2024-09-20T21:47:00.483 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-20T21:47:00.507 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-20T21:47:00.627 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-20_21:08:02-orch-squid-distro-default-smithi/7912980 branch: squid description: orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} email: ceph-qa@ceph.com first_in_suite: false job_id: '7912980' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-09-20_21:08:02-orch-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client.0: debug ms: 1 debug objecter: 20 debug rados: 20 global: ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug ms: 1 debug osd: 20 osd debug reject backfill probability: 0.3 osd delete sleep: 1 osd max backfills: 3 osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true osd snap trim sleep: 2 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - CEPHADM_STRAY_DAEMON - CEPHADM_FAILED_DAEMON - but it is still running - objects unfound and apparently lost - \(POOL_APP_NOT_ENABLED\) log-only-match: - CEPHADM_ sha1: 9e2ac70bb7aea67d34f0d50002344abbe7803736 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: 9e2ac70bb7aea67d34f0d50002344abbe7803736 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: squid sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 9344 sha1: 9e2ac70bb7aea67d34f0d50002344abbe7803736 sleep_before_teardown: 0 subset: 56/64 suite: orch suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: 4e45e6b490e6cbe082cdf155fea780438ec53fc7 targets: smithi060.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNu4q4I5cMYbiUgrPOQCFOKy/7EVJF/WJj4ZWvqG0LyAD4EGvLVpHTC9a6PS/lo0kUHeEICT9oJKRadq7zNa/7U= smithi067.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHPXzI82GQoAWhzaXUs0YpQCxnq6I6EwnGH3K2IdoimnUsxd4lFvzZ0vY+K/QpG59mBLBTuE1Dj3HDReUBtYagI= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - thrashosds: chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - full_sequential: - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 - radosbench: clients: - client.0 time: 90 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_21:08:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-20T21:47:00.627 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa; will attempt to use it 2024-09-20T21:47:00.628 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_4e45e6b490e6cbe082cdf155fea780438ec53fc7/qa/tasks 2024-09-20T21:47:00.628 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-20T21:47:00.630 INFO:teuthology.task.internal:Checking packages... 2024-09-20T21:47:00.652 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '9e2ac70bb7aea67d34f0d50002344abbe7803736' 2024-09-20T21:47:00.652 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-20T21:47:00.652 INFO:teuthology.packaging:ref: None 2024-09-20T21:47:00.652 INFO:teuthology.packaging:tag: None 2024-09-20T21:47:00.652 INFO:teuthology.packaging:branch: squid 2024-09-20T21:47:00.652 INFO:teuthology.packaging:sha1: 9e2ac70bb7aea67d34f0d50002344abbe7803736 2024-09-20T21:47:00.652 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-20T21:47:00.888 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-369.ged8ee0f2 2024-09-20T21:47:00.889 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-20T21:47:00.896 INFO:teuthology.task.internal:no buildpackages task found 2024-09-20T21:47:00.896 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-20T21:47:00.904 INFO:teuthology.task.internal:Saving configuration 2024-09-20T21:47:00.919 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-20T21:47:00.925 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-20T21:47:00.951 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi060.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-20_21:08:02-orch-squid-distro-default-smithi/7912980', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-20 21:42:03.232714', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNu4q4I5cMYbiUgrPOQCFOKy/7EVJF/WJj4ZWvqG0LyAD4EGvLVpHTC9a6PS/lo0kUHeEICT9oJKRadq7zNa/7U='} 2024-09-20T21:47:00.976 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi067.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-20_21:08:02-orch-squid-distro-default-smithi/7912980', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-20 21:42:03.234054', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHPXzI82GQoAWhzaXUs0YpQCxnq6I6EwnGH3K2IdoimnUsxd4lFvzZ0vY+K/QpG59mBLBTuE1Dj3HDReUBtYagI='} 2024-09-20T21:47:00.977 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-20T21:47:00.982 INFO:teuthology.task.internal:roles: ubuntu@smithi060.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-20T21:47:00.982 INFO:teuthology.task.internal:roles: ubuntu@smithi067.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-20T21:47:00.982 INFO:teuthology.run_tasks:Running task console_log... 2024-09-20T21:47:01.039 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f1683148160>, signals=[15]) 2024-09-20T21:47:01.039 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-20T21:47:01.045 INFO:teuthology.task.internal:Opening connections... 2024-09-20T21:47:01.045 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi060.front.sepia.ceph.com 2024-09-20T21:47:01.046 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T21:47:01.123 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi067.front.sepia.ceph.com 2024-09-20T21:47:01.124 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T21:47:01.200 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-20T21:47:01.206 DEBUG:teuthology.orchestra.run.smithi060:> uname -m 2024-09-20T21:47:01.224 INFO:teuthology.orchestra.run.smithi060.stdout:x86_64 2024-09-20T21:47:01.225 DEBUG:teuthology.orchestra.run.smithi060:> cat /etc/os-release 2024-09-20T21:47:01.282 INFO:teuthology.orchestra.run.smithi060.stdout:NAME="CentOS Stream" 2024-09-20T21:47:01.282 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION="9" 2024-09-20T21:47:01.282 INFO:teuthology.orchestra.run.smithi060.stdout:ID="centos" 2024-09-20T21:47:01.282 INFO:teuthology.orchestra.run.smithi060.stdout:ID_LIKE="rhel fedora" 2024-09-20T21:47:01.282 INFO:teuthology.orchestra.run.smithi060.stdout:VERSION_ID="9" 2024-09-20T21:47:01.282 INFO:teuthology.orchestra.run.smithi060.stdout:PLATFORM_ID="platform:el9" 2024-09-20T21:47:01.282 INFO:teuthology.orchestra.run.smithi060.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-20T21:47:01.282 INFO:teuthology.orchestra.run.smithi060.stdout:ANSI_COLOR="0;31" 2024-09-20T21:47:01.282 INFO:teuthology.orchestra.run.smithi060.stdout:LOGO="fedora-logo-icon" 2024-09-20T21:47:01.282 INFO:teuthology.orchestra.run.smithi060.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-20T21:47:01.282 INFO:teuthology.orchestra.run.smithi060.stdout:HOME_URL="https://centos.org/" 2024-09-20T21:47:01.283 INFO:teuthology.orchestra.run.smithi060.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-20T21:47:01.283 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-20T21:47:01.283 INFO:teuthology.orchestra.run.smithi060.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-20T21:47:01.283 INFO:teuthology.lock.ops:Updating smithi060.front.sepia.ceph.com on lock server 2024-09-20T21:47:01.310 DEBUG:teuthology.orchestra.run.smithi067:> uname -m 2024-09-20T21:47:01.328 INFO:teuthology.orchestra.run.smithi067.stdout:x86_64 2024-09-20T21:47:01.329 DEBUG:teuthology.orchestra.run.smithi067:> cat /etc/os-release 2024-09-20T21:47:01.386 INFO:teuthology.orchestra.run.smithi067.stdout:NAME="CentOS Stream" 2024-09-20T21:47:01.386 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION="9" 2024-09-20T21:47:01.386 INFO:teuthology.orchestra.run.smithi067.stdout:ID="centos" 2024-09-20T21:47:01.386 INFO:teuthology.orchestra.run.smithi067.stdout:ID_LIKE="rhel fedora" 2024-09-20T21:47:01.387 INFO:teuthology.orchestra.run.smithi067.stdout:VERSION_ID="9" 2024-09-20T21:47:01.387 INFO:teuthology.orchestra.run.smithi067.stdout:PLATFORM_ID="platform:el9" 2024-09-20T21:47:01.387 INFO:teuthology.orchestra.run.smithi067.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-20T21:47:01.387 INFO:teuthology.orchestra.run.smithi067.stdout:ANSI_COLOR="0;31" 2024-09-20T21:47:01.387 INFO:teuthology.orchestra.run.smithi067.stdout:LOGO="fedora-logo-icon" 2024-09-20T21:47:01.387 INFO:teuthology.orchestra.run.smithi067.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-20T21:47:01.387 INFO:teuthology.orchestra.run.smithi067.stdout:HOME_URL="https://centos.org/" 2024-09-20T21:47:01.387 INFO:teuthology.orchestra.run.smithi067.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-20T21:47:01.387 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-20T21:47:01.387 INFO:teuthology.orchestra.run.smithi067.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-20T21:47:01.387 INFO:teuthology.lock.ops:Updating smithi067.front.sepia.ceph.com on lock server 2024-09-20T21:47:01.418 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-20T21:47:01.427 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-20T21:47:01.432 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-20T21:47:01.432 DEBUG:teuthology.orchestra.run.smithi060:> test '!' -e /home/ubuntu/cephtest 2024-09-20T21:47:01.435 DEBUG:teuthology.orchestra.run.smithi067:> test '!' -e /home/ubuntu/cephtest 2024-09-20T21:47:01.452 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-20T21:47:01.458 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-20T21:47:01.458 DEBUG:teuthology.orchestra.run.smithi060:> test -z $(ls -A /var/lib/ceph) 2024-09-20T21:47:01.492 DEBUG:teuthology.orchestra.run.smithi067:> test -z $(ls -A /var/lib/ceph) 2024-09-20T21:47:01.526 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-20T21:47:01.584 INFO:teuthology.run_tasks:Running task kernel... 2024-09-20T21:47:01.598 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-20T21:47:01.598 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-20T21:47:01.599 DEBUG:teuthology.orchestra.run.smithi060:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-20T21:47:01.599 DEBUG:teuthology.orchestra.run.smithi067:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-20T21:47:01.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T21:47:01.617 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-09-20T21:47:01.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T21:47:01.618 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-09-20T21:47:01.674 INFO:teuthology.orchestra.run.smithi060.stdout:5.14.0-503.el9.x86_64 2024-09-20T21:47:01.675 INFO:teuthology.task.kernel:Running kernel on smithi060: 5.14.0-503.el9.x86_64 2024-09-20T21:47:01.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum install -y kernel 2024-09-20T21:47:01.676 INFO:teuthology.orchestra.run.smithi067.stdout:5.14.0-503.el9.x86_64 2024-09-20T21:47:01.676 INFO:teuthology.task.kernel:Running kernel on smithi067: 5.14.0-503.el9.x86_64 2024-09-20T21:47:01.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum install -y kernel 2024-09-20T21:47:04.671 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 6.7 MB/s | 8.3 MB 00:01 2024-09-20T21:47:05.180 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 4.7 MB/s | 8.3 MB 00:01 2024-09-20T21:47:06.720 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-20T21:47:06.906 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 36 MB/s | 20 MB 00:00 2024-09-20T21:47:11.429 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 9.6 MB/s | 6.5 MB 00:00 2024-09-20T21:47:12.732 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 3.6 MB/s | 6.5 MB 00:01 2024-09-20T21:47:13.388 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 19 kB 00:00 2024-09-20T21:47:14.176 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-20T21:47:14.543 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 37 kB/s | 19 kB 00:00 2024-09-20T21:47:15.662 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-20T21:47:19.899 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-20T21:47:21.360 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-20T21:47:21.973 INFO:teuthology.orchestra.run.smithi060.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-20T21:47:21.984 INFO:teuthology.orchestra.run.smithi060.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-20T21:47:22.065 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:47:22.070 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:47:22.070 INFO:teuthology.orchestra.run.smithi060.stdout: Package Architecture Version Repository Size 2024-09-20T21:47:22.070 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:47:22.071 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-20T21:47:22.071 INFO:teuthology.orchestra.run.smithi060.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-20T21:47:22.071 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-09-20T21:47:22.071 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-20T21:47:22.071 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-20T21:47:22.071 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-20T21:47:22.071 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:47:22.071 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:47:22.071 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:47:22.071 INFO:teuthology.orchestra.run.smithi060.stdout:Install 4 Packages 2024-09-20T21:47:22.071 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:47:22.072 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 79 M 2024-09-20T21:47:22.072 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 126 M 2024-09-20T21:47:22.072 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-20T21:47:22.200 INFO:teuthology.orchestra.run.smithi060.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 336 kB/s | 36 kB 00:00 2024-09-20T21:47:22.709 INFO:teuthology.orchestra.run.smithi060.stdout:(2/4): kernel-modules-core-5.14.0-508.el9.x86_6 56 MB/s | 29 MB 00:00 2024-09-20T21:47:22.935 INFO:teuthology.orchestra.run.smithi060.stdout:(3/4): kernel-core-5.14.0-508.el9.x86_64.rpm 19 MB/s | 16 MB 00:00 2024-09-20T21:47:23.325 INFO:teuthology.orchestra.run.smithi067.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-20T21:47:23.325 INFO:teuthology.orchestra.run.smithi067.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-20T21:47:23.414 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:47:23.420 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:47:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: Package Architecture Version Repository Size 2024-09-20T21:47:23.420 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:47:23.420 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-20T21:47:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-20T21:47:23.420 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-09-20T21:47:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-20T21:47:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-20T21:47:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-20T21:47:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:47:23.421 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:47:23.421 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:47:23.421 INFO:teuthology.orchestra.run.smithi067.stdout:Install 4 Packages 2024-09-20T21:47:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:47:23.421 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 79 M 2024-09-20T21:47:23.422 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 126 M 2024-09-20T21:47:23.422 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-20T21:47:23.761 INFO:teuthology.orchestra.run.smithi067.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 123 kB/s | 36 kB 00:00 2024-09-20T21:47:23.885 INFO:teuthology.orchestra.run.smithi060.stdout:(4/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 19 MB/s | 35 MB 00:01 2024-09-20T21:47:23.885 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:47:23.885 INFO:teuthology.orchestra.run.smithi060.stdout:Total 43 MB/s | 79 MB 00:01 2024-09-20T21:47:24.597 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:47:24.987 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:47:24.987 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:47:25.998 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:47:25.998 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:47:26.236 INFO:teuthology.orchestra.run.smithi067.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 5.7 MB/s | 16 MB 00:02 2024-09-20T21:47:28.559 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:47:28.836 INFO:teuthology.orchestra.run.smithi067.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 6.4 MB/s | 35 MB 00:05 2024-09-20T21:47:29.232 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-20T21:47:29.321 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-20T21:47:30.278 INFO:teuthology.orchestra.run.smithi067.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 4.4 MB/s | 29 MB 00:06 2024-09-20T21:47:30.278 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:47:30.278 INFO:teuthology.orchestra.run.smithi067.stdout:Total 11 MB/s | 79 MB 00:06 2024-09-20T21:47:30.984 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:47:31.337 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:47:31.337 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:47:32.069 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-20T21:47:32.198 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-20T21:47:32.334 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:47:32.335 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:47:34.891 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:47:35.555 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-20T21:47:35.636 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-20T21:47:37.324 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-20T21:47:37.365 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-20T21:47:38.369 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-20T21:47:38.498 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-20T21:47:42.498 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-20T21:47:43.626 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-20T21:47:43.666 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-20T21:47:48.815 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-20T21:48:21.606 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-20T21:48:21.647 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-20T21:48:22.012 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-20T21:48:22.012 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-20T21:48:22.012 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-20T21:48:22.012 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-20T21:48:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-20T21:48:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:48:22.891 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-20T21:48:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-20T21:48:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-20T21:48:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-20T21:48:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-20T21:48:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:48:22.891 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:48:23.262 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q kernel | sort -rV | head -n 1 2024-09-20T21:48:23.349 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-20T21:48:23.349 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-20T21:48:23.349 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-20T21:48:23.349 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-20T21:48:23.349 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-20T21:48:23.349 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-20T21:48:23.350 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi060.front.sepia.ceph.com, path=None, version=distro) 2024-09-20T21:48:23.350 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q kernel | sort -rV | head -n 1 2024-09-20T21:48:23.408 INFO:teuthology.orchestra.run.smithi060.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-20T21:48:23.408 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-20T21:48:23.409 DEBUG:teuthology.orchestra.run.smithi060:> sudo rpm -qi grub2-tools 2024-09-20T21:48:23.440 INFO:teuthology.orchestra.run.smithi060.stdout:Name : grub2-tools 2024-09-20T21:48:23.440 INFO:teuthology.orchestra.run.smithi060.stdout:Epoch : 1 2024-09-20T21:48:23.440 INFO:teuthology.orchestra.run.smithi060.stdout:Version : 2.06 2024-09-20T21:48:23.440 INFO:teuthology.orchestra.run.smithi060.stdout:Release : 82.el9 2024-09-20T21:48:23.440 INFO:teuthology.orchestra.run.smithi060.stdout:Architecture: x86_64 2024-09-20T21:48:23.440 INFO:teuthology.orchestra.run.smithi060.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-20T21:48:23.441 INFO:teuthology.orchestra.run.smithi060.stdout:Group : Unspecified 2024-09-20T21:48:23.441 INFO:teuthology.orchestra.run.smithi060.stdout:Size : 8274670 2024-09-20T21:48:23.441 INFO:teuthology.orchestra.run.smithi060.stdout:License : GPLv3+ 2024-09-20T21:48:23.441 INFO:teuthology.orchestra.run.smithi060.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-20T21:48:23.441 INFO:teuthology.orchestra.run.smithi060.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-20T21:48:23.441 INFO:teuthology.orchestra.run.smithi060.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-20T21:48:23.441 INFO:teuthology.orchestra.run.smithi060.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-20T21:48:23.441 INFO:teuthology.orchestra.run.smithi060.stdout:Packager : builder@centos.org 2024-09-20T21:48:23.441 INFO:teuthology.orchestra.run.smithi060.stdout:Vendor : CentOS 2024-09-20T21:48:23.441 INFO:teuthology.orchestra.run.smithi060.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-20T21:48:23.441 INFO:teuthology.orchestra.run.smithi060.stdout:Summary : Support tools for GRUB. 2024-09-20T21:48:23.441 INFO:teuthology.orchestra.run.smithi060.stdout:Description : 2024-09-20T21:48:23.441 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:48:23.441 INFO:teuthology.orchestra.run.smithi060.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-20T21:48:23.442 INFO:teuthology.orchestra.run.smithi060.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-20T21:48:23.442 INFO:teuthology.orchestra.run.smithi060.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-20T21:48:23.442 INFO:teuthology.orchestra.run.smithi060.stdout:hardware devices. 2024-09-20T21:48:23.442 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:48:23.442 INFO:teuthology.orchestra.run.smithi060.stdout:This subpackage provides tools for support of all platforms. 2024-09-20T21:48:23.443 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-20T21:48:23.443 INFO:teuthology.task.kernel:Updating grub on smithi060 to boot 5.14.0-508.el9.x86_64 2024-09-20T21:48:23.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-20T21:48:24.066 INFO:teuthology.orchestra.run.smithi060.stderr:Generating grub configuration file ... 2024-09-20T21:48:25.329 INFO:teuthology.orchestra.run.smithi060.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-20T21:48:25.365 INFO:teuthology.orchestra.run.smithi060.stderr:done 2024-09-20T21:48:25.367 DEBUG:teuthology.orchestra.run.smithi060:> mktemp 2024-09-20T21:48:25.382 INFO:teuthology.orchestra.run.smithi060.stdout:/tmp/tmp.UBpbVwXXsx 2024-09-20T21:48:25.382 DEBUG:teuthology.orchestra.run.smithi060:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.UBpbVwXXsx 2024-09-20T21:48:25.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0666 /tmp/tmp.UBpbVwXXsx 2024-09-20T21:48:25.595 DEBUG:teuthology.orchestra.remote:smithi060:/tmp/tmp.UBpbVwXXsx is 6KB 2024-09-20T21:48:25.643 DEBUG:teuthology.orchestra.run.smithi060:> rm -fr /tmp/tmp.UBpbVwXXsx 2024-09-20T21:48:25.658 DEBUG:teuthology.orchestra.run.smithi060:> sudo /bin/ls /boot/loader/entries || true 2024-09-20T21:48:25.720 INFO:teuthology.orchestra.run.smithi060.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-20T21:48:25.720 INFO:teuthology.orchestra.run.smithi060.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-20T21:48:25.720 INFO:teuthology.orchestra.run.smithi060.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-20T21:48:25.720 INFO:teuthology.orchestra.run.smithi060.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-20T21:48:25.721 DEBUG:teuthology.orchestra.run.smithi060:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-20T21:48:25.877 DEBUG:teuthology.orchestra.run.smithi060:> sudo shutdown -r now 2024-09-20T21:48:28.109 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-20T21:48:28.150 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-20T21:48:28.520 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-20T21:48:28.520 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-20T21:48:28.520 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-20T21:48:28.520 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-20T21:48:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-20T21:48:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:48:29.376 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-20T21:48:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-20T21:48:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-20T21:48:29.376 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-20T21:48:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-20T21:48:29.377 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:48:29.377 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:48:29.721 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q kernel | sort -rV | head -n 1 2024-09-20T21:48:29.808 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-20T21:48:29.808 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-20T21:48:29.808 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-20T21:48:29.809 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-20T21:48:29.809 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-20T21:48:29.809 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-20T21:48:29.809 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi067.front.sepia.ceph.com, path=None, version=distro) 2024-09-20T21:48:29.809 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q kernel | sort -rV | head -n 1 2024-09-20T21:48:29.867 INFO:teuthology.orchestra.run.smithi067.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-20T21:48:29.868 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-20T21:48:29.868 DEBUG:teuthology.orchestra.run.smithi067:> sudo rpm -qi grub2-tools 2024-09-20T21:48:29.899 INFO:teuthology.orchestra.run.smithi067.stdout:Name : grub2-tools 2024-09-20T21:48:29.899 INFO:teuthology.orchestra.run.smithi067.stdout:Epoch : 1 2024-09-20T21:48:29.899 INFO:teuthology.orchestra.run.smithi067.stdout:Version : 2.06 2024-09-20T21:48:29.899 INFO:teuthology.orchestra.run.smithi067.stdout:Release : 82.el9 2024-09-20T21:48:29.900 INFO:teuthology.orchestra.run.smithi067.stdout:Architecture: x86_64 2024-09-20T21:48:29.900 INFO:teuthology.orchestra.run.smithi067.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-20T21:48:29.900 INFO:teuthology.orchestra.run.smithi067.stdout:Group : Unspecified 2024-09-20T21:48:29.900 INFO:teuthology.orchestra.run.smithi067.stdout:Size : 8274670 2024-09-20T21:48:29.900 INFO:teuthology.orchestra.run.smithi067.stdout:License : GPLv3+ 2024-09-20T21:48:29.900 INFO:teuthology.orchestra.run.smithi067.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-20T21:48:29.900 INFO:teuthology.orchestra.run.smithi067.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-20T21:48:29.900 INFO:teuthology.orchestra.run.smithi067.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-20T21:48:29.900 INFO:teuthology.orchestra.run.smithi067.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-20T21:48:29.900 INFO:teuthology.orchestra.run.smithi067.stdout:Packager : builder@centos.org 2024-09-20T21:48:29.900 INFO:teuthology.orchestra.run.smithi067.stdout:Vendor : CentOS 2024-09-20T21:48:29.900 INFO:teuthology.orchestra.run.smithi067.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-20T21:48:29.901 INFO:teuthology.orchestra.run.smithi067.stdout:Summary : Support tools for GRUB. 2024-09-20T21:48:29.901 INFO:teuthology.orchestra.run.smithi067.stdout:Description : 2024-09-20T21:48:29.901 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:48:29.901 INFO:teuthology.orchestra.run.smithi067.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-20T21:48:29.901 INFO:teuthology.orchestra.run.smithi067.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-20T21:48:29.901 INFO:teuthology.orchestra.run.smithi067.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-20T21:48:29.901 INFO:teuthology.orchestra.run.smithi067.stdout:hardware devices. 2024-09-20T21:48:29.901 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:48:29.901 INFO:teuthology.orchestra.run.smithi067.stdout:This subpackage provides tools for support of all platforms. 2024-09-20T21:48:29.902 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-20T21:48:29.902 INFO:teuthology.task.kernel:Updating grub on smithi067 to boot 5.14.0-508.el9.x86_64 2024-09-20T21:48:29.902 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-20T21:48:30.692 INFO:teuthology.orchestra.run.smithi067.stderr:Generating grub configuration file ... 2024-09-20T21:48:32.002 INFO:teuthology.orchestra.run.smithi067.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-20T21:48:32.027 INFO:teuthology.orchestra.run.smithi067.stderr:done 2024-09-20T21:48:32.029 DEBUG:teuthology.orchestra.run.smithi067:> mktemp 2024-09-20T21:48:32.084 INFO:teuthology.orchestra.run.smithi067.stdout:/tmp/tmp.lIddXUXv2X 2024-09-20T21:48:32.084 DEBUG:teuthology.orchestra.run.smithi067:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.lIddXUXv2X 2024-09-20T21:48:32.150 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0666 /tmp/tmp.lIddXUXv2X 2024-09-20T21:48:32.315 DEBUG:teuthology.orchestra.remote:smithi067:/tmp/tmp.lIddXUXv2X is 6KB 2024-09-20T21:48:32.363 DEBUG:teuthology.orchestra.run.smithi067:> rm -fr /tmp/tmp.lIddXUXv2X 2024-09-20T21:48:32.378 DEBUG:teuthology.orchestra.run.smithi067:> sudo /bin/ls /boot/loader/entries || true 2024-09-20T21:48:32.441 INFO:teuthology.orchestra.run.smithi067.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-20T21:48:32.441 INFO:teuthology.orchestra.run.smithi067.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-20T21:48:32.441 INFO:teuthology.orchestra.run.smithi067.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-20T21:48:32.441 INFO:teuthology.orchestra.run.smithi067.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-20T21:48:32.442 DEBUG:teuthology.orchestra.run.smithi067:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-20T21:48:32.593 DEBUG:teuthology.orchestra.run.smithi067:> sudo shutdown -r now 2024-09-20T21:48:55.902 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-20T21:48:55.903 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-20T21:48:55.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T21:49:02.598 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-20T21:49:02.598 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-20T21:49:02.598 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T21:49:14.355 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.60 2024-09-20T21:49:21.007 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-09-20T21:49:23.358 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-20T21:49:23.359 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T21:49:30.013 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-20T21:49:30.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T21:49:33.071 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-09-20T21:49:45.082 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-20T21:49:45.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T21:50:03.503 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.67 2024-09-20T21:50:18.518 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-20T21:50:18.519 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T21:50:18.848 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-09-20T21:50:19.234 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-20T21:50:19.234 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-508.el9.x86_64"... 2024-09-20T21:50:19.235 DEBUG:teuthology.orchestra.run.smithi067:> uname -r 2024-09-20T21:50:19.250 INFO:teuthology.orchestra.run.smithi067.stdout:5.14.0-508.el9.x86_64 2024-09-20T21:50:19.250 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-20T21:50:19.250 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-20T21:50:19.250 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-20T21:50:20.253 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-20T21:50:20.254 DEBUG:teuthology.orchestra.run.smithi067:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-20T21:50:20.370 INFO:teuthology.orchestra.run.smithi067.stdout:ttyS1 2024-09-20T21:50:20.389 DEBUG:teuthology.parallel:result is None 2024-09-20T21:50:23.361 DEBUG:teuthology.orchestra.remote:timed out 2024-09-20T21:50:35.362 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-20T21:50:35.363 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T21:50:35.641 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-09-20T21:50:36.004 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-20T21:50:36.004 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-508.el9.x86_64"... 2024-09-20T21:50:36.004 DEBUG:teuthology.orchestra.run.smithi060:> uname -r 2024-09-20T21:50:36.020 INFO:teuthology.orchestra.run.smithi060.stdout:5.14.0-508.el9.x86_64 2024-09-20T21:50:36.020 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-20T21:50:36.020 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-20T21:50:36.020 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-20T21:50:37.022 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-20T21:50:37.022 DEBUG:teuthology.orchestra.run.smithi060:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-20T21:50:37.095 INFO:teuthology.orchestra.run.smithi060.stdout:ttyS1 2024-09-20T21:50:37.117 DEBUG:teuthology.parallel:result is None 2024-09-20T21:50:37.117 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-20T21:50:37.133 INFO:teuthology.task.internal:Creating test directory... 2024-09-20T21:50:37.133 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-20T21:50:37.136 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-20T21:50:37.157 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-20T21:50:37.166 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-20T21:50:37.191 INFO:teuthology.task.internal:Creating archive directory... 2024-09-20T21:50:37.192 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-20T21:50:37.195 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-20T21:50:37.227 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-20T21:50:37.233 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-20T21:50:37.233 DEBUG:teuthology.orchestra.run.smithi060:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-20T21:50:37.265 DEBUG:teuthology.orchestra.run.smithi067:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-20T21:50:37.297 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-20T21:50:37.306 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-20T21:50:37.308 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-20T21:50:37.318 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-20T21:50:37.320 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-20T21:50:37.327 INFO:teuthology.task.internal:Configuring sudo... 2024-09-20T21:50:37.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-20T21:50:37.362 DEBUG:teuthology.orchestra.run.smithi067:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-20T21:50:37.390 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-20T21:50:37.399 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-20T21:50:37.399 DEBUG:teuthology.orchestra.run.smithi060:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-20T21:50:37.430 DEBUG:teuthology.orchestra.run.smithi067:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-20T21:50:37.448 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-20T21:50:37.526 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-20T21:50:37.579 DEBUG:teuthology.orchestra.run.smithi060:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-20T21:50:37.663 DEBUG:teuthology.orchestra.run.smithi060:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-20T21:50:37.704 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-20T21:50:37.704 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-20T21:50:37.769 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-20T21:50:37.813 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-20T21:50:37.870 DEBUG:teuthology.orchestra.run.smithi067:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-20T21:50:37.956 DEBUG:teuthology.orchestra.run.smithi067:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-20T21:50:37.998 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-20T21:50:37.998 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-20T21:50:38.066 DEBUG:teuthology.orchestra.run.smithi060:> sudo service rsyslog restart 2024-09-20T21:50:38.069 DEBUG:teuthology.orchestra.run.smithi067:> sudo service rsyslog restart 2024-09-20T21:50:38.114 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-20T21:50:38.169 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-20T21:50:38.673 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-20T21:50:38.680 INFO:teuthology.task.internal:Starting timer... 2024-09-20T21:50:38.681 INFO:teuthology.run_tasks:Running task pcp... 2024-09-20T21:50:38.691 INFO:teuthology.run_tasks:Running task selinux... 2024-09-20T21:50:38.701 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-20T21:50:38.701 DEBUG:teuthology.orchestra.run.smithi060:> sudo service auditd rotate 2024-09-20T21:50:38.797 INFO:teuthology.orchestra.run.smithi060.stdout:Rotating logs: 2024-09-20T21:50:38.799 DEBUG:teuthology.orchestra.run.smithi067:> sudo service auditd rotate 2024-09-20T21:50:38.877 INFO:teuthology.orchestra.run.smithi067.stdout:Rotating logs: 2024-09-20T21:50:38.879 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-20T21:50:38.879 DEBUG:teuthology.orchestra.run.smithi060:> /usr/sbin/getenforce 2024-09-20T21:50:38.906 INFO:teuthology.orchestra.run.smithi060.stdout:Permissive 2024-09-20T21:50:38.906 DEBUG:teuthology.orchestra.run.smithi067:> /usr/sbin/getenforce 2024-09-20T21:50:38.934 INFO:teuthology.orchestra.run.smithi067.stdout:Permissive 2024-09-20T21:50:38.934 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi060.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi067.front.sepia.ceph.com': 'permissive'} 2024-09-20T21:50:38.934 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-20T21:50:38.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T21:50:38.976 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-20T21:50:39.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T21:50:39.005 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-20T21:50:39.005 DEBUG:teuthology.orchestra.run.smithi060:> sudo /usr/sbin/setenforce permissive 2024-09-20T21:50:39.053 DEBUG:teuthology.orchestra.run.smithi067:> sudo /usr/sbin/setenforce permissive 2024-09-20T21:50:39.087 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-20T21:50:39.096 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-20T21:50:39.098 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-20T21:50:39.218 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-20T21:50:39.228 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-20T21:50:39.228 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi060.front.sepia.ceph.com,smithi067.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-20T21:54:30.802 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi060.front.sepia.ceph.com'), Remote(name='ubuntu@smithi067.front.sepia.ceph.com')] 2024-09-20T21:54:30.803 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-20T21:54:30.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi060.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T21:54:30.878 DEBUG:teuthology.orchestra.run.smithi060:> true 2024-09-20T21:54:30.958 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi060.front.sepia.ceph.com' 2024-09-20T21:54:30.959 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-20T21:54:30.959 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi067.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-20T21:54:31.033 DEBUG:teuthology.orchestra.run.smithi067:> true 2024-09-20T21:54:31.115 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi067.front.sepia.ceph.com' 2024-09-20T21:54:31.115 INFO:teuthology.run_tasks:Running task clock... 2024-09-20T21:54:31.124 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-20T21:54:31.125 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-20T21:54:31.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-20T21:54:31.127 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-20T21:54:31.127 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-20T21:54:31.156 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-20T21:54:31.172 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-20T21:54:31.186 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-20T21:54:31.199 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: ntpd: command not found 2024-09-20T21:54:31.201 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-20T21:54:31.210 INFO:teuthology.orchestra.run.smithi060.stdout:506 Cannot talk to daemon 2024-09-20T21:54:31.225 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-20T21:54:31.227 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: ntpd: command not found 2024-09-20T21:54:31.238 INFO:teuthology.orchestra.run.smithi067.stdout:506 Cannot talk to daemon 2024-09-20T21:54:31.240 INFO:teuthology.orchestra.run.smithi060.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-20T21:54:31.254 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-20T21:54:31.269 INFO:teuthology.orchestra.run.smithi067.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-20T21:54:31.295 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2024-09-20T21:54:31.298 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-20T21:54:31.298 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2024-09-20T21:54:31.298 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T21:54:31.298 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T21:54:31.299 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T21:54:31.299 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T21:54:31.333 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2024-09-20T21:54:31.336 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-20T21:54:31.336 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-09-20T21:54:31.337 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T21:54:31.337 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T21:54:31.337 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T21:54:31.337 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T21:54:31.338 INFO:teuthology.run_tasks:Running task pexec... 2024-09-20T21:54:31.346 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-20T21:54:31.347 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi060.front.sepia.ceph.com 2024-09-20T21:54:31.347 DEBUG:teuthology.orchestra.run.smithi060:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-20T21:54:31.347 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi067.front.sepia.ceph.com 2024-09-20T21:54:31.347 DEBUG:teuthology.orchestra.run.smithi067:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-20T21:54:31.717 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:54:31.731 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:54:31.732 INFO:teuthology.orchestra.run.smithi060.stdout: Package Architecture Version Repository Size 2024-09-20T21:54:31.732 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:54:31.732 INFO:teuthology.orchestra.run.smithi060.stdout:Removing: 2024-09-20T21:54:31.732 INFO:teuthology.orchestra.run.smithi060.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-20T21:54:31.732 INFO:teuthology.orchestra.run.smithi060.stdout:Removing unused dependencies: 2024-09-20T21:54:31.732 INFO:teuthology.orchestra.run.smithi060.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-20T21:54:31.732 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:54:31.732 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:54:31.732 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:54:31.732 INFO:teuthology.orchestra.run.smithi060.stdout:Remove 2 Packages 2024-09-20T21:54:31.732 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:54:31.732 INFO:teuthology.orchestra.run.smithi060.stdout:Freed space: 5.5 M 2024-09-20T21:54:31.732 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:54:31.736 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:54:31.737 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:54:31.744 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:54:31.746 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:54:31.746 INFO:teuthology.orchestra.run.smithi067.stdout: Package Architecture Version Repository Size 2024-09-20T21:54:31.746 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:54:31.746 INFO:teuthology.orchestra.run.smithi067.stdout:Removing: 2024-09-20T21:54:31.746 INFO:teuthology.orchestra.run.smithi067.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-20T21:54:31.746 INFO:teuthology.orchestra.run.smithi067.stdout:Removing unused dependencies: 2024-09-20T21:54:31.746 INFO:teuthology.orchestra.run.smithi067.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-20T21:54:31.746 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:54:31.746 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:54:31.746 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:54:31.747 INFO:teuthology.orchestra.run.smithi067.stdout:Remove 2 Packages 2024-09-20T21:54:31.747 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:54:31.747 INFO:teuthology.orchestra.run.smithi067.stdout:Freed space: 5.5 M 2024-09-20T21:54:31.747 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:54:31.751 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:54:31.751 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:54:31.800 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:54:31.801 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:54:31.815 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:54:31.816 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:54:31.950 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:54:31.962 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:54:32.020 INFO:teuthology.orchestra.run.smithi060.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-20T21:54:32.052 INFO:teuthology.orchestra.run.smithi067.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-20T21:54:32.080 INFO:teuthology.orchestra.run.smithi060.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-20T21:54:32.109 INFO:teuthology.orchestra.run.smithi067.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-20T21:54:32.593 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-20T21:54:32.593 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-20T21:54:32.718 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-20T21:54:32.718 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-20T21:54:32.865 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-20T21:54:32.865 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:54:32.865 INFO:teuthology.orchestra.run.smithi060.stdout:Removed: 2024-09-20T21:54:32.865 INFO:teuthology.orchestra.run.smithi060.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-20T21:54:32.865 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:54:32.865 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:54:32.941 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-20T21:54:32.941 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:54:32.941 INFO:teuthology.orchestra.run.smithi067.stdout:Removed: 2024-09-20T21:54:32.942 INFO:teuthology.orchestra.run.smithi067.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-20T21:54:32.942 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:54:32.942 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:54:33.764 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:00:49 ago on Fri 20 Sep 2024 09:53:44 PM UTC. 2024-09-20T21:54:33.787 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:00:49 ago on Fri 20 Sep 2024 09:53:44 PM UTC. 2024-09-20T21:54:33.902 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:54:33.903 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:54:33.903 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-09-20T21:54:33.903 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:54:33.903 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-20T21:54:33.903 INFO:teuthology.orchestra.run.smithi067.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-20T21:54:33.903 INFO:teuthology.orchestra.run.smithi067.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-20T21:54:33.903 INFO:teuthology.orchestra.run.smithi067.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-20T21:54:33.904 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-09-20T21:54:33.904 INFO:teuthology.orchestra.run.smithi067.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-20T21:54:33.904 INFO:teuthology.orchestra.run.smithi067.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-20T21:54:33.904 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-20T21:54:33.904 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-20T21:54:33.904 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:54:33.904 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:54:33.904 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:54:33.904 INFO:teuthology.orchestra.run.smithi067.stdout:Install 7 Packages 2024-09-20T21:54:33.904 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:54:33.916 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 5.2 M 2024-09-20T21:54:33.916 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 19 M 2024-09-20T21:54:33.916 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-20T21:54:33.926 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:54:33.927 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:54:33.927 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-09-20T21:54:33.927 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:54:33.927 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-20T21:54:33.927 INFO:teuthology.orchestra.run.smithi060.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-20T21:54:33.928 INFO:teuthology.orchestra.run.smithi060.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-20T21:54:33.928 INFO:teuthology.orchestra.run.smithi060.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-20T21:54:33.928 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-09-20T21:54:33.928 INFO:teuthology.orchestra.run.smithi060.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-20T21:54:33.928 INFO:teuthology.orchestra.run.smithi060.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-20T21:54:33.928 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-20T21:54:33.928 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-20T21:54:33.928 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:54:33.928 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:54:33.928 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:54:33.928 INFO:teuthology.orchestra.run.smithi060.stdout:Install 7 Packages 2024-09-20T21:54:33.928 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:54:33.937 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 5.2 M 2024-09-20T21:54:33.937 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 19 M 2024-09-20T21:54:33.937 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-20T21:54:34.375 INFO:teuthology.orchestra.run.smithi060.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 339 kB/s | 44 kB 00:00 2024-09-20T21:54:34.409 INFO:teuthology.orchestra.run.smithi060.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 643 kB/s | 106 kB 00:00 2024-09-20T21:54:34.459 INFO:teuthology.orchestra.run.smithi060.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 859 kB/s | 72 kB 00:00 2024-09-20T21:54:34.484 INFO:teuthology.orchestra.run.smithi060.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.1 MB/s | 84 kB 00:00 2024-09-20T21:54:34.526 INFO:teuthology.orchestra.run.smithi060.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 3.1 MB/s | 906 kB 00:00 2024-09-20T21:54:34.627 INFO:teuthology.orchestra.run.smithi060.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 4.9 MB/s | 837 kB 00:00 2024-09-20T21:54:34.773 INFO:teuthology.orchestra.run.smithi067.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 77 kB/s | 44 kB 00:00 2024-09-20T21:54:34.881 INFO:teuthology.orchestra.run.smithi067.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 154 kB/s | 106 kB 00:00 2024-09-20T21:54:35.398 INFO:teuthology.orchestra.run.smithi067.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 115 kB/s | 72 kB 00:00 2024-09-20T21:54:35.574 INFO:teuthology.orchestra.run.smithi067.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 122 kB/s | 84 kB 00:00 2024-09-20T21:54:35.586 INFO:teuthology.orchestra.run.smithi060.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 2.9 MB/s | 3.2 MB 00:01 2024-09-20T21:54:35.586 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:54:35.586 INFO:teuthology.orchestra.run.smithi060.stdout:Total 3.1 MB/s | 5.2 MB 00:01 2024-09-20T21:54:35.707 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:54:35.723 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:54:35.724 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:54:35.982 INFO:teuthology.orchestra.run.smithi067.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 506 kB/s | 906 kB 00:01 2024-09-20T21:54:36.041 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:54:36.041 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:54:36.249 INFO:teuthology.orchestra.run.smithi067.stdout:(6/7): runc-1.1.13-4.el9.x86_64.rpm 4.7 MB/s | 3.2 MB 00:00 2024-09-20T21:54:36.536 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:54:36.631 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-20T21:54:36.664 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-20T21:54:36.693 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-20T21:54:36.919 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-20T21:54:36.957 INFO:teuthology.orchestra.run.smithi067.stdout:(7/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 537 kB/s | 837 kB 00:01 2024-09-20T21:54:36.958 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:54:36.958 INFO:teuthology.orchestra.run.smithi067.stdout:Total 1.7 MB/s | 5.2 MB 00:03 2024-09-20T21:54:36.965 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-20T21:54:37.078 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:54:37.095 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:54:37.095 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:54:37.418 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:54:37.418 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:54:37.434 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-20T21:54:37.460 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-20T21:54:37.583 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-20T21:54:37.649 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-20T21:54:37.911 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:54:38.002 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-20T21:54:38.045 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-20T21:54:38.082 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-20T21:54:38.303 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-20T21:54:38.338 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-20T21:54:38.614 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-20T21:54:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-20T21:54:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-20T21:54:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-20T21:54:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-20T21:54:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-20T21:54:38.615 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-20T21:54:38.819 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-20T21:54:38.848 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-20T21:54:38.968 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-20T21:54:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-20T21:54:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:54:38.977 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-20T21:54:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-20T21:54:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-20T21:54:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-20T21:54:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-20T21:54:38.977 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:54:38.977 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:54:39.038 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-20T21:54:39.324 DEBUG:teuthology.parallel:result is None 2024-09-20T21:54:39.969 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-20T21:54:39.970 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-20T21:54:39.970 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-20T21:54:39.970 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-20T21:54:39.970 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-20T21:54:39.970 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-20T21:54:39.970 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-20T21:54:40.341 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-20T21:54:40.341 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:54:40.341 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-20T21:54:40.342 INFO:teuthology.orchestra.run.smithi067.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-20T21:54:40.342 INFO:teuthology.orchestra.run.smithi067.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-20T21:54:40.342 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-20T21:54:40.342 INFO:teuthology.orchestra.run.smithi067.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-20T21:54:40.342 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:54:40.342 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:54:40.754 DEBUG:teuthology.parallel:result is None 2024-09-20T21:54:40.755 INFO:teuthology.run_tasks:Running task install... 2024-09-20T21:54:40.763 DEBUG:teuthology.task.install:project ceph 2024-09-20T21:54:40.763 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9e2ac70bb7aea67d34f0d50002344abbe7803736'}} 2024-09-20T21:54:40.764 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '9e2ac70bb7aea67d34f0d50002344abbe7803736'} 2024-09-20T21:54:40.764 INFO:teuthology.task.install:Using flavor: default 2024-09-20T21:54:40.769 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-20T21:54:40.769 INFO:teuthology.task.install:extra packages: [] 2024-09-20T21:54:40.769 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9e2ac70bb7aea67d34f0d50002344abbe7803736', 'tag': None, 'wait_for_package': False} 2024-09-20T21:54:40.770 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9e2ac70bb7aea67d34f0d50002344abbe7803736 2024-09-20T21:54:40.771 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '9e2ac70bb7aea67d34f0d50002344abbe7803736', 'tag': None, 'wait_for_package': False} 2024-09-20T21:54:40.771 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9e2ac70bb7aea67d34f0d50002344abbe7803736 2024-09-20T21:54:40.915 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/9e2ac70bb7aea67d34f0d50002344abbe7803736/centos/9/flavors/default/ 2024-09-20T21:54:40.915 INFO:teuthology.task.install.rpm:Package version is 19.1.1-367.g9e2ac70b 2024-09-20T21:54:40.920 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/squid/9e2ac70bb7aea67d34f0d50002344abbe7803736/centos/9/flavors/default/ 2024-09-20T21:54:40.920 INFO:teuthology.task.install.rpm:Package version is 19.1.1-367.g9e2ac70b 2024-09-20T21:54:41.027 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/9e2ac70bb7aea67d34f0d50002344abbe7803736/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/9e2ac70bb7aea67d34f0d50002344abbe7803736/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/9e2ac70bb7aea67d34f0d50002344abbe7803736/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-20T21:54:41.027 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-20T21:54:41.028 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-20T21:54:41.034 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/squid/9e2ac70bb7aea67d34f0d50002344abbe7803736/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/9e2ac70bb7aea67d34f0d50002344abbe7803736/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/squid/9e2ac70bb7aea67d34f0d50002344abbe7803736/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-20T21:54:41.034 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-20T21:54:41.034 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-20T21:54:41.059 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-20T21:54:41.060 DEBUG:teuthology.orchestra.run.smithi060:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9e2ac70bb7aea67d34f0d50002344abbe7803736/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-20T21:54:41.060 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-20T21:54:41.061 DEBUG:teuthology.orchestra.run.smithi067:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/9e2ac70bb7aea67d34f0d50002344abbe7803736/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-20T21:54:41.128 DEBUG:teuthology.orchestra.run.smithi067:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-20T21:54:41.133 DEBUG:teuthology.orchestra.run.smithi060:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-20T21:54:41.204 DEBUG:teuthology.orchestra.run.smithi067:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-20T21:54:41.215 DEBUG:teuthology.orchestra.run.smithi060:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-20T21:54:41.269 INFO:teuthology.orchestra.run.smithi067.stdout:check_obsoletes = 1 2024-09-20T21:54:41.271 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum clean all 2024-09-20T21:54:41.283 INFO:teuthology.orchestra.run.smithi060.stdout:check_obsoletes = 1 2024-09-20T21:54:41.285 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum clean all 2024-09-20T21:54:41.515 INFO:teuthology.orchestra.run.smithi067.stdout:45 files removed 2024-09-20T21:54:41.529 INFO:teuthology.orchestra.run.smithi060.stdout:45 files removed 2024-09-20T21:54:41.544 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-radosgw 2024-09-20T21:54:41.556 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-radosgw 2024-09-20T21:54:42.106 INFO:teuthology.orchestra.run.smithi067.stdout:ceph packages for x86_64 316 kB/s | 84 kB 00:00 2024-09-20T21:54:42.134 INFO:teuthology.orchestra.run.smithi060.stdout:ceph packages for x86_64 299 kB/s | 84 kB 00:00 2024-09-20T21:54:42.390 INFO:teuthology.orchestra.run.smithi067.stdout:ceph noarch packages 49 kB/s | 12 kB 00:00 2024-09-20T21:54:42.426 INFO:teuthology.orchestra.run.smithi060.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-20T21:54:42.665 INFO:teuthology.orchestra.run.smithi067.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-20T21:54:42.710 INFO:teuthology.orchestra.run.smithi060.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-20T21:54:43.235 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.3 MB 00:00 2024-09-20T21:54:44.174 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - BaseOS 5.6 MB/s | 8.3 MB 00:01 2024-09-20T21:54:45.070 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-20T21:54:46.200 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-20T21:54:50.671 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.5 MB 00:01 2024-09-20T21:54:50.718 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.5 MB 00:00 2024-09-20T21:54:52.305 INFO:teuthology.orchestra.run.smithi060.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 19 kB 00:00 2024-09-20T21:54:52.562 INFO:teuthology.orchestra.run.smithi067.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 19 kB 00:00 2024-09-20T21:54:53.276 INFO:teuthology.orchestra.run.smithi060.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-20T21:54:53.540 INFO:teuthology.orchestra.run.smithi067.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-20T21:54:59.032 INFO:teuthology.orchestra.run.smithi060.stdout:lab-extras 49 kB/s | 1.7 kB 00:00 2024-09-20T21:54:59.246 INFO:teuthology.orchestra.run.smithi067.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-20T21:55:01.434 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:55:01.436 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:55:01.436 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-09-20T21:55:01.436 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:55:01.436 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-20T21:55:01.436 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 11 M 2024-09-20T21:55:01.436 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrading: 2024-09-20T21:55:01.437 INFO:teuthology.orchestra.run.smithi060.stdout: librados2 x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 3.4 M 2024-09-20T21:55:01.437 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1 x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 3.2 M 2024-09-20T21:55:01.437 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-09-20T21:55:01.437 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-20T21:55:01.437 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 5.4 M 2024-09-20T21:55:01.437 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 21 M 2024-09-20T21:55:01.437 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 25 k 2024-09-20T21:55:01.437 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-20T21:55:01.437 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-20T21:55:01.437 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-20T21:55:01.437 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2 x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 752 k 2024-09-20T21:55:01.437 INFO:teuthology.orchestra.run.smithi060.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-20T21:55:01.438 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-20T21:55:01.438 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-20T21:55:01.438 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1 x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 506 k 2024-09-20T21:55:01.438 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-20T21:55:01.438 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2 x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 5.3 M 2024-09-20T21:55:01.438 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-20T21:55:01.438 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-20T21:55:01.438 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-20T21:55:01.438 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 45 k 2024-09-20T21:55:01.438 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 136 k 2024-09-20T21:55:01.438 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 161 k 2024-09-20T21:55:01.438 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-20T21:55:01.439 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 320 k 2024-09-20T21:55:01.439 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 301 k 2024-09-20T21:55:01.439 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 99 k 2024-09-20T21:55:01.439 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-20T21:55:01.439 INFO:teuthology.orchestra.run.smithi060.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-20T21:55:01.439 INFO:teuthology.orchestra.run.smithi060.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-20T21:55:01.439 INFO:teuthology.orchestra.run.smithi060.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-20T21:55:01.439 INFO:teuthology.orchestra.run.smithi060.stdout:Installing weak dependencies: 2024-09-20T21:55:01.439 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-20T21:55:01.439 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:01.439 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:55:01.439 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:55:01.439 INFO:teuthology.orchestra.run.smithi060.stdout:Install 30 Packages 2024-09-20T21:55:01.439 INFO:teuthology.orchestra.run.smithi060.stdout:Upgrade 2 Packages 2024-09-20T21:55:01.440 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:01.440 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 60 M 2024-09-20T21:55:01.440 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-20T21:55:01.581 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:55:01.584 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:55:01.584 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-09-20T21:55:01.584 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:55:01.584 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-20T21:55:01.584 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 11 M 2024-09-20T21:55:01.584 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrading: 2024-09-20T21:55:01.584 INFO:teuthology.orchestra.run.smithi067.stdout: librados2 x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 3.4 M 2024-09-20T21:55:01.584 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1 x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 3.2 M 2024-09-20T21:55:01.584 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-09-20T21:55:01.584 INFO:teuthology.orchestra.run.smithi067.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-20T21:55:01.584 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 5.4 M 2024-09-20T21:55:01.584 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 21 M 2024-09-20T21:55:01.584 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 25 k 2024-09-20T21:55:01.584 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-20T21:55:01.585 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-20T21:55:01.585 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-20T21:55:01.585 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2 x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 752 k 2024-09-20T21:55:01.585 INFO:teuthology.orchestra.run.smithi067.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-20T21:55:01.585 INFO:teuthology.orchestra.run.smithi067.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-20T21:55:01.585 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-20T21:55:01.585 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1 x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 506 k 2024-09-20T21:55:01.585 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-20T21:55:01.585 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2 x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 5.3 M 2024-09-20T21:55:01.585 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-20T21:55:01.585 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-20T21:55:01.585 INFO:teuthology.orchestra.run.smithi067.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-20T21:55:01.585 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 45 k 2024-09-20T21:55:01.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 136 k 2024-09-20T21:55:01.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 161 k 2024-09-20T21:55:01.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-20T21:55:01.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 320 k 2024-09-20T21:55:01.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 301 k 2024-09-20T21:55:01.586 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 99 k 2024-09-20T21:55:01.586 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-20T21:55:01.586 INFO:teuthology.orchestra.run.smithi067.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-20T21:55:01.586 INFO:teuthology.orchestra.run.smithi067.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-20T21:55:01.586 INFO:teuthology.orchestra.run.smithi067.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-20T21:55:01.586 INFO:teuthology.orchestra.run.smithi067.stdout:Installing weak dependencies: 2024-09-20T21:55:01.586 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-20T21:55:01.586 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:01.586 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:55:01.586 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:55:01.586 INFO:teuthology.orchestra.run.smithi067.stdout:Install 30 Packages 2024-09-20T21:55:01.587 INFO:teuthology.orchestra.run.smithi067.stdout:Upgrade 2 Packages 2024-09-20T21:55:01.587 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:01.587 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 60 M 2024-09-20T21:55:01.587 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-20T21:55:02.158 INFO:teuthology.orchestra.run.smithi060.stdout:(1/32): ceph-base-19.1.1-367.g9e2ac70b.el9.x86_ 10 MB/s | 5.4 MB 00:00 2024-09-20T21:55:02.225 INFO:teuthology.orchestra.run.smithi060.stdout:(2/32): ceph-selinux-19.1.1-367.g9e2ac70b.el9.x 378 kB/s | 25 kB 00:00 2024-09-20T21:55:02.325 INFO:teuthology.orchestra.run.smithi060.stdout:(3/32): libcephfs2-19.1.1-367.g9e2ac70b.el9.x86 7.4 MB/s | 752 kB 00:00 2024-09-20T21:55:02.442 INFO:teuthology.orchestra.run.smithi060.stdout:(4/32): ceph-radosgw-19.1.1-367.g9e2ac70b.el9.x 13 MB/s | 11 MB 00:00 2024-09-20T21:55:02.484 INFO:teuthology.orchestra.run.smithi060.stdout:(5/32): libradosstriper1-19.1.1-367.g9e2ac70b.e 3.1 MB/s | 506 kB 00:00 2024-09-20T21:55:02.543 INFO:teuthology.orchestra.run.smithi060.stdout:(6/32): python3-ceph-argparse-19.1.1-367.g9e2ac 777 kB/s | 45 kB 00:00 2024-09-20T21:55:02.626 INFO:teuthology.orchestra.run.smithi060.stdout:(7/32): python3-ceph-common-19.1.1-367.g9e2ac70 1.6 MB/s | 136 kB 00:00 2024-09-20T21:55:02.868 INFO:teuthology.orchestra.run.smithi060.stdout:(8/32): ceph-common-19.1.1-367.g9e2ac70b.el9.x8 17 MB/s | 21 MB 00:01 2024-09-20T21:55:02.893 INFO:teuthology.orchestra.run.smithi060.stdout:(9/32): python3-cephfs-19.1.1-367.g9e2ac70b.el9 605 kB/s | 161 kB 00:00 2024-09-20T21:55:02.919 INFO:teuthology.orchestra.run.smithi060.stdout:(10/32): python3-rados-19.1.1-367.g9e2ac70b.el9 6.3 MB/s | 320 kB 00:00 2024-09-20T21:55:02.953 INFO:teuthology.orchestra.run.smithi060.stdout:(11/32): python3-rbd-19.1.1-367.g9e2ac70b.el9.x 5.0 MB/s | 301 kB 00:00 2024-09-20T21:55:02.978 INFO:teuthology.orchestra.run.smithi060.stdout:(12/32): python3-rgw-19.1.1-367.g9e2ac70b.el9.x 1.6 MB/s | 99 kB 00:00 2024-09-20T21:55:03.062 INFO:teuthology.orchestra.run.smithi060.stdout:(13/32): librgw2-19.1.1-367.g9e2ac70b.el9.x86_6 8.6 MB/s | 5.3 MB 00:00 2024-09-20T21:55:03.237 INFO:teuthology.orchestra.run.smithi060.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 258 kB/s | 45 kB 00:00 2024-09-20T21:55:03.262 INFO:teuthology.orchestra.run.smithi060.stdout:(15/32): boost-program-options-1.75.0-8.el9.x86 346 kB/s | 107 kB 00:00 2024-09-20T21:55:03.287 INFO:teuthology.orchestra.run.smithi060.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 518 kB/s | 160 kB 00:00 2024-09-20T21:55:03.354 INFO:teuthology.orchestra.run.smithi060.stdout:(17/32): python3-prettytable-0.7.2-27.el9.noarc 634 kB/s | 42 kB 00:00 2024-09-20T21:55:03.380 INFO:teuthology.orchestra.run.smithi067.stdout:(1/32): ceph-radosgw-19.1.1-367.g9e2ac70b.el9.x 6.6 MB/s | 11 MB 00:01 2024-09-20T21:55:03.404 INFO:teuthology.orchestra.run.smithi060.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-09-20T21:55:03.422 INFO:teuthology.orchestra.run.smithi067.stdout:(2/32): ceph-selinux-19.1.1-367.g9e2ac70b.el9.x 607 kB/s | 25 kB 00:00 2024-09-20T21:55:03.430 INFO:teuthology.orchestra.run.smithi060.stdout:(19/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.7 MB/s | 292 kB 00:00 2024-09-20T21:55:03.463 INFO:teuthology.orchestra.run.smithi060.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 612 kB/s | 36 kB 00:00 2024-09-20T21:55:03.489 INFO:teuthology.orchestra.run.smithi060.stdout:(21/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-09-20T21:55:03.506 INFO:teuthology.orchestra.run.smithi067.stdout:(3/32): libcephfs2-19.1.1-367.g9e2ac70b.el9.x86 8.8 MB/s | 752 kB 00:00 2024-09-20T21:55:03.514 INFO:teuthology.orchestra.run.smithi060.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-09-20T21:55:03.564 INFO:teuthology.orchestra.run.smithi067.stdout:(4/32): libradosstriper1-19.1.1-367.g9e2ac70b.e 8.5 MB/s | 506 kB 00:00 2024-09-20T21:55:03.572 INFO:teuthology.orchestra.run.smithi060.stdout:(23/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.8 MB/s | 308 kB 00:00 2024-09-20T21:55:03.598 INFO:teuthology.orchestra.run.smithi060.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-20T21:55:03.623 INFO:teuthology.orchestra.run.smithi060.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 280 kB/s | 26 kB 00:00 2024-09-20T21:55:03.648 INFO:teuthology.orchestra.run.smithi060.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-20T21:55:03.682 INFO:teuthology.orchestra.run.smithi060.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 5.7 MB/s | 191 kB 00:00 2024-09-20T21:55:03.732 INFO:teuthology.orchestra.run.smithi060.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-20T21:55:03.783 INFO:teuthology.orchestra.run.smithi060.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-20T21:55:03.866 INFO:teuthology.orchestra.run.smithi060.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-20T21:55:03.931 INFO:teuthology.orchestra.run.smithi067.stdout:(5/32): ceph-common-19.1.1-367.g9e2ac70b.el9.x8 9.8 MB/s | 21 MB 00:02 2024-09-20T21:55:03.983 INFO:teuthology.orchestra.run.smithi060.stdout:(31/32): librados2-19.1.1-367.g9e2ac70b.el9.x86 14 MB/s | 3.4 MB 00:00 2024-09-20T21:55:04.015 INFO:teuthology.orchestra.run.smithi067.stdout:(6/32): librgw2-19.1.1-367.g9e2ac70b.el9.x86_64 12 MB/s | 5.3 MB 00:00 2024-09-20T21:55:04.040 INFO:teuthology.orchestra.run.smithi067.stdout:(7/32): python3-ceph-argparse-19.1.1-367.g9e2ac 416 kB/s | 45 kB 00:00 2024-09-20T21:55:04.050 INFO:teuthology.orchestra.run.smithi060.stdout:(32/32): librbd1-19.1.1-367.g9e2ac70b.el9.x86_6 12 MB/s | 3.2 MB 00:00 2024-09-20T21:55:04.056 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:55:04.056 INFO:teuthology.orchestra.run.smithi060.stdout:Total 23 MB/s | 60 MB 00:02 2024-09-20T21:55:04.065 INFO:teuthology.orchestra.run.smithi067.stdout:(8/32): python3-ceph-common-19.1.1-367.g9e2ac70 2.7 MB/s | 136 kB 00:00 2024-09-20T21:55:04.091 INFO:teuthology.orchestra.run.smithi067.stdout:(9/32): python3-cephfs-19.1.1-367.g9e2ac70b.el9 3.1 MB/s | 161 kB 00:00 2024-09-20T21:55:04.116 INFO:teuthology.orchestra.run.smithi067.stdout:(10/32): python3-rados-19.1.1-367.g9e2ac70b.el9 6.2 MB/s | 320 kB 00:00 2024-09-20T21:55:04.141 INFO:teuthology.orchestra.run.smithi067.stdout:(11/32): python3-rbd-19.1.1-367.g9e2ac70b.el9.x 5.9 MB/s | 301 kB 00:00 2024-09-20T21:55:04.165 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:55:04.167 INFO:teuthology.orchestra.run.smithi067.stdout:(12/32): python3-rgw-19.1.1-367.g9e2ac70b.el9.x 1.9 MB/s | 99 kB 00:00 2024-09-20T21:55:04.206 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:55:04.206 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:55:04.276 INFO:teuthology.orchestra.run.smithi067.stdout:(13/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-20T21:55:04.318 INFO:teuthology.orchestra.run.smithi067.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 608 kB/s | 107 kB 00:00 2024-09-20T21:55:04.351 INFO:teuthology.orchestra.run.smithi067.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 601 kB/s | 45 kB 00:00 2024-09-20T21:55:04.435 INFO:teuthology.orchestra.run.smithi067.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-09-20T21:55:04.510 INFO:teuthology.orchestra.run.smithi067.stdout:(17/32): ceph-base-19.1.1-367.g9e2ac70b.el9.x86 2.0 MB/s | 5.4 MB 00:02 2024-09-20T21:55:04.535 INFO:teuthology.orchestra.run.smithi067.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 422 kB/s | 42 kB 00:00 2024-09-20T21:55:04.569 INFO:teuthology.orchestra.run.smithi067.stdout:(19/32): librdkafka-1.6.1-102.el9.x86_64.rpm 2.6 MB/s | 662 kB 00:00 2024-09-20T21:55:04.594 INFO:teuthology.orchestra.run.smithi067.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 612 kB/s | 36 kB 00:00 2024-09-20T21:55:04.653 INFO:teuthology.orchestra.run.smithi067.stdout:(21/32): gperftools-libs-2.9.1-3.el9.x86_64.rpm 5.1 MB/s | 308 kB 00:00 2024-09-20T21:55:04.678 INFO:teuthology.orchestra.run.smithi067.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 435 kB/s | 47 kB 00:00 2024-09-20T21:55:04.720 INFO:teuthology.orchestra.run.smithi067.stdout:(23/32): qatlib-24.02.0-1.el9.x86_64.rpm 1.0 MB/s | 221 kB 00:00 2024-09-20T21:55:04.755 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:55:04.755 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:55:04.762 INFO:teuthology.orchestra.run.smithi067.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 311 kB/s | 26 kB 00:00 2024-09-20T21:55:04.787 INFO:teuthology.orchestra.run.smithi067.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 824 kB/s | 49 kB 00:00 2024-09-20T21:55:04.813 INFO:teuthology.orchestra.run.smithi067.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.4 MB/s | 67 kB 00:00 2024-09-20T21:55:04.846 INFO:teuthology.orchestra.run.smithi067.stdout:(27/32): re2-20211101-20.el9.x86_64.rpm 5.7 MB/s | 191 kB 00:00 2024-09-20T21:55:04.896 INFO:teuthology.orchestra.run.smithi067.stdout:(28/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.5 MB/s | 839 kB 00:00 2024-09-20T21:55:04.955 INFO:teuthology.orchestra.run.smithi067.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-20T21:55:05.047 INFO:teuthology.orchestra.run.smithi067.stdout:(30/32): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-20T21:55:05.181 INFO:teuthology.orchestra.run.smithi067.stdout:(31/32): librbd1-19.1.1-367.g9e2ac70b.el9.x86_6 14 MB/s | 3.2 MB 00:00 2024-09-20T21:55:05.764 INFO:teuthology.orchestra.run.smithi067.stdout:(32/32): librados2-19.1.1-367.g9e2ac70b.el9.x86 3.9 MB/s | 3.4 MB 00:00 2024-09-20T21:55:05.770 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:55:05.770 INFO:teuthology.orchestra.run.smithi067.stdout:Total 14 MB/s | 60 MB 00:04 2024-09-20T21:55:05.883 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:55:05.924 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:55:05.924 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:55:06.208 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:55:06.308 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-20T21:55:06.339 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-20T21:55:06.372 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-20T21:55:06.403 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-20T21:55:06.429 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-20T21:55:06.496 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:55:06.496 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:55:06.632 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-20T21:55:06.685 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-20T21:55:07.075 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-20T21:55:07.166 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librados2-2:19.1.1-367.g9e2ac70b.el9.x86_64 7/34 2024-09-20T21:55:07.259 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:19.1.1-367.g9e2ac70b.el9.x86_64 7/34 2024-09-20T21:55:07.357 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rados-2:19.1.1-367.g9e2ac70b.el9.x86_64 8/34 2024-09-20T21:55:07.384 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs2-2:19.1.1-367.g9e2ac70b.el9.x86_64 9/34 2024-09-20T21:55:07.430 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephfs2-2:19.1.1-367.g9e2ac70b.el9.x86_64 9/34 2024-09-20T21:55:07.467 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-20T21:55:07.511 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-argparse-2:19.1.1-367.g9e2ac70b.el9.x 11/34 2024-09-20T21:55:07.582 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cephfs-2:19.1.1-367.g9e2ac70b.el9.x86_64 12/34 2024-09-20T21:55:07.611 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libradosstriper1-2:19.1.1-367.g9e2ac70b.el9.x86_64 13/34 2024-09-20T21:55:07.676 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libradosstriper1-2:19.1.1-367.g9e2ac70b.el9.x86_64 13/34 2024-09-20T21:55:07.710 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-20T21:55:07.820 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-20T21:55:07.862 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-20T21:55:07.902 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-20T21:55:07.943 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-20T21:55:07.966 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:55:07.972 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-20T21:55:08.007 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-20T21:55:08.064 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-09-20T21:55:08.101 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-09-20T21:55:08.127 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-09-20T21:55:08.158 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-20T21:55:08.181 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-09-20T21:55:08.341 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-20T21:55:08.371 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-20T21:55:08.392 INFO:teuthology.orchestra.run.smithi060.stdout: Upgrading : librbd1-2:19.1.1-367.g9e2ac70b.el9.x86_64 22/34 2024-09-20T21:55:08.423 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-09-20T21:55:08.484 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:19.1.1-367.g9e2ac70b.el9.x86_64 22/34 2024-09-20T21:55:08.514 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rbd-2:19.1.1-367.g9e2ac70b.el9.x86_64 23/34 2024-09-20T21:55:08.550 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-20T21:55:08.787 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-09-20T21:55:08.895 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librados2-2:19.1.1-367.g9e2ac70b.el9.x86_64 7/34 2024-09-20T21:55:08.988 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:19.1.1-367.g9e2ac70b.el9.x86_64 7/34 2024-09-20T21:55:09.064 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-20T21:55:09.078 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rados-2:19.1.1-367.g9e2ac70b.el9.x86_64 8/34 2024-09-20T21:55:09.095 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librgw2-2:19.1.1-367.g9e2ac70b.el9.x86_64 26/34 2024-09-20T21:55:09.105 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs2-2:19.1.1-367.g9e2ac70b.el9.x86_64 9/34 2024-09-20T21:55:09.167 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librgw2-2:19.1.1-367.g9e2ac70b.el9.x86_64 26/34 2024-09-20T21:55:09.168 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephfs2-2:19.1.1-367.g9e2ac70b.el9.x86_64 9/34 2024-09-20T21:55:09.205 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-09-20T21:55:09.249 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-argparse-2:19.1.1-367.g9e2ac70b.el9.x 11/34 2024-09-20T21:55:09.263 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rgw-2:19.1.1-367.g9e2ac70b.el9.x86_64 27/34 2024-09-20T21:55:09.297 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ceph-common-2:19.1.1-367.g9e2ac70b.el9.x86 28/34 2024-09-20T21:55:09.329 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cephfs-2:19.1.1-367.g9e2ac70b.el9.x86_64 12/34 2024-09-20T21:55:09.482 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libradosstriper1-2:19.1.1-367.g9e2ac70b.el9.x86_64 13/34 2024-09-20T21:55:09.547 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libradosstriper1-2:19.1.1-367.g9e2ac70b.el9.x86_64 13/34 2024-09-20T21:55:09.581 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-09-20T21:55:09.700 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/34 2024-09-20T21:55:09.748 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-09-20T21:55:09.798 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-09-20T21:55:09.864 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/34 2024-09-20T21:55:09.893 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-09-20T21:55:09.926 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-09-20T21:55:10.253 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-09-20T21:55:10.288 INFO:teuthology.orchestra.run.smithi067.stdout: Upgrading : librbd1-2:19.1.1-367.g9e2ac70b.el9.x86_64 22/34 2024-09-20T21:55:10.380 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:19.1.1-367.g9e2ac70b.el9.x86_64 22/34 2024-09-20T21:55:10.410 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rbd-2:19.1.1-367.g9e2ac70b.el9.x86_64 23/34 2024-09-20T21:55:10.446 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-09-20T21:55:10.954 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-09-20T21:55:10.982 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librgw2-2:19.1.1-367.g9e2ac70b.el9.x86_64 26/34 2024-09-20T21:55:11.054 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librgw2-2:19.1.1-367.g9e2ac70b.el9.x86_64 26/34 2024-09-20T21:55:11.134 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rgw-2:19.1.1-367.g9e2ac70b.el9.x86_64 27/34 2024-09-20T21:55:11.168 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ceph-common-2:19.1.1-367.g9e2ac70b.el9.x86 28/34 2024-09-20T21:55:11.743 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:19.1.1-367.g9e2ac70b.el9.x86_64 29/34 2024-09-20T21:55:11.785 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-common-2:19.1.1-367.g9e2ac70b.el9.x86_64 29/34 2024-09-20T21:55:12.392 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-common-2:19.1.1-367.g9e2ac70b.el9.x86_64 29/34 2024-09-20T21:55:12.439 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-base-2:19.1.1-367.g9e2ac70b.el9.x86_64 30/34 2024-09-20T21:55:12.524 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-base-2:19.1.1-367.g9e2ac70b.el9.x86_64 30/34 2024-09-20T21:55:12.524 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-20T21:55:12.524 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-20T21:55:12.525 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:12.557 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-selinux-2:19.1.1-367.g9e2ac70b.el9.x86_64 31/34 2024-09-20T21:55:13.606 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:19.1.1-367.g9e2ac70b.el9.x86_64 29/34 2024-09-20T21:55:13.647 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-common-2:19.1.1-367.g9e2ac70b.el9.x86_64 29/34 2024-09-20T21:55:14.232 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-common-2:19.1.1-367.g9e2ac70b.el9.x86_64 29/34 2024-09-20T21:55:14.276 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-base-2:19.1.1-367.g9e2ac70b.el9.x86_64 30/34 2024-09-20T21:55:14.370 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-base-2:19.1.1-367.g9e2ac70b.el9.x86_64 30/34 2024-09-20T21:55:14.370 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-20T21:55:14.370 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-20T21:55:14.370 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:14.411 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-selinux-2:19.1.1-367.g9e2ac70b.el9.x86_64 31/34 2024-09-20T21:55:23.071 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-selinux-2:19.1.1-367.g9e2ac70b.el9.x86_64 31/34 2024-09-20T21:55:23.071 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /sys 2024-09-20T21:55:23.071 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /proc 2024-09-20T21:55:23.071 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /mnt 2024-09-20T21:55:23.071 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /var/tmp 2024-09-20T21:55:23.071 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /home 2024-09-20T21:55:23.071 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /root 2024-09-20T21:55:23.071 INFO:teuthology.orchestra.run.smithi060.stdout:skipping the directory /tmp 2024-09-20T21:55:23.072 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:24.081 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-radosgw-2:19.1.1-367.g9e2ac70b.el9.x86_64 32/34 2024-09-20T21:55:24.108 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-367.g9e2ac70b.el9.x86_64 32/34 2024-09-20T21:55:24.109 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T21:55:24.109 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-20T21:55:24.109 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-20T21:55:24.109 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-20T21:55:24.109 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:24.111 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-20T21:55:24.192 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-20T21:55:24.192 INFO:teuthology.orchestra.run.smithi060.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-20T21:55:24.618 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-selinux-2:19.1.1-367.g9e2ac70b.el9.x86_64 31/34 2024-09-20T21:55:24.619 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /sys 2024-09-20T21:55:24.619 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /proc 2024-09-20T21:55:24.619 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /mnt 2024-09-20T21:55:24.619 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /var/tmp 2024-09-20T21:55:24.619 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /home 2024-09-20T21:55:24.619 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /root 2024-09-20T21:55:24.619 INFO:teuthology.orchestra.run.smithi067.stdout:skipping the directory /tmp 2024-09-20T21:55:24.619 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:25.520 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-20T21:55:25.520 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-base-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/34 2024-09-20T21:55:25.520 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-common-2:19.1.1-367.g9e2ac70b.el9.x86_64 2/34 2024-09-20T21:55:25.520 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-radosgw-2:19.1.1-367.g9e2ac70b.el9.x86_64 3/34 2024-09-20T21:55:25.520 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-selinux-2:19.1.1-367.g9e2ac70b.el9.x86_64 4/34 2024-09-20T21:55:25.520 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs2-2:19.1.1-367.g9e2ac70b.el9.x86_64 5/34 2024-09-20T21:55:25.520 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libradosstriper1-2:19.1.1-367.g9e2ac70b.el9.x86_64 6/34 2024-09-20T21:55:25.521 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librgw2-2:19.1.1-367.g9e2ac70b.el9.x86_64 7/34 2024-09-20T21:55:25.521 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-argparse-2:19.1.1-367.g9e2ac70b.el9.x 8/34 2024-09-20T21:55:25.521 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ceph-common-2:19.1.1-367.g9e2ac70b.el9.x86 9/34 2024-09-20T21:55:25.521 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cephfs-2:19.1.1-367.g9e2ac70b.el9.x86_64 10/34 2024-09-20T21:55:25.521 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rados-2:19.1.1-367.g9e2ac70b.el9.x86_64 11/34 2024-09-20T21:55:25.521 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rbd-2:19.1.1-367.g9e2ac70b.el9.x86_64 12/34 2024-09-20T21:55:25.521 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rgw-2:19.1.1-367.g9e2ac70b.el9.x86_64 13/34 2024-09-20T21:55:25.521 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-20T21:55:25.521 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-20T21:55:25.521 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-20T21:55:25.521 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-20T21:55:25.521 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-20T21:55:25.521 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-20T21:55:25.521 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-20T21:55:25.522 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-20T21:55:25.522 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-20T21:55:25.522 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-20T21:55:25.522 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-20T21:55:25.522 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-20T21:55:25.522 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-20T21:55:25.522 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-20T21:55:25.522 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-20T21:55:25.522 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-20T21:55:25.522 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-20T21:55:25.522 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:19.1.1-367.g9e2ac70b.el9.x86_64 31/34 2024-09-20T21:55:25.522 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-20T21:55:25.522 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:19.1.1-367.g9e2ac70b.el9.x86_64 33/34 2024-09-20T21:55:25.660 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-radosgw-2:19.1.1-367.g9e2ac70b.el9.x86_64 32/34 2024-09-20T21:55:25.687 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-367.g9e2ac70b.el9.x86_64 32/34 2024-09-20T21:55:25.687 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T21:55:25.687 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-20T21:55:25.687 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-20T21:55:25.687 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-20T21:55:25.687 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:25.689 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-20T21:55:25.788 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-09-20T21:55:25.788 INFO:teuthology.orchestra.run.smithi067.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-20T21:55:26.718 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-20T21:55:26.718 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:26.718 INFO:teuthology.orchestra.run.smithi060.stdout:Upgraded: 2024-09-20T21:55:26.718 INFO:teuthology.orchestra.run.smithi060.stdout: librados2-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:26.718 INFO:teuthology.orchestra.run.smithi060.stdout: librbd1-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:26.718 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-20T21:55:26.719 INFO:teuthology.orchestra.run.smithi060.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-20T21:55:26.719 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-base-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:26.719 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-common-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:26.719 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-radosgw-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:26.719 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-selinux-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:26.719 INFO:teuthology.orchestra.run.smithi060.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-20T21:55:26.719 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-20T21:55:26.719 INFO:teuthology.orchestra.run.smithi060.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-20T21:55:26.719 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs2-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:26.719 INFO:teuthology.orchestra.run.smithi060.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-20T21:55:26.719 INFO:teuthology.orchestra.run.smithi060.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-20T21:55:26.719 INFO:teuthology.orchestra.run.smithi060.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-20T21:55:26.719 INFO:teuthology.orchestra.run.smithi060.stdout: libradosstriper1-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:26.719 INFO:teuthology.orchestra.run.smithi060.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-20T21:55:26.720 INFO:teuthology.orchestra.run.smithi060.stdout: librgw2-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:26.720 INFO:teuthology.orchestra.run.smithi060.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-20T21:55:26.720 INFO:teuthology.orchestra.run.smithi060.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-20T21:55:26.720 INFO:teuthology.orchestra.run.smithi060.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-20T21:55:26.720 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-argparse-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:26.720 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ceph-common-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:26.720 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cephfs-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:26.720 INFO:teuthology.orchestra.run.smithi060.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-20T21:55:26.720 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rados-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:26.720 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rbd-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:26.720 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rgw-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:26.720 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-20T21:55:26.720 INFO:teuthology.orchestra.run.smithi060.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-20T21:55:26.721 INFO:teuthology.orchestra.run.smithi060.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-20T21:55:26.721 INFO:teuthology.orchestra.run.smithi060.stdout: re2-1:20211101-20.el9.x86_64 2024-09-20T21:55:26.721 INFO:teuthology.orchestra.run.smithi060.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-20T21:55:26.721 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:26.721 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:55:26.987 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-09-20T21:55:26.988 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-base-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/34 2024-09-20T21:55:26.988 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-common-2:19.1.1-367.g9e2ac70b.el9.x86_64 2/34 2024-09-20T21:55:26.988 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-radosgw-2:19.1.1-367.g9e2ac70b.el9.x86_64 3/34 2024-09-20T21:55:26.988 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-selinux-2:19.1.1-367.g9e2ac70b.el9.x86_64 4/34 2024-09-20T21:55:26.988 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs2-2:19.1.1-367.g9e2ac70b.el9.x86_64 5/34 2024-09-20T21:55:26.988 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libradosstriper1-2:19.1.1-367.g9e2ac70b.el9.x86_64 6/34 2024-09-20T21:55:26.988 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librgw2-2:19.1.1-367.g9e2ac70b.el9.x86_64 7/34 2024-09-20T21:55:26.988 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-argparse-2:19.1.1-367.g9e2ac70b.el9.x 8/34 2024-09-20T21:55:26.988 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ceph-common-2:19.1.1-367.g9e2ac70b.el9.x86 9/34 2024-09-20T21:55:26.988 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cephfs-2:19.1.1-367.g9e2ac70b.el9.x86_64 10/34 2024-09-20T21:55:26.988 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rados-2:19.1.1-367.g9e2ac70b.el9.x86_64 11/34 2024-09-20T21:55:26.988 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rbd-2:19.1.1-367.g9e2ac70b.el9.x86_64 12/34 2024-09-20T21:55:26.989 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rgw-2:19.1.1-367.g9e2ac70b.el9.x86_64 13/34 2024-09-20T21:55:26.989 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-09-20T21:55:26.989 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-09-20T21:55:26.989 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-09-20T21:55:26.989 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-09-20T21:55:26.989 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-09-20T21:55:26.989 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-09-20T21:55:26.989 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-09-20T21:55:26.989 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-09-20T21:55:26.989 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-09-20T21:55:26.989 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 23/34 2024-09-20T21:55:26.989 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-09-20T21:55:26.989 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-09-20T21:55:26.989 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-09-20T21:55:26.989 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-09-20T21:55:26.990 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-09-20T21:55:26.990 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : re2-1:20211101-20.el9.x86_64 29/34 2024-09-20T21:55:26.990 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-09-20T21:55:26.991 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:19.1.1-367.g9e2ac70b.el9.x86_64 31/34 2024-09-20T21:55:26.991 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-09-20T21:55:26.991 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:19.1.1-367.g9e2ac70b.el9.x86_64 33/34 2024-09-20T21:55:27.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-test 2024-09-20T21:55:27.674 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:00:28 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:55:27.812 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:55:27.812 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:55:27.813 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-09-20T21:55:27.813 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:55:27.813 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-20T21:55:27.813 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 49 M 2024-09-20T21:55:27.813 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-09-20T21:55:27.813 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 163 k 2024-09-20T21:55:27.813 INFO:teuthology.orchestra.run.smithi060.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-20T21:55:27.813 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-20T21:55:27.813 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:27.813 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:55:27.813 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:55:27.813 INFO:teuthology.orchestra.run.smithi060.stdout:Install 4 Packages 2024-09-20T21:55:27.814 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:27.814 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 49 M 2024-09-20T21:55:27.814 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 209 M 2024-09-20T21:55:27.814 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-20T21:55:27.955 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-09-20T21:55:27.955 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:27.955 INFO:teuthology.orchestra.run.smithi067.stdout:Upgraded: 2024-09-20T21:55:27.955 INFO:teuthology.orchestra.run.smithi067.stdout: librados2-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:27.955 INFO:teuthology.orchestra.run.smithi067.stdout: librbd1-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:27.955 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-20T21:55:27.955 INFO:teuthology.orchestra.run.smithi067.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-20T21:55:27.955 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-base-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:27.955 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-common-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:27.955 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-radosgw-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:27.956 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-selinux-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:27.956 INFO:teuthology.orchestra.run.smithi067.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-20T21:55:27.956 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-20T21:55:27.956 INFO:teuthology.orchestra.run.smithi067.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-20T21:55:27.956 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs2-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:27.956 INFO:teuthology.orchestra.run.smithi067.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-20T21:55:27.956 INFO:teuthology.orchestra.run.smithi067.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-20T21:55:27.956 INFO:teuthology.orchestra.run.smithi067.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-20T21:55:27.956 INFO:teuthology.orchestra.run.smithi067.stdout: libradosstriper1-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:27.956 INFO:teuthology.orchestra.run.smithi067.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-20T21:55:27.956 INFO:teuthology.orchestra.run.smithi067.stdout: librgw2-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:27.956 INFO:teuthology.orchestra.run.smithi067.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-20T21:55:27.956 INFO:teuthology.orchestra.run.smithi067.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-20T21:55:27.957 INFO:teuthology.orchestra.run.smithi067.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-20T21:55:27.957 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-argparse-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:27.957 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ceph-common-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:27.957 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cephfs-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:27.957 INFO:teuthology.orchestra.run.smithi067.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-20T21:55:27.957 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rados-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:27.957 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rbd-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:27.957 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rgw-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:27.957 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-20T21:55:27.957 INFO:teuthology.orchestra.run.smithi067.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-20T21:55:27.957 INFO:teuthology.orchestra.run.smithi067.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-20T21:55:27.957 INFO:teuthology.orchestra.run.smithi067.stdout: re2-1:20211101-20.el9.x86_64 2024-09-20T21:55:27.957 INFO:teuthology.orchestra.run.smithi067.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-20T21:55:27.957 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:27.958 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:55:27.965 INFO:teuthology.orchestra.run.smithi060.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.5 MB/s | 304 kB 00:00 2024-09-20T21:55:27.999 INFO:teuthology.orchestra.run.smithi060.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2024-09-20T21:55:28.049 INFO:teuthology.orchestra.run.smithi060.stdout:(3/4): libcephsqlite-19.1.1-367.g9e2ac70b.el9.x 800 kB/s | 163 kB 00:00 2024-09-20T21:55:28.293 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-test 2024-09-20T21:55:28.906 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:00:29 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:55:29.043 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:55:29.044 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:55:29.044 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-09-20T21:55:29.044 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:55:29.044 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-20T21:55:29.044 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 49 M 2024-09-20T21:55:29.044 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-09-20T21:55:29.044 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 163 k 2024-09-20T21:55:29.044 INFO:teuthology.orchestra.run.smithi067.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-20T21:55:29.044 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-20T21:55:29.044 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:29.045 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:55:29.045 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:55:29.045 INFO:teuthology.orchestra.run.smithi067.stdout:Install 4 Packages 2024-09-20T21:55:29.045 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:29.045 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 49 M 2024-09-20T21:55:29.045 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 209 M 2024-09-20T21:55:29.045 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-20T21:55:29.308 INFO:teuthology.orchestra.run.smithi067.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.2 MB/s | 304 kB 00:00 2024-09-20T21:55:29.336 INFO:teuthology.orchestra.run.smithi067.stdout:(2/4): libcephsqlite-19.1.1-367.g9e2ac70b.el9.x 607 kB/s | 163 kB 00:00 2024-09-20T21:55:29.361 INFO:teuthology.orchestra.run.smithi067.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-20T21:55:29.391 INFO:teuthology.orchestra.run.smithi060.stdout:(4/4): ceph-test-19.1.1-367.g9e2ac70b.el9.x86_6 31 MB/s | 49 MB 00:01 2024-09-20T21:55:29.393 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:55:29.394 INFO:teuthology.orchestra.run.smithi060.stdout:Total 31 MB/s | 49 MB 00:01 2024-09-20T21:55:29.423 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:55:29.445 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:55:29.445 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:55:29.848 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:55:29.848 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:55:30.413 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:55:30.504 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-20T21:55:30.541 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-20T21:55:30.568 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephsqlite-2:19.1.1-367.g9e2ac70b.el9.x86_64 3/4 2024-09-20T21:55:30.661 INFO:teuthology.orchestra.run.smithi067.stdout:(4/4): ceph-test-19.1.1-367.g9e2ac70b.el9.x86_6 31 MB/s | 49 MB 00:01 2024-09-20T21:55:30.663 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:55:30.664 INFO:teuthology.orchestra.run.smithi067.stdout:Total 30 MB/s | 49 MB 00:01 2024-09-20T21:55:30.691 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:55:30.713 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:55:30.713 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:55:31.118 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:55:31.119 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:55:31.688 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:55:31.778 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-20T21:55:31.819 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-20T21:55:31.847 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephsqlite-2:19.1.1-367.g9e2ac70b.el9.x86_64 3/4 2024-09-20T21:55:35.445 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephsqlite-2:19.1.1-367.g9e2ac70b.el9.x86_64 3/4 2024-09-20T21:55:35.536 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-test-2:19.1.1-367.g9e2ac70b.el9.x86_64 4/4 2024-09-20T21:55:36.742 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephsqlite-2:19.1.1-367.g9e2ac70b.el9.x86_64 3/4 2024-09-20T21:55:36.831 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-test-2:19.1.1-367.g9e2ac70b.el9.x86_64 4/4 2024-09-20T21:55:37.976 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-test-2:19.1.1-367.g9e2ac70b.el9.x86_64 4/4 2024-09-20T21:55:37.976 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-test-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/4 2024-09-20T21:55:37.976 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephsqlite-2:19.1.1-367.g9e2ac70b.el9.x86_64 2/4 2024-09-20T21:55:37.976 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-20T21:55:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-20T21:55:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:38.402 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-20T21:55:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-test-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: libcephsqlite-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-20T21:55:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-20T21:55:38.402 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:38.403 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:55:38.620 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph 2024-09-20T21:55:39.214 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:00:40 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:55:39.281 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-test-2:19.1.1-367.g9e2ac70b.el9.x86_64 4/4 2024-09-20T21:55:39.281 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-test-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/4 2024-09-20T21:55:39.281 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephsqlite-2:19.1.1-367.g9e2ac70b.el9.x86_64 2/4 2024-09-20T21:55:39.281 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-20T21:55:39.353 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-09-20T21:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-20T21:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout: ceph x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 6.5 k 2024-09-20T21:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-09-20T21:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 2.4 M 2024-09-20T21:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 1.5 M 2024-09-20T21:55:39.356 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core noarch 2:19.1.1-367.g9e2ac70b.el9 ceph-noarch 249 k 2024-09-20T21:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 4.7 M 2024-09-20T21:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 17 M 2024-09-20T21:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-20T21:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-20T21:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-20T21:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-20T21:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-20T21:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-20T21:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-20T21:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-20T21:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-20T21:55:39.357 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-20T21:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-20T21:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-20T21:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-20T21:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-20T21:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-20T21:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-20T21:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-20T21:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-20T21:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-20T21:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-20T21:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-20T21:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-20T21:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-20T21:55:39.358 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-20T21:55:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-20T21:55:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-20T21:55:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-20T21:55:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-20T21:55:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-20T21:55:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-20T21:55:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-20T21:55:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:39.359 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:55:39.359 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:55:39.359 INFO:teuthology.orchestra.run.smithi060.stdout:Install 37 Packages 2024-09-20T21:55:39.359 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:39.360 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 30 M 2024-09-20T21:55:39.360 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 108 M 2024-09-20T21:55:39.360 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-20T21:55:39.684 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-20T21:55:39.684 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:39.684 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-20T21:55:39.684 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-test-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:39.684 INFO:teuthology.orchestra.run.smithi067.stdout: libcephsqlite-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:39.684 INFO:teuthology.orchestra.run.smithi067.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-20T21:55:39.685 INFO:teuthology.orchestra.run.smithi067.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-20T21:55:39.685 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:39.685 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:55:39.893 INFO:teuthology.orchestra.run.smithi060.stdout:(1/37): ceph-19.1.1-367.g9e2ac70b.el9.x86_64.rp 53 kB/s | 6.5 kB 00:00 2024-09-20T21:55:39.927 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph 2024-09-20T21:55:40.068 INFO:teuthology.orchestra.run.smithi060.stdout:(2/37): ceph-mgr-19.1.1-367.g9e2ac70b.el9.x86_6 5.1 MB/s | 1.5 MB 00:00 2024-09-20T21:55:40.169 INFO:teuthology.orchestra.run.smithi060.stdout:(3/37): ceph-mds-19.1.1-367.g9e2ac70b.el9.x86_6 6.0 MB/s | 2.4 MB 00:00 2024-09-20T21:55:40.244 INFO:teuthology.orchestra.run.smithi060.stdout:(4/37): ceph-mon-19.1.1-367.g9e2ac70b.el9.x86_6 13 MB/s | 4.7 MB 00:00 2024-09-20T21:55:40.269 INFO:teuthology.orchestra.run.smithi060.stdout:(5/37): ceph-mgr-modules-core-19.1.1-367.g9e2ac 2.4 MB/s | 249 kB 00:00 2024-09-20T21:55:40.470 INFO:teuthology.orchestra.run.smithi060.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-20T21:55:40.549 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:00:41 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:55:40.662 INFO:teuthology.orchestra.run.smithi060.stdout:(7/37): ceph-osd-19.1.1-367.g9e2ac70b.el9.x86_6 28 MB/s | 17 MB 00:00 2024-09-20T21:55:40.687 INFO:teuthology.orchestra.run.smithi060.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 490 kB/s | 106 kB 00:00 2024-09-20T21:55:40.694 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:55:40.697 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:55:40.697 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-09-20T21:55:40.697 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:55:40.697 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-20T21:55:40.697 INFO:teuthology.orchestra.run.smithi067.stdout: ceph x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 6.5 k 2024-09-20T21:55:40.697 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-09-20T21:55:40.697 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 2.4 M 2024-09-20T21:55:40.697 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 1.5 M 2024-09-20T21:55:40.698 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core noarch 2:19.1.1-367.g9e2ac70b.el9 ceph-noarch 249 k 2024-09-20T21:55:40.698 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 4.7 M 2024-09-20T21:55:40.698 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 17 M 2024-09-20T21:55:40.698 INFO:teuthology.orchestra.run.smithi067.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-20T21:55:40.698 INFO:teuthology.orchestra.run.smithi067.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-20T21:55:40.698 INFO:teuthology.orchestra.run.smithi067.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-20T21:55:40.698 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-20T21:55:40.698 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-20T21:55:40.698 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-20T21:55:40.698 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-20T21:55:40.698 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-20T21:55:40.698 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-20T21:55:40.698 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-20T21:55:40.698 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-20T21:55:40.699 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-20T21:55:40.699 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-20T21:55:40.699 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-20T21:55:40.699 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-20T21:55:40.699 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-20T21:55:40.699 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-20T21:55:40.699 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-20T21:55:40.699 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-20T21:55:40.699 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-20T21:55:40.699 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-20T21:55:40.699 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-20T21:55:40.699 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-20T21:55:40.699 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-20T21:55:40.699 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-20T21:55:40.700 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-20T21:55:40.700 INFO:teuthology.orchestra.run.smithi067.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-20T21:55:40.700 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-20T21:55:40.700 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-20T21:55:40.700 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-20T21:55:40.700 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-20T21:55:40.700 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:40.700 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:55:40.700 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:55:40.700 INFO:teuthology.orchestra.run.smithi067.stdout:Install 37 Packages 2024-09-20T21:55:40.700 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:40.701 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 30 M 2024-09-20T21:55:40.701 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 108 M 2024-09-20T21:55:40.701 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-20T21:55:40.729 INFO:teuthology.orchestra.run.smithi060.stdout:(9/37): python3-cryptography-36.0.1-4.el9.x86_6 2.7 MB/s | 1.2 MB 00:00 2024-09-20T21:55:40.777 INFO:teuthology.orchestra.run.smithi060.stdout:(10/37): python3-requests-2.25.1-8.el9.noarch.r 1.4 MB/s | 125 kB 00:00 2024-09-20T21:55:40.805 INFO:teuthology.orchestra.run.smithi060.stdout:(11/37): python3-urllib3-1.26.5-6.el9.noarch.rp 2.8 MB/s | 215 kB 00:00 2024-09-20T21:55:40.871 INFO:teuthology.orchestra.run.smithi060.stdout:(12/37): python3-pycparser-2.20-6.el9.noarch.rp 646 kB/s | 135 kB 00:00 2024-09-20T21:55:40.922 INFO:teuthology.orchestra.run.smithi060.stdout:(13/37): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-20T21:55:40.948 INFO:teuthology.orchestra.run.smithi060.stdout:(14/37): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-09-20T21:55:40.981 INFO:teuthology.orchestra.run.smithi060.stdout:(15/37): python3-markupsafe-1.1.1-12.el9.x86_64 319 kB/s | 35 kB 00:00 2024-09-20T21:55:41.006 INFO:teuthology.orchestra.run.smithi060.stdout:(16/37): python3-packaging-20.9-5.el9.noarch.rp 920 kB/s | 77 kB 00:00 2024-09-20T21:55:41.033 INFO:teuthology.orchestra.run.smithi060.stdout:(17/37): python3-pytz-2021.1-5.el9.noarch.rpm 608 kB/s | 51 kB 00:00 2024-09-20T21:55:41.057 INFO:teuthology.orchestra.run.smithi060.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 551 kB/s | 42 kB 00:00 2024-09-20T21:55:41.082 INFO:teuthology.orchestra.run.smithi060.stdout:(19/37): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-09-20T21:55:41.140 INFO:teuthology.orchestra.run.smithi060.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 747 kB/s | 43 kB 00:00 2024-09-20T21:55:41.166 INFO:teuthology.orchestra.run.smithi060.stdout:(21/37): luarocks-3.9.2-5.el9.noarch.rpm 1.1 MB/s | 151 kB 00:00 2024-09-20T21:55:41.178 INFO:teuthology.orchestra.run.smithi067.stdout:(1/37): ceph-19.1.1-367.g9e2ac70b.el9.x86_64.rp 53 kB/s | 6.5 kB 00:00 2024-09-20T21:55:41.191 INFO:teuthology.orchestra.run.smithi060.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 430 kB/s | 11 kB 00:00 2024-09-20T21:55:41.216 INFO:teuthology.orchestra.run.smithi060.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 714 kB/s | 18 kB 00:00 2024-09-20T21:55:41.250 INFO:teuthology.orchestra.run.smithi060.stdout:(24/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-20T21:55:41.275 INFO:teuthology.orchestra.run.smithi060.stdout:(25/37): python3-jaraco-collections-3.0.0-8.el9 395 kB/s | 23 kB 00:00 2024-09-20T21:55:41.300 INFO:teuthology.orchestra.run.smithi060.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-09-20T21:55:41.326 INFO:teuthology.orchestra.run.smithi060.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-09-20T21:55:41.351 INFO:teuthology.orchestra.run.smithi060.stdout:(28/37): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-09-20T21:55:41.354 INFO:teuthology.orchestra.run.smithi067.stdout:(2/37): ceph-mgr-19.1.1-367.g9e2ac70b.el9.x86_6 5.1 MB/s | 1.5 MB 00:00 2024-09-20T21:55:41.385 INFO:teuthology.orchestra.run.smithi060.stdout:(29/37): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-09-20T21:55:41.410 INFO:teuthology.orchestra.run.smithi060.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 939 kB/s | 79 kB 00:00 2024-09-20T21:55:41.429 INFO:teuthology.orchestra.run.smithi067.stdout:(3/37): ceph-mds-19.1.1-367.g9e2ac70b.el9.x86_6 6.4 MB/s | 2.4 MB 00:00 2024-09-20T21:55:41.435 INFO:teuthology.orchestra.run.smithi060.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-20T21:55:41.460 INFO:teuthology.orchestra.run.smithi060.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-20T21:55:41.485 INFO:teuthology.orchestra.run.smithi060.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-20T21:55:41.513 INFO:teuthology.orchestra.run.smithi067.stdout:(4/37): ceph-mgr-modules-core-19.1.1-367.g9e2ac 2.9 MB/s | 249 kB 00:00 2024-09-20T21:55:41.519 INFO:teuthology.orchestra.run.smithi060.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.6 MB/s | 272 kB 00:00 2024-09-20T21:55:41.545 INFO:teuthology.orchestra.run.smithi060.stdout:(35/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-09-20T21:55:41.570 INFO:teuthology.orchestra.run.smithi060.stdout:(36/37): python3-zc-lockfile-2.0-10.el9.noarch. 398 kB/s | 20 kB 00:00 2024-09-20T21:55:41.580 INFO:teuthology.orchestra.run.smithi067.stdout:(5/37): ceph-mon-19.1.1-367.g9e2ac70b.el9.x86_6 12 MB/s | 4.7 MB 00:00 2024-09-20T21:55:41.597 INFO:teuthology.orchestra.run.smithi060.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-20T21:55:41.603 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:55:41.603 INFO:teuthology.orchestra.run.smithi060.stdout:Total 14 MB/s | 30 MB 00:02 2024-09-20T21:55:41.760 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:55:41.772 INFO:teuthology.orchestra.run.smithi067.stdout:(6/37): python3-cffi-1.14.5-5.el9.x86_64.rpm 979 kB/s | 253 kB 00:00 2024-09-20T21:55:41.804 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:55:41.804 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:55:41.947 INFO:teuthology.orchestra.run.smithi067.stdout:(7/37): ceph-osd-19.1.1-367.g9e2ac70b.el9.x86_6 28 MB/s | 17 MB 00:00 2024-09-20T21:55:41.972 INFO:teuthology.orchestra.run.smithi067.stdout:(8/37): python3-ply-3.11-14.el9.noarch.rpm 532 kB/s | 106 kB 00:00 2024-09-20T21:55:42.047 INFO:teuthology.orchestra.run.smithi067.stdout:(9/37): python3-requests-2.25.1-8.el9.noarch.rp 1.6 MB/s | 125 kB 00:00 2024-09-20T21:55:42.089 INFO:teuthology.orchestra.run.smithi067.stdout:(10/37): python3-cryptography-36.0.1-4.el9.x86_ 2.4 MB/s | 1.2 MB 00:00 2024-09-20T21:55:42.123 INFO:teuthology.orchestra.run.smithi067.stdout:(11/37): python3-pycparser-2.20-6.el9.noarch.rp 770 kB/s | 135 kB 00:00 2024-09-20T21:55:42.148 INFO:teuthology.orchestra.run.smithi067.stdout:(12/37): python3-urllib3-1.26.5-6.el9.noarch.rp 2.1 MB/s | 215 kB 00:00 2024-09-20T21:55:42.282 INFO:teuthology.orchestra.run.smithi067.stdout:(13/37): python3-markupsafe-1.1.1-12.el9.x86_64 261 kB/s | 35 kB 00:00 2024-09-20T21:55:42.291 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:55:42.291 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:55:42.341 INFO:teuthology.orchestra.run.smithi067.stdout:(14/37): python3-mako-1.1.4-6.el9.noarch.rpm 793 kB/s | 172 kB 00:00 2024-09-20T21:55:42.365 INFO:teuthology.orchestra.run.smithi067.stdout:(15/37): lua-5.4.4-4.el9.x86_64.rpm 683 kB/s | 188 kB 00:00 2024-09-20T21:55:42.408 INFO:teuthology.orchestra.run.smithi067.stdout:(16/37): python3-pytz-2021.1-5.el9.noarch.rpm 764 kB/s | 51 kB 00:00 2024-09-20T21:55:42.433 INFO:teuthology.orchestra.run.smithi067.stdout:(17/37): python3-packaging-20.9-5.el9.noarch.rp 511 kB/s | 77 kB 00:00 2024-09-20T21:55:42.458 INFO:teuthology.orchestra.run.smithi067.stdout:(18/37): python3-toml-0.10.2-6.el9.noarch.rpm 451 kB/s | 42 kB 00:00 2024-09-20T21:55:42.508 INFO:teuthology.orchestra.run.smithi067.stdout:(19/37): luarocks-3.9.2-5.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-09-20T21:55:42.534 INFO:teuthology.orchestra.run.smithi067.stdout:(20/37): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 577 kB/s | 43 kB 00:00 2024-09-20T21:55:42.559 INFO:teuthology.orchestra.run.smithi067.stdout:(21/37): python3-cheroot-8.6.0-4.el9.noarch.rpm 3.5 MB/s | 172 kB 00:00 2024-09-20T21:55:42.584 INFO:teuthology.orchestra.run.smithi067.stdout:(22/37): python3-jaraco-8.2.1-3.el9.noarch.rpm 428 kB/s | 11 kB 00:00 2024-09-20T21:55:42.610 INFO:teuthology.orchestra.run.smithi067.stdout:(23/37): python3-jaraco-classes-3.2.1-5.el9.noa 707 kB/s | 18 kB 00:00 2024-09-20T21:55:42.635 INFO:teuthology.orchestra.run.smithi067.stdout:(24/37): python3-jaraco-collections-3.0.0-8.el9 923 kB/s | 23 kB 00:00 2024-09-20T21:55:42.660 INFO:teuthology.orchestra.run.smithi067.stdout:(25/37): python3-cherrypy-18.6.1-2.el9.noarch.r 2.8 MB/s | 358 kB 00:00 2024-09-20T21:55:42.686 INFO:teuthology.orchestra.run.smithi067.stdout:(26/37): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-09-20T21:55:42.711 INFO:teuthology.orchestra.run.smithi067.stdout:(27/37): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-20T21:55:42.736 INFO:teuthology.orchestra.run.smithi067.stdout:(28/37): lua-devel-5.4.4-4.el9.x86_64.rpm 68 kB/s | 22 kB 00:00 2024-09-20T21:55:42.770 INFO:teuthology.orchestra.run.smithi067.stdout:(29/37): python3-logutils-0.3.5-21.el9.noarch.r 552 kB/s | 46 kB 00:00 2024-09-20T21:55:42.795 INFO:teuthology.orchestra.run.smithi067.stdout:(30/37): python3-more-itertools-8.12.0-2.el9.no 942 kB/s | 79 kB 00:00 2024-09-20T21:55:42.820 INFO:teuthology.orchestra.run.smithi067.stdout:(31/37): python3-portend-3.1.0-2.el9.noarch.rpm 340 kB/s | 16 kB 00:00 2024-09-20T21:55:42.845 INFO:teuthology.orchestra.run.smithi067.stdout:(32/37): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-20T21:55:42.871 INFO:teuthology.orchestra.run.smithi067.stdout:(33/37): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-09-20T21:55:42.896 INFO:teuthology.orchestra.run.smithi067.stdout:(34/37): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-20T21:55:42.921 INFO:teuthology.orchestra.run.smithi067.stdout:(35/37): python3-zc-lockfile-2.0-10.el9.noarch. 801 kB/s | 20 kB 00:00 2024-09-20T21:55:42.947 INFO:teuthology.orchestra.run.smithi067.stdout:(36/37): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-20T21:55:42.980 INFO:teuthology.orchestra.run.smithi067.stdout:(37/37): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.8 MB/s | 427 kB 00:00 2024-09-20T21:55:42.983 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:55:42.984 INFO:teuthology.orchestra.run.smithi067.stdout:Total 13 MB/s | 30 MB 00:02 2024-09-20T21:55:43.140 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:55:43.184 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:55:43.185 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:55:43.641 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:55:43.660 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:55:43.660 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:55:43.720 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-20T21:55:43.764 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-20T21:55:43.794 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-20T21:55:43.869 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-20T21:55:43.910 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-20T21:55:43.943 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-20T21:55:44.040 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-20T21:55:44.124 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-20T21:55:44.171 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-20T21:55:44.205 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-20T21:55:44.235 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-20T21:55:44.269 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-20T21:55:44.303 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-20T21:55:44.344 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-20T21:55:44.387 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-20T21:55:44.429 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-20T21:55:44.479 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-20T21:55:44.518 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-20T21:55:44.594 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-20T21:55:44.739 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-20T21:55:44.820 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-20T21:55:44.872 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-20T21:55:44.918 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-20T21:55:44.985 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-20T21:55:44.999 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:55:45.049 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-20T21:55:45.058 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/37 2024-09-20T21:55:45.098 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/37 2024-09-20T21:55:45.127 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/37 2024-09-20T21:55:45.211 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/37 2024-09-20T21:55:45.251 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-20T21:55:45.268 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : luarocks-3.9.2-5.el9.noarch 5/37 2024-09-20T21:55:45.301 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/37 2024-09-20T21:55:45.305 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-20T21:55:45.395 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-20T21:55:45.397 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/37 2024-09-20T21:55:45.482 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/37 2024-09-20T21:55:45.491 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-20T21:55:45.529 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 9/37 2024-09-20T21:55:45.529 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-20T21:55:45.563 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/37 2024-09-20T21:55:45.593 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/37 2024-09-20T21:55:45.627 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/37 2024-09-20T21:55:45.661 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-20T21:55:45.669 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/37 2024-09-20T21:55:45.710 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/37 2024-09-20T21:55:45.762 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/37 2024-09-20T21:55:45.804 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/37 2024-09-20T21:55:45.825 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-367.g9e2ac70b.el9.n 32/37 2024-09-20T21:55:45.848 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-2:19.1.1-367.g9e2ac70b.el9.x86_64 33/37 2024-09-20T21:55:45.853 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/37 2024-09-20T21:55:45.875 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-2:19.1.1-367.g9e2ac70b.el9.x86_64 33/37 2024-09-20T21:55:45.875 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T21:55:45.876 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-20T21:55:45.876 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-20T21:55:45.876 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-20T21:55:45.876 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:45.893 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-packaging-20.9-5.el9.noarch 18/37 2024-09-20T21:55:45.970 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 19/37 2024-09-20T21:55:46.122 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 20/37 2024-09-20T21:55:46.205 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 21/37 2024-09-20T21:55:46.256 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 22/37 2024-09-20T21:55:46.301 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 23/37 2024-09-20T21:55:46.360 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/37 2024-09-20T21:55:46.424 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/37 2024-09-20T21:55:46.620 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/37 2024-09-20T21:55:46.671 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 27/37 2024-09-20T21:55:46.755 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/37 2024-09-20T21:55:46.854 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 29/37 2024-09-20T21:55:46.889 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/37 2024-09-20T21:55:47.011 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/37 2024-09-20T21:55:47.177 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-367.g9e2ac70b.el9.n 32/37 2024-09-20T21:55:47.207 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-2:19.1.1-367.g9e2ac70b.el9.x86_64 33/37 2024-09-20T21:55:47.235 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-2:19.1.1-367.g9e2ac70b.el9.x86_64 33/37 2024-09-20T21:55:47.235 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T21:55:47.235 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-20T21:55:47.235 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-20T21:55:47.235 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-20T21:55:47.235 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:47.406 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-osd-2:19.1.1-367.g9e2ac70b.el9.x86_64 34/37 2024-09-20T21:55:47.435 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-osd-2:19.1.1-367.g9e2ac70b.el9.x86_64 34/37 2024-09-20T21:55:47.435 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T21:55:47.435 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-20T21:55:47.436 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-20T21:55:47.436 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-20T21:55:47.436 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:47.907 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mon-2:19.1.1-367.g9e2ac70b.el9.x86_64 35/37 2024-09-20T21:55:47.932 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mon-2:19.1.1-367.g9e2ac70b.el9.x86_64 35/37 2024-09-20T21:55:47.933 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T21:55:47.933 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-20T21:55:47.933 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-20T21:55:47.933 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-20T21:55:47.933 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:48.174 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mds-2:19.1.1-367.g9e2ac70b.el9.x86_64 36/37 2024-09-20T21:55:48.199 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mds-2:19.1.1-367.g9e2ac70b.el9.x86_64 36/37 2024-09-20T21:55:48.200 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T21:55:48.200 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-20T21:55:48.200 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-20T21:55:48.200 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-20T21:55:48.200 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:48.266 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-2:19.1.1-367.g9e2ac70b.el9.x86_64 37/37 2024-09-20T21:55:48.775 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-osd-2:19.1.1-367.g9e2ac70b.el9.x86_64 34/37 2024-09-20T21:55:48.804 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-osd-2:19.1.1-367.g9e2ac70b.el9.x86_64 34/37 2024-09-20T21:55:48.804 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T21:55:48.804 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-20T21:55:48.804 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-20T21:55:48.804 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-20T21:55:48.805 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:49.275 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mon-2:19.1.1-367.g9e2ac70b.el9.x86_64 35/37 2024-09-20T21:55:49.301 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mon-2:19.1.1-367.g9e2ac70b.el9.x86_64 35/37 2024-09-20T21:55:49.301 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T21:55:49.301 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-20T21:55:49.301 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-20T21:55:49.301 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-20T21:55:49.301 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:49.543 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mds-2:19.1.1-367.g9e2ac70b.el9.x86_64 36/37 2024-09-20T21:55:49.570 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mds-2:19.1.1-367.g9e2ac70b.el9.x86_64 36/37 2024-09-20T21:55:49.570 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T21:55:49.570 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-20T21:55:49.570 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-20T21:55:49.570 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-20T21:55:49.570 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:49.635 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-2:19.1.1-367.g9e2ac70b.el9.x86_64 37/37 2024-09-20T21:55:50.088 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-2:19.1.1-367.g9e2ac70b.el9.x86_64 37/37 2024-09-20T21:55:50.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/37 2024-09-20T21:55:50.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mds-2:19.1.1-367.g9e2ac70b.el9.x86_64 2/37 2024-09-20T21:55:50.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-2:19.1.1-367.g9e2ac70b.el9.x86_64 3/37 2024-09-20T21:55:50.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mon-2:19.1.1-367.g9e2ac70b.el9.x86_64 4/37 2024-09-20T21:55:50.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-osd-2:19.1.1-367.g9e2ac70b.el9.x86_64 5/37 2024-09-20T21:55:50.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-367.g9e2ac70b.el9.n 6/37 2024-09-20T21:55:50.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-20T21:55:50.088 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-20T21:55:50.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-20T21:55:50.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-20T21:55:50.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-20T21:55:50.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-20T21:55:50.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-20T21:55:50.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-20T21:55:50.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-20T21:55:50.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-20T21:55:50.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-20T21:55:50.089 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-20T21:55:50.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-20T21:55:50.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-20T21:55:50.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-20T21:55:50.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-20T21:55:50.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-20T21:55:50.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-20T21:55:50.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-20T21:55:50.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-20T21:55:50.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-20T21:55:50.090 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-20T21:55:50.091 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-20T21:55:50.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-20T21:55:50.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-20T21:55:50.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-20T21:55:50.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-20T21:55:50.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-20T21:55:50.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-20T21:55:50.092 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-20T21:55:51.171 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-20T21:55:51.171 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:51.171 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-20T21:55:51.171 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:51.171 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mds-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:51.171 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:51.171 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-modules-core-2:19.1.1-367.g9e2ac70b.el9.noarch 2024-09-20T21:55:51.171 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mon-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:51.171 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-osd-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:51.171 INFO:teuthology.orchestra.run.smithi060.stdout: lua-5.4.4-4.el9.x86_64 2024-09-20T21:55:51.171 INFO:teuthology.orchestra.run.smithi060.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-20T21:55:51.171 INFO:teuthology.orchestra.run.smithi060.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-20T21:55:51.172 INFO:teuthology.orchestra.run.smithi060.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-20T21:55:51.172 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-20T21:55:51.172 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-20T21:55:51.172 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-20T21:55:51.172 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-20T21:55:51.172 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-20T21:55:51.172 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-20T21:55:51.172 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-20T21:55:51.172 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-20T21:55:51.172 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-20T21:55:51.172 INFO:teuthology.orchestra.run.smithi060.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-20T21:55:51.172 INFO:teuthology.orchestra.run.smithi060.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-20T21:55:51.172 INFO:teuthology.orchestra.run.smithi060.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-20T21:55:51.172 INFO:teuthology.orchestra.run.smithi060.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-20T21:55:51.173 INFO:teuthology.orchestra.run.smithi060.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-20T21:55:51.173 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-20T21:55:51.173 INFO:teuthology.orchestra.run.smithi060.stdout: python3-ply-3.11-14.el9.noarch 2024-09-20T21:55:51.173 INFO:teuthology.orchestra.run.smithi060.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-20T21:55:51.173 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-20T21:55:51.173 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-20T21:55:51.173 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-20T21:55:51.173 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-20T21:55:51.173 INFO:teuthology.orchestra.run.smithi060.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-20T21:55:51.173 INFO:teuthology.orchestra.run.smithi060.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-20T21:55:51.173 INFO:teuthology.orchestra.run.smithi060.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-20T21:55:51.173 INFO:teuthology.orchestra.run.smithi060.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-20T21:55:51.173 INFO:teuthology.orchestra.run.smithi060.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-20T21:55:51.173 INFO:teuthology.orchestra.run.smithi060.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-20T21:55:51.173 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:51.174 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:55:51.463 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-2:19.1.1-367.g9e2ac70b.el9.x86_64 37/37 2024-09-20T21:55:51.463 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/37 2024-09-20T21:55:51.463 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mds-2:19.1.1-367.g9e2ac70b.el9.x86_64 2/37 2024-09-20T21:55:51.463 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-2:19.1.1-367.g9e2ac70b.el9.x86_64 3/37 2024-09-20T21:55:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mon-2:19.1.1-367.g9e2ac70b.el9.x86_64 4/37 2024-09-20T21:55:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-osd-2:19.1.1-367.g9e2ac70b.el9.x86_64 5/37 2024-09-20T21:55:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-367.g9e2ac70b.el9.n 6/37 2024-09-20T21:55:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/37 2024-09-20T21:55:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/37 2024-09-20T21:55:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/37 2024-09-20T21:55:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/37 2024-09-20T21:55:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/37 2024-09-20T21:55:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/37 2024-09-20T21:55:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/37 2024-09-20T21:55:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/37 2024-09-20T21:55:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/37 2024-09-20T21:55:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 16/37 2024-09-20T21:55:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 17/37 2024-09-20T21:55:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 18/37 2024-09-20T21:55:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 19/37 2024-09-20T21:55:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 20/37 2024-09-20T21:55:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 21/37 2024-09-20T21:55:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 22/37 2024-09-20T21:55:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 23/37 2024-09-20T21:55:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 24/37 2024-09-20T21:55:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 25/37 2024-09-20T21:55:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 26/37 2024-09-20T21:55:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 27/37 2024-09-20T21:55:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 28/37 2024-09-20T21:55:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 29/37 2024-09-20T21:55:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 30/37 2024-09-20T21:55:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 31/37 2024-09-20T21:55:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 32/37 2024-09-20T21:55:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 33/37 2024-09-20T21:55:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 34/37 2024-09-20T21:55:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 35/37 2024-09-20T21:55:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 36/37 2024-09-20T21:55:51.576 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-base 2024-09-20T21:55:52.182 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:00:53 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:55:52.275 INFO:teuthology.orchestra.run.smithi060.stdout:Package ceph-base-2:19.1.1-367.g9e2ac70b.el9.x86_64 is already installed. 2024-09-20T21:55:52.320 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:55:52.321 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-20T21:55:52.321 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:55:52.394 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install cephadm 2024-09-20T21:55:52.546 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 37/37 2024-09-20T21:55:52.547 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:52.547 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-20T21:55:52.547 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:52.547 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mds-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:52.547 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:52.547 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-modules-core-2:19.1.1-367.g9e2ac70b.el9.noarch 2024-09-20T21:55:52.547 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mon-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:52.547 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-osd-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:52.547 INFO:teuthology.orchestra.run.smithi067.stdout: lua-5.4.4-4.el9.x86_64 2024-09-20T21:55:52.547 INFO:teuthology.orchestra.run.smithi067.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-20T21:55:52.547 INFO:teuthology.orchestra.run.smithi067.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-20T21:55:52.548 INFO:teuthology.orchestra.run.smithi067.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-20T21:55:52.548 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-20T21:55:52.548 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-20T21:55:52.548 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-20T21:55:52.548 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-20T21:55:52.548 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-20T21:55:52.548 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-20T21:55:52.548 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-20T21:55:52.548 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-20T21:55:52.548 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-20T21:55:52.548 INFO:teuthology.orchestra.run.smithi067.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-20T21:55:52.548 INFO:teuthology.orchestra.run.smithi067.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-20T21:55:52.548 INFO:teuthology.orchestra.run.smithi067.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-20T21:55:52.548 INFO:teuthology.orchestra.run.smithi067.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-20T21:55:52.549 INFO:teuthology.orchestra.run.smithi067.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-20T21:55:52.549 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-20T21:55:52.549 INFO:teuthology.orchestra.run.smithi067.stdout: python3-ply-3.11-14.el9.noarch 2024-09-20T21:55:52.549 INFO:teuthology.orchestra.run.smithi067.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-20T21:55:52.549 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-20T21:55:52.549 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-20T21:55:52.549 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-20T21:55:52.549 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-20T21:55:52.549 INFO:teuthology.orchestra.run.smithi067.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-20T21:55:52.549 INFO:teuthology.orchestra.run.smithi067.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-20T21:55:52.549 INFO:teuthology.orchestra.run.smithi067.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-20T21:55:52.549 INFO:teuthology.orchestra.run.smithi067.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-20T21:55:52.549 INFO:teuthology.orchestra.run.smithi067.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-20T21:55:52.550 INFO:teuthology.orchestra.run.smithi067.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-20T21:55:52.550 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:52.550 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:55:52.923 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-base 2024-09-20T21:55:52.995 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:00:53 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:55:53.131 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:55:53.132 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:55:53.132 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-09-20T21:55:53.132 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:55:53.132 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-20T21:55:53.133 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm noarch 2:19.1.1-367.g9e2ac70b.el9 ceph-noarch 765 k 2024-09-20T21:55:53.133 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:53.133 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:55:53.133 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:55:53.133 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2024-09-20T21:55:53.133 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:53.133 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 765 k 2024-09-20T21:55:53.133 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 771 k 2024-09-20T21:55:53.133 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-20T21:55:53.416 INFO:teuthology.orchestra.run.smithi060.stdout:cephadm-19.1.1-367.g9e2ac70b.el9.noarch.rpm 2.6 MB/s | 765 kB 00:00 2024-09-20T21:55:53.417 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:55:53.417 INFO:teuthology.orchestra.run.smithi060.stdout:Total 2.6 MB/s | 765 kB 00:00 2024-09-20T21:55:53.417 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:55:53.423 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:55:53.423 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:55:53.429 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:55:53.429 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:55:53.502 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:55:53.531 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:00:54 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:55:53.624 INFO:teuthology.orchestra.run.smithi067.stdout:Package ceph-base-2:19.1.1-367.g9e2ac70b.el9.x86_64 is already installed. 2024-09-20T21:55:53.669 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:55:53.670 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2024-09-20T21:55:53.670 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:55:53.734 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install cephadm 2024-09-20T21:55:53.976 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephadm-2:19.1.1-367.g9e2ac70b.el9.noarch 1/1 2024-09-20T21:55:54.072 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : cephadm-2:19.1.1-367.g9e2ac70b.el9.noarch 1/1 2024-09-20T21:55:54.335 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:00:55 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:55:54.444 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: cephadm-2:19.1.1-367.g9e2ac70b.el9.noarch 1/1 2024-09-20T21:55:54.474 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:55:54.474 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:55:54.474 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-09-20T21:55:54.474 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:55:54.474 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-20T21:55:54.475 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm noarch 2:19.1.1-367.g9e2ac70b.el9 ceph-noarch 765 k 2024-09-20T21:55:54.475 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:54.475 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:55:54.475 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:55:54.475 INFO:teuthology.orchestra.run.smithi067.stdout:Install 1 Package 2024-09-20T21:55:54.475 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:54.475 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 765 k 2024-09-20T21:55:54.475 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 771 k 2024-09-20T21:55:54.475 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-20T21:55:54.676 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : cephadm-2:19.1.1-367.g9e2ac70b.el9.noarch 1/1 2024-09-20T21:55:54.676 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:54.677 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-20T21:55:54.677 INFO:teuthology.orchestra.run.smithi060.stdout: cephadm-2:19.1.1-367.g9e2ac70b.el9.noarch 2024-09-20T21:55:54.677 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:54.677 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:55:54.758 INFO:teuthology.orchestra.run.smithi067.stdout:cephadm-19.1.1-367.g9e2ac70b.el9.noarch.rpm 2.6 MB/s | 765 kB 00:00 2024-09-20T21:55:54.759 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:55:54.759 INFO:teuthology.orchestra.run.smithi067.stdout:Total 2.6 MB/s | 765 kB 00:00 2024-09-20T21:55:54.759 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:55:54.765 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:55:54.765 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:55:54.771 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:55:54.771 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:55:54.851 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:55:54.896 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-immutable-object-cache 2024-09-20T21:55:55.368 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephadm-2:19.1.1-367.g9e2ac70b.el9.noarch 1/1 2024-09-20T21:55:55.465 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : cephadm-2:19.1.1-367.g9e2ac70b.el9.noarch 1/1 2024-09-20T21:55:55.509 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:00:56 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:55:55.649 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:55:55.651 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:55:55.651 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repo Size 2024-09-20T21:55:55.651 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:55:55.651 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-20T21:55:55.651 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 145 k 2024-09-20T21:55:55.651 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:55.651 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:55:55.652 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:55:55.652 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2024-09-20T21:55:55.652 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:55.652 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 145 k 2024-09-20T21:55:55.652 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 436 k 2024-09-20T21:55:55.652 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-20T21:55:55.832 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: cephadm-2:19.1.1-367.g9e2ac70b.el9.noarch 1/1 2024-09-20T21:55:55.855 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-immutable-object-cache-19.1.1-367.g9e2ac70 713 kB/s | 145 kB 00:00 2024-09-20T21:55:55.856 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:55:55.856 INFO:teuthology.orchestra.run.smithi060.stdout:Total 707 kB/s | 145 kB 00:00 2024-09-20T21:55:55.856 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:55:55.864 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:55:55.864 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:55:55.931 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:55:55.931 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:55:56.079 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : cephadm-2:19.1.1-367.g9e2ac70b.el9.noarch 1/1 2024-09-20T21:55:56.079 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:56.079 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-20T21:55:56.079 INFO:teuthology.orchestra.run.smithi067.stdout: cephadm-2:19.1.1-367.g9e2ac70b.el9.noarch 2024-09-20T21:55:56.079 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:56.080 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:55:56.095 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:55:56.198 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-367.g9e2ac70b.e 1/1 2024-09-20T21:55:56.225 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-367.g9e2ac70b.e 1/1 2024-09-20T21:55:56.225 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T21:55:56.225 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-20T21:55:56.225 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:56.289 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-immutable-object-cache 2024-09-20T21:55:56.895 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:00:57 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:55:57.035 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:55:57.035 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:55:57.035 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repo Size 2024-09-20T21:55:57.036 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:55:57.036 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-20T21:55:57.036 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 145 k 2024-09-20T21:55:57.036 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:57.036 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:55:57.036 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:55:57.036 INFO:teuthology.orchestra.run.smithi067.stdout:Install 1 Package 2024-09-20T21:55:57.036 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:57.037 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 145 k 2024-09-20T21:55:57.037 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 436 k 2024-09-20T21:55:57.037 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-20T21:55:57.199 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-367.g9e2ac70b.e 1/1 2024-09-20T21:55:57.200 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:57.200 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-20T21:55:57.200 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-immutable-object-cache-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:57.200 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:57.200 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:55:57.238 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-immutable-object-cache-19.1.1-367.g9e2ac70 721 kB/s | 145 kB 00:00 2024-09-20T21:55:57.239 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:55:57.239 INFO:teuthology.orchestra.run.smithi067.stdout:Total 714 kB/s | 145 kB 00:00 2024-09-20T21:55:57.240 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:55:57.248 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:55:57.248 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:55:57.313 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:55:57.313 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:55:57.430 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-mgr 2024-09-20T21:55:57.471 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:55:57.531 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-367.g9e2ac70b.e 1/1 2024-09-20T21:55:57.558 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-367.g9e2ac70b.e 1/1 2024-09-20T21:55:57.558 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T21:55:57.558 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-20T21:55:57.558 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:58.037 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:00:59 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:55:58.129 INFO:teuthology.orchestra.run.smithi060.stdout:Package ceph-mgr-2:19.1.1-367.g9e2ac70b.el9.x86_64 is already installed. 2024-09-20T21:55:58.175 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:55:58.176 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-20T21:55:58.176 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:55:58.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-mgr-dashboard 2024-09-20T21:55:58.497 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-367.g9e2ac70b.e 1/1 2024-09-20T21:55:58.498 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:58.498 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-20T21:55:58.498 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-immutable-object-cache-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:55:58.498 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:55:58.498 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:55:58.707 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-mgr 2024-09-20T21:55:58.854 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:00:59 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:55:58.992 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:55:58.993 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:55:58.993 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-09-20T21:55:58.994 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:55:58.994 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-20T21:55:58.994 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard noarch 2:19.1.1-367.g9e2ac70b.el9 ceph-noarch 3.6 M 2024-09-20T21:55:58.994 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-09-20T21:55:58.994 INFO:teuthology.orchestra.run.smithi060.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-20T21:55:58.994 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards noarch 2:19.1.1-367.g9e2ac70b.el9 ceph-noarch 32 k 2024-09-20T21:55:58.994 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts noarch 2:19.1.1-367.g9e2ac70b.el9 ceph-noarch 16 k 2024-09-20T21:55:58.994 INFO:teuthology.orchestra.run.smithi060.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-20T21:55:58.994 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-20T21:55:58.994 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-20T21:55:58.994 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-20T21:55:58.994 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-20T21:55:58.994 INFO:teuthology.orchestra.run.smithi060.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-20T21:55:58.995 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-20T21:55:58.995 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-20T21:55:58.995 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:58.995 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:55:58.995 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:55:58.995 INFO:teuthology.orchestra.run.smithi060.stdout:Install 12 Packages 2024-09-20T21:55:58.995 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:55:58.995 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 8.7 M 2024-09-20T21:55:58.996 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 105 M 2024-09-20T21:55:58.996 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-20T21:55:59.314 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:00 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:55:59.411 INFO:teuthology.orchestra.run.smithi067.stdout:Package ceph-mgr-2:19.1.1-367.g9e2ac70b.el9.x86_64 is already installed. 2024-09-20T21:55:59.456 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:55:59.457 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2024-09-20T21:55:59.457 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:55:59.467 INFO:teuthology.orchestra.run.smithi060.stdout:(1/12): ceph-grafana-dashboards-19.1.1-367.g9e2 205 kB/s | 32 kB 00:00 2024-09-20T21:55:59.492 INFO:teuthology.orchestra.run.smithi060.stdout:(2/12): ceph-prometheus-alerts-19.1.1-367.g9e2a 91 kB/s | 16 kB 00:00 2024-09-20T21:55:59.520 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-mgr-dashboard 2024-09-20T21:55:59.684 INFO:teuthology.orchestra.run.smithi060.stdout:(3/12): ceph-mgr-dashboard-19.1.1-367.g9e2ac70b 9.6 MB/s | 3.6 MB 00:00 2024-09-20T21:55:59.709 INFO:teuthology.orchestra.run.smithi060.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.2 MB/s | 269 kB 00:00 2024-09-20T21:55:59.752 INFO:teuthology.orchestra.run.smithi060.stdout:(5/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.6 MB/s | 1.0 MB 00:00 2024-09-20T21:55:59.802 INFO:teuthology.orchestra.run.smithi060.stdout:(6/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 5.8 MB/s | 548 kB 00:00 2024-09-20T21:55:59.852 INFO:teuthology.orchestra.run.smithi060.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 198 kB/s | 19 kB 00:00 2024-09-20T21:55:59.886 INFO:teuthology.orchestra.run.smithi060.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 4.2 MB/s | 144 kB 00:00 2024-09-20T21:55:59.911 INFO:teuthology.orchestra.run.smithi060.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 1.2 MB/s | 31 kB 00:00 2024-09-20T21:55:59.936 INFO:teuthology.orchestra.run.smithi060.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 7.4 MB/s | 188 kB 00:00 2024-09-20T21:55:59.978 INFO:teuthology.orchestra.run.smithi060.stdout:(11/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 12 MB/s | 2.0 MB 00:00 2024-09-20T21:56:00.095 INFO:teuthology.orchestra.run.smithi060.stdout:(12/12): protobuf-compiler-3.14.0-13.el9.x86_64 2.1 MB/s | 863 kB 00:00 2024-09-20T21:56:00.100 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:00.100 INFO:teuthology.orchestra.run.smithi060.stdout:Total 7.9 MB/s | 8.7 MB 00:01 2024-09-20T21:56:00.122 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:01 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:00.152 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:56:00.171 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:56:00.171 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:56:00.259 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:56:00.260 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:00.260 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-09-20T21:56:00.261 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:00.261 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-20T21:56:00.261 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard noarch 2:19.1.1-367.g9e2ac70b.el9 ceph-noarch 3.6 M 2024-09-20T21:56:00.261 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-09-20T21:56:00.261 INFO:teuthology.orchestra.run.smithi067.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-20T21:56:00.261 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards noarch 2:19.1.1-367.g9e2ac70b.el9 ceph-noarch 32 k 2024-09-20T21:56:00.261 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts noarch 2:19.1.1-367.g9e2ac70b.el9 ceph-noarch 16 k 2024-09-20T21:56:00.261 INFO:teuthology.orchestra.run.smithi067.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-20T21:56:00.261 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-20T21:56:00.261 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-20T21:56:00.261 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-20T21:56:00.261 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-20T21:56:00.261 INFO:teuthology.orchestra.run.smithi067.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-20T21:56:00.261 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-20T21:56:00.262 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-20T21:56:00.262 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:00.262 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:56:00.262 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:00.262 INFO:teuthology.orchestra.run.smithi067.stdout:Install 12 Packages 2024-09-20T21:56:00.262 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:00.262 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 8.7 M 2024-09-20T21:56:00.263 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 105 M 2024-09-20T21:56:00.263 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-20T21:56:00.385 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:56:00.385 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:56:00.858 INFO:teuthology.orchestra.run.smithi067.stdout:(1/12): ceph-grafana-dashboards-19.1.1-367.g9e2 201 kB/s | 32 kB 00:00 2024-09-20T21:56:00.883 INFO:teuthology.orchestra.run.smithi067.stdout:(2/12): ceph-prometheus-alerts-19.1.1-367.g9e2a 90 kB/s | 16 kB 00:00 2024-09-20T21:56:00.964 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:56:01.009 INFO:teuthology.orchestra.run.smithi067.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 2.1 MB/s | 269 kB 00:00 2024-09-20T21:56:01.046 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-20T21:56:01.059 INFO:teuthology.orchestra.run.smithi067.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 5.0 MB/s | 1.0 MB 00:00 2024-09-20T21:56:01.090 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-20T21:56:01.110 INFO:teuthology.orchestra.run.smithi067.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 8.4 MB/s | 863 kB 00:00 2024-09-20T21:56:01.137 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-20T21:56:01.175 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-20T21:56:01.176 INFO:teuthology.orchestra.run.smithi067.stdout:(6/12): ceph-mgr-dashboard-19.1.1-367.g9e2ac70b 7.5 MB/s | 3.6 MB 00:00 2024-09-20T21:56:01.210 INFO:teuthology.orchestra.run.smithi067.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 194 kB/s | 19 kB 00:00 2024-09-20T21:56:01.244 INFO:teuthology.orchestra.run.smithi067.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.9 MB/s | 548 kB 00:00 2024-09-20T21:56:01.269 INFO:teuthology.orchestra.run.smithi067.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.5 MB/s | 144 kB 00:00 2024-09-20T21:56:01.276 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-20T21:56:01.294 INFO:teuthology.orchestra.run.smithi067.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 614 kB/s | 31 kB 00:00 2024-09-20T21:56:01.320 INFO:teuthology.orchestra.run.smithi067.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-20T21:56:01.370 INFO:teuthology.orchestra.run.smithi067.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 11 MB/s | 2.0 MB 00:00 2024-09-20T21:56:01.374 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:01.374 INFO:teuthology.orchestra.run.smithi067.stdout:Total 7.8 MB/s | 8.7 MB 00:01 2024-09-20T21:56:01.415 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-20T21:56:01.425 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:56:01.444 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:56:01.444 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:56:01.507 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-20T21:56:01.556 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-20T21:56:01.583 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-20T21:56:01.623 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-367.g9e2ac70b.el9. 10/12 2024-09-20T21:56:01.663 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:56:01.663 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:56:02.219 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:56:02.296 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-09-20T21:56:02.339 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-09-20T21:56:02.387 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-09-20T21:56:02.416 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-09-20T21:56:02.518 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-09-20T21:56:02.657 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-09-20T21:56:02.748 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-09-20T21:56:02.752 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-367.g9e2ac70b.el9 11/12 2024-09-20T21:56:02.791 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-367.g9e2ac70b.el9.noar 12/12 2024-09-20T21:56:02.805 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-09-20T21:56:02.841 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-09-20T21:56:02.873 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-367.g9e2ac70b.el9. 10/12 2024-09-20T21:56:03.999 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-367.g9e2ac70b.el9 11/12 2024-09-20T21:56:04.031 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-367.g9e2ac70b.el9.noar 12/12 2024-09-20T21:56:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-367.g9e2ac70b.el9.noar 12/12 2024-09-20T21:56:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-367.g9e2ac70b.el9 1/12 2024-09-20T21:56:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-367.g9e2ac70b.el9.noar 2/12 2024-09-20T21:56:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-367.g9e2ac70b.el9. 3/12 2024-09-20T21:56:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-20T21:56:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-20T21:56:04.151 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-20T21:56:04.152 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-20T21:56:04.152 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-20T21:56:04.152 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-20T21:56:04.152 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-20T21:56:04.152 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-20T21:56:04.477 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-20T21:56:04.477 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:04.477 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-20T21:56:04.477 INFO:teuthology.orchestra.run.smithi060.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-20T21:56:04.477 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-grafana-dashboards-2:19.1.1-367.g9e2ac70b.el9.noarch 2024-09-20T21:56:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-dashboard-2:19.1.1-367.g9e2ac70b.el9.noarch 2024-09-20T21:56:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-prometheus-alerts-2:19.1.1-367.g9e2ac70b.el9.noarch 2024-09-20T21:56:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-20T21:56:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-20T21:56:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-20T21:56:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-20T21:56:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-20T21:56:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-20T21:56:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-20T21:56:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-20T21:56:04.478 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:04.478 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:56:04.747 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-20T21:56:05.353 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:06 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:05.357 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-367.g9e2ac70b.el9.noar 12/12 2024-09-20T21:56:05.357 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-367.g9e2ac70b.el9 1/12 2024-09-20T21:56:05.358 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-367.g9e2ac70b.el9.noar 2/12 2024-09-20T21:56:05.358 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-367.g9e2ac70b.el9. 3/12 2024-09-20T21:56:05.358 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-09-20T21:56:05.358 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-09-20T21:56:05.358 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-09-20T21:56:05.358 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-09-20T21:56:05.358 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-09-20T21:56:05.358 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-09-20T21:56:05.358 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-09-20T21:56:05.358 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-09-20T21:56:05.492 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:56:05.493 INFO:teuthology.orchestra.run.smithi060.stdout:====================================================================================== 2024-09-20T21:56:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-09-20T21:56:05.493 INFO:teuthology.orchestra.run.smithi060.stdout:====================================================================================== 2024-09-20T21:56:05.493 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-20T21:56:05.493 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-367.g9e2ac70b.el9 ceph-noarch 7.4 M 2024-09-20T21:56:05.494 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-09-20T21:56:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-20T21:56:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-20T21:56:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-20T21:56:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-20T21:56:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-20T21:56:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-20T21:56:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-20T21:56:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-20T21:56:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-20T21:56:05.494 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-20T21:56:05.495 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-20T21:56:05.495 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:05.495 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:56:05.495 INFO:teuthology.orchestra.run.smithi060.stdout:====================================================================================== 2024-09-20T21:56:05.495 INFO:teuthology.orchestra.run.smithi060.stdout:Install 12 Packages 2024-09-20T21:56:05.495 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:05.495 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 38 M 2024-09-20T21:56:05.495 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 212 M 2024-09-20T21:56:05.496 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-20T21:56:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-09-20T21:56:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:05.679 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-20T21:56:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-20T21:56:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-grafana-dashboards-2:19.1.1-367.g9e2ac70b.el9.noarch 2024-09-20T21:56:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-dashboard-2:19.1.1-367.g9e2ac70b.el9.noarch 2024-09-20T21:56:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-prometheus-alerts-2:19.1.1-367.g9e2ac70b.el9.noarch 2024-09-20T21:56:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-20T21:56:05.679 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-20T21:56:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-20T21:56:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-20T21:56:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-20T21:56:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-20T21:56:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-20T21:56:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-20T21:56:05.680 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:05.680 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:56:05.870 INFO:teuthology.orchestra.run.smithi060.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-20T21:56:05.908 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-20T21:56:05.945 INFO:teuthology.orchestra.run.smithi060.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 3.2 MB/s | 801 kB 00:00 2024-09-20T21:56:05.970 INFO:teuthology.orchestra.run.smithi060.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 326 kB/s | 33 kB 00:00 2024-09-20T21:56:06.020 INFO:teuthology.orchestra.run.smithi060.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 365 kB/s | 18 kB 00:00 2024-09-20T21:56:06.071 INFO:teuthology.orchestra.run.smithi060.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 770 kB/s | 38 kB 00:00 2024-09-20T21:56:06.196 INFO:teuthology.orchestra.run.smithi060.stdout:(6/12): ceph-mgr-diskprediction-local-19.1.1-36 15 MB/s | 7.4 MB 00:00 2024-09-20T21:56:06.272 INFO:teuthology.orchestra.run.smithi060.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.2 MB/s | 3.0 MB 00:00 2024-09-20T21:56:06.389 INFO:teuthology.orchestra.run.smithi060.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 16 MB/s | 4.9 MB 00:00 2024-09-20T21:56:06.414 INFO:teuthology.orchestra.run.smithi060.stdout:(9/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.1 MB/s | 245 kB 00:00 2024-09-20T21:56:06.448 INFO:teuthology.orchestra.run.smithi060.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 4.2 MB/s | 250 kB 00:00 2024-09-20T21:56:06.513 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:07 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:06.523 INFO:teuthology.orchestra.run.smithi060.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-09-20T21:56:06.653 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:56:06.654 INFO:teuthology.orchestra.run.smithi067.stdout:====================================================================================== 2024-09-20T21:56:06.654 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-09-20T21:56:06.655 INFO:teuthology.orchestra.run.smithi067.stdout:====================================================================================== 2024-09-20T21:56:06.655 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-20T21:56:06.655 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-367.g9e2ac70b.el9 ceph-noarch 7.4 M 2024-09-20T21:56:06.655 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-09-20T21:56:06.655 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-20T21:56:06.655 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-20T21:56:06.655 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-20T21:56:06.655 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-20T21:56:06.655 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-20T21:56:06.655 INFO:teuthology.orchestra.run.smithi067.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-20T21:56:06.655 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-20T21:56:06.655 INFO:teuthology.orchestra.run.smithi067.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-20T21:56:06.655 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-20T21:56:06.656 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-20T21:56:06.656 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-20T21:56:06.656 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:06.656 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:56:06.656 INFO:teuthology.orchestra.run.smithi067.stdout:====================================================================================== 2024-09-20T21:56:06.656 INFO:teuthology.orchestra.run.smithi067.stdout:Install 12 Packages 2024-09-20T21:56:06.656 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:06.656 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 38 M 2024-09-20T21:56:06.656 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 212 M 2024-09-20T21:56:06.656 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-20T21:56:06.928 INFO:teuthology.orchestra.run.smithi067.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.5 MB/s | 191 kB 00:00 2024-09-20T21:56:06.994 INFO:teuthology.orchestra.run.smithi067.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.1 MB/s | 801 kB 00:00 2024-09-20T21:56:07.015 INFO:teuthology.orchestra.run.smithi060.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 27 MB/s | 16 MB 00:00 2024-09-20T21:56:07.016 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:07.016 INFO:teuthology.orchestra.run.smithi060.stdout:Total 25 MB/s | 38 MB 00:01 2024-09-20T21:56:07.019 INFO:teuthology.orchestra.run.smithi067.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 356 kB/s | 33 kB 00:00 2024-09-20T21:56:07.061 INFO:teuthology.orchestra.run.smithi067.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 436 kB/s | 18 kB 00:00 2024-09-20T21:56:07.112 INFO:teuthology.orchestra.run.smithi067.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-20T21:56:07.262 INFO:teuthology.orchestra.run.smithi067.stdout:(6/12): ceph-mgr-diskprediction-local-19.1.1-36 16 MB/s | 7.4 MB 00:00 2024-09-20T21:56:07.321 INFO:teuthology.orchestra.run.smithi067.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.2 MB/s | 3.0 MB 00:00 2024-09-20T21:56:07.390 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:56:07.417 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:56:07.417 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:56:07.438 INFO:teuthology.orchestra.run.smithi067.stdout:(8/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 15 MB/s | 4.9 MB 00:00 2024-09-20T21:56:07.513 INFO:teuthology.orchestra.run.smithi067.stdout:(9/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 27 MB/s | 5.1 MB 00:00 2024-09-20T21:56:07.546 INFO:teuthology.orchestra.run.smithi067.stdout:(10/12): python3-devel-3.9.19-8.el9.x86_64.rpm 862 kB/s | 245 kB 00:00 2024-09-20T21:56:07.582 INFO:teuthology.orchestra.run.smithi067.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.7 MB/s | 250 kB 00:00 2024-09-20T21:56:07.867 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:56:07.868 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:56:07.897 INFO:teuthology.orchestra.run.smithi067.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 42 MB/s | 16 MB 00:00 2024-09-20T21:56:07.898 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:07.898 INFO:teuthology.orchestra.run.smithi067.stdout:Total 31 MB/s | 38 MB 00:01 2024-09-20T21:56:08.265 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:56:08.292 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:56:08.292 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:56:08.656 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:56:08.730 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-20T21:56:08.748 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:56:08.749 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:56:08.782 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-20T21:56:08.887 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-20T21:56:08.931 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-20T21:56:09.122 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-20T21:56:09.154 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-20T21:56:09.242 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-20T21:56:09.537 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:56:09.602 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-20T21:56:09.657 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-20T21:56:09.767 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-20T21:56:09.814 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-20T21:56:09.947 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-20T21:56:10.005 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-20T21:56:10.037 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-20T21:56:10.081 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-20T21:56:10.115 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-20T21:56:10.807 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-20T21:56:10.929 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-20T21:56:11.425 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-20T21:56:12.250 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-20T21:56:12.514 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-20T21:56:12.537 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-367.g9e2ac7 12/12 2024-09-20T21:56:13.330 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-20T21:56:13.360 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-367.g9e2ac7 12/12 2024-09-20T21:56:15.225 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-367.g9e2ac7 12/12 2024-09-20T21:56:15.225 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-367.g9e2ac7 1/12 2024-09-20T21:56:15.225 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-20T21:56:15.225 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-20T21:56:15.225 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-20T21:56:15.225 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-20T21:56:15.225 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-20T21:56:15.225 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-20T21:56:15.225 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-20T21:56:15.225 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-20T21:56:15.225 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-20T21:56:15.226 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-20T21:56:15.624 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-20T21:56:15.624 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:15.624 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-20T21:56:15.624 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-diskprediction-local-2:19.1.1-367.g9e2ac70b.el9.noarch 2024-09-20T21:56:15.625 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-20T21:56:15.625 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-20T21:56:15.625 INFO:teuthology.orchestra.run.smithi060.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-20T21:56:15.625 INFO:teuthology.orchestra.run.smithi060.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-20T21:56:15.625 INFO:teuthology.orchestra.run.smithi060.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-20T21:56:15.625 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-20T21:56:15.625 INFO:teuthology.orchestra.run.smithi060.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-20T21:56:15.625 INFO:teuthology.orchestra.run.smithi060.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-20T21:56:15.625 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-20T21:56:15.625 INFO:teuthology.orchestra.run.smithi060.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-20T21:56:15.625 INFO:teuthology.orchestra.run.smithi060.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-20T21:56:15.625 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:15.626 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:56:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-367.g9e2ac7 12/12 2024-09-20T21:56:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-367.g9e2ac7 1/12 2024-09-20T21:56:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-20T21:56:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-20T21:56:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-20T21:56:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-20T21:56:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-20T21:56:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-20T21:56:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-20T21:56:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-20T21:56:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-20T21:56:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-20T21:56:16.070 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-mgr-rook 2024-09-20T21:56:16.364 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-20T21:56:16.364 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:16.364 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-20T21:56:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-diskprediction-local-2:19.1.1-367.g9e2ac70b.el9.noarch 2024-09-20T21:56:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-20T21:56:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-20T21:56:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-20T21:56:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-20T21:56:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-20T21:56:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-20T21:56:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-20T21:56:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-20T21:56:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-20T21:56:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-20T21:56:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-20T21:56:16.365 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:16.365 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:56:16.677 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:17 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:16.773 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-mgr-rook 2024-09-20T21:56:16.816 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:56:16.817 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:16.817 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-09-20T21:56:16.817 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:16.817 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-20T21:56:16.818 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook noarch 2:19.1.1-367.g9e2ac70b.el9 ceph-noarch 49 k 2024-09-20T21:56:16.818 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-09-20T21:56:16.818 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-20T21:56:16.818 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-20T21:56:16.818 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-20T21:56:16.818 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-20T21:56:16.818 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-20T21:56:16.818 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-20T21:56:16.818 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-20T21:56:16.818 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-20T21:56:16.818 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-20T21:56:16.818 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-20T21:56:16.818 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-20T21:56:16.818 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-20T21:56:16.819 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:16.819 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:56:16.819 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:16.819 INFO:teuthology.orchestra.run.smithi060.stdout:Install 13 Packages 2024-09-20T21:56:16.819 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:16.819 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 2.2 M 2024-09-20T21:56:16.819 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 26 M 2024-09-20T21:56:16.819 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-20T21:56:17.071 INFO:teuthology.orchestra.run.smithi060.stdout:(1/13): ceph-mgr-rook-19.1.1-367.g9e2ac70b.el9. 291 kB/s | 49 kB 00:00 2024-09-20T21:56:17.382 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:18 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:17.412 INFO:teuthology.orchestra.run.smithi060.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 37 kB/s | 19 kB 00:00 2024-09-20T21:56:17.471 INFO:teuthology.orchestra.run.smithi060.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 45 kB/s | 26 kB 00:00 2024-09-20T21:56:17.521 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:56:17.522 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:17.523 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-09-20T21:56:17.523 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:17.523 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-20T21:56:17.523 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook noarch 2:19.1.1-367.g9e2ac70b.el9 ceph-noarch 49 k 2024-09-20T21:56:17.523 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-09-20T21:56:17.523 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-20T21:56:17.523 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-20T21:56:17.523 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-20T21:56:17.523 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-20T21:56:17.523 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-20T21:56:17.523 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-20T21:56:17.524 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-20T21:56:17.524 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-20T21:56:17.524 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-20T21:56:17.524 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-20T21:56:17.524 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-20T21:56:17.524 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-20T21:56:17.524 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:17.524 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:56:17.524 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:17.524 INFO:teuthology.orchestra.run.smithi067.stdout:Install 13 Packages 2024-09-20T21:56:17.524 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:17.525 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 2.2 M 2024-09-20T21:56:17.525 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 26 M 2024-09-20T21:56:17.525 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-20T21:56:17.805 INFO:teuthology.orchestra.run.smithi060.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 303 kB/s | 222 kB 00:00 2024-09-20T21:56:18.172 INFO:teuthology.orchestra.run.smithi060.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 209 kB/s | 159 kB 00:00 2024-09-20T21:56:18.214 INFO:teuthology.orchestra.run.smithi060.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 780 kB/s | 32 kB 00:00 2024-09-20T21:56:18.239 INFO:teuthology.orchestra.run.smithi060.stdout:(7/13): python3-certifi-2023.05.07-4.el9.noarch 572 kB/s | 14 kB 00:00 2024-09-20T21:56:18.261 INFO:teuthology.orchestra.run.smithi067.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 228 kB/s | 26 kB 00:00 2024-09-20T21:56:18.264 INFO:teuthology.orchestra.run.smithi060.stdout:(8/13): python3-google-auth-2.34.0-1.el9.noarch 8.6 MB/s | 220 kB 00:00 2024-09-20T21:56:18.286 INFO:teuthology.orchestra.run.smithi067.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 139 kB/s | 19 kB 00:00 2024-09-20T21:56:18.306 INFO:teuthology.orchestra.run.smithi060.stdout:(9/13): python3-kubernetes-26.1.0-2.el9.noarch. 25 MB/s | 1.0 MB 00:00 2024-09-20T21:56:18.311 INFO:teuthology.orchestra.run.smithi067.stdout:(3/13): ceph-mgr-rook-19.1.1-367.g9e2ac70b.el9. 303 kB/s | 49 kB 00:00 2024-09-20T21:56:18.331 INFO:teuthology.orchestra.run.smithi060.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-20T21:56:18.357 INFO:teuthology.orchestra.run.smithi060.stdout:(11/13): python3-pyasn1-modules-0.4.8-6.el9.noa 315 kB/s | 279 kB 00:00 2024-09-20T21:56:18.382 INFO:teuthology.orchestra.run.smithi060.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-20T21:56:18.395 INFO:teuthology.orchestra.run.smithi067.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-09-20T21:56:18.415 INFO:teuthology.orchestra.run.smithi060.stdout:(13/13): python3-requests-oauthlib-1.3.0-12.el9 88 kB/s | 54 kB 00:00 2024-09-20T21:56:18.418 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:18.418 INFO:teuthology.orchestra.run.smithi060.stdout:Total 1.4 MB/s | 2.2 MB 00:01 2024-09-20T21:56:18.420 INFO:teuthology.orchestra.run.smithi067.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.6 MB/s | 222 kB 00:00 2024-09-20T21:56:18.445 INFO:teuthology.orchestra.run.smithi067.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-20T21:56:18.471 INFO:teuthology.orchestra.run.smithi067.stdout:(7/13): python3-cachetools-4.2.4-1.el9.noarch.r 642 kB/s | 32 kB 00:00 2024-09-20T21:56:18.487 INFO:teuthology.orchestra.run.smithi067.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 353 kB/s | 14 kB 00:00 2024-09-20T21:56:18.490 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:56:18.508 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:56:18.508 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:56:18.513 INFO:teuthology.orchestra.run.smithi067.stdout:(9/13): python3-google-auth-2.34.0-1.el9.noarch 5.4 MB/s | 220 kB 00:00 2024-09-20T21:56:18.538 INFO:teuthology.orchestra.run.smithi067.stdout:(10/13): python3-pyasn1-modules-0.4.8-6.el9.noa 1.2 MB/s | 279 kB 00:00 2024-09-20T21:56:18.555 INFO:teuthology.orchestra.run.smithi067.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-20T21:56:18.605 INFO:teuthology.orchestra.run.smithi067.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 8.7 MB/s | 1.0 MB 00:00 2024-09-20T21:56:18.608 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:56:18.610 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:56:18.630 INFO:teuthology.orchestra.run.smithi067.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 977 kB/s | 90 kB 00:00 2024-09-20T21:56:18.633 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:18.633 INFO:teuthology.orchestra.run.smithi067.stdout:Total 2.0 MB/s | 2.2 MB 00:01 2024-09-20T21:56:18.703 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:56:18.720 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:56:18.720 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:56:18.821 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:56:18.821 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:56:19.127 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:56:19.305 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-20T21:56:19.334 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:56:19.368 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-20T21:56:19.433 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-20T21:56:19.468 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-20T21:56:19.510 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-20T21:56:19.512 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-20T21:56:19.634 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-20T21:56:19.652 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-20T21:56:19.698 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-20T21:56:19.732 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-20T21:56:19.759 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-20T21:56:19.775 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-20T21:56:19.813 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-20T21:56:19.882 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-20T21:56:19.994 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-20T21:56:20.044 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-20T21:56:20.581 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-20T21:56:20.642 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-20T21:56:20.685 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-20T21:56:20.733 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-20T21:56:20.762 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-rook-2:19.1.1-367.g9e2ac70b.el9.noarch 13/13 2024-09-20T21:56:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-20T21:56:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-20T21:56:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-20T21:56:20.939 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-20T21:56:20.968 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-rook-2:19.1.1-367.g9e2ac70b.el9.noarch 13/13 2024-09-20T21:56:21.506 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-367.g9e2ac70b.el9.noarch 13/13 2024-09-20T21:56:21.506 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-rook-2:19.1.1-367.g9e2ac70b.el9.noarch 1/13 2024-09-20T21:56:21.506 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-20T21:56:21.506 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-20T21:56:21.506 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-20T21:56:21.506 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-20T21:56:21.506 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-20T21:56:21.506 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-20T21:56:21.506 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-20T21:56:21.506 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-20T21:56:21.506 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-20T21:56:21.507 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-20T21:56:21.507 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-20T21:56:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-367.g9e2ac70b.el9.noarch 13/13 2024-09-20T21:56:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-rook-2:19.1.1-367.g9e2ac70b.el9.noarch 1/13 2024-09-20T21:56:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-20T21:56:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-20T21:56:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-20T21:56:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-20T21:56:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-20T21:56:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-20T21:56:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-20T21:56:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-20T21:56:21.629 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-20T21:56:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-20T21:56:21.630 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-20T21:56:21.857 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-20T21:56:21.857 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:21.857 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-20T21:56:21.857 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-rook-2:19.1.1-367.g9e2ac70b.el9.noarch 2024-09-20T21:56:21.857 INFO:teuthology.orchestra.run.smithi060.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-20T21:56:21.857 INFO:teuthology.orchestra.run.smithi060.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-20T21:56:21.857 INFO:teuthology.orchestra.run.smithi060.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-20T21:56:21.857 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-20T21:56:21.858 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-20T21:56:21.858 INFO:teuthology.orchestra.run.smithi060.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-20T21:56:21.858 INFO:teuthology.orchestra.run.smithi060.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-20T21:56:21.858 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-20T21:56:21.858 INFO:teuthology.orchestra.run.smithi060.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-20T21:56:21.858 INFO:teuthology.orchestra.run.smithi060.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-20T21:56:21.858 INFO:teuthology.orchestra.run.smithi060.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-20T21:56:21.858 INFO:teuthology.orchestra.run.smithi060.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-20T21:56:21.858 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:21.858 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:56:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-20T21:56:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:21.969 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-20T21:56:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-rook-2:19.1.1-367.g9e2ac70b.el9.noarch 2024-09-20T21:56:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-20T21:56:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-20T21:56:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-20T21:56:21.969 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-20T21:56:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-20T21:56:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-20T21:56:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-20T21:56:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-20T21:56:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-20T21:56:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-20T21:56:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-20T21:56:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-20T21:56:21.970 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:21.970 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:56:22.122 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-mgr-cephadm 2024-09-20T21:56:22.258 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-mgr-cephadm 2024-09-20T21:56:22.727 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:23 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:22.866 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:23 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:22.869 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:56:22.870 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:22.870 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-09-20T21:56:22.870 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:22.871 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-20T21:56:22.871 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm noarch 2:19.1.1-367.g9e2ac70b.el9 ceph-noarch 146 k 2024-09-20T21:56:22.871 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-09-20T21:56:22.871 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-20T21:56:22.871 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-20T21:56:22.871 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-20T21:56:22.871 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-20T21:56:22.871 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-20T21:56:22.871 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:22.871 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:56:22.871 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:22.871 INFO:teuthology.orchestra.run.smithi060.stdout:Install 6 Packages 2024-09-20T21:56:22.871 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:22.872 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 7.0 M 2024-09-20T21:56:22.872 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 33 M 2024-09-20T21:56:22.872 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-20T21:56:23.008 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:56:23.009 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:23.009 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-09-20T21:56:23.009 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:23.009 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-20T21:56:23.009 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm noarch 2:19.1.1-367.g9e2ac70b.el9 ceph-noarch 146 k 2024-09-20T21:56:23.009 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-09-20T21:56:23.009 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-20T21:56:23.009 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-20T21:56:23.009 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-20T21:56:23.009 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-20T21:56:23.010 INFO:teuthology.orchestra.run.smithi067.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-20T21:56:23.010 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:23.010 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:56:23.010 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:23.010 INFO:teuthology.orchestra.run.smithi067.stdout:Install 6 Packages 2024-09-20T21:56:23.010 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:23.010 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 7.0 M 2024-09-20T21:56:23.011 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 33 M 2024-09-20T21:56:23.011 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-20T21:56:23.268 INFO:teuthology.orchestra.run.smithi060.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.1 MB/s | 248 kB 00:00 2024-09-20T21:56:23.290 INFO:teuthology.orchestra.run.smithi067.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-20T21:56:23.293 INFO:teuthology.orchestra.run.smithi060.stdout:(2/6): ceph-mgr-cephadm-19.1.1-367.g9e2ac70b.el 615 kB/s | 146 kB 00:00 2024-09-20T21:56:23.315 INFO:teuthology.orchestra.run.smithi067.stdout:(2/6): ceph-mgr-cephadm-19.1.1-367.g9e2ac70b.el 653 kB/s | 146 kB 00:00 2024-09-20T21:56:23.346 INFO:teuthology.orchestra.run.smithi060.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-20T21:56:23.366 INFO:teuthology.orchestra.run.smithi067.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-09-20T21:56:23.369 INFO:teuthology.orchestra.run.smithi060.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 766 kB/s | 58 kB 00:00 2024-09-20T21:56:23.391 INFO:teuthology.orchestra.run.smithi067.stdout:(4/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 5.3 MB/s | 548 kB 00:00 2024-09-20T21:56:23.394 INFO:teuthology.orchestra.run.smithi060.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-20T21:56:23.416 INFO:teuthology.orchestra.run.smithi067.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-20T21:56:23.552 INFO:teuthology.orchestra.run.smithi060.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 12 MB/s | 6.0 MB 00:00 2024-09-20T21:56:23.555 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:23.555 INFO:teuthology.orchestra.run.smithi060.stdout:Total 10 MB/s | 7.0 MB 00:00 2024-09-20T21:56:23.627 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:56:23.639 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:56:23.639 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:56:23.745 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:56:23.745 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:56:24.058 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:56:24.248 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-20T21:56:24.306 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-20T21:56:24.500 INFO:teuthology.orchestra.run.smithi067.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.2 MB/s | 6.0 MB 00:01 2024-09-20T21:56:24.503 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:24.503 INFO:teuthology.orchestra.run.smithi067.stdout:Total 4.7 MB/s | 7.0 MB 00:01 2024-09-20T21:56:24.580 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:56:24.593 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:56:24.593 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:56:24.700 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:56:24.700 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:56:24.896 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-20T21:56:24.987 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-20T21:56:25.025 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:56:25.059 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-20T21:56:25.083 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-367.g9e2ac70b.el9.noarch 6/6 2024-09-20T21:56:25.191 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-20T21:56:25.245 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-20T21:56:25.797 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-20T21:56:25.880 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-367.g9e2ac70b.el9.noarch 6/6 2024-09-20T21:56:25.881 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-367.g9e2ac70b.el9.noarch 1/6 2024-09-20T21:56:25.881 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-20T21:56:25.881 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-20T21:56:25.881 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-20T21:56:25.881 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-20T21:56:25.891 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-20T21:56:25.963 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-20T21:56:25.997 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-367.g9e2ac70b.el9.noarch 6/6 2024-09-20T21:56:26.164 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-20T21:56:26.164 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:26.164 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-20T21:56:26.164 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-mgr-cephadm-2:19.1.1-367.g9e2ac70b.el9.noarch 2024-09-20T21:56:26.164 INFO:teuthology.orchestra.run.smithi060.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-20T21:56:26.165 INFO:teuthology.orchestra.run.smithi060.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-20T21:56:26.165 INFO:teuthology.orchestra.run.smithi060.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-20T21:56:26.165 INFO:teuthology.orchestra.run.smithi060.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-20T21:56:26.165 INFO:teuthology.orchestra.run.smithi060.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-20T21:56:26.165 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:26.165 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:56:26.418 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-fuse 2024-09-20T21:56:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-367.g9e2ac70b.el9.noarch 6/6 2024-09-20T21:56:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-367.g9e2ac70b.el9.noarch 1/6 2024-09-20T21:56:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-20T21:56:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-20T21:56:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-20T21:56:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-20T21:56:27.026 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:28 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:27.087 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-20T21:56:27.087 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:27.087 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-20T21:56:27.087 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-mgr-cephadm-2:19.1.1-367.g9e2ac70b.el9.noarch 2024-09-20T21:56:27.087 INFO:teuthology.orchestra.run.smithi067.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-20T21:56:27.087 INFO:teuthology.orchestra.run.smithi067.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-20T21:56:27.087 INFO:teuthology.orchestra.run.smithi067.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-20T21:56:27.088 INFO:teuthology.orchestra.run.smithi067.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-20T21:56:27.088 INFO:teuthology.orchestra.run.smithi067.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-20T21:56:27.088 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:27.088 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:56:27.167 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:56:27.167 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:27.167 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-09-20T21:56:27.167 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:27.168 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-20T21:56:27.168 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 896 k 2024-09-20T21:56:27.168 INFO:teuthology.orchestra.run.smithi060.stdout:Installing dependencies: 2024-09-20T21:56:27.168 INFO:teuthology.orchestra.run.smithi060.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-20T21:56:27.168 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:27.168 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:56:27.168 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:27.168 INFO:teuthology.orchestra.run.smithi060.stdout:Install 2 Packages 2024-09-20T21:56:27.168 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:27.169 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 976 k 2024-09-20T21:56:27.169 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 2.9 M 2024-09-20T21:56:27.169 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-20T21:56:27.366 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-fuse 2024-09-20T21:56:27.422 INFO:teuthology.orchestra.run.smithi060.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 635 kB/s | 80 kB 00:00 2024-09-20T21:56:27.588 INFO:teuthology.orchestra.run.smithi060.stdout:(2/2): ceph-fuse-19.1.1-367.g9e2ac70b.el9.x86_6 3.0 MB/s | 896 kB 00:00 2024-09-20T21:56:27.589 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:27.589 INFO:teuthology.orchestra.run.smithi060.stdout:Total 2.3 MB/s | 976 kB 00:00 2024-09-20T21:56:27.601 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:56:27.624 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:56:27.624 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:56:27.711 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:56:27.711 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:56:27.917 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:56:27.978 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:28 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:28.053 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-20T21:56:28.120 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:56:28.121 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:28.121 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-09-20T21:56:28.121 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:28.122 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-20T21:56:28.122 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 896 k 2024-09-20T21:56:28.122 INFO:teuthology.orchestra.run.smithi067.stdout:Installing dependencies: 2024-09-20T21:56:28.122 INFO:teuthology.orchestra.run.smithi067.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-20T21:56:28.122 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:28.122 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:56:28.122 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:28.123 INFO:teuthology.orchestra.run.smithi067.stdout:Install 2 Packages 2024-09-20T21:56:28.123 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:28.123 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 976 k 2024-09-20T21:56:28.123 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 2.9 M 2024-09-20T21:56:28.123 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-20T21:56:28.132 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-fuse-2:19.1.1-367.g9e2ac70b.el9.x86_64 2/2 2024-09-20T21:56:28.327 INFO:teuthology.orchestra.run.smithi067.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 434 kB/s | 80 kB 00:00 2024-09-20T21:56:28.419 INFO:teuthology.orchestra.run.smithi067.stdout:(2/2): ceph-fuse-19.1.1-367.g9e2ac70b.el9.x86_6 3.2 MB/s | 896 kB 00:00 2024-09-20T21:56:28.420 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:28.420 INFO:teuthology.orchestra.run.smithi067.stdout:Total 3.2 MB/s | 976 kB 00:00 2024-09-20T21:56:28.432 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:56:28.455 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:56:28.455 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:56:28.542 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:56:28.542 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:56:28.743 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:56:28.851 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-fuse-2:19.1.1-367.g9e2ac70b.el9.x86_64 2/2 2024-09-20T21:56:28.851 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-fuse-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/2 2024-09-20T21:56:28.883 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-20T21:56:28.963 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-fuse-2:19.1.1-367.g9e2ac70b.el9.x86_64 2/2 2024-09-20T21:56:29.139 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-20T21:56:29.139 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:29.139 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-20T21:56:29.139 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-fuse-2:19.1.1-367.g9e2ac70b.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-20T21:56:29.139 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:29.139 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:56:29.421 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install ceph-volume 2024-09-20T21:56:29.639 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-fuse-2:19.1.1-367.g9e2ac70b.el9.x86_64 2/2 2024-09-20T21:56:29.639 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-fuse-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/2 2024-09-20T21:56:29.910 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-20T21:56:29.910 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:29.910 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-20T21:56:29.911 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-fuse-2:19.1.1-367.g9e2ac70b.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-20T21:56:29.911 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:29.911 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:56:30.032 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:31 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:30.174 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:56:30.174 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:30.174 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repository Size 2024-09-20T21:56:30.175 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:30.175 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-20T21:56:30.175 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume noarch 2:19.1.1-367.g9e2ac70b.el9 ceph-noarch 285 k 2024-09-20T21:56:30.175 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:30.175 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:56:30.175 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:30.175 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2024-09-20T21:56:30.175 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:30.175 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 285 k 2024-09-20T21:56:30.175 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 1.3 M 2024-09-20T21:56:30.175 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-20T21:56:30.184 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install ceph-volume 2024-09-20T21:56:30.402 INFO:teuthology.orchestra.run.smithi060.stdout:ceph-volume-19.1.1-367.g9e2ac70b.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-20T21:56:30.402 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:30.402 INFO:teuthology.orchestra.run.smithi060.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-20T21:56:30.402 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:56:30.412 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:56:30.412 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:56:30.454 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:56:30.454 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:56:30.693 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:56:30.770 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : ceph-volume-2:19.1.1-367.g9e2ac70b.el9.noarch 1/1 2024-09-20T21:56:30.786 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:31 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:30.788 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: ceph-volume-2:19.1.1-367.g9e2ac70b.el9.noarch 1/1 2024-09-20T21:56:30.788 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T21:56:30.788 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-20T21:56:30.788 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:30.927 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:56:30.928 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:30.928 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repository Size 2024-09-20T21:56:30.928 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:30.928 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-20T21:56:30.928 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume noarch 2:19.1.1-367.g9e2ac70b.el9 ceph-noarch 285 k 2024-09-20T21:56:30.928 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:30.928 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:56:30.928 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:30.928 INFO:teuthology.orchestra.run.smithi067.stdout:Install 1 Package 2024-09-20T21:56:30.928 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:30.929 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 285 k 2024-09-20T21:56:30.929 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 1.3 M 2024-09-20T21:56:30.929 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-20T21:56:31.156 INFO:teuthology.orchestra.run.smithi067.stdout:ceph-volume-19.1.1-367.g9e2ac70b.el9.noarch.rpm 1.2 MB/s | 285 kB 00:00 2024-09-20T21:56:31.157 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:31.157 INFO:teuthology.orchestra.run.smithi067.stdout:Total 1.2 MB/s | 285 kB 00:00 2024-09-20T21:56:31.157 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:56:31.167 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:56:31.167 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:56:31.201 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:56:31.201 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:56:31.437 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:56:31.509 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : ceph-volume-2:19.1.1-367.g9e2ac70b.el9.noarch 1/1 2024-09-20T21:56:31.525 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: ceph-volume-2:19.1.1-367.g9e2ac70b.el9.noarch 1/1 2024-09-20T21:56:31.525 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T21:56:31.525 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-20T21:56:31.525 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:31.855 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : ceph-volume-2:19.1.1-367.g9e2ac70b.el9.noarch 1/1 2024-09-20T21:56:31.855 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:31.855 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-20T21:56:31.855 INFO:teuthology.orchestra.run.smithi060.stdout: ceph-volume-2:19.1.1-367.g9e2ac70b.el9.noarch 2024-09-20T21:56:31.855 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:31.855 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:56:32.192 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install librados-devel 2024-09-20T21:56:32.560 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : ceph-volume-2:19.1.1-367.g9e2ac70b.el9.noarch 1/1 2024-09-20T21:56:32.561 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:32.561 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-20T21:56:32.561 INFO:teuthology.orchestra.run.smithi067.stdout: ceph-volume-2:19.1.1-367.g9e2ac70b.el9.noarch 2024-09-20T21:56:32.561 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:32.561 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:56:32.808 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:33 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:32.858 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install librados-devel 2024-09-20T21:56:32.947 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:56:32.948 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:32.948 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repo Size 2024-09-20T21:56:32.948 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:32.948 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-20T21:56:32.948 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 127 k 2024-09-20T21:56:32.948 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:32.948 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:56:32.948 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:32.948 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2024-09-20T21:56:32.948 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:32.949 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 127 k 2024-09-20T21:56:32.949 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 456 k 2024-09-20T21:56:32.949 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-20T21:56:33.156 INFO:teuthology.orchestra.run.smithi060.stdout:librados-devel-19.1.1-367.g9e2ac70b.el9.x86_64. 613 kB/s | 127 kB 00:00 2024-09-20T21:56:33.157 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:33.157 INFO:teuthology.orchestra.run.smithi060.stdout:Total 607 kB/s | 127 kB 00:00 2024-09-20T21:56:33.157 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:56:33.163 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:56:33.163 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:56:33.232 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:56:33.232 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:56:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:56:33.473 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:34 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : librados-devel-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:33.614 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:56:33.615 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:33.615 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repo Size 2024-09-20T21:56:33.615 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:33.615 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-20T21:56:33.616 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 127 k 2024-09-20T21:56:33.616 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:33.616 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:56:33.616 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:33.616 INFO:teuthology.orchestra.run.smithi067.stdout:Install 1 Package 2024-09-20T21:56:33.616 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:33.616 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 127 k 2024-09-20T21:56:33.616 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 456 k 2024-09-20T21:56:33.616 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-20T21:56:33.828 INFO:teuthology.orchestra.run.smithi067.stdout:librados-devel-19.1.1-367.g9e2ac70b.el9.x86_64. 599 kB/s | 127 kB 00:00 2024-09-20T21:56:33.829 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:33.829 INFO:teuthology.orchestra.run.smithi067.stdout:Total 594 kB/s | 127 kB 00:00 2024-09-20T21:56:33.829 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:56:33.835 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:56:33.835 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:56:33.904 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:56:33.904 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:56:33.936 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: librados-devel-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:34.063 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:56:34.163 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : librados-devel-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:34.163 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : librados-devel-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:34.163 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:34.163 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-20T21:56:34.164 INFO:teuthology.orchestra.run.smithi060.stdout: librados-devel-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:56:34.164 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:34.164 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:56:34.450 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install libcephfs2 2024-09-20T21:56:34.588 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: librados-devel-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:34.850 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : librados-devel-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:34.850 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:34.850 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-20T21:56:34.850 INFO:teuthology.orchestra.run.smithi067.stdout: librados-devel-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:56:34.850 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:34.850 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:56:35.058 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:36 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:35.090 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install libcephfs2 2024-09-20T21:56:35.151 INFO:teuthology.orchestra.run.smithi060.stdout:Package libcephfs2-2:19.1.1-367.g9e2ac70b.el9.x86_64 is already installed. 2024-09-20T21:56:35.197 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:56:35.198 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-20T21:56:35.199 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:56:35.273 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install libcephfs-devel 2024-09-20T21:56:35.710 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:36 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:35.804 INFO:teuthology.orchestra.run.smithi067.stdout:Package libcephfs2-2:19.1.1-367.g9e2ac70b.el9.x86_64 is already installed. 2024-09-20T21:56:35.851 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:56:35.852 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2024-09-20T21:56:35.852 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:56:35.882 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:36 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:35.915 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install libcephfs-devel 2024-09-20T21:56:36.022 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:56:36.022 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:36.022 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repo Size 2024-09-20T21:56:36.022 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:36.022 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-20T21:56:36.022 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 32 k 2024-09-20T21:56:36.023 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:36.023 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:56:36.023 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:36.023 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2024-09-20T21:56:36.023 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:36.023 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 32 k 2024-09-20T21:56:36.023 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 142 k 2024-09-20T21:56:36.023 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-20T21:56:36.176 INFO:teuthology.orchestra.run.smithi060.stdout:libcephfs-devel-19.1.1-367.g9e2ac70b.el9.x86_64 207 kB/s | 32 kB 00:00 2024-09-20T21:56:36.177 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:36.177 INFO:teuthology.orchestra.run.smithi060.stdout:Total 206 kB/s | 32 kB 00:00 2024-09-20T21:56:36.177 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:56:36.181 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:56:36.181 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:56:36.197 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:56:36.197 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:56:36.291 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:56:36.413 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : libcephfs-devel-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:36.530 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:37 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:36.669 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:56:36.670 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:36.670 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repo Size 2024-09-20T21:56:36.670 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:36.670 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-20T21:56:36.670 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 32 k 2024-09-20T21:56:36.670 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:36.670 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:56:36.670 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:36.670 INFO:teuthology.orchestra.run.smithi067.stdout:Install 1 Package 2024-09-20T21:56:36.670 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:36.671 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 32 k 2024-09-20T21:56:36.671 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 142 k 2024-09-20T21:56:36.671 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-20T21:56:36.797 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:36.823 INFO:teuthology.orchestra.run.smithi067.stdout:libcephfs-devel-19.1.1-367.g9e2ac70b.el9.x86_64 208 kB/s | 32 kB 00:00 2024-09-20T21:56:36.824 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:36.824 INFO:teuthology.orchestra.run.smithi067.stdout:Total 206 kB/s | 32 kB 00:00 2024-09-20T21:56:36.824 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:56:36.828 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:56:36.828 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:56:36.848 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:56:36.849 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:56:36.948 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:56:37.055 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : libcephfs-devel-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:37.055 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:37.055 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-20T21:56:37.056 INFO:teuthology.orchestra.run.smithi060.stdout: libcephfs-devel-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:56:37.056 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:37.056 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:56:37.082 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : libcephfs-devel-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:37.244 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install librados2 2024-09-20T21:56:37.468 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: libcephfs-devel-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:37.707 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : libcephfs-devel-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:37.707 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:37.707 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-20T21:56:37.707 INFO:teuthology.orchestra.run.smithi067.stdout: libcephfs-devel-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:56:37.707 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:37.707 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:56:37.852 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:38 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:37.887 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install librados2 2024-09-20T21:56:37.944 INFO:teuthology.orchestra.run.smithi060.stdout:Package librados2-2:19.1.1-367.g9e2ac70b.el9.x86_64 is already installed. 2024-09-20T21:56:37.991 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:56:37.992 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-20T21:56:37.992 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:56:38.060 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install librbd1 2024-09-20T21:56:38.496 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:39 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:38.589 INFO:teuthology.orchestra.run.smithi067.stdout:Package librados2-2:19.1.1-367.g9e2ac70b.el9.x86_64 is already installed. 2024-09-20T21:56:38.636 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:56:38.637 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2024-09-20T21:56:38.638 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:56:38.669 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:39 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:38.693 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install librbd1 2024-09-20T21:56:38.762 INFO:teuthology.orchestra.run.smithi060.stdout:Package librbd1-2:19.1.1-367.g9e2ac70b.el9.x86_64 is already installed. 2024-09-20T21:56:38.808 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:56:38.809 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-20T21:56:38.809 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:56:38.878 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install python3-rados 2024-09-20T21:56:39.305 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:40 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:39.399 INFO:teuthology.orchestra.run.smithi067.stdout:Package librbd1-2:19.1.1-367.g9e2ac70b.el9.x86_64 is already installed. 2024-09-20T21:56:39.445 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:56:39.446 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2024-09-20T21:56:39.446 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:56:39.483 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:40 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:39.504 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install python3-rados 2024-09-20T21:56:39.578 INFO:teuthology.orchestra.run.smithi060.stdout:Package python3-rados-2:19.1.1-367.g9e2ac70b.el9.x86_64 is already installed. 2024-09-20T21:56:39.624 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:56:39.625 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-20T21:56:39.625 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:56:39.703 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install python3-rgw 2024-09-20T21:56:40.113 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:41 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:40.206 INFO:teuthology.orchestra.run.smithi067.stdout:Package python3-rados-2:19.1.1-367.g9e2ac70b.el9.x86_64 is already installed. 2024-09-20T21:56:40.252 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:56:40.253 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2024-09-20T21:56:40.254 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:56:40.311 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install python3-rgw 2024-09-20T21:56:40.311 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:41 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:40.405 INFO:teuthology.orchestra.run.smithi060.stdout:Package python3-rgw-2:19.1.1-367.g9e2ac70b.el9.x86_64 is already installed. 2024-09-20T21:56:40.451 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:56:40.452 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-20T21:56:40.452 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:56:40.577 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install python3-cephfs 2024-09-20T21:56:40.918 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:41 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:41.011 INFO:teuthology.orchestra.run.smithi067.stdout:Package python3-rgw-2:19.1.1-367.g9e2ac70b.el9.x86_64 is already installed. 2024-09-20T21:56:41.057 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:56:41.058 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2024-09-20T21:56:41.058 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:56:41.121 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install python3-cephfs 2024-09-20T21:56:41.186 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:42 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:41.280 INFO:teuthology.orchestra.run.smithi060.stdout:Package python3-cephfs-2:19.1.1-367.g9e2ac70b.el9.x86_64 is already installed. 2024-09-20T21:56:41.326 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:56:41.328 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-20T21:56:41.328 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:56:41.416 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install python3-rbd 2024-09-20T21:56:41.731 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:42 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:41.824 INFO:teuthology.orchestra.run.smithi067.stdout:Package python3-cephfs-2:19.1.1-367.g9e2ac70b.el9.x86_64 is already installed. 2024-09-20T21:56:41.871 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:56:41.872 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2024-09-20T21:56:41.872 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:56:41.935 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install python3-rbd 2024-09-20T21:56:42.025 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:43 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:42.118 INFO:teuthology.orchestra.run.smithi060.stdout:Package python3-rbd-2:19.1.1-367.g9e2ac70b.el9.x86_64 is already installed. 2024-09-20T21:56:42.164 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:56:42.165 INFO:teuthology.orchestra.run.smithi060.stdout:Nothing to do. 2024-09-20T21:56:42.165 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:56:42.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install rbd-fuse 2024-09-20T21:56:42.540 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:43 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:42.633 INFO:teuthology.orchestra.run.smithi067.stdout:Package python3-rbd-2:19.1.1-367.g9e2ac70b.el9.x86_64 is already installed. 2024-09-20T21:56:42.679 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:56:42.680 INFO:teuthology.orchestra.run.smithi067.stdout:Nothing to do. 2024-09-20T21:56:42.681 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:56:42.747 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install rbd-fuse 2024-09-20T21:56:42.993 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:43 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:43.136 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:56:43.136 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:43.137 INFO:teuthology.orchestra.run.smithi060.stdout: Package Architecture Version Repository Size 2024-09-20T21:56:43.137 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:43.137 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-20T21:56:43.137 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 85 k 2024-09-20T21:56:43.137 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:43.137 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:56:43.137 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:43.137 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2024-09-20T21:56:43.137 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:43.137 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 85 k 2024-09-20T21:56:43.137 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 231 k 2024-09-20T21:56:43.137 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-20T21:56:43.322 INFO:teuthology.orchestra.run.smithi060.stdout:rbd-fuse-19.1.1-367.g9e2ac70b.el9.x86_64.rpm 462 kB/s | 85 kB 00:00 2024-09-20T21:56:43.323 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:43.323 INFO:teuthology.orchestra.run.smithi060.stdout:Total 458 kB/s | 85 kB 00:00 2024-09-20T21:56:43.323 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:56:43.331 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:56:43.331 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:56:43.368 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:44 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:43.399 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:56:43.399 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:56:43.512 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:56:43.512 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:43.512 INFO:teuthology.orchestra.run.smithi067.stdout: Package Architecture Version Repository Size 2024-09-20T21:56:43.512 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:43.512 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-20T21:56:43.512 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 85 k 2024-09-20T21:56:43.512 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:43.512 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:56:43.513 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:43.513 INFO:teuthology.orchestra.run.smithi067.stdout:Install 1 Package 2024-09-20T21:56:43.513 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:43.513 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 85 k 2024-09-20T21:56:43.513 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 231 k 2024-09-20T21:56:43.513 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-20T21:56:43.550 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:56:43.648 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-fuse-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:43.719 INFO:teuthology.orchestra.run.smithi067.stdout:rbd-fuse-19.1.1-367.g9e2ac70b.el9.x86_64.rpm 416 kB/s | 85 kB 00:00 2024-09-20T21:56:43.719 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:43.720 INFO:teuthology.orchestra.run.smithi067.stdout:Total 412 kB/s | 85 kB 00:00 2024-09-20T21:56:43.720 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:56:43.727 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:56:43.728 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:56:43.819 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:56:43.819 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:56:44.048 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:56:44.175 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-fuse-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:44.209 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-fuse-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-fuse-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:44.411 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-20T21:56:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-fuse-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:56:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:44.411 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:56:44.604 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: rbd-fuse-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:44.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install rbd-mirror 2024-09-20T21:56:44.875 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-fuse-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:44.875 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:44.875 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-20T21:56:44.875 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-fuse-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:56:44.875 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:44.875 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:56:45.096 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install rbd-mirror 2024-09-20T21:56:45.296 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:46 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:45.435 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:56:45.435 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:45.435 INFO:teuthology.orchestra.run.smithi060.stdout: Package Arch Version Repo Size 2024-09-20T21:56:45.436 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:45.436 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-20T21:56:45.436 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 3.1 M 2024-09-20T21:56:45.436 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:45.436 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:56:45.436 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:45.436 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2024-09-20T21:56:45.436 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:45.437 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 3.1 M 2024-09-20T21:56:45.437 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 13 M 2024-09-20T21:56:45.437 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-20T21:56:45.708 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:46 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:45.783 INFO:teuthology.orchestra.run.smithi060.stdout:rbd-mirror-19.1.1-367.g9e2ac70b.el9.x86_64.rpm 9.0 MB/s | 3.1 MB 00:00 2024-09-20T21:56:45.784 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:45.784 INFO:teuthology.orchestra.run.smithi060.stdout:Total 8.9 MB/s | 3.1 MB 00:00 2024-09-20T21:56:45.784 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:56:45.794 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:56:45.794 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:56:45.851 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:56:45.851 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:45.851 INFO:teuthology.orchestra.run.smithi067.stdout: Package Arch Version Repo Size 2024-09-20T21:56:45.851 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:45.851 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-20T21:56:45.851 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 3.1 M 2024-09-20T21:56:45.851 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:45.852 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:56:45.852 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:45.852 INFO:teuthology.orchestra.run.smithi067.stdout:Install 1 Package 2024-09-20T21:56:45.852 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:45.852 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 3.1 M 2024-09-20T21:56:45.852 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 13 M 2024-09-20T21:56:45.852 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-20T21:56:45.880 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:56:45.880 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:56:46.188 INFO:teuthology.orchestra.run.smithi067.stdout:rbd-mirror-19.1.1-367.g9e2ac70b.el9.x86_64.rpm 9.2 MB/s | 3.1 MB 00:00 2024-09-20T21:56:46.188 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:46.189 INFO:teuthology.orchestra.run.smithi067.stdout:Total 9.2 MB/s | 3.1 MB 00:00 2024-09-20T21:56:46.189 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:56:46.199 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:56:46.199 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:56:46.286 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:56:46.286 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:56:46.468 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:56:46.542 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-mirror-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:46.569 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-mirror-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:46.569 INFO:teuthology.orchestra.run.smithi060.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T21:56:46.569 INFO:teuthology.orchestra.run.smithi060.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-20T21:56:46.569 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-20T21:56:46.569 INFO:teuthology.orchestra.run.smithi060.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-20T21:56:46.569 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:46.779 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:56:46.830 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-mirror-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:46.857 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: rbd-mirror-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:46.857 INFO:teuthology.orchestra.run.smithi067.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-20T21:56:46.858 INFO:teuthology.orchestra.run.smithi067.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-20T21:56:46.858 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-20T21:56:46.858 INFO:teuthology.orchestra.run.smithi067.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-20T21:56:46.858 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:47.669 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-mirror-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:47.669 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:47.669 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-20T21:56:47.669 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-mirror-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:56:47.669 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:47.669 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:56:47.940 DEBUG:teuthology.orchestra.run.smithi060:> sudo yum -y install rbd-nbd 2024-09-20T21:56:47.966 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-mirror-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:47.966 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:47.966 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-20T21:56:47.966 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-mirror-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:56:47.966 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:47.966 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:56:48.190 DEBUG:teuthology.orchestra.run.smithi067:> sudo yum -y install rbd-nbd 2024-09-20T21:56:48.551 INFO:teuthology.orchestra.run.smithi060.stdout:Last metadata expiration check: 0:01:49 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:48.689 INFO:teuthology.orchestra.run.smithi060.stdout:Dependencies resolved. 2024-09-20T21:56:48.690 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:48.690 INFO:teuthology.orchestra.run.smithi060.stdout: Package Architecture Version Repository Size 2024-09-20T21:56:48.691 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:48.691 INFO:teuthology.orchestra.run.smithi060.stdout:Installing: 2024-09-20T21:56:48.691 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 170 k 2024-09-20T21:56:48.691 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:48.691 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction Summary 2024-09-20T21:56:48.691 INFO:teuthology.orchestra.run.smithi060.stdout:================================================================================ 2024-09-20T21:56:48.691 INFO:teuthology.orchestra.run.smithi060.stdout:Install 1 Package 2024-09-20T21:56:48.691 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:48.691 INFO:teuthology.orchestra.run.smithi060.stdout:Total download size: 170 k 2024-09-20T21:56:48.691 INFO:teuthology.orchestra.run.smithi060.stdout:Installed size: 494 k 2024-09-20T21:56:48.691 INFO:teuthology.orchestra.run.smithi060.stdout:Downloading Packages: 2024-09-20T21:56:48.813 INFO:teuthology.orchestra.run.smithi067.stdout:Last metadata expiration check: 0:01:49 ago on Fri 20 Sep 2024 09:54:59 PM UTC. 2024-09-20T21:56:48.903 INFO:teuthology.orchestra.run.smithi060.stdout:rbd-nbd-19.1.1-367.g9e2ac70b.el9.x86_64.rpm 801 kB/s | 170 kB 00:00 2024-09-20T21:56:48.903 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:48.904 INFO:teuthology.orchestra.run.smithi060.stdout:Total 796 kB/s | 170 kB 00:00 2024-09-20T21:56:48.904 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction check 2024-09-20T21:56:48.912 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction check succeeded. 2024-09-20T21:56:48.912 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction test 2024-09-20T21:56:48.949 INFO:teuthology.orchestra.run.smithi067.stdout:Dependencies resolved. 2024-09-20T21:56:48.950 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:48.950 INFO:teuthology.orchestra.run.smithi067.stdout: Package Architecture Version Repository Size 2024-09-20T21:56:48.950 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:48.950 INFO:teuthology.orchestra.run.smithi067.stdout:Installing: 2024-09-20T21:56:48.950 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd x86_64 2:19.1.1-367.g9e2ac70b.el9 ceph 170 k 2024-09-20T21:56:48.950 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:48.950 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction Summary 2024-09-20T21:56:48.950 INFO:teuthology.orchestra.run.smithi067.stdout:================================================================================ 2024-09-20T21:56:48.950 INFO:teuthology.orchestra.run.smithi067.stdout:Install 1 Package 2024-09-20T21:56:48.950 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:48.951 INFO:teuthology.orchestra.run.smithi067.stdout:Total download size: 170 k 2024-09-20T21:56:48.951 INFO:teuthology.orchestra.run.smithi067.stdout:Installed size: 494 k 2024-09-20T21:56:48.951 INFO:teuthology.orchestra.run.smithi067.stdout:Downloading Packages: 2024-09-20T21:56:48.980 INFO:teuthology.orchestra.run.smithi060.stdout:Transaction test succeeded. 2024-09-20T21:56:48.981 INFO:teuthology.orchestra.run.smithi060.stdout:Running transaction 2024-09-20T21:56:49.146 INFO:teuthology.orchestra.run.smithi060.stdout: Preparing : 1/1 2024-09-20T21:56:49.166 INFO:teuthology.orchestra.run.smithi067.stdout:rbd-nbd-19.1.1-367.g9e2ac70b.el9.x86_64.rpm 794 kB/s | 170 kB 00:00 2024-09-20T21:56:49.166 INFO:teuthology.orchestra.run.smithi067.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:56:49.166 INFO:teuthology.orchestra.run.smithi067.stdout:Total 789 kB/s | 170 kB 00:00 2024-09-20T21:56:49.167 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction check 2024-09-20T21:56:49.175 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction check succeeded. 2024-09-20T21:56:49.175 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction test 2024-09-20T21:56:49.243 INFO:teuthology.orchestra.run.smithi067.stdout:Transaction test succeeded. 2024-09-20T21:56:49.244 INFO:teuthology.orchestra.run.smithi067.stdout:Running transaction 2024-09-20T21:56:49.255 INFO:teuthology.orchestra.run.smithi060.stdout: Installing : rbd-nbd-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:49.404 INFO:teuthology.orchestra.run.smithi067.stdout: Preparing : 1/1 2024-09-20T21:56:49.518 INFO:teuthology.orchestra.run.smithi067.stdout: Installing : rbd-nbd-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:49.673 INFO:teuthology.orchestra.run.smithi060.stdout: Running scriptlet: rbd-nbd-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:49.944 INFO:teuthology.orchestra.run.smithi060.stdout: Verifying : rbd-nbd-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:49.944 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:49.945 INFO:teuthology.orchestra.run.smithi060.stdout:Installed: 2024-09-20T21:56:49.945 INFO:teuthology.orchestra.run.smithi060.stdout: rbd-nbd-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:56:49.945 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:56:49.945 INFO:teuthology.orchestra.run.smithi060.stdout:Complete! 2024-09-20T21:56:49.959 INFO:teuthology.orchestra.run.smithi067.stdout: Running scriptlet: rbd-nbd-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:50.207 INFO:teuthology.orchestra.run.smithi067.stdout: Verifying : rbd-nbd-2:19.1.1-367.g9e2ac70b.el9.x86_64 1/1 2024-09-20T21:56:50.207 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:50.207 INFO:teuthology.orchestra.run.smithi067.stdout:Installed: 2024-09-20T21:56:50.208 INFO:teuthology.orchestra.run.smithi067.stdout: rbd-nbd-2:19.1.1-367.g9e2ac70b.el9.x86_64 2024-09-20T21:56:50.208 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:56:50.208 INFO:teuthology.orchestra.run.smithi067.stdout:Complete! 2024-09-20T21:56:50.266 DEBUG:teuthology.parallel:result is None 2024-09-20T21:56:50.445 DEBUG:teuthology.parallel:result is None 2024-09-20T21:56:50.446 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9e2ac70bb7aea67d34f0d50002344abbe7803736 2024-09-20T21:56:50.584 DEBUG:teuthology.orchestra.run.smithi060:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-20T21:56:50.608 INFO:teuthology.orchestra.run.smithi060.stdout:19.1.1-367.g9e2ac70b.el9 2024-09-20T21:56:50.609 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-367.g9e2ac70b.el9 2024-09-20T21:56:50.609 INFO:teuthology.task.install:The correct ceph version 19.1.1-367.g9e2ac70b is installed. 2024-09-20T21:56:50.611 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9e2ac70bb7aea67d34f0d50002344abbe7803736 2024-09-20T21:56:50.753 DEBUG:teuthology.orchestra.run.smithi067:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-20T21:56:50.777 INFO:teuthology.orchestra.run.smithi067.stdout:19.1.1-367.g9e2ac70b.el9 2024-09-20T21:56:50.777 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-367.g9e2ac70b.el9 2024-09-20T21:56:50.777 INFO:teuthology.task.install:The correct ceph version 19.1.1-367.g9e2ac70b is installed. 2024-09-20T21:56:50.779 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-20T21:56:50.779 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-20T21:56:50.779 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-20T21:56:50.808 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-20T21:56:50.808 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-20T21:56:50.846 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-20T21:56:50.846 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-20T21:56:50.846 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/daemon-helper 2024-09-20T21:56:50.876 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-20T21:56:50.946 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-20T21:56:50.946 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/daemon-helper 2024-09-20T21:56:50.973 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-20T21:56:51.040 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-20T21:56:51.040 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-20T21:56:51.040 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-20T21:56:51.067 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-20T21:56:51.134 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-20T21:56:51.134 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-20T21:56:51.168 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-20T21:56:51.235 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-20T21:56:51.235 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-20T21:56:51.235 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/usr/bin/stdin-killer 2024-09-20T21:56:51.262 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-20T21:56:51.329 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-20T21:56:51.329 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/usr/bin/stdin-killer 2024-09-20T21:56:51.356 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-20T21:56:51.422 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-20T21:56:51.516 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'CEPHADM_STRAY_DAEMON', 'CEPHADM_FAILED_DAEMON', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)'], 'log-only-match': ['CEPHADM_'], 'sha1': '9e2ac70bb7aea67d34f0d50002344abbe7803736', 'cephadm_mode': 'root'} 2024-09-20T21:56:51.516 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 2024-09-20T21:56:51.516 INFO:tasks.cephadm:Cluster fsid is 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:56:51.516 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-20T21:56:51.517 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.60', 'mon.c': '[v2:172.21.15.60:3301,v1:172.21.15.60:6790]', 'mon.b': '172.21.15.67'} 2024-09-20T21:56:51.517 INFO:tasks.cephadm:First mon is mon.a on smithi060 2024-09-20T21:56:51.517 INFO:tasks.cephadm:First mgr is y 2024-09-20T21:56:51.517 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-20T21:56:51.517 DEBUG:teuthology.orchestra.run.smithi060:> sudo hostname $(hostname -s) 2024-09-20T21:56:51.544 DEBUG:teuthology.orchestra.run.smithi067:> sudo hostname $(hostname -s) 2024-09-20T21:56:51.570 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-20T21:56:51.571 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=9e2ac70bb7aea67d34f0d50002344abbe7803736 2024-09-20T21:56:51.707 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '9e2ac70bb7aea67d34f0d50002344abbe7803736', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56118/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-367-g9e2ac70b', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-367.g9e2ac70b'}, 'url': 'https://1.chacra.ceph.com/r/ceph/squid/9e2ac70bb7aea67d34f0d50002344abbe7803736/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-19 19:23:07.775447', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/squid/9e2ac70bb7aea67d34f0d50002344abbe7803736/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-20T21:56:51.851 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref squid, sha1 9e2ac70bb7aea67d34f0d50002344abbe7803736 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=9e2ac70bb7aea67d34f0d50002344abbe7803736 2024-09-20T21:56:51.852 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/squid/9e2ac70bb7aea67d34f0d50002344abbe7803736/centos/9/x86_64/flavors/default/cephadm 2024-09-20T21:56:51.852 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/squid/9e2ac70bb7aea67d34f0d50002344abbe7803736/centos/9/x86_64/flavors/default/cephadm 2024-09-20T21:56:51.852 DEBUG:teuthology.orchestra.run.smithi060:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/9e2ac70bb7aea67d34f0d50002344abbe7803736/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-20T21:56:52.231 INFO:teuthology.orchestra.run.smithi060.stdout:-rw-r--r--. 1 ubuntu ubuntu 784381 Sep 20 21:56 /home/ubuntu/cephtest/cephadm 2024-09-20T21:56:52.232 DEBUG:teuthology.orchestra.run.smithi067:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/squid/9e2ac70bb7aea67d34f0d50002344abbe7803736/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-20T21:56:52.612 INFO:teuthology.orchestra.run.smithi067.stdout:-rw-r--r--. 1 ubuntu ubuntu 784381 Sep 20 21:56 /home/ubuntu/cephtest/cephadm 2024-09-20T21:56:52.612 DEBUG:teuthology.orchestra.run.smithi060:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-20T21:56:52.632 DEBUG:teuthology.orchestra.run.smithi067:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-20T21:56:52.662 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 on all hosts... 2024-09-20T21:56:52.663 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 pull 2024-09-20T21:56:52.675 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 pull 2024-09-20T21:56:54.318 INFO:teuthology.orchestra.run.smithi060.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736... 2024-09-20T21:56:54.401 INFO:teuthology.orchestra.run.smithi067.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736... 2024-09-20T21:57:21.268 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T21:57:21.269 INFO:teuthology.orchestra.run.smithi060.stdout: "ceph_version": "ceph version 19.1.1-367-g9e2ac70b (9e2ac70bb7aea67d34f0d50002344abbe7803736) squid (stable)", 2024-09-20T21:57:21.269 INFO:teuthology.orchestra.run.smithi060.stdout: "image_id": "cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd", 2024-09-20T21:57:21.269 INFO:teuthology.orchestra.run.smithi060.stdout: "repo_digests": [ 2024-09-20T21:57:21.269 INFO:teuthology.orchestra.run.smithi060.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5" 2024-09-20T21:57:21.270 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T21:57:21.270 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T21:57:21.952 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T21:57:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "ceph_version": "ceph version 19.1.1-367-g9e2ac70b (9e2ac70bb7aea67d34f0d50002344abbe7803736) squid (stable)", 2024-09-20T21:57:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "image_id": "cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd", 2024-09-20T21:57:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "repo_digests": [ 2024-09-20T21:57:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5" 2024-09-20T21:57:21.952 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T21:57:21.952 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T21:57:21.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo mkdir -p /etc/ceph 2024-09-20T21:57:22.015 DEBUG:teuthology.orchestra.run.smithi067:> sudo mkdir -p /etc/ceph 2024-09-20T21:57:22.045 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 777 /etc/ceph 2024-09-20T21:57:22.085 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 777 /etc/ceph 2024-09-20T21:57:22.113 INFO:tasks.cephadm:Writing seed config... 2024-09-20T21:57:22.115 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-20T21:57:22.115 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-20T21:57:22.115 INFO:tasks.cephadm: override: [client.0] debug ms = 1 2024-09-20T21:57:22.115 INFO:tasks.cephadm: override: [client.0] debug objecter = 20 2024-09-20T21:57:22.115 INFO:tasks.cephadm: override: [client.0] debug rados = 20 2024-09-20T21:57:22.116 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-20T21:57:22.116 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-20T21:57:22.116 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-20T21:57:22.116 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-20T21:57:22.116 INFO:tasks.cephadm: override: [mon] mon min osdmap epochs = 50 2024-09-20T21:57:22.116 INFO:tasks.cephadm: override: [mon] mon osdmap full prune interval = 2 2024-09-20T21:57:22.116 INFO:tasks.cephadm: override: [mon] mon osdmap full prune min = 15 2024-09-20T21:57:22.116 INFO:tasks.cephadm: override: [mon] mon osdmap full prune txsize = 2 2024-09-20T21:57:22.117 INFO:tasks.cephadm: override: [mon] paxos service trim min = 10 2024-09-20T21:57:22.117 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-20T21:57:22.117 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-20T21:57:22.117 INFO:tasks.cephadm: override: [osd] osd debug reject backfill probability = 0.3 2024-09-20T21:57:22.117 INFO:tasks.cephadm: override: [osd] osd delete sleep = 1 2024-09-20T21:57:22.117 INFO:tasks.cephadm: override: [osd] osd max backfills = 3 2024-09-20T21:57:22.117 INFO:tasks.cephadm: override: [osd] osd scrub max interval = 120 2024-09-20T21:57:22.118 INFO:tasks.cephadm: override: [osd] osd scrub min interval = 60 2024-09-20T21:57:22.118 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-20T21:57:22.118 INFO:tasks.cephadm: override: [osd] osd snap trim sleep = 2 2024-09-20T21:57:22.119 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-20T21:57:22.119 DEBUG:teuthology.orchestra.run.smithi060:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-20T21:57:22.144 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3df9647e-779b-11ef-baf5-efdc52797490 ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 120 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd debug reject backfill probability = 0.3 osd delete sleep = 1 osd max backfills = 3 osd scrub min interval = 60 osd snap trim sleep = 2 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon min osdmap epochs = 50 mon osdmap full prune interval = 2 mon osdmap full prune min = 15 mon osdmap full prune txsize = 2 paxos service trim min = 10 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client.0] debug ms = 1 debug objecter = 20 debug rados = 20 2024-09-20T21:57:22.144 DEBUG:teuthology.orchestra.run.smithi060:mon.a> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@mon.a.service 2024-09-20T21:57:22.187 DEBUG:teuthology.orchestra.run.smithi060:mgr.y> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@mgr.y.service 2024-09-20T21:57:22.230 INFO:tasks.cephadm:Bootstrapping... 2024-09-20T21:57:22.231 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 -v bootstrap --fsid 3df9647e-779b-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.60 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-20T21:57:22.444 INFO:teuthology.orchestra.run.smithi060.stdout:-------------------------------------------------------------------------------- 2024-09-20T21:57:22.444 INFO:teuthology.orchestra.run.smithi060.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736', '-v', 'bootstrap', '--fsid', '3df9647e-779b-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.60', '--skip-admin-label'] 2024-09-20T21:57:22.445 INFO:teuthology.orchestra.run.smithi060.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-20T21:57:22.445 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying podman|docker is present... 2024-09-20T21:57:22.475 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout 5.2.2 2024-09-20T21:57:22.475 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying lvm2 is present... 2024-09-20T21:57:22.475 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying time synchronization is in place... 2024-09-20T21:57:22.483 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-20T21:57:22.483 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-20T21:57:22.490 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-20T21:57:22.490 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-09-20T21:57:22.497 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout enabled 2024-09-20T21:57:22.503 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout active 2024-09-20T21:57:22.503 INFO:teuthology.orchestra.run.smithi060.stdout:Unit chronyd.service is enabled and running 2024-09-20T21:57:22.503 INFO:teuthology.orchestra.run.smithi060.stdout:Repeating the final host check... 2024-09-20T21:57:22.526 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout 5.2.2 2024-09-20T21:57:22.526 INFO:teuthology.orchestra.run.smithi060.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-20T21:57:22.526 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl is present 2024-09-20T21:57:22.526 INFO:teuthology.orchestra.run.smithi060.stdout:lvcreate is present 2024-09-20T21:57:22.533 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-20T21:57:22.533 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-20T21:57:22.539 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-20T21:57:22.539 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-09-20T21:57:22.546 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout enabled 2024-09-20T21:57:22.553 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout active 2024-09-20T21:57:22.553 INFO:teuthology.orchestra.run.smithi060.stdout:Unit chronyd.service is enabled and running 2024-09-20T21:57:22.553 INFO:teuthology.orchestra.run.smithi060.stdout:Host looks OK 2024-09-20T21:57:22.553 INFO:teuthology.orchestra.run.smithi060.stdout:Cluster fsid: 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:57:22.553 INFO:teuthology.orchestra.run.smithi060.stdout:Acquiring lock 139699570164736 on /run/cephadm/3df9647e-779b-11ef-baf5-efdc52797490.lock 2024-09-20T21:57:22.553 INFO:teuthology.orchestra.run.smithi060.stdout:Lock 139699570164736 acquired on /run/cephadm/3df9647e-779b-11ef-baf5-efdc52797490.lock 2024-09-20T21:57:22.554 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying IP 172.21.15.60 port 3300 ... 2024-09-20T21:57:22.554 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying IP 172.21.15.60 port 6789 ... 2024-09-20T21:57:22.554 INFO:teuthology.orchestra.run.smithi060.stdout:Base mon IP(s) is [172.21.15.60:3300, 172.21.15.60:6789], mon addrv is [v2:172.21.15.60:3300,v1:172.21.15.60:6789] 2024-09-20T21:57:22.557 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.60 metric 100 2024-09-20T21:57:22.557 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.60 metric 100 2024-09-20T21:57:22.560 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-20T21:57:22.560 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-20T21:57:22.560 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1792sec hoplimit 64 pref medium 2024-09-20T21:57:22.563 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-20T21:57:22.563 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-20T21:57:22.563 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-20T21:57:22.563 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-20T21:57:22.563 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1560/64 scope link 2024-09-20T21:57:22.563 INFO:teuthology.orchestra.run.smithi060.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-20T21:57:22.564 INFO:teuthology.orchestra.run.smithi060.stdout:Mon IP `172.21.15.60` is in CIDR network `172.21.0.0/20` 2024-09-20T21:57:22.564 INFO:teuthology.orchestra.run.smithi060.stdout:Mon IP `172.21.15.60` is in CIDR network `172.21.0.0/20` 2024-09-20T21:57:22.564 INFO:teuthology.orchestra.run.smithi060.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-20T21:57:22.564 INFO:teuthology.orchestra.run.smithi060.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-20T21:57:22.565 INFO:teuthology.orchestra.run.smithi060.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736... 2024-09-20T21:57:22.987 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stdout cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd 2024-09-20T21:57:22.988 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736... 2024-09-20T21:57:22.988 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Getting image source signatures 2024-09-20T21:57:22.988 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Copying blob sha256:0031407101cecb3ed6ba394cb16e83fe515e11366eae1f45bd4759e577ea92fc 2024-09-20T21:57:22.988 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-20T21:57:22.988 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Copying config sha256:cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd 2024-09-20T21:57:22.988 INFO:teuthology.orchestra.run.smithi060.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-20T21:57:23.893 INFO:teuthology.orchestra.run.smithi060.stdout:ceph: stdout ceph version 19.1.1-367-g9e2ac70b (9e2ac70bb7aea67d34f0d50002344abbe7803736) squid (stable) 2024-09-20T21:57:23.893 INFO:teuthology.orchestra.run.smithi060.stdout:Ceph version: ceph version 19.1.1-367-g9e2ac70b (9e2ac70bb7aea67d34f0d50002344abbe7803736) squid (stable) 2024-09-20T21:57:23.893 INFO:teuthology.orchestra.run.smithi060.stdout:Extracting ceph user uid/gid from container image... 2024-09-20T21:57:24.823 INFO:teuthology.orchestra.run.smithi060.stdout:stat: stdout 167 167 2024-09-20T21:57:24.823 INFO:teuthology.orchestra.run.smithi060.stdout:Creating initial keys... 2024-09-20T21:57:25.913 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph-authtool: stdout AQDF7+1mpdrWChAAect5dw1vTxH7tVAJpjnj2A== 2024-09-20T21:57:26.809 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph-authtool: stdout AQDG7+1mbOYyDRAAtJzoxnHJG4/GDFDnAZHPjA== 2024-09-20T21:57:27.824 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph-authtool: stdout AQDH7+1mq2ZnCRAAaEjbYWJNg1SMmvWViUbCHw== 2024-09-20T21:57:27.825 INFO:teuthology.orchestra.run.smithi060.stdout:Creating initial monmap... 2024-09-20T21:57:28.788 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-20T21:57:28.789 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-20T21:57:28.789 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:57:28.789 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-20T21:57:28.789 INFO:teuthology.orchestra.run.smithi060.stdout:monmaptool for a [v2:172.21.15.60:3300,v1:172.21.15.60:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-20T21:57:28.789 INFO:teuthology.orchestra.run.smithi060.stdout:setting min_mon_release = quincy 2024-09-20T21:57:28.789 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: set fsid to 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:57:28.789 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-20T21:57:28.789 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:57:28.789 INFO:teuthology.orchestra.run.smithi060.stdout:Creating mon... 2024-09-20T21:57:29.794 INFO:teuthology.orchestra.run.smithi060.stdout:create mon.a on 2024-09-20T21:57:30.022 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-20T21:57:30.171 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-20T21:57:30.377 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3df9647e-779b-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-3df9647e-779b-11ef-baf5-efdc52797490.target. 2024-09-20T21:57:30.378 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3df9647e-779b-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-3df9647e-779b-11ef-baf5-efdc52797490.target. 2024-09-20T21:57:30.616 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3df9647e-779b-11ef-baf5-efdc52797490@mon.a 2024-09-20T21:57:30.616 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3df9647e-779b-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-3df9647e-779b-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-20T21:57:30.791 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3df9647e-779b-11ef-baf5-efdc52797490.target.wants/ceph-3df9647e-779b-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-3df9647e-779b-11ef-baf5-efdc52797490@.service. 2024-09-20T21:57:31.143 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:31 smithi060 podman[29185]: 2024-09-20 21:57:30.948928634 +0000 UTC m=+0.016600273 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 2024-09-20T21:57:31.276 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-20T21:57:31.276 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-09-20T21:57:31.283 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-20T21:57:31.283 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-09-20T21:57:31.283 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-09-20T21:57:31.283 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to enable service . firewalld.service is not available 2024-09-20T21:57:31.283 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mon to start... 2024-09-20T21:57:31.283 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mon... 2024-09-20T21:57:31.518 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:31 smithi060 podman[29185]: 2024-09-20 21:57:31.202713573 +0000 UTC m=+0.270385208 container init 9132d79d3d70d9a61d6af1353b2559fc6445cf403d08865bac170557f40046b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-9e2ac70, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-20T21:57:31.518 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:31 smithi060 podman[29185]: 2024-09-20 21:57:31.20743043 +0000 UTC m=+0.275102058 container start 9132d79d3d70d9a61d6af1353b2559fc6445cf403d08865bac170557f40046b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-9e2ac70, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-20T21:57:31.518 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:31 smithi060 ceph-mon[29222]: mkfs 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:57:31.518 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:31 smithi060 ceph-mon[29222]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-20T21:57:31.518 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:31 smithi060 bash[29185]: 9132d79d3d70d9a61d6af1353b2559fc6445cf403d08865bac170557f40046b9 2024-09-20T21:57:31.518 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:31 smithi060 systemd[1]: Started Ceph mon.a for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T21:57:32.435 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:32 smithi060 ceph-mon[29222]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-20T21:57:32.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:32 smithi060 ceph-mon[29222]: monmap epoch 1 2024-09-20T21:57:32.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:32 smithi060 ceph-mon[29222]: fsid 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:57:32.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:32 smithi060 ceph-mon[29222]: last_changed 2024-09-20T21:57:28.205024+0000 2024-09-20T21:57:32.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:32 smithi060 ceph-mon[29222]: created 2024-09-20T21:57:28.205024+0000 2024-09-20T21:57:32.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:32 smithi060 ceph-mon[29222]: min_mon_release 19 (squid) 2024-09-20T21:57:32.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:32 smithi060 ceph-mon[29222]: election_strategy: 1 2024-09-20T21:57:32.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:32 smithi060 ceph-mon[29222]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-20T21:57:32.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:32 smithi060 ceph-mon[29222]: fsmap 2024-09-20T21:57:32.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:32 smithi060 ceph-mon[29222]: osdmap e1: 0 total, 0 up, 0 in 2024-09-20T21:57:32.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:32 smithi060 ceph-mon[29222]: mgrmap e1: no daemons active 2024-09-20T21:57:32.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:32 smithi060 ceph-mon[29222]: from='client.? 172.21.15.60:0/1263929463' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-20T21:57:32.440 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout cluster: 2024-09-20T21:57:32.440 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout id: 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:57:32.440 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-20T21:57:32.440 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-20T21:57:32.440 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout services: 2024-09-20T21:57:32.440 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.563808s) 2024-09-20T21:57:32.440 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-20T21:57:32.440 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-20T21:57:32.440 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-20T21:57:32.440 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout data: 2024-09-20T21:57:32.440 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-20T21:57:32.440 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-20T21:57:32.441 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-20T21:57:32.441 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout pgs: 2024-09-20T21:57:32.441 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-20T21:57:32.441 INFO:teuthology.orchestra.run.smithi060.stdout:mon is available 2024-09-20T21:57:32.441 INFO:teuthology.orchestra.run.smithi060.stdout:Assimilating anything we can from ceph.conf... 2024-09-20T21:57:33.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:33 smithi060 ceph-mon[29222]: from='client.? 172.21.15.60:0/1911744243' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-20T21:57:33.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:33 smithi060 ceph-mon[29222]: from='client.? 172.21.15.60:0/1911744243' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-20T21:57:33.503 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-20T21:57:33.503 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [global] 2024-09-20T21:57:33.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout fsid = 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:57:33.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.60:3300,v1:172.21.15.60:6789] 2024-09-20T21:57:33.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-20T21:57:33.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-20T21:57:33.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-20T21:57:33.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-20T21:57:33.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-20T21:57:33.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-20T21:57:33.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-20T21:57:33.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-20T21:57:33.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [osd] 2024-09-20T21:57:33.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-20T21:57:33.504 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-20T21:57:33.504 INFO:teuthology.orchestra.run.smithi060.stdout:Generating new minimal ceph.conf... 2024-09-20T21:57:34.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:34 smithi060 ceph-mon[29222]: from='client.? 172.21.15.60:0/356044068' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:57:34.550 INFO:teuthology.orchestra.run.smithi060.stdout:Restarting the monitor... 2024-09-20T21:57:34.820 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:34 smithi060 systemd[1]: Stopping Ceph mon.a for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T21:57:35.158 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:34 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a[29199]: 2024-09-20T21:57:34.817+0000 7f30d03c9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T21:57:35.159 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:34 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a[29199]: 2024-09-20T21:57:34.817+0000 7f30d03c9640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-20T21:57:35.159 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:34 smithi060 podman[29539]: 2024-09-20 21:57:34.941629928 +0000 UTC m=+0.226145655 container died 9132d79d3d70d9a61d6af1353b2559fc6445cf403d08865bac170557f40046b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-9e2ac70, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-20T21:57:35.459 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 podman[29539]: 2024-09-20 21:57:35.24853542 +0000 UTC m=+0.533051129 container cleanup 9132d79d3d70d9a61d6af1353b2559fc6445cf403d08865bac170557f40046b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-9e2ac70, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-20T21:57:35.459 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 bash[29539]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a 2024-09-20T21:57:35.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 podman[29559]: 2024-09-20 21:57:35.458201895 +0000 UTC m=+0.508065710 container remove 9132d79d3d70d9a61d6af1353b2559fc6445cf403d08865bac170557f40046b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-9e2ac70, org.label-schema.build-date=20240731) 2024-09-20T21:57:35.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 systemd[1]: ceph-3df9647e-779b-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-20T21:57:35.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 systemd[1]: Stopped Ceph mon.a for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T21:57:35.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 systemd[1]: Starting Ceph mon.a for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T21:57:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 podman[29656]: 2024-09-20 21:57:35.775474565 +0000 UTC m=+0.097334905 container create e363e67817222ccb42015593a937673a46aae158883950c4035ffe4bc2de6be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-9e2ac70, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-20T21:57:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 podman[29656]: 2024-09-20 21:57:35.694345597 +0000 UTC m=+0.016205948 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 2024-09-20T21:57:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 podman[29656]: 2024-09-20 21:57:35.961152216 +0000 UTC m=+0.283012565 container init e363e67817222ccb42015593a937673a46aae158883950c4035ffe4bc2de6be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-9e2ac70, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-09-20T21:57:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 podman[29656]: 2024-09-20 21:57:35.965734535 +0000 UTC m=+0.287594875 container start e363e67817222ccb42015593a937673a46aae158883950c4035ffe4bc2de6be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-9e2ac70) 2024-09-20T21:57:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: set uid:gid to 167:167 (ceph:ceph) 2024-09-20T21:57:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: ceph version 19.1.1-367-g9e2ac70b (9e2ac70bb7aea67d34f0d50002344abbe7803736) squid (stable), process ceph-mon, pid 7 2024-09-20T21:57:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: pidfile_write: ignore empty --pid-file 2024-09-20T21:57:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: load: jerasure load: lrc 2024-09-20T21:57:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: RocksDB version: 7.9.2 2024-09-20T21:57:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Git sha 0 2024-09-20T21:57:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Compile date 2024-09-19 18:20:48 2024-09-20T21:57:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: DB SUMMARY 2024-09-20T21:57:36.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: DB Session ID: UEJWGH9HTBWTZO6RWNGZ 2024-09-20T21:57:36.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: CURRENT file: CURRENT 2024-09-20T21:57:36.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: IDENTITY file: IDENTITY 2024-09-20T21:57:36.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-20T21:57:36.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-20T21:57:36.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94977 ; 2024-09-20T21:57:36.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.error_if_exists: 0 2024-09-20T21:57:36.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.create_if_missing: 0 2024-09-20T21:57:36.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.paranoid_checks: 1 2024-09-20T21:57:36.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-20T21:57:36.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-20T21:57:36.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-20T21:57:36.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.env: 0x55bcab3e0940 2024-09-20T21:57:36.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.fs: PosixFileSystem 2024-09-20T21:57:36.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.info_log: 0x55bcad8e96e0 2024-09-20T21:57:36.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_file_opening_threads: 16 2024-09-20T21:57:36.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.statistics: (nil) 2024-09-20T21:57:36.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.use_fsync: 0 2024-09-20T21:57:36.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_log_file_size: 0 2024-09-20T21:57:36.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-20T21:57:36.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-20T21:57:36.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.keep_log_file_num: 1000 2024-09-20T21:57:36.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.recycle_log_file_num: 0 2024-09-20T21:57:36.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.allow_fallocate: 1 2024-09-20T21:57:36.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.allow_mmap_reads: 0 2024-09-20T21:57:36.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.allow_mmap_writes: 0 2024-09-20T21:57:36.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.use_direct_reads: 0 2024-09-20T21:57:36.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-20T21:57:36.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.create_missing_column_families: 0 2024-09-20T21:57:36.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.db_log_dir: 2024-09-20T21:57:36.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.wal_dir: 2024-09-20T21:57:36.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-20T21:57:36.041 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-20T21:57:36.041 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-20T21:57:36.041 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-20T21:57:36.041 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-20T21:57:36.041 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-20T21:57:36.041 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.advise_random_on_open: 1 2024-09-20T21:57:36.041 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.db_write_buffer_size: 0 2024-09-20T21:57:36.041 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.write_buffer_manager: 0x55bcad8f8dc0 2024-09-20T21:57:36.041 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-20T21:57:36.041 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-20T21:57:36.041 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-20T21:57:36.041 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.rate_limiter: (nil) 2024-09-20T21:57:36.041 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-20T21:57:36.042 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.wal_recovery_mode: 2 2024-09-20T21:57:36.042 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.enable_thread_tracking: 0 2024-09-20T21:57:36.042 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.enable_pipelined_write: 0 2024-09-20T21:57:36.042 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.unordered_write: 0 2024-09-20T21:57:36.042 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-20T21:57:36.042 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-20T21:57:36.042 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-20T21:57:36.042 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-20T21:57:36.042 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.row_cache: None 2024-09-20T21:57:36.042 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.wal_filter: None 2024-09-20T21:57:36.042 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-20T21:57:36.042 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.allow_ingest_behind: 0 2024-09-20T21:57:36.042 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.two_write_queues: 0 2024-09-20T21:57:36.042 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.manual_wal_flush: 0 2024-09-20T21:57:36.043 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.wal_compression: 0 2024-09-20T21:57:36.043 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.atomic_flush: 0 2024-09-20T21:57:36.043 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-20T21:57:36.043 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-20T21:57:36.043 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-20T21:57:36.043 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.log_readahead_size: 0 2024-09-20T21:57:36.043 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-20T21:57:36.043 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.best_efforts_recovery: 0 2024-09-20T21:57:36.043 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-20T21:57:36.043 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-20T21:57:36.043 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.allow_data_in_errors: 0 2024-09-20T21:57:36.043 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.db_host_id: __hostname__ 2024-09-20T21:57:36.043 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-20T21:57:36.043 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_background_jobs: 2 2024-09-20T21:57:36.044 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_background_compactions: -1 2024-09-20T21:57:36.044 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_subcompactions: 1 2024-09-20T21:57:36.044 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-20T21:57:36.044 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-20T21:57:36.044 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-20T21:57:36.044 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_total_wal_size: 0 2024-09-20T21:57:36.044 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-20T21:57:36.044 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-20T21:57:36.044 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-20T21:57:36.044 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-20T21:57:36.044 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_open_files: -1 2024-09-20T21:57:36.044 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.bytes_per_sync: 0 2024-09-20T21:57:36.044 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-20T21:57:36.044 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-20T21:57:36.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compaction_readahead_size: 0 2024-09-20T21:57:36.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_background_flushes: -1 2024-09-20T21:57:36.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Compression algorithms supported: 2024-09-20T21:57:36.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: kZSTD supported: 0 2024-09-20T21:57:36.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: kXpressCompression supported: 0 2024-09-20T21:57:36.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: kBZip2Compression supported: 0 2024-09-20T21:57:36.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-20T21:57:36.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: kLZ4Compression supported: 1 2024-09-20T21:57:36.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: kZlibCompression supported: 1 2024-09-20T21:57:36.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: kLZ4HCCompression supported: 1 2024-09-20T21:57:36.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: kSnappyCompression supported: 1 2024-09-20T21:57:36.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-20T21:57:36.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-20T21:57:36.045 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-20T21:57:36.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-20T21:57:36.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-20T21:57:36.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.merge_operator: 2024-09-20T21:57:36.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compaction_filter: None 2024-09-20T21:57:36.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compaction_filter_factory: None 2024-09-20T21:57:36.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.sst_partitioner_factory: None 2024-09-20T21:57:36.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-20T21:57:36.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-20T21:57:36.046 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55bcad8e9320) 2024-09-20T21:57:36.046 INFO:journalctl@ceph.mon.a.smithi060.stdout: cache_index_and_filter_blocks: 1 2024-09-20T21:57:36.046 INFO:journalctl@ceph.mon.a.smithi060.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-20T21:57:36.046 INFO:journalctl@ceph.mon.a.smithi060.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-20T21:57:36.046 INFO:journalctl@ceph.mon.a.smithi060.stdout: pin_top_level_index_and_filter: 1 2024-09-20T21:57:36.047 INFO:journalctl@ceph.mon.a.smithi060.stdout: index_type: 0 2024-09-20T21:57:36.047 INFO:journalctl@ceph.mon.a.smithi060.stdout: data_block_index_type: 0 2024-09-20T21:57:36.047 INFO:journalctl@ceph.mon.a.smithi060.stdout: index_shortening: 1 2024-09-20T21:57:36.047 INFO:journalctl@ceph.mon.a.smithi060.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-20T21:57:36.047 INFO:journalctl@ceph.mon.a.smithi060.stdout: checksum: 4 2024-09-20T21:57:36.047 INFO:journalctl@ceph.mon.a.smithi060.stdout: no_block_cache: 0 2024-09-20T21:57:36.047 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache: 0x55bcad8dd1f0 2024-09-20T21:57:36.047 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache_name: BinnedLRUCache 2024-09-20T21:57:36.047 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache_options: 2024-09-20T21:57:36.047 INFO:journalctl@ceph.mon.a.smithi060.stdout: capacity : 536870912 2024-09-20T21:57:36.047 INFO:journalctl@ceph.mon.a.smithi060.stdout: num_shard_bits : 4 2024-09-20T21:57:36.047 INFO:journalctl@ceph.mon.a.smithi060.stdout: strict_capacity_limit : 0 2024-09-20T21:57:36.047 INFO:journalctl@ceph.mon.a.smithi060.stdout: high_pri_pool_ratio: 0.000 2024-09-20T21:57:36.047 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_cache_compressed: (nil) 2024-09-20T21:57:36.048 INFO:journalctl@ceph.mon.a.smithi060.stdout: persistent_cache: (nil) 2024-09-20T21:57:36.048 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_size: 4096 2024-09-20T21:57:36.048 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_size_deviation: 10 2024-09-20T21:57:36.048 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_restart_interval: 16 2024-09-20T21:57:36.048 INFO:journalctl@ceph.mon.a.smithi060.stdout: index_block_restart_interval: 1 2024-09-20T21:57:36.048 INFO:journalctl@ceph.mon.a.smithi060.stdout: metadata_block_size: 4096 2024-09-20T21:57:36.049 INFO:journalctl@ceph.mon.a.smithi060.stdout: partition_filters: 0 2024-09-20T21:57:36.049 INFO:journalctl@ceph.mon.a.smithi060.stdout: use_delta_encoding: 1 2024-09-20T21:57:36.049 INFO:journalctl@ceph.mon.a.smithi060.stdout: filter_policy: bloomfilter 2024-09-20T21:57:36.049 INFO:journalctl@ceph.mon.a.smithi060.stdout: whole_key_filtering: 1 2024-09-20T21:57:36.049 INFO:journalctl@ceph.mon.a.smithi060.stdout: verify_compression: 0 2024-09-20T21:57:36.049 INFO:journalctl@ceph.mon.a.smithi060.stdout: read_amp_bytes_per_bit: 0 2024-09-20T21:57:36.049 INFO:journalctl@ceph.mon.a.smithi060.stdout: format_version: 5 2024-09-20T21:57:36.049 INFO:journalctl@ceph.mon.a.smithi060.stdout: enable_index_compression: 1 2024-09-20T21:57:36.049 INFO:journalctl@ceph.mon.a.smithi060.stdout: block_align: 0 2024-09-20T21:57:36.049 INFO:journalctl@ceph.mon.a.smithi060.stdout: max_auto_readahead_size: 262144 2024-09-20T21:57:36.049 INFO:journalctl@ceph.mon.a.smithi060.stdout: prepopulate_block_cache: 0 2024-09-20T21:57:36.049 INFO:journalctl@ceph.mon.a.smithi060.stdout: initial_auto_readahead_size: 8192 2024-09-20T21:57:36.050 INFO:journalctl@ceph.mon.a.smithi060.stdout: num_file_reads_for_auto_readahead: 2 2024-09-20T21:57:36.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.write_buffer_size: 33554432 2024-09-20T21:57:36.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_write_buffer_number: 2 2024-09-20T21:57:36.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compression: NoCompression 2024-09-20T21:57:36.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.bottommost_compression: Disabled 2024-09-20T21:57:36.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.prefix_extractor: nullptr 2024-09-20T21:57:36.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-20T21:57:36.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.num_levels: 7 2024-09-20T21:57:36.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-20T21:57:36.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-20T21:57:36.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-20T21:57:36.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-20T21:57:36.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-20T21:57:36.050 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-20T21:57:36.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-20T21:57:36.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-20T21:57:36.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-20T21:57:36.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-20T21:57:36.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-20T21:57:36.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-20T21:57:36.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-20T21:57:36.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compression_opts.level: 32767 2024-09-20T21:57:36.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compression_opts.strategy: 0 2024-09-20T21:57:36.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-20T21:57:36.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-20T21:57:36.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-20T21:57:36.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-20T21:57:36.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compression_opts.enabled: false 2024-09-20T21:57:36.051 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-20T21:57:36.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-20T21:57:36.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-20T21:57:36.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-20T21:57:36.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.target_file_size_base: 67108864 2024-09-20T21:57:36.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-20T21:57:36.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-20T21:57:36.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-20T21:57:36.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-20T21:57:36.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-20T21:57:36.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-20T21:57:36.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-20T21:57:36.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-20T21:57:36.052 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-20T21:57:36.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-20T21:57:36.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-20T21:57:36.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-20T21:57:36.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-20T21:57:36.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-20T21:57:36.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.arena_block_size: 1048576 2024-09-20T21:57:36.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-20T21:57:36.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-20T21:57:36.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.disable_auto_compactions: 0 2024-09-20T21:57:36.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-20T21:57:36.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-20T21:57:36.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-20T21:57:36.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-20T21:57:36.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-20T21:57:36.053 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-20T21:57:36.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-20T21:57:36.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-20T21:57:36.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-20T21:57:36.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-20T21:57:36.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-20T21:57:36.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.inplace_update_support: 0 2024-09-20T21:57:36.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-20T21:57:36.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-20T21:57:36.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-20T21:57:36.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-20T21:57:36.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.bloom_locality: 0 2024-09-20T21:57:36.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.max_successive_merges: 0 2024-09-20T21:57:36.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-20T21:57:36.054 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.paranoid_file_checks: 0 2024-09-20T21:57:36.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.force_consistency_checks: 1 2024-09-20T21:57:36.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.report_bg_io_stats: 0 2024-09-20T21:57:36.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.ttl: 2592000 2024-09-20T21:57:36.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-20T21:57:36.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-20T21:57:36.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-20T21:57:36.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.enable_blob_files: false 2024-09-20T21:57:36.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.min_blob_size: 0 2024-09-20T21:57:36.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.blob_file_size: 268435456 2024-09-20T21:57:36.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-20T21:57:36.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-20T21:57:36.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-20T21:57:36.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-20T21:57:36.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-20T21:57:36.056 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.blob_file_starting_level: 0 2024-09-20T21:57:36.056 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-20T21:57:36.056 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-20T21:57:36.056 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-20T21:57:36.056 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 04c53477-d3bb-42c2-9ff5-15186bc5285f 2024-09-20T21:57:36.056 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726869455993108, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-20T21:57:36.056 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-20T21:57:36.056 INFO:teuthology.orchestra.run.smithi060.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-20T21:57:36.057 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726869455994680, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90964, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 284, "table_properties": {"data_size": 89006, "index_size": 278, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11754, "raw_average_key_size": 48, "raw_value_size": 82525, "raw_average_value_size": 339, "num_data_blocks": 12, "num_entries": 243, "num_filter_entries": 243, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726869455, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "04c53477-d3bb-42c2-9ff5-15186bc5285f", "db_session_id": "UEJWGH9HTBWTZO6RWNGZ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-20T21:57:36.057 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726869455994834, "job": 1, "event": "recovery_finished"} 2024-09-20T21:57:36.057 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-20T21:57:36.057 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-20T21:57:36.057 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55bcad90ae00 2024-09-20T21:57:36.057 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: DB pointer 0x55bcada06000 2024-09-20T21:57:36.057 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-20T21:57:36.057 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-20T21:57:36.057 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** DB Stats ** 2024-09-20T21:57:36.057 INFO:journalctl@ceph.mon.a.smithi060.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-20T21:57:36.057 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-20T21:57:36.057 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-20T21:57:36.057 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-20T21:57:36.057 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-20T21:57:36.058 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-20T21:57:36.058 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-20T21:57:36.058 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-09-20T21:57:36.058 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** Compaction Stats [default] ** 2024-09-20T21:57:36.058 INFO:journalctl@ceph.mon.a.smithi060.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-20T21:57:36.058 INFO:journalctl@ceph.mon.a.smithi060.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-20T21:57:36.058 INFO:journalctl@ceph.mon.a.smithi060.stdout: L0 2/0 90.69 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 86.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-20T21:57:36.058 INFO:journalctl@ceph.mon.a.smithi060.stdout: Sum 2/0 90.69 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 86.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-20T21:57:36.058 INFO:journalctl@ceph.mon.a.smithi060.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 86.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-20T21:57:36.058 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-09-20T21:57:36.058 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** Compaction Stats [default] ** 2024-09-20T21:57:36.058 INFO:journalctl@ceph.mon.a.smithi060.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-20T21:57:36.058 INFO:journalctl@ceph.mon.a.smithi060.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-20T21:57:36.058 INFO:journalctl@ceph.mon.a.smithi060.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 86.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-20T21:57:36.059 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-09-20T21:57:36.059 INFO:journalctl@ceph.mon.a.smithi060.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-20T21:57:36.059 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-09-20T21:57:36.059 INFO:journalctl@ceph.mon.a.smithi060.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-20T21:57:36.059 INFO:journalctl@ceph.mon.a.smithi060.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-20T21:57:36.059 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-20T21:57:36.059 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-20T21:57:36.059 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-20T21:57:36.059 INFO:journalctl@ceph.mon.a.smithi060.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-20T21:57:36.059 INFO:journalctl@ceph.mon.a.smithi060.stdout: Cumulative compaction: 0.00 GB write, 18.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-20T21:57:36.059 INFO:journalctl@ceph.mon.a.smithi060.stdout: Interval compaction: 0.00 GB write, 18.29 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-20T21:57:36.059 INFO:journalctl@ceph.mon.a.smithi060.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-20T21:57:36.059 INFO:journalctl@ceph.mon.a.smithi060.stdout: Block cache BinnedLRUCache@0x55bcad8dd1f0#7 capacity: 512.00 MB usage: 1.30 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.6e-05 secs_since: 0 2024-09-20T21:57:36.059 INFO:journalctl@ceph.mon.a.smithi060.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.83 KB,0.000157952%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-20T21:57:36.059 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-09-20T21:57:36.060 INFO:journalctl@ceph.mon.a.smithi060.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-20T21:57:36.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: starting mon.a rank 0 at public addrs [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] at bind addrs [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:57:36.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: mon.a@-1(???) e1 preinit fsid 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:57:36.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: mon.a@-1(???).mds e1 new map 2024-09-20T21:57:36.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: mon.a@-1(???).mds e1 print_map 2024-09-20T21:57:36.060 INFO:journalctl@ceph.mon.a.smithi060.stdout: e1 2024-09-20T21:57:36.060 INFO:journalctl@ceph.mon.a.smithi060.stdout: btime 2024-09-20T21:57:31:244773+0000 2024-09-20T21:57:36.060 INFO:journalctl@ceph.mon.a.smithi060.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-20T21:57:36.060 INFO:journalctl@ceph.mon.a.smithi060.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-20T21:57:36.060 INFO:journalctl@ceph.mon.a.smithi060.stdout: legacy client fscid: -1 2024-09-20T21:57:36.060 INFO:journalctl@ceph.mon.a.smithi060.stdout: 2024-09-20T21:57:36.060 INFO:journalctl@ceph.mon.a.smithi060.stdout: No filesystems configured 2024-09-20T21:57:36.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-20T21:57:36.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-20T21:57:36.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-20T21:57:36.060 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-20T21:57:36.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:35 smithi060 ceph-mon[29694]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-20T21:57:36.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:36 smithi060 bash[29656]: e363e67817222ccb42015593a937673a46aae158883950c4035ffe4bc2de6be1 2024-09-20T21:57:36.061 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:36 smithi060 systemd[1]: Started Ceph mon.a for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T21:57:36.291 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:36 smithi060 ceph-mon[29694]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-20T21:57:36.291 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:36 smithi060 ceph-mon[29694]: monmap epoch 1 2024-09-20T21:57:36.291 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:36 smithi060 ceph-mon[29694]: fsid 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:57:36.291 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:36 smithi060 ceph-mon[29694]: last_changed 2024-09-20T21:57:28.205024+0000 2024-09-20T21:57:36.291 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:36 smithi060 ceph-mon[29694]: created 2024-09-20T21:57:28.205024+0000 2024-09-20T21:57:36.291 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:36 smithi060 ceph-mon[29694]: min_mon_release 19 (squid) 2024-09-20T21:57:36.291 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:36 smithi060 ceph-mon[29694]: election_strategy: 1 2024-09-20T21:57:36.291 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:36 smithi060 ceph-mon[29694]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-20T21:57:36.291 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:36 smithi060 ceph-mon[29694]: fsmap 2024-09-20T21:57:36.291 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:36 smithi060 ceph-mon[29694]: osdmap e1: 0 total, 0 up, 0 in 2024-09-20T21:57:36.291 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:36 smithi060 ceph-mon[29694]: mgrmap e1: no daemons active 2024-09-20T21:57:37.096 INFO:teuthology.orchestra.run.smithi060.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-20T21:57:37.117 INFO:teuthology.orchestra.run.smithi060.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-20T21:57:37.118 INFO:teuthology.orchestra.run.smithi060.stdout:Creating mgr... 2024-09-20T21:57:37.118 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-20T21:57:37.118 INFO:teuthology.orchestra.run.smithi060.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-20T21:57:37.373 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3df9647e-779b-11ef-baf5-efdc52797490@mgr.y 2024-09-20T21:57:37.374 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3df9647e-779b-11ef-baf5-efdc52797490@mgr.y.service: Unit ceph-3df9647e-779b-11ef-baf5-efdc52797490@mgr.y.service not loaded. 2024-09-20T21:57:37.534 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3df9647e-779b-11ef-baf5-efdc52797490.target.wants/ceph-3df9647e-779b-11ef-baf5-efdc52797490@mgr.y.service → /etc/systemd/system/ceph-3df9647e-779b-11ef-baf5-efdc52797490@.service. 2024-09-20T21:57:37.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:37 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1287971683' entity='client.admin' 2024-09-20T21:57:37.867 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:37 smithi060 podman[29931]: 2024-09-20 21:57:37.676152353 +0000 UTC m=+0.016523050 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 2024-09-20T21:57:38.087 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-20T21:57:38.087 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-09-20T21:57:38.095 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-20T21:57:38.095 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-09-20T21:57:38.095 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-09-20T21:57:38.095 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to enable service . firewalld.service is not available 2024-09-20T21:57:38.106 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-20T21:57:38.106 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-09-20T21:57:38.115 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-20T21:57:38.115 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-09-20T21:57:38.115 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-09-20T21:57:38.115 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-20T21:57:38.115 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr to start... 2024-09-20T21:57:38.115 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr... 2024-09-20T21:57:38.238 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:37 smithi060 podman[29931]: 2024-09-20 21:57:37.97944858 +0000 UTC m=+0.319819275 container init 66443b29df2ff148d9da4279bf9b97516f42f8df2a03a597bef1dd678382db79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-9e2ac70, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-20T21:57:38.239 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:37 smithi060 podman[29931]: 2024-09-20 21:57:37.984690858 +0000 UTC m=+0.325061545 container start 66443b29df2ff148d9da4279bf9b97516f42f8df2a03a597bef1dd678382db79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y, RELEASE=squid-9e2ac70, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-20T21:57:38.239 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:38 smithi060 bash[29931]: 66443b29df2ff148d9da4279bf9b97516f42f8df2a03a597bef1dd678382db79 2024-09-20T21:57:38.239 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:38 smithi060 systemd[1]: Started Ceph mgr.y for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T21:57:38.239 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:38 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:38.113+0000 7fe11c25f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-20T21:57:38.239 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:38 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:38.175+0000 7fe11c25f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-20T21:57:38.509 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:38 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:38.333+0000 7fe11c25f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-20T21:57:38.509 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:38 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:38.420+0000 7fe11c25f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-20T21:57:38.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:38 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2883946848' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-20T21:57:38.786 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:38 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:38.577+0000 7fe11c25f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-20T21:57:38.786 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:38 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:38.729+0000 7fe11c25f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-20T21:57:39.222 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:39 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:39.062+0000 7fe11c25f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-20T21:57:39.222 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:39 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:39.125+0000 7fe11c25f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-20T21:57:39.228 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-20T21:57:39.228 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-09-20T21:57:39.229 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "3df9647e-779b-11ef-baf5-efdc52797490", 2024-09-20T21:57:39.229 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-09-20T21:57:39.229 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-20T21:57:39.229 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-20T21:57:39.229 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-20T21:57:39.229 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:39.229 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-20T21:57:39.229 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-20T21:57:39.230 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-09-20T21:57:39.230 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-09-20T21:57:39.230 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-20T21:57:39.230 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-09-20T21:57:39.230 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-09-20T21:57:39.230 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-20T21:57:39.230 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-20T21:57:39.230 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T21:57:39.230 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-20T21:57:39.230 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-20T21:57:39.230 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:39.230 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-20T21:57:39.231 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T21:57:39.231 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-20T21:57:39.231 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-20T21:57:39.231 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-20T21:57:39.231 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-20T21:57:39.231 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-20T21:57:39.231 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-20T21:57:39.231 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:39.231 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-20T21:57:39.231 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-20T21:57:39.231 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-20T21:57:39.231 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-20T21:57:39.231 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-20T21:57:39.231 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-20T21:57:39.231 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-20T21:57:39.231 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-20T21:57:39.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-20T21:57:39.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:39.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-20T21:57:39.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T21:57:39.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "btime": "2024-09-20T21:57:31:244773+0000", 2024-09-20T21:57:39.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-20T21:57:39.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-20T21:57:39.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:39.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-20T21:57:39.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-20T21:57:39.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-20T21:57:39.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-20T21:57:39.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-20T21:57:39.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-20T21:57:39.232 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-09-20T21:57:39.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-09-20T21:57:39.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-20T21:57:39.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:39.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-20T21:57:39.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T21:57:39.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-09-20T21:57:31.245398+0000", 2024-09-20T21:57:39.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-20T21:57:39.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:39.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-20T21:57:39.233 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-09-20T21:57:39.233 INFO:teuthology.orchestra.run.smithi060.stdout:mgr not available, waiting (1/15)... 2024-09-20T21:57:39.536 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:39 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:39.320+0000 7fe11c25f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-20T21:57:40.536 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:40 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:40.240+0000 7fe11c25f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-20T21:57:40.536 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:40 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:40.310+0000 7fe11c25f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-20T21:57:41.337 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:41 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:41.063+0000 7fe11c25f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-20T21:57:41.337 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:41 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:41.183+0000 7fe11c25f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-20T21:57:41.635 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:41 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:41.335+0000 7fe11c25f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-20T21:57:41.636 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:41 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:41.400+0000 7fe11c25f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-20T21:57:41.636 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:41 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:41.561+0000 7fe11c25f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-20T21:57:41.636 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:41 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:41.634+0000 7fe11c25f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-20T21:57:41.949 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:41 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1148104965' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-20T21:57:41.949 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:41 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:41.709+0000 7fe11c25f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-20T21:57:41.949 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:41 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:41.878+0000 7fe11c25f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-20T21:57:41.949 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:41 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:41.947+0000 7fe11c25f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-20T21:57:42.233 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:42 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:42.020+0000 7fe11c25f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-20T21:57:42.233 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:42 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:42.108+0000 7fe11c25f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-20T21:57:42.233 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:42 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:42.170+0000 7fe11c25f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-20T21:57:42.233 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:42 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:42.231+0000 7fe11c25f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-20T21:57:42.327 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-20T21:57:42.327 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-09-20T21:57:42.327 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "3df9647e-779b-11ef-baf5-efdc52797490", 2024-09-20T21:57:42.327 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-09-20T21:57:42.327 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-20T21:57:42.327 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-20T21:57:42.327 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-20T21:57:42.327 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:42.328 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-20T21:57:42.328 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-20T21:57:42.328 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-09-20T21:57:42.328 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-09-20T21:57:42.328 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-20T21:57:42.328 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-09-20T21:57:42.328 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-09-20T21:57:42.328 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-20T21:57:42.328 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-20T21:57:42.328 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T21:57:42.328 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-20T21:57:42.328 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-20T21:57:42.328 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:42.328 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-20T21:57:42.328 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T21:57:42.329 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-20T21:57:42.329 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-20T21:57:42.329 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-20T21:57:42.329 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-20T21:57:42.329 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-20T21:57:42.329 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-20T21:57:42.329 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:42.329 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-20T21:57:42.329 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-20T21:57:42.329 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-20T21:57:42.329 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-20T21:57:42.329 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-20T21:57:42.329 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-20T21:57:42.329 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-20T21:57:42.330 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-20T21:57:42.330 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-20T21:57:42.330 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:42.330 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-20T21:57:42.330 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T21:57:42.330 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "btime": "2024-09-20T21:57:31:244773+0000", 2024-09-20T21:57:42.330 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-20T21:57:42.330 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-20T21:57:42.330 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:42.330 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-20T21:57:42.330 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-20T21:57:42.330 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-20T21:57:42.330 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-20T21:57:42.330 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-20T21:57:42.331 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-20T21:57:42.331 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-09-20T21:57:42.331 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-09-20T21:57:42.331 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-20T21:57:42.331 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:42.331 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-20T21:57:42.331 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T21:57:42.331 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-09-20T21:57:31.245398+0000", 2024-09-20T21:57:42.331 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-20T21:57:42.331 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:42.331 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-20T21:57:42.331 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-09-20T21:57:42.331 INFO:teuthology.orchestra.run.smithi060.stdout:mgr not available, waiting (2/15)... 2024-09-20T21:57:43.536 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:43 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:43.060+0000 7fe11c25f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-20T21:57:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:43 smithi060 ceph-mon[29694]: Activating manager daemon y 2024-09-20T21:57:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:43 smithi060 ceph-mon[29694]: mgrmap e2: y(active, starting, since 0.0029267s) 2024-09-20T21:57:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:43 smithi060 ceph-mon[29694]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-20T21:57:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:43 smithi060 ceph-mon[29694]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-20T21:57:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:43 smithi060 ceph-mon[29694]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-20T21:57:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:43 smithi060 ceph-mon[29694]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T21:57:43.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:43 smithi060 ceph-mon[29694]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-20T21:57:43.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:43 smithi060 ceph-mon[29694]: Manager daemon y is now available 2024-09-20T21:57:43.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:43 smithi060 ceph-mon[29694]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-20T21:57:43.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:43 smithi060 ceph-mon[29694]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-20T21:57:43.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:43 smithi060 ceph-mon[29694]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' 2024-09-20T21:57:43.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:43 smithi060 ceph-mon[29694]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' 2024-09-20T21:57:43.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:43 smithi060 ceph-mon[29694]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' 2024-09-20T21:57:45.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:45 smithi060 ceph-mon[29694]: mgrmap e3: y(active, since 1.00747s) 2024-09-20T21:57:46.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:46 smithi060 ceph-mon[29694]: mgrmap e4: y(active, since 2s) 2024-09-20T21:57:46.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:46 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3767243270' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-20T21:57:52.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-20T21:57:52.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-09-20T21:57:52.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsid": "3df9647e-779b-11ef-baf5-efdc52797490", 2024-09-20T21:57:52.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "health": { 2024-09-20T21:57:52.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-20T21:57:52.069 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-20T21:57:52.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-20T21:57:52.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:52.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-20T21:57:52.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-20T21:57:52.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 0 2024-09-20T21:57:52.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-09-20T21:57:52.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-20T21:57:52.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "a" 2024-09-20T21:57:52.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-09-20T21:57:52.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-20T21:57:52.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-20T21:57:52.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T21:57:52.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-20T21:57:52.070 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-20T21:57:52.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:52.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-20T21:57:52.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T21:57:52.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-20T21:57:52.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-20T21:57:52.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-20T21:57:52.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-20T21:57:52.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-20T21:57:52.071 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-20T21:57:52.074 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:52.074 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-20T21:57:52.074 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-20T21:57:52.075 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-20T21:57:52.075 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-20T21:57:52.075 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-20T21:57:52.075 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-20T21:57:52.075 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-20T21:57:52.075 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-20T21:57:52.075 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-20T21:57:52.075 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:52.075 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-20T21:57:52.075 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T21:57:52.075 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "btime": "2024-09-20T21:57:31:244773+0000", 2024-09-20T21:57:52.075 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-20T21:57:52.075 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-20T21:57:52.076 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:52.076 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-20T21:57:52.076 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-20T21:57:52.076 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-20T21:57:52.076 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-20T21:57:52.076 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-20T21:57:52.076 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-20T21:57:52.076 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "restful" 2024-09-20T21:57:52.076 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ], 2024-09-20T21:57:52.076 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-20T21:57:52.076 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:52.076 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-20T21:57:52.076 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-20T21:57:52.076 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "modified": "2024-09-20T21:57:31.245398+0000", 2024-09-20T21:57:52.076 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-20T21:57:52.076 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout }, 2024-09-20T21:57:52.077 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-20T21:57:52.077 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-09-20T21:57:52.077 INFO:teuthology.orchestra.run.smithi060.stdout:mgr is available 2024-09-20T21:57:55.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:55 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1845697438' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-20T21:57:56.253 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-20T21:57:56.255 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [global] 2024-09-20T21:57:56.255 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout fsid = 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:57:56.255 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.60:3300,v1:172.21.15.60:6789] 2024-09-20T21:57:56.255 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-20T21:57:56.255 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-20T21:57:56.255 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-20T21:57:56.255 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-20T21:57:56.255 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-20T21:57:56.255 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-20T21:57:56.255 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-20T21:57:56.256 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 2024-09-20T21:57:56.256 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout [osd] 2024-09-20T21:57:56.256 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-20T21:57:56.256 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-20T21:57:56.256 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling cephadm module... 2024-09-20T21:57:57.347 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:57 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/851395620' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-20T21:57:57.348 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:57 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ignoring --setuser ceph since I am not root 2024-09-20T21:57:57.348 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:57 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ignoring --setgroup ceph since I am not root 2024-09-20T21:57:57.348 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:57 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:57.176+0000 7f9a3a516040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-20T21:57:57.348 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:57 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:57.235+0000 7f9a3a516040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-20T21:57:57.618 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:57 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:57.393+0000 7f9a3a516040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-20T21:57:57.618 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:57 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:57.463+0000 7f9a3a516040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-20T21:57:57.618 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:57 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:57.616+0000 7f9a3a516040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-20T21:57:57.959 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:57 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:57.769+0000 7f9a3a516040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-20T21:57:58.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:58 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/851395620' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-20T21:57:58.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:58 smithi060 ceph-mon[29694]: mgrmap e5: y(active, since 13s) 2024-09-20T21:57:58.278 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:58 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:58.118+0000 7f9a3a516040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-20T21:57:58.278 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:58 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:58.182+0000 7f9a3a516040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-20T21:57:58.536 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:58 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:58.386+0000 7f9a3a516040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-20T21:57:59.125 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-09-20T21:57:59.125 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-20T21:57:59.125 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-20T21:57:59.125 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-20T21:57:59.125 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-20T21:57:59.125 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-09-20T21:57:59.125 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for the mgr to restart... 2024-09-20T21:57:59.125 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr epoch 5... 2024-09-20T21:57:59.318 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:57:59 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/908676662' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-20T21:57:59.639 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:59 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:59.316+0000 7f9a3a516040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-20T21:57:59.639 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:57:59 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:57:59.389+0000 7f9a3a516040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-20T21:58:00.264 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:00 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:00.142+0000 7f9a3a516040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-20T21:58:00.264 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:00 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:00.262+0000 7f9a3a516040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-20T21:58:00.536 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:00 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:00.411+0000 7f9a3a516040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-20T21:58:00.536 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:00 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:00.475+0000 7f9a3a516040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-20T21:58:00.931 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:00 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:00.630+0000 7f9a3a516040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-20T21:58:00.931 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:00 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:00.701+0000 7f9a3a516040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-20T21:58:00.931 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:00 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:00.764+0000 7f9a3a516040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-20T21:58:00.932 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:00 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:00.929+0000 7f9a3a516040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-20T21:58:01.216 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:00 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:00.997+0000 7f9a3a516040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-20T21:58:01.216 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:01 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:01.066+0000 7f9a3a516040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-20T21:58:01.216 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:01 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:01.152+0000 7f9a3a516040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-20T21:58:01.216 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:01 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:01.214+0000 7f9a3a516040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-20T21:58:01.536 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:01 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:01.276+0000 7f9a3a516040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-20T21:58:02.536 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:02 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:02.103+0000 7f9a3a516040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-20T21:58:02.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:02 smithi060 ceph-mon[29694]: Active manager daemon y restarted 2024-09-20T21:58:02.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:02 smithi060 ceph-mon[29694]: Activating manager daemon y 2024-09-20T21:58:02.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:02 smithi060 ceph-mon[29694]: osdmap e2: 0 total, 0 up, 0 in 2024-09-20T21:58:02.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:02 smithi060 ceph-mon[29694]: mgrmap e6: y(active, starting, since 0.00332614s) 2024-09-20T21:58:02.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:02 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T21:58:02.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:02 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-20T21:58:02.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:02 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-20T21:58:02.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:02 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-20T21:58:02.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:02 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-20T21:58:02.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:02 smithi060 ceph-mon[29694]: Manager daemon y is now available 2024-09-20T21:58:02.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:02 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:02.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:02 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:02.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:02 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:58:02.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:02 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:58:02.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:02 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-20T21:58:02.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:02 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-20T21:58:03.471 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:03 smithi060 ceph-mon[29694]: Found migration_current of "None". Setting to last migration. 2024-09-20T21:58:03.471 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:03 smithi060 ceph-mon[29694]: mgrmap e7: y(active, since 1.00528s) 2024-09-20T21:58:03.697 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-09-20T21:58:03.697 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-20T21:58:03.698 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-20T21:58:03.698 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-09-20T21:58:03.698 INFO:teuthology.orchestra.run.smithi060.stdout:mgr epoch 5 is available 2024-09-20T21:58:03.698 INFO:teuthology.orchestra.run.smithi060.stdout:Setting orchestrator backend to cephadm... 2024-09-20T21:58:05.170 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:04 smithi060 ceph-mon[29694]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-20T21:58:05.171 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:04 smithi060 ceph-mon[29694]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-20T21:58:05.171 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:04 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:05.171 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:04 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:05.171 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:04 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:05.171 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:04 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:58:06.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-mon[29694]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:06.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-mon[29694]: mgrmap e8: y(active, since 2s) 2024-09-20T21:58:06.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:58:06.093 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-20T21:58:06.093 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:05.850+0000 7f99a1830640 -1 log_channel(cephadm) log [ERR] : [20/Sep/2024:21:58:05] ENGINE Error in HTTPServer.serve 2024-09-20T21:58:06.093 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: Traceback (most recent call last): 2024-09-20T21:58:06.093 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-20T21:58:06.093 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: self._connections.run(self.expiration_interval) 2024-09-20T21:58:06.093 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-20T21:58:06.093 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: self._run(expiration_interval) 2024-09-20T21:58:06.094 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-20T21:58:06.094 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: new_conn = self._from_server_socket(self.server.socket) 2024-09-20T21:58:06.094 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-20T21:58:06.094 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-20T21:58:06.094 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-20T21:58:06.094 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: s = self.context.wrap_socket( 2024-09-20T21:58:06.094 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-20T21:58:06.094 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: return self.sslsocket_class._create( 2024-09-20T21:58:06.094 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-20T21:58:06.094 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: self.do_handshake() 2024-09-20T21:58:06.094 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-20T21:58:06.094 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: self._sslobj.do_handshake() 2024-09-20T21:58:06.094 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-20T21:58:06.094 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:06.095 INFO:teuthology.orchestra.run.smithi060.stdout:Generating ssh key... 2024-09-20T21:58:07.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-mon[29694]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:07.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-mon[29694]: [20/Sep/2024:21:58:05] ENGINE Bus STARTING 2024-09-20T21:58:07.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-mon[29694]: [20/Sep/2024:21:58:05] ENGINE Serving on http://172.21.15.60:8765 2024-09-20T21:58:07.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-mon[29694]: [20/Sep/2024:21:58:05] ENGINE Serving on https://172.21.15.60:7150 2024-09-20T21:58:07.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-mon[29694]: [20/Sep/2024:21:58:05] ENGINE Bus STARTED 2024-09-20T21:58:07.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-mon[29694]: [20/Sep/2024:21:58:05] ENGINE Error in HTTPServer.serve 2024-09-20T21:58:07.287 INFO:journalctl@ceph.mon.a.smithi060.stdout: Traceback (most recent call last): 2024-09-20T21:58:07.287 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-20T21:58:07.287 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._connections.run(self.expiration_interval) 2024-09-20T21:58:07.287 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-20T21:58:07.287 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._run(expiration_interval) 2024-09-20T21:58:07.287 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-20T21:58:07.287 INFO:journalctl@ceph.mon.a.smithi060.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-20T21:58:07.287 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-20T21:58:07.287 INFO:journalctl@ceph.mon.a.smithi060.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-20T21:58:07.287 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-20T21:58:07.287 INFO:journalctl@ceph.mon.a.smithi060.stdout: s = self.context.wrap_socket( 2024-09-20T21:58:07.287 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-20T21:58:07.287 INFO:journalctl@ceph.mon.a.smithi060.stdout: return self.sslsocket_class._create( 2024-09-20T21:58:07.287 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-20T21:58:07.287 INFO:journalctl@ceph.mon.a.smithi060.stdout: self.do_handshake() 2024-09-20T21:58:07.288 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-20T21:58:07.288 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._sslobj.do_handshake() 2024-09-20T21:58:07.288 INFO:journalctl@ceph.mon.a.smithi060.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-20T21:58:07.288 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: Generating public/private rsa key pair. 2024-09-20T21:58:07.288 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: Your identification has been saved in /tmp/tmpi5u85qr8/key 2024-09-20T21:58:07.288 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: Your public key has been saved in /tmp/tmpi5u85qr8/key.pub 2024-09-20T21:58:07.288 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: The key fingerprint is: 2024-09-20T21:58:07.288 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: SHA256:9wn6YkoMqaKQ9xok33uKmoy03/YwY/SA5kAgxSepbbA ceph-3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:58:07.288 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: The key's randomart image is: 2024-09-20T21:58:07.288 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: +---[RSA 3072]----+ 2024-09-20T21:58:07.289 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: |oo.. | 2024-09-20T21:58:07.289 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: |+ + . | 2024-09-20T21:58:07.289 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: | * o | 2024-09-20T21:58:07.289 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: |E o .. | 2024-09-20T21:58:07.289 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: |.o.ooo S o | 2024-09-20T21:58:07.289 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: | =+o.oo o o . | 2024-09-20T21:58:07.289 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: |+.=..=o.. o | 2024-09-20T21:58:07.289 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: |*+.+o+= o. | 2024-09-20T21:58:07.289 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: |=++o=+o+ .. | 2024-09-20T21:58:07.289 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: +----[SHA256]-----+ 2024-09-20T21:58:08.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:07 smithi060 ceph-mon[29694]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:08.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:07 smithi060 ceph-mon[29694]: Generating ssh key... 2024-09-20T21:58:08.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:07 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:08.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:07 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:08.856 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCe93UaPk0UQX06zVm31CPkZ171ADu/1tZf+UVjGnRgT+iIVK1yOmGTLxukpcKPDxwmmGYvuJOFU4GZdOLrqUavji9JQYdIWJ6p/0iI/xDQ03BMF1J/FAHiOYlCxWAIJNbcqHlM+Xo9Mbksw1cx6f/8PyNSPonLxEJ+nWpG/Q1720O6jQn8F20k4j3dNJ/ZfLOT4LzGsyXm7lpplCp391JzVPBtzSMsbya8DMtONAqIfVhKCrdcbn59DXBaT/ZJ/2Wz/5KfWwkjcLtg4wfLl8i2p6EEUPW9Vq34cWZsERTPwJNjp/l5lRHn4SUQOm9x1HFb45vrA5i/E7dB6nctdhnGbdOJjWApcIpWPAnUPmQfcggKk+HVIWe7BaA/gn758jWpBzZxV93/9AtUKDjasg6y7GqFyao1vtyEspyFAA4fykqjvv7168q5Nu+EZ8SQuRK4rNM0c4pW5xl0hXrc+OpHsveHP68Upi4lrLTTEbtS63CbTeq/CHssT8gsTenguRc= ceph-3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:58:08.856 INFO:teuthology.orchestra.run.smithi060.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-20T21:58:08.857 INFO:teuthology.orchestra.run.smithi060.stdout:Adding key to root@localhost authorized_keys... 2024-09-20T21:58:08.861 INFO:teuthology.orchestra.run.smithi060.stdout:Adding host smithi060... 2024-09-20T21:58:09.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:09 smithi060 ceph-mon[29694]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:10.208 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:10 smithi060 ceph-mon[29694]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi060", "addr": "172.21.15.60", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:11.430 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:11 smithi060 ceph-mon[29694]: Deploying cephadm binary to smithi060 2024-09-20T21:58:13.939 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Added host 'smithi060' with addr '172.21.15.60' 2024-09-20T21:58:13.939 INFO:teuthology.orchestra.run.smithi060.stdout:Deploying unmanaged mon service... 2024-09-20T21:58:14.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:14 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:14.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:14 smithi060 ceph-mon[29694]: Added host smithi060 2024-09-20T21:58:14.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:14 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:58:15.331 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-20T21:58:15.331 INFO:teuthology.orchestra.run.smithi060.stdout:Deploying unmanaged mgr service... 2024-09-20T21:58:15.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:15 smithi060 ceph-mon[29694]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:15.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:15 smithi060 ceph-mon[29694]: Saving service mon spec with placement count:5 2024-09-20T21:58:15.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:15 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:16.882 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-20T21:58:17.267 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:17 smithi060 ceph-mon[29694]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:17.267 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:17 smithi060 ceph-mon[29694]: Saving service mgr spec with placement count:2 2024-09-20T21:58:17.268 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:17 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:18.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:18 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2194018915' entity='client.admin' 2024-09-20T21:58:18.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:18 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:18.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:18 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:19.556 INFO:teuthology.orchestra.run.smithi060.stdout:Enabling the dashboard module... 2024-09-20T21:58:20.259 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:19 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1451485985' entity='client.admin' 2024-09-20T21:58:21.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:21 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:21.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:21 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2048872448' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-20T21:58:21.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:21 smithi060 ceph-mon[29694]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:21.787 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ignoring --setuser ceph since I am not root 2024-09-20T21:58:21.787 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ignoring --setgroup ceph since I am not root 2024-09-20T21:58:21.787 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:21.577+0000 7f958b473040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-20T21:58:21.787 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:21.640+0000 7f958b473040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-20T21:58:22.182 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:21.803+0000 7f958b473040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-20T21:58:22.182 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:21.872+0000 7f958b473040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-20T21:58:22.182 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:22 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:22.030+0000 7f958b473040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-20T21:58:22.182 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:22 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:22.180+0000 7f958b473040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-20T21:58:22.467 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:22 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2048872448' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-20T21:58:22.467 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:22 smithi060 ceph-mon[29694]: mgrmap e9: y(active, since 19s) 2024-09-20T21:58:22.746 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:22 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:22.522+0000 7f958b473040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-20T21:58:22.747 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:22 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:22.586+0000 7f958b473040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-20T21:58:23.036 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:22 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:22.789+0000 7f958b473040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-20T21:58:23.889 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:23 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3726089641' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-20T21:58:23.889 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:23 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:23.737+0000 7f958b473040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-20T21:58:23.889 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:23 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:23.809+0000 7f958b473040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-20T21:58:24.222 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-09-20T21:58:24.222 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-20T21:58:24.222 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-20T21:58:24.223 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-20T21:58:24.223 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-20T21:58:24.223 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-09-20T21:58:24.223 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for the mgr to restart... 2024-09-20T21:58:24.223 INFO:teuthology.orchestra.run.smithi060.stdout:Waiting for mgr epoch 9... 2024-09-20T21:58:24.749 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:24 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:24.621+0000 7f958b473040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-20T21:58:25.036 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:24 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:24.747+0000 7f958b473040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-20T21:58:25.036 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:24 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:24.899+0000 7f958b473040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-20T21:58:25.036 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:24 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:24.965+0000 7f958b473040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-20T21:58:25.426 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:25.123+0000 7f958b473040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-20T21:58:25.426 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:25.194+0000 7f958b473040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-20T21:58:25.427 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:25.258+0000 7f958b473040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-20T21:58:25.427 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:25.424+0000 7f958b473040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-20T21:58:25.714 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:25.493+0000 7f958b473040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-20T21:58:25.714 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:25.563+0000 7f958b473040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-20T21:58:25.714 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:25.650+0000 7f958b473040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-20T21:58:25.714 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:25.712+0000 7f958b473040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-20T21:58:26.036 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:25.774+0000 7f958b473040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-20T21:58:26.864 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:26 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:26.611+0000 7f958b473040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-20T21:58:26.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:26 smithi060 ceph-mon[29694]: Active manager daemon y restarted 2024-09-20T21:58:26.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:26 smithi060 ceph-mon[29694]: Activating manager daemon y 2024-09-20T21:58:26.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:26 smithi060 ceph-mon[29694]: osdmap e3: 0 total, 0 up, 0 in 2024-09-20T21:58:26.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:26 smithi060 ceph-mon[29694]: mgrmap e10: y(active, starting, since 0.00318736s) 2024-09-20T21:58:26.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:26 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T21:58:26.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:26 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-20T21:58:26.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:26 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-20T21:58:26.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:26 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-20T21:58:26.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:26 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-20T21:58:26.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:26 smithi060 ceph-mon[29694]: Manager daemon y is now available 2024-09-20T21:58:26.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:26 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:58:26.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:26 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-20T21:58:26.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:26 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-20T21:58:28.563 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:28.502+0000 7f94f0f3c640 -1 log_channel(cephadm) log [ERR] : [20/Sep/2024:21:58:28] ENGINE Error in HTTPServer.serve 2024-09-20T21:58:28.563 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: Traceback (most recent call last): 2024-09-20T21:58:28.563 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-20T21:58:28.563 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: self._connections.run(self.expiration_interval) 2024-09-20T21:58:28.563 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-20T21:58:28.563 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: self._run(expiration_interval) 2024-09-20T21:58:28.563 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-20T21:58:28.563 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: new_conn = self._from_server_socket(self.server.socket) 2024-09-20T21:58:28.564 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-20T21:58:28.564 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-20T21:58:28.564 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-20T21:58:28.564 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: s = self.context.wrap_socket( 2024-09-20T21:58:28.564 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-20T21:58:28.564 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: return self.sslsocket_class._create( 2024-09-20T21:58:28.564 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-20T21:58:28.564 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: self.do_handshake() 2024-09-20T21:58:28.564 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-20T21:58:28.564 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: self._sslobj.do_handshake() 2024-09-20T21:58:28.564 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-20T21:58:28.564 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T21:58:28.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout { 2024-09-20T21:58:28.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-20T21:58:28.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-20T21:58:28.565 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout } 2024-09-20T21:58:28.565 INFO:teuthology.orchestra.run.smithi060.stdout:mgr epoch 9 is available 2024-09-20T21:58:28.565 INFO:teuthology.orchestra.run.smithi060.stdout:Generating a dashboard self-signed certificate... 2024-09-20T21:58:28.815 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-mon[29694]: mgrmap e11: y(active, since 1.0061s) 2024-09-20T21:58:28.815 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:28.815 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-mon[29694]: [20/Sep/2024:21:58:28] ENGINE Bus STARTING 2024-09-20T21:58:28.815 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:28 smithi060 ceph-mon[29694]: [20/Sep/2024:21:58:28] ENGINE Serving on http://172.21.15.60:8765 2024-09-20T21:58:29.904 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-20T21:58:29.904 INFO:teuthology.orchestra.run.smithi060.stdout:Creating initial admin user... 2024-09-20T21:58:30.154 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:29 smithi060 ceph-mon[29694]: [20/Sep/2024:21:58:28] ENGINE Serving on https://172.21.15.60:7150 2024-09-20T21:58:30.154 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:29 smithi060 ceph-mon[29694]: [20/Sep/2024:21:58:28] ENGINE Bus STARTED 2024-09-20T21:58:30.154 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:29 smithi060 ceph-mon[29694]: [20/Sep/2024:21:58:28] ENGINE Error in HTTPServer.serve 2024-09-20T21:58:30.154 INFO:journalctl@ceph.mon.a.smithi060.stdout: Traceback (most recent call last): 2024-09-20T21:58:30.154 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-20T21:58:30.154 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._connections.run(self.expiration_interval) 2024-09-20T21:58:30.154 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-20T21:58:30.154 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._run(expiration_interval) 2024-09-20T21:58:30.154 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-20T21:58:30.155 INFO:journalctl@ceph.mon.a.smithi060.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-20T21:58:30.155 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-20T21:58:30.155 INFO:journalctl@ceph.mon.a.smithi060.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-20T21:58:30.155 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-20T21:58:30.155 INFO:journalctl@ceph.mon.a.smithi060.stdout: s = self.context.wrap_socket( 2024-09-20T21:58:30.155 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-20T21:58:30.155 INFO:journalctl@ceph.mon.a.smithi060.stdout: return self.sslsocket_class._create( 2024-09-20T21:58:30.155 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-20T21:58:30.155 INFO:journalctl@ceph.mon.a.smithi060.stdout: self.do_handshake() 2024-09-20T21:58:30.155 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-20T21:58:30.155 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._sslobj.do_handshake() 2024-09-20T21:58:30.155 INFO:journalctl@ceph.mon.a.smithi060.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-20T21:58:30.155 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:29 smithi060 ceph-mon[29694]: mgrmap e12: y(active, since 2s) 2024-09-20T21:58:30.155 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:29 smithi060 ceph-mon[29694]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:30.156 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:29 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:30.156 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:29 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:30.156 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:29 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:31.474 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$6Utsa4BRvZUygJ1/QFeRie97CxYMc747MDw9xQv.Rw4hNkQearvg2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726869510, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-20T21:58:31.474 INFO:teuthology.orchestra.run.smithi060.stdout:Fetching dashboard port number... 2024-09-20T21:58:32.236 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:31 smithi060 ceph-mon[29694]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:32.236 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:31 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:33.132 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stdout 8443 2024-09-20T21:58:33.144 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-20T21:58:33.144 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout disabled 2024-09-20T21:58:33.153 INFO:teuthology.orchestra.run.smithi060.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-20T21:58:33.153 INFO:teuthology.orchestra.run.smithi060.stdout:systemctl: stdout inactive 2024-09-20T21:58:33.153 INFO:teuthology.orchestra.run.smithi060.stdout:firewalld.service is not enabled 2024-09-20T21:58:33.153 INFO:teuthology.orchestra.run.smithi060.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-20T21:58:33.156 INFO:teuthology.orchestra.run.smithi060.stdout:Ceph Dashboard is now available at: 2024-09-20T21:58:33.156 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:58:33.156 INFO:teuthology.orchestra.run.smithi060.stdout: URL: https://smithi060.front.sepia.ceph.com:8443/ 2024-09-20T21:58:33.156 INFO:teuthology.orchestra.run.smithi060.stdout: User: admin 2024-09-20T21:58:33.156 INFO:teuthology.orchestra.run.smithi060.stdout: Password: bis6rahtqh 2024-09-20T21:58:33.156 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:58:33.156 INFO:teuthology.orchestra.run.smithi060.stdout:Saving cluster configuration to /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config directory 2024-09-20T21:58:33.281 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:32 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/675219104' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-20T21:58:34.201 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:33 smithi060 ceph-mon[29694]: mgrmap e13: y(active, since 6s) 2024-09-20T21:58:34.201 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:33 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/665336590' entity='client.admin' 2024-09-20T21:58:34.488 INFO:teuthology.orchestra.run.smithi060.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-20T21:58:34.489 INFO:teuthology.orchestra.run.smithi060.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-20T21:58:34.489 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:58:34.489 INFO:teuthology.orchestra.run.smithi060.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-20T21:58:34.489 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:58:34.489 INFO:teuthology.orchestra.run.smithi060.stdout:Or, if you are only running a single cluster on this host: 2024-09-20T21:58:34.489 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:58:34.489 INFO:teuthology.orchestra.run.smithi060.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-20T21:58:34.489 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:58:34.489 INFO:teuthology.orchestra.run.smithi060.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-20T21:58:34.489 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:58:34.489 INFO:teuthology.orchestra.run.smithi060.stdout: ceph telemetry on 2024-09-20T21:58:34.489 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:58:34.490 INFO:teuthology.orchestra.run.smithi060.stdout:For more information see: 2024-09-20T21:58:34.490 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:58:34.490 INFO:teuthology.orchestra.run.smithi060.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-20T21:58:34.490 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:58:34.490 INFO:teuthology.orchestra.run.smithi060.stdout:Bootstrap complete. 2024-09-20T21:58:34.530 INFO:tasks.cephadm:Fetching config... 2024-09-20T21:58:34.530 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-20T21:58:34.530 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-20T21:58:34.547 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-20T21:58:34.547 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-20T21:58:34.547 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-20T21:58:34.603 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-20T21:58:34.604 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-20T21:58:34.604 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-20T21:58:34.669 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-20T21:58:34.669 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-20T21:58:34.669 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-20T21:58:34.726 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-20T21:58:34.726 DEBUG:teuthology.orchestra.run.smithi060:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCe93UaPk0UQX06zVm31CPkZ171ADu/1tZf+UVjGnRgT+iIVK1yOmGTLxukpcKPDxwmmGYvuJOFU4GZdOLrqUavji9JQYdIWJ6p/0iI/xDQ03BMF1J/FAHiOYlCxWAIJNbcqHlM+Xo9Mbksw1cx6f/8PyNSPonLxEJ+nWpG/Q1720O6jQn8F20k4j3dNJ/ZfLOT4LzGsyXm7lpplCp391JzVPBtzSMsbya8DMtONAqIfVhKCrdcbn59DXBaT/ZJ/2Wz/5KfWwkjcLtg4wfLl8i2p6EEUPW9Vq34cWZsERTPwJNjp/l5lRHn4SUQOm9x1HFb45vrA5i/E7dB6nctdhnGbdOJjWApcIpWPAnUPmQfcggKk+HVIWe7BaA/gn758jWpBzZxV93/9AtUKDjasg6y7GqFyao1vtyEspyFAA4fykqjvv7168q5Nu+EZ8SQuRK4rNM0c4pW5xl0hXrc+OpHsveHP68Upi4lrLTTEbtS63CbTeq/CHssT8gsTenguRc= ceph-3df9647e-779b-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-20T21:58:34.811 INFO:teuthology.orchestra.run.smithi060.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCe93UaPk0UQX06zVm31CPkZ171ADu/1tZf+UVjGnRgT+iIVK1yOmGTLxukpcKPDxwmmGYvuJOFU4GZdOLrqUavji9JQYdIWJ6p/0iI/xDQ03BMF1J/FAHiOYlCxWAIJNbcqHlM+Xo9Mbksw1cx6f/8PyNSPonLxEJ+nWpG/Q1720O6jQn8F20k4j3dNJ/ZfLOT4LzGsyXm7lpplCp391JzVPBtzSMsbya8DMtONAqIfVhKCrdcbn59DXBaT/ZJ/2Wz/5KfWwkjcLtg4wfLl8i2p6EEUPW9Vq34cWZsERTPwJNjp/l5lRHn4SUQOm9x1HFb45vrA5i/E7dB6nctdhnGbdOJjWApcIpWPAnUPmQfcggKk+HVIWe7BaA/gn758jWpBzZxV93/9AtUKDjasg6y7GqFyao1vtyEspyFAA4fykqjvv7168q5Nu+EZ8SQuRK4rNM0c4pW5xl0hXrc+OpHsveHP68Upi4lrLTTEbtS63CbTeq/CHssT8gsTenguRc= ceph-3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:58:34.823 DEBUG:teuthology.orchestra.run.smithi067:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCe93UaPk0UQX06zVm31CPkZ171ADu/1tZf+UVjGnRgT+iIVK1yOmGTLxukpcKPDxwmmGYvuJOFU4GZdOLrqUavji9JQYdIWJ6p/0iI/xDQ03BMF1J/FAHiOYlCxWAIJNbcqHlM+Xo9Mbksw1cx6f/8PyNSPonLxEJ+nWpG/Q1720O6jQn8F20k4j3dNJ/ZfLOT4LzGsyXm7lpplCp391JzVPBtzSMsbya8DMtONAqIfVhKCrdcbn59DXBaT/ZJ/2Wz/5KfWwkjcLtg4wfLl8i2p6EEUPW9Vq34cWZsERTPwJNjp/l5lRHn4SUQOm9x1HFb45vrA5i/E7dB6nctdhnGbdOJjWApcIpWPAnUPmQfcggKk+HVIWe7BaA/gn758jWpBzZxV93/9AtUKDjasg6y7GqFyao1vtyEspyFAA4fykqjvv7168q5Nu+EZ8SQuRK4rNM0c4pW5xl0hXrc+OpHsveHP68Upi4lrLTTEbtS63CbTeq/CHssT8gsTenguRc= ceph-3df9647e-779b-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-20T21:58:34.872 INFO:teuthology.orchestra.run.smithi067.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCe93UaPk0UQX06zVm31CPkZ171ADu/1tZf+UVjGnRgT+iIVK1yOmGTLxukpcKPDxwmmGYvuJOFU4GZdOLrqUavji9JQYdIWJ6p/0iI/xDQ03BMF1J/FAHiOYlCxWAIJNbcqHlM+Xo9Mbksw1cx6f/8PyNSPonLxEJ+nWpG/Q1720O6jQn8F20k4j3dNJ/ZfLOT4LzGsyXm7lpplCp391JzVPBtzSMsbya8DMtONAqIfVhKCrdcbn59DXBaT/ZJ/2Wz/5KfWwkjcLtg4wfLl8i2p6EEUPW9Vq34cWZsERTPwJNjp/l5lRHn4SUQOm9x1HFb45vrA5i/E7dB6nctdhnGbdOJjWApcIpWPAnUPmQfcggKk+HVIWe7BaA/gn758jWpBzZxV93/9AtUKDjasg6y7GqFyao1vtyEspyFAA4fykqjvv7168q5Nu+EZ8SQuRK4rNM0c4pW5xl0hXrc+OpHsveHP68Upi4lrLTTEbtS63CbTeq/CHssT8gsTenguRc= ceph-3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:58:34.885 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-20T21:58:35.141 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T21:58:36.922 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-20T21:58:36.923 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-20T21:58:37.202 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:36 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2355587091' entity='client.admin' 2024-09-20T21:58:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T21:58:39.130 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi067 2024-09-20T21:58:39.130 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-20T21:58:39.130 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/etc/ceph/ceph.conf 2024-09-20T21:58:39.151 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-20T21:58:39.151 DEBUG:teuthology.orchestra.run.smithi067:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-20T21:58:39.210 INFO:tasks.cephadm:Adding host smithi067 to orchestrator... 2024-09-20T21:58:39.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch host add smithi067 2024-09-20T21:58:39.476 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T21:58:39.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:39 smithi060 ceph-mon[29694]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:39.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:39 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:39.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:39 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:39.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:39 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:39.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:39 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi060", "name": "osd_memory_target"}]: dispatch 2024-09-20T21:58:39.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:39 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:58:39.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:39 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:58:40.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:40 smithi060 ceph-mon[29694]: Updating smithi060:/etc/ceph/ceph.conf 2024-09-20T21:58:40.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:40 smithi060 ceph-mon[29694]: Updating smithi060:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T21:58:40.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:40 smithi060 ceph-mon[29694]: Updating smithi060:/etc/ceph/ceph.client.admin.keyring 2024-09-20T21:58:40.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:40 smithi060 ceph-mon[29694]: Updating smithi060:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T21:58:40.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:40 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:40.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:40 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:40.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:40 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:40.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:40 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:58:40.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:40 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:58:40.288 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:40 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:58:40.288 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:40 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:41.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:41 smithi060 ceph-mon[29694]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi067", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:42.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:42 smithi060 ceph-mon[29694]: Deploying cephadm binary to smithi067 2024-09-20T21:58:44.143 INFO:teuthology.orchestra.run.smithi060.stdout:Added host 'smithi067' with addr '172.21.15.67' 2024-09-20T21:58:44.736 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-20T21:58:44.990 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T21:58:45.383 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:45 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:45.383 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:45 smithi060 ceph-mon[29694]: Added host smithi067 2024-09-20T21:58:45.384 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:45 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:58:45.384 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:45 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:45.384 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:45 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:45.669 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:58:45.669 INFO:teuthology.orchestra.run.smithi060.stdout:[{"addr": "172.21.15.60", "hostname": "smithi060", "labels": [], "status": ""}, {"addr": "172.21.15.67", "hostname": "smithi067", "labels": [], "status": ""}] 2024-09-20T21:58:46.199 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-20T21:58:46.199 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-20T21:58:46.446 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T21:58:46.782 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:46 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:46.782 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:46 smithi060 ceph-mon[29694]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T21:58:47.625 INFO:teuthology.orchestra.run.smithi060.stderr:adjusted tunables profile to default 2024-09-20T21:58:47.926 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:47 smithi060 ceph-mon[29694]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:58:47.926 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:47 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3199120284' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-20T21:58:48.218 INFO:tasks.cephadm:Adding mon.a on smithi060 2024-09-20T21:58:48.219 INFO:tasks.cephadm:Adding mon.c on smithi060 2024-09-20T21:58:48.219 INFO:tasks.cephadm:Adding mon.b on smithi067 2024-09-20T21:58:48.219 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi060:172.21.15.60=a;smithi060:[v2:172.21.15.60:3301,v1:172.21.15.60:6790]=c;smithi067:172.21.15.67=b' 2024-09-20T21:58:48.468 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-20T21:58:48.516 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-20T21:58:49.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:48 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3199120284' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-20T21:58:49.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:48 smithi060 ceph-mon[29694]: osdmap e4: 0 total, 0 up, 0 in 2024-09-20T21:58:49.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:48 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:49.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:48 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:49.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:48 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:49.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:48 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:49.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:48 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"}]: dispatch 2024-09-20T21:58:49.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:48 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:58:49.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:48 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:58:49.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:48 smithi060 ceph-mon[29694]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-20T21:58:49.333 INFO:teuthology.orchestra.run.smithi067.stdout:Scheduled mon update... 2024-09-20T21:58:49.880 DEBUG:teuthology.orchestra.run.smithi060:mon.c> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@mon.c.service 2024-09-20T21:58:49.883 DEBUG:teuthology.orchestra.run.smithi067:mon.b> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@mon.b.service 2024-09-20T21:58:49.887 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-20T21:58:49.887 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-20T21:58:50.642 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T21:58:50.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:50 smithi060 ceph-mon[29694]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:58:50.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:50 smithi060 ceph-mon[29694]: Updating smithi067:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T21:58:50.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:50 smithi060 ceph-mon[29694]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-09-20T21:58:50.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:50 smithi060 ceph-mon[29694]: Updating smithi067:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T21:58:50.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:50 smithi060 ceph-mon[29694]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi060:172.21.15.60=a;smithi060:[v2:172.21.15.60:3301,v1:172.21.15.60:6790]=c;smithi067:172.21.15.67=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:50.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:50 smithi060 ceph-mon[29694]: Saving service mon spec with placement smithi060:172.21.15.60=a;smithi060:[v2:172.21.15.60:3301,v1:172.21.15.60:6790]=c;smithi067:172.21.15.67=b;count:3 2024-09-20T21:58:50.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:50 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:50.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:50 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:50.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:50 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:50.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:50 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:50.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:50 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-20T21:58:50.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:50 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:58:50.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:50 smithi060 ceph-mon[29694]: Deploying daemon mon.b on smithi067 2024-09-20T21:58:51.411 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:58:51.411 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":1,"fsid":"3df9647e-779b-11ef-baf5-efdc52797490","modified":"2024-09-20T21:57:28.205024Z","created":"2024-09-20T21:57:28.205024Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-20T21:58:51.412 INFO:teuthology.orchestra.run.smithi067.stderr:dumped monmap epoch 1 2024-09-20T21:58:52.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:52 smithi060 ceph-mon[29694]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:58:52.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:52 smithi060 ceph-mon[29694]: from='client.? 172.21.15.67:0/4248178580' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-20T21:58:53.039 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-20T21:58:53.039 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-20T21:58:53.471 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T21:58:54.342 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:54 smithi067 systemd[1]: Starting Ceph mon.b for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T21:58:54.390 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:58:54.390 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":1,"fsid":"3df9647e-779b-11ef-baf5-efdc52797490","modified":"2024-09-20T21:57:28.205024Z","created":"2024-09-20T21:57:28.205024Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-20T21:58:54.390 INFO:teuthology.orchestra.run.smithi067.stderr:dumped monmap epoch 1 2024-09-20T21:58:54.689 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:54 smithi067 podman[33452]: 2024-09-20 21:58:54.243726148 +0000 UTC m=+0.020480877 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5 2024-09-20T21:58:54.689 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:54 smithi067 podman[33452]: 2024-09-20 21:58:54.341543801 +0000 UTC m=+0.118298516 container create 497f6b558d69936edda24dcbdf173b5a215e5bfb44eb5a3cb4f818dea5aa7ab5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-b, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-9e2ac70, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-20T21:58:54.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:58:54 smithi060 ceph-mon[29694]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:58:55.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 podman[33452]: 2024-09-20 21:58:55.047596092 +0000 UTC m=+0.824350811 container init 497f6b558d69936edda24dcbdf173b5a215e5bfb44eb5a3cb4f818dea5aa7ab5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-b, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-9e2ac70, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-20T21:58:55.340 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 podman[33452]: 2024-09-20 21:58:55.052857593 +0000 UTC m=+0.829612303 container start 497f6b558d69936edda24dcbdf173b5a215e5bfb44eb5a3cb4f818dea5aa7ab5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-b, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-9e2ac70, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-20T21:58:55.340 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: set uid:gid to 167:167 (ceph:ceph) 2024-09-20T21:58:55.340 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: ceph version 19.1.1-367-g9e2ac70b (9e2ac70bb7aea67d34f0d50002344abbe7803736) squid (stable), process ceph-mon, pid 7 2024-09-20T21:58:55.340 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: pidfile_write: ignore empty --pid-file 2024-09-20T21:58:55.341 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: load: jerasure load: lrc 2024-09-20T21:58:55.341 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: RocksDB version: 7.9.2 2024-09-20T21:58:55.341 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Git sha 0 2024-09-20T21:58:55.341 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Compile date 2024-09-19 18:20:48 2024-09-20T21:58:55.341 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: DB SUMMARY 2024-09-20T21:58:55.341 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: DB Session ID: 7IZN4AF4KPL5H0BAJDR6 2024-09-20T21:58:55.341 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: CURRENT file: CURRENT 2024-09-20T21:58:55.341 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: IDENTITY file: IDENTITY 2024-09-20T21:58:55.341 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-20T21:58:55.342 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-20T21:58:55.342 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-20T21:58:55.342 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.error_if_exists: 0 2024-09-20T21:58:55.342 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.create_if_missing: 0 2024-09-20T21:58:55.342 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.paranoid_checks: 1 2024-09-20T21:58:55.342 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-20T21:58:55.342 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-20T21:58:55.343 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-20T21:58:55.343 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.env: 0x55b82ff56940 2024-09-20T21:58:55.343 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.fs: PosixFileSystem 2024-09-20T21:58:55.343 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.info_log: 0x55b8314776e0 2024-09-20T21:58:55.343 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_file_opening_threads: 16 2024-09-20T21:58:55.343 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.statistics: (nil) 2024-09-20T21:58:55.343 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.use_fsync: 0 2024-09-20T21:58:55.343 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_log_file_size: 0 2024-09-20T21:58:55.343 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-20T21:58:55.344 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-20T21:58:55.344 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.keep_log_file_num: 1000 2024-09-20T21:58:55.344 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.recycle_log_file_num: 0 2024-09-20T21:58:55.344 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.allow_fallocate: 1 2024-09-20T21:58:55.344 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.allow_mmap_reads: 0 2024-09-20T21:58:55.344 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.allow_mmap_writes: 0 2024-09-20T21:58:55.344 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.use_direct_reads: 0 2024-09-20T21:58:55.344 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-20T21:58:55.344 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.create_missing_column_families: 0 2024-09-20T21:58:55.344 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.db_log_dir: 2024-09-20T21:58:55.345 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.wal_dir: 2024-09-20T21:58:55.345 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-20T21:58:55.345 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-20T21:58:55.345 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-20T21:58:55.345 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-20T21:58:55.345 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-20T21:58:55.345 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-20T21:58:55.345 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.advise_random_on_open: 1 2024-09-20T21:58:55.345 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.db_write_buffer_size: 0 2024-09-20T21:58:55.346 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.write_buffer_manager: 0x55b831486dc0 2024-09-20T21:58:55.346 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-20T21:58:55.346 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-20T21:58:55.346 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-20T21:58:55.346 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.rate_limiter: (nil) 2024-09-20T21:58:55.346 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-20T21:58:55.346 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.wal_recovery_mode: 2 2024-09-20T21:58:55.346 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.enable_thread_tracking: 0 2024-09-20T21:58:55.347 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.enable_pipelined_write: 0 2024-09-20T21:58:55.347 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.unordered_write: 0 2024-09-20T21:58:55.347 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-20T21:58:55.347 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-20T21:58:55.347 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-20T21:58:55.347 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-20T21:58:55.347 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.row_cache: None 2024-09-20T21:58:55.347 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.wal_filter: None 2024-09-20T21:58:55.351 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-20T21:58:55.351 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.allow_ingest_behind: 0 2024-09-20T21:58:55.351 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.two_write_queues: 0 2024-09-20T21:58:55.351 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.manual_wal_flush: 0 2024-09-20T21:58:55.351 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.wal_compression: 0 2024-09-20T21:58:55.351 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.atomic_flush: 0 2024-09-20T21:58:55.351 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-20T21:58:55.351 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-20T21:58:55.352 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-20T21:58:55.352 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.log_readahead_size: 0 2024-09-20T21:58:55.352 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-20T21:58:55.352 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.best_efforts_recovery: 0 2024-09-20T21:58:55.352 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-20T21:58:55.352 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-20T21:58:55.352 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.allow_data_in_errors: 0 2024-09-20T21:58:55.352 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.db_host_id: __hostname__ 2024-09-20T21:58:55.353 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-20T21:58:55.353 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_background_jobs: 2 2024-09-20T21:58:55.353 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_background_compactions: -1 2024-09-20T21:58:55.353 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_subcompactions: 1 2024-09-20T21:58:55.353 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-20T21:58:55.353 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-20T21:58:55.353 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-20T21:58:55.353 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_total_wal_size: 0 2024-09-20T21:58:55.353 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-20T21:58:55.353 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-20T21:58:55.354 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-20T21:58:55.354 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-20T21:58:55.354 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_open_files: -1 2024-09-20T21:58:55.354 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.bytes_per_sync: 0 2024-09-20T21:58:55.354 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-20T21:58:55.354 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-20T21:58:55.354 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compaction_readahead_size: 0 2024-09-20T21:58:55.354 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_background_flushes: -1 2024-09-20T21:58:55.354 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Compression algorithms supported: 2024-09-20T21:58:55.354 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: kZSTD supported: 0 2024-09-20T21:58:55.355 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: kXpressCompression supported: 0 2024-09-20T21:58:55.355 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: kBZip2Compression supported: 0 2024-09-20T21:58:55.355 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-20T21:58:55.355 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: kLZ4Compression supported: 1 2024-09-20T21:58:55.355 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: kZlibCompression supported: 1 2024-09-20T21:58:55.355 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: kLZ4HCCompression supported: 1 2024-09-20T21:58:55.355 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: kSnappyCompression supported: 1 2024-09-20T21:58:55.355 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-20T21:58:55.355 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-20T21:58:55.355 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2024-09-20T21:58:55.356 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-20T21:58:55.356 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-20T21:58:55.356 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.merge_operator: 2024-09-20T21:58:55.356 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compaction_filter: None 2024-09-20T21:58:55.356 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compaction_filter_factory: None 2024-09-20T21:58:55.356 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.sst_partitioner_factory: None 2024-09-20T21:58:55.356 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-20T21:58:55.356 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-20T21:58:55.356 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55b831477320) 2024-09-20T21:58:55.357 INFO:journalctl@ceph.mon.b.smithi067.stdout: cache_index_and_filter_blocks: 1 2024-09-20T21:58:55.357 INFO:journalctl@ceph.mon.b.smithi067.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-20T21:58:55.357 INFO:journalctl@ceph.mon.b.smithi067.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-20T21:58:55.357 INFO:journalctl@ceph.mon.b.smithi067.stdout: pin_top_level_index_and_filter: 1 2024-09-20T21:58:55.357 INFO:journalctl@ceph.mon.b.smithi067.stdout: index_type: 0 2024-09-20T21:58:55.357 INFO:journalctl@ceph.mon.b.smithi067.stdout: data_block_index_type: 0 2024-09-20T21:58:55.357 INFO:journalctl@ceph.mon.b.smithi067.stdout: index_shortening: 1 2024-09-20T21:58:55.357 INFO:journalctl@ceph.mon.b.smithi067.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-20T21:58:55.358 INFO:journalctl@ceph.mon.b.smithi067.stdout: checksum: 4 2024-09-20T21:58:55.358 INFO:journalctl@ceph.mon.b.smithi067.stdout: no_block_cache: 0 2024-09-20T21:58:55.358 INFO:journalctl@ceph.mon.b.smithi067.stdout: block_cache: 0x55b83146b1f0 2024-09-20T21:58:55.358 INFO:journalctl@ceph.mon.b.smithi067.stdout: block_cache_name: BinnedLRUCache 2024-09-20T21:58:55.358 INFO:journalctl@ceph.mon.b.smithi067.stdout: block_cache_options: 2024-09-20T21:58:55.358 INFO:journalctl@ceph.mon.b.smithi067.stdout: capacity : 536870912 2024-09-20T21:58:55.358 INFO:journalctl@ceph.mon.b.smithi067.stdout: num_shard_bits : 4 2024-09-20T21:58:55.358 INFO:journalctl@ceph.mon.b.smithi067.stdout: strict_capacity_limit : 0 2024-09-20T21:58:55.359 INFO:journalctl@ceph.mon.b.smithi067.stdout: high_pri_pool_ratio: 0.000 2024-09-20T21:58:55.359 INFO:journalctl@ceph.mon.b.smithi067.stdout: block_cache_compressed: (nil) 2024-09-20T21:58:55.359 INFO:journalctl@ceph.mon.b.smithi067.stdout: persistent_cache: (nil) 2024-09-20T21:58:55.359 INFO:journalctl@ceph.mon.b.smithi067.stdout: block_size: 4096 2024-09-20T21:58:55.359 INFO:journalctl@ceph.mon.b.smithi067.stdout: block_size_deviation: 10 2024-09-20T21:58:55.359 INFO:journalctl@ceph.mon.b.smithi067.stdout: block_restart_interval: 16 2024-09-20T21:58:55.359 INFO:journalctl@ceph.mon.b.smithi067.stdout: index_block_restart_interval: 1 2024-09-20T21:58:55.359 INFO:journalctl@ceph.mon.b.smithi067.stdout: metadata_block_size: 4096 2024-09-20T21:58:55.360 INFO:journalctl@ceph.mon.b.smithi067.stdout: partition_filters: 0 2024-09-20T21:58:55.360 INFO:journalctl@ceph.mon.b.smithi067.stdout: use_delta_encoding: 1 2024-09-20T21:58:55.360 INFO:journalctl@ceph.mon.b.smithi067.stdout: filter_policy: bloomfilter 2024-09-20T21:58:55.360 INFO:journalctl@ceph.mon.b.smithi067.stdout: whole_key_filtering: 1 2024-09-20T21:58:55.360 INFO:journalctl@ceph.mon.b.smithi067.stdout: verify_compression: 0 2024-09-20T21:58:55.360 INFO:journalctl@ceph.mon.b.smithi067.stdout: read_amp_bytes_per_bit: 0 2024-09-20T21:58:55.360 INFO:journalctl@ceph.mon.b.smithi067.stdout: format_version: 5 2024-09-20T21:58:55.360 INFO:journalctl@ceph.mon.b.smithi067.stdout: enable_index_compression: 1 2024-09-20T21:58:55.361 INFO:journalctl@ceph.mon.b.smithi067.stdout: block_align: 0 2024-09-20T21:58:55.361 INFO:journalctl@ceph.mon.b.smithi067.stdout: max_auto_readahead_size: 262144 2024-09-20T21:58:55.361 INFO:journalctl@ceph.mon.b.smithi067.stdout: prepopulate_block_cache: 0 2024-09-20T21:58:55.361 INFO:journalctl@ceph.mon.b.smithi067.stdout: initial_auto_readahead_size: 8192 2024-09-20T21:58:55.361 INFO:journalctl@ceph.mon.b.smithi067.stdout: num_file_reads_for_auto_readahead: 2 2024-09-20T21:58:55.361 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.write_buffer_size: 33554432 2024-09-20T21:58:55.361 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_write_buffer_number: 2 2024-09-20T21:58:55.362 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compression: NoCompression 2024-09-20T21:58:55.362 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.bottommost_compression: Disabled 2024-09-20T21:58:55.362 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.prefix_extractor: nullptr 2024-09-20T21:58:55.362 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-20T21:58:55.362 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.num_levels: 7 2024-09-20T21:58:55.362 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-20T21:58:55.362 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-20T21:58:55.363 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-20T21:58:55.363 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-20T21:58:55.363 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-20T21:58:55.363 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-20T21:58:55.363 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-20T21:58:55.363 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-20T21:58:55.363 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-20T21:58:55.363 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-20T21:58:55.364 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-20T21:58:55.364 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-20T21:58:55.364 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-20T21:58:55.364 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compression_opts.level: 32767 2024-09-20T21:58:55.365 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compression_opts.strategy: 0 2024-09-20T21:58:55.365 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-20T21:58:55.365 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-20T21:58:55.365 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-20T21:58:55.365 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-20T21:58:55.365 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compression_opts.enabled: false 2024-09-20T21:58:55.365 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-20T21:58:55.365 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-20T21:58:55.366 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-20T21:58:55.366 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-20T21:58:55.366 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.target_file_size_base: 67108864 2024-09-20T21:58:55.366 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-20T21:58:55.366 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-20T21:58:55.366 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-20T21:58:55.366 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-20T21:58:55.366 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-20T21:58:55.366 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-20T21:58:55.367 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-20T21:58:55.367 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-20T21:58:55.367 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-20T21:58:55.367 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-20T21:58:55.367 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-20T21:58:55.367 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-20T21:58:55.367 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-20T21:58:55.367 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-20T21:58:55.367 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.arena_block_size: 1048576 2024-09-20T21:58:55.368 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-20T21:58:55.368 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-20T21:58:55.368 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.disable_auto_compactions: 0 2024-09-20T21:58:55.368 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-20T21:58:55.368 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-20T21:58:55.368 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-20T21:58:55.368 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-20T21:58:55.368 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-20T21:58:55.369 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-20T21:58:55.369 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-20T21:58:55.369 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-20T21:58:55.369 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-20T21:58:55.369 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-20T21:58:55.369 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-20T21:58:55.369 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.inplace_update_support: 0 2024-09-20T21:58:55.369 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-20T21:58:55.369 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-20T21:58:55.369 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-20T21:58:55.370 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-20T21:58:55.370 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.bloom_locality: 0 2024-09-20T21:58:55.370 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.max_successive_merges: 0 2024-09-20T21:58:55.370 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-20T21:58:55.370 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.paranoid_file_checks: 0 2024-09-20T21:58:55.370 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.force_consistency_checks: 1 2024-09-20T21:58:55.370 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.report_bg_io_stats: 0 2024-09-20T21:58:55.370 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.ttl: 2592000 2024-09-20T21:58:55.370 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-20T21:58:55.371 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-20T21:58:55.371 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-20T21:58:55.371 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.enable_blob_files: false 2024-09-20T21:58:55.371 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.min_blob_size: 0 2024-09-20T21:58:55.371 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.blob_file_size: 268435456 2024-09-20T21:58:55.371 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-20T21:58:55.371 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-20T21:58:55.371 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-20T21:58:55.371 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-20T21:58:55.372 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-20T21:58:55.372 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.blob_file_starting_level: 0 2024-09-20T21:58:55.372 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-20T21:58:55.372 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-20T21:58:55.372 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-20T21:58:55.372 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 85d28748-5abd-4295-a10c-356cfad275a3 2024-09-20T21:58:55.372 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726869535082124, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-20T21:58:55.373 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-20T21:58:55.373 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726869535082576, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726869535, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "85d28748-5abd-4295-a10c-356cfad275a3", "db_session_id": "7IZN4AF4KPL5H0BAJDR6", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-20T21:58:55.373 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726869535082678, "job": 1, "event": "recovery_finished"} 2024-09-20T21:58:55.373 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-20T21:58:55.373 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-20T21:58:55.373 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55b831498e00 2024-09-20T21:58:55.373 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: DB pointer 0x55b831594000 2024-09-20T21:58:55.373 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-20T21:58:55.374 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-20T21:58:55.374 INFO:journalctl@ceph.mon.b.smithi067.stdout: ** DB Stats ** 2024-09-20T21:58:55.374 INFO:journalctl@ceph.mon.b.smithi067.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-20T21:58:55.374 INFO:journalctl@ceph.mon.b.smithi067.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-20T21:58:55.374 INFO:journalctl@ceph.mon.b.smithi067.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-20T21:58:55.374 INFO:journalctl@ceph.mon.b.smithi067.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-20T21:58:55.374 INFO:journalctl@ceph.mon.b.smithi067.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-20T21:58:55.375 INFO:journalctl@ceph.mon.b.smithi067.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-20T21:58:55.375 INFO:journalctl@ceph.mon.b.smithi067.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-20T21:58:55.375 INFO:journalctl@ceph.mon.b.smithi067.stdout: 2024-09-20T21:58:55.375 INFO:journalctl@ceph.mon.b.smithi067.stdout: ** Compaction Stats [default] ** 2024-09-20T21:58:55.375 INFO:journalctl@ceph.mon.b.smithi067.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-20T21:58:55.375 INFO:journalctl@ceph.mon.b.smithi067.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-20T21:58:55.375 INFO:journalctl@ceph.mon.b.smithi067.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-20T21:58:55.375 INFO:journalctl@ceph.mon.b.smithi067.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-20T21:58:55.375 INFO:journalctl@ceph.mon.b.smithi067.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-20T21:58:55.375 INFO:journalctl@ceph.mon.b.smithi067.stdout: 2024-09-20T21:58:55.376 INFO:journalctl@ceph.mon.b.smithi067.stdout: ** Compaction Stats [default] ** 2024-09-20T21:58:55.376 INFO:journalctl@ceph.mon.b.smithi067.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-20T21:58:55.376 INFO:journalctl@ceph.mon.b.smithi067.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-20T21:58:55.376 INFO:journalctl@ceph.mon.b.smithi067.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.7 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-20T21:58:55.376 INFO:journalctl@ceph.mon.b.smithi067.stdout: 2024-09-20T21:58:55.376 INFO:journalctl@ceph.mon.b.smithi067.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-20T21:58:55.376 INFO:journalctl@ceph.mon.b.smithi067.stdout: 2024-09-20T21:58:55.376 INFO:journalctl@ceph.mon.b.smithi067.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-20T21:58:55.376 INFO:journalctl@ceph.mon.b.smithi067.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-20T21:58:55.377 INFO:journalctl@ceph.mon.b.smithi067.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-20T21:58:55.377 INFO:journalctl@ceph.mon.b.smithi067.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-20T21:58:55.377 INFO:journalctl@ceph.mon.b.smithi067.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-20T21:58:55.377 INFO:journalctl@ceph.mon.b.smithi067.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-20T21:58:55.377 INFO:journalctl@ceph.mon.b.smithi067.stdout: Cumulative compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-20T21:58:55.377 INFO:journalctl@ceph.mon.b.smithi067.stdout: Interval compaction: 0.00 GB write, 0.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-20T21:58:55.377 INFO:journalctl@ceph.mon.b.smithi067.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-20T21:58:55.377 INFO:journalctl@ceph.mon.b.smithi067.stdout: Block cache BinnedLRUCache@0x55b83146b1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2024-09-20T21:58:55.377 INFO:journalctl@ceph.mon.b.smithi067.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-20T21:58:55.377 INFO:journalctl@ceph.mon.b.smithi067.stdout: 2024-09-20T21:58:55.378 INFO:journalctl@ceph.mon.b.smithi067.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-20T21:58:55.378 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-20T21:58:55.378 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: using public_addr v2:172.21.15.67:0/0 -> [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] 2024-09-20T21:58:55.378 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: starting mon.b rank -1 at public addrs [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] at bind addrs [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:58:55.378 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mon.b@-1(???) e0 preinit fsid 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:58:55.378 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mon.b@-1(synchronizing).mds e1 new map 2024-09-20T21:58:55.378 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-20T21:58:55.379 INFO:journalctl@ceph.mon.b.smithi067.stdout: e1 2024-09-20T21:58:55.379 INFO:journalctl@ceph.mon.b.smithi067.stdout: btime 2024-09-20T21:57:31:244773+0000 2024-09-20T21:58:55.379 INFO:journalctl@ceph.mon.b.smithi067.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-20T21:58:55.379 INFO:journalctl@ceph.mon.b.smithi067.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-20T21:58:55.379 INFO:journalctl@ceph.mon.b.smithi067.stdout: legacy client fscid: -1 2024-09-20T21:58:55.379 INFO:journalctl@ceph.mon.b.smithi067.stdout: 2024-09-20T21:58:55.379 INFO:journalctl@ceph.mon.b.smithi067.stdout: No filesystems configured 2024-09-20T21:58:55.380 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-20T21:58:55.380 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-20T21:58:55.380 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-20T21:58:55.380 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-20T21:58:55.380 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-20T21:58:55.380 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-20T21:58:55.380 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-20T21:58:55.380 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-20T21:58:55.381 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-20T21:58:55.381 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-20T21:58:55.381 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mkfs 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:58:55.381 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-20T21:58:55.381 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-20T21:58:55.381 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: monmap epoch 1 2024-09-20T21:58:55.381 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: fsid 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:58:55.382 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: last_changed 2024-09-20T21:57:28.205024+0000 2024-09-20T21:58:55.382 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: created 2024-09-20T21:57:28.205024+0000 2024-09-20T21:58:55.382 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: min_mon_release 19 (squid) 2024-09-20T21:58:55.382 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: election_strategy: 1 2024-09-20T21:58:55.382 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-20T21:58:55.382 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: fsmap 2024-09-20T21:58:55.382 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: osdmap e1: 0 total, 0 up, 0 in 2024-09-20T21:58:55.382 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mgrmap e1: no daemons active 2024-09-20T21:58:55.382 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1263929463' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-20T21:58:55.383 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1911744243' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-20T21:58:55.383 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1911744243' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-20T21:58:55.383 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/356044068' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:58:55.383 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-20T21:58:55.383 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: monmap epoch 1 2024-09-20T21:58:55.383 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: fsid 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:58:55.383 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: last_changed 2024-09-20T21:57:28.205024+0000 2024-09-20T21:58:55.383 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: created 2024-09-20T21:57:28.205024+0000 2024-09-20T21:58:55.383 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: min_mon_release 19 (squid) 2024-09-20T21:58:55.383 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: election_strategy: 1 2024-09-20T21:58:55.384 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-20T21:58:55.384 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: fsmap 2024-09-20T21:58:55.384 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: osdmap e1: 0 total, 0 up, 0 in 2024-09-20T21:58:55.384 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mgrmap e1: no daemons active 2024-09-20T21:58:55.384 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1287971683' entity='client.admin' 2024-09-20T21:58:55.384 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2883946848' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-20T21:58:55.384 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1148104965' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-20T21:58:55.384 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Activating manager daemon y 2024-09-20T21:58:55.385 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mgrmap e2: y(active, starting, since 0.0029267s) 2024-09-20T21:58:55.385 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-20T21:58:55.385 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-20T21:58:55.385 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-20T21:58:55.385 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T21:58:55.385 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-20T21:58:55.385 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Manager daemon y is now available 2024-09-20T21:58:55.385 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-20T21:58:55.385 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-20T21:58:55.386 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' 2024-09-20T21:58:55.386 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' 2024-09-20T21:58:55.386 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14100 172.21.15.60:0/3100822020' entity='mgr.y' 2024-09-20T21:58:55.386 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mgrmap e3: y(active, since 1.00747s) 2024-09-20T21:58:55.386 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mgrmap e4: y(active, since 2s) 2024-09-20T21:58:55.386 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3767243270' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-20T21:58:55.386 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1845697438' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-20T21:58:55.387 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/851395620' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-20T21:58:55.387 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/851395620' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-20T21:58:55.387 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mgrmap e5: y(active, since 13s) 2024-09-20T21:58:55.387 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/908676662' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-20T21:58:55.387 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Active manager daemon y restarted 2024-09-20T21:58:55.387 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Activating manager daemon y 2024-09-20T21:58:55.387 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: osdmap e2: 0 total, 0 up, 0 in 2024-09-20T21:58:55.387 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mgrmap e6: y(active, starting, since 0.00332614s) 2024-09-20T21:58:55.388 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T21:58:55.388 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-20T21:58:55.388 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-20T21:58:55.388 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-20T21:58:55.388 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-20T21:58:55.388 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Manager daemon y is now available 2024-09-20T21:58:55.388 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:55.388 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:55.389 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:58:55.389 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:58:55.389 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-20T21:58:55.389 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-20T21:58:55.389 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Found migration_current of "None". Setting to last migration. 2024-09-20T21:58:55.389 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mgrmap e7: y(active, since 1.00528s) 2024-09-20T21:58:55.389 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-20T21:58:55.390 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-20T21:58:55.390 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:55.390 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:55.390 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:55.390 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:58:55.390 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:55.390 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mgrmap e8: y(active, since 2s) 2024-09-20T21:58:55.390 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:58:55.390 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:55.391 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: [20/Sep/2024:21:58:05] ENGINE Bus STARTING 2024-09-20T21:58:55.391 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: [20/Sep/2024:21:58:05] ENGINE Serving on http://172.21.15.60:8765 2024-09-20T21:58:55.391 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: [20/Sep/2024:21:58:05] ENGINE Serving on https://172.21.15.60:7150 2024-09-20T21:58:55.391 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: [20/Sep/2024:21:58:05] ENGINE Bus STARTED 2024-09-20T21:58:55.391 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: [20/Sep/2024:21:58:05] ENGINE Error in HTTPServer.serve 2024-09-20T21:58:55.391 INFO:journalctl@ceph.mon.b.smithi067.stdout: Traceback (most recent call last): 2024-09-20T21:58:55.391 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-20T21:58:55.392 INFO:journalctl@ceph.mon.b.smithi067.stdout: self._connections.run(self.expiration_interval) 2024-09-20T21:58:55.392 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-20T21:58:55.392 INFO:journalctl@ceph.mon.b.smithi067.stdout: self._run(expiration_interval) 2024-09-20T21:58:55.392 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-20T21:58:55.392 INFO:journalctl@ceph.mon.b.smithi067.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-20T21:58:55.392 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-20T21:58:55.392 INFO:journalctl@ceph.mon.b.smithi067.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-20T21:58:55.393 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-20T21:58:55.393 INFO:journalctl@ceph.mon.b.smithi067.stdout: s = self.context.wrap_socket( 2024-09-20T21:58:55.393 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-20T21:58:55.393 INFO:journalctl@ceph.mon.b.smithi067.stdout: return self.sslsocket_class._create( 2024-09-20T21:58:55.393 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-20T21:58:55.393 INFO:journalctl@ceph.mon.b.smithi067.stdout: self.do_handshake() 2024-09-20T21:58:55.393 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-20T21:58:55.394 INFO:journalctl@ceph.mon.b.smithi067.stdout: self._sslobj.do_handshake() 2024-09-20T21:58:55.394 INFO:journalctl@ceph.mon.b.smithi067.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-20T21:58:55.394 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:55.394 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Generating ssh key... 2024-09-20T21:58:55.394 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:55.394 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:55.394 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:55.394 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi060", "addr": "172.21.15.60", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:55.395 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Deploying cephadm binary to smithi060 2024-09-20T21:58:55.395 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:55.395 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Added host smithi060 2024-09-20T21:58:55.395 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:58:55.395 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:55.395 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Saving service mon spec with placement count:5 2024-09-20T21:58:55.395 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:55.396 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:55.396 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Saving service mgr spec with placement count:2 2024-09-20T21:58:55.396 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:55.396 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2194018915' entity='client.admin' 2024-09-20T21:58:55.396 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:55.396 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:55.396 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1451485985' entity='client.admin' 2024-09-20T21:58:55.396 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:55.397 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2048872448' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-20T21:58:55.397 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14118 172.21.15.60:0/3015066506' entity='mgr.y' 2024-09-20T21:58:55.397 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2048872448' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-20T21:58:55.397 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mgrmap e9: y(active, since 19s) 2024-09-20T21:58:55.397 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3726089641' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-20T21:58:55.397 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Active manager daemon y restarted 2024-09-20T21:58:55.397 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Activating manager daemon y 2024-09-20T21:58:55.397 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: osdmap e3: 0 total, 0 up, 0 in 2024-09-20T21:58:55.397 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mgrmap e10: y(active, starting, since 0.00318736s) 2024-09-20T21:58:55.398 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T21:58:55.398 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-20T21:58:55.398 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-20T21:58:55.398 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-20T21:58:55.398 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-20T21:58:55.398 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Manager daemon y is now available 2024-09-20T21:58:55.398 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:58:55.398 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-20T21:58:55.399 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-20T21:58:55.399 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mgrmap e11: y(active, since 1.0061s) 2024-09-20T21:58:55.399 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.399 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: [20/Sep/2024:21:58:28] ENGINE Bus STARTING 2024-09-20T21:58:55.399 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: [20/Sep/2024:21:58:28] ENGINE Serving on http://172.21.15.60:8765 2024-09-20T21:58:55.399 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: [20/Sep/2024:21:58:28] ENGINE Serving on https://172.21.15.60:7150 2024-09-20T21:58:55.399 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: [20/Sep/2024:21:58:28] ENGINE Bus STARTED 2024-09-20T21:58:55.399 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: [20/Sep/2024:21:58:28] ENGINE Error in HTTPServer.serve 2024-09-20T21:58:55.399 INFO:journalctl@ceph.mon.b.smithi067.stdout: Traceback (most recent call last): 2024-09-20T21:58:55.400 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-20T21:58:55.400 INFO:journalctl@ceph.mon.b.smithi067.stdout: self._connections.run(self.expiration_interval) 2024-09-20T21:58:55.400 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-20T21:58:55.400 INFO:journalctl@ceph.mon.b.smithi067.stdout: self._run(expiration_interval) 2024-09-20T21:58:55.400 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-20T21:58:55.400 INFO:journalctl@ceph.mon.b.smithi067.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-20T21:58:55.400 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-20T21:58:55.400 INFO:journalctl@ceph.mon.b.smithi067.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-20T21:58:55.401 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-20T21:58:55.401 INFO:journalctl@ceph.mon.b.smithi067.stdout: s = self.context.wrap_socket( 2024-09-20T21:58:55.401 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-20T21:58:55.401 INFO:journalctl@ceph.mon.b.smithi067.stdout: return self.sslsocket_class._create( 2024-09-20T21:58:55.401 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-20T21:58:55.401 INFO:journalctl@ceph.mon.b.smithi067.stdout: self.do_handshake() 2024-09-20T21:58:55.401 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-20T21:58:55.401 INFO:journalctl@ceph.mon.b.smithi067.stdout: self._sslobj.do_handshake() 2024-09-20T21:58:55.402 INFO:journalctl@ceph.mon.b.smithi067.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-20T21:58:55.402 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mgrmap e12: y(active, since 2s) 2024-09-20T21:58:55.402 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:55.402 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.402 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.402 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.402 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:55.402 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.402 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/675219104' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-20T21:58:55.403 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mgrmap e13: y(active, since 6s) 2024-09-20T21:58:55.403 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/665336590' entity='client.admin' 2024-09-20T21:58:55.403 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2355587091' entity='client.admin' 2024-09-20T21:58:55.403 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:55.403 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.403 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.403 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.403 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi060", "name": "osd_memory_target"}]: dispatch 2024-09-20T21:58:55.403 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:58:55.403 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:58:55.404 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Updating smithi060:/etc/ceph/ceph.conf 2024-09-20T21:58:55.404 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Updating smithi060:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T21:58:55.404 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Updating smithi060:/etc/ceph/ceph.client.admin.keyring 2024-09-20T21:58:55.404 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Updating smithi060:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T21:58:55.404 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.404 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.404 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.404 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:58:55.404 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:58:55.404 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:58:55.404 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.405 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi067", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:55.405 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Deploying cephadm binary to smithi067 2024-09-20T21:58:55.405 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.405 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Added host smithi067 2024-09-20T21:58:55.405 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:58:55.405 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.405 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.405 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.406 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T21:58:55.406 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:58:55.406 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3199120284' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-20T21:58:55.406 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3199120284' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-20T21:58:55.406 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: osdmap e4: 0 total, 0 up, 0 in 2024-09-20T21:58:55.406 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.407 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.407 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.407 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.407 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi067", "name": "osd_memory_target"}]: dispatch 2024-09-20T21:58:55.407 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:58:55.407 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:58:55.407 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-20T21:58:55.408 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:58:55.408 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Updating smithi067:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T21:58:55.408 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-09-20T21:58:55.408 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Updating smithi067:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T21:58:55.408 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi060:172.21.15.60=a;smithi060:[v2:172.21.15.60:3301,v1:172.21.15.60:6790]=c;smithi067:172.21.15.67=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:58:55.408 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Saving service mon spec with placement smithi060:172.21.15.60=a;smithi060:[v2:172.21.15.60:3301,v1:172.21.15.60:6790]=c;smithi067:172.21.15.67=b;count:3 2024-09-20T21:58:55.408 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.408 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.408 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.408 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:58:55.409 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-20T21:58:55.409 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:58:55.409 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: Deploying daemon mon.b on smithi067 2024-09-20T21:58:55.409 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:58:55.409 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.67:0/4248178580' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-20T21:58:55.409 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:58:55.409 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 ceph-mon[33511]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-20T21:58:55.409 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 bash[33452]: 497f6b558d69936edda24dcbdf173b5a215e5bfb44eb5a3cb4f818dea5aa7ab5 2024-09-20T21:58:55.410 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:58:55 smithi067 systemd[1]: Started Ceph mon.b for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T21:58:55.978 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-20T21:58:55.979 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-20T21:58:56.253 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T21:59:00.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:00.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T21:59:00.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: mon.a calling monitor election 2024-09-20T21:59:00.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:00.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:00.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:00.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:00.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: mon.b calling monitor election 2024-09-20T21:59:00.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:00.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:00.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:00.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:00.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-20T21:59:00.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: monmap epoch 2 2024-09-20T21:59:00.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: fsid 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:59:00.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: last_changed 2024-09-20T21:58:55.108527+0000 2024-09-20T21:59:00.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: created 2024-09-20T21:57:28.205024+0000 2024-09-20T21:59:00.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: min_mon_release 19 (squid) 2024-09-20T21:59:00.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: election_strategy: 1 2024-09-20T21:59:00.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-20T21:59:00.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: 1: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.b 2024-09-20T21:59:00.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: fsmap 2024-09-20T21:59:00.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: osdmap e4: 0 total, 0 up, 0 in 2024-09-20T21:59:00.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: mgrmap e13: y(active, since 33s) 2024-09-20T21:59:00.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: overall HEALTH_OK 2024-09-20T21:59:00.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:00.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:00.516 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:00.516 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-20T21:59:00.516 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:00.524 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:59:00.524 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":2,"fsid":"3df9647e-779b-11ef-baf5-efdc52797490","modified":"2024-09-20T21:58:55.108527Z","created":"2024-09-20T21:57:28.205024Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-20T21:59:00.524 INFO:teuthology.orchestra.run.smithi067.stderr:dumped monmap epoch 2 2024-09-20T21:59:00.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:00.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T21:59:00.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: mon.a calling monitor election 2024-09-20T21:59:00.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:00.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:00.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:00.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: mon.b calling monitor election 2024-09-20T21:59:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-20T21:59:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: monmap epoch 2 2024-09-20T21:59:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: fsid 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:59:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: last_changed 2024-09-20T21:58:55.108527+0000 2024-09-20T21:59:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: created 2024-09-20T21:57:28.205024+0000 2024-09-20T21:59:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: min_mon_release 19 (squid) 2024-09-20T21:59:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: election_strategy: 1 2024-09-20T21:59:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-20T21:59:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: 1: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.b 2024-09-20T21:59:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: fsmap 2024-09-20T21:59:00.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: osdmap e4: 0 total, 0 up, 0 in 2024-09-20T21:59:00.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: mgrmap e13: y(active, since 33s) 2024-09-20T21:59:00.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: overall HEALTH_OK 2024-09-20T21:59:00.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:00.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:00.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:00.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-20T21:59:00.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:01.432 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:01 smithi060 ceph-mon[29694]: Deploying daemon mon.c on smithi060 2024-09-20T21:59:01.433 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:01 smithi060 ceph-mon[29694]: from='client.? 172.21.15.67:0/2486260956' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-20T21:59:01.433 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:01 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:01.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:01 smithi067 ceph-mon[33511]: Deploying daemon mon.c on smithi060 2024-09-20T21:59:01.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:01 smithi067 ceph-mon[33511]: from='client.? 172.21.15.67:0/2486260956' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-20T21:59:01.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:01 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:02.101 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-20T21:59:02.101 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-20T21:59:02.364 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T21:59:02.456 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:02 smithi060 ceph-mon[29694]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:02.485 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:02 smithi067 ceph-mon[33511]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:03.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:02 smithi060 systemd[1]: Starting Ceph mon.c for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T21:59:03.041 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:59:03.041 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":2,"fsid":"3df9647e-779b-11ef-baf5-efdc52797490","modified":"2024-09-20T21:58:55.108527Z","created":"2024-09-20T21:57:28.205024Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-20T21:59:03.041 INFO:teuthology.orchestra.run.smithi067.stderr:dumped monmap epoch 2 2024-09-20T21:59:03.335 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:03 smithi067 ceph-mon[33511]: from='client.? 172.21.15.67:0/3513242279' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-20T21:59:03.389 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[29694]: from='client.? 172.21.15.67:0/3513242279' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-20T21:59:03.389 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 podman[40252]: 2024-09-20 21:59:03.134245935 +0000 UTC m=+0.107874536 container create a7ce8d1ab286d47dc7f3f55c0f76949af7bdadf8d6bba8a5eae3301d4d36d636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-c, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=squid-9e2ac70, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-20T21:59:03.389 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 podman[40252]: 2024-09-20 21:59:03.04301771 +0000 UTC m=+0.016646266 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5 2024-09-20T21:59:03.389 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 podman[40252]: 2024-09-20 21:59:03.361727248 +0000 UTC m=+0.335355813 container init a7ce8d1ab286d47dc7f3f55c0f76949af7bdadf8d6bba8a5eae3301d4d36d636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-9e2ac70, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-20T21:59:03.389 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 podman[40252]: 2024-09-20 21:59:03.366115507 +0000 UTC m=+0.339744063 container start a7ce8d1ab286d47dc7f3f55c0f76949af7bdadf8d6bba8a5eae3301d4d36d636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-9e2ac70, ceph=True) 2024-09-20T21:59:03.389 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: set uid:gid to 167:167 (ceph:ceph) 2024-09-20T21:59:03.389 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: ceph version 19.1.1-367-g9e2ac70b (9e2ac70bb7aea67d34f0d50002344abbe7803736) squid (stable), process ceph-mon, pid 7 2024-09-20T21:59:03.390 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: pidfile_write: ignore empty --pid-file 2024-09-20T21:59:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: load: jerasure load: lrc 2024-09-20T21:59:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: RocksDB version: 7.9.2 2024-09-20T21:59:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Git sha 0 2024-09-20T21:59:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Compile date 2024-09-19 18:20:48 2024-09-20T21:59:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: DB SUMMARY 2024-09-20T21:59:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: DB Session ID: 4EFS68L4AR1KPD2B13HL 2024-09-20T21:59:03.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: CURRENT file: CURRENT 2024-09-20T21:59:03.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: IDENTITY file: IDENTITY 2024-09-20T21:59:03.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-20T21:59:03.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-20T21:59:03.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-20T21:59:03.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.error_if_exists: 0 2024-09-20T21:59:03.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.create_if_missing: 0 2024-09-20T21:59:03.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.paranoid_checks: 1 2024-09-20T21:59:03.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-20T21:59:03.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-20T21:59:03.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-20T21:59:03.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.env: 0x565188be0940 2024-09-20T21:59:03.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.fs: PosixFileSystem 2024-09-20T21:59:03.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.info_log: 0x56518acf16e0 2024-09-20T21:59:03.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_file_opening_threads: 16 2024-09-20T21:59:03.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.statistics: (nil) 2024-09-20T21:59:03.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.use_fsync: 0 2024-09-20T21:59:03.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_log_file_size: 0 2024-09-20T21:59:03.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-20T21:59:03.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-20T21:59:03.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.keep_log_file_num: 1000 2024-09-20T21:59:03.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.recycle_log_file_num: 0 2024-09-20T21:59:03.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.allow_fallocate: 1 2024-09-20T21:59:03.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.allow_mmap_reads: 0 2024-09-20T21:59:03.791 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.allow_mmap_writes: 0 2024-09-20T21:59:03.791 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.use_direct_reads: 0 2024-09-20T21:59:03.791 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-20T21:59:03.791 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.create_missing_column_families: 0 2024-09-20T21:59:03.791 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.db_log_dir: 2024-09-20T21:59:03.791 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.wal_dir: 2024-09-20T21:59:03.791 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-20T21:59:03.791 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-20T21:59:03.791 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-20T21:59:03.792 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-20T21:59:03.792 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-20T21:59:03.792 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-20T21:59:03.792 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.advise_random_on_open: 1 2024-09-20T21:59:03.792 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.db_write_buffer_size: 0 2024-09-20T21:59:03.792 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.write_buffer_manager: 0x56518ad00dc0 2024-09-20T21:59:03.793 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-20T21:59:03.793 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-20T21:59:03.793 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-20T21:59:03.793 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.rate_limiter: (nil) 2024-09-20T21:59:03.793 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-20T21:59:03.793 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.wal_recovery_mode: 2 2024-09-20T21:59:03.793 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.enable_thread_tracking: 0 2024-09-20T21:59:03.793 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.enable_pipelined_write: 0 2024-09-20T21:59:03.793 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.unordered_write: 0 2024-09-20T21:59:03.794 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-20T21:59:03.794 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-20T21:59:03.794 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-20T21:59:03.794 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-20T21:59:03.794 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.row_cache: None 2024-09-20T21:59:03.794 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.wal_filter: None 2024-09-20T21:59:03.794 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-20T21:59:03.795 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.allow_ingest_behind: 0 2024-09-20T21:59:03.795 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.two_write_queues: 0 2024-09-20T21:59:03.795 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.manual_wal_flush: 0 2024-09-20T21:59:03.795 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.wal_compression: 0 2024-09-20T21:59:03.795 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.atomic_flush: 0 2024-09-20T21:59:03.795 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-20T21:59:03.795 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-20T21:59:03.796 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-20T21:59:03.796 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.log_readahead_size: 0 2024-09-20T21:59:03.796 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-20T21:59:03.796 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.best_efforts_recovery: 0 2024-09-20T21:59:03.796 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-20T21:59:03.796 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-20T21:59:03.796 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.allow_data_in_errors: 0 2024-09-20T21:59:03.797 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.db_host_id: __hostname__ 2024-09-20T21:59:03.797 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-20T21:59:03.797 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_background_jobs: 2 2024-09-20T21:59:03.797 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_background_compactions: -1 2024-09-20T21:59:03.797 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_subcompactions: 1 2024-09-20T21:59:03.799 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-20T21:59:03.799 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-20T21:59:03.799 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-20T21:59:03.800 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_total_wal_size: 0 2024-09-20T21:59:03.800 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-20T21:59:03.800 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-20T21:59:03.800 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-20T21:59:03.800 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-20T21:59:03.800 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_open_files: -1 2024-09-20T21:59:03.801 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.bytes_per_sync: 0 2024-09-20T21:59:03.801 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-20T21:59:03.801 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-20T21:59:03.801 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compaction_readahead_size: 0 2024-09-20T21:59:03.801 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_background_flushes: -1 2024-09-20T21:59:03.801 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Compression algorithms supported: 2024-09-20T21:59:03.802 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: kZSTD supported: 0 2024-09-20T21:59:03.802 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: kXpressCompression supported: 0 2024-09-20T21:59:03.802 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: kBZip2Compression supported: 0 2024-09-20T21:59:03.802 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-20T21:59:03.802 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: kLZ4Compression supported: 1 2024-09-20T21:59:03.802 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: kZlibCompression supported: 1 2024-09-20T21:59:03.802 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: kLZ4HCCompression supported: 1 2024-09-20T21:59:03.802 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: kSnappyCompression supported: 1 2024-09-20T21:59:03.803 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-20T21:59:03.803 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-20T21:59:03.803 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-20T21:59:03.803 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-20T21:59:03.803 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-20T21:59:03.803 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.merge_operator: 2024-09-20T21:59:03.803 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compaction_filter: None 2024-09-20T21:59:03.803 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compaction_filter_factory: None 2024-09-20T21:59:03.803 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.sst_partitioner_factory: None 2024-09-20T21:59:03.803 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-20T21:59:03.803 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-20T21:59:03.804 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56518acf1320) 2024-09-20T21:59:03.804 INFO:journalctl@ceph.mon.c.smithi060.stdout: cache_index_and_filter_blocks: 1 2024-09-20T21:59:03.804 INFO:journalctl@ceph.mon.c.smithi060.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-20T21:59:03.804 INFO:journalctl@ceph.mon.c.smithi060.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-20T21:59:03.804 INFO:journalctl@ceph.mon.c.smithi060.stdout: pin_top_level_index_and_filter: 1 2024-09-20T21:59:03.804 INFO:journalctl@ceph.mon.c.smithi060.stdout: index_type: 0 2024-09-20T21:59:03.804 INFO:journalctl@ceph.mon.c.smithi060.stdout: data_block_index_type: 0 2024-09-20T21:59:03.804 INFO:journalctl@ceph.mon.c.smithi060.stdout: index_shortening: 1 2024-09-20T21:59:03.804 INFO:journalctl@ceph.mon.c.smithi060.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-20T21:59:03.804 INFO:journalctl@ceph.mon.c.smithi060.stdout: checksum: 4 2024-09-20T21:59:03.804 INFO:journalctl@ceph.mon.c.smithi060.stdout: no_block_cache: 0 2024-09-20T21:59:03.804 INFO:journalctl@ceph.mon.c.smithi060.stdout: block_cache: 0x56518ace51f0 2024-09-20T21:59:03.804 INFO:journalctl@ceph.mon.c.smithi060.stdout: block_cache_name: BinnedLRUCache 2024-09-20T21:59:03.804 INFO:journalctl@ceph.mon.c.smithi060.stdout: block_cache_options: 2024-09-20T21:59:03.804 INFO:journalctl@ceph.mon.c.smithi060.stdout: capacity : 536870912 2024-09-20T21:59:03.805 INFO:journalctl@ceph.mon.c.smithi060.stdout: num_shard_bits : 4 2024-09-20T21:59:03.805 INFO:journalctl@ceph.mon.c.smithi060.stdout: strict_capacity_limit : 0 2024-09-20T21:59:03.805 INFO:journalctl@ceph.mon.c.smithi060.stdout: high_pri_pool_ratio: 0.000 2024-09-20T21:59:03.805 INFO:journalctl@ceph.mon.c.smithi060.stdout: block_cache_compressed: (nil) 2024-09-20T21:59:03.805 INFO:journalctl@ceph.mon.c.smithi060.stdout: persistent_cache: (nil) 2024-09-20T21:59:03.805 INFO:journalctl@ceph.mon.c.smithi060.stdout: block_size: 4096 2024-09-20T21:59:03.805 INFO:journalctl@ceph.mon.c.smithi060.stdout: block_size_deviation: 10 2024-09-20T21:59:03.805 INFO:journalctl@ceph.mon.c.smithi060.stdout: block_restart_interval: 16 2024-09-20T21:59:03.805 INFO:journalctl@ceph.mon.c.smithi060.stdout: index_block_restart_interval: 1 2024-09-20T21:59:03.805 INFO:journalctl@ceph.mon.c.smithi060.stdout: metadata_block_size: 4096 2024-09-20T21:59:03.805 INFO:journalctl@ceph.mon.c.smithi060.stdout: partition_filters: 0 2024-09-20T21:59:03.805 INFO:journalctl@ceph.mon.c.smithi060.stdout: use_delta_encoding: 1 2024-09-20T21:59:03.805 INFO:journalctl@ceph.mon.c.smithi060.stdout: filter_policy: bloomfilter 2024-09-20T21:59:03.805 INFO:journalctl@ceph.mon.c.smithi060.stdout: whole_key_filtering: 1 2024-09-20T21:59:03.806 INFO:journalctl@ceph.mon.c.smithi060.stdout: verify_compression: 0 2024-09-20T21:59:03.806 INFO:journalctl@ceph.mon.c.smithi060.stdout: read_amp_bytes_per_bit: 0 2024-09-20T21:59:03.806 INFO:journalctl@ceph.mon.c.smithi060.stdout: format_version: 5 2024-09-20T21:59:03.806 INFO:journalctl@ceph.mon.c.smithi060.stdout: enable_index_compression: 1 2024-09-20T21:59:03.806 INFO:journalctl@ceph.mon.c.smithi060.stdout: block_align: 0 2024-09-20T21:59:03.806 INFO:journalctl@ceph.mon.c.smithi060.stdout: max_auto_readahead_size: 262144 2024-09-20T21:59:03.806 INFO:journalctl@ceph.mon.c.smithi060.stdout: prepopulate_block_cache: 0 2024-09-20T21:59:03.806 INFO:journalctl@ceph.mon.c.smithi060.stdout: initial_auto_readahead_size: 8192 2024-09-20T21:59:03.806 INFO:journalctl@ceph.mon.c.smithi060.stdout: num_file_reads_for_auto_readahead: 2 2024-09-20T21:59:03.806 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.write_buffer_size: 33554432 2024-09-20T21:59:03.806 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_write_buffer_number: 2 2024-09-20T21:59:03.806 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compression: NoCompression 2024-09-20T21:59:03.806 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.bottommost_compression: Disabled 2024-09-20T21:59:03.806 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.prefix_extractor: nullptr 2024-09-20T21:59:03.807 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-20T21:59:03.807 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.num_levels: 7 2024-09-20T21:59:03.807 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-20T21:59:03.807 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-20T21:59:03.807 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-20T21:59:03.807 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-20T21:59:03.807 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-20T21:59:03.807 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-20T21:59:03.807 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-20T21:59:03.807 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-20T21:59:03.807 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-20T21:59:03.807 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-20T21:59:03.807 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-20T21:59:03.807 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-20T21:59:03.808 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-20T21:59:03.808 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compression_opts.level: 32767 2024-09-20T21:59:03.808 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compression_opts.strategy: 0 2024-09-20T21:59:03.808 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-20T21:59:03.808 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-20T21:59:03.808 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-20T21:59:03.809 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-20T21:59:03.809 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compression_opts.enabled: false 2024-09-20T21:59:03.810 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-20T21:59:03.810 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-20T21:59:03.810 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-20T21:59:03.810 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-20T21:59:03.810 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.target_file_size_base: 67108864 2024-09-20T21:59:03.810 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-20T21:59:03.810 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-20T21:59:03.810 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-20T21:59:03.810 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-20T21:59:03.810 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-20T21:59:03.810 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-20T21:59:03.810 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-20T21:59:03.810 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-20T21:59:03.811 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-20T21:59:03.811 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-20T21:59:03.811 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-20T21:59:03.811 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-20T21:59:03.811 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-20T21:59:03.811 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-20T21:59:03.811 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.arena_block_size: 1048576 2024-09-20T21:59:03.811 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-20T21:59:03.811 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-20T21:59:03.811 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.disable_auto_compactions: 0 2024-09-20T21:59:03.811 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-20T21:59:03.811 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-20T21:59:03.811 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-20T21:59:03.811 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-20T21:59:03.812 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-20T21:59:03.812 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-20T21:59:03.812 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-20T21:59:03.812 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-20T21:59:03.812 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-20T21:59:03.812 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-20T21:59:03.812 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-20T21:59:03.812 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.inplace_update_support: 0 2024-09-20T21:59:03.812 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-20T21:59:03.812 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-20T21:59:03.812 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-20T21:59:03.812 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-20T21:59:03.812 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.bloom_locality: 0 2024-09-20T21:59:03.813 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.max_successive_merges: 0 2024-09-20T21:59:03.813 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-20T21:59:03.813 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.paranoid_file_checks: 0 2024-09-20T21:59:03.813 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.force_consistency_checks: 1 2024-09-20T21:59:03.813 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.report_bg_io_stats: 0 2024-09-20T21:59:03.813 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.ttl: 2592000 2024-09-20T21:59:03.813 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-20T21:59:03.813 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-20T21:59:03.813 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-20T21:59:03.813 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.enable_blob_files: false 2024-09-20T21:59:03.813 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.min_blob_size: 0 2024-09-20T21:59:03.813 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.blob_file_size: 268435456 2024-09-20T21:59:03.813 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-20T21:59:03.814 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-20T21:59:03.814 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-20T21:59:03.814 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-20T21:59:03.814 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-20T21:59:03.814 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.blob_file_starting_level: 0 2024-09-20T21:59:03.814 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-20T21:59:03.814 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-20T21:59:03.814 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-20T21:59:03.814 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 55a88fa1-cbce-48f4-96e5-8c4d08d12a07 2024-09-20T21:59:03.814 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726869543393938, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-20T21:59:03.814 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-20T21:59:03.814 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726869543394330, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1773, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726869543, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "55a88fa1-cbce-48f4-96e5-8c4d08d12a07", "db_session_id": "4EFS68L4AR1KPD2B13HL", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-20T21:59:03.815 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726869543394442, "job": 1, "event": "recovery_finished"} 2024-09-20T21:59:03.815 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-20T21:59:03.815 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-20T21:59:03.815 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x56518ad12e00 2024-09-20T21:59:03.815 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: DB pointer 0x56518ae0e000 2024-09-20T21:59:03.815 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-20T21:59:03.815 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-20T21:59:03.815 INFO:journalctl@ceph.mon.c.smithi060.stdout: ** DB Stats ** 2024-09-20T21:59:03.815 INFO:journalctl@ceph.mon.c.smithi060.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-20T21:59:03.815 INFO:journalctl@ceph.mon.c.smithi060.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-20T21:59:03.815 INFO:journalctl@ceph.mon.c.smithi060.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-20T21:59:03.815 INFO:journalctl@ceph.mon.c.smithi060.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-20T21:59:03.815 INFO:journalctl@ceph.mon.c.smithi060.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-20T21:59:03.816 INFO:journalctl@ceph.mon.c.smithi060.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-20T21:59:03.816 INFO:journalctl@ceph.mon.c.smithi060.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-20T21:59:03.816 INFO:journalctl@ceph.mon.c.smithi060.stdout: 2024-09-20T21:59:03.816 INFO:journalctl@ceph.mon.c.smithi060.stdout: ** Compaction Stats [default] ** 2024-09-20T21:59:03.816 INFO:journalctl@ceph.mon.c.smithi060.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-20T21:59:03.816 INFO:journalctl@ceph.mon.c.smithi060.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-20T21:59:03.816 INFO:journalctl@ceph.mon.c.smithi060.stdout: L0 1/0 1.73 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-20T21:59:03.816 INFO:journalctl@ceph.mon.c.smithi060.stdout: Sum 1/0 1.73 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-20T21:59:03.816 INFO:journalctl@ceph.mon.c.smithi060.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-20T21:59:03.816 INFO:journalctl@ceph.mon.c.smithi060.stdout: 2024-09-20T21:59:03.816 INFO:journalctl@ceph.mon.c.smithi060.stdout: ** Compaction Stats [default] ** 2024-09-20T21:59:03.816 INFO:journalctl@ceph.mon.c.smithi060.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-20T21:59:03.817 INFO:journalctl@ceph.mon.c.smithi060.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-20T21:59:03.817 INFO:journalctl@ceph.mon.c.smithi060.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.3 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-20T21:59:03.817 INFO:journalctl@ceph.mon.c.smithi060.stdout: 2024-09-20T21:59:03.817 INFO:journalctl@ceph.mon.c.smithi060.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-20T21:59:03.817 INFO:journalctl@ceph.mon.c.smithi060.stdout: 2024-09-20T21:59:03.817 INFO:journalctl@ceph.mon.c.smithi060.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-20T21:59:03.817 INFO:journalctl@ceph.mon.c.smithi060.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-20T21:59:03.817 INFO:journalctl@ceph.mon.c.smithi060.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-20T21:59:03.817 INFO:journalctl@ceph.mon.c.smithi060.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-20T21:59:03.817 INFO:journalctl@ceph.mon.c.smithi060.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-20T21:59:03.817 INFO:journalctl@ceph.mon.c.smithi060.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-20T21:59:03.817 INFO:journalctl@ceph.mon.c.smithi060.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-20T21:59:03.817 INFO:journalctl@ceph.mon.c.smithi060.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-20T21:59:03.818 INFO:journalctl@ceph.mon.c.smithi060.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-20T21:59:03.818 INFO:journalctl@ceph.mon.c.smithi060.stdout: Block cache BinnedLRUCache@0x56518ace51f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-20T21:59:03.818 INFO:journalctl@ceph.mon.c.smithi060.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-20T21:59:03.818 INFO:journalctl@ceph.mon.c.smithi060.stdout: 2024-09-20T21:59:03.818 INFO:journalctl@ceph.mon.c.smithi060.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-20T21:59:03.818 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-20T21:59:03.818 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: using public_addrv [v2:172.21.15.60:3301/0,v1:172.21.15.60:6790/0] 2024-09-20T21:59:03.818 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: starting mon.c rank -1 at public addrs [v2:172.21.15.60:3301/0,v1:172.21.15.60:6790/0] at bind addrs [v2:172.21.15.60:3301/0,v1:172.21.15.60:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:59:03.818 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mon.c@-1(???) e0 preinit fsid 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:59:03.818 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mon.c@-1(synchronizing).mds e1 new map 2024-09-20T21:59:03.818 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-20T21:59:03.818 INFO:journalctl@ceph.mon.c.smithi060.stdout: e1 2024-09-20T21:59:03.818 INFO:journalctl@ceph.mon.c.smithi060.stdout: btime 2024-09-20T21:57:31:244773+0000 2024-09-20T21:59:03.819 INFO:journalctl@ceph.mon.c.smithi060.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-20T21:59:03.819 INFO:journalctl@ceph.mon.c.smithi060.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-20T21:59:03.819 INFO:journalctl@ceph.mon.c.smithi060.stdout: legacy client fscid: -1 2024-09-20T21:59:03.819 INFO:journalctl@ceph.mon.c.smithi060.stdout: 2024-09-20T21:59:03.819 INFO:journalctl@ceph.mon.c.smithi060.stdout: No filesystems configured 2024-09-20T21:59:03.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-20T21:59:03.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-20T21:59:03.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-20T21:59:03.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-20T21:59:03.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-20T21:59:03.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-20T21:59:03.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-20T21:59:03.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-20T21:59:03.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-20T21:59:03.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-20T21:59:03.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:03.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T21:59:03.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mon.a calling monitor election 2024-09-20T21:59:03.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:03.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:03.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:03.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:03.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mon.b calling monitor election 2024-09-20T21:59:03.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:03.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:03.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:03.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:03.821 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-20T21:59:03.821 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: monmap epoch 2 2024-09-20T21:59:03.821 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: fsid 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:59:03.821 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: last_changed 2024-09-20T21:58:55.108527+0000 2024-09-20T21:59:03.821 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: created 2024-09-20T21:57:28.205024+0000 2024-09-20T21:59:03.821 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: min_mon_release 19 (squid) 2024-09-20T21:59:03.821 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: election_strategy: 1 2024-09-20T21:59:03.821 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-20T21:59:03.821 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: 1: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.b 2024-09-20T21:59:03.821 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: fsmap 2024-09-20T21:59:03.821 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: osdmap e4: 0 total, 0 up, 0 in 2024-09-20T21:59:03.821 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mgrmap e13: y(active, since 33s) 2024-09-20T21:59:03.822 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: overall HEALTH_OK 2024-09-20T21:59:03.822 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:03.822 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:03.822 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:03.822 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-20T21:59:03.822 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:03.822 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: Deploying daemon mon.c on smithi060 2024-09-20T21:59:03.822 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: from='client.? 172.21.15.67:0/2486260956' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-20T21:59:03.822 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:03.822 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:03.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: from='client.? 172.21.15.67:0/3513242279' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-20T21:59:03.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 ceph-mon[40290]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-20T21:59:03.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 bash[40252]: a7ce8d1ab286d47dc7f3f55c0f76949af7bdadf8d6bba8a5eae3301d4d36d636 2024-09-20T21:59:03.823 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:03 smithi060 systemd[1]: Started Ceph mon.c for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T21:59:04.612 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-20T21:59:04.613 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-20T21:59:04.871 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T21:59:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T21:59:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: mon.a calling monitor election 2024-09-20T21:59:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: mon.b calling monitor election 2024-09-20T21:59:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:08.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:08.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:08.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: mon.c calling monitor election 2024-09-20T21:59:08.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:08.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:08.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:08.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:08.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-20T21:59:08.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: monmap epoch 3 2024-09-20T21:59:08.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: fsid 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:59:08.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: last_changed 2024-09-20T21:59:03.416970+0000 2024-09-20T21:59:08.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: created 2024-09-20T21:57:28.205024+0000 2024-09-20T21:59:08.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: min_mon_release 19 (squid) 2024-09-20T21:59:08.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: election_strategy: 1 2024-09-20T21:59:08.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-20T21:59:08.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: 1: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.b 2024-09-20T21:59:08.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: 2: [v2:172.21.15.60:3301/0,v1:172.21.15.60:6790/0] mon.c 2024-09-20T21:59:08.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: fsmap 2024-09-20T21:59:08.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: osdmap e4: 0 total, 0 up, 0 in 2024-09-20T21:59:08.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: mgrmap e13: y(active, since 41s) 2024-09-20T21:59:08.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: overall HEALTH_OK 2024-09-20T21:59:08.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:08.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:08.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:08.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:08 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:08.781 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T21:59:08.781 INFO:teuthology.orchestra.run.smithi067.stdout:{"epoch":3,"fsid":"3df9647e-779b-11ef-baf5-efdc52797490","modified":"2024-09-20T21:59:03.416970Z","created":"2024-09-20T21:57:28.205024Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3300","nonce":0},{"type":"v1","addr":"172.21.15.60:6789","nonce":0}]},"addr":"172.21.15.60:6789/0","public_addr":"172.21.15.60:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:3300","nonce":0},{"type":"v1","addr":"172.21.15.67:6789","nonce":0}]},"addr":"172.21.15.67:6789/0","public_addr":"172.21.15.67:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:3301","nonce":0},{"type":"v1","addr":"172.21.15.60:6790","nonce":0}]},"addr":"172.21.15.60:6790/0","public_addr":"172.21.15.60:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-20T21:59:08.781 INFO:teuthology.orchestra.run.smithi067.stderr:dumped monmap epoch 3 2024-09-20T21:59:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T21:59:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: mon.a calling monitor election 2024-09-20T21:59:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:08.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: mon.b calling monitor election 2024-09-20T21:59:08.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:08.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:08.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:08.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: mon.c calling monitor election 2024-09-20T21:59:08.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:08.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:08.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:08.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:08.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-20T21:59:08.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: monmap epoch 3 2024-09-20T21:59:08.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: fsid 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:59:08.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: last_changed 2024-09-20T21:59:03.416970+0000 2024-09-20T21:59:08.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: created 2024-09-20T21:57:28.205024+0000 2024-09-20T21:59:08.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: min_mon_release 19 (squid) 2024-09-20T21:59:08.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: election_strategy: 1 2024-09-20T21:59:08.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-20T21:59:08.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: 1: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.b 2024-09-20T21:59:08.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: 2: [v2:172.21.15.60:3301/0,v1:172.21.15.60:6790/0] mon.c 2024-09-20T21:59:08.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: fsmap 2024-09-20T21:59:08.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: osdmap e4: 0 total, 0 up, 0 in 2024-09-20T21:59:08.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: mgrmap e13: y(active, since 41s) 2024-09-20T21:59:08.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: overall HEALTH_OK 2024-09-20T21:59:08.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:08.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:08.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:08.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:08.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:08.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T21:59:08.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: mon.a calling monitor election 2024-09-20T21:59:08.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T21:59:08.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:08.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: mon.b calling monitor election 2024-09-20T21:59:08.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:08.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:08.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:08.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: mon.c calling monitor election 2024-09-20T21:59:08.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:08.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:08.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:08.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:08.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-20T21:59:08.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: monmap epoch 3 2024-09-20T21:59:08.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: fsid 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:59:08.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: last_changed 2024-09-20T21:59:03.416970+0000 2024-09-20T21:59:08.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: created 2024-09-20T21:57:28.205024+0000 2024-09-20T21:59:08.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: min_mon_release 19 (squid) 2024-09-20T21:59:08.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: election_strategy: 1 2024-09-20T21:59:08.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: 0: [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] mon.a 2024-09-20T21:59:08.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: 1: [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] mon.b 2024-09-20T21:59:08.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: 2: [v2:172.21.15.60:3301/0,v1:172.21.15.60:6790/0] mon.c 2024-09-20T21:59:08.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: fsmap 2024-09-20T21:59:08.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: osdmap e4: 0 total, 0 up, 0 in 2024-09-20T21:59:08.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: mgrmap e13: y(active, since 41s) 2024-09-20T21:59:08.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: overall HEALTH_OK 2024-09-20T21:59:08.791 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:08.791 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:08.791 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:08.791 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:08 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:09.357 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-20T21:59:09.357 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-20T21:59:09.620 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T21:59:09.626 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:09 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:59:09.626 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:09 smithi067 ceph-mon[33511]: from='client.? 172.21.15.67:0/1218292705' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-20T21:59:09.626 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:09 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:09.742 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:09 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:59:09.742 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:09 smithi060 ceph-mon[29694]: from='client.? 172.21.15.67:0/1218292705' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-20T21:59:09.742 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:09 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:09.742 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:09 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:59:09.742 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:09 smithi060 ceph-mon[40290]: from='client.? 172.21.15.67:0/1218292705' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-20T21:59:09.742 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:09 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T21:59:10.284 INFO:teuthology.orchestra.run.smithi060.stdout:# minimal ceph.conf for 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:59:10.284 INFO:teuthology.orchestra.run.smithi060.stdout:[global] 2024-09-20T21:59:10.285 INFO:teuthology.orchestra.run.smithi060.stdout: fsid = 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T21:59:10.285 INFO:teuthology.orchestra.run.smithi060.stdout: mon_host = [v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0] [v2:172.21.15.67:3300/0,v1:172.21.15.67:6789/0] [v2:172.21.15.60:3301/0,v1:172.21.15.60:6790/0] 2024-09-20T21:59:10.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[29694]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:10.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:10.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:10.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:10.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:10.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:10.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:59:10.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[29694]: Updating smithi060:/etc/ceph/ceph.conf 2024-09-20T21:59:10.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[29694]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-20T21:59:10.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[29694]: Updating smithi067:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T21:59:10.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[29694]: Updating smithi060:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T21:59:10.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3346794260' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:10.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[40290]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:10.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:10.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:10.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:10.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:10.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:10.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:59:10.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[40290]: Updating smithi060:/etc/ceph/ceph.conf 2024-09-20T21:59:10.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[40290]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-20T21:59:10.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[40290]: Updating smithi067:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T21:59:10.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[40290]: Updating smithi060:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T21:59:10.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:10 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3346794260' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:10 smithi067 ceph-mon[33511]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:10 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:10.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:10 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:10.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:10 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:10.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:10 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:10.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:10 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:10.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:10 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:59:10.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:10 smithi067 ceph-mon[33511]: Updating smithi060:/etc/ceph/ceph.conf 2024-09-20T21:59:10.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:10 smithi067 ceph-mon[33511]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-20T21:59:10.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:10 smithi067 ceph-mon[33511]: Updating smithi067:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T21:59:10.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:10 smithi067 ceph-mon[33511]: Updating smithi060:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T21:59:10.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:10 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3346794260' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:10.952 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-20T21:59:10.952 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-20T21:59:10.952 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.conf 2024-09-20T21:59:10.981 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-20T21:59:10.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-20T21:59:11.047 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-20T21:59:11.047 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.conf 2024-09-20T21:59:11.075 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-20T21:59:11.075 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-20T21:59:11.141 INFO:tasks.cephadm:Adding mgr.y on smithi060 2024-09-20T21:59:11.141 INFO:tasks.cephadm:Adding mgr.x on smithi067 2024-09-20T21:59:11.142 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch apply mgr '2;smithi060=y;smithi067=x' 2024-09-20T21:59:11.426 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T21:59:11.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:11 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:11 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:11 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:11 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:11 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:11 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:11 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:11 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:11 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:11 smithi067 ceph-mon[33511]: Reconfiguring mon.a (unknown last config time)... 2024-09-20T21:59:11.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:11 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-20T21:59:11.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:11 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-20T21:59:11.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:11 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:11.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:11 smithi067 ceph-mon[33511]: Reconfiguring daemon mon.a on smithi060 2024-09-20T21:59:11.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:11 smithi067 ceph-mon[33511]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[29694]: Reconfiguring mon.a (unknown last config time)... 2024-09-20T21:59:11.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-20T21:59:11.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-20T21:59:11.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:11.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[29694]: Reconfiguring daemon mon.a on smithi060 2024-09-20T21:59:11.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[29694]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:11.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:11.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[40290]: Reconfiguring mon.a (unknown last config time)... 2024-09-20T21:59:11.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-20T21:59:11.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-20T21:59:11.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:11.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[40290]: Reconfiguring daemon mon.a on smithi060 2024-09-20T21:59:11.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:11 smithi060 ceph-mon[40290]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:12.087 INFO:teuthology.orchestra.run.smithi067.stdout:Scheduled mgr update... 2024-09-20T21:59:12.701 DEBUG:teuthology.orchestra.run.smithi067:mgr.x> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@mgr.x.service 2024-09-20T21:59:12.703 INFO:tasks.cephadm:Deploying OSDs... 2024-09-20T21:59:12.704 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-20T21:59:12.704 DEBUG:teuthology.orchestra.run.smithi060:> dd if=/scratch_devs of=/dev/stdout 2024-09-20T21:59:12.720 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-20T21:59:12.720 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_1 2024-09-20T21:59:12.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:12.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[40290]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi060=y;smithi067=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:59:12.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[40290]: Saving service mgr spec with placement smithi060=y;smithi067=x;count:2 2024-09-20T21:59:12.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:12.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:12.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:12.760 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[40290]: Reconfiguring mgr.y (unknown last config time)... 2024-09-20T21:59:12.760 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-20T21:59:12.760 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-20T21:59:12.760 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:12.760 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[40290]: Reconfiguring daemon mgr.y on smithi060 2024-09-20T21:59:12.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:12.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[29694]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi060=y;smithi067=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:59:12.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[29694]: Saving service mgr spec with placement smithi060=y;smithi067=x;count:2 2024-09-20T21:59:12.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:12.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:12.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:12.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[29694]: Reconfiguring mgr.y (unknown last config time)... 2024-09-20T21:59:12.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-20T21:59:12.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-20T21:59:12.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:12.761 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:12 smithi060 ceph-mon[29694]: Reconfiguring daemon mgr.y on smithi060 2024-09-20T21:59:12.777 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-20T21:59:12.778 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-20T21:59:12.778 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 975 Links: 1 2024-09-20T21:59:12.778 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-20T21:59:12.778 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-09-20T21:59:12.778 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-20 21:59:02.857671767 +0000 2024-09-20T21:59:12.778 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-20 21:54:37.555109998 +0000 2024-09-20T21:59:12.778 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-20 21:54:37.555109998 +0000 2024-09-20T21:59:12.778 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-09-20 21:54:37.555109998 +0000 2024-09-20T21:59:12.778 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-20T21:59:12.843 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-20T21:59:12.843 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-20T21:59:12.843 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000101417 s, 5.0 MB/s 2024-09-20T21:59:12.844 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-20T21:59:12.901 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_2 2024-09-20T21:59:12.957 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-20T21:59:12.957 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-20T21:59:12.957 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 960 Links: 1 2024-09-20T21:59:12.957 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-20T21:59:12.957 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-09-20T21:59:12.957 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-20 21:59:02.857671767 +0000 2024-09-20T21:59:12.957 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-20 21:54:37.554110007 +0000 2024-09-20T21:59:12.957 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-20 21:54:37.554110007 +0000 2024-09-20T21:59:12.958 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-09-20 21:54:37.554110007 +0000 2024-09-20T21:59:12.958 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-20T21:59:13.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:12 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:12 smithi067 ceph-mon[33511]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi060=y;smithi067=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:59:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:12 smithi067 ceph-mon[33511]: Saving service mgr spec with placement smithi060=y;smithi067=x;count:2 2024-09-20T21:59:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:12 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:12 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:12 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:12 smithi067 ceph-mon[33511]: Reconfiguring mgr.y (unknown last config time)... 2024-09-20T21:59:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:12 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-20T21:59:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:12 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-20T21:59:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:12 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:12 smithi067 ceph-mon[33511]: Reconfiguring daemon mgr.y on smithi060 2024-09-20T21:59:13.021 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-20T21:59:13.021 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-20T21:59:13.021 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000110996 s, 4.6 MB/s 2024-09-20T21:59:13.023 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-20T21:59:13.079 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_3 2024-09-20T21:59:13.135 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-20T21:59:13.135 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-20T21:59:13.136 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 966 Links: 1 2024-09-20T21:59:13.136 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-20T21:59:13.136 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-09-20T21:59:13.136 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-20 21:59:02.858671758 +0000 2024-09-20T21:59:13.136 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-20 21:54:37.555109998 +0000 2024-09-20T21:59:13.136 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-20 21:54:37.555109998 +0000 2024-09-20T21:59:13.136 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-09-20 21:54:37.555109998 +0000 2024-09-20T21:59:13.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-20T21:59:13.199 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-20T21:59:13.200 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-20T21:59:13.200 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 0.000105168 s, 4.9 MB/s 2024-09-20T21:59:13.201 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-20T21:59:13.258 DEBUG:teuthology.orchestra.run.smithi060:> stat /dev/vg_nvme/lv_4 2024-09-20T21:59:13.314 INFO:teuthology.orchestra.run.smithi060.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-20T21:59:13.314 INFO:teuthology.orchestra.run.smithi060.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-20T21:59:13.315 INFO:teuthology.orchestra.run.smithi060.stdout:Device: 5h/5d Inode: 978 Links: 1 2024-09-20T21:59:13.315 INFO:teuthology.orchestra.run.smithi060.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-20T21:59:13.315 INFO:teuthology.orchestra.run.smithi060.stdout:Context: system_u:object_r:device_t:s0 2024-09-20T21:59:13.315 INFO:teuthology.orchestra.run.smithi060.stdout:Access: 2024-09-20 21:59:02.858671758 +0000 2024-09-20T21:59:13.315 INFO:teuthology.orchestra.run.smithi060.stdout:Modify: 2024-09-20 21:54:37.555109998 +0000 2024-09-20T21:59:13.315 INFO:teuthology.orchestra.run.smithi060.stdout:Change: 2024-09-20 21:54:37.555109998 +0000 2024-09-20T21:59:13.315 INFO:teuthology.orchestra.run.smithi060.stdout: Birth: 2024-09-20 21:54:37.555109998 +0000 2024-09-20T21:59:13.315 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-20T21:59:13.378 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records in 2024-09-20T21:59:13.378 INFO:teuthology.orchestra.run.smithi060.stderr:1+0 records out 2024-09-20T21:59:13.378 INFO:teuthology.orchestra.run.smithi060.stderr:512 bytes copied, 9.2041e-05 s, 5.6 MB/s 2024-09-20T21:59:13.379 DEBUG:teuthology.orchestra.run.smithi060:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-20T21:59:13.438 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-20T21:59:13.438 DEBUG:teuthology.orchestra.run.smithi067:> dd if=/scratch_devs of=/dev/stdout 2024-09-20T21:59:13.458 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-20T21:59:13.458 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_1 2024-09-20T21:59:13.517 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-20T21:59:13.517 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-20T21:59:13.517 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 958 Links: 1 2024-09-20T21:59:13.517 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-20T21:59:13.517 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-09-20T21:59:13.518 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-09-20 21:58:54.028826622 +0000 2024-09-20T21:59:13.518 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-09-20 21:54:38.938606396 +0000 2024-09-20T21:59:13.518 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-09-20 21:54:38.938606396 +0000 2024-09-20T21:59:13.518 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-09-20 21:54:38.938606396 +0000 2024-09-20T21:59:13.518 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-20T21:59:13.586 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-09-20T21:59:13.587 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-09-20T21:59:13.587 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000288364 s, 1.8 MB/s 2024-09-20T21:59:13.588 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-20T21:59:13.648 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_2 2024-09-20T21:59:13.706 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-20T21:59:13.706 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-20T21:59:13.706 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 962 Links: 1 2024-09-20T21:59:13.706 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-20T21:59:13.706 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-09-20T21:59:13.706 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-09-20 21:58:54.029826600 +0000 2024-09-20T21:59:13.706 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-09-20 21:54:38.938606396 +0000 2024-09-20T21:59:13.706 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-09-20 21:54:38.938606396 +0000 2024-09-20T21:59:13.706 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-09-20 21:54:38.938606396 +0000 2024-09-20T21:59:13.707 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-20T21:59:13.774 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-09-20T21:59:13.774 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-09-20T21:59:13.774 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000309688 s, 1.7 MB/s 2024-09-20T21:59:13.776 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-20T21:59:13.835 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_3 2024-09-20T21:59:13.892 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-20T21:59:13.892 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-20T21:59:13.893 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 966 Links: 1 2024-09-20T21:59:13.893 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-20T21:59:13.893 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-09-20T21:59:13.893 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-09-20 21:58:54.029826600 +0000 2024-09-20T21:59:13.893 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-09-20 21:54:38.938606396 +0000 2024-09-20T21:59:13.893 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-09-20 21:54:38.938606396 +0000 2024-09-20T21:59:13.893 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-09-20 21:54:38.938606396 +0000 2024-09-20T21:59:13.893 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-20T21:59:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-09-20T21:59:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-09-20T21:59:13.960 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000233629 s, 2.2 MB/s 2024-09-20T21:59:13.961 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-20T21:59:14.020 DEBUG:teuthology.orchestra.run.smithi067:> stat /dev/vg_nvme/lv_4 2024-09-20T21:59:14.077 INFO:teuthology.orchestra.run.smithi067.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-20T21:59:14.077 INFO:teuthology.orchestra.run.smithi067.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-20T21:59:14.077 INFO:teuthology.orchestra.run.smithi067.stdout:Device: 5h/5d Inode: 984 Links: 1 2024-09-20T21:59:14.077 INFO:teuthology.orchestra.run.smithi067.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-20T21:59:14.077 INFO:teuthology.orchestra.run.smithi067.stdout:Context: system_u:object_r:device_t:s0 2024-09-20T21:59:14.077 INFO:teuthology.orchestra.run.smithi067.stdout:Access: 2024-09-20 21:58:54.029826600 +0000 2024-09-20T21:59:14.077 INFO:teuthology.orchestra.run.smithi067.stdout:Modify: 2024-09-20 21:54:38.941606328 +0000 2024-09-20T21:59:14.078 INFO:teuthology.orchestra.run.smithi067.stdout:Change: 2024-09-20 21:54:38.941606328 +0000 2024-09-20T21:59:14.078 INFO:teuthology.orchestra.run.smithi067.stdout: Birth: 2024-09-20 21:54:38.941606328 +0000 2024-09-20T21:59:14.078 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-20T21:59:14.142 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records in 2024-09-20T21:59:14.142 INFO:teuthology.orchestra.run.smithi067.stderr:1+0 records out 2024-09-20T21:59:14.143 INFO:teuthology.orchestra.run.smithi067.stderr:512 bytes copied, 0.000225415 s, 2.3 MB/s 2024-09-20T21:59:14.144 DEBUG:teuthology.orchestra.run.smithi067:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-20T21:59:14.201 INFO:tasks.cephadm:Deploying osd.0 on smithi060 with /dev/vg_nvme/lv_4... 2024-09-20T21:59:14.201 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-20T21:59:14.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:14 smithi067 ceph-mon[33511]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:14.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:14 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:14.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:14 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:14.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:14 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-20T21:59:14.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:14 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-20T21:59:14.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:14 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:14.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:14 smithi060 ceph-mon[29694]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:14.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:14 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:14.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:14 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:14.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:14 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-20T21:59:14.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:14 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-20T21:59:14.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:14 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:14.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:14 smithi060 ceph-mon[40290]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:14.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:14 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:14.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:14 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:14.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:14 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-20T21:59:14.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:14 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-20T21:59:14.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:14 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:14.949 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T21:59:15.202 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:15 smithi060 ceph-mon[29694]: Reconfiguring mon.c (monmap changed)... 2024-09-20T21:59:15.202 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:15 smithi060 ceph-mon[29694]: Reconfiguring daemon mon.c on smithi060 2024-09-20T21:59:15.202 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:15 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:15.202 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:15 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:15.202 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:15 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-20T21:59:15.202 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:15 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-20T21:59:15.202 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:15 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:15.202 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:15 smithi060 ceph-mon[40290]: Reconfiguring mon.c (monmap changed)... 2024-09-20T21:59:15.202 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:15 smithi060 ceph-mon[40290]: Reconfiguring daemon mon.c on smithi060 2024-09-20T21:59:15.203 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:15 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:15.203 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:15 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:15.203 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:15 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-20T21:59:15.203 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:15 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-20T21:59:15.203 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:15 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:15.389 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:15 smithi067 ceph-mon[33511]: Reconfiguring mon.c (monmap changed)... 2024-09-20T21:59:15.389 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:15 smithi067 ceph-mon[33511]: Reconfiguring daemon mon.c on smithi060 2024-09-20T21:59:15.389 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:15 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:15.390 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:15 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:15.390 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:15 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-20T21:59:15.390 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:15 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-20T21:59:15.390 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:15 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:16.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:16 smithi060 ceph-mon[40290]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:16.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:16 smithi060 ceph-mon[40290]: Reconfiguring mon.b (monmap changed)... 2024-09-20T21:59:16.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:16 smithi060 ceph-mon[40290]: Reconfiguring daemon mon.b on smithi067 2024-09-20T21:59:16.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:16 smithi060 ceph-mon[29694]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:16.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:16 smithi060 ceph-mon[29694]: Reconfiguring mon.b (monmap changed)... 2024-09-20T21:59:16.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:16 smithi060 ceph-mon[29694]: Reconfiguring daemon mon.b on smithi067 2024-09-20T21:59:16.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:16 smithi067 ceph-mon[33511]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:16.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:16 smithi067 ceph-mon[33511]: Reconfiguring mon.b (monmap changed)... 2024-09-20T21:59:16.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:16 smithi067 ceph-mon[33511]: Reconfiguring daemon mon.b on smithi067 2024-09-20T21:59:17.448 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:17.448 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:17.448 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:59:17.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:17.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:59:17.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:17.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-20T21:59:17.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-20T21:59:17.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-20T21:59:17.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:17.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:17 smithi067 ceph-mon[33511]: Deploying daemon mgr.x on smithi067 2024-09-20T21:59:17.478 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:59:17.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:17.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:17.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:59:17.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:17.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:59:17.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:17.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-20T21:59:17.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-20T21:59:17.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-20T21:59:17.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:17.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[40290]: Deploying daemon mgr.x on smithi067 2024-09-20T21:59:17.503 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:17.503 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:17.503 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:59:17.503 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:17.503 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:59:17.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:17.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-20T21:59:17.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-20T21:59:17.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-20T21:59:17.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:17.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:17 smithi060 ceph-mon[29694]: Deploying daemon mgr.x on smithi067 2024-09-20T21:59:17.505 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi060:vg_nvme/lv_4 2024-09-20T21:59:17.756 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T21:59:18.596 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:18 smithi067 ceph-mon[33511]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:18.596 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:18 smithi067 podman[35474]: 2024-09-20 21:59:18.385149546 +0000 UTC m=+0.016157683 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5 2024-09-20T21:59:18.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:18 smithi060 ceph-mon[29694]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:18.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:18 smithi060 ceph-mon[40290]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:19.013 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:18 smithi067 podman[35474]: 2024-09-20 21:59:18.693303486 +0000 UTC m=+0.324311630 container init 6062092286ad3070224d6bb5a8af2c1371cb00ccbb62cff57195475007ba5181 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=squid-9e2ac70, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-20T21:59:19.014 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:18 smithi067 podman[35474]: 2024-09-20 21:59:18.698100976 +0000 UTC m=+0.329109113 container start 6062092286ad3070224d6bb5a8af2c1371cb00ccbb62cff57195475007ba5181 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-9e2ac70, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T21:59:19.014 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:18 smithi067 ceph-mgr[35512]: -- 172.21.15.67:0/771308004 <== mon.1 v2:172.21.15.67:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ac638e90e0 con 0x55ac638c7400 2024-09-20T21:59:19.014 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:18 smithi067 bash[35474]: 6062092286ad3070224d6bb5a8af2c1371cb00ccbb62cff57195475007ba5181 2024-09-20T21:59:19.014 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:18 smithi067 systemd[1]: Started Ceph mgr.x for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T21:59:19.014 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:18 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:18.822+0000 7f61b45c7040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-20T21:59:19.014 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:18 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:18.884+0000 7f61b45c7040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-20T21:59:19.369 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:19.044+0000 7f61b45c7040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-20T21:59:19.369 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:19.114+0000 7f61b45c7040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-20T21:59:19.369 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:19.287+0000 7f61b45c7040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-20T21:59:19.732 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:19.439+0000 7f61b45c7040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-20T21:59:19.733 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:19 smithi067 ceph-mon[33511]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:59:19.733 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:19 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T21:59:19.733 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:19 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T21:59:19.733 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:19 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:19.733 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:19 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:19.733 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:19 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:19.733 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:19 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:19.733 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:19 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:19.733 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:19 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:59:19.743 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:19 smithi060 ceph-mon[40290]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:59:19.743 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:19 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T21:59:19.743 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:19 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T21:59:19.743 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:19 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:19.743 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:19 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:19.743 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:19 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:19.743 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:19 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:19.743 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:19 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:19.743 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:19 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:59:19.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:19 smithi060 ceph-mon[29694]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:59:19.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:19 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T21:59:19.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:19 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T21:59:19.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:19 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:19.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:19 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:19.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:19 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:19.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:19 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:19.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:19 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:19.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:19 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:59:20.013 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:19.776+0000 7f61b45c7040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-20T21:59:20.013 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:19.840+0000 7f61b45c7040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-20T21:59:20.370 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:20 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:20.038+0000 7f61b45c7040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-20T21:59:20.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:20 smithi067 ceph-mon[33511]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:20.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:20 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:20.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:20 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:20.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:20 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:20.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:20 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:59:20.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:20 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:20.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:20 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:59:20.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:20 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:20.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:20 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:59:20.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:20 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:20.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[40290]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:20.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:20.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:20.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:20.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:59:20.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:20.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:59:20.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:20.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:59:20.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:20.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[29694]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:20.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:20.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:20.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:20.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:59:20.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:20.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:59:20.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:20.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:59:20.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:20 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:21.263 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:20 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:20.968+0000 7f61b45c7040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-20T21:59:21.263 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:21 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:21.039+0000 7f61b45c7040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-20T21:59:21.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:21 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/695170344' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0997c01a-a9bd-421e-abcc-fbcf7abe8512"}]: dispatch 2024-09-20T21:59:21.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:21 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/695170344' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0997c01a-a9bd-421e-abcc-fbcf7abe8512"}]': finished 2024-09-20T21:59:21.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:21 smithi067 ceph-mon[33511]: osdmap e5: 1 total, 0 up, 1 in 2024-09-20T21:59:21.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:21 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T21:59:21.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:21 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3423681851' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T21:59:21.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:21 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/695170344' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0997c01a-a9bd-421e-abcc-fbcf7abe8512"}]: dispatch 2024-09-20T21:59:21.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:21 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/695170344' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0997c01a-a9bd-421e-abcc-fbcf7abe8512"}]': finished 2024-09-20T21:59:21.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:21 smithi060 ceph-mon[40290]: osdmap e5: 1 total, 0 up, 1 in 2024-09-20T21:59:21.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:21 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T21:59:21.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:21 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3423681851' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T21:59:21.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:21 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/695170344' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0997c01a-a9bd-421e-abcc-fbcf7abe8512"}]: dispatch 2024-09-20T21:59:21.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:21 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/695170344' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0997c01a-a9bd-421e-abcc-fbcf7abe8512"}]': finished 2024-09-20T21:59:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:21 smithi060 ceph-mon[29694]: osdmap e5: 1 total, 0 up, 1 in 2024-09-20T21:59:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:21 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T21:59:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:21 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3423681851' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T21:59:22.069 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:21 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:21.798+0000 7f61b45c7040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-20T21:59:22.069 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:21 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:21.919+0000 7f61b45c7040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-20T21:59:22.360 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:22.068+0000 7f61b45c7040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-20T21:59:22.360 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:22.132+0000 7f61b45c7040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-20T21:59:22.360 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:22.288+0000 7f61b45c7040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-20T21:59:22.360 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:22.359+0000 7f61b45c7040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-20T21:59:22.656 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:22 smithi067 ceph-mon[33511]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:22.656 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:22 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:22.657 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:22.422+0000 7f61b45c7040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-20T21:59:22.657 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:22.587+0000 7f61b45c7040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-20T21:59:22.657 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:22.655+0000 7f61b45c7040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-20T21:59:22.687 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:22 smithi060 ceph-mon[29694]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:22.688 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:22 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:22.688 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:22 smithi060 ceph-mon[40290]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:22.688 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:22 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:22.936 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:22.724+0000 7f61b45c7040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-20T21:59:22.936 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:22.811+0000 7f61b45c7040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-20T21:59:22.936 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:22.873+0000 7f61b45c7040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-20T21:59:22.936 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:22.934+0000 7f61b45c7040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-20T21:59:24.013 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 21:59:23 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T21:59:23.762+0000 7f61b45c7040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-20T21:59:24.719 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:24 smithi060 ceph-mon[29694]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:24.719 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:24 smithi060 ceph-mon[29694]: Standby manager daemon x started 2024-09-20T21:59:24.719 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:24 smithi060 ceph-mon[29694]: from='mgr.? 172.21.15.67:0/682845600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-20T21:59:24.719 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:24 smithi060 ceph-mon[29694]: from='mgr.? 172.21.15.67:0/682845600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-20T21:59:24.719 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:24 smithi060 ceph-mon[29694]: from='mgr.? 172.21.15.67:0/682845600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-20T21:59:24.719 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:24 smithi060 ceph-mon[29694]: from='mgr.? 172.21.15.67:0/682845600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-20T21:59:24.719 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:24 smithi060 ceph-mon[40290]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:24.719 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:24 smithi060 ceph-mon[40290]: Standby manager daemon x started 2024-09-20T21:59:24.720 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:24 smithi060 ceph-mon[40290]: from='mgr.? 172.21.15.67:0/682845600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-20T21:59:24.720 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:24 smithi060 ceph-mon[40290]: from='mgr.? 172.21.15.67:0/682845600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-20T21:59:24.720 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:24 smithi060 ceph-mon[40290]: from='mgr.? 172.21.15.67:0/682845600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-20T21:59:24.720 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:24 smithi060 ceph-mon[40290]: from='mgr.? 172.21.15.67:0/682845600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-20T21:59:24.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:24 smithi067 ceph-mon[33511]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:24.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:24 smithi067 ceph-mon[33511]: Standby manager daemon x started 2024-09-20T21:59:24.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:24 smithi067 ceph-mon[33511]: from='mgr.? 172.21.15.67:0/682845600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-20T21:59:24.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:24 smithi067 ceph-mon[33511]: from='mgr.? 172.21.15.67:0/682845600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-20T21:59:24.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:24 smithi067 ceph-mon[33511]: from='mgr.? 172.21.15.67:0/682845600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-20T21:59:24.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:24 smithi067 ceph-mon[33511]: from='mgr.? 172.21.15.67:0/682845600' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-20T21:59:25.738 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:25 smithi060 ceph-mon[29694]: mgrmap e14: y(active, since 57s), standbys: x 2024-09-20T21:59:25.739 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:25 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-20T21:59:25.739 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:25 smithi060 ceph-mon[40290]: mgrmap e14: y(active, since 57s), standbys: x 2024-09-20T21:59:25.739 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:25 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-20T21:59:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:25 smithi067 ceph-mon[33511]: mgrmap e14: y(active, since 57s), standbys: x 2024-09-20T21:59:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:25 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-20T21:59:26.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:26 smithi060 ceph-mon[40290]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:26 smithi060 ceph-mon[29694]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:26 smithi067 ceph-mon[33511]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:28.747 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:28 smithi060 ceph-mon[40290]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:28.747 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:28 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-20T21:59:28.747 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:28 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:28.748 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:28 smithi060 ceph-mon[29694]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:28.748 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:28 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-20T21:59:28.748 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:28 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:28 smithi067 ceph-mon[33511]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:28 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-20T21:59:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:28 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:29.474 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:29 smithi060 ceph-mon[29694]: Deploying daemon osd.0 on smithi060 2024-09-20T21:59:29.475 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:29 smithi060 ceph-mon[40290]: Deploying daemon osd.0 on smithi060 2024-09-20T21:59:29.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:29 smithi067 ceph-mon[33511]: Deploying daemon osd.0 on smithi060 2024-09-20T21:59:30.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:30 smithi060 ceph-mon[29694]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:30.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:30 smithi060 ceph-mon[40290]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:30 smithi067 ceph-mon[33511]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:32.687 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:32 smithi060 ceph-mon[29694]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:32.688 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:32 smithi060 ceph-mon[40290]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:32.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:32 smithi067 ceph-mon[33511]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:34.659 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:34 smithi060 ceph-mon[29694]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:34.659 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:34 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:59:34.660 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:34 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:34.660 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:34 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:34.660 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:34 smithi060 ceph-mon[40290]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:34.660 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:34 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:59:34.660 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:34 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:34.660 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:34 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:34.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:34 smithi067 ceph-mon[33511]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:34.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:34 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:59:34.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:34 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:34.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:34 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:36 smithi060 ceph-mon[29694]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:36 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:36 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:36.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:36 smithi060 ceph-mon[40290]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:36.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:36 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:36.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:36 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:36.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:36 smithi067 ceph-mon[33511]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:36.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:36 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:36.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:36 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:37.674 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:37 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:37.674 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:37 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:59:37.674 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:37 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:37.674 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:37 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:37.674 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:37 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:59:37.674 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:37 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:37.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:37 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:37.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:37 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:59:37.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:37 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:37.977 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 0 on host 'smithi060' 2024-09-20T21:59:38.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:38 smithi060 ceph-mon[29694]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:38.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:38 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:59:38.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:38 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:38.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:38 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:38.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:38 smithi060 ceph-mon[29694]: from='osd.0 [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-20T21:59:38.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:38 smithi060 ceph-mon[40290]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:38.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:38 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:59:38.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:38 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:38.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:38 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:38.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:38 smithi060 ceph-mon[40290]: from='osd.0 [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-20T21:59:38.598 DEBUG:teuthology.orchestra.run.smithi060:osd.0> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.0.service 2024-09-20T21:59:38.600 INFO:tasks.cephadm:Deploying osd.1 on smithi060 with /dev/vg_nvme/lv_3... 2024-09-20T21:59:38.601 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-20T21:59:38.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:38 smithi067 ceph-mon[33511]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:38.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:38 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T21:59:38.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:38 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:38.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:38 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:38.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:38 smithi067 ceph-mon[33511]: from='osd.0 [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-20T21:59:39.424 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T21:59:40.212 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:39 smithi060 ceph-mon[29694]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:40.212 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:39 smithi060 ceph-mon[29694]: from='osd.0 [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-20T21:59:40.212 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:39 smithi060 ceph-mon[29694]: osdmap e6: 1 total, 0 up, 1 in 2024-09-20T21:59:40.212 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:39 smithi060 ceph-mon[29694]: from='osd.0 [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-09-20T21:59:40.212 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:39 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T21:59:40.213 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:39 smithi060 ceph-mon[40290]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:40.213 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:39 smithi060 ceph-mon[40290]: from='osd.0 [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-20T21:59:40.213 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:39 smithi060 ceph-mon[40290]: osdmap e6: 1 total, 0 up, 1 in 2024-09-20T21:59:40.213 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:39 smithi060 ceph-mon[40290]: from='osd.0 [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-09-20T21:59:40.213 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:39 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T21:59:40.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:39 smithi067 ceph-mon[33511]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:40.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:39 smithi067 ceph-mon[33511]: from='osd.0 [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-20T21:59:40.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:39 smithi067 ceph-mon[33511]: osdmap e6: 1 total, 0 up, 1 in 2024-09-20T21:59:40.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:39 smithi067 ceph-mon[33511]: from='osd.0 [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-09-20T21:59:40.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:39 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T21:59:40.523 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 21:59:40 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-0[44430]: 2024-09-20T21:59:40.241+0000 7f4469842640 -1 osd.0 0 waiting for initial osdmap 2024-09-20T21:59:40.524 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 21:59:40 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-0[44430]: 2024-09-20T21:59:40.243+0000 7f4460e4e640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T21:59:41.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:40 smithi067 ceph-mon[33511]: from='osd.0 [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-20T21:59:41.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:40 smithi067 ceph-mon[33511]: osdmap e7: 1 total, 0 up, 1 in 2024-09-20T21:59:41.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:40 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T21:59:41.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:40 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T21:59:41.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:40 smithi067 ceph-mon[33511]: from='osd.0 [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550]' entity='osd.0' 2024-09-20T21:59:41.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:40 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T21:59:41.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:40 smithi060 ceph-mon[29694]: from='osd.0 [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-20T21:59:41.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:40 smithi060 ceph-mon[29694]: osdmap e7: 1 total, 0 up, 1 in 2024-09-20T21:59:41.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:40 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T21:59:41.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:40 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T21:59:41.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:40 smithi060 ceph-mon[29694]: from='osd.0 [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550]' entity='osd.0' 2024-09-20T21:59:41.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:40 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T21:59:41.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:40 smithi060 ceph-mon[40290]: from='osd.0 [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-20T21:59:41.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:40 smithi060 ceph-mon[40290]: osdmap e7: 1 total, 0 up, 1 in 2024-09-20T21:59:41.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:40 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T21:59:41.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:40 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T21:59:41.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:40 smithi060 ceph-mon[40290]: from='osd.0 [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550]' entity='osd.0' 2024-09-20T21:59:41.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:40 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T21:59:42.236 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:41 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T21:59:42.236 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:41 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T21:59:42.236 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:41 smithi060 ceph-mon[29694]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:42.237 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:41 smithi060 ceph-mon[29694]: osd.0 [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550] boot 2024-09-20T21:59:42.237 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:41 smithi060 ceph-mon[29694]: osdmap e8: 1 total, 1 up, 1 in 2024-09-20T21:59:42.237 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:41 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T21:59:42.237 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:41 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T21:59:42.237 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:41 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T21:59:42.237 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:41 smithi060 ceph-mon[40290]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:42.237 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:41 smithi060 ceph-mon[40290]: osd.0 [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550] boot 2024-09-20T21:59:42.237 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:41 smithi060 ceph-mon[40290]: osdmap e8: 1 total, 1 up, 1 in 2024-09-20T21:59:42.237 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:41 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T21:59:42.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:41 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T21:59:42.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:41 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T21:59:42.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:41 smithi067 ceph-mon[33511]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-20T21:59:42.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:41 smithi067 ceph-mon[33511]: osd.0 [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550] boot 2024-09-20T21:59:42.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:41 smithi067 ceph-mon[33511]: osdmap e8: 1 total, 1 up, 1 in 2024-09-20T21:59:42.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:41 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T21:59:43.440 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:43 smithi060 ceph-mon[29694]: Detected new or changed devices on smithi060 2024-09-20T21:59:43.440 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:43 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:43.440 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:43 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:43.441 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:43 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-20T21:59:43.441 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:43 smithi060 ceph-mon[29694]: Adjusting osd_memory_target on smithi060 to 16068M 2024-09-20T21:59:43.441 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:43 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:43.441 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:43 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:43.441 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:43 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:59:43.441 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:43 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:43.442 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:43 smithi060 ceph-mon[40290]: Detected new or changed devices on smithi060 2024-09-20T21:59:43.442 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:43 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:43.442 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:43 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:43.442 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:43 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-20T21:59:43.442 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:43 smithi060 ceph-mon[40290]: Adjusting osd_memory_target on smithi060 to 16068M 2024-09-20T21:59:43.443 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:43 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:43.443 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:43 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:43.443 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:43 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:59:43.443 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:43 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:43.444 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T21:59:43.470 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi060:vg_nvme/lv_3 2024-09-20T21:59:43.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:43 smithi067 ceph-mon[33511]: Detected new or changed devices on smithi060 2024-09-20T21:59:43.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:43 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:43.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:43 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:43.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:43 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-20T21:59:43.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:43 smithi067 ceph-mon[33511]: Adjusting osd_memory_target on smithi060 to 16068M 2024-09-20T21:59:43.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:43 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:43.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:43 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:43.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:43 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T21:59:43.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:43 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T21:59:43.723 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T21:59:44.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:44 smithi060 ceph-mon[29694]: pgmap v36: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:44.101 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:44 smithi060 ceph-mon[29694]: osdmap e9: 1 total, 1 up, 1 in 2024-09-20T21:59:44.101 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:44 smithi060 ceph-mon[40290]: pgmap v36: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:44.102 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:44 smithi060 ceph-mon[40290]: osdmap e9: 1 total, 1 up, 1 in 2024-09-20T21:59:44.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:44 smithi067 ceph-mon[33511]: pgmap v36: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:44.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:44 smithi067 ceph-mon[33511]: osdmap e9: 1 total, 1 up, 1 in 2024-09-20T21:59:45.500 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:45 smithi060 ceph-mon[29694]: from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:59:45.500 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:45 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T21:59:45.500 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:45 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T21:59:45.500 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:45 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:45.501 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:45 smithi060 ceph-mon[40290]: from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:59:45.501 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:45 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T21:59:45.501 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:45 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T21:59:45.501 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:45 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:45.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:45 smithi067 ceph-mon[33511]: from='client.14238 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T21:59:45.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:45 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T21:59:45.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:45 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T21:59:45.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:45 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:46.470 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:46 smithi060 ceph-mon[29694]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:46.471 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:46 smithi060 ceph-mon[40290]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:46.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:46 smithi067 ceph-mon[33511]: pgmap v38: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:47.379 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:47 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2742894961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c04bdcb1-d76b-47c8-a38b-c02043df6ba8"}]: dispatch 2024-09-20T21:59:47.379 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:47 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2742894961' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c04bdcb1-d76b-47c8-a38b-c02043df6ba8"}]': finished 2024-09-20T21:59:47.379 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:47 smithi060 ceph-mon[29694]: osdmap e10: 2 total, 1 up, 2 in 2024-09-20T21:59:47.379 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:47 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T21:59:47.379 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:47 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2742894961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c04bdcb1-d76b-47c8-a38b-c02043df6ba8"}]: dispatch 2024-09-20T21:59:47.379 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:47 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2742894961' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c04bdcb1-d76b-47c8-a38b-c02043df6ba8"}]': finished 2024-09-20T21:59:47.380 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:47 smithi060 ceph-mon[40290]: osdmap e10: 2 total, 1 up, 2 in 2024-09-20T21:59:47.380 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:47 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T21:59:47.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:47 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2742894961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c04bdcb1-d76b-47c8-a38b-c02043df6ba8"}]: dispatch 2024-09-20T21:59:47.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:47 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2742894961' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c04bdcb1-d76b-47c8-a38b-c02043df6ba8"}]': finished 2024-09-20T21:59:47.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:47 smithi067 ceph-mon[33511]: osdmap e10: 2 total, 1 up, 2 in 2024-09-20T21:59:47.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:47 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T21:59:48.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:48 smithi067 ceph-mon[33511]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:48.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:48 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3176482765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T21:59:48.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:48 smithi060 ceph-mon[29694]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:48.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:48 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3176482765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T21:59:48.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:48 smithi060 ceph-mon[40290]: pgmap v39: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:48.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:48 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3176482765' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T21:59:50.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:50 smithi067 ceph-mon[33511]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:50.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:50 smithi060 ceph-mon[29694]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:50.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:50 smithi060 ceph-mon[40290]: pgmap v41: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:52.321 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:52 smithi060 ceph-mon[40290]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:52.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:52 smithi060 ceph-mon[29694]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:52.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:52 smithi067 ceph-mon[33511]: pgmap v42: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:54.356 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:54 smithi060 ceph-mon[29694]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:54.356 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:54 smithi060 ceph-mon[40290]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:54.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:54 smithi067 ceph-mon[33511]: pgmap v43: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:55.512 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:55 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-20T21:59:55.513 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:55 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:55.513 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:55 smithi060 ceph-mon[40290]: Deploying daemon osd.1 on smithi060 2024-09-20T21:59:55.513 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:55 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-20T21:59:55.513 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:55 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:55.514 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:55 smithi060 ceph-mon[29694]: Deploying daemon osd.1 on smithi060 2024-09-20T21:59:55.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-20T21:59:55.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T21:59:55.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:55 smithi067 ceph-mon[33511]: Deploying daemon osd.1 on smithi060 2024-09-20T21:59:56.356 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:56 smithi060 ceph-mon[29694]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:56.357 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:56 smithi060 ceph-mon[40290]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:56.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:56 smithi067 ceph-mon[33511]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:58.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 21:59:58 smithi060 ceph-mon[29694]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:58.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 21:59:58 smithi060 ceph-mon[40290]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T21:59:58.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 21:59:58 smithi067 ceph-mon[33511]: pgmap v45: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:00.250 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:00 smithi060 ceph-mon[29694]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:00.250 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:00 smithi060 ceph-mon[29694]: overall HEALTH_OK 2024-09-20T22:00:00.251 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:00 smithi060 ceph-mon[40290]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:00.251 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:00 smithi060 ceph-mon[40290]: overall HEALTH_OK 2024-09-20T22:00:00.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:00 smithi067 ceph-mon[33511]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:00.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:00 smithi067 ceph-mon[33511]: overall HEALTH_OK 2024-09-20T22:00:01.192 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:01 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:00:01.192 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:01 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:01.192 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:01 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:01.192 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:01 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:00:01.192 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:01 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:01.192 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:01 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:01.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:01 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:00:01.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:01 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:01.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:01 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:02.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:02 smithi060 ceph-mon[29694]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:02.415 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:02 smithi060 ceph-mon[40290]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:02.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:02 smithi067 ceph-mon[33511]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:04.423 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:04 smithi060 ceph-mon[29694]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:04.423 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:04 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:04.423 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:04 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:04.423 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:04 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:04.423 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:04 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:00:04.424 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:04 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:04.424 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:04 smithi060 ceph-mon[40290]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:04.424 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:04 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:04.424 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:04 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:04.424 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:04 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:04.424 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:04 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:00:04.424 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:04 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:04.466 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 1 on host 'smithi060' 2024-09-20T22:00:04.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:04 smithi067 ceph-mon[33511]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:04.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:04 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:04.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:04 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:04.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:04 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:04.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:04 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:00:04.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:04 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:05.080 DEBUG:teuthology.orchestra.run.smithi060:osd.1> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.1.service 2024-09-20T22:00:05.082 INFO:tasks.cephadm:Deploying osd.2 on smithi060 with /dev/vg_nvme/lv_2... 2024-09-20T22:00:05.082 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-20T22:00:05.320 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:05 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:00:05.320 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:05 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:05.320 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:05 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:05.320 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:05 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:00:05.320 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:05 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:05.320 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:05 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:05.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:05 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:00:05.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:05 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:05.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:05 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:05.606 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:00:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1[50412]: 2024-09-20T22:00:05.519+0000 7f6bd9264640 -1 osd.1 0 log_to_monitors true 2024-09-20T22:00:05.908 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:00:06.721 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:06 smithi060 ceph-mon[40290]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:06.721 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:06 smithi060 ceph-mon[40290]: from='osd.1 [v2:172.21.15.60:6810/145283513,v1:172.21.15.60:6811/145283513]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-20T22:00:06.721 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:06 smithi060 ceph-mon[29694]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:06.721 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:06 smithi060 ceph-mon[29694]: from='osd.1 [v2:172.21.15.60:6810/145283513,v1:172.21.15.60:6811/145283513]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-20T22:00:06.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:06 smithi067 ceph-mon[33511]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:06.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:06 smithi067 ceph-mon[33511]: from='osd.1 [v2:172.21.15.60:6810/145283513,v1:172.21.15.60:6811/145283513]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-20T22:00:07.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:07 smithi067 ceph-mon[33511]: from='osd.1 [v2:172.21.15.60:6810/145283513,v1:172.21.15.60:6811/145283513]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-20T22:00:07.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:07 smithi067 ceph-mon[33511]: osdmap e11: 2 total, 1 up, 2 in 2024-09-20T22:00:07.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:07 smithi067 ceph-mon[33511]: from='osd.1 [v2:172.21.15.60:6810/145283513,v1:172.21.15.60:6811/145283513]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-09-20T22:00:07.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:07 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:00:07.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:07 smithi067 ceph-mon[33511]: from='osd.1 [v2:172.21.15.60:6810/145283513,v1:172.21.15.60:6811/145283513]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-20T22:00:07.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:07 smithi060 ceph-mon[29694]: from='osd.1 [v2:172.21.15.60:6810/145283513,v1:172.21.15.60:6811/145283513]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-20T22:00:07.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:07 smithi060 ceph-mon[29694]: osdmap e11: 2 total, 1 up, 2 in 2024-09-20T22:00:07.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:07 smithi060 ceph-mon[29694]: from='osd.1 [v2:172.21.15.60:6810/145283513,v1:172.21.15.60:6811/145283513]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-09-20T22:00:07.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:07 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:00:07.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:07 smithi060 ceph-mon[29694]: from='osd.1 [v2:172.21.15.60:6810/145283513,v1:172.21.15.60:6811/145283513]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-20T22:00:07.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:07 smithi060 ceph-mon[40290]: from='osd.1 [v2:172.21.15.60:6810/145283513,v1:172.21.15.60:6811/145283513]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-20T22:00:07.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:07 smithi060 ceph-mon[40290]: osdmap e11: 2 total, 1 up, 2 in 2024-09-20T22:00:07.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:07 smithi060 ceph-mon[40290]: from='osd.1 [v2:172.21.15.60:6810/145283513,v1:172.21.15.60:6811/145283513]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-09-20T22:00:07.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:07 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:00:07.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:07 smithi060 ceph-mon[40290]: from='osd.1 [v2:172.21.15.60:6810/145283513,v1:172.21.15.60:6811/145283513]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-20T22:00:07.787 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:00:07 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1[50412]: 2024-09-20T22:00:07.471+0000 7f6bd51e7640 -1 osd.1 0 waiting for initial osdmap 2024-09-20T22:00:07.787 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:00:07 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1[50412]: 2024-09-20T22:00:07.475+0000 7f6bd100f640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T22:00:08.683 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:08 smithi060 ceph-mon[29694]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:08.683 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:08 smithi060 ceph-mon[29694]: osdmap e12: 2 total, 1 up, 2 in 2024-09-20T22:00:08.683 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:08 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:00:08.683 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:08 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:00:08.683 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:08 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:00:08.683 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:08 smithi060 ceph-mon[40290]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:08.683 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:08 smithi060 ceph-mon[40290]: osdmap e12: 2 total, 1 up, 2 in 2024-09-20T22:00:08.683 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:08 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:00:08.683 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:08 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:00:08.684 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:08 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:00:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:08 smithi067 ceph-mon[33511]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:08 smithi067 ceph-mon[33511]: osdmap e12: 2 total, 1 up, 2 in 2024-09-20T22:00:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:08 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:00:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:08 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:00:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:08 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:00:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:09 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:00:09.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:09 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:00:09.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:09 smithi067 ceph-mon[33511]: osd.1 [v2:172.21.15.60:6810/145283513,v1:172.21.15.60:6811/145283513] boot 2024-09-20T22:00:09.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:09 smithi067 ceph-mon[33511]: osdmap e13: 2 total, 2 up, 2 in 2024-09-20T22:00:09.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:09 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:00:09.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:09 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:09.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:09 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:09.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:09 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:00:09.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:09 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:00:09.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:09 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:09.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:09 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:09.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:09 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:00:09.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:09 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:09.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:00:09.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:00:09.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[29694]: osd.1 [v2:172.21.15.60:6810/145283513,v1:172.21.15.60:6811/145283513] boot 2024-09-20T22:00:09.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[29694]: osdmap e13: 2 total, 2 up, 2 in 2024-09-20T22:00:09.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:00:09.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:09.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:09.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:00:09.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:00:09.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:09.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:09.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:00:09.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:09.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:00:09.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:00:09.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[40290]: osd.1 [v2:172.21.15.60:6810/145283513,v1:172.21.15.60:6811/145283513] boot 2024-09-20T22:00:09.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[40290]: osdmap e13: 2 total, 2 up, 2 in 2024-09-20T22:00:09.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:00:09.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:09.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:09.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:00:09.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:00:09.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:09.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:09.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:00:09.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:09 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:10.043 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:00:10.072 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi060:vg_nvme/lv_2 2024-09-20T22:00:10.331 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:00:10.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:10 smithi060 ceph-mon[29694]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:10.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:10 smithi060 ceph-mon[29694]: Detected new or changed devices on smithi060 2024-09-20T22:00:10.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:10 smithi060 ceph-mon[29694]: Adjusting osd_memory_target on smithi060 to 8034M 2024-09-20T22:00:10.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:10 smithi060 ceph-mon[29694]: osdmap e14: 2 total, 2 up, 2 in 2024-09-20T22:00:10.702 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:10 smithi060 ceph-mon[40290]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:10.702 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:10 smithi060 ceph-mon[40290]: Detected new or changed devices on smithi060 2024-09-20T22:00:10.702 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:10 smithi060 ceph-mon[40290]: Adjusting osd_memory_target on smithi060 to 8034M 2024-09-20T22:00:10.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:10 smithi060 ceph-mon[40290]: osdmap e14: 2 total, 2 up, 2 in 2024-09-20T22:00:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:10 smithi067 ceph-mon[33511]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-20T22:00:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:10 smithi067 ceph-mon[33511]: Detected new or changed devices on smithi060 2024-09-20T22:00:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:10 smithi067 ceph-mon[33511]: Adjusting osd_memory_target on smithi060 to 8034M 2024-09-20T22:00:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:10 smithi067 ceph-mon[33511]: osdmap e14: 2 total, 2 up, 2 in 2024-09-20T22:00:12.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:12 smithi060 ceph-mon[29694]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:12.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:12 smithi060 ceph-mon[29694]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:00:12.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:12 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T22:00:12.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:12 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T22:00:12.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:12 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:12.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:12 smithi060 ceph-mon[40290]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:12.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:12 smithi060 ceph-mon[40290]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:00:12.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:12 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T22:00:12.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:12 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T22:00:12.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:12 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:12.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:12 smithi067 ceph-mon[33511]: pgmap v56: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:12.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:12 smithi067 ceph-mon[33511]: from='client.14265 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:00:12.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:12 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T22:00:12.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:12 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T22:00:12.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:12 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:14.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:14 smithi060 ceph-mon[40290]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:14.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:14 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1669526419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32c09d19-eb98-4926-a44e-e7ddee152956"}]: dispatch 2024-09-20T22:00:14.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:14 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1669526419' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32c09d19-eb98-4926-a44e-e7ddee152956"}]': finished 2024-09-20T22:00:14.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:14 smithi060 ceph-mon[40290]: osdmap e15: 3 total, 2 up, 3 in 2024-09-20T22:00:14.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:14 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:00:14.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:14 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/596265288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T22:00:14.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:14 smithi060 ceph-mon[29694]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:14.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:14 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1669526419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32c09d19-eb98-4926-a44e-e7ddee152956"}]: dispatch 2024-09-20T22:00:14.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:14 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1669526419' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32c09d19-eb98-4926-a44e-e7ddee152956"}]': finished 2024-09-20T22:00:14.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:14 smithi060 ceph-mon[29694]: osdmap e15: 3 total, 2 up, 3 in 2024-09-20T22:00:14.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:14 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:00:14.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:14 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/596265288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T22:00:14.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:14 smithi067 ceph-mon[33511]: pgmap v57: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:14.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:14 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1669526419' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "32c09d19-eb98-4926-a44e-e7ddee152956"}]: dispatch 2024-09-20T22:00:14.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:14 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1669526419' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "32c09d19-eb98-4926-a44e-e7ddee152956"}]': finished 2024-09-20T22:00:14.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:14 smithi067 ceph-mon[33511]: osdmap e15: 3 total, 2 up, 3 in 2024-09-20T22:00:14.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:14 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:00:14.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:14 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/596265288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T22:00:16.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:16 smithi060 ceph-mon[29694]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:16.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:16 smithi060 ceph-mon[40290]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:16.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:16 smithi067 ceph-mon[33511]: pgmap v59: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:18.224 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:18 smithi060 ceph-mon[29694]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:18.225 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:18 smithi060 ceph-mon[40290]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:18.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:18 smithi067 ceph-mon[33511]: pgmap v60: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:20.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:20 smithi060 ceph-mon[29694]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:20.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:20 smithi060 ceph-mon[40290]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:20.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:20 smithi067 ceph-mon[33511]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:22.386 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:22 smithi060 ceph-mon[29694]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:22.386 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:22 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-20T22:00:22.386 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:22 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:22.386 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:22 smithi060 ceph-mon[29694]: Deploying daemon osd.2 on smithi060 2024-09-20T22:00:22.386 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:22 smithi060 ceph-mon[40290]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:22.386 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:22 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-20T22:00:22.387 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:22 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:22.387 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:22 smithi060 ceph-mon[40290]: Deploying daemon osd.2 on smithi060 2024-09-20T22:00:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:22 smithi067 ceph-mon[33511]: pgmap v62: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:22 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-20T22:00:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:22 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:22 smithi067 ceph-mon[33511]: Deploying daemon osd.2 on smithi060 2024-09-20T22:00:24.127 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:24 smithi060 ceph-mon[29694]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:24.127 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:24 smithi060 ceph-mon[40290]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:24.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:24 smithi067 ceph-mon[33511]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:26.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:26 smithi060 ceph-mon[29694]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:26.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:26 smithi060 ceph-mon[40290]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:26.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:26 smithi067 ceph-mon[33511]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:27.387 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:27 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:00:27.433 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:27 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:27.433 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:27 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:27.433 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:27 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:00:27.433 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:27 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:27.433 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:27 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:27.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:27 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:00:27.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:27 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:27.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:27 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:28.346 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:28 smithi060 ceph-mon[29694]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:28.346 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:28 smithi060 ceph-mon[40290]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:28.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:28 smithi067 ceph-mon[33511]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:30.354 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:30 smithi060 ceph-mon[29694]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:30.354 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:30 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:30.354 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:30 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:30.354 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:30 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:30.354 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:30 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:00:30.354 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:30 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:30.355 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:30 smithi060 ceph-mon[40290]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:30.355 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:30 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:30.355 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:30 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:30.355 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:30 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:30.355 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:30 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:00:30.355 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:30 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:30 smithi067 ceph-mon[33511]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:30.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:30 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:30.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:30 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:30.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:30 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:30.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:30 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:00:30.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:30 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:30.662 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 2 on host 'smithi060' 2024-09-20T22:00:31.283 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:31 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:00:31.284 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:31 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:31.284 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:31 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:31.284 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:31 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:00:31.284 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:31 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:31.284 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:31 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:31.315 DEBUG:teuthology.orchestra.run.smithi060:osd.2> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.2.service 2024-09-20T22:00:31.317 INFO:tasks.cephadm:Deploying osd.3 on smithi060 with /dev/vg_nvme/lv_1... 2024-09-20T22:00:31.317 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-20T22:00:31.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:31 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:00:31.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:31 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:31.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:31 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:32.036 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:00:31 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-2[56776]: 2024-09-20T22:00:31.657+0000 7f7325764640 -1 osd.2 0 log_to_monitors true 2024-09-20T22:00:32.046 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:00:32.399 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:32 smithi060 ceph-mon[29694]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:32.399 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:32 smithi060 ceph-mon[29694]: from='osd.2 [v2:172.21.15.60:6818/3698562274,v1:172.21.15.60:6819/3698562274]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-20T22:00:32.399 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:32 smithi060 ceph-mon[40290]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:32.399 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:32 smithi060 ceph-mon[40290]: from='osd.2 [v2:172.21.15.60:6818/3698562274,v1:172.21.15.60:6819/3698562274]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-20T22:00:32.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:32 smithi067 ceph-mon[33511]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:32.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:32 smithi067 ceph-mon[33511]: from='osd.2 [v2:172.21.15.60:6818/3698562274,v1:172.21.15.60:6819/3698562274]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-20T22:00:33.455 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:33 smithi060 ceph-mon[29694]: from='osd.2 [v2:172.21.15.60:6818/3698562274,v1:172.21.15.60:6819/3698562274]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-20T22:00:33.455 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:33 smithi060 ceph-mon[29694]: osdmap e16: 3 total, 2 up, 3 in 2024-09-20T22:00:33.455 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:33 smithi060 ceph-mon[29694]: from='osd.2 [v2:172.21.15.60:6818/3698562274,v1:172.21.15.60:6819/3698562274]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-09-20T22:00:33.455 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:33 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:00:33.455 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:33 smithi060 ceph-mon[29694]: from='osd.2 [v2:172.21.15.60:6818/3698562274,v1:172.21.15.60:6819/3698562274]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-20T22:00:33.456 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:33 smithi060 ceph-mon[40290]: from='osd.2 [v2:172.21.15.60:6818/3698562274,v1:172.21.15.60:6819/3698562274]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-20T22:00:33.456 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:33 smithi060 ceph-mon[40290]: osdmap e16: 3 total, 2 up, 3 in 2024-09-20T22:00:33.456 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:33 smithi060 ceph-mon[40290]: from='osd.2 [v2:172.21.15.60:6818/3698562274,v1:172.21.15.60:6819/3698562274]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-09-20T22:00:33.456 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:33 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:00:33.456 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:33 smithi060 ceph-mon[40290]: from='osd.2 [v2:172.21.15.60:6818/3698562274,v1:172.21.15.60:6819/3698562274]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-20T22:00:33.456 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:00:33 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-2[56776]: 2024-09-20T22:00:33.064+0000 7f73216e7640 -1 osd.2 0 waiting for initial osdmap 2024-09-20T22:00:33.456 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:00:33 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-2[56776]: 2024-09-20T22:00:33.069+0000 7f731d50f640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T22:00:33.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:33 smithi067 ceph-mon[33511]: from='osd.2 [v2:172.21.15.60:6818/3698562274,v1:172.21.15.60:6819/3698562274]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-20T22:00:33.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:33 smithi067 ceph-mon[33511]: osdmap e16: 3 total, 2 up, 3 in 2024-09-20T22:00:33.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:33 smithi067 ceph-mon[33511]: from='osd.2 [v2:172.21.15.60:6818/3698562274,v1:172.21.15.60:6819/3698562274]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-09-20T22:00:33.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:33 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:00:33.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:33 smithi067 ceph-mon[33511]: from='osd.2 [v2:172.21.15.60:6818/3698562274,v1:172.21.15.60:6819/3698562274]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-20T22:00:34.438 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:34 smithi060 ceph-mon[29694]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:34.438 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:34 smithi060 ceph-mon[29694]: osdmap e17: 3 total, 2 up, 3 in 2024-09-20T22:00:34.438 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:34 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:00:34.438 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:34 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:00:34.438 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:34 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:00:34.439 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:34 smithi060 ceph-mon[40290]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:34.439 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:34 smithi060 ceph-mon[40290]: osdmap e17: 3 total, 2 up, 3 in 2024-09-20T22:00:34.439 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:34 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:00:34.439 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:34 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:00:34.439 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:34 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:00:34.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:34 smithi067 ceph-mon[33511]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:34.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:34 smithi067 ceph-mon[33511]: osdmap e17: 3 total, 2 up, 3 in 2024-09-20T22:00:34.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:34 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:00:34.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:34 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:00:34.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:34 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:00:35.375 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:35 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:00:35.375 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:35 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:00:35.375 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:35 smithi060 ceph-mon[40290]: osd.2 [v2:172.21.15.60:6818/3698562274,v1:172.21.15.60:6819/3698562274] boot 2024-09-20T22:00:35.375 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:35 smithi060 ceph-mon[40290]: osdmap e18: 3 total, 3 up, 3 in 2024-09-20T22:00:35.375 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:35 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:00:35.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:35 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:00:35.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:35 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:00:35.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:35 smithi060 ceph-mon[29694]: osd.2 [v2:172.21.15.60:6818/3698562274,v1:172.21.15.60:6819/3698562274] boot 2024-09-20T22:00:35.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:35 smithi060 ceph-mon[29694]: osdmap e18: 3 total, 3 up, 3 in 2024-09-20T22:00:35.376 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:35 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:00:35.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:35 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:00:35.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:35 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:00:35.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:35 smithi067 ceph-mon[33511]: osd.2 [v2:172.21.15.60:6818/3698562274,v1:172.21.15.60:6819/3698562274] boot 2024-09-20T22:00:35.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:35 smithi067 ceph-mon[33511]: osdmap e18: 3 total, 3 up, 3 in 2024-09-20T22:00:35.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:35 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:00:36.415 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:36 smithi067 ceph-mon[33511]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:36.415 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:36 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:00:36.415 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:36 smithi067 ceph-mon[33511]: Detected new or changed devices on smithi060 2024-09-20T22:00:36.415 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:36 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-20T22:00:36.415 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:36 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:36.416 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:36 smithi067 ceph-mon[33511]: osdmap e19: 3 total, 3 up, 3 in 2024-09-20T22:00:36.416 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:36 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:00:36.416 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:36 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:36.416 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:36 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:00:36.416 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:36 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:00:36.416 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:36 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:00:36.416 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:36 smithi067 ceph-mon[33511]: Adjusting osd_memory_target on smithi060 to 5356M 2024-09-20T22:00:36.416 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:36 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:36.416 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:36 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:36.416 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:36 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:00:36.416 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:36 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:36.419 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[29694]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:36.419 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:00:36.419 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[29694]: Detected new or changed devices on smithi060 2024-09-20T22:00:36.419 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-20T22:00:36.419 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:36.420 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[29694]: osdmap e19: 3 total, 3 up, 3 in 2024-09-20T22:00:36.420 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:00:36.420 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:36.420 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:00:36.420 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:00:36.420 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:00:36.420 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[29694]: Adjusting osd_memory_target on smithi060 to 5356M 2024-09-20T22:00:36.420 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:36.420 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:36.420 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:00:36.420 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:36.420 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[40290]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-20T22:00:36.420 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:00:36.420 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[40290]: Detected new or changed devices on smithi060 2024-09-20T22:00:36.421 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-20T22:00:36.421 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:36.421 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[40290]: osdmap e19: 3 total, 3 up, 3 in 2024-09-20T22:00:36.421 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:00:36.421 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:36.421 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:00:36.421 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:00:36.421 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:00:36.421 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[40290]: Adjusting osd_memory_target on smithi060 to 5356M 2024-09-20T22:00:36.421 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:36.421 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:36.421 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:00:36.421 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:36 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:36.422 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:00:36.449 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi060:vg_nvme/lv_1 2024-09-20T22:00:36.720 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:00:37.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:37 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-20T22:00:37.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:37 smithi067 ceph-mon[33511]: osdmap e20: 3 total, 3 up, 3 in 2024-09-20T22:00:37.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:37 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-20T22:00:37.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:37 smithi060 ceph-mon[40290]: osdmap e20: 3 total, 3 up, 3 in 2024-09-20T22:00:37.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:37 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-20T22:00:37.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:37 smithi060 ceph-mon[29694]: osdmap e20: 3 total, 3 up, 3 in 2024-09-20T22:00:37.536 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:00:37 smithi060 sudo[60962]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-20T22:00:37.536 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:00:37 smithi060 sudo[60962]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T22:00:37.537 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:00:37 smithi060 sudo[60962]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T22:00:37.537 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:00:37 smithi060 sudo[60962]: pam_unix(sudo:session): session closed for user root 2024-09-20T22:00:37.903 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:00:37 smithi060 sudo[61059]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-20T22:00:37.904 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:00:37 smithi060 sudo[61059]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T22:00:37.904 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:00:37 smithi060 sudo[61059]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T22:00:37.904 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:00:37 smithi060 sudo[61059]: pam_unix(sudo:session): session closed for user root 2024-09-20T22:00:38.286 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:00:38 smithi060 sudo[61125]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-20T22:00:38.286 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:00:38 smithi060 sudo[61125]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T22:00:38.286 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:00:38 smithi060 sudo[61125]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T22:00:38.286 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:00:38 smithi060 sudo[61125]: pam_unix(sudo:session): session closed for user root 2024-09-20T22:00:38.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:38 smithi060 ceph-mon[29694]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:00:38.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:38 smithi060 ceph-mon[29694]: osdmap e21: 3 total, 3 up, 3 in 2024-09-20T22:00:38.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:38 smithi060 ceph-mon[29694]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:00:38.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:38 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T22:00:38.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:38 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T22:00:38.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:38 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:38.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:38 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:38.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:38 smithi060 ceph-mon[40290]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:00:38.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:38 smithi060 ceph-mon[40290]: osdmap e21: 3 total, 3 up, 3 in 2024-09-20T22:00:38.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:38 smithi060 ceph-mon[40290]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:00:38.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:38 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T22:00:38.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:38 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T22:00:38.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:38 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:38.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:38 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:38.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:38 smithi067 ceph-mon[33511]: pgmap v75: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:00:38.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:38 smithi067 ceph-mon[33511]: osdmap e21: 3 total, 3 up, 3 in 2024-09-20T22:00:38.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:38 smithi067 ceph-mon[33511]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi060:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:00:38.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:38 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T22:00:38.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:38 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T22:00:38.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:38 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:38.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:38 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:38.721 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:00:38 smithi060 sudo[61137]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-20T22:00:38.721 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:00:38 smithi060 sudo[61137]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T22:00:38.721 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:00:38 smithi060 sudo[61137]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T22:00:38.721 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:00:38 smithi060 sudo[61137]: pam_unix(sudo:session): session closed for user root 2024-09-20T22:00:39.286 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:00:39 smithi060 sudo[61164]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-20T22:00:39.286 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:00:39 smithi060 sudo[61164]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T22:00:39.286 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:00:39 smithi060 sudo[61164]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T22:00:39.286 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:00:39 smithi060 sudo[61164]: pam_unix(sudo:session): session closed for user root 2024-09-20T22:00:39.786 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:00:39 smithi060 sudo[61206]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-20T22:00:39.786 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:00:39 smithi060 sudo[61206]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T22:00:39.786 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:00:39 smithi060 sudo[61206]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T22:00:39.786 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:00:39 smithi060 sudo[61206]: pam_unix(sudo:session): session closed for user root 2024-09-20T22:00:40.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:39 smithi060 sudo[61238]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-20T22:00:40.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:39 smithi060 sudo[61238]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T22:00:40.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:39 smithi060 sudo[61238]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T22:00:40.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:39 smithi060 sudo[61238]: pam_unix(sudo:session): session closed for user root 2024-09-20T22:00:40.636 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:40 smithi060 sudo[61241]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-20T22:00:40.636 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:40 smithi060 sudo[61241]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T22:00:40.636 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:40 smithi060 sudo[61241]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T22:00:40.637 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:40 smithi060 sudo[61241]: pam_unix(sudo:session): session closed for user root 2024-09-20T22:00:40.637 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:40 smithi060 ceph-mon[29694]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-20T22:00:40.637 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:40 smithi060 ceph-mon[29694]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-20T22:00:40.637 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:40 smithi060 ceph-mon[40290]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-20T22:00:40.637 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:40 smithi060 ceph-mon[40290]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-20T22:00:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:40 smithi067 ceph-mon[33511]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-20T22:00:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:40 smithi067 ceph-mon[33511]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-20T22:00:41.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:40 smithi067 sudo[36164]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-20T22:00:41.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:40 smithi067 sudo[36164]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T22:00:41.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:40 smithi067 sudo[36164]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T22:00:41.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:40 smithi067 sudo[36164]: pam_unix(sudo:session): session closed for user root 2024-09-20T22:00:41.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:41 smithi067 sudo[36167]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-20T22:00:41.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:41 smithi067 sudo[36167]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T22:00:41.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:41 smithi067 sudo[36167]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T22:00:41.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:41 smithi067 sudo[36167]: pam_unix(sudo:session): session closed for user root 2024-09-20T22:00:42.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:41 smithi060 sudo[61322]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-20T22:00:42.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:41 smithi060 sudo[61322]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T22:00:42.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:41 smithi060 sudo[61322]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T22:00:42.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:41 smithi060 sudo[61322]: pam_unix(sudo:session): session closed for user root 2024-09-20T22:00:42.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:00:42.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1217048573' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1c2cb69-8565-49ac-b854-7ea7615c6660"}]: dispatch 2024-09-20T22:00:42.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-20T22:00:42.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T22:00:42.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1c2cb69-8565-49ac-b854-7ea7615c6660"}]: dispatch 2024-09-20T22:00:42.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T22:00:42.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T22:00:42.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1c2cb69-8565-49ac-b854-7ea7615c6660"}]': finished 2024-09-20T22:00:42.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: osdmap e22: 4 total, 3 up, 4 in 2024-09-20T22:00:42.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:00:42.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-20T22:00:42.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T22:00:42.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T22:00:42.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T22:00:42.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3977185959' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T22:00:42.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-20T22:00:42.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:42.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T22:00:42.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T22:00:42.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:42 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T22:00:42.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:00:42.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1217048573' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1c2cb69-8565-49ac-b854-7ea7615c6660"}]: dispatch 2024-09-20T22:00:42.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-20T22:00:42.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T22:00:42.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1c2cb69-8565-49ac-b854-7ea7615c6660"}]: dispatch 2024-09-20T22:00:42.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T22:00:42.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T22:00:42.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1c2cb69-8565-49ac-b854-7ea7615c6660"}]': finished 2024-09-20T22:00:42.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: osdmap e22: 4 total, 3 up, 4 in 2024-09-20T22:00:42.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:00:42.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-20T22:00:42.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T22:00:42.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T22:00:42.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T22:00:42.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3977185959' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T22:00:42.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-20T22:00:42.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:42.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T22:00:42.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T22:00:42.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T22:00:42.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 sudo[61510]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-20T22:00:42.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 sudo[61510]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-20T22:00:42.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 sudo[61510]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-20T22:00:42.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 sudo[61510]: pam_unix(sudo:session): session closed for user root 2024-09-20T22:00:42.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: pgmap v77: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:00:42.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1217048573' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1c2cb69-8565-49ac-b854-7ea7615c6660"}]: dispatch 2024-09-20T22:00:42.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-20T22:00:42.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T22:00:42.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1c2cb69-8565-49ac-b854-7ea7615c6660"}]: dispatch 2024-09-20T22:00:42.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T22:00:42.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T22:00:42.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1c2cb69-8565-49ac-b854-7ea7615c6660"}]': finished 2024-09-20T22:00:42.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: osdmap e22: 4 total, 3 up, 4 in 2024-09-20T22:00:42.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:00:42.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-20T22:00:42.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T22:00:42.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T22:00:42.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T22:00:42.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3977185959' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T22:00:42.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-20T22:00:42.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:42.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T22:00:42.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T22:00:42.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:42 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T22:00:43.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:43 smithi067 ceph-mon[33511]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:00:43.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:43 smithi067 ceph-mon[33511]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-20T22:00:43.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:43 smithi067 ceph-mon[33511]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-20T22:00:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:43 smithi060 ceph-mon[29694]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:00:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:43 smithi060 ceph-mon[29694]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-20T22:00:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:43 smithi060 ceph-mon[29694]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-20T22:00:43.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:43 smithi060 ceph-mon[40290]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:00:43.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:43 smithi060 ceph-mon[40290]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-20T22:00:43.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:43 smithi060 ceph-mon[40290]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-20T22:00:44.506 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:44 smithi060 ceph-mon[29694]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:00:44.507 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:44 smithi060 ceph-mon[40290]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:00:44.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:44 smithi067 ceph-mon[33511]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:00:46.403 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:46 smithi060 ceph-mon[29694]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 4.2 KiB/s rd, 185 KiB/s wr, 13 op/s 2024-09-20T22:00:46.404 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:46 smithi060 ceph-mon[40290]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 4.2 KiB/s rd, 185 KiB/s wr, 13 op/s 2024-09-20T22:00:46.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:46 smithi067 ceph-mon[33511]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 4.2 KiB/s rd, 185 KiB/s wr, 13 op/s 2024-09-20T22:00:48.371 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:48 smithi060 ceph-mon[29694]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 126 KiB/s wr, 8 op/s 2024-09-20T22:00:48.371 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:48 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-20T22:00:48.371 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:48 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:48.372 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:48 smithi060 ceph-mon[40290]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 126 KiB/s wr, 8 op/s 2024-09-20T22:00:48.372 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:48 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-20T22:00:48.372 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:48 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:48.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:48 smithi067 ceph-mon[33511]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 126 KiB/s wr, 8 op/s 2024-09-20T22:00:48.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:48 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-20T22:00:48.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:48 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:49.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:49 smithi067 ceph-mon[33511]: Deploying daemon osd.3 on smithi060 2024-09-20T22:00:49.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:49 smithi060 ceph-mon[29694]: Deploying daemon osd.3 on smithi060 2024-09-20T22:00:49.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:49 smithi060 ceph-mon[40290]: Deploying daemon osd.3 on smithi060 2024-09-20T22:00:50.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:50 smithi067 ceph-mon[33511]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 95 KiB/s wr, 6 op/s 2024-09-20T22:00:50.518 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:50 smithi060 ceph-mon[29694]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 95 KiB/s wr, 6 op/s 2024-09-20T22:00:50.518 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:50 smithi060 ceph-mon[40290]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 95 KiB/s wr, 6 op/s 2024-09-20T22:00:52.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:52 smithi060 ceph-mon[29694]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 77 KiB/s wr, 5 op/s 2024-09-20T22:00:52.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:52 smithi060 ceph-mon[40290]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 77 KiB/s wr, 5 op/s 2024-09-20T22:00:52.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:52 smithi067 ceph-mon[33511]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 77 KiB/s wr, 5 op/s 2024-09-20T22:00:53.370 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:53 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:00:53.370 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:53 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:53.370 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:53 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:53.370 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:53 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:00:53.370 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:53 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:53.370 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:53 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:53.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:53 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:00:53.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:53 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:53.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:53 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:54.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:54 smithi067 ceph-mon[33511]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-20T22:00:54.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:54 smithi060 ceph-mon[29694]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-20T22:00:54.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:54 smithi060 ceph-mon[40290]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-20T22:00:56.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:56 smithi060 ceph-mon[29694]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-20T22:00:56.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:56 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:56.503 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:56 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:56.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:56 smithi060 ceph-mon[40290]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-20T22:00:56.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:56 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:56.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:56 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:56.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:56 smithi067 ceph-mon[33511]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-09-20T22:00:56.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:56 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:56.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:56 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:56.542 INFO:teuthology.orchestra.run.smithi060.stdout:Created osd(s) 3 on host 'smithi060' 2024-09-20T22:00:57.201 DEBUG:teuthology.orchestra.run.smithi060:osd.3> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.3.service 2024-09-20T22:00:57.204 INFO:tasks.cephadm:Deploying osd.4 on smithi067 with /dev/vg_nvme/lv_4... 2024-09-20T22:00:57.204 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-20T22:00:57.474 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:00:57.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:00:57.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:00:57.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:57.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:00:57.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:57.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:00:57.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:57.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:57.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[29694]: osdmap e23: 4 total, 3 up, 4 in 2024-09-20T22:00:57.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:00:57.480 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:00:57.480 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:00:57.480 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:57.480 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:00:57.480 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:57.480 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:00:57.480 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:57.480 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:57.480 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[40290]: osdmap e23: 4 total, 3 up, 4 in 2024-09-20T22:00:57.480 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:00:57.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:57 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:00:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:57 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:00:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:57 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:00:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:57 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:00:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:57 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:57 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:00:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:57 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:57 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:00:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:57 smithi067 ceph-mon[33511]: osdmap e23: 4 total, 3 up, 4 in 2024-09-20T22:00:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:57 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:00:58.252 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 20 22:00:57 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-3[63042]: 2024-09-20T22:00:57.925+0000 7fe72138c640 -1 osd.3 0 log_to_monitors true 2024-09-20T22:00:58.508 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:58 smithi060 ceph-mon[29694]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:00:58.508 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:58 smithi060 ceph-mon[29694]: from='osd.3 [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-20T22:00:58.508 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:58 smithi060 ceph-mon[40290]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:00:58.509 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:58 smithi060 ceph-mon[40290]: from='osd.3 [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-20T22:00:58.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:58 smithi067 ceph-mon[33511]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:00:58.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:58 smithi067 ceph-mon[33511]: from='osd.3 [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-20T22:00:59.687 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:59 smithi060 ceph-mon[29694]: from='osd.3 [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-20T22:00:59.688 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:59 smithi060 ceph-mon[29694]: osdmap e24: 4 total, 3 up, 4 in 2024-09-20T22:00:59.688 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:59 smithi060 ceph-mon[29694]: from='osd.3 [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-09-20T22:00:59.688 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:59 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:00:59.688 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:00:59 smithi060 ceph-mon[29694]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:00:59.688 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:59 smithi060 ceph-mon[40290]: from='osd.3 [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-20T22:00:59.688 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:59 smithi060 ceph-mon[40290]: osdmap e24: 4 total, 3 up, 4 in 2024-09-20T22:00:59.688 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:59 smithi060 ceph-mon[40290]: from='osd.3 [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-09-20T22:00:59.688 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:59 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:00:59.689 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:00:59 smithi060 ceph-mon[40290]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:00:59.889 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:59 smithi067 ceph-mon[33511]: from='osd.3 [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-20T22:00:59.889 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:59 smithi067 ceph-mon[33511]: osdmap e24: 4 total, 3 up, 4 in 2024-09-20T22:00:59.889 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:59 smithi067 ceph-mon[33511]: from='osd.3 [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-09-20T22:00:59.889 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:59 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:00:59.889 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:00:59 smithi067 ceph-mon[33511]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:00:59.892 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T22:00:59.917 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi067:vg_nvme/lv_4 2024-09-20T22:01:00.021 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 20 22:00:59 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-3[63042]: 2024-09-20T22:00:59.799+0000 7fe71d30f640 -1 osd.3 0 waiting for initial osdmap 2024-09-20T22:01:00.021 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 20 22:00:59 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-3[63042]: 2024-09-20T22:00:59.803+0000 7fe71512f640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T22:01:00.171 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:01:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:00 smithi067 ceph-mon[33511]: from='osd.3 [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-20T22:01:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:00 smithi067 ceph-mon[33511]: osdmap e25: 4 total, 3 up, 4 in 2024-09-20T22:01:01.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:01:01.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:01:01.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:00 smithi067 ceph-mon[33511]: from='osd.3 [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449]' entity='osd.3' 2024-09-20T22:01:01.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:00 smithi067 ceph-mon[33511]: Detected new or changed devices on smithi060 2024-09-20T22:01:01.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:01.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:01.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:01.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:01.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:01.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:01.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:00 smithi067 ceph-mon[33511]: Adjusting osd_memory_target on smithi060 to 4017M 2024-09-20T22:01:01.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:01.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:01.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:01:01.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:00 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:01.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[40290]: from='osd.3 [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-20T22:01:01.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[40290]: osdmap e25: 4 total, 3 up, 4 in 2024-09-20T22:01:01.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:01:01.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:01:01.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[40290]: from='osd.3 [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449]' entity='osd.3' 2024-09-20T22:01:01.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[40290]: Detected new or changed devices on smithi060 2024-09-20T22:01:01.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:01.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:01.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:01.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:01.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:01.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:01.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[40290]: Adjusting osd_memory_target on smithi060 to 4017M 2024-09-20T22:01:01.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:01.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:01.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:01:01.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:01.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[29694]: from='osd.3 [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]': finished 2024-09-20T22:01:01.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[29694]: osdmap e25: 4 total, 3 up, 4 in 2024-09-20T22:01:01.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:01:01.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:01:01.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[29694]: from='osd.3 [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449]' entity='osd.3' 2024-09-20T22:01:01.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[29694]: Detected new or changed devices on smithi060 2024-09-20T22:01:01.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:01.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:01.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:01.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:01.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:01.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:01.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[29694]: Adjusting osd_memory_target on smithi060 to 4017M 2024-09-20T22:01:01.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:01.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:01.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:01:01.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:00 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:01.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:01:01.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:01:01.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:01:01.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[29694]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:01:01.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[29694]: osd.3 [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449] boot 2024-09-20T22:01:01.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[29694]: osdmap e26: 4 total, 4 up, 4 in 2024-09-20T22:01:01.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:01:01.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[29694]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:01:01.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T22:01:01.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T22:01:01.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:01.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:01:01.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:01:01.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:01:01.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[40290]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:01:01.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[40290]: osd.3 [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449] boot 2024-09-20T22:01:01.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[40290]: osdmap e26: 4 total, 4 up, 4 in 2024-09-20T22:01:01.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:01:01.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[40290]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:01:01.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T22:01:01.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T22:01:01.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:01 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:02.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:01 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:01:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:01 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:01:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:01 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:01:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:01 smithi067 ceph-mon[33511]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-20T22:01:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:01 smithi067 ceph-mon[33511]: osd.3 [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449] boot 2024-09-20T22:01:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:01 smithi067 ceph-mon[33511]: osdmap e26: 4 total, 4 up, 4 in 2024-09-20T22:01:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:01 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:01:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:01 smithi067 ceph-mon[33511]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:01:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:01 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T22:01:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:01 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T22:01:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:01 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:03.175 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:02 smithi067 ceph-mon[33511]: osdmap e27: 4 total, 4 up, 4 in 2024-09-20T22:01:03.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:02 smithi060 ceph-mon[29694]: osdmap e27: 4 total, 4 up, 4 in 2024-09-20T22:01:03.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:02 smithi060 ceph-mon[40290]: osdmap e27: 4 total, 4 up, 4 in 2024-09-20T22:01:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:03 smithi067 ceph-mon[33511]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:03 smithi067 ceph-mon[33511]: from='client.? 172.21.15.67:0/2541129560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04a547d6-fc48-43b7-a8d7-18df8a83aa96"}]: dispatch 2024-09-20T22:01:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:03 smithi067 ceph-mon[33511]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04a547d6-fc48-43b7-a8d7-18df8a83aa96"}]: dispatch 2024-09-20T22:01:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:03 smithi067 ceph-mon[33511]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04a547d6-fc48-43b7-a8d7-18df8a83aa96"}]': finished 2024-09-20T22:01:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:03 smithi067 ceph-mon[33511]: osdmap e28: 5 total, 4 up, 5 in 2024-09-20T22:01:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:03 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:01:04.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:03 smithi060 ceph-mon[29694]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:04.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:03 smithi060 ceph-mon[29694]: from='client.? 172.21.15.67:0/2541129560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04a547d6-fc48-43b7-a8d7-18df8a83aa96"}]: dispatch 2024-09-20T22:01:04.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:03 smithi060 ceph-mon[29694]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04a547d6-fc48-43b7-a8d7-18df8a83aa96"}]: dispatch 2024-09-20T22:01:04.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:03 smithi060 ceph-mon[29694]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04a547d6-fc48-43b7-a8d7-18df8a83aa96"}]': finished 2024-09-20T22:01:04.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:03 smithi060 ceph-mon[29694]: osdmap e28: 5 total, 4 up, 5 in 2024-09-20T22:01:04.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:03 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:01:04.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:03 smithi060 ceph-mon[40290]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:04.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:03 smithi060 ceph-mon[40290]: from='client.? 172.21.15.67:0/2541129560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04a547d6-fc48-43b7-a8d7-18df8a83aa96"}]: dispatch 2024-09-20T22:01:04.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:03 smithi060 ceph-mon[40290]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04a547d6-fc48-43b7-a8d7-18df8a83aa96"}]: dispatch 2024-09-20T22:01:04.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:03 smithi060 ceph-mon[40290]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04a547d6-fc48-43b7-a8d7-18df8a83aa96"}]': finished 2024-09-20T22:01:04.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:03 smithi060 ceph-mon[40290]: osdmap e28: 5 total, 4 up, 5 in 2024-09-20T22:01:04.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:03 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:01:05.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:04 smithi067 ceph-mon[33511]: from='client.? 172.21.15.67:0/2817633946' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T22:01:05.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:04 smithi060 ceph-mon[29694]: from='client.? 172.21.15.67:0/2817633946' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T22:01:05.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:04 smithi060 ceph-mon[40290]: from='client.? 172.21.15.67:0/2817633946' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T22:01:06.165 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:05 smithi067 ceph-mon[33511]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:06.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:05 smithi060 ceph-mon[29694]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:06.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:05 smithi060 ceph-mon[40290]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:08.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:08 smithi067 ceph-mon[33511]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:08.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:08 smithi060 ceph-mon[29694]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:08.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:08 smithi060 ceph-mon[40290]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:10.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:10 smithi067 ceph-mon[33511]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:10.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:10 smithi060 ceph-mon[29694]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:10.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:10 smithi060 ceph-mon[40290]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:11.329 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:11 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-20T22:01:11.329 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:11 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:11.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:11 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-20T22:01:11.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:11 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:11.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:11 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-20T22:01:11.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:11 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:12.383 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:12 smithi067 ceph-mon[33511]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:12.383 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:12 smithi067 ceph-mon[33511]: Deploying daemon osd.4 on smithi067 2024-09-20T22:01:12.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:12 smithi060 ceph-mon[29694]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:12.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:12 smithi060 ceph-mon[29694]: Deploying daemon osd.4 on smithi067 2024-09-20T22:01:12.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:12 smithi060 ceph-mon[40290]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:12.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:12 smithi060 ceph-mon[40290]: Deploying daemon osd.4 on smithi067 2024-09-20T22:01:14.290 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:14 smithi067 ceph-mon[33511]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:14.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:14 smithi060 ceph-mon[29694]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:14.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:14 smithi060 ceph-mon[40290]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:16.206 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:16 smithi067 ceph-mon[33511]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:16 smithi060 ceph-mon[29694]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:16.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:16 smithi060 ceph-mon[40290]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:18.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:18 smithi067 ceph-mon[33511]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:18.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:18 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:01:18.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:18 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:18.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:18 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:18.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:18 smithi060 ceph-mon[29694]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:18.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:18 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:01:18.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:18 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:18.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:18 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:18.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:18 smithi060 ceph-mon[40290]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:18.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:18 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:01:18.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:18 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:18.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:18 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:20.446 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:20 smithi067 ceph-mon[33511]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:20.446 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:20 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:20.446 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:20 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:20.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:20 smithi060 ceph-mon[29694]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:20.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:20 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:20.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:20 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:20.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:20 smithi060 ceph-mon[40290]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:20.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:20 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:20.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:20 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:21.018 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 4 on host 'smithi067' 2024-09-20T22:01:21.401 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:21 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:21.401 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:21 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:01:21.402 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:21 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:21.402 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:21 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:01:21.402 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:21 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:21.402 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:21 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:21.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:21 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:21.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:21 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:01:21.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:21 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:21.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:21 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:01:21.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:21 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:21.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:21 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:21.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:21 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:21.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:21 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:01:21.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:21 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:21.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:21 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:01:21.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:21 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:21.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:21 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:21.721 DEBUG:teuthology.orchestra.run.smithi067:osd.4> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.4.service 2024-09-20T22:01:21.723 INFO:tasks.cephadm:Deploying osd.5 on smithi067 with /dev/vg_nvme/lv_3... 2024-09-20T22:01:21.723 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-20T22:01:22.012 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:01:21 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[38685]: 2024-09-20T22:01:21.852+0000 7f1aff442640 -1 osd.4 0 log_to_monitors true 2024-09-20T22:01:22.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:22 smithi067 ceph-mon[33511]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:22.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:22 smithi067 ceph-mon[33511]: from='osd.4 [v2:172.21.15.67:6800/3243564239,v1:172.21.15.67:6801/3243564239]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-20T22:01:22.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:22 smithi067 ceph-mon[33511]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-20T22:01:22.438 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:22 smithi060 ceph-mon[40290]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:22.438 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:22 smithi060 ceph-mon[40290]: from='osd.4 [v2:172.21.15.67:6800/3243564239,v1:172.21.15.67:6801/3243564239]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-20T22:01:22.438 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:22 smithi060 ceph-mon[40290]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-20T22:01:22.438 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:22 smithi060 ceph-mon[29694]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:22.438 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:22 smithi060 ceph-mon[29694]: from='osd.4 [v2:172.21.15.67:6800/3243564239,v1:172.21.15.67:6801/3243564239]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-20T22:01:22.438 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:22 smithi060 ceph-mon[29694]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-20T22:01:22.764 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:01:23.412 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:23 smithi067 ceph-mon[33511]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-20T22:01:23.413 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:23 smithi067 ceph-mon[33511]: from='osd.4 [v2:172.21.15.67:6800/3243564239,v1:172.21.15.67:6801/3243564239]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:01:23.413 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:23 smithi067 ceph-mon[33511]: osdmap e29: 5 total, 4 up, 5 in 2024-09-20T22:01:23.413 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:23 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:01:23.413 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:23 smithi067 ceph-mon[33511]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:01:23.453 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:23 smithi060 ceph-mon[29694]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-20T22:01:23.453 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:23 smithi060 ceph-mon[29694]: from='osd.4 [v2:172.21.15.67:6800/3243564239,v1:172.21.15.67:6801/3243564239]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:01:23.453 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:23 smithi060 ceph-mon[29694]: osdmap e29: 5 total, 4 up, 5 in 2024-09-20T22:01:23.453 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:23 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:01:23.454 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:23 smithi060 ceph-mon[29694]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:01:23.454 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:23 smithi060 ceph-mon[40290]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-20T22:01:23.454 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:23 smithi060 ceph-mon[40290]: from='osd.4 [v2:172.21.15.67:6800/3243564239,v1:172.21.15.67:6801/3243564239]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:01:23.454 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:23 smithi060 ceph-mon[40290]: osdmap e29: 5 total, 4 up, 5 in 2024-09-20T22:01:23.454 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:23 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:01:23.454 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:23 smithi060 ceph-mon[40290]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:01:23.763 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:01:23 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[38685]: 2024-09-20T22:01:23.596+0000 7f1afb3c5640 -1 osd.4 0 waiting for initial osdmap 2024-09-20T22:01:23.763 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:01:23 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[38685]: 2024-09-20T22:01:23.599+0000 7f1af31e5640 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T22:01:24.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:24 smithi067 ceph-mon[33511]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:24.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:24 smithi067 ceph-mon[33511]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-20T22:01:24.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:24 smithi067 ceph-mon[33511]: osdmap e30: 5 total, 4 up, 5 in 2024-09-20T22:01:24.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:24 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:01:24.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:24 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:01:24.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:24 smithi067 ceph-mon[33511]: from='osd.4 ' entity='osd.4' 2024-09-20T22:01:24.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:24 smithi060 ceph-mon[29694]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:24.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:24 smithi060 ceph-mon[29694]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-20T22:01:24.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:24 smithi060 ceph-mon[29694]: osdmap e30: 5 total, 4 up, 5 in 2024-09-20T22:01:24.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:24 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:01:24.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:24 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:01:24.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:24 smithi060 ceph-mon[29694]: from='osd.4 ' entity='osd.4' 2024-09-20T22:01:24.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:24 smithi060 ceph-mon[40290]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:24.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:24 smithi060 ceph-mon[40290]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-20T22:01:24.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:24 smithi060 ceph-mon[40290]: osdmap e30: 5 total, 4 up, 5 in 2024-09-20T22:01:24.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:24 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:01:24.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:24 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:01:24.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:24 smithi060 ceph-mon[40290]: from='osd.4 ' entity='osd.4' 2024-09-20T22:01:25.423 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:25 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:01:25.423 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:25 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:01:25.423 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:25 smithi067 ceph-mon[33511]: osd.4 [v2:172.21.15.67:6800/3243564239,v1:172.21.15.67:6801/3243564239] boot 2024-09-20T22:01:25.424 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:25 smithi067 ceph-mon[33511]: osdmap e31: 5 total, 5 up, 5 in 2024-09-20T22:01:25.424 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:25 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:01:25.424 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:25 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:25.424 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:25 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:25.424 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:25 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:25.424 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:25 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:25.424 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:25 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:25.424 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:25 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:01:25.424 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:25 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:25.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:01:25.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:01:25.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[29694]: osd.4 [v2:172.21.15.67:6800/3243564239,v1:172.21.15.67:6801/3243564239] boot 2024-09-20T22:01:25.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[29694]: osdmap e31: 5 total, 5 up, 5 in 2024-09-20T22:01:25.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:01:25.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:25.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:25.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:25.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:25.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:25.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:01:25.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:25.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:01:25.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:01:25.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[40290]: osd.4 [v2:172.21.15.67:6800/3243564239,v1:172.21.15.67:6801/3243564239] boot 2024-09-20T22:01:25.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[40290]: osdmap e31: 5 total, 5 up, 5 in 2024-09-20T22:01:25.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:01:25.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:25.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:25.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:25.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:25.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:25.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:01:25.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:25 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:26.361 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:26 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:01:26.361 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:26 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:01:26.361 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:26 smithi067 ceph-mon[33511]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:26.361 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:26 smithi067 ceph-mon[33511]: Detected new or changed devices on smithi067 2024-09-20T22:01:26.361 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:26 smithi067 ceph-mon[33511]: Adjusting osd_memory_target on smithi067 to 17088M 2024-09-20T22:01:26.361 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:26 smithi067 ceph-mon[33511]: osdmap e32: 5 total, 5 up, 5 in 2024-09-20T22:01:26.363 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T22:01:26.390 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi067:vg_nvme/lv_3 2024-09-20T22:01:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:26 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:01:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:26 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:01:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:26 smithi060 ceph-mon[29694]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:26 smithi060 ceph-mon[29694]: Detected new or changed devices on smithi067 2024-09-20T22:01:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:26 smithi060 ceph-mon[29694]: Adjusting osd_memory_target on smithi067 to 17088M 2024-09-20T22:01:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:26 smithi060 ceph-mon[29694]: osdmap e32: 5 total, 5 up, 5 in 2024-09-20T22:01:26.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:26 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:01:26.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:26 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:01:26.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:26 smithi060 ceph-mon[40290]: pgmap v110: 1 pgs: 1 remapped+peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-20T22:01:26.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:26 smithi060 ceph-mon[40290]: Detected new or changed devices on smithi067 2024-09-20T22:01:26.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:26 smithi060 ceph-mon[40290]: Adjusting osd_memory_target on smithi067 to 17088M 2024-09-20T22:01:26.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:26 smithi060 ceph-mon[40290]: osdmap e32: 5 total, 5 up, 5 in 2024-09-20T22:01:26.643 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:01:27.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:27 smithi067 ceph-mon[33511]: osdmap e33: 5 total, 5 up, 5 in 2024-09-20T22:01:27.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:27 smithi060 ceph-mon[29694]: osdmap e33: 5 total, 5 up, 5 in 2024-09-20T22:01:27.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:27 smithi060 ceph-mon[40290]: osdmap e33: 5 total, 5 up, 5 in 2024-09-20T22:01:28.360 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:28 smithi067 ceph-mon[33511]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:28.360 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:28 smithi067 ceph-mon[33511]: osdmap e34: 5 total, 5 up, 5 in 2024-09-20T22:01:28.360 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:28 smithi067 ceph-mon[33511]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:01:28.360 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:28 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T22:01:28.360 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:28 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T22:01:28.361 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:28 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:28.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:28 smithi060 ceph-mon[29694]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:28.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:28 smithi060 ceph-mon[29694]: osdmap e34: 5 total, 5 up, 5 in 2024-09-20T22:01:28.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:28 smithi060 ceph-mon[29694]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:01:28.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:28 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T22:01:28.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:28 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T22:01:28.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:28 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:28.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:28 smithi060 ceph-mon[40290]: pgmap v113: 1 pgs: 1 remapped+peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:28.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:28 smithi060 ceph-mon[40290]: osdmap e34: 5 total, 5 up, 5 in 2024-09-20T22:01:28.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:28 smithi060 ceph-mon[40290]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:01:28.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:28 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T22:01:28.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:28 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T22:01:28.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:28 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:29.303 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:29 smithi067 ceph-mon[33511]: osdmap e35: 5 total, 5 up, 5 in 2024-09-20T22:01:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:29 smithi060 ceph-mon[29694]: osdmap e35: 5 total, 5 up, 5 in 2024-09-20T22:01:29.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:29 smithi060 ceph-mon[40290]: osdmap e35: 5 total, 5 up, 5 in 2024-09-20T22:01:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:30 smithi067 ceph-mon[33511]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:30 smithi067 ceph-mon[33511]: from='client.? 172.21.15.67:0/2060504863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4763aa10-c367-485b-b525-e5241e642b86"}]: dispatch 2024-09-20T22:01:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:30 smithi067 ceph-mon[33511]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4763aa10-c367-485b-b525-e5241e642b86"}]: dispatch 2024-09-20T22:01:30.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:30 smithi067 ceph-mon[33511]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4763aa10-c367-485b-b525-e5241e642b86"}]': finished 2024-09-20T22:01:30.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:30 smithi067 ceph-mon[33511]: osdmap e36: 6 total, 5 up, 6 in 2024-09-20T22:01:30.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:30 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-20T22:01:30.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:30 smithi060 ceph-mon[29694]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:30.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:30 smithi060 ceph-mon[29694]: from='client.? 172.21.15.67:0/2060504863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4763aa10-c367-485b-b525-e5241e642b86"}]: dispatch 2024-09-20T22:01:30.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:30 smithi060 ceph-mon[29694]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4763aa10-c367-485b-b525-e5241e642b86"}]: dispatch 2024-09-20T22:01:30.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:30 smithi060 ceph-mon[29694]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4763aa10-c367-485b-b525-e5241e642b86"}]': finished 2024-09-20T22:01:30.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:30 smithi060 ceph-mon[29694]: osdmap e36: 6 total, 5 up, 6 in 2024-09-20T22:01:30.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:30 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-20T22:01:30.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:30 smithi060 ceph-mon[40290]: pgmap v116: 1 pgs: 1 remapped+peering; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:30.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:30 smithi060 ceph-mon[40290]: from='client.? 172.21.15.67:0/2060504863' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4763aa10-c367-485b-b525-e5241e642b86"}]: dispatch 2024-09-20T22:01:30.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:30 smithi060 ceph-mon[40290]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4763aa10-c367-485b-b525-e5241e642b86"}]: dispatch 2024-09-20T22:01:30.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:30 smithi060 ceph-mon[40290]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4763aa10-c367-485b-b525-e5241e642b86"}]': finished 2024-09-20T22:01:30.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:30 smithi060 ceph-mon[40290]: osdmap e36: 6 total, 5 up, 6 in 2024-09-20T22:01:30.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:30 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-20T22:01:31.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:31 smithi067 ceph-mon[33511]: from='client.? 172.21.15.67:0/1138586588' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T22:01:31.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:31 smithi060 ceph-mon[29694]: from='client.? 172.21.15.67:0/1138586588' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T22:01:31.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:31 smithi060 ceph-mon[40290]: from='client.? 172.21.15.67:0/1138586588' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T22:01:32.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:32 smithi067 ceph-mon[33511]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-20T22:01:32.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:32 smithi060 ceph-mon[29694]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-20T22:01:32.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:32 smithi060 ceph-mon[40290]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 128 KiB/s, 0 objects/s recovering 2024-09-20T22:01:34.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:34 smithi067 ceph-mon[33511]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-20T22:01:34.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:34 smithi060 ceph-mon[29694]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-20T22:01:34.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:34 smithi060 ceph-mon[40290]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-20T22:01:36.375 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:36 smithi067 ceph-mon[33511]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-20T22:01:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:36 smithi060 ceph-mon[29694]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-20T22:01:36.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:36 smithi060 ceph-mon[40290]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-09-20T22:01:37.407 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:37 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-20T22:01:37.407 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:37 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:37.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:37 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-20T22:01:37.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:37 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:37.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:37 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-20T22:01:37.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:37 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:38.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:38 smithi067 ceph-mon[33511]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-20T22:01:38.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:38 smithi067 ceph-mon[33511]: Deploying daemon osd.5 on smithi067 2024-09-20T22:01:38.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:38 smithi060 ceph-mon[29694]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-20T22:01:38.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:38 smithi060 ceph-mon[29694]: Deploying daemon osd.5 on smithi067 2024-09-20T22:01:38.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:38 smithi060 ceph-mon[40290]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-09-20T22:01:38.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:38 smithi060 ceph-mon[40290]: Deploying daemon osd.5 on smithi067 2024-09-20T22:01:40.438 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:40 smithi067 ceph-mon[33511]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-20T22:01:40.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:40 smithi060 ceph-mon[29694]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-20T22:01:40.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:40 smithi060 ceph-mon[40290]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-20T22:01:42.407 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:42 smithi067 ceph-mon[33511]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:42.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:42 smithi060 ceph-mon[29694]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:42.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:42 smithi060 ceph-mon[40290]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:43.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:43 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:01:43.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:43 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:43.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:43 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:43.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:43 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:01:43.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:43 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:43.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:43 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:43 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:01:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:43 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:43 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:44.374 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:44 smithi067 ceph-mon[33511]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:44.374 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:44 smithi067 ceph-mon[33511]: 1.0 scrub starts 2024-09-20T22:01:44.374 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:44 smithi067 ceph-mon[33511]: 1.0 scrub ok 2024-09-20T22:01:44.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:44 smithi060 ceph-mon[29694]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:44.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:44 smithi060 ceph-mon[29694]: 1.0 scrub starts 2024-09-20T22:01:44.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:44 smithi060 ceph-mon[29694]: 1.0 scrub ok 2024-09-20T22:01:44.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:44 smithi060 ceph-mon[40290]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:44.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:44 smithi060 ceph-mon[40290]: 1.0 scrub starts 2024-09-20T22:01:44.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:44 smithi060 ceph-mon[40290]: 1.0 scrub ok 2024-09-20T22:01:46.451 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:46 smithi067 ceph-mon[33511]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:46.451 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:46 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:46.451 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:46 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:46.451 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:46 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:46.451 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:46 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:01:46.451 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:46 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:46.491 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 5 on host 'smithi067' 2024-09-20T22:01:46.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:46 smithi060 ceph-mon[29694]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:46.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:46 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:46.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:46 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:46.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:46 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:46.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:46 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:01:46.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:46 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:46.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:46 smithi060 ceph-mon[40290]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:46.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:46 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:46.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:46 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:46.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:46 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:46.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:46 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:01:46.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:46 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:47.202 DEBUG:teuthology.orchestra.run.smithi067:osd.5> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.5.service 2024-09-20T22:01:47.204 INFO:tasks.cephadm:Deploying osd.6 on smithi067 with /dev/vg_nvme/lv_2... 2024-09-20T22:01:47.204 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-20T22:01:47.463 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:47 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:01:47.463 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:47 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:47.463 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:47 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:47.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:47 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:01:47.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:47 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:47.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:47 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:47.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:47 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:01:47.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:47 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:47.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:47 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:47.763 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:01:47 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:01:47.496+0000 7f8f6fd52640 -1 osd.5 0 log_to_monitors true 2024-09-20T22:01:48.705 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:48 smithi067 ceph-mon[33511]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:48.705 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:48 smithi067 ceph-mon[33511]: from='osd.5 [v2:172.21.15.67:6808/1622258887,v1:172.21.15.67:6809/1622258887]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-20T22:01:48.705 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:48 smithi067 ceph-mon[33511]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-20T22:01:48.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:48 smithi060 ceph-mon[29694]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:48.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:48 smithi060 ceph-mon[29694]: from='osd.5 [v2:172.21.15.67:6808/1622258887,v1:172.21.15.67:6809/1622258887]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-20T22:01:48.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:48 smithi060 ceph-mon[29694]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-20T22:01:48.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:48 smithi060 ceph-mon[40290]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:48.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:48 smithi060 ceph-mon[40290]: from='osd.5 [v2:172.21.15.67:6808/1622258887,v1:172.21.15.67:6809/1622258887]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-20T22:01:48.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:48 smithi060 ceph-mon[40290]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-20T22:01:48.910 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:01:49.617 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:49 smithi067 ceph-mon[33511]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-20T22:01:49.617 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:49 smithi067 ceph-mon[33511]: from='osd.5 [v2:172.21.15.67:6808/1622258887,v1:172.21.15.67:6809/1622258887]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:01:49.617 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:49 smithi067 ceph-mon[33511]: osdmap e37: 6 total, 5 up, 6 in 2024-09-20T22:01:49.617 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:49 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-20T22:01:49.617 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:49 smithi067 ceph-mon[33511]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:01:49.617 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:01:49 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:01:49.503+0000 7f8f6bcd5640 -1 osd.5 0 waiting for initial osdmap 2024-09-20T22:01:49.617 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:01:49 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:01:49.507+0000 7f8f67afd640 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T22:01:49.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:49 smithi060 ceph-mon[40290]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-20T22:01:49.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:49 smithi060 ceph-mon[40290]: from='osd.5 [v2:172.21.15.67:6808/1622258887,v1:172.21.15.67:6809/1622258887]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:01:49.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:49 smithi060 ceph-mon[40290]: osdmap e37: 6 total, 5 up, 6 in 2024-09-20T22:01:49.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:49 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-20T22:01:49.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:49 smithi060 ceph-mon[40290]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:01:49.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:49 smithi060 ceph-mon[29694]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-20T22:01:49.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:49 smithi060 ceph-mon[29694]: from='osd.5 [v2:172.21.15.67:6808/1622258887,v1:172.21.15.67:6809/1622258887]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:01:49.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:49 smithi060 ceph-mon[29694]: osdmap e37: 6 total, 5 up, 6 in 2024-09-20T22:01:49.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:49 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-20T22:01:49.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:49 smithi060 ceph-mon[29694]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:01:50.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:50 smithi067 ceph-mon[33511]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:50.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:50 smithi067 ceph-mon[33511]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-20T22:01:50.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:50 smithi067 ceph-mon[33511]: osdmap e38: 6 total, 5 up, 6 in 2024-09-20T22:01:50.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:50 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-20T22:01:50.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:50 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-20T22:01:50.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:50 smithi060 ceph-mon[29694]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:50.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:50 smithi060 ceph-mon[29694]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-20T22:01:50.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:50 smithi060 ceph-mon[29694]: osdmap e38: 6 total, 5 up, 6 in 2024-09-20T22:01:50.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:50 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-20T22:01:50.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:50 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-20T22:01:50.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:50 smithi060 ceph-mon[40290]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 143 MiB used, 447 GiB / 447 GiB avail 2024-09-20T22:01:50.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:50 smithi060 ceph-mon[40290]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-20T22:01:50.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:50 smithi060 ceph-mon[40290]: osdmap e38: 6 total, 5 up, 6 in 2024-09-20T22:01:50.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:50 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-20T22:01:50.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:50 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-20T22:01:51.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:51 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:01:51.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:51 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:01:51.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:51 smithi067 ceph-mon[33511]: osd.5 [v2:172.21.15.67:6808/1622258887,v1:172.21.15.67:6809/1622258887] boot 2024-09-20T22:01:51.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:51 smithi067 ceph-mon[33511]: osdmap e39: 6 total, 6 up, 6 in 2024-09-20T22:01:51.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:51 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-20T22:01:51.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:51 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:51.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:51 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:51.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:51 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:51.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:51 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:51.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:51 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:51.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:51 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:51.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:51 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:01:51.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:51 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:51.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:51 smithi067 ceph-mon[33511]: osdmap e40: 6 total, 6 up, 6 in 2024-09-20T22:01:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:01:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:01:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[29694]: osd.5 [v2:172.21.15.67:6808/1622258887,v1:172.21.15.67:6809/1622258887] boot 2024-09-20T22:01:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[29694]: osdmap e39: 6 total, 6 up, 6 in 2024-09-20T22:01:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-20T22:01:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:51.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:51.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:51.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:51.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:51.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:01:51.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:51.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[29694]: osdmap e40: 6 total, 6 up, 6 in 2024-09-20T22:01:51.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:01:51.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:01:51.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[40290]: osd.5 [v2:172.21.15.67:6808/1622258887,v1:172.21.15.67:6809/1622258887] boot 2024-09-20T22:01:51.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[40290]: osdmap e39: 6 total, 6 up, 6 in 2024-09-20T22:01:51.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-20T22:01:51.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:51.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:51.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:51.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:01:51.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:51.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:51.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:01:51.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:01:51.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:51 smithi060 ceph-mon[40290]: osdmap e40: 6 total, 6 up, 6 in 2024-09-20T22:01:52.075 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T22:01:52.102 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi067:vg_nvme/lv_2 2024-09-20T22:01:52.356 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:01:52.665 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:52 smithi067 ceph-mon[33511]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:01:52.665 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:52 smithi067 ceph-mon[33511]: Detected new or changed devices on smithi067 2024-09-20T22:01:52.665 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:52 smithi067 ceph-mon[33511]: Adjusting osd_memory_target on smithi067 to 8544M 2024-09-20T22:01:52.665 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:52 smithi067 ceph-mon[33511]: osdmap e41: 6 total, 6 up, 6 in 2024-09-20T22:01:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:52 smithi060 ceph-mon[29694]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:01:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:52 smithi060 ceph-mon[29694]: Detected new or changed devices on smithi067 2024-09-20T22:01:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:52 smithi060 ceph-mon[29694]: Adjusting osd_memory_target on smithi067 to 8544M 2024-09-20T22:01:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:52 smithi060 ceph-mon[29694]: osdmap e41: 6 total, 6 up, 6 in 2024-09-20T22:01:52.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:52 smithi060 ceph-mon[40290]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:01:52.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:52 smithi060 ceph-mon[40290]: Detected new or changed devices on smithi067 2024-09-20T22:01:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:52 smithi060 ceph-mon[40290]: Adjusting osd_memory_target on smithi067 to 8544M 2024-09-20T22:01:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:52 smithi060 ceph-mon[40290]: osdmap e41: 6 total, 6 up, 6 in 2024-09-20T22:01:53.736 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:53 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T22:01:53.736 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:53 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T22:01:53.736 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:53 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:53.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:53 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T22:01:53.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:53 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T22:01:53.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:53 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:53.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:53 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T22:01:53.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:53 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T22:01:53.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:53 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:01:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:54 smithi067 ceph-mon[33511]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:01:54.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:54 smithi067 ceph-mon[33511]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:01:54.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:54 smithi067 ceph-mon[33511]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:01:54.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:54 smithi067 ceph-mon[33511]: osdmap e42: 6 total, 6 up, 6 in 2024-09-20T22:01:54.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:54 smithi060 ceph-mon[29694]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:01:54.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:54 smithi060 ceph-mon[29694]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:01:54.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:54 smithi060 ceph-mon[29694]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:01:54.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:54 smithi060 ceph-mon[29694]: osdmap e42: 6 total, 6 up, 6 in 2024-09-20T22:01:54.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:54 smithi060 ceph-mon[40290]: pgmap v134: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:01:54.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:54 smithi060 ceph-mon[40290]: from='client.24250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:01:54.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:54 smithi060 ceph-mon[40290]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:01:54.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:54 smithi060 ceph-mon[40290]: osdmap e42: 6 total, 6 up, 6 in 2024-09-20T22:01:55.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:55 smithi060 ceph-mon[29694]: osdmap e43: 6 total, 6 up, 6 in 2024-09-20T22:01:55.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:55 smithi060 ceph-mon[29694]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:01:55.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:55 smithi060 ceph-mon[29694]: from='client.? 172.21.15.67:0/3047339376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a359116e-c013-4e2f-8b79-86219516ee16"}]: dispatch 2024-09-20T22:01:55.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:55 smithi060 ceph-mon[29694]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a359116e-c013-4e2f-8b79-86219516ee16"}]: dispatch 2024-09-20T22:01:55.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:55 smithi060 ceph-mon[29694]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a359116e-c013-4e2f-8b79-86219516ee16"}]': finished 2024-09-20T22:01:55.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:55 smithi060 ceph-mon[29694]: osdmap e44: 7 total, 6 up, 7 in 2024-09-20T22:01:55.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:55 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:01:55.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:55 smithi060 ceph-mon[40290]: osdmap e43: 6 total, 6 up, 6 in 2024-09-20T22:01:55.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:55 smithi060 ceph-mon[40290]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:01:55.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:55 smithi060 ceph-mon[40290]: from='client.? 172.21.15.67:0/3047339376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a359116e-c013-4e2f-8b79-86219516ee16"}]: dispatch 2024-09-20T22:01:55.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:55 smithi060 ceph-mon[40290]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a359116e-c013-4e2f-8b79-86219516ee16"}]: dispatch 2024-09-20T22:01:55.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:55 smithi060 ceph-mon[40290]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a359116e-c013-4e2f-8b79-86219516ee16"}]': finished 2024-09-20T22:01:55.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:55 smithi060 ceph-mon[40290]: osdmap e44: 7 total, 6 up, 7 in 2024-09-20T22:01:55.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:55 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:01:55.874 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:55 smithi067 ceph-mon[33511]: osdmap e43: 6 total, 6 up, 6 in 2024-09-20T22:01:55.874 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:55 smithi067 ceph-mon[33511]: pgmap v137: 1 pgs: 1 remapped+peering; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:01:55.874 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.67:0/3047339376' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a359116e-c013-4e2f-8b79-86219516ee16"}]: dispatch 2024-09-20T22:01:55.874 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:55 smithi067 ceph-mon[33511]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a359116e-c013-4e2f-8b79-86219516ee16"}]: dispatch 2024-09-20T22:01:55.874 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:55 smithi067 ceph-mon[33511]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a359116e-c013-4e2f-8b79-86219516ee16"}]': finished 2024-09-20T22:01:55.874 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:55 smithi067 ceph-mon[33511]: osdmap e44: 7 total, 6 up, 7 in 2024-09-20T22:01:55.874 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:55 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:01:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:56 smithi060 ceph-mon[29694]: from='client.? 172.21.15.67:0/4246447063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T22:01:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:56 smithi060 ceph-mon[40290]: from='client.? 172.21.15.67:0/4246447063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T22:01:57.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:56 smithi067 ceph-mon[33511]: from='client.? 172.21.15.67:0/4246447063' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T22:01:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:57 smithi067 ceph-mon[33511]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-09-20T22:01:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:57 smithi060 ceph-mon[29694]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-09-20T22:01:58.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:57 smithi060 ceph-mon[40290]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-09-20T22:01:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:58 smithi067 ceph-mon[33511]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-20T22:01:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:58 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:01:59.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:58 smithi060 ceph-mon[29694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-20T22:01:59.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:58 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:01:59.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:58 smithi060 ceph-mon[40290]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-20T22:01:59.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:58 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:01:59.746 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:01:59 smithi067 ceph-mon[33511]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-20T22:02:00.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:01:59 smithi060 ceph-mon[29694]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-20T22:02:00.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:01:59 smithi060 ceph-mon[40290]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-20T22:02:02.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:01 smithi067 ceph-mon[33511]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-20T22:02:02.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:01 smithi060 ceph-mon[29694]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-20T22:02:02.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:01 smithi060 ceph-mon[40290]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-20T22:02:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:02 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:02:03.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:02 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:02:03.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:02 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-20T22:02:03.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:02 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:03.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:02 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:02:03.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:02 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:02:03.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:02 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-20T22:02:03.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:02 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:02 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:02:03.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:02 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:02:03.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:02 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-20T22:02:03.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:02 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:03.713 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:03 smithi067 ceph-mon[33511]: Deploying daemon osd.6 on smithi067 2024-09-20T22:02:03.713 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:03 smithi067 ceph-mon[33511]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-20T22:02:03.713 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:03 smithi067 ceph-mon[33511]: osdmap e45: 7 total, 6 up, 7 in 2024-09-20T22:02:03.713 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:03 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:02:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:03 smithi060 ceph-mon[29694]: Deploying daemon osd.6 on smithi067 2024-09-20T22:02:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:03 smithi060 ceph-mon[29694]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-20T22:02:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:03 smithi060 ceph-mon[29694]: osdmap e45: 7 total, 6 up, 7 in 2024-09-20T22:02:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:03 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:02:04.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:03 smithi060 ceph-mon[40290]: Deploying daemon osd.6 on smithi067 2024-09-20T22:02:04.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:03 smithi060 ceph-mon[40290]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-20T22:02:04.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:03 smithi060 ceph-mon[40290]: osdmap e45: 7 total, 6 up, 7 in 2024-09-20T22:02:04.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:03 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:02:05.984 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:05 smithi067 ceph-mon[33511]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-20T22:02:06.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:05 smithi060 ceph-mon[29694]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-20T22:02:06.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:05 smithi060 ceph-mon[40290]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 170 MiB used, 536 GiB / 536 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-20T22:02:08.290 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:08 smithi067 ceph-mon[33511]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:02:08.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:08 smithi060 ceph-mon[29694]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:02:08.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:08 smithi060 ceph-mon[40290]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:02:09.395 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:09 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:02:09.396 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:09 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:09.396 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:09 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:09.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:09 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:02:09.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:09 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:09.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:09 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:09.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:09 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:02:09.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:09 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:09.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:09 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:10.233 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:10 smithi067 ceph-mon[33511]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:02:10.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:10 smithi060 ceph-mon[29694]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:02:10.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:10 smithi060 ceph-mon[40290]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:02:11.992 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 6 on host 'smithi067' 2024-09-20T22:02:12.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:12 smithi067 ceph-mon[33511]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:02:12.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:12 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:12.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:12 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:12.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:12 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:12.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:12 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:02:12.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:12 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:12.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:12 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:02:12.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:12 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:12.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:12 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:12.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:12 smithi060 ceph-mon[29694]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:02:12.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:12 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:12.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:12 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:12.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:12 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:12.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:12 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:02:12.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:12 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:12.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:12 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:02:12.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:12 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:12.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:12 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:12.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:12 smithi060 ceph-mon[40290]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:02:12.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:12 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:12.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:12 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:12.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:12 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:12.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:12 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:02:12.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:12 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:12.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:12 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:02:12.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:12 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:12.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:12 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:12.632 DEBUG:teuthology.orchestra.run.smithi067:osd.6> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.6.service 2024-09-20T22:02:12.635 INFO:tasks.cephadm:Deploying osd.7 on smithi067 with /dev/vg_nvme/lv_1... 2024-09-20T22:02:12.635 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-20T22:02:13.432 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:02:13.513 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:02:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:02:13.367+0000 7fa9cb77f640 -1 osd.6 0 log_to_monitors true 2024-09-20T22:02:14.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:14 smithi067 ceph-mon[33511]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:02:14.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:14 smithi067 ceph-mon[33511]: from='osd.6 [v2:172.21.15.67:6816/1187114,v1:172.21.15.67:6817/1187114]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-20T22:02:14.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:14 smithi067 ceph-mon[33511]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-20T22:02:14.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:14 smithi060 ceph-mon[29694]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:02:14.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:14 smithi060 ceph-mon[29694]: from='osd.6 [v2:172.21.15.67:6816/1187114,v1:172.21.15.67:6817/1187114]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-20T22:02:14.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:14 smithi060 ceph-mon[29694]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-20T22:02:14.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:14 smithi060 ceph-mon[40290]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:02:14.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:14 smithi060 ceph-mon[40290]: from='osd.6 [v2:172.21.15.67:6816/1187114,v1:172.21.15.67:6817/1187114]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-20T22:02:14.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:14 smithi060 ceph-mon[40290]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-20T22:02:15.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:15 smithi067 ceph-mon[33511]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-20T22:02:15.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:15 smithi067 ceph-mon[33511]: osdmap e46: 7 total, 6 up, 7 in 2024-09-20T22:02:15.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:15 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:02:15.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:15 smithi067 ceph-mon[33511]: from='osd.6 [v2:172.21.15.67:6816/1187114,v1:172.21.15.67:6817/1187114]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:02:15.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:15 smithi067 ceph-mon[33511]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:02:15.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:15 smithi060 ceph-mon[29694]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-20T22:02:15.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:15 smithi060 ceph-mon[29694]: osdmap e46: 7 total, 6 up, 7 in 2024-09-20T22:02:15.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:15 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:02:15.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:15 smithi060 ceph-mon[29694]: from='osd.6 [v2:172.21.15.67:6816/1187114,v1:172.21.15.67:6817/1187114]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:02:15.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:15 smithi060 ceph-mon[29694]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:02:15.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:15 smithi060 ceph-mon[40290]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-20T22:02:15.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:15 smithi060 ceph-mon[40290]: osdmap e46: 7 total, 6 up, 7 in 2024-09-20T22:02:15.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:15 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:02:15.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:15 smithi060 ceph-mon[40290]: from='osd.6 [v2:172.21.15.67:6816/1187114,v1:172.21.15.67:6817/1187114]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:02:15.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:15 smithi060 ceph-mon[40290]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:02:16.007 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:02:15 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:02:15.708+0000 7fa9c7702640 -1 osd.6 0 waiting for initial osdmap 2024-09-20T22:02:16.007 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:02:15 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:02:15.714+0000 7fa9bf522640 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T22:02:16.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:16 smithi067 ceph-mon[33511]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:02:16.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:16 smithi067 ceph-mon[33511]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-20T22:02:16.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:16 smithi067 ceph-mon[33511]: osdmap e47: 7 total, 6 up, 7 in 2024-09-20T22:02:16.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:16 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:02:16.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:16 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:02:16.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:16 smithi067 ceph-mon[33511]: from='osd.6 ' entity='osd.6' 2024-09-20T22:02:16.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:16 smithi067 ceph-mon[33511]: osd.6 [v2:172.21.15.67:6816/1187114,v1:172.21.15.67:6817/1187114] boot 2024-09-20T22:02:16.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:16 smithi067 ceph-mon[33511]: osdmap e48: 7 total, 7 up, 7 in 2024-09-20T22:02:16.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:16 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:02:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:16 smithi060 ceph-mon[29694]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:02:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:16 smithi060 ceph-mon[29694]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-20T22:02:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:16 smithi060 ceph-mon[29694]: osdmap e47: 7 total, 6 up, 7 in 2024-09-20T22:02:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:16 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:02:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:16 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:02:16.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:16 smithi060 ceph-mon[29694]: from='osd.6 ' entity='osd.6' 2024-09-20T22:02:16.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:16 smithi060 ceph-mon[29694]: osd.6 [v2:172.21.15.67:6816/1187114,v1:172.21.15.67:6817/1187114] boot 2024-09-20T22:02:16.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:16 smithi060 ceph-mon[29694]: osdmap e48: 7 total, 7 up, 7 in 2024-09-20T22:02:16.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:16 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:02:16.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:16 smithi060 ceph-mon[40290]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 171 MiB used, 536 GiB / 536 GiB avail 2024-09-20T22:02:16.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:16 smithi060 ceph-mon[40290]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-20T22:02:16.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:16 smithi060 ceph-mon[40290]: osdmap e47: 7 total, 6 up, 7 in 2024-09-20T22:02:16.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:16 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:02:16.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:16 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:02:16.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:16 smithi060 ceph-mon[40290]: from='osd.6 ' entity='osd.6' 2024-09-20T22:02:16.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:16 smithi060 ceph-mon[40290]: osd.6 [v2:172.21.15.67:6816/1187114,v1:172.21.15.67:6817/1187114] boot 2024-09-20T22:02:16.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:16 smithi060 ceph-mon[40290]: osdmap e48: 7 total, 7 up, 7 in 2024-09-20T22:02:16.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:16 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:02:17.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:17 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:02:17.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:17 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:02:17.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:17.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:17.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:17.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:17.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:17.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:17.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:17.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:02:17.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:17 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:17.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:17 smithi067 ceph-mon[33511]: osdmap e49: 7 total, 7 up, 7 in 2024-09-20T22:02:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:02:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:02:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:02:17.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:17.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[29694]: osdmap e49: 7 total, 7 up, 7 in 2024-09-20T22:02:17.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:02:17.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:02:17.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:17.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:17.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:17.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:17.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:17.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:17.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:17.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:02:17.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:17.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:17 smithi060 ceph-mon[40290]: osdmap e49: 7 total, 7 up, 7 in 2024-09-20T22:02:18.033 INFO:teuthology.orchestra.run.smithi067.stdout: 2024-09-20T22:02:18.060 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi067:vg_nvme/lv_1 2024-09-20T22:02:18.334 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:02:18.455 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:18 smithi067 ceph-mon[33511]: Detected new or changed devices on smithi067 2024-09-20T22:02:18.455 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:18 smithi067 ceph-mon[33511]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 625 GiB / 626 GiB avail 2024-09-20T22:02:18.455 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:18 smithi067 ceph-mon[33511]: Adjusting osd_memory_target on smithi067 to 5696M 2024-09-20T22:02:18.455 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:18 smithi067 ceph-mon[33511]: osdmap e50: 7 total, 7 up, 7 in 2024-09-20T22:02:18.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:18 smithi060 ceph-mon[29694]: Detected new or changed devices on smithi067 2024-09-20T22:02:18.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:18 smithi060 ceph-mon[29694]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 625 GiB / 626 GiB avail 2024-09-20T22:02:18.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:18 smithi060 ceph-mon[29694]: Adjusting osd_memory_target on smithi067 to 5696M 2024-09-20T22:02:18.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:18 smithi060 ceph-mon[29694]: osdmap e50: 7 total, 7 up, 7 in 2024-09-20T22:02:18.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:18 smithi060 ceph-mon[40290]: Detected new or changed devices on smithi067 2024-09-20T22:02:18.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:18 smithi060 ceph-mon[40290]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 365 MiB used, 625 GiB / 626 GiB avail 2024-09-20T22:02:18.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:18 smithi060 ceph-mon[40290]: Adjusting osd_memory_target on smithi067 to 5696M 2024-09-20T22:02:18.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:18 smithi060 ceph-mon[40290]: osdmap e50: 7 total, 7 up, 7 in 2024-09-20T22:02:19.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:19 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T22:02:19.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:19 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T22:02:19.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:19 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:19.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:19 smithi067 ceph-mon[33511]: osdmap e51: 7 total, 7 up, 7 in 2024-09-20T22:02:19.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:19 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T22:02:19.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:19 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T22:02:19.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:19 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:19.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:19 smithi060 ceph-mon[29694]: osdmap e51: 7 total, 7 up, 7 in 2024-09-20T22:02:19.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:19 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-20T22:02:19.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:19 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-20T22:02:19.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:19 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:19.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:19 smithi060 ceph-mon[40290]: osdmap e51: 7 total, 7 up, 7 in 2024-09-20T22:02:20.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:20 smithi067 ceph-mon[33511]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 365 MiB used, 625 GiB / 626 GiB avail 2024-09-20T22:02:20.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:20 smithi067 ceph-mon[33511]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:02:20.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:20 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:02:20.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:20 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:02:20.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:20 smithi067 ceph-mon[33511]: osdmap e52: 7 total, 7 up, 7 in 2024-09-20T22:02:20.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:20 smithi060 ceph-mon[29694]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 365 MiB used, 625 GiB / 626 GiB avail 2024-09-20T22:02:20.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:20 smithi060 ceph-mon[29694]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:02:20.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:20 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:02:20.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:20 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:02:20.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:20 smithi060 ceph-mon[29694]: osdmap e52: 7 total, 7 up, 7 in 2024-09-20T22:02:20.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:20 smithi060 ceph-mon[40290]: pgmap v156: 1 pgs: 1 remapped+peering; 577 KiB data, 365 MiB used, 625 GiB / 626 GiB avail 2024-09-20T22:02:20.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:20 smithi060 ceph-mon[40290]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi067:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:02:20.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:20 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:02:20.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:20 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:02:20.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:20 smithi060 ceph-mon[40290]: osdmap e52: 7 total, 7 up, 7 in 2024-09-20T22:02:22.338 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:22 smithi060 ceph-mon[29694]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 365 MiB used, 625 GiB / 626 GiB avail 2024-09-20T22:02:22.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:22 smithi060 ceph-mon[29694]: from='client.? 172.21.15.67:0/3972306290' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4bf4188-f133-4a31-8897-d7e8182248d7"}]: dispatch 2024-09-20T22:02:22.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:22 smithi060 ceph-mon[29694]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4bf4188-f133-4a31-8897-d7e8182248d7"}]: dispatch 2024-09-20T22:02:22.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:22 smithi060 ceph-mon[29694]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4bf4188-f133-4a31-8897-d7e8182248d7"}]': finished 2024-09-20T22:02:22.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:22 smithi060 ceph-mon[29694]: osdmap e53: 8 total, 7 up, 8 in 2024-09-20T22:02:22.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:22 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:02:22.339 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:22 smithi060 ceph-mon[29694]: from='client.? 172.21.15.67:0/2399465044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T22:02:22.339 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:22 smithi060 ceph-mon[40290]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 365 MiB used, 625 GiB / 626 GiB avail 2024-09-20T22:02:22.340 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:22 smithi060 ceph-mon[40290]: from='client.? 172.21.15.67:0/3972306290' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4bf4188-f133-4a31-8897-d7e8182248d7"}]: dispatch 2024-09-20T22:02:22.340 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:22 smithi060 ceph-mon[40290]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4bf4188-f133-4a31-8897-d7e8182248d7"}]: dispatch 2024-09-20T22:02:22.340 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:22 smithi060 ceph-mon[40290]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4bf4188-f133-4a31-8897-d7e8182248d7"}]': finished 2024-09-20T22:02:22.340 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:22 smithi060 ceph-mon[40290]: osdmap e53: 8 total, 7 up, 8 in 2024-09-20T22:02:22.340 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:22 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:02:22.340 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:22 smithi060 ceph-mon[40290]: from='client.? 172.21.15.67:0/2399465044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T22:02:22.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:22 smithi067 ceph-mon[33511]: pgmap v159: 1 pgs: 1 remapped+peering; 577 KiB data, 365 MiB used, 625 GiB / 626 GiB avail 2024-09-20T22:02:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:22 smithi067 ceph-mon[33511]: from='client.? 172.21.15.67:0/3972306290' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4bf4188-f133-4a31-8897-d7e8182248d7"}]: dispatch 2024-09-20T22:02:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:22 smithi067 ceph-mon[33511]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4bf4188-f133-4a31-8897-d7e8182248d7"}]: dispatch 2024-09-20T22:02:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:22 smithi067 ceph-mon[33511]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4bf4188-f133-4a31-8897-d7e8182248d7"}]': finished 2024-09-20T22:02:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:22 smithi067 ceph-mon[33511]: osdmap e53: 8 total, 7 up, 8 in 2024-09-20T22:02:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:22 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:02:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:22 smithi067 ceph-mon[33511]: from='client.? 172.21.15.67:0/2399465044' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-20T22:02:24.491 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:24 smithi067 ceph-mon[33511]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-09-20T22:02:24.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:24 smithi060 ceph-mon[29694]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-09-20T22:02:24.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:24 smithi060 ceph-mon[40290]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-09-20T22:02:26.342 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:26 smithi067 ceph-mon[33511]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-20T22:02:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:26 smithi060 ceph-mon[29694]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-20T22:02:26.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:26 smithi060 ceph-mon[40290]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-20T22:02:27.230 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:27 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:02:27.230 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:27 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:02:27.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:27 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:02:27.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:27 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:02:27.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:27 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:02:27.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:27 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:02:28.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:28 smithi067 ceph-mon[33511]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-20T22:02:28.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:28 smithi067 ceph-mon[33511]: osdmap e54: 8 total, 7 up, 8 in 2024-09-20T22:02:28.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:28 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:02:28.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:28 smithi060 ceph-mon[29694]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-20T22:02:28.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:28 smithi060 ceph-mon[29694]: osdmap e54: 8 total, 7 up, 8 in 2024-09-20T22:02:28.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:28 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:02:28.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:28 smithi060 ceph-mon[40290]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-20T22:02:28.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:28 smithi060 ceph-mon[40290]: osdmap e54: 8 total, 7 up, 8 in 2024-09-20T22:02:28.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:28 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:02:29.225 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:29 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-20T22:02:29.226 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:29 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:29.226 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:29 smithi067 ceph-mon[33511]: Deploying daemon osd.7 on smithi067 2024-09-20T22:02:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:29 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-20T22:02:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:29 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:29 smithi060 ceph-mon[29694]: Deploying daemon osd.7 on smithi067 2024-09-20T22:02:29.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:29 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-20T22:02:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:29 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:29 smithi060 ceph-mon[40290]: Deploying daemon osd.7 on smithi067 2024-09-20T22:02:30.355 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:30 smithi067 ceph-mon[33511]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-20T22:02:30.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:30 smithi060 ceph-mon[29694]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-20T22:02:30.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:30 smithi060 ceph-mon[40290]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-20T22:02:32.435 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:32 smithi067 ceph-mon[33511]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-20T22:02:32.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:32 smithi060 ceph-mon[29694]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-20T22:02:32.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:32 smithi060 ceph-mon[40290]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-09-20T22:02:34.327 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:34 smithi067 ceph-mon[33511]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-20T22:02:34.327 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:34 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:02:34.327 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:34 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:34.327 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:34 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:34.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:34 smithi060 ceph-mon[29694]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-20T22:02:34.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:34 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:02:34.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:34 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:34.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:34 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:34.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:34 smithi060 ceph-mon[40290]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-20T22:02:34.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:34 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:02:34.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:34 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:34.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:34 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:36.297 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:36 smithi067 ceph-mon[33511]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-20T22:02:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:36 smithi060 ceph-mon[29694]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-20T22:02:36.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:36 smithi060 ceph-mon[40290]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-20T22:02:37.051 INFO:teuthology.orchestra.run.smithi067.stdout:Created osd(s) 7 on host 'smithi067' 2024-09-20T22:02:37.711 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:37 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:37.711 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:37 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:37.711 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:37 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:37.711 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:37 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:02:37.711 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:37 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:37.711 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:37 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:02:37.711 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:37 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:37.711 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:37 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:37.738 DEBUG:teuthology.orchestra.run.smithi067:osd.7> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.7.service 2024-09-20T22:02:37.741 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-20T22:02:37.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-20T22:02:37.769 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:37 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:37.770 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:37 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:37.770 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:37 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:37.770 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:37 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:02:37.770 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:37 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:37.770 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:37 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:02:37.770 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:37 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:37.770 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:37 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:37.770 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:37 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:37.770 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:37 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:37.770 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:37 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:37.771 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:37 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:02:37.771 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:37 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:37.771 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:37 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:02:37.771 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:37 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:37.771 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:37 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:38.011 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:02:38.439 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:38 smithi060 ceph-mon[29694]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-20T22:02:38.439 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:38 smithi060 ceph-mon[40290]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-20T22:02:38.608 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:38 smithi067 ceph-mon[33511]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-20T22:02:38.609 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:02:39.155 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:02:38 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:02:38.893+0000 7fc6f7c68640 -1 osd.7 0 log_to_monitors true 2024-09-20T22:02:39.308 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":54,"num_osds":8,"num_up_osds":7,"osd_up_since":1726869736,"num_in_osds":8,"osd_in_since":1726869741,"num_remapped_pgs":0} 2024-09-20T22:02:39.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:39 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/625659103' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-20T22:02:39.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:39 smithi067 ceph-mon[33511]: from='osd.7 [v2:172.21.15.67:6824/3021842840,v1:172.21.15.67:6825/3021842840]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-20T22:02:39.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:39 smithi067 ceph-mon[33511]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-20T22:02:39.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:39 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/625659103' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-20T22:02:39.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:39 smithi060 ceph-mon[29694]: from='osd.7 [v2:172.21.15.67:6824/3021842840,v1:172.21.15.67:6825/3021842840]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-20T22:02:39.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:39 smithi060 ceph-mon[29694]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-20T22:02:39.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:39 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/625659103' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-20T22:02:39.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:39 smithi060 ceph-mon[40290]: from='osd.7 [v2:172.21.15.67:6824/3021842840,v1:172.21.15.67:6825/3021842840]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-20T22:02:39.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:39 smithi060 ceph-mon[40290]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-20T22:02:40.309 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-20T22:02:40.414 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:40 smithi067 ceph-mon[33511]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-20T22:02:40.414 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:40 smithi067 ceph-mon[33511]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-20T22:02:40.415 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:40 smithi067 ceph-mon[33511]: from='osd.7 [v2:172.21.15.67:6824/3021842840,v1:172.21.15.67:6825/3021842840]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:02:40.415 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:40 smithi067 ceph-mon[33511]: osdmap e55: 8 total, 7 up, 8 in 2024-09-20T22:02:40.415 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:40 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:02:40.415 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:40 smithi067 ceph-mon[33511]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:02:40.587 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:02:40.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:40 smithi060 ceph-mon[29694]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-20T22:02:40.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:40 smithi060 ceph-mon[29694]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-20T22:02:40.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:40 smithi060 ceph-mon[29694]: from='osd.7 [v2:172.21.15.67:6824/3021842840,v1:172.21.15.67:6825/3021842840]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:02:40.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:40 smithi060 ceph-mon[29694]: osdmap e55: 8 total, 7 up, 8 in 2024-09-20T22:02:40.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:40 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:02:40.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:40 smithi060 ceph-mon[29694]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:02:40.710 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:40 smithi060 ceph-mon[40290]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-20T22:02:40.710 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:40 smithi060 ceph-mon[40290]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-20T22:02:40.710 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:40 smithi060 ceph-mon[40290]: from='osd.7 [v2:172.21.15.67:6824/3021842840,v1:172.21.15.67:6825/3021842840]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:02:40.710 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:40 smithi060 ceph-mon[40290]: osdmap e55: 8 total, 7 up, 8 in 2024-09-20T22:02:40.710 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:40 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:02:40.710 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:40 smithi060 ceph-mon[40290]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:02:41.230 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:02:41.263 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:02:40 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:02:40.806+0000 7fc6f3beb640 -1 osd.7 0 waiting for initial osdmap 2024-09-20T22:02:41.263 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:02:40 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:02:40.812+0000 7fc6eba0b640 -1 osd.7 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T22:02:41.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-20T22:02:41.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: osdmap e56: 8 total, 7 up, 8 in 2024-09-20T22:02:41.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:02:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:02:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: Detected new or changed devices on smithi067 2024-09-20T22:02:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:02:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: from='osd.7 ' entity='osd.7' 2024-09-20T22:02:41.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: osd.7 [v2:172.21.15.67:6824/3021842840,v1:172.21.15.67:6825/3021842840] boot 2024-09-20T22:02:41.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: osdmap e57: 8 total, 8 up, 8 in 2024-09-20T22:02:41.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:02:41.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4049934015' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-20T22:02:41.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-20T22:02:41.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: osdmap e56: 8 total, 7 up, 8 in 2024-09-20T22:02:41.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:02:41.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:02:41.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: Detected new or changed devices on smithi067 2024-09-20T22:02:41.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:41.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:02:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: from='osd.7 ' entity='osd.7' 2024-09-20T22:02:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: osd.7 [v2:172.21.15.67:6824/3021842840,v1:172.21.15.67:6825/3021842840] boot 2024-09-20T22:02:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: osdmap e57: 8 total, 8 up, 8 in 2024-09-20T22:02:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:02:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:41 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4049934015' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-20T22:02:41.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]': finished 2024-09-20T22:02:41.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: osdmap e56: 8 total, 7 up, 8 in 2024-09-20T22:02:41.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:02:41.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:02:41.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: Detected new or changed devices on smithi067 2024-09-20T22:02:41.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:41.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:41.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:41.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:41.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:41.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:02:41.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:41.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:41.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:02:41.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:41.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: from='osd.7 ' entity='osd.7' 2024-09-20T22:02:41.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: osd.7 [v2:172.21.15.67:6824/3021842840,v1:172.21.15.67:6825/3021842840] boot 2024-09-20T22:02:41.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: osdmap e57: 8 total, 8 up, 8 in 2024-09-20T22:02:41.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:02:41.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:41 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4049934015' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-20T22:02:42.034 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":57,"num_osds":8,"num_up_osds":8,"osd_up_since":1726869761,"num_in_osds":8,"osd_in_since":1726869741,"num_remapped_pgs":1} 2024-09-20T22:02:42.034 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-20T22:02:42.293 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:02:42.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:42 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:02:42.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:42 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:02:42.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:42 smithi060 ceph-mon[29694]: Adjusting osd_memory_target on smithi067 to 4272M 2024-09-20T22:02:42.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:42 smithi060 ceph-mon[29694]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-20T22:02:42.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:42 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:02:42.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:42 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:02:42.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:42 smithi060 ceph-mon[29694]: osdmap e58: 8 total, 8 up, 8 in 2024-09-20T22:02:42.630 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:42 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:02:42.630 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:42 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:02:42.631 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:42 smithi060 ceph-mon[40290]: Adjusting osd_memory_target on smithi067 to 4272M 2024-09-20T22:02:42.631 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:42 smithi060 ceph-mon[40290]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-20T22:02:42.631 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:42 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:02:42.631 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:42 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:02:42.631 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:42 smithi060 ceph-mon[40290]: osdmap e58: 8 total, 8 up, 8 in 2024-09-20T22:02:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:42 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:02:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:42 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:02:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:42 smithi067 ceph-mon[33511]: Adjusting osd_memory_target on smithi067 to 4272M 2024-09-20T22:02:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:42 smithi067 ceph-mon[33511]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 198 MiB used, 626 GiB / 626 GiB avail 2024-09-20T22:02:42.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:42 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:02:42.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:42 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:02:42.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:42 smithi067 ceph-mon[33511]: osdmap e58: 8 total, 8 up, 8 in 2024-09-20T22:02:42.899 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:02:42.900 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":58,"fsid":"3df9647e-779b-11ef-baf5-efdc52797490","created":"2024-09-20T21:57:31.244934+0000","modified":"2024-09-20T22:02:42.058780+0000","last_up_change":"2024-09-20T22:02:41.053740+0000","last_in_change":"2024-09-20T22:02:21.189225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-20T22:00:35.131125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0997c01a-a9bd-421e-abcc-fbcf7abe8512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6803","nonce":1395849550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6805","nonce":1395849550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6809","nonce":1395849550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6807","nonce":1395849550}]},"public_addr":"172.21.15.60:6803/1395849550","cluster_addr":"172.21.15.60:6805/1395849550","heartbeat_back_addr":"172.21.15.60:6809/1395849550","heartbeat_front_addr":"172.21.15.60:6807/1395849550","state":["exists","up"]},{"osd":1,"uuid":"c04bdcb1-d76b-47c8-a38b-c02043df6ba8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":145283513},{"type":"v1","addr":"172.21.15.60:6811","nonce":145283513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":145283513},{"type":"v1","addr":"172.21.15.60:6813","nonce":145283513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":145283513},{"type":"v1","addr":"172.21.15.60:6817","nonce":145283513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":145283513},{"type":"v1","addr":"172.21.15.60:6815","nonce":145283513}]},"public_addr":"172.21.15.60:6811/145283513","cluster_addr":"172.21.15.60:6813/145283513","heartbeat_back_addr":"172.21.15.60:6817/145283513","heartbeat_front_addr":"172.21.15.60:6815/145283513","state":["exists","up"]},{"osd":2,"uuid":"32c09d19-eb98-4926-a44e-e7ddee152956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6819","nonce":3698562274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6821","nonce":3698562274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6825","nonce":3698562274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6823","nonce":3698562274}]},"public_addr":"172.21.15.60:6819/3698562274","cluster_addr":"172.21.15.60:6821/3698562274","heartbeat_back_addr":"172.21.15.60:6825/3698562274","heartbeat_front_addr":"172.21.15.60:6823/3698562274","state":["exists","up"]},{"osd":3,"uuid":"b1c2cb69-8565-49ac-b854-7ea7615c6660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6827","nonce":2592322449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6829","nonce":2592322449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6833","nonce":2592322449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6831","nonce":2592322449}]},"public_addr":"172.21.15.60:6827/2592322449","cluster_addr":"172.21.15.60:6829/2592322449","heartbeat_back_addr":"172.21.15.60:6833/2592322449","heartbeat_front_addr":"172.21.15.60:6831/2592322449","state":["exists","up"]},{"osd":4,"uuid":"04a547d6-fc48-43b7-a8d7-18df8a83aa96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":3243564239},{"type":"v1","addr":"172.21.15.67:6801","nonce":3243564239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3243564239},{"type":"v1","addr":"172.21.15.67:6803","nonce":3243564239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3243564239},{"type":"v1","addr":"172.21.15.67:6807","nonce":3243564239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3243564239},{"type":"v1","addr":"172.21.15.67:6805","nonce":3243564239}]},"public_addr":"172.21.15.67:6801/3243564239","cluster_addr":"172.21.15.67:6803/3243564239","heartbeat_back_addr":"172.21.15.67:6807/3243564239","heartbeat_front_addr":"172.21.15.67:6805/3243564239","state":["exists","up"]},{"osd":5,"uuid":"4763aa10-c367-485b-b525-e5241e642b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6809","nonce":1622258887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6811","nonce":1622258887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6815","nonce":1622258887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6813","nonce":1622258887}]},"public_addr":"172.21.15.67:6809/1622258887","cluster_addr":"172.21.15.67:6811/1622258887","heartbeat_back_addr":"172.21.15.67:6815/1622258887","heartbeat_front_addr":"172.21.15.67:6813/1622258887","state":["exists","up"]},{"osd":6,"uuid":"a359116e-c013-4e2f-8b79-86219516ee16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6817","nonce":1187114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6819","nonce":1187114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6823","nonce":1187114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6821","nonce":1187114}]},"public_addr":"172.21.15.67:6817/1187114","cluster_addr":"172.21.15.67:6819/1187114","heartbeat_back_addr":"172.21.15.67:6823/1187114","heartbeat_front_addr":"172.21.15.67:6821/1187114","state":["exists","up"]},{"osd":7,"uuid":"a4bf4188-f133-4a31-8897-d7e8182248d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6825","nonce":3021842840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6827","nonce":3021842840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6831","nonce":3021842840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6829","nonce":3021842840}]},"public_addr":"172.21.15.67:6825/3021842840","cluster_addr":"172.21.15.67:6827/3021842840","heartbeat_back_addr":"172.21.15.67:6831/3021842840","heartbeat_front_addr":"172.21.15.67:6829/3021842840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:19.297917+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:40.779347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:02.406496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:26.764786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:01:22.889371+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:01:48.467136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:14.366664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:39.867765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"1.0","osds":[0,6]}],"primary_temp":[],"blocklist":{"172.21.15.60:0/1871351181":"2024-09-21T21:58:26.613414+0000","172.21.15.60:0/3927294026":"2024-09-21T21:58:26.613414+0000","172.21.15.60:6801/3734574621":"2024-09-21T21:58:26.613414+0000","172.21.15.60:6800/3734574621":"2024-09-21T21:58:26.613414+0000","172.21.15.60:0/1729862661":"2024-09-21T21:58:26.613414+0000","172.21.15.60:6801/3151907776":"2024-09-21T21:58:02.105557+0000","172.21.15.60:0/3965541528":"2024-09-21T21:58:02.105557+0000","172.21.15.60:0/4150585713":"2024-09-21T21:58:02.105557+0000","172.21.15.60:6800/3151907776":"2024-09-21T21:58:02.105557+0000","172.21.15.60:0/956264518":"2024-09-21T21:58:02.105557+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-20T22:02:43.469 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:43 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/788537270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-20T22:02:43.469 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:43 smithi060 ceph-mon[29694]: osdmap e59: 8 total, 8 up, 8 in 2024-09-20T22:02:43.469 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:43 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/788537270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-20T22:02:43.470 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:43 smithi060 ceph-mon[40290]: osdmap e59: 8 total, 8 up, 8 in 2024-09-20T22:02:43.498 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-20T22:00:35.131125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-20T22:02:43.499 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-20T22:02:43.751 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:02:43.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:43 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/788537270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-20T22:02:43.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:43 smithi067 ceph-mon[33511]: osdmap e59: 8 total, 8 up, 8 in 2024-09-20T22:02:44.352 INFO:teuthology.orchestra.run.smithi060.stdout:pg_num: 1 2024-09-20T22:02:44.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:44 smithi060 ceph-mon[29694]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-20T22:02:44.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:44 smithi060 ceph-mon[29694]: osdmap e60: 8 total, 8 up, 8 in 2024-09-20T22:02:44.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:44 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/773533636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-20T22:02:44.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:44 smithi060 ceph-mon[40290]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-20T22:02:44.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:44 smithi060 ceph-mon[40290]: osdmap e60: 8 total, 8 up, 8 in 2024-09-20T22:02:44.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:44 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/773533636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-20T22:02:44.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:44 smithi067 ceph-mon[33511]: pgmap v176: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-20T22:02:44.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:44 smithi067 ceph-mon[33511]: osdmap e60: 8 total, 8 up, 8 in 2024-09-20T22:02:44.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:44 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/773533636' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-20T22:02:44.981 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi060 2024-09-20T22:02:44.981 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch apply rgw foo.a --placement '1;smithi060=foo.a' 2024-09-20T22:02:45.252 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:02:45.984 INFO:teuthology.orchestra.run.smithi067.stdout:Scheduled rgw.foo.a update... 2024-09-20T22:02:46.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:46 smithi067 ceph-mon[33511]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-20T22:02:46.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:46 smithi067 ceph-mon[33511]: osdmap e61: 8 total, 8 up, 8 in 2024-09-20T22:02:46.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:46 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:46.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:46 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:02:46.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:46 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:46.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:46 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:02:46.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:46 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:46.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:46 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-20T22:02:46.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:46 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-20T22:02:46.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:46 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:46.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:46 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:46.444 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[29694]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-20T22:02:46.444 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[29694]: osdmap e61: 8 total, 8 up, 8 in 2024-09-20T22:02:46.444 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:46.444 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:02:46.444 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:46.444 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:02:46.444 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:46.444 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-20T22:02:46.444 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-20T22:02:46.444 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:46.444 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:46.445 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[40290]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 225 MiB used, 715 GiB / 715 GiB avail 2024-09-20T22:02:46.445 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[40290]: osdmap e61: 8 total, 8 up, 8 in 2024-09-20T22:02:46.445 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:46.445 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:02:46.445 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:46.445 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:02:46.445 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:46.445 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-20T22:02:46.445 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-20T22:02:46.445 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:46.445 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:46 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:46.606 DEBUG:teuthology.orchestra.run.smithi060:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@rgw.foo.a.service 2024-09-20T22:02:46.608 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi067 2024-09-20T22:02:46.608 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool create datapool 3 3 replicated 2024-09-20T22:02:46.867 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:02:47.139 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:47 smithi067 ceph-mon[33511]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi060=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:02:47.139 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:47 smithi067 ceph-mon[33511]: Saving service rgw.foo.a spec with placement smithi060=foo.a;count:1 2024-09-20T22:02:47.139 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:47 smithi067 ceph-mon[33511]: Deploying daemon rgw.foo.a on smithi060 2024-09-20T22:02:47.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:47 smithi060 ceph-mon[29694]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi060=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:02:47.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:47 smithi060 ceph-mon[29694]: Saving service rgw.foo.a spec with placement smithi060=foo.a;count:1 2024-09-20T22:02:47.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:47 smithi060 ceph-mon[29694]: Deploying daemon rgw.foo.a on smithi060 2024-09-20T22:02:47.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:47 smithi060 ceph-mon[40290]: from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi060=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:02:47.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:47 smithi060 ceph-mon[40290]: Saving service rgw.foo.a spec with placement smithi060=foo.a;count:1 2024-09-20T22:02:47.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:47 smithi060 ceph-mon[40290]: Deploying daemon rgw.foo.a on smithi060 2024-09-20T22:02:48.084 INFO:teuthology.orchestra.run.smithi067.stderr:pool 'datapool' created 2024-09-20T22:02:48.220 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:48 smithi060 ceph-mon[29694]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-09-20T22:02:48.220 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:48 smithi060 ceph-mon[29694]: from='client.? 172.21.15.67:0/1468678610' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-20T22:02:48.220 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:48 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-20T22:02:48.220 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:48 smithi060 ceph-mon[40290]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-09-20T22:02:48.220 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:48 smithi060 ceph-mon[40290]: from='client.? 172.21.15.67:0/1468678610' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-20T22:02:48.220 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:48 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-20T22:02:48.352 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:48 smithi067 ceph-mon[33511]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 125 KiB/s, 0 objects/s recovering 2024-09-20T22:02:48.352 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:48 smithi067 ceph-mon[33511]: from='client.? 172.21.15.67:0/1468678610' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-20T22:02:48.352 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:48 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-20T22:02:48.496 INFO:journalctl@ceph.rgw.foo.a.smithi060.stdout:Sep 20 22:02:48 smithi060 podman[67933]: 2024-09-20 22:02:48.43389575 +0000 UTC m=+0.318366363 container start e3bab278a660b7ae9c8cd3bf24dc50009dc9e130df1830a8de9b4481c6a47676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-rgw-foo-a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-9e2ac70, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-20T22:02:48.646 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- rbd pool init datapool 2024-09-20T22:02:48.786 INFO:journalctl@ceph.rgw.foo.a.smithi060.stdout:Sep 20 22:02:48 smithi060 bash[67933]: e3bab278a660b7ae9c8cd3bf24dc50009dc9e130df1830a8de9b4481c6a47676 2024-09-20T22:02:48.786 INFO:journalctl@ceph.rgw.foo.a.smithi060.stdout:Sep 20 22:02:48 smithi060 systemd[1]: Started Ceph rgw.foo.a for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T22:02:48.905 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:02:49.364 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:49 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-20T22:02:49.364 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:49 smithi067 ceph-mon[33511]: osdmap e62: 8 total, 8 up, 8 in 2024-09-20T22:02:49.365 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:49 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:49.365 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:49 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:49.365 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:49 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:49.365 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:49 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:49.365 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:49 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:49.365 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:49 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:02:49.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:49 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-20T22:02:49.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:49 smithi060 ceph-mon[29694]: osdmap e62: 8 total, 8 up, 8 in 2024-09-20T22:02:49.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:49 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:49.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:49 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:49.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:49 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:49.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:49 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:49.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:49 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:49.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:49 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:02:49.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:49 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-20T22:02:49.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:49 smithi060 ceph-mon[40290]: osdmap e62: 8 total, 8 up, 8 in 2024-09-20T22:02:49.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:49 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:49.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:49 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:49.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:49 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:49.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:49 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:49.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:49 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:49.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:49 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:02:50.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:50 smithi060 ceph-mon[29694]: pgmap v183: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-20T22:02:50.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:50 smithi060 ceph-mon[29694]: Saving service rgw.foo.a spec with placement smithi060=foo.a;count:1 2024-09-20T22:02:50.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:50 smithi060 ceph-mon[29694]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T22:02:50.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:50 smithi060 ceph-mon[29694]: osdmap e63: 8 total, 8 up, 8 in 2024-09-20T22:02:50.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:50 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2926031166' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-20T22:02:50.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:50 smithi060 ceph-mon[29694]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-20T22:02:50.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:50 smithi060 ceph-mon[29694]: from='client.? 172.21.15.67:0/609461033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-20T22:02:50.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:50 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-20T22:02:50.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:50 smithi060 ceph-mon[40290]: pgmap v183: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-20T22:02:50.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:50 smithi060 ceph-mon[40290]: Saving service rgw.foo.a spec with placement smithi060=foo.a;count:1 2024-09-20T22:02:50.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:50 smithi060 ceph-mon[40290]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T22:02:50.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:50 smithi060 ceph-mon[40290]: osdmap e63: 8 total, 8 up, 8 in 2024-09-20T22:02:50.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:50 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2926031166' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-20T22:02:50.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:50 smithi060 ceph-mon[40290]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-20T22:02:50.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:50 smithi060 ceph-mon[40290]: from='client.? 172.21.15.67:0/609461033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-20T22:02:50.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:50 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-20T22:02:50.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:50 smithi067 ceph-mon[33511]: pgmap v183: 4 pgs: 3 unknown, 1 active+clean; 577 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-20T22:02:50.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:50 smithi067 ceph-mon[33511]: Saving service rgw.foo.a spec with placement smithi060=foo.a;count:1 2024-09-20T22:02:50.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:50 smithi067 ceph-mon[33511]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T22:02:50.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:50 smithi067 ceph-mon[33511]: osdmap e63: 8 total, 8 up, 8 in 2024-09-20T22:02:50.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:50 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2926031166' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-20T22:02:50.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:50 smithi067 ceph-mon[33511]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-20T22:02:50.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:50 smithi067 ceph-mon[33511]: from='client.? 172.21.15.67:0/609461033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-20T22:02:50.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:50 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-20T22:02:51.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:51 smithi067 ceph-mon[33511]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-20T22:02:51.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:51 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-20T22:02:51.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:51 smithi067 ceph-mon[33511]: osdmap e64: 8 total, 8 up, 8 in 2024-09-20T22:02:51.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:51 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:51.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:51 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:51.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:51 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:51.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:51 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:02:51.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:51 smithi060 ceph-mon[29694]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-20T22:02:51.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:51 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-20T22:02:51.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:51 smithi060 ceph-mon[29694]: osdmap e64: 8 total, 8 up, 8 in 2024-09-20T22:02:51.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:51 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:51.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:51 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:51.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:51 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:51.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:51 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:02:51.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:51 smithi060 ceph-mon[40290]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-20T22:02:51.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:51 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-20T22:02:51.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:51 smithi060 ceph-mon[40290]: osdmap e64: 8 total, 8 up, 8 in 2024-09-20T22:02:51.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:51 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:51.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:51 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:51.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:51 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:51.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:51 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:02:52.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[29694]: Checking dashboard <-> RGW credentials 2024-09-20T22:02:52.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[29694]: pgmap v186: 36 pgs: 3 creating+peering, 21 unknown, 12 active+clean; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 182 B/s rd, 182 B/s wr, 0 op/s; 103 KiB/s, 0 objects/s recovering 2024-09-20T22:02:52.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:02:52.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:02:52.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[29694]: osdmap e65: 8 total, 8 up, 8 in 2024-09-20T22:02:52.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-20T22:02:52.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1676229616' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-20T22:02:52.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[29694]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-20T22:02:52.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[29694]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T22:02:52.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:02:52.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:52.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[40290]: Checking dashboard <-> RGW credentials 2024-09-20T22:02:52.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[40290]: pgmap v186: 36 pgs: 3 creating+peering, 21 unknown, 12 active+clean; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 182 B/s rd, 182 B/s wr, 0 op/s; 103 KiB/s, 0 objects/s recovering 2024-09-20T22:02:52.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:02:52.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:02:52.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[40290]: osdmap e65: 8 total, 8 up, 8 in 2024-09-20T22:02:52.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-20T22:02:52.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1676229616' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-20T22:02:52.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[40290]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-20T22:02:52.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[40290]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T22:02:52.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:02:52.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:52 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:52.584 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:52 smithi067 ceph-mon[33511]: Checking dashboard <-> RGW credentials 2024-09-20T22:02:52.584 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:52 smithi067 ceph-mon[33511]: pgmap v186: 36 pgs: 3 creating+peering, 21 unknown, 12 active+clean; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 182 B/s rd, 182 B/s wr, 0 op/s; 103 KiB/s, 0 objects/s recovering 2024-09-20T22:02:52.584 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:52 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:02:52.584 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:52 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:02:52.584 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:52 smithi067 ceph-mon[33511]: osdmap e65: 8 total, 8 up, 8 in 2024-09-20T22:02:52.584 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:52 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-20T22:02:52.584 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:52 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1676229616' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-20T22:02:52.585 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:52 smithi067 ceph-mon[33511]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-20T22:02:52.585 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:52 smithi067 ceph-mon[33511]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T22:02:52.585 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:52 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:02:52.585 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:52 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:52.862 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.67 --placement '1;smithi067=iscsi.a' 2024-09-20T22:02:53.122 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:02:53.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:53 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-20T22:02:53.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:53 smithi067 ceph-mon[33511]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-20T22:02:53.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:53 smithi067 ceph-mon[33511]: osdmap e66: 8 total, 8 up, 8 in 2024-09-20T22:02:53.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:53 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-20T22:02:53.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:53 smithi060 ceph-mon[40290]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-20T22:02:53.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:53 smithi060 ceph-mon[40290]: osdmap e66: 8 total, 8 up, 8 in 2024-09-20T22:02:53.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:53 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-20T22:02:53.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:53 smithi060 ceph-mon[29694]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-20T22:02:53.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:53 smithi060 ceph-mon[29694]: osdmap e66: 8 total, 8 up, 8 in 2024-09-20T22:02:53.848 INFO:teuthology.orchestra.run.smithi067.stdout:Scheduled iscsi.datapool update... 2024-09-20T22:02:54.406 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:54 smithi067 ceph-mon[33511]: pgmap v189: 68 pgs: 15 creating+peering, 20 unknown, 33 active+clean; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-20T22:02:54.406 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:54 smithi067 ceph-mon[33511]: osdmap e67: 8 total, 8 up, 8 in 2024-09-20T22:02:54.406 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:54 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1676229616' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-20T22:02:54.406 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:54 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-20T22:02:54.406 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:54 smithi067 ceph-mon[33511]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-20T22:02:54.406 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:54 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:54.434 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-20T22:02:54.435 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-20T22:02:54.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-20T22:02:54.465 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-20T22:02:54.465 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-20T22:02:54.492 DEBUG:teuthology.orchestra.run.smithi067:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@iscsi.iscsi.a.service 2024-09-20T22:02:54.535 INFO:tasks.cephadm:Adding prometheus.a on smithi067 2024-09-20T22:02:54.535 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch apply prometheus '1;smithi067=a' 2024-09-20T22:02:54.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:54 smithi060 ceph-mon[29694]: pgmap v189: 68 pgs: 15 creating+peering, 20 unknown, 33 active+clean; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-20T22:02:54.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:54 smithi060 ceph-mon[29694]: osdmap e67: 8 total, 8 up, 8 in 2024-09-20T22:02:54.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:54 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1676229616' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-20T22:02:54.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:54 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-20T22:02:54.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:54 smithi060 ceph-mon[29694]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-20T22:02:54.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:54 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:54 smithi060 ceph-mon[40290]: pgmap v189: 68 pgs: 15 creating+peering, 20 unknown, 33 active+clean; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-20T22:02:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:54 smithi060 ceph-mon[40290]: osdmap e67: 8 total, 8 up, 8 in 2024-09-20T22:02:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:54 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1676229616' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-20T22:02:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:54 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-20T22:02:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:54 smithi060 ceph-mon[40290]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-20T22:02:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:54 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:54.824 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:02:55.435 INFO:teuthology.orchestra.run.smithi067.stdout:Scheduled prometheus update... 2024-09-20T22:02:55.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:55 smithi067 ceph-mon[33511]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.67", "placement": "1;smithi067=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:02:55.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:55 smithi067 ceph-mon[33511]: Saving service iscsi.datapool spec with placement smithi067=iscsi.a;count:1 2024-09-20T22:02:55.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-20T22:02:55.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:55 smithi067 ceph-mon[33511]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-20T22:02:55.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:55 smithi067 ceph-mon[33511]: osdmap e68: 8 total, 8 up, 8 in 2024-09-20T22:02:55.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:55 smithi060 ceph-mon[40290]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.67", "placement": "1;smithi067=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:02:55.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:55 smithi060 ceph-mon[40290]: Saving service iscsi.datapool spec with placement smithi067=iscsi.a;count:1 2024-09-20T22:02:55.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:55 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-20T22:02:55.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:55 smithi060 ceph-mon[40290]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-20T22:02:55.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:55 smithi060 ceph-mon[40290]: osdmap e68: 8 total, 8 up, 8 in 2024-09-20T22:02:55.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:55 smithi060 ceph-mon[29694]: from='client.24385 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.67", "placement": "1;smithi067=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:02:55.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:55 smithi060 ceph-mon[29694]: Saving service iscsi.datapool spec with placement smithi067=iscsi.a;count:1 2024-09-20T22:02:55.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:55 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-20T22:02:55.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:55 smithi060 ceph-mon[29694]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-20T22:02:55.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:55 smithi060 ceph-mon[29694]: osdmap e68: 8 total, 8 up, 8 in 2024-09-20T22:02:56.024 DEBUG:teuthology.orchestra.run.smithi067:prometheus.a> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@prometheus.a.service 2024-09-20T22:02:56.026 INFO:tasks.cephadm:Adding node-exporter.a on smithi060 2024-09-20T22:02:56.027 INFO:tasks.cephadm:Adding node-exporter.b on smithi067 2024-09-20T22:02:56.027 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch apply node-exporter '2;smithi060=a;smithi067=b' 2024-09-20T22:02:56.325 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:02:56.569 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:56 smithi067 ceph-mon[33511]: pgmap v192: 100 pgs: 12 creating+peering, 43 unknown, 45 active+clean; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-20T22:02:56.569 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:56 smithi067 ceph-mon[33511]: osdmap e69: 8 total, 8 up, 8 in 2024-09-20T22:02:56.569 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:56 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-20T22:02:56.569 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:56 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1676229616' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-20T22:02:56.569 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:56 smithi067 ceph-mon[33511]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-20T22:02:56.570 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:56 smithi067 ceph-mon[33511]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi067=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:02:56.570 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:56 smithi067 ceph-mon[33511]: Saving service prometheus spec with placement smithi067=a;count:1 2024-09-20T22:02:56.570 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:56 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:56 smithi060 ceph-mon[29694]: pgmap v192: 100 pgs: 12 creating+peering, 43 unknown, 45 active+clean; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-20T22:02:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:56 smithi060 ceph-mon[29694]: osdmap e69: 8 total, 8 up, 8 in 2024-09-20T22:02:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:56 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-20T22:02:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:56 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1676229616' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-20T22:02:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:56 smithi060 ceph-mon[29694]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-20T22:02:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:56 smithi060 ceph-mon[29694]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi067=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:02:56.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:56 smithi060 ceph-mon[29694]: Saving service prometheus spec with placement smithi067=a;count:1 2024-09-20T22:02:56.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:56 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:56 smithi060 ceph-mon[40290]: pgmap v192: 100 pgs: 12 creating+peering, 43 unknown, 45 active+clean; 578 KiB data, 226 MiB used, 715 GiB / 715 GiB avail; 3.2 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-09-20T22:02:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:56 smithi060 ceph-mon[40290]: osdmap e69: 8 total, 8 up, 8 in 2024-09-20T22:02:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:56 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-20T22:02:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:56 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1676229616' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-20T22:02:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:56 smithi060 ceph-mon[40290]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-20T22:02:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:56 smithi060 ceph-mon[40290]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi067=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:02:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:56 smithi060 ceph-mon[40290]: Saving service prometheus spec with placement smithi067=a;count:1 2024-09-20T22:02:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:56 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:56.940 INFO:teuthology.orchestra.run.smithi067.stdout:Scheduled node-exporter update... 2024-09-20T22:02:57.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:57 smithi067 ceph-mon[33511]: 1.0 scrub starts 2024-09-20T22:02:57.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:57 smithi067 ceph-mon[33511]: 1.0 scrub ok 2024-09-20T22:02:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:57 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-20T22:02:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:57 smithi067 ceph-mon[33511]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-20T22:02:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:57 smithi067 ceph-mon[33511]: osdmap e70: 8 total, 8 up, 8 in 2024-09-20T22:02:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:57 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-20T22:02:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:57 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1676229616' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-20T22:02:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:57 smithi067 ceph-mon[33511]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-20T22:02:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:57 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:57 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-20T22:02:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:57 smithi067 ceph-mon[33511]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-20T22:02:57.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:57 smithi067 ceph-mon[33511]: osdmap e71: 8 total, 8 up, 8 in 2024-09-20T22:02:57.565 DEBUG:teuthology.orchestra.run.smithi060:node-exporter.a> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@node-exporter.a.service 2024-09-20T22:02:57.567 DEBUG:teuthology.orchestra.run.smithi067:node-exporter.b> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@node-exporter.b.service 2024-09-20T22:02:57.569 INFO:tasks.cephadm:Adding alertmanager.a on smithi060 2024-09-20T22:02:57.569 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch apply alertmanager '1;smithi060=a' 2024-09-20T22:02:57.591 INFO:journalctl@ceph.rgw.foo.a.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-rgw-foo-a[67948]: 2024-09-20T22:02:57.351+0000 7f0e6ebc8880 -1 LDAP not started since no server URIs were provided in the configuration. 2024-09-20T22:02:57.591 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[29694]: 1.0 scrub starts 2024-09-20T22:02:57.591 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[29694]: 1.0 scrub ok 2024-09-20T22:02:57.591 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-20T22:02:57.591 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[29694]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-20T22:02:57.591 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[29694]: osdmap e70: 8 total, 8 up, 8 in 2024-09-20T22:02:57.591 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-20T22:02:57.591 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1676229616' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-20T22:02:57.591 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[29694]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-20T22:02:57.591 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:57.592 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-20T22:02:57.592 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[29694]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-20T22:02:57.592 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[29694]: osdmap e71: 8 total, 8 up, 8 in 2024-09-20T22:02:57.592 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[40290]: 1.0 scrub starts 2024-09-20T22:02:57.592 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[40290]: 1.0 scrub ok 2024-09-20T22:02:57.592 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-20T22:02:57.592 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[40290]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-20T22:02:57.592 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[40290]: osdmap e70: 8 total, 8 up, 8 in 2024-09-20T22:02:57.592 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-20T22:02:57.592 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1676229616' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-20T22:02:57.592 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[40290]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-20T22:02:57.592 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:57.592 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1594620389' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-20T22:02:57.593 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[40290]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-20T22:02:57.593 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:57 smithi060 ceph-mon[40290]: osdmap e71: 8 total, 8 up, 8 in 2024-09-20T22:02:57.873 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:02:58.541 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:58 smithi067 ceph-mon[33511]: pgmap v195: 132 pgs: 9 creating+peering, 18 unknown, 105 active+clean; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-20T22:02:58.541 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:58 smithi067 ceph-mon[33511]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi060=a;smithi067=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:02:58.541 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:58 smithi067 ceph-mon[33511]: Saving service node-exporter spec with placement smithi060=a;smithi067=b;count:2 2024-09-20T22:02:58.541 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:58 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:58.541 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:58 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:58.541 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:58 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:58.541 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:58 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-20T22:02:58.541 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:58 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-20T22:02:58.541 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:58 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:58.581 INFO:teuthology.orchestra.run.smithi067.stdout:Scheduled alertmanager update... 2024-09-20T22:02:58.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:58 smithi060 ceph-mon[40290]: pgmap v195: 132 pgs: 9 creating+peering, 18 unknown, 105 active+clean; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-20T22:02:58.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:58 smithi060 ceph-mon[40290]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi060=a;smithi067=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:02:58.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:58 smithi060 ceph-mon[40290]: Saving service node-exporter spec with placement smithi060=a;smithi067=b;count:2 2024-09-20T22:02:58.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:58 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:58.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:58 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:58.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:58 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:58.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:58 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-20T22:02:58.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:58 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-20T22:02:58.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:58 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:58.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:58 smithi060 ceph-mon[29694]: pgmap v195: 132 pgs: 9 creating+peering, 18 unknown, 105 active+clean; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-20T22:02:58.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:58 smithi060 ceph-mon[29694]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi060=a;smithi067=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:02:58.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:58 smithi060 ceph-mon[29694]: Saving service node-exporter spec with placement smithi060=a;smithi067=b;count:2 2024-09-20T22:02:58.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:58 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:58.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:58 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:58.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:58 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:58.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:58 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-20T22:02:58.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:58 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-20T22:02:58.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:58 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:02:59.107 DEBUG:teuthology.orchestra.run.smithi060:alertmanager.a> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@alertmanager.a.service 2024-09-20T22:02:59.110 INFO:tasks.cephadm:Adding grafana.a on smithi067 2024-09-20T22:02:59.110 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph orch apply grafana '1;smithi067=a' 2024-09-20T22:02:59.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:59 smithi060 ceph-mon[29694]: Deploying daemon iscsi.iscsi.a on smithi067 2024-09-20T22:02:59.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:02:59 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:59.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:59 smithi060 ceph-mon[40290]: Deploying daemon iscsi.iscsi.a on smithi067 2024-09-20T22:02:59.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:02:59 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:59.907 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:59 smithi067 ceph-mon[33511]: Deploying daemon iscsi.iscsi.a on smithi067 2024-09-20T22:02:59.907 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:02:59 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:02:59.926 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:03:00.679 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:00 smithi067 ceph-mon[33511]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi060=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:03:00.679 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:00 smithi067 ceph-mon[33511]: Saving service alertmanager spec with placement smithi060=a;count:1 2024-09-20T22:03:00.679 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:00 smithi067 ceph-mon[33511]: pgmap v197: 132 pgs: 4 creating+peering, 10 unknown, 118 active+clean; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-20T22:03:00.680 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:00 smithi067 systemd[1]: Starting Ceph iscsi.iscsi.a for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:03:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:00 smithi060 ceph-mon[29694]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi060=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:03:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:00 smithi060 ceph-mon[29694]: Saving service alertmanager spec with placement smithi060=a;count:1 2024-09-20T22:03:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:00 smithi060 ceph-mon[29694]: pgmap v197: 132 pgs: 4 creating+peering, 10 unknown, 118 active+clean; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-20T22:03:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:00 smithi060 ceph-mon[40290]: from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi060=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:03:00.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:00 smithi060 ceph-mon[40290]: Saving service alertmanager spec with placement smithi060=a;count:1 2024-09-20T22:03:00.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:00 smithi060 ceph-mon[40290]: pgmap v197: 132 pgs: 4 creating+peering, 10 unknown, 118 active+clean; 578 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-09-20T22:03:00.997 INFO:teuthology.orchestra.run.smithi067.stdout:Scheduled grafana update... 2024-09-20T22:03:01.264 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:00 smithi067 podman[62342]: 2024-09-20 22:03:00.892938871 +0000 UTC m=+0.023670430 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5 2024-09-20T22:03:01.264 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 podman[62342]: 2024-09-20 22:03:01.02079976 +0000 UTC m=+0.151531308 container create 0a4eea18e1996852a18043cf3e4e52dd3a507abd1b940adbf85fbf5af5f768cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-9e2ac70, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T22:03:01.655 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 podman[62342]: 2024-09-20 22:03:01.34924158 +0000 UTC m=+0.479973129 container init 0a4eea18e1996852a18043cf3e4e52dd3a507abd1b940adbf85fbf5af5f768cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=squid-9e2ac70) 2024-09-20T22:03:01.655 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 podman[62342]: 2024-09-20 22:03:01.354271484 +0000 UTC m=+0.485003022 container start 0a4eea18e1996852a18043cf3e4e52dd3a507abd1b940adbf85fbf5af5f768cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=squid-9e2ac70, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-20T22:03:01.655 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 bash[62342]: 0a4eea18e1996852a18043cf3e4e52dd3a507abd1b940adbf85fbf5af5f768cc 2024-09-20T22:03:01.655 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 systemd[1]: Started Ceph iscsi.iscsi.a for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T22:03:01.858 DEBUG:teuthology.orchestra.run.smithi067:grafana.a> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@grafana.a.service 2024-09-20T22:03:01.861 INFO:tasks.cephadm:Setting up client nodes... 2024-09-20T22:03:01.861 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-20T22:03:01.953 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug Started the configuration object watcher 2024-09-20T22:03:01.953 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug Checking for config object changes every 1s 2024-09-20T22:03:01.953 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug Processing osd blocklist entries for this node 2024-09-20T22:03:01.953 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug Reading the configuration object to update local LIO configuration 2024-09-20T22:03:01.953 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug Configuration does not have an entry for this host(smithi067.front.sepia.ceph.com) - nothing to define to LIO 2024-09-20T22:03:01.953 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-20T22:03:02.116 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:02.237 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[29694]: pgmap v198: 132 pgs: 132 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 3.7 KiB/s wr, 102 op/s 2024-09-20T22:03:02.238 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[29694]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi067=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:03:02.238 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[29694]: Saving service grafana spec with placement smithi067=a;count:1 2024-09-20T22:03:02.238 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:02.238 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[29694]: osdmap e72: 8 total, 8 up, 8 in 2024-09-20T22:03:02.238 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:02.238 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:02.238 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:02.238 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[29694]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-20T22:03:02.238 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:02.238 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:02.238 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[29694]: from='client.? 172.21.15.67:0/1911725498' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-20T22:03:02.238 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[40290]: pgmap v198: 132 pgs: 132 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 3.7 KiB/s wr, 102 op/s 2024-09-20T22:03:02.239 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[40290]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi067=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:03:02.239 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[40290]: Saving service grafana spec with placement smithi067=a;count:1 2024-09-20T22:03:02.239 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:02.239 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[40290]: osdmap e72: 8 total, 8 up, 8 in 2024-09-20T22:03:02.239 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:02.239 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:02.239 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:02.239 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[40290]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-20T22:03:02.239 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:02.239 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:02.239 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:01 smithi060 ceph-mon[40290]: from='client.? 172.21.15.67:0/1911725498' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-20T22:03:02.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-mon[33511]: pgmap v198: 132 pgs: 132 active+clean; 580 KiB data, 230 MiB used, 715 GiB / 715 GiB avail; 42 KiB/s rd, 3.7 KiB/s wr, 102 op/s 2024-09-20T22:03:02.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-mon[33511]: from='client.24424 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi067=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:03:02.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-mon[33511]: Saving service grafana spec with placement smithi067=a;count:1 2024-09-20T22:03:02.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:02.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-mon[33511]: osdmap e72: 8 total, 8 up, 8 in 2024-09-20T22:03:02.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:02.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:02.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:02.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-mon[33511]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-20T22:03:02.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:02.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:02.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-mon[33511]: from='client.? 172.21.15.67:0/1911725498' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-20T22:03:02.265 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: * Environment: production 2024-09-20T22:03:02.265 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-20T22:03:02.265 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: Use a production WSGI server instead. 2024-09-20T22:03:02.265 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: * Debug mode: off 2024-09-20T22:03:02.265 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug * Running on all addresses. 2024-09-20T22:03:02.265 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-20T22:03:02.265 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: * Running on all addresses. 2024-09-20T22:03:02.265 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-20T22:03:02.265 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug * Running on http://[fe80::ec4:7aff:feb6:fc0b]:5000/ (Press CTRL+C to quit) 2024-09-20T22:03:02.265 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: * Running on http://[fe80::ec4:7aff:feb6:fc0b]:5000/ (Press CTRL+C to quit) 2024-09-20T22:03:02.845 INFO:teuthology.orchestra.run.smithi060.stdout:[client.0] 2024-09-20T22:03:02.846 INFO:teuthology.orchestra.run.smithi060.stdout: key = AQAW8e1mSOw5MhAA585MDguw2OAmD9ff4uEsdg== 2024-09-20T22:03:03.133 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:03 smithi060 ceph-mon[40290]: Deploying daemon prometheus.a on smithi067 2024-09-20T22:03:03.133 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:03 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1458065208' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-20T22:03:03.133 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:03 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1458065208' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-20T22:03:03.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:03 smithi060 ceph-mon[29694]: Deploying daemon prometheus.a on smithi067 2024-09-20T22:03:03.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:03 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1458065208' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-20T22:03:03.134 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:03 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1458065208' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-20T22:03:03.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:03 smithi067 ceph-mon[33511]: Deploying daemon prometheus.a on smithi067 2024-09-20T22:03:03.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:03 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1458065208' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-20T22:03:03.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:03 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1458065208' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-20T22:03:03.471 DEBUG:teuthology.orchestra.run.smithi060:> set -ex 2024-09-20T22:03:03.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-20T22:03:03.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-20T22:03:03.511 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-20T22:03:03.778 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:03:04.134 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:04 smithi067 ceph-mon[33511]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 5.3 KiB/s wr, 174 op/s 2024-09-20T22:03:04.134 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:04 smithi067 ceph-mon[33511]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-20T22:03:04.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:04 smithi060 ceph-mon[29694]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 5.3 KiB/s wr, 174 op/s 2024-09-20T22:03:04.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:04 smithi060 ceph-mon[29694]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-20T22:03:04.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:04 smithi060 ceph-mon[40290]: pgmap v200: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s rd, 5.3 KiB/s wr, 174 op/s 2024-09-20T22:03:04.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:04 smithi060 ceph-mon[40290]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-20T22:03:04.757 INFO:teuthology.orchestra.run.smithi067.stdout:[client.1] 2024-09-20T22:03:04.757 INFO:teuthology.orchestra.run.smithi067.stdout: key = AQAY8e1mybHOLBAAp0fwnI/aXCgCMoRFHJKCeQ== 2024-09-20T22:03:05.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:05 smithi067 ceph-mon[33511]: from='client.? 172.21.15.67:0/3787539860' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-20T22:03:05.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:05 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-20T22:03:05.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:05 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-20T22:03:05.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:05 smithi060 ceph-mon[29694]: from='client.? 172.21.15.67:0/3787539860' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-20T22:03:05.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:05 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-20T22:03:05.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:05 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-20T22:03:05.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:05 smithi060 ceph-mon[40290]: from='client.? 172.21.15.67:0/3787539860' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-20T22:03:05.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:05 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-20T22:03:05.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:05 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-20T22:03:05.762 DEBUG:teuthology.orchestra.run.smithi067:> set -ex 2024-09-20T22:03:05.762 DEBUG:teuthology.orchestra.run.smithi067:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-20T22:03:05.762 DEBUG:teuthology.orchestra.run.smithi067:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-20T22:03:05.803 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-20T22:03:05.804 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-20T22:03:05.804 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-20T22:03:06.072 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:06.194 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:06 smithi060 ceph-mon[29694]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 139 op/s 2024-09-20T22:03:06.195 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:06 smithi060 ceph-mon[40290]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 139 op/s 2024-09-20T22:03:06.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:06 smithi067 ceph-mon[33511]: pgmap v201: 132 pgs: 132 active+clean; 582 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 57 KiB/s rd, 4.2 KiB/s wr, 139 op/s 2024-09-20T22:03:06.779 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:03:07.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:07 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/248205974' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-20T22:03:07.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:07 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/248205974' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-20T22:03:07.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:07 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/248205974' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-20T22:03:07.676 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":16,"flags":0,"active_gid":14150,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6800","nonce":604714621},{"type":"v1","addr":"172.21.15.60:6801","nonce":604714621}]},"active_addr":"172.21.15.60:6801/604714621","active_change":"2024-09-20T21:58:26.613564+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24115,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.60:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":890154710}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":1854455098}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":3386526350}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":3311255314}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.60:0","nonce":2221262203}]}]} 2024-09-20T22:03:07.680 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-20T22:03:07.680 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-20T22:03:07.680 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-20T22:03:07.940 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:08.320 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:08 smithi060 ceph-mon[29694]: pgmap v202: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s rd, 3.7 KiB/s wr, 120 op/s 2024-09-20T22:03:08.320 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:08 smithi060 ceph-mon[40290]: pgmap v202: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s rd, 3.7 KiB/s wr, 120 op/s 2024-09-20T22:03:08.593 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:03:08.593 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":72,"fsid":"3df9647e-779b-11ef-baf5-efdc52797490","created":"2024-09-20T21:57:31.244934+0000","modified":"2024-09-20T22:03:01.257709+0000","last_up_change":"2024-09-20T22:02:41.053740+0000","last_in_change":"2024-09-20T22:02:21.189225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-20T22:00:35.131125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-20T22:02:47.974480+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-20T22:02:48.902465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-20T22:02:50.620057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-20T22:02:52.722912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-20T22:02:54.794248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0997c01a-a9bd-421e-abcc-fbcf7abe8512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6803","nonce":1395849550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6805","nonce":1395849550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6809","nonce":1395849550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6807","nonce":1395849550}]},"public_addr":"172.21.15.60:6803/1395849550","cluster_addr":"172.21.15.60:6805/1395849550","heartbeat_back_addr":"172.21.15.60:6809/1395849550","heartbeat_front_addr":"172.21.15.60:6807/1395849550","state":["exists","up"]},{"osd":1,"uuid":"c04bdcb1-d76b-47c8-a38b-c02043df6ba8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":145283513},{"type":"v1","addr":"172.21.15.60:6811","nonce":145283513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":145283513},{"type":"v1","addr":"172.21.15.60:6813","nonce":145283513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":145283513},{"type":"v1","addr":"172.21.15.60:6817","nonce":145283513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":145283513},{"type":"v1","addr":"172.21.15.60:6815","nonce":145283513}]},"public_addr":"172.21.15.60:6811/145283513","cluster_addr":"172.21.15.60:6813/145283513","heartbeat_back_addr":"172.21.15.60:6817/145283513","heartbeat_front_addr":"172.21.15.60:6815/145283513","state":["exists","up"]},{"osd":2,"uuid":"32c09d19-eb98-4926-a44e-e7ddee152956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6819","nonce":3698562274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6821","nonce":3698562274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6825","nonce":3698562274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6823","nonce":3698562274}]},"public_addr":"172.21.15.60:6819/3698562274","cluster_addr":"172.21.15.60:6821/3698562274","heartbeat_back_addr":"172.21.15.60:6825/3698562274","heartbeat_front_addr":"172.21.15.60:6823/3698562274","state":["exists","up"]},{"osd":3,"uuid":"b1c2cb69-8565-49ac-b854-7ea7615c6660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6827","nonce":2592322449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6829","nonce":2592322449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6833","nonce":2592322449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6831","nonce":2592322449}]},"public_addr":"172.21.15.60:6827/2592322449","cluster_addr":"172.21.15.60:6829/2592322449","heartbeat_back_addr":"172.21.15.60:6833/2592322449","heartbeat_front_addr":"172.21.15.60:6831/2592322449","state":["exists","up"]},{"osd":4,"uuid":"04a547d6-fc48-43b7-a8d7-18df8a83aa96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":3243564239},{"type":"v1","addr":"172.21.15.67:6801","nonce":3243564239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3243564239},{"type":"v1","addr":"172.21.15.67:6803","nonce":3243564239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3243564239},{"type":"v1","addr":"172.21.15.67:6807","nonce":3243564239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3243564239},{"type":"v1","addr":"172.21.15.67:6805","nonce":3243564239}]},"public_addr":"172.21.15.67:6801/3243564239","cluster_addr":"172.21.15.67:6803/3243564239","heartbeat_back_addr":"172.21.15.67:6807/3243564239","heartbeat_front_addr":"172.21.15.67:6805/3243564239","state":["exists","up"]},{"osd":5,"uuid":"4763aa10-c367-485b-b525-e5241e642b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6809","nonce":1622258887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6811","nonce":1622258887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6815","nonce":1622258887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6813","nonce":1622258887}]},"public_addr":"172.21.15.67:6809/1622258887","cluster_addr":"172.21.15.67:6811/1622258887","heartbeat_back_addr":"172.21.15.67:6815/1622258887","heartbeat_front_addr":"172.21.15.67:6813/1622258887","state":["exists","up"]},{"osd":6,"uuid":"a359116e-c013-4e2f-8b79-86219516ee16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6817","nonce":1187114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6819","nonce":1187114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6823","nonce":1187114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6821","nonce":1187114}]},"public_addr":"172.21.15.67:6817/1187114","cluster_addr":"172.21.15.67:6819/1187114","heartbeat_back_addr":"172.21.15.67:6823/1187114","heartbeat_front_addr":"172.21.15.67:6821/1187114","state":["exists","up"]},{"osd":7,"uuid":"a4bf4188-f133-4a31-8897-d7e8182248d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6825","nonce":3021842840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6827","nonce":3021842840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6831","nonce":3021842840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6829","nonce":3021842840}]},"public_addr":"172.21.15.67:6825/3021842840","cluster_addr":"172.21.15.67:6827/3021842840","heartbeat_back_addr":"172.21.15.67:6831/3021842840","heartbeat_front_addr":"172.21.15.67:6829/3021842840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:19.297917+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:40.779347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:02.406496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:26.764786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:51.149939+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:01:48.467136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:14.366664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:39.867765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/1871351181":"2024-09-21T21:58:26.613414+0000","172.21.15.60:0/3927294026":"2024-09-21T21:58:26.613414+0000","172.21.15.60:6801/3734574621":"2024-09-21T21:58:26.613414+0000","172.21.15.60:6800/3734574621":"2024-09-21T21:58:26.613414+0000","172.21.15.60:0/1729862661":"2024-09-21T21:58:26.613414+0000","172.21.15.60:6801/3151907776":"2024-09-21T21:58:02.105557+0000","172.21.15.60:0/3965541528":"2024-09-21T21:58:02.105557+0000","172.21.15.60:0/4150585713":"2024-09-21T21:58:02.105557+0000","172.21.15.60:6800/3151907776":"2024-09-21T21:58:02.105557+0000","172.21.15.60:0/956264518":"2024-09-21T21:58:02.105557+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-20T22:03:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:08 smithi067 ceph-mon[33511]: pgmap v202: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 49 KiB/s rd, 3.7 KiB/s wr, 120 op/s 2024-09-20T22:03:09.283 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-20T22:03:09.284 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-20T22:03:09.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:09 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/144617527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-20T22:03:09.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:09 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/144617527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-20T22:03:09.542 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:09.587 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:09 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/144617527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-20T22:03:10.183 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:03:10.183 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":72,"fsid":"3df9647e-779b-11ef-baf5-efdc52797490","created":"2024-09-20T21:57:31.244934+0000","modified":"2024-09-20T22:03:01.257709+0000","last_up_change":"2024-09-20T22:02:41.053740+0000","last_in_change":"2024-09-20T22:02:21.189225+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-20T22:00:35.131125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-20T22:02:47.974480+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-20T22:02:48.902465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-20T22:02:50.620057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-20T22:02:52.722912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-20T22:02:54.794248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0997c01a-a9bd-421e-abcc-fbcf7abe8512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6803","nonce":1395849550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6805","nonce":1395849550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6809","nonce":1395849550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6807","nonce":1395849550}]},"public_addr":"172.21.15.60:6803/1395849550","cluster_addr":"172.21.15.60:6805/1395849550","heartbeat_back_addr":"172.21.15.60:6809/1395849550","heartbeat_front_addr":"172.21.15.60:6807/1395849550","state":["exists","up"]},{"osd":1,"uuid":"c04bdcb1-d76b-47c8-a38b-c02043df6ba8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":145283513},{"type":"v1","addr":"172.21.15.60:6811","nonce":145283513}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":145283513},{"type":"v1","addr":"172.21.15.60:6813","nonce":145283513}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":145283513},{"type":"v1","addr":"172.21.15.60:6817","nonce":145283513}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":145283513},{"type":"v1","addr":"172.21.15.60:6815","nonce":145283513}]},"public_addr":"172.21.15.60:6811/145283513","cluster_addr":"172.21.15.60:6813/145283513","heartbeat_back_addr":"172.21.15.60:6817/145283513","heartbeat_front_addr":"172.21.15.60:6815/145283513","state":["exists","up"]},{"osd":2,"uuid":"32c09d19-eb98-4926-a44e-e7ddee152956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6819","nonce":3698562274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6821","nonce":3698562274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6825","nonce":3698562274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6823","nonce":3698562274}]},"public_addr":"172.21.15.60:6819/3698562274","cluster_addr":"172.21.15.60:6821/3698562274","heartbeat_back_addr":"172.21.15.60:6825/3698562274","heartbeat_front_addr":"172.21.15.60:6823/3698562274","state":["exists","up"]},{"osd":3,"uuid":"b1c2cb69-8565-49ac-b854-7ea7615c6660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6827","nonce":2592322449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6829","nonce":2592322449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6833","nonce":2592322449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6831","nonce":2592322449}]},"public_addr":"172.21.15.60:6827/2592322449","cluster_addr":"172.21.15.60:6829/2592322449","heartbeat_back_addr":"172.21.15.60:6833/2592322449","heartbeat_front_addr":"172.21.15.60:6831/2592322449","state":["exists","up"]},{"osd":4,"uuid":"04a547d6-fc48-43b7-a8d7-18df8a83aa96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":3243564239},{"type":"v1","addr":"172.21.15.67:6801","nonce":3243564239}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3243564239},{"type":"v1","addr":"172.21.15.67:6803","nonce":3243564239}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3243564239},{"type":"v1","addr":"172.21.15.67:6807","nonce":3243564239}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3243564239},{"type":"v1","addr":"172.21.15.67:6805","nonce":3243564239}]},"public_addr":"172.21.15.67:6801/3243564239","cluster_addr":"172.21.15.67:6803/3243564239","heartbeat_back_addr":"172.21.15.67:6807/3243564239","heartbeat_front_addr":"172.21.15.67:6805/3243564239","state":["exists","up"]},{"osd":5,"uuid":"4763aa10-c367-485b-b525-e5241e642b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6809","nonce":1622258887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6811","nonce":1622258887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6815","nonce":1622258887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6813","nonce":1622258887}]},"public_addr":"172.21.15.67:6809/1622258887","cluster_addr":"172.21.15.67:6811/1622258887","heartbeat_back_addr":"172.21.15.67:6815/1622258887","heartbeat_front_addr":"172.21.15.67:6813/1622258887","state":["exists","up"]},{"osd":6,"uuid":"a359116e-c013-4e2f-8b79-86219516ee16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6817","nonce":1187114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6819","nonce":1187114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6823","nonce":1187114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6821","nonce":1187114}]},"public_addr":"172.21.15.67:6817/1187114","cluster_addr":"172.21.15.67:6819/1187114","heartbeat_back_addr":"172.21.15.67:6823/1187114","heartbeat_front_addr":"172.21.15.67:6821/1187114","state":["exists","up"]},{"osd":7,"uuid":"a4bf4188-f133-4a31-8897-d7e8182248d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6825","nonce":3021842840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6827","nonce":3021842840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6831","nonce":3021842840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6829","nonce":3021842840}]},"public_addr":"172.21.15.67:6825/3021842840","cluster_addr":"172.21.15.67:6827/3021842840","heartbeat_back_addr":"172.21.15.67:6831/3021842840","heartbeat_front_addr":"172.21.15.67:6829/3021842840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:19.297917+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:40.779347+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:02.406496+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:26.764786+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:51.149939+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:01:48.467136+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:14.366664+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:02:39.867765+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/1871351181":"2024-09-21T21:58:26.613414+0000","172.21.15.60:0/3927294026":"2024-09-21T21:58:26.613414+0000","172.21.15.60:6801/3734574621":"2024-09-21T21:58:26.613414+0000","172.21.15.60:6800/3734574621":"2024-09-21T21:58:26.613414+0000","172.21.15.60:0/1729862661":"2024-09-21T21:58:26.613414+0000","172.21.15.60:6801/3151907776":"2024-09-21T21:58:02.105557+0000","172.21.15.60:0/3965541528":"2024-09-21T21:58:02.105557+0000","172.21.15.60:0/4150585713":"2024-09-21T21:58:02.105557+0000","172.21.15.60:6800/3151907776":"2024-09-21T21:58:02.105557+0000","172.21.15.60:0/956264518":"2024-09-21T21:58:02.105557+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-20T22:03:10.441 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:10 smithi067 ceph-mon[33511]: pgmap v203: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-09-20T22:03:10.442 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:10 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1578632225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-20T22:03:10.505 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:10 smithi060 ceph-mon[29694]: pgmap v203: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-09-20T22:03:10.505 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:10 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1578632225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-20T22:03:10.506 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:10 smithi060 ceph-mon[40290]: pgmap v203: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 46 KiB/s rd, 3.5 KiB/s wr, 112 op/s 2024-09-20T22:03:10.506 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:10 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1578632225' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-20T22:03:10.813 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-20T22:03:10.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-20T22:03:10.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-20T22:03:10.815 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-20T22:03:10.815 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-20T22:03:10.815 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-20T22:03:10.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-20T22:03:10.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-20T22:03:11.300 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:11.327 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:11.350 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:11.365 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:11.366 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:12.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:11 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:03:12.015 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:12.015 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:12.429 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:12 smithi067 ceph-mon[33511]: pgmap v204: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 1.6 KiB/s wr, 52 op/s 2024-09-20T22:03:12.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:12 smithi060 ceph-mon[29694]: pgmap v204: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 1.6 KiB/s wr, 52 op/s 2024-09-20T22:03:12.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:12 smithi060 ceph-mon[40290]: pgmap v204: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 21 KiB/s rd, 1.6 KiB/s wr, 52 op/s 2024-09-20T22:03:12.763 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:12 smithi067 systemd[1]: Starting Ceph prometheus.a for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:03:13.112 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:12 smithi067 podman[63312]: 2024-09-20 22:03:12.736580299 +0000 UTC m=+0.017648211 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-20T22:03:13.112 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:12 smithi067 podman[63312]: 2024-09-20 22:03:12.85923654 +0000 UTC m=+0.140304442 container create 0c9e6954fc644852e8306bef6fd7c4c2b4e19b9466730e6e79db187b3133cc07 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T22:03:13.112 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 podman[63312]: 2024-09-20 22:03:13.11133672 +0000 UTC m=+0.392404627 container init 0c9e6954fc644852e8306bef6fd7c4c2b4e19b9466730e6e79db187b3133cc07 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T22:03:13.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:03:13.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:13.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:13.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:13.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-20T22:03:13.514 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 podman[63312]: 2024-09-20 22:03:13.116930094 +0000 UTC m=+0.397997996 container start 0c9e6954fc644852e8306bef6fd7c4c2b4e19b9466730e6e79db187b3133cc07 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T22:03:13.514 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.141Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-20T22:03:13.515 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.141Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-20T22:03:13.515 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.141Z caller=main.go:570 level=info host_details="(Linux 5.14.0-508.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024 x86_64 smithi067 (none))" 2024-09-20T22:03:13.515 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.141Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-20T22:03:13.515 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.141Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-20T22:03:13.515 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.143Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-20T22:03:13.515 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.143Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-20T22:03:13.515 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.144Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-20T22:03:13.515 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.144Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-20T22:03:13.515 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.146Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-20T22:03:13.515 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.146Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.753µs 2024-09-20T22:03:13.515 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.146Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-20T22:03:13.516 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.146Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-20T22:03:13.516 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.146Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=17.915µs wal_replay_duration=172.75µs wbl_replay_duration=91ns total_replay_duration=208.799µs 2024-09-20T22:03:13.516 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.146Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-20T22:03:13.516 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.147Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-20T22:03:13.516 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.147Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-20T22:03:13.516 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.169Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=22.048204ms db_storage=723ns remote_storage=1.079µs web_handler=242ns query_engine=501ns scrape=391.846µs scrape_sd=73.638µs notify=858ns notify_sd=2.213µs rules=21.314491ms tracing=10.016µs 2024-09-20T22:03:13.516 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.169Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-20T22:03:13.516 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:03:13.169Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-20T22:03:13.516 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 bash[63312]: 0c9e6954fc644852e8306bef6fd7c4c2b4e19b9466730e6e79db187b3133cc07 2024-09-20T22:03:13.516 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:03:13 smithi067 systemd[1]: Started Ceph prometheus.a for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T22:03:13.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:13 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:03:13.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:13 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:13.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:13 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:13.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:13 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:13.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:13 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-20T22:03:13.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:13 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:03:13.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:13 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:13.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:13 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:13.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:13 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' 2024-09-20T22:03:13.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:13 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-20T22:03:13.662 INFO:teuthology.orchestra.run.smithi060.stdout:206158430221 2024-09-20T22:03:13.662 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-20T22:03:14.020 INFO:teuthology.orchestra.run.smithi060.stdout:77309411363 2024-09-20T22:03:14.021 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-20T22:03:14.542 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:14 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ignoring --setuser ceph since I am not root 2024-09-20T22:03:14.542 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:14 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ignoring --setgroup ceph since I am not root 2024-09-20T22:03:14.542 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:14 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:14.413+0000 7fa5b978c040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-20T22:03:14.542 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:14 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:14.473+0000 7fa5b978c040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-20T22:03:14.542 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:14 smithi060 ceph-mon[29694]: pgmap v205: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1.2 KiB/s wr, 36 op/s 2024-09-20T22:03:14.543 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:14 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:03:14.543 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:14 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:03:14.543 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:14 smithi060 ceph-mon[29694]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-20T22:03:14.543 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:14 smithi060 ceph-mon[29694]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-20T22:03:14.543 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:14 smithi060 ceph-mon[40290]: pgmap v205: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1.2 KiB/s wr, 36 op/s 2024-09-20T22:03:14.543 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:14 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:03:14.543 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:14 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:03:14.543 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:14 smithi060 ceph-mon[40290]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-20T22:03:14.543 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:14 smithi060 ceph-mon[40290]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-20T22:03:14.625 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:14 smithi067 ceph-mon[33511]: pgmap v205: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 15 KiB/s rd, 1.2 KiB/s wr, 36 op/s 2024-09-20T22:03:14.625 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:14 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:03:14.625 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:14 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:03:14.626 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:14 smithi067 ceph-mon[33511]: from='mgr.14150 172.21.15.60:0/3048105199' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-20T22:03:14.626 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:14 smithi067 ceph-mon[33511]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-20T22:03:14.626 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:14 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: ignoring --setuser ceph since I am not root 2024-09-20T22:03:14.626 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:14 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: ignoring --setgroup ceph since I am not root 2024-09-20T22:03:14.626 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:14 smithi067 ceph-mgr[35512]: -- 172.21.15.67:0/3886111073 <== mon.2 v2:172.21.15.60:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f7260af0e0 con 0x55f72608d400 2024-09-20T22:03:14.626 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:14 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:14.410+0000 7f5b79a85040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-20T22:03:14.626 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:14 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:14.469+0000 7f5b79a85040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-20T22:03:14.626 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:14 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:14.623+0000 7f5b79a85040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-20T22:03:14.843 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:14 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:14.633+0000 7fa5b978c040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-20T22:03:14.843 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:14 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:14.704+0000 7fa5b978c040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-20T22:03:14.998 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:14 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:14.693+0000 7f5b79a85040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-20T22:03:14.998 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:14 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:14.847+0000 7f5b79a85040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-20T22:03:15.177 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:14 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:14.865+0000 7fa5b978c040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-20T22:03:15.178 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:15.019+0000 7fa5b978c040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-20T22:03:15.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:15 smithi060 ceph-mon[29694]: osdmap e73: 8 total, 8 up, 8 in 2024-09-20T22:03:15.536 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:15.352+0000 7fa5b978c040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-20T22:03:15.536 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:15.416+0000 7fa5b978c040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-20T22:03:15.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:15 smithi060 ceph-mon[40290]: osdmap e73: 8 total, 8 up, 8 in 2024-09-20T22:03:15.584 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:15 smithi067 ceph-mon[33511]: osdmap e73: 8 total, 8 up, 8 in 2024-09-20T22:03:15.585 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:14 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:14.995+0000 7f5b79a85040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-20T22:03:15.585 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:15 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:15.325+0000 7f5b79a85040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-20T22:03:15.585 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:15 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:15.389+0000 7f5b79a85040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-20T22:03:15.585 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:15 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:15.582+0000 7f5b79a85040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-20T22:03:15.847 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:15.612+0000 7fa5b978c040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-20T22:03:16.643 INFO:teuthology.orchestra.run.smithi060.stdout:133143986201 2024-09-20T22:03:16.644 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-20T22:03:16.763 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:16 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:16.505+0000 7f5b79a85040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-20T22:03:16.763 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:16 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:16.575+0000 7f5b79a85040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-20T22:03:16.957 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:16 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:16.570+0000 7fa5b978c040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-20T22:03:16.957 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:16 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:16.645+0000 7fa5b978c040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-20T22:03:17.376 INFO:teuthology.orchestra.run.smithi060.stdout:34359738414 2024-09-20T22:03:17.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-20T22:03:17.611 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:17 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:17.339+0000 7f5b79a85040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-20T22:03:17.611 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:17 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:17.460+0000 7f5b79a85040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-20T22:03:17.698 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:17 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:17.411+0000 7fa5b978c040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-20T22:03:17.698 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:17 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:17.537+0000 7fa5b978c040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-20T22:03:17.815 INFO:teuthology.orchestra.run.smithi060.stdout:55834574888 2024-09-20T22:03:17.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-20T22:03:17.902 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:17 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:17.609+0000 7f5b79a85040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-20T22:03:17.902 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:17 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:17.672+0000 7f5b79a85040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-20T22:03:17.903 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:17 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:17.829+0000 7f5b79a85040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-20T22:03:18.006 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:17 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:17.696+0000 7fa5b978c040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-20T22:03:18.006 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:17 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:17.765+0000 7fa5b978c040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-20T22:03:18.006 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:17 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:17.929+0000 7fa5b978c040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-20T22:03:18.006 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:18 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:18.004+0000 7fa5b978c040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-20T22:03:18.198 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:17 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:17.900+0000 7f5b79a85040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-20T22:03:18.198 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:17 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:17.962+0000 7f5b79a85040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-20T22:03:18.198 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:18 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:18.128+0000 7f5b79a85040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-20T22:03:18.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:18 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:18.067+0000 7fa5b978c040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-20T22:03:18.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:18 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:18.235+0000 7fa5b978c040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-20T22:03:18.457 INFO:teuthology.orchestra.run.smithi060.stdout:111669149726 2024-09-20T22:03:18.457 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-20T22:03:18.479 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:18 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:18.196+0000 7f5b79a85040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-20T22:03:18.479 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:18 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:18.265+0000 7f5b79a85040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-20T22:03:18.479 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:18 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:18.353+0000 7f5b79a85040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-20T22:03:18.479 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:18 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:18.415+0000 7f5b79a85040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-20T22:03:18.479 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:18 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:18.476+0000 7f5b79a85040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-20T22:03:18.583 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:18 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:18.306+0000 7fa5b978c040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-20T22:03:18.584 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:18 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:18.376+0000 7fa5b978c040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-20T22:03:18.584 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:18 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:18.470+0000 7fa5b978c040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-20T22:03:18.584 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:18 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:18.536+0000 7fa5b978c040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-20T22:03:18.868 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:18 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:18.600+0000 7fa5b978c040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-20T22:03:18.883 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:18.934 INFO:teuthology.orchestra.run.smithi060.stdout:244813135881 2024-09-20T22:03:18.934 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-20T22:03:18.974 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:19.160 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:19.160 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:19.424 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-mon[29694]: Standby manager daemon x restarted 2024-09-20T22:03:19.424 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-mon[29694]: Standby manager daemon x started 2024-09-20T22:03:19.425 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-mon[29694]: from='mgr.? 172.21.15.67:0/237541836' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-20T22:03:19.425 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-mon[29694]: from='mgr.? 172.21.15.67:0/237541836' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-20T22:03:19.425 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-mon[29694]: from='mgr.? 172.21.15.67:0/237541836' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-20T22:03:19.425 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-mon[29694]: from='mgr.? 172.21.15.67:0/237541836' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-20T22:03:19.425 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-mon[40290]: Standby manager daemon x restarted 2024-09-20T22:03:19.425 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-mon[40290]: Standby manager daemon x started 2024-09-20T22:03:19.425 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-mon[40290]: from='mgr.? 172.21.15.67:0/237541836' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-20T22:03:19.425 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-mon[40290]: from='mgr.? 172.21.15.67:0/237541836' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-20T22:03:19.425 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-mon[40290]: from='mgr.? 172.21.15.67:0/237541836' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-20T22:03:19.425 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-mon[40290]: from='mgr.? 172.21.15.67:0/237541836' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-20T22:03:19.714 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:19.515+0000 7fa5b978c040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-20T22:03:19.714 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:03:19] ENGINE Bus STARTING 2024-09-20T22:03:19.714 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: CherryPy Checker: 2024-09-20T22:03:19.714 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: The Application mounted at '' has an empty config. 2024-09-20T22:03:19.714 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:19.715 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:03:19] ENGINE Serving on http://:::9283 2024-09-20T22:03:19.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:19 smithi067 ceph-mon[33511]: Standby manager daemon x restarted 2024-09-20T22:03:19.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:19 smithi067 ceph-mon[33511]: Standby manager daemon x started 2024-09-20T22:03:19.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:19 smithi067 ceph-mon[33511]: from='mgr.? 172.21.15.67:0/237541836' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-20T22:03:19.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:19 smithi067 ceph-mon[33511]: from='mgr.? 172.21.15.67:0/237541836' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-20T22:03:19.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:19 smithi067 ceph-mon[33511]: from='mgr.? 172.21.15.67:0/237541836' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-20T22:03:19.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:19 smithi067 ceph-mon[33511]: from='mgr.? 172.21.15.67:0/237541836' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-20T22:03:19.764 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:19.303+0000 7f5b79a85040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-20T22:03:19.764 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: [20/Sep/2024:22:03:19] ENGINE Bus STARTING 2024-09-20T22:03:19.764 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: CherryPy Checker: 2024-09-20T22:03:19.764 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: The Application mounted at '' has an empty config. 2024-09-20T22:03:19.764 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: 2024-09-20T22:03:19.764 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: [20/Sep/2024:22:03:19] ENGINE Serving on http://:::9283 2024-09-20T22:03:19.764 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:03:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x[35489]: [20/Sep/2024:22:03:19] ENGINE Bus STARTED 2024-09-20T22:03:19.939 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:19.972 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:19 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:03:19] ENGINE Bus STARTED 2024-09-20T22:03:20.465 INFO:teuthology.orchestra.run.smithi060.stdout:206158430221 2024-09-20T22:03:20.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-20T22:03:20.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: Active manager daemon y restarted 2024-09-20T22:03:20.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: Activating manager daemon y 2024-09-20T22:03:20.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: osdmap e74: 8 total, 8 up, 8 in 2024-09-20T22:03:20.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: mgrmap e19: y(active, starting, since 0.00867501s), standbys: x 2024-09-20T22:03:20.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T22:03:20.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T22:03:20.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T22:03:20.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-20T22:03:20.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-20T22:03:20.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T22:03:20.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:03:20.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:03:20.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:03:20.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:03:20.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-20T22:03:20.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:03:20.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:03:20.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-20T22:03:20.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-20T22:03:20.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-20T22:03:20.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: Manager daemon y is now available 2024-09-20T22:03:20.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:03:20.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:20.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:03:20.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-20T22:03:20.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-20T22:03:20.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-20T22:03:20.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: Active manager daemon y restarted 2024-09-20T22:03:20.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: Activating manager daemon y 2024-09-20T22:03:20.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: osdmap e74: 8 total, 8 up, 8 in 2024-09-20T22:03:20.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: mgrmap e19: y(active, starting, since 0.00867501s), standbys: x 2024-09-20T22:03:20.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T22:03:20.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T22:03:20.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T22:03:20.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-20T22:03:20.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-20T22:03:20.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T22:03:20.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:03:20.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:03:20.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:03:20.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:03:20.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-20T22:03:20.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:03:20.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:03:20.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-20T22:03:20.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-20T22:03:20.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-20T22:03:20.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: Manager daemon y is now available 2024-09-20T22:03:20.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:03:20.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:20.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:03:20.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-20T22:03:20.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-20T22:03:20.617 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: mgrmap e18: y(active, since 4m), standbys: x 2024-09-20T22:03:20.617 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: Active manager daemon y restarted 2024-09-20T22:03:20.617 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: Activating manager daemon y 2024-09-20T22:03:20.617 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: osdmap e74: 8 total, 8 up, 8 in 2024-09-20T22:03:20.617 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: mgrmap e19: y(active, starting, since 0.00867501s), standbys: x 2024-09-20T22:03:20.617 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-20T22:03:20.617 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-20T22:03:20.618 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-20T22:03:20.618 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-20T22:03:20.618 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-20T22:03:20.618 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-20T22:03:20.618 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:03:20.618 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-20T22:03:20.618 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-20T22:03:20.618 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:03:20.618 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-20T22:03:20.618 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-20T22:03:20.619 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-20T22:03:20.619 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-20T22:03:20.619 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-20T22:03:20.619 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-20T22:03:20.619 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: Manager daemon y is now available 2024-09-20T22:03:20.619 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:03:20.619 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:20.619 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:03:20.619 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-20T22:03:20.619 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-20T22:03:21.013 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430221 got 206158430221 for osd.6 2024-09-20T22:03:21.013 DEBUG:teuthology.parallel:result is None 2024-09-20T22:03:21.466 INFO:teuthology.orchestra.run.smithi060.stdout:55834574887 2024-09-20T22:03:21.563 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:03:21.563 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:03:21.563 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:03:21.563 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:03:21.563 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2487293044' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-20T22:03:21.563 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-mon[29694]: mgrmap e20: y(active, since 1.01275s), standbys: x 2024-09-20T22:03:21.563 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-mon[29694]: osdmap e75: 8 total, 8 up, 8 in 2024-09-20T22:03:21.563 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-mon[29694]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 95 MiB/s rd, 556 MiB/s wr, 236.57k op/s; 393 MiB/s, 1.40k objects/s recovering 2024-09-20T22:03:21.564 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:03:21.564 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:03:21.564 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:03:21.564 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:03:21.564 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2487293044' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-20T22:03:21.564 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-mon[40290]: mgrmap e20: y(active, since 1.01275s), standbys: x 2024-09-20T22:03:21.564 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-mon[40290]: osdmap e75: 8 total, 8 up, 8 in 2024-09-20T22:03:21.564 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-mon[40290]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 95 MiB/s rd, 556 MiB/s wr, 236.57k op/s; 393 MiB/s, 1.40k objects/s recovering 2024-09-20T22:03:21.661 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:21 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:03:21.661 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:21 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:03:21.661 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:21 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:03:21.661 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:21 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:03:21.661 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:21 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2487293044' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-20T22:03:21.661 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:21 smithi067 ceph-mon[33511]: mgrmap e20: y(active, since 1.01275s), standbys: x 2024-09-20T22:03:21.661 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:21 smithi067 ceph-mon[33511]: osdmap e75: 8 total, 8 up, 8 in 2024-09-20T22:03:21.662 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:21 smithi067 ceph-mon[33511]: pgmap v4: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 95 MiB/s rd, 556 MiB/s wr, 236.57k op/s; 393 MiB/s, 1.40k objects/s recovering 2024-09-20T22:03:21.844 INFO:teuthology.orchestra.run.smithi060.stdout:133143986202 2024-09-20T22:03:21.845 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:21.582+0000 7fa5106fe640 -1 log_channel(cephadm) log [ERR] : [20/Sep/2024:22:03:21] ENGINE Error in HTTPServer.serve 2024-09-20T22:03:21.845 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: Traceback (most recent call last): 2024-09-20T22:03:21.845 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-20T22:03:21.845 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: self._connections.run(self.expiration_interval) 2024-09-20T22:03:21.845 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-20T22:03:21.845 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: self._run(expiration_interval) 2024-09-20T22:03:21.845 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-20T22:03:21.845 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: new_conn = self._from_server_socket(self.server.socket) 2024-09-20T22:03:21.845 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-20T22:03:21.845 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-20T22:03:21.845 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-20T22:03:21.845 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: s = self.context.wrap_socket( 2024-09-20T22:03:21.845 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-20T22:03:21.846 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: return self.sslsocket_class._create( 2024-09-20T22:03:21.846 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-20T22:03:21.846 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: self.do_handshake() 2024-09-20T22:03:21.846 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-20T22:03:21.846 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: self._sslobj.do_handshake() 2024-09-20T22:03:21.846 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-20T22:03:21.846 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:03:22.014 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:21 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:03:22.040 INFO:teuthology.orchestra.run.smithi060.stdout:77309411364 2024-09-20T22:03:22.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:22 smithi067 ceph-mon[33511]: [20/Sep/2024:22:03:21] ENGINE Bus STARTING 2024-09-20T22:03:22.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:22 smithi067 ceph-mon[33511]: [20/Sep/2024:22:03:21] ENGINE Serving on http://172.21.15.60:8765 2024-09-20T22:03:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:22 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2790626292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-20T22:03:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:22 smithi067 ceph-mon[33511]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 418 KiB/s rd, 2.4 MiB/s wr, 1.02k op/s; 1.7 MiB/s, 6 objects/s recovering 2024-09-20T22:03:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:22 smithi067 ceph-mon[33511]: [20/Sep/2024:22:03:21] ENGINE Serving on https://172.21.15.60:7150 2024-09-20T22:03:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:22 smithi067 ceph-mon[33511]: [20/Sep/2024:22:03:21] ENGINE Bus STARTED 2024-09-20T22:03:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:22 smithi067 ceph-mon[33511]: [20/Sep/2024:22:03:21] ENGINE Error in HTTPServer.serve 2024-09-20T22:03:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout: Traceback (most recent call last): 2024-09-20T22:03:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-20T22:03:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout: self._connections.run(self.expiration_interval) 2024-09-20T22:03:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-20T22:03:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout: self._run(expiration_interval) 2024-09-20T22:03:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-20T22:03:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-20T22:03:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-20T22:03:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-20T22:03:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-20T22:03:22.515 INFO:journalctl@ceph.mon.b.smithi067.stdout: s = self.context.wrap_socket( 2024-09-20T22:03:22.515 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-20T22:03:22.515 INFO:journalctl@ceph.mon.b.smithi067.stdout: return self.sslsocket_class._create( 2024-09-20T22:03:22.515 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-20T22:03:22.515 INFO:journalctl@ceph.mon.b.smithi067.stdout: self.do_handshake() 2024-09-20T22:03:22.515 INFO:journalctl@ceph.mon.b.smithi067.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-20T22:03:22.515 INFO:journalctl@ceph.mon.b.smithi067.stdout: self._sslobj.do_handshake() 2024-09-20T22:03:22.515 INFO:journalctl@ceph.mon.b.smithi067.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-20T22:03:22.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:22 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:03:22.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:22 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1737545012' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-20T22:03:22.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:22 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/813770436' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-20T22:03:22.704 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[29694]: [20/Sep/2024:22:03:21] ENGINE Bus STARTING 2024-09-20T22:03:22.704 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[29694]: [20/Sep/2024:22:03:21] ENGINE Serving on http://172.21.15.60:8765 2024-09-20T22:03:22.704 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2790626292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-20T22:03:22.704 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[29694]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 418 KiB/s rd, 2.4 MiB/s wr, 1.02k op/s; 1.7 MiB/s, 6 objects/s recovering 2024-09-20T22:03:22.704 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[29694]: [20/Sep/2024:22:03:21] ENGINE Serving on https://172.21.15.60:7150 2024-09-20T22:03:22.705 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[29694]: [20/Sep/2024:22:03:21] ENGINE Bus STARTED 2024-09-20T22:03:22.705 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[29694]: [20/Sep/2024:22:03:21] ENGINE Error in HTTPServer.serve 2024-09-20T22:03:22.705 INFO:journalctl@ceph.mon.a.smithi060.stdout: Traceback (most recent call last): 2024-09-20T22:03:22.705 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-20T22:03:22.705 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._connections.run(self.expiration_interval) 2024-09-20T22:03:22.705 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-20T22:03:22.705 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._run(expiration_interval) 2024-09-20T22:03:22.705 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-20T22:03:22.705 INFO:journalctl@ceph.mon.a.smithi060.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-20T22:03:22.705 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-20T22:03:22.705 INFO:journalctl@ceph.mon.a.smithi060.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-20T22:03:22.705 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-20T22:03:22.705 INFO:journalctl@ceph.mon.a.smithi060.stdout: s = self.context.wrap_socket( 2024-09-20T22:03:22.706 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-20T22:03:22.706 INFO:journalctl@ceph.mon.a.smithi060.stdout: return self.sslsocket_class._create( 2024-09-20T22:03:22.706 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-20T22:03:22.706 INFO:journalctl@ceph.mon.a.smithi060.stdout: self.do_handshake() 2024-09-20T22:03:22.706 INFO:journalctl@ceph.mon.a.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-20T22:03:22.706 INFO:journalctl@ceph.mon.a.smithi060.stdout: self._sslobj.do_handshake() 2024-09-20T22:03:22.706 INFO:journalctl@ceph.mon.a.smithi060.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-20T22:03:22.706 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:03:22.706 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1737545012' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-20T22:03:22.706 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/813770436' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-20T22:03:22.706 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[40290]: [20/Sep/2024:22:03:21] ENGINE Bus STARTING 2024-09-20T22:03:22.706 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[40290]: [20/Sep/2024:22:03:21] ENGINE Serving on http://172.21.15.60:8765 2024-09-20T22:03:22.707 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2790626292' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-20T22:03:22.707 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[40290]: pgmap v5: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 418 KiB/s rd, 2.4 MiB/s wr, 1.02k op/s; 1.7 MiB/s, 6 objects/s recovering 2024-09-20T22:03:22.707 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[40290]: [20/Sep/2024:22:03:21] ENGINE Serving on https://172.21.15.60:7150 2024-09-20T22:03:22.707 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[40290]: [20/Sep/2024:22:03:21] ENGINE Bus STARTED 2024-09-20T22:03:22.707 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[40290]: [20/Sep/2024:22:03:21] ENGINE Error in HTTPServer.serve 2024-09-20T22:03:22.707 INFO:journalctl@ceph.mon.c.smithi060.stdout: Traceback (most recent call last): 2024-09-20T22:03:22.707 INFO:journalctl@ceph.mon.c.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-20T22:03:22.707 INFO:journalctl@ceph.mon.c.smithi060.stdout: self._connections.run(self.expiration_interval) 2024-09-20T22:03:22.707 INFO:journalctl@ceph.mon.c.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-20T22:03:22.707 INFO:journalctl@ceph.mon.c.smithi060.stdout: self._run(expiration_interval) 2024-09-20T22:03:22.707 INFO:journalctl@ceph.mon.c.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-20T22:03:22.707 INFO:journalctl@ceph.mon.c.smithi060.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-20T22:03:22.707 INFO:journalctl@ceph.mon.c.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-20T22:03:22.707 INFO:journalctl@ceph.mon.c.smithi060.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-20T22:03:22.707 INFO:journalctl@ceph.mon.c.smithi060.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-20T22:03:22.708 INFO:journalctl@ceph.mon.c.smithi060.stdout: s = self.context.wrap_socket( 2024-09-20T22:03:22.708 INFO:journalctl@ceph.mon.c.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-20T22:03:22.708 INFO:journalctl@ceph.mon.c.smithi060.stdout: return self.sslsocket_class._create( 2024-09-20T22:03:22.708 INFO:journalctl@ceph.mon.c.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-20T22:03:22.708 INFO:journalctl@ceph.mon.c.smithi060.stdout: self.do_handshake() 2024-09-20T22:03:22.708 INFO:journalctl@ceph.mon.c.smithi060.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-20T22:03:22.708 INFO:journalctl@ceph.mon.c.smithi060.stdout: self._sslobj.do_handshake() 2024-09-20T22:03:22.708 INFO:journalctl@ceph.mon.c.smithi060.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-20T22:03:22.708 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:03:22.708 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1737545012' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-20T22:03:22.708 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:22 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/813770436' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-20T22:03:22.731 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411363 got 77309411364 for osd.2 2024-09-20T22:03:22.731 DEBUG:teuthology.parallel:result is None 2024-09-20T22:03:23.201 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574888 got 55834574887 for osd.1 2024-09-20T22:03:23.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[29694]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-20T22:03:23.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:23.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:23.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:23.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:23.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:23.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:23.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:23.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:23.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[29694]: Adjusting osd_memory_target on smithi067 to 3760M 2024-09-20T22:03:23.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:23.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[40290]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-20T22:03:23.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:23.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:23.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:23.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:23.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:23.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:23.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:23.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:23.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[40290]: Adjusting osd_memory_target on smithi067 to 3760M 2024-09-20T22:03:23.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:23 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:23.605 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986201 got 133143986202 for osd.4 2024-09-20T22:03:23.605 DEBUG:teuthology.parallel:result is None 2024-09-20T22:03:23.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:23 smithi067 ceph-mon[33511]: mgrmap e21: y(active, since 2s), standbys: x 2024-09-20T22:03:23.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:23 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:23.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:23 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:23.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:23 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:23.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:23 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:23.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:23 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:23.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:23 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:23.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:23 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:23.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:23 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:23.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:23 smithi067 ceph-mon[33511]: Adjusting osd_memory_target on smithi067 to 3760M 2024-09-20T22:03:23.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:23 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:23.835 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:23.835 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:24.202 INFO:teuthology.orchestra.run.smithi060.stdout:167503724561 2024-09-20T22:03:24.202 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-20T22:03:24.203 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:24.203 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-20T22:03:24.446 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:24 smithi060 ceph-mon[40290]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 139 KiB/s rd, 816 KiB/s wr, 339 op/s; 577 KiB/s, 2 objects/s recovering 2024-09-20T22:03:24.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:24 smithi060 ceph-mon[29694]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 139 KiB/s rd, 816 KiB/s wr, 339 op/s; 577 KiB/s, 2 objects/s recovering 2024-09-20T22:03:24.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:24 smithi067 ceph-mon[33511]: pgmap v6: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 139 KiB/s rd, 816 KiB/s wr, 339 op/s; 577 KiB/s, 2 objects/s recovering 2024-09-20T22:03:25.113 INFO:teuthology.orchestra.run.smithi060.stdout:34359738415 2024-09-20T22:03:25.285 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:25.285 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:03:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:03:25.302 INFO:teuthology.orchestra.run.smithi060.stdout:111669149727 2024-09-20T22:03:25.555 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:25 smithi060 ceph-mon[29694]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-20T22:03:25.555 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:25 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1032327755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-20T22:03:25.555 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:25 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2897262757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-20T22:03:25.555 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:25 smithi060 ceph-mon[40290]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-20T22:03:25.555 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:25 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1032327755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-20T22:03:25.555 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:25 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2897262757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-20T22:03:25.763 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738414 got 34359738415 for osd.0 2024-09-20T22:03:25.763 DEBUG:teuthology.parallel:result is None 2024-09-20T22:03:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:25 smithi067 ceph-mon[33511]: mgrmap e22: y(active, since 4s), standbys: x 2024-09-20T22:03:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:25 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1032327755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-20T22:03:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:25 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2897262757' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-20T22:03:26.479 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:26 smithi060 ceph-mon[29694]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s rd, 489 KiB/s wr, 203 op/s; 346 KiB/s, 1 objects/s recovering 2024-09-20T22:03:26.479 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:26 smithi060 ceph-mon[40290]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s rd, 489 KiB/s wr, 203 op/s; 346 KiB/s, 1 objects/s recovering 2024-09-20T22:03:26.642 INFO:teuthology.orchestra.run.smithi060.stdout:55834574889 2024-09-20T22:03:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:26 smithi067 ceph-mon[33511]: pgmap v7: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 83 KiB/s rd, 489 KiB/s wr, 203 op/s; 346 KiB/s, 1 objects/s recovering 2024-09-20T22:03:26.785 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149726 got 111669149727 for osd.3 2024-09-20T22:03:26.786 DEBUG:teuthology.parallel:result is None 2024-09-20T22:03:26.937 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:27.663 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:27 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2896517968' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-20T22:03:27.664 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:27 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2896517968' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-20T22:03:27.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:27 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2896517968' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-20T22:03:27.796 INFO:teuthology.orchestra.run.smithi060.stdout:244813135882 2024-09-20T22:03:27.842 INFO:teuthology.orchestra.run.smithi060.stdout:167503724562 2024-09-20T22:03:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:28 smithi067 ceph-mon[33511]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 90 KiB/s rd, 350 KiB/s wr, 162 op/s; 247 KiB/s, 0 objects/s recovering 2024-09-20T22:03:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:28 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3784910676' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-20T22:03:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:28 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/725186819' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-20T22:03:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:28 smithi060 ceph-mon[29694]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 90 KiB/s rd, 350 KiB/s wr, 162 op/s; 247 KiB/s, 0 objects/s recovering 2024-09-20T22:03:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:28 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3784910676' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-20T22:03:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:28 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/725186819' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-20T22:03:28.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:28 smithi060 ceph-mon[40290]: pgmap v8: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 90 KiB/s rd, 350 KiB/s wr, 162 op/s; 247 KiB/s, 0 objects/s recovering 2024-09-20T22:03:28.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:28 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3784910676' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-20T22:03:28.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:28 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/725186819' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-20T22:03:28.932 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724561 got 167503724562 for osd.5 2024-09-20T22:03:28.932 DEBUG:teuthology.parallel:result is None 2024-09-20T22:03:29.438 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135881 got 244813135882 for osd.7 2024-09-20T22:03:29.438 DEBUG:teuthology.parallel:result is None 2024-09-20T22:03:29.812 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574888 got 55834574889 for osd.1 2024-09-20T22:03:29.812 DEBUG:teuthology.parallel:result is None 2024-09-20T22:03:29.812 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-20T22:03:29.813 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:03:30.312 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:30.756 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:30 smithi060 ceph-mon[29694]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s rd, 272 KiB/s wr, 126 op/s; 192 KiB/s, 0 objects/s recovering 2024-09-20T22:03:30.756 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:30 smithi060 ceph-mon[40290]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s rd, 272 KiB/s wr, 126 op/s; 192 KiB/s, 0 objects/s recovering 2024-09-20T22:03:30.965 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:03:30.966 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:03:31.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:30 smithi067 ceph-mon[33511]: pgmap v9: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 70 KiB/s rd, 272 KiB/s wr, 126 op/s; 192 KiB/s, 0 objects/s recovering 2024-09-20T22:03:31.716 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-09-20T22:03:29.533861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":963,"num_read_kb":809,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":601,"ondisk_log_size":601,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":244748,"kb_used_data":7708,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749716468,"statfs":{"total":767960285184,"available":767709663232,"internally_reserved":0,"allocated":7892992,"data_stored":4440577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":594692,"num_objects":122,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":0,"num_read":689,"num_read_kb":628,"num_write":449,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.998517"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072554+0000","last_change":"2024-09-20T22:02:50.111666+0000","last_active":"2024-09-20T22:03:21.072554+0000","last_peered":"2024-09-20T22:03:21.072554+0000","last_clean":"2024-09-20T22:03:21.072554+0000","last_became_active":"2024-09-20T22:02:50.109930+0000","last_became_peered":"2024-09-20T22:02:50.109930+0000","last_unstale":"2024-09-20T22:03:21.072554+0000","last_undegraded":"2024-09-20T22:03:21.072554+0000","last_fullsized":"2024-09-20T22:03:21.072554+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.513745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"71'9","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.534937+0000","last_change":"2024-09-20T22:02:52.277916+0000","last_active":"2024-09-20T22:03:20.534937+0000","last_peered":"2024-09-20T22:03:20.534937+0000","last_clean":"2024-09-20T22:03:20.534937+0000","last_became_active":"2024-09-20T22:02:52.277616+0000","last_became_peered":"2024-09-20T22:02:52.277616+0000","last_unstale":"2024-09-20T22:03:20.534937+0000","last_undegraded":"2024-09-20T22:03:20.534937+0000","last_fullsized":"2024-09-20T22:03:20.534937+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.412488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535985+0000","last_change":"2024-09-20T22:02:54.287938+0000","last_active":"2024-09-20T22:03:20.535985+0000","last_peered":"2024-09-20T22:03:20.535985+0000","last_clean":"2024-09-20T22:03:20.535985+0000","last_became_active":"2024-09-20T22:02:54.287537+0000","last_became_peered":"2024-09-20T22:02:54.287537+0000","last_unstale":"2024-09-20T22:03:20.535985+0000","last_undegraded":"2024-09-20T22:03:20.535985+0000","last_fullsized":"2024-09-20T22:03:20.535985+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.152471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535228+0000","last_change":"2024-09-20T22:02:56.297551+0000","last_active":"2024-09-20T22:03:20.535228+0000","last_peered":"2024-09-20T22:03:20.535228+0000","last_clean":"2024-09-20T22:03:20.535228+0000","last_became_active":"2024-09-20T22:02:56.297252+0000","last_became_peered":"2024-09-20T22:02:56.297252+0000","last_unstale":"2024-09-20T22:03:20.535228+0000","last_undegraded":"2024-09-20T22:03:20.535228+0000","last_fullsized":"2024-09-20T22:03:20.535228+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.977495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071617+0000","last_change":"2024-09-20T22:02:56.350387+0000","last_active":"2024-09-20T22:03:21.071617+0000","last_peered":"2024-09-20T22:03:21.071617+0000","last_clean":"2024-09-20T22:03:21.071617+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:03:21.071617+0000","last_undegraded":"2024-09-20T22:03:21.071617+0000","last_fullsized":"2024-09-20T22:03:21.071617+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.462175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072081+0000","last_change":"2024-09-20T22:02:50.107457+0000","last_active":"2024-09-20T22:03:21.072081+0000","last_peered":"2024-09-20T22:03:21.072081+0000","last_clean":"2024-09-20T22:03:21.072081+0000","last_became_active":"2024-09-20T22:02:50.107154+0000","last_became_peered":"2024-09-20T22:02:50.107154+0000","last_unstale":"2024-09-20T22:03:21.072081+0000","last_undegraded":"2024-09-20T22:03:21.072081+0000","last_fullsized":"2024-09-20T22:03:21.072081+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:11.498961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"71'15","reported_seq":54,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071355+0000","last_change":"2024-09-20T22:02:52.282478+0000","last_active":"2024-09-20T22:03:21.071355+0000","last_peered":"2024-09-20T22:03:21.071355+0000","last_clean":"2024-09-20T22:03:21.071355+0000","last_became_active":"2024-09-20T22:02:52.282101+0000","last_became_peered":"2024-09-20T22:02:52.282101+0000","last_unstale":"2024-09-20T22:03:21.071355+0000","last_undegraded":"2024-09-20T22:03:21.071355+0000","last_fullsized":"2024-09-20T22:03:21.071355+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.208571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.534947+0000","last_change":"2024-09-20T22:02:54.292666+0000","last_active":"2024-09-20T22:03:20.534947+0000","last_peered":"2024-09-20T22:03:20.534947+0000","last_clean":"2024-09-20T22:03:20.534947+0000","last_became_active":"2024-09-20T22:02:54.292343+0000","last_became_peered":"2024-09-20T22:02:54.292343+0000","last_unstale":"2024-09-20T22:03:20.534947+0000","last_undegraded":"2024-09-20T22:03:20.534947+0000","last_fullsized":"2024-09-20T22:03:20.534947+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.965688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071303+0000","last_change":"2024-09-20T22:02:50.108309+0000","last_active":"2024-09-20T22:03:21.071303+0000","last_peered":"2024-09-20T22:03:21.071303+0000","last_clean":"2024-09-20T22:03:21.071303+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:03:21.071303+0000","last_undegraded":"2024-09-20T22:03:21.071303+0000","last_fullsized":"2024-09-20T22:03:21.071303+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.039813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"71'9","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535516+0000","last_change":"2024-09-20T22:02:52.279933+0000","last_active":"2024-09-20T22:03:20.535516+0000","last_peered":"2024-09-20T22:03:20.535516+0000","last_clean":"2024-09-20T22:03:20.535516+0000","last_became_active":"2024-09-20T22:02:52.279526+0000","last_became_peered":"2024-09-20T22:02:52.279526+0000","last_unstale":"2024-09-20T22:03:20.535516+0000","last_undegraded":"2024-09-20T22:03:20.535516+0000","last_fullsized":"2024-09-20T22:03:20.535516+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.714248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070508+0000","last_change":"2024-09-20T22:02:54.293254+0000","last_active":"2024-09-20T22:03:21.070508+0000","last_peered":"2024-09-20T22:03:21.070508+0000","last_clean":"2024-09-20T22:03:21.070508+0000","last_became_active":"2024-09-20T22:02:54.292911+0000","last_became_peered":"2024-09-20T22:02:54.292911+0000","last_unstale":"2024-09-20T22:03:21.070508+0000","last_undegraded":"2024-09-20T22:03:21.070508+0000","last_fullsized":"2024-09-20T22:03:21.070508+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.908407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070886+0000","last_change":"2024-09-20T22:02:56.299059+0000","last_active":"2024-09-20T22:03:21.070886+0000","last_peered":"2024-09-20T22:03:21.070886+0000","last_clean":"2024-09-20T22:03:21.070886+0000","last_became_active":"2024-09-20T22:02:56.298660+0000","last_became_peered":"2024-09-20T22:02:56.298660+0000","last_unstale":"2024-09-20T22:03:21.070886+0000","last_undegraded":"2024-09-20T22:03:21.070886+0000","last_fullsized":"2024-09-20T22:03:21.070886+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:13.896683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070880+0000","last_change":"2024-09-20T22:02:50.106885+0000","last_active":"2024-09-20T22:03:21.070880+0000","last_peered":"2024-09-20T22:03:21.070880+0000","last_clean":"2024-09-20T22:03:21.070880+0000","last_became_active":"2024-09-20T22:02:50.106555+0000","last_became_peered":"2024-09-20T22:02:50.106555+0000","last_unstale":"2024-09-20T22:03:21.070880+0000","last_undegraded":"2024-09-20T22:03:21.070880+0000","last_fullsized":"2024-09-20T22:03:21.070880+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.204880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"71'5","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535354+0000","last_change":"2024-09-20T22:02:52.284347+0000","last_active":"2024-09-20T22:03:20.535354+0000","last_peered":"2024-09-20T22:03:20.535354+0000","last_clean":"2024-09-20T22:03:20.535354+0000","last_became_active":"2024-09-20T22:02:52.283904+0000","last_became_peered":"2024-09-20T22:02:52.283904+0000","last_unstale":"2024-09-20T22:03:20.535354+0000","last_undegraded":"2024-09-20T22:03:20.535354+0000","last_fullsized":"2024-09-20T22:03:20.535354+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:52.941046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703160+0000","last_change":"2024-09-20T22:02:54.295625+0000","last_active":"2024-09-20T22:03:20.703160+0000","last_peered":"2024-09-20T22:03:20.703160+0000","last_clean":"2024-09-20T22:03:20.703160+0000","last_became_active":"2024-09-20T22:02:54.294405+0000","last_became_peered":"2024-09-20T22:02:54.294405+0000","last_unstale":"2024-09-20T22:03:20.703160+0000","last_undegraded":"2024-09-20T22:03:20.703160+0000","last_fullsized":"2024-09-20T22:03:20.703160+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:53.468504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070421+0000","last_change":"2024-09-20T22:02:56.303701+0000","last_active":"2024-09-20T22:03:21.070421+0000","last_peered":"2024-09-20T22:03:21.070421+0000","last_clean":"2024-09-20T22:03:21.070421+0000","last_became_active":"2024-09-20T22:02:56.303338+0000","last_became_peered":"2024-09-20T22:02:56.303338+0000","last_unstale":"2024-09-20T22:03:21.070421+0000","last_undegraded":"2024-09-20T22:03:21.070421+0000","last_fullsized":"2024-09-20T22:03:21.070421+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.793299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535518+0000","last_change":"2024-09-20T22:02:56.347311+0000","last_active":"2024-09-20T22:03:20.535518+0000","last_peered":"2024-09-20T22:03:20.535518+0000","last_clean":"2024-09-20T22:03:20.535518+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:03:20.535518+0000","last_undegraded":"2024-09-20T22:03:20.535518+0000","last_fullsized":"2024-09-20T22:03:20.535518+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.297557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072043+0000","last_change":"2024-09-20T22:02:50.111842+0000","last_active":"2024-09-20T22:03:21.072043+0000","last_peered":"2024-09-20T22:03:21.072043+0000","last_clean":"2024-09-20T22:03:21.072043+0000","last_became_active":"2024-09-20T22:02:50.111517+0000","last_became_peered":"2024-09-20T22:02:50.111517+0000","last_unstale":"2024-09-20T22:03:21.072043+0000","last_undegraded":"2024-09-20T22:03:21.072043+0000","last_fullsized":"2024-09-20T22:03:21.072043+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.747690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"71'15","reported_seq":54,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071406+0000","last_change":"2024-09-20T22:02:52.283078+0000","last_active":"2024-09-20T22:03:21.071406+0000","last_peered":"2024-09-20T22:03:21.071406+0000","last_clean":"2024-09-20T22:03:21.071406+0000","last_became_active":"2024-09-20T22:02:52.282768+0000","last_became_peered":"2024-09-20T22:02:52.282768+0000","last_unstale":"2024-09-20T22:03:21.071406+0000","last_undegraded":"2024-09-20T22:03:21.071406+0000","last_fullsized":"2024-09-20T22:03:21.071406+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:52.589023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.536861+0000","last_change":"2024-09-20T22:02:54.292892+0000","last_active":"2024-09-20T22:03:20.536861+0000","last_peered":"2024-09-20T22:03:20.536861+0000","last_clean":"2024-09-20T22:03:20.536861+0000","last_became_active":"2024-09-20T22:02:54.290510+0000","last_became_peered":"2024-09-20T22:02:54.290510+0000","last_unstale":"2024-09-20T22:03:20.536861+0000","last_undegraded":"2024-09-20T22:03:20.536861+0000","last_fullsized":"2024-09-20T22:03:20.536861+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.173080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072228+0000","last_change":"2024-09-20T22:02:56.347951+0000","last_active":"2024-09-20T22:03:21.072228+0000","last_peered":"2024-09-20T22:03:21.072228+0000","last_clean":"2024-09-20T22:03:21.072228+0000","last_became_active":"2024-09-20T22:02:56.347561+0000","last_became_peered":"2024-09-20T22:02:56.347561+0000","last_unstale":"2024-09-20T22:03:21.072228+0000","last_undegraded":"2024-09-20T22:03:21.072228+0000","last_fullsized":"2024-09-20T22:03:21.072228+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.255072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535677+0000","last_change":"2024-09-20T22:02:50.108909+0000","last_active":"2024-09-20T22:03:20.535677+0000","last_peered":"2024-09-20T22:03:20.535677+0000","last_clean":"2024-09-20T22:03:20.535677+0000","last_became_active":"2024-09-20T22:02:50.107189+0000","last_became_peered":"2024-09-20T22:02:50.107189+0000","last_unstale":"2024-09-20T22:03:20.535677+0000","last_undegraded":"2024-09-20T22:03:20.535677+0000","last_fullsized":"2024-09-20T22:03:20.535677+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.636681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"71'12","reported_seq":52,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070801+0000","last_change":"2024-09-20T22:02:52.284201+0000","last_active":"2024-09-20T22:03:21.070801+0000","last_peered":"2024-09-20T22:03:21.070801+0000","last_clean":"2024-09-20T22:03:21.070801+0000","last_became_active":"2024-09-20T22:02:52.283843+0000","last_became_peered":"2024-09-20T22:02:52.283843+0000","last_unstale":"2024-09-20T22:03:21.070801+0000","last_undegraded":"2024-09-20T22:03:21.070801+0000","last_fullsized":"2024-09-20T22:03:21.070801+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.131718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535266+0000","last_change":"2024-09-20T22:02:54.293157+0000","last_active":"2024-09-20T22:03:20.535266+0000","last_peered":"2024-09-20T22:03:20.535266+0000","last_clean":"2024-09-20T22:03:20.535266+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:03:20.535266+0000","last_undegraded":"2024-09-20T22:03:20.535266+0000","last_fullsized":"2024-09-20T22:03:20.535266+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.203423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"71'1","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703332+0000","last_change":"2024-09-20T22:02:56.301720+0000","last_active":"2024-09-20T22:03:20.703332+0000","last_peered":"2024-09-20T22:03:20.703332+0000","last_clean":"2024-09-20T22:03:20.703332+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:03:20.703332+0000","last_undegraded":"2024-09-20T22:03:20.703332+0000","last_fullsized":"2024-09-20T22:03:20.703332+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.341799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"64'1","reported_seq":34,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537040+0000","last_change":"2024-09-20T22:02:50.113259+0000","last_active":"2024-09-20T22:03:20.537040+0000","last_peered":"2024-09-20T22:03:20.537040+0000","last_clean":"2024-09-20T22:03:20.537040+0000","last_became_active":"2024-09-20T22:02:50.112862+0000","last_became_peered":"2024-09-20T22:02:50.112862+0000","last_unstale":"2024-09-20T22:03:20.537040+0000","last_undegraded":"2024-09-20T22:03:20.537040+0000","last_fullsized":"2024-09-20T22:03:20.537040+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.645807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"71'10","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072457+0000","last_change":"2024-09-20T22:02:52.482300+0000","last_active":"2024-09-20T22:03:21.072457+0000","last_peered":"2024-09-20T22:03:21.072457+0000","last_clean":"2024-09-20T22:03:21.072457+0000","last_became_active":"2024-09-20T22:02:52.481980+0000","last_became_peered":"2024-09-20T22:02:52.481980+0000","last_unstale":"2024-09-20T22:03:21.072457+0000","last_undegraded":"2024-09-20T22:03:21.072457+0000","last_fullsized":"2024-09-20T22:03:21.072457+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.082587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"71'8","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071788+0000","last_change":"2024-09-20T22:02:54.297973+0000","last_active":"2024-09-20T22:03:21.071788+0000","last_peered":"2024-09-20T22:03:21.071788+0000","last_clean":"2024-09-20T22:03:21.071788+0000","last_became_active":"2024-09-20T22:02:54.297657+0000","last_became_peered":"2024-09-20T22:02:54.297657+0000","last_unstale":"2024-09-20T22:03:21.071788+0000","last_undegraded":"2024-09-20T22:03:21.071788+0000","last_fullsized":"2024-09-20T22:03:21.071788+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:20.633207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"71'15","reported_seq":54,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537626+0000","last_change":"2024-09-20T22:02:52.284890+0000","last_active":"2024-09-20T22:03:20.537626+0000","last_peered":"2024-09-20T22:03:20.537626+0000","last_clean":"2024-09-20T22:03:20.537626+0000","last_became_active":"2024-09-20T22:02:52.284027+0000","last_became_peered":"2024-09-20T22:02:52.284027+0000","last_unstale":"2024-09-20T22:03:20.537626+0000","last_undegraded":"2024-09-20T22:03:20.537626+0000","last_fullsized":"2024-09-20T22:03:20.537626+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:54.544010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070835+0000","last_change":"2024-09-20T22:02:50.108849+0000","last_active":"2024-09-20T22:03:21.070835+0000","last_peered":"2024-09-20T22:03:21.070835+0000","last_clean":"2024-09-20T22:03:21.070835+0000","last_became_active":"2024-09-20T22:02:50.108508+0000","last_became_peered":"2024-09-20T22:02:50.108508+0000","last_unstale":"2024-09-20T22:03:21.070835+0000","last_undegraded":"2024-09-20T22:03:21.070835+0000","last_fullsized":"2024-09-20T22:03:21.070835+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.430379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"71'8","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535659+0000","last_change":"2024-09-20T22:02:54.295408+0000","last_active":"2024-09-20T22:03:20.535659+0000","last_peered":"2024-09-20T22:03:20.535659+0000","last_clean":"2024-09-20T22:03:20.535659+0000","last_became_active":"2024-09-20T22:02:54.294917+0000","last_became_peered":"2024-09-20T22:02:54.294917+0000","last_unstale":"2024-09-20T22:03:20.535659+0000","last_undegraded":"2024-09-20T22:03:20.535659+0000","last_fullsized":"2024-09-20T22:03:20.535659+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.541024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071048+0000","last_change":"2024-09-20T22:02:56.296725+0000","last_active":"2024-09-20T22:03:21.071048+0000","last_peered":"2024-09-20T22:03:21.071048+0000","last_clean":"2024-09-20T22:03:21.071048+0000","last_became_active":"2024-09-20T22:02:56.296436+0000","last_became_peered":"2024-09-20T22:02:56.296436+0000","last_unstale":"2024-09-20T22:03:21.071048+0000","last_undegraded":"2024-09-20T22:03:21.071048+0000","last_fullsized":"2024-09-20T22:03:21.071048+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:19.201794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"71'18","reported_seq":61,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070609+0000","last_change":"2024-09-20T22:02:52.482408+0000","last_active":"2024-09-20T22:03:21.070609+0000","last_peered":"2024-09-20T22:03:21.070609+0000","last_clean":"2024-09-20T22:03:21.070609+0000","last_became_active":"2024-09-20T22:02:52.482119+0000","last_became_peered":"2024-09-20T22:02:52.482119+0000","last_unstale":"2024-09-20T22:03:21.070609+0000","last_undegraded":"2024-09-20T22:03:21.070609+0000","last_fullsized":"2024-09-20T22:03:21.070609+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.477571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072419+0000","last_change":"2024-09-20T22:02:50.113398+0000","last_active":"2024-09-20T22:03:21.072419+0000","last_peered":"2024-09-20T22:03:21.072419+0000","last_clean":"2024-09-20T22:03:21.072419+0000","last_became_active":"2024-09-20T22:02:50.113076+0000","last_became_peered":"2024-09-20T22:02:50.113076+0000","last_unstale":"2024-09-20T22:03:21.072419+0000","last_undegraded":"2024-09-20T22:03:21.072419+0000","last_fullsized":"2024-09-20T22:03:21.072419+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.714101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535027+0000","last_change":"2024-09-20T22:02:54.288282+0000","last_active":"2024-09-20T22:03:20.535027+0000","last_peered":"2024-09-20T22:03:20.535027+0000","last_clean":"2024-09-20T22:03:20.535027+0000","last_became_active":"2024-09-20T22:02:54.288007+0000","last_became_peered":"2024-09-20T22:02:54.288007+0000","last_unstale":"2024-09-20T22:03:20.535027+0000","last_undegraded":"2024-09-20T22:03:20.535027+0000","last_fullsized":"2024-09-20T22:03:20.535027+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.737144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535412+0000","last_change":"2024-09-20T22:02:56.303051+0000","last_active":"2024-09-20T22:03:20.535412+0000","last_peered":"2024-09-20T22:03:20.535412+0000","last_clean":"2024-09-20T22:03:20.535412+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:03:20.535412+0000","last_undegraded":"2024-09-20T22:03:20.535412+0000","last_fullsized":"2024-09-20T22:03:20.535412+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.012094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"71'14","reported_seq":50,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535136+0000","last_change":"2024-09-20T22:02:52.273553+0000","last_active":"2024-09-20T22:03:20.535136+0000","last_peered":"2024-09-20T22:03:20.535136+0000","last_clean":"2024-09-20T22:03:20.535136+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:03:20.535136+0000","last_undegraded":"2024-09-20T22:03:20.535136+0000","last_fullsized":"2024-09-20T22:03:20.535136+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:56.047961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":34,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072627+0000","last_change":"2024-09-20T22:02:50.110546+0000","last_active":"2024-09-20T22:03:21.072627+0000","last_peered":"2024-09-20T22:03:21.072627+0000","last_clean":"2024-09-20T22:03:21.072627+0000","last_became_active":"2024-09-20T22:02:50.110195+0000","last_became_peered":"2024-09-20T22:02:50.110195+0000","last_unstale":"2024-09-20T22:03:21.072627+0000","last_undegraded":"2024-09-20T22:03:21.072627+0000","last_fullsized":"2024-09-20T22:03:21.072627+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.867716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"71'8","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070476+0000","last_change":"2024-09-20T22:02:54.295841+0000","last_active":"2024-09-20T22:03:21.070476+0000","last_peered":"2024-09-20T22:03:21.070476+0000","last_clean":"2024-09-20T22:03:21.070476+0000","last_became_active":"2024-09-20T22:02:54.295553+0000","last_became_peered":"2024-09-20T22:02:54.295553+0000","last_unstale":"2024-09-20T22:03:21.070476+0000","last_undegraded":"2024-09-20T22:03:21.070476+0000","last_fullsized":"2024-09-20T22:03:21.070476+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.625795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703312+0000","last_change":"2024-09-20T22:02:56.349864+0000","last_active":"2024-09-20T22:03:20.703312+0000","last_peered":"2024-09-20T22:03:20.703312+0000","last_clean":"2024-09-20T22:03:20.703312+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:03:20.703312+0000","last_undegraded":"2024-09-20T22:03:20.703312+0000","last_fullsized":"2024-09-20T22:03:20.703312+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:57.678673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"71'10","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537327+0000","last_change":"2024-09-20T22:02:52.276282+0000","last_active":"2024-09-20T22:03:20.537327+0000","last_peered":"2024-09-20T22:03:20.537327+0000","last_clean":"2024-09-20T22:03:20.537327+0000","last_became_active":"2024-09-20T22:02:52.275958+0000","last_became_peered":"2024-09-20T22:02:52.275958+0000","last_unstale":"2024-09-20T22:03:20.537327+0000","last_undegraded":"2024-09-20T22:03:20.537327+0000","last_fullsized":"2024-09-20T22:03:20.537327+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.846987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071357+0000","last_change":"2024-09-20T22:02:50.114476+0000","last_active":"2024-09-20T22:03:21.071357+0000","last_peered":"2024-09-20T22:03:21.071357+0000","last_clean":"2024-09-20T22:03:21.071357+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:03:21.071357+0000","last_undegraded":"2024-09-20T22:03:21.071357+0000","last_fullsized":"2024-09-20T22:03:21.071357+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:51.340463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"71'8","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071587+0000","last_change":"2024-09-20T22:02:54.293220+0000","last_active":"2024-09-20T22:03:21.071587+0000","last_peered":"2024-09-20T22:03:21.071587+0000","last_clean":"2024-09-20T22:03:21.071587+0000","last_became_active":"2024-09-20T22:02:54.292819+0000","last_became_peered":"2024-09-20T22:02:54.292819+0000","last_unstale":"2024-09-20T22:03:21.071587+0000","last_undegraded":"2024-09-20T22:03:21.071587+0000","last_fullsized":"2024-09-20T22:03:21.071587+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:59.827983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071031+0000","last_change":"2024-09-20T22:02:56.304595+0000","last_active":"2024-09-20T22:03:21.071031+0000","last_peered":"2024-09-20T22:03:21.071031+0000","last_clean":"2024-09-20T22:03:21.071031+0000","last_became_active":"2024-09-20T22:02:56.304274+0000","last_became_peered":"2024-09-20T22:02:56.304274+0000","last_unstale":"2024-09-20T22:03:21.071031+0000","last_undegraded":"2024-09-20T22:03:21.071031+0000","last_fullsized":"2024-09-20T22:03:21.071031+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.850583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"71'19","reported_seq":65,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071750+0000","last_change":"2024-09-20T22:02:52.480446+0000","last_active":"2024-09-20T22:03:21.071750+0000","last_peered":"2024-09-20T22:03:21.071750+0000","last_clean":"2024-09-20T22:03:21.071750+0000","last_became_active":"2024-09-20T22:02:52.480077+0000","last_became_peered":"2024-09-20T22:02:52.480077+0000","last_unstale":"2024-09-20T22:03:21.071750+0000","last_undegraded":"2024-09-20T22:03:21.071750+0000","last_fullsized":"2024-09-20T22:03:21.071750+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.830883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537868+0000","last_change":"2024-09-20T22:02:50.104649+0000","last_active":"2024-09-20T22:03:20.537868+0000","last_peered":"2024-09-20T22:03:20.537868+0000","last_clean":"2024-09-20T22:03:20.537868+0000","last_became_active":"2024-09-20T22:02:50.104345+0000","last_became_peered":"2024-09-20T22:02:50.104345+0000","last_unstale":"2024-09-20T22:03:20.537868+0000","last_undegraded":"2024-09-20T22:03:20.537868+0000","last_fullsized":"2024-09-20T22:03:20.537868+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.766875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070730+0000","last_change":"2024-09-20T22:02:54.291638+0000","last_active":"2024-09-20T22:03:21.070730+0000","last_peered":"2024-09-20T22:03:21.070730+0000","last_clean":"2024-09-20T22:03:21.070730+0000","last_became_active":"2024-09-20T22:02:54.290787+0000","last_became_peered":"2024-09-20T22:02:54.290787+0000","last_unstale":"2024-09-20T22:03:21.070730+0000","last_undegraded":"2024-09-20T22:03:21.070730+0000","last_fullsized":"2024-09-20T22:03:21.070730+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.996942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535226+0000","last_change":"2024-09-20T22:02:56.350020+0000","last_active":"2024-09-20T22:03:20.535226+0000","last_peered":"2024-09-20T22:03:20.535226+0000","last_clean":"2024-09-20T22:03:20.535226+0000","last_became_active":"2024-09-20T22:02:56.349581+0000","last_became_peered":"2024-09-20T22:02:56.349581+0000","last_unstale":"2024-09-20T22:03:20.535226+0000","last_undegraded":"2024-09-20T22:03:20.535226+0000","last_fullsized":"2024-09-20T22:03:20.535226+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.722790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"71'28","reported_seq":85,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537851+0000","last_change":"2024-09-20T22:02:52.285811+0000","last_active":"2024-09-20T22:03:20.537851+0000","last_peered":"2024-09-20T22:03:20.537851+0000","last_clean":"2024-09-20T22:03:20.537851+0000","last_became_active":"2024-09-20T22:02:52.285341+0000","last_became_peered":"2024-09-20T22:02:52.285341+0000","last_unstale":"2024-09-20T22:03:20.537851+0000","last_undegraded":"2024-09-20T22:03:20.537851+0000","last_fullsized":"2024-09-20T22:03:20.537851+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.491983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.536083+0000","last_change":"2024-09-20T22:02:50.110040+0000","last_active":"2024-09-20T22:03:20.536083+0000","last_peered":"2024-09-20T22:03:20.536083+0000","last_clean":"2024-09-20T22:03:20.536083+0000","last_became_active":"2024-09-20T22:02:50.108457+0000","last_became_peered":"2024-09-20T22:02:50.108457+0000","last_unstale":"2024-09-20T22:03:20.536083+0000","last_undegraded":"2024-09-20T22:03:20.536083+0000","last_fullsized":"2024-09-20T22:03:20.536083+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.280284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"66'2","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071023+0000","last_change":"2024-09-20T22:02:52.273740+0000","last_active":"2024-09-20T22:03:21.071023+0000","last_peered":"2024-09-20T22:03:21.071023+0000","last_clean":"2024-09-20T22:03:21.071023+0000","last_became_active":"2024-09-20T22:02:49.096453+0000","last_became_peered":"2024-09-20T22:02:49.096453+0000","last_unstale":"2024-09-20T22:03:21.071023+0000","last_undegraded":"2024-09-20T22:03:21.071023+0000","last_fullsized":"2024-09-20T22:03:21.071023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.811834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072280+0000","last_change":"2024-09-20T22:02:54.295696+0000","last_active":"2024-09-20T22:03:21.072280+0000","last_peered":"2024-09-20T22:03:21.072280+0000","last_clean":"2024-09-20T22:03:21.072280+0000","last_became_active":"2024-09-20T22:02:54.295401+0000","last_became_peered":"2024-09-20T22:02:54.295401+0000","last_unstale":"2024-09-20T22:03:21.072280+0000","last_undegraded":"2024-09-20T22:03:21.072280+0000","last_fullsized":"2024-09-20T22:03:21.072280+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.933605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071815+0000","last_change":"2024-09-20T22:02:56.306053+0000","last_active":"2024-09-20T22:03:21.071815+0000","last_peered":"2024-09-20T22:03:21.071815+0000","last_clean":"2024-09-20T22:03:21.071815+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:03:21.071815+0000","last_undegraded":"2024-09-20T22:03:21.071815+0000","last_fullsized":"2024-09-20T22:03:21.071815+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.682717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"71'13","reported_seq":56,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537737+0000","last_change":"2024-09-20T22:02:52.280309+0000","last_active":"2024-09-20T22:03:20.537737+0000","last_peered":"2024-09-20T22:03:20.537737+0000","last_clean":"2024-09-20T22:03:20.537737+0000","last_became_active":"2024-09-20T22:02:52.280005+0000","last_became_peered":"2024-09-20T22:02:52.280005+0000","last_unstale":"2024-09-20T22:03:20.537737+0000","last_undegraded":"2024-09-20T22:03:20.537737+0000","last_fullsized":"2024-09-20T22:03:20.537737+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.190808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535654+0000","last_change":"2024-09-20T22:02:50.106023+0000","last_active":"2024-09-20T22:03:20.535654+0000","last_peered":"2024-09-20T22:03:20.535654+0000","last_clean":"2024-09-20T22:03:20.535654+0000","last_became_active":"2024-09-20T22:02:50.105633+0000","last_became_peered":"2024-09-20T22:02:50.105633+0000","last_unstale":"2024-09-20T22:03:20.535654+0000","last_undegraded":"2024-09-20T22:03:20.535654+0000","last_fullsized":"2024-09-20T22:03:20.535654+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.449401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"64'1","reported_seq":42,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071025+0000","last_change":"2024-09-20T22:02:52.270463+0000","last_active":"2024-09-20T22:03:21.071025+0000","last_peered":"2024-09-20T22:03:21.071025+0000","last_clean":"2024-09-20T22:03:21.071025+0000","last_became_active":"2024-09-20T22:02:49.099508+0000","last_became_peered":"2024-09-20T22:02:49.099508+0000","last_unstale":"2024-09-20T22:03:21.071025+0000","last_undegraded":"2024-09-20T22:03:21.071025+0000","last_fullsized":"2024-09-20T22:03:21.071025+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.955566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070945+0000","last_change":"2024-09-20T22:02:54.292450+0000","last_active":"2024-09-20T22:03:21.070945+0000","last_peered":"2024-09-20T22:03:21.070945+0000","last_clean":"2024-09-20T22:03:21.070945+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:03:21.070945+0000","last_undegraded":"2024-09-20T22:03:21.070945+0000","last_fullsized":"2024-09-20T22:03:21.070945+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.941012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703517+0000","last_change":"2024-09-20T22:02:56.349789+0000","last_active":"2024-09-20T22:03:20.703517+0000","last_peered":"2024-09-20T22:03:20.703517+0000","last_clean":"2024-09-20T22:03:20.703517+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:03:20.703517+0000","last_undegraded":"2024-09-20T22:03:20.703517+0000","last_fullsized":"2024-09-20T22:03:20.703517+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.060034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"71'12","reported_seq":47,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072508+0000","last_change":"2024-09-20T22:02:52.277878+0000","last_active":"2024-09-20T22:03:21.072508+0000","last_peered":"2024-09-20T22:03:21.072508+0000","last_clean":"2024-09-20T22:03:21.072508+0000","last_became_active":"2024-09-20T22:02:52.277548+0000","last_became_peered":"2024-09-20T22:02:52.277548+0000","last_unstale":"2024-09-20T22:03:21.072508+0000","last_undegraded":"2024-09-20T22:03:21.072508+0000","last_fullsized":"2024-09-20T22:03:21.072508+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.295996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070632+0000","last_change":"2024-09-20T22:02:50.110740+0000","last_active":"2024-09-20T22:03:21.070632+0000","last_peered":"2024-09-20T22:03:21.070632+0000","last_clean":"2024-09-20T22:03:21.070632+0000","last_became_active":"2024-09-20T22:02:50.110448+0000","last_became_peered":"2024-09-20T22:02:50.110448+0000","last_unstale":"2024-09-20T22:03:21.070632+0000","last_undegraded":"2024-09-20T22:03:21.070632+0000","last_fullsized":"2024-09-20T22:03:21.070632+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.942941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":73,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:23.698494+0000","last_change":"2024-09-20T22:02:52.480790+0000","last_active":"2024-09-20T22:03:23.698494+0000","last_peered":"2024-09-20T22:03:23.698494+0000","last_clean":"2024-09-20T22:03:23.698494+0000","last_became_active":"2024-09-20T22:02:49.098777+0000","last_became_peered":"2024-09-20T22:02:49.098777+0000","last_unstale":"2024-09-20T22:03:23.698494+0000","last_undegraded":"2024-09-20T22:03:23.698494+0000","last_fullsized":"2024-09-20T22:03:23.698494+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.872201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070884+0000","last_change":"2024-09-20T22:02:54.294806+0000","last_active":"2024-09-20T22:03:21.070884+0000","last_peered":"2024-09-20T22:03:21.070884+0000","last_clean":"2024-09-20T22:03:21.070884+0000","last_became_active":"2024-09-20T22:02:54.294495+0000","last_became_peered":"2024-09-20T22:02:54.294495+0000","last_unstale":"2024-09-20T22:03:21.070884+0000","last_undegraded":"2024-09-20T22:03:21.070884+0000","last_fullsized":"2024-09-20T22:03:21.070884+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:23.237482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537571+0000","last_change":"2024-09-20T22:02:56.308897+0000","last_active":"2024-09-20T22:03:20.537571+0000","last_peered":"2024-09-20T22:03:20.537571+0000","last_clean":"2024-09-20T22:03:20.537571+0000","last_became_active":"2024-09-20T22:02:56.308574+0000","last_became_peered":"2024-09-20T22:02:56.308574+0000","last_unstale":"2024-09-20T22:03:20.537571+0000","last_undegraded":"2024-09-20T22:03:20.537571+0000","last_fullsized":"2024-09-20T22:03:20.537571+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.064046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"71'16","reported_seq":59,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071572+0000","last_change":"2024-09-20T22:02:52.482708+0000","last_active":"2024-09-20T22:03:21.071572+0000","last_peered":"2024-09-20T22:03:21.071572+0000","last_clean":"2024-09-20T22:03:21.071572+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:03:21.071572+0000","last_undegraded":"2024-09-20T22:03:21.071572+0000","last_fullsized":"2024-09-20T22:03:21.071572+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.243856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071690+0000","last_change":"2024-09-20T22:02:50.106319+0000","last_active":"2024-09-20T22:03:21.071690+0000","last_peered":"2024-09-20T22:03:21.071690+0000","last_clean":"2024-09-20T22:03:21.071690+0000","last_became_active":"2024-09-20T22:02:50.106011+0000","last_became_peered":"2024-09-20T22:02:50.106011+0000","last_unstale":"2024-09-20T22:03:21.071690+0000","last_undegraded":"2024-09-20T22:03:21.071690+0000","last_fullsized":"2024-09-20T22:03:21.071690+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.288394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":288,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.599816+0000","last_change":"2024-09-20T22:02:55.903219+0000","last_active":"2024-09-20T22:03:21.599816+0000","last_peered":"2024-09-20T22:03:21.599816+0000","last_clean":"2024-09-20T22:03:21.599816+0000","last_became_active":"2024-09-20T22:02:45.068482+0000","last_became_peered":"2024-09-20T22:02:45.068482+0000","last_unstale":"2024-09-20T22:03:21.599816+0000","last_undegraded":"2024-09-20T22:03:21.599816+0000","last_fullsized":"2024-09-20T22:03:21.599816+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'117","last_scrub_stamp":"2024-09-20T22:02:55.903139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.903139+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.843650+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703457+0000","last_change":"2024-09-20T22:02:54.298298+0000","last_active":"2024-09-20T22:03:20.703457+0000","last_peered":"2024-09-20T22:03:20.703457+0000","last_clean":"2024-09-20T22:03:20.703457+0000","last_became_active":"2024-09-20T22:02:54.298021+0000","last_became_peered":"2024-09-20T22:02:54.298021+0000","last_unstale":"2024-09-20T22:03:20.703457+0000","last_undegraded":"2024-09-20T22:03:20.703457+0000","last_fullsized":"2024-09-20T22:03:20.703457+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:54.157709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070644+0000","last_change":"2024-09-20T22:02:56.308039+0000","last_active":"2024-09-20T22:03:21.070644+0000","last_peered":"2024-09-20T22:03:21.070644+0000","last_clean":"2024-09-20T22:03:21.070644+0000","last_became_active":"2024-09-20T22:02:56.307749+0000","last_became_peered":"2024-09-20T22:02:56.307749+0000","last_unstale":"2024-09-20T22:03:21.070644+0000","last_undegraded":"2024-09-20T22:03:21.070644+0000","last_fullsized":"2024-09-20T22:03:21.070644+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.666810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"71'11","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535748+0000","last_change":"2024-09-20T22:02:52.273177+0000","last_active":"2024-09-20T22:03:20.535748+0000","last_peered":"2024-09-20T22:03:20.535748+0000","last_clean":"2024-09-20T22:03:20.535748+0000","last_became_active":"2024-09-20T22:02:52.272766+0000","last_became_peered":"2024-09-20T22:02:52.272766+0000","last_unstale":"2024-09-20T22:03:20.535748+0000","last_undegraded":"2024-09-20T22:03:20.535748+0000","last_fullsized":"2024-09-20T22:03:20.535748+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.073826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535258+0000","last_change":"2024-09-20T22:02:50.109626+0000","last_active":"2024-09-20T22:03:20.535258+0000","last_peered":"2024-09-20T22:03:20.535258+0000","last_clean":"2024-09-20T22:03:20.535258+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:03:20.535258+0000","last_undegraded":"2024-09-20T22:03:20.535258+0000","last_fullsized":"2024-09-20T22:03:20.535258+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.659206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070499+0000","last_change":"2024-09-20T22:02:54.290285+0000","last_active":"2024-09-20T22:03:21.070499+0000","last_peered":"2024-09-20T22:03:21.070499+0000","last_clean":"2024-09-20T22:03:21.070499+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:03:21.070499+0000","last_undegraded":"2024-09-20T22:03:21.070499+0000","last_fullsized":"2024-09-20T22:03:21.070499+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.665015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071605+0000","last_change":"2024-09-20T22:02:56.349707+0000","last_active":"2024-09-20T22:03:21.071605+0000","last_peered":"2024-09-20T22:03:21.071605+0000","last_clean":"2024-09-20T22:03:21.071605+0000","last_became_active":"2024-09-20T22:02:56.349376+0000","last_became_peered":"2024-09-20T22:02:56.349376+0000","last_unstale":"2024-09-20T22:03:21.071605+0000","last_undegraded":"2024-09-20T22:03:21.071605+0000","last_fullsized":"2024-09-20T22:03:21.071605+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.370704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"71'17","reported_seq":57,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535795+0000","last_change":"2024-09-20T22:02:52.274124+0000","last_active":"2024-09-20T22:03:20.535795+0000","last_peered":"2024-09-20T22:03:20.535795+0000","last_clean":"2024-09-20T22:03:20.535795+0000","last_became_active":"2024-09-20T22:02:52.273811+0000","last_became_peered":"2024-09-20T22:02:52.273811+0000","last_unstale":"2024-09-20T22:03:20.535795+0000","last_undegraded":"2024-09-20T22:03:20.535795+0000","last_fullsized":"2024-09-20T22:03:20.535795+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:52.793675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071073+0000","last_change":"2024-09-20T22:02:50.115379+0000","last_active":"2024-09-20T22:03:21.071073+0000","last_peered":"2024-09-20T22:03:21.071073+0000","last_clean":"2024-09-20T22:03:21.071073+0000","last_became_active":"2024-09-20T22:02:50.115015+0000","last_became_peered":"2024-09-20T22:02:50.115015+0000","last_unstale":"2024-09-20T22:03:21.071073+0000","last_undegraded":"2024-09-20T22:03:21.071073+0000","last_fullsized":"2024-09-20T22:03:21.071073+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:16.806437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537571+0000","last_change":"2024-09-20T22:02:54.291468+0000","last_active":"2024-09-20T22:03:20.537571+0000","last_peered":"2024-09-20T22:03:20.537571+0000","last_clean":"2024-09-20T22:03:20.537571+0000","last_became_active":"2024-09-20T22:02:54.291161+0000","last_became_peered":"2024-09-20T22:02:54.291161+0000","last_unstale":"2024-09-20T22:03:20.537571+0000","last_undegraded":"2024-09-20T22:03:20.537571+0000","last_fullsized":"2024-09-20T22:03:20.537571+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.806525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071370+0000","last_change":"2024-09-20T22:02:56.307571+0000","last_active":"2024-09-20T22:03:21.071370+0000","last_peered":"2024-09-20T22:03:21.071370+0000","last_clean":"2024-09-20T22:03:21.071370+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:03:21.071370+0000","last_undegraded":"2024-09-20T22:03:21.071370+0000","last_fullsized":"2024-09-20T22:03:21.071370+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:20.532247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"71'10","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535440+0000","last_change":"2024-09-20T22:02:52.284306+0000","last_active":"2024-09-20T22:03:20.535440+0000","last_peered":"2024-09-20T22:03:20.535440+0000","last_clean":"2024-09-20T22:03:20.535440+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:03:20.535440+0000","last_undegraded":"2024-09-20T22:03:20.535440+0000","last_fullsized":"2024-09-20T22:03:20.535440+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:20.155009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"64'1","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072098+0000","last_change":"2024-09-20T22:02:50.110772+0000","last_active":"2024-09-20T22:03:21.072098+0000","last_peered":"2024-09-20T22:03:21.072098+0000","last_clean":"2024-09-20T22:03:21.072098+0000","last_became_active":"2024-09-20T22:02:50.110446+0000","last_became_peered":"2024-09-20T22:02:50.110446+0000","last_unstale":"2024-09-20T22:03:21.072098+0000","last_undegraded":"2024-09-20T22:03:21.072098+0000","last_fullsized":"2024-09-20T22:03:21.072098+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.664497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"71'8","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070581+0000","last_change":"2024-09-20T22:02:54.294781+0000","last_active":"2024-09-20T22:03:21.070581+0000","last_peered":"2024-09-20T22:03:21.070581+0000","last_clean":"2024-09-20T22:03:21.070581+0000","last_became_active":"2024-09-20T22:02:54.294458+0000","last_became_peered":"2024-09-20T22:02:54.294458+0000","last_unstale":"2024-09-20T22:03:21.070581+0000","last_undegraded":"2024-09-20T22:03:21.070581+0000","last_fullsized":"2024-09-20T22:03:21.070581+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.016385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535397+0000","last_change":"2024-09-20T22:02:56.300412+0000","last_active":"2024-09-20T22:03:20.535397+0000","last_peered":"2024-09-20T22:03:20.535397+0000","last_clean":"2024-09-20T22:03:20.535397+0000","last_became_active":"2024-09-20T22:02:56.300093+0000","last_became_peered":"2024-09-20T22:02:56.300093+0000","last_unstale":"2024-09-20T22:03:20.535397+0000","last_undegraded":"2024-09-20T22:03:20.535397+0000","last_fullsized":"2024-09-20T22:03:20.535397+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.204657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"71'9","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071641+0000","last_change":"2024-09-20T22:02:52.278728+0000","last_active":"2024-09-20T22:03:21.071641+0000","last_peered":"2024-09-20T22:03:21.071641+0000","last_clean":"2024-09-20T22:03:21.071641+0000","last_became_active":"2024-09-20T22:02:52.278381+0000","last_became_peered":"2024-09-20T22:02:52.278381+0000","last_unstale":"2024-09-20T22:03:21.071641+0000","last_undegraded":"2024-09-20T22:03:21.071641+0000","last_fullsized":"2024-09-20T22:03:21.071641+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:51.365937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070990+0000","last_change":"2024-09-20T22:02:50.113046+0000","last_active":"2024-09-20T22:03:21.070990+0000","last_peered":"2024-09-20T22:03:21.070990+0000","last_clean":"2024-09-20T22:03:21.070990+0000","last_became_active":"2024-09-20T22:02:50.112682+0000","last_became_peered":"2024-09-20T22:02:50.112682+0000","last_unstale":"2024-09-20T22:03:21.070990+0000","last_undegraded":"2024-09-20T22:03:21.070990+0000","last_fullsized":"2024-09-20T22:03:21.070990+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.700772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070312+0000","last_change":"2024-09-20T22:02:54.289079+0000","last_active":"2024-09-20T22:03:21.070312+0000","last_peered":"2024-09-20T22:03:21.070312+0000","last_clean":"2024-09-20T22:03:21.070312+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:03:21.070312+0000","last_undegraded":"2024-09-20T22:03:21.070312+0000","last_fullsized":"2024-09-20T22:03:21.070312+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.034123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070529+0000","last_change":"2024-09-20T22:02:56.298414+0000","last_active":"2024-09-20T22:03:21.070529+0000","last_peered":"2024-09-20T22:03:21.070529+0000","last_clean":"2024-09-20T22:03:21.070529+0000","last_became_active":"2024-09-20T22:02:56.298046+0000","last_became_peered":"2024-09-20T22:02:56.298046+0000","last_unstale":"2024-09-20T22:03:21.070529+0000","last_undegraded":"2024-09-20T22:03:21.070529+0000","last_fullsized":"2024-09-20T22:03:21.070529+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:59.723832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"71'19","reported_seq":60,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070816+0000","last_change":"2024-09-20T22:02:52.480577+0000","last_active":"2024-09-20T22:03:21.070816+0000","last_peered":"2024-09-20T22:03:21.070816+0000","last_clean":"2024-09-20T22:03:21.070816+0000","last_became_active":"2024-09-20T22:02:52.480208+0000","last_became_peered":"2024-09-20T22:02:52.480208+0000","last_unstale":"2024-09-20T22:03:21.070816+0000","last_undegraded":"2024-09-20T22:03:21.070816+0000","last_fullsized":"2024-09-20T22:03:21.070816+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.853373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703789+0000","last_change":"2024-09-20T22:02:50.109094+0000","last_active":"2024-09-20T22:03:20.703789+0000","last_peered":"2024-09-20T22:03:20.703789+0000","last_clean":"2024-09-20T22:03:20.703789+0000","last_became_active":"2024-09-20T22:02:50.108712+0000","last_became_peered":"2024-09-20T22:02:50.108712+0000","last_unstale":"2024-09-20T22:03:20.703789+0000","last_undegraded":"2024-09-20T22:03:20.703789+0000","last_fullsized":"2024-09-20T22:03:20.703789+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:08.545134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070341+0000","last_change":"2024-09-20T22:02:54.291441+0000","last_active":"2024-09-20T22:03:21.070341+0000","last_peered":"2024-09-20T22:03:21.070341+0000","last_clean":"2024-09-20T22:03:21.070341+0000","last_became_active":"2024-09-20T22:02:54.291134+0000","last_became_peered":"2024-09-20T22:02:54.291134+0000","last_unstale":"2024-09-20T22:03:21.070341+0000","last_undegraded":"2024-09-20T22:03:21.070341+0000","last_fullsized":"2024-09-20T22:03:21.070341+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.677000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703250+0000","last_change":"2024-09-20T22:02:56.308001+0000","last_active":"2024-09-20T22:03:20.703250+0000","last_peered":"2024-09-20T22:03:20.703250+0000","last_clean":"2024-09-20T22:03:20.703250+0000","last_became_active":"2024-09-20T22:02:56.307687+0000","last_became_peered":"2024-09-20T22:02:56.307687+0000","last_unstale":"2024-09-20T22:03:20.703250+0000","last_undegraded":"2024-09-20T22:03:20.703250+0000","last_fullsized":"2024-09-20T22:03:20.703250+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.249693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"71'12","reported_seq":52,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535005+0000","last_change":"2024-09-20T22:02:52.284579+0000","last_active":"2024-09-20T22:03:20.535005+0000","last_peered":"2024-09-20T22:03:20.535005+0000","last_clean":"2024-09-20T22:03:20.535005+0000","last_became_active":"2024-09-20T22:02:52.284144+0000","last_became_peered":"2024-09-20T22:02:52.284144+0000","last_unstale":"2024-09-20T22:03:20.535005+0000","last_undegraded":"2024-09-20T22:03:20.535005+0000","last_fullsized":"2024-09-20T22:03:20.535005+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.901626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703857+0000","last_change":"2024-09-20T22:02:50.111591+0000","last_active":"2024-09-20T22:03:20.703857+0000","last_peered":"2024-09-20T22:03:20.703857+0000","last_clean":"2024-09-20T22:03:20.703857+0000","last_became_active":"2024-09-20T22:02:50.111267+0000","last_became_peered":"2024-09-20T22:02:50.111267+0000","last_unstale":"2024-09-20T22:03:20.703857+0000","last_undegraded":"2024-09-20T22:03:20.703857+0000","last_fullsized":"2024-09-20T22:03:20.703857+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.038287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070129+0000","last_change":"2024-09-20T22:02:54.293804+0000","last_active":"2024-09-20T22:03:21.070129+0000","last_peered":"2024-09-20T22:03:21.070129+0000","last_clean":"2024-09-20T22:03:21.070129+0000","last_became_active":"2024-09-20T22:02:54.293377+0000","last_became_peered":"2024-09-20T22:02:54.293377+0000","last_unstale":"2024-09-20T22:03:21.070129+0000","last_undegraded":"2024-09-20T22:03:21.070129+0000","last_fullsized":"2024-09-20T22:03:21.070129+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.827756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071742+0000","last_change":"2024-09-20T22:02:56.307499+0000","last_active":"2024-09-20T22:03:21.071742+0000","last_peered":"2024-09-20T22:03:21.071742+0000","last_clean":"2024-09-20T22:03:21.071742+0000","last_became_active":"2024-09-20T22:02:56.306991+0000","last_became_peered":"2024-09-20T22:02:56.306991+0000","last_unstale":"2024-09-20T22:03:21.071742+0000","last_undegraded":"2024-09-20T22:03:21.071742+0000","last_fullsized":"2024-09-20T22:03:21.071742+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.424396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"71'15","reported_seq":54,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070130+0000","last_change":"2024-09-20T22:02:52.482395+0000","last_active":"2024-09-20T22:03:21.070130+0000","last_peered":"2024-09-20T22:03:21.070130+0000","last_clean":"2024-09-20T22:03:21.070130+0000","last_became_active":"2024-09-20T22:02:52.482008+0000","last_became_peered":"2024-09-20T22:02:52.482008+0000","last_unstale":"2024-09-20T22:03:21.070130+0000","last_undegraded":"2024-09-20T22:03:21.070130+0000","last_fullsized":"2024-09-20T22:03:21.070130+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.554179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"64'2","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703795+0000","last_change":"2024-09-20T22:02:50.115186+0000","last_active":"2024-09-20T22:03:20.703795+0000","last_peered":"2024-09-20T22:03:20.703795+0000","last_clean":"2024-09-20T22:03:20.703795+0000","last_became_active":"2024-09-20T22:02:50.114819+0000","last_became_peered":"2024-09-20T22:02:50.114819+0000","last_unstale":"2024-09-20T22:03:20.703795+0000","last_undegraded":"2024-09-20T22:03:20.703795+0000","last_fullsized":"2024-09-20T22:03:20.703795+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.801926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703218+0000","last_change":"2024-09-20T22:02:54.296823+0000","last_active":"2024-09-20T22:03:20.703218+0000","last_peered":"2024-09-20T22:03:20.703218+0000","last_clean":"2024-09-20T22:03:20.703218+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:03:20.703218+0000","last_undegraded":"2024-09-20T22:03:20.703218+0000","last_fullsized":"2024-09-20T22:03:20.703218+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.496588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070689+0000","last_change":"2024-09-20T22:02:56.347283+0000","last_active":"2024-09-20T22:03:21.070689+0000","last_peered":"2024-09-20T22:03:21.070689+0000","last_clean":"2024-09-20T22:03:21.070689+0000","last_became_active":"2024-09-20T22:02:56.346990+0000","last_became_peered":"2024-09-20T22:02:56.346990+0000","last_unstale":"2024-09-20T22:03:21.070689+0000","last_undegraded":"2024-09-20T22:03:21.070689+0000","last_fullsized":"2024-09-20T22:03:21.070689+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.718016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070498+0000","last_change":"2024-09-20T22:02:50.112245+0000","last_active":"2024-09-20T22:03:21.070498+0000","last_peered":"2024-09-20T22:03:21.070498+0000","last_clean":"2024-09-20T22:03:21.070498+0000","last_became_active":"2024-09-20T22:02:50.111935+0000","last_became_peered":"2024-09-20T22:02:50.111935+0000","last_unstale":"2024-09-20T22:03:21.070498+0000","last_undegraded":"2024-09-20T22:03:21.070498+0000","last_fullsized":"2024-09-20T22:03:21.070498+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:16.998775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"71'6","reported_seq":38,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072364+0000","last_change":"2024-09-20T22:02:52.481046+0000","last_active":"2024-09-20T22:03:21.072364+0000","last_peered":"2024-09-20T22:03:21.072364+0000","last_clean":"2024-09-20T22:03:21.072364+0000","last_became_active":"2024-09-20T22:02:52.480771+0000","last_became_peered":"2024-09-20T22:02:52.480771+0000","last_unstale":"2024-09-20T22:03:21.072364+0000","last_undegraded":"2024-09-20T22:03:21.072364+0000","last_fullsized":"2024-09-20T22:03:21.072364+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.943982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070886+0000","last_change":"2024-09-20T22:02:54.294834+0000","last_active":"2024-09-20T22:03:21.070886+0000","last_peered":"2024-09-20T22:03:21.070886+0000","last_clean":"2024-09-20T22:03:21.070886+0000","last_became_active":"2024-09-20T22:02:54.294539+0000","last_became_peered":"2024-09-20T22:02:54.294539+0000","last_unstale":"2024-09-20T22:03:21.070886+0000","last_undegraded":"2024-09-20T22:03:21.070886+0000","last_fullsized":"2024-09-20T22:03:21.070886+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.523368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703485+0000","last_change":"2024-09-20T22:02:56.350065+0000","last_active":"2024-09-20T22:03:20.703485+0000","last_peered":"2024-09-20T22:03:20.703485+0000","last_clean":"2024-09-20T22:03:20.703485+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:03:20.703485+0000","last_undegraded":"2024-09-20T22:03:20.703485+0000","last_fullsized":"2024-09-20T22:03:20.703485+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:19.909723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"71'9","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072562+0000","last_change":"2024-09-20T22:02:52.482361+0000","last_active":"2024-09-20T22:03:21.072562+0000","last_peered":"2024-09-20T22:03:21.072562+0000","last_clean":"2024-09-20T22:03:21.072562+0000","last_became_active":"2024-09-20T22:02:52.481989+0000","last_became_peered":"2024-09-20T22:02:52.481989+0000","last_unstale":"2024-09-20T22:03:21.072562+0000","last_undegraded":"2024-09-20T22:03:21.072562+0000","last_fullsized":"2024-09-20T22:03:21.072562+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.401767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703355+0000","last_change":"2024-09-20T22:02:50.108943+0000","last_active":"2024-09-20T22:03:20.703355+0000","last_peered":"2024-09-20T22:03:20.703355+0000","last_clean":"2024-09-20T22:03:20.703355+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:03:20.703355+0000","last_undegraded":"2024-09-20T22:03:20.703355+0000","last_fullsized":"2024-09-20T22:03:20.703355+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.114642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072267+0000","last_change":"2024-09-20T22:02:54.298137+0000","last_active":"2024-09-20T22:03:21.072267+0000","last_peered":"2024-09-20T22:03:21.072267+0000","last_clean":"2024-09-20T22:03:21.072267+0000","last_became_active":"2024-09-20T22:02:54.297850+0000","last_became_peered":"2024-09-20T22:02:54.297850+0000","last_unstale":"2024-09-20T22:03:21.072267+0000","last_undegraded":"2024-09-20T22:03:21.072267+0000","last_fullsized":"2024-09-20T22:03:21.072267+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.705512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071088+0000","last_change":"2024-09-20T22:02:56.299306+0000","last_active":"2024-09-20T22:03:21.071088+0000","last_peered":"2024-09-20T22:03:21.071088+0000","last_clean":"2024-09-20T22:03:21.071088+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:03:21.071088+0000","last_undegraded":"2024-09-20T22:03:21.071088+0000","last_fullsized":"2024-09-20T22:03:21.071088+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:11.499111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"71'9","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071259+0000","last_change":"2024-09-20T22:02:52.482498+0000","last_active":"2024-09-20T22:03:21.071259+0000","last_peered":"2024-09-20T22:03:21.071259+0000","last_clean":"2024-09-20T22:03:21.071259+0000","last_became_active":"2024-09-20T22:02:52.482141+0000","last_became_peered":"2024-09-20T22:02:52.482141+0000","last_unstale":"2024-09-20T22:03:21.071259+0000","last_undegraded":"2024-09-20T22:03:21.071259+0000","last_fullsized":"2024-09-20T22:03:21.071259+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.093555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070666+0000","last_change":"2024-09-20T22:02:50.112533+0000","last_active":"2024-09-20T22:03:21.070666+0000","last_peered":"2024-09-20T22:03:21.070666+0000","last_clean":"2024-09-20T22:03:21.070666+0000","last_became_active":"2024-09-20T22:02:50.110075+0000","last_became_peered":"2024-09-20T22:02:50.110075+0000","last_unstale":"2024-09-20T22:03:21.070666+0000","last_undegraded":"2024-09-20T22:03:21.070666+0000","last_fullsized":"2024-09-20T22:03:21.070666+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.714513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"71'8","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071783+0000","last_change":"2024-09-20T22:02:54.294653+0000","last_active":"2024-09-20T22:03:21.071783+0000","last_peered":"2024-09-20T22:03:21.071783+0000","last_clean":"2024-09-20T22:03:21.071783+0000","last_became_active":"2024-09-20T22:02:54.294381+0000","last_became_peered":"2024-09-20T22:02:54.294381+0000","last_unstale":"2024-09-20T22:03:21.071783+0000","last_undegraded":"2024-09-20T22:03:21.071783+0000","last_fullsized":"2024-09-20T22:03:21.071783+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.135221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535469+0000","last_change":"2024-09-20T22:02:56.300405+0000","last_active":"2024-09-20T22:03:20.535469+0000","last_peered":"2024-09-20T22:03:20.535469+0000","last_clean":"2024-09-20T22:03:20.535469+0000","last_became_active":"2024-09-20T22:02:56.300041+0000","last_became_peered":"2024-09-20T22:02:56.300041+0000","last_unstale":"2024-09-20T22:03:20.535469+0000","last_undegraded":"2024-09-20T22:03:20.535469+0000","last_fullsized":"2024-09-20T22:03:20.535469+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.312552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"71'10","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071846+0000","last_change":"2024-09-20T22:02:52.480909+0000","last_active":"2024-09-20T22:03:21.071846+0000","last_peered":"2024-09-20T22:03:21.071846+0000","last_clean":"2024-09-20T22:03:21.071846+0000","last_became_active":"2024-09-20T22:02:52.480591+0000","last_became_peered":"2024-09-20T22:02:52.480591+0000","last_unstale":"2024-09-20T22:03:21.071846+0000","last_undegraded":"2024-09-20T22:03:21.071846+0000","last_fullsized":"2024-09-20T22:03:21.071846+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.915698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703658+0000","last_change":"2024-09-20T22:02:50.112976+0000","last_active":"2024-09-20T22:03:20.703658+0000","last_peered":"2024-09-20T22:03:20.703658+0000","last_clean":"2024-09-20T22:03:20.703658+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:03:20.703658+0000","last_undegraded":"2024-09-20T22:03:20.703658+0000","last_fullsized":"2024-09-20T22:03:20.703658+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.057376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"71'8","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071189+0000","last_change":"2024-09-20T22:02:54.292909+0000","last_active":"2024-09-20T22:03:21.071189+0000","last_peered":"2024-09-20T22:03:21.071189+0000","last_clean":"2024-09-20T22:03:21.071189+0000","last_became_active":"2024-09-20T22:02:54.289548+0000","last_became_peered":"2024-09-20T22:02:54.289548+0000","last_unstale":"2024-09-20T22:03:21.071189+0000","last_undegraded":"2024-09-20T22:03:21.071189+0000","last_fullsized":"2024-09-20T22:03:21.071189+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.478336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072333+0000","last_change":"2024-09-20T22:02:56.305690+0000","last_active":"2024-09-20T22:03:21.072333+0000","last_peered":"2024-09-20T22:03:21.072333+0000","last_clean":"2024-09-20T22:03:21.072333+0000","last_became_active":"2024-09-20T22:02:56.305383+0000","last_became_peered":"2024-09-20T22:02:56.305383+0000","last_unstale":"2024-09-20T22:03:21.072333+0000","last_undegraded":"2024-09-20T22:03:21.072333+0000","last_fullsized":"2024-09-20T22:03:21.072333+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.293958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"71'11","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535982+0000","last_change":"2024-09-20T22:02:52.277987+0000","last_active":"2024-09-20T22:03:20.535982+0000","last_peered":"2024-09-20T22:03:20.535982+0000","last_clean":"2024-09-20T22:03:20.535982+0000","last_became_active":"2024-09-20T22:02:52.277479+0000","last_became_peered":"2024-09-20T22:02:52.277479+0000","last_unstale":"2024-09-20T22:03:20.535982+0000","last_undegraded":"2024-09-20T22:03:20.535982+0000","last_fullsized":"2024-09-20T22:03:20.535982+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.222570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535540+0000","last_change":"2024-09-20T22:02:50.109201+0000","last_active":"2024-09-20T22:03:20.535540+0000","last_peered":"2024-09-20T22:03:20.535540+0000","last_clean":"2024-09-20T22:03:20.535540+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:03:20.535540+0000","last_undegraded":"2024-09-20T22:03:20.535540+0000","last_fullsized":"2024-09-20T22:03:20.535540+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.129228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537256+0000","last_change":"2024-09-20T22:02:54.286502+0000","last_active":"2024-09-20T22:03:20.537256+0000","last_peered":"2024-09-20T22:03:20.537256+0000","last_clean":"2024-09-20T22:03:20.537256+0000","last_became_active":"2024-09-20T22:02:54.286200+0000","last_became_peered":"2024-09-20T22:02:54.286200+0000","last_unstale":"2024-09-20T22:03:20.537256+0000","last_undegraded":"2024-09-20T22:03:20.537256+0000","last_fullsized":"2024-09-20T22:03:20.537256+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:22.326441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070972+0000","last_change":"2024-09-20T22:02:56.307258+0000","last_active":"2024-09-20T22:03:21.070972+0000","last_peered":"2024-09-20T22:03:21.070972+0000","last_clean":"2024-09-20T22:03:21.070972+0000","last_became_active":"2024-09-20T22:02:56.306892+0000","last_became_peered":"2024-09-20T22:02:56.306892+0000","last_unstale":"2024-09-20T22:03:21.070972+0000","last_undegraded":"2024-09-20T22:03:21.070972+0000","last_fullsized":"2024-09-20T22:03:21.070972+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.948765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"71'9","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537344+0000","last_change":"2024-09-20T22:02:52.280815+0000","last_active":"2024-09-20T22:03:20.537344+0000","last_peered":"2024-09-20T22:03:20.537344+0000","last_clean":"2024-09-20T22:03:20.537344+0000","last_became_active":"2024-09-20T22:02:52.280428+0000","last_became_peered":"2024-09-20T22:02:52.280428+0000","last_unstale":"2024-09-20T22:03:20.537344+0000","last_undegraded":"2024-09-20T22:03:20.537344+0000","last_fullsized":"2024-09-20T22:03:20.537344+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.548215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703689+0000","last_change":"2024-09-20T22:02:50.110105+0000","last_active":"2024-09-20T22:03:20.703689+0000","last_peered":"2024-09-20T22:03:20.703689+0000","last_clean":"2024-09-20T22:03:20.703689+0000","last_became_active":"2024-09-20T22:02:50.109803+0000","last_became_peered":"2024-09-20T22:02:50.109803+0000","last_unstale":"2024-09-20T22:03:20.703689+0000","last_undegraded":"2024-09-20T22:03:20.703689+0000","last_fullsized":"2024-09-20T22:03:20.703689+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:53.587349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071838+0000","last_change":"2024-09-20T22:02:54.298130+0000","last_active":"2024-09-20T22:03:21.071838+0000","last_peered":"2024-09-20T22:03:21.071838+0000","last_clean":"2024-09-20T22:03:21.071838+0000","last_became_active":"2024-09-20T22:02:54.297803+0000","last_became_peered":"2024-09-20T22:02:54.297803+0000","last_unstale":"2024-09-20T22:03:21.071838+0000","last_undegraded":"2024-09-20T22:03:21.071838+0000","last_fullsized":"2024-09-20T22:03:21.071838+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.520061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070749+0000","last_change":"2024-09-20T22:02:56.298228+0000","last_active":"2024-09-20T22:03:21.070749+0000","last_peered":"2024-09-20T22:03:21.070749+0000","last_clean":"2024-09-20T22:03:21.070749+0000","last_became_active":"2024-09-20T22:02:56.297826+0000","last_became_peered":"2024-09-20T22:02:56.297826+0000","last_unstale":"2024-09-20T22:03:21.070749+0000","last_undegraded":"2024-09-20T22:03:21.070749+0000","last_fullsized":"2024-09-20T22:03:21.070749+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.126658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"71'11","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071102+0000","last_change":"2024-09-20T22:02:52.480504+0000","last_active":"2024-09-20T22:03:21.071102+0000","last_peered":"2024-09-20T22:03:21.071102+0000","last_clean":"2024-09-20T22:03:21.071102+0000","last_became_active":"2024-09-20T22:02:52.480118+0000","last_became_peered":"2024-09-20T22:02:52.480118+0000","last_unstale":"2024-09-20T22:03:21.071102+0000","last_undegraded":"2024-09-20T22:03:21.071102+0000","last_fullsized":"2024-09-20T22:03:21.071102+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.023916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071093+0000","last_change":"2024-09-20T22:02:50.110808+0000","last_active":"2024-09-20T22:03:21.071093+0000","last_peered":"2024-09-20T22:03:21.071093+0000","last_clean":"2024-09-20T22:03:21.071093+0000","last_became_active":"2024-09-20T22:02:50.110421+0000","last_became_peered":"2024-09-20T22:02:50.110421+0000","last_unstale":"2024-09-20T22:03:21.071093+0000","last_undegraded":"2024-09-20T22:03:21.071093+0000","last_fullsized":"2024-09-20T22:03:21.071093+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:53.027475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703032+0000","last_change":"2024-09-20T22:02:54.293068+0000","last_active":"2024-09-20T22:03:20.703032+0000","last_peered":"2024-09-20T22:03:20.703032+0000","last_clean":"2024-09-20T22:03:20.703032+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:03:20.703032+0000","last_undegraded":"2024-09-20T22:03:20.703032+0000","last_fullsized":"2024-09-20T22:03:20.703032+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.450710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072001+0000","last_change":"2024-09-20T22:02:56.348065+0000","last_active":"2024-09-20T22:03:21.072001+0000","last_peered":"2024-09-20T22:03:21.072001+0000","last_clean":"2024-09-20T22:03:21.072001+0000","last_became_active":"2024-09-20T22:02:56.347637+0000","last_became_peered":"2024-09-20T22:02:56.347637+0000","last_unstale":"2024-09-20T22:03:21.072001+0000","last_undegraded":"2024-09-20T22:03:21.072001+0000","last_fullsized":"2024-09-20T22:03:21.072001+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:13.568557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"71'4","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070523+0000","last_change":"2024-09-20T22:02:52.286086+0000","last_active":"2024-09-20T22:03:21.070523+0000","last_peered":"2024-09-20T22:03:21.070523+0000","last_clean":"2024-09-20T22:03:21.070523+0000","last_became_active":"2024-09-20T22:02:52.285701+0000","last_became_peered":"2024-09-20T22:02:52.285701+0000","last_unstale":"2024-09-20T22:03:21.070523+0000","last_undegraded":"2024-09-20T22:03:21.070523+0000","last_fullsized":"2024-09-20T22:03:21.070523+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.474692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072586+0000","last_change":"2024-09-20T22:02:50.111749+0000","last_active":"2024-09-20T22:03:21.072586+0000","last_peered":"2024-09-20T22:03:21.072586+0000","last_clean":"2024-09-20T22:03:21.072586+0000","last_became_active":"2024-09-20T22:02:50.111142+0000","last_became_peered":"2024-09-20T22:02:50.111142+0000","last_unstale":"2024-09-20T22:03:21.072586+0000","last_undegraded":"2024-09-20T22:03:21.072586+0000","last_fullsized":"2024-09-20T22:03:21.072586+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:57.037771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070393+0000","last_change":"2024-09-20T22:02:54.290418+0000","last_active":"2024-09-20T22:03:21.070393+0000","last_peered":"2024-09-20T22:03:21.070393+0000","last_clean":"2024-09-20T22:03:21.070393+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:03:21.070393+0000","last_undegraded":"2024-09-20T22:03:21.070393+0000","last_fullsized":"2024-09-20T22:03:21.070393+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:57.209065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703153+0000","last_change":"2024-09-20T22:02:56.300559+0000","last_active":"2024-09-20T22:03:20.703153+0000","last_peered":"2024-09-20T22:03:20.703153+0000","last_clean":"2024-09-20T22:03:20.703153+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:03:20.703153+0000","last_undegraded":"2024-09-20T22:03:20.703153+0000","last_fullsized":"2024-09-20T22:03:20.703153+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.506369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.536607+0000","last_change":"2024-09-20T22:02:56.301611+0000","last_active":"2024-09-20T22:03:20.536607+0000","last_peered":"2024-09-20T22:03:20.536607+0000","last_clean":"2024-09-20T22:03:20.536607+0000","last_became_active":"2024-09-20T22:02:56.301247+0000","last_became_peered":"2024-09-20T22:02:56.301247+0000","last_unstale":"2024-09-20T22:03:20.536607+0000","last_undegraded":"2024-09-20T22:03:20.536607+0000","last_fullsized":"2024-09-20T22:03:20.536607+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.486378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070755+0000","last_change":"2024-09-20T22:02:50.114519+0000","last_active":"2024-09-20T22:03:21.070755+0000","last_peered":"2024-09-20T22:03:21.070755+0000","last_clean":"2024-09-20T22:03:21.070755+0000","last_became_active":"2024-09-20T22:02:50.114204+0000","last_became_peered":"2024-09-20T22:02:50.114204+0000","last_unstale":"2024-09-20T22:03:21.070755+0000","last_undegraded":"2024-09-20T22:03:21.070755+0000","last_fullsized":"2024-09-20T22:03:21.070755+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:50.725534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"71'11","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071757+0000","last_change":"2024-09-20T22:02:52.279065+0000","last_active":"2024-09-20T22:03:21.071757+0000","last_peered":"2024-09-20T22:03:21.071757+0000","last_clean":"2024-09-20T22:03:21.071757+0000","last_became_active":"2024-09-20T22:02:52.278657+0000","last_became_peered":"2024-09-20T22:02:52.278657+0000","last_unstale":"2024-09-20T22:03:21.071757+0000","last_undegraded":"2024-09-20T22:03:21.071757+0000","last_fullsized":"2024-09-20T22:03:21.071757+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.405988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072426+0000","last_change":"2024-09-20T22:02:54.289401+0000","last_active":"2024-09-20T22:03:21.072426+0000","last_peered":"2024-09-20T22:03:21.072426+0000","last_clean":"2024-09-20T22:03:21.072426+0000","last_became_active":"2024-09-20T22:02:54.289078+0000","last_became_peered":"2024-09-20T22:02:54.289078+0000","last_unstale":"2024-09-20T22:03:21.072426+0000","last_undegraded":"2024-09-20T22:03:21.072426+0000","last_fullsized":"2024-09-20T22:03:21.072426+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.343711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135883,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712640,"statfs":{"total":95995035648,"available":95961743360,"internally_reserved":0,"allocated":1355776,"data_stored":923901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430223,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32492,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712660,"statfs":{"total":95995035648,"available":95961763840,"internally_reserved":0,"allocated":1339392,"data_stored":923809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724563,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27756,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717396,"statfs":{"total":95995035648,"available":95966613504,"internally_reserved":0,"allocated":745472,"data_stored":333787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986203,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31944,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713208,"statfs":{"total":95995035648,"available":95962324992,"internally_reserved":0,"allocated":770048,"data_stored":333476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574890,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":802816,"data_stored":334220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149728,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":786432,"data_stored":334768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411365,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27748,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717404,"statfs":{"total":95995035648,"available":95966621696,"internally_reserved":0,"allocated":733184,"data_stored":332583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738416,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32520,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712632,"statfs":{"total":95995035648,"available":95961735168,"internally_reserved":0,"allocated":1359872,"data_stored":924033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:03:31.739 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:03:32.009 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:32.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:31 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:03:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:31 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:31 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:31 smithi067 ceph-mon[33511]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:03:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:31 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:31 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:32.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:31 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:32.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:31 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:32.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:31 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:32.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:31 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:32.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:31 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:32.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:31 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:03:32.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:31 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:03:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[29694]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:03:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:32.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:32.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:32.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:32.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:32.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:03:32.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:03:32.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:32.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:32.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[40290]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:03:32.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:32.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:32.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:32.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:32.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:32.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:03:32.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:32.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:03:32.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:31 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:03:32.741 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:03:32.742 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:03:33.413 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[40290]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-20T22:03:33.413 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:03:33.413 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[40290]: Adjusting osd_memory_target on smithi060 to 3761M 2024-09-20T22:03:33.413 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[40290]: Updating smithi060:/etc/ceph/ceph.conf 2024-09-20T22:03:33.413 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[40290]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-20T22:03:33.414 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[40290]: Updating smithi060:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T22:03:33.414 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[40290]: Updating smithi067:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T22:03:33.414 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[40290]: Updating smithi060:/etc/ceph/ceph.client.admin.keyring 2024-09-20T22:03:33.414 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[40290]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-09-20T22:03:33.414 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:33.414 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:33.414 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:33.414 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:33.414 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:33.414 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[29694]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-20T22:03:33.414 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:03:33.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[29694]: Adjusting osd_memory_target on smithi060 to 3761M 2024-09-20T22:03:33.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[29694]: Updating smithi060:/etc/ceph/ceph.conf 2024-09-20T22:03:33.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[29694]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-20T22:03:33.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[29694]: Updating smithi060:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T22:03:33.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[29694]: Updating smithi067:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T22:03:33.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[29694]: Updating smithi060:/etc/ceph/ceph.client.admin.keyring 2024-09-20T22:03:33.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[29694]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-09-20T22:03:33.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:33.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:33.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:33.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:33.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:33 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:33.440 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2024-09-20T22:03:31.534215+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":968,"num_read_kb":814,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":601,"ondisk_log_size":601,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":244748,"kb_used_data":7708,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749716468,"statfs":{"total":767960285184,"available":767709663232,"internally_reserved":0,"allocated":7892992,"data_stored":4440577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":106,"num_read_kb":216,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.994572"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072554+0000","last_change":"2024-09-20T22:02:50.111666+0000","last_active":"2024-09-20T22:03:21.072554+0000","last_peered":"2024-09-20T22:03:21.072554+0000","last_clean":"2024-09-20T22:03:21.072554+0000","last_became_active":"2024-09-20T22:02:50.109930+0000","last_became_peered":"2024-09-20T22:02:50.109930+0000","last_unstale":"2024-09-20T22:03:21.072554+0000","last_undegraded":"2024-09-20T22:03:21.072554+0000","last_fullsized":"2024-09-20T22:03:21.072554+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.513745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"71'9","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.534937+0000","last_change":"2024-09-20T22:02:52.277916+0000","last_active":"2024-09-20T22:03:20.534937+0000","last_peered":"2024-09-20T22:03:20.534937+0000","last_clean":"2024-09-20T22:03:20.534937+0000","last_became_active":"2024-09-20T22:02:52.277616+0000","last_became_peered":"2024-09-20T22:02:52.277616+0000","last_unstale":"2024-09-20T22:03:20.534937+0000","last_undegraded":"2024-09-20T22:03:20.534937+0000","last_fullsized":"2024-09-20T22:03:20.534937+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.412488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535985+0000","last_change":"2024-09-20T22:02:54.287938+0000","last_active":"2024-09-20T22:03:20.535985+0000","last_peered":"2024-09-20T22:03:20.535985+0000","last_clean":"2024-09-20T22:03:20.535985+0000","last_became_active":"2024-09-20T22:02:54.287537+0000","last_became_peered":"2024-09-20T22:02:54.287537+0000","last_unstale":"2024-09-20T22:03:20.535985+0000","last_undegraded":"2024-09-20T22:03:20.535985+0000","last_fullsized":"2024-09-20T22:03:20.535985+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.152471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535228+0000","last_change":"2024-09-20T22:02:56.297551+0000","last_active":"2024-09-20T22:03:20.535228+0000","last_peered":"2024-09-20T22:03:20.535228+0000","last_clean":"2024-09-20T22:03:20.535228+0000","last_became_active":"2024-09-20T22:02:56.297252+0000","last_became_peered":"2024-09-20T22:02:56.297252+0000","last_unstale":"2024-09-20T22:03:20.535228+0000","last_undegraded":"2024-09-20T22:03:20.535228+0000","last_fullsized":"2024-09-20T22:03:20.535228+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.977495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071617+0000","last_change":"2024-09-20T22:02:56.350387+0000","last_active":"2024-09-20T22:03:21.071617+0000","last_peered":"2024-09-20T22:03:21.071617+0000","last_clean":"2024-09-20T22:03:21.071617+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:03:21.071617+0000","last_undegraded":"2024-09-20T22:03:21.071617+0000","last_fullsized":"2024-09-20T22:03:21.071617+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.462175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072081+0000","last_change":"2024-09-20T22:02:50.107457+0000","last_active":"2024-09-20T22:03:21.072081+0000","last_peered":"2024-09-20T22:03:21.072081+0000","last_clean":"2024-09-20T22:03:21.072081+0000","last_became_active":"2024-09-20T22:02:50.107154+0000","last_became_peered":"2024-09-20T22:02:50.107154+0000","last_unstale":"2024-09-20T22:03:21.072081+0000","last_undegraded":"2024-09-20T22:03:21.072081+0000","last_fullsized":"2024-09-20T22:03:21.072081+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:11.498961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"71'15","reported_seq":54,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071355+0000","last_change":"2024-09-20T22:02:52.282478+0000","last_active":"2024-09-20T22:03:21.071355+0000","last_peered":"2024-09-20T22:03:21.071355+0000","last_clean":"2024-09-20T22:03:21.071355+0000","last_became_active":"2024-09-20T22:02:52.282101+0000","last_became_peered":"2024-09-20T22:02:52.282101+0000","last_unstale":"2024-09-20T22:03:21.071355+0000","last_undegraded":"2024-09-20T22:03:21.071355+0000","last_fullsized":"2024-09-20T22:03:21.071355+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.208571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.534947+0000","last_change":"2024-09-20T22:02:54.292666+0000","last_active":"2024-09-20T22:03:20.534947+0000","last_peered":"2024-09-20T22:03:20.534947+0000","last_clean":"2024-09-20T22:03:20.534947+0000","last_became_active":"2024-09-20T22:02:54.292343+0000","last_became_peered":"2024-09-20T22:02:54.292343+0000","last_unstale":"2024-09-20T22:03:20.534947+0000","last_undegraded":"2024-09-20T22:03:20.534947+0000","last_fullsized":"2024-09-20T22:03:20.534947+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.965688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071303+0000","last_change":"2024-09-20T22:02:50.108309+0000","last_active":"2024-09-20T22:03:21.071303+0000","last_peered":"2024-09-20T22:03:21.071303+0000","last_clean":"2024-09-20T22:03:21.071303+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:03:21.071303+0000","last_undegraded":"2024-09-20T22:03:21.071303+0000","last_fullsized":"2024-09-20T22:03:21.071303+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.039813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"71'9","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535516+0000","last_change":"2024-09-20T22:02:52.279933+0000","last_active":"2024-09-20T22:03:20.535516+0000","last_peered":"2024-09-20T22:03:20.535516+0000","last_clean":"2024-09-20T22:03:20.535516+0000","last_became_active":"2024-09-20T22:02:52.279526+0000","last_became_peered":"2024-09-20T22:02:52.279526+0000","last_unstale":"2024-09-20T22:03:20.535516+0000","last_undegraded":"2024-09-20T22:03:20.535516+0000","last_fullsized":"2024-09-20T22:03:20.535516+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.714248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070508+0000","last_change":"2024-09-20T22:02:54.293254+0000","last_active":"2024-09-20T22:03:21.070508+0000","last_peered":"2024-09-20T22:03:21.070508+0000","last_clean":"2024-09-20T22:03:21.070508+0000","last_became_active":"2024-09-20T22:02:54.292911+0000","last_became_peered":"2024-09-20T22:02:54.292911+0000","last_unstale":"2024-09-20T22:03:21.070508+0000","last_undegraded":"2024-09-20T22:03:21.070508+0000","last_fullsized":"2024-09-20T22:03:21.070508+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.908407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070886+0000","last_change":"2024-09-20T22:02:56.299059+0000","last_active":"2024-09-20T22:03:21.070886+0000","last_peered":"2024-09-20T22:03:21.070886+0000","last_clean":"2024-09-20T22:03:21.070886+0000","last_became_active":"2024-09-20T22:02:56.298660+0000","last_became_peered":"2024-09-20T22:02:56.298660+0000","last_unstale":"2024-09-20T22:03:21.070886+0000","last_undegraded":"2024-09-20T22:03:21.070886+0000","last_fullsized":"2024-09-20T22:03:21.070886+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:13.896683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070880+0000","last_change":"2024-09-20T22:02:50.106885+0000","last_active":"2024-09-20T22:03:21.070880+0000","last_peered":"2024-09-20T22:03:21.070880+0000","last_clean":"2024-09-20T22:03:21.070880+0000","last_became_active":"2024-09-20T22:02:50.106555+0000","last_became_peered":"2024-09-20T22:02:50.106555+0000","last_unstale":"2024-09-20T22:03:21.070880+0000","last_undegraded":"2024-09-20T22:03:21.070880+0000","last_fullsized":"2024-09-20T22:03:21.070880+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.204880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"71'5","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535354+0000","last_change":"2024-09-20T22:02:52.284347+0000","last_active":"2024-09-20T22:03:20.535354+0000","last_peered":"2024-09-20T22:03:20.535354+0000","last_clean":"2024-09-20T22:03:20.535354+0000","last_became_active":"2024-09-20T22:02:52.283904+0000","last_became_peered":"2024-09-20T22:02:52.283904+0000","last_unstale":"2024-09-20T22:03:20.535354+0000","last_undegraded":"2024-09-20T22:03:20.535354+0000","last_fullsized":"2024-09-20T22:03:20.535354+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:52.941046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703160+0000","last_change":"2024-09-20T22:02:54.295625+0000","last_active":"2024-09-20T22:03:20.703160+0000","last_peered":"2024-09-20T22:03:20.703160+0000","last_clean":"2024-09-20T22:03:20.703160+0000","last_became_active":"2024-09-20T22:02:54.294405+0000","last_became_peered":"2024-09-20T22:02:54.294405+0000","last_unstale":"2024-09-20T22:03:20.703160+0000","last_undegraded":"2024-09-20T22:03:20.703160+0000","last_fullsized":"2024-09-20T22:03:20.703160+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:53.468504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070421+0000","last_change":"2024-09-20T22:02:56.303701+0000","last_active":"2024-09-20T22:03:21.070421+0000","last_peered":"2024-09-20T22:03:21.070421+0000","last_clean":"2024-09-20T22:03:21.070421+0000","last_became_active":"2024-09-20T22:02:56.303338+0000","last_became_peered":"2024-09-20T22:02:56.303338+0000","last_unstale":"2024-09-20T22:03:21.070421+0000","last_undegraded":"2024-09-20T22:03:21.070421+0000","last_fullsized":"2024-09-20T22:03:21.070421+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.793299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535518+0000","last_change":"2024-09-20T22:02:56.347311+0000","last_active":"2024-09-20T22:03:20.535518+0000","last_peered":"2024-09-20T22:03:20.535518+0000","last_clean":"2024-09-20T22:03:20.535518+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:03:20.535518+0000","last_undegraded":"2024-09-20T22:03:20.535518+0000","last_fullsized":"2024-09-20T22:03:20.535518+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.297557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072043+0000","last_change":"2024-09-20T22:02:50.111842+0000","last_active":"2024-09-20T22:03:21.072043+0000","last_peered":"2024-09-20T22:03:21.072043+0000","last_clean":"2024-09-20T22:03:21.072043+0000","last_became_active":"2024-09-20T22:02:50.111517+0000","last_became_peered":"2024-09-20T22:02:50.111517+0000","last_unstale":"2024-09-20T22:03:21.072043+0000","last_undegraded":"2024-09-20T22:03:21.072043+0000","last_fullsized":"2024-09-20T22:03:21.072043+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.747690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"71'15","reported_seq":54,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071406+0000","last_change":"2024-09-20T22:02:52.283078+0000","last_active":"2024-09-20T22:03:21.071406+0000","last_peered":"2024-09-20T22:03:21.071406+0000","last_clean":"2024-09-20T22:03:21.071406+0000","last_became_active":"2024-09-20T22:02:52.282768+0000","last_became_peered":"2024-09-20T22:02:52.282768+0000","last_unstale":"2024-09-20T22:03:21.071406+0000","last_undegraded":"2024-09-20T22:03:21.071406+0000","last_fullsized":"2024-09-20T22:03:21.071406+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:52.589023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.536861+0000","last_change":"2024-09-20T22:02:54.292892+0000","last_active":"2024-09-20T22:03:20.536861+0000","last_peered":"2024-09-20T22:03:20.536861+0000","last_clean":"2024-09-20T22:03:20.536861+0000","last_became_active":"2024-09-20T22:02:54.290510+0000","last_became_peered":"2024-09-20T22:02:54.290510+0000","last_unstale":"2024-09-20T22:03:20.536861+0000","last_undegraded":"2024-09-20T22:03:20.536861+0000","last_fullsized":"2024-09-20T22:03:20.536861+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.173080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072228+0000","last_change":"2024-09-20T22:02:56.347951+0000","last_active":"2024-09-20T22:03:21.072228+0000","last_peered":"2024-09-20T22:03:21.072228+0000","last_clean":"2024-09-20T22:03:21.072228+0000","last_became_active":"2024-09-20T22:02:56.347561+0000","last_became_peered":"2024-09-20T22:02:56.347561+0000","last_unstale":"2024-09-20T22:03:21.072228+0000","last_undegraded":"2024-09-20T22:03:21.072228+0000","last_fullsized":"2024-09-20T22:03:21.072228+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.255072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535677+0000","last_change":"2024-09-20T22:02:50.108909+0000","last_active":"2024-09-20T22:03:20.535677+0000","last_peered":"2024-09-20T22:03:20.535677+0000","last_clean":"2024-09-20T22:03:20.535677+0000","last_became_active":"2024-09-20T22:02:50.107189+0000","last_became_peered":"2024-09-20T22:02:50.107189+0000","last_unstale":"2024-09-20T22:03:20.535677+0000","last_undegraded":"2024-09-20T22:03:20.535677+0000","last_fullsized":"2024-09-20T22:03:20.535677+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.636681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"71'12","reported_seq":52,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070801+0000","last_change":"2024-09-20T22:02:52.284201+0000","last_active":"2024-09-20T22:03:21.070801+0000","last_peered":"2024-09-20T22:03:21.070801+0000","last_clean":"2024-09-20T22:03:21.070801+0000","last_became_active":"2024-09-20T22:02:52.283843+0000","last_became_peered":"2024-09-20T22:02:52.283843+0000","last_unstale":"2024-09-20T22:03:21.070801+0000","last_undegraded":"2024-09-20T22:03:21.070801+0000","last_fullsized":"2024-09-20T22:03:21.070801+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.131718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535266+0000","last_change":"2024-09-20T22:02:54.293157+0000","last_active":"2024-09-20T22:03:20.535266+0000","last_peered":"2024-09-20T22:03:20.535266+0000","last_clean":"2024-09-20T22:03:20.535266+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:03:20.535266+0000","last_undegraded":"2024-09-20T22:03:20.535266+0000","last_fullsized":"2024-09-20T22:03:20.535266+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.203423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"71'1","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703332+0000","last_change":"2024-09-20T22:02:56.301720+0000","last_active":"2024-09-20T22:03:20.703332+0000","last_peered":"2024-09-20T22:03:20.703332+0000","last_clean":"2024-09-20T22:03:20.703332+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:03:20.703332+0000","last_undegraded":"2024-09-20T22:03:20.703332+0000","last_fullsized":"2024-09-20T22:03:20.703332+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.341799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"64'1","reported_seq":34,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537040+0000","last_change":"2024-09-20T22:02:50.113259+0000","last_active":"2024-09-20T22:03:20.537040+0000","last_peered":"2024-09-20T22:03:20.537040+0000","last_clean":"2024-09-20T22:03:20.537040+0000","last_became_active":"2024-09-20T22:02:50.112862+0000","last_became_peered":"2024-09-20T22:02:50.112862+0000","last_unstale":"2024-09-20T22:03:20.537040+0000","last_undegraded":"2024-09-20T22:03:20.537040+0000","last_fullsized":"2024-09-20T22:03:20.537040+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.645807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"71'10","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072457+0000","last_change":"2024-09-20T22:02:52.482300+0000","last_active":"2024-09-20T22:03:21.072457+0000","last_peered":"2024-09-20T22:03:21.072457+0000","last_clean":"2024-09-20T22:03:21.072457+0000","last_became_active":"2024-09-20T22:02:52.481980+0000","last_became_peered":"2024-09-20T22:02:52.481980+0000","last_unstale":"2024-09-20T22:03:21.072457+0000","last_undegraded":"2024-09-20T22:03:21.072457+0000","last_fullsized":"2024-09-20T22:03:21.072457+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.082587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"71'8","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:25.171297+0000","last_change":"2024-09-20T22:02:54.297973+0000","last_active":"2024-09-20T22:03:25.171297+0000","last_peered":"2024-09-20T22:03:25.171297+0000","last_clean":"2024-09-20T22:03:25.171297+0000","last_became_active":"2024-09-20T22:02:54.297657+0000","last_became_peered":"2024-09-20T22:02:54.297657+0000","last_unstale":"2024-09-20T22:03:25.171297+0000","last_undegraded":"2024-09-20T22:03:25.171297+0000","last_fullsized":"2024-09-20T22:03:25.171297+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:20.633207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"71'15","reported_seq":54,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537626+0000","last_change":"2024-09-20T22:02:52.284890+0000","last_active":"2024-09-20T22:03:20.537626+0000","last_peered":"2024-09-20T22:03:20.537626+0000","last_clean":"2024-09-20T22:03:20.537626+0000","last_became_active":"2024-09-20T22:02:52.284027+0000","last_became_peered":"2024-09-20T22:02:52.284027+0000","last_unstale":"2024-09-20T22:03:20.537626+0000","last_undegraded":"2024-09-20T22:03:20.537626+0000","last_fullsized":"2024-09-20T22:03:20.537626+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:54.544010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070835+0000","last_change":"2024-09-20T22:02:50.108849+0000","last_active":"2024-09-20T22:03:21.070835+0000","last_peered":"2024-09-20T22:03:21.070835+0000","last_clean":"2024-09-20T22:03:21.070835+0000","last_became_active":"2024-09-20T22:02:50.108508+0000","last_became_peered":"2024-09-20T22:02:50.108508+0000","last_unstale":"2024-09-20T22:03:21.070835+0000","last_undegraded":"2024-09-20T22:03:21.070835+0000","last_fullsized":"2024-09-20T22:03:21.070835+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.430379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"71'8","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:25.171708+0000","last_change":"2024-09-20T22:02:54.295408+0000","last_active":"2024-09-20T22:03:25.171708+0000","last_peered":"2024-09-20T22:03:25.171708+0000","last_clean":"2024-09-20T22:03:25.171708+0000","last_became_active":"2024-09-20T22:02:54.294917+0000","last_became_peered":"2024-09-20T22:02:54.294917+0000","last_unstale":"2024-09-20T22:03:25.171708+0000","last_undegraded":"2024-09-20T22:03:25.171708+0000","last_fullsized":"2024-09-20T22:03:25.171708+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.541024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071048+0000","last_change":"2024-09-20T22:02:56.296725+0000","last_active":"2024-09-20T22:03:21.071048+0000","last_peered":"2024-09-20T22:03:21.071048+0000","last_clean":"2024-09-20T22:03:21.071048+0000","last_became_active":"2024-09-20T22:02:56.296436+0000","last_became_peered":"2024-09-20T22:02:56.296436+0000","last_unstale":"2024-09-20T22:03:21.071048+0000","last_undegraded":"2024-09-20T22:03:21.071048+0000","last_fullsized":"2024-09-20T22:03:21.071048+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:19.201794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"71'18","reported_seq":61,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070609+0000","last_change":"2024-09-20T22:02:52.482408+0000","last_active":"2024-09-20T22:03:21.070609+0000","last_peered":"2024-09-20T22:03:21.070609+0000","last_clean":"2024-09-20T22:03:21.070609+0000","last_became_active":"2024-09-20T22:02:52.482119+0000","last_became_peered":"2024-09-20T22:02:52.482119+0000","last_unstale":"2024-09-20T22:03:21.070609+0000","last_undegraded":"2024-09-20T22:03:21.070609+0000","last_fullsized":"2024-09-20T22:03:21.070609+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.477571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072419+0000","last_change":"2024-09-20T22:02:50.113398+0000","last_active":"2024-09-20T22:03:21.072419+0000","last_peered":"2024-09-20T22:03:21.072419+0000","last_clean":"2024-09-20T22:03:21.072419+0000","last_became_active":"2024-09-20T22:02:50.113076+0000","last_became_peered":"2024-09-20T22:02:50.113076+0000","last_unstale":"2024-09-20T22:03:21.072419+0000","last_undegraded":"2024-09-20T22:03:21.072419+0000","last_fullsized":"2024-09-20T22:03:21.072419+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.714101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535027+0000","last_change":"2024-09-20T22:02:54.288282+0000","last_active":"2024-09-20T22:03:20.535027+0000","last_peered":"2024-09-20T22:03:20.535027+0000","last_clean":"2024-09-20T22:03:20.535027+0000","last_became_active":"2024-09-20T22:02:54.288007+0000","last_became_peered":"2024-09-20T22:02:54.288007+0000","last_unstale":"2024-09-20T22:03:20.535027+0000","last_undegraded":"2024-09-20T22:03:20.535027+0000","last_fullsized":"2024-09-20T22:03:20.535027+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.737144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535412+0000","last_change":"2024-09-20T22:02:56.303051+0000","last_active":"2024-09-20T22:03:20.535412+0000","last_peered":"2024-09-20T22:03:20.535412+0000","last_clean":"2024-09-20T22:03:20.535412+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:03:20.535412+0000","last_undegraded":"2024-09-20T22:03:20.535412+0000","last_fullsized":"2024-09-20T22:03:20.535412+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.012094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"71'14","reported_seq":50,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535136+0000","last_change":"2024-09-20T22:02:52.273553+0000","last_active":"2024-09-20T22:03:20.535136+0000","last_peered":"2024-09-20T22:03:20.535136+0000","last_clean":"2024-09-20T22:03:20.535136+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:03:20.535136+0000","last_undegraded":"2024-09-20T22:03:20.535136+0000","last_fullsized":"2024-09-20T22:03:20.535136+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:56.047961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":34,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072627+0000","last_change":"2024-09-20T22:02:50.110546+0000","last_active":"2024-09-20T22:03:21.072627+0000","last_peered":"2024-09-20T22:03:21.072627+0000","last_clean":"2024-09-20T22:03:21.072627+0000","last_became_active":"2024-09-20T22:02:50.110195+0000","last_became_peered":"2024-09-20T22:02:50.110195+0000","last_unstale":"2024-09-20T22:03:21.072627+0000","last_undegraded":"2024-09-20T22:03:21.072627+0000","last_fullsized":"2024-09-20T22:03:21.072627+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.867716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"71'8","reported_seq":42,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:25.171185+0000","last_change":"2024-09-20T22:02:54.295841+0000","last_active":"2024-09-20T22:03:25.171185+0000","last_peered":"2024-09-20T22:03:25.171185+0000","last_clean":"2024-09-20T22:03:25.171185+0000","last_became_active":"2024-09-20T22:02:54.295553+0000","last_became_peered":"2024-09-20T22:02:54.295553+0000","last_unstale":"2024-09-20T22:03:25.171185+0000","last_undegraded":"2024-09-20T22:03:25.171185+0000","last_fullsized":"2024-09-20T22:03:25.171185+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.625795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703312+0000","last_change":"2024-09-20T22:02:56.349864+0000","last_active":"2024-09-20T22:03:20.703312+0000","last_peered":"2024-09-20T22:03:20.703312+0000","last_clean":"2024-09-20T22:03:20.703312+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:03:20.703312+0000","last_undegraded":"2024-09-20T22:03:20.703312+0000","last_fullsized":"2024-09-20T22:03:20.703312+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:57.678673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"71'10","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537327+0000","last_change":"2024-09-20T22:02:52.276282+0000","last_active":"2024-09-20T22:03:20.537327+0000","last_peered":"2024-09-20T22:03:20.537327+0000","last_clean":"2024-09-20T22:03:20.537327+0000","last_became_active":"2024-09-20T22:02:52.275958+0000","last_became_peered":"2024-09-20T22:02:52.275958+0000","last_unstale":"2024-09-20T22:03:20.537327+0000","last_undegraded":"2024-09-20T22:03:20.537327+0000","last_fullsized":"2024-09-20T22:03:20.537327+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.846987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071357+0000","last_change":"2024-09-20T22:02:50.114476+0000","last_active":"2024-09-20T22:03:21.071357+0000","last_peered":"2024-09-20T22:03:21.071357+0000","last_clean":"2024-09-20T22:03:21.071357+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:03:21.071357+0000","last_undegraded":"2024-09-20T22:03:21.071357+0000","last_fullsized":"2024-09-20T22:03:21.071357+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:51.340463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"71'8","reported_seq":42,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:25.171149+0000","last_change":"2024-09-20T22:02:54.293220+0000","last_active":"2024-09-20T22:03:25.171149+0000","last_peered":"2024-09-20T22:03:25.171149+0000","last_clean":"2024-09-20T22:03:25.171149+0000","last_became_active":"2024-09-20T22:02:54.292819+0000","last_became_peered":"2024-09-20T22:02:54.292819+0000","last_unstale":"2024-09-20T22:03:25.171149+0000","last_undegraded":"2024-09-20T22:03:25.171149+0000","last_fullsized":"2024-09-20T22:03:25.171149+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:59.827983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071031+0000","last_change":"2024-09-20T22:02:56.304595+0000","last_active":"2024-09-20T22:03:21.071031+0000","last_peered":"2024-09-20T22:03:21.071031+0000","last_clean":"2024-09-20T22:03:21.071031+0000","last_became_active":"2024-09-20T22:02:56.304274+0000","last_became_peered":"2024-09-20T22:02:56.304274+0000","last_unstale":"2024-09-20T22:03:21.071031+0000","last_undegraded":"2024-09-20T22:03:21.071031+0000","last_fullsized":"2024-09-20T22:03:21.071031+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.850583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"71'19","reported_seq":65,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071750+0000","last_change":"2024-09-20T22:02:52.480446+0000","last_active":"2024-09-20T22:03:21.071750+0000","last_peered":"2024-09-20T22:03:21.071750+0000","last_clean":"2024-09-20T22:03:21.071750+0000","last_became_active":"2024-09-20T22:02:52.480077+0000","last_became_peered":"2024-09-20T22:02:52.480077+0000","last_unstale":"2024-09-20T22:03:21.071750+0000","last_undegraded":"2024-09-20T22:03:21.071750+0000","last_fullsized":"2024-09-20T22:03:21.071750+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.830883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537868+0000","last_change":"2024-09-20T22:02:50.104649+0000","last_active":"2024-09-20T22:03:20.537868+0000","last_peered":"2024-09-20T22:03:20.537868+0000","last_clean":"2024-09-20T22:03:20.537868+0000","last_became_active":"2024-09-20T22:02:50.104345+0000","last_became_peered":"2024-09-20T22:02:50.104345+0000","last_unstale":"2024-09-20T22:03:20.537868+0000","last_undegraded":"2024-09-20T22:03:20.537868+0000","last_fullsized":"2024-09-20T22:03:20.537868+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.766875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070730+0000","last_change":"2024-09-20T22:02:54.291638+0000","last_active":"2024-09-20T22:03:21.070730+0000","last_peered":"2024-09-20T22:03:21.070730+0000","last_clean":"2024-09-20T22:03:21.070730+0000","last_became_active":"2024-09-20T22:02:54.290787+0000","last_became_peered":"2024-09-20T22:02:54.290787+0000","last_unstale":"2024-09-20T22:03:21.070730+0000","last_undegraded":"2024-09-20T22:03:21.070730+0000","last_fullsized":"2024-09-20T22:03:21.070730+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.996942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535226+0000","last_change":"2024-09-20T22:02:56.350020+0000","last_active":"2024-09-20T22:03:20.535226+0000","last_peered":"2024-09-20T22:03:20.535226+0000","last_clean":"2024-09-20T22:03:20.535226+0000","last_became_active":"2024-09-20T22:02:56.349581+0000","last_became_peered":"2024-09-20T22:02:56.349581+0000","last_unstale":"2024-09-20T22:03:20.535226+0000","last_undegraded":"2024-09-20T22:03:20.535226+0000","last_fullsized":"2024-09-20T22:03:20.535226+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.722790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"71'28","reported_seq":86,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:25.171479+0000","last_change":"2024-09-20T22:02:52.285811+0000","last_active":"2024-09-20T22:03:25.171479+0000","last_peered":"2024-09-20T22:03:25.171479+0000","last_clean":"2024-09-20T22:03:25.171479+0000","last_became_active":"2024-09-20T22:02:52.285341+0000","last_became_peered":"2024-09-20T22:02:52.285341+0000","last_unstale":"2024-09-20T22:03:25.171479+0000","last_undegraded":"2024-09-20T22:03:25.171479+0000","last_fullsized":"2024-09-20T22:03:25.171479+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.491983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.536083+0000","last_change":"2024-09-20T22:02:50.110040+0000","last_active":"2024-09-20T22:03:20.536083+0000","last_peered":"2024-09-20T22:03:20.536083+0000","last_clean":"2024-09-20T22:03:20.536083+0000","last_became_active":"2024-09-20T22:02:50.108457+0000","last_became_peered":"2024-09-20T22:02:50.108457+0000","last_unstale":"2024-09-20T22:03:20.536083+0000","last_undegraded":"2024-09-20T22:03:20.536083+0000","last_fullsized":"2024-09-20T22:03:20.536083+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.280284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"66'2","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071023+0000","last_change":"2024-09-20T22:02:52.273740+0000","last_active":"2024-09-20T22:03:21.071023+0000","last_peered":"2024-09-20T22:03:21.071023+0000","last_clean":"2024-09-20T22:03:21.071023+0000","last_became_active":"2024-09-20T22:02:49.096453+0000","last_became_peered":"2024-09-20T22:02:49.096453+0000","last_unstale":"2024-09-20T22:03:21.071023+0000","last_undegraded":"2024-09-20T22:03:21.071023+0000","last_fullsized":"2024-09-20T22:03:21.071023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.811834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072280+0000","last_change":"2024-09-20T22:02:54.295696+0000","last_active":"2024-09-20T22:03:21.072280+0000","last_peered":"2024-09-20T22:03:21.072280+0000","last_clean":"2024-09-20T22:03:21.072280+0000","last_became_active":"2024-09-20T22:02:54.295401+0000","last_became_peered":"2024-09-20T22:02:54.295401+0000","last_unstale":"2024-09-20T22:03:21.072280+0000","last_undegraded":"2024-09-20T22:03:21.072280+0000","last_fullsized":"2024-09-20T22:03:21.072280+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.933605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071815+0000","last_change":"2024-09-20T22:02:56.306053+0000","last_active":"2024-09-20T22:03:21.071815+0000","last_peered":"2024-09-20T22:03:21.071815+0000","last_clean":"2024-09-20T22:03:21.071815+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:03:21.071815+0000","last_undegraded":"2024-09-20T22:03:21.071815+0000","last_fullsized":"2024-09-20T22:03:21.071815+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.682717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"71'13","reported_seq":56,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537737+0000","last_change":"2024-09-20T22:02:52.280309+0000","last_active":"2024-09-20T22:03:20.537737+0000","last_peered":"2024-09-20T22:03:20.537737+0000","last_clean":"2024-09-20T22:03:20.537737+0000","last_became_active":"2024-09-20T22:02:52.280005+0000","last_became_peered":"2024-09-20T22:02:52.280005+0000","last_unstale":"2024-09-20T22:03:20.537737+0000","last_undegraded":"2024-09-20T22:03:20.537737+0000","last_fullsized":"2024-09-20T22:03:20.537737+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.190808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535654+0000","last_change":"2024-09-20T22:02:50.106023+0000","last_active":"2024-09-20T22:03:20.535654+0000","last_peered":"2024-09-20T22:03:20.535654+0000","last_clean":"2024-09-20T22:03:20.535654+0000","last_became_active":"2024-09-20T22:02:50.105633+0000","last_became_peered":"2024-09-20T22:02:50.105633+0000","last_unstale":"2024-09-20T22:03:20.535654+0000","last_undegraded":"2024-09-20T22:03:20.535654+0000","last_fullsized":"2024-09-20T22:03:20.535654+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.449401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"64'1","reported_seq":42,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071025+0000","last_change":"2024-09-20T22:02:52.270463+0000","last_active":"2024-09-20T22:03:21.071025+0000","last_peered":"2024-09-20T22:03:21.071025+0000","last_clean":"2024-09-20T22:03:21.071025+0000","last_became_active":"2024-09-20T22:02:49.099508+0000","last_became_peered":"2024-09-20T22:02:49.099508+0000","last_unstale":"2024-09-20T22:03:21.071025+0000","last_undegraded":"2024-09-20T22:03:21.071025+0000","last_fullsized":"2024-09-20T22:03:21.071025+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.955566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070945+0000","last_change":"2024-09-20T22:02:54.292450+0000","last_active":"2024-09-20T22:03:21.070945+0000","last_peered":"2024-09-20T22:03:21.070945+0000","last_clean":"2024-09-20T22:03:21.070945+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:03:21.070945+0000","last_undegraded":"2024-09-20T22:03:21.070945+0000","last_fullsized":"2024-09-20T22:03:21.070945+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.941012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703517+0000","last_change":"2024-09-20T22:02:56.349789+0000","last_active":"2024-09-20T22:03:20.703517+0000","last_peered":"2024-09-20T22:03:20.703517+0000","last_clean":"2024-09-20T22:03:20.703517+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:03:20.703517+0000","last_undegraded":"2024-09-20T22:03:20.703517+0000","last_fullsized":"2024-09-20T22:03:20.703517+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.060034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"71'12","reported_seq":47,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072508+0000","last_change":"2024-09-20T22:02:52.277878+0000","last_active":"2024-09-20T22:03:21.072508+0000","last_peered":"2024-09-20T22:03:21.072508+0000","last_clean":"2024-09-20T22:03:21.072508+0000","last_became_active":"2024-09-20T22:02:52.277548+0000","last_became_peered":"2024-09-20T22:02:52.277548+0000","last_unstale":"2024-09-20T22:03:21.072508+0000","last_undegraded":"2024-09-20T22:03:21.072508+0000","last_fullsized":"2024-09-20T22:03:21.072508+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.295996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070632+0000","last_change":"2024-09-20T22:02:50.110740+0000","last_active":"2024-09-20T22:03:21.070632+0000","last_peered":"2024-09-20T22:03:21.070632+0000","last_clean":"2024-09-20T22:03:21.070632+0000","last_became_active":"2024-09-20T22:02:50.110448+0000","last_became_peered":"2024-09-20T22:02:50.110448+0000","last_unstale":"2024-09-20T22:03:21.070632+0000","last_undegraded":"2024-09-20T22:03:21.070632+0000","last_fullsized":"2024-09-20T22:03:21.070632+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.942941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":78,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:28.707039+0000","last_change":"2024-09-20T22:02:52.480790+0000","last_active":"2024-09-20T22:03:28.707039+0000","last_peered":"2024-09-20T22:03:28.707039+0000","last_clean":"2024-09-20T22:03:28.707039+0000","last_became_active":"2024-09-20T22:02:49.098777+0000","last_became_peered":"2024-09-20T22:02:49.098777+0000","last_unstale":"2024-09-20T22:03:28.707039+0000","last_undegraded":"2024-09-20T22:03:28.707039+0000","last_fullsized":"2024-09-20T22:03:28.707039+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.872201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":34,"num_read_kb":29,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070884+0000","last_change":"2024-09-20T22:02:54.294806+0000","last_active":"2024-09-20T22:03:21.070884+0000","last_peered":"2024-09-20T22:03:21.070884+0000","last_clean":"2024-09-20T22:03:21.070884+0000","last_became_active":"2024-09-20T22:02:54.294495+0000","last_became_peered":"2024-09-20T22:02:54.294495+0000","last_unstale":"2024-09-20T22:03:21.070884+0000","last_undegraded":"2024-09-20T22:03:21.070884+0000","last_fullsized":"2024-09-20T22:03:21.070884+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:23.237482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537571+0000","last_change":"2024-09-20T22:02:56.308897+0000","last_active":"2024-09-20T22:03:20.537571+0000","last_peered":"2024-09-20T22:03:20.537571+0000","last_clean":"2024-09-20T22:03:20.537571+0000","last_became_active":"2024-09-20T22:02:56.308574+0000","last_became_peered":"2024-09-20T22:02:56.308574+0000","last_unstale":"2024-09-20T22:03:20.537571+0000","last_undegraded":"2024-09-20T22:03:20.537571+0000","last_fullsized":"2024-09-20T22:03:20.537571+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.064046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"71'16","reported_seq":60,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:25.171364+0000","last_change":"2024-09-20T22:02:52.482708+0000","last_active":"2024-09-20T22:03:25.171364+0000","last_peered":"2024-09-20T22:03:25.171364+0000","last_clean":"2024-09-20T22:03:25.171364+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:03:25.171364+0000","last_undegraded":"2024-09-20T22:03:25.171364+0000","last_fullsized":"2024-09-20T22:03:25.171364+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.243856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071690+0000","last_change":"2024-09-20T22:02:50.106319+0000","last_active":"2024-09-20T22:03:21.071690+0000","last_peered":"2024-09-20T22:03:21.071690+0000","last_clean":"2024-09-20T22:03:21.071690+0000","last_became_active":"2024-09-20T22:02:50.106011+0000","last_became_peered":"2024-09-20T22:02:50.106011+0000","last_unstale":"2024-09-20T22:03:21.071690+0000","last_undegraded":"2024-09-20T22:03:21.071690+0000","last_fullsized":"2024-09-20T22:03:21.071690+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.288394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":288,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.599816+0000","last_change":"2024-09-20T22:02:55.903219+0000","last_active":"2024-09-20T22:03:21.599816+0000","last_peered":"2024-09-20T22:03:21.599816+0000","last_clean":"2024-09-20T22:03:21.599816+0000","last_became_active":"2024-09-20T22:02:45.068482+0000","last_became_peered":"2024-09-20T22:02:45.068482+0000","last_unstale":"2024-09-20T22:03:21.599816+0000","last_undegraded":"2024-09-20T22:03:21.599816+0000","last_fullsized":"2024-09-20T22:03:21.599816+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'117","last_scrub_stamp":"2024-09-20T22:02:55.903139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.903139+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.843650+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703457+0000","last_change":"2024-09-20T22:02:54.298298+0000","last_active":"2024-09-20T22:03:20.703457+0000","last_peered":"2024-09-20T22:03:20.703457+0000","last_clean":"2024-09-20T22:03:20.703457+0000","last_became_active":"2024-09-20T22:02:54.298021+0000","last_became_peered":"2024-09-20T22:02:54.298021+0000","last_unstale":"2024-09-20T22:03:20.703457+0000","last_undegraded":"2024-09-20T22:03:20.703457+0000","last_fullsized":"2024-09-20T22:03:20.703457+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:54.157709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070644+0000","last_change":"2024-09-20T22:02:56.308039+0000","last_active":"2024-09-20T22:03:21.070644+0000","last_peered":"2024-09-20T22:03:21.070644+0000","last_clean":"2024-09-20T22:03:21.070644+0000","last_became_active":"2024-09-20T22:02:56.307749+0000","last_became_peered":"2024-09-20T22:02:56.307749+0000","last_unstale":"2024-09-20T22:03:21.070644+0000","last_undegraded":"2024-09-20T22:03:21.070644+0000","last_fullsized":"2024-09-20T22:03:21.070644+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.666810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"71'11","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535748+0000","last_change":"2024-09-20T22:02:52.273177+0000","last_active":"2024-09-20T22:03:20.535748+0000","last_peered":"2024-09-20T22:03:20.535748+0000","last_clean":"2024-09-20T22:03:20.535748+0000","last_became_active":"2024-09-20T22:02:52.272766+0000","last_became_peered":"2024-09-20T22:02:52.272766+0000","last_unstale":"2024-09-20T22:03:20.535748+0000","last_undegraded":"2024-09-20T22:03:20.535748+0000","last_fullsized":"2024-09-20T22:03:20.535748+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.073826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535258+0000","last_change":"2024-09-20T22:02:50.109626+0000","last_active":"2024-09-20T22:03:20.535258+0000","last_peered":"2024-09-20T22:03:20.535258+0000","last_clean":"2024-09-20T22:03:20.535258+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:03:20.535258+0000","last_undegraded":"2024-09-20T22:03:20.535258+0000","last_fullsized":"2024-09-20T22:03:20.535258+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.659206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070499+0000","last_change":"2024-09-20T22:02:54.290285+0000","last_active":"2024-09-20T22:03:21.070499+0000","last_peered":"2024-09-20T22:03:21.070499+0000","last_clean":"2024-09-20T22:03:21.070499+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:03:21.070499+0000","last_undegraded":"2024-09-20T22:03:21.070499+0000","last_fullsized":"2024-09-20T22:03:21.070499+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.665015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071605+0000","last_change":"2024-09-20T22:02:56.349707+0000","last_active":"2024-09-20T22:03:21.071605+0000","last_peered":"2024-09-20T22:03:21.071605+0000","last_clean":"2024-09-20T22:03:21.071605+0000","last_became_active":"2024-09-20T22:02:56.349376+0000","last_became_peered":"2024-09-20T22:02:56.349376+0000","last_unstale":"2024-09-20T22:03:21.071605+0000","last_undegraded":"2024-09-20T22:03:21.071605+0000","last_fullsized":"2024-09-20T22:03:21.071605+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.370704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"71'17","reported_seq":57,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535795+0000","last_change":"2024-09-20T22:02:52.274124+0000","last_active":"2024-09-20T22:03:20.535795+0000","last_peered":"2024-09-20T22:03:20.535795+0000","last_clean":"2024-09-20T22:03:20.535795+0000","last_became_active":"2024-09-20T22:02:52.273811+0000","last_became_peered":"2024-09-20T22:02:52.273811+0000","last_unstale":"2024-09-20T22:03:20.535795+0000","last_undegraded":"2024-09-20T22:03:20.535795+0000","last_fullsized":"2024-09-20T22:03:20.535795+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:52.793675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071073+0000","last_change":"2024-09-20T22:02:50.115379+0000","last_active":"2024-09-20T22:03:21.071073+0000","last_peered":"2024-09-20T22:03:21.071073+0000","last_clean":"2024-09-20T22:03:21.071073+0000","last_became_active":"2024-09-20T22:02:50.115015+0000","last_became_peered":"2024-09-20T22:02:50.115015+0000","last_unstale":"2024-09-20T22:03:21.071073+0000","last_undegraded":"2024-09-20T22:03:21.071073+0000","last_fullsized":"2024-09-20T22:03:21.071073+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:16.806437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537571+0000","last_change":"2024-09-20T22:02:54.291468+0000","last_active":"2024-09-20T22:03:20.537571+0000","last_peered":"2024-09-20T22:03:20.537571+0000","last_clean":"2024-09-20T22:03:20.537571+0000","last_became_active":"2024-09-20T22:02:54.291161+0000","last_became_peered":"2024-09-20T22:02:54.291161+0000","last_unstale":"2024-09-20T22:03:20.537571+0000","last_undegraded":"2024-09-20T22:03:20.537571+0000","last_fullsized":"2024-09-20T22:03:20.537571+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.806525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071370+0000","last_change":"2024-09-20T22:02:56.307571+0000","last_active":"2024-09-20T22:03:21.071370+0000","last_peered":"2024-09-20T22:03:21.071370+0000","last_clean":"2024-09-20T22:03:21.071370+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:03:21.071370+0000","last_undegraded":"2024-09-20T22:03:21.071370+0000","last_fullsized":"2024-09-20T22:03:21.071370+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:20.532247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"71'10","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535440+0000","last_change":"2024-09-20T22:02:52.284306+0000","last_active":"2024-09-20T22:03:20.535440+0000","last_peered":"2024-09-20T22:03:20.535440+0000","last_clean":"2024-09-20T22:03:20.535440+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:03:20.535440+0000","last_undegraded":"2024-09-20T22:03:20.535440+0000","last_fullsized":"2024-09-20T22:03:20.535440+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:20.155009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"64'1","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072098+0000","last_change":"2024-09-20T22:02:50.110772+0000","last_active":"2024-09-20T22:03:21.072098+0000","last_peered":"2024-09-20T22:03:21.072098+0000","last_clean":"2024-09-20T22:03:21.072098+0000","last_became_active":"2024-09-20T22:02:50.110446+0000","last_became_peered":"2024-09-20T22:02:50.110446+0000","last_unstale":"2024-09-20T22:03:21.072098+0000","last_undegraded":"2024-09-20T22:03:21.072098+0000","last_fullsized":"2024-09-20T22:03:21.072098+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.664497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"71'8","reported_seq":42,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:25.171118+0000","last_change":"2024-09-20T22:02:54.294781+0000","last_active":"2024-09-20T22:03:25.171118+0000","last_peered":"2024-09-20T22:03:25.171118+0000","last_clean":"2024-09-20T22:03:25.171118+0000","last_became_active":"2024-09-20T22:02:54.294458+0000","last_became_peered":"2024-09-20T22:02:54.294458+0000","last_unstale":"2024-09-20T22:03:25.171118+0000","last_undegraded":"2024-09-20T22:03:25.171118+0000","last_fullsized":"2024-09-20T22:03:25.171118+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.016385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535397+0000","last_change":"2024-09-20T22:02:56.300412+0000","last_active":"2024-09-20T22:03:20.535397+0000","last_peered":"2024-09-20T22:03:20.535397+0000","last_clean":"2024-09-20T22:03:20.535397+0000","last_became_active":"2024-09-20T22:02:56.300093+0000","last_became_peered":"2024-09-20T22:02:56.300093+0000","last_unstale":"2024-09-20T22:03:20.535397+0000","last_undegraded":"2024-09-20T22:03:20.535397+0000","last_fullsized":"2024-09-20T22:03:20.535397+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.204657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"71'9","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071641+0000","last_change":"2024-09-20T22:02:52.278728+0000","last_active":"2024-09-20T22:03:21.071641+0000","last_peered":"2024-09-20T22:03:21.071641+0000","last_clean":"2024-09-20T22:03:21.071641+0000","last_became_active":"2024-09-20T22:02:52.278381+0000","last_became_peered":"2024-09-20T22:02:52.278381+0000","last_unstale":"2024-09-20T22:03:21.071641+0000","last_undegraded":"2024-09-20T22:03:21.071641+0000","last_fullsized":"2024-09-20T22:03:21.071641+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:51.365937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070990+0000","last_change":"2024-09-20T22:02:50.113046+0000","last_active":"2024-09-20T22:03:21.070990+0000","last_peered":"2024-09-20T22:03:21.070990+0000","last_clean":"2024-09-20T22:03:21.070990+0000","last_became_active":"2024-09-20T22:02:50.112682+0000","last_became_peered":"2024-09-20T22:02:50.112682+0000","last_unstale":"2024-09-20T22:03:21.070990+0000","last_undegraded":"2024-09-20T22:03:21.070990+0000","last_fullsized":"2024-09-20T22:03:21.070990+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.700772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070312+0000","last_change":"2024-09-20T22:02:54.289079+0000","last_active":"2024-09-20T22:03:21.070312+0000","last_peered":"2024-09-20T22:03:21.070312+0000","last_clean":"2024-09-20T22:03:21.070312+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:03:21.070312+0000","last_undegraded":"2024-09-20T22:03:21.070312+0000","last_fullsized":"2024-09-20T22:03:21.070312+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.034123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070529+0000","last_change":"2024-09-20T22:02:56.298414+0000","last_active":"2024-09-20T22:03:21.070529+0000","last_peered":"2024-09-20T22:03:21.070529+0000","last_clean":"2024-09-20T22:03:21.070529+0000","last_became_active":"2024-09-20T22:02:56.298046+0000","last_became_peered":"2024-09-20T22:02:56.298046+0000","last_unstale":"2024-09-20T22:03:21.070529+0000","last_undegraded":"2024-09-20T22:03:21.070529+0000","last_fullsized":"2024-09-20T22:03:21.070529+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:59.723832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"71'19","reported_seq":60,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070816+0000","last_change":"2024-09-20T22:02:52.480577+0000","last_active":"2024-09-20T22:03:21.070816+0000","last_peered":"2024-09-20T22:03:21.070816+0000","last_clean":"2024-09-20T22:03:21.070816+0000","last_became_active":"2024-09-20T22:02:52.480208+0000","last_became_peered":"2024-09-20T22:02:52.480208+0000","last_unstale":"2024-09-20T22:03:21.070816+0000","last_undegraded":"2024-09-20T22:03:21.070816+0000","last_fullsized":"2024-09-20T22:03:21.070816+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.853373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703789+0000","last_change":"2024-09-20T22:02:50.109094+0000","last_active":"2024-09-20T22:03:20.703789+0000","last_peered":"2024-09-20T22:03:20.703789+0000","last_clean":"2024-09-20T22:03:20.703789+0000","last_became_active":"2024-09-20T22:02:50.108712+0000","last_became_peered":"2024-09-20T22:02:50.108712+0000","last_unstale":"2024-09-20T22:03:20.703789+0000","last_undegraded":"2024-09-20T22:03:20.703789+0000","last_fullsized":"2024-09-20T22:03:20.703789+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:08.545134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070341+0000","last_change":"2024-09-20T22:02:54.291441+0000","last_active":"2024-09-20T22:03:21.070341+0000","last_peered":"2024-09-20T22:03:21.070341+0000","last_clean":"2024-09-20T22:03:21.070341+0000","last_became_active":"2024-09-20T22:02:54.291134+0000","last_became_peered":"2024-09-20T22:02:54.291134+0000","last_unstale":"2024-09-20T22:03:21.070341+0000","last_undegraded":"2024-09-20T22:03:21.070341+0000","last_fullsized":"2024-09-20T22:03:21.070341+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.677000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703250+0000","last_change":"2024-09-20T22:02:56.308001+0000","last_active":"2024-09-20T22:03:20.703250+0000","last_peered":"2024-09-20T22:03:20.703250+0000","last_clean":"2024-09-20T22:03:20.703250+0000","last_became_active":"2024-09-20T22:02:56.307687+0000","last_became_peered":"2024-09-20T22:02:56.307687+0000","last_unstale":"2024-09-20T22:03:20.703250+0000","last_undegraded":"2024-09-20T22:03:20.703250+0000","last_fullsized":"2024-09-20T22:03:20.703250+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.249693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"71'12","reported_seq":52,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535005+0000","last_change":"2024-09-20T22:02:52.284579+0000","last_active":"2024-09-20T22:03:20.535005+0000","last_peered":"2024-09-20T22:03:20.535005+0000","last_clean":"2024-09-20T22:03:20.535005+0000","last_became_active":"2024-09-20T22:02:52.284144+0000","last_became_peered":"2024-09-20T22:02:52.284144+0000","last_unstale":"2024-09-20T22:03:20.535005+0000","last_undegraded":"2024-09-20T22:03:20.535005+0000","last_fullsized":"2024-09-20T22:03:20.535005+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.901626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703857+0000","last_change":"2024-09-20T22:02:50.111591+0000","last_active":"2024-09-20T22:03:20.703857+0000","last_peered":"2024-09-20T22:03:20.703857+0000","last_clean":"2024-09-20T22:03:20.703857+0000","last_became_active":"2024-09-20T22:02:50.111267+0000","last_became_peered":"2024-09-20T22:02:50.111267+0000","last_unstale":"2024-09-20T22:03:20.703857+0000","last_undegraded":"2024-09-20T22:03:20.703857+0000","last_fullsized":"2024-09-20T22:03:20.703857+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.038287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070129+0000","last_change":"2024-09-20T22:02:54.293804+0000","last_active":"2024-09-20T22:03:21.070129+0000","last_peered":"2024-09-20T22:03:21.070129+0000","last_clean":"2024-09-20T22:03:21.070129+0000","last_became_active":"2024-09-20T22:02:54.293377+0000","last_became_peered":"2024-09-20T22:02:54.293377+0000","last_unstale":"2024-09-20T22:03:21.070129+0000","last_undegraded":"2024-09-20T22:03:21.070129+0000","last_fullsized":"2024-09-20T22:03:21.070129+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.827756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071742+0000","last_change":"2024-09-20T22:02:56.307499+0000","last_active":"2024-09-20T22:03:21.071742+0000","last_peered":"2024-09-20T22:03:21.071742+0000","last_clean":"2024-09-20T22:03:21.071742+0000","last_became_active":"2024-09-20T22:02:56.306991+0000","last_became_peered":"2024-09-20T22:02:56.306991+0000","last_unstale":"2024-09-20T22:03:21.071742+0000","last_undegraded":"2024-09-20T22:03:21.071742+0000","last_fullsized":"2024-09-20T22:03:21.071742+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.424396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"71'15","reported_seq":54,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070130+0000","last_change":"2024-09-20T22:02:52.482395+0000","last_active":"2024-09-20T22:03:21.070130+0000","last_peered":"2024-09-20T22:03:21.070130+0000","last_clean":"2024-09-20T22:03:21.070130+0000","last_became_active":"2024-09-20T22:02:52.482008+0000","last_became_peered":"2024-09-20T22:02:52.482008+0000","last_unstale":"2024-09-20T22:03:21.070130+0000","last_undegraded":"2024-09-20T22:03:21.070130+0000","last_fullsized":"2024-09-20T22:03:21.070130+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.554179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"64'2","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703795+0000","last_change":"2024-09-20T22:02:50.115186+0000","last_active":"2024-09-20T22:03:20.703795+0000","last_peered":"2024-09-20T22:03:20.703795+0000","last_clean":"2024-09-20T22:03:20.703795+0000","last_became_active":"2024-09-20T22:02:50.114819+0000","last_became_peered":"2024-09-20T22:02:50.114819+0000","last_unstale":"2024-09-20T22:03:20.703795+0000","last_undegraded":"2024-09-20T22:03:20.703795+0000","last_fullsized":"2024-09-20T22:03:20.703795+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.801926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":42,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:25.171348+0000","last_change":"2024-09-20T22:02:54.296823+0000","last_active":"2024-09-20T22:03:25.171348+0000","last_peered":"2024-09-20T22:03:25.171348+0000","last_clean":"2024-09-20T22:03:25.171348+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:03:25.171348+0000","last_undegraded":"2024-09-20T22:03:25.171348+0000","last_fullsized":"2024-09-20T22:03:25.171348+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.496588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070689+0000","last_change":"2024-09-20T22:02:56.347283+0000","last_active":"2024-09-20T22:03:21.070689+0000","last_peered":"2024-09-20T22:03:21.070689+0000","last_clean":"2024-09-20T22:03:21.070689+0000","last_became_active":"2024-09-20T22:02:56.346990+0000","last_became_peered":"2024-09-20T22:02:56.346990+0000","last_unstale":"2024-09-20T22:03:21.070689+0000","last_undegraded":"2024-09-20T22:03:21.070689+0000","last_fullsized":"2024-09-20T22:03:21.070689+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.718016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070498+0000","last_change":"2024-09-20T22:02:50.112245+0000","last_active":"2024-09-20T22:03:21.070498+0000","last_peered":"2024-09-20T22:03:21.070498+0000","last_clean":"2024-09-20T22:03:21.070498+0000","last_became_active":"2024-09-20T22:02:50.111935+0000","last_became_peered":"2024-09-20T22:02:50.111935+0000","last_unstale":"2024-09-20T22:03:21.070498+0000","last_undegraded":"2024-09-20T22:03:21.070498+0000","last_fullsized":"2024-09-20T22:03:21.070498+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:16.998775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"71'6","reported_seq":38,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072364+0000","last_change":"2024-09-20T22:02:52.481046+0000","last_active":"2024-09-20T22:03:21.072364+0000","last_peered":"2024-09-20T22:03:21.072364+0000","last_clean":"2024-09-20T22:03:21.072364+0000","last_became_active":"2024-09-20T22:02:52.480771+0000","last_became_peered":"2024-09-20T22:02:52.480771+0000","last_unstale":"2024-09-20T22:03:21.072364+0000","last_undegraded":"2024-09-20T22:03:21.072364+0000","last_fullsized":"2024-09-20T22:03:21.072364+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.943982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070886+0000","last_change":"2024-09-20T22:02:54.294834+0000","last_active":"2024-09-20T22:03:21.070886+0000","last_peered":"2024-09-20T22:03:21.070886+0000","last_clean":"2024-09-20T22:03:21.070886+0000","last_became_active":"2024-09-20T22:02:54.294539+0000","last_became_peered":"2024-09-20T22:02:54.294539+0000","last_unstale":"2024-09-20T22:03:21.070886+0000","last_undegraded":"2024-09-20T22:03:21.070886+0000","last_fullsized":"2024-09-20T22:03:21.070886+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.523368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703485+0000","last_change":"2024-09-20T22:02:56.350065+0000","last_active":"2024-09-20T22:03:20.703485+0000","last_peered":"2024-09-20T22:03:20.703485+0000","last_clean":"2024-09-20T22:03:20.703485+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:03:20.703485+0000","last_undegraded":"2024-09-20T22:03:20.703485+0000","last_fullsized":"2024-09-20T22:03:20.703485+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:19.909723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"71'9","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072562+0000","last_change":"2024-09-20T22:02:52.482361+0000","last_active":"2024-09-20T22:03:21.072562+0000","last_peered":"2024-09-20T22:03:21.072562+0000","last_clean":"2024-09-20T22:03:21.072562+0000","last_became_active":"2024-09-20T22:02:52.481989+0000","last_became_peered":"2024-09-20T22:02:52.481989+0000","last_unstale":"2024-09-20T22:03:21.072562+0000","last_undegraded":"2024-09-20T22:03:21.072562+0000","last_fullsized":"2024-09-20T22:03:21.072562+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.401767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703355+0000","last_change":"2024-09-20T22:02:50.108943+0000","last_active":"2024-09-20T22:03:20.703355+0000","last_peered":"2024-09-20T22:03:20.703355+0000","last_clean":"2024-09-20T22:03:20.703355+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:03:20.703355+0000","last_undegraded":"2024-09-20T22:03:20.703355+0000","last_fullsized":"2024-09-20T22:03:20.703355+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.114642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072267+0000","last_change":"2024-09-20T22:02:54.298137+0000","last_active":"2024-09-20T22:03:21.072267+0000","last_peered":"2024-09-20T22:03:21.072267+0000","last_clean":"2024-09-20T22:03:21.072267+0000","last_became_active":"2024-09-20T22:02:54.297850+0000","last_became_peered":"2024-09-20T22:02:54.297850+0000","last_unstale":"2024-09-20T22:03:21.072267+0000","last_undegraded":"2024-09-20T22:03:21.072267+0000","last_fullsized":"2024-09-20T22:03:21.072267+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.705512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071088+0000","last_change":"2024-09-20T22:02:56.299306+0000","last_active":"2024-09-20T22:03:21.071088+0000","last_peered":"2024-09-20T22:03:21.071088+0000","last_clean":"2024-09-20T22:03:21.071088+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:03:21.071088+0000","last_undegraded":"2024-09-20T22:03:21.071088+0000","last_fullsized":"2024-09-20T22:03:21.071088+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:11.499111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"71'9","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071259+0000","last_change":"2024-09-20T22:02:52.482498+0000","last_active":"2024-09-20T22:03:21.071259+0000","last_peered":"2024-09-20T22:03:21.071259+0000","last_clean":"2024-09-20T22:03:21.071259+0000","last_became_active":"2024-09-20T22:02:52.482141+0000","last_became_peered":"2024-09-20T22:02:52.482141+0000","last_unstale":"2024-09-20T22:03:21.071259+0000","last_undegraded":"2024-09-20T22:03:21.071259+0000","last_fullsized":"2024-09-20T22:03:21.071259+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.093555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070666+0000","last_change":"2024-09-20T22:02:50.112533+0000","last_active":"2024-09-20T22:03:21.070666+0000","last_peered":"2024-09-20T22:03:21.070666+0000","last_clean":"2024-09-20T22:03:21.070666+0000","last_became_active":"2024-09-20T22:02:50.110075+0000","last_became_peered":"2024-09-20T22:02:50.110075+0000","last_unstale":"2024-09-20T22:03:21.070666+0000","last_undegraded":"2024-09-20T22:03:21.070666+0000","last_fullsized":"2024-09-20T22:03:21.070666+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.714513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"71'8","reported_seq":42,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:25.171545+0000","last_change":"2024-09-20T22:02:54.294653+0000","last_active":"2024-09-20T22:03:25.171545+0000","last_peered":"2024-09-20T22:03:25.171545+0000","last_clean":"2024-09-20T22:03:25.171545+0000","last_became_active":"2024-09-20T22:02:54.294381+0000","last_became_peered":"2024-09-20T22:02:54.294381+0000","last_unstale":"2024-09-20T22:03:25.171545+0000","last_undegraded":"2024-09-20T22:03:25.171545+0000","last_fullsized":"2024-09-20T22:03:25.171545+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.135221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535469+0000","last_change":"2024-09-20T22:02:56.300405+0000","last_active":"2024-09-20T22:03:20.535469+0000","last_peered":"2024-09-20T22:03:20.535469+0000","last_clean":"2024-09-20T22:03:20.535469+0000","last_became_active":"2024-09-20T22:02:56.300041+0000","last_became_peered":"2024-09-20T22:02:56.300041+0000","last_unstale":"2024-09-20T22:03:20.535469+0000","last_undegraded":"2024-09-20T22:03:20.535469+0000","last_fullsized":"2024-09-20T22:03:20.535469+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.312552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"71'10","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071846+0000","last_change":"2024-09-20T22:02:52.480909+0000","last_active":"2024-09-20T22:03:21.071846+0000","last_peered":"2024-09-20T22:03:21.071846+0000","last_clean":"2024-09-20T22:03:21.071846+0000","last_became_active":"2024-09-20T22:02:52.480591+0000","last_became_peered":"2024-09-20T22:02:52.480591+0000","last_unstale":"2024-09-20T22:03:21.071846+0000","last_undegraded":"2024-09-20T22:03:21.071846+0000","last_fullsized":"2024-09-20T22:03:21.071846+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.915698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703658+0000","last_change":"2024-09-20T22:02:50.112976+0000","last_active":"2024-09-20T22:03:20.703658+0000","last_peered":"2024-09-20T22:03:20.703658+0000","last_clean":"2024-09-20T22:03:20.703658+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:03:20.703658+0000","last_undegraded":"2024-09-20T22:03:20.703658+0000","last_fullsized":"2024-09-20T22:03:20.703658+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.057376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"71'8","reported_seq":42,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:25.171343+0000","last_change":"2024-09-20T22:02:54.292909+0000","last_active":"2024-09-20T22:03:25.171343+0000","last_peered":"2024-09-20T22:03:25.171343+0000","last_clean":"2024-09-20T22:03:25.171343+0000","last_became_active":"2024-09-20T22:02:54.289548+0000","last_became_peered":"2024-09-20T22:02:54.289548+0000","last_unstale":"2024-09-20T22:03:25.171343+0000","last_undegraded":"2024-09-20T22:03:25.171343+0000","last_fullsized":"2024-09-20T22:03:25.171343+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.478336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072333+0000","last_change":"2024-09-20T22:02:56.305690+0000","last_active":"2024-09-20T22:03:21.072333+0000","last_peered":"2024-09-20T22:03:21.072333+0000","last_clean":"2024-09-20T22:03:21.072333+0000","last_became_active":"2024-09-20T22:02:56.305383+0000","last_became_peered":"2024-09-20T22:02:56.305383+0000","last_unstale":"2024-09-20T22:03:21.072333+0000","last_undegraded":"2024-09-20T22:03:21.072333+0000","last_fullsized":"2024-09-20T22:03:21.072333+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.293958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"71'11","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535982+0000","last_change":"2024-09-20T22:02:52.277987+0000","last_active":"2024-09-20T22:03:20.535982+0000","last_peered":"2024-09-20T22:03:20.535982+0000","last_clean":"2024-09-20T22:03:20.535982+0000","last_became_active":"2024-09-20T22:02:52.277479+0000","last_became_peered":"2024-09-20T22:02:52.277479+0000","last_unstale":"2024-09-20T22:03:20.535982+0000","last_undegraded":"2024-09-20T22:03:20.535982+0000","last_fullsized":"2024-09-20T22:03:20.535982+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.222570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.535540+0000","last_change":"2024-09-20T22:02:50.109201+0000","last_active":"2024-09-20T22:03:20.535540+0000","last_peered":"2024-09-20T22:03:20.535540+0000","last_clean":"2024-09-20T22:03:20.535540+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:03:20.535540+0000","last_undegraded":"2024-09-20T22:03:20.535540+0000","last_fullsized":"2024-09-20T22:03:20.535540+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.129228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537256+0000","last_change":"2024-09-20T22:02:54.286502+0000","last_active":"2024-09-20T22:03:20.537256+0000","last_peered":"2024-09-20T22:03:20.537256+0000","last_clean":"2024-09-20T22:03:20.537256+0000","last_became_active":"2024-09-20T22:02:54.286200+0000","last_became_peered":"2024-09-20T22:02:54.286200+0000","last_unstale":"2024-09-20T22:03:20.537256+0000","last_undegraded":"2024-09-20T22:03:20.537256+0000","last_fullsized":"2024-09-20T22:03:20.537256+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:22.326441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070972+0000","last_change":"2024-09-20T22:02:56.307258+0000","last_active":"2024-09-20T22:03:21.070972+0000","last_peered":"2024-09-20T22:03:21.070972+0000","last_clean":"2024-09-20T22:03:21.070972+0000","last_became_active":"2024-09-20T22:02:56.306892+0000","last_became_peered":"2024-09-20T22:02:56.306892+0000","last_unstale":"2024-09-20T22:03:21.070972+0000","last_undegraded":"2024-09-20T22:03:21.070972+0000","last_fullsized":"2024-09-20T22:03:21.070972+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.948765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"71'9","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.537344+0000","last_change":"2024-09-20T22:02:52.280815+0000","last_active":"2024-09-20T22:03:20.537344+0000","last_peered":"2024-09-20T22:03:20.537344+0000","last_clean":"2024-09-20T22:03:20.537344+0000","last_became_active":"2024-09-20T22:02:52.280428+0000","last_became_peered":"2024-09-20T22:02:52.280428+0000","last_unstale":"2024-09-20T22:03:20.537344+0000","last_undegraded":"2024-09-20T22:03:20.537344+0000","last_fullsized":"2024-09-20T22:03:20.537344+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.548215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703689+0000","last_change":"2024-09-20T22:02:50.110105+0000","last_active":"2024-09-20T22:03:20.703689+0000","last_peered":"2024-09-20T22:03:20.703689+0000","last_clean":"2024-09-20T22:03:20.703689+0000","last_became_active":"2024-09-20T22:02:50.109803+0000","last_became_peered":"2024-09-20T22:02:50.109803+0000","last_unstale":"2024-09-20T22:03:20.703689+0000","last_undegraded":"2024-09-20T22:03:20.703689+0000","last_fullsized":"2024-09-20T22:03:20.703689+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:53.587349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071838+0000","last_change":"2024-09-20T22:02:54.298130+0000","last_active":"2024-09-20T22:03:21.071838+0000","last_peered":"2024-09-20T22:03:21.071838+0000","last_clean":"2024-09-20T22:03:21.071838+0000","last_became_active":"2024-09-20T22:02:54.297803+0000","last_became_peered":"2024-09-20T22:02:54.297803+0000","last_unstale":"2024-09-20T22:03:21.071838+0000","last_undegraded":"2024-09-20T22:03:21.071838+0000","last_fullsized":"2024-09-20T22:03:21.071838+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.520061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070749+0000","last_change":"2024-09-20T22:02:56.298228+0000","last_active":"2024-09-20T22:03:21.070749+0000","last_peered":"2024-09-20T22:03:21.070749+0000","last_clean":"2024-09-20T22:03:21.070749+0000","last_became_active":"2024-09-20T22:02:56.297826+0000","last_became_peered":"2024-09-20T22:02:56.297826+0000","last_unstale":"2024-09-20T22:03:21.070749+0000","last_undegraded":"2024-09-20T22:03:21.070749+0000","last_fullsized":"2024-09-20T22:03:21.070749+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.126658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"71'11","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071102+0000","last_change":"2024-09-20T22:02:52.480504+0000","last_active":"2024-09-20T22:03:21.071102+0000","last_peered":"2024-09-20T22:03:21.071102+0000","last_clean":"2024-09-20T22:03:21.071102+0000","last_became_active":"2024-09-20T22:02:52.480118+0000","last_became_peered":"2024-09-20T22:02:52.480118+0000","last_unstale":"2024-09-20T22:03:21.071102+0000","last_undegraded":"2024-09-20T22:03:21.071102+0000","last_fullsized":"2024-09-20T22:03:21.071102+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.023916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071093+0000","last_change":"2024-09-20T22:02:50.110808+0000","last_active":"2024-09-20T22:03:21.071093+0000","last_peered":"2024-09-20T22:03:21.071093+0000","last_clean":"2024-09-20T22:03:21.071093+0000","last_became_active":"2024-09-20T22:02:50.110421+0000","last_became_peered":"2024-09-20T22:02:50.110421+0000","last_unstale":"2024-09-20T22:03:21.071093+0000","last_undegraded":"2024-09-20T22:03:21.071093+0000","last_fullsized":"2024-09-20T22:03:21.071093+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:53.027475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703032+0000","last_change":"2024-09-20T22:02:54.293068+0000","last_active":"2024-09-20T22:03:20.703032+0000","last_peered":"2024-09-20T22:03:20.703032+0000","last_clean":"2024-09-20T22:03:20.703032+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:03:20.703032+0000","last_undegraded":"2024-09-20T22:03:20.703032+0000","last_fullsized":"2024-09-20T22:03:20.703032+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.450710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072001+0000","last_change":"2024-09-20T22:02:56.348065+0000","last_active":"2024-09-20T22:03:21.072001+0000","last_peered":"2024-09-20T22:03:21.072001+0000","last_clean":"2024-09-20T22:03:21.072001+0000","last_became_active":"2024-09-20T22:02:56.347637+0000","last_became_peered":"2024-09-20T22:02:56.347637+0000","last_unstale":"2024-09-20T22:03:21.072001+0000","last_undegraded":"2024-09-20T22:03:21.072001+0000","last_fullsized":"2024-09-20T22:03:21.072001+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:13.568557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"71'4","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070523+0000","last_change":"2024-09-20T22:02:52.286086+0000","last_active":"2024-09-20T22:03:21.070523+0000","last_peered":"2024-09-20T22:03:21.070523+0000","last_clean":"2024-09-20T22:03:21.070523+0000","last_became_active":"2024-09-20T22:02:52.285701+0000","last_became_peered":"2024-09-20T22:02:52.285701+0000","last_unstale":"2024-09-20T22:03:21.070523+0000","last_undegraded":"2024-09-20T22:03:21.070523+0000","last_fullsized":"2024-09-20T22:03:21.070523+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.474692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072586+0000","last_change":"2024-09-20T22:02:50.111749+0000","last_active":"2024-09-20T22:03:21.072586+0000","last_peered":"2024-09-20T22:03:21.072586+0000","last_clean":"2024-09-20T22:03:21.072586+0000","last_became_active":"2024-09-20T22:02:50.111142+0000","last_became_peered":"2024-09-20T22:02:50.111142+0000","last_unstale":"2024-09-20T22:03:21.072586+0000","last_undegraded":"2024-09-20T22:03:21.072586+0000","last_fullsized":"2024-09-20T22:03:21.072586+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:57.037771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070393+0000","last_change":"2024-09-20T22:02:54.290418+0000","last_active":"2024-09-20T22:03:21.070393+0000","last_peered":"2024-09-20T22:03:21.070393+0000","last_clean":"2024-09-20T22:03:21.070393+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:03:21.070393+0000","last_undegraded":"2024-09-20T22:03:21.070393+0000","last_fullsized":"2024-09-20T22:03:21.070393+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:57.209065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703153+0000","last_change":"2024-09-20T22:02:56.300559+0000","last_active":"2024-09-20T22:03:20.703153+0000","last_peered":"2024-09-20T22:03:20.703153+0000","last_clean":"2024-09-20T22:03:20.703153+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:03:20.703153+0000","last_undegraded":"2024-09-20T22:03:20.703153+0000","last_fullsized":"2024-09-20T22:03:20.703153+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.506369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.536607+0000","last_change":"2024-09-20T22:02:56.301611+0000","last_active":"2024-09-20T22:03:20.536607+0000","last_peered":"2024-09-20T22:03:20.536607+0000","last_clean":"2024-09-20T22:03:20.536607+0000","last_became_active":"2024-09-20T22:02:56.301247+0000","last_became_peered":"2024-09-20T22:02:56.301247+0000","last_unstale":"2024-09-20T22:03:20.536607+0000","last_undegraded":"2024-09-20T22:03:20.536607+0000","last_fullsized":"2024-09-20T22:03:20.536607+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.486378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070755+0000","last_change":"2024-09-20T22:02:50.114519+0000","last_active":"2024-09-20T22:03:21.070755+0000","last_peered":"2024-09-20T22:03:21.070755+0000","last_clean":"2024-09-20T22:03:21.070755+0000","last_became_active":"2024-09-20T22:02:50.114204+0000","last_became_peered":"2024-09-20T22:02:50.114204+0000","last_unstale":"2024-09-20T22:03:21.070755+0000","last_undegraded":"2024-09-20T22:03:21.070755+0000","last_fullsized":"2024-09-20T22:03:21.070755+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:50.725534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"71'11","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071757+0000","last_change":"2024-09-20T22:02:52.279065+0000","last_active":"2024-09-20T22:03:21.071757+0000","last_peered":"2024-09-20T22:03:21.071757+0000","last_clean":"2024-09-20T22:03:21.071757+0000","last_became_active":"2024-09-20T22:02:52.278657+0000","last_became_peered":"2024-09-20T22:02:52.278657+0000","last_unstale":"2024-09-20T22:03:21.071757+0000","last_undegraded":"2024-09-20T22:03:21.071757+0000","last_fullsized":"2024-09-20T22:03:21.071757+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.405988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072426+0000","last_change":"2024-09-20T22:02:54.289401+0000","last_active":"2024-09-20T22:03:21.072426+0000","last_peered":"2024-09-20T22:03:21.072426+0000","last_clean":"2024-09-20T22:03:21.072426+0000","last_became_active":"2024-09-20T22:02:54.289078+0000","last_became_peered":"2024-09-20T22:02:54.289078+0000","last_unstale":"2024-09-20T22:03:21.072426+0000","last_undegraded":"2024-09-20T22:03:21.072426+0000","last_fullsized":"2024-09-20T22:03:21.072426+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.343711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":29,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135884,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712640,"statfs":{"total":95995035648,"available":95961743360,"internally_reserved":0,"allocated":1355776,"data_stored":923901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430224,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32492,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712660,"statfs":{"total":95995035648,"available":95961763840,"internally_reserved":0,"allocated":1339392,"data_stored":923809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724564,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27756,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717396,"statfs":{"total":95995035648,"available":95966613504,"internally_reserved":0,"allocated":745472,"data_stored":333787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986204,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31944,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713208,"statfs":{"total":95995035648,"available":95962324992,"internally_reserved":0,"allocated":770048,"data_stored":333476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574891,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":802816,"data_stored":334220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149729,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":786432,"data_stored":334768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411366,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27748,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717404,"statfs":{"total":95995035648,"available":95966621696,"internally_reserved":0,"allocated":733184,"data_stored":332583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738417,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32520,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712632,"statfs":{"total":95995035648,"available":95961735168,"internally_reserved":0,"allocated":1359872,"data_stored":924033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:03:33.445 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-20T22:03:33.445 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-20T22:03:33.445 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-20T22:03:33.445 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-20T22:03:33.709 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:33.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:33 smithi067 ceph-mon[33511]: pgmap v10: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-20T22:03:33.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:33 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:03:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:33 smithi067 ceph-mon[33511]: Adjusting osd_memory_target on smithi060 to 3761M 2024-09-20T22:03:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:33 smithi067 ceph-mon[33511]: Updating smithi060:/etc/ceph/ceph.conf 2024-09-20T22:03:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:33 smithi067 ceph-mon[33511]: Updating smithi067:/etc/ceph/ceph.conf 2024-09-20T22:03:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:33 smithi067 ceph-mon[33511]: Updating smithi060:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T22:03:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:33 smithi067 ceph-mon[33511]: Updating smithi067:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-20T22:03:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:33 smithi067 ceph-mon[33511]: Updating smithi060:/etc/ceph/ceph.client.admin.keyring 2024-09-20T22:03:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:33 smithi067 ceph-mon[33511]: Updating smithi067:/etc/ceph/ceph.client.admin.keyring 2024-09-20T22:03:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:33 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:33 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:33 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:33 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:33 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:34.402 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:03:34.402 INFO:teuthology.orchestra.run.smithi060.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-20T22:03:34.402 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:34 smithi060 ceph-mon[29694]: Updating smithi060:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T22:03:34.403 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:34 smithi060 ceph-mon[29694]: Updating smithi067:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T22:03:34.403 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:34 smithi060 ceph-mon[29694]: Deploying daemon alertmanager.a on smithi060 2024-09-20T22:03:34.403 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:34 smithi060 ceph-mon[29694]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:03:34.403 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:34 smithi060 ceph-mon[40290]: Updating smithi060:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T22:03:34.403 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:34 smithi060 ceph-mon[40290]: Updating smithi067:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T22:03:34.403 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:34 smithi060 ceph-mon[40290]: Deploying daemon alertmanager.a on smithi060 2024-09-20T22:03:34.403 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:34 smithi060 ceph-mon[40290]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:03:34.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:34 smithi067 ceph-mon[33511]: Updating smithi060:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T22:03:34.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:34 smithi067 ceph-mon[33511]: Updating smithi067:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-20T22:03:34.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:34 smithi067 ceph-mon[33511]: Deploying daemon alertmanager.a on smithi060 2024-09-20T22:03:34.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:34 smithi067 ceph-mon[33511]: from='client.14730 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:03:35.165 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:03:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:03:35.193 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-20T22:03:35.194 INFO:tasks.cephadm:Setup complete, yielding 2024-09-20T22:03:35.194 INFO:teuthology.run_tasks:Running task thrashosds... 2024-09-20T22:03:35.204 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-20T22:03:35.204 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client.0': {'debug ms': 1, 'debug objecter': 20, 'debug rados': 20}, 'global': {'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd debug reject backfill probability': 0.3, 'osd delete sleep': 1, 'osd max backfills': 3, 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True, 'osd snap trim sleep': 2}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'CEPHADM_STRAY_DAEMON', 'CEPHADM_FAILED_DAEMON', 'but it is still running', 'objects unfound and apparently lost', '\\(POOL_APP_NOT_ENABLED\\)'], 'log-only-match': ['CEPHADM_'], 'sha1': '9e2ac70bb7aea67d34f0d50002344abbe7803736'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'cephadm': {'cephadm_mode': 'root'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '9e2ac70bb7aea67d34f0d50002344abbe7803736'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'squid', 'sha1': '4e45e6b490e6cbe082cdf155fea780438ec53fc7'}} 2024-09-20T22:03:35.205 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-09-20T22:03:35.205 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-09-20T22:03:35.467 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:35.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:35 smithi060 ceph-mon[29694]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-20T22:03:35.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:35 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/762042097' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-20T22:03:35.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:35 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:03:35.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:35 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:03:35.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:35 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:03:35.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:35 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:03:35.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:35 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:03:35.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:35 smithi060 ceph-mon[40290]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-20T22:03:35.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:35 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/762042097' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-20T22:03:35.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:35 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:03:35.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:35 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:03:35.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:35 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:03:35.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:35 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:03:35.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:35 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:03:35.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:35 smithi067 ceph-mon[33511]: pgmap v11: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-20T22:03:35.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:35 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/762042097' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-20T22:03:35.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:35 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:03:35.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:35 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:03:35.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:35 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:03:35.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:35 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:03:35.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:35 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:03:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:36 smithi060 ceph-mon[29694]: osdmap e76: 8 total, 8 up, 8 in 2024-09-20T22:03:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:36 smithi060 ceph-mon[29694]: osdmap e77: 8 total, 8 up, 8 in 2024-09-20T22:03:36.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:36 smithi060 ceph-mon[40290]: osdmap e76: 8 total, 8 up, 8 in 2024-09-20T22:03:36.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:36 smithi060 ceph-mon[40290]: osdmap e77: 8 total, 8 up, 8 in 2024-09-20T22:03:36.665 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:36 smithi067 ceph-mon[33511]: osdmap e76: 8 total, 8 up, 8 in 2024-09-20T22:03:36.665 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:36 smithi067 ceph-mon[33511]: osdmap e77: 8 total, 8 up, 8 in 2024-09-20T22:03:37.036 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:36 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:03:36] ENGINE Bus STOPPING 2024-09-20T22:03:37.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:37 smithi060 ceph-mon[29694]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-20T22:03:37.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:37 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' 2024-09-20T22:03:37.375 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:37 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:03:37] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-20T22:03:37.376 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:37 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:03:37] ENGINE Bus STOPPED 2024-09-20T22:03:37.376 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:37 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:03:37] ENGINE Bus STARTING 2024-09-20T22:03:37.376 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:37 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:03:37] ENGINE Serving on http://:::9283 2024-09-20T22:03:37.376 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:37 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:03:37] ENGINE Bus STARTED 2024-09-20T22:03:37.376 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:37 smithi060 ceph-mon[40290]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-20T22:03:37.376 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:37 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' 2024-09-20T22:03:37.681 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:37 smithi067 ceph-mon[33511]: pgmap v13: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 22 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-20T22:03:37.682 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:37 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' 2024-09-20T22:03:37.951 INFO:tasks.thrashosds:Beginning thrashosds... 2024-09-20T22:03:37.951 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-20T22:03:37.952 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:03:39.397 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:39.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:39 smithi060 ceph-mon[29694]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:03:39.500 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:39 smithi060 ceph-mon[40290]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:03:39.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:39 smithi067 ceph-mon[33511]: pgmap v15: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:03:40.056 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:03:40.058 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:03:40.967 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-20T22:03:39.536051+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":973,"num_read_kb":819,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":601,"ondisk_log_size":601,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":244748,"kb_used_data":7708,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749716468,"statfs":{"total":767960285184,"available":767709663232,"internally_reserved":0,"allocated":7892992,"data_stored":4440577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001836"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072554+0000","last_change":"2024-09-20T22:02:50.111666+0000","last_active":"2024-09-20T22:03:21.072554+0000","last_peered":"2024-09-20T22:03:21.072554+0000","last_clean":"2024-09-20T22:03:21.072554+0000","last_became_active":"2024-09-20T22:02:50.109930+0000","last_became_peered":"2024-09-20T22:02:50.109930+0000","last_unstale":"2024-09-20T22:03:21.072554+0000","last_undegraded":"2024-09-20T22:03:21.072554+0000","last_fullsized":"2024-09-20T22:03:21.072554+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.513745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"71'9","reported_seq":47,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279415+0000","last_change":"2024-09-20T22:02:52.277916+0000","last_active":"2024-09-20T22:03:35.279415+0000","last_peered":"2024-09-20T22:03:35.279415+0000","last_clean":"2024-09-20T22:03:35.279415+0000","last_became_active":"2024-09-20T22:02:52.277616+0000","last_became_peered":"2024-09-20T22:02:52.277616+0000","last_unstale":"2024-09-20T22:03:35.279415+0000","last_undegraded":"2024-09-20T22:03:35.279415+0000","last_fullsized":"2024-09-20T22:03:35.279415+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.412488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.278813+0000","last_change":"2024-09-20T22:02:54.287938+0000","last_active":"2024-09-20T22:03:35.278813+0000","last_peered":"2024-09-20T22:03:35.278813+0000","last_clean":"2024-09-20T22:03:35.278813+0000","last_became_active":"2024-09-20T22:02:54.287537+0000","last_became_peered":"2024-09-20T22:02:54.287537+0000","last_unstale":"2024-09-20T22:03:35.278813+0000","last_undegraded":"2024-09-20T22:03:35.278813+0000","last_fullsized":"2024-09-20T22:03:35.278813+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.152471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279633+0000","last_change":"2024-09-20T22:02:56.297551+0000","last_active":"2024-09-20T22:03:35.279633+0000","last_peered":"2024-09-20T22:03:35.279633+0000","last_clean":"2024-09-20T22:03:35.279633+0000","last_became_active":"2024-09-20T22:02:56.297252+0000","last_became_peered":"2024-09-20T22:02:56.297252+0000","last_unstale":"2024-09-20T22:03:35.279633+0000","last_undegraded":"2024-09-20T22:03:35.279633+0000","last_fullsized":"2024-09-20T22:03:35.279633+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.977495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071617+0000","last_change":"2024-09-20T22:02:56.350387+0000","last_active":"2024-09-20T22:03:21.071617+0000","last_peered":"2024-09-20T22:03:21.071617+0000","last_clean":"2024-09-20T22:03:21.071617+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:03:21.071617+0000","last_undegraded":"2024-09-20T22:03:21.071617+0000","last_fullsized":"2024-09-20T22:03:21.071617+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.462175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072081+0000","last_change":"2024-09-20T22:02:50.107457+0000","last_active":"2024-09-20T22:03:21.072081+0000","last_peered":"2024-09-20T22:03:21.072081+0000","last_clean":"2024-09-20T22:03:21.072081+0000","last_became_active":"2024-09-20T22:02:50.107154+0000","last_became_peered":"2024-09-20T22:02:50.107154+0000","last_unstale":"2024-09-20T22:03:21.072081+0000","last_undegraded":"2024-09-20T22:03:21.072081+0000","last_fullsized":"2024-09-20T22:03:21.072081+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:11.498961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"71'15","reported_seq":54,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071355+0000","last_change":"2024-09-20T22:02:52.282478+0000","last_active":"2024-09-20T22:03:21.071355+0000","last_peered":"2024-09-20T22:03:21.071355+0000","last_clean":"2024-09-20T22:03:21.071355+0000","last_became_active":"2024-09-20T22:02:52.282101+0000","last_became_peered":"2024-09-20T22:02:52.282101+0000","last_unstale":"2024-09-20T22:03:21.071355+0000","last_undegraded":"2024-09-20T22:03:21.071355+0000","last_fullsized":"2024-09-20T22:03:21.071355+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.208571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279412+0000","last_change":"2024-09-20T22:02:54.292666+0000","last_active":"2024-09-20T22:03:35.279412+0000","last_peered":"2024-09-20T22:03:35.279412+0000","last_clean":"2024-09-20T22:03:35.279412+0000","last_became_active":"2024-09-20T22:02:54.292343+0000","last_became_peered":"2024-09-20T22:02:54.292343+0000","last_unstale":"2024-09-20T22:03:35.279412+0000","last_undegraded":"2024-09-20T22:03:35.279412+0000","last_fullsized":"2024-09-20T22:03:35.279412+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.965688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071303+0000","last_change":"2024-09-20T22:02:50.108309+0000","last_active":"2024-09-20T22:03:21.071303+0000","last_peered":"2024-09-20T22:03:21.071303+0000","last_clean":"2024-09-20T22:03:21.071303+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:03:21.071303+0000","last_undegraded":"2024-09-20T22:03:21.071303+0000","last_fullsized":"2024-09-20T22:03:21.071303+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.039813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"71'9","reported_seq":47,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279833+0000","last_change":"2024-09-20T22:02:52.279933+0000","last_active":"2024-09-20T22:03:35.279833+0000","last_peered":"2024-09-20T22:03:35.279833+0000","last_clean":"2024-09-20T22:03:35.279833+0000","last_became_active":"2024-09-20T22:02:52.279526+0000","last_became_peered":"2024-09-20T22:02:52.279526+0000","last_unstale":"2024-09-20T22:03:35.279833+0000","last_undegraded":"2024-09-20T22:03:35.279833+0000","last_fullsized":"2024-09-20T22:03:35.279833+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.714248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070508+0000","last_change":"2024-09-20T22:02:54.293254+0000","last_active":"2024-09-20T22:03:21.070508+0000","last_peered":"2024-09-20T22:03:21.070508+0000","last_clean":"2024-09-20T22:03:21.070508+0000","last_became_active":"2024-09-20T22:02:54.292911+0000","last_became_peered":"2024-09-20T22:02:54.292911+0000","last_unstale":"2024-09-20T22:03:21.070508+0000","last_undegraded":"2024-09-20T22:03:21.070508+0000","last_fullsized":"2024-09-20T22:03:21.070508+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.908407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070886+0000","last_change":"2024-09-20T22:02:56.299059+0000","last_active":"2024-09-20T22:03:21.070886+0000","last_peered":"2024-09-20T22:03:21.070886+0000","last_clean":"2024-09-20T22:03:21.070886+0000","last_became_active":"2024-09-20T22:02:56.298660+0000","last_became_peered":"2024-09-20T22:02:56.298660+0000","last_unstale":"2024-09-20T22:03:21.070886+0000","last_undegraded":"2024-09-20T22:03:21.070886+0000","last_fullsized":"2024-09-20T22:03:21.070886+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:13.896683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070880+0000","last_change":"2024-09-20T22:02:50.106885+0000","last_active":"2024-09-20T22:03:21.070880+0000","last_peered":"2024-09-20T22:03:21.070880+0000","last_clean":"2024-09-20T22:03:21.070880+0000","last_became_active":"2024-09-20T22:02:50.106555+0000","last_became_peered":"2024-09-20T22:02:50.106555+0000","last_unstale":"2024-09-20T22:03:21.070880+0000","last_undegraded":"2024-09-20T22:03:21.070880+0000","last_fullsized":"2024-09-20T22:03:21.070880+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.204880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"71'5","reported_seq":41,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279823+0000","last_change":"2024-09-20T22:02:52.284347+0000","last_active":"2024-09-20T22:03:35.279823+0000","last_peered":"2024-09-20T22:03:35.279823+0000","last_clean":"2024-09-20T22:03:35.279823+0000","last_became_active":"2024-09-20T22:02:52.283904+0000","last_became_peered":"2024-09-20T22:02:52.283904+0000","last_unstale":"2024-09-20T22:03:35.279823+0000","last_undegraded":"2024-09-20T22:03:35.279823+0000","last_fullsized":"2024-09-20T22:03:35.279823+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:52.941046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703160+0000","last_change":"2024-09-20T22:02:54.295625+0000","last_active":"2024-09-20T22:03:20.703160+0000","last_peered":"2024-09-20T22:03:20.703160+0000","last_clean":"2024-09-20T22:03:20.703160+0000","last_became_active":"2024-09-20T22:02:54.294405+0000","last_became_peered":"2024-09-20T22:02:54.294405+0000","last_unstale":"2024-09-20T22:03:20.703160+0000","last_undegraded":"2024-09-20T22:03:20.703160+0000","last_fullsized":"2024-09-20T22:03:20.703160+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:53.468504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070421+0000","last_change":"2024-09-20T22:02:56.303701+0000","last_active":"2024-09-20T22:03:21.070421+0000","last_peered":"2024-09-20T22:03:21.070421+0000","last_clean":"2024-09-20T22:03:21.070421+0000","last_became_active":"2024-09-20T22:02:56.303338+0000","last_became_peered":"2024-09-20T22:02:56.303338+0000","last_unstale":"2024-09-20T22:03:21.070421+0000","last_undegraded":"2024-09-20T22:03:21.070421+0000","last_fullsized":"2024-09-20T22:03:21.070421+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.793299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.280408+0000","last_change":"2024-09-20T22:02:56.347311+0000","last_active":"2024-09-20T22:03:35.280408+0000","last_peered":"2024-09-20T22:03:35.280408+0000","last_clean":"2024-09-20T22:03:35.280408+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:03:35.280408+0000","last_undegraded":"2024-09-20T22:03:35.280408+0000","last_fullsized":"2024-09-20T22:03:35.280408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.297557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072043+0000","last_change":"2024-09-20T22:02:50.111842+0000","last_active":"2024-09-20T22:03:21.072043+0000","last_peered":"2024-09-20T22:03:21.072043+0000","last_clean":"2024-09-20T22:03:21.072043+0000","last_became_active":"2024-09-20T22:02:50.111517+0000","last_became_peered":"2024-09-20T22:02:50.111517+0000","last_unstale":"2024-09-20T22:03:21.072043+0000","last_undegraded":"2024-09-20T22:03:21.072043+0000","last_fullsized":"2024-09-20T22:03:21.072043+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.747690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"71'15","reported_seq":54,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071406+0000","last_change":"2024-09-20T22:02:52.283078+0000","last_active":"2024-09-20T22:03:21.071406+0000","last_peered":"2024-09-20T22:03:21.071406+0000","last_clean":"2024-09-20T22:03:21.071406+0000","last_became_active":"2024-09-20T22:02:52.282768+0000","last_became_peered":"2024-09-20T22:02:52.282768+0000","last_unstale":"2024-09-20T22:03:21.071406+0000","last_undegraded":"2024-09-20T22:03:21.071406+0000","last_fullsized":"2024-09-20T22:03:21.071406+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:52.589023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279696+0000","last_change":"2024-09-20T22:02:54.292892+0000","last_active":"2024-09-20T22:03:35.279696+0000","last_peered":"2024-09-20T22:03:35.279696+0000","last_clean":"2024-09-20T22:03:35.279696+0000","last_became_active":"2024-09-20T22:02:54.290510+0000","last_became_peered":"2024-09-20T22:02:54.290510+0000","last_unstale":"2024-09-20T22:03:35.279696+0000","last_undegraded":"2024-09-20T22:03:35.279696+0000","last_fullsized":"2024-09-20T22:03:35.279696+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.173080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072228+0000","last_change":"2024-09-20T22:02:56.347951+0000","last_active":"2024-09-20T22:03:21.072228+0000","last_peered":"2024-09-20T22:03:21.072228+0000","last_clean":"2024-09-20T22:03:21.072228+0000","last_became_active":"2024-09-20T22:02:56.347561+0000","last_became_peered":"2024-09-20T22:02:56.347561+0000","last_unstale":"2024-09-20T22:03:21.072228+0000","last_undegraded":"2024-09-20T22:03:21.072228+0000","last_fullsized":"2024-09-20T22:03:21.072228+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.255072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":35,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.280002+0000","last_change":"2024-09-20T22:02:50.108909+0000","last_active":"2024-09-20T22:03:35.280002+0000","last_peered":"2024-09-20T22:03:35.280002+0000","last_clean":"2024-09-20T22:03:35.280002+0000","last_became_active":"2024-09-20T22:02:50.107189+0000","last_became_peered":"2024-09-20T22:02:50.107189+0000","last_unstale":"2024-09-20T22:03:35.280002+0000","last_undegraded":"2024-09-20T22:03:35.280002+0000","last_fullsized":"2024-09-20T22:03:35.280002+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.636681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"71'12","reported_seq":52,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070801+0000","last_change":"2024-09-20T22:02:52.284201+0000","last_active":"2024-09-20T22:03:21.070801+0000","last_peered":"2024-09-20T22:03:21.070801+0000","last_clean":"2024-09-20T22:03:21.070801+0000","last_became_active":"2024-09-20T22:02:52.283843+0000","last_became_peered":"2024-09-20T22:02:52.283843+0000","last_unstale":"2024-09-20T22:03:21.070801+0000","last_undegraded":"2024-09-20T22:03:21.070801+0000","last_fullsized":"2024-09-20T22:03:21.070801+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.131718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279886+0000","last_change":"2024-09-20T22:02:54.293157+0000","last_active":"2024-09-20T22:03:35.279886+0000","last_peered":"2024-09-20T22:03:35.279886+0000","last_clean":"2024-09-20T22:03:35.279886+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:03:35.279886+0000","last_undegraded":"2024-09-20T22:03:35.279886+0000","last_fullsized":"2024-09-20T22:03:35.279886+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.203423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"71'1","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703332+0000","last_change":"2024-09-20T22:02:56.301720+0000","last_active":"2024-09-20T22:03:20.703332+0000","last_peered":"2024-09-20T22:03:20.703332+0000","last_clean":"2024-09-20T22:03:20.703332+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:03:20.703332+0000","last_undegraded":"2024-09-20T22:03:20.703332+0000","last_fullsized":"2024-09-20T22:03:20.703332+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.341799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"64'1","reported_seq":36,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.278964+0000","last_change":"2024-09-20T22:02:50.113259+0000","last_active":"2024-09-20T22:03:35.278964+0000","last_peered":"2024-09-20T22:03:35.278964+0000","last_clean":"2024-09-20T22:03:35.278964+0000","last_became_active":"2024-09-20T22:02:50.112862+0000","last_became_peered":"2024-09-20T22:02:50.112862+0000","last_unstale":"2024-09-20T22:03:35.278964+0000","last_undegraded":"2024-09-20T22:03:35.278964+0000","last_fullsized":"2024-09-20T22:03:35.278964+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.645807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"71'10","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072457+0000","last_change":"2024-09-20T22:02:52.482300+0000","last_active":"2024-09-20T22:03:21.072457+0000","last_peered":"2024-09-20T22:03:21.072457+0000","last_clean":"2024-09-20T22:03:21.072457+0000","last_became_active":"2024-09-20T22:02:52.481980+0000","last_became_peered":"2024-09-20T22:02:52.481980+0000","last_unstale":"2024-09-20T22:03:21.072457+0000","last_undegraded":"2024-09-20T22:03:21.072457+0000","last_fullsized":"2024-09-20T22:03:21.072457+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.082587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"71'8","reported_seq":46,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:30.172407+0000","last_change":"2024-09-20T22:02:54.297973+0000","last_active":"2024-09-20T22:03:30.172407+0000","last_peered":"2024-09-20T22:03:30.172407+0000","last_clean":"2024-09-20T22:03:30.172407+0000","last_became_active":"2024-09-20T22:02:54.297657+0000","last_became_peered":"2024-09-20T22:02:54.297657+0000","last_unstale":"2024-09-20T22:03:30.172407+0000","last_undegraded":"2024-09-20T22:03:30.172407+0000","last_fullsized":"2024-09-20T22:03:30.172407+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:20.633207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"71'15","reported_seq":56,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279784+0000","last_change":"2024-09-20T22:02:52.284890+0000","last_active":"2024-09-20T22:03:35.279784+0000","last_peered":"2024-09-20T22:03:35.279784+0000","last_clean":"2024-09-20T22:03:35.279784+0000","last_became_active":"2024-09-20T22:02:52.284027+0000","last_became_peered":"2024-09-20T22:02:52.284027+0000","last_unstale":"2024-09-20T22:03:35.279784+0000","last_undegraded":"2024-09-20T22:03:35.279784+0000","last_fullsized":"2024-09-20T22:03:35.279784+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:54.544010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070835+0000","last_change":"2024-09-20T22:02:50.108849+0000","last_active":"2024-09-20T22:03:21.070835+0000","last_peered":"2024-09-20T22:03:21.070835+0000","last_clean":"2024-09-20T22:03:21.070835+0000","last_became_active":"2024-09-20T22:02:50.108508+0000","last_became_peered":"2024-09-20T22:02:50.108508+0000","last_unstale":"2024-09-20T22:03:21.070835+0000","last_undegraded":"2024-09-20T22:03:21.070835+0000","last_fullsized":"2024-09-20T22:03:21.070835+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.430379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"71'8","reported_seq":48,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.280652+0000","last_change":"2024-09-20T22:02:54.295408+0000","last_active":"2024-09-20T22:03:35.280652+0000","last_peered":"2024-09-20T22:03:35.280652+0000","last_clean":"2024-09-20T22:03:35.280652+0000","last_became_active":"2024-09-20T22:02:54.294917+0000","last_became_peered":"2024-09-20T22:02:54.294917+0000","last_unstale":"2024-09-20T22:03:35.280652+0000","last_undegraded":"2024-09-20T22:03:35.280652+0000","last_fullsized":"2024-09-20T22:03:35.280652+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.541024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071048+0000","last_change":"2024-09-20T22:02:56.296725+0000","last_active":"2024-09-20T22:03:21.071048+0000","last_peered":"2024-09-20T22:03:21.071048+0000","last_clean":"2024-09-20T22:03:21.071048+0000","last_became_active":"2024-09-20T22:02:56.296436+0000","last_became_peered":"2024-09-20T22:02:56.296436+0000","last_unstale":"2024-09-20T22:03:21.071048+0000","last_undegraded":"2024-09-20T22:03:21.071048+0000","last_fullsized":"2024-09-20T22:03:21.071048+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:19.201794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"71'18","reported_seq":61,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070609+0000","last_change":"2024-09-20T22:02:52.482408+0000","last_active":"2024-09-20T22:03:21.070609+0000","last_peered":"2024-09-20T22:03:21.070609+0000","last_clean":"2024-09-20T22:03:21.070609+0000","last_became_active":"2024-09-20T22:02:52.482119+0000","last_became_peered":"2024-09-20T22:02:52.482119+0000","last_unstale":"2024-09-20T22:03:21.070609+0000","last_undegraded":"2024-09-20T22:03:21.070609+0000","last_fullsized":"2024-09-20T22:03:21.070609+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.477571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072419+0000","last_change":"2024-09-20T22:02:50.113398+0000","last_active":"2024-09-20T22:03:21.072419+0000","last_peered":"2024-09-20T22:03:21.072419+0000","last_clean":"2024-09-20T22:03:21.072419+0000","last_became_active":"2024-09-20T22:02:50.113076+0000","last_became_peered":"2024-09-20T22:02:50.113076+0000","last_unstale":"2024-09-20T22:03:21.072419+0000","last_undegraded":"2024-09-20T22:03:21.072419+0000","last_fullsized":"2024-09-20T22:03:21.072419+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.714101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279443+0000","last_change":"2024-09-20T22:02:54.288282+0000","last_active":"2024-09-20T22:03:35.279443+0000","last_peered":"2024-09-20T22:03:35.279443+0000","last_clean":"2024-09-20T22:03:35.279443+0000","last_became_active":"2024-09-20T22:02:54.288007+0000","last_became_peered":"2024-09-20T22:02:54.288007+0000","last_unstale":"2024-09-20T22:03:35.279443+0000","last_undegraded":"2024-09-20T22:03:35.279443+0000","last_fullsized":"2024-09-20T22:03:35.279443+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.737144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279780+0000","last_change":"2024-09-20T22:02:56.303051+0000","last_active":"2024-09-20T22:03:35.279780+0000","last_peered":"2024-09-20T22:03:35.279780+0000","last_clean":"2024-09-20T22:03:35.279780+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:03:35.279780+0000","last_undegraded":"2024-09-20T22:03:35.279780+0000","last_fullsized":"2024-09-20T22:03:35.279780+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.012094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"71'14","reported_seq":52,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279598+0000","last_change":"2024-09-20T22:02:52.273553+0000","last_active":"2024-09-20T22:03:35.279598+0000","last_peered":"2024-09-20T22:03:35.279598+0000","last_clean":"2024-09-20T22:03:35.279598+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:03:35.279598+0000","last_undegraded":"2024-09-20T22:03:35.279598+0000","last_fullsized":"2024-09-20T22:03:35.279598+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:56.047961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":34,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072627+0000","last_change":"2024-09-20T22:02:50.110546+0000","last_active":"2024-09-20T22:03:21.072627+0000","last_peered":"2024-09-20T22:03:21.072627+0000","last_clean":"2024-09-20T22:03:21.072627+0000","last_became_active":"2024-09-20T22:02:50.110195+0000","last_became_peered":"2024-09-20T22:02:50.110195+0000","last_unstale":"2024-09-20T22:03:21.072627+0000","last_undegraded":"2024-09-20T22:03:21.072627+0000","last_fullsized":"2024-09-20T22:03:21.072627+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.867716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"71'8","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:30.172108+0000","last_change":"2024-09-20T22:02:54.295841+0000","last_active":"2024-09-20T22:03:30.172108+0000","last_peered":"2024-09-20T22:03:30.172108+0000","last_clean":"2024-09-20T22:03:30.172108+0000","last_became_active":"2024-09-20T22:02:54.295553+0000","last_became_peered":"2024-09-20T22:02:54.295553+0000","last_unstale":"2024-09-20T22:03:30.172108+0000","last_undegraded":"2024-09-20T22:03:30.172108+0000","last_fullsized":"2024-09-20T22:03:30.172108+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.625795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703312+0000","last_change":"2024-09-20T22:02:56.349864+0000","last_active":"2024-09-20T22:03:20.703312+0000","last_peered":"2024-09-20T22:03:20.703312+0000","last_clean":"2024-09-20T22:03:20.703312+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:03:20.703312+0000","last_undegraded":"2024-09-20T22:03:20.703312+0000","last_fullsized":"2024-09-20T22:03:20.703312+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:57.678673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"71'10","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279313+0000","last_change":"2024-09-20T22:02:52.276282+0000","last_active":"2024-09-20T22:03:35.279313+0000","last_peered":"2024-09-20T22:03:35.279313+0000","last_clean":"2024-09-20T22:03:35.279313+0000","last_became_active":"2024-09-20T22:02:52.275958+0000","last_became_peered":"2024-09-20T22:02:52.275958+0000","last_unstale":"2024-09-20T22:03:35.279313+0000","last_undegraded":"2024-09-20T22:03:35.279313+0000","last_fullsized":"2024-09-20T22:03:35.279313+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.846987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071357+0000","last_change":"2024-09-20T22:02:50.114476+0000","last_active":"2024-09-20T22:03:21.071357+0000","last_peered":"2024-09-20T22:03:21.071357+0000","last_clean":"2024-09-20T22:03:21.071357+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:03:21.071357+0000","last_undegraded":"2024-09-20T22:03:21.071357+0000","last_fullsized":"2024-09-20T22:03:21.071357+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:51.340463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"71'8","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:30.172115+0000","last_change":"2024-09-20T22:02:54.293220+0000","last_active":"2024-09-20T22:03:30.172115+0000","last_peered":"2024-09-20T22:03:30.172115+0000","last_clean":"2024-09-20T22:03:30.172115+0000","last_became_active":"2024-09-20T22:02:54.292819+0000","last_became_peered":"2024-09-20T22:02:54.292819+0000","last_unstale":"2024-09-20T22:03:30.172115+0000","last_undegraded":"2024-09-20T22:03:30.172115+0000","last_fullsized":"2024-09-20T22:03:30.172115+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:59.827983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071031+0000","last_change":"2024-09-20T22:02:56.304595+0000","last_active":"2024-09-20T22:03:21.071031+0000","last_peered":"2024-09-20T22:03:21.071031+0000","last_clean":"2024-09-20T22:03:21.071031+0000","last_became_active":"2024-09-20T22:02:56.304274+0000","last_became_peered":"2024-09-20T22:02:56.304274+0000","last_unstale":"2024-09-20T22:03:21.071031+0000","last_undegraded":"2024-09-20T22:03:21.071031+0000","last_fullsized":"2024-09-20T22:03:21.071031+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.850583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"71'19","reported_seq":65,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071750+0000","last_change":"2024-09-20T22:02:52.480446+0000","last_active":"2024-09-20T22:03:21.071750+0000","last_peered":"2024-09-20T22:03:21.071750+0000","last_clean":"2024-09-20T22:03:21.071750+0000","last_became_active":"2024-09-20T22:02:52.480077+0000","last_became_peered":"2024-09-20T22:02:52.480077+0000","last_unstale":"2024-09-20T22:03:21.071750+0000","last_undegraded":"2024-09-20T22:03:21.071750+0000","last_fullsized":"2024-09-20T22:03:21.071750+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.830883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":35,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279368+0000","last_change":"2024-09-20T22:02:50.104649+0000","last_active":"2024-09-20T22:03:35.279368+0000","last_peered":"2024-09-20T22:03:35.279368+0000","last_clean":"2024-09-20T22:03:35.279368+0000","last_became_active":"2024-09-20T22:02:50.104345+0000","last_became_peered":"2024-09-20T22:02:50.104345+0000","last_unstale":"2024-09-20T22:03:35.279368+0000","last_undegraded":"2024-09-20T22:03:35.279368+0000","last_fullsized":"2024-09-20T22:03:35.279368+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.766875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070730+0000","last_change":"2024-09-20T22:02:54.291638+0000","last_active":"2024-09-20T22:03:21.070730+0000","last_peered":"2024-09-20T22:03:21.070730+0000","last_clean":"2024-09-20T22:03:21.070730+0000","last_became_active":"2024-09-20T22:02:54.290787+0000","last_became_peered":"2024-09-20T22:02:54.290787+0000","last_unstale":"2024-09-20T22:03:21.070730+0000","last_undegraded":"2024-09-20T22:03:21.070730+0000","last_fullsized":"2024-09-20T22:03:21.070730+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.996942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.278698+0000","last_change":"2024-09-20T22:02:56.350020+0000","last_active":"2024-09-20T22:03:35.278698+0000","last_peered":"2024-09-20T22:03:35.278698+0000","last_clean":"2024-09-20T22:03:35.278698+0000","last_became_active":"2024-09-20T22:02:56.349581+0000","last_became_peered":"2024-09-20T22:02:56.349581+0000","last_unstale":"2024-09-20T22:03:35.278698+0000","last_undegraded":"2024-09-20T22:03:35.278698+0000","last_fullsized":"2024-09-20T22:03:35.278698+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.722790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"71'28","reported_seq":89,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279282+0000","last_change":"2024-09-20T22:02:52.285811+0000","last_active":"2024-09-20T22:03:35.279282+0000","last_peered":"2024-09-20T22:03:35.279282+0000","last_clean":"2024-09-20T22:03:35.279282+0000","last_became_active":"2024-09-20T22:02:52.285341+0000","last_became_peered":"2024-09-20T22:02:52.285341+0000","last_unstale":"2024-09-20T22:03:35.279282+0000","last_undegraded":"2024-09-20T22:03:35.279282+0000","last_fullsized":"2024-09-20T22:03:35.279282+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.491983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":35,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.280044+0000","last_change":"2024-09-20T22:02:50.110040+0000","last_active":"2024-09-20T22:03:35.280044+0000","last_peered":"2024-09-20T22:03:35.280044+0000","last_clean":"2024-09-20T22:03:35.280044+0000","last_became_active":"2024-09-20T22:02:50.108457+0000","last_became_peered":"2024-09-20T22:02:50.108457+0000","last_unstale":"2024-09-20T22:03:35.280044+0000","last_undegraded":"2024-09-20T22:03:35.280044+0000","last_fullsized":"2024-09-20T22:03:35.280044+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.280284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"66'2","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071023+0000","last_change":"2024-09-20T22:02:52.273740+0000","last_active":"2024-09-20T22:03:21.071023+0000","last_peered":"2024-09-20T22:03:21.071023+0000","last_clean":"2024-09-20T22:03:21.071023+0000","last_became_active":"2024-09-20T22:02:49.096453+0000","last_became_peered":"2024-09-20T22:02:49.096453+0000","last_unstale":"2024-09-20T22:03:21.071023+0000","last_undegraded":"2024-09-20T22:03:21.071023+0000","last_fullsized":"2024-09-20T22:03:21.071023+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.811834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072280+0000","last_change":"2024-09-20T22:02:54.295696+0000","last_active":"2024-09-20T22:03:21.072280+0000","last_peered":"2024-09-20T22:03:21.072280+0000","last_clean":"2024-09-20T22:03:21.072280+0000","last_became_active":"2024-09-20T22:02:54.295401+0000","last_became_peered":"2024-09-20T22:02:54.295401+0000","last_unstale":"2024-09-20T22:03:21.072280+0000","last_undegraded":"2024-09-20T22:03:21.072280+0000","last_fullsized":"2024-09-20T22:03:21.072280+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.933605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071815+0000","last_change":"2024-09-20T22:02:56.306053+0000","last_active":"2024-09-20T22:03:21.071815+0000","last_peered":"2024-09-20T22:03:21.071815+0000","last_clean":"2024-09-20T22:03:21.071815+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:03:21.071815+0000","last_undegraded":"2024-09-20T22:03:21.071815+0000","last_fullsized":"2024-09-20T22:03:21.071815+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.682717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"71'13","reported_seq":58,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279678+0000","last_change":"2024-09-20T22:02:52.280309+0000","last_active":"2024-09-20T22:03:35.279678+0000","last_peered":"2024-09-20T22:03:35.279678+0000","last_clean":"2024-09-20T22:03:35.279678+0000","last_became_active":"2024-09-20T22:02:52.280005+0000","last_became_peered":"2024-09-20T22:02:52.280005+0000","last_unstale":"2024-09-20T22:03:35.279678+0000","last_undegraded":"2024-09-20T22:03:35.279678+0000","last_fullsized":"2024-09-20T22:03:35.279678+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.190808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":35,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.278883+0000","last_change":"2024-09-20T22:02:50.106023+0000","last_active":"2024-09-20T22:03:35.278883+0000","last_peered":"2024-09-20T22:03:35.278883+0000","last_clean":"2024-09-20T22:03:35.278883+0000","last_became_active":"2024-09-20T22:02:50.105633+0000","last_became_peered":"2024-09-20T22:02:50.105633+0000","last_unstale":"2024-09-20T22:03:35.278883+0000","last_undegraded":"2024-09-20T22:03:35.278883+0000","last_fullsized":"2024-09-20T22:03:35.278883+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.449401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"64'1","reported_seq":42,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071025+0000","last_change":"2024-09-20T22:02:52.270463+0000","last_active":"2024-09-20T22:03:21.071025+0000","last_peered":"2024-09-20T22:03:21.071025+0000","last_clean":"2024-09-20T22:03:21.071025+0000","last_became_active":"2024-09-20T22:02:49.099508+0000","last_became_peered":"2024-09-20T22:02:49.099508+0000","last_unstale":"2024-09-20T22:03:21.071025+0000","last_undegraded":"2024-09-20T22:03:21.071025+0000","last_fullsized":"2024-09-20T22:03:21.071025+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.955566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070945+0000","last_change":"2024-09-20T22:02:54.292450+0000","last_active":"2024-09-20T22:03:21.070945+0000","last_peered":"2024-09-20T22:03:21.070945+0000","last_clean":"2024-09-20T22:03:21.070945+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:03:21.070945+0000","last_undegraded":"2024-09-20T22:03:21.070945+0000","last_fullsized":"2024-09-20T22:03:21.070945+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.941012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703517+0000","last_change":"2024-09-20T22:02:56.349789+0000","last_active":"2024-09-20T22:03:20.703517+0000","last_peered":"2024-09-20T22:03:20.703517+0000","last_clean":"2024-09-20T22:03:20.703517+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:03:20.703517+0000","last_undegraded":"2024-09-20T22:03:20.703517+0000","last_fullsized":"2024-09-20T22:03:20.703517+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.060034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"71'12","reported_seq":47,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072508+0000","last_change":"2024-09-20T22:02:52.277878+0000","last_active":"2024-09-20T22:03:21.072508+0000","last_peered":"2024-09-20T22:03:21.072508+0000","last_clean":"2024-09-20T22:03:21.072508+0000","last_became_active":"2024-09-20T22:02:52.277548+0000","last_became_peered":"2024-09-20T22:02:52.277548+0000","last_unstale":"2024-09-20T22:03:21.072508+0000","last_undegraded":"2024-09-20T22:03:21.072508+0000","last_fullsized":"2024-09-20T22:03:21.072508+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.295996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070632+0000","last_change":"2024-09-20T22:02:50.110740+0000","last_active":"2024-09-20T22:03:21.070632+0000","last_peered":"2024-09-20T22:03:21.070632+0000","last_clean":"2024-09-20T22:03:21.070632+0000","last_became_active":"2024-09-20T22:02:50.110448+0000","last_became_peered":"2024-09-20T22:02:50.110448+0000","last_unstale":"2024-09-20T22:03:21.070632+0000","last_undegraded":"2024-09-20T22:03:21.070632+0000","last_fullsized":"2024-09-20T22:03:21.070632+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.942941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":83,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:33.715151+0000","last_change":"2024-09-20T22:02:52.480790+0000","last_active":"2024-09-20T22:03:33.715151+0000","last_peered":"2024-09-20T22:03:33.715151+0000","last_clean":"2024-09-20T22:03:33.715151+0000","last_became_active":"2024-09-20T22:02:49.098777+0000","last_became_peered":"2024-09-20T22:02:49.098777+0000","last_unstale":"2024-09-20T22:03:33.715151+0000","last_undegraded":"2024-09-20T22:03:33.715151+0000","last_fullsized":"2024-09-20T22:03:33.715151+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.872201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":34,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070884+0000","last_change":"2024-09-20T22:02:54.294806+0000","last_active":"2024-09-20T22:03:21.070884+0000","last_peered":"2024-09-20T22:03:21.070884+0000","last_clean":"2024-09-20T22:03:21.070884+0000","last_became_active":"2024-09-20T22:02:54.294495+0000","last_became_peered":"2024-09-20T22:02:54.294495+0000","last_unstale":"2024-09-20T22:03:21.070884+0000","last_undegraded":"2024-09-20T22:03:21.070884+0000","last_fullsized":"2024-09-20T22:03:21.070884+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:23.237482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279120+0000","last_change":"2024-09-20T22:02:56.308897+0000","last_active":"2024-09-20T22:03:35.279120+0000","last_peered":"2024-09-20T22:03:35.279120+0000","last_clean":"2024-09-20T22:03:35.279120+0000","last_became_active":"2024-09-20T22:02:56.308574+0000","last_became_peered":"2024-09-20T22:02:56.308574+0000","last_unstale":"2024-09-20T22:03:35.279120+0000","last_undegraded":"2024-09-20T22:03:35.279120+0000","last_fullsized":"2024-09-20T22:03:35.279120+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.064046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"71'16","reported_seq":61,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:30.172468+0000","last_change":"2024-09-20T22:02:52.482708+0000","last_active":"2024-09-20T22:03:30.172468+0000","last_peered":"2024-09-20T22:03:30.172468+0000","last_clean":"2024-09-20T22:03:30.172468+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:03:30.172468+0000","last_undegraded":"2024-09-20T22:03:30.172468+0000","last_fullsized":"2024-09-20T22:03:30.172468+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.243856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071690+0000","last_change":"2024-09-20T22:02:50.106319+0000","last_active":"2024-09-20T22:03:21.071690+0000","last_peered":"2024-09-20T22:03:21.071690+0000","last_clean":"2024-09-20T22:03:21.071690+0000","last_became_active":"2024-09-20T22:02:50.106011+0000","last_became_peered":"2024-09-20T22:02:50.106011+0000","last_unstale":"2024-09-20T22:03:21.071690+0000","last_undegraded":"2024-09-20T22:03:21.071690+0000","last_fullsized":"2024-09-20T22:03:21.071690+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.288394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":288,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.599816+0000","last_change":"2024-09-20T22:02:55.903219+0000","last_active":"2024-09-20T22:03:21.599816+0000","last_peered":"2024-09-20T22:03:21.599816+0000","last_clean":"2024-09-20T22:03:21.599816+0000","last_became_active":"2024-09-20T22:02:45.068482+0000","last_became_peered":"2024-09-20T22:02:45.068482+0000","last_unstale":"2024-09-20T22:03:21.599816+0000","last_undegraded":"2024-09-20T22:03:21.599816+0000","last_fullsized":"2024-09-20T22:03:21.599816+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'117","last_scrub_stamp":"2024-09-20T22:02:55.903139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.903139+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.843650+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703457+0000","last_change":"2024-09-20T22:02:54.298298+0000","last_active":"2024-09-20T22:03:20.703457+0000","last_peered":"2024-09-20T22:03:20.703457+0000","last_clean":"2024-09-20T22:03:20.703457+0000","last_became_active":"2024-09-20T22:02:54.298021+0000","last_became_peered":"2024-09-20T22:02:54.298021+0000","last_unstale":"2024-09-20T22:03:20.703457+0000","last_undegraded":"2024-09-20T22:03:20.703457+0000","last_fullsized":"2024-09-20T22:03:20.703457+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:54.157709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070644+0000","last_change":"2024-09-20T22:02:56.308039+0000","last_active":"2024-09-20T22:03:21.070644+0000","last_peered":"2024-09-20T22:03:21.070644+0000","last_clean":"2024-09-20T22:03:21.070644+0000","last_became_active":"2024-09-20T22:02:56.307749+0000","last_became_peered":"2024-09-20T22:02:56.307749+0000","last_unstale":"2024-09-20T22:03:21.070644+0000","last_undegraded":"2024-09-20T22:03:21.070644+0000","last_fullsized":"2024-09-20T22:03:21.070644+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.666810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"71'11","reported_seq":50,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.280534+0000","last_change":"2024-09-20T22:02:52.273177+0000","last_active":"2024-09-20T22:03:35.280534+0000","last_peered":"2024-09-20T22:03:35.280534+0000","last_clean":"2024-09-20T22:03:35.280534+0000","last_became_active":"2024-09-20T22:02:52.272766+0000","last_became_peered":"2024-09-20T22:02:52.272766+0000","last_unstale":"2024-09-20T22:03:35.280534+0000","last_undegraded":"2024-09-20T22:03:35.280534+0000","last_fullsized":"2024-09-20T22:03:35.280534+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.073826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":35,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279771+0000","last_change":"2024-09-20T22:02:50.109626+0000","last_active":"2024-09-20T22:03:35.279771+0000","last_peered":"2024-09-20T22:03:35.279771+0000","last_clean":"2024-09-20T22:03:35.279771+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:03:35.279771+0000","last_undegraded":"2024-09-20T22:03:35.279771+0000","last_fullsized":"2024-09-20T22:03:35.279771+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.659206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070499+0000","last_change":"2024-09-20T22:02:54.290285+0000","last_active":"2024-09-20T22:03:21.070499+0000","last_peered":"2024-09-20T22:03:21.070499+0000","last_clean":"2024-09-20T22:03:21.070499+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:03:21.070499+0000","last_undegraded":"2024-09-20T22:03:21.070499+0000","last_fullsized":"2024-09-20T22:03:21.070499+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.665015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071605+0000","last_change":"2024-09-20T22:02:56.349707+0000","last_active":"2024-09-20T22:03:21.071605+0000","last_peered":"2024-09-20T22:03:21.071605+0000","last_clean":"2024-09-20T22:03:21.071605+0000","last_became_active":"2024-09-20T22:02:56.349376+0000","last_became_peered":"2024-09-20T22:02:56.349376+0000","last_unstale":"2024-09-20T22:03:21.071605+0000","last_undegraded":"2024-09-20T22:03:21.071605+0000","last_fullsized":"2024-09-20T22:03:21.071605+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.370704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"71'17","reported_seq":59,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.280524+0000","last_change":"2024-09-20T22:02:52.274124+0000","last_active":"2024-09-20T22:03:35.280524+0000","last_peered":"2024-09-20T22:03:35.280524+0000","last_clean":"2024-09-20T22:03:35.280524+0000","last_became_active":"2024-09-20T22:02:52.273811+0000","last_became_peered":"2024-09-20T22:02:52.273811+0000","last_unstale":"2024-09-20T22:03:35.280524+0000","last_undegraded":"2024-09-20T22:03:35.280524+0000","last_fullsized":"2024-09-20T22:03:35.280524+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:52.793675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071073+0000","last_change":"2024-09-20T22:02:50.115379+0000","last_active":"2024-09-20T22:03:21.071073+0000","last_peered":"2024-09-20T22:03:21.071073+0000","last_clean":"2024-09-20T22:03:21.071073+0000","last_became_active":"2024-09-20T22:02:50.115015+0000","last_became_peered":"2024-09-20T22:02:50.115015+0000","last_unstale":"2024-09-20T22:03:21.071073+0000","last_undegraded":"2024-09-20T22:03:21.071073+0000","last_fullsized":"2024-09-20T22:03:21.071073+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:16.806437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279166+0000","last_change":"2024-09-20T22:02:54.291468+0000","last_active":"2024-09-20T22:03:35.279166+0000","last_peered":"2024-09-20T22:03:35.279166+0000","last_clean":"2024-09-20T22:03:35.279166+0000","last_became_active":"2024-09-20T22:02:54.291161+0000","last_became_peered":"2024-09-20T22:02:54.291161+0000","last_unstale":"2024-09-20T22:03:35.279166+0000","last_undegraded":"2024-09-20T22:03:35.279166+0000","last_fullsized":"2024-09-20T22:03:35.279166+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.806525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071370+0000","last_change":"2024-09-20T22:02:56.307571+0000","last_active":"2024-09-20T22:03:21.071370+0000","last_peered":"2024-09-20T22:03:21.071370+0000","last_clean":"2024-09-20T22:03:21.071370+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:03:21.071370+0000","last_undegraded":"2024-09-20T22:03:21.071370+0000","last_fullsized":"2024-09-20T22:03:21.071370+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:20.532247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"71'10","reported_seq":46,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.280086+0000","last_change":"2024-09-20T22:02:52.284306+0000","last_active":"2024-09-20T22:03:35.280086+0000","last_peered":"2024-09-20T22:03:35.280086+0000","last_clean":"2024-09-20T22:03:35.280086+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:03:35.280086+0000","last_undegraded":"2024-09-20T22:03:35.280086+0000","last_fullsized":"2024-09-20T22:03:35.280086+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:20.155009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"64'1","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072098+0000","last_change":"2024-09-20T22:02:50.110772+0000","last_active":"2024-09-20T22:03:21.072098+0000","last_peered":"2024-09-20T22:03:21.072098+0000","last_clean":"2024-09-20T22:03:21.072098+0000","last_became_active":"2024-09-20T22:02:50.110446+0000","last_became_peered":"2024-09-20T22:02:50.110446+0000","last_unstale":"2024-09-20T22:03:21.072098+0000","last_undegraded":"2024-09-20T22:03:21.072098+0000","last_fullsized":"2024-09-20T22:03:21.072098+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.664497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"71'8","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:30.172125+0000","last_change":"2024-09-20T22:02:54.294781+0000","last_active":"2024-09-20T22:03:30.172125+0000","last_peered":"2024-09-20T22:03:30.172125+0000","last_clean":"2024-09-20T22:03:30.172125+0000","last_became_active":"2024-09-20T22:02:54.294458+0000","last_became_peered":"2024-09-20T22:02:54.294458+0000","last_unstale":"2024-09-20T22:03:30.172125+0000","last_undegraded":"2024-09-20T22:03:30.172125+0000","last_fullsized":"2024-09-20T22:03:30.172125+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.016385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.280272+0000","last_change":"2024-09-20T22:02:56.300412+0000","last_active":"2024-09-20T22:03:35.280272+0000","last_peered":"2024-09-20T22:03:35.280272+0000","last_clean":"2024-09-20T22:03:35.280272+0000","last_became_active":"2024-09-20T22:02:56.300093+0000","last_became_peered":"2024-09-20T22:02:56.300093+0000","last_unstale":"2024-09-20T22:03:35.280272+0000","last_undegraded":"2024-09-20T22:03:35.280272+0000","last_fullsized":"2024-09-20T22:03:35.280272+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.204657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"71'9","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071641+0000","last_change":"2024-09-20T22:02:52.278728+0000","last_active":"2024-09-20T22:03:21.071641+0000","last_peered":"2024-09-20T22:03:21.071641+0000","last_clean":"2024-09-20T22:03:21.071641+0000","last_became_active":"2024-09-20T22:02:52.278381+0000","last_became_peered":"2024-09-20T22:02:52.278381+0000","last_unstale":"2024-09-20T22:03:21.071641+0000","last_undegraded":"2024-09-20T22:03:21.071641+0000","last_fullsized":"2024-09-20T22:03:21.071641+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:51.365937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070990+0000","last_change":"2024-09-20T22:02:50.113046+0000","last_active":"2024-09-20T22:03:21.070990+0000","last_peered":"2024-09-20T22:03:21.070990+0000","last_clean":"2024-09-20T22:03:21.070990+0000","last_became_active":"2024-09-20T22:02:50.112682+0000","last_became_peered":"2024-09-20T22:02:50.112682+0000","last_unstale":"2024-09-20T22:03:21.070990+0000","last_undegraded":"2024-09-20T22:03:21.070990+0000","last_fullsized":"2024-09-20T22:03:21.070990+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.700772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070312+0000","last_change":"2024-09-20T22:02:54.289079+0000","last_active":"2024-09-20T22:03:21.070312+0000","last_peered":"2024-09-20T22:03:21.070312+0000","last_clean":"2024-09-20T22:03:21.070312+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:03:21.070312+0000","last_undegraded":"2024-09-20T22:03:21.070312+0000","last_fullsized":"2024-09-20T22:03:21.070312+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.034123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070529+0000","last_change":"2024-09-20T22:02:56.298414+0000","last_active":"2024-09-20T22:03:21.070529+0000","last_peered":"2024-09-20T22:03:21.070529+0000","last_clean":"2024-09-20T22:03:21.070529+0000","last_became_active":"2024-09-20T22:02:56.298046+0000","last_became_peered":"2024-09-20T22:02:56.298046+0000","last_unstale":"2024-09-20T22:03:21.070529+0000","last_undegraded":"2024-09-20T22:03:21.070529+0000","last_fullsized":"2024-09-20T22:03:21.070529+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:59.723832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"71'19","reported_seq":60,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070816+0000","last_change":"2024-09-20T22:02:52.480577+0000","last_active":"2024-09-20T22:03:21.070816+0000","last_peered":"2024-09-20T22:03:21.070816+0000","last_clean":"2024-09-20T22:03:21.070816+0000","last_became_active":"2024-09-20T22:02:52.480208+0000","last_became_peered":"2024-09-20T22:02:52.480208+0000","last_unstale":"2024-09-20T22:03:21.070816+0000","last_undegraded":"2024-09-20T22:03:21.070816+0000","last_fullsized":"2024-09-20T22:03:21.070816+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.853373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703789+0000","last_change":"2024-09-20T22:02:50.109094+0000","last_active":"2024-09-20T22:03:20.703789+0000","last_peered":"2024-09-20T22:03:20.703789+0000","last_clean":"2024-09-20T22:03:20.703789+0000","last_became_active":"2024-09-20T22:02:50.108712+0000","last_became_peered":"2024-09-20T22:02:50.108712+0000","last_unstale":"2024-09-20T22:03:20.703789+0000","last_undegraded":"2024-09-20T22:03:20.703789+0000","last_fullsized":"2024-09-20T22:03:20.703789+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:08.545134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070341+0000","last_change":"2024-09-20T22:02:54.291441+0000","last_active":"2024-09-20T22:03:21.070341+0000","last_peered":"2024-09-20T22:03:21.070341+0000","last_clean":"2024-09-20T22:03:21.070341+0000","last_became_active":"2024-09-20T22:02:54.291134+0000","last_became_peered":"2024-09-20T22:02:54.291134+0000","last_unstale":"2024-09-20T22:03:21.070341+0000","last_undegraded":"2024-09-20T22:03:21.070341+0000","last_fullsized":"2024-09-20T22:03:21.070341+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.677000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703250+0000","last_change":"2024-09-20T22:02:56.308001+0000","last_active":"2024-09-20T22:03:20.703250+0000","last_peered":"2024-09-20T22:03:20.703250+0000","last_clean":"2024-09-20T22:03:20.703250+0000","last_became_active":"2024-09-20T22:02:56.307687+0000","last_became_peered":"2024-09-20T22:02:56.307687+0000","last_unstale":"2024-09-20T22:03:20.703250+0000","last_undegraded":"2024-09-20T22:03:20.703250+0000","last_fullsized":"2024-09-20T22:03:20.703250+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.249693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"71'12","reported_seq":54,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279487+0000","last_change":"2024-09-20T22:02:52.284579+0000","last_active":"2024-09-20T22:03:35.279487+0000","last_peered":"2024-09-20T22:03:35.279487+0000","last_clean":"2024-09-20T22:03:35.279487+0000","last_became_active":"2024-09-20T22:02:52.284144+0000","last_became_peered":"2024-09-20T22:02:52.284144+0000","last_unstale":"2024-09-20T22:03:35.279487+0000","last_undegraded":"2024-09-20T22:03:35.279487+0000","last_fullsized":"2024-09-20T22:03:35.279487+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.901626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703857+0000","last_change":"2024-09-20T22:02:50.111591+0000","last_active":"2024-09-20T22:03:20.703857+0000","last_peered":"2024-09-20T22:03:20.703857+0000","last_clean":"2024-09-20T22:03:20.703857+0000","last_became_active":"2024-09-20T22:02:50.111267+0000","last_became_peered":"2024-09-20T22:02:50.111267+0000","last_unstale":"2024-09-20T22:03:20.703857+0000","last_undegraded":"2024-09-20T22:03:20.703857+0000","last_fullsized":"2024-09-20T22:03:20.703857+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.038287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070129+0000","last_change":"2024-09-20T22:02:54.293804+0000","last_active":"2024-09-20T22:03:21.070129+0000","last_peered":"2024-09-20T22:03:21.070129+0000","last_clean":"2024-09-20T22:03:21.070129+0000","last_became_active":"2024-09-20T22:02:54.293377+0000","last_became_peered":"2024-09-20T22:02:54.293377+0000","last_unstale":"2024-09-20T22:03:21.070129+0000","last_undegraded":"2024-09-20T22:03:21.070129+0000","last_fullsized":"2024-09-20T22:03:21.070129+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.827756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071742+0000","last_change":"2024-09-20T22:02:56.307499+0000","last_active":"2024-09-20T22:03:21.071742+0000","last_peered":"2024-09-20T22:03:21.071742+0000","last_clean":"2024-09-20T22:03:21.071742+0000","last_became_active":"2024-09-20T22:02:56.306991+0000","last_became_peered":"2024-09-20T22:02:56.306991+0000","last_unstale":"2024-09-20T22:03:21.071742+0000","last_undegraded":"2024-09-20T22:03:21.071742+0000","last_fullsized":"2024-09-20T22:03:21.071742+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.424396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"71'15","reported_seq":54,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070130+0000","last_change":"2024-09-20T22:02:52.482395+0000","last_active":"2024-09-20T22:03:21.070130+0000","last_peered":"2024-09-20T22:03:21.070130+0000","last_clean":"2024-09-20T22:03:21.070130+0000","last_became_active":"2024-09-20T22:02:52.482008+0000","last_became_peered":"2024-09-20T22:02:52.482008+0000","last_unstale":"2024-09-20T22:03:21.070130+0000","last_undegraded":"2024-09-20T22:03:21.070130+0000","last_fullsized":"2024-09-20T22:03:21.070130+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.554179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"64'2","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703795+0000","last_change":"2024-09-20T22:02:50.115186+0000","last_active":"2024-09-20T22:03:20.703795+0000","last_peered":"2024-09-20T22:03:20.703795+0000","last_clean":"2024-09-20T22:03:20.703795+0000","last_became_active":"2024-09-20T22:02:50.114819+0000","last_became_peered":"2024-09-20T22:02:50.114819+0000","last_unstale":"2024-09-20T22:03:20.703795+0000","last_undegraded":"2024-09-20T22:03:20.703795+0000","last_fullsized":"2024-09-20T22:03:20.703795+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.801926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:30.172406+0000","last_change":"2024-09-20T22:02:54.296823+0000","last_active":"2024-09-20T22:03:30.172406+0000","last_peered":"2024-09-20T22:03:30.172406+0000","last_clean":"2024-09-20T22:03:30.172406+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:03:30.172406+0000","last_undegraded":"2024-09-20T22:03:30.172406+0000","last_fullsized":"2024-09-20T22:03:30.172406+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.496588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070689+0000","last_change":"2024-09-20T22:02:56.347283+0000","last_active":"2024-09-20T22:03:21.070689+0000","last_peered":"2024-09-20T22:03:21.070689+0000","last_clean":"2024-09-20T22:03:21.070689+0000","last_became_active":"2024-09-20T22:02:56.346990+0000","last_became_peered":"2024-09-20T22:02:56.346990+0000","last_unstale":"2024-09-20T22:03:21.070689+0000","last_undegraded":"2024-09-20T22:03:21.070689+0000","last_fullsized":"2024-09-20T22:03:21.070689+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.718016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070498+0000","last_change":"2024-09-20T22:02:50.112245+0000","last_active":"2024-09-20T22:03:21.070498+0000","last_peered":"2024-09-20T22:03:21.070498+0000","last_clean":"2024-09-20T22:03:21.070498+0000","last_became_active":"2024-09-20T22:02:50.111935+0000","last_became_peered":"2024-09-20T22:02:50.111935+0000","last_unstale":"2024-09-20T22:03:21.070498+0000","last_undegraded":"2024-09-20T22:03:21.070498+0000","last_fullsized":"2024-09-20T22:03:21.070498+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:16.998775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"71'6","reported_seq":38,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072364+0000","last_change":"2024-09-20T22:02:52.481046+0000","last_active":"2024-09-20T22:03:21.072364+0000","last_peered":"2024-09-20T22:03:21.072364+0000","last_clean":"2024-09-20T22:03:21.072364+0000","last_became_active":"2024-09-20T22:02:52.480771+0000","last_became_peered":"2024-09-20T22:02:52.480771+0000","last_unstale":"2024-09-20T22:03:21.072364+0000","last_undegraded":"2024-09-20T22:03:21.072364+0000","last_fullsized":"2024-09-20T22:03:21.072364+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.943982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070886+0000","last_change":"2024-09-20T22:02:54.294834+0000","last_active":"2024-09-20T22:03:21.070886+0000","last_peered":"2024-09-20T22:03:21.070886+0000","last_clean":"2024-09-20T22:03:21.070886+0000","last_became_active":"2024-09-20T22:02:54.294539+0000","last_became_peered":"2024-09-20T22:02:54.294539+0000","last_unstale":"2024-09-20T22:03:21.070886+0000","last_undegraded":"2024-09-20T22:03:21.070886+0000","last_fullsized":"2024-09-20T22:03:21.070886+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.523368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703485+0000","last_change":"2024-09-20T22:02:56.350065+0000","last_active":"2024-09-20T22:03:20.703485+0000","last_peered":"2024-09-20T22:03:20.703485+0000","last_clean":"2024-09-20T22:03:20.703485+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:03:20.703485+0000","last_undegraded":"2024-09-20T22:03:20.703485+0000","last_fullsized":"2024-09-20T22:03:20.703485+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:19.909723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"71'9","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072562+0000","last_change":"2024-09-20T22:02:52.482361+0000","last_active":"2024-09-20T22:03:21.072562+0000","last_peered":"2024-09-20T22:03:21.072562+0000","last_clean":"2024-09-20T22:03:21.072562+0000","last_became_active":"2024-09-20T22:02:52.481989+0000","last_became_peered":"2024-09-20T22:02:52.481989+0000","last_unstale":"2024-09-20T22:03:21.072562+0000","last_undegraded":"2024-09-20T22:03:21.072562+0000","last_fullsized":"2024-09-20T22:03:21.072562+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.401767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703355+0000","last_change":"2024-09-20T22:02:50.108943+0000","last_active":"2024-09-20T22:03:20.703355+0000","last_peered":"2024-09-20T22:03:20.703355+0000","last_clean":"2024-09-20T22:03:20.703355+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:03:20.703355+0000","last_undegraded":"2024-09-20T22:03:20.703355+0000","last_fullsized":"2024-09-20T22:03:20.703355+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.114642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072267+0000","last_change":"2024-09-20T22:02:54.298137+0000","last_active":"2024-09-20T22:03:21.072267+0000","last_peered":"2024-09-20T22:03:21.072267+0000","last_clean":"2024-09-20T22:03:21.072267+0000","last_became_active":"2024-09-20T22:02:54.297850+0000","last_became_peered":"2024-09-20T22:02:54.297850+0000","last_unstale":"2024-09-20T22:03:21.072267+0000","last_undegraded":"2024-09-20T22:03:21.072267+0000","last_fullsized":"2024-09-20T22:03:21.072267+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.705512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071088+0000","last_change":"2024-09-20T22:02:56.299306+0000","last_active":"2024-09-20T22:03:21.071088+0000","last_peered":"2024-09-20T22:03:21.071088+0000","last_clean":"2024-09-20T22:03:21.071088+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:03:21.071088+0000","last_undegraded":"2024-09-20T22:03:21.071088+0000","last_fullsized":"2024-09-20T22:03:21.071088+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:11.499111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"71'9","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071259+0000","last_change":"2024-09-20T22:02:52.482498+0000","last_active":"2024-09-20T22:03:21.071259+0000","last_peered":"2024-09-20T22:03:21.071259+0000","last_clean":"2024-09-20T22:03:21.071259+0000","last_became_active":"2024-09-20T22:02:52.482141+0000","last_became_peered":"2024-09-20T22:02:52.482141+0000","last_unstale":"2024-09-20T22:03:21.071259+0000","last_undegraded":"2024-09-20T22:03:21.071259+0000","last_fullsized":"2024-09-20T22:03:21.071259+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.093555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070666+0000","last_change":"2024-09-20T22:02:50.112533+0000","last_active":"2024-09-20T22:03:21.070666+0000","last_peered":"2024-09-20T22:03:21.070666+0000","last_clean":"2024-09-20T22:03:21.070666+0000","last_became_active":"2024-09-20T22:02:50.110075+0000","last_became_peered":"2024-09-20T22:02:50.110075+0000","last_unstale":"2024-09-20T22:03:21.070666+0000","last_undegraded":"2024-09-20T22:03:21.070666+0000","last_fullsized":"2024-09-20T22:03:21.070666+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.714513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"71'8","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:30.172134+0000","last_change":"2024-09-20T22:02:54.294653+0000","last_active":"2024-09-20T22:03:30.172134+0000","last_peered":"2024-09-20T22:03:30.172134+0000","last_clean":"2024-09-20T22:03:30.172134+0000","last_became_active":"2024-09-20T22:02:54.294381+0000","last_became_peered":"2024-09-20T22:02:54.294381+0000","last_unstale":"2024-09-20T22:03:30.172134+0000","last_undegraded":"2024-09-20T22:03:30.172134+0000","last_fullsized":"2024-09-20T22:03:30.172134+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.135221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.280541+0000","last_change":"2024-09-20T22:02:56.300405+0000","last_active":"2024-09-20T22:03:35.280541+0000","last_peered":"2024-09-20T22:03:35.280541+0000","last_clean":"2024-09-20T22:03:35.280541+0000","last_became_active":"2024-09-20T22:02:56.300041+0000","last_became_peered":"2024-09-20T22:02:56.300041+0000","last_unstale":"2024-09-20T22:03:35.280541+0000","last_undegraded":"2024-09-20T22:03:35.280541+0000","last_fullsized":"2024-09-20T22:03:35.280541+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.312552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"71'10","reported_seq":44,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071846+0000","last_change":"2024-09-20T22:02:52.480909+0000","last_active":"2024-09-20T22:03:21.071846+0000","last_peered":"2024-09-20T22:03:21.071846+0000","last_clean":"2024-09-20T22:03:21.071846+0000","last_became_active":"2024-09-20T22:02:52.480591+0000","last_became_peered":"2024-09-20T22:02:52.480591+0000","last_unstale":"2024-09-20T22:03:21.071846+0000","last_undegraded":"2024-09-20T22:03:21.071846+0000","last_fullsized":"2024-09-20T22:03:21.071846+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.915698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703658+0000","last_change":"2024-09-20T22:02:50.112976+0000","last_active":"2024-09-20T22:03:20.703658+0000","last_peered":"2024-09-20T22:03:20.703658+0000","last_clean":"2024-09-20T22:03:20.703658+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:03:20.703658+0000","last_undegraded":"2024-09-20T22:03:20.703658+0000","last_fullsized":"2024-09-20T22:03:20.703658+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.057376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"71'8","reported_seq":43,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:30.172406+0000","last_change":"2024-09-20T22:02:54.292909+0000","last_active":"2024-09-20T22:03:30.172406+0000","last_peered":"2024-09-20T22:03:30.172406+0000","last_clean":"2024-09-20T22:03:30.172406+0000","last_became_active":"2024-09-20T22:02:54.289548+0000","last_became_peered":"2024-09-20T22:02:54.289548+0000","last_unstale":"2024-09-20T22:03:30.172406+0000","last_undegraded":"2024-09-20T22:03:30.172406+0000","last_fullsized":"2024-09-20T22:03:30.172406+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.478336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072333+0000","last_change":"2024-09-20T22:02:56.305690+0000","last_active":"2024-09-20T22:03:21.072333+0000","last_peered":"2024-09-20T22:03:21.072333+0000","last_clean":"2024-09-20T22:03:21.072333+0000","last_became_active":"2024-09-20T22:02:56.305383+0000","last_became_peered":"2024-09-20T22:02:56.305383+0000","last_unstale":"2024-09-20T22:03:21.072333+0000","last_undegraded":"2024-09-20T22:03:21.072333+0000","last_fullsized":"2024-09-20T22:03:21.072333+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.293958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"71'11","reported_seq":50,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279851+0000","last_change":"2024-09-20T22:02:52.277987+0000","last_active":"2024-09-20T22:03:35.279851+0000","last_peered":"2024-09-20T22:03:35.279851+0000","last_clean":"2024-09-20T22:03:35.279851+0000","last_became_active":"2024-09-20T22:02:52.277479+0000","last_became_peered":"2024-09-20T22:02:52.277479+0000","last_unstale":"2024-09-20T22:03:35.279851+0000","last_undegraded":"2024-09-20T22:03:35.279851+0000","last_fullsized":"2024-09-20T22:03:35.279851+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.222570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":35,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.280347+0000","last_change":"2024-09-20T22:02:50.109201+0000","last_active":"2024-09-20T22:03:35.280347+0000","last_peered":"2024-09-20T22:03:35.280347+0000","last_clean":"2024-09-20T22:03:35.280347+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:03:35.280347+0000","last_undegraded":"2024-09-20T22:03:35.280347+0000","last_fullsized":"2024-09-20T22:03:35.280347+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.129228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":27,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279001+0000","last_change":"2024-09-20T22:02:54.286502+0000","last_active":"2024-09-20T22:03:35.279001+0000","last_peered":"2024-09-20T22:03:35.279001+0000","last_clean":"2024-09-20T22:03:35.279001+0000","last_became_active":"2024-09-20T22:02:54.286200+0000","last_became_peered":"2024-09-20T22:02:54.286200+0000","last_unstale":"2024-09-20T22:03:35.279001+0000","last_undegraded":"2024-09-20T22:03:35.279001+0000","last_fullsized":"2024-09-20T22:03:35.279001+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:22.326441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070972+0000","last_change":"2024-09-20T22:02:56.307258+0000","last_active":"2024-09-20T22:03:21.070972+0000","last_peered":"2024-09-20T22:03:21.070972+0000","last_clean":"2024-09-20T22:03:21.070972+0000","last_became_active":"2024-09-20T22:02:56.306892+0000","last_became_peered":"2024-09-20T22:02:56.306892+0000","last_unstale":"2024-09-20T22:03:21.070972+0000","last_undegraded":"2024-09-20T22:03:21.070972+0000","last_fullsized":"2024-09-20T22:03:21.070972+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.948765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"71'9","reported_seq":47,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279119+0000","last_change":"2024-09-20T22:02:52.280815+0000","last_active":"2024-09-20T22:03:35.279119+0000","last_peered":"2024-09-20T22:03:35.279119+0000","last_clean":"2024-09-20T22:03:35.279119+0000","last_became_active":"2024-09-20T22:02:52.280428+0000","last_became_peered":"2024-09-20T22:02:52.280428+0000","last_unstale":"2024-09-20T22:03:35.279119+0000","last_undegraded":"2024-09-20T22:03:35.279119+0000","last_fullsized":"2024-09-20T22:03:35.279119+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.548215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703689+0000","last_change":"2024-09-20T22:02:50.110105+0000","last_active":"2024-09-20T22:03:20.703689+0000","last_peered":"2024-09-20T22:03:20.703689+0000","last_clean":"2024-09-20T22:03:20.703689+0000","last_became_active":"2024-09-20T22:02:50.109803+0000","last_became_peered":"2024-09-20T22:02:50.109803+0000","last_unstale":"2024-09-20T22:03:20.703689+0000","last_undegraded":"2024-09-20T22:03:20.703689+0000","last_fullsized":"2024-09-20T22:03:20.703689+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:53.587349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071838+0000","last_change":"2024-09-20T22:02:54.298130+0000","last_active":"2024-09-20T22:03:21.071838+0000","last_peered":"2024-09-20T22:03:21.071838+0000","last_clean":"2024-09-20T22:03:21.071838+0000","last_became_active":"2024-09-20T22:02:54.297803+0000","last_became_peered":"2024-09-20T22:02:54.297803+0000","last_unstale":"2024-09-20T22:03:21.071838+0000","last_undegraded":"2024-09-20T22:03:21.071838+0000","last_fullsized":"2024-09-20T22:03:21.071838+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.520061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070749+0000","last_change":"2024-09-20T22:02:56.298228+0000","last_active":"2024-09-20T22:03:21.070749+0000","last_peered":"2024-09-20T22:03:21.070749+0000","last_clean":"2024-09-20T22:03:21.070749+0000","last_became_active":"2024-09-20T22:02:56.297826+0000","last_became_peered":"2024-09-20T22:02:56.297826+0000","last_unstale":"2024-09-20T22:03:21.070749+0000","last_undegraded":"2024-09-20T22:03:21.070749+0000","last_fullsized":"2024-09-20T22:03:21.070749+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.126658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"71'11","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071102+0000","last_change":"2024-09-20T22:02:52.480504+0000","last_active":"2024-09-20T22:03:21.071102+0000","last_peered":"2024-09-20T22:03:21.071102+0000","last_clean":"2024-09-20T22:03:21.071102+0000","last_became_active":"2024-09-20T22:02:52.480118+0000","last_became_peered":"2024-09-20T22:02:52.480118+0000","last_unstale":"2024-09-20T22:03:21.071102+0000","last_undegraded":"2024-09-20T22:03:21.071102+0000","last_fullsized":"2024-09-20T22:03:21.071102+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.023916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071093+0000","last_change":"2024-09-20T22:02:50.110808+0000","last_active":"2024-09-20T22:03:21.071093+0000","last_peered":"2024-09-20T22:03:21.071093+0000","last_clean":"2024-09-20T22:03:21.071093+0000","last_became_active":"2024-09-20T22:02:50.110421+0000","last_became_peered":"2024-09-20T22:02:50.110421+0000","last_unstale":"2024-09-20T22:03:21.071093+0000","last_undegraded":"2024-09-20T22:03:21.071093+0000","last_fullsized":"2024-09-20T22:03:21.071093+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:53.027475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703032+0000","last_change":"2024-09-20T22:02:54.293068+0000","last_active":"2024-09-20T22:03:20.703032+0000","last_peered":"2024-09-20T22:03:20.703032+0000","last_clean":"2024-09-20T22:03:20.703032+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:03:20.703032+0000","last_undegraded":"2024-09-20T22:03:20.703032+0000","last_fullsized":"2024-09-20T22:03:20.703032+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.450710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072001+0000","last_change":"2024-09-20T22:02:56.348065+0000","last_active":"2024-09-20T22:03:21.072001+0000","last_peered":"2024-09-20T22:03:21.072001+0000","last_clean":"2024-09-20T22:03:21.072001+0000","last_became_active":"2024-09-20T22:02:56.347637+0000","last_became_peered":"2024-09-20T22:02:56.347637+0000","last_unstale":"2024-09-20T22:03:21.072001+0000","last_undegraded":"2024-09-20T22:03:21.072001+0000","last_fullsized":"2024-09-20T22:03:21.072001+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:13.568557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"71'4","reported_seq":35,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070523+0000","last_change":"2024-09-20T22:02:52.286086+0000","last_active":"2024-09-20T22:03:21.070523+0000","last_peered":"2024-09-20T22:03:21.070523+0000","last_clean":"2024-09-20T22:03:21.070523+0000","last_became_active":"2024-09-20T22:02:52.285701+0000","last_became_peered":"2024-09-20T22:02:52.285701+0000","last_unstale":"2024-09-20T22:03:21.070523+0000","last_undegraded":"2024-09-20T22:03:21.070523+0000","last_fullsized":"2024-09-20T22:03:21.070523+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.474692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072586+0000","last_change":"2024-09-20T22:02:50.111749+0000","last_active":"2024-09-20T22:03:21.072586+0000","last_peered":"2024-09-20T22:03:21.072586+0000","last_clean":"2024-09-20T22:03:21.072586+0000","last_became_active":"2024-09-20T22:02:50.111142+0000","last_became_peered":"2024-09-20T22:02:50.111142+0000","last_unstale":"2024-09-20T22:03:21.072586+0000","last_undegraded":"2024-09-20T22:03:21.072586+0000","last_fullsized":"2024-09-20T22:03:21.072586+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:57.037771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070393+0000","last_change":"2024-09-20T22:02:54.290418+0000","last_active":"2024-09-20T22:03:21.070393+0000","last_peered":"2024-09-20T22:03:21.070393+0000","last_clean":"2024-09-20T22:03:21.070393+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:03:21.070393+0000","last_undegraded":"2024-09-20T22:03:21.070393+0000","last_fullsized":"2024-09-20T22:03:21.070393+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:57.209065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:20.703153+0000","last_change":"2024-09-20T22:02:56.300559+0000","last_active":"2024-09-20T22:03:20.703153+0000","last_peered":"2024-09-20T22:03:20.703153+0000","last_clean":"2024-09-20T22:03:20.703153+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:03:20.703153+0000","last_undegraded":"2024-09-20T22:03:20.703153+0000","last_fullsized":"2024-09-20T22:03:20.703153+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.506369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":23,"reported_epoch":76,"state":"active+clean","last_fresh":"2024-09-20T22:03:35.279492+0000","last_change":"2024-09-20T22:02:56.301611+0000","last_active":"2024-09-20T22:03:35.279492+0000","last_peered":"2024-09-20T22:03:35.279492+0000","last_clean":"2024-09-20T22:03:35.279492+0000","last_became_active":"2024-09-20T22:02:56.301247+0000","last_became_peered":"2024-09-20T22:02:56.301247+0000","last_unstale":"2024-09-20T22:03:35.279492+0000","last_undegraded":"2024-09-20T22:03:35.279492+0000","last_fullsized":"2024-09-20T22:03:35.279492+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.486378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.070755+0000","last_change":"2024-09-20T22:02:50.114519+0000","last_active":"2024-09-20T22:03:21.070755+0000","last_peered":"2024-09-20T22:03:21.070755+0000","last_clean":"2024-09-20T22:03:21.070755+0000","last_became_active":"2024-09-20T22:02:50.114204+0000","last_became_peered":"2024-09-20T22:02:50.114204+0000","last_unstale":"2024-09-20T22:03:21.070755+0000","last_undegraded":"2024-09-20T22:03:21.070755+0000","last_fullsized":"2024-09-20T22:03:21.070755+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:50.725534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"71'11","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.071757+0000","last_change":"2024-09-20T22:02:52.279065+0000","last_active":"2024-09-20T22:03:21.071757+0000","last_peered":"2024-09-20T22:03:21.071757+0000","last_clean":"2024-09-20T22:03:21.071757+0000","last_became_active":"2024-09-20T22:02:52.278657+0000","last_became_peered":"2024-09-20T22:02:52.278657+0000","last_unstale":"2024-09-20T22:03:21.071757+0000","last_undegraded":"2024-09-20T22:03:21.071757+0000","last_fullsized":"2024-09-20T22:03:21.071757+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.405988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":25,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-20T22:03:21.072426+0000","last_change":"2024-09-20T22:02:54.289401+0000","last_active":"2024-09-20T22:03:21.072426+0000","last_peered":"2024-09-20T22:03:21.072426+0000","last_clean":"2024-09-20T22:03:21.072426+0000","last_became_active":"2024-09-20T22:02:54.289078+0000","last_became_peered":"2024-09-20T22:02:54.289078+0000","last_unstale":"2024-09-20T22:03:21.072426+0000","last_undegraded":"2024-09-20T22:03:21.072426+0000","last_fullsized":"2024-09-20T22:03:21.072426+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.343711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":34,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135885,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32512,"kb_used_data":1324,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712640,"statfs":{"total":95995035648,"available":95961743360,"internally_reserved":0,"allocated":1355776,"data_stored":923901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430225,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32492,"kb_used_data":1308,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712660,"statfs":{"total":95995035648,"available":95961763840,"internally_reserved":0,"allocated":1339392,"data_stored":923809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724565,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27756,"kb_used_data":728,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717396,"statfs":{"total":95995035648,"available":95966613504,"internally_reserved":0,"allocated":745472,"data_stored":333787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986205,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31944,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713208,"statfs":{"total":95995035648,"available":95962324992,"internally_reserved":0,"allocated":770048,"data_stored":333476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":802816,"data_stored":334220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149730,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31960,"kb_used_data":768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713192,"statfs":{"total":95995035648,"available":95962308608,"internally_reserved":0,"allocated":786432,"data_stored":334768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411367,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27748,"kb_used_data":716,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717404,"statfs":{"total":95995035648,"available":95966621696,"internally_reserved":0,"allocated":733184,"data_stored":332583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738418,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32520,"kb_used_data":1328,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712632,"statfs":{"total":95995035648,"available":95961735168,"internally_reserved":0,"allocated":1359872,"data_stored":924033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:03:40.972 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:03:41.568 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:41 smithi060 ceph-mon[29694]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-20T22:03:41.568 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:41 smithi060 ceph-mon[29694]: from='client.24632 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:03:41.568 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:41 smithi060 ceph-mon[29694]: osdmap e78: 8 total, 8 up, 8 in 2024-09-20T22:03:41.568 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:41 smithi060 ceph-mon[40290]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-20T22:03:41.568 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:41 smithi060 ceph-mon[40290]: from='client.24632 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:03:41.568 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:41 smithi060 ceph-mon[40290]: osdmap e78: 8 total, 8 up, 8 in 2024-09-20T22:03:41.710 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:41 smithi067 ceph-mon[33511]: pgmap v16: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-20T22:03:41.710 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:41 smithi067 ceph-mon[33511]: from='client.24632 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:03:41.710 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:41 smithi067 ceph-mon[33511]: osdmap e78: 8 total, 8 up, 8 in 2024-09-20T22:03:42.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:41 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:03:42.304 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:43 smithi060 ceph-mon[29694]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-20T22:03:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:43 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:03:43.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:43 smithi060 ceph-mon[40290]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-20T22:03:43.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:43 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:03:43.571 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:03:43.573 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:03:43.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:43 smithi067 ceph-mon[33511]: pgmap v18: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-20T22:03:43.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:43 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:03:44.090 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-20T22:03:43.536774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":978,"num_read_kb":824,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":601,"ondisk_log_size":601,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":244932,"kb_used_data":7900,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":749716284,"statfs":{"total":767960285184,"available":767709474816,"internally_reserved":0,"allocated":8089600,"data_stored":4563313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":242535984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001698"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.295450+0000","last_change":"2024-09-20T22:02:50.111666+0000","last_active":"2024-09-20T22:03:36.295450+0000","last_peered":"2024-09-20T22:03:36.295450+0000","last_clean":"2024-09-20T22:03:36.295450+0000","last_became_active":"2024-09-20T22:02:50.109930+0000","last_became_peered":"2024-09-20T22:02:50.109930+0000","last_unstale":"2024-09-20T22:03:36.295450+0000","last_undegraded":"2024-09-20T22:03:36.295450+0000","last_fullsized":"2024-09-20T22:03:36.295450+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.513745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"71'9","reported_seq":49,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.270889+0000","last_change":"2024-09-20T22:02:52.277916+0000","last_active":"2024-09-20T22:03:36.270889+0000","last_peered":"2024-09-20T22:03:36.270889+0000","last_clean":"2024-09-20T22:03:36.270889+0000","last_became_active":"2024-09-20T22:02:52.277616+0000","last_became_peered":"2024-09-20T22:02:52.277616+0000","last_unstale":"2024-09-20T22:03:36.270889+0000","last_undegraded":"2024-09-20T22:03:36.270889+0000","last_fullsized":"2024-09-20T22:03:36.270889+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.412488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.268042+0000","last_change":"2024-09-20T22:02:54.287938+0000","last_active":"2024-09-20T22:03:36.268042+0000","last_peered":"2024-09-20T22:03:36.268042+0000","last_clean":"2024-09-20T22:03:36.268042+0000","last_became_active":"2024-09-20T22:02:54.287537+0000","last_became_peered":"2024-09-20T22:02:54.287537+0000","last_unstale":"2024-09-20T22:03:36.268042+0000","last_undegraded":"2024-09-20T22:03:36.268042+0000","last_fullsized":"2024-09-20T22:03:36.268042+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.152471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.271094+0000","last_change":"2024-09-20T22:02:56.297551+0000","last_active":"2024-09-20T22:03:36.271094+0000","last_peered":"2024-09-20T22:03:36.271094+0000","last_clean":"2024-09-20T22:03:36.271094+0000","last_became_active":"2024-09-20T22:02:56.297252+0000","last_became_peered":"2024-09-20T22:02:56.297252+0000","last_unstale":"2024-09-20T22:03:36.271094+0000","last_undegraded":"2024-09-20T22:03:36.271094+0000","last_fullsized":"2024-09-20T22:03:36.271094+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.977495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.617261+0000","last_change":"2024-09-20T22:02:56.350387+0000","last_active":"2024-09-20T22:03:36.617261+0000","last_peered":"2024-09-20T22:03:36.617261+0000","last_clean":"2024-09-20T22:03:36.617261+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:03:36.617261+0000","last_undegraded":"2024-09-20T22:03:36.617261+0000","last_fullsized":"2024-09-20T22:03:36.617261+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.462175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.618363+0000","last_change":"2024-09-20T22:02:50.107457+0000","last_active":"2024-09-20T22:03:36.618363+0000","last_peered":"2024-09-20T22:03:36.618363+0000","last_clean":"2024-09-20T22:03:36.618363+0000","last_became_active":"2024-09-20T22:02:50.107154+0000","last_became_peered":"2024-09-20T22:02:50.107154+0000","last_unstale":"2024-09-20T22:03:36.618363+0000","last_undegraded":"2024-09-20T22:03:36.618363+0000","last_fullsized":"2024-09-20T22:03:36.618363+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:11.498961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"71'15","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.617083+0000","last_change":"2024-09-20T22:02:52.282478+0000","last_active":"2024-09-20T22:03:36.617083+0000","last_peered":"2024-09-20T22:03:36.617083+0000","last_clean":"2024-09-20T22:03:36.617083+0000","last_became_active":"2024-09-20T22:02:52.282101+0000","last_became_peered":"2024-09-20T22:02:52.282101+0000","last_unstale":"2024-09-20T22:03:36.617083+0000","last_undegraded":"2024-09-20T22:03:36.617083+0000","last_fullsized":"2024-09-20T22:03:36.617083+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.208571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.270838+0000","last_change":"2024-09-20T22:02:54.292666+0000","last_active":"2024-09-20T22:03:36.270838+0000","last_peered":"2024-09-20T22:03:36.270838+0000","last_clean":"2024-09-20T22:03:36.270838+0000","last_became_active":"2024-09-20T22:02:54.292343+0000","last_became_peered":"2024-09-20T22:02:54.292343+0000","last_unstale":"2024-09-20T22:03:36.270838+0000","last_undegraded":"2024-09-20T22:03:36.270838+0000","last_fullsized":"2024-09-20T22:03:36.270838+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.965688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.587818+0000","last_change":"2024-09-20T22:02:50.108309+0000","last_active":"2024-09-20T22:03:36.587818+0000","last_peered":"2024-09-20T22:03:36.587818+0000","last_clean":"2024-09-20T22:03:36.587818+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:03:36.587818+0000","last_undegraded":"2024-09-20T22:03:36.587818+0000","last_fullsized":"2024-09-20T22:03:36.587818+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.039813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"71'9","reported_seq":49,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.271409+0000","last_change":"2024-09-20T22:02:52.279933+0000","last_active":"2024-09-20T22:03:36.271409+0000","last_peered":"2024-09-20T22:03:36.271409+0000","last_clean":"2024-09-20T22:03:36.271409+0000","last_became_active":"2024-09-20T22:02:52.279526+0000","last_became_peered":"2024-09-20T22:02:52.279526+0000","last_unstale":"2024-09-20T22:03:36.271409+0000","last_undegraded":"2024-09-20T22:03:36.271409+0000","last_fullsized":"2024-09-20T22:03:36.271409+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.714248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.587247+0000","last_change":"2024-09-20T22:02:54.293254+0000","last_active":"2024-09-20T22:03:36.587247+0000","last_peered":"2024-09-20T22:03:36.587247+0000","last_clean":"2024-09-20T22:03:36.587247+0000","last_became_active":"2024-09-20T22:02:54.292911+0000","last_became_peered":"2024-09-20T22:02:54.292911+0000","last_unstale":"2024-09-20T22:03:36.587247+0000","last_undegraded":"2024-09-20T22:03:36.587247+0000","last_fullsized":"2024-09-20T22:03:36.587247+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.908407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.294189+0000","last_change":"2024-09-20T22:02:56.299059+0000","last_active":"2024-09-20T22:03:36.294189+0000","last_peered":"2024-09-20T22:03:36.294189+0000","last_clean":"2024-09-20T22:03:36.294189+0000","last_became_active":"2024-09-20T22:02:56.298660+0000","last_became_peered":"2024-09-20T22:02:56.298660+0000","last_unstale":"2024-09-20T22:03:36.294189+0000","last_undegraded":"2024-09-20T22:03:36.294189+0000","last_fullsized":"2024-09-20T22:03:36.294189+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:13.896683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.587630+0000","last_change":"2024-09-20T22:02:50.106885+0000","last_active":"2024-09-20T22:03:36.587630+0000","last_peered":"2024-09-20T22:03:36.587630+0000","last_clean":"2024-09-20T22:03:36.587630+0000","last_became_active":"2024-09-20T22:02:50.106555+0000","last_became_peered":"2024-09-20T22:02:50.106555+0000","last_unstale":"2024-09-20T22:03:36.587630+0000","last_undegraded":"2024-09-20T22:03:36.587630+0000","last_fullsized":"2024-09-20T22:03:36.587630+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.204880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"71'5","reported_seq":43,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.271101+0000","last_change":"2024-09-20T22:02:52.284347+0000","last_active":"2024-09-20T22:03:36.271101+0000","last_peered":"2024-09-20T22:03:36.271101+0000","last_clean":"2024-09-20T22:03:36.271101+0000","last_became_active":"2024-09-20T22:02:52.283904+0000","last_became_peered":"2024-09-20T22:02:52.283904+0000","last_unstale":"2024-09-20T22:03:36.271101+0000","last_undegraded":"2024-09-20T22:03:36.271101+0000","last_fullsized":"2024-09-20T22:03:36.271101+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:52.941046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.589808+0000","last_change":"2024-09-20T22:02:54.295625+0000","last_active":"2024-09-20T22:03:36.589808+0000","last_peered":"2024-09-20T22:03:36.589808+0000","last_clean":"2024-09-20T22:03:36.589808+0000","last_became_active":"2024-09-20T22:02:54.294405+0000","last_became_peered":"2024-09-20T22:02:54.294405+0000","last_unstale":"2024-09-20T22:03:36.589808+0000","last_undegraded":"2024-09-20T22:03:36.589808+0000","last_fullsized":"2024-09-20T22:03:36.589808+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:53.468504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.586449+0000","last_change":"2024-09-20T22:02:56.303701+0000","last_active":"2024-09-20T22:03:36.586449+0000","last_peered":"2024-09-20T22:03:36.586449+0000","last_clean":"2024-09-20T22:03:36.586449+0000","last_became_active":"2024-09-20T22:02:56.303338+0000","last_became_peered":"2024-09-20T22:02:56.303338+0000","last_unstale":"2024-09-20T22:03:36.586449+0000","last_undegraded":"2024-09-20T22:03:36.586449+0000","last_fullsized":"2024-09-20T22:03:36.586449+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.793299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.271826+0000","last_change":"2024-09-20T22:02:56.347311+0000","last_active":"2024-09-20T22:03:36.271826+0000","last_peered":"2024-09-20T22:03:36.271826+0000","last_clean":"2024-09-20T22:03:36.271826+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:03:36.271826+0000","last_undegraded":"2024-09-20T22:03:36.271826+0000","last_fullsized":"2024-09-20T22:03:36.271826+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.297557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.295119+0000","last_change":"2024-09-20T22:02:50.111842+0000","last_active":"2024-09-20T22:03:36.295119+0000","last_peered":"2024-09-20T22:03:36.295119+0000","last_clean":"2024-09-20T22:03:36.295119+0000","last_became_active":"2024-09-20T22:02:50.111517+0000","last_became_peered":"2024-09-20T22:02:50.111517+0000","last_unstale":"2024-09-20T22:03:36.295119+0000","last_undegraded":"2024-09-20T22:03:36.295119+0000","last_fullsized":"2024-09-20T22:03:36.295119+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.747690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"71'15","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.618066+0000","last_change":"2024-09-20T22:02:52.283078+0000","last_active":"2024-09-20T22:03:36.618066+0000","last_peered":"2024-09-20T22:03:36.618066+0000","last_clean":"2024-09-20T22:03:36.618066+0000","last_became_active":"2024-09-20T22:02:52.282768+0000","last_became_peered":"2024-09-20T22:02:52.282768+0000","last_unstale":"2024-09-20T22:03:36.618066+0000","last_undegraded":"2024-09-20T22:03:36.618066+0000","last_fullsized":"2024-09-20T22:03:36.618066+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:52.589023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.268788+0000","last_change":"2024-09-20T22:02:54.292892+0000","last_active":"2024-09-20T22:03:36.268788+0000","last_peered":"2024-09-20T22:03:36.268788+0000","last_clean":"2024-09-20T22:03:36.268788+0000","last_became_active":"2024-09-20T22:02:54.290510+0000","last_became_peered":"2024-09-20T22:02:54.290510+0000","last_unstale":"2024-09-20T22:03:36.268788+0000","last_undegraded":"2024-09-20T22:03:36.268788+0000","last_fullsized":"2024-09-20T22:03:36.268788+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.173080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.618309+0000","last_change":"2024-09-20T22:02:56.347951+0000","last_active":"2024-09-20T22:03:36.618309+0000","last_peered":"2024-09-20T22:03:36.618309+0000","last_clean":"2024-09-20T22:03:36.618309+0000","last_became_active":"2024-09-20T22:02:56.347561+0000","last_became_peered":"2024-09-20T22:02:56.347561+0000","last_unstale":"2024-09-20T22:03:36.618309+0000","last_undegraded":"2024-09-20T22:03:36.618309+0000","last_fullsized":"2024-09-20T22:03:36.618309+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.255072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.271901+0000","last_change":"2024-09-20T22:02:50.108909+0000","last_active":"2024-09-20T22:03:36.271901+0000","last_peered":"2024-09-20T22:03:36.271901+0000","last_clean":"2024-09-20T22:03:36.271901+0000","last_became_active":"2024-09-20T22:02:50.107189+0000","last_became_peered":"2024-09-20T22:02:50.107189+0000","last_unstale":"2024-09-20T22:03:36.271901+0000","last_undegraded":"2024-09-20T22:03:36.271901+0000","last_fullsized":"2024-09-20T22:03:36.271901+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.636681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"71'12","reported_seq":55,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.617954+0000","last_change":"2024-09-20T22:02:52.284201+0000","last_active":"2024-09-20T22:03:36.617954+0000","last_peered":"2024-09-20T22:03:36.617954+0000","last_clean":"2024-09-20T22:03:36.617954+0000","last_became_active":"2024-09-20T22:02:52.283843+0000","last_became_peered":"2024-09-20T22:02:52.283843+0000","last_unstale":"2024-09-20T22:03:36.617954+0000","last_undegraded":"2024-09-20T22:03:36.617954+0000","last_fullsized":"2024-09-20T22:03:36.617954+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.131718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.271236+0000","last_change":"2024-09-20T22:02:54.293157+0000","last_active":"2024-09-20T22:03:36.271236+0000","last_peered":"2024-09-20T22:03:36.271236+0000","last_clean":"2024-09-20T22:03:36.271236+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:03:36.271236+0000","last_undegraded":"2024-09-20T22:03:36.271236+0000","last_fullsized":"2024-09-20T22:03:36.271236+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.203423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"71'1","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.589230+0000","last_change":"2024-09-20T22:02:56.301720+0000","last_active":"2024-09-20T22:03:36.589230+0000","last_peered":"2024-09-20T22:03:36.589230+0000","last_clean":"2024-09-20T22:03:36.589230+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:03:36.589230+0000","last_undegraded":"2024-09-20T22:03:36.589230+0000","last_fullsized":"2024-09-20T22:03:36.589230+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.341799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"64'1","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.268122+0000","last_change":"2024-09-20T22:02:50.113259+0000","last_active":"2024-09-20T22:03:36.268122+0000","last_peered":"2024-09-20T22:03:36.268122+0000","last_clean":"2024-09-20T22:03:36.268122+0000","last_became_active":"2024-09-20T22:02:50.112862+0000","last_became_peered":"2024-09-20T22:02:50.112862+0000","last_unstale":"2024-09-20T22:03:36.268122+0000","last_undegraded":"2024-09-20T22:03:36.268122+0000","last_fullsized":"2024-09-20T22:03:36.268122+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.645807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"71'10","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.295238+0000","last_change":"2024-09-20T22:02:52.482300+0000","last_active":"2024-09-20T22:03:36.295238+0000","last_peered":"2024-09-20T22:03:36.295238+0000","last_clean":"2024-09-20T22:03:36.295238+0000","last_became_active":"2024-09-20T22:02:52.481980+0000","last_became_peered":"2024-09-20T22:02:52.481980+0000","last_unstale":"2024-09-20T22:03:36.295238+0000","last_undegraded":"2024-09-20T22:03:36.295238+0000","last_fullsized":"2024-09-20T22:03:36.295238+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.082587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"71'8","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.433682+0000","last_change":"2024-09-20T22:02:54.297973+0000","last_active":"2024-09-20T22:03:36.433682+0000","last_peered":"2024-09-20T22:03:36.433682+0000","last_clean":"2024-09-20T22:03:36.433682+0000","last_became_active":"2024-09-20T22:02:54.297657+0000","last_became_peered":"2024-09-20T22:02:54.297657+0000","last_unstale":"2024-09-20T22:03:36.433682+0000","last_undegraded":"2024-09-20T22:03:36.433682+0000","last_fullsized":"2024-09-20T22:03:36.433682+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:20.633207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"71'15","reported_seq":58,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.269129+0000","last_change":"2024-09-20T22:02:52.284890+0000","last_active":"2024-09-20T22:03:36.269129+0000","last_peered":"2024-09-20T22:03:36.269129+0000","last_clean":"2024-09-20T22:03:36.269129+0000","last_became_active":"2024-09-20T22:02:52.284027+0000","last_became_peered":"2024-09-20T22:02:52.284027+0000","last_unstale":"2024-09-20T22:03:36.269129+0000","last_undegraded":"2024-09-20T22:03:36.269129+0000","last_fullsized":"2024-09-20T22:03:36.269129+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:54.544010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.616821+0000","last_change":"2024-09-20T22:02:50.108849+0000","last_active":"2024-09-20T22:03:36.616821+0000","last_peered":"2024-09-20T22:03:36.616821+0000","last_clean":"2024-09-20T22:03:36.616821+0000","last_became_active":"2024-09-20T22:02:50.108508+0000","last_became_peered":"2024-09-20T22:02:50.108508+0000","last_unstale":"2024-09-20T22:03:36.616821+0000","last_undegraded":"2024-09-20T22:03:36.616821+0000","last_fullsized":"2024-09-20T22:03:36.616821+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.430379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"71'8","reported_seq":51,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.271913+0000","last_change":"2024-09-20T22:02:54.295408+0000","last_active":"2024-09-20T22:03:36.271913+0000","last_peered":"2024-09-20T22:03:36.271913+0000","last_clean":"2024-09-20T22:03:36.271913+0000","last_became_active":"2024-09-20T22:02:54.294917+0000","last_became_peered":"2024-09-20T22:02:54.294917+0000","last_unstale":"2024-09-20T22:03:36.271913+0000","last_undegraded":"2024-09-20T22:03:36.271913+0000","last_fullsized":"2024-09-20T22:03:36.271913+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.541024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.586525+0000","last_change":"2024-09-20T22:02:56.296725+0000","last_active":"2024-09-20T22:03:36.586525+0000","last_peered":"2024-09-20T22:03:36.586525+0000","last_clean":"2024-09-20T22:03:36.586525+0000","last_became_active":"2024-09-20T22:02:56.296436+0000","last_became_peered":"2024-09-20T22:02:56.296436+0000","last_unstale":"2024-09-20T22:03:36.586525+0000","last_undegraded":"2024-09-20T22:03:36.586525+0000","last_fullsized":"2024-09-20T22:03:36.586525+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:19.201794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.0","version":"71'18","reported_seq":64,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.616669+0000","last_change":"2024-09-20T22:02:52.482408+0000","last_active":"2024-09-20T22:03:36.616669+0000","last_peered":"2024-09-20T22:03:36.616669+0000","last_clean":"2024-09-20T22:03:36.616669+0000","last_became_active":"2024-09-20T22:02:52.482119+0000","last_became_peered":"2024-09-20T22:02:52.482119+0000","last_unstale":"2024-09-20T22:03:36.616669+0000","last_undegraded":"2024-09-20T22:03:36.616669+0000","last_fullsized":"2024-09-20T22:03:36.616669+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.477571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.618787+0000","last_change":"2024-09-20T22:02:50.113398+0000","last_active":"2024-09-20T22:03:36.618787+0000","last_peered":"2024-09-20T22:03:36.618787+0000","last_clean":"2024-09-20T22:03:36.618787+0000","last_became_active":"2024-09-20T22:02:50.113076+0000","last_became_peered":"2024-09-20T22:02:50.113076+0000","last_unstale":"2024-09-20T22:03:36.618787+0000","last_undegraded":"2024-09-20T22:03:36.618787+0000","last_fullsized":"2024-09-20T22:03:36.618787+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.714101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.270999+0000","last_change":"2024-09-20T22:02:54.288282+0000","last_active":"2024-09-20T22:03:36.270999+0000","last_peered":"2024-09-20T22:03:36.270999+0000","last_clean":"2024-09-20T22:03:36.270999+0000","last_became_active":"2024-09-20T22:02:54.288007+0000","last_became_peered":"2024-09-20T22:02:54.288007+0000","last_unstale":"2024-09-20T22:03:36.270999+0000","last_undegraded":"2024-09-20T22:03:36.270999+0000","last_fullsized":"2024-09-20T22:03:36.270999+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.737144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.271359+0000","last_change":"2024-09-20T22:02:56.303051+0000","last_active":"2024-09-20T22:03:36.271359+0000","last_peered":"2024-09-20T22:03:36.271359+0000","last_clean":"2024-09-20T22:03:36.271359+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:03:36.271359+0000","last_undegraded":"2024-09-20T22:03:36.271359+0000","last_fullsized":"2024-09-20T22:03:36.271359+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.012094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"71'14","reported_seq":54,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.271263+0000","last_change":"2024-09-20T22:02:52.273553+0000","last_active":"2024-09-20T22:03:36.271263+0000","last_peered":"2024-09-20T22:03:36.271263+0000","last_clean":"2024-09-20T22:03:36.271263+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:03:36.271263+0000","last_undegraded":"2024-09-20T22:03:36.271263+0000","last_fullsized":"2024-09-20T22:03:36.271263+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:56.047961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.295421+0000","last_change":"2024-09-20T22:02:50.110546+0000","last_active":"2024-09-20T22:03:36.295421+0000","last_peered":"2024-09-20T22:03:36.295421+0000","last_clean":"2024-09-20T22:03:36.295421+0000","last_became_active":"2024-09-20T22:02:50.110195+0000","last_became_peered":"2024-09-20T22:02:50.110195+0000","last_unstale":"2024-09-20T22:03:36.295421+0000","last_undegraded":"2024-09-20T22:03:36.295421+0000","last_fullsized":"2024-09-20T22:03:36.295421+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.867716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"71'8","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.616573+0000","last_change":"2024-09-20T22:02:54.295841+0000","last_active":"2024-09-20T22:03:36.616573+0000","last_peered":"2024-09-20T22:03:36.616573+0000","last_clean":"2024-09-20T22:03:36.616573+0000","last_became_active":"2024-09-20T22:02:54.295553+0000","last_became_peered":"2024-09-20T22:02:54.295553+0000","last_unstale":"2024-09-20T22:03:36.616573+0000","last_undegraded":"2024-09-20T22:03:36.616573+0000","last_fullsized":"2024-09-20T22:03:36.616573+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.625795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.589210+0000","last_change":"2024-09-20T22:02:56.349864+0000","last_active":"2024-09-20T22:03:36.589210+0000","last_peered":"2024-09-20T22:03:36.589210+0000","last_clean":"2024-09-20T22:03:36.589210+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:03:36.589210+0000","last_undegraded":"2024-09-20T22:03:36.589210+0000","last_fullsized":"2024-09-20T22:03:36.589210+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:57.678673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2","version":"71'10","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.268334+0000","last_change":"2024-09-20T22:02:52.276282+0000","last_active":"2024-09-20T22:03:36.268334+0000","last_peered":"2024-09-20T22:03:36.268334+0000","last_clean":"2024-09-20T22:03:36.268334+0000","last_became_active":"2024-09-20T22:02:52.275958+0000","last_became_peered":"2024-09-20T22:02:52.275958+0000","last_unstale":"2024-09-20T22:03:36.268334+0000","last_undegraded":"2024-09-20T22:03:36.268334+0000","last_fullsized":"2024-09-20T22:03:36.268334+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.846987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.587980+0000","last_change":"2024-09-20T22:02:50.114476+0000","last_active":"2024-09-20T22:03:36.587980+0000","last_peered":"2024-09-20T22:03:36.587980+0000","last_clean":"2024-09-20T22:03:36.587980+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:03:36.587980+0000","last_undegraded":"2024-09-20T22:03:36.587980+0000","last_fullsized":"2024-09-20T22:03:36.587980+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:51.340463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"71'8","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.294636+0000","last_change":"2024-09-20T22:02:54.293220+0000","last_active":"2024-09-20T22:03:36.294636+0000","last_peered":"2024-09-20T22:03:36.294636+0000","last_clean":"2024-09-20T22:03:36.294636+0000","last_became_active":"2024-09-20T22:02:54.292819+0000","last_became_peered":"2024-09-20T22:02:54.292819+0000","last_unstale":"2024-09-20T22:03:36.294636+0000","last_undegraded":"2024-09-20T22:03:36.294636+0000","last_fullsized":"2024-09-20T22:03:36.294636+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:59.827983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.294918+0000","last_change":"2024-09-20T22:02:56.304595+0000","last_active":"2024-09-20T22:03:36.294918+0000","last_peered":"2024-09-20T22:03:36.294918+0000","last_clean":"2024-09-20T22:03:36.294918+0000","last_became_active":"2024-09-20T22:02:56.304274+0000","last_became_peered":"2024-09-20T22:02:56.304274+0000","last_unstale":"2024-09-20T22:03:36.294918+0000","last_undegraded":"2024-09-20T22:03:36.294918+0000","last_fullsized":"2024-09-20T22:03:36.294918+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.850583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"71'19","reported_seq":68,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.294884+0000","last_change":"2024-09-20T22:02:52.480446+0000","last_active":"2024-09-20T22:03:36.294884+0000","last_peered":"2024-09-20T22:03:36.294884+0000","last_clean":"2024-09-20T22:03:36.294884+0000","last_became_active":"2024-09-20T22:02:52.480077+0000","last_became_peered":"2024-09-20T22:02:52.480077+0000","last_unstale":"2024-09-20T22:03:36.294884+0000","last_undegraded":"2024-09-20T22:03:36.294884+0000","last_fullsized":"2024-09-20T22:03:36.294884+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.830883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.268653+0000","last_change":"2024-09-20T22:02:50.104649+0000","last_active":"2024-09-20T22:03:36.268653+0000","last_peered":"2024-09-20T22:03:36.268653+0000","last_clean":"2024-09-20T22:03:36.268653+0000","last_became_active":"2024-09-20T22:02:50.104345+0000","last_became_peered":"2024-09-20T22:02:50.104345+0000","last_unstale":"2024-09-20T22:03:36.268653+0000","last_undegraded":"2024-09-20T22:03:36.268653+0000","last_fullsized":"2024-09-20T22:03:36.268653+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.766875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.433068+0000","last_change":"2024-09-20T22:02:54.291638+0000","last_active":"2024-09-20T22:03:36.433068+0000","last_peered":"2024-09-20T22:03:36.433068+0000","last_clean":"2024-09-20T22:03:36.433068+0000","last_became_active":"2024-09-20T22:02:54.290787+0000","last_became_peered":"2024-09-20T22:02:54.290787+0000","last_unstale":"2024-09-20T22:03:36.433068+0000","last_undegraded":"2024-09-20T22:03:36.433068+0000","last_fullsized":"2024-09-20T22:03:36.433068+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.996942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.267956+0000","last_change":"2024-09-20T22:02:56.350020+0000","last_active":"2024-09-20T22:03:36.267956+0000","last_peered":"2024-09-20T22:03:36.267956+0000","last_clean":"2024-09-20T22:03:36.267956+0000","last_became_active":"2024-09-20T22:02:56.349581+0000","last_became_peered":"2024-09-20T22:02:56.349581+0000","last_unstale":"2024-09-20T22:03:36.267956+0000","last_undegraded":"2024-09-20T22:03:36.267956+0000","last_fullsized":"2024-09-20T22:03:36.267956+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.722790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.4","version":"71'28","reported_seq":92,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.268644+0000","last_change":"2024-09-20T22:02:52.285811+0000","last_active":"2024-09-20T22:03:36.268644+0000","last_peered":"2024-09-20T22:03:36.268644+0000","last_clean":"2024-09-20T22:03:36.268644+0000","last_became_active":"2024-09-20T22:02:52.285341+0000","last_became_peered":"2024-09-20T22:02:52.285341+0000","last_unstale":"2024-09-20T22:03:36.268644+0000","last_undegraded":"2024-09-20T22:03:36.268644+0000","last_fullsized":"2024-09-20T22:03:36.268644+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.491983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.271533+0000","last_change":"2024-09-20T22:02:50.110040+0000","last_active":"2024-09-20T22:03:36.271533+0000","last_peered":"2024-09-20T22:03:36.271533+0000","last_clean":"2024-09-20T22:03:36.271533+0000","last_became_active":"2024-09-20T22:02:50.108457+0000","last_became_peered":"2024-09-20T22:02:50.108457+0000","last_unstale":"2024-09-20T22:03:36.271533+0000","last_undegraded":"2024-09-20T22:03:36.271533+0000","last_fullsized":"2024-09-20T22:03:36.271533+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.280284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"66'2","reported_seq":46,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.587537+0000","last_change":"2024-09-20T22:02:52.273740+0000","last_active":"2024-09-20T22:03:36.587537+0000","last_peered":"2024-09-20T22:03:36.587537+0000","last_clean":"2024-09-20T22:03:36.587537+0000","last_became_active":"2024-09-20T22:02:49.096453+0000","last_became_peered":"2024-09-20T22:02:49.096453+0000","last_unstale":"2024-09-20T22:03:36.587537+0000","last_undegraded":"2024-09-20T22:03:36.587537+0000","last_fullsized":"2024-09-20T22:03:36.587537+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.811834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.294599+0000","last_change":"2024-09-20T22:02:54.295696+0000","last_active":"2024-09-20T22:03:36.294599+0000","last_peered":"2024-09-20T22:03:36.294599+0000","last_clean":"2024-09-20T22:03:36.294599+0000","last_became_active":"2024-09-20T22:02:54.295401+0000","last_became_peered":"2024-09-20T22:02:54.295401+0000","last_unstale":"2024-09-20T22:03:36.294599+0000","last_undegraded":"2024-09-20T22:03:36.294599+0000","last_fullsized":"2024-09-20T22:03:36.294599+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.933605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.617423+0000","last_change":"2024-09-20T22:02:56.306053+0000","last_active":"2024-09-20T22:03:36.617423+0000","last_peered":"2024-09-20T22:03:36.617423+0000","last_clean":"2024-09-20T22:03:36.617423+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:03:36.617423+0000","last_undegraded":"2024-09-20T22:03:36.617423+0000","last_fullsized":"2024-09-20T22:03:36.617423+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.682717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"71'13","reported_seq":60,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.269100+0000","last_change":"2024-09-20T22:02:52.280309+0000","last_active":"2024-09-20T22:03:36.269100+0000","last_peered":"2024-09-20T22:03:36.269100+0000","last_clean":"2024-09-20T22:03:36.269100+0000","last_became_active":"2024-09-20T22:02:52.280005+0000","last_became_peered":"2024-09-20T22:02:52.280005+0000","last_unstale":"2024-09-20T22:03:36.269100+0000","last_undegraded":"2024-09-20T22:03:36.269100+0000","last_fullsized":"2024-09-20T22:03:36.269100+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.190808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.267948+0000","last_change":"2024-09-20T22:02:50.106023+0000","last_active":"2024-09-20T22:03:36.267948+0000","last_peered":"2024-09-20T22:03:36.267948+0000","last_clean":"2024-09-20T22:03:36.267948+0000","last_became_active":"2024-09-20T22:02:50.105633+0000","last_became_peered":"2024-09-20T22:02:50.105633+0000","last_unstale":"2024-09-20T22:03:36.267948+0000","last_undegraded":"2024-09-20T22:03:36.267948+0000","last_fullsized":"2024-09-20T22:03:36.267948+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.449401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"64'1","reported_seq":45,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.616905+0000","last_change":"2024-09-20T22:02:52.270463+0000","last_active":"2024-09-20T22:03:36.616905+0000","last_peered":"2024-09-20T22:03:36.616905+0000","last_clean":"2024-09-20T22:03:36.616905+0000","last_became_active":"2024-09-20T22:02:49.099508+0000","last_became_peered":"2024-09-20T22:02:49.099508+0000","last_unstale":"2024-09-20T22:03:36.616905+0000","last_undegraded":"2024-09-20T22:03:36.616905+0000","last_fullsized":"2024-09-20T22:03:36.616905+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.955566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.617852+0000","last_change":"2024-09-20T22:02:54.292450+0000","last_active":"2024-09-20T22:03:36.617852+0000","last_peered":"2024-09-20T22:03:36.617852+0000","last_clean":"2024-09-20T22:03:36.617852+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:03:36.617852+0000","last_undegraded":"2024-09-20T22:03:36.617852+0000","last_fullsized":"2024-09-20T22:03:36.617852+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.941012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.589234+0000","last_change":"2024-09-20T22:02:56.349789+0000","last_active":"2024-09-20T22:03:36.589234+0000","last_peered":"2024-09-20T22:03:36.589234+0000","last_clean":"2024-09-20T22:03:36.589234+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:03:36.589234+0000","last_undegraded":"2024-09-20T22:03:36.589234+0000","last_fullsized":"2024-09-20T22:03:36.589234+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.060034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"71'12","reported_seq":50,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.295275+0000","last_change":"2024-09-20T22:02:52.277878+0000","last_active":"2024-09-20T22:03:36.295275+0000","last_peered":"2024-09-20T22:03:36.295275+0000","last_clean":"2024-09-20T22:03:36.295275+0000","last_became_active":"2024-09-20T22:02:52.277548+0000","last_became_peered":"2024-09-20T22:02:52.277548+0000","last_unstale":"2024-09-20T22:03:36.295275+0000","last_undegraded":"2024-09-20T22:03:36.295275+0000","last_fullsized":"2024-09-20T22:03:36.295275+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.295996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.294534+0000","last_change":"2024-09-20T22:02:50.110740+0000","last_active":"2024-09-20T22:03:36.294534+0000","last_peered":"2024-09-20T22:03:36.294534+0000","last_clean":"2024-09-20T22:03:36.294534+0000","last_became_active":"2024-09-20T22:02:50.110448+0000","last_became_peered":"2024-09-20T22:02:50.110448+0000","last_unstale":"2024-09-20T22:03:36.294534+0000","last_undegraded":"2024-09-20T22:03:36.294534+0000","last_fullsized":"2024-09-20T22:03:36.294534+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.942941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":92,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:38.722700+0000","last_change":"2024-09-20T22:02:52.480790+0000","last_active":"2024-09-20T22:03:38.722700+0000","last_peered":"2024-09-20T22:03:38.722700+0000","last_clean":"2024-09-20T22:03:38.722700+0000","last_became_active":"2024-09-20T22:02:49.098777+0000","last_became_peered":"2024-09-20T22:02:49.098777+0000","last_unstale":"2024-09-20T22:03:38.722700+0000","last_undegraded":"2024-09-20T22:03:38.722700+0000","last_fullsized":"2024-09-20T22:03:38.722700+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.872201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.588087+0000","last_change":"2024-09-20T22:02:54.294806+0000","last_active":"2024-09-20T22:03:36.588087+0000","last_peered":"2024-09-20T22:03:36.588087+0000","last_clean":"2024-09-20T22:03:36.588087+0000","last_became_active":"2024-09-20T22:02:54.294495+0000","last_became_peered":"2024-09-20T22:02:54.294495+0000","last_unstale":"2024-09-20T22:03:36.588087+0000","last_undegraded":"2024-09-20T22:03:36.588087+0000","last_fullsized":"2024-09-20T22:03:36.588087+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:23.237482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.268255+0000","last_change":"2024-09-20T22:02:56.308897+0000","last_active":"2024-09-20T22:03:36.268255+0000","last_peered":"2024-09-20T22:03:36.268255+0000","last_clean":"2024-09-20T22:03:36.268255+0000","last_became_active":"2024-09-20T22:02:56.308574+0000","last_became_peered":"2024-09-20T22:02:56.308574+0000","last_unstale":"2024-09-20T22:03:36.268255+0000","last_undegraded":"2024-09-20T22:03:36.268255+0000","last_fullsized":"2024-09-20T22:03:36.268255+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.064046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"71'16","reported_seq":65,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.433565+0000","last_change":"2024-09-20T22:02:52.482708+0000","last_active":"2024-09-20T22:03:36.433565+0000","last_peered":"2024-09-20T22:03:36.433565+0000","last_clean":"2024-09-20T22:03:36.433565+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:03:36.433565+0000","last_undegraded":"2024-09-20T22:03:36.433565+0000","last_fullsized":"2024-09-20T22:03:36.433565+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.243856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.617060+0000","last_change":"2024-09-20T22:02:50.106319+0000","last_active":"2024-09-20T22:03:36.617060+0000","last_peered":"2024-09-20T22:03:36.617060+0000","last_clean":"2024-09-20T22:03:36.617060+0000","last_became_active":"2024-09-20T22:02:50.106011+0000","last_became_peered":"2024-09-20T22:02:50.106011+0000","last_unstale":"2024-09-20T22:03:36.617060+0000","last_undegraded":"2024-09-20T22:03:36.617060+0000","last_fullsized":"2024-09-20T22:03:36.617060+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.288394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":292,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.589778+0000","last_change":"2024-09-20T22:02:55.903219+0000","last_active":"2024-09-20T22:03:36.589778+0000","last_peered":"2024-09-20T22:03:36.589778+0000","last_clean":"2024-09-20T22:03:36.589778+0000","last_became_active":"2024-09-20T22:02:45.068482+0000","last_became_peered":"2024-09-20T22:02:45.068482+0000","last_unstale":"2024-09-20T22:03:36.589778+0000","last_undegraded":"2024-09-20T22:03:36.589778+0000","last_fullsized":"2024-09-20T22:03:36.589778+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'117","last_scrub_stamp":"2024-09-20T22:02:55.903139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.903139+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.843650+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.590149+0000","last_change":"2024-09-20T22:02:54.298298+0000","last_active":"2024-09-20T22:03:36.590149+0000","last_peered":"2024-09-20T22:03:36.590149+0000","last_clean":"2024-09-20T22:03:36.590149+0000","last_became_active":"2024-09-20T22:02:54.298021+0000","last_became_peered":"2024-09-20T22:02:54.298021+0000","last_unstale":"2024-09-20T22:03:36.590149+0000","last_undegraded":"2024-09-20T22:03:36.590149+0000","last_fullsized":"2024-09-20T22:03:36.590149+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:54.157709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.586573+0000","last_change":"2024-09-20T22:02:56.308039+0000","last_active":"2024-09-20T22:03:36.586573+0000","last_peered":"2024-09-20T22:03:36.586573+0000","last_clean":"2024-09-20T22:03:36.586573+0000","last_became_active":"2024-09-20T22:02:56.307749+0000","last_became_peered":"2024-09-20T22:02:56.307749+0000","last_unstale":"2024-09-20T22:03:36.586573+0000","last_undegraded":"2024-09-20T22:03:36.586573+0000","last_fullsized":"2024-09-20T22:03:36.586573+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.666810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"71'11","reported_seq":52,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.272004+0000","last_change":"2024-09-20T22:02:52.273177+0000","last_active":"2024-09-20T22:03:36.272004+0000","last_peered":"2024-09-20T22:03:36.272004+0000","last_clean":"2024-09-20T22:03:36.272004+0000","last_became_active":"2024-09-20T22:02:52.272766+0000","last_became_peered":"2024-09-20T22:02:52.272766+0000","last_unstale":"2024-09-20T22:03:36.272004+0000","last_undegraded":"2024-09-20T22:03:36.272004+0000","last_fullsized":"2024-09-20T22:03:36.272004+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.073826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.271620+0000","last_change":"2024-09-20T22:02:50.109626+0000","last_active":"2024-09-20T22:03:36.271620+0000","last_peered":"2024-09-20T22:03:36.271620+0000","last_clean":"2024-09-20T22:03:36.271620+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:03:36.271620+0000","last_undegraded":"2024-09-20T22:03:36.271620+0000","last_fullsized":"2024-09-20T22:03:36.271620+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.659206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.587720+0000","last_change":"2024-09-20T22:02:54.290285+0000","last_active":"2024-09-20T22:03:36.587720+0000","last_peered":"2024-09-20T22:03:36.587720+0000","last_clean":"2024-09-20T22:03:36.587720+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:03:36.587720+0000","last_undegraded":"2024-09-20T22:03:36.587720+0000","last_fullsized":"2024-09-20T22:03:36.587720+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.665015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.618075+0000","last_change":"2024-09-20T22:02:56.349707+0000","last_active":"2024-09-20T22:03:36.618075+0000","last_peered":"2024-09-20T22:03:36.618075+0000","last_clean":"2024-09-20T22:03:36.618075+0000","last_became_active":"2024-09-20T22:02:56.349376+0000","last_became_peered":"2024-09-20T22:02:56.349376+0000","last_unstale":"2024-09-20T22:03:36.618075+0000","last_undegraded":"2024-09-20T22:03:36.618075+0000","last_fullsized":"2024-09-20T22:03:36.618075+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.370704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"71'17","reported_seq":61,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.271775+0000","last_change":"2024-09-20T22:02:52.274124+0000","last_active":"2024-09-20T22:03:36.271775+0000","last_peered":"2024-09-20T22:03:36.271775+0000","last_clean":"2024-09-20T22:03:36.271775+0000","last_became_active":"2024-09-20T22:02:52.273811+0000","last_became_peered":"2024-09-20T22:02:52.273811+0000","last_unstale":"2024-09-20T22:03:36.271775+0000","last_undegraded":"2024-09-20T22:03:36.271775+0000","last_fullsized":"2024-09-20T22:03:36.271775+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:52.793675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.433454+0000","last_change":"2024-09-20T22:02:50.115379+0000","last_active":"2024-09-20T22:03:36.433454+0000","last_peered":"2024-09-20T22:03:36.433454+0000","last_clean":"2024-09-20T22:03:36.433454+0000","last_became_active":"2024-09-20T22:02:50.115015+0000","last_became_peered":"2024-09-20T22:02:50.115015+0000","last_unstale":"2024-09-20T22:03:36.433454+0000","last_undegraded":"2024-09-20T22:03:36.433454+0000","last_fullsized":"2024-09-20T22:03:36.433454+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:16.806437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.268360+0000","last_change":"2024-09-20T22:02:54.291468+0000","last_active":"2024-09-20T22:03:36.268360+0000","last_peered":"2024-09-20T22:03:36.268360+0000","last_clean":"2024-09-20T22:03:36.268360+0000","last_became_active":"2024-09-20T22:02:54.291161+0000","last_became_peered":"2024-09-20T22:02:54.291161+0000","last_unstale":"2024-09-20T22:03:36.268360+0000","last_undegraded":"2024-09-20T22:03:36.268360+0000","last_fullsized":"2024-09-20T22:03:36.268360+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.806525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.616189+0000","last_change":"2024-09-20T22:02:56.307571+0000","last_active":"2024-09-20T22:03:36.616189+0000","last_peered":"2024-09-20T22:03:36.616189+0000","last_clean":"2024-09-20T22:03:36.616189+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:03:36.616189+0000","last_undegraded":"2024-09-20T22:03:36.616189+0000","last_fullsized":"2024-09-20T22:03:36.616189+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:20.532247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"71'10","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.271396+0000","last_change":"2024-09-20T22:02:52.284306+0000","last_active":"2024-09-20T22:03:36.271396+0000","last_peered":"2024-09-20T22:03:36.271396+0000","last_clean":"2024-09-20T22:03:36.271396+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:03:36.271396+0000","last_undegraded":"2024-09-20T22:03:36.271396+0000","last_fullsized":"2024-09-20T22:03:36.271396+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:20.155009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"64'1","reported_seq":42,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.617582+0000","last_change":"2024-09-20T22:02:50.110772+0000","last_active":"2024-09-20T22:03:36.617582+0000","last_peered":"2024-09-20T22:03:36.617582+0000","last_clean":"2024-09-20T22:03:36.617582+0000","last_became_active":"2024-09-20T22:02:50.110446+0000","last_became_peered":"2024-09-20T22:02:50.110446+0000","last_unstale":"2024-09-20T22:03:36.617582+0000","last_undegraded":"2024-09-20T22:03:36.617582+0000","last_fullsized":"2024-09-20T22:03:36.617582+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.664497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"71'8","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.617487+0000","last_change":"2024-09-20T22:02:54.294781+0000","last_active":"2024-09-20T22:03:36.617487+0000","last_peered":"2024-09-20T22:03:36.617487+0000","last_clean":"2024-09-20T22:03:36.617487+0000","last_became_active":"2024-09-20T22:02:54.294458+0000","last_became_peered":"2024-09-20T22:02:54.294458+0000","last_unstale":"2024-09-20T22:03:36.617487+0000","last_undegraded":"2024-09-20T22:03:36.617487+0000","last_fullsized":"2024-09-20T22:03:36.617487+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.016385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.271614+0000","last_change":"2024-09-20T22:02:56.300412+0000","last_active":"2024-09-20T22:03:36.271614+0000","last_peered":"2024-09-20T22:03:36.271614+0000","last_clean":"2024-09-20T22:03:36.271614+0000","last_became_active":"2024-09-20T22:02:56.300093+0000","last_became_peered":"2024-09-20T22:02:56.300093+0000","last_unstale":"2024-09-20T22:03:36.271614+0000","last_undegraded":"2024-09-20T22:03:36.271614+0000","last_fullsized":"2024-09-20T22:03:36.271614+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.204657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"71'9","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.294761+0000","last_change":"2024-09-20T22:02:52.278728+0000","last_active":"2024-09-20T22:03:36.294761+0000","last_peered":"2024-09-20T22:03:36.294761+0000","last_clean":"2024-09-20T22:03:36.294761+0000","last_became_active":"2024-09-20T22:02:52.278381+0000","last_became_peered":"2024-09-20T22:02:52.278381+0000","last_unstale":"2024-09-20T22:03:36.294761+0000","last_undegraded":"2024-09-20T22:03:36.294761+0000","last_fullsized":"2024-09-20T22:03:36.294761+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:51.365937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":42,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.587909+0000","last_change":"2024-09-20T22:02:50.113046+0000","last_active":"2024-09-20T22:03:36.587909+0000","last_peered":"2024-09-20T22:03:36.587909+0000","last_clean":"2024-09-20T22:03:36.587909+0000","last_became_active":"2024-09-20T22:02:50.112682+0000","last_became_peered":"2024-09-20T22:02:50.112682+0000","last_unstale":"2024-09-20T22:03:36.587909+0000","last_undegraded":"2024-09-20T22:03:36.587909+0000","last_fullsized":"2024-09-20T22:03:36.587909+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.700772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.617060+0000","last_change":"2024-09-20T22:02:54.289079+0000","last_active":"2024-09-20T22:03:36.617060+0000","last_peered":"2024-09-20T22:03:36.617060+0000","last_clean":"2024-09-20T22:03:36.617060+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:03:36.617060+0000","last_undegraded":"2024-09-20T22:03:36.617060+0000","last_fullsized":"2024-09-20T22:03:36.617060+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.034123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.294347+0000","last_change":"2024-09-20T22:02:56.298414+0000","last_active":"2024-09-20T22:03:36.294347+0000","last_peered":"2024-09-20T22:03:36.294347+0000","last_clean":"2024-09-20T22:03:36.294347+0000","last_became_active":"2024-09-20T22:02:56.298046+0000","last_became_peered":"2024-09-20T22:02:56.298046+0000","last_unstale":"2024-09-20T22:03:36.294347+0000","last_undegraded":"2024-09-20T22:03:36.294347+0000","last_fullsized":"2024-09-20T22:03:36.294347+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:59.723832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"71'19","reported_seq":63,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.433138+0000","last_change":"2024-09-20T22:02:52.480577+0000","last_active":"2024-09-20T22:03:36.433138+0000","last_peered":"2024-09-20T22:03:36.433138+0000","last_clean":"2024-09-20T22:03:36.433138+0000","last_became_active":"2024-09-20T22:02:52.480208+0000","last_became_peered":"2024-09-20T22:02:52.480208+0000","last_unstale":"2024-09-20T22:03:36.433138+0000","last_undegraded":"2024-09-20T22:03:36.433138+0000","last_fullsized":"2024-09-20T22:03:36.433138+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.853373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.590422+0000","last_change":"2024-09-20T22:02:50.109094+0000","last_active":"2024-09-20T22:03:36.590422+0000","last_peered":"2024-09-20T22:03:36.590422+0000","last_clean":"2024-09-20T22:03:36.590422+0000","last_became_active":"2024-09-20T22:02:50.108712+0000","last_became_peered":"2024-09-20T22:02:50.108712+0000","last_unstale":"2024-09-20T22:03:36.590422+0000","last_undegraded":"2024-09-20T22:03:36.590422+0000","last_fullsized":"2024-09-20T22:03:36.590422+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:08.545134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.617375+0000","last_change":"2024-09-20T22:02:54.291441+0000","last_active":"2024-09-20T22:03:36.617375+0000","last_peered":"2024-09-20T22:03:36.617375+0000","last_clean":"2024-09-20T22:03:36.617375+0000","last_became_active":"2024-09-20T22:02:54.291134+0000","last_became_peered":"2024-09-20T22:02:54.291134+0000","last_unstale":"2024-09-20T22:03:36.617375+0000","last_undegraded":"2024-09-20T22:03:36.617375+0000","last_fullsized":"2024-09-20T22:03:36.617375+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.677000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.589640+0000","last_change":"2024-09-20T22:02:56.308001+0000","last_active":"2024-09-20T22:03:36.589640+0000","last_peered":"2024-09-20T22:03:36.589640+0000","last_clean":"2024-09-20T22:03:36.589640+0000","last_became_active":"2024-09-20T22:02:56.307687+0000","last_became_peered":"2024-09-20T22:02:56.307687+0000","last_unstale":"2024-09-20T22:03:36.589640+0000","last_undegraded":"2024-09-20T22:03:36.589640+0000","last_fullsized":"2024-09-20T22:03:36.589640+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.249693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"71'12","reported_seq":56,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.270998+0000","last_change":"2024-09-20T22:02:52.284579+0000","last_active":"2024-09-20T22:03:36.270998+0000","last_peered":"2024-09-20T22:03:36.270998+0000","last_clean":"2024-09-20T22:03:36.270998+0000","last_became_active":"2024-09-20T22:02:52.284144+0000","last_became_peered":"2024-09-20T22:02:52.284144+0000","last_unstale":"2024-09-20T22:03:36.270998+0000","last_undegraded":"2024-09-20T22:03:36.270998+0000","last_fullsized":"2024-09-20T22:03:36.270998+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.901626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.590606+0000","last_change":"2024-09-20T22:02:50.111591+0000","last_active":"2024-09-20T22:03:36.590606+0000","last_peered":"2024-09-20T22:03:36.590606+0000","last_clean":"2024-09-20T22:03:36.590606+0000","last_became_active":"2024-09-20T22:02:50.111267+0000","last_became_peered":"2024-09-20T22:02:50.111267+0000","last_unstale":"2024-09-20T22:03:36.590606+0000","last_undegraded":"2024-09-20T22:03:36.590606+0000","last_fullsized":"2024-09-20T22:03:36.590606+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.038287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.616441+0000","last_change":"2024-09-20T22:02:54.293804+0000","last_active":"2024-09-20T22:03:36.616441+0000","last_peered":"2024-09-20T22:03:36.616441+0000","last_clean":"2024-09-20T22:03:36.616441+0000","last_became_active":"2024-09-20T22:02:54.293377+0000","last_became_peered":"2024-09-20T22:02:54.293377+0000","last_unstale":"2024-09-20T22:03:36.616441+0000","last_undegraded":"2024-09-20T22:03:36.616441+0000","last_fullsized":"2024-09-20T22:03:36.616441+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.827756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.617089+0000","last_change":"2024-09-20T22:02:56.307499+0000","last_active":"2024-09-20T22:03:36.617089+0000","last_peered":"2024-09-20T22:03:36.617089+0000","last_clean":"2024-09-20T22:03:36.617089+0000","last_became_active":"2024-09-20T22:02:56.306991+0000","last_became_peered":"2024-09-20T22:02:56.306991+0000","last_unstale":"2024-09-20T22:03:36.617089+0000","last_undegraded":"2024-09-20T22:03:36.617089+0000","last_fullsized":"2024-09-20T22:03:36.617089+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.424396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.8","version":"71'15","reported_seq":57,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.587353+0000","last_change":"2024-09-20T22:02:52.482395+0000","last_active":"2024-09-20T22:03:36.587353+0000","last_peered":"2024-09-20T22:03:36.587353+0000","last_clean":"2024-09-20T22:03:36.587353+0000","last_became_active":"2024-09-20T22:02:52.482008+0000","last_became_peered":"2024-09-20T22:02:52.482008+0000","last_unstale":"2024-09-20T22:03:36.587353+0000","last_undegraded":"2024-09-20T22:03:36.587353+0000","last_fullsized":"2024-09-20T22:03:36.587353+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.554179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"64'2","reported_seq":49,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.590615+0000","last_change":"2024-09-20T22:02:50.115186+0000","last_active":"2024-09-20T22:03:36.590615+0000","last_peered":"2024-09-20T22:03:36.590615+0000","last_clean":"2024-09-20T22:03:36.590615+0000","last_became_active":"2024-09-20T22:02:50.114819+0000","last_became_peered":"2024-09-20T22:02:50.114819+0000","last_unstale":"2024-09-20T22:03:36.590615+0000","last_undegraded":"2024-09-20T22:03:36.590615+0000","last_fullsized":"2024-09-20T22:03:36.590615+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.801926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.589783+0000","last_change":"2024-09-20T22:02:54.296823+0000","last_active":"2024-09-20T22:03:36.589783+0000","last_peered":"2024-09-20T22:03:36.589783+0000","last_clean":"2024-09-20T22:03:36.589783+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:03:36.589783+0000","last_undegraded":"2024-09-20T22:03:36.589783+0000","last_fullsized":"2024-09-20T22:03:36.589783+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.496588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.586729+0000","last_change":"2024-09-20T22:02:56.347283+0000","last_active":"2024-09-20T22:03:36.586729+0000","last_peered":"2024-09-20T22:03:36.586729+0000","last_clean":"2024-09-20T22:03:36.586729+0000","last_became_active":"2024-09-20T22:02:56.346990+0000","last_became_peered":"2024-09-20T22:02:56.346990+0000","last_unstale":"2024-09-20T22:03:36.586729+0000","last_undegraded":"2024-09-20T22:03:36.586729+0000","last_fullsized":"2024-09-20T22:03:36.586729+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.718016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.432904+0000","last_change":"2024-09-20T22:02:50.112245+0000","last_active":"2024-09-20T22:03:36.432904+0000","last_peered":"2024-09-20T22:03:36.432904+0000","last_clean":"2024-09-20T22:03:36.432904+0000","last_became_active":"2024-09-20T22:02:50.111935+0000","last_became_peered":"2024-09-20T22:02:50.111935+0000","last_unstale":"2024-09-20T22:03:36.432904+0000","last_undegraded":"2024-09-20T22:03:36.432904+0000","last_fullsized":"2024-09-20T22:03:36.432904+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:16.998775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"71'6","reported_seq":41,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.618704+0000","last_change":"2024-09-20T22:02:52.481046+0000","last_active":"2024-09-20T22:03:36.618704+0000","last_peered":"2024-09-20T22:03:36.618704+0000","last_clean":"2024-09-20T22:03:36.618704+0000","last_became_active":"2024-09-20T22:02:52.480771+0000","last_became_peered":"2024-09-20T22:02:52.480771+0000","last_unstale":"2024-09-20T22:03:36.618704+0000","last_undegraded":"2024-09-20T22:03:36.618704+0000","last_fullsized":"2024-09-20T22:03:36.618704+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.943982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.588012+0000","last_change":"2024-09-20T22:02:54.294834+0000","last_active":"2024-09-20T22:03:36.588012+0000","last_peered":"2024-09-20T22:03:36.588012+0000","last_clean":"2024-09-20T22:03:36.588012+0000","last_became_active":"2024-09-20T22:02:54.294539+0000","last_became_peered":"2024-09-20T22:02:54.294539+0000","last_unstale":"2024-09-20T22:03:36.588012+0000","last_undegraded":"2024-09-20T22:03:36.588012+0000","last_fullsized":"2024-09-20T22:03:36.588012+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.523368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.589242+0000","last_change":"2024-09-20T22:02:56.350065+0000","last_active":"2024-09-20T22:03:36.589242+0000","last_peered":"2024-09-20T22:03:36.589242+0000","last_clean":"2024-09-20T22:03:36.589242+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:03:36.589242+0000","last_undegraded":"2024-09-20T22:03:36.589242+0000","last_fullsized":"2024-09-20T22:03:36.589242+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:19.909723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"71'9","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.295364+0000","last_change":"2024-09-20T22:02:52.482361+0000","last_active":"2024-09-20T22:03:36.295364+0000","last_peered":"2024-09-20T22:03:36.295364+0000","last_clean":"2024-09-20T22:03:36.295364+0000","last_became_active":"2024-09-20T22:02:52.481989+0000","last_became_peered":"2024-09-20T22:02:52.481989+0000","last_unstale":"2024-09-20T22:03:36.295364+0000","last_undegraded":"2024-09-20T22:03:36.295364+0000","last_fullsized":"2024-09-20T22:03:36.295364+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.401767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.589927+0000","last_change":"2024-09-20T22:02:50.108943+0000","last_active":"2024-09-20T22:03:36.589927+0000","last_peered":"2024-09-20T22:03:36.589927+0000","last_clean":"2024-09-20T22:03:36.589927+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:03:36.589927+0000","last_undegraded":"2024-09-20T22:03:36.589927+0000","last_fullsized":"2024-09-20T22:03:36.589927+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.114642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.618453+0000","last_change":"2024-09-20T22:02:54.298137+0000","last_active":"2024-09-20T22:03:36.618453+0000","last_peered":"2024-09-20T22:03:36.618453+0000","last_clean":"2024-09-20T22:03:36.618453+0000","last_became_active":"2024-09-20T22:02:54.297850+0000","last_became_peered":"2024-09-20T22:02:54.297850+0000","last_unstale":"2024-09-20T22:03:36.618453+0000","last_undegraded":"2024-09-20T22:03:36.618453+0000","last_fullsized":"2024-09-20T22:03:36.618453+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.705512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.616134+0000","last_change":"2024-09-20T22:02:56.299306+0000","last_active":"2024-09-20T22:03:36.616134+0000","last_peered":"2024-09-20T22:03:36.616134+0000","last_clean":"2024-09-20T22:03:36.616134+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:03:36.616134+0000","last_undegraded":"2024-09-20T22:03:36.616134+0000","last_fullsized":"2024-09-20T22:03:36.616134+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:11.499111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"71'9","reported_seq":48,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.587875+0000","last_change":"2024-09-20T22:02:52.482498+0000","last_active":"2024-09-20T22:03:36.587875+0000","last_peered":"2024-09-20T22:03:36.587875+0000","last_clean":"2024-09-20T22:03:36.587875+0000","last_became_active":"2024-09-20T22:02:52.482141+0000","last_became_peered":"2024-09-20T22:02:52.482141+0000","last_unstale":"2024-09-20T22:03:36.587875+0000","last_undegraded":"2024-09-20T22:03:36.587875+0000","last_fullsized":"2024-09-20T22:03:36.587875+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.093555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.294710+0000","last_change":"2024-09-20T22:02:50.112533+0000","last_active":"2024-09-20T22:03:36.294710+0000","last_peered":"2024-09-20T22:03:36.294710+0000","last_clean":"2024-09-20T22:03:36.294710+0000","last_became_active":"2024-09-20T22:02:50.110075+0000","last_became_peered":"2024-09-20T22:02:50.110075+0000","last_unstale":"2024-09-20T22:03:36.294710+0000","last_undegraded":"2024-09-20T22:03:36.294710+0000","last_fullsized":"2024-09-20T22:03:36.294710+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.714513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"71'8","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.617893+0000","last_change":"2024-09-20T22:02:54.294653+0000","last_active":"2024-09-20T22:03:36.617893+0000","last_peered":"2024-09-20T22:03:36.617893+0000","last_clean":"2024-09-20T22:03:36.617893+0000","last_became_active":"2024-09-20T22:02:54.294381+0000","last_became_peered":"2024-09-20T22:02:54.294381+0000","last_unstale":"2024-09-20T22:03:36.617893+0000","last_undegraded":"2024-09-20T22:03:36.617893+0000","last_fullsized":"2024-09-20T22:03:36.617893+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.135221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.272038+0000","last_change":"2024-09-20T22:02:56.300405+0000","last_active":"2024-09-20T22:03:36.272038+0000","last_peered":"2024-09-20T22:03:36.272038+0000","last_clean":"2024-09-20T22:03:36.272038+0000","last_became_active":"2024-09-20T22:02:56.300041+0000","last_became_peered":"2024-09-20T22:02:56.300041+0000","last_unstale":"2024-09-20T22:03:36.272038+0000","last_undegraded":"2024-09-20T22:03:36.272038+0000","last_fullsized":"2024-09-20T22:03:36.272038+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.312552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"71'10","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.618645+0000","last_change":"2024-09-20T22:02:52.480909+0000","last_active":"2024-09-20T22:03:36.618645+0000","last_peered":"2024-09-20T22:03:36.618645+0000","last_clean":"2024-09-20T22:03:36.618645+0000","last_became_active":"2024-09-20T22:02:52.480591+0000","last_became_peered":"2024-09-20T22:02:52.480591+0000","last_unstale":"2024-09-20T22:03:36.618645+0000","last_undegraded":"2024-09-20T22:03:36.618645+0000","last_fullsized":"2024-09-20T22:03:36.618645+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.915698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.590144+0000","last_change":"2024-09-20T22:02:50.112976+0000","last_active":"2024-09-20T22:03:36.590144+0000","last_peered":"2024-09-20T22:03:36.590144+0000","last_clean":"2024-09-20T22:03:36.590144+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:03:36.590144+0000","last_undegraded":"2024-09-20T22:03:36.590144+0000","last_fullsized":"2024-09-20T22:03:36.590144+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.057376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"71'8","reported_seq":47,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.587487+0000","last_change":"2024-09-20T22:02:54.292909+0000","last_active":"2024-09-20T22:03:36.587487+0000","last_peered":"2024-09-20T22:03:36.587487+0000","last_clean":"2024-09-20T22:03:36.587487+0000","last_became_active":"2024-09-20T22:02:54.289548+0000","last_became_peered":"2024-09-20T22:02:54.289548+0000","last_unstale":"2024-09-20T22:03:36.587487+0000","last_undegraded":"2024-09-20T22:03:36.587487+0000","last_fullsized":"2024-09-20T22:03:36.587487+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.478336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.295004+0000","last_change":"2024-09-20T22:02:56.305690+0000","last_active":"2024-09-20T22:03:36.295004+0000","last_peered":"2024-09-20T22:03:36.295004+0000","last_clean":"2024-09-20T22:03:36.295004+0000","last_became_active":"2024-09-20T22:02:56.305383+0000","last_became_peered":"2024-09-20T22:02:56.305383+0000","last_unstale":"2024-09-20T22:03:36.295004+0000","last_undegraded":"2024-09-20T22:03:36.295004+0000","last_fullsized":"2024-09-20T22:03:36.295004+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.293958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"71'11","reported_seq":52,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.271150+0000","last_change":"2024-09-20T22:02:52.277987+0000","last_active":"2024-09-20T22:03:36.271150+0000","last_peered":"2024-09-20T22:03:36.271150+0000","last_clean":"2024-09-20T22:03:36.271150+0000","last_became_active":"2024-09-20T22:02:52.277479+0000","last_became_peered":"2024-09-20T22:02:52.277479+0000","last_unstale":"2024-09-20T22:03:36.271150+0000","last_undegraded":"2024-09-20T22:03:36.271150+0000","last_fullsized":"2024-09-20T22:03:36.271150+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.222570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.271644+0000","last_change":"2024-09-20T22:02:50.109201+0000","last_active":"2024-09-20T22:03:36.271644+0000","last_peered":"2024-09-20T22:03:36.271644+0000","last_clean":"2024-09-20T22:03:36.271644+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:03:36.271644+0000","last_undegraded":"2024-09-20T22:03:36.271644+0000","last_fullsized":"2024-09-20T22:03:36.271644+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.129228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.268166+0000","last_change":"2024-09-20T22:02:54.286502+0000","last_active":"2024-09-20T22:03:36.268166+0000","last_peered":"2024-09-20T22:03:36.268166+0000","last_clean":"2024-09-20T22:03:36.268166+0000","last_became_active":"2024-09-20T22:02:54.286200+0000","last_became_peered":"2024-09-20T22:02:54.286200+0000","last_unstale":"2024-09-20T22:03:36.268166+0000","last_undegraded":"2024-09-20T22:03:36.268166+0000","last_fullsized":"2024-09-20T22:03:36.268166+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:22.326441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.616367+0000","last_change":"2024-09-20T22:02:56.307258+0000","last_active":"2024-09-20T22:03:36.616367+0000","last_peered":"2024-09-20T22:03:36.616367+0000","last_clean":"2024-09-20T22:03:36.616367+0000","last_became_active":"2024-09-20T22:02:56.306892+0000","last_became_peered":"2024-09-20T22:02:56.306892+0000","last_unstale":"2024-09-20T22:03:36.616367+0000","last_undegraded":"2024-09-20T22:03:36.616367+0000","last_fullsized":"2024-09-20T22:03:36.616367+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.948765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"71'9","reported_seq":49,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.268387+0000","last_change":"2024-09-20T22:02:52.280815+0000","last_active":"2024-09-20T22:03:36.268387+0000","last_peered":"2024-09-20T22:03:36.268387+0000","last_clean":"2024-09-20T22:03:36.268387+0000","last_became_active":"2024-09-20T22:02:52.280428+0000","last_became_peered":"2024-09-20T22:02:52.280428+0000","last_unstale":"2024-09-20T22:03:36.268387+0000","last_undegraded":"2024-09-20T22:03:36.268387+0000","last_fullsized":"2024-09-20T22:03:36.268387+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.548215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.590319+0000","last_change":"2024-09-20T22:02:50.110105+0000","last_active":"2024-09-20T22:03:36.590319+0000","last_peered":"2024-09-20T22:03:36.590319+0000","last_clean":"2024-09-20T22:03:36.590319+0000","last_became_active":"2024-09-20T22:02:50.109803+0000","last_became_peered":"2024-09-20T22:02:50.109803+0000","last_unstale":"2024-09-20T22:03:36.590319+0000","last_undegraded":"2024-09-20T22:03:36.590319+0000","last_fullsized":"2024-09-20T22:03:36.590319+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:53.587349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.617439+0000","last_change":"2024-09-20T22:02:54.298130+0000","last_active":"2024-09-20T22:03:36.617439+0000","last_peered":"2024-09-20T22:03:36.617439+0000","last_clean":"2024-09-20T22:03:36.617439+0000","last_became_active":"2024-09-20T22:02:54.297803+0000","last_became_peered":"2024-09-20T22:02:54.297803+0000","last_unstale":"2024-09-20T22:03:36.617439+0000","last_undegraded":"2024-09-20T22:03:36.617439+0000","last_fullsized":"2024-09-20T22:03:36.617439+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.520061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.294080+0000","last_change":"2024-09-20T22:02:56.298228+0000","last_active":"2024-09-20T22:03:36.294080+0000","last_peered":"2024-09-20T22:03:36.294080+0000","last_clean":"2024-09-20T22:03:36.294080+0000","last_became_active":"2024-09-20T22:02:56.297826+0000","last_became_peered":"2024-09-20T22:02:56.297826+0000","last_unstale":"2024-09-20T22:03:36.294080+0000","last_undegraded":"2024-09-20T22:03:36.294080+0000","last_fullsized":"2024-09-20T22:03:36.294080+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.126658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"71'11","reported_seq":51,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.616693+0000","last_change":"2024-09-20T22:02:52.480504+0000","last_active":"2024-09-20T22:03:36.616693+0000","last_peered":"2024-09-20T22:03:36.616693+0000","last_clean":"2024-09-20T22:03:36.616693+0000","last_became_active":"2024-09-20T22:02:52.480118+0000","last_became_peered":"2024-09-20T22:02:52.480118+0000","last_unstale":"2024-09-20T22:03:36.616693+0000","last_undegraded":"2024-09-20T22:03:36.616693+0000","last_fullsized":"2024-09-20T22:03:36.616693+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.023916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.588012+0000","last_change":"2024-09-20T22:02:50.110808+0000","last_active":"2024-09-20T22:03:36.588012+0000","last_peered":"2024-09-20T22:03:36.588012+0000","last_clean":"2024-09-20T22:03:36.588012+0000","last_became_active":"2024-09-20T22:02:50.110421+0000","last_became_peered":"2024-09-20T22:02:50.110421+0000","last_unstale":"2024-09-20T22:03:36.588012+0000","last_undegraded":"2024-09-20T22:03:36.588012+0000","last_fullsized":"2024-09-20T22:03:36.588012+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:53.027475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":29,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.589494+0000","last_change":"2024-09-20T22:02:54.293068+0000","last_active":"2024-09-20T22:03:36.589494+0000","last_peered":"2024-09-20T22:03:36.589494+0000","last_clean":"2024-09-20T22:03:36.589494+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:03:36.589494+0000","last_undegraded":"2024-09-20T22:03:36.589494+0000","last_fullsized":"2024-09-20T22:03:36.589494+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.450710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.616965+0000","last_change":"2024-09-20T22:02:56.348065+0000","last_active":"2024-09-20T22:03:36.616965+0000","last_peered":"2024-09-20T22:03:36.616965+0000","last_clean":"2024-09-20T22:03:36.616965+0000","last_became_active":"2024-09-20T22:02:56.347637+0000","last_became_peered":"2024-09-20T22:02:56.347637+0000","last_unstale":"2024-09-20T22:03:36.616965+0000","last_undegraded":"2024-09-20T22:03:36.616965+0000","last_fullsized":"2024-09-20T22:03:36.616965+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:13.568557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"71'4","reported_seq":38,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.616624+0000","last_change":"2024-09-20T22:02:52.286086+0000","last_active":"2024-09-20T22:03:36.616624+0000","last_peered":"2024-09-20T22:03:36.616624+0000","last_clean":"2024-09-20T22:03:36.616624+0000","last_became_active":"2024-09-20T22:02:52.285701+0000","last_became_peered":"2024-09-20T22:02:52.285701+0000","last_unstale":"2024-09-20T22:03:36.616624+0000","last_undegraded":"2024-09-20T22:03:36.616624+0000","last_fullsized":"2024-09-20T22:03:36.616624+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.474692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.295452+0000","last_change":"2024-09-20T22:02:50.111749+0000","last_active":"2024-09-20T22:03:36.295452+0000","last_peered":"2024-09-20T22:03:36.295452+0000","last_clean":"2024-09-20T22:03:36.295452+0000","last_became_active":"2024-09-20T22:02:50.111142+0000","last_became_peered":"2024-09-20T22:02:50.111142+0000","last_unstale":"2024-09-20T22:03:36.295452+0000","last_undegraded":"2024-09-20T22:03:36.295452+0000","last_fullsized":"2024-09-20T22:03:36.295452+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:57.037771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.432913+0000","last_change":"2024-09-20T22:02:54.290418+0000","last_active":"2024-09-20T22:03:36.432913+0000","last_peered":"2024-09-20T22:03:36.432913+0000","last_clean":"2024-09-20T22:03:36.432913+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:03:36.432913+0000","last_undegraded":"2024-09-20T22:03:36.432913+0000","last_fullsized":"2024-09-20T22:03:36.432913+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:57.209065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.589160+0000","last_change":"2024-09-20T22:02:56.300559+0000","last_active":"2024-09-20T22:03:36.589160+0000","last_peered":"2024-09-20T22:03:36.589160+0000","last_clean":"2024-09-20T22:03:36.589160+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:03:36.589160+0000","last_undegraded":"2024-09-20T22:03:36.589160+0000","last_fullsized":"2024-09-20T22:03:36.589160+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.506369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.268612+0000","last_change":"2024-09-20T22:02:56.301611+0000","last_active":"2024-09-20T22:03:36.268612+0000","last_peered":"2024-09-20T22:03:36.268612+0000","last_clean":"2024-09-20T22:03:36.268612+0000","last_became_active":"2024-09-20T22:02:56.301247+0000","last_became_peered":"2024-09-20T22:02:56.301247+0000","last_unstale":"2024-09-20T22:03:36.268612+0000","last_undegraded":"2024-09-20T22:03:36.268612+0000","last_fullsized":"2024-09-20T22:03:36.268612+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:55.486378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":36,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.616788+0000","last_change":"2024-09-20T22:02:50.114519+0000","last_active":"2024-09-20T22:03:36.616788+0000","last_peered":"2024-09-20T22:03:36.616788+0000","last_clean":"2024-09-20T22:03:36.616788+0000","last_became_active":"2024-09-20T22:02:50.114204+0000","last_became_peered":"2024-09-20T22:02:50.114204+0000","last_unstale":"2024-09-20T22:03:36.616788+0000","last_undegraded":"2024-09-20T22:03:36.616788+0000","last_fullsized":"2024-09-20T22:03:36.616788+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:50.725534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"71'11","reported_seq":51,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.433727+0000","last_change":"2024-09-20T22:02:52.279065+0000","last_active":"2024-09-20T22:03:36.433727+0000","last_peered":"2024-09-20T22:03:36.433727+0000","last_clean":"2024-09-20T22:03:36.433727+0000","last_became_active":"2024-09-20T22:02:52.278657+0000","last_became_peered":"2024-09-20T22:02:52.278657+0000","last_unstale":"2024-09-20T22:03:36.433727+0000","last_undegraded":"2024-09-20T22:03:36.433727+0000","last_fullsized":"2024-09-20T22:03:36.433727+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.405988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-20T22:03:36.295145+0000","last_change":"2024-09-20T22:02:54.289401+0000","last_active":"2024-09-20T22:03:36.295145+0000","last_peered":"2024-09-20T22:03:36.295145+0000","last_clean":"2024-09-20T22:03:36.295145+0000","last_became_active":"2024-09-20T22:02:54.289078+0000","last_became_peered":"2024-09-20T22:02:54.289078+0000","last_unstale":"2024-09-20T22:03:36.295145+0000","last_undegraded":"2024-09-20T22:03:36.295145+0000","last_fullsized":"2024-09-20T22:03:36.295145+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.343711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135886,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32540,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712612,"statfs":{"total":95995035648,"available":95961714688,"internally_reserved":0,"allocated":1380352,"data_stored":939243,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430226,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32520,"kb_used_data":1332,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712632,"statfs":{"total":95995035648,"available":95961735168,"internally_reserved":0,"allocated":1363968,"data_stored":939151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724566,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27780,"kb_used_data":752,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717372,"statfs":{"total":95995035648,"available":95966588928,"internally_reserved":0,"allocated":770048,"data_stored":349129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986206,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31964,"kb_used_data":776,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713188,"statfs":{"total":95995035648,"available":95962304512,"internally_reserved":0,"allocated":794624,"data_stored":348818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27840,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717312,"statfs":{"total":95995035648,"available":95966527488,"internally_reserved":0,"allocated":827392,"data_stored":349562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149731,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31980,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713172,"statfs":{"total":95995035648,"available":95962288128,"internally_reserved":0,"allocated":811008,"data_stored":350110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411368,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27768,"kb_used_data":740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717384,"statfs":{"total":95995035648,"available":95966601216,"internally_reserved":0,"allocated":757760,"data_stored":347925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738419,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32540,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712612,"statfs":{"total":95995035648,"available":95961714688,"internally_reserved":0,"allocated":1384448,"data_stored":939375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:03:44.095 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-20T22:03:44.095 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd dump 2024-09-20T22:03:44.786 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:03:44 smithi060 podman[76537]: 2024-09-20 22:03:44.386042603 +0000 UTC m=+0.195724759 container create 02096472ad32bd54ce0b380bc824c4d2155f7808c499b46eeb5c81e47a9ec1cc (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T22:03:45.189 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:45.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:03:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:03:45.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:03:45] ENGINE Bus STOPPING 2024-09-20T22:03:45.287 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:03:45 smithi060 podman[76537]: 2024-09-20 22:03:45.03084152 +0000 UTC m=+0.840523687 container init 02096472ad32bd54ce0b380bc824c4d2155f7808c499b46eeb5c81e47a9ec1cc (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T22:03:45.287 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:03:45 smithi060 podman[76537]: 2024-09-20 22:03:45.036451112 +0000 UTC m=+0.846133270 container start 02096472ad32bd54ce0b380bc824c4d2155f7808c499b46eeb5c81e47a9ec1cc (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T22:03:45.287 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[76563]: ts=2024-09-20T22:03:45.051Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-20T22:03:45.287 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[76563]: ts=2024-09-20T22:03:45.051Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-20T22:03:45.287 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[76563]: ts=2024-09-20T22:03:45.052Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.60 port=9094 2024-09-20T22:03:45.287 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[76563]: ts=2024-09-20T22:03:45.053Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-20T22:03:45.287 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[76563]: ts=2024-09-20T22:03:45.077Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-20T22:03:45.287 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[76563]: ts=2024-09-20T22:03:45.078Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-20T22:03:45.287 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[76563]: ts=2024-09-20T22:03:45.080Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-20T22:03:45.287 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[76563]: ts=2024-09-20T22:03:45.080Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-20T22:03:45.288 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:03:45 smithi060 bash[76537]: 02096472ad32bd54ce0b380bc824c4d2155f7808c499b46eeb5c81e47a9ec1cc 2024-09-20T22:03:45.288 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:03:45 smithi060 systemd[1]: Started Ceph alertmanager.a for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T22:03:45.602 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:03:45] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-20T22:03:45.602 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:03:45] ENGINE Bus STOPPED 2024-09-20T22:03:45.602 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:03:45] ENGINE Bus STARTING 2024-09-20T22:03:45.602 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[29694]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:03:45.602 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[29694]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:03:45.602 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.602 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.602 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.602 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.603 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.603 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.603 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-20T22:03:45.603 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.603 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[40290]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:03:45.603 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[40290]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:03:45.603 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.603 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.603 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.603 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.603 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.604 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.604 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-20T22:03:45.604 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:45 smithi067 ceph-mon[33511]: pgmap v19: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:03:45.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:45 smithi067 ceph-mon[33511]: from='client.14754 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:03:45.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:45 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:45 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:45 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:45 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:45 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:45 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:45 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-20T22:03:45.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:45 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:45.875 INFO:teuthology.orchestra.run.smithi060.stdout:epoch 78 2024-09-20T22:03:45.875 INFO:teuthology.orchestra.run.smithi060.stdout:fsid 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T22:03:45.875 INFO:teuthology.orchestra.run.smithi060.stdout:created 2024-09-20T21:57:31.244934+0000 2024-09-20T22:03:45.875 INFO:teuthology.orchestra.run.smithi060.stdout:modified 2024-09-20T22:03:41.266640+0000 2024-09-20T22:03:45.875 INFO:teuthology.orchestra.run.smithi060.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-20T22:03:45.875 INFO:teuthology.orchestra.run.smithi060.stdout:crush_version 18 2024-09-20T22:03:45.875 INFO:teuthology.orchestra.run.smithi060.stdout:full_ratio 0.95 2024-09-20T22:03:45.875 INFO:teuthology.orchestra.run.smithi060.stdout:backfillfull_ratio 0.9 2024-09-20T22:03:45.875 INFO:teuthology.orchestra.run.smithi060.stdout:nearfull_ratio 0.85 2024-09-20T22:03:45.875 INFO:teuthology.orchestra.run.smithi060.stdout:require_min_compat_client luminous 2024-09-20T22:03:45.876 INFO:teuthology.orchestra.run.smithi060.stdout:min_compat_client jewel 2024-09-20T22:03:45.876 INFO:teuthology.orchestra.run.smithi060.stdout:require_osd_release squid 2024-09-20T22:03:45.876 INFO:teuthology.orchestra.run.smithi060.stdout:stretch_mode_enabled false 2024-09-20T22:03:45.876 INFO:teuthology.orchestra.run.smithi060.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 7.89 2024-09-20T22:03:45.876 INFO:teuthology.orchestra.run.smithi060.stdout:pool 2 'datapool' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 3 pgp_num 3 autoscale_mode off last_change 66 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 2.65 2024-09-20T22:03:45.876 INFO:teuthology.orchestra.run.smithi060.stdout:pool 3 '.rgw.root' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 65 flags hashpspool stripe_width 0 application rgw read_balance_score 1.50 2024-09-20T22:03:45.876 INFO:teuthology.orchestra.run.smithi060.stdout:pool 4 'default.rgw.log' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 67 flags hashpspool stripe_width 0 application rgw read_balance_score 2.25 2024-09-20T22:03:45.876 INFO:teuthology.orchestra.run.smithi060.stdout:pool 5 'default.rgw.control' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 69 flags hashpspool stripe_width 0 application rgw read_balance_score 1.25 2024-09-20T22:03:45.876 INFO:teuthology.orchestra.run.smithi060.stdout:pool 6 'default.rgw.meta' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 71 flags hashpspool stripe_width 0 pg_autoscale_bias 4 application rgw read_balance_score 1.75 2024-09-20T22:03:45.876 INFO:teuthology.orchestra.run.smithi060.stdout:max_osd 8 2024-09-20T22:03:45.876 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 up in weight 1 up_from 8 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550] [v2:172.21.15.60:6804/1395849550,v1:172.21.15.60:6805/1395849550] exists,up 0997c01a-a9bd-421e-abcc-fbcf7abe8512 2024-09-20T22:03:45.876 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1 up in weight 1 up_from 13 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6810/145283513,v1:172.21.15.60:6811/145283513] [v2:172.21.15.60:6812/145283513,v1:172.21.15.60:6813/145283513] exists,up c04bdcb1-d76b-47c8-a38b-c02043df6ba8 2024-09-20T22:03:45.876 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 up in weight 1 up_from 18 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6818/3698562274,v1:172.21.15.60:6819/3698562274] [v2:172.21.15.60:6820/3698562274,v1:172.21.15.60:6821/3698562274] exists,up 32c09d19-eb98-4926-a44e-e7ddee152956 2024-09-20T22:03:45.877 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3 up in weight 1 up_from 26 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449] [v2:172.21.15.60:6828/2592322449,v1:172.21.15.60:6829/2592322449] exists,up b1c2cb69-8565-49ac-b854-7ea7615c6660 2024-09-20T22:03:45.877 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 up in weight 1 up_from 31 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6800/3243564239,v1:172.21.15.67:6801/3243564239] [v2:172.21.15.67:6802/3243564239,v1:172.21.15.67:6803/3243564239] exists,up 04a547d6-fc48-43b7-a8d7-18df8a83aa96 2024-09-20T22:03:45.877 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 up in weight 1 up_from 39 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6808/1622258887,v1:172.21.15.67:6809/1622258887] [v2:172.21.15.67:6810/1622258887,v1:172.21.15.67:6811/1622258887] exists,up 4763aa10-c367-485b-b525-e5241e642b86 2024-09-20T22:03:45.877 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 up in weight 1 up_from 48 up_thru 67 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6816/1187114,v1:172.21.15.67:6817/1187114] [v2:172.21.15.67:6818/1187114,v1:172.21.15.67:6819/1187114] exists,up a359116e-c013-4e2f-8b79-86219516ee16 2024-09-20T22:03:45.877 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 up in weight 1 up_from 57 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6824/3021842840,v1:172.21.15.67:6825/3021842840] [v2:172.21.15.67:6826/3021842840,v1:172.21.15.67:6827/3021842840] exists,up a4bf4188-f133-4a31-8897-d7e8182248d7 2024-09-20T22:03:45.877 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/890154710 expires 2024-09-21T22:03:19.518405+0000 2024-09-20T22:03:45.877 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/3386526350 expires 2024-09-21T22:03:19.518405+0000 2024-09-20T22:03:45.877 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6801/604714621 expires 2024-09-21T22:03:19.518405+0000 2024-09-20T22:03:45.877 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6800/604714621 expires 2024-09-21T22:03:19.518405+0000 2024-09-20T22:03:45.877 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6800/3151907776 expires 2024-09-21T21:58:02.105557+0000 2024-09-20T22:03:45.877 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/956264518 expires 2024-09-21T21:58:02.105557+0000 2024-09-20T22:03:45.877 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/4150585713 expires 2024-09-21T21:58:02.105557+0000 2024-09-20T22:03:45.877 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/3311255314 expires 2024-09-21T22:03:19.518405+0000 2024-09-20T22:03:45.878 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/3965541528 expires 2024-09-21T21:58:02.105557+0000 2024-09-20T22:03:45.878 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6800/3734574621 expires 2024-09-21T21:58:26.613414+0000 2024-09-20T22:03:45.878 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/1729862661 expires 2024-09-21T21:58:26.613414+0000 2024-09-20T22:03:45.878 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6801/3734574621 expires 2024-09-21T21:58:26.613414+0000 2024-09-20T22:03:45.878 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/3927294026 expires 2024-09-21T21:58:26.613414+0000 2024-09-20T22:03:45.878 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:6801/3151907776 expires 2024-09-21T21:58:02.105557+0000 2024-09-20T22:03:45.878 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/1871351181 expires 2024-09-21T21:58:26.613414+0000 2024-09-20T22:03:45.878 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/1854455098 expires 2024-09-21T22:03:19.518405+0000 2024-09-20T22:03:45.878 INFO:teuthology.orchestra.run.smithi060.stdout:blocklist 172.21.15.60:0/2221262203 expires 2024-09-21T22:03:19.518405+0000 2024-09-20T22:03:45.878 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:03:45] ENGINE Serving on http://:::9283 2024-09-20T22:03:45.878 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:03:45] ENGINE Bus STARTED 2024-09-20T22:03:46.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:46 smithi060 ceph-mon[29694]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-20T22:03:46.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:46 smithi060 ceph-mon[29694]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-20T22:03:46.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:46 smithi060 ceph-mon[29694]: Deploying daemon grafana.a on smithi067 2024-09-20T22:03:46.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:46 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4075368' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-20T22:03:46.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:46 smithi060 ceph-mon[29694]: osdmap e79: 8 total, 8 up, 8 in 2024-09-20T22:03:46.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:46 smithi060 ceph-mon[40290]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-20T22:03:46.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:46 smithi060 ceph-mon[40290]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-20T22:03:46.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:46 smithi060 ceph-mon[40290]: Deploying daemon grafana.a on smithi067 2024-09-20T22:03:46.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:46 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4075368' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-20T22:03:46.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:46 smithi060 ceph-mon[40290]: osdmap e79: 8 total, 8 up, 8 in 2024-09-20T22:03:46.619 INFO:tasks.cephadm.ceph_manager.ceph:['osd.0 up in weight 1 up_from 8 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6802/1395849550,v1:172.21.15.60:6803/1395849550] [v2:172.21.15.60:6804/1395849550,v1:172.21.15.60:6805/1395849550] exists,up 0997c01a-a9bd-421e-abcc-fbcf7abe8512', 'osd.1 up in weight 1 up_from 13 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6810/145283513,v1:172.21.15.60:6811/145283513] [v2:172.21.15.60:6812/145283513,v1:172.21.15.60:6813/145283513] exists,up c04bdcb1-d76b-47c8-a38b-c02043df6ba8', 'osd.2 up in weight 1 up_from 18 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6818/3698562274,v1:172.21.15.60:6819/3698562274] [v2:172.21.15.60:6820/3698562274,v1:172.21.15.60:6821/3698562274] exists,up 32c09d19-eb98-4926-a44e-e7ddee152956', 'osd.3 up in weight 1 up_from 26 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.60:6826/2592322449,v1:172.21.15.60:6827/2592322449] [v2:172.21.15.60:6828/2592322449,v1:172.21.15.60:6829/2592322449] exists,up b1c2cb69-8565-49ac-b854-7ea7615c6660', 'osd.4 up in weight 1 up_from 31 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6800/3243564239,v1:172.21.15.67:6801/3243564239] [v2:172.21.15.67:6802/3243564239,v1:172.21.15.67:6803/3243564239] exists,up 04a547d6-fc48-43b7-a8d7-18df8a83aa96', 'osd.5 up in weight 1 up_from 39 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6808/1622258887,v1:172.21.15.67:6809/1622258887] [v2:172.21.15.67:6810/1622258887,v1:172.21.15.67:6811/1622258887] exists,up 4763aa10-c367-485b-b525-e5241e642b86', 'osd.6 up in weight 1 up_from 48 up_thru 67 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6816/1187114,v1:172.21.15.67:6817/1187114] [v2:172.21.15.67:6818/1187114,v1:172.21.15.67:6819/1187114] exists,up a359116e-c013-4e2f-8b79-86219516ee16', 'osd.7 up in weight 1 up_from 57 up_thru 69 down_at 0 last_clean_interval [0,0) [v2:172.21.15.67:6824/3021842840,v1:172.21.15.67:6825/3021842840] [v2:172.21.15.67:6826/3021842840,v1:172.21.15.67:6827/3021842840] exists,up a4bf4188-f133-4a31-8897-d7e8182248d7'] 2024-09-20T22:03:46.619 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon mon.a config show 2024-09-20T22:03:46.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:46 smithi067 ceph-mon[33511]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-20T22:03:46.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:46 smithi067 ceph-mon[33511]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-20T22:03:46.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:46 smithi067 ceph-mon[33511]: Deploying daemon grafana.a on smithi067 2024-09-20T22:03:46.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:46 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4075368' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-20T22:03:46.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:46 smithi067 ceph-mon[33511]: osdmap e79: 8 total, 8 up, 8 in 2024-09-20T22:03:46.877 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:47.235 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:03:47 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[76563]: ts=2024-09-20T22:03:47.053Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000181327s 2024-09-20T22:03:47.337 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:03:47.338 INFO:teuthology.orchestra.run.smithi060.stdout: "name": "mon.a", 2024-09-20T22:03:47.338 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster": "ceph", 2024-09-20T22:03:47.338 INFO:teuthology.orchestra.run.smithi060.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-09-20T22:03:47.338 INFO:teuthology.orchestra.run.smithi060.stdout: "admin_socket_mode": "", 2024-09-20T22:03:47.338 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_allow_insecure_global_id_reclaim": "false", 2024-09-20T22:03:47.338 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_client_required": "cephx, none", 2024-09-20T22:03:47.338 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_cluster_required": "cephx", 2024-09-20T22:03:47.338 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_debug": "true", 2024-09-20T22:03:47.338 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-09-20T22:03:47.338 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-09-20T22:03:47.338 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_service_required": "cephx", 2024-09-20T22:03:47.338 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_service_ticket_ttl": "240.000000", 2024-09-20T22:03:47.338 INFO:teuthology.orchestra.run.smithi060.stdout: "auth_supported": "", 2024-09-20T22:03:47.339 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_aio": "true", 2024-09-20T22:03:47.339 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_aio_max_queue_depth": "1024", 2024-09-20T22:03:47.339 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_aio_poll_ms": "250", 2024-09-20T22:03:47.339 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_aio_reap_max": "16", 2024-09-20T22:03:47.339 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_async_discard": "false", 2024-09-20T22:03:47.339 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_block_size": "4096", 2024-09-20T22:03:47.339 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_debug_aio": "false", 2024-09-20T22:03:47.339 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-09-20T22:03:47.339 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-09-20T22:03:47.339 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_debug_inflight_ios": "false", 2024-09-20T22:03:47.339 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_enable_discard": "false", 2024-09-20T22:03:47.339 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_flock_retry": "3", 2024-09-20T22:03:47.340 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_flock_retry_interval": "0.100000", 2024-09-20T22:03:47.340 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_inject_crash": "0", 2024-09-20T22:03:47.340 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_inject_crash_flush_delay": "2", 2024-09-20T22:03:47.340 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_ioring": "false", 2024-09-20T22:03:47.340 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_ioring_hipri": "false", 2024-09-20T22:03:47.340 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_ioring_sqthread_poll": "false", 2024-09-20T22:03:47.340 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-09-20T22:03:47.340 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_read_buffer_alignment": "4096", 2024-09-20T22:03:47.340 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-09-20T22:03:47.340 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_stalled_read_warn_lifetime": "86400", 2024-09-20T22:03:47.340 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_stalled_read_warn_threshold": "1", 2024-09-20T22:03:47.340 INFO:teuthology.orchestra.run.smithi060.stdout: "bdev_type": "", 2024-09-20T22:03:47.340 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_alloc_size": "1048576", 2024-09-20T22:03:47.341 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_allocator": "hybrid", 2024-09-20T22:03:47.341 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_buffered_io": "true", 2024-09-20T22:03:47.341 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_check_for_zeros": "false", 2024-09-20T22:03:47.341 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_check_volume_selector_often": "false", 2024-09-20T22:03:47.341 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-09-20T22:03:47.341 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_compact_log_sync": "false", 2024-09-20T22:03:47.341 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-09-20T22:03:47.341 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-09-20T22:03:47.341 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_log_compact_min_size": "16777216", 2024-09-20T22:03:47.341 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_log_replay_check_allocations": "true", 2024-09-20T22:03:47.341 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_max_log_runway": "4194304", 2024-09-20T22:03:47.341 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_max_prefetch": "1048576", 2024-09-20T22:03:47.341 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_min_flush_size": "524288", 2024-09-20T22:03:47.342 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_min_log_runway": "1048576", 2024-09-20T22:03:47.342 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_replay_recovery": "false", 2024-09-20T22:03:47.342 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-09-20T22:03:47.342 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_shared_alloc_size": "65536", 2024-09-20T22:03:47.342 INFO:teuthology.orchestra.run.smithi060.stdout: "bluefs_sync_write": "false", 2024-09-20T22:03:47.342 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-09-20T22:03:47.342 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-09-20T22:03:47.342 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-09-20T22:03:47.342 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_allocation_from_file": "true", 2024-09-20T22:03:47.342 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_allocator": "hybrid", 2024-09-20T22:03:47.342 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-09-20T22:03:47.342 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-09-20T22:03:47.342 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-09-20T22:03:47.343 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-09-20T22:03:47.343 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_bdev_label_multi": "true", 2024-09-20T22:03:47.343 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_bdev_label_multi_upgrade": "false", 2024-09-20T22:03:47.343 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_bdev_label_require_all": "true", 2024-09-20T22:03:47.343 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-09-20T22:03:47.343 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-09-20T22:03:47.343 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_blobid_prealloc": "10240", 2024-09-20T22:03:47.343 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_create": "true", 2024-09-20T22:03:47.343 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_db_create": "false", 2024-09-20T22:03:47.343 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_db_path": "", 2024-09-20T22:03:47.343 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_db_size": "0", 2024-09-20T22:03:47.343 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_path": "", 2024-09-20T22:03:47.343 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_preallocate_file": "false", 2024-09-20T22:03:47.344 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_size": "107374182400", 2024-09-20T22:03:47.344 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_wal_create": "false", 2024-09-20T22:03:47.344 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_wal_path": "", 2024-09-20T22:03:47.344 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_block_wal_size": "100663296", 2024-09-20T22:03:47.344 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_bluefs": "true", 2024-09-20T22:03:47.344 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-09-20T22:03:47.344 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_bluefs_env_mirror": "false", 2024-09-20T22:03:47.344 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-09-20T22:03:47.344 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-09-20T22:03:47.345 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-09-20T22:03:47.345 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-09-20T22:03:47.345 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-09-20T22:03:47.345 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-09-20T22:03:47.345 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_autotune": "true", 2024-09-20T22:03:47.345 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-09-20T22:03:47.345 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-09-20T22:03:47.345 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-09-20T22:03:47.345 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-09-20T22:03:47.345 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_size": "0", 2024-09-20T22:03:47.345 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-09-20T22:03:47.345 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-09-20T22:03:47.345 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-09-20T22:03:47.345 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-09-20T22:03:47.346 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cache_type": "2q", 2024-09-20T22:03:47.346 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-09-20T22:03:47.346 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_clone_cow": "true", 2024-09-20T22:03:47.346 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_compression_algorithm": "snappy", 2024-09-20T22:03:47.346 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_compression_max_blob_size": "0", 2024-09-20T22:03:47.346 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-09-20T22:03:47.346 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-09-20T22:03:47.346 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_compression_min_blob_size": "0", 2024-09-20T22:03:47.346 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-09-20T22:03:47.346 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-09-20T22:03:47.346 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_compression_mode": "none", 2024-09-20T22:03:47.346 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-09-20T22:03:47.346 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_csum_type": "crc32c", 2024-09-20T22:03:47.346 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_enforce_settings": "default", 2024-09-20T22:03:47.347 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_freelist": "false", 2024-09-20T22:03:47.347 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_fsck_abort": "false", 2024-09-20T22:03:47.347 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-09-20T22:03:47.347 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-09-20T22:03:47.347 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_inject_read_err": "false", 2024-09-20T22:03:47.347 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_legacy_omap": "false", 2024-09-20T22:03:47.347 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-09-20T22:03:47.347 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-09-20T22:03:47.347 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-09-20T22:03:47.347 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-09-20T22:03:47.347 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-09-20T22:03:47.347 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-09-20T22:03:47.347 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-09-20T22:03:47.347 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_small_allocations": "0", 2024-09-20T22:03:47.348 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-09-20T22:03:47.348 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_default_buffered_read": "true", 2024-09-20T22:03:47.348 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_default_buffered_write": "false", 2024-09-20T22:03:47.348 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_deferred_batch_ops": "0", 2024-09-20T22:03:47.348 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-09-20T22:03:47.348 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-09-20T22:03:47.348 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_elastic_shared_blobs": "true", 2024-09-20T22:03:47.348 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-09-20T22:03:47.348 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-09-20T22:03:47.348 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-09-20T22:03:47.348 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-09-20T22:03:47.348 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-09-20T22:03:47.348 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fail_eio": "false", 2024-09-20T22:03:47.349 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-09-20T22:03:47.349 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-09-20T22:03:47.349 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-09-20T22:03:47.349 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-09-20T22:03:47.349 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_on_mkfs": "true", 2024-09-20T22:03:47.349 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-09-20T22:03:47.349 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_on_mount": "false", 2024-09-20T22:03:47.349 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-09-20T22:03:47.349 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_on_umount": "false", 2024-09-20T22:03:47.349 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-09-20T22:03:47.349 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-09-20T22:03:47.349 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-09-20T22:03:47.349 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-09-20T22:03:47.349 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-09-20T22:03:47.350 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-09-20T22:03:47.350 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-09-20T22:03:47.350 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-09-20T22:03:47.350 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_ignore_data_csum": "false", 2024-09-20T22:03:47.350 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-09-20T22:03:47.350 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_kvbackend": "rocksdb", 2024-09-20T22:03:47.350 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-09-20T22:03:47.350 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-09-20T22:03:47.350 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_log_op_age": "5.000000", 2024-09-20T22:03:47.350 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_max_alloc_size": "0", 2024-09-20T22:03:47.350 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_max_blob_size": "0", 2024-09-20T22:03:47.350 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-09-20T22:03:47.350 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-09-20T22:03:47.351 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_max_defer_interval": "3.000000", 2024-09-20T22:03:47.351 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_max_deferred_txc": "32", 2024-09-20T22:03:47.351 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_min_alloc_size": "0", 2024-09-20T22:03:47.351 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-09-20T22:03:47.351 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-09-20T22:03:47.351 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_nid_prealloc": "1024", 2024-09-20T22:03:47.351 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_prefer_deferred_size": "0", 2024-09-20T22:03:47.351 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-09-20T22:03:47.351 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-09-20T22:03:47.351 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_qfsck_on_mount": "true", 2024-09-20T22:03:47.351 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_retry_disk_reads": "3", 2024-09-20T22:03:47.351 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_rocksdb_cf": "true", 2024-09-20T22:03:47.352 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-09-20T22:03:47.352 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-09-20T22:03:47.352 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_rocksdb_options_annex": "", 2024-09-20T22:03:47.352 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_slow_ops_warn_lifetime": "86400", 2024-09-20T22:03:47.352 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_slow_ops_warn_threshold": "1", 2024-09-20T22:03:47.352 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_spdk_coremask": "0x1", 2024-09-20T22:03:47.352 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_spdk_io_sleep": "5", 2024-09-20T22:03:47.352 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_spdk_max_io_completion": "0", 2024-09-20T22:03:47.352 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_spdk_mem": "512", 2024-09-20T22:03:47.352 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_sync_submit_transaction": "false", 2024-09-20T22:03:47.352 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_throttle_bytes": "67108864", 2024-09-20T22:03:47.353 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_throttle_cost_per_io": "0", 2024-09-20T22:03:47.353 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-09-20T22:03:47.353 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-09-20T22:03:47.353 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-09-20T22:03:47.353 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-09-20T22:03:47.353 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_tracing": "false", 2024-09-20T22:03:47.353 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-09-20T22:03:47.353 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-09-20T22:03:47.353 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_volume_selection_reserved": "0", 2024-09-20T22:03:47.353 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-09-20T22:03:47.353 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-09-20T22:03:47.353 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-09-20T22:03:47.353 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-09-20T22:03:47.353 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-09-20T22:03:47.354 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-09-20T22:03:47.354 INFO:teuthology.orchestra.run.smithi060.stdout: "bluestore_zero_block_detection": "false", 2024-09-20T22:03:47.354 INFO:teuthology.orchestra.run.smithi060.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-09-20T22:03:47.354 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_action_update_interval": "2", 2024-09-20T22:03:47.354 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-09-20T22:03:47.354 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-09-20T22:03:47.354 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-09-20T22:03:47.354 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-09-20T22:03:47.354 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_mount_timeout": "10", 2024-09-20T22:03:47.354 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-09-20T22:03:47.354 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-09-20T22:03:47.355 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-09-20T22:03:47.355 INFO:teuthology.orchestra.run.smithi060.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-09-20T22:03:47.355 INFO:teuthology.orchestra.run.smithi060.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-09-20T22:03:47.355 INFO:teuthology.orchestra.run.smithi060.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-09-20T22:03:47.355 INFO:teuthology.orchestra.run.smithi060.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-09-20T22:03:47.355 INFO:teuthology.orchestra.run.smithi060.stdout: "cephx_cluster_require_signatures": "false", 2024-09-20T22:03:47.355 INFO:teuthology.orchestra.run.smithi060.stdout: "cephx_cluster_require_version": "2", 2024-09-20T22:03:47.355 INFO:teuthology.orchestra.run.smithi060.stdout: "cephx_require_signatures": "false", 2024-09-20T22:03:47.355 INFO:teuthology.orchestra.run.smithi060.stdout: "cephx_require_version": "2", 2024-09-20T22:03:47.355 INFO:teuthology.orchestra.run.smithi060.stdout: "cephx_service_require_signatures": "false", 2024-09-20T22:03:47.355 INFO:teuthology.orchestra.run.smithi060.stdout: "cephx_service_require_version": "2", 2024-09-20T22:03:47.355 INFO:teuthology.orchestra.run.smithi060.stdout: "cephx_sign_messages": "true", 2024-09-20T22:03:47.355 INFO:teuthology.orchestra.run.smithi060.stdout: "chdir": "", 2024-09-20T22:03:47.355 INFO:teuthology.orchestra.run.smithi060.stdout: "client_acl_type": "", 2024-09-20T22:03:47.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client_asio_thread_count": "2", 2024-09-20T22:03:47.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client_cache_mid": "0.750000", 2024-09-20T22:03:47.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client_cache_size": "16384", 2024-09-20T22:03:47.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client_caps_release_delay": "5", 2024-09-20T22:03:47.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client_check_pool_perm": "true", 2024-09-20T22:03:47.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client_collect_and_send_global_metrics": "false", 2024-09-20T22:03:47.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client_debug_force_sync_read": "false", 2024-09-20T22:03:47.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client_debug_getattr_caps": "false", 2024-09-20T22:03:47.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client_debug_inject_features": "", 2024-09-20T22:03:47.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client_debug_inject_tick_delay": "0", 2024-09-20T22:03:47.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-09-20T22:03:47.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client_die_on_failed_remount": "false", 2024-09-20T22:03:47.356 INFO:teuthology.orchestra.run.smithi060.stdout: "client_dirsize_rbytes": "true", 2024-09-20T22:03:47.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_force_lazyio": "false", 2024-09-20T22:03:47.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_fs": "", 2024-09-20T22:03:47.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_inject_fixed_oldest_tid": "false", 2024-09-20T22:03:47.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_inject_release_failure": "false", 2024-09-20T22:03:47.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_max_inline_size": "4096", 2024-09-20T22:03:47.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_max_retries_on_remount_failure": "5", 2024-09-20T22:03:47.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_mds_namespace": "", 2024-09-20T22:03:47.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_metadata": "", 2024-09-20T22:03:47.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_mount_gid": "-1", 2024-09-20T22:03:47.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_mount_timeout": "300", 2024-09-20T22:03:47.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_mount_uid": "-1", 2024-09-20T22:03:47.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_mountpoint": "/", 2024-09-20T22:03:47.357 INFO:teuthology.orchestra.run.smithi060.stdout: "client_notify_timeout": "10", 2024-09-20T22:03:47.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_oc": "true", 2024-09-20T22:03:47.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_oc_max_dirty": "104857600", 2024-09-20T22:03:47.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_oc_max_dirty_age": "5.000000", 2024-09-20T22:03:47.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_oc_max_objects": "1000", 2024-09-20T22:03:47.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_oc_size": "209715200", 2024-09-20T22:03:47.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_oc_target_dirty": "8388608", 2024-09-20T22:03:47.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_permissions": "true", 2024-09-20T22:03:47.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_quota": "true", 2024-09-20T22:03:47.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_quota_df": "true", 2024-09-20T22:03:47.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_readahead_max_bytes": "0", 2024-09-20T22:03:47.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_readahead_max_periods": "4", 2024-09-20T22:03:47.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_readahead_min": "131072", 2024-09-20T22:03:47.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_reconnect_stale": "false", 2024-09-20T22:03:47.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_shutdown_timeout": "30", 2024-09-20T22:03:47.359 INFO:teuthology.orchestra.run.smithi060.stdout: "client_snapdir": ".snap", 2024-09-20T22:03:47.359 INFO:teuthology.orchestra.run.smithi060.stdout: "client_tick_interval": "1", 2024-09-20T22:03:47.359 INFO:teuthology.orchestra.run.smithi060.stdout: "client_trace": "", 2024-09-20T22:03:47.359 INFO:teuthology.orchestra.run.smithi060.stdout: "client_try_dentry_invalidate": "false", 2024-09-20T22:03:47.359 INFO:teuthology.orchestra.run.smithi060.stdout: "client_use_faked_inos": "false", 2024-09-20T22:03:47.359 INFO:teuthology.orchestra.run.smithi060.stdout: "client_use_random_mds": "false", 2024-09-20T22:03:47.359 INFO:teuthology.orchestra.run.smithi060.stdout: "clog_to_graylog": "false", 2024-09-20T22:03:47.359 INFO:teuthology.orchestra.run.smithi060.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-09-20T22:03:47.359 INFO:teuthology.orchestra.run.smithi060.stdout: "clog_to_graylog_port": "12201", 2024-09-20T22:03:47.359 INFO:teuthology.orchestra.run.smithi060.stdout: "clog_to_monitors": "default=true", 2024-09-20T22:03:47.359 INFO:teuthology.orchestra.run.smithi060.stdout: "clog_to_syslog": "false", 2024-09-20T22:03:47.359 INFO:teuthology.orchestra.run.smithi060.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-09-20T22:03:47.359 INFO:teuthology.orchestra.run.smithi060.stdout: "clog_to_syslog_level": "info", 2024-09-20T22:03:47.359 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "-", 2024-09-20T22:03:47.360 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_network": "", 2024-09-20T22:03:47.360 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_network_interface": "", 2024-09-20T22:03:47.360 INFO:teuthology.orchestra.run.smithi060.stdout: "compressor_zlib_isal": "false", 2024-09-20T22:03:47.360 INFO:teuthology.orchestra.run.smithi060.stdout: "compressor_zlib_level": "5", 2024-09-20T22:03:47.360 INFO:teuthology.orchestra.run.smithi060.stdout: "compressor_zlib_winsize": "-15", 2024-09-20T22:03:47.360 INFO:teuthology.orchestra.run.smithi060.stdout: "compressor_zstd_level": "1", 2024-09-20T22:03:47.360 INFO:teuthology.orchestra.run.smithi060.stdout: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5", 2024-09-20T22:03:47.360 INFO:teuthology.orchestra.run.smithi060.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-09-20T22:03:47.360 INFO:teuthology.orchestra.run.smithi060.stdout: "crimson_alien_op_num_threads": "6", 2024-09-20T22:03:47.360 INFO:teuthology.orchestra.run.smithi060.stdout: "crimson_alien_thread_cpu_cores": "", 2024-09-20T22:03:47.360 INFO:teuthology.orchestra.run.smithi060.stdout: "crimson_osd_obc_lru_size": "512", 2024-09-20T22:03:47.360 INFO:teuthology.orchestra.run.smithi060.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-09-20T22:03:47.360 INFO:teuthology.orchestra.run.smithi060.stdout: "crimson_osd_stat_interval": "0", 2024-09-20T22:03:47.361 INFO:teuthology.orchestra.run.smithi060.stdout: "crimson_seastar_cpu_cores": "", 2024-09-20T22:03:47.361 INFO:teuthology.orchestra.run.smithi060.stdout: "crimson_seastar_num_threads": "0", 2024-09-20T22:03:47.361 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_location": "", 2024-09-20T22:03:47.361 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_location_hook": "", 2024-09-20T22:03:47.361 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_location_hook_timeout": "10", 2024-09-20T22:03:47.361 INFO:teuthology.orchestra.run.smithi060.stdout: "daemonize": "false", 2024-09-20T22:03:47.361 INFO:teuthology.orchestra.run.smithi060.stdout: "daos_pool": "tank", 2024-09-20T22:03:47.361 INFO:teuthology.orchestra.run.smithi060.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-09-20T22:03:47.361 INFO:teuthology.orchestra.run.smithi060.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-09-20T22:03:47.361 INFO:teuthology.orchestra.run.smithi060.stdout: "dbstore_db_name_prefix": "dbstore", 2024-09-20T22:03:47.361 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_alienstore": "0/5", 2024-09-20T22:03:47.361 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_allow_any_pool_priority": "false", 2024-09-20T22:03:47.361 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_asok": "1/5", 2024-09-20T22:03:47.361 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_asok_assert_abort": "false", 2024-09-20T22:03:47.362 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_asserts_on_shutdown": "true", 2024-09-20T22:03:47.362 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_auth": "1/5", 2024-09-20T22:03:47.362 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_bdev": "1/3", 2024-09-20T22:03:47.362 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_bluefs": "1/5", 2024-09-20T22:03:47.362 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_bluestore": "1/5", 2024-09-20T22:03:47.362 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_buffer": "0/1", 2024-09-20T22:03:47.362 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_ceph_exporter": "1/5", 2024-09-20T22:03:47.362 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_cephfs_mirror": "0/5", 2024-09-20T22:03:47.362 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_cephsqlite": "0/5", 2024-09-20T22:03:47.362 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_client": "0/5", 2024-09-20T22:03:47.362 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_compressor": "1/5", 2024-09-20T22:03:47.362 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_context": "0/1", 2024-09-20T22:03:47.362 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_crimson_interrupt": "0/5", 2024-09-20T22:03:47.363 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_crush": "1/1", 2024-09-20T22:03:47.363 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_crypto": "1/5", 2024-09-20T22:03:47.363 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_cyanstore": "0/5", 2024-09-20T22:03:47.363 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_deliberately_leak_memory": "false", 2024-09-20T22:03:47.363 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_disable_randomized_ping": "false", 2024-09-20T22:03:47.363 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_dpdk": "1/5", 2024-09-20T22:03:47.363 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_eventtrace": "1/5", 2024-09-20T22:03:47.363 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_filer": "0/1", 2024-09-20T22:03:47.363 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_filestore": "1/3", 2024-09-20T22:03:47.363 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_finisher": "1/1", 2024-09-20T22:03:47.363 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_fuse": "1/5", 2024-09-20T22:03:47.363 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_heartbeat_testing_span": "0", 2024-09-20T22:03:47.363 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_heartbeatmap": "1/5", 2024-09-20T22:03:47.364 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_immutable_obj_cache": "0/5", 2024-09-20T22:03:47.364 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_javaclient": "1/5", 2024-09-20T22:03:47.364 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_journal": "1/3", 2024-09-20T22:03:47.364 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_journaler": "0/5", 2024-09-20T22:03:47.364 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_kstore": "1/5", 2024-09-20T22:03:47.364 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_lockdep": "0/1", 2024-09-20T22:03:47.364 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mclock": "1/5", 2024-09-20T22:03:47.364 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mds": "1/5", 2024-09-20T22:03:47.364 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mds_balancer": "1/5", 2024-09-20T22:03:47.364 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mds_locker": "1/5", 2024-09-20T22:03:47.364 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mds_log": "1/5", 2024-09-20T22:03:47.364 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mds_log_expire": "1/5", 2024-09-20T22:03:47.364 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mds_migrator": "1/5", 2024-09-20T22:03:47.365 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mds_quiesce": "3/5", 2024-09-20T22:03:47.365 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_memstore": "1/5", 2024-09-20T22:03:47.365 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mgr": "2/5", 2024-09-20T22:03:47.365 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mgrc": "1/5", 2024-09-20T22:03:47.365 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_mon": "20/20", 2024-09-20T22:03:47.365 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_monc": "0/10", 2024-09-20T22:03:47.365 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_ms": "1/1", 2024-09-20T22:03:47.365 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_none": "0/5", 2024-09-20T22:03:47.365 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_objclass": "0/5", 2024-09-20T22:03:47.365 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_objectcacher": "0/5", 2024-09-20T22:03:47.365 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_objecter": "0/1", 2024-09-20T22:03:47.365 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_optracker": "0/5", 2024-09-20T22:03:47.365 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_osd": "1/5", 2024-09-20T22:03:47.365 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_paxos": "20/20", 2024-09-20T22:03:47.366 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_perfcounter": "1/5", 2024-09-20T22:03:47.366 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_prioritycache": "1/5", 2024-09-20T22:03:47.366 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rados": "0/5", 2024-09-20T22:03:47.366 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rbd": "0/5", 2024-09-20T22:03:47.366 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rbd_mirror": "0/5", 2024-09-20T22:03:47.366 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rbd_pwl": "0/5", 2024-09-20T22:03:47.366 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rbd_replay": "0/5", 2024-09-20T22:03:47.366 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_refs": "0/0", 2024-09-20T22:03:47.366 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_reserver": "1/1", 2024-09-20T22:03:47.366 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rgw": "1/5", 2024-09-20T22:03:47.366 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rgw_access": "1/5", 2024-09-20T22:03:47.366 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rgw_datacache": "1/5", 2024-09-20T22:03:47.366 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rgw_dbstore": "1/5", 2024-09-20T22:03:47.366 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rgw_flight": "1/5", 2024-09-20T22:03:47.367 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rgw_lifecycle": "1/5", 2024-09-20T22:03:47.367 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rgw_sync": "1/5", 2024-09-20T22:03:47.367 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_rocksdb": "4/5", 2024-09-20T22:03:47.367 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore": "0/5", 2024-09-20T22:03:47.367 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_backref": "0/5", 2024-09-20T22:03:47.367 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_cache": "0/5", 2024-09-20T22:03:47.367 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_cleaner": "0/5", 2024-09-20T22:03:47.367 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_device": "0/5", 2024-09-20T22:03:47.367 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:47 smithi060 ceph-mon[29694]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 552 B/s rd, 0 op/s 2024-09-20T22:03:47.368 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:47 smithi060 ceph-mon[40290]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 552 B/s rd, 0 op/s 2024-09-20T22:03:47.368 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_epm": "0/5", 2024-09-20T22:03:47.369 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-09-20T22:03:47.369 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_journal": "0/5", 2024-09-20T22:03:47.369 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_lba": "0/5", 2024-09-20T22:03:47.369 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_odata": "0/5", 2024-09-20T22:03:47.369 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_omap": "0/5", 2024-09-20T22:03:47.369 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_onode": "0/5", 2024-09-20T22:03:47.369 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_t": "0/5", 2024-09-20T22:03:47.369 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_seastore_tm": "0/5", 2024-09-20T22:03:47.369 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_striper": "0/1", 2024-09-20T22:03:47.369 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_test": "0/5", 2024-09-20T22:03:47.369 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_throttle": "1/1", 2024-09-20T22:03:47.369 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_timer": "0/1", 2024-09-20T22:03:47.369 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_tp": "0/5", 2024-09-20T22:03:47.370 INFO:teuthology.orchestra.run.smithi060.stdout: "debug_trace": "1/5", 2024-09-20T22:03:47.370 INFO:teuthology.orchestra.run.smithi060.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-09-20T22:03:47.370 INFO:teuthology.orchestra.run.smithi060.stdout: "device_failure_prediction_mode": "none", 2024-09-20T22:03:47.370 INFO:teuthology.orchestra.run.smithi060.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-09-20T22:03:47.370 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-09-20T22:03:47.370 INFO:teuthology.orchestra.run.smithi060.stdout: "err_to_graylog": "false", 2024-09-20T22:03:47.370 INFO:teuthology.orchestra.run.smithi060.stdout: "err_to_journald": "false", 2024-09-20T22:03:47.370 INFO:teuthology.orchestra.run.smithi060.stdout: "err_to_stderr": "true", 2024-09-20T22:03:47.370 INFO:teuthology.orchestra.run.smithi060.stdout: "err_to_syslog": "false", 2024-09-20T22:03:47.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event_tracing": "false", 2024-09-20T22:03:47.370 INFO:teuthology.orchestra.run.smithi060.stdout: "exporter_addr": "0.0.0.0", 2024-09-20T22:03:47.370 INFO:teuthology.orchestra.run.smithi060.stdout: "exporter_http_port": "9926", 2024-09-20T22:03:47.370 INFO:teuthology.orchestra.run.smithi060.stdout: "exporter_prio_limit": "5", 2024-09-20T22:03:47.371 INFO:teuthology.orchestra.run.smithi060.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-09-20T22:03:47.371 INFO:teuthology.orchestra.run.smithi060.stdout: "exporter_sort_metrics": "true", 2024-09-20T22:03:47.371 INFO:teuthology.orchestra.run.smithi060.stdout: "exporter_stats_period": "5", 2024-09-20T22:03:47.371 INFO:teuthology.orchestra.run.smithi060.stdout: "fake_statfs_for_testing": "0", 2024-09-20T22:03:47.371 INFO:teuthology.orchestra.run.smithi060.stdout: "fatal_signal_handlers": "true", 2024-09-20T22:03:47.371 INFO:teuthology.orchestra.run.smithi060.stdout: "filer_max_purge_ops": "10", 2024-09-20T22:03:47.371 INFO:teuthology.orchestra.run.smithi060.stdout: "filer_max_truncate_ops": "128", 2024-09-20T22:03:47.371 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_apply_finisher_threads": "1", 2024-09-20T22:03:47.371 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_blackhole": "false", 2024-09-20T22:03:47.371 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_btrfs_clone_range": "true", 2024-09-20T22:03:47.371 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_btrfs_snap": "true", 2024-09-20T22:03:47.371 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_caller_concurrency": "10", 2024-09-20T22:03:47.371 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_collect_device_partition_information": "true", 2024-09-20T22:03:47.372 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_commit_timeout": "600.000000", 2024-09-20T22:03:47.372 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_debug_inject_read_err": "false", 2024-09-20T22:03:47.372 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_debug_omap_check": "false", 2024-09-20T22:03:47.372 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_debug_random_read_err": "0.000000", 2024-09-20T22:03:47.372 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_debug_verify_split": "false", 2024-09-20T22:03:47.372 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_dump_file": "", 2024-09-20T22:03:47.372 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-09-20T22:03:47.372 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-09-20T22:03:47.372 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_fadvise": "true", 2024-09-20T22:03:47.372 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_fail_eio": "true", 2024-09-20T22:03:47.372 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_fd_cache_shards": "16", 2024-09-20T22:03:47.372 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_fd_cache_size": "128", 2024-09-20T22:03:47.372 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_fiemap": "false", 2024-09-20T22:03:47.372 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_fiemap_threshold": "4096", 2024-09-20T22:03:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-09-20T22:03:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_index_retry_probability": "0.000000", 2024-09-20T22:03:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_inject_stall": "0", 2024-09-20T22:03:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_journal_parallel": "false", 2024-09-20T22:03:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_journal_trailing": "false", 2024-09-20T22:03:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_journal_writeahead": "false", 2024-09-20T22:03:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_kill_at": "0", 2024-09-20T22:03:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-09-20T22:03:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_inline_xattr_size": "0", 2024-09-20T22:03:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-09-20T22:03:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-09-20T22:03:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-09-20T22:03:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_inline_xattrs": "0", 2024-09-20T22:03:47.373 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-09-20T22:03:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_inline_xattrs_other": "2", 2024-09-20T22:03:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-09-20T22:03:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_sync_interval": "5.000000", 2024-09-20T22:03:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_xattr_value_size": "0", 2024-09-20T22:03:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-09-20T22:03:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-09-20T22:03:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-09-20T22:03:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_merge_threshold": "-10", 2024-09-20T22:03:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_min_sync_interval": "0.010000", 2024-09-20T22:03:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_odsync_write": "false", 2024-09-20T22:03:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_omap_backend": "rocksdb", 2024-09-20T22:03:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_omap_backend_path": "", 2024-09-20T22:03:47.374 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_omap_header_cache_size": "1024", 2024-09-20T22:03:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_ondisk_finisher_threads": "1", 2024-09-20T22:03:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-09-20T22:03:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_op_thread_timeout": "60", 2024-09-20T22:03:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_op_threads": "2", 2024-09-20T22:03:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_punch_hole": "false", 2024-09-20T22:03:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-09-20T22:03:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-09-20T22:03:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-09-20T22:03:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-09-20T22:03:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-09-20T22:03:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_max_bytes": "104857600", 2024-09-20T22:03:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-09-20T22:03:47.375 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-09-20T22:03:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-09-20T22:03:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_queue_max_ops": "50", 2024-09-20T22:03:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-09-20T22:03:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_seek_data_hole": "false", 2024-09-20T22:03:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_sloppy_crc": "false", 2024-09-20T22:03:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-09-20T22:03:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_splice": "false", 2024-09-20T22:03:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_split_multiple": "2", 2024-09-20T22:03:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_split_rand_factor": "20", 2024-09-20T22:03:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_update_to": "1000", 2024-09-20T22:03:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-09-20T22:03:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-09-20T22:03:47.376 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-09-20T22:03:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-09-20T22:03:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-09-20T22:03:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-09-20T22:03:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_enable": "true", 2024-09-20T22:03:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-09-20T22:03:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-09-20T22:03:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-09-20T22:03:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-09-20T22:03:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-09-20T22:03:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-09-20T22:03:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_xfs_extsize": "false", 2024-09-20T22:03:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "filestore_zfs_snap": "false", 2024-09-20T22:03:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "fio_dir": "/tmp/fio", 2024-09-20T22:03:47.377 INFO:teuthology.orchestra.run.smithi060.stdout: "fsid": "3df9647e-779b-11ef-baf5-efdc52797490", 2024-09-20T22:03:47.378 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_allow_other": "true", 2024-09-20T22:03:47.378 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_atomic_o_trunc": "true", 2024-09-20T22:03:47.378 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_big_writes": "true", 2024-09-20T22:03:47.378 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_debug": "false", 2024-09-20T22:03:47.378 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_default_permissions": "false", 2024-09-20T22:03:47.378 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_disable_pagecache": "false", 2024-09-20T22:03:47.378 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_max_write": "0", 2024-09-20T22:03:47.378 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_multithreaded": "true", 2024-09-20T22:03:47.378 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_require_active_mds": "true", 2024-09-20T22:03:47.378 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_set_user_groups": "true", 2024-09-20T22:03:47.378 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_splice_move": "true", 2024-09-20T22:03:47.378 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_splice_read": "true", 2024-09-20T22:03:47.378 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_splice_write": "true", 2024-09-20T22:03:47.379 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_syncfs_on_mksnap": "true", 2024-09-20T22:03:47.379 INFO:teuthology.orchestra.run.smithi060.stdout: "fuse_use_invalidate_cb": "true", 2024-09-20T22:03:47.379 INFO:teuthology.orchestra.run.smithi060.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-09-20T22:03:47.379 INFO:teuthology.orchestra.run.smithi060.stdout: "gss_target_name": "ceph", 2024-09-20T22:03:47.379 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_file": "", 2024-09-20T22:03:47.379 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_inject_failure": "0", 2024-09-20T22:03:47.379 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_interval": "5", 2024-09-20T22:03:47.379 INFO:teuthology.orchestra.run.smithi060.stdout: "host": "smithi060", 2024-09-20T22:03:47.379 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-09-20T22:03:47.379 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-09-20T22:03:47.379 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_max_size": "1073741824", 2024-09-20T22:03:47.379 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-09-20T22:03:47.379 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-09-20T22:03:47.380 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-09-20T22:03:47.380 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-09-20T22:03:47.380 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-09-20T22:03:47.380 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-09-20T22:03:47.380 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-09-20T22:03:47.380 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-09-20T22:03:47.380 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-09-20T22:03:47.380 INFO:teuthology.orchestra.run.smithi060.stdout: "immutable_object_cache_watermark": "0.900000", 2024-09-20T22:03:47.380 INFO:teuthology.orchestra.run.smithi060.stdout: "inject_early_sigterm": "false", 2024-09-20T22:03:47.380 INFO:teuthology.orchestra.run.smithi060.stdout: "jaeger_agent_port": "6799", 2024-09-20T22:03:47.380 INFO:teuthology.orchestra.run.smithi060.stdout: "jaeger_tracing_enable": "false", 2024-09-20T22:03:47.380 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_aio": "true", 2024-09-20T22:03:47.380 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_align_min_size": "65536", 2024-09-20T22:03:47.380 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_block_align": "true", 2024-09-20T22:03:47.380 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_block_size": "4096", 2024-09-20T22:03:47.381 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_dio": "true", 2024-09-20T22:03:47.381 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_discard": "false", 2024-09-20T22:03:47.381 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_force_aio": "false", 2024-09-20T22:03:47.381 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_ignore_corruption": "false", 2024-09-20T22:03:47.381 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_max_write_bytes": "10485760", 2024-09-20T22:03:47.381 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_max_write_entries": "100", 2024-09-20T22:03:47.381 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_replay_from": "0", 2024-09-20T22:03:47.381 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_throttle_high_multiple": "0.000000", 2024-09-20T22:03:47.381 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-09-20T22:03:47.381 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-09-20T22:03:47.381 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_throttle_max_multiple": "0.000000", 2024-09-20T22:03:47.381 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_write_header_frequency": "0", 2024-09-20T22:03:47.381 INFO:teuthology.orchestra.run.smithi060.stdout: "journal_zero_on_create": "false", 2024-09-20T22:03:47.381 INFO:teuthology.orchestra.run.smithi060.stdout: "journaler_prefetch_periods": "10", 2024-09-20T22:03:47.382 INFO:teuthology.orchestra.run.smithi060.stdout: "journaler_prezero_periods": "5", 2024-09-20T22:03:47.382 INFO:teuthology.orchestra.run.smithi060.stdout: "journaler_write_head_interval": "15", 2024-09-20T22:03:47.382 INFO:teuthology.orchestra.run.smithi060.stdout: "key": "", 2024-09-20T22:03:47.382 INFO:teuthology.orchestra.run.smithi060.stdout: "keyfile": "", 2024-09-20T22:03:47.382 INFO:teuthology.orchestra.run.smithi060.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-09-20T22:03:47.382 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_backend": "rocksdb", 2024-09-20T22:03:47.382 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_default_stripe_size": "65536", 2024-09-20T22:03:47.382 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_fsck_on_mount": "false", 2024-09-20T22:03:47.382 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_fsck_on_mount_deep": "true", 2024-09-20T22:03:47.382 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_max_bytes": "67108864", 2024-09-20T22:03:47.382 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_max_ops": "512", 2024-09-20T22:03:47.382 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_nid_prealloc": "1024", 2024-09-20T22:03:47.382 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_onode_map_size": "1024", 2024-09-20T22:03:47.382 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-09-20T22:03:47.382 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_sync_submit_transaction": "false", 2024-09-20T22:03:47.383 INFO:teuthology.orchestra.run.smithi060.stdout: "kstore_sync_transaction": "false", 2024-09-20T22:03:47.383 INFO:teuthology.orchestra.run.smithi060.stdout: "librados_thread_count": "2", 2024-09-20T22:03:47.383 INFO:teuthology.orchestra.run.smithi060.stdout: "lockdep": "false", 2024-09-20T22:03:47.383 INFO:teuthology.orchestra.run.smithi060.stdout: "lockdep_force_backtrace": "false", 2024-09-20T22:03:47.383 INFO:teuthology.orchestra.run.smithi060.stdout: "log_coarse_timestamps": "true", 2024-09-20T22:03:47.383 INFO:teuthology.orchestra.run.smithi060.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-09-20T22:03:47.383 INFO:teuthology.orchestra.run.smithi060.stdout: "log_flush_on_exit": "false", 2024-09-20T22:03:47.383 INFO:teuthology.orchestra.run.smithi060.stdout: "log_graylog_host": "127.0.0.1", 2024-09-20T22:03:47.383 INFO:teuthology.orchestra.run.smithi060.stdout: "log_graylog_port": "12201", 2024-09-20T22:03:47.383 INFO:teuthology.orchestra.run.smithi060.stdout: "log_max_new": "1000", 2024-09-20T22:03:47.383 INFO:teuthology.orchestra.run.smithi060.stdout: "log_max_recent": "10000", 2024-09-20T22:03:47.383 INFO:teuthology.orchestra.run.smithi060.stdout: "log_stderr_prefix": "", 2024-09-20T22:03:47.383 INFO:teuthology.orchestra.run.smithi060.stdout: "log_stop_at_utilization": "0.970000", 2024-09-20T22:03:47.383 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_file": "true", 2024-09-20T22:03:47.383 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_graylog": "false", 2024-09-20T22:03:47.384 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_journald": "false", 2024-09-20T22:03:47.384 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_stderr": "false", 2024-09-20T22:03:47.384 INFO:teuthology.orchestra.run.smithi060.stdout: "log_to_syslog": "false", 2024-09-20T22:03:47.384 INFO:teuthology.orchestra.run.smithi060.stdout: "max_rotating_auth_attempts": "10", 2024-09-20T22:03:47.384 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-09-20T22:03:47.384 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_action_on_write_error": "1", 2024-09-20T22:03:47.384 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_alternate_name_max": "8192", 2024-09-20T22:03:47.384 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_asio_thread_count": "2", 2024-09-20T22:03:47.384 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_export_pin": "true", 2024-09-20T22:03:47.384 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_fragment_dirs": "true", 2024-09-20T22:03:47.384 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-09-20T22:03:47.384 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_fragment_interval": "5", 2024-09-20T22:03:47.384 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_fragment_size_max": "100000", 2024-09-20T22:03:47.384 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_idle_threshold": "0.000000", 2024-09-20T22:03:47.385 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_interval": "10", 2024-09-20T22:03:47.385 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_max": "-1", 2024-09-20T22:03:47.385 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_max_until": "-1", 2024-09-20T22:03:47.385 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_merge_size": "50", 2024-09-20T22:03:47.385 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_midchunk": "0.300000", 2024-09-20T22:03:47.385 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_min_rebalance": "0.100000", 2024-09-20T22:03:47.385 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_min_start": "0.200000", 2024-09-20T22:03:47.385 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_minchunk": "0.001000", 2024-09-20T22:03:47.385 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_mode": "0", 2024-09-20T22:03:47.385 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_need_max": "1.200000", 2024-09-20T22:03:47.385 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_need_min": "0.800000", 2024-09-20T22:03:47.385 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_overload_epochs": "2", 2024-09-20T22:03:47.385 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-09-20T22:03:47.385 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_sample_interval": "3.000000", 2024-09-20T22:03:47.386 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_split_bits": "3", 2024-09-20T22:03:47.386 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_split_rd": "25000.000000", 2024-09-20T22:03:47.386 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_split_size": "10000", 2024-09-20T22:03:47.386 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_split_wr": "10000.000000", 2024-09-20T22:03:47.386 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_target_decay": "10.000000", 2024-09-20T22:03:47.386 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-09-20T22:03:47.386 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_beacon_grace": "15.000000", 2024-09-20T22:03:47.386 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_beacon_interval": "4.000000", 2024-09-20T22:03:47.386 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_beacon_mon_down_grace": "60", 2024-09-20T22:03:47.386 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_memory_limit": "4294967296", 2024-09-20T22:03:47.386 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_mid": "0.700000", 2024-09-20T22:03:47.386 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-09-20T22:03:47.387 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_quiesce_delay": "0", 2024-09-20T22:03:47.387 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_quiesce_sleep": "200", 2024-09-20T22:03:47.387 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_quiesce_splitauth": "true", 2024-09-20T22:03:47.387 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_quiesce_threshold": "524288", 2024-09-20T22:03:47.387 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_release_free_interval": "10", 2024-09-20T22:03:47.387 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_reservation": "0.050000", 2024-09-20T22:03:47.387 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-09-20T22:03:47.387 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_trim_interval": "1", 2024-09-20T22:03:47.387 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cache_trim_threshold": "262144", 2024-09-20T22:03:47.387 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-09-20T22:03:47.387 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-09-20T22:03:47.387 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_client_delegate_inos_pct": "50", 2024-09-20T22:03:47.387 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_client_prealloc_inos": "1000", 2024-09-20T22:03:47.388 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-09-20T22:03:47.388 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_connect_bootstrapping": "false", 2024-09-20T22:03:47.388 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_damage_table_max_entries": "10000", 2024-09-20T22:03:47.388 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-09-20T22:03:47.388 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_debug_auth_pins": "false", 2024-09-20T22:03:47.388 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_debug_frag": "false", 2024-09-20T22:03:47.388 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_debug_scatterstat": "false", 2024-09-20T22:03:47.388 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_debug_subtrees": "false", 2024-09-20T22:03:47.388 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_decay_halflife": "5.000000", 2024-09-20T22:03:47.388 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_default_dir_hash": "2", 2024-09-20T22:03:47.388 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_defer_session_stale": "true", 2024-09-20T22:03:47.388 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_deny_all_reconnect": "false", 2024-09-20T22:03:47.388 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_dir_keys_per_op": "16384", 2024-09-20T22:03:47.388 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_dir_max_commit_size": "10", 2024-09-20T22:03:47.389 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_dir_max_entries": "0", 2024-09-20T22:03:47.389 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_dir_prefetch": "true", 2024-09-20T22:03:47.389 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_dirstat_min_interval": "1.000000", 2024-09-20T22:03:47.389 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_dump_cache_after_rejoin": "false", 2024-09-20T22:03:47.389 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_dump_cache_on_map": "false", 2024-09-20T22:03:47.389 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_dump_cache_threshold_file": "0", 2024-09-20T22:03:47.389 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-09-20T22:03:47.389 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_early_reply": "true", 2024-09-20T22:03:47.389 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_enable_op_tracker": "true", 2024-09-20T22:03:47.389 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_enforce_unique_name": "true", 2024-09-20T22:03:47.389 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_export_ephemeral_distributed": "true", 2024-09-20T22:03:47.389 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-09-20T22:03:47.390 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_export_ephemeral_random": "true", 2024-09-20T22:03:47.390 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-09-20T22:03:47.390 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-09-20T22:03:47.390 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_forward_all_requests_to_auth": "false", 2024-09-20T22:03:47.390 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-09-20T22:03:47.390 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-09-20T22:03:47.390 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-09-20T22:03:47.390 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-09-20T22:03:47.390 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_health_cache_threshold": "1.500000", 2024-09-20T22:03:47.390 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_health_summarize_threshold": "10", 2024-09-20T22:03:47.390 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_heartbeat_grace": "15.000000", 2024-09-20T22:03:47.390 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_heartbeat_reset_grace": "1000", 2024-09-20T22:03:47.390 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_inject_health_dummy": "false", 2024-09-20T22:03:47.390 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-09-20T22:03:47.391 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_inject_migrator_session_race": "false", 2024-09-20T22:03:47.391 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-09-20T22:03:47.391 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-09-20T22:03:47.391 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-09-20T22:03:47.391 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_join_fs": "", 2024-09-20T22:03:47.391 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_journal_format": "1", 2024-09-20T22:03:47.391 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_after_journal_logs_flushed": "false", 2024-09-20T22:03:47.391 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_create_at": "0", 2024-09-20T22:03:47.391 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_export_at": "0", 2024-09-20T22:03:47.391 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_import_at": "0", 2024-09-20T22:03:47.391 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_journal_at": "0", 2024-09-20T22:03:47.391 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_journal_expire_at": "0", 2024-09-20T22:03:47.391 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_journal_replay_at": "0", 2024-09-20T22:03:47.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_link_at": "0", 2024-09-20T22:03:47.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_mdstable_at": "0", 2024-09-20T22:03:47.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_openc_at": "0", 2024-09-20T22:03:47.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_rename_at": "0", 2024-09-20T22:03:47.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_kill_shutdown_at": "0", 2024-09-20T22:03:47.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_event_large_threshold": "524288", 2024-09-20T22:03:47.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_events_per_segment": "1024", 2024-09-20T22:03:47.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_major_segment_event_ratio": "12", 2024-09-20T22:03:47.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_max_events": "-1", 2024-09-20T22:03:47.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_max_segments": "128", 2024-09-20T22:03:47.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_pause": "false", 2024-09-20T22:03:47.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_segment_size": "0", 2024-09-20T22:03:47.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_skip_corrupt_events": "false", 2024-09-20T22:03:47.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_skip_unbounded_events": "false", 2024-09-20T22:03:47.392 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-09-20T22:03:47.393 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_trim_threshold": "128", 2024-09-20T22:03:47.393 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-09-20T22:03:47.393 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_log_warn_factor": "2.000000", 2024-09-20T22:03:47.393 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_caps_per_client": "1048576", 2024-09-20T22:03:47.393 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_completed_flushes": "100000", 2024-09-20T22:03:47.393 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_completed_requests": "100000", 2024-09-20T22:03:47.393 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_export_size": "20971520", 2024-09-20T22:03:47.393 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_file_recover": "32", 2024-09-20T22:03:47.393 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_purge_files": "64", 2024-09-20T22:03:47.393 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_purge_ops": "8192", 2024-09-20T22:03:47.393 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-09-20T22:03:47.393 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-09-20T22:03:47.393 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_max_snaps_per_dir": "100", 2024-09-20T22:03:47.393 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_metrics_update_interval": "2", 2024-09-20T22:03:47.394 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_min_caps_per_client": "100", 2024-09-20T22:03:47.394 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_min_caps_working_set": "10000", 2024-09-20T22:03:47.394 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-09-20T22:03:47.394 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_numa_node": "-1", 2024-09-20T22:03:47.394 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-09-20T22:03:47.394 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_op_complaint_time": "30.000000", 2024-09-20T22:03:47.394 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_op_history_duration": "600", 2024-09-20T22:03:47.394 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_op_history_size": "20", 2024-09-20T22:03:47.394 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_op_history_slow_op_size": "20", 2024-09-20T22:03:47.394 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_op_history_slow_op_threshold": "10", 2024-09-20T22:03:47.394 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_op_log_threshold": "5", 2024-09-20T22:03:47.394 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_ping_grace": "15", 2024-09-20T22:03:47.394 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_ping_interval": "5", 2024-09-20T22:03:47.395 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-09-20T22:03:47.395 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-09-20T22:03:47.395 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_recall_max_caps": "30000", 2024-09-20T22:03:47.395 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-09-20T22:03:47.395 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_recall_max_decay_threshold": "131072", 2024-09-20T22:03:47.395 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-09-20T22:03:47.395 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_recall_warning_threshold": "262144", 2024-09-20T22:03:47.395 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_reconnect_timeout": "45.000000", 2024-09-20T22:03:47.395 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_replay_interval": "1.000000", 2024-09-20T22:03:47.395 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-09-20T22:03:47.395 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-09-20T22:03:47.395 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_root_ino_gid": "0", 2024-09-20T22:03:47.395 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_root_ino_uid": "0", 2024-09-20T22:03:47.395 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-09-20T22:03:47.396 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_session_blocklist_on_evict": "true", 2024-09-20T22:03:47.396 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_session_blocklist_on_timeout": "true", 2024-09-20T22:03:47.396 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-09-20T22:03:47.396 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-09-20T22:03:47.396 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-09-20T22:03:47.396 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-09-20T22:03:47.396 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-09-20T22:03:47.396 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_session_metadata_threshold": "16777216", 2024-09-20T22:03:47.396 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-09-20T22:03:47.396 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_shutdown_check": "0", 2024-09-20T22:03:47.396 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_skip_ino": "0", 2024-09-20T22:03:47.396 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_sleep_rank_change": "0.000000", 2024-09-20T22:03:47.396 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_snap_max_uid": "4294967294", 2024-09-20T22:03:47.396 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_snap_min_uid": "0", 2024-09-20T22:03:47.396 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_snap_rstat": "false", 2024-09-20T22:03:47.397 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_standby_replay_damaged": "false", 2024-09-20T22:03:47.397 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_symlink_recovery": "true", 2024-09-20T22:03:47.397 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_task_status_update_interval": "2.000000", 2024-09-20T22:03:47.397 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_thrash_exports": "0", 2024-09-20T22:03:47.397 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_thrash_fragments": "0", 2024-09-20T22:03:47.397 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_tick_interval": "5.000000", 2024-09-20T22:03:47.397 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_valgrind_exit": "false", 2024-09-20T22:03:47.397 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_verify_backtrace": "1", 2024-09-20T22:03:47.397 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_verify_scatter": "false", 2024-09-20T22:03:47.397 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_wipe_ino_prealloc": "false", 2024-09-20T22:03:47.397 INFO:teuthology.orchestra.run.smithi060.stdout: "mds_wipe_sessions": "false", 2024-09-20T22:03:47.397 INFO:teuthology.orchestra.run.smithi060.stdout: "mempool_debug": "false", 2024-09-20T22:03:47.397 INFO:teuthology.orchestra.run.smithi060.stdout: "memstore_debug_omit_block_device_write": "false", 2024-09-20T22:03:47.397 INFO:teuthology.orchestra.run.smithi060.stdout: "memstore_device_bytes": "1073741824", 2024-09-20T22:03:47.397 INFO:teuthology.orchestra.run.smithi060.stdout: "memstore_page_set": "false", 2024-09-20T22:03:47.398 INFO:teuthology.orchestra.run.smithi060.stdout: "memstore_page_size": "65536", 2024-09-20T22:03:47.398 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_client_bytes": "134217728", 2024-09-20T22:03:47.398 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_client_messages": "512", 2024-09-20T22:03:47.398 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-09-20T22:03:47.398 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_connect_retry_interval": "1.000000", 2024-09-20T22:03:47.398 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-09-20T22:03:47.398 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-09-20T22:03:47.398 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_disabled_modules": "", 2024-09-20T22:03:47.398 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_enable_op_tracker": "true", 2024-09-20T22:03:47.398 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-09-20T22:03:47.398 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_max_pg_num_change": "128", 2024-09-20T22:03:47.398 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_mds_bytes": "134217728", 2024-09-20T22:03:47.398 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_mds_messages": "128", 2024-09-20T22:03:47.398 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-09-20T22:03:47.399 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_mon_bytes": "134217728", 2024-09-20T22:03:47.399 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_mon_messages": "128", 2024-09-20T22:03:47.399 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_num_op_tracker_shard": "32", 2024-09-20T22:03:47.399 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_op_complaint_time": "30.000000", 2024-09-20T22:03:47.399 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_op_history_duration": "600", 2024-09-20T22:03:47.399 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_op_history_size": "20", 2024-09-20T22:03:47.399 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_op_history_slow_op_size": "20", 2024-09-20T22:03:47.399 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_op_history_slow_op_threshold": "10.000000", 2024-09-20T22:03:47.399 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_op_log_threshold": "5", 2024-09-20T22:03:47.399 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_osd_bytes": "536870912", 2024-09-20T22:03:47.399 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_osd_messages": "8192", 2024-09-20T22:03:47.399 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_pool": "true", 2024-09-20T22:03:47.399 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_service_beacon_grace": "60.000000", 2024-09-20T22:03:47.399 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_standby_modules": "true", 2024-09-20T22:03:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_stats_period": "5", 2024-09-20T22:03:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_stats_threshold": "5", 2024-09-20T22:03:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_tick_period": "2", 2024-09-20T22:03:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-09-20T22:03:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_accept_timeout_factor": "2.000000", 2024-09-20T22:03:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_allow_pool_delete": "true", 2024-09-20T22:03:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_allow_pool_size_one": "false", 2024-09-20T22:03:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_auth_validate_all_caps": "true", 2024-09-20T22:03:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-09-20T22:03:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-09-20T22:03:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_bytes": "104857600", 2024-09-20T22:03:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_directed_command_retry": "2", 2024-09-20T22:03:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_hunt_interval": "3.000000", 2024-09-20T22:03:47.400 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-09-20T22:03:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-09-20T22:03:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-09-20T22:03:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_hunt_parallel": "3", 2024-09-20T22:03:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_log_interval": "1.000000", 2024-09-20T22:03:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-09-20T22:03:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_ping_interval": "10.000000", 2024-09-20T22:03:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_ping_timeout": "30.000000", 2024-09-20T22:03:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_client_target_rank": "-1", 2024-09-20T22:03:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_clock_drift_allowed": "1.000000", 2024-09-20T22:03:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-09-20T22:03:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-09-20T22:03:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_file_level": "debug", 2024-09-20T22:03:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_to_file": "true", 2024-09-20T22:03:47.401 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_to_graylog": "false", 2024-09-20T22:03:47.402 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-09-20T22:03:47.402 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-09-20T22:03:47.402 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_to_journald": "true", 2024-09-20T22:03:47.402 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_to_stderr": "false", 2024-09-20T22:03:47.402 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-09-20T22:03:47.402 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-09-20T22:03:47.402 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-09-20T22:03:47.402 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_compact_on_bootstrap": "false", 2024-09-20T22:03:47.402 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_compact_on_start": "false", 2024-09-20T22:03:47.402 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_compact_on_trim": "true", 2024-09-20T22:03:47.402 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_con_tracker_persist_interval": "10", 2024-09-20T22:03:47.402 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_con_tracker_score_halflife": "43200", 2024-09-20T22:03:47.402 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_config_key_max_entry_size": "65536", 2024-09-20T22:03:47.403 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_cpu_threads": "4", 2024-09-20T22:03:47.403 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_crush_min_required_version": "hammer", 2024-09-20T22:03:47.403 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_daemon_bytes": "419430400", 2024-09-20T22:03:47.403 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-09-20T22:03:47.403 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_data_avail_crit": "5", 2024-09-20T22:03:47.403 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_data_avail_warn": "5", 2024-09-20T22:03:47.403 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_data_size_warn": "16106127360", 2024-09-20T22:03:47.403 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_block_osdmap_trim": "false", 2024-09-20T22:03:47.403 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-09-20T22:03:47.403 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_dump_json": "false", 2024-09-20T22:03:47.403 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-09-20T22:03:47.403 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_dump_transactions": "false", 2024-09-20T22:03:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_extra_checks": "false", 2024-09-20T22:03:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-09-20T22:03:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-09-20T22:03:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_no_require_reef": "false", 2024-09-20T22:03:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_no_require_squid": "false", 2024-09-20T22:03:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-09-20T22:03:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_delta_reset_interval": "10.000000", 2024-09-20T22:03:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_dns_srv_name": "ceph-mon", 2024-09-20T22:03:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_down_mkfs_grace": "60", 2024-09-20T22:03:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_down_uptime_grace": "60", 2024-09-20T22:03:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_election_default_strategy": "1", 2024-09-20T22:03:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_election_timeout": "5.000000", 2024-09-20T22:03:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-09-20T22:03:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_elector_ping_divisor": "2", 2024-09-20T22:03:47.404 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_elector_ping_timeout": "2.000000", 2024-09-20T22:03:47.405 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_enable_op_tracker": "true", 2024-09-20T22:03:47.405 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_fake_pool_delete": "false", 2024-09-20T22:03:47.405 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_force_quorum_join": "false", 2024-09-20T22:03:47.405 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_fsmap_prune_threshold": "300", 2024-09-20T22:03:47.405 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_globalid_prealloc": "10000", 2024-09-20T22:03:47.405 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_health_detail_to_clog": "true", 2024-09-20T22:03:47.405 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_health_log_update_period": "5", 2024-09-20T22:03:47.405 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_health_max_detail": "50", 2024-09-20T22:03:47.405 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_health_to_clog": "true", 2024-09-20T22:03:47.405 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_health_to_clog_interval": "600", 2024-09-20T22:03:47.405 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-09-20T22:03:47.405 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_host": "[v2:172.21.15.60:3300/0,v1:172.21.15.60:6789/0]", 2024-09-20T22:03:47.405 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_host_override": "", 2024-09-20T22:03:47.406 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_initial_members": "", 2024-09-20T22:03:47.406 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-09-20T22:03:47.406 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-09-20T22:03:47.406 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-09-20T22:03:47.406 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-09-20T22:03:47.406 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_keyvaluedb": "rocksdb", 2024-09-20T22:03:47.406 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_lease": "5.000000", 2024-09-20T22:03:47.406 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-09-20T22:03:47.406 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-09-20T22:03:47.406 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_log_full_interval": "50", 2024-09-20T22:03:47.406 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_log_max": "10000", 2024-09-20T22:03:47.406 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_log_max_summary": "50", 2024-09-20T22:03:47.407 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_max_log_entries_per_event": "4096", 2024-09-20T22:03:47.407 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_max_log_epochs": "500", 2024-09-20T22:03:47.407 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_max_mdsmap_epochs": "500", 2024-09-20T22:03:47.407 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_max_mgrmap_epochs": "500", 2024-09-20T22:03:47.407 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_max_osd": "10000", 2024-09-20T22:03:47.407 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_max_pg_per_osd": "10000", 2024-09-20T22:03:47.407 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_max_pool_pg_num": "65536", 2024-09-20T22:03:47.407 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-09-20T22:03:47.407 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-09-20T22:03:47.407 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_mds_force_trim_to": "0", 2024-09-20T22:03:47.407 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_mds_skip_sanity": "false", 2024-09-20T22:03:47.407 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_memory_autotune": "true", 2024-09-20T22:03:47.407 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_memory_target": "2147483648", 2024-09-20T22:03:47.407 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_mgr_beacon_grace": "30", 2024-09-20T22:03:47.408 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-09-20T22:03:47.408 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_mgr_digest_period": "5", 2024-09-20T22:03:47.408 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_mgr_inactive_grace": "60", 2024-09-20T22:03:47.408 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_mgr_mkfs_grace": "240", 2024-09-20T22:03:47.408 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-09-20T22:03:47.408 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_min_osdmap_epochs": "50", 2024-09-20T22:03:47.408 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_op_complaint_time": "30", 2024-09-20T22:03:47.408 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_op_history_duration": "600", 2024-09-20T22:03:47.408 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_op_history_size": "20", 2024-09-20T22:03:47.408 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_op_history_slow_op_size": "20", 2024-09-20T22:03:47.408 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_op_history_slow_op_threshold": "10", 2024-09-20T22:03:47.408 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_op_log_threshold": "5", 2024-09-20T22:03:47.408 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-09-20T22:03:47.408 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-09-20T22:03:47.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-09-20T22:03:47.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_auto_mark_in": "false", 2024-09-20T22:03:47.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_auto_mark_new_in": "true", 2024-09-20T22:03:47.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-09-20T22:03:47.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-09-20T22:03:47.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_cache_size": "500", 2024-09-20T22:03:47.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_cache_size_min": "134217728", 2024-09-20T22:03:47.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_crush_smoke_test": "true", 2024-09-20T22:03:47.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_destroyed_out_interval": "600", 2024-09-20T22:03:47.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_down_out_interval": "600", 2024-09-20T22:03:47.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-09-20T22:03:47.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-09-20T22:03:47.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_force_trim_to": "0", 2024-09-20T22:03:47.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_full_ratio": "0.950000", 2024-09-20T22:03:47.409 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-09-20T22:03:47.410 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_laggy_halflife": "3600", 2024-09-20T22:03:47.410 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_laggy_max_interval": "300", 2024-09-20T22:03:47.410 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_laggy_weight": "0.300000", 2024-09-20T22:03:47.410 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-09-20T22:03:47.410 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_max_creating_pgs": "1024", 2024-09-20T22:03:47.410 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_max_initial_pgs": "1024", 2024-09-20T22:03:47.410 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_min_down_reporters": "2", 2024-09-20T22:03:47.410 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-09-20T22:03:47.410 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-09-20T22:03:47.410 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-09-20T22:03:47.410 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_prime_pg_temp": "true", 2024-09-20T22:03:47.410 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-09-20T22:03:47.410 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-09-20T22:03:47.411 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_report_timeout": "900", 2024-09-20T22:03:47.411 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-09-20T22:03:47.411 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-09-20T22:03:47.411 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_warn_num_repaired": "10", 2024-09-20T22:03:47.411 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osd_warn_op_age": "32.000000", 2024-09-20T22:03:47.411 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-09-20T22:03:47.411 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osdmap_full_prune_interval": "2", 2024-09-20T22:03:47.411 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osdmap_full_prune_min": "15", 2024-09-20T22:03:47.411 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-09-20T22:03:47.411 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-09-20T22:03:47.411 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_pg_stuck_threshold": "60", 2024-09-20T22:03:47.411 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-09-20T22:03:47.411 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_pg_warn_min_objects": "10000", 2024-09-20T22:03:47.411 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_pg_warn_min_per_osd": "0", 2024-09-20T22:03:47.412 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-09-20T22:03:47.412 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_pool_quota_crit_threshold": "0", 2024-09-20T22:03:47.412 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_pool_quota_warn_threshold": "0", 2024-09-20T22:03:47.412 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_probe_timeout": "2.000000", 2024-09-20T22:03:47.412 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_reweight_max_change": "0.050000", 2024-09-20T22:03:47.412 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_reweight_max_osds": "4", 2024-09-20T22:03:47.412 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-09-20T22:03:47.412 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-09-20T22:03:47.412 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-09-20T22:03:47.412 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-09-20T22:03:47.412 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-09-20T22:03:47.412 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_scrub_interval": "86400", 2024-09-20T22:03:47.412 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_scrub_max_keys": "100", 2024-09-20T22:03:47.412 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_scrub_timeout": "300", 2024-09-20T22:03:47.413 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_session_timeout": "300", 2024-09-20T22:03:47.413 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_smart_report_timeout": "5", 2024-09-20T22:03:47.413 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_stat_smooth_intervals": "6", 2024-09-20T22:03:47.413 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-09-20T22:03:47.413 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_stretch_pool_min_size": "2", 2024-09-20T22:03:47.413 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_stretch_pool_size": "4", 2024-09-20T22:03:47.413 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-09-20T22:03:47.413 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_subscribe_interval": "86400.000000", 2024-09-20T22:03:47.413 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_sync_debug": "false", 2024-09-20T22:03:47.413 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_sync_max_payload_keys": "2000", 2024-09-20T22:03:47.413 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_sync_max_payload_size": "1048576", 2024-09-20T22:03:47.413 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_sync_provider_kill_at": "0", 2024-09-20T22:03:47.413 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_sync_requester_kill_at": "0", 2024-09-20T22:03:47.413 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_sync_timeout": "60.000000", 2024-09-20T22:03:47.414 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_target_pg_per_osd": "100", 2024-09-20T22:03:47.414 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_tick_interval": "5", 2024-09-20T22:03:47.414 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_timecheck_interval": "300.000000", 2024-09-20T22:03:47.414 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-09-20T22:03:47.414 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_use_min_delay_socket": "false", 2024-09-20T22:03:47.414 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_older_version_delay": "604800", 2024-09-20T22:03:47.414 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-09-20T22:03:47.414 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-09-20T22:03:47.414 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-09-20T22:03:47.414 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_filestore_osds": "true", 2024-09-20T22:03:47.414 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-09-20T22:03:47.414 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-09-20T22:03:47.414 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-09-20T22:03:47.414 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_misplaced": "false", 2024-09-20T22:03:47.415 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-09-20T22:03:47.415 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_older_version": "true", 2024-09-20T22:03:47.415 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-09-20T22:03:47.415 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_pool_no_app": "true", 2024-09-20T22:03:47.415 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_pool_no_redundancy": "true", 2024-09-20T22:03:47.415 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-09-20T22:03:47.415 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-09-20T22:03:47.415 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-09-20T22:03:47.415 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_on_too_few_osds": "false", 2024-09-20T22:03:47.415 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-09-20T22:03:47.415 INFO:teuthology.orchestra.run.smithi060.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-09-20T22:03:47.415 INFO:teuthology.orchestra.run.smithi060.stdout: "monmap": "", 2024-09-20T22:03:47.415 INFO:teuthology.orchestra.run.smithi060.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-20T22:03:47.415 INFO:teuthology.orchestra.run.smithi060.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-09-20T22:03:47.415 INFO:teuthology.orchestra.run.smithi060.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-09-20T22:03:47.416 INFO:teuthology.orchestra.run.smithi060.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-09-20T22:03:47.416 INFO:teuthology.orchestra.run.smithi060.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-09-20T22:03:47.416 INFO:teuthology.orchestra.run.smithi060.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-09-20T22:03:47.416 INFO:teuthology.orchestra.run.smithi060.stdout: "motr_tracing_enabled": "false", 2024-09-20T22:03:47.416 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_op_threads": "3", 2024-09-20T22:03:47.416 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_buffer_size": "131072", 2024-09-20T22:03:47.416 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_cm": "false", 2024-09-20T22:03:47.416 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_device_name": "", 2024-09-20T22:03:47.416 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_dscp": "96", 2024-09-20T22:03:47.416 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-09-20T22:03:47.416 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_gid_idx": "0", 2024-09-20T22:03:47.416 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_local_gid": "", 2024-09-20T22:03:47.416 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_polling_us": "1000", 2024-09-20T22:03:47.416 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_port_num": "1", 2024-09-20T22:03:47.417 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-09-20T22:03:47.417 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-09-20T22:03:47.417 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_roce_ver": "1", 2024-09-20T22:03:47.417 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_send_buffers": "1024", 2024-09-20T22:03:47.417 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_sl": "3", 2024-09-20T22:03:47.417 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_support_srq": "true", 2024-09-20T22:03:47.417 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_rdma_type": "ib", 2024-09-20T22:03:47.417 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_async_reap_threshold": "5", 2024-09-20T22:03:47.417 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_before_connect": "false", 2024-09-20T22:03:47.417 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_ipv4": "true", 2024-09-20T22:03:47.417 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_ipv6": "false", 2024-09-20T22:03:47.417 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_msgr1": "true", 2024-09-20T22:03:47.417 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_msgr2": "true", 2024-09-20T22:03:47.417 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_port_max": "7568", 2024-09-20T22:03:47.417 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_port_min": "6800", 2024-09-20T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_prefer_ipv4": "false", 2024-09-20T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_retry_count": "3", 2024-09-20T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_bind_retry_delay": "5", 2024-09-20T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_blackhole_client": "false", 2024-09-20T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_blackhole_mds": "false", 2024-09-20T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_blackhole_mgr": "false", 2024-09-20T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_blackhole_mon": "false", 2024-09-20T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_blackhole_osd": "false", 2024-09-20T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_client_mode": "crc secure", 2024-09-20T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_cluster_mode": "crc secure", 2024-09-20T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_cluster_type": "", 2024-09-20T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_compress_secure": "false", 2024-09-20T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_connection_idle_timeout": "900", 2024-09-20T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_connection_ready_timeout": "10", 2024-09-20T22:03:47.418 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_crc_data": "true", 2024-09-20T22:03:47.419 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_crc_header": "true", 2024-09-20T22:03:47.419 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_die_on_bad_msg": "false", 2024-09-20T22:03:47.419 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_die_on_bug": "true", 2024-09-20T22:03:47.419 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_die_on_old_message": "true", 2024-09-20T22:03:47.419 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_die_on_skipped_message": "false", 2024-09-20T22:03:47.419 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_die_on_unhandled_msg": "false", 2024-09-20T22:03:47.419 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-09-20T22:03:47.419 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_coremask": "0xF", 2024-09-20T22:03:47.419 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-09-20T22:03:47.419 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_devs_allowlist": "", 2024-09-20T22:03:47.419 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_enable_tso": "true", 2024-09-20T22:03:47.419 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-09-20T22:03:47.419 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-09-20T22:03:47.419 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_hugepages": "", 2024-09-20T22:03:47.419 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_hw_flow_control": "true", 2024-09-20T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-09-20T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_lro": "true", 2024-09-20T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_memory_channel": "4", 2024-09-20T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-09-20T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_pmd": "", 2024-09-20T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_port_id": "0", 2024-09-20T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-09-20T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dump_corrupt_message_level": "1", 2024-09-20T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_dump_on_send": "false", 2024-09-20T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_initial_backoff": "0.200000", 2024-09-20T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_inject_delay_max": "1.000000", 2024-09-20T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_inject_delay_probability": "0.000000", 2024-09-20T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_inject_delay_type": "", 2024-09-20T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_inject_internal_delays": "0.000000", 2024-09-20T22:03:47.420 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_inject_network_congestion": "0", 2024-09-20T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_inject_socket_failures": "0", 2024-09-20T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_learn_addr_from_peer": "true", 2024-09-20T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_max_accept_failures": "4", 2024-09-20T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_max_backoff": "15.000000", 2024-09-20T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_mon_client_mode": "secure crc", 2024-09-20T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_mon_cluster_mode": "secure crc", 2024-09-20T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_mon_service_mode": "secure crc", 2024-09-20T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_osd_compress_min_size": "1024", 2024-09-20T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_osd_compress_mode": "none", 2024-09-20T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_osd_compression_algorithm": "snappy", 2024-09-20T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-09-20T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_pq_min_cost": "65536", 2024-09-20T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_public_type": "", 2024-09-20T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_service_mode": "crc secure", 2024-09-20T22:03:47.421 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_tcp_listen_backlog": "512", 2024-09-20T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_tcp_nodelay": "true", 2024-09-20T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-09-20T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_tcp_rcvbuf": "0", 2024-09-20T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stdout: "ms_type": "async+posix", 2024-09-20T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stdout: "no_config_file": "false", 2024-09-20T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter_completion_locks_per_session": "32", 2024-09-20T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter_debug_inject_relock_delay": "false", 2024-09-20T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter_inflight_op_bytes": "104857600", 2024-09-20T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter_inflight_ops": "1024", 2024-09-20T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter_inject_no_watch_ping": "false", 2024-09-20T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-09-20T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter_tick_interval": "5.000000", 2024-09-20T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stdout: "objecter_timeout": "10.000000", 2024-09-20T22:03:47.422 INFO:teuthology.orchestra.run.smithi060.stdout: "objectstore_blackhole": "false", 2024-09-20T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-09-20T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stdout: "openssl_engine_opts": "", 2024-09-20T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_agent_delay_time": "5.000000", 2024-09-20T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_agent_hist_halflife": "1000", 2024-09-20T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_agent_max_low_ops": "2", 2024-09-20T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_agent_max_ops": "4", 2024-09-20T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-09-20T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_agent_quantize_effort": "0.100000", 2024-09-20T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_agent_slop": "0.020000", 2024-09-20T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-09-20T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_allow_recovery_below_min_size": "true", 2024-09-20T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_asio_thread_count": "2", 2024-09-20T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_async_recovery_min_cost": "100", 2024-09-20T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_auto_mark_unfound_lost": "false", 2024-09-20T22:03:47.423 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_backfill_retry_interval": "30.000000", 2024-09-20T22:03:47.424 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_backfill_scan_max": "512", 2024-09-20T22:03:47.424 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_backfill_scan_min": "64", 2024-09-20T22:03:47.424 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_backoff_on_degraded": "false", 2024-09-20T22:03:47.424 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_backoff_on_peering": "false", 2024-09-20T22:03:47.424 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_backoff_on_unfound": "true", 2024-09-20T22:03:47.424 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_beacon_report_interval": "300", 2024-09-20T22:03:47.424 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bench_duration": "30", 2024-09-20T22:03:47.424 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-09-20T22:03:47.424 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bench_max_block_size": "67108864", 2024-09-20T22:03:47.424 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_bench_small_size_max_iops": "100", 2024-09-20T22:03:47.424 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_blkin_trace_all": "false", 2024-09-20T22:03:47.424 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_blocked_scrub_grace_period": "120", 2024-09-20T22:03:47.424 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-09-20T22:03:47.424 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-09-20T22:03:47.424 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-09-20T22:03:47.425 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_check_for_log_corruption": "false", 2024-09-20T22:03:47.425 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-09-20T22:03:47.425 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-20T22:03:47.425 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-09-20T22:03:47.425 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-09-20T22:03:47.425 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_class_update_on_start": "true", 2024-09-20T22:03:47.425 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_client_message_cap": "256", 2024-09-20T22:03:47.425 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_client_message_size_cap": "524288000", 2024-09-20T22:03:47.425 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_client_op_priority": "63", 2024-09-20T22:03:47.425 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_client_watch_timeout": "30", 2024-09-20T22:03:47.425 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_command_max_records": "256", 2024-09-20T22:03:47.425 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_compact_on_start": "false", 2024-09-20T22:03:47.425 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-09-20T22:03:47.425 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_crush_chooseleaf_type": "1", 2024-09-20T22:03:47.425 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_crush_initial_weight": "-1.000000", 2024-09-20T22:03:47.426 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_crush_update_on_start": "true", 2024-09-20T22:03:47.426 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_crush_update_weight_set": "true", 2024-09-20T22:03:47.426 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-09-20T22:03:47.426 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-09-20T22:03:47.426 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-09-20T22:03:47.426 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_drop_ping_duration": "0", 2024-09-20T22:03:47.426 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-09-20T22:03:47.426 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_feed_pullee": "-1", 2024-09-20T22:03:47.426 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-09-20T22:03:47.426 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-09-20T22:03:47.426 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-09-20T22:03:47.426 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_misdirected_ops": "false", 2024-09-20T22:03:47.426 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_no_acting_change": "false", 2024-09-20T22:03:47.426 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_no_purge_strays": "false", 2024-09-20T22:03:47.427 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_op_order": "false", 2024-09-20T22:03:47.427 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_pg_log_writeout": "false", 2024-09-20T22:03:47.427 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_pretend_recovery_active": "false", 2024-09-20T22:03:47.427 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-09-20T22:03:47.427 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-09-20T22:03:47.427 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_shutdown": "false", 2024-09-20T22:03:47.427 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-09-20T22:03:47.427 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-09-20T22:03:47.427 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_trim_objects": "false", 2024-09-20T22:03:47.427 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_verify_cached_snaps": "false", 2024-09-20T22:03:47.427 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_verify_missing_on_start": "false", 2024-09-20T22:03:47.427 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_verify_snaps": "false", 2024-09-20T22:03:47.427 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-09-20T22:03:47.427 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-09-20T22:03:47.428 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_deep_scrub_keys": "1024", 2024-09-20T22:03:47.428 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-09-20T22:03:47.428 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-09-20T22:03:47.428 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-09-20T22:03:47.428 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_deep_scrub_stride": "524288", 2024-09-20T22:03:47.428 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-09-20T22:03:47.428 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_default_data_pool_replay_window": "45", 2024-09-20T22:03:47.428 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_default_notify_timeout": "30", 2024-09-20T22:03:47.428 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_delete_sleep": "0.000000", 2024-09-20T22:03:47.428 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-09-20T22:03:47.428 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-09-20T22:03:47.428 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-09-20T22:03:47.428 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_discard_disconnected_ops": "true", 2024-09-20T22:03:47.428 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_enable_op_tracker": "true", 2024-09-20T22:03:47.428 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-09-20T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_extblkdev_plugins": "vdo", 2024-09-20T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-09-20T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-09-20T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_fast_info": "true", 2024-09-20T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_fast_shutdown": "true", 2024-09-20T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-09-20T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_fast_shutdown_timeout": "15", 2024-09-20T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-09-20T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-09-20T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-09-20T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_function_tracing": "false", 2024-09-20T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_heartbeat_grace": "20", 2024-09-20T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_heartbeat_interval": "6", 2024-09-20T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-09-20T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_heartbeat_min_peers": "10", 2024-09-20T22:03:47.429 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_heartbeat_min_size": "2000", 2024-09-20T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_heartbeat_stale": "600", 2024-09-20T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-09-20T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_hit_set_max_size": "100000", 2024-09-20T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_hit_set_min_size": "1000", 2024-09-20T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-09-20T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-09-20T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-09-20T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-09-20T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-09-20T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_journal_flush_on_shutdown": "true", 2024-09-20T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_journal_size": "5120", 2024-09-20T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_kill_backfill_at": "0", 2024-09-20T22:03:47.430 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_loop_before_reset_tphandle": "64", 2024-09-20T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_map_cache_size": "50", 2024-09-20T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_map_dedup": "true", 2024-09-20T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_map_message_max": "40", 2024-09-20T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_map_message_max_bytes": "10485760", 2024-09-20T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_map_share_max_epochs": "40", 2024-09-20T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_attr_name_len": "100", 2024-09-20T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_attr_size": "0", 2024-09-20T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_backfills": "1", 2024-09-20T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_markdown_count": "5", 2024-09-20T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_markdown_period": "600", 2024-09-20T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_object_name_len": "2048", 2024-09-20T22:03:47.431 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_object_namespace_len": "256", 2024-09-20T22:03:47.432 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_object_size": "134217728", 2024-09-20T22:03:47.432 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-09-20T22:03:47.432 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_omap_entries_per_request": "1024", 2024-09-20T22:03:47.432 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_pg_blocked_by": "16", 2024-09-20T22:03:47.432 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_pg_log_entries": "10000", 2024-09-20T22:03:47.432 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-09-20T22:03:47.432 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_pgls": "1024", 2024-09-20T22:03:47.432 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_push_cost": "8388608", 2024-09-20T22:03:47.432 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_push_objects": "10", 2024-09-20T22:03:47.432 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_scrubs": "3", 2024-09-20T22:03:47.432 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-09-20T22:03:47.432 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_trimming_pgs": "2", 2024-09-20T22:03:47.432 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_write_op_reply_len": "64", 2024-09-20T22:03:47.432 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_max_write_size": "90", 2024-09-20T22:03:47.433 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-09-20T22:03:47.433 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-09-20T22:03:47.433 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-09-20T22:03:47.433 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-09-20T22:03:47.433 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-09-20T22:03:47.433 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-09-20T22:03:47.433 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-09-20T22:03:47.433 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_override_recovery_settings": "false", 2024-09-20T22:03:47.433 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_profile": "balanced", 2024-09-20T22:03:47.433 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-09-20T22:03:47.433 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-09-20T22:03:47.433 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-09-20T22:03:47.433 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-09-20T22:03:47.434 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-09-20T22:03:47.434 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-09-20T22:03:47.434 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-09-20T22:03:47.434 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-09-20T22:03:47.434 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-09-20T22:03:47.434 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-09-20T22:03:47.434 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mclock_skip_benchmark": "false", 2024-09-20T22:03:47.434 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_memory_base": "805306368", 2024-09-20T22:03:47.434 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_memory_cache_min": "134217728", 2024-09-20T22:03:47.434 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-09-20T22:03:47.434 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-09-20T22:03:47.434 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_memory_target": "4294967296", 2024-09-20T22:03:47.434 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_memory_target_autotune": "false", 2024-09-20T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-09-20T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_min_pg_log_entries": "250", 2024-09-20T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_min_recovery_priority": "0", 2024-09-20T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mon_heartbeat_interval": "30", 2024-09-20T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-09-20T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mon_report_interval": "5", 2024-09-20T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mon_report_max_in_flight": "2", 2024-09-20T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-09-20T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_num_cache_shards": "32", 2024-09-20T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_num_op_tracker_shard": "32", 2024-09-20T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_numa_auto_affinity": "true", 2024-09-20T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_numa_node": "-1", 2024-09-20T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_numa_prefer_iface": "true", 2024-09-20T22:03:47.435 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-09-20T22:03:47.436 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_objecter_finishers": "1", 2024-09-20T22:03:47.436 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_objectstore": "bluestore", 2024-09-20T22:03:47.436 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_objectstore_fuse": "false", 2024-09-20T22:03:47.436 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_objectstore_tracing": "false", 2024-09-20T22:03:47.436 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_complaint_time": "30.000000", 2024-09-20T22:03:47.436 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_history_duration": "600", 2024-09-20T22:03:47.436 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_history_size": "20", 2024-09-20T22:03:47.436 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_history_slow_op_size": "20", 2024-09-20T22:03:47.436 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-09-20T22:03:47.436 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_log_threshold": "5", 2024-09-20T22:03:47.436 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_num_shards": "0", 2024-09-20T22:03:47.436 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_num_shards_hdd": "5", 2024-09-20T22:03:47.436 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_num_shards_ssd": "8", 2024-09-20T22:03:47.437 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_num_threads_per_shard": "0", 2024-09-20T22:03:47.437 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-09-20T22:03:47.437 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-09-20T22:03:47.437 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-09-20T22:03:47.437 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_pq_min_cost": "65536", 2024-09-20T22:03:47.437 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_queue": "mclock_scheduler", 2024-09-20T22:03:47.437 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_queue_cut_off": "high", 2024-09-20T22:03:47.437 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_thread_suicide_timeout": "150", 2024-09-20T22:03:47.437 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_op_thread_timeout": "15", 2024-09-20T22:03:47.437 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_open_classes_on_start": "true", 2024-09-20T22:03:47.437 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_os_flags": "0", 2024-09-20T22:03:47.437 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_peering_op_priority": "255", 2024-09-20T22:03:47.437 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_delete_cost": "1048576", 2024-09-20T22:03:47.438 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_delete_priority": "5", 2024-09-20T22:03:47.438 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-09-20T22:03:47.438 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-09-20T22:03:47.438 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_log_dups_tracked": "3000", 2024-09-20T22:03:47.438 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_log_trim_max": "10000", 2024-09-20T22:03:47.438 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_log_trim_min": "100", 2024-09-20T22:03:47.438 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-09-20T22:03:47.438 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_object_context_cache_count": "64", 2024-09-20T22:03:47.438 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-09-20T22:03:47.438 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-09-20T22:03:47.438 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-09-20T22:03:47.438 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-09-20T22:03:47.438 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-09-20T22:03:47.438 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-09-20T22:03:47.439 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-09-20T22:03:47.439 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-09-20T22:03:47.440 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_crimson": "false", 2024-09-20T22:03:47.440 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_crush_rule": "-1", 2024-09-20T22:03:47.440 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_ec_fast_read": "false", 2024-09-20T22:03:47.440 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-09-20T22:03:47.440 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_flag_bulk": "false", 2024-09-20T22:03:47.440 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-09-20T22:03:47.440 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_flag_nodelete": "false", 2024-09-20T22:03:47.440 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-09-20T22:03:47.440 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-09-20T22:03:47.440 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_flags": "0", 2024-09-20T22:03:47.440 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-09-20T22:03:47.440 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_min_size": "0", 2024-09-20T22:03:47.440 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-09-20T22:03:47.440 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_pg_num": "32", 2024-09-20T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_pgp_num": "0", 2024-09-20T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-09-20T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_read_ratio": "70", 2024-09-20T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_size": "3", 2024-09-20T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_default_type": "replicated", 2024-09-20T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-09-20T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_pool_use_gmt_hitset": "true", 2024-09-20T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_push_per_object_cost": "1000", 2024-09-20T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_read_ec_check_for_errors": "false", 2024-09-20T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recover_clone_overlap": "true", 2024-09-20T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recover_clone_overlap_limit": "10", 2024-09-20T22:03:47.441 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_cost": "20971520", 2024-09-20T22:03:47.442 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_delay_start": "0.000000", 2024-09-20T22:03:47.442 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_max_active": "0", 2024-09-20T22:03:47.442 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_max_active_hdd": "3", 2024-09-20T22:03:47.442 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_max_active_ssd": "10", 2024-09-20T22:03:47.442 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_max_chunk": "8388608", 2024-09-20T22:03:47.442 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-09-20T22:03:47.442 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_max_single_start": "1", 2024-09-20T22:03:47.442 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_op_priority": "3", 2024-09-20T22:03:47.442 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_op_warn_multiple": "16", 2024-09-20T22:03:47.442 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_priority": "5", 2024-09-20T22:03:47.442 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_retry_interval": "30.000000", 2024-09-20T22:03:47.442 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_sleep": "0.000000", 2024-09-20T22:03:47.442 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-09-20T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-09-20T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-09-20T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_repair_during_recovery": "false", 2024-09-20T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_requested_scrub_priority": "120", 2024-09-20T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-09-20T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_rollback_to_cluster_snap": "", 2024-09-20T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_auto_repair": "false", 2024-09-20T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-09-20T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-09-20T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_begin_hour": "0", 2024-09-20T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_begin_week_day": "0", 2024-09-20T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_chunk_max": "25", 2024-09-20T22:03:47.443 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_chunk_min": "5", 2024-09-20T22:03:47.444 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_cost": "52428800", 2024-09-20T22:03:47.444 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-09-20T22:03:47.444 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_during_recovery": "false", 2024-09-20T22:03:47.444 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_end_hour": "0", 2024-09-20T22:03:47.444 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_end_week_day": "0", 2024-09-20T22:03:47.444 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_event_cost": "4096", 2024-09-20T22:03:47.444 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-09-20T22:03:47.444 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-09-20T22:03:47.444 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_invalid_stats": "true", 2024-09-20T22:03:47.444 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_load_threshold": "0.500000", 2024-09-20T22:03:47.444 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_max_interval": "604800.000000", 2024-09-20T22:03:47.444 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_max_preemptions": "5", 2024-09-20T22:03:47.444 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_min_interval": "86400.000000", 2024-09-20T22:03:47.444 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_priority": "5", 2024-09-20T22:03:47.445 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_scrub_sleep": "0.000000", 2024-09-20T22:03:47.445 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-09-20T22:03:47.445 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-09-20T22:03:47.445 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_shutdown_pgref_assert": "false", 2024-09-20T22:03:47.445 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_skip_data_digest": "false", 2024-09-20T22:03:47.445 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_smart_report_timeout": "5", 2024-09-20T22:03:47.445 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_snap_trim_cost": "1048576", 2024-09-20T22:03:47.445 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_snap_trim_priority": "5", 2024-09-20T22:03:47.445 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_snap_trim_sleep": "0.000000", 2024-09-20T22:03:47.445 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-09-20T22:03:47.445 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-09-20T22:03:47.445 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-09-20T22:03:47.445 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-09-20T22:03:47.445 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_stats_update_period_scrubbing": "15", 2024-09-20T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-09-20T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_target_transaction_size": "30", 2024-09-20T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-09-20T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-09-20T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-09-20T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-09-20T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-09-20T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-09-20T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-09-20T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_default_cache_mode": "writeback", 2024-09-20T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-09-20T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-09-20T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_tracing": "false", 2024-09-20T22:03:47.446 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_use_stale_snap": "false", 2024-09-20T22:03:47.447 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-09-20T22:03:47.447 INFO:teuthology.orchestra.run.smithi060.stdout: "osdc_blkin_trace_all": "false", 2024-09-20T22:03:47.447 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_kill_at": "0", 2024-09-20T22:03:47.447 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_max_join_drift": "10", 2024-09-20T22:03:47.447 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_min": "500", 2024-09-20T22:03:47.447 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_min_wait": "0.050000", 2024-09-20T22:03:47.447 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_propose_interval": "1.000000", 2024-09-20T22:03:47.447 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_service_trim_max": "500", 2024-09-20T22:03:47.447 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_service_trim_max_multiplier": "20", 2024-09-20T22:03:47.447 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_service_trim_min": "10", 2024-09-20T22:03:47.447 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_stash_full_interval": "25", 2024-09-20T22:03:47.447 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_trim_max": "500", 2024-09-20T22:03:47.447 INFO:teuthology.orchestra.run.smithi060.stdout: "paxos_trim_min": "250", 2024-09-20T22:03:47.448 INFO:teuthology.orchestra.run.smithi060.stdout: "perf": "true", 2024-09-20T22:03:47.448 INFO:teuthology.orchestra.run.smithi060.stdout: "pid_file": "", 2024-09-20T22:03:47.448 INFO:teuthology.orchestra.run.smithi060.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-09-20T22:03:47.448 INFO:teuthology.orchestra.run.smithi060.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-09-20T22:03:47.448 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "-", 2024-09-20T22:03:47.448 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrv": "", 2024-09-20T22:03:47.448 INFO:teuthology.orchestra.run.smithi060.stdout: "public_bind_addr": "-", 2024-09-20T22:03:47.448 INFO:teuthology.orchestra.run.smithi060.stdout: "public_network": "", 2024-09-20T22:03:47.448 INFO:teuthology.orchestra.run.smithi060.stdout: "public_network_interface": "", 2024-09-20T22:03:47.448 INFO:teuthology.orchestra.run.smithi060.stdout: "qat_compressor_busy_polling": "false", 2024-09-20T22:03:47.448 INFO:teuthology.orchestra.run.smithi060.stdout: "qat_compressor_enabled": "false", 2024-09-20T22:03:47.448 INFO:teuthology.orchestra.run.smithi060.stdout: "qat_compressor_session_max_number": "256", 2024-09-20T22:03:47.448 INFO:teuthology.orchestra.run.smithi060.stdout: "rados_mon_op_timeout": "0", 2024-09-20T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stdout: "rados_osd_op_timeout": "0", 2024-09-20T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stdout: "rados_tracing": "false", 2024-09-20T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_atime_update_interval": "60", 2024-09-20T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-09-20T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_balance_parent_reads": "false", 2024-09-20T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_balance_snap_reads": "false", 2024-09-20T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_blkin_trace_all": "false", 2024-09-20T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_blocklist_expire_seconds": "0", 2024-09-20T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_blocklist_on_break_lock": "true", 2024-09-20T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_cache": "true", 2024-09-20T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_cache_block_writes_upfront": "false", 2024-09-20T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_cache_max_dirty": "25165824", 2024-09-20T22:03:47.449 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-09-20T22:03:47.450 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_cache_max_dirty_object": "0", 2024-09-20T22:03:47.450 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_cache_policy": "writearound", 2024-09-20T22:03:47.450 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_cache_size": "33554432", 2024-09-20T22:03:47.450 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_cache_target_dirty": "16777216", 2024-09-20T22:03:47.450 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-09-20T22:03:47.450 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_clone_copy_on_read": "false", 2024-09-20T22:03:47.450 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_compression_hint": "none", 2024-09-20T22:03:47.450 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_concurrent_management_ops": "10", 2024-09-20T22:03:47.450 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-09-20T22:03:47.450 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_clone_format": "auto", 2024-09-20T22:03:47.450 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_data_pool": "", 2024-09-20T22:03:47.450 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_features": "61", 2024-09-20T22:03:47.450 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_format": "2", 2024-09-20T22:03:47.451 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_map_options": "", 2024-09-20T22:03:47.451 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_order": "22", 2024-09-20T22:03:47.451 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_pool": "rbd", 2024-09-20T22:03:47.451 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-09-20T22:03:47.451 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_stripe_count": "0", 2024-09-20T22:03:47.451 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_default_stripe_unit": "0", 2024-09-20T22:03:47.451 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_disable_zero_copy_writes": "true", 2024-09-20T22:03:47.451 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_discard_granularity_bytes": "65536", 2024-09-20T22:03:47.451 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-09-20T22:03:47.451 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_enable_alloc_hint": "true", 2024-09-20T22:03:47.451 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-09-20T22:03:47.451 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_io_scheduler": "simple", 2024-09-20T22:03:47.451 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-09-20T22:03:47.452 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_commit_age": "5.000000", 2024-09-20T22:03:47.452 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-09-20T22:03:47.452 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-09-20T22:03:47.452 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-09-20T22:03:47.452 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-09-20T22:03:47.452 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_object_flush_interval": "0", 2024-09-20T22:03:47.452 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-09-20T22:03:47.452 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-09-20T22:03:47.452 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_order": "24", 2024-09-20T22:03:47.452 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_pool": "", 2024-09-20T22:03:47.452 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_journal_splay_width": "4", 2024-09-20T22:03:47.452 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_localize_parent_reads": "false", 2024-09-20T22:03:47.452 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_localize_snap_reads": "false", 2024-09-20T22:03:47.452 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-09-20T22:03:47.453 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-09-20T22:03:47.453 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-09-20T22:03:47.453 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-09-20T22:03:47.453 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-09-20T22:03:47.453 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-09-20T22:03:47.453 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_image_policy_type": "simple", 2024-09-20T22:03:47.453 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-09-20T22:03:47.453 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-09-20T22:03:47.453 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-09-20T22:03:47.453 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-09-20T22:03:47.453 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-09-20T22:03:47.453 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-09-20T22:03:47.453 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-09-20T22:03:47.453 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_memory_autotune": "true", 2024-09-20T22:03:47.453 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_memory_base": "805306368", 2024-09-20T22:03:47.454 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-09-20T22:03:47.454 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-09-20T22:03:47.454 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-09-20T22:03:47.454 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-09-20T22:03:47.454 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_memory_target": "4294967296", 2024-09-20T22:03:47.454 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-09-20T22:03:47.454 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-09-20T22:03:47.454 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-09-20T22:03:47.454 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirroring_delete_delay": "0", 2024-09-20T22:03:47.454 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-09-20T22:03:47.454 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirroring_replay_delay": "0", 2024-09-20T22:03:47.454 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-09-20T22:03:47.454 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-09-20T22:03:47.454 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_move_to_trash_on_remove": "false", 2024-09-20T22:03:47.455 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-09-20T22:03:47.455 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_mtime_update_interval": "60", 2024-09-20T22:03:47.455 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_non_blocking_aio": "true", 2024-09-20T22:03:47.455 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_op_thread_timeout": "60", 2024-09-20T22:03:47.455 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_op_threads": "1", 2024-09-20T22:03:47.455 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_parent_cache_enabled": "false", 2024-09-20T22:03:47.455 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_persistent_cache_mode": "disabled", 2024-09-20T22:03:47.455 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_persistent_cache_path": "/tmp", 2024-09-20T22:03:47.455 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_persistent_cache_size": "1073741824", 2024-09-20T22:03:47.455 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_plugins": "", 2024-09-20T22:03:47.455 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_bps_burst": "0", 2024-09-20T22:03:47.455 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-09-20T22:03:47.455 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_bps_limit": "0", 2024-09-20T22:03:47.455 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_exclude_ops": "0", 2024-09-20T22:03:47.456 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_iops_burst": "0", 2024-09-20T22:03:47.456 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-09-20T22:03:47.456 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_iops_limit": "0", 2024-09-20T22:03:47.456 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_read_bps_burst": "0", 2024-09-20T22:03:47.456 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-09-20T22:03:47.456 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_read_bps_limit": "0", 2024-09-20T22:03:47.456 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_read_iops_burst": "0", 2024-09-20T22:03:47.456 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-09-20T22:03:47.456 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_read_iops_limit": "0", 2024-09-20T22:03:47.456 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_schedule_tick_min": "50", 2024-09-20T22:03:47.456 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_write_bps_burst": "0", 2024-09-20T22:03:47.456 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-09-20T22:03:47.457 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_write_bps_limit": "0", 2024-09-20T22:03:47.457 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_write_iops_burst": "0", 2024-09-20T22:03:47.457 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-09-20T22:03:47.457 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_qos_write_iops_limit": "0", 2024-09-20T22:03:47.457 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_quiesce_notification_attempts": "10", 2024-09-20T22:03:47.457 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_read_from_replica_policy": "default", 2024-09-20T22:03:47.457 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-09-20T22:03:47.457 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_readahead_max_bytes": "524288", 2024-09-20T22:03:47.457 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_readahead_trigger_requests": "10", 2024-09-20T22:03:47.457 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_request_timed_out_seconds": "30", 2024-09-20T22:03:47.457 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_skip_partial_discard": "true", 2024-09-20T22:03:47.457 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-09-20T22:03:47.457 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_tracing": "false", 2024-09-20T22:03:47.457 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_validate_names": "true", 2024-09-20T22:03:47.458 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd_validate_pool": "true", 2024-09-20T22:03:47.458 INFO:teuthology.orchestra.run.smithi060.stdout: "restapi_base_url": "", 2024-09-20T22:03:47.458 INFO:teuthology.orchestra.run.smithi060.stdout: "restapi_log_level": "", 2024-09-20T22:03:47.458 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-09-20T22:03:47.458 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_account_default_quota_max_size": "-1", 2024-09-20T22:03:47.458 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_acl_grants_max_num": "100", 2024-09-20T22:03:47.458 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_admin_entry": "admin", 2024-09-20T22:03:47.458 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-09-20T22:03:47.458 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_backend_store": "rados", 2024-09-20T22:03:47.458 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_barbican_url": "", 2024-09-20T22:03:47.458 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_beast_enable_async": "true", 2024-09-20T22:03:47.458 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_bucket_counters_cache": "false", 2024-09-20T22:03:47.458 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-09-20T22:03:47.458 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-09-20T22:03:47.458 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-09-20T22:03:47.459 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_bucket_index_max_aio": "128", 2024-09-20T22:03:47.459 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-09-20T22:03:47.459 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-09-20T22:03:47.459 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_bucket_quota_ttl": "600", 2024-09-20T22:03:47.459 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-09-20T22:03:47.459 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_cache_enabled": "true", 2024-09-20T22:03:47.459 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_cache_expiry_interval": "900", 2024-09-20T22:03:47.459 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_cache_lru_size": "25000", 2024-09-20T22:03:47.459 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_config_store": "rados", 2024-09-20T22:03:47.459 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_content_length_compat": "false", 2024-09-20T22:03:47.459 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_copy_obj_progress": "true", 2024-09-20T22:03:47.459 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-09-20T22:03:47.459 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_cors_rules_max_num": "100", 2024-09-20T22:03:47.459 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_cross_domain_policy": "", 2024-09-20T22:03:47.460 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_default_encryption_key": "", 2024-09-20T22:03:47.460 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_kmip_addr": "", 2024-09-20T22:03:47.460 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_kmip_ca_path": "", 2024-09-20T22:03:47.460 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_kmip_client_cert": "", 2024-09-20T22:03:47.460 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_kmip_client_key": "", 2024-09-20T22:03:47.460 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-09-20T22:03:47.460 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_kmip_password": "", 2024-09-20T22:03:47.460 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-09-20T22:03:47.460 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_kmip_username": "", 2024-09-20T22:03:47.460 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_require_ssl": "true", 2024-09-20T22:03:47.460 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-09-20T22:03:47.460 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-09-20T22:03:47.460 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-09-20T22:03:47.460 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-09-20T22:03:47.460 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-09-20T22:03:47.460 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-09-20T22:03:47.461 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-09-20T22:03:47.461 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-09-20T22:03:47.461 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-09-20T22:03:47.461 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-09-20T22:03:47.461 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-09-20T22:03:47.461 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-09-20T22:03:47.461 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-09-20T22:03:47.461 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-09-20T22:03:47.461 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_suppress_logs": "true", 2024-09-20T22:03:47.461 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_addr": "", 2024-09-20T22:03:47.461 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_auth": "token", 2024-09-20T22:03:47.461 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_namespace": "", 2024-09-20T22:03:47.461 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_prefix": "", 2024-09-20T22:03:47.462 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-09-20T22:03:47.462 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-09-20T22:03:47.462 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-09-20T22:03:47.462 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-09-20T22:03:47.462 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_token_file": "", 2024-09-20T22:03:47.462 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-09-20T22:03:47.462 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_curl_buffersize": "524288", 2024-09-20T22:03:47.462 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_curl_low_speed_limit": "1024", 2024-09-20T22:03:47.462 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_curl_low_speed_time": "300", 2024-09-20T22:03:47.462 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_curl_tcp_keepalive": "0", 2024-09-20T22:03:47.462 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-09-20T22:03:47.462 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-09-20T22:03:47.462 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-09-20T22:03:47.462 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-09-20T22:03:47.462 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-09-20T22:03:47.463 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d3n_l1_fadvise": "4", 2024-09-20T22:03:47.463 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-09-20T22:03:47.463 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-09-20T22:03:47.463 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-09-20T22:03:47.463 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d4n_host": "127.0.0.1", 2024-09-20T22:03:47.463 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_d4n_port": "6379", 2024-09-20T22:03:47.463 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-09-20T22:03:47.463 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_data_log_changes_size": "1000", 2024-09-20T22:03:47.463 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_data_log_num_shards": "128", 2024-09-20T22:03:47.463 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-09-20T22:03:47.463 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_data_log_window": "30", 2024-09-20T22:03:47.463 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_data_notify_interval_msec": "0", 2024-09-20T22:03:47.463 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_data_sync_poll_interval": "20", 2024-09-20T22:03:47.464 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_data_sync_spawn_window": "20", 2024-09-20T22:03:47.464 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-09-20T22:03:47.464 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-09-20T22:03:47.464 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-09-20T22:03:47.464 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_default_data_log_backing": "fifo", 2024-09-20T22:03:47.464 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-09-20T22:03:47.464 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_default_region_info_oid": "default.region", 2024-09-20T22:03:47.464 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-09-20T22:03:47.464 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-09-20T22:03:47.464 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_defer_to_bucket_acls": "", 2024-09-20T22:03:47.464 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-09-20T22:03:47.464 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-09-20T22:03:47.464 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-09-20T22:03:47.464 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-09-20T22:03:47.464 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-09-20T22:03:47.465 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-09-20T22:03:47.465 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-09-20T22:03:47.465 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-09-20T22:03:47.465 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_data_res": "500.000000", 2024-09-20T22:03:47.465 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-09-20T22:03:47.465 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-09-20T22:03:47.465 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-09-20T22:03:47.465 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-09-20T22:03:47.465 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dns_name": "", 2024-09-20T22:03:47.465 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dns_s3website_name": "", 2024-09-20T22:03:47.465 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_dynamic_resharding": "true", 2024-09-20T22:03:47.465 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-09-20T22:03:47.465 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_enable_gc_threads": "true", 2024-09-20T22:03:47.465 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_enable_lc_threads": "true", 2024-09-20T22:03:47.466 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_enable_ops_log": "false", 2024-09-20T22:03:47.466 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_enable_quota_threads": "true", 2024-09-20T22:03:47.466 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_enable_static_website": "false", 2024-09-20T22:03:47.466 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_enable_usage_log": "false", 2024-09-20T22:03:47.466 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_enforce_swift_acls": "true", 2024-09-20T22:03:47.466 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_exit_timeout_secs": "120", 2024-09-20T22:03:47.466 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_expose_bucket": "false", 2024-09-20T22:03:47.466 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_extended_http_attrs": "", 2024-09-20T22:03:47.466 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_filter": "none", 2024-09-20T22:03:47.466 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-09-20T22:03:47.466 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_frontends": "beast port=7480", 2024-09-20T22:03:47.466 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_gc_max_concurrent_io": "10", 2024-09-20T22:03:47.466 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_gc_max_deferred": "50", 2024-09-20T22:03:47.466 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-09-20T22:03:47.467 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_gc_max_objs": "32", 2024-09-20T22:03:47.467 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_gc_max_queue_size": "134213632", 2024-09-20T22:03:47.467 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_gc_max_trim_chunk": "16", 2024-09-20T22:03:47.467 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_gc_obj_min_wait": "7200", 2024-09-20T22:03:47.467 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_gc_processor_max_time": "3600", 2024-09-20T22:03:47.467 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_gc_processor_period": "3600", 2024-09-20T22:03:47.467 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-09-20T22:03:47.467 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_get_obj_window_size": "16777216", 2024-09-20T22:03:47.467 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_healthcheck_disabling_path": "", 2024-09-20T22:03:47.467 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ignore_get_invalid_range": "false", 2024-09-20T22:03:47.467 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_init_timeout": "300", 2024-09-20T22:03:47.467 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-09-20T22:03:47.467 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-09-20T22:03:47.467 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_kafka_connection_idle": "300", 2024-09-20T22:03:47.468 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_kafka_message_timeout": "5000", 2024-09-20T22:03:47.468 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_kafka_sleep_timeout": "10", 2024-09-20T22:03:47.468 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-09-20T22:03:47.468 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-09-20T22:03:47.468 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-09-20T22:03:47.468 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_admin_domain": "", 2024-09-20T22:03:47.468 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_admin_password": "", 2024-09-20T22:03:47.468 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_admin_password_path": "", 2024-09-20T22:03:47.468 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_admin_project": "", 2024-09-20T22:03:47.468 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_admin_tenant": "", 2024-09-20T22:03:47.468 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_admin_token": "", 2024-09-20T22:03:47.468 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_admin_token_path": "", 2024-09-20T22:03:47.468 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_admin_user": "", 2024-09-20T22:03:47.469 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_api_version": "2", 2024-09-20T22:03:47.469 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_barbican_domain": "", 2024-09-20T22:03:47.469 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_barbican_password": "", 2024-09-20T22:03:47.469 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_barbican_project": "", 2024-09-20T22:03:47.469 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_barbican_tenant": "", 2024-09-20T22:03:47.469 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_barbican_user": "", 2024-09-20T22:03:47.469 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-09-20T22:03:47.469 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_implicit_tenants": "false", 2024-09-20T22:03:47.469 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-09-20T22:03:47.469 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_service_token_enabled": "false", 2024-09-20T22:03:47.469 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_token_cache_size": "10000", 2024-09-20T22:03:47.469 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_url": "", 2024-09-20T22:03:47.469 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_keystone_verify_ssl": "true", 2024-09-20T22:03:47.469 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lc_debug_interval": "-1", 2024-09-20T22:03:47.470 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lc_lock_max_time": "90", 2024-09-20T22:03:47.470 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lc_max_objs": "32", 2024-09-20T22:03:47.470 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lc_max_rules": "1000", 2024-09-20T22:03:47.470 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lc_max_worker": "3", 2024-09-20T22:03:47.470 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lc_max_wp_worker": "3", 2024-09-20T22:03:47.470 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lc_thread_delay": "0", 2024-09-20T22:03:47.470 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-09-20T22:03:47.470 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ldap_dnattr": "uid", 2024-09-20T22:03:47.470 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-09-20T22:03:47.470 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ldap_searchfilter": "", 2024-09-20T22:03:47.470 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-09-20T22:03:47.470 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ldap_uri": "", 2024-09-20T22:03:47.470 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-09-20T22:03:47.471 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-09-20T22:03:47.471 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-09-20T22:03:47.471 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_log_http_headers": "", 2024-09-20T22:03:47.471 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_log_nonexistent_bucket": "false", 2024-09-20T22:03:47.471 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-09-20T22:03:47.471 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_log_object_name_utc": "false", 2024-09-20T22:03:47.471 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-09-20T22:03:47.471 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-09-20T22:03:47.471 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_attr_name_len": "0", 2024-09-20T22:03:47.471 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_attr_size": "0", 2024-09-20T22:03:47.471 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_attrs_num_in_req": "0", 2024-09-20T22:03:47.471 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_chunk_size": "4194304", 2024-09-20T22:03:47.472 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_concurrent_requests": "1024", 2024-09-20T22:03:47.472 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-09-20T22:03:47.472 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_dynamic_shards": "1999", 2024-09-20T22:03:47.472 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_listing_results": "1000", 2024-09-20T22:03:47.472 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_notify_retries": "10", 2024-09-20T22:03:47.472 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_objs_per_shard": "100000", 2024-09-20T22:03:47.472 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_put_param_size": "1048576", 2024-09-20T22:03:47.472 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_put_size": "5368709120", 2024-09-20T22:03:47.472 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_max_slo_entries": "1000", 2024-09-20T22:03:47.472 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_md_log_max_shards": "64", 2024-09-20T22:03:47.472 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_md_notify_interval_msec": "200", 2024-09-20T22:03:47.472 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_meta_sync_poll_interval": "20", 2024-09-20T22:03:47.472 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_meta_sync_spawn_window": "20", 2024-09-20T22:03:47.472 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-09-20T22:03:47.473 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_mp_lock_max_time": "600", 2024-09-20T22:03:47.473 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-09-20T22:03:47.473 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_multipart_min_part_size": "5242880", 2024-09-20T22:03:47.473 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-09-20T22:03:47.473 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-09-20T22:03:47.473 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_fhcache_size": "2017", 2024-09-20T22:03:47.473 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-09-20T22:03:47.473 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-09-20T22:03:47.473 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_lru_lanes": "5", 2024-09-20T22:03:47.473 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_max_gc": "300", 2024-09-20T22:03:47.473 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-09-20T22:03:47.473 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_run_gc_threads": "false", 2024-09-20T22:03:47.473 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_run_lc_threads": "false", 2024-09-20T22:03:47.474 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_run_quota_threads": "false", 2024-09-20T22:03:47.474 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_run_sync_thread": "false", 2024-09-20T22:03:47.474 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-09-20T22:03:47.474 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-09-20T22:03:47.474 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_num_async_rados_threads": "32", 2024-09-20T22:03:47.474 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_num_control_oids": "8", 2024-09-20T22:03:47.474 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_numa_node": "-1", 2024-09-20T22:03:47.474 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_obj_stripe_size": "4194304", 2024-09-20T22:03:47.474 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-09-20T22:03:47.474 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_objexp_chunk_size": "100", 2024-09-20T22:03:47.474 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_objexp_gc_interval": "600", 2024-09-20T22:03:47.474 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_objexp_hints_num_shards": "127", 2024-09-20T22:03:47.475 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-09-20T22:03:47.475 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-09-20T22:03:47.475 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_op_thread_timeout": "600", 2024-09-20T22:03:47.475 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_op_tracing": "false", 2024-09-20T22:03:47.475 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_opa_token": "", 2024-09-20T22:03:47.475 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_opa_url": "", 2024-09-20T22:03:47.475 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_opa_verify_ssl": "true", 2024-09-20T22:03:47.475 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-09-20T22:03:47.475 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-09-20T22:03:47.475 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ops_log_rados": "false", 2024-09-20T22:03:47.475 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_ops_log_socket_path": "", 2024-09-20T22:03:47.475 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-09-20T22:03:47.475 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-09-20T22:03:47.475 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-09-20T22:03:47.476 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_period_push_interval": "2.000000", 2024-09-20T22:03:47.476 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_period_push_interval_max": "30.000000", 2024-09-20T22:03:47.476 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_period_root_pool": ".rgw.root", 2024-09-20T22:03:47.476 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-09-20T22:03:47.476 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-09-20T22:03:47.476 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_posix_cache_lanes": "3", 2024-09-20T22:03:47.476 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-09-20T22:03:47.476 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_posix_cache_max_buckets": "100", 2024-09-20T22:03:47.476 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_posix_cache_partitions": "3", 2024-09-20T22:03:47.476 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-09-20T22:03:47.476 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_print_continue": "true", 2024-09-20T22:03:47.476 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_print_prohibited_content_length": "false", 2024-09-20T22:03:47.476 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-09-20T22:03:47.476 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-09-20T22:03:47.477 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-09-20T22:03:47.477 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-09-20T22:03:47.477 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_rados_tracing": "false", 2024-09-20T22:03:47.477 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_realm": "", 2024-09-20T22:03:47.477 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_realm_id": "", 2024-09-20T22:03:47.477 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-09-20T22:03:47.477 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_region": "", 2024-09-20T22:03:47.477 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_region_root_pool": ".rgw.root", 2024-09-20T22:03:47.477 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_relaxed_region_enforcement": "false", 2024-09-20T22:03:47.477 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-09-20T22:03:47.477 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-09-20T22:03:47.477 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_request_uri": "", 2024-09-20T22:03:47.477 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_reshard_batch_size": "64", 2024-09-20T22:03:47.477 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-09-20T22:03:47.478 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_reshard_max_aio": "128", 2024-09-20T22:03:47.478 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_reshard_num_logs": "16", 2024-09-20T22:03:47.478 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_reshard_thread_interval": "600", 2024-09-20T22:03:47.478 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_resolve_cname": "false", 2024-09-20T22:03:47.478 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_rest_getusage_op_compat": "false", 2024-09-20T22:03:47.478 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_run_sync_thread": "true", 2024-09-20T22:03:47.478 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-09-20T22:03:47.478 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-09-20T22:03:47.478 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_s3_auth_use_keystone": "false", 2024-09-20T22:03:47.478 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_s3_auth_use_ldap": "false", 2024-09-20T22:03:47.478 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_s3_auth_use_rados": "true", 2024-09-20T22:03:47.478 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_s3_auth_use_sts": "false", 2024-09-20T22:03:47.478 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-09-20T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_s3_success_create_obj_status": "0", 2024-09-20T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-09-20T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_scheduler_type": "throttler", 2024-09-20T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_script_uri": "", 2024-09-20T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_service_provider_name": "", 2024-09-20T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-09-20T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sts_client_id": "", 2024-09-20T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sts_client_secret": "", 2024-09-20T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sts_entry": "sts", 2024-09-20T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sts_key": "sts", 2024-09-20T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sts_max_session_duration": "43200", 2024-09-20T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sts_min_session_duration": "900", 2024-09-20T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sts_token_introspection_url": "", 2024-09-20T22:03:47.479 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_account_in_url": "false", 2024-09-20T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_auth_entry": "auth", 2024-09-20T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_auth_url": "", 2024-09-20T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_custom_header": "", 2024-09-20T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_enforce_content_length": "false", 2024-09-20T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_need_stats": "true", 2024-09-20T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_tenant_name": "", 2024-09-20T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_token_expiration": "86400", 2024-09-20T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_url": "", 2024-09-20T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_url_prefix": "swift", 2024-09-20T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_swift_versioning_enabled": "false", 2024-09-20T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-09-20T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-09-20T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_lease_period": "120", 2024-09-20T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-09-20T22:03:47.480 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_log_trim_interval": "1200", 2024-09-20T22:03:47.481 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-09-20T22:03:47.481 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-09-20T22:03:47.481 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-09-20T22:03:47.481 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_obj_etag_verify": "false", 2024-09-20T22:03:47.481 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_trace_history_size": "4096", 2024-09-20T22:03:47.481 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-09-20T22:03:47.481 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-09-20T22:03:47.481 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_thread_pool_size": "512", 2024-09-20T22:03:47.481 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_topic_persistency_max_retries": "0", 2024-09-20T22:03:47.481 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-09-20T22:03:47.481 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-09-20T22:03:47.481 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_topic_require_publish_policy": "false", 2024-09-20T22:03:47.481 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_torrent_comment": "", 2024-09-20T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_torrent_createby": "", 2024-09-20T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_torrent_encoding": "", 2024-09-20T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_torrent_flag": "false", 2024-09-20T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_torrent_max_size": "5368709120", 2024-09-20T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_torrent_origin": "", 2024-09-20T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_torrent_sha_unit": "524288", 2024-09-20T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_torrent_tracker": "", 2024-09-20T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_trust_forwarded_https": "false", 2024-09-20T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-09-20T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_usage_log_tick_interval": "30", 2024-09-20T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_usage_max_shards": "32", 2024-09-20T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_usage_max_user_shards": "1", 2024-09-20T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_use_opa_authz": "false", 2024-09-20T22:03:47.482 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_counters_cache": "false", 2024-09-20T22:03:47.483 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_counters_cache_size": "10000", 2024-09-20T22:03:47.483 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-09-20T22:03:47.483 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_default_quota_max_size": "-1", 2024-09-20T22:03:47.483 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_max_buckets": "1000", 2024-09-20T22:03:47.483 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_policies_max_num": "100", 2024-09-20T22:03:47.483 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-09-20T22:03:47.483 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-09-20T22:03:47.483 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_quota_sync_interval": "86400", 2024-09-20T22:03:47.483 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-09-20T22:03:47.483 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_user_unique_email": "true", 2024-09-20T22:03:47.483 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_verify_ssl": "true", 2024-09-20T22:03:47.483 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_website_routing_rules_max_num": "50", 2024-09-20T22:03:47.483 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_zone": "", 2024-09-20T22:03:47.483 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_zone_id": "", 2024-09-20T22:03:47.484 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-09-20T22:03:47.484 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_zonegroup": "", 2024-09-20T22:03:47.484 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_zonegroup_id": "", 2024-09-20T22:03:47.484 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-09-20T22:03:47.484 INFO:teuthology.orchestra.run.smithi060.stdout: "rgwlc_auto_session_clear": "true", 2024-09-20T22:03:47.484 INFO:teuthology.orchestra.run.smithi060.stdout: "rgwlc_skip_bucket_step": "false", 2024-09-20T22:03:47.484 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_block_size": "4096", 2024-09-20T22:03:47.484 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-09-20T22:03:47.484 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-09-20T22:03:47.484 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-09-20T22:03:47.484 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-09-20T22:03:47.484 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_cache_shard_bits": "4", 2024-09-20T22:03:47.484 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_cache_size": "536870912", 2024-09-20T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_cache_type": "binned_lru", 2024-09-20T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-09-20T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-09-20T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-09-20T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_collect_compaction_stats": "false", 2024-09-20T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_collect_extended_stats": "false", 2024-09-20T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_collect_memory_stats": "false", 2024-09-20T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-09-20T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_index_type": "binary_search", 2024-09-20T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_log_to_ceph_log": "true", 2024-09-20T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_metadata_block_size": "4096", 2024-09-20T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_partition_filters": "false", 2024-09-20T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_perf": "false", 2024-09-20T22:03:47.485 INFO:teuthology.orchestra.run.smithi060.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-09-20T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-09-20T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stdout: "rotating_keys_renewal_timeout": "10", 2024-09-20T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stdout: "run_dir": "/var/run/ceph", 2024-09-20T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_block_create": "true", 2024-09-20T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_cache_lru_size": "67108864", 2024-09-20T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_cbjournal_size": "5368709120", 2024-09-20T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_data_delta_based_overwrite": "0", 2024-09-20T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_default_max_object_size": "16777216", 2024-09-20T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-09-20T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_device_size": "53687091200", 2024-09-20T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_disable_end_to_end_data_protection": "true", 2024-09-20T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_full_integrity_check": "false", 2024-09-20T22:03:47.486 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_journal_batch_capacity": "16", 2024-09-20T22:03:47.487 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-09-20T22:03:47.487 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-09-20T22:03:47.487 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_journal_iodepth_limit": "5", 2024-09-20T22:03:47.487 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_main_device_type": "SSD", 2024-09-20T22:03:47.487 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_max_concurrent_transactions": "8", 2024-09-20T22:03:47.487 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_max_data_allocation_size": "32768", 2024-09-20T22:03:47.487 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-09-20T22:03:47.487 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-09-20T22:03:47.487 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-09-20T22:03:47.487 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-09-20T22:03:47.487 INFO:teuthology.orchestra.run.smithi060.stdout: "seastore_segment_size": "67108864", 2024-09-20T22:03:47.487 INFO:teuthology.orchestra.run.smithi060.stdout: "set_keepcaps": "false", 2024-09-20T22:03:47.487 INFO:teuthology.orchestra.run.smithi060.stdout: "setgroup": "ceph", 2024-09-20T22:03:47.487 INFO:teuthology.orchestra.run.smithi060.stdout: "setuser": "ceph", 2024-09-20T22:03:47.488 INFO:teuthology.orchestra.run.smithi060.stdout: "setuser_match_path": "", 2024-09-20T22:03:47.488 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_misplaced_ratio": "0.050000", 2024-09-20T22:03:47.488 INFO:teuthology.orchestra.run.smithi060.stdout: "thp": "false", 2024-09-20T22:03:47.488 INFO:teuthology.orchestra.run.smithi060.stdout: "threadpool_default_timeout": "60", 2024-09-20T22:03:47.488 INFO:teuthology.orchestra.run.smithi060.stdout: "threadpool_empty_queue_max_wait": "2", 2024-09-20T22:03:47.488 INFO:teuthology.orchestra.run.smithi060.stdout: "throttler_perf_counter": "true" 2024-09-20T22:03:47.488 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:03:47.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:47 smithi067 ceph-mon[33511]: pgmap v20: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 552 B/s rd, 0 op/s 2024-09-20T22:03:48.296 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-09-20T22:03:48.551 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:48.981 INFO:teuthology.orchestra.run.smithi060.stdout:mon.a: {} 2024-09-20T22:03:48.981 INFO:teuthology.orchestra.run.smithi060.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-20T22:03:48.990 INFO:teuthology.orchestra.run.smithi060.stdout:mon.b: {} 2024-09-20T22:03:48.990 INFO:teuthology.orchestra.run.smithi060.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-20T22:03:48.999 INFO:teuthology.orchestra.run.smithi060.stdout:mon.c: {} 2024-09-20T22:03:48.999 INFO:teuthology.orchestra.run.smithi060.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-20T22:03:49.532 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:49 smithi060 ceph-mon[29694]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:03:49.532 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:49 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1598741541' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-20T22:03:49.532 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:49 smithi060 ceph-mon[40290]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:03:49.532 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:49 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1598741541' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-20T22:03:49.561 INFO:teuthology.run_tasks:Running task full_sequential... 2024-09-20T22:03:49.577 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-20T22:03:49.578 INFO:tasks.radosbench:Beginning radosbench... 2024-09-20T22:03:49.578 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_0 2024-09-20T22:03:49.579 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool create unique_pool_0 16 2024-09-20T22:03:49.579 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-09-20T22:03:49.579 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T22:03:49.579 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:03:49.580 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-09-20T22:03:49.580 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:03:49.580 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-09-20T22:03:49.580 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:03:49.581 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-09-20T22:03:49.581 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T22:03:49.581 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-09-20T22:03:49.582 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:03:49.613 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:49.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:49.614 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:03:49.714 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:03:49.747 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:49.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:49.748 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:03:49.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:49 smithi067 ceph-mon[33511]: pgmap v22: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:03:49.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:49 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1598741541' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-20T22:03:49.849 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:03:49.875 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:49.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:49.876 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:03:49.900 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:49.902 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:49.903 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:49.903 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:49.976 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:03:50.002 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:50.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:50.003 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:03:50.103 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:03:50.129 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:50.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:50.130 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:03:50.230 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:03:50.252 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:50.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:50.252 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:03:50.353 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:03:50.377 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:50.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:50.378 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:03:50.478 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:03:50.502 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:50.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:50.503 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:03:50.603 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:03:50.627 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:50.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:50.628 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:03:50.673 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:50 smithi060 ceph-mon[29694]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 619 B/s rd, 0 op/s 2024-09-20T22:03:50.673 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:50 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:50.674 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:50 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:03:50.674 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:50 smithi060 ceph-mon[40290]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 619 B/s rd, 0 op/s 2024-09-20T22:03:50.674 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:50 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:50.674 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:50 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:03:50.729 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:03:50.755 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:50.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:50.756 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:03:50.856 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:03:50.882 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:50.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:50.883 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:03:50.984 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:03:51.006 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:51.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:51.007 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:03:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:50 smithi067 ceph-mon[33511]: pgmap v23: 132 pgs: 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 619 B/s rd, 0 op/s 2024-09-20T22:03:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:50 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:50 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:03:51.109 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:03:51.134 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:51.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:51.135 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:03:51.219 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:03:51.219 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:03:51.219 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-20T22:03:51.219 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:03:51.236 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:03:51.257 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:51.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:51.258 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:03:51.298 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'unique_pool_0' created 2024-09-20T22:03:51.359 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:03:51.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:51.382 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:51.382 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:03:51.483 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:03:51.504 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:51.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:51.505 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:03:51.606 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:03:51.628 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:51.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:51.629 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:03:51.729 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:03:51.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:51.752 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:51.752 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:03:51.782 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:03:51.783 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:03:51.799 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:03:51.800 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:03:51.816 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:03:51.816 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:03:51.832 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:03:51.832 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:03:51.849 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:03:51.849 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:03:51.850 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:51 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1926120546' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-20T22:03:51.850 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:51 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-20T22:03:51.850 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:51 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4234130298' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:03:51.850 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:51 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:03:51.850 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:51 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-20T22:03:51.850 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:51 smithi060 ceph-mon[40290]: osdmap e80: 8 total, 8 up, 8 in 2024-09-20T22:03:51.850 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:51 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1926120546' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-20T22:03:51.850 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:51 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-20T22:03:51.850 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:51 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4234130298' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:03:51.851 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:51 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:03:51.851 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:51 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-20T22:03:51.851 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:51 smithi060 ceph-mon[29694]: osdmap e80: 8 total, 8 up, 8 in 2024-09-20T22:03:51.853 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:03:51.867 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:03:51.867 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:03:51.876 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:51.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:51.877 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:03:51.884 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:03:51.884 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:03:51.900 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:03:51.900 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:03:51.978 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:03:51.984 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:03:51 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:03:51.984 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:51 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1926120546' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-20T22:03:51.985 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:51 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]: dispatch 2024-09-20T22:03:51.985 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:51 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4234130298' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:03:51.985 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:51 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:03:51.985 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:51 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_0", "pg_num": 16}]': finished 2024-09-20T22:03:51.985 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:51 smithi067 ceph-mon[33511]: osdmap e80: 8 total, 8 up, 8 in 2024-09-20T22:03:51.999 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:52.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:52.000 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:03:52.102 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:03:52.124 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:52.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:52.125 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:03:52.154 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T22:03:52.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:03:52.259 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:52.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:52.260 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:03:52.304 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:03:52.360 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:03:52.382 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:52.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:52.383 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:03:52.484 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:03:52.506 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:52.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:52.507 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:03:52.608 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:03:52.630 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:52.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:52.631 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:03:52.732 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:03:52.754 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:52.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:52.755 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:03:52.855 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:03:52.880 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:52.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:52.881 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:03:52.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:52 smithi060 ceph-mon[29694]: pgmap v25: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:03:52.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:52 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:03:52.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:52 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2852501489' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:03:52.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:52 smithi060 ceph-mon[29694]: 4.b scrub starts 2024-09-20T22:03:52.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:52 smithi060 ceph-mon[29694]: 4.b scrub ok 2024-09-20T22:03:52.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:52 smithi060 ceph-mon[29694]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T22:03:52.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:52 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:03:52.934 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:52 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:03:52.934 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:52 smithi060 ceph-mon[29694]: osdmap e81: 8 total, 8 up, 8 in 2024-09-20T22:03:52.934 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:52 smithi060 ceph-mon[40290]: pgmap v25: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:03:52.934 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:52 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:03:52.934 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:52 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2852501489' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:03:52.934 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:52 smithi060 ceph-mon[40290]: 4.b scrub starts 2024-09-20T22:03:52.934 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:52 smithi060 ceph-mon[40290]: 4.b scrub ok 2024-09-20T22:03:52.934 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:52 smithi060 ceph-mon[40290]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T22:03:52.934 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:52 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:03:52.934 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:52 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:03:52.935 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:52 smithi060 ceph-mon[40290]: osdmap e81: 8 total, 8 up, 8 in 2024-09-20T22:03:52.982 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:03:53.004 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:53.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:53.005 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:03:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:52 smithi067 ceph-mon[33511]: pgmap v25: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:03:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:52 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:03:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:52 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2852501489' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:03:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:52 smithi067 ceph-mon[33511]: 4.b scrub starts 2024-09-20T22:03:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:52 smithi067 ceph-mon[33511]: 4.b scrub ok 2024-09-20T22:03:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:52 smithi067 ceph-mon[33511]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T22:03:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:52 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:03:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:52 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:03:53.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:52 smithi067 ceph-mon[33511]: osdmap e81: 8 total, 8 up, 8 in 2024-09-20T22:03:53.105 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:03:53.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:53.128 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:53.128 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:03:53.229 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:03:53.252 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:53.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:53.253 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:03:53.354 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:03:53.376 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:53.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:53.377 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:03:53.478 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:03:53.501 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:53.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:53.503 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:03:53.589 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-09-20T22:03:53.604 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:03:53.625 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:53.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:53.626 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:03:53.727 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:03:53.750 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:53.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:53.751 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:03:53.852 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:03:53.875 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:53.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:53.876 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:03:53.976 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:03:54.003 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:54.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:54.004 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:03:54.040 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:54.043 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:54.105 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:03:54.131 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:54.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:54.132 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:03:54.232 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:03:54.257 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:54.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:54.258 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:03:54.359 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:03:54.385 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:54.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:54.386 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:03:54.487 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:03:54.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:54.511 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:54.512 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:03:54.604 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:54 smithi067 ceph-mon[33511]: osdmap e82: 8 total, 8 up, 8 in 2024-09-20T22:03:54.613 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:03:54.623 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:54 smithi060 ceph-mon[29694]: osdmap e82: 8 total, 8 up, 8 in 2024-09-20T22:03:54.623 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:54 smithi060 ceph-mon[40290]: osdmap e82: 8 total, 8 up, 8 in 2024-09-20T22:03:54.637 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:54.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:54.638 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:03:54.738 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:03:54.761 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:54.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:54.762 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:03:54.862 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:03:54.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:54.886 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:54.886 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:03:54.929 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:03:54.930 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:03:54.962 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:03:54.987 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:03:55.010 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:54 smithi067 systemd[1]: Starting Ceph grafana.a for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:03:55.027 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:55.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:55.028 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:03:55.128 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:03:55.152 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:55.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:55.153 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:03:55.253 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:03:55.261 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:03:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:03:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:03:55.261 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:03:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[76563]: ts=2024-09-20T22:03:55.055Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.001827072s 2024-09-20T22:03:55.263 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 podman[65749]: 2024-09-20 22:03:54.912410152 +0000 UTC m=+0.016879710 image pull 2bacad6d85d802cd176c146528adcc0e47a8a83e94bc4a8b7ba1a9b9ef2c65e7 quay.io/ceph/grafana:9.4.12 2024-09-20T22:03:55.263 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 podman[65749]: 2024-09-20 22:03:55.018013375 +0000 UTC m=+0.122482973 container create 04a820c2471a0b105c5efcfd677177fea3faf83a3dca118e276754adbc848501 (image=quay.io/ceph/grafana:9.4.12, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-20T22:03:55.263 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 podman[65749]: 2024-09-20 22:03:55.236722307 +0000 UTC m=+0.341191870 container init 04a820c2471a0b105c5efcfd677177fea3faf83a3dca118e276754adbc848501 (image=quay.io/ceph/grafana:9.4.12, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-20T22:03:55.263 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 podman[65749]: 2024-09-20 22:03:55.241730141 +0000 UTC m=+0.346199705 container start 04a820c2471a0b105c5efcfd677177fea3faf83a3dca118e276754adbc848501 (image=quay.io/ceph/grafana:9.4.12, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-20T22:03:55.276 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:55.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:55.276 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:03:55.306 INFO:teuthology.orchestra.run.smithi060.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-09-20T22:03:55.378 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:03:55.403 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:55.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:55.404 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:03:55.505 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:03:55.521 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:55 smithi060 ceph-mon[40290]: pgmap v28: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:03:55.521 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:55 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3944350921' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:03:55.522 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:55 smithi060 ceph-mon[29694]: pgmap v28: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:03:55.522 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:55 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3944350921' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:03:55.527 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:55.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:55.527 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:03:55.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T22:03:55.561 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-mon[33511]: pgmap v28: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 239 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:03:55.561 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3944350921' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:03:55.561 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 bash[65749]: 04a820c2471a0b105c5efcfd677177fea3faf83a3dca118e276754adbc848501 2024-09-20T22:03:55.562 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 systemd[1]: Started Ceph grafana.a for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T22:03:55.562 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.399198214Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-09-20T22:03:55.562 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.399376211Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-20T22:03:55.562 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.39939612Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-20T22:03:55.562 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.399400794Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-20T22:03:55.562 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.399403721Z level=info msg="Config overridden from command line" arg="default.paths.data=/var/lib/grafana" 2024-09-20T22:03:55.562 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.39940662Z level=info msg="Config overridden from command line" arg="default.paths.logs=/var/log/grafana" 2024-09-20T22:03:55.562 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.399410332Z level=info msg="Config overridden from command line" arg="default.paths.plugins=/var/lib/grafana/plugins" 2024-09-20T22:03:55.562 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.399413163Z level=info msg="Config overridden from command line" arg="default.paths.provisioning=/etc/grafana/provisioning" 2024-09-20T22:03:55.562 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.399415952Z level=info msg="Config overridden from command line" arg="default.log.mode=console" 2024-09-20T22:03:55.562 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.39941929Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-20T22:03:55.562 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.399422464Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-20T22:03:55.562 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.399425216Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-20T22:03:55.563 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.399428095Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-20T22:03:55.563 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.399430941Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-20T22:03:55.563 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.399433712Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-20T22:03:55.563 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.399436353Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-20T22:03:55.563 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.399438991Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-20T22:03:55.564 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.39944261Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-20T22:03:55.564 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=settings t=2024-09-20T22:03:55.399445541Z level=info msg="App mode production" 2024-09-20T22:03:55.564 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=sqlstore t=2024-09-20T22:03:55.399493038Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-20T22:03:55.564 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=sqlstore t=2024-09-20T22:03:55.39951458Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-20T22:03:55.564 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.400442955Z level=info msg="Starting DB migrations" 2024-09-20T22:03:55.564 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.400511679Z level=info msg="Executing migration" id="create migration_log table" 2024-09-20T22:03:55.564 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.40108369Z level=info msg="Executing migration" id="create user table" 2024-09-20T22:03:55.564 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.401554618Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-20T22:03:55.564 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.402039942Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-20T22:03:55.564 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.402497201Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-20T22:03:55.564 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.403068816Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-20T22:03:55.564 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.40347923Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-20T22:03:55.565 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.404283926Z level=info msg="Executing migration" id="create user table v2" 2024-09-20T22:03:55.565 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.404733224Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-20T22:03:55.565 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.405163264Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-20T22:03:55.565 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.405622985Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-20T22:03:55.565 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.406026696Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-20T22:03:55.565 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.406407133Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-20T22:03:55.565 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.406866698Z level=info msg="Executing migration" id="Update user table charset" 2024-09-20T22:03:55.565 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.407145071Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-20T22:03:55.565 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.407577374Z level=info msg="Executing migration" id="Add missing user data" 2024-09-20T22:03:55.565 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.4079334Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-20T22:03:55.565 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.408334484Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-20T22:03:55.565 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.408748805Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-20T22:03:55.565 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.409178102Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-20T22:03:55.566 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.410590288Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-20T22:03:55.566 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.411029953Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-20T22:03:55.566 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.411476063Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-20T22:03:55.566 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.411885599Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-20T22:03:55.566 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.412281471Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-20T22:03:55.566 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.412695639Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-20T22:03:55.566 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.412994158Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-20T22:03:55.566 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.413373884Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-20T22:03:55.566 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.41378675Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-20T22:03:55.566 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.414142703Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-20T22:03:55.566 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.414540766Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-20T22:03:55.566 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.415312944Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-20T22:03:55.566 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.415724292Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-20T22:03:55.566 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.416213028Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-20T22:03:55.567 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.416623317Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-20T22:03:55.567 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.4170201Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-20T22:03:55.567 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.417437914Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-20T22:03:55.567 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.417805715Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-20T22:03:55.567 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.418129323Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-20T22:03:55.567 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.418449771Z level=info msg="Executing migration" id="create star table" 2024-09-20T22:03:55.567 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.418828652Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-20T22:03:55.567 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.419238637Z level=info msg="Executing migration" id="create org table v1" 2024-09-20T22:03:55.567 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.419631037Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-20T22:03:55.567 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.4200379Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-20T22:03:55.567 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.42041962Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-20T22:03:55.567 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.420813826Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-20T22:03:55.567 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.421212229Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-20T22:03:55.567 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.421640599Z level=info msg="Executing migration" id="Update org table charset" 2024-09-20T22:03:55.568 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.422486237Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-20T22:03:55.568 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.423453228Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-20T22:03:55.568 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.424191069Z level=info msg="Executing migration" id="create dashboard table" 2024-09-20T22:03:55.568 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.425200745Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-20T22:03:55.568 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.425894978Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-20T22:03:55.568 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.426915887Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-20T22:03:55.568 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.427606475Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-20T22:03:55.568 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.428580683Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-20T22:03:55.568 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.42928966Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-20T22:03:55.568 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.430379452Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-20T22:03:55.568 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.431327793Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-20T22:03:55.568 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.432073717Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-20T22:03:55.568 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.432964744Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-20T22:03:55.568 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.43379495Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-20T22:03:55.569 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.434650538Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-20T22:03:55.569 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.435506397Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-20T22:03:55.569 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.436397756Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-20T22:03:55.569 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.437418938Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-20T22:03:55.569 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.438169041Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-20T22:03:55.569 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.439173646Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-20T22:03:55.569 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.439948006Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-20T22:03:55.569 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.440818823Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-20T22:03:55.569 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.441697476Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-20T22:03:55.569 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.442629248Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-20T22:03:55.569 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.443403145Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-20T22:03:55.570 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.444256266Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-20T22:03:55.570 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.444827161Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-20T22:03:55.570 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.445690025Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-20T22:03:55.570 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.446571771Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-20T22:03:55.570 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.447533798Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-20T22:03:55.570 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.448474859Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-20T22:03:55.570 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.449190763Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-20T22:03:55.570 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.450020115Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-20T22:03:55.571 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.45047469Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-20T22:03:55.571 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.450944506Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-20T22:03:55.571 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.45385253Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-20T22:03:55.571 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.456644449Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-20T22:03:55.571 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.459580318Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-20T22:03:55.571 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.462234794Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-20T22:03:55.571 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.465002388Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-20T22:03:55.571 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.46779866Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-20T22:03:55.571 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.470582386Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-20T22:03:55.571 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.473529352Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-20T22:03:55.571 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.476227532Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-20T22:03:55.571 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.479243825Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-20T22:03:55.572 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.481964781Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-20T22:03:55.572 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.484796168Z level=info msg="Executing migration" id="create data_source table" 2024-09-20T22:03:55.572 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.487721431Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-20T22:03:55.572 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.490515387Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-20T22:03:55.572 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.491006585Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-20T22:03:55.572 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.493762171Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-20T22:03:55.572 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.494200568Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-20T22:03:55.572 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.496681573Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-20T22:03:55.572 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.499345187Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-20T22:03:55.572 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.501883597Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-20T22:03:55.572 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.504497094Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-20T22:03:55.572 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.507146673Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-20T22:03:55.572 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.509813423Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-20T22:03:55.572 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.512413598Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-20T22:03:55.573 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.515146192Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-20T22:03:55.573 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.517683249Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-20T22:03:55.573 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.520350657Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-20T22:03:55.573 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.523134056Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-20T22:03:55.573 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.523489939Z level=info msg="Executing migration" id="Add uid column" 2024-09-20T22:03:55.573 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.524357676Z level=info msg="Executing migration" id="Update uid value" 2024-09-20T22:03:55.573 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.52521214Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-20T22:03:55.573 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.5260537Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-20T22:03:55.573 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.526942203Z level=info msg="Executing migration" id="create api_key table" 2024-09-20T22:03:55.573 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.527839406Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-20T22:03:55.573 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.528674079Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-20T22:03:55.573 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.529492962Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-20T22:03:55.573 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.530341088Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-20T22:03:55.573 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.531341638Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-20T22:03:55.574 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.532023347Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-20T22:03:55.574 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.532443428Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-20T22:03:55.574 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.533905929Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-20T22:03:55.574 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.534344759Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-20T22:03:55.574 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.535187093Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-20T22:03:55.574 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.536070758Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-20T22:03:55.574 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.536654403Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-20T22:03:55.574 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.537485814Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-20T22:03:55.574 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.538444704Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-20T22:03:55.574 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.539319758Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-20T22:03:55.574 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.540165859Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-20T22:03:55.574 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.541000481Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-20T22:03:55.574 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.541879609Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-20T22:03:55.574 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.54270716Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-20T22:03:55.575 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.543298805Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-20T22:03:55.575 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.544150046Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-20T22:03:55.575 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.545008599Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-20T22:03:55.575 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.545879536Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-20T22:03:55.575 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.546312827Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-20T22:03:55.575 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.547180173Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-20T22:03:55.575 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.548017528Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-20T22:03:55.575 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.548854405Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-20T22:03:55.575 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.549154935Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-20T22:03:55.575 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.550063148Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-20T22:03:55.575 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.553102699Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-20T22:03:55.575 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.557136051Z level=info msg="Executing migration" id="create quota table v1" 2024-09-20T22:03:55.575 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.557555612Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-20T22:03:55.575 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.560598985Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-20T22:03:55.576 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.563295099Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-20T22:03:55.628 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:03:55.657 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:55.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:55.658 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:03:55.758 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:03:55.785 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:55.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:55.786 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:03:55.812 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.566181775Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-20T22:03:55.812 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.566629449Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-20T22:03:55.812 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.569403344Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-20T22:03:55.813 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.572102595Z level=info msg="Executing migration" id="create session table" 2024-09-20T22:03:55.813 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.574943055Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-20T22:03:55.813 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.575248423Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-20T22:03:55.813 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.578279087Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-20T22:03:55.813 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.581165049Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-20T22:03:55.813 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.584056899Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-20T22:03:55.813 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.584374034Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-20T22:03:55.813 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.587088731Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-20T22:03:55.813 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.590167654Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-20T22:03:55.813 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.593009565Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-20T22:03:55.813 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.593555738Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-20T22:03:55.813 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.596395106Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-20T22:03:55.813 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.599092621Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-20T22:03:55.813 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.601861473Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-20T22:03:55.813 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.602536685Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-20T22:03:55.814 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.605368315Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-20T22:03:55.814 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.608284848Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-20T22:03:55.814 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.610940354Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-20T22:03:55.814 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.611632516Z level=info msg="Executing migration" id="create alert table v1" 2024-09-20T22:03:55.814 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.614340183Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-20T22:03:55.814 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.616974525Z level=info msg="Executing migration" id="add index alert state" 2024-09-20T22:03:55.814 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.619560364Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-20T22:03:55.815 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.620049837Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-20T22:03:55.815 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.623018337Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-20T22:03:55.815 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.625396154Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-20T22:03:55.815 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.62823847Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-20T22:03:55.815 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.630100378Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-20T22:03:55.815 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.630977793Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-20T22:03:55.815 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.631985583Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-20T22:03:55.815 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.632674132Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-20T22:03:55.815 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.633106867Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-20T22:03:55.815 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.633894612Z level=info msg="Executing migration" id="Add column is_default" 2024-09-20T22:03:55.815 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.634655661Z level=info msg="Executing migration" id="Add column frequency" 2024-09-20T22:03:55.815 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.635416218Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-20T22:03:55.815 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.636244901Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-20T22:03:55.815 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.637007188Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-20T22:03:55.816 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.637869687Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-20T22:03:55.816 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.638696955Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-20T22:03:55.816 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.639677683Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-20T22:03:55.816 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.6401251Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-20T22:03:55.816 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.640971906Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-20T22:03:55.816 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.641785557Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-20T22:03:55.816 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.64272046Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-20T22:03:55.816 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.643200092Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-20T22:03:55.816 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.644189377Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-20T22:03:55.816 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.645137962Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-20T22:03:55.816 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.645952551Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-20T22:03:55.816 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.646411564Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-20T22:03:55.816 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.647247665Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-20T22:03:55.816 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.64810644Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-20T22:03:55.817 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.648944909Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-20T22:03:55.817 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.649396401Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-20T22:03:55.817 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.650278898Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-20T22:03:55.817 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.651242726Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-20T22:03:55.818 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.652137957Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-20T22:03:55.818 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.652701086Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-20T22:03:55.818 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.653715982Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-20T22:03:55.818 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.654464279Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-20T22:03:55.818 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.655465956Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-20T22:03:55.818 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.656090633Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-20T22:03:55.818 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.656919851Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-20T22:03:55.818 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.65773792Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-20T22:03:55.818 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.658217131Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-20T22:03:55.818 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.661151746Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-20T22:03:55.818 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.66167188Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-20T22:03:55.819 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.664617565Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-20T22:03:55.819 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.667479833Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-20T22:03:55.819 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.670434574Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-20T22:03:55.819 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.670851044Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-20T22:03:55.819 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.674025709Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-20T22:03:55.819 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.676661857Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-20T22:03:55.819 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.679473725Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-20T22:03:55.819 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.679802328Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-20T22:03:55.819 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.682641615Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-20T22:03:55.819 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.685524773Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-20T22:03:55.819 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.68820034Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-20T22:03:55.819 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.688672411Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-20T22:03:55.819 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.691570247Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-20T22:03:55.819 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.694645469Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-20T22:03:55.819 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.697359533Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-20T22:03:55.820 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.697728647Z level=info msg="Executing migration" id="Move region to single row" 2024-09-20T22:03:55.820 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.700476547Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-20T22:03:55.820 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.70097421Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-20T22:03:55.820 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.703256501Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-20T22:03:55.820 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.703751907Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-20T22:03:55.820 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.706101082Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-20T22:03:55.820 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.706649783Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-20T22:03:55.820 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.709244125Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-20T22:03:55.820 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.712153281Z level=info msg="Executing migration" id="create test_data table" 2024-09-20T22:03:55.820 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.714848021Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-20T22:03:55.820 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.715307344Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-20T22:03:55.820 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.716157234Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-20T22:03:55.820 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.716995356Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-20T22:03:55.820 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.717851277Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-20T22:03:55.821 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.718238641Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-20T22:03:55.821 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.719082218Z level=info msg="Executing migration" id="create team table" 2024-09-20T22:03:55.821 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.719976287Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-20T22:03:55.821 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.720800714Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-20T22:03:55.821 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.721276322Z level=info msg="Executing migration" id="create team member table" 2024-09-20T22:03:55.821 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.722309617Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-20T22:03:55.821 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.723001626Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-20T22:03:55.821 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.723836988Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-20T22:03:55.821 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.724288367Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-20T22:03:55.821 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.725146815Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-20T22:03:55.821 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.726024154Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-20T22:03:55.821 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.726915775Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-20T22:03:55.821 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.727350795Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-20T22:03:55.821 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.728222521Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-20T22:03:55.821 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.729003294Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-20T22:03:55.822 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.729454037Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-20T22:03:55.822 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.7299651Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-20T22:03:55.822 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.730831638Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-20T22:03:55.822 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.73171851Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-20T22:03:55.822 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.732699537Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-20T22:03:55.822 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.733442584Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-20T22:03:55.822 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.73447996Z level=info msg="Executing migration" id="create tag table" 2024-09-20T22:03:55.822 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.735195785Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-20T22:03:55.822 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.736217582Z level=info msg="Executing migration" id="create login attempt table" 2024-09-20T22:03:55.822 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.736963899Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-20T22:03:55.822 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.73782231Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-20T22:03:55.822 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.738701676Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-20T22:03:55.822 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.741119191Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-20T22:03:55.822 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.744083521Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-20T22:03:55.822 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.746927339Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-20T22:03:55.823 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.749690812Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-20T22:03:55.823 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.752473055Z level=info msg="Executing migration" id="create user auth table" 2024-09-20T22:03:55.823 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.755192834Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-20T22:03:55.823 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.75791427Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-20T22:03:55.823 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.761192692Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-20T22:03:55.823 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.763541233Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-20T22:03:55.823 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.766532961Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-20T22:03:55.823 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.769401192Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-20T22:03:55.823 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.772217684Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-20T22:03:55.823 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.774883603Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-20T22:03:55.823 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.777607423Z level=info msg="Executing migration" id="create server_lock table" 2024-09-20T22:03:55.823 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.780560216Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-20T22:03:55.823 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.783415348Z level=info msg="Executing migration" id="create user auth token table" 2024-09-20T22:03:55.823 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.78605504Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-20T22:03:55.823 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.788919863Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-20T22:03:55.823 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.791664677Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-20T22:03:55.824 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.794197867Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-20T22:03:55.824 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.796953388Z level=info msg="Executing migration" id="create cache_data table" 2024-09-20T22:03:55.824 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.799726067Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-20T22:03:55.824 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.802470368Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-20T22:03:55.824 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.804917451Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-20T22:03:55.824 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.807595605Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-20T22:03:55.824 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.810212848Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-20T22:03:55.824 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.810537561Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-20T22:03:55.824 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.811013929Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-20T22:03:55.824 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.811494363Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-20T22:03:55.824 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.811935425Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-20T22:03:55.825 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.81222124Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-20T22:03:55.825 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.812720007Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-20T22:03:55.825 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.813160675Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-20T22:03:55.825 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.813637272Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-20T22:03:55.825 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.814095399Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-20T22:03:55.825 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.815007481Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-20T22:03:55.825 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.815475071Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-20T22:03:55.825 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.815811051Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-20T22:03:55.825 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.816212686Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-20T22:03:55.825 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.816684159Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-20T22:03:55.825 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.817133136Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-20T22:03:55.825 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.817420238Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-20T22:03:55.825 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.817923501Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-20T22:03:55.887 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:03:55.909 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:55.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:55.910 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:03:55.997 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:03:56.010 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:03:56.033 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:56.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:56.034 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:03:56.063 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.818340975Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-20T22:03:56.063 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.818797935Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-20T22:03:56.063 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.819255056Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-20T22:03:56.063 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.820174271Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-20T22:03:56.063 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.820635148Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-20T22:03:56.063 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.82108369Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-20T22:03:56.063 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.825618753Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-20T22:03:56.063 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.830140652Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-20T22:03:56.063 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.830671262Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-20T22:03:56.063 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.831123357Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-20T22:03:56.063 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.832115939Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-20T22:03:56.063 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.832564474Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-20T22:03:56.064 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.833033393Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-20T22:03:56.064 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.83357449Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-20T22:03:56.064 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.834037331Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-20T22:03:56.064 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.834331087Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-20T22:03:56.064 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.835299236Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-20T22:03:56.064 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.836241213Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-20T22:03:56.064 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.83717698Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-20T22:03:56.064 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.837632335Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-20T22:03:56.064 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.838096319Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-20T22:03:56.064 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.839042585Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-20T22:03:56.064 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.839986032Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-20T22:03:56.064 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.840460028Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-20T22:03:56.064 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.841405978Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-20T22:03:56.064 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.84233887Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-20T22:03:56.065 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.842708843Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-20T22:03:56.065 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.843179144Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-20T22:03:56.065 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.845886637Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-20T22:03:56.065 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.846339701Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-20T22:03:56.065 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.846683222Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-20T22:03:56.065 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.849389459Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-20T22:03:56.065 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.852021181Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-20T22:03:56.065 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.854881581Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-20T22:03:56.065 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.85745115Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-20T22:03:56.066 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.860260314Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-20T22:03:56.066 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.862989078Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-20T22:03:56.066 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.865549042Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-20T22:03:56.066 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.86820838Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-20T22:03:56.066 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.869141062Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-20T22:03:56.066 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.870359074Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-20T22:03:56.067 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.871213183Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-20T22:03:56.067 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.872197873Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-20T22:03:56.067 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.873188073Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-20T22:03:56.067 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.873906302Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-20T22:03:56.067 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.874905131Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-20T22:03:56.067 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.875757706Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-20T22:03:56.067 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.87660516Z level=info msg="Executing migration" id="create alert_image table" 2024-09-20T22:03:56.067 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.877603036Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-20T22:03:56.067 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.878291534Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-20T22:03:56.067 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.879268405Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-20T22:03:56.067 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.879952982Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-20T22:03:56.067 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.880389853Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-20T22:03:56.067 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.880479163Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-20T22:03:56.067 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.880789473Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-20T22:03:56.068 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.881262832Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-20T22:03:56.068 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.881744136Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-20T22:03:56.068 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.88188668Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-20T22:03:56.068 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.882013201Z level=info msg="alerts found to migrate" alerts=0 2024-09-20T22:03:56.068 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.882485288Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-20T22:03:56.068 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.883248331Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-20T22:03:56.068 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.884133824Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-20T22:03:56.068 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.885062217Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-20T22:03:56.068 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.88589731Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-20T22:03:56.068 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.888489449Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-20T22:03:56.068 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.891207404Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-20T22:03:56.068 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.893824708Z level=info msg="Executing migration" id="create data_keys table" 2024-09-20T22:03:56.068 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.896671978Z level=info msg="Executing migration" id="create secrets table" 2024-09-20T22:03:56.068 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.899426199Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-20T22:03:56.068 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.905148314Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-20T22:03:56.069 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.907936921Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-20T22:03:56.069 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.910824915Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-20T22:03:56.069 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.916675609Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-20T22:03:56.069 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.922429344Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-20T22:03:56.069 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.925092427Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-20T22:03:56.069 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.927774198Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-20T22:03:56.069 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.928754598Z level=info msg="Executing migration" id="create permission table" 2024-09-20T22:03:56.069 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.929479859Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-20T22:03:56.069 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.930479039Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-20T22:03:56.069 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.931193144Z level=info msg="Executing migration" id="create role table" 2024-09-20T22:03:56.069 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.932032862Z level=info msg="Executing migration" id="add column display_name" 2024-09-20T22:03:56.069 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.933116581Z level=info msg="Executing migration" id="add column group_name" 2024-09-20T22:03:56.069 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.934184292Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-20T22:03:56.069 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.93517156Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-20T22:03:56.069 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.935868419Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-20T22:03:56.070 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.936871952Z level=info msg="Executing migration" id="create team role table" 2024-09-20T22:03:56.070 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.937574523Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-20T22:03:56.070 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.938436394Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-20T22:03:56.070 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.939393747Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-20T22:03:56.070 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.940102222Z level=info msg="Executing migration" id="create user role table" 2024-09-20T22:03:56.070 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.94094388Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-20T22:03:56.070 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.941786814Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-20T22:03:56.070 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.942646047Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-20T22:03:56.070 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.943481428Z level=info msg="Executing migration" id="create builtin role table" 2024-09-20T22:03:56.070 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.944478299Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-20T22:03:56.070 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.945212035Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-20T22:03:56.070 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.946200241Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-20T22:03:56.070 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.947481721Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-20T22:03:56.071 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.948500214Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-20T22:03:56.071 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.949194954Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-20T22:03:56.071 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.950196252Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-20T22:03:56.071 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.950915965Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-20T22:03:56.071 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.951812279Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-20T22:03:56.071 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.952694934Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-20T22:03:56.071 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.953904239Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-20T22:03:56.071 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.954974561Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-20T22:03:56.071 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.955629498Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-20T22:03:56.071 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.956481206Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-20T22:03:56.071 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.957360097Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-20T22:03:56.071 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.958228099Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-20T22:03:56.071 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.9592376Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-20T22:03:56.071 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.962159969Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-20T22:03:56.072 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.965134949Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-20T22:03:56.072 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.967995101Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-20T22:03:56.072 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.970801422Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-20T22:03:56.072 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.973717029Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-20T22:03:56.072 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.976516179Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-20T22:03:56.072 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.979329128Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-20T22:03:56.072 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.982332142Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-20T22:03:56.072 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.985198587Z level=info msg="Executing migration" id="add correlation config column" 2024-09-20T22:03:56.072 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.98802429Z level=info msg="Executing migration" id="create entity_events table" 2024-09-20T22:03:56.072 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.991037292Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-20T22:03:56.072 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.993991605Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-20T22:03:56.072 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.994197632Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-20T22:03:56.072 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.996611323Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-20T22:03:56.072 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.996728112Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-20T22:03:56.073 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:55 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:55.999493495Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-20T22:03:56.073 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.002452964Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-20T22:03:56.073 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.005348486Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-20T22:03:56.073 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.008265981Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-20T22:03:56.073 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.011069557Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-20T22:03:56.073 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.013794979Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-20T22:03:56.073 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.016573325Z level=info msg="Executing migration" id="Drop public config table" 2024-09-20T22:03:56.073 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.019347796Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-20T22:03:56.073 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.022140492Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-20T22:03:56.073 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.025004476Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-20T22:03:56.073 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.02775585Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-20T22:03:56.073 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.030608647Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-20T22:03:56.073 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.035293341Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-20T22:03:56.073 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.037954127Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-20T22:03:56.073 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.040688108Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-20T22:03:56.074 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.043545429Z level=info msg="Executing migration" id="add share column" 2024-09-20T22:03:56.074 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.046238115Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-20T22:03:56.074 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.048855119Z level=info msg="Executing migration" id="create file table" 2024-09-20T22:03:56.074 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.051661565Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-20T22:03:56.074 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.05445656Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-20T22:03:56.074 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.057168702Z level=info msg="Executing migration" id="create file_meta table" 2024-09-20T22:03:56.074 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.059870135Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-20T22:03:56.074 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.060710247Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-20T22:03:56.074 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.061569953Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-20T22:03:56.074 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.062406001Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-20T22:03:56.074 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.063221248Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-20T22:03:56.074 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.064077293Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-20T22:03:56.135 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:03:56.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:56.158 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:56.158 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:03:56.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:03:56.275 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:56.276 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:56.288 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:56.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:56.289 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:03:56.318 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.065304583Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-20T22:03:56.318 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.066132018Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-20T22:03:56.318 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.066979236Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-20T22:03:56.318 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.067804635Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-20T22:03:56.318 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.068655758Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-20T22:03:56.318 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.069514731Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-20T22:03:56.318 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.071058218Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-20T22:03:56.318 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.072466269Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-20T22:03:56.319 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.073305701Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-20T22:03:56.319 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.086378404Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-20T22:03:56.319 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.087217769Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-20T22:03:56.319 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.088098567Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-20T22:03:56.319 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.092482181Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-20T22:03:56.319 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.093435088Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-20T22:03:56.319 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.094195325Z level=info msg="Executing migration" id="create folder table" 2024-09-20T22:03:56.319 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.095024405Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-20T22:03:56.319 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.095909696Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-20T22:03:56.319 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.096889148Z level=info msg="Executing migration" id="Update folder title length" 2024-09-20T22:03:56.319 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.097687974Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-20T22:03:56.319 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=migrator t=2024-09-20T22:03:56.098638859Z level=info msg="migrations completed" performed=485 skipped=0 duration=698.144099ms 2024-09-20T22:03:56.319 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=sqlstore t=2024-09-20T22:03:56.099114153Z level=info msg="Created default organization" 2024-09-20T22:03:56.319 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=plugin.finder t=2024-09-20T22:03:56.132537145Z level=warn msg="Skipping finding plugins as directory does not exist" path=/usr/share/grafana/plugins-bundled 2024-09-20T22:03:56.320 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=secrets t=2024-09-20T22:03:56.132637698Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-20T22:03:56.320 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=query_data t=2024-09-20T22:03:56.134854256Z level=info msg="Query Service initialization" 2024-09-20T22:03:56.320 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=live.push_http t=2024-09-20T22:03:56.138857426Z level=info msg="Live Push Gateway initialization" 2024-09-20T22:03:56.389 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:03:56.412 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:56.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:56.413 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:03:56.428 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:56 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3944350921' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T22:03:56.428 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:56 smithi060 ceph-mon[29694]: osdmap e83: 8 total, 8 up, 8 in 2024-09-20T22:03:56.428 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:56 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:56.428 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:56 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:56.428 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:56 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:56.428 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:56 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:56.428 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:56 smithi060 ceph-mon[29694]: osdmap e84: 8 total, 8 up, 8 in 2024-09-20T22:03:56.428 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:56 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3944350921' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T22:03:56.428 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:56 smithi060 ceph-mon[40290]: osdmap e83: 8 total, 8 up, 8 in 2024-09-20T22:03:56.429 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:56 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:56.429 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:56 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:56.429 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:56 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:56.429 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:56 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:56.429 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:56 smithi060 ceph-mon[40290]: osdmap e84: 8 total, 8 up, 8 in 2024-09-20T22:03:56.514 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:03:56.536 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:56.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:56.537 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:03:56.571 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:03:56.637 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:03:56.661 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:56.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:56.662 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:03:56.763 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:03:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3944350921' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_0", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T22:03:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-mon[33511]: osdmap e83: 8 total, 8 up, 8 in 2024-09-20T22:03:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:03:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-mon[33511]: osdmap e84: 8 total, 8 up, 8 in 2024-09-20T22:03:56.791 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:56.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:56.793 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:03:56.893 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:03:56.918 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:56.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:56.919 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:03:57.020 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:03:57.044 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:57.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:57.045 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:03:57.146 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:03:57.169 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:57.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:57.170 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:03:57.191 DEBUG:teuthology.orchestra.run.smithi060:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_0 bench 90 write' 2024-09-20T22:03:57.192 INFO:tasks.thrashosds.thrasher:Shrinking pool .mgr 2024-09-20T22:03:57.192 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool .mgr by 10 min 8 2024-09-20T22:03:57.192 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:03:57.194 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-20T22:03:57.194 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-20T22:03:57.263 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=infra.usagestats.collector t=2024-09-20T22:03:56.778061146Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-20T22:03:57.264 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=provisioning.datasources t=2024-09-20T22:03:56.779038398Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-20T22:03:57.264 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=provisioning.datasources t=2024-09-20T22:03:56.790340724Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-20T22:03:57.264 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=provisioning.alerting t=2024-09-20T22:03:56.801220898Z level=info msg="starting to provision alerting" 2024-09-20T22:03:57.264 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=provisioning.alerting t=2024-09-20T22:03:56.801231458Z level=info msg="finished to provision alerting" 2024-09-20T22:03:57.264 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=ngalert.state.manager t=2024-09-20T22:03:56.80130109Z level=info msg="Warming state cache for startup" 2024-09-20T22:03:57.264 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=grafanaStorageLogger t=2024-09-20T22:03:56.801454083Z level=info msg="storage starting" 2024-09-20T22:03:57.264 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=http.server t=2024-09-20T22:03:56.803118715Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-20T22:03:57.264 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=ngalert.state.manager t=2024-09-20T22:03:56.808350937Z level=info msg="State cache has been initialized" states=0 duration=7.049061ms 2024-09-20T22:03:57.264 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=ngalert.multiorg.alertmanager t=2024-09-20T22:03:56.808399211Z level=info msg="starting MultiOrg Alertmanager" 2024-09-20T22:03:57.264 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=ticker t=2024-09-20T22:03:56.80841309Z level=info msg=starting first_tick=2024-09-20T22:04:00Z 2024-09-20T22:03:57.264 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=sqlstore.transactions t=2024-09-20T22:03:56.811880504Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-20T22:03:57.264 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=sqlstore.transactions t=2024-09-20T22:03:56.823511415Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-20T22:03:57.264 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=sqlstore.transactions t=2024-09-20T22:03:56.834395525Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=2 code="database is locked" 2024-09-20T22:03:57.265 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=sqlstore.transactions t=2024-09-20T22:03:56.844766282Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=3 code="database is locked" 2024-09-20T22:03:57.265 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:03:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=sqlstore.transactions t=2024-09-20T22:03:56.855751064Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=4 code="database is locked" 2024-09-20T22:03:57.268 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:hints = 1 2024-09-20T22:03:57.268 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-20T22:03:57.269 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Object prefix: benchmark_data_smithi060_78800 2024-09-20T22:03:57.269 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:03:57.269 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 0 0 0 0 0 0 - 0 2024-09-20T22:03:57.270 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:03:57.301 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:57.436 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:03:57.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:57.437 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:03:57.537 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:03:57.566 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:57.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:57.567 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:03:57.578 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:57 smithi060 ceph-mon[29694]: Deploying daemon node-exporter.a on smithi060 2024-09-20T22:03:57.578 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:57 smithi060 ceph-mon[29694]: pgmap v30: 148 pgs: 6 unknown, 142 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-20T22:03:57.578 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:57 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:03:57.578 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:57 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:03:57.578 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:57 smithi060 ceph-mon[29694]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T22:03:57.578 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:57 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2922928878' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:03:57.579 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:57 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:03:57.579 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:57 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2922928878' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:03:57.579 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:57 smithi060 ceph-mon[29694]: osdmap e85: 8 total, 8 up, 8 in 2024-09-20T22:03:57.579 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:57 smithi060 ceph-mon[40290]: Deploying daemon node-exporter.a on smithi060 2024-09-20T22:03:57.579 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:57 smithi060 ceph-mon[40290]: pgmap v30: 148 pgs: 6 unknown, 142 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-20T22:03:57.579 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:57 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:03:57.579 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:57 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:03:57.579 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:57 smithi060 ceph-mon[40290]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T22:03:57.579 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:57 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2922928878' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:03:57.579 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:57 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:03:57.579 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:57 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2922928878' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:03:57.579 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:57 smithi060 ceph-mon[40290]: osdmap e85: 8 total, 8 up, 8 in 2024-09-20T22:03:57.607 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:03:57.607 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:03:57.607 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:03:57.607 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:03:57.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.607 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:645 7.9 7:916b61b3:::benchmark_data_smithi060_78800_object644:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.607 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.532400+0000", 2024-09-20T22:03:57.607 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.068865544000000001, 2024-09-20T22:03:57.607 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0046973609999999997, 2024-09-20T22:03:57.607 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.608 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.608 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.608 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.608 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.608 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.608 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 645 2024-09-20T22:03:57.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.609 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.609 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.609 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.532400+0000", 2024-09-20T22:03:57.609 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.609 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.609 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.532400+0000", 2024-09-20T22:03:57.609 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.609 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.610 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.532401+0000", 2024-09-20T22:03:57.610 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2759999999999999e-06 2024-09-20T22:03:57.610 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.610 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.610 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.610 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.532446+0000", 2024-09-20T22:03:57.610 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4613000000000001e-05 2024-09-20T22:03:57.610 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.611 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.611 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.611 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.532458+0000", 2024-09-20T22:03:57.611 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2362e-05 2024-09-20T22:03:57.611 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.611 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.611 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.611 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.532465+0000", 2024-09-20T22:03:57.611 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2370000000000003e-06 2024-09-20T22:03:57.611 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.611 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.611 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.612 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.532535+0000", 2024-09-20T22:03:57.612 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9809999999999997e-05 2024-09-20T22:03:57.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.612 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.612 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.612 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.532638+0000", 2024-09-20T22:03:57.612 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010256599999999999 2024-09-20T22:03:57.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.612 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.612 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:03:57.612 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.532828+0000", 2024-09-20T22:03:57.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000190411 2024-09-20T22:03:57.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.613 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.613 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.613 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.533574+0000", 2024-09-20T22:03:57.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00074627500000000002 2024-09-20T22:03:57.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.613 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.613 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.613 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.535270+0000", 2024-09-20T22:03:57.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001695228 2024-09-20T22:03:57.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.614 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.614 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.614 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.537007+0000", 2024-09-20T22:03:57.614 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017369740000000001 2024-09-20T22:03:57.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.614 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.614 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.614 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.537057+0000", 2024-09-20T22:03:57.614 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0460999999999997e-05 2024-09-20T22:03:57.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.614 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.615 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.615 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.537097+0000", 2024-09-20T22:03:57.615 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0148000000000003e-05 2024-09-20T22:03:57.615 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.615 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.615 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.615 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.615 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.615 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:670 7.4 7:20e44632:::benchmark_data_smithi060_78800_object669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.615 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.543043+0000", 2024-09-20T22:03:57.615 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.058222888, 2024-09-20T22:03:57.615 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0048063009999999998, 2024-09-20T22:03:57.615 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.616 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.616 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.616 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.616 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.616 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.616 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 670 2024-09-20T22:03:57.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.616 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.616 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.616 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.616 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.543043+0000", 2024-09-20T22:03:57.616 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.616 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.617 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.543043+0000", 2024-09-20T22:03:57.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.617 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.617 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.543044+0000", 2024-09-20T22:03:57.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3289999999999999e-06 2024-09-20T22:03:57.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.617 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.617 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.543070+0000", 2024-09-20T22:03:57.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5731000000000001e-05 2024-09-20T22:03:57.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.617 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.618 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.618 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.543148+0000", 2024-09-20T22:03:57.618 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7997000000000006e-05 2024-09-20T22:03:57.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.618 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.618 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.618 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.543158+0000", 2024-09-20T22:03:57.618 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8789999999999998e-06 2024-09-20T22:03:57.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.618 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.618 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.618 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.543214+0000", 2024-09-20T22:03:57.618 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6881e-05 2024-09-20T22:03:57.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.619 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.619 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.619 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.543351+0000", 2024-09-20T22:03:57.619 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013664999999999999 2024-09-20T22:03:57.619 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.619 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.619 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:03:57.619 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.543483+0000", 2024-09-20T22:03:57.619 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013244900000000001 2024-09-20T22:03:57.619 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.619 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.619 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.619 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.543971+0000", 2024-09-20T22:03:57.619 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00048750099999999999 2024-09-20T22:03:57.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.620 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.620 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.545110+0000", 2024-09-20T22:03:57.620 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0011394860000000001 2024-09-20T22:03:57.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.620 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.620 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.547769+0000", 2024-09-20T22:03:57.620 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0026582440000000001 2024-09-20T22:03:57.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.620 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.620 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.547814+0000", 2024-09-20T22:03:57.621 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5135000000000001e-05 2024-09-20T22:03:57.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.621 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.621 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.621 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.547849+0000", 2024-09-20T22:03:57.621 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5018999999999998e-05 2024-09-20T22:03:57.621 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.621 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.621 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.621 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.621 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:686 7.f 7:f71aa524:::benchmark_data_smithi060_78800_object685:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.621 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.550370+0000", 2024-09-20T22:03:57.622 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.050895748999999997, 2024-09-20T22:03:57.622 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0080743059999999998, 2024-09-20T22:03:57.622 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.622 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.622 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.622 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.622 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.622 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.622 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 686 2024-09-20T22:03:57.622 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.622 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.622 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.622 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.622 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.550370+0000", 2024-09-20T22:03:57.623 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.623 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.623 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.623 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.550370+0000", 2024-09-20T22:03:57.623 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.623 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.623 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.623 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.550371+0000", 2024-09-20T22:03:57.623 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.587e-06 2024-09-20T22:03:57.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.623 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.623 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.624 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.550397+0000", 2024-09-20T22:03:57.624 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5653e-05 2024-09-20T22:03:57.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.624 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.624 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.624 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.550412+0000", 2024-09-20T22:03:57.624 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5350999999999999e-05 2024-09-20T22:03:57.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.624 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.624 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.624 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.550420+0000", 2024-09-20T22:03:57.624 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0560000000000005e-06 2024-09-20T22:03:57.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.624 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.625 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.625 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.550481+0000", 2024-09-20T22:03:57.625 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0475e-05 2024-09-20T22:03:57.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.625 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.625 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.551538+0000", 2024-09-20T22:03:57.625 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00105682 2024-09-20T22:03:57.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.625 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:03:57.625 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.551743+0000", 2024-09-20T22:03:57.625 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020574699999999999 2024-09-20T22:03:57.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.626 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.626 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.555057+0000", 2024-09-20T22:03:57.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0033132259999999998 2024-09-20T22:03:57.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.626 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.626 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.555068+0000", 2024-09-20T22:03:57.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.15e-05 2024-09-20T22:03:57.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.626 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.626 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.558357+0000", 2024-09-20T22:03:57.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0032884540000000001 2024-09-20T22:03:57.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.627 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.558406+0000", 2024-09-20T22:03:57.627 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9601000000000002e-05 2024-09-20T22:03:57.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.627 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.558444+0000", 2024-09-20T22:03:57.627 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7836000000000002e-05 2024-09-20T22:03:57.627 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.627 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.627 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.628 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:687 7.6 7:6d7ce63f:::benchmark_data_smithi060_78800_object686:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.628 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.550680+0000", 2024-09-20T22:03:57.628 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.050585390000000001, 2024-09-20T22:03:57.628 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0065512859999999999, 2024-09-20T22:03:57.628 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.628 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.628 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.628 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.628 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.628 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.628 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 687 2024-09-20T22:03:57.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.628 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.629 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.629 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.550680+0000", 2024-09-20T22:03:57.629 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.629 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.629 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.629 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.550680+0000", 2024-09-20T22:03:57.629 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.629 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.629 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.629 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.550681+0000", 2024-09-20T22:03:57.629 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7000000000000003e-07 2024-09-20T22:03:57.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.629 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.630 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.630 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.550703+0000", 2024-09-20T22:03:57.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2337999999999999e-05 2024-09-20T22:03:57.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.630 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.630 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.550726+0000", 2024-09-20T22:03:57.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2892999999999999e-05 2024-09-20T22:03:57.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.630 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.630 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.550733+0000", 2024-09-20T22:03:57.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2350000000000002e-06 2024-09-20T22:03:57.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.550792+0000", 2024-09-20T22:03:57.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8841999999999997e-05 2024-09-20T22:03:57.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.550901+0000", 2024-09-20T22:03:57.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108958 2024-09-20T22:03:57.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:03:57.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.551032+0000", 2024-09-20T22:03:57.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013120800000000001 2024-09-20T22:03:57.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.555934+0000", 2024-09-20T22:03:57.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0049014940000000002 2024-09-20T22:03:57.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.557100+0000", 2024-09-20T22:03:57.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0011665079999999999 2024-09-20T22:03:57.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.557144+0000", 2024-09-20T22:03:57.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3401e-05 2024-09-20T22:03:57.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.633 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.633 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.633 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.557192+0000", 2024-09-20T22:03:57.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8346000000000001e-05 2024-09-20T22:03:57.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.633 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.633 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.633 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.557231+0000", 2024-09-20T22:03:57.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9193000000000002e-05 2024-09-20T22:03:57.633 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.633 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.633 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.633 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.634 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:691 7.f 7:ff3ec615:::benchmark_data_smithi060_78800_object690:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.634 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.551311+0000", 2024-09-20T22:03:57.634 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.049954846999999997, 2024-09-20T22:03:57.634 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0098178020000000005, 2024-09-20T22:03:57.634 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.634 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.634 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.634 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.634 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.634 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.634 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 691 2024-09-20T22:03:57.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.634 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.634 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.635 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.635 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.551311+0000", 2024-09-20T22:03:57.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.635 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.635 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.551311+0000", 2024-09-20T22:03:57.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.551311+0000", 2024-09-20T22:03:57.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5799999999999999e-07 2024-09-20T22:03:57.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.637 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.551339+0000", 2024-09-20T22:03:57.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7410999999999999e-05 2024-09-20T22:03:57.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.638 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.638 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.552016+0000", 2024-09-20T22:03:57.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00067740599999999997 2024-09-20T22:03:57.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.638 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.638 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.552024+0000", 2024-09-20T22:03:57.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8979999999999996e-06 2024-09-20T22:03:57.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.638 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.552643+0000", 2024-09-20T22:03:57.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000619394 2024-09-20T22:03:57.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.552781+0000", 2024-09-20T22:03:57.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013731099999999999 2024-09-20T22:03:57.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:03:57.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.552951+0000", 2024-09-20T22:03:57.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000170373 2024-09-20T22:03:57.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.640 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.640 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.555319+0000", 2024-09-20T22:03:57.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002368298 2024-09-20T22:03:57.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.640 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.640 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.555510+0000", 2024-09-20T22:03:57.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019089500000000001 2024-09-20T22:03:57.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.640 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.641 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.561036+0000", 2024-09-20T22:03:57.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0055255080000000002 2024-09-20T22:03:57.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.641 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.641 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.561090+0000", 2024-09-20T22:03:57.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4042000000000002e-05 2024-09-20T22:03:57.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.641 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.641 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.561128+0000", 2024-09-20T22:03:57.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8608e-05 2024-09-20T22:03:57.641 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.641 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.642 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.642 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:692 7.6 7:6b066ec2:::benchmark_data_smithi060_78800_object691:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.642 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.552044+0000", 2024-09-20T22:03:57.642 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.049221569, 2024-09-20T22:03:57.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0071203890000000004, 2024-09-20T22:03:57.642 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.642 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.642 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.642 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.642 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.642 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.643 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 692 2024-09-20T22:03:57.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.643 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.552044+0000", 2024-09-20T22:03:57.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.552044+0000", 2024-09-20T22:03:57.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.552045+0000", 2024-09-20T22:03:57.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.09e-07 2024-09-20T22:03:57.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.552079+0000", 2024-09-20T22:03:57.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4790999999999999e-05 2024-09-20T22:03:57.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.552089+0000", 2024-09-20T22:03:57.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3100000000000006e-06 2024-09-20T22:03:57.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.645 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.645 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.552095+0000", 2024-09-20T22:03:57.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.516e-06 2024-09-20T22:03:57.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.645 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.645 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.552767+0000", 2024-09-20T22:03:57.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000671252 2024-09-20T22:03:57.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.645 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.645 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.552892+0000", 2024-09-20T22:03:57.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000125735 2024-09-20T22:03:57.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:03:57.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.555216+0000", 2024-09-20T22:03:57.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023237900000000001 2024-09-20T22:03:57.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.557112+0000", 2024-09-20T22:03:57.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0018954849999999999 2024-09-20T22:03:57.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.557327+0000", 2024-09-20T22:03:57.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000215804 2024-09-20T22:03:57.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.559082+0000", 2024-09-20T22:03:57.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017545779999999999 2024-09-20T22:03:57.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.559132+0000", 2024-09-20T22:03:57.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0146999999999999e-05 2024-09-20T22:03:57.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.559164+0000", 2024-09-20T22:03:57.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2172000000000003e-05 2024-09-20T22:03:57.648 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.648 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.648 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.648 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:700 7.f 7:fe78fae2:::benchmark_data_smithi060_78800_object699:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.648 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.552978+0000", 2024-09-20T22:03:57.648 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.048287062999999998, 2024-09-20T22:03:57.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0083473769999999996, 2024-09-20T22:03:57.648 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.648 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.648 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.648 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.648 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.649 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.649 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 700 2024-09-20T22:03:57.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.649 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.552978+0000", 2024-09-20T22:03:57.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.552978+0000", 2024-09-20T22:03:57.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.552979+0000", 2024-09-20T22:03:57.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0809999999999999e-06 2024-09-20T22:03:57.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.553003+0000", 2024-09-20T22:03:57.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3322000000000001e-05 2024-09-20T22:03:57.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.553014+0000", 2024-09-20T22:03:57.651 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0998e-05 2024-09-20T22:03:57.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.651 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:03:57.651 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:03:57.651 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:03:57.651 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:03:57.652 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:03:57.652 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:03:57.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.652 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.553030+0000", 2024-09-20T22:03:57.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6512e-05 2024-09-20T22:03:57.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.554115+0000", 2024-09-20T22:03:57.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001085131 2024-09-20T22:03:57.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.554300+0000", 2024-09-20T22:03:57.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018463999999999999 2024-09-20T22:03:57.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:03:57.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.554444+0000", 2024-09-20T22:03:57.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014353799999999999 2024-09-20T22:03:57.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.555618+0000", 2024-09-20T22:03:57.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0011746370000000001 2024-09-20T22:03:57.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.556163+0000", 2024-09-20T22:03:57.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00054511700000000002 2024-09-20T22:03:57.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.561235+0000", 2024-09-20T22:03:57.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0050719909999999997 2024-09-20T22:03:57.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.561289+0000", 2024-09-20T22:03:57.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3721000000000003e-05 2024-09-20T22:03:57.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.561326+0000", 2024-09-20T22:03:57.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6689000000000001e-05 2024-09-20T22:03:57.655 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.655 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.655 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.656 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:701 7.6 7:69b8cd87:::benchmark_data_smithi060_78800_object700:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.656 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.553405+0000", 2024-09-20T22:03:57.656 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.04786004, 2024-09-20T22:03:57.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0082344940000000002, 2024-09-20T22:03:57.656 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.656 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.656 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.656 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.656 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.656 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.656 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 701 2024-09-20T22:03:57.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.656 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.553405+0000", 2024-09-20T22:03:57.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.553405+0000", 2024-09-20T22:03:57.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.553406+0000", 2024-09-20T22:03:57.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.16e-07 2024-09-20T22:03:57.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.658 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.658 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.553429+0000", 2024-09-20T22:03:57.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2388999999999999e-05 2024-09-20T22:03:57.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.658 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.658 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.553440+0000", 2024-09-20T22:03:57.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1644000000000001e-05 2024-09-20T22:03:57.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.658 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.658 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.553447+0000", 2024-09-20T22:03:57.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1949999999999997e-06 2024-09-20T22:03:57.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.555987+0000", 2024-09-20T22:03:57.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025395550000000002 2024-09-20T22:03:57.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.556134+0000", 2024-09-20T22:03:57.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014669699999999999 2024-09-20T22:03:57.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:03:57.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.556596+0000", 2024-09-20T22:03:57.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00046201499999999997 2024-09-20T22:03:57.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.558943+0000", 2024-09-20T22:03:57.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023472720000000001 2024-09-20T22:03:57.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.559818+0000", 2024-09-20T22:03:57.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00087459199999999999 2024-09-20T22:03:57.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.561494+0000", 2024-09-20T22:03:57.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0016764149999999999 2024-09-20T22:03:57.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.561582+0000", 2024-09-20T22:03:57.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8045000000000002e-05 2024-09-20T22:03:57.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.561640+0000", 2024-09-20T22:03:57.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7859000000000003e-05 2024-09-20T22:03:57.662 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.662 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.662 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.662 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:706 7.f 7:f26850f1:::benchmark_data_smithi060_78800_object705:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.662 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.556541+0000", 2024-09-20T22:03:57.662 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.044724017999999997, 2024-09-20T22:03:57.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0053630839999999997, 2024-09-20T22:03:57.662 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.662 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.662 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.663 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.663 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.663 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.663 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 706 2024-09-20T22:03:57.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.663 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.556541+0000", 2024-09-20T22:03:57.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.556541+0000", 2024-09-20T22:03:57.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.556542+0000", 2024-09-20T22:03:57.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.879999999999999e-07 2024-09-20T22:03:57.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.556567+0000", 2024-09-20T22:03:57.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4192000000000001e-05 2024-09-20T22:03:57.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.665 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.665 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.556579+0000", 2024-09-20T22:03:57.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2648e-05 2024-09-20T22:03:57.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.665 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.665 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.556587+0000", 2024-09-20T22:03:57.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4420000000000001e-06 2024-09-20T22:03:57.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.665 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.665 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.556645+0000", 2024-09-20T22:03:57.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8610000000000003e-05 2024-09-20T22:03:57.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.557303+0000", 2024-09-20T22:03:57.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00065726100000000002 2024-09-20T22:03:57.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:03:57.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.557468+0000", 2024-09-20T22:03:57.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016562900000000001 2024-09-20T22:03:57.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.558980+0000", 2024-09-20T22:03:57.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0015118320000000001 2024-09-20T22:03:57.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.560588+0000", 2024-09-20T22:03:57.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0016077190000000001 2024-09-20T22:03:57.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.561787+0000", 2024-09-20T22:03:57.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0011989609999999999 2024-09-20T22:03:57.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.561841+0000", 2024-09-20T22:03:57.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3968999999999998e-05 2024-09-20T22:03:57.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.561905+0000", 2024-09-20T22:03:57.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3832999999999997e-05 2024-09-20T22:03:57.668 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.668 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.668 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.668 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:711 7.9 7:9d8c5a85:::benchmark_data_smithi060_78800_object710:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.668 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.557408+0000", 2024-09-20T22:03:57.668 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.043857697000000001, 2024-09-20T22:03:57.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005223793, 2024-09-20T22:03:57.668 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.669 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.669 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.669 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.669 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.669 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.669 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 711 2024-09-20T22:03:57.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.669 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.557408+0000", 2024-09-20T22:03:57.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.557408+0000", 2024-09-20T22:03:57.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.557408+0000", 2024-09-20T22:03:57.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1200000000000002e-07 2024-09-20T22:03:57.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.557436+0000", 2024-09-20T22:03:57.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7107e-05 2024-09-20T22:03:57.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.557448+0000", 2024-09-20T22:03:57.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1960999999999999e-05 2024-09-20T22:03:57.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.557455+0000", 2024-09-20T22:03:57.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6420000000000007e-06 2024-09-20T22:03:57.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.557539+0000", 2024-09-20T22:03:57.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4006999999999997e-05 2024-09-20T22:03:57.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.557647+0000", 2024-09-20T22:03:57.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108058 2024-09-20T22:03:57.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:03:57.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.557790+0000", 2024-09-20T22:03:57.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014307400000000001 2024-09-20T22:03:57.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.558838+0000", 2024-09-20T22:03:57.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001048031 2024-09-20T22:03:57.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.560571+0000", 2024-09-20T22:03:57.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001733053 2024-09-20T22:03:57.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.562522+0000", 2024-09-20T22:03:57.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019506689999999999 2024-09-20T22:03:57.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.562589+0000", 2024-09-20T22:03:57.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7199999999999994e-05 2024-09-20T22:03:57.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.562632+0000", 2024-09-20T22:03:57.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2279e-05 2024-09-20T22:03:57.674 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.674 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.674 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.675 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:729 7.f 7:f178ce2d:::benchmark_data_smithi060_78800_object728:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.675 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.564962+0000", 2024-09-20T22:03:57.675 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.036303794, 2024-09-20T22:03:57.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.006511246, 2024-09-20T22:03:57.675 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.675 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.675 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.675 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.675 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.675 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.675 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 729 2024-09-20T22:03:57.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.675 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.564962+0000", 2024-09-20T22:03:57.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.564962+0000", 2024-09-20T22:03:57.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.564963+0000", 2024-09-20T22:03:57.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1659999999999999e-06 2024-09-20T22:03:57.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.564987+0000", 2024-09-20T22:03:57.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4601e-05 2024-09-20T22:03:57.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.565003+0000", 2024-09-20T22:03:57.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5721000000000001e-05 2024-09-20T22:03:57.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.565011+0000", 2024-09-20T22:03:57.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9039999999999992e-06 2024-09-20T22:03:57.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.565075+0000", 2024-09-20T22:03:57.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4338999999999995e-05 2024-09-20T22:03:57.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.565608+0000", 2024-09-20T22:03:57.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00053281400000000005 2024-09-20T22:03:57.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:03:57.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.565762+0000", 2024-09-20T22:03:57.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000154265 2024-09-20T22:03:57.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.679 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.567407+0000", 2024-09-20T22:03:57.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00164477 2024-09-20T22:03:57.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.679 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.568832+0000", 2024-09-20T22:03:57.680 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:03:57.680 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:03:57.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001424579 2024-09-20T22:03:57.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.680 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.571385+0000", 2024-09-20T22:03:57.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025530819999999999 2024-09-20T22:03:57.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.571437+0000", 2024-09-20T22:03:57.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1792999999999997e-05 2024-09-20T22:03:57.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.571473+0000", 2024-09-20T22:03:57.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6211999999999998e-05 2024-09-20T22:03:57.681 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.681 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.682 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.682 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:738 7.6 7:64561635:::benchmark_data_smithi060_78800_object737:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.682 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.567174+0000", 2024-09-20T22:03:57.682 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.034091149000000001, 2024-09-20T22:03:57.682 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0049035609999999999, 2024-09-20T22:03:57.682 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.682 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.682 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.682 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.682 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.682 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.683 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 738 2024-09-20T22:03:57.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.683 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.567174+0000", 2024-09-20T22:03:57.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.567174+0000", 2024-09-20T22:03:57.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.567176+0000", 2024-09-20T22:03:57.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.578e-06 2024-09-20T22:03:57.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.567200+0000", 2024-09-20T22:03:57.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4315e-05 2024-09-20T22:03:57.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.567240+0000", 2024-09-20T22:03:57.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0123000000000002e-05 2024-09-20T22:03:57.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.567337+0000", 2024-09-20T22:03:57.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7048999999999999e-05 2024-09-20T22:03:57.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.568126+0000", 2024-09-20T22:03:57.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00078847499999999996 2024-09-20T22:03:57.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.568227+0000", 2024-09-20T22:03:57.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100697 2024-09-20T22:03:57.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:03:57.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.568355+0000", 2024-09-20T22:03:57.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012808900000000001 2024-09-20T22:03:57.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.569585+0000", 2024-09-20T22:03:57.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001230414 2024-09-20T22:03:57.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.569947+0000", 2024-09-20T22:03:57.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00036207699999999998 2024-09-20T22:03:57.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.571976+0000", 2024-09-20T22:03:57.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020285680000000001 2024-09-20T22:03:57.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.572032+0000", 2024-09-20T22:03:57.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6660999999999998e-05 2024-09-20T22:03:57.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.572078+0000", 2024-09-20T22:03:57.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5515e-05 2024-09-20T22:03:57.688 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.688 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.688 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.688 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:760 7.f 7:fc3b144c:::benchmark_data_smithi060_78800_object759:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.688 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.576156+0000", 2024-09-20T22:03:57.688 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.025109915999999999, 2024-09-20T22:03:57.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0090028499999999997, 2024-09-20T22:03:57.688 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.688 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.689 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.689 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.689 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.689 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.689 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 760 2024-09-20T22:03:57.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.689 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.576156+0000", 2024-09-20T22:03:57.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.576156+0000", 2024-09-20T22:03:57.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.576157+0000", 2024-09-20T22:03:57.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.372e-06 2024-09-20T22:03:57.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.576179+0000", 2024-09-20T22:03:57.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1678000000000001e-05 2024-09-20T22:03:57.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.576196+0000", 2024-09-20T22:03:57.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7164e-05 2024-09-20T22:03:57.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.576203+0000", 2024-09-20T22:03:57.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3080000000000002e-06 2024-09-20T22:03:57.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.576294+0000", 2024-09-20T22:03:57.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0551999999999996e-05 2024-09-20T22:03:57.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.576414+0000", 2024-09-20T22:03:57.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000120495 2024-09-20T22:03:57.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:03:57.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.576569+0000", 2024-09-20T22:03:57.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015522600000000001 2024-09-20T22:03:57.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.581421+0000", 2024-09-20T22:03:57.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0048519119999999999 2024-09-20T22:03:57.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.582785+0000", 2024-09-20T22:03:57.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0013641840000000001 2024-09-20T22:03:57.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.585063+0000", 2024-09-20T22:03:57.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002277753 2024-09-20T22:03:57.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.585115+0000", 2024-09-20T22:03:57.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2098999999999997e-05 2024-09-20T22:03:57.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.585158+0000", 2024-09-20T22:03:57.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3106999999999999e-05 2024-09-20T22:03:57.694 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.694 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.694 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.694 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:761 7.f 7:f7238e15:::benchmark_data_smithi060_78800_object760:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.694 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.576735+0000", 2024-09-20T22:03:57.695 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024530627999999999, 2024-09-20T22:03:57.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0088473230000000007, 2024-09-20T22:03:57.695 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.695 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.695 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.695 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.695 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.695 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.695 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 761 2024-09-20T22:03:57.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.695 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.576735+0000", 2024-09-20T22:03:57.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.576735+0000", 2024-09-20T22:03:57.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.576736+0000", 2024-09-20T22:03:57.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8700000000000005e-07 2024-09-20T22:03:57.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.576759+0000", 2024-09-20T22:03:57.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3750000000000001e-05 2024-09-20T22:03:57.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.576776+0000", 2024-09-20T22:03:57.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6430999999999999e-05 2024-09-20T22:03:57.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.576784+0000", 2024-09-20T22:03:57.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2700000000000004e-06 2024-09-20T22:03:57.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.576936+0000", 2024-09-20T22:03:57.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015157699999999999 2024-09-20T22:03:57.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577078+0000", 2024-09-20T22:03:57.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014194199999999999 2024-09-20T22:03:57.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:03:57.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.580376+0000", 2024-09-20T22:03:57.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0032988140000000002 2024-09-20T22:03:57.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.582832+0000", 2024-09-20T22:03:57.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024552419999999998 2024-09-20T22:03:57.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.584938+0000", 2024-09-20T22:03:57.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021061309999999998 2024-09-20T22:03:57.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.585197+0000", 2024-09-20T22:03:57.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00025889000000000001 2024-09-20T22:03:57.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.585542+0000", 2024-09-20T22:03:57.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000345318 2024-09-20T22:03:57.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.700 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.585582+0000", 2024-09-20T22:03:57.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0170999999999999e-05 2024-09-20T22:03:57.700 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.700 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.700 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.700 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:763 7.7 e85/80 7:ecfa2f70:::benchmark_data_smithi060_78800_object762:head v 85'38, mlcod=85'36)", 2024-09-20T22:03:57.700 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.577146+0000", 2024-09-20T22:03:57.700 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.024119208999999999, 2024-09-20T22:03:57.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0056769460000000004, 2024-09-20T22:03:57.701 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.701 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.701 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.701 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577146+0000", 2024-09-20T22:03:57.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577146+0000", 2024-09-20T22:03:57.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577147+0000", 2024-09-20T22:03:57.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8700000000000001e-07 2024-09-20T22:03:57.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577177+0000", 2024-09-20T22:03:57.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0803000000000003e-05 2024-09-20T22:03:57.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577186+0000", 2024-09-20T22:03:57.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3340000000000002e-06 2024-09-20T22:03:57.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577191+0000", 2024-09-20T22:03:57.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.682e-06 2024-09-20T22:03:57.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.578566+0000", 2024-09-20T22:03:57.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00137451 2024-09-20T22:03:57.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.578585+0000", 2024-09-20T22:03:57.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8819000000000001e-05 2024-09-20T22:03:57.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.582787+0000", 2024-09-20T22:03:57.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0042020360000000001 2024-09-20T22:03:57.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.582823+0000", 2024-09-20T22:03:57.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6375000000000003e-05 2024-09-20T22:03:57.704 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.704 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.704 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.705 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:764 7.f 7:fa49c96f:::benchmark_data_smithi060_78800_object763:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.705 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.577386+0000", 2024-09-20T22:03:57.705 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023879613000000001, 2024-09-20T22:03:57.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00857634, 2024-09-20T22:03:57.705 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.705 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.705 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.705 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.705 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.705 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.705 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 764 2024-09-20T22:03:57.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.706 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577386+0000", 2024-09-20T22:03:57.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577386+0000", 2024-09-20T22:03:57.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577387+0000", 2024-09-20T22:03:57.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2999999999999999e-07 2024-09-20T22:03:57.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577410+0000", 2024-09-20T22:03:57.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3045999999999999e-05 2024-09-20T22:03:57.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.707 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:03:57.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577421+0000", 2024-09-20T22:03:57.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.101e-05 2024-09-20T22:03:57.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577428+0000", 2024-09-20T22:03:57.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.323e-06 2024-09-20T22:03:57.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.581460+0000", 2024-09-20T22:03:57.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.004031789 2024-09-20T22:03:57.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.581566+0000", 2024-09-20T22:03:57.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000106371 2024-09-20T22:03:57.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:03:57.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.581697+0000", 2024-09-20T22:03:57.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013048700000000001 2024-09-20T22:03:57.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.582836+0000", 2024-09-20T22:03:57.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001139721 2024-09-20T22:03:57.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.584982+0000", 2024-09-20T22:03:57.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021456240000000001 2024-09-20T22:03:57.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.585877+0000", 2024-09-20T22:03:57.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00089511899999999999 2024-09-20T22:03:57.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.585925+0000", 2024-09-20T22:03:57.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8179000000000001e-05 2024-09-20T22:03:57.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.585962+0000", 2024-09-20T22:03:57.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6841000000000002e-05 2024-09-20T22:03:57.712 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.712 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.712 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.713 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:771 7.f 7:f2d9efc3:::benchmark_data_smithi060_78800_object770:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.713 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.577440+0000", 2024-09-20T22:03:57.713 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.023825025, 2024-09-20T22:03:57.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.011505869, 2024-09-20T22:03:57.713 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.713 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.713 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.713 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.713 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.713 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.713 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 771 2024-09-20T22:03:57.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.713 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577440+0000", 2024-09-20T22:03:57.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577440+0000", 2024-09-20T22:03:57.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577441+0000", 2024-09-20T22:03:57.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3399999999999999e-07 2024-09-20T22:03:57.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577760+0000", 2024-09-20T22:03:57.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00031904600000000002 2024-09-20T22:03:57.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577772+0000", 2024-09-20T22:03:57.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1715e-05 2024-09-20T22:03:57.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.577779+0000", 2024-09-20T22:03:57.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9959999999999996e-06 2024-09-20T22:03:57.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.582875+0000", 2024-09-20T22:03:57.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0050962189999999999 2024-09-20T22:03:57.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.583009+0000", 2024-09-20T22:03:57.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013363399999999999 2024-09-20T22:03:57.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:03:57.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.583216+0000", 2024-09-20T22:03:57.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020703400000000001 2024-09-20T22:03:57.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.585596+0000", 2024-09-20T22:03:57.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023805779999999999 2024-09-20T22:03:57.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.586138+0000", 2024-09-20T22:03:57.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00054206299999999996 2024-09-20T22:03:57.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.588845+0000", 2024-09-20T22:03:57.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002706515 2024-09-20T22:03:57.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.588894+0000", 2024-09-20T22:03:57.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9092999999999999e-05 2024-09-20T22:03:57.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.588946+0000", 2024-09-20T22:03:57.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2441999999999997e-05 2024-09-20T22:03:57.718 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.719 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.719 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.719 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:781 7.6 7:697704a7:::benchmark_data_smithi060_78800_object780:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.719 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.583485+0000", 2024-09-20T22:03:57.719 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.017780609999999999, 2024-09-20T22:03:57.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0058400309999999999, 2024-09-20T22:03:57.719 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.719 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.719 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.719 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.719 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.719 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.720 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 781 2024-09-20T22:03:57.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.720 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.583485+0000", 2024-09-20T22:03:57.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.583485+0000", 2024-09-20T22:03:57.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.583486+0000", 2024-09-20T22:03:57.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.268e-06 2024-09-20T22:03:57.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.583504+0000", 2024-09-20T22:03:57.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8156999999999999e-05 2024-09-20T22:03:57.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.583514+0000", 2024-09-20T22:03:57.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6350000000000001e-06 2024-09-20T22:03:57.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.583519+0000", 2024-09-20T22:03:57.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1510000000000003e-06 2024-09-20T22:03:57.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.583573+0000", 2024-09-20T22:03:57.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3662999999999998e-05 2024-09-20T22:03:57.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.723 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.723 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:03:57.723 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:03:57.723 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.583645+0000", 2024-09-20T22:03:57.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2372000000000004e-05 2024-09-20T22:03:57.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:03:57.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.583736+0000", 2024-09-20T22:03:57.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1156999999999995e-05 2024-09-20T22:03:57.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.585614+0000", 2024-09-20T22:03:57.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0018782110000000001 2024-09-20T22:03:57.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.588931+0000", 2024-09-20T22:03:57.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0033167190000000001 2024-09-20T22:03:57.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.589240+0000", 2024-09-20T22:03:57.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030852599999999999 2024-09-20T22:03:57.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.589288+0000", 2024-09-20T22:03:57.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8565e-05 2024-09-20T22:03:57.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.589325+0000", 2024-09-20T22:03:57.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6606999999999997e-05 2024-09-20T22:03:57.726 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.726 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.726 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.726 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:805 7.6 7:6e8654ab:::benchmark_data_smithi060_78800_object804:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.726 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.593216+0000", 2024-09-20T22:03:57.727 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0080490460000000007, 2024-09-20T22:03:57.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0055415150000000003, 2024-09-20T22:03:57.727 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.727 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.727 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.727 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.727 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.727 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.727 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 805 2024-09-20T22:03:57.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.727 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.593216+0000", 2024-09-20T22:03:57.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.593216+0000", 2024-09-20T22:03:57.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.593218+0000", 2024-09-20T22:03:57.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.279e-06 2024-09-20T22:03:57.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.593249+0000", 2024-09-20T22:03:57.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0975000000000001e-05 2024-09-20T22:03:57.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.593260+0000", 2024-09-20T22:03:57.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1313e-05 2024-09-20T22:03:57.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.593267+0000", 2024-09-20T22:03:57.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2080000000000003e-06 2024-09-20T22:03:57.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.593380+0000", 2024-09-20T22:03:57.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112341 2024-09-20T22:03:57.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.593497+0000", 2024-09-20T22:03:57.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011702200000000001 2024-09-20T22:03:57.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:03:57.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.593622+0000", 2024-09-20T22:03:57.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012591600000000001 2024-09-20T22:03:57.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.594938+0000", 2024-09-20T22:03:57.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0013156120000000001 2024-09-20T22:03:57.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.595561+0000", 2024-09-20T22:03:57.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00062288000000000003 2024-09-20T22:03:57.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.598668+0000", 2024-09-20T22:03:57.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0031067579999999998 2024-09-20T22:03:57.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.732 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.598711+0000", 2024-09-20T22:03:57.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3071000000000001e-05 2024-09-20T22:03:57.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.732 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.598758+0000", 2024-09-20T22:03:57.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7139999999999999e-05 2024-09-20T22:03:57.732 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.732 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.732 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.733 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:807 7.9 7:9eb0bb2a:::benchmark_data_smithi060_78800_object806:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e85)", 2024-09-20T22:03:57.733 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:03:57.593286+0000", 2024-09-20T22:03:57.733 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0079796610000000007, 2024-09-20T22:03:57.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0064044970000000003, 2024-09-20T22:03:57.733 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:03:57.733 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:03:57.733 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:03:57.733 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:03:57.733 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:03:57.733 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:03:57.733 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 807 2024-09-20T22:03:57.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.733 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:03:57.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:03:57.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.593286+0000", 2024-09-20T22:03:57.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:03:57.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.593286+0000", 2024-09-20T22:03:57.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:03:57.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:03:57.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.593286+0000", 2024-09-20T22:03:57.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3200000000000005e-07 2024-09-20T22:03:57.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:03:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.593306+0000", 2024-09-20T22:03:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9928e-05 2024-09-20T22:03:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:03:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.593315+0000", 2024-09-20T22:03:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7620000000000001e-06 2024-09-20T22:03:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:03:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.593325+0000", 2024-09-20T22:03:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7869999999999996e-06 2024-09-20T22:03:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:03:57.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.593448+0000", 2024-09-20T22:03:57.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012341199999999999 2024-09-20T22:03:57.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:03:57.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.597801+0000", 2024-09-20T22:03:57.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0043526459999999999 2024-09-20T22:03:57.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:03:57.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.597938+0000", 2024-09-20T22:03:57.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000137453 2024-09-20T22:03:57.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:03:57.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.598526+0000", 2024-09-20T22:03:57.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000587602 2024-09-20T22:03:57.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.599259+0000", 2024-09-20T22:03:57.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00073288800000000001 2024-09-20T22:03:57.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:03:57.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.599602+0000", 2024-09-20T22:03:57.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00034285400000000002 2024-09-20T22:03:57.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:03:57.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.599653+0000", 2024-09-20T22:03:57.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1418000000000001e-05 2024-09-20T22:03:57.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:03:57.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:03:57.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:03:57.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:03:57.599690+0000", 2024-09-20T22:03:57.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7115e-05 2024-09-20T22:03:57.738 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.738 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.739 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.739 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:03:57.739 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:03:57.739 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:03:57.740 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:03:57.740 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:03:57.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:57.741 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:57.741 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:03:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:57 smithi067 ceph-mon[33511]: Deploying daemon node-exporter.a on smithi060 2024-09-20T22:03:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:57 smithi067 ceph-mon[33511]: pgmap v30: 148 pgs: 6 unknown, 142 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail 2024-09-20T22:03:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:57 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:03:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:57 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:03:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:57 smithi067 ceph-mon[33511]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T22:03:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:57 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2922928878' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:03:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:57 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:03:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:57 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2922928878' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:03:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:57 smithi067 ceph-mon[33511]: osdmap e85: 8 total, 8 up, 8 in 2024-09-20T22:03:57.768 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:03:57.768 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:03:57.804 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:03:57.804 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:03:57.841 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:03:57.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:57.884 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:57.884 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:03:57.985 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:03:58.026 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:58.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:58.027 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:03:58.128 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:03:58.169 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:58.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:58.170 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:03:58.269 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 1 16 2902 2886 180.364 180.375 0.00422415 0.00552345 2024-09-20T22:03:58.270 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:03:58.311 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:58.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:58.312 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:03:58.413 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:03:58.454 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:58.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:58.455 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:03:58.556 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:03:58.588 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:58.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:58.589 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:03:58.690 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:03:58.722 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:58.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:58.724 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:03:58.731 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:58 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/236668101' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:03:58.731 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:58 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/236668101' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:03:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:58 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/236668101' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:03:58.824 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:03:58.856 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:58.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:58.857 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:03:58.958 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:03:58.989 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:58.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:58.991 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:03:59.091 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:03:59.122 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:59.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:59.123 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:03:59.224 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:03:59.264 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:59.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:59.266 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:03:59.269 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 2 16 6086 6070 189.673 199 0.0044763 0.00526156 2024-09-20T22:03:59.367 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:03:59.400 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:59.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:59.402 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:03:59.502 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:03:59.533 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:03:59.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:59.535 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:03:59.635 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:03:59.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:59.677 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:59.677 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:03:59.753 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:03:59 smithi060 ceph-mon[29694]: pgmap v33: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:03:59.754 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:03:59 smithi060 ceph-mon[40290]: pgmap v33: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:03:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:03:59 smithi067 ceph-mon[33511]: pgmap v33: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:03:59.778 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:03:59.817 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:59.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:59.818 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:03:59.919 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:03:59.958 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:03:59.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:03:59.959 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:00.060 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:00.099 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:00.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:00.100 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:00.201 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:00.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T22:04:00.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:00.257 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:00.257 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:00.269 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 3 15 8955 8940 186.22 179.375 0.00712383 0.0053613 2024-09-20T22:04:00.358 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:00.408 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:00.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:00.409 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:00.510 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:00.517 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:04:00.537 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:00.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:00.538 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:00.639 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:00.669 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:00.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:00.670 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:00.771 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:00.797 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:00.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:00.798 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:00.899 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:00.941 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:00.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:00.942 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:00 smithi067 ceph-mon[33511]: pgmap v34: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:04:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:00 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:01.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:00 smithi060 ceph-mon[29694]: pgmap v34: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:04:01.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:00 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:01.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:00 smithi060 ceph-mon[40290]: pgmap v34: 148 pgs: 148 active+clean; 583 KiB data, 240 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:04:01.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:00 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:01.042 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:01.086 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:01.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:01.087 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:01.188 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:01.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:01.231 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:01.231 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:01.270 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 4 16 11481 11465 179.115 157.812 0.00602169 0.00557204 2024-09-20T22:04:01.332 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:01.361 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:01.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:01.362 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:01.462 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:01.488 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:01.490 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:01.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:01.491 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:01.592 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:01.620 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:01.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:01.621 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:01.722 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:01.752 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:01.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:01.753 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:01.854 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:01.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:01.903 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:01.903 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:02.004 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:02.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:04:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:04:02.051 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:02.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:02.052 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:02.153 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:02.183 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:02.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:02.184 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:02.270 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 5 15 13778 13763 172.014 143.625 0.00525107 0.00580431 2024-09-20T22:04:02.286 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:02.302 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:02.305 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:02.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:02.331 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:02.331 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:02.396 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:02 smithi060 ceph-mon[29694]: osdmap e86: 8 total, 8 up, 8 in 2024-09-20T22:04:02.396 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:02 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:04:02.396 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:02 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:04:02.397 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:02 smithi060 ceph-mon[40290]: osdmap e86: 8 total, 8 up, 8 in 2024-09-20T22:04:02.397 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:02 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:04:02.397 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:02 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:04:02.398 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:02 smithi060 bash[79017]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-20T22:04:02.432 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:02.473 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:02.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:02.474 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:02.578 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:04:02.580 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:02.581 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:04:02.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:02.627 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:02.628 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:02.728 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:02.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:02 smithi067 ceph-mon[33511]: osdmap e86: 8 total, 8 up, 8 in 2024-09-20T22:04:02.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:02 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:04:02.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:02 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:04:02.770 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:02.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:02.771 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:02.872 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:02.904 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:02.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:02.905 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:03.006 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:03.036 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:02 smithi060 bash[79017]: Writing manifest to image destination 2024-09-20T22:04:03.038 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:03.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:03.039 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:03.140 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:03.170 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:03.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:03.172 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:03.270 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 6 16 16246 16230 169.041 154.188 0.0104376 0.00590856 2024-09-20T22:04:03.272 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:03.300 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:03.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:03.301 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:03.402 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:03.433 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:03.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:03.434 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:03.528 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:03 smithi060 ceph-mon[29694]: pgmap v36: 148 pgs: 148 active+clean; 601 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-20T22:04:03.528 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:03 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:04:03.528 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:03 smithi060 ceph-mon[29694]: osdmap e87: 8 total, 8 up, 8 in 2024-09-20T22:04:03.528 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:03 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:04:03.528 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:03 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:04:03.528 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:03 smithi060 podman[79017]: 2024-09-20 22:04:03.18954703 +0000 UTC m=+5.287232144 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-20T22:04:03.528 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:03 smithi060 podman[79017]: 2024-09-20 22:04:03.36854405 +0000 UTC m=+5.466229159 container create f9f089f39405c767f6aa092f807245d65fb1882128788d470b8bbc25c6ca7583 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-20T22:04:03.528 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:03 smithi060 ceph-mon[40290]: pgmap v36: 148 pgs: 148 active+clean; 601 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-20T22:04:03.529 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:03 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:04:03.529 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:03 smithi060 ceph-mon[40290]: osdmap e87: 8 total, 8 up, 8 in 2024-09-20T22:04:03.529 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:03 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:04:03.529 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:03 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:04:03.535 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:03.562 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:04:03.566 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:03.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:03.567 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:03.623 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:04:03.623 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:04:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:16672 7.b 7:da67e19f:::benchmark_data_smithi060_78800_object16671:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:04:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:03.458679+0000", 2024-09-20T22:04:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0013428559999999999, 2024-09-20T22:04:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001380362, 2024-09-20T22:04:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:04:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:04:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:04:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:04:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:04:03.624 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 16672 2024-09-20T22:04:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:04:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:04:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:03.458679+0000", 2024-09-20T22:04:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:04:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:04:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:03.458679+0000", 2024-09-20T22:04:03.625 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:04:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:04:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:03.458681+0000", 2024-09-20T22:04:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.672e-06 2024-09-20T22:04:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:04:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:03.458706+0000", 2024-09-20T22:04:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5057000000000001e-05 2024-09-20T22:04:03.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:04:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:03.458725+0000", 2024-09-20T22:04:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9188999999999999e-05 2024-09-20T22:04:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:04:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:03.458734+0000", 2024-09-20T22:04:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6750000000000008e-06 2024-09-20T22:04:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:03.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:04:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:03.458801+0000", 2024-09-20T22:04:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6748999999999994e-05 2024-09-20T22:04:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:04:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:03.458930+0000", 2024-09-20T22:04:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012898599999999999 2024-09-20T22:04:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 5,6", 2024-09-20T22:04:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:03.459060+0000", 2024-09-20T22:04:03.628 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013026499999999999 2024-09-20T22:04:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:04:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:04:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:04:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:16667 7.0 e88/80)", 2024-09-20T22:04:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:03.459975+0000", 2024-09-20T22:04:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4.6539000000000003e-05, 2024-09-20T22:04:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155351, 2024-09-20T22:04:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:04:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:04:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:04:03.629 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:04:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:04:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:03.459975+0000", 2024-09-20T22:04:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:04:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:04:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:03.459975+0000", 2024-09-20T22:04:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:04:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:03.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:04:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:03.459976+0000", 2024-09-20T22:04:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5899999999999995e-07 2024-09-20T22:04:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:04:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:03.460012+0000", 2024-09-20T22:04:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5494999999999998e-05 2024-09-20T22:04:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:04:03.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:03.460027+0000", 2024-09-20T22:04:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5218999999999999e-05 2024-09-20T22:04:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:04:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:03.460034+0000", 2024-09-20T22:04:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1709999999999996e-06 2024-09-20T22:04:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:03.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:04:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:03.460122+0000", 2024-09-20T22:04:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7750999999999994e-05 2024-09-20T22:04:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:04:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:04:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:04:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:04:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:04:03.633 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 2 2024-09-20T22:04:03.633 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:04:03.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:03.699 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:03.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:03.700 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:03.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:03 smithi067 ceph-mon[33511]: pgmap v36: 148 pgs: 148 active+clean; 601 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-20T22:04:03.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:03 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:04:03.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:03 smithi067 ceph-mon[33511]: osdmap e87: 8 total, 8 up, 8 in 2024-09-20T22:04:03.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:03 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:04:03.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:03 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:04:03.801 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:03.831 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:03.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:03.833 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:03.933 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:03.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:03.981 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:03.981 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:04.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:04.132 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:04.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:04.133 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:04.234 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:04:04.257 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:04.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:04.258 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:04:04.270 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 7 16 18358 18342 163.748 132 0.06012 0.00607288 2024-09-20T22:04:04.358 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:04.401 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:04.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:04.402 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:04.503 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:04.534 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:04.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:04.535 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:04.636 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:04.679 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:04.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:04.680 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:04.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:04 smithi067 ceph-mon[33511]: from='client.24679 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:04:04.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:04 smithi067 ceph-mon[33511]: osdmap e88: 8 total, 8 up, 8 in 2024-09-20T22:04:04.781 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:04:04.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:04 smithi060 ceph-mon[40290]: from='client.24679 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:04:04.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:04 smithi060 ceph-mon[40290]: osdmap e88: 8 total, 8 up, 8 in 2024-09-20T22:04:04.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:04 smithi060 ceph-mon[29694]: from='client.24679 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:04:04.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:04 smithi060 ceph-mon[29694]: osdmap e88: 8 total, 8 up, 8 in 2024-09-20T22:04:04.824 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:04.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:04.825 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:04:04.926 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:04.967 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:04.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:04.968 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:05.069 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:05.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:05.112 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:05.112 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:05.213 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:05.245 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:05.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:05.246 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:05.270 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 8 16 20692 20676 161.513 145.875 0.00984539 0.00618434 2024-09-20T22:04:05.323 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:04:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:04:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:04:05.347 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:04:05.384 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:05.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:05.385 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:04:05.486 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:05.527 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:05.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:05.528 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:05.629 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:05.660 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:05.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:05.661 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:05.762 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:05.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:05 smithi067 ceph-mon[33511]: pgmap v39: 148 pgs: 148 active+clean; 601 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-20T22:04:05.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:05.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:05.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:04:05.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:05 smithi060 ceph-mon[40290]: pgmap v39: 148 pgs: 148 active+clean; 601 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-20T22:04:05.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:05.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:05.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:04:05.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:05 smithi060 ceph-mon[29694]: pgmap v39: 148 pgs: 148 active+clean; 601 MiB data, 1.8 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-20T22:04:05.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:05.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:05.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:04:05.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:05.802 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:05.802 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:05.903 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:05.934 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:05.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:05.935 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:06.036 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:06.079 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:06.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:06.081 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:06.181 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:06.218 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:06.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:06.219 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:06.270 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 9 15 22761 22746 157.941 129.375 0.0077403 0.00632543 2024-09-20T22:04:06.320 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:06.361 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:06.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:06.362 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:06.463 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:06.494 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:06.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:06.496 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:06.596 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:06.621 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:06.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:06.622 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:06.723 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:06.755 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:06.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:06.756 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:06.787 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 podman[79017]: 2024-09-20 22:04:06.420993192 +0000 UTC m=+8.518678309 container init f9f089f39405c767f6aa092f807245d65fb1882128788d470b8bbc25c6ca7583 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-20T22:04:06.787 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 podman[79017]: 2024-09-20 22:04:06.42614671 +0000 UTC m=+8.523831813 container start f9f089f39405c767f6aa092f807245d65fb1882128788d470b8bbc25c6ca7583 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-20T22:04:06.787 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.430Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-20T22:04:06.787 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.430Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-20T22:04:06.787 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.430Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-20T22:04:06.787 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.430Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-20T22:04:06.787 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.430Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-20T22:04:06.787 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-20T22:04:06.787 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-20T22:04:06.787 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=arp 2024-09-20T22:04:06.788 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-20T22:04:06.788 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-20T22:04:06.788 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-20T22:04:06.788 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-20T22:04:06.788 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-20T22:04:06.788 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-20T22:04:06.788 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-20T22:04:06.788 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-20T22:04:06.788 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=edac 2024-09-20T22:04:06.789 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-20T22:04:06.789 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-20T22:04:06.789 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-20T22:04:06.789 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-20T22:04:06.789 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-20T22:04:06.789 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-20T22:04:06.789 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-20T22:04:06.789 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-20T22:04:06.789 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-20T22:04:06.789 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.431Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-20T22:04:06.789 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-20T22:04:06.789 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-20T22:04:06.789 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-20T22:04:06.789 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-20T22:04:06.790 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-20T22:04:06.790 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-20T22:04:06.790 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=os 2024-09-20T22:04:06.790 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-20T22:04:06.790 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-20T22:04:06.790 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-20T22:04:06.790 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-20T22:04:06.790 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-20T22:04:06.790 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-20T22:04:06.790 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-20T22:04:06.790 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=stat 2024-09-20T22:04:06.790 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-20T22:04:06.790 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-20T22:04:06.790 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-20T22:04:06.791 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=time 2024-09-20T22:04:06.791 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-20T22:04:06.791 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=uname 2024-09-20T22:04:06.791 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-20T22:04:06.791 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-20T22:04:06.791 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-20T22:04:06.791 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-20T22:04:06.791 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a[79683]: ts=2024-09-20T22:04:06.432Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-20T22:04:06.857 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:06.884 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:06.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:06.885 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:06.986 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:07.018 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:07.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:07.019 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:07.120 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:07.151 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:07.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:07.152 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:07.253 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:07.270 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 10 16 24706 24690 154.296 121.5 0.00532645 0.00647476 2024-09-20T22:04:07.293 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:07.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:07.294 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:07.395 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:07.435 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:07.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:07.436 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:07.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:04:07.654 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:07.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:07.655 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:04:07.755 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:07.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:07 smithi067 ceph-mon[33511]: pgmap v40: 148 pgs: 148 active+clean; 735 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 122 MiB/s wr, 1.96k op/s 2024-09-20T22:04:07.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:07 smithi060 ceph-mon[40290]: pgmap v40: 148 pgs: 148 active+clean; 735 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 122 MiB/s wr, 1.96k op/s 2024-09-20T22:04:07.786 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:07 smithi060 bash[79017]: f9f089f39405c767f6aa092f807245d65fb1882128788d470b8bbc25c6ca7583 2024-09-20T22:04:07.786 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:04:07 smithi060 systemd[1]: Started Ceph node-exporter.a for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T22:04:07.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:07 smithi060 ceph-mon[29694]: pgmap v40: 148 pgs: 148 active+clean; 735 MiB data, 2.2 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 122 MiB/s wr, 1.96k op/s 2024-09-20T22:04:07.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:07.798 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:07.798 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:07.898 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:07.932 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:07.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:07.933 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:08.034 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:08.073 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:08.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:08.074 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:08.175 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:08.208 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:08.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:08.209 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:08.270 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 11 15 26803 26788 152.188 131.125 0.00776032 0.00656528 2024-09-20T22:04:08.310 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:08.343 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:08.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:08.344 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:08.445 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:04:08.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:08.486 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:08.487 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:04:08.517 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-09-20T22:04:02.318072+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":630463003,"num_objects":9810,"num_object_clones":0,"num_object_copies":29430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9810,"num_whiteouts":0,"num_read":998,"num_read_kb":844,"num_write":10226,"num_write_kb":617565,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10212,"ondisk_log_size":10212,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":1844772,"kb_used_data":1545372,"kb_used_omap":12,"kb_used_meta":299251,"kb_avail":748116444,"statfs":{"total":767960285184,"available":766071238656,"internally_reserved":0,"allocated":1582460928,"data_stored":1578662385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":306433584},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,6,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":629866496,"num_objects":9611,"num_object_clones":0,"num_object_copies":28833,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9611,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":9611,"num_write_kb":615104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.021827"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.309432+0000","last_change":"2024-09-20T22:02:50.111666+0000","last_active":"2024-09-20T22:03:57.309432+0000","last_peered":"2024-09-20T22:03:57.309432+0000","last_clean":"2024-09-20T22:03:57.309432+0000","last_became_active":"2024-09-20T22:02:50.109930+0000","last_became_peered":"2024-09-20T22:02:50.109930+0000","last_unstale":"2024-09-20T22:03:57.309432+0000","last_undegraded":"2024-09-20T22:03:57.309432+0000","last_fullsized":"2024-09-20T22:03:57.309432+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.513745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"71'9","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.306602+0000","last_change":"2024-09-20T22:02:52.277916+0000","last_active":"2024-09-20T22:03:57.306602+0000","last_peered":"2024-09-20T22:03:57.306602+0000","last_clean":"2024-09-20T22:03:57.306602+0000","last_became_active":"2024-09-20T22:02:52.277616+0000","last_became_peered":"2024-09-20T22:02:52.277616+0000","last_unstale":"2024-09-20T22:03:57.306602+0000","last_undegraded":"2024-09-20T22:03:57.306602+0000","last_fullsized":"2024-09-20T22:03:57.306602+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.412488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312814+0000","last_change":"2024-09-20T22:02:54.287938+0000","last_active":"2024-09-20T22:03:57.312814+0000","last_peered":"2024-09-20T22:03:57.312814+0000","last_clean":"2024-09-20T22:03:57.312814+0000","last_became_active":"2024-09-20T22:02:54.287537+0000","last_became_peered":"2024-09-20T22:02:54.287537+0000","last_unstale":"2024-09-20T22:03:57.312814+0000","last_undegraded":"2024-09-20T22:03:57.312814+0000","last_fullsized":"2024-09-20T22:03:57.312814+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.152471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.306581+0000","last_change":"2024-09-20T22:02:56.297551+0000","last_active":"2024-09-20T22:03:57.306581+0000","last_peered":"2024-09-20T22:03:57.306581+0000","last_clean":"2024-09-20T22:03:57.306581+0000","last_became_active":"2024-09-20T22:02:56.297252+0000","last_became_peered":"2024-09-20T22:02:56.297252+0000","last_unstale":"2024-09-20T22:03:57.306581+0000","last_undegraded":"2024-09-20T22:03:57.306581+0000","last_fullsized":"2024-09-20T22:03:57.306581+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.977495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.310330+0000","last_change":"2024-09-20T22:02:56.350387+0000","last_active":"2024-09-20T22:03:57.310330+0000","last_peered":"2024-09-20T22:03:57.310330+0000","last_clean":"2024-09-20T22:03:57.310330+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:03:57.310330+0000","last_undegraded":"2024-09-20T22:03:57.310330+0000","last_fullsized":"2024-09-20T22:03:57.310330+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.462175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.311912+0000","last_change":"2024-09-20T22:02:50.107457+0000","last_active":"2024-09-20T22:03:57.311912+0000","last_peered":"2024-09-20T22:03:57.311912+0000","last_clean":"2024-09-20T22:03:57.311912+0000","last_became_active":"2024-09-20T22:02:50.107154+0000","last_became_peered":"2024-09-20T22:02:50.107154+0000","last_unstale":"2024-09-20T22:03:57.311912+0000","last_undegraded":"2024-09-20T22:03:57.311912+0000","last_fullsized":"2024-09-20T22:03:57.311912+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:11.498961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"71'15","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.309205+0000","last_change":"2024-09-20T22:02:52.282478+0000","last_active":"2024-09-20T22:03:57.309205+0000","last_peered":"2024-09-20T22:03:57.309205+0000","last_clean":"2024-09-20T22:03:57.309205+0000","last_became_active":"2024-09-20T22:02:52.282101+0000","last_became_peered":"2024-09-20T22:02:52.282101+0000","last_unstale":"2024-09-20T22:03:57.309205+0000","last_undegraded":"2024-09-20T22:03:57.309205+0000","last_fullsized":"2024-09-20T22:03:57.309205+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.306424+0000","last_change":"2024-09-20T22:02:54.292666+0000","last_active":"2024-09-20T22:03:57.306424+0000","last_peered":"2024-09-20T22:03:57.306424+0000","last_clean":"2024-09-20T22:03:57.306424+0000","last_became_active":"2024-09-20T22:02:54.292343+0000","last_became_peered":"2024-09-20T22:02:54.292343+0000","last_unstale":"2024-09-20T22:03:57.306424+0000","last_undegraded":"2024-09-20T22:03:57.306424+0000","last_fullsized":"2024-09-20T22:03:57.306424+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.965688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.315182+0000","last_change":"2024-09-20T22:02:50.108309+0000","last_active":"2024-09-20T22:03:57.315182+0000","last_peered":"2024-09-20T22:03:57.315182+0000","last_clean":"2024-09-20T22:03:57.315182+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:03:57.315182+0000","last_undegraded":"2024-09-20T22:03:57.315182+0000","last_fullsized":"2024-09-20T22:03:57.315182+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.039813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"71'9","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.306759+0000","last_change":"2024-09-20T22:02:52.279933+0000","last_active":"2024-09-20T22:03:57.306759+0000","last_peered":"2024-09-20T22:03:57.306759+0000","last_clean":"2024-09-20T22:03:57.306759+0000","last_became_active":"2024-09-20T22:02:52.279526+0000","last_became_peered":"2024-09-20T22:02:52.279526+0000","last_unstale":"2024-09-20T22:03:57.306759+0000","last_undegraded":"2024-09-20T22:03:57.306759+0000","last_fullsized":"2024-09-20T22:03:57.306759+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.714248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.314076+0000","last_change":"2024-09-20T22:02:54.293254+0000","last_active":"2024-09-20T22:03:57.314076+0000","last_peered":"2024-09-20T22:03:57.314076+0000","last_clean":"2024-09-20T22:03:57.314076+0000","last_became_active":"2024-09-20T22:02:54.292911+0000","last_became_peered":"2024-09-20T22:02:54.292911+0000","last_unstale":"2024-09-20T22:03:57.314076+0000","last_undegraded":"2024-09-20T22:03:57.314076+0000","last_fullsized":"2024-09-20T22:03:57.314076+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.908407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.307541+0000","last_change":"2024-09-20T22:02:56.299059+0000","last_active":"2024-09-20T22:03:57.307541+0000","last_peered":"2024-09-20T22:03:57.307541+0000","last_clean":"2024-09-20T22:03:57.307541+0000","last_became_active":"2024-09-20T22:02:56.298660+0000","last_became_peered":"2024-09-20T22:02:56.298660+0000","last_unstale":"2024-09-20T22:03:57.307541+0000","last_undegraded":"2024-09-20T22:03:57.307541+0000","last_fullsized":"2024-09-20T22:03:57.307541+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:13.896683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.314461+0000","last_change":"2024-09-20T22:02:50.106885+0000","last_active":"2024-09-20T22:03:57.314461+0000","last_peered":"2024-09-20T22:03:57.314461+0000","last_clean":"2024-09-20T22:03:57.314461+0000","last_became_active":"2024-09-20T22:02:50.106555+0000","last_became_peered":"2024-09-20T22:02:50.106555+0000","last_unstale":"2024-09-20T22:03:57.314461+0000","last_undegraded":"2024-09-20T22:03:57.314461+0000","last_fullsized":"2024-09-20T22:03:57.314461+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.204880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"71'5","reported_seq":59,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.306648+0000","last_change":"2024-09-20T22:02:52.284347+0000","last_active":"2024-09-20T22:03:57.306648+0000","last_peered":"2024-09-20T22:03:57.306648+0000","last_clean":"2024-09-20T22:03:57.306648+0000","last_became_active":"2024-09-20T22:02:52.283904+0000","last_became_peered":"2024-09-20T22:02:52.283904+0000","last_unstale":"2024-09-20T22:03:57.306648+0000","last_undegraded":"2024-09-20T22:03:57.306648+0000","last_fullsized":"2024-09-20T22:03:57.306648+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.923259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312270+0000","last_change":"2024-09-20T22:02:54.295625+0000","last_active":"2024-09-20T22:03:57.312270+0000","last_peered":"2024-09-20T22:03:57.312270+0000","last_clean":"2024-09-20T22:03:57.312270+0000","last_became_active":"2024-09-20T22:02:54.294405+0000","last_became_peered":"2024-09-20T22:02:54.294405+0000","last_unstale":"2024-09-20T22:03:57.312270+0000","last_undegraded":"2024-09-20T22:03:57.312270+0000","last_fullsized":"2024-09-20T22:03:57.312270+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.674273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.313678+0000","last_change":"2024-09-20T22:02:56.303701+0000","last_active":"2024-09-20T22:03:57.313678+0000","last_peered":"2024-09-20T22:03:57.313678+0000","last_clean":"2024-09-20T22:03:57.313678+0000","last_became_active":"2024-09-20T22:02:56.303338+0000","last_became_peered":"2024-09-20T22:02:56.303338+0000","last_unstale":"2024-09-20T22:03:57.313678+0000","last_undegraded":"2024-09-20T22:03:57.313678+0000","last_fullsized":"2024-09-20T22:03:57.313678+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.793299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.307445+0000","last_change":"2024-09-20T22:02:56.347311+0000","last_active":"2024-09-20T22:03:57.307445+0000","last_peered":"2024-09-20T22:03:57.307445+0000","last_clean":"2024-09-20T22:03:57.307445+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:03:57.307445+0000","last_undegraded":"2024-09-20T22:03:57.307445+0000","last_fullsized":"2024-09-20T22:03:57.307445+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.297557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.308961+0000","last_change":"2024-09-20T22:02:50.111842+0000","last_active":"2024-09-20T22:03:57.308961+0000","last_peered":"2024-09-20T22:03:57.308961+0000","last_clean":"2024-09-20T22:03:57.308961+0000","last_became_active":"2024-09-20T22:02:50.111517+0000","last_became_peered":"2024-09-20T22:02:50.111517+0000","last_unstale":"2024-09-20T22:03:57.308961+0000","last_undegraded":"2024-09-20T22:03:57.308961+0000","last_fullsized":"2024-09-20T22:03:57.308961+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.747690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"71'15","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.317953+0000","last_change":"2024-09-20T22:02:52.283078+0000","last_active":"2024-09-20T22:03:57.317953+0000","last_peered":"2024-09-20T22:03:57.317953+0000","last_clean":"2024-09-20T22:03:57.317953+0000","last_became_active":"2024-09-20T22:02:52.282768+0000","last_became_peered":"2024-09-20T22:02:52.282768+0000","last_unstale":"2024-09-20T22:03:57.317953+0000","last_undegraded":"2024-09-20T22:03:57.317953+0000","last_fullsized":"2024-09-20T22:03:57.317953+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:57.703050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.318052+0000","last_change":"2024-09-20T22:02:54.292892+0000","last_active":"2024-09-20T22:03:57.318052+0000","last_peered":"2024-09-20T22:03:57.318052+0000","last_clean":"2024-09-20T22:03:57.318052+0000","last_became_active":"2024-09-20T22:02:54.290510+0000","last_became_peered":"2024-09-20T22:02:54.290510+0000","last_unstale":"2024-09-20T22:03:57.318052+0000","last_undegraded":"2024-09-20T22:03:57.318052+0000","last_fullsized":"2024-09-20T22:03:57.318052+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.173080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312550+0000","last_change":"2024-09-20T22:02:56.347951+0000","last_active":"2024-09-20T22:03:57.312550+0000","last_peered":"2024-09-20T22:03:57.312550+0000","last_clean":"2024-09-20T22:03:57.312550+0000","last_became_active":"2024-09-20T22:02:56.347561+0000","last_became_peered":"2024-09-20T22:02:56.347561+0000","last_unstale":"2024-09-20T22:03:57.312550+0000","last_undegraded":"2024-09-20T22:03:57.312550+0000","last_fullsized":"2024-09-20T22:03:57.312550+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.255072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.306880+0000","last_change":"2024-09-20T22:02:50.108909+0000","last_active":"2024-09-20T22:03:57.306880+0000","last_peered":"2024-09-20T22:03:57.306880+0000","last_clean":"2024-09-20T22:03:57.306880+0000","last_became_active":"2024-09-20T22:02:50.107189+0000","last_became_peered":"2024-09-20T22:02:50.107189+0000","last_unstale":"2024-09-20T22:03:57.306880+0000","last_undegraded":"2024-09-20T22:03:57.306880+0000","last_fullsized":"2024-09-20T22:03:57.306880+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.636681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"71'12","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.309079+0000","last_change":"2024-09-20T22:02:52.284201+0000","last_active":"2024-09-20T22:03:57.309079+0000","last_peered":"2024-09-20T22:03:57.309079+0000","last_clean":"2024-09-20T22:03:57.309079+0000","last_became_active":"2024-09-20T22:02:52.283843+0000","last_became_peered":"2024-09-20T22:02:52.283843+0000","last_unstale":"2024-09-20T22:03:57.309079+0000","last_undegraded":"2024-09-20T22:03:57.309079+0000","last_fullsized":"2024-09-20T22:03:57.309079+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.131718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.307200+0000","last_change":"2024-09-20T22:02:54.293157+0000","last_active":"2024-09-20T22:03:57.307200+0000","last_peered":"2024-09-20T22:03:57.307200+0000","last_clean":"2024-09-20T22:03:57.307200+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:03:57.307200+0000","last_undegraded":"2024-09-20T22:03:57.307200+0000","last_fullsized":"2024-09-20T22:03:57.307200+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.203423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"71'1","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312669+0000","last_change":"2024-09-20T22:02:56.301720+0000","last_active":"2024-09-20T22:03:57.312669+0000","last_peered":"2024-09-20T22:03:57.312669+0000","last_clean":"2024-09-20T22:03:57.312669+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:03:57.312669+0000","last_undegraded":"2024-09-20T22:03:57.312669+0000","last_fullsized":"2024-09-20T22:03:57.312669+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.341799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"64'1","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312958+0000","last_change":"2024-09-20T22:02:50.113259+0000","last_active":"2024-09-20T22:03:57.312958+0000","last_peered":"2024-09-20T22:03:57.312958+0000","last_clean":"2024-09-20T22:03:57.312958+0000","last_became_active":"2024-09-20T22:02:50.112862+0000","last_became_peered":"2024-09-20T22:02:50.112862+0000","last_unstale":"2024-09-20T22:03:57.312958+0000","last_undegraded":"2024-09-20T22:03:57.312958+0000","last_fullsized":"2024-09-20T22:03:57.312958+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.177958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"71'10","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.309272+0000","last_change":"2024-09-20T22:02:52.482300+0000","last_active":"2024-09-20T22:03:57.309272+0000","last_peered":"2024-09-20T22:03:57.309272+0000","last_clean":"2024-09-20T22:03:57.309272+0000","last_became_active":"2024-09-20T22:02:52.481980+0000","last_became_peered":"2024-09-20T22:02:52.481980+0000","last_unstale":"2024-09-20T22:03:57.309272+0000","last_undegraded":"2024-09-20T22:03:57.309272+0000","last_fullsized":"2024-09-20T22:03:57.309272+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.082587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"71'8","reported_seq":70,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.315332+0000","last_change":"2024-09-20T22:02:54.297973+0000","last_active":"2024-09-20T22:03:57.315332+0000","last_peered":"2024-09-20T22:03:57.315332+0000","last_clean":"2024-09-20T22:03:57.315332+0000","last_became_active":"2024-09-20T22:02:54.297657+0000","last_became_peered":"2024-09-20T22:02:54.297657+0000","last_unstale":"2024-09-20T22:03:57.315332+0000","last_undegraded":"2024-09-20T22:03:57.315332+0000","last_fullsized":"2024-09-20T22:03:57.315332+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:20.633207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"85'600","reported_seq":620,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:04:00.518098+0000","last_change":"2024-09-20T22:03:52.318621+0000","last_active":"2024-09-20T22:04:00.518098+0000","last_peered":"2024-09-20T22:04:00.518098+0000","last_clean":"2024-09-20T22:04:00.518098+0000","last_became_active":"2024-09-20T22:03:52.318259+0000","last_became_peered":"2024-09-20T22:03:52.318259+0000","last_unstale":"2024-09-20T22:04:00.518098+0000","last_undegraded":"2024-09-20T22:04:00.518098+0000","last_fullsized":"2024-09-20T22:04:00.518098+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":600,"log_dups_size":0,"ondisk_log_size":600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:01.035033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39321600,"num_objects":600,"num_object_clones":0,"num_object_copies":1800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":600,"num_write_kb":38400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"71'15","reported_seq":74,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.315311+0000","last_change":"2024-09-20T22:02:52.284890+0000","last_active":"2024-09-20T22:03:57.315311+0000","last_peered":"2024-09-20T22:03:57.315311+0000","last_clean":"2024-09-20T22:03:57.315311+0000","last_became_active":"2024-09-20T22:02:52.284027+0000","last_became_peered":"2024-09-20T22:02:52.284027+0000","last_unstale":"2024-09-20T22:03:57.315311+0000","last_undegraded":"2024-09-20T22:03:57.315311+0000","last_fullsized":"2024-09-20T22:03:57.315311+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.177882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.313258+0000","last_change":"2024-09-20T22:02:50.108849+0000","last_active":"2024-09-20T22:03:57.313258+0000","last_peered":"2024-09-20T22:03:57.313258+0000","last_clean":"2024-09-20T22:03:57.313258+0000","last_became_active":"2024-09-20T22:02:50.108508+0000","last_became_peered":"2024-09-20T22:02:50.108508+0000","last_unstale":"2024-09-20T22:03:57.313258+0000","last_undegraded":"2024-09-20T22:03:57.313258+0000","last_fullsized":"2024-09-20T22:03:57.313258+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.430379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"71'8","reported_seq":71,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.307583+0000","last_change":"2024-09-20T22:02:54.295408+0000","last_active":"2024-09-20T22:03:57.307583+0000","last_peered":"2024-09-20T22:03:57.307583+0000","last_clean":"2024-09-20T22:03:57.307583+0000","last_became_active":"2024-09-20T22:02:54.294917+0000","last_became_peered":"2024-09-20T22:02:54.294917+0000","last_unstale":"2024-09-20T22:03:57.307583+0000","last_undegraded":"2024-09-20T22:03:57.307583+0000","last_fullsized":"2024-09-20T22:03:57.307583+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.541024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.313567+0000","last_change":"2024-09-20T22:02:56.296725+0000","last_active":"2024-09-20T22:03:57.313567+0000","last_peered":"2024-09-20T22:03:57.313567+0000","last_clean":"2024-09-20T22:03:57.313567+0000","last_became_active":"2024-09-20T22:02:56.296436+0000","last_became_peered":"2024-09-20T22:02:56.296436+0000","last_unstale":"2024-09-20T22:03:57.313567+0000","last_undegraded":"2024-09-20T22:03:57.313567+0000","last_fullsized":"2024-09-20T22:03:57.313567+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:19.201794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3","version":"85'586","reported_seq":606,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:04:00.525838+0000","last_change":"2024-09-20T22:03:52.312639+0000","last_active":"2024-09-20T22:04:00.525838+0000","last_peered":"2024-09-20T22:04:00.525838+0000","last_clean":"2024-09-20T22:04:00.525838+0000","last_became_active":"2024-09-20T22:03:52.312376+0000","last_became_peered":"2024-09-20T22:03:52.312376+0000","last_unstale":"2024-09-20T22:04:00.525838+0000","last_undegraded":"2024-09-20T22:04:00.525838+0000","last_fullsized":"2024-09-20T22:04:00.525838+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":586,"log_dups_size":0,"ondisk_log_size":586,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:07.854596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38404096,"num_objects":586,"num_object_clones":0,"num_object_copies":1758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":586,"num_write_kb":37504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,4],"acting":[3,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"71'18","reported_seq":80,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312797+0000","last_change":"2024-09-20T22:02:52.482408+0000","last_active":"2024-09-20T22:03:57.312797+0000","last_peered":"2024-09-20T22:03:57.312797+0000","last_clean":"2024-09-20T22:03:57.312797+0000","last_became_active":"2024-09-20T22:02:52.482119+0000","last_became_peered":"2024-09-20T22:02:52.482119+0000","last_unstale":"2024-09-20T22:03:57.312797+0000","last_undegraded":"2024-09-20T22:03:57.312797+0000","last_fullsized":"2024-09-20T22:03:57.312797+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.477571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312956+0000","last_change":"2024-09-20T22:02:50.113398+0000","last_active":"2024-09-20T22:03:57.312956+0000","last_peered":"2024-09-20T22:03:57.312956+0000","last_clean":"2024-09-20T22:03:57.312956+0000","last_became_active":"2024-09-20T22:02:50.113076+0000","last_became_peered":"2024-09-20T22:02:50.113076+0000","last_unstale":"2024-09-20T22:03:57.312956+0000","last_undegraded":"2024-09-20T22:03:57.312956+0000","last_fullsized":"2024-09-20T22:03:57.312956+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.714101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.306567+0000","last_change":"2024-09-20T22:02:54.288282+0000","last_active":"2024-09-20T22:03:57.306567+0000","last_peered":"2024-09-20T22:03:57.306567+0000","last_clean":"2024-09-20T22:03:57.306567+0000","last_became_active":"2024-09-20T22:02:54.288007+0000","last_became_peered":"2024-09-20T22:02:54.288007+0000","last_unstale":"2024-09-20T22:03:57.306567+0000","last_undegraded":"2024-09-20T22:03:57.306567+0000","last_fullsized":"2024-09-20T22:03:57.306567+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.737144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.306702+0000","last_change":"2024-09-20T22:02:56.303051+0000","last_active":"2024-09-20T22:03:57.306702+0000","last_peered":"2024-09-20T22:03:57.306702+0000","last_clean":"2024-09-20T22:03:57.306702+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:03:57.306702+0000","last_undegraded":"2024-09-20T22:03:57.306702+0000","last_fullsized":"2024-09-20T22:03:57.306702+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.012094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"85'644","reported_seq":664,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:04:00.499860+0000","last_change":"2024-09-20T22:03:52.311645+0000","last_active":"2024-09-20T22:04:00.499860+0000","last_peered":"2024-09-20T22:04:00.499860+0000","last_clean":"2024-09-20T22:04:00.499860+0000","last_became_active":"2024-09-20T22:03:52.311369+0000","last_became_peered":"2024-09-20T22:03:52.311369+0000","last_unstale":"2024-09-20T22:04:00.499860+0000","last_undegraded":"2024-09-20T22:04:00.499860+0000","last_fullsized":"2024-09-20T22:04:00.499860+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:08.803963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42205184,"num_objects":644,"num_object_clones":0,"num_object_copies":1932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":644,"num_write_kb":41216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"71'14","reported_seq":70,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.306879+0000","last_change":"2024-09-20T22:02:52.273553+0000","last_active":"2024-09-20T22:03:57.306879+0000","last_peered":"2024-09-20T22:03:57.306879+0000","last_clean":"2024-09-20T22:03:57.306879+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:03:57.306879+0000","last_undegraded":"2024-09-20T22:03:57.306879+0000","last_fullsized":"2024-09-20T22:03:57.306879+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.923296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.309555+0000","last_change":"2024-09-20T22:02:50.110546+0000","last_active":"2024-09-20T22:03:57.309555+0000","last_peered":"2024-09-20T22:03:57.309555+0000","last_clean":"2024-09-20T22:03:57.309555+0000","last_became_active":"2024-09-20T22:02:50.110195+0000","last_became_peered":"2024-09-20T22:02:50.110195+0000","last_unstale":"2024-09-20T22:03:57.309555+0000","last_undegraded":"2024-09-20T22:03:57.309555+0000","last_fullsized":"2024-09-20T22:03:57.309555+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.867716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"71'8","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.310158+0000","last_change":"2024-09-20T22:02:54.295841+0000","last_active":"2024-09-20T22:03:57.310158+0000","last_peered":"2024-09-20T22:03:57.310158+0000","last_clean":"2024-09-20T22:03:57.310158+0000","last_became_active":"2024-09-20T22:02:54.295553+0000","last_became_peered":"2024-09-20T22:02:54.295553+0000","last_unstale":"2024-09-20T22:03:57.310158+0000","last_undegraded":"2024-09-20T22:03:57.310158+0000","last_fullsized":"2024-09-20T22:03:57.310158+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.625795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312503+0000","last_change":"2024-09-20T22:02:56.349864+0000","last_active":"2024-09-20T22:03:57.312503+0000","last_peered":"2024-09-20T22:03:57.312503+0000","last_clean":"2024-09-20T22:03:57.312503+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:03:57.312503+0000","last_undegraded":"2024-09-20T22:03:57.312503+0000","last_fullsized":"2024-09-20T22:03:57.312503+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:57.678673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"85'612","reported_seq":632,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:04:00.517605+0000","last_change":"2024-09-20T22:03:52.308492+0000","last_active":"2024-09-20T22:04:00.517605+0000","last_peered":"2024-09-20T22:04:00.517605+0000","last_clean":"2024-09-20T22:04:00.517605+0000","last_became_active":"2024-09-20T22:03:52.308178+0000","last_became_peered":"2024-09-20T22:03:52.308178+0000","last_unstale":"2024-09-20T22:04:00.517605+0000","last_undegraded":"2024-09-20T22:04:00.517605+0000","last_fullsized":"2024-09-20T22:04:00.517605+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":612,"log_dups_size":0,"ondisk_log_size":612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:19.708994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40108032,"num_objects":612,"num_object_clones":0,"num_object_copies":1836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":612,"num_write_kb":39168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"71'10","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.313368+0000","last_change":"2024-09-20T22:02:52.276282+0000","last_active":"2024-09-20T22:03:57.313368+0000","last_peered":"2024-09-20T22:03:57.313368+0000","last_clean":"2024-09-20T22:03:57.313368+0000","last_became_active":"2024-09-20T22:02:52.275958+0000","last_became_peered":"2024-09-20T22:02:52.275958+0000","last_unstale":"2024-09-20T22:03:57.313368+0000","last_undegraded":"2024-09-20T22:03:57.313368+0000","last_fullsized":"2024-09-20T22:03:57.313368+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.846987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.315269+0000","last_change":"2024-09-20T22:02:50.114476+0000","last_active":"2024-09-20T22:03:57.315269+0000","last_peered":"2024-09-20T22:03:57.315269+0000","last_clean":"2024-09-20T22:03:57.315269+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:03:57.315269+0000","last_undegraded":"2024-09-20T22:03:57.315269+0000","last_fullsized":"2024-09-20T22:03:57.315269+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:57.600759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"71'8","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.308589+0000","last_change":"2024-09-20T22:02:54.293220+0000","last_active":"2024-09-20T22:03:57.308589+0000","last_peered":"2024-09-20T22:03:57.308589+0000","last_clean":"2024-09-20T22:03:57.308589+0000","last_became_active":"2024-09-20T22:02:54.292819+0000","last_became_peered":"2024-09-20T22:02:54.292819+0000","last_unstale":"2024-09-20T22:03:57.308589+0000","last_undegraded":"2024-09-20T22:03:57.308589+0000","last_fullsized":"2024-09-20T22:03:57.308589+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.307685+0000","last_change":"2024-09-20T22:02:56.304595+0000","last_active":"2024-09-20T22:03:57.307685+0000","last_peered":"2024-09-20T22:03:57.307685+0000","last_clean":"2024-09-20T22:03:57.307685+0000","last_became_active":"2024-09-20T22:02:56.304274+0000","last_became_peered":"2024-09-20T22:02:56.304274+0000","last_unstale":"2024-09-20T22:03:57.307685+0000","last_undegraded":"2024-09-20T22:03:57.307685+0000","last_fullsized":"2024-09-20T22:03:57.307685+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.850583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"85'566","reported_seq":586,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:04:00.523422+0000","last_change":"2024-09-20T22:03:52.318093+0000","last_active":"2024-09-20T22:04:00.523422+0000","last_peered":"2024-09-20T22:04:00.523422+0000","last_clean":"2024-09-20T22:04:00.523422+0000","last_became_active":"2024-09-20T22:03:52.317816+0000","last_became_peered":"2024-09-20T22:03:52.317816+0000","last_unstale":"2024-09-20T22:04:00.523422+0000","last_undegraded":"2024-09-20T22:04:00.523422+0000","last_fullsized":"2024-09-20T22:04:00.523422+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:08.496980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37093376,"num_objects":566,"num_object_clones":0,"num_object_copies":1698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":566,"num_write_kb":36224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"71'19","reported_seq":84,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.308688+0000","last_change":"2024-09-20T22:02:52.480446+0000","last_active":"2024-09-20T22:03:57.308688+0000","last_peered":"2024-09-20T22:03:57.308688+0000","last_clean":"2024-09-20T22:03:57.308688+0000","last_became_active":"2024-09-20T22:02:52.480077+0000","last_became_peered":"2024-09-20T22:02:52.480077+0000","last_unstale":"2024-09-20T22:03:57.308688+0000","last_undegraded":"2024-09-20T22:03:57.308688+0000","last_fullsized":"2024-09-20T22:03:57.308688+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.830883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.314261+0000","last_change":"2024-09-20T22:02:50.104649+0000","last_active":"2024-09-20T22:03:57.314261+0000","last_peered":"2024-09-20T22:03:57.314261+0000","last_clean":"2024-09-20T22:03:57.314261+0000","last_became_active":"2024-09-20T22:02:50.104345+0000","last_became_peered":"2024-09-20T22:02:50.104345+0000","last_unstale":"2024-09-20T22:03:57.314261+0000","last_undegraded":"2024-09-20T22:03:57.314261+0000","last_fullsized":"2024-09-20T22:03:57.314261+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.766875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.314088+0000","last_change":"2024-09-20T22:02:54.291638+0000","last_active":"2024-09-20T22:03:57.314088+0000","last_peered":"2024-09-20T22:03:57.314088+0000","last_clean":"2024-09-20T22:03:57.314088+0000","last_became_active":"2024-09-20T22:02:54.290787+0000","last_became_peered":"2024-09-20T22:02:54.290787+0000","last_unstale":"2024-09-20T22:03:57.314088+0000","last_undegraded":"2024-09-20T22:03:57.314088+0000","last_fullsized":"2024-09-20T22:03:57.314088+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.996942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312713+0000","last_change":"2024-09-20T22:02:56.350020+0000","last_active":"2024-09-20T22:03:57.312713+0000","last_peered":"2024-09-20T22:03:57.312713+0000","last_clean":"2024-09-20T22:03:57.312713+0000","last_became_active":"2024-09-20T22:02:56.349581+0000","last_became_peered":"2024-09-20T22:02:56.349581+0000","last_unstale":"2024-09-20T22:03:57.312713+0000","last_undegraded":"2024-09-20T22:03:57.312713+0000","last_fullsized":"2024-09-20T22:03:57.312713+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.722790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"85'565","reported_seq":585,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:04:00.529708+0000","last_change":"2024-09-20T22:03:52.318679+0000","last_active":"2024-09-20T22:04:00.529708+0000","last_peered":"2024-09-20T22:04:00.529708+0000","last_clean":"2024-09-20T22:04:00.529708+0000","last_became_active":"2024-09-20T22:03:52.318296+0000","last_became_peered":"2024-09-20T22:03:52.318296+0000","last_unstale":"2024-09-20T22:04:00.529708+0000","last_undegraded":"2024-09-20T22:04:00.529708+0000","last_fullsized":"2024-09-20T22:04:00.529708+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":565,"log_dups_size":0,"ondisk_log_size":565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:16.272510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37027840,"num_objects":565,"num_object_clones":0,"num_object_copies":1695,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":565,"num_write_kb":36160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"71'28","reported_seq":112,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.314160+0000","last_change":"2024-09-20T22:02:52.285811+0000","last_active":"2024-09-20T22:03:57.314160+0000","last_peered":"2024-09-20T22:03:57.314160+0000","last_clean":"2024-09-20T22:03:57.314160+0000","last_became_active":"2024-09-20T22:02:52.285341+0000","last_became_peered":"2024-09-20T22:02:52.285341+0000","last_unstale":"2024-09-20T22:03:57.314160+0000","last_undegraded":"2024-09-20T22:03:57.314160+0000","last_fullsized":"2024-09-20T22:03:57.314160+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.491983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.307053+0000","last_change":"2024-09-20T22:02:50.110040+0000","last_active":"2024-09-20T22:03:57.307053+0000","last_peered":"2024-09-20T22:03:57.307053+0000","last_clean":"2024-09-20T22:03:57.307053+0000","last_became_active":"2024-09-20T22:02:50.108457+0000","last_became_peered":"2024-09-20T22:02:50.108457+0000","last_unstale":"2024-09-20T22:03:57.307053+0000","last_undegraded":"2024-09-20T22:03:57.307053+0000","last_fullsized":"2024-09-20T22:03:57.307053+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.280284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"66'2","reported_seq":62,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.315125+0000","last_change":"2024-09-20T22:02:52.273740+0000","last_active":"2024-09-20T22:03:57.315125+0000","last_peered":"2024-09-20T22:03:57.315125+0000","last_clean":"2024-09-20T22:03:57.315125+0000","last_became_active":"2024-09-20T22:02:49.096453+0000","last_became_peered":"2024-09-20T22:02:49.096453+0000","last_unstale":"2024-09-20T22:03:57.315125+0000","last_undegraded":"2024-09-20T22:03:57.315125+0000","last_fullsized":"2024-09-20T22:03:57.315125+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.811834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.308079+0000","last_change":"2024-09-20T22:02:54.295696+0000","last_active":"2024-09-20T22:03:57.308079+0000","last_peered":"2024-09-20T22:03:57.308079+0000","last_clean":"2024-09-20T22:03:57.308079+0000","last_became_active":"2024-09-20T22:02:54.295401+0000","last_became_peered":"2024-09-20T22:02:54.295401+0000","last_unstale":"2024-09-20T22:03:57.308079+0000","last_undegraded":"2024-09-20T22:03:57.308079+0000","last_fullsized":"2024-09-20T22:03:57.308079+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.933605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.311392+0000","last_change":"2024-09-20T22:02:56.306053+0000","last_active":"2024-09-20T22:03:57.311392+0000","last_peered":"2024-09-20T22:03:57.311392+0000","last_clean":"2024-09-20T22:03:57.311392+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:03:57.311392+0000","last_undegraded":"2024-09-20T22:03:57.311392+0000","last_fullsized":"2024-09-20T22:03:57.311392+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.682717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"85'591","reported_seq":611,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:04:00.509078+0000","last_change":"2024-09-20T22:03:52.312143+0000","last_active":"2024-09-20T22:04:00.509078+0000","last_peered":"2024-09-20T22:04:00.509078+0000","last_clean":"2024-09-20T22:04:00.509078+0000","last_became_active":"2024-09-20T22:03:52.311831+0000","last_became_peered":"2024-09-20T22:03:52.311831+0000","last_unstale":"2024-09-20T22:04:00.509078+0000","last_undegraded":"2024-09-20T22:04:00.509078+0000","last_fullsized":"2024-09-20T22:04:00.509078+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:17.523152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38731776,"num_objects":591,"num_object_clones":0,"num_object_copies":1773,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":591,"num_write_kb":37824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,1],"acting":[0,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"71'13","reported_seq":76,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.315479+0000","last_change":"2024-09-20T22:02:52.280309+0000","last_active":"2024-09-20T22:03:57.315479+0000","last_peered":"2024-09-20T22:03:57.315479+0000","last_clean":"2024-09-20T22:03:57.315479+0000","last_became_active":"2024-09-20T22:02:52.280005+0000","last_became_peered":"2024-09-20T22:02:52.280005+0000","last_unstale":"2024-09-20T22:03:57.315479+0000","last_undegraded":"2024-09-20T22:03:57.315479+0000","last_fullsized":"2024-09-20T22:03:57.315479+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.190808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.318385+0000","last_change":"2024-09-20T22:02:50.106023+0000","last_active":"2024-09-20T22:03:57.318385+0000","last_peered":"2024-09-20T22:03:57.318385+0000","last_clean":"2024-09-20T22:03:57.318385+0000","last_became_active":"2024-09-20T22:02:50.105633+0000","last_became_peered":"2024-09-20T22:02:50.105633+0000","last_unstale":"2024-09-20T22:03:57.318385+0000","last_undegraded":"2024-09-20T22:03:57.318385+0000","last_fullsized":"2024-09-20T22:03:57.318385+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.449401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"64'1","reported_seq":61,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.316743+0000","last_change":"2024-09-20T22:02:52.270463+0000","last_active":"2024-09-20T22:03:57.316743+0000","last_peered":"2024-09-20T22:03:57.316743+0000","last_clean":"2024-09-20T22:03:57.316743+0000","last_became_active":"2024-09-20T22:02:49.099508+0000","last_became_peered":"2024-09-20T22:02:49.099508+0000","last_unstale":"2024-09-20T22:03:57.316743+0000","last_undegraded":"2024-09-20T22:03:57.316743+0000","last_fullsized":"2024-09-20T22:03:57.316743+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.955566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.316747+0000","last_change":"2024-09-20T22:02:54.292450+0000","last_active":"2024-09-20T22:03:57.316747+0000","last_peered":"2024-09-20T22:03:57.316747+0000","last_clean":"2024-09-20T22:03:57.316747+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:03:57.316747+0000","last_undegraded":"2024-09-20T22:03:57.316747+0000","last_fullsized":"2024-09-20T22:03:57.316747+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.941012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312598+0000","last_change":"2024-09-20T22:02:56.349789+0000","last_active":"2024-09-20T22:03:57.312598+0000","last_peered":"2024-09-20T22:03:57.312598+0000","last_clean":"2024-09-20T22:03:57.312598+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:03:57.312598+0000","last_undegraded":"2024-09-20T22:03:57.312598+0000","last_fullsized":"2024-09-20T22:03:57.312598+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.060034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"85'628","reported_seq":648,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:04:00.537696+0000","last_change":"2024-09-20T22:03:52.316989+0000","last_active":"2024-09-20T22:04:00.537696+0000","last_peered":"2024-09-20T22:04:00.537696+0000","last_clean":"2024-09-20T22:04:00.537696+0000","last_became_active":"2024-09-20T22:03:52.316729+0000","last_became_peered":"2024-09-20T22:03:52.316729+0000","last_unstale":"2024-09-20T22:04:00.537696+0000","last_undegraded":"2024-09-20T22:04:00.537696+0000","last_fullsized":"2024-09-20T22:04:00.537696+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":628,"log_dups_size":0,"ondisk_log_size":628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:00.315548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41156608,"num_objects":628,"num_object_clones":0,"num_object_copies":1884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":628,"num_write_kb":40192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"71'12","reported_seq":66,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.309364+0000","last_change":"2024-09-20T22:02:52.277878+0000","last_active":"2024-09-20T22:03:57.309364+0000","last_peered":"2024-09-20T22:03:57.309364+0000","last_clean":"2024-09-20T22:03:57.309364+0000","last_became_active":"2024-09-20T22:02:52.277548+0000","last_became_peered":"2024-09-20T22:02:52.277548+0000","last_unstale":"2024-09-20T22:03:57.309364+0000","last_undegraded":"2024-09-20T22:03:57.309364+0000","last_fullsized":"2024-09-20T22:03:57.309364+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.295996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.308534+0000","last_change":"2024-09-20T22:02:50.110740+0000","last_active":"2024-09-20T22:03:57.308534+0000","last_peered":"2024-09-20T22:03:57.308534+0000","last_clean":"2024-09-20T22:03:57.308534+0000","last_became_active":"2024-09-20T22:02:50.110448+0000","last_became_peered":"2024-09-20T22:02:50.110448+0000","last_unstale":"2024-09-20T22:03:57.308534+0000","last_undegraded":"2024-09-20T22:03:57.308534+0000","last_fullsized":"2024-09-20T22:03:57.308534+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.065672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":128,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:58.750703+0000","last_change":"2024-09-20T22:02:52.480790+0000","last_active":"2024-09-20T22:03:58.750703+0000","last_peered":"2024-09-20T22:03:58.750703+0000","last_clean":"2024-09-20T22:03:58.750703+0000","last_became_active":"2024-09-20T22:02:49.098777+0000","last_became_peered":"2024-09-20T22:02:49.098777+0000","last_unstale":"2024-09-20T22:03:58.750703+0000","last_undegraded":"2024-09-20T22:03:58.750703+0000","last_fullsized":"2024-09-20T22:03:58.750703+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.872201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":64,"num_read_kb":59,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.314435+0000","last_change":"2024-09-20T22:02:54.294806+0000","last_active":"2024-09-20T22:03:57.314435+0000","last_peered":"2024-09-20T22:03:57.314435+0000","last_clean":"2024-09-20T22:03:57.314435+0000","last_became_active":"2024-09-20T22:02:54.294495+0000","last_became_peered":"2024-09-20T22:02:54.294495+0000","last_unstale":"2024-09-20T22:03:57.314435+0000","last_undegraded":"2024-09-20T22:03:57.314435+0000","last_fullsized":"2024-09-20T22:03:57.314435+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:23.237482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.313765+0000","last_change":"2024-09-20T22:02:56.308897+0000","last_active":"2024-09-20T22:03:57.313765+0000","last_peered":"2024-09-20T22:03:57.313765+0000","last_clean":"2024-09-20T22:03:57.313765+0000","last_became_active":"2024-09-20T22:02:56.308574+0000","last_became_peered":"2024-09-20T22:02:56.308574+0000","last_unstale":"2024-09-20T22:03:57.313765+0000","last_undegraded":"2024-09-20T22:03:57.313765+0000","last_fullsized":"2024-09-20T22:03:57.313765+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.064046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.6","version":"85'596","reported_seq":616,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:04:00.530966+0000","last_change":"2024-09-20T22:03:52.318412+0000","last_active":"2024-09-20T22:04:00.530966+0000","last_peered":"2024-09-20T22:04:00.530966+0000","last_clean":"2024-09-20T22:04:00.530966+0000","last_became_active":"2024-09-20T22:03:52.318009+0000","last_became_peered":"2024-09-20T22:03:52.318009+0000","last_unstale":"2024-09-20T22:04:00.530966+0000","last_undegraded":"2024-09-20T22:04:00.530966+0000","last_fullsized":"2024-09-20T22:04:00.530966+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:19.834559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39059456,"num_objects":596,"num_object_clones":0,"num_object_copies":1788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":596,"num_write_kb":38144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"71'16","reported_seq":85,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.314776+0000","last_change":"2024-09-20T22:02:52.482708+0000","last_active":"2024-09-20T22:03:57.314776+0000","last_peered":"2024-09-20T22:03:57.314776+0000","last_clean":"2024-09-20T22:03:57.314776+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:03:57.314776+0000","last_undegraded":"2024-09-20T22:03:57.314776+0000","last_fullsized":"2024-09-20T22:03:57.314776+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.243856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.309372+0000","last_change":"2024-09-20T22:02:50.106319+0000","last_active":"2024-09-20T22:03:57.309372+0000","last_peered":"2024-09-20T22:03:57.309372+0000","last_clean":"2024-09-20T22:03:57.309372+0000","last_became_active":"2024-09-20T22:02:50.106011+0000","last_became_peered":"2024-09-20T22:02:50.106011+0000","last_unstale":"2024-09-20T22:03:57.309372+0000","last_undegraded":"2024-09-20T22:03:57.309372+0000","last_fullsized":"2024-09-20T22:03:57.309372+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.288394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":308,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312156+0000","last_change":"2024-09-20T22:02:55.903219+0000","last_active":"2024-09-20T22:03:57.312156+0000","last_peered":"2024-09-20T22:03:57.312156+0000","last_clean":"2024-09-20T22:03:57.312156+0000","last_became_active":"2024-09-20T22:02:45.068482+0000","last_became_peered":"2024-09-20T22:02:45.068482+0000","last_unstale":"2024-09-20T22:03:57.312156+0000","last_undegraded":"2024-09-20T22:03:57.312156+0000","last_fullsized":"2024-09-20T22:03:57.312156+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'117","last_scrub_stamp":"2024-09-20T22:02:55.903139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.903139+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.843650+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312766+0000","last_change":"2024-09-20T22:02:54.298298+0000","last_active":"2024-09-20T22:03:57.312766+0000","last_peered":"2024-09-20T22:03:57.312766+0000","last_clean":"2024-09-20T22:03:57.312766+0000","last_became_active":"2024-09-20T22:02:54.298021+0000","last_became_peered":"2024-09-20T22:02:54.298021+0000","last_unstale":"2024-09-20T22:03:57.312766+0000","last_undegraded":"2024-09-20T22:03:57.312766+0000","last_fullsized":"2024-09-20T22:03:57.312766+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.715132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.314228+0000","last_change":"2024-09-20T22:02:56.308039+0000","last_active":"2024-09-20T22:03:57.314228+0000","last_peered":"2024-09-20T22:03:57.314228+0000","last_clean":"2024-09-20T22:03:57.314228+0000","last_became_active":"2024-09-20T22:02:56.307749+0000","last_became_peered":"2024-09-20T22:02:56.307749+0000","last_unstale":"2024-09-20T22:03:57.314228+0000","last_undegraded":"2024-09-20T22:03:57.314228+0000","last_fullsized":"2024-09-20T22:03:57.314228+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"85'636","reported_seq":656,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:04:00.530439+0000","last_change":"2024-09-20T22:03:52.310053+0000","last_active":"2024-09-20T22:04:00.530439+0000","last_peered":"2024-09-20T22:04:00.530439+0000","last_clean":"2024-09-20T22:04:00.530439+0000","last_became_active":"2024-09-20T22:03:52.309765+0000","last_became_peered":"2024-09-20T22:03:52.309765+0000","last_unstale":"2024-09-20T22:04:00.530439+0000","last_undegraded":"2024-09-20T22:04:00.530439+0000","last_fullsized":"2024-09-20T22:04:00.530439+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:00.608578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41680896,"num_objects":636,"num_object_clones":0,"num_object_copies":1908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":636,"num_write_kb":40704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"71'11","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.307641+0000","last_change":"2024-09-20T22:02:52.273177+0000","last_active":"2024-09-20T22:03:57.307641+0000","last_peered":"2024-09-20T22:03:57.307641+0000","last_clean":"2024-09-20T22:03:57.307641+0000","last_became_active":"2024-09-20T22:02:52.272766+0000","last_became_peered":"2024-09-20T22:02:52.272766+0000","last_unstale":"2024-09-20T22:03:57.307641+0000","last_undegraded":"2024-09-20T22:03:57.307641+0000","last_fullsized":"2024-09-20T22:03:57.307641+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.073826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.307018+0000","last_change":"2024-09-20T22:02:50.109626+0000","last_active":"2024-09-20T22:03:57.307018+0000","last_peered":"2024-09-20T22:03:57.307018+0000","last_clean":"2024-09-20T22:03:57.307018+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:03:57.307018+0000","last_undegraded":"2024-09-20T22:03:57.307018+0000","last_fullsized":"2024-09-20T22:03:57.307018+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.659206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.314074+0000","last_change":"2024-09-20T22:02:54.290285+0000","last_active":"2024-09-20T22:03:57.314074+0000","last_peered":"2024-09-20T22:03:57.314074+0000","last_clean":"2024-09-20T22:03:57.314074+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:03:57.314074+0000","last_undegraded":"2024-09-20T22:03:57.314074+0000","last_fullsized":"2024-09-20T22:03:57.314074+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.665015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.308907+0000","last_change":"2024-09-20T22:02:56.349707+0000","last_active":"2024-09-20T22:03:57.308907+0000","last_peered":"2024-09-20T22:03:57.308907+0000","last_clean":"2024-09-20T22:03:57.308907+0000","last_became_active":"2024-09-20T22:02:56.349376+0000","last_became_peered":"2024-09-20T22:02:56.349376+0000","last_unstale":"2024-09-20T22:03:57.308907+0000","last_undegraded":"2024-09-20T22:03:57.308907+0000","last_fullsized":"2024-09-20T22:03:57.308907+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"85'582","reported_seq":602,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:04:00.518407+0000","last_change":"2024-09-20T22:03:52.312306+0000","last_active":"2024-09-20T22:04:00.518407+0000","last_peered":"2024-09-20T22:04:00.518407+0000","last_clean":"2024-09-20T22:04:00.518407+0000","last_became_active":"2024-09-20T22:03:52.311982+0000","last_became_peered":"2024-09-20T22:03:52.311982+0000","last_unstale":"2024-09-20T22:04:00.518407+0000","last_undegraded":"2024-09-20T22:04:00.518407+0000","last_fullsized":"2024-09-20T22:04:00.518407+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":582,"log_dups_size":0,"ondisk_log_size":582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:03.397796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38141952,"num_objects":582,"num_object_clones":0,"num_object_copies":1746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":582,"num_write_kb":37248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"71'17","reported_seq":77,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.307224+0000","last_change":"2024-09-20T22:02:52.274124+0000","last_active":"2024-09-20T22:03:57.307224+0000","last_peered":"2024-09-20T22:03:57.307224+0000","last_clean":"2024-09-20T22:03:57.307224+0000","last_became_active":"2024-09-20T22:02:52.273811+0000","last_became_peered":"2024-09-20T22:02:52.273811+0000","last_unstale":"2024-09-20T22:03:57.307224+0000","last_undegraded":"2024-09-20T22:03:57.307224+0000","last_fullsized":"2024-09-20T22:03:57.307224+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:57.886989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.314370+0000","last_change":"2024-09-20T22:02:50.115379+0000","last_active":"2024-09-20T22:03:57.314370+0000","last_peered":"2024-09-20T22:03:57.314370+0000","last_clean":"2024-09-20T22:03:57.314370+0000","last_became_active":"2024-09-20T22:02:50.115015+0000","last_became_peered":"2024-09-20T22:02:50.115015+0000","last_unstale":"2024-09-20T22:03:57.314370+0000","last_undegraded":"2024-09-20T22:03:57.314370+0000","last_fullsized":"2024-09-20T22:03:57.314370+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:16.806437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.313856+0000","last_change":"2024-09-20T22:02:54.291468+0000","last_active":"2024-09-20T22:03:57.313856+0000","last_peered":"2024-09-20T22:03:57.313856+0000","last_clean":"2024-09-20T22:03:57.313856+0000","last_became_active":"2024-09-20T22:02:54.291161+0000","last_became_peered":"2024-09-20T22:02:54.291161+0000","last_unstale":"2024-09-20T22:03:57.313856+0000","last_undegraded":"2024-09-20T22:03:57.313856+0000","last_fullsized":"2024-09-20T22:03:57.313856+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.806525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.316936+0000","last_change":"2024-09-20T22:02:56.307571+0000","last_active":"2024-09-20T22:03:57.316936+0000","last_peered":"2024-09-20T22:03:57.316936+0000","last_clean":"2024-09-20T22:03:57.316936+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:03:57.316936+0000","last_undegraded":"2024-09-20T22:03:57.316936+0000","last_fullsized":"2024-09-20T22:03:57.316936+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:20.532247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"85'653","reported_seq":673,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:04:00.528547+0000","last_change":"2024-09-20T22:03:52.317134+0000","last_active":"2024-09-20T22:04:00.528547+0000","last_peered":"2024-09-20T22:04:00.528547+0000","last_clean":"2024-09-20T22:04:00.528547+0000","last_became_active":"2024-09-20T22:03:52.316825+0000","last_became_peered":"2024-09-20T22:03:52.316825+0000","last_unstale":"2024-09-20T22:04:00.528547+0000","last_undegraded":"2024-09-20T22:04:00.528547+0000","last_fullsized":"2024-09-20T22:04:00.528547+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:04.432177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":42795008,"num_objects":653,"num_object_clones":0,"num_object_copies":1959,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":653,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":653,"num_write_kb":41792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"71'10","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.307351+0000","last_change":"2024-09-20T22:02:52.284306+0000","last_active":"2024-09-20T22:03:57.307351+0000","last_peered":"2024-09-20T22:03:57.307351+0000","last_clean":"2024-09-20T22:03:57.307351+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:03:57.307351+0000","last_undegraded":"2024-09-20T22:03:57.307351+0000","last_fullsized":"2024-09-20T22:03:57.307351+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:20.155009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"64'1","reported_seq":58,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.310787+0000","last_change":"2024-09-20T22:02:50.110772+0000","last_active":"2024-09-20T22:03:57.310787+0000","last_peered":"2024-09-20T22:03:57.310787+0000","last_clean":"2024-09-20T22:03:57.310787+0000","last_became_active":"2024-09-20T22:02:50.110446+0000","last_became_peered":"2024-09-20T22:02:50.110446+0000","last_unstale":"2024-09-20T22:03:57.310787+0000","last_undegraded":"2024-09-20T22:03:57.310787+0000","last_fullsized":"2024-09-20T22:03:57.310787+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.664497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"71'8","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.316504+0000","last_change":"2024-09-20T22:02:54.294781+0000","last_active":"2024-09-20T22:03:57.316504+0000","last_peered":"2024-09-20T22:03:57.316504+0000","last_clean":"2024-09-20T22:03:57.316504+0000","last_became_active":"2024-09-20T22:02:54.294458+0000","last_became_peered":"2024-09-20T22:02:54.294458+0000","last_unstale":"2024-09-20T22:03:57.316504+0000","last_undegraded":"2024-09-20T22:03:57.316504+0000","last_fullsized":"2024-09-20T22:03:57.316504+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.307338+0000","last_change":"2024-09-20T22:02:56.300412+0000","last_active":"2024-09-20T22:03:57.307338+0000","last_peered":"2024-09-20T22:03:57.307338+0000","last_clean":"2024-09-20T22:03:57.307338+0000","last_became_active":"2024-09-20T22:02:56.300093+0000","last_became_peered":"2024-09-20T22:02:56.300093+0000","last_unstale":"2024-09-20T22:03:57.307338+0000","last_undegraded":"2024-09-20T22:03:57.307338+0000","last_fullsized":"2024-09-20T22:03:57.307338+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.204657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"85'599","reported_seq":619,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:04:00.504590+0000","last_change":"2024-09-20T22:03:52.307520+0000","last_active":"2024-09-20T22:04:00.504590+0000","last_peered":"2024-09-20T22:04:00.504590+0000","last_clean":"2024-09-20T22:04:00.504590+0000","last_became_active":"2024-09-20T22:03:52.307223+0000","last_became_peered":"2024-09-20T22:03:52.307223+0000","last_unstale":"2024-09-20T22:04:00.504590+0000","last_undegraded":"2024-09-20T22:04:00.504590+0000","last_fullsized":"2024-09-20T22:04:00.504590+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":599,"log_dups_size":0,"ondisk_log_size":599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:54.222039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39256064,"num_objects":599,"num_object_clones":0,"num_object_copies":1797,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":599,"num_write_kb":38336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"71'9","reported_seq":72,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.308637+0000","last_change":"2024-09-20T22:03:52.011299+0000","last_active":"2024-09-20T22:03:57.308637+0000","last_peered":"2024-09-20T22:03:57.308637+0000","last_clean":"2024-09-20T22:03:57.308637+0000","last_became_active":"2024-09-20T22:02:52.278381+0000","last_became_peered":"2024-09-20T22:02:52.278381+0000","last_unstale":"2024-09-20T22:03:57.308637+0000","last_undegraded":"2024-09-20T22:03:57.308637+0000","last_fullsized":"2024-09-20T22:03:57.308637+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:03:52.011201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:03:52.011201+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:14.119868+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":58,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.314604+0000","last_change":"2024-09-20T22:02:50.113046+0000","last_active":"2024-09-20T22:03:57.314604+0000","last_peered":"2024-09-20T22:03:57.314604+0000","last_clean":"2024-09-20T22:03:57.314604+0000","last_became_active":"2024-09-20T22:02:50.112682+0000","last_became_peered":"2024-09-20T22:02:50.112682+0000","last_unstale":"2024-09-20T22:03:57.314604+0000","last_undegraded":"2024-09-20T22:03:57.314604+0000","last_fullsized":"2024-09-20T22:03:57.314604+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:12.700772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.317329+0000","last_change":"2024-09-20T22:02:54.289079+0000","last_active":"2024-09-20T22:03:57.317329+0000","last_peered":"2024-09-20T22:03:57.317329+0000","last_clean":"2024-09-20T22:03:57.317329+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:03:57.317329+0000","last_undegraded":"2024-09-20T22:03:57.317329+0000","last_fullsized":"2024-09-20T22:03:57.317329+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.034123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.308441+0000","last_change":"2024-09-20T22:02:56.298414+0000","last_active":"2024-09-20T22:03:57.308441+0000","last_peered":"2024-09-20T22:03:57.308441+0000","last_clean":"2024-09-20T22:03:57.308441+0000","last_became_active":"2024-09-20T22:02:56.298046+0000","last_became_peered":"2024-09-20T22:02:56.298046+0000","last_unstale":"2024-09-20T22:03:57.308441+0000","last_undegraded":"2024-09-20T22:03:57.308441+0000","last_fullsized":"2024-09-20T22:03:57.308441+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:59.723832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"85'591","reported_seq":611,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:04:00.529595+0000","last_change":"2024-09-20T22:03:52.314546+0000","last_active":"2024-09-20T22:04:00.529595+0000","last_peered":"2024-09-20T22:04:00.529595+0000","last_clean":"2024-09-20T22:04:00.529595+0000","last_became_active":"2024-09-20T22:03:52.314199+0000","last_became_peered":"2024-09-20T22:03:52.314199+0000","last_unstale":"2024-09-20T22:04:00.529595+0000","last_undegraded":"2024-09-20T22:04:00.529595+0000","last_fullsized":"2024-09-20T22:04:00.529595+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":591,"log_dups_size":0,"ondisk_log_size":591,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:01.779244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38731776,"num_objects":591,"num_object_clones":0,"num_object_copies":1773,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":591,"num_write_kb":37824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"71'19","reported_seq":79,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.314174+0000","last_change":"2024-09-20T22:02:52.480577+0000","last_active":"2024-09-20T22:03:57.314174+0000","last_peered":"2024-09-20T22:03:57.314174+0000","last_clean":"2024-09-20T22:03:57.314174+0000","last_became_active":"2024-09-20T22:02:52.480208+0000","last_became_peered":"2024-09-20T22:02:52.480208+0000","last_unstale":"2024-09-20T22:03:57.314174+0000","last_undegraded":"2024-09-20T22:03:57.314174+0000","last_fullsized":"2024-09-20T22:03:57.314174+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.853373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312989+0000","last_change":"2024-09-20T22:02:50.109094+0000","last_active":"2024-09-20T22:03:57.312989+0000","last_peered":"2024-09-20T22:03:57.312989+0000","last_clean":"2024-09-20T22:03:57.312989+0000","last_became_active":"2024-09-20T22:02:50.108712+0000","last_became_peered":"2024-09-20T22:02:50.108712+0000","last_unstale":"2024-09-20T22:03:57.312989+0000","last_undegraded":"2024-09-20T22:03:57.312989+0000","last_fullsized":"2024-09-20T22:03:57.312989+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:08.545134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.316449+0000","last_change":"2024-09-20T22:02:54.291441+0000","last_active":"2024-09-20T22:03:57.316449+0000","last_peered":"2024-09-20T22:03:57.316449+0000","last_clean":"2024-09-20T22:03:57.316449+0000","last_became_active":"2024-09-20T22:02:54.291134+0000","last_became_peered":"2024-09-20T22:02:54.291134+0000","last_unstale":"2024-09-20T22:03:57.316449+0000","last_undegraded":"2024-09-20T22:03:57.316449+0000","last_fullsized":"2024-09-20T22:03:57.316449+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:05.677000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312053+0000","last_change":"2024-09-20T22:02:56.308001+0000","last_active":"2024-09-20T22:03:57.312053+0000","last_peered":"2024-09-20T22:03:57.312053+0000","last_clean":"2024-09-20T22:03:57.312053+0000","last_became_active":"2024-09-20T22:02:56.307687+0000","last_became_peered":"2024-09-20T22:02:56.307687+0000","last_unstale":"2024-09-20T22:03:57.312053+0000","last_undegraded":"2024-09-20T22:03:57.312053+0000","last_fullsized":"2024-09-20T22:03:57.312053+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.249693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"85'583","reported_seq":603,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:04:00.523374+0000","last_change":"2024-09-20T22:03:52.316640+0000","last_active":"2024-09-20T22:04:00.523374+0000","last_peered":"2024-09-20T22:04:00.523374+0000","last_clean":"2024-09-20T22:04:00.523374+0000","last_became_active":"2024-09-20T22:03:52.316211+0000","last_became_peered":"2024-09-20T22:03:52.316211+0000","last_unstale":"2024-09-20T22:04:00.523374+0000","last_undegraded":"2024-09-20T22:04:00.523374+0000","last_fullsized":"2024-09-20T22:04:00.523374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":583,"log_dups_size":0,"ondisk_log_size":583,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:03.907312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38207488,"num_objects":583,"num_object_clones":0,"num_object_copies":1749,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":583,"num_write_kb":37312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"71'12","reported_seq":72,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.306665+0000","last_change":"2024-09-20T22:02:52.284579+0000","last_active":"2024-09-20T22:03:57.306665+0000","last_peered":"2024-09-20T22:03:57.306665+0000","last_clean":"2024-09-20T22:03:57.306665+0000","last_became_active":"2024-09-20T22:02:52.284144+0000","last_became_peered":"2024-09-20T22:02:52.284144+0000","last_unstale":"2024-09-20T22:03:57.306665+0000","last_undegraded":"2024-09-20T22:03:57.306665+0000","last_fullsized":"2024-09-20T22:03:57.306665+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:00.901626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.313057+0000","last_change":"2024-09-20T22:02:50.111591+0000","last_active":"2024-09-20T22:03:57.313057+0000","last_peered":"2024-09-20T22:03:57.313057+0000","last_clean":"2024-09-20T22:03:57.313057+0000","last_became_active":"2024-09-20T22:02:50.111267+0000","last_became_peered":"2024-09-20T22:02:50.111267+0000","last_unstale":"2024-09-20T22:03:57.313057+0000","last_undegraded":"2024-09-20T22:03:57.313057+0000","last_fullsized":"2024-09-20T22:03:57.313057+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.038287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.318342+0000","last_change":"2024-09-20T22:02:54.293804+0000","last_active":"2024-09-20T22:03:57.318342+0000","last_peered":"2024-09-20T22:03:57.318342+0000","last_clean":"2024-09-20T22:03:57.318342+0000","last_became_active":"2024-09-20T22:02:54.293377+0000","last_became_peered":"2024-09-20T22:02:54.293377+0000","last_unstale":"2024-09-20T22:03:57.318342+0000","last_undegraded":"2024-09-20T22:03:57.318342+0000","last_fullsized":"2024-09-20T22:03:57.318342+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.827756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.310426+0000","last_change":"2024-09-20T22:02:56.307499+0000","last_active":"2024-09-20T22:03:57.310426+0000","last_peered":"2024-09-20T22:03:57.310426+0000","last_clean":"2024-09-20T22:03:57.310426+0000","last_became_active":"2024-09-20T22:02:56.306991+0000","last_became_peered":"2024-09-20T22:02:56.306991+0000","last_unstale":"2024-09-20T22:03:57.310426+0000","last_undegraded":"2024-09-20T22:03:57.310426+0000","last_fullsized":"2024-09-20T22:03:57.310426+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.424396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"85'579","reported_seq":599,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:04:00.535281+0000","last_change":"2024-09-20T22:03:52.311206+0000","last_active":"2024-09-20T22:04:00.535281+0000","last_peered":"2024-09-20T22:04:00.535281+0000","last_clean":"2024-09-20T22:04:00.535281+0000","last_became_active":"2024-09-20T22:03:52.310886+0000","last_became_peered":"2024-09-20T22:03:52.310886+0000","last_unstale":"2024-09-20T22:04:00.535281+0000","last_undegraded":"2024-09-20T22:04:00.535281+0000","last_fullsized":"2024-09-20T22:04:00.535281+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:19.836377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37945344,"num_objects":579,"num_object_clones":0,"num_object_copies":1737,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":579,"num_write_kb":37056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"71'15","reported_seq":73,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.313447+0000","last_change":"2024-09-20T22:02:52.482395+0000","last_active":"2024-09-20T22:03:57.313447+0000","last_peered":"2024-09-20T22:03:57.313447+0000","last_clean":"2024-09-20T22:03:57.313447+0000","last_became_active":"2024-09-20T22:02:52.482008+0000","last_became_peered":"2024-09-20T22:02:52.482008+0000","last_unstale":"2024-09-20T22:03:57.313447+0000","last_undegraded":"2024-09-20T22:03:57.313447+0000","last_fullsized":"2024-09-20T22:03:57.313447+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.554179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"64'2","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.313253+0000","last_change":"2024-09-20T22:02:50.115186+0000","last_active":"2024-09-20T22:03:57.313253+0000","last_peered":"2024-09-20T22:03:57.313253+0000","last_clean":"2024-09-20T22:03:57.313253+0000","last_became_active":"2024-09-20T22:02:50.114819+0000","last_became_peered":"2024-09-20T22:02:50.114819+0000","last_unstale":"2024-09-20T22:03:57.313253+0000","last_undegraded":"2024-09-20T22:03:57.313253+0000","last_fullsized":"2024-09-20T22:03:57.313253+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.801926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312049+0000","last_change":"2024-09-20T22:02:54.296823+0000","last_active":"2024-09-20T22:03:57.312049+0000","last_peered":"2024-09-20T22:03:57.312049+0000","last_clean":"2024-09-20T22:03:57.312049+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:03:57.312049+0000","last_undegraded":"2024-09-20T22:03:57.312049+0000","last_fullsized":"2024-09-20T22:03:57.312049+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.496588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.314773+0000","last_change":"2024-09-20T22:02:56.347283+0000","last_active":"2024-09-20T22:03:57.314773+0000","last_peered":"2024-09-20T22:03:57.314773+0000","last_clean":"2024-09-20T22:03:57.314773+0000","last_became_active":"2024-09-20T22:02:56.346990+0000","last_became_peered":"2024-09-20T22:02:56.346990+0000","last_unstale":"2024-09-20T22:03:57.314773+0000","last_undegraded":"2024-09-20T22:03:57.314773+0000","last_fullsized":"2024-09-20T22:03:57.314773+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:14.718016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.313964+0000","last_change":"2024-09-20T22:02:50.112245+0000","last_active":"2024-09-20T22:03:57.313964+0000","last_peered":"2024-09-20T22:03:57.313964+0000","last_clean":"2024-09-20T22:03:57.313964+0000","last_became_active":"2024-09-20T22:02:50.111935+0000","last_became_peered":"2024-09-20T22:02:50.111935+0000","last_unstale":"2024-09-20T22:03:57.313964+0000","last_undegraded":"2024-09-20T22:03:57.313964+0000","last_fullsized":"2024-09-20T22:03:57.313964+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:16.998775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"71'6","reported_seq":57,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312797+0000","last_change":"2024-09-20T22:02:52.481046+0000","last_active":"2024-09-20T22:03:57.312797+0000","last_peered":"2024-09-20T22:03:57.312797+0000","last_clean":"2024-09-20T22:03:57.312797+0000","last_became_active":"2024-09-20T22:02:52.480771+0000","last_became_peered":"2024-09-20T22:02:52.480771+0000","last_unstale":"2024-09-20T22:03:57.312797+0000","last_undegraded":"2024-09-20T22:03:57.312797+0000","last_fullsized":"2024-09-20T22:03:57.312797+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:03.943982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.314797+0000","last_change":"2024-09-20T22:02:54.294834+0000","last_active":"2024-09-20T22:03:57.314797+0000","last_peered":"2024-09-20T22:03:57.314797+0000","last_clean":"2024-09-20T22:03:57.314797+0000","last_became_active":"2024-09-20T22:02:54.294539+0000","last_became_peered":"2024-09-20T22:02:54.294539+0000","last_unstale":"2024-09-20T22:03:57.314797+0000","last_undegraded":"2024-09-20T22:03:57.314797+0000","last_fullsized":"2024-09-20T22:03:57.314797+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.523368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312510+0000","last_change":"2024-09-20T22:02:56.350065+0000","last_active":"2024-09-20T22:03:57.312510+0000","last_peered":"2024-09-20T22:03:57.312510+0000","last_clean":"2024-09-20T22:03:57.312510+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:03:57.312510+0000","last_undegraded":"2024-09-20T22:03:57.312510+0000","last_fullsized":"2024-09-20T22:03:57.312510+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:19.909723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"71'9","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.309462+0000","last_change":"2024-09-20T22:02:52.482361+0000","last_active":"2024-09-20T22:03:57.309462+0000","last_peered":"2024-09-20T22:03:57.309462+0000","last_clean":"2024-09-20T22:03:57.309462+0000","last_became_active":"2024-09-20T22:02:52.481989+0000","last_became_peered":"2024-09-20T22:02:52.481989+0000","last_unstale":"2024-09-20T22:03:57.309462+0000","last_undegraded":"2024-09-20T22:03:57.309462+0000","last_fullsized":"2024-09-20T22:03:57.309462+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.401767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312194+0000","last_change":"2024-09-20T22:02:50.108943+0000","last_active":"2024-09-20T22:03:57.312194+0000","last_peered":"2024-09-20T22:03:57.312194+0000","last_clean":"2024-09-20T22:03:57.312194+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:03:57.312194+0000","last_undegraded":"2024-09-20T22:03:57.312194+0000","last_fullsized":"2024-09-20T22:03:57.312194+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.114642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312644+0000","last_change":"2024-09-20T22:02:54.298137+0000","last_active":"2024-09-20T22:03:57.312644+0000","last_peered":"2024-09-20T22:03:57.312644+0000","last_clean":"2024-09-20T22:03:57.312644+0000","last_became_active":"2024-09-20T22:02:54.297850+0000","last_became_peered":"2024-09-20T22:02:54.297850+0000","last_unstale":"2024-09-20T22:03:57.312644+0000","last_undegraded":"2024-09-20T22:03:57.312644+0000","last_fullsized":"2024-09-20T22:03:57.312644+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.705512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.317515+0000","last_change":"2024-09-20T22:02:56.299306+0000","last_active":"2024-09-20T22:03:57.317515+0000","last_peered":"2024-09-20T22:03:57.317515+0000","last_clean":"2024-09-20T22:03:57.317515+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:03:57.317515+0000","last_undegraded":"2024-09-20T22:03:57.317515+0000","last_fullsized":"2024-09-20T22:03:57.317515+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:11.499111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"71'9","reported_seq":64,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.315088+0000","last_change":"2024-09-20T22:02:52.482498+0000","last_active":"2024-09-20T22:03:57.315088+0000","last_peered":"2024-09-20T22:03:57.315088+0000","last_clean":"2024-09-20T22:03:57.315088+0000","last_became_active":"2024-09-20T22:02:52.482141+0000","last_became_peered":"2024-09-20T22:02:52.482141+0000","last_unstale":"2024-09-20T22:03:57.315088+0000","last_undegraded":"2024-09-20T22:03:57.315088+0000","last_fullsized":"2024-09-20T22:03:57.315088+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.093555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.308204+0000","last_change":"2024-09-20T22:02:50.112533+0000","last_active":"2024-09-20T22:03:57.308204+0000","last_peered":"2024-09-20T22:03:57.308204+0000","last_clean":"2024-09-20T22:03:57.308204+0000","last_became_active":"2024-09-20T22:02:50.110075+0000","last_became_peered":"2024-09-20T22:02:50.110075+0000","last_unstale":"2024-09-20T22:03:57.308204+0000","last_undegraded":"2024-09-20T22:03:57.308204+0000","last_fullsized":"2024-09-20T22:03:57.308204+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:01.065639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"71'8","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.309085+0000","last_change":"2024-09-20T22:02:54.294653+0000","last_active":"2024-09-20T22:03:57.309085+0000","last_peered":"2024-09-20T22:03:57.309085+0000","last_clean":"2024-09-20T22:03:57.309085+0000","last_became_active":"2024-09-20T22:02:54.294381+0000","last_became_peered":"2024-09-20T22:02:54.294381+0000","last_unstale":"2024-09-20T22:03:57.309085+0000","last_undegraded":"2024-09-20T22:03:57.309085+0000","last_fullsized":"2024-09-20T22:03:57.309085+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.135221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.307310+0000","last_change":"2024-09-20T22:02:56.300405+0000","last_active":"2024-09-20T22:03:57.307310+0000","last_peered":"2024-09-20T22:03:57.307310+0000","last_clean":"2024-09-20T22:03:57.307310+0000","last_became_active":"2024-09-20T22:02:56.300041+0000","last_became_peered":"2024-09-20T22:02:56.300041+0000","last_unstale":"2024-09-20T22:03:57.307310+0000","last_undegraded":"2024-09-20T22:03:57.307310+0000","last_fullsized":"2024-09-20T22:03:57.307310+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:09.312552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"71'10","reported_seq":63,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.309202+0000","last_change":"2024-09-20T22:02:52.480909+0000","last_active":"2024-09-20T22:03:57.309202+0000","last_peered":"2024-09-20T22:03:57.309202+0000","last_clean":"2024-09-20T22:03:57.309202+0000","last_became_active":"2024-09-20T22:02:52.480591+0000","last_became_peered":"2024-09-20T22:02:52.480591+0000","last_unstale":"2024-09-20T22:03:57.309202+0000","last_undegraded":"2024-09-20T22:03:57.309202+0000","last_fullsized":"2024-09-20T22:03:57.309202+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.915698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.313122+0000","last_change":"2024-09-20T22:02:50.112976+0000","last_active":"2024-09-20T22:03:57.313122+0000","last_peered":"2024-09-20T22:03:57.313122+0000","last_clean":"2024-09-20T22:03:57.313122+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:03:57.313122+0000","last_undegraded":"2024-09-20T22:03:57.313122+0000","last_fullsized":"2024-09-20T22:03:57.313122+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.057376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"71'8","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.313774+0000","last_change":"2024-09-20T22:02:54.292909+0000","last_active":"2024-09-20T22:03:57.313774+0000","last_peered":"2024-09-20T22:03:57.313774+0000","last_clean":"2024-09-20T22:03:57.313774+0000","last_became_active":"2024-09-20T22:02:54.289548+0000","last_became_peered":"2024-09-20T22:02:54.289548+0000","last_unstale":"2024-09-20T22:03:57.313774+0000","last_undegraded":"2024-09-20T22:03:57.313774+0000","last_fullsized":"2024-09-20T22:03:57.313774+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:21.478336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.308940+0000","last_change":"2024-09-20T22:02:56.305690+0000","last_active":"2024-09-20T22:03:57.308940+0000","last_peered":"2024-09-20T22:03:57.308940+0000","last_clean":"2024-09-20T22:03:57.308940+0000","last_became_active":"2024-09-20T22:02:56.305383+0000","last_became_peered":"2024-09-20T22:02:56.305383+0000","last_unstale":"2024-09-20T22:03:57.308940+0000","last_undegraded":"2024-09-20T22:03:57.308940+0000","last_fullsized":"2024-09-20T22:03:57.308940+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.293958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"71'11","reported_seq":68,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.306843+0000","last_change":"2024-09-20T22:02:52.277987+0000","last_active":"2024-09-20T22:03:57.306843+0000","last_peered":"2024-09-20T22:03:57.306843+0000","last_clean":"2024-09-20T22:03:57.306843+0000","last_became_active":"2024-09-20T22:02:52.277479+0000","last_became_peered":"2024-09-20T22:02:52.277479+0000","last_unstale":"2024-09-20T22:03:57.306843+0000","last_undegraded":"2024-09-20T22:03:57.306843+0000","last_fullsized":"2024-09-20T22:03:57.306843+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:04.222570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.307525+0000","last_change":"2024-09-20T22:02:50.109201+0000","last_active":"2024-09-20T22:03:57.307525+0000","last_peered":"2024-09-20T22:03:57.307525+0000","last_clean":"2024-09-20T22:03:57.307525+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:03:57.307525+0000","last_undegraded":"2024-09-20T22:03:57.307525+0000","last_fullsized":"2024-09-20T22:03:57.307525+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.129228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.313220+0000","last_change":"2024-09-20T22:02:54.286502+0000","last_active":"2024-09-20T22:03:57.313220+0000","last_peered":"2024-09-20T22:03:57.313220+0000","last_clean":"2024-09-20T22:03:57.313220+0000","last_became_active":"2024-09-20T22:02:54.286200+0000","last_became_peered":"2024-09-20T22:02:54.286200+0000","last_unstale":"2024-09-20T22:03:57.313220+0000","last_undegraded":"2024-09-20T22:03:57.313220+0000","last_fullsized":"2024-09-20T22:03:57.313220+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:22.326441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.308495+0000","last_change":"2024-09-20T22:02:56.307258+0000","last_active":"2024-09-20T22:03:57.308495+0000","last_peered":"2024-09-20T22:03:57.308495+0000","last_clean":"2024-09-20T22:03:57.308495+0000","last_became_active":"2024-09-20T22:02:56.306892+0000","last_became_peered":"2024-09-20T22:02:56.306892+0000","last_unstale":"2024-09-20T22:03:57.308495+0000","last_undegraded":"2024-09-20T22:03:57.308495+0000","last_fullsized":"2024-09-20T22:03:57.308495+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:18.948765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"71'9","reported_seq":65,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.313459+0000","last_change":"2024-09-20T22:02:52.280815+0000","last_active":"2024-09-20T22:03:57.313459+0000","last_peered":"2024-09-20T22:03:57.313459+0000","last_clean":"2024-09-20T22:03:57.313459+0000","last_became_active":"2024-09-20T22:02:52.280428+0000","last_became_peered":"2024-09-20T22:02:52.280428+0000","last_unstale":"2024-09-20T22:03:57.313459+0000","last_undegraded":"2024-09-20T22:03:57.313459+0000","last_fullsized":"2024-09-20T22:03:57.313459+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:10.548215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":53,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312749+0000","last_change":"2024-09-20T22:02:50.110105+0000","last_active":"2024-09-20T22:03:57.312749+0000","last_peered":"2024-09-20T22:03:57.312749+0000","last_clean":"2024-09-20T22:03:57.312749+0000","last_became_active":"2024-09-20T22:02:50.109803+0000","last_became_peered":"2024-09-20T22:02:50.109803+0000","last_unstale":"2024-09-20T22:03:57.312749+0000","last_undegraded":"2024-09-20T22:03:57.312749+0000","last_fullsized":"2024-09-20T22:03:57.312749+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.674290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.310514+0000","last_change":"2024-09-20T22:02:54.298130+0000","last_active":"2024-09-20T22:03:57.310514+0000","last_peered":"2024-09-20T22:03:57.310514+0000","last_clean":"2024-09-20T22:03:57.310514+0000","last_became_active":"2024-09-20T22:02:54.297803+0000","last_became_peered":"2024-09-20T22:02:54.297803+0000","last_unstale":"2024-09-20T22:03:57.310514+0000","last_undegraded":"2024-09-20T22:03:57.310514+0000","last_fullsized":"2024-09-20T22:03:57.310514+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:15.520061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.307364+0000","last_change":"2024-09-20T22:02:56.298228+0000","last_active":"2024-09-20T22:03:57.307364+0000","last_peered":"2024-09-20T22:03:57.307364+0000","last_clean":"2024-09-20T22:03:57.307364+0000","last_became_active":"2024-09-20T22:02:56.297826+0000","last_became_peered":"2024-09-20T22:02:56.297826+0000","last_unstale":"2024-09-20T22:03:57.307364+0000","last_undegraded":"2024-09-20T22:03:57.307364+0000","last_fullsized":"2024-09-20T22:03:57.307364+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.126658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"71'11","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.308619+0000","last_change":"2024-09-20T22:02:52.480504+0000","last_active":"2024-09-20T22:03:57.308619+0000","last_peered":"2024-09-20T22:03:57.308619+0000","last_clean":"2024-09-20T22:03:57.308619+0000","last_became_active":"2024-09-20T22:02:52.480118+0000","last_became_peered":"2024-09-20T22:02:52.480118+0000","last_unstale":"2024-09-20T22:03:57.308619+0000","last_undegraded":"2024-09-20T22:03:57.308619+0000","last_fullsized":"2024-09-20T22:03:57.308619+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.023916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.314883+0000","last_change":"2024-09-20T22:02:50.110808+0000","last_active":"2024-09-20T22:03:57.314883+0000","last_peered":"2024-09-20T22:03:57.314883+0000","last_clean":"2024-09-20T22:03:57.314883+0000","last_became_active":"2024-09-20T22:02:50.110421+0000","last_became_peered":"2024-09-20T22:02:50.110421+0000","last_unstale":"2024-09-20T22:03:57.314883+0000","last_undegraded":"2024-09-20T22:03:57.314883+0000","last_fullsized":"2024-09-20T22:03:57.314883+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":45,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.311804+0000","last_change":"2024-09-20T22:02:54.293068+0000","last_active":"2024-09-20T22:03:57.311804+0000","last_peered":"2024-09-20T22:03:57.311804+0000","last_clean":"2024-09-20T22:03:57.311804+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:03:57.311804+0000","last_undegraded":"2024-09-20T22:03:57.311804+0000","last_fullsized":"2024-09-20T22:03:57.311804+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.450710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.310676+0000","last_change":"2024-09-20T22:02:56.348065+0000","last_active":"2024-09-20T22:03:57.310676+0000","last_peered":"2024-09-20T22:03:57.310676+0000","last_clean":"2024-09-20T22:03:57.310676+0000","last_became_active":"2024-09-20T22:02:56.347637+0000","last_became_peered":"2024-09-20T22:02:56.347637+0000","last_unstale":"2024-09-20T22:03:57.310676+0000","last_undegraded":"2024-09-20T22:03:57.310676+0000","last_fullsized":"2024-09-20T22:03:57.310676+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:13.568557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"71'4","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312684+0000","last_change":"2024-09-20T22:02:52.286086+0000","last_active":"2024-09-20T22:03:57.312684+0000","last_peered":"2024-09-20T22:03:57.312684+0000","last_clean":"2024-09-20T22:03:57.312684+0000","last_became_active":"2024-09-20T22:02:52.285701+0000","last_became_peered":"2024-09-20T22:02:52.285701+0000","last_unstale":"2024-09-20T22:03:57.312684+0000","last_undegraded":"2024-09-20T22:03:57.312684+0000","last_fullsized":"2024-09-20T22:03:57.312684+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:07.474692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.309514+0000","last_change":"2024-09-20T22:02:50.111749+0000","last_active":"2024-09-20T22:03:57.309514+0000","last_peered":"2024-09-20T22:03:57.309514+0000","last_clean":"2024-09-20T22:03:57.309514+0000","last_became_active":"2024-09-20T22:02:50.111142+0000","last_became_peered":"2024-09-20T22:02:50.111142+0000","last_unstale":"2024-09-20T22:03:57.309514+0000","last_undegraded":"2024-09-20T22:03:57.309514+0000","last_fullsized":"2024-09-20T22:03:57.309514+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.313915+0000","last_change":"2024-09-20T22:02:54.290418+0000","last_active":"2024-09-20T22:03:57.313915+0000","last_peered":"2024-09-20T22:03:57.313915+0000","last_clean":"2024-09-20T22:03:57.313915+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:03:57.313915+0000","last_undegraded":"2024-09-20T22:03:57.313915+0000","last_fullsized":"2024-09-20T22:03:57.313915+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.312170+0000","last_change":"2024-09-20T22:02:56.300559+0000","last_active":"2024-09-20T22:03:57.312170+0000","last_peered":"2024-09-20T22:03:57.312170+0000","last_clean":"2024-09-20T22:03:57.312170+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:03:57.312170+0000","last_undegraded":"2024-09-20T22:03:57.312170+0000","last_fullsized":"2024-09-20T22:03:57.312170+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:17.506369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.317804+0000","last_change":"2024-09-20T22:02:56.301611+0000","last_active":"2024-09-20T22:03:57.317804+0000","last_peered":"2024-09-20T22:03:57.317804+0000","last_clean":"2024-09-20T22:03:57.317804+0000","last_became_active":"2024-09-20T22:02:56.301247+0000","last_became_peered":"2024-09-20T22:02:56.301247+0000","last_unstale":"2024-09-20T22:03:57.317804+0000","last_undegraded":"2024-09-20T22:03:57.317804+0000","last_fullsized":"2024-09-20T22:03:57.317804+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:02.177919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":52,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.313166+0000","last_change":"2024-09-20T22:02:50.114519+0000","last_active":"2024-09-20T22:03:57.313166+0000","last_peered":"2024-09-20T22:03:57.313166+0000","last_clean":"2024-09-20T22:03:57.313166+0000","last_became_active":"2024-09-20T22:02:50.114204+0000","last_became_peered":"2024-09-20T22:02:50.114204+0000","last_unstale":"2024-09-20T22:03:57.313166+0000","last_undegraded":"2024-09-20T22:03:57.313166+0000","last_fullsized":"2024-09-20T22:03:57.313166+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:57.854945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"71'11","reported_seq":67,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.315190+0000","last_change":"2024-09-20T22:02:52.279065+0000","last_active":"2024-09-20T22:03:57.315190+0000","last_peered":"2024-09-20T22:03:57.315190+0000","last_clean":"2024-09-20T22:03:57.315190+0000","last_became_active":"2024-09-20T22:02:52.278657+0000","last_became_peered":"2024-09-20T22:02:52.278657+0000","last_unstale":"2024-09-20T22:03:57.315190+0000","last_undegraded":"2024-09-20T22:03:57.315190+0000","last_fullsized":"2024-09-20T22:03:57.315190+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:03:58.405988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":44,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-09-20T22:03:57.309188+0000","last_change":"2024-09-20T22:02:54.289401+0000","last_active":"2024-09-20T22:03:57.309188+0000","last_peered":"2024-09-20T22:03:57.309188+0000","last_clean":"2024-09-20T22:03:57.309188+0000","last_became_active":"2024-09-20T22:02:54.289078+0000","last_became_peered":"2024-09-20T22:02:54.289078+0000","last_unstale":"2024-09-20T22:03:57.309188+0000","last_undegraded":"2024-09-20T22:03:57.309188+0000","last_fullsized":"2024-09-20T22:03:57.309188+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:04:06.343711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":629866496,"num_objects":9611,"num_object_clones":0,"num_object_copies":28833,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9611,"num_write_kb":615104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1891500032,"data_stored":1891500032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9611,"ondisk_log_size":9611,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":393,"ondisk_log_size":393,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":64,"num_read_kb":59,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135890,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":197556,"kb_used_data":158052,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93547596,"statfs":{"total":95995035648,"available":95792738304,"internally_reserved":0,"allocated":161845248,"data_stored":161370107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":40434122},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430230,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":228132,"kb_used_data":188628,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93517020,"statfs":{"total":95995035648,"available":95761428480,"internally_reserved":0,"allocated":193155072,"data_stored":192696223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724570,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":143204,"kb_used_data":107856,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93601948,"statfs":{"total":95995035648,"available":95848394752,"internally_reserved":0,"allocated":110444544,"data_stored":109989593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986210,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":198140,"kb_used_data":158632,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93547012,"statfs":{"total":95995035648,"available":95792140288,"internally_reserved":0,"allocated":162439168,"data_stored":161959330,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574897,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":247832,"kb_used_data":208328,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93497320,"statfs":{"total":95995035648,"available":95741255680,"internally_reserved":0,"allocated":213327872,"data_stored":212816010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149735,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":318728,"kb_used_data":275064,"kb_used_omap":1,"kb_used_meta":43646,"kb_avail":93426424,"statfs":{"total":95995035648,"available":95668658176,"internally_reserved":0,"allocated":281665536,"data_stored":281170606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":44693959},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,3,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411372,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":205524,"kb_used_data":170180,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93539628,"statfs":{"total":95995035648,"available":95784579072,"internally_reserved":0,"allocated":174264320,"data_stored":173820453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738423,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":305656,"kb_used_data":278632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93439496,"statfs":{"total":95995035648,"available":95682043904,"internally_reserved":0,"allocated":285319168,"data_stored":284840063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":317521920,"data_stored":317521920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":236650496,"data_stored":236650496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":237502464,"data_stored":237502464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":354549760,"data_stored":354549760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":195493888,"data_stored":195493888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":151519232,"data_stored":151519232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":196214784,"data_stored":196214784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":202047488,"data_stored":202047488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:04:08.523 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj -9, currently 1 2024-09-20T22:04:08.588 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:08.629 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:08.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:08.630 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:08.731 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:08.776 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:08.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:08.777 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:08 smithi060 ceph-mon[29694]: pgmap v41: 148 pgs: 148 active+clean; 1.3 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 820 B/s rd, 116 MiB/s wr, 1.86k op/s 2024-09-20T22:04:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:08 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:08 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:08 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:08 smithi060 ceph-mon[29694]: Deploying daemon node-exporter.b on smithi067 2024-09-20T22:04:08.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:08 smithi060 ceph-mon[40290]: pgmap v41: 148 pgs: 148 active+clean; 1.3 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 820 B/s rd, 116 MiB/s wr, 1.86k op/s 2024-09-20T22:04:08.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:08 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:08.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:08 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:08.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:08 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:08.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:08 smithi060 ceph-mon[40290]: Deploying daemon node-exporter.b on smithi067 2024-09-20T22:04:08.861 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:08 smithi067 ceph-mon[33511]: pgmap v41: 148 pgs: 148 active+clean; 1.3 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 820 B/s rd, 116 MiB/s wr, 1.86k op/s 2024-09-20T22:04:08.861 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:08 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:08.862 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:08 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:08.862 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:08 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:08.862 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:08 smithi067 ceph-mon[33511]: Deploying daemon node-exporter.b on smithi067 2024-09-20T22:04:08.878 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:08.913 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:08.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:08.914 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:09.015 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:09.060 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:09.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:09.061 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:09.162 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:09.192 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:08 smithi067 systemd[1]: Starting Ceph node-exporter.b for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:04:09.195 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:09.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:09.196 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:09.270 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 12 16 28760 28744 149.693 122.25 0.00501681 0.0066745 2024-09-20T22:04:09.297 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:09.331 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:09.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:09.332 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:09.332 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:09.369 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T22:04:09.432 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:09.463 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:09.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:09.464 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:09.513 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:09 smithi067 bash[66864]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-20T22:04:09.565 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:09.610 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:09.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:09.612 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:09.712 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:09.743 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:09.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:09.744 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:09.744 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:04:09.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:09.875 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:09.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:09.876 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:09.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:09 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/725836893' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:04:09.942 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:09 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/725836893' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:04:09.977 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:10.009 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:10.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:10.010 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:10.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:09 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/725836893' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:04:10.014 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:09 smithi067 bash[66864]: Getting image source signatures 2024-09-20T22:04:10.014 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:09 smithi067 bash[66864]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-20T22:04:10.014 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:09 smithi067 bash[66864]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-20T22:04:10.014 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:09 smithi067 bash[66864]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-20T22:04:10.110 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:10.151 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:10.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:10.153 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:10.254 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:10.270 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 13 16 30647 30631 147.249 117.938 0.0046603 0.00678472 2024-09-20T22:04:10.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:10.292 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:10.292 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:10.393 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:10.430 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:10.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:10.431 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:10.532 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:10.564 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:10.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:10.565 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:10.666 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:04:10.689 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:10.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:10.690 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:04:10.790 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:10.828 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:10.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:10.829 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:10.930 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:10.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:10 smithi060 ceph-mon[29694]: pgmap v42: 148 pgs: 148 active+clean; 1.3 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 90 MiB/s wr, 1.45k op/s 2024-09-20T22:04:10.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:10 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:04:10.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:10 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/725836893' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:04:10.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:10 smithi060 ceph-mon[29694]: osdmap e89: 8 total, 8 up, 8 in 2024-09-20T22:04:10.939 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:10 smithi060 ceph-mon[40290]: pgmap v42: 148 pgs: 148 active+clean; 1.3 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 90 MiB/s wr, 1.45k op/s 2024-09-20T22:04:10.939 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:10 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:04:10.939 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:10 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/725836893' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:04:10.939 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:10 smithi060 ceph-mon[40290]: osdmap e89: 8 total, 8 up, 8 in 2024-09-20T22:04:10.960 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:10.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:10.961 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:10 smithi067 ceph-mon[33511]: pgmap v42: 148 pgs: 148 active+clean; 1.3 GiB data, 3.9 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 90 MiB/s wr, 1.45k op/s 2024-09-20T22:04:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:10 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:04:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:10 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/725836893' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:04:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:10 smithi067 ceph-mon[33511]: osdmap e89: 8 total, 8 up, 8 in 2024-09-20T22:04:11.062 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:11.087 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:11.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:11.088 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:11.106 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:04:11.106 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:04:11.124 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:04:11.124 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:04:11.150 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:11.151 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:04:11.151 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:04:11.189 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:11.196 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:04:11.196 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:04:11.231 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:04:11.231 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:04:11.244 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:11.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:11.245 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:11.266 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:04:11.266 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:04:11.271 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 14 16 32457 32441 144.805 113.125 0.0108588 0.00689981 2024-09-20T22:04:11.300 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:04:11.300 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:04:11.336 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:04:11.336 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:04:11.346 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:11.371 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:11.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:11.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:11.473 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:11.520 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:11.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:11.521 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:11.622 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:04:11.663 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:11.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:11.664 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:04:11.740 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[40290]: 6.2 scrub starts 2024-09-20T22:04:11.740 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[40290]: 6.2 scrub ok 2024-09-20T22:04:11.741 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[40290]: 4.1c scrub starts 2024-09-20T22:04:11.741 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[40290]: 4.1c scrub ok 2024-09-20T22:04:11.741 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[40290]: 6.1f scrub starts 2024-09-20T22:04:11.741 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[40290]: 6.1f scrub ok 2024-09-20T22:04:11.741 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3039286292' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:04:11.741 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[40290]: 5.19 scrub starts 2024-09-20T22:04:11.741 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[40290]: 5.19 scrub ok 2024-09-20T22:04:11.742 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[40290]: osdmap e90: 8 total, 8 up, 8 in 2024-09-20T22:04:11.742 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[29694]: 6.2 scrub starts 2024-09-20T22:04:11.742 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[29694]: 6.2 scrub ok 2024-09-20T22:04:11.742 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[29694]: 4.1c scrub starts 2024-09-20T22:04:11.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[29694]: 4.1c scrub ok 2024-09-20T22:04:11.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[29694]: 6.1f scrub starts 2024-09-20T22:04:11.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[29694]: 6.1f scrub ok 2024-09-20T22:04:11.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3039286292' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:04:11.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[29694]: 5.19 scrub starts 2024-09-20T22:04:11.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[29694]: 5.19 scrub ok 2024-09-20T22:04:11.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:11 smithi060 ceph-mon[29694]: osdmap e90: 8 total, 8 up, 8 in 2024-09-20T22:04:11.766 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:11.794 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:11.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:11.795 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:11.896 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:11.925 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:04:11.925 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:04:11.941 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:11.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:11.942 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:12.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:04:11 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:04:12.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:11 smithi067 ceph-mon[33511]: 6.2 scrub starts 2024-09-20T22:04:12.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:11 smithi067 ceph-mon[33511]: 6.2 scrub ok 2024-09-20T22:04:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:11 smithi067 ceph-mon[33511]: 4.1c scrub starts 2024-09-20T22:04:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:11 smithi067 ceph-mon[33511]: 4.1c scrub ok 2024-09-20T22:04:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:11 smithi067 ceph-mon[33511]: 6.1f scrub starts 2024-09-20T22:04:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:11 smithi067 ceph-mon[33511]: 6.1f scrub ok 2024-09-20T22:04:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:11 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3039286292' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:04:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:11 smithi067 ceph-mon[33511]: 5.19 scrub starts 2024-09-20T22:04:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:11 smithi067 ceph-mon[33511]: 5.19 scrub ok 2024-09-20T22:04:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:11 smithi067 ceph-mon[33511]: osdmap e90: 8 total, 8 up, 8 in 2024-09-20T22:04:12.043 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:12.071 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:12.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:12.072 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:12.173 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:12.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:12.205 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:12.205 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:12.271 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 15 16 34357 34341 143.067 118.75 0.00469075 0.00698384 2024-09-20T22:04:12.305 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:12.334 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:12.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:12.335 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:12.436 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:12.472 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:12.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:12.473 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:12.513 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 bash[66864]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-20T22:04:12.513 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 bash[66864]: Writing manifest to image destination 2024-09-20T22:04:12.574 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:12.613 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:12.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:12.614 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:12.715 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:12.745 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:12.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:12.746 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[29694]: 4.18 scrub starts 2024-09-20T22:04:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[29694]: 4.18 scrub ok 2024-09-20T22:04:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[29694]: 4.3 scrub starts 2024-09-20T22:04:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[29694]: 4.3 scrub ok 2024-09-20T22:04:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[29694]: pgmap v45: 148 pgs: 148 active+clean; 1.9 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 169 MiB/s wr, 2.70k op/s 2024-09-20T22:04:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[29694]: 3.2 scrub starts 2024-09-20T22:04:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:04:12.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[29694]: 3.2 scrub ok 2024-09-20T22:04:12.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[29694]: 4.12 scrub starts 2024-09-20T22:04:12.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[29694]: 4.12 scrub ok 2024-09-20T22:04:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[40290]: 4.18 scrub starts 2024-09-20T22:04:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[40290]: 4.18 scrub ok 2024-09-20T22:04:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[40290]: 4.3 scrub starts 2024-09-20T22:04:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[40290]: 4.3 scrub ok 2024-09-20T22:04:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[40290]: pgmap v45: 148 pgs: 148 active+clean; 1.9 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 169 MiB/s wr, 2.70k op/s 2024-09-20T22:04:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[40290]: 3.2 scrub starts 2024-09-20T22:04:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:04:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[40290]: 3.2 scrub ok 2024-09-20T22:04:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[40290]: 4.12 scrub starts 2024-09-20T22:04:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:12 smithi060 ceph-mon[40290]: 4.12 scrub ok 2024-09-20T22:04:12.846 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:12.873 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:04:12.875 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:12.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:12.876 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:12.905 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 podman[66864]: 2024-09-20 22:04:12.520130197 +0000 UTC m=+3.500876277 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-20T22:04:12.905 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 podman[66864]: 2024-09-20 22:04:12.626520373 +0000 UTC m=+3.607266447 container create f3ca125f7fc6e953697d711936758d1eb27bbda33bcb15fdf6c5db591d67b348 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-20T22:04:12.905 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-mon[33511]: 4.18 scrub starts 2024-09-20T22:04:12.905 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-mon[33511]: 4.18 scrub ok 2024-09-20T22:04:12.905 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-mon[33511]: 4.3 scrub starts 2024-09-20T22:04:12.905 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-mon[33511]: 4.3 scrub ok 2024-09-20T22:04:12.905 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-mon[33511]: pgmap v45: 148 pgs: 148 active+clean; 1.9 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 169 MiB/s wr, 2.70k op/s 2024-09-20T22:04:12.906 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-mon[33511]: 3.2 scrub starts 2024-09-20T22:04:12.906 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:04:12.906 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-mon[33511]: 3.2 scrub ok 2024-09-20T22:04:12.906 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-mon[33511]: 4.12 scrub starts 2024-09-20T22:04:12.906 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-mon[33511]: 4.12 scrub ok 2024-09-20T22:04:12.977 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:13.003 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:13.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:13.005 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:13.105 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:04:13.154 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:13.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:13.155 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:04:13.256 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:13.264 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 podman[66864]: 2024-09-20 22:04:12.904509065 +0000 UTC m=+3.885255143 container init f3ca125f7fc6e953697d711936758d1eb27bbda33bcb15fdf6c5db591d67b348 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-20T22:04:13.264 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 podman[66864]: 2024-09-20 22:04:12.911876084 +0000 UTC m=+3.892622159 container start f3ca125f7fc6e953697d711936758d1eb27bbda33bcb15fdf6c5db591d67b348 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-20T22:04:13.264 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.917Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-20T22:04:13.264 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.917Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-20T22:04:13.264 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.917Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-20T22:04:13.264 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.917Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-20T22:04:13.264 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.918Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-20T22:04:13.264 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.918Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-20T22:04:13.264 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-20T22:04:13.264 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=arp 2024-09-20T22:04:13.264 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-20T22:04:13.264 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-20T22:04:13.264 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-20T22:04:13.265 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-20T22:04:13.265 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-20T22:04:13.265 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-20T22:04:13.265 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-20T22:04:13.265 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-20T22:04:13.265 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=edac 2024-09-20T22:04:13.265 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-20T22:04:13.266 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-20T22:04:13.266 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-20T22:04:13.266 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-20T22:04:13.266 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-20T22:04:13.266 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-20T22:04:13.266 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-20T22:04:13.266 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-20T22:04:13.266 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-20T22:04:13.266 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-20T22:04:13.266 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-20T22:04:13.266 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-20T22:04:13.266 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-20T22:04:13.266 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-20T22:04:13.266 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-20T22:04:13.267 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-20T22:04:13.267 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=os 2024-09-20T22:04:13.267 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-20T22:04:13.267 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-20T22:04:13.267 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-20T22:04:13.267 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-20T22:04:13.267 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-20T22:04:13.267 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-20T22:04:13.267 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-20T22:04:13.267 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=stat 2024-09-20T22:04:13.267 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-20T22:04:13.267 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-20T22:04:13.267 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-20T22:04:13.267 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=time 2024-09-20T22:04:13.268 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-20T22:04:13.268 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=uname 2024-09-20T22:04:13.268 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-20T22:04:13.268 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-20T22:04:13.268 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.919Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-20T22:04:13.268 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.920Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-20T22:04:13.268 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b[67193]: ts=2024-09-20T22:04:12.920Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-20T22:04:13.268 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:13 smithi067 bash[66864]: f3ca125f7fc6e953697d711936758d1eb27bbda33bcb15fdf6c5db591d67b348 2024-09-20T22:04:13.268 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:04:13 smithi067 systemd[1]: Started Ceph node-exporter.b for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T22:04:13.273 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 16 16 36146 36130 141.102 111.812 0.00778831 0.00708184 2024-09-20T22:04:13.288 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:13.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:13.290 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:13.390 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:13.425 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:13.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:13.426 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:13.524 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T22:04:13.525 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:04:13.525 INFO:tasks.thrashosds.thrasher:Killing osd 1, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T22:04:13.525 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-20T22:04:13.525 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.1 2024-09-20T22:04:13.527 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:13.557 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:13.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:13.557 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:13.658 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:13.713 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:13.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:13.714 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:13.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:13.852 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:13.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:13.853 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:13.954 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:13.997 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:13.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:13.998 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:13 smithi067 ceph-mon[33511]: 5.d scrub starts 2024-09-20T22:04:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:13 smithi067 ceph-mon[33511]: 5.d scrub ok 2024-09-20T22:04:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:13 smithi067 ceph-mon[33511]: 3.5 scrub starts 2024-09-20T22:04:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:13 smithi067 ceph-mon[33511]: 3.5 scrub ok 2024-09-20T22:04:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:13 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:13 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:13 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:13 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:13 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:13 smithi067 ceph-mon[33511]: 4.f scrub starts 2024-09-20T22:04:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:13 smithi067 ceph-mon[33511]: 4.f scrub ok 2024-09-20T22:04:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:13 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:04:14.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[29694]: 5.d scrub starts 2024-09-20T22:04:14.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[29694]: 5.d scrub ok 2024-09-20T22:04:14.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[29694]: 3.5 scrub starts 2024-09-20T22:04:14.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[29694]: 3.5 scrub ok 2024-09-20T22:04:14.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:14.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:14.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:14.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:14.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:14.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[29694]: 4.f scrub starts 2024-09-20T22:04:14.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[29694]: 4.f scrub ok 2024-09-20T22:04:14.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:04:14.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[40290]: 5.d scrub starts 2024-09-20T22:04:14.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[40290]: 5.d scrub ok 2024-09-20T22:04:14.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[40290]: 3.5 scrub starts 2024-09-20T22:04:14.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[40290]: 3.5 scrub ok 2024-09-20T22:04:14.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:14.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:14.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:14.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:14.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:14.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[40290]: 4.f scrub starts 2024-09-20T22:04:14.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[40290]: 4.f scrub ok 2024-09-20T22:04:14.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:13 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:04:14.038 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:04:13 smithi060 systemd[1]: Stopping Ceph osd.1 for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:04:14.100 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:14.136 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:14.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:14.138 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:14.238 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:14.273 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 17 16 37739 37723 138.658 99.5625 0.00410393 0.00720572 2024-09-20T22:04:14.274 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:14.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:14.275 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:14.376 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:14.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:14.409 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:14.409 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:14.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:14.520 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:04:14.552 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:14.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:14.554 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:14.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:14.702 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:14.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:14.703 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:14.804 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:14.838 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:14.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:14.839 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:14.939 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:14.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:14 smithi060 ceph-mon[29694]: 5.8 scrub starts 2024-09-20T22:04:14.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:14 smithi060 ceph-mon[29694]: 5.8 scrub ok 2024-09-20T22:04:14.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:14 smithi060 ceph-mon[29694]: 4.d scrub starts 2024-09-20T22:04:14.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:14 smithi060 ceph-mon[29694]: 4.d scrub ok 2024-09-20T22:04:14.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:14 smithi060 ceph-mon[29694]: 3.12 scrub starts 2024-09-20T22:04:14.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:14 smithi060 ceph-mon[29694]: 3.12 scrub ok 2024-09-20T22:04:14.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:14 smithi060 ceph-mon[29694]: pgmap v46: 148 pgs: 148 active+clean; 1.9 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 152 MiB/s wr, 2.44k op/s 2024-09-20T22:04:14.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:14 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:14.958 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:14 smithi060 ceph-mon[40290]: 5.8 scrub starts 2024-09-20T22:04:14.959 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:14 smithi060 ceph-mon[40290]: 5.8 scrub ok 2024-09-20T22:04:14.959 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:14 smithi060 ceph-mon[40290]: 4.d scrub starts 2024-09-20T22:04:14.959 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:14 smithi060 ceph-mon[40290]: 4.d scrub ok 2024-09-20T22:04:14.959 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:14 smithi060 ceph-mon[40290]: 3.12 scrub starts 2024-09-20T22:04:14.959 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:14 smithi060 ceph-mon[40290]: 3.12 scrub ok 2024-09-20T22:04:14.959 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:14 smithi060 ceph-mon[40290]: pgmap v46: 148 pgs: 148 active+clean; 1.9 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 152 MiB/s wr, 2.44k op/s 2024-09-20T22:04:14.959 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:14 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:14.960 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T22:04:14.968 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:14.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:14.969 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:14 smithi067 ceph-mon[33511]: 5.8 scrub starts 2024-09-20T22:04:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:14 smithi067 ceph-mon[33511]: 5.8 scrub ok 2024-09-20T22:04:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:14 smithi067 ceph-mon[33511]: 4.d scrub starts 2024-09-20T22:04:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:14 smithi067 ceph-mon[33511]: 4.d scrub ok 2024-09-20T22:04:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:14 smithi067 ceph-mon[33511]: 3.12 scrub starts 2024-09-20T22:04:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:14 smithi067 ceph-mon[33511]: 3.12 scrub ok 2024-09-20T22:04:15.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:14 smithi067 ceph-mon[33511]: pgmap v46: 148 pgs: 148 active+clean; 1.9 GiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 152 MiB/s wr, 2.44k op/s 2024-09-20T22:04:15.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:14 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:15.070 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:15.099 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:15.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:15.100 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:15.201 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:15.214 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:15.214 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:15.225 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:15.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:15.226 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:15.273 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 18 16 39336 39320 136.493 99.8125 0.0130371 0.0073184 2024-09-20T22:04:15.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:04:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:04:15.286 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1[50412]: 2024-09-20T22:04:15.137+0000 7f6bd61fc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T22:04:15.286 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1[50412]: 2024-09-20T22:04:15.137+0000 7f6bd61fc640 -1 osd.1 90 *** Got signal Terminated *** 2024-09-20T22:04:15.286 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1[50412]: 2024-09-20T22:04:15.137+0000 7f6bd61fc640 -1 osd.1 90 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T22:04:15.327 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:15.356 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:15.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:15.357 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:15.458 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:15.493 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:15.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:15.494 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:15.595 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:15.632 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:15.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:15.634 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:15.734 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:15.757 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:15.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:15.758 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:15.859 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:15.881 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:15.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:15.882 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:15.983 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:16.006 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:16.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:16.007 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:16.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:15 smithi067 ceph-mon[33511]: 4.1b scrub starts 2024-09-20T22:04:16.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:15 smithi067 ceph-mon[33511]: 4.1b scrub ok 2024-09-20T22:04:16.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:15 smithi067 ceph-mon[33511]: 3.1 scrub starts 2024-09-20T22:04:16.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:15 smithi067 ceph-mon[33511]: 3.1 scrub ok 2024-09-20T22:04:16.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:15 smithi067 ceph-mon[33511]: 5.1a scrub starts 2024-09-20T22:04:16.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:15 smithi067 ceph-mon[33511]: 5.1a scrub ok 2024-09-20T22:04:16.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:15 smithi067 ceph-mon[33511]: 3.16 scrub starts 2024-09-20T22:04:16.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:15 smithi067 ceph-mon[33511]: 3.16 scrub ok 2024-09-20T22:04:16.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:15 smithi067 ceph-mon[33511]: osd.1 marked itself down and dead 2024-09-20T22:04:16.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:15 smithi067 ceph-mon[33511]: 6.1d scrub starts 2024-09-20T22:04:16.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:15 smithi067 ceph-mon[33511]: 6.1d scrub ok 2024-09-20T22:04:16.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[29694]: 4.1b scrub starts 2024-09-20T22:04:16.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[29694]: 4.1b scrub ok 2024-09-20T22:04:16.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[29694]: 3.1 scrub starts 2024-09-20T22:04:16.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[29694]: 3.1 scrub ok 2024-09-20T22:04:16.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[29694]: 5.1a scrub starts 2024-09-20T22:04:16.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[29694]: 5.1a scrub ok 2024-09-20T22:04:16.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[29694]: 3.16 scrub starts 2024-09-20T22:04:16.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[29694]: 3.16 scrub ok 2024-09-20T22:04:16.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[29694]: osd.1 marked itself down and dead 2024-09-20T22:04:16.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[29694]: 6.1d scrub starts 2024-09-20T22:04:16.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[29694]: 6.1d scrub ok 2024-09-20T22:04:16.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[40290]: 4.1b scrub starts 2024-09-20T22:04:16.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[40290]: 4.1b scrub ok 2024-09-20T22:04:16.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[40290]: 3.1 scrub starts 2024-09-20T22:04:16.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[40290]: 3.1 scrub ok 2024-09-20T22:04:16.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[40290]: 5.1a scrub starts 2024-09-20T22:04:16.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[40290]: 5.1a scrub ok 2024-09-20T22:04:16.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[40290]: 3.16 scrub starts 2024-09-20T22:04:16.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[40290]: 3.16 scrub ok 2024-09-20T22:04:16.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[40290]: osd.1 marked itself down and dead 2024-09-20T22:04:16.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[40290]: 6.1d scrub starts 2024-09-20T22:04:16.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:15 smithi060 ceph-mon[40290]: 6.1d scrub ok 2024-09-20T22:04:16.108 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:16.131 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:16.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:16.132 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:16.232 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:16.255 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:16.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:16.255 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:16.274 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 19 16 40029 40013 131.589 43.3125 0.00231264 0.00737588 2024-09-20T22:04:16.356 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:16.390 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:16.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:16.391 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:16.434 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:04:16 smithi060 podman[80975]: 2024-09-20 22:04:16.412533172 +0000 UTC m=+1.437870798 container died e4a4322245cf1019664093755e53cd118198ef46bc5073e6961b15a013e659d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-9e2ac70, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-20T22:04:16.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:16.526 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:16.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:16.527 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:16.628 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:16.660 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:16.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:16.661 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:16.762 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:16.795 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:16.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:16.796 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:16.897 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:16.927 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:16.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:16.928 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:17.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:16 smithi067 ceph-mon[33511]: pgmap v47: 148 pgs: 148 active+clean; 2.1 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-20T22:04:17.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:16 smithi067 ceph-mon[33511]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-20T22:04:17.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:16 smithi067 ceph-mon[33511]: osdmap e91: 8 total, 7 up, 8 in 2024-09-20T22:04:17.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:16 smithi067 ceph-mon[33511]: 4.19 scrub starts 2024-09-20T22:04:17.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:16 smithi067 ceph-mon[33511]: 4.19 scrub ok 2024-09-20T22:04:17.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:16 smithi067 ceph-mon[33511]: 3.17 scrub starts 2024-09-20T22:04:17.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:16 smithi067 ceph-mon[33511]: 3.17 scrub ok 2024-09-20T22:04:17.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:16 smithi067 ceph-mon[33511]: osdmap e92: 8 total, 7 up, 8 in 2024-09-20T22:04:17.028 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:17.036 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:04:16 smithi060 podman[80975]: 2024-09-20 22:04:16.936888515 +0000 UTC m=+1.962226124 container cleanup e4a4322245cf1019664093755e53cd118198ef46bc5073e6961b15a013e659d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-9e2ac70, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-20T22:04:17.037 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:04:16 smithi060 bash[80975]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1 2024-09-20T22:04:17.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:16 smithi060 ceph-mon[29694]: pgmap v47: 148 pgs: 148 active+clean; 2.1 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-20T22:04:17.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:16 smithi060 ceph-mon[29694]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-20T22:04:17.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:16 smithi060 ceph-mon[29694]: osdmap e91: 8 total, 7 up, 8 in 2024-09-20T22:04:17.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:16 smithi060 ceph-mon[29694]: 4.19 scrub starts 2024-09-20T22:04:17.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:16 smithi060 ceph-mon[29694]: 4.19 scrub ok 2024-09-20T22:04:17.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:16 smithi060 ceph-mon[29694]: 3.17 scrub starts 2024-09-20T22:04:17.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:16 smithi060 ceph-mon[29694]: 3.17 scrub ok 2024-09-20T22:04:17.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:16 smithi060 ceph-mon[29694]: osdmap e92: 8 total, 7 up, 8 in 2024-09-20T22:04:17.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:16 smithi060 ceph-mon[40290]: pgmap v47: 148 pgs: 148 active+clean; 2.1 GiB data, 6.3 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 104 MiB/s wr, 1.67k op/s 2024-09-20T22:04:17.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:16 smithi060 ceph-mon[40290]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-20T22:04:17.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:16 smithi060 ceph-mon[40290]: osdmap e91: 8 total, 7 up, 8 in 2024-09-20T22:04:17.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:16 smithi060 ceph-mon[40290]: 4.19 scrub starts 2024-09-20T22:04:17.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:16 smithi060 ceph-mon[40290]: 4.19 scrub ok 2024-09-20T22:04:17.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:16 smithi060 ceph-mon[40290]: 3.17 scrub starts 2024-09-20T22:04:17.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:16 smithi060 ceph-mon[40290]: 3.17 scrub ok 2024-09-20T22:04:17.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:16 smithi060 ceph-mon[40290]: osdmap e92: 8 total, 7 up, 8 in 2024-09-20T22:04:17.071 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:17.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:17.072 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:17.173 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:17.204 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:17.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:17.205 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:17.274 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:04:17.273630+0000 min lat: 0.000984708 max lat: 0.626797 avg lat: 0.00762011 2024-09-20T22:04:17.274 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:04:17.274 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 20 16 41995 41979 131.153 122.875 0.00728644 0.00762011 2024-09-20T22:04:17.306 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:17.340 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:17.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:17.341 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:17.400 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:04:17 smithi060 podman[81152]: 2024-09-20 22:04:17.147266409 +0000 UTC m=+0.906350062 container remove e4a4322245cf1019664093755e53cd118198ef46bc5073e6961b15a013e659d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1, GIT_BRANCH=HEAD, RELEASE=squid-9e2ac70, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-20T22:04:17.442 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:17.485 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:17.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:17.486 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:17.587 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:17.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:17.634 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:17.634 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:17.734 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:17.769 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:17.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:17.770 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:17.871 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:17.901 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:17.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:17.902 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:17.997 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:17 smithi060 ceph-mon[29694]: 5.11 scrub starts 2024-09-20T22:04:17.997 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:17 smithi060 ceph-mon[29694]: 5.11 scrub ok 2024-09-20T22:04:17.998 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:17 smithi060 ceph-mon[29694]: 6.9 scrub starts 2024-09-20T22:04:17.998 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:17 smithi060 ceph-mon[29694]: 6.9 scrub ok 2024-09-20T22:04:17.998 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:17 smithi060 ceph-mon[40290]: 5.11 scrub starts 2024-09-20T22:04:17.998 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:17 smithi060 ceph-mon[40290]: 5.11 scrub ok 2024-09-20T22:04:17.998 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:17 smithi060 ceph-mon[40290]: 6.9 scrub starts 2024-09-20T22:04:17.998 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:17 smithi060 ceph-mon[40290]: 6.9 scrub ok 2024-09-20T22:04:18.003 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:17 smithi067 ceph-mon[33511]: 5.11 scrub starts 2024-09-20T22:04:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:17 smithi067 ceph-mon[33511]: 5.11 scrub ok 2024-09-20T22:04:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:17 smithi067 ceph-mon[33511]: 6.9 scrub starts 2024-09-20T22:04:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:17 smithi067 ceph-mon[33511]: 6.9 scrub ok 2024-09-20T22:04:18.055 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:18.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:18.056 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:18.157 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:18.183 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:04:18.183 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:04:18.184 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:04:18.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:18.191 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:18.191 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:18.223 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:04:18.223 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:04:18.258 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:04:18.258 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:04:18.277 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 21 16 43843 43827 130.388 115.5 0.00449721 0.00766365 2024-09-20T22:04:18.292 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:18.298 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:04:18.298 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:04:18.333 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:04:18.334 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:04:18.338 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:18.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:18.340 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:18.368 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:04:18.368 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:04:18.401 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:04:18.401 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:04:18.441 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:18.481 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:18.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:18.482 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:18.583 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:18.630 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:18.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:18.631 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:18.716 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:04:18.732 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:18.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[29694]: 4.1 scrub starts 2024-09-20T22:04:18.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[29694]: 4.1 scrub ok 2024-09-20T22:04:18.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[29694]: 4.5 scrub starts 2024-09-20T22:04:18.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[29694]: 4.5 scrub ok 2024-09-20T22:04:18.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[29694]: pgmap v50: 148 pgs: 16 stale+active+clean, 132 active+clean; 2.4 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 820 B/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-20T22:04:18.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[29694]: 6.7 scrub starts 2024-09-20T22:04:18.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[29694]: 6.7 scrub ok 2024-09-20T22:04:18.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[29694]: 2.1 scrub starts 2024-09-20T22:04:18.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[29694]: 2.1 scrub ok 2024-09-20T22:04:18.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[29694]: 6.c scrub starts 2024-09-20T22:04:18.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[29694]: 6.c scrub ok 2024-09-20T22:04:18.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1882837707' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:04:18.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/261859465' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:04:18.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[40290]: 4.1 scrub starts 2024-09-20T22:04:18.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[40290]: 4.1 scrub ok 2024-09-20T22:04:18.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[40290]: 4.5 scrub starts 2024-09-20T22:04:18.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[40290]: 4.5 scrub ok 2024-09-20T22:04:18.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[40290]: pgmap v50: 148 pgs: 16 stale+active+clean, 132 active+clean; 2.4 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 820 B/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-20T22:04:18.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[40290]: 6.7 scrub starts 2024-09-20T22:04:18.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[40290]: 6.7 scrub ok 2024-09-20T22:04:18.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[40290]: 2.1 scrub starts 2024-09-20T22:04:18.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[40290]: 2.1 scrub ok 2024-09-20T22:04:18.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[40290]: 6.c scrub starts 2024-09-20T22:04:18.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[40290]: 6.c scrub ok 2024-09-20T22:04:18.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1882837707' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:04:18.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:18 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/261859465' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:04:18.790 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:04:18 smithi060 podman[81216]: 2024-09-20 22:04:18.768341661 +0000 UTC m=+1.805194058 container died 6cb1c7e7411d967203075d60aca4832c75fffb93e4b356fcd87bc386192776fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736, name=compassionate_zhukovsky, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-9e2ac70, GIT_CLEAN=True) 2024-09-20T22:04:18.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:18.791 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:18.791 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:18.892 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:18.925 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:18.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:18.926 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:18.982 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:18 smithi067 ceph-mon[33511]: 4.1 scrub starts 2024-09-20T22:04:18.982 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:18 smithi067 ceph-mon[33511]: 4.1 scrub ok 2024-09-20T22:04:18.982 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:18 smithi067 ceph-mon[33511]: 4.5 scrub starts 2024-09-20T22:04:18.982 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:18 smithi067 ceph-mon[33511]: 4.5 scrub ok 2024-09-20T22:04:18.982 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:18 smithi067 ceph-mon[33511]: pgmap v50: 148 pgs: 16 stale+active+clean, 132 active+clean; 2.4 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 820 B/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-20T22:04:18.982 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:18 smithi067 ceph-mon[33511]: 6.7 scrub starts 2024-09-20T22:04:18.982 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:18 smithi067 ceph-mon[33511]: 6.7 scrub ok 2024-09-20T22:04:18.982 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:18 smithi067 ceph-mon[33511]: 2.1 scrub starts 2024-09-20T22:04:18.982 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:18 smithi067 ceph-mon[33511]: 2.1 scrub ok 2024-09-20T22:04:18.983 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:18 smithi067 ceph-mon[33511]: 6.c scrub starts 2024-09-20T22:04:18.983 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:18 smithi067 ceph-mon[33511]: 6.c scrub ok 2024-09-20T22:04:18.983 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:18 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1882837707' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:04:18.983 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:18 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/261859465' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:04:19.027 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:19.059 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:19.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:19.060 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:19.161 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:19.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:19.201 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:19.202 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:19.277 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 22 16 45779 45763 129.961 121 0.00595838 0.00768964 2024-09-20T22:04:19.303 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:19.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:19.336 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:19.336 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:19.437 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:19.459 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:19.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:19.460 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:19.561 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:19.591 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:19.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:19.592 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:19.693 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:19.723 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:19.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:19.724 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:19.824 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:19.866 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:19.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:19.867 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:19.968 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:20.001 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:20.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:20.002 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:04:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1882837707' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:04:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: osdmap e93: 8 total, 7 up, 8 in 2024-09-20T22:04:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: 5.e scrub starts 2024-09-20T22:04:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: 5.e scrub ok 2024-09-20T22:04:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: 5.f scrub starts 2024-09-20T22:04:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: 5.f scrub ok 2024-09-20T22:04:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [4, 5]}]: dispatch 2024-09-20T22:04:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 2]}]: dispatch 2024-09-20T22:04:20.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.7", "id": [3, 0]}]: dispatch 2024-09-20T22:04:20.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.9", "id": [7, 1]}]: dispatch 2024-09-20T22:04:20.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.b", "id": [3, 0]}]: dispatch 2024-09-20T22:04:20.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.d", "id": [5, 6]}]: dispatch 2024-09-20T22:04:20.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.11", "id": [3, 1]}]: dispatch 2024-09-20T22:04:20.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.19", "id": [3, 6, 5, 0]}]: dispatch 2024-09-20T22:04:20.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "7.3", "id": [3, 1]}]: dispatch 2024-09-20T22:04:20.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:20.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:19 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:04:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:04:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1882837707' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:04:20.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: osdmap e93: 8 total, 7 up, 8 in 2024-09-20T22:04:20.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: 5.e scrub starts 2024-09-20T22:04:20.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: 5.e scrub ok 2024-09-20T22:04:20.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:20.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:20.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: 5.f scrub starts 2024-09-20T22:04:20.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: 5.f scrub ok 2024-09-20T22:04:20.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [4, 5]}]: dispatch 2024-09-20T22:04:20.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 2]}]: dispatch 2024-09-20T22:04:20.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.7", "id": [3, 0]}]: dispatch 2024-09-20T22:04:20.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.9", "id": [7, 1]}]: dispatch 2024-09-20T22:04:20.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.b", "id": [3, 0]}]: dispatch 2024-09-20T22:04:20.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.d", "id": [5, 6]}]: dispatch 2024-09-20T22:04:20.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.11", "id": [3, 1]}]: dispatch 2024-09-20T22:04:20.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.19", "id": [3, 6, 5, 0]}]: dispatch 2024-09-20T22:04:20.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "7.3", "id": [3, 1]}]: dispatch 2024-09-20T22:04:20.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:20.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:04:20.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:04:20.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1882837707' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:04:20.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: osdmap e93: 8 total, 7 up, 8 in 2024-09-20T22:04:20.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: 5.e scrub starts 2024-09-20T22:04:20.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: 5.e scrub ok 2024-09-20T22:04:20.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:20.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:20.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: 5.f scrub starts 2024-09-20T22:04:20.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: 5.f scrub ok 2024-09-20T22:04:20.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [4, 5]}]: dispatch 2024-09-20T22:04:20.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 2]}]: dispatch 2024-09-20T22:04:20.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.7", "id": [3, 0]}]: dispatch 2024-09-20T22:04:20.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.9", "id": [7, 1]}]: dispatch 2024-09-20T22:04:20.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.b", "id": [3, 0]}]: dispatch 2024-09-20T22:04:20.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.d", "id": [5, 6]}]: dispatch 2024-09-20T22:04:20.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.11", "id": [3, 1]}]: dispatch 2024-09-20T22:04:20.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.19", "id": [3, 6, 5, 0]}]: dispatch 2024-09-20T22:04:20.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "7.3", "id": [3, 1]}]: dispatch 2024-09-20T22:04:20.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:20.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:19 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:04:20.103 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:20.138 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:20.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:20.139 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:20.240 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:20.274 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:20.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:20.275 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:20.277 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 23 16 47654 47638 129.406 117.188 0.012498 0.00772172 2024-09-20T22:04:20.377 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:20.399 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:20.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:20.400 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:20.500 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:20.523 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:20.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:20.524 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:20.625 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:20.647 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:20.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:20.648 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:20.749 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:20.771 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:20.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:20.772 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:20.819 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:20.872 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:20.894 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:20.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:20.895 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:20.996 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:20 smithi067 ceph-mon[33511]: 5.2 scrub starts 2024-09-20T22:04:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:20 smithi067 ceph-mon[33511]: 5.2 scrub ok 2024-09-20T22:04:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:20 smithi067 ceph-mon[33511]: pgmap v52: 148 pgs: 16 stale+active+clean, 132 active+clean; 2.4 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-20T22:04:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:20 smithi067 ceph-mon[33511]: 5.3 scrub starts 2024-09-20T22:04:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:20 smithi067 ceph-mon[33511]: 5.3 scrub ok 2024-09-20T22:04:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [4, 5]}]': finished 2024-09-20T22:04:21.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 2]}]': finished 2024-09-20T22:04:21.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.7", "id": [3, 0]}]': finished 2024-09-20T22:04:21.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.9", "id": [7, 1]}]': finished 2024-09-20T22:04:21.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.b", "id": [3, 0]}]': finished 2024-09-20T22:04:21.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.d", "id": [5, 6]}]': finished 2024-09-20T22:04:21.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.11", "id": [3, 1]}]': finished 2024-09-20T22:04:21.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.19", "id": [3, 6, 5, 0]}]': finished 2024-09-20T22:04:21.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "7.3", "id": [3, 1]}]': finished 2024-09-20T22:04:21.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:20 smithi067 ceph-mon[33511]: osdmap e94: 8 total, 7 up, 8 in 2024-09-20T22:04:21.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:21.019 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:21.019 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:21.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[29694]: 5.2 scrub starts 2024-09-20T22:04:21.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[29694]: 5.2 scrub ok 2024-09-20T22:04:21.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[29694]: pgmap v52: 148 pgs: 16 stale+active+clean, 132 active+clean; 2.4 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-20T22:04:21.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[29694]: 5.3 scrub starts 2024-09-20T22:04:21.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[29694]: 5.3 scrub ok 2024-09-20T22:04:21.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [4, 5]}]': finished 2024-09-20T22:04:21.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 2]}]': finished 2024-09-20T22:04:21.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.7", "id": [3, 0]}]': finished 2024-09-20T22:04:21.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.9", "id": [7, 1]}]': finished 2024-09-20T22:04:21.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.b", "id": [3, 0]}]': finished 2024-09-20T22:04:21.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.d", "id": [5, 6]}]': finished 2024-09-20T22:04:21.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.11", "id": [3, 1]}]': finished 2024-09-20T22:04:21.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.19", "id": [3, 6, 5, 0]}]': finished 2024-09-20T22:04:21.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "7.3", "id": [3, 1]}]': finished 2024-09-20T22:04:21.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[29694]: osdmap e94: 8 total, 7 up, 8 in 2024-09-20T22:04:21.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[40290]: 5.2 scrub starts 2024-09-20T22:04:21.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[40290]: 5.2 scrub ok 2024-09-20T22:04:21.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[40290]: pgmap v52: 148 pgs: 16 stale+active+clean, 132 active+clean; 2.4 GiB data, 7.4 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-20T22:04:21.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[40290]: 5.3 scrub starts 2024-09-20T22:04:21.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[40290]: 5.3 scrub ok 2024-09-20T22:04:21.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1a", "id": [4, 5]}]': finished 2024-09-20T22:04:21.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1c", "id": [4, 2]}]': finished 2024-09-20T22:04:21.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.7", "id": [3, 0]}]': finished 2024-09-20T22:04:21.040 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.9", "id": [7, 1]}]': finished 2024-09-20T22:04:21.040 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.b", "id": [3, 0]}]': finished 2024-09-20T22:04:21.040 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.d", "id": [5, 6]}]': finished 2024-09-20T22:04:21.040 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.11", "id": [3, 1]}]': finished 2024-09-20T22:04:21.040 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "6.19", "id": [3, 6, 5, 0]}]': finished 2024-09-20T22:04:21.040 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "7.3", "id": [3, 1]}]': finished 2024-09-20T22:04:21.040 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:20 smithi060 ceph-mon[40290]: osdmap e94: 8 total, 7 up, 8 in 2024-09-20T22:04:21.120 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:21.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:21.143 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:21.143 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:21.244 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:21.266 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:21.267 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:04:21.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:21.268 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:04:21.268 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:21.277 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 24 16 47853 47837 124.533 12.4375 0.00171898 0.00771977 2024-09-20T22:04:21.370 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:21.402 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:21.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:21.403 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:21.504 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:21.536 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:21.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:21.537 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:21.638 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:21.670 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:21.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:21.671 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:21.772 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:21.818 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[29694]: 3.15 scrub starts 2024-09-20T22:04:21.819 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[29694]: 3.15 scrub ok 2024-09-20T22:04:21.819 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[29694]: 4.11 scrub starts 2024-09-20T22:04:21.819 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[29694]: 4.11 scrub ok 2024-09-20T22:04:21.819 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[29694]: 5.1 scrub starts 2024-09-20T22:04:21.819 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[29694]: 5.1 scrub ok 2024-09-20T22:04:21.819 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[29694]: 3.1f scrub starts 2024-09-20T22:04:21.819 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[29694]: 3.1f scrub ok 2024-09-20T22:04:21.819 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[29694]: osdmap e95: 8 total, 7 up, 8 in 2024-09-20T22:04:21.819 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[29694]: osdmap e96: 8 total, 7 up, 8 in 2024-09-20T22:04:21.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[40290]: 3.15 scrub starts 2024-09-20T22:04:21.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[40290]: 3.15 scrub ok 2024-09-20T22:04:21.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[40290]: 4.11 scrub starts 2024-09-20T22:04:21.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[40290]: 4.11 scrub ok 2024-09-20T22:04:21.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[40290]: 5.1 scrub starts 2024-09-20T22:04:21.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[40290]: 5.1 scrub ok 2024-09-20T22:04:21.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[40290]: 3.1f scrub starts 2024-09-20T22:04:21.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[40290]: 3.1f scrub ok 2024-09-20T22:04:21.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[40290]: osdmap e95: 8 total, 7 up, 8 in 2024-09-20T22:04:21.820 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:21 smithi060 ceph-mon[40290]: osdmap e96: 8 total, 7 up, 8 in 2024-09-20T22:04:21.831 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:21.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:21.832 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:21.879 INFO:teuthology.orchestra.run.smithi060.stderr:admin_socket: exception getting command descriptions: [Errno 111] Connection refused 2024-09-20T22:04:21.932 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:21.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:21.964 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:21.964 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:22.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:04:21 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:04:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:21 smithi067 ceph-mon[33511]: 3.15 scrub starts 2024-09-20T22:04:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:21 smithi067 ceph-mon[33511]: 3.15 scrub ok 2024-09-20T22:04:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:21 smithi067 ceph-mon[33511]: 4.11 scrub starts 2024-09-20T22:04:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:21 smithi067 ceph-mon[33511]: 4.11 scrub ok 2024-09-20T22:04:22.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:21 smithi067 ceph-mon[33511]: 5.1 scrub starts 2024-09-20T22:04:22.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:21 smithi067 ceph-mon[33511]: 5.1 scrub ok 2024-09-20T22:04:22.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:21 smithi067 ceph-mon[33511]: 3.1f scrub starts 2024-09-20T22:04:22.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:21 smithi067 ceph-mon[33511]: 3.1f scrub ok 2024-09-20T22:04:22.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:21 smithi067 ceph-mon[33511]: osdmap e95: 8 total, 7 up, 8 in 2024-09-20T22:04:22.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:21 smithi067 ceph-mon[33511]: osdmap e96: 8 total, 7 up, 8 in 2024-09-20T22:04:22.065 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:22.107 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:22.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:22.108 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:22.208 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:22.212 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:04:21 smithi060 podman[81732]: 2024-09-20 22:04:21.876577902 +0000 UTC m=+0.261122271 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5 2024-09-20T22:04:22.250 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:22.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:22.251 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:22.277 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 25 16 49626 49610 123.984 110.812 0.00379431 0.0080601 2024-09-20T22:04:22.352 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:22.391 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:22.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:22.392 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:22.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:22.524 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:22.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:22.525 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:22.536 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:04:22 smithi060 podman[81732]: 2024-09-20 22:04:22.211261322 +0000 UTC m=+0.595805691 container create 001999d78a9dbdff65f4de47a0fb84bc9497bbd7e4f3d47ee03a1673b3be0e4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-9e2ac70) 2024-09-20T22:04:22.626 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:22.655 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:22.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:22.656 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:22.757 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:22.785 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:22.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:22.786 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:22.791 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:04:22.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[29694]: 6.d scrub starts 2024-09-20T22:04:22.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[29694]: 6.d scrub ok 2024-09-20T22:04:22.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[29694]: pgmap v56: 148 pgs: 3 peering, 6 unknown, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 2.9 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 126 MiB/s wr, 2.01k op/s; 18046/144108 objects degraded (12.523%) 2024-09-20T22:04:22.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:04:22.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[29694]: 5.b scrub starts 2024-09-20T22:04:22.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[29694]: 5.b scrub ok 2024-09-20T22:04:22.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[29694]: 3.3 scrub starts 2024-09-20T22:04:22.847 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[29694]: 3.3 scrub ok 2024-09-20T22:04:22.847 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[29694]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-20T22:04:22.847 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[29694]: Health check failed: Degraded data redundancy: 18046/144108 objects degraded (12.523%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:22.847 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:04:22 smithi060 podman[81732]: 2024-09-20 22:04:22.548389579 +0000 UTC m=+0.932933945 container init 001999d78a9dbdff65f4de47a0fb84bc9497bbd7e4f3d47ee03a1673b3be0e4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-9e2ac70, maintainer=Guillaume Abrioux ) 2024-09-20T22:04:22.847 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:04:22 smithi060 podman[81732]: 2024-09-20 22:04:22.556860229 +0000 UTC m=+0.941404589 container start 001999d78a9dbdff65f4de47a0fb84bc9497bbd7e4f3d47ee03a1673b3be0e4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-9e2ac70, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-20T22:04:22.847 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:04:22 smithi060 podman[81732]: 2024-09-20 22:04:22.647991854 +0000 UTC m=+1.032536219 container attach 001999d78a9dbdff65f4de47a0fb84bc9497bbd7e4f3d47ee03a1673b3be0e4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-9e2ac70, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-20T22:04:22.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[40290]: 6.d scrub starts 2024-09-20T22:04:22.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[40290]: 6.d scrub ok 2024-09-20T22:04:22.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[40290]: pgmap v56: 148 pgs: 3 peering, 6 unknown, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 2.9 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 126 MiB/s wr, 2.01k op/s; 18046/144108 objects degraded (12.523%) 2024-09-20T22:04:22.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:04:22.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[40290]: 5.b scrub starts 2024-09-20T22:04:22.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[40290]: 5.b scrub ok 2024-09-20T22:04:22.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[40290]: 3.3 scrub starts 2024-09-20T22:04:22.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[40290]: 3.3 scrub ok 2024-09-20T22:04:22.849 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[40290]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-20T22:04:22.849 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:22 smithi060 ceph-mon[40290]: Health check failed: Degraded data redundancy: 18046/144108 objects degraded (12.523%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:22.887 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:22.915 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:22.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:22.917 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:23.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:22 smithi067 ceph-mon[33511]: 6.d scrub starts 2024-09-20T22:04:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:22 smithi067 ceph-mon[33511]: 6.d scrub ok 2024-09-20T22:04:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:22 smithi067 ceph-mon[33511]: pgmap v56: 148 pgs: 3 peering, 6 unknown, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 2.9 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 126 MiB/s wr, 2.01k op/s; 18046/144108 objects degraded (12.523%) 2024-09-20T22:04:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:22 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:04:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:22 smithi067 ceph-mon[33511]: 5.b scrub starts 2024-09-20T22:04:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:22 smithi067 ceph-mon[33511]: 5.b scrub ok 2024-09-20T22:04:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:22 smithi067 ceph-mon[33511]: 3.3 scrub starts 2024-09-20T22:04:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:22 smithi067 ceph-mon[33511]: 3.3 scrub ok 2024-09-20T22:04:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:22 smithi067 ceph-mon[33511]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-20T22:04:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:22 smithi067 ceph-mon[33511]: Health check failed: Degraded data redundancy: 18046/144108 objects degraded (12.523%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:23.018 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:23.074 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:23.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:23.075 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:23.176 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:23.206 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:23.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:23.207 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:23.244 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T22:04:23.244 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T22:04:23.269 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:04:23.281 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 26 16 51277 51261 123.167 103.188 0.00256975 0.00811277 2024-09-20T22:04:23.308 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:23.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:23.413 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:23.413 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:23.503 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:04:23 smithi060 podman[81732]: 2024-09-20 22:04:23.208281789 +0000 UTC m=+1.592826146 container died 001999d78a9dbdff65f4de47a0fb84bc9497bbd7e4f3d47ee03a1673b3be0e4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , RELEASE=squid-9e2ac70) 2024-09-20T22:04:23.515 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:23.540 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:23.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:23.541 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:23.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:23.668 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:23.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:23.669 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:23.770 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:23 smithi060 ceph-mon[29694]: 3.1c scrub starts 2024-09-20T22:04:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:23 smithi060 ceph-mon[29694]: 3.1c scrub ok 2024-09-20T22:04:23.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:23 smithi060 ceph-mon[40290]: 3.1c scrub starts 2024-09-20T22:04:23.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:23 smithi060 ceph-mon[40290]: 3.1c scrub ok 2024-09-20T22:04:23.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:23.812 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:23.813 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:23.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:23.958 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:23.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:23.958 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:23.974 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.1.service' 2024-09-20T22:04:24.005 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:23 smithi067 ceph-mon[33511]: 3.1c scrub starts 2024-09-20T22:04:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:23 smithi067 ceph-mon[33511]: 3.1c scrub ok 2024-09-20T22:04:24.059 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:24.124 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T22:04:24.125 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-20T22:04:24.125 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.1 2024-09-20T22:04:24.125 INFO:tasks.thrashosds.thrasher:remote for osd 1 is ubuntu@smithi060.front.sepia.ceph.com 2024-09-20T22:04:24.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -n osd.1 --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op list-pgs 2024-09-20T22:04:24.127 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:24.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:24.129 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:24.230 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:24.282 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 27 16 52595 52579 121.657 82.375 0.00962868 0.00821244 2024-09-20T22:04:24.286 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:24.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:24.287 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:24.351 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:24.388 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:24.459 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:24.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:24.460 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:24.561 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:24.591 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:24.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:24.592 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:24.693 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:24.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:24.720 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:24.821 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:24.850 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:24.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:24.851 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:24.951 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:24.978 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:24.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:24.979 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:25.007 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:24 smithi060 ceph-mon[29694]: pgmap v57: 148 pgs: 3 peering, 6 unknown, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 2.9 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 104 MiB/s wr, 1.66k op/s; 18046/144108 objects degraded (12.523%) 2024-09-20T22:04:25.007 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:24 smithi060 ceph-mon[29694]: 5.a scrub starts 2024-09-20T22:04:25.007 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:24 smithi060 ceph-mon[29694]: 5.a scrub ok 2024-09-20T22:04:25.007 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:24 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:04:25.007 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:24 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:04:25.007 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:24 smithi060 ceph-mon[29694]: 3.1d scrub starts 2024-09-20T22:04:25.007 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:24 smithi060 ceph-mon[29694]: 3.1d scrub ok 2024-09-20T22:04:25.007 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:24 smithi060 ceph-mon[40290]: pgmap v57: 148 pgs: 3 peering, 6 unknown, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 2.9 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 104 MiB/s wr, 1.66k op/s; 18046/144108 objects degraded (12.523%) 2024-09-20T22:04:25.008 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:24 smithi060 ceph-mon[40290]: 5.a scrub starts 2024-09-20T22:04:25.008 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:24 smithi060 ceph-mon[40290]: 5.a scrub ok 2024-09-20T22:04:25.008 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:24 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:04:25.008 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:24 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:04:25.008 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:24 smithi060 ceph-mon[40290]: 3.1d scrub starts 2024-09-20T22:04:25.008 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:24 smithi060 ceph-mon[40290]: 3.1d scrub ok 2024-09-20T22:04:25.008 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:24 smithi067 ceph-mon[33511]: pgmap v57: 148 pgs: 3 peering, 6 unknown, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 2.9 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 104 MiB/s wr, 1.66k op/s; 18046/144108 objects degraded (12.523%) 2024-09-20T22:04:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:24 smithi067 ceph-mon[33511]: 5.a scrub starts 2024-09-20T22:04:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:24 smithi067 ceph-mon[33511]: 5.a scrub ok 2024-09-20T22:04:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:24 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:04:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:24 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:04:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:24 smithi067 ceph-mon[33511]: 3.1d scrub starts 2024-09-20T22:04:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:24 smithi067 ceph-mon[33511]: 3.1d scrub ok 2024-09-20T22:04:25.080 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:25.134 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:04:25.134 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:04:25.134 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:25.134 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:53678 7.1 e97/80 7:8979becd:::benchmark_data_smithi060_78800_object53677:head v 97'3412, mlcod=97'3410)", 2024-09-20T22:04:25.135 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:25.077336+0000", 2024-09-20T22:04:25.135 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.001746201, 2024-09-20T22:04:25.135 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017794270000000001, 2024-09-20T22:04:25.135 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:04:25.135 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:04:25.135 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:04:25.135 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:04:25.135 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:25.135 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:04:25.135 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:25.077336+0000", 2024-09-20T22:04:25.135 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:04:25.135 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:25.135 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:25.136 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:04:25.136 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:25.077336+0000", 2024-09-20T22:04:25.136 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:04:25.136 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:25.136 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:25.136 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:04:25.136 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:25.077337+0000", 2024-09-20T22:04:25.136 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.06e-07 2024-09-20T22:04:25.136 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:25.136 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:25.136 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:04:25.136 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:25.077406+0000", 2024-09-20T22:04:25.136 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9167999999999999e-05 2024-09-20T22:04:25.137 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:25.137 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:25.137 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:04:25.137 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:25.077416+0000", 2024-09-20T22:04:25.137 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0271e-05 2024-09-20T22:04:25.137 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:25.137 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:25.137 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:04:25.137 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:25.077423+0000", 2024-09-20T22:04:25.137 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0360000000000001e-06 2024-09-20T22:04:25.137 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:25.137 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:25.137 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:04:25.137 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:25.078220+0000", 2024-09-20T22:04:25.137 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00079666600000000002 2024-09-20T22:04:25.138 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:25.138 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:25.138 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:04:25.138 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:25.078238+0000", 2024-09-20T22:04:25.138 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8182e-05 2024-09-20T22:04:25.138 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:04:25.138 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:04:25.138 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:04:25.138 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:25.138 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:25.138 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:53684 7.0 e97/91 7:0d441a84:::benchmark_data_smithi060_78800_object53683:head v 97'3344, mlcod=97'3342)", 2024-09-20T22:04:25.138 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:25.077628+0000", 2024-09-20T22:04:25.138 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0014537199999999999, 2024-09-20T22:04:25.138 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0015756749999999999, 2024-09-20T22:04:25.139 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:04:25.139 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:04:25.139 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:04:25.139 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:04:25.139 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:25.139 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:04:25.139 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:25.077628+0000", 2024-09-20T22:04:25.139 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:04:25.139 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:25.139 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:25.139 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:04:25.139 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:25.077628+0000", 2024-09-20T22:04:25.139 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:04:25.140 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:25.140 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:25.140 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:04:25.140 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:25.077629+0000", 2024-09-20T22:04:25.140 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4700000000000001e-07 2024-09-20T22:04:25.140 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:25.140 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:25.140 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:04:25.140 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:25.077684+0000", 2024-09-20T22:04:25.140 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5043000000000001e-05 2024-09-20T22:04:25.140 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:25.140 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:25.140 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:04:25.140 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:25.077697+0000", 2024-09-20T22:04:25.141 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2921e-05 2024-09-20T22:04:25.141 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:25.141 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:25.141 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:04:25.141 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:25.077704+0000", 2024-09-20T22:04:25.141 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7270000000000001e-06 2024-09-20T22:04:25.141 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:25.141 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:25.141 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:04:25.141 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:25.077779+0000", 2024-09-20T22:04:25.141 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5294000000000001e-05 2024-09-20T22:04:25.141 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:04:25.141 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:04:25.141 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:04:25.142 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:25.077803+0000", 2024-09-20T22:04:25.142 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3538999999999999e-05 2024-09-20T22:04:25.142 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:04:25.142 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:04:25.142 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:04:25.142 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:04:25.142 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:04:25.142 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 2 2024-09-20T22:04:25.142 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:04:25.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:25.143 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:25.143 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:25.244 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:25.274 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:25.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:25.275 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:25.281 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 28 16 53990 53974 120.426 87.1875 0.00458442 0.00829814 2024-09-20T22:04:25.285 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:04:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:04:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:04:25.376 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:25.403 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:25.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:25.404 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:25.505 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:25.529 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:25.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:25.530 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:25.631 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:25.674 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:25.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:25.675 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:25.776 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:25.797 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:25.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:25.798 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:25.816 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:04:25.899 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:25.941 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:25.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:25.942 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:26.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:25 smithi067 ceph-mon[33511]: osdmap e97: 8 total, 7 up, 8 in 2024-09-20T22:04:26.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:25 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3997939936' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:04:26.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:25 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:04:26.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:25 smithi060 ceph-mon[40290]: osdmap e97: 8 total, 7 up, 8 in 2024-09-20T22:04:26.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:25 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3997939936' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:04:26.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:25 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:04:26.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:25 smithi060 ceph-mon[29694]: osdmap e97: 8 total, 7 up, 8 in 2024-09-20T22:04:26.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:25 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3997939936' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:04:26.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:25 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:04:26.043 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:26.073 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:26.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:26.074 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:26.174 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:26.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:26.209 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:26.209 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:26.281 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 29 16 55716 55700 119.993 107.875 0.00557799 0.00832846 2024-09-20T22:04:26.310 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:26.350 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:26.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:26.352 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:26.452 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:26.484 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:26.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:26.485 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:26.515 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T22:04:26.586 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:26.616 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:26.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:26.617 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:26.717 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:26.745 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:26.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:26.746 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:26.847 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:26.878 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:26.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:26.879 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:26.980 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:27.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:27.026 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:27.026 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:26 smithi060 ceph-mon[29694]: pgmap v59: 148 pgs: 3 active+clean+remapped, 1 peering, 4 unknown, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 3.1 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 192 B/s rd, 129 MiB/s wr, 2.06k op/s; 19290/152907 objects degraded (12.616%) 2024-09-20T22:04:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:26 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:04:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:26 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:04:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:26 smithi060 ceph-mon[29694]: osdmap e98: 8 total, 7 up, 8 in 2024-09-20T22:04:27.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:26 smithi060 ceph-mon[40290]: pgmap v59: 148 pgs: 3 active+clean+remapped, 1 peering, 4 unknown, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 3.1 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 192 B/s rd, 129 MiB/s wr, 2.06k op/s; 19290/152907 objects degraded (12.616%) 2024-09-20T22:04:27.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:26 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:04:27.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:26 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:04:27.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:26 smithi060 ceph-mon[40290]: osdmap e98: 8 total, 7 up, 8 in 2024-09-20T22:04:27.127 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:27.168 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:27.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:27.169 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:27.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:26 smithi067 ceph-mon[33511]: pgmap v59: 148 pgs: 3 active+clean+remapped, 1 peering, 4 unknown, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 3.1 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 192 B/s rd, 129 MiB/s wr, 2.06k op/s; 19290/152907 objects degraded (12.616%) 2024-09-20T22:04:27.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:26 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:04:27.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:26 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:04:27.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:26 smithi067 ceph-mon[33511]: osdmap e98: 8 total, 7 up, 8 in 2024-09-20T22:04:27.270 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:27.282 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 30 16 57483 57467 119.674 110.438 0.0066964 0.00835075 2024-09-20T22:04:27.302 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:27.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:27.303 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:27.404 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:27.433 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:27.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:27.434 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:27.535 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:27.569 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:27.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:27.570 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:27.671 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:27.716 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:27.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:27.717 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:27.818 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:27.847 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:27.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:27.848 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:27.872 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/osd.1/config 2024-09-20T22:04:27.949 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:27.971 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:04:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:04:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:04:27.972 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:04:27.972 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:04:27.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:27.997 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:27.997 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:28.007 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:27 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 20510/164007 objects degraded (12.506%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:28.007 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:27 smithi060 ceph-mon[29694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-20T22:04:28.007 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:27 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 20510/164007 objects degraded (12.506%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:28.008 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:27 smithi060 ceph-mon[40290]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-20T22:04:28.008 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:04:28.008 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:04:28.042 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:04:28.042 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:04:28.079 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:04:28.079 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:04:28.097 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:28.114 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:04:28.114 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:04:28.129 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:28.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:28.131 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:28.148 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:04:28.148 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:04:28.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:28.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:27 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 20510/164007 objects degraded (12.506%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:28.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:27 smithi067 ceph-mon[33511]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-20T22:04:28.270 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:28.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:28.271 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:28.281 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 31 16 59168 59152 119.21 105.312 0.00288413 0.0083836 2024-09-20T22:04:28.372 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:28.419 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:28.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:28.420 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:28.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:28.571 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:28.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:28.572 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:28.673 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:28.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:28.712 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:28.712 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:28.813 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:28.850 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:28.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:28.851 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:28.952 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:28.991 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:28.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:28.992 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:29.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:28 smithi060 ceph-mon[29694]: pgmap v61: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 3.3 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 819 B/s rd, 66 MiB/s wr, 1.06k op/s; 20510/164007 objects degraded (12.506%); 3391/164007 objects misplaced (2.068%) 2024-09-20T22:04:29.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:28 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3224710983' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:04:29.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:28 smithi060 ceph-mon[40290]: pgmap v61: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 3.3 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 819 B/s rd, 66 MiB/s wr, 1.06k op/s; 20510/164007 objects degraded (12.506%); 3391/164007 objects misplaced (2.068%) 2024-09-20T22:04:29.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:28 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3224710983' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:04:29.094 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:29.155 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:04:29.155 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:04:29.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:29.174 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:29.174 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:29.186 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:29.237 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:04:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:28 smithi067 ceph-mon[33511]: pgmap v61: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 3.3 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 819 B/s rd, 66 MiB/s wr, 1.06k op/s; 20510/164007 objects degraded (12.506%); 3391/164007 objects misplaced (2.068%) 2024-09-20T22:04:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:28 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3224710983' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:04:29.275 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:29.282 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 32 16 60873 60857 118.812 106.562 0.0110946 0.00841134 2024-09-20T22:04:29.303 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:29.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:29.304 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:29.405 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:29.466 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:29.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:29.467 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:29.568 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:29.619 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:29.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:29.620 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:29.721 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:29.763 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:29.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:29.764 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:29.865 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:29.899 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:29.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:29.901 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:30.001 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:30.041 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:30.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:30.042 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:30.143 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:30.183 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:30.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:30.184 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:30.282 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 33 15 62482 62467 118.261 100.625 0.00857176 0.00845022 2024-09-20T22:04:30.285 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:30.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:30.333 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:30.334 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:30.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:30.482 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:30.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:30.484 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:30.585 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:30.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:30.628 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:30.628 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:30.729 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:30.764 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:30.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:30.765 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:30.865 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:30.905 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:30.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:30.906 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:31.007 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:30 smithi060 ceph-mon[29694]: pgmap v62: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 3.3 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 52 MiB/s wr, 829 op/s; 20510/164007 objects degraded (12.506%); 3391/164007 objects misplaced (2.068%) 2024-09-20T22:04:31.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:30 smithi060 ceph-mon[40290]: pgmap v62: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 3.3 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 52 MiB/s wr, 829 op/s; 20510/164007 objects degraded (12.506%); 3391/164007 objects misplaced (2.068%) 2024-09-20T22:04:31.037 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:31.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:31.038 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:31.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:31.156 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:04:31.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:31.173 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:31.173 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:31.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:30 smithi067 ceph-mon[33511]: pgmap v62: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 3.3 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 52 MiB/s wr, 829 op/s; 20510/164007 objects degraded (12.506%); 3391/164007 objects misplaced (2.068%) 2024-09-20T22:04:31.274 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:31.282 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 34 16 64088 64072 117.733 100.312 0.0115622 0.00848888 2024-09-20T22:04:31.302 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:31.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:31.303 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:31.404 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:31.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:31.454 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:31.454 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:31.555 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:31.583 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:31.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:31.584 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:31.638 INFO:teuthology.orchestra.run.smithi060.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-20T22:04:31.685 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:31.736 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:31.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:31.737 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:31.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:31.879 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:31.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:31.880 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:31.962 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:04:31.962 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:04:31.981 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:32.012 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:32.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:32.013 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:32.033 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:04:31 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:04:32.114 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:32.158 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:32.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:32.159 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:32.259 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:32.283 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 35 16 65642 65626 117.144 97.125 0.00802123 0.00853163 2024-09-20T22:04:32.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:32 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:32.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:32 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:32.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:32 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:04:32.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:32 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:04:32.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:32 smithi060 ceph-mon[29694]: pgmap v63: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 3.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.3 KiB/s rd, 126 MiB/s wr, 2.01k op/s; 23648/189357 objects degraded (12.489%); 3912/189357 objects misplaced (2.066%) 2024-09-20T22:04:32.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:32 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:32.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:32 smithi060 ceph-mon[29694]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-20T22:04:32.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:32 smithi060 ceph-mon[29694]: Reconfiguring daemon alertmanager.a on smithi060 2024-09-20T22:04:32.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:32 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:32.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:32 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:32.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:32 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:04:32.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:32 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:04:32.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:32 smithi060 ceph-mon[40290]: pgmap v63: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 3.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.3 KiB/s rd, 126 MiB/s wr, 2.01k op/s; 23648/189357 objects degraded (12.489%); 3912/189357 objects misplaced (2.066%) 2024-09-20T22:04:32.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:32 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:32.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:32 smithi060 ceph-mon[40290]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-20T22:04:32.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:32 smithi060 ceph-mon[40290]: Reconfiguring daemon alertmanager.a on smithi060 2024-09-20T22:04:32.294 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:32.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:32.295 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:32.396 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:32.435 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:32.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:32.436 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:32.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:32 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:32.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:32 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:32.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:32 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:04:32.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:32 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:04:32.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:32 smithi067 ceph-mon[33511]: pgmap v63: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 3.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.3 KiB/s rd, 126 MiB/s wr, 2.01k op/s; 23648/189357 objects degraded (12.489%); 3912/189357 objects misplaced (2.066%) 2024-09-20T22:04:32.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:32 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:32.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:32 smithi067 ceph-mon[33511]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-20T22:04:32.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:32 smithi067 ceph-mon[33511]: Reconfiguring daemon alertmanager.a on smithi060 2024-09-20T22:04:32.536 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:32.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:32.567 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:32.567 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:32.668 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:32.708 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:32.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:32.709 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:32.810 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:32.840 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:32.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:32.841 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:32.864 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T22:04:32.942 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:32.972 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:32.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:32.973 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:33.074 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:33.087 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:33 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:04:33.087 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:33 smithi060 ceph-mon[40290]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-20T22:04:33.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:33 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:04:33.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:33 smithi060 ceph-mon[29694]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-20T22:04:33.102 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:33.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:33.103 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:33.204 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:33.234 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:33.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:33.239 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:33.283 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 36 16 67204 67188 116.601 97.625 0.0094035 0.00857156 2024-09-20T22:04:33.339 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:33.368 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:33.370 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:33.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:33.371 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:33.472 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:33.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:33 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:04:33.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:33 smithi067 ceph-mon[33511]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2024-09-20T22:04:33.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:33.527 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:33.527 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:33.628 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:33.656 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:33.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:33.657 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:33.758 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:33.795 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:33.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:33.796 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:33.855 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:33.897 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:33.926 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:33.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:33.927 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:34.027 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:34.054 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:34.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:34.055 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:34.057 INFO:teuthology.orchestra.run.smithi060.stdout:7.d 2024-09-20T22:04:34.057 INFO:teuthology.orchestra.run.smithi060.stdout:7.b 2024-09-20T22:04:34.058 INFO:teuthology.orchestra.run.smithi060.stdout:7.4 2024-09-20T22:04:34.058 INFO:teuthology.orchestra.run.smithi060.stdout:4.7 2024-09-20T22:04:34.058 INFO:teuthology.orchestra.run.smithi060.stdout:4.1f 2024-09-20T22:04:34.058 INFO:teuthology.orchestra.run.smithi060.stdout:4.18 2024-09-20T22:04:34.058 INFO:teuthology.orchestra.run.smithi060.stdout:6.19 2024-09-20T22:04:34.058 INFO:teuthology.orchestra.run.smithi060.stdout:4.6 2024-09-20T22:04:34.058 INFO:teuthology.orchestra.run.smithi060.stdout:4.17 2024-09-20T22:04:34.058 INFO:teuthology.orchestra.run.smithi060.stdout:5.0 2024-09-20T22:04:34.058 INFO:teuthology.orchestra.run.smithi060.stdout:4.a 2024-09-20T22:04:34.058 INFO:teuthology.orchestra.run.smithi060.stdout:4.13 2024-09-20T22:04:34.058 INFO:teuthology.orchestra.run.smithi060.stdout:4.12 2024-09-20T22:04:34.059 INFO:teuthology.orchestra.run.smithi060.stdout:7.9 2024-09-20T22:04:34.059 INFO:teuthology.orchestra.run.smithi060.stdout:3.6 2024-09-20T22:04:34.059 INFO:teuthology.orchestra.run.smithi060.stdout:4.e 2024-09-20T22:04:34.059 INFO:teuthology.orchestra.run.smithi060.stdout:4.9 2024-09-20T22:04:34.059 INFO:teuthology.orchestra.run.smithi060.stdout:4.1c 2024-09-20T22:04:34.059 INFO:teuthology.orchestra.run.smithi060.stdout:3.19 2024-09-20T22:04:34.059 INFO:teuthology.orchestra.run.smithi060.stdout:6.10 2024-09-20T22:04:34.059 INFO:teuthology.orchestra.run.smithi060.stdout:4.2 2024-09-20T22:04:34.059 INFO:teuthology.orchestra.run.smithi060.stdout:3.18 2024-09-20T22:04:34.059 INFO:teuthology.orchestra.run.smithi060.stdout:4.4 2024-09-20T22:04:34.059 INFO:teuthology.orchestra.run.smithi060.stdout:3.16 2024-09-20T22:04:34.059 INFO:teuthology.orchestra.run.smithi060.stdout:3.8 2024-09-20T22:04:34.059 INFO:teuthology.orchestra.run.smithi060.stdout:3.1a 2024-09-20T22:04:34.059 INFO:teuthology.orchestra.run.smithi060.stdout:3.0 2024-09-20T22:04:34.059 INFO:teuthology.orchestra.run.smithi060.stdout:5.19 2024-09-20T22:04:34.059 INFO:teuthology.orchestra.run.smithi060.stdout:5.5 2024-09-20T22:04:34.060 INFO:teuthology.orchestra.run.smithi060.stdout:3.1c 2024-09-20T22:04:34.060 INFO:teuthology.orchestra.run.smithi060.stdout:3.4 2024-09-20T22:04:34.060 INFO:teuthology.orchestra.run.smithi060.stdout:5.16 2024-09-20T22:04:34.060 INFO:teuthology.orchestra.run.smithi060.stdout:4.14 2024-09-20T22:04:34.060 INFO:teuthology.orchestra.run.smithi060.stdout:6.1 2024-09-20T22:04:34.060 INFO:teuthology.orchestra.run.smithi060.stdout:2.0 2024-09-20T22:04:34.060 INFO:teuthology.orchestra.run.smithi060.stdout:5.13 2024-09-20T22:04:34.060 INFO:teuthology.orchestra.run.smithi060.stdout:4.1b 2024-09-20T22:04:34.060 INFO:teuthology.orchestra.run.smithi060.stdout:3.a 2024-09-20T22:04:34.060 INFO:teuthology.orchestra.run.smithi060.stdout:5.8 2024-09-20T22:04:34.060 INFO:teuthology.orchestra.run.smithi060.stdout:4.b 2024-09-20T22:04:34.060 INFO:teuthology.orchestra.run.smithi060.stdout:4.1d 2024-09-20T22:04:34.060 INFO:teuthology.orchestra.run.smithi060.stdout:4.d 2024-09-20T22:04:34.061 INFO:teuthology.orchestra.run.smithi060.stdout:4.f 2024-09-20T22:04:34.061 INFO:teuthology.orchestra.run.smithi060.stdout:5.12 2024-09-20T22:04:34.061 INFO:teuthology.orchestra.run.smithi060.stdout:4.10 2024-09-20T22:04:34.061 INFO:teuthology.orchestra.run.smithi060.stdout:6.e 2024-09-20T22:04:34.061 INFO:teuthology.orchestra.run.smithi060.stdout:5.15 2024-09-20T22:04:34.061 INFO:teuthology.orchestra.run.smithi060.stdout:5.1a 2024-09-20T22:04:34.061 INFO:teuthology.orchestra.run.smithi060.stdout:5.7 2024-09-20T22:04:34.061 INFO:teuthology.orchestra.run.smithi060.stdout:5.17 2024-09-20T22:04:34.061 INFO:teuthology.orchestra.run.smithi060.stdout:5.18 2024-09-20T22:04:34.061 INFO:teuthology.orchestra.run.smithi060.stdout:5.1d 2024-09-20T22:04:34.061 INFO:teuthology.orchestra.run.smithi060.stdout:5.c 2024-09-20T22:04:34.061 INFO:teuthology.orchestra.run.smithi060.stdout:7.2 2024-09-20T22:04:34.062 INFO:teuthology.orchestra.run.smithi060.stdout:2.2 2024-09-20T22:04:34.062 INFO:teuthology.orchestra.run.smithi060.stdout:6.18 2024-09-20T22:04:34.062 INFO:teuthology.orchestra.run.smithi060.stdout:3.e 2024-09-20T22:04:34.062 INFO:teuthology.orchestra.run.smithi060.stdout:7.0 2024-09-20T22:04:34.062 INFO:teuthology.orchestra.run.smithi060.stdout:6.1a 2024-09-20T22:04:34.062 INFO:teuthology.orchestra.run.smithi060.stdout:6.1d 2024-09-20T22:04:34.062 INFO:teuthology.orchestra.run.smithi060.stdout:6.4 2024-09-20T22:04:34.062 INFO:teuthology.orchestra.run.smithi060.stdout:6.b 2024-09-20T22:04:34.062 INFO:teuthology.orchestra.run.smithi060.stdout:6.d 2024-09-20T22:04:34.141 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:34 smithi060 ceph-mon[29694]: pgmap v64: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 3.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 114 MiB/s wr, 1.82k op/s; 23648/189357 objects degraded (12.489%); 3912/189357 objects misplaced (2.066%) 2024-09-20T22:04:34.141 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:34 smithi060 ceph-mon[40290]: pgmap v64: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 3.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 114 MiB/s wr, 1.82k op/s; 23648/189357 objects degraded (12.489%); 3912/189357 objects misplaced (2.066%) 2024-09-20T22:04:34.155 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:34.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:34.202 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:34.202 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:34.283 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 37 16 68648 68632 115.889 90.25 0.0037762 0.00862416 2024-09-20T22:04:34.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:34.341 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:34.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:34.342 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:34.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:34.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:34.483 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:34.484 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:34.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:34 smithi067 ceph-mon[33511]: pgmap v64: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 3.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 114 MiB/s wr, 1.82k op/s; 23648/189357 objects degraded (12.489%); 3912/189357 objects misplaced (2.066%) 2024-09-20T22:04:34.585 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:34.615 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:34.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:34.616 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:34.717 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:34.749 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:34.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:34.750 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:34.851 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:34.886 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:34.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:34.887 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:34.988 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:35.022 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:35.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:35.023 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:35.124 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:35.159 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:35.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:35.160 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:35.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:35.284 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 38 16 70202 70186 115.392 97.125 0.0100385 0.00865952 2024-09-20T22:04:35.296 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:35.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:35.297 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:35.337 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:04:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:04:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:04:35.398 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:35.434 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:35.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:35.435 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:35.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:35.572 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:35.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:35.573 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:35.674 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:35.710 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:35.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:35.711 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:35.812 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:35.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:35.846 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:35.847 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:35.947 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:35.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:35.980 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:35.981 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:35 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:35 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:04:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:35 smithi067 ceph-mon[33511]: pgmap v65: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 3.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 538 B/s rd, 80 MiB/s wr, 1.28k op/s; 23648/189357 objects degraded (12.489%); 3912/189357 objects misplaced (2.066%) 2024-09-20T22:04:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:35 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:35 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:04:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:35 smithi060 ceph-mon[29694]: pgmap v65: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 3.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 538 B/s rd, 80 MiB/s wr, 1.28k op/s; 23648/189357 objects degraded (12.489%); 3912/189357 objects misplaced (2.066%) 2024-09-20T22:04:36.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:35 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:04:36.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:35 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:04:36.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:35 smithi060 ceph-mon[40290]: pgmap v65: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 3.8 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 538 B/s rd, 80 MiB/s wr, 1.28k op/s; 23648/189357 objects degraded (12.489%); 3912/189357 objects misplaced (2.066%) 2024-09-20T22:04:36.082 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:36.112 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:36.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:36.113 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:36.214 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:36.239 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:36.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:36.240 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:36.284 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 39 16 71604 71588 114.68 87.625 0.00624572 0.00871422 2024-09-20T22:04:36.341 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:36.372 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:36.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:36.373 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:36.474 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:36.505 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:36.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:36.506 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:36.607 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:36.637 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:36.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:36.638 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:36.739 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:36.770 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:36.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:36.771 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:36.872 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:36.900 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:36.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:36.901 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:36.971 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:36 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 23648/189357 objects degraded (12.489%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:36.971 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:36 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 23648/189357 objects degraded (12.489%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:37.002 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:36 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 23648/189357 objects degraded (12.489%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:37.034 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:37.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:37.035 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:37.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:37.178 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:37.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:37.179 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:37.270 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:04:37.270 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:04:37.280 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:37.284 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:04:37.283929+0000 min lat: 0.000984708 max lat: 1.08039 avg lat: 0.00875507 2024-09-20T22:04:37.284 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:04:37.284 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 40 16 73092 73076 114.138 93 0.00901247 0.00875507 2024-09-20T22:04:37.331 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:37.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:37.332 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:37.432 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:37.461 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:37.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:37.462 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:37.563 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:37.601 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:37.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:37.602 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:37.703 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:37.735 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:37.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:37.736 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:37.837 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:37.866 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:37.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:37.867 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:37.967 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:38.003 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:38.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:38.004 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:38.105 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:38.139 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:38.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:38.140 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:38.241 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:38.275 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:38.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:38.276 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:38.284 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 41 16 74497 74481 113.496 87.8125 0.00529501 0.0088024 2024-09-20T22:04:38.377 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:38.404 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:38.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:38.406 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:38.507 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:38.540 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:38.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:38.541 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:38.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:38.677 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:38.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:38.678 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:38.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:38 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:04:38.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:38 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:04:38.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:38 smithi067 ceph-mon[33511]: pgmap v66: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 4.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 906 B/s rd, 89 MiB/s wr, 1.43k op/s; 26550/212319 objects degraded (12.505%); 4371/212319 objects misplaced (2.059%) 2024-09-20T22:04:38.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:38 smithi067 ceph-mon[33511]: osdmap e99: 8 total, 7 up, 8 in 2024-09-20T22:04:38.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:38 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/252319529' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:04:38.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:38 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:04:38.779 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:38.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:38 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:04:38.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:38 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:04:38.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:38 smithi060 ceph-mon[29694]: pgmap v66: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 4.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 906 B/s rd, 89 MiB/s wr, 1.43k op/s; 26550/212319 objects degraded (12.505%); 4371/212319 objects misplaced (2.059%) 2024-09-20T22:04:38.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:38 smithi060 ceph-mon[29694]: osdmap e99: 8 total, 7 up, 8 in 2024-09-20T22:04:38.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:38 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/252319529' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:04:38.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:38 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:04:38.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:38 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:04:38.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:38 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:04:38.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:38 smithi060 ceph-mon[40290]: pgmap v66: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 4.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 906 B/s rd, 89 MiB/s wr, 1.43k op/s; 26550/212319 objects degraded (12.505%); 4371/212319 objects misplaced (2.059%) 2024-09-20T22:04:38.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:38 smithi060 ceph-mon[40290]: osdmap e99: 8 total, 7 up, 8 in 2024-09-20T22:04:38.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:38 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/252319529' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:04:38.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:38 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:04:38.817 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:38.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:38.818 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:38.919 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:38.955 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:38.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:38.956 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:39.057 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:39.094 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:39.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:39.095 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:39.195 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:39.229 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:39.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:39.230 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:39.284 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 42 16 75918 75902 112.908 88.8125 0.00470159 0.00885141 2024-09-20T22:04:39.331 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:39.372 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:39.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:39.374 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:39.430 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T22:04:39.468 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:39 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:04:39.468 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:39 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:04:39.468 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:39 smithi067 ceph-mon[33511]: osdmap e100: 8 total, 7 up, 8 in 2024-09-20T22:04:39.474 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:04:39.474 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:39.516 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:39.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:39.517 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:39.617 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:39.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:39.656 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:39.656 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:39.707 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:39 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:04:39.708 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:39 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:04:39.708 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:39 smithi060 ceph-mon[29694]: osdmap e100: 8 total, 7 up, 8 in 2024-09-20T22:04:39.708 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:39 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:04:39.708 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:39 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:04:39.708 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:39 smithi060 ceph-mon[40290]: osdmap e100: 8 total, 7 up, 8 in 2024-09-20T22:04:39.756 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:39.797 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:39.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:39.798 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:39.897 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:04:39.899 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:39.913 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:39.941 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:39.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:39.942 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:40.043 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:40.084 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:40.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:40.085 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:40.186 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:40.225 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:40.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:40.226 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:40.284 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 43 16 77348 77332 112.361 89.375 0.0138044 0.00889491 2024-09-20T22:04:40.327 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:40.367 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:40.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:40.370 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:40.470 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:40.499 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -n osd.1 --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --op export --pgid 5.5 --file /var/log/ceph/exp.5.5.1 2024-09-20T22:04:40.503 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:40.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:40.504 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:40.553 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:04:40.553 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:04:40.600 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:40 smithi060 ceph-mon[29694]: pgmap v69: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 4.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 60 MiB/s wr, 956 op/s; 26550/212319 objects degraded (12.505%); 4371/212319 objects misplaced (2.059%) 2024-09-20T22:04:40.600 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:40 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:04:40.600 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:40 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:04:40.600 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:40 smithi060 ceph-mon[29694]: osdmap e101: 8 total, 7 up, 8 in 2024-09-20T22:04:40.600 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:40 smithi060 ceph-mon[40290]: pgmap v69: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 4.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 60 MiB/s wr, 956 op/s; 26550/212319 objects degraded (12.505%); 4371/212319 objects misplaced (2.059%) 2024-09-20T22:04:40.600 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:40 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:04:40.601 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:40 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:04:40.601 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:40 smithi060 ceph-mon[40290]: osdmap e101: 8 total, 7 up, 8 in 2024-09-20T22:04:40.605 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:40.612 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:40 smithi067 ceph-mon[33511]: pgmap v69: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 4.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 60 MiB/s wr, 956 op/s; 26550/212319 objects degraded (12.505%); 4371/212319 objects misplaced (2.059%) 2024-09-20T22:04:40.613 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:40 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:04:40.613 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:40 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:04:40.613 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:40 smithi067 ceph-mon[33511]: osdmap e101: 8 total, 7 up, 8 in 2024-09-20T22:04:40.637 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:40.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:40.638 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:40.739 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:40.768 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:40.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:40.769 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:40.870 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:40.903 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/osd.1/config 2024-09-20T22:04:40.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:40.919 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:40.919 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:41.019 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:41.049 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:41.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:41.050 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:41.151 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:41.195 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:41.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:41.196 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:41.284 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 44 16 78749 78733 111.797 87.5625 0.00895873 0.00893995 2024-09-20T22:04:41.297 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:41.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:41.337 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:41.337 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:41.369 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T22:04:41.438 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:41.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:41.480 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:41.480 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:41.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:41 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 26550/212319 objects degraded (12.505%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:41.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:41 smithi060 ceph-mon[29694]: osdmap e102: 8 total, 7 up, 8 in 2024-09-20T22:04:41.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:41 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 26550/212319 objects degraded (12.505%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:41.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:41 smithi060 ceph-mon[40290]: osdmap e102: 8 total, 7 up, 8 in 2024-09-20T22:04:41.581 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:41.626 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:41.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:41.627 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:41.728 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:41.755 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:04:41.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:41 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 26550/212319 objects degraded (12.505%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:41.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:41 smithi067 ceph-mon[33511]: osdmap e102: 8 total, 7 up, 8 in 2024-09-20T22:04:41.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:41.767 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:41.767 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:41.867 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:41.904 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:41.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:41.906 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:42.007 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:42.051 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:42.052 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:04:41 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:04:42.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:42.053 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:42.154 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:42.188 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:42.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:42.189 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:42.285 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 45 16 80174 80158 111.292 89.0625 0.00583334 0.00897822 2024-09-20T22:04:42.290 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:42.312 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:42.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:42.313 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:42.383 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:04:42.383 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:04:42.414 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:42.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:42.451 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:42.451 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:42.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:42.590 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:42.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:42.591 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:42.691 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:42.719 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:42.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:42.720 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:42.733 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:42 smithi067 ceph-mon[33511]: pgmap v71: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 4.7 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.7 KiB/s rd, 154 MiB/s wr, 2.47k op/s; 29184/233874 objects degraded (12.479%); 4821/233874 objects misplaced (2.061%) 2024-09-20T22:04:42.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:42 smithi060 ceph-mon[29694]: pgmap v71: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 4.7 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.7 KiB/s rd, 154 MiB/s wr, 2.47k op/s; 29184/233874 objects degraded (12.479%); 4821/233874 objects misplaced (2.061%) 2024-09-20T22:04:42.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:42 smithi060 ceph-mon[40290]: pgmap v71: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 4.7 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.7 KiB/s rd, 154 MiB/s wr, 2.47k op/s; 29184/233874 objects degraded (12.479%); 4821/233874 objects misplaced (2.061%) 2024-09-20T22:04:42.821 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:42.858 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:42.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:42.859 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:42.956 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:04:42.960 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:42.994 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:42.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:42.995 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:43.076 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T22:04:43.096 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:43.150 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:43.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:43.151 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:43.251 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:43.286 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 46 16 81467 81451 110.626 80.8125 0.0184071 0.00903381 2024-09-20T22:04:43.307 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:43.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:43.308 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:43.409 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:43.440 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:43.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:43.441 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:43.448 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:04:43.541 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:43.576 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:43.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:43.577 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:43.593 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:43 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:04:43.593 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:43 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/785545035' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:04:43.603 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:04:43.603 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:04:43.603 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:04:43.603 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:04:43.603 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:04:43.603 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:04:43.604 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:04:43.604 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:04:43.604 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:04:43.604 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:04:43.604 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:04:43.604 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:04:43.604 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:04:43.604 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:04:43.604 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:04:43.605 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:43 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:04:43.605 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:43 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/785545035' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:04:43.605 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:43 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:04:43.605 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:43 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/785545035' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:04:43.677 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:43.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:43.711 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:43.711 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:43.812 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:43.853 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:43.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:43.854 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:43.955 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:43.991 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:43.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:43.992 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:44.057 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:04:44.058 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:04:44.058 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:04:44.058 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:04:44.058 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.058 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:18348 7.1 7:8016ca7d:::benchmark_data_smithi060_78800_object18347:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:04:44.058 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:04.211319+0000", 2024-09-20T22:04:44.058 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 39.843607437999999, 2024-09-20T22:04:44.058 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.064977568999999999, 2024-09-20T22:04:44.058 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.058 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.058 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.059 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:44.059 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:44.059 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:44.059 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 18348 2024-09-20T22:04:44.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.059 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.059 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.059 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.059 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.211319+0000", 2024-09-20T22:04:44.059 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.059 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.059 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.059 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.211319+0000", 2024-09-20T22:04:44.059 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.060 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.060 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.211321+0000", 2024-09-20T22:04:44.060 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.373e-06 2024-09-20T22:04:44.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.060 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.060 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.211370+0000", 2024-09-20T22:04:44.060 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9018999999999999e-05 2024-09-20T22:04:44.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.060 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.060 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.211381+0000", 2024-09-20T22:04:44.061 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1269e-05 2024-09-20T22:04:44.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.061 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.061 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.211397+0000", 2024-09-20T22:04:44.061 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5838e-05 2024-09-20T22:04:44.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.061 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.061 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.211456+0000", 2024-09-20T22:04:44.061 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9009999999999999e-05 2024-09-20T22:04:44.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.061 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.061 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.211580+0000", 2024-09-20T22:04:44.062 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000123938 2024-09-20T22:04:44.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.062 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.062 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:04:44.062 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.211708+0000", 2024-09-20T22:04:44.062 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000128319 2024-09-20T22:04:44.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.062 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.062 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:44.062 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.212464+0000", 2024-09-20T22:04:44.062 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00075599399999999996 2024-09-20T22:04:44.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.062 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.062 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.063 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.212707+0000", 2024-09-20T22:04:44.063 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00024298200000000001 2024-09-20T22:04:44.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.063 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.063 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.063 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.276183+0000", 2024-09-20T22:04:44.063 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.063475911999999995 2024-09-20T22:04:44.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.063 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.063 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.063 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.276246+0000", 2024-09-20T22:04:44.063 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3474999999999998e-05 2024-09-20T22:04:44.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.063 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.063 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.064 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.276297+0000", 2024-09-20T22:04:44.064 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0439999999999998e-05 2024-09-20T22:04:44.064 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.064 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.064 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.064 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.064 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:18658 7.1 7:8a29fbc2:::benchmark_data_smithi060_78800_object18657:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:04:44.064 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:04.435506+0000", 2024-09-20T22:04:44.064 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 39.619420429999998, 2024-09-20T22:04:44.064 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.020549718000000002, 2024-09-20T22:04:44.064 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.064 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.064 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.064 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:44.065 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:44.065 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:44.065 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 18658 2024-09-20T22:04:44.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.065 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.065 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.065 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.065 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.435506+0000", 2024-09-20T22:04:44.065 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.065 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.065 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.065 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.435506+0000", 2024-09-20T22:04:44.066 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.066 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.066 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.066 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.435508+0000", 2024-09-20T22:04:44.066 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.711e-06 2024-09-20T22:04:44.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.066 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.066 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.066 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.435563+0000", 2024-09-20T22:04:44.066 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5588999999999999e-05 2024-09-20T22:04:44.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.066 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.066 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.067 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.435578+0000", 2024-09-20T22:04:44.067 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4980999999999999e-05 2024-09-20T22:04:44.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.067 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.067 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.435585+0000", 2024-09-20T22:04:44.067 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9609999999999999e-06 2024-09-20T22:04:44.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.067 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.067 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.435637+0000", 2024-09-20T22:04:44.067 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2043999999999998e-05 2024-09-20T22:04:44.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.068 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.068 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.435722+0000", 2024-09-20T22:04:44.068 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4612999999999998e-05 2024-09-20T22:04:44.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.068 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:04:44.068 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.435851+0000", 2024-09-20T22:04:44.068 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012898900000000001 2024-09-20T22:04:44.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.068 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:44.068 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.436782+0000", 2024-09-20T22:04:44.068 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00093117099999999995 2024-09-20T22:04:44.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.068 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.069 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.437172+0000", 2024-09-20T22:04:44.069 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00038944899999999999 2024-09-20T22:04:44.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.069 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.069 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.069 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.455947+0000", 2024-09-20T22:04:44.069 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.018775134999999998 2024-09-20T22:04:44.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.069 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.069 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.069 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.456010+0000", 2024-09-20T22:04:44.069 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3058999999999994e-05 2024-09-20T22:04:44.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.069 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.070 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.070 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:04.456056+0000", 2024-09-20T22:04:44.070 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6016000000000001e-05 2024-09-20T22:04:44.070 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.070 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.070 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.070 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.070 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:21066 7.1 7:88b2880e:::benchmark_data_smithi060_78800_object21065:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:04:44.070 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:05.425238+0000", 2024-09-20T22:04:44.070 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 38.629688942000001, 2024-09-20T22:04:44.071 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.064494489000000002, 2024-09-20T22:04:44.071 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.071 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.071 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.071 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:44.071 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:44.071 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:44.072 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 21066 2024-09-20T22:04:44.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.072 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.072 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.072 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.072 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.425238+0000", 2024-09-20T22:04:44.072 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.072 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.072 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.072 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.425238+0000", 2024-09-20T22:04:44.072 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.073 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.073 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.073 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.425239+0000", 2024-09-20T22:04:44.073 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4339999999999999e-06 2024-09-20T22:04:44.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.073 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.073 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.073 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.425320+0000", 2024-09-20T22:04:44.073 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0937000000000001e-05 2024-09-20T22:04:44.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.073 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.073 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.073 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.425333+0000", 2024-09-20T22:04:44.073 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2918000000000001e-05 2024-09-20T22:04:44.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.073 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.074 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.074 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.425340+0000", 2024-09-20T22:04:44.074 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1600000000000001e-06 2024-09-20T22:04:44.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.074 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.074 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.425413+0000", 2024-09-20T22:04:44.074 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3281999999999994e-05 2024-09-20T22:04:44.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.074 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.074 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.425556+0000", 2024-09-20T22:04:44.074 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014210199999999999 2024-09-20T22:04:44.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.075 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:04:44.075 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.425704+0000", 2024-09-20T22:04:44.075 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000148961 2024-09-20T22:04:44.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.075 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.075 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:44.075 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.426541+0000", 2024-09-20T22:04:44.075 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000836555 2024-09-20T22:04:44.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.075 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.075 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.075 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.426880+0000", 2024-09-20T22:04:44.075 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00033893200000000001 2024-09-20T22:04:44.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.076 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.076 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.489567+0000", 2024-09-20T22:04:44.076 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.062686936999999998 2024-09-20T22:04:44.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.076 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.076 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.489661+0000", 2024-09-20T22:04:44.076 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4147999999999994e-05 2024-09-20T22:04:44.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.076 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.076 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.489732+0000", 2024-09-20T22:04:44.076 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1123000000000003e-05 2024-09-20T22:04:44.076 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.077 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.077 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.077 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.077 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:21074 7.1 7:86e799a0:::benchmark_data_smithi060_78800_object21073:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:04:44.077 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:05.427895+0000", 2024-09-20T22:04:44.077 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 38.627031768000002, 2024-09-20T22:04:44.077 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.065206694999999995, 2024-09-20T22:04:44.077 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.077 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.077 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.077 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:44.077 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:44.078 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:44.078 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 21074 2024-09-20T22:04:44.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.078 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.078 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.078 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.078 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.427895+0000", 2024-09-20T22:04:44.078 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.078 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.078 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.078 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.427895+0000", 2024-09-20T22:04:44.079 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.079 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.079 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.079 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.427896+0000", 2024-09-20T22:04:44.079 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9999999999999996e-07 2024-09-20T22:04:44.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.079 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.079 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.080 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.427950+0000", 2024-09-20T22:04:44.080 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4466000000000003e-05 2024-09-20T22:04:44.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.080 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.080 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.080 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.427964+0000", 2024-09-20T22:04:44.080 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3988999999999999e-05 2024-09-20T22:04:44.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.080 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.080 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.080 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.427972+0000", 2024-09-20T22:04:44.080 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7300000000000005e-06 2024-09-20T22:04:44.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.081 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.081 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.081 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.428028+0000", 2024-09-20T22:04:44.081 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6087e-05 2024-09-20T22:04:44.081 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.081 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.081 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.081 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.428146+0000", 2024-09-20T22:04:44.081 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000118503 2024-09-20T22:04:44.081 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.081 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.081 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:04:44.081 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.428247+0000", 2024-09-20T22:04:44.081 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010062500000000001 2024-09-20T22:04:44.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.082 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.082 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:44.082 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.429061+0000", 2024-09-20T22:04:44.082 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000813969 2024-09-20T22:04:44.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.082 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.082 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.082 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.429405+0000", 2024-09-20T22:04:44.082 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00034416699999999999 2024-09-20T22:04:44.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.082 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.082 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.082 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.492997+0000", 2024-09-20T22:04:44.083 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.063591618000000003 2024-09-20T22:04:44.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.083 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.083 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.083 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.493057+0000", 2024-09-20T22:04:44.083 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0309000000000001e-05 2024-09-20T22:04:44.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.083 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.083 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.083 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:05.493102+0000", 2024-09-20T22:04:44.084 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4431999999999997e-05 2024-09-20T22:04:44.084 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.084 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.084 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.084 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.084 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.084 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:22994 7.1 7:83bb3ca8:::benchmark_data_smithi060_78800_object22993:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:04:44.084 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:06.374865+0000", 2024-09-20T22:04:44.084 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 37.680061451999997, 2024-09-20T22:04:44.084 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.061486762, 2024-09-20T22:04:44.084 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.084 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.085 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.085 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:44.085 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:44.085 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:44.085 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 22994 2024-09-20T22:04:44.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.085 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.085 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.085 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.374865+0000", 2024-09-20T22:04:44.085 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.086 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.086 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.374865+0000", 2024-09-20T22:04:44.086 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.086 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.086 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.374866+0000", 2024-09-20T22:04:44.086 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4899999999999995e-07 2024-09-20T22:04:44.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.086 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.086 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.374919+0000", 2024-09-20T22:04:44.086 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3044999999999998e-05 2024-09-20T22:04:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.374931+0000", 2024-09-20T22:04:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1997000000000001e-05 2024-09-20T22:04:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.374938+0000", 2024-09-20T22:04:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8669999999999996e-06 2024-09-20T22:04:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.087 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.374982+0000", 2024-09-20T22:04:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4724000000000001e-05 2024-09-20T22:04:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.375054+0000", 2024-09-20T22:04:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1503999999999998e-05 2024-09-20T22:04:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:04:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.375134+0000", 2024-09-20T22:04:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9870999999999995e-05 2024-09-20T22:04:44.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.375825+0000", 2024-09-20T22:04:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000691448 2024-09-20T22:04:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.375988+0000", 2024-09-20T22:04:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000162351 2024-09-20T22:04:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.436232+0000", 2024-09-20T22:04:44.089 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.060244515999999998 2024-09-20T22:04:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.436310+0000", 2024-09-20T22:04:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8088000000000002e-05 2024-09-20T22:04:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.436352+0000", 2024-09-20T22:04:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1702000000000002e-05 2024-09-20T22:04:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.090 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:23339 7.1 7:8520128b:::benchmark_data_smithi060_78800_object23338:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:04:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:06.571950+0000", 2024-09-20T22:04:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 37.482976430000001, 2024-09-20T22:04:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.034403712000000003, 2024-09-20T22:04:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 23339 2024-09-20T22:04:44.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.571950+0000", 2024-09-20T22:04:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.571950+0000", 2024-09-20T22:04:44.092 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.571952+0000", 2024-09-20T22:04:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.392e-06 2024-09-20T22:04:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.572037+0000", 2024-09-20T22:04:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5332000000000002e-05 2024-09-20T22:04:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.093 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.572052+0000", 2024-09-20T22:04:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4865000000000001e-05 2024-09-20T22:04:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.572090+0000", 2024-09-20T22:04:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7960999999999998e-05 2024-09-20T22:04:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.572151+0000", 2024-09-20T22:04:44.094 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1111999999999994e-05 2024-09-20T22:04:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.572224+0000", 2024-09-20T22:04:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3097999999999997e-05 2024-09-20T22:04:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:04:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.572312+0000", 2024-09-20T22:04:44.095 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.7992999999999996e-05 2024-09-20T22:04:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.573446+0000", 2024-09-20T22:04:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001133738 2024-09-20T22:04:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.573649+0000", 2024-09-20T22:04:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000203518 2024-09-20T22:04:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.096 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.606244+0000", 2024-09-20T22:04:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.032594351000000001 2024-09-20T22:04:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.606311+0000", 2024-09-20T22:04:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7726999999999997e-05 2024-09-20T22:04:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.606354+0000", 2024-09-20T22:04:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2624999999999999e-05 2024-09-20T22:04:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.097 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:23340 7.1 7:8d84bbb7:::benchmark_data_smithi060_78800_object23339:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:04:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:06.574405+0000", 2024-09-20T22:04:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 37.480521406000001, 2024-09-20T22:04:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.032358381999999998, 2024-09-20T22:04:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:44.098 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 23340 2024-09-20T22:04:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.574405+0000", 2024-09-20T22:04:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.574405+0000", 2024-09-20T22:04:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.099 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.574406+0000", 2024-09-20T22:04:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.031e-06 2024-09-20T22:04:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.574458+0000", 2024-09-20T22:04:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1347000000000002e-05 2024-09-20T22:04:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.100 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.574469+0000", 2024-09-20T22:04:44.100 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:44.102 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1317e-05 2024-09-20T22:04:44.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.102 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.102 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.102 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.574476+0000", 2024-09-20T22:04:44.102 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6080000000000001e-06 2024-09-20T22:04:44.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.102 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.102 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.102 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.574524+0000", 2024-09-20T22:04:44.102 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8399999999999997e-05 2024-09-20T22:04:44.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.103 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.103 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.103 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.574612+0000", 2024-09-20T22:04:44.103 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.7703999999999999e-05 2024-09-20T22:04:44.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.103 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.103 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:04:44.103 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.574695+0000", 2024-09-20T22:04:44.103 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3473999999999994e-05 2024-09-20T22:04:44.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.103 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.103 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:44.103 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.575429+0000", 2024-09-20T22:04:44.103 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00073364399999999998 2024-09-20T22:04:44.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.104 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.104 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.104 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.575580+0000", 2024-09-20T22:04:44.104 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015128200000000001 2024-09-20T22:04:44.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.104 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.104 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.104 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.606675+0000", 2024-09-20T22:04:44.104 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.031095133 2024-09-20T22:04:44.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.105 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.105 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.105 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.606728+0000", 2024-09-20T22:04:44.105 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2494000000000003e-05 2024-09-20T22:04:44.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.105 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.105 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.105 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.606764+0000", 2024-09-20T22:04:44.105 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5948000000000002e-05 2024-09-20T22:04:44.105 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.106 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.106 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.106 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.106 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:23754 7.1 7:8278a320:::benchmark_data_smithi060_78800_object23753:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:04:44.106 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:06.788242+0000", 2024-09-20T22:04:44.106 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 37.266684835, 2024-09-20T22:04:44.106 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.073252084999999995, 2024-09-20T22:04:44.106 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.106 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.107 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.107 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:44.107 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:44.107 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:44.107 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 23754 2024-09-20T22:04:44.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.107 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.107 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.107 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.788242+0000", 2024-09-20T22:04:44.107 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.108 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.108 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.788242+0000", 2024-09-20T22:04:44.108 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.108 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.108 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.788243+0000", 2024-09-20T22:04:44.108 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2500000000000004e-07 2024-09-20T22:04:44.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.108 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.108 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.788294+0000", 2024-09-20T22:04:44.108 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1440000000000002e-05 2024-09-20T22:04:44.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.109 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.109 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.788304+0000", 2024-09-20T22:04:44.109 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0169999999999999e-05 2024-09-20T22:04:44.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.109 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.109 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.788309+0000", 2024-09-20T22:04:44.109 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8189999999999999e-06 2024-09-20T22:04:44.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.109 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.109 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.788362+0000", 2024-09-20T22:04:44.109 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3362999999999997e-05 2024-09-20T22:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.788501+0000", 2024-09-20T22:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013829700000000001 2024-09-20T22:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.788648+0000", 2024-09-20T22:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014713399999999999 2024-09-20T22:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:44.110 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.789539+0000", 2024-09-20T22:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00089103900000000005 2024-09-20T22:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.789925+0000", 2024-09-20T22:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000385986 2024-09-20T22:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.861378+0000", 2024-09-20T22:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.071453336000000006 2024-09-20T22:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.111 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.861444+0000", 2024-09-20T22:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5597000000000002e-05 2024-09-20T22:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.861494+0000", 2024-09-20T22:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0078999999999999e-05 2024-09-20T22:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:23790 7.1 7:8c0e2ee0:::benchmark_data_smithi060_78800_object23789:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:06.872451+0000", 2024-09-20T22:04:44.112 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 37.182475951000001, 2024-09-20T22:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.019136837, 2024-09-20T22:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 23790 2024-09-20T22:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.872451+0000", 2024-09-20T22:04:44.113 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.872451+0000", 2024-09-20T22:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.872452+0000", 2024-09-20T22:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.347e-06 2024-09-20T22:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.114 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.872503+0000", 2024-09-20T22:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0590000000000002e-05 2024-09-20T22:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.872516+0000", 2024-09-20T22:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3743e-05 2024-09-20T22:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.872523+0000", 2024-09-20T22:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4350000000000002e-06 2024-09-20T22:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.115 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.872583+0000", 2024-09-20T22:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0647999999999999e-05 2024-09-20T22:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.872723+0000", 2024-09-20T22:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013970000000000001 2024-09-20T22:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.872864+0000", 2024-09-20T22:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014085399999999999 2024-09-20T22:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.116 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.874316+0000", 2024-09-20T22:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001452474 2024-09-20T22:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.874420+0000", 2024-09-20T22:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000103094 2024-09-20T22:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.891520+0000", 2024-09-20T22:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.017100358999999999 2024-09-20T22:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.891558+0000", 2024-09-20T22:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.8436000000000003e-05 2024-09-20T22:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.118 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.891587+0000", 2024-09-20T22:04:44.119 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.9156999999999999e-05 2024-09-20T22:04:44.119 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.119 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.119 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.119 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.24667.0:27523 7.8 e88/80 7:10cedc8f:::benchmark_data_smithi060_78800_object27522:head v 88'1756, mlcod=88'1753)", 2024-09-20T22:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:08.616286+0000", 2024-09-20T22:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 35.438640612, 2024-09-20T22:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.019452936000000001, 2024-09-20T22:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.120 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.616286+0000", 2024-09-20T22:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.122 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.122 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.616286+0000", 2024-09-20T22:04:44.122 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.122 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.122 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.616287+0000", 2024-09-20T22:04:44.123 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2900000000000004e-07 2024-09-20T22:04:44.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.123 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.123 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.616325+0000", 2024-09-20T22:04:44.123 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.8253000000000001e-05 2024-09-20T22:04:44.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.124 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.616334+0000", 2024-09-20T22:04:44.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8060000000000001e-06 2024-09-20T22:04:44.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.125 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.125 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.616339+0000", 2024-09-20T22:04:44.125 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8850000000000004e-06 2024-09-20T22:04:44.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.125 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.126 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.616396+0000", 2024-09-20T22:04:44.126 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6557000000000001e-05 2024-09-20T22:04:44.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.126 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.126 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.126 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.616414+0000", 2024-09-20T22:04:44.127 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8101000000000002e-05 2024-09-20T22:04:44.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.127 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.127 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.635689+0000", 2024-09-20T22:04:44.127 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.019274974 2024-09-20T22:04:44.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.128 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.128 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.635739+0000", 2024-09-20T22:04:44.128 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9830999999999998e-05 2024-09-20T22:04:44.128 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.128 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.129 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.129 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.129 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.24667.0:27529 7.9 e88/80 7:9d2b1959:::benchmark_data_smithi060_78800_object27528:head v 88'1678, mlcod=88'1676)", 2024-09-20T22:04:44.129 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:08.617721+0000", 2024-09-20T22:04:44.129 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 35.437205698, 2024-09-20T22:04:44.129 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.020323272, 2024-09-20T22:04:44.130 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.130 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.130 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.130 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.130 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.131 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.617721+0000", 2024-09-20T22:04:44.131 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.131 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.131 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.617721+0000", 2024-09-20T22:04:44.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.132 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.132 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.617722+0000", 2024-09-20T22:04:44.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6799999999999999e-07 2024-09-20T22:04:44.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.133 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.133 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.133 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.617758+0000", 2024-09-20T22:04:44.133 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6121999999999997e-05 2024-09-20T22:04:44.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.133 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.133 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.134 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.617770+0000", 2024-09-20T22:04:44.134 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2646000000000001e-05 2024-09-20T22:04:44.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.134 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.134 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.134 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.617778+0000", 2024-09-20T22:04:44.134 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1570000000000003e-06 2024-09-20T22:04:44.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.135 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.135 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.617953+0000", 2024-09-20T22:04:44.135 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017570699999999999 2024-09-20T22:04:44.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.135 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.617969+0000", 2024-09-20T22:04:44.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5423000000000001e-05 2024-09-20T22:04:44.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.136 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.637995+0000", 2024-09-20T22:04:44.137 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.020025869000000002 2024-09-20T22:04:44.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.137 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.137 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.137 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:08.638044+0000", 2024-09-20T22:04:44.137 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9580000000000003e-05 2024-09-20T22:04:44.138 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.138 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.138 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.138 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.138 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:29494 7.1 7:83843e4d:::benchmark_data_smithi060_78800_object29493:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:04:44.139 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:09.655302+0000", 2024-09-20T22:04:44.139 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 34.399624279999998, 2024-09-20T22:04:44.139 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.042224163000000002, 2024-09-20T22:04:44.139 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.139 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.139 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.139 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:44.140 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:44.140 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:44.140 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 29494 2024-09-20T22:04:44.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.140 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.140 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.140 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.141 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:09.655302+0000", 2024-09-20T22:04:44.141 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.141 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.141 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:09.655302+0000", 2024-09-20T22:04:44.142 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.142 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.142 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:09.655304+0000", 2024-09-20T22:04:44.142 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4410000000000001e-06 2024-09-20T22:04:44.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.143 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.143 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:09.655352+0000", 2024-09-20T22:04:44.143 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8402000000000002e-05 2024-09-20T22:04:44.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:09.655364+0000", 2024-09-20T22:04:44.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1953000000000001e-05 2024-09-20T22:04:44.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:09.655372+0000", 2024-09-20T22:04:44.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8720000000000002e-06 2024-09-20T22:04:44.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.145 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:09.655440+0000", 2024-09-20T22:04:44.145 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8058999999999993e-05 2024-09-20T22:04:44.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.145 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.145 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:09.655568+0000", 2024-09-20T22:04:44.145 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012796500000000001 2024-09-20T22:04:44.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.145 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:04:44.145 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:09.655699+0000", 2024-09-20T22:04:44.145 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013098499999999999 2024-09-20T22:04:44.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.146 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:44.146 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:09.656758+0000", 2024-09-20T22:04:44.146 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001058523 2024-09-20T22:04:44.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.146 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.146 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:09.656890+0000", 2024-09-20T22:04:44.146 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000132736 2024-09-20T22:04:44.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.146 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.146 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:09.697407+0000", 2024-09-20T22:04:44.146 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.040516306000000002 2024-09-20T22:04:44.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.147 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.147 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:09.697474+0000", 2024-09-20T22:04:44.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6956000000000001e-05 2024-09-20T22:04:44.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.147 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.147 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:09.697526+0000", 2024-09-20T22:04:44.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2964999999999999e-05 2024-09-20T22:04:44.147 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.147 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.147 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.148 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:32227 7.1 7:8ffff13b:::benchmark_data_smithi060_78800_object32226:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-20T22:04:44.148 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:11.126113+0000", 2024-09-20T22:04:44.148 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 32.928813195000004, 2024-09-20T22:04:44.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.026634893, 2024-09-20T22:04:44.148 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.148 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.148 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.148 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:44.148 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:44.148 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:44.148 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 32227 2024-09-20T22:04:44.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.148 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.149 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.149 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.126113+0000", 2024-09-20T22:04:44.149 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.149 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.149 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.126113+0000", 2024-09-20T22:04:44.149 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.149 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.149 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.126115+0000", 2024-09-20T22:04:44.149 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5939999999999999e-06 2024-09-20T22:04:44.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.150 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.150 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.126162+0000", 2024-09-20T22:04:44.150 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6508999999999998e-05 2024-09-20T22:04:44.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.150 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.150 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.126177+0000", 2024-09-20T22:04:44.150 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5410999999999999e-05 2024-09-20T22:04:44.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.150 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.150 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.126183+0000", 2024-09-20T22:04:44.151 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5549999999999999e-06 2024-09-20T22:04:44.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.151 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.151 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.126236+0000", 2024-09-20T22:04:44.151 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2246000000000001e-05 2024-09-20T22:04:44.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.151 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.151 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.126326+0000", 2024-09-20T22:04:44.151 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.0196000000000002e-05 2024-09-20T22:04:44.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.151 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:04:44.152 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.126428+0000", 2024-09-20T22:04:44.152 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000102419 2024-09-20T22:04:44.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.152 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:44.152 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.127446+0000", 2024-09-20T22:04:44.152 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0010181330000000001 2024-09-20T22:04:44.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.152 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.152 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.128949+0000", 2024-09-20T22:04:44.152 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001502335 2024-09-20T22:04:44.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.152 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.153 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.152619+0000", 2024-09-20T22:04:44.153 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.023670008999999999 2024-09-20T22:04:44.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.153 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.153 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.153 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.152692+0000", 2024-09-20T22:04:44.153 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2934000000000003e-05 2024-09-20T22:04:44.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.153 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.153 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.153 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.152748+0000", 2024-09-20T22:04:44.153 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6552000000000003e-05 2024-09-20T22:04:44.153 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.153 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.154 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.154 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.154 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:50623 7.1 7:8246a12c:::benchmark_data_smithi060_78800_object50622:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-20T22:04:44.154 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:22.849521+0000", 2024-09-20T22:04:44.154 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 21.205405176999999, 2024-09-20T22:04:44.154 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.018978198000000002, 2024-09-20T22:04:44.154 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.154 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.154 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.154 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:44.154 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:44.154 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:44.154 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 50623 2024-09-20T22:04:44.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.155 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.155 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.155 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.155 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:22.849521+0000", 2024-09-20T22:04:44.155 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.155 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.155 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.155 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:22.849521+0000", 2024-09-20T22:04:44.155 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.155 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.155 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.155 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:22.849523+0000", 2024-09-20T22:04:44.155 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.325e-06 2024-09-20T22:04:44.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.156 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.156 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.156 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:22.849586+0000", 2024-09-20T22:04:44.156 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3071000000000007e-05 2024-09-20T22:04:44.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.156 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.156 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.156 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:22.849597+0000", 2024-09-20T22:04:44.156 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1078e-05 2024-09-20T22:04:44.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.156 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.156 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.156 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:22.849604+0000", 2024-09-20T22:04:44.156 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5839999999999997e-06 2024-09-20T22:04:44.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.157 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.157 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.157 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:22.849658+0000", 2024-09-20T22:04:44.157 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3235000000000001e-05 2024-09-20T22:04:44.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.157 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.157 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.157 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:22.849743+0000", 2024-09-20T22:04:44.157 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5432000000000005e-05 2024-09-20T22:04:44.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.157 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.157 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:04:44.157 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:22.849831+0000", 2024-09-20T22:04:44.158 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8140000000000007e-05 2024-09-20T22:04:44.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.158 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.158 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:44.158 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:22.851580+0000", 2024-09-20T22:04:44.158 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001749137 2024-09-20T22:04:44.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.158 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.158 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.158 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:22.851829+0000", 2024-09-20T22:04:44.158 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00024874000000000001 2024-09-20T22:04:44.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.158 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.158 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.158 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:22.868379+0000", 2024-09-20T22:04:44.159 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.016549449000000001 2024-09-20T22:04:44.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.159 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.159 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:22.868448+0000", 2024-09-20T22:04:44.159 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8919999999999997e-05 2024-09-20T22:04:44.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.159 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.159 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:22.868500+0000", 2024-09-20T22:04:44.159 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2086999999999998e-05 2024-09-20T22:04:44.159 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.159 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.159 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.160 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.160 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:52673 7.1 7:8dfaca6a:::benchmark_data_smithi060_78800_object52672:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-20T22:04:44.160 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:24.334696+0000", 2024-09-20T22:04:44.160 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 19.720230813000001, 2024-09-20T22:04:44.160 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.027595280999999999, 2024-09-20T22:04:44.160 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.160 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.160 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.160 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:44.160 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:44.160 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:44.160 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 52673 2024-09-20T22:04:44.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.160 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.161 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.161 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.161 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.334696+0000", 2024-09-20T22:04:44.161 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.161 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.161 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.161 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.334696+0000", 2024-09-20T22:04:44.161 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.161 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.161 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.161 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.334697+0000", 2024-09-20T22:04:44.161 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6199999999999999e-06 2024-09-20T22:04:44.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.162 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.162 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.162 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.334745+0000", 2024-09-20T22:04:44.162 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7431e-05 2024-09-20T22:04:44.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.162 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.162 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.162 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.334759+0000", 2024-09-20T22:04:44.162 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3919e-05 2024-09-20T22:04:44.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.162 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.162 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.162 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.334767+0000", 2024-09-20T22:04:44.162 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4710000000000007e-06 2024-09-20T22:04:44.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.163 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.163 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.334823+0000", 2024-09-20T22:04:44.163 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6088999999999998e-05 2024-09-20T22:04:44.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.163 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.163 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.334947+0000", 2024-09-20T22:04:44.163 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000123538 2024-09-20T22:04:44.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.163 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:04:44.163 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.335089+0000", 2024-09-20T22:04:44.164 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014191299999999999 2024-09-20T22:04:44.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.164 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:44.164 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.336670+0000", 2024-09-20T22:04:44.164 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0015811169999999999 2024-09-20T22:04:44.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.164 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.164 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.336987+0000", 2024-09-20T22:04:44.164 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00031720899999999999 2024-09-20T22:04:44.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.164 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.165 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.362216+0000", 2024-09-20T22:04:44.165 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.025228984999999999 2024-09-20T22:04:44.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.165 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.165 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.165 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.362259+0000", 2024-09-20T22:04:44.165 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2985000000000003e-05 2024-09-20T22:04:44.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.165 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.165 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.165 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.362291+0000", 2024-09-20T22:04:44.165 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2004000000000001e-05 2024-09-20T22:04:44.165 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.165 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.166 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.166 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.166 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:52689 7.1 7:8f2cff25:::benchmark_data_smithi060_78800_object52688:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-20T22:04:44.166 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:24.353205+0000", 2024-09-20T22:04:44.166 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 19.70172123, 2024-09-20T22:04:44.166 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.029955072999999999, 2024-09-20T22:04:44.166 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.166 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.166 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.166 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:44.166 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:44.166 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:44.166 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 52689 2024-09-20T22:04:44.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.167 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.167 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.167 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.167 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.353205+0000", 2024-09-20T22:04:44.167 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.167 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.167 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.167 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.353205+0000", 2024-09-20T22:04:44.167 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.167 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.167 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.167 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.353207+0000", 2024-09-20T22:04:44.168 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.674e-06 2024-09-20T22:04:44.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.168 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.168 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.168 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.353266+0000", 2024-09-20T22:04:44.168 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8493999999999999e-05 2024-09-20T22:04:44.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.168 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.168 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.168 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.353289+0000", 2024-09-20T22:04:44.168 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.3827000000000001e-05 2024-09-20T22:04:44.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.168 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.168 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.353297+0000", 2024-09-20T22:04:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4889999999999998e-06 2024-09-20T22:04:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.353356+0000", 2024-09-20T22:04:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8851000000000003e-05 2024-09-20T22:04:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.353492+0000", 2024-09-20T22:04:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013621400000000001 2024-09-20T22:04:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.169 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.170 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:04:44.170 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.353628+0000", 2024-09-20T22:04:44.170 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013612300000000001 2024-09-20T22:04:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.356498+0000", 2024-09-20T22:04:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028694889999999998 2024-09-20T22:04:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.356525+0000", 2024-09-20T22:04:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.703e-05 2024-09-20T22:04:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.171 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.383056+0000", 2024-09-20T22:04:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.026530982000000002 2024-09-20T22:04:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.383111+0000", 2024-09-20T22:04:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5158000000000003e-05 2024-09-20T22:04:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.383160+0000", 2024-09-20T22:04:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9741999999999999e-05 2024-09-20T22:04:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.172 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:52691 7.1 7:8967035b:::benchmark_data_smithi060_78800_object52690:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-09-20T22:04:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:24.353318+0000", 2024-09-20T22:04:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 19.701609101999999, 2024-09-20T22:04:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.030002757000000001, 2024-09-20T22:04:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:44.173 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 52691 2024-09-20T22:04:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.353318+0000", 2024-09-20T22:04:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.353318+0000", 2024-09-20T22:04:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.174 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.353318+0000", 2024-09-20T22:04:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6100000000000001e-07 2024-09-20T22:04:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.353345+0000", 2024-09-20T22:04:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7087e-05 2024-09-20T22:04:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.353355+0000", 2024-09-20T22:04:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6120000000000005e-06 2024-09-20T22:04:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.175 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.353361+0000", 2024-09-20T22:04:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8969999999999996e-06 2024-09-20T22:04:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.355185+0000", 2024-09-20T22:04:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0018239529999999999 2024-09-20T22:04:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.355281+0000", 2024-09-20T22:04:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6798000000000004e-05 2024-09-20T22:04:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.176 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:04:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.355365+0000", 2024-09-20T22:04:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3128999999999996e-05 2024-09-20T22:04:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.356655+0000", 2024-09-20T22:04:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0012900609999999999 2024-09-20T22:04:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.356924+0000", 2024-09-20T22:04:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00026919200000000002 2024-09-20T22:04:44.177 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.383214+0000", 2024-09-20T22:04:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.026289759999999999 2024-09-20T22:04:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.383275+0000", 2024-09-20T22:04:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1818999999999999e-05 2024-09-20T22:04:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.383320+0000", 2024-09-20T22:04:44.178 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4888000000000001e-05 2024-09-20T22:04:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:53395 7.1 7:8dc1e2d3:::benchmark_data_smithi060_78800_object53394:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-20T22:04:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:24.848290+0000", 2024-09-20T22:04:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 19.206636206999999, 2024-09-20T22:04:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.019654365, 2024-09-20T22:04:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:44.179 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 53395 2024-09-20T22:04:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.848290+0000", 2024-09-20T22:04:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.848290+0000", 2024-09-20T22:04:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.180 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.848291+0000", 2024-09-20T22:04:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3399999999999998e-07 2024-09-20T22:04:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.848368+0000", 2024-09-20T22:04:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6855999999999997e-05 2024-09-20T22:04:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.848380+0000", 2024-09-20T22:04:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2234e-05 2024-09-20T22:04:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.181 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.182 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.848397+0000", 2024-09-20T22:04:44.182 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7090000000000001e-05 2024-09-20T22:04:44.182 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.182 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.182 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.182 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.850075+0000", 2024-09-20T22:04:44.182 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0016777809999999999 2024-09-20T22:04:44.182 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.182 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.182 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.182 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.850210+0000", 2024-09-20T22:04:44.182 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000134988 2024-09-20T22:04:44.182 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.182 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.183 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:04:44.183 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.850364+0000", 2024-09-20T22:04:44.183 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015412600000000001 2024-09-20T22:04:44.183 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.183 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.183 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.183 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.853902+0000", 2024-09-20T22:04:44.183 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0035377960000000002 2024-09-20T22:04:44.183 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.183 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.183 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:44.183 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.853913+0000", 2024-09-20T22:04:44.183 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0575e-05 2024-09-20T22:04:44.183 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.183 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.184 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.184 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.867843+0000", 2024-09-20T22:04:44.184 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.013930056999999999 2024-09-20T22:04:44.184 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.184 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.184 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.184 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.867901+0000", 2024-09-20T22:04:44.184 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8050999999999997e-05 2024-09-20T22:04:44.184 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.184 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.184 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.184 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:24.867945+0000", 2024-09-20T22:04:44.184 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3977000000000003e-05 2024-09-20T22:04:44.184 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.185 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.185 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.185 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.185 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.185 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:53693 7.1 7:868865af:::benchmark_data_smithi060_78800_object53692:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e97)", 2024-09-20T22:04:44.185 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:25.079141+0000", 2024-09-20T22:04:44.185 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 18.975785553000001, 2024-09-20T22:04:44.185 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.023898644, 2024-09-20T22:04:44.185 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.185 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.185 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.185 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:44.185 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:44.185 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:44.186 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 53693 2024-09-20T22:04:44.186 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.186 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.186 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.186 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.186 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:25.079141+0000", 2024-09-20T22:04:44.186 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.186 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.186 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.186 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.186 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:25.079141+0000", 2024-09-20T22:04:44.186 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.186 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.186 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.187 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.187 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:25.079142+0000", 2024-09-20T22:04:44.187 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.381e-06 2024-09-20T22:04:44.187 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.187 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.187 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.187 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:25.079196+0000", 2024-09-20T22:04:44.187 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3672999999999999e-05 2024-09-20T22:04:44.187 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.187 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.187 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.187 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:25.079248+0000", 2024-09-20T22:04:44.187 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.198e-05 2024-09-20T22:04:44.187 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.187 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.188 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.188 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:25.079257+0000", 2024-09-20T22:04:44.188 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5720000000000003e-06 2024-09-20T22:04:44.188 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.188 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.188 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.188 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:25.083926+0000", 2024-09-20T22:04:44.188 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046688629999999997 2024-09-20T22:04:44.188 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.188 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.188 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.188 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:25.084042+0000", 2024-09-20T22:04:44.188 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000116561 2024-09-20T22:04:44.188 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.188 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.189 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:04:44.189 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:25.084184+0000", 2024-09-20T22:04:44.189 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000141872 2024-09-20T22:04:44.189 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.189 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.189 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.189 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:25.085759+0000", 2024-09-20T22:04:44.189 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0015749329999999999 2024-09-20T22:04:44.189 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.189 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.189 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:44.189 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:25.085777+0000", 2024-09-20T22:04:44.189 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8457999999999998e-05 2024-09-20T22:04:44.189 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.190 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.190 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.190 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:25.102940+0000", 2024-09-20T22:04:44.190 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.017162754999999998 2024-09-20T22:04:44.190 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.190 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.190 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.190 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:25.102996+0000", 2024-09-20T22:04:44.190 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6297e-05 2024-09-20T22:04:44.190 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.190 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.190 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.190 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:25.103040+0000", 2024-09-20T22:04:44.190 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3299e-05 2024-09-20T22:04:44.191 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.191 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.191 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.191 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.191 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.191 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:82360 7.1 7:857124fa:::benchmark_data_smithi060_78800_object82359:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-20T22:04:44.191 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:43.964319+0000", 2024-09-20T22:04:44.191 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.090607208999999994, 2024-09-20T22:04:44.191 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.018494245999999999, 2024-09-20T22:04:44.191 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:44.191 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:44.191 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:44.191 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:44.191 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:44.191 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:44.192 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 82360 2024-09-20T22:04:44.192 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.192 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:44.192 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.192 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:44.192 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:43.964319+0000", 2024-09-20T22:04:44.192 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.192 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.192 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.192 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:44.192 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:43.964319+0000", 2024-09-20T22:04:44.192 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:44.192 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.192 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.192 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:44.193 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:43.964320+0000", 2024-09-20T22:04:44.193 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6599999999999995e-07 2024-09-20T22:04:44.193 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.193 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.193 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:44.193 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:43.964353+0000", 2024-09-20T22:04:44.193 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2360999999999997e-05 2024-09-20T22:04:44.193 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.193 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.193 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:44.193 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:43.964366+0000", 2024-09-20T22:04:44.193 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3174e-05 2024-09-20T22:04:44.193 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.193 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.193 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:44.194 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:43.964373+0000", 2024-09-20T22:04:44.194 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.029e-06 2024-09-20T22:04:44.194 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.194 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.194 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:44.194 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:43.968088+0000", 2024-09-20T22:04:44.194 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003715725 2024-09-20T22:04:44.194 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.194 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.194 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:44.194 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:43.968754+0000", 2024-09-20T22:04:44.194 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00066602000000000002 2024-09-20T22:04:44.194 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.194 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.195 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:04:44.195 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:43.968905+0000", 2024-09-20T22:04:44.195 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000150087 2024-09-20T22:04:44.195 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.195 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.195 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.195 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:43.974423+0000", 2024-09-20T22:04:44.195 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0055188709999999998 2024-09-20T22:04:44.195 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.195 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.195 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:44.195 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:43.978857+0000", 2024-09-20T22:04:44.195 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0044331250000000004 2024-09-20T22:04:44.195 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.195 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.196 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:44.196 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:43.982687+0000", 2024-09-20T22:04:44.196 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0038299610000000002 2024-09-20T22:04:44.196 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.196 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.196 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:44.196 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:43.982764+0000", 2024-09-20T22:04:44.196 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7589999999999994e-05 2024-09-20T22:04:44.196 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:44.196 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:44.196 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:44.196 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:43.982814+0000", 2024-09-20T22:04:44.196 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9536999999999997e-05 2024-09-20T22:04:44.196 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.196 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.197 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.197 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:44.197 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:44.197 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:04:44.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:44.198 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:44.199 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:44.286 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 47 15 82809 82794 110.059 83.9375 0.00910782 0.00908149 2024-09-20T22:04:44.299 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:44.344 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:44.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:44.345 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:44.445 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:44.486 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:44.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:44.487 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:44.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:44 smithi060 ceph-mon[29694]: pgmap v73: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 4.7 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 890 B/s rd, 78 MiB/s wr, 1.25k op/s; 29184/233874 objects degraded (12.479%); 4821/233874 objects misplaced (2.061%) 2024-09-20T22:04:44.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:44 smithi060 ceph-mon[40290]: pgmap v73: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 4.7 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 890 B/s rd, 78 MiB/s wr, 1.25k op/s; 29184/233874 objects degraded (12.479%); 4821/233874 objects misplaced (2.061%) 2024-09-20T22:04:44.588 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:44.619 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:44.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:44.620 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:44.721 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:44.730 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:04:44.731 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:04:44.752 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:44.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:44.753 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:44.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:44 smithi067 ceph-mon[33511]: pgmap v73: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 4.7 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 890 B/s rd, 78 MiB/s wr, 1.25k op/s; 29184/233874 objects degraded (12.479%); 4821/233874 objects misplaced (2.061%) 2024-09-20T22:04:44.854 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:44.901 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:44.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:44.902 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:45.002 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:45.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:45.039 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:45.039 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:45.140 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:45.170 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T22:04:45.188 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:45.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:45.189 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:45.288 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 48 15 84239 84224 109.628 89.375 0.00969449 0.00911669 2024-09-20T22:04:45.289 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:45.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:45.330 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:45.330 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:45.350 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:04:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:04:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:04:45.431 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:45.474 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:45.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:45.475 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:45.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:45 smithi067 ceph-mon[33511]: pgmap v74: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 4.7 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 760 B/s rd, 67 MiB/s wr, 1.07k op/s; 29184/233874 objects degraded (12.479%); 4821/233874 objects misplaced (2.061%) 2024-09-20T22:04:45.574 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:04:45.576 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:45.605 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:45 smithi060 ceph-mon[29694]: pgmap v74: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 4.7 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 760 B/s rd, 67 MiB/s wr, 1.07k op/s; 29184/233874 objects degraded (12.479%); 4821/233874 objects misplaced (2.061%) 2024-09-20T22:04:45.607 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:45 smithi060 ceph-mon[40290]: pgmap v74: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 4.7 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 760 B/s rd, 67 MiB/s wr, 1.07k op/s; 29184/233874 objects degraded (12.479%); 4821/233874 objects misplaced (2.061%) 2024-09-20T22:04:45.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:45.620 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:45.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:45.721 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:45.750 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:45.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:45.751 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:45.852 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:45.883 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:45.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:45.884 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:45.985 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:46.022 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:46.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:46.023 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:46.124 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:46.149 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:46.161 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:46.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:46.162 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:46.196 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:04:46.196 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:04:46.196 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:46.196 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:85500 7.0 7:0deef290:::benchmark_data_smithi060_78800_object85499:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e102)", 2024-09-20T22:04:46.196 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:46.191699+0000", 2024-09-20T22:04:46.196 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.002758178, 2024-09-20T22:04:46.196 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0027917020000000001, 2024-09-20T22:04:46.197 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:46.197 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:46.197 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:04:46.197 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:46.197 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:46.197 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:46.197 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 85500 2024-09-20T22:04:46.197 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:46.197 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:46.197 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:46.197 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:46.197 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:46.191699+0000", 2024-09-20T22:04:46.197 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:46.198 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:46.198 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:46.198 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:46.198 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:46.191699+0000", 2024-09-20T22:04:46.198 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:46.198 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:46.198 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:46.198 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:46.198 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:46.191700+0000", 2024-09-20T22:04:46.198 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3790000000000001e-06 2024-09-20T22:04:46.198 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:46.198 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:46.199 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:46.199 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:46.191756+0000", 2024-09-20T22:04:46.199 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.575e-05 2024-09-20T22:04:46.199 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:46.199 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:46.199 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:46.199 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:46.191768+0000", 2024-09-20T22:04:46.199 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2048999999999999e-05 2024-09-20T22:04:46.199 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:46.199 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:46.199 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:46.199 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:46.191777+0000", 2024-09-20T22:04:46.199 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4209999999999995e-06 2024-09-20T22:04:46.199 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:46.200 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:46.200 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:46.200 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:46.191839+0000", 2024-09-20T22:04:46.200 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1926000000000003e-05 2024-09-20T22:04:46.200 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:46.200 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:46.200 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:46.200 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:46.191935+0000", 2024-09-20T22:04:46.200 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6439000000000003e-05 2024-09-20T22:04:46.200 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:46.200 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:46.200 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:04:46.200 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:46.192058+0000", 2024-09-20T22:04:46.200 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012307099999999999 2024-09-20T22:04:46.200 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:46.201 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:46.201 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:46.201 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:46.194259+0000", 2024-09-20T22:04:46.201 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002201415 2024-09-20T22:04:46.201 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:46.201 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:46.201 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:46.201 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:46.201 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-20T22:04:46.201 INFO:teuthology.orchestra.run.smithi067.stdout: "num_ops": 1 2024-09-20T22:04:46.201 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:04:46.263 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:46.286 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 49 16 85639 85623 109.175 87.4375 0.00848257 0.00915404 2024-09-20T22:04:46.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:46.298 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:46.298 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:46.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:46.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:46.444 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:46.444 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:46.544 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:46.581 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:46.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:46.582 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:46.608 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:46 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 29184/233874 objects degraded (12.479%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:46.608 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:46 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 29184/233874 objects degraded (12.479%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:46.683 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:46.683 INFO:teuthology.orchestra.run.smithi060.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-20T22:04:46.714 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:46.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:46.716 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:46.732 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:04:46.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:46 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 29184/233874 objects degraded (12.479%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:46.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:46.861 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:46.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:46.862 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:46.936 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T22:04:46.962 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:47.020 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:47.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:47.021 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:47.121 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:47.158 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:47.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:47.159 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:47.260 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:47.286 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 50 16 86965 86949 108.649 82.875 0.0117899 0.00919912 2024-09-20T22:04:47.295 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:04:47 smithi060 systemd[1]: Stopping Ceph alertmanager.a for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:04:47.304 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:47.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:47.305 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:47.332 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:04:47.406 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:47.454 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:47.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:47.455 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:47.556 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:47.561 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:47 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:04:47.561 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:47 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:04:47.561 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:47 smithi060 ceph-mon[29694]: osdmap e103: 8 total, 7 up, 8 in 2024-09-20T22:04:47.561 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:47 smithi060 ceph-mon[29694]: pgmap v76: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.3 KiB/s rd, 114 MiB/s wr, 1.82k op/s; 31717/254544 objects degraded (12.460%); 5259/254544 objects misplaced (2.066%) 2024-09-20T22:04:47.561 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:47 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:04:47.562 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:47 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:04:47.562 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:47 smithi060 ceph-mon[40290]: osdmap e103: 8 total, 7 up, 8 in 2024-09-20T22:04:47.562 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:47 smithi060 ceph-mon[40290]: pgmap v76: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.3 KiB/s rd, 114 MiB/s wr, 1.82k op/s; 31717/254544 objects degraded (12.460%); 5259/254544 objects misplaced (2.066%) 2024-09-20T22:04:47.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:47.602 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:47.602 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:47.703 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:47.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:47.739 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:47.739 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:47.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:47 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:04:47.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:47 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:04:47.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:47 smithi067 ceph-mon[33511]: osdmap e103: 8 total, 7 up, 8 in 2024-09-20T22:04:47.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:47 smithi067 ceph-mon[33511]: pgmap v76: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.3 KiB/s rd, 114 MiB/s wr, 1.82k op/s; 31717/254544 objects degraded (12.460%); 5259/254544 objects misplaced (2.066%) 2024-09-20T22:04:47.840 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:47.883 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:47.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:47.884 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:47.985 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:48.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:48.021 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:48.021 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:48.122 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:48.151 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:48.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:48.152 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:48.253 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:48.286 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 51 16 88208 88192 108.042 77.6875 0.019805 0.00925061 2024-09-20T22:04:48.287 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:48.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:48.288 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:48.346 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:48.349 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:04:48.349 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:04:48.349 INFO:teuthology.orchestra.run.smithi067.stdout: "complaint_time": 30, 2024-09-20T22:04:48.349 INFO:teuthology.orchestra.run.smithi067.stdout: "num_blocked_ops": 0 2024-09-20T22:04:48.349 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:04:48.390 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:48.436 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:48.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:48.437 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:48.518 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:04:48.538 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:48.569 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:48.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:48.570 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:48.670 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:48.695 INFO:teuthology.orchestra.run.smithi060.stderr:Exporting 5.5 info 5.5( empty local-lis/les=67/68 n=0 ec=67/67 lis/c=67/67 les/c/f=68/68/0 sis=67) 2024-09-20T22:04:48.698 INFO:teuthology.orchestra.run.smithi060.stderr:Export successful 2024-09-20T22:04:48.699 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:48.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:48.700 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:48.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:48 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1067556784' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:04:48.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:48 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:04:48.755 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:48 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1067556784' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:04:48.755 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:48 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:04:48.801 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:48.830 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:48.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:48.831 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:48.932 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:48.979 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:48.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:48.980 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:49.003 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:48 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1067556784' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:04:49.003 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:48 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:04:49.035 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:04:48 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[76563]: ts=2024-09-20T22:04:48.750Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-20T22:04:49.046 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T22:04:49.081 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:49.129 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:49.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:49.131 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:49.231 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:49.275 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:49.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:49.276 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:49.286 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 52 16 89330 89314 107.313 70.125 0.0138959 0.00931317 2024-09-20T22:04:49.377 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:49.378 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:04:49.378 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:04:49.378 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:04:49.378 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:04:49.379 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:04:49.379 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:04:49.379 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:04:49.379 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:04:49.379 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:04:49.379 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:04:49.379 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:04:49.380 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:04:49.380 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:04:49.380 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:04:49.380 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:04:49.391 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:04:49.417 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:49.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:49.419 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:49.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:49.547 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:49.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:49.548 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:49.649 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:49.688 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:49.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:49.689 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:49.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:49 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:04:49.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:49 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:04:49.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:49 smithi060 ceph-mon[29694]: osdmap e104: 8 total, 7 up, 8 in 2024-09-20T22:04:49.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:49 smithi060 ceph-mon[29694]: pgmap v78: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 661 B/s rd, 56 MiB/s wr, 890 op/s; 31717/254544 objects degraded (12.460%); 5259/254544 objects misplaced (2.066%) 2024-09-20T22:04:49.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:49 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3423970289' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:04:49.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:49 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:04:49.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:49 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:04:49.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:49 smithi060 ceph-mon[40290]: osdmap e104: 8 total, 7 up, 8 in 2024-09-20T22:04:49.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:49 smithi060 ceph-mon[40290]: pgmap v78: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 661 B/s rd, 56 MiB/s wr, 890 op/s; 31717/254544 objects degraded (12.460%); 5259/254544 objects misplaced (2.066%) 2024-09-20T22:04:49.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:49 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3423970289' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:04:49.790 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:49.824 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:49.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:49.825 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:49.848 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:49 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:04:49.849 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:49 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:04:49.849 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:49 smithi067 ceph-mon[33511]: osdmap e104: 8 total, 7 up, 8 in 2024-09-20T22:04:49.849 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:49 smithi067 ceph-mon[33511]: pgmap v78: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 661 B/s rd, 56 MiB/s wr, 890 op/s; 31717/254544 objects degraded (12.460%); 5259/254544 objects misplaced (2.066%) 2024-09-20T22:04:49.849 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:49 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3423970289' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:04:49.926 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:49.973 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:49.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:49.974 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:50.075 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:50.112 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:50.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:50.113 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:50.214 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:50.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:50.255 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:50.256 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:50.287 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 53 16 90700 90684 106.903 85.625 0.0101853 0.00934933 2024-09-20T22:04:50.356 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:50.391 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:50.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:50.392 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:50.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:50.499 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:04:50.499 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:04:50.525 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:50.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:50.526 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:50.626 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:50.628 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:04:50.628 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:04:50.656 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:50.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:50.656 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:50.757 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:50.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:50 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:04:50.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:50 smithi067 ceph-mon[33511]: 4.15 scrub starts 2024-09-20T22:04:50.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:50 smithi067 ceph-mon[33511]: 4.15 scrub ok 2024-09-20T22:04:50.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:50.789 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:50.789 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:50.890 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:50.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:50.926 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:50.926 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:50.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:50 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:04:50.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:50 smithi060 ceph-mon[29694]: 4.15 scrub starts 2024-09-20T22:04:50.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:50 smithi060 ceph-mon[29694]: 4.15 scrub ok 2024-09-20T22:04:50.939 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:50 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:04:50.939 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:50 smithi060 ceph-mon[40290]: 4.15 scrub starts 2024-09-20T22:04:50.939 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:50 smithi060 ceph-mon[40290]: 4.15 scrub ok 2024-09-20T22:04:50.939 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:04:50 smithi060 podman[85487]: 2024-09-20 22:04:50.639234995 +0000 UTC m=+2.243594363 container died 02096472ad32bd54ce0b380bc824c4d2155f7808c499b46eeb5c81e47a9ec1cc (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T22:04:51.026 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:51.056 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:51.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:51.057 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:51.158 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:51.190 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:51.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:51.191 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:51.257 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:04:51 smithi060 podman[85487]: 2024-09-20 22:04:51.038066197 +0000 UTC m=+2.642425550 container cleanup 02096472ad32bd54ce0b380bc824c4d2155f7808c499b46eeb5c81e47a9ec1cc (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T22:04:51.257 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:04:51 smithi060 bash[85487]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a 2024-09-20T22:04:51.287 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 54 16 92080 92064 106.521 86.25 0.00695611 0.00938318 2024-09-20T22:04:51.292 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:51.312 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T22:04:51.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:51.334 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:51.334 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:51.434 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:51.467 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:51.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:51.468 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:51.536 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:04:51 smithi060 podman[85902]: 2024-09-20 22:04:51.2563754 +0000 UTC m=+2.441561926 container remove 02096472ad32bd54ce0b380bc824c4d2155f7808c499b46eeb5c81e47a9ec1cc (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T22:04:51.536 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:04:51 smithi060 podman[85902]: 2024-09-20 22:04:51.357812448 +0000 UTC m=+2.542998976 volume remove 32d35003a09f07deb6db4b8c69e9a6b41020761c4ada5673ab2a9fe598f0efc2 2024-09-20T22:04:51.569 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:51.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:51.606 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:51.607 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:51.705 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:04:51.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:51.745 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:51.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:51.746 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:51.835 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:04:51 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:04:51.835 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:51 smithi067 ceph-mon[33511]: pgmap v79: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 104 MiB/s wr, 1.67k op/s; 34132/273861 objects degraded (12.463%); 5639/273861 objects misplaced (2.059%) 2024-09-20T22:04:51.835 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:51 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 31717/254544 objects degraded (12.460%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:51.841 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:51 smithi060 ceph-mon[40290]: pgmap v79: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 104 MiB/s wr, 1.67k op/s; 34132/273861 objects degraded (12.463%); 5639/273861 objects misplaced (2.059%) 2024-09-20T22:04:51.841 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:51 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 31717/254544 objects degraded (12.460%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:51.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:51 smithi060 ceph-mon[29694]: pgmap v79: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 104 MiB/s wr, 1.67k op/s; 34132/273861 objects degraded (12.463%); 5639/273861 objects misplaced (2.059%) 2024-09-20T22:04:51.842 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:51 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 31717/254544 objects degraded (12.460%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:51.847 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:51.880 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:51.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:51.882 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:51.983 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:52.020 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:52.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:52.022 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:52.122 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:52.151 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:52.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:52.152 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:52.253 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:52.287 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 55 16 93465 93449 106.158 86.5625 0.0092261 0.00941472 2024-09-20T22:04:52.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:52.288 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:52.288 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:52.337 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:04:52.337 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:04:52.389 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:52.425 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:52.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:52.426 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:52.527 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:52.556 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:52.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:52.557 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:52.630 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:04:52.658 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:52.683 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:52.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:52.683 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:52.784 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:52 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:04:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:52 smithi060 ceph-mon[29694]: 4.16 scrub starts 2024-09-20T22:04:52.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:52 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:04:52.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:52 smithi060 ceph-mon[40290]: 4.16 scrub starts 2024-09-20T22:04:52.814 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:52.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:52.815 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:52.916 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:52.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:52.964 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:52.964 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:52 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:04:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:52 smithi067 ceph-mon[33511]: 4.16 scrub starts 2024-09-20T22:04:53.022 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -n osd.1 --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal --force --op remove --pgid 5.5 2024-09-20T22:04:53.065 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:53.077 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T22:04:53.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:53.143 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:53.144 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:53.149 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:53.244 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:53.287 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 56 16 94482 94466 105.397 63.5625 0.0160713 0.00948287 2024-09-20T22:04:53.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:53.300 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:53.301 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:53.401 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:53.416 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:04:53.440 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:53.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:53.441 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:53.464 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/osd.1/config 2024-09-20T22:04:53.541 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:53.574 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:53.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:53.575 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:53.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:53.707 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:53.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:53.708 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:53.809 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:53.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:53.839 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:53.839 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:53.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:53 smithi060 ceph-mon[29694]: 6.14 scrub starts 2024-09-20T22:04:53.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:53 smithi060 ceph-mon[29694]: 6.14 scrub ok 2024-09-20T22:04:53.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:53 smithi060 ceph-mon[29694]: 4.16 scrub ok 2024-09-20T22:04:53.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:53 smithi060 ceph-mon[29694]: 4.1a scrub starts 2024-09-20T22:04:53.861 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:53 smithi060 ceph-mon[29694]: 4.1a scrub ok 2024-09-20T22:04:53.861 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:53 smithi060 ceph-mon[29694]: pgmap v80: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 104 MiB/s wr, 1.67k op/s; 34132/273861 objects degraded (12.463%); 5639/273861 objects misplaced (2.059%) 2024-09-20T22:04:53.861 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:53 smithi060 ceph-mon[29694]: 3.1a scrub starts 2024-09-20T22:04:53.861 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:53 smithi060 ceph-mon[29694]: 3.1a scrub ok 2024-09-20T22:04:53.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:53 smithi060 ceph-mon[40290]: 6.14 scrub starts 2024-09-20T22:04:53.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:53 smithi060 ceph-mon[40290]: 6.14 scrub ok 2024-09-20T22:04:53.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:53 smithi060 ceph-mon[40290]: 4.16 scrub ok 2024-09-20T22:04:53.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:53 smithi060 ceph-mon[40290]: 4.1a scrub starts 2024-09-20T22:04:53.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:53 smithi060 ceph-mon[40290]: 4.1a scrub ok 2024-09-20T22:04:53.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:53 smithi060 ceph-mon[40290]: pgmap v80: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 104 MiB/s wr, 1.67k op/s; 34132/273861 objects degraded (12.463%); 5639/273861 objects misplaced (2.059%) 2024-09-20T22:04:53.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:53 smithi060 ceph-mon[40290]: 3.1a scrub starts 2024-09-20T22:04:53.862 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:53 smithi060 ceph-mon[40290]: 3.1a scrub ok 2024-09-20T22:04:53.902 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:53 smithi067 ceph-mon[33511]: 6.14 scrub starts 2024-09-20T22:04:53.903 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:53 smithi067 ceph-mon[33511]: 6.14 scrub ok 2024-09-20T22:04:53.903 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:53 smithi067 ceph-mon[33511]: 4.16 scrub ok 2024-09-20T22:04:53.903 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:53 smithi067 ceph-mon[33511]: 4.1a scrub starts 2024-09-20T22:04:53.903 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:53 smithi067 ceph-mon[33511]: 4.1a scrub ok 2024-09-20T22:04:53.903 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:53 smithi067 ceph-mon[33511]: pgmap v80: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 104 MiB/s wr, 1.67k op/s; 34132/273861 objects degraded (12.463%); 5639/273861 objects misplaced (2.059%) 2024-09-20T22:04:53.903 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:53 smithi067 ceph-mon[33511]: 3.1a scrub starts 2024-09-20T22:04:53.903 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:53 smithi067 ceph-mon[33511]: 3.1a scrub ok 2024-09-20T22:04:53.940 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:53.977 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:04:53.977 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:04:53.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:53.978 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:53.978 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:53.980 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:04:54.042 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:04:54.042 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:04:54.047 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:04:54.048 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:04:54.079 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:54.090 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:04:54.090 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:04:54.107 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:54.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:54.108 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:54.132 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:04:54.133 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:04:54.166 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:04:54.166 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:04:54.200 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:04:54.200 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:04:54.209 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:54.209 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:04:54.235 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:54.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:54.236 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:54.239 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:04:54.240 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:04:54.287 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 57 16 95524 95508 104.691 65.125 0.0210265 0.00954699 2024-09-20T22:04:54.337 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:54.382 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:54.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:54.383 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:54.484 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:54.501 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:04:54 smithi060 podman[86328]: 2024-09-20 22:04:54.305162099 +0000 UTC m=+0.427090326 container died 85f6e2a6285c5f62110278f6ef11ca50eea8bce641f4624439fc6ef11e0c9896 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736, name=pensive_rubin, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-9e2ac70, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-20T22:04:54.511 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:54.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:54.512 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:54.613 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:54.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:54.649 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:54.649 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:54.709 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:54 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3548888545' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:04:54.750 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:54.755 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T22:04:54.785 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:54.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:54.786 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:54.787 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:04:54 smithi060 systemd[1]: ceph-3df9647e-779b-11ef-baf5-efdc52797490@alertmanager.a.service: Deactivated successfully. 2024-09-20T22:04:54.787 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:04:54 smithi060 systemd[1]: Stopped Ceph alertmanager.a for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T22:04:54.787 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:04:54 smithi060 systemd[1]: ceph-3df9647e-779b-11ef-baf5-efdc52797490@alertmanager.a.service: Consumed 1.344s CPU time. 2024-09-20T22:04:54.787 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:04:54 smithi060 systemd[1]: Starting Ceph alertmanager.a for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:04:54.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:54 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3548888545' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:04:54.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:54 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3548888545' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:04:54.886 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:54.936 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:54.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:54.937 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:55.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:55.080 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:55.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:55.081 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:55.175 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:04:55.182 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:55.220 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:55.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:55.221 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:55.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:04:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:04:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:04:55.287 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 58 16 96796 96780 104.256 79.5 0.00664877 0.00958686 2024-09-20T22:04:55.322 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:55.354 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:55.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:55.355 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:55.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:55.494 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:55.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:55.495 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:55.596 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:55.623 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:55.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:55.624 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:55.724 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:55.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:55.764 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:55.765 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:55.789 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:04:55.790 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:04:55.815 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:04:55.865 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:55.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:55.911 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:55.912 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:55.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:55 smithi060 ceph-mon[29694]: 5.6 scrub starts 2024-09-20T22:04:55.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:55 smithi060 ceph-mon[29694]: 5.6 scrub ok 2024-09-20T22:04:55.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:55 smithi060 ceph-mon[29694]: 3.14 scrub starts 2024-09-20T22:04:55.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:55 smithi060 ceph-mon[29694]: 3.14 scrub ok 2024-09-20T22:04:55.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:55 smithi060 ceph-mon[29694]: pgmap v81: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 588 B/s rd, 46 MiB/s wr, 740 op/s; 34132/273861 objects degraded (12.463%); 5639/273861 objects misplaced (2.059%) 2024-09-20T22:04:55.934 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:55 smithi060 ceph-mon[40290]: 5.6 scrub starts 2024-09-20T22:04:55.934 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:55 smithi060 ceph-mon[40290]: 5.6 scrub ok 2024-09-20T22:04:55.934 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:55 smithi060 ceph-mon[40290]: 3.14 scrub starts 2024-09-20T22:04:55.934 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:55 smithi060 ceph-mon[40290]: 3.14 scrub ok 2024-09-20T22:04:55.934 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:55 smithi060 ceph-mon[40290]: pgmap v81: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 588 B/s rd, 46 MiB/s wr, 740 op/s; 34132/273861 objects degraded (12.463%); 5639/273861 objects misplaced (2.059%) 2024-09-20T22:04:56.012 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:55 smithi067 ceph-mon[33511]: 5.6 scrub starts 2024-09-20T22:04:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:55 smithi067 ceph-mon[33511]: 5.6 scrub ok 2024-09-20T22:04:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:55 smithi067 ceph-mon[33511]: 3.14 scrub starts 2024-09-20T22:04:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:55 smithi067 ceph-mon[33511]: 3.14 scrub ok 2024-09-20T22:04:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:55 smithi067 ceph-mon[33511]: pgmap v81: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 588 B/s rd, 46 MiB/s wr, 740 op/s; 34132/273861 objects degraded (12.463%); 5639/273861 objects misplaced (2.059%) 2024-09-20T22:04:56.040 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:56.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:56.041 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:56.141 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:56.177 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:56.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:56.177 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:56.278 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:56.287 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 59 16 98012 97996 103.777 76 0.00903991 0.00963118 2024-09-20T22:04:56.312 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:56.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:56.313 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:56.373 INFO:teuthology.orchestra.run.smithi060.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-20T22:04:56.414 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:56.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:56.465 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:56.565 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:56.597 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:56.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:56.599 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:56.699 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:56.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:56.743 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:56.743 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:56.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:56.876 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:56.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:56.877 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:56.889 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:56 smithi060 ceph-mon[29694]: 6.f scrub starts 2024-09-20T22:04:56.889 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:56 smithi060 ceph-mon[29694]: 6.f scrub ok 2024-09-20T22:04:56.889 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:56 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:04:56.890 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:56 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:04:56.890 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:56 smithi060 ceph-mon[29694]: 5.4 scrub starts 2024-09-20T22:04:56.890 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:56 smithi060 ceph-mon[29694]: 5.4 scrub ok 2024-09-20T22:04:56.890 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:56 smithi060 ceph-mon[29694]: osdmap e105: 8 total, 7 up, 8 in 2024-09-20T22:04:56.890 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:56 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 34132/273861 objects degraded (12.463%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:56.890 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:56 smithi060 ceph-mon[29694]: osdmap e106: 8 total, 7 up, 8 in 2024-09-20T22:04:56.890 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:56 smithi060 ceph-mon[40290]: 6.f scrub starts 2024-09-20T22:04:56.890 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:56 smithi060 ceph-mon[40290]: 6.f scrub ok 2024-09-20T22:04:56.890 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:56 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:04:56.890 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:56 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:04:56.890 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:56 smithi060 ceph-mon[40290]: 5.4 scrub starts 2024-09-20T22:04:56.890 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:56 smithi060 ceph-mon[40290]: 5.4 scrub ok 2024-09-20T22:04:56.891 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:56 smithi060 ceph-mon[40290]: osdmap e105: 8 total, 7 up, 8 in 2024-09-20T22:04:56.891 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:56 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 34132/273861 objects degraded (12.463%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:56.891 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:56 smithi060 ceph-mon[40290]: osdmap e106: 8 total, 7 up, 8 in 2024-09-20T22:04:56.978 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:57.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:57.014 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:57.014 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:57.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:56 smithi067 ceph-mon[33511]: 6.f scrub starts 2024-09-20T22:04:57.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:56 smithi067 ceph-mon[33511]: 6.f scrub ok 2024-09-20T22:04:57.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:56 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:04:57.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:56 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:04:57.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:56 smithi067 ceph-mon[33511]: 5.4 scrub starts 2024-09-20T22:04:57.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:56 smithi067 ceph-mon[33511]: 5.4 scrub ok 2024-09-20T22:04:57.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:56 smithi067 ceph-mon[33511]: osdmap e105: 8 total, 7 up, 8 in 2024-09-20T22:04:57.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:56 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 34132/273861 objects degraded (12.463%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:04:57.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:56 smithi067 ceph-mon[33511]: osdmap e106: 8 total, 7 up, 8 in 2024-09-20T22:04:57.114 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:57.145 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:57.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:57.146 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:57.247 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:57.286 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:57.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:57.287 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:57.288 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:04:57.287121+0000 min lat: 0.000984708 max lat: 1.08039 avg lat: 0.00968661 2024-09-20T22:04:57.288 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:04:57.288 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 60 16 99109 99093 103.19 68.5625 0.0123922 0.00968661 2024-09-20T22:04:57.388 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:57.425 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:57.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:57.426 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:57.526 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:57.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:57.561 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:57.562 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:57.662 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:57.681 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:04:57.703 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:57.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:57.704 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:57.791 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:04:57.805 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:04:57.832 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:57.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:57.833 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:04:57.934 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:57.986 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:57.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:57.987 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:57 smithi067 ceph-mon[33511]: 4.1e scrub starts 2024-09-20T22:04:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:57 smithi067 ceph-mon[33511]: 4.1e scrub ok 2024-09-20T22:04:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:57 smithi067 ceph-mon[33511]: 6.3 scrub starts 2024-09-20T22:04:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:57 smithi067 ceph-mon[33511]: 6.3 scrub ok 2024-09-20T22:04:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:57 smithi067 ceph-mon[33511]: pgmap v84: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 98 MiB/s wr, 1.57k op/s; 36416/292254 objects degraded (12.460%); 6014/292254 objects misplaced (2.058%) 2024-09-20T22:04:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:57 smithi067 ceph-mon[33511]: 3.c scrub starts 2024-09-20T22:04:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:57 smithi067 ceph-mon[33511]: 3.c scrub ok 2024-09-20T22:04:58.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:57 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2789467611' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:04:58.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:57 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:04:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:57 smithi060 ceph-mon[29694]: 4.1e scrub starts 2024-09-20T22:04:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:57 smithi060 ceph-mon[29694]: 4.1e scrub ok 2024-09-20T22:04:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:57 smithi060 ceph-mon[29694]: 6.3 scrub starts 2024-09-20T22:04:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:57 smithi060 ceph-mon[29694]: 6.3 scrub ok 2024-09-20T22:04:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:57 smithi060 ceph-mon[29694]: pgmap v84: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 98 MiB/s wr, 1.57k op/s; 36416/292254 objects degraded (12.460%); 6014/292254 objects misplaced (2.058%) 2024-09-20T22:04:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:57 smithi060 ceph-mon[29694]: 3.c scrub starts 2024-09-20T22:04:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:57 smithi060 ceph-mon[29694]: 3.c scrub ok 2024-09-20T22:04:58.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:57 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2789467611' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:04:58.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:57 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:04:58.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:57 smithi060 ceph-mon[40290]: 4.1e scrub starts 2024-09-20T22:04:58.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:57 smithi060 ceph-mon[40290]: 4.1e scrub ok 2024-09-20T22:04:58.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:57 smithi060 ceph-mon[40290]: 6.3 scrub starts 2024-09-20T22:04:58.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:57 smithi060 ceph-mon[40290]: 6.3 scrub ok 2024-09-20T22:04:58.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:57 smithi060 ceph-mon[40290]: pgmap v84: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 98 MiB/s wr, 1.57k op/s; 36416/292254 objects degraded (12.460%); 6014/292254 objects misplaced (2.058%) 2024-09-20T22:04:58.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:57 smithi060 ceph-mon[40290]: 3.c scrub starts 2024-09-20T22:04:58.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:57 smithi060 ceph-mon[40290]: 3.c scrub ok 2024-09-20T22:04:58.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:57 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2789467611' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:04:58.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:57 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:04:58.088 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:58.138 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:58.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:58.139 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:58.240 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:58.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:58.281 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:58.281 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:58.287 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 61 16 100093 100077 102.507 61.5 0.0109207 0.00975055 2024-09-20T22:04:58.307 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:04:58.307 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:04:58.307 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:04:58.307 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:04:58.307 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.307 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:22515 7.7 7:e2a23e22:::benchmark_data_smithi060_78800_object22514:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:04:58.307 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:06.081179+0000", 2024-09-20T22:04:58.307 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 51.194829882999997, 2024-09-20T22:04:58.308 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.088628218999999994, 2024-09-20T22:04:58.308 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.308 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.308 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.308 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.308 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.308 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.308 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 22515 2024-09-20T22:04:58.308 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.308 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.308 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.308 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.309 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081179+0000", 2024-09-20T22:04:58.309 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.309 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.309 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.309 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.309 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081179+0000", 2024-09-20T22:04:58.309 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.309 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.309 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.309 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.309 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081179+0000", 2024-09-20T22:04:58.309 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7300000000000002e-07 2024-09-20T22:04:58.309 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.310 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.310 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.310 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081204+0000", 2024-09-20T22:04:58.310 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.4944e-05 2024-09-20T22:04:58.310 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.310 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.310 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.310 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081211+0000", 2024-09-20T22:04:58.310 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6220000000000003e-06 2024-09-20T22:04:58.310 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.310 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.311 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.311 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081214+0000", 2024-09-20T22:04:58.311 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.557e-06 2024-09-20T22:04:58.311 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.311 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.311 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.311 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.082794+0000", 2024-09-20T22:04:58.311 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001579347 2024-09-20T22:04:58.311 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.311 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.311 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.311 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.082939+0000", 2024-09-20T22:04:58.311 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014563900000000001 2024-09-20T22:04:58.312 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.312 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.312 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:04:58.312 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083090+0000", 2024-09-20T22:04:58.312 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000150879 2024-09-20T22:04:58.312 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.312 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.312 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.312 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085586+0000", 2024-09-20T22:04:58.312 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002496042 2024-09-20T22:04:58.312 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.312 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.313 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.313 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.086685+0000", 2024-09-20T22:04:58.313 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001098903 2024-09-20T22:04:58.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.313 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.313 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.313 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.169690+0000", 2024-09-20T22:04:58.313 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.083004665000000005 2024-09-20T22:04:58.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.313 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.313 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.313 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.169761+0000", 2024-09-20T22:04:58.314 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1145999999999999e-05 2024-09-20T22:04:58.314 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.314 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.314 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.314 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.169807+0000", 2024-09-20T22:04:58.314 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6102e-05 2024-09-20T22:04:58.314 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.314 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.314 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.315 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.315 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.315 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:22520 7.7 7:e34001c7:::benchmark_data_smithi060_78800_object22519:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:04:58.315 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:06.081333+0000", 2024-09-20T22:04:58.315 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 51.194676078999997, 2024-09-20T22:04:58.315 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.088627362000000001, 2024-09-20T22:04:58.315 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.315 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.315 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.315 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.316 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.316 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.316 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 22520 2024-09-20T22:04:58.316 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.316 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.316 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.316 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.316 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081333+0000", 2024-09-20T22:04:58.316 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.316 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.316 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.316 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.317 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081333+0000", 2024-09-20T22:04:58.317 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.317 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.317 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.317 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081345+0000", 2024-09-20T22:04:58.317 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2258e-05 2024-09-20T22:04:58.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.317 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.317 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.317 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081565+0000", 2024-09-20T22:04:58.317 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00022027599999999999 2024-09-20T22:04:58.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.318 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.318 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.318 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081577+0000", 2024-09-20T22:04:58.318 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1364e-05 2024-09-20T22:04:58.318 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.318 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.318 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.318 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081596+0000", 2024-09-20T22:04:58.318 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9026999999999999e-05 2024-09-20T22:04:58.318 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.318 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.319 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.319 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083719+0000", 2024-09-20T22:04:58.319 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0021235690000000001 2024-09-20T22:04:58.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.319 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.319 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.319 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083865+0000", 2024-09-20T22:04:58.319 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014620599999999999 2024-09-20T22:04:58.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.319 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.319 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:04:58.319 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.084003+0000", 2024-09-20T22:04:58.320 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013766399999999999 2024-09-20T22:04:58.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.320 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.320 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.320 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085594+0000", 2024-09-20T22:04:58.320 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0015914270000000001 2024-09-20T22:04:58.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.320 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.320 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.320 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.088221+0000", 2024-09-20T22:04:58.320 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0026267370000000001 2024-09-20T22:04:58.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.321 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.321 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.321 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.169859+0000", 2024-09-20T22:04:58.321 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.081638302999999995 2024-09-20T22:04:58.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.321 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.321 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.321 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.169919+0000", 2024-09-20T22:04:58.321 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9468000000000001e-05 2024-09-20T22:04:58.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.321 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.321 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.322 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.169960+0000", 2024-09-20T22:04:58.322 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1063000000000003e-05 2024-09-20T22:04:58.322 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.322 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.322 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.322 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.322 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:22524 7.7 7:e83bec1e:::benchmark_data_smithi060_78800_object22523:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:04:58.322 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:06.083249+0000", 2024-09-20T22:04:58.322 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 51.192759840000001, 2024-09-20T22:04:58.322 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.086867280000000005, 2024-09-20T22:04:58.323 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.323 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.323 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.323 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.323 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.323 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.323 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 22524 2024-09-20T22:04:58.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.323 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.323 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.323 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.323 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083249+0000", 2024-09-20T22:04:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083249+0000", 2024-09-20T22:04:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083250+0000", 2024-09-20T22:04:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0240000000000001e-06 2024-09-20T22:04:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.324 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083308+0000", 2024-09-20T22:04:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8545000000000003e-05 2024-09-20T22:04:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083354+0000", 2024-09-20T22:04:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5658000000000002e-05 2024-09-20T22:04:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.325 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083361+0000", 2024-09-20T22:04:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0720000000000002e-06 2024-09-20T22:04:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.084644+0000", 2024-09-20T22:04:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0012832760000000001 2024-09-20T22:04:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.084739+0000", 2024-09-20T22:04:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4093999999999997e-05 2024-09-20T22:04:58.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:04:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.084836+0000", 2024-09-20T22:04:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7829000000000001e-05 2024-09-20T22:04:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085599+0000", 2024-09-20T22:04:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00076252199999999996 2024-09-20T22:04:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.327 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.088259+0000", 2024-09-20T22:04:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002660188 2024-09-20T22:04:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.170015+0000", 2024-09-20T22:04:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.081755456000000004 2024-09-20T22:04:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.170076+0000", 2024-09-20T22:04:58.328 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1807e-05 2024-09-20T22:04:58.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.329 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.329 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.329 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.170116+0000", 2024-09-20T22:04:58.329 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9808999999999998e-05 2024-09-20T22:04:58.329 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.329 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.329 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.329 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.330 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:22988 7.7 7:e91c34c6:::benchmark_data_smithi060_78800_object22987:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:04:58.330 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:06.370688+0000", 2024-09-20T22:04:58.330 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 50.905320949999997, 2024-09-20T22:04:58.330 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.071871357999999996, 2024-09-20T22:04:58.330 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.330 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.330 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.330 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.330 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.330 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 22988 2024-09-20T22:04:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.370688+0000", 2024-09-20T22:04:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.370688+0000", 2024-09-20T22:04:58.331 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.370689+0000", 2024-09-20T22:04:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.034e-06 2024-09-20T22:04:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.370762+0000", 2024-09-20T22:04:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2788999999999997e-05 2024-09-20T22:04:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.332 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.370771+0000", 2024-09-20T22:04:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8770000000000005e-06 2024-09-20T22:04:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.370777+0000", 2024-09-20T22:04:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0799999999999996e-06 2024-09-20T22:04:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.333 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.370829+0000", 2024-09-20T22:04:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2478000000000001e-05 2024-09-20T22:04:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.370926+0000", 2024-09-20T22:04:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7289999999999999e-05 2024-09-20T22:04:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:04:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.371050+0000", 2024-09-20T22:04:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012365999999999999 2024-09-20T22:04:58.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.371883+0000", 2024-09-20T22:04:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00083318400000000005 2024-09-20T22:04:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.372308+0000", 2024-09-20T22:04:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00042470099999999998 2024-09-20T22:04:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.335 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.442456+0000", 2024-09-20T22:04:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.070148370000000002 2024-09-20T22:04:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.442511+0000", 2024-09-20T22:04:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4917000000000003e-05 2024-09-20T22:04:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.442559+0000", 2024-09-20T22:04:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7978e-05 2024-09-20T22:04:58.336 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:23758 7.7 7:e43f7c59:::benchmark_data_smithi060_78800_object23757:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:04:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:06.790194+0000", 2024-09-20T22:04:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 50.485814298999998, 2024-09-20T22:04:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.074592989999999998, 2024-09-20T22:04:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.337 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 23758 2024-09-20T22:04:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.790194+0000", 2024-09-20T22:04:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.338 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.790194+0000", 2024-09-20T22:04:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.790195+0000", 2024-09-20T22:04:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2999999999999999e-07 2024-09-20T22:04:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.339 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.790268+0000", 2024-09-20T22:04:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2372000000000004e-05 2024-09-20T22:04:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.790281+0000", 2024-09-20T22:04:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3413e-05 2024-09-20T22:04:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.790290+0000", 2024-09-20T22:04:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.0250000000000008e-06 2024-09-20T22:04:58.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.790344+0000", 2024-09-20T22:04:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4373999999999998e-05 2024-09-20T22:04:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.790444+0000", 2024-09-20T22:04:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9523000000000002e-05 2024-09-20T22:04:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.341 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:04:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.790527+0000", 2024-09-20T22:04:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2979000000000006e-05 2024-09-20T22:04:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.791307+0000", 2024-09-20T22:04:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000780311 2024-09-20T22:04:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.864007+0000", 2024-09-20T22:04:58.342 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.072699814000000001 2024-09-20T22:04:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.864677+0000", 2024-09-20T22:04:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00067036199999999998 2024-09-20T22:04:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.864742+0000", 2024-09-20T22:04:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4376999999999997e-05 2024-09-20T22:04:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.864787+0000", 2024-09-20T22:04:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5510000000000003e-05 2024-09-20T22:04:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:32200 7.7 7:e661f05a:::benchmark_data_smithi060_78800_object32199:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-20T22:04:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:11.071733+0000", 2024-09-20T22:04:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 46.204275613, 2024-09-20T22:04:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.078344860000000002, 2024-09-20T22:04:58.344 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.345 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.345 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.345 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.345 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.345 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.345 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 32200 2024-09-20T22:04:58.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.345 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.345 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.345 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.345 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.071733+0000", 2024-09-20T22:04:58.345 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.346 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.346 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.071733+0000", 2024-09-20T22:04:58.346 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.346 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.346 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.071734+0000", 2024-09-20T22:04:58.346 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5700000000000002e-07 2024-09-20T22:04:58.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.347 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.347 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.071816+0000", 2024-09-20T22:04:58.347 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2111e-05 2024-09-20T22:04:58.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.347 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.347 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.071826+0000", 2024-09-20T22:04:58.347 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.978e-06 2024-09-20T22:04:58.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.347 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.347 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.071832+0000", 2024-09-20T22:04:58.348 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6110000000000001e-06 2024-09-20T22:04:58.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.348 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.348 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.071875+0000", 2024-09-20T22:04:58.348 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3784e-05 2024-09-20T22:04:58.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.348 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.348 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.071959+0000", 2024-09-20T22:04:58.348 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3623999999999998e-05 2024-09-20T22:04:58.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.349 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:04:58.349 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.072032+0000", 2024-09-20T22:04:58.349 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3553000000000005e-05 2024-09-20T22:04:58.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.349 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.349 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.073485+0000", 2024-09-20T22:04:58.349 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001453006 2024-09-20T22:04:58.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.349 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.349 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.077796+0000", 2024-09-20T22:04:58.349 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.004310258 2024-09-20T22:04:58.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.350 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.350 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.149986+0000", 2024-09-20T22:04:58.350 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.072190693 2024-09-20T22:04:58.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.350 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.350 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.150038+0000", 2024-09-20T22:04:58.350 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1208000000000002e-05 2024-09-20T22:04:58.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.351 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.351 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.150078+0000", 2024-09-20T22:04:58.351 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0277000000000001e-05 2024-09-20T22:04:58.351 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.351 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.351 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.351 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:32205 7.d 7:b8477c90:::benchmark_data_smithi060_78800_object32204:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-20T22:04:58.351 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:11.074949+0000", 2024-09-20T22:04:58.351 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 46.201059807, 2024-09-20T22:04:58.351 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.074949680000000005, 2024-09-20T22:04:58.351 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.352 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.352 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.352 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.352 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.352 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.352 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 32205 2024-09-20T22:04:58.352 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.352 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.352 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.352 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.352 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.074949+0000", 2024-09-20T22:04:58.352 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.353 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.353 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.353 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.074949+0000", 2024-09-20T22:04:58.353 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.353 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.353 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.353 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.074951+0000", 2024-09-20T22:04:58.353 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0789999999999999e-06 2024-09-20T22:04:58.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.353 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.353 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.354 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075006+0000", 2024-09-20T22:04:58.354 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4608000000000003e-05 2024-09-20T22:04:58.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.354 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.354 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.354 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075015+0000", 2024-09-20T22:04:58.354 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6649999999999998e-06 2024-09-20T22:04:58.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.354 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.354 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.354 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075020+0000", 2024-09-20T22:04:58.355 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9729999999999997e-06 2024-09-20T22:04:58.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.355 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.355 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075091+0000", 2024-09-20T22:04:58.355 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1218999999999997e-05 2024-09-20T22:04:58.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.355 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.355 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075222+0000", 2024-09-20T22:04:58.355 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013064399999999999 2024-09-20T22:04:58.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.356 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1", 2024-09-20T22:04:58.356 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075363+0000", 2024-09-20T22:04:58.356 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014131300000000001 2024-09-20T22:04:58.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.356 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.356 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.356 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.076629+0000", 2024-09-20T22:04:58.356 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0012658910000000001 2024-09-20T22:04:58.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.356 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.356 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.356 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.078895+0000", 2024-09-20T22:04:58.357 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0022661040000000001 2024-09-20T22:04:58.357 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.357 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.357 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.357 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.149797+0000", 2024-09-20T22:04:58.357 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.070901986 2024-09-20T22:04:58.357 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.357 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.357 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.357 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.149852+0000", 2024-09-20T22:04:58.357 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4185999999999999e-05 2024-09-20T22:04:58.357 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.357 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.358 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.358 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.149899+0000", 2024-09-20T22:04:58.358 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7012000000000003e-05 2024-09-20T22:04:58.358 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.358 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.358 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.358 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.358 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.358 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:32206 7.7 7:ed08e6b2:::benchmark_data_smithi060_78800_object32205:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-20T22:04:58.358 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:11.075810+0000", 2024-09-20T22:04:58.358 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 46.200198188000002, 2024-09-20T22:04:58.358 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.074396689000000002, 2024-09-20T22:04:58.359 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.359 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.359 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.359 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.359 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.359 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.359 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 32206 2024-09-20T22:04:58.359 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.359 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.359 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.359 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.359 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075810+0000", 2024-09-20T22:04:58.360 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.360 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.360 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.360 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.360 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075810+0000", 2024-09-20T22:04:58.360 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.360 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.360 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.360 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.360 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075812+0000", 2024-09-20T22:04:58.360 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.06e-06 2024-09-20T22:04:58.360 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.360 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.361 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.361 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075902+0000", 2024-09-20T22:04:58.361 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.0187999999999998e-05 2024-09-20T22:04:58.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.361 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.361 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.361 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075914+0000", 2024-09-20T22:04:58.361 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2632999999999999e-05 2024-09-20T22:04:58.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.361 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.361 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.361 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075922+0000", 2024-09-20T22:04:58.362 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7549999999999994e-06 2024-09-20T22:04:58.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.362 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.362 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075990+0000", 2024-09-20T22:04:58.362 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7447999999999995e-05 2024-09-20T22:04:58.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.362 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.362 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.076160+0000", 2024-09-20T22:04:58.362 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017066 2024-09-20T22:04:58.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:04:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.076318+0000", 2024-09-20T22:04:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015779499999999999 2024-09-20T22:04:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.077603+0000", 2024-09-20T22:04:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001285217 2024-09-20T22:04:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.363 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.149874+0000", 2024-09-20T22:04:58.364 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.072270757000000005 2024-09-20T22:04:58.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.364 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.364 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.150119+0000", 2024-09-20T22:04:58.364 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00024487800000000002 2024-09-20T22:04:58.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.364 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.364 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.150171+0000", 2024-09-20T22:04:58.364 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2203999999999997e-05 2024-09-20T22:04:58.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.365 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.365 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.365 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.150207+0000", 2024-09-20T22:04:58.365 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6093999999999997e-05 2024-09-20T22:04:58.365 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.365 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.365 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.365 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.365 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.365 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:32211 7.7 7:e6b36260:::benchmark_data_smithi060_78800_object32210:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-20T22:04:58.365 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:11.079364+0000", 2024-09-20T22:04:58.365 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 46.196644368999998, 2024-09-20T22:04:58.366 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.070974791999999995, 2024-09-20T22:04:58.366 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.366 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.366 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.366 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.366 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.366 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.366 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 32211 2024-09-20T22:04:58.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.366 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.366 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.366 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.366 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.079364+0000", 2024-09-20T22:04:58.367 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.367 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.367 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.367 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.079364+0000", 2024-09-20T22:04:58.367 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.367 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.367 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.367 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.079366+0000", 2024-09-20T22:04:58.367 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.654e-06 2024-09-20T22:04:58.368 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.368 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.368 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.368 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.079421+0000", 2024-09-20T22:04:58.368 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4833999999999997e-05 2024-09-20T22:04:58.368 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.368 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.368 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.368 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.079432+0000", 2024-09-20T22:04:58.368 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1202e-05 2024-09-20T22:04:58.368 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.368 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.369 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.369 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.079439+0000", 2024-09-20T22:04:58.369 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2840000000000001e-06 2024-09-20T22:04:58.369 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.369 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.369 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.369 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.079499+0000", 2024-09-20T22:04:58.369 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9388999999999997e-05 2024-09-20T22:04:58.369 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.369 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.369 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.369 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.079611+0000", 2024-09-20T22:04:58.370 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000112087 2024-09-20T22:04:58.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.370 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.370 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:04:58.370 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.079697+0000", 2024-09-20T22:04:58.370 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5964000000000006e-05 2024-09-20T22:04:58.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.370 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.370 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.370 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.081647+0000", 2024-09-20T22:04:58.370 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001950043 2024-09-20T22:04:58.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.371 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.371 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.371 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.149914+0000", 2024-09-20T22:04:58.371 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.068267204999999997 2024-09-20T22:04:58.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.371 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.371 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.371 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.150243+0000", 2024-09-20T22:04:58.371 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00032944000000000002 2024-09-20T22:04:58.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.371 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.371 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.372 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.150291+0000", 2024-09-20T22:04:58.372 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7564000000000001e-05 2024-09-20T22:04:58.372 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.372 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.372 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.372 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.150339+0000", 2024-09-20T22:04:58.372 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8126e-05 2024-09-20T22:04:58.372 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.372 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.372 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.372 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.372 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.373 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:32213 7.7 7:e774f69b:::benchmark_data_smithi060_78800_object32212:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-20T22:04:58.373 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:11.079462+0000", 2024-09-20T22:04:58.373 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 46.196546834000003, 2024-09-20T22:04:58.373 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.070996766000000003, 2024-09-20T22:04:58.373 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.373 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.373 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.373 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.373 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.373 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.373 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 32213 2024-09-20T22:04:58.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.374 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.374 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.374 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.374 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.079462+0000", 2024-09-20T22:04:58.374 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.374 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.374 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.374 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.079462+0000", 2024-09-20T22:04:58.374 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.374 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.375 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.375 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.079462+0000", 2024-09-20T22:04:58.375 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2699999999999999e-07 2024-09-20T22:04:58.375 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.375 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.375 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.375 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.079493+0000", 2024-09-20T22:04:58.375 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0488999999999999e-05 2024-09-20T22:04:58.375 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.375 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.375 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.376 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.079507+0000", 2024-09-20T22:04:58.376 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.438e-05 2024-09-20T22:04:58.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.376 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.376 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.376 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.079514+0000", 2024-09-20T22:04:58.376 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.635e-06 2024-09-20T22:04:58.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.376 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.376 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.376 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.080569+0000", 2024-09-20T22:04:58.376 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001054585 2024-09-20T22:04:58.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.377 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.377 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.377 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.080685+0000", 2024-09-20T22:04:58.377 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011648 2024-09-20T22:04:58.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.377 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.377 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:04:58.377 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.080818+0000", 2024-09-20T22:04:58.377 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000133125 2024-09-20T22:04:58.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.378 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.378 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.081811+0000", 2024-09-20T22:04:58.378 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00099250800000000002 2024-09-20T22:04:58.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.378 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.378 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.149951+0000", 2024-09-20T22:04:58.378 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.068140318000000005 2024-09-20T22:04:58.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.378 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.379 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.150373+0000", 2024-09-20T22:04:58.379 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00042166399999999998 2024-09-20T22:04:58.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.379 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.379 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.379 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.150427+0000", 2024-09-20T22:04:58.379 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3956999999999999e-05 2024-09-20T22:04:58.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.379 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.379 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.379 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.150459+0000", 2024-09-20T22:04:58.379 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1998000000000001e-05 2024-09-20T22:04:58.380 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.380 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.380 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.380 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:46036 7.7 7:e5f37aca:::benchmark_data_smithi060_78800_object46035:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T22:04:58.380 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:19.394201+0000", 2024-09-20T22:04:58.380 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 37.881808149999998, 2024-09-20T22:04:58.380 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.079244819999999994, 2024-09-20T22:04:58.380 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.380 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.381 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.381 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.381 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.381 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.381 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 46036 2024-09-20T22:04:58.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.381 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.381 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.381 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.381 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394201+0000", 2024-09-20T22:04:58.381 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.381 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.382 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.382 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394201+0000", 2024-09-20T22:04:58.382 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.382 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.382 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.382 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394202+0000", 2024-09-20T22:04:58.382 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.344e-06 2024-09-20T22:04:58.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.382 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.382 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.383 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394268+0000", 2024-09-20T22:04:58.383 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6047e-05 2024-09-20T22:04:58.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.383 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.383 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.383 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394282+0000", 2024-09-20T22:04:58.383 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3967e-05 2024-09-20T22:04:58.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.383 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.383 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.383 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394288+0000", 2024-09-20T22:04:58.383 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3199999999999996e-06 2024-09-20T22:04:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394356+0000", 2024-09-20T22:04:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7529999999999999e-05 2024-09-20T22:04:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.384 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394470+0000", 2024-09-20T22:04:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000113857 2024-09-20T22:04:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:04:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394552+0000", 2024-09-20T22:04:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2128999999999999e-05 2024-09-20T22:04:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.395833+0000", 2024-09-20T22:04:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0012817639999999999 2024-09-20T22:04:58.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.471500+0000", 2024-09-20T22:04:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.075666232 2024-09-20T22:04:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.473346+0000", 2024-09-20T22:04:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00184637 2024-09-20T22:04:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.386 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.473405+0000", 2024-09-20T22:04:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9298e-05 2024-09-20T22:04:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.473445+0000", 2024-09-20T22:04:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9962000000000001e-05 2024-09-20T22:04:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.387 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:46037 7.d 7:b4603dba:::benchmark_data_smithi060_78800_object46036:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T22:04:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:19.394327+0000", 2024-09-20T22:04:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 37.881682075000001, 2024-09-20T22:04:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.078329847999999994, 2024-09-20T22:04:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.388 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 46037 2024-09-20T22:04:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394327+0000", 2024-09-20T22:04:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394327+0000", 2024-09-20T22:04:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394327+0000", 2024-09-20T22:04:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8700000000000005e-07 2024-09-20T22:04:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394406+0000", 2024-09-20T22:04:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8265999999999999e-05 2024-09-20T22:04:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.390 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394413+0000", 2024-09-20T22:04:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9890000000000003e-06 2024-09-20T22:04:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394417+0000", 2024-09-20T22:04:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4329999999999997e-06 2024-09-20T22:04:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394475+0000", 2024-09-20T22:04:58.391 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7627000000000002e-05 2024-09-20T22:04:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394605+0000", 2024-09-20T22:04:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013037199999999999 2024-09-20T22:04:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0", 2024-09-20T22:04:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394762+0000", 2024-09-20T22:04:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015654300000000001 2024-09-20T22:04:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.392 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.395999+0000", 2024-09-20T22:04:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001237036 2024-09-20T22:04:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.472495+0000", 2024-09-20T22:04:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.076496185999999994 2024-09-20T22:04:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.393 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.472558+0000", 2024-09-20T22:04:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2849999999999996e-05 2024-09-20T22:04:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.472656+0000", 2024-09-20T22:04:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8858999999999997e-05 2024-09-20T22:04:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.394 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:98048 7.7 7:e68db34e:::benchmark_data_smithi060_78800_object98047:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-20T22:04:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.311849+0000", 2024-09-20T22:04:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.96415996400000004, 2024-09-20T22:04:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.082878767000000006, 2024-09-20T22:04:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.395 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 98048 2024-09-20T22:04:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.311849+0000", 2024-09-20T22:04:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.311849+0000", 2024-09-20T22:04:58.396 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.311850+0000", 2024-09-20T22:04:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5e-06 2024-09-20T22:04:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.311878+0000", 2024-09-20T22:04:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7756999999999999e-05 2024-09-20T22:04:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.397 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.311891+0000", 2024-09-20T22:04:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3322000000000001e-05 2024-09-20T22:04:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.311899+0000", 2024-09-20T22:04:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5340000000000002e-06 2024-09-20T22:04:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.398 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.312832+0000", 2024-09-20T22:04:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00093272699999999995 2024-09-20T22:04:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.312960+0000", 2024-09-20T22:04:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012859699999999999 2024-09-20T22:04:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:04:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.313105+0000", 2024-09-20T22:04:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014509700000000001 2024-09-20T22:04:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.399 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.316108+0000", 2024-09-20T22:04:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0030024449999999999 2024-09-20T22:04:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.394570+0000", 2024-09-20T22:04:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.078462660000000004 2024-09-20T22:04:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.394622+0000", 2024-09-20T22:04:58.400 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1662000000000001e-05 2024-09-20T22:04:58.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.401 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.401 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.401 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.394679+0000", 2024-09-20T22:04:58.401 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.664e-05 2024-09-20T22:04:58.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.401 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.401 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.401 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.394727+0000", 2024-09-20T22:04:58.401 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8826000000000003e-05 2024-09-20T22:04:58.401 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.401 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.401 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.402 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.402 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.402 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:98055 7.c 7:30ecbb09:::benchmark_data_smithi060_78800_object98054:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-20T22:04:58.402 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.318853+0000", 2024-09-20T22:04:58.402 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.95715553900000006, 2024-09-20T22:04:58.402 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.090772817000000006, 2024-09-20T22:04:58.402 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.402 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.402 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.402 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.402 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.402 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.403 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 98055 2024-09-20T22:04:58.403 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.403 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.403 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.403 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.403 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.318853+0000", 2024-09-20T22:04:58.403 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.403 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.403 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.403 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.403 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.318853+0000", 2024-09-20T22:04:58.403 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.404 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.404 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.404 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.404 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.318854+0000", 2024-09-20T22:04:58.404 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5900000000000005e-07 2024-09-20T22:04:58.404 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.404 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.404 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.404 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.318908+0000", 2024-09-20T22:04:58.404 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3850000000000001e-05 2024-09-20T22:04:58.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.405 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.405 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.405 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.318919+0000", 2024-09-20T22:04:58.405 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0708e-05 2024-09-20T22:04:58.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.405 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.405 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.405 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.318925+0000", 2024-09-20T22:04:58.405 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0820000000000004e-06 2024-09-20T22:04:58.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.406 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.406 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.406 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.318970+0000", 2024-09-20T22:04:58.406 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5626e-05 2024-09-20T22:04:58.406 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.406 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.406 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.406 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319091+0000", 2024-09-20T22:04:58.406 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000120569 2024-09-20T22:04:58.406 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.406 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.407 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:04:58.407 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319226+0000", 2024-09-20T22:04:58.407 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000134578 2024-09-20T22:04:58.407 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.407 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.407 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.407 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.397781+0000", 2024-09-20T22:04:58.407 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.078555884000000006 2024-09-20T22:04:58.407 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.407 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.407 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.407 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.398333+0000", 2024-09-20T22:04:58.407 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00055192399999999997 2024-09-20T22:04:58.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.408 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.408 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.409527+0000", 2024-09-20T22:04:58.408 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.011193318000000001 2024-09-20T22:04:58.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.408 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.408 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.409580+0000", 2024-09-20T22:04:58.408 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3573999999999999e-05 2024-09-20T22:04:58.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.409 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.409 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.409626+0000", 2024-09-20T22:04:58.409 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5745000000000003e-05 2024-09-20T22:04:58.409 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.409 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.409 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.409 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:98057 7.d 7:b86bb00f:::benchmark_data_smithi060_78800_object98056:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-20T22:04:58.409 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.319112+0000", 2024-09-20T22:04:58.409 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.95689683400000003, 2024-09-20T22:04:58.409 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.089241666999999997, 2024-09-20T22:04:58.409 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.410 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.410 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.410 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.410 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.410 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.410 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 98057 2024-09-20T22:04:58.410 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.410 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.410 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.410 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.410 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319112+0000", 2024-09-20T22:04:58.410 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.411 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.411 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.411 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319112+0000", 2024-09-20T22:04:58.411 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.411 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.411 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.411 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319113+0000", 2024-09-20T22:04:58.411 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1399999999999995e-07 2024-09-20T22:04:58.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.411 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.411 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.412 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319151+0000", 2024-09-20T22:04:58.412 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7808999999999997e-05 2024-09-20T22:04:58.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.412 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.412 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.412 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319159+0000", 2024-09-20T22:04:58.412 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.7739999999999993e-06 2024-09-20T22:04:58.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.412 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.412 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.412 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319164+0000", 2024-09-20T22:04:58.412 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8890000000000001e-06 2024-09-20T22:04:58.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.413 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.413 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.413 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319227+0000", 2024-09-20T22:04:58.413 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2630000000000002e-05 2024-09-20T22:04:58.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.413 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.413 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.413 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.394852+0000", 2024-09-20T22:04:58.413 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.075624780000000003 2024-09-20T22:04:58.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.413 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.413 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0", 2024-09-20T22:04:58.414 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.395003+0000", 2024-09-20T22:04:58.414 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000151745 2024-09-20T22:04:58.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.414 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.414 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.414 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.407269+0000", 2024-09-20T22:04:58.414 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.012265507 2024-09-20T22:04:58.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.414 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.414 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.414 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.408228+0000", 2024-09-20T22:04:58.414 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000958952 2024-09-20T22:04:58.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.415 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.415 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.415 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.408280+0000", 2024-09-20T22:04:58.415 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2077000000000003e-05 2024-09-20T22:04:58.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.415 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.415 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.415 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.408354+0000", 2024-09-20T22:04:58.415 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3590000000000005e-05 2024-09-20T22:04:58.416 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.416 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.416 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.416 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.416 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:98058 7.7 7:e4cc42e2:::benchmark_data_smithi060_78800_object98057:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-20T22:04:58.416 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.319192+0000", 2024-09-20T22:04:58.416 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.95681697700000001, 2024-09-20T22:04:58.416 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.093554769999999995, 2024-09-20T22:04:58.416 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.416 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.416 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.417 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.417 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.417 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.417 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 98058 2024-09-20T22:04:58.417 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.417 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.417 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.417 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.417 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319192+0000", 2024-09-20T22:04:58.417 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.417 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.417 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.417 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.418 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319192+0000", 2024-09-20T22:04:58.418 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.418 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.418 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.418 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319192+0000", 2024-09-20T22:04:58.418 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6800000000000001e-07 2024-09-20T22:04:58.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.418 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.418 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.418 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319261+0000", 2024-09-20T22:04:58.419 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8670999999999994e-05 2024-09-20T22:04:58.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.419 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.419 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.419 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.394159+0000", 2024-09-20T22:04:58.419 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.074898163000000004 2024-09-20T22:04:58.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.419 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.419 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.419 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.394172+0000", 2024-09-20T22:04:58.419 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.312e-05 2024-09-20T22:04:58.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.419 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.420 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.420 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.394758+0000", 2024-09-20T22:04:58.420 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00058551700000000003 2024-09-20T22:04:58.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.420 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.420 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.420 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.394898+0000", 2024-09-20T22:04:58.420 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000140044 2024-09-20T22:04:58.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.420 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.420 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:04:58.420 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.395042+0000", 2024-09-20T22:04:58.421 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000144418 2024-09-20T22:04:58.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.421 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.421 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.398913+0000", 2024-09-20T22:04:58.421 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003870975 2024-09-20T22:04:58.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.421 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.421 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.406960+0000", 2024-09-20T22:04:58.421 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0080465499999999995 2024-09-20T22:04:58.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.422 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.422 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.412648+0000", 2024-09-20T22:04:58.422 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0056887320000000002 2024-09-20T22:04:58.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.422 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.422 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.412701+0000", 2024-09-20T22:04:58.422 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2253000000000003e-05 2024-09-20T22:04:58.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.423 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.423 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.412746+0000", 2024-09-20T22:04:58.423 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5859000000000003e-05 2024-09-20T22:04:58.423 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.423 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.423 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.423 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.423 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:98517 7.d 7:bd9e5a0c:::benchmark_data_smithi060_78800_object98516:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-20T22:04:58.423 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.751855+0000", 2024-09-20T22:04:58.423 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.52415326200000001, 2024-09-20T22:04:58.424 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.081104983000000005, 2024-09-20T22:04:58.424 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.424 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.424 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.424 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.424 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.424 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.424 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 98517 2024-09-20T22:04:58.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.424 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.424 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.424 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.425 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.751855+0000", 2024-09-20T22:04:58.425 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.425 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.425 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.751855+0000", 2024-09-20T22:04:58.425 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.425 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.425 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.751857+0000", 2024-09-20T22:04:58.425 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3510000000000001e-06 2024-09-20T22:04:58.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.426 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.426 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.426 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.751910+0000", 2024-09-20T22:04:58.426 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3056999999999997e-05 2024-09-20T22:04:58.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.426 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.426 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.426 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.751918+0000", 2024-09-20T22:04:58.426 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9319999999999996e-06 2024-09-20T22:04:58.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.426 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.427 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.427 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.751922+0000", 2024-09-20T22:04:58.427 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5859999999999998e-06 2024-09-20T22:04:58.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.427 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.427 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.427 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.751962+0000", 2024-09-20T22:04:58.427 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9771000000000003e-05 2024-09-20T22:04:58.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.427 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.427 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.427 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752044+0000", 2024-09-20T22:04:58.428 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1988000000000002e-05 2024-09-20T22:04:58.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.428 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.428 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0", 2024-09-20T22:04:58.428 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752136+0000", 2024-09-20T22:04:58.428 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1710000000000001e-05 2024-09-20T22:04:58.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.428 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.428 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.428 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.831045+0000", 2024-09-20T22:04:58.428 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.078909149999999997 2024-09-20T22:04:58.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.429 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.429 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.429 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.832819+0000", 2024-09-20T22:04:58.429 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001773791 2024-09-20T22:04:58.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.429 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.429 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.429 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.832873+0000", 2024-09-20T22:04:58.429 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4725999999999997e-05 2024-09-20T22:04:58.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.429 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.429 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.429 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.832960+0000", 2024-09-20T22:04:58.430 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6921000000000003e-05 2024-09-20T22:04:58.430 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.430 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.430 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.430 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.430 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:98518 7.7 7:e03a258a:::benchmark_data_smithi060_78800_object98517:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-20T22:04:58.430 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.752091+0000", 2024-09-20T22:04:58.430 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.52391791499999996, 2024-09-20T22:04:58.430 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.092212638999999999, 2024-09-20T22:04:58.430 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.430 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.430 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.431 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.431 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.431 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.431 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 98518 2024-09-20T22:04:58.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.431 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.431 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.431 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752091+0000", 2024-09-20T22:04:58.431 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752091+0000", 2024-09-20T22:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752091+0000", 2024-09-20T22:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2499999999999995e-07 2024-09-20T22:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752154+0000", 2024-09-20T22:04:58.432 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2951000000000001e-05 2024-09-20T22:04:58.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.433 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.433 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.433 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:58.434 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752161+0000", 2024-09-20T22:04:58.434 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4880000000000004e-06 2024-09-20T22:04:58.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.435 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.435 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752165+0000", 2024-09-20T22:04:58.435 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9230000000000004e-06 2024-09-20T22:04:58.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.435 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.435 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.435 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752212+0000", 2024-09-20T22:04:58.435 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6875000000000001e-05 2024-09-20T22:04:58.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.435 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.435 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.435 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.827647+0000", 2024-09-20T22:04:58.436 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.075435385999999993 2024-09-20T22:04:58.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.436 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:04:58.436 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.827791+0000", 2024-09-20T22:04:58.436 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014404199999999999 2024-09-20T22:04:58.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.436 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.436 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.831115+0000", 2024-09-20T22:04:58.436 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0033237679999999999 2024-09-20T22:04:58.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.437 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.437 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.831125+0000", 2024-09-20T22:04:58.437 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0281e-05 2024-09-20T22:04:58.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.437 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.437 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.437 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.844201+0000", 2024-09-20T22:04:58.437 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.01307559 2024-09-20T22:04:58.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.437 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.437 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.437 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.844259+0000", 2024-09-20T22:04:58.438 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8661e-05 2024-09-20T22:04:58.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.438 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.438 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.438 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.844303+0000", 2024-09-20T22:04:58.438 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4149e-05 2024-09-20T22:04:58.438 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.438 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.438 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.439 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:98519 7.c 7:32ce32d4:::benchmark_data_smithi060_78800_object98518:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-20T22:04:58.439 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.752356+0000", 2024-09-20T22:04:58.439 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.52365304599999996, 2024-09-20T22:04:58.439 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.089684246999999995, 2024-09-20T22:04:58.439 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.439 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.439 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.439 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:04:58.439 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:04:58.439 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:04:58.439 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 98519 2024-09-20T22:04:58.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.440 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.440 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.440 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752356+0000", 2024-09-20T22:04:58.440 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.440 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.440 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752356+0000", 2024-09-20T22:04:58.440 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.441 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.441 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.441 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752356+0000", 2024-09-20T22:04:58.441 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5000000000000003e-07 2024-09-20T22:04:58.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.441 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.441 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.441 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752430+0000", 2024-09-20T22:04:58.441 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4054999999999995e-05 2024-09-20T22:04:58.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.441 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.442 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.442 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.827861+0000", 2024-09-20T22:04:58.442 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.075431127000000001 2024-09-20T22:04:58.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.442 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.442 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.827875+0000", 2024-09-20T22:04:58.442 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3183999999999999e-05 2024-09-20T22:04:58.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.442 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.442 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.827939+0000", 2024-09-20T22:04:58.443 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4628999999999994e-05 2024-09-20T22:04:58.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.443 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.443 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.828085+0000", 2024-09-20T22:04:58.443 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014621799999999999 2024-09-20T22:04:58.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.443 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:04:58.443 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.828222+0000", 2024-09-20T22:04:58.444 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000136379 2024-09-20T22:04:58.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.444 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.444 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.831909+0000", 2024-09-20T22:04:58.444 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0036872900000000002 2024-09-20T22:04:58.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.444 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:04:58.444 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.832037+0000", 2024-09-20T22:04:58.444 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012829399999999999 2024-09-20T22:04:58.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.445 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:04:58.445 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.841930+0000", 2024-09-20T22:04:58.445 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0098926190000000001 2024-09-20T22:04:58.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.445 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.445 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.841991+0000", 2024-09-20T22:04:58.445 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.101e-05 2024-09-20T22:04:58.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.445 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.446 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.842040+0000", 2024-09-20T22:04:58.446 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8891999999999999e-05 2024-09-20T22:04:58.446 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.446 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.446 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.446 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.24667.0:98529 7.b e106/91 7:d59e89dd:::benchmark_data_smithi060_78800_object98528:head v 106'6110, mlcod=106'6108)", 2024-09-20T22:04:58.446 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.760619+0000", 2024-09-20T22:04:58.446 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.51538947499999999, 2024-09-20T22:04:58.446 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.076810604000000005, 2024-09-20T22:04:58.446 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:04:58.447 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:04:58.447 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:04:58.447 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:04:58.447 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.447 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:04:58.447 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760619+0000", 2024-09-20T22:04:58.447 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.447 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.447 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:04:58.447 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760619+0000", 2024-09-20T22:04:58.447 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:04:58.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.448 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:04:58.448 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760620+0000", 2024-09-20T22:04:58.448 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.018e-06 2024-09-20T22:04:58.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.448 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:04:58.448 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760642+0000", 2024-09-20T22:04:58.448 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1852e-05 2024-09-20T22:04:58.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.448 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:04:58.449 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.830718+0000", 2024-09-20T22:04:58.449 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.070075838000000001 2024-09-20T22:04:58.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.449 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:04:58.449 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.830727+0000", 2024-09-20T22:04:58.449 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3070000000000008e-06 2024-09-20T22:04:58.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.449 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:04:58.449 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.830799+0000", 2024-09-20T22:04:58.449 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1675999999999996e-05 2024-09-20T22:04:58.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:04:58.450 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.830821+0000", 2024-09-20T22:04:58.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2291000000000001e-05 2024-09-20T22:04:58.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:04:58.450 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.837103+0000", 2024-09-20T22:04:58.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0062819729999999997 2024-09-20T22:04:58.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:04:58.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:04:58.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:04:58.451 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.837430+0000", 2024-09-20T22:04:58.451 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00032664899999999998 2024-09-20T22:04:58.451 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.451 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.451 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.451 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:04:58.451 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:04:58.451 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:04:58.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:58.464 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:58.464 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:58.565 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:58.569 INFO:teuthology.orchestra.run.smithi060.stdout: marking collection for removal 2024-09-20T22:04:58.569 INFO:teuthology.orchestra.run.smithi060.stdout:setting '_remove' omap key 2024-09-20T22:04:58.569 INFO:teuthology.orchestra.run.smithi060.stdout:finish_remove_pgs 5.5_head removing 5.5 2024-09-20T22:04:58.569 INFO:teuthology.orchestra.run.smithi060.stdout:Remove successful 2024-09-20T22:04:58.605 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:58.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:58.606 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:04:58.706 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:58.736 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:58.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:58.737 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:58.837 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:04:58.881 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:58.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:58.882 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:04:58.971 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:58 smithi060 ceph-mon[29694]: 7.8 scrub starts 2024-09-20T22:04:58.971 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:58 smithi060 ceph-mon[29694]: 5.10 scrub starts 2024-09-20T22:04:58.971 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:58 smithi060 ceph-mon[29694]: 5.10 scrub ok 2024-09-20T22:04:58.971 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:58 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:04:58.971 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:58 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:04:58.971 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:58 smithi060 ceph-mon[29694]: osdmap e107: 8 total, 7 up, 8 in 2024-09-20T22:04:58.971 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:58 smithi060 ceph-mon[29694]: 4.0 scrub starts 2024-09-20T22:04:58.971 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:58 smithi060 ceph-mon[29694]: 4.0 scrub ok 2024-09-20T22:04:58.971 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:58 smithi060 ceph-mon[40290]: 7.8 scrub starts 2024-09-20T22:04:58.971 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:58 smithi060 ceph-mon[40290]: 5.10 scrub starts 2024-09-20T22:04:58.972 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:58 smithi060 ceph-mon[40290]: 5.10 scrub ok 2024-09-20T22:04:58.972 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:58 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:04:58.972 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:58 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:04:58.972 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:58 smithi060 ceph-mon[40290]: osdmap e107: 8 total, 7 up, 8 in 2024-09-20T22:04:58.972 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:58 smithi060 ceph-mon[40290]: 4.0 scrub starts 2024-09-20T22:04:58.972 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:58 smithi060 ceph-mon[40290]: 4.0 scrub ok 2024-09-20T22:04:58.983 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:58 smithi067 ceph-mon[33511]: 7.8 scrub starts 2024-09-20T22:04:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:58 smithi067 ceph-mon[33511]: 5.10 scrub starts 2024-09-20T22:04:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:58 smithi067 ceph-mon[33511]: 5.10 scrub ok 2024-09-20T22:04:59.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:58 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:04:59.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:58 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:04:59.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:58 smithi067 ceph-mon[33511]: osdmap e107: 8 total, 7 up, 8 in 2024-09-20T22:04:59.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:58 smithi067 ceph-mon[33511]: 4.0 scrub starts 2024-09-20T22:04:59.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:58 smithi067 ceph-mon[33511]: 4.0 scrub ok 2024-09-20T22:04:59.015 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:59.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:59.016 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:59.117 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:59.144 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:59.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:59.145 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:59.246 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:59.267 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:59.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:59.268 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:59.287 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 62 16 101159 101143 101.928 66.625 0.0806923 0.00980114 2024-09-20T22:04:59.369 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:04:59.403 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:59.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:59.404 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:04:59.505 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:04:59.528 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:59.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:59.529 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:04:59.630 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:04:59.661 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:59.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:59.662 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:04:59.763 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:04:59.798 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:04:59.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:59.799 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:04:59.899 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:04:59.939 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:04:59.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:04:59.940 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:00.012 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:59 smithi060 ceph-mon[29694]: pgmap v86: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 852 B/s rd, 64 MiB/s wr, 1.02k op/s; 36416/292254 objects degraded (12.460%); 6014/292254 objects misplaced (2.058%) 2024-09-20T22:05:00.012 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:59 smithi060 ceph-mon[29694]: 3.10 scrub starts 2024-09-20T22:05:00.012 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:04:59 smithi060 ceph-mon[29694]: 3.10 scrub ok 2024-09-20T22:05:00.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:59 smithi067 ceph-mon[33511]: pgmap v86: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 852 B/s rd, 64 MiB/s wr, 1.02k op/s; 36416/292254 objects degraded (12.460%); 6014/292254 objects misplaced (2.058%) 2024-09-20T22:05:00.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:59 smithi067 ceph-mon[33511]: 3.10 scrub starts 2024-09-20T22:05:00.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:04:59 smithi067 ceph-mon[33511]: 3.10 scrub ok 2024-09-20T22:05:00.013 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:59 smithi060 ceph-mon[40290]: pgmap v86: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 5.9 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 852 B/s rd, 64 MiB/s wr, 1.02k op/s; 36416/292254 objects degraded (12.460%); 6014/292254 objects misplaced (2.058%) 2024-09-20T22:05:00.013 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:59 smithi060 ceph-mon[40290]: 3.10 scrub starts 2024-09-20T22:05:00.014 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:04:59 smithi060 ceph-mon[40290]: 3.10 scrub ok 2024-09-20T22:05:00.041 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:00.063 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -n osd.1 --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-1 --journal-path /var/lib/ceph/osd/ceph-1/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.5.5.1 2024-09-20T22:05:00.070 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:00.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:00.071 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:00.172 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:00.201 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:00.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:00.202 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:00.287 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 63 16 102374 102358 101.516 75.9375 0.0122747 0.00984577 2024-09-20T22:05:00.303 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:00.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:00.347 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:00.347 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:00.415 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/osd.1/config 2024-09-20T22:05:00.448 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:00.478 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:00.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:00.479 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:00.579 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:00.605 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:00.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:00.606 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:00.610 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:00.707 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:00.742 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:00.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:00.743 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:00 smithi060 ceph-mon[29694]: 7.8 scrub ok 2024-09-20T22:05:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:00 smithi060 ceph-mon[29694]: 6.8 scrub starts 2024-09-20T22:05:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:00 smithi060 ceph-mon[29694]: 6.8 scrub ok 2024-09-20T22:05:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:00 smithi060 ceph-mon[29694]: 6.a scrub starts 2024-09-20T22:05:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:00 smithi060 ceph-mon[29694]: 6.a scrub ok 2024-09-20T22:05:00.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:00 smithi060 ceph-mon[40290]: 7.8 scrub ok 2024-09-20T22:05:00.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:00 smithi060 ceph-mon[40290]: 6.8 scrub starts 2024-09-20T22:05:00.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:00 smithi060 ceph-mon[40290]: 6.8 scrub ok 2024-09-20T22:05:00.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:00 smithi060 ceph-mon[40290]: 6.a scrub starts 2024-09-20T22:05:00.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:00 smithi060 ceph-mon[40290]: 6.a scrub ok 2024-09-20T22:05:00.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:00.876 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:00.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:00.877 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:00.977 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:01.007 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:01.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:01.008 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:00 smithi067 ceph-mon[33511]: 7.8 scrub ok 2024-09-20T22:05:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:00 smithi067 ceph-mon[33511]: 6.8 scrub starts 2024-09-20T22:05:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:00 smithi067 ceph-mon[33511]: 6.8 scrub ok 2024-09-20T22:05:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:00 smithi067 ceph-mon[33511]: 6.a scrub starts 2024-09-20T22:05:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:00 smithi067 ceph-mon[33511]: 6.a scrub ok 2024-09-20T22:05:01.109 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:01.143 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:01.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:01.146 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:01.150 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T22:05:01.247 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:01.288 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 64 16 103644 103628 101.17 79.375 0.00466986 0.00987967 2024-09-20T22:05:01.292 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:01.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:01.293 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:01.394 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:01.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:01.437 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:01.437 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:01.538 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:01.563 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:05:01.567 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:05:01.568 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:05:01.574 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:05:01.578 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:01.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:01.579 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:01.610 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:05:01.610 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:05:01.648 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:05:01.648 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:05:01.680 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:01.683 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:05:01.683 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:05:01.711 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:01.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:01.712 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:01.716 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:05:01.717 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:05:01.751 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:05:01.751 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:05:01.751 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[29694]: 6.1e scrub starts 2024-09-20T22:05:01.752 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[29694]: 6.1e scrub ok 2024-09-20T22:05:01.752 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[29694]: 3.f scrub starts 2024-09-20T22:05:01.752 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[29694]: 3.f scrub ok 2024-09-20T22:05:01.752 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[29694]: pgmap v87: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 121 MiB/s wr, 1.94k op/s; 38432/308844 objects degraded (12.444%); 6343/308844 objects misplaced (2.054%) 2024-09-20T22:05:01.752 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 36416/292254 objects degraded (12.460%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:05:01.752 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[29694]: osdmap e108: 8 total, 7 up, 8 in 2024-09-20T22:05:01.752 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[29694]: 5.1f scrub starts 2024-09-20T22:05:01.752 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[29694]: 5.1f scrub ok 2024-09-20T22:05:01.752 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[29694]: 4.8 scrub starts 2024-09-20T22:05:01.752 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[29694]: 4.8 scrub ok 2024-09-20T22:05:01.752 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2561235702' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:01.753 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[40290]: 6.1e scrub starts 2024-09-20T22:05:01.753 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[40290]: 6.1e scrub ok 2024-09-20T22:05:01.753 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[40290]: 3.f scrub starts 2024-09-20T22:05:01.753 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[40290]: 3.f scrub ok 2024-09-20T22:05:01.753 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[40290]: pgmap v87: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 121 MiB/s wr, 1.94k op/s; 38432/308844 objects degraded (12.444%); 6343/308844 objects misplaced (2.054%) 2024-09-20T22:05:01.753 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 36416/292254 objects degraded (12.460%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:05:01.753 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[40290]: osdmap e108: 8 total, 7 up, 8 in 2024-09-20T22:05:01.753 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[40290]: 5.1f scrub starts 2024-09-20T22:05:01.753 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[40290]: 5.1f scrub ok 2024-09-20T22:05:01.753 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[40290]: 4.8 scrub starts 2024-09-20T22:05:01.753 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[40290]: 4.8 scrub ok 2024-09-20T22:05:01.753 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:01 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2561235702' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:01.790 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:05:01.790 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:05:01.812 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:01.851 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:01.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:01.852 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:01.953 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:01.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:05:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:05:01.987 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:01 smithi067 ceph-mon[33511]: 6.1e scrub starts 2024-09-20T22:05:01.987 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:01 smithi067 ceph-mon[33511]: 6.1e scrub ok 2024-09-20T22:05:01.988 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:01 smithi067 ceph-mon[33511]: 3.f scrub starts 2024-09-20T22:05:01.988 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:01 smithi067 ceph-mon[33511]: 3.f scrub ok 2024-09-20T22:05:01.988 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:01 smithi067 ceph-mon[33511]: pgmap v87: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 121 MiB/s wr, 1.94k op/s; 38432/308844 objects degraded (12.444%); 6343/308844 objects misplaced (2.054%) 2024-09-20T22:05:01.988 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:01 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 36416/292254 objects degraded (12.460%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:05:01.988 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:01 smithi067 ceph-mon[33511]: osdmap e108: 8 total, 7 up, 8 in 2024-09-20T22:05:01.988 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:01 smithi067 ceph-mon[33511]: 5.1f scrub starts 2024-09-20T22:05:01.988 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:01 smithi067 ceph-mon[33511]: 5.1f scrub ok 2024-09-20T22:05:01.988 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:01 smithi067 ceph-mon[33511]: 4.8 scrub starts 2024-09-20T22:05:01.988 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:01 smithi067 ceph-mon[33511]: 4.8 scrub ok 2024-09-20T22:05:01.988 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:01 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2561235702' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:01.992 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:01.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:01.993 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:02.094 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:02.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:02.131 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:02.132 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:02.146 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:05:02.147 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:05:02.232 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:02.263 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:02.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:02.264 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:02.288 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 65 16 104911 104895 100.832 79.1875 0.00734198 0.00991282 2024-09-20T22:05:02.364 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:02.399 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:02.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:02.400 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:02.501 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:02.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:02.533 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:02.533 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:02.535 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:05:02 smithi060 podman[87526]: 2024-09-20 22:05:02.1705813 +0000 UTC m=+1.577993074 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-20T22:05:02.588 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:05:02.634 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:02.696 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:02.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:02.697 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:02.704 INFO:teuthology.orchestra.run.smithi060.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-20T22:05:02.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:02 smithi067 ceph-mon[33511]: 6.17 scrub starts 2024-09-20T22:05:02.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:02 smithi067 ceph-mon[33511]: 6.17 scrub ok 2024-09-20T22:05:02.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:02 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:05:02.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:02 smithi067 ceph-mon[33511]: 7.c scrub starts 2024-09-20T22:05:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:02 smithi067 ceph-mon[33511]: 5.1b scrub starts 2024-09-20T22:05:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:02 smithi067 ceph-mon[33511]: 5.1b scrub ok 2024-09-20T22:05:02.798 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:02.839 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T22:05:02.841 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:02.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:02.843 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:02.944 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:02.964 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:02 smithi060 ceph-mon[40290]: 6.17 scrub starts 2024-09-20T22:05:02.964 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:02 smithi060 ceph-mon[40290]: 6.17 scrub ok 2024-09-20T22:05:02.964 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:02 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:05:02.964 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:02 smithi060 ceph-mon[40290]: 7.c scrub starts 2024-09-20T22:05:02.964 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:02 smithi060 ceph-mon[40290]: 5.1b scrub starts 2024-09-20T22:05:02.964 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:02 smithi060 ceph-mon[40290]: 5.1b scrub ok 2024-09-20T22:05:02.965 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:05:02 smithi060 podman[87526]: 2024-09-20 22:05:02.716300181 +0000 UTC m=+2.123711949 volume create b4f84d023d10072378df1d05ab06b8345e94b0707f83df6bcdec453eb47b66e0 2024-09-20T22:05:02.965 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:05:02 smithi060 podman[87526]: 2024-09-20 22:05:02.858850709 +0000 UTC m=+2.266262479 container create f4474f3f5bac57806d0cea5d47af51fdc93a4ff3202a3ce4ec1225ab09a02a7b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T22:05:02.969 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:02 smithi060 ceph-mon[29694]: 6.17 scrub starts 2024-09-20T22:05:02.969 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:02 smithi060 ceph-mon[29694]: 6.17 scrub ok 2024-09-20T22:05:02.969 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:02 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:05:02.969 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:02 smithi060 ceph-mon[29694]: 7.c scrub starts 2024-09-20T22:05:02.969 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:02 smithi060 ceph-mon[29694]: 5.1b scrub starts 2024-09-20T22:05:02.969 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:02 smithi060 ceph-mon[29694]: 5.1b scrub ok 2024-09-20T22:05:02.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:02.981 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:02.981 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:03.082 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:03.131 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:03.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:03.132 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:03.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:03.236 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:05:03.273 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:03.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:03.274 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:03.288 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 66 16 106105 106089 100.434 74.625 0.0081132 0.009952 2024-09-20T22:05:03.375 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:03.413 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:03.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:03.414 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:03.515 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:03.553 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:03.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:03.555 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:03.656 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:03.700 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:03.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:03.701 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:05:03.702 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:03.702 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:05:03.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:03 smithi060 ceph-mon[40290]: 1.0 scrub starts 2024-09-20T22:05:03.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:03 smithi060 ceph-mon[40290]: 1.0 scrub ok 2024-09-20T22:05:03.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:03 smithi060 ceph-mon[40290]: 3.7 scrub starts 2024-09-20T22:05:03.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:03 smithi060 ceph-mon[40290]: 3.7 scrub ok 2024-09-20T22:05:03.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:03 smithi060 ceph-mon[40290]: pgmap v89: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 758 B/s rd, 51 MiB/s wr, 819 op/s; 38432/308844 objects degraded (12.444%); 6343/308844 objects misplaced (2.054%) 2024-09-20T22:05:03.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:03 smithi060 ceph-mon[29694]: 1.0 scrub starts 2024-09-20T22:05:03.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:03 smithi060 ceph-mon[29694]: 1.0 scrub ok 2024-09-20T22:05:03.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:03 smithi060 ceph-mon[29694]: 3.7 scrub starts 2024-09-20T22:05:03.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:03 smithi060 ceph-mon[29694]: 3.7 scrub ok 2024-09-20T22:05:03.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:03 smithi060 ceph-mon[29694]: pgmap v89: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 758 B/s rd, 51 MiB/s wr, 819 op/s; 38432/308844 objects degraded (12.444%); 6343/308844 objects misplaced (2.054%) 2024-09-20T22:05:03.802 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:03.839 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:03.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:03.840 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:03.941 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:03.979 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:03.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:03.981 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:03.997 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:03 smithi067 ceph-mon[33511]: 1.0 scrub starts 2024-09-20T22:05:03.997 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:03 smithi067 ceph-mon[33511]: 1.0 scrub ok 2024-09-20T22:05:03.997 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:03 smithi067 ceph-mon[33511]: 3.7 scrub starts 2024-09-20T22:05:03.997 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:03 smithi067 ceph-mon[33511]: 3.7 scrub ok 2024-09-20T22:05:03.997 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:03 smithi067 ceph-mon[33511]: pgmap v89: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 758 B/s rd, 51 MiB/s wr, 819 op/s; 38432/308844 objects degraded (12.444%); 6343/308844 objects misplaced (2.054%) 2024-09-20T22:05:04.049 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:05:04.050 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:05:04.081 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:04.122 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:04.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:04.124 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:04.224 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:04.254 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:04.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:04.255 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:04.288 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 67 16 107369 107353 100.115 79 0.00680205 0.00998358 2024-09-20T22:05:04.356 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:04.367 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:05:04 smithi060 podman[87526]: 2024-09-20 22:05:04.121124938 +0000 UTC m=+3.528536712 container init f4474f3f5bac57806d0cea5d47af51fdc93a4ff3202a3ce4ec1225ab09a02a7b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T22:05:04.367 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:05:04 smithi060 podman[87526]: 2024-09-20 22:05:04.133924911 +0000 UTC m=+3.541336681 container start f4474f3f5bac57806d0cea5d47af51fdc93a4ff3202a3ce4ec1225ab09a02a7b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T22:05:04.367 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:05:04 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[87854]: ts=2024-09-20T22:05:04.161Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-20T22:05:04.367 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:05:04 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[87854]: ts=2024-09-20T22:05:04.161Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-20T22:05:04.367 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:05:04 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[87854]: ts=2024-09-20T22:05:04.162Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.60 port=9094 2024-09-20T22:05:04.367 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:05:04 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[87854]: ts=2024-09-20T22:05:04.163Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-20T22:05:04.367 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:05:04 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[87854]: ts=2024-09-20T22:05:04.215Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-20T22:05:04.367 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:05:04 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[87854]: ts=2024-09-20T22:05:04.216Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-20T22:05:04.367 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:05:04 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[87854]: ts=2024-09-20T22:05:04.220Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-20T22:05:04.367 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:05:04 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[87854]: ts=2024-09-20T22:05:04.221Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-20T22:05:04.367 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:05:04 smithi060 bash[87526]: f4474f3f5bac57806d0cea5d47af51fdc93a4ff3202a3ce4ec1225ab09a02a7b 2024-09-20T22:05:04.368 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:05:04 smithi060 systemd[1]: Started Ceph alertmanager.a for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T22:05:04.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:04.382 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:04.382 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:04.483 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:04.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:04.512 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:04.512 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:04.576 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:04.614 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:04.643 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:04.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:04.644 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:04.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:04.781 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T22:05:04.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:04.785 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:04.786 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:04.886 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:04.931 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:04.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:04.934 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:05.034 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:05.075 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:05.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:05.076 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:05.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:05.217 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:05.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:05.218 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:05.288 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 68 16 108522 108506 99.7019 72.0625 0.0117705 0.0100244 2024-09-20T22:05:05.289 INFO:teuthology.orchestra.run.smithi060.stdout:get_pg_num_history pg_num_history pg_num_history(e90 pg_nums {1={19=1},2={62=3},3={63=32},4={65=32},5={67=32},6={69=32},7={80=16}} deleted_pools ) 2024-09-20T22:05:05.289 INFO:teuthology.orchestra.run.smithi060.stdout:Importing pgid 5.5 2024-09-20T22:05:05.289 INFO:teuthology.orchestra.run.smithi060.stdout:write_pg epoch 68 info 5.5( empty local-lis/les=67/68 n=0 ec=67/67 lis/c=67/67 les/c/f=68/68/0 sis=67) 2024-09-20T22:05:05.291 INFO:teuthology.orchestra.run.smithi060.stdout:Import successful 2024-09-20T22:05:05.299 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:05:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:05:05.319 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:05.357 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:05.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:05.358 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:05.459 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:05.483 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:05.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:05.484 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:05.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:05 smithi067 ceph-mon[33511]: 7.c scrub ok 2024-09-20T22:05:05.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:05.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:05.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:05 smithi067 ceph-mon[33511]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-20T22:05:05.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:05.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:05:05.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:05 smithi067 ceph-mon[33511]: 3.1e scrub starts 2024-09-20T22:05:05.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:05 smithi067 ceph-mon[33511]: 3.1e scrub ok 2024-09-20T22:05:05.584 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:05.620 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:05.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:05.621 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:05.703 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:05:05.723 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:05.733 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:05 smithi060 ceph-mon[29694]: 7.c scrub ok 2024-09-20T22:05:05.733 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:05.733 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:05.733 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:05 smithi060 ceph-mon[29694]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-20T22:05:05.733 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:05.733 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:05:05.733 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:05 smithi060 ceph-mon[29694]: 3.1e scrub starts 2024-09-20T22:05:05.734 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:05 smithi060 ceph-mon[29694]: 3.1e scrub ok 2024-09-20T22:05:05.734 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:05 smithi060 ceph-mon[40290]: 7.c scrub ok 2024-09-20T22:05:05.734 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:05.734 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:05.734 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:05 smithi060 ceph-mon[40290]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-20T22:05:05.734 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:05.734 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:05:05.734 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:05 smithi060 ceph-mon[40290]: 3.1e scrub starts 2024-09-20T22:05:05.734 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:05 smithi060 ceph-mon[40290]: 3.1e scrub ok 2024-09-20T22:05:05.772 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:05.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:05.773 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:05.873 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:05.895 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:05.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:05.896 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:05.997 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:06.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:06.049 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:06.049 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:06.150 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:06.176 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:06.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:06.177 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:06.278 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:06.288 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 69 16 109447 109431 99.0949 57.8125 0.011204 0.0100865 2024-09-20T22:05:06.307 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:06.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:06.308 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:06.409 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:06.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:06.455 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:06.456 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:06.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:06 smithi067 ceph-mon[33511]: Reconfiguring daemon prometheus.a on smithi067 2024-09-20T22:05:06.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:06 smithi067 ceph-mon[33511]: 7.e scrub starts 2024-09-20T22:05:06.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:06 smithi067 ceph-mon[33511]: pgmap v90: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 43 MiB/s wr, 691 op/s; 38432/308844 objects degraded (12.444%); 6343/308844 objects misplaced (2.054%) 2024-09-20T22:05:06.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:06 smithi067 ceph-mon[33511]: 3.d scrub starts 2024-09-20T22:05:06.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:06 smithi067 ceph-mon[33511]: 3.d scrub ok 2024-09-20T22:05:06.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:06 smithi067 ceph-mon[33511]: 6.1b scrub starts 2024-09-20T22:05:06.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:06 smithi067 ceph-mon[33511]: 6.1b scrub ok 2024-09-20T22:05:06.536 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:05:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[87854]: ts=2024-09-20T22:05:06.164Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000405713s 2024-09-20T22:05:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:06 smithi060 ceph-mon[29694]: Reconfiguring daemon prometheus.a on smithi067 2024-09-20T22:05:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:06 smithi060 ceph-mon[29694]: 7.e scrub starts 2024-09-20T22:05:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:06 smithi060 ceph-mon[29694]: pgmap v90: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 43 MiB/s wr, 691 op/s; 38432/308844 objects degraded (12.444%); 6343/308844 objects misplaced (2.054%) 2024-09-20T22:05:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:06 smithi060 ceph-mon[29694]: 3.d scrub starts 2024-09-20T22:05:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:06 smithi060 ceph-mon[29694]: 3.d scrub ok 2024-09-20T22:05:06.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:06 smithi060 ceph-mon[29694]: 6.1b scrub starts 2024-09-20T22:05:06.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:06 smithi060 ceph-mon[29694]: 6.1b scrub ok 2024-09-20T22:05:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:06 smithi060 ceph-mon[40290]: Reconfiguring daemon prometheus.a on smithi067 2024-09-20T22:05:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:06 smithi060 ceph-mon[40290]: 7.e scrub starts 2024-09-20T22:05:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:06 smithi060 ceph-mon[40290]: pgmap v90: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 43 MiB/s wr, 691 op/s; 38432/308844 objects degraded (12.444%); 6343/308844 objects misplaced (2.054%) 2024-09-20T22:05:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:06 smithi060 ceph-mon[40290]: 3.d scrub starts 2024-09-20T22:05:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:06 smithi060 ceph-mon[40290]: 3.d scrub ok 2024-09-20T22:05:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:06 smithi060 ceph-mon[40290]: 6.1b scrub starts 2024-09-20T22:05:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:06 smithi060 ceph-mon[40290]: 6.1b scrub ok 2024-09-20T22:05:06.557 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:06.585 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:06.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:06.586 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:06.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:06.714 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:06.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:06.715 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:06.715 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:05:06.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:06.856 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:06.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:06.857 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:06.957 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:06.979 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:06.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:06.980 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:07.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:07.128 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:07.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:07.129 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:07.230 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:07.262 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:07.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:07.263 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:07.288 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 70 16 110437 110421 98.5632 61.875 0.0140132 0.0101413 2024-09-20T22:05:07.346 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:05:07.364 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:07.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:07 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 38432/308844 objects degraded (12.444%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:05:07.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:07 smithi060 ceph-mon[29694]: 5.1e scrub starts 2024-09-20T22:05:07.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:07 smithi060 ceph-mon[29694]: 5.1e scrub ok 2024-09-20T22:05:07.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:07 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1868438830' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:05:07.374 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:07 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:05:07.374 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:07 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 38432/308844 objects degraded (12.444%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:05:07.374 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:07 smithi060 ceph-mon[40290]: 5.1e scrub starts 2024-09-20T22:05:07.374 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:07 smithi060 ceph-mon[40290]: 5.1e scrub ok 2024-09-20T22:05:07.375 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:07 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1868438830' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:05:07.375 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:07 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:05:07.394 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:07.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:07.396 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:07.496 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:07.525 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:07.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:07.527 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:07.559 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:07 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 38432/308844 objects degraded (12.444%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:05:07.559 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:07 smithi067 ceph-mon[33511]: 5.1e scrub starts 2024-09-20T22:05:07.559 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:07 smithi067 ceph-mon[33511]: 5.1e scrub ok 2024-09-20T22:05:07.560 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:07 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1868438830' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:05:07.560 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:07 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:05:07.627 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:07.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:07.661 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:07.661 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:07.762 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:07.811 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:07.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:07.812 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:07.912 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:07.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:07.948 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:07.948 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:08.049 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:08.085 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:08.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:08.086 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:08.186 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:08.216 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:08.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:08.217 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:08.227 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/exp.5.5.1 2024-09-20T22:05:08.289 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 71 16 111717 111701 98.3007 80 0.0187455 0.0101681 2024-09-20T22:05:08.290 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:05:08.290 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:05:08.318 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:08.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:08.353 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:08.353 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:08.454 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:08.485 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:08.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:08.486 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:08.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:08 smithi060 ceph-mon[29694]: pgmap v91: 148 pgs: 1 active+clean+scrubbing, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 6.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 77 MiB/s wr, 1.24k op/s; 40716/327102 objects degraded (12.447%); 6746/327102 objects misplaced (2.062%) 2024-09-20T22:05:08.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:08 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:05:08.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:08 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:05:08.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:08 smithi060 ceph-mon[29694]: osdmap e109: 8 total, 7 up, 8 in 2024-09-20T22:05:08.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:08 smithi060 ceph-mon[40290]: pgmap v91: 148 pgs: 1 active+clean+scrubbing, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 6.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 77 MiB/s wr, 1.24k op/s; 40716/327102 objects degraded (12.447%); 6746/327102 objects misplaced (2.062%) 2024-09-20T22:05:08.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:08 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:05:08.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:08 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:05:08.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:08 smithi060 ceph-mon[40290]: osdmap e109: 8 total, 7 up, 8 in 2024-09-20T22:05:08.587 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:08.623 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:08.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:08.624 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:08.724 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:08.756 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:08.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:08.757 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:08 smithi067 ceph-mon[33511]: pgmap v91: 148 pgs: 1 active+clean+scrubbing, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 6.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.1 KiB/s rd, 77 MiB/s wr, 1.24k op/s; 40716/327102 objects degraded (12.447%); 6746/327102 objects misplaced (2.062%) 2024-09-20T22:05:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:08 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:05:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:08 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:05:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:08 smithi067 ceph-mon[33511]: osdmap e109: 8 total, 7 up, 8 in 2024-09-20T22:05:08.784 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:08.857 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:08.896 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:08.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:08.897 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:08.998 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:09.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:09.039 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:09.040 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:09.140 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:09.188 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:09.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:09.189 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:09.289 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 72 16 113003 112987 98.0517 80.375 0.00493089 0.0101935 2024-09-20T22:05:09.290 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:09.317 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:09.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:09.318 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:09.418 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:09.449 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:09.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:09.450 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:09.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:09.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:09.589 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:09.589 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:09.690 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:09.712 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:09.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:09.713 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:09.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:09.849 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:09.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:09.850 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:09.951 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:09.982 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:09 smithi067 systemd[1]: Stopping Ceph prometheus.a for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:05:09.985 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:09.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:09.986 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:10.087 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:10.132 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:10.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:10.133 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:10.234 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:10.265 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:05:10.265 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:05:10.272 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:05:10.275 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:10.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:10.276 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:10.289 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 73 16 114110 114094 97.6563 69.1875 0.00720795 0.010235 2024-09-20T22:05:10.314 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:05:10.315 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:05:10.349 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:05:10.349 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:05:10.378 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:10.386 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:05:10.386 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:05:10.417 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:10.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:10.418 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:10.421 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:05:10.421 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:05:10.454 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:05:10.454 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:05:10.455 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:10 smithi060 ceph-mon[29694]: pgmap v93: 148 pgs: 1 active+clean+scrubbing, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 6.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 48 MiB/s wr, 760 op/s; 40716/327102 objects degraded (12.447%); 6746/327102 objects misplaced (2.062%) 2024-09-20T22:05:10.455 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:10 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3400749779' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:10.455 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:10 smithi060 ceph-mon[40290]: pgmap v93: 148 pgs: 1 active+clean+scrubbing, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 6.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 48 MiB/s wr, 760 op/s; 40716/327102 objects degraded (12.447%); 6746/327102 objects misplaced (2.062%) 2024-09-20T22:05:10.455 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:10 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3400749779' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:10.492 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:05:10.492 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:05:10.519 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:10.541 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T22:05:10.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:10.563 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:10.563 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:10.664 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:10.715 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:10.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:10.724 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:10 smithi067 ceph-mon[33511]: pgmap v93: 148 pgs: 1 active+clean+scrubbing, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 6.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 48 MiB/s wr, 760 op/s; 40716/327102 objects degraded (12.447%); 6746/327102 objects misplaced (2.062%) 2024-09-20T22:05:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:10 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3400749779' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:10.825 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:10.833 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:05:10.858 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:10.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:10.859 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:10.960 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:10.992 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:10.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:10.995 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:11.096 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:11.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:11.128 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:11.129 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:11.229 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:11.253 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:10 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:05:10.945Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-20T22:05:11.253 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:10 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:05:10.945Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-20T22:05:11.253 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:10 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:05:10.946Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-20T22:05:11.253 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:10 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:05:10.946Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-20T22:05:11.253 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:10 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:05:10.946Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-20T22:05:11.254 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:10 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:05:10.946Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-20T22:05:11.254 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:10 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:05:10.946Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-20T22:05:11.254 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:10 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:05:10.946Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-20T22:05:11.254 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:10 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:05:10.946Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-20T22:05:11.254 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:10 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:05:10.946Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-20T22:05:11.254 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:10 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:05:10.946Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-20T22:05:11.254 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:10 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[63327]: ts=2024-09-20T22:05:10.946Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-20T22:05:11.254 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:10 smithi067 podman[73523]: 2024-09-20 22:05:10.955199148 +0000 UTC m=+0.186601073 container died 0c9e6954fc644852e8306bef6fd7c4c2b4e19b9466730e6e79db187b3133cc07 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T22:05:11.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:11.271 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:11.272 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:11.291 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 74 16 115269 115253 97.3128 72.4375 0.0363401 0.0102707 2024-09-20T22:05:11.327 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:05:11.327 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:05:11.373 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:11.406 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:11.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:11.426 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:11.426 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:11.434 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:11.513 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:11 smithi067 podman[73523]: 2024-09-20 22:05:11.349740337 +0000 UTC m=+0.581142254 container cleanup 0c9e6954fc644852e8306bef6fd7c4c2b4e19b9466730e6e79db187b3133cc07 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T22:05:11.513 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:11 smithi067 bash[73523]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a 2024-09-20T22:05:11.527 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:11.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:11.572 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:11.572 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:11.673 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:11.709 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:11.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:11.710 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:11.810 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:11.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:11.852 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:11.853 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:11.953 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:11.988 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:11.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:11.989 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:12.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:05:11 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:05:12.014 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:11 smithi067 podman[73557]: 2024-09-20 22:05:11.60176144 +0000 UTC m=+0.638842739 container remove 0c9e6954fc644852e8306bef6fd7c4c2b4e19b9466730e6e79db187b3133cc07 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T22:05:12.014 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:11 smithi067 systemd[1]: ceph-3df9647e-779b-11ef-baf5-efdc52797490@prometheus.a.service: Deactivated successfully. 2024-09-20T22:05:12.014 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:11 smithi067 systemd[1]: Stopped Ceph prometheus.a for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T22:05:12.014 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:11 smithi067 systemd[1]: ceph-3df9647e-779b-11ef-baf5-efdc52797490@prometheus.a.service: Consumed 1.457s CPU time. 2024-09-20T22:05:12.014 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:11 smithi067 systemd[1]: Starting Ceph prometheus.a for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:05:12.091 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:12.120 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:12.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:12.121 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:12.221 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:12.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:12.254 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:12.254 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:12.292 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 75 15 116530 116515 97.0669 78.875 0.0339836 0.010297 2024-09-20T22:05:12.355 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:12.385 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:12.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:12.386 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:12.487 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:12.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-mon[33511]: pgmap v94: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.0 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.0 KiB/s rd, 75 MiB/s wr, 1.21k op/s; 42834/344133 objects degraded (12.447%); 7107/344133 objects misplaced (2.065%) 2024-09-20T22:05:12.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 42834/344133 objects degraded (12.447%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:05:12.513 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 podman[73727]: 2024-09-20 22:05:12.01780861 +0000 UTC m=+0.032340434 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-20T22:05:12.514 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 podman[73727]: 2024-09-20 22:05:12.161596229 +0000 UTC m=+0.176128039 container create 8e494b97ac052a6072fc03dfde220506c002c2d00da0544b7ac8aec3ceb27739 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T22:05:12.520 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:12.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:12.522 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:12.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:12 smithi060 ceph-mon[29694]: pgmap v94: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.0 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.0 KiB/s rd, 75 MiB/s wr, 1.21k op/s; 42834/344133 objects degraded (12.447%); 7107/344133 objects misplaced (2.065%) 2024-09-20T22:05:12.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:12 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 42834/344133 objects degraded (12.447%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:05:12.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:12 smithi060 ceph-mon[40290]: pgmap v94: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.0 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.0 KiB/s rd, 75 MiB/s wr, 1.21k op/s; 42834/344133 objects degraded (12.447%); 7107/344133 objects misplaced (2.065%) 2024-09-20T22:05:12.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:12 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 42834/344133 objects degraded (12.447%), 32 pgs degraded (PG_DEGRADED) 2024-09-20T22:05:12.623 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:12.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:12.658 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:12.658 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:12.759 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:12.801 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:12.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:12.802 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:12.902 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:12.929 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:12.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:12.930 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:13.014 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 podman[73727]: 2024-09-20 22:05:12.520530959 +0000 UTC m=+0.535062771 container init 8e494b97ac052a6072fc03dfde220506c002c2d00da0544b7ac8aec3ceb27739 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T22:05:13.014 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 podman[73727]: 2024-09-20 22:05:12.529968969 +0000 UTC m=+0.544500772 container start 8e494b97ac052a6072fc03dfde220506c002c2d00da0544b7ac8aec3ceb27739 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T22:05:13.014 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.567Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-20T22:05:13.015 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.568Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-20T22:05:13.015 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.568Z caller=main.go:570 level=info host_details="(Linux 5.14.0-508.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024 x86_64 smithi067 (none))" 2024-09-20T22:05:13.015 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.568Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-20T22:05:13.015 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.568Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-20T22:05:13.015 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.570Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-20T22:05:13.015 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.571Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-20T22:05:13.015 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.574Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-20T22:05:13.015 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.575Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-20T22:05:13.015 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.578Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-20T22:05:13.015 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.578Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=3.137µs 2024-09-20T22:05:13.015 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.578Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-20T22:05:13.015 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.579Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-20T22:05:13.016 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.579Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-20T22:05:13.016 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.579Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=34.289µs wal_replay_duration=767.951µs wbl_replay_duration=99ns total_replay_duration=882.385µs 2024-09-20T22:05:13.016 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.580Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-20T22:05:13.016 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.580Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-20T22:05:13.016 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.580Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-20T22:05:13.016 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.601Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=21.372506ms db_storage=809ns remote_storage=1.553µs web_handler=390ns query_engine=616ns scrape=678.629µs scrape_sd=90.794µs notify=15.725µs notify_sd=13.909µs rules=20.238459ms tracing=5.752µs 2024-09-20T22:05:13.016 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.602Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-20T22:05:13.016 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:05:12.602Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-20T22:05:13.016 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 bash[73727]: 8e494b97ac052a6072fc03dfde220506c002c2d00da0544b7ac8aec3ceb27739 2024-09-20T22:05:13.016 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:05:12 smithi067 systemd[1]: Started Ceph prometheus.a for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T22:05:13.032 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:13.066 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:13.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:13.067 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:13.068 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:12 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:05:12] ENGINE Bus STOPPING 2024-09-20T22:05:13.068 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:12 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:05:12] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-20T22:05:13.068 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:12 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:05:12] ENGINE Bus STOPPED 2024-09-20T22:05:13.068 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:12 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:05:12] ENGINE Bus STARTING 2024-09-20T22:05:13.168 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:13.222 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:13.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:13.223 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:13.290 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3, 4, 5, 6, 7] 2024-09-20T22:05:13.290 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:05:13.290 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-09-20T22:05:13.290 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-20T22:05:13.291 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-20T22:05:13.291 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 config set filestore_inject_stall 3 2024-09-20T22:05:13.292 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 76 16 117647 117631 96.7075 69.75 0.0137118 0.0103357 2024-09-20T22:05:13.325 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:13.328 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:05:13.350 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:05:13] ENGINE Serving on http://:::9283 2024-09-20T22:05:13.350 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:05:13] ENGINE Bus STARTED 2024-09-20T22:05:13.350 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:05:13] ENGINE Bus STOPPING 2024-09-20T22:05:13.350 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:05:13] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-20T22:05:13.350 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:05:13] ENGINE Bus STOPPED 2024-09-20T22:05:13.350 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:05:13] ENGINE Bus STARTING 2024-09-20T22:05:13.350 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:05:13] ENGINE Serving on http://:::9283 2024-09-20T22:05:13.350 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:05:13] ENGINE Bus STARTED 2024-09-20T22:05:13.350 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:05:13] ENGINE Bus STOPPING 2024-09-20T22:05:13.351 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:05:13.351 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:13.351 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:13.351 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-20T22:05:13.351 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi060.front.sepia.ceph.com:9093"}]: dispatch 2024-09-20T22:05:13.351 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:13.351 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-20T22:05:13.351 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi067.front.sepia.ceph.com:9095"}]: dispatch 2024-09-20T22:05:13.351 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:13.351 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-20T22:05:13.352 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi067.front.sepia.ceph.com:3000"}]: dispatch 2024-09-20T22:05:13.352 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:13.352 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:05:13.352 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:05:13.352 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:13.352 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:13.352 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-20T22:05:13.352 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi060.front.sepia.ceph.com:9093"}]: dispatch 2024-09-20T22:05:13.352 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:13.352 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-20T22:05:13.352 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi067.front.sepia.ceph.com:9095"}]: dispatch 2024-09-20T22:05:13.352 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:13.353 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-20T22:05:13.353 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi067.front.sepia.ceph.com:3000"}]: dispatch 2024-09-20T22:05:13.353 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:13.353 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:05:13.361 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:13.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:13.362 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:13.463 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:13.492 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:13.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:13.493 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:13.593 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:13.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:13.686 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:13.686 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:13.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:13 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:05:13.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:13 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:13.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:13 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:13.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:13 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-20T22:05:13.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:13 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi060.front.sepia.ceph.com:9093"}]: dispatch 2024-09-20T22:05:13.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:13 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:13.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:13 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-20T22:05:13.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:13 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi067.front.sepia.ceph.com:9095"}]: dispatch 2024-09-20T22:05:13.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:13 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:13.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:13 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-20T22:05:13.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:13 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi067.front.sepia.ceph.com:3000"}]: dispatch 2024-09-20T22:05:13.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:13 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:13.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:13 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:05:13.787 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:13.824 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:05:13.824 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:05:13.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.824 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:118051 7.6 7:67de59c2:::benchmark_data_smithi060_78800_object118050:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-20T22:05:13.824 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:13.765121+0000", 2024-09-20T22:05:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.020647557, 2024-09-20T22:05:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.020685235999999999, 2024-09-20T22:05:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:05:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:05:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:05:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 118051 2024-09-20T22:05:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.765121+0000", 2024-09-20T22:05:13.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.765121+0000", 2024-09-20T22:05:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.765121+0000", 2024-09-20T22:05:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0200000000000003e-07 2024-09-20T22:05:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.765174+0000", 2024-09-20T22:05:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3018e-05 2024-09-20T22:05:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.765182+0000", 2024-09-20T22:05:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6939999999999995e-06 2024-09-20T22:05:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.765188+0000", 2024-09-20T22:05:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.789e-06 2024-09-20T22:05:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.768770+0000", 2024-09-20T22:05:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00358195 2024-09-20T22:05:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.768905+0000", 2024-09-20T22:05:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013520900000000001 2024-09-20T22:05:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:05:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.769064+0000", 2024-09-20T22:05:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015930799999999999 2024-09-20T22:05:13.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:05:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.773803+0000", 2024-09-20T22:05:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0047384360000000004 2024-09-20T22:05:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:05:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.778042+0000", 2024-09-20T22:05:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0042390520000000001 2024-09-20T22:05:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:118059 7.5 e109/80 7:af7f12bf:::benchmark_data_smithi060_78800_object118058:head v 109'7417, mlcod=109'7415)", 2024-09-20T22:05:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:13.779749+0000", 2024-09-20T22:05:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0060189909999999996, 2024-09-20T22:05:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0061594010000000001, 2024-09-20T22:05:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:05:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.779749+0000", 2024-09-20T22:05:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.779749+0000", 2024-09-20T22:05:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.779750+0000", 2024-09-20T22:05:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3900000000000003e-07 2024-09-20T22:05:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.779784+0000", 2024-09-20T22:05:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3977999999999999e-05 2024-09-20T22:05:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.779796+0000", 2024-09-20T22:05:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1949e-05 2024-09-20T22:05:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.779804+0000", 2024-09-20T22:05:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3839999999999999e-06 2024-09-20T22:05:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.780499+0000", 2024-09-20T22:05:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00069519600000000003 2024-09-20T22:05:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.780518+0000", 2024-09-20T22:05:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9215000000000002e-05 2024-09-20T22:05:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:05:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.783596+0000", 2024-09-20T22:05:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0030780009999999999 2024-09-20T22:05:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.833 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:118060 7.f 7:f17ba51b:::benchmark_data_smithi060_78800_object118059:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e109)", 2024-09-20T22:05:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:13.782682+0000", 2024-09-20T22:05:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0030864709999999999, 2024-09-20T22:05:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0033047369999999999, 2024-09-20T22:05:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:05:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:05:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:05:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 118060 2024-09-20T22:05:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:13.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.782682+0000", 2024-09-20T22:05:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.782682+0000", 2024-09-20T22:05:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.782684+0000", 2024-09-20T22:05:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.748e-06 2024-09-20T22:05:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:13.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.782706+0000", 2024-09-20T22:05:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2277e-05 2024-09-20T22:05:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.782730+0000", 2024-09-20T22:05:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4111999999999999e-05 2024-09-20T22:05:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.782738+0000", 2024-09-20T22:05:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2700000000000004e-06 2024-09-20T22:05:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.784404+0000", 2024-09-20T22:05:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0016651140000000001 2024-09-20T22:05:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.785888+0000", 2024-09-20T22:05:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001484034 2024-09-20T22:05:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:118064 7.7 e109/80 7:e736d897:::benchmark_data_smithi060_78800_object118063:head v 109'7401, mlcod=109'7397)", 2024-09-20T22:05:13.837 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:13.784061+0000", 2024-09-20T22:05:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.001707748, 2024-09-20T22:05:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020050089999999999, 2024-09-20T22:05:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:05:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.784061+0000", 2024-09-20T22:05:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.784061+0000", 2024-09-20T22:05:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.784061+0000", 2024-09-20T22:05:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1999999999999999e-07 2024-09-20T22:05:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.784169+0000", 2024-09-20T22:05:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000107759 2024-09-20T22:05:13.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.784183+0000", 2024-09-20T22:05:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3631e-05 2024-09-20T22:05:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.784190+0000", 2024-09-20T22:05:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2749999999999998e-06 2024-09-20T22:05:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.784536+0000", 2024-09-20T22:05:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00034618099999999998 2024-09-20T22:05:13.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:13.784555+0000", 2024-09-20T22:05:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.836e-05 2024-09-20T22:05:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:05:13.841 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 4 2024-09-20T22:05:13.841 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:05:13.866 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:13.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:13.867 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:13.968 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:14.003 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:14.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:14.004 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:14.036 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:05:13] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-20T22:05:14.037 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:05:13] ENGINE Bus STOPPED 2024-09-20T22:05:14.037 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:05:13] ENGINE Bus STARTING 2024-09-20T22:05:14.037 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:05:13] ENGINE Serving on http://:::9283 2024-09-20T22:05:14.037 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:13 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:05:13] ENGINE Bus STARTED 2024-09-20T22:05:14.104 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:14.135 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:14.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:14.135 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:14.236 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:14.274 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:14.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:14.275 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:14.291 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 77 16 118661 118645 96.2746 63.375 0.00752666 0.0103825 2024-09-20T22:05:14.376 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:14.409 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:14.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:14.411 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:14.512 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:14.536 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:05:14 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[87854]: ts=2024-09-20T22:05:14.168Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.004727594s 2024-09-20T22:05:14.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:14 smithi060 ceph-mon[29694]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-20T22:05:14.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:14 smithi060 ceph-mon[29694]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi060.front.sepia.ceph.com:9093"}]: dispatch 2024-09-20T22:05:14.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:14 smithi060 ceph-mon[29694]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-20T22:05:14.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:14 smithi060 ceph-mon[29694]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi067.front.sepia.ceph.com:9095"}]: dispatch 2024-09-20T22:05:14.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:14 smithi060 ceph-mon[29694]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-20T22:05:14.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:14 smithi060 ceph-mon[29694]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi067.front.sepia.ceph.com:3000"}]: dispatch 2024-09-20T22:05:14.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:14 smithi060 ceph-mon[29694]: pgmap v95: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.0 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 73 MiB/s wr, 1.18k op/s; 42834/344133 objects degraded (12.447%); 7107/344133 objects misplaced (2.065%) 2024-09-20T22:05:14.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:14 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3804831305' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:05:14.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:14 smithi060 ceph-mon[40290]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-20T22:05:14.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:14 smithi060 ceph-mon[40290]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi060.front.sepia.ceph.com:9093"}]: dispatch 2024-09-20T22:05:14.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:14 smithi060 ceph-mon[40290]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-20T22:05:14.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:14 smithi060 ceph-mon[40290]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi067.front.sepia.ceph.com:9095"}]: dispatch 2024-09-20T22:05:14.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:14 smithi060 ceph-mon[40290]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-20T22:05:14.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:14 smithi060 ceph-mon[40290]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi067.front.sepia.ceph.com:3000"}]: dispatch 2024-09-20T22:05:14.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:14 smithi060 ceph-mon[40290]: pgmap v95: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.0 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 73 MiB/s wr, 1.18k op/s; 42834/344133 objects degraded (12.447%); 7107/344133 objects misplaced (2.065%) 2024-09-20T22:05:14.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:14 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3804831305' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:05:14.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:14.555 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:14.555 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:14.656 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:14.701 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:14.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:14.702 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:14.744 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T22:05:14.754 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:14 smithi067 ceph-mon[33511]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-20T22:05:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:14 smithi067 ceph-mon[33511]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi060.front.sepia.ceph.com:9093"}]: dispatch 2024-09-20T22:05:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:14 smithi067 ceph-mon[33511]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-20T22:05:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:14 smithi067 ceph-mon[33511]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi067.front.sepia.ceph.com:9095"}]: dispatch 2024-09-20T22:05:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:14 smithi067 ceph-mon[33511]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-20T22:05:14.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:14 smithi067 ceph-mon[33511]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi067.front.sepia.ceph.com:3000"}]: dispatch 2024-09-20T22:05:14.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:14 smithi067 ceph-mon[33511]: pgmap v95: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.0 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 73 MiB/s wr, 1.18k op/s; 42834/344133 objects degraded (12.447%); 7107/344133 objects misplaced (2.065%) 2024-09-20T22:05:14.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:14 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3804831305' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:05:14.769 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:14.803 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:14.860 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:05:14.867 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:14.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:14.868 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:14.968 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:15.000 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:15.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:15.001 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:15.102 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:15.123 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:15.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:15.124 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:15.225 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:15.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:15.272 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:15.272 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:15.292 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 78 16 119768 119752 95.9273 69.1875 0.0065987 0.0104199 2024-09-20T22:05:15.372 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:15.405 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:15.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:15.406 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:15.507 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:15.538 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:15.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:15.539 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:15.595 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:15 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:05:15.596 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:15 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3804831305' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:05:15.596 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:15 smithi060 ceph-mon[29694]: osdmap e110: 8 total, 7 up, 8 in 2024-09-20T22:05:15.596 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:15 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:05:15.596 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:15 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3804831305' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:05:15.596 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:15 smithi060 ceph-mon[40290]: osdmap e110: 8 total, 7 up, 8 in 2024-09-20T22:05:15.639 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:15.669 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:15.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:15.670 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:15.735 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:15 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:05:15.735 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:15 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3804831305' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:05:15.735 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:15 smithi067 ceph-mon[33511]: osdmap e110: 8 total, 7 up, 8 in 2024-09-20T22:05:15.770 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:15.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:15.816 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:15.816 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:15.917 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:15.962 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:15.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:15.963 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:16.064 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:16.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:16.106 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:16.106 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:16.206 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:16.235 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:16.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:16.236 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:16.292 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 79 16 121016 121000 95.7 78 0.0123219 0.0104437 2024-09-20T22:05:16.324 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:05:16.324 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:05:16.327 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:05:16.337 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:16.369 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:05:16.369 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:05:16.375 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:16.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:16.376 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:16.406 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:05:16.406 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:05:16.440 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:05:16.440 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:05:16.475 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:16 smithi060 ceph-mon[29694]: pgmap v97: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.0 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 44 MiB/s wr, 709 op/s; 42834/344133 objects degraded (12.447%); 7107/344133 objects misplaced (2.065%) 2024-09-20T22:05:16.475 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:16 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1538639135' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:16.475 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:05:16.476 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:05:16.476 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:16 smithi060 ceph-mon[40290]: pgmap v97: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.0 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 44 MiB/s wr, 709 op/s; 42834/344133 objects degraded (12.447%); 7107/344133 objects misplaced (2.065%) 2024-09-20T22:05:16.476 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:16 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1538639135' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:16.477 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:16.509 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:05:16.509 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:05:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:16.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:16.511 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:16.542 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:05:16.542 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:05:16.613 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:16.623 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:16 smithi067 ceph-mon[33511]: pgmap v97: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.0 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 44 MiB/s wr, 709 op/s; 42834/344133 objects degraded (12.447%); 7107/344133 objects misplaced (2.065%) 2024-09-20T22:05:16.623 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:16 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1538639135' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:16.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:16.660 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:16.661 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:16.761 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:16.790 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:16.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:16.791 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:16.892 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:16.939 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:16.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:16.940 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:17.041 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:17.072 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:17.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:17.073 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:17.079 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:05:17.080 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-20T22:05:17.080 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:05:17.174 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:17.216 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:17.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:17.217 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:17.292 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:05:17.292137+0000 min lat: 0.000984708 max lat: 1.08039 avg lat: 0.0104742 2024-09-20T22:05:17.293 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:05:17.293 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 80 15 122202 122187 95.4308 74.1875 0.00517779 0.0104742 2024-09-20T22:05:17.318 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:17.365 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:17.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:17.366 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:17.467 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:17.511 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:17.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:17.512 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:17 smithi060 ceph-mon[29694]: pgmap v98: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.3 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.0 KiB/s rd, 73 MiB/s wr, 1.17k op/s; 44965/361239 objects degraded (12.447%); 7482/361239 objects misplaced (2.071%) 2024-09-20T22:05:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:17 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 44965/361239 objects degraded (12.447%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:17.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:17 smithi060 ceph-mon[40290]: pgmap v98: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.3 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.0 KiB/s rd, 73 MiB/s wr, 1.17k op/s; 44965/361239 objects degraded (12.447%); 7482/361239 objects misplaced (2.071%) 2024-09-20T22:05:17.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:17 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 44965/361239 objects degraded (12.447%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:17.613 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:17.648 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:17.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:17.649 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:17.750 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:17 smithi067 ceph-mon[33511]: pgmap v98: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.3 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.0 KiB/s rd, 73 MiB/s wr, 1.17k op/s; 44965/361239 objects degraded (12.447%); 7482/361239 objects misplaced (2.071%) 2024-09-20T22:05:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:17 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 44965/361239 objects degraded (12.447%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:17.780 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:17.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:17.781 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:17.882 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:17.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:17.925 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:17.925 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:18.026 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:18.052 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:05:18.057 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:18.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:18.058 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:18.159 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:18.185 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:18.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:18.186 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:18.287 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:18.292 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 81 16 123433 123417 95.2016 76.875 0.00720046 0.0104995 2024-09-20T22:05:18.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:18.331 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:18.331 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:18.432 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:18.467 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:18.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:18.468 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:18.569 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:18.612 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:18.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:18.613 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:18.714 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:18.760 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:18.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:18.761 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:18.768 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:05:18.769 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:05:18.862 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:18.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:18.911 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:18.911 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:18.947 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:19.013 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:19.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:19.040 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:19.040 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:19.141 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:19.181 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:19.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:19.182 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:19.283 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:19.296 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 82 16 124658 124642 94.9699 76.5625 0.00498918 0.0105244 2024-09-20T22:05:19.312 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:19.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:19.313 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:19.414 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:19.447 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:19.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:19.448 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:19.549 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:19.555 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:19.574 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:19.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:19.575 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:19.619 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:05:19.619 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:05:19.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:19.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:19.716 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:19.716 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:19.817 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:19.856 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:19.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:19.857 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:19.957 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:19.994 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:19.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:19.995 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:19 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:19 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:19 smithi060 ceph-mon[29694]: pgmap v99: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.3 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.14k op/s; 44965/361239 objects degraded (12.447%); 7482/361239 objects misplaced (2.071%) 2024-09-20T22:05:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:19 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:19 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:05:20.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:19 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:20.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:19 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:20.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:19 smithi060 ceph-mon[40290]: pgmap v99: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.3 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.14k op/s; 44965/361239 objects degraded (12.447%); 7482/361239 objects misplaced (2.071%) 2024-09-20T22:05:20.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:19 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:20.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:19 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:05:20.096 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:20.128 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:20.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:20.129 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:20.230 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:20.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:19 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:20.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:19 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:20.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:19 smithi067 ceph-mon[33511]: pgmap v99: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.3 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.14k op/s; 44965/361239 objects degraded (12.447%); 7482/361239 objects misplaced (2.071%) 2024-09-20T22:05:20.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:19 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:20.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:19 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:05:20.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:20.293 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:20.293 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:20.297 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 83 16 125840 125824 94.7151 73.875 0.0125333 0.0105532 2024-09-20T22:05:20.394 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:20.427 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:20.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:20.428 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:20.529 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:20.577 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:20.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:20.578 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:20.678 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:20.709 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:20.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:20.710 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:20.770 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:05:20.811 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:20.880 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:20.881 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:05:20.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:20.882 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:20.902 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T22:05:20.982 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:21.009 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:21.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:21.010 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:21.110 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:21.136 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:21.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:21.137 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:21.237 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:21.254 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:20 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1716199517' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:05:21.255 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:20 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1716199517' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:05:21.260 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:20 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1716199517' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:05:21.268 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:21.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:21.269 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:21.297 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 84 16 126934 126918 94.4012 68.375 0.0149299 0.0105886 2024-09-20T22:05:21.370 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:21.402 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:21.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:21.403 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:21.504 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:21.537 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:21.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:21.538 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:21.639 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:21.671 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:21.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:21.672 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:21.773 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:21.802 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:21.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:21.803 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:21.813 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:21.883 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:21.903 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:21.951 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:21.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:21.952 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:21.972 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:21 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:05:21.972 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:21 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1716199517' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:05:21.972 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:21 smithi060 ceph-mon[29694]: osdmap e111: 8 total, 7 up, 8 in 2024-09-20T22:05:21.973 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:21 smithi060 ceph-mon[29694]: pgmap v101: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.7 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 92 MiB/s wr, 1.47k op/s; 47294/379299 objects degraded (12.469%); 7843/379299 objects misplaced (2.068%) 2024-09-20T22:05:21.974 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:21 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:05:21.974 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:21 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1716199517' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:05:21.974 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:21 smithi060 ceph-mon[40290]: osdmap e111: 8 total, 7 up, 8 in 2024-09-20T22:05:21.974 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:21 smithi060 ceph-mon[40290]: pgmap v101: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.7 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 92 MiB/s wr, 1.47k op/s; 47294/379299 objects degraded (12.469%); 7843/379299 objects misplaced (2.068%) 2024-09-20T22:05:22.052 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:22.085 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:22.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:22.086 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:22.186 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:22.218 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:22.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:22.219 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:22.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:05:21 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:05:22.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:21 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:05:22.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:21 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1716199517' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:05:22.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:21 smithi067 ceph-mon[33511]: osdmap e111: 8 total, 7 up, 8 in 2024-09-20T22:05:22.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:21 smithi067 ceph-mon[33511]: pgmap v101: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.7 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 92 MiB/s wr, 1.47k op/s; 47294/379299 objects degraded (12.469%); 7843/379299 objects misplaced (2.068%) 2024-09-20T22:05:22.297 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 85 16 128147 128131 94.1824 75.8125 0.0232897 0.0106129 2024-09-20T22:05:22.320 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:22.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:22.377 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:22.377 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:22.478 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:22.504 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:22.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:22.505 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:22.606 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:22.631 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:22.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:22.631 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:22.732 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:22.753 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:05:22.754 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:05:22.756 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:05:22.759 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:22.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:22.760 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:22.795 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:05:22.796 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:05:22.836 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:05:22.836 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:05:22.861 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:22.863 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:05:22.863 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:05:22.863 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:05:22.864 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:05:22.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.864 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:22517 7.4 7:27ef929f:::benchmark_data_smithi060_78800_object22516:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:05:22.864 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:06.081505+0000", 2024-09-20T22:05:22.864 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.732742947000006, 2024-09-20T22:05:22.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087497665000000002, 2024-09-20T22:05:22.865 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.865 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.865 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:05:22.865 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:05:22.865 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:05:22.865 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:22.865 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22517 2024-09-20T22:05:22.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.866 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.081505+0000", 2024-09-20T22:05:22.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.867 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.867 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.081505+0000", 2024-09-20T22:05:22.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.868 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.081506+0000", 2024-09-20T22:05:22.868 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2900000000000002e-07 2024-09-20T22:05:22.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.869 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.081528+0000", 2024-09-20T22:05:22.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1778000000000001e-05 2024-09-20T22:05:22.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.081540+0000", 2024-09-20T22:05:22.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2238e-05 2024-09-20T22:05:22.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.081547+0000", 2024-09-20T22:05:22.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9020000000000002e-06 2024-09-20T22:05:22.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.082500+0000", 2024-09-20T22:05:22.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00095352299999999996 2024-09-20T22:05:22.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.082612+0000", 2024-09-20T22:05:22.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111366 2024-09-20T22:05:22.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:05:22.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.082753+0000", 2024-09-20T22:05:22.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014163199999999999 2024-09-20T22:05:22.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:05:22.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.087202+0000", 2024-09-20T22:05:22.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0044484909999999997 2024-09-20T22:05:22.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:05:22.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.087241+0000", 2024-09-20T22:05:22.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9391000000000003e-05 2024-09-20T22:05:22.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:05:22.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.168906+0000", 2024-09-20T22:05:22.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.081665155000000003 2024-09-20T22:05:22.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:05:22.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.168961+0000", 2024-09-20T22:05:22.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4138999999999998e-05 2024-09-20T22:05:22.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169003+0000", 2024-09-20T22:05:22.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2120999999999999e-05 2024-09-20T22:05:22.876 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.876 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.876 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.877 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:22518 7.4 7:2c0c61f8:::benchmark_data_smithi060_78800_object22517:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:05:22.877 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:06.081583+0000", 2024-09-20T22:05:22.877 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.732664490000005, 2024-09-20T22:05:22.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088096492999999998, 2024-09-20T22:05:22.877 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.877 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.877 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:05:22.877 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:05:22.877 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:05:22.877 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:22.877 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22518 2024-09-20T22:05:22.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.878 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.878 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.081583+0000", 2024-09-20T22:05:22.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.878 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.081583+0000", 2024-09-20T22:05:22.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.081584+0000", 2024-09-20T22:05:22.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2200000000000004e-07 2024-09-20T22:05:22.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.081608+0000", 2024-09-20T22:05:22.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3555000000000001e-05 2024-09-20T22:05:22.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.081617+0000", 2024-09-20T22:05:22.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9989999999999997e-06 2024-09-20T22:05:22.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.081621+0000", 2024-09-20T22:05:22.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9629999999999997e-06 2024-09-20T22:05:22.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084571+0000", 2024-09-20T22:05:22.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0029498290000000002 2024-09-20T22:05:22.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084691+0000", 2024-09-20T22:05:22.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119524 2024-09-20T22:05:22.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:05:22.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.085007+0000", 2024-09-20T22:05:22.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00031568900000000002 2024-09-20T22:05:22.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:05:22.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169043+0000", 2024-09-20T22:05:22.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084036640999999995 2024-09-20T22:05:22.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:05:22.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169053+0000", 2024-09-20T22:05:22.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0284e-05 2024-09-20T22:05:22.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:05:22.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169596+0000", 2024-09-20T22:05:22.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00054208700000000002 2024-09-20T22:05:22.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:05:22.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169644+0000", 2024-09-20T22:05:22.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8667e-05 2024-09-20T22:05:22.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169680+0000", 2024-09-20T22:05:22.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5732999999999998e-05 2024-09-20T22:05:22.883 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.883 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.883 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.883 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:22515 7.7 e88/80 7:e2a23e22:::benchmark_data_smithi060_78800_object22514:head v 88'1396, mlcod=88'1393)", 2024-09-20T22:05:22.883 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:06.084141+0000", 2024-09-20T22:05:22.883 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.730107168999993, 2024-09-20T22:05:22.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084897415000000004, 2024-09-20T22:05:22.883 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.883 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.883 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:05:22.883 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084141+0000", 2024-09-20T22:05:22.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084141+0000", 2024-09-20T22:05:22.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084141+0000", 2024-09-20T22:05:22.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4700000000000001e-07 2024-09-20T22:05:22.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084181+0000", 2024-09-20T22:05:22.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9774000000000002e-05 2024-09-20T22:05:22.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084193+0000", 2024-09-20T22:05:22.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1575e-05 2024-09-20T22:05:22.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084199+0000", 2024-09-20T22:05:22.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3629999999999999e-06 2024-09-20T22:05:22.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084448+0000", 2024-09-20T22:05:22.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00024850499999999999 2024-09-20T22:05:22.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084466+0000", 2024-09-20T22:05:22.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7912999999999999e-05 2024-09-20T22:05:22.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:05:22.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.087400+0000", 2024-09-20T22:05:22.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0029349319999999999 2024-09-20T22:05:22.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169038+0000", 2024-09-20T22:05:22.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.081637706000000004 2024-09-20T22:05:22.887 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.887 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.887 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.887 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:22520 7.7 e88/80 7:e34001c7:::benchmark_data_smithi060_78800_object22519:head v 88'1397, mlcod=88'1393)", 2024-09-20T22:05:22.888 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:06.084420+0000", 2024-09-20T22:05:22.888 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.729828002000005, 2024-09-20T22:05:22.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084649774999999997, 2024-09-20T22:05:22.888 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.888 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.888 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:05:22.888 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084420+0000", 2024-09-20T22:05:22.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084420+0000", 2024-09-20T22:05:22.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084420+0000", 2024-09-20T22:05:22.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8100000000000003e-07 2024-09-20T22:05:22.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084452+0000", 2024-09-20T22:05:22.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1696999999999998e-05 2024-09-20T22:05:22.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084462+0000", 2024-09-20T22:05:22.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.662e-06 2024-09-20T22:05:22.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084468+0000", 2024-09-20T22:05:22.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6000000000000003e-06 2024-09-20T22:05:22.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.085340+0000", 2024-09-20T22:05:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00087155499999999999 2024-09-20T22:05:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.085359+0000", 2024-09-20T22:05:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8749e-05 2024-09-20T22:05:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:05:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169044+0000", 2024-09-20T22:05:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083685223000000003 2024-09-20T22:05:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169070+0000", 2024-09-20T22:05:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5808000000000001e-05 2024-09-20T22:05:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop_reply(client.24667.0:22517 7.4 e88/80 ondisk, result = 0)", 2024-09-20T22:05:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:06.084641+0000", 2024-09-20T22:05:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.729606701999998, 2024-09-20T22:05:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084370876999999997, 2024-09-20T22:05:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:05:22.892 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084641+0000", 2024-09-20T22:05:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084641+0000", 2024-09-20T22:05:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084642+0000", 2024-09-20T22:05:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8599999999999999e-07 2024-09-20T22:05:22.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084644+0000", 2024-09-20T22:05:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1210000000000001e-06 2024-09-20T22:05:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084650+0000", 2024-09-20T22:05:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2129999999999996e-06 2024-09-20T22:05:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.084655+0000", 2024-09-20T22:05:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0679999999999996e-06 2024-09-20T22:05:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.168892+0000", 2024-09-20T22:05:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084236820000000004 2024-09-20T22:05:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.168900+0000", 2024-09-20T22:05:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3499999999999997e-06 2024-09-20T22:05:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169012+0000", 2024-09-20T22:05:22.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112019 2024-09-20T22:05:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:22524 7.7 e88/80 7:e83bec1e:::benchmark_data_smithi060_78800_object22523:head v 88'1398, mlcod=88'1393)", 2024-09-20T22:05:22.897 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:06.085171+0000", 2024-09-20T22:05:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.729077290000006, 2024-09-20T22:05:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083931570999999996, 2024-09-20T22:05:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:05:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.085171+0000", 2024-09-20T22:05:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.085171+0000", 2024-09-20T22:05:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.085171+0000", 2024-09-20T22:05:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1900000000000002e-07 2024-09-20T22:05:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.085201+0000", 2024-09-20T22:05:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9578999999999999e-05 2024-09-20T22:05:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.085210+0000", 2024-09-20T22:05:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2920000000000001e-06 2024-09-20T22:05:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.085216+0000", 2024-09-20T22:05:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7919999999999998e-06 2024-09-20T22:05:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086297+0000", 2024-09-20T22:05:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0010813579999999999 2024-09-20T22:05:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086317+0000", 2024-09-20T22:05:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.961e-05 2024-09-20T22:05:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:05:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169072+0000", 2024-09-20T22:05:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.082754964 2024-09-20T22:05:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169102+0000", 2024-09-20T22:05:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0357000000000001e-05 2024-09-20T22:05:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.901 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop_reply(client.24667.0:22518 7.4 e88/80 ondisk, result = 0)", 2024-09-20T22:05:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:06.086495+0000", 2024-09-20T22:05:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.727752697, 2024-09-20T22:05:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.082559113000000003, 2024-09-20T22:05:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:05:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086495+0000", 2024-09-20T22:05:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086495+0000", 2024-09-20T22:05:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086496+0000", 2024-09-20T22:05:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9999999999999995e-07 2024-09-20T22:05:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086501+0000", 2024-09-20T22:05:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4909999999999999e-06 2024-09-20T22:05:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086512+0000", 2024-09-20T22:05:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0847e-05 2024-09-20T22:05:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086531+0000", 2024-09-20T22:05:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.95e-05 2024-09-20T22:05:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169034+0000", 2024-09-20T22:05:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.082503002000000006 2024-09-20T22:05:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169039+0000", 2024-09-20T22:05:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2020000000000003e-06 2024-09-20T22:05:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169054+0000", 2024-09-20T22:05:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5071e-05 2024-09-20T22:05:22.905 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.906 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.906 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:22528 7.9 7:908d4937:::benchmark_data_smithi060_78800_object22527:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:05:22.906 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:06.086615+0000", 2024-09-20T22:05:22.906 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.727633109999999, 2024-09-20T22:05:22.906 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088263150999999998, 2024-09-20T22:05:22.906 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.906 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.906 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:05:22.906 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:05:22.906 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:05:22.907 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:22.907 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22528 2024-09-20T22:05:22.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.907 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086615+0000", 2024-09-20T22:05:22.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086615+0000", 2024-09-20T22:05:22.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086616+0000", 2024-09-20T22:05:22.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.421e-06 2024-09-20T22:05:22.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086650+0000", 2024-09-20T22:05:22.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3961999999999998e-05 2024-09-20T22:05:22.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086662+0000", 2024-09-20T22:05:22.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1533e-05 2024-09-20T22:05:22.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086668+0000", 2024-09-20T22:05:22.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4729999999999997e-06 2024-09-20T22:05:22.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086803+0000", 2024-09-20T22:05:22.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000134876 2024-09-20T22:05:22.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086924+0000", 2024-09-20T22:05:22.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000120794 2024-09-20T22:05:22.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:05:22.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.087053+0000", 2024-09-20T22:05:22.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012871399999999999 2024-09-20T22:05:22.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:05:22.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.168871+0000", 2024-09-20T22:05:22.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.081818881999999996 2024-09-20T22:05:22.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:05:22.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169137+0000", 2024-09-20T22:05:22.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026509199999999998 2024-09-20T22:05:22.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:05:22.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.174776+0000", 2024-09-20T22:05:22.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0056392940000000004 2024-09-20T22:05:22.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:05:22.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.174832+0000", 2024-09-20T22:05:22.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5889e-05 2024-09-20T22:05:22.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.174878+0000", 2024-09-20T22:05:22.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6220999999999997e-05 2024-09-20T22:05:22.913 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.913 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.913 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.914 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:22523 7.5 e88/80 7:a7080f75:::benchmark_data_smithi060_78800_object22522:head v 88'1419, mlcod=88'1415)", 2024-09-20T22:05:22.914 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:06.086804+0000", 2024-09-20T22:05:22.914 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.727443719999997, 2024-09-20T22:05:22.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085642177, 2024-09-20T22:05:22.914 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.914 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.914 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:05:22.914 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086804+0000", 2024-09-20T22:05:22.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086804+0000", 2024-09-20T22:05:22.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086805+0000", 2024-09-20T22:05:22.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4000000000000001e-07 2024-09-20T22:05:22.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086852+0000", 2024-09-20T22:05:22.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7256000000000003e-05 2024-09-20T22:05:22.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086862+0000", 2024-09-20T22:05:22.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7429999999999997e-06 2024-09-20T22:05:22.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086869+0000", 2024-09-20T22:05:22.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8759999999999999e-06 2024-09-20T22:05:22.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.087061+0000", 2024-09-20T22:05:22.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019199000000000001 2024-09-20T22:05:22.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.087078+0000", 2024-09-20T22:05:22.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7742000000000001e-05 2024-09-20T22:05:22.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:05:22.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.172403+0000", 2024-09-20T22:05:22.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085324538000000005 2024-09-20T22:05:22.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.172446+0000", 2024-09-20T22:05:22.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3392000000000001e-05 2024-09-20T22:05:22.918 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.918 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.918 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.918 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:22527 7.5 e88/80 7:a1def41c:::benchmark_data_smithi060_78800_object22526:head v 88'1420, mlcod=88'1415)", 2024-09-20T22:05:22.918 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:06.086887+0000", 2024-09-20T22:05:22.919 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.727361052000006, 2024-09-20T22:05:22.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086580868000000005, 2024-09-20T22:05:22.919 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.919 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.919 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:05:22.919 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086887+0000", 2024-09-20T22:05:22.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086887+0000", 2024-09-20T22:05:22.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086887+0000", 2024-09-20T22:05:22.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7099999999999997e-07 2024-09-20T22:05:22.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086932+0000", 2024-09-20T22:05:22.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4277000000000003e-05 2024-09-20T22:05:22.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086940+0000", 2024-09-20T22:05:22.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0690000000000002e-06 2024-09-20T22:05:22.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086945+0000", 2024-09-20T22:05:22.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5520000000000004e-06 2024-09-20T22:05:22.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169743+0000", 2024-09-20T22:05:22.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.082798301000000005 2024-09-20T22:05:22.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169771+0000", 2024-09-20T22:05:22.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7707000000000001e-05 2024-09-20T22:05:22.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:05:22.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.173403+0000", 2024-09-20T22:05:22.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0036322529999999998 2024-09-20T22:05:22.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.173468+0000", 2024-09-20T22:05:22.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4338000000000006e-05 2024-09-20T22:05:22.923 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.923 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.924 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.924 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:22530 7.5 e88/80 7:a570fd85:::benchmark_data_smithi060_78800_object22529:head v 88'1421, mlcod=88'1415)", 2024-09-20T22:05:22.924 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:06.086955+0000", 2024-09-20T22:05:22.924 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.727292543999994, 2024-09-20T22:05:22.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.086565141999999998, 2024-09-20T22:05:22.925 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.925 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.925 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:05:22.925 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086955+0000", 2024-09-20T22:05:22.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086955+0000", 2024-09-20T22:05:22.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086956+0000", 2024-09-20T22:05:22.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.41e-07 2024-09-20T22:05:22.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086985+0000", 2024-09-20T22:05:22.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9070000000000001e-05 2024-09-20T22:05:22.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086992+0000", 2024-09-20T22:05:22.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2429999999999999e-06 2024-09-20T22:05:22.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.086997+0000", 2024-09-20T22:05:22.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9869999999999999e-06 2024-09-20T22:05:22.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.170868+0000", 2024-09-20T22:05:22.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083870695999999995 2024-09-20T22:05:22.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.170885+0000", 2024-09-20T22:05:22.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7538999999999999e-05 2024-09-20T22:05:22.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:05:22.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.173470+0000", 2024-09-20T22:05:22.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025846380000000002 2024-09-20T22:05:22.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.173521+0000", 2024-09-20T22:05:22.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0627999999999997e-05 2024-09-20T22:05:22.933 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.933 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.933 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.934 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:22531 7.6 7:67764ed6:::benchmark_data_smithi060_78800_object22530:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:05:22.934 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:06.087490+0000", 2024-09-20T22:05:22.934 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.726757458999998, 2024-09-20T22:05:22.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088987312999999998, 2024-09-20T22:05:22.935 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.935 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.935 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:05:22.935 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:05:22.935 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:05:22.935 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:22.935 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22531 2024-09-20T22:05:22.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.936 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.087490+0000", 2024-09-20T22:05:22.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.087490+0000", 2024-09-20T22:05:22.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.087491+0000", 2024-09-20T22:05:22.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4900000000000004e-07 2024-09-20T22:05:22.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.087517+0000", 2024-09-20T22:05:22.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5570000000000001e-05 2024-09-20T22:05:22.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169111+0000", 2024-09-20T22:05:22.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.081594064999999993 2024-09-20T22:05:22.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169119+0000", 2024-09-20T22:05:22.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4090000000000003e-06 2024-09-20T22:05:22.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169172+0000", 2024-09-20T22:05:22.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2428e-05 2024-09-20T22:05:22.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169317+0000", 2024-09-20T22:05:22.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014533100000000001 2024-09-20T22:05:22.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:05:22.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169450+0000", 2024-09-20T22:05:22.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013295900000000001 2024-09-20T22:05:22.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:05:22.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.175056+0000", 2024-09-20T22:05:22.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0056053780000000003 2024-09-20T22:05:22.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:05:22.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.176168+0000", 2024-09-20T22:05:22.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001112359 2024-09-20T22:05:22.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:05:22.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.176378+0000", 2024-09-20T22:05:22.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021008100000000001 2024-09-20T22:05:22.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:05:22.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.176434+0000", 2024-09-20T22:05:22.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5497000000000001e-05 2024-09-20T22:05:22.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.176478+0000", 2024-09-20T22:05:22.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4286999999999998e-05 2024-09-20T22:05:22.947 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.947 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.947 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.947 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop_reply(client.24667.0:22528 7.9 e88/80 ondisk, result = 0)", 2024-09-20T22:05:22.948 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:06.088205+0000", 2024-09-20T22:05:22.948 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.726042805999995, 2024-09-20T22:05:22.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.080939022999999999, 2024-09-20T22:05:22.948 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.948 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.948 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:05:22.948 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.088205+0000", 2024-09-20T22:05:22.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.088205+0000", 2024-09-20T22:05:22.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.088206+0000", 2024-09-20T22:05:22.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9200000000000001e-07 2024-09-20T22:05:22.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.088210+0000", 2024-09-20T22:05:22.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4220000000000002e-06 2024-09-20T22:05:22.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169049+0000", 2024-09-20T22:05:22.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.080839024999999995 2024-09-20T22:05:22.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169056+0000", 2024-09-20T22:05:22.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8430000000000004e-06 2024-09-20T22:05:22.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169123+0000", 2024-09-20T22:05:22.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7315000000000002e-05 2024-09-20T22:05:22.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169131+0000", 2024-09-20T22:05:22.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5120000000000002e-06 2024-09-20T22:05:22.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.169144+0000", 2024-09-20T22:05:22.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3314e-05 2024-09-20T22:05:22.953 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.953 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.953 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.953 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:22536 7.5 e88/80 7:a6f0841e:::benchmark_data_smithi060_78800_object22535:head v 88'1422, mlcod=88'1417)", 2024-09-20T22:05:22.953 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:06.088688+0000", 2024-09-20T22:05:22.953 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.725560267000006, 2024-09-20T22:05:22.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085553831999999996, 2024-09-20T22:05:22.954 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.954 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.954 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:05:22.954 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.088688+0000", 2024-09-20T22:05:22.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.088688+0000", 2024-09-20T22:05:22.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.088688+0000", 2024-09-20T22:05:22.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0699999999999997e-07 2024-09-20T22:05:22.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.088795+0000", 2024-09-20T22:05:22.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000107025 2024-09-20T22:05:22.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.168773+0000", 2024-09-20T22:05:22.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.079977802000000001 2024-09-20T22:05:22.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.168783+0000", 2024-09-20T22:05:22.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9420000000000007e-06 2024-09-20T22:05:22.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.171561+0000", 2024-09-20T22:05:22.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0027779020000000001 2024-09-20T22:05:22.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.171581+0000", 2024-09-20T22:05:22.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9692000000000001e-05 2024-09-20T22:05:22.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:05:22.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.173522+0000", 2024-09-20T22:05:22.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001941688 2024-09-20T22:05:22.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:06.174242+0000", 2024-09-20T22:05:22.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00071917400000000005 2024-09-20T22:05:22.957 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.957 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.957 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.958 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:46026 7.6 7:628f6b6e:::benchmark_data_smithi060_78800_object46025:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T22:05:22.958 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:19.388760+0000", 2024-09-20T22:05:22.958 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 63.425488268999999, 2024-09-20T22:05:22.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087538124999999994, 2024-09-20T22:05:22.959 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.959 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.959 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:05:22.959 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:05:22.959 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:05:22.959 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:22.959 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 46026 2024-09-20T22:05:22.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.959 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.388760+0000", 2024-09-20T22:05:22.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.388760+0000", 2024-09-20T22:05:22.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.388761+0000", 2024-09-20T22:05:22.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7599999999999996e-07 2024-09-20T22:05:22.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.388784+0000", 2024-09-20T22:05:22.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3683e-05 2024-09-20T22:05:22.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.388793+0000", 2024-09-20T22:05:22.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3149999999999992e-06 2024-09-20T22:05:22.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.388798+0000", 2024-09-20T22:05:22.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1769999999999997e-06 2024-09-20T22:05:22.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.391458+0000", 2024-09-20T22:05:22.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002660216 2024-09-20T22:05:22.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.391579+0000", 2024-09-20T22:05:22.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121074 2024-09-20T22:05:22.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:05:22.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.391701+0000", 2024-09-20T22:05:22.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121506 2024-09-20T22:05:22.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:05:22.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.394010+0000", 2024-09-20T22:05:22.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023093530000000001 2024-09-20T22:05:22.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:05:22.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.469549+0000", 2024-09-20T22:05:22.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.075539088000000004 2024-09-20T22:05:22.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:05:22.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.475924+0000", 2024-09-20T22:05:22.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0063748880000000004 2024-09-20T22:05:22.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:05:22.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.476110+0000", 2024-09-20T22:05:22.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000186434 2024-09-20T22:05:22.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.476298+0000", 2024-09-20T22:05:22.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000187515 2024-09-20T22:05:22.964 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.964 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.964 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.964 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:46032 7.e e93/80 7:7210d5d4:::benchmark_data_smithi060_78800_object46031:head v 93'2819, mlcod=93'2817)", 2024-09-20T22:05:22.965 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:19.390807+0000", 2024-09-20T22:05:22.965 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 63.423440604, 2024-09-20T22:05:22.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.078934171999999997, 2024-09-20T22:05:22.965 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.965 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.965 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:05:22.965 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.390807+0000", 2024-09-20T22:05:22.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.390807+0000", 2024-09-20T22:05:22.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.390808+0000", 2024-09-20T22:05:22.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6400000000000001e-07 2024-09-20T22:05:22.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.390831+0000", 2024-09-20T22:05:22.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2299000000000001e-05 2024-09-20T22:05:22.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.390842+0000", 2024-09-20T22:05:22.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1396000000000001e-05 2024-09-20T22:05:22.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.390849+0000", 2024-09-20T22:05:22.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6800000000000004e-06 2024-09-20T22:05:22.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.393850+0000", 2024-09-20T22:05:22.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003001157 2024-09-20T22:05:22.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.393869+0000", 2024-09-20T22:05:22.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9500999999999998e-05 2024-09-20T22:05:22.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:05:22.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.469687+0000", 2024-09-20T22:05:22.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.075818120000000003 2024-09-20T22:05:22.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.469742+0000", 2024-09-20T22:05:22.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4154999999999999e-05 2024-09-20T22:05:22.969 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.969 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.969 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.969 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:46036 7.7 e93/80 7:e5f37aca:::benchmark_data_smithi060_78800_object46035:head v 93'2881, mlcod=93'2879)", 2024-09-20T22:05:22.969 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:19.394607+0000", 2024-09-20T22:05:22.969 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 63.419641335000001, 2024-09-20T22:05:22.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.077262500999999997, 2024-09-20T22:05:22.969 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.969 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.969 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:05:22.969 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.394607+0000", 2024-09-20T22:05:22.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.394607+0000", 2024-09-20T22:05:22.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.394607+0000", 2024-09-20T22:05:22.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7299999999999995e-07 2024-09-20T22:05:22.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.394725+0000", 2024-09-20T22:05:22.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117297 2024-09-20T22:05:22.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.467793+0000", 2024-09-20T22:05:22.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.073068316999999994 2024-09-20T22:05:22.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.467804+0000", 2024-09-20T22:05:22.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0807e-05 2024-09-20T22:05:22.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.469612+0000", 2024-09-20T22:05:22.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001808701 2024-09-20T22:05:22.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.469641+0000", 2024-09-20T22:05:22.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8223999999999999e-05 2024-09-20T22:05:22.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:05:22.973 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:05:22.973 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:05:22.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.471837+0000", 2024-09-20T22:05:22.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021961820000000001 2024-09-20T22:05:22.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:19.471869+0000", 2024-09-20T22:05:22.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2299999999999999e-05 2024-09-20T22:05:22.974 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.974 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.974 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.975 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47812 7.6 7:61c9ee9c:::benchmark_data_smithi060_78800_object47811:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:05:22.975 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.352643+0000", 2024-09-20T22:05:22.975 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.461604481000002, 2024-09-20T22:05:22.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.077802755000000001, 2024-09-20T22:05:22.975 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.975 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.975 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:05:22.975 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:05:22.975 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:05:22.975 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:22.975 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47812 2024-09-20T22:05:22.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.975 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352643+0000", 2024-09-20T22:05:22.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352643+0000", 2024-09-20T22:05:22.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352645+0000", 2024-09-20T22:05:22.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.471e-06 2024-09-20T22:05:22.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352667+0000", 2024-09-20T22:05:22.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1673000000000001e-05 2024-09-20T22:05:22.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352679+0000", 2024-09-20T22:05:22.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2488e-05 2024-09-20T22:05:22.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352686+0000", 2024-09-20T22:05:22.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.391e-06 2024-09-20T22:05:22.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352734+0000", 2024-09-20T22:05:22.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7576e-05 2024-09-20T22:05:22.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352848+0000", 2024-09-20T22:05:22.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113761 2024-09-20T22:05:22.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:05:22.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352973+0000", 2024-09-20T22:05:22.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012566300000000001 2024-09-20T22:05:22.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:05:22.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.354504+0000", 2024-09-20T22:05:22.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0015306479999999999 2024-09-20T22:05:22.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:05:22.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.354684+0000", 2024-09-20T22:05:22.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017940799999999999 2024-09-20T22:05:22.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:05:22.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.430357+0000", 2024-09-20T22:05:22.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.075673613000000001 2024-09-20T22:05:22.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:05:22.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.430405+0000", 2024-09-20T22:05:22.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7840999999999998e-05 2024-09-20T22:05:22.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.430446+0000", 2024-09-20T22:05:22.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1221999999999999e-05 2024-09-20T22:05:22.980 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.980 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.980 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.981 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:98068 7.f 7:f619711f:::benchmark_data_smithi060_78800_object98067:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-20T22:05:22.981 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:56.334751+0000", 2024-09-20T22:05:22.981 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 26.479496495999999, 2024-09-20T22:05:22.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.077434537999999997, 2024-09-20T22:05:22.981 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.981 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.981 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:05:22.981 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:05:22.981 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:05:22.981 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:22.981 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 98068 2024-09-20T22:05:22.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.981 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.334751+0000", 2024-09-20T22:05:22.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.334751+0000", 2024-09-20T22:05:22.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.334753+0000", 2024-09-20T22:05:22.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8479999999999999e-06 2024-09-20T22:05:22.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.334782+0000", 2024-09-20T22:05:22.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9002000000000001e-05 2024-09-20T22:05:22.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.334801+0000", 2024-09-20T22:05:22.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8309e-05 2024-09-20T22:05:22.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.334812+0000", 2024-09-20T22:05:22.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1240000000000001e-05 2024-09-20T22:05:22.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.334876+0000", 2024-09-20T22:05:22.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4311000000000001e-05 2024-09-20T22:05:22.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op must wait for map", 2024-09-20T22:05:22.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.334882+0000", 2024-09-20T22:05:22.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3600000000000004e-06 2024-09-20T22:05:22.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.336243+0000", 2024-09-20T22:05:22.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0013616990000000001 2024-09-20T22:05:22.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.336428+0000", 2024-09-20T22:05:22.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018430899999999999 2024-09-20T22:05:22.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:05:22.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.336617+0000", 2024-09-20T22:05:22.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018948899999999999 2024-09-20T22:05:22.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:05:22.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.340155+0000", 2024-09-20T22:05:22.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0035374920000000002 2024-09-20T22:05:22.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:05:22.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.340700+0000", 2024-09-20T22:05:22.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00054513700000000001 2024-09-20T22:05:22.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:05:22.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.412026+0000", 2024-09-20T22:05:22.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.071326652000000004 2024-09-20T22:05:22.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:05:22.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.412135+0000", 2024-09-20T22:05:22.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108622 2024-09-20T22:05:22.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.412186+0000", 2024-09-20T22:05:22.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1068e-05 2024-09-20T22:05:22.987 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.987 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.987 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.987 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:98535 7.f 7:fc9e5e4d:::benchmark_data_smithi060_78800_object98534:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-20T22:05:22.987 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:56.763706+0000", 2024-09-20T22:05:22.988 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 26.050542103000002, 2024-09-20T22:05:22.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.077044612999999998, 2024-09-20T22:05:22.988 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:22.988 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:22.988 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:05:22.988 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:05:22.988 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:05:22.988 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:22.988 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 98535 2024-09-20T22:05:22.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.988 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:22.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:22.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.763706+0000", 2024-09-20T22:05:22.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:22.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.763706+0000", 2024-09-20T22:05:22.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:22.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:22.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.763707+0000", 2024-09-20T22:05:22.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4099999999999997e-07 2024-09-20T22:05:22.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:22.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.763733+0000", 2024-09-20T22:05:22.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5891e-05 2024-09-20T22:05:22.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:22.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.763743+0000", 2024-09-20T22:05:22.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9369999999999999e-06 2024-09-20T22:05:22.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:22.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.763749+0000", 2024-09-20T22:05:22.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.314e-06 2024-09-20T22:05:22.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:22.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.763965+0000", 2024-09-20T22:05:22.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021567299999999999 2024-09-20T22:05:22.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:22.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.764164+0000", 2024-09-20T22:05:22.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019974200000000001 2024-09-20T22:05:22.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:05:22.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.764334+0000", 2024-09-20T22:05:22.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000169779 2024-09-20T22:05:22.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:05:22.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.770515+0000", 2024-09-20T22:05:22.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0061809739999999997 2024-09-20T22:05:22.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:05:22.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.771259+0000", 2024-09-20T22:05:22.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00074385 2024-09-20T22:05:22.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:05:22.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.839486+0000", 2024-09-20T22:05:22.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.068227077999999997 2024-09-20T22:05:22.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:05:22.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.840696+0000", 2024-09-20T22:05:22.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012100640000000001 2024-09-20T22:05:22.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:22.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:22.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:05:22.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:56.840750+0000", 2024-09-20T22:05:22.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4469999999999999e-05 2024-09-20T22:05:22.993 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.993 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.993 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.993 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:22.993 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:22.994 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:05:22.994 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:05:22.994 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:05:22.995 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:05:22.995 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:05:22.996 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:05:22.996 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:05:22.998 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:22.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:22.999 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:23.058 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:22 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:05:23.058 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:22 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/351510401' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:23.059 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:22 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:05:23.059 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:22 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/351510401' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:23.092 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3, 4, 5, 6, 7] 2024-09-20T22:05:23.092 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:05:23.092 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-20T22:05:23.092 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-20T22:05:23.092 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-20T22:05:23.092 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 config set filestore_inject_stall 3 2024-09-20T22:05:23.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:23.137 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:23.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:23.138 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:23.239 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:23.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:22 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:05:23.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:22 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/351510401' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:23.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:23.283 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:23.283 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:23.297 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 86 16 129150 129134 93.8162 62.6875 0.00900304 0.0106545 2024-09-20T22:05:23.384 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:23.425 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:23.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:23.426 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:23.481 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:05:23.527 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:23.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:23.561 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:23.662 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:23.693 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:23.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:23.695 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:23.724 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:05:23.795 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:23.828 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:23.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:23.829 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:23.890 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:23 smithi060 ceph-mon[29694]: 3.5 scrub starts 2024-09-20T22:05:23.930 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:23.957 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:23.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:23.958 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:24.059 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:24.092 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:24.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:24.093 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:24.125 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:23 smithi067 ceph-mon[33511]: 3.5 scrub starts 2024-09-20T22:05:24.125 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:23 smithi067 ceph-mon[33511]: 3.5 scrub ok 2024-09-20T22:05:24.125 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:23 smithi067 ceph-mon[33511]: 3.15 scrub starts 2024-09-20T22:05:24.125 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:23 smithi067 ceph-mon[33511]: 3.15 scrub ok 2024-09-20T22:05:24.125 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:23 smithi067 ceph-mon[33511]: pgmap v102: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.7 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 89 MiB/s wr, 1.43k op/s; 47294/379299 objects degraded (12.469%); 7843/379299 objects misplaced (2.068%) 2024-09-20T22:05:24.126 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:23 smithi067 ceph-mon[33511]: 7.3 scrub starts 2024-09-20T22:05:24.154 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:05:24.154 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-20T22:05:24.154 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:05:24.194 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:24.223 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:24.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:24.224 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:24.224 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:23 smithi060 ceph-mon[29694]: 3.5 scrub ok 2024-09-20T22:05:24.225 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:23 smithi060 ceph-mon[29694]: 3.15 scrub starts 2024-09-20T22:05:24.225 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:23 smithi060 ceph-mon[29694]: 3.15 scrub ok 2024-09-20T22:05:24.225 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:23 smithi060 ceph-mon[29694]: pgmap v102: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.7 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 89 MiB/s wr, 1.43k op/s; 47294/379299 objects degraded (12.469%); 7843/379299 objects misplaced (2.068%) 2024-09-20T22:05:24.225 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:23 smithi060 ceph-mon[29694]: 7.3 scrub starts 2024-09-20T22:05:24.225 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:23 smithi060 ceph-mon[40290]: 3.5 scrub starts 2024-09-20T22:05:24.225 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:23 smithi060 ceph-mon[40290]: 3.5 scrub ok 2024-09-20T22:05:24.225 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:23 smithi060 ceph-mon[40290]: 3.15 scrub starts 2024-09-20T22:05:24.225 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:23 smithi060 ceph-mon[40290]: 3.15 scrub ok 2024-09-20T22:05:24.225 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:23 smithi060 ceph-mon[40290]: pgmap v102: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.7 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 89 MiB/s wr, 1.43k op/s; 47294/379299 objects degraded (12.469%); 7843/379299 objects misplaced (2.068%) 2024-09-20T22:05:24.225 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:23 smithi060 ceph-mon[40290]: 7.3 scrub starts 2024-09-20T22:05:24.278 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:05:24.279 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:05:24.298 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 87 16 130170 130154 93.47 63.75 0.0215442 0.0106924 2024-09-20T22:05:24.324 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:24.370 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:24.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:24.371 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:24.472 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:24.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:24.521 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:24.521 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:24.622 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:24.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:24.657 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:24.657 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:24.758 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:24.789 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T22:05:24.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:24.810 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:24.810 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:24.859 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:24.911 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:24.935 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:24.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:24.936 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:25.037 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:25.095 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:25.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:25.096 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:25.197 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:25.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:25.249 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:25.250 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:25.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:05:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:05:25.302 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 88 16 131175 131159 93.1193 62.8125 0.0254976 0.0107337 2024-09-20T22:05:25.350 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:25.379 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:25.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:25.380 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:25.481 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:25.514 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:25.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:25.516 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:25.616 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:25.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:25.666 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:25.666 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:25.766 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:25.799 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:25.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:25.800 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:25.900 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:25.926 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:25.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:25.927 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:26.028 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:26.055 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:26.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:26.056 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:26.157 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:26.184 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:26.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:26.185 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:26.202 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:26.279 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:05:26.285 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:26.301 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 89 16 132136 132120 92.7479 60.0625 0.0302194 0.0107771 2024-09-20T22:05:26.315 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:26.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:26.316 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:26.416 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:26.472 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:26.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:26.473 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:26.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:26 smithi067 ceph-mon[33511]: 4.3 scrub starts 2024-09-20T22:05:26.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:26 smithi067 ceph-mon[33511]: 4.3 scrub ok 2024-09-20T22:05:26.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:26 smithi067 ceph-mon[33511]: 7.e scrub starts 2024-09-20T22:05:26.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:26 smithi067 ceph-mon[33511]: pgmap v103: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.7 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 73 MiB/s wr, 1.17k op/s; 47294/379299 objects degraded (12.469%); 7843/379299 objects misplaced (2.068%) 2024-09-20T22:05:26.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:26 smithi067 ceph-mon[33511]: 7.7 scrub starts 2024-09-20T22:05:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:26 smithi060 ceph-mon[29694]: 4.3 scrub starts 2024-09-20T22:05:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:26 smithi060 ceph-mon[29694]: 4.3 scrub ok 2024-09-20T22:05:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:26 smithi060 ceph-mon[29694]: 7.e scrub starts 2024-09-20T22:05:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:26 smithi060 ceph-mon[29694]: pgmap v103: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.7 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 73 MiB/s wr, 1.17k op/s; 47294/379299 objects degraded (12.469%); 7843/379299 objects misplaced (2.068%) 2024-09-20T22:05:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:26 smithi060 ceph-mon[29694]: 7.7 scrub starts 2024-09-20T22:05:26.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:26 smithi060 ceph-mon[40290]: 4.3 scrub starts 2024-09-20T22:05:26.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:26 smithi060 ceph-mon[40290]: 4.3 scrub ok 2024-09-20T22:05:26.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:26 smithi060 ceph-mon[40290]: 7.e scrub starts 2024-09-20T22:05:26.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:26 smithi060 ceph-mon[40290]: pgmap v103: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.7 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 1023 B/s rd, 73 MiB/s wr, 1.17k op/s; 47294/379299 objects degraded (12.469%); 7843/379299 objects misplaced (2.068%) 2024-09-20T22:05:26.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:26 smithi060 ceph-mon[40290]: 7.7 scrub starts 2024-09-20T22:05:26.573 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:26.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:26.624 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:26.625 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:26.725 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:26.754 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:26.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:26.755 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:26.856 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:26.912 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:26.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:26.913 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:27.014 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:27.047 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:27.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:27.048 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:27.148 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:05:27.148 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:05:27.148 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.148 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "rep_scrubmap(7.3 e95 from shard 4)", 2024-09-20T22:05:27.148 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:27.141130+0000", 2024-09-20T22:05:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.004728678, 2024-09-20T22:05:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0047512609999999997, 2024-09-20T22:05:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:05:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141130+0000", 2024-09-20T22:05:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.149 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141130+0000", 2024-09-20T22:05:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141131+0000", 2024-09-20T22:05:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3700000000000001e-07 2024-09-20T22:05:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141162+0000", 2024-09-20T22:05:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0942999999999998e-05 2024-09-20T22:05:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.150 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141170+0000", 2024-09-20T22:05:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1850000000000004e-06 2024-09-20T22:05:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141176+0000", 2024-09-20T22:05:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3389999999999998e-06 2024-09-20T22:05:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:132927 7.e 7:700f17f5:::benchmark_data_smithi060_78800_object132926:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-20T22:05:27.151 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:27.141210+0000", 2024-09-20T22:05:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0046494789999999998, 2024-09-20T22:05:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0047273710000000002, 2024-09-20T22:05:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:05:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:05:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:05:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 132927 2024-09-20T22:05:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:27.152 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141210+0000", 2024-09-20T22:05:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141210+0000", 2024-09-20T22:05:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141211+0000", 2024-09-20T22:05:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7549999999999999e-06 2024-09-20T22:05:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:27.153 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141238+0000", 2024-09-20T22:05:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6291999999999998e-05 2024-09-20T22:05:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141249+0000", 2024-09-20T22:05:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1747e-05 2024-09-20T22:05:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141256+0000", 2024-09-20T22:05:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6930000000000001e-06 2024-09-20T22:05:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141315+0000", 2024-09-20T22:05:27.154 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9327000000000003e-05 2024-09-20T22:05:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141483+0000", 2024-09-20T22:05:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016788099999999999 2024-09-20T22:05:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,5", 2024-09-20T22:05:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141751+0000", 2024-09-20T22:05:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026740800000000001 2024-09-20T22:05:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.155 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:132929 7.8 7:1d591aba:::benchmark_data_smithi060_78800_object132928:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e111)", 2024-09-20T22:05:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:27.141588+0000", 2024-09-20T22:05:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0042711579999999997, 2024-09-20T22:05:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0044185240000000001, 2024-09-20T22:05:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:05:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:05:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:05:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 132929 2024-09-20T22:05:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.156 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141588+0000", 2024-09-20T22:05:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141588+0000", 2024-09-20T22:05:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141589+0000", 2024-09-20T22:05:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0599999999999999e-07 2024-09-20T22:05:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.157 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141612+0000", 2024-09-20T22:05:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.338e-05 2024-09-20T22:05:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141627+0000", 2024-09-20T22:05:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4980000000000001e-05 2024-09-20T22:05:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141634+0000", 2024-09-20T22:05:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6599999999999998e-06 2024-09-20T22:05:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.158 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141689+0000", 2024-09-20T22:05:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5025000000000003e-05 2024-09-20T22:05:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141805+0000", 2024-09-20T22:05:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011566900000000001 2024-09-20T22:05:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4,7", 2024-09-20T22:05:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.141935+0000", 2024-09-20T22:05:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013018900000000001 2024-09-20T22:05:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:27.159 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:132928 7.5 e111/80 7:a7da95d0:::benchmark_data_smithi060_78800_object132927:head v 111'8332, mlcod=111'8329)", 2024-09-20T22:05:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:27.144436+0000", 2024-09-20T22:05:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00142348, 2024-09-20T22:05:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001631181, 2024-09-20T22:05:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:05:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.144436+0000", 2024-09-20T22:05:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:27.160 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.144436+0000", 2024-09-20T22:05:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.144437+0000", 2024-09-20T22:05:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6500000000000008e-07 2024-09-20T22:05:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.144480+0000", 2024-09-20T22:05:27.161 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2976999999999999e-05 2024-09-20T22:05:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.144494+0000", 2024-09-20T22:05:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4113e-05 2024-09-20T22:05:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.144502+0000", 2024-09-20T22:05:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0169999999999996e-06 2024-09-20T22:05:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.144619+0000", 2024-09-20T22:05:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011772799999999999 2024-09-20T22:05:27.162 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:27.163 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:27.163 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:27.163 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:27.144639+0000", 2024-09-20T22:05:27.163 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9723000000000001e-05 2024-09-20T22:05:27.163 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:27.163 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:27.163 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:27.163 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:27.163 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:05:27.163 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 4 2024-09-20T22:05:27.163 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:05:27.163 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:27.169 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:05:27.212 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:27.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:27.213 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:27.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:27 smithi060 ceph-mon[29694]: 3.9 scrub starts 2024-09-20T22:05:27.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:27 smithi060 ceph-mon[29694]: 3.9 scrub ok 2024-09-20T22:05:27.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:27 smithi060 ceph-mon[29694]: 5.11 scrub starts 2024-09-20T22:05:27.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:27 smithi060 ceph-mon[29694]: 5.11 scrub ok 2024-09-20T22:05:27.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:27 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3566399718' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:05:27.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:27 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:05:27.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:27 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 47294/379299 objects degraded (12.469%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:27.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:27 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:05:27.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:27 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:05:27.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:27 smithi060 ceph-mon[40290]: 3.9 scrub starts 2024-09-20T22:05:27.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:27 smithi060 ceph-mon[40290]: 3.9 scrub ok 2024-09-20T22:05:27.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:27 smithi060 ceph-mon[40290]: 5.11 scrub starts 2024-09-20T22:05:27.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:27 smithi060 ceph-mon[40290]: 5.11 scrub ok 2024-09-20T22:05:27.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:27 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3566399718' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:05:27.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:27 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:05:27.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:27 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 47294/379299 objects degraded (12.469%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:27.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:27 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:05:27.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:27 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:05:27.301 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Total time run: 90.0106 2024-09-20T22:05:27.301 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Total writes made: 133030 2024-09-20T22:05:27.301 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Write size: 65536 2024-09-20T22:05:27.301 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Object size: 65536 2024-09-20T22:05:27.301 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Bandwidth (MB/sec): 92.371 2024-09-20T22:05:27.301 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Stddev Bandwidth: 30.2196 2024-09-20T22:05:27.301 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Max bandwidth (MB/sec): 199 2024-09-20T22:05:27.301 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Min bandwidth (MB/sec): 12.4375 2024-09-20T22:05:27.301 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Average IOPS: 1477 2024-09-20T22:05:27.301 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Stddev IOPS: 483.513 2024-09-20T22:05:27.301 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Max IOPS: 3184 2024-09-20T22:05:27.301 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Min IOPS: 199 2024-09-20T22:05:27.301 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Average Latency(s): 0.0108214 2024-09-20T22:05:27.302 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Stddev Latency(s): 0.0145405 2024-09-20T22:05:27.302 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Max latency(s): 1.08039 2024-09-20T22:05:27.302 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Min latency(s): 0.000984708 2024-09-20T22:05:27.309 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Cleaning up (deleting benchmark objects) 2024-09-20T22:05:27.314 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:27.355 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:27.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:27.356 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:27.457 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:27.485 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:27.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:27.486 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:27.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:27 smithi067 ceph-mon[33511]: 3.9 scrub starts 2024-09-20T22:05:27.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:27 smithi067 ceph-mon[33511]: 3.9 scrub ok 2024-09-20T22:05:27.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:27 smithi067 ceph-mon[33511]: 5.11 scrub starts 2024-09-20T22:05:27.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:27 smithi067 ceph-mon[33511]: 5.11 scrub ok 2024-09-20T22:05:27.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:27 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3566399718' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:05:27.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:27 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:05:27.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:27 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 47294/379299 objects degraded (12.469%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:27.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:27 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:05:27.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:27 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:05:27.586 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:27.612 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:27.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:27.613 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:27.714 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:27.741 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:27.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:27.742 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:27.842 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:27.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:27.885 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:27.886 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:27.986 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:28.012 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:28.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:28.013 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:28.114 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:28.151 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:28.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:28.152 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:28.253 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:28.281 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:28.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:28.282 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:28.383 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:28.416 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:28.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:28.417 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:28.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[29694]: 4.c scrub starts 2024-09-20T22:05:28.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[29694]: 4.c scrub ok 2024-09-20T22:05:28.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[29694]: pgmap v104: 148 pgs: 2 active+clean+scrubbing, 1 active+clean+remapped+scrubbing, 7 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 80 active+clean; 8.0 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.13k op/s; 49279/395220 objects degraded (12.469%); 8220/395220 objects misplaced (2.080%) 2024-09-20T22:05:28.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:05:28.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:05:28.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[29694]: osdmap e112: 8 total, 7 up, 8 in 2024-09-20T22:05:28.498 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[29694]: 7.3 scrub ok 2024-09-20T22:05:28.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:28.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:28.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:05:28.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:05:28.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[29694]: 3.b scrub starts 2024-09-20T22:05:28.499 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[29694]: 3.b scrub ok 2024-09-20T22:05:28.499 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[40290]: 4.c scrub starts 2024-09-20T22:05:28.499 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[40290]: 4.c scrub ok 2024-09-20T22:05:28.499 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[40290]: pgmap v104: 148 pgs: 2 active+clean+scrubbing, 1 active+clean+remapped+scrubbing, 7 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 80 active+clean; 8.0 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.13k op/s; 49279/395220 objects degraded (12.469%); 8220/395220 objects misplaced (2.080%) 2024-09-20T22:05:28.499 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:05:28.499 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:05:28.500 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[40290]: osdmap e112: 8 total, 7 up, 8 in 2024-09-20T22:05:28.500 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[40290]: 7.3 scrub ok 2024-09-20T22:05:28.500 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:28.500 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:28.500 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:05:28.500 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:05:28.500 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[40290]: 3.b scrub starts 2024-09-20T22:05:28.500 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:28 smithi060 ceph-mon[40290]: 3.b scrub ok 2024-09-20T22:05:28.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:28 smithi067 ceph-mon[33511]: 4.c scrub starts 2024-09-20T22:05:28.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:28 smithi067 ceph-mon[33511]: 4.c scrub ok 2024-09-20T22:05:28.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:28 smithi067 ceph-mon[33511]: pgmap v104: 148 pgs: 2 active+clean+scrubbing, 1 active+clean+remapped+scrubbing, 7 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 80 active+clean; 8.0 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1023 B/s rd, 71 MiB/s wr, 1.13k op/s; 49279/395220 objects degraded (12.469%); 8220/395220 objects misplaced (2.080%) 2024-09-20T22:05:28.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:28 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:05:28.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:28 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:05:28.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:28 smithi067 ceph-mon[33511]: osdmap e112: 8 total, 7 up, 8 in 2024-09-20T22:05:28.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:28 smithi067 ceph-mon[33511]: 7.3 scrub ok 2024-09-20T22:05:28.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:28 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:28.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:28 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:28.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:28 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:05:28.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:28 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:05:28.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:28 smithi067 ceph-mon[33511]: 3.b scrub starts 2024-09-20T22:05:28.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:28 smithi067 ceph-mon[33511]: 3.b scrub ok 2024-09-20T22:05:28.518 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:28.546 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:28.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:28.547 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:28.648 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:28.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:28.686 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:28.786 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:28.823 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:28.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:28.824 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:28.925 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:28.953 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:28.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:28.954 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:29.055 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:29.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:29.092 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:29.092 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:29.173 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T22:05:29.193 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:29.257 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:29.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:29.259 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:29.359 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:29.391 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:29.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:29.392 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:29.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:29.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:29 smithi067 ceph-mon[33511]: 7.7 scrub ok 2024-09-20T22:05:29.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:29 smithi067 ceph-mon[33511]: 6.7 deep-scrub starts 2024-09-20T22:05:29.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:29 smithi067 ceph-mon[33511]: 6.7 deep-scrub ok 2024-09-20T22:05:29.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:29 smithi067 ceph-mon[33511]: 7.e scrub ok 2024-09-20T22:05:29.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:29 smithi067 ceph-mon[33511]: 5.1c deep-scrub starts 2024-09-20T22:05:29.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:29 smithi067 ceph-mon[33511]: 5.1c deep-scrub ok 2024-09-20T22:05:29.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:29 smithi067 ceph-mon[33511]: osdmap e113: 8 total, 7 up, 8 in 2024-09-20T22:05:29.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:29 smithi067 ceph-mon[33511]: 6.5 scrub starts 2024-09-20T22:05:29.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:29 smithi067 ceph-mon[33511]: 6.5 scrub ok 2024-09-20T22:05:29.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:29.527 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:29.528 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:29 smithi060 ceph-mon[29694]: 7.7 scrub ok 2024-09-20T22:05:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:29 smithi060 ceph-mon[29694]: 6.7 deep-scrub starts 2024-09-20T22:05:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:29 smithi060 ceph-mon[29694]: 6.7 deep-scrub ok 2024-09-20T22:05:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:29 smithi060 ceph-mon[29694]: 7.e scrub ok 2024-09-20T22:05:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:29 smithi060 ceph-mon[29694]: 5.1c deep-scrub starts 2024-09-20T22:05:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:29 smithi060 ceph-mon[29694]: 5.1c deep-scrub ok 2024-09-20T22:05:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:29 smithi060 ceph-mon[29694]: osdmap e113: 8 total, 7 up, 8 in 2024-09-20T22:05:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:29 smithi060 ceph-mon[29694]: 6.5 scrub starts 2024-09-20T22:05:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:29 smithi060 ceph-mon[29694]: 6.5 scrub ok 2024-09-20T22:05:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:29 smithi060 ceph-mon[40290]: 7.7 scrub ok 2024-09-20T22:05:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:29 smithi060 ceph-mon[40290]: 6.7 deep-scrub starts 2024-09-20T22:05:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:29 smithi060 ceph-mon[40290]: 6.7 deep-scrub ok 2024-09-20T22:05:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:29 smithi060 ceph-mon[40290]: 7.e scrub ok 2024-09-20T22:05:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:29 smithi060 ceph-mon[40290]: 5.1c deep-scrub starts 2024-09-20T22:05:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:29 smithi060 ceph-mon[40290]: 5.1c deep-scrub ok 2024-09-20T22:05:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:29 smithi060 ceph-mon[40290]: osdmap e113: 8 total, 7 up, 8 in 2024-09-20T22:05:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:29 smithi060 ceph-mon[40290]: 6.5 scrub starts 2024-09-20T22:05:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:29 smithi060 ceph-mon[40290]: 6.5 scrub ok 2024-09-20T22:05:29.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:29.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:29.684 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:29.684 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:29.705 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:29.743 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:29.785 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:29.832 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:29.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:29.834 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:29.935 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:29.965 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:29.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:29.966 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:30.067 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:30.094 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:30.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:30.095 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:30.181 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3, 4, 5, 6, 7] 2024-09-20T22:05:30.181 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:05:30.181 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-20T22:05:30.181 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-20T22:05:30.196 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:30.224 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:30.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:30.225 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:30.326 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:30.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:30.355 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:30.355 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:30.385 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:30 smithi060 ceph-mon[29694]: 6.b scrub starts 2024-09-20T22:05:30.385 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:30 smithi060 ceph-mon[29694]: 6.b scrub ok 2024-09-20T22:05:30.385 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:30 smithi060 ceph-mon[29694]: 3.1d scrub starts 2024-09-20T22:05:30.385 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:30 smithi060 ceph-mon[29694]: 3.1d scrub ok 2024-09-20T22:05:30.385 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:30 smithi060 ceph-mon[29694]: pgmap v107: 148 pgs: 2 active+clean+scrubbing, 1 active+clean+remapped+scrubbing, 7 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 80 active+clean; 8.0 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 41 MiB/s wr, 663 op/s; 49279/395220 objects degraded (12.469%); 8220/395220 objects misplaced (2.080%) 2024-09-20T22:05:30.385 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:30 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:05:30.385 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:30 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:05:30.385 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:30 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:30.386 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:30 smithi060 ceph-mon[40290]: 6.b scrub starts 2024-09-20T22:05:30.386 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:30 smithi060 ceph-mon[40290]: 6.b scrub ok 2024-09-20T22:05:30.386 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:30 smithi060 ceph-mon[40290]: 3.1d scrub starts 2024-09-20T22:05:30.386 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:30 smithi060 ceph-mon[40290]: 3.1d scrub ok 2024-09-20T22:05:30.386 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:30 smithi060 ceph-mon[40290]: pgmap v107: 148 pgs: 2 active+clean+scrubbing, 1 active+clean+remapped+scrubbing, 7 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 80 active+clean; 8.0 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 41 MiB/s wr, 663 op/s; 49279/395220 objects degraded (12.469%); 8220/395220 objects misplaced (2.080%) 2024-09-20T22:05:30.386 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:30 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:05:30.386 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:30 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:05:30.386 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:30 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:30.456 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:30.483 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:30.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:30.484 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:30 smithi067 ceph-mon[33511]: 6.b scrub starts 2024-09-20T22:05:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:30 smithi067 ceph-mon[33511]: 6.b scrub ok 2024-09-20T22:05:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:30 smithi067 ceph-mon[33511]: 3.1d scrub starts 2024-09-20T22:05:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:30 smithi067 ceph-mon[33511]: 3.1d scrub ok 2024-09-20T22:05:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:30 smithi067 ceph-mon[33511]: pgmap v107: 148 pgs: 2 active+clean+scrubbing, 1 active+clean+remapped+scrubbing, 7 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 80 active+clean; 8.0 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 41 MiB/s wr, 663 op/s; 49279/395220 objects degraded (12.469%); 8220/395220 objects misplaced (2.080%) 2024-09-20T22:05:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:30 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:05:30.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:30 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:05:30.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:30 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:30.564 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:05:30.564 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:05:30.565 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:05:30.584 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:30.599 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:05:30.599 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:05:30.615 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:30.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:30.617 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:30.639 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:05:30.640 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:05:30.672 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:05:30.672 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:05:30.714 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:05:30.714 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:05:30.718 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:30.744 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:30.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:30.745 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:30.755 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:05:30.756 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:05:30.794 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:05:30.795 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:05:30.844 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:05:30.844 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:05:30.846 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:30.881 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:30.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:30.882 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:30.983 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:31.021 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:31.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:31.022 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:31.122 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:31.155 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:31.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:31.156 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:31.257 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:31.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:31.298 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:31.298 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:31.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:31.433 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:31.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:31.434 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:31.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:31 smithi067 ceph-mon[33511]: 6.19 deep-scrub starts 2024-09-20T22:05:31.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:31 smithi067 ceph-mon[33511]: 6.19 deep-scrub ok 2024-09-20T22:05:31.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:31 smithi067 ceph-mon[33511]: pgmap v108: 148 pgs: 2 active+clean+scrubbing, 1 active+clean+remapped+scrubbing, 7 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 80 active+clean; 8.0 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 748 B/s rd, 48 MiB/s wr, 776 op/s; 49279/395220 objects degraded (12.469%); 8220/395220 objects misplaced (2.080%) 2024-09-20T22:05:31.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:31 smithi067 ceph-mon[33511]: 6.2 scrub starts 2024-09-20T22:05:31.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:31 smithi067 ceph-mon[33511]: 6.2 scrub ok 2024-09-20T22:05:31.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:31 smithi067 ceph-mon[33511]: 5.2 scrub starts 2024-09-20T22:05:31.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:31 smithi067 ceph-mon[33511]: 5.2 scrub ok 2024-09-20T22:05:31.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:31 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/965012784' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:31.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:31 smithi067 ceph-mon[33511]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-09-20T22:05:31.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:31 smithi067 ceph-mon[33511]: 5.d scrub starts 2024-09-20T22:05:31.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:31 smithi067 ceph-mon[33511]: 5.d scrub ok 2024-09-20T22:05:31.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:31 smithi067 ceph-mon[33511]: 6.13 scrub starts 2024-09-20T22:05:31.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:31 smithi067 ceph-mon[33511]: 6.13 scrub ok 2024-09-20T22:05:31.535 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:31.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[29694]: 6.19 deep-scrub starts 2024-09-20T22:05:31.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[29694]: 6.19 deep-scrub ok 2024-09-20T22:05:31.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[29694]: pgmap v108: 148 pgs: 2 active+clean+scrubbing, 1 active+clean+remapped+scrubbing, 7 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 80 active+clean; 8.0 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 748 B/s rd, 48 MiB/s wr, 776 op/s; 49279/395220 objects degraded (12.469%); 8220/395220 objects misplaced (2.080%) 2024-09-20T22:05:31.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[29694]: 6.2 scrub starts 2024-09-20T22:05:31.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[29694]: 6.2 scrub ok 2024-09-20T22:05:31.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[29694]: 5.2 scrub starts 2024-09-20T22:05:31.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[29694]: 5.2 scrub ok 2024-09-20T22:05:31.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/965012784' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:31.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[29694]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-09-20T22:05:31.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[29694]: 5.d scrub starts 2024-09-20T22:05:31.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[29694]: 5.d scrub ok 2024-09-20T22:05:31.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[29694]: 6.13 scrub starts 2024-09-20T22:05:31.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[29694]: 6.13 scrub ok 2024-09-20T22:05:31.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[40290]: 6.19 deep-scrub starts 2024-09-20T22:05:31.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[40290]: 6.19 deep-scrub ok 2024-09-20T22:05:31.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[40290]: pgmap v108: 148 pgs: 2 active+clean+scrubbing, 1 active+clean+remapped+scrubbing, 7 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 80 active+clean; 8.0 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 748 B/s rd, 48 MiB/s wr, 776 op/s; 49279/395220 objects degraded (12.469%); 8220/395220 objects misplaced (2.080%) 2024-09-20T22:05:31.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[40290]: 6.2 scrub starts 2024-09-20T22:05:31.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[40290]: 6.2 scrub ok 2024-09-20T22:05:31.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[40290]: 5.2 scrub starts 2024-09-20T22:05:31.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[40290]: 5.2 scrub ok 2024-09-20T22:05:31.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/965012784' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:31.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[40290]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2024-09-20T22:05:31.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[40290]: 5.d scrub starts 2024-09-20T22:05:31.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[40290]: 5.d scrub ok 2024-09-20T22:05:31.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[40290]: 6.13 scrub starts 2024-09-20T22:05:31.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:31 smithi060 ceph-mon[40290]: 6.13 scrub ok 2024-09-20T22:05:31.566 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:31.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:31.567 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:31.638 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:05:31.669 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:31.698 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:31.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:31.699 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:31.800 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:31.827 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:31.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:31.828 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:31.928 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:31.976 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:31.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:31.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:32.078 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:32.108 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:32.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:32.109 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:32.154 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:05:31 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:05:32.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:32.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T22:05:32.268 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:32.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:32.271 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:32.372 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:32.402 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:32.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:32.403 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:32.503 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:32.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:32 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 49279/395220 objects degraded (12.469%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:32.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:32 smithi067 ceph-mon[33511]: 3.11 scrub starts 2024-09-20T22:05:32.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:32 smithi067 ceph-mon[33511]: 3.11 scrub ok 2024-09-20T22:05:32.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:32 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 49279/395220 objects degraded (12.469%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:32.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:32 smithi060 ceph-mon[40290]: 3.11 scrub starts 2024-09-20T22:05:32.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:32 smithi060 ceph-mon[40290]: 3.11 scrub ok 2024-09-20T22:05:32.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:32 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 49279/395220 objects degraded (12.469%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:32.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:32 smithi060 ceph-mon[29694]: 3.11 scrub starts 2024-09-20T22:05:32.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:32 smithi060 ceph-mon[29694]: 3.11 scrub ok 2024-09-20T22:05:32.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:32.554 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:32.554 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:32.655 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:32.689 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:32.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:32.690 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:32.739 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:32.789 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:05:32.789 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:05:32.791 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:32.818 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:32.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:32.819 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:32.824 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:32.919 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:32.967 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:32.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:32.968 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:33.069 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:33.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:33.117 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:33.118 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:33.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:33.253 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:33.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:33.254 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:33.355 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:33.378 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:33 smithi060 ceph-mon[29694]: 3.1b scrub starts 2024-09-20T22:05:33.378 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:33 smithi060 ceph-mon[29694]: 3.1b scrub ok 2024-09-20T22:05:33.379 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:33 smithi060 ceph-mon[29694]: 5.b scrub starts 2024-09-20T22:05:33.379 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:33 smithi060 ceph-mon[29694]: 5.b scrub ok 2024-09-20T22:05:33.379 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:33 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:05:33.379 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:33 smithi060 ceph-mon[29694]: pgmap v109: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.9 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.6 KiB/s rd, 62 MiB/s wr, 1.61k op/s; 48295/387117 objects degraded (12.476%); 8061/387117 objects misplaced (2.082%) 2024-09-20T22:05:33.379 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:33 smithi060 ceph-mon[40290]: 3.1b scrub starts 2024-09-20T22:05:33.379 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:33 smithi060 ceph-mon[40290]: 3.1b scrub ok 2024-09-20T22:05:33.379 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:33 smithi060 ceph-mon[40290]: 5.b scrub starts 2024-09-20T22:05:33.379 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:33 smithi060 ceph-mon[40290]: 5.b scrub ok 2024-09-20T22:05:33.379 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:33 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:05:33.379 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:33 smithi060 ceph-mon[40290]: pgmap v109: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.9 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.6 KiB/s rd, 62 MiB/s wr, 1.61k op/s; 48295/387117 objects degraded (12.476%); 8061/387117 objects misplaced (2.082%) 2024-09-20T22:05:33.381 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:33.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:33.382 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:33.483 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:33.510 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:33.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:33.511 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:33.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:33 smithi067 ceph-mon[33511]: 3.1b scrub starts 2024-09-20T22:05:33.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:33 smithi067 ceph-mon[33511]: 3.1b scrub ok 2024-09-20T22:05:33.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:33 smithi067 ceph-mon[33511]: 5.b scrub starts 2024-09-20T22:05:33.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:33 smithi067 ceph-mon[33511]: 5.b scrub ok 2024-09-20T22:05:33.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:33 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:05:33.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:33 smithi067 ceph-mon[33511]: pgmap v109: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.9 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.6 KiB/s rd, 62 MiB/s wr, 1.61k op/s; 48295/387117 objects degraded (12.476%); 8061/387117 objects misplaced (2.082%) 2024-09-20T22:05:33.612 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:33.638 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:33.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:33.639 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:33.740 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:33.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:33.776 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:33.777 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:33.790 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:05:33.791 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:05:33.791 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 113, 2024-09-20T22:05:33.791 INFO:teuthology.orchestra.run.smithi060.stdout: "fsid": "3df9647e-779b-11ef-baf5-efdc52797490", 2024-09-20T22:05:33.791 INFO:teuthology.orchestra.run.smithi060.stdout: "created": "2024-09-20T21:57:31.244934+0000", 2024-09-20T22:05:33.791 INFO:teuthology.orchestra.run.smithi060.stdout: "modified": "2024-09-20T22:05:28.582772+0000", 2024-09-20T22:05:33.792 INFO:teuthology.orchestra.run.smithi060.stdout: "last_up_change": "2024-09-20T22:04:15.685933+0000", 2024-09-20T22:05:33.792 INFO:teuthology.orchestra.run.smithi060.stdout: "last_in_change": "2024-09-20T22:02:21.189225+0000", 2024-09-20T22:05:33.792 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-20T22:05:33.792 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_num": 5799936, 2024-09-20T22:05:33.792 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_set": [ 2024-09-20T22:05:33.792 INFO:teuthology.orchestra.run.smithi060.stdout: "pglog_hardlimit", 2024-09-20T22:05:33.792 INFO:teuthology.orchestra.run.smithi060.stdout: "purged_snapdirs", 2024-09-20T22:05:33.792 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_deletes", 2024-09-20T22:05:33.792 INFO:teuthology.orchestra.run.smithi060.stdout: "sortbitwise" 2024-09-20T22:05:33.792 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:05:33.792 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_version": 18, 2024-09-20T22:05:33.792 INFO:teuthology.orchestra.run.smithi060.stdout: "full_ratio": 0.94999998807907104, 2024-09-20T22:05:33.792 INFO:teuthology.orchestra.run.smithi060.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-20T22:05:33.793 INFO:teuthology.orchestra.run.smithi060.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-20T22:05:33.793 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_snapshot": "", 2024-09-20T22:05:33.793 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_max": 7, 2024-09-20T22:05:33.793 INFO:teuthology.orchestra.run.smithi060.stdout: "max_osd": 8, 2024-09-20T22:05:33.793 INFO:teuthology.orchestra.run.smithi060.stdout: "require_min_compat_client": "luminous", 2024-09-20T22:05:33.793 INFO:teuthology.orchestra.run.smithi060.stdout: "min_compat_client": "luminous", 2024-09-20T22:05:33.793 INFO:teuthology.orchestra.run.smithi060.stdout: "require_osd_release": "squid", 2024-09-20T22:05:33.793 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_crimson": false, 2024-09-20T22:05:33.793 INFO:teuthology.orchestra.run.smithi060.stdout: "pools": [ 2024-09-20T22:05:33.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.793 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 1, 2024-09-20T22:05:33.793 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": ".mgr", 2024-09-20T22:05:33.793 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:00:35.131125+0000", 2024-09-20T22:05:33.793 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:05:33.794 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:05:33.794 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:05:33.794 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:05:33.794 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:05:33.794 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:05:33.794 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:05:33.794 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:05:33.794 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:05:33.794 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:05:33.794 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:05:33.794 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:05:33.794 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:05:33.794 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 1, 2024-09-20T22:05:33.795 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 1, 2024-09-20T22:05:33.795 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 1, 2024-09-20T22:05:33.795 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 1, 2024-09-20T22:05:33.795 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 1, 2024-09-20T22:05:33.795 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:05:33.795 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:05:33.795 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:05:33.795 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:05:33.795 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:05:33.795 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:05:33.795 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:05:33.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.795 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "21", 2024-09-20T22:05:33.795 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:05:33.796 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:05:33.796 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:05:33.796 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:05:33.796 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:05:33.796 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:05:33.796 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:05:33.796 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:05:33.796 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:05:33.796 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:05:33.796 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:05:33.796 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:05:33.796 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:05:33.796 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:05:33.796 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:05:33.796 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:05:33.797 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:05:33.797 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:05:33.797 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:05:33.797 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:05:33.797 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:05:33.797 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:05:33.797 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:05:33.797 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:05:33.797 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:05:33.797 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:05:33.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.797 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:05:33.797 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:05:33.797 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:05:33.797 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:05:33.798 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:05:33.798 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:05:33.798 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:05:33.798 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:05:33.798 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:05:33.798 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:05:33.798 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:05:33.798 INFO:teuthology.orchestra.run.smithi060.stdout: "options": { 2024-09-20T22:05:33.798 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_max": 32, 2024-09-20T22:05:33.798 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_min": 1 2024-09-20T22:05:33.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.798 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:05:33.798 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": {} 2024-09-20T22:05:33.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.798 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:05:33.799 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:05:33.799 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 7.8899998664855957, 2024-09-20T22:05:33.799 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 7.8899998664855957, 2024-09-20T22:05:33.799 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.37999999523162842, 2024-09-20T22:05:33.799 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 3, 2024-09-20T22:05:33.799 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 3, 2024-09-20T22:05:33.799 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:05:33.799 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:05:33.799 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:05:33.799 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.799 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 2, 2024-09-20T22:05:33.799 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "datapool", 2024-09-20T22:05:33.799 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:47.974480+0000", 2024-09-20T22:05:33.799 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 8193, 2024-09-20T22:05:33.800 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-20T22:05:33.800 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:05:33.800 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:05:33.800 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:05:33.800 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:05:33.800 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:05:33.800 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:05:33.800 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:05:33.800 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:05:33.800 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:05:33.800 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:05:33.800 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:05:33.800 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 3, 2024-09-20T22:05:33.800 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 3, 2024-09-20T22:05:33.801 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 3, 2024-09-20T22:05:33.801 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 3, 2024-09-20T22:05:33.801 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 3, 2024-09-20T22:05:33.801 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:05:33.801 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:05:33.801 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:05:33.801 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:05:33.801 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:05:33.801 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:05:33.801 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:05:33.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.801 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "66", 2024-09-20T22:05:33.801 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:05:33.801 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:05:33.802 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:05:33.802 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:05:33.802 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:05:33.802 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 2, 2024-09-20T22:05:33.802 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 66, 2024-09-20T22:05:33.802 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:05:33.802 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:05:33.802 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:05:33.802 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:05:33.802 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:05:33.802 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:05:33.802 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:05:33.803 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:05:33.803 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:05:33.803 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:05:33.803 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:05:33.803 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:05:33.803 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:05:33.803 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:05:33.803 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:05:33.803 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:05:33.803 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:05:33.803 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:05:33.803 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:05:33.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.803 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:05:33.803 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:05:33.803 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:05:33.804 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:05:33.804 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:05:33.804 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:05:33.804 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:05:33.804 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:05:33.804 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:05:33.804 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:05:33.804 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:05:33.804 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:05:33.804 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:05:33.804 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd": {} 2024-09-20T22:05:33.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.804 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:05:33.804 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:05:33.804 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2.6700000762939453, 2024-09-20T22:05:33.805 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 2.6700000762939453, 2024-09-20T22:05:33.805 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.75, 2024-09-20T22:05:33.805 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2, 2024-09-20T22:05:33.805 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 2, 2024-09-20T22:05:33.805 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:05:33.805 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:05:33.805 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:05:33.805 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.805 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 3, 2024-09-20T22:05:33.805 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": ".rgw.root", 2024-09-20T22:05:33.805 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:48.902465+0000", 2024-09-20T22:05:33.805 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:05:33.806 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:05:33.806 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:05:33.806 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:05:33.806 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:05:33.806 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:05:33.806 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:05:33.806 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:05:33.806 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:05:33.806 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:05:33.806 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:05:33.806 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:05:33.806 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:05:33.806 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:05:33.806 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:05:33.807 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:05:33.807 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:05:33.807 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:05:33.807 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:05:33.807 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:05:33.807 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:05:33.807 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:05:33.807 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:05:33.807 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:05:33.808 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:05:33.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.808 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "65", 2024-09-20T22:05:33.808 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:05:33.808 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:05:33.808 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:05:33.808 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:05:33.808 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:05:33.808 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:05:33.808 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:05:33.808 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:05:33.808 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:05:33.809 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:05:33.809 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:05:33.809 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:05:33.809 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:05:33.809 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:05:33.809 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:05:33.809 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:05:33.809 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:05:33.809 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:05:33.809 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:05:33.809 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:05:33.809 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:05:33.809 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:05:33.810 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:05:33.810 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:05:33.810 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:05:33.810 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:05:33.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.810 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:05:33.810 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:05:33.810 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:05:33.810 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:05:33.810 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:05:33.810 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:05:33.810 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:05:33.810 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:05:33.811 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:05:33.811 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:05:33.811 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:05:33.811 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:05:33.811 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:05:33.811 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:05:33.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.811 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:05:33.811 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:05:33.811 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.7400000095367432, 2024-09-20T22:05:33.811 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.7400000095367432, 2024-09-20T22:05:33.811 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T22:05:33.811 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-20T22:05:33.811 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-20T22:05:33.811 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:05:33.812 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:05:33.812 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:05:33.812 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.812 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 4, 2024-09-20T22:05:33.812 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "default.rgw.log", 2024-09-20T22:05:33.812 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:50.620057+0000", 2024-09-20T22:05:33.812 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:05:33.812 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:05:33.812 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:05:33.812 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:05:33.812 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:05:33.813 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:05:33.813 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:05:33.813 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:05:33.813 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:05:33.813 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:05:33.813 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:05:33.813 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:05:33.813 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:05:33.813 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:05:33.813 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:05:33.813 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:05:33.813 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:05:33.813 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:05:33.813 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:05:33.814 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:05:33.814 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:05:33.814 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:05:33.814 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:05:33.814 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:05:33.814 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:05:33.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.814 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "67", 2024-09-20T22:05:33.814 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:05:33.814 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:05:33.814 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:05:33.814 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:05:33.814 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:05:33.815 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:05:33.815 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:05:33.815 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:05:33.815 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:05:33.815 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:05:33.815 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:05:33.815 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:05:33.815 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:05:33.815 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:05:33.815 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:05:33.815 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:05:33.815 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:05:33.815 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:05:33.816 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:05:33.816 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:05:33.816 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:05:33.816 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:05:33.816 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:05:33.816 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:05:33.816 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:05:33.816 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:05:33.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.816 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:05:33.816 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:05:33.816 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:05:33.816 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:05:33.817 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:05:33.817 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:05:33.817 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:05:33.817 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:05:33.817 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:05:33.817 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:05:33.817 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:05:33.817 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:05:33.817 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:05:33.817 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:05:33.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.817 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:05:33.817 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:05:33.818 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2.2400000095367432, 2024-09-20T22:05:33.818 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 2.2400000095367432, 2024-09-20T22:05:33.818 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T22:05:33.818 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.9700000286102295, 2024-09-20T22:05:33.818 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.9700000286102295, 2024-09-20T22:05:33.818 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:05:33.818 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:05:33.818 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:05:33.818 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.818 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 5, 2024-09-20T22:05:33.818 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "default.rgw.control", 2024-09-20T22:05:33.818 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:52.722912+0000", 2024-09-20T22:05:33.819 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:05:33.819 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:05:33.819 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:05:33.819 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:05:33.819 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:05:33.819 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:05:33.819 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:05:33.819 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:05:33.819 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:05:33.819 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:05:33.819 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:05:33.819 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:05:33.819 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:05:33.819 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:05:33.820 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:05:33.820 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:05:33.820 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:05:33.820 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:05:33.820 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:05:33.820 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:05:33.820 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:05:33.820 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:05:33.820 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:05:33.820 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:05:33.820 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:05:33.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.820 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "69", 2024-09-20T22:05:33.820 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:05:33.821 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:05:33.821 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:05:33.821 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:05:33.821 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:05:33.821 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:05:33.821 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:05:33.821 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:05:33.821 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:05:33.821 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:05:33.821 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:05:33.821 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:05:33.821 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:05:33.821 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:05:33.821 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:05:33.821 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:05:33.822 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:05:33.822 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:05:33.822 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:05:33.822 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:05:33.822 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:05:33.822 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:05:33.822 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:05:33.822 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:05:33.822 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:05:33.822 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:05:33.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.822 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:05:33.822 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:05:33.822 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:05:33.823 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:05:33.823 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:05:33.823 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:05:33.823 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:05:33.823 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:05:33.823 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:05:33.823 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:05:33.823 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:05:33.823 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:05:33.823 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:05:33.823 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:05:33.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.823 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:05:33.823 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:05:33.824 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.7400000095367432, 2024-09-20T22:05:33.824 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.7400000095367432, 2024-09-20T22:05:33.824 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T22:05:33.824 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-20T22:05:33.824 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-20T22:05:33.824 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:05:33.824 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:05:33.824 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:05:33.824 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.824 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 6, 2024-09-20T22:05:33.824 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "default.rgw.meta", 2024-09-20T22:05:33.824 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:54.794248+0000", 2024-09-20T22:05:33.824 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:05:33.825 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:05:33.825 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:05:33.825 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:05:33.825 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:05:33.825 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:05:33.825 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:05:33.825 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:05:33.825 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:05:33.825 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:05:33.825 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:05:33.825 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:05:33.825 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:05:33.825 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:05:33.825 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:05:33.825 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:05:33.825 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:05:33.826 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:05:33.826 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:05:33.826 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:05:33.826 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:05:33.826 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:05:33.826 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:05:33.826 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:05:33.826 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:05:33.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.826 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "71", 2024-09-20T22:05:33.826 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:05:33.826 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:05:33.826 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:05:33.827 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:05:33.827 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:05:33.827 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:05:33.827 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:05:33.827 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:05:33.827 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:05:33.827 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:05:33.827 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:05:33.827 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:05:33.827 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:05:33.827 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:05:33.827 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:05:33.827 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:05:33.827 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:05:33.828 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:05:33.828 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:05:33.828 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:05:33.828 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:05:33.828 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:05:33.828 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:05:33.828 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:05:33.828 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:05:33.828 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:05:33.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.828 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:05:33.828 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:05:33.828 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:05:33.828 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:05:33.828 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:05:33.828 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:05:33.829 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:05:33.829 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:05:33.829 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:05:33.829 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:05:33.829 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:05:33.829 INFO:teuthology.orchestra.run.smithi060.stdout: "options": { 2024-09-20T22:05:33.829 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_bias": 4 2024-09-20T22:05:33.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.829 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:05:33.829 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:05:33.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.829 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:05:33.829 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:05:33.829 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.9900000095367432, 2024-09-20T22:05:33.829 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.9900000095367432, 2024-09-20T22:05:33.830 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T22:05:33.830 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.75, 2024-09-20T22:05:33.830 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.75, 2024-09-20T22:05:33.830 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:05:33.830 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:05:33.830 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:05:33.830 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.830 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 7, 2024-09-20T22:05:33.830 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "unique_pool_0", 2024-09-20T22:05:33.830 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:03:51.292196+0000", 2024-09-20T22:05:33.830 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:05:33.830 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:05:33.830 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:05:33.831 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:05:33.831 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:05:33.831 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:05:33.831 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:05:33.831 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:05:33.831 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:05:33.831 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:05:33.831 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:05:33.831 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:05:33.831 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:05:33.831 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 16, 2024-09-20T22:05:33.831 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 16, 2024-09-20T22:05:33.831 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 16, 2024-09-20T22:05:33.831 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 16, 2024-09-20T22:05:33.832 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 16, 2024-09-20T22:05:33.832 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:05:33.832 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:05:33.832 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:05:33.832 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:05:33.832 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:05:33.832 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:05:33.832 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:05:33.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.832 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "83", 2024-09-20T22:05:33.832 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:05:33.832 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:05:33.832 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:05:33.832 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:05:33.832 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:05:33.833 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:05:33.833 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:05:33.833 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:05:33.833 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:05:33.833 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:05:33.833 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:05:33.833 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:05:33.833 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:05:33.833 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:05:33.833 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:05:33.833 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:05:33.833 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:05:33.833 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:05:33.833 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:05:33.833 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:05:33.833 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:05:33.834 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:05:33.834 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:05:33.834 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:05:33.834 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:05:33.834 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:05:33.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.834 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:05:33.834 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:05:33.834 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:05:33.834 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:05:33.834 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:05:33.834 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:05:33.834 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:05:33.835 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:05:33.835 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:05:33.835 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:05:33.835 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:05:33.835 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:05:33.835 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:05:33.835 INFO:teuthology.orchestra.run.smithi060.stdout: "rados": {} 2024-09-20T22:05:33.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.835 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:05:33.835 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:05:33.835 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.9900000095367432, 2024-09-20T22:05:33.835 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.9900000095367432, 2024-09-20T22:05:33.835 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T22:05:33.835 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.75, 2024-09-20T22:05:33.835 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.75, 2024-09-20T22:05:33.836 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:05:33.836 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:05:33.836 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:05:33.836 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.836 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.836 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:05:33.836 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:05:33.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.836 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 0, 2024-09-20T22:05:33.836 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "0997c01a-a9bd-421e-abcc-fbcf7abe8512", 2024-09-20T22:05:33.836 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:05:33.836 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:05:33.836 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:05:33.836 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:05:33.836 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:05:33.837 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:05:33.837 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 8, 2024-09-20T22:05:33.837 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 95, 2024-09-20T22:05:33.837 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:05:33.837 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:05:33.837 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:05:33.837 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.837 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.837 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6802", 2024-09-20T22:05:33.837 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:05:33.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.838 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.838 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6803", 2024-09-20T22:05:33.838 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:05:33.838 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.838 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.838 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:05:33.838 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.838 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.838 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6804", 2024-09-20T22:05:33.838 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:05:33.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.838 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.839 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6805", 2024-09-20T22:05:33.839 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:05:33.839 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.839 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.839 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:05:33.839 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.839 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.839 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6808", 2024-09-20T22:05:33.839 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:05:33.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.839 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.839 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6809", 2024-09-20T22:05:33.840 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:05:33.840 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.840 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.840 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:05:33.840 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.841 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.841 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6806", 2024-09-20T22:05:33.841 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:05:33.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.841 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.841 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6807", 2024-09-20T22:05:33.841 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:05:33.841 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.841 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.841 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6803/1395849550", 2024-09-20T22:05:33.841 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6805/1395849550", 2024-09-20T22:05:33.842 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6809/1395849550", 2024-09-20T22:05:33.842 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6807/1395849550", 2024-09-20T22:05:33.842 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:05:33.842 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:05:33.842 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:05:33.842 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.842 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 1, 2024-09-20T22:05:33.842 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "c04bdcb1-d76b-47c8-a38b-c02043df6ba8", 2024-09-20T22:05:33.842 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 0, 2024-09-20T22:05:33.842 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:05:33.842 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:05:33.842 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:05:33.843 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:05:33.843 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:05:33.843 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 13, 2024-09-20T22:05:33.843 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 80, 2024-09-20T22:05:33.843 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 91, 2024-09-20T22:05:33.843 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:05:33.843 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:05:33.843 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.843 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.843 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6810", 2024-09-20T22:05:33.843 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 145283513 2024-09-20T22:05:33.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.844 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.844 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6811", 2024-09-20T22:05:33.844 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 145283513 2024-09-20T22:05:33.844 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.844 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.844 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:05:33.844 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.844 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.844 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6812", 2024-09-20T22:05:33.844 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 145283513 2024-09-20T22:05:33.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.845 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.845 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6813", 2024-09-20T22:05:33.845 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 145283513 2024-09-20T22:05:33.845 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.845 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.845 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:05:33.845 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.845 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.845 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6816", 2024-09-20T22:05:33.845 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 145283513 2024-09-20T22:05:33.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.846 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.846 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6817", 2024-09-20T22:05:33.846 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 145283513 2024-09-20T22:05:33.846 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.846 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.846 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:05:33.846 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.846 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.846 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6814", 2024-09-20T22:05:33.846 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 145283513 2024-09-20T22:05:33.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.847 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.847 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6815", 2024-09-20T22:05:33.847 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 145283513 2024-09-20T22:05:33.847 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.847 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.847 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6811/145283513", 2024-09-20T22:05:33.847 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6813/145283513", 2024-09-20T22:05:33.847 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6817/145283513", 2024-09-20T22:05:33.847 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6815/145283513", 2024-09-20T22:05:33.847 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:05:33.847 INFO:teuthology.orchestra.run.smithi060.stdout: "exists" 2024-09-20T22:05:33.848 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.848 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 2, 2024-09-20T22:05:33.848 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "32c09d19-eb98-4926-a44e-e7ddee152956", 2024-09-20T22:05:33.848 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:05:33.848 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:05:33.848 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:05:33.848 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:05:33.848 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:05:33.848 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:05:33.848 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 18, 2024-09-20T22:05:33.848 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 95, 2024-09-20T22:05:33.848 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:05:33.849 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:05:33.849 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:05:33.849 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.849 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.849 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6818", 2024-09-20T22:05:33.849 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:05:33.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.849 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.849 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6819", 2024-09-20T22:05:33.849 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:05:33.849 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.850 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.850 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:05:33.850 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.850 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.850 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6820", 2024-09-20T22:05:33.850 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:05:33.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.850 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.850 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6821", 2024-09-20T22:05:33.850 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:05:33.850 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.851 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.851 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:05:33.851 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.851 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.851 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6824", 2024-09-20T22:05:33.851 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:05:33.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.851 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.851 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6825", 2024-09-20T22:05:33.851 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:05:33.851 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.852 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.852 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:05:33.852 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.852 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.852 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6822", 2024-09-20T22:05:33.852 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:05:33.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.852 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.852 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6823", 2024-09-20T22:05:33.852 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:05:33.852 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.853 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.853 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6819/3698562274", 2024-09-20T22:05:33.853 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6821/3698562274", 2024-09-20T22:05:33.853 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6825/3698562274", 2024-09-20T22:05:33.853 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6823/3698562274", 2024-09-20T22:05:33.853 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:05:33.853 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:05:33.853 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:05:33.853 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.853 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 3, 2024-09-20T22:05:33.853 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "b1c2cb69-8565-49ac-b854-7ea7615c6660", 2024-09-20T22:05:33.854 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:05:33.854 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:05:33.854 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:05:33.854 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:05:33.854 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:05:33.854 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:05:33.854 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 26, 2024-09-20T22:05:33.854 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 91, 2024-09-20T22:05:33.854 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:05:33.854 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:05:33.854 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:05:33.854 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.855 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.855 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6826", 2024-09-20T22:05:33.855 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:05:33.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.855 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.855 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6827", 2024-09-20T22:05:33.855 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:05:33.855 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.855 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.855 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:05:33.855 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.855 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.856 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6828", 2024-09-20T22:05:33.856 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:05:33.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.856 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.856 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6829", 2024-09-20T22:05:33.856 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:05:33.856 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.856 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.856 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:05:33.856 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.856 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.857 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6832", 2024-09-20T22:05:33.857 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:05:33.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.857 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.857 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6833", 2024-09-20T22:05:33.857 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:05:33.857 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.857 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.857 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:05:33.857 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.858 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.858 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6830", 2024-09-20T22:05:33.858 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:05:33.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.858 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.858 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6831", 2024-09-20T22:05:33.858 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:05:33.858 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.858 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.858 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6827/2592322449", 2024-09-20T22:05:33.858 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6829/2592322449", 2024-09-20T22:05:33.858 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6833/2592322449", 2024-09-20T22:05:33.858 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6831/2592322449", 2024-09-20T22:05:33.859 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:05:33.859 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:05:33.859 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:05:33.859 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.859 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 4, 2024-09-20T22:05:33.859 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "04a547d6-fc48-43b7-a8d7-18df8a83aa96", 2024-09-20T22:05:33.859 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:05:33.859 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:05:33.859 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:05:33.859 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:05:33.859 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:05:33.859 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:05:33.859 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 31, 2024-09-20T22:05:33.859 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 91, 2024-09-20T22:05:33.860 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:05:33.860 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:05:33.860 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:05:33.860 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.860 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.860 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6800", 2024-09-20T22:05:33.860 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:05:33.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.860 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.860 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6801", 2024-09-20T22:05:33.860 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:05:33.860 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.860 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.861 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:05:33.861 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.861 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.861 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6802", 2024-09-20T22:05:33.861 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:05:33.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.861 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.861 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6803", 2024-09-20T22:05:33.861 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:05:33.861 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.861 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.862 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:05:33.862 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.862 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.862 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6806", 2024-09-20T22:05:33.862 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:05:33.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.862 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.862 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6807", 2024-09-20T22:05:33.862 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:05:33.862 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.862 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.863 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:05:33.863 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.863 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.863 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6804", 2024-09-20T22:05:33.863 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:05:33.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.863 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.863 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6805", 2024-09-20T22:05:33.863 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:05:33.863 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.863 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.864 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6801/3243564239", 2024-09-20T22:05:33.864 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6803/3243564239", 2024-09-20T22:05:33.864 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6807/3243564239", 2024-09-20T22:05:33.864 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6805/3243564239", 2024-09-20T22:05:33.864 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:05:33.864 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:05:33.864 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:05:33.864 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.864 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 5, 2024-09-20T22:05:33.864 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "4763aa10-c367-485b-b525-e5241e642b86", 2024-09-20T22:05:33.864 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:05:33.864 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:05:33.864 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:05:33.864 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:05:33.865 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:05:33.865 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:05:33.865 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 39, 2024-09-20T22:05:33.865 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 95, 2024-09-20T22:05:33.865 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:05:33.865 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:05:33.865 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:05:33.865 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.865 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.865 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6808", 2024-09-20T22:05:33.865 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:05:33.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.865 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.866 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6809", 2024-09-20T22:05:33.866 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:05:33.866 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.866 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.866 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:05:33.866 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.866 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.866 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6810", 2024-09-20T22:05:33.866 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:05:33.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.866 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.866 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6811", 2024-09-20T22:05:33.867 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:05:33.867 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.867 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.867 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:05:33.867 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.867 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.867 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6814", 2024-09-20T22:05:33.867 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:05:33.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.867 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.867 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6815", 2024-09-20T22:05:33.867 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:05:33.868 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.868 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.868 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:05:33.868 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.868 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.868 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6812", 2024-09-20T22:05:33.868 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:05:33.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.868 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.868 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6813", 2024-09-20T22:05:33.868 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:05:33.868 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.869 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.869 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6809/1622258887", 2024-09-20T22:05:33.869 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6811/1622258887", 2024-09-20T22:05:33.869 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6815/1622258887", 2024-09-20T22:05:33.869 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6813/1622258887", 2024-09-20T22:05:33.869 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:05:33.869 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:05:33.869 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:05:33.869 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.869 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 6, 2024-09-20T22:05:33.869 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "a359116e-c013-4e2f-8b79-86219516ee16", 2024-09-20T22:05:33.870 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:05:33.870 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:05:33.870 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:05:33.870 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:05:33.870 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:05:33.870 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:05:33.870 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 48, 2024-09-20T22:05:33.870 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 95, 2024-09-20T22:05:33.870 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:05:33.870 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:05:33.870 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:05:33.870 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.870 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.870 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6816", 2024-09-20T22:05:33.871 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:05:33.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.871 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.871 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6817", 2024-09-20T22:05:33.871 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:05:33.871 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.871 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.871 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:05:33.871 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.871 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.871 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6818", 2024-09-20T22:05:33.872 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:05:33.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.872 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.872 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6819", 2024-09-20T22:05:33.872 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:05:33.872 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.872 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.872 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:05:33.872 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.872 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.872 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6822", 2024-09-20T22:05:33.872 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:05:33.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.873 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.873 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6823", 2024-09-20T22:05:33.873 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:05:33.873 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.873 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.873 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:05:33.873 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.873 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.873 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6820", 2024-09-20T22:05:33.873 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:05:33.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.874 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.874 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6821", 2024-09-20T22:05:33.874 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:05:33.874 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.874 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.874 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6817/1187114", 2024-09-20T22:05:33.874 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6819/1187114", 2024-09-20T22:05:33.874 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6823/1187114", 2024-09-20T22:05:33.874 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6821/1187114", 2024-09-20T22:05:33.874 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:05:33.874 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:05:33.874 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:05:33.874 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.875 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 7, 2024-09-20T22:05:33.875 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "a4bf4188-f133-4a31-8897-d7e8182248d7", 2024-09-20T22:05:33.875 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:05:33.875 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:05:33.875 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:05:33.875 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:05:33.875 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:05:33.875 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:05:33.875 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 57, 2024-09-20T22:05:33.875 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 91, 2024-09-20T22:05:33.875 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:05:33.875 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:05:33.875 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:05:33.876 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.876 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.876 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6824", 2024-09-20T22:05:33.876 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:05:33.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.877 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.877 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6825", 2024-09-20T22:05:33.877 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:05:33.877 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.877 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.877 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:05:33.877 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.877 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.877 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6826", 2024-09-20T22:05:33.877 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:05:33.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.877 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.878 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6827", 2024-09-20T22:05:33.878 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:05:33.878 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.878 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.878 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:05:33.878 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.878 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.878 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6830", 2024-09-20T22:05:33.878 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:05:33.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.879 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.879 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6831", 2024-09-20T22:05:33.879 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:05:33.879 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.879 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.879 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:05:33.879 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:05:33.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.879 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:05:33.879 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6828", 2024-09-20T22:05:33.879 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:05:33.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.880 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:05:33.880 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6829", 2024-09-20T22:05:33.880 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:05:33.880 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.880 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.880 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6825/3021842840", 2024-09-20T22:05:33.880 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6827/3021842840", 2024-09-20T22:05:33.880 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6831/3021842840", 2024-09-20T22:05:33.880 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6829/3021842840", 2024-09-20T22:05:33.880 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:05:33.880 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:05:33.880 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:05:33.880 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:05:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_xinfo": [ 2024-09-20T22:05:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 0, 2024-09-20T22:05:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:05:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:05:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:05:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:05:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:05:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:04:55.461948+0000", 2024-09-20T22:05:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:05:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.882 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 1, 2024-09-20T22:05:33.882 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:05:33.882 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:05:33.882 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:05:33.882 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:05:33.882 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:05:33.882 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:04:02.357858+0000", 2024-09-20T22:05:33.882 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 90 2024-09-20T22:05:33.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.882 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 2, 2024-09-20T22:05:33.882 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:05:33.882 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:05:33.882 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:05:33.883 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:05:33.883 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:05:33.883 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:04:39.208550+0000", 2024-09-20T22:05:33.883 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:05:33.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.883 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 3, 2024-09-20T22:05:33.883 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:05:33.883 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:05:33.883 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:05:33.883 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:05:33.883 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:05:33.883 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:04:46.080631+0000", 2024-09-20T22:05:33.883 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:05:33.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.884 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 4, 2024-09-20T22:05:33.884 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:05:33.884 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:05:33.884 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:05:33.884 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:05:33.884 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:05:33.884 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:05:26.679800+0000", 2024-09-20T22:05:33.884 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:05:33.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.884 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 5, 2024-09-20T22:05:33.884 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:05:33.885 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:05:33.885 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:05:33.885 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:05:33.885 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:05:33.885 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:04:36.655764+0000", 2024-09-20T22:05:33.885 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:05:33.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.885 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 6, 2024-09-20T22:05:33.885 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:05:33.885 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:05:33.885 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:05:33.885 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:05:33.885 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:05:33.886 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:04:23.791918+0000", 2024-09-20T22:05:33.886 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:05:33.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.886 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 7, 2024-09-20T22:05:33.886 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:05:33.886 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:05:33.886 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:05:33.886 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:05:33.886 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:05:33.886 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:05:27.631709+0000", 2024-09-20T22:05:33.886 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:05:33.886 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.886 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:05:33.886 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap": [], 2024-09-20T22:05:33.886 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap_items": [ 2024-09-20T22:05:33.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.887 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "3.1a", 2024-09-20T22:05:33.887 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:05:33.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.887 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 4, 2024-09-20T22:05:33.887 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 5 2024-09-20T22:05:33.887 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.887 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.887 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "3.1c", 2024-09-20T22:05:33.887 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:05:33.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.888 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 4, 2024-09-20T22:05:33.888 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 2 2024-09-20T22:05:33.888 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.888 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.888 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.7", 2024-09-20T22:05:33.888 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:05:33.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.888 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:05:33.888 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 0 2024-09-20T22:05:33.888 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.888 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.889 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.9", 2024-09-20T22:05:33.889 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:05:33.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.889 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 7, 2024-09-20T22:05:33.889 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 1 2024-09-20T22:05:33.889 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.889 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.889 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.b", 2024-09-20T22:05:33.889 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:05:33.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.889 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:05:33.889 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 0 2024-09-20T22:05:33.890 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.890 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.890 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.d", 2024-09-20T22:05:33.890 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:05:33.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.890 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 5, 2024-09-20T22:05:33.890 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 6 2024-09-20T22:05:33.890 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.890 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.890 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.11", 2024-09-20T22:05:33.891 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:05:33.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.891 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:05:33.891 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 1 2024-09-20T22:05:33.891 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.891 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.891 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.19", 2024-09-20T22:05:33.891 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:05:33.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.891 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:05:33.891 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 6 2024-09-20T22:05:33.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.892 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 5, 2024-09-20T22:05:33.892 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 0 2024-09-20T22:05:33.892 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.892 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.892 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "7.3", 2024-09-20T22:05:33.892 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:05:33.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.892 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:05:33.892 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 1 2024-09-20T22:05:33.892 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.892 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.892 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.892 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:05:33.893 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap_primaries": [], 2024-09-20T22:05:33.893 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_temp": [ 2024-09-20T22:05:33.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.893 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "3.1a", 2024-09-20T22:05:33.893 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:05:33.893 INFO:teuthology.orchestra.run.smithi060.stdout: 5, 2024-09-20T22:05:33.893 INFO:teuthology.orchestra.run.smithi060.stdout: 2, 2024-09-20T22:05:33.893 INFO:teuthology.orchestra.run.smithi060.stdout: 4 2024-09-20T22:05:33.893 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.893 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "3.1c", 2024-09-20T22:05:33.893 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:05:33.893 INFO:teuthology.orchestra.run.smithi060.stdout: 5, 2024-09-20T22:05:33.893 INFO:teuthology.orchestra.run.smithi060.stdout: 2, 2024-09-20T22:05:33.893 INFO:teuthology.orchestra.run.smithi060.stdout: 4 2024-09-20T22:05:33.894 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.894 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.9", 2024-09-20T22:05:33.894 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:05:33.894 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:05:33.894 INFO:teuthology.orchestra.run.smithi060.stdout: 2, 2024-09-20T22:05:33.894 INFO:teuthology.orchestra.run.smithi060.stdout: 7 2024-09-20T22:05:33.894 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.894 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.b", 2024-09-20T22:05:33.894 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:05:33.894 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:05:33.895 INFO:teuthology.orchestra.run.smithi060.stdout: 7, 2024-09-20T22:05:33.895 INFO:teuthology.orchestra.run.smithi060.stdout: 3 2024-09-20T22:05:33.895 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.895 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.d", 2024-09-20T22:05:33.895 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:05:33.895 INFO:teuthology.orchestra.run.smithi060.stdout: 6, 2024-09-20T22:05:33.895 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:05:33.895 INFO:teuthology.orchestra.run.smithi060.stdout: 5 2024-09-20T22:05:33.895 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.895 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.11", 2024-09-20T22:05:33.895 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:05:33.896 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:05:33.896 INFO:teuthology.orchestra.run.smithi060.stdout: 5, 2024-09-20T22:05:33.896 INFO:teuthology.orchestra.run.smithi060.stdout: 3 2024-09-20T22:05:33.896 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.896 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.19", 2024-09-20T22:05:33.896 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:05:33.896 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:05:33.896 INFO:teuthology.orchestra.run.smithi060.stdout: 6, 2024-09-20T22:05:33.896 INFO:teuthology.orchestra.run.smithi060.stdout: 3 2024-09-20T22:05:33.896 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:33.896 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "7.3", 2024-09-20T22:05:33.896 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:05:33.897 INFO:teuthology.orchestra.run.smithi060.stdout: 2, 2024-09-20T22:05:33.897 INFO:teuthology.orchestra.run.smithi060.stdout: 4, 2024-09-20T22:05:33.897 INFO:teuthology.orchestra.run.smithi060.stdout: 3 2024-09-20T22:05:33.897 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:33.897 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.897 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:05:33.897 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_temp": [], 2024-09-20T22:05:33.897 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklist": { 2024-09-20T22:05:33.897 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/890154710": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:05:33.897 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3386526350": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:05:33.897 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/604714621": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:05:33.897 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/604714621": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:05:33.897 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/3151907776": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:05:33.897 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/956264518": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:05:33.898 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/4150585713": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:05:33.898 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3311255314": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:05:33.898 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3965541528": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:05:33.898 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/3734574621": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:05:33.898 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1729862661": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:05:33.898 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/3734574621": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:05:33.898 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3927294026": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:05:33.898 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/3151907776": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:05:33.898 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1871351181": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:05:33.898 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1854455098": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:05:33.898 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/2221262203": "2024-09-21T22:03:19.518405+0000" 2024-09-20T22:05:33.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.898 INFO:teuthology.orchestra.run.smithi060.stdout: "range_blocklist": {}, 2024-09-20T22:05:33.898 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profiles": { 2024-09-20T22:05:33.899 INFO:teuthology.orchestra.run.smithi060.stdout: "default": { 2024-09-20T22:05:33.899 INFO:teuthology.orchestra.run.smithi060.stdout: "crush-failure-domain": "osd", 2024-09-20T22:05:33.899 INFO:teuthology.orchestra.run.smithi060.stdout: "k": "2", 2024-09-20T22:05:33.899 INFO:teuthology.orchestra.run.smithi060.stdout: "m": "1", 2024-09-20T22:05:33.899 INFO:teuthology.orchestra.run.smithi060.stdout: "plugin": "jerasure", 2024-09-20T22:05:33.899 INFO:teuthology.orchestra.run.smithi060.stdout: "technique": "reed_sol_van" 2024-09-20T22:05:33.899 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:33.899 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps_queue": [], 2024-09-20T22:05:33.899 INFO:teuthology.orchestra.run.smithi060.stdout: "new_removed_snaps": [], 2024-09-20T22:05:33.899 INFO:teuthology.orchestra.run.smithi060.stdout: "new_purged_snaps": [], 2024-09-20T22:05:33.899 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_node_flags": {}, 2024-09-20T22:05:33.899 INFO:teuthology.orchestra.run.smithi060.stdout: "device_class_flags": {}, 2024-09-20T22:05:33.899 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode": { 2024-09-20T22:05:33.900 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode_enabled": false, 2024-09-20T22:05:33.900 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_bucket_count": 0, 2024-09-20T22:05:33.900 INFO:teuthology.orchestra.run.smithi060.stdout: "degraded_stretch_mode": 0, 2024-09-20T22:05:33.900 INFO:teuthology.orchestra.run.smithi060.stdout: "recovering_stretch_mode": 0, 2024-09-20T22:05:33.900 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode_bucket": 0 2024-09-20T22:05:33.900 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:33.900 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:05:33.900 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:33.939 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:33.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:33.940 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:34.041 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:34.072 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:34.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:34.073 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:34.174 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:34.195 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:34.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:34.196 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:34.297 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:34.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:34.329 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:34.329 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:34.430 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:34.464 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:34.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:34.465 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:34.471 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:34 smithi060 ceph-mon[29694]: 6.1c deep-scrub starts 2024-09-20T22:05:34.471 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:34 smithi060 ceph-mon[29694]: 6.1c deep-scrub ok 2024-09-20T22:05:34.471 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:34 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3964705578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-20T22:05:34.471 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:34 smithi060 ceph-mon[40290]: 6.1c deep-scrub starts 2024-09-20T22:05:34.471 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:34 smithi060 ceph-mon[40290]: 6.1c deep-scrub ok 2024-09-20T22:05:34.471 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:34 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3964705578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-20T22:05:34.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:34 smithi067 ceph-mon[33511]: 6.1c deep-scrub starts 2024-09-20T22:05:34.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:34 smithi067 ceph-mon[33511]: 6.1c deep-scrub ok 2024-09-20T22:05:34.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:34 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3964705578' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-20T22:05:34.566 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:34.594 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:34.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:34.621 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:34.722 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:34.756 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:34.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:34.757 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:34.790 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:05:34.858 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:34.883 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:34.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:34.884 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:34.984 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:35.013 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:35.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:35.014 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:35.099 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:05:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:05:35.114 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:35.144 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:35.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:35.146 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:35.246 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:35.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:35.287 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:35.287 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:35.387 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:35.427 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:35.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:35.428 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:35.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:35 smithi067 ceph-mon[33511]: 6.11 scrub starts 2024-09-20T22:05:35.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:35 smithi067 ceph-mon[33511]: 6.11 scrub ok 2024-09-20T22:05:35.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:35 smithi067 ceph-mon[33511]: pgmap v110: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.9 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 898 B/s rd, 14 MiB/s wr, 831 op/s; 48295/387117 objects degraded (12.476%); 8061/387117 objects misplaced (2.082%) 2024-09-20T22:05:35.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:35 smithi067 ceph-mon[33511]: 5.9 scrub starts 2024-09-20T22:05:35.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:35 smithi067 ceph-mon[33511]: 5.9 scrub ok 2024-09-20T22:05:35.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:35 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:35.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:35 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:05:35.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:35 smithi067 ceph-mon[33511]: 6.6 deep-scrub starts 2024-09-20T22:05:35.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:35 smithi067 ceph-mon[33511]: 6.6 deep-scrub ok 2024-09-20T22:05:35.528 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:35.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-mon[29694]: 6.11 scrub starts 2024-09-20T22:05:35.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-mon[29694]: 6.11 scrub ok 2024-09-20T22:05:35.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-mon[29694]: pgmap v110: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.9 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 898 B/s rd, 14 MiB/s wr, 831 op/s; 48295/387117 objects degraded (12.476%); 8061/387117 objects misplaced (2.082%) 2024-09-20T22:05:35.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-mon[29694]: 5.9 scrub starts 2024-09-20T22:05:35.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-mon[29694]: 5.9 scrub ok 2024-09-20T22:05:35.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:35.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:05:35.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-mon[29694]: 6.6 deep-scrub starts 2024-09-20T22:05:35.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-mon[29694]: 6.6 deep-scrub ok 2024-09-20T22:05:35.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-mon[40290]: 6.11 scrub starts 2024-09-20T22:05:35.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-mon[40290]: 6.11 scrub ok 2024-09-20T22:05:35.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-mon[40290]: pgmap v110: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.9 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 898 B/s rd, 14 MiB/s wr, 831 op/s; 48295/387117 objects degraded (12.476%); 8061/387117 objects misplaced (2.082%) 2024-09-20T22:05:35.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-mon[40290]: 5.9 scrub starts 2024-09-20T22:05:35.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-mon[40290]: 5.9 scrub ok 2024-09-20T22:05:35.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:35.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:05:35.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-mon[40290]: 6.6 deep-scrub starts 2024-09-20T22:05:35.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:35 smithi060 ceph-mon[40290]: 6.6 deep-scrub ok 2024-09-20T22:05:35.571 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:35.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:35.572 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:35.673 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:35.707 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:35.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:35.708 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:35.809 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:35.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:35.842 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:35.842 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:35.943 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:35.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:35.979 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:35.980 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:36.080 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:36.113 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:36.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:36.114 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:36.208 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:05:36.214 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:36.249 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:36.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:36.250 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:36.351 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:36.374 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:36.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:36.375 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:36.476 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:36.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:36.510 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:36.510 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:36.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:36 smithi067 ceph-mon[33511]: 2.1 deep-scrub starts 2024-09-20T22:05:36.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:36 smithi067 ceph-mon[33511]: 2.1 deep-scrub ok 2024-09-20T22:05:36.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:36 smithi067 ceph-mon[33511]: 7.1 scrub starts 2024-09-20T22:05:36.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:36 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2021943092' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:05:36.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:36 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:05:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:36 smithi060 ceph-mon[29694]: 2.1 deep-scrub starts 2024-09-20T22:05:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:36 smithi060 ceph-mon[29694]: 2.1 deep-scrub ok 2024-09-20T22:05:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:36 smithi060 ceph-mon[29694]: 7.1 scrub starts 2024-09-20T22:05:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:36 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2021943092' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:05:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:36 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:05:36.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:36 smithi060 ceph-mon[40290]: 2.1 deep-scrub starts 2024-09-20T22:05:36.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:36 smithi060 ceph-mon[40290]: 2.1 deep-scrub ok 2024-09-20T22:05:36.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:36 smithi060 ceph-mon[40290]: 7.1 scrub starts 2024-09-20T22:05:36.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:36 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2021943092' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:05:36.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:36 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:05:36.611 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:36.632 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:36.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:36.633 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:36.734 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:36.766 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:36.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:36.767 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:36.868 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:36.904 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:36.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:36.905 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:37.006 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:37.027 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:37.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:37.028 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:37.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:37.167 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:37.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:37.168 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:37.268 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:37.300 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:37.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:37.301 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:37.402 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:37.425 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:37.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:37.426 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:37.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:37 smithi067 ceph-mon[33511]: 6.16 scrub starts 2024-09-20T22:05:37.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:37 smithi067 ceph-mon[33511]: 6.16 scrub ok 2024-09-20T22:05:37.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:37 smithi067 ceph-mon[33511]: pgmap v111: 148 pgs: 1 active+clean+scrubbing, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 7.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.3 KiB/s rd, 11 MiB/s wr, 1.34k op/s; 46018/369045 objects degraded (12.469%); 7692/369045 objects misplaced (2.084%) 2024-09-20T22:05:37.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:37 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:05:37.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:37 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:05:37.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:37 smithi067 ceph-mon[33511]: osdmap e114: 8 total, 7 up, 8 in 2024-09-20T22:05:37.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:37 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 46018/369045 objects degraded (12.469%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:37.527 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:37.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:37 smithi060 ceph-mon[29694]: 6.16 scrub starts 2024-09-20T22:05:37.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:37 smithi060 ceph-mon[29694]: 6.16 scrub ok 2024-09-20T22:05:37.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:37 smithi060 ceph-mon[29694]: pgmap v111: 148 pgs: 1 active+clean+scrubbing, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 7.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.3 KiB/s rd, 11 MiB/s wr, 1.34k op/s; 46018/369045 objects degraded (12.469%); 7692/369045 objects misplaced (2.084%) 2024-09-20T22:05:37.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:37 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:05:37.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:37 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:05:37.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:37 smithi060 ceph-mon[29694]: osdmap e114: 8 total, 7 up, 8 in 2024-09-20T22:05:37.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:37 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 46018/369045 objects degraded (12.469%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:37.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:37 smithi060 ceph-mon[40290]: 6.16 scrub starts 2024-09-20T22:05:37.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:37 smithi060 ceph-mon[40290]: 6.16 scrub ok 2024-09-20T22:05:37.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:37 smithi060 ceph-mon[40290]: pgmap v111: 148 pgs: 1 active+clean+scrubbing, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 7.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.3 KiB/s rd, 11 MiB/s wr, 1.34k op/s; 46018/369045 objects degraded (12.469%); 7692/369045 objects misplaced (2.084%) 2024-09-20T22:05:37.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:37 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:05:37.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:37 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:05:37.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:37 smithi060 ceph-mon[40290]: osdmap e114: 8 total, 7 up, 8 in 2024-09-20T22:05:37.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:37 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 46018/369045 objects degraded (12.469%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:37.557 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:37.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:37.558 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:37.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:37.689 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:37.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:37.690 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:37.790 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:37.812 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:37.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:37.813 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:37.914 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:37.942 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:37.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:37.943 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:38.044 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:38.082 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:38.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:38.083 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:38.184 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:38.206 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:38.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:38.207 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:38.308 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:38.349 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:38.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:38.350 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:38.451 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:38.472 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:38.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:38.473 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:38.574 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:38.601 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:38.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:38.602 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:38.703 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:38.735 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:38.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:38.736 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:38.837 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:38.865 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:38.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:38.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:38.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:39.002 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:39.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:39.004 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:39.104 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:39.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:39.137 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:39.137 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:39.238 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:39.260 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:39.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:39.261 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:39.362 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:39.398 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:39.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:39.399 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:39.500 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:39.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:39 smithi067 ceph-mon[33511]: pgmap v113: 148 pgs: 1 active+clean+scrubbing, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 7.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 MiB/s wr, 1.32k op/s; 46018/369045 objects degraded (12.469%); 7692/369045 objects misplaced (2.084%) 2024-09-20T22:05:39.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:39.536 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:39.536 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:39.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:39 smithi060 ceph-mon[29694]: pgmap v113: 148 pgs: 1 active+clean+scrubbing, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 7.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 MiB/s wr, 1.32k op/s; 46018/369045 objects degraded (12.469%); 7692/369045 objects misplaced (2.084%) 2024-09-20T22:05:39.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:39 smithi060 ceph-mon[40290]: pgmap v113: 148 pgs: 1 active+clean+scrubbing, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 7.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 MiB/s wr, 1.32k op/s; 46018/369045 objects degraded (12.469%); 7692/369045 objects misplaced (2.084%) 2024-09-20T22:05:39.638 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:39.670 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:39.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:39.671 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:39.772 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:39.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:39.808 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:39.808 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:39.909 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:39.931 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:39.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:39.932 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:40.033 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:40.066 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:40.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:40.067 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:40.168 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:40.206 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:40.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:40.207 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:40.308 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:40.343 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:40.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:40.344 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:40.444 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:40.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:40.487 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:40.487 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:40.588 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:40.623 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:40.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:40.624 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:40.724 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:40.758 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:40.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:40.759 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:40.859 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:40.892 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:40.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:40.893 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:40.994 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:41.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:41.020 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:41.020 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:41.121 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:41.162 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:41.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:41.163 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:41.264 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:41.299 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:41.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:41.300 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:41.400 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:41.433 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:41.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:41.434 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:41.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:41 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:05:41.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:41 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:05:41.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:41 smithi067 ceph-mon[33511]: pgmap v114: 148 pgs: 1 active+clean+scrubbing, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 7.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 9.3 MiB/s wr, 1.17k op/s; 46018/369045 objects degraded (12.469%); 7692/369045 objects misplaced (2.084%) 2024-09-20T22:05:41.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:41 smithi067 ceph-mon[33511]: osdmap e115: 8 total, 7 up, 8 in 2024-09-20T22:05:41.535 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:41.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:41 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:05:41.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:41 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:05:41.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:41 smithi060 ceph-mon[29694]: pgmap v114: 148 pgs: 1 active+clean+scrubbing, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 7.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 9.3 MiB/s wr, 1.17k op/s; 46018/369045 objects degraded (12.469%); 7692/369045 objects misplaced (2.084%) 2024-09-20T22:05:41.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:41 smithi060 ceph-mon[29694]: osdmap e115: 8 total, 7 up, 8 in 2024-09-20T22:05:41.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:41 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:05:41.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:41 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:05:41.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:41 smithi060 ceph-mon[40290]: pgmap v114: 148 pgs: 1 active+clean+scrubbing, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 7.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.1 KiB/s rd, 9.3 MiB/s wr, 1.17k op/s; 46018/369045 objects degraded (12.469%); 7692/369045 objects misplaced (2.084%) 2024-09-20T22:05:41.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:41 smithi060 ceph-mon[40290]: osdmap e115: 8 total, 7 up, 8 in 2024-09-20T22:05:41.577 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:41.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:41.578 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:41.679 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:41.708 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:41.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:41.709 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:41.810 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:41.854 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:41.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:41.855 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:41.956 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:41.991 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:41.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:41.992 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:42.092 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:42.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:42.126 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:42.126 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:42.227 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:42.233 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:05:41 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:05:42.249 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:42.253 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:42.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:42.254 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:42.355 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:42.385 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:42.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:42.387 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:42.487 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:42.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:42 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 44013/352941 objects degraded (12.470%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:42.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:42.525 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:42.525 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:42.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:42 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 44013/352941 objects degraded (12.470%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:42.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:42 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 44013/352941 objects degraded (12.470%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:42.626 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:42.657 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:42.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:42.657 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:42.758 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:42.766 INFO:tasks.thrashosds.thrasher:j is {'epoch': 113, 'fsid': '3df9647e-779b-11ef-baf5-efdc52797490', 'created': '2024-09-20T21:57:31.244934+0000', 'modified': '2024-09-20T22:05:28.582772+0000', 'last_up_change': '2024-09-20T22:04:15.685933+0000', 'last_in_change': '2024-09-20T22:02:21.189225+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 7, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-20T22:00:35.131125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-20T22:02:47.974480+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6700000762939453, 'score_stable': 2.6700000762939453, 'optimal_score': 0.75, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-20T22:02:48.902465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-20T22:02:50.620057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.240000009536743, 'score_stable': 2.240000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.9700000286102295, 'raw_score_stable': 1.9700000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-20T22:02:52.722912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-20T22:02:54.794248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-20T22:03:51.292196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '83', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '0997c01a-a9bd-421e-abcc-fbcf7abe8512', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6802', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6803', 'nonce': 1395849550}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6804', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6805', 'nonce': 1395849550}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6808', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6809', 'nonce': 1395849550}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6806', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6807', 'nonce': 1395849550}]}, 'public_addr': '172.21.15.60:6803/1395849550', 'cluster_addr': '172.21.15.60:6805/1395849550', 'heartbeat_back_addr': '172.21.15.60:6809/1395849550', 'heartbeat_front_addr': '172.21.15.60:6807/1395849550', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c04bdcb1-d76b-47c8-a38b-c02043df6ba8', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 80, 'down_at': 91, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6810', 'nonce': 145283513}, {'type': 'v1', 'addr': '172.21.15.60:6811', 'nonce': 145283513}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6812', 'nonce': 145283513}, {'type': 'v1', 'addr': '172.21.15.60:6813', 'nonce': 145283513}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6816', 'nonce': 145283513}, {'type': 'v1', 'addr': '172.21.15.60:6817', 'nonce': 145283513}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6814', 'nonce': 145283513}, {'type': 'v1', 'addr': '172.21.15.60:6815', 'nonce': 145283513}]}, 'public_addr': '172.21.15.60:6811/145283513', 'cluster_addr': '172.21.15.60:6813/145283513', 'heartbeat_back_addr': '172.21.15.60:6817/145283513', 'heartbeat_front_addr': '172.21.15.60:6815/145283513', 'state': ['exists']}, {'osd': 2, 'uuid': '32c09d19-eb98-4926-a44e-e7ddee152956', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6818', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6819', 'nonce': 3698562274}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6820', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6821', 'nonce': 3698562274}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6824', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6825', 'nonce': 3698562274}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6822', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6823', 'nonce': 3698562274}]}, 'public_addr': '172.21.15.60:6819/3698562274', 'cluster_addr': '172.21.15.60:6821/3698562274', 'heartbeat_back_addr': '172.21.15.60:6825/3698562274', 'heartbeat_front_addr': '172.21.15.60:6823/3698562274', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'b1c2cb69-8565-49ac-b854-7ea7615c6660', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 91, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6826', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6827', 'nonce': 2592322449}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6828', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6829', 'nonce': 2592322449}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6832', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6833', 'nonce': 2592322449}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6830', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6831', 'nonce': 2592322449}]}, 'public_addr': '172.21.15.60:6827/2592322449', 'cluster_addr': '172.21.15.60:6829/2592322449', 'heartbeat_back_addr': '172.21.15.60:6833/2592322449', 'heartbeat_front_addr': '172.21.15.60:6831/2592322449', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '04a547d6-fc48-43b7-a8d7-18df8a83aa96', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 91, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6800', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6801', 'nonce': 3243564239}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6802', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6803', 'nonce': 3243564239}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6806', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6807', 'nonce': 3243564239}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6804', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6805', 'nonce': 3243564239}]}, 'public_addr': '172.21.15.67:6801/3243564239', 'cluster_addr': '172.21.15.67:6803/3243564239', 'heartbeat_back_addr': '172.21.15.67:6807/3243564239', 'heartbeat_front_addr': '172.21.15.67:6805/3243564239', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4763aa10-c367-485b-b525-e5241e642b86', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6808', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6809', 'nonce': 1622258887}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6810', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6811', 'nonce': 1622258887}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6814', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6815', 'nonce': 1622258887}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6812', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6813', 'nonce': 1622258887}]}, 'public_addr': '172.21.15.67:6809/1622258887', 'cluster_addr': '172.21.15.67:6811/1622258887', 'heartbeat_back_addr': '172.21.15.67:6815/1622258887', 'heartbeat_front_addr': '172.21.15.67:6813/1622258887', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'a359116e-c013-4e2f-8b79-86219516ee16', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 48, 'up_thru': 95, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6816', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6817', 'nonce': 1187114}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6818', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6819', 'nonce': 1187114}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6822', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6823', 'nonce': 1187114}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6820', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6821', 'nonce': 1187114}]}, 'public_addr': '172.21.15.67:6817/1187114', 'cluster_addr': '172.21.15.67:6819/1187114', 'heartbeat_back_addr': '172.21.15.67:6823/1187114', 'heartbeat_front_addr': '172.21.15.67:6821/1187114', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a4bf4188-f133-4a31-8897-d7e8182248d7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 57, 'up_thru': 91, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6824', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6825', 'nonce': 3021842840}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6826', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6827', 'nonce': 3021842840}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6830', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6831', 'nonce': 3021842840}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6828', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6829', 'nonce': 3021842840}]}, 'public_addr': '172.21.15.67:6825/3021842840', 'cluster_addr': '172.21.15.67:6827/3021842840', 'heartbeat_back_addr': '172.21.15.67:6831/3021842840', 'heartbeat_front_addr': '172.21.15.67:6829/3021842840', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:04:55.461948+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:04:02.357858+0000', 'dead_epoch': 90}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:04:39.208550+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:04:46.080631+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:05:26.679800+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:04:36.655764+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:04:23.791918+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:05:27.631709+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.1a', 'mappings': [{'from': 4, 'to': 5}]}, {'pgid': '3.1c', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '6.7', 'mappings': [{'from': 3, 'to': 0}]}, {'pgid': '6.9', 'mappings': [{'from': 7, 'to': 1}]}, {'pgid': '6.b', 'mappings': [{'from': 3, 'to': 0}]}, {'pgid': '6.d', 'mappings': [{'from': 5, 'to': 6}]}, {'pgid': '6.11', 'mappings': [{'from': 3, 'to': 1}]}, {'pgid': '6.19', 'mappings': [{'from': 3, 'to': 6}, {'from': 5, 'to': 0}]}, {'pgid': '7.3', 'mappings': [{'from': 3, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.1a', 'osds': [5, 2, 4]}, {'pgid': '3.1c', 'osds': [5, 2, 4]}, {'pgid': '6.9', 'osds': [0, 2, 7]}, {'pgid': '6.b', 'osds': [0, 7, 3]}, {'pgid': '6.d', 'osds': [6, 0, 5]}, {'pgid': '6.11', 'osds': [0, 5, 3]}, {'pgid': '6.19', 'osds': [0, 6, 3]}, {'pgid': '7.3', 'osds': [2, 4, 3]}], 'primary_temp': [], 'blocklist': {'172.21.15.60:0/890154710': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:0/3386526350': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:6801/604714621': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:6800/604714621': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:6800/3151907776': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/956264518': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/4150585713': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/3311255314': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:0/3965541528': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:6800/3734574621': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:0/1729862661': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:6801/3734574621': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:0/3927294026': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:6801/3151907776': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/1871351181': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:0/1854455098': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:0/2221262203': '2024-09-21T22:03:19.518405+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-20T22:05:42.766 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-20T22:05:42.772 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:42.786 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:42.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:42.787 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:42.888 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:42.922 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:42.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:42.923 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:43.023 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:43.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:43.074 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:43.074 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:43.175 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:43.204 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:43.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:43.205 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:43.306 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:43.333 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:43.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:43.334 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:43.435 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:43.460 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:43.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:43.461 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:43.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:43 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:05:43.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:43 smithi060 ceph-mon[29694]: pgmap v116: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.2 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s; 44013/352941 objects degraded (12.470%); 7356/352941 objects misplaced (2.084%) 2024-09-20T22:05:43.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:43 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:05:43.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:43 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:05:43.504 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:43 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:05:43.504 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:43 smithi060 ceph-mon[40290]: pgmap v116: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.2 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s; 44013/352941 objects degraded (12.470%); 7356/352941 objects misplaced (2.084%) 2024-09-20T22:05:43.504 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:43 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:05:43.504 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:43 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:05:43.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:43 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:05:43.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:43 smithi067 ceph-mon[33511]: pgmap v116: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.2 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.42k op/s; 44013/352941 objects degraded (12.470%); 7356/352941 objects misplaced (2.084%) 2024-09-20T22:05:43.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:43 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:05:43.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:43 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:05:43.562 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:43.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:43.613 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:43.613 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:43.621 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:05:43.621 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:05:43.714 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:43.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:43.755 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:43.755 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:43.856 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:43.886 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:43.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:43.887 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:43.940 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:05:43.940 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:05:43.940 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:05:43.941 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:05:43.941 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:05:43.941 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:05:43.941 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:05:43.941 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:05:43.941 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:05:43.941 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:05:43.941 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:05:43.941 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:05:43.941 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:05:43.941 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:05:43.942 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:05:43.988 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:44.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:44.023 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:44.023 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:44.124 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:44.153 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:44.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:44.154 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:44.255 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:44.287 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:44.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:44.288 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:44.299 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:05:44.388 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:44.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:44.427 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:44.427 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:44.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:44 smithi067 ceph-mon[33511]: osdmap e116: 8 total, 7 up, 8 in 2024-09-20T22:05:44.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:44 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4117402915' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:44.528 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:44.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:44 smithi060 ceph-mon[29694]: osdmap e116: 8 total, 7 up, 8 in 2024-09-20T22:05:44.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:44 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4117402915' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:44.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:44 smithi060 ceph-mon[40290]: osdmap e116: 8 total, 7 up, 8 in 2024-09-20T22:05:44.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:44 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4117402915' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:44.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:44.573 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:44.573 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:44.674 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:44.707 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:44.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:44.708 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:44.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T22:05:44.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:44.859 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:44.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:44.860 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:44.961 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:44.992 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:44.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:44.993 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:45.094 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:45.123 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:45.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:45.124 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:45.222 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:05:45.223 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:05:45.224 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:45.276 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:45.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:45.277 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:45.297 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:45.323 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:45.378 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:45.406 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:45.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:45.407 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:45.449 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:05:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:05:45.449 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:45 smithi060 ceph-mon[29694]: pgmap v118: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.2 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 662 B/s rd, 0 B/s wr, 694 op/s; 44013/352941 objects degraded (12.470%); 7356/352941 objects misplaced (2.084%) 2024-09-20T22:05:45.450 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:45 smithi060 ceph-mon[40290]: pgmap v118: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.2 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 662 B/s rd, 0 B/s wr, 694 op/s; 44013/352941 objects degraded (12.470%); 7356/352941 objects misplaced (2.084%) 2024-09-20T22:05:45.508 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:45.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:45 smithi067 ceph-mon[33511]: pgmap v118: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 7.2 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 662 B/s rd, 0 B/s wr, 694 op/s; 44013/352941 objects degraded (12.470%); 7356/352941 objects misplaced (2.084%) 2024-09-20T22:05:45.552 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:45.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:45.553 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:45.654 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:45.684 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:45.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:45.685 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:45.786 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:45.816 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:45.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:45.817 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:45.918 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:45.956 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:45.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:45.957 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:46.058 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:46.099 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:46.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:46.100 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:46.201 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:46.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:46.241 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:46.241 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:46.341 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:46.371 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:46.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:46.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:46.473 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:46.510 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:46.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:46.511 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:46.612 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:46.634 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:46.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:46.635 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:46.736 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:46.772 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:46.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:46.773 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:46.874 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:46.916 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:46.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:46.918 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:47.018 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:47.066 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:47.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:47.067 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:47.168 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:47.209 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:05:47.209 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:05:47.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.210 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:156796 7.7 e116/80 7:ef1b0820:::benchmark_data_smithi060_78800_object23763:head v 116'9819, mlcod=116'9816)", 2024-09-20T22:05:47.210 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:47.203880+0000", 2024-09-20T22:05:47.210 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.001256832, 2024-09-20T22:05:47.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001286134, 2024-09-20T22:05:47.210 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:47.210 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:47.210 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:05:47.210 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:47.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.210 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:47.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.203880+0000", 2024-09-20T22:05:47.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.211 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.211 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:47.211 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.203880+0000", 2024-09-20T22:05:47.211 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.211 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.211 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:47.211 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.203880+0000", 2024-09-20T22:05:47.211 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9299999999999998e-07 2024-09-20T22:05:47.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.211 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.211 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:47.211 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.203884+0000", 2024-09-20T22:05:47.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4979999999999998e-06 2024-09-20T22:05:47.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:47.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.203891+0000", 2024-09-20T22:05:47.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6429999999999995e-06 2024-09-20T22:05:47.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:47.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.203896+0000", 2024-09-20T22:05:47.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9400000000000001e-06 2024-09-20T22:05:47.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:47.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204940+0000", 2024-09-20T22:05:47.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0010439659999999999 2024-09-20T22:05:47.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:47.213 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204959+0000", 2024-09-20T22:05:47.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9076000000000002e-05 2024-09-20T22:05:47.213 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.213 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:47.213 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.213 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:156798 7.0 e116/91 7:0a1d0c2e:::benchmark_data_smithi060_78800_object23765:head v 116'9771, mlcod=116'9768)", 2024-09-20T22:05:47.213 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:47.204358+0000", 2024-09-20T22:05:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00077860500000000003, 2024-09-20T22:05:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00088016199999999998, 2024-09-20T22:05:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:05:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204358+0000", 2024-09-20T22:05:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.214 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204358+0000", 2024-09-20T22:05:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204359+0000", 2024-09-20T22:05:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3799999999999996e-07 2024-09-20T22:05:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204364+0000", 2024-09-20T22:05:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3809999999999998e-06 2024-09-20T22:05:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204376+0000", 2024-09-20T22:05:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2203e-05 2024-09-20T22:05:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204383+0000", 2024-09-20T22:05:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3089999999999998e-06 2024-09-20T22:05:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204451+0000", 2024-09-20T22:05:47.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7306999999999998e-05 2024-09-20T22:05:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204478+0000", 2024-09-20T22:05:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7404000000000001e-05 2024-09-20T22:05:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop_reply(client.24667.0:156781 7.a e116/80)", 2024-09-20T22:05:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:47.204486+0000", 2024-09-20T22:05:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00065077199999999998, 2024-09-20T22:05:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00081568299999999999, 2024-09-20T22:05:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:05:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204486+0000", 2024-09-20T22:05:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204486+0000", 2024-09-20T22:05:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204486+0000", 2024-09-20T22:05:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.44e-07 2024-09-20T22:05:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204490+0000", 2024-09-20T22:05:47.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5920000000000001e-06 2024-09-20T22:05:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204498+0000", 2024-09-20T22:05:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2239999999999995e-06 2024-09-20T22:05:47.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204505+0000", 2024-09-20T22:05:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5989999999999998e-06 2024-09-20T22:05:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:156799 7.c e116/80 7:392b4e38:::benchmark_data_smithi060_78800_object23766:head v 116'9713, mlcod=116'9711)", 2024-09-20T22:05:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:47.204765+0000", 2024-09-20T22:05:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00037175499999999998, 2024-09-20T22:05:47.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00058775199999999998, 2024-09-20T22:05:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:05:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204765+0000", 2024-09-20T22:05:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204765+0000", 2024-09-20T22:05:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204765+0000", 2024-09-20T22:05:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1099999999999995e-07 2024-09-20T22:05:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204770+0000", 2024-09-20T22:05:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8930000000000004e-06 2024-09-20T22:05:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204782+0000", 2024-09-20T22:05:47.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2034e-05 2024-09-20T22:05:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204788+0000", 2024-09-20T22:05:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6729999999999997e-06 2024-09-20T22:05:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204842+0000", 2024-09-20T22:05:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4524000000000002e-05 2024-09-20T22:05:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204859+0000", 2024-09-20T22:05:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6560000000000001e-05 2024-09-20T22:05:47.224 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:156779 7.2 7:4117de69:::benchmark_data_smithi060_78800_object23746:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-20T22:05:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:47.184320+0000", 2024-09-20T22:05:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.020816020000000001, 2024-09-20T22:05:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021101689, 2024-09-20T22:05:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:05:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:05:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:05:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:47.225 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 156779 2024-09-20T22:05:47.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.226 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:47.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:47.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.184320+0000", 2024-09-20T22:05:47.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:47.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.184320+0000", 2024-09-20T22:05:47.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:47.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.184321+0000", 2024-09-20T22:05:47.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9600000000000008e-07 2024-09-20T22:05:47.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:47.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.184326+0000", 2024-09-20T22:05:47.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7180000000000004e-06 2024-09-20T22:05:47.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:47.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.184337+0000", 2024-09-20T22:05:47.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1279e-05 2024-09-20T22:05:47.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:47.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.184344+0000", 2024-09-20T22:05:47.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7510000000000002e-06 2024-09-20T22:05:47.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:47.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.198690+0000", 2024-09-20T22:05:47.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014345744000000001 2024-09-20T22:05:47.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:47.230 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.198806+0000", 2024-09-20T22:05:47.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000116309 2024-09-20T22:05:47.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.230 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2", 2024-09-20T22:05:47.230 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.198922+0000", 2024-09-20T22:05:47.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115455 2024-09-20T22:05:47.230 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.230 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:47.230 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.230 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:156781 7.a 7:55f8a95f:::benchmark_data_smithi060_78800_object23748:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-20T22:05:47.230 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:47.188415+0000", 2024-09-20T22:05:47.231 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016721696000000001, 2024-09-20T22:05:47.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.017088575000000002, 2024-09-20T22:05:47.231 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:47.231 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:47.231 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:05:47.231 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:05:47.231 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:05:47.231 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:47.231 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 156781 2024-09-20T22:05:47.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.231 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:47.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:47.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.188415+0000", 2024-09-20T22:05:47.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:47.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.188415+0000", 2024-09-20T22:05:47.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:47.233 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.188416+0000", 2024-09-20T22:05:47.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.471e-06 2024-09-20T22:05:47.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.233 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:47.233 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.188420+0000", 2024-09-20T22:05:47.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6689999999999999e-06 2024-09-20T22:05:47.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.234 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:47.234 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.188429+0000", 2024-09-20T22:05:47.234 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3570000000000004e-06 2024-09-20T22:05:47.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.234 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:47.234 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.188435+0000", 2024-09-20T22:05:47.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8860000000000001e-06 2024-09-20T22:05:47.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:47.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.196313+0000", 2024-09-20T22:05:47.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0078779430000000001 2024-09-20T22:05:47.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:47.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.196428+0000", 2024-09-20T22:05:47.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011524 2024-09-20T22:05:47.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:05:47.236 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.196538+0000", 2024-09-20T22:05:47.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010991499999999999 2024-09-20T22:05:47.236 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.236 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:47.236 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.236 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:156788 7.a 7:59ebf875:::benchmark_data_smithi060_78800_object23755:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-20T22:05:47.236 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:47.194768+0000", 2024-09-20T22:05:47.236 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.010368709, 2024-09-20T22:05:47.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010814473, 2024-09-20T22:05:47.237 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:47.237 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:47.237 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:05:47.237 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:05:47.237 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:05:47.237 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:47.237 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 156788 2024-09-20T22:05:47.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.238 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:47.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:47.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.194768+0000", 2024-09-20T22:05:47.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:47.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.194768+0000", 2024-09-20T22:05:47.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:47.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.194769+0000", 2024-09-20T22:05:47.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6330000000000001e-06 2024-09-20T22:05:47.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:47.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.194775+0000", 2024-09-20T22:05:47.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4210000000000003e-06 2024-09-20T22:05:47.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:47.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.194880+0000", 2024-09-20T22:05:47.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105739 2024-09-20T22:05:47.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:47.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.194890+0000", 2024-09-20T22:05:47.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6889999999999999e-06 2024-09-20T22:05:47.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:47.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.199628+0000", 2024-09-20T22:05:47.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0047374690000000002 2024-09-20T22:05:47.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.242 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:47.242 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.199840+0000", 2024-09-20T22:05:47.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000212103 2024-09-20T22:05:47.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:05:47.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.199958+0000", 2024-09-20T22:05:47.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117908 2024-09-20T22:05:47.243 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.243 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:47.243 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.244 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:156789 7.2 7.14127462 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-20T22:05:47.244 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:47.197911+0000", 2024-09-20T22:05:47.244 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.007225235, 2024-09-20T22:05:47.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:47.246 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:47.246 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:47.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0077414370000000003, 2024-09-20T22:05:47.247 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:47.247 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:47.247 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:05:47.247 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:05:47.247 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:05:47.247 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:47.247 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 156789 2024-09-20T22:05:47.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.248 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:47.248 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.248 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:47.248 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.197911+0000", 2024-09-20T22:05:47.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:47.249 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.197911+0000", 2024-09-20T22:05:47.249 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:47.250 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.197913+0000", 2024-09-20T22:05:47.250 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6789999999999999e-06 2024-09-20T22:05:47.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.250 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.250 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:47.250 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.197920+0000", 2024-09-20T22:05:47.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4030000000000001e-06 2024-09-20T22:05:47.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.251 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.251 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:47.251 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.197939+0000", 2024-09-20T22:05:47.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9148999999999999e-05 2024-09-20T22:05:47.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.252 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.252 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:47.252 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.197948+0000", 2024-09-20T22:05:47.252 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0240000000000003e-06 2024-09-20T22:05:47.252 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.252 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:47.252 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.253 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:156792 7.a 7.de2e0c2a (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-20T22:05:47.253 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:47.198749+0000", 2024-09-20T22:05:47.253 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0063878620000000002, 2024-09-20T22:05:47.253 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0069649159999999998, 2024-09-20T22:05:47.253 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:47.253 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:47.253 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:05:47.253 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:05:47.254 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:05:47.254 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:47.254 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 156792 2024-09-20T22:05:47.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.254 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:47.254 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:47.254 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.198749+0000", 2024-09-20T22:05:47.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:47.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.198749+0000", 2024-09-20T22:05:47.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.256 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:47.256 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.198749+0000", 2024-09-20T22:05:47.256 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5300000000000003e-07 2024-09-20T22:05:47.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.256 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.256 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:47.256 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.198753+0000", 2024-09-20T22:05:47.257 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9820000000000002e-06 2024-09-20T22:05:47.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.257 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.257 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:47.257 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.198766+0000", 2024-09-20T22:05:47.257 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2404e-05 2024-09-20T22:05:47.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.257 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.257 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:47.257 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.198773+0000", 2024-09-20T22:05:47.257 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4569999999999999e-06 2024-09-20T22:05:47.258 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.258 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:47.258 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.258 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.258 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop_reply(client.24667.0:156781 7.a e116/80)", 2024-09-20T22:05:47.258 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:47.200209+0000", 2024-09-20T22:05:47.258 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0049273290000000003, 2024-09-20T22:05:47.258 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0055578169999999996, 2024-09-20T22:05:47.258 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:47.258 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:47.258 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:05:47.258 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:47.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:47.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.200209+0000", 2024-09-20T22:05:47.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:47.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.200209+0000", 2024-09-20T22:05:47.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:47.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.200209+0000", 2024-09-20T22:05:47.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.39e-07 2024-09-20T22:05:47.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.200212+0000", 2024-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6359999999999998e-06 2024-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.200218+0000", 2024-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4540000000000004e-06 2024-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.200224+0000", 2024-09-20T22:05:47.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2109999999999997e-06 2024-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop_reply(client.24667.0:156788 7.a e116/80)", 2024-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:47.203687+0000", 2024-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.001449449, 2024-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021188629999999999, 2024-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.261 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:47.262 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.203687+0000", 2024-09-20T22:05:47.262 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.262 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.262 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:47.262 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.203687+0000", 2024-09-20T22:05:47.262 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.262 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.262 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:47.262 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.203688+0000", 2024-09-20T22:05:47.262 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4900000000000004e-07 2024-09-20T22:05:47.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.262 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.263 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:47.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.203694+0000", 2024-09-20T22:05:47.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1179999999999997e-06 2024-09-20T22:05:47.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.263 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:47.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.203706+0000", 2024-09-20T22:05:47.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.206e-05 2024-09-20T22:05:47.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.263 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:47.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.203713+0000", 2024-09-20T22:05:47.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9140000000000002e-06 2024-09-20T22:05:47.263 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:156793 7.3 e116/95 7:c52a25fd:::benchmark_data_smithi060_78800_object23760:head v 116'9808, mlcod=116'9804)", 2024-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:47.203817+0000", 2024-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0013192430000000001, 2024-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020410699999999999, 2024-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:05:47.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.203817+0000", 2024-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.203817+0000", 2024-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.203818+0000", 2024-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6499999999999999e-07 2024-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:05:47.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.203824+0000", 2024-09-20T22:05:47.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5370000000000003e-06 2024-09-20T22:05:47.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.266 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:05:47.266 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.203835+0000", 2024-09-20T22:05:47.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0498e-05 2024-09-20T22:05:47.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.266 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:05:47.266 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.203845+0000", 2024-09-20T22:05:47.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0052e-05 2024-09-20T22:05:47.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:05:47.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204013+0000", 2024-09-20T22:05:47.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000168166 2024-09-20T22:05:47.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:05:47.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:05:47.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:05:47.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:47.204035+0000", 2024-09-20T22:05:47.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.228e-05 2024-09-20T22:05:47.267 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.268 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:05:47.268 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.268 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:05:47.268 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:05:47.268 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 12 2024-09-20T22:05:47.268 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:05:47.269 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:05:47.347 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:47.395 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:47.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:47.396 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:47.444 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:47 smithi060 ceph-mon[29694]: pgmap v119: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s; 41700/333861 objects degraded (12.490%); 6964/333861 objects misplaced (2.086%) 2024-09-20T22:05:47.445 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:47 smithi060 ceph-mon[40290]: pgmap v119: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s; 41700/333861 objects degraded (12.490%); 6964/333861 objects misplaced (2.086%) 2024-09-20T22:05:47.497 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:47.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:47.564 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:47.564 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:47.665 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:47.707 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:47.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:47.709 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:47.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:47 smithi067 ceph-mon[33511]: pgmap v119: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s; 41700/333861 objects degraded (12.490%); 6964/333861 objects misplaced (2.086%) 2024-09-20T22:05:47.766 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3, 4, 5, 6, 7] 2024-09-20T22:05:47.766 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:05:47.767 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_0 2024-09-20T22:05:47.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:05:47.811 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:47.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:47.884 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:47.884 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:47.985 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:48.017 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:48.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:48.018 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:48.118 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:48.145 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:48.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:48.146 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:48.247 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:48.276 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:05:48.277 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:48.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:48.279 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:48.379 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:48.421 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:48.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:48.422 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:48.522 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:48.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:48 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1989477218' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:05:48.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:48 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1989477218' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:05:48.573 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:48.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:48.574 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:48.583 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T22:05:48.675 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:48.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:48.709 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:48.709 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:48.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:48 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1989477218' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:05:48.809 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:48.843 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:48.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:48.844 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:48.945 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:49.004 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:49.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:49.005 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:49.097 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:49.106 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:49.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:49.133 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:49.134 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:49.176 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:49.234 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:49.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:49.270 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:49.270 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:49.371 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:49.416 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:49.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:49.428 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:49.428 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:49.507 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:49 smithi060 ceph-mon[29694]: pgmap v120: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s; 41700/333861 objects degraded (12.490%); 6964/333861 objects misplaced (2.086%) 2024-09-20T22:05:49.507 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:49 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:05:49.507 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:49 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1989477218' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:05:49.507 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:49 smithi060 ceph-mon[29694]: osdmap e117: 8 total, 7 up, 8 in 2024-09-20T22:05:49.508 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:49 smithi060 ceph-mon[40290]: pgmap v120: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s; 41700/333861 objects degraded (12.490%); 6964/333861 objects misplaced (2.086%) 2024-09-20T22:05:49.508 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:49 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:05:49.508 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:49 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1989477218' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:05:49.508 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:49 smithi060 ceph-mon[40290]: osdmap e117: 8 total, 7 up, 8 in 2024-09-20T22:05:49.529 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:49.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:49.570 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:49.570 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:49.671 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:49.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:49.715 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:49.715 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:49.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:49 smithi067 ceph-mon[33511]: pgmap v120: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s; 41700/333861 objects degraded (12.490%); 6964/333861 objects misplaced (2.086%) 2024-09-20T22:05:49.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:49 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:05:49.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:49 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1989477218' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:05:49.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:49 smithi067 ceph-mon[33511]: osdmap e117: 8 total, 7 up, 8 in 2024-09-20T22:05:49.816 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:49.849 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:49.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:49.850 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:49.951 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:49.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:49.991 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:49.991 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:50.092 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:50.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:50.123 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:50.123 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:50.224 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:50.251 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:50.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:50.252 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:50.313 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:05:50.313 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:05:50.313 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:05:50.313 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:05:50.313 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:05:50.345 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:05:50.345 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:05:50.346 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:05:50.346 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:05:50.346 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:05:50.346 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:05:50.346 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:05:50.353 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:50.361 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:05:50.361 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:05:50.380 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:50.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:50.381 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:50.400 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:05:50.401 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:05:50.401 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:50 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:05:50.401 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:50 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:05:50.401 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:50 smithi060 ceph-mon[29694]: osdmap e118: 8 total, 7 up, 8 in 2024-09-20T22:05:50.401 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:50 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:50.401 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:50 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:05:50.401 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:50 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2005517616' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:50.402 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:50 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:05:50.402 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:50 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:05:50.402 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:50 smithi060 ceph-mon[40290]: osdmap e118: 8 total, 7 up, 8 in 2024-09-20T22:05:50.402 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:50 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:50.402 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:50 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:05:50.402 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:50 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2005517616' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:50.436 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:05:50.436 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:05:50.467 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:05:50.467 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:05:50.482 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:50.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:50.520 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:50.520 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:50.620 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:50.655 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:50.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:50.656 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:50.757 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:50.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:50 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:05:50.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:50 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:05:50.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:50 smithi067 ceph-mon[33511]: osdmap e118: 8 total, 7 up, 8 in 2024-09-20T22:05:50.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:50 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:05:50.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:50 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:05:50.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:50 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2005517616' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:50.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:50.795 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:50.795 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:50.896 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:50.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:50.943 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:50.943 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:51.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:05:51.044 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:51.075 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:51.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:51.076 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:51.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:51.219 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:51.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:51.220 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:51.321 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:51.389 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:51.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:51.390 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:51.491 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:51.502 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:05:51.502 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:05:51.520 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:51.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:51.521 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:51.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:51 smithi060 ceph-mon[29694]: pgmap v123: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 764 B/s rd, 0 B/s wr, 950 op/s; 41700/333861 objects degraded (12.490%); 6964/333861 objects misplaced (2.086%) 2024-09-20T22:05:51.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:51 smithi060 ceph-mon[40290]: pgmap v123: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 764 B/s rd, 0 B/s wr, 950 op/s; 41700/333861 objects degraded (12.490%); 6964/333861 objects misplaced (2.086%) 2024-09-20T22:05:51.560 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:05:51.561 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:05:51.622 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:51.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:51.664 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:51.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:51.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:51 smithi067 ceph-mon[33511]: pgmap v123: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.8 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 764 B/s rd, 0 B/s wr, 950 op/s; 41700/333861 objects degraded (12.490%); 6964/333861 objects misplaced (2.086%) 2024-09-20T22:05:51.766 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:51.798 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:51.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:51.799 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:51.900 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:51.936 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:51.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:51.937 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:52.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:52.061 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:52.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:52.062 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:52.163 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:52.195 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:52.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:52.196 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:52.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:05:51 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:05:52.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:52.319 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:52.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:52.320 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:52.421 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:52.449 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:52.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:52.451 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:52.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:52 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 41700/333861 objects degraded (12.490%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:52.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:52 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 41700/333861 objects degraded (12.490%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:52.551 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:52.581 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:52.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:52.582 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:52.683 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:52.712 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:52.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:52.714 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:52 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 41700/333861 objects degraded (12.490%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:52.814 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:52.843 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:52.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:52.844 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:52.945 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:52.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:52.975 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:53.075 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:53.103 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:53.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:53.104 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:53.205 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:53.234 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:53.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:53.235 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:53.336 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:53.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:53.379 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:53.379 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:53.405 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T22:05:53.480 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:53.503 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:05:53.506 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:53.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:53.507 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:53.608 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:53.635 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:53.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:53.636 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:53.736 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:53 smithi067 ceph-mon[33511]: from='client.24772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:05:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:53 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:05:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:53 smithi067 ceph-mon[33511]: pgmap v124: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.4 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.55k op/s; 39398/315612 objects degraded (12.483%); 6583/315612 objects misplaced (2.086%) 2024-09-20T22:05:53.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:53 smithi060 ceph-mon[29694]: from='client.24772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:05:53.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:53 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:05:53.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:53 smithi060 ceph-mon[29694]: pgmap v124: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.4 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.55k op/s; 39398/315612 objects degraded (12.483%); 6583/315612 objects misplaced (2.086%) 2024-09-20T22:05:53.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:53 smithi060 ceph-mon[40290]: from='client.24772 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:05:53.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:53 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:05:53.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:53 smithi060 ceph-mon[40290]: pgmap v124: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.4 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.55k op/s; 39398/315612 objects degraded (12.483%); 6583/315612 objects misplaced (2.086%) 2024-09-20T22:05:53.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:53.809 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:53.809 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:53.834 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:53.878 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-20T22:05:49.937258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7280794163,"num_objects":111287,"num_object_clones":0,"num_object_copies":333861,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41700,"num_objects_misplaced":6964,"num_objects_unfound":0,"num_objects_dirty":111287,"num_whiteouts":0,"num_read":1095,"num_read_kb":941,"num_write":155589,"num_write_kb":8516382,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92480,"ondisk_log_size":92480,"up":378,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":449,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":20526212,"kb_used_data":19732348,"kb_used_omap":12,"kb_used_meta":793523,"kb_avail":729435004,"statfs":{"total":767960285184,"available":746941444096,"internally_reserved":0,"allocated":20205924352,"data_stored":20202467906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":812568112},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,1,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-416808960,"num_objects":-6360,"num_object_clones":0,"num_object_copies":-19080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2313,"num_objects_misplaced":-392,"num_objects_unfound":0,"num_objects_dirty":-6360,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6360,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.698123"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":123,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.263237+0000","last_change":"2024-09-20T22:04:21.109795+0000","last_active":"2024-09-20T22:05:43.263237+0000","last_peered":"2024-09-20T22:05:43.263237+0000","last_clean":"2024-09-20T22:05:43.263237+0000","last_became_active":"2024-09-20T22:02:50.109930+0000","last_became_peered":"2024-09-20T22:02:50.109930+0000","last_unstale":"2024-09-20T22:05:43.263237+0000","last_undegraded":"2024-09-20T22:05:43.263237+0000","last_fullsized":"2024-09-20T22:05:43.263237+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:21.109683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:21.109683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"71'9","reported_seq":139,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.240893+0000","last_change":"2024-09-20T22:04:16.312321+0000","last_active":"2024-09-20T22:05:43.240893+0000","last_peered":"2024-09-20T22:05:43.240893+0000","last_clean":"2024-09-20T22:04:11.302960+0000","last_became_active":"2024-09-20T22:04:16.312321+0000","last_became_peered":"2024-09-20T22:04:16.312321+0000","last_unstale":"2024-09-20T22:05:43.240893+0000","last_undegraded":"2024-09-20T22:04:16.304969+0000","last_fullsized":"2024-09-20T22:04:16.304866+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:10.928348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:10.928348+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[{"shards":"4,6","objects":4}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":57,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.253318+0000","last_change":"2024-09-20T22:04:16.315219+0000","last_active":"2024-09-20T22:05:43.253318+0000","last_peered":"2024-09-20T22:05:43.253318+0000","last_clean":"2024-09-20T22:02:53.269054+0000","last_became_active":"2024-09-20T22:04:16.315219+0000","last_became_peered":"2024-09-20T22:04:16.315219+0000","last_unstale":"2024-09-20T22:05:43.253318+0000","last_undegraded":"2024-09-20T22:05:43.253318+0000","last_fullsized":"2024-09-20T22:04:16.308508+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:11.178591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:11.178591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:43.467052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":107,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.240780+0000","last_change":"2024-09-20T22:04:16.315650+0000","last_active":"2024-09-20T22:05:43.240780+0000","last_peered":"2024-09-20T22:05:43.240780+0000","last_clean":"2024-09-20T22:04:11.303058+0000","last_became_active":"2024-09-20T22:04:16.315650+0000","last_became_peered":"2024-09-20T22:04:16.315650+0000","last_unstale":"2024-09-20T22:05:43.240780+0000","last_undegraded":"2024-09-20T22:05:43.240780+0000","last_fullsized":"2024-09-20T22:04:16.303835+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":112,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.260588+0000","last_change":"2024-09-20T22:05:06.024320+0000","last_active":"2024-09-20T22:05:43.260588+0000","last_peered":"2024-09-20T22:05:43.260588+0000","last_clean":"2024-09-20T22:05:43.260588+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:05:43.260588+0000","last_undegraded":"2024-09-20T22:05:43.260588+0000","last_fullsized":"2024-09-20T22:05:43.260588+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:05:06.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:06.024051+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:11.371826+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":123,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.259353+0000","last_change":"2024-09-20T22:05:04.976359+0000","last_active":"2024-09-20T22:05:43.259353+0000","last_peered":"2024-09-20T22:05:43.259353+0000","last_clean":"2024-09-20T22:05:43.259353+0000","last_became_active":"2024-09-20T22:02:50.107154+0000","last_became_peered":"2024-09-20T22:02:50.107154+0000","last_unstale":"2024-09-20T22:05:43.259353+0000","last_undegraded":"2024-09-20T22:05:43.259353+0000","last_fullsized":"2024-09-20T22:05:43.259353+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:04.976218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:04.976218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:27.101580+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"71'15","reported_seq":144,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.258801+0000","last_change":"2024-09-20T22:04:15.767473+0000","last_active":"2024-09-20T22:05:43.258801+0000","last_peered":"2024-09-20T22:05:43.258801+0000","last_clean":"2024-09-20T22:05:43.258801+0000","last_became_active":"2024-09-20T22:02:52.282101+0000","last_became_peered":"2024-09-20T22:02:52.282101+0000","last_unstale":"2024-09-20T22:05:43.258801+0000","last_undegraded":"2024-09-20T22:05:43.258801+0000","last_fullsized":"2024-09-20T22:05:43.258801+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:04:15.767389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:15.767389+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":111,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.240612+0000","last_change":"2024-09-20T22:04:16.312826+0000","last_active":"2024-09-20T22:05:43.240612+0000","last_peered":"2024-09-20T22:05:43.240612+0000","last_clean":"2024-09-20T22:04:11.302880+0000","last_became_active":"2024-09-20T22:04:16.312826+0000","last_became_peered":"2024-09-20T22:04:16.312826+0000","last_unstale":"2024-09-20T22:05:43.240612+0000","last_undegraded":"2024-09-20T22:05:43.240612+0000","last_fullsized":"2024-09-20T22:04:16.303524+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":131,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.255405+0000","last_change":"2024-09-20T22:05:28.665445+0000","last_active":"2024-09-20T22:05:43.255405+0000","last_peered":"2024-09-20T22:05:43.255405+0000","last_clean":"2024-09-20T22:05:43.255405+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:05:43.255405+0000","last_undegraded":"2024-09-20T22:05:43.255405+0000","last_fullsized":"2024-09-20T22:05:43.255405+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.665329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.665329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:34.569519+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"71'9","reported_seq":136,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.241170+0000","last_change":"2024-09-20T22:04:52.800006+0000","last_active":"2024-09-20T22:05:43.241170+0000","last_peered":"2024-09-20T22:05:43.241170+0000","last_clean":"2024-09-20T22:05:43.241170+0000","last_became_active":"2024-09-20T22:02:52.279526+0000","last_became_peered":"2024-09-20T22:02:52.279526+0000","last_unstale":"2024-09-20T22:05:43.241170+0000","last_undegraded":"2024-09-20T22:05:43.241170+0000","last_fullsized":"2024-09-20T22:05:43.241170+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:52.799870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:52.799870+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:54.504353+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":115,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.253812+0000","last_change":"2024-09-20T22:05:02.519662+0000","last_active":"2024-09-20T22:05:43.253812+0000","last_peered":"2024-09-20T22:05:43.253812+0000","last_clean":"2024-09-20T22:05:43.253812+0000","last_became_active":"2024-09-20T22:02:54.292911+0000","last_became_peered":"2024-09-20T22:02:54.292911+0000","last_unstale":"2024-09-20T22:05:43.253812+0000","last_undegraded":"2024-09-20T22:05:43.253812+0000","last_fullsized":"2024-09-20T22:05:43.253812+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:02.519415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:02.519415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:04.396928+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":106,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.255652+0000","last_change":"2024-09-20T22:04:16.314659+0000","last_active":"2024-09-20T22:05:43.255652+0000","last_peered":"2024-09-20T22:05:43.255652+0000","last_clean":"2024-09-20T22:04:11.306271+0000","last_became_active":"2024-09-20T22:04:16.314659+0000","last_became_peered":"2024-09-20T22:04:16.314659+0000","last_unstale":"2024-09-20T22:05:43.255652+0000","last_undegraded":"2024-09-20T22:05:43.255652+0000","last_fullsized":"2024-09-20T22:04:16.302831+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:47.557411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":131,"reported_epoch":116,"state":"active+clean+remapped","last_fresh":"2024-09-20T22:05:43.253981+0000","last_change":"2024-09-20T22:04:23.456489+0000","last_active":"2024-09-20T22:05:43.253981+0000","last_peered":"2024-09-20T22:05:43.253981+0000","last_clean":"2024-09-20T22:05:43.253981+0000","last_became_active":"2024-09-20T22:04:21.311111+0000","last_became_peered":"2024-09-20T22:04:21.311111+0000","last_unstale":"2024-09-20T22:05:43.253981+0000","last_undegraded":"2024-09-20T22:05:43.253981+0000","last_fullsized":"2024-09-20T22:05:43.253981+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:23.456365+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:23.456365+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:46.532139+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2,4],"avail_no_missing":["5","2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"71'5","reported_seq":133,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.244150+0000","last_change":"2024-09-20T22:04:16.321444+0000","last_active":"2024-09-20T22:05:43.244150+0000","last_peered":"2024-09-20T22:05:43.244150+0000","last_clean":"2024-09-20T22:04:13.950323+0000","last_became_active":"2024-09-20T22:04:16.321444+0000","last_became_peered":"2024-09-20T22:04:16.321444+0000","last_unstale":"2024-09-20T22:05:43.244150+0000","last_undegraded":"2024-09-20T22:04:16.304418+0000","last_fullsized":"2024-09-20T22:04:16.304302+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'5","last_scrub_stamp":"2024-09-20T22:04:13.950197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:13.950197+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[{"shards":"3,4","objects":2}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":119,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.257409+0000","last_change":"2024-09-20T22:04:16.316889+0000","last_active":"2024-09-20T22:05:43.257409+0000","last_peered":"2024-09-20T22:05:43.257409+0000","last_clean":"2024-09-20T22:04:14.618055+0000","last_became_active":"2024-09-20T22:04:16.316889+0000","last_became_peered":"2024-09-20T22:04:16.316889+0000","last_unstale":"2024-09-20T22:05:43.257409+0000","last_undegraded":"2024-09-20T22:05:43.257409+0000","last_fullsized":"2024-09-20T22:04:16.310076+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:14.617830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:14.617830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:44.798913+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":60,"reported_epoch":116,"state":"active+clean+remapped","last_fresh":"2024-09-20T22:05:43.255399+0000","last_change":"2024-09-20T22:05:29.551389+0000","last_active":"2024-09-20T22:05:43.255399+0000","last_peered":"2024-09-20T22:05:43.255399+0000","last_clean":"2024-09-20T22:05:43.255399+0000","last_became_active":"2024-09-20T22:04:21.312179+0000","last_became_peered":"2024-09-20T22:04:21.312179+0000","last_unstale":"2024-09-20T22:05:43.255399+0000","last_undegraded":"2024-09-20T22:05:43.255399+0000","last_fullsized":"2024-09-20T22:05:43.255399+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:05:29.551172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:39.943718+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6,3],"avail_no_missing":["0","3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":112,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.241548+0000","last_change":"2024-09-20T22:04:59.903000+0000","last_active":"2024-09-20T22:05:43.241548+0000","last_peered":"2024-09-20T22:05:43.241548+0000","last_clean":"2024-09-20T22:05:43.241548+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:05:43.241548+0000","last_undegraded":"2024-09-20T22:05:43.241548+0000","last_fullsized":"2024-09-20T22:05:43.241548+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:59.902900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:59.902900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:03.610127+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":123,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.258754+0000","last_change":"2024-09-20T22:05:31.552599+0000","last_active":"2024-09-20T22:05:43.258754+0000","last_peered":"2024-09-20T22:05:43.258754+0000","last_clean":"2024-09-20T22:05:43.258754+0000","last_became_active":"2024-09-20T22:02:50.111517+0000","last_became_peered":"2024-09-20T22:02:50.111517+0000","last_unstale":"2024-09-20T22:05:43.258754+0000","last_undegraded":"2024-09-20T22:05:43.258754+0000","last_fullsized":"2024-09-20T22:05:43.258754+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:31.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:31.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:44.579423+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"71'15","reported_seq":147,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.247057+0000","last_change":"2024-09-20T22:04:16.311393+0000","last_active":"2024-09-20T22:05:43.247057+0000","last_peered":"2024-09-20T22:05:43.247057+0000","last_clean":"2024-09-20T22:04:11.306869+0000","last_became_active":"2024-09-20T22:04:16.311393+0000","last_became_peered":"2024-09-20T22:04:16.311393+0000","last_unstale":"2024-09-20T22:05:43.247057+0000","last_undegraded":"2024-09-20T22:04:16.306249+0000","last_fullsized":"2024-09-20T22:04:16.306143+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:04:10.667027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:10.667027+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":57,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.242249+0000","last_change":"2024-09-20T22:04:16.313788+0000","last_active":"2024-09-20T22:05:43.242249+0000","last_peered":"2024-09-20T22:05:43.242249+0000","last_clean":"2024-09-20T22:02:53.269054+0000","last_became_active":"2024-09-20T22:04:16.313788+0000","last_became_peered":"2024-09-20T22:04:16.313788+0000","last_unstale":"2024-09-20T22:05:43.242249+0000","last_undegraded":"2024-09-20T22:05:43.242249+0000","last_fullsized":"2024-09-20T22:04:16.304888+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":111,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.258002+0000","last_change":"2024-09-20T22:04:10.722434+0000","last_active":"2024-09-20T22:05:43.258002+0000","last_peered":"2024-09-20T22:05:43.258002+0000","last_clean":"2024-09-20T22:05:43.258002+0000","last_became_active":"2024-09-20T22:02:56.347561+0000","last_became_peered":"2024-09-20T22:02:56.347561+0000","last_unstale":"2024-09-20T22:05:43.258002+0000","last_undegraded":"2024-09-20T22:05:43.258002+0000","last_fullsized":"2024-09-20T22:05:43.258002+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:10.722324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:10.722324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":60,"reported_epoch":116,"state":"active+clean+remapped","last_fresh":"2024-09-20T22:05:43.253126+0000","last_change":"2024-09-20T22:04:53.467338+0000","last_active":"2024-09-20T22:05:43.253126+0000","last_peered":"2024-09-20T22:05:43.253126+0000","last_clean":"2024-09-20T22:05:43.253126+0000","last_became_active":"2024-09-20T22:04:21.311331+0000","last_became_peered":"2024-09-20T22:04:21.311331+0000","last_unstale":"2024-09-20T22:05:43.253126+0000","last_undegraded":"2024-09-20T22:05:43.253126+0000","last_fullsized":"2024-09-20T22:05:43.253126+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:53.467207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:53.467207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:13.830247+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2,4],"avail_no_missing":["5","2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"71'12","reported_seq":137,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.257670+0000","last_change":"2024-09-20T22:04:16.321585+0000","last_active":"2024-09-20T22:05:43.257670+0000","last_peered":"2024-09-20T22:05:43.257670+0000","last_clean":"2024-09-20T22:04:11.310784+0000","last_became_active":"2024-09-20T22:04:16.321585+0000","last_became_peered":"2024-09-20T22:04:16.321585+0000","last_unstale":"2024-09-20T22:05:43.257670+0000","last_undegraded":"2024-09-20T22:04:16.306903+0000","last_fullsized":"2024-09-20T22:04:16.306792+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[{"shards":"3,4","objects":5}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":116,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.240724+0000","last_change":"2024-09-20T22:05:28.003302+0000","last_active":"2024-09-20T22:05:43.240724+0000","last_peered":"2024-09-20T22:05:43.240724+0000","last_clean":"2024-09-20T22:05:43.240724+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:05:43.240724+0000","last_undegraded":"2024-09-20T22:05:43.240724+0000","last_fullsized":"2024-09-20T22:05:43.240724+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.003183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:33.356060+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":114,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.257372+0000","last_change":"2024-09-20T22:05:33.751965+0000","last_active":"2024-09-20T22:05:43.257372+0000","last_peered":"2024-09-20T22:05:43.257372+0000","last_clean":"2024-09-20T22:05:43.257372+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:05:43.257372+0000","last_undegraded":"2024-09-20T22:05:43.257372+0000","last_fullsized":"2024-09-20T22:05:43.257372+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:05:33.751883+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:00.978967+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"64'1","reported_seq":66,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.258443+0000","last_change":"2024-09-20T22:04:16.322089+0000","last_active":"2024-09-20T22:05:43.258443+0000","last_peered":"2024-09-20T22:05:43.258443+0000","last_clean":"2024-09-20T22:02:50.113625+0000","last_became_active":"2024-09-20T22:04:16.322089+0000","last_became_peered":"2024-09-20T22:04:16.322089+0000","last_unstale":"2024-09-20T22:05:43.258443+0000","last_undegraded":"2024-09-20T22:04:16.305892+0000","last_fullsized":"2024-09-20T22:04:16.305784+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[{"shards":"3,4","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"71'10","reported_seq":134,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.263612+0000","last_change":"2024-09-20T22:04:56.414262+0000","last_active":"2024-09-20T22:05:43.263612+0000","last_peered":"2024-09-20T22:05:43.263612+0000","last_clean":"2024-09-20T22:05:43.263612+0000","last_became_active":"2024-09-20T22:02:52.481980+0000","last_became_peered":"2024-09-20T22:02:52.481980+0000","last_unstale":"2024-09-20T22:05:43.263612+0000","last_undegraded":"2024-09-20T22:05:43.263612+0000","last_fullsized":"2024-09-20T22:05:43.263612+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:04:56.414020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:56.414020+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:57.220359+0000","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"71'8","reported_seq":162,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.255090+0000","last_change":"2024-09-20T22:05:01.351815+0000","last_active":"2024-09-20T22:05:43.255090+0000","last_peered":"2024-09-20T22:05:43.255090+0000","last_clean":"2024-09-20T22:05:43.255090+0000","last_became_active":"2024-09-20T22:02:54.297657+0000","last_became_peered":"2024-09-20T22:02:54.297657+0000","last_unstale":"2024-09-20T22:05:43.255090+0000","last_undegraded":"2024-09-20T22:05:43.255090+0000","last_fullsized":"2024-09-20T22:05:43.255090+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:05:01.351702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:01.351702+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:23.580653+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"116'9597","reported_seq":9752,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:45.598913+0000","last_change":"2024-09-20T22:05:04.128982+0000","last_active":"2024-09-20T22:05:45.598913+0000","last_peered":"2024-09-20T22:05:45.598913+0000","last_clean":"2024-09-20T22:05:45.598913+0000","last_became_active":"2024-09-20T22:03:52.318259+0000","last_became_peered":"2024-09-20T22:03:52.318259+0000","last_unstale":"2024-09-20T22:05:45.598913+0000","last_undegraded":"2024-09-20T22:05:45.598913+0000","last_fullsized":"2024-09-20T22:05:45.598913+0000","mapping_epoch":80,"log_start":"98'3500","ondisk_log_start":"98'3500","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'6604","last_scrub_stamp":"2024-09-20T22:05:04.128810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:05:04.128810+0000","objects_scrubbed":6541,"log_size":6097,"log_dups_size":0,"ondisk_log_size":6097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:10.252483+0000","scrub_duration":1801,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":451346432,"num_objects":6887,"num_object_clones":0,"num_object_copies":20661,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9597,"num_write_kb":527488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"71'15","reported_seq":101,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.258174+0000","last_change":"2024-09-20T22:04:16.323258+0000","last_active":"2024-09-20T22:05:43.258174+0000","last_peered":"2024-09-20T22:05:43.258174+0000","last_clean":"2024-09-20T22:02:57.665294+0000","last_became_active":"2024-09-20T22:04:16.323258+0000","last_became_peered":"2024-09-20T22:04:16.323258+0000","last_unstale":"2024-09-20T22:05:43.258174+0000","last_undegraded":"2024-09-20T22:04:16.312038+0000","last_fullsized":"2024-09-20T22:04:16.311916+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:04:13.235907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:13.235907+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[{"shards":"3,4","objects":7}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":118,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.260628+0000","last_change":"2024-09-20T22:04:16.317334+0000","last_active":"2024-09-20T22:05:43.260628+0000","last_peered":"2024-09-20T22:05:43.260628+0000","last_clean":"2024-09-20T22:04:11.312325+0000","last_became_active":"2024-09-20T22:04:16.317334+0000","last_became_peered":"2024-09-20T22:04:16.317334+0000","last_unstale":"2024-09-20T22:05:43.260628+0000","last_undegraded":"2024-09-20T22:05:43.260628+0000","last_fullsized":"2024-09-20T22:04:16.308050+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"71'8","reported_seq":163,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.241690+0000","last_change":"2024-09-20T22:04:18.813933+0000","last_active":"2024-09-20T22:05:43.241690+0000","last_peered":"2024-09-20T22:05:43.241690+0000","last_clean":"2024-09-20T22:05:43.241690+0000","last_became_active":"2024-09-20T22:02:54.294917+0000","last_became_peered":"2024-09-20T22:02:54.294917+0000","last_unstale":"2024-09-20T22:05:43.241690+0000","last_undegraded":"2024-09-20T22:05:43.241690+0000","last_fullsized":"2024-09-20T22:05:43.241690+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:04:18.813813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:18.813813+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":60,"reported_epoch":116,"state":"active+clean+remapped","last_fresh":"2024-09-20T22:05:43.254481+0000","last_change":"2024-09-20T22:04:21.515875+0000","last_active":"2024-09-20T22:05:43.254481+0000","last_peered":"2024-09-20T22:05:43.254481+0000","last_clean":"2024-09-20T22:05:43.254481+0000","last_became_active":"2024-09-20T22:04:21.310424+0000","last_became_peered":"2024-09-20T22:04:21.310424+0000","last_unstale":"2024-09-20T22:05:43.254481+0000","last_undegraded":"2024-09-20T22:05:43.254481+0000","last_fullsized":"2024-09-20T22:05:43.254481+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:21.515698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:21.515698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0,5],"avail_no_missing":["6","0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"116'9684","reported_seq":9861,"reported_epoch":116,"state":"active+clean+remapped","last_fresh":"2024-09-20T22:05:45.770459+0000","last_change":"2024-09-20T22:05:27.253791+0000","last_active":"2024-09-20T22:05:45.770459+0000","last_peered":"2024-09-20T22:05:45.770459+0000","last_clean":"2024-09-20T22:05:45.770459+0000","last_became_active":"2024-09-20T22:04:21.311038+0000","last_became_peered":"2024-09-20T22:04:21.311038+0000","last_unstale":"2024-09-20T22:05:45.770459+0000","last_undegraded":"2024-09-20T22:05:45.770459+0000","last_fullsized":"2024-09-20T22:05:45.770459+0000","mapping_epoch":95,"log_start":"92'2700","ondisk_log_start":"92'2700","created":80,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"112'8323","last_scrub_stamp":"2024-09-20T22:05:27.253221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:05:27.253221+0000","objects_scrubbed":8198,"log_size":6984,"log_dups_size":0,"ondisk_log_size":6984,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:52.100681+0000","scrub_duration":3507,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":456392704,"num_objects":6964,"num_object_clones":0,"num_object_copies":20892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":6964,"num_objects_unfound":0,"num_objects_dirty":6964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9684,"num_write_kb":532736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4,3],"avail_no_missing":["2","3","4"],"object_location_counts":[{"shards":"2,3,4","objects":6964}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"71'18","reported_seq":151,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.260343+0000","last_change":"2024-09-20T22:04:57.967494+0000","last_active":"2024-09-20T22:05:43.260343+0000","last_peered":"2024-09-20T22:05:43.260343+0000","last_clean":"2024-09-20T22:05:43.260343+0000","last_became_active":"2024-09-20T22:02:52.482119+0000","last_became_peered":"2024-09-20T22:02:52.482119+0000","last_unstale":"2024-09-20T22:05:43.260343+0000","last_undegraded":"2024-09-20T22:05:43.260343+0000","last_fullsized":"2024-09-20T22:05:43.260343+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'18","last_scrub_stamp":"2024-09-20T22:04:57.967216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:57.967216+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:07.936978+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":123,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.258943+0000","last_change":"2024-09-20T22:05:02.924002+0000","last_active":"2024-09-20T22:05:43.258943+0000","last_peered":"2024-09-20T22:05:43.258943+0000","last_clean":"2024-09-20T22:05:43.258943+0000","last_became_active":"2024-09-20T22:02:50.113076+0000","last_became_peered":"2024-09-20T22:02:50.113076+0000","last_unstale":"2024-09-20T22:05:43.258943+0000","last_undegraded":"2024-09-20T22:05:43.258943+0000","last_fullsized":"2024-09-20T22:05:43.258943+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:02.923565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:02.923565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:29.688625+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":116,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.241330+0000","last_change":"2024-09-20T22:04:20.841355+0000","last_active":"2024-09-20T22:05:43.241330+0000","last_peered":"2024-09-20T22:05:43.241330+0000","last_clean":"2024-09-20T22:05:43.241330+0000","last_became_active":"2024-09-20T22:02:54.288007+0000","last_became_peered":"2024-09-20T22:02:54.288007+0000","last_unstale":"2024-09-20T22:05:43.241330+0000","last_undegraded":"2024-09-20T22:05:43.241330+0000","last_fullsized":"2024-09-20T22:05:43.241330+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:20.841289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:20.841289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:45.374237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":120,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.241059+0000","last_change":"2024-09-20T22:05:29.990030+0000","last_active":"2024-09-20T22:05:43.241059+0000","last_peered":"2024-09-20T22:05:43.241059+0000","last_clean":"2024-09-20T22:05:43.241059+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:05:43.241059+0000","last_undegraded":"2024-09-20T22:05:43.241059+0000","last_fullsized":"2024-09-20T22:05:43.241059+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:29.989934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:29.989934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:34.056485+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"116'9703","reported_seq":9787,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:45.781146+0000","last_change":"2024-09-20T22:04:16.318737+0000","last_active":"2024-09-20T22:05:45.781146+0000","last_peered":"2024-09-20T22:05:45.781146+0000","last_clean":"2024-09-20T22:04:15.696640+0000","last_became_active":"2024-09-20T22:04:16.318737+0000","last_became_peered":"2024-09-20T22:04:16.318737+0000","last_unstale":"2024-09-20T22:05:45.781146+0000","last_undegraded":"2024-09-20T22:04:16.307224+0000","last_fullsized":"2024-09-20T22:04:16.307107+0000","mapping_epoch":91,"log_start":"102'5000","ondisk_log_start":"102'5000","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":4703,"log_dups_size":0,"ondisk_log_size":4703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:49.060831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":450428928,"num_objects":6873,"num_object_clones":0,"num_object_copies":20619,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6873,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9703,"num_write_kb":530432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":6873}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"71'14","reported_seq":141,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.241575+0000","last_change":"2024-09-20T22:04:16.897278+0000","last_active":"2024-09-20T22:05:43.241575+0000","last_peered":"2024-09-20T22:05:43.241575+0000","last_clean":"2024-09-20T22:05:43.241575+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:05:43.241575+0000","last_undegraded":"2024-09-20T22:05:43.241575+0000","last_fullsized":"2024-09-20T22:05:43.241575+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'14","last_scrub_stamp":"2024-09-20T22:04:16.897152+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:16.897152+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":119,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.263816+0000","last_change":"2024-09-20T22:04:16.313432+0000","last_active":"2024-09-20T22:05:43.263816+0000","last_peered":"2024-09-20T22:05:43.263816+0000","last_clean":"2024-09-20T22:04:11.308782+0000","last_became_active":"2024-09-20T22:04:16.313432+0000","last_became_peered":"2024-09-20T22:04:16.313432+0000","last_unstale":"2024-09-20T22:05:43.263816+0000","last_undegraded":"2024-09-20T22:04:16.305282+0000","last_fullsized":"2024-09-20T22:04:16.305176+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:47.557361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[{"shards":"0,4","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"92'9","reported_seq":155,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.259208+0000","last_change":"2024-09-20T22:04:16.319525+0000","last_active":"2024-09-20T22:05:43.259208+0000","last_peered":"2024-09-20T22:05:43.259208+0000","last_clean":"2024-09-20T22:04:15.181371+0000","last_became_active":"2024-09-20T22:04:16.319525+0000","last_became_peered":"2024-09-20T22:04:16.319525+0000","last_unstale":"2024-09-20T22:05:43.259208+0000","last_undegraded":"2024-09-20T22:04:16.305343+0000","last_fullsized":"2024-09-20T22:04:16.305233+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:49.060706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[{"shards":"3,4","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":112,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.257081+0000","last_change":"2024-09-20T22:04:56.605725+0000","last_active":"2024-09-20T22:05:43.257081+0000","last_peered":"2024-09-20T22:05:43.257081+0000","last_clean":"2024-09-20T22:05:43.257081+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:05:43.257081+0000","last_undegraded":"2024-09-20T22:05:43.257081+0000","last_fullsized":"2024-09-20T22:05:43.257081+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:56.605594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:56.605594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:21.394492+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"116'9880","reported_seq":10009,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:45.563718+0000","last_change":"2024-09-20T22:05:36.236684+0000","last_active":"2024-09-20T22:05:45.563718+0000","last_peered":"2024-09-20T22:05:45.563718+0000","last_clean":"2024-09-20T22:05:45.563718+0000","last_became_active":"2024-09-20T22:03:52.308178+0000","last_became_peered":"2024-09-20T22:03:52.308178+0000","last_unstale":"2024-09-20T22:05:45.563718+0000","last_undegraded":"2024-09-20T22:05:45.563718+0000","last_fullsized":"2024-09-20T22:05:45.563718+0000","mapping_epoch":80,"log_start":"102'5100","ondisk_log_start":"102'5100","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":4224,"log_size":4780,"log_dups_size":0,"ondisk_log_size":4780,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463077376,"num_objects":7066,"num_object_clones":0,"num_object_copies":21198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9880,"num_write_kb":542272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"71'10","reported_seq":91,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.253622+0000","last_change":"2024-09-20T22:04:16.317022+0000","last_active":"2024-09-20T22:05:43.253622+0000","last_peered":"2024-09-20T22:05:43.253622+0000","last_clean":"2024-09-20T22:02:57.633971+0000","last_became_active":"2024-09-20T22:04:16.317022+0000","last_became_peered":"2024-09-20T22:04:16.317022+0000","last_unstale":"2024-09-20T22:05:43.253622+0000","last_undegraded":"2024-09-20T22:04:16.307899+0000","last_fullsized":"2024-09-20T22:04:16.307784+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:43.467037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[{"shards":"4,5","objects":5}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":131,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.255618+0000","last_change":"2024-09-20T22:05:22.673440+0000","last_active":"2024-09-20T22:05:43.255618+0000","last_peered":"2024-09-20T22:05:43.255618+0000","last_clean":"2024-09-20T22:05:43.255618+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:05:43.255618+0000","last_undegraded":"2024-09-20T22:05:43.255618+0000","last_fullsized":"2024-09-20T22:05:43.255618+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:22.673310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:22.673310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:28.579242+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"71'8","reported_seq":159,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.258025+0000","last_change":"2024-09-20T22:04:20.160515+0000","last_active":"2024-09-20T22:05:43.258025+0000","last_peered":"2024-09-20T22:05:43.258025+0000","last_clean":"2024-09-20T22:05:43.258025+0000","last_became_active":"2024-09-20T22:02:54.292819+0000","last_became_peered":"2024-09-20T22:02:54.292819+0000","last_unstale":"2024-09-20T22:05:43.258025+0000","last_undegraded":"2024-09-20T22:05:43.258025+0000","last_fullsized":"2024-09-20T22:05:43.258025+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:04:20.160405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:20.160405+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":103,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.255872+0000","last_change":"2024-09-20T22:02:56.304595+0000","last_active":"2024-09-20T22:05:43.255872+0000","last_peered":"2024-09-20T22:05:43.255872+0000","last_clean":"2024-09-20T22:05:43.255872+0000","last_became_active":"2024-09-20T22:02:56.304274+0000","last_became_peered":"2024-09-20T22:02:56.304274+0000","last_unstale":"2024-09-20T22:05:43.255872+0000","last_undegraded":"2024-09-20T22:05:43.255872+0000","last_fullsized":"2024-09-20T22:05:43.255872+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"116'9665","reported_seq":9749,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:45.510977+0000","last_change":"2024-09-20T22:04:16.322804+0000","last_active":"2024-09-20T22:05:45.510977+0000","last_peered":"2024-09-20T22:05:45.510977+0000","last_clean":"2024-09-20T22:04:15.697798+0000","last_became_active":"2024-09-20T22:04:16.322804+0000","last_became_peered":"2024-09-20T22:04:16.322804+0000","last_unstale":"2024-09-20T22:05:45.510977+0000","last_undegraded":"2024-09-20T22:04:16.308985+0000","last_fullsized":"2024-09-20T22:04:16.308869+0000","mapping_epoch":91,"log_start":"98'3700","ondisk_log_start":"98'3700","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":5965,"log_dups_size":0,"ondisk_log_size":5965,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:49.409151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":463273984,"num_objects":7069,"num_object_clones":0,"num_object_copies":21207,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7069,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9665,"num_write_kb":535488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[{"shards":"3,5","objects":7069}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"71'19","reported_seq":163,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.258277+0000","last_change":"2024-09-20T22:05:24.600933+0000","last_active":"2024-09-20T22:05:43.258277+0000","last_peered":"2024-09-20T22:05:43.258277+0000","last_clean":"2024-09-20T22:05:43.258277+0000","last_became_active":"2024-09-20T22:02:52.480077+0000","last_became_peered":"2024-09-20T22:02:52.480077+0000","last_unstale":"2024-09-20T22:05:43.258277+0000","last_undegraded":"2024-09-20T22:05:43.258277+0000","last_fullsized":"2024-09-20T22:05:43.258277+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:05:24.600795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:24.600795+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:42.120234+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":58,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.247228+0000","last_change":"2024-09-20T22:04:16.311744+0000","last_active":"2024-09-20T22:05:43.247228+0000","last_peered":"2024-09-20T22:05:43.247228+0000","last_clean":"2024-09-20T22:02:49.083871+0000","last_became_active":"2024-09-20T22:04:16.311744+0000","last_became_peered":"2024-09-20T22:04:16.311744+0000","last_unstale":"2024-09-20T22:05:43.247228+0000","last_undegraded":"2024-09-20T22:05:43.247228+0000","last_fullsized":"2024-09-20T22:04:16.306448+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":123,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.252445+0000","last_change":"2024-09-20T22:05:30.141747+0000","last_active":"2024-09-20T22:05:43.252445+0000","last_peered":"2024-09-20T22:05:43.252445+0000","last_clean":"2024-09-20T22:05:43.252445+0000","last_became_active":"2024-09-20T22:02:54.290787+0000","last_became_peered":"2024-09-20T22:02:54.290787+0000","last_unstale":"2024-09-20T22:05:43.252445+0000","last_undegraded":"2024-09-20T22:05:43.252445+0000","last_fullsized":"2024-09-20T22:05:43.252445+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:30.141642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:30.141642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:49.552221+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":57,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.253885+0000","last_change":"2024-09-20T22:04:16.319315+0000","last_active":"2024-09-20T22:05:43.253885+0000","last_peered":"2024-09-20T22:05:43.253885+0000","last_clean":"2024-09-20T22:02:55.278800+0000","last_became_active":"2024-09-20T22:04:16.319315+0000","last_became_peered":"2024-09-20T22:04:16.319315+0000","last_unstale":"2024-09-20T22:05:43.253885+0000","last_undegraded":"2024-09-20T22:05:43.253885+0000","last_fullsized":"2024-09-20T22:04:16.305813+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":["6","2"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.7","version":"116'9670","reported_seq":9845,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:45.593270+0000","last_change":"2024-09-20T22:05:27.438039+0000","last_active":"2024-09-20T22:05:45.593270+0000","last_peered":"2024-09-20T22:05:45.593270+0000","last_clean":"2024-09-20T22:05:45.593270+0000","last_became_active":"2024-09-20T22:03:52.318296+0000","last_became_peered":"2024-09-20T22:03:52.318296+0000","last_unstale":"2024-09-20T22:05:45.593270+0000","last_undegraded":"2024-09-20T22:05:45.593270+0000","last_fullsized":"2024-09-20T22:05:45.593270+0000","mapping_epoch":80,"log_start":"98'3600","ondisk_log_start":"98'3600","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"112'8336","last_scrub_stamp":"2024-09-20T22:05:27.437852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:05:27.437852+0000","objects_scrubbed":8288,"log_size":6070,"log_dups_size":0,"ondisk_log_size":6070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:49.723769+0000","scrub_duration":2305,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":457572352,"num_objects":6982,"num_object_clones":0,"num_object_copies":20946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9670,"num_write_kb":532864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"92'29","reported_seq":145,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.246972+0000","last_change":"2024-09-20T22:04:16.310703+0000","last_active":"2024-09-20T22:05:43.246972+0000","last_peered":"2024-09-20T22:05:43.246972+0000","last_clean":"2024-09-20T22:02:57.636090+0000","last_became_active":"2024-09-20T22:04:16.310703+0000","last_became_peered":"2024-09-20T22:04:16.310703+0000","last_unstale":"2024-09-20T22:05:43.246972+0000","last_undegraded":"2024-09-20T22:04:16.305923+0000","last_fullsized":"2024-09-20T22:04:16.305806+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:49.110874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":124,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.244490+0000","last_change":"2024-09-20T22:04:21.796926+0000","last_active":"2024-09-20T22:05:43.244490+0000","last_peered":"2024-09-20T22:05:43.244490+0000","last_clean":"2024-09-20T22:05:43.244490+0000","last_became_active":"2024-09-20T22:02:50.108457+0000","last_became_peered":"2024-09-20T22:02:50.108457+0000","last_unstale":"2024-09-20T22:05:43.244490+0000","last_undegraded":"2024-09-20T22:05:43.244490+0000","last_fullsized":"2024-09-20T22:05:43.244490+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:21.796749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:21.796749+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"66'2","reported_seq":128,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.253969+0000","last_change":"2024-09-20T22:04:16.320295+0000","last_active":"2024-09-20T22:05:43.253969+0000","last_peered":"2024-09-20T22:05:43.253969+0000","last_clean":"2024-09-20T22:04:11.311686+0000","last_became_active":"2024-09-20T22:04:16.320295+0000","last_became_peered":"2024-09-20T22:04:16.320295+0000","last_unstale":"2024-09-20T22:05:43.253969+0000","last_undegraded":"2024-09-20T22:04:16.309132+0000","last_fullsized":"2024-09-20T22:04:16.309009+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:43.466982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[{"shards":"5,6","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":110,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.262866+0000","last_change":"2024-09-20T22:04:16.314215+0000","last_active":"2024-09-20T22:05:43.262866+0000","last_peered":"2024-09-20T22:05:43.262866+0000","last_clean":"2024-09-20T22:04:11.307965+0000","last_became_active":"2024-09-20T22:04:16.314215+0000","last_became_peered":"2024-09-20T22:04:16.314215+0000","last_unstale":"2024-09-20T22:05:43.262866+0000","last_undegraded":"2024-09-20T22:05:43.262866+0000","last_fullsized":"2024-09-20T22:04:16.304680+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:47.557400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":111,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.259107+0000","last_change":"2024-09-20T22:05:34.954357+0000","last_active":"2024-09-20T22:05:43.259107+0000","last_peered":"2024-09-20T22:05:43.259107+0000","last_clean":"2024-09-20T22:05:43.259107+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:05:43.259107+0000","last_undegraded":"2024-09-20T22:05:43.259107+0000","last_fullsized":"2024-09-20T22:05:43.259107+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:05:34.953944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:35.421579+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"116'9676","reported_seq":9761,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:45.719950+0000","last_change":"2024-09-20T22:04:16.312838+0000","last_active":"2024-09-20T22:05:45.719950+0000","last_peered":"2024-09-20T22:05:45.719950+0000","last_clean":"2024-09-20T22:04:15.698370+0000","last_became_active":"2024-09-20T22:04:16.312838+0000","last_became_peered":"2024-09-20T22:04:16.312838+0000","last_unstale":"2024-09-20T22:05:45.719950+0000","last_undegraded":"2024-09-20T22:04:16.304077+0000","last_fullsized":"2024-09-20T22:04:16.303960+0000","mapping_epoch":91,"log_start":"98'4400","ondisk_log_start":"98'4400","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":5276,"log_dups_size":0,"ondisk_log_size":5276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":453378048,"num_objects":6918,"num_object_clones":0,"num_object_copies":20754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6918,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9676,"num_write_kb":531008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[{"shards":"0,4","objects":6918}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"71'13","reported_seq":103,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.254692+0000","last_change":"2024-09-20T22:04:16.325664+0000","last_active":"2024-09-20T22:05:43.254692+0000","last_peered":"2024-09-20T22:05:43.254692+0000","last_clean":"2024-09-20T22:02:57.574176+0000","last_became_active":"2024-09-20T22:04:16.325664+0000","last_became_peered":"2024-09-20T22:04:16.325664+0000","last_unstale":"2024-09-20T22:05:43.254692+0000","last_undegraded":"2024-09-20T22:04:16.308863+0000","last_fullsized":"2024-09-20T22:04:16.308755+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:43.467020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[{"shards":"0,5","objects":5}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":58,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.252762+0000","last_change":"2024-09-20T22:04:16.319148+0000","last_active":"2024-09-20T22:05:43.252762+0000","last_peered":"2024-09-20T22:05:43.252762+0000","last_clean":"2024-09-20T22:02:49.083871+0000","last_became_active":"2024-09-20T22:04:16.319148+0000","last_became_peered":"2024-09-20T22:04:16.319148+0000","last_unstale":"2024-09-20T22:05:43.252762+0000","last_undegraded":"2024-09-20T22:05:43.252762+0000","last_fullsized":"2024-09-20T22:04:16.305482+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":["2","6"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":141,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.246436+0000","last_change":"2024-09-20T22:05:34.967627+0000","last_active":"2024-09-20T22:05:43.246436+0000","last_peered":"2024-09-20T22:05:43.246436+0000","last_clean":"2024-09-20T22:05:43.246436+0000","last_became_active":"2024-09-20T22:02:49.099508+0000","last_became_peered":"2024-09-20T22:02:49.099508+0000","last_unstale":"2024-09-20T22:05:43.246436+0000","last_undegraded":"2024-09-20T22:05:43.246436+0000","last_fullsized":"2024-09-20T22:05:43.246436+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:05:34.967504+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:34.967504+0000","last_clean_scrub_stamp":"2024-09-20T22:05:34.967504+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:45.258680+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":115,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.247631+0000","last_change":"2024-09-20T22:04:54.014490+0000","last_active":"2024-09-20T22:05:43.247631+0000","last_peered":"2024-09-20T22:05:43.247631+0000","last_clean":"2024-09-20T22:05:43.247631+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:05:43.247631+0000","last_undegraded":"2024-09-20T22:05:43.247631+0000","last_fullsized":"2024-09-20T22:05:43.247631+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:54.014237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:54.014237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:57.987950+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":112,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.257056+0000","last_change":"2024-09-20T22:05:28.780920+0000","last_active":"2024-09-20T22:05:43.257056+0000","last_peered":"2024-09-20T22:05:43.257056+0000","last_clean":"2024-09-20T22:05:43.257056+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:05:43.257056+0000","last_undegraded":"2024-09-20T22:05:43.257056+0000","last_fullsized":"2024-09-20T22:05:43.257056+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.780758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.780758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:57.033951+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"116'9700","reported_seq":9782,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:45.536249+0000","last_change":"2024-09-20T22:03:52.316989+0000","last_active":"2024-09-20T22:05:45.536249+0000","last_peered":"2024-09-20T22:05:45.536249+0000","last_clean":"2024-09-20T22:05:45.536249+0000","last_became_active":"2024-09-20T22:03:52.316729+0000","last_became_peered":"2024-09-20T22:03:52.316729+0000","last_unstale":"2024-09-20T22:05:45.536249+0000","last_undegraded":"2024-09-20T22:05:45.536249+0000","last_fullsized":"2024-09-20T22:05:45.536249+0000","mapping_epoch":80,"log_start":"98'4500","ondisk_log_start":"98'4500","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":5200,"log_dups_size":0,"ondisk_log_size":5200,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":457179136,"num_objects":6976,"num_object_clones":0,"num_object_copies":20928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9700,"num_write_kb":533632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"71'12","reported_seq":132,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.263658+0000","last_change":"2024-09-20T22:04:16.309130+0000","last_active":"2024-09-20T22:05:43.263658+0000","last_peered":"2024-09-20T22:05:43.263658+0000","last_clean":"2024-09-20T22:04:11.308694+0000","last_became_active":"2024-09-20T22:04:16.309130+0000","last_became_peered":"2024-09-20T22:04:16.309130+0000","last_unstale":"2024-09-20T22:05:43.263658+0000","last_undegraded":"2024-09-20T22:04:16.304971+0000","last_fullsized":"2024-09-20T22:04:16.304867+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:47.557376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":123,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.256516+0000","last_change":"2024-09-20T22:04:14.234738+0000","last_active":"2024-09-20T22:05:43.256516+0000","last_peered":"2024-09-20T22:05:43.256516+0000","last_clean":"2024-09-20T22:05:43.256516+0000","last_became_active":"2024-09-20T22:02:50.110448+0000","last_became_peered":"2024-09-20T22:02:50.110448+0000","last_unstale":"2024-09-20T22:05:43.256516+0000","last_undegraded":"2024-09-20T22:05:43.256516+0000","last_fullsized":"2024-09-20T22:05:43.256516+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:14.234630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:14.234630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":299,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:44.469766+0000","last_change":"2024-09-20T22:04:16.317127+0000","last_active":"2024-09-20T22:05:44.469766+0000","last_peered":"2024-09-20T22:05:44.469766+0000","last_clean":"2024-09-20T22:04:14.769746+0000","last_became_active":"2024-09-20T22:04:16.317127+0000","last_became_peered":"2024-09-20T22:04:16.317127+0000","last_unstale":"2024-09-20T22:05:44.469766+0000","last_undegraded":"2024-09-20T22:04:16.310637+0000","last_fullsized":"2024-09-20T22:04:16.310524+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":165,"num_read_kb":160,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[{"shards":"0,7","objects":1}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":110,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.255101+0000","last_change":"2024-09-20T22:04:16.325962+0000","last_active":"2024-09-20T22:05:43.255101+0000","last_peered":"2024-09-20T22:05:43.255101+0000","last_clean":"2024-09-20T22:04:11.311829+0000","last_became_active":"2024-09-20T22:04:16.325962+0000","last_became_peered":"2024-09-20T22:04:16.325962+0000","last_unstale":"2024-09-20T22:05:43.255101+0000","last_undegraded":"2024-09-20T22:05:43.255101+0000","last_fullsized":"2024-09-20T22:04:16.310570+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:43.467012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":57,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.253660+0000","last_change":"2024-09-20T22:04:16.324713+0000","last_active":"2024-09-20T22:05:43.253660+0000","last_peered":"2024-09-20T22:05:43.253660+0000","last_clean":"2024-09-20T22:02:55.278800+0000","last_became_active":"2024-09-20T22:04:16.324713+0000","last_became_peered":"2024-09-20T22:04:16.324713+0000","last_unstale":"2024-09-20T22:05:43.253660+0000","last_undegraded":"2024-09-20T22:05:43.253660+0000","last_fullsized":"2024-09-20T22:04:16.309381+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:43.467005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"116'9709","reported_seq":9791,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:45.755459+0000","last_change":"2024-09-20T22:03:52.318412+0000","last_active":"2024-09-20T22:05:45.755459+0000","last_peered":"2024-09-20T22:05:45.755459+0000","last_clean":"2024-09-20T22:05:45.755459+0000","last_became_active":"2024-09-20T22:03:52.318009+0000","last_became_peered":"2024-09-20T22:03:52.318009+0000","last_unstale":"2024-09-20T22:05:45.755459+0000","last_undegraded":"2024-09-20T22:05:45.755459+0000","last_fullsized":"2024-09-20T22:05:45.755459+0000","mapping_epoch":80,"log_start":"98'4400","ondisk_log_start":"98'4400","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":5309,"log_dups_size":0,"ondisk_log_size":5309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":453443584,"num_objects":6919,"num_object_clones":0,"num_object_copies":20757,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9709,"num_write_kb":532096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"71'16","reported_seq":177,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.255008+0000","last_change":"2024-09-20T22:04:17.518433+0000","last_active":"2024-09-20T22:05:43.255008+0000","last_peered":"2024-09-20T22:05:43.255008+0000","last_clean":"2024-09-20T22:05:43.255008+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:05:43.255008+0000","last_undegraded":"2024-09-20T22:05:43.255008+0000","last_fullsized":"2024-09-20T22:05:43.255008+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'16","last_scrub_stamp":"2024-09-20T22:04:17.518303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:17.518303+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":123,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.261055+0000","last_change":"2024-09-20T22:04:11.743850+0000","last_active":"2024-09-20T22:05:43.261055+0000","last_peered":"2024-09-20T22:05:43.261055+0000","last_clean":"2024-09-20T22:05:43.261055+0000","last_became_active":"2024-09-20T22:02:50.106011+0000","last_became_peered":"2024-09-20T22:02:50.106011+0000","last_unstale":"2024-09-20T22:05:43.261055+0000","last_undegraded":"2024-09-20T22:05:43.261055+0000","last_fullsized":"2024-09-20T22:05:43.261055+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:11.743751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:11.743751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":379,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.258888+0000","last_change":"2024-09-20T22:05:02.727025+0000","last_active":"2024-09-20T22:05:43.258888+0000","last_peered":"2024-09-20T22:05:43.258888+0000","last_clean":"2024-09-20T22:05:43.258888+0000","last_became_active":"2024-09-20T22:02:45.068482+0000","last_became_peered":"2024-09-20T22:02:45.068482+0000","last_unstale":"2024-09-20T22:05:43.258888+0000","last_undegraded":"2024-09-20T22:05:43.258888+0000","last_fullsized":"2024-09-20T22:05:43.258888+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:05:02.726765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:05:02.726765+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:21.401986+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":116,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.257681+0000","last_change":"2024-09-20T22:04:55.645343+0000","last_active":"2024-09-20T22:05:43.257681+0000","last_peered":"2024-09-20T22:05:43.257681+0000","last_clean":"2024-09-20T22:05:43.257681+0000","last_became_active":"2024-09-20T22:02:54.298021+0000","last_became_peered":"2024-09-20T22:02:54.298021+0000","last_unstale":"2024-09-20T22:05:43.257681+0000","last_undegraded":"2024-09-20T22:05:43.257681+0000","last_fullsized":"2024-09-20T22:05:43.257681+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:55.645073+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:55.645073+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:55.727604+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":125,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.254895+0000","last_change":"2024-09-20T22:05:27.659354+0000","last_active":"2024-09-20T22:05:43.254895+0000","last_peered":"2024-09-20T22:05:43.254895+0000","last_clean":"2024-09-20T22:05:43.254895+0000","last_became_active":"2024-09-20T22:04:21.237725+0000","last_became_peered":"2024-09-20T22:04:21.237725+0000","last_unstale":"2024-09-20T22:05:43.254895+0000","last_undegraded":"2024-09-20T22:05:43.254895+0000","last_fullsized":"2024-09-20T22:05:43.254895+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:05:27.659252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:40.379417+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"116'9577","reported_seq":9663,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:45.645951+0000","last_change":"2024-09-20T22:04:16.319236+0000","last_active":"2024-09-20T22:05:45.645951+0000","last_peered":"2024-09-20T22:05:45.645951+0000","last_clean":"2024-09-20T22:04:15.698310+0000","last_became_active":"2024-09-20T22:04:16.319236+0000","last_became_peered":"2024-09-20T22:04:16.319236+0000","last_unstale":"2024-09-20T22:05:45.645951+0000","last_undegraded":"2024-09-20T22:04:16.305688+0000","last_fullsized":"2024-09-20T22:04:16.305555+0000","mapping_epoch":91,"log_start":"98'3500","ondisk_log_start":"98'3500","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":6077,"log_dups_size":0,"ondisk_log_size":6077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:50.314106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":446038040,"num_objects":6807,"num_object_clones":0,"num_object_copies":20421,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6807,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6807,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9577,"num_write_kb":524225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[{"shards":"0,6","objects":6807}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"71'11","reported_seq":134,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.241745+0000","last_change":"2024-09-20T22:04:16.320499+0000","last_active":"2024-09-20T22:05:43.241745+0000","last_peered":"2024-09-20T22:05:43.241745+0000","last_clean":"2024-09-20T22:04:11.303901+0000","last_became_active":"2024-09-20T22:04:16.320499+0000","last_became_peered":"2024-09-20T22:04:16.320499+0000","last_unstale":"2024-09-20T22:05:43.241745+0000","last_undegraded":"2024-09-20T22:04:16.305535+0000","last_fullsized":"2024-09-20T22:04:16.305436+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[{"shards":"4,6","objects":5}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":124,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.241764+0000","last_change":"2024-09-20T22:05:26.006756+0000","last_active":"2024-09-20T22:05:43.241764+0000","last_peered":"2024-09-20T22:05:43.241764+0000","last_clean":"2024-09-20T22:05:43.241764+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:05:43.241764+0000","last_undegraded":"2024-09-20T22:05:43.241764+0000","last_fullsized":"2024-09-20T22:05:43.241764+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:26.006692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:26.006692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:49.724396+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":115,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.255533+0000","last_change":"2024-09-20T22:04:19.515430+0000","last_active":"2024-09-20T22:05:43.255533+0000","last_peered":"2024-09-20T22:05:43.255533+0000","last_clean":"2024-09-20T22:05:43.255533+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:05:43.255533+0000","last_undegraded":"2024-09-20T22:05:43.255533+0000","last_fullsized":"2024-09-20T22:05:43.255533+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:19.515307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:19.515307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:43.467090+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":111,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.259124+0000","last_change":"2024-09-20T22:04:17.783161+0000","last_active":"2024-09-20T22:05:43.259124+0000","last_peered":"2024-09-20T22:05:43.259124+0000","last_clean":"2024-09-20T22:05:43.259124+0000","last_became_active":"2024-09-20T22:02:56.349376+0000","last_became_peered":"2024-09-20T22:02:56.349376+0000","last_unstale":"2024-09-20T22:05:43.259124+0000","last_undegraded":"2024-09-20T22:05:43.259124+0000","last_fullsized":"2024-09-20T22:05:43.259124+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:17.783059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:17.783059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"116'9632","reported_seq":9859,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:45.744421+0000","last_change":"2024-09-20T22:05:27.949351+0000","last_active":"2024-09-20T22:05:45.744421+0000","last_peered":"2024-09-20T22:05:45.744421+0000","last_clean":"2024-09-20T22:05:45.744421+0000","last_became_active":"2024-09-20T22:03:52.311982+0000","last_became_peered":"2024-09-20T22:03:52.311982+0000","last_unstale":"2024-09-20T22:05:45.744421+0000","last_undegraded":"2024-09-20T22:05:45.744421+0000","last_fullsized":"2024-09-20T22:05:45.744421+0000","mapping_epoch":80,"log_start":"92'2600","ondisk_log_start":"92'2600","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"112'8343","last_scrub_stamp":"2024-09-20T22:05:27.949070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:05:27.949070+0000","objects_scrubbed":8239,"log_size":7032,"log_dups_size":0,"ondisk_log_size":7032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:54.550606+0000","scrub_duration":3181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":455868416,"num_objects":6956,"num_object_clones":0,"num_object_copies":20868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9632,"num_write_kb":530816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"71'17","reported_seq":151,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.241645+0000","last_change":"2024-09-20T22:04:16.312435+0000","last_active":"2024-09-20T22:05:43.241645+0000","last_peered":"2024-09-20T22:05:43.241645+0000","last_clean":"2024-09-20T22:04:12.942923+0000","last_became_active":"2024-09-20T22:04:16.312435+0000","last_became_peered":"2024-09-20T22:04:16.312435+0000","last_unstale":"2024-09-20T22:05:43.241645+0000","last_undegraded":"2024-09-20T22:04:16.305517+0000","last_fullsized":"2024-09-20T22:04:16.305404+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'17","last_scrub_stamp":"2024-09-20T22:04:12.942792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:12.942792+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[{"shards":"2,4","objects":8}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":118,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.252714+0000","last_change":"2024-09-20T22:04:16.319111+0000","last_active":"2024-09-20T22:05:43.252714+0000","last_peered":"2024-09-20T22:05:43.252714+0000","last_clean":"2024-09-20T22:04:11.306602+0000","last_became_active":"2024-09-20T22:04:16.319111+0000","last_became_peered":"2024-09-20T22:04:16.319111+0000","last_unstale":"2024-09-20T22:05:43.252714+0000","last_undegraded":"2024-09-20T22:05:43.252714+0000","last_fullsized":"2024-09-20T22:04:16.305556+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":57,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.241081+0000","last_change":"2024-09-20T22:04:16.310599+0000","last_active":"2024-09-20T22:05:43.241081+0000","last_peered":"2024-09-20T22:05:43.241081+0000","last_clean":"2024-09-20T22:02:53.269054+0000","last_became_active":"2024-09-20T22:04:16.310599+0000","last_became_peered":"2024-09-20T22:04:16.310599+0000","last_unstale":"2024-09-20T22:05:43.241081+0000","last_undegraded":"2024-09-20T22:05:43.241081+0000","last_fullsized":"2024-09-20T22:04:16.304159+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":111,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.252305+0000","last_change":"2024-09-20T22:04:54.986448+0000","last_active":"2024-09-20T22:05:43.252305+0000","last_peered":"2024-09-20T22:05:43.252305+0000","last_clean":"2024-09-20T22:05:43.252305+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:05:43.252305+0000","last_undegraded":"2024-09-20T22:05:43.252305+0000","last_fullsized":"2024-09-20T22:05:43.252305+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:54.986284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:54.986284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:55.609221+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"116'9584","reported_seq":9666,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:45.714198+0000","last_change":"2024-09-20T22:03:52.317134+0000","last_active":"2024-09-20T22:05:45.714198+0000","last_peered":"2024-09-20T22:05:45.714198+0000","last_clean":"2024-09-20T22:05:45.714198+0000","last_became_active":"2024-09-20T22:03:52.316825+0000","last_became_peered":"2024-09-20T22:03:52.316825+0000","last_unstale":"2024-09-20T22:05:45.714198+0000","last_undegraded":"2024-09-20T22:05:45.714198+0000","last_fullsized":"2024-09-20T22:05:45.714198+0000","mapping_epoch":80,"log_start":"98'4300","ondisk_log_start":"98'4300","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":5284,"log_dups_size":0,"ondisk_log_size":5284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":442105856,"num_objects":6746,"num_object_clones":0,"num_object_copies":20238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9584,"num_write_kb":522560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"71'10","reported_seq":135,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.241217+0000","last_change":"2024-09-20T22:05:27.048378+0000","last_active":"2024-09-20T22:05:43.241217+0000","last_peered":"2024-09-20T22:05:43.241217+0000","last_clean":"2024-09-20T22:05:43.241217+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:05:43.241217+0000","last_undegraded":"2024-09-20T22:05:43.241217+0000","last_fullsized":"2024-09-20T22:05:43.241217+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:05:27.048215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:27.048215+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:51.977225+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"64'1","reported_seq":129,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.261289+0000","last_change":"2024-09-20T22:05:27.895711+0000","last_active":"2024-09-20T22:05:43.261289+0000","last_peered":"2024-09-20T22:05:43.261289+0000","last_clean":"2024-09-20T22:05:43.261289+0000","last_became_active":"2024-09-20T22:02:50.110446+0000","last_became_peered":"2024-09-20T22:02:50.110446+0000","last_unstale":"2024-09-20T22:05:43.261289+0000","last_undegraded":"2024-09-20T22:05:43.261289+0000","last_fullsized":"2024-09-20T22:05:43.261289+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:05:27.895595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:27.895595+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:33.193947+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"71'8","reported_seq":167,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.247870+0000","last_change":"2024-09-20T22:05:30.744134+0000","last_active":"2024-09-20T22:05:43.247870+0000","last_peered":"2024-09-20T22:05:43.247870+0000","last_clean":"2024-09-20T22:05:43.247870+0000","last_became_active":"2024-09-20T22:02:54.294458+0000","last_became_peered":"2024-09-20T22:02:54.294458+0000","last_unstale":"2024-09-20T22:05:43.247870+0000","last_undegraded":"2024-09-20T22:05:43.247870+0000","last_fullsized":"2024-09-20T22:05:43.247870+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:05:30.743022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:30.743022+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:38.500958+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":107,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.242219+0000","last_change":"2024-09-20T22:04:16.312842+0000","last_active":"2024-09-20T22:05:43.242219+0000","last_peered":"2024-09-20T22:05:43.242219+0000","last_clean":"2024-09-20T22:04:11.303632+0000","last_became_active":"2024-09-20T22:04:16.312842+0000","last_became_peered":"2024-09-20T22:04:16.312842+0000","last_unstale":"2024-09-20T22:05:43.242219+0000","last_undegraded":"2024-09-20T22:05:43.242219+0000","last_fullsized":"2024-09-20T22:04:16.305087+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"116'9853","reported_seq":10005,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:45.784159+0000","last_change":"2024-09-20T22:04:59.310499+0000","last_active":"2024-09-20T22:05:45.784159+0000","last_peered":"2024-09-20T22:05:45.784159+0000","last_clean":"2024-09-20T22:05:45.784159+0000","last_became_active":"2024-09-20T22:03:52.307223+0000","last_became_peered":"2024-09-20T22:03:52.307223+0000","last_unstale":"2024-09-20T22:05:45.784159+0000","last_undegraded":"2024-09-20T22:05:45.784159+0000","last_fullsized":"2024-09-20T22:05:45.784159+0000","mapping_epoch":80,"log_start":"92'2800","ondisk_log_start":"92'2800","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"107'6452","last_scrub_stamp":"2024-09-20T22:04:59.310325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:04:59.310325+0000","objects_scrubbed":6357,"log_size":7053,"log_dups_size":0,"ondisk_log_size":7053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:06.545269+0000","scrub_duration":2308,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460914688,"num_objects":7033,"num_object_clones":0,"num_object_copies":21099,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9853,"num_write_kb":540352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"71'9","reported_seq":138,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.258081+0000","last_change":"2024-09-20T22:04:16.319135+0000","last_active":"2024-09-20T22:05:43.258081+0000","last_peered":"2024-09-20T22:05:43.258081+0000","last_clean":"2024-09-20T22:04:11.307387+0000","last_became_active":"2024-09-20T22:04:16.319135+0000","last_became_peered":"2024-09-20T22:04:16.319135+0000","last_unstale":"2024-09-20T22:05:43.258081+0000","last_undegraded":"2024-09-20T22:04:16.304006+0000","last_fullsized":"2024-09-20T22:04:16.303888+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:03:52.011201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:03:52.011201+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[{"shards":"0,4","objects":4}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":129,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.253863+0000","last_change":"2024-09-20T22:04:57.472466+0000","last_active":"2024-09-20T22:05:43.253863+0000","last_peered":"2024-09-20T22:05:43.253863+0000","last_clean":"2024-09-20T22:05:43.253863+0000","last_became_active":"2024-09-20T22:02:50.112682+0000","last_became_peered":"2024-09-20T22:02:50.112682+0000","last_unstale":"2024-09-20T22:05:43.253863+0000","last_undegraded":"2024-09-20T22:05:43.253863+0000","last_fullsized":"2024-09-20T22:05:43.253863+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:04:57.472316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:57.472316+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:25.283931+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":115,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.248194+0000","last_change":"2024-09-20T22:04:23.776699+0000","last_active":"2024-09-20T22:05:43.248194+0000","last_peered":"2024-09-20T22:05:43.248194+0000","last_clean":"2024-09-20T22:05:43.248194+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:05:43.248194+0000","last_undegraded":"2024-09-20T22:05:43.248194+0000","last_fullsized":"2024-09-20T22:05:43.248194+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:23.776589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:23.776589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:47.824484+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":116,"reported_epoch":116,"state":"active+clean+remapped","last_fresh":"2024-09-20T22:05:43.256333+0000","last_change":"2024-09-20T22:04:21.326390+0000","last_active":"2024-09-20T22:05:43.256333+0000","last_peered":"2024-09-20T22:05:43.256333+0000","last_clean":"2024-09-20T22:05:43.256333+0000","last_became_active":"2024-09-20T22:04:21.325835+0000","last_became_peered":"2024-09-20T22:04:21.325835+0000","last_unstale":"2024-09-20T22:05:43.256333+0000","last_undegraded":"2024-09-20T22:05:43.256333+0000","last_fullsized":"2024-09-20T22:05:43.256333+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:17.171523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:17.171523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2,7],"avail_no_missing":["0","2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"116'9637","reported_seq":9721,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:45.746158+0000","last_change":"2024-09-20T22:04:16.313312+0000","last_active":"2024-09-20T22:05:45.746158+0000","last_peered":"2024-09-20T22:05:45.746158+0000","last_clean":"2024-09-20T22:04:15.706494+0000","last_became_active":"2024-09-20T22:04:16.313312+0000","last_became_peered":"2024-09-20T22:04:16.313312+0000","last_unstale":"2024-09-20T22:05:45.746158+0000","last_undegraded":"2024-09-20T22:04:16.303611+0000","last_fullsized":"2024-09-20T22:04:16.302690+0000","mapping_epoch":91,"log_start":"98'4300","ondisk_log_start":"98'4300","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":5337,"log_dups_size":0,"ondisk_log_size":5337,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459341824,"num_objects":7009,"num_object_clones":0,"num_object_copies":21027,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9637,"num_write_kb":532672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[{"shards":"0,4","objects":7009}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"71'19","reported_seq":145,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.252481+0000","last_change":"2024-09-20T22:04:16.318494+0000","last_active":"2024-09-20T22:05:43.252481+0000","last_peered":"2024-09-20T22:05:43.252481+0000","last_clean":"2024-09-20T22:04:11.306486+0000","last_became_active":"2024-09-20T22:04:16.318494+0000","last_became_peered":"2024-09-20T22:04:16.318494+0000","last_unstale":"2024-09-20T22:05:43.252481+0000","last_undegraded":"2024-09-20T22:04:16.306185+0000","last_fullsized":"2024-09-20T22:04:16.306074+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[{"shards":"6,7","objects":9}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":124,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.257402+0000","last_change":"2024-09-20T22:05:05.659544+0000","last_active":"2024-09-20T22:05:43.257402+0000","last_peered":"2024-09-20T22:05:43.257402+0000","last_clean":"2024-09-20T22:05:43.257402+0000","last_became_active":"2024-09-20T22:02:50.108712+0000","last_became_peered":"2024-09-20T22:02:50.108712+0000","last_unstale":"2024-09-20T22:05:43.257402+0000","last_undegraded":"2024-09-20T22:05:43.257402+0000","last_fullsized":"2024-09-20T22:05:43.257402+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:05.659437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:05.659437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:25.341327+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":123,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.252055+0000","last_change":"2024-09-20T22:05:31.802337+0000","last_active":"2024-09-20T22:05:43.252055+0000","last_peered":"2024-09-20T22:05:43.252055+0000","last_clean":"2024-09-20T22:05:43.252055+0000","last_became_active":"2024-09-20T22:02:54.291134+0000","last_became_peered":"2024-09-20T22:02:54.291134+0000","last_unstale":"2024-09-20T22:05:43.252055+0000","last_undegraded":"2024-09-20T22:05:43.252055+0000","last_fullsized":"2024-09-20T22:05:43.252055+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:31.802238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:31.802238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:53.108551+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":112,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.258784+0000","last_change":"2024-09-20T22:04:59.653059+0000","last_active":"2024-09-20T22:05:43.258784+0000","last_peered":"2024-09-20T22:05:43.258784+0000","last_clean":"2024-09-20T22:05:43.258784+0000","last_became_active":"2024-09-20T22:02:56.307687+0000","last_became_peered":"2024-09-20T22:02:56.307687+0000","last_unstale":"2024-09-20T22:05:43.258784+0000","last_undegraded":"2024-09-20T22:05:43.258784+0000","last_fullsized":"2024-09-20T22:05:43.258784+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:59.652843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:59.652843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:18.749869+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"116'9759","reported_seq":9841,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:45.750670+0000","last_change":"2024-09-20T22:03:52.316640+0000","last_active":"2024-09-20T22:05:45.750670+0000","last_peered":"2024-09-20T22:05:45.750670+0000","last_clean":"2024-09-20T22:05:45.750670+0000","last_became_active":"2024-09-20T22:03:52.316211+0000","last_became_peered":"2024-09-20T22:03:52.316211+0000","last_unstale":"2024-09-20T22:05:45.750670+0000","last_undegraded":"2024-09-20T22:05:45.750670+0000","last_fullsized":"2024-09-20T22:05:45.750670+0000","mapping_epoch":80,"log_start":"102'5000","ondisk_log_start":"102'5000","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":4759,"log_dups_size":0,"ondisk_log_size":4759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:49.060859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":456851456,"num_objects":6971,"num_object_clones":0,"num_object_copies":20913,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6971,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9759,"num_write_kb":535360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"71'12","reported_seq":138,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.241471+0000","last_change":"2024-09-20T22:04:16.324704+0000","last_active":"2024-09-20T22:05:43.241471+0000","last_peered":"2024-09-20T22:05:43.241471+0000","last_clean":"2024-09-20T22:04:11.303078+0000","last_became_active":"2024-09-20T22:04:16.324704+0000","last_became_peered":"2024-09-20T22:04:16.324704+0000","last_unstale":"2024-09-20T22:05:43.241471+0000","last_undegraded":"2024-09-20T22:04:16.303915+0000","last_fullsized":"2024-09-20T22:04:16.303791+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[{"shards":"3,4","objects":5}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":119,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.258142+0000","last_change":"2024-09-20T22:04:16.316858+0000","last_active":"2024-09-20T22:05:43.258142+0000","last_peered":"2024-09-20T22:05:43.258142+0000","last_clean":"2024-09-20T22:04:11.309387+0000","last_became_active":"2024-09-20T22:04:16.316858+0000","last_became_peered":"2024-09-20T22:04:16.316858+0000","last_unstale":"2024-09-20T22:05:43.258142+0000","last_undegraded":"2024-09-20T22:05:43.258142+0000","last_fullsized":"2024-09-20T22:04:16.309391+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:44.796430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":118,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.252567+0000","last_change":"2024-09-20T22:04:16.315094+0000","last_active":"2024-09-20T22:05:43.252567+0000","last_peered":"2024-09-20T22:05:43.252567+0000","last_clean":"2024-09-20T22:04:12.729887+0000","last_became_active":"2024-09-20T22:04:16.315094+0000","last_became_peered":"2024-09-20T22:04:16.315094+0000","last_unstale":"2024-09-20T22:05:43.252567+0000","last_undegraded":"2024-09-20T22:05:43.252567+0000","last_fullsized":"2024-09-20T22:04:16.304906+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:12.729789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:12.729789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":60,"reported_epoch":116,"state":"active+clean+remapped","last_fresh":"2024-09-20T22:05:43.258497+0000","last_change":"2024-09-20T22:05:28.569254+0000","last_active":"2024-09-20T22:05:43.258497+0000","last_peered":"2024-09-20T22:05:43.258497+0000","last_clean":"2024-09-20T22:05:43.258497+0000","last_became_active":"2024-09-20T22:04:21.325506+0000","last_became_peered":"2024-09-20T22:04:21.325506+0000","last_unstale":"2024-09-20T22:05:43.258497+0000","last_undegraded":"2024-09-20T22:05:43.258497+0000","last_fullsized":"2024-09-20T22:05:43.258497+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.569045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.569045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:34.983641+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7,3],"avail_no_missing":["0","3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.b","version":"116'9648","reported_seq":9733,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:45.559127+0000","last_change":"2024-09-20T22:04:16.321983+0000","last_active":"2024-09-20T22:05:45.559127+0000","last_peered":"2024-09-20T22:05:45.559127+0000","last_clean":"2024-09-20T22:04:15.686857+0000","last_became_active":"2024-09-20T22:04:16.321983+0000","last_became_peered":"2024-09-20T22:04:16.321983+0000","last_unstale":"2024-09-20T22:05:45.559127+0000","last_undegraded":"2024-09-20T22:04:16.309082+0000","last_fullsized":"2024-09-20T22:04:16.308976+0000","mapping_epoch":91,"log_start":"98'3700","ondisk_log_start":"98'3700","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":5948,"log_dups_size":0,"ondisk_log_size":5948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:49.409175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":452984832,"num_objects":6912,"num_object_clones":0,"num_object_copies":20736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9648,"num_write_kb":529920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[{"shards":"5,6","objects":6912}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"71'15","reported_seq":144,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.253220+0000","last_change":"2024-09-20T22:05:01.515339+0000","last_active":"2024-09-20T22:05:43.253220+0000","last_peered":"2024-09-20T22:05:43.253220+0000","last_clean":"2024-09-20T22:05:43.253220+0000","last_became_active":"2024-09-20T22:02:52.482008+0000","last_became_peered":"2024-09-20T22:02:52.482008+0000","last_unstale":"2024-09-20T22:05:43.253220+0000","last_undegraded":"2024-09-20T22:05:43.253220+0000","last_fullsized":"2024-09-20T22:05:43.253220+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:05:01.515075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:01.515075+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:28.570514+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"64'2","reported_seq":136,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.258047+0000","last_change":"2024-09-20T22:05:00.690608+0000","last_active":"2024-09-20T22:05:43.258047+0000","last_peered":"2024-09-20T22:05:43.258047+0000","last_clean":"2024-09-20T22:05:43.258047+0000","last_became_active":"2024-09-20T22:02:50.114819+0000","last_became_peered":"2024-09-20T22:02:50.114819+0000","last_unstale":"2024-09-20T22:05:43.258047+0000","last_undegraded":"2024-09-20T22:05:43.258047+0000","last_fullsized":"2024-09-20T22:05:43.258047+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'2","last_scrub_stamp":"2024-09-20T22:05:00.690333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:00.690333+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:00.820041+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":160,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.256692+0000","last_change":"2024-09-20T22:05:34.732761+0000","last_active":"2024-09-20T22:05:43.256692+0000","last_peered":"2024-09-20T22:05:43.256692+0000","last_clean":"2024-09-20T22:05:43.256692+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:05:43.256692+0000","last_undegraded":"2024-09-20T22:05:43.256692+0000","last_fullsized":"2024-09-20T22:05:43.256692+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:05:34.732644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:34.732644+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:59.614780+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":111,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.253312+0000","last_change":"2024-09-20T22:05:00.532426+0000","last_active":"2024-09-20T22:05:43.253312+0000","last_peered":"2024-09-20T22:05:43.253312+0000","last_clean":"2024-09-20T22:05:43.253312+0000","last_became_active":"2024-09-20T22:02:56.346990+0000","last_became_peered":"2024-09-20T22:02:56.346990+0000","last_unstale":"2024-09-20T22:05:43.253312+0000","last_undegraded":"2024-09-20T22:05:43.253312+0000","last_fullsized":"2024-09-20T22:05:43.253312+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:00.532303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:00.532303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:24.617909+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":123,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.252232+0000","last_change":"2024-09-20T22:04:59.377318+0000","last_active":"2024-09-20T22:05:43.252232+0000","last_peered":"2024-09-20T22:05:43.252232+0000","last_clean":"2024-09-20T22:05:43.252232+0000","last_became_active":"2024-09-20T22:02:50.111935+0000","last_became_peered":"2024-09-20T22:02:50.111935+0000","last_unstale":"2024-09-20T22:05:43.252232+0000","last_undegraded":"2024-09-20T22:05:43.252232+0000","last_fullsized":"2024-09-20T22:05:43.252232+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:59.377203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:59.377203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:18.963527+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"71'6","reported_seq":123,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.258693+0000","last_change":"2024-09-20T22:04:16.318583+0000","last_active":"2024-09-20T22:05:43.258693+0000","last_peered":"2024-09-20T22:05:43.258693+0000","last_clean":"2024-09-20T22:04:11.314508+0000","last_became_active":"2024-09-20T22:04:16.318583+0000","last_became_peered":"2024-09-20T22:04:16.318583+0000","last_unstale":"2024-09-20T22:05:43.258693+0000","last_undegraded":"2024-09-20T22:04:16.312022+0000","last_fullsized":"2024-09-20T22:04:16.311757+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[{"shards":"3,7","objects":3}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":110,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.253675+0000","last_change":"2024-09-20T22:04:16.320696+0000","last_active":"2024-09-20T22:05:43.253675+0000","last_peered":"2024-09-20T22:05:43.253675+0000","last_clean":"2024-09-20T22:04:11.311311+0000","last_became_active":"2024-09-20T22:04:16.320696+0000","last_became_peered":"2024-09-20T22:04:16.320696+0000","last_unstale":"2024-09-20T22:05:43.253675+0000","last_undegraded":"2024-09-20T22:05:43.253675+0000","last_fullsized":"2024-09-20T22:04:16.311423+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:43.466998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":104,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.256977+0000","last_change":"2024-09-20T22:02:56.350065+0000","last_active":"2024-09-20T22:05:43.256977+0000","last_peered":"2024-09-20T22:05:43.256977+0000","last_clean":"2024-09-20T22:05:43.256977+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:05:43.256977+0000","last_undegraded":"2024-09-20T22:05:43.256977+0000","last_fullsized":"2024-09-20T22:05:43.256977+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:44.796515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"71'9","reported_seq":135,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.263778+0000","last_change":"2024-09-20T22:04:52.254864+0000","last_active":"2024-09-20T22:05:43.263778+0000","last_peered":"2024-09-20T22:05:43.263778+0000","last_clean":"2024-09-20T22:05:43.263778+0000","last_became_active":"2024-09-20T22:02:52.481989+0000","last_became_peered":"2024-09-20T22:02:52.481989+0000","last_unstale":"2024-09-20T22:05:43.263778+0000","last_undegraded":"2024-09-20T22:05:43.263778+0000","last_fullsized":"2024-09-20T22:05:43.263778+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:52.254706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:52.254706+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:14.729595+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":124,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.256935+0000","last_change":"2024-09-20T22:05:31.764903+0000","last_active":"2024-09-20T22:05:43.256935+0000","last_peered":"2024-09-20T22:05:43.256935+0000","last_clean":"2024-09-20T22:05:43.256935+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:05:43.256935+0000","last_undegraded":"2024-09-20T22:05:43.256935+0000","last_fullsized":"2024-09-20T22:05:43.256935+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:31.764833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:31.764833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:55.238401+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":110,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.259053+0000","last_change":"2024-09-20T22:04:16.316683+0000","last_active":"2024-09-20T22:05:43.259053+0000","last_peered":"2024-09-20T22:05:43.259053+0000","last_clean":"2024-09-20T22:04:11.314344+0000","last_became_active":"2024-09-20T22:04:16.316683+0000","last_became_peered":"2024-09-20T22:04:16.316683+0000","last_unstale":"2024-09-20T22:05:43.259053+0000","last_undegraded":"2024-09-20T22:05:43.259053+0000","last_fullsized":"2024-09-20T22:04:16.308930+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":111,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.246596+0000","last_change":"2024-09-20T22:04:52.008397+0000","last_active":"2024-09-20T22:05:43.246596+0000","last_peered":"2024-09-20T22:05:43.246596+0000","last_clean":"2024-09-20T22:05:43.246596+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:05:43.246596+0000","last_undegraded":"2024-09-20T22:05:43.246596+0000","last_fullsized":"2024-09-20T22:05:43.246596+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:52.008283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:52.008283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:14.892148+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"71'9","reported_seq":135,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.254131+0000","last_change":"2024-09-20T22:04:50.514559+0000","last_active":"2024-09-20T22:05:43.254131+0000","last_peered":"2024-09-20T22:05:43.254131+0000","last_clean":"2024-09-20T22:05:43.254131+0000","last_became_active":"2024-09-20T22:02:52.482141+0000","last_became_peered":"2024-09-20T22:02:52.482141+0000","last_unstale":"2024-09-20T22:05:43.254131+0000","last_undegraded":"2024-09-20T22:05:43.254131+0000","last_fullsized":"2024-09-20T22:05:43.254131+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:50.514410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:50.514410+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:52.693126+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":123,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.256519+0000","last_change":"2024-09-20T22:04:13.226031+0000","last_active":"2024-09-20T22:05:43.256519+0000","last_peered":"2024-09-20T22:05:43.256519+0000","last_clean":"2024-09-20T22:05:43.256519+0000","last_became_active":"2024-09-20T22:02:50.110075+0000","last_became_peered":"2024-09-20T22:02:50.110075+0000","last_unstale":"2024-09-20T22:05:43.256519+0000","last_undegraded":"2024-09-20T22:05:43.256519+0000","last_fullsized":"2024-09-20T22:05:43.256519+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:13.225925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:13.225925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"71'8","reported_seq":151,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.259322+0000","last_change":"2024-09-20T22:02:54.294653+0000","last_active":"2024-09-20T22:05:43.259322+0000","last_peered":"2024-09-20T22:05:43.259322+0000","last_clean":"2024-09-20T22:05:43.259322+0000","last_became_active":"2024-09-20T22:02:54.294381+0000","last_became_peered":"2024-09-20T22:02:54.294381+0000","last_unstale":"2024-09-20T22:05:43.259322+0000","last_undegraded":"2024-09-20T22:05:43.259322+0000","last_fullsized":"2024-09-20T22:05:43.259322+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:49.060817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":112,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.241958+0000","last_change":"2024-09-20T22:05:00.897974+0000","last_active":"2024-09-20T22:05:43.241958+0000","last_peered":"2024-09-20T22:05:43.241958+0000","last_clean":"2024-09-20T22:05:43.241958+0000","last_became_active":"2024-09-20T22:02:56.300041+0000","last_became_peered":"2024-09-20T22:02:56.300041+0000","last_unstale":"2024-09-20T22:05:43.241958+0000","last_undegraded":"2024-09-20T22:05:43.241958+0000","last_fullsized":"2024-09-20T22:05:43.241958+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:00.897869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:00.897869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:08.153745+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"71'10","reported_seq":129,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.259439+0000","last_change":"2024-09-20T22:04:16.314377+0000","last_active":"2024-09-20T22:05:43.259439+0000","last_peered":"2024-09-20T22:05:43.259439+0000","last_clean":"2024-09-20T22:04:11.310002+0000","last_became_active":"2024-09-20T22:04:16.314377+0000","last_became_peered":"2024-09-20T22:04:16.314377+0000","last_unstale":"2024-09-20T22:05:43.259439+0000","last_undegraded":"2024-09-20T22:04:16.305833+0000","last_fullsized":"2024-09-20T22:04:16.305652+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[{"shards":"3,7","objects":5}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":116,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.257370+0000","last_change":"2024-09-20T22:02:50.112976+0000","last_active":"2024-09-20T22:05:43.257370+0000","last_peered":"2024-09-20T22:05:43.257370+0000","last_clean":"2024-09-20T22:05:43.257370+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:05:43.257370+0000","last_undegraded":"2024-09-20T22:05:43.257370+0000","last_fullsized":"2024-09-20T22:05:43.257370+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:44.796489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"92'9","reported_seq":155,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.253811+0000","last_change":"2024-09-20T22:04:16.325652+0000","last_active":"2024-09-20T22:05:43.253811+0000","last_peered":"2024-09-20T22:05:43.253811+0000","last_clean":"2024-09-20T22:04:15.181393+0000","last_became_active":"2024-09-20T22:04:16.325652+0000","last_became_peered":"2024-09-20T22:04:16.325652+0000","last_unstale":"2024-09-20T22:05:43.253811+0000","last_undegraded":"2024-09-20T22:04:16.311649+0000","last_fullsized":"2024-09-20T22:04:16.311525+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:49.409125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[{"shards":"0,5","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":111,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.263441+0000","last_change":"2024-09-20T22:05:35.601891+0000","last_active":"2024-09-20T22:05:43.263441+0000","last_peered":"2024-09-20T22:05:43.263441+0000","last_clean":"2024-09-20T22:05:43.263441+0000","last_became_active":"2024-09-20T22:02:56.305383+0000","last_became_peered":"2024-09-20T22:02:56.305383+0000","last_unstale":"2024-09-20T22:05:43.263441+0000","last_undegraded":"2024-09-20T22:05:43.263441+0000","last_fullsized":"2024-09-20T22:05:43.263441+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:35.601794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:35.601794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:50.376685+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"71'11","reported_seq":134,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.244151+0000","last_change":"2024-09-20T22:04:16.316014+0000","last_active":"2024-09-20T22:05:43.244151+0000","last_peered":"2024-09-20T22:05:43.244151+0000","last_clean":"2024-09-20T22:04:11.303420+0000","last_became_active":"2024-09-20T22:04:16.316014+0000","last_became_peered":"2024-09-20T22:04:16.316014+0000","last_unstale":"2024-09-20T22:05:43.244151+0000","last_undegraded":"2024-09-20T22:04:16.304443+0000","last_fullsized":"2024-09-20T22:04:16.304329+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[{"shards":"4,6","objects":5}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":124,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.241374+0000","last_change":"2024-09-20T22:04:54.764407+0000","last_active":"2024-09-20T22:05:43.241374+0000","last_peered":"2024-09-20T22:05:43.241374+0000","last_clean":"2024-09-20T22:05:43.241374+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:05:43.241374+0000","last_undegraded":"2024-09-20T22:05:43.241374+0000","last_fullsized":"2024-09-20T22:05:43.241374+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:54.764137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:54.764137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:07.236633+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":57,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.253429+0000","last_change":"2024-09-20T22:04:16.322913+0000","last_active":"2024-09-20T22:05:43.253429+0000","last_peered":"2024-09-20T22:05:43.253429+0000","last_clean":"2024-09-20T22:02:53.269054+0000","last_became_active":"2024-09-20T22:04:16.322913+0000","last_became_peered":"2024-09-20T22:04:16.322913+0000","last_unstale":"2024-09-20T22:05:43.253429+0000","last_undegraded":"2024-09-20T22:05:43.253429+0000","last_fullsized":"2024-09-20T22:04:16.308557+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:43.466991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":61,"reported_epoch":116,"state":"active+clean+remapped","last_fresh":"2024-09-20T22:05:43.255640+0000","last_change":"2024-09-20T22:05:33.624254+0000","last_active":"2024-09-20T22:05:43.255640+0000","last_peered":"2024-09-20T22:05:43.255640+0000","last_clean":"2024-09-20T22:05:43.255640+0000","last_became_active":"2024-09-20T22:04:21.312463+0000","last_became_peered":"2024-09-20T22:04:21.312463+0000","last_unstale":"2024-09-20T22:05:43.255640+0000","last_undegraded":"2024-09-20T22:05:43.255640+0000","last_fullsized":"2024-09-20T22:05:43.255640+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:33.623816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:33.623816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:02.089979+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5,3],"avail_no_missing":["0","3","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"71'9","reported_seq":92,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.252623+0000","last_change":"2024-09-20T22:04:16.319241+0000","last_active":"2024-09-20T22:05:43.252623+0000","last_peered":"2024-09-20T22:05:43.252623+0000","last_clean":"2024-09-20T22:02:57.545555+0000","last_became_active":"2024-09-20T22:04:16.319241+0000","last_became_peered":"2024-09-20T22:04:16.319241+0000","last_unstale":"2024-09-20T22:05:43.252623+0000","last_undegraded":"2024-09-20T22:04:16.306520+0000","last_fullsized":"2024-09-20T22:04:16.306412+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:12.196518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:12.196518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":["6","2"],"object_location_counts":[{"shards":"2,6","objects":4}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":132,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.257246+0000","last_change":"2024-09-20T22:05:22.727380+0000","last_active":"2024-09-20T22:05:43.257246+0000","last_peered":"2024-09-20T22:05:43.257246+0000","last_clean":"2024-09-20T22:05:43.257246+0000","last_became_active":"2024-09-20T22:02:50.109803+0000","last_became_peered":"2024-09-20T22:02:50.109803+0000","last_unstale":"2024-09-20T22:05:43.257246+0000","last_undegraded":"2024-09-20T22:05:43.257246+0000","last_fullsized":"2024-09-20T22:05:43.257246+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:22.727282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:22.727282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:24.915671+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":110,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.260961+0000","last_change":"2024-09-20T22:04:16.312705+0000","last_active":"2024-09-20T22:05:43.260961+0000","last_peered":"2024-09-20T22:05:43.260961+0000","last_clean":"2024-09-20T22:04:11.312860+0000","last_became_active":"2024-09-20T22:04:16.312705+0000","last_became_peered":"2024-09-20T22:04:16.312705+0000","last_unstale":"2024-09-20T22:05:43.260961+0000","last_undegraded":"2024-09-20T22:05:43.260961+0000","last_fullsized":"2024-09-20T22:04:16.306905+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":106,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.255408+0000","last_change":"2024-09-20T22:04:16.315649+0000","last_active":"2024-09-20T22:05:43.255408+0000","last_peered":"2024-09-20T22:05:43.255408+0000","last_clean":"2024-09-20T22:04:11.306119+0000","last_became_active":"2024-09-20T22:04:16.315649+0000","last_became_peered":"2024-09-20T22:04:16.315649+0000","last_unstale":"2024-09-20T22:05:43.255408+0000","last_undegraded":"2024-09-20T22:05:43.255408+0000","last_fullsized":"2024-09-20T22:04:16.302635+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:47.557388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"71'11","reported_seq":138,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.258304+0000","last_change":"2024-09-20T22:04:20.835713+0000","last_active":"2024-09-20T22:05:43.258304+0000","last_peered":"2024-09-20T22:05:43.258304+0000","last_clean":"2024-09-20T22:05:43.258304+0000","last_became_active":"2024-09-20T22:02:52.480118+0000","last_became_peered":"2024-09-20T22:02:52.480118+0000","last_unstale":"2024-09-20T22:05:43.258304+0000","last_undegraded":"2024-09-20T22:05:43.258304+0000","last_fullsized":"2024-09-20T22:05:43.258304+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:04:20.835626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:20.835626+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":126,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.253964+0000","last_change":"2024-09-20T22:04:16.325403+0000","last_active":"2024-09-20T22:05:43.253964+0000","last_peered":"2024-09-20T22:05:43.253964+0000","last_clean":"2024-09-20T22:04:14.620696+0000","last_became_active":"2024-09-20T22:04:16.325403+0000","last_became_peered":"2024-09-20T22:04:16.325403+0000","last_unstale":"2024-09-20T22:05:43.253964+0000","last_undegraded":"2024-09-20T22:05:43.253964+0000","last_fullsized":"2024-09-20T22:04:16.313799+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:14.620563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:14.620563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:43.467073+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":116,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.257728+0000","last_change":"2024-09-20T22:04:57.641872+0000","last_active":"2024-09-20T22:05:43.257728+0000","last_peered":"2024-09-20T22:05:43.257728+0000","last_clean":"2024-09-20T22:05:43.257728+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:05:43.257728+0000","last_undegraded":"2024-09-20T22:05:43.257728+0000","last_fullsized":"2024-09-20T22:05:43.257728+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:57.641748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:57.641748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:08.376222+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":111,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.261203+0000","last_change":"2024-09-20T22:05:30.899063+0000","last_active":"2024-09-20T22:05:43.261203+0000","last_peered":"2024-09-20T22:05:43.261203+0000","last_clean":"2024-09-20T22:05:43.261203+0000","last_became_active":"2024-09-20T22:02:56.347637+0000","last_became_peered":"2024-09-20T22:02:56.347637+0000","last_unstale":"2024-09-20T22:05:43.261203+0000","last_undegraded":"2024-09-20T22:05:43.261203+0000","last_fullsized":"2024-09-20T22:05:43.261203+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:30.898956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:30.898956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:31.255983+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"71'4","reported_seq":120,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.260101+0000","last_change":"2024-09-20T22:04:16.310098+0000","last_active":"2024-09-20T22:05:43.260101+0000","last_peered":"2024-09-20T22:05:43.260101+0000","last_clean":"2024-09-20T22:04:11.310261+0000","last_became_active":"2024-09-20T22:04:16.310098+0000","last_became_peered":"2024-09-20T22:04:16.310098+0000","last_unstale":"2024-09-20T22:05:43.260101+0000","last_undegraded":"2024-09-20T22:04:16.306006+0000","last_fullsized":"2024-09-20T22:04:16.305667+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[{"shards":"3,6","objects":2}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":123,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.263278+0000","last_change":"2024-09-20T22:04:16.198264+0000","last_active":"2024-09-20T22:05:43.263278+0000","last_peered":"2024-09-20T22:05:43.263278+0000","last_clean":"2024-09-20T22:05:43.263278+0000","last_became_active":"2024-09-20T22:02:50.111142+0000","last_became_peered":"2024-09-20T22:02:50.111142+0000","last_unstale":"2024-09-20T22:05:43.263278+0000","last_undegraded":"2024-09-20T22:05:43.263278+0000","last_fullsized":"2024-09-20T22:05:43.263278+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:16.198153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:16.198153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:44.335348+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":123,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.254062+0000","last_change":"2024-09-20T22:05:26.146698+0000","last_active":"2024-09-20T22:05:43.254062+0000","last_peered":"2024-09-20T22:05:43.254062+0000","last_clean":"2024-09-20T22:05:43.254062+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:05:43.254062+0000","last_undegraded":"2024-09-20T22:05:43.254062+0000","last_fullsized":"2024-09-20T22:05:43.254062+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:26.146625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:26.146625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:34.163023+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":104,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.257306+0000","last_change":"2024-09-20T22:02:56.300559+0000","last_active":"2024-09-20T22:05:43.257306+0000","last_peered":"2024-09-20T22:05:43.257306+0000","last_clean":"2024-09-20T22:05:43.257306+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:05:43.257306+0000","last_undegraded":"2024-09-20T22:05:43.257306+0000","last_fullsized":"2024-09-20T22:05:43.257306+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:44.796461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":57,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.253909+0000","last_change":"2024-09-20T22:04:16.324831+0000","last_active":"2024-09-20T22:05:43.253909+0000","last_peered":"2024-09-20T22:05:43.253909+0000","last_clean":"2024-09-20T22:02:55.278800+0000","last_became_active":"2024-09-20T22:04:16.324831+0000","last_became_peered":"2024-09-20T22:04:16.324831+0000","last_unstale":"2024-09-20T22:05:43.253909+0000","last_undegraded":"2024-09-20T22:05:43.253909+0000","last_fullsized":"2024-09-20T22:04:16.311809+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:15.303300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:15.303300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:05:43.467066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":118,"reported_epoch":116,"state":"active+undersized","last_fresh":"2024-09-20T22:05:43.260478+0000","last_change":"2024-09-20T22:04:16.309674+0000","last_active":"2024-09-20T22:05:43.260478+0000","last_peered":"2024-09-20T22:05:43.260478+0000","last_clean":"2024-09-20T22:04:11.311023+0000","last_became_active":"2024-09-20T22:04:16.309674+0000","last_became_peered":"2024-09-20T22:04:16.309674+0000","last_unstale":"2024-09-20T22:05:43.260478+0000","last_undegraded":"2024-09-20T22:05:43.260478+0000","last_fullsized":"2024-09-20T22:04:16.304826+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"71'11","reported_seq":133,"reported_epoch":116,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:05:43.255109+0000","last_change":"2024-09-20T22:04:16.326833+0000","last_active":"2024-09-20T22:05:43.255109+0000","last_peered":"2024-09-20T22:05:43.255109+0000","last_clean":"2024-09-20T22:04:11.307316+0000","last_became_active":"2024-09-20T22:04:16.326833+0000","last_became_peered":"2024-09-20T22:04:16.326833+0000","last_unstale":"2024-09-20T22:05:43.255109+0000","last_undegraded":"2024-09-20T22:04:16.307528+0000","last_fullsized":"2024-09-20T22:04:16.307421+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[{"shards":"5,6","objects":5}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":115,"reported_epoch":116,"state":"active+clean","last_fresh":"2024-09-20T22:05:43.263570+0000","last_change":"2024-09-20T22:05:06.706314+0000","last_active":"2024-09-20T22:05:43.263570+0000","last_peered":"2024-09-20T22:05:43.263570+0000","last_clean":"2024-09-20T22:05:43.263570+0000","last_became_active":"2024-09-20T22:02:54.289078+0000","last_became_peered":"2024-09-20T22:02:54.289078+0000","last_unstale":"2024-09-20T22:05:43.263570+0000","last_undegraded":"2024-09-20T22:05:43.263570+0000","last_fullsized":"2024-09-20T22:05:43.263570+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:06.681123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:06.681123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:18.782996+0000","scrub_duration":82,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":16,"stat_sum":{"num_bytes":7280197656,"num_objects":111088,"num_object_clones":0,"num_object_copies":333264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41588,"num_objects_misplaced":6964,"num_objects_unfound":0,"num_objects_dirty":111088,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":154974,"num_write_kb":8513921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20090396672,"data_stored":20090388528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91874,"ondisk_log_size":91874,"up":41,"acting":42,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":84,"acting":89,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":82,"acting":82,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":76,"acting":76,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":85,"acting":87,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":165,"num_read_kb":160,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":7,"acting":7,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135911,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2311352,"kb_used_data":2239912,"kb_used_omap":1,"kb_used_meta":71422,"kb_avail":91433800,"statfs":{"total":95995035648,"available":93628211200,"internally_reserved":0,"allocated":2293669888,"data_stored":2293247840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":73136586},"hb_peers":[0,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430251,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2334896,"kb_used_data":2227356,"kb_used_omap":1,"kb_used_meta":107518,"kb_avail":91410256,"statfs":{"total":95995035648,"available":93604102144,"internally_reserved":0,"allocated":2280812544,"data_stored":2280402716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":110098886},"hb_peers":[0,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724591,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1878756,"kb_used_data":1793300,"kb_used_omap":1,"kb_used_meta":85310,"kb_avail":91866396,"statfs":{"total":95995035648,"available":94071189504,"internally_reserved":0,"allocated":1836339200,"data_stored":1835937342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87357893},"hb_peers":[0,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986231,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2371712,"kb_used_data":2259948,"kb_used_omap":1,"kb_used_meta":111678,"kb_avail":91373440,"statfs":{"total":95995035648,"available":93566402560,"internally_reserved":0,"allocated":2314186752,"data_stored":2313760007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":114358725},"hb_peers":[0,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":968916,"kb_used_data":941892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":92776236,"statfs":{"total":95995035648,"available":95002865664,"internally_reserved":0,"allocated":964497408,"data_stored":963964136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149756,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4164172,"kb_used_data":4028860,"kb_used_omap":1,"kb_used_meta":135294,"kb_avail":89580980,"statfs":{"total":95995035648,"available":91730923520,"internally_reserved":0,"allocated":4125552640,"data_stored":4125110803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":138541511},"hb_peers":[0,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411393,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2801624,"kb_used_data":2694088,"kb_used_omap":1,"kb_used_meta":107518,"kb_avail":90943528,"statfs":{"total":95995035648,"available":93126172672,"internally_reserved":0,"allocated":2758746112,"data_stored":2758355338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110098885},"hb_peers":[0,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738444,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3694784,"kb_used_data":3546992,"kb_used_omap":1,"kb_used_meta":147774,"kb_avail":90050368,"statfs":{"total":95995035648,"available":92211576832,"internally_reserved":0,"allocated":3632119808,"data_stored":3631689724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":151321029},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3622047744,"data_stored":3622043672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":973668352,"data_stored":973668352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2736193536,"data_stored":2736193536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4089970688,"data_stored":4089970688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2298019840,"data_stored":2298019840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1830813696,"data_stored":1830813696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2270302208,"data_stored":2270298136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2269380608,"data_stored":2269380608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:05:53.884 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-20T22:05:53.884 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool set unique_pool_0 pg_num 26 2024-09-20T22:05:53.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:53.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:53.963 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:53.963 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:54.064 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:54.124 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:54.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:54.125 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:54.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:54.272 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:54.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:54.273 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:54.374 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:54.399 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:54.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:54.400 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:54.492 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:54 smithi060 ceph-mon[29694]: pgmap v125: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.4 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 760 op/s; 39398/315612 objects degraded (12.483%); 6583/315612 objects misplaced (2.086%) 2024-09-20T22:05:54.501 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:54.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:54 smithi060 ceph-mon[40290]: pgmap v125: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.4 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 760 op/s; 39398/315612 objects degraded (12.483%); 6583/315612 objects misplaced (2.086%) 2024-09-20T22:05:54.508 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:54.525 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:54.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:54.526 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:54.580 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:54.626 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:54.652 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:54.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:54.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:54.754 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:54 smithi067 ceph-mon[33511]: pgmap v125: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.4 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 760 op/s; 39398/315612 objects degraded (12.483%); 6583/315612 objects misplaced (2.086%) 2024-09-20T22:05:54.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:54.801 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:54.801 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:54.847 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:05:54.902 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:54.931 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:54.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:54.932 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:55.033 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:55.075 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:55.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:55.076 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:55.170 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:05:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:05:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:05:55.177 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:55.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:55.206 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:55.206 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:55.306 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:55.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:55.334 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:55.334 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:55.399 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:05:55.433 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:55 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3946807089' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:05:55.433 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:55 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3946807089' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:05:55.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:55.461 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:05:55.461 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:05:55.488 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:55.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:55.489 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:55.590 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:55.624 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:55.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:55.625 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:55.726 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3946807089' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:05:55.766 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:55.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:55.767 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:55.810 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:05:55.811 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:05:55.811 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:05:55.811 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:05:55.811 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:05:55.811 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:05:55.811 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:05:55.831 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:05:55.831 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:05:55.864 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:05:55.864 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:05:55.867 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:55.901 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:05:55.901 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:05:55.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:55.916 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:55.916 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:55.935 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:05:55.935 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:05:56.017 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:56.050 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:56.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:56.051 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:56.151 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:56.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:56.188 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:56.188 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:56.289 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:56.317 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:56.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:56.318 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:56.418 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:56.449 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:56.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:56.450 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:56.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:56 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:05:56.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:56 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3946807089' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:05:56.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:56 smithi060 ceph-mon[29694]: osdmap e119: 8 total, 7 up, 8 in 2024-09-20T22:05:56.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:56 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4153162260' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:56.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:56 smithi060 ceph-mon[29694]: pgmap v127: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.50k op/s; 37390/299295 objects degraded (12.493%); 6280/299295 objects misplaced (2.098%) 2024-09-20T22:05:56.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:56 smithi060 ceph-mon[29694]: 5.14 scrub starts 2024-09-20T22:05:56.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:56 smithi060 ceph-mon[29694]: 5.14 scrub ok 2024-09-20T22:05:56.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:56 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 39398/315612 objects degraded (12.483%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:56 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:05:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:56 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3946807089' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:05:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:56 smithi060 ceph-mon[40290]: osdmap e119: 8 total, 7 up, 8 in 2024-09-20T22:05:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:56 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4153162260' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:56 smithi060 ceph-mon[40290]: pgmap v127: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.50k op/s; 37390/299295 objects degraded (12.493%); 6280/299295 objects misplaced (2.098%) 2024-09-20T22:05:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:56 smithi060 ceph-mon[40290]: 5.14 scrub starts 2024-09-20T22:05:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:56 smithi060 ceph-mon[40290]: 5.14 scrub ok 2024-09-20T22:05:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:56 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 39398/315612 objects degraded (12.483%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:56.550 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:56.578 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:56.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:56.579 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:56.680 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:56.713 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:56.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:56.714 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:56 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:05:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:56 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3946807089' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:05:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:56 smithi067 ceph-mon[33511]: osdmap e119: 8 total, 7 up, 8 in 2024-09-20T22:05:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:56 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4153162260' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:05:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:56 smithi067 ceph-mon[33511]: pgmap v127: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.50k op/s; 37390/299295 objects degraded (12.493%); 6280/299295 objects misplaced (2.098%) 2024-09-20T22:05:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:56 smithi067 ceph-mon[33511]: 5.14 scrub starts 2024-09-20T22:05:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:56 smithi067 ceph-mon[33511]: 5.14 scrub ok 2024-09-20T22:05:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:56 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 39398/315612 objects degraded (12.483%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:05:56.815 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:56.845 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:56.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:56.846 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:56.946 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:56.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:56.987 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:56.987 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:57.088 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:57.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:57.125 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:57.125 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:57.226 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:57.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:57.258 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:57.259 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:57.359 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:57.399 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:57.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:57.400 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:57.406 INFO:teuthology.orchestra.run.smithi060.stderr:set pool 7 pg_num to 26 2024-09-20T22:05:57.439 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T22:05:57.501 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:57.540 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:57.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:57.541 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:57.641 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:05:57.675 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:57.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:57.676 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:05:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:57 smithi067 ceph-mon[33511]: 5.4 scrub starts 2024-09-20T22:05:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:57 smithi067 ceph-mon[33511]: 5.4 scrub ok 2024-09-20T22:05:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:57 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2023971617' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num", "val": "26"}]: dispatch 2024-09-20T22:05:57.771 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:57 smithi060 ceph-mon[29694]: 5.4 scrub starts 2024-09-20T22:05:57.771 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:57 smithi060 ceph-mon[29694]: 5.4 scrub ok 2024-09-20T22:05:57.771 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:57 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2023971617' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num", "val": "26"}]: dispatch 2024-09-20T22:05:57.771 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:57 smithi060 ceph-mon[40290]: 5.4 scrub starts 2024-09-20T22:05:57.771 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:57 smithi060 ceph-mon[40290]: 5.4 scrub ok 2024-09-20T22:05:57.771 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:57 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2023971617' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num", "val": "26"}]: dispatch 2024-09-20T22:05:57.776 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:57.813 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:57.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:57.814 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:57.821 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:05:57.915 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:57.953 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:57.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:57.954 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:58.055 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:58.092 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:58.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:58.093 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:58.194 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:58.217 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:58.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:58.218 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:58.222 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:05:58.223 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:05:58.318 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:58.358 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:58.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:58.359 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:58.460 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:58.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:58.497 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:58.498 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:58.598 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:58.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:58.624 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:58.625 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:58.677 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:58 smithi067 ceph-mon[33511]: 6.d scrub starts 2024-09-20T22:05:58.677 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:58 smithi067 ceph-mon[33511]: 6.d scrub ok 2024-09-20T22:05:58.677 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:58 smithi067 ceph-mon[33511]: 5.f scrub starts 2024-09-20T22:05:58.677 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:58 smithi067 ceph-mon[33511]: 5.f scrub ok 2024-09-20T22:05:58.677 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:58 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2023971617' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num", "val": "26"}]': finished 2024-09-20T22:05:58.677 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:58 smithi067 ceph-mon[33511]: osdmap e120: 8 total, 7 up, 8 in 2024-09-20T22:05:58.677 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:58 smithi067 ceph-mon[33511]: 7.5 scrub starts 2024-09-20T22:05:58.678 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:58 smithi067 ceph-mon[33511]: pgmap v129: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.44k op/s; 37390/299295 objects degraded (12.493%); 6280/299295 objects misplaced (2.098%) 2024-09-20T22:05:58.678 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:58 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-20T22:05:58.678 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:58 smithi067 ceph-mon[33511]: 4.5 scrub starts 2024-09-20T22:05:58.678 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:58 smithi067 ceph-mon[33511]: 4.5 scrub ok 2024-09-20T22:05:58.711 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:05:58.711 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:05:58.711 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.711 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169955 7.1 7.da7beda1 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:05:58.711 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.440021+0000", 2024-09-20T22:05:58.711 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.26946128800000002, 2024-09-20T22:05:58.712 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.26948521399999997, 2024-09-20T22:05:58.712 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:05:58.712 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:05:58.712 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:05:58.712 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:05:58.712 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:05:58.712 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:58.712 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169955 2024-09-20T22:05:58.712 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.712 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:05:58.712 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.712 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:05:58.712 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440021+0000", 2024-09-20T22:05:58.712 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:05:58.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.713 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.713 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:05:58.713 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440021+0000", 2024-09-20T22:05:58.713 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:05:58.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.713 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.713 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:05:58.713 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440025+0000", 2024-09-20T22:05:58.713 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4429999999999996e-06 2024-09-20T22:05:58.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.713 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.713 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:05:58.713 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440034+0000", 2024-09-20T22:05:58.713 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.7190000000000007e-06 2024-09-20T22:05:58.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.713 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.714 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:05:58.714 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440055+0000", 2024-09-20T22:05:58.714 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1146999999999999e-05 2024-09-20T22:05:58.714 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.714 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.714 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:05:58.714 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440081+0000", 2024-09-20T22:05:58.714 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.5432e-05 2024-09-20T22:05:58.714 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.714 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.714 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:05:58.714 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601707+0000", 2024-09-20T22:05:58.714 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.161626409 2024-09-20T22:05:58.714 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.714 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.715 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:05:58.715 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601721+0000", 2024-09-20T22:05:58.715 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.36e-05 2024-09-20T22:05:58.715 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:05:58.715 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:05:58.715 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:05:58.715 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.715 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.715 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169956 7.1 7.16b721a1 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:05:58.715 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.440104+0000", 2024-09-20T22:05:58.715 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.26937806800000003, 2024-09-20T22:05:58.715 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.269446203, 2024-09-20T22:05:58.715 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:05:58.715 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:05:58.716 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:05:58.716 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:05:58.716 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:05:58.716 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:58.716 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169956 2024-09-20T22:05:58.716 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.716 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:05:58.716 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.716 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:05:58.716 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440104+0000", 2024-09-20T22:05:58.716 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:05:58.716 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.716 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.716 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:05:58.716 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440104+0000", 2024-09-20T22:05:58.716 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:05:58.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.717 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.717 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:05:58.717 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440105+0000", 2024-09-20T22:05:58.717 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.68e-07 2024-09-20T22:05:58.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.717 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.717 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:05:58.717 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440108+0000", 2024-09-20T22:05:58.717 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.92e-06 2024-09-20T22:05:58.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.717 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.717 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:05:58.717 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440116+0000", 2024-09-20T22:05:58.717 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.276e-06 2024-09-20T22:05:58.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.718 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.718 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:05:58.718 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440121+0000", 2024-09-20T22:05:58.718 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5940000000000001e-06 2024-09-20T22:05:58.718 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.718 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.718 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:05:58.718 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601779+0000", 2024-09-20T22:05:58.718 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.16165766000000001 2024-09-20T22:05:58.718 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.718 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.718 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:05:58.718 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601788+0000", 2024-09-20T22:05:58.718 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.149e-06 2024-09-20T22:05:58.718 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:05:58.719 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:05:58.719 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:05:58.719 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.719 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.719 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169965 7.11 7.31051ab1 (undecoded) ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:05:58.719 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.440131+0000", 2024-09-20T22:05:58.719 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.26935115100000001, 2024-09-20T22:05:58.719 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.26945542500000003, 2024-09-20T22:05:58.719 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:05:58.719 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:05:58.719 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:05:58.719 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:05:58.719 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:05:58.719 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:58.719 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169965 2024-09-20T22:05:58.720 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.720 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:05:58.720 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.720 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:05:58.720 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440131+0000", 2024-09-20T22:05:58.720 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:05:58.720 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.720 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.720 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:05:58.720 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440131+0000", 2024-09-20T22:05:58.720 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:05:58.720 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.720 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.720 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:05:58.720 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440131+0000", 2024-09-20T22:05:58.720 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9900000000000001e-07 2024-09-20T22:05:58.720 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.721 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.721 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:05:58.721 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440134+0000", 2024-09-20T22:05:58.721 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1100000000000001e-06 2024-09-20T22:05:58.721 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.721 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.721 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:05:58.721 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440144+0000", 2024-09-20T22:05:58.721 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0339999999999999e-05 2024-09-20T22:05:58.721 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.721 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.721 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:05:58.721 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440149+0000", 2024-09-20T22:05:58.721 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8339999999999998e-06 2024-09-20T22:05:58.721 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.722 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.722 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:05:58.722 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601764+0000", 2024-09-20T22:05:58.722 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.16161526700000001 2024-09-20T22:05:58.722 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.722 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.722 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:05:58.722 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601772+0000", 2024-09-20T22:05:58.722 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2220000000000002e-06 2024-09-20T22:05:58.722 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:05:58.722 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:05:58.722 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:05:58.722 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.722 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.723 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169972 7.1 7.b1944aa1 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:05:58.723 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.441015+0000", 2024-09-20T22:05:58.723 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.26846723700000003, 2024-09-20T22:05:58.723 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.26860649599999997, 2024-09-20T22:05:58.723 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:05:58.723 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:05:58.723 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:05:58.723 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:05:58.723 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:05:58.723 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:05:58.723 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169972 2024-09-20T22:05:58.723 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.723 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:05:58.723 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.723 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:05:58.723 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.441015+0000", 2024-09-20T22:05:58.724 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:05:58.724 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.724 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.724 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:05:58.724 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.441015+0000", 2024-09-20T22:05:58.724 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:05:58.724 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.724 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.724 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:05:58.724 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.441017+0000", 2024-09-20T22:05:58.724 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.604e-06 2024-09-20T22:05:58.724 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.724 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.724 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:05:58.724 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.441025+0000", 2024-09-20T22:05:58.724 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2949999999999994e-06 2024-09-20T22:05:58.724 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.725 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.725 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:05:58.725 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.441044+0000", 2024-09-20T22:05:58.725 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8695e-05 2024-09-20T22:05:58.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.725 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.725 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:05:58.725 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.441052+0000", 2024-09-20T22:05:58.725 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2109999999999998e-06 2024-09-20T22:05:58.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.725 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.725 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:05:58.725 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601834+0000", 2024-09-20T22:05:58.725 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.16078221000000001 2024-09-20T22:05:58.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:05:58.725 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:05:58.726 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:05:58.726 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601844+0000", 2024-09-20T22:05:58.726 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9599999999999995e-06 2024-09-20T22:05:58.726 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:05:58.726 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:05:58.726 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:05:58.726 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:05:58.726 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-20T22:05:58.726 INFO:teuthology.orchestra.run.smithi067.stdout: "num_ops": 4 2024-09-20T22:05:58.726 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:05:58.726 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:58.748 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:58.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:58.749 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:58.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[29694]: 6.d scrub starts 2024-09-20T22:05:58.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[29694]: 6.d scrub ok 2024-09-20T22:05:58.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[29694]: 5.f scrub starts 2024-09-20T22:05:58.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[29694]: 5.f scrub ok 2024-09-20T22:05:58.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2023971617' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num", "val": "26"}]': finished 2024-09-20T22:05:58.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[29694]: osdmap e120: 8 total, 7 up, 8 in 2024-09-20T22:05:58.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[29694]: 7.5 scrub starts 2024-09-20T22:05:58.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[29694]: pgmap v129: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.44k op/s; 37390/299295 objects degraded (12.493%); 6280/299295 objects misplaced (2.098%) 2024-09-20T22:05:58.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-20T22:05:58.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[29694]: 4.5 scrub starts 2024-09-20T22:05:58.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[29694]: 4.5 scrub ok 2024-09-20T22:05:58.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[40290]: 6.d scrub starts 2024-09-20T22:05:58.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[40290]: 6.d scrub ok 2024-09-20T22:05:58.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[40290]: 5.f scrub starts 2024-09-20T22:05:58.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[40290]: 5.f scrub ok 2024-09-20T22:05:58.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2023971617' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num", "val": "26"}]': finished 2024-09-20T22:05:58.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[40290]: osdmap e120: 8 total, 7 up, 8 in 2024-09-20T22:05:58.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[40290]: 7.5 scrub starts 2024-09-20T22:05:58.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[40290]: pgmap v129: 148 pgs: 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.44k op/s; 37390/299295 objects degraded (12.493%); 6280/299295 objects misplaced (2.098%) 2024-09-20T22:05:58.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-20T22:05:58.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[40290]: 4.5 scrub starts 2024-09-20T22:05:58.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:58 smithi060 ceph-mon[40290]: 4.5 scrub ok 2024-09-20T22:05:58.850 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:58.873 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:58.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:58.874 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:58.974 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:05:58.996 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:58.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:58.997 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:05:59.098 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:59.120 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:59.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:59.121 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:59.222 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:05:59.244 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:59.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:59.245 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:05:59.345 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:05:59.367 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:05:59.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:59.368 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:05:59.469 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:05:59.502 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:59.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:59.503 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:05:59.543 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T22:05:59.603 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:59.666 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:59.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:59.668 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:59 smithi067 ceph-mon[33511]: 7.1 scrub starts 2024-09-20T22:05:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:59 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-20T22:05:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:05:59 smithi067 ceph-mon[33511]: osdmap e121: 8 total, 7 up, 8 in 2024-09-20T22:05:59.769 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:05:59.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:59 smithi060 ceph-mon[29694]: 7.1 scrub starts 2024-09-20T22:05:59.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:59 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-20T22:05:59.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:05:59 smithi060 ceph-mon[29694]: osdmap e121: 8 total, 7 up, 8 in 2024-09-20T22:05:59.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:59 smithi060 ceph-mon[40290]: 7.1 scrub starts 2024-09-20T22:05:59.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:59 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-20T22:05:59.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:05:59 smithi060 ceph-mon[40290]: osdmap e121: 8 total, 7 up, 8 in 2024-09-20T22:05:59.809 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:59.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:59.810 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:05:59.911 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:05:59.953 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:05:59.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:05:59.954 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:05:59.991 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:06:00.055 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:00.098 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:00.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:00.099 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:00.200 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:00.236 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:06:00.281 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:00.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:00.282 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:00.383 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:00.415 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:00.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:00.416 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:00.517 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:00.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[40290]: 3.3 scrub starts 2024-09-20T22:06:00.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[40290]: 3.3 scrub ok 2024-09-20T22:06:00.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[40290]: osdmap e122: 8 total, 7 up, 8 in 2024-09-20T22:06:00.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[40290]: 7.6 scrub starts 2024-09-20T22:06:00.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[40290]: 7.5 scrub starts 2024-09-20T22:06:00.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[40290]: 7.13 scrub starts 2024-09-20T22:06:00.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[40290]: pgmap v132: 158 pgs: 10 unknown, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.20k op/s; 37390/299295 objects degraded (12.493%); 6280/299295 objects misplaced (2.098%) 2024-09-20T22:06:00.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-20T22:06:00.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[40290]: 7.6 scrub ok 2024-09-20T22:06:00.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[40290]: 7.7 scrub starts 2024-09-20T22:06:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[29694]: 3.3 scrub starts 2024-09-20T22:06:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[29694]: 3.3 scrub ok 2024-09-20T22:06:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[29694]: osdmap e122: 8 total, 7 up, 8 in 2024-09-20T22:06:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[29694]: 7.6 scrub starts 2024-09-20T22:06:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[29694]: 7.5 scrub starts 2024-09-20T22:06:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[29694]: 7.13 scrub starts 2024-09-20T22:06:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[29694]: pgmap v132: 158 pgs: 10 unknown, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.20k op/s; 37390/299295 objects degraded (12.493%); 6280/299295 objects misplaced (2.098%) 2024-09-20T22:06:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-20T22:06:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[29694]: 7.6 scrub ok 2024-09-20T22:06:00.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:00 smithi060 ceph-mon[29694]: 7.7 scrub starts 2024-09-20T22:06:00.545 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:00.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:00.546 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:00.619 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:00.628 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:06:00.628 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:06:00.628 INFO:teuthology.orchestra.run.smithi067.stdout: "complaint_time": 30, 2024-09-20T22:06:00.628 INFO:teuthology.orchestra.run.smithi067.stdout: "num_blocked_ops": 0 2024-09-20T22:06:00.628 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:06:00.648 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:00.672 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:00.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:00.673 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:00 smithi067 ceph-mon[33511]: 3.3 scrub starts 2024-09-20T22:06:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:00 smithi067 ceph-mon[33511]: 3.3 scrub ok 2024-09-20T22:06:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:00 smithi067 ceph-mon[33511]: osdmap e122: 8 total, 7 up, 8 in 2024-09-20T22:06:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:00 smithi067 ceph-mon[33511]: 7.6 scrub starts 2024-09-20T22:06:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:00 smithi067 ceph-mon[33511]: 7.5 scrub starts 2024-09-20T22:06:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:00 smithi067 ceph-mon[33511]: 7.13 scrub starts 2024-09-20T22:06:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:00 smithi067 ceph-mon[33511]: pgmap v132: 158 pgs: 10 unknown, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 82 active+clean; 6.1 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.20k op/s; 37390/299295 objects degraded (12.493%); 6280/299295 objects misplaced (2.098%) 2024-09-20T22:06:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:00 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-20T22:06:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:00 smithi067 ceph-mon[33511]: 7.6 scrub ok 2024-09-20T22:06:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:00 smithi067 ceph-mon[33511]: 7.7 scrub starts 2024-09-20T22:06:00.774 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:00.796 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:00.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:00.797 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:00.898 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:00.920 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:00.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:00.921 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:01.022 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:01.045 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:01.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:01.046 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:01.147 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:01.169 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:01.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:01.170 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:01.207 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:06:01.208 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:06:01.208 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:06:01.223 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:06:01.223 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:06:01.240 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:06:01.241 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:06:01.258 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:06:01.258 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:06:01.270 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:01.275 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:06:01.275 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:06:01.293 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:06:01.293 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:06:01.294 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:01.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:01.295 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:01.310 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:06:01.310 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:06:01.339 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:06:01.378 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T22:06:01.395 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:01.444 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:01.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:01.445 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:01.546 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:01.568 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:01.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:01.569 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:01.639 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:06:01.669 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:01.691 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:01.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:01.692 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:01.760 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:01 smithi067 ceph-mon[33511]: 7.1 scrub starts 2024-09-20T22:06:01.760 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:01 smithi067 ceph-mon[33511]: 7.3 scrub starts 2024-09-20T22:06:01.760 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:01 smithi067 ceph-mon[33511]: 7.5 scrub ok 2024-09-20T22:06:01.760 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:01 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-20T22:06:01.760 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:01 smithi067 ceph-mon[33511]: osdmap e123: 8 total, 7 up, 8 in 2024-09-20T22:06:01.760 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:01 smithi067 ceph-mon[33511]: 7.1 scrub ok 2024-09-20T22:06:01.761 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:01 smithi067 ceph-mon[33511]: 6.0 scrub starts 2024-09-20T22:06:01.761 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:01 smithi067 ceph-mon[33511]: 6.0 scrub ok 2024-09-20T22:06:01.761 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:01 smithi067 ceph-mon[33511]: 7.3 scrub ok 2024-09-20T22:06:01.761 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:01 smithi067 ceph-mon[33511]: 7.7 scrub ok 2024-09-20T22:06:01.761 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:01 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1721205745' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:01.761 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:01 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 37390/299295 objects degraded (12.493%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:01.761 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:01 smithi067 ceph-mon[33511]: osdmap e124: 8 total, 7 up, 8 in 2024-09-20T22:06:01.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[29694]: 7.1 scrub starts 2024-09-20T22:06:01.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[29694]: 7.3 scrub starts 2024-09-20T22:06:01.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[29694]: 7.5 scrub ok 2024-09-20T22:06:01.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-20T22:06:01.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[29694]: osdmap e123: 8 total, 7 up, 8 in 2024-09-20T22:06:01.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[29694]: 7.1 scrub ok 2024-09-20T22:06:01.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[29694]: 6.0 scrub starts 2024-09-20T22:06:01.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[29694]: 6.0 scrub ok 2024-09-20T22:06:01.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[29694]: 7.3 scrub ok 2024-09-20T22:06:01.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[29694]: 7.7 scrub ok 2024-09-20T22:06:01.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1721205745' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:01.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 37390/299295 objects degraded (12.493%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:01.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[29694]: osdmap e124: 8 total, 7 up, 8 in 2024-09-20T22:06:01.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[40290]: 7.1 scrub starts 2024-09-20T22:06:01.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[40290]: 7.3 scrub starts 2024-09-20T22:06:01.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[40290]: 7.5 scrub ok 2024-09-20T22:06:01.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_0", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-20T22:06:01.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[40290]: osdmap e123: 8 total, 7 up, 8 in 2024-09-20T22:06:01.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[40290]: 7.1 scrub ok 2024-09-20T22:06:01.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[40290]: 6.0 scrub starts 2024-09-20T22:06:01.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[40290]: 6.0 scrub ok 2024-09-20T22:06:01.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[40290]: 7.3 scrub ok 2024-09-20T22:06:01.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[40290]: 7.7 scrub ok 2024-09-20T22:06:01.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1721205745' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:01.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 37390/299295 objects degraded (12.493%), 32 pgs degraded, 58 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:01.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:01 smithi060 ceph-mon[40290]: osdmap e124: 8 total, 7 up, 8 in 2024-09-20T22:06:01.793 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:01.815 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:01.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:01.816 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:01.904 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:06:01.905 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:06:01.916 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:01.941 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:01.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:01.942 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:01.945 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:02.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:06:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:06:02.042 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:02.065 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:02.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:02.066 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:02.166 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:02.189 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:02.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:02.190 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:02.193 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:06:02.193 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:06:02.291 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:02.315 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:02.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:02.316 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:02.416 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:02.447 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:02.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:02.448 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:02.501 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: 4.1 scrub starts 2024-09-20T22:06:02.501 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: 4.1 scrub ok 2024-09-20T22:06:02.501 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: 7.8 scrub starts 2024-09-20T22:06:02.501 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: 7.8 scrub ok 2024-09-20T22:06:02.501 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: 6.12 scrub starts 2024-09-20T22:06:02.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: 6.12 scrub ok 2024-09-20T22:06:02.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:06:02.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: 7.a scrub starts 2024-09-20T22:06:02.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: pgmap v135: 158 pgs: 1 active+clean+scrubbing, 10 unknown, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 23681/213126 objects degraded (11.111%); 2982/213126 objects misplaced (1.399%); 0 B/s, 0 objects/s recovering 2024-09-20T22:06:02.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: osdmap e125: 8 total, 7 up, 8 in 2024-09-20T22:06:02.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: 7.16 starting backfill to osd.4 from (0'0,0'0] MAX to 122'10693 2024-09-20T22:06:02.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: 7.19 starting backfill to osd.2 from (0'0,0'0] MAX to 122'10603 2024-09-20T22:06:02.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: 7.14 starting backfill to osd.2 from (0'0,0'0] MAX to 122'10650 2024-09-20T22:06:02.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: 7.17 starting backfill to osd.2 from (0'0,0'0] MAX to 122'10681 2024-09-20T22:06:02.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: 7.14 starting backfill to osd.3 from (0'0,0'0] MAX to 122'10650 2024-09-20T22:06:02.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: 7.19 starting backfill to osd.6 from (0'0,0'0] MAX to 122'10603 2024-09-20T22:06:02.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: 7.19 starting backfill to osd.7 from (0'0,0'0] MAX to 122'10603 2024-09-20T22:06:02.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: 7.17 starting backfill to osd.5 from (0'0,0'0] MAX to 122'10681 2024-09-20T22:06:02.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[29694]: 7.14 starting backfill to osd.5 from (0'0,0'0] MAX to 122'10650 2024-09-20T22:06:02.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: 4.1 scrub starts 2024-09-20T22:06:02.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: 4.1 scrub ok 2024-09-20T22:06:02.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: 7.8 scrub starts 2024-09-20T22:06:02.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: 7.8 scrub ok 2024-09-20T22:06:02.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: 6.12 scrub starts 2024-09-20T22:06:02.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: 6.12 scrub ok 2024-09-20T22:06:02.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:06:02.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: 7.a scrub starts 2024-09-20T22:06:02.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: pgmap v135: 158 pgs: 1 active+clean+scrubbing, 10 unknown, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 23681/213126 objects degraded (11.111%); 2982/213126 objects misplaced (1.399%); 0 B/s, 0 objects/s recovering 2024-09-20T22:06:02.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: osdmap e125: 8 total, 7 up, 8 in 2024-09-20T22:06:02.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: 7.16 starting backfill to osd.4 from (0'0,0'0] MAX to 122'10693 2024-09-20T22:06:02.503 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: 7.19 starting backfill to osd.2 from (0'0,0'0] MAX to 122'10603 2024-09-20T22:06:02.504 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: 7.14 starting backfill to osd.2 from (0'0,0'0] MAX to 122'10650 2024-09-20T22:06:02.504 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: 7.17 starting backfill to osd.2 from (0'0,0'0] MAX to 122'10681 2024-09-20T22:06:02.504 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: 7.14 starting backfill to osd.3 from (0'0,0'0] MAX to 122'10650 2024-09-20T22:06:02.504 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: 7.19 starting backfill to osd.6 from (0'0,0'0] MAX to 122'10603 2024-09-20T22:06:02.504 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: 7.19 starting backfill to osd.7 from (0'0,0'0] MAX to 122'10603 2024-09-20T22:06:02.504 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: 7.17 starting backfill to osd.5 from (0'0,0'0] MAX to 122'10681 2024-09-20T22:06:02.504 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:02 smithi060 ceph-mon[40290]: 7.14 starting backfill to osd.5 from (0'0,0'0] MAX to 122'10650 2024-09-20T22:06:02.548 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:02.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:02.579 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:02.580 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:02.680 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:02.710 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:02.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:02.711 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:02.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: 4.1 scrub starts 2024-09-20T22:06:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: 4.1 scrub ok 2024-09-20T22:06:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: 7.8 scrub starts 2024-09-20T22:06:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: 7.8 scrub ok 2024-09-20T22:06:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: 6.12 scrub starts 2024-09-20T22:06:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: 6.12 scrub ok 2024-09-20T22:06:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:06:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: 7.a scrub starts 2024-09-20T22:06:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: pgmap v135: 158 pgs: 1 active+clean+scrubbing, 10 unknown, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 23681/213126 objects degraded (11.111%); 2982/213126 objects misplaced (1.399%); 0 B/s, 0 objects/s recovering 2024-09-20T22:06:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: osdmap e125: 8 total, 7 up, 8 in 2024-09-20T22:06:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: 7.16 starting backfill to osd.4 from (0'0,0'0] MAX to 122'10693 2024-09-20T22:06:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: 7.19 starting backfill to osd.2 from (0'0,0'0] MAX to 122'10603 2024-09-20T22:06:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: 7.14 starting backfill to osd.2 from (0'0,0'0] MAX to 122'10650 2024-09-20T22:06:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: 7.17 starting backfill to osd.2 from (0'0,0'0] MAX to 122'10681 2024-09-20T22:06:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: 7.14 starting backfill to osd.3 from (0'0,0'0] MAX to 122'10650 2024-09-20T22:06:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: 7.19 starting backfill to osd.6 from (0'0,0'0] MAX to 122'10603 2024-09-20T22:06:02.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: 7.19 starting backfill to osd.7 from (0'0,0'0] MAX to 122'10603 2024-09-20T22:06:02.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: 7.17 starting backfill to osd.5 from (0'0,0'0] MAX to 122'10681 2024-09-20T22:06:02.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:02 smithi067 ceph-mon[33511]: 7.14 starting backfill to osd.5 from (0'0,0'0] MAX to 122'10650 2024-09-20T22:06:02.812 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:02.857 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:02.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:02.858 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:02.958 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:02.994 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:02.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:02.995 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:03.096 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:03.125 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:03.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:03.126 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:03.227 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:03.258 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:03.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:03.259 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:03.360 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:03.393 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:03.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:03.394 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:03.495 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:03.496 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:06:03.540 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:03.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:03.541 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:03.642 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:03.655 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T22:06:03.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:03.687 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:03.688 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:03.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:03 smithi067 ceph-mon[33511]: 5.a scrub starts 2024-09-20T22:06:03.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:03 smithi067 ceph-mon[33511]: 5.a scrub ok 2024-09-20T22:06:03.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:03 smithi067 ceph-mon[33511]: 7.18 starting backfill to osd.0 from (0'0,0'0] MAX to 122'10861 2024-09-20T22:06:03.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:03 smithi067 ceph-mon[33511]: 7.12 starting backfill to osd.0 from (0'0,0'0] MAX to 122'10633 2024-09-20T22:06:03.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:03 smithi067 ceph-mon[33511]: 7.13 starting backfill to osd.0 from (0'0,0'0] MAX to 122'10620 2024-09-20T22:06:03.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:03 smithi067 ceph-mon[33511]: 7.10 starting backfill to osd.2 from (0'0,0'0] MAX to 122'10697 2024-09-20T22:06:03.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:03 smithi067 ceph-mon[33511]: 7.11 starting backfill to osd.7 from (0'0,0'0] MAX to 122'10894 2024-09-20T22:06:03.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:03 smithi067 ceph-mon[33511]: 7.18 starting backfill to osd.6 from (0'0,0'0] MAX to 122'10861 2024-09-20T22:06:03.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:03 smithi067 ceph-mon[33511]: 7.12 starting backfill to osd.6 from (0'0,0'0] MAX to 122'10633 2024-09-20T22:06:03.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:03 smithi067 ceph-mon[33511]: 7.13 starting backfill to osd.7 from (0'0,0'0] MAX to 122'10620 2024-09-20T22:06:03.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:03 smithi067 ceph-mon[33511]: 7.17 starting backfill to osd.7 from (0'0,0'0] MAX to 122'10681 2024-09-20T22:06:03.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:03 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/500370242' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:06:03.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:03 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:06:03.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[29694]: 5.a scrub starts 2024-09-20T22:06:03.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[29694]: 5.a scrub ok 2024-09-20T22:06:03.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[29694]: 7.18 starting backfill to osd.0 from (0'0,0'0] MAX to 122'10861 2024-09-20T22:06:03.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[29694]: 7.12 starting backfill to osd.0 from (0'0,0'0] MAX to 122'10633 2024-09-20T22:06:03.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[29694]: 7.13 starting backfill to osd.0 from (0'0,0'0] MAX to 122'10620 2024-09-20T22:06:03.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[29694]: 7.10 starting backfill to osd.2 from (0'0,0'0] MAX to 122'10697 2024-09-20T22:06:03.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[29694]: 7.11 starting backfill to osd.7 from (0'0,0'0] MAX to 122'10894 2024-09-20T22:06:03.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[29694]: 7.18 starting backfill to osd.6 from (0'0,0'0] MAX to 122'10861 2024-09-20T22:06:03.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[29694]: 7.12 starting backfill to osd.6 from (0'0,0'0] MAX to 122'10633 2024-09-20T22:06:03.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[29694]: 7.13 starting backfill to osd.7 from (0'0,0'0] MAX to 122'10620 2024-09-20T22:06:03.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[29694]: 7.17 starting backfill to osd.7 from (0'0,0'0] MAX to 122'10681 2024-09-20T22:06:03.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/500370242' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:06:03.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:06:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[40290]: 5.a scrub starts 2024-09-20T22:06:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[40290]: 5.a scrub ok 2024-09-20T22:06:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[40290]: 7.18 starting backfill to osd.0 from (0'0,0'0] MAX to 122'10861 2024-09-20T22:06:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[40290]: 7.12 starting backfill to osd.0 from (0'0,0'0] MAX to 122'10633 2024-09-20T22:06:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[40290]: 7.13 starting backfill to osd.0 from (0'0,0'0] MAX to 122'10620 2024-09-20T22:06:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[40290]: 7.10 starting backfill to osd.2 from (0'0,0'0] MAX to 122'10697 2024-09-20T22:06:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[40290]: 7.11 starting backfill to osd.7 from (0'0,0'0] MAX to 122'10894 2024-09-20T22:06:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[40290]: 7.18 starting backfill to osd.6 from (0'0,0'0] MAX to 122'10861 2024-09-20T22:06:03.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[40290]: 7.12 starting backfill to osd.6 from (0'0,0'0] MAX to 122'10633 2024-09-20T22:06:03.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[40290]: 7.13 starting backfill to osd.7 from (0'0,0'0] MAX to 122'10620 2024-09-20T22:06:03.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[40290]: 7.17 starting backfill to osd.7 from (0'0,0'0] MAX to 122'10681 2024-09-20T22:06:03.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/500370242' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:06:03.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:03 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:06:03.788 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:03.819 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:03.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:03.821 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:03.852 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3, 4, 5, 6, 7] 2024-09-20T22:06:03.852 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:06:03.852 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:06:03.906 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:06:03.921 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:03.954 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:03.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:03.956 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:04.026 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:06:04.057 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:04.095 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:04.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:04.096 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:04.197 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:04.225 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:04.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:04.226 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:04.326 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:04.371 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:04.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:04.372 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:04.473 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:04.513 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:04.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:04.515 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:04.616 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:04.632 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:06:04.633 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:06:04.653 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:04.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:04.654 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:04.754 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:04.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:04 smithi067 ceph-mon[33511]: 7.a scrub ok 2024-09-20T22:06:04.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:04 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:06:04.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:04 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:06:04.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:04 smithi067 ceph-mon[33511]: osdmap e126: 8 total, 7 up, 8 in 2024-09-20T22:06:04.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:04 smithi067 ceph-mon[33511]: pgmap v138: 158 pgs: 1 active+clean+scrubbing, 10 unknown, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 23681/213126 objects degraded (11.111%); 2982/213126 objects misplaced (1.399%); 0 B/s, 0 objects/s recovering 2024-09-20T22:06:04.784 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:04.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:04.785 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:04.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:04 smithi060 ceph-mon[29694]: 7.a scrub ok 2024-09-20T22:06:04.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:04 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:06:04.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:04 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:06:04.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:04 smithi060 ceph-mon[29694]: osdmap e126: 8 total, 7 up, 8 in 2024-09-20T22:06:04.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:04 smithi060 ceph-mon[29694]: pgmap v138: 158 pgs: 1 active+clean+scrubbing, 10 unknown, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 23681/213126 objects degraded (11.111%); 2982/213126 objects misplaced (1.399%); 0 B/s, 0 objects/s recovering 2024-09-20T22:06:04.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:04 smithi060 ceph-mon[40290]: 7.a scrub ok 2024-09-20T22:06:04.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:04 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:06:04.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:04 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:06:04.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:04 smithi060 ceph-mon[40290]: osdmap e126: 8 total, 7 up, 8 in 2024-09-20T22:06:04.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:04 smithi060 ceph-mon[40290]: pgmap v138: 158 pgs: 1 active+clean+scrubbing, 10 unknown, 8 active+clean+remapped, 26 active+undersized, 32 active+undersized+degraded, 81 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 23681/213126 objects degraded (11.111%); 2982/213126 objects misplaced (1.399%); 0 B/s, 0 objects/s recovering 2024-09-20T22:06:04.885 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:04.907 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:04.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:04.908 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:05.008 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:05.051 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:05.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:05.052 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:05.153 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:05.186 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:05.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:05.187 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:05.288 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:05.321 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:05.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:05.322 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:05.406 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T22:06:05.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:05.469 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:05.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:05.471 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:05.512 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:06:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:06:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:06:05.572 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:05.613 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:05.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:05.614 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:05.715 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:05.760 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:05.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:05.761 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:05.787 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:06:05.862 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:05.889 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:05.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:05.890 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:05.976 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:05.988 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:05.990 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:06.059 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:06.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:06.060 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:06.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:06.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:05 smithi060 ceph-mon[29694]: 5.e scrub starts 2024-09-20T22:06:06.124 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:06:06.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:05 smithi060 ceph-mon[29694]: 5.e scrub ok 2024-09-20T22:06:06.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:05 smithi060 ceph-mon[29694]: 3.1c scrub starts 2024-09-20T22:06:06.125 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:05 smithi060 ceph-mon[29694]: 3.1c scrub ok 2024-09-20T22:06:06.125 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:06.125 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:05 smithi060 ceph-mon[40290]: 5.e scrub starts 2024-09-20T22:06:06.125 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:06:06.125 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:05 smithi060 ceph-mon[40290]: 5.e scrub ok 2024-09-20T22:06:06.125 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:05 smithi060 ceph-mon[40290]: 3.1c scrub starts 2024-09-20T22:06:06.125 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:05 smithi060 ceph-mon[40290]: 3.1c scrub ok 2024-09-20T22:06:06.131 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:06.131 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:05 smithi067 ceph-mon[33511]: 5.e scrub starts 2024-09-20T22:06:06.131 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:06:06.131 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:05 smithi067 ceph-mon[33511]: 5.e scrub ok 2024-09-20T22:06:06.132 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:05 smithi067 ceph-mon[33511]: 3.1c scrub starts 2024-09-20T22:06:06.132 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:05 smithi067 ceph-mon[33511]: 3.1c scrub ok 2024-09-20T22:06:06.161 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:06.193 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:06.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:06.194 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:06.294 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:06.330 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:06.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:06.331 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:06.431 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:06.434 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:06:06.434 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:06:06.466 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:06.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:06.467 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:06.568 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:06.598 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:06.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:06.599 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:06.699 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:06.742 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:06.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:06.743 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:06.844 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:06.886 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:06.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:06.888 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:06.892 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:06:06.892 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:06:06.896 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:06:06.941 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:06:06.941 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:06 smithi060 ceph-mon[29694]: 5.1 scrub starts 2024-09-20T22:06:06.941 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:06 smithi060 ceph-mon[29694]: 5.1 scrub ok 2024-09-20T22:06:06.941 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:06 smithi060 ceph-mon[29694]: pgmap v139: 158 pgs: 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 1 active+undersized+degraded+remapped+backfilling, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 5.5 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s; 33681/272109 objects degraded (12.378%); 46808/272109 objects misplaced (17.202%); 13 MiB/s, 206 objects/s recovering 2024-09-20T22:06:06.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:06 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 23681/213126 objects degraded (11.111%), 32 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:06.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:06 smithi060 ceph-mon[29694]: osdmap e127: 8 total, 7 up, 8 in 2024-09-20T22:06:06.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:06 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2812707501' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:06.942 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:06 smithi060 ceph-mon[40290]: 5.1 scrub starts 2024-09-20T22:06:06.942 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:06 smithi060 ceph-mon[40290]: 5.1 scrub ok 2024-09-20T22:06:06.942 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:06 smithi060 ceph-mon[40290]: pgmap v139: 158 pgs: 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 1 active+undersized+degraded+remapped+backfilling, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 5.5 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s; 33681/272109 objects degraded (12.378%); 46808/272109 objects misplaced (17.202%); 13 MiB/s, 206 objects/s recovering 2024-09-20T22:06:06.942 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:06 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 23681/213126 objects degraded (11.111%), 32 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:06.942 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:06 smithi060 ceph-mon[40290]: osdmap e127: 8 total, 7 up, 8 in 2024-09-20T22:06:06.942 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:06 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2812707501' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:06.943 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:06:06.989 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:06.992 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:06:06.992 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:06:07.035 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:07.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:07.036 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:07.037 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:06:07.037 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:06:07.072 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:06:07.072 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:06:07.118 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:06:07.119 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:06:07.137 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:07.140 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:06 smithi067 ceph-mon[33511]: 5.1 scrub starts 2024-09-20T22:06:07.140 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:06 smithi067 ceph-mon[33511]: 5.1 scrub ok 2024-09-20T22:06:07.140 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:06 smithi067 ceph-mon[33511]: pgmap v139: 158 pgs: 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 1 active+undersized+degraded+remapped+backfilling, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 5.5 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s; 33681/272109 objects degraded (12.378%); 46808/272109 objects misplaced (17.202%); 13 MiB/s, 206 objects/s recovering 2024-09-20T22:06:07.140 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:06 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 23681/213126 objects degraded (11.111%), 32 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:07.140 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:06 smithi067 ceph-mon[33511]: osdmap e127: 8 total, 7 up, 8 in 2024-09-20T22:06:07.140 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:06 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2812707501' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:07.158 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:06:07.158 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:06:07.173 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:07.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:07.174 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:07.190 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T22:06:07.275 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:07.312 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:07.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:07.313 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:07.414 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:07.444 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:07.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:07.445 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:07.463 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:06:07.465 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:06:07.546 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:07.587 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:07.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:07.588 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:07.612 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:06:07.688 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:07.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:07.727 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:07.728 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:07.828 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:07.865 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:07.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:07.866 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:07.889 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:06:07.935 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:06:07.935 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:06:07.967 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:08.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:07 smithi067 ceph-mon[33511]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:06:08.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:07 smithi067 ceph-mon[33511]: 4.15 scrub starts 2024-09-20T22:06:08.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:07 smithi067 ceph-mon[33511]: 4.15 scrub ok 2024-09-20T22:06:08.023 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:08.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:08.024 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:08.125 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:08.156 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:08.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:08.157 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:08.216 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:06:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:06:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:06:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:06:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:23759 7.0 7:09825cc1:::benchmark_data_smithi060_78800_object23758:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:06:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:06.790275+0000", 2024-09-20T22:06:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 121.42045744399999, 2024-09-20T22:06:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.073367153000000004, 2024-09-20T22:06:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.216 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 23759 2024-09-20T22:06:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.790275+0000", 2024-09-20T22:06:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.217 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.790275+0000", 2024-09-20T22:06:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.790277+0000", 2024-09-20T22:06:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6500000000000001e-06 2024-09-20T22:06:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.790333+0000", 2024-09-20T22:06:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6239000000000001e-05 2024-09-20T22:06:08.218 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.790347+0000", 2024-09-20T22:06:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4374e-05 2024-09-20T22:06:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.790355+0000", 2024-09-20T22:06:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4109999999999999e-06 2024-09-20T22:06:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.219 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.790408+0000", 2024-09-20T22:06:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2880999999999997e-05 2024-09-20T22:06:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.790489+0000", 2024-09-20T22:06:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1546999999999997e-05 2024-09-20T22:06:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,3", 2024-09-20T22:06:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.790575+0000", 2024-09-20T22:06:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5933000000000005e-05 2024-09-20T22:06:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:08.220 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.791345+0000", 2024-09-20T22:06:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00076988000000000002 2024-09-20T22:06:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.791826+0000", 2024-09-20T22:06:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00048114799999999998 2024-09-20T22:06:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.863523+0000", 2024-09-20T22:06:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.071696704 2024-09-20T22:06:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.221 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.863586+0000", 2024-09-20T22:06:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2842000000000006e-05 2024-09-20T22:06:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.863642+0000", 2024-09-20T22:06:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6543999999999999e-05 2024-09-20T22:06:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.222 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:46027 7.0 7:0265a40b:::benchmark_data_smithi060_78800_object46026:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T22:06:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:19.389108+0000", 2024-09-20T22:06:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 108.821625059, 2024-09-20T22:06:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.079311146999999999, 2024-09-20T22:06:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 46027 2024-09-20T22:06:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.223 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389108+0000", 2024-09-20T22:06:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389108+0000", 2024-09-20T22:06:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389109+0000", 2024-09-20T22:06:08.224 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1289999999999999e-06 2024-09-20T22:06:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389169+0000", 2024-09-20T22:06:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0485000000000001e-05 2024-09-20T22:06:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389177+0000", 2024-09-20T22:06:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.089e-06 2024-09-20T22:06:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389182+0000", 2024-09-20T22:06:08.225 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9749999999999998e-06 2024-09-20T22:06:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389275+0000", 2024-09-20T22:06:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2967000000000006e-05 2024-09-20T22:06:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389390+0000", 2024-09-20T22:06:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000115148 2024-09-20T22:06:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:06:08.226 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389524+0000", 2024-09-20T22:06:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013312900000000001 2024-09-20T22:06:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.391831+0000", 2024-09-20T22:06:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0023078769999999998 2024-09-20T22:06:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.468296+0000", 2024-09-20T22:06:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.076464536999999999 2024-09-20T22:06:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.227 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.468337+0000", 2024-09-20T22:06:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1102000000000001e-05 2024-09-20T22:06:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.468419+0000", 2024-09-20T22:06:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1709e-05 2024-09-20T22:06:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:46028 7.0 7:04febef9:::benchmark_data_smithi060_78800_object46027:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T22:06:08.228 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:19.389375+0000", 2024-09-20T22:06:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 108.82135732899999, 2024-09-20T22:06:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.079186260999999994, 2024-09-20T22:06:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 46028 2024-09-20T22:06:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.229 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389375+0000", 2024-09-20T22:06:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389375+0000", 2024-09-20T22:06:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389376+0000", 2024-09-20T22:06:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8000000000000004e-07 2024-09-20T22:06:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.230 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389437+0000", 2024-09-20T22:06:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0291999999999998e-05 2024-09-20T22:06:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389446+0000", 2024-09-20T22:06:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8130000000000007e-06 2024-09-20T22:06:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389453+0000", 2024-09-20T22:06:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0469999999999996e-06 2024-09-20T22:06:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.231 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.390649+0000", 2024-09-20T22:06:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001195392 2024-09-20T22:06:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.390768+0000", 2024-09-20T22:06:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000118863 2024-09-20T22:06:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:06:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.390902+0000", 2024-09-20T22:06:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013449100000000001 2024-09-20T22:06:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.391959+0000", 2024-09-20T22:06:08.232 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001056415 2024-09-20T22:06:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.468456+0000", 2024-09-20T22:06:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.076497323000000006 2024-09-20T22:06:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.468490+0000", 2024-09-20T22:06:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.3887000000000003e-05 2024-09-20T22:06:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.233 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.468562+0000", 2024-09-20T22:06:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1858000000000002e-05 2024-09-20T22:06:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:97617 7.0 7:094cbc91:::benchmark_data_smithi060_78800_object97616:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-20T22:06:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:55.961747+0000", 2024-09-20T22:06:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 72.248985239999996, 2024-09-20T22:06:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.069398526000000002, 2024-09-20T22:06:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.234 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 97617 2024-09-20T22:06:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.961747+0000", 2024-09-20T22:06:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.961747+0000", 2024-09-20T22:06:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.961749+0000", 2024-09-20T22:06:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.085e-06 2024-09-20T22:06:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.961795+0000", 2024-09-20T22:06:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6579000000000002e-05 2024-09-20T22:06:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.961806+0000", 2024-09-20T22:06:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1099e-05 2024-09-20T22:06:08.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.961811+0000", 2024-09-20T22:06:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0760000000000002e-06 2024-09-20T22:06:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.961866+0000", 2024-09-20T22:06:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4364000000000003e-05 2024-09-20T22:06:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.961980+0000", 2024-09-20T22:06:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000114681 2024-09-20T22:06:08.237 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:06:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.962120+0000", 2024-09-20T22:06:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000139963 2024-09-20T22:06:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.030907+0000", 2024-09-20T22:06:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.068786981999999997 2024-09-20T22:06:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.030970+0000", 2024-09-20T22:06:08.238 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2273999999999994e-05 2024-09-20T22:06:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.031036+0000", 2024-09-20T22:06:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6243999999999998e-05 2024-09-20T22:06:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.031146+0000", 2024-09-20T22:06:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000110179 2024-09-20T22:06:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:97619 7.0 7:076260e1:::benchmark_data_smithi060_78800_object97618:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-20T22:06:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:55.963183+0000", 2024-09-20T22:06:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 72.247550122000007, 2024-09-20T22:06:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.068230031999999996, 2024-09-20T22:06:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 97619 2024-09-20T22:06:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.240 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.963183+0000", 2024-09-20T22:06:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.963183+0000", 2024-09-20T22:06:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.963183+0000", 2024-09-20T22:06:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6799999999999999e-07 2024-09-20T22:06:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.241 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.963247+0000", 2024-09-20T22:06:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3689999999999995e-05 2024-09-20T22:06:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.963256+0000", 2024-09-20T22:06:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3479999999999993e-06 2024-09-20T22:06:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.242 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.963261+0000", 2024-09-20T22:06:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9830000000000004e-06 2024-09-20T22:06:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.243 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.964614+0000", 2024-09-20T22:06:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00135249 2024-09-20T22:06:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.964706+0000", 2024-09-20T22:06:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2381999999999997e-05 2024-09-20T22:06:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:06:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.964789+0000", 2024-09-20T22:06:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3253999999999999e-05 2024-09-20T22:06:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.244 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.031225+0000", 2024-09-20T22:06:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.066435055000000007 2024-09-20T22:06:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.031246+0000", 2024-09-20T22:06:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1645e-05 2024-09-20T22:06:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.031309+0000", 2024-09-20T22:06:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3338999999999998e-05 2024-09-20T22:06:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.031413+0000", 2024-09-20T22:06:08.245 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000103078 2024-09-20T22:06:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:97624 7.0 7:00ae6b61:::benchmark_data_smithi060_78800_object97623:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-20T22:06:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:55.964064+0000", 2024-09-20T22:06:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 72.246668850999995, 2024-09-20T22:06:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.081791942000000006, 2024-09-20T22:06:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.246 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 97624 2024-09-20T22:06:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.964064+0000", 2024-09-20T22:06:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.964064+0000", 2024-09-20T22:06:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.247 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.964064+0000", 2024-09-20T22:06:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8800000000000002e-07 2024-09-20T22:06:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.964102+0000", 2024-09-20T22:06:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7379000000000002e-05 2024-09-20T22:06:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.248 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.964109+0000", 2024-09-20T22:06:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4719999999999998e-06 2024-09-20T22:06:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.964115+0000", 2024-09-20T22:06:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7270000000000004e-06 2024-09-20T22:06:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.027873+0000", 2024-09-20T22:06:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.063758327000000004 2024-09-20T22:06:08.249 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.027995+0000", 2024-09-20T22:06:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000121808 2024-09-20T22:06:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:06:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.028169+0000", 2024-09-20T22:06:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017430500000000001 2024-09-20T22:06:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.034984+0000", 2024-09-20T22:06:08.250 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0068145080000000004 2024-09-20T22:06:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.045758+0000", 2024-09-20T22:06:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.010773731 2024-09-20T22:06:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.045800+0000", 2024-09-20T22:06:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2626000000000002e-05 2024-09-20T22:06:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.251 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.045856+0000", 2024-09-20T22:06:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5470999999999998e-05 2024-09-20T22:06:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:97634 7.0 7:02fe1766:::benchmark_data_smithi060_78800_object97633:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-20T22:06:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:55.975497+0000", 2024-09-20T22:06:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 72.235235708000005, 2024-09-20T22:06:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.070573945999999999, 2024-09-20T22:06:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:08.252 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 97634 2024-09-20T22:06:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.975497+0000", 2024-09-20T22:06:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.975497+0000", 2024-09-20T22:06:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.975499+0000", 2024-09-20T22:06:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6139999999999999e-06 2024-09-20T22:06:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.975560+0000", 2024-09-20T22:06:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1314000000000003e-05 2024-09-20T22:06:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.024807+0000", 2024-09-20T22:06:08.254 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.049247434999999999 2024-09-20T22:06:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.024824+0000", 2024-09-20T22:06:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7059999999999999e-05 2024-09-20T22:06:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.031664+0000", 2024-09-20T22:06:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0068397270000000003 2024-09-20T22:06:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.031789+0000", 2024-09-20T22:06:08.255 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012484 2024-09-20T22:06:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:06:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.031946+0000", 2024-09-20T22:06:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015695600000000001 2024-09-20T22:06:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.037968+0000", 2024-09-20T22:06:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0060224099999999997 2024-09-20T22:06:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.045903+0000", 2024-09-20T22:06:08.256 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0079344029999999996 2024-09-20T22:06:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.045966+0000", 2024-09-20T22:06:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2849999999999996e-05 2024-09-20T22:06:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.046071+0000", 2024-09-20T22:06:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000105337 2024-09-20T22:06:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.257 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:98062 7.b 7:d93ed38f:::benchmark_data_smithi060_78800_object98061:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-20T22:06:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.322852+0000", 2024-09-20T22:06:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 71.887880959, 2024-09-20T22:06:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.084887676999999995, 2024-09-20T22:06:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 98062 2024-09-20T22:06:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.258 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322852+0000", 2024-09-20T22:06:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322852+0000", 2024-09-20T22:06:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322853+0000", 2024-09-20T22:06:08.259 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.376e-06 2024-09-20T22:06:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322895+0000", 2024-09-20T22:06:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1971999999999997e-05 2024-09-20T22:06:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322907+0000", 2024-09-20T22:06:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1758e-05 2024-09-20T22:06:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322914+0000", 2024-09-20T22:06:08.260 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1840000000000002e-06 2024-09-20T22:06:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322965+0000", 2024-09-20T22:06:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0584000000000002e-05 2024-09-20T22:06:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.323058+0000", 2024-09-20T22:06:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3249000000000001e-05 2024-09-20T22:06:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.261 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6", 2024-09-20T22:06:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.323142+0000", 2024-09-20T22:06:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4429000000000001e-05 2024-09-20T22:06:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.325301+0000", 2024-09-20T22:06:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0021587440000000002 2024-09-20T22:06:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.407511+0000", 2024-09-20T22:06:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.082209656000000006 2024-09-20T22:06:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.262 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.407630+0000", 2024-09-20T22:06:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011909900000000001 2024-09-20T22:06:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.407739+0000", 2024-09-20T22:06:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000109626 2024-09-20T22:06:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:98529 7.b 7:d59e89dd:::benchmark_data_smithi060_78800_object98528:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-20T22:06:08.263 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.760245+0000", 2024-09-20T22:06:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 71.450487902999996, 2024-09-20T22:06:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.077949966999999995, 2024-09-20T22:06:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 98529 2024-09-20T22:06:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.264 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760245+0000", 2024-09-20T22:06:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760245+0000", 2024-09-20T22:06:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760246+0000", 2024-09-20T22:06:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.062e-06 2024-09-20T22:06:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.265 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760304+0000", 2024-09-20T22:06:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7834999999999998e-05 2024-09-20T22:06:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760313+0000", 2024-09-20T22:06:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1819999999999995e-06 2024-09-20T22:06:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760318+0000", 2024-09-20T22:06:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0239999999999997e-06 2024-09-20T22:06:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.266 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760361+0000", 2024-09-20T22:06:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3383000000000001e-05 2024-09-20T22:06:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760464+0000", 2024-09-20T22:06:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010275700000000001 2024-09-20T22:06:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6", 2024-09-20T22:06:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760550+0000", 2024-09-20T22:06:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6180000000000005e-05 2024-09-20T22:06:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.267 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.762778+0000", 2024-09-20T22:06:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0022282690000000002 2024-09-20T22:06:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.838033+0000", 2024-09-20T22:06:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.075254673999999994 2024-09-20T22:06:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.838095+0000", 2024-09-20T22:06:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2068000000000003e-05 2024-09-20T22:06:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.268 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.838195+0000", 2024-09-20T22:06:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9532999999999997e-05 2024-09-20T22:06:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:99611 7.b 7:d551c118:::benchmark_data_smithi060_78800_object99610:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-20T22:06:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:57.750045+0000", 2024-09-20T22:06:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 70.460687387999997, 2024-09-20T22:06:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.076272598999999996, 2024-09-20T22:06:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.269 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 99611 2024-09-20T22:06:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750045+0000", 2024-09-20T22:06:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.270 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750045+0000", 2024-09-20T22:06:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750047+0000", 2024-09-20T22:06:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6419999999999999e-06 2024-09-20T22:06:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750097+0000", 2024-09-20T22:06:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0220999999999999e-05 2024-09-20T22:06:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.271 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750110+0000", 2024-09-20T22:06:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2577e-05 2024-09-20T22:06:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750117+0000", 2024-09-20T22:06:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5939999999999996e-06 2024-09-20T22:06:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750172+0000", 2024-09-20T22:06:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4586999999999998e-05 2024-09-20T22:06:08.272 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750253+0000", 2024-09-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0924e-05 2024-09-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6", 2024-09-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750350+0000", 2024-09-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6871000000000001e-05 2024-09-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.755920+0000", 2024-09-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0055701470000000001 2024-09-20T22:06:08.273 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.826093+0000", 2024-09-20T22:06:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.070173630000000001 2024-09-20T22:06:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.826212+0000", 2024-09-20T22:06:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011807 2024-09-20T22:06:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.826318+0000", 2024-09-20T22:06:08.274 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010633600000000001 2024-09-20T22:06:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:99613 7.b 7:d5b26ca4:::benchmark_data_smithi060_78800_object99612:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-20T22:06:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:57.750430+0000", 2024-09-20T22:06:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 70.460302620999997, 2024-09-20T22:06:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.077303389, 2024-09-20T22:06:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:08.275 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 99613 2024-09-20T22:06:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750430+0000", 2024-09-20T22:06:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750430+0000", 2024-09-20T22:06:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.276 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750431+0000", 2024-09-20T22:06:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.64e-07 2024-09-20T22:06:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750527+0000", 2024-09-20T22:06:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6439000000000003e-05 2024-09-20T22:06:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750539+0000", 2024-09-20T22:06:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1778e-05 2024-09-20T22:06:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.277 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750547+0000", 2024-09-20T22:06:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3629999999999996e-06 2024-09-20T22:06:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.753106+0000", 2024-09-20T22:06:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0025595510000000002 2024-09-20T22:06:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.753223+0000", 2024-09-20T22:06:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000116927 2024-09-20T22:06:08.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6", 2024-09-20T22:06:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.753368+0000", 2024-09-20T22:06:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014529199999999999 2024-09-20T22:06:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.756019+0000", 2024-09-20T22:06:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0026511180000000001 2024-09-20T22:06:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.827375+0000", 2024-09-20T22:06:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.071355298999999997 2024-09-20T22:06:08.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.827482+0000", 2024-09-20T22:06:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000106963 2024-09-20T22:06:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.827733+0000", 2024-09-20T22:06:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025169500000000002 2024-09-20T22:06:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.280 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:101399 7.b 7:dad621f3:::benchmark_data_smithi060_78800_object101398:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-20T22:06:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:59.478423+0000", 2024-09-20T22:06:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 68.732309404999995, 2024-09-20T22:06:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.079270373000000005, 2024-09-20T22:06:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 101399 2024-09-20T22:06:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.281 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.478423+0000", 2024-09-20T22:06:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.478423+0000", 2024-09-20T22:06:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.478425+0000", 2024-09-20T22:06:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.341e-06 2024-09-20T22:06:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.282 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.283 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:08.284 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:07 smithi060 ceph-mon[29694]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:06:08.284 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:07 smithi060 ceph-mon[29694]: 4.15 scrub starts 2024-09-20T22:06:08.284 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:07 smithi060 ceph-mon[29694]: 4.15 scrub ok 2024-09-20T22:06:08.284 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:07 smithi060 ceph-mon[40290]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:06:08.284 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:07 smithi060 ceph-mon[40290]: 4.15 scrub starts 2024-09-20T22:06:08.284 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:07 smithi060 ceph-mon[40290]: 4.15 scrub ok 2024-09-20T22:06:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.478475+0000", 2024-09-20T22:06:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0312999999999998e-05 2024-09-20T22:06:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.478486+0000", 2024-09-20T22:06:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0813e-05 2024-09-20T22:06:08.285 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.478491+0000", 2024-09-20T22:06:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5130000000000004e-06 2024-09-20T22:06:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.478531+0000", 2024-09-20T22:06:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9489000000000001e-05 2024-09-20T22:06:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.478621+0000", 2024-09-20T22:06:08.286 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9827999999999994e-05 2024-09-20T22:06:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6", 2024-09-20T22:06:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.478707+0000", 2024-09-20T22:06:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6021999999999997e-05 2024-09-20T22:06:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.557429+0000", 2024-09-20T22:06:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.078721970000000002 2024-09-20T22:06:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:08.287 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.557513+0000", 2024-09-20T22:06:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4500999999999996e-05 2024-09-20T22:06:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.557576+0000", 2024-09-20T22:06:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2600000000000004e-05 2024-09-20T22:06:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.557694+0000", 2024-09-20T22:06:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000117983 2024-09-20T22:06:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:101402 7.0 7:025f111c:::benchmark_data_smithi060_78800_object101401:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-20T22:06:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:59.479446+0000", 2024-09-20T22:06:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 68.731286515999997, 2024-09-20T22:06:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.079768565, 2024-09-20T22:06:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 101402 2024-09-20T22:06:08.289 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479446+0000", 2024-09-20T22:06:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479446+0000", 2024-09-20T22:06:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.290 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479447+0000", 2024-09-20T22:06:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.68e-07 2024-09-20T22:06:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479505+0000", 2024-09-20T22:06:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8579999999999998e-05 2024-09-20T22:06:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479514+0000", 2024-09-20T22:06:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1920000000000005e-06 2024-09-20T22:06:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.291 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479518+0000", 2024-09-20T22:06:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6569999999999996e-06 2024-09-20T22:06:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479565+0000", 2024-09-20T22:06:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6820999999999998e-05 2024-09-20T22:06:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479642+0000", 2024-09-20T22:06:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6768999999999996e-05 2024-09-20T22:06:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:06:08.292 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479726+0000", 2024-09-20T22:06:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3759999999999998e-05 2024-09-20T22:06:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559070+0000", 2024-09-20T22:06:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.079344425999999996 2024-09-20T22:06:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559081+0000", 2024-09-20T22:06:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1223000000000001e-05 2024-09-20T22:06:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559129+0000", 2024-09-20T22:06:08.293 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.778e-05 2024-09-20T22:06:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559215+0000", 2024-09-20T22:06:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5688999999999999e-05 2024-09-20T22:06:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:101403 7.b 7:dcd99ced:::benchmark_data_smithi060_78800_object101402:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-20T22:06:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:59.479831+0000", 2024-09-20T22:06:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 68.730901584999998, 2024-09-20T22:06:08.294 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.078021717000000004, 2024-09-20T22:06:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 101403 2024-09-20T22:06:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479831+0000", 2024-09-20T22:06:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.295 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479831+0000", 2024-09-20T22:06:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479832+0000", 2024-09-20T22:06:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.7400000000000002e-07 2024-09-20T22:06:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479896+0000", 2024-09-20T22:06:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4144000000000001e-05 2024-09-20T22:06:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.296 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479903+0000", 2024-09-20T22:06:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4629999999999998e-06 2024-09-20T22:06:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479906+0000", 2024-09-20T22:06:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7900000000000001e-06 2024-09-20T22:06:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480804+0000", 2024-09-20T22:06:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00089766900000000003 2024-09-20T22:06:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.297 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480913+0000", 2024-09-20T22:06:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000109407 2024-09-20T22:06:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6", 2024-09-20T22:06:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.481053+0000", 2024-09-20T22:06:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013912000000000001 2024-09-20T22:06:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.557496+0000", 2024-09-20T22:06:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.076443120000000003 2024-09-20T22:06:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.298 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.557701+0000", 2024-09-20T22:06:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000205479 2024-09-20T22:06:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.557753+0000", 2024-09-20T22:06:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1357999999999999e-05 2024-09-20T22:06:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.557853+0000", 2024-09-20T22:06:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100293 2024-09-20T22:06:08.299 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:101405 7.b 7:d7e4e064:::benchmark_data_smithi060_78800_object101404:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-20T22:06:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:59.480104+0000", 2024-09-20T22:06:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 68.730628760000002, 2024-09-20T22:06:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.081978562000000005, 2024-09-20T22:06:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:08.300 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 101405 2024-09-20T22:06:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480104+0000", 2024-09-20T22:06:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480104+0000", 2024-09-20T22:06:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.301 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480104+0000", 2024-09-20T22:06:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1200000000000003e-07 2024-09-20T22:06:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480188+0000", 2024-09-20T22:06:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3659999999999995e-05 2024-09-20T22:06:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480194+0000", 2024-09-20T22:06:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.66e-06 2024-09-20T22:06:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.302 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480197+0000", 2024-09-20T22:06:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.478e-06 2024-09-20T22:06:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554223+0000", 2024-09-20T22:06:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.074025653999999996 2024-09-20T22:06:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554363+0000", 2024-09-20T22:06:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000139727 2024-09-20T22:06:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.303 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6", 2024-09-20T22:06:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554528+0000", 2024-09-20T22:06:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000165129 2024-09-20T22:06:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.557863+0000", 2024-09-20T22:06:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0033354779999999998 2024-09-20T22:06:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.561924+0000", 2024-09-20T22:06:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0040608839999999998 2024-09-20T22:06:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.304 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.561987+0000", 2024-09-20T22:06:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2547999999999998e-05 2024-09-20T22:06:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.562082+0000", 2024-09-20T22:06:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5832e-05 2024-09-20T22:06:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:101407 7.0 7:04db00de:::benchmark_data_smithi060_78800_object101406:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-20T22:06:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:59.480934+0000", 2024-09-20T22:06:08.305 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 68.729798977000002, 2024-09-20T22:06:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.078444652000000004, 2024-09-20T22:06:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 101407 2024-09-20T22:06:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480934+0000", 2024-09-20T22:06:08.306 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480934+0000", 2024-09-20T22:06:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480934+0000", 2024-09-20T22:06:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7000000000000004e-07 2024-09-20T22:06:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.307 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480993+0000", 2024-09-20T22:06:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8697999999999999e-05 2024-09-20T22:06:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.481001+0000", 2024-09-20T22:06:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0399999999999993e-06 2024-09-20T22:06:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.481005+0000", 2024-09-20T22:06:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2189999999999998e-06 2024-09-20T22:06:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.308 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.482346+0000", 2024-09-20T22:06:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0013407670000000001 2024-09-20T22:06:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.482463+0000", 2024-09-20T22:06:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011674400000000001 2024-09-20T22:06:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:06:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554159+0000", 2024-09-20T22:06:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.071695851000000005 2024-09-20T22:06:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.309 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559254+0000", 2024-09-20T22:06:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0050953149999999996 2024-09-20T22:06:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559265+0000", 2024-09-20T22:06:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0514e-05 2024-09-20T22:06:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559307+0000", 2024-09-20T22:06:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2038e-05 2024-09-20T22:06:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.310 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559378+0000", 2024-09-20T22:06:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1796000000000002e-05 2024-09-20T22:06:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop_reply(client.24667.0:101402 7.0 e107/91 ondisk, result = 0)", 2024-09-20T22:06:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:59.483798+0000", 2024-09-20T22:06:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 68.726935011999998, 2024-09-20T22:06:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.075419815000000001, 2024-09-20T22:06:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.311 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:06:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.483798+0000", 2024-09-20T22:06:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.483798+0000", 2024-09-20T22:06:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.312 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.483799+0000", 2024-09-20T22:06:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4099999999999997e-07 2024-09-20T22:06:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.483807+0000", 2024-09-20T22:06:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3529999999999995e-06 2024-09-20T22:06:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554126+0000", 2024-09-20T22:06:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.070319507000000003 2024-09-20T22:06:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.313 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554150+0000", 2024-09-20T22:06:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.3555999999999999e-05 2024-09-20T22:06:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559057+0000", 2024-09-20T22:06:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0049072120000000002 2024-09-20T22:06:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559064+0000", 2024-09-20T22:06:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9489999999999999e-06 2024-09-20T22:06:08.314 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559217+0000", 2024-09-20T22:06:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015329699999999999 2024-09-20T22:06:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.24667.0:101413 7.e e107/80 7:7bfa84cf:::benchmark_data_smithi060_78800_object101412:head v 107'6308, mlcod=107'6306)", 2024-09-20T22:06:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:59.487069+0000", 2024-09-20T22:06:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 68.723663267999996, 2024-09-20T22:06:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.090496713000000006, 2024-09-20T22:06:08.315 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.487069+0000", 2024-09-20T22:06:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.487069+0000", 2024-09-20T22:06:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.316 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.487070+0000", 2024-09-20T22:06:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9599999999999998e-07 2024-09-20T22:06:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.487131+0000", 2024-09-20T22:06:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0526000000000003e-05 2024-09-20T22:06:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554190+0000", 2024-09-20T22:06:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.067059399000000006 2024-09-20T22:06:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.317 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554197+0000", 2024-09-20T22:06:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0010000000000004e-06 2024-09-20T22:06:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554563+0000", 2024-09-20T22:06:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000365483 2024-09-20T22:06:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554581+0000", 2024-09-20T22:06:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8481000000000001e-05 2024-09-20T22:06:08.318 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.577514+0000", 2024-09-20T22:06:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.022933044 2024-09-20T22:06:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.577566+0000", 2024-09-20T22:06:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1983e-05 2024-09-20T22:06:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.319 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:101418 7.0 7:0f8fb5c9:::benchmark_data_smithi060_78800_object101417:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-20T22:06:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:59.491096+0000", 2024-09-20T22:06:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 68.719636707999996, 2024-09-20T22:06:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.071683293999999995, 2024-09-20T22:06:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 101418 2024-09-20T22:06:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.320 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.491096+0000", 2024-09-20T22:06:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.491096+0000", 2024-09-20T22:06:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.491097+0000", 2024-09-20T22:06:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3120000000000001e-06 2024-09-20T22:06:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.321 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.491153+0000", 2024-09-20T22:06:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5287000000000001e-05 2024-09-20T22:06:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554969+0000", 2024-09-20T22:06:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.063816169000000006 2024-09-20T22:06:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554979+0000", 2024-09-20T22:06:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0747999999999999e-05 2024-09-20T22:06:08.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559418+0000", 2024-09-20T22:06:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00443875 2024-09-20T22:06:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559554+0000", 2024-09-20T22:06:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000135723 2024-09-20T22:06:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:06:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559690+0000", 2024-09-20T22:06:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013608700000000001 2024-09-20T22:06:08.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.562518+0000", 2024-09-20T22:06:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028279249999999998 2024-09-20T22:06:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.562632+0000", 2024-09-20T22:06:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011374000000000001 2024-09-20T22:06:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.562685+0000", 2024-09-20T22:06:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3569000000000002e-05 2024-09-20T22:06:08.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.562779+0000", 2024-09-20T22:06:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3984e-05 2024-09-20T22:06:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169974 7.10 7:0a082cf8:::benchmark_data_smithi060_78800_object36941:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:06:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:06:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 9.5861188689999999, 2024-09-20T22:06:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87805208899999998, 2024-09-20T22:06:08.325 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169974 2024-09-20T22:06:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:06:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.326 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:06:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624616+0000", 2024-09-20T22:06:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1739999999999999e-06 2024-09-20T22:06:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624626+0000", 2024-09-20T22:06:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7909999999999999e-06 2024-09-20T22:06:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.327 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624654+0000", 2024-09-20T22:06:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.8524999999999999e-05 2024-09-20T22:06:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624665+0000", 2024-09-20T22:06:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0985000000000001e-05 2024-09-20T22:06:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.705237+0000", 2024-09-20T22:06:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.080571485999999998 2024-09-20T22:06:08.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:06:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.705245+0000", 2024-09-20T22:06:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8460000000000008e-06 2024-09-20T22:06:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498359+0000", 2024-09-20T22:06:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79311461999999999 2024-09-20T22:06:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498498+0000", 2024-09-20T22:06:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013918599999999999 2024-09-20T22:06:08.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:06:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498751+0000", 2024-09-20T22:06:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025235999999999998 2024-09-20T22:06:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.500697+0000", 2024-09-20T22:06:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0019460969999999999 2024-09-20T22:06:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502528+0000", 2024-09-20T22:06:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00183161 2024-09-20T22:06:08.330 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502597+0000", 2024-09-20T22:06:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8761000000000001e-05 2024-09-20T22:06:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502666+0000", 2024-09-20T22:06:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8647999999999997e-05 2024-09-20T22:06:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:08.331 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:08.332 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:06:08.363 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:08.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:08.364 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:08.465 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:08.505 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:08.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:08.506 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:08.607 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:08.621 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-20T22:06:06.347179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5931768371,"num_objects":90703,"num_object_clones":0,"num_object_copies":272109,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33681,"num_objects_misplaced":46808,"num_objects_unfound":0,"num_objects_dirty":90703,"num_whiteouts":0,"num_read":1115,"num_read_kb":961,"num_write":143652,"num_write_kb":6909406,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1144,"num_bytes_recovered":75564640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148897,"ondisk_log_size":148897,"up":406,"acting":412,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":488,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":17347376,"kb_used_data":16429916,"kb_used_omap":12,"kb_used_meta":917107,"kb_avail":732613840,"statfs":{"total":767960285184,"available":750196572160,"internally_reserved":0,"allocated":16824233984,"data_stored":16820244222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":939118128},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,5],"upper_bound":2048},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1288372224,"num_objects":19661,"num_object_clones":0,"num_object_copies":58983,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10000,"num_objects_misplaced":43826,"num_objects_unfound":0,"num_objects_dirty":19661,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":36193,"num_write_kb":1600608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1135,"num_bytes_recovered":73793536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.010434"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":143,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.499122+0000","last_change":"2024-09-20T22:04:21.109795+0000","last_active":"2024-09-20T22:06:03.499122+0000","last_peered":"2024-09-20T22:06:03.499122+0000","last_clean":"2024-09-20T22:06:03.499122+0000","last_became_active":"2024-09-20T22:02:50.109930+0000","last_became_peered":"2024-09-20T22:02:50.109930+0000","last_unstale":"2024-09-20T22:06:03.499122+0000","last_undegraded":"2024-09-20T22:06:03.499122+0000","last_fullsized":"2024-09-20T22:06:03.499122+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:21.109683+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:21.109683+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"71'9","reported_seq":159,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.498176+0000","last_change":"2024-09-20T22:04:16.312321+0000","last_active":"2024-09-20T22:06:03.498176+0000","last_peered":"2024-09-20T22:06:03.498176+0000","last_clean":"2024-09-20T22:04:11.302960+0000","last_became_active":"2024-09-20T22:04:16.312321+0000","last_became_peered":"2024-09-20T22:04:16.312321+0000","last_unstale":"2024-09-20T22:06:03.498176+0000","last_undegraded":"2024-09-20T22:04:16.304969+0000","last_fullsized":"2024-09-20T22:04:16.304866+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:10.928348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:10.928348+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:03.964588+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[{"shards":"4,6","objects":4}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":77,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.493945+0000","last_change":"2024-09-20T22:04:16.315219+0000","last_active":"2024-09-20T22:06:03.493945+0000","last_peered":"2024-09-20T22:06:03.493945+0000","last_clean":"2024-09-20T22:02:53.269054+0000","last_became_active":"2024-09-20T22:04:16.315219+0000","last_became_peered":"2024-09-20T22:04:16.315219+0000","last_unstale":"2024-09-20T22:06:03.493945+0000","last_undegraded":"2024-09-20T22:06:03.493945+0000","last_fullsized":"2024-09-20T22:04:16.308508+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:11.178591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:11.178591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":127,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.497842+0000","last_change":"2024-09-20T22:04:16.315650+0000","last_active":"2024-09-20T22:06:03.497842+0000","last_peered":"2024-09-20T22:06:03.497842+0000","last_clean":"2024-09-20T22:04:11.303058+0000","last_became_active":"2024-09-20T22:04:16.315650+0000","last_became_peered":"2024-09-20T22:04:16.315650+0000","last_unstale":"2024-09-20T22:06:03.497842+0000","last_undegraded":"2024-09-20T22:06:03.497842+0000","last_fullsized":"2024-09-20T22:04:16.303835+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":132,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.504786+0000","last_change":"2024-09-20T22:05:06.024320+0000","last_active":"2024-09-20T22:06:03.504786+0000","last_peered":"2024-09-20T22:06:03.504786+0000","last_clean":"2024-09-20T22:06:03.504786+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:06:03.504786+0000","last_undegraded":"2024-09-20T22:06:03.504786+0000","last_fullsized":"2024-09-20T22:06:03.504786+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:05:06.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:06.024051+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:11.371826+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":143,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.507228+0000","last_change":"2024-09-20T22:05:04.976359+0000","last_active":"2024-09-20T22:06:03.507228+0000","last_peered":"2024-09-20T22:06:03.507228+0000","last_clean":"2024-09-20T22:06:03.507228+0000","last_became_active":"2024-09-20T22:02:50.107154+0000","last_became_peered":"2024-09-20T22:02:50.107154+0000","last_unstale":"2024-09-20T22:06:03.507228+0000","last_undegraded":"2024-09-20T22:06:03.507228+0000","last_fullsized":"2024-09-20T22:06:03.507228+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:04.976218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:04.976218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:27.101580+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"71'15","reported_seq":164,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.507528+0000","last_change":"2024-09-20T22:04:15.767473+0000","last_active":"2024-09-20T22:06:03.507528+0000","last_peered":"2024-09-20T22:06:03.507528+0000","last_clean":"2024-09-20T22:06:03.507528+0000","last_became_active":"2024-09-20T22:02:52.282101+0000","last_became_peered":"2024-09-20T22:02:52.282101+0000","last_unstale":"2024-09-20T22:06:03.507528+0000","last_undegraded":"2024-09-20T22:06:03.507528+0000","last_fullsized":"2024-09-20T22:06:03.507528+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:04:15.767389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:15.767389+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":131,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.497838+0000","last_change":"2024-09-20T22:04:16.312826+0000","last_active":"2024-09-20T22:06:03.497838+0000","last_peered":"2024-09-20T22:06:03.497838+0000","last_clean":"2024-09-20T22:04:11.302880+0000","last_became_active":"2024-09-20T22:04:16.312826+0000","last_became_peered":"2024-09-20T22:04:16.312826+0000","last_unstale":"2024-09-20T22:06:03.497838+0000","last_undegraded":"2024-09-20T22:06:03.497838+0000","last_fullsized":"2024-09-20T22:04:16.303524+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.19","version":"126'10731","reported_seq":10849,"reported_epoch":126,"state":"active+undersized+degraded+remapped+backfill_toofull","last_fresh":"2024-09-20T22:06:05.780346+0000","last_change":"2024-09-20T22:06:02.490027+0000","last_active":"2024-09-20T22:06:05.780346+0000","last_peered":"2024-09-20T22:06:05.780346+0000","last_clean":"2024-09-20T22:04:15.706494+0000","last_became_active":"2024-09-20T22:06:02.449423+0000","last_became_peered":"2024-09-20T22:06:02.449423+0000","last_unstale":"2024-09-20T22:06:05.780346+0000","last_undegraded":"2024-09-20T22:06:02.482226+0000","last_fullsized":"2024-09-20T22:06:02.342084+0000","mapping_epoch":124,"log_start":"102'5200","ondisk_log_start":"102'5200","created":121,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":5531,"log_dups_size":0,"ondisk_log_size":5531,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187957248,"num_objects":2868,"num_object_clones":0,"num_object_copies":8604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2868,"num_objects_misplaced":5736,"num_objects_unfound":0,"num_objects_dirty":2868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5455,"num_write_kb":266336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[{"shards":"0,4","objects":2868}],"blocked_by":[],"up_primary":6,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":151,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.495129+0000","last_change":"2024-09-20T22:05:28.665445+0000","last_active":"2024-09-20T22:06:03.495129+0000","last_peered":"2024-09-20T22:06:03.495129+0000","last_clean":"2024-09-20T22:06:03.495129+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:06:03.495129+0000","last_undegraded":"2024-09-20T22:06:03.495129+0000","last_fullsized":"2024-09-20T22:06:03.495129+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.665329+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.665329+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:34.569519+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"71'9","reported_seq":156,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.498105+0000","last_change":"2024-09-20T22:04:52.800006+0000","last_active":"2024-09-20T22:06:03.498105+0000","last_peered":"2024-09-20T22:06:03.498105+0000","last_clean":"2024-09-20T22:06:03.498105+0000","last_became_active":"2024-09-20T22:02:52.279526+0000","last_became_peered":"2024-09-20T22:02:52.279526+0000","last_unstale":"2024-09-20T22:06:03.498105+0000","last_undegraded":"2024-09-20T22:06:03.498105+0000","last_fullsized":"2024-09-20T22:06:03.498105+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:52.799870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:52.799870+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":135,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.494268+0000","last_change":"2024-09-20T22:05:02.519662+0000","last_active":"2024-09-20T22:06:03.494268+0000","last_peered":"2024-09-20T22:06:03.494268+0000","last_clean":"2024-09-20T22:06:03.494268+0000","last_became_active":"2024-09-20T22:02:54.292911+0000","last_became_peered":"2024-09-20T22:02:54.292911+0000","last_unstale":"2024-09-20T22:06:03.494268+0000","last_undegraded":"2024-09-20T22:06:03.494268+0000","last_fullsized":"2024-09-20T22:06:03.494268+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:02.519415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:02.519415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:04.396928+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":126,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.492405+0000","last_change":"2024-09-20T22:04:16.314659+0000","last_active":"2024-09-20T22:06:03.492405+0000","last_peered":"2024-09-20T22:06:03.492405+0000","last_clean":"2024-09-20T22:04:11.306271+0000","last_became_active":"2024-09-20T22:04:16.314659+0000","last_became_peered":"2024-09-20T22:04:16.314659+0000","last_unstale":"2024-09-20T22:06:03.492405+0000","last_undegraded":"2024-09-20T22:06:03.492405+0000","last_fullsized":"2024-09-20T22:04:16.302831+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:05.570159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.18","version":"126'10994","reported_seq":12306,"reported_epoch":126,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T22:06:05.804483+0000","last_change":"2024-09-20T22:06:02.516477+0000","last_active":"2024-09-20T22:06:05.804483+0000","last_peered":"2024-09-20T22:06:05.804483+0000","last_clean":"2024-09-20T22:06:00.420952+0000","last_became_active":"2024-09-20T22:06:02.455341+0000","last_became_peered":"2024-09-20T22:06:02.455341+0000","last_unstale":"2024-09-20T22:06:05.804483+0000","last_undegraded":"2024-09-20T22:06:05.804483+0000","last_fullsized":"2024-09-20T22:06:05.804483+0000","mapping_epoch":124,"log_start":"98'3800","ondisk_log_start":"98'3800","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":5,"last_scrub":"107'6452","last_scrub_stamp":"2024-09-20T22:04:59.310325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:04:59.310325+0000","objects_scrubbed":6357,"log_size":7194,"log_dups_size":0,"ondisk_log_size":7194,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2308,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187105280,"num_objects":2855,"num_object_clones":0,"num_object_copies":8565,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4988,"num_objects_unfound":0,"num_objects_dirty":2855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5588,"num_write_kb":270176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":24510464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[2,7,4],"avail_no_missing":["2","4","7"],"object_location_counts":[{"shards":"2,4,7","objects":2855}],"blocked_by":[],"up_primary":0,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":159,"reported_epoch":126,"state":"active+clean+remapped","last_fresh":"2024-09-20T22:06:05.351947+0000","last_change":"2024-09-20T22:06:05.352020+0000","last_active":"2024-09-20T22:06:05.351947+0000","last_peered":"2024-09-20T22:06:05.351947+0000","last_clean":"2024-09-20T22:06:05.351947+0000","last_became_active":"2024-09-20T22:04:21.311111+0000","last_became_peered":"2024-09-20T22:04:21.311111+0000","last_unstale":"2024-09-20T22:06:05.351947+0000","last_undegraded":"2024-09-20T22:06:05.351947+0000","last_fullsized":"2024-09-20T22:06:05.351947+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:05.351834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:05.351834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:25.354007+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2,4],"avail_no_missing":["5","2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"71'5","reported_seq":153,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.498361+0000","last_change":"2024-09-20T22:04:16.321444+0000","last_active":"2024-09-20T22:06:03.498361+0000","last_peered":"2024-09-20T22:06:03.498361+0000","last_clean":"2024-09-20T22:04:13.950323+0000","last_became_active":"2024-09-20T22:04:16.321444+0000","last_became_peered":"2024-09-20T22:04:16.321444+0000","last_unstale":"2024-09-20T22:06:03.498361+0000","last_undegraded":"2024-09-20T22:04:16.304418+0000","last_fullsized":"2024-09-20T22:04:16.304302+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'5","last_scrub_stamp":"2024-09-20T22:04:13.950197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:13.950197+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:03.964569+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[{"shards":"3,4","objects":2}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":139,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.499582+0000","last_change":"2024-09-20T22:04:16.316889+0000","last_active":"2024-09-20T22:06:03.499582+0000","last_peered":"2024-09-20T22:06:03.499582+0000","last_clean":"2024-09-20T22:04:14.618055+0000","last_became_active":"2024-09-20T22:04:16.316889+0000","last_became_peered":"2024-09-20T22:04:16.316889+0000","last_unstale":"2024-09-20T22:06:03.499582+0000","last_undegraded":"2024-09-20T22:06:03.499582+0000","last_fullsized":"2024-09-20T22:04:16.310076+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:14.617830+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:14.617830+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":80,"reported_epoch":126,"state":"active+clean+remapped","last_fresh":"2024-09-20T22:06:03.492902+0000","last_change":"2024-09-20T22:05:29.551389+0000","last_active":"2024-09-20T22:06:03.492902+0000","last_peered":"2024-09-20T22:06:03.492902+0000","last_clean":"2024-09-20T22:06:03.492902+0000","last_became_active":"2024-09-20T22:04:21.312179+0000","last_became_peered":"2024-09-20T22:04:21.312179+0000","last_unstale":"2024-09-20T22:06:03.492902+0000","last_undegraded":"2024-09-20T22:06:03.492902+0000","last_fullsized":"2024-09-20T22:06:03.492902+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:05:29.551172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:39.943718+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6,3],"avail_no_missing":["0","3","6"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":132,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.498849+0000","last_change":"2024-09-20T22:04:59.903000+0000","last_active":"2024-09-20T22:06:03.498849+0000","last_peered":"2024-09-20T22:06:03.498849+0000","last_clean":"2024-09-20T22:06:03.498849+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:06:03.498849+0000","last_undegraded":"2024-09-20T22:06:03.498849+0000","last_fullsized":"2024-09-20T22:06:03.498849+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:59.902900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:59.902900+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:03.610127+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":143,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.497829+0000","last_change":"2024-09-20T22:05:31.552599+0000","last_active":"2024-09-20T22:06:03.497829+0000","last_peered":"2024-09-20T22:06:03.497829+0000","last_clean":"2024-09-20T22:06:03.497829+0000","last_became_active":"2024-09-20T22:02:50.111517+0000","last_became_peered":"2024-09-20T22:02:50.111517+0000","last_unstale":"2024-09-20T22:06:03.497829+0000","last_undegraded":"2024-09-20T22:06:03.497829+0000","last_fullsized":"2024-09-20T22:06:03.497829+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:31.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:31.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:44.579423+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"71'15","reported_seq":167,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.497255+0000","last_change":"2024-09-20T22:04:16.311393+0000","last_active":"2024-09-20T22:06:03.497255+0000","last_peered":"2024-09-20T22:06:03.497255+0000","last_clean":"2024-09-20T22:04:11.306869+0000","last_became_active":"2024-09-20T22:04:16.311393+0000","last_became_peered":"2024-09-20T22:04:16.311393+0000","last_unstale":"2024-09-20T22:06:03.497255+0000","last_undegraded":"2024-09-20T22:04:16.306249+0000","last_fullsized":"2024-09-20T22:04:16.306143+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:04:10.667027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:10.667027+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:07.071091+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":7}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":77,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.498410+0000","last_change":"2024-09-20T22:04:16.313788+0000","last_active":"2024-09-20T22:06:03.498410+0000","last_peered":"2024-09-20T22:06:03.498410+0000","last_clean":"2024-09-20T22:02:53.269054+0000","last_became_active":"2024-09-20T22:04:16.313788+0000","last_became_peered":"2024-09-20T22:04:16.313788+0000","last_unstale":"2024-09-20T22:06:03.498410+0000","last_undegraded":"2024-09-20T22:06:03.498410+0000","last_fullsized":"2024-09-20T22:04:16.304888+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":131,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.513023+0000","last_change":"2024-09-20T22:04:10.722434+0000","last_active":"2024-09-20T22:06:03.513023+0000","last_peered":"2024-09-20T22:06:03.513023+0000","last_clean":"2024-09-20T22:06:03.513023+0000","last_became_active":"2024-09-20T22:02:56.347561+0000","last_became_peered":"2024-09-20T22:02:56.347561+0000","last_unstale":"2024-09-20T22:06:03.513023+0000","last_undegraded":"2024-09-20T22:06:03.513023+0000","last_fullsized":"2024-09-20T22:06:03.513023+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:10.722324+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:10.722324+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":80,"reported_epoch":126,"state":"active+clean+remapped","last_fresh":"2024-09-20T22:06:03.494005+0000","last_change":"2024-09-20T22:04:53.467338+0000","last_active":"2024-09-20T22:06:03.494005+0000","last_peered":"2024-09-20T22:06:03.494005+0000","last_clean":"2024-09-20T22:06:03.494005+0000","last_became_active":"2024-09-20T22:04:21.311331+0000","last_became_peered":"2024-09-20T22:04:21.311331+0000","last_unstale":"2024-09-20T22:06:03.494005+0000","last_undegraded":"2024-09-20T22:06:03.494005+0000","last_fullsized":"2024-09-20T22:06:03.494005+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:53.467207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:53.467207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:13.830247+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2,4],"avail_no_missing":["5","2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"71'12","reported_seq":157,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.504770+0000","last_change":"2024-09-20T22:04:16.321585+0000","last_active":"2024-09-20T22:06:03.504770+0000","last_peered":"2024-09-20T22:06:03.504770+0000","last_clean":"2024-09-20T22:04:11.310784+0000","last_became_active":"2024-09-20T22:04:16.321585+0000","last_became_peered":"2024-09-20T22:04:16.321585+0000","last_unstale":"2024-09-20T22:06:03.504770+0000","last_undegraded":"2024-09-20T22:04:16.306903+0000","last_fullsized":"2024-09-20T22:04:16.306792+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:06.925503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[{"shards":"3,4","objects":5}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":136,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.498020+0000","last_change":"2024-09-20T22:05:28.003302+0000","last_active":"2024-09-20T22:06:03.498020+0000","last_peered":"2024-09-20T22:06:03.498020+0000","last_clean":"2024-09-20T22:06:03.498020+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:06:03.498020+0000","last_undegraded":"2024-09-20T22:06:03.498020+0000","last_fullsized":"2024-09-20T22:06:03.498020+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.003183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:33.356060+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":134,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.500723+0000","last_change":"2024-09-20T22:05:33.751965+0000","last_active":"2024-09-20T22:06:03.500723+0000","last_peered":"2024-09-20T22:06:03.500723+0000","last_clean":"2024-09-20T22:06:03.500723+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:06:03.500723+0000","last_undegraded":"2024-09-20T22:06:03.500723+0000","last_fullsized":"2024-09-20T22:06:03.500723+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:05:33.751883+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:00.978967+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"64'1","reported_seq":86,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.507299+0000","last_change":"2024-09-20T22:04:16.322089+0000","last_active":"2024-09-20T22:06:03.507299+0000","last_peered":"2024-09-20T22:06:03.507299+0000","last_clean":"2024-09-20T22:02:50.113625+0000","last_became_active":"2024-09-20T22:04:16.322089+0000","last_became_peered":"2024-09-20T22:04:16.322089+0000","last_unstale":"2024-09-20T22:06:03.507299+0000","last_undegraded":"2024-09-20T22:04:16.305892+0000","last_fullsized":"2024-09-20T22:04:16.305784+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:06.925457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[{"shards":"3,4","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"71'10","reported_seq":154,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.505590+0000","last_change":"2024-09-20T22:04:56.414262+0000","last_active":"2024-09-20T22:06:03.505590+0000","last_peered":"2024-09-20T22:06:03.505590+0000","last_clean":"2024-09-20T22:06:03.505590+0000","last_became_active":"2024-09-20T22:02:52.481980+0000","last_became_peered":"2024-09-20T22:02:52.481980+0000","last_unstale":"2024-09-20T22:06:03.505590+0000","last_undegraded":"2024-09-20T22:06:03.505590+0000","last_fullsized":"2024-09-20T22:06:03.505590+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:04:56.414020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:56.414020+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"71'8","reported_seq":186,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.491269+0000","last_change":"2024-09-20T22:05:01.351815+0000","last_active":"2024-09-20T22:06:03.491269+0000","last_peered":"2024-09-20T22:06:03.491269+0000","last_clean":"2024-09-20T22:06:03.491269+0000","last_became_active":"2024-09-20T22:02:54.297657+0000","last_became_peered":"2024-09-20T22:02:54.297657+0000","last_unstale":"2024-09-20T22:06:03.491269+0000","last_undegraded":"2024-09-20T22:06:03.491269+0000","last_fullsized":"2024-09-20T22:06:03.491269+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:05:01.351702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:01.351702+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:23.580653+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.c","version":"126'10875","reported_seq":11053,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:05.647742+0000","last_change":"2024-09-20T22:05:04.128982+0000","last_active":"2024-09-20T22:06:05.647742+0000","last_peered":"2024-09-20T22:06:05.647742+0000","last_clean":"2024-09-20T22:06:05.647742+0000","last_became_active":"2024-09-20T22:03:52.318259+0000","last_became_peered":"2024-09-20T22:03:52.318259+0000","last_unstale":"2024-09-20T22:06:05.647742+0000","last_undegraded":"2024-09-20T22:06:05.647742+0000","last_fullsized":"2024-09-20T22:06:05.647742+0000","mapping_epoch":80,"log_start":"103'5368","ondisk_log_start":"103'5368","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"108'6604","last_scrub_stamp":"2024-09-20T22:05:04.128810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:05:04.128810+0000","objects_scrubbed":6541,"log_size":5507,"log_dups_size":0,"ondisk_log_size":5507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:19.797130+0000","scrub_duration":1801,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":367591424,"num_objects":5609,"num_object_clones":0,"num_object_copies":16827,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10875,"num_write_kb":527488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.f","version":"71'15","reported_seq":121,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.513124+0000","last_change":"2024-09-20T22:04:16.323258+0000","last_active":"2024-09-20T22:06:03.513124+0000","last_peered":"2024-09-20T22:06:03.513124+0000","last_clean":"2024-09-20T22:02:57.665294+0000","last_became_active":"2024-09-20T22:04:16.323258+0000","last_became_peered":"2024-09-20T22:04:16.323258+0000","last_unstale":"2024-09-20T22:06:03.513124+0000","last_undegraded":"2024-09-20T22:04:16.312038+0000","last_fullsized":"2024-09-20T22:04:16.311916+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:04:13.235907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:13.235907+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[{"shards":"3,4","objects":7}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":138,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.511545+0000","last_change":"2024-09-20T22:04:16.317334+0000","last_active":"2024-09-20T22:06:03.511545+0000","last_peered":"2024-09-20T22:06:03.511545+0000","last_clean":"2024-09-20T22:04:11.312325+0000","last_became_active":"2024-09-20T22:04:16.317334+0000","last_became_peered":"2024-09-20T22:04:16.317334+0000","last_unstale":"2024-09-20T22:06:03.511545+0000","last_undegraded":"2024-09-20T22:06:03.511545+0000","last_fullsized":"2024-09-20T22:04:16.308050+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:06.925510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"71'8","reported_seq":195,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:04.874166+0000","last_change":"2024-09-20T22:06:04.874166+0000","last_active":"2024-09-20T22:06:04.874166+0000","last_peered":"2024-09-20T22:06:04.874166+0000","last_clean":"2024-09-20T22:06:04.874166+0000","last_became_active":"2024-09-20T22:02:54.294917+0000","last_became_peered":"2024-09-20T22:02:54.294917+0000","last_unstale":"2024-09-20T22:06:04.874166+0000","last_undegraded":"2024-09-20T22:06:04.874166+0000","last_fullsized":"2024-09-20T22:06:04.874166+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:06:04.874042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:04.874042+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:28.727825+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":88,"reported_epoch":126,"state":"active+clean+remapped","last_fresh":"2024-09-20T22:06:03.490395+0000","last_change":"2024-09-20T22:05:57.309472+0000","last_active":"2024-09-20T22:06:03.490395+0000","last_peered":"2024-09-20T22:06:03.490395+0000","last_clean":"2024-09-20T22:06:03.490395+0000","last_became_active":"2024-09-20T22:04:21.310424+0000","last_became_peered":"2024-09-20T22:04:21.310424+0000","last_unstale":"2024-09-20T22:06:03.490395+0000","last_undegraded":"2024-09-20T22:06:03.490395+0000","last_fullsized":"2024-09-20T22:06:03.490395+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:57.309262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:57.309262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:09.184401+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0,5],"avail_no_missing":["6","0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.3","version":"126'10770","reported_seq":11010,"reported_epoch":126,"state":"active+clean+remapped","last_fresh":"2024-09-20T22:06:05.764515+0000","last_change":"2024-09-20T22:06:00.679079+0000","last_active":"2024-09-20T22:06:05.764515+0000","last_peered":"2024-09-20T22:06:05.764515+0000","last_clean":"2024-09-20T22:06:05.764515+0000","last_became_active":"2024-09-20T22:05:59.459981+0000","last_became_peered":"2024-09-20T22:05:59.459981+0000","last_unstale":"2024-09-20T22:06:05.764515+0000","last_undegraded":"2024-09-20T22:06:05.764515+0000","last_fullsized":"2024-09-20T22:06:05.764515+0000","mapping_epoch":95,"log_start":"98'3600","ondisk_log_start":"98'3600","created":80,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"122'10632","last_scrub_stamp":"2024-09-20T22:06:00.678805+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.678805+0000","objects_scrubbed":2987,"log_size":7170,"log_dups_size":0,"ondisk_log_size":7170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:28.823229+0000","scrub_duration":607,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":186384384,"num_objects":2844,"num_object_clones":0,"num_object_copies":8532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2844,"num_objects_unfound":0,"num_objects_dirty":2844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":143,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4,3],"avail_no_missing":["2","3","4"],"object_location_counts":[{"shards":"2,3,4","objects":2844}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"71'18","reported_seq":171,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.509234+0000","last_change":"2024-09-20T22:04:57.967494+0000","last_active":"2024-09-20T22:06:03.509234+0000","last_peered":"2024-09-20T22:06:03.509234+0000","last_clean":"2024-09-20T22:06:03.509234+0000","last_became_active":"2024-09-20T22:02:52.482119+0000","last_became_peered":"2024-09-20T22:02:52.482119+0000","last_unstale":"2024-09-20T22:06:03.509234+0000","last_undegraded":"2024-09-20T22:06:03.509234+0000","last_fullsized":"2024-09-20T22:06:03.509234+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'18","last_scrub_stamp":"2024-09-20T22:04:57.967216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:57.967216+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:07.936978+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":143,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.513372+0000","last_change":"2024-09-20T22:05:02.924002+0000","last_active":"2024-09-20T22:06:03.513372+0000","last_peered":"2024-09-20T22:06:03.513372+0000","last_clean":"2024-09-20T22:06:03.513372+0000","last_became_active":"2024-09-20T22:02:50.113076+0000","last_became_peered":"2024-09-20T22:02:50.113076+0000","last_unstale":"2024-09-20T22:06:03.513372+0000","last_undegraded":"2024-09-20T22:06:03.513372+0000","last_fullsized":"2024-09-20T22:06:03.513372+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:02.923565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:02.923565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:29.688625+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":136,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.497994+0000","last_change":"2024-09-20T22:04:20.841355+0000","last_active":"2024-09-20T22:06:03.497994+0000","last_peered":"2024-09-20T22:06:03.497994+0000","last_clean":"2024-09-20T22:06:03.497994+0000","last_became_active":"2024-09-20T22:02:54.288007+0000","last_became_peered":"2024-09-20T22:02:54.288007+0000","last_unstale":"2024-09-20T22:06:03.497994+0000","last_undegraded":"2024-09-20T22:06:03.497994+0000","last_fullsized":"2024-09-20T22:06:03.497994+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:20.841289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:20.841289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":140,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.498097+0000","last_change":"2024-09-20T22:05:29.990030+0000","last_active":"2024-09-20T22:06:03.498097+0000","last_peered":"2024-09-20T22:06:03.498097+0000","last_clean":"2024-09-20T22:06:03.498097+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:06:03.498097+0000","last_undegraded":"2024-09-20T22:06:03.498097+0000","last_fullsized":"2024-09-20T22:06:03.498097+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:29.989934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:29.989934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:34.056485+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"126'10796","reported_seq":10907,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:05.823696+0000","last_change":"2024-09-20T22:05:59.451475+0000","last_active":"2024-09-20T22:06:05.823696+0000","last_peered":"2024-09-20T22:06:05.823696+0000","last_clean":"2024-09-20T22:04:15.696640+0000","last_became_active":"2024-09-20T22:05:59.451475+0000","last_became_peered":"2024-09-20T22:05:59.451475+0000","last_unstale":"2024-09-20T22:06:05.823696+0000","last_undegraded":"2024-09-20T22:05:59.445776+0000","last_fullsized":"2024-09-20T22:05:59.445640+0000","mapping_epoch":91,"log_start":"104'5900","ondisk_log_start":"104'5900","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":4896,"log_dups_size":0,"ondisk_log_size":4896,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:05.926244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":182616064,"num_objects":2786,"num_object_clones":0,"num_object_copies":8358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2786,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5501,"num_write_kb":265216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":["3","2"],"object_location_counts":[{"shards":"2,3","objects":2786}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"71'14","reported_seq":169,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.498337+0000","last_change":"2024-09-20T22:06:00.944172+0000","last_active":"2024-09-20T22:06:03.498337+0000","last_peered":"2024-09-20T22:06:03.498337+0000","last_clean":"2024-09-20T22:06:03.498337+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:06:03.498337+0000","last_undegraded":"2024-09-20T22:06:03.498337+0000","last_fullsized":"2024-09-20T22:06:03.498337+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'14","last_scrub_stamp":"2024-09-20T22:06:00.944086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.944086+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:21.819374+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":139,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.505793+0000","last_change":"2024-09-20T22:04:16.313432+0000","last_active":"2024-09-20T22:06:03.505793+0000","last_peered":"2024-09-20T22:06:03.505793+0000","last_clean":"2024-09-20T22:04:11.308782+0000","last_became_active":"2024-09-20T22:04:16.313432+0000","last_became_peered":"2024-09-20T22:04:16.313432+0000","last_unstale":"2024-09-20T22:06:03.505793+0000","last_undegraded":"2024-09-20T22:04:16.305282+0000","last_fullsized":"2024-09-20T22:04:16.305176+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:05.570058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[{"shards":"0,4","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"92'9","reported_seq":179,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.504377+0000","last_change":"2024-09-20T22:04:16.319525+0000","last_active":"2024-09-20T22:06:03.504377+0000","last_peered":"2024-09-20T22:06:03.504377+0000","last_clean":"2024-09-20T22:04:15.181371+0000","last_became_active":"2024-09-20T22:04:16.319525+0000","last_became_peered":"2024-09-20T22:04:16.319525+0000","last_unstale":"2024-09-20T22:06:03.504377+0000","last_undegraded":"2024-09-20T22:04:16.305343+0000","last_fullsized":"2024-09-20T22:04:16.305233+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:06.925480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":["3","4"],"object_location_counts":[{"shards":"3,4","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":132,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.500036+0000","last_change":"2024-09-20T22:04:56.605725+0000","last_active":"2024-09-20T22:06:03.500036+0000","last_peered":"2024-09-20T22:06:03.500036+0000","last_clean":"2024-09-20T22:06:03.500036+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:06:03.500036+0000","last_undegraded":"2024-09-20T22:06:03.500036+0000","last_fullsized":"2024-09-20T22:06:03.500036+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:56.605594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:56.605594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:21.394492+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1","version":"126'11037","reported_seq":11249,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:05.559610+0000","last_change":"2024-09-20T22:06:00.562515+0000","last_active":"2024-09-20T22:06:05.559610+0000","last_peered":"2024-09-20T22:06:05.559610+0000","last_clean":"2024-09-20T22:06:05.559610+0000","last_became_active":"2024-09-20T22:05:59.470086+0000","last_became_peered":"2024-09-20T22:05:59.470086+0000","last_unstale":"2024-09-20T22:06:05.559610+0000","last_undegraded":"2024-09-20T22:06:05.559610+0000","last_fullsized":"2024-09-20T22:06:05.559610+0000","mapping_epoch":80,"log_start":"104'6100","ondisk_log_start":"104'6100","created":80,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"122'10913","last_scrub_stamp":"2024-09-20T22:06:00.562376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.562376+0000","objects_scrubbed":2968,"log_size":4937,"log_dups_size":0,"ondisk_log_size":4937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:25.342701+0000","scrub_duration":592,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":186056704,"num_objects":2839,"num_object_clones":0,"num_object_copies":8517,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":129,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"71'10","reported_seq":111,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.494290+0000","last_change":"2024-09-20T22:04:16.317022+0000","last_active":"2024-09-20T22:06:03.494290+0000","last_peered":"2024-09-20T22:06:03.494290+0000","last_clean":"2024-09-20T22:02:57.633971+0000","last_became_active":"2024-09-20T22:04:16.317022+0000","last_became_peered":"2024-09-20T22:04:16.317022+0000","last_unstale":"2024-09-20T22:06:03.494290+0000","last_undegraded":"2024-09-20T22:04:16.307899+0000","last_fullsized":"2024-09-20T22:04:16.307784+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[{"shards":"4,5","objects":5}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":151,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.495222+0000","last_change":"2024-09-20T22:05:22.673440+0000","last_active":"2024-09-20T22:06:03.495222+0000","last_peered":"2024-09-20T22:06:03.495222+0000","last_clean":"2024-09-20T22:06:03.495222+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:06:03.495222+0000","last_undegraded":"2024-09-20T22:06:03.495222+0000","last_fullsized":"2024-09-20T22:06:03.495222+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:22.673310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:22.673310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:28.579242+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"71'8","reported_seq":183,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.492883+0000","last_change":"2024-09-20T22:04:20.160515+0000","last_active":"2024-09-20T22:06:03.492883+0000","last_peered":"2024-09-20T22:06:03.492883+0000","last_clean":"2024-09-20T22:06:03.492883+0000","last_became_active":"2024-09-20T22:02:54.292819+0000","last_became_peered":"2024-09-20T22:02:54.292819+0000","last_unstale":"2024-09-20T22:06:03.492883+0000","last_undegraded":"2024-09-20T22:06:03.492883+0000","last_fullsized":"2024-09-20T22:06:03.492883+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:04:20.160405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:20.160405+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":131,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.493653+0000","last_change":"2024-09-20T22:06:00.572347+0000","last_active":"2024-09-20T22:06:03.493653+0000","last_peered":"2024-09-20T22:06:03.493653+0000","last_clean":"2024-09-20T22:06:03.493653+0000","last_became_active":"2024-09-20T22:02:56.304274+0000","last_became_peered":"2024-09-20T22:02:56.304274+0000","last_unstale":"2024-09-20T22:06:03.493653+0000","last_undegraded":"2024-09-20T22:06:03.493653+0000","last_fullsized":"2024-09-20T22:06:03.493653+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:00.572243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.572243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:21.230297+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.0","version":"126'10813","reported_seq":10925,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:05.438854+0000","last_change":"2024-09-20T22:05:59.456631+0000","last_active":"2024-09-20T22:06:05.438854+0000","last_peered":"2024-09-20T22:06:05.438854+0000","last_clean":"2024-09-20T22:04:15.697798+0000","last_became_active":"2024-09-20T22:05:59.456631+0000","last_became_peered":"2024-09-20T22:05:59.456631+0000","last_unstale":"2024-09-20T22:06:05.438854+0000","last_undegraded":"2024-09-20T22:05:59.452323+0000","last_fullsized":"2024-09-20T22:05:59.452179+0000","mapping_epoch":91,"log_start":"99'4700","ondisk_log_start":"99'4700","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":6113,"log_dups_size":0,"ondisk_log_size":6113,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:06.391060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":188841984,"num_objects":2881,"num_object_clones":0,"num_object_copies":8643,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2881,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5485,"num_write_kb":267744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[{"shards":"3,5","objects":2881}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3","version":"71'19","reported_seq":183,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.493199+0000","last_change":"2024-09-20T22:05:24.600933+0000","last_active":"2024-09-20T22:06:03.493199+0000","last_peered":"2024-09-20T22:06:03.493199+0000","last_clean":"2024-09-20T22:06:03.493199+0000","last_became_active":"2024-09-20T22:02:52.480077+0000","last_became_peered":"2024-09-20T22:02:52.480077+0000","last_unstale":"2024-09-20T22:06:03.493199+0000","last_undegraded":"2024-09-20T22:06:03.493199+0000","last_fullsized":"2024-09-20T22:06:03.493199+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:05:24.600795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:24.600795+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:42.120234+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":78,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.497479+0000","last_change":"2024-09-20T22:04:16.311744+0000","last_active":"2024-09-20T22:06:03.497479+0000","last_peered":"2024-09-20T22:06:03.497479+0000","last_clean":"2024-09-20T22:02:49.083871+0000","last_became_active":"2024-09-20T22:04:16.311744+0000","last_became_peered":"2024-09-20T22:04:16.311744+0000","last_unstale":"2024-09-20T22:06:03.497479+0000","last_undegraded":"2024-09-20T22:06:03.497479+0000","last_fullsized":"2024-09-20T22:04:16.306448+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:07.071084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":143,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.490553+0000","last_change":"2024-09-20T22:05:30.141747+0000","last_active":"2024-09-20T22:06:03.490553+0000","last_peered":"2024-09-20T22:06:03.490553+0000","last_clean":"2024-09-20T22:06:03.490553+0000","last_became_active":"2024-09-20T22:02:54.290787+0000","last_became_peered":"2024-09-20T22:02:54.290787+0000","last_unstale":"2024-09-20T22:06:03.490553+0000","last_undegraded":"2024-09-20T22:06:03.490553+0000","last_fullsized":"2024-09-20T22:06:03.490553+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:30.141642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:30.141642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:49.552221+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":77,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.491895+0000","last_change":"2024-09-20T22:04:16.319315+0000","last_active":"2024-09-20T22:06:03.491895+0000","last_peered":"2024-09-20T22:06:03.491895+0000","last_clean":"2024-09-20T22:02:55.278800+0000","last_became_active":"2024-09-20T22:04:16.319315+0000","last_became_peered":"2024-09-20T22:04:16.319315+0000","last_unstale":"2024-09-20T22:06:03.491895+0000","last_undegraded":"2024-09-20T22:06:03.491895+0000","last_fullsized":"2024-09-20T22:04:16.305813+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":["6","2"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.7","version":"126'10795","reported_seq":11036,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:05.642808+0000","last_change":"2024-09-20T22:06:00.734708+0000","last_active":"2024-09-20T22:06:05.642808+0000","last_peered":"2024-09-20T22:06:05.642808+0000","last_clean":"2024-09-20T22:06:05.642808+0000","last_became_active":"2024-09-20T22:05:59.483490+0000","last_became_peered":"2024-09-20T22:05:59.483490+0000","last_unstale":"2024-09-20T22:06:05.642808+0000","last_undegraded":"2024-09-20T22:06:05.642808+0000","last_fullsized":"2024-09-20T22:06:05.642808+0000","mapping_epoch":80,"log_start":"99'4600","ondisk_log_start":"99'4600","created":80,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"123'10686","last_scrub_stamp":"2024-09-20T22:06:00.734586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.734586+0000","objects_scrubbed":2875,"log_size":6195,"log_dups_size":0,"ondisk_log_size":6195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:05.158329+0000","scrub_duration":408,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":181272576,"num_objects":2766,"num_object_clones":0,"num_object_copies":8298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":109,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.4","version":"92'29","reported_seq":169,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.497134+0000","last_change":"2024-09-20T22:04:16.310703+0000","last_active":"2024-09-20T22:06:03.497134+0000","last_peered":"2024-09-20T22:06:03.497134+0000","last_clean":"2024-09-20T22:02:57.636090+0000","last_became_active":"2024-09-20T22:04:16.310703+0000","last_became_peered":"2024-09-20T22:04:16.310703+0000","last_unstale":"2024-09-20T22:06:03.497134+0000","last_undegraded":"2024-09-20T22:04:16.305923+0000","last_fullsized":"2024-09-20T22:04:16.305806+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:07.071075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":10}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":152,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.498679+0000","last_change":"2024-09-20T22:05:58.966616+0000","last_active":"2024-09-20T22:06:03.498679+0000","last_peered":"2024-09-20T22:06:03.498679+0000","last_clean":"2024-09-20T22:06:03.498679+0000","last_became_active":"2024-09-20T22:02:50.108457+0000","last_became_peered":"2024-09-20T22:02:50.108457+0000","last_unstale":"2024-09-20T22:06:03.498679+0000","last_undegraded":"2024-09-20T22:06:03.498679+0000","last_fullsized":"2024-09-20T22:06:03.498679+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:58.966541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:58.966541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:11.649235+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"66'2","reported_seq":148,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.494632+0000","last_change":"2024-09-20T22:04:16.320295+0000","last_active":"2024-09-20T22:06:03.494632+0000","last_peered":"2024-09-20T22:06:03.494632+0000","last_clean":"2024-09-20T22:04:11.311686+0000","last_became_active":"2024-09-20T22:04:16.320295+0000","last_became_peered":"2024-09-20T22:04:16.320295+0000","last_unstale":"2024-09-20T22:06:03.494632+0000","last_undegraded":"2024-09-20T22:04:16.309132+0000","last_fullsized":"2024-09-20T22:04:16.309009+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[{"shards":"5,6","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":130,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.498379+0000","last_change":"2024-09-20T22:04:16.314215+0000","last_active":"2024-09-20T22:06:03.498379+0000","last_peered":"2024-09-20T22:06:03.498379+0000","last_clean":"2024-09-20T22:04:11.307965+0000","last_became_active":"2024-09-20T22:04:16.314215+0000","last_became_peered":"2024-09-20T22:04:16.314215+0000","last_unstale":"2024-09-20T22:06:03.498379+0000","last_undegraded":"2024-09-20T22:06:03.498379+0000","last_fullsized":"2024-09-20T22:04:16.304680+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:05.570147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":131,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.506938+0000","last_change":"2024-09-20T22:05:34.954357+0000","last_active":"2024-09-20T22:06:03.506938+0000","last_peered":"2024-09-20T22:06:03.506938+0000","last_clean":"2024-09-20T22:06:03.506938+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:06:03.506938+0000","last_undegraded":"2024-09-20T22:06:03.506938+0000","last_fullsized":"2024-09-20T22:06:03.506938+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:05:34.953944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:35.421579+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"126'10778","reported_seq":10890,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:05.759932+0000","last_change":"2024-09-20T22:05:59.463592+0000","last_active":"2024-09-20T22:06:05.759932+0000","last_peered":"2024-09-20T22:06:05.759932+0000","last_clean":"2024-09-20T22:04:15.698370+0000","last_became_active":"2024-09-20T22:05:59.463592+0000","last_became_peered":"2024-09-20T22:05:59.463592+0000","last_unstale":"2024-09-20T22:06:05.759932+0000","last_undegraded":"2024-09-20T22:05:59.448501+0000","last_fullsized":"2024-09-20T22:05:59.448387+0000","mapping_epoch":91,"log_start":"102'5300","ondisk_log_start":"102'5300","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":5478,"log_dups_size":0,"ondisk_log_size":5478,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:10.652674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184516608,"num_objects":2815,"num_object_clones":0,"num_object_copies":8445,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2815,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5481,"num_write_kb":265504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[{"shards":"0,4","objects":2815}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"71'13","reported_seq":123,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.495405+0000","last_change":"2024-09-20T22:04:16.325664+0000","last_active":"2024-09-20T22:06:03.495405+0000","last_peered":"2024-09-20T22:06:03.495405+0000","last_clean":"2024-09-20T22:02:57.574176+0000","last_became_active":"2024-09-20T22:04:16.325664+0000","last_became_peered":"2024-09-20T22:04:16.325664+0000","last_unstale":"2024-09-20T22:06:03.495405+0000","last_undegraded":"2024-09-20T22:04:16.308863+0000","last_fullsized":"2024-09-20T22:04:16.308755+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[{"shards":"0,5","objects":5}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":78,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.497292+0000","last_change":"2024-09-20T22:04:16.319148+0000","last_active":"2024-09-20T22:06:03.497292+0000","last_peered":"2024-09-20T22:06:03.497292+0000","last_clean":"2024-09-20T22:02:49.083871+0000","last_became_active":"2024-09-20T22:04:16.319148+0000","last_became_peered":"2024-09-20T22:04:16.319148+0000","last_unstale":"2024-09-20T22:06:03.497292+0000","last_undegraded":"2024-09-20T22:06:03.497292+0000","last_fullsized":"2024-09-20T22:04:16.305482+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:07.071065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":["2","6"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":161,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.497062+0000","last_change":"2024-09-20T22:05:34.967627+0000","last_active":"2024-09-20T22:06:03.497062+0000","last_peered":"2024-09-20T22:06:03.497062+0000","last_clean":"2024-09-20T22:06:03.497062+0000","last_became_active":"2024-09-20T22:02:49.099508+0000","last_became_peered":"2024-09-20T22:02:49.099508+0000","last_unstale":"2024-09-20T22:06:03.497062+0000","last_undegraded":"2024-09-20T22:06:03.497062+0000","last_fullsized":"2024-09-20T22:06:03.497062+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:05:34.967504+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:34.967504+0000","last_clean_scrub_stamp":"2024-09-20T22:05:34.967504+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:45.258680+0000","scrub_duration":26,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":135,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.503246+0000","last_change":"2024-09-20T22:04:54.014490+0000","last_active":"2024-09-20T22:06:03.503246+0000","last_peered":"2024-09-20T22:06:03.503246+0000","last_clean":"2024-09-20T22:06:03.503246+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:06:03.503246+0000","last_undegraded":"2024-09-20T22:06:03.503246+0000","last_fullsized":"2024-09-20T22:06:03.503246+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:54.014237+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:54.014237+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":132,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.500514+0000","last_change":"2024-09-20T22:05:28.780920+0000","last_active":"2024-09-20T22:06:03.500514+0000","last_peered":"2024-09-20T22:06:03.500514+0000","last_clean":"2024-09-20T22:06:03.500514+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:06:03.500514+0000","last_undegraded":"2024-09-20T22:06:03.500514+0000","last_fullsized":"2024-09-20T22:06:03.500514+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.780758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.780758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:57.033951+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"126'10800","reported_seq":10977,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:05.551137+0000","last_change":"2024-09-20T22:06:00.374826+0000","last_active":"2024-09-20T22:06:05.551137+0000","last_peered":"2024-09-20T22:06:05.551137+0000","last_clean":"2024-09-20T22:06:05.551137+0000","last_became_active":"2024-09-20T22:05:59.467347+0000","last_became_peered":"2024-09-20T22:05:59.467347+0000","last_unstale":"2024-09-20T22:06:05.551137+0000","last_undegraded":"2024-09-20T22:06:05.551137+0000","last_fullsized":"2024-09-20T22:06:05.551137+0000","mapping_epoch":80,"log_start":"103'5400","ondisk_log_start":"103'5400","created":80,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"122'10691","last_scrub_stamp":"2024-09-20T22:06:00.373859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.373859+0000","objects_scrubbed":2996,"log_size":5400,"log_dups_size":0,"ondisk_log_size":5400,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:11.955718+0000","scrub_duration":680,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":188416000,"num_objects":2875,"num_object_clones":0,"num_object_copies":8625,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2875,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"71'12","reported_seq":152,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.505669+0000","last_change":"2024-09-20T22:04:16.309130+0000","last_active":"2024-09-20T22:06:03.505669+0000","last_peered":"2024-09-20T22:06:03.505669+0000","last_clean":"2024-09-20T22:04:11.308694+0000","last_became_active":"2024-09-20T22:04:16.309130+0000","last_became_peered":"2024-09-20T22:04:16.309130+0000","last_unstale":"2024-09-20T22:06:03.505669+0000","last_undegraded":"2024-09-20T22:04:16.304971+0000","last_fullsized":"2024-09-20T22:04:16.304867+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:05.570119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0,2","objects":6}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":143,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.493709+0000","last_change":"2024-09-20T22:04:14.234738+0000","last_active":"2024-09-20T22:06:03.493709+0000","last_peered":"2024-09-20T22:06:03.493709+0000","last_clean":"2024-09-20T22:06:03.493709+0000","last_became_active":"2024-09-20T22:02:50.110448+0000","last_became_peered":"2024-09-20T22:02:50.110448+0000","last_unstale":"2024-09-20T22:06:03.493709+0000","last_undegraded":"2024-09-20T22:06:03.493709+0000","last_fullsized":"2024-09-20T22:06:03.493709+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:14.234630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:14.234630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":339,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:04.496254+0000","last_change":"2024-09-20T22:04:16.317127+0000","last_active":"2024-09-20T22:06:04.496254+0000","last_peered":"2024-09-20T22:06:04.496254+0000","last_clean":"2024-09-20T22:04:14.769746+0000","last_became_active":"2024-09-20T22:04:16.317127+0000","last_became_peered":"2024-09-20T22:04:16.317127+0000","last_unstale":"2024-09-20T22:06:04.496254+0000","last_undegraded":"2024-09-20T22:04:16.310637+0000","last_fullsized":"2024-09-20T22:04:16.310524+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:02:48.075477+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:06.619524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":185,"num_read_kb":180,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[{"shards":"0,7","objects":1}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":130,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.495704+0000","last_change":"2024-09-20T22:04:16.325962+0000","last_active":"2024-09-20T22:06:03.495704+0000","last_peered":"2024-09-20T22:06:03.495704+0000","last_clean":"2024-09-20T22:04:11.311829+0000","last_became_active":"2024-09-20T22:04:16.325962+0000","last_became_peered":"2024-09-20T22:04:16.325962+0000","last_unstale":"2024-09-20T22:06:03.495704+0000","last_undegraded":"2024-09-20T22:06:03.495704+0000","last_fullsized":"2024-09-20T22:04:16.310570+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":77,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.494461+0000","last_change":"2024-09-20T22:04:16.324713+0000","last_active":"2024-09-20T22:06:03.494461+0000","last_peered":"2024-09-20T22:06:03.494461+0000","last_clean":"2024-09-20T22:02:55.278800+0000","last_became_active":"2024-09-20T22:04:16.324713+0000","last_became_peered":"2024-09-20T22:04:16.324713+0000","last_unstale":"2024-09-20T22:06:03.494461+0000","last_undegraded":"2024-09-20T22:06:03.494461+0000","last_fullsized":"2024-09-20T22:04:16.309381+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.6","version":"126'10824","reported_seq":10971,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:05.765447+0000","last_change":"2024-09-20T22:06:00.331675+0000","last_active":"2024-09-20T22:06:05.765447+0000","last_peered":"2024-09-20T22:06:05.765447+0000","last_clean":"2024-09-20T22:06:05.765447+0000","last_became_active":"2024-09-20T22:05:59.454153+0000","last_became_peered":"2024-09-20T22:05:59.454153+0000","last_unstale":"2024-09-20T22:06:05.765447+0000","last_undegraded":"2024-09-20T22:06:05.765447+0000","last_fullsized":"2024-09-20T22:06:05.765447+0000","mapping_epoch":80,"log_start":"102'5300","ondisk_log_start":"102'5300","created":80,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"122'10699","last_scrub_stamp":"2024-09-20T22:06:00.331499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.331499+0000","objects_scrubbed":2914,"log_size":5524,"log_dups_size":0,"ondisk_log_size":5524,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:29.358323+0000","scrub_duration":784,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":181927936,"num_objects":2776,"num_object_clones":0,"num_object_copies":8328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":138,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"71'16","reported_seq":209,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.491809+0000","last_change":"2024-09-20T22:05:58.335330+0000","last_active":"2024-09-20T22:06:03.491809+0000","last_peered":"2024-09-20T22:06:03.491809+0000","last_clean":"2024-09-20T22:06:03.491809+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:06:03.491809+0000","last_undegraded":"2024-09-20T22:06:03.491809+0000","last_fullsized":"2024-09-20T22:06:03.491809+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'16","last_scrub_stamp":"2024-09-20T22:05:58.335199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:58.335199+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:02.188245+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":143,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.505180+0000","last_change":"2024-09-20T22:04:11.743850+0000","last_active":"2024-09-20T22:06:03.505180+0000","last_peered":"2024-09-20T22:06:03.505180+0000","last_clean":"2024-09-20T22:06:03.505180+0000","last_became_active":"2024-09-20T22:02:50.106011+0000","last_became_peered":"2024-09-20T22:02:50.106011+0000","last_unstale":"2024-09-20T22:06:03.505180+0000","last_undegraded":"2024-09-20T22:06:03.505180+0000","last_fullsized":"2024-09-20T22:06:03.505180+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:11.743751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:11.743751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":399,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.500234+0000","last_change":"2024-09-20T22:05:02.727025+0000","last_active":"2024-09-20T22:06:03.500234+0000","last_peered":"2024-09-20T22:06:03.500234+0000","last_clean":"2024-09-20T22:06:03.500234+0000","last_became_active":"2024-09-20T22:02:45.068482+0000","last_became_peered":"2024-09-20T22:02:45.068482+0000","last_unstale":"2024-09-20T22:06:03.500234+0000","last_undegraded":"2024-09-20T22:06:03.500234+0000","last_fullsized":"2024-09-20T22:06:03.500234+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:05:02.726765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:05:02.726765+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:21.401986+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":144,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.500872+0000","last_change":"2024-09-20T22:05:56.679105+0000","last_active":"2024-09-20T22:06:03.500872+0000","last_peered":"2024-09-20T22:06:03.500872+0000","last_clean":"2024-09-20T22:06:03.500872+0000","last_became_active":"2024-09-20T22:02:54.298021+0000","last_became_peered":"2024-09-20T22:02:54.298021+0000","last_unstale":"2024-09-20T22:06:03.500872+0000","last_undegraded":"2024-09-20T22:06:03.500872+0000","last_fullsized":"2024-09-20T22:06:03.500872+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:56.678980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:56.678980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:19.526891+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":145,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.495412+0000","last_change":"2024-09-20T22:05:27.659354+0000","last_active":"2024-09-20T22:06:03.495412+0000","last_peered":"2024-09-20T22:06:03.495412+0000","last_clean":"2024-09-20T22:06:03.495412+0000","last_became_active":"2024-09-20T22:04:21.237725+0000","last_became_peered":"2024-09-20T22:04:21.237725+0000","last_unstale":"2024-09-20T22:06:03.495412+0000","last_undegraded":"2024-09-20T22:06:03.495412+0000","last_fullsized":"2024-09-20T22:06:03.495412+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:05:27.659252+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:40.379417+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"126'10828","reported_seq":10937,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:05.642271+0000","last_change":"2024-09-20T22:04:16.319236+0000","last_active":"2024-09-20T22:06:05.642271+0000","last_peered":"2024-09-20T22:06:05.642271+0000","last_clean":"2024-09-20T22:04:15.698310+0000","last_became_active":"2024-09-20T22:04:16.319236+0000","last_became_peered":"2024-09-20T22:04:16.319236+0000","last_unstale":"2024-09-20T22:06:05.642271+0000","last_undegraded":"2024-09-20T22:04:16.305688+0000","last_fullsized":"2024-09-20T22:04:16.305555+0000","mapping_epoch":91,"log_start":"103'5339","ondisk_log_start":"103'5339","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":5489,"log_dups_size":0,"ondisk_log_size":5489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:06.346733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":364052504,"num_objects":5556,"num_object_clones":0,"num_object_copies":16668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5556,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5556,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10828,"num_write_kb":524225,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[{"shards":"0,6","objects":5556}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"71'11","reported_seq":154,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.498976+0000","last_change":"2024-09-20T22:04:16.320499+0000","last_active":"2024-09-20T22:06:03.498976+0000","last_peered":"2024-09-20T22:06:03.498976+0000","last_clean":"2024-09-20T22:04:11.303901+0000","last_became_active":"2024-09-20T22:04:16.320499+0000","last_became_peered":"2024-09-20T22:04:16.320499+0000","last_unstale":"2024-09-20T22:06:03.498976+0000","last_undegraded":"2024-09-20T22:04:16.305535+0000","last_fullsized":"2024-09-20T22:04:16.305436+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[{"shards":"4,6","objects":5}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":144,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.498516+0000","last_change":"2024-09-20T22:05:26.006756+0000","last_active":"2024-09-20T22:06:03.498516+0000","last_peered":"2024-09-20T22:06:03.498516+0000","last_clean":"2024-09-20T22:06:03.498516+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:06:03.498516+0000","last_undegraded":"2024-09-20T22:06:03.498516+0000","last_fullsized":"2024-09-20T22:06:03.498516+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:26.006692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:26.006692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:49.724396+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":143,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.495303+0000","last_change":"2024-09-20T22:05:57.397917+0000","last_active":"2024-09-20T22:06:03.495303+0000","last_peered":"2024-09-20T22:06:03.495303+0000","last_clean":"2024-09-20T22:06:03.495303+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:06:03.495303+0000","last_undegraded":"2024-09-20T22:06:03.495303+0000","last_fullsized":"2024-09-20T22:06:03.495303+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:57.397807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:57.397807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:19.095166+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":131,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.509625+0000","last_change":"2024-09-20T22:04:17.783161+0000","last_active":"2024-09-20T22:06:03.509625+0000","last_peered":"2024-09-20T22:06:03.509625+0000","last_clean":"2024-09-20T22:06:03.509625+0000","last_became_active":"2024-09-20T22:02:56.349376+0000","last_became_peered":"2024-09-20T22:02:56.349376+0000","last_unstale":"2024-09-20T22:06:03.509625+0000","last_undegraded":"2024-09-20T22:06:03.509625+0000","last_fullsized":"2024-09-20T22:06:03.509625+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:17.783059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:17.783059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.e","version":"126'10882","reported_seq":11132,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:05.789447+0000","last_change":"2024-09-20T22:05:27.949351+0000","last_active":"2024-09-20T22:06:05.789447+0000","last_peered":"2024-09-20T22:06:05.789447+0000","last_clean":"2024-09-20T22:06:05.789447+0000","last_became_active":"2024-09-20T22:03:52.311982+0000","last_became_peered":"2024-09-20T22:03:52.311982+0000","last_unstale":"2024-09-20T22:06:05.789447+0000","last_undegraded":"2024-09-20T22:06:05.789447+0000","last_fullsized":"2024-09-20T22:06:05.789447+0000","mapping_epoch":80,"log_start":"102'4973","ondisk_log_start":"102'4973","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"112'8343","last_scrub_stamp":"2024-09-20T22:05:27.949070+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:05:27.949070+0000","objects_scrubbed":8239,"log_size":5909,"log_dups_size":0,"ondisk_log_size":5909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:52.098447+0000","scrub_duration":3181,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":373948416,"num_objects":5706,"num_object_clones":0,"num_object_copies":17118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10882,"num_write_kb":530816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"71'17","reported_seq":171,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.498770+0000","last_change":"2024-09-20T22:04:16.312435+0000","last_active":"2024-09-20T22:06:03.498770+0000","last_peered":"2024-09-20T22:06:03.498770+0000","last_clean":"2024-09-20T22:04:12.942923+0000","last_became_active":"2024-09-20T22:04:16.312435+0000","last_became_peered":"2024-09-20T22:04:16.312435+0000","last_unstale":"2024-09-20T22:06:03.498770+0000","last_undegraded":"2024-09-20T22:04:16.305517+0000","last_fullsized":"2024-09-20T22:04:16.305404+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'17","last_scrub_stamp":"2024-09-20T22:04:12.942792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:12.942792+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:03.964579+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[{"shards":"2,4","objects":8}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":138,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.491141+0000","last_change":"2024-09-20T22:04:16.319111+0000","last_active":"2024-09-20T22:06:03.491141+0000","last_peered":"2024-09-20T22:06:03.491141+0000","last_clean":"2024-09-20T22:04:11.306602+0000","last_became_active":"2024-09-20T22:04:16.319111+0000","last_became_peered":"2024-09-20T22:04:16.319111+0000","last_unstale":"2024-09-20T22:06:03.491141+0000","last_undegraded":"2024-09-20T22:06:03.491141+0000","last_fullsized":"2024-09-20T22:04:16.305556+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":["6","4"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":77,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.501662+0000","last_change":"2024-09-20T22:04:16.310599+0000","last_active":"2024-09-20T22:06:03.501662+0000","last_peered":"2024-09-20T22:06:03.501662+0000","last_clean":"2024-09-20T22:02:53.269054+0000","last_became_active":"2024-09-20T22:04:16.310599+0000","last_became_peered":"2024-09-20T22:04:16.310599+0000","last_unstale":"2024-09-20T22:06:03.501662+0000","last_undegraded":"2024-09-20T22:06:03.501662+0000","last_fullsized":"2024-09-20T22:04:16.304159+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":["4","0"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":131,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.500684+0000","last_change":"2024-09-20T22:04:54.986448+0000","last_active":"2024-09-20T22:06:03.500684+0000","last_peered":"2024-09-20T22:06:03.500684+0000","last_clean":"2024-09-20T22:06:03.500684+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:06:03.500684+0000","last_undegraded":"2024-09-20T22:06:03.500684+0000","last_fullsized":"2024-09-20T22:06:03.500684+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:54.986284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:54.986284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"126'10785","reported_seq":10890,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:05.765004+0000","last_change":"2024-09-20T22:03:52.317134+0000","last_active":"2024-09-20T22:06:05.765004+0000","last_peered":"2024-09-20T22:06:05.765004+0000","last_clean":"2024-09-20T22:06:05.765004+0000","last_became_active":"2024-09-20T22:03:52.316825+0000","last_became_peered":"2024-09-20T22:03:52.316825+0000","last_unstale":"2024-09-20T22:06:05.765004+0000","last_undegraded":"2024-09-20T22:06:05.765004+0000","last_fullsized":"2024-09-20T22:06:05.765004+0000","mapping_epoch":80,"log_start":"107'6145","ondisk_log_start":"107'6145","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":4640,"log_dups_size":0,"ondisk_log_size":4640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":363397120,"num_objects":5545,"num_object_clones":0,"num_object_copies":16635,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10785,"num_write_kb":522560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"71'10","reported_seq":155,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.498700+0000","last_change":"2024-09-20T22:05:27.048378+0000","last_active":"2024-09-20T22:06:03.498700+0000","last_peered":"2024-09-20T22:06:03.498700+0000","last_clean":"2024-09-20T22:06:03.498700+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:06:03.498700+0000","last_undegraded":"2024-09-20T22:06:03.498700+0000","last_fullsized":"2024-09-20T22:06:03.498700+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:05:27.048215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:27.048215+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:51.977225+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"64'1","reported_seq":149,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.505288+0000","last_change":"2024-09-20T22:05:27.895711+0000","last_active":"2024-09-20T22:06:03.505288+0000","last_peered":"2024-09-20T22:06:03.505288+0000","last_clean":"2024-09-20T22:06:03.505288+0000","last_became_active":"2024-09-20T22:02:50.110446+0000","last_became_peered":"2024-09-20T22:02:50.110446+0000","last_unstale":"2024-09-20T22:06:03.505288+0000","last_undegraded":"2024-09-20T22:06:03.505288+0000","last_fullsized":"2024-09-20T22:06:03.505288+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:05:27.895595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:27.895595+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:33.193947+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"71'8","reported_seq":191,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.497925+0000","last_change":"2024-09-20T22:05:30.744134+0000","last_active":"2024-09-20T22:06:03.497925+0000","last_peered":"2024-09-20T22:06:03.497925+0000","last_clean":"2024-09-20T22:06:03.497925+0000","last_became_active":"2024-09-20T22:02:54.294458+0000","last_became_peered":"2024-09-20T22:02:54.294458+0000","last_unstale":"2024-09-20T22:06:03.497925+0000","last_undegraded":"2024-09-20T22:06:03.497925+0000","last_fullsized":"2024-09-20T22:06:03.497925+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:05:30.743022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:30.743022+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:38.500958+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":127,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.498790+0000","last_change":"2024-09-20T22:04:16.312842+0000","last_active":"2024-09-20T22:06:03.498790+0000","last_peered":"2024-09-20T22:06:03.498790+0000","last_clean":"2024-09-20T22:04:11.303632+0000","last_became_active":"2024-09-20T22:04:16.312842+0000","last_became_peered":"2024-09-20T22:04:16.312842+0000","last_unstale":"2024-09-20T22:06:03.498790+0000","last_undegraded":"2024-09-20T22:06:03.498790+0000","last_fullsized":"2024-09-20T22:04:16.305087+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":["4","2"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.8","version":"126'10973","reported_seq":11196,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:05.796164+0000","last_change":"2024-09-20T22:06:01.437051+0000","last_active":"2024-09-20T22:06:05.796164+0000","last_peered":"2024-09-20T22:06:05.796164+0000","last_clean":"2024-09-20T22:06:05.796164+0000","last_became_active":"2024-09-20T22:05:59.476491+0000","last_became_peered":"2024-09-20T22:05:59.476491+0000","last_unstale":"2024-09-20T22:06:05.796164+0000","last_undegraded":"2024-09-20T22:06:05.796164+0000","last_fullsized":"2024-09-20T22:06:05.796164+0000","mapping_epoch":80,"log_start":"98'3800","ondisk_log_start":"98'3800","created":80,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"122'10858","last_scrub_stamp":"2024-09-20T22:06:01.436936+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:06:01.436936+0000","objects_scrubbed":2956,"log_size":7173,"log_dups_size":0,"ondisk_log_size":7173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:30.059324+0000","scrub_duration":328,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":186187776,"num_objects":2841,"num_object_clones":0,"num_object_copies":8523,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"71'9","reported_seq":158,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.492936+0000","last_change":"2024-09-20T22:04:16.319135+0000","last_active":"2024-09-20T22:06:03.492936+0000","last_peered":"2024-09-20T22:06:03.492936+0000","last_clean":"2024-09-20T22:04:11.307387+0000","last_became_active":"2024-09-20T22:04:16.319135+0000","last_became_peered":"2024-09-20T22:04:16.319135+0000","last_unstale":"2024-09-20T22:06:03.492936+0000","last_undegraded":"2024-09-20T22:04:16.304006+0000","last_fullsized":"2024-09-20T22:04:16.303888+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:03:52.011201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:03:52.011201+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[{"shards":"0,4","objects":4}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":149,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.494768+0000","last_change":"2024-09-20T22:04:57.472466+0000","last_active":"2024-09-20T22:06:03.494768+0000","last_peered":"2024-09-20T22:06:03.494768+0000","last_clean":"2024-09-20T22:06:03.494768+0000","last_became_active":"2024-09-20T22:02:50.112682+0000","last_became_peered":"2024-09-20T22:02:50.112682+0000","last_unstale":"2024-09-20T22:06:03.494768+0000","last_undegraded":"2024-09-20T22:06:03.494768+0000","last_fullsized":"2024-09-20T22:06:03.494768+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:04:57.472316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:57.472316+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:25.283931+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":143,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.499952+0000","last_change":"2024-09-20T22:06:02.072685+0000","last_active":"2024-09-20T22:06:03.499952+0000","last_peered":"2024-09-20T22:06:03.499952+0000","last_clean":"2024-09-20T22:06:03.499952+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:06:03.499952+0000","last_undegraded":"2024-09-20T22:06:03.499952+0000","last_fullsized":"2024-09-20T22:06:03.499952+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:02.072607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:02.072607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:21.069580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":136,"reported_epoch":126,"state":"active+clean+remapped","last_fresh":"2024-09-20T22:06:03.493600+0000","last_change":"2024-09-20T22:04:21.326390+0000","last_active":"2024-09-20T22:06:03.493600+0000","last_peered":"2024-09-20T22:06:03.493600+0000","last_clean":"2024-09-20T22:06:03.493600+0000","last_became_active":"2024-09-20T22:04:21.325835+0000","last_became_peered":"2024-09-20T22:04:21.325835+0000","last_unstale":"2024-09-20T22:06:03.493600+0000","last_undegraded":"2024-09-20T22:06:03.493600+0000","last_fullsized":"2024-09-20T22:06:03.493600+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:17.171523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:17.171523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2,7],"avail_no_missing":["0","2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"126'10715","reported_seq":10824,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:05.783277+0000","last_change":"2024-09-20T22:05:59.474094+0000","last_active":"2024-09-20T22:06:05.783277+0000","last_peered":"2024-09-20T22:06:05.783277+0000","last_clean":"2024-09-20T22:04:15.706494+0000","last_became_active":"2024-09-20T22:05:59.474094+0000","last_became_peered":"2024-09-20T22:05:59.474094+0000","last_unstale":"2024-09-20T22:06:05.783277+0000","last_undegraded":"2024-09-20T22:05:59.447549+0000","last_fullsized":"2024-09-20T22:05:59.447430+0000","mapping_epoch":91,"log_start":"102'5200","ondisk_log_start":"102'5200","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":5515,"log_dups_size":0,"ondisk_log_size":5515,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:10.652689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":189005824,"num_objects":2884,"num_object_clones":0,"num_object_copies":8652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2884,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5439,"num_write_kb":266336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[{"shards":"0,4","objects":2884}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"71'19","reported_seq":165,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.490640+0000","last_change":"2024-09-20T22:04:16.318494+0000","last_active":"2024-09-20T22:06:03.490640+0000","last_peered":"2024-09-20T22:06:03.490640+0000","last_clean":"2024-09-20T22:04:11.306486+0000","last_became_active":"2024-09-20T22:04:16.318494+0000","last_became_peered":"2024-09-20T22:04:16.318494+0000","last_unstale":"2024-09-20T22:06:03.490640+0000","last_undegraded":"2024-09-20T22:04:16.306185+0000","last_fullsized":"2024-09-20T22:04:16.306074+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[{"shards":"6,7","objects":9}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":144,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.500865+0000","last_change":"2024-09-20T22:05:05.659544+0000","last_active":"2024-09-20T22:06:03.500865+0000","last_peered":"2024-09-20T22:06:03.500865+0000","last_clean":"2024-09-20T22:06:03.500865+0000","last_became_active":"2024-09-20T22:02:50.108712+0000","last_became_peered":"2024-09-20T22:02:50.108712+0000","last_unstale":"2024-09-20T22:06:03.500865+0000","last_undegraded":"2024-09-20T22:06:03.500865+0000","last_fullsized":"2024-09-20T22:06:03.500865+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:05.659437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:05.659437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:25.341327+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":143,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.498528+0000","last_change":"2024-09-20T22:05:31.802337+0000","last_active":"2024-09-20T22:06:03.498528+0000","last_peered":"2024-09-20T22:06:03.498528+0000","last_clean":"2024-09-20T22:06:03.498528+0000","last_became_active":"2024-09-20T22:02:54.291134+0000","last_became_peered":"2024-09-20T22:02:54.291134+0000","last_unstale":"2024-09-20T22:06:03.498528+0000","last_undegraded":"2024-09-20T22:06:03.498528+0000","last_fullsized":"2024-09-20T22:06:03.498528+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:31.802238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:31.802238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:53.108551+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":132,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.500143+0000","last_change":"2024-09-20T22:04:59.653059+0000","last_active":"2024-09-20T22:06:03.500143+0000","last_peered":"2024-09-20T22:06:03.500143+0000","last_clean":"2024-09-20T22:06:03.500143+0000","last_became_active":"2024-09-20T22:02:56.307687+0000","last_became_peered":"2024-09-20T22:02:56.307687+0000","last_unstale":"2024-09-20T22:06:03.500143+0000","last_undegraded":"2024-09-20T22:06:03.500143+0000","last_fullsized":"2024-09-20T22:06:03.500143+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:59.652843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:59.652843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:18.749869+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"126'11033","reported_seq":11206,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:05.833918+0000","last_change":"2024-09-20T22:06:03.160538+0000","last_active":"2024-09-20T22:06:05.833918+0000","last_peered":"2024-09-20T22:06:05.833918+0000","last_clean":"2024-09-20T22:06:05.833918+0000","last_became_active":"2024-09-20T22:03:52.316211+0000","last_became_peered":"2024-09-20T22:03:52.316211+0000","last_unstale":"2024-09-20T22:06:05.833918+0000","last_undegraded":"2024-09-20T22:06:05.833918+0000","last_fullsized":"2024-09-20T22:06:05.833918+0000","mapping_epoch":80,"log_start":"108'6736","ondisk_log_start":"108'6736","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"125'10809","last_scrub_stamp":"2024-09-20T22:06:03.160287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:06:03.160287+0000","objects_scrubbed":5957,"log_size":4297,"log_dups_size":0,"ondisk_log_size":4297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:15.768132+0000","scrub_duration":1234,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":373358592,"num_objects":5697,"num_object_clones":0,"num_object_copies":17091,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11033,"num_write_kb":535360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"71'12","reported_seq":158,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.498081+0000","last_change":"2024-09-20T22:04:16.324704+0000","last_active":"2024-09-20T22:06:03.498081+0000","last_peered":"2024-09-20T22:06:03.498081+0000","last_clean":"2024-09-20T22:04:11.303078+0000","last_became_active":"2024-09-20T22:04:16.324704+0000","last_became_peered":"2024-09-20T22:04:16.324704+0000","last_unstale":"2024-09-20T22:06:03.498081+0000","last_undegraded":"2024-09-20T22:04:16.303915+0000","last_fullsized":"2024-09-20T22:04:16.303791+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[{"shards":"3,4","objects":5}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":139,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.500500+0000","last_change":"2024-09-20T22:04:16.316858+0000","last_active":"2024-09-20T22:06:03.500500+0000","last_peered":"2024-09-20T22:06:03.500500+0000","last_clean":"2024-09-20T22:04:11.309387+0000","last_became_active":"2024-09-20T22:04:16.316858+0000","last_became_peered":"2024-09-20T22:04:16.316858+0000","last_unstale":"2024-09-20T22:06:03.500500+0000","last_undegraded":"2024-09-20T22:06:03.500500+0000","last_fullsized":"2024-09-20T22:04:16.309391+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:06.619534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":138,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.497118+0000","last_change":"2024-09-20T22:04:16.315094+0000","last_active":"2024-09-20T22:06:03.497118+0000","last_peered":"2024-09-20T22:06:03.497118+0000","last_clean":"2024-09-20T22:04:12.729887+0000","last_became_active":"2024-09-20T22:04:16.315094+0000","last_became_peered":"2024-09-20T22:04:16.315094+0000","last_unstale":"2024-09-20T22:06:03.497118+0000","last_undegraded":"2024-09-20T22:06:03.497118+0000","last_fullsized":"2024-09-20T22:04:16.304906+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:12.729789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:12.729789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:07.071098+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":80,"reported_epoch":126,"state":"active+clean+remapped","last_fresh":"2024-09-20T22:06:03.495318+0000","last_change":"2024-09-20T22:05:28.569254+0000","last_active":"2024-09-20T22:06:03.495318+0000","last_peered":"2024-09-20T22:06:03.495318+0000","last_clean":"2024-09-20T22:06:03.495318+0000","last_became_active":"2024-09-20T22:04:21.325506+0000","last_became_peered":"2024-09-20T22:04:21.325506+0000","last_unstale":"2024-09-20T22:06:03.495318+0000","last_undegraded":"2024-09-20T22:06:03.495318+0000","last_fullsized":"2024-09-20T22:06:03.495318+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.569045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.569045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:34.983641+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7,3],"avail_no_missing":["0","3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.b","version":"126'10909","reported_seq":11017,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:05.567033+0000","last_change":"2024-09-20T22:04:16.321983+0000","last_active":"2024-09-20T22:06:05.567033+0000","last_peered":"2024-09-20T22:06:05.567033+0000","last_clean":"2024-09-20T22:04:15.686857+0000","last_became_active":"2024-09-20T22:04:16.321983+0000","last_became_peered":"2024-09-20T22:04:16.321983+0000","last_unstale":"2024-09-20T22:06:05.567033+0000","last_undegraded":"2024-09-20T22:04:16.309082+0000","last_fullsized":"2024-09-20T22:04:16.308976+0000","mapping_epoch":91,"log_start":"103'5324","ondisk_log_start":"103'5324","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":5585,"log_dups_size":0,"ondisk_log_size":5585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:10.348541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":370343936,"num_objects":5651,"num_object_clones":0,"num_object_copies":16953,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10909,"num_write_kb":529920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[{"shards":"5,6","objects":5651}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"71'15","reported_seq":164,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.494000+0000","last_change":"2024-09-20T22:05:01.515339+0000","last_active":"2024-09-20T22:06:03.494000+0000","last_peered":"2024-09-20T22:06:03.494000+0000","last_clean":"2024-09-20T22:06:03.494000+0000","last_became_active":"2024-09-20T22:02:52.482008+0000","last_became_peered":"2024-09-20T22:02:52.482008+0000","last_unstale":"2024-09-20T22:06:03.494000+0000","last_undegraded":"2024-09-20T22:06:03.494000+0000","last_fullsized":"2024-09-20T22:06:03.494000+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:05:01.515075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:01.515075+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:28.570514+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"64'2","reported_seq":156,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.500858+0000","last_change":"2024-09-20T22:05:00.690608+0000","last_active":"2024-09-20T22:06:03.500858+0000","last_peered":"2024-09-20T22:06:03.500858+0000","last_clean":"2024-09-20T22:06:03.500858+0000","last_became_active":"2024-09-20T22:02:50.114819+0000","last_became_peered":"2024-09-20T22:02:50.114819+0000","last_unstale":"2024-09-20T22:06:03.500858+0000","last_undegraded":"2024-09-20T22:06:03.500858+0000","last_fullsized":"2024-09-20T22:06:03.500858+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'2","last_scrub_stamp":"2024-09-20T22:05:00.690333+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:00.690333+0000","objects_scrubbed":2,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":184,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.499749+0000","last_change":"2024-09-20T22:05:34.732761+0000","last_active":"2024-09-20T22:06:03.499749+0000","last_peered":"2024-09-20T22:06:03.499749+0000","last_clean":"2024-09-20T22:06:03.499749+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:06:03.499749+0000","last_undegraded":"2024-09-20T22:06:03.499749+0000","last_fullsized":"2024-09-20T22:06:03.499749+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:05:34.732644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:34.732644+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:59.614780+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":131,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.494045+0000","last_change":"2024-09-20T22:05:00.532426+0000","last_active":"2024-09-20T22:06:03.494045+0000","last_peered":"2024-09-20T22:06:03.494045+0000","last_clean":"2024-09-20T22:06:03.494045+0000","last_became_active":"2024-09-20T22:02:56.346990+0000","last_became_peered":"2024-09-20T22:02:56.346990+0000","last_unstale":"2024-09-20T22:06:03.494045+0000","last_undegraded":"2024-09-20T22:06:03.494045+0000","last_fullsized":"2024-09-20T22:06:03.494045+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:00.532303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:00.532303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:24.617909+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.14","version":"126'10782","reported_seq":10902,"reported_epoch":126,"state":"active+undersized+degraded+remapped+backfill_toofull","last_fresh":"2024-09-20T22:06:05.792310+0000","last_change":"2024-09-20T22:06:02.523900+0000","last_active":"2024-09-20T22:06:05.792310+0000","last_peered":"2024-09-20T22:06:05.792310+0000","last_clean":"2024-09-20T22:04:15.698370+0000","last_became_active":"2024-09-20T22:06:02.467069+0000","last_became_peered":"2024-09-20T22:06:02.467069+0000","last_unstale":"2024-09-20T22:06:05.792310+0000","last_undegraded":"2024-09-20T22:06:02.509391+0000","last_fullsized":"2024-09-20T22:06:02.342126+0000","mapping_epoch":124,"log_start":"102'5300","ondisk_log_start":"102'5300","created":121,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":5482,"log_dups_size":0,"ondisk_log_size":5482,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184254464,"num_objects":2812,"num_object_clones":0,"num_object_copies":8436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2812,"num_objects_misplaced":5624,"num_objects_unfound":0,"num_objects_dirty":2812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5486,"num_write_kb":265504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[{"shards":"0,4","objects":2812}],"blocked_by":[],"up_primary":3,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":143,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.490567+0000","last_change":"2024-09-20T22:04:59.377318+0000","last_active":"2024-09-20T22:06:03.490567+0000","last_peered":"2024-09-20T22:06:03.490567+0000","last_clean":"2024-09-20T22:06:03.490567+0000","last_became_active":"2024-09-20T22:02:50.111935+0000","last_became_peered":"2024-09-20T22:02:50.111935+0000","last_unstale":"2024-09-20T22:06:03.490567+0000","last_undegraded":"2024-09-20T22:06:03.490567+0000","last_fullsized":"2024-09-20T22:06:03.490567+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:59.377203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:59.377203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:18.963527+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"71'6","reported_seq":143,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.513259+0000","last_change":"2024-09-20T22:04:16.318583+0000","last_active":"2024-09-20T22:06:03.513259+0000","last_peered":"2024-09-20T22:06:03.513259+0000","last_clean":"2024-09-20T22:04:11.314508+0000","last_became_active":"2024-09-20T22:04:16.318583+0000","last_became_peered":"2024-09-20T22:04:16.318583+0000","last_unstale":"2024-09-20T22:06:03.513259+0000","last_undegraded":"2024-09-20T22:04:16.312022+0000","last_fullsized":"2024-09-20T22:04:16.311757+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:06.925518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[{"shards":"3,7","objects":3}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":130,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.495134+0000","last_change":"2024-09-20T22:04:16.320696+0000","last_active":"2024-09-20T22:06:03.495134+0000","last_peered":"2024-09-20T22:06:03.495134+0000","last_clean":"2024-09-20T22:04:11.311311+0000","last_became_active":"2024-09-20T22:04:16.320696+0000","last_became_peered":"2024-09-20T22:04:16.320696+0000","last_unstale":"2024-09-20T22:06:03.495134+0000","last_undegraded":"2024-09-20T22:06:03.495134+0000","last_fullsized":"2024-09-20T22:04:16.311423+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":124,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.500414+0000","last_change":"2024-09-20T22:02:56.350065+0000","last_active":"2024-09-20T22:06:03.500414+0000","last_peered":"2024-09-20T22:06:03.500414+0000","last_clean":"2024-09-20T22:06:03.500414+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:06:03.500414+0000","last_undegraded":"2024-09-20T22:06:03.500414+0000","last_fullsized":"2024-09-20T22:06:03.500414+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"126'10800","reported_seq":10957,"reported_epoch":126,"state":"active+clean+remapped","last_fresh":"2024-09-20T22:06:05.521023+0000","last_change":"2024-09-20T22:06:02.383199+0000","last_active":"2024-09-20T22:06:05.521023+0000","last_peered":"2024-09-20T22:06:05.521023+0000","last_clean":"2024-09-20T22:06:05.521023+0000","last_became_active":"2024-09-20T22:06:02.355017+0000","last_became_peered":"2024-09-20T22:06:02.355017+0000","last_unstale":"2024-09-20T22:06:05.521023+0000","last_undegraded":"2024-09-20T22:06:05.521023+0000","last_fullsized":"2024-09-20T22:06:05.521023+0000","mapping_epoch":124,"log_start":"103'5400","ondisk_log_start":"103'5400","created":121,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":2649,"log_size":5400,"log_dups_size":0,"ondisk_log_size":5400,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:07.640592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187760640,"num_objects":2865,"num_object_clones":0,"num_object_copies":8595,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2865,"num_objects_unfound":0,"num_objects_dirty":2865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5473,"num_write_kb":266816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2,0],"avail_no_missing":["7","0","2"],"object_location_counts":[{"shards":"0,2,7","objects":2865}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"71'9","reported_seq":155,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.505688+0000","last_change":"2024-09-20T22:04:52.254864+0000","last_active":"2024-09-20T22:06:03.505688+0000","last_peered":"2024-09-20T22:06:03.505688+0000","last_clean":"2024-09-20T22:06:03.505688+0000","last_became_active":"2024-09-20T22:02:52.481989+0000","last_became_peered":"2024-09-20T22:02:52.481989+0000","last_unstale":"2024-09-20T22:06:03.505688+0000","last_undegraded":"2024-09-20T22:06:03.505688+0000","last_fullsized":"2024-09-20T22:06:03.505688+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:52.254706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:52.254706+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:14.729595+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":144,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.500008+0000","last_change":"2024-09-20T22:05:31.764903+0000","last_active":"2024-09-20T22:06:03.500008+0000","last_peered":"2024-09-20T22:06:03.500008+0000","last_clean":"2024-09-20T22:06:03.500008+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:06:03.500008+0000","last_undegraded":"2024-09-20T22:06:03.500008+0000","last_fullsized":"2024-09-20T22:06:03.500008+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:31.764833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:31.764833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:55.238401+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":130,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.513063+0000","last_change":"2024-09-20T22:04:16.316683+0000","last_active":"2024-09-20T22:06:03.513063+0000","last_peered":"2024-09-20T22:06:03.513063+0000","last_clean":"2024-09-20T22:04:11.314344+0000","last_became_active":"2024-09-20T22:04:16.316683+0000","last_became_peered":"2024-09-20T22:04:16.316683+0000","last_unstale":"2024-09-20T22:06:03.513063+0000","last_undegraded":"2024-09-20T22:06:03.513063+0000","last_fullsized":"2024-09-20T22:04:16.308930+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:06.925472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":131,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.496647+0000","last_change":"2024-09-20T22:04:52.008397+0000","last_active":"2024-09-20T22:06:03.496647+0000","last_peered":"2024-09-20T22:06:03.496647+0000","last_clean":"2024-09-20T22:06:03.496647+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:06:03.496647+0000","last_undegraded":"2024-09-20T22:06:03.496647+0000","last_fullsized":"2024-09-20T22:06:03.496647+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:52.008283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:52.008283+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:14.892148+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.16","version":"126'10829","reported_seq":10945,"reported_epoch":126,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T22:06:05.789134+0000","last_change":"2024-09-20T22:06:02.448582+0000","last_active":"2024-09-20T22:06:05.789134+0000","last_peered":"2024-09-20T22:06:05.789134+0000","last_clean":"2024-09-20T22:06:00.443191+0000","last_became_active":"2024-09-20T22:06:02.394187+0000","last_became_peered":"2024-09-20T22:06:02.394187+0000","last_unstale":"2024-09-20T22:06:05.789134+0000","last_undegraded":"2024-09-20T22:06:05.789134+0000","last_fullsized":"2024-09-20T22:06:05.789134+0000","mapping_epoch":124,"log_start":"102'5300","ondisk_log_start":"102'5300","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":5529,"log_dups_size":0,"ondisk_log_size":5529,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184418304,"num_objects":2814,"num_object_clones":0,"num_object_copies":8442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2814,"num_objects_unfound":0,"num_objects_dirty":2814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5500,"num_write_kb":266048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[0,7,3],"avail_no_missing":["0","3","7"],"object_location_counts":[{"shards":"0,3,7","objects":2814}],"blocked_by":[],"up_primary":4,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"71'9","reported_seq":155,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.495110+0000","last_change":"2024-09-20T22:04:50.514559+0000","last_active":"2024-09-20T22:06:03.495110+0000","last_peered":"2024-09-20T22:06:03.495110+0000","last_clean":"2024-09-20T22:06:03.495110+0000","last_became_active":"2024-09-20T22:02:52.482141+0000","last_became_peered":"2024-09-20T22:02:52.482141+0000","last_unstale":"2024-09-20T22:06:03.495110+0000","last_undegraded":"2024-09-20T22:06:03.495110+0000","last_fullsized":"2024-09-20T22:06:03.495110+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:50.514410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:50.514410+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":143,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.492294+0000","last_change":"2024-09-20T22:04:13.226031+0000","last_active":"2024-09-20T22:06:03.492294+0000","last_peered":"2024-09-20T22:06:03.492294+0000","last_clean":"2024-09-20T22:06:03.492294+0000","last_became_active":"2024-09-20T22:02:50.110075+0000","last_became_peered":"2024-09-20T22:02:50.110075+0000","last_unstale":"2024-09-20T22:06:03.492294+0000","last_undegraded":"2024-09-20T22:06:03.492294+0000","last_fullsized":"2024-09-20T22:06:03.492294+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:13.225925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:13.225925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"71'8","reported_seq":183,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.509802+0000","last_change":"2024-09-20T22:05:55.972684+0000","last_active":"2024-09-20T22:06:03.509802+0000","last_peered":"2024-09-20T22:06:03.509802+0000","last_clean":"2024-09-20T22:06:03.509802+0000","last_became_active":"2024-09-20T22:02:54.294381+0000","last_became_peered":"2024-09-20T22:02:54.294381+0000","last_unstale":"2024-09-20T22:06:03.509802+0000","last_undegraded":"2024-09-20T22:06:03.509802+0000","last_fullsized":"2024-09-20T22:06:03.509802+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:05:55.972565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:55.972565+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:04.374900+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":132,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.499234+0000","last_change":"2024-09-20T22:05:00.897974+0000","last_active":"2024-09-20T22:06:03.499234+0000","last_peered":"2024-09-20T22:06:03.499234+0000","last_clean":"2024-09-20T22:06:03.499234+0000","last_became_active":"2024-09-20T22:02:56.300041+0000","last_became_peered":"2024-09-20T22:02:56.300041+0000","last_unstale":"2024-09-20T22:06:03.499234+0000","last_undegraded":"2024-09-20T22:06:03.499234+0000","last_fullsized":"2024-09-20T22:06:03.499234+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:00.897869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:00.897869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:08.153745+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"126'10830","reported_seq":11038,"reported_epoch":126,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T22:06:05.792822+0000","last_change":"2024-09-20T22:06:02.509509+0000","last_active":"2024-09-20T22:06:05.792822+0000","last_peered":"2024-09-20T22:06:05.792822+0000","last_clean":"2024-09-20T22:06:00.337539+0000","last_became_active":"2024-09-20T22:06:02.480114+0000","last_became_peered":"2024-09-20T22:06:02.480114+0000","last_unstale":"2024-09-20T22:06:05.792822+0000","last_undegraded":"2024-09-20T22:06:05.792822+0000","last_fullsized":"2024-09-20T22:06:05.792822+0000","mapping_epoch":124,"log_start":"99'4600","ondisk_log_start":"99'4600","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":5,"last_scrub":"112'8336","last_scrub_stamp":"2024-09-20T22:05:27.437852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:05:27.437852+0000","objects_scrubbed":8288,"log_size":6230,"log_dups_size":0,"ondisk_log_size":6230,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2305,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184647680,"num_objects":2818,"num_object_clones":0,"num_object_copies":8454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":8454,"num_objects_unfound":0,"num_objects_dirty":2818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5509,"num_write_kb":266432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[0,3,6],"avail_no_missing":["0","3","6"],"object_location_counts":[{"shards":"0,3,6","objects":2818}],"blocked_by":[],"up_primary":7,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"71'10","reported_seq":149,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.509901+0000","last_change":"2024-09-20T22:04:16.314377+0000","last_active":"2024-09-20T22:06:03.509901+0000","last_peered":"2024-09-20T22:06:03.509901+0000","last_clean":"2024-09-20T22:04:11.310002+0000","last_became_active":"2024-09-20T22:04:16.314377+0000","last_became_peered":"2024-09-20T22:04:16.314377+0000","last_unstale":"2024-09-20T22:06:03.509901+0000","last_undegraded":"2024-09-20T22:04:16.305833+0000","last_fullsized":"2024-09-20T22:04:16.305652+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:06.925488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":["3","7"],"object_location_counts":[{"shards":"3,7","objects":5}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":136,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.500464+0000","last_change":"2024-09-20T22:02:50.112976+0000","last_active":"2024-09-20T22:06:03.500464+0000","last_peered":"2024-09-20T22:06:03.500464+0000","last_clean":"2024-09-20T22:06:03.500464+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:06:03.500464+0000","last_undegraded":"2024-09-20T22:06:03.500464+0000","last_fullsized":"2024-09-20T22:06:03.500464+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"92'9","reported_seq":179,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.494972+0000","last_change":"2024-09-20T22:04:16.325652+0000","last_active":"2024-09-20T22:06:03.494972+0000","last_peered":"2024-09-20T22:06:03.494972+0000","last_clean":"2024-09-20T22:04:15.181393+0000","last_became_active":"2024-09-20T22:04:16.325652+0000","last_became_peered":"2024-09-20T22:04:16.325652+0000","last_unstale":"2024-09-20T22:06:03.494972+0000","last_undegraded":"2024-09-20T22:04:16.311649+0000","last_fullsized":"2024-09-20T22:04:16.311525+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[{"shards":"0,5","objects":1}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":131,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.505189+0000","last_change":"2024-09-20T22:05:35.601891+0000","last_active":"2024-09-20T22:06:03.505189+0000","last_peered":"2024-09-20T22:06:03.505189+0000","last_clean":"2024-09-20T22:06:03.505189+0000","last_became_active":"2024-09-20T22:02:56.305383+0000","last_became_peered":"2024-09-20T22:02:56.305383+0000","last_unstale":"2024-09-20T22:06:03.505189+0000","last_undegraded":"2024-09-20T22:06:03.505189+0000","last_fullsized":"2024-09-20T22:06:03.505189+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:35.601794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:35.601794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:50.376685+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"126'10839","reported_seq":11658,"reported_epoch":126,"state":"active+undersized+degraded+remapped+backfilling","last_fresh":"2024-09-20T22:06:05.835051+0000","last_change":"2024-09-20T22:06:02.529398+0000","last_active":"2024-09-20T22:06:05.835051+0000","last_peered":"2024-09-20T22:06:05.835051+0000","last_clean":"2024-09-20T22:04:15.697798+0000","last_became_active":"2024-09-20T22:06:02.503584+0000","last_became_peered":"2024-09-20T22:06:02.503584+0000","last_unstale":"2024-09-20T22:06:05.835051+0000","last_undegraded":"2024-09-20T22:06:02.362885+0000","last_fullsized":"2024-09-20T22:06:02.344182+0000","mapping_epoch":124,"log_start":"99'4700","ondisk_log_start":"99'4700","created":121,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":6139,"log_dups_size":0,"ondisk_log_size":6139,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187138048,"num_objects":2856,"num_object_clones":0,"num_object_copies":8568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2511,"num_objects_misplaced":2856,"num_objects_unfound":0,"num_objects_dirty":2856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5512,"num_write_kb":267744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":352,"num_bytes_recovered":23068672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,5],"avail_no_missing":["3","5"],"object_location_counts":[{"shards":"3,5","objects":2856}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"71'11","reported_seq":154,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.498396+0000","last_change":"2024-09-20T22:04:16.316014+0000","last_active":"2024-09-20T22:06:03.498396+0000","last_peered":"2024-09-20T22:06:03.498396+0000","last_clean":"2024-09-20T22:04:11.303420+0000","last_became_active":"2024-09-20T22:04:16.316014+0000","last_became_peered":"2024-09-20T22:04:16.316014+0000","last_unstale":"2024-09-20T22:06:03.498396+0000","last_undegraded":"2024-09-20T22:04:16.304443+0000","last_fullsized":"2024-09-20T22:04:16.304329+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[{"shards":"4,6","objects":5}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":144,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.498906+0000","last_change":"2024-09-20T22:04:54.764407+0000","last_active":"2024-09-20T22:06:03.498906+0000","last_peered":"2024-09-20T22:06:03.498906+0000","last_clean":"2024-09-20T22:06:03.498906+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:06:03.498906+0000","last_undegraded":"2024-09-20T22:06:03.498906+0000","last_fullsized":"2024-09-20T22:06:03.498906+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:54.764137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:54.764137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:07.236633+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":77,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.494392+0000","last_change":"2024-09-20T22:04:16.322913+0000","last_active":"2024-09-20T22:06:03.494392+0000","last_peered":"2024-09-20T22:06:03.494392+0000","last_clean":"2024-09-20T22:02:53.269054+0000","last_became_active":"2024-09-20T22:04:16.322913+0000","last_became_peered":"2024-09-20T22:04:16.322913+0000","last_unstale":"2024-09-20T22:06:03.494392+0000","last_undegraded":"2024-09-20T22:06:03.494392+0000","last_fullsized":"2024-09-20T22:04:16.308557+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":81,"reported_epoch":126,"state":"active+clean+remapped","last_fresh":"2024-09-20T22:06:03.493294+0000","last_change":"2024-09-20T22:05:33.624254+0000","last_active":"2024-09-20T22:06:03.493294+0000","last_peered":"2024-09-20T22:06:03.493294+0000","last_clean":"2024-09-20T22:06:03.493294+0000","last_became_active":"2024-09-20T22:04:21.312463+0000","last_became_peered":"2024-09-20T22:04:21.312463+0000","last_unstale":"2024-09-20T22:06:03.493294+0000","last_undegraded":"2024-09-20T22:06:03.493294+0000","last_fullsized":"2024-09-20T22:06:03.493294+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:33.623816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:33.623816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:02.089979+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5,3],"avail_no_missing":["0","3","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"126'11020","reported_seq":11198,"reported_epoch":126,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T22:06:05.810815+0000","last_change":"2024-09-20T22:06:02.442872+0000","last_active":"2024-09-20T22:06:05.810815+0000","last_peered":"2024-09-20T22:06:05.810815+0000","last_clean":"2024-09-20T22:06:00.240366+0000","last_became_active":"2024-09-20T22:06:02.406172+0000","last_became_peered":"2024-09-20T22:06:02.406172+0000","last_unstale":"2024-09-20T22:06:05.810815+0000","last_undegraded":"2024-09-20T22:06:05.810815+0000","last_fullsized":"2024-09-20T22:06:05.810815+0000","mapping_epoch":124,"log_start":"104'6100","ondisk_log_start":"104'6100","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":1349,"log_size":4920,"log_dups_size":0,"ondisk_log_size":4920,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":189005824,"num_objects":2884,"num_object_clones":0,"num_object_copies":8652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2884,"num_objects_unfound":0,"num_objects_dirty":2884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5589,"num_write_kb":271136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[3,6,4],"avail_no_missing":["3","4","6"],"object_location_counts":[{"shards":"3,4,6","objects":2884}],"blocked_by":[],"up_primary":7,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"71'9","reported_seq":112,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.490936+0000","last_change":"2024-09-20T22:04:16.319241+0000","last_active":"2024-09-20T22:06:03.490936+0000","last_peered":"2024-09-20T22:06:03.490936+0000","last_clean":"2024-09-20T22:02:57.545555+0000","last_became_active":"2024-09-20T22:04:16.319241+0000","last_became_peered":"2024-09-20T22:04:16.319241+0000","last_unstale":"2024-09-20T22:06:03.490936+0000","last_undegraded":"2024-09-20T22:04:16.306520+0000","last_fullsized":"2024-09-20T22:04:16.306412+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:12.196518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:12.196518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":["6","2"],"object_location_counts":[{"shards":"2,6","objects":4}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":152,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.500721+0000","last_change":"2024-09-20T22:05:22.727380+0000","last_active":"2024-09-20T22:06:03.500721+0000","last_peered":"2024-09-20T22:06:03.500721+0000","last_clean":"2024-09-20T22:06:03.500721+0000","last_became_active":"2024-09-20T22:02:50.109803+0000","last_became_peered":"2024-09-20T22:02:50.109803+0000","last_unstale":"2024-09-20T22:06:03.500721+0000","last_undegraded":"2024-09-20T22:06:03.500721+0000","last_fullsized":"2024-09-20T22:06:03.500721+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:22.727282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:22.727282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:24.915671+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":130,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.504948+0000","last_change":"2024-09-20T22:04:16.312705+0000","last_active":"2024-09-20T22:06:03.504948+0000","last_peered":"2024-09-20T22:06:03.504948+0000","last_clean":"2024-09-20T22:04:11.312860+0000","last_became_active":"2024-09-20T22:04:16.312705+0000","last_became_peered":"2024-09-20T22:04:16.312705+0000","last_unstale":"2024-09-20T22:06:03.504948+0000","last_undegraded":"2024-09-20T22:06:03.504948+0000","last_fullsized":"2024-09-20T22:04:16.306905+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:06.925465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":126,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.491644+0000","last_change":"2024-09-20T22:04:16.315649+0000","last_active":"2024-09-20T22:06:03.491644+0000","last_peered":"2024-09-20T22:06:03.491644+0000","last_clean":"2024-09-20T22:04:11.306119+0000","last_became_active":"2024-09-20T22:04:16.315649+0000","last_became_peered":"2024-09-20T22:04:16.315649+0000","last_unstale":"2024-09-20T22:06:03.491644+0000","last_undegraded":"2024-09-20T22:06:03.491644+0000","last_fullsized":"2024-09-20T22:04:16.302635+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:05.570135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"126'10778","reported_seq":10892,"reported_epoch":126,"state":"active+undersized+degraded+remapped+backfill_toofull","last_fresh":"2024-09-20T22:06:05.771302+0000","last_change":"2024-09-20T22:06:02.495738+0000","last_active":"2024-09-20T22:06:05.771302+0000","last_peered":"2024-09-20T22:06:05.771302+0000","last_clean":"2024-09-20T22:04:15.696640+0000","last_became_active":"2024-09-20T22:06:02.452182+0000","last_became_peered":"2024-09-20T22:06:02.452182+0000","last_unstale":"2024-09-20T22:06:05.771302+0000","last_undegraded":"2024-09-20T22:06:02.364382+0000","last_fullsized":"2024-09-20T22:06:02.338781+0000","mapping_epoch":124,"log_start":"104'5900","ondisk_log_start":"104'5900","created":121,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:03:51.292681+0000","objects_scrubbed":0,"log_size":4878,"log_dups_size":0,"ondisk_log_size":4878,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":183795712,"num_objects":2805,"num_object_clones":0,"num_object_copies":8415,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2805,"num_objects_misplaced":2805,"num_objects_unfound":0,"num_objects_dirty":2805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5484,"num_write_kb":265216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,2],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[{"shards":"2,3","objects":2805}],"blocked_by":[],"up_primary":6,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"71'11","reported_seq":158,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.507123+0000","last_change":"2024-09-20T22:04:20.835713+0000","last_active":"2024-09-20T22:06:03.507123+0000","last_peered":"2024-09-20T22:06:03.507123+0000","last_clean":"2024-09-20T22:06:03.507123+0000","last_became_active":"2024-09-20T22:02:52.480118+0000","last_became_peered":"2024-09-20T22:02:52.480118+0000","last_unstale":"2024-09-20T22:06:03.507123+0000","last_undegraded":"2024-09-20T22:06:03.507123+0000","last_fullsized":"2024-09-20T22:06:03.507123+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:04:20.835626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:20.835626+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":146,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.495205+0000","last_change":"2024-09-20T22:04:16.325403+0000","last_active":"2024-09-20T22:06:03.495205+0000","last_peered":"2024-09-20T22:06:03.495205+0000","last_clean":"2024-09-20T22:04:14.620696+0000","last_became_active":"2024-09-20T22:04:16.325403+0000","last_became_peered":"2024-09-20T22:04:16.325403+0000","last_unstale":"2024-09-20T22:06:03.495205+0000","last_undegraded":"2024-09-20T22:06:03.495205+0000","last_fullsized":"2024-09-20T22:04:16.313799+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:14.620563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:14.620563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":136,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.499421+0000","last_change":"2024-09-20T22:04:57.641872+0000","last_active":"2024-09-20T22:06:03.499421+0000","last_peered":"2024-09-20T22:06:03.499421+0000","last_clean":"2024-09-20T22:06:03.499421+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:06:03.499421+0000","last_undegraded":"2024-09-20T22:06:03.499421+0000","last_fullsized":"2024-09-20T22:06:03.499421+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:57.641748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:57.641748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:08.376222+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":131,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.505179+0000","last_change":"2024-09-20T22:05:30.899063+0000","last_active":"2024-09-20T22:06:03.505179+0000","last_peered":"2024-09-20T22:06:03.505179+0000","last_clean":"2024-09-20T22:06:03.505179+0000","last_became_active":"2024-09-20T22:02:56.347637+0000","last_became_peered":"2024-09-20T22:02:56.347637+0000","last_unstale":"2024-09-20T22:06:03.505179+0000","last_undegraded":"2024-09-20T22:06:03.505179+0000","last_fullsized":"2024-09-20T22:06:03.505179+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:30.898956+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:30.898956+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:31.255983+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"126'10760","reported_seq":12201,"reported_epoch":126,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T22:06:05.834377+0000","last_change":"2024-09-20T22:06:02.518919+0000","last_active":"2024-09-20T22:06:05.834377+0000","last_peered":"2024-09-20T22:06:05.834377+0000","last_clean":"2024-09-20T22:06:00.436623+0000","last_became_active":"2024-09-20T22:06:02.457626+0000","last_became_peered":"2024-09-20T22:06:02.457626+0000","last_unstale":"2024-09-20T22:06:05.834377+0000","last_undegraded":"2024-09-20T22:06:05.834377+0000","last_fullsized":"2024-09-20T22:06:05.834377+0000","mapping_epoch":124,"log_start":"98'3600","ondisk_log_start":"98'3600","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":5,"last_scrub":"112'8323","last_scrub_stamp":"2024-09-20T22:05:27.253221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:03:51.292681+0000","last_clean_scrub_stamp":"2024-09-20T22:05:27.253221+0000","objects_scrubbed":1849,"log_size":7160,"log_dups_size":0,"ondisk_log_size":7160,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":4,"scrub_schedule":"no scrub is scheduled","scrub_duration":3507,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187170816,"num_objects":2856,"num_object_clones":0,"num_object_copies":8568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4938,"num_objects_unfound":0,"num_objects_dirty":2856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5468,"num_write_kb":266368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":403,"num_bytes_recovered":26214400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[3,2,4],"avail_no_missing":["3","2","4"],"object_location_counts":[{"shards":"2,3,4","objects":2856}],"blocked_by":[],"up_primary":0,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"71'4","reported_seq":140,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.505271+0000","last_change":"2024-09-20T22:04:16.310098+0000","last_active":"2024-09-20T22:06:03.505271+0000","last_peered":"2024-09-20T22:06:03.505271+0000","last_clean":"2024-09-20T22:04:11.310261+0000","last_became_active":"2024-09-20T22:04:16.310098+0000","last_became_peered":"2024-09-20T22:04:16.310098+0000","last_unstale":"2024-09-20T22:06:03.505271+0000","last_undegraded":"2024-09-20T22:04:16.306006+0000","last_fullsized":"2024-09-20T22:04:16.305667+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:06.925495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":["3","6"],"object_location_counts":[{"shards":"3,6","objects":2}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":143,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.499212+0000","last_change":"2024-09-20T22:04:16.198264+0000","last_active":"2024-09-20T22:06:03.499212+0000","last_peered":"2024-09-20T22:06:03.499212+0000","last_clean":"2024-09-20T22:06:03.499212+0000","last_became_active":"2024-09-20T22:02:50.111142+0000","last_became_peered":"2024-09-20T22:02:50.111142+0000","last_unstale":"2024-09-20T22:06:03.499212+0000","last_undegraded":"2024-09-20T22:06:03.499212+0000","last_fullsized":"2024-09-20T22:06:03.499212+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:16.198153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:16.198153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":143,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.492078+0000","last_change":"2024-09-20T22:05:26.146698+0000","last_active":"2024-09-20T22:06:03.492078+0000","last_peered":"2024-09-20T22:06:03.492078+0000","last_clean":"2024-09-20T22:06:03.492078+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:06:03.492078+0000","last_undegraded":"2024-09-20T22:06:03.492078+0000","last_fullsized":"2024-09-20T22:06:03.492078+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:26.146625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:26.146625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:34.163023+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":132,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.499576+0000","last_change":"2024-09-20T22:06:01.621567+0000","last_active":"2024-09-20T22:06:03.499576+0000","last_peered":"2024-09-20T22:06:03.499576+0000","last_clean":"2024-09-20T22:06:03.499576+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:06:03.499576+0000","last_undegraded":"2024-09-20T22:06:03.499576+0000","last_fullsized":"2024-09-20T22:06:03.499576+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:01.621489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:01.621489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:03.902306+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":77,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.494923+0000","last_change":"2024-09-20T22:04:16.324831+0000","last_active":"2024-09-20T22:06:03.494923+0000","last_peered":"2024-09-20T22:06:03.494923+0000","last_clean":"2024-09-20T22:02:55.278800+0000","last_became_active":"2024-09-20T22:04:16.324831+0000","last_became_peered":"2024-09-20T22:04:16.324831+0000","last_unstale":"2024-09-20T22:06:03.494923+0000","last_undegraded":"2024-09-20T22:06:03.494923+0000","last_fullsized":"2024-09-20T22:04:16.311809+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:15.303300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:15.303300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":138,"reported_epoch":126,"state":"active+undersized","last_fresh":"2024-09-20T22:06:03.509425+0000","last_change":"2024-09-20T22:04:16.309674+0000","last_active":"2024-09-20T22:06:03.509425+0000","last_peered":"2024-09-20T22:06:03.509425+0000","last_clean":"2024-09-20T22:04:11.311023+0000","last_became_active":"2024-09-20T22:04:16.309674+0000","last_became_peered":"2024-09-20T22:04:16.309674+0000","last_unstale":"2024-09-20T22:06:03.509425+0000","last_undegraded":"2024-09-20T22:06:03.509425+0000","last_fullsized":"2024-09-20T22:04:16.304826+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:06.925447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":["3","0"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"71'11","reported_seq":153,"reported_epoch":126,"state":"active+undersized+degraded","last_fresh":"2024-09-20T22:06:03.491321+0000","last_change":"2024-09-20T22:04:16.326833+0000","last_active":"2024-09-20T22:06:03.491321+0000","last_peered":"2024-09-20T22:06:03.491321+0000","last_clean":"2024-09-20T22:04:11.307316+0000","last_became_active":"2024-09-20T22:04:16.326833+0000","last_became_peered":"2024-09-20T22:04:16.326833+0000","last_unstale":"2024-09-20T22:06:03.491321+0000","last_undegraded":"2024-09-20T22:04:16.307528+0000","last_fullsized":"2024-09-20T22:04:16.307421+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[{"shards":"5,6","objects":5}],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":135,"reported_epoch":126,"state":"active+clean","last_fresh":"2024-09-20T22:06:03.505493+0000","last_change":"2024-09-20T22:05:06.706314+0000","last_active":"2024-09-20T22:06:03.505493+0000","last_peered":"2024-09-20T22:06:03.505493+0000","last_clean":"2024-09-20T22:06:03.505493+0000","last_became_active":"2024-09-20T22:02:54.289078+0000","last_became_peered":"2024-09-20T22:02:54.289078+0000","last_unstale":"2024-09-20T22:06:03.505493+0000","last_undegraded":"2024-09-20T22:06:03.505493+0000","last_fullsized":"2024-09-20T22:06:03.505493+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:06.681123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:06.681123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:06:18.782996+0000","scrub_duration":82,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":26,"stat_sum":{"num_bytes":5931171864,"num_objects":90504,"num_object_clones":0,"num_object_copies":271512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33569,"num_objects_misplaced":46808,"num_objects_unfound":0,"num_objects_dirty":90504,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143037,"num_write_kb":6906945,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1138,"num_bytes_recovered":73793536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16704806912,"data_stored":16704798768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148291,"ondisk_log_size":148291,"up":69,"acting":68,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":84,"acting":89,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66,"ondisk_log_size":66,"up":82,"acting":82,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":394,"ondisk_log_size":394,"up":76,"acting":76,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":85,"acting":87,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":185,"num_read_kb":180,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":7,"acting":7,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135915,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1954460,"kb_used_data":1850952,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":91790692,"statfs":{"total":95995035648,"available":93993668608,"internally_reserved":0,"allocated":1895374848,"data_stored":1894876612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":105839050},"hb_peers":[0,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,5],"upper_bound":2048},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430255,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1939084,"kb_used_data":1835708,"kb_used_omap":1,"kb_used_meta":103358,"kb_avail":91806068,"statfs":{"total":95995035648,"available":94009413632,"internally_reserved":0,"allocated":1879764992,"data_stored":1879278976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":105839046},"hb_peers":[0,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724595,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1576520,"kb_used_data":1468980,"kb_used_omap":1,"kb_used_meta":107518,"kb_avail":92168632,"statfs":{"total":95995035648,"available":94380679168,"internally_reserved":0,"allocated":1504235520,"data_stored":1503757474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":110098885},"hb_peers":[0,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986235,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1975260,"kb_used_data":1853836,"kb_used_omap":1,"kb_used_meta":121406,"kb_avail":91769892,"statfs":{"total":95995035648,"available":93972369408,"internally_reserved":0,"allocated":1898328064,"data_stored":1897825131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124320197},"hb_peers":[0,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574900,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":968916,"kb_used_data":941892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":92776236,"statfs":{"total":95995035648,"available":95002865664,"internally_reserved":0,"allocated":964497408,"data_stored":963964136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149760,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3479796,"kb_used_data":3304220,"kb_used_omap":1,"kb_used_meta":175550,"kb_avail":90265356,"statfs":{"total":95995035648,"available":92431724544,"internally_reserved":0,"allocated":3383521280,"data_stored":3383003255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":179763655},"hb_peers":[0,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411397,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2351036,"kb_used_data":2229544,"kb_used_omap":1,"kb_used_meta":121406,"kb_avail":91394116,"statfs":{"total":95995035648,"available":93587574784,"internally_reserved":0,"allocated":2283053056,"data_stored":2282586094,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":124320197},"hb_peers":[0,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3102304,"kb_used_data":2944784,"kb_used_omap":1,"kb_used_meta":157502,"kb_avail":90642848,"statfs":{"total":95995035648,"available":92818276352,"internally_reserved":0,"allocated":3015458816,"data_stored":3014952544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":161282501},"hb_peers":[2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3009220608,"data_stored":3009216536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":973668352,"data_stored":973668352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2260271104,"data_stored":2260271104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3335192576,"data_stored":3335192576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1877803008,"data_stored":1877803008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1497759744,"data_stored":1497759744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1873547264,"data_stored":1873543192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1877344256,"data_stored":1877344256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:06:08.628 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg force-recovery 4.18 7.19 4.1c 4.1d 3.19 4.f 7.2 5.0 7.0 4.4 7.4 4.7 4.6 2.0 4.d 4.b 7.9 4.a 4.9 7.b 4.17 4.14 7.10 4.13 4.12 4.1f 2024-09-20T22:06:08.635 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:08.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:08.636 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:08.737 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:08.803 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:08.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:08.804 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:08.820 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:08.896 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:08 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:06:08.896 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:08 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:06:08.897 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:08 smithi060 ceph-mon[40290]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:06:08.897 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:08 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:06:08.897 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:08 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:06:08.897 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:08 smithi060 ceph-mon[40290]: osdmap e128: 8 total, 7 up, 8 in 2024-09-20T22:06:08.897 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:08 smithi060 ceph-mon[40290]: pgmap v142: 158 pgs: 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 1 active+undersized+degraded+remapped+backfilling, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 5.5 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 912 B/s rd, 279 MiB/s wr, 6.45k op/s; 33681/272109 objects degraded (12.378%); 46808/272109 objects misplaced (17.202%); 13 MiB/s, 202 objects/s recovering 2024-09-20T22:06:08.905 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:08.936 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:08.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:08.937 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:09.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:08 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:06:09.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:08 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:06:09.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:08 smithi067 ceph-mon[33511]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:06:09.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:08 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:06:09.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:08 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:06:09.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:08 smithi067 ceph-mon[33511]: osdmap e128: 8 total, 7 up, 8 in 2024-09-20T22:06:09.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:08 smithi067 ceph-mon[33511]: pgmap v142: 158 pgs: 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 1 active+undersized+degraded+remapped+backfilling, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 5.5 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 912 B/s rd, 279 MiB/s wr, 6.45k op/s; 33681/272109 objects degraded (12.378%); 46808/272109 objects misplaced (17.202%); 13 MiB/s, 202 objects/s recovering 2024-09-20T22:06:09.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:09.075 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:09.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:09.076 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:09.159 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:08 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:06:09.160 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:08 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:06:09.160 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:08 smithi060 ceph-mon[29694]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:06:09.160 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:08 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:06:09.160 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:08 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:06:09.160 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:08 smithi060 ceph-mon[29694]: osdmap e128: 8 total, 7 up, 8 in 2024-09-20T22:06:09.160 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:08 smithi060 ceph-mon[29694]: pgmap v142: 158 pgs: 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 1 active+undersized+degraded+remapped+backfilling, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 5.5 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 912 B/s rd, 279 MiB/s wr, 6.45k op/s; 33681/272109 objects degraded (12.378%); 46808/272109 objects misplaced (17.202%); 13 MiB/s, 202 objects/s recovering 2024-09-20T22:06:09.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:09.220 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:09.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:09.221 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:09.322 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:09.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:09.368 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:09.368 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:09.469 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:09.490 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:09.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:09.491 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:09.592 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:09.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:09.632 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:09.633 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:09.733 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:09.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:09.778 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:09.778 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:09.878 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:09.919 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:09.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:09.920 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:09.936 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:06:09.997 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T22:06:10.002 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:06:10.021 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:10.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:10.084 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:10.085 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:10.186 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:10.252 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:10.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:10.253 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:10.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:09 smithi067 ceph-mon[33511]: 5.1b scrub starts 2024-09-20T22:06:10.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:09 smithi067 ceph-mon[33511]: 5.1b scrub ok 2024-09-20T22:06:10.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:09 smithi067 ceph-mon[33511]: 4.1a scrub starts 2024-09-20T22:06:10.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:09 smithi067 ceph-mon[33511]: 4.1a scrub ok 2024-09-20T22:06:10.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:09 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2519165434' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:06:10.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:09 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:06:10.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:09 smithi060 ceph-mon[29694]: 5.1b scrub starts 2024-09-20T22:06:10.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:09 smithi060 ceph-mon[29694]: 5.1b scrub ok 2024-09-20T22:06:10.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:09 smithi060 ceph-mon[29694]: 4.1a scrub starts 2024-09-20T22:06:10.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:09 smithi060 ceph-mon[29694]: 4.1a scrub ok 2024-09-20T22:06:10.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:09 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2519165434' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:06:10.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:09 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:06:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:09 smithi060 ceph-mon[40290]: 5.1b scrub starts 2024-09-20T22:06:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:09 smithi060 ceph-mon[40290]: 5.1b scrub ok 2024-09-20T22:06:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:09 smithi060 ceph-mon[40290]: 4.1a scrub starts 2024-09-20T22:06:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:09 smithi060 ceph-mon[40290]: 4.1a scrub ok 2024-09-20T22:06:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:09 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2519165434' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:06:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:09 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:06:10.353 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:10.404 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:10.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:10.405 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:10.408 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:06:10.506 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:10.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:10.546 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:10.547 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:10.647 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:10.681 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:10.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:10.683 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:10.783 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:10.819 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:10.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:10.820 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:10.921 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:10.960 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:10.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:10.961 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:10 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:06:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:10 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:06:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:10 smithi067 ceph-mon[33511]: osdmap e129: 8 total, 7 up, 8 in 2024-09-20T22:06:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:10 smithi067 ceph-mon[33511]: pgmap v144: 158 pgs: 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 1 active+undersized+degraded+remapped+backfilling, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 5.5 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 260 MiB/s wr, 6.03k op/s; 33681/272109 objects degraded (12.378%); 46808/272109 objects misplaced (17.202%); 12 MiB/s, 189 objects/s recovering 2024-09-20T22:06:11.062 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:11.095 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:11.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:11.096 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:11.197 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:11.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:11.237 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:11.237 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:10 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:06:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:10 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:06:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:10 smithi060 ceph-mon[29694]: osdmap e129: 8 total, 7 up, 8 in 2024-09-20T22:06:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:10 smithi060 ceph-mon[29694]: pgmap v144: 158 pgs: 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 1 active+undersized+degraded+remapped+backfilling, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 5.5 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 260 MiB/s wr, 6.03k op/s; 33681/272109 objects degraded (12.378%); 46808/272109 objects misplaced (17.202%); 12 MiB/s, 189 objects/s recovering 2024-09-20T22:06:11.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:10 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:06:11.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:10 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:06:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:10 smithi060 ceph-mon[40290]: osdmap e129: 8 total, 7 up, 8 in 2024-09-20T22:06:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:10 smithi060 ceph-mon[40290]: pgmap v144: 158 pgs: 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 1 active+undersized+degraded+remapped+backfilling, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 5.5 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 260 MiB/s wr, 6.03k op/s; 33681/272109 objects degraded (12.378%); 46808/272109 objects misplaced (17.202%); 12 MiB/s, 189 objects/s recovering 2024-09-20T22:06:11.338 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:11.348 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:06:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:06:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:182314 7.d 7:b821e8ed:::benchmark_data_smithi060_78800_object49281:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-20T22:06:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:11.323214+0000", 2024-09-20T22:06:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.021380112999999999, 2024-09-20T22:06:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.021412985999999998, 2024-09-20T22:06:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:06:11.348 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 182314 2024-09-20T22:06:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.323214+0000", 2024-09-20T22:06:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:11.349 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.323214+0000", 2024-09-20T22:06:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.323216+0000", 2024-09-20T22:06:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1820000000000001e-06 2024-09-20T22:06:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.323221+0000", 2024-09-20T22:06:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6550000000000001e-06 2024-09-20T22:06:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.350 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.323233+0000", 2024-09-20T22:06:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1385e-05 2024-09-20T22:06:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.323240+0000", 2024-09-20T22:06:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6170000000000001e-06 2024-09-20T22:06:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.324278+0000", 2024-09-20T22:06:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0010378690000000001 2024-09-20T22:06:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.352 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:11.352 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.324350+0000", 2024-09-20T22:06:11.352 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1855999999999998e-05 2024-09-20T22:06:11.352 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.352 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.352 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0", 2024-09-20T22:06:11.352 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.324422+0000", 2024-09-20T22:06:11.352 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2337999999999998e-05 2024-09-20T22:06:11.352 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.352 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.352 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:11.352 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.326722+0000", 2024-09-20T22:06:11.352 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0022999409999999998 2024-09-20T22:06:11.352 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:11.353 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:11.353 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:11.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.353 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.353 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop_reply(client.24667.0:182294 7.7 e129/121 ondisk, result = 0)", 2024-09-20T22:06:11.353 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:11.344443+0000", 2024-09-20T22:06:11.353 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.00015130300000000001, 2024-09-20T22:06:11.353 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00026689900000000002, 2024-09-20T22:06:11.353 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:11.353 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:11.353 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:06:11.353 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:11.353 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.353 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:11.353 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.344443+0000", 2024-09-20T22:06:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.344443+0000", 2024-09-20T22:06:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.344444+0000", 2024-09-20T22:06:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5699999999999998e-07 2024-09-20T22:06:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:11.354 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.344448+0000", 2024-09-20T22:06:11.355 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9430000000000002e-06 2024-09-20T22:06:11.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.355 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:11.355 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.344457+0000", 2024-09-20T22:06:11.355 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8300000000000008e-06 2024-09-20T22:06:11.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.355 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:11.355 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.344464+0000", 2024-09-20T22:06:11.355 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6479999999999997e-06 2024-09-20T22:06:11.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.355 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:11.356 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.344669+0000", 2024-09-20T22:06:11.356 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020501300000000001 2024-09-20T22:06:11.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.356 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.356 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:11.356 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.344674+0000", 2024-09-20T22:06:11.356 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8539999999999996e-06 2024-09-20T22:06:11.356 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:11.356 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:11.356 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:11.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.356 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.356 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop_reply(client.24667.0:182302 7.7 e129/121)", 2024-09-20T22:06:11.356 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:11.344482+0000", 2024-09-20T22:06:11.357 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.00011203499999999999, 2024-09-20T22:06:11.357 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00029160200000000002, 2024-09-20T22:06:11.357 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:11.357 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:11.357 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:06:11.357 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:11.357 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.357 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:11.357 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.344482+0000", 2024-09-20T22:06:11.357 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:11.357 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.357 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.357 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:11.357 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.344482+0000", 2024-09-20T22:06:11.358 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:11.358 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.358 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.358 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:11.358 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.344483+0000", 2024-09-20T22:06:11.358 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.15e-07 2024-09-20T22:06:11.358 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.358 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.358 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:11.358 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.344485+0000", 2024-09-20T22:06:11.358 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.3149999999999999e-06 2024-09-20T22:06:11.358 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.359 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.359 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:11.359 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.344492+0000", 2024-09-20T22:06:11.359 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0790000000000003e-06 2024-09-20T22:06:11.359 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.359 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.359 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:11.359 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.344498+0000", 2024-09-20T22:06:11.359 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3669999999999996e-06 2024-09-20T22:06:11.359 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.360 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.360 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:11.360 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.344768+0000", 2024-09-20T22:06:11.360 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00027025399999999999 2024-09-20T22:06:11.360 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.360 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.360 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:11.360 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.344775+0000", 2024-09-20T22:06:11.360 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2359999999999998e-06 2024-09-20T22:06:11.360 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:11.361 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:11.361 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:11.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.361 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.361 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:182302 7.7 7:e1cf309b:::benchmark_data_smithi060_78800_object49269:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-20T22:06:11.361 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:11.310707+0000", 2024-09-20T22:06:11.361 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.033887668000000003, 2024-09-20T22:06:11.361 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.034128370999999998, 2024-09-20T22:06:11.361 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:11.361 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:11.361 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:06:11.361 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:11.361 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:11.362 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:11.362 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 182302 2024-09-20T22:06:11.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.362 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:11.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.362 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:11.362 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.310707+0000", 2024-09-20T22:06:11.362 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:11.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.362 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:11.362 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.310707+0000", 2024-09-20T22:06:11.363 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:11.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.363 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:11.363 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.310708+0000", 2024-09-20T22:06:11.363 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.195e-06 2024-09-20T22:06:11.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.363 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:11.363 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.310713+0000", 2024-09-20T22:06:11.363 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1279999999999999e-06 2024-09-20T22:06:11.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.364 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:11.364 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.310729+0000", 2024-09-20T22:06:11.364 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6143000000000001e-05 2024-09-20T22:06:11.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.364 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:11.364 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.310738+0000", 2024-09-20T22:06:11.364 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4060000000000005e-06 2024-09-20T22:06:11.365 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.365 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.365 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:11.365 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.310791+0000", 2024-09-20T22:06:11.365 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2815999999999997e-05 2024-09-20T22:06:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.310905+0000", 2024-09-20T22:06:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000114073 2024-09-20T22:06:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:06:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.311170+0000", 2024-09-20T22:06:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00026570400000000002 2024-09-20T22:06:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.366 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:11.367 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.313622+0000", 2024-09-20T22:06:11.367 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0024518740000000002 2024-09-20T22:06:11.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.367 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.367 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:11.367 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.317757+0000", 2024-09-20T22:06:11.367 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0041352009999999998 2024-09-20T22:06:11.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.367 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.367 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:11.367 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.344783+0000", 2024-09-20T22:06:11.367 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.027025242000000001 2024-09-20T22:06:11.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:11.367 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:11.368 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:11.368 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:11.344842+0000", 2024-09-20T22:06:11.368 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9004e-05 2024-09-20T22:06:11.368 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:11.368 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:11.368 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:11.368 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:11.368 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-20T22:06:11.368 INFO:teuthology.orchestra.run.smithi067.stdout: "num_ops": 4 2024-09-20T22:06:11.368 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:06:11.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:11.372 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:11.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:11.473 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:11.505 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:11.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:11.506 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:11.607 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:11.650 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:11.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:11.651 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:11.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:11.789 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:11.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:11.790 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:11.891 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:11.920 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:11.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:11.921 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:12.022 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:12.054 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:12.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:12.055 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:12.156 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:12.197 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:12.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:12.198 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:12.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:11 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 33681/272109 objects degraded (12.378%), 36 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:12.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:06:11 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:06:12.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:11 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 33681/272109 objects degraded (12.378%), 36 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:12.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:11 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 33681/272109 objects degraded (12.378%), 36 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:12.298 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:12.303 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T22:06:12.327 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:12.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:12.329 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:12.429 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:12.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:12.472 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:12.472 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:12.573 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:12.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:12.620 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:12.620 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:12.716 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:06:12.721 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:12.747 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:12.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:12.748 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:12.849 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:12.882 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:12.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:12.884 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:12.985 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:13.015 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:13.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:13.016 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:13.117 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:13.148 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:12 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:06:13.148 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:12 smithi067 ceph-mon[33511]: pgmap v145: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 5.1 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.04k op/s; 30807/253407 objects degraded (12.157%); 41370/253407 objects misplaced (16.326%); 17 MiB/s, 272 objects/s recovering 2024-09-20T22:06:13.157 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:13.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:13.158 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:13.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:13.283 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:13.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:13.284 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:13.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:12 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:06:13.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:12 smithi060 ceph-mon[29694]: pgmap v145: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 5.1 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.04k op/s; 30807/253407 objects degraded (12.157%); 41370/253407 objects misplaced (16.326%); 17 MiB/s, 272 objects/s recovering 2024-09-20T22:06:13.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:12 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:06:13.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:12 smithi060 ceph-mon[40290]: pgmap v145: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 5.1 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.04k op/s; 30807/253407 objects degraded (12.157%); 41370/253407 objects misplaced (16.326%); 17 MiB/s, 272 objects/s recovering 2024-09-20T22:06:13.385 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:13.424 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:13.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:13.425 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:13.526 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:13.547 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:13.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:13.548 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:13.649 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:13.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:13.683 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:13.683 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:13.784 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:13.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:13.825 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:13.825 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:13.898 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:06:13.898 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:06:13.898 INFO:teuthology.orchestra.run.smithi067.stdout: "complaint_time": 30, 2024-09-20T22:06:13.898 INFO:teuthology.orchestra.run.smithi067.stdout: "num_blocked_ops": 0 2024-09-20T22:06:13.898 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:06:13.927 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:13.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:13.962 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:13.962 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:14.063 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:14.084 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:14.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:14.085 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:14.186 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:14.217 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:14.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:14.218 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:14.319 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:14.350 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:14.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:14.351 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:14.452 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:14.490 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:14.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:14.491 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:14.592 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:14.614 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:14.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:14.615 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:14.715 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:14.750 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:14.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:14.751 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:14.852 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:14.886 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:14.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:14.887 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:14.978 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T22:06:14.988 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:15.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:15.046 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:15.047 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:15.147 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:15.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:15.190 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:15.190 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:15.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:15 smithi067 ceph-mon[33511]: pgmap v146: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 5.1 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 673 B/s rd, 0 B/s wr, 820 op/s; 30807/253407 objects degraded (12.157%); 41370/253407 objects misplaced (16.326%); 13 MiB/s, 215 objects/s recovering 2024-09-20T22:06:15.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:06:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:06:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:06:15.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:15 smithi060 ceph-mon[29694]: pgmap v146: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 5.1 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 673 B/s rd, 0 B/s wr, 820 op/s; 30807/253407 objects degraded (12.157%); 41370/253407 objects misplaced (16.326%); 13 MiB/s, 215 objects/s recovering 2024-09-20T22:06:15.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:15 smithi060 ceph-mon[40290]: pgmap v146: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 5.1 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 673 B/s rd, 0 B/s wr, 820 op/s; 30807/253407 objects degraded (12.157%); 41370/253407 objects misplaced (16.326%); 13 MiB/s, 215 objects/s recovering 2024-09-20T22:06:15.291 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:15.329 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:15.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:15.330 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:15.342 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:06:15.431 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:15.432 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:15.458 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:15.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:15.459 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:15.496 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:15.560 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:15.592 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:15.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:15.593 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:15.694 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:15.728 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:15.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:15.730 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:15.831 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:15.868 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:15.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:15.869 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:15.956 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:06:15.956 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:06:15.956 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:06:15.956 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:06:15.956 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.956 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:22515 7.7 7:e2a23e22:::benchmark_data_smithi060_78800_object22514:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:06:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:06.081179+0000", 2024-09-20T22:06:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 129.87114451100001, 2024-09-20T22:06:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.088628218999999994, 2024-09-20T22:06:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 22515 2024-09-20T22:06:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.957 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081179+0000", 2024-09-20T22:06:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081179+0000", 2024-09-20T22:06:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081179+0000", 2024-09-20T22:06:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7300000000000002e-07 2024-09-20T22:06:15.958 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081204+0000", 2024-09-20T22:06:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.4944e-05 2024-09-20T22:06:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081211+0000", 2024-09-20T22:06:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6220000000000003e-06 2024-09-20T22:06:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081214+0000", 2024-09-20T22:06:15.959 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.557e-06 2024-09-20T22:06:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.082794+0000", 2024-09-20T22:06:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001579347 2024-09-20T22:06:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.082939+0000", 2024-09-20T22:06:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014563900000000001 2024-09-20T22:06:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:06:15.960 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083090+0000", 2024-09-20T22:06:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000150879 2024-09-20T22:06:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085586+0000", 2024-09-20T22:06:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002496042 2024-09-20T22:06:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.086685+0000", 2024-09-20T22:06:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001098903 2024-09-20T22:06:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:15.961 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.169690+0000", 2024-09-20T22:06:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.083004665000000005 2024-09-20T22:06:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.169761+0000", 2024-09-20T22:06:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1145999999999999e-05 2024-09-20T22:06:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.169807+0000", 2024-09-20T22:06:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6102e-05 2024-09-20T22:06:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:15.962 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:22520 7.7 7:e34001c7:::benchmark_data_smithi060_78800_object22519:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:06:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:06.081333+0000", 2024-09-20T22:06:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 129.870990707, 2024-09-20T22:06:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.088627362000000001, 2024-09-20T22:06:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:15.963 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 22520 2024-09-20T22:06:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081333+0000", 2024-09-20T22:06:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081333+0000", 2024-09-20T22:06:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.964 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081345+0000", 2024-09-20T22:06:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2258e-05 2024-09-20T22:06:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081565+0000", 2024-09-20T22:06:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00022027599999999999 2024-09-20T22:06:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081577+0000", 2024-09-20T22:06:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1364e-05 2024-09-20T22:06:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.965 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.081596+0000", 2024-09-20T22:06:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9026999999999999e-05 2024-09-20T22:06:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083719+0000", 2024-09-20T22:06:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0021235690000000001 2024-09-20T22:06:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083865+0000", 2024-09-20T22:06:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014620599999999999 2024-09-20T22:06:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.966 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:06:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.084003+0000", 2024-09-20T22:06:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013766399999999999 2024-09-20T22:06:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085594+0000", 2024-09-20T22:06:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0015914270000000001 2024-09-20T22:06:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.088221+0000", 2024-09-20T22:06:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0026267370000000001 2024-09-20T22:06:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.967 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.169859+0000", 2024-09-20T22:06:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.081638302999999995 2024-09-20T22:06:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.169919+0000", 2024-09-20T22:06:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9468000000000001e-05 2024-09-20T22:06:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.169960+0000", 2024-09-20T22:06:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1063000000000003e-05 2024-09-20T22:06:15.968 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:22524 7.7 7:e83bec1e:::benchmark_data_smithi060_78800_object22523:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:06:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:06.083249+0000", 2024-09-20T22:06:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 129.86907446800001, 2024-09-20T22:06:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.086867280000000005, 2024-09-20T22:06:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:15.969 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 22524 2024-09-20T22:06:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083249+0000", 2024-09-20T22:06:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083249+0000", 2024-09-20T22:06:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.970 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083250+0000", 2024-09-20T22:06:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0240000000000001e-06 2024-09-20T22:06:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083308+0000", 2024-09-20T22:06:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8545000000000003e-05 2024-09-20T22:06:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083354+0000", 2024-09-20T22:06:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5658000000000002e-05 2024-09-20T22:06:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.971 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083361+0000", 2024-09-20T22:06:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0720000000000002e-06 2024-09-20T22:06:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.084644+0000", 2024-09-20T22:06:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0012832760000000001 2024-09-20T22:06:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.084739+0000", 2024-09-20T22:06:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4093999999999997e-05 2024-09-20T22:06:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.972 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:06:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.084836+0000", 2024-09-20T22:06:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7829000000000001e-05 2024-09-20T22:06:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085599+0000", 2024-09-20T22:06:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00076252199999999996 2024-09-20T22:06:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.088259+0000", 2024-09-20T22:06:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002660188 2024-09-20T22:06:15.973 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.170015+0000", 2024-09-20T22:06:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.081755456000000004 2024-09-20T22:06:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.170076+0000", 2024-09-20T22:06:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1807e-05 2024-09-20T22:06:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:15.974 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.170116+0000", 2024-09-20T22:06:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9808999999999998e-05 2024-09-20T22:06:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:32200 7.7 7:e661f05a:::benchmark_data_smithi060_78800_object32199:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-20T22:06:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:11.071733+0000", 2024-09-20T22:06:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 124.88059024099999, 2024-09-20T22:06:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.078344860000000002, 2024-09-20T22:06:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:15.975 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 32200 2024-09-20T22:06:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.071733+0000", 2024-09-20T22:06:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.071733+0000", 2024-09-20T22:06:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:15.976 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.071734+0000", 2024-09-20T22:06:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5700000000000002e-07 2024-09-20T22:06:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.071816+0000", 2024-09-20T22:06:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2111e-05 2024-09-20T22:06:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.071826+0000", 2024-09-20T22:06:15.977 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.978e-06 2024-09-20T22:06:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.071832+0000", 2024-09-20T22:06:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6110000000000001e-06 2024-09-20T22:06:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.071875+0000", 2024-09-20T22:06:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3784e-05 2024-09-20T22:06:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:15.978 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.071959+0000", 2024-09-20T22:06:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3623999999999998e-05 2024-09-20T22:06:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:06:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.072032+0000", 2024-09-20T22:06:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3553000000000005e-05 2024-09-20T22:06:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.073485+0000", 2024-09-20T22:06:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001453006 2024-09-20T22:06:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.979 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.077796+0000", 2024-09-20T22:06:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.004310258 2024-09-20T22:06:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.149986+0000", 2024-09-20T22:06:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.072190693 2024-09-20T22:06:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.150038+0000", 2024-09-20T22:06:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1208000000000002e-05 2024-09-20T22:06:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.980 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.150078+0000", 2024-09-20T22:06:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0277000000000001e-05 2024-09-20T22:06:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:32205 7.d 7:b8477c90:::benchmark_data_smithi060_78800_object32204:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e89)", 2024-09-20T22:06:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:11.074949+0000", 2024-09-20T22:06:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 124.87737443499999, 2024-09-20T22:06:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.074949680000000005, 2024-09-20T22:06:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:15.981 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 32205 2024-09-20T22:06:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.074949+0000", 2024-09-20T22:06:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.982 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.983 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:15.983 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.074949+0000", 2024-09-20T22:06:15.983 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:15.983 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.983 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.983 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:15.984 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:15.984 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.074951+0000", 2024-09-20T22:06:15.984 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0789999999999999e-06 2024-09-20T22:06:15.984 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.984 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075006+0000", 2024-09-20T22:06:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4608000000000003e-05 2024-09-20T22:06:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075015+0000", 2024-09-20T22:06:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6649999999999998e-06 2024-09-20T22:06:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075020+0000", 2024-09-20T22:06:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9729999999999997e-06 2024-09-20T22:06:15.985 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075091+0000", 2024-09-20T22:06:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1218999999999997e-05 2024-09-20T22:06:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075222+0000", 2024-09-20T22:06:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013064399999999999 2024-09-20T22:06:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1", 2024-09-20T22:06:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.075363+0000", 2024-09-20T22:06:15.986 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014131300000000001 2024-09-20T22:06:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.076629+0000", 2024-09-20T22:06:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0012658910000000001 2024-09-20T22:06:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.078895+0000", 2024-09-20T22:06:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0022661040000000001 2024-09-20T22:06:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:15.987 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.149797+0000", 2024-09-20T22:06:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.070901986 2024-09-20T22:06:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.149852+0000", 2024-09-20T22:06:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4185999999999999e-05 2024-09-20T22:06:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:11.149899+0000", 2024-09-20T22:06:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7012000000000003e-05 2024-09-20T22:06:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:15.988 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:46036 7.7 7:e5f37aca:::benchmark_data_smithi060_78800_object46035:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T22:06:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:19.394201+0000", 2024-09-20T22:06:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 116.558122778, 2024-09-20T22:06:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.079244819999999994, 2024-09-20T22:06:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 46036 2024-09-20T22:06:15.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394201+0000", 2024-09-20T22:06:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394201+0000", 2024-09-20T22:06:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:15.990 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394202+0000", 2024-09-20T22:06:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.344e-06 2024-09-20T22:06:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394268+0000", 2024-09-20T22:06:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6047e-05 2024-09-20T22:06:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394282+0000", 2024-09-20T22:06:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3967e-05 2024-09-20T22:06:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.991 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394288+0000", 2024-09-20T22:06:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3199999999999996e-06 2024-09-20T22:06:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394356+0000", 2024-09-20T22:06:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7529999999999999e-05 2024-09-20T22:06:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394470+0000", 2024-09-20T22:06:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000113857 2024-09-20T22:06:15.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:06:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394552+0000", 2024-09-20T22:06:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2128999999999999e-05 2024-09-20T22:06:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.395833+0000", 2024-09-20T22:06:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0012817639999999999 2024-09-20T22:06:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:15.993 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.471500+0000", 2024-09-20T22:06:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.075666232 2024-09-20T22:06:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.473346+0000", 2024-09-20T22:06:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00184637 2024-09-20T22:06:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.473405+0000", 2024-09-20T22:06:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9298e-05 2024-09-20T22:06:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.994 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.473445+0000", 2024-09-20T22:06:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9962000000000001e-05 2024-09-20T22:06:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:46037 7.d 7:b4603dba:::benchmark_data_smithi060_78800_object46036:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T22:06:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:19.394327+0000", 2024-09-20T22:06:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 116.557996703, 2024-09-20T22:06:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.078329847999999994, 2024-09-20T22:06:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:15.995 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 46037 2024-09-20T22:06:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394327+0000", 2024-09-20T22:06:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394327+0000", 2024-09-20T22:06:15.996 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394327+0000", 2024-09-20T22:06:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8700000000000005e-07 2024-09-20T22:06:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394406+0000", 2024-09-20T22:06:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8265999999999999e-05 2024-09-20T22:06:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:15.997 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394413+0000", 2024-09-20T22:06:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9890000000000003e-06 2024-09-20T22:06:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394417+0000", 2024-09-20T22:06:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4329999999999997e-06 2024-09-20T22:06:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394475+0000", 2024-09-20T22:06:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7627000000000002e-05 2024-09-20T22:06:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.998 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394605+0000", 2024-09-20T22:06:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013037199999999999 2024-09-20T22:06:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0", 2024-09-20T22:06:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.394762+0000", 2024-09-20T22:06:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015654300000000001 2024-09-20T22:06:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.395999+0000", 2024-09-20T22:06:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001237036 2024-09-20T22:06:15.999 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.472495+0000", 2024-09-20T22:06:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.076496185999999994 2024-09-20T22:06:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.472558+0000", 2024-09-20T22:06:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2849999999999996e-05 2024-09-20T22:06:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.472656+0000", 2024-09-20T22:06:16.000 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8858999999999997e-05 2024-09-20T22:06:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:98048 7.7 7:e68db34e:::benchmark_data_smithi060_78800_object98047:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-20T22:06:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.311849+0000", 2024-09-20T22:06:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 79.640474592000004, 2024-09-20T22:06:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.082878767000000006, 2024-09-20T22:06:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:16.001 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 98048 2024-09-20T22:06:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.311849+0000", 2024-09-20T22:06:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.311849+0000", 2024-09-20T22:06:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.002 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.311850+0000", 2024-09-20T22:06:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5e-06 2024-09-20T22:06:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.311878+0000", 2024-09-20T22:06:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7756999999999999e-05 2024-09-20T22:06:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.311891+0000", 2024-09-20T22:06:16.003 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3322000000000001e-05 2024-09-20T22:06:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.311899+0000", 2024-09-20T22:06:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5340000000000002e-06 2024-09-20T22:06:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.312832+0000", 2024-09-20T22:06:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00093272699999999995 2024-09-20T22:06:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:16.004 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.312960+0000", 2024-09-20T22:06:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012859699999999999 2024-09-20T22:06:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:06:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.313105+0000", 2024-09-20T22:06:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014509700000000001 2024-09-20T22:06:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.316108+0000", 2024-09-20T22:06:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0030024449999999999 2024-09-20T22:06:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.005 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.394570+0000", 2024-09-20T22:06:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.078462660000000004 2024-09-20T22:06:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.394622+0000", 2024-09-20T22:06:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1662000000000001e-05 2024-09-20T22:06:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.394679+0000", 2024-09-20T22:06:16.006 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.664e-05 2024-09-20T22:06:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.394727+0000", 2024-09-20T22:06:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8826000000000003e-05 2024-09-20T22:06:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:98055 7.c 7:30ecbb09:::benchmark_data_smithi060_78800_object98054:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-20T22:06:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.318853+0000", 2024-09-20T22:06:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 79.633470166999999, 2024-09-20T22:06:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.090772817000000006, 2024-09-20T22:06:16.007 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 98055 2024-09-20T22:06:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:16.008 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.318853+0000", 2024-09-20T22:06:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.318853+0000", 2024-09-20T22:06:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.318854+0000", 2024-09-20T22:06:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5900000000000005e-07 2024-09-20T22:06:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.009 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.318908+0000", 2024-09-20T22:06:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3850000000000001e-05 2024-09-20T22:06:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.318919+0000", 2024-09-20T22:06:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0708e-05 2024-09-20T22:06:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.318925+0000", 2024-09-20T22:06:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0820000000000004e-06 2024-09-20T22:06:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.010 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.318970+0000", 2024-09-20T22:06:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5626e-05 2024-09-20T22:06:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319091+0000", 2024-09-20T22:06:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000120569 2024-09-20T22:06:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:06:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319226+0000", 2024-09-20T22:06:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000134578 2024-09-20T22:06:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.011 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.397781+0000", 2024-09-20T22:06:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.078555884000000006 2024-09-20T22:06:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.398333+0000", 2024-09-20T22:06:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00055192399999999997 2024-09-20T22:06:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.012 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.409527+0000", 2024-09-20T22:06:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.011193318000000001 2024-09-20T22:06:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.409580+0000", 2024-09-20T22:06:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3573999999999999e-05 2024-09-20T22:06:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.409626+0000", 2024-09-20T22:06:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5745000000000003e-05 2024-09-20T22:06:16.013 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:98057 7.d 7:b86bb00f:::benchmark_data_smithi060_78800_object98056:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-20T22:06:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.319112+0000", 2024-09-20T22:06:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 79.633211462000006, 2024-09-20T22:06:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.089241666999999997, 2024-09-20T22:06:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:16.014 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 98057 2024-09-20T22:06:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319112+0000", 2024-09-20T22:06:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319112+0000", 2024-09-20T22:06:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.015 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319113+0000", 2024-09-20T22:06:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1399999999999995e-07 2024-09-20T22:06:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319151+0000", 2024-09-20T22:06:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7808999999999997e-05 2024-09-20T22:06:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319159+0000", 2024-09-20T22:06:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.7739999999999993e-06 2024-09-20T22:06:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.016 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319164+0000", 2024-09-20T22:06:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8890000000000001e-06 2024-09-20T22:06:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319227+0000", 2024-09-20T22:06:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2630000000000002e-05 2024-09-20T22:06:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.394852+0000", 2024-09-20T22:06:16.017 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.075624780000000003 2024-09-20T22:06:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0", 2024-09-20T22:06:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.395003+0000", 2024-09-20T22:06:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000151745 2024-09-20T22:06:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.407269+0000", 2024-09-20T22:06:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.012265507 2024-09-20T22:06:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.018 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.408228+0000", 2024-09-20T22:06:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000958952 2024-09-20T22:06:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.408280+0000", 2024-09-20T22:06:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2077000000000003e-05 2024-09-20T22:06:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.408354+0000", 2024-09-20T22:06:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3590000000000005e-05 2024-09-20T22:06:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.019 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:98058 7.7 7:e4cc42e2:::benchmark_data_smithi060_78800_object98057:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-20T22:06:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.319192+0000", 2024-09-20T22:06:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 79.633131605000003, 2024-09-20T22:06:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.093554769999999995, 2024-09-20T22:06:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 98058 2024-09-20T22:06:16.020 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319192+0000", 2024-09-20T22:06:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319192+0000", 2024-09-20T22:06:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:16.021 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319192+0000", 2024-09-20T22:06:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6800000000000001e-07 2024-09-20T22:06:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.319261+0000", 2024-09-20T22:06:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8670999999999994e-05 2024-09-20T22:06:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.394159+0000", 2024-09-20T22:06:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.074898163000000004 2024-09-20T22:06:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.022 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.394172+0000", 2024-09-20T22:06:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.312e-05 2024-09-20T22:06:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.394758+0000", 2024-09-20T22:06:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00058551700000000003 2024-09-20T22:06:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.394898+0000", 2024-09-20T22:06:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000140044 2024-09-20T22:06:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.023 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:06:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.395042+0000", 2024-09-20T22:06:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000144418 2024-09-20T22:06:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.398913+0000", 2024-09-20T22:06:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003870975 2024-09-20T22:06:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.406960+0000", 2024-09-20T22:06:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0080465499999999995 2024-09-20T22:06:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.024 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:16.025 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.412648+0000", 2024-09-20T22:06:16.025 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0056887320000000002 2024-09-20T22:06:16.025 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.025 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.025 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:16.025 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.412701+0000", 2024-09-20T22:06:16.026 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2253000000000003e-05 2024-09-20T22:06:16.026 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.026 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.026 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:16.026 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.412746+0000", 2024-09-20T22:06:16.026 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5859000000000003e-05 2024-09-20T22:06:16.026 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:98517 7.d 7:bd9e5a0c:::benchmark_data_smithi060_78800_object98516:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-20T22:06:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.751855+0000", 2024-09-20T22:06:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 79.200467889999999, 2024-09-20T22:06:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.081104983000000005, 2024-09-20T22:06:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:16.027 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 98517 2024-09-20T22:06:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.751855+0000", 2024-09-20T22:06:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.751855+0000", 2024-09-20T22:06:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.028 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.751857+0000", 2024-09-20T22:06:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3510000000000001e-06 2024-09-20T22:06:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.751910+0000", 2024-09-20T22:06:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3056999999999997e-05 2024-09-20T22:06:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.751918+0000", 2024-09-20T22:06:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9319999999999996e-06 2024-09-20T22:06:16.029 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.751922+0000", 2024-09-20T22:06:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5859999999999998e-06 2024-09-20T22:06:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.751962+0000", 2024-09-20T22:06:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9771000000000003e-05 2024-09-20T22:06:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752044+0000", 2024-09-20T22:06:16.030 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1988000000000002e-05 2024-09-20T22:06:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0", 2024-09-20T22:06:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752136+0000", 2024-09-20T22:06:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1710000000000001e-05 2024-09-20T22:06:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.831045+0000", 2024-09-20T22:06:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.078909149999999997 2024-09-20T22:06:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.031 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.832819+0000", 2024-09-20T22:06:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001773791 2024-09-20T22:06:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.832873+0000", 2024-09-20T22:06:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4725999999999997e-05 2024-09-20T22:06:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.832960+0000", 2024-09-20T22:06:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6921000000000003e-05 2024-09-20T22:06:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:16.032 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:98518 7.7 7:e03a258a:::benchmark_data_smithi060_78800_object98517:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-20T22:06:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.752091+0000", 2024-09-20T22:06:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 79.200232542999998, 2024-09-20T22:06:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.092212638999999999, 2024-09-20T22:06:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 98518 2024-09-20T22:06:16.033 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752091+0000", 2024-09-20T22:06:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752091+0000", 2024-09-20T22:06:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:16.034 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752091+0000", 2024-09-20T22:06:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2499999999999995e-07 2024-09-20T22:06:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752154+0000", 2024-09-20T22:06:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2951000000000001e-05 2024-09-20T22:06:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752161+0000", 2024-09-20T22:06:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4880000000000004e-06 2024-09-20T22:06:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:16.035 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752165+0000", 2024-09-20T22:06:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9230000000000004e-06 2024-09-20T22:06:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752212+0000", 2024-09-20T22:06:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6875000000000001e-05 2024-09-20T22:06:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.827647+0000", 2024-09-20T22:06:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.075435385999999993 2024-09-20T22:06:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.036 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:06:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.827791+0000", 2024-09-20T22:06:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014404199999999999 2024-09-20T22:06:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.831115+0000", 2024-09-20T22:06:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0033237679999999999 2024-09-20T22:06:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.831125+0000", 2024-09-20T22:06:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0281e-05 2024-09-20T22:06:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.037 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.844201+0000", 2024-09-20T22:06:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.01307559 2024-09-20T22:06:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.844259+0000", 2024-09-20T22:06:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8661e-05 2024-09-20T22:06:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.844303+0000", 2024-09-20T22:06:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4149e-05 2024-09-20T22:06:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.038 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:98519 7.c 7:32ce32d4:::benchmark_data_smithi060_78800_object98518:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-20T22:06:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.752356+0000", 2024-09-20T22:06:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 79.199967674000007, 2024-09-20T22:06:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.089684246999999995, 2024-09-20T22:06:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:16.039 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 98519 2024-09-20T22:06:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752356+0000", 2024-09-20T22:06:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752356+0000", 2024-09-20T22:06:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:16.040 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752356+0000", 2024-09-20T22:06:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5000000000000003e-07 2024-09-20T22:06:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.752430+0000", 2024-09-20T22:06:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4054999999999995e-05 2024-09-20T22:06:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.827861+0000", 2024-09-20T22:06:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.075431127000000001 2024-09-20T22:06:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.041 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.827875+0000", 2024-09-20T22:06:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3183999999999999e-05 2024-09-20T22:06:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.827939+0000", 2024-09-20T22:06:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4628999999999994e-05 2024-09-20T22:06:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.828085+0000", 2024-09-20T22:06:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014621799999999999 2024-09-20T22:06:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.042 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:06:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.828222+0000", 2024-09-20T22:06:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000136379 2024-09-20T22:06:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.831909+0000", 2024-09-20T22:06:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0036872900000000002 2024-09-20T22:06:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.832037+0000", 2024-09-20T22:06:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012829399999999999 2024-09-20T22:06:16.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.841930+0000", 2024-09-20T22:06:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0098926190000000001 2024-09-20T22:06:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.841991+0000", 2024-09-20T22:06:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.101e-05 2024-09-20T22:06:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.842040+0000", 2024-09-20T22:06:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8891999999999999e-05 2024-09-20T22:06:16.044 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.24667.0:98529 7.b e106/91 7:d59e89dd:::benchmark_data_smithi060_78800_object98528:head v 106'6110, mlcod=106'6108)", 2024-09-20T22:06:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.760619+0000", 2024-09-20T22:06:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 79.191704103000006, 2024-09-20T22:06:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.076810604000000005, 2024-09-20T22:06:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:16.045 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760619+0000", 2024-09-20T22:06:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760619+0000", 2024-09-20T22:06:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:16.046 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760620+0000", 2024-09-20T22:06:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.018e-06 2024-09-20T22:06:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760642+0000", 2024-09-20T22:06:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1852e-05 2024-09-20T22:06:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.830718+0000", 2024-09-20T22:06:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.070075838000000001 2024-09-20T22:06:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.047 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.830727+0000", 2024-09-20T22:06:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3070000000000008e-06 2024-09-20T22:06:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.830799+0000", 2024-09-20T22:06:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1675999999999996e-05 2024-09-20T22:06:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.830821+0000", 2024-09-20T22:06:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2291000000000001e-05 2024-09-20T22:06:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.048 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.837103+0000", 2024-09-20T22:06:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0062819729999999997 2024-09-20T22:06:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.837430+0000", 2024-09-20T22:06:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00032664899999999998 2024-09-20T22:06:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:99601 7.d 7:ba9751c2:::benchmark_data_smithi060_78800_object99600:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-20T22:06:16.049 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:57.738365+0000", 2024-09-20T22:06:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 78.213958624, 2024-09-20T22:06:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.083724046999999996, 2024-09-20T22:06:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 99601 2024-09-20T22:06:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:16.050 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.738365+0000", 2024-09-20T22:06:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.738365+0000", 2024-09-20T22:06:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.738366+0000", 2024-09-20T22:06:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0330000000000001e-06 2024-09-20T22:06:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.051 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:16.052 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.738419+0000", 2024-09-20T22:06:16.052 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3532999999999997e-05 2024-09-20T22:06:16.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.053 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.053 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:16.053 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.738428+0000", 2024-09-20T22:06:16.053 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6710000000000005e-06 2024-09-20T22:06:16.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.053 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.738433+0000", 2024-09-20T22:06:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7260000000000002e-06 2024-09-20T22:06:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.738480+0000", 2024-09-20T22:06:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7423000000000003e-05 2024-09-20T22:06:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.738556+0000", 2024-09-20T22:06:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5673999999999994e-05 2024-09-20T22:06:16.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0", 2024-09-20T22:06:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.738640+0000", 2024-09-20T22:06:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4581999999999998e-05 2024-09-20T22:06:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.741237+0000", 2024-09-20T22:06:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002596807 2024-09-20T22:06:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.821932+0000", 2024-09-20T22:06:16.055 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.080694504 2024-09-20T22:06:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.821993+0000", 2024-09-20T22:06:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1396999999999995e-05 2024-09-20T22:06:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.822089+0000", 2024-09-20T22:06:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5697000000000002e-05 2024-09-20T22:06:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:99607 7.d 7:b7d673d2:::benchmark_data_smithi060_78800_object99606:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-20T22:06:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:57.744424+0000", 2024-09-20T22:06:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 78.207899495000007, 2024-09-20T22:06:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.077971463000000005, 2024-09-20T22:06:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 99607 2024-09-20T22:06:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:16.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.744424+0000", 2024-09-20T22:06:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.744424+0000", 2024-09-20T22:06:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.744425+0000", 2024-09-20T22:06:16.058 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1399999999999995e-07 2024-09-20T22:06:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.744476+0000", 2024-09-20T22:06:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0834999999999997e-05 2024-09-20T22:06:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.744484+0000", 2024-09-20T22:06:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6920000000000008e-06 2024-09-20T22:06:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.059 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.744489+0000", 2024-09-20T22:06:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7609999999999998e-06 2024-09-20T22:06:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.744532+0000", 2024-09-20T22:06:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3195999999999998e-05 2024-09-20T22:06:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.744618+0000", 2024-09-20T22:06:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6069999999999994e-05 2024-09-20T22:06:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0", 2024-09-20T22:06:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.744722+0000", 2024-09-20T22:06:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010355 2024-09-20T22:06:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.746896+0000", 2024-09-20T22:06:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002173843 2024-09-20T22:06:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.822243+0000", 2024-09-20T22:06:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.075347757000000001 2024-09-20T22:06:16.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.822298+0000", 2024-09-20T22:06:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4648999999999998e-05 2024-09-20T22:06:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.822395+0000", 2024-09-20T22:06:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7195999999999996e-05 2024-09-20T22:06:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:185011 7.7 7:e5ecae64:::benchmark_data_smithi060_78800_object51978:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-20T22:06:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:13.519863+0000", 2024-09-20T22:06:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 2.4324601659999998, 2024-09-20T22:06:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.091476746999999997, 2024-09-20T22:06:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 185011 2024-09-20T22:06:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.063 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:13.519863+0000", 2024-09-20T22:06:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:13.519863+0000", 2024-09-20T22:06:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:13.519865+0000", 2024-09-20T22:06:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5e-06 2024-09-20T22:06:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.064 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:13.519869+0000", 2024-09-20T22:06:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7049999999999998e-06 2024-09-20T22:06:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:13.519882+0000", 2024-09-20T22:06:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2821e-05 2024-09-20T22:06:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:16.065 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:13.519891+0000", 2024-09-20T22:06:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.0399999999999998e-06 2024-09-20T22:06:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:13.519945+0000", 2024-09-20T22:06:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3884000000000001e-05 2024-09-20T22:06:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:13.520161+0000", 2024-09-20T22:06:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00021546 2024-09-20T22:06:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.066 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:06:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:13.520720+0000", 2024-09-20T22:06:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00055939399999999995 2024-09-20T22:06:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:13.522245+0000", 2024-09-20T22:06:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0015248589999999999 2024-09-20T22:06:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:13.578565+0000", 2024-09-20T22:06:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.056320275000000003 2024-09-20T22:06:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.067 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:13.611220+0000", 2024-09-20T22:06:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.032655002000000002 2024-09-20T22:06:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:13.611275+0000", 2024-09-20T22:06:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5368999999999998e-05 2024-09-20T22:06:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:13.611340+0000", 2024-09-20T22:06:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4437999999999995e-05 2024-09-20T22:06:16.068 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:185616 7.c 7:3b5c0a23:::benchmark_data_smithi060_78800_object52583:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-20T22:06:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:14.042032+0000", 2024-09-20T22:06:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.910291505, 2024-09-20T22:06:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.080282892999999994, 2024-09-20T22:06:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:16.069 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 185616 2024-09-20T22:06:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.042032+0000", 2024-09-20T22:06:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.042032+0000", 2024-09-20T22:06:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.070 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.042033+0000", 2024-09-20T22:06:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5199999999999995e-07 2024-09-20T22:06:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.042038+0000", 2024-09-20T22:06:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9760000000000003e-06 2024-09-20T22:06:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.042049+0000", 2024-09-20T22:06:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1242e-05 2024-09-20T22:06:16.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.042056+0000", 2024-09-20T22:06:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9110000000000004e-06 2024-09-20T22:06:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.042419+0000", 2024-09-20T22:06:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00036318800000000002 2024-09-20T22:06:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.042527+0000", 2024-09-20T22:06:16.072 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000107682 2024-09-20T22:06:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:06:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.042650+0000", 2024-09-20T22:06:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012368700000000001 2024-09-20T22:06:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.045117+0000", 2024-09-20T22:06:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002466111 2024-09-20T22:06:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.073 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.045351+0000", 2024-09-20T22:06:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00023475300000000001 2024-09-20T22:06:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.122207+0000", 2024-09-20T22:06:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.076856010000000002 2024-09-20T22:06:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.122268+0000", 2024-09-20T22:06:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1017000000000002e-05 2024-09-20T22:06:16.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.122315+0000", 2024-09-20T22:06:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6363999999999999e-05 2024-09-20T22:06:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:186317 7.d 7:bbd5fb02:::benchmark_data_smithi060_78800_object53284:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-20T22:06:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:14.637161+0000", 2024-09-20T22:06:16.075 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.3151621849999999, 2024-09-20T22:06:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.076624783000000002, 2024-09-20T22:06:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 186317 2024-09-20T22:06:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:16.076 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.637161+0000", 2024-09-20T22:06:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.637161+0000", 2024-09-20T22:06:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.637162+0000", 2024-09-20T22:06:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.173e-06 2024-09-20T22:06:16.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.637167+0000", 2024-09-20T22:06:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6489999999999999e-06 2024-09-20T22:06:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.637179+0000", 2024-09-20T22:06:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1555e-05 2024-09-20T22:06:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.637185+0000", 2024-09-20T22:06:16.078 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5950000000000004e-06 2024-09-20T22:06:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.637233+0000", 2024-09-20T22:06:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8140000000000003e-05 2024-09-20T22:06:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.637314+0000", 2024-09-20T22:06:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1027999999999997e-05 2024-09-20T22:06:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.079 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0", 2024-09-20T22:06:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.637401+0000", 2024-09-20T22:06:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6767999999999993e-05 2024-09-20T22:06:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.640319+0000", 2024-09-20T22:06:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0029177299999999999 2024-09-20T22:06:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.713610+0000", 2024-09-20T22:06:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.073291545999999999 2024-09-20T22:06:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.080 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.713694+0000", 2024-09-20T22:06:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3285e-05 2024-09-20T22:06:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:14.713786+0000", 2024-09-20T22:06:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2313999999999997e-05 2024-09-20T22:06:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:16.081 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:16.081 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:06:16.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:16.083 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:16.083 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:16.184 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:16.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:16.239 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:16.240 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:16.340 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:16.380 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:16.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:16.381 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:16.481 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:16.511 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:16.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:16.512 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:16.613 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:16.639 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:16.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:16.640 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:16.733 INFO:teuthology.orchestra.run.smithi060.stderr:instructing pg(s) [4.6,4.b,7.4,7.9,7.19] on osd.0 to force-recovery; instructing pg(s) [4.4,4.1c] on osd.2 to force-recovery; instructing pg(s) [3.19,4.f,4.14,4.17,4.1d,5.0,7.2,7.10] on osd.3 to force-recovery; instructing pg(s) [4.9,4.d,4.13,4.18] on osd.4 to force-recovery; instructing pg(s) [4.7,7.0,7.b] on osd.5 to force-recovery; instructing pg(s) [4.a,4.12,4.1f] on osd.6 to force-recovery; instructing pg(s) [2.0] on osd.7 to force-recovery; 2024-09-20T22:06:16.733 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-09-20T22:06:16.740 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:16.770 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:16.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:16.771 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:16.872 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:16.904 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:16.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:16.905 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:17.000 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:06:17.000 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:06:17.002 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:06:17.005 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:17.041 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:06:17.041 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:06:17.047 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:17.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:17.049 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:17.082 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:06:17.082 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:06:17.118 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:06:17.118 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:06:17.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:17.156 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:06:17.157 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:06:17.157 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:17 smithi060 ceph-mon[29694]: pgmap v147: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 4.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.51k op/s; 28100/235485 objects degraded (11.933%); 36234/235485 objects misplaced (15.387%); 26 MiB/s, 415 objects/s recovering 2024-09-20T22:06:17.157 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:17 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 28100/235485 objects degraded (11.933%), 36 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:17.157 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:17 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2036379686' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:17.158 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:17 smithi060 ceph-mon[40290]: pgmap v147: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 4.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.51k op/s; 28100/235485 objects degraded (11.933%); 36234/235485 objects misplaced (15.387%); 26 MiB/s, 415 objects/s recovering 2024-09-20T22:06:17.158 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:17 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 28100/235485 objects degraded (11.933%), 36 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:17.158 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:17 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2036379686' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:17.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:17.190 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:17.191 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:17.201 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:06:17.201 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:06:17.237 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:06:17.238 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:06:17.292 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:17.330 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:17.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:17.331 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:17.396 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:06:17.432 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:17.486 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:17.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:17.487 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:17.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:17 smithi067 ceph-mon[33511]: pgmap v147: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 4.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.51k op/s; 28100/235485 objects degraded (11.933%); 36234/235485 objects misplaced (15.387%); 26 MiB/s, 415 objects/s recovering 2024-09-20T22:06:17.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:17 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 28100/235485 objects degraded (11.933%), 36 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:17.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:17 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2036379686' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:17.588 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:17.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:17.625 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:17.625 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:17.726 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:17.754 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:17.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:17.755 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:17.855 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:17.887 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:17.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:17.888 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:17.989 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:18.038 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:18.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:18.039 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:18.140 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:18.173 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:18.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:18.174 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:18.275 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:18.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:18 smithi060 ceph-mon[29694]: from='client.24869 -' entity='client.admin' cmd=[{"prefix": "pg force-recovery", "pgid": ["4.18", "7.19", "4.1c", "4.1d", "3.19", "4.f", "7.2", "5.0", "7.0", "4.4", "7.4", "4.7", "4.6", "2.0", "4.d", "4.b", "7.9", "4.a", "4.9", "7.b", "4.17", "4.14", "7.10", "4.13", "4.12", "4.1f"], "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:06:18.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:18 smithi060 ceph-mon[40290]: from='client.24869 -' entity='client.admin' cmd=[{"prefix": "pg force-recovery", "pgid": ["4.18", "7.19", "4.1c", "4.1d", "3.19", "4.f", "7.2", "5.0", "7.0", "4.4", "7.4", "4.7", "4.6", "2.0", "4.d", "4.b", "7.9", "4.a", "4.9", "7.b", "4.17", "4.14", "7.10", "4.13", "4.12", "4.1f"], "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:06:18.304 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:18.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:18.305 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:18.406 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:18.422 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:06:18.423 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:06:18.447 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:18.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:18.448 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:18.449 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:18.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:18 smithi067 ceph-mon[33511]: from='client.24869 -' entity='client.admin' cmd=[{"prefix": "pg force-recovery", "pgid": ["4.18", "7.19", "4.1c", "4.1d", "3.19", "4.f", "7.2", "5.0", "7.0", "4.4", "7.4", "4.7", "4.6", "2.0", "4.d", "4.b", "7.9", "4.a", "4.9", "7.b", "4.17", "4.14", "7.10", "4.13", "4.12", "4.1f"], "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:06:18.548 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:18.583 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:18.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:18.584 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:18.648 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T22:06:18.685 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:18.727 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:18.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:18.728 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:18.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:18.865 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:18.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:18.866 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:18.967 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:19.004 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:19.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:19.005 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:19.031 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:06:19.105 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:19.135 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:19.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:19.136 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:19.237 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:19.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:19.277 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:19.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:19 smithi060 ceph-mon[29694]: pgmap v148: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 4.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1022 B/s rd, 0 B/s wr, 1.22k op/s; 28100/235485 objects degraded (11.933%); 36234/235485 objects misplaced (15.387%); 21 MiB/s, 334 objects/s recovering 2024-09-20T22:06:19.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:19 smithi060 ceph-mon[40290]: pgmap v148: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 4.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1022 B/s rd, 0 B/s wr, 1.22k op/s; 28100/235485 objects degraded (11.933%); 36234/235485 objects misplaced (15.387%); 21 MiB/s, 334 objects/s recovering 2024-09-20T22:06:19.378 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:19.405 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:19.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:19.406 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:19.421 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:19 smithi067 ceph-mon[33511]: pgmap v148: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 4.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1022 B/s rd, 0 B/s wr, 1.22k op/s; 28100/235485 objects degraded (11.933%); 36234/235485 objects misplaced (15.387%); 21 MiB/s, 334 objects/s recovering 2024-09-20T22:06:19.507 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:19.543 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:19.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:19.544 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:19.645 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:19.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:19.681 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:19.682 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:19.782 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:19.812 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:19.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:19.813 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:19.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:19.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:19.942 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:19.942 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:20.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:20.082 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:20.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:20.083 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:20.098 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:06:20.184 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:20.216 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:20.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:20.217 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:20.318 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:20.348 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:20.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:20.350 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:20.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:06:20.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:20.462 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:20 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2869731371' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:06:20.462 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:20 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:06:20.462 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:20.462 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:06:20.463 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:20 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2869731371' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:06:20.463 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:20 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:06:20.463 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:20.463 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:06:20.479 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:20.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:20.480 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:20.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:20 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2869731371' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:06:20.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:20 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:06:20.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:20.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:06:20.581 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:20.628 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:20.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:20.629 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:20.730 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:20.761 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:20.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:20.762 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:20.863 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:20.892 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:20.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:20.893 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:20.916 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:20.968 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:06:20.968 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:06:20.994 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:21.028 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:21.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:21.029 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:21.130 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:21.165 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:21.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:21.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:21.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:21 smithi067 ceph-mon[33511]: pgmap v149: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 4.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1019 B/s rd, 0 B/s wr, 1.22k op/s; 28100/235485 objects degraded (11.933%); 36234/235485 objects misplaced (15.387%); 21 MiB/s, 333 objects/s recovering 2024-09-20T22:06:21.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:21 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:06:21.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:21 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:06:21.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:21 smithi067 ceph-mon[33511]: osdmap e130: 8 total, 7 up, 8 in 2024-09-20T22:06:21.267 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:21.288 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:21.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:21.289 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:21.390 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:21.424 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:21.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:21.425 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:21.526 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:21.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:21 smithi060 ceph-mon[29694]: pgmap v149: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 4.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1019 B/s rd, 0 B/s wr, 1.22k op/s; 28100/235485 objects degraded (11.933%); 36234/235485 objects misplaced (15.387%); 21 MiB/s, 333 objects/s recovering 2024-09-20T22:06:21.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:21 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:06:21.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:21 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:06:21.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:21 smithi060 ceph-mon[29694]: osdmap e130: 8 total, 7 up, 8 in 2024-09-20T22:06:21.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:21 smithi060 ceph-mon[40290]: pgmap v149: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 3 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 4.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1019 B/s rd, 0 B/s wr, 1.22k op/s; 28100/235485 objects degraded (11.933%); 36234/235485 objects misplaced (15.387%); 21 MiB/s, 333 objects/s recovering 2024-09-20T22:06:21.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:21 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:06:21.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:21 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:06:21.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:21 smithi060 ceph-mon[40290]: osdmap e130: 8 total, 7 up, 8 in 2024-09-20T22:06:21.556 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:21.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:21.557 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:21.658 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:21.691 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:21.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:21.692 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:21.793 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:21.835 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:21.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:21.836 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:21.840 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T22:06:21.936 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:21.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:21.978 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:21.979 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:22.079 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:22.115 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:22.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:22.117 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:22.217 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:22.223 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:06:22.240 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:22.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:22.241 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:22.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:06:21 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:06:22.342 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:22.372 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:22.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:22.373 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:22.474 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:22.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:22.497 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:22.497 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:22.598 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:22.640 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:22.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:22.641 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:22.671 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:22 smithi067 ceph-mon[33511]: osdmap e131: 8 total, 7 up, 8 in 2024-09-20T22:06:22.683 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:22 smithi060 ceph-mon[29694]: osdmap e131: 8 total, 7 up, 8 in 2024-09-20T22:06:22.683 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:22 smithi060 ceph-mon[40290]: osdmap e131: 8 total, 7 up, 8 in 2024-09-20T22:06:22.742 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:22.776 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:22.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:22.777 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:22.793 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3, 4, 5, 6, 7] 2024-09-20T22:06:22.793 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:06:22.793 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-20T22:06:22.793 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-20T22:06:22.823 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:06:22.823 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:06:22.830 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:06:22.878 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:22.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:22.906 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:22.906 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:23.007 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:23.033 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:23.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:23.034 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:23.136 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:23.164 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:23.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:23.165 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:23.266 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:23.291 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:23.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:23.292 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:23.393 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:23.454 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:23.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:23.456 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:23.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:23 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:06:23.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:23 smithi067 ceph-mon[33511]: pgmap v152: 158 pgs: 1 active+forced_recovery+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 4.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.53k op/s; 25351/216675 objects degraded (11.700%); 31230/216675 objects misplaced (14.413%); 26 MiB/s, 413 objects/s recovering 2024-09-20T22:06:23.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:23 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 25351/216675 objects degraded (11.700%), 36 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:23.556 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:23.592 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:23.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:23.593 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:23.605 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:23 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:06:23.605 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:23 smithi060 ceph-mon[29694]: pgmap v152: 158 pgs: 1 active+forced_recovery+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 4.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.53k op/s; 25351/216675 objects degraded (11.700%); 31230/216675 objects misplaced (14.413%); 26 MiB/s, 413 objects/s recovering 2024-09-20T22:06:23.606 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:23 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 25351/216675 objects degraded (11.700%), 36 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:23.606 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:23 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:06:23.606 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:23 smithi060 ceph-mon[40290]: pgmap v152: 158 pgs: 1 active+forced_recovery+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 4.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.53k op/s; 25351/216675 objects degraded (11.700%); 31230/216675 objects misplaced (14.413%); 26 MiB/s, 413 objects/s recovering 2024-09-20T22:06:23.606 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:23 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 25351/216675 objects degraded (11.700%), 36 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:23.694 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:23.729 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T22:06:23.735 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:23.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:23.736 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:23.837 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:23.874 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:23.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:23.875 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:23.976 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:24.014 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:24.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:24.016 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:24.098 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:06:24.116 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:24.138 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:24.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:24.139 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:24.239 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:24.274 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:24.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:24.275 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:24.330 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:06:24.331 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:06:24.331 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:06:24.331 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:06:24.331 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:06:24.331 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:06:24.331 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:06:24.331 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:06:24.332 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:06:24.332 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:06:24.332 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:06:24.332 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:06:24.332 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:06:24.332 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:06:24.332 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:06:24.376 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:24.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:24.418 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:24.418 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:24.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:24 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/247589301' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:24.519 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:24.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:24.560 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:24.560 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:24.660 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:24.686 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:24.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:24.687 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:24.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:24 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/247589301' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:24.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:24 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/247589301' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:24.787 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:24.823 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:24.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:24.824 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:24.925 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:24.948 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:24.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:24.949 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:24.993 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:25.001 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:25.050 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:25.058 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:06:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:06:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:06:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:06:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:22523 7.5 7:a7080f75:::benchmark_data_smithi060_78800_object22522:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:06:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:06.083191+0000", 2024-09-20T22:06:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 138.577802967, 2024-09-20T22:06:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.089521636000000002, 2024-09-20T22:06:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:25.059 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:25.060 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 22523 2024-09-20T22:06:25.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.060 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.060 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.060 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083191+0000", 2024-09-20T22:06:25.060 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.060 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083191+0000", 2024-09-20T22:06:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083192+0000", 2024-09-20T22:06:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5200000000000001e-06 2024-09-20T22:06:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083288+0000", 2024-09-20T22:06:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5377999999999994e-05 2024-09-20T22:06:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083310+0000", 2024-09-20T22:06:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2943e-05 2024-09-20T22:06:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083318+0000", 2024-09-20T22:06:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4240000000000004e-06 2024-09-20T22:06:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083630+0000", 2024-09-20T22:06:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000311883 2024-09-20T22:06:25.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083768+0000", 2024-09-20T22:06:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000137918 2024-09-20T22:06:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:06:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.083906+0000", 2024-09-20T22:06:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013816800000000001 2024-09-20T22:06:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.084597+0000", 2024-09-20T22:06:25.063 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00069096000000000001 2024-09-20T22:06:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.086112+0000", 2024-09-20T22:06:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0015148690000000001 2024-09-20T22:06:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.172612+0000", 2024-09-20T22:06:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.086500683999999994 2024-09-20T22:06:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:25.064 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.172669+0000", 2024-09-20T22:06:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6591000000000001e-05 2024-09-20T22:06:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.172712+0000", 2024-09-20T22:06:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3297999999999998e-05 2024-09-20T22:06:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:22527 7.5 7:a1def41c:::benchmark_data_smithi060_78800_object22526:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:06:25.065 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:06.085000+0000", 2024-09-20T22:06:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 138.575993848, 2024-09-20T22:06:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.088773002000000004, 2024-09-20T22:06:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 22527 2024-09-20T22:06:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.066 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085000+0000", 2024-09-20T22:06:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085000+0000", 2024-09-20T22:06:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085001+0000", 2024-09-20T22:06:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5000000000000001e-07 2024-09-20T22:06:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.067 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085050+0000", 2024-09-20T22:06:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9370999999999999e-05 2024-09-20T22:06:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085061+0000", 2024-09-20T22:06:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0736e-05 2024-09-20T22:06:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085066+0000", 2024-09-20T22:06:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2310000000000004e-06 2024-09-20T22:06:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.068 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085116+0000", 2024-09-20T22:06:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9740000000000001e-05 2024-09-20T22:06:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085241+0000", 2024-09-20T22:06:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012504000000000001 2024-09-20T22:06:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:06:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085465+0000", 2024-09-20T22:06:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000224186 2024-09-20T22:06:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.069 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.086126+0000", 2024-09-20T22:06:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00066131399999999998 2024-09-20T22:06:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.088208+0000", 2024-09-20T22:06:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0020822430000000001 2024-09-20T22:06:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.173622+0000", 2024-09-20T22:06:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.085413878999999998 2024-09-20T22:06:25.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.173737+0000", 2024-09-20T22:06:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011482200000000001 2024-09-20T22:06:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.173773+0000", 2024-09-20T22:06:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5590000000000003e-05 2024-09-20T22:06:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.071 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:22530 7.5 7:a570fd85:::benchmark_data_smithi060_78800_object22529:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:06:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:06.085828+0000", 2024-09-20T22:06:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 138.575165325, 2024-09-20T22:06:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.088066444999999993, 2024-09-20T22:06:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 22530 2024-09-20T22:06:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.072 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085828+0000", 2024-09-20T22:06:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085828+0000", 2024-09-20T22:06:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085829+0000", 2024-09-20T22:06:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2900000000000002e-07 2024-09-20T22:06:25.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085893+0000", 2024-09-20T22:06:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3791e-05 2024-09-20T22:06:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085907+0000", 2024-09-20T22:06:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4347e-05 2024-09-20T22:06:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.085915+0000", 2024-09-20T22:06:25.074 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3640000000000001e-06 2024-09-20T22:06:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.086151+0000", 2024-09-20T22:06:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00023587 2024-09-20T22:06:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.086228+0000", 2024-09-20T22:06:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7668000000000002e-05 2024-09-20T22:06:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:06:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.086309+0000", 2024-09-20T22:06:25.075 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0574000000000005e-05 2024-09-20T22:06:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.086922+0000", 2024-09-20T22:06:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000613183 2024-09-20T22:06:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.088354+0000", 2024-09-20T22:06:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0014318899999999999 2024-09-20T22:06:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.173817+0000", 2024-09-20T22:06:25.076 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.085462815999999997 2024-09-20T22:06:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.173854+0000", 2024-09-20T22:06:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6823000000000003e-05 2024-09-20T22:06:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.173895+0000", 2024-09-20T22:06:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1189999999999997e-05 2024-09-20T22:06:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:22536 7.5 7:a6f0841e:::benchmark_data_smithi060_78800_object22535:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-09-20T22:06:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:06.088349+0000", 2024-09-20T22:06:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 138.572644489, 2024-09-20T22:06:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.085696201999999999, 2024-09-20T22:06:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 22536 2024-09-20T22:06:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.078 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.088349+0000", 2024-09-20T22:06:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.088349+0000", 2024-09-20T22:06:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.088350+0000", 2024-09-20T22:06:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9100000000000003e-07 2024-09-20T22:06:25.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.088422+0000", 2024-09-20T22:06:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2127999999999998e-05 2024-09-20T22:06:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.088432+0000", 2024-09-20T22:06:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5710000000000004e-06 2024-09-20T22:06:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.088436+0000", 2024-09-20T22:06:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6589999999999998e-06 2024-09-20T22:06:25.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.088477+0000", 2024-09-20T22:06:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0574000000000002e-05 2024-09-20T22:06:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.088564+0000", 2024-09-20T22:06:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.7464000000000001e-05 2024-09-20T22:06:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:06:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.088645+0000", 2024-09-20T22:06:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0471999999999998e-05 2024-09-20T22:06:25.081 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.089160+0000", 2024-09-20T22:06:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000515706 2024-09-20T22:06:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.089567+0000", 2024-09-20T22:06:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000406396 2024-09-20T22:06:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.173941+0000", 2024-09-20T22:06:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.084374271000000001 2024-09-20T22:06:25.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.174003+0000", 2024-09-20T22:06:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2321000000000002e-05 2024-09-20T22:06:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:06.174045+0000", 2024-09-20T22:06:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1848999999999999e-05 2024-09-20T22:06:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.24667.0:98054 7.8 e105/80 7:1885fe64:::benchmark_data_smithi060_78800_object98053:head v 105'6246, mlcod=105'6244)", 2024-09-20T22:06:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.322413+0000", 2024-09-20T22:06:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 88.338581011000002, 2024-09-20T22:06:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.085866539000000006, 2024-09-20T22:06:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:25.084 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322413+0000", 2024-09-20T22:06:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322413+0000", 2024-09-20T22:06:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322413+0000", 2024-09-20T22:06:25.085 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4600000000000004e-07 2024-09-20T22:06:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322462+0000", 2024-09-20T22:06:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8789000000000003e-05 2024-09-20T22:06:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322487+0000", 2024-09-20T22:06:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.5109e-05 2024-09-20T22:06:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.086 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322492+0000", 2024-09-20T22:06:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.378e-06 2024-09-20T22:06:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322536+0000", 2024-09-20T22:06:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3197e-05 2024-09-20T22:06:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322558+0000", 2024-09-20T22:06:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2178e-05 2024-09-20T22:06:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.087 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.407595+0000", 2024-09-20T22:06:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.085037410999999993 2024-09-20T22:06:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.408279+0000", 2024-09-20T22:06:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00068383099999999998 2024-09-20T22:06:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:155123 7.5 7:a8f8d6b5:::benchmark_data_smithi060_78800_object22090:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e116)", 2024-09-20T22:06:25.088 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:45.805472+0000", 2024-09-20T22:06:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 38.855521897000003, 2024-09-20T22:06:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.078412549999999998, 2024-09-20T22:06:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 155123 2024-09-20T22:06:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:45.805472+0000", 2024-09-20T22:06:25.089 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:45.805472+0000", 2024-09-20T22:06:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:45.805473+0000", 2024-09-20T22:06:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5300000000000002e-07 2024-09-20T22:06:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:45.805477+0000", 2024-09-20T22:06:25.090 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6079999999999998e-06 2024-09-20T22:06:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:45.807802+0000", 2024-09-20T22:06:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0023246030000000002 2024-09-20T22:06:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:45.807815+0000", 2024-09-20T22:06:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2729e-05 2024-09-20T22:06:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.091 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:45.813781+0000", 2024-09-20T22:06:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0059665630000000002 2024-09-20T22:06:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:45.813901+0000", 2024-09-20T22:06:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011940800000000001 2024-09-20T22:06:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:06:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:45.814027+0000", 2024-09-20T22:06:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012614899999999999 2024-09-20T22:06:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:25.092 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:45.817356+0000", 2024-09-20T22:06:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0033290099999999999 2024-09-20T22:06:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:45.867120+0000", 2024-09-20T22:06:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.049764405999999997 2024-09-20T22:06:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:45.883771+0000", 2024-09-20T22:06:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.016650761 2024-09-20T22:06:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.093 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:45.883827+0000", 2024-09-20T22:06:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5640000000000003e-05 2024-09-20T22:06:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:45.883884+0000", 2024-09-20T22:06:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7720000000000003e-05 2024-09-20T22:06:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169963 7.5 7:acd7b1d6:::benchmark_data_smithi060_78800_object36930:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-20T22:06:25.094 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.430484+0000", 2024-09-20T22:06:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 26.230509250000001, 2024-09-20T22:06:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.119160301, 2024-09-20T22:06:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:06:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169963 2024-09-20T22:06:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430484+0000", 2024-09-20T22:06:25.095 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430484+0000", 2024-09-20T22:06:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430486+0000", 2024-09-20T22:06:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6789999999999999e-06 2024-09-20T22:06:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430492+0000", 2024-09-20T22:06:25.096 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3459999999999999e-06 2024-09-20T22:06:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430505+0000", 2024-09-20T22:06:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3072e-05 2024-09-20T22:06:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430513+0000", 2024-09-20T22:06:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5089999999999996e-06 2024-09-20T22:06:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.435965+0000", 2024-09-20T22:06:25.097 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0054521500000000002 2024-09-20T22:06:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.436089+0000", 2024-09-20T22:06:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012344799999999999 2024-09-20T22:06:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:06:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.436212+0000", 2024-09-20T22:06:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012380099999999999 2024-09-20T22:06:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:25.098 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.442642+0000", 2024-09-20T22:06:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0064292689999999996 2024-09-20T22:06:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549633+0000", 2024-09-20T22:06:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.106991168 2024-09-20T22:06:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549645+0000", 2024-09-20T22:06:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1858999999999999e-05 2024-09-20T22:06:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169964 7.5 7:aca939f2:::benchmark_data_smithi060_78800_object36931:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-20T22:06:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.430999+0000", 2024-09-20T22:06:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 26.229994612999999, 2024-09-20T22:06:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.11886621999999999, 2024-09-20T22:06:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:06:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169964 2024-09-20T22:06:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.100 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430999+0000", 2024-09-20T22:06:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430999+0000", 2024-09-20T22:06:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.101 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.431000+0000", 2024-09-20T22:06:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5000000000000001e-07 2024-09-20T22:06:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.431008+0000", 2024-09-20T22:06:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2479999999999996e-06 2024-09-20T22:06:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.431020+0000", 2024-09-20T22:06:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2082999999999999e-05 2024-09-20T22:06:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.102 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.431028+0000", 2024-09-20T22:06:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4950000000000002e-06 2024-09-20T22:06:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.438979+0000", 2024-09-20T22:06:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0079512799999999998 2024-09-20T22:06:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.439105+0000", 2024-09-20T22:06:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012618800000000001 2024-09-20T22:06:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.103 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:06:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.439232+0000", 2024-09-20T22:06:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012722800000000001 2024-09-20T22:06:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.443003+0000", 2024-09-20T22:06:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0037705920000000001 2024-09-20T22:06:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549861+0000", 2024-09-20T22:06:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.106857492 2024-09-20T22:06:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.104 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549865+0000", 2024-09-20T22:06:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6639999999999997e-06 2024-09-20T22:06:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169963 7.15 7:acd7b1d6:::benchmark_data_smithi060_78800_object36930:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:06:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.440422+0000", 2024-09-20T22:06:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 26.220571330999999, 2024-09-20T22:06:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0806133010000001, 2024-09-20T22:06:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.105 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:06:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169963 2024-09-20T22:06:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440422+0000", 2024-09-20T22:06:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.106 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440422+0000", 2024-09-20T22:06:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440424+0000", 2024-09-20T22:06:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.762e-06 2024-09-20T22:06:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440432+0000", 2024-09-20T22:06:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4530000000000002e-06 2024-09-20T22:06:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.107 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440448+0000", 2024-09-20T22:06:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5084e-05 2024-09-20T22:06:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440456+0000", 2024-09-20T22:06:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0020000000000006e-06 2024-09-20T22:06:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549826+0000", 2024-09-20T22:06:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.109370678 2024-09-20T22:06:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:06:25.108 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549832+0000", 2024-09-20T22:06:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3850000000000001e-06 2024-09-20T22:06:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.503733+0000", 2024-09-20T22:06:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.95390151700000003 2024-09-20T22:06:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for missing object", 2024-09-20T22:06:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.503868+0000", 2024-09-20T22:06:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013515399999999999 2024-09-20T22:06:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.109 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.520960+0000", 2024-09-20T22:06:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.017092006 2024-09-20T22:06:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.521036+0000", 2024-09-20T22:06:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5259999999999994e-05 2024-09-20T22:06:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.110 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169964 7.15 7:aca939f2:::benchmark_data_smithi060_78800_object36931:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:06:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.440473+0000", 2024-09-20T22:06:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 26.220520881999999, 2024-09-20T22:06:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.081014133, 2024-09-20T22:06:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:06:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169964 2024-09-20T22:06:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.111 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440473+0000", 2024-09-20T22:06:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440473+0000", 2024-09-20T22:06:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440473+0000", 2024-09-20T22:06:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0499999999999999e-07 2024-09-20T22:06:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.112 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440476+0000", 2024-09-20T22:06:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0750000000000002e-06 2024-09-20T22:06:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440487+0000", 2024-09-20T22:06:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1048e-05 2024-09-20T22:06:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440493+0000", 2024-09-20T22:06:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.31e-06 2024-09-20T22:06:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.113 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549842+0000", 2024-09-20T22:06:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.109349718 2024-09-20T22:06:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:06:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549845+0000", 2024-09-20T22:06:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0029999999999999e-06 2024-09-20T22:06:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.504130+0000", 2024-09-20T22:06:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.95428438100000001 2024-09-20T22:06:25.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for missing object", 2024-09-20T22:06:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.504241+0000", 2024-09-20T22:06:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011154099999999999 2024-09-20T22:06:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.521422+0000", 2024-09-20T22:06:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.017180540000000001 2024-09-20T22:06:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.521487+0000", 2024-09-20T22:06:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5111999999999996e-05 2024-09-20T22:06:25.115 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "rep_scrubmap(7.5 e80 from shard 2)", 2024-09-20T22:06:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.453781+0000", 2024-09-20T22:06:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 26.207212976000001, 2024-09-20T22:06:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.095955683999999999, 2024-09-20T22:06:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:06:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.116 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.453781+0000", 2024-09-20T22:06:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.453781+0000", 2024-09-20T22:06:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.453782+0000", 2024-09-20T22:06:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.793e-06 2024-09-20T22:06:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.117 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.453819+0000", 2024-09-20T22:06:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7036999999999998e-05 2024-09-20T22:06:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.453853+0000", 2024-09-20T22:06:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.3436999999999999e-05 2024-09-20T22:06:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.453866+0000", 2024-09-20T22:06:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3594000000000001e-05 2024-09-20T22:06:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.118 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549665+0000", 2024-09-20T22:06:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.095798344999999993 2024-09-20T22:06:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549736+0000", 2024-09-20T22:06:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1477999999999996e-05 2024-09-20T22:06:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop_reply(client.24667.0:169963 7.5 e120/80)", 2024-09-20T22:06:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.454092+0000", 2024-09-20T22:06:25.119 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 26.206901616, 2024-09-20T22:06:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.095685120999999998, 2024-09-20T22:06:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:06:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.454092+0000", 2024-09-20T22:06:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.454092+0000", 2024-09-20T22:06:25.120 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.454092+0000", 2024-09-20T22:06:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7399999999999999e-07 2024-09-20T22:06:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.454097+0000", 2024-09-20T22:06:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2370000000000003e-06 2024-09-20T22:06:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.454107+0000", 2024-09-20T22:06:25.121 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0192e-05 2024-09-20T22:06:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.454114+0000", 2024-09-20T22:06:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1810000000000004e-06 2024-09-20T22:06:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549769+0000", 2024-09-20T22:06:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.095654840000000005 2024-09-20T22:06:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549777+0000", 2024-09-20T22:06:25.122 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2970000000000004e-06 2024-09-20T22:06:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop_reply(client.24667.0:169964 7.5 e120/80)", 2024-09-20T22:06:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.454154+0000", 2024-09-20T22:06:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 26.206839148, 2024-09-20T22:06:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.095641905999999999, 2024-09-20T22:06:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:06:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.454154+0000", 2024-09-20T22:06:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.454154+0000", 2024-09-20T22:06:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.454155+0000", 2024-09-20T22:06:25.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8299999999999997e-07 2024-09-20T22:06:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.454158+0000", 2024-09-20T22:06:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7609999999999999e-06 2024-09-20T22:06:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.454165+0000", 2024-09-20T22:06:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6930000000000007e-06 2024-09-20T22:06:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.125 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.454171+0000", 2024-09-20T22:06:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5559999999999998e-06 2024-09-20T22:06:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549789+0000", 2024-09-20T22:06:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.095618442999999997 2024-09-20T22:06:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549796+0000", 2024-09-20T22:06:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9700000000000002e-06 2024-09-20T22:06:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.126 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:186720 7.15 7:ab58254d:::benchmark_data_smithi060_78800_object53687:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-20T22:06:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:15.008883+0000", 2024-09-20T22:06:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 9.652110789, 2024-09-20T22:06:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.096851640000000003, 2024-09-20T22:06:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 186720 2024-09-20T22:06:25.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.008883+0000", 2024-09-20T22:06:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.008883+0000", 2024-09-20T22:06:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.008884+0000", 2024-09-20T22:06:25.128 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6759999999999999e-06 2024-09-20T22:06:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.008891+0000", 2024-09-20T22:06:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7850000000000003e-06 2024-09-20T22:06:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.008909+0000", 2024-09-20T22:06:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7374e-05 2024-09-20T22:06:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.129 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.008917+0000", 2024-09-20T22:06:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1550000000000007e-06 2024-09-20T22:06:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.009022+0000", 2024-09-20T22:06:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000105698 2024-09-20T22:06:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.009170+0000", 2024-09-20T22:06:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014745200000000001 2024-09-20T22:06:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:06:25.130 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.009474+0000", 2024-09-20T22:06:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00030385999999999998 2024-09-20T22:06:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.011531+0000", 2024-09-20T22:06:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0020571690000000002 2024-09-20T22:06:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.017781+0000", 2024-09-20T22:06:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0062499060000000004 2024-09-20T22:06:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.131 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.105542+0000", 2024-09-20T22:06:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.087760705999999994 2024-09-20T22:06:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.105609+0000", 2024-09-20T22:06:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7339e-05 2024-09-20T22:06:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.105734+0000", 2024-09-20T22:06:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012552 2024-09-20T22:06:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.132 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:186730 7.15 7:a8af47ec:::benchmark_data_smithi060_78800_object53697:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e129)", 2024-09-20T22:06:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:15.015899+0000", 2024-09-20T22:06:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 9.645094812, 2024-09-20T22:06:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.092255730999999994, 2024-09-20T22:06:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 186730 2024-09-20T22:06:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.133 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.015899+0000", 2024-09-20T22:06:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.015899+0000", 2024-09-20T22:06:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.015900+0000", 2024-09-20T22:06:25.134 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.463e-06 2024-09-20T22:06:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.015906+0000", 2024-09-20T22:06:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4790000000000004e-06 2024-09-20T22:06:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.015923+0000", 2024-09-20T22:06:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6866000000000001e-05 2024-09-20T22:06:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.015939+0000", 2024-09-20T22:06:25.135 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6597999999999999e-05 2024-09-20T22:06:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.016017+0000", 2024-09-20T22:06:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8294999999999995e-05 2024-09-20T22:06:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.016142+0000", 2024-09-20T22:06:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012494200000000001 2024-09-20T22:06:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:06:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.016459+0000", 2024-09-20T22:06:25.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000316803 2024-09-20T22:06:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.017989+0000", 2024-09-20T22:06:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001529613 2024-09-20T22:06:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.019908+0000", 2024-09-20T22:06:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0019187609999999999 2024-09-20T22:06:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.107996+0000", 2024-09-20T22:06:25.137 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.088087952999999997 2024-09-20T22:06:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.108055+0000", 2024-09-20T22:06:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9259000000000003e-05 2024-09-20T22:06:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:15.108154+0000", 2024-09-20T22:06:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9698999999999995e-05 2024-09-20T22:06:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:197380 7.15 7:ab8602e9:::benchmark_data_smithi060_78800_object64347:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-20T22:06:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:24.106584+0000", 2024-09-20T22:06:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.55440983700000002, 2024-09-20T22:06:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.088668811, 2024-09-20T22:06:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 197380 2024-09-20T22:06:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.139 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.106584+0000", 2024-09-20T22:06:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.106584+0000", 2024-09-20T22:06:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.106585+0000", 2024-09-20T22:06:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.359e-06 2024-09-20T22:06:25.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.106590+0000", 2024-09-20T22:06:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2039999999999996e-06 2024-09-20T22:06:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.106602+0000", 2024-09-20T22:06:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2075999999999999e-05 2024-09-20T22:06:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.106608+0000", 2024-09-20T22:06:25.141 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0190000000000003e-06 2024-09-20T22:06:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.106661+0000", 2024-09-20T22:06:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3087000000000002e-05 2024-09-20T22:06:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.106782+0000", 2024-09-20T22:06:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000120617 2024-09-20T22:06:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:06:25.142 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.107119+0000", 2024-09-20T22:06:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00033711100000000002 2024-09-20T22:06:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.109542+0000", 2024-09-20T22:06:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0024232730000000001 2024-09-20T22:06:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.118122+0000", 2024-09-20T22:06:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0085794170000000006 2024-09-20T22:06:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.143 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.195002+0000", 2024-09-20T22:06:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.076880395000000004 2024-09-20T22:06:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.195072+0000", 2024-09-20T22:06:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9418999999999994e-05 2024-09-20T22:06:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.195253+0000", 2024-09-20T22:06:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000180834 2024-09-20T22:06:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.144 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:197381 7.5 7:a5eefb89:::benchmark_data_smithi060_78800_object64348:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-20T22:06:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:24.106630+0000", 2024-09-20T22:06:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.55436371500000003, 2024-09-20T22:06:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.088583848000000007, 2024-09-20T22:06:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 197381 2024-09-20T22:06:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.145 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.106630+0000", 2024-09-20T22:06:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.106630+0000", 2024-09-20T22:06:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.106630+0000", 2024-09-20T22:06:25.146 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0200000000000003e-07 2024-09-20T22:06:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.106633+0000", 2024-09-20T22:06:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.6910000000000002e-06 2024-09-20T22:06:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.106642+0000", 2024-09-20T22:06:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.867e-06 2024-09-20T22:06:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.106645+0000", 2024-09-20T22:06:25.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6320000000000001e-06 2024-09-20T22:06:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.106692+0000", 2024-09-20T22:06:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6468999999999998e-05 2024-09-20T22:06:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.106824+0000", 2024-09-20T22:06:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013244 2024-09-20T22:06:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:06:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.107167+0000", 2024-09-20T22:06:25.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00034227900000000002 2024-09-20T22:06:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.109728+0000", 2024-09-20T22:06:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0025610590000000001 2024-09-20T22:06:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.119676+0000", 2024-09-20T22:06:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0099478550000000002 2024-09-20T22:06:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.195037+0000", 2024-09-20T22:06:25.149 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.075361601 2024-09-20T22:06:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.195087+0000", 2024-09-20T22:06:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0158999999999999e-05 2024-09-20T22:06:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.195214+0000", 2024-09-20T22:06:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000126394 2024-09-20T22:06:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:197385 7.5 7:a12b9ef0:::benchmark_data_smithi060_78800_object64352:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-20T22:06:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:24.110432+0000", 2024-09-20T22:06:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.55056194199999997, 2024-09-20T22:06:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.084912652000000005, 2024-09-20T22:06:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 197385 2024-09-20T22:06:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.152 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.110432+0000", 2024-09-20T22:06:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.110432+0000", 2024-09-20T22:06:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.110433+0000", 2024-09-20T22:06:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7409999999999999e-06 2024-09-20T22:06:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.154 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.110439+0000", 2024-09-20T22:06:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2480000000000004e-06 2024-09-20T22:06:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.110460+0000", 2024-09-20T22:06:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1498999999999999e-05 2024-09-20T22:06:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.155 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.110467+0000", 2024-09-20T22:06:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2609999999999996e-06 2024-09-20T22:06:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.110659+0000", 2024-09-20T22:06:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019204700000000001 2024-09-20T22:06:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.110783+0000", 2024-09-20T22:06:25.156 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012318600000000001 2024-09-20T22:06:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:06:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.111092+0000", 2024-09-20T22:06:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00030942600000000002 2024-09-20T22:06:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.113393+0000", 2024-09-20T22:06:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0023010320000000002 2024-09-20T22:06:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.119724+0000", 2024-09-20T22:06:25.157 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0063306400000000002 2024-09-20T22:06:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.195268+0000", 2024-09-20T22:06:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.075543856000000006 2024-09-20T22:06:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.195303+0000", 2024-09-20T22:06:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4981000000000003e-05 2024-09-20T22:06:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.158 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.195344+0000", 2024-09-20T22:06:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1734999999999999e-05 2024-09-20T22:06:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:197389 7.5 7:a4b23080:::benchmark_data_smithi060_78800_object64356:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-20T22:06:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:24.116055+0000", 2024-09-20T22:06:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.54493877800000001, 2024-09-20T22:06:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.085477304000000004, 2024-09-20T22:06:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:25.159 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 197389 2024-09-20T22:06:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.116055+0000", 2024-09-20T22:06:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.116055+0000", 2024-09-20T22:06:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.116056+0000", 2024-09-20T22:06:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.618e-06 2024-09-20T22:06:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.116062+0000", 2024-09-20T22:06:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3920000000000002e-06 2024-09-20T22:06:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.161 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.116078+0000", 2024-09-20T22:06:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5832e-05 2024-09-20T22:06:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.116085+0000", 2024-09-20T22:06:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9240000000000001e-06 2024-09-20T22:06:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.116139+0000", 2024-09-20T22:06:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4574000000000003e-05 2024-09-20T22:06:25.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.116263+0000", 2024-09-20T22:06:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012376299999999999 2024-09-20T22:06:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:06:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.116599+0000", 2024-09-20T22:06:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00033597299999999999 2024-09-20T22:06:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.119738+0000", 2024-09-20T22:06:25.163 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0031390580000000001 2024-09-20T22:06:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.195376+0000", 2024-09-20T22:06:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.075638104999999997 2024-09-20T22:06:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.201466+0000", 2024-09-20T22:06:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0060900679999999997 2024-09-20T22:06:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.201502+0000", 2024-09-20T22:06:25.164 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5794999999999999e-05 2024-09-20T22:06:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.201532+0000", 2024-09-20T22:06:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0202e-05 2024-09-20T22:06:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:197392 7.5 7:a7b851c9:::benchmark_data_smithi060_78800_object64359:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e131)", 2024-09-20T22:06:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:24.116862+0000", 2024-09-20T22:06:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.544131213, 2024-09-20T22:06:25.165 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.085464007999999994, 2024-09-20T22:06:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:06:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:06:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:06:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:06:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 197392 2024-09-20T22:06:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:06:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:06:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.116862+0000", 2024-09-20T22:06:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.166 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:06:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.116862+0000", 2024-09-20T22:06:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:06:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:06:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.116864+0000", 2024-09-20T22:06:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.192e-06 2024-09-20T22:06:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:06:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.116869+0000", 2024-09-20T22:06:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5729999999999998e-06 2024-09-20T22:06:25.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:06:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.116883+0000", 2024-09-20T22:06:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4260999999999999e-05 2024-09-20T22:06:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:06:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.116891+0000", 2024-09-20T22:06:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0010000000000001e-06 2024-09-20T22:06:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:06:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.118133+0000", 2024-09-20T22:06:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001241563 2024-09-20T22:06:25.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:06:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.118270+0000", 2024-09-20T22:06:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013682200000000001 2024-09-20T22:06:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:06:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.118508+0000", 2024-09-20T22:06:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00023843099999999999 2024-09-20T22:06:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:06:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.119754+0000", 2024-09-20T22:06:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0012462389999999999 2024-09-20T22:06:25.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.201707+0000", 2024-09-20T22:06:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.081952264999999996 2024-09-20T22:06:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.202253+0000", 2024-09-20T22:06:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00054635700000000003 2024-09-20T22:06:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:06:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.202289+0000", 2024-09-20T22:06:25.170 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5893999999999999e-05 2024-09-20T22:06:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:06:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:06:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:06:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.202326+0000", 2024-09-20T22:06:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7410000000000003e-05 2024-09-20T22:06:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:06:25.171 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:06:25.171 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:06:25.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:25.172 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:25.173 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:25.273 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:25.297 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:25.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:25.298 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:25.360 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:06:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:06:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:06:25.399 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:25.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:25.425 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:25.426 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:25.526 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:25.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:25.559 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:25.560 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:25.660 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:25.689 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:25.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:25.690 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:25.756 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:25 smithi067 ceph-mon[33511]: pgmap v153: 158 pgs: 1 active+forced_recovery+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 4.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 783 op/s; 25351/216675 objects degraded (11.700%); 31230/216675 objects misplaced (14.413%); 12 MiB/s, 199 objects/s recovering 2024-09-20T22:06:25.757 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:25 smithi067 ceph-mon[33511]: osdmap e132: 8 total, 7 up, 8 in 2024-09-20T22:06:25.790 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:25.851 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:25 smithi060 ceph-mon[29694]: pgmap v153: 158 pgs: 1 active+forced_recovery+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 4.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 783 op/s; 25351/216675 objects degraded (11.700%); 31230/216675 objects misplaced (14.413%); 12 MiB/s, 199 objects/s recovering 2024-09-20T22:06:25.851 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:25 smithi060 ceph-mon[29694]: osdmap e132: 8 total, 7 up, 8 in 2024-09-20T22:06:25.852 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:25 smithi060 ceph-mon[40290]: pgmap v153: 158 pgs: 1 active+forced_recovery+undersized+degraded+remapped+backfilling, 2 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 82 active+clean; 4.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 783 op/s; 25351/216675 objects degraded (11.700%); 31230/216675 objects misplaced (14.413%); 12 MiB/s, 199 objects/s recovering 2024-09-20T22:06:25.852 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:25 smithi060 ceph-mon[40290]: osdmap e132: 8 total, 7 up, 8 in 2024-09-20T22:06:25.873 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:25.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:25.874 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:25.974 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:25.997 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:25.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:25.998 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:26.099 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:26.122 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:26.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:26.123 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:26.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T22:06:26.224 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:26.247 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:26.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:26.248 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:26.349 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:26.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:26.371 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:26.372 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:26.472 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:26.513 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:26.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:26.515 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:26.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:26 smithi060 ceph-mon[29694]: osdmap e133: 8 total, 7 up, 8 in 2024-09-20T22:06:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:26 smithi060 ceph-mon[29694]: pgmap v156: 158 pgs: 1 peering, 1 remapped+peering, 2 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.84k op/s; 23311/203124 objects degraded (11.476%); 25270/203124 objects misplaced (12.441%); 33 MiB/s, 533 objects/s recovering 2024-09-20T22:06:26.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:26 smithi060 ceph-mon[40290]: osdmap e133: 8 total, 7 up, 8 in 2024-09-20T22:06:26.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:26 smithi060 ceph-mon[40290]: pgmap v156: 158 pgs: 1 peering, 1 remapped+peering, 2 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.84k op/s; 23311/203124 objects degraded (11.476%); 25270/203124 objects misplaced (12.441%); 33 MiB/s, 533 objects/s recovering 2024-09-20T22:06:26.616 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:26.646 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:26.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:26.647 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:26.748 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:26 smithi067 ceph-mon[33511]: osdmap e133: 8 total, 7 up, 8 in 2024-09-20T22:06:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:26 smithi067 ceph-mon[33511]: pgmap v156: 158 pgs: 1 peering, 1 remapped+peering, 2 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.84k op/s; 23311/203124 objects degraded (11.476%); 25270/203124 objects misplaced (12.441%); 33 MiB/s, 533 objects/s recovering 2024-09-20T22:06:26.780 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:26.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:26.782 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:26.883 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:26.913 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:26.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:26.914 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:27.015 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:27.046 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:27.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:27.047 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:27.148 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:27.187 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:27.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:27.188 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:27.289 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:27.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:27.330 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:27.431 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:27.463 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:27.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:27.464 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:27.489 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:06:27.489 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:06:27.565 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:27.600 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:27.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:27.601 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:27.702 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:27.739 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:27.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:27.741 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:27.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:27 smithi067 ceph-mon[33511]: osdmap e134: 8 total, 7 up, 8 in 2024-09-20T22:06:27.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:27 smithi060 ceph-mon[29694]: osdmap e134: 8 total, 7 up, 8 in 2024-09-20T22:06:27.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:27 smithi060 ceph-mon[40290]: osdmap e134: 8 total, 7 up, 8 in 2024-09-20T22:06:27.841 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:27.864 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:27.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:27.865 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:27.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:28.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:28.003 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:28.004 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:28.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:28.148 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:28.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:28.149 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:28.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:28.282 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:28.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:28.283 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:28.384 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:28.418 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:28.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:28.419 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:28.520 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:28.542 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:28.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:28.543 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:28.643 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:28.684 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:28.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:28.686 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:28 smithi067 ceph-mon[33511]: pgmap v158: 158 pgs: 1 peering, 1 remapped+peering, 2 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 753 op/s; 23311/203124 objects degraded (11.476%); 25270/203124 objects misplaced (12.441%); 16 MiB/s, 255 objects/s recovering 2024-09-20T22:06:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:28 smithi060 ceph-mon[29694]: pgmap v158: 158 pgs: 1 peering, 1 remapped+peering, 2 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 753 op/s; 23311/203124 objects degraded (11.476%); 25270/203124 objects misplaced (12.441%); 16 MiB/s, 255 objects/s recovering 2024-09-20T22:06:28.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:28 smithi060 ceph-mon[40290]: pgmap v158: 158 pgs: 1 peering, 1 remapped+peering, 2 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 753 op/s; 23311/203124 objects degraded (11.476%); 25270/203124 objects misplaced (12.441%); 16 MiB/s, 255 objects/s recovering 2024-09-20T22:06:28.786 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:28.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:28.817 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:28.817 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:28.918 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:28.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:28.940 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:28.940 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:29.041 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:29.075 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:29.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:29.076 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:29.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:29.213 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:29.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:29.214 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:29.314 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:29.344 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:29.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:29.345 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:29.446 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:29.490 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:06:29.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:29.493 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:29.493 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:29.594 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:29.658 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:29.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:29.662 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:29.763 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:29.814 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:29.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:29.815 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:29.916 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:29.944 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:29.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:29.946 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:30.046 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:30.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:30.077 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:30.077 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:30.178 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:30.232 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:30.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:30.233 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:30.257 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:30.334 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:30.334 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:30.363 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:30.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:30.364 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:30.465 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:30.503 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:30.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:30.504 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:30.605 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:30.655 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:30.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:30.656 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:30.671 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:06:30.673 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:06:30.673 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 134, 2024-09-20T22:06:30.673 INFO:teuthology.orchestra.run.smithi060.stdout: "fsid": "3df9647e-779b-11ef-baf5-efdc52797490", 2024-09-20T22:06:30.673 INFO:teuthology.orchestra.run.smithi060.stdout: "created": "2024-09-20T21:57:31.244934+0000", 2024-09-20T22:06:30.673 INFO:teuthology.orchestra.run.smithi060.stdout: "modified": "2024-09-20T22:06:26.396885+0000", 2024-09-20T22:06:30.673 INFO:teuthology.orchestra.run.smithi060.stdout: "last_up_change": "2024-09-20T22:04:15.685933+0000", 2024-09-20T22:06:30.673 INFO:teuthology.orchestra.run.smithi060.stdout: "last_in_change": "2024-09-20T22:02:21.189225+0000", 2024-09-20T22:06:30.673 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-20T22:06:30.673 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_num": 5806080, 2024-09-20T22:06:30.673 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_set": [ 2024-09-20T22:06:30.673 INFO:teuthology.orchestra.run.smithi060.stdout: "nodeep-scrub", 2024-09-20T22:06:30.674 INFO:teuthology.orchestra.run.smithi060.stdout: "noscrub", 2024-09-20T22:06:30.674 INFO:teuthology.orchestra.run.smithi060.stdout: "pglog_hardlimit", 2024-09-20T22:06:30.674 INFO:teuthology.orchestra.run.smithi060.stdout: "purged_snapdirs", 2024-09-20T22:06:30.674 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_deletes", 2024-09-20T22:06:30.674 INFO:teuthology.orchestra.run.smithi060.stdout: "sortbitwise" 2024-09-20T22:06:30.674 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:06:30.674 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_version": 18, 2024-09-20T22:06:30.674 INFO:teuthology.orchestra.run.smithi060.stdout: "full_ratio": 0.94999998807907104, 2024-09-20T22:06:30.674 INFO:teuthology.orchestra.run.smithi060.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-20T22:06:30.674 INFO:teuthology.orchestra.run.smithi060.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-20T22:06:30.674 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_snapshot": "", 2024-09-20T22:06:30.674 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_max": 7, 2024-09-20T22:06:30.674 INFO:teuthology.orchestra.run.smithi060.stdout: "max_osd": 8, 2024-09-20T22:06:30.675 INFO:teuthology.orchestra.run.smithi060.stdout: "require_min_compat_client": "luminous", 2024-09-20T22:06:30.675 INFO:teuthology.orchestra.run.smithi060.stdout: "min_compat_client": "luminous", 2024-09-20T22:06:30.675 INFO:teuthology.orchestra.run.smithi060.stdout: "require_osd_release": "squid", 2024-09-20T22:06:30.675 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_crimson": false, 2024-09-20T22:06:30.675 INFO:teuthology.orchestra.run.smithi060.stdout: "pools": [ 2024-09-20T22:06:30.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.675 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 1, 2024-09-20T22:06:30.675 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": ".mgr", 2024-09-20T22:06:30.675 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:00:35.131125+0000", 2024-09-20T22:06:30.675 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:06:30.675 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:06:30.675 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:06:30.675 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:06:30.676 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:06:30.676 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:06:30.676 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:06:30.676 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:06:30.676 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:06:30.676 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:06:30.676 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:06:30.676 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:06:30.676 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:06:30.676 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 1, 2024-09-20T22:06:30.676 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 1, 2024-09-20T22:06:30.676 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 1, 2024-09-20T22:06:30.676 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 1, 2024-09-20T22:06:30.677 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 1, 2024-09-20T22:06:30.677 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:06:30.677 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:06:30.677 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:06:30.677 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:06:30.677 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:06:30.677 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:06:30.677 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:06:30.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.677 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "21", 2024-09-20T22:06:30.677 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:06:30.677 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:06:30.677 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:06:30.677 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:06:30.678 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:06:30.678 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:06:30.678 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:06:30.678 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:06:30.678 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:06:30.678 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:06:30.678 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:06:30.678 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:06:30.678 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:06:30.678 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:06:30.678 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:06:30.678 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:06:30.678 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:06:30.678 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:06:30.679 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:06:30.679 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:06:30.679 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:06:30.679 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:06:30.679 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:06:30.679 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:06:30.679 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:06:30.679 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:06:30.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.679 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:06:30.679 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:06:30.679 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:06:30.679 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:06:30.679 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:06:30.680 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:06:30.680 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:06:30.680 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:06:30.680 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:06:30.680 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:06:30.680 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:06:30.680 INFO:teuthology.orchestra.run.smithi060.stdout: "options": { 2024-09-20T22:06:30.680 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_max": 32, 2024-09-20T22:06:30.680 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_min": 1 2024-09-20T22:06:30.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.680 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:06:30.680 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": {} 2024-09-20T22:06:30.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.681 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:06:30.681 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:06:30.681 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 7.8899998664855957, 2024-09-20T22:06:30.681 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 7.8899998664855957, 2024-09-20T22:06:30.681 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.37999999523162842, 2024-09-20T22:06:30.681 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 3, 2024-09-20T22:06:30.681 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 3, 2024-09-20T22:06:30.681 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:06:30.681 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:06:30.681 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:06:30.681 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.681 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 2, 2024-09-20T22:06:30.682 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "datapool", 2024-09-20T22:06:30.682 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:47.974480+0000", 2024-09-20T22:06:30.682 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 8193, 2024-09-20T22:06:30.682 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-20T22:06:30.682 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:06:30.682 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:06:30.682 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:06:30.682 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:06:30.682 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:06:30.682 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:06:30.682 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:06:30.682 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:06:30.682 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:06:30.682 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:06:30.683 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:06:30.683 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 3, 2024-09-20T22:06:30.683 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 3, 2024-09-20T22:06:30.683 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 3, 2024-09-20T22:06:30.683 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 3, 2024-09-20T22:06:30.683 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 3, 2024-09-20T22:06:30.683 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:06:30.683 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:06:30.683 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:06:30.683 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:06:30.683 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:06:30.686 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:06:30.686 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:06:30.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.686 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "66", 2024-09-20T22:06:30.686 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:06:30.686 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:06:30.686 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:06:30.686 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:06:30.686 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:06:30.686 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 2, 2024-09-20T22:06:30.686 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 66, 2024-09-20T22:06:30.686 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:06:30.686 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:06:30.686 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:06:30.687 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:06:30.687 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:06:30.687 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:06:30.687 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:06:30.687 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:06:30.687 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:06:30.687 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:06:30.687 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:06:30.687 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:06:30.687 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:06:30.687 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:06:30.687 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:06:30.687 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:06:30.687 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:06:30.687 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:06:30.688 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:06:30.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.688 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:06:30.688 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:06:30.688 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:06:30.688 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:06:30.688 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:06:30.688 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:06:30.688 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:06:30.688 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:06:30.688 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:06:30.688 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:06:30.688 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:06:30.688 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:06:30.689 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:06:30.689 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd": {} 2024-09-20T22:06:30.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.689 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:06:30.689 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:06:30.689 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2.6700000762939453, 2024-09-20T22:06:30.689 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 2.6700000762939453, 2024-09-20T22:06:30.689 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.75, 2024-09-20T22:06:30.689 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2, 2024-09-20T22:06:30.689 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 2, 2024-09-20T22:06:30.689 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:06:30.689 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:06:30.689 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:06:30.689 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.690 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 3, 2024-09-20T22:06:30.690 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": ".rgw.root", 2024-09-20T22:06:30.690 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:48.902465+0000", 2024-09-20T22:06:30.690 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:06:30.690 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:06:30.690 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:06:30.690 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:06:30.690 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:06:30.690 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:06:30.690 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:06:30.690 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:06:30.691 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:06:30.691 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:06:30.691 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:06:30.691 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:06:30.691 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:06:30.691 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:06:30.691 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:06:30.691 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:06:30.691 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:06:30.691 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:06:30.691 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:06:30.691 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:06:30.692 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:06:30.692 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:06:30.692 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:06:30.692 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:06:30.692 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:06:30.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.692 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "65", 2024-09-20T22:06:30.692 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:06:30.692 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:06:30.692 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:06:30.692 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:06:30.692 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:06:30.692 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:06:30.692 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:06:30.693 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:06:30.693 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:06:30.693 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:06:30.693 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:06:30.693 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:06:30.693 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:06:30.693 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:06:30.693 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:06:30.693 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:06:30.693 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:06:30.693 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:06:30.693 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:06:30.693 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:06:30.694 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:06:30.694 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:06:30.694 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:06:30.694 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:06:30.694 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:06:30.694 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:06:30.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.694 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:06:30.694 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:06:30.694 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:06:30.694 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:06:30.694 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:06:30.694 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:06:30.694 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:06:30.695 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:06:30.695 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:06:30.695 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:06:30.695 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:06:30.695 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:06:30.695 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:06:30.695 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:06:30.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.695 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:06:30.695 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:06:30.695 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.7400000095367432, 2024-09-20T22:06:30.695 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.7400000095367432, 2024-09-20T22:06:30.696 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T22:06:30.696 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-20T22:06:30.696 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-20T22:06:30.696 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:06:30.696 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:06:30.696 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:06:30.696 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.696 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 4, 2024-09-20T22:06:30.696 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "default.rgw.log", 2024-09-20T22:06:30.696 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:50.620057+0000", 2024-09-20T22:06:30.696 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:06:30.696 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:06:30.696 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:06:30.697 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:06:30.697 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:06:30.697 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:06:30.697 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:06:30.697 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:06:30.697 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:06:30.697 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:06:30.697 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:06:30.697 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:06:30.697 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:06:30.697 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:06:30.697 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:06:30.697 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:06:30.697 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:06:30.698 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:06:30.698 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:06:30.698 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:06:30.698 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:06:30.698 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:06:30.698 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:06:30.698 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:06:30.698 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:06:30.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.698 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "67", 2024-09-20T22:06:30.698 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:06:30.698 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:06:30.698 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:06:30.698 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:06:30.699 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:06:30.699 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:06:30.699 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:06:30.699 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:06:30.699 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:06:30.699 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:06:30.699 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:06:30.699 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:06:30.699 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:06:30.699 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:06:30.699 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:06:30.699 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:06:30.699 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:06:30.699 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:06:30.700 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:06:30.700 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:06:30.700 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:06:30.700 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:06:30.700 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:06:30.700 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:06:30.700 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:06:30.700 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:06:30.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.700 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:06:30.700 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:06:30.700 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:06:30.700 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:06:30.700 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:06:30.701 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:06:30.701 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:06:30.701 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:06:30.701 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:06:30.701 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:06:30.701 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:06:30.701 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:06:30.701 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:06:30.701 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:06:30.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.701 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:06:30.701 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:06:30.701 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2.2400000095367432, 2024-09-20T22:06:30.701 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 2.2400000095367432, 2024-09-20T22:06:30.702 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T22:06:30.702 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.9700000286102295, 2024-09-20T22:06:30.702 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.9700000286102295, 2024-09-20T22:06:30.702 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:06:30.702 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:06:30.702 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:06:30.702 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.702 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 5, 2024-09-20T22:06:30.702 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "default.rgw.control", 2024-09-20T22:06:30.702 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:52.722912+0000", 2024-09-20T22:06:30.702 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:06:30.702 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:06:30.703 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:06:30.703 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:06:30.703 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:06:30.703 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:06:30.703 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:06:30.703 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:06:30.703 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:06:30.703 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:06:30.703 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:06:30.703 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:06:30.703 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:06:30.703 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:06:30.703 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:06:30.703 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:06:30.704 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:06:30.704 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:06:30.704 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:06:30.704 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:06:30.704 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:06:30.704 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:06:30.704 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:06:30.704 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:06:30.704 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:06:30.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.704 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "69", 2024-09-20T22:06:30.704 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:06:30.704 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:06:30.704 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:06:30.705 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:06:30.705 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:06:30.705 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:06:30.705 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:06:30.705 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:06:30.705 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:06:30.705 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:06:30.705 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:06:30.705 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:06:30.705 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:06:30.705 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:06:30.705 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:06:30.705 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:06:30.705 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:06:30.705 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:06:30.706 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:06:30.706 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:06:30.706 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:06:30.706 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:06:30.706 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:06:30.706 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:06:30.706 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:06:30.706 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:06:30.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.706 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:06:30.706 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:06:30.706 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:06:30.706 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:06:30.706 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:06:30.706 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:06:30.707 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:06:30.707 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:06:30.707 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:06:30.707 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:06:30.707 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:06:30.707 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:06:30.707 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:06:30.707 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:06:30.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.707 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:06:30.707 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:06:30.707 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.7400000095367432, 2024-09-20T22:06:30.707 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.7400000095367432, 2024-09-20T22:06:30.707 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T22:06:30.707 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.5299999713897705, 2024-09-20T22:06:30.708 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.5299999713897705, 2024-09-20T22:06:30.708 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:06:30.708 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:06:30.708 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:06:30.708 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.709 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 6, 2024-09-20T22:06:30.709 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "default.rgw.meta", 2024-09-20T22:06:30.709 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:54.794248+0000", 2024-09-20T22:06:30.709 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:06:30.709 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:06:30.709 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:06:30.709 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:06:30.709 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:06:30.709 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:06:30.709 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:06:30.709 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:06:30.709 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:06:30.709 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:06:30.709 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:06:30.710 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:06:30.710 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:06:30.710 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:06:30.710 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:06:30.710 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:06:30.710 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:06:30.710 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:06:30.710 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:06:30.710 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:06:30.710 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:06:30.710 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:06:30.710 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:06:30.710 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:06:30.711 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:06:30.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.711 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "71", 2024-09-20T22:06:30.711 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:06:30.711 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:06:30.711 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:06:30.711 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:06:30.711 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:06:30.711 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:06:30.711 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:06:30.711 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:06:30.711 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:06:30.711 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:06:30.711 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:06:30.712 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:06:30.712 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:06:30.712 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:06:30.712 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:06:30.712 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:06:30.712 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:06:30.712 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:06:30.712 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:06:30.712 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:06:30.712 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:06:30.712 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:06:30.712 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:06:30.712 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:06:30.712 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:06:30.713 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:06:30.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.713 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:06:30.713 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:06:30.713 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:06:30.713 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:06:30.713 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:06:30.713 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:06:30.713 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:06:30.713 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:06:30.713 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:06:30.713 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:06:30.713 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:06:30.713 INFO:teuthology.orchestra.run.smithi060.stdout: "options": { 2024-09-20T22:06:30.713 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_bias": 4 2024-09-20T22:06:30.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.714 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:06:30.714 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:06:30.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.714 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:06:30.714 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:06:30.714 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.9900000095367432, 2024-09-20T22:06:30.714 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.9900000095367432, 2024-09-20T22:06:30.714 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T22:06:30.714 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.75, 2024-09-20T22:06:30.714 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.75, 2024-09-20T22:06:30.714 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:06:30.714 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:06:30.714 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:06:30.715 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.715 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 7, 2024-09-20T22:06:30.715 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "unique_pool_0", 2024-09-20T22:06:30.715 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:03:51.292196+0000", 2024-09-20T22:06:30.715 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:06:30.715 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:06:30.715 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:06:30.715 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:06:30.715 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:06:30.715 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:06:30.715 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:06:30.715 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:06:30.716 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:06:30.716 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:06:30.716 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:06:30.716 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:06:30.716 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:06:30.716 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 26, 2024-09-20T22:06:30.716 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 26, 2024-09-20T22:06:30.716 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 26, 2024-09-20T22:06:30.716 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 26, 2024-09-20T22:06:30.716 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 26, 2024-09-20T22:06:30.716 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:06:30.716 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:06:30.716 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:06:30.716 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:06:30.716 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:06:30.717 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:06:30.717 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:06:30.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.717 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "123", 2024-09-20T22:06:30.717 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:06:30.717 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:06:30.717 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "121", 2024-09-20T22:06:30.717 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:06:30.717 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:06:30.717 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:06:30.717 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:06:30.717 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:06:30.717 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:06:30.717 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:06:30.718 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:06:30.718 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:06:30.718 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:06:30.718 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:06:30.718 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:06:30.718 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:06:30.718 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:06:30.718 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:06:30.718 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:06:30.718 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:06:30.718 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:06:30.718 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:06:30.718 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:06:30.718 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:06:30.719 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:06:30.719 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:06:30.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.719 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:06:30.719 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:06:30.719 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:06:30.719 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:06:30.719 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:06:30.719 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:06:30.719 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:06:30.719 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:06:30.719 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:06:30.719 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:06:30.719 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:06:30.719 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:06:30.720 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:06:30.720 INFO:teuthology.orchestra.run.smithi060.stdout: "rados": {} 2024-09-20T22:06:30.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.720 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:06:30.720 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:06:30.720 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 3.059999942779541, 2024-09-20T22:06:30.720 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.8400000333786011, 2024-09-20T22:06:30.720 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T22:06:30.720 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2.690000057220459, 2024-09-20T22:06:30.720 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.6200000047683716, 2024-09-20T22:06:30.720 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:06:30.720 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:06:30.720 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:06:30.720 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.721 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.721 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:06:30.721 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:06:30.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.721 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 0, 2024-09-20T22:06:30.721 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "0997c01a-a9bd-421e-abcc-fbcf7abe8512", 2024-09-20T22:06:30.721 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:06:30.721 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:06:30.721 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:06:30.721 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:06:30.721 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:06:30.721 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:06:30.721 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 8, 2024-09-20T22:06:30.722 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 133, 2024-09-20T22:06:30.722 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:06:30.722 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:06:30.722 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:06:30.722 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.722 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.722 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6802", 2024-09-20T22:06:30.722 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:06:30.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.722 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.722 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6803", 2024-09-20T22:06:30.722 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:06:30.722 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.723 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.723 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:06:30.723 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.723 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.723 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6804", 2024-09-20T22:06:30.723 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:06:30.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.723 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.723 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6805", 2024-09-20T22:06:30.723 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:06:30.723 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.724 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.724 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:06:30.724 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.724 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.724 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6808", 2024-09-20T22:06:30.724 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:06:30.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.724 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.724 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6809", 2024-09-20T22:06:30.724 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:06:30.724 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.725 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.725 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:06:30.725 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.725 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.725 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6806", 2024-09-20T22:06:30.725 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:06:30.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.725 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.725 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6807", 2024-09-20T22:06:30.725 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:06:30.725 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.726 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.726 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6803/1395849550", 2024-09-20T22:06:30.726 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6805/1395849550", 2024-09-20T22:06:30.726 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6809/1395849550", 2024-09-20T22:06:30.726 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6807/1395849550", 2024-09-20T22:06:30.726 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:06:30.726 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:06:30.726 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:06:30.726 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.726 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 1, 2024-09-20T22:06:30.726 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "c04bdcb1-d76b-47c8-a38b-c02043df6ba8", 2024-09-20T22:06:30.727 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 0, 2024-09-20T22:06:30.727 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:06:30.727 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:06:30.727 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:06:30.727 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:06:30.727 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:06:30.727 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 13, 2024-09-20T22:06:30.727 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 80, 2024-09-20T22:06:30.727 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 91, 2024-09-20T22:06:30.727 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:06:30.727 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:06:30.727 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.727 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.728 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6810", 2024-09-20T22:06:30.728 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 145283513 2024-09-20T22:06:30.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.728 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.728 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6811", 2024-09-20T22:06:30.728 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 145283513 2024-09-20T22:06:30.728 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.728 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.728 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:06:30.728 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.728 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.729 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6812", 2024-09-20T22:06:30.729 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 145283513 2024-09-20T22:06:30.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.729 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.729 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6813", 2024-09-20T22:06:30.729 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 145283513 2024-09-20T22:06:30.729 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.729 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.729 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:06:30.729 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.729 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.729 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6816", 2024-09-20T22:06:30.730 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 145283513 2024-09-20T22:06:30.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.730 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.730 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6817", 2024-09-20T22:06:30.730 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 145283513 2024-09-20T22:06:30.730 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.730 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.730 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:06:30.730 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.730 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.730 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6814", 2024-09-20T22:06:30.731 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 145283513 2024-09-20T22:06:30.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.731 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.731 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6815", 2024-09-20T22:06:30.731 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 145283513 2024-09-20T22:06:30.731 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.731 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.731 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6811/145283513", 2024-09-20T22:06:30.731 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6813/145283513", 2024-09-20T22:06:30.731 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6817/145283513", 2024-09-20T22:06:30.731 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6815/145283513", 2024-09-20T22:06:30.731 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:06:30.732 INFO:teuthology.orchestra.run.smithi060.stdout: "exists" 2024-09-20T22:06:30.732 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.732 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 2, 2024-09-20T22:06:30.732 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "32c09d19-eb98-4926-a44e-e7ddee152956", 2024-09-20T22:06:30.732 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:06:30.732 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:06:30.732 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:06:30.732 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:06:30.732 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:06:30.732 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:06:30.732 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 18, 2024-09-20T22:06:30.732 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 132, 2024-09-20T22:06:30.732 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:06:30.733 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:06:30.733 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:06:30.733 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.733 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.733 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6818", 2024-09-20T22:06:30.733 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:06:30.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.733 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.733 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6819", 2024-09-20T22:06:30.733 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:06:30.733 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.733 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.734 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:06:30.734 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.734 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.734 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6820", 2024-09-20T22:06:30.734 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:06:30.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.734 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.734 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6821", 2024-09-20T22:06:30.734 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:06:30.734 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.734 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.735 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:06:30.735 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.735 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.735 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6824", 2024-09-20T22:06:30.735 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:06:30.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.735 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.735 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6825", 2024-09-20T22:06:30.735 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:06:30.735 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.735 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.735 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:06:30.736 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.736 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.736 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6822", 2024-09-20T22:06:30.736 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:06:30.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.736 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.736 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6823", 2024-09-20T22:06:30.736 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:06:30.736 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.736 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.736 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6819/3698562274", 2024-09-20T22:06:30.737 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6821/3698562274", 2024-09-20T22:06:30.737 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6825/3698562274", 2024-09-20T22:06:30.737 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6823/3698562274", 2024-09-20T22:06:30.737 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:06:30.737 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:06:30.737 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:06:30.737 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.737 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 3, 2024-09-20T22:06:30.737 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "b1c2cb69-8565-49ac-b854-7ea7615c6660", 2024-09-20T22:06:30.737 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:06:30.737 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:06:30.737 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:06:30.738 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:06:30.738 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:06:30.738 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:06:30.738 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 26, 2024-09-20T22:06:30.738 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 133, 2024-09-20T22:06:30.738 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:06:30.738 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:06:30.738 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:06:30.738 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.738 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.738 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6826", 2024-09-20T22:06:30.738 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:06:30.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6827", 2024-09-20T22:06:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:06:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:06:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6828", 2024-09-20T22:06:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:06:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.740 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.740 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6829", 2024-09-20T22:06:30.740 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:06:30.740 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.740 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.740 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:06:30.740 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.740 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.740 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6832", 2024-09-20T22:06:30.740 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:06:30.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.741 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.741 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6833", 2024-09-20T22:06:30.741 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:06:30.741 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.741 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.741 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:06:30.741 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.741 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.741 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6830", 2024-09-20T22:06:30.741 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:06:30.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.742 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.742 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6831", 2024-09-20T22:06:30.742 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:06:30.742 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.742 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.742 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6827/2592322449", 2024-09-20T22:06:30.742 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6829/2592322449", 2024-09-20T22:06:30.743 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6833/2592322449", 2024-09-20T22:06:30.743 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6831/2592322449", 2024-09-20T22:06:30.743 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:06:30.743 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:06:30.743 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:06:30.743 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.743 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 4, 2024-09-20T22:06:30.743 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "04a547d6-fc48-43b7-a8d7-18df8a83aa96", 2024-09-20T22:06:30.743 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:06:30.743 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:06:30.744 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:06:30.744 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:06:30.744 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:06:30.744 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:06:30.744 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 31, 2024-09-20T22:06:30.744 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 123, 2024-09-20T22:06:30.744 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:06:30.744 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:06:30.744 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:06:30.744 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.744 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.744 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6800", 2024-09-20T22:06:30.745 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:06:30.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.745 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.745 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6801", 2024-09-20T22:06:30.745 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:06:30.745 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.745 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.745 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:06:30.745 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.745 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.745 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6802", 2024-09-20T22:06:30.746 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:06:30.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.746 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.746 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6803", 2024-09-20T22:06:30.746 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:06:30.746 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.746 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.746 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:06:30.746 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.746 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.746 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6806", 2024-09-20T22:06:30.747 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:06:30.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.747 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.747 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6807", 2024-09-20T22:06:30.747 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:06:30.747 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.747 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.747 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:06:30.747 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.747 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.747 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6804", 2024-09-20T22:06:30.748 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:06:30.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.748 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.748 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6805", 2024-09-20T22:06:30.748 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:06:30.748 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.748 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.748 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6801/3243564239", 2024-09-20T22:06:30.748 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6803/3243564239", 2024-09-20T22:06:30.748 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6807/3243564239", 2024-09-20T22:06:30.748 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6805/3243564239", 2024-09-20T22:06:30.748 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:06:30.749 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:06:30.749 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:06:30.749 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.749 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 5, 2024-09-20T22:06:30.749 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "4763aa10-c367-485b-b525-e5241e642b86", 2024-09-20T22:06:30.749 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:06:30.749 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:06:30.749 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:06:30.749 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:06:30.749 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:06:30.749 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:06:30.749 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 39, 2024-09-20T22:06:30.749 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 121, 2024-09-20T22:06:30.750 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:06:30.750 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:06:30.750 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:06:30.750 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.750 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.750 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6808", 2024-09-20T22:06:30.750 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:06:30.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.750 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.750 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6809", 2024-09-20T22:06:30.750 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:06:30.750 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.751 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.751 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:06:30.751 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.751 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.751 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6810", 2024-09-20T22:06:30.751 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:06:30.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.751 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.751 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6811", 2024-09-20T22:06:30.751 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:06:30.751 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.752 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.752 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:06:30.752 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.752 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.752 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6814", 2024-09-20T22:06:30.752 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:06:30.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.752 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.752 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6815", 2024-09-20T22:06:30.752 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:06:30.752 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.752 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.753 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:06:30.753 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.753 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.753 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6812", 2024-09-20T22:06:30.753 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:06:30.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.753 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.753 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6813", 2024-09-20T22:06:30.753 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:06:30.753 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.753 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.754 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6809/1622258887", 2024-09-20T22:06:30.754 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6811/1622258887", 2024-09-20T22:06:30.754 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6815/1622258887", 2024-09-20T22:06:30.754 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6813/1622258887", 2024-09-20T22:06:30.754 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:06:30.754 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:06:30.754 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:06:30.754 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.754 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 6, 2024-09-20T22:06:30.754 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "a359116e-c013-4e2f-8b79-86219516ee16", 2024-09-20T22:06:30.754 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:06:30.754 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:06:30.755 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:06:30.755 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:06:30.755 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:06:30.755 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:06:30.755 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 48, 2024-09-20T22:06:30.755 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 123, 2024-09-20T22:06:30.755 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:06:30.755 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:06:30.755 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:06:30.755 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.755 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.755 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6816", 2024-09-20T22:06:30.755 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:06:30.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.756 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.756 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6817", 2024-09-20T22:06:30.756 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:06:30.756 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.756 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.756 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:06:30.756 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.756 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.756 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6818", 2024-09-20T22:06:30.756 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:06:30.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.756 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.757 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6819", 2024-09-20T22:06:30.757 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:06:30.757 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.757 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.757 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:06:30.757 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.757 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.757 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6822", 2024-09-20T22:06:30.757 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:06:30.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.757 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.758 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6823", 2024-09-20T22:06:30.758 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:06:30.758 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.758 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.758 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:06:30.758 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.758 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.758 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6820", 2024-09-20T22:06:30.758 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:06:30.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.758 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.758 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6821", 2024-09-20T22:06:30.759 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:06:30.759 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.759 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.759 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6817/1187114", 2024-09-20T22:06:30.759 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6819/1187114", 2024-09-20T22:06:30.759 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6823/1187114", 2024-09-20T22:06:30.759 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6821/1187114", 2024-09-20T22:06:30.759 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:06:30.759 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:06:30.759 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:06:30.759 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.760 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 7, 2024-09-20T22:06:30.760 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "a4bf4188-f133-4a31-8897-d7e8182248d7", 2024-09-20T22:06:30.760 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:06:30.760 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:06:30.760 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:06:30.760 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:06:30.760 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:06:30.760 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:06:30.760 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 57, 2024-09-20T22:06:30.760 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 124, 2024-09-20T22:06:30.760 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:06:30.760 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:30.761 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:06:30.761 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:06:30.761 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.762 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.762 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6824", 2024-09-20T22:06:30.762 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:06:30.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.762 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.762 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6825", 2024-09-20T22:06:30.762 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:06:30.762 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.762 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.762 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:06:30.762 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.763 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.763 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6826", 2024-09-20T22:06:30.763 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:06:30.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.763 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.763 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6827", 2024-09-20T22:06:30.763 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:06:30.763 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.763 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.763 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:06:30.764 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.764 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.764 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6830", 2024-09-20T22:06:30.764 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:06:30.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.764 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.764 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6831", 2024-09-20T22:06:30.764 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:06:30.764 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.764 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.764 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:06:30.764 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:06:30.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.765 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:06:30.765 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6828", 2024-09-20T22:06:30.765 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:06:30.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.765 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:06:30.765 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6829", 2024-09-20T22:06:30.765 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:06:30.765 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.765 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.765 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6825/3021842840", 2024-09-20T22:06:30.765 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6827/3021842840", 2024-09-20T22:06:30.765 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6831/3021842840", 2024-09-20T22:06:30.766 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6829/3021842840", 2024-09-20T22:06:30.766 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:06:30.766 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:06:30.766 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:06:30.766 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.766 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.766 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:06:30.766 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_xinfo": [ 2024-09-20T22:06:30.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.766 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 0, 2024-09-20T22:06:30.766 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:06:30.766 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:06:30.766 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:06:30.766 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:06:30.767 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:06:30.767 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:06:07.734923+0000", 2024-09-20T22:06:30.767 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:06:30.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.767 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 1, 2024-09-20T22:06:30.767 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:06:30.767 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:06:30.767 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:06:30.767 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:06:30.767 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:06:30.767 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:04:02.357858+0000", 2024-09-20T22:06:30.767 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 90 2024-09-20T22:06:30.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.768 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 2, 2024-09-20T22:06:30.768 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:06:30.768 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:06:30.768 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:06:30.768 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:06:30.768 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:06:30.768 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:05:42.634604+0000", 2024-09-20T22:06:30.768 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:06:30.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.768 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 3, 2024-09-20T22:06:30.768 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:06:30.768 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:06:30.769 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:06:30.769 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:06:30.769 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:06:30.769 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:06:07.263859+0000", 2024-09-20T22:06:30.769 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:06:30.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.769 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 4, 2024-09-20T22:06:30.769 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:06:30.769 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:06:30.769 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:06:30.769 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:06:30.769 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:06:30.769 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:05:26.679800+0000", 2024-09-20T22:06:30.770 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:06:30.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.770 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 5, 2024-09-20T22:06:30.770 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:06:30.770 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:06:30.770 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:06:30.770 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:06:30.770 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:06:30.770 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:05:39.734002+0000", 2024-09-20T22:06:30.770 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:06:30.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.770 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 6, 2024-09-20T22:06:30.771 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:06:30.771 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:06:30.771 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:06:30.771 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:06:30.771 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:06:30.771 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:05:48.482150+0000", 2024-09-20T22:06:30.771 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:06:30.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.771 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 7, 2024-09-20T22:06:30.771 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:06:30.771 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:06:30.771 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:06:30.771 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:06:30.772 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:06:30.772 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:05:27.631709+0000", 2024-09-20T22:06:30.772 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:06:30.772 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.772 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:06:30.772 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap": [], 2024-09-20T22:06:30.772 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap_items": [ 2024-09-20T22:06:30.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.772 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "3.1a", 2024-09-20T22:06:30.772 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:06:30.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.772 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 4, 2024-09-20T22:06:30.772 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 5 2024-09-20T22:06:30.773 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.773 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.773 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "3.1c", 2024-09-20T22:06:30.773 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:06:30.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.773 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 4, 2024-09-20T22:06:30.773 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 2 2024-09-20T22:06:30.773 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.773 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.773 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.7", 2024-09-20T22:06:30.773 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:06:30.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.774 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:06:30.774 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 0 2024-09-20T22:06:30.774 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.774 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.774 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.9", 2024-09-20T22:06:30.774 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:06:30.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.774 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 7, 2024-09-20T22:06:30.774 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 1 2024-09-20T22:06:30.774 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.774 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.775 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.b", 2024-09-20T22:06:30.775 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:06:30.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.775 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:06:30.775 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 0 2024-09-20T22:06:30.775 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.775 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.775 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.d", 2024-09-20T22:06:30.775 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:06:30.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.776 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 5, 2024-09-20T22:06:30.776 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 6 2024-09-20T22:06:30.776 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.776 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.776 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.11", 2024-09-20T22:06:30.776 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:06:30.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.776 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:06:30.776 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 1 2024-09-20T22:06:30.776 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.776 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.777 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.19", 2024-09-20T22:06:30.777 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:06:30.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.777 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:06:30.777 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 6 2024-09-20T22:06:30.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.777 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 5, 2024-09-20T22:06:30.777 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 0 2024-09-20T22:06:30.777 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.777 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.778 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "7.3", 2024-09-20T22:06:30.778 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:06:30.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.778 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:06:30.778 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 1 2024-09-20T22:06:30.778 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.778 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.778 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.778 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:06:30.778 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap_primaries": [], 2024-09-20T22:06:30.778 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_temp": [ 2024-09-20T22:06:30.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.779 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "3.1a", 2024-09-20T22:06:30.779 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:06:30.779 INFO:teuthology.orchestra.run.smithi060.stdout: 5, 2024-09-20T22:06:30.779 INFO:teuthology.orchestra.run.smithi060.stdout: 2, 2024-09-20T22:06:30.779 INFO:teuthology.orchestra.run.smithi060.stdout: 4 2024-09-20T22:06:30.779 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.779 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "3.1c", 2024-09-20T22:06:30.779 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:06:30.779 INFO:teuthology.orchestra.run.smithi060.stdout: 5, 2024-09-20T22:06:30.779 INFO:teuthology.orchestra.run.smithi060.stdout: 2, 2024-09-20T22:06:30.779 INFO:teuthology.orchestra.run.smithi060.stdout: 4 2024-09-20T22:06:30.779 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.780 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.9", 2024-09-20T22:06:30.780 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:06:30.780 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:06:30.780 INFO:teuthology.orchestra.run.smithi060.stdout: 2, 2024-09-20T22:06:30.780 INFO:teuthology.orchestra.run.smithi060.stdout: 7 2024-09-20T22:06:30.780 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.780 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.b", 2024-09-20T22:06:30.780 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:06:30.780 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:06:30.780 INFO:teuthology.orchestra.run.smithi060.stdout: 7, 2024-09-20T22:06:30.781 INFO:teuthology.orchestra.run.smithi060.stdout: 3 2024-09-20T22:06:30.781 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.781 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.d", 2024-09-20T22:06:30.781 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:06:30.781 INFO:teuthology.orchestra.run.smithi060.stdout: 6, 2024-09-20T22:06:30.781 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:06:30.781 INFO:teuthology.orchestra.run.smithi060.stdout: 5 2024-09-20T22:06:30.781 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.781 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.11", 2024-09-20T22:06:30.781 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:06:30.782 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:06:30.782 INFO:teuthology.orchestra.run.smithi060.stdout: 5, 2024-09-20T22:06:30.782 INFO:teuthology.orchestra.run.smithi060.stdout: 3 2024-09-20T22:06:30.782 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.782 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.19", 2024-09-20T22:06:30.782 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:06:30.782 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:06:30.782 INFO:teuthology.orchestra.run.smithi060.stdout: 6, 2024-09-20T22:06:30.782 INFO:teuthology.orchestra.run.smithi060.stdout: 3 2024-09-20T22:06:30.782 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.782 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "7.3", 2024-09-20T22:06:30.783 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:06:30.783 INFO:teuthology.orchestra.run.smithi060.stdout: 2, 2024-09-20T22:06:30.783 INFO:teuthology.orchestra.run.smithi060.stdout: 4, 2024-09-20T22:06:30.783 INFO:teuthology.orchestra.run.smithi060.stdout: 3 2024-09-20T22:06:30.783 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.783 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "7.10", 2024-09-20T22:06:30.783 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:06:30.783 INFO:teuthology.orchestra.run.smithi060.stdout: 3, 2024-09-20T22:06:30.783 INFO:teuthology.orchestra.run.smithi060.stdout: 2, 2024-09-20T22:06:30.783 INFO:teuthology.orchestra.run.smithi060.stdout: 5 2024-09-20T22:06:30.783 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.784 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "7.11", 2024-09-20T22:06:30.784 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:06:30.784 INFO:teuthology.orchestra.run.smithi060.stdout: 3, 2024-09-20T22:06:30.784 INFO:teuthology.orchestra.run.smithi060.stdout: 6, 2024-09-20T22:06:30.784 INFO:teuthology.orchestra.run.smithi060.stdout: 4 2024-09-20T22:06:30.784 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.784 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "7.12", 2024-09-20T22:06:30.784 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:06:30.784 INFO:teuthology.orchestra.run.smithi060.stdout: 2, 2024-09-20T22:06:30.784 INFO:teuthology.orchestra.run.smithi060.stdout: 3 2024-09-20T22:06:30.784 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.785 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "7.14", 2024-09-20T22:06:30.785 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:06:30.785 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:06:30.785 INFO:teuthology.orchestra.run.smithi060.stdout: 4 2024-09-20T22:06:30.785 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.785 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "7.15", 2024-09-20T22:06:30.785 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:06:30.785 INFO:teuthology.orchestra.run.smithi060.stdout: 7, 2024-09-20T22:06:30.785 INFO:teuthology.orchestra.run.smithi060.stdout: 2, 2024-09-20T22:06:30.785 INFO:teuthology.orchestra.run.smithi060.stdout: 0 2024-09-20T22:06:30.785 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.786 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "7.16", 2024-09-20T22:06:30.786 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:06:30.786 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:06:30.786 INFO:teuthology.orchestra.run.smithi060.stdout: 7, 2024-09-20T22:06:30.786 INFO:teuthology.orchestra.run.smithi060.stdout: 3 2024-09-20T22:06:30.786 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.786 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "7.17", 2024-09-20T22:06:30.786 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:06:30.786 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:06:30.786 INFO:teuthology.orchestra.run.smithi060.stdout: 3, 2024-09-20T22:06:30.786 INFO:teuthology.orchestra.run.smithi060.stdout: 6 2024-09-20T22:06:30.786 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:30.787 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "7.19", 2024-09-20T22:06:30.787 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:06:30.787 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:06:30.787 INFO:teuthology.orchestra.run.smithi060.stdout: 4 2024-09-20T22:06:30.787 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:30.787 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.787 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:06:30.787 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_temp": [], 2024-09-20T22:06:30.787 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklist": { 2024-09-20T22:06:30.787 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/890154710": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:06:30.787 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3386526350": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:06:30.787 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/604714621": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:06:30.787 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/604714621": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:06:30.787 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/3151907776": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:06:30.788 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/956264518": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:06:30.788 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/4150585713": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:06:30.788 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3311255314": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:06:30.788 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3965541528": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:06:30.788 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/3734574621": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:06:30.788 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1729862661": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:06:30.788 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/3734574621": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:06:30.788 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3927294026": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:06:30.788 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/3151907776": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:06:30.788 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1871351181": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:06:30.788 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1854455098": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:06:30.788 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/2221262203": "2024-09-21T22:03:19.518405+0000" 2024-09-20T22:06:30.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.788 INFO:teuthology.orchestra.run.smithi060.stdout: "range_blocklist": {}, 2024-09-20T22:06:30.788 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profiles": { 2024-09-20T22:06:30.789 INFO:teuthology.orchestra.run.smithi060.stdout: "default": { 2024-09-20T22:06:30.789 INFO:teuthology.orchestra.run.smithi060.stdout: "crush-failure-domain": "osd", 2024-09-20T22:06:30.789 INFO:teuthology.orchestra.run.smithi060.stdout: "k": "2", 2024-09-20T22:06:30.789 INFO:teuthology.orchestra.run.smithi060.stdout: "m": "1", 2024-09-20T22:06:30.789 INFO:teuthology.orchestra.run.smithi060.stdout: "plugin": "jerasure", 2024-09-20T22:06:30.789 INFO:teuthology.orchestra.run.smithi060.stdout: "technique": "reed_sol_van" 2024-09-20T22:06:30.789 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:30.789 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps_queue": [], 2024-09-20T22:06:30.789 INFO:teuthology.orchestra.run.smithi060.stdout: "new_removed_snaps": [], 2024-09-20T22:06:30.789 INFO:teuthology.orchestra.run.smithi060.stdout: "new_purged_snaps": [], 2024-09-20T22:06:30.789 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_node_flags": {}, 2024-09-20T22:06:30.789 INFO:teuthology.orchestra.run.smithi060.stdout: "device_class_flags": {}, 2024-09-20T22:06:30.789 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode": { 2024-09-20T22:06:30.790 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode_enabled": false, 2024-09-20T22:06:30.790 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_bucket_count": 0, 2024-09-20T22:06:30.790 INFO:teuthology.orchestra.run.smithi060.stdout: "degraded_stretch_mode": 0, 2024-09-20T22:06:30.790 INFO:teuthology.orchestra.run.smithi060.stdout: "recovering_stretch_mode": 0, 2024-09-20T22:06:30.790 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode_bucket": 0 2024-09-20T22:06:30.790 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:30.790 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:06:30.825 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:30.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:30.826 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:30.927 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:30.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:30.979 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:30.979 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:31.027 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:06:31.080 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:31.117 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:31.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:31.118 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:31.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:31.237 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:31 smithi067 ceph-mon[33511]: pgmap v159: 158 pgs: 1 peering, 1 remapped+peering, 2 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 753 op/s; 23311/203124 objects degraded (11.476%); 25270/203124 objects misplaced (12.441%); 16 MiB/s, 255 objects/s recovering 2024-09-20T22:06:31.237 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:31 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4051108503' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:06:31.237 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:31 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:06:31.237 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:31 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2281188926' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-20T22:06:31.260 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:31.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:31.261 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:31.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:31 smithi060 ceph-mon[29694]: pgmap v159: 158 pgs: 1 peering, 1 remapped+peering, 2 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 753 op/s; 23311/203124 objects degraded (11.476%); 25270/203124 objects misplaced (12.441%); 16 MiB/s, 255 objects/s recovering 2024-09-20T22:06:31.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:31 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4051108503' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:06:31.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:31 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:06:31.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:31 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2281188926' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-20T22:06:31.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:31 smithi060 ceph-mon[40290]: pgmap v159: 158 pgs: 1 peering, 1 remapped+peering, 2 active+undersized+degraded+remapped+backfill_toofull, 9 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 4.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 753 op/s; 23311/203124 objects degraded (11.476%); 25270/203124 objects misplaced (12.441%); 16 MiB/s, 255 objects/s recovering 2024-09-20T22:06:31.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:31 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4051108503' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:06:31.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:31 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:06:31.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:31 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2281188926' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-20T22:06:31.362 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:31.393 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:31.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:31.394 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:31.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:31.536 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:31.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:31.537 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:31.623 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:06:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:06:31.624 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:06:31.638 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:31.663 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:06:31.664 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:06:31.669 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:31.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:31.670 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:31.690 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:06:31.690 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:06:31.707 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:06:31.707 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:06:31.724 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:06:31.724 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:06:31.751 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:06:31.752 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:06:31.771 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:31.790 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:06:31.790 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:06:31.814 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:31.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:31.816 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:31.917 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:31.955 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:31.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:31.956 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:32.057 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:32.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:32.100 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:32.100 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:32.201 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:32.255 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:32.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:32.256 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:32.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:06:31 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:06:32.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:32 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:06:32.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:32 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4051108503' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:06:32.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:32 smithi067 ceph-mon[33511]: osdmap e135: 8 total, 7 up, 8 in 2024-09-20T22:06:32.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:32 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 23311/203124 objects degraded (11.476%), 35 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:32.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:32 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2435396661' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:32.342 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:32 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:06:32.342 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:32 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4051108503' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:06:32.342 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:32 smithi060 ceph-mon[29694]: osdmap e135: 8 total, 7 up, 8 in 2024-09-20T22:06:32.342 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:32 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 23311/203124 objects degraded (11.476%), 35 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:32.342 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:32 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2435396661' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:32.342 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:32 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:06:32.342 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:32 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4051108503' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:06:32.342 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:32 smithi060 ceph-mon[40290]: osdmap e135: 8 total, 7 up, 8 in 2024-09-20T22:06:32.343 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:32 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 23311/203124 objects degraded (11.476%), 35 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:32.343 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:32 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2435396661' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:32.357 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:32.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:32.385 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:32.386 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:32.401 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:06:32.401 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:06:32.486 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:32.524 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:32.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:32.525 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:32.626 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:32.660 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:32.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:32.661 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:32.762 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:32.796 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:32.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:32.797 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:32.862 INFO:tasks.thrashosds.thrasher:j is {'epoch': 134, 'fsid': '3df9647e-779b-11ef-baf5-efdc52797490', 'created': '2024-09-20T21:57:31.244934+0000', 'modified': '2024-09-20T22:06:26.396885+0000', 'last_up_change': '2024-09-20T22:04:15.685933+0000', 'last_in_change': '2024-09-20T22:02:21.189225+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 18, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 7, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-20T22:00:35.131125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-20T22:02:47.974480+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6700000762939453, 'score_stable': 2.6700000762939453, 'optimal_score': 0.75, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-20T22:02:48.902465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-20T22:02:50.620057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.240000009536743, 'score_stable': 2.240000009536743, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.9700000286102295, 'raw_score_stable': 1.9700000286102295, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-20T22:02:52.722912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.7400000095367432, 'score_stable': 1.7400000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.5299999713897705, 'raw_score_stable': 1.5299999713897705, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-20T22:02:54.794248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'unique_pool_0', 'create_time': '2024-09-20T22:03:51.292196+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 26, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '121', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3.059999942779541, 'score_stable': 1.840000033378601, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.690000057220459, 'raw_score_stable': 1.6200000047683716, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '0997c01a-a9bd-421e-abcc-fbcf7abe8512', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 133, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6802', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6803', 'nonce': 1395849550}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6804', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6805', 'nonce': 1395849550}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6808', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6809', 'nonce': 1395849550}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6806', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6807', 'nonce': 1395849550}]}, 'public_addr': '172.21.15.60:6803/1395849550', 'cluster_addr': '172.21.15.60:6805/1395849550', 'heartbeat_back_addr': '172.21.15.60:6809/1395849550', 'heartbeat_front_addr': '172.21.15.60:6807/1395849550', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c04bdcb1-d76b-47c8-a38b-c02043df6ba8', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 80, 'down_at': 91, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6810', 'nonce': 145283513}, {'type': 'v1', 'addr': '172.21.15.60:6811', 'nonce': 145283513}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6812', 'nonce': 145283513}, {'type': 'v1', 'addr': '172.21.15.60:6813', 'nonce': 145283513}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6816', 'nonce': 145283513}, {'type': 'v1', 'addr': '172.21.15.60:6817', 'nonce': 145283513}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6814', 'nonce': 145283513}, {'type': 'v1', 'addr': '172.21.15.60:6815', 'nonce': 145283513}]}, 'public_addr': '172.21.15.60:6811/145283513', 'cluster_addr': '172.21.15.60:6813/145283513', 'heartbeat_back_addr': '172.21.15.60:6817/145283513', 'heartbeat_front_addr': '172.21.15.60:6815/145283513', 'state': ['exists']}, {'osd': 2, 'uuid': '32c09d19-eb98-4926-a44e-e7ddee152956', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 132, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6818', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6819', 'nonce': 3698562274}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6820', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6821', 'nonce': 3698562274}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6824', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6825', 'nonce': 3698562274}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6822', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6823', 'nonce': 3698562274}]}, 'public_addr': '172.21.15.60:6819/3698562274', 'cluster_addr': '172.21.15.60:6821/3698562274', 'heartbeat_back_addr': '172.21.15.60:6825/3698562274', 'heartbeat_front_addr': '172.21.15.60:6823/3698562274', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'b1c2cb69-8565-49ac-b854-7ea7615c6660', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 133, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6826', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6827', 'nonce': 2592322449}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6828', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6829', 'nonce': 2592322449}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6832', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6833', 'nonce': 2592322449}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6830', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6831', 'nonce': 2592322449}]}, 'public_addr': '172.21.15.60:6827/2592322449', 'cluster_addr': '172.21.15.60:6829/2592322449', 'heartbeat_back_addr': '172.21.15.60:6833/2592322449', 'heartbeat_front_addr': '172.21.15.60:6831/2592322449', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '04a547d6-fc48-43b7-a8d7-18df8a83aa96', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 123, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6800', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6801', 'nonce': 3243564239}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6802', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6803', 'nonce': 3243564239}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6806', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6807', 'nonce': 3243564239}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6804', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6805', 'nonce': 3243564239}]}, 'public_addr': '172.21.15.67:6801/3243564239', 'cluster_addr': '172.21.15.67:6803/3243564239', 'heartbeat_back_addr': '172.21.15.67:6807/3243564239', 'heartbeat_front_addr': '172.21.15.67:6805/3243564239', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4763aa10-c367-485b-b525-e5241e642b86', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 121, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6808', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6809', 'nonce': 1622258887}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6810', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6811', 'nonce': 1622258887}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6814', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6815', 'nonce': 1622258887}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6812', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6813', 'nonce': 1622258887}]}, 'public_addr': '172.21.15.67:6809/1622258887', 'cluster_addr': '172.21.15.67:6811/1622258887', 'heartbeat_back_addr': '172.21.15.67:6815/1622258887', 'heartbeat_front_addr': '172.21.15.67:6813/1622258887', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'a359116e-c013-4e2f-8b79-86219516ee16', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 48, 'up_thru': 123, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6816', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6817', 'nonce': 1187114}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6818', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6819', 'nonce': 1187114}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6822', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6823', 'nonce': 1187114}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6820', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6821', 'nonce': 1187114}]}, 'public_addr': '172.21.15.67:6817/1187114', 'cluster_addr': '172.21.15.67:6819/1187114', 'heartbeat_back_addr': '172.21.15.67:6823/1187114', 'heartbeat_front_addr': '172.21.15.67:6821/1187114', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a4bf4188-f133-4a31-8897-d7e8182248d7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 57, 'up_thru': 124, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6824', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6825', 'nonce': 3021842840}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6826', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6827', 'nonce': 3021842840}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6830', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6831', 'nonce': 3021842840}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6828', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6829', 'nonce': 3021842840}]}, 'public_addr': '172.21.15.67:6825/3021842840', 'cluster_addr': '172.21.15.67:6827/3021842840', 'heartbeat_back_addr': '172.21.15.67:6831/3021842840', 'heartbeat_front_addr': '172.21.15.67:6829/3021842840', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:06:07.734923+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:04:02.357858+0000', 'dead_epoch': 90}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:05:42.634604+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:06:07.263859+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:05:26.679800+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:05:39.734002+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:05:48.482150+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:05:27.631709+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.1a', 'mappings': [{'from': 4, 'to': 5}]}, {'pgid': '3.1c', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '6.7', 'mappings': [{'from': 3, 'to': 0}]}, {'pgid': '6.9', 'mappings': [{'from': 7, 'to': 1}]}, {'pgid': '6.b', 'mappings': [{'from': 3, 'to': 0}]}, {'pgid': '6.d', 'mappings': [{'from': 5, 'to': 6}]}, {'pgid': '6.11', 'mappings': [{'from': 3, 'to': 1}]}, {'pgid': '6.19', 'mappings': [{'from': 3, 'to': 6}, {'from': 5, 'to': 0}]}, {'pgid': '7.3', 'mappings': [{'from': 3, 'to': 1}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '3.1a', 'osds': [5, 2, 4]}, {'pgid': '3.1c', 'osds': [5, 2, 4]}, {'pgid': '6.9', 'osds': [0, 2, 7]}, {'pgid': '6.b', 'osds': [0, 7, 3]}, {'pgid': '6.d', 'osds': [6, 0, 5]}, {'pgid': '6.11', 'osds': [0, 5, 3]}, {'pgid': '6.19', 'osds': [0, 6, 3]}, {'pgid': '7.3', 'osds': [2, 4, 3]}, {'pgid': '7.10', 'osds': [3, 2, 5]}, {'pgid': '7.11', 'osds': [3, 6, 4]}, {'pgid': '7.12', 'osds': [2, 3]}, {'pgid': '7.14', 'osds': [0, 4]}, {'pgid': '7.15', 'osds': [7, 2, 0]}, {'pgid': '7.16', 'osds': [0, 7, 3]}, {'pgid': '7.17', 'osds': [0, 3, 6]}, {'pgid': '7.19', 'osds': [0, 4]}], 'primary_temp': [], 'blocklist': {'172.21.15.60:0/890154710': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:0/3386526350': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:6801/604714621': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:6800/604714621': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:6800/3151907776': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/956264518': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/4150585713': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/3311255314': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:0/3965541528': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:6800/3734574621': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:0/1729862661': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:6801/3734574621': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:0/3927294026': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:6801/3151907776': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/1871351181': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:0/1854455098': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:0/2221262203': '2024-09-21T22:03:19.518405+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-20T22:06:32.862 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-09-20T22:06:32.898 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:32.932 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:32.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:32.933 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:33.033 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:33.073 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:33.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:33.074 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:33.175 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:33.208 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:33.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:33.209 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:33.242 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:33 smithi060 ceph-mon[29694]: 7.18 scrub starts 2024-09-20T22:06:33.242 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:33 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:06:33.242 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:33 smithi060 ceph-mon[29694]: pgmap v161: 158 pgs: 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.8 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.53k op/s; 21382/186450 objects degraded (11.468%); 25089/186450 objects misplaced (13.456%); 15 MiB/s, 233 objects/s recovering 2024-09-20T22:06:33.242 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:33 smithi060 ceph-mon[29694]: 7.10 scrub starts 2024-09-20T22:06:33.242 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:33 smithi060 ceph-mon[29694]: 7.18 scrub ok 2024-09-20T22:06:33.242 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:33 smithi060 ceph-mon[29694]: 3.1a scrub starts 2024-09-20T22:06:33.242 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:33 smithi060 ceph-mon[29694]: 3.1a scrub ok 2024-09-20T22:06:33.242 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:33 smithi060 ceph-mon[40290]: 7.18 scrub starts 2024-09-20T22:06:33.242 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:33 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:06:33.242 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:33 smithi060 ceph-mon[40290]: pgmap v161: 158 pgs: 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.8 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.53k op/s; 21382/186450 objects degraded (11.468%); 25089/186450 objects misplaced (13.456%); 15 MiB/s, 233 objects/s recovering 2024-09-20T22:06:33.243 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:33 smithi060 ceph-mon[40290]: 7.10 scrub starts 2024-09-20T22:06:33.243 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:33 smithi060 ceph-mon[40290]: 7.18 scrub ok 2024-09-20T22:06:33.243 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:33 smithi060 ceph-mon[40290]: 3.1a scrub starts 2024-09-20T22:06:33.243 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:33 smithi060 ceph-mon[40290]: 3.1a scrub ok 2024-09-20T22:06:33.293 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:06:33.293 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:06:33.309 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:33.342 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:33.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:33.343 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:33.405 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:33 smithi067 ceph-mon[33511]: 7.18 scrub starts 2024-09-20T22:06:33.405 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:33 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:06:33.405 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:33 smithi067 ceph-mon[33511]: pgmap v161: 158 pgs: 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.8 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.53k op/s; 21382/186450 objects degraded (11.468%); 25089/186450 objects misplaced (13.456%); 15 MiB/s, 233 objects/s recovering 2024-09-20T22:06:33.405 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:33 smithi067 ceph-mon[33511]: 7.10 scrub starts 2024-09-20T22:06:33.405 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:33 smithi067 ceph-mon[33511]: 7.18 scrub ok 2024-09-20T22:06:33.405 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:33 smithi067 ceph-mon[33511]: 3.1a scrub starts 2024-09-20T22:06:33.406 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:33 smithi067 ceph-mon[33511]: 3.1a scrub ok 2024-09-20T22:06:33.444 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:33.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:33.487 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:33.487 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:33.587 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:33.625 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:33.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:33.626 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:33.726 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:33.764 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:33.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:33.765 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:33.866 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:33.902 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:33.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:33.903 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:34.004 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:34.032 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:34.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:34.033 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:34.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:34.159 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:34.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:34.160 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:34.261 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:34.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:34 smithi060 ceph-mon[29694]: 7.10 scrub ok 2024-09-20T22:06:34.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:34 smithi060 ceph-mon[29694]: 6.13 scrub starts 2024-09-20T22:06:34.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:34 smithi060 ceph-mon[29694]: 6.13 scrub ok 2024-09-20T22:06:34.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:34 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:06:34.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:34 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:06:34.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:34 smithi060 ceph-mon[40290]: 7.10 scrub ok 2024-09-20T22:06:34.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:34 smithi060 ceph-mon[40290]: 6.13 scrub starts 2024-09-20T22:06:34.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:34 smithi060 ceph-mon[40290]: 6.13 scrub ok 2024-09-20T22:06:34.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:34 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:06:34.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:34 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:06:34.301 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:34.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:34.302 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:34.403 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:34.445 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:34.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:34.446 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:34.511 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T22:06:34.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:34 smithi067 ceph-mon[33511]: 7.10 scrub ok 2024-09-20T22:06:34.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:34 smithi067 ceph-mon[33511]: 6.13 scrub starts 2024-09-20T22:06:34.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:34 smithi067 ceph-mon[33511]: 6.13 scrub ok 2024-09-20T22:06:34.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:34 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:06:34.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:34 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:06:34.547 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:34.574 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:34.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:34.575 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:34.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:34.713 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:34.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:34.714 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:34.814 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:34.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:34.855 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:34.855 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:34.956 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:35.012 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:35.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:35.013 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:35.113 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:35.145 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:35.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:35.146 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:35.247 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:35.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:35 smithi067 ceph-mon[33511]: 3.b scrub starts 2024-09-20T22:06:35.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:35 smithi067 ceph-mon[33511]: 3.b scrub ok 2024-09-20T22:06:35.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:35 smithi067 ceph-mon[33511]: pgmap v162: 158 pgs: 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.8 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 694 op/s; 21382/186450 objects degraded (11.468%); 25089/186450 objects misplaced (13.456%) 2024-09-20T22:06:35.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:35 smithi067 ceph-mon[33511]: osdmap e136: 8 total, 7 up, 8 in 2024-09-20T22:06:35.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:35 smithi067 ceph-mon[33511]: 3.10 scrub starts 2024-09-20T22:06:35.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:35 smithi067 ceph-mon[33511]: 3.10 scrub ok 2024-09-20T22:06:35.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:35 smithi067 ceph-mon[33511]: 7.15 scrub starts 2024-09-20T22:06:35.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:35 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:06:35.285 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:06:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:06:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:06:35.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:35 smithi060 ceph-mon[29694]: 3.b scrub starts 2024-09-20T22:06:35.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:35 smithi060 ceph-mon[29694]: 3.b scrub ok 2024-09-20T22:06:35.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:35 smithi060 ceph-mon[29694]: pgmap v162: 158 pgs: 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.8 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 694 op/s; 21382/186450 objects degraded (11.468%); 25089/186450 objects misplaced (13.456%) 2024-09-20T22:06:35.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:35 smithi060 ceph-mon[29694]: osdmap e136: 8 total, 7 up, 8 in 2024-09-20T22:06:35.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:35 smithi060 ceph-mon[29694]: 3.10 scrub starts 2024-09-20T22:06:35.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:35 smithi060 ceph-mon[29694]: 3.10 scrub ok 2024-09-20T22:06:35.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:35 smithi060 ceph-mon[29694]: 7.15 scrub starts 2024-09-20T22:06:35.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:35 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:06:35.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:35 smithi060 ceph-mon[40290]: 3.b scrub starts 2024-09-20T22:06:35.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:35 smithi060 ceph-mon[40290]: 3.b scrub ok 2024-09-20T22:06:35.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:35 smithi060 ceph-mon[40290]: pgmap v162: 158 pgs: 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 3 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.8 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 694 op/s; 21382/186450 objects degraded (11.468%); 25089/186450 objects misplaced (13.456%) 2024-09-20T22:06:35.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:35 smithi060 ceph-mon[40290]: osdmap e136: 8 total, 7 up, 8 in 2024-09-20T22:06:35.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:35 smithi060 ceph-mon[40290]: 3.10 scrub starts 2024-09-20T22:06:35.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:35 smithi060 ceph-mon[40290]: 3.10 scrub ok 2024-09-20T22:06:35.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:35 smithi060 ceph-mon[40290]: 7.15 scrub starts 2024-09-20T22:06:35.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:35 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:06:35.293 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:35.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:35.294 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:35.294 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:06:35.394 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:35.423 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:35.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:35.424 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:35.524 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:35.552 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:35.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:35.553 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:35.572 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:35.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:35.716 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:35.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:35.717 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:35.818 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:35.853 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:35.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:35.854 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:35.954 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:35.987 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:35.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:35.989 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:36.042 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:06:36.089 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:36.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:36.121 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:36.222 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:36.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:36 smithi067 ceph-mon[33511]: 6.1f scrub starts 2024-09-20T22:06:36.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:36 smithi067 ceph-mon[33511]: 6.1f scrub ok 2024-09-20T22:06:36.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:36 smithi067 ceph-mon[33511]: 7.15 scrub ok 2024-09-20T22:06:36.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:36 smithi067 ceph-mon[33511]: 6.a scrub starts 2024-09-20T22:06:36.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:36 smithi067 ceph-mon[33511]: 6.a scrub ok 2024-09-20T22:06:36.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:36 smithi067 ceph-mon[33511]: 3.13 scrub starts 2024-09-20T22:06:36.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:36 smithi067 ceph-mon[33511]: 3.13 scrub ok 2024-09-20T22:06:36.264 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:06:36.264 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:06:36.276 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:36.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:36.277 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:36.378 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:36.417 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:36.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:36.418 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:36.519 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:36 smithi060 ceph-mon[29694]: 6.1f scrub starts 2024-09-20T22:06:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:36 smithi060 ceph-mon[29694]: 6.1f scrub ok 2024-09-20T22:06:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:36 smithi060 ceph-mon[29694]: 7.15 scrub ok 2024-09-20T22:06:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:36 smithi060 ceph-mon[29694]: 6.a scrub starts 2024-09-20T22:06:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:36 smithi060 ceph-mon[29694]: 6.a scrub ok 2024-09-20T22:06:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:36 smithi060 ceph-mon[29694]: 3.13 scrub starts 2024-09-20T22:06:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:36 smithi060 ceph-mon[29694]: 3.13 scrub ok 2024-09-20T22:06:36.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:36 smithi060 ceph-mon[40290]: 6.1f scrub starts 2024-09-20T22:06:36.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:36 smithi060 ceph-mon[40290]: 6.1f scrub ok 2024-09-20T22:06:36.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:36 smithi060 ceph-mon[40290]: 7.15 scrub ok 2024-09-20T22:06:36.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:36 smithi060 ceph-mon[40290]: 6.a scrub starts 2024-09-20T22:06:36.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:36 smithi060 ceph-mon[40290]: 6.a scrub ok 2024-09-20T22:06:36.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:36 smithi060 ceph-mon[40290]: 3.13 scrub starts 2024-09-20T22:06:36.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:36 smithi060 ceph-mon[40290]: 3.13 scrub ok 2024-09-20T22:06:36.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:36.574 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:36.574 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:36.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:36.711 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:36.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:36.713 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:36.813 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:36.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:36.858 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:36.858 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:36.959 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:37.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:37.009 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:37.009 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:37.026 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T22:06:37.035 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:37.110 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:37.140 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:37.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:37.142 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:37.223 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:37 smithi060 ceph-mon[40290]: 6.f scrub starts 2024-09-20T22:06:37.223 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:37 smithi060 ceph-mon[40290]: 6.f scrub ok 2024-09-20T22:06:37.223 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:37 smithi060 ceph-mon[40290]: pgmap v164: 158 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 19236/167157 objects degraded (11.508%); 21935/167157 objects misplaced (13.122%) 2024-09-20T22:06:37.224 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:37 smithi060 ceph-mon[40290]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:06:37.224 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:37 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 19236/167157 objects degraded (11.508%), 35 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:37.224 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:37 smithi060 ceph-mon[40290]: osdmap e137: 8 total, 7 up, 8 in 2024-09-20T22:06:37.224 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:37 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:37.224 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:37 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:37.224 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:37 smithi060 ceph-mon[29694]: 6.f scrub starts 2024-09-20T22:06:37.224 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:37 smithi060 ceph-mon[29694]: 6.f scrub ok 2024-09-20T22:06:37.224 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:37 smithi060 ceph-mon[29694]: pgmap v164: 158 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 19236/167157 objects degraded (11.508%); 21935/167157 objects misplaced (13.122%) 2024-09-20T22:06:37.224 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:37 smithi060 ceph-mon[29694]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:06:37.224 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:37 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 19236/167157 objects degraded (11.508%), 35 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:37.224 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:37 smithi060 ceph-mon[29694]: osdmap e137: 8 total, 7 up, 8 in 2024-09-20T22:06:37.225 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:37 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:37.225 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:37 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:37.242 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:37.274 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:37.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:37.275 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:37.375 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:37.433 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:37.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:37.434 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:37.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:37 smithi067 ceph-mon[33511]: 6.f scrub starts 2024-09-20T22:06:37.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:37 smithi067 ceph-mon[33511]: 6.f scrub ok 2024-09-20T22:06:37.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:37 smithi067 ceph-mon[33511]: pgmap v164: 158 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 19236/167157 objects degraded (11.508%); 21935/167157 objects misplaced (13.122%) 2024-09-20T22:06:37.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:37 smithi067 ceph-mon[33511]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:06:37.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:37 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 19236/167157 objects degraded (11.508%), 35 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:37.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:37 smithi067 ceph-mon[33511]: osdmap e137: 8 total, 7 up, 8 in 2024-09-20T22:06:37.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:37 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:37.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:37 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:37.534 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:37.562 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:37.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:37.564 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:37.655 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:37.665 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:37.742 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:37.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:37.743 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:37.748 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:06:37.748 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:06:37.753 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:06:37.788 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:06:37.788 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:06:37.830 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:06:37.831 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:06:37.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:37.863 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [1] live_osds: [0, 2, 3, 4, 5, 6, 7] 2024-09-20T22:06:37.863 INFO:tasks.thrashosds.thrasher:Reviving osd 1 2024-09-20T22:06:37.863 INFO:tasks.cephadm.osd.1:Restarting osd.1 (starting--it wasn't running)... 2024-09-20T22:06:37.864 DEBUG:teuthology.orchestra.run.smithi060:osd.1> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.1.service 2024-09-20T22:06:37.866 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl start ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.1 2024-09-20T22:06:37.867 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:06:37.867 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:06:37.875 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:37.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:37.876 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:37.914 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:06:37.914 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:06:37.946 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:06:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:06:37.977 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:37.981 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:06:37.981 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:06:38.002 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:38.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:38.003 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:38.043 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:37 smithi060 systemd[1]: Starting Ceph osd.1 for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:06:38.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:38.149 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:38.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:38.150 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:38.251 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:38.282 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:38.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:38.284 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:38.384 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:38.417 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:38.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:38.417 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:38.465 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:38 smithi060 ceph-mon[29694]: 3.c scrub starts 2024-09-20T22:06:38.465 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:38 smithi060 ceph-mon[29694]: 3.c scrub ok 2024-09-20T22:06:38.465 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:38 smithi060 ceph-mon[29694]: 5.6 scrub starts 2024-09-20T22:06:38.465 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:38 smithi060 ceph-mon[29694]: 5.6 scrub ok 2024-09-20T22:06:38.466 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:38 smithi060 ceph-mon[29694]: 4.11 scrub starts 2024-09-20T22:06:38.466 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:38 smithi060 ceph-mon[29694]: 4.11 scrub ok 2024-09-20T22:06:38.466 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:38 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1172123494' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:38.466 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:38 smithi060 ceph-mon[40290]: 3.c scrub starts 2024-09-20T22:06:38.466 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:38 smithi060 ceph-mon[40290]: 3.c scrub ok 2024-09-20T22:06:38.466 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:38 smithi060 ceph-mon[40290]: 5.6 scrub starts 2024-09-20T22:06:38.466 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:38 smithi060 ceph-mon[40290]: 5.6 scrub ok 2024-09-20T22:06:38.466 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:38 smithi060 ceph-mon[40290]: 4.11 scrub starts 2024-09-20T22:06:38.467 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:38 smithi060 ceph-mon[40290]: 4.11 scrub ok 2024-09-20T22:06:38.467 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:38 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1172123494' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:38.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:38 smithi067 ceph-mon[33511]: 3.c scrub starts 2024-09-20T22:06:38.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:38 smithi067 ceph-mon[33511]: 3.c scrub ok 2024-09-20T22:06:38.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:38 smithi067 ceph-mon[33511]: 5.6 scrub starts 2024-09-20T22:06:38.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:38 smithi067 ceph-mon[33511]: 5.6 scrub ok 2024-09-20T22:06:38.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:38 smithi067 ceph-mon[33511]: 4.11 scrub starts 2024-09-20T22:06:38.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:38 smithi067 ceph-mon[33511]: 4.11 scrub ok 2024-09-20T22:06:38.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:38 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1172123494' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:38.518 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:38.558 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:38.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:38.559 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:38.660 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:38.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:38.691 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:38.691 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:38.792 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:38.830 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:38.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:38.831 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:38.931 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:38.933 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:38.959 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:38.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:38.960 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:38.974 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:06:38.975 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:06:39.060 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:39.092 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:39.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:39.093 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:39.135 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:06:39.194 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:39.212 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:39 smithi060 ceph-mon[29694]: pgmap v166: 158 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 737 B/s rd, 0 op/s; 19236/167157 objects degraded (11.508%); 21935/167157 objects misplaced (13.122%) 2024-09-20T22:06:39.212 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:39 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2067152295' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:06:39.212 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:39 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:06:39.212 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:39 smithi060 ceph-mon[40290]: pgmap v166: 158 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 737 B/s rd, 0 op/s; 19236/167157 objects degraded (11.508%); 21935/167157 objects misplaced (13.122%) 2024-09-20T22:06:39.213 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:39 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2067152295' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:06:39.213 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:39 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:06:39.223 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:39.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:39.224 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:39.325 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:39.359 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:39.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:39.360 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:39.414 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:39 smithi067 ceph-mon[33511]: pgmap v166: 158 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 737 B/s rd, 0 op/s; 19236/167157 objects degraded (11.508%); 21935/167157 objects misplaced (13.122%) 2024-09-20T22:06:39.414 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:39 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2067152295' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:06:39.414 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:39 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:06:39.461 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:39.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:39.498 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:39.499 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:39.599 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:39.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:39.653 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:39.654 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:39.754 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:39.794 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:39.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:39.795 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:39.802 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:06:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:06:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:06:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:06:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169948 7.9 7:94ca5e9f:::benchmark_data_smithi060_78800_object36915:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:06:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440369+0000", 2024-09-20T22:06:39.802 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 41.357019682000001, 2024-09-20T22:06:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.04459061, 2024-09-20T22:06:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:06:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169948 2024-09-20T22:06:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440369+0000", 2024-09-20T22:06:39.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440369+0000", 2024-09-20T22:06:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440371+0000", 2024-09-20T22:06:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.567e-06 2024-09-20T22:06:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440375+0000", 2024-09-20T22:06:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7910000000000004e-06 2024-09-20T22:06:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440388+0000", 2024-09-20T22:06:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2354999999999999e-05 2024-09-20T22:06:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440395+0000", 2024-09-20T22:06:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5739999999999998e-06 2024-09-20T22:06:39.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.650274+0000", 2024-09-20T22:06:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.209878595 2024-09-20T22:06:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.650283+0000", 2024-09-20T22:06:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6070000000000008e-06 2024-09-20T22:06:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.484885+0000", 2024-09-20T22:06:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.83460233500000003 2024-09-20T22:06:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.484960+0000", 2024-09-20T22:06:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4785999999999999e-05 2024-09-20T22:06:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169959 7.19 7:9aa79aaf:::benchmark_data_smithi060_78800_object36926:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:06:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440410+0000", 2024-09-20T22:06:39.807 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 41.356978746000003, 2024-09-20T22:06:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0776166780000001, 2024-09-20T22:06:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:06:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169959 2024-09-20T22:06:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440410+0000", 2024-09-20T22:06:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440410+0000", 2024-09-20T22:06:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440410+0000", 2024-09-20T22:06:39.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9299999999999999e-07 2024-09-20T22:06:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440413+0000", 2024-09-20T22:06:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7219999999999999e-06 2024-09-20T22:06:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440419+0000", 2024-09-20T22:06:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0449999999999997e-06 2024-09-20T22:06:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440424+0000", 2024-09-20T22:06:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7090000000000001e-06 2024-09-20T22:06:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.651028+0000", 2024-09-20T22:06:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.21060456799999999 2024-09-20T22:06:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.651036+0000", 2024-09-20T22:06:39.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8720000000000002e-06 2024-09-20T22:06:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.517832+0000", 2024-09-20T22:06:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.86679538099999998 2024-09-20T22:06:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.518027+0000", 2024-09-20T22:06:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000194988 2024-09-20T22:06:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169973 7.19 7:9cb6a65c:::benchmark_data_smithi060_78800_object36940:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:06:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.452156+0000", 2024-09-20T22:06:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 41.345232983999999, 2024-09-20T22:06:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.079163852, 2024-09-20T22:06:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169973 2024-09-20T22:06:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.452156+0000", 2024-09-20T22:06:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.452156+0000", 2024-09-20T22:06:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.452158+0000", 2024-09-20T22:06:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0999999999999998e-06 2024-09-20T22:06:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.452165+0000", 2024-09-20T22:06:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5039999999999996e-06 2024-09-20T22:06:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.452181+0000", 2024-09-20T22:06:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5226e-05 2024-09-20T22:06:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.452190+0000", 2024-09-20T22:06:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0380000000000005e-06 2024-09-20T22:06:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.651057+0000", 2024-09-20T22:06:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.19886704599999999 2024-09-20T22:06:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.651062+0000", 2024-09-20T22:06:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7230000000000001e-06 2024-09-20T22:06:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.518364+0000", 2024-09-20T22:06:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.86730196400000004 2024-09-20T22:06:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:06:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.518458+0000", 2024-09-20T22:06:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3105999999999999e-05 2024-09-20T22:06:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4", 2024-09-20T22:06:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.518738+0000", 2024-09-20T22:06:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000280005 2024-09-20T22:06:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:06:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.521963+0000", 2024-09-20T22:06:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003225374 2024-09-20T22:06:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.531163+0000", 2024-09-20T22:06:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0091998740000000002 2024-09-20T22:06:39.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:06:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.531225+0000", 2024-09-20T22:06:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2119000000000006e-05 2024-09-20T22:06:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.531320+0000", 2024-09-20T22:06:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4772999999999995e-05 2024-09-20T22:06:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.819 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169975 7.6 7:60dd5d68:::benchmark_data_smithi060_78800_object36942:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:06:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.684899+0000", 2024-09-20T22:06:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 41.112490190000003, 2024-09-20T22:06:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.80166559299999995, 2024-09-20T22:06:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169975 2024-09-20T22:06:39.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.684899+0000", 2024-09-20T22:06:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.684899+0000", 2024-09-20T22:06:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.684900+0000", 2024-09-20T22:06:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.531e-06 2024-09-20T22:06:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.684908+0000", 2024-09-20T22:06:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2719999999999997e-06 2024-09-20T22:06:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.684924+0000", 2024-09-20T22:06:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5821e-05 2024-09-20T22:06:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.684933+0000", 2024-09-20T22:06:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7360000000000007e-06 2024-09-20T22:06:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.685008+0000", 2024-09-20T22:06:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5261999999999999e-05 2024-09-20T22:06:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.685021+0000", 2024-09-20T22:06:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2286000000000001e-05 2024-09-20T22:06:39.823 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.466392+0000", 2024-09-20T22:06:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.78137169799999995 2024-09-20T22:06:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:06:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.466517+0000", 2024-09-20T22:06:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012508199999999999 2024-09-20T22:06:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:06:39.824 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.466835+0000", 2024-09-20T22:06:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000317592 2024-09-20T22:06:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.481276+0000", 2024-09-20T22:06:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014440876 2024-09-20T22:06:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.483189+0000", 2024-09-20T22:06:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001913355 2024-09-20T22:06:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:06:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.486212+0000", 2024-09-20T22:06:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0030228260000000002 2024-09-20T22:06:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:06:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.486474+0000", 2024-09-20T22:06:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026204800000000002 2024-09-20T22:06:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.486564+0000", 2024-09-20T22:06:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0208000000000001e-05 2024-09-20T22:06:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.826 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:197601 7.13 7:c8cc3338:::benchmark_data_smithi060_78800_object64568:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e132)", 2024-09-20T22:06:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:24.370429+0000", 2024-09-20T22:06:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 15.426959463999999, 2024-09-20T22:06:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0703220390000001, 2024-09-20T22:06:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:06:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.827 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 197601 2024-09-20T22:06:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370429+0000", 2024-09-20T22:06:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370429+0000", 2024-09-20T22:06:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370430+0000", 2024-09-20T22:06:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6200000000000006e-07 2024-09-20T22:06:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370435+0000", 2024-09-20T22:06:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5589999999999999e-06 2024-09-20T22:06:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370447+0000", 2024-09-20T22:06:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2058e-05 2024-09-20T22:06:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370454+0000", 2024-09-20T22:06:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4449999999999999e-06 2024-09-20T22:06:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370854+0000", 2024-09-20T22:06:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00039970400000000002 2024-09-20T22:06:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op must wait for map", 2024-09-20T22:06:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370861+0000", 2024-09-20T22:06:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8190000000000003e-06 2024-09-20T22:06:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.373710+0000", 2024-09-20T22:06:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0028495270000000001 2024-09-20T22:06:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.373724+0000", 2024-09-20T22:06:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.395e-05 2024-09-20T22:06:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.435399+0000", 2024-09-20T22:06:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0616748069999999 2024-09-20T22:06:39.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:06:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.435541+0000", 2024-09-20T22:06:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014141499999999999 2024-09-20T22:06:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.440680+0000", 2024-09-20T22:06:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0051391170000000003 2024-09-20T22:06:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.440751+0000", 2024-09-20T22:06:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1675999999999996e-05 2024-09-20T22:06:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:197609 7.13 7:c869088e:::benchmark_data_smithi060_78800_object64576:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e132)", 2024-09-20T22:06:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:24.370476+0000", 2024-09-20T22:06:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 15.426913183, 2024-09-20T22:06:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.075397427, 2024-09-20T22:06:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:39.833 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 197609 2024-09-20T22:06:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370476+0000", 2024-09-20T22:06:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370476+0000", 2024-09-20T22:06:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370476+0000", 2024-09-20T22:06:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4299999999999998e-07 2024-09-20T22:06:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370479+0000", 2024-09-20T22:06:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5299999999999999e-06 2024-09-20T22:06:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370486+0000", 2024-09-20T22:06:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.968e-06 2024-09-20T22:06:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370491+0000", 2024-09-20T22:06:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6359999999999999e-06 2024-09-20T22:06:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370914+0000", 2024-09-20T22:06:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00042270099999999999 2024-09-20T22:06:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting_for_map not empty", 2024-09-20T22:06:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370919+0000", 2024-09-20T22:06:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6019999999999999e-06 2024-09-20T22:06:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.373986+0000", 2024-09-20T22:06:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0030664049999999999 2024-09-20T22:06:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.373993+0000", 2024-09-20T22:06:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5979999999999999e-06 2024-09-20T22:06:39.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.440778+0000", 2024-09-20T22:06:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0667841199999999 2024-09-20T22:06:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:06:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.440952+0000", 2024-09-20T22:06:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017486199999999999 2024-09-20T22:06:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:06:39.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.441236+0000", 2024-09-20T22:06:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00028399600000000001 2024-09-20T22:06:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445645+0000", 2024-09-20T22:06:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0044080669999999999 2024-09-20T22:06:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:06:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445661+0000", 2024-09-20T22:06:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6928000000000001e-05 2024-09-20T22:06:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.839 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445783+0000", 2024-09-20T22:06:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121438 2024-09-20T22:06:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:06:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445816+0000", 2024-09-20T22:06:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3413e-05 2024-09-20T22:06:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445873+0000", 2024-09-20T22:06:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6719999999999999e-05 2024-09-20T22:06:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.840 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:197635 7.13 7:c81e98ce:::benchmark_data_smithi060_78800_object64602:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-20T22:06:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:24.390574+0000", 2024-09-20T22:06:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 15.406814491, 2024-09-20T22:06:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0554612830000001, 2024-09-20T22:06:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.841 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 197635 2024-09-20T22:06:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.390574+0000", 2024-09-20T22:06:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.390574+0000", 2024-09-20T22:06:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.390576+0000", 2024-09-20T22:06:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5349999999999999e-06 2024-09-20T22:06:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.390582+0000", 2024-09-20T22:06:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6840000000000001e-06 2024-09-20T22:06:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.390596+0000", 2024-09-20T22:06:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4088e-05 2024-09-20T22:06:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.390603+0000", 2024-09-20T22:06:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8269999999999998e-06 2024-09-20T22:06:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.390654+0000", 2024-09-20T22:06:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0074000000000002e-05 2024-09-20T22:06:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.844 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.390662+0000", 2024-09-20T22:06:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5609999999999999e-06 2024-09-20T22:06:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.442542+0000", 2024-09-20T22:06:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0518796610000001 2024-09-20T22:06:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:06:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.442657+0000", 2024-09-20T22:06:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115215 2024-09-20T22:06:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:06:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.442883+0000", 2024-09-20T22:06:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000225715 2024-09-20T22:06:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:06:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445668+0000", 2024-09-20T22:06:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0027853349999999999 2024-09-20T22:06:39.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445727+0000", 2024-09-20T22:06:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9296000000000003e-05 2024-09-20T22:06:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445924+0000", 2024-09-20T22:06:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000197104 2024-09-20T22:06:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:06:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445954+0000", 2024-09-20T22:06:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0094999999999999e-05 2024-09-20T22:06:39.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.446036+0000", 2024-09-20T22:06:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1093000000000004e-05 2024-09-20T22:06:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:197657 7.13 7:cbcc9238:::benchmark_data_smithi060_78800_object64624:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-20T22:06:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:24.409026+0000", 2024-09-20T22:06:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 15.388362854, 2024-09-20T22:06:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.037258835, 2024-09-20T22:06:39.848 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 197657 2024-09-20T22:06:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.409026+0000", 2024-09-20T22:06:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.409026+0000", 2024-09-20T22:06:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.409027+0000", 2024-09-20T22:06:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.141e-06 2024-09-20T22:06:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.409032+0000", 2024-09-20T22:06:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7199999999999997e-06 2024-09-20T22:06:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.409042+0000", 2024-09-20T22:06:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7450000000000007e-06 2024-09-20T22:06:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.409047+0000", 2024-09-20T22:06:39.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1460000000000004e-06 2024-09-20T22:06:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.409095+0000", 2024-09-20T22:06:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8331000000000002e-05 2024-09-20T22:06:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.409103+0000", 2024-09-20T22:06:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3270000000000001e-06 2024-09-20T22:06:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.444150+0000", 2024-09-20T22:06:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.035046707 2024-09-20T22:06:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:06:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.444279+0000", 2024-09-20T22:06:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012934000000000001 2024-09-20T22:06:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:06:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.444483+0000", 2024-09-20T22:06:39.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020355199999999999 2024-09-20T22:06:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:06:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445886+0000", 2024-09-20T22:06:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001403114 2024-09-20T22:06:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.446099+0000", 2024-09-20T22:06:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021258799999999999 2024-09-20T22:06:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.854 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.446146+0000", 2024-09-20T22:06:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7144000000000001e-05 2024-09-20T22:06:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:06:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.446194+0000", 2024-09-20T22:06:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8158999999999997e-05 2024-09-20T22:06:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.446285+0000", 2024-09-20T22:06:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0821000000000003e-05 2024-09-20T22:06:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.855 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:197833 7.13 7:c9e4b84d:::benchmark_data_smithi060_78800_object64800:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-20T22:06:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:24.692243+0000", 2024-09-20T22:06:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 15.105145889999999, 2024-09-20T22:06:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.76879329299999999, 2024-09-20T22:06:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.856 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 197833 2024-09-20T22:06:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.692243+0000", 2024-09-20T22:06:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.692243+0000", 2024-09-20T22:06:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.692244+0000", 2024-09-20T22:06:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.376e-06 2024-09-20T22:06:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.692249+0000", 2024-09-20T22:06:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6750000000000001e-06 2024-09-20T22:06:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.692260+0000", 2024-09-20T22:06:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.114e-05 2024-09-20T22:06:39.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.692267+0000", 2024-09-20T22:06:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0979999999999996e-06 2024-09-20T22:06:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.692325+0000", 2024-09-20T22:06:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7989999999999999e-05 2024-09-20T22:06:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.692334+0000", 2024-09-20T22:06:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3930000000000008e-06 2024-09-20T22:06:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.446327+0000", 2024-09-20T22:06:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.75399364499999999 2024-09-20T22:06:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:06:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.446429+0000", 2024-09-20T22:06:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000102073 2024-09-20T22:06:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.860 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:06:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.446589+0000", 2024-09-20T22:06:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015977799999999999 2024-09-20T22:06:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.451168+0000", 2024-09-20T22:06:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0045788189999999996 2024-09-20T22:06:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.451249+0000", 2024-09-20T22:06:39.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.14e-05 2024-09-20T22:06:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:06:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.460856+0000", 2024-09-20T22:06:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0096071639999999996 2024-09-20T22:06:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:06:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.460913+0000", 2024-09-20T22:06:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6427e-05 2024-09-20T22:06:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.862 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.461036+0000", 2024-09-20T22:06:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012331500000000001 2024-09-20T22:06:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:197843 7.13 7:ccfb7743:::benchmark_data_smithi060_78800_object64810:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-20T22:06:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:24.698206+0000", 2024-09-20T22:06:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 15.099183011999999, 2024-09-20T22:06:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.76305993000000005, 2024-09-20T22:06:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:39.863 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 197843 2024-09-20T22:06:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.698206+0000", 2024-09-20T22:06:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.698206+0000", 2024-09-20T22:06:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.698207+0000", 2024-09-20T22:06:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5880000000000001e-06 2024-09-20T22:06:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.698213+0000", 2024-09-20T22:06:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5779999999999998e-06 2024-09-20T22:06:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.698225+0000", 2024-09-20T22:06:39.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2411e-05 2024-09-20T22:06:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.698233+0000", 2024-09-20T22:06:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7910000000000005e-06 2024-09-20T22:06:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.698300+0000", 2024-09-20T22:06:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6579000000000001e-05 2024-09-20T22:06:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.698311+0000", 2024-09-20T22:06:39.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0791000000000001e-05 2024-09-20T22:06:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.447804+0000", 2024-09-20T22:06:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.74949309200000003 2024-09-20T22:06:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:06:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.447932+0000", 2024-09-20T22:06:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012875899999999999 2024-09-20T22:06:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:06:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.448165+0000", 2024-09-20T22:06:39.867 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023215399999999999 2024-09-20T22:06:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.451210+0000", 2024-09-20T22:06:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003045556 2024-09-20T22:06:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.451288+0000", 2024-09-20T22:06:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8413999999999998e-05 2024-09-20T22:06:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:06:39.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.461097+0000", 2024-09-20T22:06:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0098084709999999992 2024-09-20T22:06:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:06:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.461147+0000", 2024-09-20T22:06:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0435000000000001e-05 2024-09-20T22:06:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.461266+0000", 2024-09-20T22:06:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118311 2024-09-20T22:06:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:197871 7.13 7:cbbabb7b:::benchmark_data_smithi060_78800_object64838:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-20T22:06:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:24.721106+0000", 2024-09-20T22:06:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 15.076282431999999, 2024-09-20T22:06:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.74031054600000001, 2024-09-20T22:06:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 197871 2024-09-20T22:06:39.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.721106+0000", 2024-09-20T22:06:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.721106+0000", 2024-09-20T22:06:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.721108+0000", 2024-09-20T22:06:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2869999999999999e-06 2024-09-20T22:06:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.721113+0000", 2024-09-20T22:06:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0660000000000003e-06 2024-09-20T22:06:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.721125+0000", 2024-09-20T22:06:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2057e-05 2024-09-20T22:06:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.721281+0000", 2024-09-20T22:06:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015638300000000001 2024-09-20T22:06:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.721342+0000", 2024-09-20T22:06:39.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1076999999999998e-05 2024-09-20T22:06:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.721350+0000", 2024-09-20T22:06:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0239999999999997e-06 2024-09-20T22:06:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.449554+0000", 2024-09-20T22:06:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.72820337800000001 2024-09-20T22:06:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:06:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.449622+0000", 2024-09-20T22:06:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7898999999999995e-05 2024-09-20T22:06:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:06:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.449813+0000", 2024-09-20T22:06:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000191151 2024-09-20T22:06:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.451329+0000", 2024-09-20T22:06:39.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0015165109999999999 2024-09-20T22:06:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.454996+0000", 2024-09-20T22:06:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0036671070000000002 2024-09-20T22:06:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:06:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.461278+0000", 2024-09-20T22:06:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.006281928 2024-09-20T22:06:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:06:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.461327+0000", 2024-09-20T22:06:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9169999999999998e-05 2024-09-20T22:06:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.461417+0000", 2024-09-20T22:06:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9507999999999998e-05 2024-09-20T22:06:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:197934 7.13 7:cc84ed6d:::benchmark_data_smithi060_78800_object64901:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-20T22:06:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:24.772217+0000", 2024-09-20T22:06:39.877 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 15.025171897, 2024-09-20T22:06:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.68934804100000002, 2024-09-20T22:06:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 197934 2024-09-20T22:06:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.772217+0000", 2024-09-20T22:06:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.772217+0000", 2024-09-20T22:06:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.772218+0000", 2024-09-20T22:06:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.401e-06 2024-09-20T22:06:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.772224+0000", 2024-09-20T22:06:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5319999999999997e-06 2024-09-20T22:06:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.772232+0000", 2024-09-20T22:06:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0569999999999993e-06 2024-09-20T22:06:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.772237+0000", 2024-09-20T22:06:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7489999999999998e-06 2024-09-20T22:06:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.772286+0000", 2024-09-20T22:06:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9762999999999998e-05 2024-09-20T22:06:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.772295+0000", 2024-09-20T22:06:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.969e-06 2024-09-20T22:06:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.451362+0000", 2024-09-20T22:06:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.67906666599999999 2024-09-20T22:06:39.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:06:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.451453+0000", 2024-09-20T22:06:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1188999999999997e-05 2024-09-20T22:06:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:06:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.451666+0000", 2024-09-20T22:06:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000213203 2024-09-20T22:06:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.455037+0000", 2024-09-20T22:06:39.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0033702770000000001 2024-09-20T22:06:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.455081+0000", 2024-09-20T22:06:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4138999999999999e-05 2024-09-20T22:06:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:06:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.461427+0000", 2024-09-20T22:06:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0063465689999999998 2024-09-20T22:06:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:06:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.461471+0000", 2024-09-20T22:06:39.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3668000000000003e-05 2024-09-20T22:06:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.461565+0000", 2024-09-20T22:06:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3858999999999997e-05 2024-09-20T22:06:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:197968 7.13 7:c9ceb089:::benchmark_data_smithi060_78800_object64935:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-20T22:06:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:24.795195+0000", 2024-09-20T22:06:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 15.002193481000001, 2024-09-20T22:06:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.66655309399999996, 2024-09-20T22:06:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.884 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 197968 2024-09-20T22:06:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.795195+0000", 2024-09-20T22:06:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.795195+0000", 2024-09-20T22:06:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.795196+0000", 2024-09-20T22:06:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4600000000000005e-07 2024-09-20T22:06:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.795199+0000", 2024-09-20T22:06:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.954e-06 2024-09-20T22:06:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.795207+0000", 2024-09-20T22:06:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8809999999999996e-06 2024-09-20T22:06:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.795213+0000", 2024-09-20T22:06:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0270000000000001e-06 2024-09-20T22:06:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.796088+0000", 2024-09-20T22:06:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00087562600000000001 2024-09-20T22:06:39.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.796097+0000", 2024-09-20T22:06:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8389999999999996e-06 2024-09-20T22:06:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.453178+0000", 2024-09-20T22:06:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.65708096400000005 2024-09-20T22:06:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:06:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.453289+0000", 2024-09-20T22:06:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000110624 2024-09-20T22:06:39.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:06:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.453554+0000", 2024-09-20T22:06:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026488199999999998 2024-09-20T22:06:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.457436+0000", 2024-09-20T22:06:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038824060000000001 2024-09-20T22:06:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.457475+0000", 2024-09-20T22:06:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8593999999999997e-05 2024-09-20T22:06:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:06:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.461578+0000", 2024-09-20T22:06:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0041035710000000003 2024-09-20T22:06:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:06:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.461641+0000", 2024-09-20T22:06:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2632000000000006e-05 2024-09-20T22:06:39.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.461748+0000", 2024-09-20T22:06:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010754799999999999 2024-09-20T22:06:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198099 7.18 7:1dc840d9:::benchmark_data_smithi060_78800_object65066:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:06:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.455807+0000", 2024-09-20T22:06:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.341582041000001, 2024-09-20T22:06:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97129763599999996, 2024-09-20T22:06:39.891 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198099 2024-09-20T22:06:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.455807+0000", 2024-09-20T22:06:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.455807+0000", 2024-09-20T22:06:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.455808+0000", 2024-09-20T22:06:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7199999999999998e-07 2024-09-20T22:06:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.455811+0000", 2024-09-20T22:06:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7290000000000002e-06 2024-09-20T22:06:39.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.455822+0000", 2024-09-20T22:06:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0331e-05 2024-09-20T22:06:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.455826+0000", 2024-09-20T22:06:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5719999999999996e-06 2024-09-20T22:06:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.455881+0000", 2024-09-20T22:06:39.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5068999999999997e-05 2024-09-20T22:06:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.455889+0000", 2024-09-20T22:06:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5429999999999996e-06 2024-09-20T22:06:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.422697+0000", 2024-09-20T22:06:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.96680820099999998 2024-09-20T22:06:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:06:39.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.422844+0000", 2024-09-20T22:06:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147128 2024-09-20T22:06:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:06:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.423134+0000", 2024-09-20T22:06:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00028993600000000002 2024-09-20T22:06:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:06:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.426960+0000", 2024-09-20T22:06:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003825803 2024-09-20T22:06:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.427001+0000", 2024-09-20T22:06:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1100000000000003e-05 2024-09-20T22:06:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.427049+0000", 2024-09-20T22:06:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8378999999999999e-05 2024-09-20T22:06:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:06:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.427080+0000", 2024-09-20T22:06:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0397999999999999e-05 2024-09-20T22:06:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.427104+0000", 2024-09-20T22:06:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4675e-05 2024-09-20T22:06:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198205 7.18 7:1bc47e83:::benchmark_data_smithi060_78800_object65172:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:06:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.532610+0000", 2024-09-20T22:06:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.264778376000001, 2024-09-20T22:06:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.89743818799999997, 2024-09-20T22:06:39.898 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198205 2024-09-20T22:06:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.532610+0000", 2024-09-20T22:06:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.532610+0000", 2024-09-20T22:06:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.532611+0000", 2024-09-20T22:06:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2099999999999995e-07 2024-09-20T22:06:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.532616+0000", 2024-09-20T22:06:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8500000000000002e-06 2024-09-20T22:06:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.532628+0000", 2024-09-20T22:06:39.901 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1331e-05 2024-09-20T22:06:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.532634+0000", 2024-09-20T22:06:39.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6880000000000002e-06 2024-09-20T22:06:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.532705+0000", 2024-09-20T22:06:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0949000000000001e-05 2024-09-20T22:06:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.532713+0000", 2024-09-20T22:06:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.542e-06 2024-09-20T22:06:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.424810+0000", 2024-09-20T22:06:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.892097052 2024-09-20T22:06:39.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:06:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.424880+0000", 2024-09-20T22:06:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9961000000000003e-05 2024-09-20T22:06:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:06:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.425039+0000", 2024-09-20T22:06:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000159314 2024-09-20T22:06:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:06:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.427185+0000", 2024-09-20T22:06:39.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021459679999999998 2024-09-20T22:06:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.429908+0000", 2024-09-20T22:06:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002723442 2024-09-20T22:06:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.429955+0000", 2024-09-20T22:06:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6597000000000001e-05 2024-09-20T22:06:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:06:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.429996+0000", 2024-09-20T22:06:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1334000000000001e-05 2024-09-20T22:06:39.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430049+0000", 2024-09-20T22:06:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2238999999999999e-05 2024-09-20T22:06:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198207 7.18 7:18fdfd72:::benchmark_data_smithi060_78800_object65174:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:06:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.534448+0000", 2024-09-20T22:06:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.262940796000001, 2024-09-20T22:06:39.906 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.89581232600000005, 2024-09-20T22:06:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198207 2024-09-20T22:06:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.534448+0000", 2024-09-20T22:06:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.534448+0000", 2024-09-20T22:06:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.534449+0000", 2024-09-20T22:06:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0999999999999999e-07 2024-09-20T22:06:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.534451+0000", 2024-09-20T22:06:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.34e-06 2024-09-20T22:06:39.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.534459+0000", 2024-09-20T22:06:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9519999999999994e-06 2024-09-20T22:06:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.534464+0000", 2024-09-20T22:06:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.293e-06 2024-09-20T22:06:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.534516+0000", 2024-09-20T22:06:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1468999999999998e-05 2024-09-20T22:06:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.534533+0000", 2024-09-20T22:06:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7223000000000001e-05 2024-09-20T22:06:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.425960+0000", 2024-09-20T22:06:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.89142684400000005 2024-09-20T22:06:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:06:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.426045+0000", 2024-09-20T22:06:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4882000000000005e-05 2024-09-20T22:06:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:06:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.426171+0000", 2024-09-20T22:06:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012685600000000001 2024-09-20T22:06:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:06:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.429924+0000", 2024-09-20T22:06:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037527900000000002 2024-09-20T22:06:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430118+0000", 2024-09-20T22:06:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000193649 2024-09-20T22:06:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430169+0000", 2024-09-20T22:06:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1153000000000003e-05 2024-09-20T22:06:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:06:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430221+0000", 2024-09-20T22:06:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1561999999999999e-05 2024-09-20T22:06:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430260+0000", 2024-09-20T22:06:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9802999999999998e-05 2024-09-20T22:06:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198209 7.18 7:1b4b9a8f:::benchmark_data_smithi060_78800_object65176:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:06:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.535076+0000", 2024-09-20T22:06:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.262313112999999, 2024-09-20T22:06:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.89904252500000004, 2024-09-20T22:06:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.913 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198209 2024-09-20T22:06:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.535076+0000", 2024-09-20T22:06:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.535076+0000", 2024-09-20T22:06:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.535077+0000", 2024-09-20T22:06:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9100000000000002e-07 2024-09-20T22:06:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.535081+0000", 2024-09-20T22:06:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8060000000000002e-06 2024-09-20T22:06:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.535096+0000", 2024-09-20T22:06:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4491999999999999e-05 2024-09-20T22:06:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.535103+0000", 2024-09-20T22:06:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8689999999999998e-06 2024-09-20T22:06:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.535153+0000", 2024-09-20T22:06:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0510000000000003e-05 2024-09-20T22:06:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.535161+0000", 2024-09-20T22:06:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3930000000000002e-06 2024-09-20T22:06:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.427218+0000", 2024-09-20T22:06:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.89205771099999998 2024-09-20T22:06:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:06:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.427309+0000", 2024-09-20T22:06:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0476999999999994e-05 2024-09-20T22:06:39.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:06:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.427501+0000", 2024-09-20T22:06:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019246899999999999 2024-09-20T22:06:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:06:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430275+0000", 2024-09-20T22:06:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0027736309999999999 2024-09-20T22:06:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430313+0000", 2024-09-20T22:06:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8383000000000002e-05 2024-09-20T22:06:39.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.434027+0000", 2024-09-20T22:06:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037134170000000001 2024-09-20T22:06:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:06:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.434077+0000", 2024-09-20T22:06:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0408000000000003e-05 2024-09-20T22:06:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.434118+0000", 2024-09-20T22:06:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1068000000000001e-05 2024-09-20T22:06:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198292 7.18 7:1a05a732:::benchmark_data_smithi060_78800_object65259:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:06:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.610019+0000", 2024-09-20T22:06:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.187370190999999, 2024-09-20T22:06:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.82429133399999999, 2024-09-20T22:06:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:39.920 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198292 2024-09-20T22:06:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.610019+0000", 2024-09-20T22:06:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.610019+0000", 2024-09-20T22:06:39.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.610020+0000", 2024-09-20T22:06:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3549999999999999e-06 2024-09-20T22:06:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.610026+0000", 2024-09-20T22:06:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.614e-06 2024-09-20T22:06:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.610038+0000", 2024-09-20T22:06:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2867999999999999e-05 2024-09-20T22:06:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.610046+0000", 2024-09-20T22:06:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0080000000000002e-06 2024-09-20T22:06:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.610228+0000", 2024-09-20T22:06:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000181249 2024-09-20T22:06:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.610235+0000", 2024-09-20T22:06:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3209999999999999e-06 2024-09-20T22:06:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.428696+0000", 2024-09-20T22:06:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.81846141400000005 2024-09-20T22:06:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:06:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.428815+0000", 2024-09-20T22:06:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011861799999999999 2024-09-20T22:06:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:06:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.428968+0000", 2024-09-20T22:06:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000152621 2024-09-20T22:06:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:06:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430324+0000", 2024-09-20T22:06:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001356714 2024-09-20T22:06:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.434162+0000", 2024-09-20T22:06:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038374590000000001 2024-09-20T22:06:39.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.434219+0000", 2024-09-20T22:06:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7608999999999997e-05 2024-09-20T22:06:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:06:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.434268+0000", 2024-09-20T22:06:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8322000000000003e-05 2024-09-20T22:06:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.434310+0000", 2024-09-20T22:06:39.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2162000000000001e-05 2024-09-20T22:06:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198319 7.18 7:1800bb3a:::benchmark_data_smithi060_78800_object65286:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:06:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.628287+0000", 2024-09-20T22:06:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.169101404999999, 2024-09-20T22:06:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.808638721, 2024-09-20T22:06:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.927 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198319 2024-09-20T22:06:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.628287+0000", 2024-09-20T22:06:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.628287+0000", 2024-09-20T22:06:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.628288+0000", 2024-09-20T22:06:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7800000000000001e-07 2024-09-20T22:06:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.628291+0000", 2024-09-20T22:06:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3270000000000001e-06 2024-09-20T22:06:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.628300+0000", 2024-09-20T22:06:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3999999999999992e-06 2024-09-20T22:06:39.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.628305+0000", 2024-09-20T22:06:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6690000000000003e-06 2024-09-20T22:06:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.628365+0000", 2024-09-20T22:06:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9434000000000001e-05 2024-09-20T22:06:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.628372+0000", 2024-09-20T22:06:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6070000000000002e-06 2024-09-20T22:06:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430359+0000", 2024-09-20T22:06:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.80198674299999995 2024-09-20T22:06:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:06:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430465+0000", 2024-09-20T22:06:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010555100000000001 2024-09-20T22:06:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:06:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430692+0000", 2024-09-20T22:06:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022686700000000001 2024-09-20T22:06:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.434346+0000", 2024-09-20T22:06:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0036543069999999999 2024-09-20T22:06:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.434380+0000", 2024-09-20T22:06:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3639000000000001e-05 2024-09-20T22:06:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:06:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.436714+0000", 2024-09-20T22:06:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023347279999999999 2024-09-20T22:06:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:06:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.436774+0000", 2024-09-20T22:06:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9274000000000002e-05 2024-09-20T22:06:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.436926+0000", 2024-09-20T22:06:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000152597 2024-09-20T22:06:39.933 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198322 7.18 7:180783d3:::benchmark_data_smithi060_78800_object65289:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:06:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.631851+0000", 2024-09-20T22:06:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.165537897, 2024-09-20T22:06:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.80517613899999996, 2024-09-20T22:06:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:06:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:06:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:06:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:06:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:06:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:06:39.934 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198322 2024-09-20T22:06:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:06:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:06:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.631851+0000", 2024-09-20T22:06:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:06:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.631851+0000", 2024-09-20T22:06:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:06:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:06:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.631852+0000", 2024-09-20T22:06:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.167e-06 2024-09-20T22:06:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:06:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.631858+0000", 2024-09-20T22:06:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5439999999999998e-06 2024-09-20T22:06:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:06:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.631869+0000", 2024-09-20T22:06:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1759e-05 2024-09-20T22:06:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:06:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.631877+0000", 2024-09-20T22:06:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4710000000000001e-06 2024-09-20T22:06:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.631935+0000", 2024-09-20T22:06:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8587e-05 2024-09-20T22:06:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:06:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.631943+0000", 2024-09-20T22:06:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9330000000000001e-06 2024-09-20T22:06:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:06:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.432138+0000", 2024-09-20T22:06:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.80019480099999996 2024-09-20T22:06:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:06:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.432229+0000", 2024-09-20T22:06:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1063999999999994e-05 2024-09-20T22:06:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:06:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.432442+0000", 2024-09-20T22:06:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021241700000000001 2024-09-20T22:06:39.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.434412+0000", 2024-09-20T22:06:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019708310000000001 2024-09-20T22:06:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:06:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.436701+0000", 2024-09-20T22:06:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022885330000000001 2024-09-20T22:06:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:06:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.436940+0000", 2024-09-20T22:06:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000238846 2024-09-20T22:06:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:06:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.436989+0000", 2024-09-20T22:06:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9076000000000002e-05 2024-09-20T22:06:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:06:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:06:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:06:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.437027+0000", 2024-09-20T22:06:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8109999999999999e-05 2024-09-20T22:06:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:06:39.940 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:06:39.940 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:06:39.977 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:39.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:39.978 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:40.079 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:40.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:40 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:06:40.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:40 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:06:40.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:40 smithi060 ceph-mon[29694]: osdmap e138: 8 total, 7 up, 8 in 2024-09-20T22:06:40.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:40 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:06:40.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:40 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:06:40.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:40 smithi060 ceph-mon[29694]: 6.15 scrub starts 2024-09-20T22:06:40.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:40 smithi060 ceph-mon[29694]: 6.15 scrub ok 2024-09-20T22:06:40.120 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:40.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:40.122 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:40.222 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:40.227 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:40 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:06:40.227 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:40 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:06:40.227 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:40 smithi067 ceph-mon[33511]: osdmap e138: 8 total, 7 up, 8 in 2024-09-20T22:06:40.227 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:40 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:06:40.227 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:40 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:06:40.228 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:40 smithi067 ceph-mon[33511]: 6.15 scrub starts 2024-09-20T22:06:40.228 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:40 smithi067 ceph-mon[33511]: 6.15 scrub ok 2024-09-20T22:06:40.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:40.262 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:40.262 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:40.363 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:40.399 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:40.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:40.400 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:40.501 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:40.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:40 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:06:40.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:40 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:06:40.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:40 smithi060 ceph-mon[40290]: osdmap e138: 8 total, 7 up, 8 in 2024-09-20T22:06:40.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:40 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:06:40.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:40 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:06:40.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:40 smithi060 ceph-mon[40290]: 6.15 scrub starts 2024-09-20T22:06:40.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:40 smithi060 ceph-mon[40290]: 6.15 scrub ok 2024-09-20T22:06:40.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:40.540 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:40.540 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:40.641 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:40.671 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:40.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:40.673 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:40.773 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:40.814 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:40.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:40.816 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:40.916 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:40.955 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:40.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:40.956 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:40.976 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:06:41.056 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:41.108 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:41.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:41.109 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:41.139 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:40 smithi060 podman[100099]: 2024-09-20 22:06:40.697030401 +0000 UTC m=+0.373046959 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5 2024-09-20T22:06:41.205 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T22:06:41.210 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:41.242 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:41.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:41.243 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:41.343 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:41.400 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:41 smithi067 ceph-mon[33511]: 7.c scrub starts 2024-09-20T22:06:41.401 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:41 smithi067 ceph-mon[33511]: 6.14 scrub starts 2024-09-20T22:06:41.401 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:41 smithi067 ceph-mon[33511]: 6.14 scrub ok 2024-09-20T22:06:41.401 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:41 smithi067 ceph-mon[33511]: pgmap v168: 158 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 19236/167157 objects degraded (11.508%); 21935/167157 objects misplaced (13.122%) 2024-09-20T22:06:41.401 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:41 smithi067 ceph-mon[33511]: osdmap e139: 8 total, 7 up, 8 in 2024-09-20T22:06:41.401 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:41 smithi067 ceph-mon[33511]: 7.c scrub ok 2024-09-20T22:06:41.401 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:41 smithi067 ceph-mon[33511]: 3.f deep-scrub starts 2024-09-20T22:06:41.401 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:41 smithi067 ceph-mon[33511]: 3.f deep-scrub ok 2024-09-20T22:06:41.408 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:41.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:41.410 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:41.472 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:41 smithi060 ceph-mon[29694]: 7.c scrub starts 2024-09-20T22:06:41.472 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:41 smithi060 ceph-mon[29694]: 6.14 scrub starts 2024-09-20T22:06:41.473 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:41 smithi060 ceph-mon[29694]: 6.14 scrub ok 2024-09-20T22:06:41.473 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:41 smithi060 ceph-mon[29694]: pgmap v168: 158 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 19236/167157 objects degraded (11.508%); 21935/167157 objects misplaced (13.122%) 2024-09-20T22:06:41.473 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:41 smithi060 ceph-mon[29694]: osdmap e139: 8 total, 7 up, 8 in 2024-09-20T22:06:41.473 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:41 smithi060 ceph-mon[29694]: 7.c scrub ok 2024-09-20T22:06:41.473 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:41 smithi060 ceph-mon[29694]: 3.f deep-scrub starts 2024-09-20T22:06:41.473 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:41 smithi060 ceph-mon[29694]: 3.f deep-scrub ok 2024-09-20T22:06:41.473 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:41 smithi060 ceph-mon[40290]: 7.c scrub starts 2024-09-20T22:06:41.473 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:41 smithi060 ceph-mon[40290]: 6.14 scrub starts 2024-09-20T22:06:41.474 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:41 smithi060 ceph-mon[40290]: 6.14 scrub ok 2024-09-20T22:06:41.474 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:41 smithi060 ceph-mon[40290]: pgmap v168: 158 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.4 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 19236/167157 objects degraded (11.508%); 21935/167157 objects misplaced (13.122%) 2024-09-20T22:06:41.474 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:41 smithi060 ceph-mon[40290]: osdmap e139: 8 total, 7 up, 8 in 2024-09-20T22:06:41.474 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:41 smithi060 ceph-mon[40290]: 7.c scrub ok 2024-09-20T22:06:41.474 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:41 smithi060 ceph-mon[40290]: 3.f deep-scrub starts 2024-09-20T22:06:41.474 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:41 smithi060 ceph-mon[40290]: 3.f deep-scrub ok 2024-09-20T22:06:41.474 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:41 smithi060 podman[100099]: 2024-09-20 22:06:41.294097494 +0000 UTC m=+0.970114031 container create 411d640160256879a1d4f92a897ada64ea27bc8706e97f85974ded12494f52b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-activate, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-9e2ac70, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-20T22:06:41.511 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:41.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:41.557 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:41.557 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:41.658 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:41.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:41.711 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:41.711 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:41.790 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:41 smithi060 podman[100099]: 2024-09-20 22:06:41.608073208 +0000 UTC m=+1.284089750 container init 411d640160256879a1d4f92a897ada64ea27bc8706e97f85974ded12494f52b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-activate, RELEASE=squid-9e2ac70, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-20T22:06:41.790 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:41 smithi060 podman[100099]: 2024-09-20 22:06:41.616611709 +0000 UTC m=+1.292628248 container start 411d640160256879a1d4f92a897ada64ea27bc8706e97f85974ded12494f52b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-activate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-9e2ac70, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-20T22:06:41.790 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:41 smithi060 podman[100099]: 2024-09-20 22:06:41.714830436 +0000 UTC m=+1.390846976 container attach 411d640160256879a1d4f92a897ada64ea27bc8706e97f85974ded12494f52b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-activate, RELEASE=squid-9e2ac70, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-20T22:06:41.805 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:41.812 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:41.827 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:41.882 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:41.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:41.883 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:41.984 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:42.035 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:42.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:42.036 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:42.137 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:42.165 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:42.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:42.166 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:42.182 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:41 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-activate[100234]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T22:06:42.182 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:41 smithi060 bash[100099]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T22:06:42.182 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:41 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-activate[100234]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T22:06:42.182 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:41 smithi060 bash[100099]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T22:06:42.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:06:41 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:06:42.266 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:42.296 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:42.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:42.297 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:42.302 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:06:42.398 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:42.424 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:42.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:42.425 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:42.480 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:42 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-activate[100234]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-20T22:06:42.481 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:42 smithi060 bash[100099]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-20T22:06:42.481 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:42 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-activate[100234]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-20T22:06:42.481 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:42 smithi060 bash[100099]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-20T22:06:42.481 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:42 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-activate[100234]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-20T22:06:42.481 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:42 smithi060 bash[100099]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-20T22:06:42.481 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:42 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-activate[100234]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-20T22:06:42.481 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:42 smithi060 bash[100099]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-20T22:06:42.481 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:42 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-activate[100234]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-09-20T22:06:42.481 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:42 smithi060 bash[100099]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-09-20T22:06:42.481 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:42 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-activate[100234]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-20T22:06:42.481 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:42 smithi060 bash[100099]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-20T22:06:42.481 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:42 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-activate[100234]: --> ceph-volume raw activate successful for osd ID: 1 2024-09-20T22:06:42.481 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:42 smithi060 bash[100099]: --> ceph-volume raw activate successful for osd ID: 1 2024-09-20T22:06:42.481 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:42 smithi060 conmon[100234]: conmon 411d640160256879a1d4 : Failed to open cgroups file: /sys/fs/cgroup/machine.slice/libpod-411d640160256879a1d4f92a897ada64ea27bc8706e97f85974ded12494f52b9.scope/memory.events 2024-09-20T22:06:42.482 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:42 smithi060 ceph-mon[29694]: 5.d deep-scrub starts 2024-09-20T22:06:42.482 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:42 smithi060 ceph-mon[29694]: 5.d deep-scrub ok 2024-09-20T22:06:42.482 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:42 smithi060 ceph-mon[29694]: 5.1f scrub starts 2024-09-20T22:06:42.482 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:42 smithi060 ceph-mon[29694]: 5.1f scrub ok 2024-09-20T22:06:42.482 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:42 smithi060 ceph-mon[40290]: 5.d deep-scrub starts 2024-09-20T22:06:42.482 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:42 smithi060 ceph-mon[40290]: 5.d deep-scrub ok 2024-09-20T22:06:42.482 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:42 smithi060 ceph-mon[40290]: 5.1f scrub starts 2024-09-20T22:06:42.482 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:42 smithi060 ceph-mon[40290]: 5.1f scrub ok 2024-09-20T22:06:42.526 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:42.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:42.556 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:42.556 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:42.657 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:42.686 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:42.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:42.687 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:42.742 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:42 smithi060 podman[100099]: 2024-09-20 22:06:42.618884526 +0000 UTC m=+2.294901061 container died 411d640160256879a1d4f92a897ada64ea27bc8706e97f85974ded12494f52b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-9e2ac70, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-20T22:06:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:42 smithi067 ceph-mon[33511]: 5.d deep-scrub starts 2024-09-20T22:06:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:42 smithi067 ceph-mon[33511]: 5.d deep-scrub ok 2024-09-20T22:06:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:42 smithi067 ceph-mon[33511]: 5.1f scrub starts 2024-09-20T22:06:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:42 smithi067 ceph-mon[33511]: 5.1f scrub ok 2024-09-20T22:06:42.788 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:42.824 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:42.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:42.825 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:42.926 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:42.979 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:42.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:42.981 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:43.081 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:43.115 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:43.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:43.118 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:43.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:43.254 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:43.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:43.255 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:43.286 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:43 smithi060 podman[100618]: 2024-09-20 22:06:43.261200137 +0000 UTC m=+0.859930614 container remove 411d640160256879a1d4f92a897ada64ea27bc8706e97f85974ded12494f52b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-activate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-9e2ac70) 2024-09-20T22:06:43.356 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:43.391 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:43.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:43.392 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:43.492 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:43.523 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:43.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:43.524 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:43.529 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:06:43.529 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:06:43.529 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:06:43.529 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:06:43.530 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:06:43.530 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:06:43.530 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:06:43.530 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:06:43.530 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:06:43.530 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:06:43.530 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:06:43.530 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:06:43.531 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:06:43.531 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:06:43.531 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:06:43.624 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:43.652 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:43.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:43.653 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:43.754 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:43.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:43 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:06:43.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:43 smithi067 ceph-mon[33511]: pgmap v170: 158 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.01k op/s; 17210/149025 objects degraded (11.548%); 18768/149025 objects misplaced (12.594%); 7.8 MiB/s, 124 objects/s recovering 2024-09-20T22:06:43.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:43 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 17210/149025 objects degraded (11.548%), 35 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:43.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:43 smithi067 ceph-mon[33511]: 5.11 scrub starts 2024-09-20T22:06:43.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:43 smithi067 ceph-mon[33511]: 5.11 scrub ok 2024-09-20T22:06:43.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:43 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2513186063' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:43.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:43 smithi067 ceph-mon[33511]: 4.8 scrub starts 2024-09-20T22:06:43.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:43 smithi067 ceph-mon[33511]: 4.8 scrub ok 2024-09-20T22:06:43.780 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:43.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:43.781 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:43.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:43 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:06:43.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:43 smithi060 ceph-mon[29694]: pgmap v170: 158 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.01k op/s; 17210/149025 objects degraded (11.548%); 18768/149025 objects misplaced (12.594%); 7.8 MiB/s, 124 objects/s recovering 2024-09-20T22:06:43.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:43 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 17210/149025 objects degraded (11.548%), 35 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:43.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:43 smithi060 ceph-mon[29694]: 5.11 scrub starts 2024-09-20T22:06:43.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:43 smithi060 ceph-mon[29694]: 5.11 scrub ok 2024-09-20T22:06:43.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:43 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2513186063' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:43.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:43 smithi060 ceph-mon[29694]: 4.8 scrub starts 2024-09-20T22:06:43.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:43 smithi060 ceph-mon[29694]: 4.8 scrub ok 2024-09-20T22:06:43.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:43 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:06:43.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:43 smithi060 ceph-mon[40290]: pgmap v170: 158 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.01k op/s; 17210/149025 objects degraded (11.548%); 18768/149025 objects misplaced (12.594%); 7.8 MiB/s, 124 objects/s recovering 2024-09-20T22:06:43.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:43 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 17210/149025 objects degraded (11.548%), 35 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:43.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:43 smithi060 ceph-mon[40290]: 5.11 scrub starts 2024-09-20T22:06:43.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:43 smithi060 ceph-mon[40290]: 5.11 scrub ok 2024-09-20T22:06:43.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:43 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2513186063' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:43.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:43 smithi060 ceph-mon[40290]: 4.8 scrub starts 2024-09-20T22:06:43.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:43 smithi060 ceph-mon[40290]: 4.8 scrub ok 2024-09-20T22:06:43.881 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:43.907 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:43.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:43.907 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:44.008 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:44.031 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:44.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:44.031 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:44.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:44.158 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:44.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:44.159 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:44.165 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:06:44.165 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:06:44.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:44.288 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:44.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:44.289 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:44.390 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:44.416 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:44.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:44.417 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:44.518 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:44.553 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:44.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:44.554 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:44.603 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:06:44.603 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:06:44.656 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:44.685 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:44.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:44.686 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:44.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:44 smithi067 ceph-mon[33511]: osdmap e140: 8 total, 7 up, 8 in 2024-09-20T22:06:44.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:44 smithi067 ceph-mon[33511]: pgmap v172: 158 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.01k op/s; 17210/149025 objects degraded (11.548%); 18768/149025 objects misplaced (12.594%); 7.8 MiB/s, 124 objects/s recovering 2024-09-20T22:06:44.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:44 smithi060 ceph-mon[29694]: osdmap e140: 8 total, 7 up, 8 in 2024-09-20T22:06:44.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:44 smithi060 ceph-mon[29694]: pgmap v172: 158 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.01k op/s; 17210/149025 objects degraded (11.548%); 18768/149025 objects misplaced (12.594%); 7.8 MiB/s, 124 objects/s recovering 2024-09-20T22:06:44.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:44 smithi060 ceph-mon[40290]: osdmap e140: 8 total, 7 up, 8 in 2024-09-20T22:06:44.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:44 smithi060 ceph-mon[40290]: pgmap v172: 158 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 84 active+clean; 3.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.01k op/s; 17210/149025 objects degraded (11.548%); 18768/149025 objects misplaced (12.594%); 7.8 MiB/s, 124 objects/s recovering 2024-09-20T22:06:44.786 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:44.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:44.838 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:44.839 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:44.939 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:44.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:44.975 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:44.975 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:45.076 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:45.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:45.105 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:45.105 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:45.206 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:45.229 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:45.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:45.231 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:45.332 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:45.360 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:45.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:45.361 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:45.392 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:06:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:06:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:06:45.461 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:45.492 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:45.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:45.493 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:45.593 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:45.634 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:45.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:45.635 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:45.735 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:45.741 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:45 smithi060 ceph-mon[29694]: 3.5 deep-scrub starts 2024-09-20T22:06:45.741 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:45 smithi060 ceph-mon[29694]: 3.5 deep-scrub ok 2024-09-20T22:06:45.741 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:45 smithi060 ceph-mon[29694]: 7.f deep-scrub starts 2024-09-20T22:06:45.741 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:45 smithi060 ceph-mon[29694]: osdmap e141: 8 total, 7 up, 8 in 2024-09-20T22:06:45.741 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:45 smithi060 ceph-mon[29694]: 7.16 scrub starts 2024-09-20T22:06:45.741 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:45 smithi060 ceph-mon[29694]: 6.c scrub starts 2024-09-20T22:06:45.741 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:45 smithi060 ceph-mon[29694]: 6.c scrub ok 2024-09-20T22:06:45.741 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:45 smithi060 ceph-mon[40290]: 3.5 deep-scrub starts 2024-09-20T22:06:45.741 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:45 smithi060 ceph-mon[40290]: 3.5 deep-scrub ok 2024-09-20T22:06:45.742 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:45 smithi060 ceph-mon[40290]: 7.f deep-scrub starts 2024-09-20T22:06:45.742 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:45 smithi060 ceph-mon[40290]: osdmap e141: 8 total, 7 up, 8 in 2024-09-20T22:06:45.742 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:45 smithi060 ceph-mon[40290]: 7.16 scrub starts 2024-09-20T22:06:45.742 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:45 smithi060 ceph-mon[40290]: 6.c scrub starts 2024-09-20T22:06:45.742 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:45 smithi060 ceph-mon[40290]: 6.c scrub ok 2024-09-20T22:06:45.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:45 smithi067 ceph-mon[33511]: 3.5 deep-scrub starts 2024-09-20T22:06:45.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:45 smithi067 ceph-mon[33511]: 3.5 deep-scrub ok 2024-09-20T22:06:45.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:45 smithi067 ceph-mon[33511]: 7.f deep-scrub starts 2024-09-20T22:06:45.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:45 smithi067 ceph-mon[33511]: osdmap e141: 8 total, 7 up, 8 in 2024-09-20T22:06:45.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:45 smithi067 ceph-mon[33511]: 7.16 scrub starts 2024-09-20T22:06:45.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:45 smithi067 ceph-mon[33511]: 6.c scrub starts 2024-09-20T22:06:45.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:45 smithi067 ceph-mon[33511]: 6.c scrub ok 2024-09-20T22:06:45.798 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T22:06:45.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:45.799 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:45.799 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:45.900 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:45.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:45.948 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:45.948 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:46.049 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:46.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:46.099 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:46.099 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:46.167 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:06:46.200 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:46.225 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:46.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:46.226 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:46.286 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:46.327 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:46.353 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:46.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:46.355 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:46.455 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:46.507 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:46.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:46.508 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:46.609 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:46.661 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:46.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:46.662 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:46.679 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:46 smithi060 ceph-mon[29694]: 3.1 deep-scrub starts 2024-09-20T22:06:46.679 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:46 smithi060 ceph-mon[29694]: 3.1 deep-scrub ok 2024-09-20T22:06:46.679 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:46 smithi060 ceph-mon[29694]: 7.16 scrub ok 2024-09-20T22:06:46.679 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:46 smithi060 ceph-mon[29694]: 5.10 scrub starts 2024-09-20T22:06:46.679 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:46 smithi060 ceph-mon[29694]: 5.10 scrub ok 2024-09-20T22:06:46.679 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:46 smithi060 ceph-mon[29694]: pgmap v174: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 2.7 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 703 op/s; 15472/133818 objects degraded (11.562%); 16448/133818 objects misplaced (12.291%) 2024-09-20T22:06:46.680 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:46 smithi060 ceph-mon[40290]: 3.1 deep-scrub starts 2024-09-20T22:06:46.680 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:46 smithi060 ceph-mon[40290]: 3.1 deep-scrub ok 2024-09-20T22:06:46.680 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:46 smithi060 ceph-mon[40290]: 7.16 scrub ok 2024-09-20T22:06:46.680 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:46 smithi060 ceph-mon[40290]: 5.10 scrub starts 2024-09-20T22:06:46.680 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:46 smithi060 ceph-mon[40290]: 5.10 scrub ok 2024-09-20T22:06:46.680 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:46 smithi060 ceph-mon[40290]: pgmap v174: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 2.7 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 703 op/s; 15472/133818 objects degraded (11.562%); 16448/133818 objects misplaced (12.291%) 2024-09-20T22:06:46.680 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:46 smithi060 podman[101025]: 2024-09-20 22:06:46.291393153 +0000 UTC m=+0.367148752 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5 2024-09-20T22:06:46.680 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:46 smithi060 podman[101025]: 2024-09-20 22:06:46.4252224 +0000 UTC m=+0.500977984 container create ec8a7318fda765e76c05c0474fc38573c84b1da012032b49870e2acca31a4a29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=squid-9e2ac70) 2024-09-20T22:06:46.763 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:46.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:46 smithi067 ceph-mon[33511]: 3.1 deep-scrub starts 2024-09-20T22:06:46.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:46 smithi067 ceph-mon[33511]: 3.1 deep-scrub ok 2024-09-20T22:06:46.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:46 smithi067 ceph-mon[33511]: 7.16 scrub ok 2024-09-20T22:06:46.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:46 smithi067 ceph-mon[33511]: 5.10 scrub starts 2024-09-20T22:06:46.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:46 smithi067 ceph-mon[33511]: 5.10 scrub ok 2024-09-20T22:06:46.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:46 smithi067 ceph-mon[33511]: pgmap v174: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 2.7 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 703 op/s; 15472/133818 objects degraded (11.562%); 16448/133818 objects misplaced (12.291%) 2024-09-20T22:06:46.791 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:46.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:46.792 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:46.893 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:46.918 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:46.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:46.919 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:47.020 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:47.055 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:47.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:47.056 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:47.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:47.207 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:47.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:47.208 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:47.309 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:47.335 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:47.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:47.336 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:47.437 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:47.464 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:47.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:47.465 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:47.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:47 smithi060 ceph-mon[29694]: 3.12 scrub starts 2024-09-20T22:06:47.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:47 smithi060 ceph-mon[29694]: 3.12 scrub ok 2024-09-20T22:06:47.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:47 smithi060 ceph-mon[40290]: 3.12 scrub starts 2024-09-20T22:06:47.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:47 smithi060 ceph-mon[40290]: 3.12 scrub ok 2024-09-20T22:06:47.566 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:47.593 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:47.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:47.595 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:47.695 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:47.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:47.733 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:47.733 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:47.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:47 smithi067 ceph-mon[33511]: 3.12 scrub starts 2024-09-20T22:06:47.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:47 smithi067 ceph-mon[33511]: 3.12 scrub ok 2024-09-20T22:06:47.835 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:47.911 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:47.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:47.912 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:48.012 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:48.066 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:48.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:48.067 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:48.168 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:48.195 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:48.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:48.196 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:48.200 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T22:06:48.292 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:48.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:48.307 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:06:48.320 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:48.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:48.321 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:48.410 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:06:48.412 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:48 smithi060 podman[101025]: 2024-09-20 22:06:48.072390893 +0000 UTC m=+2.148146491 container init ec8a7318fda765e76c05c0474fc38573c84b1da012032b49870e2acca31a4a29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-9e2ac70, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-20T22:06:48.412 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:48 smithi060 podman[101025]: 2024-09-20 22:06:48.087442273 +0000 UTC m=+2.163197864 container start ec8a7318fda765e76c05c0474fc38573c84b1da012032b49870e2acca31a4a29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=squid-9e2ac70, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-20T22:06:48.412 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-osd[101453]: -- 172.21.15.60:0/3873690561 <== mon.0 v2:172.21.15.60:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5634c162fd40 con 0x5634c17a6400 2024-09-20T22:06:48.412 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-osd[101453]: -- 172.21.15.60:0/3873690561 <== mon.0 v2:172.21.15.60:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5634c15efa40 con 0x5634c17a6400 2024-09-20T22:06:48.412 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:48 smithi060 bash[101025]: ec8a7318fda765e76c05c0474fc38573c84b1da012032b49870e2acca31a4a29 2024-09-20T22:06:48.412 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:48 smithi060 systemd[1]: Started Ceph osd.1 for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T22:06:48.421 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:48.447 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:48.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:48.448 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:48.549 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:48.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:48.610 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:48.611 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:48.711 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:48.742 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:48.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:48.743 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:48.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:48 smithi067 ceph-mon[33511]: 3.1d scrub starts 2024-09-20T22:06:48.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:48 smithi067 ceph-mon[33511]: 3.1d scrub ok 2024-09-20T22:06:48.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:48 smithi067 ceph-mon[33511]: 7.13 deep-scrub starts 2024-09-20T22:06:48.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:48 smithi067 ceph-mon[33511]: 2.1 scrub starts 2024-09-20T22:06:48.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:48 smithi067 ceph-mon[33511]: 2.1 scrub ok 2024-09-20T22:06:48.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:48 smithi067 ceph-mon[33511]: 6.1e deep-scrub starts 2024-09-20T22:06:48.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:48 smithi067 ceph-mon[33511]: 6.1e deep-scrub ok 2024-09-20T22:06:48.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:48 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1818286361' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:06:48.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:48 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:48.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:48 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:48.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:48 smithi067 ceph-mon[33511]: pgmap v175: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 2.7 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 538 op/s; 15472/133818 objects degraded (11.562%); 16448/133818 objects misplaced (12.291%) 2024-09-20T22:06:48.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:48 smithi067 ceph-mon[33511]: 3.2 scrub starts 2024-09-20T22:06:48.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:48 smithi067 ceph-mon[33511]: 3.2 scrub ok 2024-09-20T22:06:48.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[29694]: 3.1d scrub starts 2024-09-20T22:06:48.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[29694]: 3.1d scrub ok 2024-09-20T22:06:48.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[29694]: 7.13 deep-scrub starts 2024-09-20T22:06:48.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[29694]: 2.1 scrub starts 2024-09-20T22:06:48.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[29694]: 2.1 scrub ok 2024-09-20T22:06:48.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[29694]: 6.1e deep-scrub starts 2024-09-20T22:06:48.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[29694]: 6.1e deep-scrub ok 2024-09-20T22:06:48.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1818286361' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:06:48.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:48.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:48.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[29694]: pgmap v175: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 2.7 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 538 op/s; 15472/133818 objects degraded (11.562%); 16448/133818 objects misplaced (12.291%) 2024-09-20T22:06:48.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[29694]: 3.2 scrub starts 2024-09-20T22:06:48.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[29694]: 3.2 scrub ok 2024-09-20T22:06:48.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[40290]: 3.1d scrub starts 2024-09-20T22:06:48.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[40290]: 3.1d scrub ok 2024-09-20T22:06:48.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[40290]: 7.13 deep-scrub starts 2024-09-20T22:06:48.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[40290]: 2.1 scrub starts 2024-09-20T22:06:48.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[40290]: 2.1 scrub ok 2024-09-20T22:06:48.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[40290]: 6.1e deep-scrub starts 2024-09-20T22:06:48.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[40290]: 6.1e deep-scrub ok 2024-09-20T22:06:48.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1818286361' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:06:48.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:48.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:48.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[40290]: pgmap v175: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 2.7 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 538 op/s; 15472/133818 objects degraded (11.562%); 16448/133818 objects misplaced (12.291%) 2024-09-20T22:06:48.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[40290]: 3.2 scrub starts 2024-09-20T22:06:48.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:48 smithi060 ceph-mon[40290]: 3.2 scrub ok 2024-09-20T22:06:48.844 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:48.892 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:48.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:48.893 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:48.994 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:49.031 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:49.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:49.032 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:49.133 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:49.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:49.175 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:49.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:49.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:49.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:49.321 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:49.321 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:49.422 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:49.459 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:49.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:49.460 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:49.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[29694]: 6.7 scrub starts 2024-09-20T22:06:49.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[29694]: 6.7 scrub ok 2024-09-20T22:06:49.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[29694]: 3.1f deep-scrub starts 2024-09-20T22:06:49.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[29694]: 3.1f deep-scrub ok 2024-09-20T22:06:49.480 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:06:49.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1818286361' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:06:49.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[29694]: osdmap e142: 8 total, 7 up, 8 in 2024-09-20T22:06:49.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:06:49.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:06:49.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:49.481 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1868504307' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:49.481 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1[101165]: 2024-09-20T22:06:49.258+0000 7f15a9397640 -1 Falling back to public interface 2024-09-20T22:06:49.481 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[40290]: 6.7 scrub starts 2024-09-20T22:06:49.481 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[40290]: 6.7 scrub ok 2024-09-20T22:06:49.482 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[40290]: 3.1f deep-scrub starts 2024-09-20T22:06:49.482 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[40290]: 3.1f deep-scrub ok 2024-09-20T22:06:49.482 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:06:49.482 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1818286361' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:06:49.482 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[40290]: osdmap e142: 8 total, 7 up, 8 in 2024-09-20T22:06:49.482 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:06:49.482 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:06:49.482 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:49.482 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:49 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1868504307' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:49.561 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:49.586 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:49.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:49.587 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:49.688 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:49.735 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:49.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:49.736 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:49.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:49 smithi067 ceph-mon[33511]: 6.7 scrub starts 2024-09-20T22:06:49.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:49 smithi067 ceph-mon[33511]: 6.7 scrub ok 2024-09-20T22:06:49.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:49 smithi067 ceph-mon[33511]: 3.1f deep-scrub starts 2024-09-20T22:06:49.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:49 smithi067 ceph-mon[33511]: 3.1f deep-scrub ok 2024-09-20T22:06:49.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:49 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:06:49.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:49 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1818286361' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:06:49.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:49 smithi067 ceph-mon[33511]: osdmap e142: 8 total, 7 up, 8 in 2024-09-20T22:06:49.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:49 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:06:49.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:49 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:06:49.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:49 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:06:49.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:49 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1868504307' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:06:49.837 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:49.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:49.883 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:49.883 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:49.984 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:50.006 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:50.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:50.007 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:50.107 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:50.136 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:50.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:50.137 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:50.237 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:50.264 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:50.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:50.265 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:50.365 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:50.390 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:50.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:50.391 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:50.492 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:50.537 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:50.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:50.538 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:50.638 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:50.661 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:50.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:50.662 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:50.763 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:50.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:50 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:06:50.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:50 smithi067 ceph-mon[33511]: pgmap v177: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 2.7 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 776 B/s rd, 0 op/s; 15472/133818 objects degraded (11.562%); 16448/133818 objects misplaced (12.291%) 2024-09-20T22:06:50.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:50 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:06:50.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:50 smithi060 ceph-mon[29694]: pgmap v177: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 2.7 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 776 B/s rd, 0 op/s; 15472/133818 objects degraded (11.562%); 16448/133818 objects misplaced (12.291%) 2024-09-20T22:06:50.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:50 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:06:50.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:50 smithi060 ceph-mon[40290]: pgmap v177: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 2.7 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 776 B/s rd, 0 op/s; 15472/133818 objects degraded (11.562%); 16448/133818 objects misplaced (12.291%) 2024-09-20T22:06:50.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:50.787 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:50.787 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:50.888 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:50.916 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:50.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:50.917 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:51.018 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:51.044 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:51.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:51.045 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:51.146 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:51.177 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:51.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:51.178 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:51.279 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:51.316 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:51.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:51.317 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:51.418 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:51.440 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:51.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:51.441 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:51.542 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:51.576 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:51.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:51.577 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:51.606 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:06:51.606 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:06:51.606 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:06:51.606 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:06:51.606 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:06:51.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:51.712 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:51.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:51.713 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:51.750 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:06:51.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:51 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 15472/133818 objects degraded (11.562%), 35 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:51.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:51 smithi067 ceph-mon[33511]: osdmap e143: 8 total, 7 up, 8 in 2024-09-20T22:06:51.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:51 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 15472/133818 objects degraded (11.562%), 35 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:51.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:51 smithi060 ceph-mon[29694]: osdmap e143: 8 total, 7 up, 8 in 2024-09-20T22:06:51.779 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:51 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 15472/133818 objects degraded (11.562%), 35 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:51.779 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:51 smithi060 ceph-mon[40290]: osdmap e143: 8 total, 7 up, 8 in 2024-09-20T22:06:51.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:51.878 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:51.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:51.879 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:51.980 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:52.003 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:52.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:52.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:52.105 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:52.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:52.149 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:52.149 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:52.250 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:52.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:06:51 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:06:52.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:52.293 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:52.293 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:52.394 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:52.433 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:52.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:52.434 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:52.535 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:52.559 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:52.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:52.560 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:52.661 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:52.683 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:52.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:52.684 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:52 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:06:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:52 smithi067 ceph-mon[33511]: pgmap v179: 158 pgs: 2 active+clean+scrubbing+deep, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 2.5 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 296 op/s; 14034/121605 objects degraded (11.541%); 14855/121605 objects misplaced (12.216%) 2024-09-20T22:06:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:52 smithi067 ceph-mon[33511]: from='osd.1 [v2:172.21.15.60:6810/2970874921,v1:172.21.15.60:6811/2970874921]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-20T22:06:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:52 smithi067 ceph-mon[33511]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-20T22:06:52.785 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:52 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:06:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:52 smithi060 ceph-mon[29694]: pgmap v179: 158 pgs: 2 active+clean+scrubbing+deep, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 2.5 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 296 op/s; 14034/121605 objects degraded (11.541%); 14855/121605 objects misplaced (12.216%) 2024-09-20T22:06:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:52 smithi060 ceph-mon[29694]: from='osd.1 [v2:172.21.15.60:6810/2970874921,v1:172.21.15.60:6811/2970874921]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-20T22:06:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:52 smithi060 ceph-mon[29694]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-20T22:06:52.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:52 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:06:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:52 smithi060 ceph-mon[40290]: pgmap v179: 158 pgs: 2 active+clean+scrubbing+deep, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 2.5 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 296 op/s; 14034/121605 objects degraded (11.541%); 14855/121605 objects misplaced (12.216%) 2024-09-20T22:06:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:52 smithi060 ceph-mon[40290]: from='osd.1 [v2:172.21.15.60:6810/2970874921,v1:172.21.15.60:6811/2970874921]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-20T22:06:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:52 smithi060 ceph-mon[40290]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-20T22:06:52.787 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:52 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1[101165]: 2024-09-20T22:06:52.401+0000 7f15a9397640 -1 osd.1 90 log_to_monitors true 2024-09-20T22:06:52.812 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:52.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:52.813 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:52.914 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:52.966 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:52.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:52.967 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:53.068 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:53.097 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:53.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:53.098 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:53.199 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:53.221 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:53.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:53.222 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:53.286 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:06:52 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1[101165]: 2024-09-20T22:06:52.968+0000 7f159d13a640 -1 osd.1 144 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T22:06:53.322 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:53.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:53.349 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:53.349 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:53.450 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:53.481 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:53.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:53.481 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:53.582 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:53.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:53.627 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:53.628 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:53.665 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:53 smithi060 ceph-mon[29694]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-20T22:06:53.665 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:53 smithi060 ceph-mon[29694]: from='osd.1 [v2:172.21.15.60:6810/2970874921,v1:172.21.15.60:6811/2970874921]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-09-20T22:06:53.666 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:53 smithi060 ceph-mon[29694]: osdmap e144: 8 total, 7 up, 8 in 2024-09-20T22:06:53.666 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:53 smithi060 ceph-mon[29694]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-09-20T22:06:53.666 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:53 smithi060 ceph-mon[29694]: from='osd.1 ' entity='osd.1' 2024-09-20T22:06:53.666 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:53 smithi060 ceph-mon[40290]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-20T22:06:53.666 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:53 smithi060 ceph-mon[40290]: from='osd.1 [v2:172.21.15.60:6810/2970874921,v1:172.21.15.60:6811/2970874921]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-09-20T22:06:53.666 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:53 smithi060 ceph-mon[40290]: osdmap e144: 8 total, 7 up, 8 in 2024-09-20T22:06:53.666 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:53 smithi060 ceph-mon[40290]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-09-20T22:06:53.666 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:53 smithi060 ceph-mon[40290]: from='osd.1 ' entity='osd.1' 2024-09-20T22:06:53.729 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:53.753 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:53.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:53.754 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:53 smithi067 ceph-mon[33511]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-20T22:06:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:53 smithi067 ceph-mon[33511]: from='osd.1 [v2:172.21.15.60:6810/2970874921,v1:172.21.15.60:6811/2970874921]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-09-20T22:06:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:53 smithi067 ceph-mon[33511]: osdmap e144: 8 total, 7 up, 8 in 2024-09-20T22:06:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:53 smithi067 ceph-mon[33511]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi060", "root=default"]}]: dispatch 2024-09-20T22:06:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:53 smithi067 ceph-mon[33511]: from='osd.1 ' entity='osd.1' 2024-09-20T22:06:53.855 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:53.877 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:53.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:53.879 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:53.979 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:54.006 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:54.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:54.007 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:54.108 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:54.129 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:54.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:54.130 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:54.231 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:54.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:54.254 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:54.254 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:54.355 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:54.377 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:54.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:54.378 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:54.479 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:54.501 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:54.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:54.502 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:54.602 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:54.625 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:54.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:54.626 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:54.726 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:54.748 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:54.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:54.749 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:54 smithi067 ceph-mon[33511]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-20T22:06:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:54 smithi067 ceph-mon[33511]: osd.1 [v2:172.21.15.60:6810/2970874921,v1:172.21.15.60:6811/2970874921] boot 2024-09-20T22:06:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:54 smithi067 ceph-mon[33511]: osdmap e145: 8 total, 8 up, 8 in 2024-09-20T22:06:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:54 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:06:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:54 smithi067 ceph-mon[33511]: pgmap v182: 158 pgs: 2 active+clean+scrubbing+deep, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 2.5 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 919 B/s rd, 0 B/s wr, 731 op/s; 14034/121605 objects degraded (11.541%); 14855/121605 objects misplaced (12.216%) 2024-09-20T22:06:54.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:54 smithi060 ceph-mon[29694]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-20T22:06:54.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:54 smithi060 ceph-mon[29694]: osd.1 [v2:172.21.15.60:6810/2970874921,v1:172.21.15.60:6811/2970874921] boot 2024-09-20T22:06:54.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:54 smithi060 ceph-mon[29694]: osdmap e145: 8 total, 8 up, 8 in 2024-09-20T22:06:54.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:54 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:06:54.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:54 smithi060 ceph-mon[29694]: pgmap v182: 158 pgs: 2 active+clean+scrubbing+deep, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 2.5 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 919 B/s rd, 0 B/s wr, 731 op/s; 14034/121605 objects degraded (11.541%); 14855/121605 objects misplaced (12.216%) 2024-09-20T22:06:54.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:54 smithi060 ceph-mon[40290]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-20T22:06:54.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:54 smithi060 ceph-mon[40290]: osd.1 [v2:172.21.15.60:6810/2970874921,v1:172.21.15.60:6811/2970874921] boot 2024-09-20T22:06:54.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:54 smithi060 ceph-mon[40290]: osdmap e145: 8 total, 8 up, 8 in 2024-09-20T22:06:54.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:54 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-20T22:06:54.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:54 smithi060 ceph-mon[40290]: pgmap v182: 158 pgs: 2 active+clean+scrubbing+deep, 2 active+undersized+degraded+remapped+backfill_toofull, 10 active+clean+remapped, 1 active+forced_recovery+undersized+degraded+remapped+backfill_toofull, 26 active+undersized, 2 active+remapped+backfill_toofull, 32 active+undersized+degraded, 83 active+clean; 2.5 GiB data, 8.8 GiB used, 706 GiB / 715 GiB avail; 919 B/s rd, 0 B/s wr, 731 op/s; 14034/121605 objects degraded (11.541%); 14855/121605 objects misplaced (12.216%) 2024-09-20T22:06:54.850 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:54.872 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:54.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:54.873 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:54.974 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:54.999 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:55.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:55.000 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:55.101 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:55.125 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:55.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:55.125 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:55.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:55.249 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:55.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:55.250 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:55.319 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:06:55.319 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:06:55.319 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:06:55.319 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:06:55.319 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:06:55.319 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:06:55.319 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:06:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:06:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.1 2024-09-20T22:06:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:06:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:06:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:06:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:06:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:06:55.320 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:06:55.320 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:06:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:06:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:06:55.351 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:55.376 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:55.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:55.376 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:55.477 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:55.499 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:55.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:55.500 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:55.601 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:55.623 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:55.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:55.624 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:55.725 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:55.747 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:55.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:55.748 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:55 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:06:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:55 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:06:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:55 smithi067 ceph-mon[33511]: 7.13 deep-scrub ok 2024-09-20T22:06:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:55 smithi067 ceph-mon[33511]: 7.10 starting backfill to osd.1 from (0'0,90'2478] MAX to 144'12522 2024-09-20T22:06:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:55 smithi067 ceph-mon[33511]: osdmap e146: 8 total, 8 up, 8 in 2024-09-20T22:06:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:55 smithi067 ceph-mon[33511]: 7.3 starting backfill to osd.1 from (0'0,0'0] MAX to 144'12407 2024-09-20T22:06:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:55 smithi067 ceph-mon[33511]: 7.15 starting backfill to osd.1 from (0'0,0'0] MAX to 144'12455 2024-09-20T22:06:55.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:55 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:06:55.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:55 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:06:55.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:55 smithi060 ceph-mon[29694]: 7.13 deep-scrub ok 2024-09-20T22:06:55.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:55 smithi060 ceph-mon[29694]: 7.10 starting backfill to osd.1 from (0'0,90'2478] MAX to 144'12522 2024-09-20T22:06:55.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:55 smithi060 ceph-mon[29694]: osdmap e146: 8 total, 8 up, 8 in 2024-09-20T22:06:55.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:55 smithi060 ceph-mon[29694]: 7.3 starting backfill to osd.1 from (0'0,0'0] MAX to 144'12407 2024-09-20T22:06:55.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:55 smithi060 ceph-mon[29694]: 7.15 starting backfill to osd.1 from (0'0,0'0] MAX to 144'12455 2024-09-20T22:06:55.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:55 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:06:55.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:55 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:06:55.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:55 smithi060 ceph-mon[40290]: 7.13 deep-scrub ok 2024-09-20T22:06:55.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:55 smithi060 ceph-mon[40290]: 7.10 starting backfill to osd.1 from (0'0,90'2478] MAX to 144'12522 2024-09-20T22:06:55.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:55 smithi060 ceph-mon[40290]: osdmap e146: 8 total, 8 up, 8 in 2024-09-20T22:06:55.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:55 smithi060 ceph-mon[40290]: 7.3 starting backfill to osd.1 from (0'0,0'0] MAX to 144'12407 2024-09-20T22:06:55.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:55 smithi060 ceph-mon[40290]: 7.15 starting backfill to osd.1 from (0'0,0'0] MAX to 144'12455 2024-09-20T22:06:55.849 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:55.885 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:55.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:55.886 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:55.986 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:56.008 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:56.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:56.009 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:56.110 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:56.140 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:56.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:56.141 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:56.242 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:56.284 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:56.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:56.285 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:56.386 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:56.408 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:56.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:56.409 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:56.509 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:56.562 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:56.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:56.563 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:56.664 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:56.690 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:56.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:56.691 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:56.792 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:56.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:56.826 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:56.826 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:56.926 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:56.973 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:56.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:56.974 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:57.074 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:57.099 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:57.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:57.100 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:57.200 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:57.233 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:57.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:57.234 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:57.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:56 smithi067 ceph-mon[33511]: 7.19 continuing backfill to osd.2 from (102'5250,145'12396] MIN to 145'12396 2024-09-20T22:06:57.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:56 smithi067 ceph-mon[33511]: 7.19 continuing backfill to osd.6 from (102'5250,145'12396] MIN to 145'12396 2024-09-20T22:06:57.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:56 smithi067 ceph-mon[33511]: 7.19 continuing backfill to osd.7 from (102'5250,145'12396] MIN to 145'12396 2024-09-20T22:06:57.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:56 smithi067 ceph-mon[33511]: 7.4 starting backfill to osd.1 from (0'0,90'2497] MAX to 144'12335 2024-09-20T22:06:57.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:56 smithi067 ceph-mon[33511]: 7.9 starting backfill to osd.1 from (0'0,90'2419] MAX to 144'12350 2024-09-20T22:06:57.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:56 smithi067 ceph-mon[33511]: 7.b starting backfill to osd.1 from (0'0,90'2491] MAX to 144'14141 2024-09-20T22:06:57.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:56 smithi067 ceph-mon[33511]: 7.d starting backfill to osd.1 from (0'0,90'2517] MAX to 144'14011 2024-09-20T22:06:57.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:56 smithi067 ceph-mon[33511]: osdmap e147: 8 total, 8 up, 8 in 2024-09-20T22:06:57.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:56 smithi067 ceph-mon[33511]: 7.0 starting backfill to osd.1 from (0'0,90'2478] MAX to 144'12491 2024-09-20T22:06:57.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:56 smithi067 ceph-mon[33511]: 7.2 starting backfill to osd.1 from (0'0,90'2520] MAX to 144'12389 2024-09-20T22:06:57.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:56 smithi067 ceph-mon[33511]: pgmap v185: 158 pgs: 3 peering, 4 remapped+peering, 2 active+clean+scrubbing+deep, 2 active+undersized+degraded+remapped+backfill_toofull, 5 active+clean+remapped, 16 active+undersized, 4 active+remapped+backfill_toofull, 17 active+undersized+degraded, 105 active+clean; 2.4 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 286 op/s; 6350/118167 objects degraded (5.374%); 12124/118167 objects misplaced (10.260%) 2024-09-20T22:06:57.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:56 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 14034/121605 objects degraded (11.541%), 35 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:57.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[29694]: 7.19 continuing backfill to osd.2 from (102'5250,145'12396] MIN to 145'12396 2024-09-20T22:06:57.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[29694]: 7.19 continuing backfill to osd.6 from (102'5250,145'12396] MIN to 145'12396 2024-09-20T22:06:57.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[29694]: 7.19 continuing backfill to osd.7 from (102'5250,145'12396] MIN to 145'12396 2024-09-20T22:06:57.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[29694]: 7.4 starting backfill to osd.1 from (0'0,90'2497] MAX to 144'12335 2024-09-20T22:06:57.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[29694]: 7.9 starting backfill to osd.1 from (0'0,90'2419] MAX to 144'12350 2024-09-20T22:06:57.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[29694]: 7.b starting backfill to osd.1 from (0'0,90'2491] MAX to 144'14141 2024-09-20T22:06:57.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[29694]: 7.d starting backfill to osd.1 from (0'0,90'2517] MAX to 144'14011 2024-09-20T22:06:57.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[29694]: osdmap e147: 8 total, 8 up, 8 in 2024-09-20T22:06:57.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[29694]: 7.0 starting backfill to osd.1 from (0'0,90'2478] MAX to 144'12491 2024-09-20T22:06:57.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[29694]: 7.2 starting backfill to osd.1 from (0'0,90'2520] MAX to 144'12389 2024-09-20T22:06:57.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[29694]: pgmap v185: 158 pgs: 3 peering, 4 remapped+peering, 2 active+clean+scrubbing+deep, 2 active+undersized+degraded+remapped+backfill_toofull, 5 active+clean+remapped, 16 active+undersized, 4 active+remapped+backfill_toofull, 17 active+undersized+degraded, 105 active+clean; 2.4 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 286 op/s; 6350/118167 objects degraded (5.374%); 12124/118167 objects misplaced (10.260%) 2024-09-20T22:06:57.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 14034/121605 objects degraded (11.541%), 35 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:57.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[40290]: 7.19 continuing backfill to osd.2 from (102'5250,145'12396] MIN to 145'12396 2024-09-20T22:06:57.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[40290]: 7.19 continuing backfill to osd.6 from (102'5250,145'12396] MIN to 145'12396 2024-09-20T22:06:57.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[40290]: 7.19 continuing backfill to osd.7 from (102'5250,145'12396] MIN to 145'12396 2024-09-20T22:06:57.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[40290]: 7.4 starting backfill to osd.1 from (0'0,90'2497] MAX to 144'12335 2024-09-20T22:06:57.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[40290]: 7.9 starting backfill to osd.1 from (0'0,90'2419] MAX to 144'12350 2024-09-20T22:06:57.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[40290]: 7.b starting backfill to osd.1 from (0'0,90'2491] MAX to 144'14141 2024-09-20T22:06:57.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[40290]: 7.d starting backfill to osd.1 from (0'0,90'2517] MAX to 144'14011 2024-09-20T22:06:57.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[40290]: osdmap e147: 8 total, 8 up, 8 in 2024-09-20T22:06:57.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[40290]: 7.0 starting backfill to osd.1 from (0'0,90'2478] MAX to 144'12491 2024-09-20T22:06:57.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[40290]: 7.2 starting backfill to osd.1 from (0'0,90'2520] MAX to 144'12389 2024-09-20T22:06:57.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[40290]: pgmap v185: 158 pgs: 3 peering, 4 remapped+peering, 2 active+clean+scrubbing+deep, 2 active+undersized+degraded+remapped+backfill_toofull, 5 active+clean+remapped, 16 active+undersized, 4 active+remapped+backfill_toofull, 17 active+undersized+degraded, 105 active+clean; 2.4 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 511 B/s rd, 0 B/s wr, 286 op/s; 6350/118167 objects degraded (5.374%); 12124/118167 objects misplaced (10.260%) 2024-09-20T22:06:57.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:56 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 14034/121605 objects degraded (11.541%), 35 pgs degraded, 54 pgs undersized (PG_DEGRADED) 2024-09-20T22:06:57.335 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:57.366 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:57.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:57.367 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:57.467 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:06:57.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:57.490 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:57.490 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:06:57.591 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:57.630 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:57.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:57.631 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:57.732 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:57.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:57.764 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:57.765 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:57.865 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:57.895 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:57.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:57.896 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:57.996 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:58.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:58.029 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:58.029 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:58.130 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:58.170 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:58.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:58.171 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:58.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:57 smithi067 ceph-mon[33511]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:06:58.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:57 smithi067 ceph-mon[33511]: 7.f deep-scrub ok 2024-09-20T22:06:58.271 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:58.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:57 smithi060 ceph-mon[29694]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:06:58.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:57 smithi060 ceph-mon[29694]: 7.f deep-scrub ok 2024-09-20T22:06:58.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:57 smithi060 ceph-mon[40290]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:06:58.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:57 smithi060 ceph-mon[40290]: 7.f deep-scrub ok 2024-09-20T22:06:58.303 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:58.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:58.304 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:58.405 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:58.437 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:58.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:58.438 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:06:58.539 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:58.579 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:58.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:58.580 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:58.681 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:58.706 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:58.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:58.707 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:58.808 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:58.843 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:58.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:58.844 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:58.945 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:58.969 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:58.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:58.970 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:59.071 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:06:59.107 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:59.108 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:59.108 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:06:59.209 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:06:59.244 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:59.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:59.245 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:06:59.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:06:58 smithi067 ceph-mon[33511]: pgmap v186: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 3 peering, 2 active+undersized+degraded+remapped+backfill_wait, 4 remapped+peering, 1 active+clean+scrubbing+deep, 2 active+undersized+degraded+remapped+backfill_toofull, 5 active+remapped+backfill_toofull, 140 active+clean; 2.3 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 370 B/s rd, 0 op/s; 6060/115518 objects degraded (5.246%); 11654/115518 objects misplaced (10.088%) 2024-09-20T22:06:59.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:06:58 smithi060 ceph-mon[29694]: pgmap v186: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 3 peering, 2 active+undersized+degraded+remapped+backfill_wait, 4 remapped+peering, 1 active+clean+scrubbing+deep, 2 active+undersized+degraded+remapped+backfill_toofull, 5 active+remapped+backfill_toofull, 140 active+clean; 2.3 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 370 B/s rd, 0 op/s; 6060/115518 objects degraded (5.246%); 11654/115518 objects misplaced (10.088%) 2024-09-20T22:06:59.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:06:58 smithi060 ceph-mon[40290]: pgmap v186: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 3 peering, 2 active+undersized+degraded+remapped+backfill_wait, 4 remapped+peering, 1 active+clean+scrubbing+deep, 2 active+undersized+degraded+remapped+backfill_toofull, 5 active+remapped+backfill_toofull, 140 active+clean; 2.3 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 370 B/s rd, 0 op/s; 6060/115518 objects degraded (5.246%); 11654/115518 objects misplaced (10.088%) 2024-09-20T22:06:59.346 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:06:59.394 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:59.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:59.395 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:06:59.495 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:59.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:59.535 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:59.535 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:59.636 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:06:59.685 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:59.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:59.686 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:06:59.786 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:06:59.828 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:06:59.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:59.829 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:06:59.930 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:06:59.964 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:06:59.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:06:59.965 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:00.066 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:00.093 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:00.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:00.094 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:00.195 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:00.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:00.229 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:00.229 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:00.330 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:00.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:00.364 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:00.364 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:00.465 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:00.497 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:00.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:00.499 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:00.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:00.636 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:00.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:00.637 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:00.738 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:00.759 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:00.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:00.760 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:00.861 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:00.885 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:00.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:00.886 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:00.987 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:01.009 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:01.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:01.009 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:01.110 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:01.144 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:01.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:01.146 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:01.246 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:01.289 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:01.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:01.290 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:01.391 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:01.422 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:01.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:01.423 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:01.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:01 smithi067 ceph-mon[33511]: pgmap v187: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 3 peering, 2 active+undersized+degraded+remapped+backfill_wait, 4 remapped+peering, 1 active+clean+scrubbing+deep, 2 active+undersized+degraded+remapped+backfill_toofull, 5 active+remapped+backfill_toofull, 140 active+clean; 2.3 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 330 B/s rd, 0 op/s; 6060/115518 objects degraded (5.246%); 11654/115518 objects misplaced (10.088%) 2024-09-20T22:07:01.524 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:01.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:01 smithi060 ceph-mon[29694]: pgmap v187: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 3 peering, 2 active+undersized+degraded+remapped+backfill_wait, 4 remapped+peering, 1 active+clean+scrubbing+deep, 2 active+undersized+degraded+remapped+backfill_toofull, 5 active+remapped+backfill_toofull, 140 active+clean; 2.3 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 330 B/s rd, 0 op/s; 6060/115518 objects degraded (5.246%); 11654/115518 objects misplaced (10.088%) 2024-09-20T22:07:01.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:01 smithi060 ceph-mon[40290]: pgmap v187: 158 pgs: 1 active+undersized+degraded+remapped+backfilling, 3 peering, 2 active+undersized+degraded+remapped+backfill_wait, 4 remapped+peering, 1 active+clean+scrubbing+deep, 2 active+undersized+degraded+remapped+backfill_toofull, 5 active+remapped+backfill_toofull, 140 active+clean; 2.3 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 330 B/s rd, 0 op/s; 6060/115518 objects degraded (5.246%); 11654/115518 objects misplaced (10.088%) 2024-09-20T22:07:01.560 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:01.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:01.561 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:01.662 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:01.695 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:01.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:01.696 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:01.769 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:07:01.770 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:07:01.796 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:01.827 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:01.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:01.828 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:01.929 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:01.969 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:01.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:01.970 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:02.071 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:02.110 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:02.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:02.111 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:02.212 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:02.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:02.240 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:02.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:02 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 6060/115518 objects degraded (5.246%), 5 pgs degraded (PG_DEGRADED) 2024-09-20T22:07:02.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:07:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:07:02.275 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T22:07:02.311 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:02.323 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:02.335 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:02 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 6060/115518 objects degraded (5.246%), 5 pgs degraded (PG_DEGRADED) 2024-09-20T22:07:02.337 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:02 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 6060/115518 objects degraded (5.246%), 5 pgs degraded (PG_DEGRADED) 2024-09-20T22:07:02.341 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:02.369 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:02.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:02.370 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:02.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:02.535 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:02.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:02.536 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:02.637 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:02.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:02.716 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:02.717 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:02.817 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:02.861 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:02.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:02.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:02.963 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:02.996 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:02.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:02.997 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:03.016 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:07:03.016 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:07:03.016 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-20T22:07:03.016 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:07:03.098 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:03.132 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:03.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:03.133 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:03.234 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:03.282 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:03.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:03.283 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:03.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:03 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:07:03.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:03 smithi060 ceph-mon[29694]: pgmap v188: 158 pgs: 4 active+undersized+degraded+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_wait, 3 active+undersized+degraded+remapped+backfill_toofull, 5 active+remapped+backfill_toofull, 144 active+clean; 2.1 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 895 B/s rd, 0 op/s; 9919/103977 objects degraded (9.540%); 11418/103977 objects misplaced (10.981%) 2024-09-20T22:07:03.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:03 smithi060 ceph-mon[29694]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 8 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:07:03.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:03 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:07:03.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:03 smithi060 ceph-mon[40290]: pgmap v188: 158 pgs: 4 active+undersized+degraded+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_wait, 3 active+undersized+degraded+remapped+backfill_toofull, 5 active+remapped+backfill_toofull, 144 active+clean; 2.1 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 895 B/s rd, 0 op/s; 9919/103977 objects degraded (9.540%); 11418/103977 objects misplaced (10.981%) 2024-09-20T22:07:03.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:03 smithi060 ceph-mon[40290]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 8 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:07:03.384 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:03.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:03.415 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:03.415 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:03.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:03 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:07:03.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:03 smithi067 ceph-mon[33511]: pgmap v188: 158 pgs: 4 active+undersized+degraded+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_wait, 3 active+undersized+degraded+remapped+backfill_toofull, 5 active+remapped+backfill_toofull, 144 active+clean; 2.1 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 895 B/s rd, 0 op/s; 9919/103977 objects degraded (9.540%); 11418/103977 objects misplaced (10.981%) 2024-09-20T22:07:03.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:03 smithi067 ceph-mon[33511]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 8 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:07:03.516 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:03.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:03.544 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:03.544 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:03.645 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:03.702 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:03.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:03.703 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:03.770 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:07:03.804 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:03.867 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:03.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:03.868 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:03.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:04.025 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:04.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:04.027 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:04.046 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 0 1 2024-09-20T22:07:04.071 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:07:04.127 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:04.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:04.217 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:04.217 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:04.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:04 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3570995223' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:04.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:04 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:04.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:04 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3570995223' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:04.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:04 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:04.317 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:04.345 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:04.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:04.346 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:04.447 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:04.475 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:04.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:04.476 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:04.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:04 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3570995223' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:04.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:04 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:04.577 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:04.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:04.631 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:04.631 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:04.732 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:04.760 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:04.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:04.761 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:04.824 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:04.862 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:04.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:04.939 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:04.939 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:05.040 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:05.068 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:05.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:05.069 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:05.170 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:05.201 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:05.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:05.202 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:05.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:05.317 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:05 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:07:05.317 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:05 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:07:05.318 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:05 smithi060 ceph-mon[29694]: pgmap v189: 158 pgs: 4 active+undersized+degraded+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_wait, 3 active+undersized+degraded+remapped+backfill_toofull, 5 active+remapped+backfill_toofull, 144 active+clean; 2.1 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 780 B/s rd, 0 op/s; 9919/103977 objects degraded (9.540%); 11418/103977 objects misplaced (10.981%) 2024-09-20T22:07:05.318 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:05 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:07:05.318 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:05 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:07:05.318 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:05 smithi060 ceph-mon[29694]: osdmap e148: 8 total, 8 up, 8 in 2024-09-20T22:07:05.318 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:07:05.318 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:07:05.318 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:05 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:07:05.318 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:05 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:07:05.318 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:05 smithi060 ceph-mon[40290]: pgmap v189: 158 pgs: 4 active+undersized+degraded+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_wait, 3 active+undersized+degraded+remapped+backfill_toofull, 5 active+remapped+backfill_toofull, 144 active+clean; 2.1 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 780 B/s rd, 0 op/s; 9919/103977 objects degraded (9.540%); 11418/103977 objects misplaced (10.981%) 2024-09-20T22:07:05.318 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:05 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:07:05.318 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:05 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:07:05.319 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:05 smithi060 ceph-mon[40290]: osdmap e148: 8 total, 8 up, 8 in 2024-09-20T22:07:05.319 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:07:05.319 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:07:05.319 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:07:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:07:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:07:05.341 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:05.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:05.342 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:05.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:05.474 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:05.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:05.475 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:05.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:05 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:07:05.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:05 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:07:05.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:05 smithi067 ceph-mon[33511]: pgmap v189: 158 pgs: 4 active+undersized+degraded+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_wait, 3 active+undersized+degraded+remapped+backfill_toofull, 5 active+remapped+backfill_toofull, 144 active+clean; 2.1 GiB data, 7.9 GiB used, 707 GiB / 715 GiB avail; 780 B/s rd, 0 op/s; 9919/103977 objects degraded (9.540%); 11418/103977 objects misplaced (10.981%) 2024-09-20T22:07:05.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:05 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:07:05.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:05 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:07:05.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:05 smithi067 ceph-mon[33511]: osdmap e148: 8 total, 8 up, 8 in 2024-09-20T22:07:05.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:07:05.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:07:05.575 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:05.599 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:05.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:05.600 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:05.700 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:05.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:05.736 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:05.736 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:05.836 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:05.872 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:05.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:05.873 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:05.881 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:05.946 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:05.956 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:07:05.956 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:07:05.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:07:05.956 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:07:05.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.956 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47598 7.3 7:c6c29109:::benchmark_data_smithi060_78800_object47597:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:07:05.956 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.238604+0000", 2024-09-20T22:07:05.956 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 165.711609127, 2024-09-20T22:07:05.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0797809089999999, 2024-09-20T22:07:05.956 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:05.956 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:05.956 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:05.956 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:05.957 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:05.957 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:05.957 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47598 2024-09-20T22:07:05.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.957 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:05.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:05.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238604+0000", 2024-09-20T22:07:05.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:05.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:05.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238604+0000", 2024-09-20T22:07:05.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:05.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:05.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238605+0000", 2024-09-20T22:07:05.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3099999999999996e-07 2024-09-20T22:07:05.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:05.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238626+0000", 2024-09-20T22:07:05.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1192999999999999e-05 2024-09-20T22:07:05.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:05.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238637+0000", 2024-09-20T22:07:05.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0516000000000001e-05 2024-09-20T22:07:05.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:05.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238644+0000", 2024-09-20T22:07:05.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3969999999999997e-06 2024-09-20T22:07:05.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:05.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.240757+0000", 2024-09-20T22:07:05.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021131100000000001 2024-09-20T22:07:05.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op must wait for map", 2024-09-20T22:07:05.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.240764+0000", 2024-09-20T22:07:05.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9550000000000003e-06 2024-09-20T22:07:05.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:05.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.246730+0000", 2024-09-20T22:07:05.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0059659020000000004 2024-09-20T22:07:05.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:05.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.246741+0000", 2024-09-20T22:07:05.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0855e-05 2024-09-20T22:07:05.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:05.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.235644+0000", 2024-09-20T22:07:05.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98890363400000003 2024-09-20T22:07:05.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:05.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.235657+0000", 2024-09-20T22:07:05.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3040000000000001e-05 2024-09-20T22:07:05.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:05.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.313762+0000", 2024-09-20T22:07:05.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.078104783999999997 2024-09-20T22:07:05.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:05.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.314529+0000", 2024-09-20T22:07:05.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00076653400000000003 2024-09-20T22:07:05.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:07:05.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.314632+0000", 2024-09-20T22:07:05.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103224 2024-09-20T22:07:05.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:05.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318215+0000", 2024-09-20T22:07:05.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0035831439999999999 2024-09-20T22:07:05.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:05.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318247+0000", 2024-09-20T22:07:05.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1804999999999998e-05 2024-09-20T22:07:05.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:05.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318292+0000", 2024-09-20T22:07:05.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4605999999999999e-05 2024-09-20T22:07:05.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:05.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318329+0000", 2024-09-20T22:07:05.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7175999999999998e-05 2024-09-20T22:07:05.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:05.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318385+0000", 2024-09-20T22:07:05.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6202999999999997e-05 2024-09-20T22:07:05.964 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:05.964 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:05.964 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:05.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.964 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47618 7.3 7:c8aeb189:::benchmark_data_smithi060_78800_object47617:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:07:05.964 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.257444+0000", 2024-09-20T22:07:05.964 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 165.69276914400001, 2024-09-20T22:07:05.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0612655090000001, 2024-09-20T22:07:05.964 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:05.965 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:05.965 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:05.965 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:05.965 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:05.965 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:05.965 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47618 2024-09-20T22:07:05.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.965 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:05.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:05.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257444+0000", 2024-09-20T22:07:05.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:05.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:05.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257444+0000", 2024-09-20T22:07:05.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:05.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:05.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257445+0000", 2024-09-20T22:07:05.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3790000000000001e-06 2024-09-20T22:07:05.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:05.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257468+0000", 2024-09-20T22:07:05.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2078000000000001e-05 2024-09-20T22:07:05.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:05.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257480+0000", 2024-09-20T22:07:05.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.253e-05 2024-09-20T22:07:05.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:05.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257488+0000", 2024-09-20T22:07:05.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4900000000000003e-06 2024-09-20T22:07:05.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:05.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257693+0000", 2024-09-20T22:07:05.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000205071 2024-09-20T22:07:05.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:05.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257702+0000", 2024-09-20T22:07:05.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6099999999999995e-06 2024-09-20T22:07:05.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:05.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236024+0000", 2024-09-20T22:07:05.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97832186300000001 2024-09-20T22:07:05.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:05.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236034+0000", 2024-09-20T22:07:05.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9189999999999994e-06 2024-09-20T22:07:05.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:05.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.316117+0000", 2024-09-20T22:07:05.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.080083484999999996 2024-09-20T22:07:05.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:05.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.316218+0000", 2024-09-20T22:07:05.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100846 2024-09-20T22:07:05.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:07:05.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.316363+0000", 2024-09-20T22:07:05.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014473099999999999 2024-09-20T22:07:05.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:05.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318472+0000", 2024-09-20T22:07:05.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021087039999999999 2024-09-20T22:07:05.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:05.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318532+0000", 2024-09-20T22:07:05.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0418999999999999e-05 2024-09-20T22:07:05.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:05.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318622+0000", 2024-09-20T22:07:05.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0074000000000005e-05 2024-09-20T22:07:05.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:05.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318659+0000", 2024-09-20T22:07:05.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6260999999999997e-05 2024-09-20T22:07:05.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:05.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318710+0000", 2024-09-20T22:07:05.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1048999999999999e-05 2024-09-20T22:07:05.971 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:05.971 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:05.971 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:05.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.971 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47683 7.3 7:cf1d2c6c:::benchmark_data_smithi060_78800_object47682:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:07:05.971 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.290834+0000", 2024-09-20T22:07:05.972 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 165.65937909100001, 2024-09-20T22:07:05.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.029069585, 2024-09-20T22:07:05.972 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:05.972 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:05.972 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:05.972 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:05.972 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:05.972 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:05.972 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47683 2024-09-20T22:07:05.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.972 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:05.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:05.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290834+0000", 2024-09-20T22:07:05.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:05.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:05.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290834+0000", 2024-09-20T22:07:05.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:05.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:05.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290835+0000", 2024-09-20T22:07:05.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2020000000000001e-06 2024-09-20T22:07:05.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:05.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290856+0000", 2024-09-20T22:07:05.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0585999999999999e-05 2024-09-20T22:07:05.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:05.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290868+0000", 2024-09-20T22:07:05.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2160999999999999e-05 2024-09-20T22:07:05.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:05.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290875+0000", 2024-09-20T22:07:05.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3200000000000002e-06 2024-09-20T22:07:05.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:05.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290955+0000", 2024-09-20T22:07:05.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9592999999999995e-05 2024-09-20T22:07:05.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:05.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290966+0000", 2024-09-20T22:07:05.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.119e-05 2024-09-20T22:07:05.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:05.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236104+0000", 2024-09-20T22:07:05.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94513756400000004 2024-09-20T22:07:05.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:05.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236112+0000", 2024-09-20T22:07:05.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2220000000000002e-06 2024-09-20T22:07:05.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:05.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.317361+0000", 2024-09-20T22:07:05.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.081249110999999999 2024-09-20T22:07:05.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:05.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.317444+0000", 2024-09-20T22:07:05.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2936999999999996e-05 2024-09-20T22:07:05.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:07:05.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.317551+0000", 2024-09-20T22:07:05.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010741100000000001 2024-09-20T22:07:05.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:05.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318732+0000", 2024-09-20T22:07:05.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0011802379999999999 2024-09-20T22:07:05.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:05.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319739+0000", 2024-09-20T22:07:05.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001007457 2024-09-20T22:07:05.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:05.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319786+0000", 2024-09-20T22:07:05.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6845999999999998e-05 2024-09-20T22:07:05.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:05.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319820+0000", 2024-09-20T22:07:05.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4103000000000002e-05 2024-09-20T22:07:05.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:05.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319904+0000", 2024-09-20T22:07:05.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3644000000000001e-05 2024-09-20T22:07:05.979 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:05.979 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:05.979 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:05.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.979 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47686 7.3 7:c4eaa487:::benchmark_data_smithi060_78800_object47685:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:07:05.979 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.291328+0000", 2024-09-20T22:07:05.979 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 165.65888551899999, 2024-09-20T22:07:05.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.03577915, 2024-09-20T22:07:05.979 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:05.980 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:05.980 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:05.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:05.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:05.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:05.980 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47686 2024-09-20T22:07:05.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.980 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:05.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:05.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291328+0000", 2024-09-20T22:07:05.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:05.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:05.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291328+0000", 2024-09-20T22:07:05.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:05.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:05.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291328+0000", 2024-09-20T22:07:05.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0699999999999996e-07 2024-09-20T22:07:05.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:05.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291353+0000", 2024-09-20T22:07:05.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4134999999999998e-05 2024-09-20T22:07:05.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:05.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291364+0000", 2024-09-20T22:07:05.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1501999999999999e-05 2024-09-20T22:07:05.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.982 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291371+0000", 2024-09-20T22:07:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1210000000000001e-06 2024-09-20T22:07:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291460+0000", 2024-09-20T22:07:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8869999999999994e-05 2024-09-20T22:07:05.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:05.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291473+0000", 2024-09-20T22:07:05.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2899e-05 2024-09-20T22:07:05.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:05.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236140+0000", 2024-09-20T22:07:05.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94466707500000002 2024-09-20T22:07:05.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:05.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236149+0000", 2024-09-20T22:07:05.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9509999999999995e-06 2024-09-20T22:07:05.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:05.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318750+0000", 2024-09-20T22:07:05.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.082601349000000004 2024-09-20T22:07:05.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:05.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318813+0000", 2024-09-20T22:07:05.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2306999999999998e-05 2024-09-20T22:07:05.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:07:05.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318890+0000", 2024-09-20T22:07:05.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7565999999999996e-05 2024-09-20T22:07:05.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:05.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323067+0000", 2024-09-20T22:07:05.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0041765129999999998 2024-09-20T22:07:05.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:05.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323117+0000", 2024-09-20T22:07:05.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0297000000000003e-05 2024-09-20T22:07:05.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:05.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.326959+0000", 2024-09-20T22:07:05.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003841957 2024-09-20T22:07:05.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:05.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327009+0000", 2024-09-20T22:07:05.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0328999999999999e-05 2024-09-20T22:07:05.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:05.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327107+0000", 2024-09-20T22:07:05.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7472000000000005e-05 2024-09-20T22:07:05.988 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:05.988 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:05.988 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:05.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.988 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47726 7.3 7:c33b26c6:::benchmark_data_smithi060_78800_object47725:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:07:05.988 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.311743+0000", 2024-09-20T22:07:05.988 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 165.63846970899999, 2024-09-20T22:07:05.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0155469880000001, 2024-09-20T22:07:05.988 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:05.988 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:05.988 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:05.989 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:05.989 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:05.989 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:05.989 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47726 2024-09-20T22:07:05.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.989 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:05.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:05.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311743+0000", 2024-09-20T22:07:05.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:05.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:05.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311743+0000", 2024-09-20T22:07:05.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:05.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:05.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311745+0000", 2024-09-20T22:07:05.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.333e-06 2024-09-20T22:07:05.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:05.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311769+0000", 2024-09-20T22:07:05.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3688000000000001e-05 2024-09-20T22:07:05.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:05.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311781+0000", 2024-09-20T22:07:05.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2099999999999999e-05 2024-09-20T22:07:05.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:05.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311788+0000", 2024-09-20T22:07:05.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2269999999999996e-06 2024-09-20T22:07:05.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:05.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311860+0000", 2024-09-20T22:07:05.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1823000000000007e-05 2024-09-20T22:07:05.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:05.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311870+0000", 2024-09-20T22:07:05.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0079e-05 2024-09-20T22:07:05.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:05.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236177+0000", 2024-09-20T22:07:05.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.92430713399999997 2024-09-20T22:07:05.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:05.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236185+0000", 2024-09-20T22:07:05.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.106e-06 2024-09-20T22:07:05.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:05.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319949+0000", 2024-09-20T22:07:05.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083764076000000007 2024-09-20T22:07:05.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:05.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.320088+0000", 2024-09-20T22:07:05.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013926300000000001 2024-09-20T22:07:05.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:07:05.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.320263+0000", 2024-09-20T22:07:05.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000174978 2024-09-20T22:07:05.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:05.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323166+0000", 2024-09-20T22:07:05.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0029027060000000001 2024-09-20T22:07:05.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:05.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323208+0000", 2024-09-20T22:07:05.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2373999999999998e-05 2024-09-20T22:07:05.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:05.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327156+0000", 2024-09-20T22:07:05.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0039471910000000001 2024-09-20T22:07:05.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:05.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327201+0000", 2024-09-20T22:07:05.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5062999999999999e-05 2024-09-20T22:07:05.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:05.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327290+0000", 2024-09-20T22:07:05.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9846999999999995e-05 2024-09-20T22:07:05.995 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:05.996 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:05.996 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:05.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.996 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47771 7.3 7:c7433b13:::benchmark_data_smithi060_78800_object47770:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:07:05.996 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.332925+0000", 2024-09-20T22:07:05.996 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 165.61728866300001, 2024-09-20T22:07:05.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99455883199999995, 2024-09-20T22:07:05.996 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:05.996 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:05.996 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:05.996 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:05.996 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:05.997 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:05.997 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47771 2024-09-20T22:07:05.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.997 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:05.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:05.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332925+0000", 2024-09-20T22:07:05.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:05.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:05.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332925+0000", 2024-09-20T22:07:05.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:05.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:05.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332926+0000", 2024-09-20T22:07:05.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4959999999999999e-06 2024-09-20T22:07:05.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:05.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332954+0000", 2024-09-20T22:07:05.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7495e-05 2024-09-20T22:07:05.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:05.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332967+0000", 2024-09-20T22:07:05.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3005e-05 2024-09-20T22:07:05.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:05.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332974+0000", 2024-09-20T22:07:05.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9719999999999992e-06 2024-09-20T22:07:05.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:05.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:05.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333056+0000", 2024-09-20T22:07:05.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1390999999999994e-05 2024-09-20T22:07:05.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:05.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333067+0000", 2024-09-20T22:07:06.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0896999999999999e-05 2024-09-20T22:07:06.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236223+0000", 2024-09-20T22:07:06.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90315600299999999 2024-09-20T22:07:06.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236243+0000", 2024-09-20T22:07:06.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9928999999999998e-05 2024-09-20T22:07:06.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.321565+0000", 2024-09-20T22:07:06.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085322377000000005 2024-09-20T22:07:06.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:06.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.321707+0000", 2024-09-20T22:07:06.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014187100000000001 2024-09-20T22:07:06.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:07:06.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.321808+0000", 2024-09-20T22:07:06.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100536 2024-09-20T22:07:06.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323255+0000", 2024-09-20T22:07:06.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00144754 2024-09-20T22:07:06.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.326947+0000", 2024-09-20T22:07:06.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0036917360000000001 2024-09-20T22:07:06.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:06.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327341+0000", 2024-09-20T22:07:06.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00039394499999999998 2024-09-20T22:07:06.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:06.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327391+0000", 2024-09-20T22:07:06.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9836999999999997e-05 2024-09-20T22:07:06.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:06.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327483+0000", 2024-09-20T22:07:06.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2801999999999997e-05 2024-09-20T22:07:06.003 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.004 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:06.004 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.004 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47772 7.3 7:c5f9ddaf:::benchmark_data_smithi060_78800_object47771:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:07:06.004 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.333899+0000", 2024-09-20T22:07:06.004 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 165.61631414600001, 2024-09-20T22:07:06.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99376266499999999, 2024-09-20T22:07:06.004 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:06.004 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:06.004 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:06.004 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:06.004 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:06.004 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:06.005 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47772 2024-09-20T22:07:06.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.005 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:06.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:06.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333899+0000", 2024-09-20T22:07:06.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:06.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333899+0000", 2024-09-20T22:07:06.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:06.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333900+0000", 2024-09-20T22:07:06.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0049999999999999e-06 2024-09-20T22:07:06.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:06.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333927+0000", 2024-09-20T22:07:06.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.675e-05 2024-09-20T22:07:06.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:06.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333942+0000", 2024-09-20T22:07:06.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5678000000000001e-05 2024-09-20T22:07:06.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:06.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333950+0000", 2024-09-20T22:07:06.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7060000000000004e-06 2024-09-20T22:07:06.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.334006+0000", 2024-09-20T22:07:06.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5631000000000003e-05 2024-09-20T22:07:06.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.334031+0000", 2024-09-20T22:07:06.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5213999999999999e-05 2024-09-20T22:07:06.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236276+0000", 2024-09-20T22:07:06.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90224500100000005 2024-09-20T22:07:06.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236285+0000", 2024-09-20T22:07:06.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7139999999999999e-06 2024-09-20T22:07:06.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323289+0000", 2024-09-20T22:07:06.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087004215999999995 2024-09-20T22:07:06.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:06.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323404+0000", 2024-09-20T22:07:06.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011502000000000001 2024-09-20T22:07:06.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:07:06.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323545+0000", 2024-09-20T22:07:06.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014127700000000001 2024-09-20T22:07:06.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327330+0000", 2024-09-20T22:07:06.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037843400000000002 2024-09-20T22:07:06.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:06.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327491+0000", 2024-09-20T22:07:06.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016180099999999999 2024-09-20T22:07:06.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327529+0000", 2024-09-20T22:07:06.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8081999999999999e-05 2024-09-20T22:07:06.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:06.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327576+0000", 2024-09-20T22:07:06.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6881000000000001e-05 2024-09-20T22:07:06.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:06.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327662+0000", 2024-09-20T22:07:06.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5348999999999999e-05 2024-09-20T22:07:06.012 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.012 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:06.012 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.013 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47798 7.3 7:cc6426c7:::benchmark_data_smithi060_78800_object47797:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:07:06.013 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.346085+0000", 2024-09-20T22:07:06.013 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 165.60412775200001, 2024-09-20T22:07:06.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98393741899999998, 2024-09-20T22:07:06.013 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:06.013 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:06.013 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:06.013 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:06.013 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:06.013 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:06.013 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47798 2024-09-20T22:07:06.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.013 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:06.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:06.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346085+0000", 2024-09-20T22:07:06.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:06.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346085+0000", 2024-09-20T22:07:06.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:06.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346087+0000", 2024-09-20T22:07:06.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3289999999999999e-06 2024-09-20T22:07:06.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:06.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346109+0000", 2024-09-20T22:07:06.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2359999999999999e-05 2024-09-20T22:07:06.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:06.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346123+0000", 2024-09-20T22:07:06.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.367e-05 2024-09-20T22:07:06.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:06.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346133+0000", 2024-09-20T22:07:06.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0519999999999999e-05 2024-09-20T22:07:06.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346206+0000", 2024-09-20T22:07:06.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3011999999999998e-05 2024-09-20T22:07:06.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346218+0000", 2024-09-20T22:07:06.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2089000000000001e-05 2024-09-20T22:07:06.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236825+0000", 2024-09-20T22:07:06.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.89060705600000001 2024-09-20T22:07:06.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236839+0000", 2024-09-20T22:07:06.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3685e-05 2024-09-20T22:07:06.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.324961+0000", 2024-09-20T22:07:06.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088121685000000005 2024-09-20T22:07:06.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:06.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.325083+0000", 2024-09-20T22:07:06.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012199199999999999 2024-09-20T22:07:06.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:07:06.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.325216+0000", 2024-09-20T22:07:06.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000132862 2024-09-20T22:07:06.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:06.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327496+0000", 2024-09-20T22:07:06.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022805669999999998 2024-09-20T22:07:06.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327698+0000", 2024-09-20T22:07:06.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020205099999999999 2024-09-20T22:07:06.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.329834+0000", 2024-09-20T22:07:06.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021359249999999999 2024-09-20T22:07:06.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:06.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.329894+0000", 2024-09-20T22:07:06.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9929000000000002e-05 2024-09-20T22:07:06.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:06.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.330023+0000", 2024-09-20T22:07:06.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000128687 2024-09-20T22:07:06.020 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.020 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:06.020 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.020 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47809 7.3 7:caa74520:::benchmark_data_smithi060_78800_object47808:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:07:06.020 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.351924+0000", 2024-09-20T22:07:06.020 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 165.59828961400001, 2024-09-20T22:07:06.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.982167177, 2024-09-20T22:07:06.021 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:06.021 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:06.021 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:06.021 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:06.021 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:06.021 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:06.021 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47809 2024-09-20T22:07:06.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.021 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:06.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:06.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.351924+0000", 2024-09-20T22:07:06.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:06.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.351924+0000", 2024-09-20T22:07:06.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:06.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.351925+0000", 2024-09-20T22:07:06.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1680000000000001e-06 2024-09-20T22:07:06.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:06.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352100+0000", 2024-09-20T22:07:06.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017479600000000001 2024-09-20T22:07:06.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:06.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352112+0000", 2024-09-20T22:07:06.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.204e-05 2024-09-20T22:07:06.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:06.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352119+0000", 2024-09-20T22:07:06.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8960000000000003e-06 2024-09-20T22:07:06.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352199+0000", 2024-09-20T22:07:06.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9747999999999996e-05 2024-09-20T22:07:06.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352208+0000", 2024-09-20T22:07:06.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2709999999999998e-06 2024-09-20T22:07:06.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236880+0000", 2024-09-20T22:07:06.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.88467109200000005 2024-09-20T22:07:06.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236889+0000", 2024-09-20T22:07:06.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0580000000000004e-06 2024-09-20T22:07:06.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327738+0000", 2024-09-20T22:07:06.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.090849691999999996 2024-09-20T22:07:06.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:06.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327931+0000", 2024-09-20T22:07:06.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019231900000000001 2024-09-20T22:07:06.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:07:06.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.328060+0000", 2024-09-20T22:07:06.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012906699999999999 2024-09-20T22:07:06.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:06.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.331899+0000", 2024-09-20T22:07:06.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038394620000000001 2024-09-20T22:07:06.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.333872+0000", 2024-09-20T22:07:06.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019725540000000001 2024-09-20T22:07:06.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.333936+0000", 2024-09-20T22:07:06.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3849000000000005e-05 2024-09-20T22:07:06.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:06.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.333992+0000", 2024-09-20T22:07:06.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6730000000000001e-05 2024-09-20T22:07:06.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:06.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334091+0000", 2024-09-20T22:07:06.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8435000000000002e-05 2024-09-20T22:07:06.028 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.028 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:06.028 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.028 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47810 7.3 7:c6434cb9:::benchmark_data_smithi060_78800_object47809:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:07:06.028 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.352140+0000", 2024-09-20T22:07:06.028 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 165.59807361899999, 2024-09-20T22:07:06.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.985672357, 2024-09-20T22:07:06.028 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:06.028 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:06.028 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:06.028 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:06.029 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:06.029 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:06.029 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47810 2024-09-20T22:07:06.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.029 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:06.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:06.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352140+0000", 2024-09-20T22:07:06.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:06.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352140+0000", 2024-09-20T22:07:06.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:06.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352140+0000", 2024-09-20T22:07:06.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9299999999999997e-07 2024-09-20T22:07:06.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:06.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352187+0000", 2024-09-20T22:07:06.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6468000000000003e-05 2024-09-20T22:07:06.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:06.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352196+0000", 2024-09-20T22:07:06.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9670000000000007e-06 2024-09-20T22:07:06.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:06.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352202+0000", 2024-09-20T22:07:06.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0120000000000002e-06 2024-09-20T22:07:06.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352230+0000", 2024-09-20T22:07:06.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8289e-05 2024-09-20T22:07:06.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352236+0000", 2024-09-20T22:07:06.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.455e-06 2024-09-20T22:07:06.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236921+0000", 2024-09-20T22:07:06.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.88468433300000004 2024-09-20T22:07:06.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236931+0000", 2024-09-20T22:07:06.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0322e-05 2024-09-20T22:07:06.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.330063+0000", 2024-09-20T22:07:06.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.093132375000000003 2024-09-20T22:07:06.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:06.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.330188+0000", 2024-09-20T22:07:06.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000124664 2024-09-20T22:07:06.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:07:06.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.330329+0000", 2024-09-20T22:07:06.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014119299999999999 2024-09-20T22:07:06.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:06.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334105+0000", 2024-09-20T22:07:06.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037757799999999998 2024-09-20T22:07:06.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334140+0000", 2024-09-20T22:07:06.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5015000000000003e-05 2024-09-20T22:07:06.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.035 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337652+0000", 2024-09-20T22:07:06.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0035119819999999999 2024-09-20T22:07:06.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.035 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:06.035 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337709+0000", 2024-09-20T22:07:06.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6446000000000001e-05 2024-09-20T22:07:06.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.035 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:06.035 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337812+0000", 2024-09-20T22:07:06.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010336300000000001 2024-09-20T22:07:06.035 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.035 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:06.036 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.036 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47831 7.3 7:cf7c6d80:::benchmark_data_smithi060_78800_object47830:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:07:06.036 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.435232+0000", 2024-09-20T22:07:06.036 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 165.514981027, 2024-09-20T22:07:06.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90280988699999998, 2024-09-20T22:07:06.036 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:06.036 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:06.036 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:06.036 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:06.036 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:06.036 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:06.037 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47831 2024-09-20T22:07:06.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.037 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:06.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:06.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435232+0000", 2024-09-20T22:07:06.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:06.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435232+0000", 2024-09-20T22:07:06.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:06.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435233+0000", 2024-09-20T22:07:06.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0920000000000001e-06 2024-09-20T22:07:06.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:06.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435256+0000", 2024-09-20T22:07:06.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2840000000000002e-05 2024-09-20T22:07:06.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:06.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435270+0000", 2024-09-20T22:07:06.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3944000000000001e-05 2024-09-20T22:07:06.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:06.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435277+0000", 2024-09-20T22:07:06.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1420000000000004e-06 2024-09-20T22:07:06.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435341+0000", 2024-09-20T22:07:06.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3431999999999998e-05 2024-09-20T22:07:06.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435350+0000", 2024-09-20T22:07:06.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2920000000000001e-06 2024-09-20T22:07:06.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236961+0000", 2024-09-20T22:07:06.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.80161062500000002 2024-09-20T22:07:06.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236968+0000", 2024-09-20T22:07:06.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8529999999999992e-06 2024-09-20T22:07:06.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.331930+0000", 2024-09-20T22:07:06.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.094961952000000002 2024-09-20T22:07:06.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:06.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.332051+0000", 2024-09-20T22:07:06.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000120683 2024-09-20T22:07:06.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:07:06.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.332208+0000", 2024-09-20T22:07:06.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015661399999999999 2024-09-20T22:07:06.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334177+0000", 2024-09-20T22:07:06.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019690350000000001 2024-09-20T22:07:06.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:06.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334187+0000", 2024-09-20T22:07:06.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8870000000000004e-06 2024-09-20T22:07:06.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337890+0000", 2024-09-20T22:07:06.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037037379999999998 2024-09-20T22:07:06.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:06.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337940+0000", 2024-09-20T22:07:06.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0144e-05 2024-09-20T22:07:06.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:06.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.338042+0000", 2024-09-20T22:07:06.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010161400000000001 2024-09-20T22:07:06.044 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.044 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:06.044 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.045 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47833 7.3 7:c90762ed:::benchmark_data_smithi060_78800_object47832:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:07:06.045 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.437390+0000", 2024-09-20T22:07:06.045 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 165.512823067, 2024-09-20T22:07:06.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90566071400000003, 2024-09-20T22:07:06.045 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:06.045 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:06.045 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:06.045 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:06.045 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:06.045 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:06.045 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47833 2024-09-20T22:07:06.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.046 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:06.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:06.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437390+0000", 2024-09-20T22:07:06.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:06.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437390+0000", 2024-09-20T22:07:06.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:06.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437391+0000", 2024-09-20T22:07:06.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.085e-06 2024-09-20T22:07:06.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:06.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437415+0000", 2024-09-20T22:07:06.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3479e-05 2024-09-20T22:07:06.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:06.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437430+0000", 2024-09-20T22:07:06.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5485000000000002e-05 2024-09-20T22:07:06.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:06.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437439+0000", 2024-09-20T22:07:06.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3520000000000007e-06 2024-09-20T22:07:06.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437507+0000", 2024-09-20T22:07:06.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8011999999999999e-05 2024-09-20T22:07:06.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437520+0000", 2024-09-20T22:07:06.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3022e-05 2024-09-20T22:07:06.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237002+0000", 2024-09-20T22:07:06.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.79948219600000003 2024-09-20T22:07:06.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237011+0000", 2024-09-20T22:07:06.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8049999999999996e-06 2024-09-20T22:07:06.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334208+0000", 2024-09-20T22:07:06.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.097197622999999997 2024-09-20T22:07:06.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:06.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334306+0000", 2024-09-20T22:07:06.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8068999999999999e-05 2024-09-20T22:07:06.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:07:06.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334475+0000", 2024-09-20T22:07:06.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016830799999999999 2024-09-20T22:07:06.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337852+0000", 2024-09-20T22:07:06.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00337747 2024-09-20T22:07:06.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:06.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.340086+0000", 2024-09-20T22:07:06.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002234036 2024-09-20T22:07:06.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.342602+0000", 2024-09-20T22:07:06.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025159929999999998 2024-09-20T22:07:06.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:06.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.342808+0000", 2024-09-20T22:07:06.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020551099999999999 2024-09-20T22:07:06.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:06.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343051+0000", 2024-09-20T22:07:06.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00024326799999999999 2024-09-20T22:07:06.052 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.052 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:06.052 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.053 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47842 7.3 7:c97c3aed:::benchmark_data_smithi060_78800_object47841:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:07:06.053 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.442613+0000", 2024-09-20T22:07:06.053 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 165.50760045999999, 2024-09-20T22:07:06.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90079891400000001, 2024-09-20T22:07:06.053 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:06.053 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:06.053 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:06.053 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:06.053 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:06.053 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:06.053 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47842 2024-09-20T22:07:06.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.053 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:06.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:06.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442613+0000", 2024-09-20T22:07:06.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:06.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442613+0000", 2024-09-20T22:07:06.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:06.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442614+0000", 2024-09-20T22:07:06.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3e-06 2024-09-20T22:07:06.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:06.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442640+0000", 2024-09-20T22:07:06.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5544000000000001e-05 2024-09-20T22:07:06.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:06.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442651+0000", 2024-09-20T22:07:06.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1218e-05 2024-09-20T22:07:06.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:06.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442658+0000", 2024-09-20T22:07:06.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1980000000000004e-06 2024-09-20T22:07:06.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442734+0000", 2024-09-20T22:07:06.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6279999999999995e-05 2024-09-20T22:07:06.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442747+0000", 2024-09-20T22:07:06.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2393e-05 2024-09-20T22:07:06.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237041+0000", 2024-09-20T22:07:06.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.79429414499999995 2024-09-20T22:07:06.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237049+0000", 2024-09-20T22:07:06.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8150000000000006e-06 2024-09-20T22:07:06.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.335958+0000", 2024-09-20T22:07:06.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.098909056999999995 2024-09-20T22:07:06.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:06.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.336074+0000", 2024-09-20T22:07:06.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011582900000000001 2024-09-20T22:07:06.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:07:06.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.336216+0000", 2024-09-20T22:07:06.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014236899999999999 2024-09-20T22:07:06.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:06.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.342529+0000", 2024-09-20T22:07:06.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.006312799 2024-09-20T22:07:06.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343211+0000", 2024-09-20T22:07:06.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00068264900000000002 2024-09-20T22:07:06.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343252+0000", 2024-09-20T22:07:06.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1084000000000002e-05 2024-09-20T22:07:06.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:06.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343315+0000", 2024-09-20T22:07:06.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2989000000000003e-05 2024-09-20T22:07:06.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:06.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343412+0000", 2024-09-20T22:07:06.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6244999999999998e-05 2024-09-20T22:07:06.060 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.060 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:06.060 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.060 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47846 7.3 7:c61861ce:::benchmark_data_smithi060_78800_object47845:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:07:06.060 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.445377+0000", 2024-09-20T22:07:06.060 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 165.50483659099999, 2024-09-20T22:07:06.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91076886199999996, 2024-09-20T22:07:06.060 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:06.061 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:06.061 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:06.061 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:06.061 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:06.061 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:06.061 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47846 2024-09-20T22:07:06.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.061 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:06.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:06.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445377+0000", 2024-09-20T22:07:06.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:06.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445377+0000", 2024-09-20T22:07:06.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:06.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445378+0000", 2024-09-20T22:07:06.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.02e-06 2024-09-20T22:07:06.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:06.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445407+0000", 2024-09-20T22:07:06.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9654000000000001e-05 2024-09-20T22:07:06.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:06.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445423+0000", 2024-09-20T22:07:06.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6107999999999999e-05 2024-09-20T22:07:06.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:06.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445432+0000", 2024-09-20T22:07:06.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6770000000000001e-06 2024-09-20T22:07:06.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445522+0000", 2024-09-20T22:07:06.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0211999999999996e-05 2024-09-20T22:07:06.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445535+0000", 2024-09-20T22:07:06.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2279000000000001e-05 2024-09-20T22:07:06.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237076+0000", 2024-09-20T22:07:06.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.79154110300000002 2024-09-20T22:07:06.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.065 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.065 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237085+0000", 2024-09-20T22:07:06.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.611e-06 2024-09-20T22:07:06.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.065 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.065 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.338112+0000", 2024-09-20T22:07:06.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10102702700000001 2024-09-20T22:07:06.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.065 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:06.065 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.338234+0000", 2024-09-20T22:07:06.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121853 2024-09-20T22:07:06.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:07:06.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.338375+0000", 2024-09-20T22:07:06.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140428 2024-09-20T22:07:06.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343463+0000", 2024-09-20T22:07:06.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0050883710000000004 2024-09-20T22:07:06.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343504+0000", 2024-09-20T22:07:06.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1185e-05 2024-09-20T22:07:06.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:06.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.355992+0000", 2024-09-20T22:07:06.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.012487992 2024-09-20T22:07:06.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:06.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:06.068 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:06.068 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:06.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356046+0000", 2024-09-20T22:07:06.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4033000000000003e-05 2024-09-20T22:07:06.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:06.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356145+0000", 2024-09-20T22:07:06.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9308999999999994e-05 2024-09-20T22:07:06.069 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.069 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:06.069 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.069 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47847 7.3 7:c0c5cc37:::benchmark_data_smithi060_78800_object47846:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:07:06.069 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.445583+0000", 2024-09-20T22:07:06.069 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 165.50463031199999, 2024-09-20T22:07:06.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91077994900000003, 2024-09-20T22:07:06.070 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:06.070 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:06.070 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:06.070 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:06.070 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:06.070 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:06.070 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47847 2024-09-20T22:07:06.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.070 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:06.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:06.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445583+0000", 2024-09-20T22:07:06.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:06.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445583+0000", 2024-09-20T22:07:06.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:06.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445584+0000", 2024-09-20T22:07:06.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4900000000000005e-07 2024-09-20T22:07:06.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:06.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445607+0000", 2024-09-20T22:07:06.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3153e-05 2024-09-20T22:07:06.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:06.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445621+0000", 2024-09-20T22:07:06.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3638000000000001e-05 2024-09-20T22:07:06.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:06.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445628+0000", 2024-09-20T22:07:06.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.729e-06 2024-09-20T22:07:06.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445676+0000", 2024-09-20T22:07:06.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7911000000000003e-05 2024-09-20T22:07:06.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445685+0000", 2024-09-20T22:07:06.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8289999999999997e-06 2024-09-20T22:07:06.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237113+0000", 2024-09-20T22:07:06.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.79142811599999996 2024-09-20T22:07:06.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237121+0000", 2024-09-20T22:07:06.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8310000000000001e-06 2024-09-20T22:07:06.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.340131+0000", 2024-09-20T22:07:06.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103009795 2024-09-20T22:07:06.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:06.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.340324+0000", 2024-09-20T22:07:06.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000193034 2024-09-20T22:07:06.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:07:06.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.340466+0000", 2024-09-20T22:07:06.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000142032 2024-09-20T22:07:06.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343542+0000", 2024-09-20T22:07:06.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0030759860000000002 2024-09-20T22:07:06.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343584+0000", 2024-09-20T22:07:06.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1990999999999998e-05 2024-09-20T22:07:06.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:06.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356203+0000", 2024-09-20T22:07:06.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.012619361000000001 2024-09-20T22:07:06.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:06.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356268+0000", 2024-09-20T22:07:06.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4596000000000003e-05 2024-09-20T22:07:06.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:06.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356363+0000", 2024-09-20T22:07:06.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5098000000000003e-05 2024-09-20T22:07:06.077 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.077 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:06.077 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.077 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47853 7.3 7:c21485b4:::benchmark_data_smithi060_78800_object47852:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:07:06.077 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.457698+0000", 2024-09-20T22:07:06.077 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 165.49251501500001, 2024-09-20T22:07:06.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.89889923500000002, 2024-09-20T22:07:06.077 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:06.077 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:06.077 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:06.078 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:06.078 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:06.078 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:06.078 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47853 2024-09-20T22:07:06.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.078 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:06.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:06.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457698+0000", 2024-09-20T22:07:06.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:06.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457698+0000", 2024-09-20T22:07:06.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:06.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457699+0000", 2024-09-20T22:07:06.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.159e-06 2024-09-20T22:07:06.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:06.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457753+0000", 2024-09-20T22:07:06.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4067999999999998e-05 2024-09-20T22:07:06.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:06.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457765+0000", 2024-09-20T22:07:06.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1430000000000001e-05 2024-09-20T22:07:06.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:06.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457770+0000", 2024-09-20T22:07:06.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4410000000000001e-06 2024-09-20T22:07:06.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457821+0000", 2024-09-20T22:07:06.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1156999999999998e-05 2024-09-20T22:07:06.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457829+0000", 2024-09-20T22:07:06.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5259999999999996e-06 2024-09-20T22:07:06.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237712+0000", 2024-09-20T22:07:06.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.77988319699999997 2024-09-20T22:07:06.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237742+0000", 2024-09-20T22:07:06.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0224000000000001e-05 2024-09-20T22:07:06.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343621+0000", 2024-09-20T22:07:06.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10587846400000001 2024-09-20T22:07:06.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:06.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343759+0000", 2024-09-20T22:07:06.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000137947 2024-09-20T22:07:06.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:07:06.083 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343960+0000", 2024-09-20T22:07:06.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000201201 2024-09-20T22:07:06.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.083 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.349430+0000", 2024-09-20T22:07:06.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0054698710000000003 2024-09-20T22:07:06.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.083 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.349473+0000", 2024-09-20T22:07:06.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2865999999999999e-05 2024-09-20T22:07:06.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:06.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356374+0000", 2024-09-20T22:07:06.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0069012359999999998 2024-09-20T22:07:06.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:06.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356434+0000", 2024-09-20T22:07:06.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9669000000000001e-05 2024-09-20T22:07:06.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:06.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356597+0000", 2024-09-20T22:07:06.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016377899999999999 2024-09-20T22:07:06.084 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.084 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:06.084 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.085 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169951 7.3 7:c27f31c3:::benchmark_data_smithi060_78800_object36918:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:07:06.085 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440033+0000", 2024-09-20T22:07:06.085 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 67.510179824000005, 2024-09-20T22:07:06.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0308894289999999, 2024-09-20T22:07:06.085 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:06.085 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:06.085 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:07:06.085 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:06.085 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:06.085 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:06.085 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169951 2024-09-20T22:07:06.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.085 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:06.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:06.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440033+0000", 2024-09-20T22:07:06.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:06.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440033+0000", 2024-09-20T22:07:06.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:06.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440035+0000", 2024-09-20T22:07:06.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3170000000000001e-06 2024-09-20T22:07:06.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:06.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440041+0000", 2024-09-20T22:07:06.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8869999999999997e-06 2024-09-20T22:07:06.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:06.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440053+0000", 2024-09-20T22:07:06.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2146e-05 2024-09-20T22:07:06.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:06.088 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440061+0000", 2024-09-20T22:07:06.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3049999999999993e-06 2024-09-20T22:07:06.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.088 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.088 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440616+0000", 2024-09-20T22:07:06.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00055526700000000002 2024-09-20T22:07:06.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.088 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op must wait for map", 2024-09-20T22:07:06.088 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440623+0000", 2024-09-20T22:07:06.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9290000000000001e-06 2024-09-20T22:07:06.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.089 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.089 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.659494+0000", 2024-09-20T22:07:06.089 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.21887063800000001 2024-09-20T22:07:06.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.089 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.089 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.089 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.659506+0000", 2024-09-20T22:07:06.089 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2313000000000001e-05 2024-09-20T22:07:06.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.089 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.089 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.089 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.470588+0000", 2024-09-20T22:07:06.089 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.81108138900000004 2024-09-20T22:07:06.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:06.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.470923+0000", 2024-09-20T22:07:06.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033523800000000002 2024-09-20T22:07:06.090 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.090 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:06.090 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.090 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169962 7.8 7:1412370b:::benchmark_data_smithi060_78800_object36929:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:07:06.090 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440138+0000", 2024-09-20T22:07:06.090 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 67.510074794000005, 2024-09-20T22:07:06.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0713576439999999, 2024-09-20T22:07:06.090 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:06.091 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:06.091 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:07:06.091 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:06.091 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:06.091 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:06.091 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169962 2024-09-20T22:07:06.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.091 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:06.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:06.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440138+0000", 2024-09-20T22:07:06.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:06.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440138+0000", 2024-09-20T22:07:06.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:06.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440139+0000", 2024-09-20T22:07:06.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1399999999999996e-07 2024-09-20T22:07:06.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:06.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440143+0000", 2024-09-20T22:07:06.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8029999999999998e-06 2024-09-20T22:07:06.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.093 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:06.093 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440152+0000", 2024-09-20T22:07:06.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-20T22:07:06.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.093 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:06.093 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440158+0000", 2024-09-20T22:07:06.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3319999999999997e-06 2024-09-20T22:07:06.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.093 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.093 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440263+0000", 2024-09-20T22:07:06.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105258 2024-09-20T22:07:06.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op must wait for map", 2024-09-20T22:07:06.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440269+0000", 2024-09-20T22:07:06.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8549999999999998e-06 2024-09-20T22:07:06.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.647667+0000", 2024-09-20T22:07:06.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.207398358 2024-09-20T22:07:06.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.647675+0000", 2024-09-20T22:07:06.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5979999999999999e-06 2024-09-20T22:07:06.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.490134+0000", 2024-09-20T22:07:06.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.84245882699999997 2024-09-20T22:07:06.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:07:06.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.490253+0000", 2024-09-20T22:07:06.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011938999999999999 2024-09-20T22:07:06.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.511425+0000", 2024-09-20T22:07:06.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021171453999999999 2024-09-20T22:07:06.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:06.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.511496+0000", 2024-09-20T22:07:06.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1359000000000006e-05 2024-09-20T22:07:06.096 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.096 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:06.097 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.097 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.097 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169969 7.18 7:18661b50:::benchmark_data_smithi060_78800_object36936:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:07:06.097 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440181+0000", 2024-09-20T22:07:06.097 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 67.510032375999998, 2024-09-20T22:07:06.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.074892379, 2024-09-20T22:07:06.097 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:06.097 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:06.097 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:07:06.097 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:06.097 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:06.097 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:06.097 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169969 2024-09-20T22:07:06.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.098 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:06.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.098 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:06.098 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440181+0000", 2024-09-20T22:07:06.098 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.098 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:06.098 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440181+0000", 2024-09-20T22:07:06.098 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.098 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:06.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440181+0000", 2024-09-20T22:07:06.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7100000000000002e-07 2024-09-20T22:07:06.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.099 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:06.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440185+0000", 2024-09-20T22:07:06.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3840000000000001e-06 2024-09-20T22:07:06.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.099 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:06.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440193+0000", 2024-09-20T22:07:06.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5790000000000004e-06 2024-09-20T22:07:06.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:06.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440200+0000", 2024-09-20T22:07:06.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2310000000000001e-06 2024-09-20T22:07:06.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.647597+0000", 2024-09-20T22:07:06.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.20739776200000001 2024-09-20T22:07:06.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.647606+0000", 2024-09-20T22:07:06.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6580000000000008e-06 2024-09-20T22:07:06.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.510928+0000", 2024-09-20T22:07:06.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.86332191400000002 2024-09-20T22:07:06.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:07:06.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.511051+0000", 2024-09-20T22:07:06.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012358099999999999 2024-09-20T22:07:06.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.514989+0000", 2024-09-20T22:07:06.102 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003937744 2024-09-20T22:07:06.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.102 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:06.102 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.515073+0000", 2024-09-20T22:07:06.102 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3955000000000005e-05 2024-09-20T22:07:06.102 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.102 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:06.102 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.102 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227626 7.3 7:c62c9d8d:::benchmark_data_smithi060_78800_object94593:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:07:06.102 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.821701+0000", 2024-09-20T22:07:06.103 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 12.128512567, 2024-09-20T22:07:06.103 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1262373590000001, 2024-09-20T22:07:06.103 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:06.103 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:06.103 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:06.103 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:06.103 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:06.103 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:06.103 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227626 2024-09-20T22:07:06.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.103 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:06.103 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.103 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:06.103 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821701+0000", 2024-09-20T22:07:06.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.104 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:06.104 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821701+0000", 2024-09-20T22:07:06.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:06.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.104 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:06.104 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821702+0000", 2024-09-20T22:07:06.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.102e-06 2024-09-20T22:07:06.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.104 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:06.104 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821707+0000", 2024-09-20T22:07:06.105 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0579999999999997e-06 2024-09-20T22:07:06.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.105 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:06.105 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821733+0000", 2024-09-20T22:07:06.105 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6642999999999999e-05 2024-09-20T22:07:06.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.105 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:06.105 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821741+0000", 2024-09-20T22:07:06.105 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7009999999999996e-06 2024-09-20T22:07:06.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.105 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821806+0000", 2024-09-20T22:07:06.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4882000000000007e-05 2024-09-20T22:07:06.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:06.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821817+0000", 2024-09-20T22:07:06.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0852999999999999e-05 2024-09-20T22:07:06.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:06.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.941998+0000", 2024-09-20T22:07:06.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1201808070000001 2024-09-20T22:07:06.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.107 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:06.107 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.942120+0000", 2024-09-20T22:07:06.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012227900000000001 2024-09-20T22:07:06.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.107 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.107 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,3,4", 2024-09-20T22:07:06.107 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.942615+0000", 2024-09-20T22:07:06.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00049518700000000001 2024-09-20T22:07:06.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.107 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.107 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:06.107 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946370+0000", 2024-09-20T22:07:06.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003755141 2024-09-20T22:07:06.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.108 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.108 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.108 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946870+0000", 2024-09-20T22:07:06.108 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00049939899999999998 2024-09-20T22:07:06.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.108 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.108 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.108 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947042+0000", 2024-09-20T22:07:06.108 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000172199 2024-09-20T22:07:06.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.108 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.108 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:06.108 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947739+0000", 2024-09-20T22:07:06.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00069714899999999999 2024-09-20T22:07:06.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:06.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947809+0000", 2024-09-20T22:07:06.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9530999999999995e-05 2024-09-20T22:07:06.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:06.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:06.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:06.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947938+0000", 2024-09-20T22:07:06.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012942800000000001 2024-09-20T22:07:06.109 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.109 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:06.109 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.110 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:06.110 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:06.110 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:07:06.168 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:06.217 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:06.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:06.218 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:06.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:06 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:07:06.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:06 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:07:06.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:06 smithi060 ceph-mon[29694]: osdmap e149: 8 total, 8 up, 8 in 2024-09-20T22:07:06.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:06 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:07:06.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:06 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:07:06.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:06 smithi060 ceph-mon[40290]: osdmap e149: 8 total, 8 up, 8 in 2024-09-20T22:07:06.319 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:06.356 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:06.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:06.357 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:06.458 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:06.489 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:06.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:06.490 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:06.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:06 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:07:06.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:06 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:07:06.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:06 smithi067 ceph-mon[33511]: osdmap e149: 8 total, 8 up, 8 in 2024-09-20T22:07:06.591 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:06.623 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:06.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:06.624 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:06.725 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:06.768 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:06.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:06.769 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:06.870 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:06.878 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:07:06.899 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:06.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:06.900 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:07.001 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:07.035 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:07.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:07.036 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:07.137 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:07.169 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:07.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:07.170 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:07.271 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:07.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:07 smithi060 ceph-mon[29694]: pgmap v192: 158 pgs: 1 peering, 4 active+undersized+degraded+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_wait, 2 active+undersized+degraded+remapped+backfill_toofull, 5 active+remapped+backfill_toofull, 144 active+clean; 2.0 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 755 op/s; 7785/97404 objects degraded (7.992%); 9772/97404 objects misplaced (10.032%); 29 MiB/s, 460 objects/s recovering 2024-09-20T22:07:07.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:07 smithi060 ceph-mon[29694]: osdmap e150: 8 total, 8 up, 8 in 2024-09-20T22:07:07.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:07 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 7785/97404 objects degraded (7.992%), 8 pgs degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-20T22:07:07.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:07 smithi060 ceph-mon[29694]: osdmap e151: 8 total, 8 up, 8 in 2024-09-20T22:07:07.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:07 smithi060 ceph-mon[40290]: pgmap v192: 158 pgs: 1 peering, 4 active+undersized+degraded+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_wait, 2 active+undersized+degraded+remapped+backfill_toofull, 5 active+remapped+backfill_toofull, 144 active+clean; 2.0 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 755 op/s; 7785/97404 objects degraded (7.992%); 9772/97404 objects misplaced (10.032%); 29 MiB/s, 460 objects/s recovering 2024-09-20T22:07:07.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:07 smithi060 ceph-mon[40290]: osdmap e150: 8 total, 8 up, 8 in 2024-09-20T22:07:07.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:07 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 7785/97404 objects degraded (7.992%), 8 pgs degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-20T22:07:07.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:07 smithi060 ceph-mon[40290]: osdmap e151: 8 total, 8 up, 8 in 2024-09-20T22:07:07.319 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:07.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:07.320 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:07.420 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:07.450 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:07.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:07.451 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:07.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:07 smithi067 ceph-mon[33511]: pgmap v192: 158 pgs: 1 peering, 4 active+undersized+degraded+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_wait, 2 active+undersized+degraded+remapped+backfill_toofull, 5 active+remapped+backfill_toofull, 144 active+clean; 2.0 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 755 op/s; 7785/97404 objects degraded (7.992%); 9772/97404 objects misplaced (10.032%); 29 MiB/s, 460 objects/s recovering 2024-09-20T22:07:07.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:07 smithi067 ceph-mon[33511]: osdmap e150: 8 total, 8 up, 8 in 2024-09-20T22:07:07.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:07 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 7785/97404 objects degraded (7.992%), 8 pgs degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-20T22:07:07.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:07 smithi067 ceph-mon[33511]: osdmap e151: 8 total, 8 up, 8 in 2024-09-20T22:07:07.551 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:07.578 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:07.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:07.579 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:07.680 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:07.706 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:07.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:07.707 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:07.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:07.833 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:07.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:07.834 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:07.935 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:07.968 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:07.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:07.969 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:08.070 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:08.094 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:08.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:08.095 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:08.196 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:08.222 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:08.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:08.223 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:08.324 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:08.355 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:08.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:08.356 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:08.457 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:08.495 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:08.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:08.496 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:08.596 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:08.626 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:08.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:08.627 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:08.728 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:08 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:07:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:08 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:07:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:08 smithi067 ceph-mon[33511]: osdmap e152: 8 total, 8 up, 8 in 2024-09-20T22:07:08.775 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:08.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:08.776 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:08 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:07:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:08 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:07:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:08 smithi060 ceph-mon[29694]: osdmap e152: 8 total, 8 up, 8 in 2024-09-20T22:07:08.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:08 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:07:08.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:08 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:07:08.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:08 smithi060 ceph-mon[40290]: osdmap e152: 8 total, 8 up, 8 in 2024-09-20T22:07:08.877 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:08.937 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:08.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:08.938 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:09.039 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:09.068 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:09.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:09.069 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:09.170 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:09.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:09.204 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:09.205 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:09.306 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:09.344 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:09.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:09.345 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:09.446 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:09.474 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:09.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:09.475 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:09.576 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:09.608 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:09.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:09.609 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:09.664 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:09 smithi067 ceph-mon[33511]: pgmap v196: 158 pgs: 1 peering, 5 active+undersized+degraded+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_wait, 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 144 active+clean; 1.9 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 974 op/s; 6100/92505 objects degraded (6.594%); 7525/92505 objects misplaced (8.135%); 59 MiB/s, 937 objects/s recovering 2024-09-20T22:07:09.664 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:09 smithi067 ceph-mon[33511]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:07:09.664 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:09 smithi067 ceph-mon[33511]: osdmap e153: 8 total, 8 up, 8 in 2024-09-20T22:07:09.710 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:09.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:09.745 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:09.745 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:09.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:09 smithi060 ceph-mon[29694]: pgmap v196: 158 pgs: 1 peering, 5 active+undersized+degraded+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_wait, 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 144 active+clean; 1.9 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 974 op/s; 6100/92505 objects degraded (6.594%); 7525/92505 objects misplaced (8.135%); 59 MiB/s, 937 objects/s recovering 2024-09-20T22:07:09.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:09 smithi060 ceph-mon[29694]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:07:09.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:09 smithi060 ceph-mon[29694]: osdmap e153: 8 total, 8 up, 8 in 2024-09-20T22:07:09.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:09 smithi060 ceph-mon[40290]: pgmap v196: 158 pgs: 1 peering, 5 active+undersized+degraded+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_wait, 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 144 active+clean; 1.9 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 974 op/s; 6100/92505 objects degraded (6.594%); 7525/92505 objects misplaced (8.135%); 59 MiB/s, 937 objects/s recovering 2024-09-20T22:07:09.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:09 smithi060 ceph-mon[40290]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 5 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:07:09.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:09 smithi060 ceph-mon[40290]: osdmap e153: 8 total, 8 up, 8 in 2024-09-20T22:07:09.846 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:09.876 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:09.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:09.877 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:09.977 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:10.008 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:10.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:10.009 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:10.110 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:10.131 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:10.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:10.132 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:10.180 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:07:10.180 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:07:10.180 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:07:10.180 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:07:10.181 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:07:10.181 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:07:10.198 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:07:10.198 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:07:10.218 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:07:10.218 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:07:10.233 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:10.240 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:07:10.240 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:07:10.260 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:10.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:10.261 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:10.261 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:07:10.261 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:07:10.282 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:07:10.282 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:07:10.362 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:10.389 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:10.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:10.390 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:10.491 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:10.522 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:10 smithi060 ceph-mon[29694]: osdmap e154: 8 total, 8 up, 8 in 2024-09-20T22:07:10.522 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:10 smithi060 ceph-mon[29694]: pgmap v199: 158 pgs: 1 peering, 5 active+undersized+degraded+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_wait, 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 144 active+clean; 1.9 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 274 op/s; 6100/92505 objects degraded (6.594%); 7525/92505 objects misplaced (8.135%); 14 MiB/s, 221 objects/s recovering 2024-09-20T22:07:10.522 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:10 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3834249343' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:10.523 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:10 smithi060 ceph-mon[40290]: osdmap e154: 8 total, 8 up, 8 in 2024-09-20T22:07:10.523 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:10 smithi060 ceph-mon[40290]: pgmap v199: 158 pgs: 1 peering, 5 active+undersized+degraded+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_wait, 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 144 active+clean; 1.9 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 274 op/s; 6100/92505 objects degraded (6.594%); 7525/92505 objects misplaced (8.135%); 14 MiB/s, 221 objects/s recovering 2024-09-20T22:07:10.523 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:10 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3834249343' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:10.529 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:10.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:10.530 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:10.631 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:10.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:10.653 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:10.653 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:10.754 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:10 smithi067 ceph-mon[33511]: osdmap e154: 8 total, 8 up, 8 in 2024-09-20T22:07:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:10 smithi067 ceph-mon[33511]: pgmap v199: 158 pgs: 1 peering, 5 active+undersized+degraded+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_wait, 1 active+remapped+backfilling, 1 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 144 active+clean; 1.9 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 0 B/s wr, 274 op/s; 6100/92505 objects degraded (6.594%); 7525/92505 objects misplaced (8.135%); 14 MiB/s, 221 objects/s recovering 2024-09-20T22:07:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:10 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3834249343' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:10.775 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:10.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:10.776 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:10.877 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:10.899 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:10.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:10.900 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:11.000 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:11.023 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:11.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:11.024 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:11.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:11.147 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:11.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:11.149 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:11.250 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:11.273 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:11.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:11.274 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:11.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:11.400 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:11.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:11.401 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:11.502 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:11.547 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:11.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:11.548 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:11.648 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:11.681 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:11 smithi060 ceph-mon[29694]: osdmap e155: 8 total, 8 up, 8 in 2024-09-20T22:07:11.681 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:11 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 6100/92505 objects degraded (6.594%), 8 pgs degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-20T22:07:11.681 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:11 smithi060 ceph-mon[29694]: osdmap e156: 8 total, 8 up, 8 in 2024-09-20T22:07:11.681 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:11 smithi060 ceph-mon[40290]: osdmap e155: 8 total, 8 up, 8 in 2024-09-20T22:07:11.682 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:11 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 6100/92505 objects degraded (6.594%), 8 pgs degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-20T22:07:11.682 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:11 smithi060 ceph-mon[40290]: osdmap e156: 8 total, 8 up, 8 in 2024-09-20T22:07:11.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:11.697 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:11.697 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:11.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:11 smithi067 ceph-mon[33511]: osdmap e155: 8 total, 8 up, 8 in 2024-09-20T22:07:11.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:11 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 6100/92505 objects degraded (6.594%), 8 pgs degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-20T22:07:11.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:11 smithi067 ceph-mon[33511]: osdmap e156: 8 total, 8 up, 8 in 2024-09-20T22:07:11.798 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:11.826 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:11.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:11.828 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:11.928 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:11.962 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:11.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:11.963 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:12.030 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:12.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T22:07:12.064 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:12.086 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:12.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:12.087 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:12.188 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:12.210 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:12.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:12.211 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:12.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:07:11 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:07:12.312 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:12.335 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:12.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:12.335 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:12.364 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.0 to 1 (10000) 2024-09-20T22:07:12.436 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:12.479 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:12.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:12.480 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:12.581 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:12.609 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:12.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:12.610 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:12.711 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:12.761 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:12.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:12.762 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:12.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:12 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:07:12.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:12 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1233854217' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:07:12.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:12 smithi067 ceph-mon[33511]: pgmap v202: 158 pgs: 1 active+undersized+remapped, 2 peering, 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 149 active+clean; 1.7 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 684 op/s; 2225/84309 objects degraded (2.639%); 4999/84309 objects misplaced (5.929%); 57 MiB/s, 914 objects/s recovering 2024-09-20T22:07:12.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:12 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1233854217' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:07:12.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:12 smithi067 ceph-mon[33511]: osdmap e157: 8 total, 8 up, 8 in 2024-09-20T22:07:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:12 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:07:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:12 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1233854217' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:07:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:12 smithi060 ceph-mon[29694]: pgmap v202: 158 pgs: 1 active+undersized+remapped, 2 peering, 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 149 active+clean; 1.7 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 684 op/s; 2225/84309 objects degraded (2.639%); 4999/84309 objects misplaced (5.929%); 57 MiB/s, 914 objects/s recovering 2024-09-20T22:07:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:12 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1233854217' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:07:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:12 smithi060 ceph-mon[29694]: osdmap e157: 8 total, 8 up, 8 in 2024-09-20T22:07:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:12 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:07:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:12 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1233854217' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:07:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:12 smithi060 ceph-mon[40290]: pgmap v202: 158 pgs: 1 active+undersized+remapped, 2 peering, 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 149 active+clean; 1.7 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 684 op/s; 2225/84309 objects degraded (2.639%); 4999/84309 objects misplaced (5.929%); 57 MiB/s, 914 objects/s recovering 2024-09-20T22:07:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:12 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1233854217' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:07:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:12 smithi060 ceph-mon[40290]: osdmap e157: 8 total, 8 up, 8 in 2024-09-20T22:07:12.863 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:12.891 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:12.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:12.892 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:12.993 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:13.025 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:13.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:13.026 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:13.127 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:13.173 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:13.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:13.174 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:13.233 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 1 1 2024-09-20T22:07:13.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:13.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:07:13.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:13.329 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:13.329 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:13.430 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:13.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:13 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3582607817' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:07:13.456 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:07:13.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:13.523 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:13.523 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:13.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:13.645 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:13.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:13.646 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:13.747 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:13.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:13 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3582607817' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:07:13.782 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:13.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:13.783 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:13.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:13 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3582607817' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:07:13.884 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:13.921 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:13.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:13.922 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:14.022 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:14.057 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:14.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:14.058 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:14.159 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:14.199 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:14.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:14.200 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:14.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:14.345 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:14.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:14.346 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:14.447 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:14.489 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:14.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:14.490 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:14.591 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:14.621 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:14.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:14.622 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:14.722 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:14.755 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:14.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:14.756 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:14 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:07:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:14 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3582607817' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:07:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:14 smithi067 ceph-mon[33511]: osdmap e158: 8 total, 8 up, 8 in 2024-09-20T22:07:14.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:14 smithi067 ceph-mon[33511]: 4.e scrub starts 2024-09-20T22:07:14.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:14 smithi067 ceph-mon[33511]: 4.e scrub ok 2024-09-20T22:07:14.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:14 smithi067 ceph-mon[33511]: 7.5 scrub starts 2024-09-20T22:07:14.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:14 smithi067 ceph-mon[33511]: 7.5 scrub ok 2024-09-20T22:07:14.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:14 smithi067 ceph-mon[33511]: pgmap v205: 158 pgs: 1 active+undersized+remapped, 2 peering, 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 149 active+clean; 1.7 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 683 op/s; 2225/84309 objects degraded (2.639%); 4999/84309 objects misplaced (5.929%); 57 MiB/s, 914 objects/s recovering 2024-09-20T22:07:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:14 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:07:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:14 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3582607817' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:07:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:14 smithi060 ceph-mon[29694]: osdmap e158: 8 total, 8 up, 8 in 2024-09-20T22:07:14.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:14 smithi060 ceph-mon[29694]: 4.e scrub starts 2024-09-20T22:07:14.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:14 smithi060 ceph-mon[29694]: 4.e scrub ok 2024-09-20T22:07:14.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:14 smithi060 ceph-mon[29694]: 7.5 scrub starts 2024-09-20T22:07:14.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:14 smithi060 ceph-mon[29694]: 7.5 scrub ok 2024-09-20T22:07:14.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:14 smithi060 ceph-mon[29694]: pgmap v205: 158 pgs: 1 active+undersized+remapped, 2 peering, 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 149 active+clean; 1.7 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 683 op/s; 2225/84309 objects degraded (2.639%); 4999/84309 objects misplaced (5.929%); 57 MiB/s, 914 objects/s recovering 2024-09-20T22:07:14.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:14 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:07:14.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:14 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3582607817' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:07:14.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:14 smithi060 ceph-mon[40290]: osdmap e158: 8 total, 8 up, 8 in 2024-09-20T22:07:14.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:14 smithi060 ceph-mon[40290]: 4.e scrub starts 2024-09-20T22:07:14.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:14 smithi060 ceph-mon[40290]: 4.e scrub ok 2024-09-20T22:07:14.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:14 smithi060 ceph-mon[40290]: 7.5 scrub starts 2024-09-20T22:07:14.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:14 smithi060 ceph-mon[40290]: 7.5 scrub ok 2024-09-20T22:07:14.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:14 smithi060 ceph-mon[40290]: pgmap v205: 158 pgs: 1 active+undersized+remapped, 2 peering, 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 149 active+clean; 1.7 GiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 683 op/s; 2225/84309 objects degraded (2.639%); 4999/84309 objects misplaced (5.929%); 57 MiB/s, 914 objects/s recovering 2024-09-20T22:07:14.857 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:14.878 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:14.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:14.879 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:14.980 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:15.012 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:15.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:15.013 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:15.113 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:15.149 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:15.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:15.150 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:15.251 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:15.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:15.303 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:15.320 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:15.405 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:15.438 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:07:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:07:15.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:15.459 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:15.459 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:15.560 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:15.597 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:15.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:15.598 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:15.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:15.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:15.733 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:15.733 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:15.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[29694]: 7.2 scrub starts 2024-09-20T22:07:15.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[29694]: 3.4 scrub starts 2024-09-20T22:07:15.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[29694]: 3.4 scrub ok 2024-09-20T22:07:15.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[29694]: 7.9 scrub starts 2024-09-20T22:07:15.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[29694]: 6.1a scrub starts 2024-09-20T22:07:15.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[29694]: 6.1a scrub ok 2024-09-20T22:07:15.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[29694]: 7.11 scrub starts 2024-09-20T22:07:15.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[29694]: 7.2 scrub ok 2024-09-20T22:07:15.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[29694]: 7.11 scrub ok 2024-09-20T22:07:15.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[29694]: 7.d scrub starts 2024-09-20T22:07:15.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[40290]: 7.2 scrub starts 2024-09-20T22:07:15.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[40290]: 3.4 scrub starts 2024-09-20T22:07:15.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[40290]: 3.4 scrub ok 2024-09-20T22:07:15.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[40290]: 7.9 scrub starts 2024-09-20T22:07:15.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[40290]: 6.1a scrub starts 2024-09-20T22:07:15.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[40290]: 6.1a scrub ok 2024-09-20T22:07:15.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[40290]: 7.11 scrub starts 2024-09-20T22:07:15.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[40290]: 7.2 scrub ok 2024-09-20T22:07:15.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[40290]: 7.11 scrub ok 2024-09-20T22:07:15.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:15 smithi060 ceph-mon[40290]: 7.d scrub starts 2024-09-20T22:07:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:15 smithi067 ceph-mon[33511]: 7.2 scrub starts 2024-09-20T22:07:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:15 smithi067 ceph-mon[33511]: 3.4 scrub starts 2024-09-20T22:07:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:15 smithi067 ceph-mon[33511]: 3.4 scrub ok 2024-09-20T22:07:15.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:15 smithi067 ceph-mon[33511]: 7.9 scrub starts 2024-09-20T22:07:15.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:15 smithi067 ceph-mon[33511]: 6.1a scrub starts 2024-09-20T22:07:15.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:15 smithi067 ceph-mon[33511]: 6.1a scrub ok 2024-09-20T22:07:15.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:15 smithi067 ceph-mon[33511]: 7.11 scrub starts 2024-09-20T22:07:15.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:15 smithi067 ceph-mon[33511]: 7.2 scrub ok 2024-09-20T22:07:15.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:15 smithi067 ceph-mon[33511]: 7.11 scrub ok 2024-09-20T22:07:15.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:15 smithi067 ceph-mon[33511]: 7.d scrub starts 2024-09-20T22:07:15.834 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:15.891 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:15.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:15.892 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:15.992 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:16.032 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:16.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:16.033 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:16.134 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:16.171 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:16.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:16.172 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:16.272 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:16.296 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:16.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:16.297 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:16.398 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:16.435 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:16.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:16.436 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:16.537 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:16.567 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:16.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:16.568 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:16.668 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:16.715 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:16.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:16.717 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:16 smithi067 ceph-mon[33511]: 7.9 scrub ok 2024-09-20T22:07:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:16 smithi067 ceph-mon[33511]: 5.1b scrub starts 2024-09-20T22:07:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:16 smithi067 ceph-mon[33511]: 5.1b scrub ok 2024-09-20T22:07:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:16 smithi067 ceph-mon[33511]: 5.8 scrub starts 2024-09-20T22:07:16.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:16 smithi067 ceph-mon[33511]: 5.8 scrub ok 2024-09-20T22:07:16.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:16 smithi067 ceph-mon[33511]: 6.18 scrub starts 2024-09-20T22:07:16.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:16 smithi067 ceph-mon[33511]: 6.18 scrub ok 2024-09-20T22:07:16.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:16 smithi067 ceph-mon[33511]: 4.13 scrub starts 2024-09-20T22:07:16.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:16 smithi067 ceph-mon[33511]: 4.13 scrub ok 2024-09-20T22:07:16.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:16 smithi067 ceph-mon[33511]: 7.d scrub ok 2024-09-20T22:07:16.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:16 smithi067 ceph-mon[33511]: pgmap v206: 158 pgs: 1 active+clean+scrubbing, 1 active+undersized+remapped, 1 peering, 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 149 active+clean; 1.6 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s; 1925/76938 objects degraded (2.502%); 4725/76938 objects misplaced (6.141%); 30 MiB/s, 477 objects/s recovering 2024-09-20T22:07:16.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:16 smithi067 ceph-mon[33511]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:07:16.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:16 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 2225/84309 objects degraded (2.639%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T22:07:16.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:16 smithi067 ceph-mon[33511]: osdmap e159: 8 total, 8 up, 8 in 2024-09-20T22:07:16.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[29694]: 7.9 scrub ok 2024-09-20T22:07:16.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[29694]: 5.1b scrub starts 2024-09-20T22:07:16.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[29694]: 5.1b scrub ok 2024-09-20T22:07:16.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[29694]: 5.8 scrub starts 2024-09-20T22:07:16.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[29694]: 5.8 scrub ok 2024-09-20T22:07:16.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[29694]: 6.18 scrub starts 2024-09-20T22:07:16.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[29694]: 6.18 scrub ok 2024-09-20T22:07:16.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[29694]: 4.13 scrub starts 2024-09-20T22:07:16.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[29694]: 4.13 scrub ok 2024-09-20T22:07:16.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[29694]: 7.d scrub ok 2024-09-20T22:07:16.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[29694]: pgmap v206: 158 pgs: 1 active+clean+scrubbing, 1 active+undersized+remapped, 1 peering, 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 149 active+clean; 1.6 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s; 1925/76938 objects degraded (2.502%); 4725/76938 objects misplaced (6.141%); 30 MiB/s, 477 objects/s recovering 2024-09-20T22:07:16.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[29694]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:07:16.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 2225/84309 objects degraded (2.639%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T22:07:16.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[29694]: osdmap e159: 8 total, 8 up, 8 in 2024-09-20T22:07:16.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[40290]: 7.9 scrub ok 2024-09-20T22:07:16.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[40290]: 5.1b scrub starts 2024-09-20T22:07:16.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[40290]: 5.1b scrub ok 2024-09-20T22:07:16.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[40290]: 5.8 scrub starts 2024-09-20T22:07:16.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[40290]: 5.8 scrub ok 2024-09-20T22:07:16.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[40290]: 6.18 scrub starts 2024-09-20T22:07:16.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[40290]: 6.18 scrub ok 2024-09-20T22:07:16.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[40290]: 4.13 scrub starts 2024-09-20T22:07:16.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[40290]: 4.13 scrub ok 2024-09-20T22:07:16.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[40290]: 7.d scrub ok 2024-09-20T22:07:16.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[40290]: pgmap v206: 158 pgs: 1 active+clean+scrubbing, 1 active+undersized+remapped, 1 peering, 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 149 active+clean; 1.6 GiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 1.8 KiB/s rd, 1 op/s; 1925/76938 objects degraded (2.502%); 4725/76938 objects misplaced (6.141%); 30 MiB/s, 477 objects/s recovering 2024-09-20T22:07:16.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[40290]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 6 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:07:16.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 2225/84309 objects degraded (2.639%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T22:07:16.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:16 smithi060 ceph-mon[40290]: osdmap e159: 8 total, 8 up, 8 in 2024-09-20T22:07:16.817 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:16.862 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:16.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:16.863 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:16.881 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:16.959 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:16.963 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:07:16.963 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:07:16.964 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:16.990 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:16.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:16.991 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:17.092 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:17.134 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:17.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:17.135 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:17.236 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:17.281 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:17.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:17.282 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:17.383 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:17.422 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:17.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:17.423 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:17.524 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:17.545 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:17.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:17.546 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:17.647 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:17.676 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:17.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:17.677 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:17 smithi067 ceph-mon[33511]: 5.15 scrub starts 2024-09-20T22:07:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:17 smithi067 ceph-mon[33511]: 5.15 scrub ok 2024-09-20T22:07:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:17 smithi067 ceph-mon[33511]: 4.1c scrub starts 2024-09-20T22:07:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:17 smithi067 ceph-mon[33511]: 4.1c scrub ok 2024-09-20T22:07:17.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:17 smithi067 ceph-mon[33511]: 2.0 scrub starts 2024-09-20T22:07:17.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:17 smithi067 ceph-mon[33511]: 2.0 scrub ok 2024-09-20T22:07:17.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:17 smithi067 ceph-mon[33511]: 7.19 scrub starts 2024-09-20T22:07:17.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:17 smithi067 ceph-mon[33511]: 7.19 scrub ok 2024-09-20T22:07:17.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:17 smithi060 ceph-mon[29694]: 5.15 scrub starts 2024-09-20T22:07:17.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:17 smithi060 ceph-mon[29694]: 5.15 scrub ok 2024-09-20T22:07:17.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:17 smithi060 ceph-mon[29694]: 4.1c scrub starts 2024-09-20T22:07:17.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:17 smithi060 ceph-mon[29694]: 4.1c scrub ok 2024-09-20T22:07:17.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:17 smithi060 ceph-mon[29694]: 2.0 scrub starts 2024-09-20T22:07:17.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:17 smithi060 ceph-mon[29694]: 2.0 scrub ok 2024-09-20T22:07:17.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:17 smithi060 ceph-mon[29694]: 7.19 scrub starts 2024-09-20T22:07:17.777 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:17 smithi060 ceph-mon[29694]: 7.19 scrub ok 2024-09-20T22:07:17.777 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:17 smithi060 ceph-mon[40290]: 5.15 scrub starts 2024-09-20T22:07:17.777 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:17 smithi060 ceph-mon[40290]: 5.15 scrub ok 2024-09-20T22:07:17.778 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:17 smithi060 ceph-mon[40290]: 4.1c scrub starts 2024-09-20T22:07:17.778 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:17 smithi060 ceph-mon[40290]: 4.1c scrub ok 2024-09-20T22:07:17.778 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:17 smithi060 ceph-mon[40290]: 2.0 scrub starts 2024-09-20T22:07:17.778 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:17 smithi060 ceph-mon[40290]: 2.0 scrub ok 2024-09-20T22:07:17.778 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:17 smithi060 ceph-mon[40290]: 7.19 scrub starts 2024-09-20T22:07:17.778 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:17 smithi060 ceph-mon[40290]: 7.19 scrub ok 2024-09-20T22:07:17.778 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:17.809 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:07:17.810 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:17.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:17.811 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:17.912 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:17.979 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:17.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:17.980 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:18.009 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:07:18.009 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:07:18.064 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:07:18.064 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:07:18.080 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:18.103 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:07:18.104 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:07:18.133 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:18.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:18.134 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:18.142 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:07:18.143 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:07:18.181 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:07:18.181 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:07:18.214 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:07:18.215 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:07:18.235 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:18.251 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:07:18.252 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:07:18.275 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:18.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:18.276 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:18.284 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:07:18.284 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:07:18.360 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T22:07:18.377 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:18.413 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:18.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:18.414 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:18.515 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:18.540 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:18.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:18.541 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:18.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:18.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[29694]: 7.e scrub starts 2024-09-20T22:07:18.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[29694]: 5.5 scrub starts 2024-09-20T22:07:18.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[29694]: 5.18 scrub starts 2024-09-20T22:07:18.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[29694]: 5.18 scrub ok 2024-09-20T22:07:18.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[29694]: 5.5 scrub ok 2024-09-20T22:07:18.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:07:18.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:07:18.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1891331954' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:18.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[29694]: pgmap v208: 158 pgs: 1 active+clean+scrubbing, 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 151 active+clean; 1.4 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 1925/70947 objects degraded (2.713%); 4168/70947 objects misplaced (5.875%) 2024-09-20T22:07:18.643 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[29694]: 7.12 scrub starts 2024-09-20T22:07:18.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[29694]: 7.12 scrub ok 2024-09-20T22:07:18.644 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[40290]: 7.e scrub starts 2024-09-20T22:07:18.644 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[40290]: 5.5 scrub starts 2024-09-20T22:07:18.644 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[40290]: 5.18 scrub starts 2024-09-20T22:07:18.644 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[40290]: 5.18 scrub ok 2024-09-20T22:07:18.644 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[40290]: 5.5 scrub ok 2024-09-20T22:07:18.644 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:07:18.644 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:07:18.644 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1891331954' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:18.644 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[40290]: pgmap v208: 158 pgs: 1 active+clean+scrubbing, 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 151 active+clean; 1.4 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 1925/70947 objects degraded (2.713%); 4168/70947 objects misplaced (5.875%) 2024-09-20T22:07:18.644 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[40290]: 7.12 scrub starts 2024-09-20T22:07:18.644 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:18 smithi060 ceph-mon[40290]: 7.12 scrub ok 2024-09-20T22:07:18.669 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:18.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:18.670 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:18 smithi067 ceph-mon[33511]: 7.e scrub starts 2024-09-20T22:07:18.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:18 smithi067 ceph-mon[33511]: 5.5 scrub starts 2024-09-20T22:07:18.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:18 smithi067 ceph-mon[33511]: 5.18 scrub starts 2024-09-20T22:07:18.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:18 smithi067 ceph-mon[33511]: 5.18 scrub ok 2024-09-20T22:07:18.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:18 smithi067 ceph-mon[33511]: 5.5 scrub ok 2024-09-20T22:07:18.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:18 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:07:18.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:18 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:07:18.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:18 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1891331954' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:18.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:18 smithi067 ceph-mon[33511]: pgmap v208: 158 pgs: 1 active+clean+scrubbing, 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 151 active+clean; 1.4 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 1925/70947 objects degraded (2.713%); 4168/70947 objects misplaced (5.875%) 2024-09-20T22:07:18.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:18 smithi067 ceph-mon[33511]: 7.12 scrub starts 2024-09-20T22:07:18.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:18 smithi067 ceph-mon[33511]: 7.12 scrub ok 2024-09-20T22:07:18.772 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:18.799 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:18.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:18.800 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:18.901 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:18.935 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:18.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:18.936 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:19.037 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:19.090 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:19.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:19.092 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:19.192 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:19.253 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:19.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:19.254 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:19.355 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:19.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:19.398 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:19.398 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:19.499 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:19.530 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:19.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:19.531 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:19.532 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:19.598 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:19.631 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:19.657 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:19.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:19.658 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:19.759 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:19.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:19 smithi067 ceph-mon[33511]: 7.e scrub ok 2024-09-20T22:07:19.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:19 smithi067 ceph-mon[33511]: 7.14 scrub starts 2024-09-20T22:07:19.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:19 smithi067 ceph-mon[33511]: 7.4 scrub starts 2024-09-20T22:07:19.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:19 smithi067 ceph-mon[33511]: osdmap e160: 8 total, 8 up, 8 in 2024-09-20T22:07:19.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:19 smithi067 ceph-mon[33511]: 7.14 scrub ok 2024-09-20T22:07:19.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:19 smithi067 ceph-mon[33511]: 7.4 scrub ok 2024-09-20T22:07:19.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:19 smithi067 ceph-mon[33511]: 4.1f scrub starts 2024-09-20T22:07:19.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:19 smithi067 ceph-mon[33511]: 4.1f scrub ok 2024-09-20T22:07:19.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:19 smithi060 ceph-mon[29694]: 7.e scrub ok 2024-09-20T22:07:19.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:19 smithi060 ceph-mon[29694]: 7.14 scrub starts 2024-09-20T22:07:19.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:19 smithi060 ceph-mon[29694]: 7.4 scrub starts 2024-09-20T22:07:19.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:19 smithi060 ceph-mon[29694]: osdmap e160: 8 total, 8 up, 8 in 2024-09-20T22:07:19.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:19 smithi060 ceph-mon[29694]: 7.14 scrub ok 2024-09-20T22:07:19.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:19 smithi060 ceph-mon[29694]: 7.4 scrub ok 2024-09-20T22:07:19.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:19 smithi060 ceph-mon[29694]: 4.1f scrub starts 2024-09-20T22:07:19.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:19 smithi060 ceph-mon[29694]: 4.1f scrub ok 2024-09-20T22:07:19.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:19 smithi060 ceph-mon[40290]: 7.e scrub ok 2024-09-20T22:07:19.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:19 smithi060 ceph-mon[40290]: 7.14 scrub starts 2024-09-20T22:07:19.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:19 smithi060 ceph-mon[40290]: 7.4 scrub starts 2024-09-20T22:07:19.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:19 smithi060 ceph-mon[40290]: osdmap e160: 8 total, 8 up, 8 in 2024-09-20T22:07:19.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:19 smithi060 ceph-mon[40290]: 7.14 scrub ok 2024-09-20T22:07:19.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:19 smithi060 ceph-mon[40290]: 7.4 scrub ok 2024-09-20T22:07:19.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:19 smithi060 ceph-mon[40290]: 4.1f scrub starts 2024-09-20T22:07:19.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:19 smithi060 ceph-mon[40290]: 4.1f scrub ok 2024-09-20T22:07:19.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:19.810 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:19.810 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:19.911 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:19.954 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:19.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:19.955 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:20.055 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:20.095 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:20.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:20.096 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:20.196 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:20.224 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:20.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:20.225 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:20.326 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:20.357 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:20.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:20.358 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:20.459 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:20.508 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.1 to 1 (10000) 2024-09-20T22:07:20.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:20.515 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:20.516 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:20.616 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:20.670 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:20.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:20.672 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:20.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:20 smithi067 ceph-mon[33511]: 4.10 scrub starts 2024-09-20T22:07:20.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:20 smithi067 ceph-mon[33511]: 4.10 scrub ok 2024-09-20T22:07:20.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:20 smithi067 ceph-mon[33511]: 5.c scrub starts 2024-09-20T22:07:20.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:20 smithi067 ceph-mon[33511]: 5.c scrub ok 2024-09-20T22:07:20.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:20 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/779976617' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:07:20.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:20 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:07:20.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:07:20.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:20 smithi067 ceph-mon[33511]: pgmap v210: 158 pgs: 1 active+clean+scrubbing, 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 151 active+clean; 1.4 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 777 B/s rd, 0 op/s; 1925/70947 objects degraded (2.713%); 4168/70947 objects misplaced (5.875%) 2024-09-20T22:07:20.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:20 smithi060 ceph-mon[29694]: 4.10 scrub starts 2024-09-20T22:07:20.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:20 smithi060 ceph-mon[29694]: 4.10 scrub ok 2024-09-20T22:07:20.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:20 smithi060 ceph-mon[29694]: 5.c scrub starts 2024-09-20T22:07:20.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:20 smithi060 ceph-mon[29694]: 5.c scrub ok 2024-09-20T22:07:20.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:20 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/779976617' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:07:20.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:20 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:07:20.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:07:20.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:20 smithi060 ceph-mon[29694]: pgmap v210: 158 pgs: 1 active+clean+scrubbing, 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 151 active+clean; 1.4 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 777 B/s rd, 0 op/s; 1925/70947 objects degraded (2.713%); 4168/70947 objects misplaced (5.875%) 2024-09-20T22:07:20.769 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:20 smithi060 ceph-mon[40290]: 4.10 scrub starts 2024-09-20T22:07:20.769 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:20 smithi060 ceph-mon[40290]: 4.10 scrub ok 2024-09-20T22:07:20.769 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:20 smithi060 ceph-mon[40290]: 5.c scrub starts 2024-09-20T22:07:20.769 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:20 smithi060 ceph-mon[40290]: 5.c scrub ok 2024-09-20T22:07:20.769 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:20 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/779976617' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:07:20.769 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:20 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:07:20.769 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:07:20.769 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:20 smithi060 ceph-mon[40290]: pgmap v210: 158 pgs: 1 active+clean+scrubbing, 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 151 active+clean; 1.4 GiB data, 6.0 GiB used, 709 GiB / 715 GiB avail; 777 B/s rd, 0 op/s; 1925/70947 objects degraded (2.713%); 4168/70947 objects misplaced (5.875%) 2024-09-20T22:07:20.772 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:20.801 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:20.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:20.802 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:20.822 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:07:20.823 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:07:20.823 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:07:20.823 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:07:20.823 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:07:20.903 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:20.935 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:20.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:20.936 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:21.036 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:21.074 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:21.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:21.075 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:21.176 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:21.206 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:21.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:21.207 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:21.243 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:07:21.307 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:21.338 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:21.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:21.339 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:21.440 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:21.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:21.484 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:21.484 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:21.535 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:07:21.585 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:21.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:21.615 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:21.615 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:21.716 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:21.756 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:21.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:21.757 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:21.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[29694]: 5.16 scrub starts 2024-09-20T22:07:21.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[29694]: 5.16 scrub ok 2024-09-20T22:07:21.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[29694]: 5.b scrub starts 2024-09-20T22:07:21.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[29694]: 5.b scrub ok 2024-09-20T22:07:21.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[29694]: 3.0 scrub starts 2024-09-20T22:07:21.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[29694]: 3.0 scrub ok 2024-09-20T22:07:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T22:07:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[29694]: osdmap e161: 8 total, 8 up, 8 in 2024-09-20T22:07:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/100990634' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[29694]: 3.a scrub starts 2024-09-20T22:07:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[29694]: 3.a scrub ok 2024-09-20T22:07:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 1925/70947 objects degraded (2.713%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T22:07:21.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[40290]: 5.16 scrub starts 2024-09-20T22:07:21.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[40290]: 5.16 scrub ok 2024-09-20T22:07:21.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[40290]: 5.b scrub starts 2024-09-20T22:07:21.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[40290]: 5.b scrub ok 2024-09-20T22:07:21.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[40290]: 3.0 scrub starts 2024-09-20T22:07:21.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[40290]: 3.0 scrub ok 2024-09-20T22:07:21.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T22:07:21.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[40290]: osdmap e161: 8 total, 8 up, 8 in 2024-09-20T22:07:21.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/100990634' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:21.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:21.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[40290]: 3.a scrub starts 2024-09-20T22:07:21.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[40290]: 3.a scrub ok 2024-09-20T22:07:21.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:21 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 1925/70947 objects degraded (2.713%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T22:07:21.858 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:21.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:21.893 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:21.893 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:21.945 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:21 smithi067 ceph-mon[33511]: 5.16 scrub starts 2024-09-20T22:07:21.945 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:21 smithi067 ceph-mon[33511]: 5.16 scrub ok 2024-09-20T22:07:21.945 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:21 smithi067 ceph-mon[33511]: 5.b scrub starts 2024-09-20T22:07:21.945 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:21 smithi067 ceph-mon[33511]: 5.b scrub ok 2024-09-20T22:07:21.945 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:21 smithi067 ceph-mon[33511]: 3.0 scrub starts 2024-09-20T22:07:21.945 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:21 smithi067 ceph-mon[33511]: 3.0 scrub ok 2024-09-20T22:07:21.945 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:21 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T22:07:21.945 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:21 smithi067 ceph-mon[33511]: osdmap e161: 8 total, 8 up, 8 in 2024-09-20T22:07:21.945 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:21 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/100990634' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:21.945 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:21 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:21.945 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:21 smithi067 ceph-mon[33511]: 3.a scrub starts 2024-09-20T22:07:21.946 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:21 smithi067 ceph-mon[33511]: 3.a scrub ok 2024-09-20T22:07:21.946 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:21 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 1925/70947 objects degraded (2.713%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T22:07:21.977 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T22:07:21.994 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:22.025 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:22.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:22.026 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:22.126 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:22.155 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:22.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:22.156 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:22.257 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:22.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:07:21 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:07:22.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:22.300 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:22.300 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:22.401 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:22.446 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:22.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:22.447 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:22.548 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:22.585 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:22.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:22.586 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:22.649 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:22 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:07:22.649 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:22 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:07:22.649 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:22 smithi060 ceph-mon[29694]: osdmap e162: 8 total, 8 up, 8 in 2024-09-20T22:07:22.649 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:22 smithi060 ceph-mon[29694]: 3.e scrub starts 2024-09-20T22:07:22.649 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:22 smithi060 ceph-mon[29694]: 3.e scrub ok 2024-09-20T22:07:22.649 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:22 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:07:22.649 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:22 smithi060 ceph-mon[29694]: pgmap v213: 158 pgs: 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 152 active+clean; 1.1 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 909 B/s rd, 0 op/s; 1518/55395 objects degraded (2.740%); 3198/55395 objects misplaced (5.773%) 2024-09-20T22:07:22.650 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:22 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:07:22.650 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:22 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:07:22.650 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:22 smithi060 ceph-mon[40290]: osdmap e162: 8 total, 8 up, 8 in 2024-09-20T22:07:22.650 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:22 smithi060 ceph-mon[40290]: 3.e scrub starts 2024-09-20T22:07:22.650 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:22 smithi060 ceph-mon[40290]: 3.e scrub ok 2024-09-20T22:07:22.650 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:22 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:07:22.650 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:22 smithi060 ceph-mon[40290]: pgmap v213: 158 pgs: 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 152 active+clean; 1.1 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 909 B/s rd, 0 op/s; 1518/55395 objects degraded (2.740%); 3198/55395 objects misplaced (5.773%) 2024-09-20T22:07:22.688 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:22.698 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 2 1 2024-09-20T22:07:22.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:22.741 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:22.742 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:22.843 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:22.869 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:22.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:22.871 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:22.971 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:23.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:22 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:07:23.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:22 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:07:23.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:22 smithi067 ceph-mon[33511]: osdmap e162: 8 total, 8 up, 8 in 2024-09-20T22:07:23.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:22 smithi067 ceph-mon[33511]: 3.e scrub starts 2024-09-20T22:07:23.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:22 smithi067 ceph-mon[33511]: 3.e scrub ok 2024-09-20T22:07:23.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:22 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:07:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:22 smithi067 ceph-mon[33511]: pgmap v213: 158 pgs: 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 152 active+clean; 1.1 GiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 909 B/s rd, 0 op/s; 1518/55395 objects degraded (2.740%); 3198/55395 objects misplaced (5.773%) 2024-09-20T22:07:23.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:23.020 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:23.020 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:23.121 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:23.169 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:23.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:23.170 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:23.270 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:23.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:23.302 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:23.302 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:23.403 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:23.435 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:23.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:23.437 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:23.537 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:23.567 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:23.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:23.568 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:23.669 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:23.701 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:23.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:23.702 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:23 smithi060 ceph-mon[29694]: 2.2 deep-scrub starts 2024-09-20T22:07:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:23 smithi060 ceph-mon[29694]: 2.2 deep-scrub ok 2024-09-20T22:07:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:23 smithi060 ceph-mon[29694]: 5.a scrub starts 2024-09-20T22:07:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:23 smithi060 ceph-mon[29694]: 5.a scrub ok 2024-09-20T22:07:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:23 smithi060 ceph-mon[29694]: 4.9 scrub starts 2024-09-20T22:07:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:23 smithi060 ceph-mon[29694]: 4.9 scrub ok 2024-09-20T22:07:23.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:23 smithi060 ceph-mon[40290]: 2.2 deep-scrub starts 2024-09-20T22:07:23.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:23 smithi060 ceph-mon[40290]: 2.2 deep-scrub ok 2024-09-20T22:07:23.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:23 smithi060 ceph-mon[40290]: 5.a scrub starts 2024-09-20T22:07:23.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:23 smithi060 ceph-mon[40290]: 5.a scrub ok 2024-09-20T22:07:23.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:23 smithi060 ceph-mon[40290]: 4.9 scrub starts 2024-09-20T22:07:23.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:23 smithi060 ceph-mon[40290]: 4.9 scrub ok 2024-09-20T22:07:23.803 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:23.835 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:23.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:23.836 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:23.937 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:23.976 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:23.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:23.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:23 smithi067 ceph-mon[33511]: 2.2 deep-scrub starts 2024-09-20T22:07:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:23 smithi067 ceph-mon[33511]: 2.2 deep-scrub ok 2024-09-20T22:07:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:23 smithi067 ceph-mon[33511]: 5.a scrub starts 2024-09-20T22:07:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:23 smithi067 ceph-mon[33511]: 5.a scrub ok 2024-09-20T22:07:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:23 smithi067 ceph-mon[33511]: 4.9 scrub starts 2024-09-20T22:07:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:23 smithi067 ceph-mon[33511]: 4.9 scrub ok 2024-09-20T22:07:24.078 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:24.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:24.117 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:24.117 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:24.217 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:24.261 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:24.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:24.262 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:24.363 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:24.389 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:24.396 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:24.407 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:24.458 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:24.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:24.459 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:24.560 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:24.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:24.612 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:24.613 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:24.713 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:24.755 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:24.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:24.756 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:24.857 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:24.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:24 smithi060 ceph-mon[29694]: 5.7 scrub starts 2024-09-20T22:07:24.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:24 smithi060 ceph-mon[29694]: 5.7 scrub ok 2024-09-20T22:07:24.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:24 smithi060 ceph-mon[29694]: 3.8 scrub starts 2024-09-20T22:07:24.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:24 smithi060 ceph-mon[29694]: 3.8 scrub ok 2024-09-20T22:07:24.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:24 smithi060 ceph-mon[29694]: 6.e deep-scrub starts 2024-09-20T22:07:24.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:24 smithi060 ceph-mon[29694]: 6.e deep-scrub ok 2024-09-20T22:07:24.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:24 smithi060 ceph-mon[29694]: 6.10 scrub starts 2024-09-20T22:07:24.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:24 smithi060 ceph-mon[29694]: 6.10 scrub ok 2024-09-20T22:07:24.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:24 smithi060 ceph-mon[29694]: pgmap v214: 158 pgs: 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 152 active+clean; 1.1 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1518/55395 objects degraded (2.740%); 3198/55395 objects misplaced (5.773%) 2024-09-20T22:07:24.867 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:24 smithi060 ceph-mon[40290]: 5.7 scrub starts 2024-09-20T22:07:24.867 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:24 smithi060 ceph-mon[40290]: 5.7 scrub ok 2024-09-20T22:07:24.867 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:24 smithi060 ceph-mon[40290]: 3.8 scrub starts 2024-09-20T22:07:24.868 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:24 smithi060 ceph-mon[40290]: 3.8 scrub ok 2024-09-20T22:07:24.868 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:24 smithi060 ceph-mon[40290]: 6.e deep-scrub starts 2024-09-20T22:07:24.868 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:24 smithi060 ceph-mon[40290]: 6.e deep-scrub ok 2024-09-20T22:07:24.868 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:24 smithi060 ceph-mon[40290]: 6.10 scrub starts 2024-09-20T22:07:24.868 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:24 smithi060 ceph-mon[40290]: 6.10 scrub ok 2024-09-20T22:07:24.868 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:24 smithi060 ceph-mon[40290]: pgmap v214: 158 pgs: 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 152 active+clean; 1.1 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1518/55395 objects degraded (2.740%); 3198/55395 objects misplaced (5.773%) 2024-09-20T22:07:24.885 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:24.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:24.886 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:24.987 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:24 smithi067 ceph-mon[33511]: 5.7 scrub starts 2024-09-20T22:07:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:24 smithi067 ceph-mon[33511]: 5.7 scrub ok 2024-09-20T22:07:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:24 smithi067 ceph-mon[33511]: 3.8 scrub starts 2024-09-20T22:07:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:24 smithi067 ceph-mon[33511]: 3.8 scrub ok 2024-09-20T22:07:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:24 smithi067 ceph-mon[33511]: 6.e deep-scrub starts 2024-09-20T22:07:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:24 smithi067 ceph-mon[33511]: 6.e deep-scrub ok 2024-09-20T22:07:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:24 smithi067 ceph-mon[33511]: 6.10 scrub starts 2024-09-20T22:07:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:24 smithi067 ceph-mon[33511]: 6.10 scrub ok 2024-09-20T22:07:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:24 smithi067 ceph-mon[33511]: pgmap v214: 158 pgs: 2 active+undersized+degraded+remapped+backfill_toofull, 4 active+remapped+backfill_toofull, 152 active+clean; 1.1 GiB data, 5.0 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 1518/55395 objects degraded (2.740%); 3198/55395 objects misplaced (5.773%) 2024-09-20T22:07:25.029 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:25.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:25.030 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:25.131 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:25.141 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:07:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:07:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:07:25.158 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:25.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:25.159 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:25.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:25.306 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:25.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:25.307 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:25.408 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:25.433 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:25.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:25.434 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:25.535 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:25.568 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:07:25.568 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:07:25.568 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:07:25.568 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:07:25.568 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.569 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169961 7.13 7:cf59cdb9:::benchmark_data_smithi060_78800_object36928:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:07:25.569 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440329+0000", 2024-09-20T22:07:25.569 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.952062181000002, 2024-09-20T22:07:25.569 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.067711163, 2024-09-20T22:07:25.569 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.569 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.569 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:07:25.569 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.569 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.569 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.569 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169961 2024-09-20T22:07:25.569 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.569 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.569 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.570 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.570 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440329+0000", 2024-09-20T22:07:25.570 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.570 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.570 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.570 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.570 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440329+0000", 2024-09-20T22:07:25.570 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.570 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.570 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.570 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.570 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440330+0000", 2024-09-20T22:07:25.570 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2800000000000005e-07 2024-09-20T22:07:25.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.571 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.571 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.571 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440335+0000", 2024-09-20T22:07:25.571 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9520000000000002e-06 2024-09-20T22:07:25.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.571 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.571 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.571 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440347+0000", 2024-09-20T22:07:25.571 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1732999999999999e-05 2024-09-20T22:07:25.571 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.571 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.571 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.571 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440354+0000", 2024-09-20T22:07:25.572 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6420000000000007e-06 2024-09-20T22:07:25.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.572 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.572 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.572 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.698470+0000", 2024-09-20T22:07:25.572 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.25811523800000002 2024-09-20T22:07:25.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.572 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.572 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.572 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.698477+0000", 2024-09-20T22:07:25.572 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9999999999999999e-06 2024-09-20T22:07:25.572 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.572 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.572 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.573 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.507960+0000", 2024-09-20T22:07:25.573 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.809483493 2024-09-20T22:07:25.573 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.573 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.573 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.573 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.508040+0000", 2024-09-20T22:07:25.573 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0177000000000002e-05 2024-09-20T22:07:25.573 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.573 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.573 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.573 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.573 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.573 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169966 7.13 7:c9a28d28:::benchmark_data_smithi060_78800_object36933:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:07:25.573 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440368+0000", 2024-09-20T22:07:25.573 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.952022924999994, 2024-09-20T22:07:25.574 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0730672910000001, 2024-09-20T22:07:25.574 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.574 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.574 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:07:25.574 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.574 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.574 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.574 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169966 2024-09-20T22:07:25.574 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.574 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.574 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.574 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.574 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440368+0000", 2024-09-20T22:07:25.574 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.575 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.575 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.575 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.575 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440368+0000", 2024-09-20T22:07:25.575 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.575 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.575 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.575 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.575 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440369+0000", 2024-09-20T22:07:25.575 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9100000000000004e-07 2024-09-20T22:07:25.575 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.575 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.575 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.575 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440371+0000", 2024-09-20T22:07:25.576 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5280000000000002e-06 2024-09-20T22:07:25.576 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.576 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.576 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.576 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440378+0000", 2024-09-20T22:07:25.576 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7340000000000002e-06 2024-09-20T22:07:25.576 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.576 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.576 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.576 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440383+0000", 2024-09-20T22:07:25.576 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3750000000000002e-06 2024-09-20T22:07:25.576 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.576 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.576 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.576 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.698495+0000", 2024-09-20T22:07:25.577 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.258111427 2024-09-20T22:07:25.577 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.577 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.577 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.577 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.698499+0000", 2024-09-20T22:07:25.577 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.658e-06 2024-09-20T22:07:25.577 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.577 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.577 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.577 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.509530+0000", 2024-09-20T22:07:25.577 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.81103158399999997 2024-09-20T22:07:25.577 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.577 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.577 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:07:25.577 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.509656+0000", 2024-09-20T22:07:25.578 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012591300000000001 2024-09-20T22:07:25.578 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.578 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.578 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.578 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.513371+0000", 2024-09-20T22:07:25.578 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037147370000000001 2024-09-20T22:07:25.578 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.578 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.578 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.578 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.513436+0000", 2024-09-20T22:07:25.578 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4844000000000005e-05 2024-09-20T22:07:25.578 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.578 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.578 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.579 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.579 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.579 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198108 7.10 7:0b18f923:::benchmark_data_smithi060_78800_object65075:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:07:25.579 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.462286+0000", 2024-09-20T22:07:25.579 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 59.930105570999999, 2024-09-20T22:07:25.579 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97321715200000003, 2024-09-20T22:07:25.579 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.579 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.579 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:25.579 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.579 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.579 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.579 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198108 2024-09-20T22:07:25.579 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.580 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.580 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.580 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.580 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462286+0000", 2024-09-20T22:07:25.580 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.580 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.580 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.580 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462286+0000", 2024-09-20T22:07:25.580 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.580 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.580 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.580 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462287+0000", 2024-09-20T22:07:25.580 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.668e-06 2024-09-20T22:07:25.580 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.581 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.581 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.581 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462293+0000", 2024-09-20T22:07:25.581 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0020000000000003e-06 2024-09-20T22:07:25.581 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.581 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.581 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.581 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462315+0000", 2024-09-20T22:07:25.581 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1809000000000001e-05 2024-09-20T22:07:25.581 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.581 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.581 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.581 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462324+0000", 2024-09-20T22:07:25.581 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6209999999999993e-06 2024-09-20T22:07:25.581 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.582 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.582 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.582 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462389+0000", 2024-09-20T22:07:25.582 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5007999999999998e-05 2024-09-20T22:07:25.582 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.582 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.582 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.582 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462397+0000", 2024-09-20T22:07:25.582 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3690000000000007e-06 2024-09-20T22:07:25.582 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.582 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.582 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.583 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430729+0000", 2024-09-20T22:07:25.583 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.96833147100000005 2024-09-20T22:07:25.583 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.583 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.583 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:25.583 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430856+0000", 2024-09-20T22:07:25.583 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012690800000000001 2024-09-20T22:07:25.583 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.583 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.583 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:25.583 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.431205+0000", 2024-09-20T22:07:25.583 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00034923899999999999 2024-09-20T22:07:25.583 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.583 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.584 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.584 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435090+0000", 2024-09-20T22:07:25.584 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038847090000000001 2024-09-20T22:07:25.584 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.584 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.584 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:25.584 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435103+0000", 2024-09-20T22:07:25.584 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169999999999999e-05 2024-09-20T22:07:25.584 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.584 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.584 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.584 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435254+0000", 2024-09-20T22:07:25.584 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015143900000000001 2024-09-20T22:07:25.584 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.584 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.584 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:25.585 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435404+0000", 2024-09-20T22:07:25.585 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014957600000000001 2024-09-20T22:07:25.585 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.585 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.585 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.585 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435503+0000", 2024-09-20T22:07:25.585 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9162999999999999e-05 2024-09-20T22:07:25.585 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.585 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.585 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.585 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.585 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.585 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198131 7.10 7:0b2c8a99:::benchmark_data_smithi060_78800_object65098:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:07:25.585 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.479927+0000", 2024-09-20T22:07:25.585 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 59.912464235000002, 2024-09-20T22:07:25.586 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.96409556600000001, 2024-09-20T22:07:25.586 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.586 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.586 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:25.586 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.586 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.586 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.586 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198131 2024-09-20T22:07:25.586 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.586 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.586 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.586 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.586 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479927+0000", 2024-09-20T22:07:25.586 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.586 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.587 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.587 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.587 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479927+0000", 2024-09-20T22:07:25.587 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.587 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.587 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.587 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.587 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479929+0000", 2024-09-20T22:07:25.587 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5850000000000001e-06 2024-09-20T22:07:25.587 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.587 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.587 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.587 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479934+0000", 2024-09-20T22:07:25.587 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1869999999999996e-06 2024-09-20T22:07:25.588 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.588 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.588 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.588 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479955+0000", 2024-09-20T22:07:25.588 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1353e-05 2024-09-20T22:07:25.588 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.588 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.588 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.588 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479962+0000", 2024-09-20T22:07:25.588 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7549999999999997e-06 2024-09-20T22:07:25.588 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.588 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.588 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.588 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.480018+0000", 2024-09-20T22:07:25.588 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6342999999999999e-05 2024-09-20T22:07:25.588 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.589 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.589 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.589 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.480027+0000", 2024-09-20T22:07:25.589 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7469999999999994e-06 2024-09-20T22:07:25.589 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.589 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.589 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.589 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.433323+0000", 2024-09-20T22:07:25.589 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.95329631199999998 2024-09-20T22:07:25.589 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.589 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.589 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:25.589 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.433425+0000", 2024-09-20T22:07:25.589 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000101271 2024-09-20T22:07:25.589 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.590 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.590 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:25.590 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.433690+0000", 2024-09-20T22:07:25.590 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026574900000000002 2024-09-20T22:07:25.590 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.590 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.590 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:25.590 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.437320+0000", 2024-09-20T22:07:25.590 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003630075 2024-09-20T22:07:25.590 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.590 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.590 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.590 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.437363+0000", 2024-09-20T22:07:25.590 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2104000000000002e-05 2024-09-20T22:07:25.590 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.591 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.591 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.591 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.443794+0000", 2024-09-20T22:07:25.591 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0064310729999999998 2024-09-20T22:07:25.591 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.591 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.591 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:25.591 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.443902+0000", 2024-09-20T22:07:25.591 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010815 2024-09-20T22:07:25.591 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.591 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.591 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.591 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.444023+0000", 2024-09-20T22:07:25.591 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000120862 2024-09-20T22:07:25.591 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.592 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.592 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.592 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.592 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.592 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198163 7.10 7:086b2df9:::benchmark_data_smithi060_78800_object65130:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:07:25.592 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.501070+0000", 2024-09-20T22:07:25.592 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 59.891320862000001, 2024-09-20T22:07:25.592 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94904837799999997, 2024-09-20T22:07:25.592 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.592 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.592 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:25.592 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.592 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.592 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.592 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198163 2024-09-20T22:07:25.593 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.593 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.593 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.593 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.593 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501070+0000", 2024-09-20T22:07:25.593 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.593 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.593 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.593 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.593 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501070+0000", 2024-09-20T22:07:25.593 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.593 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.593 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.593 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.593 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501072+0000", 2024-09-20T22:07:25.593 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.742e-06 2024-09-20T22:07:25.594 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.594 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.594 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.594 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501078+0000", 2024-09-20T22:07:25.594 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3859999999999998e-06 2024-09-20T22:07:25.594 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.594 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.594 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.594 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501090+0000", 2024-09-20T22:07:25.594 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2003e-05 2024-09-20T22:07:25.594 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.594 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.594 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.594 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501097+0000", 2024-09-20T22:07:25.595 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2089999999999999e-06 2024-09-20T22:07:25.595 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.595 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.595 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.595 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501154+0000", 2024-09-20T22:07:25.595 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6901000000000003e-05 2024-09-20T22:07:25.595 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.595 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.595 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.595 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501162+0000", 2024-09-20T22:07:25.595 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3440000000000001e-06 2024-09-20T22:07:25.595 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.595 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.595 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.596 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435556+0000", 2024-09-20T22:07:25.596 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.93439354699999999 2024-09-20T22:07:25.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.596 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.596 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:25.596 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435672+0000", 2024-09-20T22:07:25.596 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011694 2024-09-20T22:07:25.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.596 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.596 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:25.596 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435963+0000", 2024-09-20T22:07:25.596 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000290654 2024-09-20T22:07:25.596 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.596 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.596 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:25.596 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.440444+0000", 2024-09-20T22:07:25.597 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0044806450000000001 2024-09-20T22:07:25.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.597 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.597 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.597 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.440486+0000", 2024-09-20T22:07:25.597 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2036999999999997e-05 2024-09-20T22:07:25.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.597 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.597 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.597 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.449968+0000", 2024-09-20T22:07:25.597 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0094824560000000002 2024-09-20T22:07:25.597 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.597 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.597 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:25.598 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450024+0000", 2024-09-20T22:07:25.598 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5374000000000002e-05 2024-09-20T22:07:25.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.598 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.598 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.598 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450119+0000", 2024-09-20T22:07:25.598 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5140000000000001e-05 2024-09-20T22:07:25.598 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.598 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.598 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.598 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.598 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.598 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198198 7.10 7:0c0c59e3:::benchmark_data_smithi060_78800_object65165:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:07:25.598 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.527682+0000", 2024-09-20T22:07:25.599 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 59.864708884999999, 2024-09-20T22:07:25.599 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.92260386900000002, 2024-09-20T22:07:25.599 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.599 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.599 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:25.599 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.599 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.599 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.599 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198198 2024-09-20T22:07:25.599 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.599 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.602 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.602 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.602 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.527682+0000", 2024-09-20T22:07:25.602 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.602 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.602 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.602 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.527682+0000", 2024-09-20T22:07:25.602 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.602 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.602 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.603 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.527684+0000", 2024-09-20T22:07:25.603 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.423e-06 2024-09-20T22:07:25.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.603 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.603 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.603 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.527689+0000", 2024-09-20T22:07:25.603 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.259e-06 2024-09-20T22:07:25.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.603 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.603 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.603 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.527702+0000", 2024-09-20T22:07:25.603 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2738000000000001e-05 2024-09-20T22:07:25.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.603 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.527727+0000", 2024-09-20T22:07:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5088000000000001e-05 2024-09-20T22:07:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.527787+0000", 2024-09-20T22:07:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0043000000000001e-05 2024-09-20T22:07:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.527795+0000", 2024-09-20T22:07:25.604 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4209999999999995e-06 2024-09-20T22:07:25.605 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.605 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.605 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.605 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.437388+0000", 2024-09-20T22:07:25.605 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90959268900000001 2024-09-20T22:07:25.605 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.605 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.605 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:25.605 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.437497+0000", 2024-09-20T22:07:25.605 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010881400000000001 2024-09-20T22:07:25.605 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.605 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.605 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:25.605 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.437727+0000", 2024-09-20T22:07:25.606 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022985 2024-09-20T22:07:25.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.606 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.606 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:25.606 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.440498+0000", 2024-09-20T22:07:25.606 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002771267 2024-09-20T22:07:25.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.606 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.606 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.606 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.444097+0000", 2024-09-20T22:07:25.606 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0035986540000000002 2024-09-20T22:07:25.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.606 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.606 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.607 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450156+0000", 2024-09-20T22:07:25.607 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.006059699 2024-09-20T22:07:25.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.607 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:25.607 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450203+0000", 2024-09-20T22:07:25.607 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6325000000000001e-05 2024-09-20T22:07:25.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.607 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.607 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450286+0000", 2024-09-20T22:07:25.607 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3598999999999997e-05 2024-09-20T22:07:25.607 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.608 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.608 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.608 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.608 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198259 7.10 7:0ad0ad2c:::benchmark_data_smithi060_78800_object65226:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:07:25.608 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.583584+0000", 2024-09-20T22:07:25.608 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 59.808807492, 2024-09-20T22:07:25.608 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.86690348100000003, 2024-09-20T22:07:25.608 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.608 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.608 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:25.608 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.608 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.608 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.608 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198259 2024-09-20T22:07:25.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.609 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.609 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.609 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.583584+0000", 2024-09-20T22:07:25.609 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.609 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.609 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.583584+0000", 2024-09-20T22:07:25.609 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.610 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.610 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.583585+0000", 2024-09-20T22:07:25.610 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4780000000000001e-06 2024-09-20T22:07:25.610 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.610 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.610 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.610 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.583591+0000", 2024-09-20T22:07:25.610 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9719999999999997e-06 2024-09-20T22:07:25.610 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.610 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.610 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.610 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.583603+0000", 2024-09-20T22:07:25.610 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2211999999999999e-05 2024-09-20T22:07:25.611 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.611 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.611 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.611 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.583619+0000", 2024-09-20T22:07:25.611 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5098e-05 2024-09-20T22:07:25.611 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.611 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.611 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.611 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.583677+0000", 2024-09-20T22:07:25.611 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8724999999999998e-05 2024-09-20T22:07:25.611 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.611 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.611 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.612 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.583686+0000", 2024-09-20T22:07:25.612 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3319999999999992e-06 2024-09-20T22:07:25.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.612 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.612 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.612 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.439048+0000", 2024-09-20T22:07:25.612 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85536227600000003 2024-09-20T22:07:25.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.612 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.612 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:25.612 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.439148+0000", 2024-09-20T22:07:25.612 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9788000000000001e-05 2024-09-20T22:07:25.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.612 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.613 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:25.613 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.439365+0000", 2024-09-20T22:07:25.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000217269 2024-09-20T22:07:25.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.613 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.613 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:25.613 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.443753+0000", 2024-09-20T22:07:25.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0043879169999999999 2024-09-20T22:07:25.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.613 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.613 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.613 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.449854+0000", 2024-09-20T22:07:25.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0061015959999999999 2024-09-20T22:07:25.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.614 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.614 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.614 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450358+0000", 2024-09-20T22:07:25.614 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00050349100000000001 2024-09-20T22:07:25.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.614 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.614 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:25.614 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450404+0000", 2024-09-20T22:07:25.614 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5806000000000002e-05 2024-09-20T22:07:25.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.614 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.614 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.614 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450487+0000", 2024-09-20T22:07:25.614 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3521000000000002e-05 2024-09-20T22:07:25.615 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.615 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.615 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.615 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.615 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.615 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198351 7.10 7:0faceefa:::benchmark_data_smithi060_78800_object65318:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:07:25.615 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.655692+0000", 2024-09-20T22:07:25.615 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 59.736699191, 2024-09-20T22:07:25.615 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.79496427599999997, 2024-09-20T22:07:25.615 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.615 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.615 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:25.615 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.615 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.616 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.616 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198351 2024-09-20T22:07:25.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.616 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.616 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.616 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.616 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.655692+0000", 2024-09-20T22:07:25.616 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.616 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.616 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.616 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.655692+0000", 2024-09-20T22:07:25.616 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.617 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.617 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.655693+0000", 2024-09-20T22:07:25.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3599999999999999e-06 2024-09-20T22:07:25.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.617 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.617 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.655699+0000", 2024-09-20T22:07:25.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3419999999999998e-06 2024-09-20T22:07:25.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.617 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.617 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.655711+0000", 2024-09-20T22:07:25.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2448e-05 2024-09-20T22:07:25.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.618 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.618 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.618 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.655729+0000", 2024-09-20T22:07:25.618 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8014000000000001e-05 2024-09-20T22:07:25.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.618 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.618 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.618 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.655792+0000", 2024-09-20T22:07:25.618 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2817999999999994e-05 2024-09-20T22:07:25.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.618 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.618 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.618 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.655800+0000", 2024-09-20T22:07:25.618 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0800000000000006e-06 2024-09-20T22:07:25.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.619 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.619 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.619 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.440525+0000", 2024-09-20T22:07:25.619 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.78472440200000004 2024-09-20T22:07:25.619 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.619 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.619 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:25.619 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.440622+0000", 2024-09-20T22:07:25.619 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7503999999999993e-05 2024-09-20T22:07:25.619 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.619 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.619 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:25.619 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.440832+0000", 2024-09-20T22:07:25.619 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020958199999999999 2024-09-20T22:07:25.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.620 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:25.620 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.444056+0000", 2024-09-20T22:07:25.620 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0032243939999999998 2024-09-20T22:07:25.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.620 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.620 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.449896+0000", 2024-09-20T22:07:25.620 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005840273 2024-09-20T22:07:25.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.620 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.620 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450523+0000", 2024-09-20T22:07:25.620 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00062680600000000004 2024-09-20T22:07:25.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.621 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.621 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:25.621 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450572+0000", 2024-09-20T22:07:25.621 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9317999999999998e-05 2024-09-20T22:07:25.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.621 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.621 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.621 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450656+0000", 2024-09-20T22:07:25.621 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3935000000000002e-05 2024-09-20T22:07:25.621 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.621 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.621 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.621 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.622 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227634 7.10 7:0ea5dec3:::benchmark_data_smithi060_78800_object94601:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:07:25.622 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.846865+0000", 2024-09-20T22:07:25.622 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 31.545526035999998, 2024-09-20T22:07:25.622 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1004788329999999, 2024-09-20T22:07:25.622 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.622 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.622 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:25.622 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.622 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.622 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.622 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227634 2024-09-20T22:07:25.622 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.622 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.622 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.623 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.623 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846865+0000", 2024-09-20T22:07:25.623 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.623 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.623 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.623 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846865+0000", 2024-09-20T22:07:25.623 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.623 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.623 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.623 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846867+0000", 2024-09-20T22:07:25.624 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.641e-06 2024-09-20T22:07:25.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.624 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.624 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.624 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846873+0000", 2024-09-20T22:07:25.624 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1489999999999999e-06 2024-09-20T22:07:25.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.624 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.624 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.624 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846887+0000", 2024-09-20T22:07:25.624 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4113e-05 2024-09-20T22:07:25.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.624 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.625 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.625 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846895+0000", 2024-09-20T22:07:25.625 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6699999999999994e-06 2024-09-20T22:07:25.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.625 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.625 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.852299+0000", 2024-09-20T22:07:25.625 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0054038059999999997 2024-09-20T22:07:25.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.625 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.625 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.852311+0000", 2024-09-20T22:07:25.625 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2278e-05 2024-09-20T22:07:25.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.626 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.626 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.941457+0000", 2024-09-20T22:07:25.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0891458009999999 2024-09-20T22:07:25.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.626 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:25.626 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.941594+0000", 2024-09-20T22:07:25.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013708000000000001 2024-09-20T22:07:25.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.626 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2,5", 2024-09-20T22:07:25.626 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.941949+0000", 2024-09-20T22:07:25.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00035502599999999999 2024-09-20T22:07:25.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.627 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946263+0000", 2024-09-20T22:07:25.627 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0043141150000000003 2024-09-20T22:07:25.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:25.627 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946286+0000", 2024-09-20T22:07:25.627 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3419000000000001e-05 2024-09-20T22:07:25.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.627 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946351+0000", 2024-09-20T22:07:25.627 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4231000000000002e-05 2024-09-20T22:07:25.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.628 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.628 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947173+0000", 2024-09-20T22:07:25.628 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00082238199999999997 2024-09-20T22:07:25.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.628 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:25.628 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947237+0000", 2024-09-20T22:07:25.628 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4462999999999996e-05 2024-09-20T22:07:25.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.628 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.628 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947344+0000", 2024-09-20T22:07:25.628 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000106659 2024-09-20T22:07:25.629 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.629 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.629 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.629 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.629 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227639 7.10 7:0834985a:::benchmark_data_smithi060_78800_object94606:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:07:25.629 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.856347+0000", 2024-09-20T22:07:25.629 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 31.536044473, 2024-09-20T22:07:25.629 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0915877439999999, 2024-09-20T22:07:25.629 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.629 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.629 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:25.629 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.629 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.630 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.630 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227639 2024-09-20T22:07:25.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.630 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.630 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.630 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856347+0000", 2024-09-20T22:07:25.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.630 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.630 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856347+0000", 2024-09-20T22:07:25.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856349+0000", 2024-09-20T22:07:25.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7090000000000001e-06 2024-09-20T22:07:25.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856355+0000", 2024-09-20T22:07:25.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4729999999999997e-06 2024-09-20T22:07:25.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856369+0000", 2024-09-20T22:07:25.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3832000000000001e-05 2024-09-20T22:07:25.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856378+0000", 2024-09-20T22:07:25.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8689999999999993e-06 2024-09-20T22:07:25.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856556+0000", 2024-09-20T22:07:25.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000178529 2024-09-20T22:07:25.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.633 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856568+0000", 2024-09-20T22:07:25.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2281e-05 2024-09-20T22:07:25.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.633 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.633 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.633 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.944093+0000", 2024-09-20T22:07:25.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.087524438 2024-09-20T22:07:25.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.633 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.633 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:25.633 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.944232+0000", 2024-09-20T22:07:25.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000139178 2024-09-20T22:07:25.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.633 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.633 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2,5", 2024-09-20T22:07:25.634 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.944533+0000", 2024-09-20T22:07:25.634 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030114999999999997 2024-09-20T22:07:25.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.634 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.634 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.634 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946726+0000", 2024-09-20T22:07:25.634 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021925809999999999 2024-09-20T22:07:25.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.634 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.634 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:25.634 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946797+0000", 2024-09-20T22:07:25.634 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1570999999999996e-05 2024-09-20T22:07:25.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.635 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.635 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947394+0000", 2024-09-20T22:07:25.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00059673799999999998 2024-09-20T22:07:25.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.635 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.635 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947773+0000", 2024-09-20T22:07:25.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00037865499999999999 2024-09-20T22:07:25.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.635 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:25.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947831+0000", 2024-09-20T22:07:25.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8557000000000002e-05 2024-09-20T22:07:25.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947935+0000", 2024-09-20T22:07:25.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103183 2024-09-20T22:07:25.636 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.636 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.636 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.636 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227649 7.2 7:46c04214:::benchmark_data_smithi060_78800_object94616:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:07:25.637 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.900917+0000", 2024-09-20T22:07:25.637 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 31.491474428, 2024-09-20T22:07:25.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1912150110000002, 2024-09-20T22:07:25.637 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.637 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.637 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:25.637 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.637 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.637 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.637 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227649 2024-09-20T22:07:25.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.637 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.638 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900917+0000", 2024-09-20T22:07:25.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.638 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.638 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900917+0000", 2024-09-20T22:07:25.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.638 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.638 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900919+0000", 2024-09-20T22:07:25.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7400000000000001e-06 2024-09-20T22:07:25.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900924+0000", 2024-09-20T22:07:25.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6439999999999997e-06 2024-09-20T22:07:25.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900941+0000", 2024-09-20T22:07:25.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7244e-05 2024-09-20T22:07:25.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.640 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.640 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900949+0000", 2024-09-20T22:07:25.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0829999999999998e-06 2024-09-20T22:07:25.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.640 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.640 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.901025+0000", 2024-09-20T22:07:25.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6643999999999993e-05 2024-09-20T22:07:25.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.640 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.640 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.901037+0000", 2024-09-20T22:07:25.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1824999999999999e-05 2024-09-20T22:07:25.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.641 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.641 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.817605+0000", 2024-09-20T22:07:25.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91656772500000006 2024-09-20T22:07:25.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.641 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.641 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.817617+0000", 2024-09-20T22:07:25.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2320999999999999e-05 2024-09-20T22:07:25.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.642 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.642 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.067011+0000", 2024-09-20T22:07:25.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2493934259999999 2024-09-20T22:07:25.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.642 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:25.642 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.067109+0000", 2024-09-20T22:07:25.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8904e-05 2024-09-20T22:07:25.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.642 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:07:25.642 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.067424+0000", 2024-09-20T22:07:25.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00031460400000000001 2024-09-20T22:07:25.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:25.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.069092+0000", 2024-09-20T22:07:25.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001667836 2024-09-20T22:07:25.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.070743+0000", 2024-09-20T22:07:25.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001650763 2024-09-20T22:07:25.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.091949+0000", 2024-09-20T22:07:25.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021206434999999999 2024-09-20T22:07:25.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:25.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092012+0000", 2024-09-20T22:07:25.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2651999999999996e-05 2024-09-20T22:07:25.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092132+0000", 2024-09-20T22:07:25.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000120165 2024-09-20T22:07:25.644 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.644 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.644 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.645 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227675 7.2 7:45bb0f01:::benchmark_data_smithi060_78800_object94642:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-20T22:07:25.645 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:54.993128+0000", 2024-09-20T22:07:25.645 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 30.399263196, 2024-09-20T22:07:25.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.09956863, 2024-09-20T22:07:25.645 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.645 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.645 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:25.645 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.645 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.645 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.645 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227675 2024-09-20T22:07:25.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.645 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993128+0000", 2024-09-20T22:07:25.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993128+0000", 2024-09-20T22:07:25.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993130+0000", 2024-09-20T22:07:25.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5400000000000001e-06 2024-09-20T22:07:25.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993136+0000", 2024-09-20T22:07:25.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.669e-06 2024-09-20T22:07:25.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993153+0000", 2024-09-20T22:07:25.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6538999999999999e-05 2024-09-20T22:07:25.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993162+0000", 2024-09-20T22:07:25.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2529999999999993e-06 2024-09-20T22:07:25.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993239+0000", 2024-09-20T22:07:25.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6499000000000001e-05 2024-09-20T22:07:25.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993253+0000", 2024-09-20T22:07:25.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4278e-05 2024-09-20T22:07:25.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.069151+0000", 2024-09-20T22:07:25.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0758977059999999 2024-09-20T22:07:25.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:25.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.069260+0000", 2024-09-20T22:07:25.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109175 2024-09-20T22:07:25.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:07:25.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.069491+0000", 2024-09-20T22:07:25.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000231716 2024-09-20T22:07:25.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:25.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.070761+0000", 2024-09-20T22:07:25.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012690850000000001 2024-09-20T22:07:25.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.071295+0000", 2024-09-20T22:07:25.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00053413799999999997 2024-09-20T22:07:25.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092555+0000", 2024-09-20T22:07:25.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021260233 2024-09-20T22:07:25.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:25.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092609+0000", 2024-09-20T22:07:25.651 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4147999999999997e-05 2024-09-20T22:07:25.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.651 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.651 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092697+0000", 2024-09-20T22:07:25.651 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7651000000000005e-05 2024-09-20T22:07:25.651 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.651 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.651 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.651 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236464 7.2 7:41e7f20e:::benchmark_data_smithi060_78800_object103431:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:07:25.651 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.420278+0000", 2024-09-20T22:07:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 17.972113178000001, 2024-09-20T22:07:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99395603799999999, 2024-09-20T22:07:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236464 2024-09-20T22:07:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420278+0000", 2024-09-20T22:07:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420278+0000", 2024-09-20T22:07:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420280+0000", 2024-09-20T22:07:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0049999999999999e-06 2024-09-20T22:07:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420287+0000", 2024-09-20T22:07:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7429999999999996e-06 2024-09-20T22:07:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420305+0000", 2024-09-20T22:07:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.836e-05 2024-09-20T22:07:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420314+0000", 2024-09-20T22:07:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7329999999999992e-06 2024-09-20T22:07:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.421421+0000", 2024-09-20T22:07:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001107455 2024-09-20T22:07:25.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.421429+0000", 2024-09-20T22:07:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5669999999999997e-06 2024-09-20T22:07:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.405308+0000", 2024-09-20T22:07:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98387922900000002 2024-09-20T22:07:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.405428+0000", 2024-09-20T22:07:25.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012023699999999999 2024-09-20T22:07:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:07:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.405861+0000", 2024-09-20T22:07:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00043245600000000001 2024-09-20T22:07:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414023+0000", 2024-09-20T22:07:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0081622280000000005 2024-09-20T22:07:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414092+0000", 2024-09-20T22:07:25.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9129999999999997e-05 2024-09-20T22:07:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414141+0000", 2024-09-20T22:07:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8426e-05 2024-09-20T22:07:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414196+0000", 2024-09-20T22:07:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5226999999999998e-05 2024-09-20T22:07:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414234+0000", 2024-09-20T22:07:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8241999999999997e-05 2024-09-20T22:07:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236481 7.2 7:417f7fb8:::benchmark_data_smithi060_78800_object103448:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:07:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.436984+0000", 2024-09-20T22:07:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 17.955406958000001, 2024-09-20T22:07:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98983631900000002, 2024-09-20T22:07:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.658 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236481 2024-09-20T22:07:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.436984+0000", 2024-09-20T22:07:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.436984+0000", 2024-09-20T22:07:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.436985+0000", 2024-09-20T22:07:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-07 2024-09-20T22:07:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.436990+0000", 2024-09-20T22:07:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7770000000000002e-06 2024-09-20T22:07:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.437004+0000", 2024-09-20T22:07:25.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3572e-05 2024-09-20T22:07:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.437010+0000", 2024-09-20T22:07:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9469999999999997e-06 2024-09-20T22:07:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.439285+0000", 2024-09-20T22:07:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022742539999999999 2024-09-20T22:07:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.439294+0000", 2024-09-20T22:07:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0720000000000006e-06 2024-09-20T22:07:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.408421+0000", 2024-09-20T22:07:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.96912711200000001 2024-09-20T22:07:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.408517+0000", 2024-09-20T22:07:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.624e-05 2024-09-20T22:07:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:07:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.408884+0000", 2024-09-20T22:07:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00036694800000000002 2024-09-20T22:07:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414277+0000", 2024-09-20T22:07:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0053932440000000002 2024-09-20T22:07:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.419733+0000", 2024-09-20T22:07:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0054556500000000003 2024-09-20T22:07:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426386+0000", 2024-09-20T22:07:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0066532680000000004 2024-09-20T22:07:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426769+0000", 2024-09-20T22:07:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000382347 2024-09-20T22:07:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426821+0000", 2024-09-20T22:07:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2008000000000001e-05 2024-09-20T22:07:25.664 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236521 7.2 7:400e2278:::benchmark_data_smithi060_78800_object103488:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:07:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.486226+0000", 2024-09-20T22:07:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 17.906165189999999, 2024-09-20T22:07:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94626404099999994, 2024-09-20T22:07:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.665 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236521 2024-09-20T22:07:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.486226+0000", 2024-09-20T22:07:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.486226+0000", 2024-09-20T22:07:25.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.486228+0000", 2024-09-20T22:07:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0090000000000002e-06 2024-09-20T22:07:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.486235+0000", 2024-09-20T22:07:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.55e-06 2024-09-20T22:07:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.486249+0000", 2024-09-20T22:07:25.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3993e-05 2024-09-20T22:07:25.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.486256+0000", 2024-09-20T22:07:25.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7829999999999999e-06 2024-09-20T22:07:25.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.487970+0000", 2024-09-20T22:07:25.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017137630000000001 2024-09-20T22:07:25.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.487977+0000", 2024-09-20T22:07:25.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0319999999999998e-06 2024-09-20T22:07:25.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.411021+0000", 2024-09-20T22:07:25.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.92304408900000001 2024-09-20T22:07:25.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:25.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.411137+0000", 2024-09-20T22:07:25.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115716 2024-09-20T22:07:25.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:07:25.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.411662+0000", 2024-09-20T22:07:25.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00052523800000000003 2024-09-20T22:07:25.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.419783+0000", 2024-09-20T22:07:25.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0081203500000000001 2024-09-20T22:07:25.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:25.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426834+0000", 2024-09-20T22:07:25.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.007051163 2024-09-20T22:07:25.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432386+0000", 2024-09-20T22:07:25.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0055518499999999997 2024-09-20T22:07:25.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:25.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432445+0000", 2024-09-20T22:07:25.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8984999999999999e-05 2024-09-20T22:07:25.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432490+0000", 2024-09-20T22:07:25.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5519999999999998e-05 2024-09-20T22:07:25.671 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.671 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.672 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.672 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236528 7.2 7:47f1bc45:::benchmark_data_smithi060_78800_object103495:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:07:25.672 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.491029+0000", 2024-09-20T22:07:25.672 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 17.901362138, 2024-09-20T22:07:25.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.941676018, 2024-09-20T22:07:25.672 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.672 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.672 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:25.672 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.672 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.672 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.672 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236528 2024-09-20T22:07:25.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.673 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.491029+0000", 2024-09-20T22:07:25.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.491029+0000", 2024-09-20T22:07:25.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.491030+0000", 2024-09-20T22:07:25.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2300000000000001e-06 2024-09-20T22:07:25.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.491035+0000", 2024-09-20T22:07:25.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4800000000000003e-06 2024-09-20T22:07:25.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.491044+0000", 2024-09-20T22:07:25.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6780000000000006e-06 2024-09-20T22:07:25.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.491048+0000", 2024-09-20T22:07:25.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7940000000000002e-06 2024-09-20T22:07:25.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.491102+0000", 2024-09-20T22:07:25.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4011999999999997e-05 2024-09-20T22:07:25.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.491111+0000", 2024-09-20T22:07:25.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3639999999999999e-06 2024-09-20T22:07:25.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414308+0000", 2024-09-20T22:07:25.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.92319775000000004 2024-09-20T22:07:25.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:25.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414417+0000", 2024-09-20T22:07:25.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108238 2024-09-20T22:07:25.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:07:25.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414770+0000", 2024-09-20T22:07:25.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00035358200000000002 2024-09-20T22:07:25.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.419827+0000", 2024-09-20T22:07:25.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0050570479999999998 2024-09-20T22:07:25.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:25.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426890+0000", 2024-09-20T22:07:25.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0070631629999999999 2024-09-20T22:07:25.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432621+0000", 2024-09-20T22:07:25.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0057304740000000002 2024-09-20T22:07:25.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:25.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432667+0000", 2024-09-20T22:07:25.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5770999999999999e-05 2024-09-20T22:07:25.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432705+0000", 2024-09-20T22:07:25.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8433999999999998e-05 2024-09-20T22:07:25.678 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.678 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.678 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.679 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236532 7.2 7:46848d74:::benchmark_data_smithi060_78800_object103499:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:07:25.679 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.496306+0000", 2024-09-20T22:07:25.679 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 17.896085148000001, 2024-09-20T22:07:25.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.93652918600000001, 2024-09-20T22:07:25.679 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.679 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.679 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:25.679 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.679 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.679 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.679 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236532 2024-09-20T22:07:25.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.679 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.680 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.496306+0000", 2024-09-20T22:07:25.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.680 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.496306+0000", 2024-09-20T22:07:25.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.680 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.496307+0000", 2024-09-20T22:07:25.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2279999999999999e-06 2024-09-20T22:07:25.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.496313+0000", 2024-09-20T22:07:25.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4330000000000003e-06 2024-09-20T22:07:25.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.496329+0000", 2024-09-20T22:07:25.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6271000000000001e-05 2024-09-20T22:07:25.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.496336+0000", 2024-09-20T22:07:25.682 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1770000000000001e-06 2024-09-20T22:07:25.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.682 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.682 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.496443+0000", 2024-09-20T22:07:25.682 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010645999999999999 2024-09-20T22:07:25.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.682 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.682 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.496451+0000", 2024-09-20T22:07:25.682 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8790000000000003e-06 2024-09-20T22:07:25.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.682 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.416957+0000", 2024-09-20T22:07:25.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.92050670800000001 2024-09-20T22:07:25.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:25.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.417078+0000", 2024-09-20T22:07:25.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012116300000000001 2024-09-20T22:07:25.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:07:25.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.417454+0000", 2024-09-20T22:07:25.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00037505600000000002 2024-09-20T22:07:25.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426372+0000", 2024-09-20T22:07:25.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0089183890000000005 2024-09-20T22:07:25.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:25.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426896+0000", 2024-09-20T22:07:25.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00052363099999999999 2024-09-20T22:07:25.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432754+0000", 2024-09-20T22:07:25.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0058583769999999997 2024-09-20T22:07:25.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:25.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432800+0000", 2024-09-20T22:07:25.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5955999999999998e-05 2024-09-20T22:07:25.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432835+0000", 2024-09-20T22:07:25.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5457999999999998e-05 2024-09-20T22:07:25.685 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.685 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.685 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.685 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236566 7.2 7:4368a0c6:::benchmark_data_smithi060_78800_object103533:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:07:25.686 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.564354+0000", 2024-09-20T22:07:25.686 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 17.828036869999998, 2024-09-20T22:07:25.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.86859894900000001, 2024-09-20T22:07:25.686 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.686 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.686 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:25.686 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.686 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.686 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.686 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236566 2024-09-20T22:07:25.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.686 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.564354+0000", 2024-09-20T22:07:25.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.564354+0000", 2024-09-20T22:07:25.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.564356+0000", 2024-09-20T22:07:25.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4100000000000001e-06 2024-09-20T22:07:25.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.564362+0000", 2024-09-20T22:07:25.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6789999999999999e-06 2024-09-20T22:07:25.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.564377+0000", 2024-09-20T22:07:25.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4226000000000001e-05 2024-09-20T22:07:25.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.564385+0000", 2024-09-20T22:07:25.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8699999999999992e-06 2024-09-20T22:07:25.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.564447+0000", 2024-09-20T22:07:25.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1966999999999998e-05 2024-09-20T22:07:25.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.564455+0000", 2024-09-20T22:07:25.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-20T22:07:25.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.419861+0000", 2024-09-20T22:07:25.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85540531399999997 2024-09-20T22:07:25.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:25.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.419968+0000", 2024-09-20T22:07:25.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000107507 2024-09-20T22:07:25.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:07:25.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.420312+0000", 2024-09-20T22:07:25.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00034424199999999997 2024-09-20T22:07:25.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426878+0000", 2024-09-20T22:07:25.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0065659409999999996 2024-09-20T22:07:25.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:25.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426900+0000", 2024-09-20T22:07:25.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1934999999999999e-05 2024-09-20T22:07:25.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432876+0000", 2024-09-20T22:07:25.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0059759380000000001 2024-09-20T22:07:25.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:25.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432919+0000", 2024-09-20T22:07:25.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3010000000000003e-05 2024-09-20T22:07:25.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432953+0000", 2024-09-20T22:07:25.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4109999999999997e-05 2024-09-20T22:07:25.692 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.692 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.692 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.693 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236574 7.2 7:47f87b9b:::benchmark_data_smithi060_78800_object103541:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:07:25.693 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.580098+0000", 2024-09-20T22:07:25.693 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 17.812293646000001, 2024-09-20T22:07:25.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85343583899999997, 2024-09-20T22:07:25.693 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.693 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.693 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:25.693 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.693 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.693 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.693 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236574 2024-09-20T22:07:25.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.693 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.580098+0000", 2024-09-20T22:07:25.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.580098+0000", 2024-09-20T22:07:25.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.580099+0000", 2024-09-20T22:07:25.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.643e-06 2024-09-20T22:07:25.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.580106+0000", 2024-09-20T22:07:25.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4609999999999996e-06 2024-09-20T22:07:25.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.580119+0000", 2024-09-20T22:07:25.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3094e-05 2024-09-20T22:07:25.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.580127+0000", 2024-09-20T22:07:25.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2740000000000007e-06 2024-09-20T22:07:25.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.580201+0000", 2024-09-20T22:07:25.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3867000000000003e-05 2024-09-20T22:07:25.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.580211+0000", 2024-09-20T22:07:25.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0225e-05 2024-09-20T22:07:25.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.422840+0000", 2024-09-20T22:07:25.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.84262903099999997 2024-09-20T22:07:25.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:25.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.422949+0000", 2024-09-20T22:07:25.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108801 2024-09-20T22:07:25.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:07:25.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.423319+0000", 2024-09-20T22:07:25.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00036967100000000002 2024-09-20T22:07:25.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:25.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426904+0000", 2024-09-20T22:07:25.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00358553 2024-09-20T22:07:25.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432578+0000", 2024-09-20T22:07:25.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005673984 2024-09-20T22:07:25.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.433451+0000", 2024-09-20T22:07:25.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00087312200000000005 2024-09-20T22:07:25.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:25.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.433497+0000", 2024-09-20T22:07:25.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5547000000000003e-05 2024-09-20T22:07:25.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.433533+0000", 2024-09-20T22:07:25.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6588999999999998e-05 2024-09-20T22:07:25.699 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.699 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.700 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.700 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236582 7.2 7:444ff2c7:::benchmark_data_smithi060_78800_object103549:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:07:25.700 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.613480+0000", 2024-09-20T22:07:25.700 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 17.778911083000001, 2024-09-20T22:07:25.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.82262310000000005, 2024-09-20T22:07:25.700 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:07:25.700 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:07:25.700 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:25.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:07:25.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:07:25.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:25.701 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236582 2024-09-20T22:07:25.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.701 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:07:25.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:07:25.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.613480+0000", 2024-09-20T22:07:25.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:07:25.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.613480+0000", 2024-09-20T22:07:25.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:07:25.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:07:25.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.613482+0000", 2024-09-20T22:07:25.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3319999999999999e-06 2024-09-20T22:07:25.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:07:25.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.613486+0000", 2024-09-20T22:07:25.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.848e-06 2024-09-20T22:07:25.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:07:25.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.613500+0000", 2024-09-20T22:07:25.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3609e-05 2024-09-20T22:07:25.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:07:25.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.613505+0000", 2024-09-20T22:07:25.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4970000000000001e-06 2024-09-20T22:07:25.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.613567+0000", 2024-09-20T22:07:25.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1502999999999997e-05 2024-09-20T22:07:25.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:07:25.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.613577+0000", 2024-09-20T22:07:25.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0372e-05 2024-09-20T22:07:25.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:07:25.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.429648+0000", 2024-09-20T22:07:25.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.81607059699999995 2024-09-20T22:07:25.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:07:25.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.429792+0000", 2024-09-20T22:07:25.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014414699999999999 2024-09-20T22:07:25.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:07:25.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.430189+0000", 2024-09-20T22:07:25.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00039722899999999997 2024-09-20T22:07:25.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.434069+0000", 2024-09-20T22:07:25.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038798769999999999 2024-09-20T22:07:25.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:07:25.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.434144+0000", 2024-09-20T22:07:25.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4635000000000006e-05 2024-09-20T22:07:25.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:25.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.436012+0000", 2024-09-20T22:07:25.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0018679300000000001 2024-09-20T22:07:25.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:07:25.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.436065+0000", 2024-09-20T22:07:25.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2753999999999997e-05 2024-09-20T22:07:25.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:07:25.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:07:25.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:07:25.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.436103+0000", 2024-09-20T22:07:25.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8770000000000003e-05 2024-09-20T22:07:25.706 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.706 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.707 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.707 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:07:25.707 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:07:25.707 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:07:25.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:25.708 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:25.708 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:25.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:25 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:07:25.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:25 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:07:25.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:25 smithi060 ceph-mon[29694]: 5.0 scrub starts 2024-09-20T22:07:25.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:25 smithi060 ceph-mon[29694]: 5.0 scrub ok 2024-09-20T22:07:25.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:25 smithi060 ceph-mon[29694]: osdmap e163: 8 total, 8 up, 8 in 2024-09-20T22:07:25.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:25 smithi060 ceph-mon[29694]: 4.a scrub starts 2024-09-20T22:07:25.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:25 smithi060 ceph-mon[29694]: 4.a scrub ok 2024-09-20T22:07:25.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:25 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:07:25.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:25 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:07:25.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:25 smithi060 ceph-mon[40290]: 5.0 scrub starts 2024-09-20T22:07:25.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:25 smithi060 ceph-mon[40290]: 5.0 scrub ok 2024-09-20T22:07:25.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:25 smithi060 ceph-mon[40290]: osdmap e163: 8 total, 8 up, 8 in 2024-09-20T22:07:25.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:25 smithi060 ceph-mon[40290]: 4.a scrub starts 2024-09-20T22:07:25.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:25 smithi060 ceph-mon[40290]: 4.a scrub ok 2024-09-20T22:07:25.809 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:25.837 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:25.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:25.838 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:25.939 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:25.986 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:25.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:25.987 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:26.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:25 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:07:26.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:25 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:07:26.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:25 smithi067 ceph-mon[33511]: 5.0 scrub starts 2024-09-20T22:07:26.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:25 smithi067 ceph-mon[33511]: 5.0 scrub ok 2024-09-20T22:07:26.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:25 smithi067 ceph-mon[33511]: osdmap e163: 8 total, 8 up, 8 in 2024-09-20T22:07:26.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:25 smithi067 ceph-mon[33511]: 4.a scrub starts 2024-09-20T22:07:26.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:25 smithi067 ceph-mon[33511]: 4.a scrub ok 2024-09-20T22:07:26.087 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:26.129 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:26.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:26.131 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:26.232 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:26.262 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:26.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:26.263 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:26.356 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:07:26.363 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:26.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:26.468 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:26.468 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:26.503 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:07:26.503 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:07:26.542 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:07:26.543 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:07:26.568 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:26.585 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:07:26.585 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:07:26.597 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:26.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:26.598 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:26.640 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:07:26.640 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:07:26.683 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:07:26.683 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:07:26.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:26.715 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:07:26.715 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:07:26.728 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:26.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:26.729 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:26.753 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:07:26.753 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:07:26.788 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:07:26.789 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:07:26.830 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:26.877 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:26.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:26.878 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:26.978 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:27.010 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:27.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:27.011 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:27.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:26 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/784297564' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:07:27.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:26 smithi067 ceph-mon[33511]: pgmap v216: 158 pgs: 2 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 2 active+remapped+backfill_toofull, 152 active+clean; 942 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 1093/45795 objects degraded (2.387%); 2684/45795 objects misplaced (5.861%) 2024-09-20T22:07:27.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:26 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 1518/55395 objects degraded (2.740%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T22:07:27.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:26 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1108152664' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:26 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/784297564' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:07:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:26 smithi060 ceph-mon[29694]: pgmap v216: 158 pgs: 2 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 2 active+remapped+backfill_toofull, 152 active+clean; 942 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 1093/45795 objects degraded (2.387%); 2684/45795 objects misplaced (5.861%) 2024-09-20T22:07:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:26 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 1518/55395 objects degraded (2.740%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T22:07:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:26 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1108152664' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:27.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:26 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/784297564' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:07:27.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:26 smithi060 ceph-mon[40290]: pgmap v216: 158 pgs: 2 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 2 active+remapped+backfill_toofull, 152 active+clean; 942 MiB data, 4.5 GiB used, 711 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 1093/45795 objects degraded (2.387%); 2684/45795 objects misplaced (5.861%) 2024-09-20T22:07:27.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:26 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 1518/55395 objects degraded (2.740%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T22:07:27.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:26 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1108152664' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:27.112 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:27.140 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:27.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:27.141 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:27.145 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T22:07:27.242 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:27.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:27.276 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:27.276 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:27.377 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:27.407 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:27.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:27.408 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:27.473 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.2 to 1 (10000) 2024-09-20T22:07:27.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:27.547 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:27.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:27.548 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:27.586 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:07:27.649 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:27.679 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:27.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:27.680 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:27.781 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:27.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:27 smithi060 ceph-mon[29694]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:07:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:27 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/784297564' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:07:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:27 smithi060 ceph-mon[29694]: osdmap e164: 8 total, 8 up, 8 in 2024-09-20T22:07:27.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:27 smithi060 ceph-mon[40290]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:07:27.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:27 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/784297564' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:07:27.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:27 smithi060 ceph-mon[40290]: osdmap e164: 8 total, 8 up, 8 in 2024-09-20T22:07:27.823 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:27.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:27.824 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:27.893 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:27 smithi067 ceph-mon[33511]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:07:27.893 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:27 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/784297564' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:07:27.894 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:27 smithi067 ceph-mon[33511]: osdmap e164: 8 total, 8 up, 8 in 2024-09-20T22:07:27.925 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:27.967 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:27.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:27.968 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:28.069 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:28.103 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:28.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:28.104 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:28.172 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 3 1 2024-09-20T22:07:28.202 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:07:28.203 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:07:28.205 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:28.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:28.264 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:28.264 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:28.365 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:28.393 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:28.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:28.394 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:28.495 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:28.523 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:28.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:28.525 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:28.626 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:28.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:28.689 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:28.689 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:28.717 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:28.789 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:28.818 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:28.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:28.819 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:28.880 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:28 smithi060 ceph-mon[29694]: pgmap v218: 158 pgs: 3 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 152 active+clean; 805 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 791 B/s rd, 0 B/s wr, 834 op/s; 1093/39216 objects degraded (2.787%); 1976/39216 objects misplaced (5.039%); 2.4 MiB/s, 38 objects/s recovering 2024-09-20T22:07:28.880 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:28 smithi060 ceph-mon[40290]: pgmap v218: 158 pgs: 3 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 152 active+clean; 805 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 791 B/s rd, 0 B/s wr, 834 op/s; 1093/39216 objects degraded (2.787%); 1976/39216 objects misplaced (5.039%); 2.4 MiB/s, 38 objects/s recovering 2024-09-20T22:07:28.920 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:28.963 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:28.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:28.964 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:29.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:28 smithi067 ceph-mon[33511]: pgmap v218: 158 pgs: 3 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 152 active+clean; 805 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 791 B/s rd, 0 B/s wr, 834 op/s; 1093/39216 objects degraded (2.787%); 1976/39216 objects misplaced (5.039%); 2.4 MiB/s, 38 objects/s recovering 2024-09-20T22:07:29.061 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:29.065 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:29.066 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T22:07:29.133 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:29.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:29.135 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:29.235 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:29.270 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:29.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:29.271 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:29.372 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:29.409 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:29.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:29.410 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:29.441 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:07:29.510 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:29.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:29.540 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:29.540 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:29.641 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:29.670 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:29.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:29.671 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:29.772 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:29.804 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:29.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:29.805 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:29.851 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:29 smithi060 ceph-mon[29694]: osdmap e165: 8 total, 8 up, 8 in 2024-09-20T22:07:29.852 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:29 smithi060 ceph-mon[29694]: 5.1a scrub starts 2024-09-20T22:07:29.852 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:29 smithi060 ceph-mon[29694]: 5.1a scrub ok 2024-09-20T22:07:29.852 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:29 smithi060 ceph-mon[40290]: osdmap e165: 8 total, 8 up, 8 in 2024-09-20T22:07:29.852 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:29 smithi060 ceph-mon[40290]: 5.1a scrub starts 2024-09-20T22:07:29.852 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:29 smithi060 ceph-mon[40290]: 5.1a scrub ok 2024-09-20T22:07:29.907 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:29.926 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:29 smithi067 ceph-mon[33511]: osdmap e165: 8 total, 8 up, 8 in 2024-09-20T22:07:29.926 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:29 smithi067 ceph-mon[33511]: 5.1a scrub starts 2024-09-20T22:07:29.926 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:29 smithi067 ceph-mon[33511]: 5.1a scrub ok 2024-09-20T22:07:29.932 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:29.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:29.934 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:30.034 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:30.050 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:07:30.050 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:07:30.057 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:30.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:30.058 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:30.159 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:30.182 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:30.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:30.183 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:30.284 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:30.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:30.307 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:30.307 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:30.408 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:30.430 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:30.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:30.431 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:30.531 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:30.554 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:30.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:30.555 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:30.591 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.3 to 1 (10000) 2024-09-20T22:07:30.591 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:07:30.649 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:07:30.656 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:30.738 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:30.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:30.739 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:30 smithi067 ceph-mon[33511]: osdmap e166: 8 total, 8 up, 8 in 2024-09-20T22:07:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:30 smithi067 ceph-mon[33511]: 4.6 scrub starts 2024-09-20T22:07:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:30 smithi067 ceph-mon[33511]: 4.6 scrub ok 2024-09-20T22:07:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:30 smithi067 ceph-mon[33511]: 6.8 scrub starts 2024-09-20T22:07:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:30 smithi067 ceph-mon[33511]: 6.8 scrub ok 2024-09-20T22:07:30.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:30 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3059180556' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:07:30.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:30 smithi067 ceph-mon[33511]: pgmap v221: 158 pgs: 3 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 152 active+clean; 805 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 938 B/s rd, 0 B/s wr, 989 op/s; 1093/39216 objects degraded (2.787%); 1976/39216 objects misplaced (5.039%); 2.8 MiB/s, 45 objects/s recovering 2024-09-20T22:07:30.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:30 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2826347136' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:07:30.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:30 smithi067 ceph-mon[33511]: 5.2 scrub starts 2024-09-20T22:07:30.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:30 smithi067 ceph-mon[33511]: 5.2 scrub ok 2024-09-20T22:07:30.840 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:30.854 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T22:07:30.863 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:30.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:30.864 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:30.965 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:30.988 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:30.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:30.989 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[29694]: osdmap e166: 8 total, 8 up, 8 in 2024-09-20T22:07:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[29694]: 4.6 scrub starts 2024-09-20T22:07:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[29694]: 4.6 scrub ok 2024-09-20T22:07:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[29694]: 6.8 scrub starts 2024-09-20T22:07:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[29694]: 6.8 scrub ok 2024-09-20T22:07:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3059180556' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:07:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[29694]: pgmap v221: 158 pgs: 3 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 152 active+clean; 805 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 938 B/s rd, 0 B/s wr, 989 op/s; 1093/39216 objects degraded (2.787%); 1976/39216 objects misplaced (5.039%); 2.8 MiB/s, 45 objects/s recovering 2024-09-20T22:07:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2826347136' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:07:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[29694]: 5.2 scrub starts 2024-09-20T22:07:31.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[29694]: 5.2 scrub ok 2024-09-20T22:07:31.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[40290]: osdmap e166: 8 total, 8 up, 8 in 2024-09-20T22:07:31.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[40290]: 4.6 scrub starts 2024-09-20T22:07:31.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[40290]: 4.6 scrub ok 2024-09-20T22:07:31.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[40290]: 6.8 scrub starts 2024-09-20T22:07:31.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[40290]: 6.8 scrub ok 2024-09-20T22:07:31.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3059180556' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:07:31.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[40290]: pgmap v221: 158 pgs: 3 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 152 active+clean; 805 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 938 B/s rd, 0 B/s wr, 989 op/s; 1093/39216 objects degraded (2.787%); 1976/39216 objects misplaced (5.039%); 2.8 MiB/s, 45 objects/s recovering 2024-09-20T22:07:31.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2826347136' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:07:31.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[40290]: 5.2 scrub starts 2024-09-20T22:07:31.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:30 smithi060 ceph-mon[40290]: 5.2 scrub ok 2024-09-20T22:07:31.090 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:31.112 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:07:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:31.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:31.114 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:31.214 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:31.236 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:31.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:31.237 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:31.338 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:31.360 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:31.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:31.362 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:31.462 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:31.485 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:31.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:31.486 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:31.586 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:31.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:31.615 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:31.615 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:31.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:31.756 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:31.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:31.757 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:31.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:31 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:07:31.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:31 smithi060 ceph-mon[29694]: 6.3 scrub starts 2024-09-20T22:07:31.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:31 smithi060 ceph-mon[29694]: 6.3 scrub ok 2024-09-20T22:07:31.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:31 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3059180556' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:07:31.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:31 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2826347136' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:07:31.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:31 smithi060 ceph-mon[29694]: osdmap e167: 8 total, 8 up, 8 in 2024-09-20T22:07:31.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:31 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 1093/39216 objects degraded (2.787%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T22:07:31.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:31 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:07:31.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:31 smithi060 ceph-mon[40290]: 6.3 scrub starts 2024-09-20T22:07:31.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:31 smithi060 ceph-mon[40290]: 6.3 scrub ok 2024-09-20T22:07:31.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:31 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3059180556' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:07:31.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:31 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2826347136' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:07:31.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:31 smithi060 ceph-mon[40290]: osdmap e167: 8 total, 8 up, 8 in 2024-09-20T22:07:31.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:31 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 1093/39216 objects degraded (2.787%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T22:07:31.857 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:31.887 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:31.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:31.888 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:31.957 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:31 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:07:31.957 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:31 smithi067 ceph-mon[33511]: 6.3 scrub starts 2024-09-20T22:07:31.957 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:31 smithi067 ceph-mon[33511]: 6.3 scrub ok 2024-09-20T22:07:31.957 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:31 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3059180556' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:07:31.958 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:31 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2826347136' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:07:31.958 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:31 smithi067 ceph-mon[33511]: osdmap e167: 8 total, 8 up, 8 in 2024-09-20T22:07:31.958 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:31 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 1093/39216 objects degraded (2.787%), 2 pgs degraded (PG_DEGRADED) 2024-09-20T22:07:31.988 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:32.024 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:32.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:32.026 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:32.126 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:32.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:32.164 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:32.164 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:32.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:07:31 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:07:32.266 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:32.303 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:32.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:32.304 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:32.404 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:32.441 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:32.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:32.442 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:32.542 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:32.569 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:32.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:32.570 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:32.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:32.677 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:32 smithi067 ceph-mon[33511]: osdmap e168: 8 total, 8 up, 8 in 2024-09-20T22:07:32.678 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:32 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:07:32.678 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:32 smithi067 ceph-mon[33511]: pgmap v224: 158 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 154 active+clean; 584 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 885 op/s; 1043/28599 objects degraded (3.647%); 965/28599 objects misplaced (3.374%); 9.1 MiB/s, 146 objects/s recovering 2024-09-20T22:07:32.702 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:07:32.702 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:07:32.709 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:32.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:32.711 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:32.811 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:32.855 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:32.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:32.856 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:32.957 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:32.996 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:32.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:32.998 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:33.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:32 smithi060 ceph-mon[29694]: osdmap e168: 8 total, 8 up, 8 in 2024-09-20T22:07:33.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:32 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:07:33.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:32 smithi060 ceph-mon[29694]: pgmap v224: 158 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 154 active+clean; 584 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 885 op/s; 1043/28599 objects degraded (3.647%); 965/28599 objects misplaced (3.374%); 9.1 MiB/s, 146 objects/s recovering 2024-09-20T22:07:33.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:32 smithi060 ceph-mon[40290]: osdmap e168: 8 total, 8 up, 8 in 2024-09-20T22:07:33.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:32 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:07:33.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:32 smithi060 ceph-mon[40290]: pgmap v224: 158 pgs: 1 active+remapped+backfilling, 2 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 154 active+clean; 584 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 885 op/s; 1043/28599 objects degraded (3.647%); 965/28599 objects misplaced (3.374%); 9.1 MiB/s, 146 objects/s recovering 2024-09-20T22:07:33.099 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:33.131 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:33.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:33.132 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:33.233 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:33.269 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:33.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:33.270 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:33.370 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:33.400 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:33.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:33.401 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:33.502 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:33.520 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T22:07:33.550 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:33.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:33.551 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:33.653 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:33.698 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:33.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:33.699 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:33.799 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:33.836 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:33.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:33.837 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:33.890 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:07:33.938 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:33.974 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:33.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:33.975 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:34.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:34.123 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:34.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:34.124 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:34.225 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:34.240 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:34 smithi060 ceph-mon[29694]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:07:34.240 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:34 smithi060 ceph-mon[40290]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:07:34.260 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:34.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:34.262 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:34.304 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 4 1 2024-09-20T22:07:34.340 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:34.362 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:34.363 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:34 smithi067 ceph-mon[33511]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:07:34.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:34.419 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:34.419 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:34.508 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:07:34.508 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:07:34.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:34.549 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:34.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:34.550 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:34.651 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:34.679 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:34.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:34.680 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:34.780 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:34.816 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:34.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:34.817 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:34.918 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:34.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:34.965 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:34.965 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:35.029 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:35.030 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:07:35.030 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:07:35.066 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:35.071 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:07:35.071 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:07:35.108 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:07:35.109 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:07:35.109 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:07:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:07:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:07:35.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:35.130 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:35.131 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:35.142 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:07:35.143 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:07:35.177 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:07:35.178 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:07:35.211 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:07:35.212 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:07:35.231 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:35.245 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:07:35.245 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:07:35.279 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:35.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:35.280 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:35.281 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:07:35.281 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:07:35.381 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:35.387 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:35 smithi060 ceph-mon[29694]: pgmap v225: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 156 active+clean; 553 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 943 B/s rd, 0 B/s wr, 744 op/s; 158/27120 objects degraded (0.583%); 612/27120 objects misplaced (2.257%); 18 MiB/s, 282 objects/s recovering 2024-09-20T22:07:35.387 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:35 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3238036036' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:35.387 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:35 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:07:35.387 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:35 smithi060 ceph-mon[40290]: pgmap v225: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 156 active+clean; 553 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 943 B/s rd, 0 B/s wr, 744 op/s; 158/27120 objects degraded (0.583%); 612/27120 objects misplaced (2.257%); 18 MiB/s, 282 objects/s recovering 2024-09-20T22:07:35.387 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:35 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3238036036' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:35.387 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:35 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:07:35.408 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:35.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:35.409 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:35.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:35.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:35 smithi067 ceph-mon[33511]: pgmap v225: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 156 active+clean; 553 MiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 943 B/s rd, 0 B/s wr, 744 op/s; 158/27120 objects degraded (0.583%); 612/27120 objects misplaced (2.257%); 18 MiB/s, 282 objects/s recovering 2024-09-20T22:07:35.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:35 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3238036036' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:35.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:35 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:07:35.552 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:35.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:35.553 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:35.562 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:07:35.653 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:35.680 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:35.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:35.681 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:35.781 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:35.808 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:35.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:35.810 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:35.910 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:35.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:35.946 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:35.946 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:36.047 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:36.075 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:36.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:36.076 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:36.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:36.202 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:36.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:36.203 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:36.304 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:36.342 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:36.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:36.343 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:36.444 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:36.455 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:36.471 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:36.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:36.473 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:36.573 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:36.602 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:36.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:36.603 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:36.704 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:36.730 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:36.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:36.731 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:36.832 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:36.854 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:36.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:36.855 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:36.956 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:36.979 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:36.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:36.980 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:37.080 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:37.103 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:37.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:37.104 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:37.131 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.4 to 1 (10000) 2024-09-20T22:07:37.205 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:37.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:37.247 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:37.247 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:37.324 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T22:07:37.348 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:37.414 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:37.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:37.416 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:37.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:37 smithi067 ceph-mon[33511]: pgmap v226: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 156 active+clean; 412 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 981 op/s; 90/20334 objects degraded (0.443%); 612/20334 objects misplaced (3.010%); 15 MiB/s, 238 objects/s recovering 2024-09-20T22:07:37.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:37 smithi067 ceph-mon[33511]: osdmap e169: 8 total, 8 up, 8 in 2024-09-20T22:07:37.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:37 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3078676904' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:07:37.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:37 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 90/20334 objects degraded (0.443%), 1 pg degraded (PG_DEGRADED) 2024-09-20T22:07:37.517 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:37.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:37 smithi060 ceph-mon[29694]: pgmap v226: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 156 active+clean; 412 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 981 op/s; 90/20334 objects degraded (0.443%); 612/20334 objects misplaced (3.010%); 15 MiB/s, 238 objects/s recovering 2024-09-20T22:07:37.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:37 smithi060 ceph-mon[29694]: osdmap e169: 8 total, 8 up, 8 in 2024-09-20T22:07:37.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:37 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3078676904' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:07:37.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:37 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 90/20334 objects degraded (0.443%), 1 pg degraded (PG_DEGRADED) 2024-09-20T22:07:37.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:37 smithi060 ceph-mon[40290]: pgmap v226: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 156 active+clean; 412 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 981 op/s; 90/20334 objects degraded (0.443%); 612/20334 objects misplaced (3.010%); 15 MiB/s, 238 objects/s recovering 2024-09-20T22:07:37.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:37 smithi060 ceph-mon[40290]: osdmap e169: 8 total, 8 up, 8 in 2024-09-20T22:07:37.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:37 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3078676904' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:07:37.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:37 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 90/20334 objects degraded (0.443%), 1 pg degraded (PG_DEGRADED) 2024-09-20T22:07:37.559 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:37.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:37.560 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:37.661 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:37.692 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:37.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:37.693 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:37.707 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:07:37.794 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:37.825 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:37.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:37.826 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:37.927 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:37.949 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:37.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:37.950 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:38.051 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:38.090 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:38.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:38.091 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:38.192 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:38.233 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:38.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:38.234 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:38.305 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:07:38.305 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:07:38.305 INFO:teuthology.orchestra.run.smithi067.stdout: "complaint_time": 30, 2024-09-20T22:07:38.305 INFO:teuthology.orchestra.run.smithi067.stdout: "num_blocked_ops": 0 2024-09-20T22:07:38.305 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:07:38.335 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:38.374 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:38.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:38.375 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:38.378 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:07:38.457 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:38 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3078676904' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T22:07:38.457 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:38 smithi060 ceph-mon[29694]: osdmap e170: 8 total, 8 up, 8 in 2024-09-20T22:07:38.457 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:38 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3078676904' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T22:07:38.458 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:38 smithi060 ceph-mon[40290]: osdmap e170: 8 total, 8 up, 8 in 2024-09-20T22:07:38.476 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:38.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:38 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3078676904' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T22:07:38.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:38 smithi067 ceph-mon[33511]: osdmap e170: 8 total, 8 up, 8 in 2024-09-20T22:07:38.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:38.515 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:38.516 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:38.616 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:38.645 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:38.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:38.646 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:38.747 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:38.773 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:38.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:38.774 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:38.875 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:38.909 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:38.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:38.910 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:39.011 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:39.046 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:39.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:39.047 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:39.147 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:39.151 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T22:07:39.180 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:39.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:39.181 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:39.282 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:39.322 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:39.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:39.323 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:39.424 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:39.463 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:39.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:39.464 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:39.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:39 smithi067 ceph-mon[33511]: pgmap v229: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 156 active+clean; 269 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 798 B/s rd, 0 B/s wr, 785 op/s; 90/13491 objects degraded (0.667%); 612/13491 objects misplaced (4.536%); 9.2 MiB/s, 147 objects/s recovering 2024-09-20T22:07:39.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:39 smithi060 ceph-mon[29694]: pgmap v229: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 156 active+clean; 269 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 798 B/s rd, 0 B/s wr, 785 op/s; 90/13491 objects degraded (0.667%); 612/13491 objects misplaced (4.536%); 9.2 MiB/s, 147 objects/s recovering 2024-09-20T22:07:39.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:39 smithi060 ceph-mon[40290]: pgmap v229: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 156 active+clean; 269 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 798 B/s rd, 0 B/s wr, 785 op/s; 90/13491 objects degraded (0.667%); 612/13491 objects misplaced (4.536%); 9.2 MiB/s, 147 objects/s recovering 2024-09-20T22:07:39.565 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:39.569 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:07:39.605 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:39.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:39.606 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:39.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:39.745 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:39.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:39.746 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:39.847 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:39.892 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:39.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:39.894 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:39.917 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 5 1 2024-09-20T22:07:39.994 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:40.046 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:40.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:40.047 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:40.148 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:40.189 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:40.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:40.190 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:40.211 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:07:40.212 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:07:40.212 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:07:40.212 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:07:40.212 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.212 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:46027 7.0 7:0265a40b:::benchmark_data_smithi060_78800_object46026:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T22:07:40.212 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:19.389108+0000", 2024-09-20T22:07:40.212 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 200.81742861500001, 2024-09-20T22:07:40.212 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.079311146999999999, 2024-09-20T22:07:40.212 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.212 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.212 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.213 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.213 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.213 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.213 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 46027 2024-09-20T22:07:40.213 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.213 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.213 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.213 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.213 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389108+0000", 2024-09-20T22:07:40.213 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.213 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.213 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.213 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.213 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389108+0000", 2024-09-20T22:07:40.214 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.214 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.214 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.214 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.214 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389109+0000", 2024-09-20T22:07:40.214 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1289999999999999e-06 2024-09-20T22:07:40.214 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.214 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.214 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.214 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389169+0000", 2024-09-20T22:07:40.214 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0485000000000001e-05 2024-09-20T22:07:40.214 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.215 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.215 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.215 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389177+0000", 2024-09-20T22:07:40.215 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.089e-06 2024-09-20T22:07:40.215 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.215 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.215 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.215 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389182+0000", 2024-09-20T22:07:40.215 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9749999999999998e-06 2024-09-20T22:07:40.215 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.215 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.215 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.215 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389275+0000", 2024-09-20T22:07:40.215 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2967000000000006e-05 2024-09-20T22:07:40.215 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.216 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.216 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.216 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389390+0000", 2024-09-20T22:07:40.216 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000115148 2024-09-20T22:07:40.216 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.216 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.216 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:07:40.216 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389524+0000", 2024-09-20T22:07:40.216 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013312900000000001 2024-09-20T22:07:40.216 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.216 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.216 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.216 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.391831+0000", 2024-09-20T22:07:40.217 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0023078769999999998 2024-09-20T22:07:40.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.217 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.217 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.217 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.468296+0000", 2024-09-20T22:07:40.217 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.076464536999999999 2024-09-20T22:07:40.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.217 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.217 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.217 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.468337+0000", 2024-09-20T22:07:40.217 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1102000000000001e-05 2024-09-20T22:07:40.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.217 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.217 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.217 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.468419+0000", 2024-09-20T22:07:40.218 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1709e-05 2024-09-20T22:07:40.218 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.218 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.218 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.218 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.218 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.218 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:46028 7.0 7:04febef9:::benchmark_data_smithi060_78800_object46027:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e93)", 2024-09-20T22:07:40.218 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:19.389375+0000", 2024-09-20T22:07:40.218 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 200.81716088499999, 2024-09-20T22:07:40.218 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.079186260999999994, 2024-09-20T22:07:40.218 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.218 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.219 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.219 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.219 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.219 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.219 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 46028 2024-09-20T22:07:40.219 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.219 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.219 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.219 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.219 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389375+0000", 2024-09-20T22:07:40.219 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.219 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.219 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.219 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.220 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389375+0000", 2024-09-20T22:07:40.220 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.220 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.220 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.220 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389376+0000", 2024-09-20T22:07:40.220 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8000000000000004e-07 2024-09-20T22:07:40.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.220 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.220 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.220 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389437+0000", 2024-09-20T22:07:40.220 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0291999999999998e-05 2024-09-20T22:07:40.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.220 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.221 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.221 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389446+0000", 2024-09-20T22:07:40.221 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8130000000000007e-06 2024-09-20T22:07:40.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.221 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.221 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.221 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.389453+0000", 2024-09-20T22:07:40.221 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0469999999999996e-06 2024-09-20T22:07:40.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.221 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.221 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.221 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.390649+0000", 2024-09-20T22:07:40.222 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001195392 2024-09-20T22:07:40.222 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.222 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.222 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.222 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.390768+0000", 2024-09-20T22:07:40.222 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000118863 2024-09-20T22:07:40.222 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.222 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.222 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:07:40.222 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.390902+0000", 2024-09-20T22:07:40.222 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013449100000000001 2024-09-20T22:07:40.222 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.223 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.223 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.223 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.391959+0000", 2024-09-20T22:07:40.223 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001056415 2024-09-20T22:07:40.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.223 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.223 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.223 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.468456+0000", 2024-09-20T22:07:40.223 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.076497323000000006 2024-09-20T22:07:40.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.223 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.223 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.223 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.468490+0000", 2024-09-20T22:07:40.224 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.3887000000000003e-05 2024-09-20T22:07:40.224 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.224 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.224 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.224 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:19.468562+0000", 2024-09-20T22:07:40.224 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1858000000000002e-05 2024-09-20T22:07:40.224 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.224 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.224 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.224 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.224 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.224 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:97624 7.0 7:00ae6b61:::benchmark_data_smithi060_78800_object97623:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-20T22:07:40.224 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:55.964064+0000", 2024-09-20T22:07:40.224 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 164.24247240700001, 2024-09-20T22:07:40.224 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.081791942000000006, 2024-09-20T22:07:40.225 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.225 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.225 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.225 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.225 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.225 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.225 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 97624 2024-09-20T22:07:40.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.225 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.225 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.225 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.225 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.964064+0000", 2024-09-20T22:07:40.225 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.225 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.226 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.226 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.964064+0000", 2024-09-20T22:07:40.226 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.226 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.226 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.226 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.226 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.964064+0000", 2024-09-20T22:07:40.226 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8800000000000002e-07 2024-09-20T22:07:40.226 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.226 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.226 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.226 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.964102+0000", 2024-09-20T22:07:40.227 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7379000000000002e-05 2024-09-20T22:07:40.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.227 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.227 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.227 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.964109+0000", 2024-09-20T22:07:40.227 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4719999999999998e-06 2024-09-20T22:07:40.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.227 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.227 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.227 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:55.964115+0000", 2024-09-20T22:07:40.227 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7270000000000004e-06 2024-09-20T22:07:40.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.227 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.227 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.228 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.027873+0000", 2024-09-20T22:07:40.228 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.063758327000000004 2024-09-20T22:07:40.228 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.228 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.228 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.228 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.027995+0000", 2024-09-20T22:07:40.228 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000121808 2024-09-20T22:07:40.228 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.228 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.228 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:07:40.228 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.028169+0000", 2024-09-20T22:07:40.228 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017430500000000001 2024-09-20T22:07:40.228 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.228 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.229 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.229 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.034984+0000", 2024-09-20T22:07:40.229 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0068145080000000004 2024-09-20T22:07:40.229 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.229 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.229 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.229 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.045758+0000", 2024-09-20T22:07:40.229 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.010773731 2024-09-20T22:07:40.229 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.229 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.229 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.229 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.045800+0000", 2024-09-20T22:07:40.229 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2626000000000002e-05 2024-09-20T22:07:40.230 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.230 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.230 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.230 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.045856+0000", 2024-09-20T22:07:40.230 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5470999999999998e-05 2024-09-20T22:07:40.230 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.230 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.230 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.230 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.230 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.230 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:98062 7.b 7:d93ed38f:::benchmark_data_smithi060_78800_object98061:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e105)", 2024-09-20T22:07:40.230 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.322852+0000", 2024-09-20T22:07:40.230 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 163.883684515, 2024-09-20T22:07:40.230 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.084887676999999995, 2024-09-20T22:07:40.231 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.231 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.231 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.231 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.231 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.231 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.231 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 98062 2024-09-20T22:07:40.231 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.231 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.231 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.231 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.231 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322852+0000", 2024-09-20T22:07:40.231 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.232 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.232 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.232 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.232 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322852+0000", 2024-09-20T22:07:40.232 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.232 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.232 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.232 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.232 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322853+0000", 2024-09-20T22:07:40.232 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.376e-06 2024-09-20T22:07:40.232 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.232 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.232 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.232 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322895+0000", 2024-09-20T22:07:40.232 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1971999999999997e-05 2024-09-20T22:07:40.233 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.233 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.233 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.233 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322907+0000", 2024-09-20T22:07:40.233 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1758e-05 2024-09-20T22:07:40.233 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.233 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.233 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.233 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322914+0000", 2024-09-20T22:07:40.233 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1840000000000002e-06 2024-09-20T22:07:40.233 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.233 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.233 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.233 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.322965+0000", 2024-09-20T22:07:40.234 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0584000000000002e-05 2024-09-20T22:07:40.234 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.234 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.234 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.234 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.323058+0000", 2024-09-20T22:07:40.234 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3249000000000001e-05 2024-09-20T22:07:40.234 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.234 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.234 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6", 2024-09-20T22:07:40.234 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.323142+0000", 2024-09-20T22:07:40.234 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4429000000000001e-05 2024-09-20T22:07:40.234 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.234 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.234 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.235 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.325301+0000", 2024-09-20T22:07:40.235 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0021587440000000002 2024-09-20T22:07:40.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.235 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.235 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.235 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.407511+0000", 2024-09-20T22:07:40.235 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.082209656000000006 2024-09-20T22:07:40.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.235 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.235 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.235 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.407630+0000", 2024-09-20T22:07:40.235 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011909900000000001 2024-09-20T22:07:40.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.235 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.235 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.236 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.407739+0000", 2024-09-20T22:07:40.236 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000109626 2024-09-20T22:07:40.236 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.236 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.236 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.236 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.236 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:98529 7.b 7:d59e89dd:::benchmark_data_smithi060_78800_object98528:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e106)", 2024-09-20T22:07:40.236 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:56.760245+0000", 2024-09-20T22:07:40.236 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 163.44629145900001, 2024-09-20T22:07:40.236 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.077949966999999995, 2024-09-20T22:07:40.236 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.236 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.236 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.237 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.237 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.237 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.237 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 98529 2024-09-20T22:07:40.237 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.237 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.237 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.237 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.237 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760245+0000", 2024-09-20T22:07:40.237 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.237 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.237 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.237 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.237 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760245+0000", 2024-09-20T22:07:40.237 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.238 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.238 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.238 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760246+0000", 2024-09-20T22:07:40.238 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.062e-06 2024-09-20T22:07:40.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.238 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.238 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.238 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760304+0000", 2024-09-20T22:07:40.238 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7834999999999998e-05 2024-09-20T22:07:40.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.238 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.238 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.239 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760313+0000", 2024-09-20T22:07:40.239 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1819999999999995e-06 2024-09-20T22:07:40.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.239 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.239 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.239 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760318+0000", 2024-09-20T22:07:40.239 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0239999999999997e-06 2024-09-20T22:07:40.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.239 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.240 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.240 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760361+0000", 2024-09-20T22:07:40.240 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3383000000000001e-05 2024-09-20T22:07:40.240 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.240 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.240 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.240 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760464+0000", 2024-09-20T22:07:40.240 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010275700000000001 2024-09-20T22:07:40.240 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.241 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.241 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6", 2024-09-20T22:07:40.241 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.760550+0000", 2024-09-20T22:07:40.241 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6180000000000005e-05 2024-09-20T22:07:40.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.241 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.241 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.241 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.762778+0000", 2024-09-20T22:07:40.241 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0022282690000000002 2024-09-20T22:07:40.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.241 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.241 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.242 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.838033+0000", 2024-09-20T22:07:40.242 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.075254673999999994 2024-09-20T22:07:40.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.242 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.242 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.242 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.838095+0000", 2024-09-20T22:07:40.242 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2068000000000003e-05 2024-09-20T22:07:40.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.242 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.242 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.242 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:56.838195+0000", 2024-09-20T22:07:40.242 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9532999999999997e-05 2024-09-20T22:07:40.243 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.243 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.243 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.243 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.243 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.243 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:99611 7.b 7:d551c118:::benchmark_data_smithi060_78800_object99610:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-20T22:07:40.243 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:57.750045+0000", 2024-09-20T22:07:40.243 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 162.456490944, 2024-09-20T22:07:40.243 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.076272598999999996, 2024-09-20T22:07:40.243 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.243 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.243 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.243 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.243 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.243 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.244 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 99611 2024-09-20T22:07:40.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.244 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.244 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.244 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.244 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750045+0000", 2024-09-20T22:07:40.244 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.244 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.244 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.244 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750045+0000", 2024-09-20T22:07:40.244 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.244 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.245 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.245 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750047+0000", 2024-09-20T22:07:40.245 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6419999999999999e-06 2024-09-20T22:07:40.245 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.245 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.245 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.245 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750097+0000", 2024-09-20T22:07:40.245 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0220999999999999e-05 2024-09-20T22:07:40.245 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.245 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.245 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.245 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750110+0000", 2024-09-20T22:07:40.245 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2577e-05 2024-09-20T22:07:40.245 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.246 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.246 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.246 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750117+0000", 2024-09-20T22:07:40.246 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5939999999999996e-06 2024-09-20T22:07:40.246 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.246 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.246 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.246 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750172+0000", 2024-09-20T22:07:40.246 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4586999999999998e-05 2024-09-20T22:07:40.246 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.246 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.246 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.246 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750253+0000", 2024-09-20T22:07:40.247 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0924e-05 2024-09-20T22:07:40.247 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.247 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.247 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6", 2024-09-20T22:07:40.247 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750350+0000", 2024-09-20T22:07:40.247 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6871000000000001e-05 2024-09-20T22:07:40.247 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.247 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.247 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.247 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.755920+0000", 2024-09-20T22:07:40.247 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0055701470000000001 2024-09-20T22:07:40.247 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.247 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.247 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.247 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.826093+0000", 2024-09-20T22:07:40.248 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.070173630000000001 2024-09-20T22:07:40.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.248 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.248 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.248 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.826212+0000", 2024-09-20T22:07:40.248 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011807 2024-09-20T22:07:40.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.248 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.248 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.248 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.826318+0000", 2024-09-20T22:07:40.248 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010633600000000001 2024-09-20T22:07:40.248 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.248 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.248 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.249 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.249 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.249 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:99613 7.b 7:d5b26ca4:::benchmark_data_smithi060_78800_object99612:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-20T22:07:40.249 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:57.750430+0000", 2024-09-20T22:07:40.249 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 162.45610617700001, 2024-09-20T22:07:40.249 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.077303389, 2024-09-20T22:07:40.249 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.249 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.249 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.249 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.249 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.249 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.249 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 99613 2024-09-20T22:07:40.249 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.250 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.250 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.250 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.250 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750430+0000", 2024-09-20T22:07:40.250 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.250 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.250 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.250 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.250 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750430+0000", 2024-09-20T22:07:40.250 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.250 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.250 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.250 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.250 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750431+0000", 2024-09-20T22:07:40.251 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.64e-07 2024-09-20T22:07:40.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.251 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.251 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.251 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750527+0000", 2024-09-20T22:07:40.251 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6439000000000003e-05 2024-09-20T22:07:40.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.251 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.251 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.251 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750539+0000", 2024-09-20T22:07:40.251 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1778e-05 2024-09-20T22:07:40.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.251 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.252 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.252 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.750547+0000", 2024-09-20T22:07:40.252 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3629999999999996e-06 2024-09-20T22:07:40.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.252 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.252 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.252 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.753106+0000", 2024-09-20T22:07:40.252 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0025595510000000002 2024-09-20T22:07:40.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.252 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.252 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.252 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.753223+0000", 2024-09-20T22:07:40.252 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000116927 2024-09-20T22:07:40.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.252 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.253 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6", 2024-09-20T22:07:40.253 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.753368+0000", 2024-09-20T22:07:40.253 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014529199999999999 2024-09-20T22:07:40.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.253 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.253 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.253 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.756019+0000", 2024-09-20T22:07:40.253 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0026511180000000001 2024-09-20T22:07:40.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.253 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.253 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.253 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.827375+0000", 2024-09-20T22:07:40.253 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.071355298999999997 2024-09-20T22:07:40.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.254 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.254 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.827482+0000", 2024-09-20T22:07:40.254 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000106963 2024-09-20T22:07:40.254 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.254 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.254 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:57.827733+0000", 2024-09-20T22:07:40.254 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025169500000000002 2024-09-20T22:07:40.254 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.254 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.254 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.254 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.255 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:101399 7.b 7:dad621f3:::benchmark_data_smithi060_78800_object101398:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-20T22:07:40.255 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:59.478423+0000", 2024-09-20T22:07:40.255 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 160.72811296099999, 2024-09-20T22:07:40.255 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.079270373000000005, 2024-09-20T22:07:40.255 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.255 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.255 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.255 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.255 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.255 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.255 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 101399 2024-09-20T22:07:40.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.255 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.255 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.256 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.256 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.478423+0000", 2024-09-20T22:07:40.256 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.256 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.256 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.478423+0000", 2024-09-20T22:07:40.256 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.256 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.256 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.478425+0000", 2024-09-20T22:07:40.256 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.341e-06 2024-09-20T22:07:40.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.257 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.257 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.257 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.478475+0000", 2024-09-20T22:07:40.257 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0312999999999998e-05 2024-09-20T22:07:40.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.257 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.257 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.257 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.478486+0000", 2024-09-20T22:07:40.257 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0813e-05 2024-09-20T22:07:40.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.257 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.257 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.257 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.478491+0000", 2024-09-20T22:07:40.257 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5130000000000004e-06 2024-09-20T22:07:40.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.258 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.258 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.258 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.478531+0000", 2024-09-20T22:07:40.258 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9489000000000001e-05 2024-09-20T22:07:40.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.258 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.258 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.258 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.478621+0000", 2024-09-20T22:07:40.258 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9827999999999994e-05 2024-09-20T22:07:40.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.258 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.258 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6", 2024-09-20T22:07:40.259 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.478707+0000", 2024-09-20T22:07:40.259 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6021999999999997e-05 2024-09-20T22:07:40.259 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.259 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.259 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.259 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.557429+0000", 2024-09-20T22:07:40.259 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.078721970000000002 2024-09-20T22:07:40.259 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.259 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.259 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.259 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.557513+0000", 2024-09-20T22:07:40.259 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4500999999999996e-05 2024-09-20T22:07:40.259 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.260 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.260 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.260 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.557576+0000", 2024-09-20T22:07:40.260 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2600000000000004e-05 2024-09-20T22:07:40.260 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.260 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.260 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.260 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.557694+0000", 2024-09-20T22:07:40.260 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000117983 2024-09-20T22:07:40.260 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.260 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.260 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.261 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.261 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.261 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:101402 7.0 7:025f111c:::benchmark_data_smithi060_78800_object101401:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-20T22:07:40.261 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:59.479446+0000", 2024-09-20T22:07:40.261 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 160.72709007200001, 2024-09-20T22:07:40.261 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.079768565, 2024-09-20T22:07:40.261 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.261 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.261 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.261 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.261 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.261 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.261 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 101402 2024-09-20T22:07:40.261 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.262 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.262 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.262 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.262 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479446+0000", 2024-09-20T22:07:40.262 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.262 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.262 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.262 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.262 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479446+0000", 2024-09-20T22:07:40.262 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.262 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.262 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.262 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.263 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479447+0000", 2024-09-20T22:07:40.263 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.68e-07 2024-09-20T22:07:40.263 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.263 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.263 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.263 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479505+0000", 2024-09-20T22:07:40.263 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8579999999999998e-05 2024-09-20T22:07:40.263 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.263 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.263 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.263 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479514+0000", 2024-09-20T22:07:40.263 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1920000000000005e-06 2024-09-20T22:07:40.263 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.263 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.264 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.264 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479518+0000", 2024-09-20T22:07:40.264 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6569999999999996e-06 2024-09-20T22:07:40.264 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.264 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.264 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.264 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479565+0000", 2024-09-20T22:07:40.264 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6820999999999998e-05 2024-09-20T22:07:40.264 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.264 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.264 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.264 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479642+0000", 2024-09-20T22:07:40.264 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6768999999999996e-05 2024-09-20T22:07:40.265 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.265 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.265 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:07:40.265 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479726+0000", 2024-09-20T22:07:40.265 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3759999999999998e-05 2024-09-20T22:07:40.265 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.265 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.265 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.265 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559070+0000", 2024-09-20T22:07:40.265 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.079344425999999996 2024-09-20T22:07:40.265 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.265 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.265 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.265 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559081+0000", 2024-09-20T22:07:40.266 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1223000000000001e-05 2024-09-20T22:07:40.266 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.266 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.266 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.266 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559129+0000", 2024-09-20T22:07:40.266 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.778e-05 2024-09-20T22:07:40.266 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.266 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.266 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.266 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559215+0000", 2024-09-20T22:07:40.266 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5688999999999999e-05 2024-09-20T22:07:40.266 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.266 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.267 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.267 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.267 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.267 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:101403 7.b 7:dcd99ced:::benchmark_data_smithi060_78800_object101402:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-20T22:07:40.267 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:59.479831+0000", 2024-09-20T22:07:40.267 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 160.726705141, 2024-09-20T22:07:40.267 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.078021717000000004, 2024-09-20T22:07:40.267 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.267 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.267 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.267 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.267 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.267 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.267 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 101403 2024-09-20T22:07:40.267 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.268 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.268 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.268 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.268 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479831+0000", 2024-09-20T22:07:40.268 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.268 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.268 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.268 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.268 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479831+0000", 2024-09-20T22:07:40.268 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.268 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.268 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.268 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.268 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479832+0000", 2024-09-20T22:07:40.269 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.7400000000000002e-07 2024-09-20T22:07:40.269 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.269 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.269 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.269 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479896+0000", 2024-09-20T22:07:40.269 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4144000000000001e-05 2024-09-20T22:07:40.269 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.269 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.269 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.269 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479903+0000", 2024-09-20T22:07:40.269 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4629999999999998e-06 2024-09-20T22:07:40.269 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.269 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.270 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.270 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.479906+0000", 2024-09-20T22:07:40.270 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7900000000000001e-06 2024-09-20T22:07:40.270 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.270 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.270 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.270 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480804+0000", 2024-09-20T22:07:40.270 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00089766900000000003 2024-09-20T22:07:40.270 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.270 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.270 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.270 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480913+0000", 2024-09-20T22:07:40.270 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000109407 2024-09-20T22:07:40.271 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.271 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.271 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6", 2024-09-20T22:07:40.271 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.481053+0000", 2024-09-20T22:07:40.271 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013912000000000001 2024-09-20T22:07:40.271 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.271 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.271 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.271 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.557496+0000", 2024-09-20T22:07:40.271 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.076443120000000003 2024-09-20T22:07:40.271 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.271 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.271 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.272 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.557701+0000", 2024-09-20T22:07:40.272 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000205479 2024-09-20T22:07:40.272 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.272 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.272 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.272 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.557753+0000", 2024-09-20T22:07:40.272 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1357999999999999e-05 2024-09-20T22:07:40.272 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.272 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.272 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.272 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.557853+0000", 2024-09-20T22:07:40.272 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100293 2024-09-20T22:07:40.272 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.272 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.273 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.273 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.273 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.273 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:101405 7.b 7:d7e4e064:::benchmark_data_smithi060_78800_object101404:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-20T22:07:40.273 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:59.480104+0000", 2024-09-20T22:07:40.273 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 160.726432316, 2024-09-20T22:07:40.273 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.081978562000000005, 2024-09-20T22:07:40.273 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.273 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.273 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.273 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.273 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.273 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.273 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 101405 2024-09-20T22:07:40.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.274 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.274 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.274 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.274 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480104+0000", 2024-09-20T22:07:40.274 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.274 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.274 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.274 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480104+0000", 2024-09-20T22:07:40.274 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.274 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.274 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.275 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480104+0000", 2024-09-20T22:07:40.275 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1200000000000003e-07 2024-09-20T22:07:40.275 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.275 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.275 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.275 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480188+0000", 2024-09-20T22:07:40.275 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3659999999999995e-05 2024-09-20T22:07:40.275 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.275 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.275 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.275 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480194+0000", 2024-09-20T22:07:40.275 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.66e-06 2024-09-20T22:07:40.275 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.275 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.276 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.276 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480197+0000", 2024-09-20T22:07:40.276 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.478e-06 2024-09-20T22:07:40.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.276 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.276 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.276 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554223+0000", 2024-09-20T22:07:40.276 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.074025653999999996 2024-09-20T22:07:40.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.276 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.276 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.276 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554363+0000", 2024-09-20T22:07:40.276 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000139727 2024-09-20T22:07:40.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.277 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.277 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6", 2024-09-20T22:07:40.277 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554528+0000", 2024-09-20T22:07:40.277 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000165129 2024-09-20T22:07:40.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.277 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.277 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.277 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.557863+0000", 2024-09-20T22:07:40.277 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0033354779999999998 2024-09-20T22:07:40.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.277 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.277 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.277 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.561924+0000", 2024-09-20T22:07:40.277 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0040608839999999998 2024-09-20T22:07:40.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.278 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.278 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.278 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.561987+0000", 2024-09-20T22:07:40.278 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2547999999999998e-05 2024-09-20T22:07:40.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.278 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.278 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.278 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.562082+0000", 2024-09-20T22:07:40.278 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5832e-05 2024-09-20T22:07:40.278 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.278 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.278 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.279 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.279 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:101407 7.0 7:04db00de:::benchmark_data_smithi060_78800_object101406:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e107)", 2024-09-20T22:07:40.279 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:59.480934+0000", 2024-09-20T22:07:40.279 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 160.725602533, 2024-09-20T22:07:40.279 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.078444652000000004, 2024-09-20T22:07:40.279 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.279 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.279 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.279 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.279 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.279 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.279 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 101407 2024-09-20T22:07:40.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.279 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.280 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.280 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.280 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480934+0000", 2024-09-20T22:07:40.280 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.280 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.280 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.280 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480934+0000", 2024-09-20T22:07:40.280 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.280 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.280 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.280 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480934+0000", 2024-09-20T22:07:40.281 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7000000000000004e-07 2024-09-20T22:07:40.281 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.281 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.281 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.281 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.480993+0000", 2024-09-20T22:07:40.281 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8697999999999999e-05 2024-09-20T22:07:40.281 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.281 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.281 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.282 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.481001+0000", 2024-09-20T22:07:40.283 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0399999999999993e-06 2024-09-20T22:07:40.283 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.283 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.283 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.283 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.481005+0000", 2024-09-20T22:07:40.283 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2189999999999998e-06 2024-09-20T22:07:40.283 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.283 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.283 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.283 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.482346+0000", 2024-09-20T22:07:40.283 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0013407670000000001 2024-09-20T22:07:40.283 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.283 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.284 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.284 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.482463+0000", 2024-09-20T22:07:40.284 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011674400000000001 2024-09-20T22:07:40.284 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.284 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.284 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:07:40.284 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554159+0000", 2024-09-20T22:07:40.284 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.071695851000000005 2024-09-20T22:07:40.284 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.284 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.284 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.284 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559254+0000", 2024-09-20T22:07:40.284 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0050953149999999996 2024-09-20T22:07:40.284 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.284 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.285 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.285 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559265+0000", 2024-09-20T22:07:40.285 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0514e-05 2024-09-20T22:07:40.285 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.285 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.285 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.285 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559307+0000", 2024-09-20T22:07:40.285 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2038e-05 2024-09-20T22:07:40.285 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.285 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.285 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.285 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.559378+0000", 2024-09-20T22:07:40.285 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1796000000000002e-05 2024-09-20T22:07:40.285 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.285 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.286 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.286 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.286 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.286 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.24667.0:101413 7.e e107/80 7:7bfa84cf:::benchmark_data_smithi060_78800_object101412:head v 107'6308, mlcod=107'6306)", 2024-09-20T22:07:40.286 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:04:59.487069+0000", 2024-09-20T22:07:40.286 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 160.71946682399999, 2024-09-20T22:07:40.286 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.090496713000000006, 2024-09-20T22:07:40.286 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.286 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.286 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.286 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.286 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.286 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.286 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.487069+0000", 2024-09-20T22:07:40.287 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.287 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.287 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.287 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.287 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.487069+0000", 2024-09-20T22:07:40.287 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.287 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.287 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.287 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.287 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.487070+0000", 2024-09-20T22:07:40.287 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9599999999999998e-07 2024-09-20T22:07:40.287 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.287 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.287 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.287 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.487131+0000", 2024-09-20T22:07:40.288 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0526000000000003e-05 2024-09-20T22:07:40.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.288 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.288 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.288 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554190+0000", 2024-09-20T22:07:40.288 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.067059399000000006 2024-09-20T22:07:40.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.288 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.288 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.288 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554197+0000", 2024-09-20T22:07:40.288 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0010000000000004e-06 2024-09-20T22:07:40.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.288 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.288 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.288 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554563+0000", 2024-09-20T22:07:40.289 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000365483 2024-09-20T22:07:40.289 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.289 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.289 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.289 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.554581+0000", 2024-09-20T22:07:40.289 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8481000000000001e-05 2024-09-20T22:07:40.289 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.289 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.289 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.289 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.577514+0000", 2024-09-20T22:07:40.289 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.022933044 2024-09-20T22:07:40.289 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.289 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.289 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.289 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:04:59.577566+0000", 2024-09-20T22:07:40.289 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1983e-05 2024-09-20T22:07:40.290 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.290 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.290 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.290 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.290 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.290 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169974 7.10 7:0a082cf8:::benchmark_data_smithi060_78800_object36941:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:07:40.290 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:07:40.290 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 101.581922425, 2024-09-20T22:07:40.290 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87805208899999998, 2024-09-20T22:07:40.290 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.290 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.290 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.290 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.291 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.291 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.291 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169974 2024-09-20T22:07:40.291 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.291 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.291 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.291 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.291 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:07:40.291 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.291 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.291 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.291 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.291 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:07:40.291 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.292 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.292 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.292 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.292 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624616+0000", 2024-09-20T22:07:40.292 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1739999999999999e-06 2024-09-20T22:07:40.292 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.292 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.292 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.292 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624626+0000", 2024-09-20T22:07:40.292 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7909999999999999e-06 2024-09-20T22:07:40.292 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.292 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.292 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.292 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624654+0000", 2024-09-20T22:07:40.293 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.8524999999999999e-05 2024-09-20T22:07:40.293 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.293 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.293 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.293 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624665+0000", 2024-09-20T22:07:40.293 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0985000000000001e-05 2024-09-20T22:07:40.293 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.293 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.293 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.293 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.705237+0000", 2024-09-20T22:07:40.293 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.080571485999999998 2024-09-20T22:07:40.293 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.293 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.293 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:40.294 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.705245+0000", 2024-09-20T22:07:40.294 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8460000000000008e-06 2024-09-20T22:07:40.294 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.294 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.294 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.294 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498359+0000", 2024-09-20T22:07:40.294 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79311461999999999 2024-09-20T22:07:40.294 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.294 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.294 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.294 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498498+0000", 2024-09-20T22:07:40.294 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013918599999999999 2024-09-20T22:07:40.294 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.294 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.295 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:07:40.295 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498751+0000", 2024-09-20T22:07:40.295 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025235999999999998 2024-09-20T22:07:40.295 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.295 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.295 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.295 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.500697+0000", 2024-09-20T22:07:40.295 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0019460969999999999 2024-09-20T22:07:40.295 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.295 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.295 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.295 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502528+0000", 2024-09-20T22:07:40.295 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00183161 2024-09-20T22:07:40.295 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.295 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.296 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.296 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502597+0000", 2024-09-20T22:07:40.296 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8761000000000001e-05 2024-09-20T22:07:40.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.296 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.296 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.296 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502666+0000", 2024-09-20T22:07:40.296 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8647999999999997e-05 2024-09-20T22:07:40.296 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.296 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.296 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.296 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.296 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:197815 7.0 7:0212144b:::benchmark_data_smithi060_78800_object64782:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-20T22:07:40.297 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:24.564029+0000", 2024-09-20T22:07:40.297 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 75.642506764999993, 2024-09-20T22:07:40.297 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.127570877, 2024-09-20T22:07:40.297 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.297 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.297 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.297 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.297 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.297 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.297 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 197815 2024-09-20T22:07:40.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.297 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.297 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.297 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.298 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.564029+0000", 2024-09-20T22:07:40.298 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.298 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.298 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.298 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.564029+0000", 2024-09-20T22:07:40.298 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.298 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.298 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.298 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.564031+0000", 2024-09-20T22:07:40.298 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.364e-06 2024-09-20T22:07:40.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.298 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.298 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.299 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.564038+0000", 2024-09-20T22:07:40.299 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.559e-06 2024-09-20T22:07:40.299 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.299 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.299 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.299 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.564069+0000", 2024-09-20T22:07:40.299 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0824999999999997e-05 2024-09-20T22:07:40.299 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.299 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.299 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.299 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.564076+0000", 2024-09-20T22:07:40.299 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6669999999999999e-06 2024-09-20T22:07:40.299 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.299 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.299 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.299 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.564130+0000", 2024-09-20T22:07:40.299 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3866999999999998e-05 2024-09-20T22:07:40.300 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.300 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.300 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.300 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.564261+0000", 2024-09-20T22:07:40.300 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013081400000000001 2024-09-20T22:07:40.300 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.300 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.300 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:07:40.300 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.564658+0000", 2024-09-20T22:07:40.300 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000397554 2024-09-20T22:07:40.300 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.300 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.300 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.300 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.567831+0000", 2024-09-20T22:07:40.301 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0031727970000000002 2024-09-20T22:07:40.301 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.301 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.301 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.301 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.691477+0000", 2024-09-20T22:07:40.301 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.123646334 2024-09-20T22:07:40.301 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.301 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.301 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.301 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.691512+0000", 2024-09-20T22:07:40.301 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5113000000000001e-05 2024-09-20T22:07:40.301 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.301 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.301 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.302 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:24.691600+0000", 2024-09-20T22:07:40.302 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.7983000000000001e-05 2024-09-20T22:07:40.302 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.302 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.302 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.302 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.302 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227618 7.0 7:0598f762:::benchmark_data_smithi060_78800_object94585:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-20T22:07:40.302 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:53.805369+0000", 2024-09-20T22:07:40.302 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 46.401167399999999, 2024-09-20T22:07:40.302 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2615457110000001, 2024-09-20T22:07:40.302 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.302 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.303 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.303 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.303 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.303 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.303 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227618 2024-09-20T22:07:40.303 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.303 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.303 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.303 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.303 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805369+0000", 2024-09-20T22:07:40.303 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.303 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.303 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.303 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.303 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805369+0000", 2024-09-20T22:07:40.304 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.304 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.304 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.304 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.304 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805371+0000", 2024-09-20T22:07:40.304 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.827e-06 2024-09-20T22:07:40.304 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.304 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.304 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.304 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805377+0000", 2024-09-20T22:07:40.304 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.161e-06 2024-09-20T22:07:40.304 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.304 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.305 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.305 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805398+0000", 2024-09-20T22:07:40.305 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1208000000000001e-05 2024-09-20T22:07:40.305 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.305 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.305 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.305 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805406+0000", 2024-09-20T22:07:40.305 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3820000000000004e-06 2024-09-20T22:07:40.305 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.305 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.305 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.305 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.813177+0000", 2024-09-20T22:07:40.305 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0077708919999999997 2024-09-20T22:07:40.305 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.305 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.305 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:40.306 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.813194+0000", 2024-09-20T22:07:40.306 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7107000000000001e-05 2024-09-20T22:07:40.306 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.306 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.306 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.306 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.813273+0000", 2024-09-20T22:07:40.306 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.000078609 2024-09-20T22:07:40.306 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.306 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.306 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:40.306 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.813290+0000", 2024-09-20T22:07:40.306 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7382999999999999e-05 2024-09-20T22:07:40.306 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.307 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.307 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.307 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.960476+0000", 2024-09-20T22:07:40.307 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1471854260000001 2024-09-20T22:07:40.307 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.307 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.307 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.307 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.960616+0000", 2024-09-20T22:07:40.307 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014035100000000001 2024-09-20T22:07:40.307 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.307 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.307 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,3", 2024-09-20T22:07:40.307 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.961013+0000", 2024-09-20T22:07:40.307 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00039649999999999999 2024-09-20T22:07:40.308 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.308 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.308 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.308 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.963511+0000", 2024-09-20T22:07:40.308 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0024986919999999998 2024-09-20T22:07:40.308 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.308 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.308 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.308 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.963553+0000", 2024-09-20T22:07:40.308 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1783999999999998e-05 2024-09-20T22:07:40.308 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.308 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.309 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.309 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.066713+0000", 2024-09-20T22:07:40.309 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.103160088 2024-09-20T22:07:40.309 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.309 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.309 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.309 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.066794+0000", 2024-09-20T22:07:40.309 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0924e-05 2024-09-20T22:07:40.309 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.309 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.309 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.309 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.066915+0000", 2024-09-20T22:07:40.309 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000120377 2024-09-20T22:07:40.309 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.310 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.310 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.310 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.310 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.310 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227635 7.b 7:da8bd73f:::benchmark_data_smithi060_78800_object94602:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e146)", 2024-09-20T22:07:40.310 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:54.820720+0000", 2024-09-20T22:07:40.310 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 45.385816523999999, 2024-09-20T22:07:40.310 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1621617440000001, 2024-09-20T22:07:40.310 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.310 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.310 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.311 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.311 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.311 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.311 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227635 2024-09-20T22:07:40.311 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.311 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.311 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.311 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.311 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820720+0000", 2024-09-20T22:07:40.311 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.311 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.311 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.311 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.312 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820720+0000", 2024-09-20T22:07:40.312 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.312 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.312 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.312 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.312 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820721+0000", 2024-09-20T22:07:40.312 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.593e-06 2024-09-20T22:07:40.312 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.312 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.312 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.312 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820728+0000", 2024-09-20T22:07:40.312 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3799999999999999e-06 2024-09-20T22:07:40.312 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.313 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.313 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.313 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820740+0000", 2024-09-20T22:07:40.313 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.279e-05 2024-09-20T22:07:40.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.313 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.313 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.313 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820749+0000", 2024-09-20T22:07:40.313 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8000000000000004e-06 2024-09-20T22:07:40.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.313 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.313 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.313 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820942+0000", 2024-09-20T22:07:40.313 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000192263 2024-09-20T22:07:40.314 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.314 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.314 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:40.314 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820953+0000", 2024-09-20T22:07:40.314 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1953000000000001e-05 2024-09-20T22:07:40.314 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.314 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.314 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.314 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.974865+0000", 2024-09-20T22:07:40.314 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.15391142 2024-09-20T22:07:40.314 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.314 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.315 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.315 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.974939+0000", 2024-09-20T22:07:40.315 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3798000000000001e-05 2024-09-20T22:07:40.315 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.315 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.315 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,6", 2024-09-20T22:07:40.315 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.975019+0000", 2024-09-20T22:07:40.315 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0692999999999995e-05 2024-09-20T22:07:40.315 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.315 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.315 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.315 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.977853+0000", 2024-09-20T22:07:40.315 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028332269999999998 2024-09-20T22:07:40.316 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.316 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.316 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.316 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982635+0000", 2024-09-20T22:07:40.316 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0047819840000000004 2024-09-20T22:07:40.316 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.316 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.316 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.316 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982688+0000", 2024-09-20T22:07:40.316 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.384e-05 2024-09-20T22:07:40.316 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.316 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.316 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.316 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982747+0000", 2024-09-20T22:07:40.317 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9004e-05 2024-09-20T22:07:40.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.317 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.317 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.317 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982881+0000", 2024-09-20T22:07:40.317 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013399899999999999 2024-09-20T22:07:40.317 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.317 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.317 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.317 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.317 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227641 7.b 7:d29062b1:::benchmark_data_smithi060_78800_object94608:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e146)", 2024-09-20T22:07:40.317 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:07:40.317 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 45.385765413000001, 2024-09-20T22:07:40.317 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2739681169999999, 2024-09-20T22:07:40.318 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.318 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.318 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.318 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.318 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.318 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.318 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227641 2024-09-20T22:07:40.318 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.318 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.318 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.318 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.318 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:07:40.318 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.319 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.319 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.319 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:07:40.319 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.319 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.319 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.319 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:07:40.319 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3900000000000005e-07 2024-09-20T22:07:40.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.319 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.319 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.319 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820775+0000", 2024-09-20T22:07:40.320 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4589999999999998e-06 2024-09-20T22:07:40.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.320 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.320 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.320 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820783+0000", 2024-09-20T22:07:40.320 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8949999999999998e-06 2024-09-20T22:07:40.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.320 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.320 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.320 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820788+0000", 2024-09-20T22:07:40.320 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5759999999999996e-06 2024-09-20T22:07:40.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.320 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.321 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.321 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820979+0000", 2024-09-20T22:07:40.321 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019122 2024-09-20T22:07:40.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.321 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.321 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:40.321 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820988+0000", 2024-09-20T22:07:40.321 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1480000000000006e-06 2024-09-20T22:07:40.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.321 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.321 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.321 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.977915+0000", 2024-09-20T22:07:40.321 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.156927834 2024-09-20T22:07:40.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.322 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.322 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.322 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.978045+0000", 2024-09-20T22:07:40.322 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012967 2024-09-20T22:07:40.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.322 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.322 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,6", 2024-09-20T22:07:40.322 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.978183+0000", 2024-09-20T22:07:40.322 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013767900000000001 2024-09-20T22:07:40.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.322 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.322 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.322 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982676+0000", 2024-09-20T22:07:40.323 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0044928210000000001 2024-09-20T22:07:40.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.323 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.323 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.323 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982891+0000", 2024-09-20T22:07:40.323 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00021542299999999999 2024-09-20T22:07:40.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.323 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.323 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.323 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.094638+0000", 2024-09-20T22:07:40.323 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.111746554 2024-09-20T22:07:40.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.323 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.323 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.323 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.094677+0000", 2024-09-20T22:07:40.324 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9069000000000002e-05 2024-09-20T22:07:40.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.324 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.324 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.324 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.094739+0000", 2024-09-20T22:07:40.324 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2230000000000006e-05 2024-09-20T22:07:40.324 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.324 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.324 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.324 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.324 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:263421 7.0 7:031b5a22:::benchmark_data_smithi060_78800_object130388:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e170)", 2024-09-20T22:07:40.324 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:37.875322+0000", 2024-09-20T22:07:40.324 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 2.3312145420000001, 2024-09-20T22:07:40.325 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.131628833, 2024-09-20T22:07:40.325 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.325 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.325 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.325 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.325 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.325 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.325 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 263421 2024-09-20T22:07:40.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.325 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.325 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.325 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.325 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.875322+0000", 2024-09-20T22:07:40.325 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.326 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.326 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.326 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.875322+0000", 2024-09-20T22:07:40.326 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.326 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.326 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.326 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.875323+0000", 2024-09-20T22:07:40.326 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.238e-06 2024-09-20T22:07:40.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.326 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.326 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.326 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.875328+0000", 2024-09-20T22:07:40.326 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8570000000000003e-06 2024-09-20T22:07:40.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.327 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.327 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.327 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.875339+0000", 2024-09-20T22:07:40.327 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1032e-05 2024-09-20T22:07:40.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.327 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.327 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.327 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.875346+0000", 2024-09-20T22:07:40.327 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5190000000000003e-06 2024-09-20T22:07:40.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.327 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.327 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.327 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.875410+0000", 2024-09-20T22:07:40.328 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3749000000000002e-05 2024-09-20T22:07:40.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.328 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.328 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.328 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.875560+0000", 2024-09-20T22:07:40.328 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014982799999999999 2024-09-20T22:07:40.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.328 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.328 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,3", 2024-09-20T22:07:40.328 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.875941+0000", 2024-09-20T22:07:40.328 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00038139800000000002 2024-09-20T22:07:40.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.328 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.328 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.328 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.877870+0000", 2024-09-20T22:07:40.329 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0019284759999999999 2024-09-20T22:07:40.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.329 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.329 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.329 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.877955+0000", 2024-09-20T22:07:40.329 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5447999999999999e-05 2024-09-20T22:07:40.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.329 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.329 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.329 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:38.006773+0000", 2024-09-20T22:07:40.329 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.12881790400000001 2024-09-20T22:07:40.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.329 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.330 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.330 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:38.006840+0000", 2024-09-20T22:07:40.330 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7070999999999995e-05 2024-09-20T22:07:40.330 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.330 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.330 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.330 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:38.006951+0000", 2024-09-20T22:07:40.330 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000110313 2024-09-20T22:07:40.330 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.330 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.330 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.330 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.330 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.330 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:263726 7.0 7:05709e6b:::benchmark_data_smithi060_78800_object130693:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e170)", 2024-09-20T22:07:40.331 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:38.235648+0000", 2024-09-20T22:07:40.331 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.9708881730000001, 2024-09-20T22:07:40.331 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.092065175999999999, 2024-09-20T22:07:40.331 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:40.331 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:40.331 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:40.331 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:40.331 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:40.331 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:40.331 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 263726 2024-09-20T22:07:40.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.331 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:40.331 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.332 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:40.332 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:38.235648+0000", 2024-09-20T22:07:40.332 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.332 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.332 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:40.332 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:38.235648+0000", 2024-09-20T22:07:40.332 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:40.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.332 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.332 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:40.332 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:38.235650+0000", 2024-09-20T22:07:40.332 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6700000000000001e-06 2024-09-20T22:07:40.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.332 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.332 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:40.333 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:38.235656+0000", 2024-09-20T22:07:40.333 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4729999999999997e-06 2024-09-20T22:07:40.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.333 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.333 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:40.333 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:38.235674+0000", 2024-09-20T22:07:40.333 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7774e-05 2024-09-20T22:07:40.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.333 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.333 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:40.333 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:38.235681+0000", 2024-09-20T22:07:40.333 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7259999999999996e-06 2024-09-20T22:07:40.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.333 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.334 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:40.334 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:38.235741+0000", 2024-09-20T22:07:40.334 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0145000000000001e-05 2024-09-20T22:07:40.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.334 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.334 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:40.334 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:38.235881+0000", 2024-09-20T22:07:40.334 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013971599999999999 2024-09-20T22:07:40.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.334 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.334 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,3", 2024-09-20T22:07:40.334 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:38.236340+0000", 2024-09-20T22:07:40.334 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000459438 2024-09-20T22:07:40.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.334 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.335 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:40.335 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:38.239065+0000", 2024-09-20T22:07:40.335 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002724771 2024-09-20T22:07:40.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.335 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.335 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.335 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:38.240633+0000", 2024-09-20T22:07:40.335 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0015682599999999999 2024-09-20T22:07:40.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.335 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.335 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:40.335 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:38.327521+0000", 2024-09-20T22:07:40.335 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.086888205999999996 2024-09-20T22:07:40.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.336 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.336 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:40.336 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:38.327594+0000", 2024-09-20T22:07:40.336 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2496000000000005e-05 2024-09-20T22:07:40.336 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:40.336 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:40.336 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:40.336 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:38.327713+0000", 2024-09-20T22:07:40.336 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011950100000000001 2024-09-20T22:07:40.336 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.336 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.336 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.336 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:40.337 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:40.337 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:07:40.337 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Removed 133030 objects 2024-09-20T22:07:40.337 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Clean up completed and total clean up time :132.964 2024-09-20T22:07:40.337 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:40.338 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:40.362 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:40.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:40.363 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:40.387 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:40.463 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:40.485 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:40.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:40.486 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:40.587 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:40.611 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:40.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:40.612 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:40.713 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:40.736 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:40.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:40.737 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:40.838 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:40.860 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:40.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:40.861 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:40.909 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:07:40.909 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:07:40.909 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:07:40.909 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:07:40.909 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:07:40.910 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:07:40.910 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:07:40.910 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:07:40.910 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:07:40.910 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:07:40.912 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:07:40.912 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:07:40.928 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:07:40.929 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:07:40.945 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:07:40.945 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:07:40.961 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:40.983 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:40.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:40.984 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:41.084 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:41.106 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:41.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:41.107 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:41.163 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:07:41.208 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:41.230 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:41.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:41.231 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:41.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:41 smithi060 ceph-mon[29694]: pgmap v230: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 156 active+clean; 269 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 629 op/s; 90/13491 objects degraded (0.667%); 612/13491 objects misplaced (4.536%); 7.4 MiB/s, 118 objects/s recovering 2024-09-20T22:07:41.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:41 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3336274152' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:41.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:41 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:41.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:41 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1044060698' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:41.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:41 smithi060 ceph-mon[40290]: pgmap v230: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 156 active+clean; 269 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 629 op/s; 90/13491 objects degraded (0.667%); 612/13491 objects misplaced (4.536%); 7.4 MiB/s, 118 objects/s recovering 2024-09-20T22:07:41.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:41 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3336274152' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:41.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:41 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:41.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:41 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1044060698' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:41.332 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:41.353 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:41.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:41.354 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:41.455 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:41.477 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:41.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:41.478 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:41.480 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:41 smithi067 ceph-mon[33511]: pgmap v230: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 1 active+remapped+backfill_toofull, 156 active+clean; 269 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 629 op/s; 90/13491 objects degraded (0.667%); 612/13491 objects misplaced (4.536%); 7.4 MiB/s, 118 objects/s recovering 2024-09-20T22:07:41.480 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:41 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3336274152' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:41.480 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:41 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:41.481 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:41 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1044060698' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:41.578 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:41.600 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:41.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:41.601 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:41.702 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:41.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:41.725 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:41.725 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:41.826 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:41.848 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:41.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:41.849 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:41.950 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:41.972 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:41.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:41.973 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:42.074 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:42.096 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:42.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:42.097 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:42.198 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:42.220 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:42.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:42.221 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:42.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:07:41 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:07:42.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:42 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:07:42.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:42 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:07:42.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:42 smithi067 ceph-mon[33511]: osdmap e171: 8 total, 8 up, 8 in 2024-09-20T22:07:42.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:42 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 90/13491 objects degraded (0.667%), 1 pg degraded (PG_DEGRADED) 2024-09-20T22:07:42.322 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:42.344 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:42.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:42.345 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:42.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:42 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:07:42.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:42 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:07:42.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:42 smithi060 ceph-mon[29694]: osdmap e171: 8 total, 8 up, 8 in 2024-09-20T22:07:42.446 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:42 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 90/13491 objects degraded (0.667%), 1 pg degraded (PG_DEGRADED) 2024-09-20T22:07:42.446 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:42 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:07:42.446 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:42 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:07:42.446 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:42 smithi060 ceph-mon[40290]: osdmap e171: 8 total, 8 up, 8 in 2024-09-20T22:07:42.446 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:42 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 90/13491 objects degraded (0.667%), 1 pg degraded (PG_DEGRADED) 2024-09-20T22:07:42.447 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:42.473 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:42.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:42.474 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:42.575 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:42.601 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:42.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:42.602 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:42.703 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:42.725 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:42.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:42.726 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:42.827 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:42.849 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:42.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:42.850 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:42.951 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:42.973 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:42.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:42.974 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:43.074 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:43.085 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T22:07:43.091 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_0 2024-09-20T22:07:43.091 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-09-20T22:07:43.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:43.097 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:43.097 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:43.198 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:43.221 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:43.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:43.222 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:43.323 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:43.344 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:07:43.345 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:43.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:43.346 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:43.353 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:43.379 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.5 to 1 (10000) 2024-09-20T22:07:43.379 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:43 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:07:43.379 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:43 smithi060 ceph-mon[40290]: pgmap v232: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 157 active+clean; 23 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.05k op/s; 34/1650 objects degraded (2.061%); 2.1 MiB/s, 33 objects/s recovering 2024-09-20T22:07:43.379 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:43 smithi060 ceph-mon[40290]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:07:43.379 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:43 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3468444178' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:07:43.379 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:43 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:07:43.380 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:43 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:07:43.380 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:43 smithi060 ceph-mon[29694]: pgmap v232: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 157 active+clean; 23 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.05k op/s; 34/1650 objects degraded (2.061%); 2.1 MiB/s, 33 objects/s recovering 2024-09-20T22:07:43.380 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:43 smithi060 ceph-mon[29694]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:07:43.380 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:43 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3468444178' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:07:43.380 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:43 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:07:43.448 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:43.465 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:43 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:07:43.465 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:43 smithi067 ceph-mon[33511]: pgmap v232: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 157 active+clean; 23 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.05k op/s; 34/1650 objects degraded (2.061%); 2.1 MiB/s, 33 objects/s recovering 2024-09-20T22:07:43.466 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:43 smithi067 ceph-mon[33511]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:07:43.466 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:43 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3468444178' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:07:43.466 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:43 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:07:43.469 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:43.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:43.470 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:43.571 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:43.595 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:43.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:43.596 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:43.697 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:43.720 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:43.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:43.721 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:43.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:07:43.821 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:43.837 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:07:43.838 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:07:43.843 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:43.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:43.844 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:43.945 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:43.967 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:43.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:43.968 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:44.069 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:44.093 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:44.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:44.093 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:44.194 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:44.216 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:44.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:44.217 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:44.318 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:44.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:44.342 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:44.342 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:44.355 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 6 1 2024-09-20T22:07:44.443 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:44.462 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:44 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:07:44.462 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:44 smithi067 ceph-mon[33511]: osdmap e172: 8 total, 8 up, 8 in 2024-09-20T22:07:44.467 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:44.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:44.467 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:44.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:07:44.490 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T22:07:44.569 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:44.590 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:44.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:44.591 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:44.596 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:44 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:07:44.596 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:44 smithi060 ceph-mon[40290]: osdmap e172: 8 total, 8 up, 8 in 2024-09-20T22:07:44.597 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:44 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:07:44.597 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:44 smithi060 ceph-mon[29694]: osdmap e172: 8 total, 8 up, 8 in 2024-09-20T22:07:44.690 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:44.692 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:44.721 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:44.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:44.723 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:44.784 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:07:44.823 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:44.845 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:44.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:44.846 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:44.947 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:44.969 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:44.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:44.970 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:45.071 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:45.093 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:45.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:45.094 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:45.195 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:45.217 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:45.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:45.218 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:45.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:07:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:07:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:07:45.298 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:07:45.299 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:07:45.319 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:45.341 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:45.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:45.342 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:45.395 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'unique_pool_0' removed 2024-09-20T22:07:45.442 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:45.464 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:45.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:45.465 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:45.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:45 smithi067 ceph-mon[33511]: pgmap v234: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 157 active+clean; 2.7 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 744 B/s rd, 0 B/s wr, 706 op/s; 34/699 objects degraded (4.864%); 1.8 MiB/s, 29 objects/s recovering 2024-09-20T22:07:45.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:45 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1918660264' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:07:45.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:45 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:07:45.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:45 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2542758242' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:07:45.566 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:45.587 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:45.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:45.588 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:45.689 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:45.711 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:45.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:45.712 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:45.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:45 smithi060 ceph-mon[29694]: pgmap v234: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 157 active+clean; 2.7 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 744 B/s rd, 0 B/s wr, 706 op/s; 34/699 objects degraded (4.864%); 1.8 MiB/s, 29 objects/s recovering 2024-09-20T22:07:45.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:45 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1918660264' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:07:45.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:45 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:07:45.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:45 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2542758242' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:07:45.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:45 smithi060 ceph-mon[40290]: pgmap v234: 158 pgs: 1 active+undersized+degraded+remapped+backfill_toofull, 157 active+clean; 2.7 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 744 B/s rd, 0 B/s wr, 706 op/s; 34/699 objects degraded (4.864%); 1.8 MiB/s, 29 objects/s recovering 2024-09-20T22:07:45.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:45 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1918660264' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:07:45.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:45 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:07:45.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:45 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2542758242' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:07:45.813 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:45.817 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.6 to 1 (10000) 2024-09-20T22:07:45.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:45.851 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:45.851 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:45.952 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:45.975 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:45.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:45.976 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:46.077 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:46.100 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:46.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:46.101 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:46.148 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T22:07:46.202 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:46.224 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:46.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:46.225 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:46.249 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:46.250 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:46.325 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:46.347 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:46.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:46.348 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:46.400 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:07:46.449 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:46.471 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:46.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:46.472 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:46.572 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:46.594 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:46.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:46.595 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:46.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:46.696 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:46 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T22:07:46.697 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:46 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2542758242' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:07:46.697 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:46 smithi067 ceph-mon[33511]: osdmap e173: 8 total, 8 up, 8 in 2024-09-20T22:07:46.697 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:46 smithi067 ceph-mon[33511]: pgmap v236: 132 pgs: 132 active+clean; 583 KiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-20T22:07:46.697 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:46 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 34/699 objects degraded (4.864%), 1 pg degraded (PG_DEGRADED) 2024-09-20T22:07:46.715 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:46 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T22:07:46.715 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:46 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2542758242' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:07:46.715 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:46 smithi060 ceph-mon[29694]: osdmap e173: 8 total, 8 up, 8 in 2024-09-20T22:07:46.716 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:46 smithi060 ceph-mon[29694]: pgmap v236: 132 pgs: 132 active+clean; 583 KiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-20T22:07:46.716 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:46 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 34/699 objects degraded (4.864%), 1 pg degraded (PG_DEGRADED) 2024-09-20T22:07:46.716 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:46 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_0", "pool2": "unique_pool_0", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T22:07:46.716 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:46 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2542758242' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:07:46.716 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:46 smithi060 ceph-mon[40290]: osdmap e173: 8 total, 8 up, 8 in 2024-09-20T22:07:46.716 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:46 smithi060 ceph-mon[40290]: pgmap v236: 132 pgs: 132 active+clean; 583 KiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-20T22:07:46.716 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:46 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 34/699 objects degraded (4.864%), 1 pg degraded (PG_DEGRADED) 2024-09-20T22:07:46.718 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:46.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:46.719 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:46.820 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:46.843 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:46.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:46.844 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:46.940 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:07:46.940 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:07:46.945 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:46.967 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:46.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:46.968 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:47.069 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:47.091 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:47.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:47.092 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:47.193 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:47.217 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:47.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:47.218 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:47.318 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:47.340 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:47.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:47.341 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:47.442 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:47.444 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 7 1 2024-09-20T22:07:47.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:47.507 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:47.508 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:47.508 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:47 smithi060 ceph-mon[29694]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-20T22:07:47.508 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:47 smithi060 ceph-mon[29694]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/699 objects degraded (4.864%), 1 pg degraded) 2024-09-20T22:07:47.508 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:47 smithi060 ceph-mon[40290]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-20T22:07:47.509 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:47 smithi060 ceph-mon[40290]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/699 objects degraded (4.864%), 1 pg degraded) 2024-09-20T22:07:47.608 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:47.631 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:47.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:47.632 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:47.659 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:47 smithi067 ceph-mon[33511]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-20T22:07:47.659 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:47 smithi067 ceph-mon[33511]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/699 objects degraded (4.864%), 1 pg degraded) 2024-09-20T22:07:47.691 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T22:07:47.711 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:07:47.712 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:07:47.730 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:07:47.730 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:07:47.733 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:47.746 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:07:47.746 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:07:47.758 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:47.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:47.759 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:47.768 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:07:47.768 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:07:47.785 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:07:47.785 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:07:47.803 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:07:47.803 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:07:47.821 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:07:47.821 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:07:47.838 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:07:47.838 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:07:47.860 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:47.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:47.884 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:47.884 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:47.950 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:07:47.986 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:48.008 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:48.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:48.009 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:48.110 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:48.132 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:48.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:48.134 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:48.171 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-20T22:07:48.172 INFO:tasks.radosbench:Beginning radosbench... 2024-09-20T22:07:48.172 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_1 2024-09-20T22:07:48.172 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool create unique_pool_1 16 2024-09-20T22:07:48.235 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:48.257 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:48.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:48.258 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:48.359 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:48.381 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:48.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:48.382 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:48.413 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:07:48.419 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:48.444 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:07:48.444 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:07:48.445 INFO:teuthology.orchestra.run.smithi067.stdout: "num_ops": 0 2024-09-20T22:07:48.445 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:07:48.483 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:48.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:48.507 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:48.508 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:48.608 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:48.630 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:48.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:48.631 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:48.732 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:48.751 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:48 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3302770188' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:07:48.752 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:48 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:07:48.752 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:48 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2620620544' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:48.752 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:48 smithi067 ceph-mon[33511]: pgmap v237: 132 pgs: 132 active+clean; 583 KiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 597 B/s rd, 0 op/s 2024-09-20T22:07:48.755 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:48.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:48.756 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:48.778 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:48 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3302770188' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:07:48.778 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:48 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:07:48.778 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:48 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2620620544' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:48.778 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:48 smithi060 ceph-mon[29694]: pgmap v237: 132 pgs: 132 active+clean; 583 KiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 597 B/s rd, 0 op/s 2024-09-20T22:07:48.778 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:48 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3302770188' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:07:48.778 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:48 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:07:48.779 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:48 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2620620544' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:48.779 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:48 smithi060 ceph-mon[40290]: pgmap v237: 132 pgs: 132 active+clean; 583 KiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 597 B/s rd, 0 op/s 2024-09-20T22:07:48.856 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:48.880 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:48.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:48.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:48.982 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:49.004 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:49.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:49.005 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:49.105 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:49.128 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:49.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:49.129 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:49.212 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T22:07:49.229 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:49.278 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:49.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:49.279 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:49.379 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:49.401 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:49.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:49.402 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:49.498 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:07:49.503 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:49.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:49 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:07:49.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:49 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:07:49.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:49 smithi067 ceph-mon[33511]: osdmap e174: 8 total, 8 up, 8 in 2024-09-20T22:07:49.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:49 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:07:49.527 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:49.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:49.528 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:49.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:49 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:07:49.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:49 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:07:49.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:49 smithi060 ceph-mon[29694]: osdmap e174: 8 total, 8 up, 8 in 2024-09-20T22:07:49.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:49 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:07:49.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:49 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:07:49.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:49 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:07:49.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:49 smithi060 ceph-mon[40290]: osdmap e174: 8 total, 8 up, 8 in 2024-09-20T22:07:49.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:49 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:07:49.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:49.653 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:49.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:49.654 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:49.755 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:49.777 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:49.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:49.778 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:49.879 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:49.902 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:49.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:49.903 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:49.945 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:50.003 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:50.006 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:07:50.006 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:07:50.007 INFO:teuthology.orchestra.run.smithi067.stdout: "complaint_time": 30, 2024-09-20T22:07:50.007 INFO:teuthology.orchestra.run.smithi067.stdout: "num_blocked_ops": 0 2024-09-20T22:07:50.007 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:07:50.027 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:50.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:50.028 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:50.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:50.151 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:50.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:50.152 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:50.253 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:50.275 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:50.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:50.276 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:50.377 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:50.399 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:50.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:50.400 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:50.410 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.7 to 1 (10000) 2024-09-20T22:07:50.501 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:50.522 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:50.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:50.523 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:50.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:50 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/861791628' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:07:50.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:50 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:07:50.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:50 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:07:50.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:50 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:07:50.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:50 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:07:50.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:50 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:07:50.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:50 smithi060 ceph-mon[40290]: pgmap v239: 132 pgs: 132 active+clean; 583 KiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 616 B/s rd, 0 op/s 2024-09-20T22:07:50.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:50 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:07:50.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:50 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:07:50.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:50 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/861791628' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:07:50.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:50 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:07:50.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:50 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:07:50.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:50 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:07:50.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:50 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:07:50.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:50 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:07:50.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:50 smithi060 ceph-mon[29694]: pgmap v239: 132 pgs: 132 active+clean; 583 KiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 616 B/s rd, 0 op/s 2024-09-20T22:07:50.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:50 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:07:50.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:50 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:07:50.619 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:50 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/861791628' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:07:50.619 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:50 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:07:50.619 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:50 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:07:50.619 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:50 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:07:50.619 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:50 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:07:50.619 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:50 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:07:50.619 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:50 smithi067 ceph-mon[33511]: pgmap v239: 132 pgs: 132 active+clean; 583 KiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 616 B/s rd, 0 op/s 2024-09-20T22:07:50.620 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:50 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:07:50.620 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:50 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:07:50.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:50.646 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:50.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:50.646 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:50.647 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T22:07:50.747 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:50.769 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:50.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:50.770 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:50.871 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:50.893 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:50.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:50.894 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:50.905 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:07:50.994 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:51.016 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:51.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:51.017 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:51.056 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-20T22:07:51.056 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:07:51.117 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:51.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:51.162 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:51.162 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:51.263 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:51.285 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:51.285 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:07:51.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:51.287 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:51.356 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:51.388 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:51.411 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:51.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:51.412 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:51.414 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'unique_pool_1' created 2024-09-20T22:07:51.448 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:07:51.448 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:07:51.448 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:07:51.448 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:07:51.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.448 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169963 7.15 7:acd7b1d6:::benchmark_data_smithi060_78800_object36930:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:07:51.448 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.440422+0000", 2024-09-20T22:07:51.448 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 113.00460400199999, 2024-09-20T22:07:51.448 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0806133010000001, 2024-09-20T22:07:51.448 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.448 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.448 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:07:51.449 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.449 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.449 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.449 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169963 2024-09-20T22:07:51.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.449 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.449 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.449 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440422+0000", 2024-09-20T22:07:51.449 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.449 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.449 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440422+0000", 2024-09-20T22:07:51.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.450 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440424+0000", 2024-09-20T22:07:51.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.762e-06 2024-09-20T22:07:51.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.450 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440432+0000", 2024-09-20T22:07:51.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4530000000000002e-06 2024-09-20T22:07:51.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.451 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440448+0000", 2024-09-20T22:07:51.451 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5084e-05 2024-09-20T22:07:51.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.451 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.451 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440456+0000", 2024-09-20T22:07:51.451 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0020000000000006e-06 2024-09-20T22:07:51.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.451 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.451 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549826+0000", 2024-09-20T22:07:51.451 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.109370678 2024-09-20T22:07:51.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.452 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.452 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549832+0000", 2024-09-20T22:07:51.452 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3850000000000001e-06 2024-09-20T22:07:51.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.452 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.452 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.503733+0000", 2024-09-20T22:07:51.452 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.95390151700000003 2024-09-20T22:07:51.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.452 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for missing object", 2024-09-20T22:07:51.452 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.503868+0000", 2024-09-20T22:07:51.452 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013515399999999999 2024-09-20T22:07:51.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.453 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.453 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.520960+0000", 2024-09-20T22:07:51.453 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.017092006 2024-09-20T22:07:51.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.453 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.453 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.453 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.521036+0000", 2024-09-20T22:07:51.453 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5259999999999994e-05 2024-09-20T22:07:51.453 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.453 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.453 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.453 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.453 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169964 7.15 7:aca939f2:::benchmark_data_smithi060_78800_object36931:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:07:51.454 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.440473+0000", 2024-09-20T22:07:51.454 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 113.00455355299999, 2024-09-20T22:07:51.454 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.081014133, 2024-09-20T22:07:51.454 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.454 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.454 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:07:51.454 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.454 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.454 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.454 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169964 2024-09-20T22:07:51.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.454 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.454 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.454 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.455 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440473+0000", 2024-09-20T22:07:51.455 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.455 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.455 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440473+0000", 2024-09-20T22:07:51.455 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.455 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.455 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440473+0000", 2024-09-20T22:07:51.455 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0499999999999999e-07 2024-09-20T22:07:51.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.455 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.456 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440476+0000", 2024-09-20T22:07:51.456 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0750000000000002e-06 2024-09-20T22:07:51.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.456 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.456 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440487+0000", 2024-09-20T22:07:51.456 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1048e-05 2024-09-20T22:07:51.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.456 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.456 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440493+0000", 2024-09-20T22:07:51.456 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.31e-06 2024-09-20T22:07:51.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.457 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549842+0000", 2024-09-20T22:07:51.457 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.109349718 2024-09-20T22:07:51.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.457 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549845+0000", 2024-09-20T22:07:51.457 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0029999999999999e-06 2024-09-20T22:07:51.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.457 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.504130+0000", 2024-09-20T22:07:51.457 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.95428438100000001 2024-09-20T22:07:51.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.458 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for missing object", 2024-09-20T22:07:51.458 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.504241+0000", 2024-09-20T22:07:51.458 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011154099999999999 2024-09-20T22:07:51.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.458 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.458 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.521422+0000", 2024-09-20T22:07:51.458 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.017180540000000001 2024-09-20T22:07:51.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.458 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.458 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.521487+0000", 2024-09-20T22:07:51.458 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5111999999999996e-05 2024-09-20T22:07:51.459 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.459 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.459 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.459 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:236437 7.11 7:8eaecf20:::benchmark_data_smithi060_78800_object103404:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:07:51.459 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:07.392155+0000", 2024-09-20T22:07:51.459 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 44.052871099000001, 2024-09-20T22:07:51.459 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.019005036, 2024-09-20T22:07:51.459 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.459 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.459 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:51.459 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.459 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.459 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.460 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 236437 2024-09-20T22:07:51.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.460 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.460 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.460 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392155+0000", 2024-09-20T22:07:51.460 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.460 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.460 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392155+0000", 2024-09-20T22:07:51.460 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.461 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.461 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392157+0000", 2024-09-20T22:07:51.461 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9599999999999999e-06 2024-09-20T22:07:51.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.462 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.462 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392162+0000", 2024-09-20T22:07:51.462 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1719999999999998e-06 2024-09-20T22:07:51.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.463 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.463 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392180+0000", 2024-09-20T22:07:51.463 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8088e-05 2024-09-20T22:07:51.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.463 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.463 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.463 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392188+0000", 2024-09-20T22:07:51.463 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9929999999999995e-06 2024-09-20T22:07:51.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.463 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.463 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.463 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392254+0000", 2024-09-20T22:07:51.463 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6137999999999996e-05 2024-09-20T22:07:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392260+0000", 2024-09-20T22:07:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7089999999999999e-06 2024-09-20T22:07:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.406485+0000", 2024-09-20T22:07:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0142251280000001 2024-09-20T22:07:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.406593+0000", 2024-09-20T22:07:51.464 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000107672 2024-09-20T22:07:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:07:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.406954+0000", 2024-09-20T22:07:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000360973 2024-09-20T22:07:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411014+0000", 2024-09-20T22:07:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00406033 2024-09-20T22:07:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.465 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411059+0000", 2024-09-20T22:07:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4907999999999998e-05 2024-09-20T22:07:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411090+0000", 2024-09-20T22:07:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0874000000000003e-05 2024-09-20T22:07:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411121+0000", 2024-09-20T22:07:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0518999999999997e-05 2024-09-20T22:07:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.466 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411160+0000", 2024-09-20T22:07:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9572e-05 2024-09-20T22:07:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:236538 7.11 7:8ee390f6:::benchmark_data_smithi060_78800_object103505:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:07:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:07.506607+0000", 2024-09-20T22:07:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 43.938419473000003, 2024-09-20T22:07:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.90916602000000002, 2024-09-20T22:07:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.467 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:51.468 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.468 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.468 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.468 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 236538 2024-09-20T22:07:51.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.468 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.468 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.468 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.468 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.506607+0000", 2024-09-20T22:07:51.468 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.468 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.468 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.468 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.506607+0000", 2024-09-20T22:07:51.469 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.469 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.469 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.506608+0000", 2024-09-20T22:07:51.469 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4759999999999999e-06 2024-09-20T22:07:51.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.469 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.469 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.506614+0000", 2024-09-20T22:07:51.469 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.344e-06 2024-09-20T22:07:51.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.469 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.469 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.506630+0000", 2024-09-20T22:07:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6305000000000001e-05 2024-09-20T22:07:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.506638+0000", 2024-09-20T22:07:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8660000000000006e-06 2024-09-20T22:07:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.506716+0000", 2024-09-20T22:07:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8469999999999999e-05 2024-09-20T22:07:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.470 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.506725+0000", 2024-09-20T22:07:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4509999999999992e-06 2024-09-20T22:07:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.409302+0000", 2024-09-20T22:07:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.90257712300000004 2024-09-20T22:07:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.409402+0000", 2024-09-20T22:07:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100457 2024-09-20T22:07:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.471 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:07:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.409647+0000", 2024-09-20T22:07:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00024485799999999998 2024-09-20T22:07:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411214+0000", 2024-09-20T22:07:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0015664159999999999 2024-09-20T22:07:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411405+0000", 2024-09-20T22:07:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019131399999999999 2024-09-20T22:07:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.415666+0000", 2024-09-20T22:07:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0042612099999999997 2024-09-20T22:07:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.415726+0000", 2024-09-20T22:07:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0269999999999997e-05 2024-09-20T22:07:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.415773+0000", 2024-09-20T22:07:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6459999999999999e-05 2024-09-20T22:07:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.473 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261924 7.17 7:ec34c39a:::benchmark_data_smithi060_78800_object128891:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:07:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.137969+0000", 2024-09-20T22:07:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 15.307056799, 2024-09-20T22:07:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0378232009999999, 2024-09-20T22:07:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:07:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.474 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261924 2024-09-20T22:07:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137969+0000", 2024-09-20T22:07:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137969+0000", 2024-09-20T22:07:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.475 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137971+0000", 2024-09-20T22:07:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.19e-06 2024-09-20T22:07:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137976+0000", 2024-09-20T22:07:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8429999999999998e-06 2024-09-20T22:07:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137987+0000", 2024-09-20T22:07:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0587e-05 2024-09-20T22:07:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.477 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.477 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137992+0000", 2024-09-20T22:07:51.477 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4090000000000002e-06 2024-09-20T22:07:51.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.477 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.477 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138048+0000", 2024-09-20T22:07:51.477 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5241999999999997e-05 2024-09-20T22:07:51.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.477 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.477 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138056+0000", 2024-09-20T22:07:51.477 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4649999999999994e-06 2024-09-20T22:07:51.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.478 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.478 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.175735+0000", 2024-09-20T22:07:51.478 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.037678871 2024-09-20T22:07:51.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.478 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.478 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.175793+0000", 2024-09-20T22:07:51.478 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7593999999999998e-05 2024-09-20T22:07:51.478 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.478 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.478 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.479 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261927 7.17 7:ed776cb1:::benchmark_data_smithi060_78800_object128894:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:07:51.479 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.138007+0000", 2024-09-20T22:07:51.479 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 15.307019371999999, 2024-09-20T22:07:51.479 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0471626510000001, 2024-09-20T22:07:51.479 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.479 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.479 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:07:51.479 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.479 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.479 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.479 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261927 2024-09-20T22:07:51.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.479 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.480 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.480 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138007+0000", 2024-09-20T22:07:51.480 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.480 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.480 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138007+0000", 2024-09-20T22:07:51.480 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.480 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.480 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138007+0000", 2024-09-20T22:07:51.480 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5600000000000001e-07 2024-09-20T22:07:51.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138009+0000", 2024-09-20T22:07:51.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6729999999999999e-06 2024-09-20T22:07:51.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138013+0000", 2024-09-20T22:07:51.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.16e-06 2024-09-20T22:07:51.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138016+0000", 2024-09-20T22:07:51.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1609999999999999e-06 2024-09-20T22:07:51.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.482 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.482 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138084+0000", 2024-09-20T22:07:51.482 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8044000000000001e-05 2024-09-20T22:07:51.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.482 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.482 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138093+0000", 2024-09-20T22:07:51.482 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.49e-06 2024-09-20T22:07:51.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.482 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.482 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177120+0000", 2024-09-20T22:07:51.482 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0390272819999999 2024-09-20T22:07:51.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.483 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for missing object", 2024-09-20T22:07:51.483 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177222+0000", 2024-09-20T22:07:51.483 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000102272 2024-09-20T22:07:51.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.483 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.483 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185103+0000", 2024-09-20T22:07:51.483 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0078803929999999994 2024-09-20T22:07:51.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.483 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.483 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185170+0000", 2024-09-20T22:07:51.483 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6820000000000001e-05 2024-09-20T22:07:51.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.484 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.484 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261940 7.17 7:e9ae652d:::benchmark_data_smithi060_78800_object128907:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:07:51.484 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.144358+0000", 2024-09-20T22:07:51.484 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 15.300668455, 2024-09-20T22:07:51.484 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.040411067, 2024-09-20T22:07:51.484 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.484 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.484 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:51.484 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.484 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.484 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.485 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261940 2024-09-20T22:07:51.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.485 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.485 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.485 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144358+0000", 2024-09-20T22:07:51.485 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.485 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.485 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144358+0000", 2024-09-20T22:07:51.485 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.486 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.486 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144359+0000", 2024-09-20T22:07:51.486 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9299999999999996e-07 2024-09-20T22:07:51.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.486 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.486 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144362+0000", 2024-09-20T22:07:51.486 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6799999999999999e-06 2024-09-20T22:07:51.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.486 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.486 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144369+0000", 2024-09-20T22:07:51.486 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8070000000000002e-06 2024-09-20T22:07:51.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.487 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.487 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.487 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144374+0000", 2024-09-20T22:07:51.487 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8999999999999997e-06 2024-09-20T22:07:51.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.487 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.487 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.487 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144416+0000", 2024-09-20T22:07:51.487 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2049999999999999e-05 2024-09-20T22:07:51.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.487 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.487 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.487 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144421+0000", 2024-09-20T22:07:51.487 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6550000000000003e-06 2024-09-20T22:07:51.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.488 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.488 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.488 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177283+0000", 2024-09-20T22:07:51.488 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0328618039999999 2024-09-20T22:07:51.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.488 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.488 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:51.488 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177468+0000", 2024-09-20T22:07:51.488 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00018551900000000001 2024-09-20T22:07:51.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.488 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.488 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:51.488 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177846+0000", 2024-09-20T22:07:51.488 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00037771700000000002 2024-09-20T22:07:51.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.489 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.489 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184561+0000", 2024-09-20T22:07:51.489 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0067147600000000002 2024-09-20T22:07:51.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.489 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:51.489 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184572+0000", 2024-09-20T22:07:51.489 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1433999999999999e-05 2024-09-20T22:07:51.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.489 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.490 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184635+0000", 2024-09-20T22:07:51.490 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3459000000000003e-05 2024-09-20T22:07:51.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.490 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:51.490 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184692+0000", 2024-09-20T22:07:51.490 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6759999999999999e-05 2024-09-20T22:07:51.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.490 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.490 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184769+0000", 2024-09-20T22:07:51.490 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6629000000000001e-05 2024-09-20T22:07:51.490 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.490 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.491 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.491 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.491 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261963 7.17 7:eacd681d:::benchmark_data_smithi060_78800_object128930:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:07:51.491 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.165356+0000", 2024-09-20T22:07:51.491 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 15.279669877, 2024-09-20T22:07:51.491 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.020053374, 2024-09-20T22:07:51.491 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.491 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.491 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:51.491 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.491 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.491 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.492 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261963 2024-09-20T22:07:51.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.492 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.492 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.492 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.492 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165356+0000", 2024-09-20T22:07:51.492 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.492 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.492 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.492 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165356+0000", 2024-09-20T22:07:51.492 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.492 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.493 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.493 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165357+0000", 2024-09-20T22:07:51.493 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.3299999999999998e-07 2024-09-20T22:07:51.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.493 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.493 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.493 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165359+0000", 2024-09-20T22:07:51.493 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.046e-06 2024-09-20T22:07:51.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.493 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.493 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.493 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165364+0000", 2024-09-20T22:07:51.493 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.502e-06 2024-09-20T22:07:51.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.494 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.494 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.494 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165369+0000", 2024-09-20T22:07:51.494 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6319999999999999e-06 2024-09-20T22:07:51.494 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.494 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.494 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.494 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.171567+0000", 2024-09-20T22:07:51.494 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061976460000000002 2024-09-20T22:07:51.494 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.494 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.494 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.494 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.171575+0000", 2024-09-20T22:07:51.494 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2390000000000002e-06 2024-09-20T22:07:51.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.495 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.495 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.495 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.180431+0000", 2024-09-20T22:07:51.495 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0088558569999999 2024-09-20T22:07:51.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.495 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.495 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:51.495 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.180570+0000", 2024-09-20T22:07:51.495 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013935799999999999 2024-09-20T22:07:51.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.495 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.495 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:51.495 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.180897+0000", 2024-09-20T22:07:51.496 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00032708599999999998 2024-09-20T22:07:51.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.496 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.496 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:51.496 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184575+0000", 2024-09-20T22:07:51.496 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0036776790000000001 2024-09-20T22:07:51.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.496 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.496 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.496 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185202+0000", 2024-09-20T22:07:51.496 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00062773099999999997 2024-09-20T22:07:51.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.496 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.496 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.497 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185253+0000", 2024-09-20T22:07:51.497 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0742999999999999e-05 2024-09-20T22:07:51.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.497 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.497 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:51.497 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185303+0000", 2024-09-20T22:07:51.497 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9339000000000003e-05 2024-09-20T22:07:51.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.497 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.497 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.497 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185410+0000", 2024-09-20T22:07:51.497 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010718299999999999 2024-09-20T22:07:51.497 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.497 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.497 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.498 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.498 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261986 7.17 7:ebad35fd:::benchmark_data_smithi060_78800_object128953:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:07:51.498 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.200930+0000", 2024-09-20T22:07:51.498 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 15.244095785000001, 2024-09-20T22:07:51.498 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98696583999999998, 2024-09-20T22:07:51.498 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.498 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.498 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:51.498 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.498 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.498 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.498 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261986 2024-09-20T22:07:51.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.499 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.499 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.499 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.499 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200930+0000", 2024-09-20T22:07:51.499 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.499 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.499 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.499 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200930+0000", 2024-09-20T22:07:51.499 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.499 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.499 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.499 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200931+0000", 2024-09-20T22:07:51.499 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8800000000000001e-07 2024-09-20T22:07:51.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.500 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.500 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.500 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200936+0000", 2024-09-20T22:07:51.500 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3939999999999998e-06 2024-09-20T22:07:51.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.500 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.500 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.500 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200946+0000", 2024-09-20T22:07:51.500 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0706999999999999e-05 2024-09-20T22:07:51.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.500 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.500 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.500 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200953+0000", 2024-09-20T22:07:51.500 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9260000000000003e-06 2024-09-20T22:07:51.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.501 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.501 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.201017+0000", 2024-09-20T22:07:51.501 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3793000000000004e-05 2024-09-20T22:07:51.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.501 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.501 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.201028+0000", 2024-09-20T22:07:51.501 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0577000000000001e-05 2024-09-20T22:07:51.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.501 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.501 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.183246+0000", 2024-09-20T22:07:51.502 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98221816100000003 2024-09-20T22:07:51.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.502 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:51.502 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.183318+0000", 2024-09-20T22:07:51.502 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1890000000000005e-05 2024-09-20T22:07:51.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.502 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:51.502 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.183503+0000", 2024-09-20T22:07:51.502 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00018529399999999999 2024-09-20T22:07:51.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.502 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:51.502 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185423+0000", 2024-09-20T22:07:51.503 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0019203390000000001 2024-09-20T22:07:51.503 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:51 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/861791628' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:07:51.503 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:51 smithi067 ceph-mon[33511]: osdmap e175: 8 total, 8 up, 8 in 2024-09-20T22:07:51.503 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:51 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3792371549' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-20T22:07:51.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.504 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.504 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.504 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185478+0000", 2024-09-20T22:07:51.504 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5041999999999999e-05 2024-09-20T22:07:51.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.505 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.505 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.505 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.187745+0000", 2024-09-20T22:07:51.505 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0022662239999999998 2024-09-20T22:07:51.505 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.505 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.505 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:51.505 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.187795+0000", 2024-09-20T22:07:51.505 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0668999999999999e-05 2024-09-20T22:07:51.505 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.505 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.505 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.505 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.187896+0000", 2024-09-20T22:07:51.505 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100936 2024-09-20T22:07:51.505 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.506 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.506 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.506 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.506 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262055 7.17 7:eca2ebed:::benchmark_data_smithi060_78800_object129022:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:07:51.506 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.253316+0000", 2024-09-20T22:07:51.506 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 15.191710646000001, 2024-09-20T22:07:51.506 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93859176700000002, 2024-09-20T22:07:51.506 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.506 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.506 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:51.506 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.506 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.506 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.507 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262055 2024-09-20T22:07:51.507 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.507 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.507 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.507 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.507 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253316+0000", 2024-09-20T22:07:51.507 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.507 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.507 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.507 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.507 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253316+0000", 2024-09-20T22:07:51.507 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.507 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.507 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.507 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.508 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253316+0000", 2024-09-20T22:07:51.508 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2999999999999999e-07 2024-09-20T22:07:51.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.508 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.508 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.508 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253320+0000", 2024-09-20T22:07:51.508 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9129999999999996e-06 2024-09-20T22:07:51.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.508 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.508 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.508 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253333+0000", 2024-09-20T22:07:51.508 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2762000000000001e-05 2024-09-20T22:07:51.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.509 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.509 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.509 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253340+0000", 2024-09-20T22:07:51.509 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0180000000000004e-06 2024-09-20T22:07:51.509 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.509 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.509 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.509 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253402+0000", 2024-09-20T22:07:51.509 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2211000000000005e-05 2024-09-20T22:07:51.509 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.509 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.509 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.509 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253409+0000", 2024-09-20T22:07:51.509 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.985e-06 2024-09-20T22:07:51.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.510 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.510 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.510 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185521+0000", 2024-09-20T22:07:51.510 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93211197400000001 2024-09-20T22:07:51.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.510 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.510 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:51.510 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185620+0000", 2024-09-20T22:07:51.510 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9162999999999999e-05 2024-09-20T22:07:51.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.510 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.510 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:51.510 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185870+0000", 2024-09-20T22:07:51.511 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00024926499999999998 2024-09-20T22:07:51.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.511 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.511 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.511 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191631+0000", 2024-09-20T22:07:51.511 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.005761375 2024-09-20T22:07:51.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.511 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.511 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:51.511 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191650+0000", 2024-09-20T22:07:51.511 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8533999999999999e-05 2024-09-20T22:07:51.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.511 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.511 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.511 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191717+0000", 2024-09-20T22:07:51.512 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7731999999999995e-05 2024-09-20T22:07:51.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.512 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.512 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:51.512 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191780+0000", 2024-09-20T22:07:51.512 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2447999999999996e-05 2024-09-20T22:07:51.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.512 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.512 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.512 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191907+0000", 2024-09-20T22:07:51.512 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012755700000000001 2024-09-20T22:07:51.512 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.512 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.512 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.513 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.513 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262136 7.17 7:e9d369cd:::benchmark_data_smithi060_78800_object129103:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:07:51.513 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.316296+0000", 2024-09-20T22:07:51.513 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 15.12873051, 2024-09-20T22:07:51.513 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87816596999999996, 2024-09-20T22:07:51.513 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.513 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.513 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:51.513 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.513 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.513 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.513 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262136 2024-09-20T22:07:51.513 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.513 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.514 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.514 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.514 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.316296+0000", 2024-09-20T22:07:51.514 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.514 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.514 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.514 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.514 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.316296+0000", 2024-09-20T22:07:51.514 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.514 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.514 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.514 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.514 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.316297+0000", 2024-09-20T22:07:51.514 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.297e-06 2024-09-20T22:07:51.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.515 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.515 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.515 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.316301+0000", 2024-09-20T22:07:51.515 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2690000000000001e-06 2024-09-20T22:07:51.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.515 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.515 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.515 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.316312+0000", 2024-09-20T22:07:51.515 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0891e-05 2024-09-20T22:07:51.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.515 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.515 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.515 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.316319+0000", 2024-09-20T22:07:51.515 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8539999999999999e-06 2024-09-20T22:07:51.516 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.516 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.516 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.516 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.316378+0000", 2024-09-20T22:07:51.516 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8956000000000003e-05 2024-09-20T22:07:51.516 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.516 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.516 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.516 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.316392+0000", 2024-09-20T22:07:51.516 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.379e-05 2024-09-20T22:07:51.516 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.516 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.516 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.516 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.187931+0000", 2024-09-20T22:07:51.516 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87153904400000004 2024-09-20T22:07:51.516 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.188062+0000", 2024-09-20T22:07:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013087299999999999 2024-09-20T22:07:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.188381+0000", 2024-09-20T22:07:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00031884699999999998 2024-09-20T22:07:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194156+0000", 2024-09-20T22:07:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0057755100000000002 2024-09-20T22:07:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194246+0000", 2024-09-20T22:07:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9977999999999998e-05 2024-09-20T22:07:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194281+0000", 2024-09-20T22:07:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5389000000000003e-05 2024-09-20T22:07:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194335+0000", 2024-09-20T22:07:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3434999999999999e-05 2024-09-20T22:07:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194462+0000", 2024-09-20T22:07:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012683700000000001 2024-09-20T22:07:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262158 7.17 7:ef5434de:::benchmark_data_smithi060_78800_object129125:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:07:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.334932+0000", 2024-09-20T22:07:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 15.110094016, 2024-09-20T22:07:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.85976260999999998, 2024-09-20T22:07:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262158 2024-09-20T22:07:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.334932+0000", 2024-09-20T22:07:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.334932+0000", 2024-09-20T22:07:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.334933+0000", 2024-09-20T22:07:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.234e-06 2024-09-20T22:07:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.334938+0000", 2024-09-20T22:07:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2939999999999999e-06 2024-09-20T22:07:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346642+0000", 2024-09-20T22:07:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.011703743000000001 2024-09-20T22:07:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346653+0000", 2024-09-20T22:07:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1065e-05 2024-09-20T22:07:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346718+0000", 2024-09-20T22:07:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5714000000000001e-05 2024-09-20T22:07:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346727+0000", 2024-09-20T22:07:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9740000000000008e-06 2024-09-20T22:07:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.189854+0000", 2024-09-20T22:07:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.84312635899999999 2024-09-20T22:07:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:51.524 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.189977+0000", 2024-09-20T22:07:51.524 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000123745 2024-09-20T22:07:51.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.524 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.524 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:51.524 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.190187+0000", 2024-09-20T22:07:51.524 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020927200000000001 2024-09-20T22:07:51.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.524 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.524 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.524 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194231+0000", 2024-09-20T22:07:51.524 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0040443370000000003 2024-09-20T22:07:51.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.524 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.524 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194475+0000", 2024-09-20T22:07:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00024404299999999999 2024-09-20T22:07:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194528+0000", 2024-09-20T22:07:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3361e-05 2024-09-20T22:07:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194584+0000", 2024-09-20T22:07:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5337999999999998e-05 2024-09-20T22:07:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194695+0000", 2024-09-20T22:07:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000111131 2024-09-20T22:07:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262159 7.17 7:e923d856:::benchmark_data_smithi060_78800_object129126:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:07:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.346686+0000", 2024-09-20T22:07:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 15.098340314, 2024-09-20T22:07:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.85042171700000002, 2024-09-20T22:07:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262159 2024-09-20T22:07:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346686+0000", 2024-09-20T22:07:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346686+0000", 2024-09-20T22:07:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346687+0000", 2024-09-20T22:07:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.361e-06 2024-09-20T22:07:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346692+0000", 2024-09-20T22:07:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4819999999999996e-06 2024-09-20T22:07:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346702+0000", 2024-09-20T22:07:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0436e-05 2024-09-20T22:07:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346708+0000", 2024-09-20T22:07:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9710000000000001e-06 2024-09-20T22:07:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346754+0000", 2024-09-20T22:07:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6001999999999998e-05 2024-09-20T22:07:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346761+0000", 2024-09-20T22:07:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9970000000000001e-06 2024-09-20T22:07:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191946+0000", 2024-09-20T22:07:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.84518505200000005 2024-09-20T22:07:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.192060+0000", 2024-09-20T22:07:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000113701 2024-09-20T22:07:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.192312+0000", 2024-09-20T22:07:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025226399999999998 2024-09-20T22:07:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194742+0000", 2024-09-20T22:07:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0024301380000000001 2024-09-20T22:07:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194798+0000", 2024-09-20T22:07:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5547999999999997e-05 2024-09-20T22:07:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.196951+0000", 2024-09-20T22:07:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0021531459999999999 2024-09-20T22:07:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.197003+0000", 2024-09-20T22:07:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1712999999999998e-05 2024-09-20T22:07:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.197108+0000", 2024-09-20T22:07:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000104906 2024-09-20T22:07:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262176 7.17 7:e9deef4b:::benchmark_data_smithi060_78800_object129143:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:07:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.358433+0000", 2024-09-20T22:07:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 15.086593344000001, 2024-09-20T22:07:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.84352511399999996, 2024-09-20T22:07:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262176 2024-09-20T22:07:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.358433+0000", 2024-09-20T22:07:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.358433+0000", 2024-09-20T22:07:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.358434+0000", 2024-09-20T22:07:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4470000000000001e-06 2024-09-20T22:07:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.358439+0000", 2024-09-20T22:07:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1479999999999997e-06 2024-09-20T22:07:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.358454+0000", 2024-09-20T22:07:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4423999999999999e-05 2024-09-20T22:07:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.358461+0000", 2024-09-20T22:07:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2080000000000003e-06 2024-09-20T22:07:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.358517+0000", 2024-09-20T22:07:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5819999999999997e-05 2024-09-20T22:07:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.358525+0000", 2024-09-20T22:07:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1559999999999995e-06 2024-09-20T22:07:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194849+0000", 2024-09-20T22:07:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.83632406999999997 2024-09-20T22:07:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194922+0000", 2024-09-20T22:07:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3261000000000002e-05 2024-09-20T22:07:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.195218+0000", 2024-09-20T22:07:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000295302 2024-09-20T22:07:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.201692+0000", 2024-09-20T22:07:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0064745840000000002 2024-09-20T22:07:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.201706+0000", 2024-09-20T22:07:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4034e-05 2024-09-20T22:07:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.201754+0000", 2024-09-20T22:07:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7614999999999997e-05 2024-09-20T22:07:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.201809+0000", 2024-09-20T22:07:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4833999999999997e-05 2024-09-20T22:07:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.201958+0000", 2024-09-20T22:07:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014921100000000001 2024-09-20T22:07:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262238 7.17 7:eff2d775:::benchmark_data_smithi060_78800_object129205:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:07:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.423242+0000", 2024-09-20T22:07:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 15.021784409, 2024-09-20T22:07:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.77916317400000001, 2024-09-20T22:07:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262238 2024-09-20T22:07:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.423242+0000", 2024-09-20T22:07:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.423242+0000", 2024-09-20T22:07:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.423242+0000", 2024-09-20T22:07:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4999999999999998e-07 2024-09-20T22:07:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.423245+0000", 2024-09-20T22:07:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7080000000000002e-06 2024-09-20T22:07:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.423254+0000", 2024-09-20T22:07:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.448e-06 2024-09-20T22:07:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.423262+0000", 2024-09-20T22:07:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0790000000000003e-06 2024-09-20T22:07:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.423353+0000", 2024-09-20T22:07:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1340000000000003e-05 2024-09-20T22:07:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.423362+0000", 2024-09-20T22:07:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.0389999999999993e-06 2024-09-20T22:07:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.197143+0000", 2024-09-20T22:07:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.77378096799999996 2024-09-20T22:07:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.197253+0000", 2024-09-20T22:07:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000110289 2024-09-20T22:07:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.197537+0000", 2024-09-20T22:07:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00028388399999999998 2024-09-20T22:07:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.201968+0000", 2024-09-20T22:07:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0044307490000000003 2024-09-20T22:07:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.202082+0000", 2024-09-20T22:07:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000113998 2024-09-20T22:07:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.202203+0000", 2024-09-20T22:07:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012076199999999999 2024-09-20T22:07:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.202248+0000", 2024-09-20T22:07:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5782999999999999e-05 2024-09-20T22:07:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.202405+0000", 2024-09-20T22:07:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000156677 2024-09-20T22:07:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262347 7.17 7:ed5b7b4e:::benchmark_data_smithi060_78800_object129314:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:07:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.540334+0000", 2024-09-20T22:07:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 14.904692733999999, 2024-09-20T22:07:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.67407287100000002, 2024-09-20T22:07:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262347 2024-09-20T22:07:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.540334+0000", 2024-09-20T22:07:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.540334+0000", 2024-09-20T22:07:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.540335+0000", 2024-09-20T22:07:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.052e-06 2024-09-20T22:07:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.540339+0000", 2024-09-20T22:07:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2289999999999997e-06 2024-09-20T22:07:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.540349+0000", 2024-09-20T22:07:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0353000000000001e-05 2024-09-20T22:07:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.540355+0000", 2024-09-20T22:07:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4140000000000002e-06 2024-09-20T22:07:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.540414+0000", 2024-09-20T22:07:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9268999999999998e-05 2024-09-20T22:07:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.540422+0000", 2024-09-20T22:07:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3340000000000002e-06 2024-09-20T22:07:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.199217+0000", 2024-09-20T22:07:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.65879443900000001 2024-09-20T22:07:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.199321+0000", 2024-09-20T22:07:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010479700000000001 2024-09-20T22:07:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.199736+0000", 2024-09-20T22:07:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00041412300000000002 2024-09-20T22:07:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.202094+0000", 2024-09-20T22:07:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0023584320000000001 2024-09-20T22:07:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.209581+0000", 2024-09-20T22:07:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0074874870000000001 2024-09-20T22:07:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214225+0000", 2024-09-20T22:07:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046432519999999996 2024-09-20T22:07:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214276+0000", 2024-09-20T22:07:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0958999999999998e-05 2024-09-20T22:07:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214406+0000", 2024-09-20T22:07:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013073100000000001 2024-09-20T22:07:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262355 7.17 7:eb2c2f7e:::benchmark_data_smithi060_78800_object129322:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:07:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.547037+0000", 2024-09-20T22:07:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 14.89798929, 2024-09-20T22:07:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.66768182899999995, 2024-09-20T22:07:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262355 2024-09-20T22:07:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.547037+0000", 2024-09-20T22:07:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.547037+0000", 2024-09-20T22:07:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.547038+0000", 2024-09-20T22:07:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.062e-06 2024-09-20T22:07:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.547042+0000", 2024-09-20T22:07:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3540000000000002e-06 2024-09-20T22:07:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.547055+0000", 2024-09-20T22:07:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3033e-05 2024-09-20T22:07:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.547063+0000", 2024-09-20T22:07:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3810000000000001e-06 2024-09-20T22:07:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.547120+0000", 2024-09-20T22:07:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6947999999999998e-05 2024-09-20T22:07:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.547127+0000", 2024-09-20T22:07:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2760000000000003e-06 2024-09-20T22:07:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.202433+0000", 2024-09-20T22:07:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.65530552500000006 2024-09-20T22:07:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.202537+0000", 2024-09-20T22:07:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000104053 2024-09-20T22:07:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.202883+0000", 2024-09-20T22:07:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00034644800000000001 2024-09-20T22:07:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214187+0000", 2024-09-20T22:07:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.011303674999999999 2024-09-20T22:07:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214475+0000", 2024-09-20T22:07:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00028836600000000002 2024-09-20T22:07:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214588+0000", 2024-09-20T22:07:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000112427 2024-09-20T22:07:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214631+0000", 2024-09-20T22:07:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3140999999999999e-05 2024-09-20T22:07:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214719+0000", 2024-09-20T22:07:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8140000000000007e-05 2024-09-20T22:07:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262398 7.17 7:eff8302a:::benchmark_data_smithi060_78800_object129365:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:07:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.601400+0000", 2024-09-20T22:07:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 14.843625911, 2024-09-20T22:07:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.61813722900000001, 2024-09-20T22:07:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262398 2024-09-20T22:07:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.601400+0000", 2024-09-20T22:07:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.601400+0000", 2024-09-20T22:07:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.601401+0000", 2024-09-20T22:07:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6899999999999996e-07 2024-09-20T22:07:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.601406+0000", 2024-09-20T22:07:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5929999999999999e-06 2024-09-20T22:07:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.601419+0000", 2024-09-20T22:07:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2692e-05 2024-09-20T22:07:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.601426+0000", 2024-09-20T22:07:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4889999999999998e-06 2024-09-20T22:07:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.601489+0000", 2024-09-20T22:07:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2947999999999994e-05 2024-09-20T22:07:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.601500+0000", 2024-09-20T22:07:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0874e-05 2024-09-20T22:07:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.207212+0000", 2024-09-20T22:07:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.60571198900000001 2024-09-20T22:07:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.207311+0000", 2024-09-20T22:07:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9314999999999994e-05 2024-09-20T22:07:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.207652+0000", 2024-09-20T22:07:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00034120500000000002 2024-09-20T22:07:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214432+0000", 2024-09-20T22:07:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0067799280000000002 2024-09-20T22:07:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214511+0000", 2024-09-20T22:07:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8968999999999996e-05 2024-09-20T22:07:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.219314+0000", 2024-09-20T22:07:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0048025580000000002 2024-09-20T22:07:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.219367+0000", 2024-09-20T22:07:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3010999999999998e-05 2024-09-20T22:07:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.219538+0000", 2024-09-20T22:07:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017068899999999999 2024-09-20T22:07:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262528 7.17 7:e945594e:::benchmark_data_smithi060_78800_object129495:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:07:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.816844+0000", 2024-09-20T22:07:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 14.628182591, 2024-09-20T22:07:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.40292846700000001, 2024-09-20T22:07:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262528 2024-09-20T22:07:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.816844+0000", 2024-09-20T22:07:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.816844+0000", 2024-09-20T22:07:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.816845+0000", 2024-09-20T22:07:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9800000000000002e-07 2024-09-20T22:07:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.816848+0000", 2024-09-20T22:07:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6289999999999999e-06 2024-09-20T22:07:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.816855+0000", 2024-09-20T22:07:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.562e-06 2024-09-20T22:07:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.816859+0000", 2024-09-20T22:07:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.087e-06 2024-09-20T22:07:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.816896+0000", 2024-09-20T22:07:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7061000000000003e-05 2024-09-20T22:07:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.816901+0000", 2024-09-20T22:07:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2619999999999998e-06 2024-09-20T22:07:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.209624+0000", 2024-09-20T22:07:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.39272300500000001 2024-09-20T22:07:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.209746+0000", 2024-09-20T22:07:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000121616 2024-09-20T22:07:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.210041+0000", 2024-09-20T22:07:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00029491199999999998 2024-09-20T22:07:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214437+0000", 2024-09-20T22:07:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0043966819999999998 2024-09-20T22:07:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214546+0000", 2024-09-20T22:07:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000108587 2024-09-20T22:07:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.219594+0000", 2024-09-20T22:07:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0050477040000000001 2024-09-20T22:07:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.219665+0000", 2024-09-20T22:07:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1067000000000002e-05 2024-09-20T22:07:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.219772+0000", 2024-09-20T22:07:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000107395 2024-09-20T22:07:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262536 7.17 7:ea4f1fb9:::benchmark_data_smithi060_78800_object129503:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:07:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.839563+0000", 2024-09-20T22:07:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 14.605462995, 2024-09-20T22:07:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.38047786300000003, 2024-09-20T22:07:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:07:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:07:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:07:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:07:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:07:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:07:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262536 2024-09-20T22:07:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:07:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:07:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.839563+0000", 2024-09-20T22:07:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:07:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.839563+0000", 2024-09-20T22:07:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:07:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:07:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.839565+0000", 2024-09-20T22:07:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6610000000000001e-06 2024-09-20T22:07:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:07:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.839571+0000", 2024-09-20T22:07:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2979999999999997e-06 2024-09-20T22:07:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:07:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.839585+0000", 2024-09-20T22:07:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3927999999999999e-05 2024-09-20T22:07:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:07:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.839594+0000", 2024-09-20T22:07:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9740000000000008e-06 2024-09-20T22:07:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.839658+0000", 2024-09-20T22:07:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3668000000000001e-05 2024-09-20T22:07:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:07:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.839666+0000", 2024-09-20T22:07:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7729999999999999e-06 2024-09-20T22:07:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:07:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.211937+0000", 2024-09-20T22:07:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.37227109600000002 2024-09-20T22:07:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:07:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.212063+0000", 2024-09-20T22:07:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012673200000000001 2024-09-20T22:07:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:07:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.212367+0000", 2024-09-20T22:07:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00030342800000000001 2024-09-20T22:07:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:07:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.216985+0000", 2024-09-20T22:07:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046182640000000004 2024-09-20T22:07:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.219833+0000", 2024-09-20T22:07:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028474920000000001 2024-09-20T22:07:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:07:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.219875+0000", 2024-09-20T22:07:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2901999999999997e-05 2024-09-20T22:07:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:07:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.219927+0000", 2024-09-20T22:07:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1649e-05 2024-09-20T22:07:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:07:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:07:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:07:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.220041+0000", 2024-09-20T22:07:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000113998 2024-09-20T22:07:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:07:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:07:51.580 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:07:51.580 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:51.603 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:51.603 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:51 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/861791628' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:07:51.604 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:51 smithi060 ceph-mon[29694]: osdmap e175: 8 total, 8 up, 8 in 2024-09-20T22:07:51.604 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:51 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3792371549' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-20T22:07:51.604 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:51 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/861791628' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:07:51.604 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:51 smithi060 ceph-mon[40290]: osdmap e175: 8 total, 8 up, 8 in 2024-09-20T22:07:51.604 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:51 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3792371549' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]: dispatch 2024-09-20T22:07:51.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:51.605 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:51.705 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:51.728 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:51.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:51.729 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:51.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:51.851 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:51.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:51.852 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:51.953 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:51.970 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:07:51.976 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:51.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:51.977 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:52.078 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:52.100 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:52.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:52.101 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:52.205 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:07:52.207 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:07:52.209 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:52.231 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:52.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:52.232 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:52.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:07:51 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:07:52.333 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:52.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:52.355 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:52.355 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:52.452 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool application enable unique_pool_1 rados --yes-i-really-mean-it || true 2024-09-20T22:07:52.456 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:52.475 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:52 smithi060 ceph-mon[29694]: 3.16 scrub starts 2024-09-20T22:07:52.475 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:52 smithi060 ceph-mon[29694]: 3.16 scrub ok 2024-09-20T22:07:52.475 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:52 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3792371549' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-20T22:07:52.475 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:52 smithi060 ceph-mon[29694]: osdmap e176: 8 total, 8 up, 8 in 2024-09-20T22:07:52.475 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:52 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:07:52.475 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:52 smithi060 ceph-mon[29694]: pgmap v242: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:07:52.475 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:52 smithi060 ceph-mon[29694]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:07:52.475 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:52 smithi060 ceph-mon[40290]: 3.16 scrub starts 2024-09-20T22:07:52.475 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:52 smithi060 ceph-mon[40290]: 3.16 scrub ok 2024-09-20T22:07:52.475 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:52 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3792371549' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-20T22:07:52.475 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:52 smithi060 ceph-mon[40290]: osdmap e176: 8 total, 8 up, 8 in 2024-09-20T22:07:52.476 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:52 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:07:52.476 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:52 smithi060 ceph-mon[40290]: pgmap v242: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:07:52.476 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:52 smithi060 ceph-mon[40290]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:07:52.478 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:52.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:52.478 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:52.579 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:52.591 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T22:07:52.601 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:52.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:52.602 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:52.703 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:52.726 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:52.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:52.727 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:52 smithi067 ceph-mon[33511]: 3.16 scrub starts 2024-09-20T22:07:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:52 smithi067 ceph-mon[33511]: 3.16 scrub ok 2024-09-20T22:07:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:52 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3792371549' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_1", "pg_num": 16}]': finished 2024-09-20T22:07:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:52 smithi067 ceph-mon[33511]: osdmap e176: 8 total, 8 up, 8 in 2024-09-20T22:07:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:52 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:07:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:52 smithi067 ceph-mon[33511]: pgmap v242: 148 pgs: 16 unknown, 132 active+clean; 583 KiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:07:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:52 smithi067 ceph-mon[33511]: from='client.15159 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:07:52.828 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:52.918 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:52.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:52.919 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:53.020 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:53.043 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:53.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:53.044 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:53.145 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:53.168 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:53.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:53.169 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:53.178 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-20T22:07:52.013900+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":1216,"num_read_kb":1062,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":613,"ondisk_log_size":613,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":414,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":1601776,"kb_used_data":14528,"kb_used_omap":15,"kb_used_meta":1586096,"kb_avail":748359440,"statfs":{"total":767960285184,"available":766320066560,"internally_reserved":0,"allocated":14876672,"data_stored":8813698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":1624163091},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":40,"apply_latency_ms":40,"commit_latency_ns":40000000,"apply_latency_ns":40000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001421"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":250,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412553+0000","last_change":"2024-09-20T22:06:48.018571+0000","last_active":"2024-09-20T22:07:50.412553+0000","last_peered":"2024-09-20T22:07:50.412553+0000","last_clean":"2024-09-20T22:07:50.412553+0000","last_became_active":"2024-09-20T22:02:50.109930+0000","last_became_peered":"2024-09-20T22:02:50.109930+0000","last_unstale":"2024-09-20T22:07:50.412553+0000","last_undegraded":"2024-09-20T22:07:50.412553+0000","last_fullsized":"2024-09-20T22:07:50.412553+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_clean_scrub_stamp":"2024-09-20T22:06:48.018157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:02.812901+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"71'9","reported_seq":264,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.411778+0000","last_change":"2024-09-20T22:06:54.831868+0000","last_active":"2024-09-20T22:07:50.411778+0000","last_peered":"2024-09-20T22:07:50.411778+0000","last_clean":"2024-09-20T22:07:50.411778+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:07:50.411778+0000","last_undegraded":"2024-09-20T22:07:50.411778+0000","last_fullsized":"2024-09-20T22:07:50.411778+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:10.928348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:10.928348+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.603090+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":127,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-20T22:07:45.387010+0000","last_change":"2024-09-20T22:06:54.843328+0000","last_active":"2024-09-20T22:07:45.387010+0000","last_peered":"2024-09-20T22:07:45.387010+0000","last_clean":"2024-09-20T22:07:45.387010+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:07:45.387010+0000","last_undegraded":"2024-09-20T22:07:45.387010+0000","last_fullsized":"2024-09-20T22:07:45.387010+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:11.178591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:11.178591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:48.255869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":239,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.411941+0000","last_change":"2024-09-20T22:07:14.535725+0000","last_active":"2024-09-20T22:07:50.411941+0000","last_peered":"2024-09-20T22:07:50.411941+0000","last_clean":"2024-09-20T22:07:50.411941+0000","last_became_active":"2024-09-20T22:06:54.830769+0000","last_became_peered":"2024-09-20T22:06:54.830769+0000","last_unstale":"2024-09-20T22:07:50.411941+0000","last_undegraded":"2024-09-20T22:07:50.411941+0000","last_fullsized":"2024-09-20T22:07:50.411941+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.535618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.535618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:36.228377+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":230,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.530911+0000","last_change":"2024-09-20T22:05:06.024320+0000","last_active":"2024-09-20T22:07:50.530911+0000","last_peered":"2024-09-20T22:07:50.530911+0000","last_clean":"2024-09-20T22:07:50.530911+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:07:50.530911+0000","last_undegraded":"2024-09-20T22:07:50.530911+0000","last_fullsized":"2024-09-20T22:07:50.530911+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:05:06.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:06.024051+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.452684+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":242,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531830+0000","last_change":"2024-09-20T22:05:04.976359+0000","last_active":"2024-09-20T22:07:50.531830+0000","last_peered":"2024-09-20T22:07:50.531830+0000","last_clean":"2024-09-20T22:07:50.531830+0000","last_became_active":"2024-09-20T22:02:50.107154+0000","last_became_peered":"2024-09-20T22:02:50.107154+0000","last_unstale":"2024-09-20T22:07:50.531830+0000","last_undegraded":"2024-09-20T22:07:50.531830+0000","last_fullsized":"2024-09-20T22:07:50.531830+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:04.976218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:04.976218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.452698+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"71'15","reported_seq":263,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531171+0000","last_change":"2024-09-20T22:04:15.767473+0000","last_active":"2024-09-20T22:07:50.531171+0000","last_peered":"2024-09-20T22:07:50.531171+0000","last_clean":"2024-09-20T22:07:50.531171+0000","last_became_active":"2024-09-20T22:02:52.282101+0000","last_became_peered":"2024-09-20T22:02:52.282101+0000","last_unstale":"2024-09-20T22:07:50.531171+0000","last_undegraded":"2024-09-20T22:07:50.531171+0000","last_fullsized":"2024-09-20T22:07:50.531171+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:04:15.767389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:15.767389+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.452653+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":244,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.411738+0000","last_change":"2024-09-20T22:07:17.494965+0000","last_active":"2024-09-20T22:07:50.411738+0000","last_peered":"2024-09-20T22:07:50.411738+0000","last_clean":"2024-09-20T22:07:50.411738+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:07:50.411738+0000","last_undegraded":"2024-09-20T22:07:50.411738+0000","last_fullsized":"2024-09-20T22:07:50.411738+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:17.494852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:17.494852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:40.846112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":258,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.533101+0000","last_change":"2024-09-20T22:06:46.616824+0000","last_active":"2024-09-20T22:07:50.533101+0000","last_peered":"2024-09-20T22:07:50.533101+0000","last_clean":"2024-09-20T22:07:50.533101+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:07:50.533101+0000","last_undegraded":"2024-09-20T22:07:50.533101+0000","last_fullsized":"2024-09-20T22:07:50.533101+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:46.616725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:46.616725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:55.441670+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"71'9","reported_seq":262,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412096+0000","last_change":"2024-09-20T22:06:09.729638+0000","last_active":"2024-09-20T22:07:50.412096+0000","last_peered":"2024-09-20T22:07:50.412096+0000","last_clean":"2024-09-20T22:07:50.412096+0000","last_became_active":"2024-09-20T22:02:52.279526+0000","last_became_peered":"2024-09-20T22:02:52.279526+0000","last_unstale":"2024-09-20T22:07:50.412096+0000","last_undegraded":"2024-09-20T22:07:50.412096+0000","last_fullsized":"2024-09-20T22:07:50.412096+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:06:09.729513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:09.729513+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.603269+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":249,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532890+0000","last_change":"2024-09-20T22:07:14.890410+0000","last_active":"2024-09-20T22:07:50.532890+0000","last_peered":"2024-09-20T22:07:50.532890+0000","last_clean":"2024-09-20T22:07:50.532890+0000","last_became_active":"2024-09-20T22:02:54.292911+0000","last_became_peered":"2024-09-20T22:02:54.292911+0000","last_unstale":"2024-09-20T22:07:50.532890+0000","last_undegraded":"2024-09-20T22:07:50.532890+0000","last_fullsized":"2024-09-20T22:07:50.532890+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.890308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.890308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:16.602195+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":239,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412038+0000","last_change":"2024-09-20T22:07:15.416312+0000","last_active":"2024-09-20T22:07:50.412038+0000","last_peered":"2024-09-20T22:07:50.412038+0000","last_clean":"2024-09-20T22:07:50.412038+0000","last_became_active":"2024-09-20T22:06:54.841095+0000","last_became_peered":"2024-09-20T22:06:54.841095+0000","last_unstale":"2024-09-20T22:07:50.412038+0000","last_undegraded":"2024-09-20T22:07:50.412038+0000","last_fullsized":"2024-09-20T22:07:50.412038+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:15.416176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.416176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:19.458530+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":263,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532107+0000","last_change":"2024-09-20T22:06:55.816225+0000","last_active":"2024-09-20T22:07:50.532107+0000","last_peered":"2024-09-20T22:07:50.532107+0000","last_clean":"2024-09-20T22:07:50.532107+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:07:50.532107+0000","last_undegraded":"2024-09-20T22:07:50.532107+0000","last_fullsized":"2024-09-20T22:07:50.532107+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:05.351834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:05.351834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"71'5","reported_seq":257,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.411941+0000","last_change":"2024-09-20T22:06:54.831254+0000","last_active":"2024-09-20T22:07:50.411941+0000","last_peered":"2024-09-20T22:07:50.411941+0000","last_clean":"2024-09-20T22:07:50.411941+0000","last_became_active":"2024-09-20T22:06:54.830844+0000","last_became_peered":"2024-09-20T22:06:54.830844+0000","last_unstale":"2024-09-20T22:07:50.411941+0000","last_undegraded":"2024-09-20T22:07:50.411941+0000","last_fullsized":"2024-09-20T22:07:50.411941+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'5","last_scrub_stamp":"2024-09-20T22:04:13.950197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:13.950197+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.603108+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":251,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531264+0000","last_change":"2024-09-20T22:07:28.654301+0000","last_active":"2024-09-20T22:07:50.531264+0000","last_peered":"2024-09-20T22:07:50.531264+0000","last_clean":"2024-09-20T22:07:50.531264+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:07:50.531264+0000","last_undegraded":"2024-09-20T22:07:50.531264+0000","last_fullsized":"2024-09-20T22:07:50.531264+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:28.654202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:28.654202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:41.331246+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":184,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.411297+0000","last_change":"2024-09-20T22:06:55.824661+0000","last_active":"2024-09-20T22:07:50.411297+0000","last_peered":"2024-09-20T22:07:50.411297+0000","last_clean":"2024-09-20T22:07:50.411297+0000","last_became_active":"2024-09-20T22:06:55.824168+0000","last_became_peered":"2024-09-20T22:06:55.824168+0000","last_unstale":"2024-09-20T22:07:50.411297+0000","last_undegraded":"2024-09-20T22:07:50.411297+0000","last_fullsized":"2024-09-20T22:07:50.411297+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:05:29.551172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":239,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412508+0000","last_change":"2024-09-20T22:06:47.667293+0000","last_active":"2024-09-20T22:07:50.412508+0000","last_peered":"2024-09-20T22:07:50.412508+0000","last_clean":"2024-09-20T22:07:50.412508+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:07:50.412508+0000","last_undegraded":"2024-09-20T22:07:50.412508+0000","last_fullsized":"2024-09-20T22:07:50.412508+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:06:47.667180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:57.957033+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":242,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412213+0000","last_change":"2024-09-20T22:05:31.552599+0000","last_active":"2024-09-20T22:07:50.412213+0000","last_peered":"2024-09-20T22:07:50.412213+0000","last_clean":"2024-09-20T22:07:50.412213+0000","last_became_active":"2024-09-20T22:02:50.111517+0000","last_became_peered":"2024-09-20T22:02:50.111517+0000","last_unstale":"2024-09-20T22:07:50.412213+0000","last_undegraded":"2024-09-20T22:07:50.412213+0000","last_fullsized":"2024-09-20T22:07:50.412213+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:31.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:31.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"71'15","reported_seq":279,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531323+0000","last_change":"2024-09-20T22:07:16.215766+0000","last_active":"2024-09-20T22:07:50.531323+0000","last_peered":"2024-09-20T22:07:50.531323+0000","last_clean":"2024-09-20T22:07:50.531323+0000","last_became_active":"2024-09-20T22:06:54.846111+0000","last_became_peered":"2024-09-20T22:06:54.846111+0000","last_unstale":"2024-09-20T22:07:50.531323+0000","last_undegraded":"2024-09-20T22:07:50.531323+0000","last_fullsized":"2024-09-20T22:07:50.531323+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:07:16.215638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:16.215638+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:18.878161+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":76,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-20T22:07:45.389503+0000","last_change":"2024-09-20T22:06:54.848811+0000","last_active":"2024-09-20T22:07:45.389503+0000","last_peered":"2024-09-20T22:07:45.389503+0000","last_clean":"2024-09-20T22:07:45.389503+0000","last_became_active":"2024-09-20T22:06:54.848505+0000","last_became_peered":"2024-09-20T22:06:54.848505+0000","last_unstale":"2024-09-20T22:07:45.389503+0000","last_undegraded":"2024-09-20T22:07:45.389503+0000","last_fullsized":"2024-09-20T22:07:45.389503+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:48.255502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":238,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531729+0000","last_change":"2024-09-20T22:06:34.944093+0000","last_active":"2024-09-20T22:07:50.531729+0000","last_peered":"2024-09-20T22:07:50.531729+0000","last_clean":"2024-09-20T22:07:50.531729+0000","last_became_active":"2024-09-20T22:02:56.347561+0000","last_became_peered":"2024-09-20T22:02:56.347561+0000","last_unstale":"2024-09-20T22:07:50.531729+0000","last_undegraded":"2024-09-20T22:07:50.531729+0000","last_fullsized":"2024-09-20T22:07:50.531729+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:34.940758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:34.940758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.452778+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":192,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.530971+0000","last_change":"2024-09-20T22:06:55.817107+0000","last_active":"2024-09-20T22:07:50.530971+0000","last_peered":"2024-09-20T22:07:50.530971+0000","last_clean":"2024-09-20T22:07:50.530971+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:07:50.530971+0000","last_undegraded":"2024-09-20T22:07:50.530971+0000","last_fullsized":"2024-09-20T22:07:50.530971+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:32.472041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:32.472041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"71'12","reported_seq":262,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531582+0000","last_change":"2024-09-20T22:06:54.831302+0000","last_active":"2024-09-20T22:07:50.531582+0000","last_peered":"2024-09-20T22:07:50.531582+0000","last_clean":"2024-09-20T22:07:50.531582+0000","last_became_active":"2024-09-20T22:06:54.830946+0000","last_became_peered":"2024-09-20T22:06:54.830946+0000","last_unstale":"2024-09-20T22:07:50.531582+0000","last_undegraded":"2024-09-20T22:07:50.531582+0000","last_fullsized":"2024-09-20T22:07:50.531582+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.452560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":234,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412006+0000","last_change":"2024-09-20T22:05:28.003302+0000","last_active":"2024-09-20T22:07:50.412006+0000","last_peered":"2024-09-20T22:07:50.412006+0000","last_clean":"2024-09-20T22:07:50.412006+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:07:50.412006+0000","last_undegraded":"2024-09-20T22:07:50.412006+0000","last_fullsized":"2024-09-20T22:07:50.412006+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.003183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.603151+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":232,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532075+0000","last_change":"2024-09-20T22:05:33.751965+0000","last_active":"2024-09-20T22:07:50.532075+0000","last_peered":"2024-09-20T22:07:50.532075+0000","last_clean":"2024-09-20T22:07:50.532075+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:07:50.532075+0000","last_undegraded":"2024-09-20T22:07:50.532075+0000","last_fullsized":"2024-09-20T22:07:50.532075+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:05:33.751883+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:52.559563+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"64'1","reported_seq":78,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-20T22:07:45.387211+0000","last_change":"2024-09-20T22:06:54.854003+0000","last_active":"2024-09-20T22:07:45.387211+0000","last_peered":"2024-09-20T22:07:45.387211+0000","last_clean":"2024-09-20T22:07:45.387211+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:07:45.387211+0000","last_undegraded":"2024-09-20T22:07:45.387211+0000","last_fullsized":"2024-09-20T22:07:45.387211+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:48.255644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"71'10","reported_seq":253,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412724+0000","last_change":"2024-09-20T22:04:56.414262+0000","last_active":"2024-09-20T22:07:50.412724+0000","last_peered":"2024-09-20T22:07:50.412724+0000","last_clean":"2024-09-20T22:07:50.412724+0000","last_became_active":"2024-09-20T22:02:52.481980+0000","last_became_peered":"2024-09-20T22:02:52.481980+0000","last_unstale":"2024-09-20T22:07:50.412724+0000","last_undegraded":"2024-09-20T22:07:50.412724+0000","last_fullsized":"2024-09-20T22:07:50.412724+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:04:56.414020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:56.414020+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"71'8","reported_seq":314,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532032+0000","last_change":"2024-09-20T22:06:41.394065+0000","last_active":"2024-09-20T22:07:50.532032+0000","last_peered":"2024-09-20T22:07:50.532032+0000","last_clean":"2024-09-20T22:07:50.532032+0000","last_became_active":"2024-09-20T22:02:54.297657+0000","last_became_peered":"2024-09-20T22:02:54.297657+0000","last_unstale":"2024-09-20T22:07:50.532032+0000","last_undegraded":"2024-09-20T22:07:50.532032+0000","last_fullsized":"2024-09-20T22:07:50.532032+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:06:41.393961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:41.393961+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:02.614370+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-20T22:07:51.409224+0000","last_change":"2024-09-20T22:07:51.409224+0000","last_active":"2024-09-20T22:07:51.409224+0000","last_peered":"2024-09-20T22:07:51.409224+0000","last_clean":"2024-09-20T22:07:51.409224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T22:07:51.409224+0000","last_undegraded":"2024-09-20T22:07:51.409224+0000","last_fullsized":"2024-09-20T22:07:51.409224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"71'15","reported_seq":157,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-20T22:07:45.386241+0000","last_change":"2024-09-20T22:06:54.860305+0000","last_active":"2024-09-20T22:07:45.386241+0000","last_peered":"2024-09-20T22:07:45.386241+0000","last_clean":"2024-09-20T22:07:45.386241+0000","last_became_active":"2024-09-20T22:06:54.859918+0000","last_became_peered":"2024-09-20T22:06:54.859918+0000","last_unstale":"2024-09-20T22:07:45.386241+0000","last_undegraded":"2024-09-20T22:07:45.386241+0000","last_fullsized":"2024-09-20T22:07:45.386241+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:04:13.235907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:13.235907+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:48.255808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":251,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531180+0000","last_change":"2024-09-20T22:07:23.292560+0000","last_active":"2024-09-20T22:07:50.531180+0000","last_peered":"2024-09-20T22:07:50.531180+0000","last_clean":"2024-09-20T22:07:50.531180+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:07:50.531180+0000","last_undegraded":"2024-09-20T22:07:50.531180+0000","last_fullsized":"2024-09-20T22:07:50.531180+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.292451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.292451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:25.739658+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"71'8","reported_seq":315,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412349+0000","last_change":"2024-09-20T22:06:04.874166+0000","last_active":"2024-09-20T22:07:50.412349+0000","last_peered":"2024-09-20T22:07:50.412349+0000","last_clean":"2024-09-20T22:07:50.412349+0000","last_became_active":"2024-09-20T22:02:54.294917+0000","last_became_peered":"2024-09-20T22:02:54.294917+0000","last_unstale":"2024-09-20T22:07:50.412349+0000","last_undegraded":"2024-09-20T22:07:50.412349+0000","last_fullsized":"2024-09-20T22:07:50.412349+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:06:04.874042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:04.874042+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.603239+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":192,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531903+0000","last_change":"2024-09-20T22:06:55.823217+0000","last_active":"2024-09-20T22:07:50.531903+0000","last_peered":"2024-09-20T22:07:50.531903+0000","last_clean":"2024-09-20T22:07:50.531903+0000","last_became_active":"2024-09-20T22:06:55.822780+0000","last_became_peered":"2024-09-20T22:06:55.822780+0000","last_unstale":"2024-09-20T22:07:50.531903+0000","last_undegraded":"2024-09-20T22:07:50.531903+0000","last_fullsized":"2024-09-20T22:07:50.531903+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:57.309262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:57.309262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:51.394983+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-20T22:07:51.409224+0000","last_change":"2024-09-20T22:07:51.409224+0000","last_active":"2024-09-20T22:07:51.409224+0000","last_peered":"2024-09-20T22:07:51.409224+0000","last_clean":"2024-09-20T22:07:51.409224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T22:07:51.409224+0000","last_undegraded":"2024-09-20T22:07:51.409224+0000","last_fullsized":"2024-09-20T22:07:51.409224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"71'18","reported_seq":270,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.530943+0000","last_change":"2024-09-20T22:04:57.967494+0000","last_active":"2024-09-20T22:07:50.530943+0000","last_peered":"2024-09-20T22:07:50.530943+0000","last_clean":"2024-09-20T22:07:50.530943+0000","last_became_active":"2024-09-20T22:02:52.482119+0000","last_became_peered":"2024-09-20T22:02:52.482119+0000","last_unstale":"2024-09-20T22:07:50.530943+0000","last_undegraded":"2024-09-20T22:07:50.530943+0000","last_fullsized":"2024-09-20T22:07:50.530943+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'18","last_scrub_stamp":"2024-09-20T22:04:57.967216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:57.967216+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.452668+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":242,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531887+0000","last_change":"2024-09-20T22:05:02.924002+0000","last_active":"2024-09-20T22:07:50.531887+0000","last_peered":"2024-09-20T22:07:50.531887+0000","last_clean":"2024-09-20T22:07:50.531887+0000","last_became_active":"2024-09-20T22:02:50.113076+0000","last_became_peered":"2024-09-20T22:02:50.113076+0000","last_unstale":"2024-09-20T22:07:50.531887+0000","last_undegraded":"2024-09-20T22:07:50.531887+0000","last_fullsized":"2024-09-20T22:07:50.531887+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:02.923565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:02.923565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.452712+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":243,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.411797+0000","last_change":"2024-09-20T22:06:05.894144+0000","last_active":"2024-09-20T22:07:50.411797+0000","last_peered":"2024-09-20T22:07:50.411797+0000","last_clean":"2024-09-20T22:07:50.411797+0000","last_became_active":"2024-09-20T22:02:54.288007+0000","last_became_peered":"2024-09-20T22:02:54.288007+0000","last_unstale":"2024-09-20T22:07:50.411797+0000","last_undegraded":"2024-09-20T22:07:50.411797+0000","last_fullsized":"2024-09-20T22:07:50.411797+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:05.894031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:05.894031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.603253+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":239,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412112+0000","last_change":"2024-09-20T22:05:29.990030+0000","last_active":"2024-09-20T22:07:50.412112+0000","last_peered":"2024-09-20T22:07:50.412112+0000","last_clean":"2024-09-20T22:07:50.412112+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:07:50.412112+0000","last_undegraded":"2024-09-20T22:07:50.412112+0000","last_fullsized":"2024-09-20T22:07:50.412112+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:29.989934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:29.989934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.603165+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-20T22:07:51.409224+0000","last_change":"2024-09-20T22:07:51.409224+0000","last_active":"2024-09-20T22:07:51.409224+0000","last_peered":"2024-09-20T22:07:51.409224+0000","last_clean":"2024-09-20T22:07:51.409224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T22:07:51.409224+0000","last_undegraded":"2024-09-20T22:07:51.409224+0000","last_fullsized":"2024-09-20T22:07:51.409224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"71'14","reported_seq":268,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.411920+0000","last_change":"2024-09-20T22:06:00.944172+0000","last_active":"2024-09-20T22:07:50.411920+0000","last_peered":"2024-09-20T22:07:50.411920+0000","last_clean":"2024-09-20T22:07:50.411920+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:07:50.411920+0000","last_undegraded":"2024-09-20T22:07:50.411920+0000","last_fullsized":"2024-09-20T22:07:50.411920+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'14","last_scrub_stamp":"2024-09-20T22:06:00.944086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.944086+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.603223+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":244,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412916+0000","last_change":"2024-09-20T22:06:54.842558+0000","last_active":"2024-09-20T22:07:50.412916+0000","last_peered":"2024-09-20T22:07:50.412916+0000","last_clean":"2024-09-20T22:07:50.412916+0000","last_became_active":"2024-09-20T22:06:54.841165+0000","last_became_peered":"2024-09-20T22:06:54.841165+0000","last_unstale":"2024-09-20T22:07:50.412916+0000","last_undegraded":"2024-09-20T22:07:50.412916+0000","last_fullsized":"2024-09-20T22:07:50.412916+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"146'10","reported_seq":314,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.530711+0000","last_change":"2024-09-20T22:07:24.333937+0000","last_active":"2024-09-20T22:07:50.530711+0000","last_peered":"2024-09-20T22:07:50.530711+0000","last_clean":"2024-09-20T22:07:50.530711+0000","last_became_active":"2024-09-20T22:06:54.847742+0000","last_became_peered":"2024-09-20T22:06:54.847742+0000","last_unstale":"2024-09-20T22:07:50.530711+0000","last_undegraded":"2024-09-20T22:07:50.530711+0000","last_fullsized":"2024-09-20T22:07:50.530711+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:07:24.333809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:24.333809+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:46.378931+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":239,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531546+0000","last_change":"2024-09-20T22:07:30.589937+0000","last_active":"2024-09-20T22:07:50.531546+0000","last_peered":"2024-09-20T22:07:50.531546+0000","last_clean":"2024-09-20T22:07:50.531546+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:07:50.531546+0000","last_undegraded":"2024-09-20T22:07:50.531546+0000","last_fullsized":"2024-09-20T22:07:50.531546+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:30.589849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:30.589849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:57.117488+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-20T22:07:51.409224+0000","last_change":"2024-09-20T22:07:51.409224+0000","last_active":"2024-09-20T22:07:51.409224+0000","last_peered":"2024-09-20T22:07:51.409224+0000","last_clean":"2024-09-20T22:07:51.409224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T22:07:51.409224+0000","last_undegraded":"2024-09-20T22:07:51.409224+0000","last_fullsized":"2024-09-20T22:07:51.409224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"71'10","reported_seq":103,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-20T22:07:45.385118+0000","last_change":"2024-09-20T22:06:54.845001+0000","last_active":"2024-09-20T22:07:45.385118+0000","last_peered":"2024-09-20T22:07:45.385118+0000","last_clean":"2024-09-20T22:07:45.385118+0000","last_became_active":"2024-09-20T22:06:54.844681+0000","last_became_peered":"2024-09-20T22:06:54.844681+0000","last_unstale":"2024-09-20T22:07:45.385118+0000","last_undegraded":"2024-09-20T22:07:45.385118+0000","last_fullsized":"2024-09-20T22:07:45.385118+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:48.255468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":258,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.533102+0000","last_change":"2024-09-20T22:06:43.598035+0000","last_active":"2024-09-20T22:07:50.533102+0000","last_peered":"2024-09-20T22:07:50.533102+0000","last_clean":"2024-09-20T22:07:50.533102+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:07:50.533102+0000","last_undegraded":"2024-09-20T22:07:50.533102+0000","last_fullsized":"2024-09-20T22:07:50.533102+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:06:43.597924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:51.777419+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"71'8","reported_seq":303,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.411574+0000","last_change":"2024-09-20T22:04:20.160515+0000","last_active":"2024-09-20T22:07:50.411574+0000","last_peered":"2024-09-20T22:07:50.411574+0000","last_clean":"2024-09-20T22:07:50.411574+0000","last_became_active":"2024-09-20T22:02:54.292819+0000","last_became_peered":"2024-09-20T22:02:54.292819+0000","last_unstale":"2024-09-20T22:07:50.411574+0000","last_undegraded":"2024-09-20T22:07:50.411574+0000","last_fullsized":"2024-09-20T22:07:50.411574+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:04:20.160405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:20.160405+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":230,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412150+0000","last_change":"2024-09-20T22:06:00.572347+0000","last_active":"2024-09-20T22:07:50.412150+0000","last_peered":"2024-09-20T22:07:50.412150+0000","last_clean":"2024-09-20T22:07:50.412150+0000","last_became_active":"2024-09-20T22:02:56.304274+0000","last_became_peered":"2024-09-20T22:02:56.304274+0000","last_unstale":"2024-09-20T22:07:50.412150+0000","last_undegraded":"2024-09-20T22:07:50.412150+0000","last_fullsized":"2024-09-20T22:07:50.412150+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:00.572243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.572243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-20T22:07:51.409224+0000","last_change":"2024-09-20T22:07:51.409224+0000","last_active":"2024-09-20T22:07:51.409224+0000","last_peered":"2024-09-20T22:07:51.409224+0000","last_clean":"2024-09-20T22:07:51.409224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T22:07:51.409224+0000","last_undegraded":"2024-09-20T22:07:51.409224+0000","last_fullsized":"2024-09-20T22:07:51.409224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"71'19","reported_seq":282,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.411833+0000","last_change":"2024-09-20T22:05:24.600933+0000","last_active":"2024-09-20T22:07:50.411833+0000","last_peered":"2024-09-20T22:07:50.411833+0000","last_clean":"2024-09-20T22:07:50.411833+0000","last_became_active":"2024-09-20T22:02:52.480077+0000","last_became_peered":"2024-09-20T22:02:52.480077+0000","last_unstale":"2024-09-20T22:07:50.411833+0000","last_undegraded":"2024-09-20T22:07:50.411833+0000","last_fullsized":"2024-09-20T22:07:50.411833+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:05:24.600795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:24.600795+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":85,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-20T22:07:45.386783+0000","last_change":"2024-09-20T22:07:14.382822+0000","last_active":"2024-09-20T22:07:45.386783+0000","last_peered":"2024-09-20T22:07:45.386783+0000","last_clean":"2024-09-20T22:07:45.386783+0000","last_became_active":"2024-09-20T22:06:54.829037+0000","last_became_peered":"2024-09-20T22:06:54.829037+0000","last_unstale":"2024-09-20T22:07:45.386783+0000","last_undegraded":"2024-09-20T22:07:45.386783+0000","last_fullsized":"2024-09-20T22:07:45.386783+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.382712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.382712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.271274+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":250,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531563+0000","last_change":"2024-09-20T22:07:30.169138+0000","last_active":"2024-09-20T22:07:50.531563+0000","last_peered":"2024-09-20T22:07:50.531563+0000","last_clean":"2024-09-20T22:07:50.531563+0000","last_became_active":"2024-09-20T22:02:54.290787+0000","last_became_peered":"2024-09-20T22:02:54.290787+0000","last_unstale":"2024-09-20T22:07:50.531563+0000","last_undegraded":"2024-09-20T22:07:50.531563+0000","last_fullsized":"2024-09-20T22:07:50.531563+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:30.169038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:30.169038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:38.467894+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":76,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-20T22:07:45.386868+0000","last_change":"2024-09-20T22:06:54.820385+0000","last_active":"2024-09-20T22:07:45.386868+0000","last_peered":"2024-09-20T22:07:45.386868+0000","last_clean":"2024-09-20T22:07:45.386868+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:07:45.386868+0000","last_undegraded":"2024-09-20T22:07:45.386868+0000","last_fullsized":"2024-09-20T22:07:45.386868+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:48.255676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-20T22:07:51.409224+0000","last_change":"2024-09-20T22:07:51.409224+0000","last_active":"2024-09-20T22:07:51.409224+0000","last_peered":"2024-09-20T22:07:51.409224+0000","last_clean":"2024-09-20T22:07:51.409224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T22:07:51.409224+0000","last_undegraded":"2024-09-20T22:07:51.409224+0000","last_fullsized":"2024-09-20T22:07:51.409224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"146'30","reported_seq":149,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-20T22:07:45.386920+0000","last_change":"2024-09-20T22:06:54.854264+0000","last_active":"2024-09-20T22:07:45.386920+0000","last_peered":"2024-09-20T22:07:45.386920+0000","last_clean":"2024-09-20T22:07:45.386920+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:07:45.386920+0000","last_undegraded":"2024-09-20T22:07:45.386920+0000","last_fullsized":"2024-09-20T22:07:45.386920+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:48.255424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":251,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412238+0000","last_change":"2024-09-20T22:05:58.966616+0000","last_active":"2024-09-20T22:07:50.412238+0000","last_peered":"2024-09-20T22:07:50.412238+0000","last_clean":"2024-09-20T22:07:50.412238+0000","last_became_active":"2024-09-20T22:02:50.108457+0000","last_became_peered":"2024-09-20T22:02:50.108457+0000","last_unstale":"2024-09-20T22:07:50.412238+0000","last_undegraded":"2024-09-20T22:07:50.412238+0000","last_fullsized":"2024-09-20T22:07:50.412238+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:58.966541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:58.966541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.603208+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":262,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532422+0000","last_change":"2024-09-20T22:07:21.886172+0000","last_active":"2024-09-20T22:07:50.532422+0000","last_peered":"2024-09-20T22:07:50.532422+0000","last_clean":"2024-09-20T22:07:50.532422+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:07:50.532422+0000","last_undegraded":"2024-09-20T22:07:50.532422+0000","last_fullsized":"2024-09-20T22:07:50.532422+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_clean_scrub_stamp":"2024-09-20T22:07:21.886038+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.052265+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":243,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412450+0000","last_change":"2024-09-20T22:07:17.566774+0000","last_active":"2024-09-20T22:07:50.412450+0000","last_peered":"2024-09-20T22:07:50.412450+0000","last_clean":"2024-09-20T22:07:50.412450+0000","last_became_active":"2024-09-20T22:06:54.841419+0000","last_became_peered":"2024-09-20T22:06:54.841419+0000","last_unstale":"2024-09-20T22:07:50.412450+0000","last_undegraded":"2024-09-20T22:07:50.412450+0000","last_fullsized":"2024-09-20T22:07:50.412450+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:17.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:17.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:32.923210+0000","scrub_duration":120,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":230,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531647+0000","last_change":"2024-09-20T22:05:34.954357+0000","last_active":"2024-09-20T22:07:50.531647+0000","last_peered":"2024-09-20T22:07:50.531647+0000","last_clean":"2024-09-20T22:07:50.531647+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:07:50.531647+0000","last_undegraded":"2024-09-20T22:07:50.531647+0000","last_fullsized":"2024-09-20T22:07:50.531647+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:05:34.953944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.452733+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-20T22:07:51.409224+0000","last_change":"2024-09-20T22:07:51.409224+0000","last_active":"2024-09-20T22:07:51.409224+0000","last_peered":"2024-09-20T22:07:51.409224+0000","last_clean":"2024-09-20T22:07:51.409224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T22:07:51.409224+0000","last_undegraded":"2024-09-20T22:07:51.409224+0000","last_fullsized":"2024-09-20T22:07:51.409224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"71'13","reported_seq":115,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-20T22:07:45.386443+0000","last_change":"2024-09-20T22:06:54.851246+0000","last_active":"2024-09-20T22:07:45.386443+0000","last_peered":"2024-09-20T22:07:45.386443+0000","last_clean":"2024-09-20T22:07:45.386443+0000","last_became_active":"2024-09-20T22:06:54.850696+0000","last_became_peered":"2024-09-20T22:06:54.850696+0000","last_unstale":"2024-09-20T22:07:45.386443+0000","last_undegraded":"2024-09-20T22:07:45.386443+0000","last_fullsized":"2024-09-20T22:07:45.386443+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:48.255572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":85,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-20T22:07:45.386321+0000","last_change":"2024-09-20T22:07:20.341524+0000","last_active":"2024-09-20T22:07:45.386321+0000","last_peered":"2024-09-20T22:07:45.386321+0000","last_clean":"2024-09-20T22:07:45.386321+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:07:45.386321+0000","last_undegraded":"2024-09-20T22:07:45.386321+0000","last_fullsized":"2024-09-20T22:07:45.386321+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:20.338388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:20.338388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:40.900548+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":268,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.530659+0000","last_change":"2024-09-20T22:06:47.015379+0000","last_active":"2024-09-20T22:07:50.530659+0000","last_peered":"2024-09-20T22:07:50.530659+0000","last_clean":"2024-09-20T22:07:50.530659+0000","last_became_active":"2024-09-20T22:02:49.099508+0000","last_became_peered":"2024-09-20T22:02:49.099508+0000","last_unstale":"2024-09-20T22:07:50.530659+0000","last_undegraded":"2024-09-20T22:07:50.530659+0000","last_fullsized":"2024-09-20T22:07:50.530659+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:06:47.015270+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:34.967504+0000","last_clean_scrub_stamp":"2024-09-20T22:06:47.015270+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:09.454334+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":242,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531918+0000","last_change":"2024-09-20T22:06:36.901630+0000","last_active":"2024-09-20T22:07:50.531918+0000","last_peered":"2024-09-20T22:07:50.531918+0000","last_clean":"2024-09-20T22:07:50.531918+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:07:50.531918+0000","last_undegraded":"2024-09-20T22:07:50.531918+0000","last_fullsized":"2024-09-20T22:07:50.531918+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:36.901522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:36.901522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:00.298120+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532450+0000","last_change":"2024-09-20T22:05:28.780920+0000","last_active":"2024-09-20T22:07:50.532450+0000","last_peered":"2024-09-20T22:07:50.532450+0000","last_clean":"2024-09-20T22:07:50.532450+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:07:50.532450+0000","last_undegraded":"2024-09-20T22:07:50.532450+0000","last_fullsized":"2024-09-20T22:07:50.532450+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.780758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.780758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:52.559531+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-20T22:07:51.409224+0000","last_change":"2024-09-20T22:07:51.409224+0000","last_active":"2024-09-20T22:07:51.409224+0000","last_peered":"2024-09-20T22:07:51.409224+0000","last_clean":"2024-09-20T22:07:51.409224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T22:07:51.409224+0000","last_undegraded":"2024-09-20T22:07:51.409224+0000","last_fullsized":"2024-09-20T22:07:51.409224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"71'12","reported_seq":265,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412794+0000","last_change":"2024-09-20T22:07:29.614630+0000","last_active":"2024-09-20T22:07:50.412794+0000","last_peered":"2024-09-20T22:07:50.412794+0000","last_clean":"2024-09-20T22:07:50.412794+0000","last_became_active":"2024-09-20T22:06:54.848670+0000","last_became_peered":"2024-09-20T22:06:54.848670+0000","last_unstale":"2024-09-20T22:07:50.412794+0000","last_undegraded":"2024-09-20T22:07:50.412794+0000","last_fullsized":"2024-09-20T22:07:50.412794+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:29.614551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:29.614551+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:50.884056+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":250,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.411649+0000","last_change":"2024-09-20T22:06:44.862452+0000","last_active":"2024-09-20T22:07:50.411649+0000","last_peered":"2024-09-20T22:07:50.411649+0000","last_clean":"2024-09-20T22:07:50.411649+0000","last_became_active":"2024-09-20T22:02:50.110448+0000","last_became_peered":"2024-09-20T22:02:50.110448+0000","last_unstale":"2024-09-20T22:07:50.411649+0000","last_undegraded":"2024-09-20T22:07:50.411649+0000","last_fullsized":"2024-09-20T22:07:50.411649+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:06:44.862338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:59.230368+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":556,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531098+0000","last_change":"2024-09-20T22:07:16.590582+0000","last_active":"2024-09-20T22:07:50.531098+0000","last_peered":"2024-09-20T22:07:50.531098+0000","last_clean":"2024-09-20T22:07:50.531098+0000","last_became_active":"2024-09-20T22:06:54.844454+0000","last_became_peered":"2024-09-20T22:06:54.844454+0000","last_unstale":"2024-09-20T22:07:50.531098+0000","last_undegraded":"2024-09-20T22:07:50.531098+0000","last_fullsized":"2024-09-20T22:07:50.531098+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:07:16.590475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:07:16.590475+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.092001+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":287,"num_read_kb":282,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":243,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532768+0000","last_change":"2024-09-20T22:07:22.884981+0000","last_active":"2024-09-20T22:07:50.532768+0000","last_peered":"2024-09-20T22:07:50.532768+0000","last_clean":"2024-09-20T22:07:50.532768+0000","last_became_active":"2024-09-20T22:06:54.841596+0000","last_became_peered":"2024-09-20T22:06:54.841596+0000","last_unstale":"2024-09-20T22:07:50.532768+0000","last_undegraded":"2024-09-20T22:07:50.532768+0000","last_fullsized":"2024-09-20T22:07:50.532768+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:22.884907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.884907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:41.462917+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":76,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-20T22:07:45.386602+0000","last_change":"2024-09-20T22:06:54.853957+0000","last_active":"2024-09-20T22:07:45.386602+0000","last_peered":"2024-09-20T22:07:45.386602+0000","last_clean":"2024-09-20T22:07:45.386602+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:07:45.386602+0000","last_undegraded":"2024-09-20T22:07:45.386602+0000","last_fullsized":"2024-09-20T22:07:45.386602+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:48.255533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-20T22:07:51.409224+0000","last_change":"2024-09-20T22:07:51.409224+0000","last_active":"2024-09-20T22:07:51.409224+0000","last_peered":"2024-09-20T22:07:51.409224+0000","last_clean":"2024-09-20T22:07:51.409224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T22:07:51.409224+0000","last_undegraded":"2024-09-20T22:07:51.409224+0000","last_fullsized":"2024-09-20T22:07:51.409224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"71'16","reported_seq":329,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531928+0000","last_change":"2024-09-20T22:05:58.335330+0000","last_active":"2024-09-20T22:07:50.531928+0000","last_peered":"2024-09-20T22:07:50.531928+0000","last_clean":"2024-09-20T22:07:50.531928+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:07:50.531928+0000","last_undegraded":"2024-09-20T22:07:50.531928+0000","last_fullsized":"2024-09-20T22:07:50.531928+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'16","last_scrub_stamp":"2024-09-20T22:05:58.335199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:58.335199+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:51.395022+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":250,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531180+0000","last_change":"2024-09-20T22:06:48.105501+0000","last_active":"2024-09-20T22:07:50.531180+0000","last_peered":"2024-09-20T22:07:50.531180+0000","last_clean":"2024-09-20T22:07:50.531180+0000","last_became_active":"2024-09-20T22:02:50.106011+0000","last_became_peered":"2024-09-20T22:02:50.106011+0000","last_unstale":"2024-09-20T22:07:50.531180+0000","last_undegraded":"2024-09-20T22:07:50.531180+0000","last_fullsized":"2024-09-20T22:07:50.531180+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:48.105403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:48.105403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:17.663438+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":498,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531288+0000","last_change":"2024-09-20T22:05:02.727025+0000","last_active":"2024-09-20T22:07:50.531288+0000","last_peered":"2024-09-20T22:07:50.531288+0000","last_clean":"2024-09-20T22:07:50.531288+0000","last_became_active":"2024-09-20T22:02:45.068482+0000","last_became_peered":"2024-09-20T22:02:45.068482+0000","last_unstale":"2024-09-20T22:07:50.531288+0000","last_undegraded":"2024-09-20T22:07:50.531288+0000","last_fullsized":"2024-09-20T22:07:50.531288+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:05:02.726765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:05:02.726765+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:52.559468+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":243,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532231+0000","last_change":"2024-09-20T22:05:56.679105+0000","last_active":"2024-09-20T22:07:50.532231+0000","last_peered":"2024-09-20T22:07:50.532231+0000","last_clean":"2024-09-20T22:07:50.532231+0000","last_became_active":"2024-09-20T22:02:54.298021+0000","last_became_peered":"2024-09-20T22:02:54.298021+0000","last_unstale":"2024-09-20T22:07:50.532231+0000","last_undegraded":"2024-09-20T22:07:50.532231+0000","last_fullsized":"2024-09-20T22:07:50.532231+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:56.678980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:56.678980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:52.559591+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":252,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532585+0000","last_change":"2024-09-20T22:06:47.664985+0000","last_active":"2024-09-20T22:07:50.532585+0000","last_peered":"2024-09-20T22:07:50.532585+0000","last_clean":"2024-09-20T22:07:50.532585+0000","last_became_active":"2024-09-20T22:04:21.237725+0000","last_became_peered":"2024-09-20T22:04:21.237725+0000","last_unstale":"2024-09-20T22:07:50.532585+0000","last_undegraded":"2024-09-20T22:07:50.532585+0000","last_fullsized":"2024-09-20T22:07:50.532585+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:47.664877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:06:47.664877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:10.795971+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-20T22:07:51.409224+0000","last_change":"2024-09-20T22:07:51.409224+0000","last_active":"2024-09-20T22:07:51.409224+0000","last_peered":"2024-09-20T22:07:51.409224+0000","last_clean":"2024-09-20T22:07:51.409224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T22:07:51.409224+0000","last_undegraded":"2024-09-20T22:07:51.409224+0000","last_fullsized":"2024-09-20T22:07:51.409224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"71'11","reported_seq":267,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412530+0000","last_change":"2024-09-20T22:07:13.511239+0000","last_active":"2024-09-20T22:07:50.412530+0000","last_peered":"2024-09-20T22:07:50.412530+0000","last_clean":"2024-09-20T22:07:50.412530+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:07:50.412530+0000","last_undegraded":"2024-09-20T22:07:50.412530+0000","last_fullsized":"2024-09-20T22:07:50.412530+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:13.511159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:13.511159+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:38.687052+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":243,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412057+0000","last_change":"2024-09-20T22:05:26.006756+0000","last_active":"2024-09-20T22:07:50.412057+0000","last_peered":"2024-09-20T22:07:50.412057+0000","last_clean":"2024-09-20T22:07:50.412057+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:07:50.412057+0000","last_undegraded":"2024-09-20T22:07:50.412057+0000","last_fullsized":"2024-09-20T22:07:50.412057+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:26.006692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:26.006692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.603179+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":242,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532424+0000","last_change":"2024-09-20T22:05:57.397917+0000","last_active":"2024-09-20T22:07:50.532424+0000","last_peered":"2024-09-20T22:07:50.532424+0000","last_clean":"2024-09-20T22:07:50.532424+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:07:50.532424+0000","last_undegraded":"2024-09-20T22:07:50.532424+0000","last_fullsized":"2024-09-20T22:07:50.532424+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:57.397807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:57.397807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":238,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531318+0000","last_change":"2024-09-20T22:06:45.030549+0000","last_active":"2024-09-20T22:07:50.531318+0000","last_peered":"2024-09-20T22:07:50.531318+0000","last_clean":"2024-09-20T22:07:50.531318+0000","last_became_active":"2024-09-20T22:02:56.349376+0000","last_became_peered":"2024-09-20T22:02:56.349376+0000","last_unstale":"2024-09-20T22:07:50.531318+0000","last_undegraded":"2024-09-20T22:07:50.531318+0000","last_fullsized":"2024-09-20T22:07:50.531318+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.030289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.030289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:55.774314+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-20T22:07:51.409224+0000","last_change":"2024-09-20T22:07:51.409224+0000","last_active":"2024-09-20T22:07:51.409224+0000","last_peered":"2024-09-20T22:07:51.409224+0000","last_clean":"2024-09-20T22:07:51.409224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T22:07:51.409224+0000","last_undegraded":"2024-09-20T22:07:51.409224+0000","last_fullsized":"2024-09-20T22:07:51.409224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"71'17","reported_seq":276,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412352+0000","last_change":"2024-09-20T22:06:54.824419+0000","last_active":"2024-09-20T22:07:50.412352+0000","last_peered":"2024-09-20T22:07:50.412352+0000","last_clean":"2024-09-20T22:07:50.412352+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:07:50.412352+0000","last_undegraded":"2024-09-20T22:07:50.412352+0000","last_fullsized":"2024-09-20T22:07:50.412352+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'17","last_scrub_stamp":"2024-09-20T22:04:12.942792+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:12.942792+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.603030+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":251,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531827+0000","last_change":"2024-09-20T22:07:21.124897+0000","last_active":"2024-09-20T22:07:50.531827+0000","last_peered":"2024-09-20T22:07:50.531827+0000","last_clean":"2024-09-20T22:07:50.531827+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:07:50.531827+0000","last_undegraded":"2024-09-20T22:07:50.531827+0000","last_fullsized":"2024-09-20T22:07:50.531827+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:21.124789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:21.124789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:29.000115+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":84,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-20T22:07:45.386693+0000","last_change":"2024-09-20T22:07:19.376008+0000","last_active":"2024-09-20T22:07:45.386693+0000","last_peered":"2024-09-20T22:07:45.386693+0000","last_clean":"2024-09-20T22:07:45.386693+0000","last_became_active":"2024-09-20T22:06:54.848275+0000","last_became_peered":"2024-09-20T22:06:54.848275+0000","last_unstale":"2024-09-20T22:07:45.386693+0000","last_undegraded":"2024-09-20T22:07:45.386693+0000","last_fullsized":"2024-09-20T22:07:45.386693+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:19.375867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.375867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:49.161984+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":238,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531643+0000","last_change":"2024-09-20T22:06:35.916240+0000","last_active":"2024-09-20T22:07:50.531643+0000","last_peered":"2024-09-20T22:07:50.531643+0000","last_clean":"2024-09-20T22:07:50.531643+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:07:50.531643+0000","last_undegraded":"2024-09-20T22:07:50.531643+0000","last_fullsized":"2024-09-20T22:07:50.531643+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:35.916120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:35.916120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.228674+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-20T22:07:51.409224+0000","last_change":"2024-09-20T22:07:51.409224+0000","last_active":"2024-09-20T22:07:51.409224+0000","last_peered":"2024-09-20T22:07:51.409224+0000","last_clean":"2024-09-20T22:07:51.409224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T22:07:51.409224+0000","last_undegraded":"2024-09-20T22:07:51.409224+0000","last_fullsized":"2024-09-20T22:07:51.409224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"71'10","reported_seq":254,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412294+0000","last_change":"2024-09-20T22:05:27.048378+0000","last_active":"2024-09-20T22:07:50.412294+0000","last_peered":"2024-09-20T22:07:50.412294+0000","last_clean":"2024-09-20T22:07:50.412294+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:07:50.412294+0000","last_undegraded":"2024-09-20T22:07:50.412294+0000","last_fullsized":"2024-09-20T22:07:50.412294+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:05:27.048215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:27.048215+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.603194+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"64'1","reported_seq":256,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531253+0000","last_change":"2024-09-20T22:06:33.966005+0000","last_active":"2024-09-20T22:07:50.531253+0000","last_peered":"2024-09-20T22:07:50.531253+0000","last_clean":"2024-09-20T22:07:50.531253+0000","last_became_active":"2024-09-20T22:02:50.110446+0000","last_became_peered":"2024-09-20T22:02:50.110446+0000","last_unstale":"2024-09-20T22:07:50.531253+0000","last_undegraded":"2024-09-20T22:07:50.531253+0000","last_fullsized":"2024-09-20T22:07:50.531253+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:06:33.965866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:33.965866+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.452794+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"139'9","reported_seq":320,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531049+0000","last_change":"2024-09-20T22:06:40.910132+0000","last_active":"2024-09-20T22:07:50.531049+0000","last_peered":"2024-09-20T22:07:50.531049+0000","last_clean":"2024-09-20T22:07:50.531049+0000","last_became_active":"2024-09-20T22:02:54.294458+0000","last_became_peered":"2024-09-20T22:02:54.294458+0000","last_unstale":"2024-09-20T22:07:50.531049+0000","last_undegraded":"2024-09-20T22:07:50.531049+0000","last_fullsized":"2024-09-20T22:07:50.531049+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'9","last_scrub_stamp":"2024-09-20T22:06:40.910011+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:06:40.910011+0000","last_clean_scrub_stamp":"2024-09-20T22:06:40.910011+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:51.594963+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":240,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412238+0000","last_change":"2024-09-20T22:07:23.482514+0000","last_active":"2024-09-20T22:07:50.412238+0000","last_peered":"2024-09-20T22:07:50.412238+0000","last_clean":"2024-09-20T22:07:50.412238+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:07:50.412238+0000","last_undegraded":"2024-09-20T22:07:50.412238+0000","last_fullsized":"2024-09-20T22:07:50.412238+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.482372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.886444+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-20T22:07:51.409224+0000","last_change":"2024-09-20T22:07:51.409224+0000","last_active":"2024-09-20T22:07:51.409224+0000","last_peered":"2024-09-20T22:07:51.409224+0000","last_clean":"2024-09-20T22:07:51.409224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T22:07:51.409224+0000","last_undegraded":"2024-09-20T22:07:51.409224+0000","last_fullsized":"2024-09-20T22:07:51.409224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"71'9","reported_seq":263,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.411739+0000","last_change":"2024-09-20T22:06:54.848100+0000","last_active":"2024-09-20T22:07:50.411739+0000","last_peered":"2024-09-20T22:07:50.411739+0000","last_clean":"2024-09-20T22:07:50.411739+0000","last_became_active":"2024-09-20T22:06:54.847770+0000","last_became_peered":"2024-09-20T22:06:54.847770+0000","last_unstale":"2024-09-20T22:07:50.411739+0000","last_undegraded":"2024-09-20T22:07:50.411739+0000","last_fullsized":"2024-09-20T22:07:50.411739+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:03:52.011201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:03:52.011201+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":256,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532209+0000","last_change":"2024-09-20T22:06:36.582728+0000","last_active":"2024-09-20T22:07:50.532209+0000","last_peered":"2024-09-20T22:07:50.532209+0000","last_clean":"2024-09-20T22:07:50.532209+0000","last_became_active":"2024-09-20T22:02:50.112682+0000","last_became_peered":"2024-09-20T22:02:50.112682+0000","last_unstale":"2024-09-20T22:07:50.532209+0000","last_undegraded":"2024-09-20T22:07:50.532209+0000","last_fullsized":"2024-09-20T22:07:50.532209+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:06:36.582537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:36.582537+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:51.777438+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":250,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531351+0000","last_change":"2024-09-20T22:07:22.203673+0000","last_active":"2024-09-20T22:07:50.531351+0000","last_peered":"2024-09-20T22:07:50.531351+0000","last_clean":"2024-09-20T22:07:50.531351+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:07:50.531351+0000","last_undegraded":"2024-09-20T22:07:50.531351+0000","last_fullsized":"2024-09-20T22:07:50.531351+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:22.203576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.203576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.064543+0000","scrub_duration":49,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":240,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.411561+0000","last_change":"2024-09-20T22:06:55.824784+0000","last_active":"2024-09-20T22:07:50.411561+0000","last_peered":"2024-09-20T22:07:50.411561+0000","last_clean":"2024-09-20T22:07:50.411561+0000","last_became_active":"2024-09-20T22:06:55.824229+0000","last_became_peered":"2024-09-20T22:06:55.824229+0000","last_unstale":"2024-09-20T22:07:50.411561+0000","last_undegraded":"2024-09-20T22:07:50.411561+0000","last_fullsized":"2024-09-20T22:07:50.411561+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:17.171523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:17.171523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-20T22:07:51.409224+0000","last_change":"2024-09-20T22:07:51.409224+0000","last_active":"2024-09-20T22:07:51.409224+0000","last_peered":"2024-09-20T22:07:51.409224+0000","last_clean":"2024-09-20T22:07:51.409224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T22:07:51.409224+0000","last_undegraded":"2024-09-20T22:07:51.409224+0000","last_fullsized":"2024-09-20T22:07:51.409224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"71'19","reported_seq":278,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531651+0000","last_change":"2024-09-20T22:07:25.155015+0000","last_active":"2024-09-20T22:07:50.531651+0000","last_peered":"2024-09-20T22:07:50.531651+0000","last_clean":"2024-09-20T22:07:50.531651+0000","last_became_active":"2024-09-20T22:06:54.829170+0000","last_became_peered":"2024-09-20T22:06:54.829170+0000","last_unstale":"2024-09-20T22:07:50.531651+0000","last_undegraded":"2024-09-20T22:07:50.531651+0000","last_fullsized":"2024-09-20T22:07:50.531651+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:07:25.154919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:25.154919+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:43.954499+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":243,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532664+0000","last_change":"2024-09-20T22:05:05.659544+0000","last_active":"2024-09-20T22:07:50.532664+0000","last_peered":"2024-09-20T22:07:50.532664+0000","last_clean":"2024-09-20T22:07:50.532664+0000","last_became_active":"2024-09-20T22:02:50.108712+0000","last_became_peered":"2024-09-20T22:02:50.108712+0000","last_unstale":"2024-09-20T22:07:50.532664+0000","last_undegraded":"2024-09-20T22:07:50.532664+0000","last_fullsized":"2024-09-20T22:07:50.532664+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:05.659437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:05.659437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:52.559502+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":250,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.530775+0000","last_change":"2024-09-20T22:07:20.217991+0000","last_active":"2024-09-20T22:07:50.530775+0000","last_peered":"2024-09-20T22:07:50.530775+0000","last_clean":"2024-09-20T22:07:50.530775+0000","last_became_active":"2024-09-20T22:02:54.291134+0000","last_became_peered":"2024-09-20T22:02:54.291134+0000","last_unstale":"2024-09-20T22:07:50.530775+0000","last_undegraded":"2024-09-20T22:07:50.530775+0000","last_fullsized":"2024-09-20T22:07:50.530775+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:20.217761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:20.217761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:42.617706+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":239,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531194+0000","last_change":"2024-09-20T22:07:29.645352+0000","last_active":"2024-09-20T22:07:50.531194+0000","last_peered":"2024-09-20T22:07:50.531194+0000","last_clean":"2024-09-20T22:07:50.531194+0000","last_became_active":"2024-09-20T22:02:56.307687+0000","last_became_peered":"2024-09-20T22:02:56.307687+0000","last_unstale":"2024-09-20T22:07:50.531194+0000","last_undegraded":"2024-09-20T22:07:50.531194+0000","last_fullsized":"2024-09-20T22:07:50.531194+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:29.645127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:29.645127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:36.059611+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-20T22:07:51.409224+0000","last_change":"2024-09-20T22:07:51.409224+0000","last_active":"2024-09-20T22:07:51.409224+0000","last_peered":"2024-09-20T22:07:51.409224+0000","last_clean":"2024-09-20T22:07:51.409224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T22:07:51.409224+0000","last_undegraded":"2024-09-20T22:07:51.409224+0000","last_fullsized":"2024-09-20T22:07:51.409224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"71'12","reported_seq":271,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.411828+0000","last_change":"2024-09-20T22:07:22.458421+0000","last_active":"2024-09-20T22:07:50.411828+0000","last_peered":"2024-09-20T22:07:50.411828+0000","last_clean":"2024-09-20T22:07:50.411828+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:07:50.411828+0000","last_undegraded":"2024-09-20T22:07:50.411828+0000","last_fullsized":"2024-09-20T22:07:50.411828+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:22.458279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.458279+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:33.412138+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":252,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532912+0000","last_change":"2024-09-20T22:07:21.584606+0000","last_active":"2024-09-20T22:07:50.532912+0000","last_peered":"2024-09-20T22:07:50.532912+0000","last_clean":"2024-09-20T22:07:50.532912+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:07:50.532912+0000","last_undegraded":"2024-09-20T22:07:50.532912+0000","last_fullsized":"2024-09-20T22:07:50.532912+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:21.584499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:21.584499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:24.004976+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":251,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.530539+0000","last_change":"2024-09-20T22:07:15.260254+0000","last_active":"2024-09-20T22:07:50.530539+0000","last_peered":"2024-09-20T22:07:50.530539+0000","last_clean":"2024-09-20T22:07:50.530539+0000","last_became_active":"2024-09-20T22:06:54.843906+0000","last_became_peered":"2024-09-20T22:06:54.843906+0000","last_unstale":"2024-09-20T22:07:50.530539+0000","last_undegraded":"2024-09-20T22:07:50.530539+0000","last_fullsized":"2024-09-20T22:07:50.530539+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:15.260148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.260148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:20.976588+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":184,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412037+0000","last_change":"2024-09-20T22:06:55.822136+0000","last_active":"2024-09-20T22:07:50.412037+0000","last_peered":"2024-09-20T22:07:50.412037+0000","last_clean":"2024-09-20T22:07:50.412037+0000","last_became_active":"2024-09-20T22:06:55.821647+0000","last_became_peered":"2024-09-20T22:06:55.821647+0000","last_unstale":"2024-09-20T22:07:50.412037+0000","last_undegraded":"2024-09-20T22:07:50.412037+0000","last_fullsized":"2024-09-20T22:07:50.412037+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.569045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.569045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-20T22:07:51.409224+0000","last_change":"2024-09-20T22:07:51.409224+0000","last_active":"2024-09-20T22:07:51.409224+0000","last_peered":"2024-09-20T22:07:51.409224+0000","last_clean":"2024-09-20T22:07:51.409224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T22:07:51.409224+0000","last_undegraded":"2024-09-20T22:07:51.409224+0000","last_fullsized":"2024-09-20T22:07:51.409224+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"71'15","reported_seq":271,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531098+0000","last_change":"2024-09-20T22:06:42.563403+0000","last_active":"2024-09-20T22:07:50.531098+0000","last_peered":"2024-09-20T22:07:50.531098+0000","last_clean":"2024-09-20T22:07:50.531098+0000","last_became_active":"2024-09-20T22:02:52.482008+0000","last_became_peered":"2024-09-20T22:02:52.482008+0000","last_unstale":"2024-09-20T22:07:50.531098+0000","last_undegraded":"2024-09-20T22:07:50.531098+0000","last_fullsized":"2024-09-20T22:07:50.531098+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:06:42.563266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:42.563266+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:02.694507+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":265,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532699+0000","last_change":"2024-09-20T22:06:40.813723+0000","last_active":"2024-09-20T22:07:50.532699+0000","last_peered":"2024-09-20T22:07:50.532699+0000","last_clean":"2024-09-20T22:07:50.532699+0000","last_became_active":"2024-09-20T22:02:50.114819+0000","last_became_peered":"2024-09-20T22:02:50.114819+0000","last_unstale":"2024-09-20T22:07:50.532699+0000","last_undegraded":"2024-09-20T22:07:50.532699+0000","last_fullsized":"2024-09-20T22:07:50.532699+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:06:40.813590+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":304,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532174+0000","last_change":"2024-09-20T22:05:34.732761+0000","last_active":"2024-09-20T22:07:50.532174+0000","last_peered":"2024-09-20T22:07:50.532174+0000","last_clean":"2024-09-20T22:07:50.532174+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:07:50.532174+0000","last_undegraded":"2024-09-20T22:07:50.532174+0000","last_fullsized":"2024-09-20T22:07:50.532174+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:05:34.732644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:34.732644+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:52.559547+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":238,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532103+0000","last_change":"2024-09-20T22:06:35.559426+0000","last_active":"2024-09-20T22:07:50.532103+0000","last_peered":"2024-09-20T22:07:50.532103+0000","last_clean":"2024-09-20T22:07:50.532103+0000","last_became_active":"2024-09-20T22:02:56.346990+0000","last_became_peered":"2024-09-20T22:02:56.346990+0000","last_unstale":"2024-09-20T22:07:50.532103+0000","last_undegraded":"2024-09-20T22:07:50.532103+0000","last_fullsized":"2024-09-20T22:07:50.532103+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:35.559306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:35.559306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:52.655537+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":250,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531318+0000","last_change":"2024-09-20T22:06:34.392493+0000","last_active":"2024-09-20T22:07:50.531318+0000","last_peered":"2024-09-20T22:07:50.531318+0000","last_clean":"2024-09-20T22:07:50.531318+0000","last_became_active":"2024-09-20T22:02:50.111935+0000","last_became_peered":"2024-09-20T22:02:50.111935+0000","last_unstale":"2024-09-20T22:07:50.531318+0000","last_undegraded":"2024-09-20T22:07:50.531318+0000","last_fullsized":"2024-09-20T22:07:50.531318+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:34.391904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:34.391904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:58.191440+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"71'6","reported_seq":248,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531832+0000","last_change":"2024-09-20T22:06:54.862697+0000","last_active":"2024-09-20T22:07:50.531832+0000","last_peered":"2024-09-20T22:07:50.531832+0000","last_clean":"2024-09-20T22:07:50.531832+0000","last_became_active":"2024-09-20T22:06:54.862227+0000","last_became_peered":"2024-09-20T22:06:54.862227+0000","last_unstale":"2024-09-20T22:07:50.531832+0000","last_undegraded":"2024-09-20T22:07:50.531832+0000","last_fullsized":"2024-09-20T22:07:50.531832+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.452593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":243,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531670+0000","last_change":"2024-09-20T22:07:19.865906+0000","last_active":"2024-09-20T22:07:50.531670+0000","last_peered":"2024-09-20T22:07:50.531670+0000","last_clean":"2024-09-20T22:07:50.531670+0000","last_became_active":"2024-09-20T22:06:54.836200+0000","last_became_peered":"2024-09-20T22:06:54.836200+0000","last_unstale":"2024-09-20T22:07:50.531670+0000","last_undegraded":"2024-09-20T22:07:50.531670+0000","last_fullsized":"2024-09-20T22:07:50.531670+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:19.865809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.865809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:42.730889+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532418+0000","last_change":"2024-09-20T22:06:39.783526+0000","last_active":"2024-09-20T22:07:50.532418+0000","last_peered":"2024-09-20T22:07:50.532418+0000","last_clean":"2024-09-20T22:07:50.532418+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:07:50.532418+0000","last_undegraded":"2024-09-20T22:07:50.532418+0000","last_fullsized":"2024-09-20T22:07:50.532418+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:39.783456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:39.783456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:59.986149+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"71'9","reported_seq":254,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412819+0000","last_change":"2024-09-20T22:04:52.254864+0000","last_active":"2024-09-20T22:07:50.412819+0000","last_peered":"2024-09-20T22:07:50.412819+0000","last_clean":"2024-09-20T22:07:50.412819+0000","last_became_active":"2024-09-20T22:02:52.481989+0000","last_became_peered":"2024-09-20T22:02:52.481989+0000","last_unstale":"2024-09-20T22:07:50.412819+0000","last_undegraded":"2024-09-20T22:07:50.412819+0000","last_fullsized":"2024-09-20T22:07:50.412819+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:52.254706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:52.254706+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":243,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532330+0000","last_change":"2024-09-20T22:05:31.764903+0000","last_active":"2024-09-20T22:07:50.532330+0000","last_peered":"2024-09-20T22:07:50.532330+0000","last_clean":"2024-09-20T22:07:50.532330+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:07:50.532330+0000","last_undegraded":"2024-09-20T22:07:50.532330+0000","last_fullsized":"2024-09-20T22:07:50.532330+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:31.764833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:31.764833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:52.559517+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":235,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531675+0000","last_change":"2024-09-20T22:06:54.847169+0000","last_active":"2024-09-20T22:07:50.531675+0000","last_peered":"2024-09-20T22:07:50.531675+0000","last_clean":"2024-09-20T22:07:50.531675+0000","last_became_active":"2024-09-20T22:06:54.845704+0000","last_became_peered":"2024-09-20T22:06:54.845704+0000","last_unstale":"2024-09-20T22:07:50.531675+0000","last_undegraded":"2024-09-20T22:07:50.531675+0000","last_fullsized":"2024-09-20T22:07:50.531675+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.452622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":238,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.530875+0000","last_change":"2024-09-20T22:06:39.942296+0000","last_active":"2024-09-20T22:07:50.530875+0000","last_peered":"2024-09-20T22:07:50.530875+0000","last_clean":"2024-09-20T22:07:50.530875+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:07:50.530875+0000","last_undegraded":"2024-09-20T22:07:50.530875+0000","last_fullsized":"2024-09-20T22:07:50.530875+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:39.942186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:39.942186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:04.710791+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"71'9","reported_seq":262,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.533043+0000","last_change":"2024-09-20T22:06:07.333745+0000","last_active":"2024-09-20T22:07:50.533043+0000","last_peered":"2024-09-20T22:07:50.533043+0000","last_clean":"2024-09-20T22:07:50.533043+0000","last_became_active":"2024-09-20T22:02:52.482141+0000","last_became_peered":"2024-09-20T22:02:52.482141+0000","last_unstale":"2024-09-20T22:07:50.533043+0000","last_undegraded":"2024-09-20T22:07:50.533043+0000","last_fullsized":"2024-09-20T22:07:50.533043+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:06:07.333624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:07.333624+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":250,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.411246+0000","last_change":"2024-09-20T22:06:45.908734+0000","last_active":"2024-09-20T22:07:50.411246+0000","last_peered":"2024-09-20T22:07:50.411246+0000","last_clean":"2024-09-20T22:07:50.411246+0000","last_became_active":"2024-09-20T22:02:50.110075+0000","last_became_peered":"2024-09-20T22:02:50.110075+0000","last_unstale":"2024-09-20T22:07:50.411246+0000","last_undegraded":"2024-09-20T22:07:50.411246+0000","last_fullsized":"2024-09-20T22:07:50.411246+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.908616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.908616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:56.429713+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"71'8","reported_seq":303,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531412+0000","last_change":"2024-09-20T22:05:55.972684+0000","last_active":"2024-09-20T22:07:50.531412+0000","last_peered":"2024-09-20T22:07:50.531412+0000","last_clean":"2024-09-20T22:07:50.531412+0000","last_became_active":"2024-09-20T22:02:54.294381+0000","last_became_peered":"2024-09-20T22:02:54.294381+0000","last_unstale":"2024-09-20T22:07:50.531412+0000","last_undegraded":"2024-09-20T22:07:50.531412+0000","last_fullsized":"2024-09-20T22:07:50.531412+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:05:55.972565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:55.972565+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.452750+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412518+0000","last_change":"2024-09-20T22:05:00.897974+0000","last_active":"2024-09-20T22:07:50.412518+0000","last_peered":"2024-09-20T22:07:50.412518+0000","last_clean":"2024-09-20T22:07:50.412518+0000","last_became_active":"2024-09-20T22:02:56.300041+0000","last_became_peered":"2024-09-20T22:02:56.300041+0000","last_unstale":"2024-09-20T22:07:50.412518+0000","last_undegraded":"2024-09-20T22:07:50.412518+0000","last_fullsized":"2024-09-20T22:07:50.412518+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:00.897869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:00.897869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.603137+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"71'10","reported_seq":254,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531520+0000","last_change":"2024-09-20T22:06:54.862610+0000","last_active":"2024-09-20T22:07:50.531520+0000","last_peered":"2024-09-20T22:07:50.531520+0000","last_clean":"2024-09-20T22:07:50.531520+0000","last_became_active":"2024-09-20T22:06:54.862289+0000","last_became_peered":"2024-09-20T22:06:54.862289+0000","last_unstale":"2024-09-20T22:07:50.531520+0000","last_undegraded":"2024-09-20T22:07:50.531520+0000","last_fullsized":"2024-09-20T22:07:50.531520+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.452637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":243,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531890+0000","last_change":"2024-09-20T22:06:35.687169+0000","last_active":"2024-09-20T22:07:50.531890+0000","last_peered":"2024-09-20T22:07:50.531890+0000","last_clean":"2024-09-20T22:07:50.531890+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:07:50.531890+0000","last_undegraded":"2024-09-20T22:07:50.531890+0000","last_fullsized":"2024-09-20T22:07:50.531890+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:35.687065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:35.687065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:58.652707+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"146'10","reported_seq":314,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532948+0000","last_change":"2024-09-20T22:07:15.870748+0000","last_active":"2024-09-20T22:07:50.532948+0000","last_peered":"2024-09-20T22:07:50.532948+0000","last_clean":"2024-09-20T22:07:50.532948+0000","last_became_active":"2024-09-20T22:06:54.843097+0000","last_became_peered":"2024-09-20T22:06:54.843097+0000","last_unstale":"2024-09-20T22:07:50.532948+0000","last_undegraded":"2024-09-20T22:07:50.532948+0000","last_fullsized":"2024-09-20T22:07:50.532948+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:07:15.870646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.870646+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:36.803368+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":230,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412560+0000","last_change":"2024-09-20T22:05:35.601891+0000","last_active":"2024-09-20T22:07:50.412560+0000","last_peered":"2024-09-20T22:07:50.412560+0000","last_clean":"2024-09-20T22:07:50.412560+0000","last_became_active":"2024-09-20T22:02:56.305383+0000","last_became_peered":"2024-09-20T22:02:56.305383+0000","last_unstale":"2024-09-20T22:07:50.412560+0000","last_undegraded":"2024-09-20T22:07:50.412560+0000","last_fullsized":"2024-09-20T22:07:50.412560+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:35.601794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:35.601794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"71'11","reported_seq":267,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412002+0000","last_change":"2024-09-20T22:07:15.544790+0000","last_active":"2024-09-20T22:07:50.412002+0000","last_peered":"2024-09-20T22:07:50.412002+0000","last_clean":"2024-09-20T22:07:50.412002+0000","last_became_active":"2024-09-20T22:06:54.823678+0000","last_became_peered":"2024-09-20T22:06:54.823678+0000","last_unstale":"2024-09-20T22:07:50.412002+0000","last_undegraded":"2024-09-20T22:07:50.412002+0000","last_fullsized":"2024-09-20T22:07:50.412002+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:15.544705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.544705+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:32.779498+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":243,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412366+0000","last_change":"2024-09-20T22:04:54.764407+0000","last_active":"2024-09-20T22:07:50.412366+0000","last_peered":"2024-09-20T22:07:50.412366+0000","last_clean":"2024-09-20T22:07:50.412366+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:07:50.412366+0000","last_undegraded":"2024-09-20T22:07:50.412366+0000","last_fullsized":"2024-09-20T22:07:50.412366+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:54.764137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:54.764137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.603122+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":76,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-20T22:07:45.384973+0000","last_change":"2024-09-20T22:06:54.853857+0000","last_active":"2024-09-20T22:07:45.384973+0000","last_peered":"2024-09-20T22:07:45.384973+0000","last_clean":"2024-09-20T22:07:45.384973+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:07:45.384973+0000","last_undegraded":"2024-09-20T22:07:45.384973+0000","last_fullsized":"2024-09-20T22:07:45.384973+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:48.255605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":63,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-20T22:07:45.386784+0000","last_change":"2024-09-20T22:06:55.835713+0000","last_active":"2024-09-20T22:07:45.386784+0000","last_peered":"2024-09-20T22:07:45.386784+0000","last_clean":"2024-09-20T22:07:45.386784+0000","last_became_active":"2024-09-20T22:06:55.835393+0000","last_became_peered":"2024-09-20T22:06:55.835393+0000","last_unstale":"2024-09-20T22:07:45.386784+0000","last_undegraded":"2024-09-20T22:07:45.386784+0000","last_fullsized":"2024-09-20T22:07:45.386784+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:33.623816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:33.623816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:48.255901+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"71'9","reported_seq":148,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-20T22:07:45.385271+0000","last_change":"2024-09-20T22:06:54.844349+0000","last_active":"2024-09-20T22:07:45.385271+0000","last_peered":"2024-09-20T22:07:45.385271+0000","last_clean":"2024-09-20T22:07:45.385271+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:07:45.385271+0000","last_undegraded":"2024-09-20T22:07:45.385271+0000","last_fullsized":"2024-09-20T22:07:45.385271+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:12.196518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:12.196518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:48.255768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":251,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532584+0000","last_change":"2024-09-20T22:05:22.727380+0000","last_active":"2024-09-20T22:07:50.532584+0000","last_peered":"2024-09-20T22:07:50.532584+0000","last_clean":"2024-09-20T22:07:50.532584+0000","last_became_active":"2024-09-20T22:02:50.109803+0000","last_became_peered":"2024-09-20T22:02:50.109803+0000","last_unstale":"2024-09-20T22:07:50.532584+0000","last_undegraded":"2024-09-20T22:07:50.532584+0000","last_fullsized":"2024-09-20T22:07:50.532584+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:22.727282+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:22.727282+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:52.559486+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":235,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531011+0000","last_change":"2024-09-20T22:06:54.844570+0000","last_active":"2024-09-20T22:07:50.531011+0000","last_peered":"2024-09-20T22:07:50.531011+0000","last_clean":"2024-09-20T22:07:50.531011+0000","last_became_active":"2024-09-20T22:06:54.844271+0000","last_became_peered":"2024-09-20T22:06:54.844271+0000","last_unstale":"2024-09-20T22:07:50.531011+0000","last_undegraded":"2024-09-20T22:07:50.531011+0000","last_fullsized":"2024-09-20T22:07:50.531011+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.452608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":239,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.411087+0000","last_change":"2024-09-20T22:07:23.495005+0000","last_active":"2024-09-20T22:07:50.411087+0000","last_peered":"2024-09-20T22:07:50.411087+0000","last_clean":"2024-09-20T22:07:50.411087+0000","last_became_active":"2024-09-20T22:06:54.848842+0000","last_became_peered":"2024-09-20T22:06:54.848842+0000","last_unstale":"2024-09-20T22:07:50.411087+0000","last_undegraded":"2024-09-20T22:07:50.411087+0000","last_fullsized":"2024-09-20T22:07:50.411087+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.494905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.494905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.999867+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"71'11","reported_seq":265,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.530878+0000","last_change":"2024-09-20T22:06:36.942847+0000","last_active":"2024-09-20T22:07:50.530878+0000","last_peered":"2024-09-20T22:07:50.530878+0000","last_clean":"2024-09-20T22:07:50.530878+0000","last_became_active":"2024-09-20T22:02:52.480118+0000","last_became_peered":"2024-09-20T22:02:52.480118+0000","last_unstale":"2024-09-20T22:07:50.530878+0000","last_undegraded":"2024-09-20T22:07:50.530878+0000","last_fullsized":"2024-09-20T22:07:50.530878+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:06:36.942620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:36.942620+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:55.637693+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":251,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531754+0000","last_change":"2024-09-20T22:06:54.829473+0000","last_active":"2024-09-20T22:07:50.531754+0000","last_peered":"2024-09-20T22:07:50.531754+0000","last_clean":"2024-09-20T22:07:50.531754+0000","last_became_active":"2024-09-20T22:06:54.828826+0000","last_became_peered":"2024-09-20T22:06:54.828826+0000","last_unstale":"2024-09-20T22:07:50.531754+0000","last_undegraded":"2024-09-20T22:07:50.531754+0000","last_fullsized":"2024-09-20T22:07:50.531754+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:14.620563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:14.620563+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":243,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.530819+0000","last_change":"2024-09-20T22:06:45.780268+0000","last_active":"2024-09-20T22:07:50.530819+0000","last_peered":"2024-09-20T22:07:50.530819+0000","last_clean":"2024-09-20T22:07:50.530819+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:07:50.530819+0000","last_undegraded":"2024-09-20T22:07:50.530819+0000","last_fullsized":"2024-09-20T22:07:50.530819+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.780200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.780200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:06.694833+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":238,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531159+0000","last_change":"2024-09-20T22:06:33.002428+0000","last_active":"2024-09-20T22:07:50.531159+0000","last_peered":"2024-09-20T22:07:50.531159+0000","last_clean":"2024-09-20T22:07:50.531159+0000","last_became_active":"2024-09-20T22:02:56.347637+0000","last_became_peered":"2024-09-20T22:02:56.347637+0000","last_unstale":"2024-09-20T22:07:50.531159+0000","last_undegraded":"2024-09-20T22:07:50.531159+0000","last_fullsized":"2024-09-20T22:07:50.531159+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:33.002326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:33.002326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.452765+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"71'4","reported_seq":253,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.530834+0000","last_change":"2024-09-20T22:07:19.309318+0000","last_active":"2024-09-20T22:07:50.530834+0000","last_peered":"2024-09-20T22:07:50.530834+0000","last_clean":"2024-09-20T22:07:50.530834+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:07:50.530834+0000","last_undegraded":"2024-09-20T22:07:50.530834+0000","last_fullsized":"2024-09-20T22:07:50.530834+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'4","last_scrub_stamp":"2024-09-20T22:07:19.309187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.309187+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:33.061321+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":242,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412636+0000","last_change":"2024-09-20T22:04:16.198264+0000","last_active":"2024-09-20T22:07:50.412636+0000","last_peered":"2024-09-20T22:07:50.412636+0000","last_clean":"2024-09-20T22:07:50.412636+0000","last_became_active":"2024-09-20T22:02:50.111142+0000","last_became_peered":"2024-09-20T22:02:50.111142+0000","last_unstale":"2024-09-20T22:07:50.412636+0000","last_undegraded":"2024-09-20T22:07:50.412636+0000","last_fullsized":"2024-09-20T22:07:50.412636+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:16.198153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:16.198153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":250,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531552+0000","last_change":"2024-09-20T22:06:42.412537+0000","last_active":"2024-09-20T22:07:50.531552+0000","last_peered":"2024-09-20T22:07:50.531552+0000","last_clean":"2024-09-20T22:07:50.531552+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:07:50.531552+0000","last_undegraded":"2024-09-20T22:07:50.531552+0000","last_fullsized":"2024-09-20T22:07:50.531552+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:42.412461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:42.412461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:52.350866+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":231,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531155+0000","last_change":"2024-09-20T22:06:01.621567+0000","last_active":"2024-09-20T22:07:50.531155+0000","last_peered":"2024-09-20T22:07:50.531155+0000","last_clean":"2024-09-20T22:07:50.531155+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:07:50.531155+0000","last_undegraded":"2024-09-20T22:07:50.531155+0000","last_fullsized":"2024-09-20T22:07:50.531155+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:01.621489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:01.621489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:52.559577+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":124,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-20T22:07:45.389336+0000","last_change":"2024-09-20T22:06:54.828772+0000","last_active":"2024-09-20T22:07:45.389336+0000","last_peered":"2024-09-20T22:07:45.389336+0000","last_clean":"2024-09-20T22:07:45.389336+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:07:45.389336+0000","last_undegraded":"2024-09-20T22:07:45.389336+0000","last_fullsized":"2024-09-20T22:07:45.389336+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:15.303300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:15.303300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:48.255839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":243,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.531092+0000","last_change":"2024-09-20T22:06:54.848978+0000","last_active":"2024-09-20T22:07:50.531092+0000","last_peered":"2024-09-20T22:07:50.531092+0000","last_clean":"2024-09-20T22:07:50.531092+0000","last_became_active":"2024-09-20T22:06:54.848032+0000","last_became_peered":"2024-09-20T22:06:54.848032+0000","last_unstale":"2024-09-20T22:07:50.531092+0000","last_undegraded":"2024-09-20T22:07:50.531092+0000","last_fullsized":"2024-09-20T22:07:50.531092+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.452577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"71'11","reported_seq":266,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.532121+0000","last_change":"2024-09-20T22:07:19.214716+0000","last_active":"2024-09-20T22:07:50.532121+0000","last_peered":"2024-09-20T22:07:50.532121+0000","last_clean":"2024-09-20T22:07:50.532121+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:07:50.532121+0000","last_undegraded":"2024-09-20T22:07:50.532121+0000","last_fullsized":"2024-09-20T22:07:50.532121+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:19.214636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.214636+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:32.168748+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":234,"reported_epoch":175,"state":"active+clean","last_fresh":"2024-09-20T22:07:50.412670+0000","last_change":"2024-09-20T22:05:06.706314+0000","last_active":"2024-09-20T22:07:50.412670+0000","last_peered":"2024-09-20T22:07:50.412670+0000","last_clean":"2024-09-20T22:07:50.412670+0000","last_became_active":"2024-09-20T22:02:54.289078+0000","last_became_peered":"2024-09-20T22:02:54.289078+0000","last_unstale":"2024-09-20T22:07:50.412670+0000","last_undegraded":"2024-09-20T22:07:50.412670+0000","last_fullsized":"2024-09-20T22:07:50.412670+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:06.681123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:06.681123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":82,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":287,"num_read_kb":282,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135936,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":186248,"kb_used_data":2148,"kb_used_omap":1,"kb_used_meta":183870,"kb_avail":93558904,"statfs":{"total":95995035648,"available":95804317696,"internally_reserved":0,"allocated":2199552,"data_stored":1454082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":188283338},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430276,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":168168,"kb_used_data":2144,"kb_used_omap":1,"kb_used_meta":165822,"kb_avail":93576984,"statfs":{"total":95995035648,"available":95822831616,"internally_reserved":0,"allocated":2195456,"data_stored":1461363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":169802182},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724616,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":137060,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":135294,"kb_avail":93608092,"statfs":{"total":95995035648,"available":95854686208,"internally_reserved":0,"allocated":1601536,"data_stored":871341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138541509},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986256,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":154964,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":153342,"kb_avail":93590188,"statfs":{"total":95995035648,"available":95836352512,"internally_reserved":0,"allocated":1613824,"data_stored":863657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":157022661},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770257932,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":109200,"kb_used_data":1788,"kb_used_omap":4,"kb_used_meta":107387,"kb_avail":93635952,"statfs":{"total":95995035648,"available":95883214848,"internally_reserved":0,"allocated":1830912,"data_stored":966582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":109964968},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149781,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":312012,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":93433140,"statfs":{"total":95995035648,"available":95675535360,"internally_reserved":0,"allocated":1642496,"data_stored":872322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":317651399},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":217560,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":215806,"kb_avail":93527592,"statfs":{"total":95995035648,"available":95772254208,"internally_reserved":0,"allocated":1589248,"data_stored":870137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":220985797},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":316564,"kb_used_data":2152,"kb_used_omap":1,"kb_used_meta":314366,"kb_avail":93428588,"statfs":{"total":95995035648,"available":95670874112,"internally_reserved":0,"allocated":2203648,"data_stored":1454214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":321911237},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:07:53.184 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:07:53.245 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:53.245 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:53.269 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:53.295 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:53.299 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:53.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:53.300 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:53.401 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:53.425 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:53.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:53.426 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:53.526 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:53.550 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:53.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:53.551 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:53.651 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:53.677 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:53.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:53.678 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:53.728 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:53 smithi060 ceph-mon[29694]: 6.d scrub starts 2024-09-20T22:07:53.728 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:53 smithi060 ceph-mon[29694]: 6.d scrub ok 2024-09-20T22:07:53.728 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:53 smithi060 ceph-mon[29694]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T22:07:53.728 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:53 smithi060 ceph-mon[29694]: osdmap e177: 8 total, 8 up, 8 in 2024-09-20T22:07:53.729 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:53 smithi060 ceph-mon[40290]: 6.d scrub starts 2024-09-20T22:07:53.729 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:53 smithi060 ceph-mon[40290]: 6.d scrub ok 2024-09-20T22:07:53.729 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:53 smithi060 ceph-mon[40290]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T22:07:53.729 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:53 smithi060 ceph-mon[40290]: osdmap e177: 8 total, 8 up, 8 in 2024-09-20T22:07:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:53 smithi067 ceph-mon[33511]: 6.d scrub starts 2024-09-20T22:07:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:53 smithi067 ceph-mon[33511]: 6.d scrub ok 2024-09-20T22:07:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:53 smithi067 ceph-mon[33511]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T22:07:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:53 smithi067 ceph-mon[33511]: osdmap e177: 8 total, 8 up, 8 in 2024-09-20T22:07:53.779 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:53.801 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:53.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:53.802 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:53.902 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:53.924 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:53.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:53.925 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:54.026 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:54.053 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:54.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:54.053 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:54.154 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:54.179 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:54.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:54.180 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:54.281 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:54.308 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:54.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:54.309 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:54.410 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:54.437 INFO:teuthology.orchestra.run.smithi060.stderr:enabled application 'rados' on pool 'unique_pool_1' 2024-09-20T22:07:54.438 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:54.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:54.439 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:54.535 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:07:54.535 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:07:54.535 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:07:54.535 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:07:54.535 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:07:54.535 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:07:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:07:54.535 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:07:54.536 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:07:54.536 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:07:54.536 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:07:54.536 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:07:54.536 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:07:54.536 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:07:54.536 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:07:54.536 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:07:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[40290]: 4.15 scrub starts 2024-09-20T22:07:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[40290]: 4.15 scrub ok 2024-09-20T22:07:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[40290]: 4.5 scrub starts 2024-09-20T22:07:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[40290]: 4.5 scrub ok 2024-09-20T22:07:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[40290]: osdmap e178: 8 total, 8 up, 8 in 2024-09-20T22:07:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[40290]: 1.0 scrub starts 2024-09-20T22:07:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[40290]: 1.0 scrub ok 2024-09-20T22:07:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[40290]: pgmap v245: 148 pgs: 2 creating+activating, 12 unknown, 134 active+clean; 583 KiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:07:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2688202838' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:07:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:07:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/847400602' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:54.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[29694]: 4.15 scrub starts 2024-09-20T22:07:54.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[29694]: 4.15 scrub ok 2024-09-20T22:07:54.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[29694]: 4.5 scrub starts 2024-09-20T22:07:54.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[29694]: 4.5 scrub ok 2024-09-20T22:07:54.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[29694]: osdmap e178: 8 total, 8 up, 8 in 2024-09-20T22:07:54.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[29694]: 1.0 scrub starts 2024-09-20T22:07:54.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[29694]: 1.0 scrub ok 2024-09-20T22:07:54.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[29694]: pgmap v245: 148 pgs: 2 creating+activating, 12 unknown, 134 active+clean; 583 KiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:07:54.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2688202838' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:07:54.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:07:54.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:54 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/847400602' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:54.539 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:54.564 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:54.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:54.566 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:54.667 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:54.692 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:54.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:54.694 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:54 smithi067 ceph-mon[33511]: 4.15 scrub starts 2024-09-20T22:07:54.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:54 smithi067 ceph-mon[33511]: 4.15 scrub ok 2024-09-20T22:07:54.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:54 smithi067 ceph-mon[33511]: 4.5 scrub starts 2024-09-20T22:07:54.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:54 smithi067 ceph-mon[33511]: 4.5 scrub ok 2024-09-20T22:07:54.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:54 smithi067 ceph-mon[33511]: osdmap e178: 8 total, 8 up, 8 in 2024-09-20T22:07:54.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:54 smithi067 ceph-mon[33511]: 1.0 scrub starts 2024-09-20T22:07:54.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:54 smithi067 ceph-mon[33511]: 1.0 scrub ok 2024-09-20T22:07:54.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:54 smithi067 ceph-mon[33511]: pgmap v245: 148 pgs: 2 creating+activating, 12 unknown, 134 active+clean; 583 KiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:07:54.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:54 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2688202838' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:07:54.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:54 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:07:54.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:54 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/847400602' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:07:54.795 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:54.819 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:54.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:54.820 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:54.921 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:54.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:54.945 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:54.946 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:55.046 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:55.073 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:55.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:55.074 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:55.174 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:55.200 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:55.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:55.200 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:55.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:55.315 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:07:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:07:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:07:55.326 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:55.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:55.327 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:55.428 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:55.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:55.451 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:55.452 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:55.552 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:55.575 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:55.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:55.576 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:55.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:55 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T22:07:55.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:55 smithi060 ceph-mon[29694]: osdmap e179: 8 total, 8 up, 8 in 2024-09-20T22:07:55.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:55 smithi060 ceph-mon[29694]: 4.d scrub starts 2024-09-20T22:07:55.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:55 smithi060 ceph-mon[29694]: 4.d scrub ok 2024-09-20T22:07:55.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:55 smithi060 ceph-mon[29694]: 3.15 scrub starts 2024-09-20T22:07:55.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:55 smithi060 ceph-mon[29694]: 3.15 scrub ok 2024-09-20T22:07:55.634 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:55 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T22:07:55.634 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:55 smithi060 ceph-mon[40290]: osdmap e179: 8 total, 8 up, 8 in 2024-09-20T22:07:55.634 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:55 smithi060 ceph-mon[40290]: 4.d scrub starts 2024-09-20T22:07:55.634 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:55 smithi060 ceph-mon[40290]: 4.d scrub ok 2024-09-20T22:07:55.634 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:55 smithi060 ceph-mon[40290]: 3.15 scrub starts 2024-09-20T22:07:55.634 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:55 smithi060 ceph-mon[40290]: 3.15 scrub ok 2024-09-20T22:07:55.676 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:55.698 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:55.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:55.699 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:55 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_1", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T22:07:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:55 smithi067 ceph-mon[33511]: osdmap e179: 8 total, 8 up, 8 in 2024-09-20T22:07:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:55 smithi067 ceph-mon[33511]: 4.d scrub starts 2024-09-20T22:07:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:55 smithi067 ceph-mon[33511]: 4.d scrub ok 2024-09-20T22:07:55.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:55 smithi067 ceph-mon[33511]: 3.15 scrub starts 2024-09-20T22:07:55.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:55 smithi067 ceph-mon[33511]: 3.15 scrub ok 2024-09-20T22:07:55.800 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:55.822 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:55.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:55.823 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:55.923 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:55.946 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:55.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:55.947 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:56.035 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:56.048 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:56.073 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:56.074 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:56.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:56.075 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:56.176 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:56.203 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:56.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:56.204 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:56.305 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:56.331 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:56.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:56.332 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:56.433 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:56.441 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:07:56.460 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:56.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:56.461 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[40290]: 6.f scrub starts 2024-09-20T22:07:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[40290]: 6.f scrub ok 2024-09-20T22:07:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[40290]: 5.11 scrub starts 2024-09-20T22:07:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[40290]: 5.11 scrub ok 2024-09-20T22:07:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[40290]: 4.4 scrub starts 2024-09-20T22:07:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[40290]: 4.4 scrub ok 2024-09-20T22:07:56.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[40290]: 4.1d scrub starts 2024-09-20T22:07:56.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[40290]: 4.1d scrub ok 2024-09-20T22:07:56.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[40290]: 4.18 scrub starts 2024-09-20T22:07:56.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[40290]: 4.18 scrub ok 2024-09-20T22:07:56.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2644317844' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:56.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:56.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[40290]: pgmap v247: 148 pgs: 2 creating+activating, 6 unknown, 140 active+clean; 583 KiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T22:07:56.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[29694]: 6.f scrub starts 2024-09-20T22:07:56.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[29694]: 6.f scrub ok 2024-09-20T22:07:56.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[29694]: 5.11 scrub starts 2024-09-20T22:07:56.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[29694]: 5.11 scrub ok 2024-09-20T22:07:56.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[29694]: 4.4 scrub starts 2024-09-20T22:07:56.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[29694]: 4.4 scrub ok 2024-09-20T22:07:56.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[29694]: 4.1d scrub starts 2024-09-20T22:07:56.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[29694]: 4.1d scrub ok 2024-09-20T22:07:56.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[29694]: 4.18 scrub starts 2024-09-20T22:07:56.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[29694]: 4.18 scrub ok 2024-09-20T22:07:56.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2644317844' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:56.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:56.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:56 smithi060 ceph-mon[29694]: pgmap v247: 148 pgs: 2 creating+activating, 6 unknown, 140 active+clean; 583 KiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T22:07:56.562 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:56.588 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:56.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:56.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:56.690 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:56.717 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:56.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:56.718 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:56 smithi067 ceph-mon[33511]: 6.f scrub starts 2024-09-20T22:07:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:56 smithi067 ceph-mon[33511]: 6.f scrub ok 2024-09-20T22:07:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:56 smithi067 ceph-mon[33511]: 5.11 scrub starts 2024-09-20T22:07:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:56 smithi067 ceph-mon[33511]: 5.11 scrub ok 2024-09-20T22:07:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:56 smithi067 ceph-mon[33511]: 4.4 scrub starts 2024-09-20T22:07:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:56 smithi067 ceph-mon[33511]: 4.4 scrub ok 2024-09-20T22:07:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:56 smithi067 ceph-mon[33511]: 4.1d scrub starts 2024-09-20T22:07:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:56 smithi067 ceph-mon[33511]: 4.1d scrub ok 2024-09-20T22:07:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:56 smithi067 ceph-mon[33511]: 4.18 scrub starts 2024-09-20T22:07:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:56 smithi067 ceph-mon[33511]: 4.18 scrub ok 2024-09-20T22:07:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:56 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2644317844' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:56 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:07:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:56 smithi067 ceph-mon[33511]: pgmap v247: 148 pgs: 2 creating+activating, 6 unknown, 140 active+clean; 583 KiB data, 1.5 GiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T22:07:56.818 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:56.842 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:56.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:56.843 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:56.944 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:56.970 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:56.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:56.971 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:57.072 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:57.072 DEBUG:teuthology.orchestra.run.smithi060:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_1 bench 90 write' 2024-09-20T22:07:57.075 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-20T22:07:57.075 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-20T22:07:57.098 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:57.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:57.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:57.131 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:07:57.131 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:hints = 1 2024-09-20T22:07:57.132 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-20T22:07:57.132 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Object prefix: benchmark_data_smithi060_111051 2024-09-20T22:07:57.134 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:07:57.134 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 0 0 0 0 0 0 - 0 2024-09-20T22:07:57.138 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:07:57.201 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:57.236 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:57.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:57.237 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:57.338 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:57.347 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:07:57.376 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:57.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:57.379 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:57.480 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:57.521 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:57.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:57.522 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:57.623 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:57.662 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:57.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:57.663 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:57.689 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:57 smithi060 ceph-mon[29694]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T22:07:57.689 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:57 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:07:57.689 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:57 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:07:57.689 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:57 smithi060 ceph-mon[29694]: osdmap e180: 8 total, 8 up, 8 in 2024-09-20T22:07:57.689 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:57 smithi060 ceph-mon[29694]: from='client.25061 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:07:57.689 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:57 smithi060 ceph-mon[40290]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T22:07:57.689 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:57 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:07:57.689 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:57 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:07:57.690 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:57 smithi060 ceph-mon[40290]: osdmap e180: 8 total, 8 up, 8 in 2024-09-20T22:07:57.690 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:57 smithi060 ceph-mon[40290]: from='client.25061 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:07:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:57 smithi067 ceph-mon[33511]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T22:07:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:57 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:07:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:57 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:07:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:57 smithi067 ceph-mon[33511]: osdmap e180: 8 total, 8 up, 8 in 2024-09-20T22:07:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:57 smithi067 ceph-mon[33511]: from='client.25061 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:07:57.764 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:57.804 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:57.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:57.805 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:57.905 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:57.942 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:57.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:57.943 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:58.044 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:07:58.054 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-20T22:07:56.436710+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":1221,"num_read_kb":1067,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":613,"ondisk_log_size":613,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":428,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":1605124,"kb_used_data":14636,"kb_used_omap":15,"kb_used_meta":1590256,"kb_avail":748356092,"statfs":{"total":767960285184,"available":766316638208,"internally_reserved":0,"allocated":14987264,"data_stored":8919708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":1628422931},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.422810"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":258,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208329+0000","last_change":"2024-09-20T22:06:48.018571+0000","last_active":"2024-09-20T22:07:55.208329+0000","last_peered":"2024-09-20T22:07:55.208329+0000","last_clean":"2024-09-20T22:07:55.208329+0000","last_became_active":"2024-09-20T22:02:50.109930+0000","last_became_peered":"2024-09-20T22:02:50.109930+0000","last_unstale":"2024-09-20T22:07:55.208329+0000","last_undegraded":"2024-09-20T22:07:55.208329+0000","last_fullsized":"2024-09-20T22:07:55.208329+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_clean_scrub_stamp":"2024-09-20T22:06:48.018157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:02.812901+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"71'9","reported_seq":280,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.492913+0000","last_change":"2024-09-20T22:07:55.492953+0000","last_active":"2024-09-20T22:07:55.492913+0000","last_peered":"2024-09-20T22:07:55.492913+0000","last_clean":"2024-09-20T22:07:55.492913+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:07:55.492913+0000","last_undegraded":"2024-09-20T22:07:55.492913+0000","last_fullsized":"2024-09-20T22:07:55.492913+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:55.492841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.492841+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:24.417494+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":135,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-20T22:07:52.423666+0000","last_change":"2024-09-20T22:06:54.843328+0000","last_active":"2024-09-20T22:07:52.423666+0000","last_peered":"2024-09-20T22:07:52.423666+0000","last_clean":"2024-09-20T22:07:52.423666+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:07:52.423666+0000","last_undegraded":"2024-09-20T22:07:52.423666+0000","last_fullsized":"2024-09-20T22:07:52.423666+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:11.178591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:11.178591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.299692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":247,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.527012+0000","last_change":"2024-09-20T22:07:14.535725+0000","last_active":"2024-09-20T22:07:54.527012+0000","last_peered":"2024-09-20T22:07:54.527012+0000","last_clean":"2024-09-20T22:07:54.527012+0000","last_became_active":"2024-09-20T22:06:54.830769+0000","last_became_peered":"2024-09-20T22:06:54.830769+0000","last_unstale":"2024-09-20T22:07:54.527012+0000","last_undegraded":"2024-09-20T22:07:54.527012+0000","last_fullsized":"2024-09-20T22:07:54.527012+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.535618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.535618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:36.228377+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":239,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.192561+0000","last_change":"2024-09-20T22:05:06.024320+0000","last_active":"2024-09-20T22:07:55.192561+0000","last_peered":"2024-09-20T22:07:55.192561+0000","last_clean":"2024-09-20T22:07:55.192561+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:07:55.192561+0000","last_undegraded":"2024-09-20T22:07:55.192561+0000","last_fullsized":"2024-09-20T22:07:55.192561+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:05:06.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:06.024051+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":250,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.192978+0000","last_change":"2024-09-20T22:05:04.976359+0000","last_active":"2024-09-20T22:07:55.192978+0000","last_peered":"2024-09-20T22:07:55.192978+0000","last_clean":"2024-09-20T22:07:55.192978+0000","last_became_active":"2024-09-20T22:02:50.107154+0000","last_became_peered":"2024-09-20T22:02:50.107154+0000","last_unstale":"2024-09-20T22:07:55.192978+0000","last_undegraded":"2024-09-20T22:07:55.192978+0000","last_fullsized":"2024-09-20T22:07:55.192978+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:04.976218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:04.976218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"71'15","reported_seq":271,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.192338+0000","last_change":"2024-09-20T22:04:15.767473+0000","last_active":"2024-09-20T22:07:55.192338+0000","last_peered":"2024-09-20T22:07:55.192338+0000","last_clean":"2024-09-20T22:07:55.192338+0000","last_became_active":"2024-09-20T22:02:52.282101+0000","last_became_peered":"2024-09-20T22:02:52.282101+0000","last_unstale":"2024-09-20T22:07:55.192338+0000","last_undegraded":"2024-09-20T22:07:55.192338+0000","last_fullsized":"2024-09-20T22:07:55.192338+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:04:15.767389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:15.767389+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":252,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.526787+0000","last_change":"2024-09-20T22:07:17.494965+0000","last_active":"2024-09-20T22:07:54.526787+0000","last_peered":"2024-09-20T22:07:54.526787+0000","last_clean":"2024-09-20T22:07:54.526787+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:07:54.526787+0000","last_undegraded":"2024-09-20T22:07:54.526787+0000","last_fullsized":"2024-09-20T22:07:54.526787+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:17.494852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:17.494852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:40.846112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":266,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.433654+0000","last_change":"2024-09-20T22:06:46.616824+0000","last_active":"2024-09-20T22:07:54.433654+0000","last_peered":"2024-09-20T22:07:54.433654+0000","last_clean":"2024-09-20T22:07:54.433654+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:07:54.433654+0000","last_undegraded":"2024-09-20T22:07:54.433654+0000","last_fullsized":"2024-09-20T22:07:54.433654+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:46.616725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:46.616725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:55.441670+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"71'9","reported_seq":271,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.527308+0000","last_change":"2024-09-20T22:06:09.729638+0000","last_active":"2024-09-20T22:07:54.527308+0000","last_peered":"2024-09-20T22:07:54.527308+0000","last_clean":"2024-09-20T22:07:54.527308+0000","last_became_active":"2024-09-20T22:02:52.279526+0000","last_became_peered":"2024-09-20T22:02:52.279526+0000","last_unstale":"2024-09-20T22:07:54.527308+0000","last_undegraded":"2024-09-20T22:07:54.527308+0000","last_fullsized":"2024-09-20T22:07:54.527308+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:06:09.729513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:09.729513+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":258,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.433178+0000","last_change":"2024-09-20T22:07:14.890410+0000","last_active":"2024-09-20T22:07:54.433178+0000","last_peered":"2024-09-20T22:07:54.433178+0000","last_clean":"2024-09-20T22:07:54.433178+0000","last_became_active":"2024-09-20T22:02:54.292911+0000","last_became_peered":"2024-09-20T22:02:54.292911+0000","last_unstale":"2024-09-20T22:07:54.433178+0000","last_undegraded":"2024-09-20T22:07:54.433178+0000","last_fullsized":"2024-09-20T22:07:54.433178+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.890308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.890308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:16.602195+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":247,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.207726+0000","last_change":"2024-09-20T22:07:15.416312+0000","last_active":"2024-09-20T22:07:55.207726+0000","last_peered":"2024-09-20T22:07:55.207726+0000","last_clean":"2024-09-20T22:07:55.207726+0000","last_became_active":"2024-09-20T22:06:54.841095+0000","last_became_peered":"2024-09-20T22:06:54.841095+0000","last_unstale":"2024-09-20T22:07:55.207726+0000","last_undegraded":"2024-09-20T22:07:55.207726+0000","last_fullsized":"2024-09-20T22:07:55.207726+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:15.416176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.416176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:19.458530+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":271,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.433368+0000","last_change":"2024-09-20T22:06:55.816225+0000","last_active":"2024-09-20T22:07:54.433368+0000","last_peered":"2024-09-20T22:07:54.433368+0000","last_clean":"2024-09-20T22:07:54.433368+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:07:54.433368+0000","last_undegraded":"2024-09-20T22:07:54.433368+0000","last_fullsized":"2024-09-20T22:07:54.433368+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:05.351834+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:05.351834+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"71'5","reported_seq":266,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.527311+0000","last_change":"2024-09-20T22:06:54.831254+0000","last_active":"2024-09-20T22:07:54.527311+0000","last_peered":"2024-09-20T22:07:54.527311+0000","last_clean":"2024-09-20T22:07:54.527311+0000","last_became_active":"2024-09-20T22:06:54.830844+0000","last_became_peered":"2024-09-20T22:06:54.830844+0000","last_unstale":"2024-09-20T22:07:54.527311+0000","last_undegraded":"2024-09-20T22:07:54.527311+0000","last_fullsized":"2024-09-20T22:07:54.527311+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'5","last_scrub_stamp":"2024-09-20T22:04:13.950197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:13.950197+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":260,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.685797+0000","last_change":"2024-09-20T22:07:28.654301+0000","last_active":"2024-09-20T22:07:54.685797+0000","last_peered":"2024-09-20T22:07:54.685797+0000","last_clean":"2024-09-20T22:07:54.685797+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:07:54.685797+0000","last_undegraded":"2024-09-20T22:07:54.685797+0000","last_fullsized":"2024-09-20T22:07:54.685797+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:28.654202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:28.654202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:41.331246+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":192,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.207808+0000","last_change":"2024-09-20T22:06:55.824661+0000","last_active":"2024-09-20T22:07:55.207808+0000","last_peered":"2024-09-20T22:07:55.207808+0000","last_clean":"2024-09-20T22:07:55.207808+0000","last_became_active":"2024-09-20T22:06:55.824168+0000","last_became_peered":"2024-09-20T22:06:55.824168+0000","last_unstale":"2024-09-20T22:07:55.207808+0000","last_undegraded":"2024-09-20T22:07:55.207808+0000","last_fullsized":"2024-09-20T22:07:55.207808+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:05:29.551172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":247,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.527593+0000","last_change":"2024-09-20T22:06:47.667293+0000","last_active":"2024-09-20T22:07:54.527593+0000","last_peered":"2024-09-20T22:07:54.527593+0000","last_clean":"2024-09-20T22:07:54.527593+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:07:54.527593+0000","last_undegraded":"2024-09-20T22:07:54.527593+0000","last_fullsized":"2024-09-20T22:07:54.527593+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:06:47.667180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:57.957033+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":250,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208582+0000","last_change":"2024-09-20T22:05:31.552599+0000","last_active":"2024-09-20T22:07:55.208582+0000","last_peered":"2024-09-20T22:07:55.208582+0000","last_clean":"2024-09-20T22:07:55.208582+0000","last_became_active":"2024-09-20T22:02:50.111517+0000","last_became_peered":"2024-09-20T22:02:50.111517+0000","last_unstale":"2024-09-20T22:07:55.208582+0000","last_undegraded":"2024-09-20T22:07:55.208582+0000","last_fullsized":"2024-09-20T22:07:55.208582+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:31.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:31.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"71'15","reported_seq":288,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.433009+0000","last_change":"2024-09-20T22:07:16.215766+0000","last_active":"2024-09-20T22:07:54.433009+0000","last_peered":"2024-09-20T22:07:54.433009+0000","last_clean":"2024-09-20T22:07:54.433009+0000","last_became_active":"2024-09-20T22:06:54.846111+0000","last_became_peered":"2024-09-20T22:06:54.846111+0000","last_unstale":"2024-09-20T22:07:54.433009+0000","last_undegraded":"2024-09-20T22:07:54.433009+0000","last_fullsized":"2024-09-20T22:07:54.433009+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:07:16.215638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:16.215638+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:18.878161+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":84,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-20T22:07:52.424841+0000","last_change":"2024-09-20T22:06:54.848811+0000","last_active":"2024-09-20T22:07:52.424841+0000","last_peered":"2024-09-20T22:07:52.424841+0000","last_clean":"2024-09-20T22:07:52.424841+0000","last_became_active":"2024-09-20T22:06:54.848505+0000","last_became_peered":"2024-09-20T22:06:54.848505+0000","last_unstale":"2024-09-20T22:07:52.424841+0000","last_undegraded":"2024-09-20T22:07:52.424841+0000","last_fullsized":"2024-09-20T22:07:52.424841+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.299348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":246,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.192976+0000","last_change":"2024-09-20T22:06:34.944093+0000","last_active":"2024-09-20T22:07:55.192976+0000","last_peered":"2024-09-20T22:07:55.192976+0000","last_clean":"2024-09-20T22:07:55.192976+0000","last_became_active":"2024-09-20T22:02:56.347561+0000","last_became_peered":"2024-09-20T22:02:56.347561+0000","last_unstale":"2024-09-20T22:07:55.192976+0000","last_undegraded":"2024-09-20T22:07:55.192976+0000","last_fullsized":"2024-09-20T22:07:55.192976+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:34.940758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:34.940758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":200,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.432704+0000","last_change":"2024-09-20T22:06:55.817107+0000","last_active":"2024-09-20T22:07:54.432704+0000","last_peered":"2024-09-20T22:07:54.432704+0000","last_clean":"2024-09-20T22:07:54.432704+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:07:54.432704+0000","last_undegraded":"2024-09-20T22:07:54.432704+0000","last_fullsized":"2024-09-20T22:07:54.432704+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:32.472041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:32.472041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"71'12","reported_seq":278,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.383871+0000","last_change":"2024-09-20T22:07:55.383903+0000","last_active":"2024-09-20T22:07:55.383871+0000","last_peered":"2024-09-20T22:07:55.383871+0000","last_clean":"2024-09-20T22:07:55.383871+0000","last_became_active":"2024-09-20T22:06:54.830946+0000","last_became_peered":"2024-09-20T22:06:54.830946+0000","last_unstale":"2024-09-20T22:07:55.383871+0000","last_undegraded":"2024-09-20T22:07:55.383871+0000","last_fullsized":"2024-09-20T22:07:55.383871+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:55.383818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.383818+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:56.321505+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":243,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.527132+0000","last_change":"2024-09-20T22:05:28.003302+0000","last_active":"2024-09-20T22:07:54.527132+0000","last_peered":"2024-09-20T22:07:54.527132+0000","last_clean":"2024-09-20T22:07:54.527132+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:07:54.527132+0000","last_undegraded":"2024-09-20T22:07:54.527132+0000","last_fullsized":"2024-09-20T22:07:54.527132+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.003183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":241,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.686115+0000","last_change":"2024-09-20T22:05:33.751965+0000","last_active":"2024-09-20T22:07:54.686115+0000","last_peered":"2024-09-20T22:07:54.686115+0000","last_clean":"2024-09-20T22:07:54.686115+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:07:54.686115+0000","last_undegraded":"2024-09-20T22:07:54.686115+0000","last_fullsized":"2024-09-20T22:07:54.686115+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:05:33.751883+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"64'1","reported_seq":86,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-20T22:07:52.423751+0000","last_change":"2024-09-20T22:06:54.854003+0000","last_active":"2024-09-20T22:07:52.423751+0000","last_peered":"2024-09-20T22:07:52.423751+0000","last_clean":"2024-09-20T22:07:52.423751+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:07:52.423751+0000","last_undegraded":"2024-09-20T22:07:52.423751+0000","last_fullsized":"2024-09-20T22:07:52.423751+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.299518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"71'10","reported_seq":261,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208599+0000","last_change":"2024-09-20T22:04:56.414262+0000","last_active":"2024-09-20T22:07:55.208599+0000","last_peered":"2024-09-20T22:07:55.208599+0000","last_clean":"2024-09-20T22:07:55.208599+0000","last_became_active":"2024-09-20T22:02:52.481980+0000","last_became_peered":"2024-09-20T22:02:52.481980+0000","last_unstale":"2024-09-20T22:07:55.208599+0000","last_undegraded":"2024-09-20T22:07:55.208599+0000","last_fullsized":"2024-09-20T22:07:55.208599+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:04:56.414020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:56.414020+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"71'8","reported_seq":323,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208854+0000","last_change":"2024-09-20T22:06:41.394065+0000","last_active":"2024-09-20T22:07:55.208854+0000","last_peered":"2024-09-20T22:07:55.208854+0000","last_clean":"2024-09-20T22:07:55.208854+0000","last_became_active":"2024-09-20T22:02:54.297657+0000","last_became_peered":"2024-09-20T22:02:54.297657+0000","last_unstale":"2024-09-20T22:07:55.208854+0000","last_undegraded":"2024-09-20T22:07:55.208854+0000","last_fullsized":"2024-09-20T22:07:55.208854+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:06:41.393961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:41.393961+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:02.614370+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-20T22:07:52.429532+0000","last_change":"2024-09-20T22:07:52.429150+0000","last_active":"2024-09-20T22:07:52.429532+0000","last_peered":"2024-09-20T22:07:52.429532+0000","last_clean":"2024-09-20T22:07:52.429532+0000","last_became_active":"2024-09-20T22:07:52.428105+0000","last_became_peered":"2024-09-20T22:07:52.428105+0000","last_unstale":"2024-09-20T22:07:52.429532+0000","last_undegraded":"2024-09-20T22:07:52.429532+0000","last_fullsized":"2024-09-20T22:07:52.429532+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:13.187262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"71'15","reported_seq":165,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-20T22:07:52.424574+0000","last_change":"2024-09-20T22:06:54.860305+0000","last_active":"2024-09-20T22:07:52.424574+0000","last_peered":"2024-09-20T22:07:52.424574+0000","last_clean":"2024-09-20T22:07:52.424574+0000","last_became_active":"2024-09-20T22:06:54.859918+0000","last_became_peered":"2024-09-20T22:06:54.859918+0000","last_unstale":"2024-09-20T22:07:52.424574+0000","last_undegraded":"2024-09-20T22:07:52.424574+0000","last_fullsized":"2024-09-20T22:07:52.424574+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:04:13.235907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:13.235907+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.299629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":259,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.192423+0000","last_change":"2024-09-20T22:07:23.292560+0000","last_active":"2024-09-20T22:07:55.192423+0000","last_peered":"2024-09-20T22:07:55.192423+0000","last_clean":"2024-09-20T22:07:55.192423+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:07:55.192423+0000","last_undegraded":"2024-09-20T22:07:55.192423+0000","last_fullsized":"2024-09-20T22:07:55.192423+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.292451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.292451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:25.739658+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"71'8","reported_seq":324,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.527937+0000","last_change":"2024-09-20T22:06:04.874166+0000","last_active":"2024-09-20T22:07:54.527937+0000","last_peered":"2024-09-20T22:07:54.527937+0000","last_clean":"2024-09-20T22:07:54.527937+0000","last_became_active":"2024-09-20T22:02:54.294917+0000","last_became_peered":"2024-09-20T22:02:54.294917+0000","last_unstale":"2024-09-20T22:07:54.527937+0000","last_undegraded":"2024-09-20T22:07:54.527937+0000","last_fullsized":"2024-09-20T22:07:54.527937+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:06:04.874042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:04.874042+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":208,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208571+0000","last_change":"2024-09-20T22:07:52.334831+0000","last_active":"2024-09-20T22:07:55.208571+0000","last_peered":"2024-09-20T22:07:55.208571+0000","last_clean":"2024-09-20T22:07:55.208571+0000","last_became_active":"2024-09-20T22:06:55.822780+0000","last_became_peered":"2024-09-20T22:06:55.822780+0000","last_unstale":"2024-09-20T22:07:55.208571+0000","last_undegraded":"2024-09-20T22:07:55.208571+0000","last_fullsized":"2024-09-20T22:07:55.208571+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:52.334748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:52.334748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:53.481596+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":16,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208076+0000","last_change":"2024-09-20T22:07:52.674862+0000","last_active":"2024-09-20T22:07:55.208076+0000","last_peered":"2024-09-20T22:07:55.208076+0000","last_clean":"2024-09-20T22:07:55.208076+0000","last_became_active":"2024-09-20T22:07:52.674521+0000","last_became_peered":"2024-09-20T22:07:52.674521+0000","last_unstale":"2024-09-20T22:07:55.208076+0000","last_undegraded":"2024-09-20T22:07:55.208076+0000","last_fullsized":"2024-09-20T22:07:55.208076+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:14.102427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"71'18","reported_seq":278,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.192269+0000","last_change":"2024-09-20T22:04:57.967494+0000","last_active":"2024-09-20T22:07:55.192269+0000","last_peered":"2024-09-20T22:07:55.192269+0000","last_clean":"2024-09-20T22:07:55.192269+0000","last_became_active":"2024-09-20T22:02:52.482119+0000","last_became_peered":"2024-09-20T22:02:52.482119+0000","last_unstale":"2024-09-20T22:07:55.192269+0000","last_undegraded":"2024-09-20T22:07:55.192269+0000","last_fullsized":"2024-09-20T22:07:55.192269+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'18","last_scrub_stamp":"2024-09-20T22:04:57.967216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:57.967216+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":250,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.193313+0000","last_change":"2024-09-20T22:05:02.924002+0000","last_active":"2024-09-20T22:07:55.193313+0000","last_peered":"2024-09-20T22:07:55.193313+0000","last_clean":"2024-09-20T22:07:55.193313+0000","last_became_active":"2024-09-20T22:02:50.113076+0000","last_became_peered":"2024-09-20T22:02:50.113076+0000","last_unstale":"2024-09-20T22:07:55.193313+0000","last_undegraded":"2024-09-20T22:07:55.193313+0000","last_fullsized":"2024-09-20T22:07:55.193313+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:02.923565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:02.923565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":251,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.526860+0000","last_change":"2024-09-20T22:06:05.894144+0000","last_active":"2024-09-20T22:07:54.526860+0000","last_peered":"2024-09-20T22:07:54.526860+0000","last_clean":"2024-09-20T22:07:54.526860+0000","last_became_active":"2024-09-20T22:02:54.288007+0000","last_became_peered":"2024-09-20T22:02:54.288007+0000","last_unstale":"2024-09-20T22:07:54.526860+0000","last_undegraded":"2024-09-20T22:07:54.526860+0000","last_fullsized":"2024-09-20T22:07:54.526860+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:05.894031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:05.894031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":247,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.527198+0000","last_change":"2024-09-20T22:05:29.990030+0000","last_active":"2024-09-20T22:07:54.527198+0000","last_peered":"2024-09-20T22:07:54.527198+0000","last_clean":"2024-09-20T22:07:54.527198+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:07:54.527198+0000","last_undegraded":"2024-09-20T22:07:54.527198+0000","last_fullsized":"2024-09-20T22:07:54.527198+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:29.989934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:29.989934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":16,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.433149+0000","last_change":"2024-09-20T22:07:52.940943+0000","last_active":"2024-09-20T22:07:54.433149+0000","last_peered":"2024-09-20T22:07:54.433149+0000","last_clean":"2024-09-20T22:07:54.433149+0000","last_became_active":"2024-09-20T22:07:52.940627+0000","last_became_peered":"2024-09-20T22:07:52.940627+0000","last_unstale":"2024-09-20T22:07:54.433149+0000","last_undegraded":"2024-09-20T22:07:54.433149+0000","last_fullsized":"2024-09-20T22:07:54.433149+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:11.794238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"71'14","reported_seq":276,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.527841+0000","last_change":"2024-09-20T22:06:00.944172+0000","last_active":"2024-09-20T22:07:54.527841+0000","last_peered":"2024-09-20T22:07:54.527841+0000","last_clean":"2024-09-20T22:07:54.527841+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:07:54.527841+0000","last_undegraded":"2024-09-20T22:07:54.527841+0000","last_fullsized":"2024-09-20T22:07:54.527841+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'14","last_scrub_stamp":"2024-09-20T22:06:00.944086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.944086+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":252,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208732+0000","last_change":"2024-09-20T22:06:54.842558+0000","last_active":"2024-09-20T22:07:55.208732+0000","last_peered":"2024-09-20T22:07:55.208732+0000","last_clean":"2024-09-20T22:07:55.208732+0000","last_became_active":"2024-09-20T22:06:54.841165+0000","last_became_peered":"2024-09-20T22:06:54.841165+0000","last_unstale":"2024-09-20T22:07:55.208732+0000","last_undegraded":"2024-09-20T22:07:55.208732+0000","last_fullsized":"2024-09-20T22:07:55.208732+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"146'10","reported_seq":323,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.192206+0000","last_change":"2024-09-20T22:07:24.333937+0000","last_active":"2024-09-20T22:07:55.192206+0000","last_peered":"2024-09-20T22:07:55.192206+0000","last_clean":"2024-09-20T22:07:55.192206+0000","last_became_active":"2024-09-20T22:06:54.847742+0000","last_became_peered":"2024-09-20T22:06:54.847742+0000","last_unstale":"2024-09-20T22:07:55.192206+0000","last_undegraded":"2024-09-20T22:07:55.192206+0000","last_fullsized":"2024-09-20T22:07:55.192206+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:07:24.333809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:24.333809+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:46.378931+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":247,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.685896+0000","last_change":"2024-09-20T22:07:30.589937+0000","last_active":"2024-09-20T22:07:54.685896+0000","last_peered":"2024-09-20T22:07:54.685896+0000","last_clean":"2024-09-20T22:07:54.685896+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:07:54.685896+0000","last_undegraded":"2024-09-20T22:07:54.685896+0000","last_fullsized":"2024-09-20T22:07:54.685896+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:30.589849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:30.589849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:57.117488+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":16,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208176+0000","last_change":"2024-09-20T22:07:52.940978+0000","last_active":"2024-09-20T22:07:55.208176+0000","last_peered":"2024-09-20T22:07:55.208176+0000","last_clean":"2024-09-20T22:07:55.208176+0000","last_became_active":"2024-09-20T22:07:52.940673+0000","last_became_peered":"2024-09-20T22:07:52.940673+0000","last_unstale":"2024-09-20T22:07:55.208176+0000","last_undegraded":"2024-09-20T22:07:55.208176+0000","last_fullsized":"2024-09-20T22:07:55.208176+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.115401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"71'10","reported_seq":111,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-20T22:07:52.422436+0000","last_change":"2024-09-20T22:06:54.845001+0000","last_active":"2024-09-20T22:07:52.422436+0000","last_peered":"2024-09-20T22:07:52.422436+0000","last_clean":"2024-09-20T22:07:52.422436+0000","last_became_active":"2024-09-20T22:06:54.844681+0000","last_became_peered":"2024-09-20T22:06:54.844681+0000","last_unstale":"2024-09-20T22:07:52.422436+0000","last_undegraded":"2024-09-20T22:07:52.422436+0000","last_fullsized":"2024-09-20T22:07:52.422436+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.299315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":266,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.433755+0000","last_change":"2024-09-20T22:06:43.598035+0000","last_active":"2024-09-20T22:07:54.433755+0000","last_peered":"2024-09-20T22:07:54.433755+0000","last_clean":"2024-09-20T22:07:54.433755+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:07:54.433755+0000","last_undegraded":"2024-09-20T22:07:54.433755+0000","last_fullsized":"2024-09-20T22:07:54.433755+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:06:43.597924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"71'8","reported_seq":312,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208290+0000","last_change":"2024-09-20T22:04:20.160515+0000","last_active":"2024-09-20T22:07:55.208290+0000","last_peered":"2024-09-20T22:07:55.208290+0000","last_clean":"2024-09-20T22:07:55.208290+0000","last_became_active":"2024-09-20T22:02:54.292819+0000","last_became_peered":"2024-09-20T22:02:54.292819+0000","last_unstale":"2024-09-20T22:07:55.208290+0000","last_undegraded":"2024-09-20T22:07:55.208290+0000","last_fullsized":"2024-09-20T22:07:55.208290+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:04:20.160405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:20.160405+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":238,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.207849+0000","last_change":"2024-09-20T22:06:00.572347+0000","last_active":"2024-09-20T22:07:55.207849+0000","last_peered":"2024-09-20T22:07:55.207849+0000","last_clean":"2024-09-20T22:07:55.207849+0000","last_became_active":"2024-09-20T22:02:56.304274+0000","last_became_peered":"2024-09-20T22:02:56.304274+0000","last_unstale":"2024-09-20T22:07:55.207849+0000","last_undegraded":"2024-09-20T22:07:55.207849+0000","last_fullsized":"2024-09-20T22:07:55.207849+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:00.572243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.572243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":16,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.193276+0000","last_change":"2024-09-20T22:07:52.940027+0000","last_active":"2024-09-20T22:07:55.193276+0000","last_peered":"2024-09-20T22:07:55.193276+0000","last_clean":"2024-09-20T22:07:55.193276+0000","last_became_active":"2024-09-20T22:07:52.939692+0000","last_became_peered":"2024-09-20T22:07:52.939692+0000","last_unstale":"2024-09-20T22:07:55.193276+0000","last_undegraded":"2024-09-20T22:07:55.193276+0000","last_fullsized":"2024-09-20T22:07:55.193276+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:12.848256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"71'19","reported_seq":290,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208407+0000","last_change":"2024-09-20T22:05:24.600933+0000","last_active":"2024-09-20T22:07:55.208407+0000","last_peered":"2024-09-20T22:07:55.208407+0000","last_clean":"2024-09-20T22:07:55.208407+0000","last_became_active":"2024-09-20T22:02:52.480077+0000","last_became_peered":"2024-09-20T22:02:52.480077+0000","last_unstale":"2024-09-20T22:07:55.208407+0000","last_undegraded":"2024-09-20T22:07:55.208407+0000","last_fullsized":"2024-09-20T22:07:55.208407+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:05:24.600795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:24.600795+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":93,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-20T22:07:52.424725+0000","last_change":"2024-09-20T22:07:14.382822+0000","last_active":"2024-09-20T22:07:52.424725+0000","last_peered":"2024-09-20T22:07:52.424725+0000","last_clean":"2024-09-20T22:07:52.424725+0000","last_became_active":"2024-09-20T22:06:54.829037+0000","last_became_peered":"2024-09-20T22:06:54.829037+0000","last_unstale":"2024-09-20T22:07:52.424725+0000","last_undegraded":"2024-09-20T22:07:52.424725+0000","last_fullsized":"2024-09-20T22:07:52.424725+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.382712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.382712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.271274+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":258,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208064+0000","last_change":"2024-09-20T22:07:30.169138+0000","last_active":"2024-09-20T22:07:55.208064+0000","last_peered":"2024-09-20T22:07:55.208064+0000","last_clean":"2024-09-20T22:07:55.208064+0000","last_became_active":"2024-09-20T22:02:54.290787+0000","last_became_peered":"2024-09-20T22:02:54.290787+0000","last_unstale":"2024-09-20T22:07:55.208064+0000","last_undegraded":"2024-09-20T22:07:55.208064+0000","last_fullsized":"2024-09-20T22:07:55.208064+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:30.169038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:30.169038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:38.467894+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":84,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-20T22:07:52.423536+0000","last_change":"2024-09-20T22:06:54.820385+0000","last_active":"2024-09-20T22:07:52.423536+0000","last_peered":"2024-09-20T22:07:52.423536+0000","last_clean":"2024-09-20T22:07:52.423536+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:07:52.423536+0000","last_undegraded":"2024-09-20T22:07:52.423536+0000","last_fullsized":"2024-09-20T22:07:52.423536+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.299552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":16,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.432544+0000","last_change":"2024-09-20T22:07:52.674766+0000","last_active":"2024-09-20T22:07:54.432544+0000","last_peered":"2024-09-20T22:07:54.432544+0000","last_clean":"2024-09-20T22:07:54.432544+0000","last_became_active":"2024-09-20T22:07:52.674363+0000","last_became_peered":"2024-09-20T22:07:52.674363+0000","last_unstale":"2024-09-20T22:07:54.432544+0000","last_undegraded":"2024-09-20T22:07:54.432544+0000","last_fullsized":"2024-09-20T22:07:54.432544+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:16.336670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"146'30","reported_seq":158,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-20T22:07:52.424818+0000","last_change":"2024-09-20T22:06:54.854264+0000","last_active":"2024-09-20T22:07:52.424818+0000","last_peered":"2024-09-20T22:07:52.424818+0000","last_clean":"2024-09-20T22:07:52.424818+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:07:52.424818+0000","last_undegraded":"2024-09-20T22:07:52.424818+0000","last_fullsized":"2024-09-20T22:07:52.424818+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.299277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":259,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.527808+0000","last_change":"2024-09-20T22:05:58.966616+0000","last_active":"2024-09-20T22:07:54.527808+0000","last_peered":"2024-09-20T22:07:54.527808+0000","last_clean":"2024-09-20T22:07:54.527808+0000","last_became_active":"2024-09-20T22:02:50.108457+0000","last_became_peered":"2024-09-20T22:02:50.108457+0000","last_unstale":"2024-09-20T22:07:54.527808+0000","last_undegraded":"2024-09-20T22:07:54.527808+0000","last_fullsized":"2024-09-20T22:07:54.527808+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:58.966541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:58.966541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":270,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.433075+0000","last_change":"2024-09-20T22:07:21.886172+0000","last_active":"2024-09-20T22:07:54.433075+0000","last_peered":"2024-09-20T22:07:54.433075+0000","last_clean":"2024-09-20T22:07:54.433075+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:07:54.433075+0000","last_undegraded":"2024-09-20T22:07:54.433075+0000","last_fullsized":"2024-09-20T22:07:54.433075+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_clean_scrub_stamp":"2024-09-20T22:07:21.886038+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.052265+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":251,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208371+0000","last_change":"2024-09-20T22:07:17.566774+0000","last_active":"2024-09-20T22:07:55.208371+0000","last_peered":"2024-09-20T22:07:55.208371+0000","last_clean":"2024-09-20T22:07:55.208371+0000","last_became_active":"2024-09-20T22:06:54.841419+0000","last_became_peered":"2024-09-20T22:06:54.841419+0000","last_unstale":"2024-09-20T22:07:55.208371+0000","last_undegraded":"2024-09-20T22:07:55.208371+0000","last_fullsized":"2024-09-20T22:07:55.208371+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:17.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:17.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:32.923210+0000","scrub_duration":120,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":238,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.192784+0000","last_change":"2024-09-20T22:05:34.954357+0000","last_active":"2024-09-20T22:07:55.192784+0000","last_peered":"2024-09-20T22:07:55.192784+0000","last_clean":"2024-09-20T22:07:55.192784+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:07:55.192784+0000","last_undegraded":"2024-09-20T22:07:55.192784+0000","last_fullsized":"2024-09-20T22:07:55.192784+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:05:34.953944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":16,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.192509+0000","last_change":"2024-09-20T22:07:52.429893+0000","last_active":"2024-09-20T22:07:55.192509+0000","last_peered":"2024-09-20T22:07:55.192509+0000","last_clean":"2024-09-20T22:07:55.192509+0000","last_became_active":"2024-09-20T22:07:52.429554+0000","last_became_peered":"2024-09-20T22:07:52.429554+0000","last_unstale":"2024-09-20T22:07:55.192509+0000","last_undegraded":"2024-09-20T22:07:55.192509+0000","last_fullsized":"2024-09-20T22:07:55.192509+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:20.427594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"71'13","reported_seq":123,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-20T22:07:52.425037+0000","last_change":"2024-09-20T22:06:54.851246+0000","last_active":"2024-09-20T22:07:52.425037+0000","last_peered":"2024-09-20T22:07:52.425037+0000","last_clean":"2024-09-20T22:07:52.425037+0000","last_became_active":"2024-09-20T22:06:54.850696+0000","last_became_peered":"2024-09-20T22:06:54.850696+0000","last_unstale":"2024-09-20T22:07:52.425037+0000","last_undegraded":"2024-09-20T22:07:52.425037+0000","last_fullsized":"2024-09-20T22:07:52.425037+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.299443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":93,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-20T22:07:52.422060+0000","last_change":"2024-09-20T22:07:20.341524+0000","last_active":"2024-09-20T22:07:52.422060+0000","last_peered":"2024-09-20T22:07:52.422060+0000","last_clean":"2024-09-20T22:07:52.422060+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:07:52.422060+0000","last_undegraded":"2024-09-20T22:07:52.422060+0000","last_fullsized":"2024-09-20T22:07:52.422060+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:20.338388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:20.338388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:40.900548+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":276,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.432239+0000","last_change":"2024-09-20T22:06:47.015379+0000","last_active":"2024-09-20T22:07:54.432239+0000","last_peered":"2024-09-20T22:07:54.432239+0000","last_clean":"2024-09-20T22:07:54.432239+0000","last_became_active":"2024-09-20T22:02:49.099508+0000","last_became_peered":"2024-09-20T22:02:49.099508+0000","last_unstale":"2024-09-20T22:07:54.432239+0000","last_undegraded":"2024-09-20T22:07:54.432239+0000","last_fullsized":"2024-09-20T22:07:54.432239+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:06:47.015270+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:34.967504+0000","last_clean_scrub_stamp":"2024-09-20T22:06:47.015270+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:09.454334+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":250,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.433688+0000","last_change":"2024-09-20T22:06:36.901630+0000","last_active":"2024-09-20T22:07:54.433688+0000","last_peered":"2024-09-20T22:07:54.433688+0000","last_clean":"2024-09-20T22:07:54.433688+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:07:54.433688+0000","last_undegraded":"2024-09-20T22:07:54.433688+0000","last_fullsized":"2024-09-20T22:07:54.433688+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:36.901522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:36.901522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:00.298120+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":239,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.686250+0000","last_change":"2024-09-20T22:05:28.780920+0000","last_active":"2024-09-20T22:07:54.686250+0000","last_peered":"2024-09-20T22:07:54.686250+0000","last_clean":"2024-09-20T22:07:54.686250+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:07:54.686250+0000","last_undegraded":"2024-09-20T22:07:54.686250+0000","last_fullsized":"2024-09-20T22:07:54.686250+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.780758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.780758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":4,"reported_epoch":177,"state":"creating+activating","last_fresh":"2024-09-20T22:07:52.428227+0000","last_change":"2024-09-20T22:07:52.420395+0000","last_active":"2024-09-20T22:07:51.409224+0000","last_peered":"2024-09-20T22:07:51.409224+0000","last_clean":"2024-09-20T22:07:51.409224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T22:07:52.428227+0000","last_undegraded":"2024-09-20T22:07:52.428227+0000","last_fullsized":"2024-09-20T22:07:52.428227+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":["1","2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"71'12","reported_seq":273,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208618+0000","last_change":"2024-09-20T22:07:29.614630+0000","last_active":"2024-09-20T22:07:55.208618+0000","last_peered":"2024-09-20T22:07:55.208618+0000","last_clean":"2024-09-20T22:07:55.208618+0000","last_became_active":"2024-09-20T22:06:54.848670+0000","last_became_peered":"2024-09-20T22:06:54.848670+0000","last_unstale":"2024-09-20T22:07:55.208618+0000","last_undegraded":"2024-09-20T22:07:55.208618+0000","last_fullsized":"2024-09-20T22:07:55.208618+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:29.614551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:29.614551+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:50.884056+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":258,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208286+0000","last_change":"2024-09-20T22:06:44.862452+0000","last_active":"2024-09-20T22:07:55.208286+0000","last_peered":"2024-09-20T22:07:55.208286+0000","last_clean":"2024-09-20T22:07:55.208286+0000","last_became_active":"2024-09-20T22:02:50.110448+0000","last_became_peered":"2024-09-20T22:02:50.110448+0000","last_unstale":"2024-09-20T22:07:55.208286+0000","last_undegraded":"2024-09-20T22:07:55.208286+0000","last_fullsized":"2024-09-20T22:07:55.208286+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:06:44.862338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:59.230368+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":569,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.685854+0000","last_change":"2024-09-20T22:07:16.590582+0000","last_active":"2024-09-20T22:07:54.685854+0000","last_peered":"2024-09-20T22:07:54.685854+0000","last_clean":"2024-09-20T22:07:54.685854+0000","last_became_active":"2024-09-20T22:06:54.844454+0000","last_became_peered":"2024-09-20T22:06:54.844454+0000","last_unstale":"2024-09-20T22:07:54.685854+0000","last_undegraded":"2024-09-20T22:07:54.685854+0000","last_fullsized":"2024-09-20T22:07:54.685854+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:07:16.590475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:07:16.590475+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.092001+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":292,"num_read_kb":287,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":251,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.433924+0000","last_change":"2024-09-20T22:07:22.884981+0000","last_active":"2024-09-20T22:07:54.433924+0000","last_peered":"2024-09-20T22:07:54.433924+0000","last_clean":"2024-09-20T22:07:54.433924+0000","last_became_active":"2024-09-20T22:06:54.841596+0000","last_became_peered":"2024-09-20T22:06:54.841596+0000","last_unstale":"2024-09-20T22:07:54.433924+0000","last_undegraded":"2024-09-20T22:07:54.433924+0000","last_fullsized":"2024-09-20T22:07:54.433924+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:22.884907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.884907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:41.462917+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":84,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-20T22:07:52.424611+0000","last_change":"2024-09-20T22:06:54.853957+0000","last_active":"2024-09-20T22:07:52.424611+0000","last_peered":"2024-09-20T22:07:52.424611+0000","last_clean":"2024-09-20T22:07:52.424611+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:07:52.424611+0000","last_undegraded":"2024-09-20T22:07:52.424611+0000","last_fullsized":"2024-09-20T22:07:52.424611+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.299380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":16,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208114+0000","last_change":"2024-09-20T22:07:52.430313+0000","last_active":"2024-09-20T22:07:55.208114+0000","last_peered":"2024-09-20T22:07:55.208114+0000","last_clean":"2024-09-20T22:07:55.208114+0000","last_became_active":"2024-09-20T22:07:52.430017+0000","last_became_peered":"2024-09-20T22:07:52.430017+0000","last_unstale":"2024-09-20T22:07:55.208114+0000","last_undegraded":"2024-09-20T22:07:55.208114+0000","last_fullsized":"2024-09-20T22:07:55.208114+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:04.804126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"71'16","reported_seq":346,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208818+0000","last_change":"2024-09-20T22:07:53.297530+0000","last_active":"2024-09-20T22:07:55.208818+0000","last_peered":"2024-09-20T22:07:55.208818+0000","last_clean":"2024-09-20T22:07:55.208818+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:07:55.208818+0000","last_undegraded":"2024-09-20T22:07:55.208818+0000","last_fullsized":"2024-09-20T22:07:55.208818+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'16","last_scrub_stamp":"2024-09-20T22:07:53.297442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.297442+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:20.417428+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":258,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.192538+0000","last_change":"2024-09-20T22:06:48.105501+0000","last_active":"2024-09-20T22:07:55.192538+0000","last_peered":"2024-09-20T22:07:55.192538+0000","last_clean":"2024-09-20T22:07:55.192538+0000","last_became_active":"2024-09-20T22:02:50.106011+0000","last_became_peered":"2024-09-20T22:02:50.106011+0000","last_unstale":"2024-09-20T22:07:55.192538+0000","last_undegraded":"2024-09-20T22:07:55.192538+0000","last_fullsized":"2024-09-20T22:07:55.192538+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:48.105403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:48.105403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:17.663438+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":514,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.686507+0000","last_change":"2024-09-20T22:07:53.674419+0000","last_active":"2024-09-20T22:07:54.686507+0000","last_peered":"2024-09-20T22:07:54.686507+0000","last_clean":"2024-09-20T22:07:54.686507+0000","last_became_active":"2024-09-20T22:02:45.068482+0000","last_became_peered":"2024-09-20T22:02:45.068482+0000","last_unstale":"2024-09-20T22:07:54.686507+0000","last_undegraded":"2024-09-20T22:07:54.686507+0000","last_fullsized":"2024-09-20T22:07:54.686507+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:07:53.673691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.673691+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.005158+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":251,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.686352+0000","last_change":"2024-09-20T22:05:56.679105+0000","last_active":"2024-09-20T22:07:54.686352+0000","last_peered":"2024-09-20T22:07:54.686352+0000","last_clean":"2024-09-20T22:07:54.686352+0000","last_became_active":"2024-09-20T22:02:54.298021+0000","last_became_peered":"2024-09-20T22:02:54.298021+0000","last_unstale":"2024-09-20T22:07:54.686352+0000","last_undegraded":"2024-09-20T22:07:54.686352+0000","last_fullsized":"2024-09-20T22:07:54.686352+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:56.678980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:56.678980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":260,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.433794+0000","last_change":"2024-09-20T22:06:47.664985+0000","last_active":"2024-09-20T22:07:54.433794+0000","last_peered":"2024-09-20T22:07:54.433794+0000","last_clean":"2024-09-20T22:07:54.433794+0000","last_became_active":"2024-09-20T22:04:21.237725+0000","last_became_peered":"2024-09-20T22:04:21.237725+0000","last_unstale":"2024-09-20T22:07:54.433794+0000","last_undegraded":"2024-09-20T22:07:54.433794+0000","last_fullsized":"2024-09-20T22:07:54.433794+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:47.664877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:06:47.664877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:10.795971+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":16,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.526986+0000","last_change":"2024-09-20T22:07:52.429509+0000","last_active":"2024-09-20T22:07:54.526986+0000","last_peered":"2024-09-20T22:07:54.526986+0000","last_clean":"2024-09-20T22:07:54.526986+0000","last_became_active":"2024-09-20T22:07:52.429069+0000","last_became_peered":"2024-09-20T22:07:52.429069+0000","last_unstale":"2024-09-20T22:07:54.526986+0000","last_undegraded":"2024-09-20T22:07:54.526986+0000","last_fullsized":"2024-09-20T22:07:54.526986+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.758907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"71'11","reported_seq":275,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.527994+0000","last_change":"2024-09-20T22:07:13.511239+0000","last_active":"2024-09-20T22:07:54.527994+0000","last_peered":"2024-09-20T22:07:54.527994+0000","last_clean":"2024-09-20T22:07:54.527994+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:07:54.527994+0000","last_undegraded":"2024-09-20T22:07:54.527994+0000","last_fullsized":"2024-09-20T22:07:54.527994+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:13.511159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:13.511159+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:38.687052+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":251,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.528105+0000","last_change":"2024-09-20T22:05:26.006756+0000","last_active":"2024-09-20T22:07:54.528105+0000","last_peered":"2024-09-20T22:07:54.528105+0000","last_clean":"2024-09-20T22:07:54.528105+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:07:54.528105+0000","last_undegraded":"2024-09-20T22:07:54.528105+0000","last_fullsized":"2024-09-20T22:07:54.528105+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:26.006692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:26.006692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":250,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.433647+0000","last_change":"2024-09-20T22:05:57.397917+0000","last_active":"2024-09-20T22:07:54.433647+0000","last_peered":"2024-09-20T22:07:54.433647+0000","last_clean":"2024-09-20T22:07:54.433647+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:07:54.433647+0000","last_undegraded":"2024-09-20T22:07:54.433647+0000","last_fullsized":"2024-09-20T22:07:54.433647+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:57.397807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:57.397807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":246,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.192589+0000","last_change":"2024-09-20T22:06:45.030549+0000","last_active":"2024-09-20T22:07:55.192589+0000","last_peered":"2024-09-20T22:07:55.192589+0000","last_clean":"2024-09-20T22:07:55.192589+0000","last_became_active":"2024-09-20T22:02:56.349376+0000","last_became_peered":"2024-09-20T22:02:56.349376+0000","last_unstale":"2024-09-20T22:07:55.192589+0000","last_undegraded":"2024-09-20T22:07:55.192589+0000","last_fullsized":"2024-09-20T22:07:55.192589+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.030289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.030289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:55.774314+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":4,"reported_epoch":177,"state":"creating+activating","last_fresh":"2024-09-20T22:07:52.425297+0000","last_change":"2024-09-20T22:07:52.420769+0000","last_active":"2024-09-20T22:07:51.409224+0000","last_peered":"2024-09-20T22:07:51.409224+0000","last_clean":"2024-09-20T22:07:51.409224+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-20T22:07:52.425297+0000","last_undegraded":"2024-09-20T22:07:52.425297+0000","last_fullsized":"2024-09-20T22:07:52.425297+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":["1","0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"71'17","reported_seq":292,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.528815+0000","last_change":"2024-09-20T22:07:54.528854+0000","last_active":"2024-09-20T22:07:54.528815+0000","last_peered":"2024-09-20T22:07:54.528815+0000","last_clean":"2024-09-20T22:07:54.528815+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:07:54.528815+0000","last_undegraded":"2024-09-20T22:07:54.528815+0000","last_fullsized":"2024-09-20T22:07:54.528815+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'17","last_scrub_stamp":"2024-09-20T22:07:54.528759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:54.528759+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.170193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":259,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208470+0000","last_change":"2024-09-20T22:07:21.124897+0000","last_active":"2024-09-20T22:07:55.208470+0000","last_peered":"2024-09-20T22:07:55.208470+0000","last_clean":"2024-09-20T22:07:55.208470+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:07:55.208470+0000","last_undegraded":"2024-09-20T22:07:55.208470+0000","last_fullsized":"2024-09-20T22:07:55.208470+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:21.124789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:21.124789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:29.000115+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":92,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-20T22:07:52.424651+0000","last_change":"2024-09-20T22:07:19.376008+0000","last_active":"2024-09-20T22:07:52.424651+0000","last_peered":"2024-09-20T22:07:52.424651+0000","last_clean":"2024-09-20T22:07:52.424651+0000","last_became_active":"2024-09-20T22:06:54.848275+0000","last_became_peered":"2024-09-20T22:06:54.848275+0000","last_unstale":"2024-09-20T22:07:52.424651+0000","last_undegraded":"2024-09-20T22:07:52.424651+0000","last_fullsized":"2024-09-20T22:07:52.424651+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:19.375867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.375867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:49.161984+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":254,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.195195+0000","last_change":"2024-09-20T22:07:55.195229+0000","last_active":"2024-09-20T22:07:55.195195+0000","last_peered":"2024-09-20T22:07:55.195195+0000","last_clean":"2024-09-20T22:07:55.195195+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:07:55.195195+0000","last_undegraded":"2024-09-20T22:07:55.195195+0000","last_fullsized":"2024-09-20T22:07:55.195195+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:55.195131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.195131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:10.187674+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":16,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.192147+0000","last_change":"2024-09-20T22:07:52.429344+0000","last_active":"2024-09-20T22:07:55.192147+0000","last_peered":"2024-09-20T22:07:55.192147+0000","last_clean":"2024-09-20T22:07:55.192147+0000","last_became_active":"2024-09-20T22:07:52.429054+0000","last_became_peered":"2024-09-20T22:07:52.429054+0000","last_unstale":"2024-09-20T22:07:55.192147+0000","last_undegraded":"2024-09-20T22:07:55.192147+0000","last_fullsized":"2024-09-20T22:07:55.192147+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:16.620658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"71'10","reported_seq":262,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.527279+0000","last_change":"2024-09-20T22:05:27.048378+0000","last_active":"2024-09-20T22:07:54.527279+0000","last_peered":"2024-09-20T22:07:54.527279+0000","last_clean":"2024-09-20T22:07:54.527279+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:07:54.527279+0000","last_undegraded":"2024-09-20T22:07:54.527279+0000","last_fullsized":"2024-09-20T22:07:54.527279+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:05:27.048215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:27.048215+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"64'1","reported_seq":264,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.192702+0000","last_change":"2024-09-20T22:06:33.966005+0000","last_active":"2024-09-20T22:07:55.192702+0000","last_peered":"2024-09-20T22:07:55.192702+0000","last_clean":"2024-09-20T22:07:55.192702+0000","last_became_active":"2024-09-20T22:02:50.110446+0000","last_became_peered":"2024-09-20T22:02:50.110446+0000","last_unstale":"2024-09-20T22:07:55.192702+0000","last_undegraded":"2024-09-20T22:07:55.192702+0000","last_fullsized":"2024-09-20T22:07:55.192702+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:06:33.965866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:33.965866+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"139'9","reported_seq":329,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.433210+0000","last_change":"2024-09-20T22:06:40.910132+0000","last_active":"2024-09-20T22:07:54.433210+0000","last_peered":"2024-09-20T22:07:54.433210+0000","last_clean":"2024-09-20T22:07:54.433210+0000","last_became_active":"2024-09-20T22:02:54.294458+0000","last_became_peered":"2024-09-20T22:02:54.294458+0000","last_unstale":"2024-09-20T22:07:54.433210+0000","last_undegraded":"2024-09-20T22:07:54.433210+0000","last_fullsized":"2024-09-20T22:07:54.433210+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'9","last_scrub_stamp":"2024-09-20T22:06:40.910011+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:06:40.910011+0000","last_clean_scrub_stamp":"2024-09-20T22:06:40.910011+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":248,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.527926+0000","last_change":"2024-09-20T22:07:23.482514+0000","last_active":"2024-09-20T22:07:54.527926+0000","last_peered":"2024-09-20T22:07:54.527926+0000","last_clean":"2024-09-20T22:07:54.527926+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:07:54.527926+0000","last_undegraded":"2024-09-20T22:07:54.527926+0000","last_fullsized":"2024-09-20T22:07:54.527926+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.482372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.886444+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":10,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-20T22:07:52.430032+0000","last_change":"2024-09-20T22:07:52.429071+0000","last_active":"2024-09-20T22:07:52.430032+0000","last_peered":"2024-09-20T22:07:52.430032+0000","last_clean":"2024-09-20T22:07:52.430032+0000","last_became_active":"2024-09-20T22:07:52.428789+0000","last_became_peered":"2024-09-20T22:07:52.428789+0000","last_unstale":"2024-09-20T22:07:52.430032+0000","last_undegraded":"2024-09-20T22:07:52.430032+0000","last_fullsized":"2024-09-20T22:07:52.430032+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:14.700506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"71'9","reported_seq":271,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208249+0000","last_change":"2024-09-20T22:06:54.848100+0000","last_active":"2024-09-20T22:07:55.208249+0000","last_peered":"2024-09-20T22:07:55.208249+0000","last_clean":"2024-09-20T22:07:55.208249+0000","last_became_active":"2024-09-20T22:06:54.847770+0000","last_became_peered":"2024-09-20T22:06:54.847770+0000","last_unstale":"2024-09-20T22:07:55.208249+0000","last_undegraded":"2024-09-20T22:07:55.208249+0000","last_fullsized":"2024-09-20T22:07:55.208249+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:03:52.011201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:03:52.011201+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":264,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.433482+0000","last_change":"2024-09-20T22:06:36.582728+0000","last_active":"2024-09-20T22:07:54.433482+0000","last_peered":"2024-09-20T22:07:54.433482+0000","last_clean":"2024-09-20T22:07:54.433482+0000","last_became_active":"2024-09-20T22:02:50.112682+0000","last_became_peered":"2024-09-20T22:02:50.112682+0000","last_unstale":"2024-09-20T22:07:54.433482+0000","last_undegraded":"2024-09-20T22:07:54.433482+0000","last_fullsized":"2024-09-20T22:07:54.433482+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:06:36.582537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:36.582537+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":258,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.432375+0000","last_change":"2024-09-20T22:07:22.203673+0000","last_active":"2024-09-20T22:07:54.432375+0000","last_peered":"2024-09-20T22:07:54.432375+0000","last_clean":"2024-09-20T22:07:54.432375+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:07:54.432375+0000","last_undegraded":"2024-09-20T22:07:54.432375+0000","last_fullsized":"2024-09-20T22:07:54.432375+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:22.203576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.203576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.064543+0000","scrub_duration":49,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":248,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208245+0000","last_change":"2024-09-20T22:06:55.824784+0000","last_active":"2024-09-20T22:07:55.208245+0000","last_peered":"2024-09-20T22:07:55.208245+0000","last_clean":"2024-09-20T22:07:55.208245+0000","last_became_active":"2024-09-20T22:06:55.824229+0000","last_became_peered":"2024-09-20T22:06:55.824229+0000","last_unstale":"2024-09-20T22:07:55.208245+0000","last_undegraded":"2024-09-20T22:07:55.208245+0000","last_fullsized":"2024-09-20T22:07:55.208245+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:17.171523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:17.171523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":16,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208535+0000","last_change":"2024-09-20T22:07:52.674668+0000","last_active":"2024-09-20T22:07:55.208535+0000","last_peered":"2024-09-20T22:07:55.208535+0000","last_clean":"2024-09-20T22:07:55.208535+0000","last_became_active":"2024-09-20T22:07:52.674301+0000","last_became_peered":"2024-09-20T22:07:52.674301+0000","last_unstale":"2024-09-20T22:07:55.208535+0000","last_undegraded":"2024-09-20T22:07:55.208535+0000","last_fullsized":"2024-09-20T22:07:55.208535+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:06.988552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"71'19","reported_seq":286,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208149+0000","last_change":"2024-09-20T22:07:25.155015+0000","last_active":"2024-09-20T22:07:55.208149+0000","last_peered":"2024-09-20T22:07:55.208149+0000","last_clean":"2024-09-20T22:07:55.208149+0000","last_became_active":"2024-09-20T22:06:54.829170+0000","last_became_peered":"2024-09-20T22:06:54.829170+0000","last_unstale":"2024-09-20T22:07:55.208149+0000","last_undegraded":"2024-09-20T22:07:55.208149+0000","last_fullsized":"2024-09-20T22:07:55.208149+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:07:25.154919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:25.154919+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:43.954499+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":251,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.686545+0000","last_change":"2024-09-20T22:05:05.659544+0000","last_active":"2024-09-20T22:07:54.686545+0000","last_peered":"2024-09-20T22:07:54.686545+0000","last_clean":"2024-09-20T22:07:54.686545+0000","last_became_active":"2024-09-20T22:02:50.108712+0000","last_became_peered":"2024-09-20T22:02:50.108712+0000","last_unstale":"2024-09-20T22:07:54.686545+0000","last_undegraded":"2024-09-20T22:07:54.686545+0000","last_fullsized":"2024-09-20T22:07:54.686545+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:05.659437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:05.659437+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":258,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.432191+0000","last_change":"2024-09-20T22:07:20.217991+0000","last_active":"2024-09-20T22:07:54.432191+0000","last_peered":"2024-09-20T22:07:54.432191+0000","last_clean":"2024-09-20T22:07:54.432191+0000","last_became_active":"2024-09-20T22:02:54.291134+0000","last_became_peered":"2024-09-20T22:02:54.291134+0000","last_unstale":"2024-09-20T22:07:54.432191+0000","last_undegraded":"2024-09-20T22:07:54.432191+0000","last_fullsized":"2024-09-20T22:07:54.432191+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:20.217761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:20.217761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:42.617706+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":247,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.686636+0000","last_change":"2024-09-20T22:07:29.645352+0000","last_active":"2024-09-20T22:07:54.686636+0000","last_peered":"2024-09-20T22:07:54.686636+0000","last_clean":"2024-09-20T22:07:54.686636+0000","last_became_active":"2024-09-20T22:02:56.307687+0000","last_became_peered":"2024-09-20T22:02:56.307687+0000","last_unstale":"2024-09-20T22:07:54.686636+0000","last_undegraded":"2024-09-20T22:07:54.686636+0000","last_fullsized":"2024-09-20T22:07:54.686636+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:29.645127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:29.645127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:36.059611+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":16,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208579+0000","last_change":"2024-09-20T22:07:52.675309+0000","last_active":"2024-09-20T22:07:55.208579+0000","last_peered":"2024-09-20T22:07:55.208579+0000","last_clean":"2024-09-20T22:07:55.208579+0000","last_became_active":"2024-09-20T22:07:52.674921+0000","last_became_peered":"2024-09-20T22:07:52.674921+0000","last_unstale":"2024-09-20T22:07:55.208579+0000","last_undegraded":"2024-09-20T22:07:55.208579+0000","last_fullsized":"2024-09-20T22:07:55.208579+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:17.687123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"71'12","reported_seq":279,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.526901+0000","last_change":"2024-09-20T22:07:22.458421+0000","last_active":"2024-09-20T22:07:54.526901+0000","last_peered":"2024-09-20T22:07:54.526901+0000","last_clean":"2024-09-20T22:07:54.526901+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:07:54.526901+0000","last_undegraded":"2024-09-20T22:07:54.526901+0000","last_fullsized":"2024-09-20T22:07:54.526901+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:22.458279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.458279+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:33.412138+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":260,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.686859+0000","last_change":"2024-09-20T22:07:21.584606+0000","last_active":"2024-09-20T22:07:54.686859+0000","last_peered":"2024-09-20T22:07:54.686859+0000","last_clean":"2024-09-20T22:07:54.686859+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:07:54.686859+0000","last_undegraded":"2024-09-20T22:07:54.686859+0000","last_fullsized":"2024-09-20T22:07:54.686859+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:21.584499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:21.584499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:24.004976+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":259,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.432217+0000","last_change":"2024-09-20T22:07:15.260254+0000","last_active":"2024-09-20T22:07:54.432217+0000","last_peered":"2024-09-20T22:07:54.432217+0000","last_clean":"2024-09-20T22:07:54.432217+0000","last_became_active":"2024-09-20T22:06:54.843906+0000","last_became_peered":"2024-09-20T22:06:54.843906+0000","last_unstale":"2024-09-20T22:07:54.432217+0000","last_undegraded":"2024-09-20T22:07:54.432217+0000","last_fullsized":"2024-09-20T22:07:54.432217+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:15.260148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.260148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:20.976588+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":192,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208515+0000","last_change":"2024-09-20T22:06:55.822136+0000","last_active":"2024-09-20T22:07:55.208515+0000","last_peered":"2024-09-20T22:07:55.208515+0000","last_clean":"2024-09-20T22:07:55.208515+0000","last_became_active":"2024-09-20T22:06:55.821647+0000","last_became_peered":"2024-09-20T22:06:55.821647+0000","last_unstale":"2024-09-20T22:07:55.208515+0000","last_undegraded":"2024-09-20T22:07:55.208515+0000","last_fullsized":"2024-09-20T22:07:55.208515+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.569045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.569045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":16,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.433111+0000","last_change":"2024-09-20T22:07:52.939846+0000","last_active":"2024-09-20T22:07:54.433111+0000","last_peered":"2024-09-20T22:07:54.433111+0000","last_clean":"2024-09-20T22:07:54.433111+0000","last_became_active":"2024-09-20T22:07:52.939563+0000","last_became_peered":"2024-09-20T22:07:52.939563+0000","last_unstale":"2024-09-20T22:07:54.433111+0000","last_undegraded":"2024-09-20T22:07:54.433111+0000","last_fullsized":"2024-09-20T22:07:54.433111+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:55.690792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"71'15","reported_seq":279,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.432756+0000","last_change":"2024-09-20T22:06:42.563403+0000","last_active":"2024-09-20T22:07:54.432756+0000","last_peered":"2024-09-20T22:07:54.432756+0000","last_clean":"2024-09-20T22:07:54.432756+0000","last_became_active":"2024-09-20T22:02:52.482008+0000","last_became_peered":"2024-09-20T22:02:52.482008+0000","last_unstale":"2024-09-20T22:07:54.432756+0000","last_undegraded":"2024-09-20T22:07:54.432756+0000","last_fullsized":"2024-09-20T22:07:54.432756+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:06:42.563266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:42.563266+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:02.694507+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":273,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.686323+0000","last_change":"2024-09-20T22:06:40.813723+0000","last_active":"2024-09-20T22:07:54.686323+0000","last_peered":"2024-09-20T22:07:54.686323+0000","last_clean":"2024-09-20T22:07:54.686323+0000","last_became_active":"2024-09-20T22:02:50.114819+0000","last_became_peered":"2024-09-20T22:02:50.114819+0000","last_unstale":"2024-09-20T22:07:54.686323+0000","last_undegraded":"2024-09-20T22:07:54.686323+0000","last_fullsized":"2024-09-20T22:07:54.686323+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:06:40.813590+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":313,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.685744+0000","last_change":"2024-09-20T22:05:34.732761+0000","last_active":"2024-09-20T22:07:54.685744+0000","last_peered":"2024-09-20T22:07:54.685744+0000","last_clean":"2024-09-20T22:07:54.685744+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:07:54.685744+0000","last_undegraded":"2024-09-20T22:07:54.685744+0000","last_fullsized":"2024-09-20T22:07:54.685744+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:05:34.732644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:34.732644+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":246,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.432743+0000","last_change":"2024-09-20T22:06:35.559426+0000","last_active":"2024-09-20T22:07:54.432743+0000","last_peered":"2024-09-20T22:07:54.432743+0000","last_clean":"2024-09-20T22:07:54.432743+0000","last_became_active":"2024-09-20T22:02:56.346990+0000","last_became_peered":"2024-09-20T22:02:56.346990+0000","last_unstale":"2024-09-20T22:07:54.432743+0000","last_undegraded":"2024-09-20T22:07:54.432743+0000","last_fullsized":"2024-09-20T22:07:54.432743+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:35.559306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:35.559306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":258,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208148+0000","last_change":"2024-09-20T22:06:34.392493+0000","last_active":"2024-09-20T22:07:55.208148+0000","last_peered":"2024-09-20T22:07:55.208148+0000","last_clean":"2024-09-20T22:07:55.208148+0000","last_became_active":"2024-09-20T22:02:50.111935+0000","last_became_peered":"2024-09-20T22:02:50.111935+0000","last_unstale":"2024-09-20T22:07:55.208148+0000","last_undegraded":"2024-09-20T22:07:55.208148+0000","last_fullsized":"2024-09-20T22:07:55.208148+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:34.391904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:34.391904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:58.191440+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"71'6","reported_seq":256,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.193153+0000","last_change":"2024-09-20T22:06:54.862697+0000","last_active":"2024-09-20T22:07:55.193153+0000","last_peered":"2024-09-20T22:07:55.193153+0000","last_clean":"2024-09-20T22:07:55.193153+0000","last_became_active":"2024-09-20T22:06:54.862227+0000","last_became_peered":"2024-09-20T22:06:54.862227+0000","last_unstale":"2024-09-20T22:07:55.193153+0000","last_undegraded":"2024-09-20T22:07:55.193153+0000","last_fullsized":"2024-09-20T22:07:55.193153+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":251,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.433593+0000","last_change":"2024-09-20T22:07:19.865906+0000","last_active":"2024-09-20T22:07:54.433593+0000","last_peered":"2024-09-20T22:07:54.433593+0000","last_clean":"2024-09-20T22:07:54.433593+0000","last_became_active":"2024-09-20T22:06:54.836200+0000","last_became_peered":"2024-09-20T22:06:54.836200+0000","last_unstale":"2024-09-20T22:07:54.433593+0000","last_undegraded":"2024-09-20T22:07:54.433593+0000","last_fullsized":"2024-09-20T22:07:54.433593+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:19.865809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.865809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:42.730889+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":239,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.686211+0000","last_change":"2024-09-20T22:06:39.783526+0000","last_active":"2024-09-20T22:07:54.686211+0000","last_peered":"2024-09-20T22:07:54.686211+0000","last_clean":"2024-09-20T22:07:54.686211+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:07:54.686211+0000","last_undegraded":"2024-09-20T22:07:54.686211+0000","last_fullsized":"2024-09-20T22:07:54.686211+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:39.783456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:39.783456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:59.986149+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"71'9","reported_seq":262,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208726+0000","last_change":"2024-09-20T22:04:52.254864+0000","last_active":"2024-09-20T22:07:55.208726+0000","last_peered":"2024-09-20T22:07:55.208726+0000","last_clean":"2024-09-20T22:07:55.208726+0000","last_became_active":"2024-09-20T22:02:52.481989+0000","last_became_peered":"2024-09-20T22:02:52.481989+0000","last_unstale":"2024-09-20T22:07:55.208726+0000","last_undegraded":"2024-09-20T22:07:55.208726+0000","last_fullsized":"2024-09-20T22:07:55.208726+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:52.254706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:52.254706+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":251,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.685968+0000","last_change":"2024-09-20T22:05:31.764903+0000","last_active":"2024-09-20T22:07:54.685968+0000","last_peered":"2024-09-20T22:07:54.685968+0000","last_clean":"2024-09-20T22:07:54.685968+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:07:54.685968+0000","last_undegraded":"2024-09-20T22:07:54.685968+0000","last_fullsized":"2024-09-20T22:07:54.685968+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:31.764833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:31.764833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":243,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.193058+0000","last_change":"2024-09-20T22:06:54.847169+0000","last_active":"2024-09-20T22:07:55.193058+0000","last_peered":"2024-09-20T22:07:55.193058+0000","last_clean":"2024-09-20T22:07:55.193058+0000","last_became_active":"2024-09-20T22:06:54.845704+0000","last_became_peered":"2024-09-20T22:06:54.845704+0000","last_unstale":"2024-09-20T22:07:55.193058+0000","last_undegraded":"2024-09-20T22:07:55.193058+0000","last_fullsized":"2024-09-20T22:07:55.193058+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":246,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.432408+0000","last_change":"2024-09-20T22:06:39.942296+0000","last_active":"2024-09-20T22:07:54.432408+0000","last_peered":"2024-09-20T22:07:54.432408+0000","last_clean":"2024-09-20T22:07:54.432408+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:07:54.432408+0000","last_undegraded":"2024-09-20T22:07:54.432408+0000","last_fullsized":"2024-09-20T22:07:54.432408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:39.942186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:39.942186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:04.710791+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"71'9","reported_seq":278,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.433585+0000","last_change":"2024-09-20T22:07:52.834986+0000","last_active":"2024-09-20T22:07:54.433585+0000","last_peered":"2024-09-20T22:07:54.433585+0000","last_clean":"2024-09-20T22:07:54.433585+0000","last_became_active":"2024-09-20T22:02:52.482141+0000","last_became_peered":"2024-09-20T22:02:52.482141+0000","last_unstale":"2024-09-20T22:07:54.433585+0000","last_undegraded":"2024-09-20T22:07:54.433585+0000","last_fullsized":"2024-09-20T22:07:54.433585+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:52.834898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:52.834898+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.590814+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":258,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208001+0000","last_change":"2024-09-20T22:06:45.908734+0000","last_active":"2024-09-20T22:07:55.208001+0000","last_peered":"2024-09-20T22:07:55.208001+0000","last_clean":"2024-09-20T22:07:55.208001+0000","last_became_active":"2024-09-20T22:02:50.110075+0000","last_became_peered":"2024-09-20T22:02:50.110075+0000","last_unstale":"2024-09-20T22:07:55.208001+0000","last_undegraded":"2024-09-20T22:07:55.208001+0000","last_fullsized":"2024-09-20T22:07:55.208001+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.908616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.908616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:56.429713+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"71'8","reported_seq":312,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.193045+0000","last_change":"2024-09-20T22:05:55.972684+0000","last_active":"2024-09-20T22:07:55.193045+0000","last_peered":"2024-09-20T22:07:55.193045+0000","last_clean":"2024-09-20T22:07:55.193045+0000","last_became_active":"2024-09-20T22:02:54.294381+0000","last_became_peered":"2024-09-20T22:02:54.294381+0000","last_unstale":"2024-09-20T22:07:55.193045+0000","last_undegraded":"2024-09-20T22:07:55.193045+0000","last_fullsized":"2024-09-20T22:07:55.193045+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:05:55.972565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:55.972565+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":239,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.527759+0000","last_change":"2024-09-20T22:05:00.897974+0000","last_active":"2024-09-20T22:07:54.527759+0000","last_peered":"2024-09-20T22:07:54.527759+0000","last_clean":"2024-09-20T22:07:54.527759+0000","last_became_active":"2024-09-20T22:02:56.300041+0000","last_became_peered":"2024-09-20T22:02:56.300041+0000","last_unstale":"2024-09-20T22:07:54.527759+0000","last_undegraded":"2024-09-20T22:07:54.527759+0000","last_fullsized":"2024-09-20T22:07:54.527759+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:00.897869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:00.897869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"71'10","reported_seq":262,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.193078+0000","last_change":"2024-09-20T22:06:54.862610+0000","last_active":"2024-09-20T22:07:55.193078+0000","last_peered":"2024-09-20T22:07:55.193078+0000","last_clean":"2024-09-20T22:07:55.193078+0000","last_became_active":"2024-09-20T22:06:54.862289+0000","last_became_peered":"2024-09-20T22:06:54.862289+0000","last_unstale":"2024-09-20T22:07:55.193078+0000","last_undegraded":"2024-09-20T22:07:55.193078+0000","last_fullsized":"2024-09-20T22:07:55.193078+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":251,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.686244+0000","last_change":"2024-09-20T22:06:35.687169+0000","last_active":"2024-09-20T22:07:54.686244+0000","last_peered":"2024-09-20T22:07:54.686244+0000","last_clean":"2024-09-20T22:07:54.686244+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:07:54.686244+0000","last_undegraded":"2024-09-20T22:07:54.686244+0000","last_fullsized":"2024-09-20T22:07:54.686244+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:35.687065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:35.687065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:58.652707+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"146'10","reported_seq":323,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.433380+0000","last_change":"2024-09-20T22:07:15.870748+0000","last_active":"2024-09-20T22:07:54.433380+0000","last_peered":"2024-09-20T22:07:54.433380+0000","last_clean":"2024-09-20T22:07:54.433380+0000","last_became_active":"2024-09-20T22:06:54.843097+0000","last_became_peered":"2024-09-20T22:06:54.843097+0000","last_unstale":"2024-09-20T22:07:54.433380+0000","last_undegraded":"2024-09-20T22:07:54.433380+0000","last_fullsized":"2024-09-20T22:07:54.433380+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:07:15.870646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.870646+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:36.803368+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":238,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208238+0000","last_change":"2024-09-20T22:05:35.601891+0000","last_active":"2024-09-20T22:07:55.208238+0000","last_peered":"2024-09-20T22:07:55.208238+0000","last_clean":"2024-09-20T22:07:55.208238+0000","last_became_active":"2024-09-20T22:02:56.305383+0000","last_became_peered":"2024-09-20T22:02:56.305383+0000","last_unstale":"2024-09-20T22:07:55.208238+0000","last_undegraded":"2024-09-20T22:07:55.208238+0000","last_fullsized":"2024-09-20T22:07:55.208238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:35.601794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:35.601794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"71'11","reported_seq":275,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.527360+0000","last_change":"2024-09-20T22:07:15.544790+0000","last_active":"2024-09-20T22:07:54.527360+0000","last_peered":"2024-09-20T22:07:54.527360+0000","last_clean":"2024-09-20T22:07:54.527360+0000","last_became_active":"2024-09-20T22:06:54.823678+0000","last_became_peered":"2024-09-20T22:06:54.823678+0000","last_unstale":"2024-09-20T22:07:54.527360+0000","last_undegraded":"2024-09-20T22:07:54.527360+0000","last_fullsized":"2024-09-20T22:07:54.527360+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:15.544705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.544705+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:32.779498+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":251,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.527877+0000","last_change":"2024-09-20T22:04:54.764407+0000","last_active":"2024-09-20T22:07:54.527877+0000","last_peered":"2024-09-20T22:07:54.527877+0000","last_clean":"2024-09-20T22:07:54.527877+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:07:54.527877+0000","last_undegraded":"2024-09-20T22:07:54.527877+0000","last_fullsized":"2024-09-20T22:07:54.527877+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:54.764137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:54.764137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":84,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-20T22:07:52.422296+0000","last_change":"2024-09-20T22:06:54.853857+0000","last_active":"2024-09-20T22:07:52.422296+0000","last_peered":"2024-09-20T22:07:52.422296+0000","last_clean":"2024-09-20T22:07:52.422296+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:07:52.422296+0000","last_undegraded":"2024-09-20T22:07:52.422296+0000","last_fullsized":"2024-09-20T22:07:52.422296+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.299478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":71,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-20T22:07:52.423443+0000","last_change":"2024-09-20T22:06:55.835713+0000","last_active":"2024-09-20T22:07:52.423443+0000","last_peered":"2024-09-20T22:07:52.423443+0000","last_clean":"2024-09-20T22:07:52.423443+0000","last_became_active":"2024-09-20T22:06:55.835393+0000","last_became_peered":"2024-09-20T22:06:55.835393+0000","last_unstale":"2024-09-20T22:07:52.423443+0000","last_undegraded":"2024-09-20T22:07:52.423443+0000","last_fullsized":"2024-09-20T22:07:52.423443+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:33.623816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:33.623816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.299731+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"71'9","reported_seq":156,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-20T22:07:52.422578+0000","last_change":"2024-09-20T22:06:54.844349+0000","last_active":"2024-09-20T22:07:52.422578+0000","last_peered":"2024-09-20T22:07:52.422578+0000","last_clean":"2024-09-20T22:07:52.422578+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:07:52.422578+0000","last_undegraded":"2024-09-20T22:07:52.422578+0000","last_fullsized":"2024-09-20T22:07:52.422578+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:12.196518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:12.196518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.299591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":267,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.686442+0000","last_change":"2024-09-20T22:07:54.685916+0000","last_active":"2024-09-20T22:07:54.686442+0000","last_peered":"2024-09-20T22:07:54.686442+0000","last_clean":"2024-09-20T22:07:54.686442+0000","last_became_active":"2024-09-20T22:02:50.109803+0000","last_became_peered":"2024-09-20T22:02:50.109803+0000","last_unstale":"2024-09-20T22:07:54.686442+0000","last_undegraded":"2024-09-20T22:07:54.686442+0000","last_fullsized":"2024-09-20T22:07:54.686442+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:54.685751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:54.685751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:04.385036+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":243,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.192565+0000","last_change":"2024-09-20T22:06:54.844570+0000","last_active":"2024-09-20T22:07:55.192565+0000","last_peered":"2024-09-20T22:07:55.192565+0000","last_clean":"2024-09-20T22:07:55.192565+0000","last_became_active":"2024-09-20T22:06:54.844271+0000","last_became_peered":"2024-09-20T22:06:54.844271+0000","last_unstale":"2024-09-20T22:07:55.192565+0000","last_undegraded":"2024-09-20T22:07:55.192565+0000","last_fullsized":"2024-09-20T22:07:55.192565+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":247,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.207640+0000","last_change":"2024-09-20T22:07:23.495005+0000","last_active":"2024-09-20T22:07:55.207640+0000","last_peered":"2024-09-20T22:07:55.207640+0000","last_clean":"2024-09-20T22:07:55.207640+0000","last_became_active":"2024-09-20T22:06:54.848842+0000","last_became_peered":"2024-09-20T22:06:54.848842+0000","last_unstale":"2024-09-20T22:07:55.207640+0000","last_undegraded":"2024-09-20T22:07:55.207640+0000","last_fullsized":"2024-09-20T22:07:55.207640+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.494905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.494905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.999867+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"71'11","reported_seq":273,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.192119+0000","last_change":"2024-09-20T22:06:36.942847+0000","last_active":"2024-09-20T22:07:55.192119+0000","last_peered":"2024-09-20T22:07:55.192119+0000","last_clean":"2024-09-20T22:07:55.192119+0000","last_became_active":"2024-09-20T22:02:52.480118+0000","last_became_peered":"2024-09-20T22:02:52.480118+0000","last_unstale":"2024-09-20T22:07:55.192119+0000","last_undegraded":"2024-09-20T22:07:55.192119+0000","last_fullsized":"2024-09-20T22:07:55.192119+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:06:36.942620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:36.942620+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:55.637693+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":267,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.433685+0000","last_change":"2024-09-20T22:07:50.851948+0000","last_active":"2024-09-20T22:07:54.433685+0000","last_peered":"2024-09-20T22:07:54.433685+0000","last_clean":"2024-09-20T22:07:54.433685+0000","last_became_active":"2024-09-20T22:06:54.828826+0000","last_became_peered":"2024-09-20T22:06:54.828826+0000","last_unstale":"2024-09-20T22:07:54.433685+0000","last_undegraded":"2024-09-20T22:07:54.433685+0000","last_fullsized":"2024-09-20T22:07:54.433685+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:50.851839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:50.851839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:57.182737+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":251,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.685526+0000","last_change":"2024-09-20T22:06:45.780268+0000","last_active":"2024-09-20T22:07:54.685526+0000","last_peered":"2024-09-20T22:07:54.685526+0000","last_clean":"2024-09-20T22:07:54.685526+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:07:54.685526+0000","last_undegraded":"2024-09-20T22:07:54.685526+0000","last_fullsized":"2024-09-20T22:07:54.685526+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.780200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.780200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:06.694833+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":246,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.192636+0000","last_change":"2024-09-20T22:06:33.002428+0000","last_active":"2024-09-20T22:07:55.192636+0000","last_peered":"2024-09-20T22:07:55.192636+0000","last_clean":"2024-09-20T22:07:55.192636+0000","last_became_active":"2024-09-20T22:02:56.347637+0000","last_became_peered":"2024-09-20T22:02:56.347637+0000","last_unstale":"2024-09-20T22:07:55.192636+0000","last_undegraded":"2024-09-20T22:07:55.192636+0000","last_fullsized":"2024-09-20T22:07:55.192636+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:33.002326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:33.002326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"71'4","reported_seq":261,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.192211+0000","last_change":"2024-09-20T22:07:19.309318+0000","last_active":"2024-09-20T22:07:55.192211+0000","last_peered":"2024-09-20T22:07:55.192211+0000","last_clean":"2024-09-20T22:07:55.192211+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:07:55.192211+0000","last_undegraded":"2024-09-20T22:07:55.192211+0000","last_fullsized":"2024-09-20T22:07:55.192211+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'4","last_scrub_stamp":"2024-09-20T22:07:19.309187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.309187+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:33.061321+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":250,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208422+0000","last_change":"2024-09-20T22:04:16.198264+0000","last_active":"2024-09-20T22:07:55.208422+0000","last_peered":"2024-09-20T22:07:55.208422+0000","last_clean":"2024-09-20T22:07:55.208422+0000","last_became_active":"2024-09-20T22:02:50.111142+0000","last_became_peered":"2024-09-20T22:02:50.111142+0000","last_unstale":"2024-09-20T22:07:55.208422+0000","last_undegraded":"2024-09-20T22:07:55.208422+0000","last_fullsized":"2024-09-20T22:07:55.208422+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:16.198153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:16.198153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":266,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.255911+0000","last_change":"2024-09-20T22:07:55.255933+0000","last_active":"2024-09-20T22:07:55.255911+0000","last_peered":"2024-09-20T22:07:55.255911+0000","last_clean":"2024-09-20T22:07:55.255911+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:07:55.255911+0000","last_undegraded":"2024-09-20T22:07:55.255911+0000","last_fullsized":"2024-09-20T22:07:55.255911+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:55.255854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.255854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.819675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":239,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:54.685741+0000","last_change":"2024-09-20T22:06:01.621567+0000","last_active":"2024-09-20T22:07:54.685741+0000","last_peered":"2024-09-20T22:07:54.685741+0000","last_clean":"2024-09-20T22:07:54.685741+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:07:54.685741+0000","last_undegraded":"2024-09-20T22:07:54.685741+0000","last_fullsized":"2024-09-20T22:07:54.685741+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:01.621489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:01.621489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":132,"reported_epoch":177,"state":"active+clean","last_fresh":"2024-09-20T22:07:52.424705+0000","last_change":"2024-09-20T22:06:54.828772+0000","last_active":"2024-09-20T22:07:52.424705+0000","last_peered":"2024-09-20T22:07:52.424705+0000","last_clean":"2024-09-20T22:07:52.424705+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:07:52.424705+0000","last_undegraded":"2024-09-20T22:07:52.424705+0000","last_fullsized":"2024-09-20T22:07:52.424705+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:15.303300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:15.303300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:07:53.299662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":251,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.192529+0000","last_change":"2024-09-20T22:06:54.848978+0000","last_active":"2024-09-20T22:07:55.192529+0000","last_peered":"2024-09-20T22:07:55.192529+0000","last_clean":"2024-09-20T22:07:55.192529+0000","last_became_active":"2024-09-20T22:06:54.848032+0000","last_became_peered":"2024-09-20T22:06:54.848032+0000","last_unstale":"2024-09-20T22:07:55.192529+0000","last_undegraded":"2024-09-20T22:07:55.192529+0000","last_fullsized":"2024-09-20T22:07:55.192529+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"71'11","reported_seq":274,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208829+0000","last_change":"2024-09-20T22:07:19.214716+0000","last_active":"2024-09-20T22:07:55.208829+0000","last_peered":"2024-09-20T22:07:55.208829+0000","last_clean":"2024-09-20T22:07:55.208829+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:07:55.208829+0000","last_undegraded":"2024-09-20T22:07:55.208829+0000","last_fullsized":"2024-09-20T22:07:55.208829+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:19.214636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.214636+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:32.168748+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":242,"reported_epoch":179,"state":"active+clean","last_fresh":"2024-09-20T22:07:55.208512+0000","last_change":"2024-09-20T22:05:06.706314+0000","last_active":"2024-09-20T22:07:55.208512+0000","last_peered":"2024-09-20T22:07:55.208512+0000","last_clean":"2024-09-20T22:07:55.208512+0000","last_became_active":"2024-09-20T22:02:54.289078+0000","last_became_peered":"2024-09-20T22:02:54.289078+0000","last_unstale":"2024-09-20T22:07:55.208512+0000","last_undegraded":"2024-09-20T22:07:55.208512+0000","last_fullsized":"2024-09-20T22:07:55.208512+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:06.681123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:06.681123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":82,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":292,"num_read_kb":287,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135937,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":186084,"kb_used_data":2184,"kb_used_omap":1,"kb_used_meta":183870,"kb_avail":93559068,"statfs":{"total":95995035648,"available":95804485632,"internally_reserved":0,"allocated":2236416,"data_stored":1479468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":188283338},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430277,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":172188,"kb_used_data":2180,"kb_used_omap":1,"kb_used_meta":169982,"kb_avail":93572964,"statfs":{"total":95995035648,"available":95818715136,"internally_reserved":0,"allocated":2232320,"data_stored":1487942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":174062022},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724617,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":136924,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":135294,"kb_avail":93608228,"statfs":{"total":95995035648,"available":95854825472,"internally_reserved":0,"allocated":1638400,"data_stored":897920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":138541509},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986257,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":154984,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":153342,"kb_avail":93590168,"statfs":{"total":95995035648,"available":95836332032,"internally_reserved":0,"allocated":1662976,"data_stored":897609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":157022661},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770257933,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":109060,"kb_used_data":1620,"kb_used_omap":4,"kb_used_meta":107387,"kb_avail":93636092,"statfs":{"total":95995035648,"available":95883358208,"internally_reserved":0,"allocated":1658880,"data_stored":872986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":109964968},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149782,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":311868,"kb_used_data":1640,"kb_used_omap":1,"kb_used_meta":310206,"kb_avail":93433284,"statfs":{"total":95995035648,"available":95675682816,"internally_reserved":0,"allocated":1679360,"data_stored":898901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":317651399},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411419,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":217424,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":215806,"kb_avail":93527728,"statfs":{"total":95995035648,"available":95772393472,"internally_reserved":0,"allocated":1626112,"data_stored":896716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":220985797},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738470,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":316592,"kb_used_data":2200,"kb_used_omap":1,"kb_used_meta":314366,"kb_avail":93428560,"statfs":{"total":95995035648,"available":95670845440,"internally_reserved":0,"allocated":2252800,"data_stored":1488166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":321911237},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:07:58.060 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph status --format=json 2024-09-20T22:07:58.079 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:58.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:58.080 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:07:58.132 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 1 16 3375 3359 209.745 209.938 0.00302707 0.00474588 2024-09-20T22:07:58.181 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:58.211 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:58.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:58.212 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:58.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:58.345 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:58.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:58.346 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:58.384 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:07:58.384 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:07:58.447 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:58.480 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:58.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:58.481 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:58.582 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:58.616 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:58.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:58.617 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:58.718 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:58.746 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:58 smithi060 ceph-mon[29694]: 5.d scrub starts 2024-09-20T22:07:58.747 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:58 smithi060 ceph-mon[29694]: 5.d scrub ok 2024-09-20T22:07:58.747 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:58 smithi060 ceph-mon[29694]: 3.18 scrub starts 2024-09-20T22:07:58.747 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:58 smithi060 ceph-mon[29694]: 3.18 scrub ok 2024-09-20T22:07:58.747 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:58 smithi060 ceph-mon[29694]: pgmap v249: 148 pgs: 148 active+clean; 26 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 914 B/s rd, 4.5 MiB/s wr, 72 op/s 2024-09-20T22:07:58.747 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:58 smithi060 ceph-mon[40290]: 5.d scrub starts 2024-09-20T22:07:58.747 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:58 smithi060 ceph-mon[40290]: 5.d scrub ok 2024-09-20T22:07:58.747 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:58 smithi060 ceph-mon[40290]: 3.18 scrub starts 2024-09-20T22:07:58.747 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:58 smithi060 ceph-mon[40290]: 3.18 scrub ok 2024-09-20T22:07:58.747 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:58 smithi060 ceph-mon[40290]: pgmap v249: 148 pgs: 148 active+clean; 26 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 914 B/s rd, 4.5 MiB/s wr, 72 op/s 2024-09-20T22:07:58.760 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:58.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:58.761 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:58 smithi067 ceph-mon[33511]: 5.d scrub starts 2024-09-20T22:07:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:58 smithi067 ceph-mon[33511]: 5.d scrub ok 2024-09-20T22:07:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:58 smithi067 ceph-mon[33511]: 3.18 scrub starts 2024-09-20T22:07:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:58 smithi067 ceph-mon[33511]: 3.18 scrub ok 2024-09-20T22:07:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:58 smithi067 ceph-mon[33511]: pgmap v249: 148 pgs: 148 active+clean; 26 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 914 B/s rd, 4.5 MiB/s wr, 72 op/s 2024-09-20T22:07:58.862 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:07:58.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:58.904 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:58.904 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:07:59.005 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:07:59.047 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:59.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:59.048 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:07:59.132 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 2 16 6567 6551 204.615 199.5 0.00376551 0.00487759 2024-09-20T22:07:59.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:59.174 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:59.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:59.175 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:07:59.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:07:59.317 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:59.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:59.318 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:07:59.358 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:07:59.419 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:07:59.449 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:59.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:59.450 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:07:59.459 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T22:07:59.484 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:07:59.551 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:59.581 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:59.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:59.583 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:59.663 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:59 smithi060 ceph-mon[40290]: 3.6 scrub starts 2024-09-20T22:07:59.663 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:07:59 smithi060 ceph-mon[40290]: 3.6 scrub ok 2024-09-20T22:07:59.663 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:59 smithi060 ceph-mon[29694]: 3.6 scrub starts 2024-09-20T22:07:59.664 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:07:59 smithi060 ceph-mon[29694]: 3.6 scrub ok 2024-09-20T22:07:59.683 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:07:59.714 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:59.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:59.715 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:07:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:59 smithi067 ceph-mon[33511]: 3.6 scrub starts 2024-09-20T22:07:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:07:59 smithi067 ceph-mon[33511]: 3.6 scrub ok 2024-09-20T22:07:59.816 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:07:59.843 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:07:59.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:59.844 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:07:59.945 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:07:59.983 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:07:59.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:07:59.985 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:00.086 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:00.111 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:00.133 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 3 16 9215 9199 191.575 165.5 0.00801927 0.00520477 2024-09-20T22:08:00.143 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:00.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:00.144 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:00.205 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:08:00.206 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:08:00.245 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:00.247 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:08:00.247 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:08:00.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:00.285 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:00.285 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:00.285 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:08:00.285 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:08:00.325 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:08:00.325 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:08:00.344 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:00.360 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:08:00.361 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:08:00.386 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:00.398 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:08:00.398 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:08:00.420 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:00.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:00.421 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:00.432 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:08:00.432 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:08:00.466 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:08:00.467 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:08:00.503 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:00 smithi060 ceph-mon[29694]: 4.2 scrub starts 2024-09-20T22:08:00.503 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:00 smithi060 ceph-mon[29694]: 4.2 scrub ok 2024-09-20T22:08:00.503 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:00 smithi060 ceph-mon[29694]: 4.b scrub starts 2024-09-20T22:08:00.503 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:00 smithi060 ceph-mon[29694]: 4.b scrub ok 2024-09-20T22:08:00.503 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:00 smithi060 ceph-mon[29694]: pgmap v250: 148 pgs: 148 active+clean; 26 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 776 B/s rd, 3.8 MiB/s wr, 61 op/s 2024-09-20T22:08:00.503 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:00 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2483250359' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:00.504 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:00 smithi060 ceph-mon[40290]: 4.2 scrub starts 2024-09-20T22:08:00.504 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:00 smithi060 ceph-mon[40290]: 4.2 scrub ok 2024-09-20T22:08:00.504 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:00 smithi060 ceph-mon[40290]: 4.b scrub starts 2024-09-20T22:08:00.504 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:00 smithi060 ceph-mon[40290]: 4.b scrub ok 2024-09-20T22:08:00.504 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:00 smithi060 ceph-mon[40290]: pgmap v250: 148 pgs: 148 active+clean; 26 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 776 B/s rd, 3.8 MiB/s wr, 61 op/s 2024-09-20T22:08:00.504 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:00 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2483250359' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:00.522 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:00.564 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:00.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:00.565 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:00.666 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:00.707 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:00.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:00.708 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:00 smithi067 ceph-mon[33511]: 4.2 scrub starts 2024-09-20T22:08:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:00 smithi067 ceph-mon[33511]: 4.2 scrub ok 2024-09-20T22:08:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:00 smithi067 ceph-mon[33511]: 4.b scrub starts 2024-09-20T22:08:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:00 smithi067 ceph-mon[33511]: 4.b scrub ok 2024-09-20T22:08:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:00 smithi067 ceph-mon[33511]: pgmap v250: 148 pgs: 148 active+clean; 26 MiB data, 1.6 GiB used, 714 GiB / 715 GiB avail; 776 B/s rd, 3.8 MiB/s wr, 61 op/s 2024-09-20T22:08:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:00 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2483250359' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:00.809 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:00.846 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:00.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:00.847 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:00.947 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:00.982 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:00.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:00.984 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:01.085 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:01.122 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:01.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:01.123 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:01.132 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 4 16 12041 12025 187.835 176.625 0.00515271 0.00531717 2024-09-20T22:08:01.224 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:01.271 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:01.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:01.272 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:01.373 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:01.418 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:01.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:01.419 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:01.520 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:01.562 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:01.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:01.563 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:01.664 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:01.709 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:01.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:01.710 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:01 smithi067 ceph-mon[33511]: 3.1c deep-scrub starts 2024-09-20T22:08:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:01 smithi067 ceph-mon[33511]: 3.1c deep-scrub ok 2024-09-20T22:08:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:01 smithi067 ceph-mon[33511]: 5.1d scrub starts 2024-09-20T22:08:01.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:01 smithi067 ceph-mon[33511]: 5.1d scrub ok 2024-09-20T22:08:01.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:01 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:08:01.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:01 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:08:01.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:01 smithi067 ceph-mon[33511]: 3.d scrub starts 2024-09-20T22:08:01.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:01 smithi067 ceph-mon[33511]: 3.d scrub ok 2024-09-20T22:08:01.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:01 smithi067 ceph-mon[33511]: 3.10 scrub starts 2024-09-20T22:08:01.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:01 smithi067 ceph-mon[33511]: 3.10 scrub ok 2024-09-20T22:08:01.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:01 smithi067 ceph-mon[33511]: osdmap e181: 8 total, 8 up, 8 in 2024-09-20T22:08:01.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[29694]: 3.1c deep-scrub starts 2024-09-20T22:08:01.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[29694]: 3.1c deep-scrub ok 2024-09-20T22:08:01.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[29694]: 5.1d scrub starts 2024-09-20T22:08:01.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[29694]: 5.1d scrub ok 2024-09-20T22:08:01.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:08:01.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:08:01.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[29694]: 3.d scrub starts 2024-09-20T22:08:01.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[29694]: 3.d scrub ok 2024-09-20T22:08:01.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[29694]: 3.10 scrub starts 2024-09-20T22:08:01.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[29694]: 3.10 scrub ok 2024-09-20T22:08:01.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[29694]: osdmap e181: 8 total, 8 up, 8 in 2024-09-20T22:08:01.766 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[40290]: 3.1c deep-scrub starts 2024-09-20T22:08:01.766 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[40290]: 3.1c deep-scrub ok 2024-09-20T22:08:01.766 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[40290]: 5.1d scrub starts 2024-09-20T22:08:01.766 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[40290]: 5.1d scrub ok 2024-09-20T22:08:01.767 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:08:01.767 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:08:01.767 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[40290]: 3.d scrub starts 2024-09-20T22:08:01.767 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[40290]: 3.d scrub ok 2024-09-20T22:08:01.767 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[40290]: 3.10 scrub starts 2024-09-20T22:08:01.767 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[40290]: 3.10 scrub ok 2024-09-20T22:08:01.767 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:01 smithi060 ceph-mon[40290]: osdmap e181: 8 total, 8 up, 8 in 2024-09-20T22:08:01.811 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:01.842 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:01.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:01.843 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:01.929 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:01.943 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:01.973 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:01.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:01.974 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:02.075 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:02.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:02.127 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:02.128 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:02.133 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 5 16 14485 14469 180.812 152.75 0.00466571 0.00552193 2024-09-20T22:08:02.228 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:02.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:08:01 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:08:02.267 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:08:02.267 INFO:teuthology.orchestra.run.smithi060.stdout:{"fsid":"3df9647e-779b-11ef-baf5-efdc52797490","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":533,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":181,"num_osds":8,"num_up_osds":8,"osd_up_since":1726870013,"num_in_osds":8,"osd_in_since":1726869741,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":148}],"num_pgs":148,"num_pools":7,"num_objects":600,"data_bytes":26876443,"bytes_used":1668112384,"bytes_avail":766292172800,"bytes_total":767960285184,"read_bytes_sec":776,"write_bytes_sec":3987487,"read_op_per_sec":0,"write_op_per_sec":60},"fsmap":{"epoch":1,"btime":"2024-09-20T21:57:31:244773+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"services":{"dashboard":"https://172.21.15.60:8443/","prometheus":"http://172.21.15.60:9283/"}},"servicemap":{"epoch":14,"modified":"2024-09-20T22:07:54.014274+0000","services":{"rgw":{"daemons":{"summary":"","14445":{"start_epoch":12,"start_stamp":"2024-09-20T22:02:57.389055+0000","gid":14445,"addr":"172.21.15.60:0/1594620389","metadata":{"arch":"x86_64","ceph_release":"squid","ceph_version":"ceph version 19.1.1-367-g9e2ac70b (9e2ac70bb7aea67d34f0d50002344abbe7803736) squid (stable)","ceph_version_short":"19.1.1-367-g9e2ac70b","container_hostname":"smithi060","container_image":"quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5","cpu":"Intel(R) Xeon(R) CPU E5-1620 v3 @ 3.50GHz","distro":"centos","distro_description":"CentOS Stream 9","distro_version":"9","frontend_config#0":"beast port=80","frontend_type#0":"beast","hostname":"smithi060","id":"foo.a","kernel_description":"#1 SMP PREEMPT_DYNAMIC Thu Sep 12 15:49:37 UTC 2024","kernel_version":"5.14.0-508.el9.x86_64","mem_swap_kb":"0","mem_total_kb":"32493484","num_handles":"1","os":"Linux","pid":"7","realm_id":"","realm_name":"","zone_id":"959e729f-3275-43b7-b0ce-fa01b06b5bb7","zone_name":"default","zonegroup_id":"9a789018-66fe-48e9-ab7f-a2c56aa55415","zonegroup_name":"default"},"task_status":{}}}}}},"progress_events":{}} 2024-09-20T22:08:02.269 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:02.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:02.271 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:02.372 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:02.404 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:02.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:02.405 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:02.478 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:08:02.506 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:02.535 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:02.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:02.536 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:02.637 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:02.667 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:02.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:02.668 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:02.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:02 smithi067 ceph-mon[33511]: 4.1b scrub starts 2024-09-20T22:08:02.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:02 smithi067 ceph-mon[33511]: 4.1b scrub ok 2024-09-20T22:08:02.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:02 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:08:02.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:02 smithi067 ceph-mon[33511]: pgmap v252: 148 pgs: 148 active+clean; 561 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-20T22:08:02.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:02 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/150133058' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:08:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:02 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:08:02.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:02 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/59065494' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-20T22:08:02.769 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:02.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:02 smithi060 ceph-mon[29694]: 4.1b scrub starts 2024-09-20T22:08:02.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:02 smithi060 ceph-mon[29694]: 4.1b scrub ok 2024-09-20T22:08:02.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:02 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:08:02.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:02 smithi060 ceph-mon[29694]: pgmap v252: 148 pgs: 148 active+clean; 561 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-20T22:08:02.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:02 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/150133058' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:08:02.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:02 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:08:02.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:02 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/59065494' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-20T22:08:02.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:02 smithi060 ceph-mon[40290]: 4.1b scrub starts 2024-09-20T22:08:02.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:02 smithi060 ceph-mon[40290]: 4.1b scrub ok 2024-09-20T22:08:02.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:02 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:08:02.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:02 smithi060 ceph-mon[40290]: pgmap v252: 148 pgs: 148 active+clean; 561 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 74 MiB/s wr, 1.18k op/s 2024-09-20T22:08:02.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:02 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/150133058' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:08:02.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:02 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:08:02.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:02 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/59065494' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-09-20T22:08:02.801 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:02.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:02.802 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:02.903 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:02.933 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:08:02.933 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:08:02.933 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:08:02.933 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:08:02.934 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:08:02.941 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:02.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:02.942 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:03.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:03.087 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:03.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:03.088 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:03.133 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 6 16 17049 17033 177.383 160.25 0.00413717 0.00562996 2024-09-20T22:08:03.189 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:03.239 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:03.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:03.240 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:03.244 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:08:03.341 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:03.372 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:03.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:03.373 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:03.474 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:03.518 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:03.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:03.519 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:03.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:03 smithi060 ceph-mon[29694]: 5.6 deep-scrub starts 2024-09-20T22:08:03.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:03 smithi060 ceph-mon[29694]: 5.6 deep-scrub ok 2024-09-20T22:08:03.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:03 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:08:03.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:03 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:08:03.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:03 smithi060 ceph-mon[29694]: osdmap e182: 8 total, 8 up, 8 in 2024-09-20T22:08:03.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:03 smithi060 ceph-mon[40290]: 5.6 deep-scrub starts 2024-09-20T22:08:03.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:03 smithi060 ceph-mon[40290]: 5.6 deep-scrub ok 2024-09-20T22:08:03.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:03 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:08:03.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:03 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:08:03.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:03 smithi060 ceph-mon[40290]: osdmap e182: 8 total, 8 up, 8 in 2024-09-20T22:08:03.621 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:03.658 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:03.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:03.659 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:03.760 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:03.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:03 smithi067 ceph-mon[33511]: 5.6 deep-scrub starts 2024-09-20T22:08:03.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:03 smithi067 ceph-mon[33511]: 5.6 deep-scrub ok 2024-09-20T22:08:03.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:03 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:08:03.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:03 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:08:03.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:03 smithi067 ceph-mon[33511]: osdmap e182: 8 total, 8 up, 8 in 2024-09-20T22:08:03.793 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:03.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:03.794 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:03.874 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T22:08:03.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:03.931 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:03.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:03.932 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:04.033 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:04.065 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:04.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:04.066 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:04.133 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 7 16 19495 19479 173.881 152.875 0.00462863 0.00574462 2024-09-20T22:08:04.166 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:04.210 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:04.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:04.211 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:04.312 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:04.351 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:04.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:04.352 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:04.453 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:04.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:04.486 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:04.486 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:04.587 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:04.610 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:04.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:04.611 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:04.712 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:04.747 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:04.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:04.748 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:04.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:04 smithi067 ceph-mon[33511]: pgmap v254: 148 pgs: 148 active+clean; 777 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 675 B/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-20T22:08:04.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:04 smithi060 ceph-mon[29694]: pgmap v254: 148 pgs: 148 active+clean; 777 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 675 B/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-20T22:08:04.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:04 smithi060 ceph-mon[40290]: pgmap v254: 148 pgs: 148 active+clean; 777 MiB data, 3.5 GiB used, 712 GiB / 715 GiB avail; 675 B/s rd, 102 MiB/s wr, 1.64k op/s 2024-09-20T22:08:04.849 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:04.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:04.886 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:04.887 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:04.987 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:05.019 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:05.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:05.020 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:05.121 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:05.133 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 8 16 21713 21697 169.473 138.625 0.00374233 0.00589384 2024-09-20T22:08:05.162 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:05.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:05.163 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:05.264 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:05.292 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:05.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:05.293 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:05.394 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:05.426 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:05.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:05.427 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:05.482 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:08:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:08:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:08:05.528 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:05.565 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:05.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:05.566 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:05.667 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:05.690 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:05.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:05.691 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:05.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:08:05.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:08:05.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:08:05.792 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:05.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:05.832 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:05.832 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:05.933 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:05.972 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:05.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:05.973 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:06.074 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:06.105 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:06.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:06.106 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:06.133 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 9 16 23972 23956 166.329 141.188 0.00758654 0.0060061 2024-09-20T22:08:06.207 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:06.229 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:06.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:06.230 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:06.330 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:06.371 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:06.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:06.372 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:06.472 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:06.505 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:06.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:06.507 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:06.607 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:06.640 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:06.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:06.641 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:06.742 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:06.776 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:06.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:06.777 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:06.878 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:06.908 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:06.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:06.910 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:07.010 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:07.047 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:07.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:07.048 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:07.133 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 10 16 26235 26219 163.839 141.438 0.0100039 0.006097 2024-09-20T22:08:07.148 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:07.195 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:07.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:07.196 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:07.297 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:07.359 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:07.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:07.360 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:07.367 INFO:tasks.cephadm.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-20T22:08:07.367 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:08:07.408 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:07.461 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:07.515 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:07.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:07.516 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:07.617 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:07.668 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:07.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:07.669 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:07.735 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:07 smithi060 ceph-mon[29694]: pgmap v255: 148 pgs: 148 active+clean; 1.0 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 130 MiB/s wr, 2.08k op/s 2024-09-20T22:08:07.735 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:07 smithi060 ceph-mon[29694]: osdmap e183: 8 total, 8 up, 8 in 2024-09-20T22:08:07.735 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:07 smithi060 ceph-mon[40290]: pgmap v255: 148 pgs: 148 active+clean; 1.0 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 130 MiB/s wr, 2.08k op/s 2024-09-20T22:08:07.735 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:07 smithi060 ceph-mon[40290]: osdmap e183: 8 total, 8 up, 8 in 2024-09-20T22:08:07.740 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:07.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:07 smithi067 ceph-mon[33511]: pgmap v255: 148 pgs: 148 active+clean; 1.0 GiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 1.2 KiB/s rd, 130 MiB/s wr, 2.08k op/s 2024-09-20T22:08:07.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:07 smithi067 ceph-mon[33511]: osdmap e183: 8 total, 8 up, 8 in 2024-09-20T22:08:07.770 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:07.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:07.799 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:07.799 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:07.900 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:07.944 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:07.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:07.945 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:08.046 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:08.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:08.097 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:08.097 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:08.133 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 11 16 28185 28169 160.023 121.875 0.00647493 0.00624153 2024-09-20T22:08:08.197 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:08.201 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:08:08.201 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:08:08.201 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:08:08.202 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:08:08.202 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.202 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227635 7.b 7.fcebd15b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:08:08.202 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.849145+0000", 2024-09-20T22:08:08.202 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 74.247813106999999, 2024-09-20T22:08:08.202 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.958058509, 2024-09-20T22:08:08.202 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.202 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.202 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:08:08.202 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:08.202 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:08.202 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:08.202 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227635 2024-09-20T22:08:08.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.203 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.203 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849145+0000", 2024-09-20T22:08:08.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.203 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849145+0000", 2024-09-20T22:08:08.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849146+0000", 2024-09-20T22:08:08.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.562e-06 2024-09-20T22:08:08.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849152+0000", 2024-09-20T22:08:08.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5999999999999997e-06 2024-09-20T22:08:08.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849162+0000", 2024-09-20T22:08:08.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9620000000000005e-06 2024-09-20T22:08:08.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849171+0000", 2024-09-20T22:08:08.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4569999999999994e-06 2024-09-20T22:08:08.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.205 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849336+0000", 2024-09-20T22:08:08.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016478700000000001 2024-09-20T22:08:08.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.205 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:08.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849344+0000", 2024-09-20T22:08:08.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3950000000000001e-06 2024-09-20T22:08:08.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.205 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.206 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.807192+0000", 2024-09-20T22:08:08.206 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.95784742599999995 2024-09-20T22:08:08.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.206 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.206 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.206 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.807203+0000", 2024-09-20T22:08:08.206 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.132e-05 2024-09-20T22:08:08.206 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.206 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.206 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.206 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.206 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227641 7.b 7.8d46094b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:08:08.207 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.858720+0000", 2024-09-20T22:08:08.207 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 74.238237619000003, 2024-09-20T22:08:08.207 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.95058797399999995, 2024-09-20T22:08:08.207 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.207 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.207 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:08:08.207 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:08.207 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:08.207 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:08.207 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227641 2024-09-20T22:08:08.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.207 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.207 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.207 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858720+0000", 2024-09-20T22:08:08.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.208 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.208 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858720+0000", 2024-09-20T22:08:08.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.208 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.208 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858722+0000", 2024-09-20T22:08:08.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3969999999999999e-06 2024-09-20T22:08:08.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.208 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858726+0000", 2024-09-20T22:08:08.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7999999999999998e-06 2024-09-20T22:08:08.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858737+0000", 2024-09-20T22:08:08.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0403e-05 2024-09-20T22:08:08.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858746+0000", 2024-09-20T22:08:08.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2739999999999996e-06 2024-09-20T22:08:08.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858806+0000", 2024-09-20T22:08:08.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0315000000000001e-05 2024-09-20T22:08:08.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.210 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:08.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858818+0000", 2024-09-20T22:08:08.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2079000000000001e-05 2024-09-20T22:08:08.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.210 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.809293+0000", 2024-09-20T22:08:08.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.95047507899999994 2024-09-20T22:08:08.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.211 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.211 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.809308+0000", 2024-09-20T22:08:08.211 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4627000000000001e-05 2024-09-20T22:08:08.211 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.211 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.211 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.211 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.211 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.211 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:227661 7.d e147/146 7:b23bd03b:::benchmark_data_smithi060_78800_object94628:head v 147'14015, mlcod=0'0)", 2024-09-20T22:08:08.211 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:55.977923+0000", 2024-09-20T22:08:08.211 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 72.119034686000006, 2024-09-20T22:08:08.211 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.110252976, 2024-09-20T22:08:08.211 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.211 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.212 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:08.212 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.977923+0000", 2024-09-20T22:08:08.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.977923+0000", 2024-09-20T22:08:08.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.977924+0000", 2024-09-20T22:08:08.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2099999999999997e-07 2024-09-20T22:08:08.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.213 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.977927+0000", 2024-09-20T22:08:08.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3689999999999998e-06 2024-09-20T22:08:08.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.213 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.977934+0000", 2024-09-20T22:08:08.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7190000000000004e-06 2024-09-20T22:08:08.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.977938+0000", 2024-09-20T22:08:08.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0289999999999999e-06 2024-09-20T22:08:08.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.214 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.977971+0000", 2024-09-20T22:08:08.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3662999999999999e-05 2024-09-20T22:08:08.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.214 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:08.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.977983+0000", 2024-09-20T22:08:08.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.133e-05 2024-09-20T22:08:08.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.214 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:08.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.086650+0000", 2024-09-20T22:08:08.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10866683100000001 2024-09-20T22:08:08.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.215 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.088176+0000", 2024-09-20T22:08:08.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0015265140000000001 2024-09-20T22:08:08.215 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.215 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.215 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.215 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:227641 7.b e147/146 7:d29062b1:::benchmark_data_smithi060_78800_object94608:head v 147'14143, mlcod=0'0)", 2024-09-20T22:08:08.215 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:55.978277+0000", 2024-09-20T22:08:08.215 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 72.118680620999996, 2024-09-20T22:08:08.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11623739600000001, 2024-09-20T22:08:08.216 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.216 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.216 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:08.216 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.216 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.216 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.978277+0000", 2024-09-20T22:08:08.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.216 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.216 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.978277+0000", 2024-09-20T22:08:08.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.978278+0000", 2024-09-20T22:08:08.217 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1100000000000001e-07 2024-09-20T22:08:08.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.978282+0000", 2024-09-20T22:08:08.217 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5800000000000002e-06 2024-09-20T22:08:08.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.978288+0000", 2024-09-20T22:08:08.217 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5940000000000001e-06 2024-09-20T22:08:08.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.218 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.978291+0000", 2024-09-20T22:08:08.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3419999999999999e-06 2024-09-20T22:08:08.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.218 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.978327+0000", 2024-09-20T22:08:08.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6396000000000002e-05 2024-09-20T22:08:08.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.218 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:08.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.978339+0000", 2024-09-20T22:08:08.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1744e-05 2024-09-20T22:08:08.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:08.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.094471+0000", 2024-09-20T22:08:08.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11613159300000001 2024-09-20T22:08:08.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.094515+0000", 2024-09-20T22:08:08.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3736000000000003e-05 2024-09-20T22:08:08.219 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.219 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.219 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.220 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.24667.0:227670 7.d e147/146 7:b1d0b44c:::benchmark_data_smithi060_78800_object94637:head v 147'14016, mlcod=0'0)", 2024-09-20T22:08:08.220 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:55.980859+0000", 2024-09-20T22:08:08.220 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 72.116098317999999, 2024-09-20T22:08:08.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.12541506799999999, 2024-09-20T22:08:08.220 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.220 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.220 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:08.220 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.220 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.220 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.220 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.980859+0000", 2024-09-20T22:08:08.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.220 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.220 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.221 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.980859+0000", 2024-09-20T22:08:08.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.221 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.221 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.980861+0000", 2024-09-20T22:08:08.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3230000000000001e-06 2024-09-20T22:08:08.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.221 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.221 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:55.980870+0000", 2024-09-20T22:08:08.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5270000000000004e-06 2024-09-20T22:08:08.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.086703+0000", 2024-09-20T22:08:08.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.105832975 2024-09-20T22:08:08.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.086713+0000", 2024-09-20T22:08:08.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2399999999999996e-06 2024-09-20T22:08:08.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.088206+0000", 2024-09-20T22:08:08.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001493643 2024-09-20T22:08:08.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:08.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.088226+0000", 2024-09-20T22:08:08.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9916e-05 2024-09-20T22:08:08.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:08.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.106006+0000", 2024-09-20T22:08:08.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.017780133 2024-09-20T22:08:08.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.106275+0000", 2024-09-20T22:08:08.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026831100000000003 2024-09-20T22:08:08.223 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.223 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.224 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.224 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "MOSDPGPush(7.d 147/146 [PushOp(7:b09456b1:::benchmark_data_smithi060_78800_object118582:head, version: 109'7308, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7:b09456b1:::benchmark_data_smithi060_78800_object118582:head@109'7308, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-20T22:08:08.224 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:56.714902+0000", 2024-09-20T22:08:08.224 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.382055539000007, 2024-09-20T22:08:08.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084510889000000006, 2024-09-20T22:08:08.224 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.224 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.224 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:08:08.224 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.714902+0000", 2024-09-20T22:08:08.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.225 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.225 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.225 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.714902+0000", 2024-09-20T22:08:08.225 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.225 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.225 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.225 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.714904+0000", 2024-09-20T22:08:08.225 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5200000000000001e-06 2024-09-20T22:08:08.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.225 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.225 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.225 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.714971+0000", 2024-09-20T22:08:08.225 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7549e-05 2024-09-20T22:08:08.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.714983+0000", 2024-09-20T22:08:08.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1491999999999999e-05 2024-09-20T22:08:08.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.714991+0000", 2024-09-20T22:08:08.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.473e-06 2024-09-20T22:08:08.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.715058+0000", 2024-09-20T22:08:08.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6359000000000006e-05 2024-09-20T22:08:08.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:08.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.799163+0000", 2024-09-20T22:08:08.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084105498000000001 2024-09-20T22:08:08.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.799413+0000", 2024-09-20T22:08:08.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00024999799999999997 2024-09-20T22:08:08.227 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.227 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.227 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.228 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "pg_backfill(progress 7.d e 147/147 lb 7:b0937ed8:::benchmark_data_smithi060_78800_object102515:head)", 2024-09-20T22:08:08.228 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:56.715005+0000", 2024-09-20T22:08:08.228 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.381953215999999, 2024-09-20T22:08:08.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084510486999999995, 2024-09-20T22:08:08.228 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.228 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.228 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:08:08.228 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.715005+0000", 2024-09-20T22:08:08.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.715005+0000", 2024-09-20T22:08:08.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.715005+0000", 2024-09-20T22:08:08.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.29e-07 2024-09-20T22:08:08.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.715009+0000", 2024-09-20T22:08:08.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1620000000000001e-06 2024-09-20T22:08:08.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.230 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.230 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.715013+0000", 2024-09-20T22:08:08.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1579999999999998e-06 2024-09-20T22:08:08.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.230 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.230 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.715016+0000", 2024-09-20T22:08:08.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9900000000000002e-06 2024-09-20T22:08:08.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.230 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.799459+0000", 2024-09-20T22:08:08.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.084442969000000007 2024-09-20T22:08:08.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:08.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.799474+0000", 2024-09-20T22:08:08.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5252999999999999e-05 2024-09-20T22:08:08.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.799515+0000", 2024-09-20T22:08:08.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0726000000000003e-05 2024-09-20T22:08:08.231 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.231 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.232 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.232 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.232 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "MOSDPGPush(7.d 147/146 [PushOp(7:b0d36332:::benchmark_data_smithi060_78800_object103958:head, version: 108'6425, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7:b0d36332:::benchmark_data_smithi060_78800_object103958:head@108'6425, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-20T22:08:08.232 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:57.095768+0000", 2024-09-20T22:08:08.232 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 71.001189284000006, 2024-09-20T22:08:08.232 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.066703399999999996, 2024-09-20T22:08:08.232 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.232 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.232 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:08:08.232 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.232 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.232 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.232 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:57.095768+0000", 2024-09-20T22:08:08.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.233 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.233 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:57.095768+0000", 2024-09-20T22:08:08.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.233 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.233 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:57.095769+0000", 2024-09-20T22:08:08.233 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6400000000000003e-07 2024-09-20T22:08:08.233 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.233 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.233 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.233 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:57.095823+0000", 2024-09-20T22:08:08.234 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3851999999999999e-05 2024-09-20T22:08:08.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.234 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.234 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:57.161859+0000", 2024-09-20T22:08:08.234 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.066035857000000003 2024-09-20T22:08:08.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.234 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.234 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:57.161887+0000", 2024-09-20T22:08:08.234 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8272e-05 2024-09-20T22:08:08.234 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.234 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:57.162126+0000", 2024-09-20T22:08:08.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023934499999999999 2024-09-20T22:08:08.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:08.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:57.162139+0000", 2024-09-20T22:08:08.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3132000000000001e-05 2024-09-20T22:08:08.235 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.235 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.235 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.235 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:57.162472+0000", 2024-09-20T22:08:08.235 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033247799999999998 2024-09-20T22:08:08.235 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.235 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.236 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.236 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.236 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "pg_backfill(progress 7.d e 147/147 lb 7:b23c2b50:::benchmark_data_smithi060_78800_object11618:head)", 2024-09-20T22:08:08.236 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:58.591621+0000", 2024-09-20T22:08:08.236 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 69.505337155000007, 2024-09-20T22:08:08.236 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.081050223000000005, 2024-09-20T22:08:08.236 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.236 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.236 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:08:08.236 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.236 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.236 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.236 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:58.591621+0000", 2024-09-20T22:08:08.237 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.237 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.237 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:58.591621+0000", 2024-09-20T22:08:08.237 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.237 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.237 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:58.591621+0000", 2024-09-20T22:08:08.237 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4999999999999998e-07 2024-09-20T22:08:08.237 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.237 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.237 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:58.591627+0000", 2024-09-20T22:08:08.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0090000000000004e-06 2024-09-20T22:08:08.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:58.591635+0000", 2024-09-20T22:08:08.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5080000000000005e-06 2024-09-20T22:08:08.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.238 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.238 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:58.591641+0000", 2024-09-20T22:08:08.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0109999999999997e-06 2024-09-20T22:08:08.238 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:58.591784+0000", 2024-09-20T22:08:08.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014293800000000001 2024-09-20T22:08:08.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:08.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:58.591792+0000", 2024-09-20T22:08:08.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1910000000000003e-06 2024-09-20T22:08:08.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.239 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.239 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:58.672671+0000", 2024-09-20T22:08:08.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.080879216000000004 2024-09-20T22:08:08.239 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.240 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.240 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.240 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "MOSDPGPush(7.d 147/146 [PushOp(7:b305d997:::benchmark_data_smithi060_78800_object130555:head, version: 111'8039, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7:b305d997:::benchmark_data_smithi060_78800_object130555:head@111'8039, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-20T22:08:08.240 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:59.401143+0000", 2024-09-20T22:08:08.240 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 68.695814587000001, 2024-09-20T22:08:08.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.097936297000000005, 2024-09-20T22:08:08.240 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.240 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.240 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:08:08.240 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:59.401143+0000", 2024-09-20T22:08:08.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:59.401143+0000", 2024-09-20T22:08:08.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:59.401144+0000", 2024-09-20T22:08:08.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2800000000000002e-07 2024-09-20T22:08:08.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:59.401205+0000", 2024-09-20T22:08:08.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1748999999999994e-05 2024-09-20T22:08:08.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.242 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.242 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:59.401216+0000", 2024-09-20T22:08:08.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0837e-05 2024-09-20T22:08:08.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.242 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.242 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:59.401224+0000", 2024-09-20T22:08:08.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1780000000000003e-06 2024-09-20T22:08:08.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.242 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.242 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:59.401273+0000", 2024-09-20T22:08:08.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9002000000000003e-05 2024-09-20T22:08:08.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:08.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:59.401280+0000", 2024-09-20T22:08:08.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6000000000000003e-06 2024-09-20T22:08:08.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:59.499079+0000", 2024-09-20T22:08:08.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.097799502999999996 2024-09-20T22:08:08.243 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.243 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.243 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.244 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "pg_backfill(progress 7.d e 147/147 lb 7:b3054005:::benchmark_data_smithi060_78800_object126816:head)", 2024-09-20T22:08:08.244 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:59.401237+0000", 2024-09-20T22:08:08.244 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 68.695720571999999, 2024-09-20T22:08:08.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.097924472999999998, 2024-09-20T22:08:08.244 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.244 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.244 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:08:08.244 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.244 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.244 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:59.401237+0000", 2024-09-20T22:08:08.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:59.401237+0000", 2024-09-20T22:08:08.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:59.401237+0000", 2024-09-20T22:08:08.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6e-07 2024-09-20T22:08:08.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:59.401241+0000", 2024-09-20T22:08:08.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.963e-06 2024-09-20T22:08:08.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.246 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:59.401246+0000", 2024-09-20T22:08:08.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.121e-06 2024-09-20T22:08:08.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.246 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.246 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:59.401249+0000", 2024-09-20T22:08:08.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9830000000000001e-06 2024-09-20T22:08:08.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.246 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.246 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:59.499116+0000", 2024-09-20T22:08:08.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.097867185999999995 2024-09-20T22:08:08.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.246 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:08.247 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:59.499130+0000", 2024-09-20T22:08:08.247 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4078e-05 2024-09-20T22:08:08.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.247 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.247 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:59.499162+0000", 2024-09-20T22:08:08.247 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1881999999999997e-05 2024-09-20T22:08:08.247 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.247 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.247 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.247 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "pg_backfill(progress 7.d e 147/147 lb 7:b3960d05:::benchmark_data_smithi060_78800_object29544:head)", 2024-09-20T22:08:08.248 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:00.095109+0000", 2024-09-20T22:08:08.248 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 68.001848635000002, 2024-09-20T22:08:08.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10902100000000001, 2024-09-20T22:08:08.248 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.248 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.248 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:08:08.248 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.248 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.248 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.248 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:00.095109+0000", 2024-09-20T22:08:08.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.248 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.248 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.248 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:00.095109+0000", 2024-09-20T22:08:08.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.249 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:00.095109+0000", 2024-09-20T22:08:08.249 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7700000000000001e-07 2024-09-20T22:08:08.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.249 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:00.095114+0000", 2024-09-20T22:08:08.249 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9320000000000004e-06 2024-09-20T22:08:08.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.249 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:00.095119+0000", 2024-09-20T22:08:08.249 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8319999999999996e-06 2024-09-20T22:08:08.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.250 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.250 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.250 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:00.095123+0000", 2024-09-20T22:08:08.250 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4470000000000002e-06 2024-09-20T22:08:08.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.250 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.250 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.250 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:00.095274+0000", 2024-09-20T22:08:08.250 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015136199999999999 2024-09-20T22:08:08.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.250 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.250 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:08.250 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:00.203803+0000", 2024-09-20T22:08:08.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10852873 2024-09-20T22:08:08.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.251 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.251 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.251 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:00.204130+0000", 2024-09-20T22:08:08.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00032741999999999998 2024-09-20T22:08:08.251 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.251 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.251 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.251 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.251 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "backfill_remove(7.9 e149 7:969f466a:::benchmark_data_smithi060_78800_object15494:head,87'928)", 2024-09-20T22:08:08.251 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:05.475491+0000", 2024-09-20T22:08:08.251 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.621466407, 2024-09-20T22:08:08.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.13473716799999999, 2024-09-20T22:08:08.252 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.252 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.252 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:08:08.252 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.252 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.252 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.252 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:05.475491+0000", 2024-09-20T22:08:08.252 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.252 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.252 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.252 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:05.475491+0000", 2024-09-20T22:08:08.252 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.253 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.253 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:05.475492+0000", 2024-09-20T22:08:08.253 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8800000000000002e-07 2024-09-20T22:08:08.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.253 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.253 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:05.475497+0000", 2024-09-20T22:08:08.253 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1399999999999999e-06 2024-09-20T22:08:08.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.254 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:05.610076+0000", 2024-09-20T22:08:08.254 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.134578221 2024-09-20T22:08:08.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.254 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.254 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:05.610091+0000", 2024-09-20T22:08:08.254 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5960999999999998e-05 2024-09-20T22:08:08.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.254 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.254 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:05.610154+0000", 2024-09-20T22:08:08.254 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2055000000000001e-05 2024-09-20T22:08:08.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:08.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:05.610170+0000", 2024-09-20T22:08:08.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6384000000000001e-05 2024-09-20T22:08:08.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:05.610229+0000", 2024-09-20T22:08:08.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8619000000000002e-05 2024-09-20T22:08:08.255 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.255 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.255 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.256 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "MOSDPGPush(7.d 149/146 [PushOp(7:b9319363:::benchmark_data_smithi060_78800_object113160:head, version: 109'6980, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(7:b9319363:::benchmark_data_smithi060_78800_object113160:head@109'6980, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-20T22:08:08.256 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:05.477037+0000", 2024-09-20T22:08:08.256 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 62.619920851000003, 2024-09-20T22:08:08.256 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.133814722, 2024-09-20T22:08:08.256 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.256 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.256 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:08:08.256 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.256 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.256 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.256 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:05.477037+0000", 2024-09-20T22:08:08.256 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.256 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.256 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.257 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:05.477037+0000", 2024-09-20T22:08:08.257 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.257 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.257 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.257 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:05.477038+0000", 2024-09-20T22:08:08.257 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2200000000000003e-07 2024-09-20T22:08:08.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.257 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.257 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.258 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:05.477083+0000", 2024-09-20T22:08:08.258 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4947000000000002e-05 2024-09-20T22:08:08.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.258 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.258 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.258 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:05.610089+0000", 2024-09-20T22:08:08.258 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.133005914 2024-09-20T22:08:08.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.258 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.258 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.258 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:05.610116+0000", 2024-09-20T22:08:08.258 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7421999999999999e-05 2024-09-20T22:08:08.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.258 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:05.610653+0000", 2024-09-20T22:08:08.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00053740700000000005 2024-09-20T22:08:08.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:08.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:05.610666+0000", 2024-09-20T22:08:08.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2798e-05 2024-09-20T22:08:08.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:05.610852+0000", 2024-09-20T22:08:08.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018541200000000001 2024-09-20T22:08:08.259 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.260 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.260 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.260 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:261258 7.b 7:dcf35b41:::benchmark_data_smithi060_78800_object128225:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e168)", 2024-09-20T22:08:08.260 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:35.440957+0000", 2024-09-20T22:08:08.260 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 32.656001013000001, 2024-09-20T22:08:08.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059117718, 2024-09-20T22:08:08.260 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.260 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.260 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:08.260 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:08.260 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:08.260 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:08.260 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 261258 2024-09-20T22:08:08.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.261 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.261 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.261 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.440957+0000", 2024-09-20T22:08:08.261 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.261 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.261 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.440957+0000", 2024-09-20T22:08:08.261 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.261 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.261 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.440959+0000", 2024-09-20T22:08:08.262 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.762e-06 2024-09-20T22:08:08.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.262 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.262 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.262 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.440965+0000", 2024-09-20T22:08:08.262 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3509999999999998e-06 2024-09-20T22:08:08.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.262 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.262 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.262 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.440985+0000", 2024-09-20T22:08:08.262 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0168000000000001e-05 2024-09-20T22:08:08.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.262 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.263 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.440994+0000", 2024-09-20T22:08:08.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9700000000000005e-06 2024-09-20T22:08:08.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.263 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.441296+0000", 2024-09-20T22:08:08.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030204800000000002 2024-09-20T22:08:08.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.263 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:08.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.441423+0000", 2024-09-20T22:08:08.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000126511 2024-09-20T22:08:08.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 5,6", 2024-09-20T22:08:08.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.441990+0000", 2024-09-20T22:08:08.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000567784 2024-09-20T22:08:08.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:08.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.448755+0000", 2024-09-20T22:08:08.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0067649449999999996 2024-09-20T22:08:08.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:08.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.449007+0000", 2024-09-20T22:08:08.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00025161999999999998 2024-09-20T22:08:08.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:08.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.499920+0000", 2024-09-20T22:08:08.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.050913236000000001 2024-09-20T22:08:08.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:08.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.499980+0000", 2024-09-20T22:08:08.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9401000000000003e-05 2024-09-20T22:08:08.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.500074+0000", 2024-09-20T22:08:08.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4921999999999997e-05 2024-09-20T22:08:08.265 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.265 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.266 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.266 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:261262 7.b 7:d97f69c9:::benchmark_data_smithi060_78800_object128229:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e168)", 2024-09-20T22:08:08.266 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:35.441029+0000", 2024-09-20T22:08:08.266 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 32.655928778000003, 2024-09-20T22:08:08.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.065113121999999996, 2024-09-20T22:08:08.266 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.266 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.266 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:08.266 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:08.266 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:08.266 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:08.266 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 261262 2024-09-20T22:08:08.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.267 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.441029+0000", 2024-09-20T22:08:08.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.441029+0000", 2024-09-20T22:08:08.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.441030+0000", 2024-09-20T22:08:08.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6400000000000002e-07 2024-09-20T22:08:08.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.441033+0000", 2024-09-20T22:08:08.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0419999999999998e-06 2024-09-20T22:08:08.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.441041+0000", 2024-09-20T22:08:08.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.242e-06 2024-09-20T22:08:08.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.441047+0000", 2024-09-20T22:08:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9499999999999998e-06 2024-09-20T22:08:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.446001+0000", 2024-09-20T22:08:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0049546690000000001 2024-09-20T22:08:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.446320+0000", 2024-09-20T22:08:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00031895600000000001 2024-09-20T22:08:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 5,6", 2024-09-20T22:08:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.446694+0000", 2024-09-20T22:08:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00037359600000000002 2024-09-20T22:08:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.449046+0000", 2024-09-20T22:08:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023517569999999999 2024-09-20T22:08:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.449173+0000", 2024-09-20T22:08:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012772200000000001 2024-09-20T22:08:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.506035+0000", 2024-09-20T22:08:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.056861314000000003 2024-09-20T22:08:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.506096+0000", 2024-09-20T22:08:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1496999999999998e-05 2024-09-20T22:08:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:35.506142+0000", 2024-09-20T22:08:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5813000000000003e-05 2024-09-20T22:08:08.271 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:20571 8.7 8:e4985f7d:::benchmark_data_smithi060_111051_object20570:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-20T22:08:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:08:04.552949+0000", 2024-09-20T22:08:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 3.5440083539999998, 2024-09-20T22:08:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.094103943999999995, 2024-09-20T22:08:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:08:08.272 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:08:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 20571 2024-09-20T22:08:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:04.552949+0000", 2024-09-20T22:08:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:04.552949+0000", 2024-09-20T22:08:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:04.552951+0000", 2024-09-20T22:08:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3400000000000001e-06 2024-09-20T22:08:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:04.552974+0000", 2024-09-20T22:08:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3518000000000001e-05 2024-09-20T22:08:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:04.552986+0000", 2024-09-20T22:08:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1803e-05 2024-09-20T22:08:08.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:04.552994+0000", 2024-09-20T22:08:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8089999999999993e-06 2024-09-20T22:08:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:04.553057+0000", 2024-09-20T22:08:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2939000000000002e-05 2024-09-20T22:08:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:08.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:04.553179+0000", 2024-09-20T22:08:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012232000000000001 2024-09-20T22:08:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:08:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:04.553330+0000", 2024-09-20T22:08:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015133900000000001 2024-09-20T22:08:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:04.554470+0000", 2024-09-20T22:08:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0011391929999999999 2024-09-20T22:08:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:08.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:04.554693+0000", 2024-09-20T22:08:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022330200000000001 2024-09-20T22:08:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:04.646959+0000", 2024-09-20T22:08:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.092265774999999994 2024-09-20T22:08:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:04.647010+0000", 2024-09-20T22:08:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1422999999999999e-05 2024-09-20T22:08:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:04.647053+0000", 2024-09-20T22:08:08.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3183000000000003e-05 2024-09-20T22:08:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:22942 8.1 8:8a3debf9:::benchmark_data_smithi060_111051_object22941:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-20T22:08:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:08:05.642784+0000", 2024-09-20T22:08:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 2.4541741049999999, 2024-09-20T22:08:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.071263980000000005, 2024-09-20T22:08:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:08.278 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:08:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:08:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22942 2024-09-20T22:08:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.642784+0000", 2024-09-20T22:08:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.642784+0000", 2024-09-20T22:08:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.642785+0000", 2024-09-20T22:08:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5099999999999998e-07 2024-09-20T22:08:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.642805+0000", 2024-09-20T22:08:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0812999999999999e-05 2024-09-20T22:08:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.642819+0000", 2024-09-20T22:08:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3798000000000001e-05 2024-09-20T22:08:08.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.642826+0000", 2024-09-20T22:08:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8920000000000003e-06 2024-09-20T22:08:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.643851+0000", 2024-09-20T22:08:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00102501 2024-09-20T22:08:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.643967+0000", 2024-09-20T22:08:08.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011567599999999999 2024-09-20T22:08:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:08:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.644263+0000", 2024-09-20T22:08:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029623300000000002 2024-09-20T22:08:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.645931+0000", 2024-09-20T22:08:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0016680919999999999 2024-09-20T22:08:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.648638+0000", 2024-09-20T22:08:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0027072139999999999 2024-09-20T22:08:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.713890+0000", 2024-09-20T22:08:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.065251348000000001 2024-09-20T22:08:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.713997+0000", 2024-09-20T22:08:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010758 2024-09-20T22:08:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.714048+0000", 2024-09-20T22:08:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0473000000000003e-05 2024-09-20T22:08:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:22960 8.1 8:8c82d049:::benchmark_data_smithi060_111051_object22959:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-20T22:08:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:08:05.650096+0000", 2024-09-20T22:08:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 2.4468620639999998, 2024-09-20T22:08:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.066399028999999998, 2024-09-20T22:08:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.284 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:08:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:08:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22960 2024-09-20T22:08:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.650096+0000", 2024-09-20T22:08:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.650096+0000", 2024-09-20T22:08:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.650097+0000", 2024-09-20T22:08:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4050000000000001e-06 2024-09-20T22:08:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.650119+0000", 2024-09-20T22:08:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2138999999999999e-05 2024-09-20T22:08:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.650131+0000", 2024-09-20T22:08:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1851999999999999e-05 2024-09-20T22:08:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.650138+0000", 2024-09-20T22:08:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0559999999999999e-06 2024-09-20T22:08:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.650194+0000", 2024-09-20T22:08:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6181999999999998e-05 2024-09-20T22:08:08.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.650315+0000", 2024-09-20T22:08:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012058 2024-09-20T22:08:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:08:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.650438+0000", 2024-09-20T22:08:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012332399999999999 2024-09-20T22:08:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.651569+0000", 2024-09-20T22:08:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0011306860000000001 2024-09-20T22:08:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.651959+0000", 2024-09-20T22:08:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000390355 2024-09-20T22:08:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.716394+0000", 2024-09-20T22:08:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.064434951000000004 2024-09-20T22:08:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:08.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.716449+0000", 2024-09-20T22:08:08.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5120000000000001e-05 2024-09-20T22:08:08.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.716495+0000", 2024-09-20T22:08:08.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5379e-05 2024-09-20T22:08:08.290 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.290 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.290 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:22969 8.1 8:8d1e003d:::benchmark_data_smithi060_111051_object22968:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e182)", 2024-09-20T22:08:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:08:05.652974+0000", 2024-09-20T22:08:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 2.4439836179999999, 2024-09-20T22:08:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.063651828999999993, 2024-09-20T22:08:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:08:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:08:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 22969 2024-09-20T22:08:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.291 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.652974+0000", 2024-09-20T22:08:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.652974+0000", 2024-09-20T22:08:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.652975+0000", 2024-09-20T22:08:08.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8400000000000004e-07 2024-09-20T22:08:08.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:08.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.652991+0000", 2024-09-20T22:08:08.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6518999999999999e-05 2024-09-20T22:08:08.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:08.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.688260+0000", 2024-09-20T22:08:08.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.035268233000000003 2024-09-20T22:08:08.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:08.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.688285+0000", 2024-09-20T22:08:08.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5452e-05 2024-09-20T22:08:08.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:08.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.688369+0000", 2024-09-20T22:08:08.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3857999999999996e-05 2024-09-20T22:08:08.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:08.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.688489+0000", 2024-09-20T22:08:08.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011974800000000001 2024-09-20T22:08:08.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:08:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.688630+0000", 2024-09-20T22:08:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014114499999999999 2024-09-20T22:08:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.690201+0000", 2024-09-20T22:08:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0015715429999999999 2024-09-20T22:08:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.690525+0000", 2024-09-20T22:08:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00032329 2024-09-20T22:08:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.716538+0000", 2024-09-20T22:08:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.026013418999999999 2024-09-20T22:08:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.716588+0000", 2024-09-20T22:08:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0133999999999998e-05 2024-09-20T22:08:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:05.716626+0000", 2024-09-20T22:08:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7803999999999999e-05 2024-09-20T22:08:08.296 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.297 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.297 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.297 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:08.297 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:08.297 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:08:08.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:08.298 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:08.298 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:08.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:08.442 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:08.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:08.443 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:08.544 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:08.585 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:08.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:08.586 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:08.687 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:08.718 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:08.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:08.719 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:08.820 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:08.853 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:08.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:08.855 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:08.955 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:08.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:08.990 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:08.990 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:09.090 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:09.125 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:09.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:09.127 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:09.133 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 12 16 30388 30372 158.161 137.688 0.00472101 0.00631682 2024-09-20T22:08:09.171 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:08:09.228 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:09.275 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:09.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:09.276 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:09.377 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:09.406 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:09.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:09.407 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:09.410 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T22:08:09.467 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:09 smithi060 ceph-mon[29694]: pgmap v257: 148 pgs: 148 active+clean; 1.5 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 771 B/s rd, 145 MiB/s wr, 2.32k op/s 2024-09-20T22:08:09.467 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:09 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:08:09.467 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:09 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:08:09.467 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:09 smithi060 ceph-mon[29694]: osdmap e184: 8 total, 8 up, 8 in 2024-09-20T22:08:09.467 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:09 smithi060 ceph-mon[40290]: pgmap v257: 148 pgs: 148 active+clean; 1.5 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 771 B/s rd, 145 MiB/s wr, 2.32k op/s 2024-09-20T22:08:09.468 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:09 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:08:09.468 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:09 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:08:09.468 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:09 smithi060 ceph-mon[40290]: osdmap e184: 8 total, 8 up, 8 in 2024-09-20T22:08:09.508 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:09.529 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:09.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:09.530 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:09.631 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:09.658 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:08:09.658 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:08:09.704 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:09.705 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:08:09.705 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:08:09.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:09.715 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:09.715 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:09.747 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:08:09.747 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:08:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:09 smithi067 ceph-mon[33511]: pgmap v257: 148 pgs: 148 active+clean; 1.5 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 771 B/s rd, 145 MiB/s wr, 2.32k op/s 2024-09-20T22:08:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:09 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:08:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:09 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:08:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:09 smithi067 ceph-mon[33511]: osdmap e184: 8 total, 8 up, 8 in 2024-09-20T22:08:09.784 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:08:09.785 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:08:09.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:09.821 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:08:09.821 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:08:09.850 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:09.859 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:08:09.859 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:08:09.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:09.869 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:09.869 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:09.900 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:08:09.900 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:08:09.935 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:08:09.935 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:08:09.938 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:09.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:09.994 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:09.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:09.995 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:10.096 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:10.134 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 13 16 31797 31781 152.769 88.0625 0.00830669 0.00654008 2024-09-20T22:08:10.137 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:10.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:10.138 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:10.239 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:10.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:10.278 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:10.278 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:10.379 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:10.417 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:10.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:10.419 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:10.520 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:10.551 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:10.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:10.552 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:10.652 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:10.694 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:10.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:10.695 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:10 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/732286643' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:10.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:10 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/732286643' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:10.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:10 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/732286643' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:10.795 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:10.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:10.826 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:10.826 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:10.927 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:10.970 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:10.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:10.971 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:11.072 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:11.111 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:11.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:11.112 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:11.133 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 14 16 33538 33522 149.628 108.812 0.00606606 0.00667748 2024-09-20T22:08:11.213 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:11.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:11.243 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:11.243 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:11.344 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:11.379 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:11.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:11.380 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:11.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:11.517 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:11.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:11.518 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:11.619 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:11.642 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:11.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:11.643 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:11.743 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:11.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:11 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:08:11.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:11 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:08:11.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:11 smithi067 ceph-mon[33511]: pgmap v259: 148 pgs: 148 active+clean; 1.5 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 678 B/s rd, 128 MiB/s wr, 2.04k op/s 2024-09-20T22:08:11.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:11 smithi067 ceph-mon[33511]: osdmap e185: 8 total, 8 up, 8 in 2024-09-20T22:08:11.779 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:11.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:11.780 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:11 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:08:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:11 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:08:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:11 smithi060 ceph-mon[29694]: pgmap v259: 148 pgs: 148 active+clean; 1.5 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 678 B/s rd, 128 MiB/s wr, 2.04k op/s 2024-09-20T22:08:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:11 smithi060 ceph-mon[29694]: osdmap e185: 8 total, 8 up, 8 in 2024-09-20T22:08:11.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:11 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:08:11.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:11 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:08:11.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:11 smithi060 ceph-mon[40290]: pgmap v259: 148 pgs: 148 active+clean; 1.5 GiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 678 B/s rd, 128 MiB/s wr, 2.04k op/s 2024-09-20T22:08:11.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:11 smithi060 ceph-mon[40290]: osdmap e185: 8 total, 8 up, 8 in 2024-09-20T22:08:11.881 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:11.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:11.915 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:11.915 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:12.016 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:12.050 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:12.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:12.051 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:12.134 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 15 16 35437 35421 147.556 118.688 0.0060654 0.00677126 2024-09-20T22:08:12.152 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:12.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:12.175 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:12.176 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:12.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:08:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:08:12.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:12.317 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:12.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:12.318 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:12.418 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:12.450 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:12.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:12.451 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:12.552 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:12.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:12.595 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:12.595 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:12.696 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:12.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:12.726 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:12.726 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:12.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:12 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:08:12.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:12 smithi067 ceph-mon[33511]: pgmap v261: 148 pgs: 148 active+clean; 1.9 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 152 MiB/s wr, 2.44k op/s 2024-09-20T22:08:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:12 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:08:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:12 smithi060 ceph-mon[29694]: pgmap v261: 148 pgs: 148 active+clean; 1.9 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 152 MiB/s wr, 2.44k op/s 2024-09-20T22:08:12.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:12 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:08:12.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:12 smithi060 ceph-mon[40290]: pgmap v261: 148 pgs: 148 active+clean; 1.9 GiB data, 7.3 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 152 MiB/s wr, 2.44k op/s 2024-09-20T22:08:12.828 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:12.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:12.857 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:12.857 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:12.958 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:12.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:12.998 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:12.998 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:13.099 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:13.121 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:13.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:13.122 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:13.135 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 16 16 36899 36883 144.045 91.375 0.00327794 0.0069158 2024-09-20T22:08:13.222 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:13.253 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:13.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:13.254 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:13.355 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:13.380 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:13.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:13.381 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:13.482 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:13.519 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:13.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:13.520 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:13.620 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:13.642 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:13.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:13.644 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:13.745 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:13.776 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:13.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:13.777 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:13.878 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:13.906 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:13.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:13.907 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:14.008 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:14.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:14.042 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:14.043 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:14.135 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 17 15 38538 38523 141.601 102.5 0.00700237 0.00705686 2024-09-20T22:08:14.143 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:14.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:14.176 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:14.176 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:14.277 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:14.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:14.312 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:14.312 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:14.413 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:14.446 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:14.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:14.448 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:14.549 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:14.586 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:14.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:14.587 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:14.688 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:14.717 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:14.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:14.718 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:14.819 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:14.858 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:14.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:14.859 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:14.960 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:14.996 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:14.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:14.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:15.098 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:15.135 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 18 16 40459 40443 140.4 120 0.0062583 0.00711605 2024-09-20T22:08:15.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:15.138 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:15.138 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:15.239 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:15.271 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:15.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:15.272 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:15.373 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:15.410 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:15.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:15.411 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:15.500 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:08:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:08:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:08:15.500 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:15 smithi060 ceph-mon[29694]: pgmap v262: 148 pgs: 148 active+clean; 2.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 670 B/s rd, 102 MiB/s wr, 1.63k op/s 2024-09-20T22:08:15.501 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:15 smithi060 ceph-mon[40290]: pgmap v262: 148 pgs: 148 active+clean; 2.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 670 B/s rd, 102 MiB/s wr, 1.63k op/s 2024-09-20T22:08:15.512 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:15.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:15 smithi067 ceph-mon[33511]: pgmap v262: 148 pgs: 148 active+clean; 2.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 670 B/s rd, 102 MiB/s wr, 1.63k op/s 2024-09-20T22:08:15.540 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:15.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:15.541 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:15.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:15.677 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:15.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:15.678 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:15.779 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:15.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:15.823 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:15.823 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:15.924 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:15.954 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:15.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:15.955 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:16.056 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:16.087 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:16.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:16.088 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:16.135 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 19 16 42327 42311 139.155 116.75 0.00442312 0.00718035 2024-09-20T22:08:16.188 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:16.215 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:16.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:16.216 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:16.292 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:08:16.292 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:08:16.317 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:16.356 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:16.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:16.357 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:16.420 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:08:16.422 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:08:16.458 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:16.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:16.504 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:16.505 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:16.605 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:16.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:16.644 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:16.644 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:16.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:16.782 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:16.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:16.783 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:16.884 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:16.928 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:16.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:16.930 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:17.030 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:17.067 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:17.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:17.068 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:17.135 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:08:17.134810+0000 min lat: 0.00126498 max lat: 0.154538 avg lat: 0.0072745 2024-09-20T22:08:17.135 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:08:17.135 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 20 16 43974 43958 137.344 102.938 0.012746 0.0072745 2024-09-20T22:08:17.169 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:17.199 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:17.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:17.200 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:17.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:17 smithi060 ceph-mon[29694]: pgmap v263: 148 pgs: 148 active+clean; 2.3 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 100 MiB/s wr, 1.61k op/s 2024-09-20T22:08:17.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:17 smithi060 ceph-mon[40290]: pgmap v263: 148 pgs: 148 active+clean; 2.3 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 100 MiB/s wr, 1.61k op/s 2024-09-20T22:08:17.300 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:17.330 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:17.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:17.331 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:17.432 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:17.470 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:17.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:17.471 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:17.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:17 smithi067 ceph-mon[33511]: pgmap v263: 148 pgs: 148 active+clean; 2.3 GiB data, 8.6 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 100 MiB/s wr, 1.61k op/s 2024-09-20T22:08:17.555 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:08:17.573 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:17.602 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:17.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:17.603 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:17.704 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:17.745 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:17.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:17.746 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:17.847 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:17.889 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:17.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:17.891 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:17.991 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:18.034 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:18.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:18.036 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:18.098 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:08:18.135 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 21 16 45548 45532 135.488 98.375 0.00882208 0.00737576 2024-09-20T22:08:18.137 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:18.176 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:18.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:18.177 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:18.278 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:18.315 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:18.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:18.316 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:18.416 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:18.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:18.455 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:18.455 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:18.464 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:18 smithi060 ceph-mon[29694]: from='client.25097 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:08:18.464 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:18 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/493389926' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:08:18.464 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:18 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:08:18.465 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:18 smithi060 ceph-mon[40290]: from='client.25097 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:08:18.466 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:18 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/493389926' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:08:18.466 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:18 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:08:18.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:18 smithi067 ceph-mon[33511]: from='client.25097 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:08:18.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:18 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/493389926' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:08:18.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:18 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:08:18.523 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T22:08:18.557 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:18.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:18.614 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:18.614 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:18.715 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:18.750 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:18.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:18.751 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:18.852 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:18.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:18.904 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:18.904 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:19.005 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:19.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:19.033 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:19.034 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:19.134 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:19.135 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 22 16 47150 47134 133.88 100.125 0.0127636 0.00746442 2024-09-20T22:08:19.165 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:19.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:19.167 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:19.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:19.322 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:19.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:19.324 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:19.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:19 smithi060 ceph-mon[29694]: pgmap v264: 148 pgs: 148 active+clean; 2.6 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 1.0 KiB/s rd, 119 MiB/s wr, 1.90k op/s 2024-09-20T22:08:19.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:19 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:08:19.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:19 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:08:19.364 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:19 smithi060 ceph-mon[29694]: osdmap e186: 8 total, 8 up, 8 in 2024-09-20T22:08:19.364 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:19 smithi060 ceph-mon[40290]: pgmap v264: 148 pgs: 148 active+clean; 2.6 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 1.0 KiB/s rd, 119 MiB/s wr, 1.90k op/s 2024-09-20T22:08:19.364 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:19 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:08:19.364 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:19 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:08:19.364 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:19 smithi060 ceph-mon[40290]: osdmap e186: 8 total, 8 up, 8 in 2024-09-20T22:08:19.425 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:19.460 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:19.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:19.461 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:19.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:19 smithi067 ceph-mon[33511]: pgmap v264: 148 pgs: 148 active+clean; 2.6 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 1.0 KiB/s rd, 119 MiB/s wr, 1.90k op/s 2024-09-20T22:08:19.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:19 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:08:19.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:19 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:08:19.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:19 smithi067 ceph-mon[33511]: osdmap e186: 8 total, 8 up, 8 in 2024-09-20T22:08:19.562 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:19.600 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:19.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:19.601 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:19.702 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:19.715 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":263,"stamp":"2024-09-20T22:08:16.021642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2440370715,"num_objects":37427,"num_object_clones":0,"num_object_copies":112281,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37427,"num_whiteouts":0,"num_read":1241,"num_read_kb":1087,"num_write":37843,"num_write_kb":2385053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37841,"ondisk_log_size":37841,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":8979040,"kb_used_data":7288668,"kb_used_omap":15,"kb_used_meta":1690224,"kb_avail":740982176,"statfs":{"total":767960285184,"available":758765748224,"internally_reserved":0,"allocated":7463596032,"data_stored":7457319481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":1730790163},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,1,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":842137600,"num_objects":12850,"num_object_clones":0,"num_object_copies":38550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12850,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":12850,"num_write_kb":822400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002856"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":270,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.414568+0000","last_change":"2024-09-20T22:06:48.018571+0000","last_active":"2024-09-20T22:08:10.414568+0000","last_peered":"2024-09-20T22:08:10.414568+0000","last_clean":"2024-09-20T22:08:10.414568+0000","last_became_active":"2024-09-20T22:02:50.109930+0000","last_became_peered":"2024-09-20T22:02:50.109930+0000","last_unstale":"2024-09-20T22:08:10.414568+0000","last_undegraded":"2024-09-20T22:08:10.414568+0000","last_fullsized":"2024-09-20T22:08:10.414568+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_clean_scrub_stamp":"2024-09-20T22:06:48.018157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"71'9","reported_seq":292,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407342+0000","last_change":"2024-09-20T22:07:55.492953+0000","last_active":"2024-09-20T22:08:10.407342+0000","last_peered":"2024-09-20T22:08:10.407342+0000","last_clean":"2024-09-20T22:08:10.407342+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:08:10.407342+0000","last_undegraded":"2024-09-20T22:08:10.407342+0000","last_fullsized":"2024-09-20T22:08:10.407342+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:55.492841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.492841+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:24.417494+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":151,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.416643+0000","last_change":"2024-09-20T22:06:54.843328+0000","last_active":"2024-09-20T22:08:10.416643+0000","last_peered":"2024-09-20T22:08:10.416643+0000","last_clean":"2024-09-20T22:08:10.416643+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:08:10.416643+0000","last_undegraded":"2024-09-20T22:08:10.416643+0000","last_fullsized":"2024-09-20T22:08:10.416643+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:11.178591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:11.178591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:11.268504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":260,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.408011+0000","last_change":"2024-09-20T22:07:14.535725+0000","last_active":"2024-09-20T22:08:10.408011+0000","last_peered":"2024-09-20T22:08:10.408011+0000","last_clean":"2024-09-20T22:08:10.408011+0000","last_became_active":"2024-09-20T22:06:54.830769+0000","last_became_peered":"2024-09-20T22:06:54.830769+0000","last_unstale":"2024-09-20T22:08:10.408011+0000","last_undegraded":"2024-09-20T22:08:10.408011+0000","last_fullsized":"2024-09-20T22:08:10.408011+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.535618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.535618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:36.228377+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":251,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420399+0000","last_change":"2024-09-20T22:05:06.024320+0000","last_active":"2024-09-20T22:08:10.420399+0000","last_peered":"2024-09-20T22:08:10.420399+0000","last_clean":"2024-09-20T22:08:10.420399+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:08:10.420399+0000","last_undegraded":"2024-09-20T22:08:10.420399+0000","last_fullsized":"2024-09-20T22:08:10.420399+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:05:06.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:06.024051+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":262,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.421192+0000","last_change":"2024-09-20T22:05:04.976359+0000","last_active":"2024-09-20T22:08:10.421192+0000","last_peered":"2024-09-20T22:08:10.421192+0000","last_clean":"2024-09-20T22:08:10.421192+0000","last_became_active":"2024-09-20T22:02:50.107154+0000","last_became_peered":"2024-09-20T22:02:50.107154+0000","last_unstale":"2024-09-20T22:08:10.421192+0000","last_undegraded":"2024-09-20T22:08:10.421192+0000","last_fullsized":"2024-09-20T22:08:10.421192+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:04.976218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:04.976218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"71'15","reported_seq":283,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420152+0000","last_change":"2024-09-20T22:04:15.767473+0000","last_active":"2024-09-20T22:08:10.420152+0000","last_peered":"2024-09-20T22:08:10.420152+0000","last_clean":"2024-09-20T22:08:10.420152+0000","last_became_active":"2024-09-20T22:02:52.282101+0000","last_became_peered":"2024-09-20T22:02:52.282101+0000","last_unstale":"2024-09-20T22:08:10.420152+0000","last_undegraded":"2024-09-20T22:08:10.420152+0000","last_fullsized":"2024-09-20T22:08:10.420152+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:04:15.767389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:15.767389+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":264,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407332+0000","last_change":"2024-09-20T22:07:17.494965+0000","last_active":"2024-09-20T22:08:10.407332+0000","last_peered":"2024-09-20T22:08:10.407332+0000","last_clean":"2024-09-20T22:08:10.407332+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:08:10.407332+0000","last_undegraded":"2024-09-20T22:08:10.407332+0000","last_fullsized":"2024-09-20T22:08:10.407332+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:17.494852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:17.494852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:40.846112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":278,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.414097+0000","last_change":"2024-09-20T22:06:46.616824+0000","last_active":"2024-09-20T22:08:10.414097+0000","last_peered":"2024-09-20T22:08:10.414097+0000","last_clean":"2024-09-20T22:08:10.414097+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:08:10.414097+0000","last_undegraded":"2024-09-20T22:08:10.414097+0000","last_fullsized":"2024-09-20T22:08:10.414097+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:46.616725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:46.616725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:13.947589+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"71'9","reported_seq":283,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.408542+0000","last_change":"2024-09-20T22:06:09.729638+0000","last_active":"2024-09-20T22:08:10.408542+0000","last_peered":"2024-09-20T22:08:10.408542+0000","last_clean":"2024-09-20T22:08:10.408542+0000","last_became_active":"2024-09-20T22:02:52.279526+0000","last_became_peered":"2024-09-20T22:02:52.279526+0000","last_unstale":"2024-09-20T22:08:10.408542+0000","last_undegraded":"2024-09-20T22:08:10.408542+0000","last_fullsized":"2024-09-20T22:08:10.408542+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:06:09.729513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:09.729513+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:13.566535+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":270,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.413704+0000","last_change":"2024-09-20T22:07:14.890410+0000","last_active":"2024-09-20T22:08:10.413704+0000","last_peered":"2024-09-20T22:08:10.413704+0000","last_clean":"2024-09-20T22:08:10.413704+0000","last_became_active":"2024-09-20T22:02:54.292911+0000","last_became_peered":"2024-09-20T22:02:54.292911+0000","last_unstale":"2024-09-20T22:08:10.413704+0000","last_undegraded":"2024-09-20T22:08:10.413704+0000","last_fullsized":"2024-09-20T22:08:10.413704+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.890308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.890308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:16.602195+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":259,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.411468+0000","last_change":"2024-09-20T22:07:15.416312+0000","last_active":"2024-09-20T22:08:10.411468+0000","last_peered":"2024-09-20T22:08:10.411468+0000","last_clean":"2024-09-20T22:08:10.411468+0000","last_became_active":"2024-09-20T22:06:54.841095+0000","last_became_peered":"2024-09-20T22:06:54.841095+0000","last_unstale":"2024-09-20T22:08:10.411468+0000","last_undegraded":"2024-09-20T22:08:10.411468+0000","last_fullsized":"2024-09-20T22:08:10.411468+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:15.416176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.416176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:19.458530+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":291,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.413941+0000","last_change":"2024-09-20T22:07:59.947914+0000","last_active":"2024-09-20T22:08:10.413941+0000","last_peered":"2024-09-20T22:08:10.413941+0000","last_clean":"2024-09-20T22:08:10.413941+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:08:10.413941+0000","last_undegraded":"2024-09-20T22:08:10.413941+0000","last_fullsized":"2024-09-20T22:08:10.413941+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:59.947741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:59.947741+0000","last_clean_scrub_stamp":"2024-09-20T22:07:59.947741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:29.533391+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"71'5","reported_seq":286,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407458+0000","last_change":"2024-09-20T22:08:01.462684+0000","last_active":"2024-09-20T22:08:10.407458+0000","last_peered":"2024-09-20T22:08:10.407458+0000","last_clean":"2024-09-20T22:08:10.407458+0000","last_became_active":"2024-09-20T22:06:54.830844+0000","last_became_peered":"2024-09-20T22:06:54.830844+0000","last_unstale":"2024-09-20T22:08:10.407458+0000","last_undegraded":"2024-09-20T22:08:10.407458+0000","last_fullsized":"2024-09-20T22:08:10.407458+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'5","last_scrub_stamp":"2024-09-20T22:08:01.462581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:01.462581+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:11.282642+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":272,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420373+0000","last_change":"2024-09-20T22:07:28.654301+0000","last_active":"2024-09-20T22:08:10.420373+0000","last_peered":"2024-09-20T22:08:10.420373+0000","last_clean":"2024-09-20T22:08:10.420373+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:08:10.420373+0000","last_undegraded":"2024-09-20T22:08:10.420373+0000","last_fullsized":"2024-09-20T22:08:10.420373+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:28.654202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:28.654202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:41.331246+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":204,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.411731+0000","last_change":"2024-09-20T22:06:55.824661+0000","last_active":"2024-09-20T22:08:10.411731+0000","last_peered":"2024-09-20T22:08:10.411731+0000","last_clean":"2024-09-20T22:08:10.411731+0000","last_became_active":"2024-09-20T22:06:55.824168+0000","last_became_peered":"2024-09-20T22:06:55.824168+0000","last_unstale":"2024-09-20T22:08:10.411731+0000","last_undegraded":"2024-09-20T22:08:10.411731+0000","last_fullsized":"2024-09-20T22:08:10.411731+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:05:29.551172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":259,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407850+0000","last_change":"2024-09-20T22:06:47.667293+0000","last_active":"2024-09-20T22:08:10.407850+0000","last_peered":"2024-09-20T22:08:10.407850+0000","last_clean":"2024-09-20T22:08:10.407850+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:08:10.407850+0000","last_undegraded":"2024-09-20T22:08:10.407850+0000","last_fullsized":"2024-09-20T22:08:10.407850+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:06:47.667180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:13.566556+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":262,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.412953+0000","last_change":"2024-09-20T22:05:31.552599+0000","last_active":"2024-09-20T22:08:10.412953+0000","last_peered":"2024-09-20T22:08:10.412953+0000","last_clean":"2024-09-20T22:08:10.412953+0000","last_became_active":"2024-09-20T22:02:50.111517+0000","last_became_peered":"2024-09-20T22:02:50.111517+0000","last_unstale":"2024-09-20T22:08:10.412953+0000","last_undegraded":"2024-09-20T22:08:10.412953+0000","last_fullsized":"2024-09-20T22:08:10.412953+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:31.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:31.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"71'15","reported_seq":300,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407890+0000","last_change":"2024-09-20T22:07:16.215766+0000","last_active":"2024-09-20T22:08:10.407890+0000","last_peered":"2024-09-20T22:08:10.407890+0000","last_clean":"2024-09-20T22:08:10.407890+0000","last_became_active":"2024-09-20T22:06:54.846111+0000","last_became_peered":"2024-09-20T22:06:54.846111+0000","last_unstale":"2024-09-20T22:08:10.407890+0000","last_undegraded":"2024-09-20T22:08:10.407890+0000","last_fullsized":"2024-09-20T22:08:10.407890+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:07:16.215638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:16.215638+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:18.878161+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":108,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.416030+0000","last_change":"2024-09-20T22:08:00.369883+0000","last_active":"2024-09-20T22:08:10.416030+0000","last_peered":"2024-09-20T22:08:10.416030+0000","last_clean":"2024-09-20T22:08:10.416030+0000","last_became_active":"2024-09-20T22:06:54.848505+0000","last_became_peered":"2024-09-20T22:06:54.848505+0000","last_unstale":"2024-09-20T22:08:10.416030+0000","last_undegraded":"2024-09-20T22:08:10.416030+0000","last_fullsized":"2024-09-20T22:08:10.416030+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:00.369767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:00.369767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:20.935739+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":258,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.421587+0000","last_change":"2024-09-20T22:06:34.944093+0000","last_active":"2024-09-20T22:08:10.421587+0000","last_peered":"2024-09-20T22:08:10.421587+0000","last_clean":"2024-09-20T22:08:10.421587+0000","last_became_active":"2024-09-20T22:02:56.347561+0000","last_became_peered":"2024-09-20T22:02:56.347561+0000","last_unstale":"2024-09-20T22:08:10.421587+0000","last_undegraded":"2024-09-20T22:08:10.421587+0000","last_fullsized":"2024-09-20T22:08:10.421587+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:34.940758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:34.940758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":212,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.414426+0000","last_change":"2024-09-20T22:06:55.817107+0000","last_active":"2024-09-20T22:08:10.414426+0000","last_peered":"2024-09-20T22:08:10.414426+0000","last_clean":"2024-09-20T22:08:10.414426+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:08:10.414426+0000","last_undegraded":"2024-09-20T22:08:10.414426+0000","last_fullsized":"2024-09-20T22:08:10.414426+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:32.472041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:32.472041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:13.947487+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"71'12","reported_seq":290,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420597+0000","last_change":"2024-09-20T22:07:55.383903+0000","last_active":"2024-09-20T22:08:10.420597+0000","last_peered":"2024-09-20T22:08:10.420597+0000","last_clean":"2024-09-20T22:08:10.420597+0000","last_became_active":"2024-09-20T22:06:54.830946+0000","last_became_peered":"2024-09-20T22:06:54.830946+0000","last_unstale":"2024-09-20T22:08:10.420597+0000","last_undegraded":"2024-09-20T22:08:10.420597+0000","last_fullsized":"2024-09-20T22:08:10.420597+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:55.383818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.383818+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:56.321505+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":255,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407538+0000","last_change":"2024-09-20T22:05:28.003302+0000","last_active":"2024-09-20T22:08:10.407538+0000","last_peered":"2024-09-20T22:08:10.407538+0000","last_clean":"2024-09-20T22:08:10.407538+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:08:10.407538+0000","last_undegraded":"2024-09-20T22:08:10.407538+0000","last_fullsized":"2024-09-20T22:08:10.407538+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.003183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:13.566281+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":253,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420214+0000","last_change":"2024-09-20T22:05:33.751965+0000","last_active":"2024-09-20T22:08:10.420214+0000","last_peered":"2024-09-20T22:08:10.420214+0000","last_clean":"2024-09-20T22:08:10.420214+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:08:10.420214+0000","last_undegraded":"2024-09-20T22:08:10.420214+0000","last_fullsized":"2024-09-20T22:08:10.420214+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:05:33.751883+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"64'1","reported_seq":102,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.416731+0000","last_change":"2024-09-20T22:06:54.854003+0000","last_active":"2024-09-20T22:08:10.416731+0000","last_peered":"2024-09-20T22:08:10.416731+0000","last_clean":"2024-09-20T22:08:10.416731+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:08:10.416731+0000","last_undegraded":"2024-09-20T22:08:10.416731+0000","last_fullsized":"2024-09-20T22:08:10.416731+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:11.268426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"71'10","reported_seq":273,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.413500+0000","last_change":"2024-09-20T22:04:56.414262+0000","last_active":"2024-09-20T22:08:10.413500+0000","last_peered":"2024-09-20T22:08:10.413500+0000","last_clean":"2024-09-20T22:08:10.413500+0000","last_became_active":"2024-09-20T22:02:52.481980+0000","last_became_peered":"2024-09-20T22:02:52.481980+0000","last_unstale":"2024-09-20T22:08:10.413500+0000","last_undegraded":"2024-09-20T22:08:10.413500+0000","last_fullsized":"2024-09-20T22:08:10.413500+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:04:56.414020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:56.414020+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"71'8","reported_seq":339,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.417644+0000","last_change":"2024-09-20T22:06:41.394065+0000","last_active":"2024-09-20T22:08:10.417644+0000","last_peered":"2024-09-20T22:08:10.417644+0000","last_clean":"2024-09-20T22:08:10.417644+0000","last_became_active":"2024-09-20T22:02:54.297657+0000","last_became_peered":"2024-09-20T22:02:54.297657+0000","last_unstale":"2024-09-20T22:08:10.417644+0000","last_undegraded":"2024-09-20T22:08:10.417644+0000","last_fullsized":"2024-09-20T22:08:10.417644+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:06:41.393961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:41.393961+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:17.260962+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"185'2290","reported_seq":2318,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:12.624322+0000","last_change":"2024-09-20T22:07:52.429150+0000","last_active":"2024-09-20T22:08:12.624322+0000","last_peered":"2024-09-20T22:08:12.624322+0000","last_clean":"2024-09-20T22:08:12.624322+0000","last_became_active":"2024-09-20T22:07:52.428105+0000","last_became_peered":"2024-09-20T22:07:52.428105+0000","last_unstale":"2024-09-20T22:08:12.624322+0000","last_undegraded":"2024-09-20T22:08:12.624322+0000","last_fullsized":"2024-09-20T22:08:12.624322+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":2290,"log_dups_size":0,"ondisk_log_size":2290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:04.623212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150077440,"num_objects":2290,"num_object_clones":0,"num_object_copies":6870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2290,"num_write_kb":146560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"71'15","reported_seq":181,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.417358+0000","last_change":"2024-09-20T22:06:54.860305+0000","last_active":"2024-09-20T22:08:10.417358+0000","last_peered":"2024-09-20T22:08:10.417358+0000","last_clean":"2024-09-20T22:08:10.417358+0000","last_became_active":"2024-09-20T22:06:54.859918+0000","last_became_peered":"2024-09-20T22:06:54.859918+0000","last_unstale":"2024-09-20T22:08:10.417358+0000","last_undegraded":"2024-09-20T22:08:10.417358+0000","last_fullsized":"2024-09-20T22:08:10.417358+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:04:13.235907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:13.235907+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:11.268474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":271,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.421234+0000","last_change":"2024-09-20T22:07:23.292560+0000","last_active":"2024-09-20T22:08:10.421234+0000","last_peered":"2024-09-20T22:08:10.421234+0000","last_clean":"2024-09-20T22:08:10.421234+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:08:10.421234+0000","last_undegraded":"2024-09-20T22:08:10.421234+0000","last_fullsized":"2024-09-20T22:08:10.421234+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.292451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.292451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:25.739658+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"71'8","reported_seq":340,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407996+0000","last_change":"2024-09-20T22:06:04.874166+0000","last_active":"2024-09-20T22:08:10.407996+0000","last_peered":"2024-09-20T22:08:10.407996+0000","last_clean":"2024-09-20T22:08:10.407996+0000","last_became_active":"2024-09-20T22:02:54.294917+0000","last_became_peered":"2024-09-20T22:02:54.294917+0000","last_unstale":"2024-09-20T22:08:10.407996+0000","last_undegraded":"2024-09-20T22:08:10.407996+0000","last_fullsized":"2024-09-20T22:08:10.407996+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:06:04.874042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:04.874042+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":220,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.417391+0000","last_change":"2024-09-20T22:07:52.334831+0000","last_active":"2024-09-20T22:08:10.417391+0000","last_peered":"2024-09-20T22:08:10.417391+0000","last_clean":"2024-09-20T22:08:10.417391+0000","last_became_active":"2024-09-20T22:06:55.822780+0000","last_became_peered":"2024-09-20T22:06:55.822780+0000","last_unstale":"2024-09-20T22:08:10.417391+0000","last_undegraded":"2024-09-20T22:08:10.417391+0000","last_fullsized":"2024-09-20T22:08:10.417391+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:52.334748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:52.334748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:53.481596+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.c","version":"185'2111","reported_seq":2139,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:11.226284+0000","last_change":"2024-09-20T22:07:52.674862+0000","last_active":"2024-09-20T22:08:11.226284+0000","last_peered":"2024-09-20T22:08:11.226284+0000","last_clean":"2024-09-20T22:08:11.226284+0000","last_became_active":"2024-09-20T22:07:52.674521+0000","last_became_peered":"2024-09-20T22:07:52.674521+0000","last_unstale":"2024-09-20T22:08:11.226284+0000","last_undegraded":"2024-09-20T22:08:11.226284+0000","last_fullsized":"2024-09-20T22:08:11.226284+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":2111,"log_dups_size":0,"ondisk_log_size":2111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:14.102427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138346496,"num_objects":2111,"num_object_clones":0,"num_object_copies":6333,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2111,"num_write_kb":135104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"71'18","reported_seq":290,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.419294+0000","last_change":"2024-09-20T22:04:57.967494+0000","last_active":"2024-09-20T22:08:10.419294+0000","last_peered":"2024-09-20T22:08:10.419294+0000","last_clean":"2024-09-20T22:08:10.419294+0000","last_became_active":"2024-09-20T22:02:52.482119+0000","last_became_peered":"2024-09-20T22:02:52.482119+0000","last_unstale":"2024-09-20T22:08:10.419294+0000","last_undegraded":"2024-09-20T22:08:10.419294+0000","last_fullsized":"2024-09-20T22:08:10.419294+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'18","last_scrub_stamp":"2024-09-20T22:04:57.967216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:57.967216+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":262,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.422043+0000","last_change":"2024-09-20T22:05:02.924002+0000","last_active":"2024-09-20T22:08:10.422043+0000","last_peered":"2024-09-20T22:08:10.422043+0000","last_clean":"2024-09-20T22:08:10.422043+0000","last_became_active":"2024-09-20T22:02:50.113076+0000","last_became_peered":"2024-09-20T22:02:50.113076+0000","last_unstale":"2024-09-20T22:08:10.422043+0000","last_undegraded":"2024-09-20T22:08:10.422043+0000","last_fullsized":"2024-09-20T22:08:10.422043+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:02.923565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:02.923565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":263,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407753+0000","last_change":"2024-09-20T22:06:05.894144+0000","last_active":"2024-09-20T22:08:10.407753+0000","last_peered":"2024-09-20T22:08:10.407753+0000","last_clean":"2024-09-20T22:08:10.407753+0000","last_became_active":"2024-09-20T22:02:54.288007+0000","last_became_peered":"2024-09-20T22:02:54.288007+0000","last_unstale":"2024-09-20T22:08:10.407753+0000","last_undegraded":"2024-09-20T22:08:10.407753+0000","last_fullsized":"2024-09-20T22:08:10.407753+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:05.894031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:05.894031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:13.566505+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":259,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.408454+0000","last_change":"2024-09-20T22:05:29.990030+0000","last_active":"2024-09-20T22:08:10.408454+0000","last_peered":"2024-09-20T22:08:10.408454+0000","last_clean":"2024-09-20T22:08:10.408454+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:08:10.408454+0000","last_undegraded":"2024-09-20T22:08:10.408454+0000","last_fullsized":"2024-09-20T22:08:10.408454+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:29.989934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:29.989934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:13.566313+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"185'2577","reported_seq":2605,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:15.627562+0000","last_change":"2024-09-20T22:07:52.940943+0000","last_active":"2024-09-20T22:08:15.627562+0000","last_peered":"2024-09-20T22:08:15.627562+0000","last_clean":"2024-09-20T22:08:15.627562+0000","last_became_active":"2024-09-20T22:07:52.940627+0000","last_became_peered":"2024-09-20T22:07:52.940627+0000","last_unstale":"2024-09-20T22:08:15.627562+0000","last_undegraded":"2024-09-20T22:08:15.627562+0000","last_fullsized":"2024-09-20T22:08:15.627562+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":2577,"log_dups_size":0,"ondisk_log_size":2577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:11.794238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168886272,"num_objects":2577,"num_object_clones":0,"num_object_copies":7731,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2577,"num_write_kb":164928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"71'14","reported_seq":288,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.408020+0000","last_change":"2024-09-20T22:06:00.944172+0000","last_active":"2024-09-20T22:08:10.408020+0000","last_peered":"2024-09-20T22:08:10.408020+0000","last_clean":"2024-09-20T22:08:10.408020+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:08:10.408020+0000","last_undegraded":"2024-09-20T22:08:10.408020+0000","last_fullsized":"2024-09-20T22:08:10.408020+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'14","last_scrub_stamp":"2024-09-20T22:06:00.944086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.944086+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:13.566438+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":272,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.414039+0000","last_change":"2024-09-20T22:07:58.783500+0000","last_active":"2024-09-20T22:08:10.414039+0000","last_peered":"2024-09-20T22:08:10.414039+0000","last_clean":"2024-09-20T22:08:10.414039+0000","last_became_active":"2024-09-20T22:06:54.841165+0000","last_became_peered":"2024-09-20T22:06:54.841165+0000","last_unstale":"2024-09-20T22:08:10.414039+0000","last_undegraded":"2024-09-20T22:08:10.414039+0000","last_fullsized":"2024-09-20T22:08:10.414039+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:07:58.783380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:58.783380+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:25.086988+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"146'10","reported_seq":338,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.419119+0000","last_change":"2024-09-20T22:07:24.333937+0000","last_active":"2024-09-20T22:08:10.419119+0000","last_peered":"2024-09-20T22:08:10.419119+0000","last_clean":"2024-09-20T22:08:10.419119+0000","last_became_active":"2024-09-20T22:06:54.847742+0000","last_became_peered":"2024-09-20T22:06:54.847742+0000","last_unstale":"2024-09-20T22:08:10.419119+0000","last_undegraded":"2024-09-20T22:08:10.419119+0000","last_fullsized":"2024-09-20T22:08:10.419119+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:07:24.333809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:24.333809+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:46.378931+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":259,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420541+0000","last_change":"2024-09-20T22:07:30.589937+0000","last_active":"2024-09-20T22:08:10.420541+0000","last_peered":"2024-09-20T22:08:10.420541+0000","last_clean":"2024-09-20T22:08:10.420541+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:08:10.420541+0000","last_undegraded":"2024-09-20T22:08:10.420541+0000","last_fullsized":"2024-09-20T22:08:10.420541+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:30.589849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:30.589849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:57.117488+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"185'2165","reported_seq":2193,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:11.221096+0000","last_change":"2024-09-20T22:07:52.940978+0000","last_active":"2024-09-20T22:08:11.221096+0000","last_peered":"2024-09-20T22:08:11.221096+0000","last_clean":"2024-09-20T22:08:11.221096+0000","last_became_active":"2024-09-20T22:07:52.940673+0000","last_became_peered":"2024-09-20T22:07:52.940673+0000","last_unstale":"2024-09-20T22:08:11.221096+0000","last_undegraded":"2024-09-20T22:08:11.221096+0000","last_fullsized":"2024-09-20T22:08:11.221096+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":2165,"log_dups_size":0,"ondisk_log_size":2165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.115401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":141885440,"num_objects":2165,"num_object_clones":0,"num_object_copies":6495,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2165,"num_write_kb":138560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"71'10","reported_seq":135,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.425865+0000","last_change":"2024-09-20T22:07:59.424108+0000","last_active":"2024-09-20T22:08:10.425865+0000","last_peered":"2024-09-20T22:08:10.425865+0000","last_clean":"2024-09-20T22:08:10.425865+0000","last_became_active":"2024-09-20T22:06:54.844681+0000","last_became_peered":"2024-09-20T22:06:54.844681+0000","last_unstale":"2024-09-20T22:08:10.425865+0000","last_undegraded":"2024-09-20T22:08:10.425865+0000","last_fullsized":"2024-09-20T22:08:10.425865+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:07:59.423970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:59.423970+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:05.043224+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":278,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.414218+0000","last_change":"2024-09-20T22:06:43.598035+0000","last_active":"2024-09-20T22:08:10.414218+0000","last_peered":"2024-09-20T22:08:10.414218+0000","last_clean":"2024-09-20T22:08:10.414218+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:08:10.414218+0000","last_undegraded":"2024-09-20T22:08:10.414218+0000","last_fullsized":"2024-09-20T22:08:10.414218+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:06:43.597924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:13.947512+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"71'8","reported_seq":327,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.412145+0000","last_change":"2024-09-20T22:04:20.160515+0000","last_active":"2024-09-20T22:08:10.412145+0000","last_peered":"2024-09-20T22:08:10.412145+0000","last_clean":"2024-09-20T22:08:10.412145+0000","last_became_active":"2024-09-20T22:02:54.292819+0000","last_became_peered":"2024-09-20T22:02:54.292819+0000","last_unstale":"2024-09-20T22:08:10.412145+0000","last_undegraded":"2024-09-20T22:08:10.412145+0000","last_fullsized":"2024-09-20T22:08:10.412145+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:04:20.160405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:20.160405+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":250,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.411603+0000","last_change":"2024-09-20T22:06:00.572347+0000","last_active":"2024-09-20T22:08:10.411603+0000","last_peered":"2024-09-20T22:08:10.411603+0000","last_clean":"2024-09-20T22:08:10.411603+0000","last_became_active":"2024-09-20T22:02:56.304274+0000","last_became_peered":"2024-09-20T22:02:56.304274+0000","last_unstale":"2024-09-20T22:08:10.411603+0000","last_undegraded":"2024-09-20T22:08:10.411603+0000","last_fullsized":"2024-09-20T22:08:10.411603+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:00.572243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.572243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"185'2151","reported_seq":2179,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:11.090751+0000","last_change":"2024-09-20T22:07:52.940027+0000","last_active":"2024-09-20T22:08:11.090751+0000","last_peered":"2024-09-20T22:08:11.090751+0000","last_clean":"2024-09-20T22:08:11.090751+0000","last_became_active":"2024-09-20T22:07:52.939692+0000","last_became_peered":"2024-09-20T22:07:52.939692+0000","last_unstale":"2024-09-20T22:08:11.090751+0000","last_undegraded":"2024-09-20T22:08:11.090751+0000","last_fullsized":"2024-09-20T22:08:11.090751+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":2151,"log_dups_size":0,"ondisk_log_size":2151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:12.848256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140967936,"num_objects":2151,"num_object_clones":0,"num_object_copies":6453,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2151,"num_write_kb":137664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"71'19","reported_seq":302,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.412338+0000","last_change":"2024-09-20T22:05:24.600933+0000","last_active":"2024-09-20T22:08:10.412338+0000","last_peered":"2024-09-20T22:08:10.412338+0000","last_clean":"2024-09-20T22:08:10.412338+0000","last_became_active":"2024-09-20T22:02:52.480077+0000","last_became_peered":"2024-09-20T22:02:52.480077+0000","last_unstale":"2024-09-20T22:08:10.412338+0000","last_undegraded":"2024-09-20T22:08:10.412338+0000","last_fullsized":"2024-09-20T22:08:10.412338+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:05:24.600795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:24.600795+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":109,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.416079+0000","last_change":"2024-09-20T22:07:14.382822+0000","last_active":"2024-09-20T22:08:10.416079+0000","last_peered":"2024-09-20T22:08:10.416079+0000","last_clean":"2024-09-20T22:08:10.416079+0000","last_became_active":"2024-09-20T22:06:54.829037+0000","last_became_peered":"2024-09-20T22:06:54.829037+0000","last_unstale":"2024-09-20T22:08:10.416079+0000","last_undegraded":"2024-09-20T22:08:10.416079+0000","last_fullsized":"2024-09-20T22:08:10.416079+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.382712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.382712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.271274+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":270,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.417072+0000","last_change":"2024-09-20T22:07:30.169138+0000","last_active":"2024-09-20T22:08:10.417072+0000","last_peered":"2024-09-20T22:08:10.417072+0000","last_clean":"2024-09-20T22:08:10.417072+0000","last_became_active":"2024-09-20T22:02:54.290787+0000","last_became_peered":"2024-09-20T22:02:54.290787+0000","last_unstale":"2024-09-20T22:08:10.417072+0000","last_undegraded":"2024-09-20T22:08:10.417072+0000","last_fullsized":"2024-09-20T22:08:10.417072+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:30.169038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:30.169038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:38.467894+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":100,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.415712+0000","last_change":"2024-09-20T22:06:54.820385+0000","last_active":"2024-09-20T22:08:10.415712+0000","last_peered":"2024-09-20T22:08:10.415712+0000","last_clean":"2024-09-20T22:08:10.415712+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:08:10.415712+0000","last_undegraded":"2024-09-20T22:08:10.415712+0000","last_fullsized":"2024-09-20T22:08:10.415712+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:11.268442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"185'2633","reported_seq":2661,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:15.623373+0000","last_change":"2024-09-20T22:07:52.674766+0000","last_active":"2024-09-20T22:08:15.623373+0000","last_peered":"2024-09-20T22:08:15.623373+0000","last_clean":"2024-09-20T22:08:15.623373+0000","last_became_active":"2024-09-20T22:07:52.674363+0000","last_became_peered":"2024-09-20T22:07:52.674363+0000","last_unstale":"2024-09-20T22:08:15.623373+0000","last_undegraded":"2024-09-20T22:08:15.623373+0000","last_fullsized":"2024-09-20T22:08:15.623373+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":2633,"log_dups_size":0,"ondisk_log_size":2633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:16.336670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":172556288,"num_objects":2633,"num_object_clones":0,"num_object_copies":7899,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2633,"num_write_kb":168512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"146'30","reported_seq":186,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.416206+0000","last_change":"2024-09-20T22:07:55.357829+0000","last_active":"2024-09-20T22:08:10.416206+0000","last_peered":"2024-09-20T22:08:10.416206+0000","last_clean":"2024-09-20T22:08:10.416206+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:08:10.416206+0000","last_undegraded":"2024-09-20T22:08:10.416206+0000","last_fullsized":"2024-09-20T22:08:10.416206+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'30","last_scrub_stamp":"2024-09-20T22:07:55.357726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.357726+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:10.172184+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":271,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407783+0000","last_change":"2024-09-20T22:05:58.966616+0000","last_active":"2024-09-20T22:08:10.407783+0000","last_peered":"2024-09-20T22:08:10.407783+0000","last_clean":"2024-09-20T22:08:10.407783+0000","last_became_active":"2024-09-20T22:02:50.108457+0000","last_became_peered":"2024-09-20T22:02:50.108457+0000","last_unstale":"2024-09-20T22:08:10.407783+0000","last_undegraded":"2024-09-20T22:08:10.407783+0000","last_fullsized":"2024-09-20T22:08:10.407783+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:58.966541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:58.966541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:13.566399+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":282,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.414806+0000","last_change":"2024-09-20T22:07:21.886172+0000","last_active":"2024-09-20T22:08:10.414806+0000","last_peered":"2024-09-20T22:08:10.414806+0000","last_clean":"2024-09-20T22:08:10.414806+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:08:10.414806+0000","last_undegraded":"2024-09-20T22:08:10.414806+0000","last_fullsized":"2024-09-20T22:08:10.414806+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_clean_scrub_stamp":"2024-09-20T22:07:21.886038+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.052265+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":263,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.412426+0000","last_change":"2024-09-20T22:07:17.566774+0000","last_active":"2024-09-20T22:08:10.412426+0000","last_peered":"2024-09-20T22:08:10.412426+0000","last_clean":"2024-09-20T22:08:10.412426+0000","last_became_active":"2024-09-20T22:06:54.841419+0000","last_became_peered":"2024-09-20T22:06:54.841419+0000","last_unstale":"2024-09-20T22:08:10.412426+0000","last_undegraded":"2024-09-20T22:08:10.412426+0000","last_fullsized":"2024-09-20T22:08:10.412426+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:17.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:17.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:32.923210+0000","scrub_duration":120,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":250,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420920+0000","last_change":"2024-09-20T22:05:34.954357+0000","last_active":"2024-09-20T22:08:10.420920+0000","last_peered":"2024-09-20T22:08:10.420920+0000","last_clean":"2024-09-20T22:08:10.420920+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:08:10.420920+0000","last_undegraded":"2024-09-20T22:08:10.420920+0000","last_fullsized":"2024-09-20T22:08:10.420920+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:05:34.953944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"185'2037","reported_seq":2065,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:11.117350+0000","last_change":"2024-09-20T22:07:52.429893+0000","last_active":"2024-09-20T22:08:11.117350+0000","last_peered":"2024-09-20T22:08:11.117350+0000","last_clean":"2024-09-20T22:08:11.117350+0000","last_became_active":"2024-09-20T22:07:52.429554+0000","last_became_peered":"2024-09-20T22:07:52.429554+0000","last_unstale":"2024-09-20T22:08:11.117350+0000","last_undegraded":"2024-09-20T22:08:11.117350+0000","last_fullsized":"2024-09-20T22:08:11.117350+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":2037,"log_dups_size":0,"ondisk_log_size":2037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:20.427594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133496832,"num_objects":2037,"num_object_clones":0,"num_object_copies":6111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2037,"num_write_kb":130368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"71'13","reported_seq":139,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.417470+0000","last_change":"2024-09-20T22:06:54.851246+0000","last_active":"2024-09-20T22:08:10.417470+0000","last_peered":"2024-09-20T22:08:10.417470+0000","last_clean":"2024-09-20T22:08:10.417470+0000","last_became_active":"2024-09-20T22:06:54.850696+0000","last_became_peered":"2024-09-20T22:06:54.850696+0000","last_unstale":"2024-09-20T22:08:10.417470+0000","last_undegraded":"2024-09-20T22:08:10.417470+0000","last_fullsized":"2024-09-20T22:08:10.417470+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:11.268394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":109,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420105+0000","last_change":"2024-09-20T22:07:20.341524+0000","last_active":"2024-09-20T22:08:10.420105+0000","last_peered":"2024-09-20T22:08:10.420105+0000","last_clean":"2024-09-20T22:08:10.420105+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:08:10.420105+0000","last_undegraded":"2024-09-20T22:08:10.420105+0000","last_fullsized":"2024-09-20T22:08:10.420105+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:20.338388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:20.338388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:40.900548+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":288,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407219+0000","last_change":"2024-09-20T22:06:47.015379+0000","last_active":"2024-09-20T22:08:10.407219+0000","last_peered":"2024-09-20T22:08:10.407219+0000","last_clean":"2024-09-20T22:08:10.407219+0000","last_became_active":"2024-09-20T22:02:49.099508+0000","last_became_peered":"2024-09-20T22:02:49.099508+0000","last_unstale":"2024-09-20T22:08:10.407219+0000","last_undegraded":"2024-09-20T22:08:10.407219+0000","last_fullsized":"2024-09-20T22:08:10.407219+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:06:47.015270+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:34.967504+0000","last_clean_scrub_stamp":"2024-09-20T22:06:47.015270+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":270,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.408153+0000","last_change":"2024-09-20T22:08:02.168865+0000","last_active":"2024-09-20T22:08:10.408153+0000","last_peered":"2024-09-20T22:08:10.408153+0000","last_clean":"2024-09-20T22:08:10.408153+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:08:10.408153+0000","last_undegraded":"2024-09-20T22:08:10.408153+0000","last_fullsized":"2024-09-20T22:08:10.408153+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:08:02.167030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:15.048786+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":251,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.421337+0000","last_change":"2024-09-20T22:05:28.780920+0000","last_active":"2024-09-20T22:08:10.421337+0000","last_peered":"2024-09-20T22:08:10.421337+0000","last_clean":"2024-09-20T22:08:10.421337+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:08:10.421337+0000","last_undegraded":"2024-09-20T22:08:10.421337+0000","last_fullsized":"2024-09-20T22:08:10.421337+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.780758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.780758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"185'2241","reported_seq":2268,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:12.632162+0000","last_change":"2024-09-20T22:07:52.939933+0000","last_active":"2024-09-20T22:08:12.632162+0000","last_peered":"2024-09-20T22:08:12.632162+0000","last_clean":"2024-09-20T22:08:12.632162+0000","last_became_active":"2024-09-20T22:07:52.939626+0000","last_became_peered":"2024-09-20T22:07:52.939626+0000","last_unstale":"2024-09-20T22:08:12.632162+0000","last_undegraded":"2024-09-20T22:08:12.632162+0000","last_fullsized":"2024-09-20T22:08:12.632162+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":2241,"log_dups_size":0,"ondisk_log_size":2241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:10.985426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146866176,"num_objects":2241,"num_object_clones":0,"num_object_copies":6723,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2241,"num_write_kb":143424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"71'12","reported_seq":285,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.413641+0000","last_change":"2024-09-20T22:07:29.614630+0000","last_active":"2024-09-20T22:08:10.413641+0000","last_peered":"2024-09-20T22:08:10.413641+0000","last_clean":"2024-09-20T22:08:10.413641+0000","last_became_active":"2024-09-20T22:06:54.848670+0000","last_became_peered":"2024-09-20T22:06:54.848670+0000","last_unstale":"2024-09-20T22:08:10.413641+0000","last_undegraded":"2024-09-20T22:08:10.413641+0000","last_fullsized":"2024-09-20T22:08:10.413641+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:29.614551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:29.614551+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:50.884056+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":270,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.411902+0000","last_change":"2024-09-20T22:06:44.862452+0000","last_active":"2024-09-20T22:08:10.411902+0000","last_peered":"2024-09-20T22:08:10.411902+0000","last_clean":"2024-09-20T22:08:10.411902+0000","last_became_active":"2024-09-20T22:02:50.110448+0000","last_became_peered":"2024-09-20T22:02:50.110448+0000","last_unstale":"2024-09-20T22:08:10.411902+0000","last_undegraded":"2024-09-20T22:08:10.411902+0000","last_fullsized":"2024-09-20T22:08:10.411902+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:06:44.862338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":601,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:13.943313+0000","last_change":"2024-09-20T22:07:16.590582+0000","last_active":"2024-09-20T22:08:13.943313+0000","last_peered":"2024-09-20T22:08:13.943313+0000","last_clean":"2024-09-20T22:08:13.943313+0000","last_became_active":"2024-09-20T22:06:54.844454+0000","last_became_peered":"2024-09-20T22:06:54.844454+0000","last_unstale":"2024-09-20T22:08:13.943313+0000","last_undegraded":"2024-09-20T22:08:13.943313+0000","last_fullsized":"2024-09-20T22:08:13.943313+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:07:16.590475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:07:16.590475+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.092001+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":312,"num_read_kb":307,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":263,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.414408+0000","last_change":"2024-09-20T22:07:22.884981+0000","last_active":"2024-09-20T22:08:10.414408+0000","last_peered":"2024-09-20T22:08:10.414408+0000","last_clean":"2024-09-20T22:08:10.414408+0000","last_became_active":"2024-09-20T22:06:54.841596+0000","last_became_peered":"2024-09-20T22:06:54.841596+0000","last_unstale":"2024-09-20T22:08:10.414408+0000","last_undegraded":"2024-09-20T22:08:10.414408+0000","last_fullsized":"2024-09-20T22:08:10.414408+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:22.884907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.884907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:41.462917+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":100,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.415844+0000","last_change":"2024-09-20T22:06:54.853957+0000","last_active":"2024-09-20T22:08:10.415844+0000","last_peered":"2024-09-20T22:08:10.415844+0000","last_clean":"2024-09-20T22:08:10.415844+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:08:10.415844+0000","last_undegraded":"2024-09-20T22:08:10.415844+0000","last_fullsized":"2024-09-20T22:08:10.415844+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:11.268375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"185'2611","reported_seq":2639,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:15.706258+0000","last_change":"2024-09-20T22:07:52.430313+0000","last_active":"2024-09-20T22:08:15.706258+0000","last_peered":"2024-09-20T22:08:15.706258+0000","last_clean":"2024-09-20T22:08:15.706258+0000","last_became_active":"2024-09-20T22:07:52.430017+0000","last_became_peered":"2024-09-20T22:07:52.430017+0000","last_unstale":"2024-09-20T22:08:15.706258+0000","last_undegraded":"2024-09-20T22:08:15.706258+0000","last_fullsized":"2024-09-20T22:08:15.706258+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":2611,"log_dups_size":0,"ondisk_log_size":2611,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:04.804126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171114496,"num_objects":2611,"num_object_clones":0,"num_object_copies":7833,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2611,"num_write_kb":167104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"71'16","reported_seq":362,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.417635+0000","last_change":"2024-09-20T22:07:53.297530+0000","last_active":"2024-09-20T22:08:10.417635+0000","last_peered":"2024-09-20T22:08:10.417635+0000","last_clean":"2024-09-20T22:08:10.417635+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:08:10.417635+0000","last_undegraded":"2024-09-20T22:08:10.417635+0000","last_fullsized":"2024-09-20T22:08:10.417635+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'16","last_scrub_stamp":"2024-09-20T22:07:53.297442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.297442+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:20.417428+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":270,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.419900+0000","last_change":"2024-09-20T22:06:48.105501+0000","last_active":"2024-09-20T22:08:10.419900+0000","last_peered":"2024-09-20T22:08:10.419900+0000","last_clean":"2024-09-20T22:08:10.419900+0000","last_became_active":"2024-09-20T22:02:50.106011+0000","last_became_peered":"2024-09-20T22:02:50.106011+0000","last_unstale":"2024-09-20T22:08:10.419900+0000","last_undegraded":"2024-09-20T22:08:10.419900+0000","last_fullsized":"2024-09-20T22:08:10.419900+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:48.105403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:48.105403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:17.663438+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":526,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.421405+0000","last_change":"2024-09-20T22:07:53.674419+0000","last_active":"2024-09-20T22:08:10.421405+0000","last_peered":"2024-09-20T22:08:10.421405+0000","last_clean":"2024-09-20T22:08:10.421405+0000","last_became_active":"2024-09-20T22:02:45.068482+0000","last_became_peered":"2024-09-20T22:02:45.068482+0000","last_unstale":"2024-09-20T22:08:10.421405+0000","last_undegraded":"2024-09-20T22:08:10.421405+0000","last_fullsized":"2024-09-20T22:08:10.421405+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:07:53.673691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.673691+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.005158+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":263,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420450+0000","last_change":"2024-09-20T22:05:56.679105+0000","last_active":"2024-09-20T22:08:10.420450+0000","last_peered":"2024-09-20T22:08:10.420450+0000","last_clean":"2024-09-20T22:08:10.420450+0000","last_became_active":"2024-09-20T22:02:54.298021+0000","last_became_peered":"2024-09-20T22:02:54.298021+0000","last_unstale":"2024-09-20T22:08:10.420450+0000","last_undegraded":"2024-09-20T22:08:10.420450+0000","last_fullsized":"2024-09-20T22:08:10.420450+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:56.678980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:56.678980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":272,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.414278+0000","last_change":"2024-09-20T22:06:47.664985+0000","last_active":"2024-09-20T22:08:10.414278+0000","last_peered":"2024-09-20T22:08:10.414278+0000","last_clean":"2024-09-20T22:08:10.414278+0000","last_became_active":"2024-09-20T22:04:21.237725+0000","last_became_peered":"2024-09-20T22:04:21.237725+0000","last_unstale":"2024-09-20T22:08:10.414278+0000","last_undegraded":"2024-09-20T22:08:10.414278+0000","last_fullsized":"2024-09-20T22:08:10.414278+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:47.664877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:06:47.664877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:10.795971+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"185'2504","reported_seq":2532,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:15.597739+0000","last_change":"2024-09-20T22:07:52.429509+0000","last_active":"2024-09-20T22:08:15.597739+0000","last_peered":"2024-09-20T22:08:15.597739+0000","last_clean":"2024-09-20T22:08:15.597739+0000","last_became_active":"2024-09-20T22:07:52.429069+0000","last_became_peered":"2024-09-20T22:07:52.429069+0000","last_unstale":"2024-09-20T22:08:15.597739+0000","last_undegraded":"2024-09-20T22:08:15.597739+0000","last_fullsized":"2024-09-20T22:08:15.597739+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":2504,"log_dups_size":0,"ondisk_log_size":2504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.758907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":164102144,"num_objects":2504,"num_object_clones":0,"num_object_copies":7512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2504,"num_write_kb":160256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"71'11","reported_seq":287,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.408067+0000","last_change":"2024-09-20T22:07:13.511239+0000","last_active":"2024-09-20T22:08:10.408067+0000","last_peered":"2024-09-20T22:08:10.408067+0000","last_clean":"2024-09-20T22:08:10.408067+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:08:10.408067+0000","last_undegraded":"2024-09-20T22:08:10.408067+0000","last_fullsized":"2024-09-20T22:08:10.408067+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:13.511159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:13.511159+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:38.687052+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":263,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.408127+0000","last_change":"2024-09-20T22:05:26.006756+0000","last_active":"2024-09-20T22:08:10.408127+0000","last_peered":"2024-09-20T22:08:10.408127+0000","last_clean":"2024-09-20T22:08:10.408127+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:08:10.408127+0000","last_undegraded":"2024-09-20T22:08:10.408127+0000","last_fullsized":"2024-09-20T22:08:10.408127+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:26.006692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:26.006692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:13.566335+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":262,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.414143+0000","last_change":"2024-09-20T22:05:57.397917+0000","last_active":"2024-09-20T22:08:10.414143+0000","last_peered":"2024-09-20T22:08:10.414143+0000","last_clean":"2024-09-20T22:08:10.414143+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:08:10.414143+0000","last_undegraded":"2024-09-20T22:08:10.414143+0000","last_fullsized":"2024-09-20T22:08:10.414143+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:57.397807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:57.397807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:13.947459+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":258,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420847+0000","last_change":"2024-09-20T22:06:45.030549+0000","last_active":"2024-09-20T22:08:10.420847+0000","last_peered":"2024-09-20T22:08:10.420847+0000","last_clean":"2024-09-20T22:08:10.420847+0000","last_became_active":"2024-09-20T22:02:56.349376+0000","last_became_peered":"2024-09-20T22:02:56.349376+0000","last_unstale":"2024-09-20T22:08:10.420847+0000","last_undegraded":"2024-09-20T22:08:10.420847+0000","last_fullsized":"2024-09-20T22:08:10.420847+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.030289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.030289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"185'2243","reported_seq":2271,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:12.624932+0000","last_change":"2024-09-20T22:07:52.939890+0000","last_active":"2024-09-20T22:08:12.624932+0000","last_peered":"2024-09-20T22:08:12.624932+0000","last_clean":"2024-09-20T22:08:12.624932+0000","last_became_active":"2024-09-20T22:07:52.939569+0000","last_became_peered":"2024-09-20T22:07:52.939569+0000","last_unstale":"2024-09-20T22:08:12.624932+0000","last_undegraded":"2024-09-20T22:08:12.624932+0000","last_fullsized":"2024-09-20T22:08:12.624932+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":2243,"log_dups_size":0,"ondisk_log_size":2243,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:15.331372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146997248,"num_objects":2243,"num_object_clones":0,"num_object_copies":6729,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2243,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2243,"num_write_kb":143552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"71'17","reported_seq":304,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407826+0000","last_change":"2024-09-20T22:07:54.528854+0000","last_active":"2024-09-20T22:08:10.407826+0000","last_peered":"2024-09-20T22:08:10.407826+0000","last_clean":"2024-09-20T22:08:10.407826+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:08:10.407826+0000","last_undegraded":"2024-09-20T22:08:10.407826+0000","last_fullsized":"2024-09-20T22:08:10.407826+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'17","last_scrub_stamp":"2024-09-20T22:07:54.528759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:54.528759+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.170193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":271,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.417355+0000","last_change":"2024-09-20T22:07:21.124897+0000","last_active":"2024-09-20T22:08:10.417355+0000","last_peered":"2024-09-20T22:08:10.417355+0000","last_clean":"2024-09-20T22:08:10.417355+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:08:10.417355+0000","last_undegraded":"2024-09-20T22:08:10.417355+0000","last_fullsized":"2024-09-20T22:08:10.417355+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:21.124789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:21.124789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:29.000115+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":108,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.415971+0000","last_change":"2024-09-20T22:07:19.376008+0000","last_active":"2024-09-20T22:08:10.415971+0000","last_peered":"2024-09-20T22:08:10.415971+0000","last_clean":"2024-09-20T22:08:10.415971+0000","last_became_active":"2024-09-20T22:06:54.848275+0000","last_became_peered":"2024-09-20T22:06:54.848275+0000","last_unstale":"2024-09-20T22:08:10.415971+0000","last_undegraded":"2024-09-20T22:08:10.415971+0000","last_fullsized":"2024-09-20T22:08:10.415971+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:19.375867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.375867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:49.161984+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":266,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407677+0000","last_change":"2024-09-20T22:07:55.195229+0000","last_active":"2024-09-20T22:08:10.407677+0000","last_peered":"2024-09-20T22:08:10.407677+0000","last_clean":"2024-09-20T22:08:10.407677+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:08:10.407677+0000","last_undegraded":"2024-09-20T22:08:10.407677+0000","last_fullsized":"2024-09-20T22:08:10.407677+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:55.195131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.195131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:10.187674+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"185'2054","reported_seq":2082,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:11.122135+0000","last_change":"2024-09-20T22:07:52.429344+0000","last_active":"2024-09-20T22:08:11.122135+0000","last_peered":"2024-09-20T22:08:11.122135+0000","last_clean":"2024-09-20T22:08:11.122135+0000","last_became_active":"2024-09-20T22:07:52.429054+0000","last_became_peered":"2024-09-20T22:07:52.429054+0000","last_unstale":"2024-09-20T22:08:11.122135+0000","last_undegraded":"2024-09-20T22:08:11.122135+0000","last_fullsized":"2024-09-20T22:08:11.122135+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":2054,"log_dups_size":0,"ondisk_log_size":2054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:16.620658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134610944,"num_objects":2054,"num_object_clones":0,"num_object_copies":6162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2054,"num_write_kb":131456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"71'10","reported_seq":274,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407690+0000","last_change":"2024-09-20T22:05:27.048378+0000","last_active":"2024-09-20T22:08:10.407690+0000","last_peered":"2024-09-20T22:08:10.407690+0000","last_clean":"2024-09-20T22:08:10.407690+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:08:10.407690+0000","last_undegraded":"2024-09-20T22:08:10.407690+0000","last_fullsized":"2024-09-20T22:08:10.407690+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:05:27.048215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:27.048215+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:13.566361+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"64'1","reported_seq":276,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420762+0000","last_change":"2024-09-20T22:06:33.966005+0000","last_active":"2024-09-20T22:08:10.420762+0000","last_peered":"2024-09-20T22:08:10.420762+0000","last_clean":"2024-09-20T22:08:10.420762+0000","last_became_active":"2024-09-20T22:02:50.110446+0000","last_became_peered":"2024-09-20T22:02:50.110446+0000","last_unstale":"2024-09-20T22:08:10.420762+0000","last_undegraded":"2024-09-20T22:08:10.420762+0000","last_fullsized":"2024-09-20T22:08:10.420762+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:06:33.965866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:33.965866+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"139'9","reported_seq":353,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407483+0000","last_change":"2024-09-20T22:07:57.138071+0000","last_active":"2024-09-20T22:08:10.407483+0000","last_peered":"2024-09-20T22:08:10.407483+0000","last_clean":"2024-09-20T22:08:10.407483+0000","last_became_active":"2024-09-20T22:02:54.294458+0000","last_became_peered":"2024-09-20T22:02:54.294458+0000","last_unstale":"2024-09-20T22:08:10.407483+0000","last_undegraded":"2024-09-20T22:08:10.407483+0000","last_fullsized":"2024-09-20T22:08:10.407483+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'9","last_scrub_stamp":"2024-09-20T22:07:57.137943+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:06:40.910011+0000","last_clean_scrub_stamp":"2024-09-20T22:07:57.137943+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:23.015596+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":260,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407730+0000","last_change":"2024-09-20T22:07:23.482514+0000","last_active":"2024-09-20T22:08:10.407730+0000","last_peered":"2024-09-20T22:08:10.407730+0000","last_clean":"2024-09-20T22:08:10.407730+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:08:10.407730+0000","last_undegraded":"2024-09-20T22:08:10.407730+0000","last_fullsized":"2024-09-20T22:08:10.407730+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.482372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.886444+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"185'2234","reported_seq":2262,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:12.614195+0000","last_change":"2024-09-20T22:07:52.429071+0000","last_active":"2024-09-20T22:08:12.614195+0000","last_peered":"2024-09-20T22:08:12.614195+0000","last_clean":"2024-09-20T22:08:12.614195+0000","last_became_active":"2024-09-20T22:07:52.428789+0000","last_became_peered":"2024-09-20T22:07:52.428789+0000","last_unstale":"2024-09-20T22:08:12.614195+0000","last_undegraded":"2024-09-20T22:08:12.614195+0000","last_fullsized":"2024-09-20T22:08:12.614195+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":2234,"log_dups_size":0,"ondisk_log_size":2234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:01.547587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146407424,"num_objects":2234,"num_object_clones":0,"num_object_copies":6702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2234,"num_write_kb":142976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"71'9","reported_seq":291,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.412242+0000","last_change":"2024-09-20T22:07:59.740218+0000","last_active":"2024-09-20T22:08:10.412242+0000","last_peered":"2024-09-20T22:08:10.412242+0000","last_clean":"2024-09-20T22:08:10.412242+0000","last_became_active":"2024-09-20T22:06:54.847770+0000","last_became_peered":"2024-09-20T22:06:54.847770+0000","last_unstale":"2024-09-20T22:08:10.412242+0000","last_undegraded":"2024-09-20T22:08:10.412242+0000","last_fullsized":"2024-09-20T22:08:10.412242+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:59.740089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:59.740089+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:19.034130+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":276,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.414102+0000","last_change":"2024-09-20T22:06:36.582728+0000","last_active":"2024-09-20T22:08:10.414102+0000","last_peered":"2024-09-20T22:08:10.414102+0000","last_clean":"2024-09-20T22:08:10.414102+0000","last_became_active":"2024-09-20T22:02:50.112682+0000","last_became_peered":"2024-09-20T22:02:50.112682+0000","last_unstale":"2024-09-20T22:08:10.414102+0000","last_undegraded":"2024-09-20T22:08:10.414102+0000","last_fullsized":"2024-09-20T22:08:10.414102+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:06:36.582537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:36.582537+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:13.947541+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":270,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407157+0000","last_change":"2024-09-20T22:07:22.203673+0000","last_active":"2024-09-20T22:08:10.407157+0000","last_peered":"2024-09-20T22:08:10.407157+0000","last_clean":"2024-09-20T22:08:10.407157+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:08:10.407157+0000","last_undegraded":"2024-09-20T22:08:10.407157+0000","last_fullsized":"2024-09-20T22:08:10.407157+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:22.203576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.203576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.064543+0000","scrub_duration":49,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":260,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.411883+0000","last_change":"2024-09-20T22:06:55.824784+0000","last_active":"2024-09-20T22:08:10.411883+0000","last_peered":"2024-09-20T22:08:10.411883+0000","last_clean":"2024-09-20T22:08:10.411883+0000","last_became_active":"2024-09-20T22:06:55.824229+0000","last_became_peered":"2024-09-20T22:06:55.824229+0000","last_unstale":"2024-09-20T22:08:10.411883+0000","last_undegraded":"2024-09-20T22:08:10.411883+0000","last_fullsized":"2024-09-20T22:08:10.411883+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:17.171523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:17.171523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"185'2129","reported_seq":2157,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:11.203391+0000","last_change":"2024-09-20T22:07:52.674668+0000","last_active":"2024-09-20T22:08:11.203391+0000","last_peered":"2024-09-20T22:08:11.203391+0000","last_clean":"2024-09-20T22:08:11.203391+0000","last_became_active":"2024-09-20T22:07:52.674301+0000","last_became_peered":"2024-09-20T22:07:52.674301+0000","last_unstale":"2024-09-20T22:08:11.203391+0000","last_undegraded":"2024-09-20T22:08:11.203391+0000","last_fullsized":"2024-09-20T22:08:11.203391+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":2129,"log_dups_size":0,"ondisk_log_size":2129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:06.988552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":139526144,"num_objects":2129,"num_object_clones":0,"num_object_copies":6387,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2129,"num_write_kb":136256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"71'19","reported_seq":298,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.417248+0000","last_change":"2024-09-20T22:07:25.155015+0000","last_active":"2024-09-20T22:08:10.417248+0000","last_peered":"2024-09-20T22:08:10.417248+0000","last_clean":"2024-09-20T22:08:10.417248+0000","last_became_active":"2024-09-20T22:06:54.829170+0000","last_became_peered":"2024-09-20T22:06:54.829170+0000","last_unstale":"2024-09-20T22:08:10.417248+0000","last_undegraded":"2024-09-20T22:08:10.417248+0000","last_fullsized":"2024-09-20T22:08:10.417248+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:07:25.154919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:25.154919+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:43.954499+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":271,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.421683+0000","last_change":"2024-09-20T22:08:00.560208+0000","last_active":"2024-09-20T22:08:10.421683+0000","last_peered":"2024-09-20T22:08:10.421683+0000","last_clean":"2024-09-20T22:08:10.421683+0000","last_became_active":"2024-09-20T22:02:50.108712+0000","last_became_peered":"2024-09-20T22:02:50.108712+0000","last_unstale":"2024-09-20T22:08:10.421683+0000","last_undegraded":"2024-09-20T22:08:10.421683+0000","last_fullsized":"2024-09-20T22:08:10.421683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:00.560098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:00.560098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:09.631655+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":270,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.406804+0000","last_change":"2024-09-20T22:07:20.217991+0000","last_active":"2024-09-20T22:08:10.406804+0000","last_peered":"2024-09-20T22:08:10.406804+0000","last_clean":"2024-09-20T22:08:10.406804+0000","last_became_active":"2024-09-20T22:02:54.291134+0000","last_became_peered":"2024-09-20T22:02:54.291134+0000","last_unstale":"2024-09-20T22:08:10.406804+0000","last_undegraded":"2024-09-20T22:08:10.406804+0000","last_fullsized":"2024-09-20T22:08:10.406804+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:20.217761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:20.217761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:42.617706+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":259,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.421303+0000","last_change":"2024-09-20T22:07:29.645352+0000","last_active":"2024-09-20T22:08:10.421303+0000","last_peered":"2024-09-20T22:08:10.421303+0000","last_clean":"2024-09-20T22:08:10.421303+0000","last_became_active":"2024-09-20T22:02:56.307687+0000","last_became_peered":"2024-09-20T22:02:56.307687+0000","last_unstale":"2024-09-20T22:08:10.421303+0000","last_undegraded":"2024-09-20T22:08:10.421303+0000","last_fullsized":"2024-09-20T22:08:10.421303+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:29.645127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:29.645127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:36.059611+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"185'2644","reported_seq":2671,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:15.683662+0000","last_change":"2024-09-20T22:07:52.675309+0000","last_active":"2024-09-20T22:08:15.683662+0000","last_peered":"2024-09-20T22:08:15.683662+0000","last_clean":"2024-09-20T22:08:15.683662+0000","last_became_active":"2024-09-20T22:07:52.674921+0000","last_became_peered":"2024-09-20T22:07:52.674921+0000","last_unstale":"2024-09-20T22:08:15.683662+0000","last_undegraded":"2024-09-20T22:08:15.683662+0000","last_fullsized":"2024-09-20T22:08:15.683662+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":2644,"log_dups_size":0,"ondisk_log_size":2644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:17.687123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173277184,"num_objects":2644,"num_object_clones":0,"num_object_copies":7932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2644,"num_write_kb":169216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"71'12","reported_seq":291,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407860+0000","last_change":"2024-09-20T22:07:22.458421+0000","last_active":"2024-09-20T22:08:10.407860+0000","last_peered":"2024-09-20T22:08:10.407860+0000","last_clean":"2024-09-20T22:08:10.407860+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:08:10.407860+0000","last_undegraded":"2024-09-20T22:08:10.407860+0000","last_fullsized":"2024-09-20T22:08:10.407860+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:22.458279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.458279+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:33.412138+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":272,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.422370+0000","last_change":"2024-09-20T22:07:21.584606+0000","last_active":"2024-09-20T22:08:10.422370+0000","last_peered":"2024-09-20T22:08:10.422370+0000","last_clean":"2024-09-20T22:08:10.422370+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:08:10.422370+0000","last_undegraded":"2024-09-20T22:08:10.422370+0000","last_fullsized":"2024-09-20T22:08:10.422370+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:21.584499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:21.584499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:24.004976+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":271,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.406829+0000","last_change":"2024-09-20T22:07:15.260254+0000","last_active":"2024-09-20T22:08:10.406829+0000","last_peered":"2024-09-20T22:08:10.406829+0000","last_clean":"2024-09-20T22:08:10.406829+0000","last_became_active":"2024-09-20T22:06:54.843906+0000","last_became_peered":"2024-09-20T22:06:54.843906+0000","last_unstale":"2024-09-20T22:08:10.406829+0000","last_undegraded":"2024-09-20T22:08:10.406829+0000","last_fullsized":"2024-09-20T22:08:10.406829+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:15.260148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.260148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:20.976588+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":204,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.412787+0000","last_change":"2024-09-20T22:06:55.822136+0000","last_active":"2024-09-20T22:08:10.412787+0000","last_peered":"2024-09-20T22:08:10.412787+0000","last_clean":"2024-09-20T22:08:10.412787+0000","last_became_active":"2024-09-20T22:06:55.821647+0000","last_became_peered":"2024-09-20T22:06:55.821647+0000","last_unstale":"2024-09-20T22:08:10.412787+0000","last_undegraded":"2024-09-20T22:08:10.412787+0000","last_fullsized":"2024-09-20T22:08:10.412787+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.569045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.569045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"185'2604","reported_seq":2632,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:15.634076+0000","last_change":"2024-09-20T22:07:52.939846+0000","last_active":"2024-09-20T22:08:15.634076+0000","last_peered":"2024-09-20T22:08:15.634076+0000","last_clean":"2024-09-20T22:08:15.634076+0000","last_became_active":"2024-09-20T22:07:52.939563+0000","last_became_peered":"2024-09-20T22:07:52.939563+0000","last_unstale":"2024-09-20T22:08:15.634076+0000","last_undegraded":"2024-09-20T22:08:15.634076+0000","last_fullsized":"2024-09-20T22:08:15.634076+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":2604,"log_dups_size":0,"ondisk_log_size":2604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:55.690792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170655744,"num_objects":2604,"num_object_clones":0,"num_object_copies":7812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2604,"num_write_kb":166656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"71'15","reported_seq":291,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.413584+0000","last_change":"2024-09-20T22:06:42.563403+0000","last_active":"2024-09-20T22:08:10.413584+0000","last_peered":"2024-09-20T22:08:10.413584+0000","last_clean":"2024-09-20T22:08:10.413584+0000","last_became_active":"2024-09-20T22:02:52.482008+0000","last_became_peered":"2024-09-20T22:02:52.482008+0000","last_unstale":"2024-09-20T22:08:10.413584+0000","last_undegraded":"2024-09-20T22:08:10.413584+0000","last_fullsized":"2024-09-20T22:08:10.413584+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:06:42.563266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:42.563266+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:13.947617+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":285,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420781+0000","last_change":"2024-09-20T22:06:40.813723+0000","last_active":"2024-09-20T22:08:10.420781+0000","last_peered":"2024-09-20T22:08:10.420781+0000","last_clean":"2024-09-20T22:08:10.420781+0000","last_became_active":"2024-09-20T22:02:50.114819+0000","last_became_peered":"2024-09-20T22:02:50.114819+0000","last_unstale":"2024-09-20T22:08:10.420781+0000","last_undegraded":"2024-09-20T22:08:10.420781+0000","last_fullsized":"2024-09-20T22:08:10.420781+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:06:40.813590+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":329,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420101+0000","last_change":"2024-09-20T22:05:34.732761+0000","last_active":"2024-09-20T22:08:10.420101+0000","last_peered":"2024-09-20T22:08:10.420101+0000","last_clean":"2024-09-20T22:08:10.420101+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:08:10.420101+0000","last_undegraded":"2024-09-20T22:08:10.420101+0000","last_fullsized":"2024-09-20T22:08:10.420101+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:05:34.732644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:34.732644+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:15.767018+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":258,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.414503+0000","last_change":"2024-09-20T22:06:35.559426+0000","last_active":"2024-09-20T22:08:10.414503+0000","last_peered":"2024-09-20T22:08:10.414503+0000","last_clean":"2024-09-20T22:08:10.414503+0000","last_became_active":"2024-09-20T22:02:56.346990+0000","last_became_peered":"2024-09-20T22:02:56.346990+0000","last_unstale":"2024-09-20T22:08:10.414503+0000","last_undegraded":"2024-09-20T22:08:10.414503+0000","last_fullsized":"2024-09-20T22:08:10.414503+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:35.559306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:35.559306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:13.947565+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":278,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.416965+0000","last_change":"2024-09-20T22:08:01.261458+0000","last_active":"2024-09-20T22:08:10.416965+0000","last_peered":"2024-09-20T22:08:10.416965+0000","last_clean":"2024-09-20T22:08:10.416965+0000","last_became_active":"2024-09-20T22:02:50.111935+0000","last_became_peered":"2024-09-20T22:02:50.111935+0000","last_unstale":"2024-09-20T22:08:10.416965+0000","last_undegraded":"2024-09-20T22:08:10.416965+0000","last_fullsized":"2024-09-20T22:08:10.416965+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:01.261328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:01.261328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:06.169471+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"71'6","reported_seq":268,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.421826+0000","last_change":"2024-09-20T22:06:54.862697+0000","last_active":"2024-09-20T22:08:10.421826+0000","last_peered":"2024-09-20T22:08:10.421826+0000","last_clean":"2024-09-20T22:08:10.421826+0000","last_became_active":"2024-09-20T22:06:54.862227+0000","last_became_peered":"2024-09-20T22:06:54.862227+0000","last_unstale":"2024-09-20T22:08:10.421826+0000","last_undegraded":"2024-09-20T22:08:10.421826+0000","last_fullsized":"2024-09-20T22:08:10.421826+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":263,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.414031+0000","last_change":"2024-09-20T22:07:19.865906+0000","last_active":"2024-09-20T22:08:10.414031+0000","last_peered":"2024-09-20T22:08:10.414031+0000","last_clean":"2024-09-20T22:08:10.414031+0000","last_became_active":"2024-09-20T22:06:54.836200+0000","last_became_peered":"2024-09-20T22:06:54.836200+0000","last_unstale":"2024-09-20T22:08:10.414031+0000","last_undegraded":"2024-09-20T22:08:10.414031+0000","last_fullsized":"2024-09-20T22:08:10.414031+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:19.865809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.865809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:42.730889+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":251,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.421248+0000","last_change":"2024-09-20T22:06:39.783526+0000","last_active":"2024-09-20T22:08:10.421248+0000","last_peered":"2024-09-20T22:08:10.421248+0000","last_clean":"2024-09-20T22:08:10.421248+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:08:10.421248+0000","last_undegraded":"2024-09-20T22:08:10.421248+0000","last_fullsized":"2024-09-20T22:08:10.421248+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:39.783456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:39.783456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"71'9","reported_seq":274,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.413763+0000","last_change":"2024-09-20T22:04:52.254864+0000","last_active":"2024-09-20T22:08:10.413763+0000","last_peered":"2024-09-20T22:08:10.413763+0000","last_clean":"2024-09-20T22:08:10.413763+0000","last_became_active":"2024-09-20T22:02:52.481989+0000","last_became_peered":"2024-09-20T22:02:52.481989+0000","last_unstale":"2024-09-20T22:08:10.413763+0000","last_undegraded":"2024-09-20T22:08:10.413763+0000","last_fullsized":"2024-09-20T22:08:10.413763+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:52.254706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:52.254706+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":263,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420319+0000","last_change":"2024-09-20T22:05:31.764903+0000","last_active":"2024-09-20T22:08:10.420319+0000","last_peered":"2024-09-20T22:08:10.420319+0000","last_clean":"2024-09-20T22:08:10.420319+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:08:10.420319+0000","last_undegraded":"2024-09-20T22:08:10.420319+0000","last_fullsized":"2024-09-20T22:08:10.420319+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:31.764833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:31.764833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":255,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.421682+0000","last_change":"2024-09-20T22:06:54.847169+0000","last_active":"2024-09-20T22:08:10.421682+0000","last_peered":"2024-09-20T22:08:10.421682+0000","last_clean":"2024-09-20T22:08:10.421682+0000","last_became_active":"2024-09-20T22:06:54.845704+0000","last_became_peered":"2024-09-20T22:06:54.845704+0000","last_unstale":"2024-09-20T22:08:10.421682+0000","last_undegraded":"2024-09-20T22:08:10.421682+0000","last_fullsized":"2024-09-20T22:08:10.421682+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":258,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407406+0000","last_change":"2024-09-20T22:06:39.942296+0000","last_active":"2024-09-20T22:08:10.407406+0000","last_peered":"2024-09-20T22:08:10.407406+0000","last_clean":"2024-09-20T22:08:10.407406+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:08:10.407406+0000","last_undegraded":"2024-09-20T22:08:10.407406+0000","last_fullsized":"2024-09-20T22:08:10.407406+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:39.942186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:39.942186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"71'9","reported_seq":290,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.414025+0000","last_change":"2024-09-20T22:07:52.834986+0000","last_active":"2024-09-20T22:08:10.414025+0000","last_peered":"2024-09-20T22:08:10.414025+0000","last_clean":"2024-09-20T22:08:10.414025+0000","last_became_active":"2024-09-20T22:02:52.482141+0000","last_became_peered":"2024-09-20T22:02:52.482141+0000","last_unstale":"2024-09-20T22:08:10.414025+0000","last_undegraded":"2024-09-20T22:08:10.414025+0000","last_fullsized":"2024-09-20T22:08:10.414025+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:52.834898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:52.834898+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.590814+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":270,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.412193+0000","last_change":"2024-09-20T22:06:45.908734+0000","last_active":"2024-09-20T22:08:10.412193+0000","last_peered":"2024-09-20T22:08:10.412193+0000","last_clean":"2024-09-20T22:08:10.412193+0000","last_became_active":"2024-09-20T22:02:50.110075+0000","last_became_peered":"2024-09-20T22:02:50.110075+0000","last_unstale":"2024-09-20T22:08:10.412193+0000","last_undegraded":"2024-09-20T22:08:10.412193+0000","last_fullsized":"2024-09-20T22:08:10.412193+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.908616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.908616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"71'8","reported_seq":327,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.421210+0000","last_change":"2024-09-20T22:05:55.972684+0000","last_active":"2024-09-20T22:08:10.421210+0000","last_peered":"2024-09-20T22:08:10.421210+0000","last_clean":"2024-09-20T22:08:10.421210+0000","last_became_active":"2024-09-20T22:02:54.294381+0000","last_became_peered":"2024-09-20T22:02:54.294381+0000","last_unstale":"2024-09-20T22:08:10.421210+0000","last_undegraded":"2024-09-20T22:08:10.421210+0000","last_fullsized":"2024-09-20T22:08:10.421210+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:05:55.972565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:55.972565+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":251,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.408139+0000","last_change":"2024-09-20T22:05:00.897974+0000","last_active":"2024-09-20T22:08:10.408139+0000","last_peered":"2024-09-20T22:08:10.408139+0000","last_clean":"2024-09-20T22:08:10.408139+0000","last_became_active":"2024-09-20T22:02:56.300041+0000","last_became_peered":"2024-09-20T22:02:56.300041+0000","last_unstale":"2024-09-20T22:08:10.408139+0000","last_undegraded":"2024-09-20T22:08:10.408139+0000","last_fullsized":"2024-09-20T22:08:10.408139+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:00.897869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:00.897869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:13.566252+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"71'10","reported_seq":274,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.421320+0000","last_change":"2024-09-20T22:06:54.862610+0000","last_active":"2024-09-20T22:08:10.421320+0000","last_peered":"2024-09-20T22:08:10.421320+0000","last_clean":"2024-09-20T22:08:10.421320+0000","last_became_active":"2024-09-20T22:06:54.862289+0000","last_became_peered":"2024-09-20T22:06:54.862289+0000","last_unstale":"2024-09-20T22:08:10.421320+0000","last_undegraded":"2024-09-20T22:08:10.421320+0000","last_fullsized":"2024-09-20T22:08:10.421320+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":263,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420853+0000","last_change":"2024-09-20T22:06:35.687169+0000","last_active":"2024-09-20T22:08:10.420853+0000","last_peered":"2024-09-20T22:08:10.420853+0000","last_clean":"2024-09-20T22:08:10.420853+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:08:10.420853+0000","last_undegraded":"2024-09-20T22:08:10.420853+0000","last_fullsized":"2024-09-20T22:08:10.420853+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:35.687065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:35.687065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"146'10","reported_seq":339,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.413968+0000","last_change":"2024-09-20T22:07:15.870748+0000","last_active":"2024-09-20T22:08:10.413968+0000","last_peered":"2024-09-20T22:08:10.413968+0000","last_clean":"2024-09-20T22:08:10.413968+0000","last_became_active":"2024-09-20T22:06:54.843097+0000","last_became_peered":"2024-09-20T22:06:54.843097+0000","last_unstale":"2024-09-20T22:08:10.413968+0000","last_undegraded":"2024-09-20T22:08:10.413968+0000","last_fullsized":"2024-09-20T22:08:10.413968+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:07:15.870646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.870646+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:36.803368+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":250,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.412441+0000","last_change":"2024-09-20T22:05:35.601891+0000","last_active":"2024-09-20T22:08:10.412441+0000","last_peered":"2024-09-20T22:08:10.412441+0000","last_clean":"2024-09-20T22:08:10.412441+0000","last_became_active":"2024-09-20T22:02:56.305383+0000","last_became_peered":"2024-09-20T22:02:56.305383+0000","last_unstale":"2024-09-20T22:08:10.412441+0000","last_undegraded":"2024-09-20T22:08:10.412441+0000","last_fullsized":"2024-09-20T22:08:10.412441+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:35.601794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:35.601794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"71'11","reported_seq":287,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407464+0000","last_change":"2024-09-20T22:07:15.544790+0000","last_active":"2024-09-20T22:08:10.407464+0000","last_peered":"2024-09-20T22:08:10.407464+0000","last_clean":"2024-09-20T22:08:10.407464+0000","last_became_active":"2024-09-20T22:06:54.823678+0000","last_became_peered":"2024-09-20T22:06:54.823678+0000","last_unstale":"2024-09-20T22:08:10.407464+0000","last_undegraded":"2024-09-20T22:08:10.407464+0000","last_fullsized":"2024-09-20T22:08:10.407464+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:15.544705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.544705+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:32.779498+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":263,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.407986+0000","last_change":"2024-09-20T22:04:54.764407+0000","last_active":"2024-09-20T22:08:10.407986+0000","last_peered":"2024-09-20T22:08:10.407986+0000","last_clean":"2024-09-20T22:08:10.407986+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:08:10.407986+0000","last_undegraded":"2024-09-20T22:08:10.407986+0000","last_fullsized":"2024-09-20T22:08:10.407986+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:54.764137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:54.764137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:13.566218+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":100,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.425833+0000","last_change":"2024-09-20T22:06:54.853857+0000","last_active":"2024-09-20T22:08:10.425833+0000","last_peered":"2024-09-20T22:08:10.425833+0000","last_clean":"2024-09-20T22:08:10.425833+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:08:10.425833+0000","last_undegraded":"2024-09-20T22:08:10.425833+0000","last_fullsized":"2024-09-20T22:08:10.425833+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:11.268410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":87,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.415618+0000","last_change":"2024-09-20T22:06:55.835713+0000","last_active":"2024-09-20T22:08:10.415618+0000","last_peered":"2024-09-20T22:08:10.415618+0000","last_clean":"2024-09-20T22:08:10.415618+0000","last_became_active":"2024-09-20T22:06:55.835393+0000","last_became_peered":"2024-09-20T22:06:55.835393+0000","last_unstale":"2024-09-20T22:08:10.415618+0000","last_undegraded":"2024-09-20T22:08:10.415618+0000","last_fullsized":"2024-09-20T22:08:10.415618+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:33.623816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:33.623816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:11.268518+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"71'9","reported_seq":172,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.425929+0000","last_change":"2024-09-20T22:06:54.844349+0000","last_active":"2024-09-20T22:08:10.425929+0000","last_peered":"2024-09-20T22:08:10.425929+0000","last_clean":"2024-09-20T22:08:10.425929+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:08:10.425929+0000","last_undegraded":"2024-09-20T22:08:10.425929+0000","last_fullsized":"2024-09-20T22:08:10.425929+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:12.196518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:12.196518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:11.268458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":279,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.421548+0000","last_change":"2024-09-20T22:07:54.685916+0000","last_active":"2024-09-20T22:08:10.421548+0000","last_peered":"2024-09-20T22:08:10.421548+0000","last_clean":"2024-09-20T22:08:10.421548+0000","last_became_active":"2024-09-20T22:02:50.109803+0000","last_became_peered":"2024-09-20T22:02:50.109803+0000","last_unstale":"2024-09-20T22:08:10.421548+0000","last_undegraded":"2024-09-20T22:08:10.421548+0000","last_fullsized":"2024-09-20T22:08:10.421548+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:54.685751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:54.685751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:04.385036+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":255,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420502+0000","last_change":"2024-09-20T22:06:54.844570+0000","last_active":"2024-09-20T22:08:10.420502+0000","last_peered":"2024-09-20T22:08:10.420502+0000","last_clean":"2024-09-20T22:08:10.420502+0000","last_became_active":"2024-09-20T22:06:54.844271+0000","last_became_peered":"2024-09-20T22:06:54.844271+0000","last_unstale":"2024-09-20T22:08:10.420502+0000","last_undegraded":"2024-09-20T22:08:10.420502+0000","last_fullsized":"2024-09-20T22:08:10.420502+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":259,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.411212+0000","last_change":"2024-09-20T22:07:23.495005+0000","last_active":"2024-09-20T22:08:10.411212+0000","last_peered":"2024-09-20T22:08:10.411212+0000","last_clean":"2024-09-20T22:08:10.411212+0000","last_became_active":"2024-09-20T22:06:54.848842+0000","last_became_peered":"2024-09-20T22:06:54.848842+0000","last_unstale":"2024-09-20T22:08:10.411212+0000","last_undegraded":"2024-09-20T22:08:10.411212+0000","last_fullsized":"2024-09-20T22:08:10.411212+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.494905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.494905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.999867+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"71'11","reported_seq":285,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.419978+0000","last_change":"2024-09-20T22:06:36.942847+0000","last_active":"2024-09-20T22:08:10.419978+0000","last_peered":"2024-09-20T22:08:10.419978+0000","last_clean":"2024-09-20T22:08:10.419978+0000","last_became_active":"2024-09-20T22:02:52.480118+0000","last_became_peered":"2024-09-20T22:02:52.480118+0000","last_unstale":"2024-09-20T22:08:10.419978+0000","last_undegraded":"2024-09-20T22:08:10.419978+0000","last_fullsized":"2024-09-20T22:08:10.419978+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:06:36.942620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:36.942620+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":279,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.414125+0000","last_change":"2024-09-20T22:07:50.851948+0000","last_active":"2024-09-20T22:08:10.414125+0000","last_peered":"2024-09-20T22:08:10.414125+0000","last_clean":"2024-09-20T22:08:10.414125+0000","last_became_active":"2024-09-20T22:06:54.828826+0000","last_became_peered":"2024-09-20T22:06:54.828826+0000","last_unstale":"2024-09-20T22:08:10.414125+0000","last_undegraded":"2024-09-20T22:08:10.414125+0000","last_fullsized":"2024-09-20T22:08:10.414125+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:50.851839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:50.851839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:57.182737+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":263,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420947+0000","last_change":"2024-09-20T22:06:45.780268+0000","last_active":"2024-09-20T22:08:10.420947+0000","last_peered":"2024-09-20T22:08:10.420947+0000","last_clean":"2024-09-20T22:08:10.420947+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:08:10.420947+0000","last_undegraded":"2024-09-20T22:08:10.420947+0000","last_fullsized":"2024-09-20T22:08:10.420947+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.780200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.780200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":258,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420655+0000","last_change":"2024-09-20T22:06:33.002428+0000","last_active":"2024-09-20T22:08:10.420655+0000","last_peered":"2024-09-20T22:08:10.420655+0000","last_clean":"2024-09-20T22:08:10.420655+0000","last_became_active":"2024-09-20T22:02:56.347637+0000","last_became_peered":"2024-09-20T22:02:56.347637+0000","last_unstale":"2024-09-20T22:08:10.420655+0000","last_undegraded":"2024-09-20T22:08:10.420655+0000","last_fullsized":"2024-09-20T22:08:10.420655+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:33.002326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:33.002326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"71'4","reported_seq":273,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.419203+0000","last_change":"2024-09-20T22:07:19.309318+0000","last_active":"2024-09-20T22:08:10.419203+0000","last_peered":"2024-09-20T22:08:10.419203+0000","last_clean":"2024-09-20T22:08:10.419203+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:08:10.419203+0000","last_undegraded":"2024-09-20T22:08:10.419203+0000","last_fullsized":"2024-09-20T22:08:10.419203+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'4","last_scrub_stamp":"2024-09-20T22:07:19.309187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.309187+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:33.061321+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":262,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.414653+0000","last_change":"2024-09-20T22:04:16.198264+0000","last_active":"2024-09-20T22:08:10.414653+0000","last_peered":"2024-09-20T22:08:10.414653+0000","last_clean":"2024-09-20T22:08:10.414653+0000","last_became_active":"2024-09-20T22:02:50.111142+0000","last_became_peered":"2024-09-20T22:02:50.111142+0000","last_unstale":"2024-09-20T22:08:10.414653+0000","last_undegraded":"2024-09-20T22:08:10.414653+0000","last_fullsized":"2024-09-20T22:08:10.414653+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:16.198153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:16.198153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":278,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.417006+0000","last_change":"2024-09-20T22:07:55.255933+0000","last_active":"2024-09-20T22:08:10.417006+0000","last_peered":"2024-09-20T22:08:10.417006+0000","last_clean":"2024-09-20T22:08:10.417006+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:08:10.417006+0000","last_undegraded":"2024-09-20T22:08:10.417006+0000","last_fullsized":"2024-09-20T22:08:10.417006+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:55.255854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.255854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.819675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":251,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.420263+0000","last_change":"2024-09-20T22:06:01.621567+0000","last_active":"2024-09-20T22:08:10.420263+0000","last_peered":"2024-09-20T22:08:10.420263+0000","last_clean":"2024-09-20T22:08:10.420263+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:08:10.420263+0000","last_undegraded":"2024-09-20T22:08:10.420263+0000","last_fullsized":"2024-09-20T22:08:10.420263+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:01.621489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:01.621489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":148,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.415577+0000","last_change":"2024-09-20T22:06:54.828772+0000","last_active":"2024-09-20T22:08:10.415577+0000","last_peered":"2024-09-20T22:08:10.415577+0000","last_clean":"2024-09-20T22:08:10.415577+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:08:10.415577+0000","last_undegraded":"2024-09-20T22:08:10.415577+0000","last_fullsized":"2024-09-20T22:08:10.415577+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:15.303300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:15.303300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:11.268490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":271,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.421198+0000","last_change":"2024-09-20T22:07:57.428285+0000","last_active":"2024-09-20T22:08:10.421198+0000","last_peered":"2024-09-20T22:08:10.421198+0000","last_clean":"2024-09-20T22:08:10.421198+0000","last_became_active":"2024-09-20T22:06:54.848032+0000","last_became_peered":"2024-09-20T22:06:54.848032+0000","last_unstale":"2024-09-20T22:08:10.421198+0000","last_undegraded":"2024-09-20T22:08:10.421198+0000","last_fullsized":"2024-09-20T22:08:10.421198+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:57.428165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:57.428165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:04.979638+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"71'11","reported_seq":286,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.417698+0000","last_change":"2024-09-20T22:07:19.214716+0000","last_active":"2024-09-20T22:08:10.417698+0000","last_peered":"2024-09-20T22:08:10.417698+0000","last_clean":"2024-09-20T22:08:10.417698+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:08:10.417698+0000","last_undegraded":"2024-09-20T22:08:10.417698+0000","last_fullsized":"2024-09-20T22:08:10.417698+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:19.214636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.214636+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:32.168748+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":254,"reported_epoch":185,"state":"active+clean","last_fresh":"2024-09-20T22:08:10.413409+0000","last_change":"2024-09-20T22:05:06.706314+0000","last_active":"2024-09-20T22:08:10.413409+0000","last_peered":"2024-09-20T22:08:10.413409+0000","last_clean":"2024-09-20T22:08:10.413409+0000","last_became_active":"2024-09-20T22:02:54.289078+0000","last_became_peered":"2024-09-20T22:02:54.289078+0000","last_unstale":"2024-09-20T22:08:10.413409+0000","last_undegraded":"2024-09-20T22:08:10.413409+0000","last_fullsized":"2024-09-20T22:08:10.413409+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:06.681123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:06.681123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":82,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":2439774208,"num_objects":37228,"num_object_clones":0,"num_object_copies":111684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":37228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":37228,"num_write_kb":2382592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7633436672,"data_stored":7633436672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":37228,"ondisk_log_size":37228,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":312,"num_read_kb":307,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135941,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1009452,"kb_used_data":807516,"kb_used_omap":1,"kb_used_meta":201918,"kb_avail":92735700,"statfs":{"total":95995035648,"available":94961356800,"internally_reserved":0,"allocated":826896384,"data_stored":826111492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":206764490},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430281,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1006300,"kb_used_data":822412,"kb_used_omap":1,"kb_used_meta":183870,"kb_avail":92738852,"statfs":{"total":95995035648,"available":94964584448,"internally_reserved":0,"allocated":842149888,"data_stored":841381288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":188283334},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724621,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1463768,"kb_used_data":1284040,"kb_used_omap":1,"kb_used_meta":179710,"kb_avail":92281384,"statfs":{"total":95995035648,"available":94496137216,"internally_reserved":0,"allocated":1314856960,"data_stored":1314092258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":184023493},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986261,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":998512,"kb_used_data":827104,"kb_used_omap":1,"kb_used_meta":171390,"kb_avail":92746640,"statfs":{"total":95995035648,"available":94972559360,"internally_reserved":0,"allocated":846954496,"data_stored":846164907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":175503813},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770257937,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1104208,"kb_used_data":996800,"kb_used_omap":4,"kb_used_meta":107387,"kb_avail":92640944,"statfs":{"total":95995035648,"available":94864326656,"internally_reserved":0,"allocated":1020723200,"data_stored":1019901587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":109964968},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149785,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1082116,"kb_used_data":780208,"kb_used_omap":1,"kb_used_meta":301886,"kb_avail":92663036,"statfs":{"total":95995035648,"available":94886948864,"internally_reserved":0,"allocated":798932992,"data_stored":798128311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":309131719},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1202316,"kb_used_data":982332,"kb_used_omap":1,"kb_used_meta":219966,"kb_avail":92542836,"statfs":{"total":95995035648,"available":94763864064,"internally_reserved":0,"allocated":1005907968,"data_stored":1005154350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":225245637},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738473,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1112368,"kb_used_data":788256,"kb_used_omap":1,"kb_used_meta":324094,"kb_avail":92632784,"statfs":{"total":95995035648,"available":94855970816,"internally_reserved":0,"allocated":807174144,"data_stored":806385288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":331872709},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":827850752,"data_stored":827850752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1034747904,"data_stored":1034747904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1039335424,"data_stored":1039335424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":821559296,"data_stored":821559296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":846462976,"data_stored":846462976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1353580544,"data_stored":1353580544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":856162304,"data_stored":856162304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":853737472,"data_stored":853737472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:08:19.734 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:19.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:19.735 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:19.753 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:19.757 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:19.835 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:19.866 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:19.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:19.867 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:19.968 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:20.012 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:20.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:20.013 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:20.114 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:20.135 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 23 16 48830 48814 132.624 105 0.00505738 0.00753463 2024-09-20T22:08:20.147 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:20.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:20.148 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:20.184 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:08:20.184 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:08:20.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:20.296 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:20.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:20.297 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:20.398 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:20.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:20.438 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:20.438 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:20.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:08:20.538 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:20.578 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:20.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:20.579 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:20.638 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:08:20.638 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:08:20.680 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:20.712 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:20.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:20.713 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:20.814 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:20.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:20.850 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:20.850 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:20.952 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:20.958 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:08:20.958 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:08:20.958 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:08:20.958 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:08:20.958 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:08:20.958 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:08:20.958 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:08:20.958 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:08:20.958 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:08:20.959 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:08:20.959 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:08:20.959 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:08:20.959 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:08:20.959 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:08:20.969 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:08:20.969 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:08:20.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:20.999 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:20.999 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:21.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:21.135 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 24 16 50379 50363 131.132 96.8125 0.0170529 0.00761881 2024-09-20T22:08:21.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:21.147 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:21.148 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:21.198 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:08:21.248 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:21.290 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:21.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:21.291 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:21.391 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:21.424 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:21 smithi060 ceph-mon[40290]: pgmap v266: 148 pgs: 148 active+clean; 2.6 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 1.0 KiB/s rd, 119 MiB/s wr, 1.90k op/s 2024-09-20T22:08:21.424 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-20T22:08:21.424 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2024-09-20T22:08:21.424 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 5]}]: dispatch 2024-09-20T22:08:21.424 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:21 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3110252011' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:21.424 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:21 smithi060 ceph-mon[29694]: pgmap v266: 148 pgs: 148 active+clean; 2.6 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 1.0 KiB/s rd, 119 MiB/s wr, 1.90k op/s 2024-09-20T22:08:21.424 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-20T22:08:21.425 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2024-09-20T22:08:21.425 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 5]}]: dispatch 2024-09-20T22:08:21.425 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:21 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3110252011' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:21.451 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:21.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:21.452 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:21.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:21 smithi067 ceph-mon[33511]: pgmap v266: 148 pgs: 148 active+clean; 2.6 GiB data, 9.2 GiB used, 706 GiB / 715 GiB avail; 1.0 KiB/s rd, 119 MiB/s wr, 1.90k op/s 2024-09-20T22:08:21.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]: dispatch 2024-09-20T22:08:21.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]: dispatch 2024-09-20T22:08:21.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 5]}]: dispatch 2024-09-20T22:08:21.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:21 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3110252011' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:21.553 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:21.583 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:21.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:21.584 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:21.684 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:21.720 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:21.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:21.721 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:21.822 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:21.861 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:21.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:21.862 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:21.934 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T22:08:21.964 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:22.024 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:22.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:22.025 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:22.117 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:08:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:08:22.125 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:22.135 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 25 16 51905 51889 129.701 95.375 0.0093835 0.0077036 2024-09-20T22:08:22.157 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:22.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:22.158 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:22.259 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:22.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:22 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-20T22:08:22.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:22 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]': finished 2024-09-20T22:08:22.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:22 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 5]}]': finished 2024-09-20T22:08:22.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:22 smithi060 ceph-mon[40290]: osdmap e187: 8 total, 8 up, 8 in 2024-09-20T22:08:22.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:22 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-20T22:08:22.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:22 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]': finished 2024-09-20T22:08:22.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:22 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 5]}]': finished 2024-09-20T22:08:22.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:22 smithi060 ceph-mon[29694]: osdmap e187: 8 total, 8 up, 8 in 2024-09-20T22:08:22.290 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:22.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:22.291 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:22.392 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:22.436 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:22.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:22.437 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:22.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:22 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [1, 2]}]': finished 2024-09-20T22:08:22.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:22 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.13", "id": [1, 2]}]': finished 2024-09-20T22:08:22.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:22 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 5]}]': finished 2024-09-20T22:08:22.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:22 smithi067 ceph-mon[33511]: osdmap e187: 8 total, 8 up, 8 in 2024-09-20T22:08:22.538 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:22.575 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:22.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:22.577 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:22.677 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:22.709 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:22.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:22.711 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:22.722 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:08:22.788 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:22.812 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:22.840 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:22.840 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:22.841 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:22.941 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:22.975 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:22.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:22.977 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:23.078 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:23.122 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:23.139 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 26 16 53340 53324 128.156 89.6875 0.00845017 0.00779656 2024-09-20T22:08:23.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:23.140 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:23.140 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:23.241 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:23.272 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:23 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:08:23.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:23 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:08:23.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:23 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:08:23.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:23 smithi060 ceph-mon[29694]: pgmap v268: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 116 MiB/s wr, 1.85k op/s 2024-09-20T22:08:23.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:23 smithi060 ceph-mon[29694]: osdmap e188: 8 total, 8 up, 8 in 2024-09-20T22:08:23.274 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:23 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:08:23.274 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:23 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:08:23.274 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:23 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:08:23.274 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:23 smithi060 ceph-mon[40290]: pgmap v268: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 116 MiB/s wr, 1.85k op/s 2024-09-20T22:08:23.274 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:23 smithi060 ceph-mon[40290]: osdmap e188: 8 total, 8 up, 8 in 2024-09-20T22:08:23.288 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:23.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:23.289 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:23.390 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:23.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:23.435 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:23.435 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:23.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:23 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:08:23.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:23 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:08:23.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:23 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:08:23.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:23 smithi067 ceph-mon[33511]: pgmap v268: 148 pgs: 148 active+clean; 3.0 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 116 MiB/s wr, 1.85k op/s 2024-09-20T22:08:23.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:23 smithi067 ceph-mon[33511]: osdmap e188: 8 total, 8 up, 8 in 2024-09-20T22:08:23.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:23.580 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:23.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:23.581 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:23.682 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:23.712 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:23.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:23.713 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:23.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:23.857 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:23.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:23.858 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:23.882 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:08:23.882 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:08:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:08:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:08:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169948 7.9 7:94ca5e9f:::benchmark_data_smithi060_78800_object36915:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:08:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440369+0000", 2024-09-20T22:08:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 145.40024557800001, 2024-09-20T22:08:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.04459061, 2024-09-20T22:08:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:08:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169948 2024-09-20T22:08:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440369+0000", 2024-09-20T22:08:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440369+0000", 2024-09-20T22:08:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440371+0000", 2024-09-20T22:08:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.567e-06 2024-09-20T22:08:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440375+0000", 2024-09-20T22:08:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7910000000000004e-06 2024-09-20T22:08:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440388+0000", 2024-09-20T22:08:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2354999999999999e-05 2024-09-20T22:08:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440395+0000", 2024-09-20T22:08:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5739999999999998e-06 2024-09-20T22:08:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.650274+0000", 2024-09-20T22:08:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.209878595 2024-09-20T22:08:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.650283+0000", 2024-09-20T22:08:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6070000000000008e-06 2024-09-20T22:08:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.484885+0000", 2024-09-20T22:08:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.83460233500000003 2024-09-20T22:08:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.484960+0000", 2024-09-20T22:08:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4785999999999999e-05 2024-09-20T22:08:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169959 7.19 7:9aa79aaf:::benchmark_data_smithi060_78800_object36926:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:08:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440410+0000", 2024-09-20T22:08:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 145.40020464200001, 2024-09-20T22:08:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0776166780000001, 2024-09-20T22:08:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:08:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169959 2024-09-20T22:08:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:23.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:23.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440410+0000", 2024-09-20T22:08:23.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:23.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440410+0000", 2024-09-20T22:08:23.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:23.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440410+0000", 2024-09-20T22:08:23.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9299999999999999e-07 2024-09-20T22:08:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440413+0000", 2024-09-20T22:08:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7219999999999999e-06 2024-09-20T22:08:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440419+0000", 2024-09-20T22:08:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0449999999999997e-06 2024-09-20T22:08:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440424+0000", 2024-09-20T22:08:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7090000000000001e-06 2024-09-20T22:08:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.651028+0000", 2024-09-20T22:08:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.21060456799999999 2024-09-20T22:08:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.651036+0000", 2024-09-20T22:08:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8720000000000002e-06 2024-09-20T22:08:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.517832+0000", 2024-09-20T22:08:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.86679538099999998 2024-09-20T22:08:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.518027+0000", 2024-09-20T22:08:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000194988 2024-09-20T22:08:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169973 7.19 7:9cb6a65c:::benchmark_data_smithi060_78800_object36940:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:08:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.452156+0000", 2024-09-20T22:08:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 145.38845888, 2024-09-20T22:08:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.079163852, 2024-09-20T22:08:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169973 2024-09-20T22:08:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.452156+0000", 2024-09-20T22:08:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.452156+0000", 2024-09-20T22:08:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.452158+0000", 2024-09-20T22:08:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0999999999999998e-06 2024-09-20T22:08:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.452165+0000", 2024-09-20T22:08:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5039999999999996e-06 2024-09-20T22:08:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.452181+0000", 2024-09-20T22:08:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5226e-05 2024-09-20T22:08:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.452190+0000", 2024-09-20T22:08:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0380000000000005e-06 2024-09-20T22:08:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.651057+0000", 2024-09-20T22:08:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.19886704599999999 2024-09-20T22:08:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.651062+0000", 2024-09-20T22:08:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7230000000000001e-06 2024-09-20T22:08:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.518364+0000", 2024-09-20T22:08:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.86730196400000004 2024-09-20T22:08:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.518458+0000", 2024-09-20T22:08:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3105999999999999e-05 2024-09-20T22:08:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4", 2024-09-20T22:08:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.518738+0000", 2024-09-20T22:08:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000280005 2024-09-20T22:08:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.521963+0000", 2024-09-20T22:08:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003225374 2024-09-20T22:08:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.531163+0000", 2024-09-20T22:08:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0091998740000000002 2024-09-20T22:08:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.531225+0000", 2024-09-20T22:08:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2119000000000006e-05 2024-09-20T22:08:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.531320+0000", 2024-09-20T22:08:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4772999999999995e-05 2024-09-20T22:08:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:197601 7.13 7:c8cc3338:::benchmark_data_smithi060_78800_object64568:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e132)", 2024-09-20T22:08:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:24.370429+0000", 2024-09-20T22:08:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 119.47018536, 2024-09-20T22:08:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0703220390000001, 2024-09-20T22:08:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:08:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 197601 2024-09-20T22:08:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370429+0000", 2024-09-20T22:08:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370429+0000", 2024-09-20T22:08:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:23.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370430+0000", 2024-09-20T22:08:23.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6200000000000006e-07 2024-09-20T22:08:23.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:23.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370435+0000", 2024-09-20T22:08:23.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5589999999999999e-06 2024-09-20T22:08:23.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:23.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370447+0000", 2024-09-20T22:08:23.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2058e-05 2024-09-20T22:08:23.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:23.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370454+0000", 2024-09-20T22:08:23.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4449999999999999e-06 2024-09-20T22:08:23.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370854+0000", 2024-09-20T22:08:23.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00039970400000000002 2024-09-20T22:08:23.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op must wait for map", 2024-09-20T22:08:23.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370861+0000", 2024-09-20T22:08:23.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8190000000000003e-06 2024-09-20T22:08:23.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.373710+0000", 2024-09-20T22:08:23.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0028495270000000001 2024-09-20T22:08:23.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.373724+0000", 2024-09-20T22:08:23.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.395e-05 2024-09-20T22:08:23.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.435399+0000", 2024-09-20T22:08:23.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0616748069999999 2024-09-20T22:08:23.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:08:23.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.435541+0000", 2024-09-20T22:08:23.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014141499999999999 2024-09-20T22:08:23.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.440680+0000", 2024-09-20T22:08:23.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0051391170000000003 2024-09-20T22:08:23.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.905 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:23.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.440751+0000", 2024-09-20T22:08:23.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1675999999999996e-05 2024-09-20T22:08:23.905 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.905 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:23.905 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.905 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:197609 7.13 7:c869088e:::benchmark_data_smithi060_78800_object64576:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e132)", 2024-09-20T22:08:23.905 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:24.370476+0000", 2024-09-20T22:08:23.905 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 119.47013907900001, 2024-09-20T22:08:23.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.075397427, 2024-09-20T22:08:23.906 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:23.906 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:23.906 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:23.906 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:23.906 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:23.906 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:23.906 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 197609 2024-09-20T22:08:23.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.906 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:23.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.906 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:23.906 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370476+0000", 2024-09-20T22:08:23.906 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:23.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370476+0000", 2024-09-20T22:08:23.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:23.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370476+0000", 2024-09-20T22:08:23.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4299999999999998e-07 2024-09-20T22:08:23.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:23.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370479+0000", 2024-09-20T22:08:23.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5299999999999999e-06 2024-09-20T22:08:23.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:23.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370486+0000", 2024-09-20T22:08:23.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.968e-06 2024-09-20T22:08:23.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:23.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370491+0000", 2024-09-20T22:08:23.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6359999999999999e-06 2024-09-20T22:08:23.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370914+0000", 2024-09-20T22:08:23.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00042270099999999999 2024-09-20T22:08:23.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting_for_map not empty", 2024-09-20T22:08:23.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370919+0000", 2024-09-20T22:08:23.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6019999999999999e-06 2024-09-20T22:08:23.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.373986+0000", 2024-09-20T22:08:23.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0030664049999999999 2024-09-20T22:08:23.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.373993+0000", 2024-09-20T22:08:23.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5979999999999999e-06 2024-09-20T22:08:23.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.440778+0000", 2024-09-20T22:08:23.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0667841199999999 2024-09-20T22:08:23.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:23.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.440952+0000", 2024-09-20T22:08:23.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017486199999999999 2024-09-20T22:08:23.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:08:23.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.441236+0000", 2024-09-20T22:08:23.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00028399600000000001 2024-09-20T22:08:23.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445645+0000", 2024-09-20T22:08:23.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0044080669999999999 2024-09-20T22:08:23.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:23.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445661+0000", 2024-09-20T22:08:23.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6928000000000001e-05 2024-09-20T22:08:23.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445783+0000", 2024-09-20T22:08:23.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121438 2024-09-20T22:08:23.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:23.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445816+0000", 2024-09-20T22:08:23.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3413e-05 2024-09-20T22:08:23.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:23.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445873+0000", 2024-09-20T22:08:23.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6719999999999999e-05 2024-09-20T22:08:23.913 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.913 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:23.913 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.913 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:197635 7.13 7:c81e98ce:::benchmark_data_smithi060_78800_object64602:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-20T22:08:23.913 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:24.390574+0000", 2024-09-20T22:08:23.913 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 119.450040387, 2024-09-20T22:08:23.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0554612830000001, 2024-09-20T22:08:23.913 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:23.914 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:23.914 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:23.914 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:23.914 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:23.914 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:23.914 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 197635 2024-09-20T22:08:23.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.914 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:23.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:23.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.390574+0000", 2024-09-20T22:08:23.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:23.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.390574+0000", 2024-09-20T22:08:23.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:23.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.390576+0000", 2024-09-20T22:08:23.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5349999999999999e-06 2024-09-20T22:08:23.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:23.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.390582+0000", 2024-09-20T22:08:23.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6840000000000001e-06 2024-09-20T22:08:23.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:23.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.390596+0000", 2024-09-20T22:08:23.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4088e-05 2024-09-20T22:08:23.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:23.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.390603+0000", 2024-09-20T22:08:23.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8269999999999998e-06 2024-09-20T22:08:23.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.390654+0000", 2024-09-20T22:08:23.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0074000000000002e-05 2024-09-20T22:08:23.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.390662+0000", 2024-09-20T22:08:23.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5609999999999999e-06 2024-09-20T22:08:23.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.442542+0000", 2024-09-20T22:08:23.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0518796610000001 2024-09-20T22:08:23.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:23.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.442657+0000", 2024-09-20T22:08:23.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115215 2024-09-20T22:08:23.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:08:23.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.442883+0000", 2024-09-20T22:08:23.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000225715 2024-09-20T22:08:23.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:23.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445668+0000", 2024-09-20T22:08:23.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0027853349999999999 2024-09-20T22:08:23.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445727+0000", 2024-09-20T22:08:23.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9296000000000003e-05 2024-09-20T22:08:23.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445924+0000", 2024-09-20T22:08:23.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000197104 2024-09-20T22:08:23.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:23.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445954+0000", 2024-09-20T22:08:23.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0094999999999999e-05 2024-09-20T22:08:23.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:23.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.446036+0000", 2024-09-20T22:08:23.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1093000000000004e-05 2024-09-20T22:08:23.920 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.920 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:23.920 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.920 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:197657 7.13 7:cbcc9238:::benchmark_data_smithi060_78800_object64624:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e132)", 2024-09-20T22:08:23.920 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:24.409026+0000", 2024-09-20T22:08:23.920 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 119.43158875, 2024-09-20T22:08:23.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.037258835, 2024-09-20T22:08:23.920 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:23.920 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:23.920 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:23.921 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:23.921 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:23.921 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:23.921 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 197657 2024-09-20T22:08:23.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.921 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:23.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:23.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.409026+0000", 2024-09-20T22:08:23.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:23.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.409026+0000", 2024-09-20T22:08:23.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:23.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.409027+0000", 2024-09-20T22:08:23.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.141e-06 2024-09-20T22:08:23.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:23.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.409032+0000", 2024-09-20T22:08:23.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7199999999999997e-06 2024-09-20T22:08:23.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:23.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.409042+0000", 2024-09-20T22:08:23.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7450000000000007e-06 2024-09-20T22:08:23.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:23.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.409047+0000", 2024-09-20T22:08:23.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1460000000000004e-06 2024-09-20T22:08:23.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.409095+0000", 2024-09-20T22:08:23.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8331000000000002e-05 2024-09-20T22:08:23.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.409103+0000", 2024-09-20T22:08:23.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3270000000000001e-06 2024-09-20T22:08:23.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.444150+0000", 2024-09-20T22:08:23.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.035046707 2024-09-20T22:08:23.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:23.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.444279+0000", 2024-09-20T22:08:23.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012934000000000001 2024-09-20T22:08:23.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:08:23.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.444483+0000", 2024-09-20T22:08:23.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020355199999999999 2024-09-20T22:08:23.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:23.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445886+0000", 2024-09-20T22:08:23.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001403114 2024-09-20T22:08:23.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.446099+0000", 2024-09-20T22:08:23.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021258799999999999 2024-09-20T22:08:23.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.446146+0000", 2024-09-20T22:08:23.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7144000000000001e-05 2024-09-20T22:08:23.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:23.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.446194+0000", 2024-09-20T22:08:23.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8158999999999997e-05 2024-09-20T22:08:23.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:23.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.446285+0000", 2024-09-20T22:08:23.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0821000000000003e-05 2024-09-20T22:08:23.927 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.927 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:23.927 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.928 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198099 7.18 7:1dc840d9:::benchmark_data_smithi060_78800_object65066:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:08:23.928 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.455807+0000", 2024-09-20T22:08:23.928 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.38480793700001, 2024-09-20T22:08:23.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97129763599999996, 2024-09-20T22:08:23.928 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:23.928 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:23.928 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:23.928 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:23.928 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:23.928 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:23.928 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198099 2024-09-20T22:08:23.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.928 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:23.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:23.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.455807+0000", 2024-09-20T22:08:23.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:23.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.455807+0000", 2024-09-20T22:08:23.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:23.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.455808+0000", 2024-09-20T22:08:23.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7199999999999998e-07 2024-09-20T22:08:23.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:23.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.455811+0000", 2024-09-20T22:08:23.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7290000000000002e-06 2024-09-20T22:08:23.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:23.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.455822+0000", 2024-09-20T22:08:23.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0331e-05 2024-09-20T22:08:23.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.455826+0000", 2024-09-20T22:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5719999999999996e-06 2024-09-20T22:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.455881+0000", 2024-09-20T22:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5068999999999997e-05 2024-09-20T22:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.455889+0000", 2024-09-20T22:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5429999999999996e-06 2024-09-20T22:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.422697+0000", 2024-09-20T22:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.96680820099999998 2024-09-20T22:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.422844+0000", 2024-09-20T22:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147128 2024-09-20T22:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.423134+0000", 2024-09-20T22:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00028993600000000002 2024-09-20T22:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.426960+0000", 2024-09-20T22:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003825803 2024-09-20T22:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.427001+0000", 2024-09-20T22:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1100000000000003e-05 2024-09-20T22:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.427049+0000", 2024-09-20T22:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8378999999999999e-05 2024-09-20T22:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.427080+0000", 2024-09-20T22:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0397999999999999e-05 2024-09-20T22:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.427104+0000", 2024-09-20T22:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4675e-05 2024-09-20T22:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.934 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198205 7.18 7:1bc47e83:::benchmark_data_smithi060_78800_object65172:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.532610+0000", 2024-09-20T22:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.30800427200001, 2024-09-20T22:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.89743818799999997, 2024-09-20T22:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198205 2024-09-20T22:08:23.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.532610+0000", 2024-09-20T22:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.532610+0000", 2024-09-20T22:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:23.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.532611+0000", 2024-09-20T22:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2099999999999995e-07 2024-09-20T22:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.532616+0000", 2024-09-20T22:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8500000000000002e-06 2024-09-20T22:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.532628+0000", 2024-09-20T22:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1331e-05 2024-09-20T22:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.532634+0000", 2024-09-20T22:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6880000000000002e-06 2024-09-20T22:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.532705+0000", 2024-09-20T22:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0949000000000001e-05 2024-09-20T22:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.532713+0000", 2024-09-20T22:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.542e-06 2024-09-20T22:08:23.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.424810+0000", 2024-09-20T22:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.892097052 2024-09-20T22:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.424880+0000", 2024-09-20T22:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9961000000000003e-05 2024-09-20T22:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.425039+0000", 2024-09-20T22:08:23.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000159314 2024-09-20T22:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.427185+0000", 2024-09-20T22:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021459679999999998 2024-09-20T22:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.429908+0000", 2024-09-20T22:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002723442 2024-09-20T22:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.429955+0000", 2024-09-20T22:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6597000000000001e-05 2024-09-20T22:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.429996+0000", 2024-09-20T22:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1334000000000001e-05 2024-09-20T22:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430049+0000", 2024-09-20T22:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2238999999999999e-05 2024-09-20T22:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:23.941 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198209 7.18 7:1b4b9a8f:::benchmark_data_smithi060_78800_object65176:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.535076+0000", 2024-09-20T22:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 118.305539009, 2024-09-20T22:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.89904252500000004, 2024-09-20T22:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198209 2024-09-20T22:08:23.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.535076+0000", 2024-09-20T22:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.535076+0000", 2024-09-20T22:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.535077+0000", 2024-09-20T22:08:23.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9100000000000002e-07 2024-09-20T22:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.535081+0000", 2024-09-20T22:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8060000000000002e-06 2024-09-20T22:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.535096+0000", 2024-09-20T22:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4491999999999999e-05 2024-09-20T22:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.535103+0000", 2024-09-20T22:08:23.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8689999999999998e-06 2024-09-20T22:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.535153+0000", 2024-09-20T22:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0510000000000003e-05 2024-09-20T22:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.535161+0000", 2024-09-20T22:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3930000000000002e-06 2024-09-20T22:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.427218+0000", 2024-09-20T22:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.89205771099999998 2024-09-20T22:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.427309+0000", 2024-09-20T22:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0476999999999994e-05 2024-09-20T22:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.427501+0000", 2024-09-20T22:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019246899999999999 2024-09-20T22:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430275+0000", 2024-09-20T22:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0027736309999999999 2024-09-20T22:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430313+0000", 2024-09-20T22:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8383000000000002e-05 2024-09-20T22:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.434027+0000", 2024-09-20T22:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037134170000000001 2024-09-20T22:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.434077+0000", 2024-09-20T22:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0408000000000003e-05 2024-09-20T22:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.434118+0000", 2024-09-20T22:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1068000000000001e-05 2024-09-20T22:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227609 7.4 7:26fdb431:::benchmark_data_smithi060_78800_object94576:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-20T22:08:23.948 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.802526+0000", 2024-09-20T22:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.038088278999993, 2024-09-20T22:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.327846133, 2024-09-20T22:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227609 2024-09-20T22:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802526+0000", 2024-09-20T22:08:23.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802526+0000", 2024-09-20T22:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802528+0000", 2024-09-20T22:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2550000000000001e-06 2024-09-20T22:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802533+0000", 2024-09-20T22:08:23.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5500000000000002e-06 2024-09-20T22:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802542+0000", 2024-09-20T22:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6950000000000006e-06 2024-09-20T22:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802548+0000", 2024-09-20T22:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8100000000000003e-06 2024-09-20T22:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802787+0000", 2024-09-20T22:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023914699999999999 2024-09-20T22:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802897+0000", 2024-09-20T22:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109645 2024-09-20T22:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4", 2024-09-20T22:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.803676+0000", 2024-09-20T22:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000779085 2024-09-20T22:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.814426+0000", 2024-09-20T22:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010750857000000001 2024-09-20T22:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.814439+0000", 2024-09-20T22:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2549e-05 2024-09-20T22:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.823484+0000", 2024-09-20T22:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0090446120000001 2024-09-20T22:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.823496+0000", 2024-09-20T22:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2413e-05 2024-09-20T22:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.115278+0000", 2024-09-20T22:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.291782427 2024-09-20T22:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.115476+0000", 2024-09-20T22:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019718599999999999 2024-09-20T22:08:23.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130062+0000", 2024-09-20T22:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014586541 2024-09-20T22:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130373+0000", 2024-09-20T22:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00031036099999999999 2024-09-20T22:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.956 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227614 7.4 7:22c1d0bc:::benchmark_data_smithi060_78800_object94581:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-20T22:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.804108+0000", 2024-09-20T22:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 90.036506281000001, 2024-09-20T22:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3264465649999999, 2024-09-20T22:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227614 2024-09-20T22:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.804108+0000", 2024-09-20T22:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.804108+0000", 2024-09-20T22:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.804110+0000", 2024-09-20T22:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1510000000000001e-06 2024-09-20T22:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.804115+0000", 2024-09-20T22:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0969999999999996e-06 2024-09-20T22:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.804128+0000", 2024-09-20T22:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2899e-05 2024-09-20T22:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.804136+0000", 2024-09-20T22:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0479999999999999e-06 2024-09-20T22:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.814469+0000", 2024-09-20T22:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010333187000000001 2024-09-20T22:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.814478+0000", 2024-09-20T22:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9530000000000005e-06 2024-09-20T22:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.823863+0000", 2024-09-20T22:08:23.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0093847810000001 2024-09-20T22:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.823872+0000", 2024-09-20T22:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.312e-06 2024-09-20T22:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.115762+0000", 2024-09-20T22:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.291889702 2024-09-20T22:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.115904+0000", 2024-09-20T22:08:23.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014230800000000001 2024-09-20T22:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.116318+0000", 2024-09-20T22:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00041407100000000001 2024-09-20T22:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.123684+0000", 2024-09-20T22:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0073657769999999996 2024-09-20T22:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:23.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.129612+0000", 2024-09-20T22:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0059283519999999996 2024-09-20T22:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130418+0000", 2024-09-20T22:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00080579299999999996 2024-09-20T22:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130473+0000", 2024-09-20T22:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4740000000000001e-05 2024-09-20T22:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:23.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130555+0000", 2024-09-20T22:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2393999999999998e-05 2024-09-20T22:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227652 7.4 7:213b8ad4:::benchmark_data_smithi060_78800_object94619:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.904523+0000", 2024-09-20T22:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.936092172000002, 2024-09-20T22:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2262305599999999, 2024-09-20T22:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:23.964 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227652 2024-09-20T22:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904523+0000", 2024-09-20T22:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904523+0000", 2024-09-20T22:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904524+0000", 2024-09-20T22:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0100000000000001e-06 2024-09-20T22:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904528+0000", 2024-09-20T22:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4710000000000001e-06 2024-09-20T22:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:23.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904537+0000", 2024-09-20T22:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2289999999999992e-06 2024-09-20T22:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904544+0000", 2024-09-20T22:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6900000000000003e-06 2024-09-20T22:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904608+0000", 2024-09-20T22:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3598999999999999e-05 2024-09-20T22:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904617+0000", 2024-09-20T22:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8179999999999999e-06 2024-09-20T22:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.823899+0000", 2024-09-20T22:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.919281338 2024-09-20T22:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.823907+0000", 2024-09-20T22:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9449999999999993e-06 2024-09-20T22:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.118500+0000", 2024-09-20T22:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2945934700000001 2024-09-20T22:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.118634+0000", 2024-09-20T22:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013352700000000001 2024-09-20T22:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.119027+0000", 2024-09-20T22:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000393533 2024-09-20T22:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.129598+0000", 2024-09-20T22:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010570941 2024-09-20T22:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.129619+0000", 2024-09-20T22:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0854000000000001e-05 2024-09-20T22:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130623+0000", 2024-09-20T22:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001003947 2024-09-20T22:08:23.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130663+0000", 2024-09-20T22:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0139000000000003e-05 2024-09-20T22:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130753+0000", 2024-09-20T22:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0049000000000005e-05 2024-09-20T22:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227664 7.4 7:264fd802:::benchmark_data_smithi060_78800_object94631:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.969131+0000", 2024-09-20T22:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 89.871483396000002, 2024-09-20T22:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1748220109999998, 2024-09-20T22:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227664 2024-09-20T22:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:23.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969131+0000", 2024-09-20T22:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969131+0000", 2024-09-20T22:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969133+0000", 2024-09-20T22:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.421e-06 2024-09-20T22:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969138+0000", 2024-09-20T22:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9239999999999998e-06 2024-09-20T22:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969148+0000", 2024-09-20T22:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0696000000000001e-05 2024-09-20T22:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969155+0000", 2024-09-20T22:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7780000000000002e-06 2024-09-20T22:08:23.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969217+0000", 2024-09-20T22:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2304000000000005e-05 2024-09-20T22:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969229+0000", 2024-09-20T22:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1100999999999999e-05 2024-09-20T22:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.824751+0000", 2024-09-20T22:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85552231099999998 2024-09-20T22:08:23.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.824761+0000", 2024-09-20T22:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9210000000000004e-06 2024-09-20T22:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.126989+0000", 2024-09-20T22:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.302228449 2024-09-20T22:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.127107+0000", 2024-09-20T22:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117712 2024-09-20T22:08:23.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.127530+0000", 2024-09-20T22:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000423056 2024-09-20T22:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130029+0000", 2024-09-20T22:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024987020000000002 2024-09-20T22:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.133706+0000", 2024-09-20T22:08:23.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0036771659999999999 2024-09-20T22:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.143463+0000", 2024-09-20T22:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0097571510000000004 2024-09-20T22:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.143714+0000", 2024-09-20T22:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00025083199999999999 2024-09-20T22:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:23.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.143953+0000", 2024-09-20T22:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023948700000000001 2024-09-20T22:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236412 7.9 7:953e1373:::benchmark_data_smithi060_78800_object103379:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-20T22:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.364320+0000", 2024-09-20T22:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.476294781000007, 2024-09-20T22:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1050446030000001, 2024-09-20T22:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:08:23.979 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236412 2024-09-20T22:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364320+0000", 2024-09-20T22:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364320+0000", 2024-09-20T22:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364321+0000", 2024-09-20T22:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.533e-06 2024-09-20T22:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364328+0000", 2024-09-20T22:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.241e-06 2024-09-20T22:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364346+0000", 2024-09-20T22:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8546000000000002e-05 2024-09-20T22:08:23.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364354+0000", 2024-09-20T22:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6839999999999996e-06 2024-09-20T22:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364429+0000", 2024-09-20T22:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4618999999999998e-05 2024-09-20T22:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364584+0000", 2024-09-20T22:08:23.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155725 2024-09-20T22:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.367359+0000", 2024-09-20T22:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002774421 2024-09-20T22:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.379883+0000", 2024-09-20T22:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.012524208 2024-09-20T22:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.379895+0000", 2024-09-20T22:08:23.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2268e-05 2024-09-20T22:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.421975+0000", 2024-09-20T22:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042079623 2024-09-20T22:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.422090+0000", 2024-09-20T22:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011518000000000001 2024-09-20T22:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.469296+0000", 2024-09-20T22:08:23.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.047205907999999998 2024-09-20T22:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.469365+0000", 2024-09-20T22:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8646999999999995e-05 2024-09-20T22:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236413 7.9 7:92cb2bc7:::benchmark_data_smithi060_78800_object103380:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-20T22:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.364373+0000", 2024-09-20T22:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.476241521999995, 2024-09-20T22:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.105379702, 2024-09-20T22:08:23.985 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236413 2024-09-20T22:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364373+0000", 2024-09-20T22:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364373+0000", 2024-09-20T22:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364374+0000", 2024-09-20T22:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2100000000000002e-07 2024-09-20T22:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364376+0000", 2024-09-20T22:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5440000000000001e-06 2024-09-20T22:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364383+0000", 2024-09-20T22:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7870000000000004e-06 2024-09-20T22:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364389+0000", 2024-09-20T22:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6540000000000004e-06 2024-09-20T22:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.374885+0000", 2024-09-20T22:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010496871 2024-09-20T22:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.375004+0000", 2024-09-20T22:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118956 2024-09-20T22:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.375466+0000", 2024-09-20T22:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00046160599999999998 2024-09-20T22:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.379926+0000", 2024-09-20T22:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0044598850000000002 2024-09-20T22:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.379935+0000", 2024-09-20T22:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8140000000000007e-06 2024-09-20T22:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.422377+0000", 2024-09-20T22:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042442028 2024-09-20T22:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.422482+0000", 2024-09-20T22:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010542 2024-09-20T22:08:23.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.469677+0000", 2024-09-20T22:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.047194699 2024-09-20T22:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.469753+0000", 2024-09-20T22:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6017000000000001e-05 2024-09-20T22:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236450 7.9 7:95eabd46:::benchmark_data_smithi060_78800_object103417:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.406110+0000", 2024-09-20T22:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.434504985000004, 2024-09-20T22:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0608129049999999, 2024-09-20T22:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236450 2024-09-20T22:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:23.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.406110+0000", 2024-09-20T22:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.406110+0000", 2024-09-20T22:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.406111+0000", 2024-09-20T22:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.248e-06 2024-09-20T22:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.406115+0000", 2024-09-20T22:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3830000000000002e-06 2024-09-20T22:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.406130+0000", 2024-09-20T22:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4456e-05 2024-09-20T22:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.406137+0000", 2024-09-20T22:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5329999999999997e-06 2024-09-20T22:08:23.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.406191+0000", 2024-09-20T22:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3764999999999998e-05 2024-09-20T22:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.406200+0000", 2024-09-20T22:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6710000000000005e-06 2024-09-20T22:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.422520+0000", 2024-09-20T22:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0163197589999999 2024-09-20T22:08:23.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.422622+0000", 2024-09-20T22:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010295 2024-09-20T22:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.423898+0000", 2024-09-20T22:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001275272 2024-09-20T22:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.447631+0000", 2024-09-20T22:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.023732896 2024-09-20T22:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.448017+0000", 2024-09-20T22:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000386723 2024-09-20T22:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.466783+0000", 2024-09-20T22:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.01876566 2024-09-20T22:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.466838+0000", 2024-09-20T22:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5241000000000002e-05 2024-09-20T22:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.466923+0000", 2024-09-20T22:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4347999999999999e-05 2024-09-20T22:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:23.999 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236490 7.9 7:97b0ba39:::benchmark_data_smithi060_78800_object103457:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.446394+0000", 2024-09-20T22:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.394220271999998, 2024-09-20T22:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.021274571, 2024-09-20T22:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236490 2024-09-20T22:08:24.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.446394+0000", 2024-09-20T22:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.446394+0000", 2024-09-20T22:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:24.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.446396+0000", 2024-09-20T22:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.271e-06 2024-09-20T22:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.446401+0000", 2024-09-20T22:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8230000000000002e-06 2024-09-20T22:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.446413+0000", 2024-09-20T22:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.234e-05 2024-09-20T22:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:24.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.446516+0000", 2024-09-20T22:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103064 2024-09-20T22:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.446596+0000", 2024-09-20T22:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0208000000000002e-05 2024-09-20T22:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.446604+0000", 2024-09-20T22:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0050000000000004e-06 2024-09-20T22:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:24.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426372+0000", 2024-09-20T22:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97976804299999998 2024-09-20T22:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426684+0000", 2024-09-20T22:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00031145599999999999 2024-09-20T22:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.427744+0000", 2024-09-20T22:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0010606910000000001 2024-09-20T22:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:24.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.447792+0000", 2024-09-20T22:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.020047586999999999 2024-09-20T22:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.448023+0000", 2024-09-20T22:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000230926 2024-09-20T22:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.466958+0000", 2024-09-20T22:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.018935508 2024-09-20T22:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:24.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.466998+0000", 2024-09-20T22:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9381999999999997e-05 2024-09-20T22:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.467669+0000", 2024-09-20T22:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00067126700000000002 2024-09-20T22:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236505 7.9 7:949e4424:::benchmark_data_smithi060_78800_object103472:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.465770+0000", 2024-09-20T22:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.374844956000004, 2024-09-20T22:08:24.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.002128857, 2024-09-20T22:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236505 2024-09-20T22:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:24.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.465770+0000", 2024-09-20T22:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.465770+0000", 2024-09-20T22:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.465771+0000", 2024-09-20T22:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1370000000000001e-06 2024-09-20T22:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.465776+0000", 2024-09-20T22:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9320000000000004e-06 2024-09-20T22:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.465786+0000", 2024-09-20T22:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0131e-05 2024-09-20T22:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.465793+0000", 2024-09-20T22:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7560000000000002e-06 2024-09-20T22:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.465849+0000", 2024-09-20T22:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5983000000000003e-05 2024-09-20T22:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.465856+0000", 2024-09-20T22:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6499999999999996e-06 2024-09-20T22:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:24.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432586+0000", 2024-09-20T22:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.96672997599999999 2024-09-20T22:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432706+0000", 2024-09-20T22:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119381 2024-09-20T22:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.433124+0000", 2024-09-20T22:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00041824400000000002 2024-09-20T22:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.447842+0000", 2024-09-20T22:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014717661 2024-09-20T22:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.448027+0000", 2024-09-20T22:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000185706 2024-09-20T22:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.467735+0000", 2024-09-20T22:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.019707730999999999 2024-09-20T22:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.467789+0000", 2024-09-20T22:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3752999999999998e-05 2024-09-20T22:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.467899+0000", 2024-09-20T22:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010981599999999999 2024-09-20T22:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236526 7.9 7:91b5ec90:::benchmark_data_smithi060_78800_object103493:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:08:24.013 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.490578+0000", 2024-09-20T22:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 76.350036375000002, 2024-09-20T22:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97751770000000004, 2024-09-20T22:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236526 2024-09-20T22:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:24.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.490578+0000", 2024-09-20T22:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.490578+0000", 2024-09-20T22:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.490580+0000", 2024-09-20T22:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.888e-06 2024-09-20T22:08:24.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.490586+0000", 2024-09-20T22:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7429999999999999e-06 2024-09-20T22:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.490598+0000", 2024-09-20T22:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1985e-05 2024-09-20T22:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.490606+0000", 2024-09-20T22:08:24.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9219999999999997e-06 2024-09-20T22:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.490756+0000", 2024-09-20T22:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014965900000000001 2024-09-20T22:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.490766+0000", 2024-09-20T22:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0036e-05 2024-09-20T22:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.438540+0000", 2024-09-20T22:08:24.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94777446200000004 2024-09-20T22:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.438653+0000", 2024-09-20T22:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112793 2024-09-20T22:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.442289+0000", 2024-09-20T22:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003636168 2024-09-20T22:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:24.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.447999+0000", 2024-09-20T22:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005709591 2024-09-20T22:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.448032+0000", 2024-09-20T22:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3626e-05 2024-09-20T22:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.467952+0000", 2024-09-20T22:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.019919849 2024-09-20T22:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.468006+0000", 2024-09-20T22:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4296999999999999e-05 2024-09-20T22:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.468096+0000", 2024-09-20T22:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9680999999999997e-05 2024-09-20T22:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:24.020 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:08:24.021 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:24.062 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:24.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:24.063 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:24.137 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 27 16 54750 54734 126.674 88.125 0.00776588 0.00788851 2024-09-20T22:08:24.141 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:08:24.165 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:24.205 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:24.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:24.206 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:24.307 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:24.347 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:24.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:24.348 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:24.448 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:24.479 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:24.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:24.480 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:24.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:24 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1225296798' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:08:24.520 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:24.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:24 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1225296798' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:08:24.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:24 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1225296798' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:08:24.581 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:24.616 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:24.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:24.617 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:24.718 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:24.749 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:24.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:24.750 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:24.768 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:08:24.850 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:24.883 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:24.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:24.884 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:24.984 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:25.030 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:25.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:25.031 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:25.131 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:25.138 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 28 16 56259 56243 125.512 94.3125 0.0140576 0.00796191 2024-09-20T22:08:25.171 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:25.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:25.172 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:25.273 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:25.309 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:25.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:25.311 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:25.328 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:08:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:08:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:08:25.329 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:25 smithi060 ceph-mon[29694]: pgmap v270: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-20T22:08:25.329 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:25 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:08:25.329 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:25 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1225296798' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:08:25.329 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:25 smithi060 ceph-mon[29694]: osdmap e189: 8 total, 8 up, 8 in 2024-09-20T22:08:25.329 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:25 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:08:25.329 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:25 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:08:25.329 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:25 smithi060 ceph-mon[40290]: pgmap v270: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-20T22:08:25.329 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:25 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:08:25.329 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:25 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1225296798' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:08:25.330 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:25 smithi060 ceph-mon[40290]: osdmap e189: 8 total, 8 up, 8 in 2024-09-20T22:08:25.330 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:25 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:08:25.330 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:25 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:08:25.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T22:08:25.412 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:25.487 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:25.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:25.489 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:25.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:25 smithi067 ceph-mon[33511]: pgmap v270: 148 pgs: 148 active+clean; 3.1 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 84 MiB/s wr, 1.35k op/s 2024-09-20T22:08:25.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:25 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:08:25.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:25 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1225296798' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:08:25.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:25 smithi067 ceph-mon[33511]: osdmap e189: 8 total, 8 up, 8 in 2024-09-20T22:08:25.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:25 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:08:25.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:25 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:08:25.590 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:25.641 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:25.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:25.642 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:25.743 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:25.773 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:25.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:25.774 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:25.875 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:25.910 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:25.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:25.911 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:26.011 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:26.013 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:26.040 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:26.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:26.041 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:26.138 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 29 16 57726 57710 124.346 91.6875 0.00910494 0.00803662 2024-09-20T22:08:26.142 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:26.168 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:26 smithi060 ceph-mon[29694]: osdmap e190: 8 total, 8 up, 8 in 2024-09-20T22:08:26.169 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:26 smithi060 ceph-mon[40290]: osdmap e190: 8 total, 8 up, 8 in 2024-09-20T22:08:26.174 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:26.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:26.175 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:26.276 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:26.305 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:26.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:26.306 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:26.407 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:26.441 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:26.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:26.441 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:26.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:26 smithi067 ceph-mon[33511]: osdmap e190: 8 total, 8 up, 8 in 2024-09-20T22:08:26.542 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:26.571 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:26.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:26.572 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:26.673 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:26.714 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:26.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:26.715 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:26.815 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:26.843 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:26.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:26.844 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:26.945 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:26.990 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:08:26.990 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:08:26.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:26.990 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.15192.0:58938 8.4 e191/176 8:21b28c60:::benchmark_data_smithi060_111051_object58937:head v 191'3632, mlcod=191'3630)", 2024-09-20T22:08:26.990 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:08:26.985558+0000", 2024-09-20T22:08:26.990 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.001740115, 2024-09-20T22:08:26.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001769849, 2024-09-20T22:08:26.991 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:26.991 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:26.991 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:08:26.991 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:26.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:26.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:26.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:26.985558+0000", 2024-09-20T22:08:26.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:26.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:26.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:26.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:26.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:26.985558+0000", 2024-09-20T22:08:26.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:26.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:26.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:26.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:26.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:26.985559+0000", 2024-09-20T22:08:26.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.08e-06 2024-09-20T22:08:26.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:26.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:26.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:26.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:26.985589+0000", 2024-09-20T22:08:26.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0088000000000001e-05 2024-09-20T22:08:26.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:26.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:26.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:26.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:26.985600+0000", 2024-09-20T22:08:26.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1603000000000001e-05 2024-09-20T22:08:26.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:26.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:26.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:26.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:26.985622+0000", 2024-09-20T22:08:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1256000000000001e-05 2024-09-20T22:08:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:26.985730+0000", 2024-09-20T22:08:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108553 2024-09-20T22:08:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:26.985749+0000", 2024-09-20T22:08:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9221000000000001e-05 2024-09-20T22:08:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:26.993 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:26.994 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:08:26.994 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 1 2024-09-20T22:08:26.994 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:08:26.996 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:26.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:26.997 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:27.098 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:27.126 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:27.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:27.127 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:27.138 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 30 16 59128 59112 123.122 87.625 0.013902 0.00811675 2024-09-20T22:08:27.228 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:27.266 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:08:27.268 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:08:27.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:27.269 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:27.269 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:27.370 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:27.400 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:27.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:27.401 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:27.502 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:27.525 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:27 smithi060 ceph-mon[29694]: pgmap v273: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.0 KiB/s rd, 98 MiB/s wr, 1.57k op/s; 67 B/s, 2 objects/s recovering 2024-09-20T22:08:27.526 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:27 smithi060 ceph-mon[29694]: osdmap e191: 8 total, 8 up, 8 in 2024-09-20T22:08:27.526 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:27 smithi060 ceph-mon[40290]: pgmap v273: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.0 KiB/s rd, 98 MiB/s wr, 1.57k op/s; 67 B/s, 2 objects/s recovering 2024-09-20T22:08:27.526 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:27 smithi060 ceph-mon[40290]: osdmap e191: 8 total, 8 up, 8 in 2024-09-20T22:08:27.542 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:27.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:27.543 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:27.644 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:27.675 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:27.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:27.676 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:27.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:27 smithi067 ceph-mon[33511]: pgmap v273: 148 pgs: 148 active+clean; 3.3 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.0 KiB/s rd, 98 MiB/s wr, 1.57k op/s; 67 B/s, 2 objects/s recovering 2024-09-20T22:08:27.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:27 smithi067 ceph-mon[33511]: osdmap e191: 8 total, 8 up, 8 in 2024-09-20T22:08:27.777 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:27.813 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:27.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:27.814 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:27.915 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:27.950 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:27.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:27.951 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:27.976 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:08:28.052 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:28.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:28.098 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:28.098 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:28.138 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 31 16 60535 60519 121.987 87.9375 0.00989635 0.00819178 2024-09-20T22:08:28.199 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:28.227 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:28.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:28.228 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:28.329 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:28.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:28.370 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:28.370 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:28.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:28.510 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:28.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:28.511 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:28.612 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:28.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:28.643 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:28.643 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:28.744 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:28.804 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:28 smithi067 ceph-mon[33511]: 5.f scrub starts 2024-09-20T22:08:28.805 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:28 smithi067 ceph-mon[33511]: 5.f scrub ok 2024-09-20T22:08:28.805 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:28 smithi067 ceph-mon[33511]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:08:28.805 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:28 smithi067 ceph-mon[33511]: 6.9 scrub starts 2024-09-20T22:08:28.805 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:28 smithi067 ceph-mon[33511]: 6.9 scrub ok 2024-09-20T22:08:28.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:28 smithi060 ceph-mon[29694]: 5.f scrub starts 2024-09-20T22:08:28.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:28 smithi060 ceph-mon[29694]: 5.f scrub ok 2024-09-20T22:08:28.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:28 smithi060 ceph-mon[29694]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:08:28.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:28 smithi060 ceph-mon[29694]: 6.9 scrub starts 2024-09-20T22:08:28.805 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:28 smithi060 ceph-mon[29694]: 6.9 scrub ok 2024-09-20T22:08:28.806 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:28 smithi060 ceph-mon[40290]: 5.f scrub starts 2024-09-20T22:08:28.806 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:28 smithi060 ceph-mon[40290]: 5.f scrub ok 2024-09-20T22:08:28.806 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:28 smithi060 ceph-mon[40290]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:08:28.806 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:28 smithi060 ceph-mon[40290]: 6.9 scrub starts 2024-09-20T22:08:28.806 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:28 smithi060 ceph-mon[40290]: 6.9 scrub ok 2024-09-20T22:08:28.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:28.843 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:28.944 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:28.984 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:28.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:28.985 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:29.086 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:29.120 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:29.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:29.121 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:29.138 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 32 15 61967 61952 120.973 89.5625 0.0115267 0.00826135 2024-09-20T22:08:29.222 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:29.262 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:29.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:29.263 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:29.364 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:29.401 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:29.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:29.402 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:29.492 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T22:08:29.502 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:29.583 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:29.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:29.584 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:29.685 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:29.716 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:29.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:29.717 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:29.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:29 smithi067 ceph-mon[33511]: pgmap v275: 148 pgs: 148 active+clean; 3.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 867 B/s rd, 99 MiB/s wr, 1.58k op/s; 55 B/s, 2 objects/s recovering 2024-09-20T22:08:29.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:29 smithi067 ceph-mon[33511]: 3.1a scrub starts 2024-09-20T22:08:29.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:29 smithi067 ceph-mon[33511]: 3.1a scrub ok 2024-09-20T22:08:29.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:29 smithi067 ceph-mon[33511]: 3.11 scrub starts 2024-09-20T22:08:29.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:29 smithi067 ceph-mon[33511]: 3.11 scrub ok 2024-09-20T22:08:29.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:29 smithi067 ceph-mon[33511]: 5.3 scrub starts 2024-09-20T22:08:29.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:29 smithi067 ceph-mon[33511]: 5.3 scrub ok 2024-09-20T22:08:29.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:29 smithi060 ceph-mon[29694]: pgmap v275: 148 pgs: 148 active+clean; 3.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 867 B/s rd, 99 MiB/s wr, 1.58k op/s; 55 B/s, 2 objects/s recovering 2024-09-20T22:08:29.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:29 smithi060 ceph-mon[29694]: 3.1a scrub starts 2024-09-20T22:08:29.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:29 smithi060 ceph-mon[29694]: 3.1a scrub ok 2024-09-20T22:08:29.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:29 smithi060 ceph-mon[29694]: 3.11 scrub starts 2024-09-20T22:08:29.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:29 smithi060 ceph-mon[29694]: 3.11 scrub ok 2024-09-20T22:08:29.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:29 smithi060 ceph-mon[29694]: 5.3 scrub starts 2024-09-20T22:08:29.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:29 smithi060 ceph-mon[29694]: 5.3 scrub ok 2024-09-20T22:08:29.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:29 smithi060 ceph-mon[40290]: pgmap v275: 148 pgs: 148 active+clean; 3.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 867 B/s rd, 99 MiB/s wr, 1.58k op/s; 55 B/s, 2 objects/s recovering 2024-09-20T22:08:29.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:29 smithi060 ceph-mon[40290]: 3.1a scrub starts 2024-09-20T22:08:29.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:29 smithi060 ceph-mon[40290]: 3.1a scrub ok 2024-09-20T22:08:29.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:29 smithi060 ceph-mon[40290]: 3.11 scrub starts 2024-09-20T22:08:29.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:29 smithi060 ceph-mon[40290]: 3.11 scrub ok 2024-09-20T22:08:29.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:29 smithi060 ceph-mon[40290]: 5.3 scrub starts 2024-09-20T22:08:29.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:29 smithi060 ceph-mon[40290]: 5.3 scrub ok 2024-09-20T22:08:29.818 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:29.850 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:29.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:29.851 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:29.952 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:29.968 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:30.008 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:30.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:30.009 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:30.110 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:30.139 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 33 16 63305 63289 119.838 83.5625 0.0104542 0.00833939 2024-09-20T22:08:30.158 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:30.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:30.159 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:30.260 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:30.293 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:30.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:30.294 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:30.395 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:30.436 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:30.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:30.437 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:30.538 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:30.577 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-09-20T22:08:26.391424+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3686013467,"num_objects":56434,"num_object_clones":0,"num_object_copies":169302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56434,"num_whiteouts":0,"num_read":1251,"num_read_kb":1097,"num_write":56850,"num_write_kb":3601501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1771434,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56848,"ondisk_log_size":56848,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":447,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":12516328,"kb_used_data":10742540,"kb_used_omap":15,"kb_used_meta":1773552,"kb_avail":737444888,"statfs":{"total":767960285184,"available":755143565312,"internally_reserved":0,"allocated":11000360960,"data_stored":10993942087,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":1816118035},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[9,1,3,7,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":492896256,"num_objects":7521,"num_object_clones":0,"num_object_copies":22563,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7521,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":7521,"num_write_kb":481344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.367315"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":280,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154163+0000","last_change":"2024-09-20T22:06:48.018571+0000","last_active":"2024-09-20T22:08:25.154163+0000","last_peered":"2024-09-20T22:08:25.154163+0000","last_clean":"2024-09-20T22:08:25.154163+0000","last_became_active":"2024-09-20T22:02:50.109930+0000","last_became_peered":"2024-09-20T22:02:50.109930+0000","last_unstale":"2024-09-20T22:08:25.154163+0000","last_undegraded":"2024-09-20T22:08:25.154163+0000","last_fullsized":"2024-09-20T22:08:25.154163+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_clean_scrub_stamp":"2024-09-20T22:06:48.018157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"71'9","reported_seq":302,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.150917+0000","last_change":"2024-09-20T22:07:55.492953+0000","last_active":"2024-09-20T22:08:25.150917+0000","last_peered":"2024-09-20T22:08:25.150917+0000","last_clean":"2024-09-20T22:08:25.150917+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:08:25.150917+0000","last_undegraded":"2024-09-20T22:08:25.150917+0000","last_fullsized":"2024-09-20T22:08:25.150917+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:55.492841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.492841+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:24.417494+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":157,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.138354+0000","last_change":"2024-09-20T22:06:54.843328+0000","last_active":"2024-09-20T22:08:22.138354+0000","last_peered":"2024-09-20T22:08:22.138354+0000","last_clean":"2024-09-20T22:08:22.138354+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:08:22.138354+0000","last_undegraded":"2024-09-20T22:08:22.138354+0000","last_fullsized":"2024-09-20T22:08:22.138354+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:11.178591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:11.178591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":270,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.151817+0000","last_change":"2024-09-20T22:07:14.535725+0000","last_active":"2024-09-20T22:08:25.151817+0000","last_peered":"2024-09-20T22:08:25.151817+0000","last_clean":"2024-09-20T22:08:25.151817+0000","last_became_active":"2024-09-20T22:06:54.830769+0000","last_became_peered":"2024-09-20T22:06:54.830769+0000","last_unstale":"2024-09-20T22:08:25.151817+0000","last_undegraded":"2024-09-20T22:08:25.151817+0000","last_fullsized":"2024-09-20T22:08:25.151817+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.535618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.535618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:36.228377+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":261,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.159615+0000","last_change":"2024-09-20T22:05:06.024320+0000","last_active":"2024-09-20T22:08:25.159615+0000","last_peered":"2024-09-20T22:08:25.159615+0000","last_clean":"2024-09-20T22:08:25.159615+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:08:25.159615+0000","last_undegraded":"2024-09-20T22:08:25.159615+0000","last_fullsized":"2024-09-20T22:08:25.159615+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:05:06.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:06.024051+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":272,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.160185+0000","last_change":"2024-09-20T22:05:04.976359+0000","last_active":"2024-09-20T22:08:25.160185+0000","last_peered":"2024-09-20T22:08:25.160185+0000","last_clean":"2024-09-20T22:08:25.160185+0000","last_became_active":"2024-09-20T22:02:50.107154+0000","last_became_peered":"2024-09-20T22:02:50.107154+0000","last_unstale":"2024-09-20T22:08:25.160185+0000","last_undegraded":"2024-09-20T22:08:25.160185+0000","last_fullsized":"2024-09-20T22:08:25.160185+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:04.976218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:04.976218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"71'15","reported_seq":293,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.159235+0000","last_change":"2024-09-20T22:04:15.767473+0000","last_active":"2024-09-20T22:08:25.159235+0000","last_peered":"2024-09-20T22:08:25.159235+0000","last_clean":"2024-09-20T22:08:25.159235+0000","last_became_active":"2024-09-20T22:02:52.282101+0000","last_became_peered":"2024-09-20T22:02:52.282101+0000","last_unstale":"2024-09-20T22:08:25.159235+0000","last_undegraded":"2024-09-20T22:08:25.159235+0000","last_fullsized":"2024-09-20T22:08:25.159235+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:04:15.767389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:15.767389+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":274,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.150822+0000","last_change":"2024-09-20T22:07:17.494965+0000","last_active":"2024-09-20T22:08:25.150822+0000","last_peered":"2024-09-20T22:08:25.150822+0000","last_clean":"2024-09-20T22:08:25.150822+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:08:25.150822+0000","last_undegraded":"2024-09-20T22:08:25.150822+0000","last_fullsized":"2024-09-20T22:08:25.150822+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:17.494852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:17.494852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:40.846112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":288,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154701+0000","last_change":"2024-09-20T22:06:46.616824+0000","last_active":"2024-09-20T22:08:25.154701+0000","last_peered":"2024-09-20T22:08:25.154701+0000","last_clean":"2024-09-20T22:08:25.154701+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:08:25.154701+0000","last_undegraded":"2024-09-20T22:08:25.154701+0000","last_fullsized":"2024-09-20T22:08:25.154701+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:46.616725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:46.616725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:26.085900+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"71'9","reported_seq":293,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.152074+0000","last_change":"2024-09-20T22:06:09.729638+0000","last_active":"2024-09-20T22:08:25.152074+0000","last_peered":"2024-09-20T22:08:25.152074+0000","last_clean":"2024-09-20T22:08:25.152074+0000","last_became_active":"2024-09-20T22:02:52.279526+0000","last_became_peered":"2024-09-20T22:02:52.279526+0000","last_unstale":"2024-09-20T22:08:25.152074+0000","last_undegraded":"2024-09-20T22:08:25.152074+0000","last_fullsized":"2024-09-20T22:08:25.152074+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:06:09.729513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:09.729513+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547520+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":280,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154101+0000","last_change":"2024-09-20T22:07:14.890410+0000","last_active":"2024-09-20T22:08:25.154101+0000","last_peered":"2024-09-20T22:08:25.154101+0000","last_clean":"2024-09-20T22:08:25.154101+0000","last_became_active":"2024-09-20T22:02:54.292911+0000","last_became_peered":"2024-09-20T22:02:54.292911+0000","last_unstale":"2024-09-20T22:08:25.154101+0000","last_undegraded":"2024-09-20T22:08:25.154101+0000","last_fullsized":"2024-09-20T22:08:25.154101+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.890308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.890308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:29.100670+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":269,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.153070+0000","last_change":"2024-09-20T22:07:15.416312+0000","last_active":"2024-09-20T22:08:25.153070+0000","last_peered":"2024-09-20T22:08:25.153070+0000","last_clean":"2024-09-20T22:08:25.153070+0000","last_became_active":"2024-09-20T22:06:54.841095+0000","last_became_peered":"2024-09-20T22:06:54.841095+0000","last_unstale":"2024-09-20T22:08:25.153070+0000","last_undegraded":"2024-09-20T22:08:25.153070+0000","last_fullsized":"2024-09-20T22:08:25.153070+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:15.416176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.416176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":301,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154491+0000","last_change":"2024-09-20T22:07:59.947914+0000","last_active":"2024-09-20T22:08:25.154491+0000","last_peered":"2024-09-20T22:08:25.154491+0000","last_clean":"2024-09-20T22:08:25.154491+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:08:25.154491+0000","last_undegraded":"2024-09-20T22:08:25.154491+0000","last_fullsized":"2024-09-20T22:08:25.154491+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:59.947741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:59.947741+0000","last_clean_scrub_stamp":"2024-09-20T22:07:59.947741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:29.533391+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"71'5","reported_seq":308,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.152228+0000","last_change":"2024-09-20T22:08:22.151586+0000","last_active":"2024-09-20T22:08:25.152228+0000","last_peered":"2024-09-20T22:08:25.152228+0000","last_clean":"2024-09-20T22:08:25.152228+0000","last_became_active":"2024-09-20T22:08:22.144364+0000","last_became_peered":"2024-09-20T22:08:22.144364+0000","last_unstale":"2024-09-20T22:08:25.152228+0000","last_undegraded":"2024-09-20T22:08:25.152228+0000","last_fullsized":"2024-09-20T22:08:25.152228+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'5","last_scrub_stamp":"2024-09-20T22:08:01.462581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:01.462581+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:24.969371+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":282,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.156730+0000","last_change":"2024-09-20T22:07:28.654301+0000","last_active":"2024-09-20T22:08:25.156730+0000","last_peered":"2024-09-20T22:08:25.156730+0000","last_clean":"2024-09-20T22:08:25.156730+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:08:25.156730+0000","last_undegraded":"2024-09-20T22:08:25.156730+0000","last_fullsized":"2024-09-20T22:08:25.156730+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:28.654202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:28.654202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:41.331246+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":214,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.153753+0000","last_change":"2024-09-20T22:06:55.824661+0000","last_active":"2024-09-20T22:08:25.153753+0000","last_peered":"2024-09-20T22:08:25.153753+0000","last_clean":"2024-09-20T22:08:25.153753+0000","last_became_active":"2024-09-20T22:06:55.824168+0000","last_became_peered":"2024-09-20T22:06:55.824168+0000","last_unstale":"2024-09-20T22:08:25.153753+0000","last_undegraded":"2024-09-20T22:08:25.153753+0000","last_fullsized":"2024-09-20T22:08:25.153753+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:05:29.551172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":269,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.152852+0000","last_change":"2024-09-20T22:06:47.667293+0000","last_active":"2024-09-20T22:08:25.152852+0000","last_peered":"2024-09-20T22:08:25.152852+0000","last_clean":"2024-09-20T22:08:25.152852+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:08:25.152852+0000","last_undegraded":"2024-09-20T22:08:25.152852+0000","last_fullsized":"2024-09-20T22:08:25.152852+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:06:47.667180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547546+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":272,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154658+0000","last_change":"2024-09-20T22:05:31.552599+0000","last_active":"2024-09-20T22:08:25.154658+0000","last_peered":"2024-09-20T22:08:25.154658+0000","last_clean":"2024-09-20T22:08:25.154658+0000","last_became_active":"2024-09-20T22:02:50.111517+0000","last_became_peered":"2024-09-20T22:02:50.111517+0000","last_unstale":"2024-09-20T22:08:25.154658+0000","last_undegraded":"2024-09-20T22:08:25.154658+0000","last_fullsized":"2024-09-20T22:08:25.154658+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:31.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:31.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"71'15","reported_seq":310,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.155710+0000","last_change":"2024-09-20T22:07:16.215766+0000","last_active":"2024-09-20T22:08:25.155710+0000","last_peered":"2024-09-20T22:08:25.155710+0000","last_clean":"2024-09-20T22:08:25.155710+0000","last_became_active":"2024-09-20T22:06:54.846111+0000","last_became_peered":"2024-09-20T22:06:54.846111+0000","last_unstale":"2024-09-20T22:08:25.155710+0000","last_undegraded":"2024-09-20T22:08:25.155710+0000","last_fullsized":"2024-09-20T22:08:25.155710+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:07:16.215638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:16.215638+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":114,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.131503+0000","last_change":"2024-09-20T22:08:00.369883+0000","last_active":"2024-09-20T22:08:22.131503+0000","last_peered":"2024-09-20T22:08:22.131503+0000","last_clean":"2024-09-20T22:08:22.131503+0000","last_became_active":"2024-09-20T22:06:54.848505+0000","last_became_peered":"2024-09-20T22:06:54.848505+0000","last_unstale":"2024-09-20T22:08:22.131503+0000","last_undegraded":"2024-09-20T22:08:22.131503+0000","last_fullsized":"2024-09-20T22:08:22.131503+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:00.369767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:00.369767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:20.935739+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":268,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.160194+0000","last_change":"2024-09-20T22:06:34.944093+0000","last_active":"2024-09-20T22:08:25.160194+0000","last_peered":"2024-09-20T22:08:25.160194+0000","last_clean":"2024-09-20T22:08:25.160194+0000","last_became_active":"2024-09-20T22:02:56.347561+0000","last_became_peered":"2024-09-20T22:02:56.347561+0000","last_unstale":"2024-09-20T22:08:25.160194+0000","last_undegraded":"2024-09-20T22:08:25.160194+0000","last_fullsized":"2024-09-20T22:08:25.160194+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:34.940758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:34.940758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":222,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154278+0000","last_change":"2024-09-20T22:06:55.817107+0000","last_active":"2024-09-20T22:08:25.154278+0000","last_peered":"2024-09-20T22:08:25.154278+0000","last_clean":"2024-09-20T22:08:25.154278+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:08:25.154278+0000","last_undegraded":"2024-09-20T22:08:25.154278+0000","last_fullsized":"2024-09-20T22:08:25.154278+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:32.472041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:32.472041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:26.085798+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"71'12","reported_seq":300,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.159880+0000","last_change":"2024-09-20T22:07:55.383903+0000","last_active":"2024-09-20T22:08:25.159880+0000","last_peered":"2024-09-20T22:08:25.159880+0000","last_clean":"2024-09-20T22:08:25.159880+0000","last_became_active":"2024-09-20T22:06:54.830946+0000","last_became_peered":"2024-09-20T22:06:54.830946+0000","last_unstale":"2024-09-20T22:08:25.159880+0000","last_undegraded":"2024-09-20T22:08:25.159880+0000","last_fullsized":"2024-09-20T22:08:25.159880+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:55.383818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.383818+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:56.321505+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":265,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.152439+0000","last_change":"2024-09-20T22:05:28.003302+0000","last_active":"2024-09-20T22:08:25.152439+0000","last_peered":"2024-09-20T22:08:25.152439+0000","last_clean":"2024-09-20T22:08:25.152439+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:08:25.152439+0000","last_undegraded":"2024-09-20T22:08:25.152439+0000","last_fullsized":"2024-09-20T22:08:25.152439+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.003183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547263+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":263,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.157399+0000","last_change":"2024-09-20T22:05:33.751965+0000","last_active":"2024-09-20T22:08:25.157399+0000","last_peered":"2024-09-20T22:08:25.157399+0000","last_clean":"2024-09-20T22:08:25.157399+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:08:25.157399+0000","last_undegraded":"2024-09-20T22:08:25.157399+0000","last_fullsized":"2024-09-20T22:08:25.157399+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:05:33.751883+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"64'1","reported_seq":108,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.138477+0000","last_change":"2024-09-20T22:06:54.854003+0000","last_active":"2024-09-20T22:08:22.138477+0000","last_peered":"2024-09-20T22:08:22.138477+0000","last_clean":"2024-09-20T22:08:22.138477+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:08:22.138477+0000","last_undegraded":"2024-09-20T22:08:22.138477+0000","last_fullsized":"2024-09-20T22:08:22.138477+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"71'10","reported_seq":283,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.155657+0000","last_change":"2024-09-20T22:04:56.414262+0000","last_active":"2024-09-20T22:08:25.155657+0000","last_peered":"2024-09-20T22:08:25.155657+0000","last_clean":"2024-09-20T22:08:25.155657+0000","last_became_active":"2024-09-20T22:02:52.481980+0000","last_became_peered":"2024-09-20T22:02:52.481980+0000","last_unstale":"2024-09-20T22:08:25.155657+0000","last_undegraded":"2024-09-20T22:08:25.155657+0000","last_fullsized":"2024-09-20T22:08:25.155657+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:04:56.414020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:56.414020+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"71'8","reported_seq":351,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154199+0000","last_change":"2024-09-20T22:06:41.394065+0000","last_active":"2024-09-20T22:08:25.154199+0000","last_peered":"2024-09-20T22:08:25.154199+0000","last_clean":"2024-09-20T22:08:25.154199+0000","last_became_active":"2024-09-20T22:02:54.297657+0000","last_became_peered":"2024-09-20T22:02:54.297657+0000","last_unstale":"2024-09-20T22:08:25.154199+0000","last_undegraded":"2024-09-20T22:08:25.154199+0000","last_fullsized":"2024-09-20T22:08:25.154199+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:06:41.393961+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:41.393961+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:28.370169+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"188'3314","reported_seq":3348,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.625951+0000","last_change":"2024-09-20T22:07:52.429150+0000","last_active":"2024-09-20T22:08:22.625951+0000","last_peered":"2024-09-20T22:08:22.625951+0000","last_clean":"2024-09-20T22:08:22.625951+0000","last_became_active":"2024-09-20T22:07:52.428105+0000","last_became_peered":"2024-09-20T22:07:52.428105+0000","last_unstale":"2024-09-20T22:08:22.625951+0000","last_undegraded":"2024-09-20T22:08:22.625951+0000","last_fullsized":"2024-09-20T22:08:22.625951+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3314,"log_dups_size":0,"ondisk_log_size":3314,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:04.623212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":217186304,"num_objects":3314,"num_object_clones":0,"num_object_copies":9942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3314,"num_write_kb":212096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"71'15","reported_seq":31,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.155301+0000","last_change":"2024-09-20T22:08:22.190283+0000","last_active":"2024-09-20T22:08:25.155301+0000","last_peered":"2024-09-20T22:08:25.155301+0000","last_clean":"2024-09-20T22:08:25.155301+0000","last_became_active":"2024-09-20T22:08:22.145408+0000","last_became_peered":"2024-09-20T22:08:22.145408+0000","last_unstale":"2024-09-20T22:08:25.155301+0000","last_undegraded":"2024-09-20T22:08:25.155301+0000","last_fullsized":"2024-09-20T22:08:25.155301+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:04:13.235907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:13.235907+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:28.184828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":281,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.160386+0000","last_change":"2024-09-20T22:07:23.292560+0000","last_active":"2024-09-20T22:08:25.160386+0000","last_peered":"2024-09-20T22:08:25.160386+0000","last_clean":"2024-09-20T22:08:25.160386+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:08:25.160386+0000","last_undegraded":"2024-09-20T22:08:25.160386+0000","last_fullsized":"2024-09-20T22:08:25.160386+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.292451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.292451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:25.739658+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"71'8","reported_seq":352,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.152997+0000","last_change":"2024-09-20T22:06:04.874166+0000","last_active":"2024-09-20T22:08:25.152997+0000","last_peered":"2024-09-20T22:08:25.152997+0000","last_clean":"2024-09-20T22:08:25.152997+0000","last_became_active":"2024-09-20T22:02:54.294917+0000","last_became_peered":"2024-09-20T22:02:54.294917+0000","last_unstale":"2024-09-20T22:08:25.152997+0000","last_undegraded":"2024-09-20T22:08:25.152997+0000","last_fullsized":"2024-09-20T22:08:25.152997+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:06:04.874042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:04.874042+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547486+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":230,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.153647+0000","last_change":"2024-09-20T22:07:52.334831+0000","last_active":"2024-09-20T22:08:25.153647+0000","last_peered":"2024-09-20T22:08:25.153647+0000","last_clean":"2024-09-20T22:08:25.153647+0000","last_became_active":"2024-09-20T22:06:55.822780+0000","last_became_peered":"2024-09-20T22:06:55.822780+0000","last_unstale":"2024-09-20T22:08:25.153647+0000","last_undegraded":"2024-09-20T22:08:25.153647+0000","last_fullsized":"2024-09-20T22:08:25.153647+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:52.334748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:52.334748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:53.481596+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.c","version":"190'3600","reported_seq":3638,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.266318+0000","last_change":"2024-09-20T22:07:52.674862+0000","last_active":"2024-09-20T22:08:26.266318+0000","last_peered":"2024-09-20T22:08:26.266318+0000","last_clean":"2024-09-20T22:08:26.266318+0000","last_became_active":"2024-09-20T22:07:52.674521+0000","last_became_peered":"2024-09-20T22:07:52.674521+0000","last_unstale":"2024-09-20T22:08:26.266318+0000","last_undegraded":"2024-09-20T22:08:26.266318+0000","last_fullsized":"2024-09-20T22:08:26.266318+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3600,"log_dups_size":0,"ondisk_log_size":3600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:14.102427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":235929600,"num_objects":3600,"num_object_clones":0,"num_object_copies":10800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3600,"num_write_kb":230400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"71'18","reported_seq":300,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.160060+0000","last_change":"2024-09-20T22:04:57.967494+0000","last_active":"2024-09-20T22:08:25.160060+0000","last_peered":"2024-09-20T22:08:25.160060+0000","last_clean":"2024-09-20T22:08:25.160060+0000","last_became_active":"2024-09-20T22:02:52.482119+0000","last_became_peered":"2024-09-20T22:02:52.482119+0000","last_unstale":"2024-09-20T22:08:25.160060+0000","last_undegraded":"2024-09-20T22:08:25.160060+0000","last_fullsized":"2024-09-20T22:08:25.160060+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'18","last_scrub_stamp":"2024-09-20T22:04:57.967216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:57.967216+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":272,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.160406+0000","last_change":"2024-09-20T22:05:02.924002+0000","last_active":"2024-09-20T22:08:25.160406+0000","last_peered":"2024-09-20T22:08:25.160406+0000","last_clean":"2024-09-20T22:08:25.160406+0000","last_became_active":"2024-09-20T22:02:50.113076+0000","last_became_peered":"2024-09-20T22:02:50.113076+0000","last_unstale":"2024-09-20T22:08:25.160406+0000","last_undegraded":"2024-09-20T22:08:25.160406+0000","last_fullsized":"2024-09-20T22:08:25.160406+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:02.923565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:02.923565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":273,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.150853+0000","last_change":"2024-09-20T22:06:05.894144+0000","last_active":"2024-09-20T22:08:25.150853+0000","last_peered":"2024-09-20T22:08:25.150853+0000","last_clean":"2024-09-20T22:08:25.150853+0000","last_became_active":"2024-09-20T22:02:54.288007+0000","last_became_peered":"2024-09-20T22:02:54.288007+0000","last_unstale":"2024-09-20T22:08:25.150853+0000","last_undegraded":"2024-09-20T22:08:25.150853+0000","last_fullsized":"2024-09-20T22:08:25.150853+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:05.894031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:05.894031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547501+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":269,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.151996+0000","last_change":"2024-09-20T22:05:29.990030+0000","last_active":"2024-09-20T22:08:25.151996+0000","last_peered":"2024-09-20T22:08:25.151996+0000","last_clean":"2024-09-20T22:08:25.151996+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:08:25.151996+0000","last_undegraded":"2024-09-20T22:08:25.151996+0000","last_fullsized":"2024-09-20T22:08:25.151996+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:29.989934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:29.989934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547290+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"190'3577","reported_seq":3615,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.630274+0000","last_change":"2024-09-20T22:07:52.940943+0000","last_active":"2024-09-20T22:08:25.630274+0000","last_peered":"2024-09-20T22:08:25.630274+0000","last_clean":"2024-09-20T22:08:25.630274+0000","last_became_active":"2024-09-20T22:07:52.940627+0000","last_became_peered":"2024-09-20T22:07:52.940627+0000","last_unstale":"2024-09-20T22:08:25.630274+0000","last_undegraded":"2024-09-20T22:08:25.630274+0000","last_fullsized":"2024-09-20T22:08:25.630274+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3577,"log_dups_size":0,"ondisk_log_size":3577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:11.794238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234422272,"num_objects":3577,"num_object_clones":0,"num_object_copies":10731,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3577,"num_write_kb":228928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"71'14","reported_seq":298,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.151073+0000","last_change":"2024-09-20T22:06:00.944172+0000","last_active":"2024-09-20T22:08:25.151073+0000","last_peered":"2024-09-20T22:08:25.151073+0000","last_clean":"2024-09-20T22:08:25.151073+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:08:25.151073+0000","last_undegraded":"2024-09-20T22:08:25.151073+0000","last_fullsized":"2024-09-20T22:08:25.151073+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'14","last_scrub_stamp":"2024-09-20T22:06:00.944086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.944086+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547461+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":282,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.155907+0000","last_change":"2024-09-20T22:07:58.783500+0000","last_active":"2024-09-20T22:08:25.155907+0000","last_peered":"2024-09-20T22:08:25.155907+0000","last_clean":"2024-09-20T22:08:25.155907+0000","last_became_active":"2024-09-20T22:06:54.841165+0000","last_became_peered":"2024-09-20T22:06:54.841165+0000","last_unstale":"2024-09-20T22:08:25.155907+0000","last_undegraded":"2024-09-20T22:08:25.155907+0000","last_fullsized":"2024-09-20T22:08:25.155907+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:07:58.783380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:58.783380+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:25.086988+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"146'10","reported_seq":351,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.159590+0000","last_change":"2024-09-20T22:07:24.333937+0000","last_active":"2024-09-20T22:08:25.159590+0000","last_peered":"2024-09-20T22:08:25.159590+0000","last_clean":"2024-09-20T22:08:25.159590+0000","last_became_active":"2024-09-20T22:06:54.847742+0000","last_became_peered":"2024-09-20T22:06:54.847742+0000","last_unstale":"2024-09-20T22:08:25.159590+0000","last_undegraded":"2024-09-20T22:08:25.159590+0000","last_fullsized":"2024-09-20T22:08:25.159590+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:07:24.333809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:24.333809+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:46.378931+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":269,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.156968+0000","last_change":"2024-09-20T22:07:30.589937+0000","last_active":"2024-09-20T22:08:25.156968+0000","last_peered":"2024-09-20T22:08:25.156968+0000","last_clean":"2024-09-20T22:08:25.156968+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:08:25.156968+0000","last_undegraded":"2024-09-20T22:08:25.156968+0000","last_fullsized":"2024-09-20T22:08:25.156968+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:30.589849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:30.589849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:57.117488+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"190'3676","reported_seq":3714,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.263404+0000","last_change":"2024-09-20T22:07:52.940978+0000","last_active":"2024-09-20T22:08:26.263404+0000","last_peered":"2024-09-20T22:08:26.263404+0000","last_clean":"2024-09-20T22:08:26.263404+0000","last_became_active":"2024-09-20T22:07:52.940673+0000","last_became_peered":"2024-09-20T22:07:52.940673+0000","last_unstale":"2024-09-20T22:08:26.263404+0000","last_undegraded":"2024-09-20T22:08:26.263404+0000","last_fullsized":"2024-09-20T22:08:26.263404+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3676,"log_dups_size":0,"ondisk_log_size":3676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.115401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240910336,"num_objects":3676,"num_object_clones":0,"num_object_copies":11028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3676,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3676,"num_write_kb":235264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"71'10","reported_seq":141,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.131062+0000","last_change":"2024-09-20T22:07:59.424108+0000","last_active":"2024-09-20T22:08:22.131062+0000","last_peered":"2024-09-20T22:08:22.131062+0000","last_clean":"2024-09-20T22:08:22.131062+0000","last_became_active":"2024-09-20T22:06:54.844681+0000","last_became_peered":"2024-09-20T22:06:54.844681+0000","last_unstale":"2024-09-20T22:08:22.131062+0000","last_undegraded":"2024-09-20T22:08:22.131062+0000","last_fullsized":"2024-09-20T22:08:22.131062+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:07:59.423970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:59.423970+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:05.043224+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":288,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154770+0000","last_change":"2024-09-20T22:06:43.598035+0000","last_active":"2024-09-20T22:08:25.154770+0000","last_peered":"2024-09-20T22:08:25.154770+0000","last_clean":"2024-09-20T22:08:25.154770+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:08:25.154770+0000","last_undegraded":"2024-09-20T22:08:25.154770+0000","last_fullsized":"2024-09-20T22:08:25.154770+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:06:43.597924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:26.085822+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"71'8","reported_seq":340,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.153503+0000","last_change":"2024-09-20T22:04:20.160515+0000","last_active":"2024-09-20T22:08:25.153503+0000","last_peered":"2024-09-20T22:08:25.153503+0000","last_clean":"2024-09-20T22:08:25.153503+0000","last_became_active":"2024-09-20T22:02:54.292819+0000","last_became_peered":"2024-09-20T22:02:54.292819+0000","last_unstale":"2024-09-20T22:08:25.153503+0000","last_undegraded":"2024-09-20T22:08:25.153503+0000","last_fullsized":"2024-09-20T22:08:25.153503+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:04:20.160405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:20.160405+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":260,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.153212+0000","last_change":"2024-09-20T22:06:00.572347+0000","last_active":"2024-09-20T22:08:25.153212+0000","last_peered":"2024-09-20T22:08:25.153212+0000","last_clean":"2024-09-20T22:08:25.153212+0000","last_became_active":"2024-09-20T22:02:56.304274+0000","last_became_peered":"2024-09-20T22:02:56.304274+0000","last_unstale":"2024-09-20T22:08:25.153212+0000","last_undegraded":"2024-09-20T22:08:25.153212+0000","last_fullsized":"2024-09-20T22:08:25.153212+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:00.572243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.572243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"190'3674","reported_seq":3712,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.130976+0000","last_change":"2024-09-20T22:07:52.940027+0000","last_active":"2024-09-20T22:08:26.130976+0000","last_peered":"2024-09-20T22:08:26.130976+0000","last_clean":"2024-09-20T22:08:26.130976+0000","last_became_active":"2024-09-20T22:07:52.939692+0000","last_became_peered":"2024-09-20T22:07:52.939692+0000","last_unstale":"2024-09-20T22:08:26.130976+0000","last_undegraded":"2024-09-20T22:08:26.130976+0000","last_fullsized":"2024-09-20T22:08:26.130976+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3674,"log_dups_size":0,"ondisk_log_size":3674,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:12.848256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240779264,"num_objects":3674,"num_object_clones":0,"num_object_copies":11022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3674,"num_write_kb":235136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"71'19","reported_seq":312,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.153655+0000","last_change":"2024-09-20T22:05:24.600933+0000","last_active":"2024-09-20T22:08:25.153655+0000","last_peered":"2024-09-20T22:08:25.153655+0000","last_clean":"2024-09-20T22:08:25.153655+0000","last_became_active":"2024-09-20T22:02:52.480077+0000","last_became_peered":"2024-09-20T22:02:52.480077+0000","last_unstale":"2024-09-20T22:08:25.153655+0000","last_undegraded":"2024-09-20T22:08:25.153655+0000","last_fullsized":"2024-09-20T22:08:25.153655+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:05:24.600795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:24.600795+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":115,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.130973+0000","last_change":"2024-09-20T22:07:14.382822+0000","last_active":"2024-09-20T22:08:22.130973+0000","last_peered":"2024-09-20T22:08:22.130973+0000","last_clean":"2024-09-20T22:08:22.130973+0000","last_became_active":"2024-09-20T22:06:54.829037+0000","last_became_peered":"2024-09-20T22:06:54.829037+0000","last_unstale":"2024-09-20T22:08:22.130973+0000","last_undegraded":"2024-09-20T22:08:22.130973+0000","last_fullsized":"2024-09-20T22:08:22.130973+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.382712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.382712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.271274+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":280,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.153360+0000","last_change":"2024-09-20T22:07:30.169138+0000","last_active":"2024-09-20T22:08:25.153360+0000","last_peered":"2024-09-20T22:08:25.153360+0000","last_clean":"2024-09-20T22:08:25.153360+0000","last_became_active":"2024-09-20T22:02:54.290787+0000","last_became_peered":"2024-09-20T22:02:54.290787+0000","last_unstale":"2024-09-20T22:08:25.153360+0000","last_undegraded":"2024-09-20T22:08:25.153360+0000","last_fullsized":"2024-09-20T22:08:25.153360+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:30.169038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:30.169038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:38.467894+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":106,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.132078+0000","last_change":"2024-09-20T22:06:54.820385+0000","last_active":"2024-09-20T22:08:22.132078+0000","last_peered":"2024-09-20T22:08:22.132078+0000","last_clean":"2024-09-20T22:08:22.132078+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:08:22.132078+0000","last_undegraded":"2024-09-20T22:08:22.132078+0000","last_fullsized":"2024-09-20T22:08:22.132078+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"190'3590","reported_seq":3628,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.617825+0000","last_change":"2024-09-20T22:07:52.674766+0000","last_active":"2024-09-20T22:08:25.617825+0000","last_peered":"2024-09-20T22:08:25.617825+0000","last_clean":"2024-09-20T22:08:25.617825+0000","last_became_active":"2024-09-20T22:07:52.674363+0000","last_became_peered":"2024-09-20T22:07:52.674363+0000","last_unstale":"2024-09-20T22:08:25.617825+0000","last_undegraded":"2024-09-20T22:08:25.617825+0000","last_fullsized":"2024-09-20T22:08:25.617825+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3590,"log_dups_size":0,"ondisk_log_size":3590,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:16.336670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":235274240,"num_objects":3590,"num_object_clones":0,"num_object_copies":10770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3590,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3590,"num_write_kb":229760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"146'30","reported_seq":194,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.131109+0000","last_change":"2024-09-20T22:07:55.357829+0000","last_active":"2024-09-20T22:08:22.131109+0000","last_peered":"2024-09-20T22:08:22.131109+0000","last_clean":"2024-09-20T22:08:22.131109+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:08:22.131109+0000","last_undegraded":"2024-09-20T22:08:22.131109+0000","last_fullsized":"2024-09-20T22:08:22.131109+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'30","last_scrub_stamp":"2024-09-20T22:07:55.357726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.357726+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:10.172184+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":281,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.152663+0000","last_change":"2024-09-20T22:05:58.966616+0000","last_active":"2024-09-20T22:08:25.152663+0000","last_peered":"2024-09-20T22:08:25.152663+0000","last_clean":"2024-09-20T22:08:25.152663+0000","last_became_active":"2024-09-20T22:02:50.108457+0000","last_became_peered":"2024-09-20T22:02:50.108457+0000","last_unstale":"2024-09-20T22:08:25.152663+0000","last_undegraded":"2024-09-20T22:08:25.152663+0000","last_fullsized":"2024-09-20T22:08:25.152663+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:58.966541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:58.966541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547423+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":292,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154648+0000","last_change":"2024-09-20T22:07:21.886172+0000","last_active":"2024-09-20T22:08:25.154648+0000","last_peered":"2024-09-20T22:08:25.154648+0000","last_clean":"2024-09-20T22:08:25.154648+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:08:25.154648+0000","last_undegraded":"2024-09-20T22:08:25.154648+0000","last_fullsized":"2024-09-20T22:08:25.154648+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_clean_scrub_stamp":"2024-09-20T22:07:21.886038+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.052265+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":273,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.153761+0000","last_change":"2024-09-20T22:07:17.566774+0000","last_active":"2024-09-20T22:08:25.153761+0000","last_peered":"2024-09-20T22:08:25.153761+0000","last_clean":"2024-09-20T22:08:25.153761+0000","last_became_active":"2024-09-20T22:06:54.841419+0000","last_became_peered":"2024-09-20T22:06:54.841419+0000","last_unstale":"2024-09-20T22:08:25.153761+0000","last_undegraded":"2024-09-20T22:08:25.153761+0000","last_fullsized":"2024-09-20T22:08:25.153761+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:17.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:17.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:32.923210+0000","scrub_duration":120,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":260,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.160067+0000","last_change":"2024-09-20T22:05:34.954357+0000","last_active":"2024-09-20T22:08:25.160067+0000","last_peered":"2024-09-20T22:08:25.160067+0000","last_clean":"2024-09-20T22:08:25.160067+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:08:25.160067+0000","last_undegraded":"2024-09-20T22:08:25.160067+0000","last_fullsized":"2024-09-20T22:08:25.160067+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:05:34.953944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"190'3557","reported_seq":3595,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.141081+0000","last_change":"2024-09-20T22:07:52.429893+0000","last_active":"2024-09-20T22:08:26.141081+0000","last_peered":"2024-09-20T22:08:26.141081+0000","last_clean":"2024-09-20T22:08:26.141081+0000","last_became_active":"2024-09-20T22:07:52.429554+0000","last_became_peered":"2024-09-20T22:07:52.429554+0000","last_unstale":"2024-09-20T22:08:26.141081+0000","last_undegraded":"2024-09-20T22:08:26.141081+0000","last_fullsized":"2024-09-20T22:08:26.141081+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3557,"log_dups_size":0,"ondisk_log_size":3557,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:20.427594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233111552,"num_objects":3557,"num_object_clones":0,"num_object_copies":10671,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3557,"num_write_kb":227648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"71'13","reported_seq":145,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.135306+0000","last_change":"2024-09-20T22:06:54.851246+0000","last_active":"2024-09-20T22:08:22.135306+0000","last_peered":"2024-09-20T22:08:22.135306+0000","last_clean":"2024-09-20T22:08:22.135306+0000","last_became_active":"2024-09-20T22:06:54.850696+0000","last_became_peered":"2024-09-20T22:06:54.850696+0000","last_unstale":"2024-09-20T22:08:22.135306+0000","last_undegraded":"2024-09-20T22:08:22.135306+0000","last_fullsized":"2024-09-20T22:08:22.135306+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":115,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.131416+0000","last_change":"2024-09-20T22:07:20.341524+0000","last_active":"2024-09-20T22:08:22.131416+0000","last_peered":"2024-09-20T22:08:22.131416+0000","last_clean":"2024-09-20T22:08:22.131416+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:08:22.131416+0000","last_undegraded":"2024-09-20T22:08:22.131416+0000","last_fullsized":"2024-09-20T22:08:22.131416+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:20.338388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:20.338388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:40.900548+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":298,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.155501+0000","last_change":"2024-09-20T22:06:47.015379+0000","last_active":"2024-09-20T22:08:25.155501+0000","last_peered":"2024-09-20T22:08:25.155501+0000","last_clean":"2024-09-20T22:08:25.155501+0000","last_became_active":"2024-09-20T22:02:49.099508+0000","last_became_peered":"2024-09-20T22:02:49.099508+0000","last_unstale":"2024-09-20T22:08:25.155501+0000","last_undegraded":"2024-09-20T22:08:25.155501+0000","last_fullsized":"2024-09-20T22:08:25.155501+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:06:47.015270+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:34.967504+0000","last_clean_scrub_stamp":"2024-09-20T22:06:47.015270+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:28.184890+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":280,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.156084+0000","last_change":"2024-09-20T22:08:02.168865+0000","last_active":"2024-09-20T22:08:25.156084+0000","last_peered":"2024-09-20T22:08:25.156084+0000","last_clean":"2024-09-20T22:08:25.156084+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:08:25.156084+0000","last_undegraded":"2024-09-20T22:08:25.156084+0000","last_fullsized":"2024-09-20T22:08:25.156084+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:08:02.167030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:15.048786+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":261,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.158026+0000","last_change":"2024-09-20T22:05:28.780920+0000","last_active":"2024-09-20T22:08:25.158026+0000","last_peered":"2024-09-20T22:08:25.158026+0000","last_clean":"2024-09-20T22:08:25.158026+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:08:25.158026+0000","last_undegraded":"2024-09-20T22:08:25.158026+0000","last_fullsized":"2024-09-20T22:08:25.158026+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.780758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.780758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"188'3297","reported_seq":3331,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.638535+0000","last_change":"2024-09-20T22:07:52.939933+0000","last_active":"2024-09-20T22:08:22.638535+0000","last_peered":"2024-09-20T22:08:22.638535+0000","last_clean":"2024-09-20T22:08:22.638535+0000","last_became_active":"2024-09-20T22:07:52.939626+0000","last_became_peered":"2024-09-20T22:07:52.939626+0000","last_unstale":"2024-09-20T22:08:22.638535+0000","last_undegraded":"2024-09-20T22:08:22.638535+0000","last_fullsized":"2024-09-20T22:08:22.638535+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3297,"log_dups_size":0,"ondisk_log_size":3297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:10.985426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":216072192,"num_objects":3297,"num_object_clones":0,"num_object_copies":9891,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3297,"num_write_kb":211008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"71'12","reported_seq":295,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.155763+0000","last_change":"2024-09-20T22:07:29.614630+0000","last_active":"2024-09-20T22:08:25.155763+0000","last_peered":"2024-09-20T22:08:25.155763+0000","last_clean":"2024-09-20T22:08:25.155763+0000","last_became_active":"2024-09-20T22:06:54.848670+0000","last_became_peered":"2024-09-20T22:06:54.848670+0000","last_unstale":"2024-09-20T22:08:25.155763+0000","last_undegraded":"2024-09-20T22:08:25.155763+0000","last_fullsized":"2024-09-20T22:08:25.155763+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:29.614551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:29.614551+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:50.884056+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":280,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.153979+0000","last_change":"2024-09-20T22:06:44.862452+0000","last_active":"2024-09-20T22:08:25.153979+0000","last_peered":"2024-09-20T22:08:25.153979+0000","last_clean":"2024-09-20T22:08:25.153979+0000","last_became_active":"2024-09-20T22:02:50.110448+0000","last_became_peered":"2024-09-20T22:02:50.110448+0000","last_unstale":"2024-09-20T22:08:25.153979+0000","last_undegraded":"2024-09-20T22:08:25.153979+0000","last_fullsized":"2024-09-20T22:08:25.153979+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:06:44.862338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":621,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.158623+0000","last_change":"2024-09-20T22:07:16.590582+0000","last_active":"2024-09-20T22:08:25.158623+0000","last_peered":"2024-09-20T22:08:25.158623+0000","last_clean":"2024-09-20T22:08:25.158623+0000","last_became_active":"2024-09-20T22:06:54.844454+0000","last_became_peered":"2024-09-20T22:06:54.844454+0000","last_unstale":"2024-09-20T22:08:25.158623+0000","last_undegraded":"2024-09-20T22:08:25.158623+0000","last_fullsized":"2024-09-20T22:08:25.158623+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:07:16.590475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:07:16.590475+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.092001+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":322,"num_read_kb":317,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":273,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154576+0000","last_change":"2024-09-20T22:07:22.884981+0000","last_active":"2024-09-20T22:08:25.154576+0000","last_peered":"2024-09-20T22:08:25.154576+0000","last_clean":"2024-09-20T22:08:25.154576+0000","last_became_active":"2024-09-20T22:06:54.841596+0000","last_became_peered":"2024-09-20T22:06:54.841596+0000","last_unstale":"2024-09-20T22:08:25.154576+0000","last_undegraded":"2024-09-20T22:08:25.154576+0000","last_fullsized":"2024-09-20T22:08:25.154576+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:22.884907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.884907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:41.462917+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":106,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.130790+0000","last_change":"2024-09-20T22:06:54.853957+0000","last_active":"2024-09-20T22:08:22.130790+0000","last_peered":"2024-09-20T22:08:22.130790+0000","last_clean":"2024-09-20T22:08:22.130790+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:08:22.130790+0000","last_undegraded":"2024-09-20T22:08:22.130790+0000","last_fullsized":"2024-09-20T22:08:22.130790+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"190'3580","reported_seq":3617,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.723155+0000","last_change":"2024-09-20T22:07:52.430313+0000","last_active":"2024-09-20T22:08:25.723155+0000","last_peered":"2024-09-20T22:08:25.723155+0000","last_clean":"2024-09-20T22:08:25.723155+0000","last_became_active":"2024-09-20T22:07:52.430017+0000","last_became_peered":"2024-09-20T22:07:52.430017+0000","last_unstale":"2024-09-20T22:08:25.723155+0000","last_undegraded":"2024-09-20T22:08:25.723155+0000","last_fullsized":"2024-09-20T22:08:25.723155+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3580,"log_dups_size":0,"ondisk_log_size":3580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:04.804126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234618880,"num_objects":3580,"num_object_clones":0,"num_object_copies":10740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3580,"num_write_kb":229120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"71'16","reported_seq":374,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154216+0000","last_change":"2024-09-20T22:07:53.297530+0000","last_active":"2024-09-20T22:08:25.154216+0000","last_peered":"2024-09-20T22:08:25.154216+0000","last_clean":"2024-09-20T22:08:25.154216+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:08:25.154216+0000","last_undegraded":"2024-09-20T22:08:25.154216+0000","last_fullsized":"2024-09-20T22:08:25.154216+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'16","last_scrub_stamp":"2024-09-20T22:07:53.297442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.297442+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:20.417428+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":280,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.159564+0000","last_change":"2024-09-20T22:06:48.105501+0000","last_active":"2024-09-20T22:08:25.159564+0000","last_peered":"2024-09-20T22:08:25.159564+0000","last_clean":"2024-09-20T22:08:25.159564+0000","last_became_active":"2024-09-20T22:02:50.106011+0000","last_became_peered":"2024-09-20T22:02:50.106011+0000","last_unstale":"2024-09-20T22:08:25.159564+0000","last_undegraded":"2024-09-20T22:08:25.159564+0000","last_fullsized":"2024-09-20T22:08:25.159564+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:48.105403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:48.105403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":536,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.158814+0000","last_change":"2024-09-20T22:07:53.674419+0000","last_active":"2024-09-20T22:08:25.158814+0000","last_peered":"2024-09-20T22:08:25.158814+0000","last_clean":"2024-09-20T22:08:25.158814+0000","last_became_active":"2024-09-20T22:02:45.068482+0000","last_became_peered":"2024-09-20T22:02:45.068482+0000","last_unstale":"2024-09-20T22:08:25.158814+0000","last_undegraded":"2024-09-20T22:08:25.158814+0000","last_fullsized":"2024-09-20T22:08:25.158814+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:07:53.673691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.673691+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.005158+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":273,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.157630+0000","last_change":"2024-09-20T22:05:56.679105+0000","last_active":"2024-09-20T22:08:25.157630+0000","last_peered":"2024-09-20T22:08:25.157630+0000","last_clean":"2024-09-20T22:08:25.157630+0000","last_became_active":"2024-09-20T22:02:54.298021+0000","last_became_peered":"2024-09-20T22:02:54.298021+0000","last_unstale":"2024-09-20T22:08:25.157630+0000","last_undegraded":"2024-09-20T22:08:25.157630+0000","last_fullsized":"2024-09-20T22:08:25.157630+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:56.678980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:56.678980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":282,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154398+0000","last_change":"2024-09-20T22:06:47.664985+0000","last_active":"2024-09-20T22:08:25.154398+0000","last_peered":"2024-09-20T22:08:25.154398+0000","last_clean":"2024-09-20T22:08:25.154398+0000","last_became_active":"2024-09-20T22:04:21.237725+0000","last_became_peered":"2024-09-20T22:04:21.237725+0000","last_unstale":"2024-09-20T22:08:25.154398+0000","last_undegraded":"2024-09-20T22:08:25.154398+0000","last_fullsized":"2024-09-20T22:08:25.154398+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:47.664877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:06:47.664877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:26.085952+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"190'3438","reported_seq":3476,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.626566+0000","last_change":"2024-09-20T22:07:52.429509+0000","last_active":"2024-09-20T22:08:25.626566+0000","last_peered":"2024-09-20T22:08:25.626566+0000","last_clean":"2024-09-20T22:08:25.626566+0000","last_became_active":"2024-09-20T22:07:52.429069+0000","last_became_peered":"2024-09-20T22:07:52.429069+0000","last_unstale":"2024-09-20T22:08:25.626566+0000","last_undegraded":"2024-09-20T22:08:25.626566+0000","last_fullsized":"2024-09-20T22:08:25.626566+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3438,"log_dups_size":0,"ondisk_log_size":3438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.758907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":225312768,"num_objects":3438,"num_object_clones":0,"num_object_copies":10314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3438,"num_write_kb":220032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"71'11","reported_seq":297,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.153019+0000","last_change":"2024-09-20T22:07:13.511239+0000","last_active":"2024-09-20T22:08:25.153019+0000","last_peered":"2024-09-20T22:08:25.153019+0000","last_clean":"2024-09-20T22:08:25.153019+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:08:25.153019+0000","last_undegraded":"2024-09-20T22:08:25.153019+0000","last_fullsized":"2024-09-20T22:08:25.153019+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:13.511159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:13.511159+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:38.687052+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":273,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.151186+0000","last_change":"2024-09-20T22:05:26.006756+0000","last_active":"2024-09-20T22:08:25.151186+0000","last_peered":"2024-09-20T22:08:25.151186+0000","last_clean":"2024-09-20T22:08:25.151186+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:08:25.151186+0000","last_undegraded":"2024-09-20T22:08:25.151186+0000","last_fullsized":"2024-09-20T22:08:25.151186+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:26.006692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:26.006692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547316+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":272,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154239+0000","last_change":"2024-09-20T22:05:57.397917+0000","last_active":"2024-09-20T22:08:25.154239+0000","last_peered":"2024-09-20T22:08:25.154239+0000","last_clean":"2024-09-20T22:08:25.154239+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:08:25.154239+0000","last_undegraded":"2024-09-20T22:08:25.154239+0000","last_fullsized":"2024-09-20T22:08:25.154239+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:57.397807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:57.397807+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:26.085770+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":268,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.159703+0000","last_change":"2024-09-20T22:06:45.030549+0000","last_active":"2024-09-20T22:08:25.159703+0000","last_peered":"2024-09-20T22:08:25.159703+0000","last_clean":"2024-09-20T22:08:25.159703+0000","last_became_active":"2024-09-20T22:02:56.349376+0000","last_became_peered":"2024-09-20T22:02:56.349376+0000","last_unstale":"2024-09-20T22:08:25.159703+0000","last_undegraded":"2024-09-20T22:08:25.159703+0000","last_fullsized":"2024-09-20T22:08:25.159703+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.030289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.030289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"188'3259","reported_seq":3293,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.646006+0000","last_change":"2024-09-20T22:07:52.939890+0000","last_active":"2024-09-20T22:08:22.646006+0000","last_peered":"2024-09-20T22:08:22.646006+0000","last_clean":"2024-09-20T22:08:22.646006+0000","last_became_active":"2024-09-20T22:07:52.939569+0000","last_became_peered":"2024-09-20T22:07:52.939569+0000","last_unstale":"2024-09-20T22:08:22.646006+0000","last_undegraded":"2024-09-20T22:08:22.646006+0000","last_fullsized":"2024-09-20T22:08:22.646006+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3259,"log_dups_size":0,"ondisk_log_size":3259,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:15.331372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213581824,"num_objects":3259,"num_object_clones":0,"num_object_copies":9777,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3259,"num_write_kb":208576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"71'17","reported_seq":314,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.152583+0000","last_change":"2024-09-20T22:07:54.528854+0000","last_active":"2024-09-20T22:08:25.152583+0000","last_peered":"2024-09-20T22:08:25.152583+0000","last_clean":"2024-09-20T22:08:25.152583+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:08:25.152583+0000","last_undegraded":"2024-09-20T22:08:25.152583+0000","last_fullsized":"2024-09-20T22:08:25.152583+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'17","last_scrub_stamp":"2024-09-20T22:07:54.528759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:54.528759+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.170193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":281,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.153638+0000","last_change":"2024-09-20T22:07:21.124897+0000","last_active":"2024-09-20T22:08:25.153638+0000","last_peered":"2024-09-20T22:08:25.153638+0000","last_clean":"2024-09-20T22:08:25.153638+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:08:25.153638+0000","last_undegraded":"2024-09-20T22:08:25.153638+0000","last_fullsized":"2024-09-20T22:08:25.153638+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:21.124789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:21.124789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:29.000115+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":114,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.130890+0000","last_change":"2024-09-20T22:07:19.376008+0000","last_active":"2024-09-20T22:08:22.130890+0000","last_peered":"2024-09-20T22:08:22.130890+0000","last_clean":"2024-09-20T22:08:22.130890+0000","last_became_active":"2024-09-20T22:06:54.848275+0000","last_became_peered":"2024-09-20T22:06:54.848275+0000","last_unstale":"2024-09-20T22:08:22.130890+0000","last_undegraded":"2024-09-20T22:08:22.130890+0000","last_fullsized":"2024-09-20T22:08:22.130890+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:19.375867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.375867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:49.161984+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":276,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.155482+0000","last_change":"2024-09-20T22:07:55.195229+0000","last_active":"2024-09-20T22:08:25.155482+0000","last_peered":"2024-09-20T22:08:25.155482+0000","last_clean":"2024-09-20T22:08:25.155482+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:08:25.155482+0000","last_undegraded":"2024-09-20T22:08:25.155482+0000","last_fullsized":"2024-09-20T22:08:25.155482+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:55.195131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.195131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:10.187674+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"190'3582","reported_seq":3620,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.116542+0000","last_change":"2024-09-20T22:07:52.429344+0000","last_active":"2024-09-20T22:08:26.116542+0000","last_peered":"2024-09-20T22:08:26.116542+0000","last_clean":"2024-09-20T22:08:26.116542+0000","last_became_active":"2024-09-20T22:07:52.429054+0000","last_became_peered":"2024-09-20T22:07:52.429054+0000","last_unstale":"2024-09-20T22:08:26.116542+0000","last_undegraded":"2024-09-20T22:08:26.116542+0000","last_fullsized":"2024-09-20T22:08:26.116542+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3582,"log_dups_size":0,"ondisk_log_size":3582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:16.620658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234749952,"num_objects":3582,"num_object_clones":0,"num_object_copies":10746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3582,"num_write_kb":229248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"71'10","reported_seq":284,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.152573+0000","last_change":"2024-09-20T22:05:27.048378+0000","last_active":"2024-09-20T22:08:25.152573+0000","last_peered":"2024-09-20T22:08:25.152573+0000","last_clean":"2024-09-20T22:08:25.152573+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:08:25.152573+0000","last_undegraded":"2024-09-20T22:08:25.152573+0000","last_fullsized":"2024-09-20T22:08:25.152573+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:05:27.048215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:27.048215+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547351+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"64'1","reported_seq":286,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.159916+0000","last_change":"2024-09-20T22:06:33.966005+0000","last_active":"2024-09-20T22:08:25.159916+0000","last_peered":"2024-09-20T22:08:25.159916+0000","last_clean":"2024-09-20T22:08:25.159916+0000","last_became_active":"2024-09-20T22:02:50.110446+0000","last_became_peered":"2024-09-20T22:02:50.110446+0000","last_unstale":"2024-09-20T22:08:25.159916+0000","last_undegraded":"2024-09-20T22:08:25.159916+0000","last_fullsized":"2024-09-20T22:08:25.159916+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:06:33.965866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:33.965866+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"139'9","reported_seq":365,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.155060+0000","last_change":"2024-09-20T22:07:57.138071+0000","last_active":"2024-09-20T22:08:25.155060+0000","last_peered":"2024-09-20T22:08:25.155060+0000","last_clean":"2024-09-20T22:08:25.155060+0000","last_became_active":"2024-09-20T22:02:54.294458+0000","last_became_peered":"2024-09-20T22:02:54.294458+0000","last_unstale":"2024-09-20T22:08:25.155060+0000","last_undegraded":"2024-09-20T22:08:25.155060+0000","last_fullsized":"2024-09-20T22:08:25.155060+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'9","last_scrub_stamp":"2024-09-20T22:07:57.137943+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:06:40.910011+0000","last_clean_scrub_stamp":"2024-09-20T22:07:57.137943+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:23.015596+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":270,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.152868+0000","last_change":"2024-09-20T22:07:23.482514+0000","last_active":"2024-09-20T22:08:25.152868+0000","last_peered":"2024-09-20T22:08:25.152868+0000","last_clean":"2024-09-20T22:08:25.152868+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:08:25.152868+0000","last_undegraded":"2024-09-20T22:08:25.152868+0000","last_fullsized":"2024-09-20T22:08:25.152868+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.482372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.886444+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"188'3272","reported_seq":3306,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.628815+0000","last_change":"2024-09-20T22:07:52.429071+0000","last_active":"2024-09-20T22:08:22.628815+0000","last_peered":"2024-09-20T22:08:22.628815+0000","last_clean":"2024-09-20T22:08:22.628815+0000","last_became_active":"2024-09-20T22:07:52.428789+0000","last_became_peered":"2024-09-20T22:07:52.428789+0000","last_unstale":"2024-09-20T22:08:22.628815+0000","last_undegraded":"2024-09-20T22:08:22.628815+0000","last_fullsized":"2024-09-20T22:08:22.628815+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3272,"log_dups_size":0,"ondisk_log_size":3272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:01.547587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":214433792,"num_objects":3272,"num_object_clones":0,"num_object_copies":9816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3272,"num_write_kb":209408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"71'9","reported_seq":301,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.153556+0000","last_change":"2024-09-20T22:07:59.740218+0000","last_active":"2024-09-20T22:08:25.153556+0000","last_peered":"2024-09-20T22:08:25.153556+0000","last_clean":"2024-09-20T22:08:25.153556+0000","last_became_active":"2024-09-20T22:06:54.847770+0000","last_became_peered":"2024-09-20T22:06:54.847770+0000","last_unstale":"2024-09-20T22:08:25.153556+0000","last_undegraded":"2024-09-20T22:08:25.153556+0000","last_fullsized":"2024-09-20T22:08:25.153556+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:59.740089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:59.740089+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:19.034130+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":286,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154702+0000","last_change":"2024-09-20T22:06:36.582728+0000","last_active":"2024-09-20T22:08:25.154702+0000","last_peered":"2024-09-20T22:08:25.154702+0000","last_clean":"2024-09-20T22:08:25.154702+0000","last_became_active":"2024-09-20T22:02:50.112682+0000","last_became_peered":"2024-09-20T22:02:50.112682+0000","last_unstale":"2024-09-20T22:08:25.154702+0000","last_undegraded":"2024-09-20T22:08:25.154702+0000","last_fullsized":"2024-09-20T22:08:25.154702+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:06:36.582537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:36.582537+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:26.085850+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":280,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154590+0000","last_change":"2024-09-20T22:07:22.203673+0000","last_active":"2024-09-20T22:08:25.154590+0000","last_peered":"2024-09-20T22:08:25.154590+0000","last_clean":"2024-09-20T22:08:25.154590+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:08:25.154590+0000","last_undegraded":"2024-09-20T22:08:25.154590+0000","last_fullsized":"2024-09-20T22:08:25.154590+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:22.203576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.203576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.064543+0000","scrub_duration":49,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":270,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.153967+0000","last_change":"2024-09-20T22:06:55.824784+0000","last_active":"2024-09-20T22:08:25.153967+0000","last_peered":"2024-09-20T22:08:25.153967+0000","last_clean":"2024-09-20T22:08:25.153967+0000","last_became_active":"2024-09-20T22:06:55.824229+0000","last_became_peered":"2024-09-20T22:06:55.824229+0000","last_unstale":"2024-09-20T22:08:25.153967+0000","last_undegraded":"2024-09-20T22:08:25.153967+0000","last_fullsized":"2024-09-20T22:08:25.153967+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:17.171523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:17.171523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"190'3663","reported_seq":3701,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.233155+0000","last_change":"2024-09-20T22:07:52.674668+0000","last_active":"2024-09-20T22:08:26.233155+0000","last_peered":"2024-09-20T22:08:26.233155+0000","last_clean":"2024-09-20T22:08:26.233155+0000","last_became_active":"2024-09-20T22:07:52.674301+0000","last_became_peered":"2024-09-20T22:07:52.674301+0000","last_unstale":"2024-09-20T22:08:26.233155+0000","last_undegraded":"2024-09-20T22:08:26.233155+0000","last_fullsized":"2024-09-20T22:08:26.233155+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3663,"log_dups_size":0,"ondisk_log_size":3663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:06.988552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240058368,"num_objects":3663,"num_object_clones":0,"num_object_copies":10989,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3663,"num_write_kb":234432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"71'19","reported_seq":308,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.153428+0000","last_change":"2024-09-20T22:07:25.155015+0000","last_active":"2024-09-20T22:08:25.153428+0000","last_peered":"2024-09-20T22:08:25.153428+0000","last_clean":"2024-09-20T22:08:25.153428+0000","last_became_active":"2024-09-20T22:06:54.829170+0000","last_became_peered":"2024-09-20T22:06:54.829170+0000","last_unstale":"2024-09-20T22:08:25.153428+0000","last_undegraded":"2024-09-20T22:08:25.153428+0000","last_fullsized":"2024-09-20T22:08:25.153428+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:07:25.154919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:25.154919+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:43.954499+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":281,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.158372+0000","last_change":"2024-09-20T22:08:00.560208+0000","last_active":"2024-09-20T22:08:25.158372+0000","last_peered":"2024-09-20T22:08:25.158372+0000","last_clean":"2024-09-20T22:08:25.158372+0000","last_became_active":"2024-09-20T22:02:50.108712+0000","last_became_peered":"2024-09-20T22:02:50.108712+0000","last_unstale":"2024-09-20T22:08:25.158372+0000","last_undegraded":"2024-09-20T22:08:25.158372+0000","last_fullsized":"2024-09-20T22:08:25.158372+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:00.560098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:00.560098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:09.631655+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":280,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154442+0000","last_change":"2024-09-20T22:07:20.217991+0000","last_active":"2024-09-20T22:08:25.154442+0000","last_peered":"2024-09-20T22:08:25.154442+0000","last_clean":"2024-09-20T22:08:25.154442+0000","last_became_active":"2024-09-20T22:02:54.291134+0000","last_became_peered":"2024-09-20T22:02:54.291134+0000","last_unstale":"2024-09-20T22:08:25.154442+0000","last_undegraded":"2024-09-20T22:08:25.154442+0000","last_fullsized":"2024-09-20T22:08:25.154442+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:20.217761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:20.217761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:42.617706+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":269,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.158718+0000","last_change":"2024-09-20T22:07:29.645352+0000","last_active":"2024-09-20T22:08:25.158718+0000","last_peered":"2024-09-20T22:08:25.158718+0000","last_clean":"2024-09-20T22:08:25.158718+0000","last_became_active":"2024-09-20T22:02:56.307687+0000","last_became_peered":"2024-09-20T22:02:56.307687+0000","last_unstale":"2024-09-20T22:08:25.158718+0000","last_undegraded":"2024-09-20T22:08:25.158718+0000","last_fullsized":"2024-09-20T22:08:25.158718+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:29.645127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:29.645127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:36.059611+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"190'3636","reported_seq":3673,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.725337+0000","last_change":"2024-09-20T22:07:52.675309+0000","last_active":"2024-09-20T22:08:25.725337+0000","last_peered":"2024-09-20T22:08:25.725337+0000","last_clean":"2024-09-20T22:08:25.725337+0000","last_became_active":"2024-09-20T22:07:52.674921+0000","last_became_peered":"2024-09-20T22:07:52.674921+0000","last_unstale":"2024-09-20T22:08:25.725337+0000","last_undegraded":"2024-09-20T22:08:25.725337+0000","last_fullsized":"2024-09-20T22:08:25.725337+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3636,"log_dups_size":0,"ondisk_log_size":3636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:17.687123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238288896,"num_objects":3636,"num_object_clones":0,"num_object_copies":10908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3636,"num_write_kb":232704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"71'12","reported_seq":301,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.150938+0000","last_change":"2024-09-20T22:07:22.458421+0000","last_active":"2024-09-20T22:08:25.150938+0000","last_peered":"2024-09-20T22:08:25.150938+0000","last_clean":"2024-09-20T22:08:25.150938+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:08:25.150938+0000","last_undegraded":"2024-09-20T22:08:25.150938+0000","last_fullsized":"2024-09-20T22:08:25.150938+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:22.458279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.458279+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:33.412138+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":282,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.157503+0000","last_change":"2024-09-20T22:07:21.584606+0000","last_active":"2024-09-20T22:08:25.157503+0000","last_peered":"2024-09-20T22:08:25.157503+0000","last_clean":"2024-09-20T22:08:25.157503+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:08:25.157503+0000","last_undegraded":"2024-09-20T22:08:25.157503+0000","last_fullsized":"2024-09-20T22:08:25.157503+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:21.584499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:21.584499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":281,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154471+0000","last_change":"2024-09-20T22:07:15.260254+0000","last_active":"2024-09-20T22:08:25.154471+0000","last_peered":"2024-09-20T22:08:25.154471+0000","last_clean":"2024-09-20T22:08:25.154471+0000","last_became_active":"2024-09-20T22:06:54.843906+0000","last_became_peered":"2024-09-20T22:06:54.843906+0000","last_unstale":"2024-09-20T22:08:25.154471+0000","last_undegraded":"2024-09-20T22:08:25.154471+0000","last_fullsized":"2024-09-20T22:08:25.154471+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:15.260148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.260148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:26.224893+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":214,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154516+0000","last_change":"2024-09-20T22:06:55.822136+0000","last_active":"2024-09-20T22:08:25.154516+0000","last_peered":"2024-09-20T22:08:25.154516+0000","last_clean":"2024-09-20T22:08:25.154516+0000","last_became_active":"2024-09-20T22:06:55.821647+0000","last_became_peered":"2024-09-20T22:06:55.821647+0000","last_unstale":"2024-09-20T22:08:25.154516+0000","last_undegraded":"2024-09-20T22:08:25.154516+0000","last_fullsized":"2024-09-20T22:08:25.154516+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.569045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.569045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"190'3520","reported_seq":3557,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.596737+0000","last_change":"2024-09-20T22:07:52.939846+0000","last_active":"2024-09-20T22:08:25.596737+0000","last_peered":"2024-09-20T22:08:25.596737+0000","last_clean":"2024-09-20T22:08:25.596737+0000","last_became_active":"2024-09-20T22:07:52.939563+0000","last_became_peered":"2024-09-20T22:07:52.939563+0000","last_unstale":"2024-09-20T22:08:25.596737+0000","last_undegraded":"2024-09-20T22:08:25.596737+0000","last_fullsized":"2024-09-20T22:08:25.596737+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3520,"log_dups_size":0,"ondisk_log_size":3520,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:55.690792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":230686720,"num_objects":3520,"num_object_clones":0,"num_object_copies":10560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3520,"num_write_kb":225280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"71'15","reported_seq":301,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.160403+0000","last_change":"2024-09-20T22:06:42.563403+0000","last_active":"2024-09-20T22:08:25.160403+0000","last_peered":"2024-09-20T22:08:25.160403+0000","last_clean":"2024-09-20T22:08:25.160403+0000","last_became_active":"2024-09-20T22:02:52.482008+0000","last_became_peered":"2024-09-20T22:02:52.482008+0000","last_unstale":"2024-09-20T22:08:25.160403+0000","last_undegraded":"2024-09-20T22:08:25.160403+0000","last_fullsized":"2024-09-20T22:08:25.160403+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:06:42.563266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:42.563266+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:26.085928+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":295,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.158194+0000","last_change":"2024-09-20T22:06:40.813723+0000","last_active":"2024-09-20T22:08:25.158194+0000","last_peered":"2024-09-20T22:08:25.158194+0000","last_clean":"2024-09-20T22:08:25.158194+0000","last_became_active":"2024-09-20T22:02:50.114819+0000","last_became_peered":"2024-09-20T22:02:50.114819+0000","last_unstale":"2024-09-20T22:08:25.158194+0000","last_undegraded":"2024-09-20T22:08:25.158194+0000","last_fullsized":"2024-09-20T22:08:25.158194+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:06:40.813590+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":341,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.156347+0000","last_change":"2024-09-20T22:05:34.732761+0000","last_active":"2024-09-20T22:08:25.156347+0000","last_peered":"2024-09-20T22:08:25.156347+0000","last_clean":"2024-09-20T22:08:25.156347+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:08:25.156347+0000","last_undegraded":"2024-09-20T22:08:25.156347+0000","last_fullsized":"2024-09-20T22:08:25.156347+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:05:34.732644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:34.732644+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":268,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154323+0000","last_change":"2024-09-20T22:06:35.559426+0000","last_active":"2024-09-20T22:08:25.154323+0000","last_peered":"2024-09-20T22:08:25.154323+0000","last_clean":"2024-09-20T22:08:25.154323+0000","last_became_active":"2024-09-20T22:02:56.346990+0000","last_became_peered":"2024-09-20T22:02:56.346990+0000","last_unstale":"2024-09-20T22:08:25.154323+0000","last_undegraded":"2024-09-20T22:08:25.154323+0000","last_fullsized":"2024-09-20T22:08:25.154323+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:35.559306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:35.559306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:26.085876+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":288,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.152572+0000","last_change":"2024-09-20T22:08:01.261458+0000","last_active":"2024-09-20T22:08:25.152572+0000","last_peered":"2024-09-20T22:08:25.152572+0000","last_clean":"2024-09-20T22:08:25.152572+0000","last_became_active":"2024-09-20T22:02:50.111935+0000","last_became_peered":"2024-09-20T22:02:50.111935+0000","last_unstale":"2024-09-20T22:08:25.152572+0000","last_undegraded":"2024-09-20T22:08:25.152572+0000","last_fullsized":"2024-09-20T22:08:25.152572+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:01.261328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:01.261328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:06.169471+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"71'6","reported_seq":278,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.160318+0000","last_change":"2024-09-20T22:06:54.862697+0000","last_active":"2024-09-20T22:08:25.160318+0000","last_peered":"2024-09-20T22:08:25.160318+0000","last_clean":"2024-09-20T22:08:25.160318+0000","last_became_active":"2024-09-20T22:06:54.862227+0000","last_became_peered":"2024-09-20T22:06:54.862227+0000","last_unstale":"2024-09-20T22:08:25.160318+0000","last_undegraded":"2024-09-20T22:08:25.160318+0000","last_fullsized":"2024-09-20T22:08:25.160318+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":273,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154744+0000","last_change":"2024-09-20T22:07:19.865906+0000","last_active":"2024-09-20T22:08:25.154744+0000","last_peered":"2024-09-20T22:08:25.154744+0000","last_clean":"2024-09-20T22:08:25.154744+0000","last_became_active":"2024-09-20T22:06:54.836200+0000","last_became_peered":"2024-09-20T22:06:54.836200+0000","last_unstale":"2024-09-20T22:08:25.154744+0000","last_undegraded":"2024-09-20T22:08:25.154744+0000","last_fullsized":"2024-09-20T22:08:25.154744+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:19.865809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.865809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:42.730889+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":261,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.157936+0000","last_change":"2024-09-20T22:06:39.783526+0000","last_active":"2024-09-20T22:08:25.157936+0000","last_peered":"2024-09-20T22:08:25.157936+0000","last_clean":"2024-09-20T22:08:25.157936+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:08:25.157936+0000","last_undegraded":"2024-09-20T22:08:25.157936+0000","last_fullsized":"2024-09-20T22:08:25.157936+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:39.783456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:39.783456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"71'9","reported_seq":284,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.155876+0000","last_change":"2024-09-20T22:04:52.254864+0000","last_active":"2024-09-20T22:08:25.155876+0000","last_peered":"2024-09-20T22:08:25.155876+0000","last_clean":"2024-09-20T22:08:25.155876+0000","last_became_active":"2024-09-20T22:02:52.481989+0000","last_became_peered":"2024-09-20T22:02:52.481989+0000","last_unstale":"2024-09-20T22:08:25.155876+0000","last_undegraded":"2024-09-20T22:08:25.155876+0000","last_fullsized":"2024-09-20T22:08:25.155876+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:52.254706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:52.254706+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":273,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.157814+0000","last_change":"2024-09-20T22:05:31.764903+0000","last_active":"2024-09-20T22:08:25.157814+0000","last_peered":"2024-09-20T22:08:25.157814+0000","last_clean":"2024-09-20T22:08:25.157814+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:08:25.157814+0000","last_undegraded":"2024-09-20T22:08:25.157814+0000","last_fullsized":"2024-09-20T22:08:25.157814+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:31.764833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:31.764833+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":265,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.160255+0000","last_change":"2024-09-20T22:06:54.847169+0000","last_active":"2024-09-20T22:08:25.160255+0000","last_peered":"2024-09-20T22:08:25.160255+0000","last_clean":"2024-09-20T22:08:25.160255+0000","last_became_active":"2024-09-20T22:06:54.845704+0000","last_became_peered":"2024-09-20T22:06:54.845704+0000","last_unstale":"2024-09-20T22:08:25.160255+0000","last_undegraded":"2024-09-20T22:08:25.160255+0000","last_fullsized":"2024-09-20T22:08:25.160255+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":268,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.155241+0000","last_change":"2024-09-20T22:06:39.942296+0000","last_active":"2024-09-20T22:08:25.155241+0000","last_peered":"2024-09-20T22:08:25.155241+0000","last_clean":"2024-09-20T22:08:25.155241+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:08:25.155241+0000","last_undegraded":"2024-09-20T22:08:25.155241+0000","last_fullsized":"2024-09-20T22:08:25.155241+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:39.942186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:39.942186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:28.184860+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"71'9","reported_seq":300,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154599+0000","last_change":"2024-09-20T22:07:52.834986+0000","last_active":"2024-09-20T22:08:25.154599+0000","last_peered":"2024-09-20T22:08:25.154599+0000","last_clean":"2024-09-20T22:08:25.154599+0000","last_became_active":"2024-09-20T22:02:52.482141+0000","last_became_peered":"2024-09-20T22:02:52.482141+0000","last_unstale":"2024-09-20T22:08:25.154599+0000","last_undegraded":"2024-09-20T22:08:25.154599+0000","last_fullsized":"2024-09-20T22:08:25.154599+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:52.834898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:52.834898+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.590814+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":280,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.153663+0000","last_change":"2024-09-20T22:06:45.908734+0000","last_active":"2024-09-20T22:08:25.153663+0000","last_peered":"2024-09-20T22:08:25.153663+0000","last_clean":"2024-09-20T22:08:25.153663+0000","last_became_active":"2024-09-20T22:02:50.110075+0000","last_became_peered":"2024-09-20T22:02:50.110075+0000","last_unstale":"2024-09-20T22:08:25.153663+0000","last_undegraded":"2024-09-20T22:08:25.153663+0000","last_fullsized":"2024-09-20T22:08:25.153663+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.908616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.908616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"71'8","reported_seq":340,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.159949+0000","last_change":"2024-09-20T22:05:55.972684+0000","last_active":"2024-09-20T22:08:25.159949+0000","last_peered":"2024-09-20T22:08:25.159949+0000","last_clean":"2024-09-20T22:08:25.159949+0000","last_became_active":"2024-09-20T22:02:54.294381+0000","last_became_peered":"2024-09-20T22:02:54.294381+0000","last_unstale":"2024-09-20T22:08:25.159949+0000","last_undegraded":"2024-09-20T22:08:25.159949+0000","last_fullsized":"2024-09-20T22:08:25.159949+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:05:55.972565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:55.972565+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":261,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.153080+0000","last_change":"2024-09-20T22:05:00.897974+0000","last_active":"2024-09-20T22:08:25.153080+0000","last_peered":"2024-09-20T22:08:25.153080+0000","last_clean":"2024-09-20T22:08:25.153080+0000","last_became_active":"2024-09-20T22:02:56.300041+0000","last_became_peered":"2024-09-20T22:02:56.300041+0000","last_unstale":"2024-09-20T22:08:25.153080+0000","last_undegraded":"2024-09-20T22:08:25.153080+0000","last_fullsized":"2024-09-20T22:08:25.153080+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:00.897869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:00.897869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547236+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"71'10","reported_seq":284,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.160401+0000","last_change":"2024-09-20T22:06:54.862610+0000","last_active":"2024-09-20T22:08:25.160401+0000","last_peered":"2024-09-20T22:08:25.160401+0000","last_clean":"2024-09-20T22:08:25.160401+0000","last_became_active":"2024-09-20T22:06:54.862289+0000","last_became_peered":"2024-09-20T22:06:54.862289+0000","last_unstale":"2024-09-20T22:08:25.160401+0000","last_undegraded":"2024-09-20T22:08:25.160401+0000","last_fullsized":"2024-09-20T22:08:25.160401+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":273,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.158037+0000","last_change":"2024-09-20T22:06:35.687169+0000","last_active":"2024-09-20T22:08:25.158037+0000","last_peered":"2024-09-20T22:08:25.158037+0000","last_clean":"2024-09-20T22:08:25.158037+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:08:25.158037+0000","last_undegraded":"2024-09-20T22:08:25.158037+0000","last_fullsized":"2024-09-20T22:08:25.158037+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:35.687065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:35.687065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"146'10","reported_seq":351,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154397+0000","last_change":"2024-09-20T22:07:15.870748+0000","last_active":"2024-09-20T22:08:25.154397+0000","last_peered":"2024-09-20T22:08:25.154397+0000","last_clean":"2024-09-20T22:08:25.154397+0000","last_became_active":"2024-09-20T22:06:54.843097+0000","last_became_peered":"2024-09-20T22:06:54.843097+0000","last_unstale":"2024-09-20T22:08:25.154397+0000","last_undegraded":"2024-09-20T22:08:25.154397+0000","last_fullsized":"2024-09-20T22:08:25.154397+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:07:15.870646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.870646+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:36.803368+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":260,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154975+0000","last_change":"2024-09-20T22:05:35.601891+0000","last_active":"2024-09-20T22:08:25.154975+0000","last_peered":"2024-09-20T22:08:25.154975+0000","last_clean":"2024-09-20T22:08:25.154975+0000","last_became_active":"2024-09-20T22:02:56.305383+0000","last_became_peered":"2024-09-20T22:02:56.305383+0000","last_unstale":"2024-09-20T22:08:25.154975+0000","last_undegraded":"2024-09-20T22:08:25.154975+0000","last_fullsized":"2024-09-20T22:08:25.154975+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:35.601794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:35.601794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"71'11","reported_seq":315,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.152340+0000","last_change":"2024-09-20T22:08:22.159810+0000","last_active":"2024-09-20T22:08:25.152340+0000","last_peered":"2024-09-20T22:08:25.152340+0000","last_clean":"2024-09-20T22:08:25.152340+0000","last_became_active":"2024-09-20T22:08:22.139581+0000","last_became_peered":"2024-09-20T22:08:22.139581+0000","last_unstale":"2024-09-20T22:08:25.152340+0000","last_undegraded":"2024-09-20T22:08:25.152340+0000","last_fullsized":"2024-09-20T22:08:25.152340+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:15.544705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.544705+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:28.946270+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":273,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.152677+0000","last_change":"2024-09-20T22:04:54.764407+0000","last_active":"2024-09-20T22:08:25.152677+0000","last_peered":"2024-09-20T22:08:25.152677+0000","last_clean":"2024-09-20T22:08:25.152677+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:08:25.152677+0000","last_undegraded":"2024-09-20T22:08:25.152677+0000","last_fullsized":"2024-09-20T22:08:25.152677+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:54.764137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:54.764137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547208+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":106,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.130806+0000","last_change":"2024-09-20T22:06:54.853857+0000","last_active":"2024-09-20T22:08:22.130806+0000","last_peered":"2024-09-20T22:08:22.130806+0000","last_clean":"2024-09-20T22:08:22.130806+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:08:22.130806+0000","last_undegraded":"2024-09-20T22:08:22.130806+0000","last_fullsized":"2024-09-20T22:08:22.130806+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":93,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.131968+0000","last_change":"2024-09-20T22:06:55.835713+0000","last_active":"2024-09-20T22:08:22.131968+0000","last_peered":"2024-09-20T22:08:22.131968+0000","last_clean":"2024-09-20T22:08:22.131968+0000","last_became_active":"2024-09-20T22:06:55.835393+0000","last_became_peered":"2024-09-20T22:06:55.835393+0000","last_unstale":"2024-09-20T22:08:22.131968+0000","last_undegraded":"2024-09-20T22:08:22.131968+0000","last_fullsized":"2024-09-20T22:08:22.131968+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:33.623816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:33.623816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"71'9","reported_seq":178,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.131202+0000","last_change":"2024-09-20T22:06:54.844349+0000","last_active":"2024-09-20T22:08:22.131202+0000","last_peered":"2024-09-20T22:08:22.131202+0000","last_clean":"2024-09-20T22:08:22.131202+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:08:22.131202+0000","last_undegraded":"2024-09-20T22:08:22.131202+0000","last_fullsized":"2024-09-20T22:08:22.131202+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:12.196518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:12.196518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":289,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.158224+0000","last_change":"2024-09-20T22:07:54.685916+0000","last_active":"2024-09-20T22:08:25.158224+0000","last_peered":"2024-09-20T22:08:25.158224+0000","last_clean":"2024-09-20T22:08:25.158224+0000","last_became_active":"2024-09-20T22:02:50.109803+0000","last_became_peered":"2024-09-20T22:02:50.109803+0000","last_unstale":"2024-09-20T22:08:25.158224+0000","last_undegraded":"2024-09-20T22:08:25.158224+0000","last_fullsized":"2024-09-20T22:08:25.158224+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:54.685751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:54.685751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:04.385036+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":265,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.159812+0000","last_change":"2024-09-20T22:06:54.844570+0000","last_active":"2024-09-20T22:08:25.159812+0000","last_peered":"2024-09-20T22:08:25.159812+0000","last_clean":"2024-09-20T22:08:25.159812+0000","last_became_active":"2024-09-20T22:06:54.844271+0000","last_became_peered":"2024-09-20T22:06:54.844271+0000","last_unstale":"2024-09-20T22:08:25.159812+0000","last_undegraded":"2024-09-20T22:08:25.159812+0000","last_fullsized":"2024-09-20T22:08:25.159812+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":269,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.152909+0000","last_change":"2024-09-20T22:07:23.495005+0000","last_active":"2024-09-20T22:08:25.152909+0000","last_peered":"2024-09-20T22:08:25.152909+0000","last_clean":"2024-09-20T22:08:25.152909+0000","last_became_active":"2024-09-20T22:06:54.848842+0000","last_became_peered":"2024-09-20T22:06:54.848842+0000","last_unstale":"2024-09-20T22:08:25.152909+0000","last_undegraded":"2024-09-20T22:08:25.152909+0000","last_fullsized":"2024-09-20T22:08:25.152909+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.494905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.494905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.999867+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"71'11","reported_seq":295,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.159002+0000","last_change":"2024-09-20T22:06:36.942847+0000","last_active":"2024-09-20T22:08:25.159002+0000","last_peered":"2024-09-20T22:08:25.159002+0000","last_clean":"2024-09-20T22:08:25.159002+0000","last_became_active":"2024-09-20T22:02:52.480118+0000","last_became_peered":"2024-09-20T22:02:52.480118+0000","last_unstale":"2024-09-20T22:08:25.159002+0000","last_undegraded":"2024-09-20T22:08:25.159002+0000","last_fullsized":"2024-09-20T22:08:25.159002+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:06:36.942620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:36.942620+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":289,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.155873+0000","last_change":"2024-09-20T22:07:50.851948+0000","last_active":"2024-09-20T22:08:25.155873+0000","last_peered":"2024-09-20T22:08:25.155873+0000","last_clean":"2024-09-20T22:08:25.155873+0000","last_became_active":"2024-09-20T22:06:54.828826+0000","last_became_peered":"2024-09-20T22:06:54.828826+0000","last_unstale":"2024-09-20T22:08:25.155873+0000","last_undegraded":"2024-09-20T22:08:25.155873+0000","last_fullsized":"2024-09-20T22:08:25.155873+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:50.851839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:50.851839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:57.182737+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":273,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.158340+0000","last_change":"2024-09-20T22:06:45.780268+0000","last_active":"2024-09-20T22:08:25.158340+0000","last_peered":"2024-09-20T22:08:25.158340+0000","last_clean":"2024-09-20T22:08:25.158340+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:08:25.158340+0000","last_undegraded":"2024-09-20T22:08:25.158340+0000","last_fullsized":"2024-09-20T22:08:25.158340+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.780200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.780200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":268,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.159896+0000","last_change":"2024-09-20T22:06:33.002428+0000","last_active":"2024-09-20T22:08:25.159896+0000","last_peered":"2024-09-20T22:08:25.159896+0000","last_clean":"2024-09-20T22:08:25.159896+0000","last_became_active":"2024-09-20T22:02:56.347637+0000","last_became_peered":"2024-09-20T22:02:56.347637+0000","last_unstale":"2024-09-20T22:08:25.159896+0000","last_undegraded":"2024-09-20T22:08:25.159896+0000","last_fullsized":"2024-09-20T22:08:25.159896+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:33.002326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:33.002326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"71'4","reported_seq":283,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.159649+0000","last_change":"2024-09-20T22:07:19.309318+0000","last_active":"2024-09-20T22:08:25.159649+0000","last_peered":"2024-09-20T22:08:25.159649+0000","last_clean":"2024-09-20T22:08:25.159649+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:08:25.159649+0000","last_undegraded":"2024-09-20T22:08:25.159649+0000","last_fullsized":"2024-09-20T22:08:25.159649+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'4","last_scrub_stamp":"2024-09-20T22:07:19.309187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.309187+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:33.061321+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":272,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154247+0000","last_change":"2024-09-20T22:04:16.198264+0000","last_active":"2024-09-20T22:08:25.154247+0000","last_peered":"2024-09-20T22:08:25.154247+0000","last_clean":"2024-09-20T22:08:25.154247+0000","last_became_active":"2024-09-20T22:02:50.111142+0000","last_became_peered":"2024-09-20T22:02:50.111142+0000","last_unstale":"2024-09-20T22:08:25.154247+0000","last_undegraded":"2024-09-20T22:08:25.154247+0000","last_fullsized":"2024-09-20T22:08:25.154247+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:16.198153+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:04:16.198153+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":288,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.153290+0000","last_change":"2024-09-20T22:07:55.255933+0000","last_active":"2024-09-20T22:08:25.153290+0000","last_peered":"2024-09-20T22:08:25.153290+0000","last_clean":"2024-09-20T22:08:25.153290+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:08:25.153290+0000","last_undegraded":"2024-09-20T22:08:25.153290+0000","last_fullsized":"2024-09-20T22:08:25.153290+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:55.255854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.255854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.819675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":261,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.156644+0000","last_change":"2024-09-20T22:06:01.621567+0000","last_active":"2024-09-20T22:08:25.156644+0000","last_peered":"2024-09-20T22:08:25.156644+0000","last_clean":"2024-09-20T22:08:25.156644+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:08:25.156644+0000","last_undegraded":"2024-09-20T22:08:25.156644+0000","last_fullsized":"2024-09-20T22:08:25.156644+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:01.621489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:01.621489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":154,"reported_epoch":188,"state":"active+clean","last_fresh":"2024-09-20T22:08:22.131340+0000","last_change":"2024-09-20T22:06:54.828772+0000","last_active":"2024-09-20T22:08:22.131340+0000","last_peered":"2024-09-20T22:08:22.131340+0000","last_clean":"2024-09-20T22:08:22.131340+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:08:22.131340+0000","last_undegraded":"2024-09-20T22:08:22.131340+0000","last_fullsized":"2024-09-20T22:08:22.131340+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:15.303300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:15.303300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":281,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.160287+0000","last_change":"2024-09-20T22:07:57.428285+0000","last_active":"2024-09-20T22:08:25.160287+0000","last_peered":"2024-09-20T22:08:25.160287+0000","last_clean":"2024-09-20T22:08:25.160287+0000","last_became_active":"2024-09-20T22:06:54.848032+0000","last_became_peered":"2024-09-20T22:06:54.848032+0000","last_unstale":"2024-09-20T22:08:25.160287+0000","last_undegraded":"2024-09-20T22:08:25.160287+0000","last_fullsized":"2024-09-20T22:08:25.160287+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:57.428165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:57.428165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:04.979638+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"71'11","reported_seq":296,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.154249+0000","last_change":"2024-09-20T22:07:19.214716+0000","last_active":"2024-09-20T22:08:25.154249+0000","last_peered":"2024-09-20T22:08:25.154249+0000","last_clean":"2024-09-20T22:08:25.154249+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:08:25.154249+0000","last_undegraded":"2024-09-20T22:08:25.154249+0000","last_fullsized":"2024-09-20T22:08:25.154249+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:19.214636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.214636+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:32.168748+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":264,"reported_epoch":190,"state":"active+clean","last_fresh":"2024-09-20T22:08:25.155129+0000","last_change":"2024-09-20T22:05:06.706314+0000","last_active":"2024-09-20T22:08:25.155129+0000","last_peered":"2024-09-20T22:08:25.155129+0000","last_clean":"2024-09-20T22:08:25.155129+0000","last_became_active":"2024-09-20T22:02:54.289078+0000","last_became_peered":"2024-09-20T22:02:54.289078+0000","last_unstale":"2024-09-20T22:08:25.155129+0000","last_undegraded":"2024-09-20T22:08:25.155129+0000","last_fullsized":"2024-09-20T22:08:25.155129+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:06.681123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:06.681123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":82,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":3685416960,"num_objects":56235,"num_object_clones":0,"num_object_copies":168705,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56235,"num_write_kb":3599040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11136204800,"data_stored":11136204800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56235,"ondisk_log_size":56235,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":322,"num_read_kb":317,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135943,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1326176,"kb_used_data":1124236,"kb_used_omap":1,"kb_used_meta":201918,"kb_avail":92418976,"statfs":{"total":95995035648,"available":94637031424,"internally_reserved":0,"allocated":1151217664,"data_stored":1150416938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":206764490},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430283,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1311904,"kb_used_data":1140488,"kb_used_omap":1,"kb_used_meta":171390,"kb_avail":92433248,"statfs":{"total":95995035648,"available":94651645952,"internally_reserved":0,"allocated":1167859712,"data_stored":1167071353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":175503814},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724623,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1968332,"kb_used_data":1792764,"kb_used_omap":1,"kb_used_meta":175550,"kb_avail":91776820,"statfs":{"total":95995035648,"available":93979463680,"internally_reserved":0,"allocated":1835790336,"data_stored":1835005814,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":179763653},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986263,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1326572,"kb_used_data":1132956,"kb_used_omap":1,"kb_used_meta":193598,"kb_avail":92418580,"statfs":{"total":95995035648,"available":94636625920,"internally_reserved":0,"allocated":1160146944,"data_stored":1159337596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":198244805},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770257939,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1593776,"kb_used_data":1459928,"kb_used_omap":4,"kb_used_meta":133755,"kb_avail":92151376,"statfs":{"total":95995035648,"available":94363009024,"internally_reserved":0,"allocated":1494966272,"data_stored":1494144758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":136965800},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149788,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1695164,"kb_used_data":1361324,"kb_used_omap":1,"kb_used_meta":333822,"kb_avail":92049988,"statfs":{"total":95995035648,"available":94259187712,"internally_reserved":0,"allocated":1393995776,"data_stored":1393171336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":341834183},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411425,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1584216,"kb_used_data":1358656,"kb_used_omap":1,"kb_used_meta":225534,"kb_avail":92160936,"statfs":{"total":95995035648,"available":94372798464,"internally_reserved":0,"allocated":1391263744,"data_stored":1390482395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":230947269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738476,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1710188,"kb_used_data":1372188,"kb_used_omap":1,"kb_used_meta":337982,"kb_avail":92034964,"statfs":{"total":95995035648,"available":94243803136,"internally_reserved":0,"allocated":1405120512,"data_stored":1404311897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":346094021},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1419509760,"data_stored":1419509760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1507262464,"data_stored":1507262464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1418067968,"data_stored":1418067968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1416232960,"data_stored":1416232960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1161756672,"data_stored":1161756672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1860435968,"data_stored":1860435968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1177681920,"data_stored":1177681920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1175257088,"data_stored":1175257088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:08:30.583 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-20T22:08:30.583 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:08:30.583 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2024-09-20T22:08:30.583 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:08:30.594 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:30.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:30.595 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:30.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:30.718 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:30.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:30.719 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:30 smithi067 ceph-mon[33511]: 3.5 scrub starts 2024-09-20T22:08:30.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:30 smithi067 ceph-mon[33511]: 3.5 scrub ok 2024-09-20T22:08:30.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:30 smithi067 ceph-mon[33511]: 5.1f scrub starts 2024-09-20T22:08:30.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:30 smithi067 ceph-mon[33511]: 5.1f scrub ok 2024-09-20T22:08:30.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:30 smithi067 ceph-mon[33511]: 3.14 scrub starts 2024-09-20T22:08:30.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:30 smithi067 ceph-mon[33511]: 3.14 scrub ok 2024-09-20T22:08:30.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:30 smithi067 ceph-mon[33511]: 3.17 scrub starts 2024-09-20T22:08:30.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:30 smithi067 ceph-mon[33511]: 3.17 scrub ok 2024-09-20T22:08:30.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:30 smithi067 ceph-mon[33511]: pgmap v276: 148 pgs: 148 active+clean; 3.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 76 MiB/s wr, 1.22k op/s; 54 B/s, 2 objects/s recovering 2024-09-20T22:08:30.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:30 smithi060 ceph-mon[29694]: 3.5 scrub starts 2024-09-20T22:08:30.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:30 smithi060 ceph-mon[29694]: 3.5 scrub ok 2024-09-20T22:08:30.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:30 smithi060 ceph-mon[29694]: 5.1f scrub starts 2024-09-20T22:08:30.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:30 smithi060 ceph-mon[29694]: 5.1f scrub ok 2024-09-20T22:08:30.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:30 smithi060 ceph-mon[29694]: 3.14 scrub starts 2024-09-20T22:08:30.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:30 smithi060 ceph-mon[29694]: 3.14 scrub ok 2024-09-20T22:08:30.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:30 smithi060 ceph-mon[29694]: 3.17 scrub starts 2024-09-20T22:08:30.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:30 smithi060 ceph-mon[29694]: 3.17 scrub ok 2024-09-20T22:08:30.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:30 smithi060 ceph-mon[29694]: pgmap v276: 148 pgs: 148 active+clean; 3.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 76 MiB/s wr, 1.22k op/s; 54 B/s, 2 objects/s recovering 2024-09-20T22:08:30.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:30 smithi060 ceph-mon[40290]: 3.5 scrub starts 2024-09-20T22:08:30.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:30 smithi060 ceph-mon[40290]: 3.5 scrub ok 2024-09-20T22:08:30.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:30 smithi060 ceph-mon[40290]: 5.1f scrub starts 2024-09-20T22:08:30.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:30 smithi060 ceph-mon[40290]: 5.1f scrub ok 2024-09-20T22:08:30.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:30 smithi060 ceph-mon[40290]: 3.14 scrub starts 2024-09-20T22:08:30.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:30 smithi060 ceph-mon[40290]: 3.14 scrub ok 2024-09-20T22:08:30.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:30 smithi060 ceph-mon[40290]: 3.17 scrub starts 2024-09-20T22:08:30.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:30 smithi060 ceph-mon[40290]: 3.17 scrub ok 2024-09-20T22:08:30.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:30 smithi060 ceph-mon[40290]: pgmap v276: 148 pgs: 148 active+clean; 3.5 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 852 B/s rd, 76 MiB/s wr, 1.22k op/s; 54 B/s, 2 objects/s recovering 2024-09-20T22:08:30.820 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:30.858 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:30.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:30.859 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:30.959 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:31.002 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:31.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:31.003 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:31.103 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:31.130 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:31.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:31.131 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:31.140 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 34 16 64592 64576 118.675 80.4375 0.013735 0.00842124 2024-09-20T22:08:31.231 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:31.267 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:31.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:31.268 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:31.369 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:31.379 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:31.399 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:31.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:31.401 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:31.412 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:31.461 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:31.466 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:08:31.466 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:08:31.500 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:08:31.501 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:08:31.501 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:31.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:31 smithi060 ceph-mon[40290]: 4.f scrub starts 2024-09-20T22:08:31.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:31 smithi060 ceph-mon[40290]: 4.f scrub ok 2024-09-20T22:08:31.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:31 smithi060 ceph-mon[40290]: 3.a scrub starts 2024-09-20T22:08:31.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:31 smithi060 ceph-mon[40290]: 3.a scrub ok 2024-09-20T22:08:31.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:31 smithi060 ceph-mon[40290]: 4.1e scrub starts 2024-09-20T22:08:31.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:31 smithi060 ceph-mon[40290]: 4.1e scrub ok 2024-09-20T22:08:31.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:31 smithi060 ceph-mon[29694]: 4.f scrub starts 2024-09-20T22:08:31.502 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:31 smithi060 ceph-mon[29694]: 4.f scrub ok 2024-09-20T22:08:31.503 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:31 smithi060 ceph-mon[29694]: 3.a scrub starts 2024-09-20T22:08:31.503 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:31 smithi060 ceph-mon[29694]: 3.a scrub ok 2024-09-20T22:08:31.503 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:31 smithi060 ceph-mon[29694]: 4.1e scrub starts 2024-09-20T22:08:31.503 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:31 smithi060 ceph-mon[29694]: 4.1e scrub ok 2024-09-20T22:08:31.535 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:31.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:31.536 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:31.539 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:08:31.540 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:08:31.574 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:08:31.574 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:08:31.612 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:08:31.612 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:08:31.637 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:31.646 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:08:31.646 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:08:31.678 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:31.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:31.679 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:31.682 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:08:31.682 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:08:31.719 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:08:31.719 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:08:31.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:31 smithi067 ceph-mon[33511]: 4.f scrub starts 2024-09-20T22:08:31.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:31 smithi067 ceph-mon[33511]: 4.f scrub ok 2024-09-20T22:08:31.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:31 smithi067 ceph-mon[33511]: 3.a scrub starts 2024-09-20T22:08:31.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:31 smithi067 ceph-mon[33511]: 3.a scrub ok 2024-09-20T22:08:31.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:31 smithi067 ceph-mon[33511]: 4.1e scrub starts 2024-09-20T22:08:31.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:31 smithi067 ceph-mon[33511]: 4.1e scrub ok 2024-09-20T22:08:31.780 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:31.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:31.823 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:31.823 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:31.924 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:31.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:31.960 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:31.960 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:32.061 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:32.091 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:32.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:32.092 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:32.140 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 35 16 65851 65835 117.533 78.6875 0.0128931 0.00850287 2024-09-20T22:08:32.194 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:32.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:32.244 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:32.245 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:32.345 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:32.392 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:32.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:32.393 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:32.422 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:08:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:08:32.494 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:32.520 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:32.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:32.521 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:32.622 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:32.651 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:32.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:32.652 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:32.705 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:08:32.706 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:32 smithi060 ceph-mon[29694]: 3.c scrub starts 2024-09-20T22:08:32.706 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:32 smithi060 ceph-mon[29694]: 3.c scrub ok 2024-09-20T22:08:32.706 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:32 smithi060 ceph-mon[29694]: 6.14 scrub starts 2024-09-20T22:08:32.706 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:32 smithi060 ceph-mon[29694]: 6.14 scrub ok 2024-09-20T22:08:32.706 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:32 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/992663595' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:32.707 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:32 smithi060 ceph-mon[29694]: 6.5 scrub starts 2024-09-20T22:08:32.707 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:32 smithi060 ceph-mon[29694]: 6.5 scrub ok 2024-09-20T22:08:32.707 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:32 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:08:32.707 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:32 smithi060 ceph-mon[29694]: pgmap v277: 148 pgs: 148 active+clean; 3.9 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.3 KiB/s rd, 99 MiB/s wr, 1.58k op/s; 41 B/s, 1 objects/s recovering 2024-09-20T22:08:32.707 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:32 smithi060 ceph-mon[40290]: 3.c scrub starts 2024-09-20T22:08:32.707 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:32 smithi060 ceph-mon[40290]: 3.c scrub ok 2024-09-20T22:08:32.707 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:32 smithi060 ceph-mon[40290]: 6.14 scrub starts 2024-09-20T22:08:32.707 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:32 smithi060 ceph-mon[40290]: 6.14 scrub ok 2024-09-20T22:08:32.707 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:32 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/992663595' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:32.708 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:32 smithi060 ceph-mon[40290]: 6.5 scrub starts 2024-09-20T22:08:32.708 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:32 smithi060 ceph-mon[40290]: 6.5 scrub ok 2024-09-20T22:08:32.708 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:32 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:08:32.708 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:32 smithi060 ceph-mon[40290]: pgmap v277: 148 pgs: 148 active+clean; 3.9 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.3 KiB/s rd, 99 MiB/s wr, 1.58k op/s; 41 B/s, 1 objects/s recovering 2024-09-20T22:08:32.713 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:08:32.753 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:32.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:32 smithi067 ceph-mon[33511]: 3.c scrub starts 2024-09-20T22:08:32.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:32 smithi067 ceph-mon[33511]: 3.c scrub ok 2024-09-20T22:08:32.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:32 smithi067 ceph-mon[33511]: 6.14 scrub starts 2024-09-20T22:08:32.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:32 smithi067 ceph-mon[33511]: 6.14 scrub ok 2024-09-20T22:08:32.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:32 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/992663595' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:32.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:32 smithi067 ceph-mon[33511]: 6.5 scrub starts 2024-09-20T22:08:32.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:32 smithi067 ceph-mon[33511]: 6.5 scrub ok 2024-09-20T22:08:32.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:32 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:08:32.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:32 smithi067 ceph-mon[33511]: pgmap v277: 148 pgs: 148 active+clean; 3.9 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.3 KiB/s rd, 99 MiB/s wr, 1.58k op/s; 41 B/s, 1 objects/s recovering 2024-09-20T22:08:32.781 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:32.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:32.782 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:32.883 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:32.917 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:32.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:32.918 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:33.019 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:33.059 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:33.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:33.060 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:33.142 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 36 16 67117 67101 116.46 79.125 0.00982214 0.00858227 2024-09-20T22:08:33.161 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:33.191 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:33.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:33.192 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:33.293 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:33.327 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:33.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:33.328 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:33.429 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:33.450 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:08:33.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:33.513 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:33.513 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:33.614 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:33.648 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:33.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:33.649 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:33.750 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:33.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:33 smithi067 ceph-mon[33511]: 4.16 scrub starts 2024-09-20T22:08:33.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:33 smithi067 ceph-mon[33511]: 4.16 scrub ok 2024-09-20T22:08:33.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:33 smithi067 ceph-mon[33511]: 6.a scrub starts 2024-09-20T22:08:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:33 smithi067 ceph-mon[33511]: 6.a scrub ok 2024-09-20T22:08:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:33 smithi067 ceph-mon[33511]: 4.1f scrub starts 2024-09-20T22:08:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:33 smithi067 ceph-mon[33511]: 4.1f scrub ok 2024-09-20T22:08:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:33 smithi067 ceph-mon[33511]: 6.17 scrub starts 2024-09-20T22:08:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:33 smithi067 ceph-mon[33511]: 6.17 scrub ok 2024-09-20T22:08:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:33 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1343525017' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:08:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:33 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:08:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:33 smithi067 ceph-mon[33511]: from='client.25124 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:08:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:33 smithi067 ceph-mon[33511]: 5.9 scrub starts 2024-09-20T22:08:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:33 smithi067 ceph-mon[33511]: 5.9 scrub ok 2024-09-20T22:08:33.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:33 smithi067 ceph-mon[33511]: 5.1e scrub starts 2024-09-20T22:08:33.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:33 smithi067 ceph-mon[33511]: 5.1e scrub ok 2024-09-20T22:08:33.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[29694]: 4.16 scrub starts 2024-09-20T22:08:33.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[29694]: 4.16 scrub ok 2024-09-20T22:08:33.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[29694]: 6.a scrub starts 2024-09-20T22:08:33.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[29694]: 6.a scrub ok 2024-09-20T22:08:33.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[29694]: 4.1f scrub starts 2024-09-20T22:08:33.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[29694]: 4.1f scrub ok 2024-09-20T22:08:33.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[29694]: 6.17 scrub starts 2024-09-20T22:08:33.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[29694]: 6.17 scrub ok 2024-09-20T22:08:33.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1343525017' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:08:33.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:08:33.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[29694]: from='client.25124 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:08:33.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[29694]: 5.9 scrub starts 2024-09-20T22:08:33.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[29694]: 5.9 scrub ok 2024-09-20T22:08:33.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[29694]: 5.1e scrub starts 2024-09-20T22:08:33.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[29694]: 5.1e scrub ok 2024-09-20T22:08:33.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[40290]: 4.16 scrub starts 2024-09-20T22:08:33.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[40290]: 4.16 scrub ok 2024-09-20T22:08:33.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[40290]: 6.a scrub starts 2024-09-20T22:08:33.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[40290]: 6.a scrub ok 2024-09-20T22:08:33.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[40290]: 4.1f scrub starts 2024-09-20T22:08:33.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[40290]: 4.1f scrub ok 2024-09-20T22:08:33.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[40290]: 6.17 scrub starts 2024-09-20T22:08:33.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[40290]: 6.17 scrub ok 2024-09-20T22:08:33.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1343525017' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:08:33.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:08:33.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[40290]: from='client.25124 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:08:33.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[40290]: 5.9 scrub starts 2024-09-20T22:08:33.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[40290]: 5.9 scrub ok 2024-09-20T22:08:33.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[40290]: 5.1e scrub starts 2024-09-20T22:08:33.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:33 smithi060 ceph-mon[40290]: 5.1e scrub ok 2024-09-20T22:08:33.791 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:33.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:33.792 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:33.893 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:33.927 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:33.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:33.928 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:34.029 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:34.059 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:34.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:34.060 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:34.142 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 37 16 68528 68512 115.696 88.1875 0.00628349 0.00863817 2024-09-20T22:08:34.161 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:34.197 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:34.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:34.198 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:34.299 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:34.332 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:34.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:34.333 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:34.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:34.472 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:34.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:34.473 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:34.574 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:34.625 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:34.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:34.626 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:34.718 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:34 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:08:34.718 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:34 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:08:34.718 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:34 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:08:34.718 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:34 smithi060 ceph-mon[29694]: osdmap e192: 8 total, 8 up, 8 in 2024-09-20T22:08:34.719 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:34 smithi060 ceph-mon[29694]: 4.17 scrub starts 2024-09-20T22:08:34.719 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:34 smithi060 ceph-mon[29694]: 4.17 scrub ok 2024-09-20T22:08:34.719 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:34 smithi060 ceph-mon[29694]: 4.3 scrub starts 2024-09-20T22:08:34.719 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:34 smithi060 ceph-mon[29694]: 4.3 scrub ok 2024-09-20T22:08:34.719 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:34 smithi060 ceph-mon[29694]: pgmap v279: 148 pgs: 148 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 90 MiB/s wr, 1.44k op/s 2024-09-20T22:08:34.719 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:34 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:08:34.719 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:34 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:08:34.719 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:34 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:08:34.719 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:34 smithi060 ceph-mon[40290]: osdmap e192: 8 total, 8 up, 8 in 2024-09-20T22:08:34.720 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:34 smithi060 ceph-mon[40290]: 4.17 scrub starts 2024-09-20T22:08:34.720 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:34 smithi060 ceph-mon[40290]: 4.17 scrub ok 2024-09-20T22:08:34.720 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:34 smithi060 ceph-mon[40290]: 4.3 scrub starts 2024-09-20T22:08:34.720 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:34 smithi060 ceph-mon[40290]: 4.3 scrub ok 2024-09-20T22:08:34.720 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:34 smithi060 ceph-mon[40290]: pgmap v279: 148 pgs: 148 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 90 MiB/s wr, 1.44k op/s 2024-09-20T22:08:34.727 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:34.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:34 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:08:34.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:34 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:08:34.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:34 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:08:34.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:34 smithi067 ceph-mon[33511]: osdmap e192: 8 total, 8 up, 8 in 2024-09-20T22:08:34.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:34 smithi067 ceph-mon[33511]: 4.17 scrub starts 2024-09-20T22:08:34.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:34 smithi067 ceph-mon[33511]: 4.17 scrub ok 2024-09-20T22:08:34.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:34 smithi067 ceph-mon[33511]: 4.3 scrub starts 2024-09-20T22:08:34.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:34 smithi067 ceph-mon[33511]: 4.3 scrub ok 2024-09-20T22:08:34.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:34 smithi067 ceph-mon[33511]: pgmap v279: 148 pgs: 148 active+clean; 4.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 90 MiB/s wr, 1.44k op/s 2024-09-20T22:08:34.768 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:34.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:34.769 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:34.870 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:34.901 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:34.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:34.902 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:35.003 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:35.043 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:35.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:35.044 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:35.142 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 38 16 69864 69848 114.849 83.5 0.0102607 0.00870206 2024-09-20T22:08:35.145 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:35.182 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:35.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:35.183 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:35.284 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:35.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:08:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:08:35.320 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:35.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:35.321 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:35.417 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-20T22:08:32.030244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4142602779,"num_objects":63401,"num_object_clones":0,"num_object_copies":190203,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63401,"num_whiteouts":0,"num_read":1256,"num_read_kb":1102,"num_write":63817,"num_write_kb":4047389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1771434,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63815,"ondisk_log_size":63815,"up":444,"acting":444,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":444,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":13921184,"kb_used_data":12096156,"kb_used_omap":15,"kb_used_meta":1824880,"kb_avail":736040032,"statfs":{"total":767960285184,"available":753704992768,"internally_reserved":0,"allocated":12386463744,"data_stored":12379997321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":1868677907},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,5,8,6,9],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":817954816,"num_objects":12481,"num_object_clones":0,"num_object_copies":37443,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12481,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":12481,"num_write_kb":798784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.888367"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":282,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395193+0000","last_change":"2024-09-20T22:06:48.018571+0000","last_active":"2024-09-20T22:08:26.395193+0000","last_peered":"2024-09-20T22:08:26.395193+0000","last_clean":"2024-09-20T22:08:26.395193+0000","last_became_active":"2024-09-20T22:02:50.109930+0000","last_became_peered":"2024-09-20T22:02:50.109930+0000","last_unstale":"2024-09-20T22:08:26.395193+0000","last_undegraded":"2024-09-20T22:08:26.395193+0000","last_fullsized":"2024-09-20T22:08:26.395193+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_clean_scrub_stamp":"2024-09-20T22:06:48.018157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"71'9","reported_seq":304,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.394607+0000","last_change":"2024-09-20T22:07:55.492953+0000","last_active":"2024-09-20T22:08:26.394607+0000","last_peered":"2024-09-20T22:08:26.394607+0000","last_clean":"2024-09-20T22:08:26.394607+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:08:26.394607+0000","last_undegraded":"2024-09-20T22:08:26.394607+0000","last_fullsized":"2024-09-20T22:08:26.394607+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:55.492841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.492841+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:24.417494+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":163,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.396805+0000","last_change":"2024-09-20T22:06:54.843328+0000","last_active":"2024-09-20T22:08:26.396805+0000","last_peered":"2024-09-20T22:08:26.396805+0000","last_clean":"2024-09-20T22:08:26.396805+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:08:26.396805+0000","last_undegraded":"2024-09-20T22:08:26.396805+0000","last_fullsized":"2024-09-20T22:08:26.396805+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:11.178591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:11.178591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:28.363460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":272,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.394799+0000","last_change":"2024-09-20T22:07:14.535725+0000","last_active":"2024-09-20T22:08:26.394799+0000","last_peered":"2024-09-20T22:08:26.394799+0000","last_clean":"2024-09-20T22:08:26.394799+0000","last_became_active":"2024-09-20T22:06:54.830769+0000","last_became_peered":"2024-09-20T22:06:54.830769+0000","last_unstale":"2024-09-20T22:08:26.394799+0000","last_undegraded":"2024-09-20T22:08:26.394799+0000","last_fullsized":"2024-09-20T22:08:26.394799+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.535618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.535618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:36.228377+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":263,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.401355+0000","last_change":"2024-09-20T22:05:06.024320+0000","last_active":"2024-09-20T22:08:26.401355+0000","last_peered":"2024-09-20T22:08:26.401355+0000","last_clean":"2024-09-20T22:08:26.401355+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:08:26.401355+0000","last_undegraded":"2024-09-20T22:08:26.401355+0000","last_fullsized":"2024-09-20T22:08:26.401355+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:05:06.024051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:06.024051+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":274,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.398991+0000","last_change":"2024-09-20T22:05:04.976359+0000","last_active":"2024-09-20T22:08:26.398991+0000","last_peered":"2024-09-20T22:08:26.398991+0000","last_clean":"2024-09-20T22:08:26.398991+0000","last_became_active":"2024-09-20T22:02:50.107154+0000","last_became_peered":"2024-09-20T22:02:50.107154+0000","last_unstale":"2024-09-20T22:08:26.398991+0000","last_undegraded":"2024-09-20T22:08:26.398991+0000","last_fullsized":"2024-09-20T22:08:26.398991+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:04.976218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:04.976218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"71'15","reported_seq":295,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.399177+0000","last_change":"2024-09-20T22:04:15.767473+0000","last_active":"2024-09-20T22:08:26.399177+0000","last_peered":"2024-09-20T22:08:26.399177+0000","last_clean":"2024-09-20T22:08:26.399177+0000","last_became_active":"2024-09-20T22:02:52.282101+0000","last_became_peered":"2024-09-20T22:02:52.282101+0000","last_unstale":"2024-09-20T22:08:26.399177+0000","last_undegraded":"2024-09-20T22:08:26.399177+0000","last_fullsized":"2024-09-20T22:08:26.399177+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:04:15.767389+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:15.767389+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":276,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.394514+0000","last_change":"2024-09-20T22:07:17.494965+0000","last_active":"2024-09-20T22:08:26.394514+0000","last_peered":"2024-09-20T22:08:26.394514+0000","last_clean":"2024-09-20T22:08:26.394514+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:08:26.394514+0000","last_undegraded":"2024-09-20T22:08:26.394514+0000","last_fullsized":"2024-09-20T22:08:26.394514+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:17.494852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:17.494852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:40.846112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":290,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.399858+0000","last_change":"2024-09-20T22:06:46.616824+0000","last_active":"2024-09-20T22:08:26.399858+0000","last_peered":"2024-09-20T22:08:26.399858+0000","last_clean":"2024-09-20T22:08:26.399858+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:08:26.399858+0000","last_undegraded":"2024-09-20T22:08:26.399858+0000","last_fullsized":"2024-09-20T22:08:26.399858+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:46.616725+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:46.616725+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"71'9","reported_seq":295,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395124+0000","last_change":"2024-09-20T22:06:09.729638+0000","last_active":"2024-09-20T22:08:26.395124+0000","last_peered":"2024-09-20T22:08:26.395124+0000","last_clean":"2024-09-20T22:08:26.395124+0000","last_became_active":"2024-09-20T22:02:52.279526+0000","last_became_peered":"2024-09-20T22:02:52.279526+0000","last_unstale":"2024-09-20T22:08:26.395124+0000","last_undegraded":"2024-09-20T22:08:26.395124+0000","last_fullsized":"2024-09-20T22:08:26.395124+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:06:09.729513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:09.729513+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547520+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":282,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.398970+0000","last_change":"2024-09-20T22:07:14.890410+0000","last_active":"2024-09-20T22:08:26.398970+0000","last_peered":"2024-09-20T22:08:26.398970+0000","last_clean":"2024-09-20T22:08:26.398970+0000","last_became_active":"2024-09-20T22:02:54.292911+0000","last_became_peered":"2024-09-20T22:02:54.292911+0000","last_unstale":"2024-09-20T22:08:26.398970+0000","last_undegraded":"2024-09-20T22:08:26.398970+0000","last_fullsized":"2024-09-20T22:08:26.398970+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.890308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.890308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:29.100670+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":271,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.393874+0000","last_change":"2024-09-20T22:07:15.416312+0000","last_active":"2024-09-20T22:08:26.393874+0000","last_peered":"2024-09-20T22:08:26.393874+0000","last_clean":"2024-09-20T22:08:26.393874+0000","last_became_active":"2024-09-20T22:06:54.841095+0000","last_became_peered":"2024-09-20T22:06:54.841095+0000","last_unstale":"2024-09-20T22:08:26.393874+0000","last_undegraded":"2024-09-20T22:08:26.393874+0000","last_fullsized":"2024-09-20T22:08:26.393874+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:15.416176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.416176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":303,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.399259+0000","last_change":"2024-09-20T22:07:59.947914+0000","last_active":"2024-09-20T22:08:26.399259+0000","last_peered":"2024-09-20T22:08:26.399259+0000","last_clean":"2024-09-20T22:08:26.399259+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:08:26.399259+0000","last_undegraded":"2024-09-20T22:08:26.399259+0000","last_fullsized":"2024-09-20T22:08:26.399259+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:59.947741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:59.947741+0000","last_clean_scrub_stamp":"2024-09-20T22:07:59.947741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:29.533391+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"71'5","reported_seq":310,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395341+0000","last_change":"2024-09-20T22:08:22.151586+0000","last_active":"2024-09-20T22:08:26.395341+0000","last_peered":"2024-09-20T22:08:26.395341+0000","last_clean":"2024-09-20T22:08:26.395341+0000","last_became_active":"2024-09-20T22:08:22.144364+0000","last_became_peered":"2024-09-20T22:08:22.144364+0000","last_unstale":"2024-09-20T22:08:26.395341+0000","last_undegraded":"2024-09-20T22:08:26.395341+0000","last_fullsized":"2024-09-20T22:08:26.395341+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'5","last_scrub_stamp":"2024-09-20T22:08:01.462581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:01.462581+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:24.969371+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":284,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.401591+0000","last_change":"2024-09-20T22:07:28.654301+0000","last_active":"2024-09-20T22:08:26.401591+0000","last_peered":"2024-09-20T22:08:26.401591+0000","last_clean":"2024-09-20T22:08:26.401591+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:08:26.401591+0000","last_undegraded":"2024-09-20T22:08:26.401591+0000","last_fullsized":"2024-09-20T22:08:26.401591+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:28.654202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:28.654202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:41.331246+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":216,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.394539+0000","last_change":"2024-09-20T22:06:55.824661+0000","last_active":"2024-09-20T22:08:26.394539+0000","last_peered":"2024-09-20T22:08:26.394539+0000","last_clean":"2024-09-20T22:08:26.394539+0000","last_became_active":"2024-09-20T22:06:55.824168+0000","last_became_peered":"2024-09-20T22:06:55.824168+0000","last_unstale":"2024-09-20T22:08:26.394539+0000","last_undegraded":"2024-09-20T22:08:26.394539+0000","last_fullsized":"2024-09-20T22:08:26.394539+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:05:29.551172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":271,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395809+0000","last_change":"2024-09-20T22:06:47.667293+0000","last_active":"2024-09-20T22:08:26.395809+0000","last_peered":"2024-09-20T22:08:26.395809+0000","last_clean":"2024-09-20T22:08:26.395809+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:08:26.395809+0000","last_undegraded":"2024-09-20T22:08:26.395809+0000","last_fullsized":"2024-09-20T22:08:26.395809+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:06:47.667180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547546+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":274,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395437+0000","last_change":"2024-09-20T22:05:31.552599+0000","last_active":"2024-09-20T22:08:26.395437+0000","last_peered":"2024-09-20T22:08:26.395437+0000","last_clean":"2024-09-20T22:08:26.395437+0000","last_became_active":"2024-09-20T22:02:50.111517+0000","last_became_peered":"2024-09-20T22:02:50.111517+0000","last_unstale":"2024-09-20T22:08:26.395437+0000","last_undegraded":"2024-09-20T22:08:26.395437+0000","last_fullsized":"2024-09-20T22:08:26.395437+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:31.552474+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:31.552474+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"71'15","reported_seq":312,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.403868+0000","last_change":"2024-09-20T22:07:16.215766+0000","last_active":"2024-09-20T22:08:26.403868+0000","last_peered":"2024-09-20T22:08:26.403868+0000","last_clean":"2024-09-20T22:08:26.403868+0000","last_became_active":"2024-09-20T22:06:54.846111+0000","last_became_peered":"2024-09-20T22:06:54.846111+0000","last_unstale":"2024-09-20T22:08:26.403868+0000","last_undegraded":"2024-09-20T22:08:26.403868+0000","last_fullsized":"2024-09-20T22:08:26.403868+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:07:16.215638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:16.215638+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":120,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.396927+0000","last_change":"2024-09-20T22:08:00.369883+0000","last_active":"2024-09-20T22:08:26.396927+0000","last_peered":"2024-09-20T22:08:26.396927+0000","last_clean":"2024-09-20T22:08:26.396927+0000","last_became_active":"2024-09-20T22:06:54.848505+0000","last_became_peered":"2024-09-20T22:06:54.848505+0000","last_unstale":"2024-09-20T22:08:26.396927+0000","last_undegraded":"2024-09-20T22:08:26.396927+0000","last_fullsized":"2024-09-20T22:08:26.396927+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:00.369767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:00.369767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:20.935739+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":270,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.400091+0000","last_change":"2024-09-20T22:06:34.944093+0000","last_active":"2024-09-20T22:08:26.400091+0000","last_peered":"2024-09-20T22:08:26.400091+0000","last_clean":"2024-09-20T22:08:26.400091+0000","last_became_active":"2024-09-20T22:02:56.347561+0000","last_became_peered":"2024-09-20T22:02:56.347561+0000","last_unstale":"2024-09-20T22:08:26.400091+0000","last_undegraded":"2024-09-20T22:08:26.400091+0000","last_fullsized":"2024-09-20T22:08:26.400091+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:34.940758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:34.940758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":232,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:28.058565+0000","last_change":"2024-09-20T22:08:28.058586+0000","last_active":"2024-09-20T22:08:28.058565+0000","last_peered":"2024-09-20T22:08:28.058565+0000","last_clean":"2024-09-20T22:08:28.058565+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:08:28.058565+0000","last_undegraded":"2024-09-20T22:08:28.058565+0000","last_fullsized":"2024-09-20T22:08:28.058565+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:28.058516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:28.058516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:50.545327+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"71'12","reported_seq":302,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.398755+0000","last_change":"2024-09-20T22:07:55.383903+0000","last_active":"2024-09-20T22:08:26.398755+0000","last_peered":"2024-09-20T22:08:26.398755+0000","last_clean":"2024-09-20T22:08:26.398755+0000","last_became_active":"2024-09-20T22:06:54.830946+0000","last_became_peered":"2024-09-20T22:06:54.830946+0000","last_unstale":"2024-09-20T22:08:26.398755+0000","last_undegraded":"2024-09-20T22:08:26.398755+0000","last_fullsized":"2024-09-20T22:08:26.398755+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:55.383818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.383818+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:56.321505+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":267,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.394864+0000","last_change":"2024-09-20T22:05:28.003302+0000","last_active":"2024-09-20T22:08:26.394864+0000","last_peered":"2024-09-20T22:08:26.394864+0000","last_clean":"2024-09-20T22:08:26.394864+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:08:26.394864+0000","last_undegraded":"2024-09-20T22:08:26.394864+0000","last_fullsized":"2024-09-20T22:08:26.394864+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.003183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547263+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":265,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.401827+0000","last_change":"2024-09-20T22:05:33.751965+0000","last_active":"2024-09-20T22:08:26.401827+0000","last_peered":"2024-09-20T22:08:26.401827+0000","last_clean":"2024-09-20T22:08:26.401827+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:08:26.401827+0000","last_undegraded":"2024-09-20T22:08:26.401827+0000","last_fullsized":"2024-09-20T22:08:26.401827+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:05:33.751883+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"64'1","reported_seq":114,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.396881+0000","last_change":"2024-09-20T22:06:54.854003+0000","last_active":"2024-09-20T22:08:26.396881+0000","last_peered":"2024-09-20T22:08:26.396881+0000","last_clean":"2024-09-20T22:08:26.396881+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:08:26.396881+0000","last_undegraded":"2024-09-20T22:08:26.396881+0000","last_fullsized":"2024-09-20T22:08:26.396881+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:28.363362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"71'10","reported_seq":293,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:30.868876+0000","last_change":"2024-09-20T22:08:30.868926+0000","last_active":"2024-09-20T22:08:30.868876+0000","last_peered":"2024-09-20T22:08:30.868876+0000","last_clean":"2024-09-20T22:08:30.868876+0000","last_became_active":"2024-09-20T22:02:52.481980+0000","last_became_peered":"2024-09-20T22:02:52.481980+0000","last_unstale":"2024-09-20T22:08:30.868876+0000","last_undegraded":"2024-09-20T22:08:30.868876+0000","last_fullsized":"2024-09-20T22:08:30.868876+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:08:30.868781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:30.868781+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:54.682307+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"71'8","reported_seq":362,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:29.377058+0000","last_change":"2024-09-20T22:08:29.377058+0000","last_active":"2024-09-20T22:08:29.377058+0000","last_peered":"2024-09-20T22:08:29.377058+0000","last_clean":"2024-09-20T22:08:29.377058+0000","last_became_active":"2024-09-20T22:02:54.297657+0000","last_became_peered":"2024-09-20T22:02:54.297657+0000","last_unstale":"2024-09-20T22:08:29.377058+0000","last_undegraded":"2024-09-20T22:08:29.377058+0000","last_fullsized":"2024-09-20T22:08:29.377058+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:08:29.376948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:29.376948+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:56.622742+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"191'3771","reported_seq":3811,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:27.646380+0000","last_change":"2024-09-20T22:07:52.429150+0000","last_active":"2024-09-20T22:08:27.646380+0000","last_peered":"2024-09-20T22:08:27.646380+0000","last_clean":"2024-09-20T22:08:27.646380+0000","last_became_active":"2024-09-20T22:07:52.428105+0000","last_became_peered":"2024-09-20T22:07:52.428105+0000","last_unstale":"2024-09-20T22:08:27.646380+0000","last_undegraded":"2024-09-20T22:08:27.646380+0000","last_fullsized":"2024-09-20T22:08:27.646380+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3771,"log_dups_size":0,"ondisk_log_size":3771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:04.623212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":247136256,"num_objects":3771,"num_object_clones":0,"num_object_copies":11313,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3771,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3771,"num_write_kb":241344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"71'15","reported_seq":41,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:30.118861+0000","last_change":"2024-09-20T22:08:30.118912+0000","last_active":"2024-09-20T22:08:30.118861+0000","last_peered":"2024-09-20T22:08:30.118861+0000","last_clean":"2024-09-20T22:08:30.118861+0000","last_became_active":"2024-09-20T22:08:22.145408+0000","last_became_peered":"2024-09-20T22:08:22.145408+0000","last_unstale":"2024-09-20T22:08:30.118861+0000","last_undegraded":"2024-09-20T22:08:30.118861+0000","last_fullsized":"2024-09-20T22:08:30.118861+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:08:30.118638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:30.118638+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:30.162802+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":283,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.400270+0000","last_change":"2024-09-20T22:07:23.292560+0000","last_active":"2024-09-20T22:08:26.400270+0000","last_peered":"2024-09-20T22:08:26.400270+0000","last_clean":"2024-09-20T22:08:26.400270+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:08:26.400270+0000","last_undegraded":"2024-09-20T22:08:26.400270+0000","last_fullsized":"2024-09-20T22:08:26.400270+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.292451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.292451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"71'8","reported_seq":355,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395931+0000","last_change":"2024-09-20T22:06:04.874166+0000","last_active":"2024-09-20T22:08:26.395931+0000","last_peered":"2024-09-20T22:08:26.395931+0000","last_clean":"2024-09-20T22:08:26.395931+0000","last_became_active":"2024-09-20T22:02:54.294917+0000","last_became_peered":"2024-09-20T22:02:54.294917+0000","last_unstale":"2024-09-20T22:08:26.395931+0000","last_undegraded":"2024-09-20T22:08:26.395931+0000","last_fullsized":"2024-09-20T22:08:26.395931+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:06:04.874042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:04.874042+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":232,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.397194+0000","last_change":"2024-09-20T22:07:52.334831+0000","last_active":"2024-09-20T22:08:26.397194+0000","last_peered":"2024-09-20T22:08:26.397194+0000","last_clean":"2024-09-20T22:08:26.397194+0000","last_became_active":"2024-09-20T22:06:55.822780+0000","last_became_peered":"2024-09-20T22:06:55.822780+0000","last_unstale":"2024-09-20T22:08:26.397194+0000","last_undegraded":"2024-09-20T22:08:26.397194+0000","last_fullsized":"2024-09-20T22:08:26.397194+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:52.334748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:52.334748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:53.481596+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.c","version":"191'4025","reported_seq":4064,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:31.300820+0000","last_change":"2024-09-20T22:07:52.674862+0000","last_active":"2024-09-20T22:08:31.300820+0000","last_peered":"2024-09-20T22:08:31.300820+0000","last_clean":"2024-09-20T22:08:31.300820+0000","last_became_active":"2024-09-20T22:07:52.674521+0000","last_became_peered":"2024-09-20T22:07:52.674521+0000","last_unstale":"2024-09-20T22:08:31.300820+0000","last_undegraded":"2024-09-20T22:08:31.300820+0000","last_fullsized":"2024-09-20T22:08:31.300820+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4025,"log_dups_size":0,"ondisk_log_size":4025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:14.102427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":263782400,"num_objects":4025,"num_object_clones":0,"num_object_copies":12075,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4025,"num_write_kb":257600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"71'18","reported_seq":302,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.400038+0000","last_change":"2024-09-20T22:04:57.967494+0000","last_active":"2024-09-20T22:08:26.400038+0000","last_peered":"2024-09-20T22:08:26.400038+0000","last_clean":"2024-09-20T22:08:26.400038+0000","last_became_active":"2024-09-20T22:02:52.482119+0000","last_became_peered":"2024-09-20T22:02:52.482119+0000","last_unstale":"2024-09-20T22:08:26.400038+0000","last_undegraded":"2024-09-20T22:08:26.400038+0000","last_fullsized":"2024-09-20T22:08:26.400038+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'18","last_scrub_stamp":"2024-09-20T22:04:57.967216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:57.967216+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":274,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.400520+0000","last_change":"2024-09-20T22:05:02.924002+0000","last_active":"2024-09-20T22:08:26.400520+0000","last_peered":"2024-09-20T22:08:26.400520+0000","last_clean":"2024-09-20T22:08:26.400520+0000","last_became_active":"2024-09-20T22:02:50.113076+0000","last_became_peered":"2024-09-20T22:02:50.113076+0000","last_unstale":"2024-09-20T22:08:26.400520+0000","last_undegraded":"2024-09-20T22:08:26.400520+0000","last_fullsized":"2024-09-20T22:08:26.400520+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:02.923565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:02.923565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":275,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.394788+0000","last_change":"2024-09-20T22:06:05.894144+0000","last_active":"2024-09-20T22:08:26.394788+0000","last_peered":"2024-09-20T22:08:26.394788+0000","last_clean":"2024-09-20T22:08:26.394788+0000","last_became_active":"2024-09-20T22:02:54.288007+0000","last_became_peered":"2024-09-20T22:02:54.288007+0000","last_unstale":"2024-09-20T22:08:26.394788+0000","last_undegraded":"2024-09-20T22:08:26.394788+0000","last_fullsized":"2024-09-20T22:08:26.394788+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:05.894031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:05.894031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547501+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":271,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395042+0000","last_change":"2024-09-20T22:05:29.990030+0000","last_active":"2024-09-20T22:08:26.395042+0000","last_peered":"2024-09-20T22:08:26.395042+0000","last_clean":"2024-09-20T22:08:26.395042+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:08:26.395042+0000","last_undegraded":"2024-09-20T22:08:26.395042+0000","last_fullsized":"2024-09-20T22:08:26.395042+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:29.989934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:29.989934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547290+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.d","version":"191'4008","reported_seq":4046,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:30.646780+0000","last_change":"2024-09-20T22:07:52.940943+0000","last_active":"2024-09-20T22:08:30.646780+0000","last_peered":"2024-09-20T22:08:30.646780+0000","last_clean":"2024-09-20T22:08:30.646780+0000","last_became_active":"2024-09-20T22:07:52.940627+0000","last_became_peered":"2024-09-20T22:07:52.940627+0000","last_unstale":"2024-09-20T22:08:30.646780+0000","last_undegraded":"2024-09-20T22:08:30.646780+0000","last_fullsized":"2024-09-20T22:08:30.646780+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4008,"log_dups_size":0,"ondisk_log_size":4008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:11.794238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262668288,"num_objects":4008,"num_object_clones":0,"num_object_copies":12024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4008,"num_write_kb":256512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"71'14","reported_seq":300,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395060+0000","last_change":"2024-09-20T22:06:00.944172+0000","last_active":"2024-09-20T22:08:26.395060+0000","last_peered":"2024-09-20T22:08:26.395060+0000","last_clean":"2024-09-20T22:08:26.395060+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:08:26.395060+0000","last_undegraded":"2024-09-20T22:08:26.395060+0000","last_fullsized":"2024-09-20T22:08:26.395060+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'14","last_scrub_stamp":"2024-09-20T22:06:00.944086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.944086+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547461+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":284,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395600+0000","last_change":"2024-09-20T22:07:58.783500+0000","last_active":"2024-09-20T22:08:26.395600+0000","last_peered":"2024-09-20T22:08:26.395600+0000","last_clean":"2024-09-20T22:08:26.395600+0000","last_became_active":"2024-09-20T22:06:54.841165+0000","last_became_peered":"2024-09-20T22:06:54.841165+0000","last_unstale":"2024-09-20T22:08:26.395600+0000","last_undegraded":"2024-09-20T22:08:26.395600+0000","last_fullsized":"2024-09-20T22:08:26.395600+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:07:58.783380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:58.783380+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:25.086988+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"146'10","reported_seq":354,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.399885+0000","last_change":"2024-09-20T22:07:24.333937+0000","last_active":"2024-09-20T22:08:26.399885+0000","last_peered":"2024-09-20T22:08:26.399885+0000","last_clean":"2024-09-20T22:08:26.399885+0000","last_became_active":"2024-09-20T22:06:54.847742+0000","last_became_peered":"2024-09-20T22:06:54.847742+0000","last_unstale":"2024-09-20T22:08:26.399885+0000","last_undegraded":"2024-09-20T22:08:26.399885+0000","last_fullsized":"2024-09-20T22:08:26.399885+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:07:24.333809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:24.333809+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:46.378931+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":271,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.401863+0000","last_change":"2024-09-20T22:07:30.589937+0000","last_active":"2024-09-20T22:08:26.401863+0000","last_peered":"2024-09-20T22:08:26.401863+0000","last_clean":"2024-09-20T22:08:26.401863+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:08:26.401863+0000","last_undegraded":"2024-09-20T22:08:26.401863+0000","last_fullsized":"2024-09-20T22:08:26.401863+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:30.589849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:30.589849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:57.117488+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.e","version":"191'4091","reported_seq":4130,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:31.277856+0000","last_change":"2024-09-20T22:07:52.940978+0000","last_active":"2024-09-20T22:08:31.277856+0000","last_peered":"2024-09-20T22:08:31.277856+0000","last_clean":"2024-09-20T22:08:31.277856+0000","last_became_active":"2024-09-20T22:07:52.940673+0000","last_became_peered":"2024-09-20T22:07:52.940673+0000","last_unstale":"2024-09-20T22:08:31.277856+0000","last_undegraded":"2024-09-20T22:08:31.277856+0000","last_fullsized":"2024-09-20T22:08:31.277856+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4091,"log_dups_size":0,"ondisk_log_size":4091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.115401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268107776,"num_objects":4091,"num_object_clones":0,"num_object_copies":12273,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4091,"num_write_kb":261824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"71'10","reported_seq":147,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.397659+0000","last_change":"2024-09-20T22:07:59.424108+0000","last_active":"2024-09-20T22:08:26.397659+0000","last_peered":"2024-09-20T22:08:26.397659+0000","last_clean":"2024-09-20T22:08:26.397659+0000","last_became_active":"2024-09-20T22:06:54.844681+0000","last_became_peered":"2024-09-20T22:06:54.844681+0000","last_unstale":"2024-09-20T22:08:26.397659+0000","last_undegraded":"2024-09-20T22:08:26.397659+0000","last_fullsized":"2024-09-20T22:08:26.397659+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:07:59.423970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:59.423970+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:05.043224+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":298,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:29.034854+0000","last_change":"2024-09-20T22:08:29.034890+0000","last_active":"2024-09-20T22:08:29.034854+0000","last_peered":"2024-09-20T22:08:29.034854+0000","last_clean":"2024-09-20T22:08:29.034854+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:08:29.034854+0000","last_undegraded":"2024-09-20T22:08:29.034854+0000","last_fullsized":"2024-09-20T22:08:29.034854+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:29.034641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:08:29.034641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:32.281368+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"71'8","reported_seq":351,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:28.861124+0000","last_change":"2024-09-20T22:08:28.861124+0000","last_active":"2024-09-20T22:08:28.861124+0000","last_peered":"2024-09-20T22:08:28.861124+0000","last_clean":"2024-09-20T22:08:28.861124+0000","last_became_active":"2024-09-20T22:02:54.292819+0000","last_became_peered":"2024-09-20T22:02:54.292819+0000","last_unstale":"2024-09-20T22:08:28.861124+0000","last_undegraded":"2024-09-20T22:08:28.861124+0000","last_fullsized":"2024-09-20T22:08:28.861124+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:08:28.860982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:28.860982+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:39.280471+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":262,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.394015+0000","last_change":"2024-09-20T22:06:00.572347+0000","last_active":"2024-09-20T22:08:26.394015+0000","last_peered":"2024-09-20T22:08:26.394015+0000","last_clean":"2024-09-20T22:08:26.394015+0000","last_became_active":"2024-09-20T22:02:56.304274+0000","last_became_peered":"2024-09-20T22:02:56.304274+0000","last_unstale":"2024-09-20T22:08:26.394015+0000","last_undegraded":"2024-09-20T22:08:26.394015+0000","last_fullsized":"2024-09-20T22:08:26.394015+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:00.572243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.572243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.f","version":"191'4129","reported_seq":4169,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:31.118557+0000","last_change":"2024-09-20T22:07:52.940027+0000","last_active":"2024-09-20T22:08:31.118557+0000","last_peered":"2024-09-20T22:08:31.118557+0000","last_clean":"2024-09-20T22:08:31.118557+0000","last_became_active":"2024-09-20T22:07:52.939692+0000","last_became_peered":"2024-09-20T22:07:52.939692+0000","last_unstale":"2024-09-20T22:08:31.118557+0000","last_undegraded":"2024-09-20T22:08:31.118557+0000","last_fullsized":"2024-09-20T22:08:31.118557+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4129,"log_dups_size":0,"ondisk_log_size":4129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:12.848256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":270598144,"num_objects":4129,"num_object_clones":0,"num_object_copies":12387,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4129,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4129,"num_write_kb":264256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"71'19","reported_seq":314,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.394964+0000","last_change":"2024-09-20T22:05:24.600933+0000","last_active":"2024-09-20T22:08:26.394964+0000","last_peered":"2024-09-20T22:08:26.394964+0000","last_clean":"2024-09-20T22:08:26.394964+0000","last_became_active":"2024-09-20T22:02:52.480077+0000","last_became_peered":"2024-09-20T22:02:52.480077+0000","last_unstale":"2024-09-20T22:08:26.394964+0000","last_undegraded":"2024-09-20T22:08:26.394964+0000","last_fullsized":"2024-09-20T22:08:26.394964+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:05:24.600795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:24.600795+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":121,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.396496+0000","last_change":"2024-09-20T22:07:14.382822+0000","last_active":"2024-09-20T22:08:26.396496+0000","last_peered":"2024-09-20T22:08:26.396496+0000","last_clean":"2024-09-20T22:08:26.396496+0000","last_became_active":"2024-09-20T22:06:54.829037+0000","last_became_peered":"2024-09-20T22:06:54.829037+0000","last_unstale":"2024-09-20T22:08:26.396496+0000","last_undegraded":"2024-09-20T22:08:26.396496+0000","last_fullsized":"2024-09-20T22:08:26.396496+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.382712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.382712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.271274+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":282,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.396569+0000","last_change":"2024-09-20T22:07:30.169138+0000","last_active":"2024-09-20T22:08:26.396569+0000","last_peered":"2024-09-20T22:08:26.396569+0000","last_clean":"2024-09-20T22:08:26.396569+0000","last_became_active":"2024-09-20T22:02:54.290787+0000","last_became_peered":"2024-09-20T22:02:54.290787+0000","last_unstale":"2024-09-20T22:08:26.396569+0000","last_undegraded":"2024-09-20T22:08:26.396569+0000","last_fullsized":"2024-09-20T22:08:26.396569+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:30.169038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:30.169038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:38.467894+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":112,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.396669+0000","last_change":"2024-09-20T22:06:54.820385+0000","last_active":"2024-09-20T22:08:26.396669+0000","last_peered":"2024-09-20T22:08:26.396669+0000","last_clean":"2024-09-20T22:08:26.396669+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:08:26.396669+0000","last_undegraded":"2024-09-20T22:08:26.396669+0000","last_fullsized":"2024-09-20T22:08:26.396669+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:28.363385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"191'4002","reported_seq":4042,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:30.640406+0000","last_change":"2024-09-20T22:07:52.674766+0000","last_active":"2024-09-20T22:08:30.640406+0000","last_peered":"2024-09-20T22:08:30.640406+0000","last_clean":"2024-09-20T22:08:30.640406+0000","last_became_active":"2024-09-20T22:07:52.674363+0000","last_became_peered":"2024-09-20T22:07:52.674363+0000","last_unstale":"2024-09-20T22:08:30.640406+0000","last_undegraded":"2024-09-20T22:08:30.640406+0000","last_fullsized":"2024-09-20T22:08:30.640406+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4002,"log_dups_size":0,"ondisk_log_size":4002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:16.336670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262275072,"num_objects":4002,"num_object_clones":0,"num_object_copies":12006,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4002,"num_write_kb":256128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"146'30","reported_seq":201,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.396567+0000","last_change":"2024-09-20T22:07:55.357829+0000","last_active":"2024-09-20T22:08:26.396567+0000","last_peered":"2024-09-20T22:08:26.396567+0000","last_clean":"2024-09-20T22:08:26.396567+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:08:26.396567+0000","last_undegraded":"2024-09-20T22:08:26.396567+0000","last_fullsized":"2024-09-20T22:08:26.396567+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'30","last_scrub_stamp":"2024-09-20T22:07:55.357726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.357726+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:10.172184+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":283,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395578+0000","last_change":"2024-09-20T22:05:58.966616+0000","last_active":"2024-09-20T22:08:26.395578+0000","last_peered":"2024-09-20T22:08:26.395578+0000","last_clean":"2024-09-20T22:08:26.395578+0000","last_became_active":"2024-09-20T22:02:50.108457+0000","last_became_peered":"2024-09-20T22:02:50.108457+0000","last_unstale":"2024-09-20T22:08:26.395578+0000","last_undegraded":"2024-09-20T22:08:26.395578+0000","last_fullsized":"2024-09-20T22:08:26.395578+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:58.966541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:58.966541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547423+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":294,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.399120+0000","last_change":"2024-09-20T22:07:21.886172+0000","last_active":"2024-09-20T22:08:26.399120+0000","last_peered":"2024-09-20T22:08:26.399120+0000","last_clean":"2024-09-20T22:08:26.399120+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:08:26.399120+0000","last_undegraded":"2024-09-20T22:08:26.399120+0000","last_fullsized":"2024-09-20T22:08:26.399120+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_clean_scrub_stamp":"2024-09-20T22:07:21.886038+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.052265+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":275,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.394780+0000","last_change":"2024-09-20T22:07:17.566774+0000","last_active":"2024-09-20T22:08:26.394780+0000","last_peered":"2024-09-20T22:08:26.394780+0000","last_clean":"2024-09-20T22:08:26.394780+0000","last_became_active":"2024-09-20T22:06:54.841419+0000","last_became_peered":"2024-09-20T22:06:54.841419+0000","last_unstale":"2024-09-20T22:08:26.394780+0000","last_undegraded":"2024-09-20T22:08:26.394780+0000","last_fullsized":"2024-09-20T22:08:26.394780+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:17.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:17.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:32.923210+0000","scrub_duration":120,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":262,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.398714+0000","last_change":"2024-09-20T22:05:34.954357+0000","last_active":"2024-09-20T22:08:26.398714+0000","last_peered":"2024-09-20T22:08:26.398714+0000","last_clean":"2024-09-20T22:08:26.398714+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:08:26.398714+0000","last_undegraded":"2024-09-20T22:08:26.398714+0000","last_fullsized":"2024-09-20T22:08:26.398714+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:05:34.953944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.b","version":"191'3979","reported_seq":4018,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:31.153618+0000","last_change":"2024-09-20T22:07:52.429893+0000","last_active":"2024-09-20T22:08:31.153618+0000","last_peered":"2024-09-20T22:08:31.153618+0000","last_clean":"2024-09-20T22:08:31.153618+0000","last_became_active":"2024-09-20T22:07:52.429554+0000","last_became_peered":"2024-09-20T22:07:52.429554+0000","last_unstale":"2024-09-20T22:08:31.153618+0000","last_undegraded":"2024-09-20T22:08:31.153618+0000","last_fullsized":"2024-09-20T22:08:31.153618+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3979,"log_dups_size":0,"ondisk_log_size":3979,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:20.427594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":260767744,"num_objects":3979,"num_object_clones":0,"num_object_copies":11937,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3979,"num_write_kb":254656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"71'13","reported_seq":151,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.398917+0000","last_change":"2024-09-20T22:06:54.851246+0000","last_active":"2024-09-20T22:08:26.398917+0000","last_peered":"2024-09-20T22:08:26.398917+0000","last_clean":"2024-09-20T22:08:26.398917+0000","last_became_active":"2024-09-20T22:06:54.850696+0000","last_became_peered":"2024-09-20T22:06:54.850696+0000","last_unstale":"2024-09-20T22:08:26.398917+0000","last_undegraded":"2024-09-20T22:08:26.398917+0000","last_fullsized":"2024-09-20T22:08:26.398917+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:28.363310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":121,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.402651+0000","last_change":"2024-09-20T22:07:20.341524+0000","last_active":"2024-09-20T22:08:26.402651+0000","last_peered":"2024-09-20T22:08:26.402651+0000","last_clean":"2024-09-20T22:08:26.402651+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:08:26.402651+0000","last_undegraded":"2024-09-20T22:08:26.402651+0000","last_fullsized":"2024-09-20T22:08:26.402651+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:20.338388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:20.338388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:40.900548+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":300,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.404450+0000","last_change":"2024-09-20T22:06:47.015379+0000","last_active":"2024-09-20T22:08:26.404450+0000","last_peered":"2024-09-20T22:08:26.404450+0000","last_clean":"2024-09-20T22:08:26.404450+0000","last_became_active":"2024-09-20T22:02:49.099508+0000","last_became_peered":"2024-09-20T22:02:49.099508+0000","last_unstale":"2024-09-20T22:08:26.404450+0000","last_undegraded":"2024-09-20T22:08:26.404450+0000","last_fullsized":"2024-09-20T22:08:26.404450+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:06:47.015270+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:34.967504+0000","last_clean_scrub_stamp":"2024-09-20T22:06:47.015270+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:28.184890+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":282,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.403759+0000","last_change":"2024-09-20T22:08:02.168865+0000","last_active":"2024-09-20T22:08:26.403759+0000","last_peered":"2024-09-20T22:08:26.403759+0000","last_clean":"2024-09-20T22:08:26.403759+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:08:26.403759+0000","last_undegraded":"2024-09-20T22:08:26.403759+0000","last_fullsized":"2024-09-20T22:08:26.403759+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:08:02.167030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:15.048786+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":263,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.403921+0000","last_change":"2024-09-20T22:05:28.780920+0000","last_active":"2024-09-20T22:08:26.403921+0000","last_peered":"2024-09-20T22:08:26.403921+0000","last_clean":"2024-09-20T22:08:26.403921+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:08:26.403921+0000","last_undegraded":"2024-09-20T22:08:26.403921+0000","last_fullsized":"2024-09-20T22:08:26.403921+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.780758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.780758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"191'3759","reported_seq":3798,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:27.635994+0000","last_change":"2024-09-20T22:07:52.939933+0000","last_active":"2024-09-20T22:08:27.635994+0000","last_peered":"2024-09-20T22:08:27.635994+0000","last_clean":"2024-09-20T22:08:27.635994+0000","last_became_active":"2024-09-20T22:07:52.939626+0000","last_became_peered":"2024-09-20T22:07:52.939626+0000","last_unstale":"2024-09-20T22:08:27.635994+0000","last_undegraded":"2024-09-20T22:08:27.635994+0000","last_fullsized":"2024-09-20T22:08:27.635994+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3759,"log_dups_size":0,"ondisk_log_size":3759,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:10.985426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":246349824,"num_objects":3759,"num_object_clones":0,"num_object_copies":11277,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3759,"num_write_kb":240576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"71'12","reported_seq":297,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395409+0000","last_change":"2024-09-20T22:07:29.614630+0000","last_active":"2024-09-20T22:08:26.395409+0000","last_peered":"2024-09-20T22:08:26.395409+0000","last_clean":"2024-09-20T22:08:26.395409+0000","last_became_active":"2024-09-20T22:06:54.848670+0000","last_became_peered":"2024-09-20T22:06:54.848670+0000","last_unstale":"2024-09-20T22:08:26.395409+0000","last_undegraded":"2024-09-20T22:08:26.395409+0000","last_fullsized":"2024-09-20T22:08:26.395409+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:29.614551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:29.614551+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:50.884056+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":282,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.394798+0000","last_change":"2024-09-20T22:06:44.862452+0000","last_active":"2024-09-20T22:08:26.394798+0000","last_peered":"2024-09-20T22:08:26.394798+0000","last_clean":"2024-09-20T22:08:26.394798+0000","last_became_active":"2024-09-20T22:02:50.110448+0000","last_became_peered":"2024-09-20T22:02:50.110448+0000","last_unstale":"2024-09-20T22:08:26.394798+0000","last_undegraded":"2024-09-20T22:08:26.394798+0000","last_fullsized":"2024-09-20T22:08:26.394798+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:06:44.862338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":628,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:28.962712+0000","last_change":"2024-09-20T22:07:16.590582+0000","last_active":"2024-09-20T22:08:28.962712+0000","last_peered":"2024-09-20T22:08:28.962712+0000","last_clean":"2024-09-20T22:08:28.962712+0000","last_became_active":"2024-09-20T22:06:54.844454+0000","last_became_peered":"2024-09-20T22:06:54.844454+0000","last_unstale":"2024-09-20T22:08:28.962712+0000","last_undegraded":"2024-09-20T22:08:28.962712+0000","last_fullsized":"2024-09-20T22:08:28.962712+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:07:16.590475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:07:16.590475+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.092001+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":327,"num_read_kb":322,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":275,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.399296+0000","last_change":"2024-09-20T22:07:22.884981+0000","last_active":"2024-09-20T22:08:26.399296+0000","last_peered":"2024-09-20T22:08:26.399296+0000","last_clean":"2024-09-20T22:08:26.399296+0000","last_became_active":"2024-09-20T22:06:54.841596+0000","last_became_peered":"2024-09-20T22:06:54.841596+0000","last_unstale":"2024-09-20T22:08:26.399296+0000","last_undegraded":"2024-09-20T22:08:26.399296+0000","last_fullsized":"2024-09-20T22:08:26.399296+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:22.884907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.884907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:41.462917+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":112,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.396392+0000","last_change":"2024-09-20T22:06:54.853957+0000","last_active":"2024-09-20T22:08:26.396392+0000","last_peered":"2024-09-20T22:08:26.396392+0000","last_clean":"2024-09-20T22:08:26.396392+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:08:26.396392+0000","last_undegraded":"2024-09-20T22:08:26.396392+0000","last_fullsized":"2024-09-20T22:08:26.396392+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:28.363280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"191'4006","reported_seq":4046,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:30.643010+0000","last_change":"2024-09-20T22:07:52.430313+0000","last_active":"2024-09-20T22:08:30.643010+0000","last_peered":"2024-09-20T22:08:30.643010+0000","last_clean":"2024-09-20T22:08:30.643010+0000","last_became_active":"2024-09-20T22:07:52.430017+0000","last_became_peered":"2024-09-20T22:07:52.430017+0000","last_unstale":"2024-09-20T22:08:30.643010+0000","last_undegraded":"2024-09-20T22:08:30.643010+0000","last_fullsized":"2024-09-20T22:08:30.643010+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4006,"log_dups_size":0,"ondisk_log_size":4006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:04.804126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262537216,"num_objects":4006,"num_object_clones":0,"num_object_copies":12018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4006,"num_write_kb":256384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"71'16","reported_seq":377,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.397323+0000","last_change":"2024-09-20T22:07:53.297530+0000","last_active":"2024-09-20T22:08:26.397323+0000","last_peered":"2024-09-20T22:08:26.397323+0000","last_clean":"2024-09-20T22:08:26.397323+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:08:26.397323+0000","last_undegraded":"2024-09-20T22:08:26.397323+0000","last_fullsized":"2024-09-20T22:08:26.397323+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'16","last_scrub_stamp":"2024-09-20T22:07:53.297442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.297442+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:20.417428+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":282,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.400005+0000","last_change":"2024-09-20T22:06:48.105501+0000","last_active":"2024-09-20T22:08:26.400005+0000","last_peered":"2024-09-20T22:08:26.400005+0000","last_clean":"2024-09-20T22:08:26.400005+0000","last_became_active":"2024-09-20T22:02:50.106011+0000","last_became_peered":"2024-09-20T22:02:50.106011+0000","last_unstale":"2024-09-20T22:08:26.400005+0000","last_undegraded":"2024-09-20T22:08:26.400005+0000","last_fullsized":"2024-09-20T22:08:26.400005+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:48.105403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:48.105403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":538,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.401725+0000","last_change":"2024-09-20T22:07:53.674419+0000","last_active":"2024-09-20T22:08:26.401725+0000","last_peered":"2024-09-20T22:08:26.401725+0000","last_clean":"2024-09-20T22:08:26.401725+0000","last_became_active":"2024-09-20T22:02:45.068482+0000","last_became_peered":"2024-09-20T22:02:45.068482+0000","last_unstale":"2024-09-20T22:08:26.401725+0000","last_undegraded":"2024-09-20T22:08:26.401725+0000","last_fullsized":"2024-09-20T22:08:26.401725+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:07:53.673691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.673691+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.005158+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":275,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.402368+0000","last_change":"2024-09-20T22:05:56.679105+0000","last_active":"2024-09-20T22:08:26.402368+0000","last_peered":"2024-09-20T22:08:26.402368+0000","last_clean":"2024-09-20T22:08:26.402368+0000","last_became_active":"2024-09-20T22:02:54.298021+0000","last_became_peered":"2024-09-20T22:02:54.298021+0000","last_unstale":"2024-09-20T22:08:26.402368+0000","last_undegraded":"2024-09-20T22:08:26.402368+0000","last_fullsized":"2024-09-20T22:08:26.402368+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:56.678980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:56.678980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":284,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.399186+0000","last_change":"2024-09-20T22:06:47.664985+0000","last_active":"2024-09-20T22:08:26.399186+0000","last_peered":"2024-09-20T22:08:26.399186+0000","last_clean":"2024-09-20T22:08:26.399186+0000","last_became_active":"2024-09-20T22:04:21.237725+0000","last_became_peered":"2024-09-20T22:04:21.237725+0000","last_unstale":"2024-09-20T22:08:26.399186+0000","last_undegraded":"2024-09-20T22:08:26.399186+0000","last_fullsized":"2024-09-20T22:08:26.399186+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:47.664877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:06:47.664877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"191'3885","reported_seq":3925,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:30.622578+0000","last_change":"2024-09-20T22:07:52.429509+0000","last_active":"2024-09-20T22:08:30.622578+0000","last_peered":"2024-09-20T22:08:30.622578+0000","last_clean":"2024-09-20T22:08:30.622578+0000","last_became_active":"2024-09-20T22:07:52.429069+0000","last_became_peered":"2024-09-20T22:07:52.429069+0000","last_unstale":"2024-09-20T22:08:30.622578+0000","last_undegraded":"2024-09-20T22:08:30.622578+0000","last_fullsized":"2024-09-20T22:08:30.622578+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3885,"log_dups_size":0,"ondisk_log_size":3885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.758907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":254607360,"num_objects":3885,"num_object_clones":0,"num_object_copies":11655,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3885,"num_write_kb":248640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"71'11","reported_seq":299,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395989+0000","last_change":"2024-09-20T22:07:13.511239+0000","last_active":"2024-09-20T22:08:26.395989+0000","last_peered":"2024-09-20T22:08:26.395989+0000","last_clean":"2024-09-20T22:08:26.395989+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:08:26.395989+0000","last_undegraded":"2024-09-20T22:08:26.395989+0000","last_fullsized":"2024-09-20T22:08:26.395989+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:13.511159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:13.511159+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:38.687052+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":275,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.394986+0000","last_change":"2024-09-20T22:05:26.006756+0000","last_active":"2024-09-20T22:08:26.394986+0000","last_peered":"2024-09-20T22:08:26.394986+0000","last_clean":"2024-09-20T22:08:26.394986+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:08:26.394986+0000","last_undegraded":"2024-09-20T22:08:26.394986+0000","last_fullsized":"2024-09-20T22:08:26.394986+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:26.006692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:26.006692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547316+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":282,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:27.057915+0000","last_change":"2024-09-20T22:08:27.057946+0000","last_active":"2024-09-20T22:08:27.057915+0000","last_peered":"2024-09-20T22:08:27.057915+0000","last_clean":"2024-09-20T22:08:27.057915+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:08:27.057915+0000","last_undegraded":"2024-09-20T22:08:27.057915+0000","last_fullsized":"2024-09-20T22:08:27.057915+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:27.057849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:27.057849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:36.999687+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":270,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.399222+0000","last_change":"2024-09-20T22:06:45.030549+0000","last_active":"2024-09-20T22:08:26.399222+0000","last_peered":"2024-09-20T22:08:26.399222+0000","last_clean":"2024-09-20T22:08:26.399222+0000","last_became_active":"2024-09-20T22:02:56.349376+0000","last_became_peered":"2024-09-20T22:02:56.349376+0000","last_unstale":"2024-09-20T22:08:26.399222+0000","last_undegraded":"2024-09-20T22:08:26.399222+0000","last_fullsized":"2024-09-20T22:08:26.399222+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.030289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.030289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1","version":"191'3702","reported_seq":3742,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:27.635937+0000","last_change":"2024-09-20T22:07:52.939890+0000","last_active":"2024-09-20T22:08:27.635937+0000","last_peered":"2024-09-20T22:08:27.635937+0000","last_clean":"2024-09-20T22:08:27.635937+0000","last_became_active":"2024-09-20T22:07:52.939569+0000","last_became_peered":"2024-09-20T22:07:52.939569+0000","last_unstale":"2024-09-20T22:08:27.635937+0000","last_undegraded":"2024-09-20T22:08:27.635937+0000","last_fullsized":"2024-09-20T22:08:27.635937+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3702,"log_dups_size":0,"ondisk_log_size":3702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:15.331372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242614272,"num_objects":3702,"num_object_clones":0,"num_object_copies":11106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3702,"num_write_kb":236928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"71'17","reported_seq":316,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.396100+0000","last_change":"2024-09-20T22:07:54.528854+0000","last_active":"2024-09-20T22:08:26.396100+0000","last_peered":"2024-09-20T22:08:26.396100+0000","last_clean":"2024-09-20T22:08:26.396100+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:08:26.396100+0000","last_undegraded":"2024-09-20T22:08:26.396100+0000","last_fullsized":"2024-09-20T22:08:26.396100+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'17","last_scrub_stamp":"2024-09-20T22:07:54.528759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:54.528759+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.170193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":291,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:30.429815+0000","last_change":"2024-09-20T22:08:30.429880+0000","last_active":"2024-09-20T22:08:30.429815+0000","last_peered":"2024-09-20T22:08:30.429815+0000","last_clean":"2024-09-20T22:08:30.429815+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:08:30.429815+0000","last_undegraded":"2024-09-20T22:08:30.429815+0000","last_fullsized":"2024-09-20T22:08:30.429815+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:30.429704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:30.429704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:36.273776+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":120,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.396405+0000","last_change":"2024-09-20T22:07:19.376008+0000","last_active":"2024-09-20T22:08:26.396405+0000","last_peered":"2024-09-20T22:08:26.396405+0000","last_clean":"2024-09-20T22:08:26.396405+0000","last_became_active":"2024-09-20T22:06:54.848275+0000","last_became_peered":"2024-09-20T22:06:54.848275+0000","last_unstale":"2024-09-20T22:08:26.396405+0000","last_undegraded":"2024-09-20T22:08:26.396405+0000","last_fullsized":"2024-09-20T22:08:26.396405+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:19.375867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.375867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:49.161984+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":278,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.404337+0000","last_change":"2024-09-20T22:07:55.195229+0000","last_active":"2024-09-20T22:08:26.404337+0000","last_peered":"2024-09-20T22:08:26.404337+0000","last_clean":"2024-09-20T22:08:26.404337+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:08:26.404337+0000","last_undegraded":"2024-09-20T22:08:26.404337+0000","last_fullsized":"2024-09-20T22:08:26.404337+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:55.195131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.195131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:10.187674+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.0","version":"191'4008","reported_seq":4048,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:31.118770+0000","last_change":"2024-09-20T22:07:52.429344+0000","last_active":"2024-09-20T22:08:31.118770+0000","last_peered":"2024-09-20T22:08:31.118770+0000","last_clean":"2024-09-20T22:08:31.118770+0000","last_became_active":"2024-09-20T22:07:52.429054+0000","last_became_peered":"2024-09-20T22:07:52.429054+0000","last_unstale":"2024-09-20T22:08:31.118770+0000","last_undegraded":"2024-09-20T22:08:31.118770+0000","last_fullsized":"2024-09-20T22:08:31.118770+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4008,"log_dups_size":0,"ondisk_log_size":4008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:16.620658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262668288,"num_objects":4008,"num_object_clones":0,"num_object_copies":12024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4008,"num_write_kb":256512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"71'10","reported_seq":286,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395003+0000","last_change":"2024-09-20T22:05:27.048378+0000","last_active":"2024-09-20T22:08:26.395003+0000","last_peered":"2024-09-20T22:08:26.395003+0000","last_clean":"2024-09-20T22:08:26.395003+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:08:26.395003+0000","last_undegraded":"2024-09-20T22:08:26.395003+0000","last_fullsized":"2024-09-20T22:08:26.395003+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:05:27.048215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:27.048215+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547351+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"64'1","reported_seq":288,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.402054+0000","last_change":"2024-09-20T22:06:33.966005+0000","last_active":"2024-09-20T22:08:26.402054+0000","last_peered":"2024-09-20T22:08:26.402054+0000","last_clean":"2024-09-20T22:08:26.402054+0000","last_became_active":"2024-09-20T22:02:50.110446+0000","last_became_peered":"2024-09-20T22:02:50.110446+0000","last_unstale":"2024-09-20T22:08:26.402054+0000","last_undegraded":"2024-09-20T22:08:26.402054+0000","last_fullsized":"2024-09-20T22:08:26.402054+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:06:33.965866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:33.965866+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"139'9","reported_seq":368,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.403256+0000","last_change":"2024-09-20T22:07:57.138071+0000","last_active":"2024-09-20T22:08:26.403256+0000","last_peered":"2024-09-20T22:08:26.403256+0000","last_clean":"2024-09-20T22:08:26.403256+0000","last_became_active":"2024-09-20T22:02:54.294458+0000","last_became_peered":"2024-09-20T22:02:54.294458+0000","last_unstale":"2024-09-20T22:08:26.403256+0000","last_undegraded":"2024-09-20T22:08:26.403256+0000","last_fullsized":"2024-09-20T22:08:26.403256+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'9","last_scrub_stamp":"2024-09-20T22:07:57.137943+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:06:40.910011+0000","last_clean_scrub_stamp":"2024-09-20T22:07:57.137943+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:23.015596+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":272,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395786+0000","last_change":"2024-09-20T22:07:23.482514+0000","last_active":"2024-09-20T22:08:26.395786+0000","last_peered":"2024-09-20T22:08:26.395786+0000","last_clean":"2024-09-20T22:08:26.395786+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:08:26.395786+0000","last_undegraded":"2024-09-20T22:08:26.395786+0000","last_fullsized":"2024-09-20T22:08:26.395786+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.482372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.886444+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"191'3715","reported_seq":3755,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:27.644184+0000","last_change":"2024-09-20T22:07:52.429071+0000","last_active":"2024-09-20T22:08:27.644184+0000","last_peered":"2024-09-20T22:08:27.644184+0000","last_clean":"2024-09-20T22:08:27.644184+0000","last_became_active":"2024-09-20T22:07:52.428789+0000","last_became_peered":"2024-09-20T22:07:52.428789+0000","last_unstale":"2024-09-20T22:08:27.644184+0000","last_undegraded":"2024-09-20T22:08:27.644184+0000","last_fullsized":"2024-09-20T22:08:27.644184+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3715,"log_dups_size":0,"ondisk_log_size":3715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:01.547587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243466240,"num_objects":3715,"num_object_clones":0,"num_object_copies":11145,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3715,"num_write_kb":237760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"71'9","reported_seq":303,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.394865+0000","last_change":"2024-09-20T22:07:59.740218+0000","last_active":"2024-09-20T22:08:26.394865+0000","last_peered":"2024-09-20T22:08:26.394865+0000","last_clean":"2024-09-20T22:08:26.394865+0000","last_became_active":"2024-09-20T22:06:54.847770+0000","last_became_peered":"2024-09-20T22:06:54.847770+0000","last_unstale":"2024-09-20T22:08:26.394865+0000","last_undegraded":"2024-09-20T22:08:26.394865+0000","last_fullsized":"2024-09-20T22:08:26.394865+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:59.740089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:59.740089+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:19.034130+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":288,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.399374+0000","last_change":"2024-09-20T22:06:36.582728+0000","last_active":"2024-09-20T22:08:26.399374+0000","last_peered":"2024-09-20T22:08:26.399374+0000","last_clean":"2024-09-20T22:08:26.399374+0000","last_became_active":"2024-09-20T22:02:50.112682+0000","last_became_peered":"2024-09-20T22:02:50.112682+0000","last_unstale":"2024-09-20T22:08:26.399374+0000","last_undegraded":"2024-09-20T22:08:26.399374+0000","last_fullsized":"2024-09-20T22:08:26.399374+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:06:36.582537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:36.582537+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":282,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.402939+0000","last_change":"2024-09-20T22:07:22.203673+0000","last_active":"2024-09-20T22:08:26.402939+0000","last_peered":"2024-09-20T22:08:26.402939+0000","last_clean":"2024-09-20T22:08:26.402939+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:08:26.402939+0000","last_undegraded":"2024-09-20T22:08:26.402939+0000","last_fullsized":"2024-09-20T22:08:26.402939+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:22.203576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.203576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.064543+0000","scrub_duration":49,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":280,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:27.817917+0000","last_change":"2024-09-20T22:08:27.817949+0000","last_active":"2024-09-20T22:08:27.817917+0000","last_peered":"2024-09-20T22:08:27.817917+0000","last_clean":"2024-09-20T22:08:27.817917+0000","last_became_active":"2024-09-20T22:06:55.824229+0000","last_became_peered":"2024-09-20T22:06:55.824229+0000","last_unstale":"2024-09-20T22:08:27.817917+0000","last_undegraded":"2024-09-20T22:08:27.817917+0000","last_fullsized":"2024-09-20T22:08:27.817917+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:27.817850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:27.817850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:45.479132+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"191'4086","reported_seq":4126,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:31.282309+0000","last_change":"2024-09-20T22:07:52.674668+0000","last_active":"2024-09-20T22:08:31.282309+0000","last_peered":"2024-09-20T22:08:31.282309+0000","last_clean":"2024-09-20T22:08:31.282309+0000","last_became_active":"2024-09-20T22:07:52.674301+0000","last_became_peered":"2024-09-20T22:07:52.674301+0000","last_unstale":"2024-09-20T22:08:31.282309+0000","last_undegraded":"2024-09-20T22:08:31.282309+0000","last_fullsized":"2024-09-20T22:08:31.282309+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4086,"log_dups_size":0,"ondisk_log_size":4086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:06.988552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":267780096,"num_objects":4086,"num_object_clones":0,"num_object_copies":12258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4086,"num_write_kb":261504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"71'19","reported_seq":310,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.396654+0000","last_change":"2024-09-20T22:07:25.155015+0000","last_active":"2024-09-20T22:08:26.396654+0000","last_peered":"2024-09-20T22:08:26.396654+0000","last_clean":"2024-09-20T22:08:26.396654+0000","last_became_active":"2024-09-20T22:06:54.829170+0000","last_became_peered":"2024-09-20T22:06:54.829170+0000","last_unstale":"2024-09-20T22:08:26.396654+0000","last_undegraded":"2024-09-20T22:08:26.396654+0000","last_fullsized":"2024-09-20T22:08:26.396654+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:07:25.154919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:25.154919+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:43.954499+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":283,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.404220+0000","last_change":"2024-09-20T22:08:00.560208+0000","last_active":"2024-09-20T22:08:26.404220+0000","last_peered":"2024-09-20T22:08:26.404220+0000","last_clean":"2024-09-20T22:08:26.404220+0000","last_became_active":"2024-09-20T22:02:50.108712+0000","last_became_peered":"2024-09-20T22:02:50.108712+0000","last_unstale":"2024-09-20T22:08:26.404220+0000","last_undegraded":"2024-09-20T22:08:26.404220+0000","last_fullsized":"2024-09-20T22:08:26.404220+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:00.560098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:00.560098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:09.631655+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":282,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.402621+0000","last_change":"2024-09-20T22:07:20.217991+0000","last_active":"2024-09-20T22:08:26.402621+0000","last_peered":"2024-09-20T22:08:26.402621+0000","last_clean":"2024-09-20T22:08:26.402621+0000","last_became_active":"2024-09-20T22:02:54.291134+0000","last_became_peered":"2024-09-20T22:02:54.291134+0000","last_unstale":"2024-09-20T22:08:26.402621+0000","last_undegraded":"2024-09-20T22:08:26.402621+0000","last_fullsized":"2024-09-20T22:08:26.402621+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:20.217761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:20.217761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:42.617706+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":271,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.401631+0000","last_change":"2024-09-20T22:07:29.645352+0000","last_active":"2024-09-20T22:08:26.401631+0000","last_peered":"2024-09-20T22:08:26.401631+0000","last_clean":"2024-09-20T22:08:26.401631+0000","last_became_active":"2024-09-20T22:02:56.307687+0000","last_became_peered":"2024-09-20T22:02:56.307687+0000","last_unstale":"2024-09-20T22:08:26.401631+0000","last_undegraded":"2024-09-20T22:08:26.401631+0000","last_fullsized":"2024-09-20T22:08:26.401631+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:29.645127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:29.645127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:36.059611+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"191'4101","reported_seq":4141,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:30.678092+0000","last_change":"2024-09-20T22:07:52.675309+0000","last_active":"2024-09-20T22:08:30.678092+0000","last_peered":"2024-09-20T22:08:30.678092+0000","last_clean":"2024-09-20T22:08:30.678092+0000","last_became_active":"2024-09-20T22:07:52.674921+0000","last_became_peered":"2024-09-20T22:07:52.674921+0000","last_unstale":"2024-09-20T22:08:30.678092+0000","last_undegraded":"2024-09-20T22:08:30.678092+0000","last_fullsized":"2024-09-20T22:08:30.678092+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4101,"log_dups_size":0,"ondisk_log_size":4101,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:17.687123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":268763136,"num_objects":4101,"num_object_clones":0,"num_object_copies":12303,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4101,"num_write_kb":262464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"71'12","reported_seq":303,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.394599+0000","last_change":"2024-09-20T22:07:22.458421+0000","last_active":"2024-09-20T22:08:26.394599+0000","last_peered":"2024-09-20T22:08:26.394599+0000","last_clean":"2024-09-20T22:08:26.394599+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:08:26.394599+0000","last_undegraded":"2024-09-20T22:08:26.394599+0000","last_fullsized":"2024-09-20T22:08:26.394599+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:22.458279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.458279+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:33.412138+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":284,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.403060+0000","last_change":"2024-09-20T22:07:21.584606+0000","last_active":"2024-09-20T22:08:26.403060+0000","last_peered":"2024-09-20T22:08:26.403060+0000","last_clean":"2024-09-20T22:08:26.403060+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:08:26.403060+0000","last_undegraded":"2024-09-20T22:08:26.403060+0000","last_fullsized":"2024-09-20T22:08:26.403060+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:21.584499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:21.584499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":283,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.402371+0000","last_change":"2024-09-20T22:07:15.260254+0000","last_active":"2024-09-20T22:08:26.402371+0000","last_peered":"2024-09-20T22:08:26.402371+0000","last_clean":"2024-09-20T22:08:26.402371+0000","last_became_active":"2024-09-20T22:06:54.843906+0000","last_became_peered":"2024-09-20T22:06:54.843906+0000","last_unstale":"2024-09-20T22:08:26.402371+0000","last_undegraded":"2024-09-20T22:08:26.402371+0000","last_fullsized":"2024-09-20T22:08:26.402371+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:15.260148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.260148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":216,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395175+0000","last_change":"2024-09-20T22:06:55.822136+0000","last_active":"2024-09-20T22:08:26.395175+0000","last_peered":"2024-09-20T22:08:26.395175+0000","last_clean":"2024-09-20T22:08:26.395175+0000","last_became_active":"2024-09-20T22:06:55.821647+0000","last_became_peered":"2024-09-20T22:06:55.821647+0000","last_unstale":"2024-09-20T22:08:26.395175+0000","last_undegraded":"2024-09-20T22:08:26.395175+0000","last_fullsized":"2024-09-20T22:08:26.395175+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:28.569045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:28.569045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"191'3935","reported_seq":3975,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:30.635598+0000","last_change":"2024-09-20T22:07:52.939846+0000","last_active":"2024-09-20T22:08:30.635598+0000","last_peered":"2024-09-20T22:08:30.635598+0000","last_clean":"2024-09-20T22:08:30.635598+0000","last_became_active":"2024-09-20T22:07:52.939563+0000","last_became_peered":"2024-09-20T22:07:52.939563+0000","last_unstale":"2024-09-20T22:08:30.635598+0000","last_undegraded":"2024-09-20T22:08:30.635598+0000","last_fullsized":"2024-09-20T22:08:30.635598+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":3935,"log_dups_size":0,"ondisk_log_size":3935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:55.690792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":257884160,"num_objects":3935,"num_object_clones":0,"num_object_copies":11805,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3935,"num_write_kb":251840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"71'15","reported_seq":303,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.400399+0000","last_change":"2024-09-20T22:06:42.563403+0000","last_active":"2024-09-20T22:08:26.400399+0000","last_peered":"2024-09-20T22:08:26.400399+0000","last_clean":"2024-09-20T22:08:26.400399+0000","last_became_active":"2024-09-20T22:02:52.482008+0000","last_became_peered":"2024-09-20T22:02:52.482008+0000","last_unstale":"2024-09-20T22:08:26.400399+0000","last_undegraded":"2024-09-20T22:08:26.400399+0000","last_fullsized":"2024-09-20T22:08:26.400399+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:06:42.563266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:42.563266+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":297,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.402517+0000","last_change":"2024-09-20T22:06:40.813723+0000","last_active":"2024-09-20T22:08:26.402517+0000","last_peered":"2024-09-20T22:08:26.402517+0000","last_clean":"2024-09-20T22:08:26.402517+0000","last_became_active":"2024-09-20T22:02:50.114819+0000","last_became_peered":"2024-09-20T22:02:50.114819+0000","last_unstale":"2024-09-20T22:08:26.402517+0000","last_undegraded":"2024-09-20T22:08:26.402517+0000","last_fullsized":"2024-09-20T22:08:26.402517+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:06:40.813590+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":344,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.401568+0000","last_change":"2024-09-20T22:05:34.732761+0000","last_active":"2024-09-20T22:08:26.401568+0000","last_peered":"2024-09-20T22:08:26.401568+0000","last_clean":"2024-09-20T22:08:26.401568+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:08:26.401568+0000","last_undegraded":"2024-09-20T22:08:26.401568+0000","last_fullsized":"2024-09-20T22:08:26.401568+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:05:34.732644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:34.732644+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":270,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.399090+0000","last_change":"2024-09-20T22:06:35.559426+0000","last_active":"2024-09-20T22:08:26.399090+0000","last_peered":"2024-09-20T22:08:26.399090+0000","last_clean":"2024-09-20T22:08:26.399090+0000","last_became_active":"2024-09-20T22:02:56.346990+0000","last_became_peered":"2024-09-20T22:02:56.346990+0000","last_unstale":"2024-09-20T22:08:26.399090+0000","last_undegraded":"2024-09-20T22:08:26.399090+0000","last_fullsized":"2024-09-20T22:08:26.399090+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:35.559306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:35.559306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":290,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.396441+0000","last_change":"2024-09-20T22:08:01.261458+0000","last_active":"2024-09-20T22:08:26.396441+0000","last_peered":"2024-09-20T22:08:26.396441+0000","last_clean":"2024-09-20T22:08:26.396441+0000","last_became_active":"2024-09-20T22:02:50.111935+0000","last_became_peered":"2024-09-20T22:02:50.111935+0000","last_unstale":"2024-09-20T22:08:26.396441+0000","last_undegraded":"2024-09-20T22:08:26.396441+0000","last_fullsized":"2024-09-20T22:08:26.396441+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:01.261328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:01.261328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:06.169471+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"71'6","reported_seq":280,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.400310+0000","last_change":"2024-09-20T22:06:54.862697+0000","last_active":"2024-09-20T22:08:26.400310+0000","last_peered":"2024-09-20T22:08:26.400310+0000","last_clean":"2024-09-20T22:08:26.400310+0000","last_became_active":"2024-09-20T22:06:54.862227+0000","last_became_peered":"2024-09-20T22:06:54.862227+0000","last_unstale":"2024-09-20T22:08:26.400310+0000","last_undegraded":"2024-09-20T22:08:26.400310+0000","last_fullsized":"2024-09-20T22:08:26.400310+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":275,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.398947+0000","last_change":"2024-09-20T22:07:19.865906+0000","last_active":"2024-09-20T22:08:26.398947+0000","last_peered":"2024-09-20T22:08:26.398947+0000","last_clean":"2024-09-20T22:08:26.398947+0000","last_became_active":"2024-09-20T22:06:54.836200+0000","last_became_peered":"2024-09-20T22:06:54.836200+0000","last_unstale":"2024-09-20T22:08:26.398947+0000","last_undegraded":"2024-09-20T22:08:26.398947+0000","last_fullsized":"2024-09-20T22:08:26.398947+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:19.865809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.865809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:42.730889+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":263,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.403873+0000","last_change":"2024-09-20T22:06:39.783526+0000","last_active":"2024-09-20T22:08:26.403873+0000","last_peered":"2024-09-20T22:08:26.403873+0000","last_clean":"2024-09-20T22:08:26.403873+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:08:26.403873+0000","last_undegraded":"2024-09-20T22:08:26.403873+0000","last_fullsized":"2024-09-20T22:08:26.403873+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:39.783456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:39.783456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"71'9","reported_seq":286,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395509+0000","last_change":"2024-09-20T22:04:52.254864+0000","last_active":"2024-09-20T22:08:26.395509+0000","last_peered":"2024-09-20T22:08:26.395509+0000","last_clean":"2024-09-20T22:08:26.395509+0000","last_became_active":"2024-09-20T22:02:52.481989+0000","last_became_peered":"2024-09-20T22:02:52.481989+0000","last_unstale":"2024-09-20T22:08:26.395509+0000","last_undegraded":"2024-09-20T22:08:26.395509+0000","last_fullsized":"2024-09-20T22:08:26.395509+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:52.254706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:52.254706+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":283,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:28.755392+0000","last_change":"2024-09-20T22:08:28.755412+0000","last_active":"2024-09-20T22:08:28.755392+0000","last_peered":"2024-09-20T22:08:28.755392+0000","last_clean":"2024-09-20T22:08:28.755392+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:08:28.755392+0000","last_undegraded":"2024-09-20T22:08:28.755392+0000","last_fullsized":"2024-09-20T22:08:28.755392+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:28.755335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:28.755335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:33.713076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":267,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.400177+0000","last_change":"2024-09-20T22:06:54.847169+0000","last_active":"2024-09-20T22:08:26.400177+0000","last_peered":"2024-09-20T22:08:26.400177+0000","last_clean":"2024-09-20T22:08:26.400177+0000","last_became_active":"2024-09-20T22:06:54.845704+0000","last_became_peered":"2024-09-20T22:06:54.845704+0000","last_unstale":"2024-09-20T22:08:26.400177+0000","last_undegraded":"2024-09-20T22:08:26.400177+0000","last_fullsized":"2024-09-20T22:08:26.400177+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":270,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.403452+0000","last_change":"2024-09-20T22:06:39.942296+0000","last_active":"2024-09-20T22:08:26.403452+0000","last_peered":"2024-09-20T22:08:26.403452+0000","last_clean":"2024-09-20T22:08:26.403452+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:08:26.403452+0000","last_undegraded":"2024-09-20T22:08:26.403452+0000","last_fullsized":"2024-09-20T22:08:26.403452+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:39.942186+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:39.942186+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:28.184860+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"71'9","reported_seq":302,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.399852+0000","last_change":"2024-09-20T22:07:52.834986+0000","last_active":"2024-09-20T22:08:26.399852+0000","last_peered":"2024-09-20T22:08:26.399852+0000","last_clean":"2024-09-20T22:08:26.399852+0000","last_became_active":"2024-09-20T22:02:52.482141+0000","last_became_peered":"2024-09-20T22:02:52.482141+0000","last_unstale":"2024-09-20T22:08:26.399852+0000","last_undegraded":"2024-09-20T22:08:26.399852+0000","last_fullsized":"2024-09-20T22:08:26.399852+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:52.834898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:52.834898+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.590814+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":282,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.394530+0000","last_change":"2024-09-20T22:06:45.908734+0000","last_active":"2024-09-20T22:08:26.394530+0000","last_peered":"2024-09-20T22:08:26.394530+0000","last_clean":"2024-09-20T22:08:26.394530+0000","last_became_active":"2024-09-20T22:02:50.110075+0000","last_became_peered":"2024-09-20T22:02:50.110075+0000","last_unstale":"2024-09-20T22:08:26.394530+0000","last_undegraded":"2024-09-20T22:08:26.394530+0000","last_fullsized":"2024-09-20T22:08:26.394530+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.908616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.908616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"71'8","reported_seq":343,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.399485+0000","last_change":"2024-09-20T22:05:55.972684+0000","last_active":"2024-09-20T22:08:26.399485+0000","last_peered":"2024-09-20T22:08:26.399485+0000","last_clean":"2024-09-20T22:08:26.399485+0000","last_became_active":"2024-09-20T22:02:54.294381+0000","last_became_peered":"2024-09-20T22:02:54.294381+0000","last_unstale":"2024-09-20T22:08:26.399485+0000","last_undegraded":"2024-09-20T22:08:26.399485+0000","last_fullsized":"2024-09-20T22:08:26.399485+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:05:55.972565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:55.972565+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":263,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395847+0000","last_change":"2024-09-20T22:05:00.897974+0000","last_active":"2024-09-20T22:08:26.395847+0000","last_peered":"2024-09-20T22:08:26.395847+0000","last_clean":"2024-09-20T22:08:26.395847+0000","last_became_active":"2024-09-20T22:02:56.300041+0000","last_became_peered":"2024-09-20T22:02:56.300041+0000","last_unstale":"2024-09-20T22:08:26.395847+0000","last_undegraded":"2024-09-20T22:08:26.395847+0000","last_fullsized":"2024-09-20T22:08:26.395847+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:00.897869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:00.897869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:27.547236+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"71'10","reported_seq":286,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.399604+0000","last_change":"2024-09-20T22:06:54.862610+0000","last_active":"2024-09-20T22:08:26.399604+0000","last_peered":"2024-09-20T22:08:26.399604+0000","last_clean":"2024-09-20T22:08:26.399604+0000","last_became_active":"2024-09-20T22:06:54.862289+0000","last_became_peered":"2024-09-20T22:06:54.862289+0000","last_unstale":"2024-09-20T22:08:26.399604+0000","last_undegraded":"2024-09-20T22:08:26.399604+0000","last_fullsized":"2024-09-20T22:08:26.399604+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:02:51.256183+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":275,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.402025+0000","last_change":"2024-09-20T22:06:35.687169+0000","last_active":"2024-09-20T22:08:26.402025+0000","last_peered":"2024-09-20T22:08:26.402025+0000","last_clean":"2024-09-20T22:08:26.402025+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:08:26.402025+0000","last_undegraded":"2024-09-20T22:08:26.402025+0000","last_fullsized":"2024-09-20T22:08:26.402025+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:35.687065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:35.687065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"146'10","reported_seq":354,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.399740+0000","last_change":"2024-09-20T22:07:15.870748+0000","last_active":"2024-09-20T22:08:26.399740+0000","last_peered":"2024-09-20T22:08:26.399740+0000","last_clean":"2024-09-20T22:08:26.399740+0000","last_became_active":"2024-09-20T22:06:54.843097+0000","last_became_peered":"2024-09-20T22:06:54.843097+0000","last_unstale":"2024-09-20T22:08:26.399740+0000","last_undegraded":"2024-09-20T22:08:26.399740+0000","last_fullsized":"2024-09-20T22:08:26.399740+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:07:15.870646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.870646+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:36.803368+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":262,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395018+0000","last_change":"2024-09-20T22:05:35.601891+0000","last_active":"2024-09-20T22:08:26.395018+0000","last_peered":"2024-09-20T22:08:26.395018+0000","last_clean":"2024-09-20T22:08:26.395018+0000","last_became_active":"2024-09-20T22:02:56.305383+0000","last_became_peered":"2024-09-20T22:02:56.305383+0000","last_unstale":"2024-09-20T22:08:26.395018+0000","last_undegraded":"2024-09-20T22:08:26.395018+0000","last_fullsized":"2024-09-20T22:08:26.395018+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:35.601794+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:35.601794+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"71'11","reported_seq":317,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395346+0000","last_change":"2024-09-20T22:08:22.159810+0000","last_active":"2024-09-20T22:08:26.395346+0000","last_peered":"2024-09-20T22:08:26.395346+0000","last_clean":"2024-09-20T22:08:26.395346+0000","last_became_active":"2024-09-20T22:08:22.139581+0000","last_became_peered":"2024-09-20T22:08:22.139581+0000","last_unstale":"2024-09-20T22:08:26.395346+0000","last_undegraded":"2024-09-20T22:08:26.395346+0000","last_fullsized":"2024-09-20T22:08:26.395346+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:15.544705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.544705+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:28.946270+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":283,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:29.474852+0000","last_change":"2024-09-20T22:08:29.474891+0000","last_active":"2024-09-20T22:08:29.474852+0000","last_peered":"2024-09-20T22:08:29.474852+0000","last_clean":"2024-09-20T22:08:29.474852+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:08:29.474852+0000","last_undegraded":"2024-09-20T22:08:29.474852+0000","last_fullsized":"2024-09-20T22:08:29.474852+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:29.474781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:29.474781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:48.958420+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":112,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.397501+0000","last_change":"2024-09-20T22:06:54.853857+0000","last_active":"2024-09-20T22:08:26.397501+0000","last_peered":"2024-09-20T22:08:26.397501+0000","last_clean":"2024-09-20T22:08:26.397501+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:08:26.397501+0000","last_undegraded":"2024-09-20T22:08:26.397501+0000","last_fullsized":"2024-09-20T22:08:26.397501+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:28.363334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":99,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.396586+0000","last_change":"2024-09-20T22:06:55.835713+0000","last_active":"2024-09-20T22:08:26.396586+0000","last_peered":"2024-09-20T22:08:26.396586+0000","last_clean":"2024-09-20T22:08:26.396586+0000","last_became_active":"2024-09-20T22:06:55.835393+0000","last_became_peered":"2024-09-20T22:06:55.835393+0000","last_unstale":"2024-09-20T22:08:26.396586+0000","last_undegraded":"2024-09-20T22:08:26.396586+0000","last_fullsized":"2024-09-20T22:08:26.396586+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:33.623816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:33.623816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:28.363483+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"71'9","reported_seq":184,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.397831+0000","last_change":"2024-09-20T22:06:54.844349+0000","last_active":"2024-09-20T22:08:26.397831+0000","last_peered":"2024-09-20T22:08:26.397831+0000","last_clean":"2024-09-20T22:08:26.397831+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:08:26.397831+0000","last_undegraded":"2024-09-20T22:08:26.397831+0000","last_fullsized":"2024-09-20T22:08:26.397831+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:12.196518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:12.196518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:28.363413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":291,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.404119+0000","last_change":"2024-09-20T22:07:54.685916+0000","last_active":"2024-09-20T22:08:26.404119+0000","last_peered":"2024-09-20T22:08:26.404119+0000","last_clean":"2024-09-20T22:08:26.404119+0000","last_became_active":"2024-09-20T22:02:50.109803+0000","last_became_peered":"2024-09-20T22:02:50.109803+0000","last_unstale":"2024-09-20T22:08:26.404119+0000","last_undegraded":"2024-09-20T22:08:26.404119+0000","last_fullsized":"2024-09-20T22:08:26.404119+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:54.685751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:54.685751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:04.385036+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":267,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.401630+0000","last_change":"2024-09-20T22:06:54.844570+0000","last_active":"2024-09-20T22:08:26.401630+0000","last_peered":"2024-09-20T22:08:26.401630+0000","last_clean":"2024-09-20T22:08:26.401630+0000","last_became_active":"2024-09-20T22:06:54.844271+0000","last_became_peered":"2024-09-20T22:06:54.844271+0000","last_unstale":"2024-09-20T22:08:26.401630+0000","last_undegraded":"2024-09-20T22:08:26.401630+0000","last_fullsized":"2024-09-20T22:08:26.401630+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:02:53.269054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":271,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.393701+0000","last_change":"2024-09-20T22:07:23.495005+0000","last_active":"2024-09-20T22:08:26.393701+0000","last_peered":"2024-09-20T22:08:26.393701+0000","last_clean":"2024-09-20T22:08:26.393701+0000","last_became_active":"2024-09-20T22:06:54.848842+0000","last_became_peered":"2024-09-20T22:06:54.848842+0000","last_unstale":"2024-09-20T22:08:26.393701+0000","last_undegraded":"2024-09-20T22:08:26.393701+0000","last_fullsized":"2024-09-20T22:08:26.393701+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.494905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.494905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:44.999867+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"71'11","reported_seq":297,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.398930+0000","last_change":"2024-09-20T22:06:36.942847+0000","last_active":"2024-09-20T22:08:26.398930+0000","last_peered":"2024-09-20T22:08:26.398930+0000","last_clean":"2024-09-20T22:08:26.398930+0000","last_became_active":"2024-09-20T22:02:52.480118+0000","last_became_peered":"2024-09-20T22:02:52.480118+0000","last_unstale":"2024-09-20T22:08:26.398930+0000","last_undegraded":"2024-09-20T22:08:26.398930+0000","last_fullsized":"2024-09-20T22:08:26.398930+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:06:36.942620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:36.942620+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":291,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.399011+0000","last_change":"2024-09-20T22:07:50.851948+0000","last_active":"2024-09-20T22:08:26.399011+0000","last_peered":"2024-09-20T22:08:26.399011+0000","last_clean":"2024-09-20T22:08:26.399011+0000","last_became_active":"2024-09-20T22:06:54.828826+0000","last_became_peered":"2024-09-20T22:06:54.828826+0000","last_unstale":"2024-09-20T22:08:26.399011+0000","last_undegraded":"2024-09-20T22:08:26.399011+0000","last_fullsized":"2024-09-20T22:08:26.399011+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:50.851839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:50.851839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:57.182737+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":275,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.401271+0000","last_change":"2024-09-20T22:06:45.780268+0000","last_active":"2024-09-20T22:08:26.401271+0000","last_peered":"2024-09-20T22:08:26.401271+0000","last_clean":"2024-09-20T22:08:26.401271+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:08:26.401271+0000","last_undegraded":"2024-09-20T22:08:26.401271+0000","last_fullsized":"2024-09-20T22:08:26.401271+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.780200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.780200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":270,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.401956+0000","last_change":"2024-09-20T22:06:33.002428+0000","last_active":"2024-09-20T22:08:26.401956+0000","last_peered":"2024-09-20T22:08:26.401956+0000","last_clean":"2024-09-20T22:08:26.401956+0000","last_became_active":"2024-09-20T22:02:56.347637+0000","last_became_peered":"2024-09-20T22:02:56.347637+0000","last_unstale":"2024-09-20T22:08:26.401956+0000","last_undegraded":"2024-09-20T22:08:26.401956+0000","last_fullsized":"2024-09-20T22:08:26.401956+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:33.002326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:33.002326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"71'4","reported_seq":285,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.399938+0000","last_change":"2024-09-20T22:07:19.309318+0000","last_active":"2024-09-20T22:08:26.399938+0000","last_peered":"2024-09-20T22:08:26.399938+0000","last_clean":"2024-09-20T22:08:26.399938+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:08:26.399938+0000","last_undegraded":"2024-09-20T22:08:26.399938+0000","last_fullsized":"2024-09-20T22:08:26.399938+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'4","last_scrub_stamp":"2024-09-20T22:07:19.309187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.309187+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:33.061321+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":282,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:29.852576+0000","last_change":"2024-09-20T22:08:29.852608+0000","last_active":"2024-09-20T22:08:29.852576+0000","last_peered":"2024-09-20T22:08:29.852576+0000","last_clean":"2024-09-20T22:08:29.852576+0000","last_became_active":"2024-09-20T22:02:50.111142+0000","last_became_peered":"2024-09-20T22:02:50.111142+0000","last_unstale":"2024-09-20T22:08:29.852576+0000","last_undegraded":"2024-09-20T22:08:29.852576+0000","last_fullsized":"2024-09-20T22:08:29.852576+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:29.852502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:29.852502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:45.967665+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":290,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.396646+0000","last_change":"2024-09-20T22:07:55.255933+0000","last_active":"2024-09-20T22:08:26.396646+0000","last_peered":"2024-09-20T22:08:26.396646+0000","last_clean":"2024-09-20T22:08:26.396646+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:08:26.396646+0000","last_undegraded":"2024-09-20T22:08:26.396646+0000","last_fullsized":"2024-09-20T22:08:26.396646+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:55.255854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.255854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.819675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":263,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.401488+0000","last_change":"2024-09-20T22:06:01.621567+0000","last_active":"2024-09-20T22:08:26.401488+0000","last_peered":"2024-09-20T22:08:26.401488+0000","last_clean":"2024-09-20T22:08:26.401488+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:08:26.401488+0000","last_undegraded":"2024-09-20T22:08:26.401488+0000","last_fullsized":"2024-09-20T22:08:26.401488+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:01.621489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:01.621489+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":160,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.396797+0000","last_change":"2024-09-20T22:06:54.828772+0000","last_active":"2024-09-20T22:08:26.396797+0000","last_peered":"2024-09-20T22:08:26.396797+0000","last_clean":"2024-09-20T22:08:26.396797+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:08:26.396797+0000","last_undegraded":"2024-09-20T22:08:26.396797+0000","last_fullsized":"2024-09-20T22:08:26.396797+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:15.303300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:15.303300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:28.363437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":283,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.400215+0000","last_change":"2024-09-20T22:07:57.428285+0000","last_active":"2024-09-20T22:08:26.400215+0000","last_peered":"2024-09-20T22:08:26.400215+0000","last_clean":"2024-09-20T22:08:26.400215+0000","last_became_active":"2024-09-20T22:06:54.848032+0000","last_became_peered":"2024-09-20T22:06:54.848032+0000","last_unstale":"2024-09-20T22:08:26.400215+0000","last_undegraded":"2024-09-20T22:08:26.400215+0000","last_fullsized":"2024-09-20T22:08:26.400215+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:57.428165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:57.428165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:04.979638+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"71'11","reported_seq":298,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.397434+0000","last_change":"2024-09-20T22:07:19.214716+0000","last_active":"2024-09-20T22:08:26.397434+0000","last_peered":"2024-09-20T22:08:26.397434+0000","last_clean":"2024-09-20T22:08:26.397434+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:08:26.397434+0000","last_undegraded":"2024-09-20T22:08:26.397434+0000","last_fullsized":"2024-09-20T22:08:26.397434+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:19.214636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.214636+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:32.168748+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":266,"reported_epoch":191,"state":"active+clean","last_fresh":"2024-09-20T22:08:26.395214+0000","last_change":"2024-09-20T22:05:06.706314+0000","last_active":"2024-09-20T22:08:26.395214+0000","last_peered":"2024-09-20T22:08:26.395214+0000","last_clean":"2024-09-20T22:08:26.395214+0000","last_became_active":"2024-09-20T22:02:54.289078+0000","last_became_peered":"2024-09-20T22:02:54.289078+0000","last_unstale":"2024-09-20T22:08:26.395214+0000","last_undegraded":"2024-09-20T22:08:26.395214+0000","last_fullsized":"2024-09-20T22:08:26.395214+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:06.681123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:06.681123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":82,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":4142006272,"num_objects":63202,"num_object_clones":0,"num_object_copies":189606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63202,"num_write_kb":4044928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12509708288,"data_stored":12509708288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63202,"ondisk_log_size":63202,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":69,"ondisk_log_size":69,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":327,"num_read_kb":322,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135944,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1458164,"kb_used_data":1264548,"kb_used_omap":1,"kb_used_meta":193598,"kb_avail":92286988,"statfs":{"total":95995035648,"available":94501875712,"internally_reserved":0,"allocated":1294897152,"data_stored":1294088526,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":198244810},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430284,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1486244,"kb_used_data":1288468,"kb_used_omap":1,"kb_used_meta":197758,"kb_avail":92258908,"statfs":{"total":95995035648,"available":94473121792,"internally_reserved":0,"allocated":1319391232,"data_stored":1318598898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":202504646},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724624,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2188644,"kb_used_data":2017236,"kb_used_omap":1,"kb_used_meta":171390,"kb_avail":91556508,"statfs":{"total":95995035648,"available":93753864192,"internally_reserved":0,"allocated":2065649664,"data_stored":2064857242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":175503813},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986264,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1471224,"kb_used_data":1269288,"kb_used_omap":1,"kb_used_meta":201918,"kb_avail":92273928,"statfs":{"total":95995035648,"available":94488502272,"internally_reserved":0,"allocated":1299750912,"data_stored":1298937589,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":206764485},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770257940,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1793168,"kb_used_data":1663548,"kb_used_omap":4,"kb_used_meta":129595,"kb_avail":91951984,"statfs":{"total":95995035648,"available":94158831616,"internally_reserved":0,"allocated":1703473152,"data_stored":1702639763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":132705960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,2,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149789,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1880008,"kb_used_data":1533688,"kb_used_omap":1,"kb_used_meta":346302,"kb_avail":91865144,"statfs":{"total":95995035648,"available":94069907456,"internally_reserved":0,"allocated":1570496512,"data_stored":1569668097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":354613703},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,2,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1757276,"kb_used_data":1523404,"kb_used_omap":1,"kb_used_meta":233854,"kb_avail":91987876,"statfs":{"total":95995035648,"available":94195585024,"internally_reserved":0,"allocated":1559965696,"data_stored":1559180372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":239466949},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738477,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1886456,"kb_used_data":1535976,"kb_used_omap":1,"kb_used_meta":350462,"kb_avail":91858696,"statfs":{"total":95995035648,"available":94063304704,"internally_reserved":0,"allocated":1572839424,"data_stored":1572026834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":358873541},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1585905664,"data_stored":1585905664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1715535872,"data_stored":1715535872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1586036736,"data_stored":1586036736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1586102272,"data_stored":1586102272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1303379968,"data_stored":1303379968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2090401792,"data_stored":2090401792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1324941312,"data_stored":1324941312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1317404672,"data_stored":1317404672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:08:35.423 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-20T22:08:35.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool set .mgr pg_num 11 2024-09-20T22:08:35.424 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:35.457 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:35.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:35.458 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:35.504 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:08:35.558 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:35.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:35.587 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:35.587 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[29694]: 6.4 deep-scrub starts 2024-09-20T22:08:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[29694]: 6.4 deep-scrub ok 2024-09-20T22:08:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[29694]: 5.13 scrub starts 2024-09-20T22:08:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[29694]: 5.13 scrub ok 2024-09-20T22:08:35.636 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[29694]: 6.1c scrub starts 2024-09-20T22:08:35.637 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[29694]: 6.1c scrub ok 2024-09-20T22:08:35.637 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[29694]: 3.1b deep-scrub starts 2024-09-20T22:08:35.637 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[29694]: 3.1b deep-scrub ok 2024-09-20T22:08:35.637 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:35.637 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:08:35.637 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[40290]: 6.4 deep-scrub starts 2024-09-20T22:08:35.637 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[40290]: 6.4 deep-scrub ok 2024-09-20T22:08:35.637 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[40290]: 5.13 scrub starts 2024-09-20T22:08:35.637 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[40290]: 5.13 scrub ok 2024-09-20T22:08:35.637 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[40290]: 6.1c scrub starts 2024-09-20T22:08:35.638 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[40290]: 6.1c scrub ok 2024-09-20T22:08:35.638 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[40290]: 3.1b deep-scrub starts 2024-09-20T22:08:35.638 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[40290]: 3.1b deep-scrub ok 2024-09-20T22:08:35.638 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:35.638 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:35 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:08:35.687 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:35.711 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:35.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:35.712 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:35.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:35 smithi067 ceph-mon[33511]: 6.4 deep-scrub starts 2024-09-20T22:08:35.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:35 smithi067 ceph-mon[33511]: 6.4 deep-scrub ok 2024-09-20T22:08:35.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:35 smithi067 ceph-mon[33511]: 5.13 scrub starts 2024-09-20T22:08:35.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:35 smithi067 ceph-mon[33511]: 5.13 scrub ok 2024-09-20T22:08:35.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:35 smithi067 ceph-mon[33511]: 6.1c scrub starts 2024-09-20T22:08:35.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:35 smithi067 ceph-mon[33511]: 6.1c scrub ok 2024-09-20T22:08:35.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:35 smithi067 ceph-mon[33511]: 3.1b deep-scrub starts 2024-09-20T22:08:35.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:35 smithi067 ceph-mon[33511]: 3.1b deep-scrub ok 2024-09-20T22:08:35.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:35 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:35.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:35 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:08:35.813 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:35.831 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:08:35.831 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:08:35.842 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:35.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:35.843 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:35.944 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:35.988 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:35.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:35.989 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:36.090 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:36.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:36.120 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:36.120 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:36.142 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 39 16 71101 71085 113.886 77.3125 0.0118204 0.00877615 2024-09-20T22:08:36.221 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:36.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:36.263 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:36.264 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:36.364 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:36.397 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:36.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:36.398 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:36.502 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:36.531 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:36.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:36.532 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:36.615 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:08:36.633 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:36.705 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:36.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:36.706 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:36.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:36 smithi067 ceph-mon[33511]: 3.1d deep-scrub starts 2024-09-20T22:08:36.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:36 smithi067 ceph-mon[33511]: 3.1d deep-scrub ok 2024-09-20T22:08:36.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:36 smithi067 ceph-mon[33511]: 4.7 scrub starts 2024-09-20T22:08:36.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:36 smithi067 ceph-mon[33511]: 4.7 scrub ok 2024-09-20T22:08:36.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:36 smithi067 ceph-mon[33511]: 6.16 scrub starts 2024-09-20T22:08:36.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:36 smithi067 ceph-mon[33511]: 6.16 scrub ok 2024-09-20T22:08:36.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:36 smithi067 ceph-mon[33511]: pgmap v280: 148 pgs: 148 active+clean; 4.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.0 KiB/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-20T22:08:36.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:36 smithi067 ceph-mon[33511]: osdmap e193: 8 total, 8 up, 8 in 2024-09-20T22:08:36.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:36 smithi060 ceph-mon[29694]: 3.1d deep-scrub starts 2024-09-20T22:08:36.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:36 smithi060 ceph-mon[29694]: 3.1d deep-scrub ok 2024-09-20T22:08:36.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:36 smithi060 ceph-mon[29694]: 4.7 scrub starts 2024-09-20T22:08:36.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:36 smithi060 ceph-mon[29694]: 4.7 scrub ok 2024-09-20T22:08:36.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:36 smithi060 ceph-mon[29694]: 6.16 scrub starts 2024-09-20T22:08:36.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:36 smithi060 ceph-mon[29694]: 6.16 scrub ok 2024-09-20T22:08:36.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:36 smithi060 ceph-mon[29694]: pgmap v280: 148 pgs: 148 active+clean; 4.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.0 KiB/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-20T22:08:36.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:36 smithi060 ceph-mon[29694]: osdmap e193: 8 total, 8 up, 8 in 2024-09-20T22:08:36.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:36 smithi060 ceph-mon[40290]: 3.1d deep-scrub starts 2024-09-20T22:08:36.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:36 smithi060 ceph-mon[40290]: 3.1d deep-scrub ok 2024-09-20T22:08:36.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:36 smithi060 ceph-mon[40290]: 4.7 scrub starts 2024-09-20T22:08:36.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:36 smithi060 ceph-mon[40290]: 4.7 scrub ok 2024-09-20T22:08:36.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:36 smithi060 ceph-mon[40290]: 6.16 scrub starts 2024-09-20T22:08:36.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:36 smithi060 ceph-mon[40290]: 6.16 scrub ok 2024-09-20T22:08:36.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:36 smithi060 ceph-mon[40290]: pgmap v280: 148 pgs: 148 active+clean; 4.1 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.0 KiB/s rd, 72 MiB/s wr, 1.15k op/s 2024-09-20T22:08:36.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:36 smithi060 ceph-mon[40290]: osdmap e193: 8 total, 8 up, 8 in 2024-09-20T22:08:36.807 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:36.842 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:36.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:36.843 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:36.943 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:36.972 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:36.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:36.973 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:37.074 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:37.109 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:37.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:37.110 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:37.142 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:08:37.142200+0000 min lat: 0.00126498 max lat: 0.154538 avg lat: 0.00883874 2024-09-20T22:08:37.143 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:08:37.143 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 40 16 72329 72313 112.958 76.75 0.00254245 0.00883874 2024-09-20T22:08:37.211 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:37.245 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:37.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:37.247 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:37.347 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:37.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:37.370 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:37.370 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:37.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:37.513 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:37.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:37.514 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:37.615 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:37.654 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:37.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:37.655 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:37.756 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:37.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:37 smithi067 ceph-mon[33511]: 2.1 deep-scrub starts 2024-09-20T22:08:37.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:37 smithi067 ceph-mon[33511]: 2.1 deep-scrub ok 2024-09-20T22:08:37.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:37 smithi067 ceph-mon[33511]: 5.1c scrub starts 2024-09-20T22:08:37.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:37 smithi067 ceph-mon[33511]: 5.1c scrub ok 2024-09-20T22:08:37.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:37 smithi067 ceph-mon[33511]: 5.17 scrub starts 2024-09-20T22:08:37.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:37 smithi067 ceph-mon[33511]: 5.17 scrub ok 2024-09-20T22:08:37.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:37 smithi067 ceph-mon[33511]: 6.b scrub starts 2024-09-20T22:08:37.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:37 smithi067 ceph-mon[33511]: 6.b scrub ok 2024-09-20T22:08:37.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:37 smithi060 ceph-mon[29694]: 2.1 deep-scrub starts 2024-09-20T22:08:37.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:37 smithi060 ceph-mon[29694]: 2.1 deep-scrub ok 2024-09-20T22:08:37.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:37 smithi060 ceph-mon[29694]: 5.1c scrub starts 2024-09-20T22:08:37.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:37 smithi060 ceph-mon[29694]: 5.1c scrub ok 2024-09-20T22:08:37.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:37 smithi060 ceph-mon[29694]: 5.17 scrub starts 2024-09-20T22:08:37.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:37 smithi060 ceph-mon[29694]: 5.17 scrub ok 2024-09-20T22:08:37.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:37 smithi060 ceph-mon[29694]: 6.b scrub starts 2024-09-20T22:08:37.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:37 smithi060 ceph-mon[29694]: 6.b scrub ok 2024-09-20T22:08:37.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:37 smithi060 ceph-mon[40290]: 2.1 deep-scrub starts 2024-09-20T22:08:37.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:37 smithi060 ceph-mon[40290]: 2.1 deep-scrub ok 2024-09-20T22:08:37.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:37 smithi060 ceph-mon[40290]: 5.1c scrub starts 2024-09-20T22:08:37.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:37 smithi060 ceph-mon[40290]: 5.1c scrub ok 2024-09-20T22:08:37.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:37 smithi060 ceph-mon[40290]: 5.17 scrub starts 2024-09-20T22:08:37.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:37 smithi060 ceph-mon[40290]: 5.17 scrub ok 2024-09-20T22:08:37.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:37 smithi060 ceph-mon[40290]: 6.b scrub starts 2024-09-20T22:08:37.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:37 smithi060 ceph-mon[40290]: 6.b scrub ok 2024-09-20T22:08:37.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:37.788 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:37.788 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:37.889 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:37.930 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:37.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:37.932 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:38.032 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:38.065 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:38.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:38.066 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:38.142 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 41 16 73567 73551 112.09 77.375 0.00822317 0.00891646 2024-09-20T22:08:38.167 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:38.206 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:38.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:38.207 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:38.307 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:38.341 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:38.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:38.342 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:38.443 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:38.475 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:38.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:38.476 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:38.577 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:38.606 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:38.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:38.607 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:38.708 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:38.739 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:38.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:38.740 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:38.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:38 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:08:38.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:38 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:08:38.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:38 smithi067 ceph-mon[33511]: osdmap e194: 8 total, 8 up, 8 in 2024-09-20T22:08:38.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:38 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:08:38.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:38 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:08:38.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:38 smithi067 ceph-mon[33511]: pgmap v283: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 85 MiB/s wr, 1.36k op/s 2024-09-20T22:08:38.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:38 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:08:38.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:38 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:08:38.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:38 smithi060 ceph-mon[29694]: osdmap e194: 8 total, 8 up, 8 in 2024-09-20T22:08:38.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:38 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:08:38.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:38 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:08:38.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:38 smithi060 ceph-mon[29694]: pgmap v283: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 85 MiB/s wr, 1.36k op/s 2024-09-20T22:08:38.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:38 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:08:38.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:38 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:08:38.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:38 smithi060 ceph-mon[40290]: osdmap e194: 8 total, 8 up, 8 in 2024-09-20T22:08:38.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:38 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:08:38.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:38 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:08:38.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:38 smithi060 ceph-mon[40290]: pgmap v283: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 85 MiB/s wr, 1.36k op/s 2024-09-20T22:08:38.841 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:38.916 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T22:08:38.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:38.947 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:38.967 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:38.980 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:39.048 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:39.085 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:39.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:39.086 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:39.142 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 42 16 74702 74686 111.11 70.9375 0.0110838 0.00899494 2024-09-20T22:08:39.187 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:39.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:39.218 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:39.218 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:39.319 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:39.378 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:39.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:39.380 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:39.480 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:39.509 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:39.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:39.511 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:39.611 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:39.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:39.662 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:39.663 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:39.763 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:39.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:39 smithi067 ceph-mon[33511]: 4.8 scrub starts 2024-09-20T22:08:39.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:39 smithi067 ceph-mon[33511]: 4.8 scrub ok 2024-09-20T22:08:39.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:39 smithi067 ceph-mon[33511]: osdmap e195: 8 total, 8 up, 8 in 2024-09-20T22:08:39.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:39 smithi067 ceph-mon[33511]: 6.2 scrub starts 2024-09-20T22:08:39.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:39 smithi067 ceph-mon[33511]: 6.2 scrub ok 2024-09-20T22:08:39.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:39 smithi067 ceph-mon[33511]: 4.14 scrub starts 2024-09-20T22:08:39.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:39 smithi067 ceph-mon[33511]: 4.14 scrub ok 2024-09-20T22:08:39.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:39 smithi060 ceph-mon[40290]: 4.8 scrub starts 2024-09-20T22:08:39.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:39 smithi060 ceph-mon[40290]: 4.8 scrub ok 2024-09-20T22:08:39.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:39 smithi060 ceph-mon[40290]: osdmap e195: 8 total, 8 up, 8 in 2024-09-20T22:08:39.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:39 smithi060 ceph-mon[40290]: 6.2 scrub starts 2024-09-20T22:08:39.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:39 smithi060 ceph-mon[40290]: 6.2 scrub ok 2024-09-20T22:08:39.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:39 smithi060 ceph-mon[40290]: 4.14 scrub starts 2024-09-20T22:08:39.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:39 smithi060 ceph-mon[40290]: 4.14 scrub ok 2024-09-20T22:08:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:39 smithi060 ceph-mon[29694]: 4.8 scrub starts 2024-09-20T22:08:39.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:39 smithi060 ceph-mon[29694]: 4.8 scrub ok 2024-09-20T22:08:39.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:39 smithi060 ceph-mon[29694]: osdmap e195: 8 total, 8 up, 8 in 2024-09-20T22:08:39.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:39 smithi060 ceph-mon[29694]: 6.2 scrub starts 2024-09-20T22:08:39.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:39 smithi060 ceph-mon[29694]: 6.2 scrub ok 2024-09-20T22:08:39.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:39 smithi060 ceph-mon[29694]: 4.14 scrub starts 2024-09-20T22:08:39.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:39 smithi060 ceph-mon[29694]: 4.14 scrub ok 2024-09-20T22:08:39.799 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:39.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:39.800 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:39.901 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:39.940 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:39.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:39.941 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:40.042 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:40.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:40.075 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:40.075 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:40.143 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 43 16 75934 75918 110.317 77 0.0135871 0.00905946 2024-09-20T22:08:40.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:40.198 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:40.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:40.199 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:40.300 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:40.340 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:40.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:40.341 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:40.441 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:40.475 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:40.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:40.476 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:40.577 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:40.609 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:40.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:40.610 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:40.684 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[29694]: 6.7 scrub starts 2024-09-20T22:08:40.684 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[29694]: 6.7 scrub ok 2024-09-20T22:08:40.684 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[29694]: 5.12 scrub starts 2024-09-20T22:08:40.684 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[29694]: 5.12 scrub ok 2024-09-20T22:08:40.685 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[29694]: 4.1c deep-scrub starts 2024-09-20T22:08:40.685 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[29694]: 4.1c deep-scrub ok 2024-09-20T22:08:40.685 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[29694]: 4.19 scrub starts 2024-09-20T22:08:40.685 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[29694]: 4.19 scrub ok 2024-09-20T22:08:40.685 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[29694]: 6.12 scrub starts 2024-09-20T22:08:40.685 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[29694]: 6.12 scrub ok 2024-09-20T22:08:40.685 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[29694]: pgmap v285: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 67 MiB/s wr, 1.08k op/s 2024-09-20T22:08:40.685 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[40290]: 6.7 scrub starts 2024-09-20T22:08:40.685 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[40290]: 6.7 scrub ok 2024-09-20T22:08:40.686 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[40290]: 5.12 scrub starts 2024-09-20T22:08:40.686 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[40290]: 5.12 scrub ok 2024-09-20T22:08:40.686 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[40290]: 4.1c deep-scrub starts 2024-09-20T22:08:40.686 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[40290]: 4.1c deep-scrub ok 2024-09-20T22:08:40.686 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[40290]: 4.19 scrub starts 2024-09-20T22:08:40.686 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[40290]: 4.19 scrub ok 2024-09-20T22:08:40.686 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[40290]: 6.12 scrub starts 2024-09-20T22:08:40.686 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[40290]: 6.12 scrub ok 2024-09-20T22:08:40.686 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:40 smithi060 ceph-mon[40290]: pgmap v285: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 67 MiB/s wr, 1.08k op/s 2024-09-20T22:08:40.710 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:40.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:40.752 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:40.753 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:40 smithi067 ceph-mon[33511]: 6.7 scrub starts 2024-09-20T22:08:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:40 smithi067 ceph-mon[33511]: 6.7 scrub ok 2024-09-20T22:08:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:40 smithi067 ceph-mon[33511]: 5.12 scrub starts 2024-09-20T22:08:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:40 smithi067 ceph-mon[33511]: 5.12 scrub ok 2024-09-20T22:08:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:40 smithi067 ceph-mon[33511]: 4.1c deep-scrub starts 2024-09-20T22:08:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:40 smithi067 ceph-mon[33511]: 4.1c deep-scrub ok 2024-09-20T22:08:40.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:40 smithi067 ceph-mon[33511]: 4.19 scrub starts 2024-09-20T22:08:40.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:40 smithi067 ceph-mon[33511]: 4.19 scrub ok 2024-09-20T22:08:40.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:40 smithi067 ceph-mon[33511]: 6.12 scrub starts 2024-09-20T22:08:40.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:40 smithi067 ceph-mon[33511]: 6.12 scrub ok 2024-09-20T22:08:40.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:40 smithi067 ceph-mon[33511]: pgmap v285: 148 pgs: 148 active+clean; 4.4 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 67 MiB/s wr, 1.08k op/s 2024-09-20T22:08:40.853 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:40.889 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:40.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:40.891 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:40.991 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:41.032 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:41.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:41.033 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:41.133 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:41.143 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 44 16 77028 77012 109.364 68.375 0.0217696 0.00913836 2024-09-20T22:08:41.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:41.162 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:41.162 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:41.263 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:41.320 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:41.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:41.321 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:41.421 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:41.452 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:41.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:41.453 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:41.488 INFO:teuthology.orchestra.run.smithi060.stderr:set pool 1 pg_num to 11 2024-09-20T22:08:41.510 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:41 smithi060 ceph-mon[29694]: 3.9 scrub starts 2024-09-20T22:08:41.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:41 smithi060 ceph-mon[29694]: 3.9 scrub ok 2024-09-20T22:08:41.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:41 smithi060 ceph-mon[29694]: 4.0 scrub starts 2024-09-20T22:08:41.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:41 smithi060 ceph-mon[29694]: 4.0 scrub ok 2024-09-20T22:08:41.511 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:41 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1489035959' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-20T22:08:41.511 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:41 smithi060 ceph-mon[40290]: 3.9 scrub starts 2024-09-20T22:08:41.511 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:41 smithi060 ceph-mon[40290]: 3.9 scrub ok 2024-09-20T22:08:41.511 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:41 smithi060 ceph-mon[40290]: 4.0 scrub starts 2024-09-20T22:08:41.511 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:41 smithi060 ceph-mon[40290]: 4.0 scrub ok 2024-09-20T22:08:41.511 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:41 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1489035959' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-20T22:08:41.554 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:41.573 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:08:41.573 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:08:41.605 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:41.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:41.606 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:41.630 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:08:41.630 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:08:41.667 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:08:41.667 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:08:41.708 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:41.713 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:08:41.713 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:08:41.738 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:41.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:41.739 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:41.755 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:08:41.755 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:08:41.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:41 smithi067 ceph-mon[33511]: 3.9 scrub starts 2024-09-20T22:08:41.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:41 smithi067 ceph-mon[33511]: 3.9 scrub ok 2024-09-20T22:08:41.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:41 smithi067 ceph-mon[33511]: 4.0 scrub starts 2024-09-20T22:08:41.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:41 smithi067 ceph-mon[33511]: 4.0 scrub ok 2024-09-20T22:08:41.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:41 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1489035959' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]: dispatch 2024-09-20T22:08:41.797 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:08:41.798 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:08:41.840 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:41.848 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:08:41.848 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:08:41.879 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:41.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:41.880 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:41.887 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:08:41.888 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:08:41.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:42.018 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:42.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:42.019 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:42.120 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:42.143 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 45 15 78140 78125 108.479 69.5625 0.0102421 0.00921381 2024-09-20T22:08:42.157 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:42.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:42.158 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:42.259 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:42.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:42.298 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:42.298 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:42.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:42.453 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:42.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:42.454 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:42.513 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:08:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:08:42.521 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:08:42.554 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:42.588 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:42.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:42.589 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:42.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1489035959' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-20T22:08:42.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[29694]: osdmap e196: 8 total, 8 up, 8 in 2024-09-20T22:08:42.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/594928178' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:42.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[29694]: 6.1b scrub starts 2024-09-20T22:08:42.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[29694]: 6.1b scrub ok 2024-09-20T22:08:42.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[29694]: 6.19 scrub starts 2024-09-20T22:08:42.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/998597797' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:08:42.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[29694]: 6.19 scrub ok 2024-09-20T22:08:42.649 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:08:42.649 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[29694]: pgmap v287: 148 pgs: 148 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 907 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-20T22:08:42.649 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-20T22:08:42.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1489035959' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-20T22:08:42.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[40290]: osdmap e196: 8 total, 8 up, 8 in 2024-09-20T22:08:42.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/594928178' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:42.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[40290]: 6.1b scrub starts 2024-09-20T22:08:42.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[40290]: 6.1b scrub ok 2024-09-20T22:08:42.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[40290]: 6.19 scrub starts 2024-09-20T22:08:42.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/998597797' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:08:42.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[40290]: 6.19 scrub ok 2024-09-20T22:08:42.650 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:08:42.650 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[40290]: pgmap v287: 148 pgs: 148 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 907 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-20T22:08:42.650 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:42 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-20T22:08:42.689 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:42.729 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:42.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:42.730 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:42.831 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:42.865 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:42.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:42.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:42.967 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:43.012 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:43.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:43.013 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:42 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1489035959' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "11"}]': finished 2024-09-20T22:08:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:42 smithi067 ceph-mon[33511]: osdmap e196: 8 total, 8 up, 8 in 2024-09-20T22:08:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:42 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/594928178' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:42 smithi067 ceph-mon[33511]: 6.1b scrub starts 2024-09-20T22:08:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:42 smithi067 ceph-mon[33511]: 6.1b scrub ok 2024-09-20T22:08:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:42 smithi067 ceph-mon[33511]: 6.19 scrub starts 2024-09-20T22:08:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:42 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/998597797' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:08:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:42 smithi067 ceph-mon[33511]: 6.19 scrub ok 2024-09-20T22:08:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:42 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:08:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:42 smithi067 ceph-mon[33511]: pgmap v287: 148 pgs: 148 active+clean; 4.6 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 907 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-20T22:08:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:42 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]: dispatch 2024-09-20T22:08:43.114 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:43.143 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 46 16 79431 79415 107.874 80.625 0.0100893 0.00926396 2024-09-20T22:08:43.147 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:43.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:43.148 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:43.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:43.287 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:43.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:43.288 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:43.389 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:43.438 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:43.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:43.439 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:43.540 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:43.585 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:43.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:43.586 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:43.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:43.731 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:43.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:43.732 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:43.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:43 smithi060 ceph-mon[29694]: 5.1b scrub starts 2024-09-20T22:08:43.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:43 smithi060 ceph-mon[29694]: 5.1b scrub ok 2024-09-20T22:08:43.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:43 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:08:43.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:43 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/998597797' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:08:43.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:43 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-20T22:08:43.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:43 smithi060 ceph-mon[29694]: osdmap e197: 8 total, 8 up, 8 in 2024-09-20T22:08:43.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:43 smithi060 ceph-mon[40290]: 5.1b scrub starts 2024-09-20T22:08:43.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:43 smithi060 ceph-mon[40290]: 5.1b scrub ok 2024-09-20T22:08:43.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:43 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:08:43.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:43 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/998597797' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:08:43.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:43 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-20T22:08:43.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:43 smithi060 ceph-mon[40290]: osdmap e197: 8 total, 8 up, 8 in 2024-09-20T22:08:43.832 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:43.868 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:43.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:43.869 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:43.970 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:44.001 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:44.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:44.002 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:43 smithi067 ceph-mon[33511]: 5.1b scrub starts 2024-09-20T22:08:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:43 smithi067 ceph-mon[33511]: 5.1b scrub ok 2024-09-20T22:08:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:43 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:08:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:43 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/998597797' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:08:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:43 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "11"}]': finished 2024-09-20T22:08:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:43 smithi067 ceph-mon[33511]: osdmap e197: 8 total, 8 up, 8 in 2024-09-20T22:08:44.041 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:44.103 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:44.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:44.133 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:44.133 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:44.143 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 47 16 80671 80655 107.227 77.5 0.0108182 0.00932156 2024-09-20T22:08:44.234 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:44.268 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:44.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:44.269 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:44.370 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:44.412 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:44.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:44.413 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:44.514 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:44.554 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:44.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:44.555 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:44.655 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:44.686 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:44.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:44.688 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:44.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:44 smithi060 ceph-mon[29694]: osdmap e198: 8 total, 8 up, 8 in 2024-09-20T22:08:44.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:44 smithi060 ceph-mon[29694]: pgmap v290: 158 pgs: 10 unknown, 148 active+clean; 4.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 920 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-20T22:08:44.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:44 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-20T22:08:44.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:44 smithi060 ceph-mon[40290]: osdmap e198: 8 total, 8 up, 8 in 2024-09-20T22:08:44.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:44 smithi060 ceph-mon[40290]: pgmap v290: 158 pgs: 10 unknown, 148 active+clean; 4.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 920 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-20T22:08:44.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:44 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-20T22:08:44.788 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:44.815 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:44.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:44.816 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:44.917 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:44.939 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:08:44.939 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:08:44.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:08:44.939 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:08:44.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.939 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47598 7.3 7:c6c29109:::benchmark_data_smithi060_78800_object47597:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:08:44.939 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.238604+0000", 2024-09-20T22:08:44.939 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 264.69588709999999, 2024-09-20T22:08:44.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0797809089999999, 2024-09-20T22:08:44.939 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:44.939 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:44.939 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:44.940 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:44.940 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:44.940 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:44.940 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47598 2024-09-20T22:08:44.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.940 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:44.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:44.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238604+0000", 2024-09-20T22:08:44.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:44.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:44.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238604+0000", 2024-09-20T22:08:44.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:44.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:44.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238605+0000", 2024-09-20T22:08:44.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3099999999999996e-07 2024-09-20T22:08:44.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:44.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238626+0000", 2024-09-20T22:08:44.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1192999999999999e-05 2024-09-20T22:08:44.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:44.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238637+0000", 2024-09-20T22:08:44.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0516000000000001e-05 2024-09-20T22:08:44.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:44.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238644+0000", 2024-09-20T22:08:44.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3969999999999997e-06 2024-09-20T22:08:44.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.240757+0000", 2024-09-20T22:08:44.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021131100000000001 2024-09-20T22:08:44.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op must wait for map", 2024-09-20T22:08:44.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.240764+0000", 2024-09-20T22:08:44.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9550000000000003e-06 2024-09-20T22:08:44.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.246730+0000", 2024-09-20T22:08:44.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0059659020000000004 2024-09-20T22:08:44.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:44.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.246741+0000", 2024-09-20T22:08:44.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0855e-05 2024-09-20T22:08:44.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.235644+0000", 2024-09-20T22:08:44.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98890363400000003 2024-09-20T22:08:44.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:44.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.235657+0000", 2024-09-20T22:08:44.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3040000000000001e-05 2024-09-20T22:08:44.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.313762+0000", 2024-09-20T22:08:44.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.078104783999999997 2024-09-20T22:08:44.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:44.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.314529+0000", 2024-09-20T22:08:44.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00076653400000000003 2024-09-20T22:08:44.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:08:44.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.314632+0000", 2024-09-20T22:08:44.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103224 2024-09-20T22:08:44.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:44.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318215+0000", 2024-09-20T22:08:44.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0035831439999999999 2024-09-20T22:08:44.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:44.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318247+0000", 2024-09-20T22:08:44.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1804999999999998e-05 2024-09-20T22:08:44.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:44.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318292+0000", 2024-09-20T22:08:44.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4605999999999999e-05 2024-09-20T22:08:44.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:44.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318329+0000", 2024-09-20T22:08:44.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7175999999999998e-05 2024-09-20T22:08:44.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:44.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318385+0000", 2024-09-20T22:08:44.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6202999999999997e-05 2024-09-20T22:08:44.947 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:44.947 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:44.947 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:44.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.948 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47618 7.3 7:c8aeb189:::benchmark_data_smithi060_78800_object47617:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:08:44.948 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.257444+0000", 2024-09-20T22:08:44.948 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 264.67704711699997, 2024-09-20T22:08:44.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0612655090000001, 2024-09-20T22:08:44.948 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:44.948 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:44.948 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:44.948 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:44.948 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:44.948 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:44.948 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47618 2024-09-20T22:08:44.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.948 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:44.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:44.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257444+0000", 2024-09-20T22:08:44.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:44.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:44.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257444+0000", 2024-09-20T22:08:44.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:44.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:44.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257445+0000", 2024-09-20T22:08:44.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3790000000000001e-06 2024-09-20T22:08:44.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:44.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257468+0000", 2024-09-20T22:08:44.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2078000000000001e-05 2024-09-20T22:08:44.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:44.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257480+0000", 2024-09-20T22:08:44.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.253e-05 2024-09-20T22:08:44.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:44.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257488+0000", 2024-09-20T22:08:44.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4900000000000003e-06 2024-09-20T22:08:44.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257693+0000", 2024-09-20T22:08:44.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000205071 2024-09-20T22:08:44.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:44.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257702+0000", 2024-09-20T22:08:44.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6099999999999995e-06 2024-09-20T22:08:44.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236024+0000", 2024-09-20T22:08:44.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97832186300000001 2024-09-20T22:08:44.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:44.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236034+0000", 2024-09-20T22:08:44.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9189999999999994e-06 2024-09-20T22:08:44.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.316117+0000", 2024-09-20T22:08:44.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.080083484999999996 2024-09-20T22:08:44.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:44.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.316218+0000", 2024-09-20T22:08:44.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100846 2024-09-20T22:08:44.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:08:44.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.316363+0000", 2024-09-20T22:08:44.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014473099999999999 2024-09-20T22:08:44.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:44.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318472+0000", 2024-09-20T22:08:44.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021087039999999999 2024-09-20T22:08:44.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:44.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318532+0000", 2024-09-20T22:08:44.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0418999999999999e-05 2024-09-20T22:08:44.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:44.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318622+0000", 2024-09-20T22:08:44.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0074000000000005e-05 2024-09-20T22:08:44.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:44.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318659+0000", 2024-09-20T22:08:44.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6260999999999997e-05 2024-09-20T22:08:44.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:44.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318710+0000", 2024-09-20T22:08:44.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1048999999999999e-05 2024-09-20T22:08:44.955 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:44.955 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:44.955 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:44.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.955 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47683 7.3 7:cf1d2c6c:::benchmark_data_smithi060_78800_object47682:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:08:44.955 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.290834+0000", 2024-09-20T22:08:44.956 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 264.64365706400002, 2024-09-20T22:08:44.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.029069585, 2024-09-20T22:08:44.956 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:44.956 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:44.956 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:44.956 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:44.956 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:44.956 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:44.956 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47683 2024-09-20T22:08:44.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.956 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:44.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:44.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290834+0000", 2024-09-20T22:08:44.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:44.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:44.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290834+0000", 2024-09-20T22:08:44.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:44.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:44.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290835+0000", 2024-09-20T22:08:44.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2020000000000001e-06 2024-09-20T22:08:44.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:44.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290856+0000", 2024-09-20T22:08:44.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0585999999999999e-05 2024-09-20T22:08:44.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:44.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290868+0000", 2024-09-20T22:08:44.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2160999999999999e-05 2024-09-20T22:08:44.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:44.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290875+0000", 2024-09-20T22:08:44.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3200000000000002e-06 2024-09-20T22:08:44.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290955+0000", 2024-09-20T22:08:44.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9592999999999995e-05 2024-09-20T22:08:44.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:44.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290966+0000", 2024-09-20T22:08:44.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.119e-05 2024-09-20T22:08:44.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236104+0000", 2024-09-20T22:08:44.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94513756400000004 2024-09-20T22:08:44.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:44.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236112+0000", 2024-09-20T22:08:44.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2220000000000002e-06 2024-09-20T22:08:44.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.317361+0000", 2024-09-20T22:08:44.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.081249110999999999 2024-09-20T22:08:44.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:44.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.317444+0000", 2024-09-20T22:08:44.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2936999999999996e-05 2024-09-20T22:08:44.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:08:44.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.317551+0000", 2024-09-20T22:08:44.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010741100000000001 2024-09-20T22:08:44.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:44.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318732+0000", 2024-09-20T22:08:44.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0011802379999999999 2024-09-20T22:08:44.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:44.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319739+0000", 2024-09-20T22:08:44.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001007457 2024-09-20T22:08:44.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:44.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319786+0000", 2024-09-20T22:08:44.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6845999999999998e-05 2024-09-20T22:08:44.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:44.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319820+0000", 2024-09-20T22:08:44.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4103000000000002e-05 2024-09-20T22:08:44.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:44.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319904+0000", 2024-09-20T22:08:44.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3644000000000001e-05 2024-09-20T22:08:44.963 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:44.963 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:44.963 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:44.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.963 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47686 7.3 7:c4eaa487:::benchmark_data_smithi060_78800_object47685:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:08:44.963 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.291328+0000", 2024-09-20T22:08:44.963 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 264.64316349199999, 2024-09-20T22:08:44.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.03577915, 2024-09-20T22:08:44.963 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:44.963 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:44.963 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:44.964 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:44.964 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:44.964 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:44.964 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47686 2024-09-20T22:08:44.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.964 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:44.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:44.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291328+0000", 2024-09-20T22:08:44.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:44.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:44.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291328+0000", 2024-09-20T22:08:44.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:44.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:44.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291328+0000", 2024-09-20T22:08:44.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0699999999999996e-07 2024-09-20T22:08:44.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:44.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291353+0000", 2024-09-20T22:08:44.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4134999999999998e-05 2024-09-20T22:08:44.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:44.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291364+0000", 2024-09-20T22:08:44.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1501999999999999e-05 2024-09-20T22:08:44.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:44.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291371+0000", 2024-09-20T22:08:44.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1210000000000001e-06 2024-09-20T22:08:44.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291460+0000", 2024-09-20T22:08:44.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8869999999999994e-05 2024-09-20T22:08:44.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:44.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291473+0000", 2024-09-20T22:08:44.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2899e-05 2024-09-20T22:08:44.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236140+0000", 2024-09-20T22:08:44.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94466707500000002 2024-09-20T22:08:44.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:44.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236149+0000", 2024-09-20T22:08:44.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9509999999999995e-06 2024-09-20T22:08:44.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318750+0000", 2024-09-20T22:08:44.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.082601349000000004 2024-09-20T22:08:44.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:44.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318813+0000", 2024-09-20T22:08:44.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2306999999999998e-05 2024-09-20T22:08:44.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:08:44.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318890+0000", 2024-09-20T22:08:44.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7565999999999996e-05 2024-09-20T22:08:44.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:44.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323067+0000", 2024-09-20T22:08:44.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0041765129999999998 2024-09-20T22:08:44.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:44.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323117+0000", 2024-09-20T22:08:44.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0297000000000003e-05 2024-09-20T22:08:44.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:44.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.326959+0000", 2024-09-20T22:08:44.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003841957 2024-09-20T22:08:44.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:44.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327009+0000", 2024-09-20T22:08:44.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0328999999999999e-05 2024-09-20T22:08:44.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:44.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327107+0000", 2024-09-20T22:08:44.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7472000000000005e-05 2024-09-20T22:08:44.970 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:44.971 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:44.971 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:44.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.971 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47726 7.3 7:c33b26c6:::benchmark_data_smithi060_78800_object47725:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:08:44.971 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.311743+0000", 2024-09-20T22:08:44.971 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 264.62274768200001, 2024-09-20T22:08:44.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0155469880000001, 2024-09-20T22:08:44.971 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:44.971 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:44.971 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:44.971 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:44.971 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:44.971 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:44.972 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47726 2024-09-20T22:08:44.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.972 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:44.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:44.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311743+0000", 2024-09-20T22:08:44.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:44.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:44.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311743+0000", 2024-09-20T22:08:44.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:44.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:44.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311745+0000", 2024-09-20T22:08:44.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.333e-06 2024-09-20T22:08:44.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:44.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311769+0000", 2024-09-20T22:08:44.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3688000000000001e-05 2024-09-20T22:08:44.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:44.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311781+0000", 2024-09-20T22:08:44.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2099999999999999e-05 2024-09-20T22:08:44.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:44.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311788+0000", 2024-09-20T22:08:44.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2269999999999996e-06 2024-09-20T22:08:44.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311860+0000", 2024-09-20T22:08:44.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1823000000000007e-05 2024-09-20T22:08:44.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:44.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311870+0000", 2024-09-20T22:08:44.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0079e-05 2024-09-20T22:08:44.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236177+0000", 2024-09-20T22:08:44.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.92430713399999997 2024-09-20T22:08:44.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:44.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236185+0000", 2024-09-20T22:08:44.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.106e-06 2024-09-20T22:08:44.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319949+0000", 2024-09-20T22:08:44.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083764076000000007 2024-09-20T22:08:44.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:44.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.320088+0000", 2024-09-20T22:08:44.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013926300000000001 2024-09-20T22:08:44.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:08:44.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.320263+0000", 2024-09-20T22:08:44.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000174978 2024-09-20T22:08:44.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:44.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323166+0000", 2024-09-20T22:08:44.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0029027060000000001 2024-09-20T22:08:44.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:44.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323208+0000", 2024-09-20T22:08:44.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2373999999999998e-05 2024-09-20T22:08:44.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:44.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327156+0000", 2024-09-20T22:08:44.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0039471910000000001 2024-09-20T22:08:44.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:44.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327201+0000", 2024-09-20T22:08:44.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5062999999999999e-05 2024-09-20T22:08:44.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:44.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327290+0000", 2024-09-20T22:08:44.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9846999999999995e-05 2024-09-20T22:08:44.978 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:44.978 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:44.978 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:44.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.979 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47771 7.3 7:c7433b13:::benchmark_data_smithi060_78800_object47770:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:08:44.979 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.332925+0000", 2024-09-20T22:08:44.979 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 264.60156663599997, 2024-09-20T22:08:44.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99455883199999995, 2024-09-20T22:08:44.979 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:44.979 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:44.979 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:44.979 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:44.979 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:44.979 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:44.980 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47771 2024-09-20T22:08:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332925+0000", 2024-09-20T22:08:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332925+0000", 2024-09-20T22:08:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:44.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332926+0000", 2024-09-20T22:08:44.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4959999999999999e-06 2024-09-20T22:08:44.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:44.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332954+0000", 2024-09-20T22:08:44.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7495e-05 2024-09-20T22:08:44.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:44.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332967+0000", 2024-09-20T22:08:44.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3005e-05 2024-09-20T22:08:44.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:44.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332974+0000", 2024-09-20T22:08:44.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9719999999999992e-06 2024-09-20T22:08:44.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333056+0000", 2024-09-20T22:08:44.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1390999999999994e-05 2024-09-20T22:08:44.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:44.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333067+0000", 2024-09-20T22:08:44.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0896999999999999e-05 2024-09-20T22:08:44.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236223+0000", 2024-09-20T22:08:44.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90315600299999999 2024-09-20T22:08:44.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:44.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236243+0000", 2024-09-20T22:08:44.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9928999999999998e-05 2024-09-20T22:08:44.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.321565+0000", 2024-09-20T22:08:44.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085322377000000005 2024-09-20T22:08:44.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:44.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.321707+0000", 2024-09-20T22:08:44.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014187100000000001 2024-09-20T22:08:44.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:08:44.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.321808+0000", 2024-09-20T22:08:44.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100536 2024-09-20T22:08:44.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:44.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323255+0000", 2024-09-20T22:08:44.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00144754 2024-09-20T22:08:44.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:44.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.326947+0000", 2024-09-20T22:08:44.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0036917360000000001 2024-09-20T22:08:44.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:44.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327341+0000", 2024-09-20T22:08:44.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00039394499999999998 2024-09-20T22:08:44.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:44.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327391+0000", 2024-09-20T22:08:44.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9836999999999997e-05 2024-09-20T22:08:44.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:44.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327483+0000", 2024-09-20T22:08:44.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2801999999999997e-05 2024-09-20T22:08:44.987 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:44.987 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:44.987 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:44.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.988 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47772 7.3 7:c5f9ddaf:::benchmark_data_smithi060_78800_object47771:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:08:44.988 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.333899+0000", 2024-09-20T22:08:44.988 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 264.600592119, 2024-09-20T22:08:44.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99376266499999999, 2024-09-20T22:08:44.988 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:44.988 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:44.988 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:44.988 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:44.988 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:44.988 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:44.989 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47772 2024-09-20T22:08:44.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.989 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:44.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:44.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333899+0000", 2024-09-20T22:08:44.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:44.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:44.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333899+0000", 2024-09-20T22:08:44.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:44.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:44.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333900+0000", 2024-09-20T22:08:44.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0049999999999999e-06 2024-09-20T22:08:44.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:44.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333927+0000", 2024-09-20T22:08:44.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.675e-05 2024-09-20T22:08:44.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:44.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333942+0000", 2024-09-20T22:08:44.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5678000000000001e-05 2024-09-20T22:08:44.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:44.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333950+0000", 2024-09-20T22:08:44.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7060000000000004e-06 2024-09-20T22:08:44.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.334006+0000", 2024-09-20T22:08:44.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5631000000000003e-05 2024-09-20T22:08:44.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:44.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.334031+0000", 2024-09-20T22:08:44.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5213999999999999e-05 2024-09-20T22:08:44.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236276+0000", 2024-09-20T22:08:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90224500100000005 2024-09-20T22:08:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236285+0000", 2024-09-20T22:08:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7139999999999999e-06 2024-09-20T22:08:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323289+0000", 2024-09-20T22:08:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087004215999999995 2024-09-20T22:08:44.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323404+0000", 2024-09-20T22:08:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011502000000000001 2024-09-20T22:08:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:08:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323545+0000", 2024-09-20T22:08:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014127700000000001 2024-09-20T22:08:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327330+0000", 2024-09-20T22:08:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037843400000000002 2024-09-20T22:08:44.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327491+0000", 2024-09-20T22:08:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016180099999999999 2024-09-20T22:08:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327529+0000", 2024-09-20T22:08:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8081999999999999e-05 2024-09-20T22:08:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327576+0000", 2024-09-20T22:08:44.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6881000000000001e-05 2024-09-20T22:08:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327662+0000", 2024-09-20T22:08:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5348999999999999e-05 2024-09-20T22:08:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47798 7.3 7:cc6426c7:::benchmark_data_smithi060_78800_object47797:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:08:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.346085+0000", 2024-09-20T22:08:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 264.58840572499997, 2024-09-20T22:08:44.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98393741899999998, 2024-09-20T22:08:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47798 2024-09-20T22:08:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346085+0000", 2024-09-20T22:08:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:44.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346085+0000", 2024-09-20T22:08:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346087+0000", 2024-09-20T22:08:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3289999999999999e-06 2024-09-20T22:08:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346109+0000", 2024-09-20T22:08:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2359999999999999e-05 2024-09-20T22:08:44.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346123+0000", 2024-09-20T22:08:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.367e-05 2024-09-20T22:08:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346133+0000", 2024-09-20T22:08:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0519999999999999e-05 2024-09-20T22:08:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346206+0000", 2024-09-20T22:08:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3011999999999998e-05 2024-09-20T22:08:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346218+0000", 2024-09-20T22:08:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2089000000000001e-05 2024-09-20T22:08:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236825+0000", 2024-09-20T22:08:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.89060705600000001 2024-09-20T22:08:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:44.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236839+0000", 2024-09-20T22:08:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3685e-05 2024-09-20T22:08:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.324961+0000", 2024-09-20T22:08:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088121685000000005 2024-09-20T22:08:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.325083+0000", 2024-09-20T22:08:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012199199999999999 2024-09-20T22:08:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:08:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.325216+0000", 2024-09-20T22:08:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000132862 2024-09-20T22:08:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327496+0000", 2024-09-20T22:08:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022805669999999998 2024-09-20T22:08:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327698+0000", 2024-09-20T22:08:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020205099999999999 2024-09-20T22:08:45.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.329834+0000", 2024-09-20T22:08:45.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021359249999999999 2024-09-20T22:08:45.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:45.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.329894+0000", 2024-09-20T22:08:45.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9929000000000002e-05 2024-09-20T22:08:45.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:45.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.330023+0000", 2024-09-20T22:08:45.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000128687 2024-09-20T22:08:45.002 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.003 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:45.003 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.003 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47809 7.3 7:caa74520:::benchmark_data_smithi060_78800_object47808:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:08:45.003 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.351924+0000", 2024-09-20T22:08:45.003 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 264.58256758700003, 2024-09-20T22:08:45.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.982167177, 2024-09-20T22:08:45.003 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:45.003 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:45.003 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:45.003 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:45.003 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:45.003 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:45.003 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47809 2024-09-20T22:08:45.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.004 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:45.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:45.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.351924+0000", 2024-09-20T22:08:45.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:45.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.351924+0000", 2024-09-20T22:08:45.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:45.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.351925+0000", 2024-09-20T22:08:45.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1680000000000001e-06 2024-09-20T22:08:45.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:45.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352100+0000", 2024-09-20T22:08:45.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017479600000000001 2024-09-20T22:08:45.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:45.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352112+0000", 2024-09-20T22:08:45.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.204e-05 2024-09-20T22:08:45.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:45.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352119+0000", 2024-09-20T22:08:45.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8960000000000003e-06 2024-09-20T22:08:45.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352199+0000", 2024-09-20T22:08:45.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9747999999999996e-05 2024-09-20T22:08:45.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352208+0000", 2024-09-20T22:08:45.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2709999999999998e-06 2024-09-20T22:08:45.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236880+0000", 2024-09-20T22:08:45.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.88467109200000005 2024-09-20T22:08:45.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236889+0000", 2024-09-20T22:08:45.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0580000000000004e-06 2024-09-20T22:08:45.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327738+0000", 2024-09-20T22:08:45.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.090849691999999996 2024-09-20T22:08:45.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:45.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327931+0000", 2024-09-20T22:08:45.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019231900000000001 2024-09-20T22:08:45.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:08:45.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.328060+0000", 2024-09-20T22:08:45.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012906699999999999 2024-09-20T22:08:45.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:45.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.331899+0000", 2024-09-20T22:08:45.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038394620000000001 2024-09-20T22:08:45.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.333872+0000", 2024-09-20T22:08:45.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019725540000000001 2024-09-20T22:08:45.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.333936+0000", 2024-09-20T22:08:45.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3849000000000005e-05 2024-09-20T22:08:45.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:45.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.333992+0000", 2024-09-20T22:08:45.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6730000000000001e-05 2024-09-20T22:08:45.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:45.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334091+0000", 2024-09-20T22:08:45.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8435000000000002e-05 2024-09-20T22:08:45.010 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.010 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:45.010 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.010 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47810 7.3 7:c6434cb9:::benchmark_data_smithi060_78800_object47809:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:08:45.010 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.352140+0000", 2024-09-20T22:08:45.010 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 264.58235159200001, 2024-09-20T22:08:45.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.985672357, 2024-09-20T22:08:45.010 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:45.010 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:45.011 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:45.011 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:45.011 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:45.011 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:45.011 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47810 2024-09-20T22:08:45.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.011 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:45.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:45.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352140+0000", 2024-09-20T22:08:45.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:45.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352140+0000", 2024-09-20T22:08:45.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:45.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352140+0000", 2024-09-20T22:08:45.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9299999999999997e-07 2024-09-20T22:08:45.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:45.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352187+0000", 2024-09-20T22:08:45.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6468000000000003e-05 2024-09-20T22:08:45.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352196+0000", 2024-09-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9670000000000007e-06 2024-09-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352202+0000", 2024-09-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0120000000000002e-06 2024-09-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352230+0000", 2024-09-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8289e-05 2024-09-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352236+0000", 2024-09-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.455e-06 2024-09-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236921+0000", 2024-09-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.88468433300000004 2024-09-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236931+0000", 2024-09-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0322e-05 2024-09-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.330063+0000", 2024-09-20T22:08:45.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.093132375000000003 2024-09-20T22:08:45.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:45.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.330188+0000", 2024-09-20T22:08:45.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000124664 2024-09-20T22:08:45.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:08:45.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.330329+0000", 2024-09-20T22:08:45.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014119299999999999 2024-09-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334105+0000", 2024-09-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037757799999999998 2024-09-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334140+0000", 2024-09-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5015000000000003e-05 2024-09-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337652+0000", 2024-09-20T22:08:45.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0035119819999999999 2024-09-20T22:08:45.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:45.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337709+0000", 2024-09-20T22:08:45.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6446000000000001e-05 2024-09-20T22:08:45.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:45.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337812+0000", 2024-09-20T22:08:45.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010336300000000001 2024-09-20T22:08:45.017 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.017 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:45.017 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.018 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47831 7.3 7:cf7c6d80:::benchmark_data_smithi060_78800_object47830:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:08:45.018 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.435232+0000", 2024-09-20T22:08:45.018 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 264.499259, 2024-09-20T22:08:45.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90280988699999998, 2024-09-20T22:08:45.018 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:45.018 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:45.018 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:45.018 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:45.018 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:45.018 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:45.018 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47831 2024-09-20T22:08:45.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.018 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435232+0000", 2024-09-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435232+0000", 2024-09-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435233+0000", 2024-09-20T22:08:45.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0920000000000001e-06 2024-09-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435256+0000", 2024-09-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2840000000000002e-05 2024-09-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435270+0000", 2024-09-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3944000000000001e-05 2024-09-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:45.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:45.021 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:45.021 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:45.022 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:44 smithi067 ceph-mon[33511]: osdmap e198: 8 total, 8 up, 8 in 2024-09-20T22:08:45.022 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:44 smithi067 ceph-mon[33511]: pgmap v290: 158 pgs: 10 unknown, 148 active+clean; 4.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 920 B/s rd, 66 MiB/s wr, 1.06k op/s 2024-09-20T22:08:45.022 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:44 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]: dispatch 2024-09-20T22:08:45.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435277+0000", 2024-09-20T22:08:45.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1420000000000004e-06 2024-09-20T22:08:45.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435341+0000", 2024-09-20T22:08:45.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3431999999999998e-05 2024-09-20T22:08:45.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435350+0000", 2024-09-20T22:08:45.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2920000000000001e-06 2024-09-20T22:08:45.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236961+0000", 2024-09-20T22:08:45.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.80161062500000002 2024-09-20T22:08:45.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236968+0000", 2024-09-20T22:08:45.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8529999999999992e-06 2024-09-20T22:08:45.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.331930+0000", 2024-09-20T22:08:45.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.094961952000000002 2024-09-20T22:08:45.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:45.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.332051+0000", 2024-09-20T22:08:45.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000120683 2024-09-20T22:08:45.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:08:45.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.332208+0000", 2024-09-20T22:08:45.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015661399999999999 2024-09-20T22:08:45.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334177+0000", 2024-09-20T22:08:45.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019690350000000001 2024-09-20T22:08:45.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:45.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334187+0000", 2024-09-20T22:08:45.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8870000000000004e-06 2024-09-20T22:08:45.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337890+0000", 2024-09-20T22:08:45.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037037379999999998 2024-09-20T22:08:45.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:45.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337940+0000", 2024-09-20T22:08:45.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0144e-05 2024-09-20T22:08:45.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:45.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.338042+0000", 2024-09-20T22:08:45.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010161400000000001 2024-09-20T22:08:45.027 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.027 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:45.027 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.027 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47833 7.3 7:c90762ed:::benchmark_data_smithi060_78800_object47832:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:08:45.027 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.437390+0000", 2024-09-20T22:08:45.027 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 264.49710104000002, 2024-09-20T22:08:45.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90566071400000003, 2024-09-20T22:08:45.027 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:45.027 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:45.028 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:45.028 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:45.028 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:45.028 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:45.028 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47833 2024-09-20T22:08:45.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.028 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:45.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:45.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437390+0000", 2024-09-20T22:08:45.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:45.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437390+0000", 2024-09-20T22:08:45.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:45.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437391+0000", 2024-09-20T22:08:45.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.085e-06 2024-09-20T22:08:45.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:45.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437415+0000", 2024-09-20T22:08:45.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3479e-05 2024-09-20T22:08:45.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:45.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437430+0000", 2024-09-20T22:08:45.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5485000000000002e-05 2024-09-20T22:08:45.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:45.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437439+0000", 2024-09-20T22:08:45.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3520000000000007e-06 2024-09-20T22:08:45.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437507+0000", 2024-09-20T22:08:45.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8011999999999999e-05 2024-09-20T22:08:45.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437520+0000", 2024-09-20T22:08:45.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3022e-05 2024-09-20T22:08:45.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237002+0000", 2024-09-20T22:08:45.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.79948219600000003 2024-09-20T22:08:45.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237011+0000", 2024-09-20T22:08:45.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8049999999999996e-06 2024-09-20T22:08:45.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334208+0000", 2024-09-20T22:08:45.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.097197622999999997 2024-09-20T22:08:45.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:45.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334306+0000", 2024-09-20T22:08:45.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8068999999999999e-05 2024-09-20T22:08:45.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:08:45.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334475+0000", 2024-09-20T22:08:45.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016830799999999999 2024-09-20T22:08:45.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337852+0000", 2024-09-20T22:08:45.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00337747 2024-09-20T22:08:45.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:45.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.340086+0000", 2024-09-20T22:08:45.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002234036 2024-09-20T22:08:45.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.342602+0000", 2024-09-20T22:08:45.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025159929999999998 2024-09-20T22:08:45.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.342808+0000", 2024-09-20T22:08:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020551099999999999 2024-09-20T22:08:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343051+0000", 2024-09-20T22:08:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00024326799999999999 2024-09-20T22:08:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.034 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47842 7.3 7:c97c3aed:::benchmark_data_smithi060_78800_object47841:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:08:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.442613+0000", 2024-09-20T22:08:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 264.49187843300001, 2024-09-20T22:08:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90079891400000001, 2024-09-20T22:08:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47842 2024-09-20T22:08:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.035 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442613+0000", 2024-09-20T22:08:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442613+0000", 2024-09-20T22:08:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442614+0000", 2024-09-20T22:08:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3e-06 2024-09-20T22:08:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442640+0000", 2024-09-20T22:08:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5544000000000001e-05 2024-09-20T22:08:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442651+0000", 2024-09-20T22:08:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1218e-05 2024-09-20T22:08:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442658+0000", 2024-09-20T22:08:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1980000000000004e-06 2024-09-20T22:08:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442734+0000", 2024-09-20T22:08:45.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6279999999999995e-05 2024-09-20T22:08:45.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442747+0000", 2024-09-20T22:08:45.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2393e-05 2024-09-20T22:08:45.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237041+0000", 2024-09-20T22:08:45.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.79429414499999995 2024-09-20T22:08:45.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237049+0000", 2024-09-20T22:08:45.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8150000000000006e-06 2024-09-20T22:08:45.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.335958+0000", 2024-09-20T22:08:45.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.098909056999999995 2024-09-20T22:08:45.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:45.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.336074+0000", 2024-09-20T22:08:45.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011582900000000001 2024-09-20T22:08:45.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:08:45.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.336216+0000", 2024-09-20T22:08:45.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014236899999999999 2024-09-20T22:08:45.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:45.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.342529+0000", 2024-09-20T22:08:45.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.006312799 2024-09-20T22:08:45.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343211+0000", 2024-09-20T22:08:45.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00068264900000000002 2024-09-20T22:08:45.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343252+0000", 2024-09-20T22:08:45.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1084000000000002e-05 2024-09-20T22:08:45.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:45.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343315+0000", 2024-09-20T22:08:45.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2989000000000003e-05 2024-09-20T22:08:45.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:45.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343412+0000", 2024-09-20T22:08:45.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6244999999999998e-05 2024-09-20T22:08:45.042 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.042 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:45.042 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.042 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47846 7.3 7:c61861ce:::benchmark_data_smithi060_78800_object47845:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:08:45.042 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.445377+0000", 2024-09-20T22:08:45.042 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 264.48911456399998, 2024-09-20T22:08:45.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91076886199999996, 2024-09-20T22:08:45.042 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:45.042 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:45.042 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:45.042 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:45.042 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:45.043 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:45.043 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47846 2024-09-20T22:08:45.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.043 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:45.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:45.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445377+0000", 2024-09-20T22:08:45.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:45.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445377+0000", 2024-09-20T22:08:45.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:45.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445378+0000", 2024-09-20T22:08:45.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.02e-06 2024-09-20T22:08:45.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:45.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445407+0000", 2024-09-20T22:08:45.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9654000000000001e-05 2024-09-20T22:08:45.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:45.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445423+0000", 2024-09-20T22:08:45.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6107999999999999e-05 2024-09-20T22:08:45.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:45.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445432+0000", 2024-09-20T22:08:45.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6770000000000001e-06 2024-09-20T22:08:45.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445522+0000", 2024-09-20T22:08:45.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0211999999999996e-05 2024-09-20T22:08:45.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445535+0000", 2024-09-20T22:08:45.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2279000000000001e-05 2024-09-20T22:08:45.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237076+0000", 2024-09-20T22:08:45.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.79154110300000002 2024-09-20T22:08:45.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237085+0000", 2024-09-20T22:08:45.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.611e-06 2024-09-20T22:08:45.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.338112+0000", 2024-09-20T22:08:45.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10102702700000001 2024-09-20T22:08:45.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:45.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.338234+0000", 2024-09-20T22:08:45.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121853 2024-09-20T22:08:45.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:08:45.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.338375+0000", 2024-09-20T22:08:45.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140428 2024-09-20T22:08:45.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343463+0000", 2024-09-20T22:08:45.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0050883710000000004 2024-09-20T22:08:45.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343504+0000", 2024-09-20T22:08:45.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1185e-05 2024-09-20T22:08:45.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:45.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.355992+0000", 2024-09-20T22:08:45.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.012487992 2024-09-20T22:08:45.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:45.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356046+0000", 2024-09-20T22:08:45.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4033000000000003e-05 2024-09-20T22:08:45.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:45.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356145+0000", 2024-09-20T22:08:45.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9308999999999994e-05 2024-09-20T22:08:45.049 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.049 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:45.049 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.049 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47847 7.3 7:c0c5cc37:::benchmark_data_smithi060_78800_object47846:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:08:45.049 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.445583+0000", 2024-09-20T22:08:45.049 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 264.48890828499998, 2024-09-20T22:08:45.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91077994900000003, 2024-09-20T22:08:45.049 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:45.049 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:45.050 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:45.050 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:45.050 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:45.050 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:45.050 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47847 2024-09-20T22:08:45.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.050 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:45.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:45.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445583+0000", 2024-09-20T22:08:45.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:45.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445583+0000", 2024-09-20T22:08:45.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:45.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445584+0000", 2024-09-20T22:08:45.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4900000000000005e-07 2024-09-20T22:08:45.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:45.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445607+0000", 2024-09-20T22:08:45.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3153e-05 2024-09-20T22:08:45.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:45.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445621+0000", 2024-09-20T22:08:45.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3638000000000001e-05 2024-09-20T22:08:45.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:45.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445628+0000", 2024-09-20T22:08:45.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.729e-06 2024-09-20T22:08:45.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445676+0000", 2024-09-20T22:08:45.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7911000000000003e-05 2024-09-20T22:08:45.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445685+0000", 2024-09-20T22:08:45.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8289999999999997e-06 2024-09-20T22:08:45.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237113+0000", 2024-09-20T22:08:45.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.79142811599999996 2024-09-20T22:08:45.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237121+0000", 2024-09-20T22:08:45.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8310000000000001e-06 2024-09-20T22:08:45.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.340131+0000", 2024-09-20T22:08:45.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103009795 2024-09-20T22:08:45.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:45.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.340324+0000", 2024-09-20T22:08:45.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000193034 2024-09-20T22:08:45.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:08:45.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.340466+0000", 2024-09-20T22:08:45.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000142032 2024-09-20T22:08:45.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343542+0000", 2024-09-20T22:08:45.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0030759860000000002 2024-09-20T22:08:45.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343584+0000", 2024-09-20T22:08:45.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1990999999999998e-05 2024-09-20T22:08:45.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:45.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356203+0000", 2024-09-20T22:08:45.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.012619361000000001 2024-09-20T22:08:45.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:45.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356268+0000", 2024-09-20T22:08:45.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4596000000000003e-05 2024-09-20T22:08:45.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:45.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356363+0000", 2024-09-20T22:08:45.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5098000000000003e-05 2024-09-20T22:08:45.056 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.056 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:45.056 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.057 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47853 7.3 7:c21485b4:::benchmark_data_smithi060_78800_object47852:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:08:45.057 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.457698+0000", 2024-09-20T22:08:45.057 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 264.476792988, 2024-09-20T22:08:45.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.89889923500000002, 2024-09-20T22:08:45.057 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:45.057 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:45.057 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:45.057 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:45.057 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:45.057 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:45.057 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47853 2024-09-20T22:08:45.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.057 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:45.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:45.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457698+0000", 2024-09-20T22:08:45.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:45.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457698+0000", 2024-09-20T22:08:45.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:45.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457699+0000", 2024-09-20T22:08:45.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.159e-06 2024-09-20T22:08:45.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:45.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457753+0000", 2024-09-20T22:08:45.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4067999999999998e-05 2024-09-20T22:08:45.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:45.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457765+0000", 2024-09-20T22:08:45.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1430000000000001e-05 2024-09-20T22:08:45.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:45.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457770+0000", 2024-09-20T22:08:45.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4410000000000001e-06 2024-09-20T22:08:45.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457821+0000", 2024-09-20T22:08:45.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1156999999999998e-05 2024-09-20T22:08:45.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457829+0000", 2024-09-20T22:08:45.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5259999999999996e-06 2024-09-20T22:08:45.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237712+0000", 2024-09-20T22:08:45.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.77988319699999997 2024-09-20T22:08:45.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237742+0000", 2024-09-20T22:08:45.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0224000000000001e-05 2024-09-20T22:08:45.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343621+0000", 2024-09-20T22:08:45.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10587846400000001 2024-09-20T22:08:45.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:45.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343759+0000", 2024-09-20T22:08:45.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000137947 2024-09-20T22:08:45.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:08:45.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343960+0000", 2024-09-20T22:08:45.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000201201 2024-09-20T22:08:45.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.349430+0000", 2024-09-20T22:08:45.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0054698710000000003 2024-09-20T22:08:45.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.349473+0000", 2024-09-20T22:08:45.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2865999999999999e-05 2024-09-20T22:08:45.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:45.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356374+0000", 2024-09-20T22:08:45.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0069012359999999998 2024-09-20T22:08:45.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:45.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356434+0000", 2024-09-20T22:08:45.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9669000000000001e-05 2024-09-20T22:08:45.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:45.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356597+0000", 2024-09-20T22:08:45.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016377899999999999 2024-09-20T22:08:45.064 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.064 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:45.064 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.065 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169951 7.3 7:c27f31c3:::benchmark_data_smithi060_78800_object36918:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:08:45.065 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440033+0000", 2024-09-20T22:08:45.065 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 166.494457797, 2024-09-20T22:08:45.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0308894289999999, 2024-09-20T22:08:45.065 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:45.065 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:45.065 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:08:45.065 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:45.065 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:45.065 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:45.065 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169951 2024-09-20T22:08:45.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.065 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:45.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:45.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440033+0000", 2024-09-20T22:08:45.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:45.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440033+0000", 2024-09-20T22:08:45.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:45.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440035+0000", 2024-09-20T22:08:45.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3170000000000001e-06 2024-09-20T22:08:45.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:45.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440041+0000", 2024-09-20T22:08:45.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8869999999999997e-06 2024-09-20T22:08:45.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:45.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440053+0000", 2024-09-20T22:08:45.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2146e-05 2024-09-20T22:08:45.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:45.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440061+0000", 2024-09-20T22:08:45.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3049999999999993e-06 2024-09-20T22:08:45.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440616+0000", 2024-09-20T22:08:45.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00055526700000000002 2024-09-20T22:08:45.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op must wait for map", 2024-09-20T22:08:45.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440623+0000", 2024-09-20T22:08:45.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9290000000000001e-06 2024-09-20T22:08:45.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.659494+0000", 2024-09-20T22:08:45.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.21887063800000001 2024-09-20T22:08:45.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.659506+0000", 2024-09-20T22:08:45.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2313000000000001e-05 2024-09-20T22:08:45.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.470588+0000", 2024-09-20T22:08:45.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.81108138900000004 2024-09-20T22:08:45.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:45.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.470923+0000", 2024-09-20T22:08:45.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033523800000000002 2024-09-20T22:08:45.070 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.070 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:45.070 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.070 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169962 7.8 7:1412370b:::benchmark_data_smithi060_78800_object36929:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:08:45.070 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440138+0000", 2024-09-20T22:08:45.070 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 166.49435276700001, 2024-09-20T22:08:45.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0713576439999999, 2024-09-20T22:08:45.070 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:45.070 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:45.070 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:08:45.071 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:45.071 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:45.071 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:45.071 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169962 2024-09-20T22:08:45.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.071 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:45.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:45.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440138+0000", 2024-09-20T22:08:45.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:45.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440138+0000", 2024-09-20T22:08:45.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440139+0000", 2024-09-20T22:08:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1399999999999996e-07 2024-09-20T22:08:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440143+0000", 2024-09-20T22:08:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8029999999999998e-06 2024-09-20T22:08:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:45.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440152+0000", 2024-09-20T22:08:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-20T22:08:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440158+0000", 2024-09-20T22:08:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3319999999999997e-06 2024-09-20T22:08:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440263+0000", 2024-09-20T22:08:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105258 2024-09-20T22:08:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op must wait for map", 2024-09-20T22:08:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440269+0000", 2024-09-20T22:08:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8549999999999998e-06 2024-09-20T22:08:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.647667+0000", 2024-09-20T22:08:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.207398358 2024-09-20T22:08:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.647675+0000", 2024-09-20T22:08:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5979999999999999e-06 2024-09-20T22:08:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.490134+0000", 2024-09-20T22:08:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.84245882699999997 2024-09-20T22:08:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:08:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.490253+0000", 2024-09-20T22:08:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011938999999999999 2024-09-20T22:08:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.511425+0000", 2024-09-20T22:08:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021171453999999999 2024-09-20T22:08:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.511496+0000", 2024-09-20T22:08:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1359000000000006e-05 2024-09-20T22:08:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169969 7.18 7:18661b50:::benchmark_data_smithi060_78800_object36936:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:08:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440181+0000", 2024-09-20T22:08:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 166.49431034899999, 2024-09-20T22:08:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.074892379, 2024-09-20T22:08:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:45.076 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:08:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169969 2024-09-20T22:08:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440181+0000", 2024-09-20T22:08:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:45.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440181+0000", 2024-09-20T22:08:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440181+0000", 2024-09-20T22:08:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7100000000000002e-07 2024-09-20T22:08:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440185+0000", 2024-09-20T22:08:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3840000000000001e-06 2024-09-20T22:08:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:45.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440193+0000", 2024-09-20T22:08:45.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5790000000000004e-06 2024-09-20T22:08:45.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:45.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440200+0000", 2024-09-20T22:08:45.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2310000000000001e-06 2024-09-20T22:08:45.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.647597+0000", 2024-09-20T22:08:45.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.20739776200000001 2024-09-20T22:08:45.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.647606+0000", 2024-09-20T22:08:45.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6580000000000008e-06 2024-09-20T22:08:45.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.510928+0000", 2024-09-20T22:08:45.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.86332191400000002 2024-09-20T22:08:45.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:08:45.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.511051+0000", 2024-09-20T22:08:45.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012358099999999999 2024-09-20T22:08:45.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.514989+0000", 2024-09-20T22:08:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003937744 2024-09-20T22:08:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.515073+0000", 2024-09-20T22:08:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3955000000000005e-05 2024-09-20T22:08:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.081 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227626 7.3 7:c62c9d8d:::benchmark_data_smithi060_78800_object94593:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:08:45.082 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.821701+0000", 2024-09-20T22:08:45.082 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 111.11279054000001, 2024-09-20T22:08:45.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1262373590000001, 2024-09-20T22:08:45.082 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:45.082 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:45.082 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:45.082 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:45.082 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:45.082 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:45.082 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227626 2024-09-20T22:08:45.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.082 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:45.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821701+0000", 2024-09-20T22:08:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821701+0000", 2024-09-20T22:08:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821702+0000", 2024-09-20T22:08:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.102e-06 2024-09-20T22:08:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:45.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821707+0000", 2024-09-20T22:08:45.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0579999999999997e-06 2024-09-20T22:08:45.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:45.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821733+0000", 2024-09-20T22:08:45.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6642999999999999e-05 2024-09-20T22:08:45.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:45.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821741+0000", 2024-09-20T22:08:45.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7009999999999996e-06 2024-09-20T22:08:45.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821806+0000", 2024-09-20T22:08:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4882000000000007e-05 2024-09-20T22:08:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821817+0000", 2024-09-20T22:08:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0852999999999999e-05 2024-09-20T22:08:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.941998+0000", 2024-09-20T22:08:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1201808070000001 2024-09-20T22:08:45.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:45.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.942120+0000", 2024-09-20T22:08:45.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012227900000000001 2024-09-20T22:08:45.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,3,4", 2024-09-20T22:08:45.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.942615+0000", 2024-09-20T22:08:45.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00049518700000000001 2024-09-20T22:08:45.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:45.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946370+0000", 2024-09-20T22:08:45.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003755141 2024-09-20T22:08:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946870+0000", 2024-09-20T22:08:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00049939899999999998 2024-09-20T22:08:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947042+0000", 2024-09-20T22:08:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000172199 2024-09-20T22:08:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947739+0000", 2024-09-20T22:08:45.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00069714899999999999 2024-09-20T22:08:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947809+0000", 2024-09-20T22:08:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9530999999999995e-05 2024-09-20T22:08:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947938+0000", 2024-09-20T22:08:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012942800000000001 2024-09-20T22:08:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:45.088 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:45.089 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:08:45.122 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:45.143 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 48 16 81897 81881 106.589 76.625 0.0150035 0.0093773 2024-09-20T22:08:45.157 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:45.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:45.158 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:45.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:45.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:08:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:08:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:08:45.296 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:45.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:45.297 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:45.398 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:45.421 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:08:45.428 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:45.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:45.429 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:45.529 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:45.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:45.623 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:45.624 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:45.724 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:45.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:45.771 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:45.771 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:45.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:45 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-20T22:08:45.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:45 smithi060 ceph-mon[29694]: osdmap e199: 8 total, 8 up, 8 in 2024-09-20T22:08:45.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:45 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-20T22:08:45.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:45 smithi060 ceph-mon[40290]: osdmap e199: 8 total, 8 up, 8 in 2024-09-20T22:08:45.872 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:45.914 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:45.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:45.915 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:45.994 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:08:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:45 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "11"}]': finished 2024-09-20T22:08:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:45 smithi067 ceph-mon[33511]: osdmap e199: 8 total, 8 up, 8 in 2024-09-20T22:08:46.016 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:46.052 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:46.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:46.053 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:46.144 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 49 16 83088 83072 105.933 74.4375 0.0116949 0.00943483 2024-09-20T22:08:46.154 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:46.181 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:46.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:46.182 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:46.275 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T22:08:46.282 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:46.392 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:46.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:46.393 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:46.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:46.503 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:46.537 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:46.564 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:46.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:46.566 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:46.651 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:46 smithi060 ceph-mon[40290]: osdmap e200: 8 total, 8 up, 8 in 2024-09-20T22:08:46.651 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:46 smithi060 ceph-mon[40290]: pgmap v293: 158 pgs: 4 remapped+peering, 6 unknown, 148 active+clean; 4.9 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 58 MiB/s wr, 905 op/s 2024-09-20T22:08:46.651 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:46 smithi060 ceph-mon[40290]: osdmap e201: 8 total, 8 up, 8 in 2024-09-20T22:08:46.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:46 smithi060 ceph-mon[29694]: osdmap e200: 8 total, 8 up, 8 in 2024-09-20T22:08:46.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:46 smithi060 ceph-mon[29694]: pgmap v293: 158 pgs: 4 remapped+peering, 6 unknown, 148 active+clean; 4.9 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 58 MiB/s wr, 905 op/s 2024-09-20T22:08:46.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:46 smithi060 ceph-mon[29694]: osdmap e201: 8 total, 8 up, 8 in 2024-09-20T22:08:46.667 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:46.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:46.703 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:46.703 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:46.804 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:46.834 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:46.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:46.835 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:46.935 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:46.968 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:46.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:46.969 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:46 smithi067 ceph-mon[33511]: osdmap e200: 8 total, 8 up, 8 in 2024-09-20T22:08:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:46 smithi067 ceph-mon[33511]: pgmap v293: 158 pgs: 4 remapped+peering, 6 unknown, 148 active+clean; 4.9 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 58 MiB/s wr, 905 op/s 2024-09-20T22:08:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:46 smithi067 ceph-mon[33511]: osdmap e201: 8 total, 8 up, 8 in 2024-09-20T22:08:47.070 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:47.104 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:47.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:47.105 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:47.143 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 50 16 84169 84153 105.166 67.5625 0.0122241 0.00950414 2024-09-20T22:08:47.205 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:47.238 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:47.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:47.239 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:47.293 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:08:47.293 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:08:47.336 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:08:47.336 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:08:47.340 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:47.369 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:47.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:47.370 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:47.380 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:08:47.380 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:08:47.422 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:08:47.422 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:08:47.422 INFO:tasks.thrashosds.thrasher:Reweighting osd 7 to 0.7327013547825585 2024-09-20T22:08:47.422 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 7 0.7327013547825585 2024-09-20T22:08:47.423 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:08:47.423 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:08:47.458 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:08:47.458 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:08:47.471 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:47.491 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:08:47.492 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:08:47.499 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:47.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:47.500 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:47.525 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:08:47.525 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:08:47.565 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:08:47.565 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:08:47.601 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:47.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:47.644 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:47.644 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:47.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:47.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:47 smithi060 ceph-mon[40290]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-20T22:08:47.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:47 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3319579936' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:47.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:47 smithi060 ceph-mon[40290]: osdmap e202: 8 total, 8 up, 8 in 2024-09-20T22:08:47.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:47 smithi060 ceph-mon[29694]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-20T22:08:47.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:47 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3319579936' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:47.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:47 smithi060 ceph-mon[29694]: osdmap e202: 8 total, 8 up, 8 in 2024-09-20T22:08:47.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:47.804 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:47.805 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:47.905 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:47.935 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:47.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:47.936 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:47 smithi067 ceph-mon[33511]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-20T22:08:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:47 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3319579936' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:47 smithi067 ceph-mon[33511]: osdmap e202: 8 total, 8 up, 8 in 2024-09-20T22:08:48.037 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:48.084 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:48.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:48.085 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:48.144 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 51 16 85198 85182 104.365 64.3125 0.00713645 0.00957684 2024-09-20T22:08:48.186 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:48.223 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:48.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:48.224 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:48.324 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:48.359 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:48.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:48.361 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:48.434 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:08:48.462 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:48.509 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:48.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:48.510 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:48.611 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:48.645 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:48.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:48.646 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:48.731 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:48.747 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:48.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:48 smithi060 ceph-mon[40290]: pgmap v296: 158 pgs: 1 peering, 9 remapped+peering, 148 active+clean; 5.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 95 MiB/s wr, 1.51k op/s 2024-09-20T22:08:48.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:48 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2375719448' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:08:48.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:48 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:08:48.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:48 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:08:48.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:48 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:08:48.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:48 smithi060 ceph-mon[40290]: osdmap e203: 8 total, 8 up, 8 in 2024-09-20T22:08:48.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:48 smithi060 ceph-mon[29694]: pgmap v296: 158 pgs: 1 peering, 9 remapped+peering, 148 active+clean; 5.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 95 MiB/s wr, 1.51k op/s 2024-09-20T22:08:48.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:48 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2375719448' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:08:48.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:48 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:08:48.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:48 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:08:48.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:48 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:08:48.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:48 smithi060 ceph-mon[29694]: osdmap e203: 8 total, 8 up, 8 in 2024-09-20T22:08:48.794 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:48.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:48.795 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:48.895 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:48.929 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:48.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:48.930 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:49.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:48 smithi067 ceph-mon[33511]: pgmap v296: 158 pgs: 1 peering, 9 remapped+peering, 148 active+clean; 5.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 95 MiB/s wr, 1.51k op/s 2024-09-20T22:08:49.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:48 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2375719448' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:08:49.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:48 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:08:49.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:48 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:08:49.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:48 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:08:49.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:48 smithi067 ceph-mon[33511]: osdmap e203: 8 total, 8 up, 8 in 2024-09-20T22:08:49.030 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:49.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:49.064 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:49.064 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:49.144 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 52 16 86456 86440 103.869 78.625 0.00796723 0.00962222 2024-09-20T22:08:49.165 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:49.198 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:49.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:49.199 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:49.236 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:49.300 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:49.346 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:49.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:49.348 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:49.448 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:49.501 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:49.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:49.502 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:49.602 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:49.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:49.644 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:49.644 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:49.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:49.789 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:49.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:49.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:49.890 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:49.928 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:49.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:49.929 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:50.003 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:49 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:08:50.003 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:49 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:08:50.030 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:50.065 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:50.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:50.066 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:50.074 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:08:50.074 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:08:50.144 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 53 16 87601 87585 103.26 71.5625 0.0170484 0.00967825 2024-09-20T22:08:50.166 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:50.211 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:50.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:50.212 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:50.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:49 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:08:50.312 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:50.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:50.360 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:50.360 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:50.461 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:50.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:08:50.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:50.496 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:50.496 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:50.597 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:50.628 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:50.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:50.629 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:50.729 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:50.784 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:50.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:50.785 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:50.886 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:50.922 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:50.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:50.923 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:51.024 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:51.054 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:51.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:51.055 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:51.144 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 54 16 88613 88597 102.519 63.25 0.00786309 0.00974963 2024-09-20T22:08:51.156 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:51.197 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:51.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:51.198 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:51.233 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:08:51.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:51 smithi067 ceph-mon[33511]: pgmap v298: 158 pgs: 1 peering, 9 remapped+peering, 148 active+clean; 5.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 85 MiB/s wr, 1.35k op/s 2024-09-20T22:08:51.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:51.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:08:51.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:51.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:51.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:51.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:51.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:51 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/660238569' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 0.7327013547825585}]: dispatch 2024-09-20T22:08:51.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:51 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 0.7327013547825585}]: dispatch 2024-09-20T22:08:51.299 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:51.323 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:51.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:51.324 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:51.351 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:51 smithi060 ceph-mon[29694]: pgmap v298: 158 pgs: 1 peering, 9 remapped+peering, 148 active+clean; 5.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 85 MiB/s wr, 1.35k op/s 2024-09-20T22:08:51.351 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:51.351 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:08:51.351 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:51.351 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:51.351 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:51.351 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:51.351 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:51 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/660238569' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 0.7327013547825585}]: dispatch 2024-09-20T22:08:51.352 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:51 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 0.7327013547825585}]: dispatch 2024-09-20T22:08:51.352 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:51 smithi060 ceph-mon[40290]: pgmap v298: 158 pgs: 1 peering, 9 remapped+peering, 148 active+clean; 5.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 85 MiB/s wr, 1.35k op/s 2024-09-20T22:08:51.352 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:51.352 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:08:51.352 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:51.352 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:51.352 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:51.352 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:51.352 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:51 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/660238569' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 0.7327013547825585}]: dispatch 2024-09-20T22:08:51.352 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:51 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 0.7327013547825585}]: dispatch 2024-09-20T22:08:51.425 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:51.452 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:51.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:51.453 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:51.553 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:51.588 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:51.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:51.589 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:51.689 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:51.711 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:51.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:51.712 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:51.813 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:51.835 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:51.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:51.836 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:51.937 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:51.959 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:51.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:51.960 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:52.061 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:52.083 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:52.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:52.084 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:52.144 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 55 16 89027 89011 101.126 25.875 0.00340869 0.00977391 2024-09-20T22:08:52.185 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:52.211 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:52.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:52.212 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:52.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:52.336 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:52.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:52.337 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:52.337 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:08:52 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:08:52.438 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:52.462 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:52.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:52.463 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:52.564 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:52.587 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:52.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:52.588 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:52.689 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:52.714 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:52.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:52.715 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:52 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 0.7327013547825585}]': finished 2024-09-20T22:08:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:52 smithi067 ceph-mon[33511]: osdmap e204: 8 total, 8 up, 8 in 2024-09-20T22:08:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:52 smithi067 ceph-mon[33511]: osdmap e205: 8 total, 8 up, 8 in 2024-09-20T22:08:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:52 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 0.7327013547825585}]': finished 2024-09-20T22:08:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:52 smithi060 ceph-mon[29694]: osdmap e204: 8 total, 8 up, 8 in 2024-09-20T22:08:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:52 smithi060 ceph-mon[29694]: osdmap e205: 8 total, 8 up, 8 in 2024-09-20T22:08:52.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:52 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 0.7327013547825585}]': finished 2024-09-20T22:08:52.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:52 smithi060 ceph-mon[40290]: osdmap e204: 8 total, 8 up, 8 in 2024-09-20T22:08:52.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:52 smithi060 ceph-mon[40290]: osdmap e205: 8 total, 8 up, 8 in 2024-09-20T22:08:52.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:52.843 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:52.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:52.844 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:52.944 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:52.971 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:52.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:52.972 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:53.073 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:53.099 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:53.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:53.100 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:53.144 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 56 16 89027 89011 99.3198 0 - 0.00977391 2024-09-20T22:08:53.184 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T22:08:53.200 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:53.227 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:53.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:53.228 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:53.235 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.7 to 0.732701 (bb92) 2024-09-20T22:08:53.329 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:53.351 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:53.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:53.352 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:53.453 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:53.476 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:53.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:53.476 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:53.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:53 smithi060 ceph-mon[29694]: pgmap v301: 158 pgs: 9 peering, 149 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 667 B/s rd, 76 MiB/s wr, 1.22k op/s; 125 KiB/s, 0 objects/s recovering 2024-09-20T22:08:53.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:53 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:08:53.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:53 smithi060 ceph-mon[29694]: Health check update: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-20T22:08:53.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:53 smithi060 ceph-mon[29694]: osdmap e206: 8 total, 8 up, 8 in 2024-09-20T22:08:53.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:53 smithi060 ceph-mon[40290]: pgmap v301: 158 pgs: 9 peering, 149 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 667 B/s rd, 76 MiB/s wr, 1.22k op/s; 125 KiB/s, 0 objects/s recovering 2024-09-20T22:08:53.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:53 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:08:53.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:53 smithi060 ceph-mon[40290]: Health check update: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-20T22:08:53.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:53 smithi060 ceph-mon[40290]: osdmap e206: 8 total, 8 up, 8 in 2024-09-20T22:08:53.578 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:53.605 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:53.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:53.606 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:53.706 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:53.729 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:53.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:53.730 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:53 smithi067 ceph-mon[33511]: pgmap v301: 158 pgs: 9 peering, 149 active+clean; 5.3 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 667 B/s rd, 76 MiB/s wr, 1.22k op/s; 125 KiB/s, 0 objects/s recovering 2024-09-20T22:08:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:53 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:08:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:53 smithi067 ceph-mon[33511]: Health check update: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-09-20T22:08:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:53 smithi067 ceph-mon[33511]: osdmap e206: 8 total, 8 up, 8 in 2024-09-20T22:08:53.831 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:53.862 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:53.863 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:53.863 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:53.964 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:53.976 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:53.989 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:53.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:53.991 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:54.091 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:54.091 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:54.098 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:54.146 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 57 16 89411 89395 97.9962 12 0.0117228 0.0101998 2024-09-20T22:08:54.156 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:54.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:54.157 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:54.257 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:54.319 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:54.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:54.321 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:54.421 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:54.449 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:54.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:54.450 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:54.550 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:54.578 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:54.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:54.580 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:54.580 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:54 smithi060 ceph-mon[29694]: osdmap e207: 8 total, 8 up, 8 in 2024-09-20T22:08:54.580 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:54 smithi060 ceph-mon[29694]: 8.c starting backfill to osd.5 from (0'0,0'0] MAX to 203'5495 2024-09-20T22:08:54.580 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:54 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:08:54.580 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:54 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:08:54.580 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:54 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:54.581 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:54 smithi060 ceph-mon[40290]: osdmap e207: 8 total, 8 up, 8 in 2024-09-20T22:08:54.581 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:54 smithi060 ceph-mon[40290]: 8.c starting backfill to osd.5 from (0'0,0'0] MAX to 203'5495 2024-09-20T22:08:54.581 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:54 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:08:54.581 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:54 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:08:54.581 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:54 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:54.681 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:54.710 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:54.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:54.711 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:54 smithi067 ceph-mon[33511]: osdmap e207: 8 total, 8 up, 8 in 2024-09-20T22:08:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:54 smithi067 ceph-mon[33511]: 8.c starting backfill to osd.5 from (0'0,0'0] MAX to 203'5495 2024-09-20T22:08:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:54 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:08:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:54 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:08:54.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:54 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:08:54.812 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:54.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:54.861 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:54.962 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:54.996 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:54.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:54.997 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:55.098 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:55.118 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:08:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:08:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:08:55.145 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:55.145 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 58 16 90583 90567 97.5695 73.25 0.00848718 0.010244 2024-09-20T22:08:55.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:55.146 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:55.247 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:55.280 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:55.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:55.281 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:55.381 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:55.411 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:08:55.411 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:08:55.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:55.412 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:55.413 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:55.513 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:55.519 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:55 smithi060 ceph-mon[29694]: pgmap v304: 158 pgs: 8 peering, 150 active+clean; 5.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 767 B/s rd, 81 MiB/s wr, 1.30k op/s; 144 KiB/s, 0 objects/s recovering 2024-09-20T22:08:55.520 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:55 smithi060 ceph-mon[29694]: osdmap e208: 8 total, 8 up, 8 in 2024-09-20T22:08:55.520 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:55 smithi060 ceph-mon[29694]: osdmap e209: 8 total, 8 up, 8 in 2024-09-20T22:08:55.520 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:55 smithi060 ceph-mon[40290]: pgmap v304: 158 pgs: 8 peering, 150 active+clean; 5.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 767 B/s rd, 81 MiB/s wr, 1.30k op/s; 144 KiB/s, 0 objects/s recovering 2024-09-20T22:08:55.520 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:55 smithi060 ceph-mon[40290]: osdmap e208: 8 total, 8 up, 8 in 2024-09-20T22:08:55.520 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:55 smithi060 ceph-mon[40290]: osdmap e209: 8 total, 8 up, 8 in 2024-09-20T22:08:55.551 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:55.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:55.552 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:55.653 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:55.697 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:55.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:55.698 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:55 smithi067 ceph-mon[33511]: pgmap v304: 158 pgs: 8 peering, 150 active+clean; 5.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 767 B/s rd, 81 MiB/s wr, 1.30k op/s; 144 KiB/s, 0 objects/s recovering 2024-09-20T22:08:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:55 smithi067 ceph-mon[33511]: osdmap e208: 8 total, 8 up, 8 in 2024-09-20T22:08:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:55 smithi067 ceph-mon[33511]: osdmap e209: 8 total, 8 up, 8 in 2024-09-20T22:08:55.799 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:55.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:55.844 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:55.844 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:55.857 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:08:55.857 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:08:55.857 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:08:55.857 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:08:55.858 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:08:55.858 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:08:55.858 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:08:55.858 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:08:55.858 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:08:55.858 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:08:55.858 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:08:55.859 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:08:55.859 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:08:55.859 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:08:55.859 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:08:55.859 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:08:55.946 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:55.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:55.987 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:55.987 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:56.088 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:56.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:56.129 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:56.129 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:56.146 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 59 16 91645 91629 97.0407 66.375 0.0236425 0.0103 2024-09-20T22:08:56.231 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:56.266 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:56.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:56.267 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:56.368 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:56.402 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:56.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:56.403 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:56.475 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:56 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3784539732' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:56.475 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:56 smithi060 ceph-mon[29694]: pgmap v307: 158 pgs: 8 peering, 150 active+clean; 5.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 36 MiB/s wr, 571 op/s; 77 B/s, 4 objects/s recovering 2024-09-20T22:08:56.475 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:56 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4195370574' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:08:56.475 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:56 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:08:56.475 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:56 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3784539732' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:56.475 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:56 smithi060 ceph-mon[40290]: pgmap v307: 158 pgs: 8 peering, 150 active+clean; 5.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 36 MiB/s wr, 571 op/s; 77 B/s, 4 objects/s recovering 2024-09-20T22:08:56.475 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:56 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4195370574' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:08:56.476 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:56 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:08:56.504 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:56.530 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:56.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:56.531 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:56.632 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:56.672 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:56.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:56.674 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:56.694 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T22:08:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:56 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3784539732' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:08:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:56 smithi067 ceph-mon[33511]: pgmap v307: 158 pgs: 8 peering, 150 active+clean; 5.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 36 MiB/s wr, 571 op/s; 77 B/s, 4 objects/s recovering 2024-09-20T22:08:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:56 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4195370574' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:08:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:56 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:08:56.775 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:56.804 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:56.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:56.806 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:56.907 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:56.938 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:56.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:56.939 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:57.040 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:57.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:57.072 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:57.072 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:57.146 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:08:57.145939+0000 min lat: 0.00126498 max lat: 2.31982 avg lat: 0.0103441 2024-09-20T22:08:57.146 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:08:57.146 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 60 16 92800 92784 96.6262 72.1875 0.00702854 0.0103441 2024-09-20T22:08:57.173 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:57.197 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:08:57.219 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:57.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:57.220 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:57.321 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:57.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:57.358 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:57.358 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:57.459 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:57.482 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:57.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:57.483 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:57.584 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:57.619 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:57.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:57.620 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:57.720 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:57 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:08:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:57 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:08:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:57 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:08:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:57 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:08:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:57 smithi067 ceph-mon[33511]: osdmap e210: 8 total, 8 up, 8 in 2024-09-20T22:08:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:57 smithi067 ceph-mon[33511]: 3.13 scrub starts 2024-09-20T22:08:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:57 smithi067 ceph-mon[33511]: 3.13 scrub ok 2024-09-20T22:08:57.767 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:57.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:57.769 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:57 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:08:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:57 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:08:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:57 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:08:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:57 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:08:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:57 smithi060 ceph-mon[29694]: osdmap e210: 8 total, 8 up, 8 in 2024-09-20T22:08:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:57 smithi060 ceph-mon[29694]: 3.13 scrub starts 2024-09-20T22:08:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:57 smithi060 ceph-mon[29694]: 3.13 scrub ok 2024-09-20T22:08:57.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:57 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:08:57.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:57 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:08:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:57 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:08:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:57 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:08:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:57 smithi060 ceph-mon[40290]: osdmap e210: 8 total, 8 up, 8 in 2024-09-20T22:08:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:57 smithi060 ceph-mon[40290]: 3.13 scrub starts 2024-09-20T22:08:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:57 smithi060 ceph-mon[40290]: 3.13 scrub ok 2024-09-20T22:08:57.870 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:57.906 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:57.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:57.907 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:58.008 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:58.043 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:58.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:58.044 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:58.145 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:58.146 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 61 15 93995 93980 96.2675 74.75 0.0196782 0.0103827 2024-09-20T22:08:58.185 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:58.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:58.186 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:58.287 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:58.306 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:08:58.316 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:58.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:58.316 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:58.417 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:58.468 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:58.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:58.469 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:58.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:58 smithi060 ceph-mon[29694]: pgmap v309: 158 pgs: 1 active+remapped+backfilling, 157 active+clean; 5.6 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.1 KiB/s rd, 47 MiB/s wr, 748 op/s; 5373/276642 objects misplaced (1.942%); 4.6 MiB/s, 82 objects/s recovering 2024-09-20T22:08:58.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:58 smithi060 ceph-mon[40290]: pgmap v309: 158 pgs: 1 active+remapped+backfilling, 157 active+clean; 5.6 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.1 KiB/s rd, 47 MiB/s wr, 748 op/s; 5373/276642 objects misplaced (1.942%); 4.6 MiB/s, 82 objects/s recovering 2024-09-20T22:08:58.570 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:08:58.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:58.605 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:58.605 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:08:58.706 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:58.745 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:58.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:58.746 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:58 smithi067 ceph-mon[33511]: pgmap v309: 158 pgs: 1 active+remapped+backfilling, 157 active+clean; 5.6 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.1 KiB/s rd, 47 MiB/s wr, 748 op/s; 5373/276642 objects misplaced (1.942%); 4.6 MiB/s, 82 objects/s recovering 2024-09-20T22:08:58.847 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:58.893 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:58.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:58.894 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:08:58.951 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:08:58.952 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:08:58.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:08:58.952 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:08:58.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.952 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169961 7.13 7:cf59cdb9:::benchmark_data_smithi060_78800_object36928:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:08:58.952 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440329+0000", 2024-09-20T22:08:58.952 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 180.50704868599999, 2024-09-20T22:08:58.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.067711163, 2024-09-20T22:08:58.952 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:58.952 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:58.952 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:08:58.952 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:58.952 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:58.953 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:58.953 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169961 2024-09-20T22:08:58.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.953 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:58.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:58.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440329+0000", 2024-09-20T22:08:58.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:58.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:58.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440329+0000", 2024-09-20T22:08:58.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:58.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:58.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440330+0000", 2024-09-20T22:08:58.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2800000000000005e-07 2024-09-20T22:08:58.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:58.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440335+0000", 2024-09-20T22:08:58.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9520000000000002e-06 2024-09-20T22:08:58.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:58.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440347+0000", 2024-09-20T22:08:58.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1732999999999999e-05 2024-09-20T22:08:58.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:58.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440354+0000", 2024-09-20T22:08:58.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6420000000000007e-06 2024-09-20T22:08:58.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:58.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.698470+0000", 2024-09-20T22:08:58.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.25811523800000002 2024-09-20T22:08:58.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:58.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.698477+0000", 2024-09-20T22:08:58.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9999999999999999e-06 2024-09-20T22:08:58.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:58.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.507960+0000", 2024-09-20T22:08:58.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.809483493 2024-09-20T22:08:58.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:58.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.508040+0000", 2024-09-20T22:08:58.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0177000000000002e-05 2024-09-20T22:08:58.956 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:58.957 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:58.957 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:58.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.957 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169966 7.13 7:c9a28d28:::benchmark_data_smithi060_78800_object36933:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:08:58.957 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440368+0000", 2024-09-20T22:08:58.957 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 180.50700943000001, 2024-09-20T22:08:58.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0730672910000001, 2024-09-20T22:08:58.957 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:58.957 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:58.957 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:08:58.957 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:58.957 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:58.957 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:58.957 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169966 2024-09-20T22:08:58.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.958 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:58.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:58.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440368+0000", 2024-09-20T22:08:58.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:58.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:58.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440368+0000", 2024-09-20T22:08:58.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:58.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:58.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440369+0000", 2024-09-20T22:08:58.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9100000000000004e-07 2024-09-20T22:08:58.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:58.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440371+0000", 2024-09-20T22:08:58.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5280000000000002e-06 2024-09-20T22:08:58.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:58.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440378+0000", 2024-09-20T22:08:58.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7340000000000002e-06 2024-09-20T22:08:58.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:58.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440383+0000", 2024-09-20T22:08:58.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3750000000000002e-06 2024-09-20T22:08:58.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:58.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.698495+0000", 2024-09-20T22:08:58.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.258111427 2024-09-20T22:08:58.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:58.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.698499+0000", 2024-09-20T22:08:58.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.658e-06 2024-09-20T22:08:58.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:58.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.509530+0000", 2024-09-20T22:08:58.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.81103158399999997 2024-09-20T22:08:58.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:08:58.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.509656+0000", 2024-09-20T22:08:58.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012591300000000001 2024-09-20T22:08:58.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:58.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.513371+0000", 2024-09-20T22:08:58.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037147370000000001 2024-09-20T22:08:58.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:58.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.513436+0000", 2024-09-20T22:08:58.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4844000000000005e-05 2024-09-20T22:08:58.962 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:58.962 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:58.962 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:58.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.962 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198108 7.10 7:0b18f923:::benchmark_data_smithi060_78800_object65075:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:08:58.962 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.462286+0000", 2024-09-20T22:08:58.962 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 153.485092076, 2024-09-20T22:08:58.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97321715200000003, 2024-09-20T22:08:58.963 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:58.963 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:58.963 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:58.963 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:58.963 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:58.963 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:58.963 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198108 2024-09-20T22:08:58.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.963 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:58.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:58.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462286+0000", 2024-09-20T22:08:58.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:58.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:58.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462286+0000", 2024-09-20T22:08:58.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:58.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:58.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462287+0000", 2024-09-20T22:08:58.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.668e-06 2024-09-20T22:08:58.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:58.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462293+0000", 2024-09-20T22:08:58.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0020000000000003e-06 2024-09-20T22:08:58.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:58.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462315+0000", 2024-09-20T22:08:58.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1809000000000001e-05 2024-09-20T22:08:58.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:58.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462324+0000", 2024-09-20T22:08:58.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6209999999999993e-06 2024-09-20T22:08:58.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:58.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462389+0000", 2024-09-20T22:08:58.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5007999999999998e-05 2024-09-20T22:08:58.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:58.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462397+0000", 2024-09-20T22:08:58.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3690000000000007e-06 2024-09-20T22:08:58.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:58.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430729+0000", 2024-09-20T22:08:58.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.96833147100000005 2024-09-20T22:08:58.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:58.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430856+0000", 2024-09-20T22:08:58.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012690800000000001 2024-09-20T22:08:58.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:08:58.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.431205+0000", 2024-09-20T22:08:58.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00034923899999999999 2024-09-20T22:08:58.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:58.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435090+0000", 2024-09-20T22:08:58.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038847090000000001 2024-09-20T22:08:58.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:58.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435103+0000", 2024-09-20T22:08:58.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169999999999999e-05 2024-09-20T22:08:58.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:58.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435254+0000", 2024-09-20T22:08:58.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015143900000000001 2024-09-20T22:08:58.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:58.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435404+0000", 2024-09-20T22:08:58.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014957600000000001 2024-09-20T22:08:58.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:58.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435503+0000", 2024-09-20T22:08:58.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9162999999999999e-05 2024-09-20T22:08:58.970 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:58.970 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:58.970 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:58.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.970 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198131 7.10 7:0b2c8a99:::benchmark_data_smithi060_78800_object65098:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:08:58.970 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.479927+0000", 2024-09-20T22:08:58.970 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 153.46745074, 2024-09-20T22:08:58.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.96409556600000001, 2024-09-20T22:08:58.970 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:58.970 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:58.971 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:58.971 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:58.971 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:58.971 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:58.971 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198131 2024-09-20T22:08:58.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.971 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:58.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:58.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479927+0000", 2024-09-20T22:08:58.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:58.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:58.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479927+0000", 2024-09-20T22:08:58.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:58.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:58.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479929+0000", 2024-09-20T22:08:58.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5850000000000001e-06 2024-09-20T22:08:58.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:58.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479934+0000", 2024-09-20T22:08:58.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1869999999999996e-06 2024-09-20T22:08:58.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:58.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479955+0000", 2024-09-20T22:08:58.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1353e-05 2024-09-20T22:08:58.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:58.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479962+0000", 2024-09-20T22:08:58.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7549999999999997e-06 2024-09-20T22:08:58.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:58.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.480018+0000", 2024-09-20T22:08:58.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6342999999999999e-05 2024-09-20T22:08:58.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:58.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.480027+0000", 2024-09-20T22:08:58.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7469999999999994e-06 2024-09-20T22:08:58.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:58.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.433323+0000", 2024-09-20T22:08:58.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.95329631199999998 2024-09-20T22:08:58.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:58.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.433425+0000", 2024-09-20T22:08:58.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000101271 2024-09-20T22:08:58.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:08:58.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.433690+0000", 2024-09-20T22:08:58.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026574900000000002 2024-09-20T22:08:58.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:58.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.437320+0000", 2024-09-20T22:08:58.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003630075 2024-09-20T22:08:58.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:58.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.437363+0000", 2024-09-20T22:08:58.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2104000000000002e-05 2024-09-20T22:08:58.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:58.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.443794+0000", 2024-09-20T22:08:58.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0064310729999999998 2024-09-20T22:08:58.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:58.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.443902+0000", 2024-09-20T22:08:58.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010815 2024-09-20T22:08:58.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:58.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.444023+0000", 2024-09-20T22:08:58.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000120862 2024-09-20T22:08:58.977 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:58.977 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:58.977 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:58.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.977 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198163 7.10 7:086b2df9:::benchmark_data_smithi060_78800_object65130:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:08:58.977 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.501070+0000", 2024-09-20T22:08:58.977 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 153.446307367, 2024-09-20T22:08:58.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94904837799999997, 2024-09-20T22:08:58.977 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:58.977 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:58.977 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:58.977 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:58.978 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:58.978 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:58.978 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198163 2024-09-20T22:08:58.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.978 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:58.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:58.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501070+0000", 2024-09-20T22:08:58.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:58.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:58.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501070+0000", 2024-09-20T22:08:58.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:58.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:58.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501072+0000", 2024-09-20T22:08:58.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.742e-06 2024-09-20T22:08:58.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:58.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501078+0000", 2024-09-20T22:08:58.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3859999999999998e-06 2024-09-20T22:08:58.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:58.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501090+0000", 2024-09-20T22:08:58.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2003e-05 2024-09-20T22:08:58.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:58.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501097+0000", 2024-09-20T22:08:58.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2089999999999999e-06 2024-09-20T22:08:58.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:58.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501154+0000", 2024-09-20T22:08:58.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6901000000000003e-05 2024-09-20T22:08:58.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:58.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501162+0000", 2024-09-20T22:08:58.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3440000000000001e-06 2024-09-20T22:08:58.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:58.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435556+0000", 2024-09-20T22:08:58.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.93439354699999999 2024-09-20T22:08:58.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:58.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435672+0000", 2024-09-20T22:08:58.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011694 2024-09-20T22:08:58.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:08:58.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435963+0000", 2024-09-20T22:08:58.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000290654 2024-09-20T22:08:58.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:58.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.440444+0000", 2024-09-20T22:08:58.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0044806450000000001 2024-09-20T22:08:58.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:58.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.440486+0000", 2024-09-20T22:08:58.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2036999999999997e-05 2024-09-20T22:08:58.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:58.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.449968+0000", 2024-09-20T22:08:58.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0094824560000000002 2024-09-20T22:08:58.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:58.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450024+0000", 2024-09-20T22:08:58.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5374000000000002e-05 2024-09-20T22:08:58.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:58.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450119+0000", 2024-09-20T22:08:58.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5140000000000001e-05 2024-09-20T22:08:58.984 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:58.984 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:58.984 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:58.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.984 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198198 7.10 7:0c0c59e3:::benchmark_data_smithi060_78800_object65165:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:08:58.984 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.527682+0000", 2024-09-20T22:08:58.984 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 153.41969538999999, 2024-09-20T22:08:58.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.92260386900000002, 2024-09-20T22:08:58.984 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:58.984 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:58.984 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:58.984 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:58.984 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:58.985 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:58.985 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198198 2024-09-20T22:08:58.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.985 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:58.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:58.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.527682+0000", 2024-09-20T22:08:58.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:58.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:58.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.527682+0000", 2024-09-20T22:08:58.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:58.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:58.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.527684+0000", 2024-09-20T22:08:58.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.423e-06 2024-09-20T22:08:58.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:58.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.527689+0000", 2024-09-20T22:08:58.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.259e-06 2024-09-20T22:08:58.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:58.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.527702+0000", 2024-09-20T22:08:58.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2738000000000001e-05 2024-09-20T22:08:58.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:58.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.527727+0000", 2024-09-20T22:08:58.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5088000000000001e-05 2024-09-20T22:08:58.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:58.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.527787+0000", 2024-09-20T22:08:58.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0043000000000001e-05 2024-09-20T22:08:58.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:58.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.527795+0000", 2024-09-20T22:08:58.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4209999999999995e-06 2024-09-20T22:08:58.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:58.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.437388+0000", 2024-09-20T22:08:58.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90959268900000001 2024-09-20T22:08:58.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:58.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.437497+0000", 2024-09-20T22:08:58.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010881400000000001 2024-09-20T22:08:58.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:08:58.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.437727+0000", 2024-09-20T22:08:58.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022985 2024-09-20T22:08:58.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:58.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.440498+0000", 2024-09-20T22:08:58.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002771267 2024-09-20T22:08:58.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:58.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.444097+0000", 2024-09-20T22:08:58.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0035986540000000002 2024-09-20T22:08:58.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:58.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450156+0000", 2024-09-20T22:08:58.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.006059699 2024-09-20T22:08:58.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:58.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450203+0000", 2024-09-20T22:08:58.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6325000000000001e-05 2024-09-20T22:08:58.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:58.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450286+0000", 2024-09-20T22:08:58.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3598999999999997e-05 2024-09-20T22:08:58.990 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:58.990 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:58.990 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198259 7.10 7:0ad0ad2c:::benchmark_data_smithi060_78800_object65226:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.583584+0000", 2024-09-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 153.36379399699999, 2024-09-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.86690348100000003, 2024-09-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198259 2024-09-20T22:08:58.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.992 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:58.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:58.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.583584+0000", 2024-09-20T22:08:58.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:58.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:58.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.583584+0000", 2024-09-20T22:08:58.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:58.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:58.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.583585+0000", 2024-09-20T22:08:58.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4780000000000001e-06 2024-09-20T22:08:58.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:58.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.583591+0000", 2024-09-20T22:08:58.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9719999999999997e-06 2024-09-20T22:08:58.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:58.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.583603+0000", 2024-09-20T22:08:58.993 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:08:58.993 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:08:58.994 INFO:tasks.thrashosds.thrasher:Growing pool unique_pool_1 2024-09-20T22:08:58.994 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:08:58.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2211999999999999e-05 2024-09-20T22:08:58.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:58.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.583619+0000", 2024-09-20T22:08:58.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5098e-05 2024-09-20T22:08:58.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:58.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.583677+0000", 2024-09-20T22:08:58.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8724999999999998e-05 2024-09-20T22:08:58.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:58.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.583686+0000", 2024-09-20T22:08:58.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3319999999999992e-06 2024-09-20T22:08:58.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:58.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.439048+0000", 2024-09-20T22:08:58.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85536227600000003 2024-09-20T22:08:58.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:58.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.439148+0000", 2024-09-20T22:08:58.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9788000000000001e-05 2024-09-20T22:08:58.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:08:58.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.439365+0000", 2024-09-20T22:08:58.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000217269 2024-09-20T22:08:58.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:58.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.443753+0000", 2024-09-20T22:08:58.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0043879169999999999 2024-09-20T22:08:58.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:58.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.449854+0000", 2024-09-20T22:08:58.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0061015959999999999 2024-09-20T22:08:58.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:58.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450358+0000", 2024-09-20T22:08:58.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00050349100000000001 2024-09-20T22:08:58.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:58.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450404+0000", 2024-09-20T22:08:58.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5806000000000002e-05 2024-09-20T22:08:58.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:58.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450487+0000", 2024-09-20T22:08:58.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3521000000000002e-05 2024-09-20T22:08:58.999 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:58.999 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:58.999 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:58.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:58.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:58.999 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198351 7.10 7:0faceefa:::benchmark_data_smithi060_78800_object65318:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:08:58.999 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.655692+0000", 2024-09-20T22:08:58.999 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 153.291685696, 2024-09-20T22:08:58.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.79496427599999997, 2024-09-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198351 2024-09-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.655692+0000", 2024-09-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.655692+0000", 2024-09-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.655693+0000", 2024-09-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3599999999999999e-06 2024-09-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.655699+0000", 2024-09-20T22:08:59.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3419999999999998e-06 2024-09-20T22:08:59.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:59.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.655711+0000", 2024-09-20T22:08:59.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2448e-05 2024-09-20T22:08:59.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:59.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.655729+0000", 2024-09-20T22:08:59.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8014000000000001e-05 2024-09-20T22:08:59.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.655792+0000", 2024-09-20T22:08:59.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2817999999999994e-05 2024-09-20T22:08:59.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:59.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.655800+0000", 2024-09-20T22:08:59.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0800000000000006e-06 2024-09-20T22:08:59.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.440525+0000", 2024-09-20T22:08:59.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.78472440200000004 2024-09-20T22:08:59.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:59.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.440622+0000", 2024-09-20T22:08:59.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7503999999999993e-05 2024-09-20T22:08:59.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:08:59.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.440832+0000", 2024-09-20T22:08:59.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020958199999999999 2024-09-20T22:08:59.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:59.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.444056+0000", 2024-09-20T22:08:59.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0032243939999999998 2024-09-20T22:08:59.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.449896+0000", 2024-09-20T22:08:59.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005840273 2024-09-20T22:08:59.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450523+0000", 2024-09-20T22:08:59.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00062680600000000004 2024-09-20T22:08:59.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:59.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450572+0000", 2024-09-20T22:08:59.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9317999999999998e-05 2024-09-20T22:08:59.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:59.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450656+0000", 2024-09-20T22:08:59.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3935000000000002e-05 2024-09-20T22:08:59.006 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.006 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:59.006 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.006 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227634 7.10 7:0ea5dec3:::benchmark_data_smithi060_78800_object94601:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:08:59.006 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.846865+0000", 2024-09-20T22:08:59.006 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 125.100512541, 2024-09-20T22:08:59.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1004788329999999, 2024-09-20T22:08:59.006 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:59.006 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:59.007 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:59.007 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:59.007 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:59.007 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:59.007 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227634 2024-09-20T22:08:59.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.007 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:59.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:59.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846865+0000", 2024-09-20T22:08:59.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:59.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846865+0000", 2024-09-20T22:08:59.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:59.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846867+0000", 2024-09-20T22:08:59.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.641e-06 2024-09-20T22:08:59.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:59.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846873+0000", 2024-09-20T22:08:59.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1489999999999999e-06 2024-09-20T22:08:59.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:59.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846887+0000", 2024-09-20T22:08:59.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4113e-05 2024-09-20T22:08:59.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:59.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846895+0000", 2024-09-20T22:08:59.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6699999999999994e-06 2024-09-20T22:08:59.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.852299+0000", 2024-09-20T22:08:59.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0054038059999999997 2024-09-20T22:08:59.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:59.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.852311+0000", 2024-09-20T22:08:59.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2278e-05 2024-09-20T22:08:59.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.941457+0000", 2024-09-20T22:08:59.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0891458009999999 2024-09-20T22:08:59.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:59.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.941594+0000", 2024-09-20T22:08:59.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013708000000000001 2024-09-20T22:08:59.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2,5", 2024-09-20T22:08:59.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.941949+0000", 2024-09-20T22:08:59.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00035502599999999999 2024-09-20T22:08:59.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946263+0000", 2024-09-20T22:08:59.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0043141150000000003 2024-09-20T22:08:59.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:59.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946286+0000", 2024-09-20T22:08:59.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3419000000000001e-05 2024-09-20T22:08:59.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946351+0000", 2024-09-20T22:08:59.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4231000000000002e-05 2024-09-20T22:08:59.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947173+0000", 2024-09-20T22:08:59.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00082238199999999997 2024-09-20T22:08:59.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:59.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947237+0000", 2024-09-20T22:08:59.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4462999999999996e-05 2024-09-20T22:08:59.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:59.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947344+0000", 2024-09-20T22:08:59.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000106659 2024-09-20T22:08:59.013 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.013 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:59.013 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.013 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227639 7.10 7:0834985a:::benchmark_data_smithi060_78800_object94606:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:08:59.013 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.856347+0000", 2024-09-20T22:08:59.014 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 125.09103097800001, 2024-09-20T22:08:59.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0915877439999999, 2024-09-20T22:08:59.014 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:59.014 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:59.014 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:59.014 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:59.014 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:59.014 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:59.014 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227639 2024-09-20T22:08:59.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.014 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:59.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:59.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856347+0000", 2024-09-20T22:08:59.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:59.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856347+0000", 2024-09-20T22:08:59.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:59.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856349+0000", 2024-09-20T22:08:59.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7090000000000001e-06 2024-09-20T22:08:59.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:59.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856355+0000", 2024-09-20T22:08:59.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4729999999999997e-06 2024-09-20T22:08:59.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:59.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856369+0000", 2024-09-20T22:08:59.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3832000000000001e-05 2024-09-20T22:08:59.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:59.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856378+0000", 2024-09-20T22:08:59.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8689999999999993e-06 2024-09-20T22:08:59.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856556+0000", 2024-09-20T22:08:59.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000178529 2024-09-20T22:08:59.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:59.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856568+0000", 2024-09-20T22:08:59.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2281e-05 2024-09-20T22:08:59.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.944093+0000", 2024-09-20T22:08:59.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.087524438 2024-09-20T22:08:59.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:59.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.944232+0000", 2024-09-20T22:08:59.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000139178 2024-09-20T22:08:59.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2,5", 2024-09-20T22:08:59.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.944533+0000", 2024-09-20T22:08:59.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030114999999999997 2024-09-20T22:08:59.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946726+0000", 2024-09-20T22:08:59.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021925809999999999 2024-09-20T22:08:59.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:59.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946797+0000", 2024-09-20T22:08:59.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1570999999999996e-05 2024-09-20T22:08:59.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947394+0000", 2024-09-20T22:08:59.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00059673799999999998 2024-09-20T22:08:59.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947773+0000", 2024-09-20T22:08:59.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00037865499999999999 2024-09-20T22:08:59.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:59.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947831+0000", 2024-09-20T22:08:59.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8557000000000002e-05 2024-09-20T22:08:59.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:59.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947935+0000", 2024-09-20T22:08:59.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103183 2024-09-20T22:08:59.020 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.020 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:59.020 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.021 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227649 7.2 7:46c04214:::benchmark_data_smithi060_78800_object94616:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:08:59.021 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.900917+0000", 2024-09-20T22:08:59.021 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 125.04646093300001, 2024-09-20T22:08:59.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1912150110000002, 2024-09-20T22:08:59.021 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:59.021 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:59.021 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:59.021 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:59.021 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:59.021 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:59.021 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227649 2024-09-20T22:08:59.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.021 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:59.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:59.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900917+0000", 2024-09-20T22:08:59.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.022 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:08:59.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:59.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900917+0000", 2024-09-20T22:08:59.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:59.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900919+0000", 2024-09-20T22:08:59.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7400000000000001e-06 2024-09-20T22:08:59.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:59.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900924+0000", 2024-09-20T22:08:59.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6439999999999997e-06 2024-09-20T22:08:59.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:59.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900941+0000", 2024-09-20T22:08:59.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7244e-05 2024-09-20T22:08:59.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:59.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900949+0000", 2024-09-20T22:08:59.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0829999999999998e-06 2024-09-20T22:08:59.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.901025+0000", 2024-09-20T22:08:59.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6643999999999993e-05 2024-09-20T22:08:59.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:59.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.901037+0000", 2024-09-20T22:08:59.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1824999999999999e-05 2024-09-20T22:08:59.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.817605+0000", 2024-09-20T22:08:59.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91656772500000006 2024-09-20T22:08:59.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:59.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.817617+0000", 2024-09-20T22:08:59.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2320999999999999e-05 2024-09-20T22:08:59.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.067011+0000", 2024-09-20T22:08:59.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2493934259999999 2024-09-20T22:08:59.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:59.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.067109+0000", 2024-09-20T22:08:59.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8904e-05 2024-09-20T22:08:59.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:08:59.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.067424+0000", 2024-09-20T22:08:59.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00031460400000000001 2024-09-20T22:08:59.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:59.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.069092+0000", 2024-09-20T22:08:59.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001667836 2024-09-20T22:08:59.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.070743+0000", 2024-09-20T22:08:59.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001650763 2024-09-20T22:08:59.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.091949+0000", 2024-09-20T22:08:59.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021206434999999999 2024-09-20T22:08:59.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:59.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092012+0000", 2024-09-20T22:08:59.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2651999999999996e-05 2024-09-20T22:08:59.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:59.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092132+0000", 2024-09-20T22:08:59.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000120165 2024-09-20T22:08:59.029 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.029 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:59.029 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.029 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227675 7.2 7:45bb0f01:::benchmark_data_smithi060_78800_object94642:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-20T22:08:59.029 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:54.993128+0000", 2024-09-20T22:08:59.029 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 123.95424970099999, 2024-09-20T22:08:59.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.09956863, 2024-09-20T22:08:59.030 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:59.030 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:59.030 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:59.030 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:59.030 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:59.030 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:59.030 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227675 2024-09-20T22:08:59.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.030 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:59.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:59.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993128+0000", 2024-09-20T22:08:59.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:59.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993128+0000", 2024-09-20T22:08:59.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:59.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993130+0000", 2024-09-20T22:08:59.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5400000000000001e-06 2024-09-20T22:08:59.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:59.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993136+0000", 2024-09-20T22:08:59.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.669e-06 2024-09-20T22:08:59.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:59.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993153+0000", 2024-09-20T22:08:59.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6538999999999999e-05 2024-09-20T22:08:59.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:59.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993162+0000", 2024-09-20T22:08:59.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2529999999999993e-06 2024-09-20T22:08:59.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993239+0000", 2024-09-20T22:08:59.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6499000000000001e-05 2024-09-20T22:08:59.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:59.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993253+0000", 2024-09-20T22:08:59.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4278e-05 2024-09-20T22:08:59.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.069151+0000", 2024-09-20T22:08:59.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0758977059999999 2024-09-20T22:08:59.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:59.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.069260+0000", 2024-09-20T22:08:59.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109175 2024-09-20T22:08:59.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:08:59.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.069491+0000", 2024-09-20T22:08:59.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000231716 2024-09-20T22:08:59.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:59.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.070761+0000", 2024-09-20T22:08:59.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012690850000000001 2024-09-20T22:08:59.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.035 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.035 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.071295+0000", 2024-09-20T22:08:59.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00053413799999999997 2024-09-20T22:08:59.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.035 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.035 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092555+0000", 2024-09-20T22:08:59.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021260233 2024-09-20T22:08:59.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.035 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:59.035 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092609+0000", 2024-09-20T22:08:59.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4147999999999997e-05 2024-09-20T22:08:59.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:59.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092697+0000", 2024-09-20T22:08:59.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7651000000000005e-05 2024-09-20T22:08:59.036 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.036 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:59.036 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.036 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236464 7.2 7:41e7f20e:::benchmark_data_smithi060_78800_object103431:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:08:59.036 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.420278+0000", 2024-09-20T22:08:59.036 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 111.527099683, 2024-09-20T22:08:59.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99395603799999999, 2024-09-20T22:08:59.037 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:59.037 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:59.037 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:59.037 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:59.037 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:59.037 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:59.037 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236464 2024-09-20T22:08:59.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.037 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:59.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:59.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420278+0000", 2024-09-20T22:08:59.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:59.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420278+0000", 2024-09-20T22:08:59.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:59.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420280+0000", 2024-09-20T22:08:59.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0049999999999999e-06 2024-09-20T22:08:59.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:59.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420287+0000", 2024-09-20T22:08:59.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7429999999999996e-06 2024-09-20T22:08:59.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:59.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420305+0000", 2024-09-20T22:08:59.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.836e-05 2024-09-20T22:08:59.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:59.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420314+0000", 2024-09-20T22:08:59.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7329999999999992e-06 2024-09-20T22:08:59.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.421421+0000", 2024-09-20T22:08:59.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001107455 2024-09-20T22:08:59.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:59.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.421429+0000", 2024-09-20T22:08:59.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5669999999999997e-06 2024-09-20T22:08:59.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.405308+0000", 2024-09-20T22:08:59.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98387922900000002 2024-09-20T22:08:59.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:59.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.405428+0000", 2024-09-20T22:08:59.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012023699999999999 2024-09-20T22:08:59.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:08:59.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.405861+0000", 2024-09-20T22:08:59.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00043245600000000001 2024-09-20T22:08:59.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:59.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414023+0000", 2024-09-20T22:08:59.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0081622280000000005 2024-09-20T22:08:59.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414092+0000", 2024-09-20T22:08:59.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9129999999999997e-05 2024-09-20T22:08:59.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414141+0000", 2024-09-20T22:08:59.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8426e-05 2024-09-20T22:08:59.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:59.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414196+0000", 2024-09-20T22:08:59.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5226999999999998e-05 2024-09-20T22:08:59.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:59.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414234+0000", 2024-09-20T22:08:59.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8241999999999997e-05 2024-09-20T22:08:59.043 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.043 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:59.043 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.043 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236481 7.2 7:417f7fb8:::benchmark_data_smithi060_78800_object103448:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:08:59.043 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.436984+0000", 2024-09-20T22:08:59.044 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 111.510393463, 2024-09-20T22:08:59.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98983631900000002, 2024-09-20T22:08:59.044 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:59.044 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:59.044 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:59.044 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:59.044 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:59.044 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:59.044 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236481 2024-09-20T22:08:59.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.044 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:59.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:59.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.436984+0000", 2024-09-20T22:08:59.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:59.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.436984+0000", 2024-09-20T22:08:59.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:59.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.436985+0000", 2024-09-20T22:08:59.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-07 2024-09-20T22:08:59.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:59.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.436990+0000", 2024-09-20T22:08:59.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7770000000000002e-06 2024-09-20T22:08:59.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:59.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.437004+0000", 2024-09-20T22:08:59.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3572e-05 2024-09-20T22:08:59.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:59.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.437010+0000", 2024-09-20T22:08:59.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9469999999999997e-06 2024-09-20T22:08:59.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.439285+0000", 2024-09-20T22:08:59.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022742539999999999 2024-09-20T22:08:59.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:59.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.439294+0000", 2024-09-20T22:08:59.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0720000000000006e-06 2024-09-20T22:08:59.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.408421+0000", 2024-09-20T22:08:59.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.96912711200000001 2024-09-20T22:08:59.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:59.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.408517+0000", 2024-09-20T22:08:59.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.624e-05 2024-09-20T22:08:59.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:08:59.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.408884+0000", 2024-09-20T22:08:59.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00036694800000000002 2024-09-20T22:08:59.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414277+0000", 2024-09-20T22:08:59.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0053932440000000002 2024-09-20T22:08:59.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.419733+0000", 2024-09-20T22:08:59.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0054556500000000003 2024-09-20T22:08:59.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:59.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426386+0000", 2024-09-20T22:08:59.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0066532680000000004 2024-09-20T22:08:59.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:59.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426769+0000", 2024-09-20T22:08:59.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000382347 2024-09-20T22:08:59.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:59.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426821+0000", 2024-09-20T22:08:59.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2008000000000001e-05 2024-09-20T22:08:59.050 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.050 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:59.050 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.051 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236521 7.2 7:400e2278:::benchmark_data_smithi060_78800_object103488:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:08:59.052 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.486226+0000", 2024-09-20T22:08:59.052 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 111.461151695, 2024-09-20T22:08:59.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94626404099999994, 2024-09-20T22:08:59.052 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:59.052 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:59.052 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:59.052 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:59.052 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:59.052 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:59.052 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236521 2024-09-20T22:08:59.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.052 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:59.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:59.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.486226+0000", 2024-09-20T22:08:59.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:59.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.486226+0000", 2024-09-20T22:08:59.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:59.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.486228+0000", 2024-09-20T22:08:59.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0090000000000002e-06 2024-09-20T22:08:59.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:59.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.486235+0000", 2024-09-20T22:08:59.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.55e-06 2024-09-20T22:08:59.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:59.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.486249+0000", 2024-09-20T22:08:59.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3993e-05 2024-09-20T22:08:59.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:59.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.486256+0000", 2024-09-20T22:08:59.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7829999999999999e-06 2024-09-20T22:08:59.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.487970+0000", 2024-09-20T22:08:59.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017137630000000001 2024-09-20T22:08:59.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:59.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.487977+0000", 2024-09-20T22:08:59.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0319999999999998e-06 2024-09-20T22:08:59.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.411021+0000", 2024-09-20T22:08:59.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.92304408900000001 2024-09-20T22:08:59.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:59.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.411137+0000", 2024-09-20T22:08:59.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115716 2024-09-20T22:08:59.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:08:59.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.411662+0000", 2024-09-20T22:08:59.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00052523800000000003 2024-09-20T22:08:59.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.419783+0000", 2024-09-20T22:08:59.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0081203500000000001 2024-09-20T22:08:59.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:59.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426834+0000", 2024-09-20T22:08:59.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.007051163 2024-09-20T22:08:59.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432386+0000", 2024-09-20T22:08:59.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0055518499999999997 2024-09-20T22:08:59.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:59.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432445+0000", 2024-09-20T22:08:59.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8984999999999999e-05 2024-09-20T22:08:59.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:59.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432490+0000", 2024-09-20T22:08:59.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5519999999999998e-05 2024-09-20T22:08:59.059 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.059 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:59.059 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.059 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236528 7.2 7:47f1bc45:::benchmark_data_smithi060_78800_object103495:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:08:59.059 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.491029+0000", 2024-09-20T22:08:59.059 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 111.456348643, 2024-09-20T22:08:59.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.941676018, 2024-09-20T22:08:59.059 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:59.059 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:59.059 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:59.059 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:59.059 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:59.059 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:59.060 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236528 2024-09-20T22:08:59.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.060 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:59.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:59.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.491029+0000", 2024-09-20T22:08:59.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:59.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.491029+0000", 2024-09-20T22:08:59.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:59.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.491030+0000", 2024-09-20T22:08:59.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2300000000000001e-06 2024-09-20T22:08:59.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:59.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.491035+0000", 2024-09-20T22:08:59.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4800000000000003e-06 2024-09-20T22:08:59.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:59.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.491044+0000", 2024-09-20T22:08:59.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6780000000000006e-06 2024-09-20T22:08:59.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:59.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.491048+0000", 2024-09-20T22:08:59.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7940000000000002e-06 2024-09-20T22:08:59.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.491102+0000", 2024-09-20T22:08:59.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4011999999999997e-05 2024-09-20T22:08:59.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:59.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.491111+0000", 2024-09-20T22:08:59.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3639999999999999e-06 2024-09-20T22:08:59.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414308+0000", 2024-09-20T22:08:59.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.92319775000000004 2024-09-20T22:08:59.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:59.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414417+0000", 2024-09-20T22:08:59.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108238 2024-09-20T22:08:59.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:08:59.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414770+0000", 2024-09-20T22:08:59.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00035358200000000002 2024-09-20T22:08:59.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.419827+0000", 2024-09-20T22:08:59.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0050570479999999998 2024-09-20T22:08:59.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:59.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426890+0000", 2024-09-20T22:08:59.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0070631629999999999 2024-09-20T22:08:59.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.065 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432621+0000", 2024-09-20T22:08:59.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0057304740000000002 2024-09-20T22:08:59.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.065 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:59.065 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432667+0000", 2024-09-20T22:08:59.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5770999999999999e-05 2024-09-20T22:08:59.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.065 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:59.065 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432705+0000", 2024-09-20T22:08:59.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8433999999999998e-05 2024-09-20T22:08:59.065 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.065 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:59.066 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.066 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236532 7.2 7:46848d74:::benchmark_data_smithi060_78800_object103499:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:08:59.066 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.496306+0000", 2024-09-20T22:08:59.066 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 111.451071653, 2024-09-20T22:08:59.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.93652918600000001, 2024-09-20T22:08:59.066 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:59.066 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:59.066 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:59.066 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:59.066 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:59.066 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:59.067 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236532 2024-09-20T22:08:59.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.067 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:59.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:59.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.496306+0000", 2024-09-20T22:08:59.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:59.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.496306+0000", 2024-09-20T22:08:59.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:59.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.496307+0000", 2024-09-20T22:08:59.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2279999999999999e-06 2024-09-20T22:08:59.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:59.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.496313+0000", 2024-09-20T22:08:59.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4330000000000003e-06 2024-09-20T22:08:59.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:59.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.496329+0000", 2024-09-20T22:08:59.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6271000000000001e-05 2024-09-20T22:08:59.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:59.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.496336+0000", 2024-09-20T22:08:59.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1770000000000001e-06 2024-09-20T22:08:59.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.496443+0000", 2024-09-20T22:08:59.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010645999999999999 2024-09-20T22:08:59.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:59.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.496451+0000", 2024-09-20T22:08:59.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8790000000000003e-06 2024-09-20T22:08:59.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.416957+0000", 2024-09-20T22:08:59.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.92050670800000001 2024-09-20T22:08:59.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:59.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.417078+0000", 2024-09-20T22:08:59.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012116300000000001 2024-09-20T22:08:59.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:08:59.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.417454+0000", 2024-09-20T22:08:59.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00037505600000000002 2024-09-20T22:08:59.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426372+0000", 2024-09-20T22:08:59.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0089183890000000005 2024-09-20T22:08:59.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:59.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426896+0000", 2024-09-20T22:08:59.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00052363099999999999 2024-09-20T22:08:59.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432754+0000", 2024-09-20T22:08:59.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0058583769999999997 2024-09-20T22:08:59.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:59.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432800+0000", 2024-09-20T22:08:59.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5955999999999998e-05 2024-09-20T22:08:59.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:59.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432835+0000", 2024-09-20T22:08:59.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5457999999999998e-05 2024-09-20T22:08:59.073 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.073 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:59.073 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.073 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236566 7.2 7:4368a0c6:::benchmark_data_smithi060_78800_object103533:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:08:59.073 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.564354+0000", 2024-09-20T22:08:59.073 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 111.38302337499999, 2024-09-20T22:08:59.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.86859894900000001, 2024-09-20T22:08:59.073 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:59.073 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:59.073 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:59.073 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:59.073 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:59.074 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:59.074 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236566 2024-09-20T22:08:59.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.074 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:59.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:59.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.564354+0000", 2024-09-20T22:08:59.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:59.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.564354+0000", 2024-09-20T22:08:59.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:59.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.564356+0000", 2024-09-20T22:08:59.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4100000000000001e-06 2024-09-20T22:08:59.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:59.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.564362+0000", 2024-09-20T22:08:59.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6789999999999999e-06 2024-09-20T22:08:59.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:59.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.564377+0000", 2024-09-20T22:08:59.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4226000000000001e-05 2024-09-20T22:08:59.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:59.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.564385+0000", 2024-09-20T22:08:59.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8699999999999992e-06 2024-09-20T22:08:59.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.564447+0000", 2024-09-20T22:08:59.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1966999999999998e-05 2024-09-20T22:08:59.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:59.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.564455+0000", 2024-09-20T22:08:59.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-06 2024-09-20T22:08:59.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.419861+0000", 2024-09-20T22:08:59.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85540531399999997 2024-09-20T22:08:59.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:59.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.419968+0000", 2024-09-20T22:08:59.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000107507 2024-09-20T22:08:59.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:08:59.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.420312+0000", 2024-09-20T22:08:59.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00034424199999999997 2024-09-20T22:08:59.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426878+0000", 2024-09-20T22:08:59.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0065659409999999996 2024-09-20T22:08:59.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:59.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426900+0000", 2024-09-20T22:08:59.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1934999999999999e-05 2024-09-20T22:08:59.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432876+0000", 2024-09-20T22:08:59.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0059759380000000001 2024-09-20T22:08:59.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:59.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432919+0000", 2024-09-20T22:08:59.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3010000000000003e-05 2024-09-20T22:08:59.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:59.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432953+0000", 2024-09-20T22:08:59.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4109999999999997e-05 2024-09-20T22:08:59.081 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.081 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:59.081 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.081 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236574 7.2 7:47f87b9b:::benchmark_data_smithi060_78800_object103541:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:08:59.081 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.580098+0000", 2024-09-20T22:08:59.081 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 111.367280151, 2024-09-20T22:08:59.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85343583899999997, 2024-09-20T22:08:59.081 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:59.081 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:59.081 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:59.081 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:59.081 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:59.082 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:59.082 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236574 2024-09-20T22:08:59.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.082 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:59.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:59.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.580098+0000", 2024-09-20T22:08:59.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:59.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.580098+0000", 2024-09-20T22:08:59.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:59.083 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.580099+0000", 2024-09-20T22:08:59.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.643e-06 2024-09-20T22:08:59.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:59.083 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.580106+0000", 2024-09-20T22:08:59.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4609999999999996e-06 2024-09-20T22:08:59.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:59.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.580119+0000", 2024-09-20T22:08:59.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3094e-05 2024-09-20T22:08:59.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:59.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.580127+0000", 2024-09-20T22:08:59.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2740000000000007e-06 2024-09-20T22:08:59.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.580201+0000", 2024-09-20T22:08:59.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3867000000000003e-05 2024-09-20T22:08:59.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:59.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.580211+0000", 2024-09-20T22:08:59.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0225e-05 2024-09-20T22:08:59.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.422840+0000", 2024-09-20T22:08:59.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.84262903099999997 2024-09-20T22:08:59.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:59.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.422949+0000", 2024-09-20T22:08:59.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108801 2024-09-20T22:08:59.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:08:59.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.423319+0000", 2024-09-20T22:08:59.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00036967100000000002 2024-09-20T22:08:59.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:59.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426904+0000", 2024-09-20T22:08:59.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00358553 2024-09-20T22:08:59.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.432578+0000", 2024-09-20T22:08:59.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005673984 2024-09-20T22:08:59.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.433451+0000", 2024-09-20T22:08:59.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00087312200000000005 2024-09-20T22:08:59.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:59.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.433497+0000", 2024-09-20T22:08:59.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5547000000000003e-05 2024-09-20T22:08:59.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:59.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.433533+0000", 2024-09-20T22:08:59.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6588999999999998e-05 2024-09-20T22:08:59.088 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.088 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:59.088 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.088 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236582 7.2 7:444ff2c7:::benchmark_data_smithi060_78800_object103549:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:08:59.088 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.613480+0000", 2024-09-20T22:08:59.088 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 111.333897588, 2024-09-20T22:08:59.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.82262310000000005, 2024-09-20T22:08:59.088 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:08:59.088 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:08:59.088 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:08:59.088 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:08:59.089 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:08:59.089 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:08:59.089 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236582 2024-09-20T22:08:59.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.089 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:08:59.089 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.089 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:08:59.089 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.613480+0000", 2024-09-20T22:08:59.089 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.089 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.089 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:08:59.089 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.613480+0000", 2024-09-20T22:08:59.089 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:08:59.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:08:59.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.613482+0000", 2024-09-20T22:08:59.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3319999999999999e-06 2024-09-20T22:08:59.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:08:59.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.613486+0000", 2024-09-20T22:08:59.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.848e-06 2024-09-20T22:08:59.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:08:59.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.613500+0000", 2024-09-20T22:08:59.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3609e-05 2024-09-20T22:08:59.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:08:59.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.613505+0000", 2024-09-20T22:08:59.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4970000000000001e-06 2024-09-20T22:08:59.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.613567+0000", 2024-09-20T22:08:59.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1502999999999997e-05 2024-09-20T22:08:59.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:08:59.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.613577+0000", 2024-09-20T22:08:59.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0372e-05 2024-09-20T22:08:59.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:08:59.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.429648+0000", 2024-09-20T22:08:59.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.81607059699999995 2024-09-20T22:08:59.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:08:59.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.429792+0000", 2024-09-20T22:08:59.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014414699999999999 2024-09-20T22:08:59.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.093 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:08:59.093 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.430189+0000", 2024-09-20T22:08:59.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00039722899999999997 2024-09-20T22:08:59.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.093 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.093 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.434069+0000", 2024-09-20T22:08:59.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038798769999999999 2024-09-20T22:08:59.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.093 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:08:59.093 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.434144+0000", 2024-09-20T22:08:59.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4635000000000006e-05 2024-09-20T22:08:59.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:08:59.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.436012+0000", 2024-09-20T22:08:59.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0018679300000000001 2024-09-20T22:08:59.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:08:59.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.436065+0000", 2024-09-20T22:08:59.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2753999999999997e-05 2024-09-20T22:08:59.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:08:59.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:08:59.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:08:59.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.436103+0000", 2024-09-20T22:08:59.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8770000000000003e-05 2024-09-20T22:08:59.095 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.095 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:59.095 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.095 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:08:59.095 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:08:59.095 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:08:59.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:59.096 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:59.096 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:08:59.146 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:08:59.147 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 62 16 95140 95124 95.868 71.5 0.0078116 0.0104257 2024-09-20T22:08:59.196 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:08:59.250 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:59.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:59.251 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:08:59.352 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:08:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:59.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:59.405 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:08:59.505 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:08:59.548 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:59.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:59.550 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:08:59.650 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:08:59.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:59.690 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:08:59.690 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:08:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:59 smithi067 ceph-mon[33511]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-20T22:08:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:59 smithi067 ceph-mon[33511]: 1.9 scrub starts 2024-09-20T22:08:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:59 smithi067 ceph-mon[33511]: 1.9 scrub ok 2024-09-20T22:08:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:59 smithi067 ceph-mon[33511]: 1.4 scrub starts 2024-09-20T22:08:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:08:59 smithi067 ceph-mon[33511]: 1.4 scrub ok 2024-09-20T22:08:59.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:59 smithi060 ceph-mon[40290]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-20T22:08:59.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:59 smithi060 ceph-mon[40290]: 1.9 scrub starts 2024-09-20T22:08:59.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:59 smithi060 ceph-mon[40290]: 1.9 scrub ok 2024-09-20T22:08:59.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:59 smithi060 ceph-mon[40290]: 1.4 scrub starts 2024-09-20T22:08:59.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:08:59 smithi060 ceph-mon[40290]: 1.4 scrub ok 2024-09-20T22:08:59.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:59 smithi060 ceph-mon[29694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-09-20T22:08:59.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:59 smithi060 ceph-mon[29694]: 1.9 scrub starts 2024-09-20T22:08:59.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:59 smithi060 ceph-mon[29694]: 1.9 scrub ok 2024-09-20T22:08:59.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:59 smithi060 ceph-mon[29694]: 1.4 scrub starts 2024-09-20T22:08:59.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:08:59 smithi060 ceph-mon[29694]: 1.4 scrub ok 2024-09-20T22:08:59.791 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:08:59.832 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:59.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:59.833 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:08:59.934 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:08:59.975 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:08:59.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:08:59.976 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:00.077 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:00.111 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:00.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:00.112 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:00.146 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 63 16 96318 96302 95.5149 73.625 0.00606692 0.0104648 2024-09-20T22:09:00.213 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:00.250 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:00.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:00.251 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:00.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:09:00.352 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:00.399 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:00.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:00.400 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:00.500 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:00.525 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:00.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:00.526 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:00.627 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:00.667 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:00.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:00.668 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:00 smithi067 ceph-mon[33511]: 5.4 scrub starts 2024-09-20T22:09:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:00 smithi067 ceph-mon[33511]: 5.4 scrub ok 2024-09-20T22:09:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:00 smithi067 ceph-mon[33511]: 1.3 scrub starts 2024-09-20T22:09:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:00 smithi067 ceph-mon[33511]: 1.3 scrub ok 2024-09-20T22:09:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:00 smithi067 ceph-mon[33511]: 1.5 scrub starts 2024-09-20T22:09:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:00 smithi067 ceph-mon[33511]: 1.5 scrub ok 2024-09-20T22:09:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:00 smithi067 ceph-mon[33511]: pgmap v310: 158 pgs: 1 active+remapped+backfilling, 157 active+clean; 5.6 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 36 MiB/s wr, 577 op/s; 5373/276642 objects misplaced (1.942%); 3.5 MiB/s, 63 objects/s recovering 2024-09-20T22:09:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:00 smithi067 ceph-mon[33511]: 1.6 scrub starts 2024-09-20T22:09:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:00 smithi067 ceph-mon[33511]: 1.6 scrub ok 2024-09-20T22:09:00.769 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:00 smithi060 ceph-mon[40290]: 5.4 scrub starts 2024-09-20T22:09:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:00 smithi060 ceph-mon[40290]: 5.4 scrub ok 2024-09-20T22:09:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:00 smithi060 ceph-mon[40290]: 1.3 scrub starts 2024-09-20T22:09:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:00 smithi060 ceph-mon[40290]: 1.3 scrub ok 2024-09-20T22:09:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:00 smithi060 ceph-mon[40290]: 1.5 scrub starts 2024-09-20T22:09:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:00 smithi060 ceph-mon[40290]: 1.5 scrub ok 2024-09-20T22:09:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:00 smithi060 ceph-mon[40290]: pgmap v310: 158 pgs: 1 active+remapped+backfilling, 157 active+clean; 5.6 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 36 MiB/s wr, 577 op/s; 5373/276642 objects misplaced (1.942%); 3.5 MiB/s, 63 objects/s recovering 2024-09-20T22:09:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:00 smithi060 ceph-mon[40290]: 1.6 scrub starts 2024-09-20T22:09:00.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:00 smithi060 ceph-mon[40290]: 1.6 scrub ok 2024-09-20T22:09:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:00 smithi060 ceph-mon[29694]: 5.4 scrub starts 2024-09-20T22:09:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:00 smithi060 ceph-mon[29694]: 5.4 scrub ok 2024-09-20T22:09:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:00 smithi060 ceph-mon[29694]: 1.3 scrub starts 2024-09-20T22:09:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:00 smithi060 ceph-mon[29694]: 1.3 scrub ok 2024-09-20T22:09:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:00 smithi060 ceph-mon[29694]: 1.5 scrub starts 2024-09-20T22:09:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:00 smithi060 ceph-mon[29694]: 1.5 scrub ok 2024-09-20T22:09:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:00 smithi060 ceph-mon[29694]: pgmap v310: 158 pgs: 1 active+remapped+backfilling, 157 active+clean; 5.6 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 36 MiB/s wr, 577 op/s; 5373/276642 objects misplaced (1.942%); 3.5 MiB/s, 63 objects/s recovering 2024-09-20T22:09:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:00 smithi060 ceph-mon[29694]: 1.6 scrub starts 2024-09-20T22:09:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:00 smithi060 ceph-mon[29694]: 1.6 scrub ok 2024-09-20T22:09:00.800 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:00.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:00.801 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:00.902 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:00.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:00.942 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:00.942 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:01.043 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:01.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:01.076 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:01.076 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:01.146 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 64 16 97496 97480 95.1728 73.625 0.00952623 0.0105023 2024-09-20T22:09:01.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:01.215 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:01.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:01.216 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:01.317 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:01.350 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:01.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:01.351 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:01.452 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:01.484 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:01.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:01.485 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:01.585 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:01.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:01.611 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:01.612 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:01.712 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:01.754 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:01.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:01.755 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:01 smithi067 ceph-mon[33511]: 6.15 scrub starts 2024-09-20T22:09:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:01 smithi067 ceph-mon[33511]: 6.15 scrub ok 2024-09-20T22:09:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:01 smithi067 ceph-mon[33511]: 3.1e scrub starts 2024-09-20T22:09:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:01 smithi067 ceph-mon[33511]: 3.1e scrub ok 2024-09-20T22:09:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:01 smithi067 ceph-mon[33511]: 5.11 scrub starts 2024-09-20T22:09:01.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:01 smithi067 ceph-mon[33511]: 5.11 scrub ok 2024-09-20T22:09:01.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:01 smithi067 ceph-mon[33511]: osdmap e211: 8 total, 8 up, 8 in 2024-09-20T22:09:01.779 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:01 smithi060 ceph-mon[40290]: 6.15 scrub starts 2024-09-20T22:09:01.779 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:01 smithi060 ceph-mon[40290]: 6.15 scrub ok 2024-09-20T22:09:01.779 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:01 smithi060 ceph-mon[40290]: 3.1e scrub starts 2024-09-20T22:09:01.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:01 smithi060 ceph-mon[40290]: 3.1e scrub ok 2024-09-20T22:09:01.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:01 smithi060 ceph-mon[40290]: 5.11 scrub starts 2024-09-20T22:09:01.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:01 smithi060 ceph-mon[40290]: 5.11 scrub ok 2024-09-20T22:09:01.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:01 smithi060 ceph-mon[40290]: osdmap e211: 8 total, 8 up, 8 in 2024-09-20T22:09:01.780 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:01 smithi060 ceph-mon[29694]: 6.15 scrub starts 2024-09-20T22:09:01.780 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:01 smithi060 ceph-mon[29694]: 6.15 scrub ok 2024-09-20T22:09:01.780 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:01 smithi060 ceph-mon[29694]: 3.1e scrub starts 2024-09-20T22:09:01.780 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:01 smithi060 ceph-mon[29694]: 3.1e scrub ok 2024-09-20T22:09:01.780 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:01 smithi060 ceph-mon[29694]: 5.11 scrub starts 2024-09-20T22:09:01.780 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:01 smithi060 ceph-mon[29694]: 5.11 scrub ok 2024-09-20T22:09:01.780 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:01 smithi060 ceph-mon[29694]: osdmap e211: 8 total, 8 up, 8 in 2024-09-20T22:09:01.834 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T22:09:01.856 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:01.869 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:01.919 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:01.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:01.920 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:02.021 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:02.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:02.059 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:02.059 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:02.146 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 65 15 98557 98542 94.7298 66.375 0.00829866 0.0105517 2024-09-20T22:09:02.160 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:02.221 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:02.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:02.222 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:02.235 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:09:02.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:09:02 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:09:02.323 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:02.359 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:02.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:02.360 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:02.461 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:02.470 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:02.475 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:02.516 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:02.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:02.517 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:02.564 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:09:02.564 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:09:02.611 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:09:02.612 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:09:02.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[29694]: 5.8 scrub starts 2024-09-20T22:09:02.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[29694]: 5.8 scrub ok 2024-09-20T22:09:02.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[29694]: 4.c scrub starts 2024-09-20T22:09:02.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[29694]: 4.c scrub ok 2024-09-20T22:09:02.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[29694]: 5.10 scrub starts 2024-09-20T22:09:02.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[29694]: 5.10 scrub ok 2024-09-20T22:09:02.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[29694]: 3.7 scrub starts 2024-09-20T22:09:02.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[29694]: 3.7 scrub ok 2024-09-20T22:09:02.612 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[29694]: pgmap v312: 158 pgs: 1 active+remapped+backfilling, 157 active+clean; 5.9 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 74 MiB/s wr, 1.17k op/s; 4988/289671 objects misplaced (1.722%); 9.7 MiB/s, 163 objects/s recovering 2024-09-20T22:09:02.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:09:02.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[29694]: 1.7 scrub starts 2024-09-20T22:09:02.613 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[29694]: 1.7 scrub ok 2024-09-20T22:09:02.613 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[40290]: 5.8 scrub starts 2024-09-20T22:09:02.613 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[40290]: 5.8 scrub ok 2024-09-20T22:09:02.613 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[40290]: 4.c scrub starts 2024-09-20T22:09:02.613 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[40290]: 4.c scrub ok 2024-09-20T22:09:02.613 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[40290]: 5.10 scrub starts 2024-09-20T22:09:02.613 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[40290]: 5.10 scrub ok 2024-09-20T22:09:02.613 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[40290]: 3.7 scrub starts 2024-09-20T22:09:02.613 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[40290]: 3.7 scrub ok 2024-09-20T22:09:02.613 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[40290]: pgmap v312: 158 pgs: 1 active+remapped+backfilling, 157 active+clean; 5.9 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 74 MiB/s wr, 1.17k op/s; 4988/289671 objects misplaced (1.722%); 9.7 MiB/s, 163 objects/s recovering 2024-09-20T22:09:02.614 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:09:02.614 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[40290]: 1.7 scrub starts 2024-09-20T22:09:02.614 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:02 smithi060 ceph-mon[40290]: 1.7 scrub ok 2024-09-20T22:09:02.618 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:02.635 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:02 smithi067 ceph-mon[33511]: 5.8 scrub starts 2024-09-20T22:09:02.635 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:02 smithi067 ceph-mon[33511]: 5.8 scrub ok 2024-09-20T22:09:02.635 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:02 smithi067 ceph-mon[33511]: 4.c scrub starts 2024-09-20T22:09:02.635 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:02 smithi067 ceph-mon[33511]: 4.c scrub ok 2024-09-20T22:09:02.635 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:02 smithi067 ceph-mon[33511]: 5.10 scrub starts 2024-09-20T22:09:02.635 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:02 smithi067 ceph-mon[33511]: 5.10 scrub ok 2024-09-20T22:09:02.636 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:02 smithi067 ceph-mon[33511]: 3.7 scrub starts 2024-09-20T22:09:02.636 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:02 smithi067 ceph-mon[33511]: 3.7 scrub ok 2024-09-20T22:09:02.636 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:02 smithi067 ceph-mon[33511]: pgmap v312: 158 pgs: 1 active+remapped+backfilling, 157 active+clean; 5.9 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 74 MiB/s wr, 1.17k op/s; 4988/289671 objects misplaced (1.722%); 9.7 MiB/s, 163 objects/s recovering 2024-09-20T22:09:02.636 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:02 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:09:02.636 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:02 smithi067 ceph-mon[33511]: 1.7 scrub starts 2024-09-20T22:09:02.636 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:02 smithi067 ceph-mon[33511]: 1.7 scrub ok 2024-09-20T22:09:02.652 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:09:02.652 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:09:02.657 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:02.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:02.658 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:02.685 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:09:02.686 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:09:02.719 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:09:02.719 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:09:02.754 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:09:02.754 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:09:02.759 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:02.790 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:09:02.790 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:09:02.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:02.807 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:02.808 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:02.828 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:09:02.828 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:09:02.909 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:02.922 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:09:02.922 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:09:02.945 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:02.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:02.946 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:03.047 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:03.075 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:03.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:03.076 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:03.147 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 66 16 99651 99635 94.3294 68.3125 0.0146831 0.0105962 2024-09-20T22:09:03.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:03.220 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:03.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:03.221 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:03.322 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:03.354 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:03.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:03.355 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:03.456 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:03.483 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:03.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:03.484 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:03.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:03 smithi060 ceph-mon[29694]: 3.19 scrub starts 2024-09-20T22:09:03.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:03 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1158057315' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:09:03.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:03 smithi060 ceph-mon[29694]: 3.19 scrub ok 2024-09-20T22:09:03.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:03 smithi060 ceph-mon[29694]: 3.3 scrub starts 2024-09-20T22:09:03.539 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:03 smithi060 ceph-mon[29694]: 3.3 scrub ok 2024-09-20T22:09:03.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:03 smithi060 ceph-mon[40290]: 3.19 scrub starts 2024-09-20T22:09:03.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:03 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1158057315' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:09:03.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:03 smithi060 ceph-mon[40290]: 3.19 scrub ok 2024-09-20T22:09:03.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:03 smithi060 ceph-mon[40290]: 3.3 scrub starts 2024-09-20T22:09:03.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:03 smithi060 ceph-mon[40290]: 3.3 scrub ok 2024-09-20T22:09:03.585 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:03.609 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:03.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:03.610 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:03.711 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:03.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:03 smithi067 ceph-mon[33511]: 3.19 scrub starts 2024-09-20T22:09:03.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:03 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1158057315' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:09:03.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:03 smithi067 ceph-mon[33511]: 3.19 scrub ok 2024-09-20T22:09:03.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:03 smithi067 ceph-mon[33511]: 3.3 scrub starts 2024-09-20T22:09:03.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:03 smithi067 ceph-mon[33511]: 3.3 scrub ok 2024-09-20T22:09:03.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:03.765 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:03.765 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:03.820 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T22:09:03.866 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:03.904 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:03.907 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:09:03.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:03.908 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:03.914 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:09:04.010 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:04.047 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:04.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:04.049 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:04.147 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 67 16 100751 100735 93.9476 68.75 0.0108182 0.0106396 2024-09-20T22:09:04.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:04.186 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:04.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:04.187 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:04.223 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:09:04.288 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:04.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:04.340 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:04.340 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:04.441 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:04.483 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:04.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:04.484 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:04.496 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:09:04.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:04 smithi060 ceph-mon[29694]: 6.1 scrub starts 2024-09-20T22:09:04.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:04 smithi060 ceph-mon[29694]: 6.1 scrub ok 2024-09-20T22:09:04.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:04 smithi060 ceph-mon[29694]: 4.1 scrub starts 2024-09-20T22:09:04.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:04 smithi060 ceph-mon[29694]: 4.1 scrub ok 2024-09-20T22:09:04.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:04 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3608236593' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:09:04.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:04 smithi060 ceph-mon[29694]: 3.e scrub starts 2024-09-20T22:09:04.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:04 smithi060 ceph-mon[29694]: 3.e scrub ok 2024-09-20T22:09:04.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:04 smithi060 ceph-mon[29694]: from='client.25163 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:09:04.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:04 smithi060 ceph-mon[29694]: pgmap v313: 158 pgs: 1 active+remapped+backfilling, 157 active+clean; 6.0 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 63 MiB/s wr, 998 op/s; 4988/293943 objects misplaced (1.697%); 8.0 MiB/s, 132 objects/s recovering 2024-09-20T22:09:04.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:04 smithi060 ceph-mon[40290]: 6.1 scrub starts 2024-09-20T22:09:04.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:04 smithi060 ceph-mon[40290]: 6.1 scrub ok 2024-09-20T22:09:04.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:04 smithi060 ceph-mon[40290]: 4.1 scrub starts 2024-09-20T22:09:04.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:04 smithi060 ceph-mon[40290]: 4.1 scrub ok 2024-09-20T22:09:04.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:04 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3608236593' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:09:04.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:04 smithi060 ceph-mon[40290]: 3.e scrub starts 2024-09-20T22:09:04.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:04 smithi060 ceph-mon[40290]: 3.e scrub ok 2024-09-20T22:09:04.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:04 smithi060 ceph-mon[40290]: from='client.25163 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:09:04.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:04 smithi060 ceph-mon[40290]: pgmap v313: 158 pgs: 1 active+remapped+backfilling, 157 active+clean; 6.0 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 63 MiB/s wr, 998 op/s; 4988/293943 objects misplaced (1.697%); 8.0 MiB/s, 132 objects/s recovering 2024-09-20T22:09:04.586 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:04.626 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:04.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:04.627 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:04.715 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:04 smithi067 ceph-mon[33511]: 6.1 scrub starts 2024-09-20T22:09:04.716 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:04 smithi067 ceph-mon[33511]: 6.1 scrub ok 2024-09-20T22:09:04.716 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:04 smithi067 ceph-mon[33511]: 4.1 scrub starts 2024-09-20T22:09:04.716 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:04 smithi067 ceph-mon[33511]: 4.1 scrub ok 2024-09-20T22:09:04.716 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:04 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3608236593' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:09:04.716 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:04 smithi067 ceph-mon[33511]: 3.e scrub starts 2024-09-20T22:09:04.716 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:04 smithi067 ceph-mon[33511]: 3.e scrub ok 2024-09-20T22:09:04.716 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:04 smithi067 ceph-mon[33511]: from='client.25163 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:09:04.716 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:04 smithi067 ceph-mon[33511]: pgmap v313: 158 pgs: 1 active+remapped+backfilling, 157 active+clean; 6.0 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 63 MiB/s wr, 998 op/s; 4988/293943 objects misplaced (1.697%); 8.0 MiB/s, 132 objects/s recovering 2024-09-20T22:09:04.729 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:04.764 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:04.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:04.765 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:04.866 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:04.880 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:09:04.880 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:09:04.906 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:04.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:04.907 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:05.007 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:05.048 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:05.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:05.049 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:05.147 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 68 16 101969 101953 93.6854 76.125 0.0136482 0.0106693 2024-09-20T22:09:05.150 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:05.192 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:05.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:05.193 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:05.218 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:09:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:09:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:09:05.271 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-20T22:09:02.042025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6315561859,"num_objects":96557,"num_object_clones":0,"num_object_copies":289671,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4988,"num_objects_unfound":0,"num_objects_dirty":96557,"num_whiteouts":0,"num_read":1200,"num_read_kb":1013,"num_write":96908,"num_write_kb":6168430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1104,"num_bytes_recovered":69679348,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78511,"ondisk_log_size":78511,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":20516476,"kb_used_data":18521980,"kb_used_omap":15,"kb_used_meta":1994288,"kb_avail":729444740,"statfs":{"total":767960285184,"available":746951413760,"internally_reserved":0,"allocated":18966507520,"data_stored":18959399961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":2042151699},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,7,4,3,5,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":511687016,"num_objects":7807,"num_object_clones":0,"num_object_copies":23421,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4988,"num_objects_unfound":0,"num_objects_dirty":7807,"num_whiteouts":0,"num_read":-63,"num_read_kb":-99,"num_write":7742,"num_write_kb":498705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1082,"num_bytes_recovered":67317546,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.619078"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":322,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.426632+0000","last_change":"2024-09-20T22:06:48.018571+0000","last_active":"2024-09-20T22:09:01.426632+0000","last_peered":"2024-09-20T22:09:01.426632+0000","last_clean":"2024-09-20T22:09:01.426632+0000","last_became_active":"2024-09-20T22:02:50.109930+0000","last_became_peered":"2024-09-20T22:02:50.109930+0000","last_unstale":"2024-09-20T22:09:01.426632+0000","last_undegraded":"2024-09-20T22:09:01.426632+0000","last_fullsized":"2024-09-20T22:09:01.426632+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_clean_scrub_stamp":"2024-09-20T22:06:48.018157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"71'9","reported_seq":342,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455249+0000","last_change":"2024-09-20T22:07:55.492953+0000","last_active":"2024-09-20T22:08:56.455249+0000","last_peered":"2024-09-20T22:08:56.455249+0000","last_clean":"2024-09-20T22:08:56.455249+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:08:56.455249+0000","last_undegraded":"2024-09-20T22:08:56.455249+0000","last_fullsized":"2024-09-20T22:08:56.455249+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:55.492841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.492841+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:24.417494+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":200,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.456511+0000","last_change":"2024-09-20T22:06:54.843328+0000","last_active":"2024-09-20T22:08:56.456511+0000","last_peered":"2024-09-20T22:08:56.456511+0000","last_clean":"2024-09-20T22:08:56.456511+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:08:56.456511+0000","last_undegraded":"2024-09-20T22:08:56.456511+0000","last_fullsized":"2024-09-20T22:08:56.456511+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:11.178591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:11.178591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.394594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":310,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.456567+0000","last_change":"2024-09-20T22:07:14.535725+0000","last_active":"2024-09-20T22:08:56.456567+0000","last_peered":"2024-09-20T22:08:56.456567+0000","last_clean":"2024-09-20T22:08:56.456567+0000","last_became_active":"2024-09-20T22:06:54.830769+0000","last_became_peered":"2024-09-20T22:06:54.830769+0000","last_unstale":"2024-09-20T22:08:56.456567+0000","last_undegraded":"2024-09-20T22:08:56.456567+0000","last_fullsized":"2024-09-20T22:08:56.456567+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.535618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.535618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:58.622589+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":309,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.449553+0000","last_change":"2024-09-20T22:08:41.770593+0000","last_active":"2024-09-20T22:08:56.449553+0000","last_peered":"2024-09-20T22:08:56.449553+0000","last_clean":"2024-09-20T22:08:56.449553+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:08:56.449553+0000","last_undegraded":"2024-09-20T22:08:56.449553+0000","last_fullsized":"2024-09-20T22:08:56.449553+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:08:41.770351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:41.770351+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:07.625199+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":321,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:09:00.839312+0000","last_change":"2024-09-20T22:09:00.839346+0000","last_active":"2024-09-20T22:09:00.839312+0000","last_peered":"2024-09-20T22:09:00.839312+0000","last_clean":"2024-09-20T22:09:00.839312+0000","last_became_active":"2024-09-20T22:02:50.107154+0000","last_became_peered":"2024-09-20T22:02:50.107154+0000","last_unstale":"2024-09-20T22:09:00.839312+0000","last_undegraded":"2024-09-20T22:09:00.839312+0000","last_fullsized":"2024-09-20T22:09:00.839312+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:00.839239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:00.839239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:07.151739+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"71'15","reported_seq":341,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.449821+0000","last_change":"2024-09-20T22:08:39.705641+0000","last_active":"2024-09-20T22:08:56.449821+0000","last_peered":"2024-09-20T22:08:56.449821+0000","last_clean":"2024-09-20T22:08:56.449821+0000","last_became_active":"2024-09-20T22:02:52.282101+0000","last_became_peered":"2024-09-20T22:02:52.282101+0000","last_unstale":"2024-09-20T22:08:56.449821+0000","last_undegraded":"2024-09-20T22:08:56.449821+0000","last_fullsized":"2024-09-20T22:08:56.449821+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:08:39.705510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:39.705510+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:08.518121+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":314,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455156+0000","last_change":"2024-09-20T22:07:17.494965+0000","last_active":"2024-09-20T22:08:56.455156+0000","last_peered":"2024-09-20T22:08:56.455156+0000","last_clean":"2024-09-20T22:08:56.455156+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:08:56.455156+0000","last_undegraded":"2024-09-20T22:08:56.455156+0000","last_fullsized":"2024-09-20T22:08:56.455156+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:17.494852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:17.494852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:58.622623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":336,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.447620+0000","last_change":"2024-09-20T22:08:35.146849+0000","last_active":"2024-09-20T22:08:56.447620+0000","last_peered":"2024-09-20T22:08:56.447620+0000","last_clean":"2024-09-20T22:08:56.447620+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:08:56.447620+0000","last_undegraded":"2024-09-20T22:08:56.447620+0000","last_fullsized":"2024-09-20T22:08:56.447620+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:08:35.146554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:50.212690+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"71'9","reported_seq":333,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.456866+0000","last_change":"2024-09-20T22:06:09.729638+0000","last_active":"2024-09-20T22:08:56.456866+0000","last_peered":"2024-09-20T22:08:56.456866+0000","last_clean":"2024-09-20T22:08:56.456866+0000","last_became_active":"2024-09-20T22:02:52.279526+0000","last_became_peered":"2024-09-20T22:02:52.279526+0000","last_unstale":"2024-09-20T22:08:56.456866+0000","last_undegraded":"2024-09-20T22:08:56.456866+0000","last_fullsized":"2024-09-20T22:08:56.456866+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:06:09.729513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:09.729513+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:58.622481+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":334,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.447047+0000","last_change":"2024-09-20T22:08:51.456300+0000","last_active":"2024-09-20T22:08:56.447047+0000","last_peered":"2024-09-20T22:08:56.447047+0000","last_clean":"2024-09-20T22:08:56.447047+0000","last_became_active":"2024-09-20T22:08:51.455924+0000","last_became_peered":"2024-09-20T22:08:51.455924+0000","last_unstale":"2024-09-20T22:08:56.447047+0000","last_undegraded":"2024-09-20T22:08:56.447047+0000","last_fullsized":"2024-09-20T22:08:56.447047+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:42.143066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:42.143066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.990159+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":311,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.426896+0000","last_change":"2024-09-20T22:07:15.416312+0000","last_active":"2024-09-20T22:09:01.426896+0000","last_peered":"2024-09-20T22:09:01.426896+0000","last_clean":"2024-09-20T22:09:01.426896+0000","last_became_active":"2024-09-20T22:06:54.841095+0000","last_became_peered":"2024-09-20T22:06:54.841095+0000","last_unstale":"2024-09-20T22:09:01.426896+0000","last_undegraded":"2024-09-20T22:09:01.426896+0000","last_fullsized":"2024-09-20T22:09:01.426896+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:15.416176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.416176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":341,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.447089+0000","last_change":"2024-09-20T22:07:59.947914+0000","last_active":"2024-09-20T22:08:56.447089+0000","last_peered":"2024-09-20T22:08:56.447089+0000","last_clean":"2024-09-20T22:08:56.447089+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:08:56.447089+0000","last_undegraded":"2024-09-20T22:08:56.447089+0000","last_fullsized":"2024-09-20T22:08:56.447089+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:59.947741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:59.947741+0000","last_clean_scrub_stamp":"2024-09-20T22:07:59.947741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:29.533391+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"71'5","reported_seq":348,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.456093+0000","last_change":"2024-09-20T22:08:22.151586+0000","last_active":"2024-09-20T22:08:56.456093+0000","last_peered":"2024-09-20T22:08:56.456093+0000","last_clean":"2024-09-20T22:08:56.456093+0000","last_became_active":"2024-09-20T22:08:22.144364+0000","last_became_peered":"2024-09-20T22:08:22.144364+0000","last_unstale":"2024-09-20T22:08:56.456093+0000","last_undegraded":"2024-09-20T22:08:56.456093+0000","last_fullsized":"2024-09-20T22:08:56.456093+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'5","last_scrub_stamp":"2024-09-20T22:08:01.462581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:01.462581+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:24.969371+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":322,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.454310+0000","last_change":"2024-09-20T22:07:28.654301+0000","last_active":"2024-09-20T22:08:56.454310+0000","last_peered":"2024-09-20T22:08:56.454310+0000","last_clean":"2024-09-20T22:08:56.454310+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:08:56.454310+0000","last_undegraded":"2024-09-20T22:08:56.454310+0000","last_fullsized":"2024-09-20T22:08:56.454310+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:28.654202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:28.654202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":264,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.422045+0000","last_change":"2024-09-20T22:08:41.836934+0000","last_active":"2024-09-20T22:09:01.422045+0000","last_peered":"2024-09-20T22:09:01.422045+0000","last_clean":"2024-09-20T22:09:01.422045+0000","last_became_active":"2024-09-20T22:06:55.824168+0000","last_became_peered":"2024-09-20T22:06:55.824168+0000","last_unstale":"2024-09-20T22:09:01.422045+0000","last_undegraded":"2024-09-20T22:09:01.422045+0000","last_fullsized":"2024-09-20T22:09:01.422045+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:41.836827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:08:41.836827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:04.667178+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":310,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.456509+0000","last_change":"2024-09-20T22:06:47.667293+0000","last_active":"2024-09-20T22:08:56.456509+0000","last_peered":"2024-09-20T22:08:56.456509+0000","last_clean":"2024-09-20T22:08:56.456509+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:08:56.456509+0000","last_undegraded":"2024-09-20T22:08:56.456509+0000","last_fullsized":"2024-09-20T22:08:56.456509+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:06:47.667180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":322,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.422179+0000","last_change":"2024-09-20T22:08:34.815041+0000","last_active":"2024-09-20T22:09:01.422179+0000","last_peered":"2024-09-20T22:09:01.422179+0000","last_clean":"2024-09-20T22:09:01.422179+0000","last_became_active":"2024-09-20T22:02:50.111517+0000","last_became_peered":"2024-09-20T22:02:50.111517+0000","last_unstale":"2024-09-20T22:09:01.422179+0000","last_undegraded":"2024-09-20T22:09:01.422179+0000","last_fullsized":"2024-09-20T22:09:01.422179+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:08:34.814930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:42.718933+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"195'22","reported_seq":364,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.464613+0000","last_change":"2024-09-20T22:08:39.358759+0000","last_active":"2024-09-20T22:08:56.464613+0000","last_peered":"2024-09-20T22:08:56.464613+0000","last_clean":"2024-09-20T22:08:56.464613+0000","last_became_active":"2024-09-20T22:06:54.846111+0000","last_became_peered":"2024-09-20T22:06:54.846111+0000","last_unstale":"2024-09-20T22:08:56.464613+0000","last_undegraded":"2024-09-20T22:08:56.464613+0000","last_fullsized":"2024-09-20T22:08:56.464613+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'22","last_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:08:39.355859+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:53.211286+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":158,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.458809+0000","last_change":"2024-09-20T22:08:00.369883+0000","last_active":"2024-09-20T22:08:56.458809+0000","last_peered":"2024-09-20T22:08:56.458809+0000","last_clean":"2024-09-20T22:08:56.458809+0000","last_became_active":"2024-09-20T22:06:54.848505+0000","last_became_peered":"2024-09-20T22:06:54.848505+0000","last_unstale":"2024-09-20T22:08:56.458809+0000","last_undegraded":"2024-09-20T22:08:56.458809+0000","last_fullsized":"2024-09-20T22:08:56.458809+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:00.369767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:00.369767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:20.935739+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":308,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.454869+0000","last_change":"2024-09-20T22:06:34.944093+0000","last_active":"2024-09-20T22:08:56.454869+0000","last_peered":"2024-09-20T22:08:56.454869+0000","last_clean":"2024-09-20T22:08:56.454869+0000","last_became_active":"2024-09-20T22:02:56.347561+0000","last_became_peered":"2024-09-20T22:02:56.347561+0000","last_unstale":"2024-09-20T22:08:56.454869+0000","last_undegraded":"2024-09-20T22:08:56.454869+0000","last_fullsized":"2024-09-20T22:08:56.454869+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:34.940758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:34.940758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:00.825125+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":270,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.447943+0000","last_change":"2024-09-20T22:08:28.058586+0000","last_active":"2024-09-20T22:08:56.447943+0000","last_peered":"2024-09-20T22:08:56.447943+0000","last_clean":"2024-09-20T22:08:56.447943+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:08:56.447943+0000","last_undegraded":"2024-09-20T22:08:56.447943+0000","last_fullsized":"2024-09-20T22:08:56.447943+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:28.058516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:28.058516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:50.545327+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"71'12","reported_seq":340,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.452291+0000","last_change":"2024-09-20T22:07:55.383903+0000","last_active":"2024-09-20T22:08:56.452291+0000","last_peered":"2024-09-20T22:08:56.452291+0000","last_clean":"2024-09-20T22:08:56.452291+0000","last_became_active":"2024-09-20T22:06:54.830946+0000","last_became_peered":"2024-09-20T22:06:54.830946+0000","last_unstale":"2024-09-20T22:08:56.452291+0000","last_undegraded":"2024-09-20T22:08:56.452291+0000","last_fullsized":"2024-09-20T22:08:56.452291+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:55.383818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.383818+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":313,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455231+0000","last_change":"2024-09-20T22:08:36.461149+0000","last_active":"2024-09-20T22:08:56.455231+0000","last_peered":"2024-09-20T22:08:56.455231+0000","last_clean":"2024-09-20T22:08:56.455231+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:08:56.455231+0000","last_undegraded":"2024-09-20T22:08:56.455231+0000","last_fullsized":"2024-09-20T22:08:56.455231+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:36.460897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:36.460897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:51.153875+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":311,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.454229+0000","last_change":"2024-09-20T22:08:34.802125+0000","last_active":"2024-09-20T22:08:56.454229+0000","last_peered":"2024-09-20T22:08:56.454229+0000","last_clean":"2024-09-20T22:08:56.454229+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:08:56.454229+0000","last_undegraded":"2024-09-20T22:08:56.454229+0000","last_fullsized":"2024-09-20T22:08:56.454229+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:08:34.801894+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:08:34.801894+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:01.262234+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"64'1","reported_seq":151,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.456587+0000","last_change":"2024-09-20T22:06:54.854003+0000","last_active":"2024-09-20T22:08:56.456587+0000","last_peered":"2024-09-20T22:08:56.456587+0000","last_clean":"2024-09-20T22:08:56.456587+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:08:56.456587+0000","last_undegraded":"2024-09-20T22:08:56.456587+0000","last_fullsized":"2024-09-20T22:08:56.456587+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:02:49.083871+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.394492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"71'10","reported_seq":351,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.430296+0000","last_change":"2024-09-20T22:08:52.434346+0000","last_active":"2024-09-20T22:09:01.430296+0000","last_peered":"2024-09-20T22:09:01.430296+0000","last_clean":"2024-09-20T22:09:01.430296+0000","last_became_active":"2024-09-20T22:08:52.424697+0000","last_became_peered":"2024-09-20T22:08:52.424697+0000","last_unstale":"2024-09-20T22:09:01.430296+0000","last_undegraded":"2024-09-20T22:09:01.430296+0000","last_fullsized":"2024-09-20T22:09:01.430296+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:08:30.868781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:30.868781+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:40.662035+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"208'9","reported_seq":416,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.450192+0000","last_change":"2024-09-20T22:08:51.459956+0000","last_active":"2024-09-20T22:08:56.450192+0000","last_peered":"2024-09-20T22:08:56.450192+0000","last_clean":"2024-09-20T22:08:56.450192+0000","last_became_active":"2024-09-20T22:08:51.448751+0000","last_became_peered":"2024-09-20T22:08:51.448751+0000","last_unstale":"2024-09-20T22:08:56.450192+0000","last_undegraded":"2024-09-20T22:08:56.450192+0000","last_fullsized":"2024-09-20T22:08:56.450192+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:08:29.376948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:29.376948+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:41.132625+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":316,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.447838+0000","last_change":"2024-09-20T22:08:48.453361+0000","last_active":"2024-09-20T22:08:56.447838+0000","last_peered":"2024-09-20T22:08:56.447838+0000","last_clean":"2024-09-20T22:08:56.447838+0000","last_became_active":"2024-09-20T22:08:48.452786+0000","last_became_peered":"2024-09-20T22:08:48.452786+0000","last_unstale":"2024-09-20T22:08:56.447838+0000","last_undegraded":"2024-09-20T22:08:56.447838+0000","last_fullsized":"2024-09-20T22:08:56.447838+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:07:53.673691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.673691+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:00.053536+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53669,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":26,"num_write":16,"num_write_kb":219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":161009,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3","version":"210'5884","reported_seq":5961,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:57.718666+0000","last_change":"2024-09-20T22:07:52.429150+0000","last_active":"2024-09-20T22:08:57.718666+0000","last_peered":"2024-09-20T22:08:57.718666+0000","last_clean":"2024-09-20T22:08:57.718666+0000","last_became_active":"2024-09-20T22:07:52.428105+0000","last_became_peered":"2024-09-20T22:07:52.428105+0000","last_unstale":"2024-09-20T22:08:57.718666+0000","last_undegraded":"2024-09-20T22:08:57.718666+0000","last_fullsized":"2024-09-20T22:08:57.718666+0000","mapping_epoch":176,"log_start":"182'1486","ondisk_log_start":"182'1486","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4398,"log_dups_size":0,"ondisk_log_size":4398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:04.623212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":385613824,"num_objects":5884,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5884,"num_write_kb":376576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"71'15","reported_seq":79,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.463484+0000","last_change":"2024-09-20T22:08:30.118912+0000","last_active":"2024-09-20T22:08:56.463484+0000","last_peered":"2024-09-20T22:08:56.463484+0000","last_clean":"2024-09-20T22:08:56.463484+0000","last_became_active":"2024-09-20T22:08:22.145408+0000","last_became_peered":"2024-09-20T22:08:22.145408+0000","last_unstale":"2024-09-20T22:08:56.463484+0000","last_undegraded":"2024-09-20T22:08:56.463484+0000","last_fullsized":"2024-09-20T22:08:56.463484+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:08:30.118638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:30.118638+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:30.162802+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":321,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.449855+0000","last_change":"2024-09-20T22:07:23.292560+0000","last_active":"2024-09-20T22:08:56.449855+0000","last_peered":"2024-09-20T22:08:56.449855+0000","last_clean":"2024-09-20T22:08:56.449855+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:08:56.449855+0000","last_undegraded":"2024-09-20T22:08:56.449855+0000","last_fullsized":"2024-09-20T22:08:56.449855+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.292451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.292451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:00.825236+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"71'8","reported_seq":399,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.456598+0000","last_change":"2024-09-20T22:06:04.874166+0000","last_active":"2024-09-20T22:08:56.456598+0000","last_peered":"2024-09-20T22:08:56.456598+0000","last_clean":"2024-09-20T22:08:56.456598+0000","last_became_active":"2024-09-20T22:02:54.294917+0000","last_became_peered":"2024-09-20T22:02:54.294917+0000","last_unstale":"2024-09-20T22:08:56.456598+0000","last_undegraded":"2024-09-20T22:08:56.456598+0000","last_fullsized":"2024-09-20T22:08:56.456598+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:06:04.874042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:04.874042+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":271,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.449961+0000","last_change":"2024-09-20T22:07:52.334831+0000","last_active":"2024-09-20T22:08:56.449961+0000","last_peered":"2024-09-20T22:08:56.449961+0000","last_clean":"2024-09-20T22:08:56.449961+0000","last_became_active":"2024-09-20T22:06:55.822780+0000","last_became_peered":"2024-09-20T22:06:55.822780+0000","last_unstale":"2024-09-20T22:08:56.449961+0000","last_undegraded":"2024-09-20T22:08:56.449961+0000","last_fullsized":"2024-09-20T22:08:56.449961+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:52.334748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:52.334748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:01.399176+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":324,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:59.562970+0000","last_change":"2024-09-20T22:08:59.563006+0000","last_active":"2024-09-20T22:08:59.562970+0000","last_peered":"2024-09-20T22:08:59.562970+0000","last_clean":"2024-09-20T22:08:59.562970+0000","last_became_active":"2024-09-20T22:08:48.452475+0000","last_became_peered":"2024-09-20T22:08:48.452475+0000","last_unstale":"2024-09-20T22:08:59.562970+0000","last_undegraded":"2024-09-20T22:08:59.562970+0000","last_fullsized":"2024-09-20T22:08:59.562970+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:08:59.562897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:08:59.562897+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:26.912800+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"211'6083","reported_seq":8252,"reported_epoch":211,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T22:09:01.461530+0000","last_change":"2024-09-20T22:08:53.752102+0000","last_active":"2024-09-20T22:09:01.461530+0000","last_peered":"2024-09-20T22:09:01.461530+0000","last_clean":"2024-09-20T22:08:51.290755+0000","last_became_active":"2024-09-20T22:08:53.741655+0000","last_became_peered":"2024-09-20T22:08:53.741655+0000","last_unstale":"2024-09-20T22:09:01.461530+0000","last_undegraded":"2024-09-20T22:09:01.461530+0000","last_fullsized":"2024-09-20T22:09:01.461530+0000","mapping_epoch":206,"log_start":"182'1083","ondisk_log_start":"182'1083","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":5000,"log_dups_size":0,"ondisk_log_size":5000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398655488,"num_objects":6083,"num_object_clones":0,"num_object_copies":18249,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4988,"num_objects_unfound":0,"num_objects_dirty":6083,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6083,"num_write_kb":389312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1042,"num_bytes_recovered":68288512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,7],"avail_no_missing":["0","3","7"],"object_location_counts":[{"shards":"0,3,7","objects":6083}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"71'18","reported_seq":372,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.449618+0000","last_change":"2024-09-20T22:08:52.436122+0000","last_active":"2024-09-20T22:08:56.449618+0000","last_peered":"2024-09-20T22:08:56.449618+0000","last_clean":"2024-09-20T22:08:56.449618+0000","last_became_active":"2024-09-20T22:08:52.421420+0000","last_became_peered":"2024-09-20T22:08:52.421420+0000","last_unstale":"2024-09-20T22:08:56.449618+0000","last_undegraded":"2024-09-20T22:08:56.449618+0000","last_fullsized":"2024-09-20T22:08:56.449618+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'18","last_scrub_stamp":"2024-09-20T22:08:40.737338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:40.737338+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:48.648335+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":311,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455357+0000","last_change":"2024-09-20T22:05:02.924002+0000","last_active":"2024-09-20T22:08:56.455357+0000","last_peered":"2024-09-20T22:08:56.455357+0000","last_clean":"2024-09-20T22:08:56.455357+0000","last_became_active":"2024-09-20T22:02:50.113076+0000","last_became_peered":"2024-09-20T22:02:50.113076+0000","last_unstale":"2024-09-20T22:08:56.455357+0000","last_undegraded":"2024-09-20T22:08:56.455357+0000","last_fullsized":"2024-09-20T22:08:56.455357+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:02.923565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:02.923565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:00.825013+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":319,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455291+0000","last_change":"2024-09-20T22:08:51.455584+0000","last_active":"2024-09-20T22:08:56.455291+0000","last_peered":"2024-09-20T22:08:56.455291+0000","last_clean":"2024-09-20T22:08:56.455291+0000","last_became_active":"2024-09-20T22:08:51.455159+0000","last_became_peered":"2024-09-20T22:08:51.455159+0000","last_unstale":"2024-09-20T22:08:56.455291+0000","last_undegraded":"2024-09-20T22:08:56.455291+0000","last_fullsized":"2024-09-20T22:08:56.455291+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:05.894031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:05.894031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:58.622444+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":317,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.456775+0000","last_change":"2024-09-20T22:08:38.488454+0000","last_active":"2024-09-20T22:08:56.456775+0000","last_peered":"2024-09-20T22:08:56.456775+0000","last_clean":"2024-09-20T22:08:56.456775+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:08:56.456775+0000","last_undegraded":"2024-09-20T22:08:56.456775+0000","last_fullsized":"2024-09-20T22:08:56.456775+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:38.488310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:38.488310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.111826+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":324,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:58.858236+0000","last_change":"2024-09-20T22:08:58.858275+0000","last_active":"2024-09-20T22:08:58.858236+0000","last_peered":"2024-09-20T22:08:58.858236+0000","last_clean":"2024-09-20T22:08:58.858236+0000","last_became_active":"2024-09-20T22:08:48.476212+0000","last_became_peered":"2024-09-20T22:08:48.476212+0000","last_unstale":"2024-09-20T22:08:58.858236+0000","last_undegraded":"2024-09-20T22:08:58.858236+0000","last_fullsized":"2024-09-20T22:08:58.858236+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:08:58.858164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:08:58.858164+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:01.714721+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"210'5993","reported_seq":6071,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:09:00.662013+0000","last_change":"2024-09-20T22:07:52.940943+0000","last_active":"2024-09-20T22:09:00.662013+0000","last_peered":"2024-09-20T22:09:00.662013+0000","last_clean":"2024-09-20T22:09:00.662013+0000","last_became_active":"2024-09-20T22:07:52.940627+0000","last_became_peered":"2024-09-20T22:07:52.940627+0000","last_unstale":"2024-09-20T22:09:00.662013+0000","last_undegraded":"2024-09-20T22:09:00.662013+0000","last_fullsized":"2024-09-20T22:09:00.662013+0000","mapping_epoch":176,"log_start":"182'1178","ondisk_log_start":"182'1178","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4815,"log_dups_size":0,"ondisk_log_size":4815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:11.794238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":392757248,"num_objects":5993,"num_object_clones":0,"num_object_copies":17979,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5993,"num_write_kb":383552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"71'14","reported_seq":338,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455565+0000","last_change":"2024-09-20T22:06:00.944172+0000","last_active":"2024-09-20T22:08:56.455565+0000","last_peered":"2024-09-20T22:08:56.455565+0000","last_clean":"2024-09-20T22:08:56.455565+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:08:56.455565+0000","last_undegraded":"2024-09-20T22:08:56.455565+0000","last_fullsized":"2024-09-20T22:08:56.455565+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'14","last_scrub_stamp":"2024-09-20T22:06:00.944086+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.944086+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:58.622427+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":324,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.430573+0000","last_change":"2024-09-20T22:07:58.783500+0000","last_active":"2024-09-20T22:09:01.430573+0000","last_peered":"2024-09-20T22:09:01.430573+0000","last_clean":"2024-09-20T22:09:01.430573+0000","last_became_active":"2024-09-20T22:06:54.841165+0000","last_became_peered":"2024-09-20T22:06:54.841165+0000","last_unstale":"2024-09-20T22:09:01.430573+0000","last_undegraded":"2024-09-20T22:09:01.430573+0000","last_fullsized":"2024-09-20T22:09:01.430573+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:07:58.783380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:58.783380+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:25.086988+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"146'10","reported_seq":399,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:09:00.241857+0000","last_change":"2024-09-20T22:07:24.333937+0000","last_active":"2024-09-20T22:09:00.241857+0000","last_peered":"2024-09-20T22:09:00.241857+0000","last_clean":"2024-09-20T22:09:00.241857+0000","last_became_active":"2024-09-20T22:06:54.847742+0000","last_became_peered":"2024-09-20T22:06:54.847742+0000","last_unstale":"2024-09-20T22:09:00.241857+0000","last_undegraded":"2024-09-20T22:09:00.241857+0000","last_fullsized":"2024-09-20T22:09:00.241857+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:07:24.333809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:24.333809+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":309,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.454416+0000","last_change":"2024-09-20T22:07:30.589937+0000","last_active":"2024-09-20T22:08:56.454416+0000","last_peered":"2024-09-20T22:08:56.454416+0000","last_clean":"2024-09-20T22:08:56.454416+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:08:56.454416+0000","last_undegraded":"2024-09-20T22:08:56.454416+0000","last_fullsized":"2024-09-20T22:08:56.454416+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:30.589849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:30.589849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:57.117488+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":329,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.449935+0000","last_change":"2024-09-20T22:08:55.450144+0000","last_active":"2024-09-20T22:08:56.449935+0000","last_peered":"2024-09-20T22:08:56.449935+0000","last_clean":"2024-09-20T22:08:56.449935+0000","last_became_active":"2024-09-20T22:08:55.449793+0000","last_became_peered":"2024-09-20T22:08:55.449793+0000","last_unstale":"2024-09-20T22:08:56.449935+0000","last_undegraded":"2024-09-20T22:08:56.449935+0000","last_fullsized":"2024-09-20T22:08:56.449935+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:07:53.673691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.673691+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:01.399078+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":27,"num_write":16,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":161009,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"211'6178","reported_seq":6258,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.421704+0000","last_change":"2024-09-20T22:07:52.940978+0000","last_active":"2024-09-20T22:09:01.421704+0000","last_peered":"2024-09-20T22:09:01.421704+0000","last_clean":"2024-09-20T22:09:01.421704+0000","last_became_active":"2024-09-20T22:07:52.940673+0000","last_became_peered":"2024-09-20T22:07:52.940673+0000","last_unstale":"2024-09-20T22:09:01.421704+0000","last_undegraded":"2024-09-20T22:09:01.421704+0000","last_fullsized":"2024-09-20T22:09:01.421704+0000","mapping_epoch":176,"log_start":"182'1222","ondisk_log_start":"182'1222","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4956,"log_dups_size":0,"ondisk_log_size":4956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.115401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":404881408,"num_objects":6178,"num_object_clones":0,"num_object_copies":18534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6178,"num_write_kb":395392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"71'10","reported_seq":185,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.461545+0000","last_change":"2024-09-20T22:07:59.424108+0000","last_active":"2024-09-20T22:08:56.461545+0000","last_peered":"2024-09-20T22:08:56.461545+0000","last_clean":"2024-09-20T22:08:56.461545+0000","last_became_active":"2024-09-20T22:06:54.844681+0000","last_became_peered":"2024-09-20T22:06:54.844681+0000","last_unstale":"2024-09-20T22:08:56.461545+0000","last_undegraded":"2024-09-20T22:08:56.461545+0000","last_fullsized":"2024-09-20T22:08:56.461545+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:07:59.423970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:59.423970+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:05.043224+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":336,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.447716+0000","last_change":"2024-09-20T22:08:29.034890+0000","last_active":"2024-09-20T22:08:56.447716+0000","last_peered":"2024-09-20T22:08:56.447716+0000","last_clean":"2024-09-20T22:08:56.447716+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:08:56.447716+0000","last_undegraded":"2024-09-20T22:08:56.447716+0000","last_fullsized":"2024-09-20T22:08:56.447716+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:29.034641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:08:29.034641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:32.281368+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"71'8","reported_seq":397,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.421563+0000","last_change":"2024-09-20T22:08:28.861124+0000","last_active":"2024-09-20T22:09:01.421563+0000","last_peered":"2024-09-20T22:09:01.421563+0000","last_clean":"2024-09-20T22:09:01.421563+0000","last_became_active":"2024-09-20T22:02:54.292819+0000","last_became_peered":"2024-09-20T22:02:54.292819+0000","last_unstale":"2024-09-20T22:09:01.421563+0000","last_undegraded":"2024-09-20T22:09:01.421563+0000","last_fullsized":"2024-09-20T22:09:01.421563+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:08:28.860982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:28.860982+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:39.280471+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":303,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.427086+0000","last_change":"2024-09-20T22:06:00.572347+0000","last_active":"2024-09-20T22:09:01.427086+0000","last_peered":"2024-09-20T22:09:01.427086+0000","last_clean":"2024-09-20T22:09:01.427086+0000","last_became_active":"2024-09-20T22:02:56.304274+0000","last_became_peered":"2024-09-20T22:02:56.304274+0000","last_unstale":"2024-09-20T22:09:01.427086+0000","last_undegraded":"2024-09-20T22:09:01.427086+0000","last_fullsized":"2024-09-20T22:09:01.427086+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:00.572243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.572243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":329,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:09:00.372445+0000","last_change":"2024-09-20T22:09:00.372470+0000","last_active":"2024-09-20T22:09:00.372445+0000","last_peered":"2024-09-20T22:09:00.372445+0000","last_clean":"2024-09-20T22:09:00.372445+0000","last_became_active":"2024-09-20T22:08:48.466328+0000","last_became_peered":"2024-09-20T22:08:48.466328+0000","last_unstale":"2024-09-20T22:09:00.372445+0000","last_undegraded":"2024-09-20T22:09:00.372445+0000","last_fullsized":"2024-09-20T22:09:00.372445+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:09:00.372393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:09:00.372393+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:28.799282+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.f","version":"210'6179","reported_seq":6257,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.178746+0000","last_change":"2024-09-20T22:07:52.940027+0000","last_active":"2024-09-20T22:09:01.178746+0000","last_peered":"2024-09-20T22:09:01.178746+0000","last_clean":"2024-09-20T22:09:01.178746+0000","last_became_active":"2024-09-20T22:07:52.939692+0000","last_became_peered":"2024-09-20T22:07:52.939692+0000","last_unstale":"2024-09-20T22:09:01.178746+0000","last_undegraded":"2024-09-20T22:09:01.178746+0000","last_fullsized":"2024-09-20T22:09:01.178746+0000","mapping_epoch":176,"log_start":"182'1370","ondisk_log_start":"182'1370","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4809,"log_dups_size":0,"ondisk_log_size":4809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:12.848256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":404946944,"num_objects":6179,"num_object_clones":0,"num_object_copies":18537,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6179,"num_write_kb":395456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"71'19","reported_seq":362,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.421772+0000","last_change":"2024-09-20T22:08:33.852966+0000","last_active":"2024-09-20T22:09:01.421772+0000","last_peered":"2024-09-20T22:09:01.421772+0000","last_clean":"2024-09-20T22:09:01.421772+0000","last_became_active":"2024-09-20T22:02:52.480077+0000","last_became_peered":"2024-09-20T22:02:52.480077+0000","last_unstale":"2024-09-20T22:09:01.421772+0000","last_undegraded":"2024-09-20T22:09:01.421772+0000","last_fullsized":"2024-09-20T22:09:01.421772+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:08:33.852806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:33.852806+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:59.554741+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":160,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.458513+0000","last_change":"2024-09-20T22:07:14.382822+0000","last_active":"2024-09-20T22:08:56.458513+0000","last_peered":"2024-09-20T22:08:56.458513+0000","last_clean":"2024-09-20T22:08:56.458513+0000","last_became_active":"2024-09-20T22:06:54.829037+0000","last_became_peered":"2024-09-20T22:06:54.829037+0000","last_unstale":"2024-09-20T22:08:56.458513+0000","last_undegraded":"2024-09-20T22:08:56.458513+0000","last_fullsized":"2024-09-20T22:08:56.458513+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.382712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.382712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.394667+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":320,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.449236+0000","last_change":"2024-09-20T22:07:30.169138+0000","last_active":"2024-09-20T22:08:56.449236+0000","last_peered":"2024-09-20T22:08:56.449236+0000","last_clean":"2024-09-20T22:08:56.449236+0000","last_became_active":"2024-09-20T22:02:54.290787+0000","last_became_peered":"2024-09-20T22:02:54.290787+0000","last_unstale":"2024-09-20T22:08:56.449236+0000","last_undegraded":"2024-09-20T22:08:56.449236+0000","last_fullsized":"2024-09-20T22:08:56.449236+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:30.169038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:30.169038+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:01.399110+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":150,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.456421+0000","last_change":"2024-09-20T22:06:54.820385+0000","last_active":"2024-09-20T22:08:56.456421+0000","last_peered":"2024-09-20T22:08:56.456421+0000","last_clean":"2024-09-20T22:08:56.456421+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:08:56.456421+0000","last_undegraded":"2024-09-20T22:08:56.456421+0000","last_fullsized":"2024-09-20T22:08:56.456421+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:02:55.278800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.394517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":316,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.446361+0000","last_change":"2024-09-20T22:08:48.452711+0000","last_active":"2024-09-20T22:08:56.446361+0000","last_peered":"2024-09-20T22:08:56.446361+0000","last_clean":"2024-09-20T22:08:56.446361+0000","last_became_active":"2024-09-20T22:08:48.452234+0000","last_became_peered":"2024-09-20T22:08:48.452234+0000","last_unstale":"2024-09-20T22:08:56.446361+0000","last_undegraded":"2024-09-20T22:08:56.446361+0000","last_fullsized":"2024-09-20T22:08:56.446361+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:07:53.673691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.673691+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:00.053505+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":27,"num_write":17,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":161010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"210'6107","reported_seq":6185,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:09:00.634241+0000","last_change":"2024-09-20T22:07:52.674766+0000","last_active":"2024-09-20T22:09:00.634241+0000","last_peered":"2024-09-20T22:09:00.634241+0000","last_clean":"2024-09-20T22:09:00.634241+0000","last_became_active":"2024-09-20T22:07:52.674363+0000","last_became_peered":"2024-09-20T22:07:52.674363+0000","last_unstale":"2024-09-20T22:09:00.634241+0000","last_undegraded":"2024-09-20T22:09:00.634241+0000","last_fullsized":"2024-09-20T22:09:00.634241+0000","mapping_epoch":176,"log_start":"182'1259","ondisk_log_start":"182'1259","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4848,"log_dups_size":0,"ondisk_log_size":4848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:16.336670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":400228352,"num_objects":6107,"num_object_clones":0,"num_object_copies":18321,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6107,"num_write_kb":390848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"146'30","reported_seq":246,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.458637+0000","last_change":"2024-09-20T22:07:55.357829+0000","last_active":"2024-09-20T22:08:56.458637+0000","last_peered":"2024-09-20T22:08:56.458637+0000","last_clean":"2024-09-20T22:08:56.458637+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:08:56.458637+0000","last_undegraded":"2024-09-20T22:08:56.458637+0000","last_fullsized":"2024-09-20T22:08:56.458637+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'30","last_scrub_stamp":"2024-09-20T22:07:55.357726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.357726+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:10.172184+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":321,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.457183+0000","last_change":"2024-09-20T22:05:58.966616+0000","last_active":"2024-09-20T22:08:56.457183+0000","last_peered":"2024-09-20T22:08:56.457183+0000","last_clean":"2024-09-20T22:08:56.457183+0000","last_became_active":"2024-09-20T22:02:50.108457+0000","last_became_peered":"2024-09-20T22:02:50.108457+0000","last_unstale":"2024-09-20T22:08:56.457183+0000","last_undegraded":"2024-09-20T22:08:56.457183+0000","last_fullsized":"2024-09-20T22:08:56.457183+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:58.966541+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:05:58.966541+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:58.622407+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":332,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.448445+0000","last_change":"2024-09-20T22:07:21.886172+0000","last_active":"2024-09-20T22:08:56.448445+0000","last_peered":"2024-09-20T22:08:56.448445+0000","last_clean":"2024-09-20T22:08:56.448445+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:08:56.448445+0000","last_undegraded":"2024-09-20T22:08:56.448445+0000","last_fullsized":"2024-09-20T22:08:56.448445+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_clean_scrub_stamp":"2024-09-20T22:07:21.886038+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":315,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.425830+0000","last_change":"2024-09-20T22:07:17.566774+0000","last_active":"2024-09-20T22:09:01.425830+0000","last_peered":"2024-09-20T22:09:01.425830+0000","last_clean":"2024-09-20T22:09:01.425830+0000","last_became_active":"2024-09-20T22:06:54.841419+0000","last_became_peered":"2024-09-20T22:06:54.841419+0000","last_unstale":"2024-09-20T22:09:01.425830+0000","last_undegraded":"2024-09-20T22:09:01.425830+0000","last_fullsized":"2024-09-20T22:09:01.425830+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:17.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:17.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":120,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":300,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.450522+0000","last_change":"2024-09-20T22:05:34.954357+0000","last_active":"2024-09-20T22:08:56.450522+0000","last_peered":"2024-09-20T22:08:56.450522+0000","last_clean":"2024-09-20T22:08:56.450522+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:08:56.450522+0000","last_undegraded":"2024-09-20T22:08:56.450522+0000","last_fullsized":"2024-09-20T22:08:56.450522+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:05:34.953944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:00.825040+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":312,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.421802+0000","last_change":"2024-09-20T22:08:55.444848+0000","last_active":"2024-09-20T22:09:01.421802+0000","last_peered":"2024-09-20T22:09:01.421802+0000","last_clean":"2024-09-20T22:09:01.421802+0000","last_became_active":"2024-09-20T22:08:55.444469+0000","last_became_peered":"2024-09-20T22:08:55.444469+0000","last_unstale":"2024-09-20T22:09:01.421802+0000","last_undegraded":"2024-09-20T22:09:01.421802+0000","last_fullsized":"2024-09-20T22:09:01.421802+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:07:53.673691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.673691+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:05.868712+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":27,"num_write":17,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":161010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"210'5979","reported_seq":6057,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.165328+0000","last_change":"2024-09-20T22:07:52.429893+0000","last_active":"2024-09-20T22:09:01.165328+0000","last_peered":"2024-09-20T22:09:01.165328+0000","last_clean":"2024-09-20T22:09:01.165328+0000","last_became_active":"2024-09-20T22:07:52.429554+0000","last_became_peered":"2024-09-20T22:07:52.429554+0000","last_unstale":"2024-09-20T22:09:01.165328+0000","last_undegraded":"2024-09-20T22:09:01.165328+0000","last_fullsized":"2024-09-20T22:09:01.165328+0000","mapping_epoch":176,"log_start":"182'1197","ondisk_log_start":"182'1197","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4782,"log_dups_size":0,"ondisk_log_size":4782,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:20.427594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":391839744,"num_objects":5979,"num_object_clones":0,"num_object_copies":17937,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5979,"num_write_kb":382656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"71'13","reported_seq":196,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.459861+0000","last_change":"2024-09-20T22:08:35.413541+0000","last_active":"2024-09-20T22:08:56.459861+0000","last_peered":"2024-09-20T22:08:56.459861+0000","last_clean":"2024-09-20T22:08:56.459861+0000","last_became_active":"2024-09-20T22:06:54.850696+0000","last_became_peered":"2024-09-20T22:06:54.850696+0000","last_unstale":"2024-09-20T22:08:56.459861+0000","last_undegraded":"2024-09-20T22:08:56.459861+0000","last_fullsized":"2024-09-20T22:08:56.459861+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'13","last_scrub_stamp":"2024-09-20T22:08:35.413221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:35.413221+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:39.827104+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":159,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455707+0000","last_change":"2024-09-20T22:07:20.341524+0000","last_active":"2024-09-20T22:08:56.455707+0000","last_peered":"2024-09-20T22:08:56.455707+0000","last_clean":"2024-09-20T22:08:56.455707+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:08:56.455707+0000","last_undegraded":"2024-09-20T22:08:56.455707+0000","last_fullsized":"2024-09-20T22:08:56.455707+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:20.338388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:20.338388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.394642+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":346,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.464370+0000","last_change":"2024-09-20T22:08:36.251074+0000","last_active":"2024-09-20T22:08:56.464370+0000","last_peered":"2024-09-20T22:08:56.464370+0000","last_clean":"2024-09-20T22:08:56.464370+0000","last_became_active":"2024-09-20T22:02:49.099508+0000","last_became_peered":"2024-09-20T22:02:49.099508+0000","last_unstale":"2024-09-20T22:08:56.464370+0000","last_undegraded":"2024-09-20T22:08:56.464370+0000","last_fullsized":"2024-09-20T22:08:56.464370+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:08:36.250748+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:56.158190+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":320,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.463448+0000","last_change":"2024-09-20T22:08:02.168865+0000","last_active":"2024-09-20T22:08:56.463448+0000","last_peered":"2024-09-20T22:08:56.463448+0000","last_clean":"2024-09-20T22:08:56.463448+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:08:56.463448+0000","last_undegraded":"2024-09-20T22:08:56.463448+0000","last_fullsized":"2024-09-20T22:08:56.463448+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:08:02.167030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:15.048786+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":309,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455467+0000","last_change":"2024-09-20T22:08:31.832304+0000","last_active":"2024-09-20T22:08:56.455467+0000","last_peered":"2024-09-20T22:08:56.455467+0000","last_clean":"2024-09-20T22:08:56.455467+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:08:56.455467+0000","last_undegraded":"2024-09-20T22:08:56.455467+0000","last_fullsized":"2024-09-20T22:08:56.455467+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:31.832107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:31.832107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:53.771648+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":315,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.460643+0000","last_change":"2024-09-20T22:08:48.447550+0000","last_active":"2024-09-20T22:08:56.460643+0000","last_peered":"2024-09-20T22:08:56.460643+0000","last_clean":"2024-09-20T22:08:56.460643+0000","last_became_active":"2024-09-20T22:08:48.446840+0000","last_became_peered":"2024-09-20T22:08:48.446840+0000","last_unstale":"2024-09-20T22:08:56.460643+0000","last_undegraded":"2024-09-20T22:08:56.460643+0000","last_fullsized":"2024-09-20T22:08:56.460643+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:07:53.673691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.673691+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.394460+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":27,"num_write":17,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":161010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"210'5898","reported_seq":5974,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:57.749146+0000","last_change":"2024-09-20T22:07:52.939933+0000","last_active":"2024-09-20T22:08:57.749146+0000","last_peered":"2024-09-20T22:08:57.749146+0000","last_clean":"2024-09-20T22:08:57.749146+0000","last_became_active":"2024-09-20T22:07:52.939626+0000","last_became_peered":"2024-09-20T22:07:52.939626+0000","last_unstale":"2024-09-20T22:08:57.749146+0000","last_undegraded":"2024-09-20T22:08:57.749146+0000","last_fullsized":"2024-09-20T22:08:57.749146+0000","mapping_epoch":176,"log_start":"182'1488","ondisk_log_start":"182'1488","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4410,"log_dups_size":0,"ondisk_log_size":4410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:10.985426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":386531328,"num_objects":5898,"num_object_clones":0,"num_object_copies":17694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5898,"num_write_kb":377472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"71'12","reported_seq":337,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.430390+0000","last_change":"2024-09-20T22:07:29.614630+0000","last_active":"2024-09-20T22:09:01.430390+0000","last_peered":"2024-09-20T22:09:01.430390+0000","last_clean":"2024-09-20T22:09:01.430390+0000","last_became_active":"2024-09-20T22:06:54.848670+0000","last_became_peered":"2024-09-20T22:06:54.848670+0000","last_unstale":"2024-09-20T22:09:01.430390+0000","last_undegraded":"2024-09-20T22:09:01.430390+0000","last_fullsized":"2024-09-20T22:09:01.430390+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:29.614551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:29.614551+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":322,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.422435+0000","last_change":"2024-09-20T22:06:44.862452+0000","last_active":"2024-09-20T22:09:01.422435+0000","last_peered":"2024-09-20T22:09:01.422435+0000","last_clean":"2024-09-20T22:09:01.422435+0000","last_became_active":"2024-09-20T22:02:50.110448+0000","last_became_peered":"2024-09-20T22:02:50.110448+0000","last_unstale":"2024-09-20T22:09:01.422435+0000","last_undegraded":"2024-09-20T22:09:01.422435+0000","last_fullsized":"2024-09-20T22:09:01.422435+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:06:44.862338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":697,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:59.001975+0000","last_change":"2024-09-20T22:07:16.590582+0000","last_active":"2024-09-20T22:08:59.001975+0000","last_peered":"2024-09-20T22:08:59.001975+0000","last_clean":"2024-09-20T22:08:59.001975+0000","last_became_active":"2024-09-20T22:06:54.844454+0000","last_became_peered":"2024-09-20T22:06:54.844454+0000","last_unstale":"2024-09-20T22:08:59.001975+0000","last_undegraded":"2024-09-20T22:08:59.001975+0000","last_fullsized":"2024-09-20T22:08:59.001975+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:07:16.590475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:07:16.590475+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":357,"num_read_kb":352,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":313,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.447592+0000","last_change":"2024-09-20T22:07:22.884981+0000","last_active":"2024-09-20T22:08:56.447592+0000","last_peered":"2024-09-20T22:08:56.447592+0000","last_clean":"2024-09-20T22:08:56.447592+0000","last_became_active":"2024-09-20T22:06:54.841596+0000","last_became_peered":"2024-09-20T22:06:54.841596+0000","last_unstale":"2024-09-20T22:08:56.447592+0000","last_undegraded":"2024-09-20T22:08:56.447592+0000","last_fullsized":"2024-09-20T22:08:56.447592+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:22.884907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.884907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":159,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455337+0000","last_change":"2024-09-20T22:08:34.415569+0000","last_active":"2024-09-20T22:08:56.455337+0000","last_peered":"2024-09-20T22:08:56.455337+0000","last_clean":"2024-09-20T22:08:56.455337+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:08:56.455337+0000","last_undegraded":"2024-09-20T22:08:56.455337+0000","last_fullsized":"2024-09-20T22:08:56.455337+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:34.415468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.415468+0000","last_clean_scrub_stamp":"2024-09-20T22:08:34.415468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:39.709664+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"210'6059","reported_seq":6136,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:09:00.726366+0000","last_change":"2024-09-20T22:07:52.430313+0000","last_active":"2024-09-20T22:09:00.726366+0000","last_peered":"2024-09-20T22:09:00.726366+0000","last_clean":"2024-09-20T22:09:00.726366+0000","last_became_active":"2024-09-20T22:07:52.430017+0000","last_became_peered":"2024-09-20T22:07:52.430017+0000","last_unstale":"2024-09-20T22:09:00.726366+0000","last_undegraded":"2024-09-20T22:09:00.726366+0000","last_fullsized":"2024-09-20T22:09:00.726366+0000","mapping_epoch":176,"log_start":"180'700","ondisk_log_start":"180'700","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":5359,"log_dups_size":0,"ondisk_log_size":5359,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:04.804126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":397082624,"num_objects":6059,"num_object_clones":0,"num_object_copies":18177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6059,"num_write_kb":387776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"71'16","reported_seq":421,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.450179+0000","last_change":"2024-09-20T22:07:53.297530+0000","last_active":"2024-09-20T22:08:56.450179+0000","last_peered":"2024-09-20T22:08:56.450179+0000","last_clean":"2024-09-20T22:08:56.450179+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:08:56.450179+0000","last_undegraded":"2024-09-20T22:08:56.450179+0000","last_fullsized":"2024-09-20T22:08:56.450179+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'16","last_scrub_stamp":"2024-09-20T22:07:53.297442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.297442+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:20.417428+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":320,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.450310+0000","last_change":"2024-09-20T22:06:48.105501+0000","last_active":"2024-09-20T22:08:56.450310+0000","last_peered":"2024-09-20T22:08:56.450310+0000","last_clean":"2024-09-20T22:08:56.450310+0000","last_became_active":"2024-09-20T22:02:50.106011+0000","last_became_peered":"2024-09-20T22:02:50.106011+0000","last_unstale":"2024-09-20T22:08:56.450310+0000","last_undegraded":"2024-09-20T22:08:56.450310+0000","last_fullsized":"2024-09-20T22:08:56.450310+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:48.105403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:48.105403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:00.825210+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":312,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.427290+0000","last_change":"2024-09-20T22:08:55.451233+0000","last_active":"2024-09-20T22:09:01.427290+0000","last_peered":"2024-09-20T22:09:01.427290+0000","last_clean":"2024-09-20T22:09:01.427290+0000","last_became_active":"2024-09-20T22:08:55.450855+0000","last_became_peered":"2024-09-20T22:08:55.450855+0000","last_unstale":"2024-09-20T22:09:01.427290+0000","last_undegraded":"2024-09-20T22:09:01.427290+0000","last_fullsized":"2024-09-20T22:09:01.427290+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:07:53.673691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.673691+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:05.868744+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53669,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":26,"num_write":16,"num_write_kb":219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":161009,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":29,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:59.475351+0000","last_change":"2024-09-20T22:08:59.475385+0000","last_active":"2024-09-20T22:08:59.475351+0000","last_peered":"2024-09-20T22:08:59.475351+0000","last_clean":"2024-09-20T22:08:59.475351+0000","last_became_active":"2024-09-20T22:08:52.423678+0000","last_became_peered":"2024-09-20T22:08:52.423678+0000","last_unstale":"2024-09-20T22:08:59.475351+0000","last_undegraded":"2024-09-20T22:08:59.475351+0000","last_fullsized":"2024-09-20T22:08:59.475351+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:59.475260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:59.475260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:28.970779+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":330,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.447499+0000","last_change":"2024-09-20T22:08:39.101879+0000","last_active":"2024-09-20T22:08:56.447499+0000","last_peered":"2024-09-20T22:08:56.447499+0000","last_clean":"2024-09-20T22:08:56.447499+0000","last_became_active":"2024-09-20T22:04:21.237725+0000","last_became_peered":"2024-09-20T22:04:21.237725+0000","last_unstale":"2024-09-20T22:08:56.447499+0000","last_undegraded":"2024-09-20T22:08:56.447499+0000","last_fullsized":"2024-09-20T22:08:56.447499+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:39.101763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:08:39.101763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:05.527492+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"210'6001","reported_seq":6079,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:09:00.595032+0000","last_change":"2024-09-20T22:07:52.429509+0000","last_active":"2024-09-20T22:09:00.595032+0000","last_peered":"2024-09-20T22:09:00.595032+0000","last_clean":"2024-09-20T22:09:00.595032+0000","last_became_active":"2024-09-20T22:07:52.429069+0000","last_became_peered":"2024-09-20T22:07:52.429069+0000","last_unstale":"2024-09-20T22:09:00.595032+0000","last_undegraded":"2024-09-20T22:09:00.595032+0000","last_fullsized":"2024-09-20T22:09:00.595032+0000","mapping_epoch":176,"log_start":"182'1432","ondisk_log_start":"182'1432","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4569,"log_dups_size":0,"ondisk_log_size":4569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.758907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393281536,"num_objects":6001,"num_object_clones":0,"num_object_copies":18003,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6001,"num_write_kb":384064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"71'11","reported_seq":337,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.456647+0000","last_change":"2024-09-20T22:07:13.511239+0000","last_active":"2024-09-20T22:08:56.456647+0000","last_peered":"2024-09-20T22:08:56.456647+0000","last_clean":"2024-09-20T22:08:56.456647+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:08:56.456647+0000","last_undegraded":"2024-09-20T22:08:56.456647+0000","last_fullsized":"2024-09-20T22:08:56.456647+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:13.511159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:13.511159+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:58.622607+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":321,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455764+0000","last_change":"2024-09-20T22:08:40.493022+0000","last_active":"2024-09-20T22:08:56.455764+0000","last_peered":"2024-09-20T22:08:56.455764+0000","last_clean":"2024-09-20T22:08:56.455764+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:08:56.455764+0000","last_undegraded":"2024-09-20T22:08:56.455764+0000","last_fullsized":"2024-09-20T22:08:56.455764+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:40.492952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:40.492952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:45.281101+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":320,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.446754+0000","last_change":"2024-09-20T22:08:27.057946+0000","last_active":"2024-09-20T22:08:56.446754+0000","last_peered":"2024-09-20T22:08:56.446754+0000","last_clean":"2024-09-20T22:08:56.446754+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:08:56.446754+0000","last_undegraded":"2024-09-20T22:08:56.446754+0000","last_fullsized":"2024-09-20T22:08:56.446754+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:27.057849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:27.057849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:36.999687+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":308,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.451131+0000","last_change":"2024-09-20T22:06:45.030549+0000","last_active":"2024-09-20T22:08:56.451131+0000","last_peered":"2024-09-20T22:08:56.451131+0000","last_clean":"2024-09-20T22:08:56.451131+0000","last_became_active":"2024-09-20T22:02:56.349376+0000","last_became_peered":"2024-09-20T22:02:56.349376+0000","last_unstale":"2024-09-20T22:08:56.451131+0000","last_undegraded":"2024-09-20T22:08:56.451131+0000","last_fullsized":"2024-09-20T22:08:56.451131+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.030289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.030289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:00.825183+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":316,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.446522+0000","last_change":"2024-09-20T22:08:48.461425+0000","last_active":"2024-09-20T22:08:56.446522+0000","last_peered":"2024-09-20T22:08:56.446522+0000","last_clean":"2024-09-20T22:08:56.446522+0000","last_became_active":"2024-09-20T22:08:48.460964+0000","last_became_peered":"2024-09-20T22:08:48.460964+0000","last_unstale":"2024-09-20T22:08:56.446522+0000","last_undegraded":"2024-09-20T22:08:56.446522+0000","last_fullsized":"2024-09-20T22:08:56.446522+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:07:53.673691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.673691+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:00.053559+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":27,"num_write":16,"num_write_kb":219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":423697,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"210'5823","reported_seq":5898,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:57.723235+0000","last_change":"2024-09-20T22:07:52.939890+0000","last_active":"2024-09-20T22:08:57.723235+0000","last_peered":"2024-09-20T22:08:57.723235+0000","last_clean":"2024-09-20T22:08:57.723235+0000","last_became_active":"2024-09-20T22:07:52.939569+0000","last_became_peered":"2024-09-20T22:07:52.939569+0000","last_unstale":"2024-09-20T22:08:57.723235+0000","last_undegraded":"2024-09-20T22:08:57.723235+0000","last_fullsized":"2024-09-20T22:08:57.723235+0000","mapping_epoch":176,"log_start":"182'1385","ondisk_log_start":"182'1385","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4438,"log_dups_size":0,"ondisk_log_size":4438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:15.331372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":381616128,"num_objects":5823,"num_object_clones":0,"num_object_copies":17469,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5823,"num_write_kb":372672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"71'17","reported_seq":354,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.456629+0000","last_change":"2024-09-20T22:07:54.528854+0000","last_active":"2024-09-20T22:08:56.456629+0000","last_peered":"2024-09-20T22:08:56.456629+0000","last_clean":"2024-09-20T22:08:56.456629+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:08:56.456629+0000","last_undegraded":"2024-09-20T22:08:56.456629+0000","last_fullsized":"2024-09-20T22:08:56.456629+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'17","last_scrub_stamp":"2024-09-20T22:07:54.528759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:54.528759+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.170193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":329,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.449457+0000","last_change":"2024-09-20T22:08:30.429880+0000","last_active":"2024-09-20T22:08:56.449457+0000","last_peered":"2024-09-20T22:08:56.449457+0000","last_clean":"2024-09-20T22:08:56.449457+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:08:56.449457+0000","last_undegraded":"2024-09-20T22:08:56.449457+0000","last_fullsized":"2024-09-20T22:08:56.449457+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:30.429704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:30.429704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:36.273776+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":158,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455391+0000","last_change":"2024-09-20T22:07:19.376008+0000","last_active":"2024-09-20T22:08:56.455391+0000","last_peered":"2024-09-20T22:08:56.455391+0000","last_clean":"2024-09-20T22:08:56.455391+0000","last_became_active":"2024-09-20T22:06:54.848275+0000","last_became_peered":"2024-09-20T22:06:54.848275+0000","last_unstale":"2024-09-20T22:08:56.455391+0000","last_undegraded":"2024-09-20T22:08:56.455391+0000","last_fullsized":"2024-09-20T22:08:56.455391+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:19.375867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.375867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.394691+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":316,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.463773+0000","last_change":"2024-09-20T22:07:55.195229+0000","last_active":"2024-09-20T22:08:56.463773+0000","last_peered":"2024-09-20T22:08:56.463773+0000","last_clean":"2024-09-20T22:08:56.463773+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:08:56.463773+0000","last_undegraded":"2024-09-20T22:08:56.463773+0000","last_fullsized":"2024-09-20T22:08:56.463773+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:55.195131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.195131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:10.187674+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":601,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:58.773448+0000","last_change":"2024-09-20T22:08:58.773488+0000","last_active":"2024-09-20T22:08:58.773448+0000","last_peered":"2024-09-20T22:08:58.773448+0000","last_clean":"2024-09-20T22:08:58.773448+0000","last_became_active":"2024-09-20T22:08:48.455288+0000","last_became_peered":"2024-09-20T22:08:48.455288+0000","last_unstale":"2024-09-20T22:08:58.773448+0000","last_undegraded":"2024-09-20T22:08:58.773448+0000","last_fullsized":"2024-09-20T22:08:58.773448+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:08:58.773367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:08:58.773367+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:00.937529+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"210'6080","reported_seq":6158,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.171679+0000","last_change":"2024-09-20T22:07:52.429344+0000","last_active":"2024-09-20T22:09:01.171679+0000","last_peered":"2024-09-20T22:09:01.171679+0000","last_clean":"2024-09-20T22:09:01.171679+0000","last_became_active":"2024-09-20T22:07:52.429054+0000","last_became_peered":"2024-09-20T22:07:52.429054+0000","last_unstale":"2024-09-20T22:09:01.171679+0000","last_undegraded":"2024-09-20T22:09:01.171679+0000","last_fullsized":"2024-09-20T22:09:01.171679+0000","mapping_epoch":176,"log_start":"182'1294","ondisk_log_start":"182'1294","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4786,"log_dups_size":0,"ondisk_log_size":4786,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:16.620658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398458880,"num_objects":6080,"num_object_clones":0,"num_object_copies":18240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6080,"num_write_kb":389120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"71'10","reported_seq":324,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455988+0000","last_change":"2024-09-20T22:05:27.048378+0000","last_active":"2024-09-20T22:08:56.455988+0000","last_peered":"2024-09-20T22:08:56.455988+0000","last_clean":"2024-09-20T22:08:56.455988+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:08:56.455988+0000","last_undegraded":"2024-09-20T22:08:56.455988+0000","last_fullsized":"2024-09-20T22:08:56.455988+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:05:27.048215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:05:27.048215+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:58.622378+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"64'1","reported_seq":326,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.449939+0000","last_change":"2024-09-20T22:06:33.966005+0000","last_active":"2024-09-20T22:08:56.449939+0000","last_peered":"2024-09-20T22:08:56.449939+0000","last_clean":"2024-09-20T22:08:56.449939+0000","last_became_active":"2024-09-20T22:02:50.110446+0000","last_became_peered":"2024-09-20T22:02:50.110446+0000","last_unstale":"2024-09-20T22:08:56.449939+0000","last_undegraded":"2024-09-20T22:08:56.449939+0000","last_fullsized":"2024-09-20T22:08:56.449939+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:06:33.965866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:33.965866+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:00.825156+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"139'9","reported_seq":412,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.464039+0000","last_change":"2024-09-20T22:07:57.138071+0000","last_active":"2024-09-20T22:08:56.464039+0000","last_peered":"2024-09-20T22:08:56.464039+0000","last_clean":"2024-09-20T22:08:56.464039+0000","last_became_active":"2024-09-20T22:02:54.294458+0000","last_became_peered":"2024-09-20T22:02:54.294458+0000","last_unstale":"2024-09-20T22:08:56.464039+0000","last_undegraded":"2024-09-20T22:08:56.464039+0000","last_fullsized":"2024-09-20T22:08:56.464039+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'9","last_scrub_stamp":"2024-09-20T22:07:57.137943+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:06:40.910011+0000","last_clean_scrub_stamp":"2024-09-20T22:07:57.137943+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:23.015596+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":310,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.456921+0000","last_change":"2024-09-20T22:07:23.482514+0000","last_active":"2024-09-20T22:08:56.456921+0000","last_peered":"2024-09-20T22:08:56.456921+0000","last_clean":"2024-09-20T22:08:56.456921+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:08:56.456921+0000","last_undegraded":"2024-09-20T22:08:56.456921+0000","last_fullsized":"2024-09-20T22:08:56.456921+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.482372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:58.622536+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"210'5891","reported_seq":5968,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:57.726986+0000","last_change":"2024-09-20T22:07:52.429071+0000","last_active":"2024-09-20T22:08:57.726986+0000","last_peered":"2024-09-20T22:08:57.726986+0000","last_clean":"2024-09-20T22:08:57.726986+0000","last_became_active":"2024-09-20T22:07:52.428789+0000","last_became_peered":"2024-09-20T22:07:52.428789+0000","last_unstale":"2024-09-20T22:08:57.726986+0000","last_undegraded":"2024-09-20T22:08:57.726986+0000","last_fullsized":"2024-09-20T22:08:57.726986+0000","mapping_epoch":176,"log_start":"182'1456","ondisk_log_start":"182'1456","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4435,"log_dups_size":0,"ondisk_log_size":4435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:01.547587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":386072576,"num_objects":5891,"num_object_clones":0,"num_object_copies":17673,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5891,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5891,"num_write_kb":377024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"71'9","reported_seq":343,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.421667+0000","last_change":"2024-09-20T22:07:59.740218+0000","last_active":"2024-09-20T22:09:01.421667+0000","last_peered":"2024-09-20T22:09:01.421667+0000","last_clean":"2024-09-20T22:09:01.421667+0000","last_became_active":"2024-09-20T22:06:54.847770+0000","last_became_peered":"2024-09-20T22:06:54.847770+0000","last_unstale":"2024-09-20T22:09:01.421667+0000","last_undegraded":"2024-09-20T22:09:01.421667+0000","last_fullsized":"2024-09-20T22:09:01.421667+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:59.740089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:59.740089+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:19.034130+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":334,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.447222+0000","last_change":"2024-09-20T22:08:31.071694+0000","last_active":"2024-09-20T22:08:56.447222+0000","last_peered":"2024-09-20T22:08:56.447222+0000","last_clean":"2024-09-20T22:08:56.447222+0000","last_became_active":"2024-09-20T22:02:50.112682+0000","last_became_peered":"2024-09-20T22:02:50.112682+0000","last_unstale":"2024-09-20T22:08:56.447222+0000","last_undegraded":"2024-09-20T22:08:56.447222+0000","last_fullsized":"2024-09-20T22:08:56.447222+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:08:31.071610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:31.071610+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:35.205784+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":320,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.466048+0000","last_change":"2024-09-20T22:07:22.203673+0000","last_active":"2024-09-20T22:08:56.466048+0000","last_peered":"2024-09-20T22:08:56.466048+0000","last_clean":"2024-09-20T22:08:56.466048+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:08:56.466048+0000","last_undegraded":"2024-09-20T22:08:56.466048+0000","last_fullsized":"2024-09-20T22:08:56.466048+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:22.203576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.203576+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:57.492617+0000","scrub_duration":49,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":320,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.422346+0000","last_change":"2024-09-20T22:08:27.817949+0000","last_active":"2024-09-20T22:09:01.422346+0000","last_peered":"2024-09-20T22:09:01.422346+0000","last_clean":"2024-09-20T22:09:01.422346+0000","last_became_active":"2024-09-20T22:06:55.824229+0000","last_became_peered":"2024-09-20T22:06:55.824229+0000","last_unstale":"2024-09-20T22:09:01.422346+0000","last_undegraded":"2024-09-20T22:09:01.422346+0000","last_fullsized":"2024-09-20T22:09:01.422346+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:27.817850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:27.817850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:45.479132+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"211'6094","reported_seq":6173,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.447275+0000","last_change":"2024-09-20T22:07:52.674668+0000","last_active":"2024-09-20T22:09:01.447275+0000","last_peered":"2024-09-20T22:09:01.447275+0000","last_clean":"2024-09-20T22:09:01.447275+0000","last_became_active":"2024-09-20T22:07:52.674301+0000","last_became_peered":"2024-09-20T22:07:52.674301+0000","last_unstale":"2024-09-20T22:09:01.447275+0000","last_undegraded":"2024-09-20T22:09:01.447275+0000","last_fullsized":"2024-09-20T22:09:01.447275+0000","mapping_epoch":176,"log_start":"182'1126","ondisk_log_start":"182'1126","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4968,"log_dups_size":0,"ondisk_log_size":4968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:06.988552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":399376384,"num_objects":6094,"num_object_clones":0,"num_object_copies":18282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6094,"num_write_kb":390016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"71'19","reported_seq":374,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.449204+0000","last_change":"2024-09-20T22:08:51.467188+0000","last_active":"2024-09-20T22:08:56.449204+0000","last_peered":"2024-09-20T22:08:56.449204+0000","last_clean":"2024-09-20T22:08:56.449204+0000","last_became_active":"2024-09-20T22:08:51.451319+0000","last_became_peered":"2024-09-20T22:08:51.451319+0000","last_unstale":"2024-09-20T22:08:56.449204+0000","last_undegraded":"2024-09-20T22:08:56.449204+0000","last_fullsized":"2024-09-20T22:08:56.449204+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:07:25.154919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:25.154919+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:01.399146+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":321,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455684+0000","last_change":"2024-09-20T22:08:00.560208+0000","last_active":"2024-09-20T22:08:56.455684+0000","last_peered":"2024-09-20T22:08:56.455684+0000","last_clean":"2024-09-20T22:08:56.455684+0000","last_became_active":"2024-09-20T22:02:50.108712+0000","last_became_peered":"2024-09-20T22:02:50.108712+0000","last_unstale":"2024-09-20T22:08:56.455684+0000","last_undegraded":"2024-09-20T22:08:56.455684+0000","last_fullsized":"2024-09-20T22:08:56.455684+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:00.560098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:00.560098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:09.631655+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":320,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.463277+0000","last_change":"2024-09-20T22:07:20.217991+0000","last_active":"2024-09-20T22:08:56.463277+0000","last_peered":"2024-09-20T22:08:56.463277+0000","last_clean":"2024-09-20T22:08:56.463277+0000","last_became_active":"2024-09-20T22:02:54.291134+0000","last_became_peered":"2024-09-20T22:02:54.291134+0000","last_unstale":"2024-09-20T22:08:56.463277+0000","last_undegraded":"2024-09-20T22:08:56.463277+0000","last_fullsized":"2024-09-20T22:08:56.463277+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:20.217761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:20.217761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:57.492631+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":21,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.463576+0000","last_change":"2024-09-20T22:08:52.421026+0000","last_active":"2024-09-20T22:08:56.463576+0000","last_peered":"2024-09-20T22:08:56.463576+0000","last_clean":"2024-09-20T22:08:56.463576+0000","last_became_active":"2024-09-20T22:08:52.420555+0000","last_became_peered":"2024-09-20T22:08:52.420555+0000","last_unstale":"2024-09-20T22:08:56.463576+0000","last_undegraded":"2024-09-20T22:08:56.463576+0000","last_fullsized":"2024-09-20T22:08:56.463576+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:29.645127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:29.645127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:57.492645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"210'6166","reported_seq":6244,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:09:00.716730+0000","last_change":"2024-09-20T22:07:52.675309+0000","last_active":"2024-09-20T22:09:00.716730+0000","last_peered":"2024-09-20T22:09:00.716730+0000","last_clean":"2024-09-20T22:09:00.716730+0000","last_became_active":"2024-09-20T22:07:52.674921+0000","last_became_peered":"2024-09-20T22:07:52.674921+0000","last_unstale":"2024-09-20T22:09:00.716730+0000","last_undegraded":"2024-09-20T22:09:00.716730+0000","last_fullsized":"2024-09-20T22:09:00.716730+0000","mapping_epoch":176,"log_start":"181'900","ondisk_log_start":"181'900","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":5266,"log_dups_size":0,"ondisk_log_size":5266,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:17.687123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":404094976,"num_objects":6166,"num_object_clones":0,"num_object_copies":18498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6166,"num_write_kb":394624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"71'12","reported_seq":341,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455394+0000","last_change":"2024-09-20T22:07:22.458421+0000","last_active":"2024-09-20T22:08:56.455394+0000","last_peered":"2024-09-20T22:08:56.455394+0000","last_clean":"2024-09-20T22:08:56.455394+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:08:56.455394+0000","last_undegraded":"2024-09-20T22:08:56.455394+0000","last_fullsized":"2024-09-20T22:08:56.455394+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:22.458279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.458279+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:58.622573+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":322,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455142+0000","last_change":"2024-09-20T22:07:21.584606+0000","last_active":"2024-09-20T22:08:56.455142+0000","last_peered":"2024-09-20T22:08:56.455142+0000","last_clean":"2024-09-20T22:08:56.455142+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:08:56.455142+0000","last_undegraded":"2024-09-20T22:08:56.455142+0000","last_fullsized":"2024-09-20T22:08:56.455142+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:21.584499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:21.584499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":321,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.463708+0000","last_change":"2024-09-20T22:07:15.260254+0000","last_active":"2024-09-20T22:08:56.463708+0000","last_peered":"2024-09-20T22:08:56.463708+0000","last_clean":"2024-09-20T22:08:56.463708+0000","last_became_active":"2024-09-20T22:06:54.843906+0000","last_became_peered":"2024-09-20T22:06:54.843906+0000","last_unstale":"2024-09-20T22:08:56.463708+0000","last_undegraded":"2024-09-20T22:08:56.463708+0000","last_fullsized":"2024-09-20T22:08:56.463708+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:15.260148+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.260148+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:57.492602+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":270,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.421997+0000","last_change":"2024-09-20T22:08:52.418883+0000","last_active":"2024-09-20T22:09:01.421997+0000","last_peered":"2024-09-20T22:09:01.421997+0000","last_clean":"2024-09-20T22:09:01.421997+0000","last_became_active":"2024-09-20T22:08:52.418544+0000","last_became_peered":"2024-09-20T22:08:52.418544+0000","last_unstale":"2024-09-20T22:09:01.421997+0000","last_undegraded":"2024-09-20T22:09:01.421997+0000","last_fullsized":"2024-09-20T22:09:01.421997+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:36.768146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:36.768146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:55.646733+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"210'5942","reported_seq":6020,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:09:00.669595+0000","last_change":"2024-09-20T22:07:52.939846+0000","last_active":"2024-09-20T22:09:00.669595+0000","last_peered":"2024-09-20T22:09:00.669595+0000","last_clean":"2024-09-20T22:09:00.669595+0000","last_became_active":"2024-09-20T22:07:52.939563+0000","last_became_peered":"2024-09-20T22:07:52.939563+0000","last_unstale":"2024-09-20T22:09:00.669595+0000","last_undegraded":"2024-09-20T22:09:00.669595+0000","last_fullsized":"2024-09-20T22:09:00.669595+0000","mapping_epoch":176,"log_start":"182'1171","ondisk_log_start":"182'1171","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4771,"log_dups_size":0,"ondisk_log_size":4771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":389414912,"num_objects":5942,"num_object_clones":0,"num_object_copies":17826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5942,"num_write_kb":380288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"71'15","reported_seq":371,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.446766+0000","last_change":"2024-09-20T22:08:51.413659+0000","last_active":"2024-09-20T22:08:56.446766+0000","last_peered":"2024-09-20T22:08:56.446766+0000","last_clean":"2024-09-20T22:08:56.446766+0000","last_became_active":"2024-09-20T22:08:51.405122+0000","last_became_peered":"2024-09-20T22:08:51.405122+0000","last_unstale":"2024-09-20T22:08:56.446766+0000","last_undegraded":"2024-09-20T22:08:56.446766+0000","last_fullsized":"2024-09-20T22:08:56.446766+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:08:38.115162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:38.115162+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:53.864987+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":213,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.456879+0000","last_change":"2024-09-20T22:08:51.463648+0000","last_active":"2024-09-20T22:08:56.456879+0000","last_peered":"2024-09-20T22:08:56.456879+0000","last_clean":"2024-09-20T22:08:56.456879+0000","last_became_active":"2024-09-20T22:08:51.454410+0000","last_became_peered":"2024-09-20T22:08:51.454410+0000","last_unstale":"2024-09-20T22:08:56.456879+0000","last_undegraded":"2024-09-20T22:08:56.456879+0000","last_fullsized":"2024-09-20T22:08:56.456879+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:06:40.813590+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:58.622501+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":396,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.454051+0000","last_change":"2024-09-20T22:08:32.824544+0000","last_active":"2024-09-20T22:08:56.454051+0000","last_peered":"2024-09-20T22:08:56.454051+0000","last_clean":"2024-09-20T22:08:56.454051+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:08:56.454051+0000","last_undegraded":"2024-09-20T22:08:56.454051+0000","last_fullsized":"2024-09-20T22:08:56.454051+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:08:32.824413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.824413+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:01.392553+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":316,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.448016+0000","last_change":"2024-09-20T22:08:32.081639+0000","last_active":"2024-09-20T22:08:56.448016+0000","last_peered":"2024-09-20T22:08:56.448016+0000","last_clean":"2024-09-20T22:08:56.448016+0000","last_became_active":"2024-09-20T22:02:56.346990+0000","last_became_peered":"2024-09-20T22:02:56.346990+0000","last_unstale":"2024-09-20T22:08:56.448016+0000","last_undegraded":"2024-09-20T22:08:56.448016+0000","last_fullsized":"2024-09-20T22:08:56.448016+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:32.081527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.081527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:01.717616+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":328,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.449100+0000","last_change":"2024-09-20T22:08:01.261458+0000","last_active":"2024-09-20T22:08:56.449100+0000","last_peered":"2024-09-20T22:08:56.449100+0000","last_clean":"2024-09-20T22:08:56.449100+0000","last_became_active":"2024-09-20T22:02:50.111935+0000","last_became_peered":"2024-09-20T22:02:50.111935+0000","last_unstale":"2024-09-20T22:08:56.449100+0000","last_undegraded":"2024-09-20T22:08:56.449100+0000","last_fullsized":"2024-09-20T22:08:56.449100+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:01.261328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:01.261328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:06.169471+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"71'6","reported_seq":339,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455136+0000","last_change":"2024-09-20T22:08:52.431426+0000","last_active":"2024-09-20T22:08:56.455136+0000","last_peered":"2024-09-20T22:08:56.455136+0000","last_clean":"2024-09-20T22:08:56.455136+0000","last_became_active":"2024-09-20T22:08:52.419521+0000","last_became_peered":"2024-09-20T22:08:52.419521+0000","last_unstale":"2024-09-20T22:08:56.455136+0000","last_undegraded":"2024-09-20T22:08:56.455136+0000","last_fullsized":"2024-09-20T22:08:56.455136+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'6","last_scrub_stamp":"2024-09-20T22:08:33.574900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:33.574900+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:38.149614+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":313,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.447318+0000","last_change":"2024-09-20T22:07:19.865906+0000","last_active":"2024-09-20T22:08:56.447318+0000","last_peered":"2024-09-20T22:08:56.447318+0000","last_clean":"2024-09-20T22:08:56.447318+0000","last_became_active":"2024-09-20T22:06:54.836200+0000","last_became_peered":"2024-09-20T22:06:54.836200+0000","last_unstale":"2024-09-20T22:08:56.447318+0000","last_undegraded":"2024-09-20T22:08:56.447318+0000","last_fullsized":"2024-09-20T22:08:56.447318+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:19.865809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.865809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":301,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455395+0000","last_change":"2024-09-20T22:06:39.783526+0000","last_active":"2024-09-20T22:08:56.455395+0000","last_peered":"2024-09-20T22:08:56.455395+0000","last_clean":"2024-09-20T22:08:56.455395+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:08:56.455395+0000","last_undegraded":"2024-09-20T22:08:56.455395+0000","last_fullsized":"2024-09-20T22:08:56.455395+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:39.783456+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:39.783456+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"71'9","reported_seq":349,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.430482+0000","last_change":"2024-09-20T22:08:52.433690+0000","last_active":"2024-09-20T22:09:01.430482+0000","last_peered":"2024-09-20T22:09:01.430482+0000","last_clean":"2024-09-20T22:09:01.430482+0000","last_became_active":"2024-09-20T22:08:52.423437+0000","last_became_peered":"2024-09-20T22:08:52.423437+0000","last_unstale":"2024-09-20T22:09:01.430482+0000","last_undegraded":"2024-09-20T22:09:01.430482+0000","last_fullsized":"2024-09-20T22:09:01.430482+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:08:31.897253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:31.897253+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:53.330127+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":321,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.454165+0000","last_change":"2024-09-20T22:08:28.755412+0000","last_active":"2024-09-20T22:08:56.454165+0000","last_peered":"2024-09-20T22:08:56.454165+0000","last_clean":"2024-09-20T22:08:56.454165+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:08:56.454165+0000","last_undegraded":"2024-09-20T22:08:56.454165+0000","last_fullsized":"2024-09-20T22:08:56.454165+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:28.755335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:28.755335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:33.713076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":313,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.454954+0000","last_change":"2024-09-20T22:08:36.681072+0000","last_active":"2024-09-20T22:08:56.454954+0000","last_peered":"2024-09-20T22:08:56.454954+0000","last_clean":"2024-09-20T22:08:56.454954+0000","last_became_active":"2024-09-20T22:06:54.845704+0000","last_became_peered":"2024-09-20T22:06:54.845704+0000","last_unstale":"2024-09-20T22:08:56.454954+0000","last_undegraded":"2024-09-20T22:08:56.454954+0000","last_fullsized":"2024-09-20T22:08:56.454954+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:36.680959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:36.680959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:37.238356+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":316,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.464184+0000","last_change":"2024-09-20T22:08:31.095724+0000","last_active":"2024-09-20T22:08:56.464184+0000","last_peered":"2024-09-20T22:08:56.464184+0000","last_clean":"2024-09-20T22:08:56.464184+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:08:56.464184+0000","last_undegraded":"2024-09-20T22:08:56.464184+0000","last_fullsized":"2024-09-20T22:08:56.464184+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:31.095323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:31.095323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:32.516870+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"71'9","reported_seq":340,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.447423+0000","last_change":"2024-09-20T22:07:52.834986+0000","last_active":"2024-09-20T22:08:56.447423+0000","last_peered":"2024-09-20T22:08:56.447423+0000","last_clean":"2024-09-20T22:08:56.447423+0000","last_became_active":"2024-09-20T22:02:52.482141+0000","last_became_peered":"2024-09-20T22:02:52.482141+0000","last_unstale":"2024-09-20T22:08:56.447423+0000","last_undegraded":"2024-09-20T22:08:56.447423+0000","last_fullsized":"2024-09-20T22:08:56.447423+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:52.834898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:52.834898+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.590814+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":327,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.422309+0000","last_change":"2024-09-20T22:08:52.423987+0000","last_active":"2024-09-20T22:09:01.422309+0000","last_peered":"2024-09-20T22:09:01.422309+0000","last_clean":"2024-09-20T22:09:01.422309+0000","last_became_active":"2024-09-20T22:08:52.423263+0000","last_became_peered":"2024-09-20T22:08:52.423263+0000","last_unstale":"2024-09-20T22:09:01.422309+0000","last_undegraded":"2024-09-20T22:09:01.422309+0000","last_fullsized":"2024-09-20T22:09:01.422309+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.908616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.908616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"71'8","reported_seq":387,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.451389+0000","last_change":"2024-09-20T22:05:55.972684+0000","last_active":"2024-09-20T22:08:56.451389+0000","last_peered":"2024-09-20T22:08:56.451389+0000","last_clean":"2024-09-20T22:08:56.451389+0000","last_became_active":"2024-09-20T22:02:54.294381+0000","last_became_peered":"2024-09-20T22:02:54.294381+0000","last_unstale":"2024-09-20T22:08:56.451389+0000","last_undegraded":"2024-09-20T22:08:56.451389+0000","last_fullsized":"2024-09-20T22:08:56.451389+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:05:55.972565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:05:55.972565+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:00.825073+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":309,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.456649+0000","last_change":"2024-09-20T22:08:32.435371+0000","last_active":"2024-09-20T22:08:56.456649+0000","last_peered":"2024-09-20T22:08:56.456649+0000","last_clean":"2024-09-20T22:08:56.456649+0000","last_became_active":"2024-09-20T22:02:56.300041+0000","last_became_peered":"2024-09-20T22:02:56.300041+0000","last_unstale":"2024-09-20T22:08:56.456649+0000","last_undegraded":"2024-09-20T22:08:56.456649+0000","last_fullsized":"2024-09-20T22:08:56.456649+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:32.435297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.435297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:53.430793+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"71'10","reported_seq":332,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.451495+0000","last_change":"2024-09-20T22:08:38.681999+0000","last_active":"2024-09-20T22:08:56.451495+0000","last_peered":"2024-09-20T22:08:56.451495+0000","last_clean":"2024-09-20T22:08:56.451495+0000","last_became_active":"2024-09-20T22:06:54.862289+0000","last_became_peered":"2024-09-20T22:06:54.862289+0000","last_unstale":"2024-09-20T22:08:56.451495+0000","last_undegraded":"2024-09-20T22:08:56.451495+0000","last_fullsized":"2024-09-20T22:08:56.451495+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:08:38.681831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:38.681831+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:47.639920+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":321,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.746714+0000","last_change":"2024-09-20T22:08:56.746747+0000","last_active":"2024-09-20T22:08:56.746714+0000","last_peered":"2024-09-20T22:08:56.746714+0000","last_clean":"2024-09-20T22:08:56.746714+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:08:56.746714+0000","last_undegraded":"2024-09-20T22:08:56.746714+0000","last_fullsized":"2024-09-20T22:08:56.746714+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:56.746651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:56.746651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:08.258731+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"146'10","reported_seq":398,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.447099+0000","last_change":"2024-09-20T22:07:15.870748+0000","last_active":"2024-09-20T22:08:56.447099+0000","last_peered":"2024-09-20T22:08:56.447099+0000","last_clean":"2024-09-20T22:08:56.447099+0000","last_became_active":"2024-09-20T22:06:54.843097+0000","last_became_peered":"2024-09-20T22:06:54.843097+0000","last_unstale":"2024-09-20T22:08:56.447099+0000","last_undegraded":"2024-09-20T22:08:56.447099+0000","last_fullsized":"2024-09-20T22:08:56.447099+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:07:15.870646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.870646+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":315,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.421765+0000","last_change":"2024-09-20T22:08:52.424184+0000","last_active":"2024-09-20T22:09:01.421765+0000","last_peered":"2024-09-20T22:09:01.421765+0000","last_clean":"2024-09-20T22:09:01.421765+0000","last_became_active":"2024-09-20T22:08:52.422795+0000","last_became_peered":"2024-09-20T22:08:52.422795+0000","last_unstale":"2024-09-20T22:09:01.421765+0000","last_undegraded":"2024-09-20T22:09:01.421765+0000","last_fullsized":"2024-09-20T22:09:01.421765+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:35.824973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:35.824973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:43.837757+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"71'11","reported_seq":355,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.456207+0000","last_change":"2024-09-20T22:08:22.159810+0000","last_active":"2024-09-20T22:08:56.456207+0000","last_peered":"2024-09-20T22:08:56.456207+0000","last_clean":"2024-09-20T22:08:56.456207+0000","last_became_active":"2024-09-20T22:08:22.139581+0000","last_became_peered":"2024-09-20T22:08:22.139581+0000","last_unstale":"2024-09-20T22:08:56.456207+0000","last_undegraded":"2024-09-20T22:08:56.456207+0000","last_fullsized":"2024-09-20T22:08:56.456207+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:15.544705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.544705+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:58.622554+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":321,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.456149+0000","last_change":"2024-09-20T22:08:29.474891+0000","last_active":"2024-09-20T22:08:56.456149+0000","last_peered":"2024-09-20T22:08:56.456149+0000","last_clean":"2024-09-20T22:08:56.456149+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:08:56.456149+0000","last_undegraded":"2024-09-20T22:08:56.456149+0000","last_fullsized":"2024-09-20T22:08:56.456149+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:29.474781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:29.474781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:48.958420+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":157,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.461111+0000","last_change":"2024-09-20T22:08:39.327541+0000","last_active":"2024-09-20T22:08:56.461111+0000","last_peered":"2024-09-20T22:08:56.461111+0000","last_clean":"2024-09-20T22:08:56.461111+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:08:56.461111+0000","last_undegraded":"2024-09-20T22:08:56.461111+0000","last_fullsized":"2024-09-20T22:08:56.461111+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:39.326984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:39.326984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.317069+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":136,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.456307+0000","last_change":"2024-09-20T22:06:55.835713+0000","last_active":"2024-09-20T22:08:56.456307+0000","last_peered":"2024-09-20T22:08:56.456307+0000","last_clean":"2024-09-20T22:08:56.456307+0000","last_became_active":"2024-09-20T22:06:55.835393+0000","last_became_peered":"2024-09-20T22:06:55.835393+0000","last_unstale":"2024-09-20T22:08:56.456307+0000","last_undegraded":"2024-09-20T22:08:56.456307+0000","last_fullsized":"2024-09-20T22:08:56.456307+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:33.623816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:33.623816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.394618+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"71'9","reported_seq":221,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.461645+0000","last_change":"2024-09-20T22:06:54.844349+0000","last_active":"2024-09-20T22:08:56.461645+0000","last_peered":"2024-09-20T22:08:56.461645+0000","last_clean":"2024-09-20T22:08:56.461645+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:08:56.461645+0000","last_undegraded":"2024-09-20T22:08:56.461645+0000","last_fullsized":"2024-09-20T22:08:56.461645+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:12.196518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:12.196518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.394546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.452139+0000","last_change":"2024-09-20T22:08:52.418861+0000","last_active":"2024-09-20T22:08:56.452139+0000","last_peered":"2024-09-20T22:08:56.452139+0000","last_clean":"2024-09-20T22:08:56.452139+0000","last_became_active":"2024-09-20T22:08:52.418520+0000","last_became_peered":"2024-09-20T22:08:52.418520+0000","last_unstale":"2024-09-20T22:08:56.452139+0000","last_undegraded":"2024-09-20T22:08:56.452139+0000","last_fullsized":"2024-09-20T22:08:56.452139+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:54.685751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:54.685751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:02.475449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":313,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.449658+0000","last_change":"2024-09-20T22:08:34.657517+0000","last_active":"2024-09-20T22:08:56.449658+0000","last_peered":"2024-09-20T22:08:56.449658+0000","last_clean":"2024-09-20T22:08:56.449658+0000","last_became_active":"2024-09-20T22:06:54.844271+0000","last_became_peered":"2024-09-20T22:06:54.844271+0000","last_unstale":"2024-09-20T22:08:56.449658+0000","last_undegraded":"2024-09-20T22:08:56.449658+0000","last_fullsized":"2024-09-20T22:08:56.449658+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:34.657278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:34.657278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:56.363742+0000","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":311,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.420968+0000","last_change":"2024-09-20T22:07:23.495005+0000","last_active":"2024-09-20T22:09:01.420968+0000","last_peered":"2024-09-20T22:09:01.420968+0000","last_clean":"2024-09-20T22:09:01.420968+0000","last_became_active":"2024-09-20T22:06:54.848842+0000","last_became_peered":"2024-09-20T22:06:54.848842+0000","last_unstale":"2024-09-20T22:09:01.420968+0000","last_undegraded":"2024-09-20T22:09:01.420968+0000","last_fullsized":"2024-09-20T22:09:01.420968+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.494905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.494905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"71'11","reported_seq":352,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.449547+0000","last_change":"2024-09-20T22:08:52.431501+0000","last_active":"2024-09-20T22:08:56.449547+0000","last_peered":"2024-09-20T22:08:56.449547+0000","last_clean":"2024-09-20T22:08:56.449547+0000","last_became_active":"2024-09-20T22:08:52.417107+0000","last_became_peered":"2024-09-20T22:08:52.417107+0000","last_unstale":"2024-09-20T22:08:56.449547+0000","last_undegraded":"2024-09-20T22:08:56.449547+0000","last_fullsized":"2024-09-20T22:08:56.449547+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:06:36.942620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:36.942620+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:57.813003+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":329,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.447414+0000","last_change":"2024-09-20T22:07:50.851948+0000","last_active":"2024-09-20T22:08:56.447414+0000","last_peered":"2024-09-20T22:08:56.447414+0000","last_clean":"2024-09-20T22:08:56.447414+0000","last_became_active":"2024-09-20T22:06:54.828826+0000","last_became_peered":"2024-09-20T22:06:54.828826+0000","last_unstale":"2024-09-20T22:08:56.447414+0000","last_undegraded":"2024-09-20T22:08:56.447414+0000","last_fullsized":"2024-09-20T22:08:56.447414+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:50.851839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:50.851839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:57.182737+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":313,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455508+0000","last_change":"2024-09-20T22:06:45.780268+0000","last_active":"2024-09-20T22:08:56.455508+0000","last_peered":"2024-09-20T22:08:56.455508+0000","last_clean":"2024-09-20T22:08:56.455508+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:08:56.455508+0000","last_undegraded":"2024-09-20T22:08:56.455508+0000","last_fullsized":"2024-09-20T22:08:56.455508+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.780200+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.780200+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":308,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.449835+0000","last_change":"2024-09-20T22:06:33.002428+0000","last_active":"2024-09-20T22:08:56.449835+0000","last_peered":"2024-09-20T22:08:56.449835+0000","last_clean":"2024-09-20T22:08:56.449835+0000","last_became_active":"2024-09-20T22:02:56.347637+0000","last_became_peered":"2024-09-20T22:02:56.347637+0000","last_unstale":"2024-09-20T22:08:56.449835+0000","last_undegraded":"2024-09-20T22:08:56.449835+0000","last_fullsized":"2024-09-20T22:08:56.449835+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:33.002326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:33.002326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:00.825099+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"71'4","reported_seq":323,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.449525+0000","last_change":"2024-09-20T22:07:19.309318+0000","last_active":"2024-09-20T22:08:56.449525+0000","last_peered":"2024-09-20T22:08:56.449525+0000","last_clean":"2024-09-20T22:08:56.449525+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:08:56.449525+0000","last_undegraded":"2024-09-20T22:08:56.449525+0000","last_fullsized":"2024-09-20T22:08:56.449525+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'4","last_scrub_stamp":"2024-09-20T22:07:19.309187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.309187+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:00.825285+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":322,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.426723+0000","last_change":"2024-09-20T22:08:29.852608+0000","last_active":"2024-09-20T22:09:01.426723+0000","last_peered":"2024-09-20T22:09:01.426723+0000","last_clean":"2024-09-20T22:09:01.426723+0000","last_became_active":"2024-09-20T22:02:50.111142+0000","last_became_peered":"2024-09-20T22:02:50.111142+0000","last_unstale":"2024-09-20T22:09:01.426723+0000","last_undegraded":"2024-09-20T22:09:01.426723+0000","last_fullsized":"2024-09-20T22:09:01.426723+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:29.852502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:29.852502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:45.967665+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":328,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.449122+0000","last_change":"2024-09-20T22:07:55.255933+0000","last_active":"2024-09-20T22:08:56.449122+0000","last_peered":"2024-09-20T22:08:56.449122+0000","last_clean":"2024-09-20T22:08:56.449122+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:08:56.449122+0000","last_undegraded":"2024-09-20T22:08:56.449122+0000","last_fullsized":"2024-09-20T22:08:56.449122+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:55.255854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.255854+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.819675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":309,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.454219+0000","last_change":"2024-09-20T22:08:39.772520+0000","last_active":"2024-09-20T22:08:56.454219+0000","last_peered":"2024-09-20T22:08:56.454219+0000","last_clean":"2024-09-20T22:08:56.454219+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:08:56.454219+0000","last_undegraded":"2024-09-20T22:08:56.454219+0000","last_fullsized":"2024-09-20T22:08:56.454219+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:39.772379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:39.772379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:55.873541+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":198,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.455650+0000","last_change":"2024-09-20T22:06:54.828772+0000","last_active":"2024-09-20T22:08:56.455650+0000","last_peered":"2024-09-20T22:08:56.455650+0000","last_clean":"2024-09-20T22:08:56.455650+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:08:56.455650+0000","last_undegraded":"2024-09-20T22:08:56.455650+0000","last_fullsized":"2024-09-20T22:08:56.455650+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:15.303300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:15.303300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:08:59.394570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":321,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.449775+0000","last_change":"2024-09-20T22:07:57.428285+0000","last_active":"2024-09-20T22:08:56.449775+0000","last_peered":"2024-09-20T22:08:56.449775+0000","last_clean":"2024-09-20T22:08:56.449775+0000","last_became_active":"2024-09-20T22:06:54.848032+0000","last_became_peered":"2024-09-20T22:06:54.848032+0000","last_unstale":"2024-09-20T22:08:56.449775+0000","last_undegraded":"2024-09-20T22:08:56.449775+0000","last_fullsized":"2024-09-20T22:08:56.449775+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:57.428165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:57.428165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:04.979638+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"71'11","reported_seq":344,"reported_epoch":210,"state":"active+clean","last_fresh":"2024-09-20T22:08:56.450195+0000","last_change":"2024-09-20T22:08:32.401623+0000","last_active":"2024-09-20T22:08:56.450195+0000","last_peered":"2024-09-20T22:08:56.450195+0000","last_clean":"2024-09-20T22:08:56.450195+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:08:56.450195+0000","last_undegraded":"2024-09-20T22:08:56.450195+0000","last_fullsized":"2024-09-20T22:08:56.450195+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:08:32.401501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.401501+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:50.388509+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":315,"reported_epoch":211,"state":"active+clean","last_fresh":"2024-09-20T22:09:01.430239+0000","last_change":"2024-09-20T22:08:32.854748+0000","last_active":"2024-09-20T22:09:01.430239+0000","last_peered":"2024-09-20T22:09:01.430239+0000","last_clean":"2024-09-20T22:09:01.430239+0000","last_became_active":"2024-09-20T22:02:54.289078+0000","last_became_peered":"2024-09-20T22:02:54.289078+0000","last_unstale":"2024-09-20T22:09:01.430239+0000","last_undegraded":"2024-09-20T22:09:01.430239+0000","last_fullsized":"2024-09-20T22:09:01.430239+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:32.854221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.854221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:55.721860+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":16,"stat_sum":{"num_bytes":6314852352,"num_objects":96357,"num_object_clones":0,"num_object_copies":289071,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4988,"num_objects_unfound":0,"num_objects_dirty":96357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":96357,"num_write_kb":6166848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1042,"num_bytes_recovered":68288512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":19067174912,"data_stored":19067174912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76610,"ondisk_log_size":76610,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":402,"ondisk_log_size":402,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":357,"num_read_kb":352,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":703368,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":126,"num_read_kb":187,"num_write":115,"num_write_kb":1537,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1389754,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1408,"ondisk_log_size":1408,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135950,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2158844,"kb_used_data":1920812,"kb_used_omap":1,"kb_used_meta":238014,"kb_avail":91586308,"statfs":{"total":95995035648,"available":93784379392,"internally_reserved":0,"allocated":1966911488,"data_stored":1966058288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":243726794},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430290,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2133196,"kb_used_data":1943740,"kb_used_omap":1,"kb_used_meta":189438,"kb_avail":91611956,"statfs":{"total":95995035648,"available":93810642944,"internally_reserved":0,"allocated":1990389760,"data_stored":1989520836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":193984966},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724630,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3340508,"kb_used_data":3128780,"kb_used_omap":1,"kb_used_meta":211646,"kb_avail":90404644,"statfs":{"total":95995035648,"available":92574355456,"internally_reserved":0,"allocated":3203870720,"data_stored":3202986632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":216725957},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2,1,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986270,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2118312,"kb_used_data":1928856,"kb_used_omap":1,"kb_used_meta":189438,"kb_avail":91626840,"statfs":{"total":95995035648,"available":93825884160,"internally_reserved":0,"allocated":1975148544,"data_stored":1974251093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":193984965},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770257946,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2763592,"kb_used_data":2615928,"kb_used_omap":4,"kb_used_meta":147643,"kb_avail":90981560,"statfs":{"total":95995035648,"available":93165117440,"internally_reserved":0,"allocated":2678710272,"data_stored":2677780669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":151187112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149795,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2688312,"kb_used_data":2332264,"kb_used_omap":1,"kb_used_meta":356030,"kb_avail":91056840,"statfs":{"total":95995035648,"available":93242204160,"internally_reserved":0,"allocated":2388238336,"data_stored":2387326227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":364575175},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2624592,"kb_used_data":2322688,"kb_used_omap":1,"kb_used_meta":301886,"kb_avail":91120560,"statfs":{"total":95995035648,"available":93307453440,"internally_reserved":0,"allocated":2378432512,"data_stored":2377559508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":309131717},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2],"upper_bound":4},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2689120,"kb_used_data":2328912,"kb_used_omap":1,"kb_used_meta":360190,"kb_avail":91056032,"statfs":{"total":95995035648,"available":93241376768,"internally_reserved":0,"allocated":2384805888,"data_stored":2383916708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":368835013},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2401173504,"data_stored":2401173504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2685272064,"data_stored":2685272064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2393505792,"data_stored":2393505792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2398027776,"data_stored":2398027776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1975451648,"data_stored":1975451648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3232890880,"data_stored":3232890880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1995898880,"data_stored":1995898880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1984954368,"data_stored":1984954368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:09:05.277 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-20T22:09:05.277 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool set unique_pool_1 pg_num 26 2024-09-20T22:09:05.294 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:05.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:05.368 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:05.369 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:05.469 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:05.521 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:05.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:05.522 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:05.607 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:05 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:09:05.608 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:05 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:09:05.608 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:05 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3608236593' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:09:05.608 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:05 smithi067 ceph-mon[33511]: osdmap e212: 8 total, 8 up, 8 in 2024-09-20T22:09:05.608 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:09:05.608 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:09:05.622 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:05.648 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T22:09:05.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:05.681 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:05.682 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:05.765 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:05.782 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:05.789 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:05 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:09:05.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:05 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:09:05.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:05 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3608236593' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:09:05.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:05 smithi060 ceph-mon[29694]: osdmap e212: 8 total, 8 up, 8 in 2024-09-20T22:09:05.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:09:05.790 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:09:05.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:05 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:09:05.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:05 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:09:05.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:05 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3608236593' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:09:05.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:05 smithi060 ceph-mon[40290]: osdmap e212: 8 total, 8 up, 8 in 2024-09-20T22:09:05.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:09:05.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:09:05.829 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:05.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:05.831 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:05.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:05.972 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:05.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:05.973 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:06.067 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:09:06.074 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:06.123 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:06.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:06.124 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:06.147 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 69 16 103095 103079 93.3476 70.375 0.0071859 0.0107073 2024-09-20T22:09:06.225 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:06.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:06.265 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:06.265 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:06.343 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:09:06.366 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:06.449 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:06.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:06.450 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:06.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:06 smithi060 ceph-mon[40290]: 5.1 scrub starts 2024-09-20T22:09:06.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:06 smithi060 ceph-mon[40290]: 5.1 scrub ok 2024-09-20T22:09:06.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:06 smithi060 ceph-mon[40290]: 6.6 scrub starts 2024-09-20T22:09:06.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:06 smithi060 ceph-mon[40290]: 6.6 scrub ok 2024-09-20T22:09:06.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:06 smithi060 ceph-mon[40290]: pgmap v315: 158 pgs: 1 active+remapped+backfilling, 157 active+clean; 6.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 61 MiB/s wr, 969 op/s; 4988/300210 objects misplaced (1.662%); 5.3 MiB/s, 87 objects/s recovering 2024-09-20T22:09:06.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:06 smithi060 ceph-mon[40290]: osdmap e213: 8 total, 8 up, 8 in 2024-09-20T22:09:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:06 smithi060 ceph-mon[29694]: 5.1 scrub starts 2024-09-20T22:09:06.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:06 smithi060 ceph-mon[29694]: 5.1 scrub ok 2024-09-20T22:09:06.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:06 smithi060 ceph-mon[29694]: 6.6 scrub starts 2024-09-20T22:09:06.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:06 smithi060 ceph-mon[29694]: 6.6 scrub ok 2024-09-20T22:09:06.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:06 smithi060 ceph-mon[29694]: pgmap v315: 158 pgs: 1 active+remapped+backfilling, 157 active+clean; 6.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 61 MiB/s wr, 969 op/s; 4988/300210 objects misplaced (1.662%); 5.3 MiB/s, 87 objects/s recovering 2024-09-20T22:09:06.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:06 smithi060 ceph-mon[29694]: osdmap e213: 8 total, 8 up, 8 in 2024-09-20T22:09:06.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:06.606 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:06.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:06.608 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:06.708 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:06.721 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:06 smithi067 ceph-mon[33511]: 5.1 scrub starts 2024-09-20T22:09:06.721 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:06 smithi067 ceph-mon[33511]: 5.1 scrub ok 2024-09-20T22:09:06.721 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:06 smithi067 ceph-mon[33511]: 6.6 scrub starts 2024-09-20T22:09:06.721 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:06 smithi067 ceph-mon[33511]: 6.6 scrub ok 2024-09-20T22:09:06.721 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:06 smithi067 ceph-mon[33511]: pgmap v315: 158 pgs: 1 active+remapped+backfilling, 157 active+clean; 6.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 61 MiB/s wr, 969 op/s; 4988/300210 objects misplaced (1.662%); 5.3 MiB/s, 87 objects/s recovering 2024-09-20T22:09:06.721 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:06 smithi067 ceph-mon[33511]: osdmap e213: 8 total, 8 up, 8 in 2024-09-20T22:09:06.743 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:06.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:06.744 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:06.757 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:09:06.757 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:09:06.820 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:06.845 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:06.913 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:06.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:06.914 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:07.014 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:07.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:07.061 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:07.061 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:07.147 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 70 16 104180 104164 92.9828 67.8125 0.0208299 0.01075 2024-09-20T22:09:07.162 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:07.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:07.195 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:07.195 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:07.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:07.327 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:07.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:07.328 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:07.429 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:07.457 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:07.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:07.458 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:07.511 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:09:07.511 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:09:07.530 INFO:teuthology.orchestra.run.smithi060.stderr:set pool 8 pg_num to 26 2024-09-20T22:09:07.554 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:09:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:09:07.560 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:07.600 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:07.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:07.601 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:07.604 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:07 smithi060 ceph-mon[29694]: 5.a deep-scrub starts 2024-09-20T22:09:07.604 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:07 smithi060 ceph-mon[29694]: 5.e scrub starts 2024-09-20T22:09:07.604 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:07 smithi060 ceph-mon[29694]: 5.a deep-scrub ok 2024-09-20T22:09:07.604 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:07 smithi060 ceph-mon[29694]: 5.e scrub ok 2024-09-20T22:09:07.604 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:07 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3651127141' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pg_num", "val": "26"}]: dispatch 2024-09-20T22:09:07.604 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:07 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pg_num", "val": "26"}]: dispatch 2024-09-20T22:09:07.604 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:07 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/26641339' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:09:07.604 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:07 smithi060 ceph-mon[40290]: 5.a deep-scrub starts 2024-09-20T22:09:07.604 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:07 smithi060 ceph-mon[40290]: 5.e scrub starts 2024-09-20T22:09:07.605 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:07 smithi060 ceph-mon[40290]: 5.a deep-scrub ok 2024-09-20T22:09:07.605 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:07 smithi060 ceph-mon[40290]: 5.e scrub ok 2024-09-20T22:09:07.605 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:07 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3651127141' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pg_num", "val": "26"}]: dispatch 2024-09-20T22:09:07.605 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:07 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pg_num", "val": "26"}]: dispatch 2024-09-20T22:09:07.605 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:07 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/26641339' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:09:07.605 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:09:07.605 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:09:07.645 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:09:07.645 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:09:07.678 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:09:07.678 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:09:07.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:09:07.702 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:07.724 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:07.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:07.725 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:07.733 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:09:07.733 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:09:07.763 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:09:07.763 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:09:07.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:07 smithi067 ceph-mon[33511]: 5.a deep-scrub starts 2024-09-20T22:09:07.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:07 smithi067 ceph-mon[33511]: 5.e scrub starts 2024-09-20T22:09:07.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:07 smithi067 ceph-mon[33511]: 5.a deep-scrub ok 2024-09-20T22:09:07.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:07 smithi067 ceph-mon[33511]: 5.e scrub ok 2024-09-20T22:09:07.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:07 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3651127141' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pg_num", "val": "26"}]: dispatch 2024-09-20T22:09:07.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:07 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pg_num", "val": "26"}]: dispatch 2024-09-20T22:09:07.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:07 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/26641339' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:09:07.802 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:09:07.803 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:09:07.826 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:07.854 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:07.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:07.855 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:07.956 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:07.992 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:07.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:07.993 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:08.094 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:08.130 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:08.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:08.131 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:08.147 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 71 16 105289 105273 92.6493 69.3125 0.00813314 0.0107883 2024-09-20T22:09:08.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:08.271 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:08.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:08.272 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:08.373 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:08.404 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:08.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:08.405 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:08.506 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:08.548 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:08.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:08.549 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:08.649 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:08.697 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:08.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:08.698 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:08.790 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T22:09:08.798 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:08.812 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:08 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pg_num", "val": "26"}]': finished 2024-09-20T22:09:08.812 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:08 smithi067 ceph-mon[33511]: osdmap e214: 8 total, 8 up, 8 in 2024-09-20T22:09:08.813 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:08 smithi067 ceph-mon[33511]: 5.b scrub starts 2024-09-20T22:09:08.813 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:08 smithi067 ceph-mon[33511]: 5.b scrub ok 2024-09-20T22:09:08.813 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:08 smithi067 ceph-mon[33511]: 5.14 scrub starts 2024-09-20T22:09:08.813 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:08 smithi067 ceph-mon[33511]: 5.14 scrub ok 2024-09-20T22:09:08.813 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:08 smithi067 ceph-mon[33511]: pgmap v318: 158 pgs: 1 active+remapped+backfilling, 157 active+clean; 6.3 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 73 MiB/s wr, 1.16k op/s; 4608/310692 objects misplaced (1.483%); 6.8 MiB/s, 109 objects/s recovering 2024-09-20T22:09:08.813 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:08 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-20T22:09:08.813 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:08 smithi067 ceph-mon[33511]: 5.2 deep-scrub starts 2024-09-20T22:09:08.813 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:08 smithi067 ceph-mon[33511]: 5.2 deep-scrub ok 2024-09-20T22:09:08.836 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:08.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:08.837 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:08.938 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:08.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:08.963 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:08.963 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:09.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pg_num", "val": "26"}]': finished 2024-09-20T22:09:09.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[29694]: osdmap e214: 8 total, 8 up, 8 in 2024-09-20T22:09:09.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[29694]: 5.b scrub starts 2024-09-20T22:09:09.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[29694]: 5.b scrub ok 2024-09-20T22:09:09.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[29694]: 5.14 scrub starts 2024-09-20T22:09:09.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[29694]: 5.14 scrub ok 2024-09-20T22:09:09.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[29694]: pgmap v318: 158 pgs: 1 active+remapped+backfilling, 157 active+clean; 6.3 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 73 MiB/s wr, 1.16k op/s; 4608/310692 objects misplaced (1.483%); 6.8 MiB/s, 109 objects/s recovering 2024-09-20T22:09:09.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-20T22:09:09.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[29694]: 5.2 deep-scrub starts 2024-09-20T22:09:09.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[29694]: 5.2 deep-scrub ok 2024-09-20T22:09:09.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pg_num", "val": "26"}]': finished 2024-09-20T22:09:09.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[40290]: osdmap e214: 8 total, 8 up, 8 in 2024-09-20T22:09:09.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[40290]: 5.b scrub starts 2024-09-20T22:09:09.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[40290]: 5.b scrub ok 2024-09-20T22:09:09.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[40290]: 5.14 scrub starts 2024-09-20T22:09:09.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[40290]: 5.14 scrub ok 2024-09-20T22:09:09.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[40290]: pgmap v318: 158 pgs: 1 active+remapped+backfilling, 157 active+clean; 6.3 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 73 MiB/s wr, 1.16k op/s; 4608/310692 objects misplaced (1.483%); 6.8 MiB/s, 109 objects/s recovering 2024-09-20T22:09:09.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pg_num_actual", "val": "26"}]: dispatch 2024-09-20T22:09:09.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[40290]: 5.2 deep-scrub starts 2024-09-20T22:09:09.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:08 smithi060 ceph-mon[40290]: 5.2 deep-scrub ok 2024-09-20T22:09:09.064 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:09.064 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:09:09.087 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:09.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:09.088 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:09.147 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 72 16 105673 105657 91.696 24 0.0321394 0.0108188 2024-09-20T22:09:09.189 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:09.213 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:09.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:09.214 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:09.315 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:09.339 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:09.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:09.340 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:09.441 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:09.463 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:09.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:09.464 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:09.565 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:09.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:09.619 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:09.619 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:09.638 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:09:09.638 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:09:09.638 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.638 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105717 8.18 8:1a687098:::benchmark_data_smithi060_111051_object105716:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e216)", 2024-09-20T22:09:09.638 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:09.627781+0000", 2024-09-20T22:09:09.638 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.007780304, 2024-09-20T22:09:09.639 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0078134050000000007, 2024-09-20T22:09:09.639 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:09.639 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:09.639 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:09:09.639 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:09.639 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:09:09.639 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:09.639 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105717 2024-09-20T22:09:09.639 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.639 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:09.639 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.639 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:09.639 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.627781+0000", 2024-09-20T22:09:09.639 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:09.640 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.640 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.640 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:09.640 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.627781+0000", 2024-09-20T22:09:09.640 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:09.640 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.640 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.640 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:09.640 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.627783+0000", 2024-09-20T22:09:09.640 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.046e-06 2024-09-20T22:09:09.640 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.640 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.640 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:09.641 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.627815+0000", 2024-09-20T22:09:09.641 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2128999999999997e-05 2024-09-20T22:09:09.641 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.641 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.641 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:09.641 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.627828+0000", 2024-09-20T22:09:09.641 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3089e-05 2024-09-20T22:09:09.641 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.641 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.641 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:09.641 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.627837+0000", 2024-09-20T22:09:09.641 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1749999999999994e-06 2024-09-20T22:09:09.641 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.641 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.642 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:09.642 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.628024+0000", 2024-09-20T22:09:09.642 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00018694999999999999 2024-09-20T22:09:09.642 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.642 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.642 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:09.642 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.628778+0000", 2024-09-20T22:09:09.642 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00075364500000000005 2024-09-20T22:09:09.642 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.642 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.642 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:09:09.642 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.629047+0000", 2024-09-20T22:09:09.642 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00026972900000000002 2024-09-20T22:09:09.643 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.643 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.643 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:09.643 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.630316+0000", 2024-09-20T22:09:09.643 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0012687309999999999 2024-09-20T22:09:09.643 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.643 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.643 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:09.643 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.634141+0000", 2024-09-20T22:09:09.643 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0038245750000000002 2024-09-20T22:09:09.643 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:09.643 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:09.643 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:09.644 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.644 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.644 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105718 8.d 8:b0c56a85:::benchmark_data_smithi060_111051_object105717:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e216)", 2024-09-20T22:09:09.644 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:09.627963+0000", 2024-09-20T22:09:09.644 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0075981349999999998, 2024-09-20T22:09:09.644 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0077331730000000003, 2024-09-20T22:09:09.644 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:09.644 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:09.644 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:09:09.644 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:09.644 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:09:09.644 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:09.644 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105718 2024-09-20T22:09:09.644 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.645 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:09.645 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.645 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:09.645 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.627963+0000", 2024-09-20T22:09:09.645 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:09.645 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.645 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.645 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:09.645 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.627963+0000", 2024-09-20T22:09:09.645 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:09.645 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.645 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.645 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:09.646 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.627964+0000", 2024-09-20T22:09:09.646 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.7300000000000005e-07 2024-09-20T22:09:09.646 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.646 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.646 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:09.646 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.627990+0000", 2024-09-20T22:09:09.646 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.6339e-05 2024-09-20T22:09:09.646 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.646 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.646 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:09.646 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.628001+0000", 2024-09-20T22:09:09.646 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0937999999999999e-05 2024-09-20T22:09:09.646 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.647 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.647 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:09.647 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.628009+0000", 2024-09-20T22:09:09.647 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6109999999999996e-06 2024-09-20T22:09:09.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.647 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.647 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:09.647 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.628070+0000", 2024-09-20T22:09:09.647 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1612000000000006e-05 2024-09-20T22:09:09.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.647 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.647 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:09.647 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.628216+0000", 2024-09-20T22:09:09.647 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014541399999999999 2024-09-20T22:09:09.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.648 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.648 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:09:09.648 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.628375+0000", 2024-09-20T22:09:09.648 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015907599999999999 2024-09-20T22:09:09.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.648 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.648 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:09.648 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.633745+0000", 2024-09-20T22:09:09.648 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0053703739999999998 2024-09-20T22:09:09.648 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:09.648 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:09.648 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:09.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.649 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.649 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105721 8.18 8:1c39a084:::benchmark_data_smithi060_111051_object105720:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e216)", 2024-09-20T22:09:09.649 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:09.628182+0000", 2024-09-20T22:09:09.649 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0073788639999999997, 2024-09-20T22:09:09.649 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0076014150000000003, 2024-09-20T22:09:09.649 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:09.649 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:09.649 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:09:09.649 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:09.649 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:09:09.649 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:09.649 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105721 2024-09-20T22:09:09.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.650 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:09.650 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.650 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:09.650 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.628182+0000", 2024-09-20T22:09:09.650 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:09.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.650 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.650 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:09.650 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.628182+0000", 2024-09-20T22:09:09.650 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:09.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.650 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.650 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:09.651 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.628183+0000", 2024-09-20T22:09:09.651 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.7000000000000003e-07 2024-09-20T22:09:09.651 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.651 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.651 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:09.651 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.630928+0000", 2024-09-20T22:09:09.651 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0027451939999999998 2024-09-20T22:09:09.651 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.651 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.651 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:09.651 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.630946+0000", 2024-09-20T22:09:09.651 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7915e-05 2024-09-20T22:09:09.651 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.652 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.652 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:09.652 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.630954+0000", 2024-09-20T22:09:09.652 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3699999999999995e-06 2024-09-20T22:09:09.652 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.652 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.652 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:09.652 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.631103+0000", 2024-09-20T22:09:09.653 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000148429 2024-09-20T22:09:09.653 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.653 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.653 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:09.653 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.631242+0000", 2024-09-20T22:09:09.653 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013935900000000001 2024-09-20T22:09:09.653 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.653 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.653 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:09:09.653 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.631485+0000", 2024-09-20T22:09:09.653 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000242995 2024-09-20T22:09:09.653 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.653 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.653 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:09.653 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.634167+0000", 2024-09-20T22:09:09.654 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0026813990000000001 2024-09-20T22:09:09.654 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:09.654 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:09.654 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:09.654 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.654 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.654 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105727 8.18 8:19f936bc:::benchmark_data_smithi060_111051_object105726:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e216)", 2024-09-20T22:09:09.654 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:09.631499+0000", 2024-09-20T22:09:09.654 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0040622009999999997, 2024-09-20T22:09:09.654 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0043633880000000002, 2024-09-20T22:09:09.654 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:09.654 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:09.654 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:09:09.655 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:09.655 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:09:09.655 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:09.655 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105727 2024-09-20T22:09:09.655 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.655 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:09.655 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.655 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:09.655 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.631499+0000", 2024-09-20T22:09:09.655 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:09.655 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.655 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.655 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:09.655 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.631499+0000", 2024-09-20T22:09:09.656 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:09.656 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.656 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.656 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:09.656 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.631500+0000", 2024-09-20T22:09:09.656 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.248e-06 2024-09-20T22:09:09.656 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.656 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.656 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:09.656 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.631548+0000", 2024-09-20T22:09:09.656 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.774e-05 2024-09-20T22:09:09.656 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.656 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.656 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:09.657 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.631559+0000", 2024-09-20T22:09:09.657 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1501000000000001e-05 2024-09-20T22:09:09.657 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.657 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.657 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:09.657 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.631566+0000", 2024-09-20T22:09:09.657 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8770000000000004e-06 2024-09-20T22:09:09.657 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.657 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.657 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:09.657 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.632538+0000", 2024-09-20T22:09:09.657 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00097178800000000001 2024-09-20T22:09:09.657 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.657 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.657 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:09.658 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.632659+0000", 2024-09-20T22:09:09.658 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012102199999999999 2024-09-20T22:09:09.658 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.658 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.658 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:09:09.658 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.632991+0000", 2024-09-20T22:09:09.658 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000332317 2024-09-20T22:09:09.658 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.658 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.658 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:09.658 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.634369+0000", 2024-09-20T22:09:09.658 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0013778429999999999 2024-09-20T22:09:09.658 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:09.658 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:09.659 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:09.659 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.659 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.659 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.15192.0:105724 8.c e216/206 8:3e3d52d7:::benchmark_data_smithi060_111051_object105723:head v 216'6590, mlcod=216'6588)", 2024-09-20T22:09:09.659 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:09.632863+0000", 2024-09-20T22:09:09.659 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0026974640000000001, 2024-09-20T22:09:09.659 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003070896, 2024-09-20T22:09:09.659 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:09.659 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:09.659 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:09:09.659 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:09.659 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.659 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:09.660 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.632863+0000", 2024-09-20T22:09:09.660 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:09.660 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.660 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.660 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:09.660 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.632863+0000", 2024-09-20T22:09:09.660 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:09.660 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.660 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.660 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:09.660 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.632864+0000", 2024-09-20T22:09:09.660 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4799999999999997e-07 2024-09-20T22:09:09.660 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.660 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.660 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:09.661 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.632870+0000", 2024-09-20T22:09:09.661 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0260000000000004e-06 2024-09-20T22:09:09.661 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.661 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.661 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:09.661 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.632879+0000", 2024-09-20T22:09:09.661 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.7139999999999999e-06 2024-09-20T22:09:09.661 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.661 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.661 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:09.661 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.632886+0000", 2024-09-20T22:09:09.661 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.533e-06 2024-09-20T22:09:09.661 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.661 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.662 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:09.662 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.633005+0000", 2024-09-20T22:09:09.662 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011958799999999999 2024-09-20T22:09:09.662 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.662 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.662 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:09.662 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.633026+0000", 2024-09-20T22:09:09.662 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0516000000000001e-05 2024-09-20T22:09:09.662 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:09.662 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:09.662 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:09.662 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.662 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.662 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.15192.0:105725 8.a e216/176 8:54104870:::benchmark_data_smithi060_111051_object105724:head v 216'6666, mlcod=214'6663)", 2024-09-20T22:09:09.663 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:09.632920+0000", 2024-09-20T22:09:09.663 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0026406369999999999, 2024-09-20T22:09:09.663 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0030687589999999999, 2024-09-20T22:09:09.663 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:09.663 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:09.663 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:09:09.663 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:09.663 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.663 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:09.663 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.632920+0000", 2024-09-20T22:09:09.663 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:09.663 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.663 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.663 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:09.664 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.632920+0000", 2024-09-20T22:09:09.664 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:09.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.664 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.664 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:09.664 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.632921+0000", 2024-09-20T22:09:09.664 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8100000000000003e-07 2024-09-20T22:09:09.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.664 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.664 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:09.664 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.632947+0000", 2024-09-20T22:09:09.664 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.6384e-05 2024-09-20T22:09:09.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.664 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.664 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:09.665 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.632955+0000", 2024-09-20T22:09:09.665 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7680000000000008e-06 2024-09-20T22:09:09.665 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.665 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.665 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:09.665 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.632960+0000", 2024-09-20T22:09:09.665 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0699999999999997e-06 2024-09-20T22:09:09.665 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.665 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.665 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:09.665 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.633729+0000", 2024-09-20T22:09:09.665 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00076908399999999996 2024-09-20T22:09:09.665 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.666 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.666 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:09.666 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.633747+0000", 2024-09-20T22:09:09.666 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7799999999999999e-05 2024-09-20T22:09:09.666 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:09.666 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:09.666 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:09.666 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.666 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.666 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.15192.0:105728 8.13 e216/215)", 2024-09-20T22:09:09.666 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:09.635152+0000", 2024-09-20T22:09:09.666 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.00040845999999999998, 2024-09-20T22:09:09.666 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00089017000000000002, 2024-09-20T22:09:09.666 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:09.667 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:09.667 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "queued for pg", 2024-09-20T22:09:09.667 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:09.667 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.667 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:09.667 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.635152+0000", 2024-09-20T22:09:09.667 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:09.667 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.667 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.667 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:09.667 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.635152+0000", 2024-09-20T22:09:09.667 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:09.667 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.667 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.668 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:09.668 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.635153+0000", 2024-09-20T22:09:09.668 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6099999999999999e-07 2024-09-20T22:09:09.668 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.668 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.668 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:09.668 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.635184+0000", 2024-09-20T22:09:09.668 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.082e-05 2024-09-20T22:09:09.668 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.668 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.668 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:09.668 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.635201+0000", 2024-09-20T22:09:09.668 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6305000000000001e-05 2024-09-20T22:09:09.668 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:09.668 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:09.668 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:09.668 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.635207+0000", 2024-09-20T22:09:09.669 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7909999999999999e-06 2024-09-20T22:09:09.669 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:09.669 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:09.669 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:09.669 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:09.669 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-20T22:09:09.669 INFO:teuthology.orchestra.run.smithi067.stdout: "num_ops": 7 2024-09-20T22:09:09.669 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:09:09.719 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:09.760 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:09.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:09.761 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:09 smithi067 ceph-mon[33511]: 6.8 scrub starts 2024-09-20T22:09:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:09 smithi067 ceph-mon[33511]: 6.8 scrub ok 2024-09-20T22:09:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:09 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-20T22:09:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:09 smithi067 ceph-mon[33511]: osdmap e215: 8 total, 8 up, 8 in 2024-09-20T22:09:09.862 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:09.900 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:09.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:09.901 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:09.999 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:09 smithi060 ceph-mon[29694]: 6.8 scrub starts 2024-09-20T22:09:09.999 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:09 smithi060 ceph-mon[29694]: 6.8 scrub ok 2024-09-20T22:09:09.999 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:09 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-20T22:09:09.999 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:09 smithi060 ceph-mon[29694]: osdmap e215: 8 total, 8 up, 8 in 2024-09-20T22:09:09.999 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:09 smithi060 ceph-mon[40290]: 6.8 scrub starts 2024-09-20T22:09:10.000 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:09 smithi060 ceph-mon[40290]: 6.8 scrub ok 2024-09-20T22:09:10.000 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:09 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pg_num_actual", "val": "26"}]': finished 2024-09-20T22:09:10.000 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:09 smithi060 ceph-mon[40290]: osdmap e215: 8 total, 8 up, 8 in 2024-09-20T22:09:10.002 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:10.056 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:10.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:10.057 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:10.147 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 73 16 106477 106461 91.1281 50.25 0.00476144 0.0109686 2024-09-20T22:09:10.159 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:10.212 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:10.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:10.213 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:10.270 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:10.314 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:10.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:10.350 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:10.350 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:10.451 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:10.468 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T22:09:10.496 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:10.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:10.497 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:10.597 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:10.631 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:10.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:10.633 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:10.733 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:10.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:10 smithi060 ceph-mon[29694]: osdmap e216: 8 total, 8 up, 8 in 2024-09-20T22:09:10.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:10 smithi060 ceph-mon[29694]: 8.0 scrub starts 2024-09-20T22:09:10.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:10 smithi060 ceph-mon[29694]: pgmap v321: 168 pgs: 10 unknown, 1 active+remapped+backfilling, 157 active+clean; 6.3 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 55 MiB/s wr, 872 op/s; 4608/310692 objects misplaced (1.483%); 10 MiB/s, 165 objects/s recovering 2024-09-20T22:09:10.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:10 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-20T22:09:10.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:10 smithi060 ceph-mon[29694]: 8.9 scrub starts 2024-09-20T22:09:10.744 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:10 smithi060 ceph-mon[40290]: osdmap e216: 8 total, 8 up, 8 in 2024-09-20T22:09:10.744 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:10 smithi060 ceph-mon[40290]: 8.0 scrub starts 2024-09-20T22:09:10.744 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:10 smithi060 ceph-mon[40290]: pgmap v321: 168 pgs: 10 unknown, 1 active+remapped+backfilling, 157 active+clean; 6.3 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 55 MiB/s wr, 872 op/s; 4608/310692 objects misplaced (1.483%); 10 MiB/s, 165 objects/s recovering 2024-09-20T22:09:10.744 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:10 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-20T22:09:10.744 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:10 smithi060 ceph-mon[40290]: 8.9 scrub starts 2024-09-20T22:09:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:10 smithi067 ceph-mon[33511]: osdmap e216: 8 total, 8 up, 8 in 2024-09-20T22:09:10.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:10 smithi067 ceph-mon[33511]: 8.0 scrub starts 2024-09-20T22:09:10.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:10 smithi067 ceph-mon[33511]: pgmap v321: 168 pgs: 10 unknown, 1 active+remapped+backfilling, 157 active+clean; 6.3 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 55 MiB/s wr, 872 op/s; 4608/310692 objects misplaced (1.483%); 10 MiB/s, 165 objects/s recovering 2024-09-20T22:09:10.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:10 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num_actual", "val": "26"}]: dispatch 2024-09-20T22:09:10.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:10 smithi067 ceph-mon[33511]: 8.9 scrub starts 2024-09-20T22:09:10.765 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:10.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:10.766 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:10.867 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:10.898 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:10.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:10.899 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:10.905 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:09:10.999 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:11.023 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:11.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:11.024 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:11.124 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:11.148 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 74 16 107013 106997 90.3494 33.5 0.00496373 0.0109809 2024-09-20T22:09:11.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:11.148 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:11.149 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:11.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:11.273 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:11.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:11.274 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:11.375 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:11.399 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:11.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:11.400 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:11.417 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:09:11.469 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:09:11.469 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:09:11.469 INFO:teuthology.orchestra.run.smithi067.stdout: "complaint_time": 30, 2024-09-20T22:09:11.469 INFO:teuthology.orchestra.run.smithi067.stdout: "num_blocked_ops": 0 2024-09-20T22:09:11.469 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:09:11.500 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:11.523 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:11.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:11.524 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:11.625 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:11.649 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:11.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:11.650 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:11.751 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:11.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:11 smithi067 ceph-mon[33511]: 8.1 scrub starts 2024-09-20T22:09:11.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:11 smithi067 ceph-mon[33511]: 8.2 scrub starts 2024-09-20T22:09:11.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:11 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-20T22:09:11.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:11 smithi067 ceph-mon[33511]: osdmap e217: 8 total, 8 up, 8 in 2024-09-20T22:09:11.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:11 smithi067 ceph-mon[33511]: 8.0 scrub ok 2024-09-20T22:09:11.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:11 smithi067 ceph-mon[33511]: 8.9 scrub ok 2024-09-20T22:09:11.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:11 smithi067 ceph-mon[33511]: 6.13 scrub starts 2024-09-20T22:09:11.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:11 smithi067 ceph-mon[33511]: 6.13 scrub ok 2024-09-20T22:09:11.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:11 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1273429096' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:09:11.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:11 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:09:11.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:11 smithi067 ceph-mon[33511]: 8.5 scrub starts 2024-09-20T22:09:11.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:11 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:09:11.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:11 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:09:11.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:11 smithi067 ceph-mon[33511]: osdmap e218: 8 total, 8 up, 8 in 2024-09-20T22:09:11.776 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:11.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:11.777 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[29694]: 8.1 scrub starts 2024-09-20T22:09:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[29694]: 8.2 scrub starts 2024-09-20T22:09:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-20T22:09:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[29694]: osdmap e217: 8 total, 8 up, 8 in 2024-09-20T22:09:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[29694]: 8.0 scrub ok 2024-09-20T22:09:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[29694]: 8.9 scrub ok 2024-09-20T22:09:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[29694]: 6.13 scrub starts 2024-09-20T22:09:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[29694]: 6.13 scrub ok 2024-09-20T22:09:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1273429096' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:09:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:09:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[29694]: 8.5 scrub starts 2024-09-20T22:09:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:09:11.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:09:11.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[29694]: osdmap e218: 8 total, 8 up, 8 in 2024-09-20T22:09:11.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[40290]: 8.1 scrub starts 2024-09-20T22:09:11.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[40290]: 8.2 scrub starts 2024-09-20T22:09:11.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_1", "var": "pgp_num_actual", "val": "26"}]': finished 2024-09-20T22:09:11.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[40290]: osdmap e217: 8 total, 8 up, 8 in 2024-09-20T22:09:11.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[40290]: 8.0 scrub ok 2024-09-20T22:09:11.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[40290]: 8.9 scrub ok 2024-09-20T22:09:11.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[40290]: 6.13 scrub starts 2024-09-20T22:09:11.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[40290]: 6.13 scrub ok 2024-09-20T22:09:11.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1273429096' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:09:11.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:09:11.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[40290]: 8.5 scrub starts 2024-09-20T22:09:11.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:09:11.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:09:11.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:11 smithi060 ceph-mon[40290]: osdmap e218: 8 total, 8 up, 8 in 2024-09-20T22:09:11.877 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:11.901 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:11.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:11.902 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:12.003 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:12.025 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:12.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:12.026 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:12.064 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:09:12.127 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:12.147 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 75 16 107013 106997 89.1449 0 - 0.0109809 2024-09-20T22:09:12.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:12.150 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:12.150 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:12.207 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:09:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:09:12.237 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T22:09:12.251 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:12.275 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:12.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:12.276 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:12.377 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:12.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:12.402 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:12.402 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:12.503 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:12.540 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:12.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:12.541 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:12.603 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:09:12.642 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:12.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:12.688 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:12.688 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:12.788 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:12.834 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:12.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:12.835 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:12.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:12 smithi060 ceph-mon[29694]: 8.1 scrub ok 2024-09-20T22:09:12.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:12 smithi060 ceph-mon[29694]: 8.2 scrub ok 2024-09-20T22:09:12.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:12 smithi060 ceph-mon[29694]: pgmap v324: 168 pgs: 1 active+remapped+backfilling, 2 active+clean+scrubbing, 3 remapped+peering, 7 unknown, 155 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 3200/309561 objects misplaced (1.034%); 24 MiB/s, 385 objects/s recovering 2024-09-20T22:09:12.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:12 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:09:12.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:12 smithi060 ceph-mon[29694]: 8.18 starting backfill to osd.4 from (0'0,0'0] MAX to 216'6726 2024-09-20T22:09:12.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:12 smithi060 ceph-mon[29694]: osdmap e219: 8 total, 8 up, 8 in 2024-09-20T22:09:12.860 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:12 smithi060 ceph-mon[40290]: 8.1 scrub ok 2024-09-20T22:09:12.860 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:12 smithi060 ceph-mon[40290]: 8.2 scrub ok 2024-09-20T22:09:12.860 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:12 smithi060 ceph-mon[40290]: pgmap v324: 168 pgs: 1 active+remapped+backfilling, 2 active+clean+scrubbing, 3 remapped+peering, 7 unknown, 155 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 3200/309561 objects misplaced (1.034%); 24 MiB/s, 385 objects/s recovering 2024-09-20T22:09:12.860 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:12 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:09:12.860 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:12 smithi060 ceph-mon[40290]: 8.18 starting backfill to osd.4 from (0'0,0'0] MAX to 216'6726 2024-09-20T22:09:12.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:12 smithi060 ceph-mon[40290]: osdmap e219: 8 total, 8 up, 8 in 2024-09-20T22:09:12.925 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:12 smithi067 ceph-mon[33511]: 8.1 scrub ok 2024-09-20T22:09:12.925 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:12 smithi067 ceph-mon[33511]: 8.2 scrub ok 2024-09-20T22:09:12.926 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:12 smithi067 ceph-mon[33511]: pgmap v324: 168 pgs: 1 active+remapped+backfilling, 2 active+clean+scrubbing, 3 remapped+peering, 7 unknown, 155 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 3200/309561 objects misplaced (1.034%); 24 MiB/s, 385 objects/s recovering 2024-09-20T22:09:12.926 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:12 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:09:12.926 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:12 smithi067 ceph-mon[33511]: 8.18 starting backfill to osd.4 from (0'0,0'0] MAX to 216'6726 2024-09-20T22:09:12.926 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:12 smithi067 ceph-mon[33511]: osdmap e219: 8 total, 8 up, 8 in 2024-09-20T22:09:12.936 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:12.978 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:12.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:12.979 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:13.058 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:13.079 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:13.121 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:13.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:13.122 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:13.148 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 76 16 107713 107697 88.5476 21.875 0.0206859 0.0112882 2024-09-20T22:09:13.222 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:13.254 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:09:13.254 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:09:13.254 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:09:13.254 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:09:13.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.255 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169974 7.10 7:0a082cf8:::benchmark_data_smithi060_78800_object36941:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:09:13.255 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:09:13.255 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 194.625455663, 2024-09-20T22:09:13.255 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87805208899999998, 2024-09-20T22:09:13.255 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.255 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.255 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:13.255 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:13.255 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:13.255 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:13.255 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169974 2024-09-20T22:09:13.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.255 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.255 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.256 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.256 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:09:13.256 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.256 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.256 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:09:13.256 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.256 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.256 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624616+0000", 2024-09-20T22:09:13.256 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1739999999999999e-06 2024-09-20T22:09:13.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.256 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.257 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624626+0000", 2024-09-20T22:09:13.257 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7909999999999999e-06 2024-09-20T22:09:13.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.257 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.257 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.257 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624654+0000", 2024-09-20T22:09:13.257 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.8524999999999999e-05 2024-09-20T22:09:13.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.257 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.257 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.257 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624665+0000", 2024-09-20T22:09:13.257 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0985000000000001e-05 2024-09-20T22:09:13.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.257 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.257 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.257 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.705237+0000", 2024-09-20T22:09:13.258 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.080571485999999998 2024-09-20T22:09:13.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.258 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.258 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:13.258 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.705245+0000", 2024-09-20T22:09:13.258 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8460000000000008e-06 2024-09-20T22:09:13.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.258 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.258 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.258 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498359+0000", 2024-09-20T22:09:13.258 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79311461999999999 2024-09-20T22:09:13.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.258 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.258 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:13.258 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498498+0000", 2024-09-20T22:09:13.259 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013918599999999999 2024-09-20T22:09:13.259 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.259 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.259 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:09:13.259 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498751+0000", 2024-09-20T22:09:13.259 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025235999999999998 2024-09-20T22:09:13.259 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.259 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.259 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.259 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.500697+0000", 2024-09-20T22:09:13.259 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0019460969999999999 2024-09-20T22:09:13.259 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.259 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.260 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:13.260 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502528+0000", 2024-09-20T22:09:13.260 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00183161 2024-09-20T22:09:13.260 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.260 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.260 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:13.260 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502597+0000", 2024-09-20T22:09:13.260 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8761000000000001e-05 2024-09-20T22:09:13.260 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.260 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.260 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.260 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502666+0000", 2024-09-20T22:09:13.260 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8647999999999997e-05 2024-09-20T22:09:13.260 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.260 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.261 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.261 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.261 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.261 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227618 7.0 7:0598f762:::benchmark_data_smithi060_78800_object94585:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-20T22:09:13.261 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:53.805369+0000", 2024-09-20T22:09:13.261 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 139.444700638, 2024-09-20T22:09:13.261 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2615457110000001, 2024-09-20T22:09:13.261 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.261 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.261 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:13.261 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:13.261 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:13.261 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:13.261 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227618 2024-09-20T22:09:13.261 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.261 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.261 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.262 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.262 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805369+0000", 2024-09-20T22:09:13.262 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.262 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.262 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.262 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.262 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805369+0000", 2024-09-20T22:09:13.262 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.262 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.262 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.262 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.262 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805371+0000", 2024-09-20T22:09:13.262 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.827e-06 2024-09-20T22:09:13.262 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.262 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.263 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.263 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805377+0000", 2024-09-20T22:09:13.263 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.161e-06 2024-09-20T22:09:13.263 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.263 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.263 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.263 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805398+0000", 2024-09-20T22:09:13.263 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1208000000000001e-05 2024-09-20T22:09:13.263 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.263 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.263 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.263 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805406+0000", 2024-09-20T22:09:13.263 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3820000000000004e-06 2024-09-20T22:09:13.263 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.263 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.264 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.264 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.813177+0000", 2024-09-20T22:09:13.264 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0077708919999999997 2024-09-20T22:09:13.264 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.264 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.264 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:13.264 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.813194+0000", 2024-09-20T22:09:13.264 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7107000000000001e-05 2024-09-20T22:09:13.264 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.264 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.264 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.264 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.813273+0000", 2024-09-20T22:09:13.264 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.000078609 2024-09-20T22:09:13.264 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.264 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.264 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:13.265 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.813290+0000", 2024-09-20T22:09:13.265 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7382999999999999e-05 2024-09-20T22:09:13.265 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.265 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.265 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.265 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.960476+0000", 2024-09-20T22:09:13.265 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1471854260000001 2024-09-20T22:09:13.265 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.265 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.265 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:13.265 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.960616+0000", 2024-09-20T22:09:13.265 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014035100000000001 2024-09-20T22:09:13.265 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.265 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.265 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,3", 2024-09-20T22:09:13.266 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.961013+0000", 2024-09-20T22:09:13.266 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00039649999999999999 2024-09-20T22:09:13.266 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.266 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.266 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:13.266 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.963511+0000", 2024-09-20T22:09:13.266 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0024986919999999998 2024-09-20T22:09:13.266 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.266 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.266 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.266 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.963553+0000", 2024-09-20T22:09:13.266 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1783999999999998e-05 2024-09-20T22:09:13.266 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.266 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.266 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.267 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.066713+0000", 2024-09-20T22:09:13.267 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.103160088 2024-09-20T22:09:13.267 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.267 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.268 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:13.268 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.066794+0000", 2024-09-20T22:09:13.268 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0924e-05 2024-09-20T22:09:13.268 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.268 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.268 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.268 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.066915+0000", 2024-09-20T22:09:13.268 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000120377 2024-09-20T22:09:13.268 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.268 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.268 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.268 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.269 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.269 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227635 7.b 7:da8bd73f:::benchmark_data_smithi060_78800_object94602:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e146)", 2024-09-20T22:09:13.269 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:54.820720+0000", 2024-09-20T22:09:13.269 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 138.42934976199999, 2024-09-20T22:09:13.269 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1621617440000001, 2024-09-20T22:09:13.269 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.269 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.269 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:13.269 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:13.269 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:13.269 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:13.269 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227635 2024-09-20T22:09:13.269 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.269 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.269 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.269 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.270 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820720+0000", 2024-09-20T22:09:13.270 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.270 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.270 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.270 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.270 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820720+0000", 2024-09-20T22:09:13.270 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.270 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.270 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.270 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.270 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820721+0000", 2024-09-20T22:09:13.270 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.593e-06 2024-09-20T22:09:13.270 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.270 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.271 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.271 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820728+0000", 2024-09-20T22:09:13.271 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3799999999999999e-06 2024-09-20T22:09:13.271 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.271 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.271 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.271 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820740+0000", 2024-09-20T22:09:13.271 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.279e-05 2024-09-20T22:09:13.271 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.271 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.271 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.271 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820749+0000", 2024-09-20T22:09:13.271 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8000000000000004e-06 2024-09-20T22:09:13.271 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.271 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.271 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.272 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820942+0000", 2024-09-20T22:09:13.272 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000192263 2024-09-20T22:09:13.272 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.272 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.272 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:13.272 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820953+0000", 2024-09-20T22:09:13.272 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1953000000000001e-05 2024-09-20T22:09:13.272 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.272 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.272 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.272 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.974865+0000", 2024-09-20T22:09:13.272 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.15391142 2024-09-20T22:09:13.272 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.272 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.272 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:13.273 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.974939+0000", 2024-09-20T22:09:13.273 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3798000000000001e-05 2024-09-20T22:09:13.273 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.273 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.273 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,6", 2024-09-20T22:09:13.273 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.975019+0000", 2024-09-20T22:09:13.273 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0692999999999995e-05 2024-09-20T22:09:13.273 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.273 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.273 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.273 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.977853+0000", 2024-09-20T22:09:13.273 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028332269999999998 2024-09-20T22:09:13.273 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.273 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.273 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.273 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982635+0000", 2024-09-20T22:09:13.274 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0047819840000000004 2024-09-20T22:09:13.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.274 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.274 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:13.274 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982688+0000", 2024-09-20T22:09:13.274 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.384e-05 2024-09-20T22:09:13.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.274 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.274 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:13.274 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982747+0000", 2024-09-20T22:09:13.274 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9004e-05 2024-09-20T22:09:13.274 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.274 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.274 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.274 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982881+0000", 2024-09-20T22:09:13.275 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013399899999999999 2024-09-20T22:09:13.275 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.275 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.275 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.275 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.275 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.275 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227641 7.b 7:d29062b1:::benchmark_data_smithi060_78800_object94608:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e146)", 2024-09-20T22:09:13.275 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:09:13.275 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 138.42929865100001, 2024-09-20T22:09:13.275 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2739681169999999, 2024-09-20T22:09:13.275 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.275 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.275 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:13.275 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:13.275 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:13.275 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:13.276 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227641 2024-09-20T22:09:13.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.276 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.276 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.276 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.276 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:09:13.276 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.276 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.276 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.276 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:09:13.276 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.276 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.276 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.276 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.276 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:09:13.277 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3900000000000005e-07 2024-09-20T22:09:13.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.277 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.277 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.277 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820775+0000", 2024-09-20T22:09:13.277 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4589999999999998e-06 2024-09-20T22:09:13.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.277 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.277 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.277 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820783+0000", 2024-09-20T22:09:13.277 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8949999999999998e-06 2024-09-20T22:09:13.277 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.277 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.277 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.277 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820788+0000", 2024-09-20T22:09:13.277 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5759999999999996e-06 2024-09-20T22:09:13.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.278 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.278 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.278 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820979+0000", 2024-09-20T22:09:13.278 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019122 2024-09-20T22:09:13.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.278 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.278 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:13.278 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820988+0000", 2024-09-20T22:09:13.278 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1480000000000006e-06 2024-09-20T22:09:13.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.278 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.278 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.278 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.977915+0000", 2024-09-20T22:09:13.278 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.156927834 2024-09-20T22:09:13.278 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.279 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.279 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:13.279 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.978045+0000", 2024-09-20T22:09:13.279 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012967 2024-09-20T22:09:13.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.279 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.279 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,6", 2024-09-20T22:09:13.279 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.978183+0000", 2024-09-20T22:09:13.279 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013767900000000001 2024-09-20T22:09:13.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.279 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.279 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.279 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982676+0000", 2024-09-20T22:09:13.279 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0044928210000000001 2024-09-20T22:09:13.279 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.279 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.280 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:13.280 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982891+0000", 2024-09-20T22:09:13.280 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00021542299999999999 2024-09-20T22:09:13.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.280 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.280 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.280 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.094638+0000", 2024-09-20T22:09:13.280 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.111746554 2024-09-20T22:09:13.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.280 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.280 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:13.280 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.094677+0000", 2024-09-20T22:09:13.280 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9069000000000002e-05 2024-09-20T22:09:13.280 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.280 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.280 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.281 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.094739+0000", 2024-09-20T22:09:13.281 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2230000000000006e-05 2024-09-20T22:09:13.281 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.281 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.281 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.281 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.281 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.281 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:31005 8.8 8:12186b0c:::benchmark_data_smithi060_111051_object31004:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e184)", 2024-09-20T22:09:13.281 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:08:09.453336+0000", 2024-09-20T22:09:13.281 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 63.796733670999998, 2024-09-20T22:09:13.281 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.14879588099999999, 2024-09-20T22:09:13.281 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.281 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.281 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:13.281 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:13.282 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:09:13.282 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:13.282 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 31005 2024-09-20T22:09:13.282 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.282 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.282 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.282 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.282 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.453336+0000", 2024-09-20T22:09:13.282 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.282 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.282 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.282 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.282 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.453336+0000", 2024-09-20T22:09:13.282 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.282 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.283 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.283 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.283 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.453337+0000", 2024-09-20T22:09:13.283 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.031e-06 2024-09-20T22:09:13.283 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.283 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.283 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.283 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.453415+0000", 2024-09-20T22:09:13.283 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8064000000000003e-05 2024-09-20T22:09:13.283 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.283 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.283 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.283 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.453424+0000", 2024-09-20T22:09:13.283 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5610000000000004e-06 2024-09-20T22:09:13.284 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.284 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.284 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.284 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.453431+0000", 2024-09-20T22:09:13.284 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1589999999999998e-06 2024-09-20T22:09:13.284 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.284 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.284 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.284 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.453479+0000", 2024-09-20T22:09:13.284 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8825000000000001e-05 2024-09-20T22:09:13.284 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.284 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.284 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:13.284 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.453561+0000", 2024-09-20T22:09:13.284 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1983000000000004e-05 2024-09-20T22:09:13.285 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.285 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.285 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:09:13.285 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.453638+0000", 2024-09-20T22:09:13.285 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6470000000000005e-05 2024-09-20T22:09:13.285 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.285 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.285 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:13.285 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.454908+0000", 2024-09-20T22:09:13.285 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001269903 2024-09-20T22:09:13.285 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.285 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.285 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.285 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.455027+0000", 2024-09-20T22:09:13.286 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011891 2024-09-20T22:09:13.286 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.286 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.286 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.286 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.602039+0000", 2024-09-20T22:09:13.286 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.14701270799999999 2024-09-20T22:09:13.286 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.286 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.286 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:13.286 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.602090+0000", 2024-09-20T22:09:13.286 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0411999999999998e-05 2024-09-20T22:09:13.286 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.286 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.287 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.287 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.602132+0000", 2024-09-20T22:09:13.287 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1854999999999998e-05 2024-09-20T22:09:13.287 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.287 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.287 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.287 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.287 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.287 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:31014 8.d 8:b73fa061:::benchmark_data_smithi060_111051_object31013:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e184)", 2024-09-20T22:09:13.287 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:08:09.456738+0000", 2024-09-20T22:09:13.287 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 63.793331817999999, 2024-09-20T22:09:13.287 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.14460163400000001, 2024-09-20T22:09:13.287 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.287 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.287 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:13.288 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:13.288 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:09:13.288 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:13.288 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 31014 2024-09-20T22:09:13.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.288 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.288 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.288 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.288 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.456738+0000", 2024-09-20T22:09:13.288 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.288 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.288 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.288 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.456738+0000", 2024-09-20T22:09:13.288 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.288 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.289 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.289 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.289 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.456739+0000", 2024-09-20T22:09:13.289 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5349999999999999e-06 2024-09-20T22:09:13.289 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.289 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.289 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.289 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.456816+0000", 2024-09-20T22:09:13.289 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6328000000000005e-05 2024-09-20T22:09:13.289 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.289 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.289 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.289 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.456828+0000", 2024-09-20T22:09:13.289 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2398999999999999e-05 2024-09-20T22:09:13.290 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.290 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.290 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.290 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.456835+0000", 2024-09-20T22:09:13.290 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4150000000000002e-06 2024-09-20T22:09:13.290 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.290 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.290 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.290 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.456882+0000", 2024-09-20T22:09:13.290 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6758000000000002e-05 2024-09-20T22:09:13.290 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.290 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.290 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:13.290 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.456968+0000", 2024-09-20T22:09:13.290 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6254000000000005e-05 2024-09-20T22:09:13.291 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.291 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.291 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:09:13.291 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.457058+0000", 2024-09-20T22:09:13.291 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9191999999999996e-05 2024-09-20T22:09:13.291 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.291 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.291 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.291 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.601189+0000", 2024-09-20T22:09:13.291 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.14413159 2024-09-20T22:09:13.291 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.291 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.291 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.291 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.601226+0000", 2024-09-20T22:09:13.291 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.697e-05 2024-09-20T22:09:13.292 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.292 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.292 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:13.292 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.601248+0000", 2024-09-20T22:09:13.292 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1705e-05 2024-09-20T22:09:13.292 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.292 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.292 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:13.292 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.601300+0000", 2024-09-20T22:09:13.292 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1844e-05 2024-09-20T22:09:13.292 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.292 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.292 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.292 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.601339+0000", 2024-09-20T22:09:13.292 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9644000000000002e-05 2024-09-20T22:09:13.292 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.293 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.293 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.293 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.293 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:13.293 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:13.293 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:13.294 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:31015 8.4 8:224fd3d7:::benchmark_data_smithi060_111051_object31014:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e184)", 2024-09-20T22:09:13.294 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:08:09.456856+0000", 2024-09-20T22:09:13.294 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 63.793213563999998, 2024-09-20T22:09:13.294 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.149031996, 2024-09-20T22:09:13.294 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.294 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.294 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:13.294 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:13.294 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:09:13.294 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:13.295 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 31015 2024-09-20T22:09:13.295 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.295 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.295 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.295 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.295 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.456856+0000", 2024-09-20T22:09:13.295 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.295 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.295 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.295 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.295 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.456856+0000", 2024-09-20T22:09:13.295 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.295 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.295 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.296 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.296 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.456856+0000", 2024-09-20T22:09:13.296 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9299999999999998e-07 2024-09-20T22:09:13.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.296 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.296 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.296 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.456883+0000", 2024-09-20T22:09:13.296 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.6169e-05 2024-09-20T22:09:13.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.296 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.296 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.296 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.456891+0000", 2024-09-20T22:09:13.296 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0800000000000006e-06 2024-09-20T22:09:13.296 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.296 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.297 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.297 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.456896+0000", 2024-09-20T22:09:13.297 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6779999999999997e-06 2024-09-20T22:09:13.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.297 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.297 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.297 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.456937+0000", 2024-09-20T22:09:13.297 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0488999999999998e-05 2024-09-20T22:09:13.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.297 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.297 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:13.297 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.457012+0000", 2024-09-20T22:09:13.297 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4820000000000005e-05 2024-09-20T22:09:13.297 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.297 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.298 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,4", 2024-09-20T22:09:13.298 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.457090+0000", 2024-09-20T22:09:13.298 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8353e-05 2024-09-20T22:09:13.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.298 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.298 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.298 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.602527+0000", 2024-09-20T22:09:13.298 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.14543661599999999 2024-09-20T22:09:13.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.298 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.298 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.298 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.602565+0000", 2024-09-20T22:09:13.298 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.8646999999999998e-05 2024-09-20T22:09:13.298 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.298 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.299 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:13.299 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.605833+0000", 2024-09-20T22:09:13.299 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003267416 2024-09-20T22:09:13.299 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.299 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.299 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:13.299 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.605864+0000", 2024-09-20T22:09:13.299 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0960000000000002e-05 2024-09-20T22:09:13.299 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.299 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.299 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.299 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.605888+0000", 2024-09-20T22:09:13.299 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.4175000000000001e-05 2024-09-20T22:09:13.299 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.300 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.300 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.300 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.300 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.300 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:31016 8.4 8:2f040713:::benchmark_data_smithi060_111051_object31015:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e184)", 2024-09-20T22:09:13.300 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:08:09.457020+0000", 2024-09-20T22:09:13.300 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 63.793049678000003, 2024-09-20T22:09:13.300 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.148918882, 2024-09-20T22:09:13.300 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.300 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.300 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:13.300 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:13.300 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:09:13.300 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:13.300 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 31016 2024-09-20T22:09:13.300 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.301 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.301 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.301 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.301 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.457020+0000", 2024-09-20T22:09:13.301 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.301 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.301 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.301 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.301 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.457020+0000", 2024-09-20T22:09:13.301 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.301 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.301 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.301 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.301 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.457020+0000", 2024-09-20T22:09:13.301 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8199999999999999e-07 2024-09-20T22:09:13.301 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.302 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.302 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.302 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.457082+0000", 2024-09-20T22:09:13.302 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1499000000000002e-05 2024-09-20T22:09:13.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.302 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.302 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.302 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.457091+0000", 2024-09-20T22:09:13.302 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2890000000000003e-06 2024-09-20T22:09:13.302 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.302 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.302 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.302 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.457097+0000", 2024-09-20T22:09:13.302 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9769999999999997e-06 2024-09-20T22:09:13.303 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.303 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.303 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.303 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.600987+0000", 2024-09-20T22:09:13.303 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.14389021199999999 2024-09-20T22:09:13.303 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.303 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.303 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:13.303 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.601154+0000", 2024-09-20T22:09:13.303 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00016654699999999999 2024-09-20T22:09:13.303 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.303 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.303 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,4", 2024-09-20T22:09:13.303 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.601304+0000", 2024-09-20T22:09:13.303 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000150347 2024-09-20T22:09:13.304 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.304 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.304 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.304 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.605752+0000", 2024-09-20T22:09:13.304 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0044477029999999999 2024-09-20T22:09:13.304 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.304 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.304 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.304 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.605791+0000", 2024-09-20T22:09:13.304 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.8594999999999999e-05 2024-09-20T22:09:13.304 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.304 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.304 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:13.304 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.605896+0000", 2024-09-20T22:09:13.304 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000105152 2024-09-20T22:09:13.304 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.305 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.305 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:13.305 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.605921+0000", 2024-09-20T22:09:13.305 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.4915000000000001e-05 2024-09-20T22:09:13.305 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.305 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.305 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.305 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.605939+0000", 2024-09-20T22:09:13.305 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7963999999999999e-05 2024-09-20T22:09:13.305 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.305 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.305 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.305 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.305 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.305 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:31018 8.8 8:1b2b9543:::benchmark_data_smithi060_111051_object31017:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e184)", 2024-09-20T22:09:13.305 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:08:09.457167+0000", 2024-09-20T22:09:13.306 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 63.792902112, 2024-09-20T22:09:13.306 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.14739764599999999, 2024-09-20T22:09:13.306 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.306 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.306 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:13.306 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:13.306 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:09:13.306 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:13.306 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 31018 2024-09-20T22:09:13.306 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.306 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.306 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.306 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.457167+0000", 2024-09-20T22:09:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.457167+0000", 2024-09-20T22:09:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.457168+0000", 2024-09-20T22:09:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.63e-07 2024-09-20T22:09:13.307 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.457236+0000", 2024-09-20T22:09:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8005999999999999e-05 2024-09-20T22:09:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.457244+0000", 2024-09-20T22:09:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9990000000000008e-06 2024-09-20T22:09:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.457249+0000", 2024-09-20T22:09:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9429999999999999e-06 2024-09-20T22:09:13.308 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.457292+0000", 2024-09-20T22:09:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3640999999999998e-05 2024-09-20T22:09:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.457399+0000", 2024-09-20T22:09:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000106945 2024-09-20T22:09:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:09:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.457518+0000", 2024-09-20T22:09:13.309 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000118658 2024-09-20T22:09:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.602169+0000", 2024-09-20T22:09:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.14465055600000001 2024-09-20T22:09:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.603132+0000", 2024-09-20T22:09:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00096369800000000003 2024-09-20T22:09:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:13.310 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.604469+0000", 2024-09-20T22:09:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001337232 2024-09-20T22:09:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.604525+0000", 2024-09-20T22:09:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5872999999999999e-05 2024-09-20T22:09:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.604565+0000", 2024-09-20T22:09:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9632000000000003e-05 2024-09-20T22:09:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.311 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop_reply(client.15192.0:31014 8.d e184/176 ondisk, result = 0)", 2024-09-20T22:09:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:08:09.458479+0000", 2024-09-20T22:09:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 63.791590905, 2024-09-20T22:09:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.14275339400000001, 2024-09-20T22:09:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:09:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.458479+0000", 2024-09-20T22:09:13.312 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.458479+0000", 2024-09-20T22:09:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.458480+0000", 2024-09-20T22:09:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.125e-06 2024-09-20T22:09:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.313 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.458484+0000", 2024-09-20T22:09:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2490000000000003e-06 2024-09-20T22:09:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.601055+0000", 2024-09-20T22:09:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.14257064799999999 2024-09-20T22:09:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.601064+0000", 2024-09-20T22:09:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3589999999999997e-06 2024-09-20T22:09:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.314 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.601216+0000", 2024-09-20T22:09:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000152254 2024-09-20T22:09:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.601222+0000", 2024-09-20T22:09:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4770000000000002e-06 2024-09-20T22:09:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.601232+0000", 2024-09-20T22:09:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0282000000000001e-05 2024-09-20T22:09:13.315 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop_reply(client.15192.0:31015 8.4 e184/176 ondisk, result = 0)", 2024-09-20T22:09:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:08:09.458684+0000", 2024-09-20T22:09:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 63.791385480999999, 2024-09-20T22:09:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.14384971399999999, 2024-09-20T22:09:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:09:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.316 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.458684+0000", 2024-09-20T22:09:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.458684+0000", 2024-09-20T22:09:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.458685+0000", 2024-09-20T22:09:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9699999999999995e-07 2024-09-20T22:09:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.317 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.458689+0000", 2024-09-20T22:09:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0060000000000003e-06 2024-09-20T22:09:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.600548+0000", 2024-09-20T22:09:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.14185889099999999 2024-09-20T22:09:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.600565+0000", 2024-09-20T22:09:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7357e-05 2024-09-20T22:09:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.318 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.602513+0000", 2024-09-20T22:09:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0019477730000000001 2024-09-20T22:09:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.602521+0000", 2024-09-20T22:09:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1480000000000006e-06 2024-09-20T22:09:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.602534+0000", 2024-09-20T22:09:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2741999999999999e-05 2024-09-20T22:09:13.319 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.320 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.320 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.320 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.320 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.320 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.15192.0:31013 8.3 e184/176 8:c65e4777:::benchmark_data_smithi060_111051_object31012:head v 184'1972, mlcod=184'1970)", 2024-09-20T22:09:13.320 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:08:09.459343+0000", 2024-09-20T22:09:13.320 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 63.790726595999999, 2024-09-20T22:09:13.320 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.14457510700000001, 2024-09-20T22:09:13.320 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.320 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.321 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:13.321 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.321 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.321 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.321 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.459343+0000", 2024-09-20T22:09:13.321 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.321 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.321 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.321 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.459343+0000", 2024-09-20T22:09:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.459344+0000", 2024-09-20T22:09:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5300000000000004e-07 2024-09-20T22:09:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.459438+0000", 2024-09-20T22:09:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4562999999999996e-05 2024-09-20T22:09:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.322 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.600874+0000", 2024-09-20T22:09:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.14143639399999999 2024-09-20T22:09:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.600885+0000", 2024-09-20T22:09:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0684e-05 2024-09-20T22:09:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.600986+0000", 2024-09-20T22:09:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100647 2024-09-20T22:09:13.323 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.324 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.324 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:13.324 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.601012+0000", 2024-09-20T22:09:13.324 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.5982e-05 2024-09-20T22:09:13.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.324 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.324 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:13.324 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.603868+0000", 2024-09-20T22:09:13.324 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028563579999999998 2024-09-20T22:09:13.324 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.324 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.324 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.324 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.603918+0000", 2024-09-20T22:09:13.324 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9826e-05 2024-09-20T22:09:13.324 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.325 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.325 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.325 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.325 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.325 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105650 8.4 8:29da2845:::benchmark_data_smithi060_111051_object105649:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e214)", 2024-09-20T22:09:13.325 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.523636+0000", 2024-09-20T22:09:13.325 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 4.7264336560000002, 2024-09-20T22:09:13.325 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.17888942399999999, 2024-09-20T22:09:13.325 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.325 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.325 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:09:13.325 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:13.325 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:09:13.326 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:13.326 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105650 2024-09-20T22:09:13.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.326 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.326 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.326 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.326 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523636+0000", 2024-09-20T22:09:13.326 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.326 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.326 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.326 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523636+0000", 2024-09-20T22:09:13.326 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.326 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.327 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.327 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.327 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523637+0000", 2024-09-20T22:09:13.327 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.686e-06 2024-09-20T22:09:13.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.327 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.327 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.327 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523679+0000", 2024-09-20T22:09:13.327 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1081999999999997e-05 2024-09-20T22:09:13.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.327 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.327 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.327 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523690+0000", 2024-09-20T22:09:13.327 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1545e-05 2024-09-20T22:09:13.327 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.328 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.328 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.328 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523696+0000", 2024-09-20T22:09:13.328 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3879999999999999e-06 2024-09-20T22:09:13.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.328 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.328 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.328 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523738+0000", 2024-09-20T22:09:13.328 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2460999999999999e-05 2024-09-20T22:09:13.328 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.328 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.328 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:13.328 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523826+0000", 2024-09-20T22:09:13.329 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8088000000000001e-05 2024-09-20T22:09:13.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.329 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.329 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,4", 2024-09-20T22:09:13.329 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523912+0000", 2024-09-20T22:09:13.329 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6074000000000003e-05 2024-09-20T22:09:13.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.329 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.329 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.329 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.528900+0000", 2024-09-20T22:09:13.329 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0049876720000000003 2024-09-20T22:09:13.329 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.329 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.329 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:13.330 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.529517+0000", 2024-09-20T22:09:13.330 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00061690400000000004 2024-09-20T22:09:13.330 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.330 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.330 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.330 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.702506+0000", 2024-09-20T22:09:13.330 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.17298888600000001 2024-09-20T22:09:13.330 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.330 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.330 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.330 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.702525+0000", 2024-09-20T22:09:13.330 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9638000000000001e-05 2024-09-20T22:09:13.330 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.331 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.331 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.331 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.331 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.331 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105650 8.14 8:29da2845:::benchmark_data_smithi060_111051_object105649:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:09:13.331 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.545671+0000", 2024-09-20T22:09:13.331 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 4.704398426, 2024-09-20T22:09:13.331 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.038415869, 2024-09-20T22:09:13.331 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.331 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.331 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:09:13.331 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:13.331 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:09:13.331 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:13.331 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105650 2024-09-20T22:09:13.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.332 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.332 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.332 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.332 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545671+0000", 2024-09-20T22:09:13.332 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.332 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.332 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.332 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545671+0000", 2024-09-20T22:09:13.332 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.332 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.332 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.332 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.332 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545673+0000", 2024-09-20T22:09:13.333 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.17e-06 2024-09-20T22:09:13.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.333 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.333 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.333 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545723+0000", 2024-09-20T22:09:13.333 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9509000000000003e-05 2024-09-20T22:09:13.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.333 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.333 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.333 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545741+0000", 2024-09-20T22:09:13.333 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8634000000000002e-05 2024-09-20T22:09:13.333 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.333 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.334 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.334 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545751+0000", 2024-09-20T22:09:13.334 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4420000000000004e-06 2024-09-20T22:09:13.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.334 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.334 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.334 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.707696+0000", 2024-09-20T22:09:13.334 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.16194560099999999 2024-09-20T22:09:13.334 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.334 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.334 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:13.334 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.707706+0000", 2024-09-20T22:09:13.334 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6860000000000001e-06 2024-09-20T22:09:13.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.335 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.335 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.335 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584005+0000", 2024-09-20T22:09:13.335 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87629895400000002 2024-09-20T22:09:13.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.335 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.335 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.335 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584087+0000", 2024-09-20T22:09:13.335 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1872999999999994e-05 2024-09-20T22:09:13.335 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.335 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.335 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.335 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.336 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.336 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "MOSDPGPush(8.c 215/206 [PushOp(8:355e0c9a:::benchmark_data_smithi060_111051_object46967:head, version: 186'2869, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8:355e0c9a:::benchmark_data_smithi060_111051_object46967:head@186'2869, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-20T22:09:13.336 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.555562+0000", 2024-09-20T22:09:13.336 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 4.6945070590000002, 2024-09-20T22:09:13.336 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.15072329400000001, 2024-09-20T22:09:13.336 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.336 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.336 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:09:13.336 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.336 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.336 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.336 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.555562+0000", 2024-09-20T22:09:13.336 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.336 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.555562+0000", 2024-09-20T22:09:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.555564+0000", 2024-09-20T22:09:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4139999999999999e-06 2024-09-20T22:09:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.555625+0000", 2024-09-20T22:09:13.337 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1282e-05 2024-09-20T22:09:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572138+0000", 2024-09-20T22:09:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.016512466999999999 2024-09-20T22:09:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572188+0000", 2024-09-20T22:09:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0773000000000003e-05 2024-09-20T22:09:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.338 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.706058+0000", 2024-09-20T22:09:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.13387001800000001 2024-09-20T22:09:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.706071+0000", 2024-09-20T22:09:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2876e-05 2024-09-20T22:09:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.706286+0000", 2024-09-20T22:09:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00021446400000000001 2024-09-20T22:09:13.339 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105665 8.18 8:1b9c2a37:::benchmark_data_smithi060_111051_object105664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:09:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.556635+0000", 2024-09-20T22:09:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 4.6934343969999999, 2024-09-20T22:09:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.051209888, 2024-09-20T22:09:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:09:13.340 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105665 2024-09-20T22:09:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556635+0000", 2024-09-20T22:09:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556635+0000", 2024-09-20T22:09:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.341 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556638+0000", 2024-09-20T22:09:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.6400000000000001e-06 2024-09-20T22:09:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556714+0000", 2024-09-20T22:09:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6594999999999994e-05 2024-09-20T22:09:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556736+0000", 2024-09-20T22:09:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1662e-05 2024-09-20T22:09:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.342 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556751+0000", 2024-09-20T22:09:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4941e-05 2024-09-20T22:09:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.750414+0000", 2024-09-20T22:09:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.193663171 2024-09-20T22:09:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.750424+0000", 2024-09-20T22:09:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0397999999999999e-05 2024-09-20T22:09:13.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.596184+0000", 2024-09-20T22:09:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.84575938799999995 2024-09-20T22:09:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.596303+0000", 2024-09-20T22:09:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000119503 2024-09-20T22:09:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:09:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.596588+0000", 2024-09-20T22:09:13.344 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00028446100000000001 2024-09-20T22:09:13.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.603603+0000", 2024-09-20T22:09:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0070155649999999996 2024-09-20T22:09:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607589+0000", 2024-09-20T22:09:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0039854110000000003 2024-09-20T22:09:13.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607743+0000", 2024-09-20T22:09:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015391499999999999 2024-09-20T22:09:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607802+0000", 2024-09-20T22:09:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9472000000000003e-05 2024-09-20T22:09:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607845+0000", 2024-09-20T22:09:13.347 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2765999999999997e-05 2024-09-20T22:09:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.15192.0:105664 8.c e215/206 8:37c43236:::benchmark_data_smithi060_111051_object105663:head v 215'6587, mlcod=214'6584)", 2024-09-20T22:09:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.572293+0000", 2024-09-20T22:09:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 4.6777762740000002, 2024-09-20T22:09:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.15892877499999999, 2024-09-20T22:09:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.348 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572293+0000", 2024-09-20T22:09:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572293+0000", 2024-09-20T22:09:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572294+0000", 2024-09-20T22:09:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.65e-07 2024-09-20T22:09:13.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572300+0000", 2024-09-20T22:09:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7599999999999997e-06 2024-09-20T22:09:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572308+0000", 2024-09-20T22:09:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2490000000000004e-06 2024-09-20T22:09:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.350 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572313+0000", 2024-09-20T22:09:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6590000000000004e-06 2024-09-20T22:09:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.703998+0000", 2024-09-20T22:09:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.13168523900000001 2024-09-20T22:09:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.704019+0000", 2024-09-20T22:09:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0267000000000001e-05 2024-09-20T22:09:13.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.731164+0000", 2024-09-20T22:09:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.027144784000000002 2024-09-20T22:09:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.731222+0000", 2024-09-20T22:09:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8452000000000002e-05 2024-09-20T22:09:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.352 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105673 8.14 8:2f269e39:::benchmark_data_smithi060_111051_object105672:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:09:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.761620+0000", 2024-09-20T22:09:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 4.4884498439999998, 2024-09-20T22:09:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.83821859799999998, 2024-09-20T22:09:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:09:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105673 2024-09-20T22:09:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.353 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.354 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761620+0000", 2024-09-20T22:09:13.354 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.354 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.354 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.354 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761620+0000", 2024-09-20T22:09:13.354 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.354 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.354 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.354 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761621+0000", 2024-09-20T22:09:13.354 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.392e-06 2024-09-20T22:09:13.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.355 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.355 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761674+0000", 2024-09-20T22:09:13.355 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2677999999999999e-05 2024-09-20T22:09:13.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.355 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.355 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761686+0000", 2024-09-20T22:09:13.355 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1926000000000001e-05 2024-09-20T22:09:13.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.355 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.355 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761697+0000", 2024-09-20T22:09:13.355 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1364e-05 2024-09-20T22:09:13.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.356 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.356 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.356 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761770+0000", 2024-09-20T22:09:13.356 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2904999999999994e-05 2024-09-20T22:09:13.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.356 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.356 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:13.356 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761777+0000", 2024-09-20T22:09:13.356 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8029999999999999e-06 2024-09-20T22:09:13.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.356 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.356 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.356 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584113+0000", 2024-09-20T22:09:13.357 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.82233672499999999 2024-09-20T22:09:13.357 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.357 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.357 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:13.357 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584256+0000", 2024-09-20T22:09:13.357 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000142686 2024-09-20T22:09:13.357 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.357 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.357 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,4", 2024-09-20T22:09:13.357 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584528+0000", 2024-09-20T22:09:13.357 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00027199499999999997 2024-09-20T22:09:13.357 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.357 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.357 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.357 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.590292+0000", 2024-09-20T22:09:13.358 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.005764333 2024-09-20T22:09:13.358 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.358 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.358 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:13.358 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.590315+0000", 2024-09-20T22:09:13.358 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2768e-05 2024-09-20T22:09:13.358 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.358 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.358 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.358 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.599740+0000", 2024-09-20T22:09:13.358 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0094248200000000004 2024-09-20T22:09:13.358 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.358 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.358 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:13.358 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.599795+0000", 2024-09-20T22:09:13.359 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5108000000000002e-05 2024-09-20T22:09:13.359 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.359 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.359 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.359 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.599838+0000", 2024-09-20T22:09:13.359 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3095e-05 2024-09-20T22:09:13.359 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.359 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.359 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.359 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.359 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.359 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:106978 8.19 8:9e674fff:::benchmark_data_smithi060_111051_object106977:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:09:13.359 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.576265+0000", 2024-09-20T22:09:13.359 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 2.6738047300000001, 2024-09-20T22:09:13.360 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9488434429999999, 2024-09-20T22:09:13.360 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.360 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.360 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:13.360 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:13.360 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:09:13.360 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:13.360 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 106978 2024-09-20T22:09:13.360 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.360 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.360 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.360 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.360 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576265+0000", 2024-09-20T22:09:13.360 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.361 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.361 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.361 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576265+0000", 2024-09-20T22:09:13.361 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.361 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.361 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.361 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576267+0000", 2024-09-20T22:09:13.361 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0509999999999999e-06 2024-09-20T22:09:13.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.361 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.361 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.361 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576299+0000", 2024-09-20T22:09:13.361 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1962000000000003e-05 2024-09-20T22:09:13.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.362 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.362 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576312+0000", 2024-09-20T22:09:13.362 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3423e-05 2024-09-20T22:09:13.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.362 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.363 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576322+0000", 2024-09-20T22:09:13.363 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0118e-05 2024-09-20T22:09:13.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.363 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.363 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576468+0000", 2024-09-20T22:09:13.363 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000145575 2024-09-20T22:09:13.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.363 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:13.363 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576481+0000", 2024-09-20T22:09:13.363 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3023999999999999e-05 2024-09-20T22:09:13.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.364 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.364 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410333+0000", 2024-09-20T22:09:13.364 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.83385187800000005 2024-09-20T22:09:13.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.364 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:13.364 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410345+0000", 2024-09-20T22:09:13.364 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.258e-05 2024-09-20T22:09:13.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.364 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.364 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.503436+0000", 2024-09-20T22:09:13.364 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.093091088 2024-09-20T22:09:13.365 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.365 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.365 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:13.365 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.503599+0000", 2024-09-20T22:09:13.365 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00016286999999999999 2024-09-20T22:09:13.365 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.365 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.365 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4,6", 2024-09-20T22:09:13.365 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.505264+0000", 2024-09-20T22:09:13.365 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001664591 2024-09-20T22:09:13.365 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.365 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.365 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.365 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.508504+0000", 2024-09-20T22:09:13.366 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0032404920000000002 2024-09-20T22:09:13.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.366 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.366 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.366 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.508547+0000", 2024-09-20T22:09:13.366 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2734000000000001e-05 2024-09-20T22:09:13.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.366 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.366 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:13.366 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.513180+0000", 2024-09-20T22:09:13.366 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046330490000000002 2024-09-20T22:09:13.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.366 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.366 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:13.367 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.524891+0000", 2024-09-20T22:09:13.367 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.011710379999999999 2024-09-20T22:09:13.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.367 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.367 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:13.367 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.524966+0000", 2024-09-20T22:09:13.367 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5290999999999995e-05 2024-09-20T22:09:13.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.367 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.367 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.367 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.525108+0000", 2024-09-20T22:09:13.367 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000142337 2024-09-20T22:09:13.367 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.367 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.368 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.368 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.368 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.368 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:107002 8.11 8.e0f52171 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:09:13.368 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.617682+0000", 2024-09-20T22:09:13.368 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 2.6323872320000001, 2024-09-20T22:09:13.368 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79265608300000001, 2024-09-20T22:09:13.368 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:13.368 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:13.368 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:09:13.368 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:13.368 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:09:13.368 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:13.368 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 107002 2024-09-20T22:09:13.369 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.369 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:13.369 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.369 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:13.369 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617682+0000", 2024-09-20T22:09:13.369 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.369 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.369 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.369 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:13.369 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617682+0000", 2024-09-20T22:09:13.369 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:13.369 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.369 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.369 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617684+0000", 2024-09-20T22:09:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1299999999999999e-06 2024-09-20T22:09:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617725+0000", 2024-09-20T22:09:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0839e-05 2024-09-20T22:09:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617738+0000", 2024-09-20T22:09:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2310000000000001e-05 2024-09-20T22:09:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.370 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617746+0000", 2024-09-20T22:09:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3939999999999996e-06 2024-09-20T22:09:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617856+0000", 2024-09-20T22:09:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000109917 2024-09-20T22:09:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617878+0000", 2024-09-20T22:09:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2495999999999999e-05 2024-09-20T22:09:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.371 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410330+0000", 2024-09-20T22:09:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79245202299999995 2024-09-20T22:09:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410338+0000", 2024-09-20T22:09:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9740000000000002e-06 2024-09-20T22:09:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:13.372 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:13.372 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:09:13.394 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:13.424 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:13.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:13.425 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:13.526 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:13.569 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:13.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:13.570 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:13.671 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:13.705 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:13.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:13.707 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:13.807 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:13.845 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[29694]: 8.10 starting backfill to osd.4 from (0'0,0'0] MAX to 216'6685 2024-09-20T22:09:13.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[29694]: 8.16 starting backfill to osd.2 from (0'0,0'0] MAX to 216'6607 2024-09-20T22:09:13.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[29694]: 8.11 starting backfill to osd.5 from (0'0,0'0] MAX to 216'6591 2024-09-20T22:09:13.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[29694]: 8.19 starting backfill to osd.4 from (0'0,0'0] MAX to 216'6621 2024-09-20T22:09:13.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[29694]: 8.12 starting backfill to osd.2 from (0'0,0'0] MAX to 216'6592 2024-09-20T22:09:13.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[29694]: 8.13 starting backfill to osd.2 from (0'0,0'0] MAX to 216'6651 2024-09-20T22:09:13.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[29694]: 8.17 starting backfill to osd.0 from (0'0,0'0] MAX to 216'6660 2024-09-20T22:09:13.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[29694]: 8.14 starting backfill to osd.0 from (0'0,0'0] MAX to 216'6522 2024-09-20T22:09:13.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[29694]: 8.15 starting backfill to osd.1 from (0'0,0'0] MAX to 216'6755 2024-09-20T22:09:13.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[29694]: 8.10 starting backfill to osd.7 from (0'0,0'0] MAX to 216'6685 2024-09-20T22:09:13.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[29694]: 8.17 starting backfill to osd.7 from (0'0,0'0] MAX to 216'6660 2024-09-20T22:09:13.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[29694]: 8.13 starting backfill to osd.3 from (0'0,0'0] MAX to 216'6651 2024-09-20T22:09:13.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[29694]: 8.11 starting backfill to osd.6 from (0'0,0'0] MAX to 216'6591 2024-09-20T22:09:13.846 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[29694]: 8.12 starting backfill to osd.6 from (0'0,0'0] MAX to 216'6592 2024-09-20T22:09:13.847 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[29694]: 8.11 starting backfill to osd.7 from (0'0,0'0] MAX to 216'6591 2024-09-20T22:09:13.847 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[29694]: 8.12 starting backfill to osd.7 from (0'0,0'0] MAX to 216'6592 2024-09-20T22:09:13.847 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[29694]: 8.16 starting backfill to osd.3 from (0'0,0'0] MAX to 216'6607 2024-09-20T22:09:13.847 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[29694]: 8.16 starting backfill to osd.5 from (0'0,0'0] MAX to 216'6607 2024-09-20T22:09:13.847 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[40290]: 8.10 starting backfill to osd.4 from (0'0,0'0] MAX to 216'6685 2024-09-20T22:09:13.847 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[40290]: 8.16 starting backfill to osd.2 from (0'0,0'0] MAX to 216'6607 2024-09-20T22:09:13.847 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[40290]: 8.11 starting backfill to osd.5 from (0'0,0'0] MAX to 216'6591 2024-09-20T22:09:13.847 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[40290]: 8.19 starting backfill to osd.4 from (0'0,0'0] MAX to 216'6621 2024-09-20T22:09:13.847 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[40290]: 8.12 starting backfill to osd.2 from (0'0,0'0] MAX to 216'6592 2024-09-20T22:09:13.847 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[40290]: 8.13 starting backfill to osd.2 from (0'0,0'0] MAX to 216'6651 2024-09-20T22:09:13.847 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[40290]: 8.17 starting backfill to osd.0 from (0'0,0'0] MAX to 216'6660 2024-09-20T22:09:13.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[40290]: 8.14 starting backfill to osd.0 from (0'0,0'0] MAX to 216'6522 2024-09-20T22:09:13.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[40290]: 8.15 starting backfill to osd.1 from (0'0,0'0] MAX to 216'6755 2024-09-20T22:09:13.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[40290]: 8.10 starting backfill to osd.7 from (0'0,0'0] MAX to 216'6685 2024-09-20T22:09:13.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[40290]: 8.17 starting backfill to osd.7 from (0'0,0'0] MAX to 216'6660 2024-09-20T22:09:13.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[40290]: 8.13 starting backfill to osd.3 from (0'0,0'0] MAX to 216'6651 2024-09-20T22:09:13.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[40290]: 8.11 starting backfill to osd.6 from (0'0,0'0] MAX to 216'6591 2024-09-20T22:09:13.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[40290]: 8.12 starting backfill to osd.6 from (0'0,0'0] MAX to 216'6592 2024-09-20T22:09:13.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[40290]: 8.11 starting backfill to osd.7 from (0'0,0'0] MAX to 216'6591 2024-09-20T22:09:13.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[40290]: 8.12 starting backfill to osd.7 from (0'0,0'0] MAX to 216'6592 2024-09-20T22:09:13.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[40290]: 8.16 starting backfill to osd.3 from (0'0,0'0] MAX to 216'6607 2024-09-20T22:09:13.848 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:13 smithi060 ceph-mon[40290]: 8.16 starting backfill to osd.5 from (0'0,0'0] MAX to 216'6607 2024-09-20T22:09:13.849 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:13.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:13.850 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:13.878 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:13 smithi067 ceph-mon[33511]: 8.10 starting backfill to osd.4 from (0'0,0'0] MAX to 216'6685 2024-09-20T22:09:13.878 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:13 smithi067 ceph-mon[33511]: 8.16 starting backfill to osd.2 from (0'0,0'0] MAX to 216'6607 2024-09-20T22:09:13.878 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:13 smithi067 ceph-mon[33511]: 8.11 starting backfill to osd.5 from (0'0,0'0] MAX to 216'6591 2024-09-20T22:09:13.878 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:13 smithi067 ceph-mon[33511]: 8.19 starting backfill to osd.4 from (0'0,0'0] MAX to 216'6621 2024-09-20T22:09:13.878 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:13 smithi067 ceph-mon[33511]: 8.12 starting backfill to osd.2 from (0'0,0'0] MAX to 216'6592 2024-09-20T22:09:13.879 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:13 smithi067 ceph-mon[33511]: 8.13 starting backfill to osd.2 from (0'0,0'0] MAX to 216'6651 2024-09-20T22:09:13.879 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:13 smithi067 ceph-mon[33511]: 8.17 starting backfill to osd.0 from (0'0,0'0] MAX to 216'6660 2024-09-20T22:09:13.879 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:13 smithi067 ceph-mon[33511]: 8.14 starting backfill to osd.0 from (0'0,0'0] MAX to 216'6522 2024-09-20T22:09:13.879 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:13 smithi067 ceph-mon[33511]: 8.15 starting backfill to osd.1 from (0'0,0'0] MAX to 216'6755 2024-09-20T22:09:13.879 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:13 smithi067 ceph-mon[33511]: 8.10 starting backfill to osd.7 from (0'0,0'0] MAX to 216'6685 2024-09-20T22:09:13.879 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:13 smithi067 ceph-mon[33511]: 8.17 starting backfill to osd.7 from (0'0,0'0] MAX to 216'6660 2024-09-20T22:09:13.879 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:13 smithi067 ceph-mon[33511]: 8.13 starting backfill to osd.3 from (0'0,0'0] MAX to 216'6651 2024-09-20T22:09:13.879 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:13 smithi067 ceph-mon[33511]: 8.11 starting backfill to osd.6 from (0'0,0'0] MAX to 216'6591 2024-09-20T22:09:13.879 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:13 smithi067 ceph-mon[33511]: 8.12 starting backfill to osd.6 from (0'0,0'0] MAX to 216'6592 2024-09-20T22:09:13.879 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:13 smithi067 ceph-mon[33511]: 8.11 starting backfill to osd.7 from (0'0,0'0] MAX to 216'6591 2024-09-20T22:09:13.879 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:13 smithi067 ceph-mon[33511]: 8.12 starting backfill to osd.7 from (0'0,0'0] MAX to 216'6592 2024-09-20T22:09:13.879 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:13 smithi067 ceph-mon[33511]: 8.16 starting backfill to osd.3 from (0'0,0'0] MAX to 216'6607 2024-09-20T22:09:13.879 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:13 smithi067 ceph-mon[33511]: 8.16 starting backfill to osd.5 from (0'0,0'0] MAX to 216'6607 2024-09-20T22:09:13.951 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:13.981 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:09:13.981 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:09:13.981 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:09:13.981 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:09:13.981 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:09:13.982 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:09:13.982 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:09:13.982 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:09:13.982 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:09:13.982 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:09:13.987 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:13.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:13.988 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:14.001 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:09:14.002 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:09:14.038 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:09:14.038 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:09:14.087 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:09:14.087 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:09:14.088 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:14.132 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:14.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:14.133 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:14.147 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 77 16 108810 108794 88.288 68.5625 0.00829796 0.0113216 2024-09-20T22:09:14.234 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:14.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:14.269 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:14.269 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:14.370 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:14.408 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:14.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:14.409 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:14.510 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:14.514 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T22:09:14.534 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:14.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:14.535 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:14.636 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:14.676 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:14.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:14.677 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:14 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2303511328' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:09:14.776 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:14 smithi067 ceph-mon[33511]: pgmap v326: 168 pgs: 3 active+remapped+backfilling, 2 active+clean+scrubbing, 3 remapped+peering, 4 unknown, 1 active+remapped+backfill_toofull, 155 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 23284/311616 objects misplaced (7.472%); 22 MiB/s, 354 objects/s recovering 2024-09-20T22:09:14.778 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:14.816 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:14.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:14.817 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:14.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:09:14.918 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:14.953 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:14 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2303511328' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:09:14.953 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:14 smithi060 ceph-mon[29694]: pgmap v326: 168 pgs: 3 active+remapped+backfilling, 2 active+clean+scrubbing, 3 remapped+peering, 4 unknown, 1 active+remapped+backfill_toofull, 155 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 23284/311616 objects misplaced (7.472%); 22 MiB/s, 354 objects/s recovering 2024-09-20T22:09:14.953 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:14 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2303511328' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:09:14.954 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:14 smithi060 ceph-mon[40290]: pgmap v326: 168 pgs: 3 active+remapped+backfilling, 2 active+clean+scrubbing, 3 remapped+peering, 4 unknown, 1 active+remapped+backfill_toofull, 155 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 23284/311616 objects misplaced (7.472%); 22 MiB/s, 354 objects/s recovering 2024-09-20T22:09:14.956 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:09:15.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:15.009 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:15.009 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:15.110 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:15.148 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 78 16 109637 109621 87.8181 51.6875 0.0220343 0.0113819 2024-09-20T22:09:15.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:15.159 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:15.159 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:15.245 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:09:15.246 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:09:15.246 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-09-20T22:09:15.246 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-20T22:09:15.246 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-20T22:09:15.246 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 config set filestore_inject_stall 3 2024-09-20T22:09:15.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:15.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:09:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:09:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:09:15.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:15.308 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:15.308 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:15.409 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:15.434 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:15.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:15.435 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:15.536 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:15.577 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:15.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:15.578 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:15.618 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:09:15.618 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:09:15.679 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:15.702 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:15.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:15.703 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:15 smithi067 ceph-mon[33511]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:09:15.803 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:15.841 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:15.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:15.842 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:15.943 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:15.965 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:15.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:15.966 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:16.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:15 smithi060 ceph-mon[29694]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:09:16.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:15 smithi060 ceph-mon[40290]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:09:16.068 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:16.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:16.105 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:16.105 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:16.148 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 79 15 110475 110460 87.3703 52.4375 0.0150283 0.0114409 2024-09-20T22:09:16.206 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:16.239 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:16.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:16.240 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:16.341 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:16.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:16.375 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:16.376 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:16.476 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:16.504 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:16.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:16.505 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:16.605 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:16.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:16.649 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:16.649 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:16.750 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:16.783 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:16.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:16.784 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:16.885 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:16.920 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:16.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:16.921 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:17.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:16 smithi067 ceph-mon[33511]: pgmap v327: 168 pgs: 5 active+remapped+backfilling, 3 remapped+peering, 1 unknown, 2 active+remapped+backfill_toofull, 157 active+clean; 6.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 32360/314541 objects misplaced (10.288%); 29 MiB/s, 467 objects/s recovering 2024-09-20T22:09:17.021 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:17.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:16 smithi060 ceph-mon[29694]: pgmap v327: 168 pgs: 5 active+remapped+backfilling, 3 remapped+peering, 1 unknown, 2 active+remapped+backfill_toofull, 157 active+clean; 6.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 32360/314541 objects misplaced (10.288%); 29 MiB/s, 467 objects/s recovering 2024-09-20T22:09:17.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:16 smithi060 ceph-mon[40290]: pgmap v327: 168 pgs: 5 active+remapped+backfilling, 3 remapped+peering, 1 unknown, 2 active+remapped+backfill_toofull, 157 active+clean; 6.4 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s; 32360/314541 objects misplaced (10.288%); 29 MiB/s, 467 objects/s recovering 2024-09-20T22:09:17.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:17.064 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:17.065 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:17.149 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:09:17.148481+0000 min lat: 0.00126498 max lat: 2.31982 avg lat: 0.0114992 2024-09-20T22:09:17.149 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:09:17.149 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 80 16 111278 111262 86.9046 50.125 0.00380215 0.0114992 2024-09-20T22:09:17.165 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:17.193 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:17.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:17.194 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:17.295 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:17.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:17.319 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:17.319 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:17.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:17.463 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:17.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:17.465 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:17.538 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T22:09:17.565 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:09:17.565 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:17.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:17.614 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:17.614 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:17.714 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:17.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:17.765 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:17.765 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:17.866 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:17.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:17.894 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:17.895 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:17.995 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:18.040 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:18.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:18.041 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:18.141 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:18.150 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 81 16 112127 112111 86.4856 53.0625 0.0160458 0.0115576 2024-09-20T22:09:18.182 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:18.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:18.183 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:18.275 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:09:18.275 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "" 2024-09-20T22:09:18.275 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:09:18.281 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:09:18.284 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:18.308 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:18.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:18.310 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:18.410 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:18.446 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:18.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:18.447 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:18.548 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:18.586 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:18.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:18.587 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:18.688 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:18.733 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:18.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:18.734 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:18.835 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:18.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:18.873 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:18.873 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:18.974 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:19.007 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:19.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:19.008 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:19.109 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:19.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:19.150 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:19.150 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:19.150 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 82 16 113065 113049 86.1458 58.625 0.013359 0.0116033 2024-09-20T22:09:19.251 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:19.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:19.290 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:19.290 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:19.391 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:19.409 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:19 smithi067 ceph-mon[33511]: pgmap v328: 168 pgs: 7 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 62999/332709 objects misplaced (18.935%); 48 MiB/s, 769 objects/s recovering 2024-09-20T22:09:19.413 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:19.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:19.414 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:19.423 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:09:19.423 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:09:19.514 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:19.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:19 smithi060 ceph-mon[29694]: pgmap v328: 168 pgs: 7 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 62999/332709 objects misplaced (18.935%); 48 MiB/s, 769 objects/s recovering 2024-09-20T22:09:19.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:19 smithi060 ceph-mon[40290]: pgmap v328: 168 pgs: 7 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s; 62999/332709 objects misplaced (18.935%); 48 MiB/s, 769 objects/s recovering 2024-09-20T22:09:19.553 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:19.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:19.554 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:19.655 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:19.690 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:19.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:19.691 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:19.792 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:19.829 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:19.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:19.831 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:19.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:19.953 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:19.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:19.954 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:20.055 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:20.087 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:20.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:20.088 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:20.150 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 83 16 113965 113949 85.7856 56.25 0.0181201 0.0116526 2024-09-20T22:09:20.159 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T22:09:20.189 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:20.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:20.275 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:20.275 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:20.376 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:20.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:20.416 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:20.417 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:20.517 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:20.560 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:20.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:20.561 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:20.662 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:20.695 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:20.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:20.696 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:20.797 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:20.799 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:09:20.841 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:20.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:20.843 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:20.943 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:20.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:20.967 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:20.967 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:21.068 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:21.108 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:21.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:21.109 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:21.150 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 84 16 114785 114769 85.3745 51.25 0.00993987 0.011708 2024-09-20T22:09:21.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:21.254 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:21.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:21.255 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:21.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:21 smithi067 ceph-mon[33511]: pgmap v329: 168 pgs: 7 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 592 B/s rd, 36 MiB/s wr, 581 op/s; 62999/332709 objects misplaced (18.935%); 42 MiB/s, 667 objects/s recovering 2024-09-20T22:09:21.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:09:21.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:09:21.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:21 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:09:21.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:21 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:09:21.356 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:21.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:21.401 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:21.401 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:21.444 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:09:21.445 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:09:21.502 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:21.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:21 smithi060 ceph-mon[29694]: pgmap v329: 168 pgs: 7 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 592 B/s rd, 36 MiB/s wr, 581 op/s; 62999/332709 objects misplaced (18.935%); 42 MiB/s, 667 objects/s recovering 2024-09-20T22:09:21.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:09:21.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:09:21.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:21 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:09:21.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:21 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:09:21.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:21 smithi060 ceph-mon[40290]: pgmap v329: 168 pgs: 7 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 6.8 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 592 B/s rd, 36 MiB/s wr, 581 op/s; 62999/332709 objects misplaced (18.935%); 42 MiB/s, 667 objects/s recovering 2024-09-20T22:09:21.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:09:21.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:09:21.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:21 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:09:21.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:21 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:09:21.538 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:21.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:21.540 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:21.641 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:21.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:21.679 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:21.679 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:21.780 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:21.810 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:21.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:21.812 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:21.912 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:21.950 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:21.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:21.951 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:22.051 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:22.084 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:22.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:22.085 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:22.150 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 85 16 115754 115738 85.0826 60.5625 0.0180249 0.0117487 2024-09-20T22:09:22.159 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:09:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:09:22.186 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:22.211 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T22:09:22.222 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:22.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:22.223 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:22.324 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:22.364 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:22.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:22.365 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:22.430 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:22 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:09:22.430 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:22 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:09:22.431 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:22 smithi060 ceph-mon[40290]: osdmap e220: 8 total, 8 up, 8 in 2024-09-20T22:09:22.431 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:22 smithi060 ceph-mon[40290]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:09:22.431 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:22 smithi060 ceph-mon[40290]: osdmap e221: 8 total, 8 up, 8 in 2024-09-20T22:09:22.431 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:22 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:09:22.431 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:22 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:09:22.431 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:22 smithi060 ceph-mon[29694]: osdmap e220: 8 total, 8 up, 8 in 2024-09-20T22:09:22.431 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:22 smithi060 ceph-mon[29694]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:09:22.431 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:22 smithi060 ceph-mon[29694]: osdmap e221: 8 total, 8 up, 8 in 2024-09-20T22:09:22.466 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:22.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:22.506 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:22.506 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:22.511 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:22.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:22 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:09:22.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:22 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:09:22.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:22 smithi067 ceph-mon[33511]: osdmap e220: 8 total, 8 up, 8 in 2024-09-20T22:09:22.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:22 smithi067 ceph-mon[33511]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:09:22.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:22 smithi067 ceph-mon[33511]: osdmap e221: 8 total, 8 up, 8 in 2024-09-20T22:09:22.607 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:22.648 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:09:22.651 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:22.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:22.653 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:22.753 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:22.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:22.795 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:22.796 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:22.896 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:22.941 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:22.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:22.942 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:23.043 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:23.078 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:23.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:23.079 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:23.150 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 86 16 116846 116830 84.8868 68.25 0.0328459 0.0117748 2024-09-20T22:09:23.180 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:23.205 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:23.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:23.206 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:23.307 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:23.341 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:23.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:23.342 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:23.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:23.487 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:23.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:23.488 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:23.589 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:23.627 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:23.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:23.628 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:23.728 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:23.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:23 smithi067 ceph-mon[33511]: pgmap v332: 168 pgs: 7 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 29 MiB/s wr, 464 op/s; 60686/342717 objects misplaced (17.707%); 58 MiB/s, 921 objects/s recovering 2024-09-20T22:09:23.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:23 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:09:23.767 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:23.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:23.768 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:23.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:23 smithi060 ceph-mon[40290]: pgmap v332: 168 pgs: 7 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 29 MiB/s wr, 464 op/s; 60686/342717 objects misplaced (17.707%); 58 MiB/s, 921 objects/s recovering 2024-09-20T22:09:23.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:23 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:09:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:23 smithi060 ceph-mon[29694]: pgmap v332: 168 pgs: 7 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 29 MiB/s wr, 464 op/s; 60686/342717 objects misplaced (17.707%); 58 MiB/s, 921 objects/s recovering 2024-09-20T22:09:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:23 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:09:23.869 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:23.909 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:23.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:23.910 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:24.010 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:24.054 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:24.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:24.055 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:24.150 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 87 16 117841 117825 84.6258 62.1875 0.0165169 0.0118121 2024-09-20T22:09:24.156 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:24.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:24.181 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:24.281 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:24.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:24.330 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:24.330 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:24.430 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:24.433 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:09:24.434 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:09:24.452 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:24.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:24.453 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:24.554 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:24.587 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:24.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:24.588 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:24.689 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:24.735 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:24.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:24.736 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:24.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:24 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2064667562' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:09:24.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:24 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:09:24.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:24 smithi067 ceph-mon[33511]: pgmap v333: 168 pgs: 7 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 82 MiB/s wr, 1.31k op/s; 59596/346092 objects misplaced (17.220%); 55 MiB/s, 882 objects/s recovering 2024-09-20T22:09:24.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:24 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2064667562' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:09:24.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:24 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:09:24.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:24 smithi060 ceph-mon[40290]: pgmap v333: 168 pgs: 7 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 82 MiB/s wr, 1.31k op/s; 59596/346092 objects misplaced (17.220%); 55 MiB/s, 882 objects/s recovering 2024-09-20T22:09:24.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:24 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2064667562' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:09:24.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:24 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:09:24.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:24 smithi060 ceph-mon[29694]: pgmap v333: 168 pgs: 7 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 7.0 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 639 B/s rd, 82 MiB/s wr, 1.31k op/s; 59596/346092 objects misplaced (17.220%); 55 MiB/s, 882 objects/s recovering 2024-09-20T22:09:24.837 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:24.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:24.894 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:24.895 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:24.995 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:25.021 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:25.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:25.022 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:25.123 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:25.150 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 88 16 118720 118704 84.2885 54.9375 0.0158604 0.0118593 2024-09-20T22:09:25.161 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:25.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:25.162 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:25.263 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:25.287 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:09:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:09:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:09:25.305 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:25.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:25.306 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:25.407 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:25.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:25.457 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:25.458 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:25.551 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:25.558 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:25.619 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:25.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:25.620 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:25.674 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:09:25.675 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:09:25.675 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-20T22:09:25.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-20T22:09:25.695 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:25 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:09:25.695 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:25 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:09:25.695 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:25 smithi060 ceph-mon[29694]: osdmap e222: 8 total, 8 up, 8 in 2024-09-20T22:09:25.695 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:25 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:09:25.695 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:25 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:09:25.695 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:25 smithi060 ceph-mon[40290]: osdmap e222: 8 total, 8 up, 8 in 2024-09-20T22:09:25.720 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:25 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:09:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:25 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:09:25.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:25 smithi067 ceph-mon[33511]: osdmap e222: 8 total, 8 up, 8 in 2024-09-20T22:09:25.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:25.765 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:25.765 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:25.866 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:25.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:25.898 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:25.898 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:25.973 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:09:25.973 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:09:25.999 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:26.028 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:26.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:26.029 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:26.130 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:26.151 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 89 16 119342 119326 83.7782 38.875 0.004084 0.0118765 2024-09-20T22:09:26.164 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:26.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:26.165 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:26.266 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:26.301 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:26.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:26.302 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:26.402 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:26.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:26.434 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:26.511 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:26.534 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:26.535 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:09:26.536 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:09:26.569 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:26.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:26.570 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:26.587 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:09:26.587 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:09:26.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:26 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:09:26.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:26 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:09:26.628 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:26 smithi060 ceph-mon[29694]: osdmap e223: 8 total, 8 up, 8 in 2024-09-20T22:09:26.628 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:26 smithi060 ceph-mon[29694]: pgmap v336: 168 pgs: 1 peering, 6 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 7.1 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.0 KiB/s rd, 58 MiB/s wr, 925 op/s; 58313/350469 objects misplaced (16.639%); 50 MiB/s, 805 objects/s recovering 2024-09-20T22:09:26.628 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:26 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:09:26.628 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:26 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:09:26.628 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:26 smithi060 ceph-mon[40290]: osdmap e223: 8 total, 8 up, 8 in 2024-09-20T22:09:26.628 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:26 smithi060 ceph-mon[40290]: pgmap v336: 168 pgs: 1 peering, 6 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 7.1 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.0 KiB/s rd, 58 MiB/s wr, 925 op/s; 58313/350469 objects misplaced (16.639%); 50 MiB/s, 805 objects/s recovering 2024-09-20T22:09:26.628 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:09:26.629 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:09:26.663 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:09:26.663 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:09:26.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:26.701 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:09:26.701 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:09:26.716 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:26.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:26.717 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:26.739 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:09:26.740 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:09:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:26 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:09:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:26 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:09:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:26 smithi067 ceph-mon[33511]: osdmap e223: 8 total, 8 up, 8 in 2024-09-20T22:09:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:26 smithi067 ceph-mon[33511]: pgmap v336: 168 pgs: 1 peering, 6 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 7.1 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.0 KiB/s rd, 58 MiB/s wr, 925 op/s; 58313/350469 objects misplaced (16.639%); 50 MiB/s, 805 objects/s recovering 2024-09-20T22:09:26.773 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:09:26.773 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:09:26.807 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:09:26.808 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:09:26.818 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:26.850 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:26.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:26.851 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:26.952 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:26.996 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:26.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:26.997 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:27.098 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:27.147 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:27.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:27.148 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:27.151 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Total time run: 90.0103 2024-09-20T22:09:27.151 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Total writes made: 119996 2024-09-20T22:09:27.151 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Write size: 65536 2024-09-20T22:09:27.151 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Object size: 65536 2024-09-20T22:09:27.151 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Bandwidth (MB/sec): 83.321 2024-09-20T22:09:27.151 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Stddev Bandwidth: 38.3959 2024-09-20T22:09:27.151 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Max bandwidth (MB/sec): 209.938 2024-09-20T22:09:27.151 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Min bandwidth (MB/sec): 0 2024-09-20T22:09:27.151 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Average IOPS: 1333 2024-09-20T22:09:27.151 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Stddev IOPS: 614.334 2024-09-20T22:09:27.151 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Max IOPS: 3359 2024-09-20T22:09:27.152 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Min IOPS: 0 2024-09-20T22:09:27.152 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Average Latency(s): 0.0119976 2024-09-20T22:09:27.152 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Stddev Latency(s): 0.0382698 2024-09-20T22:09:27.152 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Max latency(s): 2.31982 2024-09-20T22:09:27.152 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Min latency(s): 0.00126498 2024-09-20T22:09:27.155 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Cleaning up (deleting benchmark objects) 2024-09-20T22:09:27.248 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:27.282 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:27.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:27.283 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:27.384 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:27.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:27.429 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:27.429 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:27.433 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:09:27.530 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:27.558 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:27.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:27.559 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:27.660 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:27.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:27.693 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:27.694 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:27.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:27 smithi067 ceph-mon[33511]: osdmap e224: 8 total, 8 up, 8 in 2024-09-20T22:09:27.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:27 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2088127260' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:09:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:27 smithi060 ceph-mon[29694]: osdmap e224: 8 total, 8 up, 8 in 2024-09-20T22:09:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:27 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2088127260' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:09:27.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:27 smithi060 ceph-mon[40290]: osdmap e224: 8 total, 8 up, 8 in 2024-09-20T22:09:27.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:27 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2088127260' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:09:27.795 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:27.827 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:27.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:27.828 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:27.929 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:27.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:27.966 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:27.966 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:28.067 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:28.095 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:28.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:28.096 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:28.197 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:28.234 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:28.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:28.235 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:28.336 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:28.359 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:28.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:28.360 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:28.460 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:28.492 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:28.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:28.493 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:28.594 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:28.635 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:28.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:28.636 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:28.737 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:28 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:09:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:28 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:09:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:28 smithi067 ceph-mon[33511]: osdmap e225: 8 total, 8 up, 8 in 2024-09-20T22:09:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:28 smithi067 ceph-mon[33511]: pgmap v339: 168 pgs: 1 peering, 6 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 7.3 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 67 MiB/s wr, 1.12k op/s; 54091/358515 objects misplaced (15.088%); 79 MiB/s, 1.26k objects/s recovering 2024-09-20T22:09:28.775 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:28.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:28.776 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:28 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:09:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:28 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:09:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:28 smithi060 ceph-mon[29694]: osdmap e225: 8 total, 8 up, 8 in 2024-09-20T22:09:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:28 smithi060 ceph-mon[29694]: pgmap v339: 168 pgs: 1 peering, 6 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 7.3 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 67 MiB/s wr, 1.12k op/s; 54091/358515 objects misplaced (15.088%); 79 MiB/s, 1.26k objects/s recovering 2024-09-20T22:09:28.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:28 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:09:28.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:28 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:09:28.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:28 smithi060 ceph-mon[40290]: osdmap e225: 8 total, 8 up, 8 in 2024-09-20T22:09:28.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:28 smithi060 ceph-mon[40290]: pgmap v339: 168 pgs: 1 peering, 6 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 7.3 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 67 MiB/s wr, 1.12k op/s; 54091/358515 objects misplaced (15.088%); 79 MiB/s, 1.26k objects/s recovering 2024-09-20T22:09:28.877 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:28.914 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:28.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:28.915 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:29.016 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:29.058 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:29.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:29.059 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:29.161 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:29.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:29.202 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:29.202 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:29.242 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T22:09:29.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:29.343 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:29.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:29.344 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:29.445 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:29.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:29.467 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:29.467 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:29.568 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:29.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:29.607 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:29.607 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:29.610 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:09:29.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:29.731 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:29.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:29.732 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:29.832 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:29.876 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:29.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:29.877 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:29.932 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:29 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/148758570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-20T22:09:29.933 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:09:29.933 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:09:29.933 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 225, 2024-09-20T22:09:29.933 INFO:teuthology.orchestra.run.smithi060.stdout: "fsid": "3df9647e-779b-11ef-baf5-efdc52797490", 2024-09-20T22:09:29.933 INFO:teuthology.orchestra.run.smithi060.stdout: "created": "2024-09-20T21:57:31.244934+0000", 2024-09-20T22:09:29.933 INFO:teuthology.orchestra.run.smithi060.stdout: "modified": "2024-09-20T22:09:27.465687+0000", 2024-09-20T22:09:29.933 INFO:teuthology.orchestra.run.smithi060.stdout: "last_up_change": "2024-09-20T22:06:53.796679+0000", 2024-09-20T22:09:29.934 INFO:teuthology.orchestra.run.smithi060.stdout: "last_in_change": "2024-09-20T22:02:21.189225+0000", 2024-09-20T22:09:29.934 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": "noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-20T22:09:29.934 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_num": 5806080, 2024-09-20T22:09:29.934 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_set": [ 2024-09-20T22:09:29.934 INFO:teuthology.orchestra.run.smithi060.stdout: "nodeep-scrub", 2024-09-20T22:09:29.934 INFO:teuthology.orchestra.run.smithi060.stdout: "noscrub", 2024-09-20T22:09:29.934 INFO:teuthology.orchestra.run.smithi060.stdout: "pglog_hardlimit", 2024-09-20T22:09:29.934 INFO:teuthology.orchestra.run.smithi060.stdout: "purged_snapdirs", 2024-09-20T22:09:29.934 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_deletes", 2024-09-20T22:09:29.934 INFO:teuthology.orchestra.run.smithi060.stdout: "sortbitwise" 2024-09-20T22:09:29.934 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:09:29.934 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_version": 19, 2024-09-20T22:09:29.935 INFO:teuthology.orchestra.run.smithi060.stdout: "full_ratio": 0.94999998807907104, 2024-09-20T22:09:29.935 INFO:teuthology.orchestra.run.smithi060.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-20T22:09:29.935 INFO:teuthology.orchestra.run.smithi060.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-20T22:09:29.935 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_snapshot": "", 2024-09-20T22:09:29.935 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_max": 8, 2024-09-20T22:09:29.935 INFO:teuthology.orchestra.run.smithi060.stdout: "max_osd": 8, 2024-09-20T22:09:29.935 INFO:teuthology.orchestra.run.smithi060.stdout: "require_min_compat_client": "luminous", 2024-09-20T22:09:29.935 INFO:teuthology.orchestra.run.smithi060.stdout: "min_compat_client": "luminous", 2024-09-20T22:09:29.935 INFO:teuthology.orchestra.run.smithi060.stdout: "require_osd_release": "squid", 2024-09-20T22:09:29.935 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_crimson": false, 2024-09-20T22:09:29.935 INFO:teuthology.orchestra.run.smithi060.stdout: "pools": [ 2024-09-20T22:09:29.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:29.936 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 1, 2024-09-20T22:09:29.936 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": ".mgr", 2024-09-20T22:09:29.936 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:00:35.131125+0000", 2024-09-20T22:09:29.936 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:09:29.936 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:09:29.936 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:09:29.936 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:09:29.936 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:09:29.936 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:09:29.936 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:09:29.936 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:09:29.936 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:09:29.936 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:09:29.937 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:09:29.937 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:09:29.937 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:09:29.937 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 11, 2024-09-20T22:09:29.937 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 11, 2024-09-20T22:09:29.937 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 11, 2024-09-20T22:09:29.937 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 11, 2024-09-20T22:09:29.937 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 11, 2024-09-20T22:09:29.937 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:09:29.937 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:09:29.937 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:09:29.938 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:09:29.938 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:09:29.938 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:09:29.938 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:09:29.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.938 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "199", 2024-09-20T22:09:29.938 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:09:29.938 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:09:29.938 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "197", 2024-09-20T22:09:29.938 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:09:29.938 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:09:29.939 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:09:29.939 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:09:29.939 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:09:29.939 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:09:29.939 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:09:29.939 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:09:29.939 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:09:29.939 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:09:29.939 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:09:29.939 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:09:29.939 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:09:29.940 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:09:29.940 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:09:29.940 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:09:29.940 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:09:29.940 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:09:29.940 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:09:29.940 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:09:29.940 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:09:29.940 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:09:29.940 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:09:29.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.941 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:09:29.941 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:09:29.941 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:09:29.941 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:09:29.941 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:09:29.941 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:09:29.941 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:09:29.941 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:09:29.941 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:09:29.942 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:09:29.942 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:09:29.942 INFO:teuthology.orchestra.run.smithi060.stdout: "options": { 2024-09-20T22:09:29.942 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_max": 32, 2024-09-20T22:09:29.942 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_min": 1 2024-09-20T22:09:29.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.942 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:09:29.942 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": {} 2024-09-20T22:09:29.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.942 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:09:29.942 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:09:29.943 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2.1800000667572021, 2024-09-20T22:09:29.943 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 2.1800000667572021, 2024-09-20T22:09:29.943 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:09:29.943 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2.1800000667572021, 2024-09-20T22:09:29.943 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 2.1800000667572021, 2024-09-20T22:09:29.943 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:09:29.943 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:09:29.943 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:09:29.943 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:29.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:29.943 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 2, 2024-09-20T22:09:29.943 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "datapool", 2024-09-20T22:09:29.944 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:47.974480+0000", 2024-09-20T22:09:29.944 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 8193, 2024-09-20T22:09:29.944 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-20T22:09:29.944 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:09:29.944 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:09:29.944 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:09:29.944 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:09:29.944 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:09:29.944 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:09:29.944 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:09:29.944 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:09:29.945 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:09:29.945 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:09:29.945 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:09:29.945 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 3, 2024-09-20T22:09:29.945 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 3, 2024-09-20T22:09:29.945 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 3, 2024-09-20T22:09:29.945 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 3, 2024-09-20T22:09:29.945 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 3, 2024-09-20T22:09:29.945 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:09:29.945 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:09:29.945 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:09:29.945 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:09:29.945 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:09:29.946 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:09:29.946 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:09:29.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.946 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "66", 2024-09-20T22:09:29.946 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:09:29.946 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:09:29.946 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:09:29.946 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:09:29.946 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:09:29.946 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 2, 2024-09-20T22:09:29.946 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 66, 2024-09-20T22:09:29.946 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:09:29.947 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:09:29.947 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:09:29.947 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:09:29.947 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:09:29.947 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:09:29.947 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:09:29.947 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:09:29.947 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:09:29.947 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:09:29.947 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:09:29.947 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:09:29.947 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:09:29.948 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:09:29.948 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:09:29.948 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:09:29.948 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:09:29.948 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:09:29.948 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:09:29.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.948 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:09:29.948 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:09:29.948 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:09:29.949 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:09:29.949 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:09:29.949 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:09:29.949 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:09:29.949 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:09:29.949 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:09:29.949 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:09:29.949 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:09:29.949 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:09:29.949 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:09:29.949 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd": {} 2024-09-20T22:09:29.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.950 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:09:29.950 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:09:29.950 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2.6500000953674316, 2024-09-20T22:09:29.950 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 2.6500000953674316, 2024-09-20T22:09:29.950 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T22:09:29.950 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-20T22:09:29.950 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-20T22:09:29.950 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:09:29.950 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:09:29.950 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:09:29.951 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:29.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:29.951 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 3, 2024-09-20T22:09:29.951 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": ".rgw.root", 2024-09-20T22:09:29.951 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:48.902465+0000", 2024-09-20T22:09:29.951 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:09:29.951 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:09:29.951 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:09:29.951 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:09:29.951 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:09:29.951 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:09:29.952 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:09:29.952 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:09:29.952 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:09:29.952 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:09:29.952 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:09:29.952 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:09:29.952 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:09:29.952 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:09:29.952 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:09:29.952 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:09:29.953 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:09:29.953 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:09:29.953 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:09:29.953 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:09:29.953 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:09:29.953 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:09:29.953 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:09:29.953 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:09:29.953 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:09:29.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.953 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "65", 2024-09-20T22:09:29.953 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:09:29.954 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:09:29.954 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:09:29.954 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:09:29.954 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:09:29.954 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:09:29.954 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:09:29.954 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:09:29.954 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:09:29.954 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:09:29.954 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:09:29.955 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:09:29.955 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:09:29.955 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:09:29.955 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:09:29.955 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:09:29.955 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:09:29.955 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:09:29.955 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:09:29.955 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:09:29.955 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:09:29.955 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:09:29.956 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:09:29.956 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:09:29.956 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:09:29.956 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:09:29.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.956 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:09:29.956 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:09:29.956 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:09:29.956 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:09:29.956 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:09:29.956 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:09:29.956 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:09:29.957 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:09:29.957 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:09:29.957 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:09:29.957 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:09:29.957 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:09:29.957 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:09:29.957 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:09:29.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.957 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:09:29.957 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:09:29.957 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.75, 2024-09-20T22:09:29.958 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.75, 2024-09-20T22:09:29.958 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:09:29.958 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.75, 2024-09-20T22:09:29.958 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.75, 2024-09-20T22:09:29.958 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:09:29.958 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:09:29.958 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:09:29.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:29.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:29.958 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 4, 2024-09-20T22:09:29.959 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "default.rgw.log", 2024-09-20T22:09:29.959 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:50.620057+0000", 2024-09-20T22:09:29.959 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:09:29.959 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:09:29.959 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:09:29.959 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:09:29.959 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:09:29.959 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:09:29.959 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:09:29.959 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:09:29.959 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:09:29.959 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:09:29.960 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:09:29.960 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:09:29.960 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:09:29.960 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:09:29.960 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:09:29.960 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:09:29.960 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:09:29.960 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:09:29.960 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:09:29.960 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:09:29.960 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:09:29.960 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:09:29.960 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:09:29.961 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:09:29.961 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:09:29.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.961 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "67", 2024-09-20T22:09:29.961 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:09:29.961 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:09:29.961 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:09:29.961 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:09:29.961 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:09:29.961 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:09:29.961 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:09:29.962 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:09:29.962 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:09:29.962 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:09:29.962 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:09:29.962 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:09:29.962 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:09:29.962 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:09:29.962 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:09:29.962 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:09:29.962 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:09:29.962 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:09:29.962 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:09:29.962 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:09:29.963 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:09:29.963 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:09:29.963 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:09:29.963 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:09:29.963 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:09:29.963 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:09:29.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.963 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:09:29.963 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:09:29.963 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:09:29.963 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:09:29.963 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:09:29.964 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:09:29.964 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:09:29.964 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:09:29.964 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:09:29.964 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:09:29.964 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:09:29.964 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:09:29.964 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:09:29.964 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:09:29.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.964 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:09:29.964 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:09:29.965 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2.25, 2024-09-20T22:09:29.965 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 2.25, 2024-09-20T22:09:29.965 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:09:29.965 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2.25, 2024-09-20T22:09:29.965 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 2.25, 2024-09-20T22:09:29.965 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:09:29.965 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:09:29.965 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:09:29.965 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:29.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:29.965 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 5, 2024-09-20T22:09:29.966 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "default.rgw.control", 2024-09-20T22:09:29.966 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:52.722912+0000", 2024-09-20T22:09:29.966 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:09:29.966 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:09:29.966 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:09:29.966 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:09:29.966 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:09:29.966 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:09:29.966 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:09:29.966 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:09:29.966 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:09:29.966 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:09:29.966 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:09:29.967 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:09:29.967 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:09:29.967 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:09:29.967 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:09:29.967 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:09:29.967 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:09:29.967 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:09:29.967 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:09:29.967 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:09:29.967 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:09:29.967 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:09:29.967 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:09:29.968 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:09:29.968 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:09:29.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.968 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "69", 2024-09-20T22:09:29.968 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:09:29.968 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:09:29.968 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:09:29.968 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:09:29.968 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:09:29.968 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:09:29.968 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:09:29.968 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:09:29.968 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:09:29.969 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:09:29.969 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:09:29.969 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:09:29.969 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:09:29.969 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:09:29.969 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:09:29.969 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:09:29.969 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:09:29.969 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:09:29.969 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:09:29.969 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:09:29.969 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:09:29.970 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:09:29.970 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:09:29.970 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:09:29.970 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:09:29.970 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:09:29.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.970 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:09:29.970 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:09:29.970 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:09:29.970 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:09:29.970 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:09:29.971 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:09:29.973 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:09:29.973 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:09:29.974 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:09:29.974 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:09:29.974 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:09:29.974 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:09:29.974 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:09:29.974 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:09:29.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.974 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:09:29.974 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:09:29.974 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.5, 2024-09-20T22:09:29.974 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.5, 2024-09-20T22:09:29.975 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:09:29.975 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.5, 2024-09-20T22:09:29.975 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.5, 2024-09-20T22:09:29.975 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:09:29.975 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:09:29.975 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:09:29.975 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:29.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:29.975 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 6, 2024-09-20T22:09:29.975 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "default.rgw.meta", 2024-09-20T22:09:29.976 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:54.794248+0000", 2024-09-20T22:09:29.976 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:09:29.976 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:09:29.976 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:09:29.976 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:09:29.976 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:09:29.976 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:09:29.976 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:09:29.976 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:09:29.976 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:09:29.976 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:09:29.977 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:09:29.977 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:09:29.977 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:09:29.977 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:09:29.977 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:09:29.977 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:09:29.977 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:09:29.977 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:09:29.977 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:09:29.977 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:09:29.977 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:09:29.977 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:09:29.978 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:09:29.978 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:09:29.978 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:09:29.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.978 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "71", 2024-09-20T22:09:29.978 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:09:29.978 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:09:29.978 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:09:29.978 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:09:29.978 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:09:29.978 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:09:29.978 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:09:29.979 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:09:29.979 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:09:29.979 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:09:29.979 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:09:29.979 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:09:29.979 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:09:29.979 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:09:29.979 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:09:29.979 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:09:29.979 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:09:29.979 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:09:29.979 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:09:29.980 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:09:29.980 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:09:29.980 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:09:29.980 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:09:29.980 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:09:29.980 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:09:29.980 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:09:29.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.980 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:09:29.980 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:09:29.980 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:09:29.981 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:09:29.981 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:09:29.981 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:09:29.981 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:09:29.981 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:09:29.981 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:09:29.981 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:09:29.981 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:09:29.981 INFO:teuthology.orchestra.run.smithi060.stdout: "options": { 2024-09-20T22:09:29.981 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_bias": 4 2024-09-20T22:09:29.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.982 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:09:29.982 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:09:29.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.982 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:09:29.982 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:09:29.982 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.75, 2024-09-20T22:09:29.982 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.75, 2024-09-20T22:09:29.982 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:09:29.982 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.75, 2024-09-20T22:09:29.982 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.75, 2024-09-20T22:09:29.983 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:09:29.983 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:09:29.983 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:09:29.983 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:29.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:29.983 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 8, 2024-09-20T22:09:29.983 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "unique_pool_1", 2024-09-20T22:09:29.983 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:07:51.043037+0000", 2024-09-20T22:09:29.983 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:09:29.984 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:09:29.984 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:09:29.984 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:09:29.984 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:09:29.984 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:09:29.984 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:09:29.984 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:09:29.984 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:09:29.984 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:09:29.984 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:09:29.984 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:09:29.984 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:09:29.985 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 26, 2024-09-20T22:09:29.985 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 26, 2024-09-20T22:09:29.985 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 26, 2024-09-20T22:09:29.985 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 26, 2024-09-20T22:09:29.985 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 26, 2024-09-20T22:09:29.985 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:09:29.985 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:09:29.985 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:09:29.985 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:09:29.985 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:09:29.985 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:09:29.985 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:09:29.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.986 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "217", 2024-09-20T22:09:29.986 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:09:29.986 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:09:29.986 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "215", 2024-09-20T22:09:29.986 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:09:29.986 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:09:29.986 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:09:29.986 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:09:29.986 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:09:29.986 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:09:29.987 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:09:29.987 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:09:29.987 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:09:29.987 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:09:29.987 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:09:29.987 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:09:29.987 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:09:29.987 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:09:29.987 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:09:29.987 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:09:29.987 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:09:29.987 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:09:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:09:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:09:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:09:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:09:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:09:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:09:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:09:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:09:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:09:29.988 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:09:29.989 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:09:29.989 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:09:29.989 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:09:29.989 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:09:29.989 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:09:29.989 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:09:29.989 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:09:29.989 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:09:29.989 INFO:teuthology.orchestra.run.smithi060.stdout: "rados": {} 2024-09-20T22:09:29.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.989 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:09:29.989 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:09:29.989 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2.1500000953674316, 2024-09-20T22:09:29.990 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.5399999618530273, 2024-09-20T22:09:29.990 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:09:29.990 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2.1500000953674316, 2024-09-20T22:09:29.990 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.5399999618530273, 2024-09-20T22:09:29.990 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:09:29.990 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:09:29.990 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:09:29.990 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:29.990 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:29.990 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:09:29.990 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:09:29.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:29.991 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 0, 2024-09-20T22:09:29.991 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "0997c01a-a9bd-421e-abcc-fbcf7abe8512", 2024-09-20T22:09:29.991 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:09:29.991 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:09:29.991 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:09:29.991 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:09:29.991 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:09:29.991 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:09:29.991 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 8, 2024-09-20T22:09:29.991 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 218, 2024-09-20T22:09:29.991 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:09:29.992 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:09:29.992 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:09:29.992 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:29.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:29.992 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:29.992 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6802", 2024-09-20T22:09:29.992 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:09:29.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:29.992 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:29.992 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6803", 2024-09-20T22:09:29.992 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:09:29.993 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:29.993 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:29.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.993 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:09:29.993 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:29.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:29.993 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:29.993 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6804", 2024-09-20T22:09:29.993 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:09:29.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:29.993 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:29.994 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6805", 2024-09-20T22:09:29.994 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:09:29.994 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:29.994 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:29.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.994 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:09:29.994 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:29.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:29.994 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:29.994 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6808", 2024-09-20T22:09:29.994 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:09:29.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:29.995 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:29.995 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6809", 2024-09-20T22:09:29.995 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:09:29.995 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:29.995 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:29.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.995 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:09:29.995 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:29.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:29.996 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:29.996 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:29 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/148758570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-20T22:09:29.996 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6806", 2024-09-20T22:09:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:09:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6807", 2024-09-20T22:09:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:09:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6803/1395849550", 2024-09-20T22:09:29.997 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6805/1395849550", 2024-09-20T22:09:29.998 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6809/1395849550", 2024-09-20T22:09:29.998 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6807/1395849550", 2024-09-20T22:09:29.998 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:09:29.998 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:09:29.998 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:09:29.998 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:29.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:29.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:29.998 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 1, 2024-09-20T22:09:29.998 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "c04bdcb1-d76b-47c8-a38b-c02043df6ba8", 2024-09-20T22:09:29.998 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:09:29.999 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:09:29.999 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:09:29.999 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:09:29.999 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 13, 2024-09-20T22:09:29.999 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 90, 2024-09-20T22:09:29.999 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 145, 2024-09-20T22:09:29.999 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 218, 2024-09-20T22:09:29.999 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 91, 2024-09-20T22:09:29.999 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:09:29.999 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:09:29.999 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:29.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.000 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.000 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6810", 2024-09-20T22:09:30.000 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:09:30.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.000 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.000 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6811", 2024-09-20T22:09:30.000 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:09:30.000 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.000 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.001 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:09:30.001 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.001 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.001 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6812", 2024-09-20T22:09:30.001 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:09:30.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.001 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.001 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6813", 2024-09-20T22:09:30.001 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:09:30.001 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.002 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.002 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:09:30.002 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.002 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.002 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6816", 2024-09-20T22:09:30.002 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:09:30.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.002 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.002 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6817", 2024-09-20T22:09:30.003 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:09:30.003 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.003 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.003 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:09:30.003 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.003 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.003 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6814", 2024-09-20T22:09:30.003 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:09:30.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.004 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.004 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6815", 2024-09-20T22:09:30.004 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:09:30.004 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.004 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.004 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6811/2970874921", 2024-09-20T22:09:30.004 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6813/2970874921", 2024-09-20T22:09:30.004 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6817/2970874921", 2024-09-20T22:09:30.004 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6815/2970874921", 2024-09-20T22:09:30.004 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:09:30.004 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:09:30.005 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:09:30.005 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.005 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 2, 2024-09-20T22:09:30.005 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "32c09d19-eb98-4926-a44e-e7ddee152956", 2024-09-20T22:09:30.005 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:09:30.005 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:09:30.005 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:09:30.005 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:09:30.005 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:09:30.005 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:09:30.006 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 18, 2024-09-20T22:09:30.006 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 204, 2024-09-20T22:09:30.006 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:09:30.006 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:09:30.006 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:09:30.006 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.006 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.006 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6818", 2024-09-20T22:09:30.006 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:09:30.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.007 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.007 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6819", 2024-09-20T22:09:30.007 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:09:30.007 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.007 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.007 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:09:30.007 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.007 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.008 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6820", 2024-09-20T22:09:30.008 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:09:30.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.008 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.008 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6821", 2024-09-20T22:09:30.008 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:09:30.008 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.008 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.008 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:09:30.008 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.009 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.009 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6824", 2024-09-20T22:09:30.009 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:09:30.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.009 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.009 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6825", 2024-09-20T22:09:30.009 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:09:30.009 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.009 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.010 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:09:30.010 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.010 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.010 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6822", 2024-09-20T22:09:30.010 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:09:30.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.010 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.010 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6823", 2024-09-20T22:09:30.010 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:09:30.010 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.011 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.011 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6819/3698562274", 2024-09-20T22:09:30.011 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6821/3698562274", 2024-09-20T22:09:30.011 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6825/3698562274", 2024-09-20T22:09:30.011 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6823/3698562274", 2024-09-20T22:09:30.011 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:09:30.011 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:09:30.011 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:09:30.011 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.012 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 3, 2024-09-20T22:09:30.012 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "b1c2cb69-8565-49ac-b854-7ea7615c6660", 2024-09-20T22:09:30.012 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:09:30.012 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:09:30.012 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:09:30.012 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:09:30.012 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:09:30.012 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:09:30.012 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 26, 2024-09-20T22:09:30.012 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 218, 2024-09-20T22:09:30.012 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:09:30.012 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:09:30.013 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:09:30.013 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.013 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.013 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6826", 2024-09-20T22:09:30.013 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:09:30.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.013 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.013 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6827", 2024-09-20T22:09:30.014 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:09:30.014 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.014 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.014 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:09:30.014 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.014 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.014 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6828", 2024-09-20T22:09:30.014 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:09:30.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.015 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.015 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6829", 2024-09-20T22:09:30.015 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:09:30.015 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.015 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.015 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:09:30.015 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.015 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.015 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6832", 2024-09-20T22:09:30.015 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:09:30.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.016 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.016 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6833", 2024-09-20T22:09:30.016 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:09:30.016 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.016 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.016 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:09:30.016 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.016 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.017 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6830", 2024-09-20T22:09:30.017 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:09:30.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.017 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.017 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6831", 2024-09-20T22:09:30.017 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:09:30.017 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.017 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.017 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6827/2592322449", 2024-09-20T22:09:30.017 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6829/2592322449", 2024-09-20T22:09:30.018 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6833/2592322449", 2024-09-20T22:09:30.018 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6831/2592322449", 2024-09-20T22:09:30.018 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:09:30.018 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:09:30.018 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:09:30.018 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.018 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 4, 2024-09-20T22:09:30.018 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "04a547d6-fc48-43b7-a8d7-18df8a83aa96", 2024-09-20T22:09:30.018 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:09:30.019 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:09:30.019 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:09:30.019 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:09:30.019 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:09:30.019 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:09:30.019 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 31, 2024-09-20T22:09:30.019 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 218, 2024-09-20T22:09:30.019 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:09:30.019 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:09:30.019 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:09:30.019 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.020 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.020 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6800", 2024-09-20T22:09:30.020 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:09:30.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.020 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.020 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6801", 2024-09-20T22:09:30.020 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:09:30.020 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.020 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.021 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:09:30.021 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.021 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.021 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6802", 2024-09-20T22:09:30.021 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:09:30.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.021 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.021 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6803", 2024-09-20T22:09:30.021 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:09:30.022 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.022 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.022 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:09:30.022 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.022 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.022 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6806", 2024-09-20T22:09:30.022 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:09:30.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.022 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.023 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6807", 2024-09-20T22:09:30.023 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:09:30.023 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.023 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.023 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:09:30.023 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.023 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.023 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6804", 2024-09-20T22:09:30.023 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:09:30.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.024 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.024 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6805", 2024-09-20T22:09:30.024 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:09:30.024 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.024 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.024 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6801/3243564239", 2024-09-20T22:09:30.024 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6803/3243564239", 2024-09-20T22:09:30.024 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6807/3243564239", 2024-09-20T22:09:30.024 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6805/3243564239", 2024-09-20T22:09:30.024 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:09:30.025 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:09:30.025 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:09:30.025 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.025 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 5, 2024-09-20T22:09:30.025 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "4763aa10-c367-485b-b525-e5241e642b86", 2024-09-20T22:09:30.025 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:09:30.025 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:09:30.025 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:09:30.025 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:09:30.025 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:09:30.026 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:09:30.026 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 39, 2024-09-20T22:09:30.026 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 223, 2024-09-20T22:09:30.026 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:09:30.026 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:09:30.026 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:09:30.026 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.026 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.026 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6808", 2024-09-20T22:09:30.026 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:09:30.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.027 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.027 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6809", 2024-09-20T22:09:30.027 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:09:30.027 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.027 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.027 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:09:30.027 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.027 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.027 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6810", 2024-09-20T22:09:30.027 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:09:30.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.028 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.028 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6811", 2024-09-20T22:09:30.028 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:09:30.028 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.028 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.028 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:09:30.028 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.028 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.029 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6814", 2024-09-20T22:09:30.029 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:09:30.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.029 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.029 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6815", 2024-09-20T22:09:30.029 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:09:30.029 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.029 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.029 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:09:30.029 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.030 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.030 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6812", 2024-09-20T22:09:30.030 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:09:30.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.030 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.030 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6813", 2024-09-20T22:09:30.030 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:09:30.030 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.030 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.031 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6809/1622258887", 2024-09-20T22:09:30.031 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6811/1622258887", 2024-09-20T22:09:30.031 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6815/1622258887", 2024-09-20T22:09:30.031 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6813/1622258887", 2024-09-20T22:09:30.031 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:09:30.031 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:09:30.031 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:09:30.031 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.032 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 6, 2024-09-20T22:09:30.032 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "a359116e-c013-4e2f-8b79-86219516ee16", 2024-09-20T22:09:30.032 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:09:30.032 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:09:30.032 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:09:30.032 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:09:30.032 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:09:30.032 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:09:30.032 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 48, 2024-09-20T22:09:30.032 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 217, 2024-09-20T22:09:30.032 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:09:30.032 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:09:30.033 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:09:30.033 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.033 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.033 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6816", 2024-09-20T22:09:30.033 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:09:30.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.033 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.033 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6817", 2024-09-20T22:09:30.033 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:09:30.033 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.033 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.034 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:09:30.034 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.034 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.034 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6818", 2024-09-20T22:09:30.034 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:09:30.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.034 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.034 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6819", 2024-09-20T22:09:30.035 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:09:30.035 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.035 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.035 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:09:30.035 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.035 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.035 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6822", 2024-09-20T22:09:30.035 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:09:30.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.036 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.036 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6823", 2024-09-20T22:09:30.036 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:09:30.036 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.036 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.036 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:09:30.036 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.036 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.036 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6820", 2024-09-20T22:09:30.036 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:09:30.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.037 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.037 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6821", 2024-09-20T22:09:30.037 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:09:30.037 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.037 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.037 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6817/1187114", 2024-09-20T22:09:30.037 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6819/1187114", 2024-09-20T22:09:30.037 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6823/1187114", 2024-09-20T22:09:30.037 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6821/1187114", 2024-09-20T22:09:30.037 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:09:30.038 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:09:30.038 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:09:30.038 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.038 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 7, 2024-09-20T22:09:30.038 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "a4bf4188-f133-4a31-8897-d7e8182248d7", 2024-09-20T22:09:30.038 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:09:30.038 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:09:30.038 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 0.732696533203125, 2024-09-20T22:09:30.038 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:09:30.039 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:09:30.039 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:09:30.039 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 57, 2024-09-20T22:09:30.039 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 218, 2024-09-20T22:09:30.039 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:09:30.039 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:09:30.039 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:09:30.039 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.039 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.039 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6824", 2024-09-20T22:09:30.039 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:09:30.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.040 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.040 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6825", 2024-09-20T22:09:30.040 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:09:30.040 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.040 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.040 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:09:30.040 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.041 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.041 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6826", 2024-09-20T22:09:30.041 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:09:30.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.041 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.041 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6827", 2024-09-20T22:09:30.041 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:09:30.041 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.041 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.042 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:09:30.042 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.042 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.042 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6830", 2024-09-20T22:09:30.042 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:09:30.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.042 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.042 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6831", 2024-09-20T22:09:30.042 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:09:30.043 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.043 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.043 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:09:30.043 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:09:30.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.043 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:09:30.043 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6828", 2024-09-20T22:09:30.043 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:09:30.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.044 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:09:30.044 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6829", 2024-09-20T22:09:30.044 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:09:30.044 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.044 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.044 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6825/3021842840", 2024-09-20T22:09:30.044 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6827/3021842840", 2024-09-20T22:09:30.044 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6831/3021842840", 2024-09-20T22:09:30.044 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6829/3021842840", 2024-09-20T22:09:30.044 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:09:30.044 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:09:30.044 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:09:30.045 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.045 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.045 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:09:30.045 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_xinfo": [ 2024-09-20T22:09:30.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.045 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 0, 2024-09-20T22:09:30.045 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:09:30.045 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:09:30.045 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:09:30.045 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:09:30.045 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:09:30.045 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:08:37.160825+0000", 2024-09-20T22:09:30.046 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:09:30.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.046 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 1, 2024-09-20T22:09:30.046 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:09:30.046 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:09:30.046 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:09:30.046 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:09:30.046 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:09:30.046 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:09:20.725454+0000", 2024-09-20T22:09:30.046 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 90 2024-09-20T22:09:30.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.047 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 2, 2024-09-20T22:09:30.047 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:09:30.047 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:09:30.047 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:09:30.047 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:09:30.047 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:09:30.047 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:08:24.811711+0000", 2024-09-20T22:09:30.047 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:09:30.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.047 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 3, 2024-09-20T22:09:30.047 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:09:30.048 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:09:30.048 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:09:30.048 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:09:30.048 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:09:30.048 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:08:37.826751+0000", 2024-09-20T22:09:30.048 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:09:30.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.048 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 4, 2024-09-20T22:09:30.048 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:09:30.048 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:09:30.048 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:09:30.049 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:09:30.049 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:09:30.049 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:08:55.939475+0000", 2024-09-20T22:09:30.049 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:09:30.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.049 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 5, 2024-09-20T22:09:30.049 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:09:30.049 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:09:30.049 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:09:30.049 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:09:30.050 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:09:30.050 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:09:21.083547+0000", 2024-09-20T22:09:30.050 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:09:30.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.050 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 6, 2024-09-20T22:09:30.050 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:09:30.050 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:09:30.050 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:09:30.050 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:09:30.050 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:09:30.051 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:09:25.296562+0000", 2024-09-20T22:09:30.051 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:09:30.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.051 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 7, 2024-09-20T22:09:30.051 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:09:30.051 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:09:30.051 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:09:30.051 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:09:30.051 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:09:30.051 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:09:27.456118+0000", 2024-09-20T22:09:30.051 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:09:30.052 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.052 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:09:30.052 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap": [], 2024-09-20T22:09:30.052 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap_items": [ 2024-09-20T22:09:30.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.052 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "3.1a", 2024-09-20T22:09:30.052 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:09:30.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.052 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 4, 2024-09-20T22:09:30.052 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 5 2024-09-20T22:09:30.052 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.052 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.053 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "3.1c", 2024-09-20T22:09:30.053 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:09:30.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.053 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 4, 2024-09-20T22:09:30.053 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 2 2024-09-20T22:09:30.053 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.053 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.053 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "4.f", 2024-09-20T22:09:30.054 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:09:30.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.054 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 1, 2024-09-20T22:09:30.054 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 2 2024-09-20T22:09:30.054 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.054 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.054 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "4.13", 2024-09-20T22:09:30.054 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:09:30.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.055 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 1, 2024-09-20T22:09:30.055 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 2 2024-09-20T22:09:30.055 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.055 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.055 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "4.1b", 2024-09-20T22:09:30.055 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:09:30.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.055 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 1, 2024-09-20T22:09:30.055 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 5 2024-09-20T22:09:30.055 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.056 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.056 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.7", 2024-09-20T22:09:30.056 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:09:30.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.056 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:09:30.056 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 0 2024-09-20T22:09:30.056 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.056 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.057 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.9", 2024-09-20T22:09:30.057 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:09:30.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.057 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 7, 2024-09-20T22:09:30.057 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 1 2024-09-20T22:09:30.057 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.057 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.057 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.b", 2024-09-20T22:09:30.057 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:09:30.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.057 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:09:30.058 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 0 2024-09-20T22:09:30.058 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.058 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.058 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.d", 2024-09-20T22:09:30.058 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:09:30.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.058 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 5, 2024-09-20T22:09:30.058 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 6 2024-09-20T22:09:30.058 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.058 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.059 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.11", 2024-09-20T22:09:30.059 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:09:30.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.059 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:09:30.059 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 1 2024-09-20T22:09:30.059 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.059 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.059 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.19", 2024-09-20T22:09:30.059 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:09:30.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.060 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:09:30.060 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 6 2024-09-20T22:09:30.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.060 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 5, 2024-09-20T22:09:30.060 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 0 2024-09-20T22:09:30.060 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.060 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.060 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.060 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:09:30.060 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap_primaries": [], 2024-09-20T22:09:30.060 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_temp": [ 2024-09-20T22:09:30.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.061 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "8.c", 2024-09-20T22:09:30.061 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:09:30.061 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:09:30.061 INFO:teuthology.orchestra.run.smithi060.stdout: 3, 2024-09-20T22:09:30.061 INFO:teuthology.orchestra.run.smithi060.stdout: 7 2024-09-20T22:09:30.061 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.061 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "8.10", 2024-09-20T22:09:30.061 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:09:30.062 INFO:teuthology.orchestra.run.smithi060.stdout: 1, 2024-09-20T22:09:30.062 INFO:teuthology.orchestra.run.smithi060.stdout: 5, 2024-09-20T22:09:30.062 INFO:teuthology.orchestra.run.smithi060.stdout: 3 2024-09-20T22:09:30.062 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.062 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "8.11", 2024-09-20T22:09:30.062 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:09:30.062 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:09:30.062 INFO:teuthology.orchestra.run.smithi060.stdout: 1, 2024-09-20T22:09:30.062 INFO:teuthology.orchestra.run.smithi060.stdout: 2 2024-09-20T22:09:30.063 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.063 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "8.12", 2024-09-20T22:09:30.063 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:09:30.063 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:09:30.063 INFO:teuthology.orchestra.run.smithi060.stdout: 4, 2024-09-20T22:09:30.063 INFO:teuthology.orchestra.run.smithi060.stdout: 5 2024-09-20T22:09:30.063 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.063 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "8.13", 2024-09-20T22:09:30.063 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:09:30.064 INFO:teuthology.orchestra.run.smithi060.stdout: 1, 2024-09-20T22:09:30.064 INFO:teuthology.orchestra.run.smithi060.stdout: 5, 2024-09-20T22:09:30.064 INFO:teuthology.orchestra.run.smithi060.stdout: 6 2024-09-20T22:09:30.064 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.064 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "8.14", 2024-09-20T22:09:30.064 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:09:30.064 INFO:teuthology.orchestra.run.smithi060.stdout: 4, 2024-09-20T22:09:30.064 INFO:teuthology.orchestra.run.smithi060.stdout: 2, 2024-09-20T22:09:30.064 INFO:teuthology.orchestra.run.smithi060.stdout: 5 2024-09-20T22:09:30.065 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.065 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "8.15", 2024-09-20T22:09:30.065 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:09:30.065 INFO:teuthology.orchestra.run.smithi060.stdout: 7, 2024-09-20T22:09:30.065 INFO:teuthology.orchestra.run.smithi060.stdout: 6, 2024-09-20T22:09:30.065 INFO:teuthology.orchestra.run.smithi060.stdout: 4 2024-09-20T22:09:30.065 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.065 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "8.16", 2024-09-20T22:09:30.065 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:09:30.065 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:09:30.066 INFO:teuthology.orchestra.run.smithi060.stdout: 6, 2024-09-20T22:09:30.066 INFO:teuthology.orchestra.run.smithi060.stdout: 7 2024-09-20T22:09:30.066 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.066 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "8.17", 2024-09-20T22:09:30.066 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:09:30.066 INFO:teuthology.orchestra.run.smithi060.stdout: 1, 2024-09-20T22:09:30.066 INFO:teuthology.orchestra.run.smithi060.stdout: 6, 2024-09-20T22:09:30.066 INFO:teuthology.orchestra.run.smithi060.stdout: 3 2024-09-20T22:09:30.067 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:30.067 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "8.18", 2024-09-20T22:09:30.067 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:09:30.067 INFO:teuthology.orchestra.run.smithi060.stdout: 3, 2024-09-20T22:09:30.067 INFO:teuthology.orchestra.run.smithi060.stdout: 5, 2024-09-20T22:09:30.067 INFO:teuthology.orchestra.run.smithi060.stdout: 7 2024-09-20T22:09:30.067 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:30.067 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.067 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:09:30.068 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_temp": [], 2024-09-20T22:09:30.068 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklist": { 2024-09-20T22:09:30.068 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/890154710": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:09:30.068 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3386526350": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:09:30.068 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/604714621": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:09:30.068 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/604714621": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:09:30.068 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/3151907776": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:09:30.068 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/956264518": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:09:30.068 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/4150585713": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:09:30.068 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3311255314": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:09:30.068 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3965541528": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:09:30.068 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/3734574621": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:09:30.068 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1729862661": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:09:30.068 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/3734574621": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:09:30.069 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3927294026": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:09:30.069 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/3151907776": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:09:30.069 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1871351181": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:09:30.069 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1854455098": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:09:30.069 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/2221262203": "2024-09-21T22:03:19.518405+0000" 2024-09-20T22:09:30.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.069 INFO:teuthology.orchestra.run.smithi060.stdout: "range_blocklist": {}, 2024-09-20T22:09:30.069 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profiles": { 2024-09-20T22:09:30.069 INFO:teuthology.orchestra.run.smithi060.stdout: "default": { 2024-09-20T22:09:30.069 INFO:teuthology.orchestra.run.smithi060.stdout: "crush-failure-domain": "osd", 2024-09-20T22:09:30.069 INFO:teuthology.orchestra.run.smithi060.stdout: "k": "2", 2024-09-20T22:09:30.069 INFO:teuthology.orchestra.run.smithi060.stdout: "m": "1", 2024-09-20T22:09:30.069 INFO:teuthology.orchestra.run.smithi060.stdout: "plugin": "jerasure", 2024-09-20T22:09:30.069 INFO:teuthology.orchestra.run.smithi060.stdout: "technique": "reed_sol_van" 2024-09-20T22:09:30.070 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:30.070 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps_queue": [], 2024-09-20T22:09:30.070 INFO:teuthology.orchestra.run.smithi060.stdout: "new_removed_snaps": [], 2024-09-20T22:09:30.070 INFO:teuthology.orchestra.run.smithi060.stdout: "new_purged_snaps": [], 2024-09-20T22:09:30.070 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_node_flags": {}, 2024-09-20T22:09:30.070 INFO:teuthology.orchestra.run.smithi060.stdout: "device_class_flags": {}, 2024-09-20T22:09:30.070 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode": { 2024-09-20T22:09:30.070 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode_enabled": false, 2024-09-20T22:09:30.070 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_bucket_count": 0, 2024-09-20T22:09:30.070 INFO:teuthology.orchestra.run.smithi060.stdout: "degraded_stretch_mode": 0, 2024-09-20T22:09:30.070 INFO:teuthology.orchestra.run.smithi060.stdout: "recovering_stretch_mode": 0, 2024-09-20T22:09:30.070 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode_bucket": 0 2024-09-20T22:09:30.070 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:30.071 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:09:30.071 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:30.072 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:29 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/148758570' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-20T22:09:30.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:30.131 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:30.131 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:30.232 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:30.236 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:09:30.237 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:09:30.237 INFO:teuthology.orchestra.run.smithi067.stdout: "complaint_time": 30, 2024-09-20T22:09:30.237 INFO:teuthology.orchestra.run.smithi067.stdout: "num_blocked_ops": 0 2024-09-20T22:09:30.237 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:09:30.271 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:30.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:30.272 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:30.373 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:30.433 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:30.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:30.434 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:30.535 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:30.570 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:30.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:30.571 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:30.672 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:30.709 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:30.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:30.710 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:30.811 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:30.854 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:30.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:30.855 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:30.956 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:30.967 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:30 smithi067 ceph-mon[33511]: pgmap v340: 168 pgs: 1 peering, 6 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 7.3 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 909 B/s rd, 48 MiB/s wr, 796 op/s; 54091/358515 objects misplaced (15.088%); 56 MiB/s, 899 objects/s recovering 2024-09-20T22:09:30.988 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:09:30.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:30.996 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:30.997 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:31.008 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T22:09:31.031 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:30 smithi060 ceph-mon[40290]: pgmap v340: 168 pgs: 1 peering, 6 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 7.3 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 909 B/s rd, 48 MiB/s wr, 796 op/s; 54091/358515 objects misplaced (15.088%); 56 MiB/s, 899 objects/s recovering 2024-09-20T22:09:31.032 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:30 smithi060 ceph-mon[29694]: pgmap v340: 168 pgs: 1 peering, 6 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 157 active+clean; 7.3 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 909 B/s rd, 48 MiB/s wr, 796 op/s; 54091/358515 objects misplaced (15.088%); 56 MiB/s, 899 objects/s recovering 2024-09-20T22:09:31.098 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:31.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:31.135 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:31.135 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:31.236 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:31.240 INFO:tasks.thrashosds.thrasher:j is {'epoch': 225, 'fsid': '3df9647e-779b-11ef-baf5-efdc52797490', 'created': '2024-09-20T21:57:31.244934+0000', 'modified': '2024-09-20T22:09:27.465687+0000', 'last_up_change': '2024-09-20T22:06:53.796679+0000', 'last_in_change': '2024-09-20T22:02:21.189225+0000', 'flags': 'noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5806080, 'flags_set': ['nodeep-scrub', 'noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 8, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-20T22:00:35.131125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '199', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '197', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.180000066757202, 'score_stable': 2.180000066757202, 'optimal_score': 1, 'raw_score_acting': 2.180000066757202, 'raw_score_stable': 2.180000066757202, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-20T22:02:47.974480+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6500000953674316, 'score_stable': 2.6500000953674316, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-20T22:02:48.902465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-20T22:02:50.620057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-20T22:02:52.722912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-20T22:02:54.794248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'unique_pool_1', 'create_time': '2024-09-20T22:07:51.043037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 26, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '217', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '215', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.1500000953674316, 'score_stable': 1.5399999618530273, 'optimal_score': 1, 'raw_score_acting': 2.1500000953674316, 'raw_score_stable': 1.5399999618530273, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '0997c01a-a9bd-421e-abcc-fbcf7abe8512', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 218, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6802', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6803', 'nonce': 1395849550}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6804', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6805', 'nonce': 1395849550}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6808', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6809', 'nonce': 1395849550}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6806', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6807', 'nonce': 1395849550}]}, 'public_addr': '172.21.15.60:6803/1395849550', 'cluster_addr': '172.21.15.60:6805/1395849550', 'heartbeat_back_addr': '172.21.15.60:6809/1395849550', 'heartbeat_front_addr': '172.21.15.60:6807/1395849550', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c04bdcb1-d76b-47c8-a38b-c02043df6ba8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 90, 'up_from': 145, 'up_thru': 218, 'down_at': 91, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6810', 'nonce': 2970874921}, {'type': 'v1', 'addr': '172.21.15.60:6811', 'nonce': 2970874921}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6812', 'nonce': 2970874921}, {'type': 'v1', 'addr': '172.21.15.60:6813', 'nonce': 2970874921}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6816', 'nonce': 2970874921}, {'type': 'v1', 'addr': '172.21.15.60:6817', 'nonce': 2970874921}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6814', 'nonce': 2970874921}, {'type': 'v1', 'addr': '172.21.15.60:6815', 'nonce': 2970874921}]}, 'public_addr': '172.21.15.60:6811/2970874921', 'cluster_addr': '172.21.15.60:6813/2970874921', 'heartbeat_back_addr': '172.21.15.60:6817/2970874921', 'heartbeat_front_addr': '172.21.15.60:6815/2970874921', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '32c09d19-eb98-4926-a44e-e7ddee152956', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 204, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6818', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6819', 'nonce': 3698562274}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6820', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6821', 'nonce': 3698562274}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6824', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6825', 'nonce': 3698562274}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6822', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6823', 'nonce': 3698562274}]}, 'public_addr': '172.21.15.60:6819/3698562274', 'cluster_addr': '172.21.15.60:6821/3698562274', 'heartbeat_back_addr': '172.21.15.60:6825/3698562274', 'heartbeat_front_addr': '172.21.15.60:6823/3698562274', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'b1c2cb69-8565-49ac-b854-7ea7615c6660', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 218, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6826', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6827', 'nonce': 2592322449}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6828', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6829', 'nonce': 2592322449}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6832', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6833', 'nonce': 2592322449}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6830', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6831', 'nonce': 2592322449}]}, 'public_addr': '172.21.15.60:6827/2592322449', 'cluster_addr': '172.21.15.60:6829/2592322449', 'heartbeat_back_addr': '172.21.15.60:6833/2592322449', 'heartbeat_front_addr': '172.21.15.60:6831/2592322449', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '04a547d6-fc48-43b7-a8d7-18df8a83aa96', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 218, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6800', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6801', 'nonce': 3243564239}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6802', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6803', 'nonce': 3243564239}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6806', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6807', 'nonce': 3243564239}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6804', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6805', 'nonce': 3243564239}]}, 'public_addr': '172.21.15.67:6801/3243564239', 'cluster_addr': '172.21.15.67:6803/3243564239', 'heartbeat_back_addr': '172.21.15.67:6807/3243564239', 'heartbeat_front_addr': '172.21.15.67:6805/3243564239', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4763aa10-c367-485b-b525-e5241e642b86', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 223, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6808', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6809', 'nonce': 1622258887}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6810', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6811', 'nonce': 1622258887}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6814', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6815', 'nonce': 1622258887}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6812', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6813', 'nonce': 1622258887}]}, 'public_addr': '172.21.15.67:6809/1622258887', 'cluster_addr': '172.21.15.67:6811/1622258887', 'heartbeat_back_addr': '172.21.15.67:6815/1622258887', 'heartbeat_front_addr': '172.21.15.67:6813/1622258887', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'a359116e-c013-4e2f-8b79-86219516ee16', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 48, 'up_thru': 217, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6816', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6817', 'nonce': 1187114}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6818', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6819', 'nonce': 1187114}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6822', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6823', 'nonce': 1187114}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6820', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6821', 'nonce': 1187114}]}, 'public_addr': '172.21.15.67:6817/1187114', 'cluster_addr': '172.21.15.67:6819/1187114', 'heartbeat_back_addr': '172.21.15.67:6823/1187114', 'heartbeat_front_addr': '172.21.15.67:6821/1187114', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a4bf4188-f133-4a31-8897-d7e8182248d7', 'up': 1, 'in': 1, 'weight': 0.732696533203125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 57, 'up_thru': 218, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6824', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6825', 'nonce': 3021842840}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6826', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6827', 'nonce': 3021842840}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6830', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6831', 'nonce': 3021842840}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6828', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6829', 'nonce': 3021842840}]}, 'public_addr': '172.21.15.67:6825/3021842840', 'cluster_addr': '172.21.15.67:6827/3021842840', 'heartbeat_back_addr': '172.21.15.67:6831/3021842840', 'heartbeat_front_addr': '172.21.15.67:6829/3021842840', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:08:37.160825+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:09:20.725454+0000', 'dead_epoch': 90}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:08:24.811711+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:08:37.826751+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:08:55.939475+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:09:21.083547+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:09:25.296562+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:09:27.456118+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [{'pgid': '3.1a', 'mappings': [{'from': 4, 'to': 5}]}, {'pgid': '3.1c', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.1b', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '6.7', 'mappings': [{'from': 3, 'to': 0}]}, {'pgid': '6.9', 'mappings': [{'from': 7, 'to': 1}]}, {'pgid': '6.b', 'mappings': [{'from': 3, 'to': 0}]}, {'pgid': '6.d', 'mappings': [{'from': 5, 'to': 6}]}, {'pgid': '6.11', 'mappings': [{'from': 3, 'to': 1}]}, {'pgid': '6.19', 'mappings': [{'from': 3, 'to': 6}, {'from': 5, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '8.c', 'osds': [0, 3, 7]}, {'pgid': '8.10', 'osds': [1, 5, 3]}, {'pgid': '8.11', 'osds': [0, 1, 2]}, {'pgid': '8.12', 'osds': [0, 4, 5]}, {'pgid': '8.13', 'osds': [1, 5, 6]}, {'pgid': '8.14', 'osds': [4, 2, 5]}, {'pgid': '8.15', 'osds': [7, 6, 4]}, {'pgid': '8.16', 'osds': [0, 6, 7]}, {'pgid': '8.17', 'osds': [1, 6, 3]}, {'pgid': '8.18', 'osds': [3, 5, 7]}], 'primary_temp': [], 'blocklist': {'172.21.15.60:0/890154710': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:0/3386526350': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:6801/604714621': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:6800/604714621': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:6800/3151907776': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/956264518': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/4150585713': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/3311255314': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:0/3965541528': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:6800/3734574621': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:0/1729862661': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:6801/3734574621': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:0/3927294026': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:6801/3151907776': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/1871351181': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:0/1854455098': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:0/2221262203': '2024-09-21T22:03:19.518405+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-20T22:09:31.241 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:09:31.289 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:31.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:31.300 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:31.300 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:31.380 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:09:31.401 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:31.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:31.474 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:31.475 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:31.575 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:31.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:31.626 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:31.626 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:31.727 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:31.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:31.783 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:31.783 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:31.884 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:31.922 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:31.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:31.923 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:32.017 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:32.019 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:09:32.019 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:09:32.019 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:09:32.019 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:09:32.019 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.019 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169963 7.15 7:acd7b1d6:::benchmark_data_smithi060_78800_object36930:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:09:32.019 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.440422+0000", 2024-09-20T22:09:32.019 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 213.575071571, 2024-09-20T22:09:32.019 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0806133010000001, 2024-09-20T22:09:32.020 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.020 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.020 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:09:32.020 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.020 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:32.020 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:32.020 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169963 2024-09-20T22:09:32.020 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.020 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.022 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.022 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.022 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440422+0000", 2024-09-20T22:09:32.022 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.022 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.022 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.022 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.022 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440422+0000", 2024-09-20T22:09:32.022 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.022 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.023 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.023 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.023 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440424+0000", 2024-09-20T22:09:32.023 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.762e-06 2024-09-20T22:09:32.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.023 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.023 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.023 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440432+0000", 2024-09-20T22:09:32.023 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4530000000000002e-06 2024-09-20T22:09:32.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.023 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.023 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.023 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440448+0000", 2024-09-20T22:09:32.023 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5084e-05 2024-09-20T22:09:32.024 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.024 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.024 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.024 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440456+0000", 2024-09-20T22:09:32.024 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0020000000000006e-06 2024-09-20T22:09:32.024 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.024 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.024 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.024 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549826+0000", 2024-09-20T22:09:32.024 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.109370678 2024-09-20T22:09:32.024 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.024 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.024 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.024 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549832+0000", 2024-09-20T22:09:32.025 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3850000000000001e-06 2024-09-20T22:09:32.025 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.025 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.025 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.025 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.503733+0000", 2024-09-20T22:09:32.025 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.95390151700000003 2024-09-20T22:09:32.025 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.025 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.025 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for missing object", 2024-09-20T22:09:32.025 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.503868+0000", 2024-09-20T22:09:32.025 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013515399999999999 2024-09-20T22:09:32.025 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.025 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.025 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.026 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.520960+0000", 2024-09-20T22:09:32.026 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.017092006 2024-09-20T22:09:32.026 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.026 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.026 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.026 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.521036+0000", 2024-09-20T22:09:32.026 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5259999999999994e-05 2024-09-20T22:09:32.026 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.026 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.026 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.026 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.026 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.026 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169964 7.15 7:aca939f2:::benchmark_data_smithi060_78800_object36931:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:09:32.027 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.440473+0000", 2024-09-20T22:09:32.027 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 213.57502112200001, 2024-09-20T22:09:32.027 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.081014133, 2024-09-20T22:09:32.027 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.027 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.027 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:09:32.027 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.027 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:32.027 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:32.027 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169964 2024-09-20T22:09:32.027 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.027 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.027 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.027 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440473+0000", 2024-09-20T22:09:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440473+0000", 2024-09-20T22:09:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440473+0000", 2024-09-20T22:09:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0499999999999999e-07 2024-09-20T22:09:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.028 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.029 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.029 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440476+0000", 2024-09-20T22:09:32.029 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0750000000000002e-06 2024-09-20T22:09:32.029 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.029 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.029 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.029 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440487+0000", 2024-09-20T22:09:32.029 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1048e-05 2024-09-20T22:09:32.029 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.029 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.029 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.029 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440493+0000", 2024-09-20T22:09:32.029 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.31e-06 2024-09-20T22:09:32.029 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.030 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.030 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.030 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549842+0000", 2024-09-20T22:09:32.030 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.109349718 2024-09-20T22:09:32.030 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.030 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.030 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.030 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549845+0000", 2024-09-20T22:09:32.030 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0029999999999999e-06 2024-09-20T22:09:32.030 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.030 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.031 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.031 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.504130+0000", 2024-09-20T22:09:32.031 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.95428438100000001 2024-09-20T22:09:32.031 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.031 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.031 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for missing object", 2024-09-20T22:09:32.031 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.504241+0000", 2024-09-20T22:09:32.031 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011154099999999999 2024-09-20T22:09:32.031 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.031 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.031 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.031 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.521422+0000", 2024-09-20T22:09:32.032 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.017180540000000001 2024-09-20T22:09:32.032 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.032 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.032 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.032 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.521487+0000", 2024-09-20T22:09:32.032 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5111999999999996e-05 2024-09-20T22:09:32.032 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.032 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.032 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.032 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.032 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.032 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:236437 7.11 7:8eaecf20:::benchmark_data_smithi060_78800_object103404:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:09:32.032 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:07.392155+0000", 2024-09-20T22:09:32.032 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 144.623338668, 2024-09-20T22:09:32.033 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.019005036, 2024-09-20T22:09:32.033 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.033 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.033 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:32.033 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.033 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:32.033 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:32.033 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 236437 2024-09-20T22:09:32.033 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.033 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.033 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.033 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.033 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392155+0000", 2024-09-20T22:09:32.033 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.034 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.034 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.034 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.034 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392155+0000", 2024-09-20T22:09:32.034 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.034 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.034 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.034 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.034 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392157+0000", 2024-09-20T22:09:32.034 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9599999999999999e-06 2024-09-20T22:09:32.034 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.034 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.034 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.034 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392162+0000", 2024-09-20T22:09:32.035 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1719999999999998e-06 2024-09-20T22:09:32.035 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.035 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.035 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:32.036 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:32.036 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.036 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392180+0000", 2024-09-20T22:09:32.036 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8088e-05 2024-09-20T22:09:32.036 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.036 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.037 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.037 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392188+0000", 2024-09-20T22:09:32.037 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9929999999999995e-06 2024-09-20T22:09:32.037 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.037 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.037 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.037 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392254+0000", 2024-09-20T22:09:32.037 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6137999999999996e-05 2024-09-20T22:09:32.037 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.037 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.037 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.037 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392260+0000", 2024-09-20T22:09:32.037 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7089999999999999e-06 2024-09-20T22:09:32.038 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.038 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.038 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.038 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.406485+0000", 2024-09-20T22:09:32.038 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0142251280000001 2024-09-20T22:09:32.038 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.038 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.038 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:32.038 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.406593+0000", 2024-09-20T22:09:32.038 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000107672 2024-09-20T22:09:32.038 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.038 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.038 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:09:32.039 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.406954+0000", 2024-09-20T22:09:32.039 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000360973 2024-09-20T22:09:32.039 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.039 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.039 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:32.039 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411014+0000", 2024-09-20T22:09:32.039 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00406033 2024-09-20T22:09:32.039 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.039 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.039 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.039 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411059+0000", 2024-09-20T22:09:32.039 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4907999999999998e-05 2024-09-20T22:09:32.039 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.040 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.040 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.040 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411090+0000", 2024-09-20T22:09:32.040 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0874000000000003e-05 2024-09-20T22:09:32.040 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.040 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.040 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:32.040 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411121+0000", 2024-09-20T22:09:32.040 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0518999999999997e-05 2024-09-20T22:09:32.040 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.040 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.040 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.040 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411160+0000", 2024-09-20T22:09:32.040 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9572e-05 2024-09-20T22:09:32.040 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.041 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.041 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.041 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.041 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.041 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:236538 7.11 7:8ee390f6:::benchmark_data_smithi060_78800_object103505:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:09:32.041 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:07.506607+0000", 2024-09-20T22:09:32.041 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 144.508887042, 2024-09-20T22:09:32.041 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.90916602000000002, 2024-09-20T22:09:32.041 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.041 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.041 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:32.041 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.041 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:32.041 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:32.041 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 236538 2024-09-20T22:09:32.042 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.042 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.042 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.042 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.042 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.506607+0000", 2024-09-20T22:09:32.042 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.042 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.042 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.042 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.042 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.506607+0000", 2024-09-20T22:09:32.042 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.042 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.042 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.042 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.043 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.506608+0000", 2024-09-20T22:09:32.043 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4759999999999999e-06 2024-09-20T22:09:32.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.043 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.043 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.043 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.506614+0000", 2024-09-20T22:09:32.043 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.344e-06 2024-09-20T22:09:32.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.043 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.043 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.043 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.506630+0000", 2024-09-20T22:09:32.043 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6305000000000001e-05 2024-09-20T22:09:32.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.043 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.044 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.044 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.506638+0000", 2024-09-20T22:09:32.044 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8660000000000006e-06 2024-09-20T22:09:32.044 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.044 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.044 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.044 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.506716+0000", 2024-09-20T22:09:32.044 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8469999999999999e-05 2024-09-20T22:09:32.044 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.044 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.044 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.044 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.506725+0000", 2024-09-20T22:09:32.044 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4509999999999992e-06 2024-09-20T22:09:32.044 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.044 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.045 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.045 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.409302+0000", 2024-09-20T22:09:32.045 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.90257712300000004 2024-09-20T22:09:32.045 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.045 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.045 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:32.045 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.409402+0000", 2024-09-20T22:09:32.045 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100457 2024-09-20T22:09:32.045 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.045 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.045 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:09:32.045 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.409647+0000", 2024-09-20T22:09:32.045 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00024485799999999998 2024-09-20T22:09:32.045 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.046 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.046 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:32.046 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411214+0000", 2024-09-20T22:09:32.046 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0015664159999999999 2024-09-20T22:09:32.046 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.046 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.046 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.046 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411405+0000", 2024-09-20T22:09:32.046 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019131399999999999 2024-09-20T22:09:32.046 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.046 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.046 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.046 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.415666+0000", 2024-09-20T22:09:32.046 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0042612099999999997 2024-09-20T22:09:32.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.047 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.047 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:32.047 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.415726+0000", 2024-09-20T22:09:32.047 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0269999999999997e-05 2024-09-20T22:09:32.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.047 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.047 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.047 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.415773+0000", 2024-09-20T22:09:32.047 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6459999999999999e-05 2024-09-20T22:09:32.047 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.047 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.047 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.048 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.048 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.048 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261924 7.17 7:ec34c39a:::benchmark_data_smithi060_78800_object128891:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:09:32.048 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.137969+0000", 2024-09-20T22:09:32.048 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 115.877524368, 2024-09-20T22:09:32.048 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0378232009999999, 2024-09-20T22:09:32.048 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.048 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.048 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:09:32.048 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.048 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:32.048 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:32.048 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261924 2024-09-20T22:09:32.048 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.049 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.049 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.049 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.049 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137969+0000", 2024-09-20T22:09:32.049 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.049 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.049 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.049 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.049 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137969+0000", 2024-09-20T22:09:32.049 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.049 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.049 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.049 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.049 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137971+0000", 2024-09-20T22:09:32.049 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.19e-06 2024-09-20T22:09:32.050 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.050 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.050 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.050 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137976+0000", 2024-09-20T22:09:32.050 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8429999999999998e-06 2024-09-20T22:09:32.050 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.050 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.050 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.050 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137987+0000", 2024-09-20T22:09:32.050 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0587e-05 2024-09-20T22:09:32.050 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.050 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.050 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.051 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137992+0000", 2024-09-20T22:09:32.051 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4090000000000002e-06 2024-09-20T22:09:32.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.051 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.051 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.051 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138048+0000", 2024-09-20T22:09:32.051 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5241999999999997e-05 2024-09-20T22:09:32.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.051 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.051 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.051 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138056+0000", 2024-09-20T22:09:32.051 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4649999999999994e-06 2024-09-20T22:09:32.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.051 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.052 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.052 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.175735+0000", 2024-09-20T22:09:32.052 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.037678871 2024-09-20T22:09:32.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.052 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.052 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.175793+0000", 2024-09-20T22:09:32.052 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7593999999999998e-05 2024-09-20T22:09:32.052 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.052 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.052 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.052 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261927 7.17 7:ed776cb1:::benchmark_data_smithi060_78800_object128894:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:09:32.053 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.138007+0000", 2024-09-20T22:09:32.053 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 115.877486941, 2024-09-20T22:09:32.053 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0471626510000001, 2024-09-20T22:09:32.053 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.053 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.053 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:09:32.053 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.053 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:32.053 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:32.053 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261927 2024-09-20T22:09:32.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.053 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.053 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.053 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.054 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138007+0000", 2024-09-20T22:09:32.054 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.054 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.055 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.055 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138007+0000", 2024-09-20T22:09:32.055 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.055 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.055 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138007+0000", 2024-09-20T22:09:32.055 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5600000000000001e-07 2024-09-20T22:09:32.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.056 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.056 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.056 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138009+0000", 2024-09-20T22:09:32.056 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6729999999999999e-06 2024-09-20T22:09:32.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.056 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.056 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.056 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138013+0000", 2024-09-20T22:09:32.056 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.16e-06 2024-09-20T22:09:32.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.056 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.056 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.056 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138016+0000", 2024-09-20T22:09:32.056 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1609999999999999e-06 2024-09-20T22:09:32.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.057 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.057 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138084+0000", 2024-09-20T22:09:32.057 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8044000000000001e-05 2024-09-20T22:09:32.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.057 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.057 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138093+0000", 2024-09-20T22:09:32.057 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.49e-06 2024-09-20T22:09:32.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.057 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.058 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177120+0000", 2024-09-20T22:09:32.058 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0390272819999999 2024-09-20T22:09:32.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.058 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.058 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for missing object", 2024-09-20T22:09:32.058 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177222+0000", 2024-09-20T22:09:32.058 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000102272 2024-09-20T22:09:32.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.058 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.058 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.058 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185103+0000", 2024-09-20T22:09:32.058 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0078803929999999994 2024-09-20T22:09:32.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.059 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.059 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.059 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185170+0000", 2024-09-20T22:09:32.059 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6820000000000001e-05 2024-09-20T22:09:32.059 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.059 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.059 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.059 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.059 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261940 7.17 7:e9ae652d:::benchmark_data_smithi060_78800_object128907:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:09:32.059 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.144358+0000", 2024-09-20T22:09:32.059 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 115.87113602399999, 2024-09-20T22:09:32.059 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.040411067, 2024-09-20T22:09:32.059 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.060 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.060 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:32.060 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.060 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:32.060 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:32.060 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261940 2024-09-20T22:09:32.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.060 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.060 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.060 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144358+0000", 2024-09-20T22:09:32.060 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.060 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.061 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144358+0000", 2024-09-20T22:09:32.061 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.061 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.061 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144359+0000", 2024-09-20T22:09:32.061 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9299999999999996e-07 2024-09-20T22:09:32.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.061 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.061 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144362+0000", 2024-09-20T22:09:32.061 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6799999999999999e-06 2024-09-20T22:09:32.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.062 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.062 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144369+0000", 2024-09-20T22:09:32.062 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8070000000000002e-06 2024-09-20T22:09:32.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.062 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.062 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.062 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144374+0000", 2024-09-20T22:09:32.062 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8999999999999997e-06 2024-09-20T22:09:32.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.062 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.062 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.062 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144416+0000", 2024-09-20T22:09:32.062 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2049999999999999e-05 2024-09-20T22:09:32.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.063 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.063 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.063 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144421+0000", 2024-09-20T22:09:32.063 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6550000000000003e-06 2024-09-20T22:09:32.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.063 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.063 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.063 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177283+0000", 2024-09-20T22:09:32.063 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0328618039999999 2024-09-20T22:09:32.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.063 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.063 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:32.063 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177468+0000", 2024-09-20T22:09:32.063 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00018551900000000001 2024-09-20T22:09:32.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.064 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.064 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:09:32.064 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177846+0000", 2024-09-20T22:09:32.064 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00037771700000000002 2024-09-20T22:09:32.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.064 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.064 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.064 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184561+0000", 2024-09-20T22:09:32.064 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0067147600000000002 2024-09-20T22:09:32.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.064 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.064 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:32.064 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184572+0000", 2024-09-20T22:09:32.064 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1433999999999999e-05 2024-09-20T22:09:32.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.065 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.065 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.065 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184635+0000", 2024-09-20T22:09:32.065 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3459000000000003e-05 2024-09-20T22:09:32.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.065 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.065 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:32.065 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184692+0000", 2024-09-20T22:09:32.065 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6759999999999999e-05 2024-09-20T22:09:32.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.065 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.065 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.066 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184769+0000", 2024-09-20T22:09:32.066 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6629000000000001e-05 2024-09-20T22:09:32.066 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.066 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.066 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.066 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.066 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261963 7.17 7:eacd681d:::benchmark_data_smithi060_78800_object128930:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:09:32.066 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.165356+0000", 2024-09-20T22:09:32.066 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 115.85013744600001, 2024-09-20T22:09:32.066 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.020053374, 2024-09-20T22:09:32.066 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.066 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.067 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:32.067 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.067 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:32.067 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:32.067 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261963 2024-09-20T22:09:32.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.067 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.067 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.067 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165356+0000", 2024-09-20T22:09:32.067 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.067 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.068 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165356+0000", 2024-09-20T22:09:32.068 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.068 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.068 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165357+0000", 2024-09-20T22:09:32.068 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.3299999999999998e-07 2024-09-20T22:09:32.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.068 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.068 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165359+0000", 2024-09-20T22:09:32.068 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.046e-06 2024-09-20T22:09:32.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.068 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.069 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165364+0000", 2024-09-20T22:09:32.069 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.502e-06 2024-09-20T22:09:32.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.069 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.069 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.069 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165369+0000", 2024-09-20T22:09:32.069 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6319999999999999e-06 2024-09-20T22:09:32.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.069 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.069 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.069 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.171567+0000", 2024-09-20T22:09:32.069 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061976460000000002 2024-09-20T22:09:32.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.069 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.069 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.069 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.171575+0000", 2024-09-20T22:09:32.070 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2390000000000002e-06 2024-09-20T22:09:32.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.070 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.070 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.070 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.180431+0000", 2024-09-20T22:09:32.070 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0088558569999999 2024-09-20T22:09:32.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.070 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.070 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:32.070 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.180570+0000", 2024-09-20T22:09:32.070 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013935799999999999 2024-09-20T22:09:32.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.070 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.071 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:09:32.071 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.180897+0000", 2024-09-20T22:09:32.071 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00032708599999999998 2024-09-20T22:09:32.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.071 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.071 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:32.071 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184575+0000", 2024-09-20T22:09:32.071 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0036776790000000001 2024-09-20T22:09:32.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.071 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.071 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.071 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185202+0000", 2024-09-20T22:09:32.071 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00062773099999999997 2024-09-20T22:09:32.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.072 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.072 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.072 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185253+0000", 2024-09-20T22:09:32.072 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0742999999999999e-05 2024-09-20T22:09:32.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.072 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.072 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:32.072 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185303+0000", 2024-09-20T22:09:32.072 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9339000000000003e-05 2024-09-20T22:09:32.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.072 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.072 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.072 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185410+0000", 2024-09-20T22:09:32.072 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010718299999999999 2024-09-20T22:09:32.072 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.073 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.073 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.073 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.073 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261986 7.17 7:ebad35fd:::benchmark_data_smithi060_78800_object128953:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:09:32.073 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.200930+0000", 2024-09-20T22:09:32.073 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 115.814563354, 2024-09-20T22:09:32.073 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98696583999999998, 2024-09-20T22:09:32.073 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.073 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.073 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:32.073 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.073 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:32.073 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:32.073 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261986 2024-09-20T22:09:32.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.074 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.074 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.074 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200930+0000", 2024-09-20T22:09:32.074 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.074 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.074 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200930+0000", 2024-09-20T22:09:32.074 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.075 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.075 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200931+0000", 2024-09-20T22:09:32.075 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8800000000000001e-07 2024-09-20T22:09:32.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.075 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.075 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.075 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200936+0000", 2024-09-20T22:09:32.075 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3939999999999998e-06 2024-09-20T22:09:32.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.075 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.075 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.075 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200946+0000", 2024-09-20T22:09:32.075 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0706999999999999e-05 2024-09-20T22:09:32.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.076 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.076 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200953+0000", 2024-09-20T22:09:32.076 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9260000000000003e-06 2024-09-20T22:09:32.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.076 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.076 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.201017+0000", 2024-09-20T22:09:32.076 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3793000000000004e-05 2024-09-20T22:09:32.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.076 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.076 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.201028+0000", 2024-09-20T22:09:32.076 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0577000000000001e-05 2024-09-20T22:09:32.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.077 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.077 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.077 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.183246+0000", 2024-09-20T22:09:32.077 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98221816100000003 2024-09-20T22:09:32.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.077 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.077 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:32.077 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.183318+0000", 2024-09-20T22:09:32.077 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1890000000000005e-05 2024-09-20T22:09:32.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.077 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.077 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:09:32.077 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.183503+0000", 2024-09-20T22:09:32.077 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00018529399999999999 2024-09-20T22:09:32.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.078 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.078 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:32.078 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185423+0000", 2024-09-20T22:09:32.078 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0019203390000000001 2024-09-20T22:09:32.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.079 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.079 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.079 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185478+0000", 2024-09-20T22:09:32.079 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5041999999999999e-05 2024-09-20T22:09:32.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.080 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.080 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.080 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.187745+0000", 2024-09-20T22:09:32.080 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0022662239999999998 2024-09-20T22:09:32.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.080 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.080 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:32.080 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.187795+0000", 2024-09-20T22:09:32.080 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0668999999999999e-05 2024-09-20T22:09:32.080 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.080 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.080 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.080 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.187896+0000", 2024-09-20T22:09:32.080 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100936 2024-09-20T22:09:32.081 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.081 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.081 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.081 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.081 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.081 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262055 7.17 7:eca2ebed:::benchmark_data_smithi060_78800_object129022:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:09:32.081 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.253316+0000", 2024-09-20T22:09:32.081 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 115.76217821500001, 2024-09-20T22:09:32.081 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93859176700000002, 2024-09-20T22:09:32.081 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.081 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.081 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:32.081 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.081 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:32.082 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:32.082 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262055 2024-09-20T22:09:32.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.082 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.082 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.082 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.082 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253316+0000", 2024-09-20T22:09:32.082 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.082 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.082 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.082 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253316+0000", 2024-09-20T22:09:32.082 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.083 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.083 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.083 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253316+0000", 2024-09-20T22:09:32.083 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2999999999999999e-07 2024-09-20T22:09:32.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.083 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.083 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.083 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253320+0000", 2024-09-20T22:09:32.083 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9129999999999996e-06 2024-09-20T22:09:32.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.083 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.083 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.083 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253333+0000", 2024-09-20T22:09:32.084 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2762000000000001e-05 2024-09-20T22:09:32.084 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.084 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.084 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.084 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253340+0000", 2024-09-20T22:09:32.084 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0180000000000004e-06 2024-09-20T22:09:32.084 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.084 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.084 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.084 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253402+0000", 2024-09-20T22:09:32.084 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2211000000000005e-05 2024-09-20T22:09:32.084 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.084 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.084 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.084 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253409+0000", 2024-09-20T22:09:32.085 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.985e-06 2024-09-20T22:09:32.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.085 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.085 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185521+0000", 2024-09-20T22:09:32.085 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93211197400000001 2024-09-20T22:09:32.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.085 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:32.085 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185620+0000", 2024-09-20T22:09:32.085 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9162999999999999e-05 2024-09-20T22:09:32.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.085 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:09:32.085 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185870+0000", 2024-09-20T22:09:32.086 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00024926499999999998 2024-09-20T22:09:32.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.086 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.086 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191631+0000", 2024-09-20T22:09:32.086 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.005761375 2024-09-20T22:09:32.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.086 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:32.086 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191650+0000", 2024-09-20T22:09:32.086 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8533999999999999e-05 2024-09-20T22:09:32.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.086 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.087 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191717+0000", 2024-09-20T22:09:32.087 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7731999999999995e-05 2024-09-20T22:09:32.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.087 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.087 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:32.087 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191780+0000", 2024-09-20T22:09:32.087 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2447999999999996e-05 2024-09-20T22:09:32.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.087 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.087 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.087 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191907+0000", 2024-09-20T22:09:32.087 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012755700000000001 2024-09-20T22:09:32.087 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.087 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.087 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.088 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.088 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262136 7.17 7:e9d369cd:::benchmark_data_smithi060_78800_object129103:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:09:32.088 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.316296+0000", 2024-09-20T22:09:32.088 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 115.699198079, 2024-09-20T22:09:32.088 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87816596999999996, 2024-09-20T22:09:32.088 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.088 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.088 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:32.088 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.088 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:32.088 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:32.088 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262136 2024-09-20T22:09:32.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.089 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.089 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.089 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.089 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.316296+0000", 2024-09-20T22:09:32.089 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.089 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.089 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.089 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.316296+0000", 2024-09-20T22:09:32.089 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.089 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.089 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.089 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.316297+0000", 2024-09-20T22:09:32.089 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.297e-06 2024-09-20T22:09:32.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.090 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.090 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.090 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.316301+0000", 2024-09-20T22:09:32.090 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2690000000000001e-06 2024-09-20T22:09:32.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.090 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.090 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.090 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.316312+0000", 2024-09-20T22:09:32.090 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0891e-05 2024-09-20T22:09:32.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.090 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.090 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.090 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.316319+0000", 2024-09-20T22:09:32.090 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8539999999999999e-06 2024-09-20T22:09:32.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.091 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.091 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.091 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.316378+0000", 2024-09-20T22:09:32.091 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8956000000000003e-05 2024-09-20T22:09:32.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.091 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.091 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.091 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.316392+0000", 2024-09-20T22:09:32.091 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.379e-05 2024-09-20T22:09:32.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.091 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.091 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.091 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.187931+0000", 2024-09-20T22:09:32.092 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87153904400000004 2024-09-20T22:09:32.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.092 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.092 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:32.092 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.188062+0000", 2024-09-20T22:09:32.092 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013087299999999999 2024-09-20T22:09:32.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.092 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.092 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:09:32.092 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.188381+0000", 2024-09-20T22:09:32.092 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00031884699999999998 2024-09-20T22:09:32.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.092 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.092 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.093 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194156+0000", 2024-09-20T22:09:32.093 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0057755100000000002 2024-09-20T22:09:32.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.093 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.093 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:32.093 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194246+0000", 2024-09-20T22:09:32.093 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9977999999999998e-05 2024-09-20T22:09:32.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.093 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.093 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.093 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194281+0000", 2024-09-20T22:09:32.093 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5389000000000003e-05 2024-09-20T22:09:32.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.093 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.094 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:32.094 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194335+0000", 2024-09-20T22:09:32.094 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3434999999999999e-05 2024-09-20T22:09:32.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.094 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.094 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.094 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194462+0000", 2024-09-20T22:09:32.094 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012683700000000001 2024-09-20T22:09:32.094 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.094 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.094 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.094 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.094 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262158 7.17 7:ef5434de:::benchmark_data_smithi060_78800_object129125:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:09:32.095 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.334932+0000", 2024-09-20T22:09:32.095 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 115.68056158500001, 2024-09-20T22:09:32.095 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.85976260999999998, 2024-09-20T22:09:32.095 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.095 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.095 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:32.095 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.095 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:32.095 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:32.095 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262158 2024-09-20T22:09:32.095 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.095 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.095 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.095 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.096 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.334932+0000", 2024-09-20T22:09:32.096 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.096 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.096 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.096 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.334932+0000", 2024-09-20T22:09:32.096 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.096 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.096 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.096 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.334933+0000", 2024-09-20T22:09:32.096 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.234e-06 2024-09-20T22:09:32.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.096 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.097 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.097 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.334938+0000", 2024-09-20T22:09:32.097 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2939999999999999e-06 2024-09-20T22:09:32.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.097 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.097 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.097 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346642+0000", 2024-09-20T22:09:32.097 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.011703743000000001 2024-09-20T22:09:32.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.097 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.097 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.097 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346653+0000", 2024-09-20T22:09:32.097 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1065e-05 2024-09-20T22:09:32.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.098 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.098 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.098 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346718+0000", 2024-09-20T22:09:32.098 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5714000000000001e-05 2024-09-20T22:09:32.098 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.098 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.098 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.098 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346727+0000", 2024-09-20T22:09:32.098 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9740000000000008e-06 2024-09-20T22:09:32.098 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.098 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.098 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.098 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.189854+0000", 2024-09-20T22:09:32.099 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.84312635899999999 2024-09-20T22:09:32.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.099 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.099 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:32.099 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.189977+0000", 2024-09-20T22:09:32.099 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000123745 2024-09-20T22:09:32.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.099 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.099 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:09:32.099 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.190187+0000", 2024-09-20T22:09:32.099 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020927200000000001 2024-09-20T22:09:32.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.099 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.099 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.100 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194231+0000", 2024-09-20T22:09:32.100 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0040443370000000003 2024-09-20T22:09:32.100 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.100 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.100 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:32.100 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194475+0000", 2024-09-20T22:09:32.100 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00024404299999999999 2024-09-20T22:09:32.100 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.100 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.100 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.100 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194528+0000", 2024-09-20T22:09:32.100 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3361e-05 2024-09-20T22:09:32.100 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.100 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194584+0000", 2024-09-20T22:09:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5337999999999998e-05 2024-09-20T22:09:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194695+0000", 2024-09-20T22:09:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000111131 2024-09-20T22:09:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262159 7.17 7:e923d856:::benchmark_data_smithi060_78800_object129126:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:09:32.101 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.346686+0000", 2024-09-20T22:09:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 115.668807883, 2024-09-20T22:09:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.85042171700000002, 2024-09-20T22:09:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262159 2024-09-20T22:09:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.102 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.103 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346686+0000", 2024-09-20T22:09:32.103 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.103 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.103 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.103 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346686+0000", 2024-09-20T22:09:32.103 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.103 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.103 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.103 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346687+0000", 2024-09-20T22:09:32.103 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.361e-06 2024-09-20T22:09:32.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.103 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.104 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.104 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346692+0000", 2024-09-20T22:09:32.104 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4819999999999996e-06 2024-09-20T22:09:32.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.104 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.104 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.104 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346702+0000", 2024-09-20T22:09:32.104 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0436e-05 2024-09-20T22:09:32.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.104 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.104 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.104 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346708+0000", 2024-09-20T22:09:32.104 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9710000000000001e-06 2024-09-20T22:09:32.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.105 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.105 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.105 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346754+0000", 2024-09-20T22:09:32.105 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6001999999999998e-05 2024-09-20T22:09:32.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.105 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.105 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.105 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.346761+0000", 2024-09-20T22:09:32.105 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9970000000000001e-06 2024-09-20T22:09:32.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.105 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.105 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.105 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191946+0000", 2024-09-20T22:09:32.105 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.84518505200000005 2024-09-20T22:09:32.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.106 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.106 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:32.106 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.192060+0000", 2024-09-20T22:09:32.106 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000113701 2024-09-20T22:09:32.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.106 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.106 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:09:32.106 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.192312+0000", 2024-09-20T22:09:32.106 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025226399999999998 2024-09-20T22:09:32.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.106 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.106 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.107 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194742+0000", 2024-09-20T22:09:32.107 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0024301380000000001 2024-09-20T22:09:32.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.107 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.107 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194798+0000", 2024-09-20T22:09:32.107 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5547999999999997e-05 2024-09-20T22:09:32.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.107 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:32.107 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.196951+0000", 2024-09-20T22:09:32.107 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0021531459999999999 2024-09-20T22:09:32.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.108 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:32.108 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.197003+0000", 2024-09-20T22:09:32.108 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1712999999999998e-05 2024-09-20T22:09:32.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.108 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.108 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.197108+0000", 2024-09-20T22:09:32.108 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000104906 2024-09-20T22:09:32.108 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.108 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.108 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.108 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262176 7.17 7:e9deef4b:::benchmark_data_smithi060_78800_object129143:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:09:32.109 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.358433+0000", 2024-09-20T22:09:32.109 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 115.657060913, 2024-09-20T22:09:32.109 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.84352511399999996, 2024-09-20T22:09:32.109 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.109 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.109 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:32.109 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.109 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:32.109 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:32.109 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262176 2024-09-20T22:09:32.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.109 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.110 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.110 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.358433+0000", 2024-09-20T22:09:32.110 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.110 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.110 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.358433+0000", 2024-09-20T22:09:32.110 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.110 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.110 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.358434+0000", 2024-09-20T22:09:32.110 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4470000000000001e-06 2024-09-20T22:09:32.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.111 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.111 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.111 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.358439+0000", 2024-09-20T22:09:32.111 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1479999999999997e-06 2024-09-20T22:09:32.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.111 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.111 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.111 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.358454+0000", 2024-09-20T22:09:32.111 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4423999999999999e-05 2024-09-20T22:09:32.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.111 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.111 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.111 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.358461+0000", 2024-09-20T22:09:32.111 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2080000000000003e-06 2024-09-20T22:09:32.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.112 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.112 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.358517+0000", 2024-09-20T22:09:32.112 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5819999999999997e-05 2024-09-20T22:09:32.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.112 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.112 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.358525+0000", 2024-09-20T22:09:32.112 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1559999999999995e-06 2024-09-20T22:09:32.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.112 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.113 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194849+0000", 2024-09-20T22:09:32.113 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.83632406999999997 2024-09-20T22:09:32.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.113 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.113 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:32.113 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.194922+0000", 2024-09-20T22:09:32.113 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3261000000000002e-05 2024-09-20T22:09:32.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.113 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.113 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:09:32.113 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.195218+0000", 2024-09-20T22:09:32.113 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000295302 2024-09-20T22:09:32.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.114 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.114 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.201692+0000", 2024-09-20T22:09:32.114 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0064745840000000002 2024-09-20T22:09:32.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.114 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:32.114 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.201706+0000", 2024-09-20T22:09:32.114 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4034e-05 2024-09-20T22:09:32.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.114 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.114 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.201754+0000", 2024-09-20T22:09:32.114 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7614999999999997e-05 2024-09-20T22:09:32.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.115 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:32.115 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.201809+0000", 2024-09-20T22:09:32.115 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4833999999999997e-05 2024-09-20T22:09:32.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.115 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.115 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.201958+0000", 2024-09-20T22:09:32.115 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014921100000000001 2024-09-20T22:09:32.115 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.115 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.115 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.116 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.116 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262238 7.17 7:eff2d775:::benchmark_data_smithi060_78800_object129205:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:09:32.116 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.423242+0000", 2024-09-20T22:09:32.116 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 115.59225197799999, 2024-09-20T22:09:32.116 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.77916317400000001, 2024-09-20T22:09:32.116 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.116 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.116 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:32.116 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.116 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:32.116 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:32.116 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262238 2024-09-20T22:09:32.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.116 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.117 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.117 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.423242+0000", 2024-09-20T22:09:32.117 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.117 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.117 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.423242+0000", 2024-09-20T22:09:32.117 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.117 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.117 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.423242+0000", 2024-09-20T22:09:32.117 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4999999999999998e-07 2024-09-20T22:09:32.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.118 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.118 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.423245+0000", 2024-09-20T22:09:32.118 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7080000000000002e-06 2024-09-20T22:09:32.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.118 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.118 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.423254+0000", 2024-09-20T22:09:32.118 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.448e-06 2024-09-20T22:09:32.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.118 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.118 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.423262+0000", 2024-09-20T22:09:32.119 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0790000000000003e-06 2024-09-20T22:09:32.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.119 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.119 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.423353+0000", 2024-09-20T22:09:32.119 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1340000000000003e-05 2024-09-20T22:09:32.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.119 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.119 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.423362+0000", 2024-09-20T22:09:32.119 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.0389999999999993e-06 2024-09-20T22:09:32.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.197143+0000", 2024-09-20T22:09:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.77378096799999996 2024-09-20T22:09:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.197253+0000", 2024-09-20T22:09:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000110289 2024-09-20T22:09:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:09:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.197537+0000", 2024-09-20T22:09:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00028388399999999998 2024-09-20T22:09:32.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.201968+0000", 2024-09-20T22:09:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0044307490000000003 2024-09-20T22:09:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.202082+0000", 2024-09-20T22:09:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000113998 2024-09-20T22:09:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.202203+0000", 2024-09-20T22:09:32.121 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012076199999999999 2024-09-20T22:09:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.202248+0000", 2024-09-20T22:09:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5782999999999999e-05 2024-09-20T22:09:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.202405+0000", 2024-09-20T22:09:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000156677 2024-09-20T22:09:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262347 7.17 7:ed5b7b4e:::benchmark_data_smithi060_78800_object129314:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:09:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.540334+0000", 2024-09-20T22:09:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 115.475160303, 2024-09-20T22:09:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.67407287100000002, 2024-09-20T22:09:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262347 2024-09-20T22:09:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.540334+0000", 2024-09-20T22:09:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.540334+0000", 2024-09-20T22:09:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.540335+0000", 2024-09-20T22:09:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.052e-06 2024-09-20T22:09:32.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.540339+0000", 2024-09-20T22:09:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2289999999999997e-06 2024-09-20T22:09:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.540349+0000", 2024-09-20T22:09:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0353000000000001e-05 2024-09-20T22:09:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.540355+0000", 2024-09-20T22:09:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4140000000000002e-06 2024-09-20T22:09:32.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.540414+0000", 2024-09-20T22:09:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9268999999999998e-05 2024-09-20T22:09:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.540422+0000", 2024-09-20T22:09:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3340000000000002e-06 2024-09-20T22:09:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.199217+0000", 2024-09-20T22:09:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.65879443900000001 2024-09-20T22:09:32.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.199321+0000", 2024-09-20T22:09:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010479700000000001 2024-09-20T22:09:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:09:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.199736+0000", 2024-09-20T22:09:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00041412300000000002 2024-09-20T22:09:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.202094+0000", 2024-09-20T22:09:32.127 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0023584320000000001 2024-09-20T22:09:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.209581+0000", 2024-09-20T22:09:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0074874870000000001 2024-09-20T22:09:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214225+0000", 2024-09-20T22:09:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046432519999999996 2024-09-20T22:09:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214276+0000", 2024-09-20T22:09:32.128 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0958999999999998e-05 2024-09-20T22:09:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214406+0000", 2024-09-20T22:09:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013073100000000001 2024-09-20T22:09:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262355 7.17 7:eb2c2f7e:::benchmark_data_smithi060_78800_object129322:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:09:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.547037+0000", 2024-09-20T22:09:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 115.468456859, 2024-09-20T22:09:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.66768182899999995, 2024-09-20T22:09:32.129 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:09:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262355 2024-09-20T22:09:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.547037+0000", 2024-09-20T22:09:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.130 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.547037+0000", 2024-09-20T22:09:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.547038+0000", 2024-09-20T22:09:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.062e-06 2024-09-20T22:09:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.547042+0000", 2024-09-20T22:09:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3540000000000002e-06 2024-09-20T22:09:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.131 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.547055+0000", 2024-09-20T22:09:32.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3033e-05 2024-09-20T22:09:32.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.132 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.132 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.547063+0000", 2024-09-20T22:09:32.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3810000000000001e-06 2024-09-20T22:09:32.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.132 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.132 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.547120+0000", 2024-09-20T22:09:32.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6947999999999998e-05 2024-09-20T22:09:32.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.132 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.132 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.547127+0000", 2024-09-20T22:09:32.133 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2760000000000003e-06 2024-09-20T22:09:32.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.133 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.133 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.133 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.202433+0000", 2024-09-20T22:09:32.134 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.65530552500000006 2024-09-20T22:09:32.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.134 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.134 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:32.134 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.202537+0000", 2024-09-20T22:09:32.134 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000104053 2024-09-20T22:09:32.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.134 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.135 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:09:32.135 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.202883+0000", 2024-09-20T22:09:32.135 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00034644800000000001 2024-09-20T22:09:32.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.135 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:32.135 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214187+0000", 2024-09-20T22:09:32.135 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.011303674999999999 2024-09-20T22:09:32.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.135 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.135 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214475+0000", 2024-09-20T22:09:32.135 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00028836600000000002 2024-09-20T22:09:32.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214588+0000", 2024-09-20T22:09:32.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000112427 2024-09-20T22:09:32.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.136 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:32.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214631+0000", 2024-09-20T22:09:32.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3140999999999999e-05 2024-09-20T22:09:32.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.136 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.214719+0000", 2024-09-20T22:09:32.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8140000000000007e-05 2024-09-20T22:09:32.136 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.137 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.137 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.137 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.137 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:106968 8.15 8:a8961342:::benchmark_data_smithi060_111051_object106967:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:09:32.137 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.566651+0000", 2024-09-20T22:09:32.137 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 21.448842801000001, 2024-09-20T22:09:32.137 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9227000670000001, 2024-09-20T22:09:32.137 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.137 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.137 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:09:32.137 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.137 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:09:32.138 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:32.138 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 106968 2024-09-20T22:09:32.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.138 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.138 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.138 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.138 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566651+0000", 2024-09-20T22:09:32.138 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.138 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.138 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.138 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566651+0000", 2024-09-20T22:09:32.138 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.139 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.139 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.139 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566653+0000", 2024-09-20T22:09:32.139 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.674e-06 2024-09-20T22:09:32.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.139 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.139 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.139 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566809+0000", 2024-09-20T22:09:32.139 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015637600000000001 2024-09-20T22:09:32.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.139 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.139 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.139 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566820+0000", 2024-09-20T22:09:32.140 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.097e-05 2024-09-20T22:09:32.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.140 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.140 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.140 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566830+0000", 2024-09-20T22:09:32.140 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4690000000000003e-06 2024-09-20T22:09:32.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.140 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.140 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.140 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.567309+0000", 2024-09-20T22:09:32.140 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00047940599999999998 2024-09-20T22:09:32.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.140 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.140 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.140 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.567321+0000", 2024-09-20T22:09:32.141 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2451e-05 2024-09-20T22:09:32.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.141 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.141 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417364+0000", 2024-09-20T22:09:32.141 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.85004267200000005 2024-09-20T22:09:32.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.141 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.141 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417390+0000", 2024-09-20T22:09:32.141 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.5959999999999999e-05 2024-09-20T22:09:32.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.142 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.142 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489254+0000", 2024-09-20T22:09:32.142 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0718642620000001 2024-09-20T22:09:32.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.142 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.142 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489351+0000", 2024-09-20T22:09:32.142 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6827e-05 2024-09-20T22:09:32.142 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.142 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.142 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.142 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:106993 8.15 8:ae0aa858:::benchmark_data_smithi060_111051_object106992:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:09:32.143 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.600689+0000", 2024-09-20T22:09:32.143 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 21.414805136999998, 2024-09-20T22:09:32.143 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.911249717, 2024-09-20T22:09:32.143 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.143 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.143 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:32.143 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.143 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:09:32.143 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:32.143 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 106993 2024-09-20T22:09:32.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.143 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600689+0000", 2024-09-20T22:09:32.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600689+0000", 2024-09-20T22:09:32.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600691+0000", 2024-09-20T22:09:32.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.052e-06 2024-09-20T22:09:32.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.145 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.145 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600724+0000", 2024-09-20T22:09:32.145 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.3429000000000001e-05 2024-09-20T22:09:32.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.145 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.145 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600747+0000", 2024-09-20T22:09:32.145 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2663e-05 2024-09-20T22:09:32.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.145 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.145 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600756+0000", 2024-09-20T22:09:32.145 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4099999999999997e-06 2024-09-20T22:09:32.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.146 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.146 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600862+0000", 2024-09-20T22:09:32.146 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000106135 2024-09-20T22:09:32.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.146 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.146 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600873+0000", 2024-09-20T22:09:32.146 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0794e-05 2024-09-20T22:09:32.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.146 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.147 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417486+0000", 2024-09-20T22:09:32.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.81661262800000001 2024-09-20T22:09:32.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.147 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.147 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417495+0000", 2024-09-20T22:09:32.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1160000000000005e-06 2024-09-20T22:09:32.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.147 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.147 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489416+0000", 2024-09-20T22:09:32.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.071920784 2024-09-20T22:09:32.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.148 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:32.148 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489581+0000", 2024-09-20T22:09:32.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00016577399999999999 2024-09-20T22:09:32.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.148 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,4,6", 2024-09-20T22:09:32.148 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489742+0000", 2024-09-20T22:09:32.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000160204 2024-09-20T22:09:32.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.148 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:32.148 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.494442+0000", 2024-09-20T22:09:32.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0047004660000000004 2024-09-20T22:09:32.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.149 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.149 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.495057+0000", 2024-09-20T22:09:32.149 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000614677 2024-09-20T22:09:32.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.149 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.149 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.495091+0000", 2024-09-20T22:09:32.149 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4581e-05 2024-09-20T22:09:32.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.149 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.149 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.511743+0000", 2024-09-20T22:09:32.150 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.016651142000000001 2024-09-20T22:09:32.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.150 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:32.150 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.511810+0000", 2024-09-20T22:09:32.150 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7065999999999998e-05 2024-09-20T22:09:32.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.150 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.150 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.511938+0000", 2024-09-20T22:09:32.150 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000128796 2024-09-20T22:09:32.150 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.150 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.151 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.151 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:107013 8.15 8:ab48afac:::benchmark_data_smithi060_111051_object107012:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:09:32.151 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.648456+0000", 2024-09-20T22:09:32.151 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 21.367037435, 2024-09-20T22:09:32.151 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8646870719999999, 2024-09-20T22:09:32.151 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:09:32.151 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:09:32.151 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:32.151 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:09:32.151 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:09:32.151 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:32.151 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 107013 2024-09-20T22:09:32.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.152 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:09:32.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.152 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:09:32.152 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648456+0000", 2024-09-20T22:09:32.152 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.152 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:09:32.152 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648456+0000", 2024-09-20T22:09:32.152 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:09:32.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.152 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:09:32.152 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648459+0000", 2024-09-20T22:09:32.153 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2000000000000001e-06 2024-09-20T22:09:32.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.153 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.153 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:09:32.153 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648501+0000", 2024-09-20T22:09:32.153 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2453000000000002e-05 2024-09-20T22:09:32.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.153 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.153 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:09:32.153 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648525+0000", 2024-09-20T22:09:32.153 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.4034000000000001e-05 2024-09-20T22:09:32.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.153 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.153 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:09:32.154 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648535+0000", 2024-09-20T22:09:32.154 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0355e-05 2024-09-20T22:09:32.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.154 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.154 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.154 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648641+0000", 2024-09-20T22:09:32.154 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000105448 2024-09-20T22:09:32.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.154 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.154 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.154 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648654+0000", 2024-09-20T22:09:32.154 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2904999999999999e-05 2024-09-20T22:09:32.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.154 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.155 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.155 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417525+0000", 2024-09-20T22:09:32.155 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.76887095999999999 2024-09-20T22:09:32.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.155 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.155 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:09:32.155 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417959+0000", 2024-09-20T22:09:32.155 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00043430399999999999 2024-09-20T22:09:32.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.155 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.155 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:09:32.155 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.492147+0000", 2024-09-20T22:09:32.155 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0741883080000001 2024-09-20T22:09:32.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.156 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.156 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:09:32.156 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.492291+0000", 2024-09-20T22:09:32.156 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014341700000000001 2024-09-20T22:09:32.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.156 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.156 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,4,6", 2024-09-20T22:09:32.156 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.492463+0000", 2024-09-20T22:09:32.156 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017243400000000001 2024-09-20T22:09:32.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.156 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.156 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.156 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.499029+0000", 2024-09-20T22:09:32.156 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0065655460000000002 2024-09-20T22:09:32.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.157 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.157 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.157 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.499078+0000", 2024-09-20T22:09:32.157 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9527000000000002e-05 2024-09-20T22:09:32.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.157 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.157 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:09:32.157 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.499106+0000", 2024-09-20T22:09:32.157 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7280999999999999e-05 2024-09-20T22:09:32.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.157 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.157 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:32.157 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.512989+0000", 2024-09-20T22:09:32.158 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.013883929 2024-09-20T22:09:32.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.158 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.158 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:09:32.158 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.513048+0000", 2024-09-20T22:09:32.158 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8261999999999999e-05 2024-09-20T22:09:32.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:09:32.158 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:09:32.158 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:09:32.158 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.513143+0000", 2024-09-20T22:09:32.158 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5709000000000001e-05 2024-09-20T22:09:32.158 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.158 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.159 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.159 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:09:32.159 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:09:32.159 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:09:32.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:32.160 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:32.160 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:32.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:32.303 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:32.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:32.304 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:32.405 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:32.420 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:09:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:09:32.431 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:09:32.432 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:32 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/289416619' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:09:32.433 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:32 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/289416619' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:09:32.452 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:32.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:32.453 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:32.554 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:32.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:32.602 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:32.602 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:32.703 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:32.745 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:32.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:32.746 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:32.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:32 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/289416619' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:09:32.846 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:32.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:32.890 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:32.890 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:32.991 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:33.023 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:33.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:33.024 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:33.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:33.146 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T22:09:33.166 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:33.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:33.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:33.268 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:33.299 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:33.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:33.300 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:33.400 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:33.448 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:09:33.449 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:09:33.469 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:33.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:33.471 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:33.500 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:09:33.500 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:09:33.501 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:33 smithi060 ceph-mon[29694]: pgmap v341: 168 pgs: 6 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 158 active+clean; 7.1 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.7 KiB/s rd, 46 MiB/s wr, 1.24k op/s; 49458/350424 objects misplaced (14.114%); 70 MiB/s, 1.13k objects/s recovering 2024-09-20T22:09:33.501 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:33 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:09:33.501 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:33 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:09:33.501 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:33 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/289416619' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:09:33.501 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:33 smithi060 ceph-mon[29694]: osdmap e226: 8 total, 8 up, 8 in 2024-09-20T22:09:33.501 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:33 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1195699609' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:09:33.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:33 smithi060 ceph-mon[40290]: pgmap v341: 168 pgs: 6 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 158 active+clean; 7.1 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.7 KiB/s rd, 46 MiB/s wr, 1.24k op/s; 49458/350424 objects misplaced (14.114%); 70 MiB/s, 1.13k objects/s recovering 2024-09-20T22:09:33.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:33 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:09:33.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:33 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:09:33.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:33 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/289416619' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:09:33.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:33 smithi060 ceph-mon[40290]: osdmap e226: 8 total, 8 up, 8 in 2024-09-20T22:09:33.502 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:33 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1195699609' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:09:33.540 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:09:33.541 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:09:33.572 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:33.579 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:09:33.579 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:09:33.622 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:33.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:33.623 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:33.631 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:09:33.631 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:09:33.682 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:09:33.682 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:09:33.722 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:09:33.722 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:09:33.724 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:33.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:33 smithi067 ceph-mon[33511]: pgmap v341: 168 pgs: 6 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 158 active+clean; 7.1 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.7 KiB/s rd, 46 MiB/s wr, 1.24k op/s; 49458/350424 objects misplaced (14.114%); 70 MiB/s, 1.13k objects/s recovering 2024-09-20T22:09:33.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:33 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:09:33.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:33 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:09:33.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:33 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/289416619' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:09:33.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:33 smithi067 ceph-mon[33511]: osdmap e226: 8 total, 8 up, 8 in 2024-09-20T22:09:33.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:33 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1195699609' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:09:33.771 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:09:33.771 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:09:33.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:33.775 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:33.775 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:33.876 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:33.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:33.924 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:33.925 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:34.025 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:34.055 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:34.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:34.056 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:34.146 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:09:34.148 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:09:34.157 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:34.188 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:34.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:34.189 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:34.290 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:34.321 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:34.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:34.322 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:34.423 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:34.454 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:34.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:34.455 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:34.556 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:34.591 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:34.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:34.592 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:34.693 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:34.726 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:34.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:34.727 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:34.827 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:34.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:34.860 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:34.860 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:34.961 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:34.994 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:34.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:34.995 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:35.096 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:35.147 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-09-20T22:09:32.434823+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7642525775,"num_objects":116808,"num_object_clones":0,"num_object_copies":350424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":49458,"num_objects_unfound":0,"num_objects_dirty":116808,"num_whiteouts":0,"num_read":1195,"num_read_kb":990,"num_write":110588,"num_write_kb":6829015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19073,"num_bytes_recovered":1247039249,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134683,"ondisk_log_size":134683,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":524,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":26471344,"kb_used_data":23967336,"kb_used_omap":15,"kb_used_meta":2503792,"kb_avail":723489872,"statfs":{"total":767960285184,"available":740853628928,"internally_reserved":0,"allocated":24542552064,"data_stored":24534731281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":2563883795},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,7,6,8,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1048552,"num_objects":-15,"num_object_clones":0,"num_object_copies":-45,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-8855,"num_objects_unfound":0,"num_objects_dirty":-15,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":6751,"num_write_kb":215489,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5838,"num_bytes_recovered":382599168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.382053"},"pg_stats":[{"pgid":"8.14","version":"225'7044","reported_seq":10326,"reported_epoch":225,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T22:09:30.655339+0000","last_change":"2024-09-20T22:09:12.497001+0000","last_active":"2024-09-20T22:09:30.655339+0000","last_peered":"2024-09-20T22:09:30.655339+0000","last_clean":"2024-09-20T22:09:10.504152+0000","last_became_active":"2024-09-20T22:09:12.491220+0000","last_became_peered":"2024-09-20T22:09:12.491220+0000","last_unstale":"2024-09-20T22:09:30.655339+0000","last_undegraded":"2024-09-20T22:09:30.655339+0000","last_fullsized":"2024-09-20T22:09:30.655339+0000","mapping_epoch":218,"log_start":"183'1671","ondisk_log_start":"183'1671","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":5373,"log_dups_size":0,"ondisk_log_size":5373,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233209856,"num_objects":3559,"num_object_clones":0,"num_object_copies":10677,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1951,"num_objects_unfound":0,"num_objects_dirty":3559,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3799,"num_write_kb":235424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1581,"num_bytes_recovered":103612416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,2,5],"avail_no_missing":["4","2","5"],"object_location_counts":[{"shards":"2,4,5","objects":3559}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":351,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.484207+0000","last_change":"2024-09-20T22:06:48.018571+0000","last_active":"2024-09-20T22:09:27.484207+0000","last_peered":"2024-09-20T22:09:27.484207+0000","last_clean":"2024-09-20T22:09:27.484207+0000","last_became_active":"2024-09-20T22:02:50.109930+0000","last_became_peered":"2024-09-20T22:02:50.109930+0000","last_unstale":"2024-09-20T22:09:27.484207+0000","last_undegraded":"2024-09-20T22:09:27.484207+0000","last_fullsized":"2024-09-20T22:09:27.484207+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_clean_scrub_stamp":"2024-09-20T22:06:48.018157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"71'9","reported_seq":372,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.472814+0000","last_change":"2024-09-20T22:07:55.492953+0000","last_active":"2024-09-20T22:09:27.472814+0000","last_peered":"2024-09-20T22:09:27.472814+0000","last_clean":"2024-09-20T22:09:27.472814+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:09:27.472814+0000","last_undegraded":"2024-09-20T22:09:27.472814+0000","last_fullsized":"2024-09-20T22:09:27.472814+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:55.492841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.492841+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":231,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.486628+0000","last_change":"2024-09-20T22:06:54.843328+0000","last_active":"2024-09-20T22:09:27.486628+0000","last_peered":"2024-09-20T22:09:27.486628+0000","last_clean":"2024-09-20T22:09:27.486628+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:09:27.486628+0000","last_undegraded":"2024-09-20T22:09:27.486628+0000","last_fullsized":"2024-09-20T22:09:27.486628+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:11.178591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:11.178591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":340,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.471892+0000","last_change":"2024-09-20T22:07:14.535725+0000","last_active":"2024-09-20T22:09:27.471892+0000","last_peered":"2024-09-20T22:09:27.471892+0000","last_clean":"2024-09-20T22:09:27.471892+0000","last_became_active":"2024-09-20T22:06:54.830769+0000","last_became_peered":"2024-09-20T22:06:54.830769+0000","last_unstale":"2024-09-20T22:09:27.471892+0000","last_undegraded":"2024-09-20T22:09:27.471892+0000","last_fullsized":"2024-09-20T22:09:27.471892+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.535618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.535618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"225'7300","reported_seq":7422,"reported_epoch":225,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T22:09:30.563460+0000","last_change":"2024-09-20T22:09:12.494773+0000","last_active":"2024-09-20T22:09:30.563460+0000","last_peered":"2024-09-20T22:09:30.563460+0000","last_clean":"2024-09-20T22:09:10.532398+0000","last_became_active":"2024-09-20T22:09:12.486597+0000","last_became_peered":"2024-09-20T22:09:12.486597+0000","last_unstale":"2024-09-20T22:09:30.563460+0000","last_undegraded":"2024-09-20T22:09:30.563460+0000","last_fullsized":"2024-09-20T22:09:30.563460+0000","mapping_epoch":218,"log_start":"182'1400","ondisk_log_start":"182'1400","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":5900,"log_dups_size":0,"ondisk_log_size":5900,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243040256,"num_objects":3709,"num_object_clones":0,"num_object_copies":11127,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3709,"num_objects_unfound":0,"num_objects_dirty":3709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3943,"num_write_kb":244832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,6,4],"avail_no_missing":["7","4","6"],"object_location_counts":[{"shards":"4,6,7","objects":3709}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":339,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.473232+0000","last_change":"2024-09-20T22:08:41.770593+0000","last_active":"2024-09-20T22:09:27.473232+0000","last_peered":"2024-09-20T22:09:27.473232+0000","last_clean":"2024-09-20T22:09:27.473232+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:09:27.473232+0000","last_undegraded":"2024-09-20T22:09:27.473232+0000","last_fullsized":"2024-09-20T22:09:27.473232+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:08:41.770351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:41.770351+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:07.625199+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":351,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.474905+0000","last_change":"2024-09-20T22:09:00.839346+0000","last_active":"2024-09-20T22:09:27.474905+0000","last_peered":"2024-09-20T22:09:27.474905+0000","last_clean":"2024-09-20T22:09:27.474905+0000","last_became_active":"2024-09-20T22:02:50.107154+0000","last_became_peered":"2024-09-20T22:02:50.107154+0000","last_unstale":"2024-09-20T22:09:27.474905+0000","last_undegraded":"2024-09-20T22:09:27.474905+0000","last_fullsized":"2024-09-20T22:09:27.474905+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:00.839239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:00.839239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:07.151739+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"71'15","reported_seq":371,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.473358+0000","last_change":"2024-09-20T22:08:39.705641+0000","last_active":"2024-09-20T22:09:27.473358+0000","last_peered":"2024-09-20T22:09:27.473358+0000","last_clean":"2024-09-20T22:09:27.473358+0000","last_became_active":"2024-09-20T22:02:52.282101+0000","last_became_peered":"2024-09-20T22:02:52.282101+0000","last_unstale":"2024-09-20T22:09:27.473358+0000","last_undegraded":"2024-09-20T22:09:27.473358+0000","last_fullsized":"2024-09-20T22:09:27.473358+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:08:39.705510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:39.705510+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:08.518121+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":344,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.472677+0000","last_change":"2024-09-20T22:07:17.494965+0000","last_active":"2024-09-20T22:09:27.472677+0000","last_peered":"2024-09-20T22:09:27.472677+0000","last_clean":"2024-09-20T22:09:27.472677+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:09:27.472677+0000","last_undegraded":"2024-09-20T22:09:27.472677+0000","last_fullsized":"2024-09-20T22:09:27.472677+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:17.494852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:17.494852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"225'7165","reported_seq":7287,"reported_epoch":225,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T22:09:31.484987+0000","last_change":"2024-09-20T22:09:12.533136+0000","last_active":"2024-09-20T22:09:31.484987+0000","last_peered":"2024-09-20T22:09:31.484987+0000","last_clean":"2024-09-20T22:09:10.566074+0000","last_became_active":"2024-09-20T22:09:12.508336+0000","last_became_peered":"2024-09-20T22:09:12.508336+0000","last_unstale":"2024-09-20T22:09:31.484987+0000","last_undegraded":"2024-09-20T22:09:31.484987+0000","last_fullsized":"2024-09-20T22:09:31.484987+0000","mapping_epoch":218,"log_start":"183'1626","ondisk_log_start":"183'1626","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":5539,"log_dups_size":0,"ondisk_log_size":5539,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234389504,"num_objects":3577,"num_object_clones":0,"num_object_copies":10731,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10731,"num_objects_unfound":0,"num_objects_dirty":3577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3885,"num_write_kb":238752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[0,6,7],"avail_no_missing":["0","6","7"],"object_location_counts":[{"shards":"0,6,7","objects":3577}],"blocked_by":[],"up_primary":3,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":366,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477356+0000","last_change":"2024-09-20T22:08:35.146849+0000","last_active":"2024-09-20T22:09:27.477356+0000","last_peered":"2024-09-20T22:09:27.477356+0000","last_clean":"2024-09-20T22:09:27.477356+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:09:27.477356+0000","last_undegraded":"2024-09-20T22:09:27.477356+0000","last_fullsized":"2024-09-20T22:09:27.477356+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:08:35.146554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:50.212690+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"71'9","reported_seq":363,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.472162+0000","last_change":"2024-09-20T22:06:09.729638+0000","last_active":"2024-09-20T22:09:27.472162+0000","last_peered":"2024-09-20T22:09:27.472162+0000","last_clean":"2024-09-20T22:09:27.472162+0000","last_became_active":"2024-09-20T22:02:52.279526+0000","last_became_peered":"2024-09-20T22:02:52.279526+0000","last_unstale":"2024-09-20T22:09:27.472162+0000","last_undegraded":"2024-09-20T22:09:27.472162+0000","last_fullsized":"2024-09-20T22:09:27.472162+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:06:09.729513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:09.729513+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":364,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.478269+0000","last_change":"2024-09-20T22:08:51.456300+0000","last_active":"2024-09-20T22:09:27.478269+0000","last_peered":"2024-09-20T22:09:27.478269+0000","last_clean":"2024-09-20T22:09:27.478269+0000","last_became_active":"2024-09-20T22:08:51.455924+0000","last_became_peered":"2024-09-20T22:08:51.455924+0000","last_unstale":"2024-09-20T22:09:27.478269+0000","last_undegraded":"2024-09-20T22:09:27.478269+0000","last_fullsized":"2024-09-20T22:09:27.478269+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:42.143066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:42.143066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.990159+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":339,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.481613+0000","last_change":"2024-09-20T22:07:15.416312+0000","last_active":"2024-09-20T22:09:27.481613+0000","last_peered":"2024-09-20T22:09:27.481613+0000","last_clean":"2024-09-20T22:09:27.481613+0000","last_became_active":"2024-09-20T22:06:54.841095+0000","last_became_peered":"2024-09-20T22:06:54.841095+0000","last_unstale":"2024-09-20T22:09:27.481613+0000","last_undegraded":"2024-09-20T22:09:27.481613+0000","last_fullsized":"2024-09-20T22:09:27.481613+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:15.416176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.416176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"225'7080","reported_seq":7202,"reported_epoch":225,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T22:09:27.783503+0000","last_change":"2024-09-20T22:09:12.526408+0000","last_active":"2024-09-20T22:09:27.783503+0000","last_peered":"2024-09-20T22:09:27.783503+0000","last_clean":"2024-09-20T22:09:10.546593+0000","last_became_active":"2024-09-20T22:09:12.517484+0000","last_became_peered":"2024-09-20T22:09:12.517484+0000","last_unstale":"2024-09-20T22:09:27.783503+0000","last_undegraded":"2024-09-20T22:09:27.783503+0000","last_fullsized":"2024-09-20T22:09:27.783503+0000","mapping_epoch":218,"log_start":"185'2256","ondisk_log_start":"185'2256","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4824,"log_dups_size":0,"ondisk_log_size":4824,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243957760,"num_objects":3723,"num_object_clones":0,"num_object_copies":11169,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7446,"num_objects_unfound":0,"num_objects_dirty":3723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3775,"num_write_kb":239904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[1,6,3],"avail_no_missing":["1","3","6"],"object_location_counts":[{"shards":"1,3,6","objects":3723}],"blocked_by":[],"up_primary":7,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":371,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.478268+0000","last_change":"2024-09-20T22:07:59.947914+0000","last_active":"2024-09-20T22:09:27.478268+0000","last_peered":"2024-09-20T22:09:27.478268+0000","last_clean":"2024-09-20T22:09:27.478268+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:09:27.478268+0000","last_undegraded":"2024-09-20T22:09:27.478268+0000","last_fullsized":"2024-09-20T22:09:27.478268+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:59.947741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:59.947741+0000","last_clean_scrub_stamp":"2024-09-20T22:07:59.947741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:29.533391+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"71'5","reported_seq":378,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.472157+0000","last_change":"2024-09-20T22:08:22.151586+0000","last_active":"2024-09-20T22:09:27.472157+0000","last_peered":"2024-09-20T22:09:27.472157+0000","last_clean":"2024-09-20T22:09:27.472157+0000","last_became_active":"2024-09-20T22:08:22.144364+0000","last_became_peered":"2024-09-20T22:08:22.144364+0000","last_unstale":"2024-09-20T22:09:27.472157+0000","last_undegraded":"2024-09-20T22:09:27.472157+0000","last_fullsized":"2024-09-20T22:09:27.472157+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'5","last_scrub_stamp":"2024-09-20T22:08:01.462581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:01.462581+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":352,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.489599+0000","last_change":"2024-09-20T22:07:28.654301+0000","last_active":"2024-09-20T22:09:27.489599+0000","last_peered":"2024-09-20T22:09:27.489599+0000","last_clean":"2024-09-20T22:09:27.489599+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:09:27.489599+0000","last_undegraded":"2024-09-20T22:09:27.489599+0000","last_fullsized":"2024-09-20T22:09:27.489599+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:28.654202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:28.654202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":293,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.484685+0000","last_change":"2024-09-20T22:08:41.836934+0000","last_active":"2024-09-20T22:09:27.484685+0000","last_peered":"2024-09-20T22:09:27.484685+0000","last_clean":"2024-09-20T22:09:27.484685+0000","last_became_active":"2024-09-20T22:06:55.824168+0000","last_became_peered":"2024-09-20T22:06:55.824168+0000","last_unstale":"2024-09-20T22:09:27.484685+0000","last_undegraded":"2024-09-20T22:09:27.484685+0000","last_fullsized":"2024-09-20T22:09:27.484685+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:41.836827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:08:41.836827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:04.667178+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.10","version":"225'7095","reported_seq":12303,"reported_epoch":225,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T22:09:27.797519+0000","last_change":"2024-09-20T22:09:12.531190+0000","last_active":"2024-09-20T22:09:27.797519+0000","last_peered":"2024-09-20T22:09:27.797519+0000","last_clean":"2024-09-20T22:09:10.486900+0000","last_became_active":"2024-09-20T22:09:12.525967+0000","last_became_peered":"2024-09-20T22:09:12.525967+0000","last_unstale":"2024-09-20T22:09:27.797519+0000","last_undegraded":"2024-09-20T22:09:27.797519+0000","last_fullsized":"2024-09-20T22:09:27.797519+0000","mapping_epoch":218,"log_start":"184'1794","ondisk_log_start":"184'1794","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":5301,"log_dups_size":0,"ondisk_log_size":5301,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":243793920,"num_objects":3720,"num_object_clones":0,"num_object_copies":11160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3934,"num_objects_unfound":0,"num_objects_dirty":3720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3776,"num_write_kb":239872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1696,"num_bytes_recovered":111149056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[1,5,3],"avail_no_missing":["1","3","5"],"object_location_counts":[{"shards":"1,3,5","objects":3720}],"blocked_by":[],"up_primary":7,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":340,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.473999+0000","last_change":"2024-09-20T22:06:47.667293+0000","last_active":"2024-09-20T22:09:27.473999+0000","last_peered":"2024-09-20T22:09:27.473999+0000","last_clean":"2024-09-20T22:09:27.473999+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:09:27.473999+0000","last_undegraded":"2024-09-20T22:09:27.473999+0000","last_fullsized":"2024-09-20T22:09:27.473999+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:06:47.667180+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":350,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.483405+0000","last_change":"2024-09-20T22:08:34.815041+0000","last_active":"2024-09-20T22:09:27.483405+0000","last_peered":"2024-09-20T22:09:27.483405+0000","last_clean":"2024-09-20T22:09:27.483405+0000","last_became_active":"2024-09-20T22:02:50.111517+0000","last_became_peered":"2024-09-20T22:02:50.111517+0000","last_unstale":"2024-09-20T22:09:27.483405+0000","last_undegraded":"2024-09-20T22:09:27.483405+0000","last_fullsized":"2024-09-20T22:09:27.483405+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:08:34.814930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:42.718933+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"195'22","reported_seq":394,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.482304+0000","last_change":"2024-09-20T22:08:39.358759+0000","last_active":"2024-09-20T22:09:27.482304+0000","last_peered":"2024-09-20T22:09:27.482304+0000","last_clean":"2024-09-20T22:09:27.482304+0000","last_became_active":"2024-09-20T22:06:54.846111+0000","last_became_peered":"2024-09-20T22:06:54.846111+0000","last_unstale":"2024-09-20T22:09:27.482304+0000","last_undegraded":"2024-09-20T22:09:27.482304+0000","last_fullsized":"2024-09-20T22:09:27.482304+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'22","last_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:08:39.355859+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:53.211286+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":189,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.487821+0000","last_change":"2024-09-20T22:08:00.369883+0000","last_active":"2024-09-20T22:09:27.487821+0000","last_peered":"2024-09-20T22:09:27.487821+0000","last_clean":"2024-09-20T22:09:27.487821+0000","last_became_active":"2024-09-20T22:06:54.848505+0000","last_became_peered":"2024-09-20T22:06:54.848505+0000","last_unstale":"2024-09-20T22:09:27.487821+0000","last_undegraded":"2024-09-20T22:09:27.487821+0000","last_fullsized":"2024-09-20T22:09:27.487821+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:00.369767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:00.369767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"225'7173","reported_seq":7292,"reported_epoch":225,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T22:09:31.509149+0000","last_change":"2024-09-20T22:09:12.551370+0000","last_active":"2024-09-20T22:09:31.509149+0000","last_peered":"2024-09-20T22:09:31.509149+0000","last_clean":"2024-09-20T22:09:10.466832+0000","last_became_active":"2024-09-20T22:09:12.526523+0000","last_became_peered":"2024-09-20T22:09:12.526523+0000","last_unstale":"2024-09-20T22:09:31.509149+0000","last_undegraded":"2024-09-20T22:09:31.509149+0000","last_fullsized":"2024-09-20T22:09:31.509149+0000","mapping_epoch":218,"log_start":"185'2185","ondisk_log_start":"185'2185","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4988,"log_dups_size":0,"ondisk_log_size":4988,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":235044864,"num_objects":3587,"num_object_clones":0,"num_object_copies":10761,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10761,"num_objects_unfound":0,"num_objects_dirty":3587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3901,"num_write_kb":239584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[0,1,2],"avail_no_missing":["0","1","2"],"object_location_counts":[{"shards":"0,1,2","objects":3587}],"blocked_by":[],"up_primary":5,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":339,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.475403+0000","last_change":"2024-09-20T22:06:34.944093+0000","last_active":"2024-09-20T22:09:27.475403+0000","last_peered":"2024-09-20T22:09:27.475403+0000","last_clean":"2024-09-20T22:09:27.475403+0000","last_became_active":"2024-09-20T22:02:56.347561+0000","last_became_peered":"2024-09-20T22:02:56.347561+0000","last_unstale":"2024-09-20T22:09:27.475403+0000","last_undegraded":"2024-09-20T22:09:27.475403+0000","last_fullsized":"2024-09-20T22:09:27.475403+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:34.940758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:34.940758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":301,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477898+0000","last_change":"2024-09-20T22:08:28.058586+0000","last_active":"2024-09-20T22:09:27.477898+0000","last_peered":"2024-09-20T22:09:27.477898+0000","last_clean":"2024-09-20T22:09:27.477898+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:09:27.477898+0000","last_undegraded":"2024-09-20T22:09:27.477898+0000","last_fullsized":"2024-09-20T22:09:27.477898+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:28.058516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:28.058516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:50.545327+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"71'12","reported_seq":370,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.478731+0000","last_change":"2024-09-20T22:07:55.383903+0000","last_active":"2024-09-20T22:09:27.478731+0000","last_peered":"2024-09-20T22:09:27.478731+0000","last_clean":"2024-09-20T22:09:27.478731+0000","last_became_active":"2024-09-20T22:06:54.830946+0000","last_became_peered":"2024-09-20T22:06:54.830946+0000","last_unstale":"2024-09-20T22:09:27.478731+0000","last_undegraded":"2024-09-20T22:09:27.478731+0000","last_fullsized":"2024-09-20T22:09:27.478731+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:55.383818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.383818+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":343,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.471898+0000","last_change":"2024-09-20T22:08:36.461149+0000","last_active":"2024-09-20T22:09:27.471898+0000","last_peered":"2024-09-20T22:09:27.471898+0000","last_clean":"2024-09-20T22:09:27.471898+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:09:27.471898+0000","last_undegraded":"2024-09-20T22:09:27.471898+0000","last_fullsized":"2024-09-20T22:09:27.471898+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:36.460897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:36.460897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:51.153875+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.12","version":"225'7115","reported_seq":14284,"reported_epoch":225,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T22:09:31.521349+0000","last_change":"2024-09-20T22:09:12.529968+0000","last_active":"2024-09-20T22:09:31.521349+0000","last_peered":"2024-09-20T22:09:31.521349+0000","last_clean":"2024-09-20T22:09:10.505850+0000","last_became_active":"2024-09-20T22:09:12.505502+0000","last_became_peered":"2024-09-20T22:09:12.505502+0000","last_unstale":"2024-09-20T22:09:31.521349+0000","last_undegraded":"2024-09-20T22:09:31.521349+0000","last_fullsized":"2024-09-20T22:09:31.521349+0000","mapping_epoch":218,"log_start":"185'2032","ondisk_log_start":"185'2032","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":5083,"log_dups_size":0,"ondisk_log_size":5083,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233373696,"num_objects":3561,"num_object_clones":0,"num_object_copies":10683,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":5391,"num_objects_unfound":0,"num_objects_dirty":3561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3839,"num_write_kb":236800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1762,"num_bytes_recovered":115474432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[0,4,5],"avail_no_missing":["0","4","5"],"object_location_counts":[{"shards":"0,4,5","objects":3561}],"blocked_by":[],"up_primary":6,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":341,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.489844+0000","last_change":"2024-09-20T22:08:34.802125+0000","last_active":"2024-09-20T22:09:27.489844+0000","last_peered":"2024-09-20T22:09:27.489844+0000","last_clean":"2024-09-20T22:09:27.489844+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:09:27.489844+0000","last_undegraded":"2024-09-20T22:09:27.489844+0000","last_fullsized":"2024-09-20T22:09:27.489844+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:08:34.801894+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:08:34.801894+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:01.262234+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"64'1","reported_seq":190,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.486705+0000","last_change":"2024-09-20T22:09:02.517233+0000","last_active":"2024-09-20T22:09:27.486705+0000","last_peered":"2024-09-20T22:09:27.486705+0000","last_clean":"2024-09-20T22:09:27.486705+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:09:27.486705+0000","last_undegraded":"2024-09-20T22:09:27.486705+0000","last_fullsized":"2024-09-20T22:09:27.486705+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:09:02.516942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:02.516942+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:20.902402+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"71'10","reported_seq":379,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.484667+0000","last_change":"2024-09-20T22:08:52.434346+0000","last_active":"2024-09-20T22:09:27.484667+0000","last_peered":"2024-09-20T22:09:27.484667+0000","last_clean":"2024-09-20T22:09:27.484667+0000","last_became_active":"2024-09-20T22:08:52.424697+0000","last_became_peered":"2024-09-20T22:08:52.424697+0000","last_unstale":"2024-09-20T22:09:27.484667+0000","last_undegraded":"2024-09-20T22:09:27.484667+0000","last_fullsized":"2024-09-20T22:09:27.484667+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:08:30.868781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:30.868781+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:40.662035+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"208'9","reported_seq":453,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.478882+0000","last_change":"2024-09-20T22:08:51.459956+0000","last_active":"2024-09-20T22:09:27.478882+0000","last_peered":"2024-09-20T22:09:27.478882+0000","last_clean":"2024-09-20T22:09:27.478882+0000","last_became_active":"2024-09-20T22:08:51.448751+0000","last_became_peered":"2024-09-20T22:08:51.448751+0000","last_unstale":"2024-09-20T22:09:27.478882+0000","last_undegraded":"2024-09-20T22:09:27.478882+0000","last_fullsized":"2024-09-20T22:09:27.478882+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:08:29.376948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:29.376948+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:41.132625+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":346,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477671+0000","last_change":"2024-09-20T22:08:48.453361+0000","last_active":"2024-09-20T22:09:27.477671+0000","last_peered":"2024-09-20T22:09:27.477671+0000","last_clean":"2024-09-20T22:09:27.477671+0000","last_became_active":"2024-09-20T22:08:48.452786+0000","last_became_peered":"2024-09-20T22:08:48.452786+0000","last_unstale":"2024-09-20T22:09:27.477671+0000","last_undegraded":"2024-09-20T22:09:27.477671+0000","last_fullsized":"2024-09-20T22:09:27.477671+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:07:53.673691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.673691+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:32.262245+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53669,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":26,"num_write":16,"num_write_kb":219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":161009,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3","version":"225'7076","reported_seq":7192,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.664449+0000","last_change":"2024-09-20T22:09:09.569263+0000","last_active":"2024-09-20T22:09:27.664449+0000","last_peered":"2024-09-20T22:09:27.664449+0000","last_clean":"2024-09-20T22:09:27.664449+0000","last_became_active":"2024-09-20T22:09:09.566687+0000","last_became_peered":"2024-09-20T22:09:09.566687+0000","last_unstale":"2024-09-20T22:09:27.664449+0000","last_undegraded":"2024-09-20T22:09:27.664449+0000","last_fullsized":"2024-09-20T22:09:27.664449+0000","mapping_epoch":176,"log_start":"185'2186","ondisk_log_start":"185'2186","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4890,"log_dups_size":0,"ondisk_log_size":4890,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":244449280,"num_objects":3730,"num_object_clones":0,"num_object_copies":11190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3770,"num_write_kb":240000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"71'15","reported_seq":109,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.481261+0000","last_change":"2024-09-20T22:08:30.118912+0000","last_active":"2024-09-20T22:09:27.481261+0000","last_peered":"2024-09-20T22:09:27.481261+0000","last_clean":"2024-09-20T22:09:27.481261+0000","last_became_active":"2024-09-20T22:08:22.145408+0000","last_became_peered":"2024-09-20T22:08:22.145408+0000","last_unstale":"2024-09-20T22:09:27.481261+0000","last_undegraded":"2024-09-20T22:09:27.481261+0000","last_fullsized":"2024-09-20T22:09:27.481261+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:08:30.118638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:30.118638+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:30.162802+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":352,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.478270+0000","last_change":"2024-09-20T22:07:23.292560+0000","last_active":"2024-09-20T22:09:27.478270+0000","last_peered":"2024-09-20T22:09:27.478270+0000","last_clean":"2024-09-20T22:09:27.478270+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:09:27.478270+0000","last_undegraded":"2024-09-20T22:09:27.478270+0000","last_fullsized":"2024-09-20T22:09:27.478270+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.292451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.292451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"71'8","reported_seq":443,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.474067+0000","last_change":"2024-09-20T22:09:06.534110+0000","last_active":"2024-09-20T22:09:27.474067+0000","last_peered":"2024-09-20T22:09:27.474067+0000","last_clean":"2024-09-20T22:09:27.474067+0000","last_became_active":"2024-09-20T22:02:54.294917+0000","last_became_peered":"2024-09-20T22:02:54.294917+0000","last_unstale":"2024-09-20T22:09:27.474067+0000","last_undegraded":"2024-09-20T22:09:27.474067+0000","last_fullsized":"2024-09-20T22:09:27.474067+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:09:06.534030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:06.534030+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:08.011991+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":301,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.478511+0000","last_change":"2024-09-20T22:07:52.334831+0000","last_active":"2024-09-20T22:09:27.478511+0000","last_peered":"2024-09-20T22:09:27.478511+0000","last_clean":"2024-09-20T22:09:27.478511+0000","last_became_active":"2024-09-20T22:06:55.822780+0000","last_became_peered":"2024-09-20T22:06:55.822780+0000","last_unstale":"2024-09-20T22:09:27.478511+0000","last_undegraded":"2024-09-20T22:09:27.478511+0000","last_fullsized":"2024-09-20T22:09:27.478511+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:52.334748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:52.334748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":354,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.472907+0000","last_change":"2024-09-20T22:08:59.563006+0000","last_active":"2024-09-20T22:09:27.472907+0000","last_peered":"2024-09-20T22:09:27.472907+0000","last_clean":"2024-09-20T22:09:27.472907+0000","last_became_active":"2024-09-20T22:08:48.452475+0000","last_became_peered":"2024-09-20T22:08:48.452475+0000","last_unstale":"2024-09-20T22:09:27.472907+0000","last_undegraded":"2024-09-20T22:09:27.472907+0000","last_fullsized":"2024-09-20T22:09:27.472907+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:08:59.562897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:08:59.562897+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:26.912800+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"225'7789","reported_seq":19989,"reported_epoch":225,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T22:09:31.523350+0000","last_change":"2024-09-20T22:08:53.752102+0000","last_active":"2024-09-20T22:09:31.523350+0000","last_peered":"2024-09-20T22:09:31.523350+0000","last_clean":"2024-09-20T22:08:51.290755+0000","last_became_active":"2024-09-20T22:08:53.741655+0000","last_became_peered":"2024-09-20T22:08:53.741655+0000","last_unstale":"2024-09-20T22:09:31.523350+0000","last_undegraded":"2024-09-20T22:09:31.523350+0000","last_fullsized":"2024-09-20T22:09:31.523350+0000","mapping_epoch":206,"log_start":"188'3300","ondisk_log_start":"188'3300","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4489,"log_dups_size":0,"ondisk_log_size":4489,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":471400448,"num_objects":7193,"num_object_clones":0,"num_object_copies":21579,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":589,"num_objects_unfound":0,"num_objects_dirty":7193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7789,"num_write_kb":479424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6042,"num_bytes_recovered":395968512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,7],"avail_no_missing":["0","3","7"],"object_location_counts":[{"shards":"0,3,7","objects":7193}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"71'18","reported_seq":402,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477809+0000","last_change":"2024-09-20T22:08:52.436122+0000","last_active":"2024-09-20T22:09:27.477809+0000","last_peered":"2024-09-20T22:09:27.477809+0000","last_clean":"2024-09-20T22:09:27.477809+0000","last_became_active":"2024-09-20T22:08:52.421420+0000","last_became_peered":"2024-09-20T22:08:52.421420+0000","last_unstale":"2024-09-20T22:09:27.477809+0000","last_undegraded":"2024-09-20T22:09:27.477809+0000","last_fullsized":"2024-09-20T22:09:27.477809+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'18","last_scrub_stamp":"2024-09-20T22:08:40.737338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:40.737338+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:48.648335+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":350,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.475963+0000","last_change":"2024-09-20T22:09:01.889907+0000","last_active":"2024-09-20T22:09:27.475963+0000","last_peered":"2024-09-20T22:09:27.475963+0000","last_clean":"2024-09-20T22:09:27.475963+0000","last_became_active":"2024-09-20T22:02:50.113076+0000","last_became_peered":"2024-09-20T22:02:50.113076+0000","last_unstale":"2024-09-20T22:09:27.475963+0000","last_undegraded":"2024-09-20T22:09:27.475963+0000","last_fullsized":"2024-09-20T22:09:27.475963+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:01.889795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:01.889795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:27.566891+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":358,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.471554+0000","last_change":"2024-09-20T22:09:05.535483+0000","last_active":"2024-09-20T22:09:27.471554+0000","last_peered":"2024-09-20T22:09:27.471554+0000","last_clean":"2024-09-20T22:09:27.471554+0000","last_became_active":"2024-09-20T22:08:51.455159+0000","last_became_peered":"2024-09-20T22:08:51.455159+0000","last_unstale":"2024-09-20T22:09:27.471554+0000","last_undegraded":"2024-09-20T22:09:27.471554+0000","last_fullsized":"2024-09-20T22:09:27.471554+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:05.535412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:05.535412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:23.611809+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":348,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.472066+0000","last_change":"2024-09-20T22:08:38.488454+0000","last_active":"2024-09-20T22:09:27.472066+0000","last_peered":"2024-09-20T22:09:27.472066+0000","last_clean":"2024-09-20T22:09:27.472066+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:09:27.472066+0000","last_undegraded":"2024-09-20T22:09:27.472066+0000","last_fullsized":"2024-09-20T22:09:27.472066+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:38.488310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:38.488310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.111826+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":354,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.473057+0000","last_change":"2024-09-20T22:08:58.858275+0000","last_active":"2024-09-20T22:09:27.473057+0000","last_peered":"2024-09-20T22:09:27.473057+0000","last_clean":"2024-09-20T22:09:27.473057+0000","last_became_active":"2024-09-20T22:08:48.476212+0000","last_became_peered":"2024-09-20T22:08:48.476212+0000","last_unstale":"2024-09-20T22:09:27.473057+0000","last_undegraded":"2024-09-20T22:09:27.473057+0000","last_fullsized":"2024-09-20T22:09:27.473057+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:08:58.858164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:08:58.858164+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:01.714721+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"225'7685","reported_seq":7797,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:30.685934+0000","last_change":"2024-09-20T22:07:52.940943+0000","last_active":"2024-09-20T22:09:30.685934+0000","last_peered":"2024-09-20T22:09:30.685934+0000","last_clean":"2024-09-20T22:09:30.685934+0000","last_became_active":"2024-09-20T22:07:52.940627+0000","last_became_peered":"2024-09-20T22:07:52.940627+0000","last_unstale":"2024-09-20T22:09:30.685934+0000","last_undegraded":"2024-09-20T22:09:30.685934+0000","last_fullsized":"2024-09-20T22:09:30.685934+0000","mapping_epoch":176,"log_start":"188'3366","ondisk_log_start":"188'3366","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4319,"log_dups_size":0,"ondisk_log_size":4319,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":471859224,"num_objects":7201,"num_object_clones":0,"num_object_copies":21603,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7201,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7685,"num_write_kb":476289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"71'14","reported_seq":377,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.471706+0000","last_change":"2024-09-20T22:09:03.569008+0000","last_active":"2024-09-20T22:09:27.471706+0000","last_peered":"2024-09-20T22:09:27.471706+0000","last_clean":"2024-09-20T22:09:27.471706+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:09:27.471706+0000","last_undegraded":"2024-09-20T22:09:27.471706+0000","last_fullsized":"2024-09-20T22:09:27.471706+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'14","last_scrub_stamp":"2024-09-20T22:09:03.568900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:09:03.568900+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:31.777928+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":353,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.484817+0000","last_change":"2024-09-20T22:07:58.783500+0000","last_active":"2024-09-20T22:09:27.484817+0000","last_peered":"2024-09-20T22:09:27.484817+0000","last_clean":"2024-09-20T22:09:27.484817+0000","last_became_active":"2024-09-20T22:06:54.841165+0000","last_became_peered":"2024-09-20T22:06:54.841165+0000","last_unstale":"2024-09-20T22:09:27.484817+0000","last_undegraded":"2024-09-20T22:09:27.484817+0000","last_fullsized":"2024-09-20T22:09:27.484817+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:07:58.783380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:58.783380+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"146'10","reported_seq":435,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477425+0000","last_change":"2024-09-20T22:07:24.333937+0000","last_active":"2024-09-20T22:09:27.477425+0000","last_peered":"2024-09-20T22:09:27.477425+0000","last_clean":"2024-09-20T22:09:27.477425+0000","last_became_active":"2024-09-20T22:06:54.847742+0000","last_became_peered":"2024-09-20T22:06:54.847742+0000","last_unstale":"2024-09-20T22:09:27.477425+0000","last_undegraded":"2024-09-20T22:09:27.477425+0000","last_fullsized":"2024-09-20T22:09:27.477425+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:07:24.333809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:24.333809+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":340,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.489510+0000","last_change":"2024-09-20T22:07:30.589937+0000","last_active":"2024-09-20T22:09:27.489510+0000","last_peered":"2024-09-20T22:09:27.489510+0000","last_clean":"2024-09-20T22:09:27.489510+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:09:27.489510+0000","last_undegraded":"2024-09-20T22:09:27.489510+0000","last_fullsized":"2024-09-20T22:09:27.489510+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:30.589849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:30.589849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":367,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.478584+0000","last_change":"2024-09-20T22:09:02.350959+0000","last_active":"2024-09-20T22:09:27.478584+0000","last_peered":"2024-09-20T22:09:27.478584+0000","last_clean":"2024-09-20T22:09:27.478584+0000","last_became_active":"2024-09-20T22:08:55.449793+0000","last_became_peered":"2024-09-20T22:08:55.449793+0000","last_unstale":"2024-09-20T22:09:27.478584+0000","last_undegraded":"2024-09-20T22:09:27.478584+0000","last_fullsized":"2024-09-20T22:09:27.478584+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:09:02.350748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:09:02.350748+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:20.165516+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"225'7928","reported_seq":8039,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:31.516500+0000","last_change":"2024-09-20T22:07:52.940978+0000","last_active":"2024-09-20T22:09:31.516500+0000","last_peered":"2024-09-20T22:09:31.516500+0000","last_clean":"2024-09-20T22:09:31.516500+0000","last_became_active":"2024-09-20T22:07:52.940673+0000","last_became_peered":"2024-09-20T22:07:52.940673+0000","last_unstale":"2024-09-20T22:09:31.516500+0000","last_undegraded":"2024-09-20T22:09:31.516500+0000","last_fullsized":"2024-09-20T22:09:31.516500+0000","mapping_epoch":176,"log_start":"188'3421","ondisk_log_start":"188'3421","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4507,"log_dups_size":0,"ondisk_log_size":4507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":479854592,"num_objects":7322,"num_object_clones":0,"num_object_copies":21966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7928,"num_write_kb":488000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"71'10","reported_seq":215,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.486246+0000","last_change":"2024-09-20T22:07:59.424108+0000","last_active":"2024-09-20T22:09:27.486246+0000","last_peered":"2024-09-20T22:09:27.486246+0000","last_clean":"2024-09-20T22:09:27.486246+0000","last_became_active":"2024-09-20T22:06:54.844681+0000","last_became_peered":"2024-09-20T22:06:54.844681+0000","last_unstale":"2024-09-20T22:09:27.486246+0000","last_undegraded":"2024-09-20T22:09:27.486246+0000","last_fullsized":"2024-09-20T22:09:27.486246+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:07:59.423970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:59.423970+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":367,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477415+0000","last_change":"2024-09-20T22:08:29.034890+0000","last_active":"2024-09-20T22:09:27.477415+0000","last_peered":"2024-09-20T22:09:27.477415+0000","last_clean":"2024-09-20T22:09:27.477415+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:09:27.477415+0000","last_undegraded":"2024-09-20T22:09:27.477415+0000","last_fullsized":"2024-09-20T22:09:27.477415+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:29.034641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:08:29.034641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:32.281368+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"71'8","reported_seq":432,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.482801+0000","last_change":"2024-09-20T22:08:28.861124+0000","last_active":"2024-09-20T22:09:27.482801+0000","last_peered":"2024-09-20T22:09:27.482801+0000","last_clean":"2024-09-20T22:09:27.482801+0000","last_became_active":"2024-09-20T22:02:54.292819+0000","last_became_peered":"2024-09-20T22:02:54.292819+0000","last_unstale":"2024-09-20T22:09:27.482801+0000","last_undegraded":"2024-09-20T22:09:27.482801+0000","last_fullsized":"2024-09-20T22:09:27.482801+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:08:28.860982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:28.860982+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:39.280471+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":331,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.481783+0000","last_change":"2024-09-20T22:06:00.572347+0000","last_active":"2024-09-20T22:09:27.481783+0000","last_peered":"2024-09-20T22:09:27.481783+0000","last_clean":"2024-09-20T22:09:27.481783+0000","last_became_active":"2024-09-20T22:02:56.304274+0000","last_became_peered":"2024-09-20T22:02:56.304274+0000","last_unstale":"2024-09-20T22:09:27.481783+0000","last_undegraded":"2024-09-20T22:09:27.481783+0000","last_fullsized":"2024-09-20T22:09:27.481783+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:00.572243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.572243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":359,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.478189+0000","last_change":"2024-09-20T22:09:00.372470+0000","last_active":"2024-09-20T22:09:27.478189+0000","last_peered":"2024-09-20T22:09:27.478189+0000","last_clean":"2024-09-20T22:09:27.478189+0000","last_became_active":"2024-09-20T22:08:48.466328+0000","last_became_peered":"2024-09-20T22:08:48.466328+0000","last_unstale":"2024-09-20T22:09:27.478189+0000","last_undegraded":"2024-09-20T22:09:27.478189+0000","last_fullsized":"2024-09-20T22:09:27.478189+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:09:00.372393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:09:00.372393+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:28.799282+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.f","version":"225'7876","reported_seq":7984,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:31.210795+0000","last_change":"2024-09-20T22:07:52.940027+0000","last_active":"2024-09-20T22:09:31.210795+0000","last_peered":"2024-09-20T22:09:31.210795+0000","last_clean":"2024-09-20T22:09:31.210795+0000","last_became_active":"2024-09-20T22:07:52.939692+0000","last_became_peered":"2024-09-20T22:07:52.939692+0000","last_unstale":"2024-09-20T22:09:31.210795+0000","last_undegraded":"2024-09-20T22:09:31.210795+0000","last_fullsized":"2024-09-20T22:09:31.210795+0000","mapping_epoch":176,"log_start":"188'3436","ondisk_log_start":"188'3436","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4440,"log_dups_size":0,"ondisk_log_size":4440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":481820672,"num_objects":7352,"num_object_clones":0,"num_object_copies":22056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7876,"num_write_kb":487296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"71'19","reported_seq":391,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.482948+0000","last_change":"2024-09-20T22:08:33.852966+0000","last_active":"2024-09-20T22:09:27.482948+0000","last_peered":"2024-09-20T22:09:27.482948+0000","last_clean":"2024-09-20T22:09:27.482948+0000","last_became_active":"2024-09-20T22:02:52.480077+0000","last_became_peered":"2024-09-20T22:02:52.480077+0000","last_unstale":"2024-09-20T22:09:27.482948+0000","last_undegraded":"2024-09-20T22:09:27.482948+0000","last_fullsized":"2024-09-20T22:09:27.482948+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:08:33.852806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:33.852806+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:59.554741+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":190,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.485581+0000","last_change":"2024-09-20T22:07:14.382822+0000","last_active":"2024-09-20T22:09:27.485581+0000","last_peered":"2024-09-20T22:09:27.485581+0000","last_clean":"2024-09-20T22:09:27.485581+0000","last_became_active":"2024-09-20T22:06:54.829037+0000","last_became_peered":"2024-09-20T22:06:54.829037+0000","last_unstale":"2024-09-20T22:09:27.485581+0000","last_undegraded":"2024-09-20T22:09:27.485581+0000","last_fullsized":"2024-09-20T22:09:27.485581+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.382712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.382712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":359,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477555+0000","last_change":"2024-09-20T22:09:08.234635+0000","last_active":"2024-09-20T22:09:27.477555+0000","last_peered":"2024-09-20T22:09:27.477555+0000","last_clean":"2024-09-20T22:09:27.477555+0000","last_became_active":"2024-09-20T22:02:54.290787+0000","last_became_peered":"2024-09-20T22:02:54.290787+0000","last_unstale":"2024-09-20T22:09:27.477555+0000","last_undegraded":"2024-09-20T22:09:27.477555+0000","last_fullsized":"2024-09-20T22:09:27.477555+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:09:08.234520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:09.917123+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":188,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.486520+0000","last_change":"2024-09-20T22:09:03.521037+0000","last_active":"2024-09-20T22:09:27.486520+0000","last_peered":"2024-09-20T22:09:27.486520+0000","last_clean":"2024-09-20T22:09:27.486520+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:09:27.486520+0000","last_undegraded":"2024-09-20T22:09:27.486520+0000","last_fullsized":"2024-09-20T22:09:27.486520+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:03.520930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:03.520930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:30.299784+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":346,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.476968+0000","last_change":"2024-09-20T22:08:48.452711+0000","last_active":"2024-09-20T22:09:27.476968+0000","last_peered":"2024-09-20T22:09:27.476968+0000","last_clean":"2024-09-20T22:09:27.476968+0000","last_became_active":"2024-09-20T22:08:48.452234+0000","last_became_peered":"2024-09-20T22:08:48.452234+0000","last_unstale":"2024-09-20T22:09:27.476968+0000","last_undegraded":"2024-09-20T22:09:27.476968+0000","last_fullsized":"2024-09-20T22:09:27.476968+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:07:53.673691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.673691+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:32.262219+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":27,"num_write":17,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":161010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"225'7232","reported_seq":7349,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:30.644200+0000","last_change":"2024-09-20T22:09:09.580125+0000","last_active":"2024-09-20T22:09:30.644200+0000","last_peered":"2024-09-20T22:09:30.644200+0000","last_clean":"2024-09-20T22:09:30.644200+0000","last_became_active":"2024-09-20T22:09:09.561977+0000","last_became_peered":"2024-09-20T22:09:09.561977+0000","last_unstale":"2024-09-20T22:09:30.644200+0000","last_undegraded":"2024-09-20T22:09:30.644200+0000","last_fullsized":"2024-09-20T22:09:30.644200+0000","mapping_epoch":176,"log_start":"184'1859","ondisk_log_start":"184'1859","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":5373,"log_dups_size":0,"ondisk_log_size":5373,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:32.262293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239927296,"num_objects":3661,"num_object_clones":0,"num_object_copies":10983,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3893,"num_write_kb":241728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"146'30","reported_seq":282,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.485692+0000","last_change":"2024-09-20T22:07:55.357829+0000","last_active":"2024-09-20T22:09:27.485692+0000","last_peered":"2024-09-20T22:09:27.485692+0000","last_clean":"2024-09-20T22:09:27.485692+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:09:27.485692+0000","last_undegraded":"2024-09-20T22:09:27.485692+0000","last_fullsized":"2024-09-20T22:09:27.485692+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'30","last_scrub_stamp":"2024-09-20T22:07:55.357726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.357726+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":360,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.472991+0000","last_change":"2024-09-20T22:09:02.547888+0000","last_active":"2024-09-20T22:09:27.472991+0000","last_peered":"2024-09-20T22:09:27.472991+0000","last_clean":"2024-09-20T22:09:27.472991+0000","last_became_active":"2024-09-20T22:02:50.108457+0000","last_became_peered":"2024-09-20T22:02:50.108457+0000","last_unstale":"2024-09-20T22:09:27.472991+0000","last_undegraded":"2024-09-20T22:09:27.472991+0000","last_fullsized":"2024-09-20T22:09:27.472991+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:02.547780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:02.547780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:13.447545+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":363,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.478507+0000","last_change":"2024-09-20T22:07:21.886172+0000","last_active":"2024-09-20T22:09:27.478507+0000","last_peered":"2024-09-20T22:09:27.478507+0000","last_clean":"2024-09-20T22:09:27.478507+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:09:27.478507+0000","last_undegraded":"2024-09-20T22:09:27.478507+0000","last_fullsized":"2024-09-20T22:09:27.478507+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_clean_scrub_stamp":"2024-09-20T22:07:21.886038+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":344,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.483745+0000","last_change":"2024-09-20T22:07:17.566774+0000","last_active":"2024-09-20T22:09:27.483745+0000","last_peered":"2024-09-20T22:09:27.483745+0000","last_clean":"2024-09-20T22:09:27.483745+0000","last_became_active":"2024-09-20T22:06:54.841419+0000","last_became_peered":"2024-09-20T22:06:54.841419+0000","last_unstale":"2024-09-20T22:09:27.483745+0000","last_undegraded":"2024-09-20T22:09:27.483745+0000","last_fullsized":"2024-09-20T22:09:27.483745+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:17.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:17.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":120,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":339,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.474430+0000","last_change":"2024-09-20T22:09:05.892350+0000","last_active":"2024-09-20T22:09:27.474430+0000","last_peered":"2024-09-20T22:09:27.474430+0000","last_clean":"2024-09-20T22:09:27.474430+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:09:27.474430+0000","last_undegraded":"2024-09-20T22:09:27.474430+0000","last_fullsized":"2024-09-20T22:09:27.474430+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:05.892235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:09:05.892235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:30.714492+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":340,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.489017+0000","last_change":"2024-09-20T22:08:55.444848+0000","last_active":"2024-09-20T22:09:27.489017+0000","last_peered":"2024-09-20T22:09:27.489017+0000","last_clean":"2024-09-20T22:09:27.489017+0000","last_became_active":"2024-09-20T22:08:55.444469+0000","last_became_peered":"2024-09-20T22:08:55.444469+0000","last_unstale":"2024-09-20T22:09:27.489017+0000","last_undegraded":"2024-09-20T22:09:27.489017+0000","last_fullsized":"2024-09-20T22:09:27.489017+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:07:53.673691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.673691+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:27.998742+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":27,"num_write":17,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":161010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"225'7604","reported_seq":7715,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:31.141993+0000","last_change":"2024-09-20T22:07:52.429893+0000","last_active":"2024-09-20T22:09:31.141993+0000","last_peered":"2024-09-20T22:09:31.141993+0000","last_clean":"2024-09-20T22:09:31.141993+0000","last_became_active":"2024-09-20T22:07:52.429554+0000","last_became_peered":"2024-09-20T22:07:52.429554+0000","last_unstale":"2024-09-20T22:09:31.141993+0000","last_undegraded":"2024-09-20T22:09:31.141993+0000","last_fullsized":"2024-09-20T22:09:31.141993+0000","mapping_epoch":176,"log_start":"186'3125","ondisk_log_start":"186'3125","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4479,"log_dups_size":0,"ondisk_log_size":4479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":461766656,"num_objects":7046,"num_object_clones":0,"num_object_copies":21138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7604,"num_write_kb":468800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"71'13","reported_seq":227,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.487551+0000","last_change":"2024-09-20T22:08:35.413541+0000","last_active":"2024-09-20T22:09:27.487551+0000","last_peered":"2024-09-20T22:09:27.487551+0000","last_clean":"2024-09-20T22:09:27.487551+0000","last_became_active":"2024-09-20T22:06:54.850696+0000","last_became_peered":"2024-09-20T22:06:54.850696+0000","last_unstale":"2024-09-20T22:09:27.487551+0000","last_undegraded":"2024-09-20T22:09:27.487551+0000","last_fullsized":"2024-09-20T22:09:27.487551+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'13","last_scrub_stamp":"2024-09-20T22:08:35.413221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:35.413221+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:39.827104+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":189,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.491581+0000","last_change":"2024-09-20T22:07:20.341524+0000","last_active":"2024-09-20T22:09:27.491581+0000","last_peered":"2024-09-20T22:09:27.491581+0000","last_clean":"2024-09-20T22:09:27.491581+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:09:27.491581+0000","last_undegraded":"2024-09-20T22:09:27.491581+0000","last_fullsized":"2024-09-20T22:09:27.491581+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:20.338388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:20.338388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":377,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.481455+0000","last_change":"2024-09-20T22:08:36.251074+0000","last_active":"2024-09-20T22:09:27.481455+0000","last_peered":"2024-09-20T22:09:27.481455+0000","last_clean":"2024-09-20T22:09:27.481455+0000","last_became_active":"2024-09-20T22:02:49.099508+0000","last_became_peered":"2024-09-20T22:02:49.099508+0000","last_unstale":"2024-09-20T22:09:27.481455+0000","last_undegraded":"2024-09-20T22:09:27.481455+0000","last_fullsized":"2024-09-20T22:09:27.481455+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:08:36.250748+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:56.158190+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":351,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.483691+0000","last_change":"2024-09-20T22:08:02.168865+0000","last_active":"2024-09-20T22:09:27.483691+0000","last_peered":"2024-09-20T22:09:27.483691+0000","last_clean":"2024-09-20T22:09:27.483691+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:09:27.483691+0000","last_undegraded":"2024-09-20T22:09:27.483691+0000","last_fullsized":"2024-09-20T22:09:27.483691+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:08:02.167030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":340,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.490052+0000","last_change":"2024-09-20T22:08:31.832304+0000","last_active":"2024-09-20T22:09:27.490052+0000","last_peered":"2024-09-20T22:09:27.490052+0000","last_clean":"2024-09-20T22:09:27.490052+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:09:27.490052+0000","last_undegraded":"2024-09-20T22:09:27.490052+0000","last_fullsized":"2024-09-20T22:09:27.490052+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:31.832107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:31.832107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:53.771648+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":353,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.491250+0000","last_change":"2024-09-20T22:08:59.488297+0000","last_active":"2024-09-20T22:09:27.491250+0000","last_peered":"2024-09-20T22:09:27.491250+0000","last_clean":"2024-09-20T22:09:27.491250+0000","last_became_active":"2024-09-20T22:08:48.446840+0000","last_became_peered":"2024-09-20T22:08:48.446840+0000","last_unstale":"2024-09-20T22:09:27.491250+0000","last_undegraded":"2024-09-20T22:09:27.491250+0000","last_fullsized":"2024-09-20T22:09:27.491250+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:08:59.488188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:08:59.488188+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:20.578312+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"225'7593","reported_seq":7703,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.783905+0000","last_change":"2024-09-20T22:07:52.939933+0000","last_active":"2024-09-20T22:09:27.783905+0000","last_peered":"2024-09-20T22:09:27.783905+0000","last_clean":"2024-09-20T22:09:27.783905+0000","last_became_active":"2024-09-20T22:07:52.939626+0000","last_became_peered":"2024-09-20T22:07:52.939626+0000","last_unstale":"2024-09-20T22:09:27.783905+0000","last_undegraded":"2024-09-20T22:09:27.783905+0000","last_fullsized":"2024-09-20T22:09:27.783905+0000","mapping_epoch":176,"log_start":"189'3497","ondisk_log_start":"189'3497","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4096,"log_dups_size":0,"ondisk_log_size":4096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":492896256,"num_objects":7521,"num_object_clones":0,"num_object_copies":22563,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7593,"num_write_kb":483648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"71'12","reported_seq":366,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.484704+0000","last_change":"2024-09-20T22:07:29.614630+0000","last_active":"2024-09-20T22:09:27.484704+0000","last_peered":"2024-09-20T22:09:27.484704+0000","last_clean":"2024-09-20T22:09:27.484704+0000","last_became_active":"2024-09-20T22:06:54.848670+0000","last_became_peered":"2024-09-20T22:06:54.848670+0000","last_unstale":"2024-09-20T22:09:27.484704+0000","last_undegraded":"2024-09-20T22:09:27.484704+0000","last_fullsized":"2024-09-20T22:09:27.484704+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:29.614551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:29.614551+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":351,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.485068+0000","last_change":"2024-09-20T22:06:44.862452+0000","last_active":"2024-09-20T22:09:27.485068+0000","last_peered":"2024-09-20T22:09:27.485068+0000","last_clean":"2024-09-20T22:09:27.485068+0000","last_became_active":"2024-09-20T22:02:50.110448+0000","last_became_peered":"2024-09-20T22:02:50.110448+0000","last_unstale":"2024-09-20T22:09:27.485068+0000","last_undegraded":"2024-09-20T22:09:27.485068+0000","last_fullsized":"2024-09-20T22:09:27.485068+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:06:44.862338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":757,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:29.048514+0000","last_change":"2024-09-20T22:07:16.590582+0000","last_active":"2024-09-20T22:09:29.048514+0000","last_peered":"2024-09-20T22:09:29.048514+0000","last_clean":"2024-09-20T22:09:29.048514+0000","last_became_active":"2024-09-20T22:06:54.844454+0000","last_became_peered":"2024-09-20T22:06:54.844454+0000","last_unstale":"2024-09-20T22:09:29.048514+0000","last_undegraded":"2024-09-20T22:09:29.048514+0000","last_fullsized":"2024-09-20T22:09:29.048514+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:07:16.590475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:07:16.590475+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":387,"num_read_kb":382,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":344,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.478442+0000","last_change":"2024-09-20T22:07:22.884981+0000","last_active":"2024-09-20T22:09:27.478442+0000","last_peered":"2024-09-20T22:09:27.478442+0000","last_clean":"2024-09-20T22:09:27.478442+0000","last_became_active":"2024-09-20T22:06:54.841596+0000","last_became_peered":"2024-09-20T22:06:54.841596+0000","last_unstale":"2024-09-20T22:09:27.478442+0000","last_undegraded":"2024-09-20T22:09:27.478442+0000","last_fullsized":"2024-09-20T22:09:27.478442+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:22.884907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.884907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":189,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.485421+0000","last_change":"2024-09-20T22:08:34.415569+0000","last_active":"2024-09-20T22:09:27.485421+0000","last_peered":"2024-09-20T22:09:27.485421+0000","last_clean":"2024-09-20T22:09:27.485421+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:09:27.485421+0000","last_undegraded":"2024-09-20T22:09:27.485421+0000","last_fullsized":"2024-09-20T22:09:27.485421+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:34.415468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.415468+0000","last_clean_scrub_stamp":"2024-09-20T22:08:34.415468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:39.709664+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"225'7170","reported_seq":7329,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:30.698227+0000","last_change":"2024-09-20T22:09:10.870815+0000","last_active":"2024-09-20T22:09:30.698227+0000","last_peered":"2024-09-20T22:09:30.698227+0000","last_clean":"2024-09-20T22:09:30.698227+0000","last_became_active":"2024-09-20T22:09:09.578463+0000","last_became_peered":"2024-09-20T22:09:09.578463+0000","last_unstale":"2024-09-20T22:09:30.698227+0000","last_undegraded":"2024-09-20T22:09:30.698227+0000","last_fullsized":"2024-09-20T22:09:30.698227+0000","mapping_epoch":176,"log_start":"182'1200","ondisk_log_start":"182'1200","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"216'6610","last_scrub_stamp":"2024-09-20T22:09:10.870629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:09:10.870629+0000","objects_scrubbed":3287,"log_size":5970,"log_dups_size":0,"ondisk_log_size":5970,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:18.397304+0000","scrub_duration":670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234749952,"num_objects":3582,"num_object_clones":0,"num_object_copies":10746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":561,"num_write_kb":27392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"71'16","reported_seq":458,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.478983+0000","last_change":"2024-09-20T22:07:53.297530+0000","last_active":"2024-09-20T22:09:27.478983+0000","last_peered":"2024-09-20T22:09:27.478983+0000","last_clean":"2024-09-20T22:09:27.478983+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:09:27.478983+0000","last_undegraded":"2024-09-20T22:09:27.478983+0000","last_fullsized":"2024-09-20T22:09:27.478983+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'16","last_scrub_stamp":"2024-09-20T22:07:53.297442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.297442+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":351,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.473490+0000","last_change":"2024-09-20T22:06:48.105501+0000","last_active":"2024-09-20T22:09:27.473490+0000","last_peered":"2024-09-20T22:09:27.473490+0000","last_clean":"2024-09-20T22:09:27.473490+0000","last_became_active":"2024-09-20T22:02:50.106011+0000","last_became_peered":"2024-09-20T22:02:50.106011+0000","last_unstale":"2024-09-20T22:09:27.473490+0000","last_undegraded":"2024-09-20T22:09:27.473490+0000","last_fullsized":"2024-09-20T22:09:27.473490+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:48.105403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:48.105403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":340,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.482044+0000","last_change":"2024-09-20T22:08:55.451233+0000","last_active":"2024-09-20T22:09:27.482044+0000","last_peered":"2024-09-20T22:09:27.482044+0000","last_clean":"2024-09-20T22:09:27.482044+0000","last_became_active":"2024-09-20T22:08:55.450855+0000","last_became_peered":"2024-09-20T22:08:55.450855+0000","last_unstale":"2024-09-20T22:09:27.482044+0000","last_undegraded":"2024-09-20T22:09:27.482044+0000","last_fullsized":"2024-09-20T22:09:27.482044+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:07:53.673691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.673691+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:27.998765+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53669,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":26,"num_write":16,"num_write_kb":219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":161009,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":59,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.481948+0000","last_change":"2024-09-20T22:08:59.475385+0000","last_active":"2024-09-20T22:09:27.481948+0000","last_peered":"2024-09-20T22:09:27.481948+0000","last_clean":"2024-09-20T22:09:27.481948+0000","last_became_active":"2024-09-20T22:08:52.423678+0000","last_became_peered":"2024-09-20T22:08:52.423678+0000","last_unstale":"2024-09-20T22:09:27.481948+0000","last_undegraded":"2024-09-20T22:09:27.481948+0000","last_fullsized":"2024-09-20T22:09:27.481948+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:59.475260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:59.475260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:28.970779+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":361,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477995+0000","last_change":"2024-09-20T22:08:39.101879+0000","last_active":"2024-09-20T22:09:27.477995+0000","last_peered":"2024-09-20T22:09:27.477995+0000","last_clean":"2024-09-20T22:09:27.477995+0000","last_became_active":"2024-09-20T22:04:21.237725+0000","last_became_peered":"2024-09-20T22:04:21.237725+0000","last_unstale":"2024-09-20T22:09:27.477995+0000","last_undegraded":"2024-09-20T22:09:27.477995+0000","last_fullsized":"2024-09-20T22:09:27.477995+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:39.101763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:08:39.101763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:05.527492+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"225'7101","reported_seq":7260,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:30.616635+0000","last_change":"2024-09-20T22:09:11.017877+0000","last_active":"2024-09-20T22:09:30.616635+0000","last_peered":"2024-09-20T22:09:30.616635+0000","last_clean":"2024-09-20T22:09:30.616635+0000","last_became_active":"2024-09-20T22:09:09.551695+0000","last_became_peered":"2024-09-20T22:09:09.551695+0000","last_unstale":"2024-09-20T22:09:30.616635+0000","last_undegraded":"2024-09-20T22:09:30.616635+0000","last_fullsized":"2024-09-20T22:09:30.616635+0000","mapping_epoch":176,"log_start":"185'2032","ondisk_log_start":"185'2032","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"217'6588","last_scrub_stamp":"2024-09-20T22:09:11.017735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:09:11.017735+0000","objects_scrubbed":3372,"log_size":5069,"log_dups_size":0,"ondisk_log_size":5069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:11.622652+0000","scrub_duration":565,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":239927296,"num_objects":3661,"num_object_clones":0,"num_object_copies":10983,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":513,"num_write_kb":25664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"71'11","reported_seq":367,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.474112+0000","last_change":"2024-09-20T22:07:13.511239+0000","last_active":"2024-09-20T22:09:27.474112+0000","last_peered":"2024-09-20T22:09:27.474112+0000","last_clean":"2024-09-20T22:09:27.474112+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:09:27.474112+0000","last_undegraded":"2024-09-20T22:09:27.474112+0000","last_fullsized":"2024-09-20T22:09:27.474112+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:13.511159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:13.511159+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":352,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.472040+0000","last_change":"2024-09-20T22:08:40.493022+0000","last_active":"2024-09-20T22:09:27.472040+0000","last_peered":"2024-09-20T22:09:27.472040+0000","last_clean":"2024-09-20T22:09:27.472040+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:09:27.472040+0000","last_undegraded":"2024-09-20T22:09:27.472040+0000","last_fullsized":"2024-09-20T22:09:27.472040+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:40.492952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:40.492952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:45.281101+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":350,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477899+0000","last_change":"2024-09-20T22:08:27.057946+0000","last_active":"2024-09-20T22:09:27.477899+0000","last_peered":"2024-09-20T22:09:27.477899+0000","last_clean":"2024-09-20T22:09:27.477899+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:09:27.477899+0000","last_undegraded":"2024-09-20T22:09:27.477899+0000","last_fullsized":"2024-09-20T22:09:27.477899+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:27.057849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:27.057849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:36.999687+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":338,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.473162+0000","last_change":"2024-09-20T22:06:45.030549+0000","last_active":"2024-09-20T22:09:27.473162+0000","last_peered":"2024-09-20T22:09:27.473162+0000","last_clean":"2024-09-20T22:09:27.473162+0000","last_became_active":"2024-09-20T22:02:56.349376+0000","last_became_peered":"2024-09-20T22:02:56.349376+0000","last_unstale":"2024-09-20T22:09:27.473162+0000","last_undegraded":"2024-09-20T22:09:27.473162+0000","last_fullsized":"2024-09-20T22:09:27.473162+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.030289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.030289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":346,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.476655+0000","last_change":"2024-09-20T22:08:48.461425+0000","last_active":"2024-09-20T22:09:27.476655+0000","last_peered":"2024-09-20T22:09:27.476655+0000","last_clean":"2024-09-20T22:09:27.476655+0000","last_became_active":"2024-09-20T22:08:48.460964+0000","last_became_peered":"2024-09-20T22:08:48.460964+0000","last_unstale":"2024-09-20T22:09:27.476655+0000","last_undegraded":"2024-09-20T22:09:27.476655+0000","last_fullsized":"2024-09-20T22:09:27.476655+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:07:53.673691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.673691+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:32.262269+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":27,"num_write":16,"num_write_kb":219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":423697,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"225'7025","reported_seq":7181,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.779136+0000","last_change":"2024-09-20T22:09:10.957765+0000","last_active":"2024-09-20T22:09:27.779136+0000","last_peered":"2024-09-20T22:09:27.779136+0000","last_clean":"2024-09-20T22:09:27.779136+0000","last_became_active":"2024-09-20T22:09:09.551097+0000","last_became_peered":"2024-09-20T22:09:09.551097+0000","last_unstale":"2024-09-20T22:09:27.779136+0000","last_undegraded":"2024-09-20T22:09:27.779136+0000","last_fullsized":"2024-09-20T22:09:27.779136+0000","mapping_epoch":176,"log_start":"185'2185","ondisk_log_start":"185'2185","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"217'6597","last_scrub_stamp":"2024-09-20T22:09:10.957585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:09:10.957585+0000","objects_scrubbed":3328,"log_size":4840,"log_dups_size":0,"ondisk_log_size":4840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:28.897373+0000","scrub_duration":523,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":244056064,"num_objects":3724,"num_object_clones":0,"num_object_copies":11172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":428,"num_write_kb":26368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"71'17","reported_seq":384,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.472828+0000","last_change":"2024-09-20T22:07:54.528854+0000","last_active":"2024-09-20T22:09:27.472828+0000","last_peered":"2024-09-20T22:09:27.472828+0000","last_clean":"2024-09-20T22:09:27.472828+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:09:27.472828+0000","last_undegraded":"2024-09-20T22:09:27.472828+0000","last_fullsized":"2024-09-20T22:09:27.472828+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'17","last_scrub_stamp":"2024-09-20T22:07:54.528759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:54.528759+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":360,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477922+0000","last_change":"2024-09-20T22:08:30.429880+0000","last_active":"2024-09-20T22:09:27.477922+0000","last_peered":"2024-09-20T22:09:27.477922+0000","last_clean":"2024-09-20T22:09:27.477922+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:09:27.477922+0000","last_undegraded":"2024-09-20T22:09:27.477922+0000","last_fullsized":"2024-09-20T22:09:27.477922+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:30.429704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:30.429704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:36.273776+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":189,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.485510+0000","last_change":"2024-09-20T22:07:19.376008+0000","last_active":"2024-09-20T22:09:27.485510+0000","last_peered":"2024-09-20T22:09:27.485510+0000","last_clean":"2024-09-20T22:09:27.485510+0000","last_became_active":"2024-09-20T22:06:54.848275+0000","last_became_peered":"2024-09-20T22:06:54.848275+0000","last_unstale":"2024-09-20T22:09:27.485510+0000","last_undegraded":"2024-09-20T22:09:27.485510+0000","last_fullsized":"2024-09-20T22:09:27.485510+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:19.375867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.375867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":346,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.481482+0000","last_change":"2024-09-20T22:07:55.195229+0000","last_active":"2024-09-20T22:09:27.481482+0000","last_peered":"2024-09-20T22:09:27.481482+0000","last_clean":"2024-09-20T22:09:27.481482+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:09:27.481482+0000","last_undegraded":"2024-09-20T22:09:27.481482+0000","last_fullsized":"2024-09-20T22:09:27.481482+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:55.195131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.195131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":631,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.489950+0000","last_change":"2024-09-20T22:08:58.773488+0000","last_active":"2024-09-20T22:09:27.489950+0000","last_peered":"2024-09-20T22:09:27.489950+0000","last_clean":"2024-09-20T22:09:27.489950+0000","last_became_active":"2024-09-20T22:08:48.455288+0000","last_became_peered":"2024-09-20T22:08:48.455288+0000","last_unstale":"2024-09-20T22:09:27.489950+0000","last_undegraded":"2024-09-20T22:09:27.489950+0000","last_fullsized":"2024-09-20T22:09:27.489950+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:08:58.773367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:08:58.773367+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:00.937529+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"225'7183","reported_seq":7341,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:31.027087+0000","last_change":"2024-09-20T22:09:10.725332+0000","last_active":"2024-09-20T22:09:31.027087+0000","last_peered":"2024-09-20T22:09:31.027087+0000","last_clean":"2024-09-20T22:09:31.027087+0000","last_became_active":"2024-09-20T22:09:09.591212+0000","last_became_peered":"2024-09-20T22:09:09.591212+0000","last_unstale":"2024-09-20T22:09:31.027087+0000","last_undegraded":"2024-09-20T22:09:31.027087+0000","last_fullsized":"2024-09-20T22:09:31.027087+0000","mapping_epoch":176,"log_start":"184'1794","ondisk_log_start":"184'1794","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"217'6680","last_scrub_stamp":"2024-09-20T22:09:10.725099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:09:10.725099+0000","objects_scrubbed":3317,"log_size":5389,"log_dups_size":0,"ondisk_log_size":5389,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:40.073825+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234684416,"num_objects":3581,"num_object_clones":0,"num_object_copies":10743,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":518,"num_write_kb":25024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"71'10","reported_seq":362,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.472059+0000","last_change":"2024-09-20T22:09:01.543757+0000","last_active":"2024-09-20T22:09:27.472059+0000","last_peered":"2024-09-20T22:09:27.472059+0000","last_clean":"2024-09-20T22:09:27.472059+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:09:27.472059+0000","last_undegraded":"2024-09-20T22:09:27.472059+0000","last_fullsized":"2024-09-20T22:09:27.472059+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:09:01.543678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:09:01.543678+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:02.970566+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"64'1","reported_seq":356,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.473613+0000","last_change":"2024-09-20T22:06:33.966005+0000","last_active":"2024-09-20T22:09:27.473613+0000","last_peered":"2024-09-20T22:09:27.473613+0000","last_clean":"2024-09-20T22:09:27.473613+0000","last_became_active":"2024-09-20T22:02:50.110446+0000","last_became_peered":"2024-09-20T22:02:50.110446+0000","last_unstale":"2024-09-20T22:09:27.473613+0000","last_undegraded":"2024-09-20T22:09:27.473613+0000","last_fullsized":"2024-09-20T22:09:27.473613+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:06:33.965866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:33.965866+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"139'9","reported_seq":448,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.483421+0000","last_change":"2024-09-20T22:07:57.138071+0000","last_active":"2024-09-20T22:09:27.483421+0000","last_peered":"2024-09-20T22:09:27.483421+0000","last_clean":"2024-09-20T22:09:27.483421+0000","last_became_active":"2024-09-20T22:02:54.294458+0000","last_became_peered":"2024-09-20T22:02:54.294458+0000","last_unstale":"2024-09-20T22:09:27.483421+0000","last_undegraded":"2024-09-20T22:09:27.483421+0000","last_fullsized":"2024-09-20T22:09:27.483421+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'9","last_scrub_stamp":"2024-09-20T22:07:57.137943+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:06:40.910011+0000","last_clean_scrub_stamp":"2024-09-20T22:07:57.137943+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":340,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.473921+0000","last_change":"2024-09-20T22:07:23.482514+0000","last_active":"2024-09-20T22:09:27.473921+0000","last_peered":"2024-09-20T22:09:27.473921+0000","last_clean":"2024-09-20T22:09:27.473921+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:09:27.473921+0000","last_undegraded":"2024-09-20T22:09:27.473921+0000","last_fullsized":"2024-09-20T22:09:27.473921+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.482372+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"225'7101","reported_seq":7217,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.782754+0000","last_change":"2024-09-20T22:09:09.571096+0000","last_active":"2024-09-20T22:09:27.782754+0000","last_peered":"2024-09-20T22:09:27.782754+0000","last_clean":"2024-09-20T22:09:27.782754+0000","last_became_active":"2024-09-20T22:09:09.566984+0000","last_became_peered":"2024-09-20T22:09:09.566984+0000","last_unstale":"2024-09-20T22:09:27.782754+0000","last_undegraded":"2024-09-20T22:09:27.782754+0000","last_fullsized":"2024-09-20T22:09:27.782754+0000","mapping_epoch":176,"log_start":"185'2256","ondisk_log_start":"185'2256","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4845,"log_dups_size":0,"ondisk_log_size":4845,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":245989376,"num_objects":3753,"num_object_clones":0,"num_object_copies":11259,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3795,"num_write_kb":241568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"71'9","reported_seq":371,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.482851+0000","last_change":"2024-09-20T22:07:59.740218+0000","last_active":"2024-09-20T22:09:27.482851+0000","last_peered":"2024-09-20T22:09:27.482851+0000","last_clean":"2024-09-20T22:09:27.482851+0000","last_became_active":"2024-09-20T22:06:54.847770+0000","last_became_peered":"2024-09-20T22:06:54.847770+0000","last_unstale":"2024-09-20T22:09:27.482851+0000","last_undegraded":"2024-09-20T22:09:27.482851+0000","last_fullsized":"2024-09-20T22:09:27.482851+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:59.740089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:59.740089+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":364,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.478484+0000","last_change":"2024-09-20T22:08:31.071694+0000","last_active":"2024-09-20T22:09:27.478484+0000","last_peered":"2024-09-20T22:09:27.478484+0000","last_clean":"2024-09-20T22:09:27.478484+0000","last_became_active":"2024-09-20T22:02:50.112682+0000","last_became_peered":"2024-09-20T22:02:50.112682+0000","last_unstale":"2024-09-20T22:09:27.478484+0000","last_undegraded":"2024-09-20T22:09:27.478484+0000","last_fullsized":"2024-09-20T22:09:27.478484+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:08:31.071610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:31.071610+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:35.205784+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":359,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.482002+0000","last_change":"2024-09-20T22:09:06.528889+0000","last_active":"2024-09-20T22:09:27.482002+0000","last_peered":"2024-09-20T22:09:27.482002+0000","last_clean":"2024-09-20T22:09:27.482002+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:09:27.482002+0000","last_undegraded":"2024-09-20T22:09:27.482002+0000","last_fullsized":"2024-09-20T22:09:27.482002+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:09:06.528588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:07.052667+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":349,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.485016+0000","last_change":"2024-09-20T22:08:27.817949+0000","last_active":"2024-09-20T22:09:27.485016+0000","last_peered":"2024-09-20T22:09:27.485016+0000","last_clean":"2024-09-20T22:09:27.485016+0000","last_became_active":"2024-09-20T22:06:55.824229+0000","last_became_peered":"2024-09-20T22:06:55.824229+0000","last_unstale":"2024-09-20T22:09:27.485016+0000","last_undegraded":"2024-09-20T22:09:27.485016+0000","last_fullsized":"2024-09-20T22:09:27.485016+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:27.817850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:27.817850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:45.479132+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"225'7181","reported_seq":7297,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:31.475298+0000","last_change":"2024-09-20T22:09:09.567024+0000","last_active":"2024-09-20T22:09:31.475298+0000","last_peered":"2024-09-20T22:09:31.475298+0000","last_clean":"2024-09-20T22:09:31.475298+0000","last_became_active":"2024-09-20T22:09:09.564622+0000","last_became_peered":"2024-09-20T22:09:09.564622+0000","last_unstale":"2024-09-20T22:09:31.475298+0000","last_undegraded":"2024-09-20T22:09:31.475298+0000","last_fullsized":"2024-09-20T22:09:31.475298+0000","mapping_epoch":176,"log_start":"183'1626","ondisk_log_start":"183'1626","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":5555,"log_dups_size":0,"ondisk_log_size":5555,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:33.068696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":236224512,"num_objects":3604,"num_object_clones":0,"num_object_copies":10812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3900,"num_write_kb":240160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"71'19","reported_seq":405,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477651+0000","last_change":"2024-09-20T22:08:51.467188+0000","last_active":"2024-09-20T22:09:27.477651+0000","last_peered":"2024-09-20T22:09:27.477651+0000","last_clean":"2024-09-20T22:09:27.477651+0000","last_became_active":"2024-09-20T22:08:51.451319+0000","last_became_peered":"2024-09-20T22:08:51.451319+0000","last_unstale":"2024-09-20T22:09:27.477651+0000","last_undegraded":"2024-09-20T22:09:27.477651+0000","last_fullsized":"2024-09-20T22:09:27.477651+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:07:25.154919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:25.154919+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":351,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.490202+0000","last_change":"2024-09-20T22:08:00.560208+0000","last_active":"2024-09-20T22:09:27.490202+0000","last_peered":"2024-09-20T22:09:27.490202+0000","last_clean":"2024-09-20T22:09:27.490202+0000","last_became_active":"2024-09-20T22:02:50.108712+0000","last_became_peered":"2024-09-20T22:02:50.108712+0000","last_unstale":"2024-09-20T22:09:27.490202+0000","last_undegraded":"2024-09-20T22:09:27.490202+0000","last_fullsized":"2024-09-20T22:09:27.490202+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:00.560098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:00.560098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":358,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.480959+0000","last_change":"2024-09-20T22:09:07.554566+0000","last_active":"2024-09-20T22:09:27.480959+0000","last_peered":"2024-09-20T22:09:27.480959+0000","last_clean":"2024-09-20T22:09:27.480959+0000","last_became_active":"2024-09-20T22:02:54.291134+0000","last_became_peered":"2024-09-20T22:02:54.291134+0000","last_unstale":"2024-09-20T22:09:27.480959+0000","last_undegraded":"2024-09-20T22:09:27.480959+0000","last_fullsized":"2024-09-20T22:09:27.480959+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:07.554455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:07.554455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:21.068448+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":59,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.480890+0000","last_change":"2024-09-20T22:09:08.495817+0000","last_active":"2024-09-20T22:09:27.480890+0000","last_peered":"2024-09-20T22:09:27.480890+0000","last_clean":"2024-09-20T22:09:27.480890+0000","last_became_active":"2024-09-20T22:08:52.420555+0000","last_became_peered":"2024-09-20T22:08:52.420555+0000","last_unstale":"2024-09-20T22:09:27.480890+0000","last_undegraded":"2024-09-20T22:09:27.480890+0000","last_fullsized":"2024-09-20T22:09:27.480890+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:08.495599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:08.495599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:30.744649+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"225'7282","reported_seq":7426,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:30.656196+0000","last_change":"2024-09-20T22:09:12.421515+0000","last_active":"2024-09-20T22:09:30.656196+0000","last_peered":"2024-09-20T22:09:30.656196+0000","last_clean":"2024-09-20T22:09:30.656196+0000","last_became_active":"2024-09-20T22:09:09.583278+0000","last_became_peered":"2024-09-20T22:09:09.583278+0000","last_unstale":"2024-09-20T22:09:30.656196+0000","last_undegraded":"2024-09-20T22:09:30.656196+0000","last_fullsized":"2024-09-20T22:09:30.656196+0000","mapping_epoch":176,"log_start":"182'1400","ondisk_log_start":"182'1400","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":2399,"log_size":5882,"log_dups_size":0,"ondisk_log_size":5882,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:34.068724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":240025600,"num_objects":3662,"num_object_clones":0,"num_object_copies":10986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3924,"num_write_kb":242784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"71'12","reported_seq":372,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.471596+0000","last_change":"2024-09-20T22:07:22.458421+0000","last_active":"2024-09-20T22:09:27.471596+0000","last_peered":"2024-09-20T22:09:27.471596+0000","last_clean":"2024-09-20T22:09:27.471596+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:09:27.471596+0000","last_undegraded":"2024-09-20T22:09:27.471596+0000","last_fullsized":"2024-09-20T22:09:27.471596+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:22.458279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.458279+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":360,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.490135+0000","last_change":"2024-09-20T22:09:03.831996+0000","last_active":"2024-09-20T22:09:27.490135+0000","last_peered":"2024-09-20T22:09:27.490135+0000","last_clean":"2024-09-20T22:09:27.490135+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:09:27.490135+0000","last_undegraded":"2024-09-20T22:09:27.490135+0000","last_fullsized":"2024-09-20T22:09:27.490135+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:03.831774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:03.831774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:30.562595+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":360,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.481053+0000","last_change":"2024-09-20T22:09:01.509627+0000","last_active":"2024-09-20T22:09:27.481053+0000","last_peered":"2024-09-20T22:09:27.481053+0000","last_clean":"2024-09-20T22:09:27.481053+0000","last_became_active":"2024-09-20T22:06:54.843906+0000","last_became_peered":"2024-09-20T22:06:54.843906+0000","last_unstale":"2024-09-20T22:09:27.481053+0000","last_undegraded":"2024-09-20T22:09:27.481053+0000","last_fullsized":"2024-09-20T22:09:27.481053+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:01.509511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:01.509511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:02.825783+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":298,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.483191+0000","last_change":"2024-09-20T22:08:52.418883+0000","last_active":"2024-09-20T22:09:27.483191+0000","last_peered":"2024-09-20T22:09:27.483191+0000","last_clean":"2024-09-20T22:09:27.483191+0000","last_became_active":"2024-09-20T22:08:52.418544+0000","last_became_peered":"2024-09-20T22:08:52.418544+0000","last_unstale":"2024-09-20T22:09:27.483191+0000","last_undegraded":"2024-09-20T22:09:27.483191+0000","last_fullsized":"2024-09-20T22:09:27.483191+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:36.768146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:36.768146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:55.646733+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"225'7082","reported_seq":7198,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:30.677209+0000","last_change":"2024-09-20T22:09:09.573570+0000","last_active":"2024-09-20T22:09:30.677209+0000","last_peered":"2024-09-20T22:09:30.677209+0000","last_clean":"2024-09-20T22:09:30.677209+0000","last_became_active":"2024-09-20T22:09:09.561556+0000","last_became_peered":"2024-09-20T22:09:09.561556+0000","last_unstale":"2024-09-20T22:09:30.677209+0000","last_undegraded":"2024-09-20T22:09:30.677209+0000","last_fullsized":"2024-09-20T22:09:30.677209+0000","mapping_epoch":176,"log_start":"183'1671","ondisk_log_start":"183'1671","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":5411,"log_dups_size":0,"ondisk_log_size":5411,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:32.263390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":233996288,"num_objects":3570,"num_object_clones":0,"num_object_copies":10710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3836,"num_write_kb":237024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"71'15","reported_seq":402,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.476787+0000","last_change":"2024-09-20T22:08:51.413659+0000","last_active":"2024-09-20T22:09:27.476787+0000","last_peered":"2024-09-20T22:09:27.476787+0000","last_clean":"2024-09-20T22:09:27.476787+0000","last_became_active":"2024-09-20T22:08:51.405122+0000","last_became_peered":"2024-09-20T22:08:51.405122+0000","last_unstale":"2024-09-20T22:09:27.476787+0000","last_undegraded":"2024-09-20T22:09:27.476787+0000","last_fullsized":"2024-09-20T22:09:27.476787+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:08:38.115162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:38.115162+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:53.864987+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":243,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.472764+0000","last_change":"2024-09-20T22:08:51.463648+0000","last_active":"2024-09-20T22:09:27.472764+0000","last_peered":"2024-09-20T22:09:27.472764+0000","last_clean":"2024-09-20T22:09:27.472764+0000","last_became_active":"2024-09-20T22:08:51.454410+0000","last_became_peered":"2024-09-20T22:08:51.454410+0000","last_unstale":"2024-09-20T22:09:27.472764+0000","last_undegraded":"2024-09-20T22:09:27.472764+0000","last_fullsized":"2024-09-20T22:09:27.472764+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:06:40.813590+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":433,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.489837+0000","last_change":"2024-09-20T22:08:32.824544+0000","last_active":"2024-09-20T22:09:27.489837+0000","last_peered":"2024-09-20T22:09:27.489837+0000","last_clean":"2024-09-20T22:09:27.489837+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:09:27.489837+0000","last_undegraded":"2024-09-20T22:09:27.489837+0000","last_fullsized":"2024-09-20T22:09:27.489837+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:08:32.824413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.824413+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:01.392553+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":347,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477990+0000","last_change":"2024-09-20T22:08:32.081639+0000","last_active":"2024-09-20T22:09:27.477990+0000","last_peered":"2024-09-20T22:09:27.477990+0000","last_clean":"2024-09-20T22:09:27.477990+0000","last_became_active":"2024-09-20T22:02:56.346990+0000","last_became_peered":"2024-09-20T22:02:56.346990+0000","last_unstale":"2024-09-20T22:09:27.477990+0000","last_undegraded":"2024-09-20T22:09:27.477990+0000","last_fullsized":"2024-09-20T22:09:27.477990+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:32.081527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.081527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:01.717616+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":358,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.476984+0000","last_change":"2024-09-20T22:08:01.261458+0000","last_active":"2024-09-20T22:09:27.476984+0000","last_peered":"2024-09-20T22:09:27.476984+0000","last_clean":"2024-09-20T22:09:27.476984+0000","last_became_active":"2024-09-20T22:02:50.111935+0000","last_became_peered":"2024-09-20T22:02:50.111935+0000","last_unstale":"2024-09-20T22:09:27.476984+0000","last_undegraded":"2024-09-20T22:09:27.476984+0000","last_fullsized":"2024-09-20T22:09:27.476984+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:01.261328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:01.261328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"71'6","reported_seq":369,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.475746+0000","last_change":"2024-09-20T22:08:52.431426+0000","last_active":"2024-09-20T22:09:27.475746+0000","last_peered":"2024-09-20T22:09:27.475746+0000","last_clean":"2024-09-20T22:09:27.475746+0000","last_became_active":"2024-09-20T22:08:52.419521+0000","last_became_peered":"2024-09-20T22:08:52.419521+0000","last_unstale":"2024-09-20T22:09:27.475746+0000","last_undegraded":"2024-09-20T22:09:27.475746+0000","last_fullsized":"2024-09-20T22:09:27.475746+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'6","last_scrub_stamp":"2024-09-20T22:08:33.574900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:33.574900+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:38.149614+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":343,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477531+0000","last_change":"2024-09-20T22:07:19.865906+0000","last_active":"2024-09-20T22:09:27.477531+0000","last_peered":"2024-09-20T22:09:27.477531+0000","last_clean":"2024-09-20T22:09:27.477531+0000","last_became_active":"2024-09-20T22:06:54.836200+0000","last_became_peered":"2024-09-20T22:06:54.836200+0000","last_unstale":"2024-09-20T22:09:27.477531+0000","last_undegraded":"2024-09-20T22:09:27.477531+0000","last_fullsized":"2024-09-20T22:09:27.477531+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:19.865809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.865809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":339,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.489626+0000","last_change":"2024-09-20T22:09:00.737324+0000","last_active":"2024-09-20T22:09:27.489626+0000","last_peered":"2024-09-20T22:09:27.489626+0000","last_clean":"2024-09-20T22:09:27.489626+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:09:27.489626+0000","last_undegraded":"2024-09-20T22:09:27.489626+0000","last_fullsized":"2024-09-20T22:09:27.489626+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:00.737219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:00.737219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:04.114790+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"71'9","reported_seq":377,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.484796+0000","last_change":"2024-09-20T22:08:52.433690+0000","last_active":"2024-09-20T22:09:27.484796+0000","last_peered":"2024-09-20T22:09:27.484796+0000","last_clean":"2024-09-20T22:09:27.484796+0000","last_became_active":"2024-09-20T22:08:52.423437+0000","last_became_peered":"2024-09-20T22:08:52.423437+0000","last_unstale":"2024-09-20T22:09:27.484796+0000","last_undegraded":"2024-09-20T22:09:27.484796+0000","last_fullsized":"2024-09-20T22:09:27.484796+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:08:31.897253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:31.897253+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:53.330127+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":351,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.490134+0000","last_change":"2024-09-20T22:08:28.755412+0000","last_active":"2024-09-20T22:09:27.490134+0000","last_peered":"2024-09-20T22:09:27.490134+0000","last_clean":"2024-09-20T22:09:27.490134+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:09:27.490134+0000","last_undegraded":"2024-09-20T22:09:27.490134+0000","last_fullsized":"2024-09-20T22:09:27.490134+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:28.755335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:28.755335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:33.713076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":343,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.475548+0000","last_change":"2024-09-20T22:08:36.681072+0000","last_active":"2024-09-20T22:09:27.475548+0000","last_peered":"2024-09-20T22:09:27.475548+0000","last_clean":"2024-09-20T22:09:27.475548+0000","last_became_active":"2024-09-20T22:06:54.845704+0000","last_became_peered":"2024-09-20T22:06:54.845704+0000","last_unstale":"2024-09-20T22:09:27.475548+0000","last_undegraded":"2024-09-20T22:09:27.475548+0000","last_fullsized":"2024-09-20T22:09:27.475548+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:36.680959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:36.680959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:37.238356+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":346,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.481781+0000","last_change":"2024-09-20T22:08:31.095724+0000","last_active":"2024-09-20T22:09:27.481781+0000","last_peered":"2024-09-20T22:09:27.481781+0000","last_clean":"2024-09-20T22:09:27.481781+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:09:27.481781+0000","last_undegraded":"2024-09-20T22:09:27.481781+0000","last_fullsized":"2024-09-20T22:09:27.481781+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:31.095323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:31.095323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:32.516870+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"225'7128","reported_seq":14372,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:30.667317+0000","last_change":"2024-09-20T22:09:26.472135+0000","last_active":"2024-09-20T22:09:30.667317+0000","last_peered":"2024-09-20T22:09:30.667317+0000","last_clean":"2024-09-20T22:09:30.667317+0000","last_became_active":"2024-09-20T22:09:26.467651+0000","last_became_peered":"2024-09-20T22:09:26.467651+0000","last_unstale":"2024-09-20T22:09:30.667317+0000","last_undegraded":"2024-09-20T22:09:30.667317+0000","last_fullsized":"2024-09-20T22:09:30.667317+0000","mapping_epoch":223,"log_start":"182'1200","ondisk_log_start":"182'1200","created":215,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":5928,"log_dups_size":0,"ondisk_log_size":5928,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:32.263425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238878720,"num_objects":3645,"num_object_clones":0,"num_object_copies":10935,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3645,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3847,"num_write_kb":239744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":233111552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"71'9","reported_seq":370,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477307+0000","last_change":"2024-09-20T22:07:52.834986+0000","last_active":"2024-09-20T22:09:27.477307+0000","last_peered":"2024-09-20T22:09:27.477307+0000","last_clean":"2024-09-20T22:09:27.477307+0000","last_became_active":"2024-09-20T22:02:52.482141+0000","last_became_peered":"2024-09-20T22:02:52.482141+0000","last_unstale":"2024-09-20T22:09:27.477307+0000","last_undegraded":"2024-09-20T22:09:27.477307+0000","last_fullsized":"2024-09-20T22:09:27.477307+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:52.834898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:52.834898+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":355,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.489316+0000","last_change":"2024-09-20T22:08:52.423987+0000","last_active":"2024-09-20T22:09:27.489316+0000","last_peered":"2024-09-20T22:09:27.489316+0000","last_clean":"2024-09-20T22:09:27.489316+0000","last_became_active":"2024-09-20T22:08:52.423263+0000","last_became_peered":"2024-09-20T22:08:52.423263+0000","last_unstale":"2024-09-20T22:09:27.489316+0000","last_undegraded":"2024-09-20T22:09:27.489316+0000","last_fullsized":"2024-09-20T22:09:27.489316+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.908616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.908616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"71'8","reported_seq":431,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.474672+0000","last_change":"2024-09-20T22:09:07.902620+0000","last_active":"2024-09-20T22:09:27.474672+0000","last_peered":"2024-09-20T22:09:27.474672+0000","last_clean":"2024-09-20T22:09:27.474672+0000","last_became_active":"2024-09-20T22:02:54.294381+0000","last_became_peered":"2024-09-20T22:02:54.294381+0000","last_unstale":"2024-09-20T22:09:27.474672+0000","last_undegraded":"2024-09-20T22:09:27.474672+0000","last_fullsized":"2024-09-20T22:09:27.474672+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:09:07.902502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:07.902502+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:25.791850+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":339,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.472562+0000","last_change":"2024-09-20T22:08:32.435371+0000","last_active":"2024-09-20T22:09:27.472562+0000","last_peered":"2024-09-20T22:09:27.472562+0000","last_clean":"2024-09-20T22:09:27.472562+0000","last_became_active":"2024-09-20T22:02:56.300041+0000","last_became_peered":"2024-09-20T22:02:56.300041+0000","last_unstale":"2024-09-20T22:09:27.472562+0000","last_undegraded":"2024-09-20T22:09:27.472562+0000","last_fullsized":"2024-09-20T22:09:27.472562+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:32.435297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.435297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:53.430793+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"225'7231","reported_seq":12796,"reported_epoch":225,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T22:09:31.231175+0000","last_change":"2024-09-20T22:09:12.514001+0000","last_active":"2024-09-20T22:09:31.231175+0000","last_peered":"2024-09-20T22:09:31.231175+0000","last_clean":"2024-09-20T22:09:10.539959+0000","last_became_active":"2024-09-20T22:09:12.499931+0000","last_became_peered":"2024-09-20T22:09:12.499931+0000","last_unstale":"2024-09-20T22:09:31.231175+0000","last_undegraded":"2024-09-20T22:09:31.231175+0000","last_fullsized":"2024-09-20T22:09:31.231175+0000","mapping_epoch":218,"log_start":"184'1859","ondisk_log_start":"184'1859","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":5372,"log_dups_size":0,"ondisk_log_size":5372,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":236322816,"num_objects":3606,"num_object_clones":0,"num_object_copies":10818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":876,"num_objects_unfound":0,"num_objects_dirty":3606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3892,"num_write_kb":239936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2723,"num_bytes_recovered":178454528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[3,5,7],"avail_no_missing":["3","5","7"],"object_location_counts":[{"shards":"3,5,7","objects":3606}],"blocked_by":[],"up_primary":4,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"71'10","reported_seq":362,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.474786+0000","last_change":"2024-09-20T22:08:38.681999+0000","last_active":"2024-09-20T22:09:27.474786+0000","last_peered":"2024-09-20T22:09:27.474786+0000","last_clean":"2024-09-20T22:09:27.474786+0000","last_became_active":"2024-09-20T22:06:54.862289+0000","last_became_peered":"2024-09-20T22:06:54.862289+0000","last_unstale":"2024-09-20T22:09:27.474786+0000","last_undegraded":"2024-09-20T22:09:27.474786+0000","last_fullsized":"2024-09-20T22:09:27.474786+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:08:38.681831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:38.681831+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:47.639920+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":351,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.489665+0000","last_change":"2024-09-20T22:08:56.746747+0000","last_active":"2024-09-20T22:09:27.489665+0000","last_peered":"2024-09-20T22:09:27.489665+0000","last_clean":"2024-09-20T22:09:27.489665+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:09:27.489665+0000","last_undegraded":"2024-09-20T22:09:27.489665+0000","last_fullsized":"2024-09-20T22:09:27.489665+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:56.746651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:56.746651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:08.258731+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"146'10","reported_seq":434,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477221+0000","last_change":"2024-09-20T22:07:15.870748+0000","last_active":"2024-09-20T22:09:27.477221+0000","last_peered":"2024-09-20T22:09:27.477221+0000","last_clean":"2024-09-20T22:09:27.477221+0000","last_became_active":"2024-09-20T22:06:54.843097+0000","last_became_peered":"2024-09-20T22:06:54.843097+0000","last_unstale":"2024-09-20T22:09:27.477221+0000","last_undegraded":"2024-09-20T22:09:27.477221+0000","last_fullsized":"2024-09-20T22:09:27.477221+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:07:15.870646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.870646+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":343,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.484353+0000","last_change":"2024-09-20T22:08:52.424184+0000","last_active":"2024-09-20T22:09:27.484353+0000","last_peered":"2024-09-20T22:09:27.484353+0000","last_clean":"2024-09-20T22:09:27.484353+0000","last_became_active":"2024-09-20T22:08:52.422795+0000","last_became_peered":"2024-09-20T22:08:52.422795+0000","last_unstale":"2024-09-20T22:09:27.484353+0000","last_undegraded":"2024-09-20T22:09:27.484353+0000","last_fullsized":"2024-09-20T22:09:27.484353+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:35.824973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:35.824973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:43.837757+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"71'11","reported_seq":385,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.472094+0000","last_change":"2024-09-20T22:08:22.159810+0000","last_active":"2024-09-20T22:09:27.472094+0000","last_peered":"2024-09-20T22:09:27.472094+0000","last_clean":"2024-09-20T22:09:27.472094+0000","last_became_active":"2024-09-20T22:08:22.139581+0000","last_became_peered":"2024-09-20T22:08:22.139581+0000","last_unstale":"2024-09-20T22:09:27.472094+0000","last_undegraded":"2024-09-20T22:09:27.472094+0000","last_fullsized":"2024-09-20T22:09:27.472094+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:15.544705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.544705+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":351,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.473094+0000","last_change":"2024-09-20T22:08:29.474891+0000","last_active":"2024-09-20T22:09:27.473094+0000","last_peered":"2024-09-20T22:09:27.473094+0000","last_clean":"2024-09-20T22:09:27.473094+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:09:27.473094+0000","last_undegraded":"2024-09-20T22:09:27.473094+0000","last_fullsized":"2024-09-20T22:09:27.473094+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:29.474781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:29.474781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:48.958420+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":188,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.486119+0000","last_change":"2024-09-20T22:08:39.327541+0000","last_active":"2024-09-20T22:09:27.486119+0000","last_peered":"2024-09-20T22:09:27.486119+0000","last_clean":"2024-09-20T22:09:27.486119+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:09:27.486119+0000","last_undegraded":"2024-09-20T22:09:27.486119+0000","last_fullsized":"2024-09-20T22:09:27.486119+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:39.326984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:39.326984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.317069+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":167,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.486421+0000","last_change":"2024-09-20T22:06:55.835713+0000","last_active":"2024-09-20T22:09:27.486421+0000","last_peered":"2024-09-20T22:09:27.486421+0000","last_clean":"2024-09-20T22:09:27.486421+0000","last_became_active":"2024-09-20T22:06:55.835393+0000","last_became_peered":"2024-09-20T22:06:55.835393+0000","last_unstale":"2024-09-20T22:09:27.486421+0000","last_undegraded":"2024-09-20T22:09:27.486421+0000","last_fullsized":"2024-09-20T22:09:27.486421+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:33.623816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:33.623816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"71'9","reported_seq":252,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.486380+0000","last_change":"2024-09-20T22:06:54.844349+0000","last_active":"2024-09-20T22:09:27.486380+0000","last_peered":"2024-09-20T22:09:27.486380+0000","last_clean":"2024-09-20T22:09:27.486380+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:09:27.486380+0000","last_undegraded":"2024-09-20T22:09:27.486380+0000","last_fullsized":"2024-09-20T22:09:27.486380+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:12.196518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:12.196518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":51,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.474157+0000","last_change":"2024-09-20T22:08:52.418861+0000","last_active":"2024-09-20T22:09:27.474157+0000","last_peered":"2024-09-20T22:09:27.474157+0000","last_clean":"2024-09-20T22:09:27.474157+0000","last_became_active":"2024-09-20T22:08:52.418520+0000","last_became_peered":"2024-09-20T22:08:52.418520+0000","last_unstale":"2024-09-20T22:09:27.474157+0000","last_undegraded":"2024-09-20T22:09:27.474157+0000","last_fullsized":"2024-09-20T22:09:27.474157+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:54.685751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:54.685751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":343,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.473346+0000","last_change":"2024-09-20T22:08:34.657517+0000","last_active":"2024-09-20T22:09:27.473346+0000","last_peered":"2024-09-20T22:09:27.473346+0000","last_clean":"2024-09-20T22:09:27.473346+0000","last_became_active":"2024-09-20T22:06:54.844271+0000","last_became_peered":"2024-09-20T22:06:54.844271+0000","last_unstale":"2024-09-20T22:09:27.473346+0000","last_undegraded":"2024-09-20T22:09:27.473346+0000","last_fullsized":"2024-09-20T22:09:27.473346+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:34.657278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:34.657278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:56.363742+0000","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":339,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.481431+0000","last_change":"2024-09-20T22:07:23.495005+0000","last_active":"2024-09-20T22:09:27.481431+0000","last_peered":"2024-09-20T22:09:27.481431+0000","last_clean":"2024-09-20T22:09:27.481431+0000","last_became_active":"2024-09-20T22:06:54.848842+0000","last_became_peered":"2024-09-20T22:06:54.848842+0000","last_unstale":"2024-09-20T22:09:27.481431+0000","last_undegraded":"2024-09-20T22:09:27.481431+0000","last_fullsized":"2024-09-20T22:09:27.481431+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.494905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.494905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"71'11","reported_seq":382,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.473116+0000","last_change":"2024-09-20T22:08:52.431501+0000","last_active":"2024-09-20T22:09:27.473116+0000","last_peered":"2024-09-20T22:09:27.473116+0000","last_clean":"2024-09-20T22:09:27.473116+0000","last_became_active":"2024-09-20T22:08:52.417107+0000","last_became_peered":"2024-09-20T22:08:52.417107+0000","last_unstale":"2024-09-20T22:09:27.473116+0000","last_undegraded":"2024-09-20T22:09:27.473116+0000","last_fullsized":"2024-09-20T22:09:27.473116+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:06:36.942620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:06:36.942620+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":359,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477623+0000","last_change":"2024-09-20T22:07:50.851948+0000","last_active":"2024-09-20T22:09:27.477623+0000","last_peered":"2024-09-20T22:09:27.477623+0000","last_clean":"2024-09-20T22:09:27.477623+0000","last_became_active":"2024-09-20T22:06:54.828826+0000","last_became_peered":"2024-09-20T22:06:54.828826+0000","last_unstale":"2024-09-20T22:09:27.477623+0000","last_undegraded":"2024-09-20T22:09:27.477623+0000","last_fullsized":"2024-09-20T22:09:27.477623+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:50.851839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:50.851839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":351,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.489339+0000","last_change":"2024-09-20T22:09:01.751444+0000","last_active":"2024-09-20T22:09:27.489339+0000","last_peered":"2024-09-20T22:09:27.489339+0000","last_clean":"2024-09-20T22:09:27.489339+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:09:27.489339+0000","last_undegraded":"2024-09-20T22:09:27.489339+0000","last_fullsized":"2024-09-20T22:09:27.489339+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:01.751321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:01.751321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:12.987039+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":346,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.473515+0000","last_change":"2024-09-20T22:09:10.873782+0000","last_active":"2024-09-20T22:09:27.473515+0000","last_peered":"2024-09-20T22:09:27.473515+0000","last_clean":"2024-09-20T22:09:27.473515+0000","last_became_active":"2024-09-20T22:02:56.347637+0000","last_became_peered":"2024-09-20T22:02:56.347637+0000","last_unstale":"2024-09-20T22:09:27.473515+0000","last_undegraded":"2024-09-20T22:09:27.473515+0000","last_fullsized":"2024-09-20T22:09:27.473515+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:10.873660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:10.873660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:40.635513+0000","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"71'4","reported_seq":353,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477623+0000","last_change":"2024-09-20T22:07:19.309318+0000","last_active":"2024-09-20T22:09:27.477623+0000","last_peered":"2024-09-20T22:09:27.477623+0000","last_clean":"2024-09-20T22:09:27.477623+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:09:27.477623+0000","last_undegraded":"2024-09-20T22:09:27.477623+0000","last_fullsized":"2024-09-20T22:09:27.477623+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'4","last_scrub_stamp":"2024-09-20T22:07:19.309187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.309187+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":350,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.484299+0000","last_change":"2024-09-20T22:08:29.852608+0000","last_active":"2024-09-20T22:09:27.484299+0000","last_peered":"2024-09-20T22:09:27.484299+0000","last_clean":"2024-09-20T22:09:27.484299+0000","last_became_active":"2024-09-20T22:02:50.111142+0000","last_became_peered":"2024-09-20T22:02:50.111142+0000","last_unstale":"2024-09-20T22:09:27.484299+0000","last_undegraded":"2024-09-20T22:09:27.484299+0000","last_fullsized":"2024-09-20T22:09:27.484299+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:29.852502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:29.852502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:45.967665+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":366,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477374+0000","last_change":"2024-09-20T22:09:01.395413+0000","last_active":"2024-09-20T22:09:27.477374+0000","last_peered":"2024-09-20T22:09:27.477374+0000","last_clean":"2024-09-20T22:09:27.477374+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:09:27.477374+0000","last_undegraded":"2024-09-20T22:09:27.477374+0000","last_fullsized":"2024-09-20T22:09:27.477374+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:01.395328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:01.395328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:24.061199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":339,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.489476+0000","last_change":"2024-09-20T22:08:39.772520+0000","last_active":"2024-09-20T22:09:27.489476+0000","last_peered":"2024-09-20T22:09:27.489476+0000","last_clean":"2024-09-20T22:09:27.489476+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:09:27.489476+0000","last_undegraded":"2024-09-20T22:09:27.489476+0000","last_fullsized":"2024-09-20T22:09:27.489476+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:39.772379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:39.772379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:55.873541+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.13","version":"225'7106","reported_seq":12180,"reported_epoch":225,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T22:09:27.795188+0000","last_change":"2024-09-20T22:09:12.486510+0000","last_active":"2024-09-20T22:09:27.795188+0000","last_peered":"2024-09-20T22:09:27.795188+0000","last_clean":"2024-09-20T22:09:10.553921+0000","last_became_active":"2024-09-20T22:09:12.477947+0000","last_became_peered":"2024-09-20T22:09:12.477947+0000","last_unstale":"2024-09-20T22:09:27.795188+0000","last_undegraded":"2024-09-20T22:09:27.795188+0000","last_fullsized":"2024-09-20T22:09:27.795188+0000","mapping_epoch":218,"log_start":"185'2186","ondisk_log_start":"185'2186","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4920,"log_dups_size":0,"ondisk_log_size":4920,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":246284288,"num_objects":3758,"num_object_clones":0,"num_object_copies":11274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":4070,"num_objects_unfound":0,"num_objects_dirty":3758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3800,"num_write_kb":241856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1651,"num_bytes_recovered":108199936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[1,5,6],"avail_no_missing":["1","5","6"],"object_location_counts":[{"shards":"1,5,6","objects":3758}],"blocked_by":[],"up_primary":3,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":229,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.478320+0000","last_change":"2024-09-20T22:06:54.828772+0000","last_active":"2024-09-20T22:09:27.478320+0000","last_peered":"2024-09-20T22:09:27.478320+0000","last_clean":"2024-09-20T22:09:27.478320+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:09:27.478320+0000","last_undegraded":"2024-09-20T22:09:27.478320+0000","last_fullsized":"2024-09-20T22:09:27.478320+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:15.303300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:15.303300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":351,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.477957+0000","last_change":"2024-09-20T22:07:57.428285+0000","last_active":"2024-09-20T22:09:27.477957+0000","last_peered":"2024-09-20T22:09:27.477957+0000","last_clean":"2024-09-20T22:09:27.477957+0000","last_became_active":"2024-09-20T22:06:54.848032+0000","last_became_peered":"2024-09-20T22:06:54.848032+0000","last_unstale":"2024-09-20T22:09:27.477957+0000","last_undegraded":"2024-09-20T22:09:27.477957+0000","last_fullsized":"2024-09-20T22:09:27.477957+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:57.428165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:57.428165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"71'11","reported_seq":375,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.478933+0000","last_change":"2024-09-20T22:08:32.401623+0000","last_active":"2024-09-20T22:09:27.478933+0000","last_peered":"2024-09-20T22:09:27.478933+0000","last_clean":"2024-09-20T22:09:27.478933+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:09:27.478933+0000","last_undegraded":"2024-09-20T22:09:27.478933+0000","last_fullsized":"2024-09-20T22:09:27.478933+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:08:32.401501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.401501+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:50.388509+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":343,"reported_epoch":225,"state":"active+clean","last_fresh":"2024-09-20T22:09:27.484624+0000","last_change":"2024-09-20T22:08:32.854748+0000","last_active":"2024-09-20T22:09:27.484624+0000","last_peered":"2024-09-20T22:09:27.484624+0000","last_clean":"2024-09-20T22:09:27.484624+0000","last_became_active":"2024-09-20T22:02:54.289078+0000","last_became_peered":"2024-09-20T22:02:54.289078+0000","last_unstale":"2024-09-20T22:09:27.484624+0000","last_undegraded":"2024-09-20T22:09:27.484624+0000","last_fullsized":"2024-09-20T22:09:27.484624+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:32.854221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.854221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:09:55.721860+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":26,"stat_sum":{"num_bytes":7641923608,"num_objects":116608,"num_object_clones":0,"num_object_copies":349824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":49458,"num_objects_unfound":0,"num_objects_dirty":116608,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":110070,"num_write_kb":6827873,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19012,"num_bytes_recovered":1245970432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24469188608,"data_stored":24469176392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132782,"ondisk_log_size":132782,"up":78,"acting":78,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":402,"ondisk_log_size":402,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":387,"num_read_kb":382,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":596028,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":90,"num_read_kb":133,"num_write":82,"num_write_kb":1097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1067735,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1408,"ondisk_log_size":1408,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135956,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2886616,"kb_used_data":2580548,"kb_used_omap":1,"kb_used_meta":306046,"kb_avail":90858536,"statfs":{"total":95995035648,"available":93039140864,"internally_reserved":0,"allocated":2642481152,"data_stored":2641536254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":313391562},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430296,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2509536,"kb_used_data":2217360,"kb_used_omap":1,"kb_used_meta":292158,"kb_avail":91235616,"statfs":{"total":95995035648,"available":93425270784,"internally_reserved":0,"allocated":2270576640,"data_stored":2269620090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":299170246},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724636,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4458740,"kb_used_data":4142948,"kb_used_omap":1,"kb_used_meta":315774,"kb_avail":89286412,"statfs":{"total":95995035648,"available":91429285888,"internally_reserved":0,"allocated":4242378752,"data_stored":4241402966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":323353029},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986276,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3162044,"kb_used_data":2855980,"kb_used_omap":1,"kb_used_meta":306046,"kb_avail":90583108,"statfs":{"total":95995035648,"available":92757102592,"internally_reserved":0,"allocated":2924523520,"data_stored":2923538443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":313391557},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770257952,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3520928,"kb_used_data":3345484,"kb_used_omap":4,"kb_used_meta":175419,"kb_avail":90224224,"statfs":{"total":95995035648,"available":92389605376,"internally_reserved":0,"allocated":3425775616,"data_stored":3424758387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":179629736},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149801,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3283276,"kb_used_data":2904956,"kb_used_omap":1,"kb_used_meta":378238,"kb_avail":90461876,"statfs":{"total":95995035648,"available":92632961024,"internally_reserved":0,"allocated":2974674944,"data_stored":2973675209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":387316167},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411438,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3368360,"kb_used_data":3034520,"kb_used_omap":1,"kb_used_meta":333822,"kb_avail":90376792,"statfs":{"total":95995035648,"available":92545835008,"internally_reserved":0,"allocated":3107348480,"data_stored":3106383778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":341834181},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3281844,"kb_used_data":2885540,"kb_used_omap":1,"kb_used_meta":396286,"kb_avail":90463308,"statfs":{"total":95995035648,"available":92634427392,"internally_reserved":0,"allocated":2954792960,"data_stored":2953816154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":405797317},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2943877120,"data_stored":2943877120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3423469568,"data_stored":3423469568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3096645632,"data_stored":3096641560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2968584192,"data_stored":2968584192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2921791488,"data_stored":2921791488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4227076096,"data_stored":4227072024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2256797696,"data_stored":2256797696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2630946816,"data_stored":2630942744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:09:35.153 INFO:tasks.thrashosds.thrasher:Setting 6.13 to [0, 7, 6] 2024-09-20T22:09:35.154 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '6.13', '0', '7', '6'] 2024-09-20T22:09:35.154 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pg-upmap 6.13 0 7 6 2024-09-20T22:09:35.166 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:35.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:35.167 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:35.199 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:35.268 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:35.295 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:35.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:35.296 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:35.314 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:09:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:09:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:09:35.398 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:35.450 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:35.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:35.451 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:35.552 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:35.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:35.595 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:35.596 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:35.662 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:35 smithi060 ceph-mon[29694]: from='client.25211 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:09:35.662 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:35 smithi060 ceph-mon[29694]: pgmap v343: 168 pgs: 6 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 158 active+clean; 7.0 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 807 B/s rd, 21 MiB/s wr, 950 op/s; 46710/346581 objects misplaced (13.477%); 50 MiB/s, 800 objects/s recovering 2024-09-20T22:09:35.662 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:35 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:09:35.662 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:35 smithi060 ceph-mon[40290]: from='client.25211 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:09:35.662 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:35 smithi060 ceph-mon[40290]: pgmap v343: 168 pgs: 6 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 158 active+clean; 7.0 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 807 B/s rd, 21 MiB/s wr, 950 op/s; 46710/346581 objects misplaced (13.477%); 50 MiB/s, 800 objects/s recovering 2024-09-20T22:09:35.662 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:35 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:09:35.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:35.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:35 smithi067 ceph-mon[33511]: from='client.25211 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:09:35.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:35 smithi067 ceph-mon[33511]: pgmap v343: 168 pgs: 6 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 158 active+clean; 7.0 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 807 B/s rd, 21 MiB/s wr, 950 op/s; 46710/346581 objects misplaced (13.477%); 50 MiB/s, 800 objects/s recovering 2024-09-20T22:09:35.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:35 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:09:35.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:35.786 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:35.786 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:35.887 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:35.921 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:35.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:35.922 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:36.023 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:36.060 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:36.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:36.061 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:36.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:09:36.162 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:36.193 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:36.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:36.195 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:36.295 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:36.357 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:36.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:36.358 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:36.360 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:09:36.360 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:09:36.459 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:36.514 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:36.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:36.515 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:36.616 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:36.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:36.652 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:36.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:36.753 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:36.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:36.802 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:36.802 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:36.903 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:36.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:36.941 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:36.941 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:37.042 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:37.081 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:37.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:37.082 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:37.128 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:37.183 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:37.204 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T22:09:37.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:37.252 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:37.253 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:37.353 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:37.384 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:37.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:37.385 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:37.485 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:37.542 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:37.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:37.543 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:37.644 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:37.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:37.677 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:37.677 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:37.732 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:09:37.763 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:37 smithi060 ceph-mon[40290]: pgmap v344: 168 pgs: 6 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 158 active+clean; 6.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.3 KiB/s rd, 11 MiB/s wr, 905 op/s; 46027/341610 objects misplaced (13.474%); 38 MiB/s, 614 objects/s recovering 2024-09-20T22:09:37.763 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:37 smithi060 ceph-mon[40290]: osdmap e227: 8 total, 8 up, 8 in 2024-09-20T22:09:37.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:37 smithi060 ceph-mon[29694]: pgmap v344: 168 pgs: 6 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 158 active+clean; 6.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.3 KiB/s rd, 11 MiB/s wr, 905 op/s; 46027/341610 objects misplaced (13.474%); 38 MiB/s, 614 objects/s recovering 2024-09-20T22:09:37.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:37 smithi060 ceph-mon[29694]: osdmap e227: 8 total, 8 up, 8 in 2024-09-20T22:09:37.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:37 smithi067 ceph-mon[33511]: pgmap v344: 168 pgs: 6 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 158 active+clean; 6.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.3 KiB/s rd, 11 MiB/s wr, 905 op/s; 46027/341610 objects misplaced (13.474%); 38 MiB/s, 614 objects/s recovering 2024-09-20T22:09:37.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:37 smithi067 ceph-mon[33511]: osdmap e227: 8 total, 8 up, 8 in 2024-09-20T22:09:37.773 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:37.779 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:37.804 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:37.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:37.805 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:37.906 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:37.941 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:37.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:37.942 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:38.043 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:38.069 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:38.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:38.070 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:38.171 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:38.196 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:38.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:38.197 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:38.298 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:38.330 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:38.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:38.331 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:38.432 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:38.445 INFO:teuthology.orchestra.run.smithi060.stderr:set 6.13 pg_upmap mapping to [0,7,6] 2024-09-20T22:09:38.461 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:38.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:38.462 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:38.562 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:38.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:38.608 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:38.608 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:38.709 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:38.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:38.747 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:38.748 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:38.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:38 smithi067 ceph-mon[33511]: osdmap e228: 8 total, 8 up, 8 in 2024-09-20T22:09:38.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:38 smithi067 ceph-mon[33511]: pgmap v347: 168 pgs: 1 active+clean+remapped, 5 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 158 active+clean; 6.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 851 B/s rd, 0 B/s wr, 1.08k op/s; 39328/330921 objects misplaced (11.884%); 46 MiB/s, 740 objects/s recovering 2024-09-20T22:09:38.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:38 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1150869120' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "6.13", "id": [0, 7, 6]}]: dispatch 2024-09-20T22:09:38.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:38 smithi060 ceph-mon[40290]: osdmap e228: 8 total, 8 up, 8 in 2024-09-20T22:09:38.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:38 smithi060 ceph-mon[40290]: pgmap v347: 168 pgs: 1 active+clean+remapped, 5 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 158 active+clean; 6.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 851 B/s rd, 0 B/s wr, 1.08k op/s; 39328/330921 objects misplaced (11.884%); 46 MiB/s, 740 objects/s recovering 2024-09-20T22:09:38.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:38 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1150869120' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "6.13", "id": [0, 7, 6]}]: dispatch 2024-09-20T22:09:38.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:38 smithi060 ceph-mon[29694]: osdmap e228: 8 total, 8 up, 8 in 2024-09-20T22:09:38.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:38 smithi060 ceph-mon[29694]: pgmap v347: 168 pgs: 1 active+clean+remapped, 5 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 158 active+clean; 6.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 851 B/s rd, 0 B/s wr, 1.08k op/s; 39328/330921 objects misplaced (11.884%); 46 MiB/s, 740 objects/s recovering 2024-09-20T22:09:38.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:38 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1150869120' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "6.13", "id": [0, 7, 6]}]: dispatch 2024-09-20T22:09:38.849 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:38.881 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:38.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:38.882 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:38.983 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:39.005 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:39.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:39.006 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:39.106 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:39.143 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:39.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:39.144 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:39.245 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:39.285 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:39.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:39.286 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:39.386 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:39.418 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:39.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:39.419 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:39.520 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:39.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:39.557 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:39.557 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:39.609 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:09:39.657 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:39.695 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:39.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:39.696 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:39.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:39 smithi067 ceph-mon[33511]: 1.1 scrub starts 2024-09-20T22:09:39.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:39 smithi067 ceph-mon[33511]: 1.1 scrub ok 2024-09-20T22:09:39.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:39 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1150869120' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "6.13", "id": [0, 7, 6]}]': finished 2024-09-20T22:09:39.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:39 smithi067 ceph-mon[33511]: osdmap e229: 8 total, 8 up, 8 in 2024-09-20T22:09:39.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:39 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3990436243' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:09:39.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:39 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:09:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:39 smithi060 ceph-mon[29694]: 1.1 scrub starts 2024-09-20T22:09:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:39 smithi060 ceph-mon[29694]: 1.1 scrub ok 2024-09-20T22:09:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:39 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1150869120' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "6.13", "id": [0, 7, 6]}]': finished 2024-09-20T22:09:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:39 smithi060 ceph-mon[29694]: osdmap e229: 8 total, 8 up, 8 in 2024-09-20T22:09:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:39 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3990436243' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:09:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:39 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:09:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:39 smithi060 ceph-mon[40290]: 1.1 scrub starts 2024-09-20T22:09:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:39 smithi060 ceph-mon[40290]: 1.1 scrub ok 2024-09-20T22:09:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:39 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1150869120' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "6.13", "id": [0, 7, 6]}]': finished 2024-09-20T22:09:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:39 smithi060 ceph-mon[40290]: osdmap e229: 8 total, 8 up, 8 in 2024-09-20T22:09:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:39 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3990436243' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:09:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:39 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:09:39.797 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:39.834 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:39.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:39.835 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:39.936 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:39.958 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:39.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:39.959 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:40.060 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:40.095 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:40.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:40.096 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:40.197 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:40.238 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:40.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:40.239 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:40.340 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:40.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:40.384 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:40.384 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:40.485 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:40.508 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:40.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:40.510 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:40.610 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:40.641 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:40.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:40.642 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:40.743 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:40 smithi067 ceph-mon[33511]: 1.a scrub starts 2024-09-20T22:09:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:40 smithi067 ceph-mon[33511]: 1.a scrub ok 2024-09-20T22:09:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:40 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:09:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:40 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:09:40.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:40 smithi067 ceph-mon[33511]: osdmap e230: 8 total, 8 up, 8 in 2024-09-20T22:09:40.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:40 smithi067 ceph-mon[33511]: pgmap v350: 168 pgs: 1 active+clean+remapped, 5 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 158 active+clean; 6.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 0 B/s wr, 890 op/s; 39328/330921 objects misplaced (11.884%); 46 MiB/s, 741 objects/s recovering 2024-09-20T22:09:40.775 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:40.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:40.776 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:40.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:40 smithi060 ceph-mon[40290]: 1.a scrub starts 2024-09-20T22:09:40.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:40 smithi060 ceph-mon[40290]: 1.a scrub ok 2024-09-20T22:09:40.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:40 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:09:40.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:40 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:09:40.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:40 smithi060 ceph-mon[40290]: osdmap e230: 8 total, 8 up, 8 in 2024-09-20T22:09:40.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:40 smithi060 ceph-mon[40290]: pgmap v350: 168 pgs: 1 active+clean+remapped, 5 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 158 active+clean; 6.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 0 B/s wr, 890 op/s; 39328/330921 objects misplaced (11.884%); 46 MiB/s, 741 objects/s recovering 2024-09-20T22:09:40.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:40 smithi060 ceph-mon[29694]: 1.a scrub starts 2024-09-20T22:09:40.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:40 smithi060 ceph-mon[29694]: 1.a scrub ok 2024-09-20T22:09:40.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:40 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:09:40.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:40 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:09:40.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:40 smithi060 ceph-mon[29694]: osdmap e230: 8 total, 8 up, 8 in 2024-09-20T22:09:40.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:40 smithi060 ceph-mon[29694]: pgmap v350: 168 pgs: 1 active+clean+remapped, 5 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 158 active+clean; 6.7 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 0 B/s wr, 890 op/s; 39328/330921 objects misplaced (11.884%); 46 MiB/s, 741 objects/s recovering 2024-09-20T22:09:40.876 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:40.916 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:40.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:40.917 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:41.018 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:41.040 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:41.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:41.041 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:41.142 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:41.186 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:41.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:41.187 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:41.288 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:41.324 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:41.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:41.325 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:41.425 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:41.463 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:41.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:41.464 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:41.564 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:41.595 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:41.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:41.596 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:41.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:41.736 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:41.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:41.737 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:41 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:09:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:41 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:09:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:41 smithi060 ceph-mon[29694]: 1.8 scrub starts 2024-09-20T22:09:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:41 smithi060 ceph-mon[29694]: 1.8 scrub ok 2024-09-20T22:09:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:41 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:09:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:41 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:09:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:41 smithi060 ceph-mon[29694]: osdmap e231: 8 total, 8 up, 8 in 2024-09-20T22:09:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:41 smithi060 ceph-mon[29694]: osdmap e232: 8 total, 8 up, 8 in 2024-09-20T22:09:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:41 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:09:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:41 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:09:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:41 smithi060 ceph-mon[40290]: 1.8 scrub starts 2024-09-20T22:09:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:41 smithi060 ceph-mon[40290]: 1.8 scrub ok 2024-09-20T22:09:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:41 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:09:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:41 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:09:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:41 smithi060 ceph-mon[40290]: osdmap e231: 8 total, 8 up, 8 in 2024-09-20T22:09:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:41 smithi060 ceph-mon[40290]: osdmap e232: 8 total, 8 up, 8 in 2024-09-20T22:09:41.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:41.874 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:41.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:41.875 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:41.976 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:42.000 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:42.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:42.001 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:41 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:09:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:41 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:09:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:41 smithi067 ceph-mon[33511]: 1.8 scrub starts 2024-09-20T22:09:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:41 smithi067 ceph-mon[33511]: 1.8 scrub ok 2024-09-20T22:09:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:41 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:09:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:41 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:09:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:41 smithi067 ceph-mon[33511]: osdmap e231: 8 total, 8 up, 8 in 2024-09-20T22:09:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:41 smithi067 ceph-mon[33511]: osdmap e232: 8 total, 8 up, 8 in 2024-09-20T22:09:42.102 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:42.135 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:42.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:42.136 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:42.237 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:42.259 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:42.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:42.259 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:42.360 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:42.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:42.393 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:42.394 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:42.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:42.513 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:09:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:09:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:42.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:42.541 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:42.641 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:42.684 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:42.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:42.685 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:42.785 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:42.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:42 smithi060 ceph-mon[29694]: 8.8 scrub starts 2024-09-20T22:09:42.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:42 smithi060 ceph-mon[29694]: pgmap v353: 168 pgs: 4 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 160 active+clean; 6.5 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 0 B/s wr, 789 op/s; 35441/321297 objects misplaced (11.031%); 23 MiB/s, 375 objects/s recovering 2024-09-20T22:09:42.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:42 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:09:42.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:42 smithi060 ceph-mon[29694]: osdmap e233: 8 total, 8 up, 8 in 2024-09-20T22:09:42.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:42 smithi060 ceph-mon[40290]: 8.8 scrub starts 2024-09-20T22:09:42.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:42 smithi060 ceph-mon[40290]: pgmap v353: 168 pgs: 4 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 160 active+clean; 6.5 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 0 B/s wr, 789 op/s; 35441/321297 objects misplaced (11.031%); 23 MiB/s, 375 objects/s recovering 2024-09-20T22:09:42.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:42 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:09:42.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:42 smithi060 ceph-mon[40290]: osdmap e233: 8 total, 8 up, 8 in 2024-09-20T22:09:42.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:42.830 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:42.830 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:42.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:42.957 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:42.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:42.958 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:42 smithi067 ceph-mon[33511]: 8.8 scrub starts 2024-09-20T22:09:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:42 smithi067 ceph-mon[33511]: pgmap v353: 168 pgs: 4 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 160 active+clean; 6.5 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 0 B/s wr, 789 op/s; 35441/321297 objects misplaced (11.031%); 23 MiB/s, 375 objects/s recovering 2024-09-20T22:09:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:42 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:09:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:42 smithi067 ceph-mon[33511]: osdmap e233: 8 total, 8 up, 8 in 2024-09-20T22:09:43.058 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:43.095 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:43.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:43.096 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:43.197 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:43.218 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:43.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:43.219 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:43.320 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:43.354 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:43.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:43.356 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:43.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:43.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:43.479 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:43.479 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:43.580 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:43.612 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:43.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:43.614 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:43.714 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:43.749 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:43.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:43.750 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:43.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:43 smithi060 ceph-mon[29694]: 8.8 scrub ok 2024-09-20T22:09:43.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:43 smithi060 ceph-mon[29694]: 8.4 scrub starts 2024-09-20T22:09:43.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:43 smithi060 ceph-mon[40290]: 8.8 scrub ok 2024-09-20T22:09:43.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:43 smithi060 ceph-mon[40290]: 8.4 scrub starts 2024-09-20T22:09:43.851 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:43.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:43.882 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:43.883 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:43.983 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:43 smithi067 ceph-mon[33511]: 8.8 scrub ok 2024-09-20T22:09:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:43 smithi067 ceph-mon[33511]: 8.4 scrub starts 2024-09-20T22:09:44.014 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:44.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:44.015 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:44.116 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:44.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:44.151 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:44.152 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:44.252 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:44.284 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:44.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:44.285 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:44.385 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:44.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:44.423 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:44.423 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:44.524 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:44.549 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:44.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:44.550 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:44.651 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:44.694 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:44.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:44.695 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:44.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:44 smithi060 ceph-mon[29694]: 8.4 scrub ok 2024-09-20T22:09:44.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:44 smithi060 ceph-mon[29694]: pgmap v355: 168 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 160 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 0 B/s wr, 877 op/s; 33659/318651 objects misplaced (10.563%); 30 MiB/s, 488 objects/s recovering 2024-09-20T22:09:44.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:44 smithi060 ceph-mon[40290]: 8.4 scrub ok 2024-09-20T22:09:44.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:44 smithi060 ceph-mon[40290]: pgmap v355: 168 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 160 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 0 B/s wr, 877 op/s; 33659/318651 objects misplaced (10.563%); 30 MiB/s, 488 objects/s recovering 2024-09-20T22:09:44.796 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:44.834 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:44.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:44.835 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:44.935 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:44.976 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:44.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:44.977 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:44 smithi067 ceph-mon[33511]: 8.4 scrub ok 2024-09-20T22:09:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:44 smithi067 ceph-mon[33511]: pgmap v355: 168 pgs: 5 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 160 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 0 B/s wr, 877 op/s; 33659/318651 objects misplaced (10.563%); 30 MiB/s, 488 objects/s recovering 2024-09-20T22:09:45.078 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:45.100 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:45.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:45.101 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:45.202 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:45.243 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:45.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:45.244 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:45.345 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:45.381 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:45.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:45.382 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:45.482 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:45.512 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:45.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:45.513 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:45.531 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:09:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:09:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:09:45.614 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:45.648 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:45.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:45.649 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:45.750 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:45.782 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:45 smithi060 ceph-mon[40290]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:09:45.782 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:45 smithi060 ceph-mon[29694]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:09:45.788 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:45.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:45.790 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:45.890 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:45.922 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:45.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:45.923 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:45 smithi067 ceph-mon[33511]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:09:46.024 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:46.052 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:46.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:46.053 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:46.154 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:46.188 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:46.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:46.189 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:46.290 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:46.320 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:46.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:46.321 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:46.422 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:46.455 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:46.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:46.456 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:46.557 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:46.589 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:46.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:46.590 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:46.691 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:46.721 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:46.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:46.722 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:46.823 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:46.850 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:46.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:46.851 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:46.952 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:46.980 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:46.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:46.981 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:46 smithi067 ceph-mon[33511]: osdmap e234: 8 total, 8 up, 8 in 2024-09-20T22:09:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:46 smithi067 ceph-mon[33511]: pgmap v357: 168 pgs: 1 peering, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 161 active+clean; 6.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 31893/313632 objects misplaced (10.169%); 38 MiB/s, 608 objects/s recovering 2024-09-20T22:09:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:46 smithi067 ceph-mon[33511]: osdmap e235: 8 total, 8 up, 8 in 2024-09-20T22:09:47.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:46 smithi060 ceph-mon[29694]: osdmap e234: 8 total, 8 up, 8 in 2024-09-20T22:09:47.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:46 smithi060 ceph-mon[29694]: pgmap v357: 168 pgs: 1 peering, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 161 active+clean; 6.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 31893/313632 objects misplaced (10.169%); 38 MiB/s, 608 objects/s recovering 2024-09-20T22:09:47.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:46 smithi060 ceph-mon[29694]: osdmap e235: 8 total, 8 up, 8 in 2024-09-20T22:09:47.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:46 smithi060 ceph-mon[40290]: osdmap e234: 8 total, 8 up, 8 in 2024-09-20T22:09:47.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:46 smithi060 ceph-mon[40290]: pgmap v357: 168 pgs: 1 peering, 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 161 active+clean; 6.4 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 31893/313632 objects misplaced (10.169%); 38 MiB/s, 608 objects/s recovering 2024-09-20T22:09:47.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:46 smithi060 ceph-mon[40290]: osdmap e235: 8 total, 8 up, 8 in 2024-09-20T22:09:47.083 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:47.111 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:47.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:47.112 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:47.212 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:47.238 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:47.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:47.239 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:47.340 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:47.365 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:47.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:47.366 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:47.467 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:47.500 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:47.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:47.501 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:47.602 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:47.623 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:47.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:47.624 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:47.725 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:47.764 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:47.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:47.765 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:47.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:09:47.866 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:47.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:47 smithi060 ceph-mon[29694]: 6.f scrub starts 2024-09-20T22:09:47.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:47 smithi060 ceph-mon[29694]: 6.f scrub ok 2024-09-20T22:09:47.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:47 smithi060 ceph-mon[29694]: 8.14 scrub starts 2024-09-20T22:09:47.870 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:47 smithi060 ceph-mon[29694]: osdmap e236: 8 total, 8 up, 8 in 2024-09-20T22:09:47.870 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:47 smithi060 ceph-mon[40290]: 6.f scrub starts 2024-09-20T22:09:47.871 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:47 smithi060 ceph-mon[40290]: 6.f scrub ok 2024-09-20T22:09:47.871 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:47 smithi060 ceph-mon[40290]: 8.14 scrub starts 2024-09-20T22:09:47.871 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:47 smithi060 ceph-mon[40290]: osdmap e236: 8 total, 8 up, 8 in 2024-09-20T22:09:47.891 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:47.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:47.892 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:47.993 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:47 smithi067 ceph-mon[33511]: 6.f scrub starts 2024-09-20T22:09:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:47 smithi067 ceph-mon[33511]: 6.f scrub ok 2024-09-20T22:09:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:47 smithi067 ceph-mon[33511]: 8.14 scrub starts 2024-09-20T22:09:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:47 smithi067 ceph-mon[33511]: osdmap e236: 8 total, 8 up, 8 in 2024-09-20T22:09:48.023 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:48.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:48.024 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:48.125 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:48.154 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:48.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:48.155 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:48.159 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:48.256 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:48.278 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:48.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:48.279 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:48.379 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:48.425 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:48.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:48.426 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:48.527 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:48.554 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:48.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:48.555 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:48.656 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:48.686 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:48.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:48.687 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:48.788 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:48.811 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:48.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:48.812 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:48.912 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:48.956 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:48.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:48.957 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:49.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:48 smithi067 ceph-mon[33511]: 8.14 scrub ok 2024-09-20T22:09:49.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:48 smithi067 ceph-mon[33511]: 4.1a scrub starts 2024-09-20T22:09:49.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:48 smithi067 ceph-mon[33511]: 4.1a scrub ok 2024-09-20T22:09:49.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:48 smithi067 ceph-mon[33511]: pgmap v360: 168 pgs: 1 peering, 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 161 active+clean; 6.2 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 907 B/s rd, 0 op/s; 27758/306519 objects misplaced (9.056%); 36 MiB/s, 568 objects/s recovering 2024-09-20T22:09:49.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:48 smithi067 ceph-mon[33511]: 8.13 scrub starts 2024-09-20T22:09:49.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:48 smithi060 ceph-mon[29694]: 8.14 scrub ok 2024-09-20T22:09:49.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:48 smithi060 ceph-mon[29694]: 4.1a scrub starts 2024-09-20T22:09:49.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:48 smithi060 ceph-mon[29694]: 4.1a scrub ok 2024-09-20T22:09:49.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:48 smithi060 ceph-mon[29694]: pgmap v360: 168 pgs: 1 peering, 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 161 active+clean; 6.2 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 907 B/s rd, 0 op/s; 27758/306519 objects misplaced (9.056%); 36 MiB/s, 568 objects/s recovering 2024-09-20T22:09:49.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:48 smithi060 ceph-mon[29694]: 8.13 scrub starts 2024-09-20T22:09:49.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:48 smithi060 ceph-mon[40290]: 8.14 scrub ok 2024-09-20T22:09:49.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:48 smithi060 ceph-mon[40290]: 4.1a scrub starts 2024-09-20T22:09:49.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:48 smithi060 ceph-mon[40290]: 4.1a scrub ok 2024-09-20T22:09:49.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:48 smithi060 ceph-mon[40290]: pgmap v360: 168 pgs: 1 peering, 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 161 active+clean; 6.2 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 907 B/s rd, 0 op/s; 27758/306519 objects misplaced (9.056%); 36 MiB/s, 568 objects/s recovering 2024-09-20T22:09:49.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:48 smithi060 ceph-mon[40290]: 8.13 scrub starts 2024-09-20T22:09:49.058 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:49.081 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:49.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:49.082 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:49.183 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:49.213 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:49.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:49.214 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:49.314 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:49.345 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:49.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:49.346 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:49.447 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:49.478 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:49.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:49.479 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:49.579 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:49.613 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:49.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:49.615 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:49.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:49.739 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:49.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:49.740 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:49.841 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:49.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:49.884 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:49.884 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:49.985 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:50.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:49 smithi067 ceph-mon[33511]: 5.6 scrub starts 2024-09-20T22:09:50.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:49 smithi067 ceph-mon[33511]: 5.6 scrub ok 2024-09-20T22:09:50.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:49 smithi067 ceph-mon[33511]: 3.f scrub starts 2024-09-20T22:09:50.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:49 smithi067 ceph-mon[33511]: 3.f scrub ok 2024-09-20T22:09:50.020 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:50.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:50.021 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:50.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:49 smithi060 ceph-mon[40290]: 5.6 scrub starts 2024-09-20T22:09:50.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:49 smithi060 ceph-mon[40290]: 5.6 scrub ok 2024-09-20T22:09:50.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:49 smithi060 ceph-mon[40290]: 3.f scrub starts 2024-09-20T22:09:50.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:49 smithi060 ceph-mon[40290]: 3.f scrub ok 2024-09-20T22:09:50.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:49 smithi060 ceph-mon[29694]: 5.6 scrub starts 2024-09-20T22:09:50.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:49 smithi060 ceph-mon[29694]: 5.6 scrub ok 2024-09-20T22:09:50.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:49 smithi060 ceph-mon[29694]: 3.f scrub starts 2024-09-20T22:09:50.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:49 smithi060 ceph-mon[29694]: 3.f scrub ok 2024-09-20T22:09:50.122 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:50.146 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:50.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:50.146 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:50.247 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:50.291 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:50.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:50.293 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:50.393 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:50.415 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:50.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:50.416 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:50.517 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:50.553 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:50.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:50.555 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:50.655 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:50.692 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:50.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:50.694 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:50.794 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:50.838 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:50.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:50.839 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:50.940 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:50.977 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:50.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:50.978 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:50 smithi067 ceph-mon[33511]: 8.13 scrub ok 2024-09-20T22:09:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:50 smithi067 ceph-mon[33511]: 8.18 scrub starts 2024-09-20T22:09:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:50 smithi067 ceph-mon[33511]: pgmap v361: 168 pgs: 1 peering, 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 161 active+clean; 6.2 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 27758/306519 objects misplaced (9.056%); 26 MiB/s, 409 objects/s recovering 2024-09-20T22:09:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:50 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:09:51.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:50 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:09:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:50 smithi060 ceph-mon[29694]: 8.13 scrub ok 2024-09-20T22:09:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:50 smithi060 ceph-mon[29694]: 8.18 scrub starts 2024-09-20T22:09:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:50 smithi060 ceph-mon[29694]: pgmap v361: 168 pgs: 1 peering, 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 161 active+clean; 6.2 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 27758/306519 objects misplaced (9.056%); 26 MiB/s, 409 objects/s recovering 2024-09-20T22:09:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:50 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:09:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:50 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:09:51.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:50 smithi060 ceph-mon[40290]: 8.13 scrub ok 2024-09-20T22:09:51.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:50 smithi060 ceph-mon[40290]: 8.18 scrub starts 2024-09-20T22:09:51.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:50 smithi060 ceph-mon[40290]: pgmap v361: 168 pgs: 1 peering, 4 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 161 active+clean; 6.2 GiB data, 23 GiB used, 693 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 27758/306519 objects misplaced (9.056%); 26 MiB/s, 409 objects/s recovering 2024-09-20T22:09:51.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:50 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:09:51.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:50 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:09:51.079 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:51.119 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:51.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:51.120 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:51.221 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:51.253 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:51.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:51.254 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:51.355 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:51.376 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:51.381 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:51.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:51.382 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:51.483 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:51.515 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:51.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:51.516 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:51.617 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:51.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:51 smithi060 ceph-mon[29694]: 5.d deep-scrub starts 2024-09-20T22:09:51.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:51 smithi060 ceph-mon[29694]: 5.d deep-scrub ok 2024-09-20T22:09:51.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:51 smithi060 ceph-mon[29694]: 8.18 scrub ok 2024-09-20T22:09:51.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:51 smithi060 ceph-mon[29694]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:09:51.627 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:51 smithi060 ceph-mon[29694]: osdmap e237: 8 total, 8 up, 8 in 2024-09-20T22:09:51.629 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:51 smithi060 ceph-mon[40290]: 5.d deep-scrub starts 2024-09-20T22:09:51.629 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:51 smithi060 ceph-mon[40290]: 5.d deep-scrub ok 2024-09-20T22:09:51.629 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:51 smithi060 ceph-mon[40290]: 8.18 scrub ok 2024-09-20T22:09:51.629 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:51 smithi060 ceph-mon[40290]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:09:51.629 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:51 smithi060 ceph-mon[40290]: osdmap e237: 8 total, 8 up, 8 in 2024-09-20T22:09:51.677 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:51.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:51.678 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:51.779 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:51.826 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:51.827 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:51.827 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:51.928 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:51.974 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:51.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:51.975 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:51 smithi067 ceph-mon[33511]: 5.d deep-scrub starts 2024-09-20T22:09:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:51 smithi067 ceph-mon[33511]: 5.d deep-scrub ok 2024-09-20T22:09:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:51 smithi067 ceph-mon[33511]: 8.18 scrub ok 2024-09-20T22:09:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:51 smithi067 ceph-mon[33511]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:09:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:51 smithi067 ceph-mon[33511]: osdmap e237: 8 total, 8 up, 8 in 2024-09-20T22:09:52.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:52.109 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:09:52.110 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:09:52.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:52.121 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:52.222 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:52.254 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:52.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:52.255 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:52.356 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:52.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:52.386 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:52.386 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:52.487 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:52.513 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:09:52 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:09:52.522 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:52.522 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:52.523 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:52.624 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:52.653 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:52.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:52.654 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:52.743 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:09:52.744 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:09:52.755 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:52.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:52 smithi060 ceph-mon[29694]: 4.f scrub starts 2024-09-20T22:09:52.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:52 smithi060 ceph-mon[29694]: 4.f scrub ok 2024-09-20T22:09:52.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:52 smithi060 ceph-mon[29694]: 6.1e scrub starts 2024-09-20T22:09:52.791 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:52 smithi060 ceph-mon[29694]: 6.1e scrub ok 2024-09-20T22:09:52.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:52 smithi060 ceph-mon[29694]: pgmap v363: 168 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 163 active+clean; 6.0 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 22300/295233 objects misplaced (7.553%) 2024-09-20T22:09:52.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:52 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:09:52.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:52 smithi060 ceph-mon[29694]: 6.1f scrub starts 2024-09-20T22:09:52.792 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:52 smithi060 ceph-mon[29694]: 6.1f scrub ok 2024-09-20T22:09:52.792 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:52 smithi060 ceph-mon[40290]: 4.f scrub starts 2024-09-20T22:09:52.792 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:52 smithi060 ceph-mon[40290]: 4.f scrub ok 2024-09-20T22:09:52.792 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:52 smithi060 ceph-mon[40290]: 6.1e scrub starts 2024-09-20T22:09:52.792 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:52 smithi060 ceph-mon[40290]: 6.1e scrub ok 2024-09-20T22:09:52.792 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:52 smithi060 ceph-mon[40290]: pgmap v363: 168 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 163 active+clean; 6.0 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 22300/295233 objects misplaced (7.553%) 2024-09-20T22:09:52.792 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:52 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:09:52.792 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:52 smithi060 ceph-mon[40290]: 6.1f scrub starts 2024-09-20T22:09:52.793 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:52 smithi060 ceph-mon[40290]: 6.1f scrub ok 2024-09-20T22:09:52.793 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:09:52.793 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:09:52.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:52.801 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:52.801 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:52.824 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:09:52.825 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:09:52.858 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:09:52.858 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:09:52.889 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:09:52.889 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:09:52.902 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:52.924 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:09:52.924 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:09:52.938 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:52.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:52.939 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:52.955 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:09:52.955 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:09:52.993 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:09:52.993 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:09:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:52 smithi067 ceph-mon[33511]: 4.f scrub starts 2024-09-20T22:09:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:52 smithi067 ceph-mon[33511]: 4.f scrub ok 2024-09-20T22:09:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:52 smithi067 ceph-mon[33511]: 6.1e scrub starts 2024-09-20T22:09:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:52 smithi067 ceph-mon[33511]: 6.1e scrub ok 2024-09-20T22:09:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:52 smithi067 ceph-mon[33511]: pgmap v363: 168 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 163 active+clean; 6.0 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 22300/295233 objects misplaced (7.553%) 2024-09-20T22:09:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:52 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:09:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:52 smithi067 ceph-mon[33511]: 6.1f scrub starts 2024-09-20T22:09:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:52 smithi067 ceph-mon[33511]: 6.1f scrub ok 2024-09-20T22:09:53.040 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:53.076 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:53.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:53.077 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:53.179 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:53.212 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:53.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:53.213 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:53.273 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T22:09:53.314 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:53.349 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:53.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:53.350 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:53.448 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:53.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:53.481 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:53.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:53.482 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:53.582 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:53.631 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:53.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:53.631 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:53.732 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:53.770 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:53.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:53.771 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:53.872 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:53.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:53 smithi060 ceph-mon[29694]: 6.14 scrub starts 2024-09-20T22:09:53.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:53 smithi060 ceph-mon[29694]: 6.14 scrub ok 2024-09-20T22:09:53.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:53 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/38262498' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:09:53.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:53 smithi060 ceph-mon[29694]: 3.b deep-scrub starts 2024-09-20T22:09:53.879 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:53 smithi060 ceph-mon[29694]: 3.b deep-scrub ok 2024-09-20T22:09:53.880 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:53 smithi060 ceph-mon[40290]: 6.14 scrub starts 2024-09-20T22:09:53.880 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:53 smithi060 ceph-mon[40290]: 6.14 scrub ok 2024-09-20T22:09:53.880 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:53 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/38262498' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:09:53.880 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:53 smithi060 ceph-mon[40290]: 3.b deep-scrub starts 2024-09-20T22:09:53.880 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:53 smithi060 ceph-mon[40290]: 3.b deep-scrub ok 2024-09-20T22:09:53.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:53.913 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:53.913 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:54.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:53 smithi067 ceph-mon[33511]: 6.14 scrub starts 2024-09-20T22:09:54.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:53 smithi067 ceph-mon[33511]: 6.14 scrub ok 2024-09-20T22:09:54.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:53 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/38262498' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:09:54.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:53 smithi067 ceph-mon[33511]: 3.b deep-scrub starts 2024-09-20T22:09:54.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:53 smithi067 ceph-mon[33511]: 3.b deep-scrub ok 2024-09-20T22:09:54.014 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:54.041 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:54.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:54.042 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:54.143 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:54.172 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:54.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:54.173 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:54.274 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:54.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:54.317 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:54.318 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:54.418 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:54.463 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:54.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:54.464 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:54.555 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:54.564 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:54.594 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:54.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:54.595 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:54.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:54.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:54.747 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:54.747 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:54.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:54 smithi060 ceph-mon[29694]: 6.e scrub starts 2024-09-20T22:09:54.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:54 smithi060 ceph-mon[29694]: 6.e scrub ok 2024-09-20T22:09:54.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:54 smithi060 ceph-mon[29694]: pgmap v364: 168 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 163 active+clean; 5.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 668 B/s rd, 0 op/s; 21058/292128 objects misplaced (7.208%) 2024-09-20T22:09:54.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:54 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:09:54.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:54 smithi060 ceph-mon[40290]: 6.e scrub starts 2024-09-20T22:09:54.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:54 smithi060 ceph-mon[40290]: 6.e scrub ok 2024-09-20T22:09:54.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:54 smithi060 ceph-mon[40290]: pgmap v364: 168 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 163 active+clean; 5.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 668 B/s rd, 0 op/s; 21058/292128 objects misplaced (7.208%) 2024-09-20T22:09:54.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:54 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:09:54.848 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:54.918 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:54.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:54.919 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:55.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:54 smithi067 ceph-mon[33511]: 6.e scrub starts 2024-09-20T22:09:55.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:54 smithi067 ceph-mon[33511]: 6.e scrub ok 2024-09-20T22:09:55.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:54 smithi067 ceph-mon[33511]: pgmap v364: 168 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 163 active+clean; 5.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 668 B/s rd, 0 op/s; 21058/292128 objects misplaced (7.208%) 2024-09-20T22:09:55.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:54 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:09:55.020 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:55.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:55.053 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:55.054 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:55.154 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:55.190 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:55.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:55.191 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:55.219 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:09:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:09:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:09:55.238 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:09:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:09:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:09:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:09:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227635 7.b 7.fcebd15b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:09:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.849145+0000", 2024-09-20T22:09:55.238 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 181.36688122999999, 2024-09-20T22:09:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.958058509, 2024-09-20T22:09:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:09:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:09:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:09:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227635 2024-09-20T22:09:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.239 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849145+0000", 2024-09-20T22:09:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849145+0000", 2024-09-20T22:09:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.240 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849146+0000", 2024-09-20T22:09:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.562e-06 2024-09-20T22:09:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849152+0000", 2024-09-20T22:09:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5999999999999997e-06 2024-09-20T22:09:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849162+0000", 2024-09-20T22:09:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9620000000000005e-06 2024-09-20T22:09:55.241 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849171+0000", 2024-09-20T22:09:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4569999999999994e-06 2024-09-20T22:09:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849336+0000", 2024-09-20T22:09:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016478700000000001 2024-09-20T22:09:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.242 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:09:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849344+0000", 2024-09-20T22:09:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3950000000000001e-06 2024-09-20T22:09:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.807192+0000", 2024-09-20T22:09:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.95784742599999995 2024-09-20T22:09:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.807203+0000", 2024-09-20T22:09:55.243 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.132e-05 2024-09-20T22:09:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227641 7.b 7.8d46094b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:09:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.858720+0000", 2024-09-20T22:09:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 181.35730574199999, 2024-09-20T22:09:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.95058797399999995, 2024-09-20T22:09:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.244 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:09:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:09:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:09:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:09:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227641 2024-09-20T22:09:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858720+0000", 2024-09-20T22:09:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.245 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858720+0000", 2024-09-20T22:09:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858722+0000", 2024-09-20T22:09:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3969999999999999e-06 2024-09-20T22:09:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858726+0000", 2024-09-20T22:09:55.246 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7999999999999998e-06 2024-09-20T22:09:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858737+0000", 2024-09-20T22:09:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0403e-05 2024-09-20T22:09:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858746+0000", 2024-09-20T22:09:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2739999999999996e-06 2024-09-20T22:09:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.247 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858806+0000", 2024-09-20T22:09:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0315000000000001e-05 2024-09-20T22:09:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:09:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858818+0000", 2024-09-20T22:09:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2079000000000001e-05 2024-09-20T22:09:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.809293+0000", 2024-09-20T22:09:55.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.95047507899999994 2024-09-20T22:09:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.809308+0000", 2024-09-20T22:09:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4627000000000001e-05 2024-09-20T22:09:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:31013 8.3 8:c65e4777:::benchmark_data_smithi060_111051_object31012:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e184)", 2024-09-20T22:09:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:08:09.458037+0000", 2024-09-20T22:09:55.249 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 105.757988797, 2024-09-20T22:09:55.250 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.14882263700000001, 2024-09-20T22:09:55.250 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.250 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.250 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:55.250 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:09:55.250 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:09:55.250 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:55.250 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 31013 2024-09-20T22:09:55.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.250 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.251 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.251 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.251 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.458037+0000", 2024-09-20T22:09:55.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.251 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.251 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.251 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.458037+0000", 2024-09-20T22:09:55.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.251 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.251 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.252 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.458039+0000", 2024-09-20T22:09:55.252 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.455e-06 2024-09-20T22:09:55.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.252 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.252 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.252 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.458064+0000", 2024-09-20T22:09:55.252 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5369000000000001e-05 2024-09-20T22:09:55.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.252 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.252 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.252 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.458076+0000", 2024-09-20T22:09:55.252 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2571999999999999e-05 2024-09-20T22:09:55.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.253 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.253 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.458084+0000", 2024-09-20T22:09:55.253 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3880000000000002e-06 2024-09-20T22:09:55.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.253 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.253 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.458150+0000", 2024-09-20T22:09:55.253 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6569999999999995e-05 2024-09-20T22:09:55.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:09:55.254 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.458313+0000", 2024-09-20T22:09:55.254 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016227700000000001 2024-09-20T22:09:55.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.254 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 5,6", 2024-09-20T22:09:55.254 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.458450+0000", 2024-09-20T22:09:55.254 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136942 2024-09-20T22:09:55.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.254 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:55.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.460989+0000", 2024-09-20T22:09:55.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025389499999999999 2024-09-20T22:09:55.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:09:55.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.461205+0000", 2024-09-20T22:09:55.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000216257 2024-09-20T22:09:55.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:55.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.606691+0000", 2024-09-20T22:09:55.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.14548624399999999 2024-09-20T22:09:55.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.256 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.256 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:09:55.256 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.606811+0000", 2024-09-20T22:09:55.256 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011961200000000001 2024-09-20T22:09:55.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.256 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.256 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.256 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.606860+0000", 2024-09-20T22:09:55.256 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9001000000000001e-05 2024-09-20T22:09:55.257 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.257 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.257 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.257 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.257 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:31020 8.a 8:5de616bc:::benchmark_data_smithi060_111051_object31019:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e184)", 2024-09-20T22:09:55.257 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:08:09.459011+0000", 2024-09-20T22:09:55.257 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 105.757014791, 2024-09-20T22:09:55.257 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.14547617700000001, 2024-09-20T22:09:55.257 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.257 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.258 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:55.258 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:09:55.258 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:09:55.258 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:55.258 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 31020 2024-09-20T22:09:55.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.258 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.258 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.258 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.258 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.459011+0000", 2024-09-20T22:09:55.258 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.459011+0000", 2024-09-20T22:09:55.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.459012+0000", 2024-09-20T22:09:55.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8999999999999996e-07 2024-09-20T22:09:55.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.459032+0000", 2024-09-20T22:09:55.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9819000000000001e-05 2024-09-20T22:09:55.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.459040+0000", 2024-09-20T22:09:55.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4719999999999995e-06 2024-09-20T22:09:55.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.459046+0000", 2024-09-20T22:09:55.261 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8830000000000003e-06 2024-09-20T22:09:55.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.261 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.261 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.459477+0000", 2024-09-20T22:09:55.261 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00043077300000000002 2024-09-20T22:09:55.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.261 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:09:55.261 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.459613+0000", 2024-09-20T22:09:55.262 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013607699999999999 2024-09-20T22:09:55.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.262 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.262 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:09:55.262 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.459740+0000", 2024-09-20T22:09:55.262 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012731099999999999 2024-09-20T22:09:55.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.262 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.262 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:09:55.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.461208+0000", 2024-09-20T22:09:55.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0014682969999999999 2024-09-20T22:09:55.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.263 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:55.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.465828+0000", 2024-09-20T22:09:55.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0046195159999999997 2024-09-20T22:09:55.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.263 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:55.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.604293+0000", 2024-09-20T22:09:55.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.13846460099999999 2024-09-20T22:09:55.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:09:55.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.604393+0000", 2024-09-20T22:09:55.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100163 2024-09-20T22:09:55.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:08:09.604487+0000", 2024-09-20T22:09:55.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4574999999999995e-05 2024-09-20T22:09:55.264 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.264 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.264 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.265 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105654 8.1 8:83678158:::benchmark_data_smithi060_111051_object105653:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e214)", 2024-09-20T22:09:55.265 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.527812+0000", 2024-09-20T22:09:55.265 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 46.688213881999999, 2024-09-20T22:09:55.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.17380899499999999, 2024-09-20T22:09:55.265 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.265 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.265 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:09:55.265 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:09:55.265 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:09:55.265 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:55.265 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105654 2024-09-20T22:09:55.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.266 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.266 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.266 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.527812+0000", 2024-09-20T22:09:55.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.266 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.266 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.527812+0000", 2024-09-20T22:09:55.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.527814+0000", 2024-09-20T22:09:55.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.564e-06 2024-09-20T22:09:55.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.527839+0000", 2024-09-20T22:09:55.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5341999999999999e-05 2024-09-20T22:09:55.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.527853+0000", 2024-09-20T22:09:55.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3719e-05 2024-09-20T22:09:55.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.527860+0000", 2024-09-20T22:09:55.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1890000000000001e-06 2024-09-20T22:09:55.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.529439+0000", 2024-09-20T22:09:55.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001579425 2024-09-20T22:09:55.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:09:55.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.529561+0000", 2024-09-20T22:09:55.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012155300000000001 2024-09-20T22:09:55.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:09:55.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.529705+0000", 2024-09-20T22:09:55.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014421900000000001 2024-09-20T22:09:55.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.701564+0000", 2024-09-20T22:09:55.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.171859179 2024-09-20T22:09:55.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.270 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.270 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.701621+0000", 2024-09-20T22:09:55.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6805000000000002e-05 2024-09-20T22:09:55.270 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.270 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.270 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.271 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105657 8.7 8.6bf50607 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e214)", 2024-09-20T22:09:55.271 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.533270+0000", 2024-09-20T22:09:55.271 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 46.682755663000002, 2024-09-20T22:09:55.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.18481678800000001, 2024-09-20T22:09:55.271 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.271 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.271 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:09:55.271 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:09:55.271 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:09:55.271 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:55.271 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105657 2024-09-20T22:09:55.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.272 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533270+0000", 2024-09-20T22:09:55.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533270+0000", 2024-09-20T22:09:55.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533272+0000", 2024-09-20T22:09:55.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.751e-06 2024-09-20T22:09:55.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533297+0000", 2024-09-20T22:09:55.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5105000000000001e-05 2024-09-20T22:09:55.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533424+0000", 2024-09-20T22:09:55.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000127181 2024-09-20T22:09:55.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533434+0000", 2024-09-20T22:09:55.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4229999999999994e-06 2024-09-20T22:09:55.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.718074+0000", 2024-09-20T22:09:55.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.18464032699999999 2024-09-20T22:09:55.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.718087+0000", 2024-09-20T22:09:55.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3001e-05 2024-09-20T22:09:55.275 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.275 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.275 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.276 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.15192.0:105656 8.b e214/176 8:dc90bde5:::benchmark_data_smithi060_111051_object105655:head v 214'6463, mlcod=214'6461)", 2024-09-20T22:09:55.276 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.533332+0000", 2024-09-20T22:09:55.276 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 46.682693395999998, 2024-09-20T22:09:55.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.16766616300000001, 2024-09-20T22:09:55.276 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.276 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.276 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:55.276 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533332+0000", 2024-09-20T22:09:55.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533332+0000", 2024-09-20T22:09:55.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533334+0000", 2024-09-20T22:09:55.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.012e-06 2024-09-20T22:09:55.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533357+0000", 2024-09-20T22:09:55.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3448999999999999e-05 2024-09-20T22:09:55.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.535888+0000", 2024-09-20T22:09:55.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025313219999999999 2024-09-20T22:09:55.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.535901+0000", 2024-09-20T22:09:55.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2496e-05 2024-09-20T22:09:55.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542308+0000", 2024-09-20T22:09:55.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0064067480000000003 2024-09-20T22:09:55.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:09:55.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542330+0000", 2024-09-20T22:09:55.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2860000000000001e-05 2024-09-20T22:09:55.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:09:55.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.700965+0000", 2024-09-20T22:09:55.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15863476900000001 2024-09-20T22:09:55.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.700999+0000", 2024-09-20T22:09:55.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3507000000000003e-05 2024-09-20T22:09:55.280 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.280 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.281 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.281 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105658 8.1 8.93657901 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e214)", 2024-09-20T22:09:55.281 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.533463+0000", 2024-09-20T22:09:55.281 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 46.682562763, 2024-09-20T22:09:55.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.168771484, 2024-09-20T22:09:55.281 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.281 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.281 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:09:55.281 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:09:55.282 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:09:55.282 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:55.282 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105658 2024-09-20T22:09:55.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.282 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533463+0000", 2024-09-20T22:09:55.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533463+0000", 2024-09-20T22:09:55.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533464+0000", 2024-09-20T22:09:55.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2600000000000002e-07 2024-09-20T22:09:55.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533489+0000", 2024-09-20T22:09:55.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4811e-05 2024-09-20T22:09:55.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533498+0000", 2024-09-20T22:09:55.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7019999999999996e-06 2024-09-20T22:09:55.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533505+0000", 2024-09-20T22:09:55.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2369999999999997e-06 2024-09-20T22:09:55.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702223+0000", 2024-09-20T22:09:55.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.16871866599999999 2024-09-20T22:09:55.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702235+0000", 2024-09-20T22:09:55.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1342000000000001e-05 2024-09-20T22:09:55.285 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.286 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.286 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.286 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105659 8.1 8.ff390301 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e214)", 2024-09-20T22:09:55.286 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.538166+0000", 2024-09-20T22:09:55.286 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 46.677859984000001, 2024-09-20T22:09:55.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.16410761300000001, 2024-09-20T22:09:55.286 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.286 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.287 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:09:55.287 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:09:55.287 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:09:55.287 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:55.287 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105659 2024-09-20T22:09:55.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.287 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.538166+0000", 2024-09-20T22:09:55.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.538166+0000", 2024-09-20T22:09:55.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.538167+0000", 2024-09-20T22:09:55.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2979999999999999e-06 2024-09-20T22:09:55.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.538197+0000", 2024-09-20T22:09:55.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9439999999999999e-05 2024-09-20T22:09:55.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.538206+0000", 2024-09-20T22:09:55.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6830000000000003e-06 2024-09-20T22:09:55.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.538215+0000", 2024-09-20T22:09:55.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4829999999999999e-06 2024-09-20T22:09:55.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702263+0000", 2024-09-20T22:09:55.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.16404848899999999 2024-09-20T22:09:55.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702274+0000", 2024-09-20T22:09:55.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.022e-05 2024-09-20T22:09:55.291 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.292 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.292 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.292 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105660 8.3 8.129f9cf3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e214)", 2024-09-20T22:09:55.292 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.538229+0000", 2024-09-20T22:09:55.292 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 46.677797333999997, 2024-09-20T22:09:55.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.16266391899999999, 2024-09-20T22:09:55.292 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.292 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.292 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:09:55.292 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:09:55.293 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:09:55.293 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:55.293 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105660 2024-09-20T22:09:55.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.293 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.538229+0000", 2024-09-20T22:09:55.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.538229+0000", 2024-09-20T22:09:55.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.538229+0000", 2024-09-20T22:09:55.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7e-07 2024-09-20T22:09:55.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.538252+0000", 2024-09-20T22:09:55.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3272e-05 2024-09-20T22:09:55.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.538259+0000", 2024-09-20T22:09:55.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0099999999999998e-06 2024-09-20T22:09:55.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.538264+0000", 2024-09-20T22:09:55.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.916e-06 2024-09-20T22:09:55.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.700876+0000", 2024-09-20T22:09:55.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.162611805 2024-09-20T22:09:55.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.700892+0000", 2024-09-20T22:09:55.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6446000000000001e-05 2024-09-20T22:09:55.296 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.296 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.296 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.297 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.15192.0:105653 8.f e214/176 8:f20d820f:::benchmark_data_smithi060_111051_object105652:head v 214'6689, mlcod=214'6685)", 2024-09-20T22:09:55.297 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.542169+0000", 2024-09-20T22:09:55.297 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 46.673856483000002, 2024-09-20T22:09:55.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.186805894, 2024-09-20T22:09:55.297 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.297 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.297 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:55.297 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542169+0000", 2024-09-20T22:09:55.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542169+0000", 2024-09-20T22:09:55.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542170+0000", 2024-09-20T22:09:55.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7800000000000001e-07 2024-09-20T22:09:55.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.299 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.299 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542205+0000", 2024-09-20T22:09:55.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4589999999999999e-05 2024-09-20T22:09:55.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.299 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.299 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542219+0000", 2024-09-20T22:09:55.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3879999999999999e-05 2024-09-20T22:09:55.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.299 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.299 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542229+0000", 2024-09-20T22:09:55.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0797e-05 2024-09-20T22:09:55.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.300 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.718074+0000", 2024-09-20T22:09:55.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.17584509000000001 2024-09-20T22:09:55.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:09:55.300 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.718102+0000", 2024-09-20T22:09:55.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7668999999999999e-05 2024-09-20T22:09:55.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:09:55.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.728943+0000", 2024-09-20T22:09:55.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.01084106 2024-09-20T22:09:55.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.728975+0000", 2024-09-20T22:09:55.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2129999999999999e-05 2024-09-20T22:09:55.301 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.301 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.302 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.302 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop_reply(client.15192.0:105654 8.1 e214/176)", 2024-09-20T22:09:55.302 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.542958+0000", 2024-09-20T22:09:55.302 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 46.673068305000001, 2024-09-20T22:09:55.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.158737298, 2024-09-20T22:09:55.302 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.302 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.302 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:09:55.302 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542958+0000", 2024-09-20T22:09:55.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542958+0000", 2024-09-20T22:09:55.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.304 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.304 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542958+0000", 2024-09-20T22:09:55.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5700000000000002e-07 2024-09-20T22:09:55.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.304 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.304 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542963+0000", 2024-09-20T22:09:55.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5170000000000001e-06 2024-09-20T22:09:55.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.304 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542973+0000", 2024-09-20T22:09:55.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0276e-05 2024-09-20T22:09:55.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542980+0000", 2024-09-20T22:09:55.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0829999999999998e-06 2024-09-20T22:09:55.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.701603+0000", 2024-09-20T22:09:55.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15862241799999999 2024-09-20T22:09:55.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.701695+0000", 2024-09-20T22:09:55.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2347000000000002e-05 2024-09-20T22:09:55.306 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.306 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.306 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.306 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105654 8.1 8:83678158:::benchmark_data_smithi060_111051_object105653:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:09:55.306 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.549939+0000", 2024-09-20T22:09:55.306 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 46.666087374, 2024-09-20T22:09:55.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.003362782, 2024-09-20T22:09:55.307 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.307 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.307 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:09:55.307 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:09:55.307 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:09:55.307 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:55.307 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105654 2024-09-20T22:09:55.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.307 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549939+0000", 2024-09-20T22:09:55.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549939+0000", 2024-09-20T22:09:55.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549940+0000", 2024-09-20T22:09:55.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.95e-06 2024-09-20T22:09:55.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549973+0000", 2024-09-20T22:09:55.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2798e-05 2024-09-20T22:09:55.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549997+0000", 2024-09-20T22:09:55.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3649e-05 2024-09-20T22:09:55.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550007+0000", 2024-09-20T22:09:55.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8190000000000003e-06 2024-09-20T22:09:55.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702300+0000", 2024-09-20T22:09:55.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.152292975 2024-09-20T22:09:55.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:09:55.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702310+0000", 2024-09-20T22:09:55.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0453e-05 2024-09-20T22:09:55.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.553223+0000", 2024-09-20T22:09:55.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85091275799999999 2024-09-20T22:09:55.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.553301+0000", 2024-09-20T22:09:55.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8380000000000005e-05 2024-09-20T22:09:55.311 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.312 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.312 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.312 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105657 8.7 8:e060afd6:::benchmark_data_smithi060_111051_object105656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:09:55.312 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.550025+0000", 2024-09-20T22:09:55.312 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 46.666000857999997, 2024-09-20T22:09:55.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.029495933, 2024-09-20T22:09:55.312 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.312 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.312 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:55.312 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:09:55.313 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:09:55.313 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:55.313 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105657 2024-09-20T22:09:55.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.313 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550025+0000", 2024-09-20T22:09:55.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.314 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550025+0000", 2024-09-20T22:09:55.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.314 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550026+0000", 2024-09-20T22:09:55.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1799999999999995e-07 2024-09-20T22:09:55.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550051+0000", 2024-09-20T22:09:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5554999999999998e-05 2024-09-20T22:09:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550062+0000", 2024-09-20T22:09:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.137e-05 2024-09-20T22:09:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550068+0000", 2024-09-20T22:09:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8289999999999996e-06 2024-09-20T22:09:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720577+0000", 2024-09-20T22:09:55.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.170508778 2024-09-20T22:09:55.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:09:55.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720588+0000", 2024-09-20T22:09:55.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0808000000000001e-05 2024-09-20T22:09:55.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.569210+0000", 2024-09-20T22:09:55.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.84862162900000004 2024-09-20T22:09:55.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:09:55.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.569360+0000", 2024-09-20T22:09:55.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015044799999999999 2024-09-20T22:09:55.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:09:55.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.569667+0000", 2024-09-20T22:09:55.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030655599999999998 2024-09-20T22:09:55.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:09:55.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.572342+0000", 2024-09-20T22:09:55.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0026757840000000001 2024-09-20T22:09:55.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:55.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.573025+0000", 2024-09-20T22:09:55.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00068279199999999999 2024-09-20T22:09:55.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:55.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579397+0000", 2024-09-20T22:09:55.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.006372071 2024-09-20T22:09:55.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:09:55.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579465+0000", 2024-09-20T22:09:55.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7520000000000004e-05 2024-09-20T22:09:55.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579521+0000", 2024-09-20T22:09:55.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6274999999999999e-05 2024-09-20T22:09:55.319 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.319 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.319 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.320 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105658 8.1 8:809ea6c9:::benchmark_data_smithi060_111051_object105657:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:09:55.320 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.550079+0000", 2024-09-20T22:09:55.320 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 46.665947275000001, 2024-09-20T22:09:55.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.010609463, 2024-09-20T22:09:55.320 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.320 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.320 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:55.320 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:09:55.320 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:09:55.320 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:55.321 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105658 2024-09-20T22:09:55.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.321 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550079+0000", 2024-09-20T22:09:55.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550079+0000", 2024-09-20T22:09:55.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.322 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.322 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550079+0000", 2024-09-20T22:09:55.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2599999999999998e-07 2024-09-20T22:09:55.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.322 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.322 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550101+0000", 2024-09-20T22:09:55.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1628e-05 2024-09-20T22:09:55.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550111+0000", 2024-09-20T22:09:55.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9620000000000005e-06 2024-09-20T22:09:55.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550116+0000", 2024-09-20T22:09:55.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9810000000000003e-06 2024-09-20T22:09:55.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702335+0000", 2024-09-20T22:09:55.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15221938700000001 2024-09-20T22:09:55.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:09:55.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702343+0000", 2024-09-20T22:09:55.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2120000000000003e-06 2024-09-20T22:09:55.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.553649+0000", 2024-09-20T22:09:55.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85130619600000002 2024-09-20T22:09:55.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:09:55.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.553784+0000", 2024-09-20T22:09:55.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000134499 2024-09-20T22:09:55.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:09:55.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.554060+0000", 2024-09-20T22:09:55.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000276579 2024-09-20T22:09:55.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:09:55.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.557338+0000", 2024-09-20T22:09:55.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003277608 2024-09-20T22:09:55.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:55.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.559599+0000", 2024-09-20T22:09:55.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002261304 2024-09-20T22:09:55.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:55.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560617+0000", 2024-09-20T22:09:55.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001017354 2024-09-20T22:09:55.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:09:55.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560660+0000", 2024-09-20T22:09:55.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3167999999999997e-05 2024-09-20T22:09:55.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560688+0000", 2024-09-20T22:09:55.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8158999999999999e-05 2024-09-20T22:09:55.327 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.327 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.327 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.327 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105659 8.1 8:80c09cff:::benchmark_data_smithi060_111051_object105658:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:09:55.328 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.550123+0000", 2024-09-20T22:09:55.328 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 46.665902541000001, 2024-09-20T22:09:55.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0107473760000001, 2024-09-20T22:09:55.328 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.328 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.328 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:55.328 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:09:55.328 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:09:55.328 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:55.328 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105659 2024-09-20T22:09:55.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.328 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550123+0000", 2024-09-20T22:09:55.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550123+0000", 2024-09-20T22:09:55.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550124+0000", 2024-09-20T22:09:55.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.39e-07 2024-09-20T22:09:55.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550148+0000", 2024-09-20T22:09:55.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3944e-05 2024-09-20T22:09:55.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.331 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550157+0000", 2024-09-20T22:09:55.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5410000000000006e-06 2024-09-20T22:09:55.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.331 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550162+0000", 2024-09-20T22:09:55.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8069999999999999e-06 2024-09-20T22:09:55.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702368+0000", 2024-09-20T22:09:55.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15220560899999999 2024-09-20T22:09:55.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:09:55.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702374+0000", 2024-09-20T22:09:55.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7939999999999997e-06 2024-09-20T22:09:55.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.555649+0000", 2024-09-20T22:09:55.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85327487000000002 2024-09-20T22:09:55.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:09:55.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.555819+0000", 2024-09-20T22:09:55.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016938200000000001 2024-09-20T22:09:55.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:09:55.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.556100+0000", 2024-09-20T22:09:55.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00028165699999999999 2024-09-20T22:09:55.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:09:55.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.558925+0000", 2024-09-20T22:09:55.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0028247290000000002 2024-09-20T22:09:55.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:55.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.559791+0000", 2024-09-20T22:09:55.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00086584599999999998 2024-09-20T22:09:55.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:55.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560771+0000", 2024-09-20T22:09:55.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00097968300000000007 2024-09-20T22:09:55.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:09:55.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560821+0000", 2024-09-20T22:09:55.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0460999999999997e-05 2024-09-20T22:09:55.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560871+0000", 2024-09-20T22:09:55.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9614000000000003e-05 2024-09-20T22:09:55.335 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.335 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.335 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.336 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105660 8.13 8:cf39f948:::benchmark_data_smithi060_111051_object105659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:09:55.336 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.550169+0000", 2024-09-20T22:09:55.336 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 46.665856808000001, 2024-09-20T22:09:55.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0659826960000001, 2024-09-20T22:09:55.336 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.336 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.336 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:55.336 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:09:55.336 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:09:55.336 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:55.336 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105660 2024-09-20T22:09:55.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.336 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550169+0000", 2024-09-20T22:09:55.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550169+0000", 2024-09-20T22:09:55.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550169+0000", 2024-09-20T22:09:55.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9099999999999999e-07 2024-09-20T22:09:55.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550221+0000", 2024-09-20T22:09:55.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1844e-05 2024-09-20T22:09:55.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550233+0000", 2024-09-20T22:09:55.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1871e-05 2024-09-20T22:09:55.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550238+0000", 2024-09-20T22:09:55.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6750000000000001e-06 2024-09-20T22:09:55.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.701152+0000", 2024-09-20T22:09:55.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15091443500000001 2024-09-20T22:09:55.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:09:55.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.701161+0000", 2024-09-20T22:09:55.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2679999999999994e-06 2024-09-20T22:09:55.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.605154+0000", 2024-09-20T22:09:55.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90399336100000005 2024-09-20T22:09:55.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:09:55.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.605243+0000", 2024-09-20T22:09:55.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8661999999999999e-05 2024-09-20T22:09:55.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 5,6", 2024-09-20T22:09:55.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.605424+0000", 2024-09-20T22:09:55.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018181800000000001 2024-09-20T22:09:55.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:09:55.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.606520+0000", 2024-09-20T22:09:55.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001095333 2024-09-20T22:09:55.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:55.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.612488+0000", 2024-09-20T22:09:55.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0059685019999999997 2024-09-20T22:09:55.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:55.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.616043+0000", 2024-09-20T22:09:55.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0035544249999999999 2024-09-20T22:09:55.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:09:55.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.616108+0000", 2024-09-20T22:09:55.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5506999999999995e-05 2024-09-20T22:09:55.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.616152+0000", 2024-09-20T22:09:55.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3603999999999998e-05 2024-09-20T22:09:55.343 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.343 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.343 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.343 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.15192.0:105655 8.f e215/176 8:ff82ed31:::benchmark_data_smithi060_111051_object105654:head v 215'6690, mlcod=214'6687)", 2024-09-20T22:09:55.343 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.562042+0000", 2024-09-20T22:09:55.343 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 46.653983853, 2024-09-20T22:09:55.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.16695152799999999, 2024-09-20T22:09:55.344 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.344 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.344 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:55.344 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.562042+0000", 2024-09-20T22:09:55.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.562042+0000", 2024-09-20T22:09:55.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.562044+0000", 2024-09-20T22:09:55.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.657e-06 2024-09-20T22:09:55.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.562073+0000", 2024-09-20T22:09:55.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9498000000000001e-05 2024-09-20T22:09:55.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.562091+0000", 2024-09-20T22:09:55.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.732e-05 2024-09-20T22:09:55.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.562101+0000", 2024-09-20T22:09:55.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0243e-05 2024-09-20T22:09:55.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720131+0000", 2024-09-20T22:09:55.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15803002899999999 2024-09-20T22:09:55.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:09:55.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720151+0000", 2024-09-20T22:09:55.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0579999999999999e-05 2024-09-20T22:09:55.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:09:55.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.728980+0000", 2024-09-20T22:09:55.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0088283089999999995 2024-09-20T22:09:55.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.728994+0000", 2024-09-20T22:09:55.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3892e-05 2024-09-20T22:09:55.348 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.348 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.348 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.348 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105668 8.7 8:e0173451:::benchmark_data_smithi060_111051_object105667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:09:55.348 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.706681+0000", 2024-09-20T22:09:55.348 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 46.509345007, 2024-09-20T22:09:55.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.87300058000000003, 2024-09-20T22:09:55.349 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.349 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.349 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:55.349 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:09:55.349 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:09:55.349 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:55.349 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105668 2024-09-20T22:09:55.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.349 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706681+0000", 2024-09-20T22:09:55.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706681+0000", 2024-09-20T22:09:55.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706683+0000", 2024-09-20T22:09:55.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.928e-06 2024-09-20T22:09:55.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706710+0000", 2024-09-20T22:09:55.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6911e-05 2024-09-20T22:09:55.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706734+0000", 2024-09-20T22:09:55.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3992999999999999e-05 2024-09-20T22:09:55.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706743+0000", 2024-09-20T22:09:55.352 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:55.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1209999999999996e-06 2024-09-20T22:09:55.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720612+0000", 2024-09-20T22:09:55.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.013869283 2024-09-20T22:09:55.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:09:55.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720619+0000", 2024-09-20T22:09:55.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7100000000000001e-06 2024-09-20T22:09:55.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.571247+0000", 2024-09-20T22:09:55.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85062784499999999 2024-09-20T22:09:55.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:09:55.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.571381+0000", 2024-09-20T22:09:55.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000134167 2024-09-20T22:09:55.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:09:55.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.571594+0000", 2024-09-20T22:09:55.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021272700000000001 2024-09-20T22:09:55.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:09:55.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.572500+0000", 2024-09-20T22:09:55.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00090607800000000005 2024-09-20T22:09:55.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:55.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.574997+0000", 2024-09-20T22:09:55.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024976759999999999 2024-09-20T22:09:55.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:55.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579580+0000", 2024-09-20T22:09:55.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0045830269999999999 2024-09-20T22:09:55.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:09:55.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579636+0000", 2024-09-20T22:09:55.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5413e-05 2024-09-20T22:09:55.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579681+0000", 2024-09-20T22:09:55.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5701000000000002e-05 2024-09-20T22:09:55.357 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.357 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.357 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.358 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106973 8.13 8:cbbe7a4a:::benchmark_data_smithi060_111051_object106972:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:09:55.358 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.414028+0000", 2024-09-20T22:09:55.358 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 43.801997565999997, 2024-09-20T22:09:55.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0848815410000001, 2024-09-20T22:09:55.358 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:09:55.358 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:09:55.358 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:09:55.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:09:55.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:09:55.358 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:09:55.358 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106973 2024-09-20T22:09:55.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.359 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:09:55.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:09:55.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414028+0000", 2024-09-20T22:09:55.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:09:55.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414028+0000", 2024-09-20T22:09:55.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:09:55.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:09:55.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414030+0000", 2024-09-20T22:09:55.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.866e-06 2024-09-20T22:09:55.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:09:55.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414059+0000", 2024-09-20T22:09:55.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9043999999999998e-05 2024-09-20T22:09:55.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:09:55.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414080+0000", 2024-09-20T22:09:55.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1254999999999999e-05 2024-09-20T22:09:55.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:09:55.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414090+0000", 2024-09-20T22:09:55.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2129999999999997e-06 2024-09-20T22:09:55.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.415798+0000", 2024-09-20T22:09:55.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017082340000000001 2024-09-20T22:09:55.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:09:55.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.415809+0000", 2024-09-20T22:09:55.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1433e-05 2024-09-20T22:09:55.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:09:55.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.479861+0000", 2024-09-20T22:09:55.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.064051522 2024-09-20T22:09:55.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:09:55.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.479994+0000", 2024-09-20T22:09:55.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013297800000000001 2024-09-20T22:09:55.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5,6", 2024-09-20T22:09:55.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.480283+0000", 2024-09-20T22:09:55.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00028927100000000002 2024-09-20T22:09:55.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:55.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.482942+0000", 2024-09-20T22:09:55.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002658737 2024-09-20T22:09:55.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:55.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.483991+0000", 2024-09-20T22:09:55.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001049267 2024-09-20T22:09:55.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:09:55.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.484017+0000", 2024-09-20T22:09:55.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6057000000000001e-05 2024-09-20T22:09:55.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:55.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.486044+0000", 2024-09-20T22:09:55.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020266189999999999 2024-09-20T22:09:55.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:09:55.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.498642+0000", 2024-09-20T22:09:55.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.012598417000000001 2024-09-20T22:09:55.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:09:55.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.498776+0000", 2024-09-20T22:09:55.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000133764 2024-09-20T22:09:55.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:09:55.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:09:55.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:09:55.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.498910+0000", 2024-09-20T22:09:55.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000133864 2024-09-20T22:09:55.366 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.366 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.366 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.366 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:09:55.366 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:09:55.366 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:09:55.400 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:55.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:55.401 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:55.502 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:55.533 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:55.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:55.534 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:55.634 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:09:55.635 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:55.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:55.683 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:55.683 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:55.784 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:55.827 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:55.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:55.828 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:55.929 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:55.930 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:09:55.959 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:55.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:55.960 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:55.968 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:55 smithi060 ceph-mon[29694]: 4.13 scrub starts 2024-09-20T22:09:55.968 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:55 smithi060 ceph-mon[29694]: 4.13 scrub ok 2024-09-20T22:09:55.968 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:55 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1179719930' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:09:55.968 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:55 smithi060 ceph-mon[29694]: 4.11 scrub starts 2024-09-20T22:09:55.968 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:55 smithi060 ceph-mon[29694]: 4.11 scrub ok 2024-09-20T22:09:55.969 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:55 smithi060 ceph-mon[40290]: 4.13 scrub starts 2024-09-20T22:09:55.969 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:55 smithi060 ceph-mon[40290]: 4.13 scrub ok 2024-09-20T22:09:55.969 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:55 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1179719930' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:09:55.969 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:55 smithi060 ceph-mon[40290]: 4.11 scrub starts 2024-09-20T22:09:55.969 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:55 smithi060 ceph-mon[40290]: 4.11 scrub ok 2024-09-20T22:09:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:55 smithi067 ceph-mon[33511]: 4.13 scrub starts 2024-09-20T22:09:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:55 smithi067 ceph-mon[33511]: 4.13 scrub ok 2024-09-20T22:09:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1179719930' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:09:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:55 smithi067 ceph-mon[33511]: 4.11 scrub starts 2024-09-20T22:09:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:55 smithi067 ceph-mon[33511]: 4.11 scrub ok 2024-09-20T22:09:56.061 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:56.092 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:56.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:56.093 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:56.194 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:56.233 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:56.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:56.234 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:56.335 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:56.383 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:56.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:56.384 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:56.394 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:09:56.394 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:09:56.395 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-20T22:09:56.395 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-20T22:09:56.485 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:56.548 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:56.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:56.549 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:56.650 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:56.672 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:56.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:56.673 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:56.774 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:56 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:09:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:56 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1179719930' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:09:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:56 smithi060 ceph-mon[40290]: osdmap e238: 8 total, 8 up, 8 in 2024-09-20T22:09:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:56 smithi060 ceph-mon[40290]: pgmap v366: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 20882/284673 objects misplaced (7.335%) 2024-09-20T22:09:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:56 smithi060 ceph-mon[40290]: osdmap e239: 8 total, 8 up, 8 in 2024-09-20T22:09:56.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:56 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:09:56.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:56 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1179719930' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:09:56.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:56 smithi060 ceph-mon[29694]: osdmap e238: 8 total, 8 up, 8 in 2024-09-20T22:09:56.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:56 smithi060 ceph-mon[29694]: pgmap v366: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 20882/284673 objects misplaced (7.335%) 2024-09-20T22:09:56.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:56 smithi060 ceph-mon[29694]: osdmap e239: 8 total, 8 up, 8 in 2024-09-20T22:09:56.811 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:56.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:56.812 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:56.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:56.954 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:56.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:56.955 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:57.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:56 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:09:57.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:56 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1179719930' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:09:57.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:56 smithi067 ceph-mon[33511]: osdmap e238: 8 total, 8 up, 8 in 2024-09-20T22:09:57.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:56 smithi067 ceph-mon[33511]: pgmap v366: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 20882/284673 objects misplaced (7.335%) 2024-09-20T22:09:57.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:56 smithi067 ceph-mon[33511]: osdmap e239: 8 total, 8 up, 8 in 2024-09-20T22:09:57.056 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:57.086 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:57.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:57.087 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:57.189 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:09:57.222 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:57.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:57.223 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:09:57.323 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:09:57.366 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:57.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:57.367 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:09:57.468 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:57.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:57.507 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:57.507 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:57.608 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:57.639 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:57.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:57.640 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:57.740 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:57.772 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:57.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:57.773 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:57.874 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:57.912 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:57.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:57.913 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:58.014 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:58.052 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:58.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:58.053 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:58.154 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:58.193 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:58.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:58.195 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:58.296 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:58.332 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:58.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:58.333 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:58.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:09:58.470 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:58.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:58.471 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:09:58.572 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:58.598 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:58.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:58.600 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:58.631 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T22:09:58.700 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:58.730 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:58.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:58.731 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:58.832 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:58.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:58.892 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:58.893 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:58.993 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:59.029 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:59.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:59.031 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:09:59.132 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:09:59.160 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:59.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:59.161 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:09:59.262 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:59.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:59 smithi060 ceph-mon[29694]: pgmap v368: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.6 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 769 B/s rd, 0 B/s wr, 538 op/s; 17284/274179 objects misplaced (6.304%) 2024-09-20T22:09:59.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:59 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:09:59.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:59 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:09:59.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:09:59 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:09:59.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:59 smithi060 ceph-mon[40290]: pgmap v368: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.6 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 769 B/s rd, 0 B/s wr, 538 op/s; 17284/274179 objects misplaced (6.304%) 2024-09-20T22:09:59.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:59 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:09:59.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:59 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:09:59.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:09:59 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:09:59.303 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:59.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:59.304 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:59.404 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:09:59.439 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:59.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:59.440 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:09:59.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:59 smithi067 ceph-mon[33511]: pgmap v368: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.6 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 769 B/s rd, 0 B/s wr, 538 op/s; 17284/274179 objects misplaced (6.304%) 2024-09-20T22:09:59.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:59 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:09:59.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:59 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:09:59.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:09:59 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:09:59.542 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:09:59.586 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:59.615 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:09:59.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:59.659 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:59.659 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:09:59.760 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:09:59.812 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:09:59.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:59.813 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:09:59.914 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:09:59.947 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:09:59.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:09:59.948 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:00.049 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:00.081 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:00.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:00.082 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:00.182 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:00.229 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:00.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:00.231 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:00.331 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:00.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:00 smithi060 ceph-mon[29694]: Health detail: HEALTH_WARN noscrub flag(s) set; Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull 2024-09-20T22:10:00.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:00 smithi060 ceph-mon[29694]: [WRN] OSDMAP_FLAGS: noscrub flag(s) set 2024-09-20T22:10:00.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:00 smithi060 ceph-mon[29694]: [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull 2024-09-20T22:10:00.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:00 smithi060 ceph-mon[29694]: pg 8.15 is active+remapped+backfill_toofull, acting [7,6,4] 2024-09-20T22:10:00.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:00 smithi060 ceph-mon[29694]: pg 8.16 is active+remapped+backfill_toofull, acting [0,6,7] 2024-09-20T22:10:00.364 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:00 smithi060 ceph-mon[40290]: Health detail: HEALTH_WARN noscrub flag(s) set; Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull 2024-09-20T22:10:00.364 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:00 smithi060 ceph-mon[40290]: [WRN] OSDMAP_FLAGS: noscrub flag(s) set 2024-09-20T22:10:00.364 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:00 smithi060 ceph-mon[40290]: [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull 2024-09-20T22:10:00.364 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:00 smithi060 ceph-mon[40290]: pg 8.15 is active+remapped+backfill_toofull, acting [7,6,4] 2024-09-20T22:10:00.364 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:00 smithi060 ceph-mon[40290]: pg 8.16 is active+remapped+backfill_toofull, acting [0,6,7] 2024-09-20T22:10:00.381 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:00.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:00.382 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:00.483 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:00.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:00 smithi067 ceph-mon[33511]: Health detail: HEALTH_WARN noscrub flag(s) set; Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull 2024-09-20T22:10:00.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:00 smithi067 ceph-mon[33511]: [WRN] OSDMAP_FLAGS: noscrub flag(s) set 2024-09-20T22:10:00.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:00 smithi067 ceph-mon[33511]: [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull 2024-09-20T22:10:00.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:00 smithi067 ceph-mon[33511]: pg 8.15 is active+remapped+backfill_toofull, acting [7,6,4] 2024-09-20T22:10:00.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:00 smithi067 ceph-mon[33511]: pg 8.16 is active+remapped+backfill_toofull, acting [0,6,7] 2024-09-20T22:10:00.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:00.515 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:00.515 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:00.569 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:10:00.569 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:10:00.577 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:10:00.578 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:10:00.615 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:10:00.615 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:10:00.615 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:00.654 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:10:00.654 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:10:00.661 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:00.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:00.662 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:00.677 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:00.694 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:10:00.694 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:10:00.742 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:10:00.743 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:10:00.763 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:00.784 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:10:00.784 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:10:00.806 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:00.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:00.807 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:00.809 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:10:00.810 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:10:00.894 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:10:00.894 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:10:00.894 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 239, 2024-09-20T22:10:00.894 INFO:teuthology.orchestra.run.smithi060.stdout: "fsid": "3df9647e-779b-11ef-baf5-efdc52797490", 2024-09-20T22:10:00.894 INFO:teuthology.orchestra.run.smithi060.stdout: "created": "2024-09-20T21:57:31.244934+0000", 2024-09-20T22:10:00.894 INFO:teuthology.orchestra.run.smithi060.stdout: "modified": "2024-09-20T22:09:56.421224+0000", 2024-09-20T22:10:00.894 INFO:teuthology.orchestra.run.smithi060.stdout: "last_up_change": "2024-09-20T22:06:53.796679+0000", 2024-09-20T22:10:00.895 INFO:teuthology.orchestra.run.smithi060.stdout: "last_in_change": "2024-09-20T22:02:21.189225+0000", 2024-09-20T22:10:00.895 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-20T22:10:00.895 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_num": 5801984, 2024-09-20T22:10:00.895 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_set": [ 2024-09-20T22:10:00.895 INFO:teuthology.orchestra.run.smithi060.stdout: "noscrub", 2024-09-20T22:10:00.895 INFO:teuthology.orchestra.run.smithi060.stdout: "pglog_hardlimit", 2024-09-20T22:10:00.895 INFO:teuthology.orchestra.run.smithi060.stdout: "purged_snapdirs", 2024-09-20T22:10:00.895 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_deletes", 2024-09-20T22:10:00.895 INFO:teuthology.orchestra.run.smithi060.stdout: "sortbitwise" 2024-09-20T22:10:00.895 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:10:00.895 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_version": 19, 2024-09-20T22:10:00.895 INFO:teuthology.orchestra.run.smithi060.stdout: "full_ratio": 0.94999998807907104, 2024-09-20T22:10:00.895 INFO:teuthology.orchestra.run.smithi060.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-20T22:10:00.896 INFO:teuthology.orchestra.run.smithi060.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-20T22:10:00.896 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_snapshot": "", 2024-09-20T22:10:00.896 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_max": 8, 2024-09-20T22:10:00.896 INFO:teuthology.orchestra.run.smithi060.stdout: "max_osd": 8, 2024-09-20T22:10:00.896 INFO:teuthology.orchestra.run.smithi060.stdout: "require_min_compat_client": "luminous", 2024-09-20T22:10:00.896 INFO:teuthology.orchestra.run.smithi060.stdout: "min_compat_client": "luminous", 2024-09-20T22:10:00.896 INFO:teuthology.orchestra.run.smithi060.stdout: "require_osd_release": "squid", 2024-09-20T22:10:00.896 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_crimson": false, 2024-09-20T22:10:00.896 INFO:teuthology.orchestra.run.smithi060.stdout: "pools": [ 2024-09-20T22:10:00.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.896 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 1, 2024-09-20T22:10:00.896 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": ".mgr", 2024-09-20T22:10:00.896 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:00:35.131125+0000", 2024-09-20T22:10:00.896 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:10:00.897 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:10:00.897 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:10:00.897 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:10:00.897 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:10:00.897 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:10:00.897 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:10:00.897 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:10:00.897 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:10:00.897 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:10:00.897 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:10:00.897 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:10:00.897 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:10:00.897 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 11, 2024-09-20T22:10:00.897 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 11, 2024-09-20T22:10:00.898 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 11, 2024-09-20T22:10:00.898 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 11, 2024-09-20T22:10:00.898 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 11, 2024-09-20T22:10:00.898 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:10:00.898 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:10:00.898 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:10:00.898 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:10:00.898 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:10:00.898 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:10:00.898 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:10:00.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.898 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "199", 2024-09-20T22:10:00.898 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:10:00.898 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:10:00.899 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "197", 2024-09-20T22:10:00.899 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:10:00.899 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:10:00.899 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:10:00.899 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:10:00.899 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:10:00.899 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:10:00.899 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:10:00.899 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:10:00.899 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:10:00.899 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:10:00.899 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:10:00.899 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:10:00.899 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:10:00.900 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:10:00.900 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:10:00.900 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:10:00.900 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:10:00.900 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:10:00.900 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:10:00.900 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:10:00.900 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:10:00.900 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:10:00.900 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:10:00.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.900 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:10:00.900 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:10:00.901 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:10:00.901 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:10:00.901 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:10:00.901 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:10:00.901 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:10:00.901 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:10:00.901 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:10:00.901 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:10:00.901 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:10:00.901 INFO:teuthology.orchestra.run.smithi060.stdout: "options": { 2024-09-20T22:10:00.901 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_max": 32, 2024-09-20T22:10:00.901 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_min": 1 2024-09-20T22:10:00.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.901 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:10:00.902 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": {} 2024-09-20T22:10:00.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.902 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:10:00.902 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:10:00.902 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2.1800000667572021, 2024-09-20T22:10:00.902 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 2.1800000667572021, 2024-09-20T22:10:00.902 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:10:00.902 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2.1800000667572021, 2024-09-20T22:10:00.902 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 2.1800000667572021, 2024-09-20T22:10:00.902 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:10:00.902 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:10:00.902 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:10:00.903 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.903 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 2, 2024-09-20T22:10:00.903 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "datapool", 2024-09-20T22:10:00.903 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:47.974480+0000", 2024-09-20T22:10:00.903 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 8193, 2024-09-20T22:10:00.903 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-20T22:10:00.903 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:10:00.903 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:10:00.903 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:10:00.903 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:10:00.903 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:10:00.903 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:10:00.904 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:10:00.904 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:10:00.904 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:10:00.904 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:10:00.904 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:10:00.904 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 3, 2024-09-20T22:10:00.904 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 3, 2024-09-20T22:10:00.904 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 3, 2024-09-20T22:10:00.904 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 3, 2024-09-20T22:10:00.904 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 3, 2024-09-20T22:10:00.904 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:10:00.904 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:10:00.904 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:10:00.904 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:10:00.905 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:10:00.905 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:10:00.905 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:10:00.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.905 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "66", 2024-09-20T22:10:00.905 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:10:00.905 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:10:00.905 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:10:00.905 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:10:00.905 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:10:00.905 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 2, 2024-09-20T22:10:00.905 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 66, 2024-09-20T22:10:00.905 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:10:00.906 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:10:00.906 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:10:00.906 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:10:00.906 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:10:00.906 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:10:00.906 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:10:00.906 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:10:00.906 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:10:00.906 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:10:00.906 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:10:00.906 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:10:00.906 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:10:00.906 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:10:00.906 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:10:00.907 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:10:00.907 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:10:00.907 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:10:00.907 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:10:00.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.907 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:10:00.907 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:10:00.907 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:10:00.907 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:10:00.907 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:10:00.907 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:10:00.907 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:10:00.907 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:10:00.907 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:10:00.908 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:10:00.908 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:10:00.908 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:10:00.908 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:10:00.908 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd": {} 2024-09-20T22:10:00.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.908 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:10:00.908 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:10:00.908 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2.6500000953674316, 2024-09-20T22:10:00.908 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 2.6500000953674316, 2024-09-20T22:10:00.908 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T22:10:00.908 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-20T22:10:00.908 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-20T22:10:00.908 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:10:00.908 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:10:00.909 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:10:00.909 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.909 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 3, 2024-09-20T22:10:00.909 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": ".rgw.root", 2024-09-20T22:10:00.909 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:48.902465+0000", 2024-09-20T22:10:00.909 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:10:00.909 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:10:00.909 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:10:00.909 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:10:00.909 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:10:00.909 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:10:00.909 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:10:00.909 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:10:00.909 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:10:00.910 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:10:00.910 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:10:00.910 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:10:00.910 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:10:00.910 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:10:00.910 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:10:00.910 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:10:00.910 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:10:00.910 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:10:00.910 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:10:00.910 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:10:00.910 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:10:00.910 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:10:00.911 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:00.911 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:10:00.912 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:10:00.912 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:10:00.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.912 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "65", 2024-09-20T22:10:00.912 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:10:00.912 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:10:00.912 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:10:00.912 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:10:00.912 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:10:00.912 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:10:00.912 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:10:00.912 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:10:00.912 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:10:00.913 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:10:00.913 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:10:00.913 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:10:00.913 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:10:00.913 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:10:00.913 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:10:00.913 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:10:00.913 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:10:00.913 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:10:00.913 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:10:00.913 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:10:00.913 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:10:00.913 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:10:00.913 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:10:00.914 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:10:00.914 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:10:00.914 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:10:00.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.914 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:10:00.914 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:10:00.914 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:10:00.914 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:10:00.914 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:10:00.914 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:10:00.914 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:10:00.914 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:10:00.914 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:10:00.914 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:10:00.915 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:10:00.915 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:10:00.915 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:10:00.915 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:10:00.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.915 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:10:00.915 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:10:00.915 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.75, 2024-09-20T22:10:00.915 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.75, 2024-09-20T22:10:00.915 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:10:00.915 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.75, 2024-09-20T22:10:00.915 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.75, 2024-09-20T22:10:00.915 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:10:00.915 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:10:00.915 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:10:00.916 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.916 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 4, 2024-09-20T22:10:00.916 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "default.rgw.log", 2024-09-20T22:10:00.916 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:50.620057+0000", 2024-09-20T22:10:00.916 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:10:00.916 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:10:00.916 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:10:00.916 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:10:00.916 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:10:00.916 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:10:00.916 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:10:00.916 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:10:00.917 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:10:00.917 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:10:00.917 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:10:00.917 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:10:00.917 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:10:00.917 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:10:00.917 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:10:00.917 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:10:00.917 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:10:00.917 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:10:00.917 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:10:00.917 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:10:00.917 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:10:00.917 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:10:00.918 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:10:00.918 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:10:00.918 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:10:00.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.918 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "67", 2024-09-20T22:10:00.918 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:10:00.918 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:10:00.918 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:10:00.918 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:10:00.918 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:10:00.918 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:10:00.918 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:10:00.918 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:10:00.918 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:10:00.918 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:10:00.919 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:10:00.919 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:10:00.919 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:10:00.919 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:10:00.919 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:10:00.919 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:10:00.919 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:10:00.919 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:10:00.919 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:10:00.919 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:10:00.919 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:10:00.919 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:10:00.919 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:10:00.919 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:10:00.920 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:10:00.920 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:10:00.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.920 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:10:00.920 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:10:00.920 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:10:00.920 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:10:00.920 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:10:00.920 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:10:00.920 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:10:00.920 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:10:00.920 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:10:00.920 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:10:00.920 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:10:00.920 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:10:00.921 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:10:00.921 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:10:00.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.921 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:10:00.921 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:10:00.921 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2.25, 2024-09-20T22:10:00.921 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 2.25, 2024-09-20T22:10:00.921 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:10:00.921 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2.25, 2024-09-20T22:10:00.921 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 2.25, 2024-09-20T22:10:00.921 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:10:00.921 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:10:00.921 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:10:00.921 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.922 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 5, 2024-09-20T22:10:00.922 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "default.rgw.control", 2024-09-20T22:10:00.922 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:52.722912+0000", 2024-09-20T22:10:00.922 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:10:00.922 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:10:00.922 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:10:00.922 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:10:00.922 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:10:00.922 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:10:00.922 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:10:00.922 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:10:00.922 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:10:00.923 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:10:00.923 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:10:00.923 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:10:00.923 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:10:00.923 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:10:00.923 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:10:00.923 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:10:00.923 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:10:00.923 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:10:00.923 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:10:00.923 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:10:00.923 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:10:00.923 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:10:00.923 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:10:00.923 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:10:00.924 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:10:00.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.924 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "69", 2024-09-20T22:10:00.924 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:10:00.924 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:10:00.924 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:10:00.924 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:10:00.924 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:10:00.924 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:10:00.924 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:10:00.924 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:10:00.924 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:10:00.924 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:10:00.924 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:10:00.924 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:10:00.925 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:10:00.925 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:10:00.925 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:10:00.925 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:10:00.925 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:10:00.925 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:10:00.925 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:10:00.925 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:10:00.925 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:10:00.925 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:10:00.925 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:10:00.925 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:10:00.925 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:10:00.925 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:10:00.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.926 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:10:00.926 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:10:00.926 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:10:00.926 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:10:00.926 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:10:00.926 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:10:00.926 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:10:00.926 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:10:00.926 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:10:00.926 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:10:00.926 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:10:00.926 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:10:00.926 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:10:00.927 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:10:00.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.927 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:10:00.927 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:10:00.927 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.5, 2024-09-20T22:10:00.927 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.5, 2024-09-20T22:10:00.927 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:10:00.927 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.5, 2024-09-20T22:10:00.927 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.5, 2024-09-20T22:10:00.927 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:10:00.927 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:10:00.927 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:10:00.927 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.928 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 6, 2024-09-20T22:10:00.928 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "default.rgw.meta", 2024-09-20T22:10:00.928 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:54.794248+0000", 2024-09-20T22:10:00.928 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:10:00.928 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:10:00.928 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:10:00.928 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:10:00.928 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:10:00.928 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:10:00.928 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:10:00.928 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:10:00.928 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:10:00.929 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:10:00.929 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:10:00.929 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:10:00.929 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:10:00.929 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:10:00.929 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:10:00.929 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:10:00.929 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:10:00.929 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:10:00.929 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:10:00.929 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:10:00.929 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:10:00.929 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:10:00.929 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:10:00.929 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:10:00.930 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:10:00.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.930 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "71", 2024-09-20T22:10:00.930 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:10:00.930 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:10:00.930 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:10:00.930 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:10:00.930 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:10:00.930 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:10:00.930 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:10:00.930 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:10:00.930 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:10:00.930 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:10:00.931 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:10:00.931 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:10:00.931 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:10:00.931 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:10:00.931 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:10:00.931 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:10:00.931 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:10:00.931 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:10:00.931 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:10:00.931 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:10:00.931 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:10:00.931 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:10:00.931 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:10:00.931 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:10:00.932 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:10:00.932 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:10:00.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.932 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:10:00.932 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:10:00.932 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:10:00.932 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:10:00.932 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:10:00.932 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:10:00.932 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:10:00.932 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:10:00.932 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:10:00.932 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:10:00.932 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:10:00.932 INFO:teuthology.orchestra.run.smithi060.stdout: "options": { 2024-09-20T22:10:00.933 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_bias": 4 2024-09-20T22:10:00.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.933 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:10:00.933 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:10:00.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.933 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:10:00.933 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:10:00.933 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2, 2024-09-20T22:10:00.933 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 2, 2024-09-20T22:10:00.933 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:10:00.933 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2, 2024-09-20T22:10:00.933 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 2, 2024-09-20T22:10:00.933 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:10:00.934 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:10:00.934 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:10:00.934 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.934 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 8, 2024-09-20T22:10:00.934 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "unique_pool_1", 2024-09-20T22:10:00.934 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:07:51.043037+0000", 2024-09-20T22:10:00.934 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:10:00.934 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:10:00.934 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:10:00.934 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:10:00.934 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:10:00.934 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:10:00.935 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:10:00.935 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:10:00.935 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:10:00.935 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:10:00.935 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:10:00.935 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:10:00.935 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:10:00.935 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 26, 2024-09-20T22:10:00.935 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 26, 2024-09-20T22:10:00.935 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 26, 2024-09-20T22:10:00.935 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 26, 2024-09-20T22:10:00.935 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 26, 2024-09-20T22:10:00.935 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:10:00.935 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:10:00.936 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:10:00.936 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:10:00.936 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:10:00.936 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:10:00.936 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:10:00.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.936 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "217", 2024-09-20T22:10:00.936 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:10:00.936 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:10:00.936 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "215", 2024-09-20T22:10:00.936 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:10:00.936 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:10:00.936 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:10:00.936 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:10:00.937 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:10:00.937 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:10:00.937 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:10:00.937 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:10:00.937 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:10:00.937 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:10:00.937 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:10:00.937 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:10:00.937 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:10:00.937 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:10:00.937 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:10:00.937 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:10:00.937 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:10:00.937 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:10:00.937 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:10:00.938 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:10:00.938 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:10:00.938 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:10:00.938 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:10:00.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.938 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:10:00.938 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:10:00.938 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:10:00.938 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:10:00.938 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:10:00.938 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:10:00.938 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:10:00.938 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:10:00.938 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:10:00.939 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:10:00.939 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:10:00.939 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:10:00.939 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:10:00.939 INFO:teuthology.orchestra.run.smithi060.stdout: "rados": {} 2024-09-20T22:10:00.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.939 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:10:00.939 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:10:00.939 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.5399999618530273, 2024-09-20T22:10:00.939 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.5399999618530273, 2024-09-20T22:10:00.939 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:10:00.939 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.5399999618530273, 2024-09-20T22:10:00.939 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.5399999618530273, 2024-09-20T22:10:00.939 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 1, 2024-09-20T22:10:00.940 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 1, 2024-09-20T22:10:00.940 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 1 2024-09-20T22:10:00.940 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.940 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.940 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:10:00.940 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:10:00.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.940 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 0, 2024-09-20T22:10:00.940 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "0997c01a-a9bd-421e-abcc-fbcf7abe8512", 2024-09-20T22:10:00.940 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:10:00.940 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:10:00.940 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:10:00.940 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:10:00.940 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:10:00.940 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:10:00.941 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 8, 2024-09-20T22:10:00.941 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 234, 2024-09-20T22:10:00.941 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:10:00.941 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:10:00.941 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:10:00.941 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.941 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.941 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6802", 2024-09-20T22:10:00.941 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:10:00.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.941 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.941 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6803", 2024-09-20T22:10:00.941 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:10:00.942 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.942 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.942 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:10:00.942 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.942 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.942 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6804", 2024-09-20T22:10:00.942 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:10:00.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.942 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.942 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6805", 2024-09-20T22:10:00.942 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:10:00.943 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.943 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.943 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:10:00.943 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.943 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.943 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6808", 2024-09-20T22:10:00.943 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:10:00.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.943 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.943 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6809", 2024-09-20T22:10:00.943 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:10:00.944 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.944 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.944 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:10:00.944 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.944 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.944 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6806", 2024-09-20T22:10:00.945 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:10:00.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.945 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.945 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6807", 2024-09-20T22:10:00.945 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:10:00.945 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.945 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.945 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6803/1395849550", 2024-09-20T22:10:00.946 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6805/1395849550", 2024-09-20T22:10:00.946 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6809/1395849550", 2024-09-20T22:10:00.946 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6807/1395849550", 2024-09-20T22:10:00.946 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:10:00.946 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:10:00.946 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:10:00.946 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.946 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 1, 2024-09-20T22:10:00.946 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "c04bdcb1-d76b-47c8-a38b-c02043df6ba8", 2024-09-20T22:10:00.946 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:10:00.946 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:10:00.946 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:10:00.946 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:10:00.947 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 13, 2024-09-20T22:10:00.947 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 90, 2024-09-20T22:10:00.947 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 145, 2024-09-20T22:10:00.947 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 234, 2024-09-20T22:10:00.947 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 91, 2024-09-20T22:10:00.947 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:10:00.947 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:10:00.947 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.947 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.947 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6810", 2024-09-20T22:10:00.947 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:10:00.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.947 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.948 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6811", 2024-09-20T22:10:00.948 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:10:00.948 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.948 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.948 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:10:00.948 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.948 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.948 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6812", 2024-09-20T22:10:00.948 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:10:00.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.948 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.948 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6813", 2024-09-20T22:10:00.949 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:10:00.949 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.949 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.949 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:10:00.949 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.949 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.949 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6816", 2024-09-20T22:10:00.949 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:10:00.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.949 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.949 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6817", 2024-09-20T22:10:00.949 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:10:00.949 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.950 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.950 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:10:00.950 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.950 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.950 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6814", 2024-09-20T22:10:00.950 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:10:00.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.950 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.950 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6815", 2024-09-20T22:10:00.950 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:10:00.950 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.951 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.951 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6811/2970874921", 2024-09-20T22:10:00.951 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6813/2970874921", 2024-09-20T22:10:00.951 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6817/2970874921", 2024-09-20T22:10:00.951 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6815/2970874921", 2024-09-20T22:10:00.951 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:10:00.951 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:10:00.951 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:10:00.951 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.951 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 2, 2024-09-20T22:10:00.951 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "32c09d19-eb98-4926-a44e-e7ddee152956", 2024-09-20T22:10:00.951 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:10:00.952 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:10:00.952 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:10:00.952 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:10:00.952 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:10:00.952 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:10:00.952 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 18, 2024-09-20T22:10:00.952 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 204, 2024-09-20T22:10:00.952 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:10:00.952 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:10:00.952 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:10:00.952 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.952 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.952 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6818", 2024-09-20T22:10:00.952 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:10:00.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.953 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.953 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6819", 2024-09-20T22:10:00.953 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:10:00.953 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.953 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.953 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:10:00.953 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.953 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.953 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6820", 2024-09-20T22:10:00.953 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:10:00.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.954 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.954 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6821", 2024-09-20T22:10:00.954 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:10:00.954 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.954 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.954 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:10:00.954 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.954 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.954 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6824", 2024-09-20T22:10:00.954 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:10:00.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.955 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.955 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6825", 2024-09-20T22:10:00.955 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:10:00.955 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.955 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.955 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:10:00.955 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.955 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.955 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6822", 2024-09-20T22:10:00.955 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:10:00.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.956 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.956 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6823", 2024-09-20T22:10:00.956 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:10:00.956 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.956 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.956 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6819/3698562274", 2024-09-20T22:10:00.956 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6821/3698562274", 2024-09-20T22:10:00.956 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6825/3698562274", 2024-09-20T22:10:00.956 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6823/3698562274", 2024-09-20T22:10:00.956 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:10:00.956 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:10:00.956 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:10:00.956 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.957 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 3, 2024-09-20T22:10:00.957 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "b1c2cb69-8565-49ac-b854-7ea7615c6660", 2024-09-20T22:10:00.957 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:10:00.957 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:10:00.957 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:10:00.957 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:10:00.957 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:10:00.957 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:10:00.957 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 26, 2024-09-20T22:10:00.957 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 235, 2024-09-20T22:10:00.957 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:10:00.957 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:10:00.957 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:10:00.958 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.958 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.958 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6826", 2024-09-20T22:10:00.958 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:10:00.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.958 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.958 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6827", 2024-09-20T22:10:00.958 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:10:00.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.958 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.958 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:10:00.958 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.959 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.959 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6828", 2024-09-20T22:10:00.959 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:10:00.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.959 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.959 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6829", 2024-09-20T22:10:00.959 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:10:00.959 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.959 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.959 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:10:00.959 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.960 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.960 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6832", 2024-09-20T22:10:00.960 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:10:00.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.960 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.960 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6833", 2024-09-20T22:10:00.960 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:10:00.960 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.960 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.960 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:10:00.960 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.960 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.961 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6830", 2024-09-20T22:10:00.961 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:10:00.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.961 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.961 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6831", 2024-09-20T22:10:00.961 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:10:00.961 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.961 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.961 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6827/2592322449", 2024-09-20T22:10:00.961 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6829/2592322449", 2024-09-20T22:10:00.961 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6833/2592322449", 2024-09-20T22:10:00.961 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6831/2592322449", 2024-09-20T22:10:00.961 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:10:00.962 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:10:00.962 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:10:00.962 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.962 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 4, 2024-09-20T22:10:00.962 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "04a547d6-fc48-43b7-a8d7-18df8a83aa96", 2024-09-20T22:10:00.962 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:10:00.962 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:10:00.962 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:10:00.962 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:10:00.962 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:10:00.962 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:10:00.962 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 31, 2024-09-20T22:10:00.962 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 234, 2024-09-20T22:10:00.963 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:10:00.963 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:10:00.963 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:10:00.963 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.963 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.963 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6800", 2024-09-20T22:10:00.963 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:10:00.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.963 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.963 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6801", 2024-09-20T22:10:00.963 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:10:00.963 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.964 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.964 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:10:00.964 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.964 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.964 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6802", 2024-09-20T22:10:00.964 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:10:00.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.964 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.964 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6803", 2024-09-20T22:10:00.964 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:10:00.964 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.965 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.965 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:10:00.965 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.965 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.965 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6806", 2024-09-20T22:10:00.965 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:10:00.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.965 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.965 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6807", 2024-09-20T22:10:00.965 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:10:00.965 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.966 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.966 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:10:00.966 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.966 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.966 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6804", 2024-09-20T22:10:00.966 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:10:00.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.966 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.966 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6805", 2024-09-20T22:10:00.966 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:10:00.966 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.967 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.967 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6801/3243564239", 2024-09-20T22:10:00.967 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6803/3243564239", 2024-09-20T22:10:00.967 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6807/3243564239", 2024-09-20T22:10:00.967 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6805/3243564239", 2024-09-20T22:10:00.967 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:10:00.967 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:10:00.967 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:10:00.967 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.967 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 5, 2024-09-20T22:10:00.967 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "4763aa10-c367-485b-b525-e5241e642b86", 2024-09-20T22:10:00.968 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:10:00.968 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:10:00.968 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:10:00.968 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:10:00.968 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:10:00.968 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:10:00.968 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 39, 2024-09-20T22:10:00.968 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 223, 2024-09-20T22:10:00.968 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:10:00.968 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:10:00.968 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:10:00.968 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.968 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.969 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6808", 2024-09-20T22:10:00.969 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:10:00.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.969 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.969 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6809", 2024-09-20T22:10:00.969 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:10:00.969 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.969 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.969 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:10:00.969 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.970 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.970 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6810", 2024-09-20T22:10:00.970 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:10:00.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.970 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.970 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6811", 2024-09-20T22:10:00.970 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:10:00.970 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.970 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.970 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:10:00.970 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.971 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.971 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6814", 2024-09-20T22:10:00.971 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:10:00.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.971 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.971 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6815", 2024-09-20T22:10:00.971 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:10:00.971 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.971 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.971 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:10:00.971 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.972 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.972 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6812", 2024-09-20T22:10:00.972 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:10:00.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.972 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.972 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6813", 2024-09-20T22:10:00.972 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:10:00.972 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.972 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.972 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6809/1622258887", 2024-09-20T22:10:00.972 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6811/1622258887", 2024-09-20T22:10:00.973 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6815/1622258887", 2024-09-20T22:10:00.973 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6813/1622258887", 2024-09-20T22:10:00.973 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:10:00.973 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:10:00.973 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:10:00.973 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.973 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 6, 2024-09-20T22:10:00.973 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "a359116e-c013-4e2f-8b79-86219516ee16", 2024-09-20T22:10:00.973 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:10:00.973 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:10:00.973 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:10:00.973 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:10:00.973 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:10:00.974 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:10:00.974 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 48, 2024-09-20T22:10:00.974 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 235, 2024-09-20T22:10:00.974 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:10:00.974 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:10:00.974 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:10:00.974 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.974 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.974 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6816", 2024-09-20T22:10:00.974 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:10:00.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.974 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.974 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6817", 2024-09-20T22:10:00.975 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:10:00.975 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.975 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.975 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:10:00.975 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.975 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.975 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6818", 2024-09-20T22:10:00.975 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:10:00.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.975 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.976 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6819", 2024-09-20T22:10:00.976 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:10:00.976 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.976 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.976 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:10:00.976 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.976 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.976 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6822", 2024-09-20T22:10:00.976 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:10:00.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.976 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.976 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6823", 2024-09-20T22:10:00.976 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:10:00.977 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.977 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.977 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:10:00.977 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.977 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.977 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6820", 2024-09-20T22:10:00.977 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:10:00.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.977 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.977 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6821", 2024-09-20T22:10:00.977 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:10:00.978 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.978 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.978 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6817/1187114", 2024-09-20T22:10:00.978 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6819/1187114", 2024-09-20T22:10:00.978 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6823/1187114", 2024-09-20T22:10:00.978 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6821/1187114", 2024-09-20T22:10:00.978 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:10:00.978 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:10:00.978 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:10:00.978 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.978 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 7, 2024-09-20T22:10:00.979 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "a4bf4188-f133-4a31-8897-d7e8182248d7", 2024-09-20T22:10:00.979 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:10:00.979 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:10:00.979 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 0.732696533203125, 2024-09-20T22:10:00.979 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:10:00.979 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:10:00.979 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:10:00.979 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 57, 2024-09-20T22:10:00.979 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 232, 2024-09-20T22:10:00.979 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:10:00.979 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:10:00.979 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:10:00.979 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.979 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.980 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6824", 2024-09-20T22:10:00.980 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:10:00.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.981 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.981 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6825", 2024-09-20T22:10:00.981 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:10:00.981 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.981 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.981 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:10:00.981 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.981 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.982 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6826", 2024-09-20T22:10:00.982 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:10:00.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.982 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.982 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6827", 2024-09-20T22:10:00.982 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:10:00.982 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.982 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.982 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:10:00.982 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.982 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.982 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6830", 2024-09-20T22:10:00.983 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:10:00.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.983 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.983 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6831", 2024-09-20T22:10:00.983 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:10:00.983 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.983 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.983 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:10:00.983 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:10:00.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.983 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:10:00.983 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6828", 2024-09-20T22:10:00.984 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:10:00.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.984 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:10:00.984 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6829", 2024-09-20T22:10:00.984 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:10:00.984 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.984 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.984 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6825/3021842840", 2024-09-20T22:10:00.984 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6827/3021842840", 2024-09-20T22:10:00.984 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6831/3021842840", 2024-09-20T22:10:00.984 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6829/3021842840", 2024-09-20T22:10:00.984 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:10:00.984 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:10:00.985 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:10:00.985 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.985 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.985 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:10:00.985 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_xinfo": [ 2024-09-20T22:10:00.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.985 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 0, 2024-09-20T22:10:00.985 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:10:00.985 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:10:00.985 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:10:00.985 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:10:00.985 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:10:00.985 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:09:40.512797+0000", 2024-09-20T22:10:00.985 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:10:00.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.986 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 1, 2024-09-20T22:10:00.986 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:10:00.986 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:10:00.986 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:10:00.986 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:10:00.986 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:10:00.986 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:09:20.725454+0000", 2024-09-20T22:10:00.986 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 90 2024-09-20T22:10:00.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.986 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 2, 2024-09-20T22:10:00.986 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:10:00.987 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:10:00.987 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:10:00.987 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:10:00.987 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:10:00.987 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:09:39.953596+0000", 2024-09-20T22:10:00.987 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:10:00.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.987 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 3, 2024-09-20T22:10:00.987 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:10:00.987 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:10:00.987 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:10:00.988 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:10:00.988 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:10:00.988 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:08:37.826751+0000", 2024-09-20T22:10:00.988 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:10:00.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.988 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 4, 2024-09-20T22:10:00.988 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:10:00.988 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:10:00.988 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:10:00.988 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:10:00.988 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:10:00.988 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:08:55.939475+0000", 2024-09-20T22:10:00.988 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:10:00.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.989 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 5, 2024-09-20T22:10:00.989 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:10:00.989 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:10:00.989 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:10:00.989 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:10:00.989 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:10:00.989 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:09:21.083547+0000", 2024-09-20T22:10:00.989 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:10:00.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.989 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 6, 2024-09-20T22:10:00.990 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:10:00.990 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:10:00.990 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:10:00.990 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:10:00.990 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:10:00.990 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:09:25.296562+0000", 2024-09-20T22:10:00.990 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:10:00.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.990 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 7, 2024-09-20T22:10:00.990 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:10:00.990 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:10:00.990 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:10:00.990 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:10:00.991 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:10:00.991 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:09:27.456118+0000", 2024-09-20T22:10:00.991 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:10:00.991 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.991 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:10:00.991 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap": [ 2024-09-20T22:10:00.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.991 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.13", 2024-09-20T22:10:00.991 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:10:00.991 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:10:00.991 INFO:teuthology.orchestra.run.smithi060.stdout: 7, 2024-09-20T22:10:00.991 INFO:teuthology.orchestra.run.smithi060.stdout: 6 2024-09-20T22:10:00.991 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.991 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.992 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:10:00.992 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap_items": [ 2024-09-20T22:10:00.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.992 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "3.1a", 2024-09-20T22:10:00.992 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:10:00.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.992 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 4, 2024-09-20T22:10:00.992 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 5 2024-09-20T22:10:00.992 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.992 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.992 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "3.1c", 2024-09-20T22:10:00.992 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:10:00.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.993 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 4, 2024-09-20T22:10:00.993 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 2 2024-09-20T22:10:00.993 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.993 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.993 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "4.f", 2024-09-20T22:10:00.993 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:10:00.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.993 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 1, 2024-09-20T22:10:00.993 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 2 2024-09-20T22:10:00.993 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.993 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.994 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "4.13", 2024-09-20T22:10:00.994 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:10:00.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.994 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 1, 2024-09-20T22:10:00.994 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 2 2024-09-20T22:10:00.994 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.994 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.994 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "4.1b", 2024-09-20T22:10:00.994 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:10:00.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.994 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 1, 2024-09-20T22:10:00.994 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 5 2024-09-20T22:10:00.995 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.995 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.995 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.7", 2024-09-20T22:10:00.995 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:10:00.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.995 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:10:00.995 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 0 2024-09-20T22:10:00.995 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.995 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.995 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.9", 2024-09-20T22:10:00.996 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:10:00.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.996 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 7, 2024-09-20T22:10:00.996 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 1 2024-09-20T22:10:00.996 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.996 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.996 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.b", 2024-09-20T22:10:00.996 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:10:00.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.996 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:10:00.996 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 0 2024-09-20T22:10:00.996 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.996 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.997 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.d", 2024-09-20T22:10:00.997 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:10:00.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.997 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 5, 2024-09-20T22:10:00.997 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 6 2024-09-20T22:10:00.997 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.997 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.997 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.11", 2024-09-20T22:10:00.997 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:10:00.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.998 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:10:00.998 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 1 2024-09-20T22:10:00.998 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.998 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.998 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.19", 2024-09-20T22:10:00.998 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:10:00.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.998 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:10:00.998 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 6 2024-09-20T22:10:00.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:00.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.998 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 5, 2024-09-20T22:10:00.998 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 0 2024-09-20T22:10:00.999 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.999 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.999 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:00.999 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:10:00.999 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap_primaries": [], 2024-09-20T22:10:00.999 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_temp": [ 2024-09-20T22:10:00.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:00.999 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "8.11", 2024-09-20T22:10:00.999 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:10:00.999 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:10:00.999 INFO:teuthology.orchestra.run.smithi060.stdout: 1, 2024-09-20T22:10:00.999 INFO:teuthology.orchestra.run.smithi060.stdout: 2 2024-09-20T22:10:00.999 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:00.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:01.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:01.000 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "8.15", 2024-09-20T22:10:01.000 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:10:01.000 INFO:teuthology.orchestra.run.smithi060.stdout: 7, 2024-09-20T22:10:01.000 INFO:teuthology.orchestra.run.smithi060.stdout: 6, 2024-09-20T22:10:01.000 INFO:teuthology.orchestra.run.smithi060.stdout: 4 2024-09-20T22:10:01.000 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:01.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:01.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:01.000 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "8.16", 2024-09-20T22:10:01.000 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:10:01.000 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:10:01.000 INFO:teuthology.orchestra.run.smithi060.stdout: 6, 2024-09-20T22:10:01.000 INFO:teuthology.orchestra.run.smithi060.stdout: 7 2024-09-20T22:10:01.000 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:01.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:01.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:01.001 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "8.17", 2024-09-20T22:10:01.001 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:10:01.001 INFO:teuthology.orchestra.run.smithi060.stdout: 1, 2024-09-20T22:10:01.001 INFO:teuthology.orchestra.run.smithi060.stdout: 6, 2024-09-20T22:10:01.001 INFO:teuthology.orchestra.run.smithi060.stdout: 3 2024-09-20T22:10:01.001 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:01.001 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:01.001 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:10:01.001 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_temp": [], 2024-09-20T22:10:01.001 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklist": { 2024-09-20T22:10:01.001 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/890154710": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:10:01.001 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3386526350": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:10:01.001 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/604714621": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:10:01.002 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/604714621": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:10:01.002 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/3151907776": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:10:01.002 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/956264518": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:10:01.002 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/4150585713": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:10:01.002 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3311255314": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:10:01.002 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3965541528": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:10:01.002 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/3734574621": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:10:01.002 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1729862661": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:10:01.002 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/3734574621": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:10:01.002 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3927294026": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:10:01.002 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/3151907776": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:10:01.002 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1871351181": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:10:01.002 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1854455098": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:10:01.002 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/2221262203": "2024-09-21T22:03:19.518405+0000" 2024-09-20T22:10:01.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:01.003 INFO:teuthology.orchestra.run.smithi060.stdout: "range_blocklist": {}, 2024-09-20T22:10:01.003 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profiles": { 2024-09-20T22:10:01.003 INFO:teuthology.orchestra.run.smithi060.stdout: "default": { 2024-09-20T22:10:01.003 INFO:teuthology.orchestra.run.smithi060.stdout: "crush-failure-domain": "osd", 2024-09-20T22:10:01.003 INFO:teuthology.orchestra.run.smithi060.stdout: "k": "2", 2024-09-20T22:10:01.003 INFO:teuthology.orchestra.run.smithi060.stdout: "m": "1", 2024-09-20T22:10:01.003 INFO:teuthology.orchestra.run.smithi060.stdout: "plugin": "jerasure", 2024-09-20T22:10:01.003 INFO:teuthology.orchestra.run.smithi060.stdout: "technique": "reed_sol_van" 2024-09-20T22:10:01.003 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:01.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:01.003 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps_queue": [], 2024-09-20T22:10:01.003 INFO:teuthology.orchestra.run.smithi060.stdout: "new_removed_snaps": [], 2024-09-20T22:10:01.003 INFO:teuthology.orchestra.run.smithi060.stdout: "new_purged_snaps": [], 2024-09-20T22:10:01.003 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_node_flags": {}, 2024-09-20T22:10:01.004 INFO:teuthology.orchestra.run.smithi060.stdout: "device_class_flags": {}, 2024-09-20T22:10:01.004 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode": { 2024-09-20T22:10:01.004 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode_enabled": false, 2024-09-20T22:10:01.004 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_bucket_count": 0, 2024-09-20T22:10:01.004 INFO:teuthology.orchestra.run.smithi060.stdout: "degraded_stretch_mode": 0, 2024-09-20T22:10:01.004 INFO:teuthology.orchestra.run.smithi060.stdout: "recovering_stretch_mode": 0, 2024-09-20T22:10:01.004 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode_bucket": 0 2024-09-20T22:10:01.004 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:01.004 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:10:01.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:01.005 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:01.005 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:01.106 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:01.142 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:01.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:01.143 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:01.244 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:01.288 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:01.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:01.289 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:01.390 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:01.422 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:01.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:01.423 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:01.489 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:01 smithi060 ceph-mon[29694]: pgmap v369: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.6 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 322 op/s; 17284/274179 objects misplaced (6.304%) 2024-09-20T22:10:01.490 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:01 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:10:01.490 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:01 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:10:01.490 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:01 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2786663958' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:01.490 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:01 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1113482871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-20T22:10:01.490 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:01 smithi060 ceph-mon[40290]: pgmap v369: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.6 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 322 op/s; 17284/274179 objects misplaced (6.304%) 2024-09-20T22:10:01.490 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:01 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:10:01.490 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:01 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:10:01.490 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:01 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2786663958' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:01.491 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:01 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1113482871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-20T22:10:01.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:01 smithi067 ceph-mon[33511]: pgmap v369: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.6 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 322 op/s; 17284/274179 objects misplaced (6.304%) 2024-09-20T22:10:01.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:01 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:10:01.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:01 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:10:01.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:01 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2786663958' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:01.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:01 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1113482871' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-20T22:10:01.525 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:01.559 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:10:01.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:01.560 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:01.560 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:01.661 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:01.692 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:01.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:01.693 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:01.794 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:01.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:01.824 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:01.824 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:01.925 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:01.977 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:01.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:01.978 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:02.079 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:02.115 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:02.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:02.116 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:02.216 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:02.253 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:02.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:02.255 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:02.355 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:02.386 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:02.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:02.387 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:02.488 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:02.512 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:02.513 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:10:02 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:10:02.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:02.514 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:02.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:02 smithi067 ceph-mon[33511]: osdmap e240: 8 total, 8 up, 8 in 2024-09-20T22:10:02.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:02 smithi067 ceph-mon[33511]: osdmap e241: 8 total, 8 up, 8 in 2024-09-20T22:10:02.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:02 smithi060 ceph-mon[29694]: osdmap e240: 8 total, 8 up, 8 in 2024-09-20T22:10:02.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:02 smithi060 ceph-mon[29694]: osdmap e241: 8 total, 8 up, 8 in 2024-09-20T22:10:02.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:02 smithi060 ceph-mon[40290]: osdmap e240: 8 total, 8 up, 8 in 2024-09-20T22:10:02.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:02 smithi060 ceph-mon[40290]: osdmap e241: 8 total, 8 up, 8 in 2024-09-20T22:10:02.614 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:02.672 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:02.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:02.673 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:02.773 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:02.803 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:02.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:02.804 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:02.904 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:02.949 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:02.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:02.950 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:03.050 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:03.088 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:03.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:03.089 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:03.190 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:03.224 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:03.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:03.225 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:03.326 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:03.357 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:03.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:03.358 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:03.459 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:03.493 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:03.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:03.495 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:03.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:03 smithi067 ceph-mon[33511]: pgmap v372: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.37k op/s; 14903/260037 objects misplaced (5.731%); 16 MiB/s, 249 objects/s recovering 2024-09-20T22:10:03.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:03 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:10:03.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:03 smithi060 ceph-mon[29694]: pgmap v372: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.37k op/s; 14903/260037 objects misplaced (5.731%); 16 MiB/s, 249 objects/s recovering 2024-09-20T22:10:03.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:03 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:10:03.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:03 smithi060 ceph-mon[40290]: pgmap v372: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.37k op/s; 14903/260037 objects misplaced (5.731%); 16 MiB/s, 249 objects/s recovering 2024-09-20T22:10:03.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:03 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:10:03.596 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:03.642 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:03.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:03.643 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:03.744 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:03.785 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:03.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:03.786 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:03.887 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:03.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:03.921 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:03.922 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:04.022 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:04.055 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:04.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:04.056 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:04.157 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:04.187 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:04.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:04.188 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:04.289 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:04.322 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:04.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:04.325 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:04.426 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:04.460 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:04.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:04.462 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:04.562 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:04.595 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:04.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:04.596 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:04.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:04.750 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:04.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:04.751 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:04.852 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:04.884 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:04.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:04.886 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:04.986 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:05.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:05.029 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:05.030 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:05.131 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:05.170 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:05.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:05.171 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:05.193 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:10:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:10:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:10:05.272 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:05.307 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:05.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:05.308 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:05.409 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:05.444 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:05.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:05.445 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:05.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:05 smithi067 ceph-mon[33511]: pgmap v373: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.2 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 668 B/s rd, 0 B/s wr, 1.07k op/s; 13861/256878 objects misplaced (5.396%); 16 MiB/s, 255 objects/s recovering 2024-09-20T22:10:05.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:05 smithi060 ceph-mon[29694]: pgmap v373: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.2 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 668 B/s rd, 0 B/s wr, 1.07k op/s; 13861/256878 objects misplaced (5.396%); 16 MiB/s, 255 objects/s recovering 2024-09-20T22:10:05.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:05 smithi060 ceph-mon[40290]: pgmap v373: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.2 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 668 B/s rd, 0 B/s wr, 1.07k op/s; 13861/256878 objects misplaced (5.396%); 16 MiB/s, 255 objects/s recovering 2024-09-20T22:10:05.545 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:05.577 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:05.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:05.578 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:05.661 INFO:tasks.thrashosds.thrasher:j is {'epoch': 239, 'fsid': '3df9647e-779b-11ef-baf5-efdc52797490', 'created': '2024-09-20T21:57:31.244934+0000', 'modified': '2024-09-20T22:09:56.421224+0000', 'last_up_change': '2024-09-20T22:06:53.796679+0000', 'last_in_change': '2024-09-20T22:02:21.189225+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 8, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-20T22:00:35.131125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '199', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '197', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.180000066757202, 'score_stable': 2.180000066757202, 'optimal_score': 1, 'raw_score_acting': 2.180000066757202, 'raw_score_stable': 2.180000066757202, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-20T22:02:47.974480+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6500000953674316, 'score_stable': 2.6500000953674316, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-20T22:02:48.902465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-20T22:02:50.620057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-20T22:02:52.722912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-20T22:02:54.794248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'unique_pool_1', 'create_time': '2024-09-20T22:07:51.043037+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 26, 'pg_placement_num': 26, 'pg_placement_num_target': 26, 'pg_num_target': 26, 'pg_num_pending': 26, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '217', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '215', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5399999618530273, 'score_stable': 1.5399999618530273, 'optimal_score': 1, 'raw_score_acting': 1.5399999618530273, 'raw_score_stable': 1.5399999618530273, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}], 'osds': [{'osd': 0, 'uuid': '0997c01a-a9bd-421e-abcc-fbcf7abe8512', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 234, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6802', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6803', 'nonce': 1395849550}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6804', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6805', 'nonce': 1395849550}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6808', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6809', 'nonce': 1395849550}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6806', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6807', 'nonce': 1395849550}]}, 'public_addr': '172.21.15.60:6803/1395849550', 'cluster_addr': '172.21.15.60:6805/1395849550', 'heartbeat_back_addr': '172.21.15.60:6809/1395849550', 'heartbeat_front_addr': '172.21.15.60:6807/1395849550', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c04bdcb1-d76b-47c8-a38b-c02043df6ba8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 90, 'up_from': 145, 'up_thru': 234, 'down_at': 91, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6810', 'nonce': 2970874921}, {'type': 'v1', 'addr': '172.21.15.60:6811', 'nonce': 2970874921}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6812', 'nonce': 2970874921}, {'type': 'v1', 'addr': '172.21.15.60:6813', 'nonce': 2970874921}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6816', 'nonce': 2970874921}, {'type': 'v1', 'addr': '172.21.15.60:6817', 'nonce': 2970874921}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6814', 'nonce': 2970874921}, {'type': 'v1', 'addr': '172.21.15.60:6815', 'nonce': 2970874921}]}, 'public_addr': '172.21.15.60:6811/2970874921', 'cluster_addr': '172.21.15.60:6813/2970874921', 'heartbeat_back_addr': '172.21.15.60:6817/2970874921', 'heartbeat_front_addr': '172.21.15.60:6815/2970874921', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '32c09d19-eb98-4926-a44e-e7ddee152956', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 204, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6818', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6819', 'nonce': 3698562274}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6820', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6821', 'nonce': 3698562274}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6824', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6825', 'nonce': 3698562274}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6822', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6823', 'nonce': 3698562274}]}, 'public_addr': '172.21.15.60:6819/3698562274', 'cluster_addr': '172.21.15.60:6821/3698562274', 'heartbeat_back_addr': '172.21.15.60:6825/3698562274', 'heartbeat_front_addr': '172.21.15.60:6823/3698562274', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'b1c2cb69-8565-49ac-b854-7ea7615c6660', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 235, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6826', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6827', 'nonce': 2592322449}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6828', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6829', 'nonce': 2592322449}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6832', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6833', 'nonce': 2592322449}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6830', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6831', 'nonce': 2592322449}]}, 'public_addr': '172.21.15.60:6827/2592322449', 'cluster_addr': '172.21.15.60:6829/2592322449', 'heartbeat_back_addr': '172.21.15.60:6833/2592322449', 'heartbeat_front_addr': '172.21.15.60:6831/2592322449', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '04a547d6-fc48-43b7-a8d7-18df8a83aa96', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 234, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6800', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6801', 'nonce': 3243564239}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6802', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6803', 'nonce': 3243564239}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6806', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6807', 'nonce': 3243564239}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6804', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6805', 'nonce': 3243564239}]}, 'public_addr': '172.21.15.67:6801/3243564239', 'cluster_addr': '172.21.15.67:6803/3243564239', 'heartbeat_back_addr': '172.21.15.67:6807/3243564239', 'heartbeat_front_addr': '172.21.15.67:6805/3243564239', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4763aa10-c367-485b-b525-e5241e642b86', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 223, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6808', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6809', 'nonce': 1622258887}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6810', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6811', 'nonce': 1622258887}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6814', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6815', 'nonce': 1622258887}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6812', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6813', 'nonce': 1622258887}]}, 'public_addr': '172.21.15.67:6809/1622258887', 'cluster_addr': '172.21.15.67:6811/1622258887', 'heartbeat_back_addr': '172.21.15.67:6815/1622258887', 'heartbeat_front_addr': '172.21.15.67:6813/1622258887', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'a359116e-c013-4e2f-8b79-86219516ee16', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 48, 'up_thru': 235, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6816', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6817', 'nonce': 1187114}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6818', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6819', 'nonce': 1187114}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6822', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6823', 'nonce': 1187114}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6820', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6821', 'nonce': 1187114}]}, 'public_addr': '172.21.15.67:6817/1187114', 'cluster_addr': '172.21.15.67:6819/1187114', 'heartbeat_back_addr': '172.21.15.67:6823/1187114', 'heartbeat_front_addr': '172.21.15.67:6821/1187114', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a4bf4188-f133-4a31-8897-d7e8182248d7', 'up': 1, 'in': 1, 'weight': 0.732696533203125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 57, 'up_thru': 232, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6824', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6825', 'nonce': 3021842840}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6826', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6827', 'nonce': 3021842840}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6830', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6831', 'nonce': 3021842840}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6828', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6829', 'nonce': 3021842840}]}, 'public_addr': '172.21.15.67:6825/3021842840', 'cluster_addr': '172.21.15.67:6827/3021842840', 'heartbeat_back_addr': '172.21.15.67:6831/3021842840', 'heartbeat_front_addr': '172.21.15.67:6829/3021842840', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:09:40.512797+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:09:20.725454+0000', 'dead_epoch': 90}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:09:39.953596+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:08:37.826751+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:08:55.939475+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:09:21.083547+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:09:25.296562+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:09:27.456118+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '6.13', 'osds': [0, 7, 6]}], 'pg_upmap_items': [{'pgid': '3.1a', 'mappings': [{'from': 4, 'to': 5}]}, {'pgid': '3.1c', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.1b', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '6.7', 'mappings': [{'from': 3, 'to': 0}]}, {'pgid': '6.9', 'mappings': [{'from': 7, 'to': 1}]}, {'pgid': '6.b', 'mappings': [{'from': 3, 'to': 0}]}, {'pgid': '6.d', 'mappings': [{'from': 5, 'to': 6}]}, {'pgid': '6.11', 'mappings': [{'from': 3, 'to': 1}]}, {'pgid': '6.19', 'mappings': [{'from': 3, 'to': 6}, {'from': 5, 'to': 0}]}], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '8.11', 'osds': [0, 1, 2]}, {'pgid': '8.15', 'osds': [7, 6, 4]}, {'pgid': '8.16', 'osds': [0, 6, 7]}, {'pgid': '8.17', 'osds': [1, 6, 3]}], 'primary_temp': [], 'blocklist': {'172.21.15.60:0/890154710': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:0/3386526350': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:6801/604714621': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:6800/604714621': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:6800/3151907776': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/956264518': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/4150585713': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/3311255314': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:0/3965541528': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:6800/3734574621': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:0/1729862661': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:6801/3734574621': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:0/3927294026': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:6801/3151907776': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/1871351181': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:0/1854455098': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:0/2221262203': '2024-09-21T22:03:19.518405+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-20T22:10:05.661 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:10:05.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:05.753 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:05.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:05.754 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:05.855 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:05.883 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:05.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:05.884 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:05.963 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:10:05.964 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:10:05.984 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:05.990 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:06.043 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:06.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:06.044 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:06.145 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:06.183 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:06.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:06.184 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:06.285 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:06.327 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:06.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:06.328 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:06.429 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:06.467 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:06.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:06.467 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:06.519 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:10:06.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:10:06.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:10:06.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:10:06.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:10:06.568 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:06.637 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:06.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:06.638 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:06.738 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:06.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:10:06.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:10:06.779 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:06.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:06.780 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:06.880 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:06.884 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T22:10:06.909 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:06.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:06.910 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:07.010 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:07.047 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:07.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:07.048 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:07.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:07.219 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:07.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:07.220 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:07.320 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:07.356 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:07.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:07.357 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:07.458 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:07.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:07.491 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:07.491 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:07.592 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:07.656 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:07.686 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:07.696 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:07.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:07.697 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:07.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:07 smithi067 ceph-mon[33511]: pgmap v374: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.02k op/s; 13681/249708 objects misplaced (5.479%); 7.4 MiB/s, 118 objects/s recovering 2024-09-20T22:10:07.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:07 smithi060 ceph-mon[29694]: pgmap v374: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.02k op/s; 13681/249708 objects misplaced (5.479%); 7.4 MiB/s, 118 objects/s recovering 2024-09-20T22:10:07.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:07 smithi060 ceph-mon[40290]: pgmap v374: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 5.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.02k op/s; 13681/249708 objects misplaced (5.479%); 7.4 MiB/s, 118 objects/s recovering 2024-09-20T22:10:07.798 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:07.846 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:07.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:07.847 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:07.939 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:07.947 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:07.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:07.993 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:07.993 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:08.094 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:08.121 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:08.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:08.122 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:08.222 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:08.251 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:08.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:08.252 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:08.353 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:08.397 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:08.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:08.398 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:08.450 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:10:08.499 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:08.541 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:08.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:08.542 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:08.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:08 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3432745377' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:10:08.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:08 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:10:08.630 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:08 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3432745377' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:10:08.630 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:08 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:10:08.643 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:08.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:08.684 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:08.684 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:08.719 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:10:08.719 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:10:08.720 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:10:08.720 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:10:08.720 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:10:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:08 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3432745377' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:10:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:08 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:10:08.784 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:08.813 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:08.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:08.814 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:08.914 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:08.946 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:08.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:08.947 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:09.048 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:09.071 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:09.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:09.072 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:09.172 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:09.195 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:09.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:09.195 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:09.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:09.318 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:09.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:09.319 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:09.420 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:09.443 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:09.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:09.444 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:09.544 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:09.576 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:09.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:09.577 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:09.633 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:09 smithi060 ceph-mon[29694]: pgmap v375: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 4.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.46k op/s; 10328/239124 objects misplaced (4.319%); 16 MiB/s, 248 objects/s recovering 2024-09-20T22:10:09.634 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:09 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:10:09.634 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:09 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:10:09.634 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:09 smithi060 ceph-mon[29694]: osdmap e242: 8 total, 8 up, 8 in 2024-09-20T22:10:09.634 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:09 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2858709366' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:09.634 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:09 smithi060 ceph-mon[40290]: pgmap v375: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 4.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.46k op/s; 10328/239124 objects misplaced (4.319%); 16 MiB/s, 248 objects/s recovering 2024-09-20T22:10:09.634 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:09 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:10:09.634 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:09 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:10:09.634 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:09 smithi060 ceph-mon[40290]: osdmap e242: 8 total, 8 up, 8 in 2024-09-20T22:10:09.634 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:09 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2858709366' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:09.678 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:09.706 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:09.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:09.707 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:09 smithi067 ceph-mon[33511]: pgmap v375: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 4.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.46k op/s; 10328/239124 objects misplaced (4.319%); 16 MiB/s, 248 objects/s recovering 2024-09-20T22:10:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:09 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:10:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:09 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:10:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:09 smithi067 ceph-mon[33511]: osdmap e242: 8 total, 8 up, 8 in 2024-09-20T22:10:09.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:09 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2858709366' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:09.809 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:09.834 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:09.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:09.835 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:09.936 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:09.958 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:09.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:09.959 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:10.036 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:10:10.036 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:10:10.036 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:10:10.036 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:10:10.036 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:10:10.036 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:10:10.036 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:10:10.037 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:10:10.037 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:10:10.037 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:10:10.037 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:10:10.037 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:10:10.037 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:10:10.037 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:10:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:10:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:10:10.061 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:10:10.063 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:10:10.064 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:10.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:10.088 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:10.088 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:10.189 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:10.212 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:10.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:10.213 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:10.314 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:10.337 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:10.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:10.338 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:10.439 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:10.465 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:10.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:10.466 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:10.567 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:10.599 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:10.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:10.601 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:10.701 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:10.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:10.735 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:10.735 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:10 smithi067 ceph-mon[33511]: osdmap e243: 8 total, 8 up, 8 in 2024-09-20T22:10:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:10 smithi067 ceph-mon[33511]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:10:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:10 smithi067 ceph-mon[33511]: pgmap v378: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 4.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 871 op/s; 10328/239124 objects misplaced (4.319%); 12 MiB/s, 187 objects/s recovering 2024-09-20T22:10:10.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:10 smithi060 ceph-mon[29694]: osdmap e243: 8 total, 8 up, 8 in 2024-09-20T22:10:10.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:10 smithi060 ceph-mon[29694]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:10:10.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:10 smithi060 ceph-mon[29694]: pgmap v378: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 4.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 871 op/s; 10328/239124 objects misplaced (4.319%); 12 MiB/s, 187 objects/s recovering 2024-09-20T22:10:10.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:10 smithi060 ceph-mon[40290]: osdmap e243: 8 total, 8 up, 8 in 2024-09-20T22:10:10.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:10 smithi060 ceph-mon[40290]: from='client.15387 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:10:10.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:10 smithi060 ceph-mon[40290]: pgmap v378: 168 pgs: 2 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 164 active+clean; 4.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 871 op/s; 10328/239124 objects misplaced (4.319%); 12 MiB/s, 187 objects/s recovering 2024-09-20T22:10:10.836 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:10.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:10.875 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:10.875 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:10.976 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:11.016 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:11.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:11.017 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:11.118 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:11.152 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:11.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:11.153 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:11.254 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:11.289 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:11.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:11.290 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:11.391 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:11.392 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:10:11.430 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:11.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:11.431 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:11.531 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:11.541 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-09-20T22:10:09.445181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5211241854,"num_objects":79708,"num_object_clones":0,"num_object_copies":239124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10328,"num_objects_unfound":0,"num_objects_dirty":79708,"num_whiteouts":0,"num_read":1176,"num_read_kb":945,"num_write":126065,"num_write_kb":5632389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22782,"num_bytes_recovered":1489301021,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157877,"ondisk_log_size":157877,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":516,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":19480104,"kb_used_data":16638696,"kb_used_omap":15,"kb_used_meta":2841136,"kb_avail":730481112,"statfs":{"total":767960285184,"available":748012658688,"internally_reserved":0,"allocated":17038024704,"data_stored":17029203969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":2909324051},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-539230208,"num_objects":-8228,"num_object_clones":0,"num_object_copies":-24684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-6765,"num_objects_unfound":0,"num_objects_dirty":-8228,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":8228,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1988,"num_bytes_recovered":130285568,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.012679"},"pg_stats":[{"pgid":"8.14","version":"241'8127","reported_seq":15116,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:05.619673+0000","last_change":"2024-09-20T22:09:47.155778+0000","last_active":"2024-09-20T22:10:05.619673+0000","last_peered":"2024-09-20T22:10:05.619673+0000","last_clean":"2024-09-20T22:10:05.619673+0000","last_became_active":"2024-09-20T22:09:46.430707+0000","last_became_peered":"2024-09-20T22:09:46.430707+0000","last_unstale":"2024-09-20T22:10:05.619673+0000","last_undegraded":"2024-09-20T22:10:05.619673+0000","last_fullsized":"2024-09-20T22:10:05.619673+0000","mapping_epoch":234,"log_start":"183'1671","ondisk_log_start":"183'1671","created":215,"last_epoch_clean":235,"parent":"0.0","parent_split_bits":5,"last_scrub":"235'7486","last_scrub_stamp":"2024-09-20T22:09:47.155594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:09:47.155594+0000","objects_scrubbed":3099,"log_size":6456,"log_dups_size":0,"ondisk_log_size":6456,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:56.972924+0000","scrub_duration":449,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161087488,"num_objects":2458,"num_object_clones":0,"num_object_copies":7374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":641,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2],"acting":[4,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":383,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.431575+0000","last_change":"2024-09-20T22:06:48.018571+0000","last_active":"2024-09-20T22:10:01.431575+0000","last_peered":"2024-09-20T22:10:01.431575+0000","last_clean":"2024-09-20T22:10:01.431575+0000","last_became_active":"2024-09-20T22:02:50.109930+0000","last_became_peered":"2024-09-20T22:02:50.109930+0000","last_unstale":"2024-09-20T22:10:01.431575+0000","last_undegraded":"2024-09-20T22:10:01.431575+0000","last_fullsized":"2024-09-20T22:10:01.431575+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_clean_scrub_stamp":"2024-09-20T22:06:48.018157+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"71'9","reported_seq":404,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.441926+0000","last_change":"2024-09-20T22:07:55.492953+0000","last_active":"2024-09-20T22:10:01.441926+0000","last_peered":"2024-09-20T22:10:01.441926+0000","last_clean":"2024-09-20T22:10:01.441926+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:10:01.441926+0000","last_undegraded":"2024-09-20T22:10:01.441926+0000","last_fullsized":"2024-09-20T22:10:01.441926+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:55.492841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.492841+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.733043+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":263,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.443425+0000","last_change":"2024-09-20T22:06:54.843328+0000","last_active":"2024-09-20T22:10:01.443425+0000","last_peered":"2024-09-20T22:10:01.443425+0000","last_clean":"2024-09-20T22:10:01.443425+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:10:01.443425+0000","last_undegraded":"2024-09-20T22:10:01.443425+0000","last_fullsized":"2024-09-20T22:10:01.443425+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:11.178591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:04:11.178591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":372,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.441862+0000","last_change":"2024-09-20T22:07:14.535725+0000","last_active":"2024-09-20T22:10:01.441862+0000","last_peered":"2024-09-20T22:10:01.441862+0000","last_clean":"2024-09-20T22:10:01.441862+0000","last_became_active":"2024-09-20T22:06:54.830769+0000","last_became_peered":"2024-09-20T22:06:54.830769+0000","last_unstale":"2024-09-20T22:10:01.441862+0000","last_undegraded":"2024-09-20T22:10:01.441862+0000","last_fullsized":"2024-09-20T22:10:01.441862+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.535618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.535618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.732923+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.15","version":"241'8441","reported_seq":8599,"reported_epoch":241,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T22:10:05.594941+0000","last_change":"2024-09-20T22:09:42.496841+0000","last_active":"2024-09-20T22:10:05.594941+0000","last_peered":"2024-09-20T22:10:05.594941+0000","last_clean":"2024-09-20T22:09:10.532398+0000","last_became_active":"2024-09-20T22:09:12.486597+0000","last_became_peered":"2024-09-20T22:09:12.486597+0000","last_unstale":"2024-09-20T22:10:05.594941+0000","last_undegraded":"2024-09-20T22:10:05.594941+0000","last_fullsized":"2024-09-20T22:10:05.594941+0000","mapping_epoch":218,"log_start":"182'1400","ondisk_log_start":"182'1400","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":7041,"log_dups_size":0,"ondisk_log_size":7041,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168263680,"num_objects":2568,"num_object_clones":0,"num_object_copies":7704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2568,"num_objects_unfound":0,"num_objects_dirty":2568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5084,"num_write_kb":244832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,6,4],"avail_no_missing":["7","4","6"],"object_location_counts":[{"shards":"4,6,7","objects":2568}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":371,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.453637+0000","last_change":"2024-09-20T22:08:41.770593+0000","last_active":"2024-09-20T22:10:01.453637+0000","last_peered":"2024-09-20T22:10:01.453637+0000","last_clean":"2024-09-20T22:10:01.453637+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:10:01.453637+0000","last_undegraded":"2024-09-20T22:10:01.453637+0000","last_fullsized":"2024-09-20T22:10:01.453637+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:08:41.770351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:41.770351+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:07.625199+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":383,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.454317+0000","last_change":"2024-09-20T22:09:00.839346+0000","last_active":"2024-09-20T22:10:01.454317+0000","last_peered":"2024-09-20T22:10:01.454317+0000","last_clean":"2024-09-20T22:10:01.454317+0000","last_became_active":"2024-09-20T22:02:50.107154+0000","last_became_peered":"2024-09-20T22:02:50.107154+0000","last_unstale":"2024-09-20T22:10:01.454317+0000","last_undegraded":"2024-09-20T22:10:01.454317+0000","last_fullsized":"2024-09-20T22:10:01.454317+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:00.839239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:00.839239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:07.151739+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"71'15","reported_seq":403,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.453295+0000","last_change":"2024-09-20T22:08:39.705641+0000","last_active":"2024-09-20T22:10:01.453295+0000","last_peered":"2024-09-20T22:10:01.453295+0000","last_clean":"2024-09-20T22:10:01.453295+0000","last_became_active":"2024-09-20T22:02:52.282101+0000","last_became_peered":"2024-09-20T22:02:52.282101+0000","last_unstale":"2024-09-20T22:10:01.453295+0000","last_undegraded":"2024-09-20T22:10:01.453295+0000","last_fullsized":"2024-09-20T22:10:01.453295+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:08:39.705510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:39.705510+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:08.518121+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":376,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.441736+0000","last_change":"2024-09-20T22:07:17.494965+0000","last_active":"2024-09-20T22:10:01.441736+0000","last_peered":"2024-09-20T22:10:01.441736+0000","last_clean":"2024-09-20T22:10:01.441736+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:10:01.441736+0000","last_undegraded":"2024-09-20T22:10:01.441736+0000","last_fullsized":"2024-09-20T22:10:01.441736+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:17.494852+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:17.494852+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.732982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"241'8322","reported_seq":8481,"reported_epoch":241,"state":"active+remapped+backfill_toofull","last_fresh":"2024-09-20T22:10:06.547152+0000","last_change":"2024-09-20T22:09:42.550574+0000","last_active":"2024-09-20T22:10:06.547152+0000","last_peered":"2024-09-20T22:10:06.547152+0000","last_clean":"2024-09-20T22:09:10.566074+0000","last_became_active":"2024-09-20T22:09:12.508336+0000","last_became_peered":"2024-09-20T22:09:12.508336+0000","last_unstale":"2024-09-20T22:10:06.547152+0000","last_undegraded":"2024-09-20T22:10:06.547152+0000","last_fullsized":"2024-09-20T22:10:06.547152+0000","mapping_epoch":218,"log_start":"183'1626","ondisk_log_start":"183'1626","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":6696,"log_dups_size":0,"ondisk_log_size":6696,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":158564352,"num_objects":2420,"num_object_clones":0,"num_object_copies":7260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":7260,"num_objects_unfound":0,"num_objects_dirty":2420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5042,"num_write_kb":238752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[0,6,7],"avail_no_missing":["0","6","7"],"object_location_counts":[{"shards":"0,6,7","objects":2420}],"blocked_by":[],"up_primary":3,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":398,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.436315+0000","last_change":"2024-09-20T22:08:35.146849+0000","last_active":"2024-09-20T22:10:01.436315+0000","last_peered":"2024-09-20T22:10:01.436315+0000","last_clean":"2024-09-20T22:10:01.436315+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:10:01.436315+0000","last_undegraded":"2024-09-20T22:10:01.436315+0000","last_fullsized":"2024-09-20T22:10:01.436315+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:08:35.146554+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"71'9","reported_seq":403,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.443272+0000","last_change":"2024-09-20T22:09:47.733504+0000","last_active":"2024-09-20T22:10:01.443272+0000","last_peered":"2024-09-20T22:10:01.443272+0000","last_clean":"2024-09-20T22:10:01.443272+0000","last_became_active":"2024-09-20T22:02:52.279526+0000","last_became_peered":"2024-09-20T22:02:52.279526+0000","last_unstale":"2024-09-20T22:10:01.443272+0000","last_undegraded":"2024-09-20T22:10:01.443272+0000","last_fullsized":"2024-09-20T22:10:01.443272+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:09:47.733368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:09:47.733368+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:56.454959+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":396,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.435817+0000","last_change":"2024-09-20T22:08:51.456300+0000","last_active":"2024-09-20T22:10:01.435817+0000","last_peered":"2024-09-20T22:10:01.435817+0000","last_clean":"2024-09-20T22:10:01.435817+0000","last_became_active":"2024-09-20T22:08:51.455924+0000","last_became_peered":"2024-09-20T22:08:51.455924+0000","last_unstale":"2024-09-20T22:10:01.435817+0000","last_undegraded":"2024-09-20T22:10:01.435817+0000","last_fullsized":"2024-09-20T22:10:01.435817+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:42.143066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:42.143066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.990159+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":371,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.430607+0000","last_change":"2024-09-20T22:07:15.416312+0000","last_active":"2024-09-20T22:10:01.430607+0000","last_peered":"2024-09-20T22:10:01.430607+0000","last_clean":"2024-09-20T22:10:01.430607+0000","last_became_active":"2024-09-20T22:06:54.841095+0000","last_became_peered":"2024-09-20T22:06:54.841095+0000","last_unstale":"2024-09-20T22:10:01.430607+0000","last_undegraded":"2024-09-20T22:10:01.430607+0000","last_fullsized":"2024-09-20T22:10:01.430607+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:15.416176+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.416176+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"241'8340","reported_seq":17026,"reported_epoch":241,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T22:10:07.862899+0000","last_change":"2024-09-20T22:09:42.535829+0000","last_active":"2024-09-20T22:10:07.862899+0000","last_peered":"2024-09-20T22:10:07.862899+0000","last_clean":"2024-09-20T22:09:10.546593+0000","last_became_active":"2024-09-20T22:09:12.517484+0000","last_became_peered":"2024-09-20T22:09:12.517484+0000","last_unstale":"2024-09-20T22:10:07.862899+0000","last_undegraded":"2024-09-20T22:10:07.862899+0000","last_fullsized":"2024-09-20T22:10:07.862899+0000","mapping_epoch":218,"log_start":"185'2256","ondisk_log_start":"185'2256","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":6084,"log_dups_size":0,"ondisk_log_size":6084,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161382400,"num_objects":2463,"num_object_clones":0,"num_object_copies":7389,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":116,"num_objects_unfound":0,"num_objects_dirty":2463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5035,"num_write_kb":239904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2843,"num_bytes_recovered":186318848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[1,6,3],"avail_no_missing":["1","3","6"],"object_location_counts":[{"shards":"1,3,6","objects":2463}],"blocked_by":[],"up_primary":7,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":403,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.436984+0000","last_change":"2024-09-20T22:07:59.947914+0000","last_active":"2024-09-20T22:10:01.436984+0000","last_peered":"2024-09-20T22:10:01.436984+0000","last_clean":"2024-09-20T22:10:01.436984+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:10:01.436984+0000","last_undegraded":"2024-09-20T22:10:01.436984+0000","last_fullsized":"2024-09-20T22:10:01.436984+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:59.947741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:59.947741+0000","last_clean_scrub_stamp":"2024-09-20T22:07:59.947741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"71'5","reported_seq":410,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.442561+0000","last_change":"2024-09-20T22:08:22.151586+0000","last_active":"2024-09-20T22:10:01.442561+0000","last_peered":"2024-09-20T22:10:01.442561+0000","last_clean":"2024-09-20T22:10:01.442561+0000","last_became_active":"2024-09-20T22:08:22.144364+0000","last_became_peered":"2024-09-20T22:08:22.144364+0000","last_unstale":"2024-09-20T22:10:01.442561+0000","last_undegraded":"2024-09-20T22:10:01.442561+0000","last_fullsized":"2024-09-20T22:10:01.442561+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'5","last_scrub_stamp":"2024-09-20T22:08:01.462581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:01.462581+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.733075+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":384,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.435773+0000","last_change":"2024-09-20T22:07:28.654301+0000","last_active":"2024-09-20T22:10:01.435773+0000","last_peered":"2024-09-20T22:10:01.435773+0000","last_clean":"2024-09-20T22:10:01.435773+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:10:01.435773+0000","last_undegraded":"2024-09-20T22:10:01.435773+0000","last_fullsized":"2024-09-20T22:10:01.435773+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:28.654202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:28.654202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":325,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.433976+0000","last_change":"2024-09-20T22:08:41.836934+0000","last_active":"2024-09-20T22:10:01.433976+0000","last_peered":"2024-09-20T22:10:01.433976+0000","last_clean":"2024-09-20T22:10:01.433976+0000","last_became_active":"2024-09-20T22:06:55.824168+0000","last_became_peered":"2024-09-20T22:06:55.824168+0000","last_unstale":"2024-09-20T22:10:01.433976+0000","last_undegraded":"2024-09-20T22:10:01.433976+0000","last_fullsized":"2024-09-20T22:10:01.433976+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:41.836827+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:08:41.836827+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:04.667178+0000","scrub_duration":37,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.10","version":"241'8291","reported_seq":19192,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:05.638036+0000","last_change":"2024-09-20T22:09:42.487436+0000","last_active":"2024-09-20T22:10:05.638036+0000","last_peered":"2024-09-20T22:10:05.638036+0000","last_clean":"2024-09-20T22:10:05.638036+0000","last_became_active":"2024-09-20T22:09:42.443918+0000","last_became_peered":"2024-09-20T22:09:42.443918+0000","last_unstale":"2024-09-20T22:10:05.638036+0000","last_undegraded":"2024-09-20T22:10:05.638036+0000","last_fullsized":"2024-09-20T22:10:05.638036+0000","mapping_epoch":232,"log_start":"184'1794","ondisk_log_start":"184'1794","created":215,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":6497,"log_dups_size":0,"ondisk_log_size":6497,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:09.052451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165412864,"num_objects":2524,"num_object_clones":0,"num_object_copies":7572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4972,"num_write_kb":239872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3581,"num_bytes_recovered":234487808,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,4],"acting":[7,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":380,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.445067+0000","last_change":"2024-09-20T22:09:51.759271+0000","last_active":"2024-09-20T22:10:01.445067+0000","last_peered":"2024-09-20T22:10:01.445067+0000","last_clean":"2024-09-20T22:10:01.445067+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:10:01.445067+0000","last_undegraded":"2024-09-20T22:10:01.445067+0000","last_fullsized":"2024-09-20T22:10:01.445067+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:51.759042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:09:51.759042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:18.845958+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":382,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.431574+0000","last_change":"2024-09-20T22:08:34.815041+0000","last_active":"2024-09-20T22:10:01.431574+0000","last_peered":"2024-09-20T22:10:01.431574+0000","last_clean":"2024-09-20T22:10:01.431574+0000","last_became_active":"2024-09-20T22:02:50.111517+0000","last_became_peered":"2024-09-20T22:02:50.111517+0000","last_unstale":"2024-09-20T22:10:01.431574+0000","last_undegraded":"2024-09-20T22:10:01.431574+0000","last_fullsized":"2024-09-20T22:10:01.431574+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:08:34.814930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"195'22","reported_seq":426,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.444939+0000","last_change":"2024-09-20T22:08:39.358759+0000","last_active":"2024-09-20T22:10:01.444939+0000","last_peered":"2024-09-20T22:10:01.444939+0000","last_clean":"2024-09-20T22:10:01.444939+0000","last_became_active":"2024-09-20T22:06:54.846111+0000","last_became_peered":"2024-09-20T22:06:54.846111+0000","last_unstale":"2024-09-20T22:10:01.444939+0000","last_undegraded":"2024-09-20T22:10:01.444939+0000","last_fullsized":"2024-09-20T22:10:01.444939+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'22","last_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:08:39.355859+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:01.487736+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":221,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.433275+0000","last_change":"2024-09-20T22:08:00.369883+0000","last_active":"2024-09-20T22:10:01.433275+0000","last_peered":"2024-09-20T22:10:01.433275+0000","last_clean":"2024-09-20T22:10:01.433275+0000","last_became_active":"2024-09-20T22:06:54.848505+0000","last_became_peered":"2024-09-20T22:06:54.848505+0000","last_unstale":"2024-09-20T22:10:01.433275+0000","last_undegraded":"2024-09-20T22:10:01.433275+0000","last_fullsized":"2024-09-20T22:10:01.433275+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:00.369767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:00.369767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"241'8286","reported_seq":19353,"reported_epoch":241,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T22:10:06.576165+0000","last_change":"2024-09-20T22:09:42.640287+0000","last_active":"2024-09-20T22:10:06.576165+0000","last_peered":"2024-09-20T22:10:06.576165+0000","last_clean":"2024-09-20T22:09:10.466832+0000","last_became_active":"2024-09-20T22:09:12.526523+0000","last_became_peered":"2024-09-20T22:09:12.526523+0000","last_unstale":"2024-09-20T22:10:06.576165+0000","last_undegraded":"2024-09-20T22:10:06.576165+0000","last_fullsized":"2024-09-20T22:10:06.576165+0000","mapping_epoch":218,"log_start":"185'2185","ondisk_log_start":"185'2185","created":215,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":6101,"log_dups_size":0,"ondisk_log_size":6101,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":162103296,"num_objects":2474,"num_object_clones":0,"num_object_copies":7422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":384,"num_objects_unfound":0,"num_objects_dirty":2474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5014,"num_write_kb":239584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2728,"num_bytes_recovered":178782208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[0,1,2],"avail_no_missing":["0","1","2"],"object_location_counts":[{"shards":"0,1,2","objects":2474}],"blocked_by":[],"up_primary":5,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":379,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.454308+0000","last_change":"2024-09-20T22:09:52.318025+0000","last_active":"2024-09-20T22:10:01.454308+0000","last_peered":"2024-09-20T22:10:01.454308+0000","last_clean":"2024-09-20T22:10:01.454308+0000","last_became_active":"2024-09-20T22:02:56.347561+0000","last_became_peered":"2024-09-20T22:02:56.347561+0000","last_unstale":"2024-09-20T22:10:01.454308+0000","last_undegraded":"2024-09-20T22:10:01.454308+0000","last_fullsized":"2024-09-20T22:10:01.454308+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:52.317931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:52.317931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:12.366536+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":333,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.435229+0000","last_change":"2024-09-20T22:08:28.058586+0000","last_active":"2024-09-20T22:10:01.435229+0000","last_peered":"2024-09-20T22:10:01.435229+0000","last_clean":"2024-09-20T22:10:01.435229+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:10:01.435229+0000","last_undegraded":"2024-09-20T22:10:01.435229+0000","last_fullsized":"2024-09-20T22:10:01.435229+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:28.058516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:28.058516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"71'12","reported_seq":402,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.453583+0000","last_change":"2024-09-20T22:07:55.383903+0000","last_active":"2024-09-20T22:10:01.453583+0000","last_peered":"2024-09-20T22:10:01.453583+0000","last_clean":"2024-09-20T22:10:01.453583+0000","last_became_active":"2024-09-20T22:06:54.830946+0000","last_became_peered":"2024-09-20T22:06:54.830946+0000","last_unstale":"2024-09-20T22:10:01.453583+0000","last_undegraded":"2024-09-20T22:10:01.453583+0000","last_fullsized":"2024-09-20T22:10:01.453583+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:55.383818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.383818+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":375,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.441828+0000","last_change":"2024-09-20T22:08:36.461149+0000","last_active":"2024-09-20T22:10:01.441828+0000","last_peered":"2024-09-20T22:10:01.441828+0000","last_clean":"2024-09-20T22:10:01.441828+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:10:01.441828+0000","last_undegraded":"2024-09-20T22:10:01.441828+0000","last_fullsized":"2024-09-20T22:10:01.441828+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:36.460897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:36.460897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.733153+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.12","version":"241'8144","reported_seq":21882,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:05.676446+0000","last_change":"2024-09-20T22:09:47.438259+0000","last_active":"2024-09-20T22:10:05.676446+0000","last_peered":"2024-09-20T22:10:05.676446+0000","last_clean":"2024-09-20T22:10:05.676446+0000","last_became_active":"2024-09-20T22:09:47.430478+0000","last_became_peered":"2024-09-20T22:09:47.430478+0000","last_unstale":"2024-09-20T22:10:05.676446+0000","last_undegraded":"2024-09-20T22:10:05.676446+0000","last_fullsized":"2024-09-20T22:10:05.676446+0000","mapping_epoch":235,"log_start":"185'2032","ondisk_log_start":"185'2032","created":215,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":6112,"log_dups_size":0,"ondisk_log_size":6112,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:10.169227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165937152,"num_objects":2532,"num_object_clones":0,"num_object_copies":7596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4868,"num_write_kb":236800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3394,"num_bytes_recovered":222429184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":373,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.435585+0000","last_change":"2024-09-20T22:08:34.802125+0000","last_active":"2024-09-20T22:10:01.435585+0000","last_peered":"2024-09-20T22:10:01.435585+0000","last_clean":"2024-09-20T22:10:01.435585+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:10:01.435585+0000","last_undegraded":"2024-09-20T22:10:01.435585+0000","last_fullsized":"2024-09-20T22:10:01.435585+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:08:34.801894+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:08:34.801894+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"64'1","reported_seq":222,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.443431+0000","last_change":"2024-09-20T22:09:02.517233+0000","last_active":"2024-09-20T22:10:01.443431+0000","last_peered":"2024-09-20T22:10:01.443431+0000","last_clean":"2024-09-20T22:10:01.443431+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:10:01.443431+0000","last_undegraded":"2024-09-20T22:10:01.443431+0000","last_fullsized":"2024-09-20T22:10:01.443431+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:09:02.516942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:02.516942+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:20.902402+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"71'10","reported_seq":411,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.432365+0000","last_change":"2024-09-20T22:08:52.434346+0000","last_active":"2024-09-20T22:10:01.432365+0000","last_peered":"2024-09-20T22:10:01.432365+0000","last_clean":"2024-09-20T22:10:01.432365+0000","last_became_active":"2024-09-20T22:08:52.424697+0000","last_became_peered":"2024-09-20T22:08:52.424697+0000","last_unstale":"2024-09-20T22:10:01.432365+0000","last_undegraded":"2024-09-20T22:10:01.432365+0000","last_fullsized":"2024-09-20T22:10:01.432365+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:08:30.868781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:30.868781+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"208'9","reported_seq":492,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.438924+0000","last_change":"2024-09-20T22:08:51.459956+0000","last_active":"2024-09-20T22:10:01.438924+0000","last_peered":"2024-09-20T22:10:01.438924+0000","last_clean":"2024-09-20T22:10:01.438924+0000","last_became_active":"2024-09-20T22:08:51.448751+0000","last_became_peered":"2024-09-20T22:08:51.448751+0000","last_unstale":"2024-09-20T22:10:01.438924+0000","last_undegraded":"2024-09-20T22:10:01.438924+0000","last_fullsized":"2024-09-20T22:10:01.438924+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:08:29.376948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:29.376948+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":386,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.435069+0000","last_change":"2024-09-20T22:09:39.334307+0000","last_active":"2024-09-20T22:10:01.435069+0000","last_peered":"2024-09-20T22:10:01.435069+0000","last_clean":"2024-09-20T22:10:01.435069+0000","last_became_active":"2024-09-20T22:08:48.452786+0000","last_became_peered":"2024-09-20T22:08:48.452786+0000","last_unstale":"2024-09-20T22:10:01.435069+0000","last_undegraded":"2024-09-20T22:10:01.435069+0000","last_fullsized":"2024-09-20T22:10:01.435069+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:09:39.334096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:09:39.334096+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:07.278009+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3","version":"241'8380","reported_seq":8529,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:07.763478+0000","last_change":"2024-09-20T22:09:09.569263+0000","last_active":"2024-09-20T22:10:07.763478+0000","last_peered":"2024-09-20T22:10:07.763478+0000","last_clean":"2024-09-20T22:10:07.763478+0000","last_became_active":"2024-09-20T22:09:09.566687+0000","last_became_peered":"2024-09-20T22:09:09.566687+0000","last_unstale":"2024-09-20T22:10:07.763478+0000","last_undegraded":"2024-09-20T22:10:07.763478+0000","last_fullsized":"2024-09-20T22:10:07.763478+0000","mapping_epoch":176,"log_start":"185'2186","ondisk_log_start":"185'2186","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":6194,"log_dups_size":0,"ondisk_log_size":6194,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:08.461396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":158990336,"num_objects":2426,"num_object_clones":0,"num_object_copies":7278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5074,"num_write_kb":240000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6],"acting":[1,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"71'15","reported_seq":149,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.444178+0000","last_change":"2024-09-20T22:09:51.530815+0000","last_active":"2024-09-20T22:10:01.444178+0000","last_peered":"2024-09-20T22:10:01.444178+0000","last_clean":"2024-09-20T22:10:01.444178+0000","last_became_active":"2024-09-20T22:08:22.145408+0000","last_became_peered":"2024-09-20T22:08:22.145408+0000","last_unstale":"2024-09-20T22:10:01.444178+0000","last_undegraded":"2024-09-20T22:10:01.444178+0000","last_fullsized":"2024-09-20T22:10:01.444178+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:09:51.529877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:09:51.529877+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:12.144008+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":384,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.453399+0000","last_change":"2024-09-20T22:07:23.292560+0000","last_active":"2024-09-20T22:10:01.453399+0000","last_peered":"2024-09-20T22:10:01.453399+0000","last_clean":"2024-09-20T22:10:01.453399+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:10:01.453399+0000","last_undegraded":"2024-09-20T22:10:01.453399+0000","last_fullsized":"2024-09-20T22:10:01.453399+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.292451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.292451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"71'8","reported_seq":483,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.445178+0000","last_change":"2024-09-20T22:09:06.534110+0000","last_active":"2024-09-20T22:10:01.445178+0000","last_peered":"2024-09-20T22:10:01.445178+0000","last_clean":"2024-09-20T22:10:01.445178+0000","last_became_active":"2024-09-20T22:02:54.294917+0000","last_became_peered":"2024-09-20T22:02:54.294917+0000","last_unstale":"2024-09-20T22:10:01.445178+0000","last_undegraded":"2024-09-20T22:10:01.445178+0000","last_fullsized":"2024-09-20T22:10:01.445178+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:09:06.534030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:06.534030+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:08.011991+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":333,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.439003+0000","last_change":"2024-09-20T22:07:52.334831+0000","last_active":"2024-09-20T22:10:01.439003+0000","last_peered":"2024-09-20T22:10:01.439003+0000","last_clean":"2024-09-20T22:10:01.439003+0000","last_became_active":"2024-09-20T22:06:55.822780+0000","last_became_peered":"2024-09-20T22:06:55.822780+0000","last_unstale":"2024-09-20T22:10:01.439003+0000","last_undegraded":"2024-09-20T22:10:01.439003+0000","last_fullsized":"2024-09-20T22:10:01.439003+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:52.334748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:52.334748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":386,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.444621+0000","last_change":"2024-09-20T22:08:59.563006+0000","last_active":"2024-09-20T22:10:01.444621+0000","last_peered":"2024-09-20T22:10:01.444621+0000","last_clean":"2024-09-20T22:10:01.444621+0000","last_became_active":"2024-09-20T22:08:48.452475+0000","last_became_peered":"2024-09-20T22:08:48.452475+0000","last_unstale":"2024-09-20T22:10:01.444621+0000","last_undegraded":"2024-09-20T22:10:01.444621+0000","last_fullsized":"2024-09-20T22:10:01.444621+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:08:59.562897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:08:59.562897+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:26.912800+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.c","version":"241'10016","reported_seq":23413,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:06.564298+0000","last_change":"2024-09-20T22:09:38.472776+0000","last_active":"2024-09-20T22:10:06.564298+0000","last_peered":"2024-09-20T22:10:06.564298+0000","last_clean":"2024-09-20T22:10:06.564298+0000","last_became_active":"2024-09-20T22:09:38.454952+0000","last_became_peered":"2024-09-20T22:09:38.454952+0000","last_unstale":"2024-09-20T22:10:06.564298+0000","last_undegraded":"2024-09-20T22:10:06.564298+0000","last_fullsized":"2024-09-20T22:10:06.564298+0000","mapping_epoch":228,"log_start":"203'5400","ondisk_log_start":"203'5400","created":176,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4616,"log_dups_size":0,"ondisk_log_size":4616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":325451776,"num_objects":4966,"num_object_clones":0,"num_object_copies":14898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10016,"num_write_kb":479424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6620,"num_bytes_recovered":433848320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.0","version":"71'18","reported_seq":434,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.453161+0000","last_change":"2024-09-20T22:08:52.436122+0000","last_active":"2024-09-20T22:10:01.453161+0000","last_peered":"2024-09-20T22:10:01.453161+0000","last_clean":"2024-09-20T22:10:01.453161+0000","last_became_active":"2024-09-20T22:08:52.421420+0000","last_became_peered":"2024-09-20T22:08:52.421420+0000","last_unstale":"2024-09-20T22:10:01.453161+0000","last_undegraded":"2024-09-20T22:10:01.453161+0000","last_fullsized":"2024-09-20T22:10:01.453161+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'18","last_scrub_stamp":"2024-09-20T22:08:40.737338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:40.737338+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":382,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.454863+0000","last_change":"2024-09-20T22:09:01.889907+0000","last_active":"2024-09-20T22:10:01.454863+0000","last_peered":"2024-09-20T22:10:01.454863+0000","last_clean":"2024-09-20T22:10:01.454863+0000","last_became_active":"2024-09-20T22:02:50.113076+0000","last_became_peered":"2024-09-20T22:02:50.113076+0000","last_unstale":"2024-09-20T22:10:01.454863+0000","last_undegraded":"2024-09-20T22:10:01.454863+0000","last_fullsized":"2024-09-20T22:10:01.454863+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:01.889795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:01.889795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:27.566891+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":390,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.442528+0000","last_change":"2024-09-20T22:09:05.535483+0000","last_active":"2024-09-20T22:10:01.442528+0000","last_peered":"2024-09-20T22:10:01.442528+0000","last_clean":"2024-09-20T22:10:01.442528+0000","last_became_active":"2024-09-20T22:08:51.455159+0000","last_became_peered":"2024-09-20T22:08:51.455159+0000","last_unstale":"2024-09-20T22:10:01.442528+0000","last_undegraded":"2024-09-20T22:10:01.442528+0000","last_fullsized":"2024-09-20T22:10:01.442528+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:05.535412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:05.535412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:23.611809+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":380,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.443183+0000","last_change":"2024-09-20T22:08:38.488454+0000","last_active":"2024-09-20T22:10:01.443183+0000","last_peered":"2024-09-20T22:10:01.443183+0000","last_clean":"2024-09-20T22:10:01.443183+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:10:01.443183+0000","last_undegraded":"2024-09-20T22:10:01.443183+0000","last_fullsized":"2024-09-20T22:10:01.443183+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:38.488310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:38.488310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.111826+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":386,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.453838+0000","last_change":"2024-09-20T22:08:58.858275+0000","last_active":"2024-09-20T22:10:01.453838+0000","last_peered":"2024-09-20T22:10:01.453838+0000","last_clean":"2024-09-20T22:10:01.453838+0000","last_became_active":"2024-09-20T22:08:48.476212+0000","last_became_peered":"2024-09-20T22:08:48.476212+0000","last_unstale":"2024-09-20T22:10:01.453838+0000","last_undegraded":"2024-09-20T22:10:01.453838+0000","last_fullsized":"2024-09-20T22:10:01.453838+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:08:58.858164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:08:58.858164+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:01.714721+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"241'9911","reported_seq":10055,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:05.693569+0000","last_change":"2024-09-20T22:07:52.940943+0000","last_active":"2024-09-20T22:10:05.693569+0000","last_peered":"2024-09-20T22:10:05.693569+0000","last_clean":"2024-09-20T22:10:05.693569+0000","last_became_active":"2024-09-20T22:07:52.940627+0000","last_became_peered":"2024-09-20T22:07:52.940627+0000","last_unstale":"2024-09-20T22:10:05.693569+0000","last_undegraded":"2024-09-20T22:10:05.693569+0000","last_fullsized":"2024-09-20T22:10:05.693569+0000","mapping_epoch":176,"log_start":"209'5666","ondisk_log_start":"209'5666","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4245,"log_dups_size":0,"ondisk_log_size":4245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":325976088,"num_objects":4975,"num_object_clones":0,"num_object_copies":14925,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4975,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9911,"num_write_kb":476289,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1","version":"71'14","reported_seq":409,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.442765+0000","last_change":"2024-09-20T22:09:03.569008+0000","last_active":"2024-09-20T22:10:01.442765+0000","last_peered":"2024-09-20T22:10:01.442765+0000","last_clean":"2024-09-20T22:10:01.442765+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:10:01.442765+0000","last_undegraded":"2024-09-20T22:10:01.442765+0000","last_fullsized":"2024-09-20T22:10:01.442765+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'14","last_scrub_stamp":"2024-09-20T22:09:03.568900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:09:03.568900+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:31.777928+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":385,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.432652+0000","last_change":"2024-09-20T22:07:58.783500+0000","last_active":"2024-09-20T22:10:01.432652+0000","last_peered":"2024-09-20T22:10:01.432652+0000","last_clean":"2024-09-20T22:10:01.432652+0000","last_became_active":"2024-09-20T22:06:54.841165+0000","last_became_peered":"2024-09-20T22:06:54.841165+0000","last_unstale":"2024-09-20T22:10:01.432652+0000","last_undegraded":"2024-09-20T22:10:01.432652+0000","last_fullsized":"2024-09-20T22:10:01.432652+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:07:58.783380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:58.783380+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"146'10","reported_seq":474,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.453016+0000","last_change":"2024-09-20T22:07:24.333937+0000","last_active":"2024-09-20T22:10:01.453016+0000","last_peered":"2024-09-20T22:10:01.453016+0000","last_clean":"2024-09-20T22:10:01.453016+0000","last_became_active":"2024-09-20T22:06:54.847742+0000","last_became_peered":"2024-09-20T22:06:54.847742+0000","last_unstale":"2024-09-20T22:10:01.453016+0000","last_undegraded":"2024-09-20T22:10:01.453016+0000","last_fullsized":"2024-09-20T22:10:01.453016+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:07:24.333809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:24.333809+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:08.222086+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":372,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.435766+0000","last_change":"2024-09-20T22:07:30.589937+0000","last_active":"2024-09-20T22:10:01.435766+0000","last_peered":"2024-09-20T22:10:01.435766+0000","last_clean":"2024-09-20T22:10:01.435766+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:10:01.435766+0000","last_undegraded":"2024-09-20T22:10:01.435766+0000","last_fullsized":"2024-09-20T22:10:01.435766+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:30.589849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:30.589849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":399,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.438500+0000","last_change":"2024-09-20T22:09:02.350959+0000","last_active":"2024-09-20T22:10:01.438500+0000","last_peered":"2024-09-20T22:10:01.438500+0000","last_clean":"2024-09-20T22:10:01.438500+0000","last_became_active":"2024-09-20T22:08:55.449793+0000","last_became_peered":"2024-09-20T22:08:55.449793+0000","last_unstale":"2024-09-20T22:10:01.438500+0000","last_undegraded":"2024-09-20T22:10:01.438500+0000","last_fullsized":"2024-09-20T22:10:01.438500+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:09:02.350748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:09:02.350748+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:20.165516+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.e","version":"241'10201","reported_seq":10344,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:06.554890+0000","last_change":"2024-09-20T22:07:52.940978+0000","last_active":"2024-09-20T22:10:06.554890+0000","last_peered":"2024-09-20T22:10:06.554890+0000","last_clean":"2024-09-20T22:10:06.554890+0000","last_became_active":"2024-09-20T22:07:52.940673+0000","last_became_peered":"2024-09-20T22:07:52.940673+0000","last_unstale":"2024-09-20T22:10:06.554890+0000","last_undegraded":"2024-09-20T22:10:06.554890+0000","last_fullsized":"2024-09-20T22:10:06.554890+0000","mapping_epoch":176,"log_start":"207'5621","ondisk_log_start":"207'5621","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4580,"log_dups_size":0,"ondisk_log_size":4580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330891264,"num_objects":5049,"num_object_clones":0,"num_object_copies":15147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10201,"num_write_kb":488000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"71'10","reported_seq":247,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.433094+0000","last_change":"2024-09-20T22:07:59.424108+0000","last_active":"2024-09-20T22:10:01.433094+0000","last_peered":"2024-09-20T22:10:01.433094+0000","last_clean":"2024-09-20T22:10:01.433094+0000","last_became_active":"2024-09-20T22:06:54.844681+0000","last_became_peered":"2024-09-20T22:06:54.844681+0000","last_unstale":"2024-09-20T22:10:01.433094+0000","last_undegraded":"2024-09-20T22:10:01.433094+0000","last_fullsized":"2024-09-20T22:10:01.433094+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:07:59.423970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:59.423970+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":399,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.436408+0000","last_change":"2024-09-20T22:08:29.034890+0000","last_active":"2024-09-20T22:10:01.436408+0000","last_peered":"2024-09-20T22:10:01.436408+0000","last_clean":"2024-09-20T22:10:01.436408+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:10:01.436408+0000","last_undegraded":"2024-09-20T22:10:01.436408+0000","last_fullsized":"2024-09-20T22:10:01.436408+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:29.034641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:08:29.034641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"71'8","reported_seq":471,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.431081+0000","last_change":"2024-09-20T22:08:28.861124+0000","last_active":"2024-09-20T22:10:01.431081+0000","last_peered":"2024-09-20T22:10:01.431081+0000","last_clean":"2024-09-20T22:10:01.431081+0000","last_became_active":"2024-09-20T22:02:54.292819+0000","last_became_peered":"2024-09-20T22:02:54.292819+0000","last_unstale":"2024-09-20T22:10:01.431081+0000","last_undegraded":"2024-09-20T22:10:01.431081+0000","last_fullsized":"2024-09-20T22:10:01.431081+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:08:28.860982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:28.860982+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":363,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.430817+0000","last_change":"2024-09-20T22:06:00.572347+0000","last_active":"2024-09-20T22:10:01.430817+0000","last_peered":"2024-09-20T22:10:01.430817+0000","last_clean":"2024-09-20T22:10:01.430817+0000","last_became_active":"2024-09-20T22:02:56.304274+0000","last_became_peered":"2024-09-20T22:02:56.304274+0000","last_unstale":"2024-09-20T22:10:01.430817+0000","last_undegraded":"2024-09-20T22:10:01.430817+0000","last_fullsized":"2024-09-20T22:10:01.430817+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:00.572243+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:00.572243+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":391,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.437393+0000","last_change":"2024-09-20T22:09:00.372470+0000","last_active":"2024-09-20T22:10:01.437393+0000","last_peered":"2024-09-20T22:10:01.437393+0000","last_clean":"2024-09-20T22:10:01.437393+0000","last_became_active":"2024-09-20T22:08:48.466328+0000","last_became_peered":"2024-09-20T22:08:48.466328+0000","last_unstale":"2024-09-20T22:10:01.437393+0000","last_undegraded":"2024-09-20T22:10:01.437393+0000","last_fullsized":"2024-09-20T22:10:01.437393+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:09:00.372393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:09:00.372393+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:28.799282+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.f","version":"241'10189","reported_seq":10332,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:06.254828+0000","last_change":"2024-09-20T22:07:52.940027+0000","last_active":"2024-09-20T22:10:06.254828+0000","last_peered":"2024-09-20T22:10:06.254828+0000","last_clean":"2024-09-20T22:10:06.254828+0000","last_became_active":"2024-09-20T22:07:52.939692+0000","last_became_peered":"2024-09-20T22:07:52.939692+0000","last_unstale":"2024-09-20T22:10:06.254828+0000","last_undegraded":"2024-09-20T22:10:06.254828+0000","last_fullsized":"2024-09-20T22:10:06.254828+0000","mapping_epoch":176,"log_start":"205'5636","ondisk_log_start":"205'5636","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4553,"log_dups_size":0,"ondisk_log_size":4553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:08.222113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330235904,"num_objects":5039,"num_object_clones":0,"num_object_copies":15117,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10189,"num_write_kb":487296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3","version":"71'19","reported_seq":423,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.431223+0000","last_change":"2024-09-20T22:08:33.852966+0000","last_active":"2024-09-20T22:10:01.431223+0000","last_peered":"2024-09-20T22:10:01.431223+0000","last_clean":"2024-09-20T22:10:01.431223+0000","last_became_active":"2024-09-20T22:02:52.480077+0000","last_became_peered":"2024-09-20T22:02:52.480077+0000","last_unstale":"2024-09-20T22:10:01.431223+0000","last_undegraded":"2024-09-20T22:10:01.431223+0000","last_fullsized":"2024-09-20T22:10:01.431223+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:08:33.852806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:33.852806+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":222,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.432736+0000","last_change":"2024-09-20T22:07:14.382822+0000","last_active":"2024-09-20T22:10:01.432736+0000","last_peered":"2024-09-20T22:10:01.432736+0000","last_clean":"2024-09-20T22:10:01.432736+0000","last_became_active":"2024-09-20T22:06:54.829037+0000","last_became_peered":"2024-09-20T22:06:54.829037+0000","last_unstale":"2024-09-20T22:10:01.432736+0000","last_undegraded":"2024-09-20T22:10:01.432736+0000","last_fullsized":"2024-09-20T22:10:01.432736+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:14.382712+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:14.382712+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":391,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.438254+0000","last_change":"2024-09-20T22:09:08.234635+0000","last_active":"2024-09-20T22:10:01.438254+0000","last_peered":"2024-09-20T22:10:01.438254+0000","last_clean":"2024-09-20T22:10:01.438254+0000","last_became_active":"2024-09-20T22:02:54.290787+0000","last_became_peered":"2024-09-20T22:02:54.290787+0000","last_unstale":"2024-09-20T22:10:01.438254+0000","last_undegraded":"2024-09-20T22:10:01.438254+0000","last_fullsized":"2024-09-20T22:10:01.438254+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:09:08.234520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:09.917123+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":220,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.443337+0000","last_change":"2024-09-20T22:09:03.521037+0000","last_active":"2024-09-20T22:10:01.443337+0000","last_peered":"2024-09-20T22:10:01.443337+0000","last_clean":"2024-09-20T22:10:01.443337+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:10:01.443337+0000","last_undegraded":"2024-09-20T22:10:01.443337+0000","last_fullsized":"2024-09-20T22:10:01.443337+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:03.520930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:03.520930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:30.299784+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":386,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.434608+0000","last_change":"2024-09-20T22:09:38.316197+0000","last_active":"2024-09-20T22:10:01.434608+0000","last_peered":"2024-09-20T22:10:01.434608+0000","last_clean":"2024-09-20T22:10:01.434608+0000","last_became_active":"2024-09-20T22:08:48.452234+0000","last_became_peered":"2024-09-20T22:08:48.452234+0000","last_unstale":"2024-09-20T22:10:01.434608+0000","last_undegraded":"2024-09-20T22:10:01.434608+0000","last_fullsized":"2024-09-20T22:10:01.434608+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:09:38.316108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:09:38.316108+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:55.990624+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"241'8356","reported_seq":8547,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:05.716036+0000","last_change":"2024-09-20T22:09:41.961273+0000","last_active":"2024-09-20T22:10:05.716036+0000","last_peered":"2024-09-20T22:10:05.716036+0000","last_clean":"2024-09-20T22:10:05.716036+0000","last_became_active":"2024-09-20T22:09:09.561977+0000","last_became_peered":"2024-09-20T22:09:09.561977+0000","last_unstale":"2024-09-20T22:10:05.716036+0000","last_undegraded":"2024-09-20T22:10:05.716036+0000","last_fullsized":"2024-09-20T22:10:05.716036+0000","mapping_epoch":176,"log_start":"184'1859","ondisk_log_start":"184'1859","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"232'7571","last_scrub_stamp":"2024-09-20T22:09:41.961143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:09:41.961143+0000","objects_scrubbed":3345,"log_size":6497,"log_dups_size":0,"ondisk_log_size":6497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:10.666900+0000","scrub_duration":652,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167051264,"num_objects":2549,"num_object_clones":0,"num_object_copies":7647,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":796,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"146'30","reported_seq":322,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.432931+0000","last_change":"2024-09-20T22:07:55.357829+0000","last_active":"2024-09-20T22:10:01.432931+0000","last_peered":"2024-09-20T22:10:01.432931+0000","last_clean":"2024-09-20T22:10:01.432931+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:10:01.432931+0000","last_undegraded":"2024-09-20T22:10:01.432931+0000","last_fullsized":"2024-09-20T22:10:01.432931+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'30","last_scrub_stamp":"2024-09-20T22:07:55.357726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:55.357726+0000","objects_scrubbed":10,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":392,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.442960+0000","last_change":"2024-09-20T22:09:02.547888+0000","last_active":"2024-09-20T22:10:01.442960+0000","last_peered":"2024-09-20T22:10:01.442960+0000","last_clean":"2024-09-20T22:10:01.442960+0000","last_became_active":"2024-09-20T22:02:50.108457+0000","last_became_peered":"2024-09-20T22:02:50.108457+0000","last_unstale":"2024-09-20T22:10:01.442960+0000","last_undegraded":"2024-09-20T22:10:01.442960+0000","last_fullsized":"2024-09-20T22:10:01.442960+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:02.547780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:02.547780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:13.447545+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":395,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.435622+0000","last_change":"2024-09-20T22:07:21.886172+0000","last_active":"2024-09-20T22:10:01.435622+0000","last_peered":"2024-09-20T22:10:01.435622+0000","last_clean":"2024-09-20T22:10:01.435622+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:10:01.435622+0000","last_undegraded":"2024-09-20T22:10:01.435622+0000","last_fullsized":"2024-09-20T22:10:01.435622+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_clean_scrub_stamp":"2024-09-20T22:07:21.886038+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":376,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.431414+0000","last_change":"2024-09-20T22:07:17.566774+0000","last_active":"2024-09-20T22:10:01.431414+0000","last_peered":"2024-09-20T22:10:01.431414+0000","last_clean":"2024-09-20T22:10:01.431414+0000","last_became_active":"2024-09-20T22:06:54.841419+0000","last_became_peered":"2024-09-20T22:06:54.841419+0000","last_unstale":"2024-09-20T22:10:01.431414+0000","last_undegraded":"2024-09-20T22:10:01.431414+0000","last_fullsized":"2024-09-20T22:10:01.431414+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:17.566115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:17.566115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":120,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":371,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.454047+0000","last_change":"2024-09-20T22:09:05.892350+0000","last_active":"2024-09-20T22:10:01.454047+0000","last_peered":"2024-09-20T22:10:01.454047+0000","last_clean":"2024-09-20T22:10:01.454047+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:10:01.454047+0000","last_undegraded":"2024-09-20T22:10:01.454047+0000","last_fullsized":"2024-09-20T22:10:01.454047+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:05.892235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:09:05.892235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:30.714492+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":372,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.430481+0000","last_change":"2024-09-20T22:08:55.444848+0000","last_active":"2024-09-20T22:10:01.430481+0000","last_peered":"2024-09-20T22:10:01.430481+0000","last_clean":"2024-09-20T22:10:01.430481+0000","last_became_active":"2024-09-20T22:08:55.444469+0000","last_became_peered":"2024-09-20T22:08:55.444469+0000","last_unstale":"2024-09-20T22:10:01.430481+0000","last_undegraded":"2024-09-20T22:10:01.430481+0000","last_fullsized":"2024-09-20T22:10:01.430481+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:07:53.673691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.673691+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:05.028091+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53670,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":18,"num_read_kb":27,"num_write":17,"num_write_kb":220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":161010,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"241'9818","reported_seq":9961,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:06.260426+0000","last_change":"2024-09-20T22:07:52.429893+0000","last_active":"2024-09-20T22:10:06.260426+0000","last_peered":"2024-09-20T22:10:06.260426+0000","last_clean":"2024-09-20T22:10:06.260426+0000","last_became_active":"2024-09-20T22:07:52.429554+0000","last_became_peered":"2024-09-20T22:07:52.429554+0000","last_unstale":"2024-09-20T22:10:06.260426+0000","last_undegraded":"2024-09-20T22:10:06.260426+0000","last_fullsized":"2024-09-20T22:10:06.260426+0000","mapping_epoch":176,"log_start":"202'5225","ondisk_log_start":"202'5225","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4593,"log_dups_size":0,"ondisk_log_size":4593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:08.222186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":316669952,"num_objects":4832,"num_object_clones":0,"num_object_copies":14496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9818,"num_write_kb":468800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,1],"acting":[3,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"71'13","reported_seq":259,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.433652+0000","last_change":"2024-09-20T22:08:35.413541+0000","last_active":"2024-09-20T22:10:01.433652+0000","last_peered":"2024-09-20T22:10:01.433652+0000","last_clean":"2024-09-20T22:10:01.433652+0000","last_became_active":"2024-09-20T22:06:54.850696+0000","last_became_peered":"2024-09-20T22:06:54.850696+0000","last_unstale":"2024-09-20T22:10:01.433652+0000","last_undegraded":"2024-09-20T22:10:01.433652+0000","last_fullsized":"2024-09-20T22:10:01.433652+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'13","last_scrub_stamp":"2024-09-20T22:08:35.413221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:35.413221+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":221,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.440751+0000","last_change":"2024-09-20T22:07:20.341524+0000","last_active":"2024-09-20T22:10:01.440751+0000","last_peered":"2024-09-20T22:10:01.440751+0000","last_clean":"2024-09-20T22:10:01.440751+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:10:01.440751+0000","last_undegraded":"2024-09-20T22:10:01.440751+0000","last_fullsized":"2024-09-20T22:10:01.440751+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:20.338388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:20.338388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":409,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.437378+0000","last_change":"2024-09-20T22:08:36.251074+0000","last_active":"2024-09-20T22:10:01.437378+0000","last_peered":"2024-09-20T22:10:01.437378+0000","last_clean":"2024-09-20T22:10:01.437378+0000","last_became_active":"2024-09-20T22:02:49.099508+0000","last_became_peered":"2024-09-20T22:02:49.099508+0000","last_unstale":"2024-09-20T22:10:01.437378+0000","last_undegraded":"2024-09-20T22:10:01.437378+0000","last_fullsized":"2024-09-20T22:10:01.437378+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:08:36.250748+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:01.487768+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":391,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.445303+0000","last_change":"2024-09-20T22:09:48.504346+0000","last_active":"2024-09-20T22:10:01.445303+0000","last_peered":"2024-09-20T22:10:01.445303+0000","last_clean":"2024-09-20T22:10:01.445303+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:10:01.445303+0000","last_undegraded":"2024-09-20T22:10:01.445303+0000","last_fullsized":"2024-09-20T22:10:01.445303+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:48.504119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:09:48.504119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:57.245456+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":372,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.437547+0000","last_change":"2024-09-20T22:08:31.832304+0000","last_active":"2024-09-20T22:10:01.437547+0000","last_peered":"2024-09-20T22:10:01.437547+0000","last_clean":"2024-09-20T22:10:01.437547+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:10:01.437547+0000","last_undegraded":"2024-09-20T22:10:01.437547+0000","last_fullsized":"2024-09-20T22:10:01.437547+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:31.832107+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:31.832107+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":385,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.433430+0000","last_change":"2024-09-20T22:08:59.488297+0000","last_active":"2024-09-20T22:10:01.433430+0000","last_peered":"2024-09-20T22:10:01.433430+0000","last_clean":"2024-09-20T22:10:01.433430+0000","last_became_active":"2024-09-20T22:08:48.446840+0000","last_became_peered":"2024-09-20T22:08:48.446840+0000","last_unstale":"2024-09-20T22:10:01.433430+0000","last_undegraded":"2024-09-20T22:10:01.433430+0000","last_fullsized":"2024-09-20T22:10:01.433430+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:08:59.488188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:08:59.488188+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:20.578312+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.a","version":"241'10168","reported_seq":10311,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:07.850742+0000","last_change":"2024-09-20T22:07:52.939933+0000","last_active":"2024-09-20T22:10:07.850742+0000","last_peered":"2024-09-20T22:10:07.850742+0000","last_clean":"2024-09-20T22:10:07.850742+0000","last_became_active":"2024-09-20T22:07:52.939626+0000","last_became_peered":"2024-09-20T22:07:52.939626+0000","last_unstale":"2024-09-20T22:10:07.850742+0000","last_undegraded":"2024-09-20T22:10:07.850742+0000","last_fullsized":"2024-09-20T22:10:07.850742+0000","mapping_epoch":176,"log_start":"210'5997","ondisk_log_start":"210'5997","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":4171,"log_dups_size":0,"ondisk_log_size":4171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324141056,"num_objects":4946,"num_object_clones":0,"num_object_copies":14838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10168,"num_write_kb":483648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"71'12","reported_seq":398,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.432467+0000","last_change":"2024-09-20T22:07:29.614630+0000","last_active":"2024-09-20T22:10:01.432467+0000","last_peered":"2024-09-20T22:10:01.432467+0000","last_clean":"2024-09-20T22:10:01.432467+0000","last_became_active":"2024-09-20T22:06:54.848670+0000","last_became_peered":"2024-09-20T22:06:54.848670+0000","last_unstale":"2024-09-20T22:10:01.432467+0000","last_undegraded":"2024-09-20T22:10:01.432467+0000","last_fullsized":"2024-09-20T22:10:01.432467+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:29.614551+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:29.614551+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":383,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.434507+0000","last_change":"2024-09-20T22:06:44.862452+0000","last_active":"2024-09-20T22:10:01.434507+0000","last_peered":"2024-09-20T22:10:01.434507+0000","last_clean":"2024-09-20T22:10:01.434507+0000","last_became_active":"2024-09-20T22:02:50.110448+0000","last_became_peered":"2024-09-20T22:02:50.110448+0000","last_unstale":"2024-09-20T22:10:01.434507+0000","last_undegraded":"2024-09-20T22:10:01.434507+0000","last_fullsized":"2024-09-20T22:10:01.434507+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:06:44.862338+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":824,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:04.098023+0000","last_change":"2024-09-20T22:07:16.590582+0000","last_active":"2024-09-20T22:10:04.098023+0000","last_peered":"2024-09-20T22:10:04.098023+0000","last_clean":"2024-09-20T22:10:04.098023+0000","last_became_active":"2024-09-20T22:06:54.844454+0000","last_became_peered":"2024-09-20T22:06:54.844454+0000","last_unstale":"2024-09-20T22:10:04.098023+0000","last_undegraded":"2024-09-20T22:10:04.098023+0000","last_fullsized":"2024-09-20T22:10:04.098023+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:07:16.590475+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:07:16.590475+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":422,"num_read_kb":417,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":376,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.436781+0000","last_change":"2024-09-20T22:07:22.884981+0000","last_active":"2024-09-20T22:10:01.436781+0000","last_peered":"2024-09-20T22:10:01.436781+0000","last_clean":"2024-09-20T22:10:01.436781+0000","last_became_active":"2024-09-20T22:06:54.841596+0000","last_became_peered":"2024-09-20T22:06:54.841596+0000","last_unstale":"2024-09-20T22:10:01.436781+0000","last_undegraded":"2024-09-20T22:10:01.436781+0000","last_fullsized":"2024-09-20T22:10:01.436781+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:22.884907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.884907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":221,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.432550+0000","last_change":"2024-09-20T22:08:34.415569+0000","last_active":"2024-09-20T22:10:01.432550+0000","last_peered":"2024-09-20T22:10:01.432550+0000","last_clean":"2024-09-20T22:10:01.432550+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:10:01.432550+0000","last_undegraded":"2024-09-20T22:10:01.432550+0000","last_fullsized":"2024-09-20T22:10:01.432550+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:34.415468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.415468+0000","last_clean_scrub_stamp":"2024-09-20T22:08:34.415468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.9","version":"241'8260","reported_seq":8451,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:05.670861+0000","last_change":"2024-09-20T22:09:10.870815+0000","last_active":"2024-09-20T22:10:05.670861+0000","last_peered":"2024-09-20T22:10:05.670861+0000","last_clean":"2024-09-20T22:10:05.670861+0000","last_became_active":"2024-09-20T22:09:09.578463+0000","last_became_peered":"2024-09-20T22:09:09.578463+0000","last_unstale":"2024-09-20T22:10:05.670861+0000","last_undegraded":"2024-09-20T22:10:05.670861+0000","last_fullsized":"2024-09-20T22:10:05.670861+0000","mapping_epoch":176,"log_start":"182'1200","ondisk_log_start":"182'1200","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"216'6610","last_scrub_stamp":"2024-09-20T22:09:10.870629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:09:10.870629+0000","objects_scrubbed":3287,"log_size":7060,"log_dups_size":0,"ondisk_log_size":7060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:18.397304+0000","scrub_duration":670,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163315712,"num_objects":2492,"num_object_clones":0,"num_object_copies":7476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1651,"num_write_kb":27392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"71'16","reported_seq":497,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.439229+0000","last_change":"2024-09-20T22:07:53.297530+0000","last_active":"2024-09-20T22:10:01.439229+0000","last_peered":"2024-09-20T22:10:01.439229+0000","last_clean":"2024-09-20T22:10:01.439229+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:10:01.439229+0000","last_undegraded":"2024-09-20T22:10:01.439229+0000","last_fullsized":"2024-09-20T22:10:01.439229+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'16","last_scrub_stamp":"2024-09-20T22:07:53.297442+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.297442+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":383,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.453438+0000","last_change":"2024-09-20T22:06:48.105501+0000","last_active":"2024-09-20T22:10:01.453438+0000","last_peered":"2024-09-20T22:10:01.453438+0000","last_clean":"2024-09-20T22:10:01.453438+0000","last_became_active":"2024-09-20T22:02:50.106011+0000","last_became_peered":"2024-09-20T22:02:50.106011+0000","last_unstale":"2024-09-20T22:10:01.453438+0000","last_undegraded":"2024-09-20T22:10:01.453438+0000","last_fullsized":"2024-09-20T22:10:01.453438+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:48.105403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:48.105403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":372,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.430993+0000","last_change":"2024-09-20T22:08:55.451233+0000","last_active":"2024-09-20T22:10:01.430993+0000","last_peered":"2024-09-20T22:10:01.430993+0000","last_clean":"2024-09-20T22:10:01.430993+0000","last_became_active":"2024-09-20T22:08:55.450855+0000","last_became_peered":"2024-09-20T22:08:55.450855+0000","last_unstale":"2024-09-20T22:10:01.430993+0000","last_undegraded":"2024-09-20T22:10:01.430993+0000","last_fullsized":"2024-09-20T22:10:01.430993+0000","mapping_epoch":208,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:07:53.673691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:07:53.673691+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:05.028115+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":53669,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":18,"num_read_kb":26,"num_write":16,"num_write_kb":219,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":161009,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,1],"acting":[0,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":91,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.445513+0000","last_change":"2024-09-20T22:08:59.475385+0000","last_active":"2024-09-20T22:10:01.445513+0000","last_peered":"2024-09-20T22:10:01.445513+0000","last_clean":"2024-09-20T22:10:01.445513+0000","last_became_active":"2024-09-20T22:08:52.423678+0000","last_became_peered":"2024-09-20T22:08:52.423678+0000","last_unstale":"2024-09-20T22:10:01.445513+0000","last_undegraded":"2024-09-20T22:10:01.445513+0000","last_fullsized":"2024-09-20T22:10:01.445513+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:59.475260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:59.475260+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:28.970779+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":393,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.436580+0000","last_change":"2024-09-20T22:08:39.101879+0000","last_active":"2024-09-20T22:10:01.436580+0000","last_peered":"2024-09-20T22:10:01.436580+0000","last_clean":"2024-09-20T22:10:01.436580+0000","last_became_active":"2024-09-20T22:04:21.237725+0000","last_became_peered":"2024-09-20T22:04:21.237725+0000","last_unstale":"2024-09-20T22:10:01.436580+0000","last_undegraded":"2024-09-20T22:10:01.436580+0000","last_fullsized":"2024-09-20T22:10:01.436580+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:39.101763+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:08:39.101763+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:05.527492+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2","version":"241'8229","reported_seq":8420,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:05.668447+0000","last_change":"2024-09-20T22:09:11.017877+0000","last_active":"2024-09-20T22:10:05.668447+0000","last_peered":"2024-09-20T22:10:05.668447+0000","last_clean":"2024-09-20T22:10:05.668447+0000","last_became_active":"2024-09-20T22:09:09.551695+0000","last_became_peered":"2024-09-20T22:09:09.551695+0000","last_unstale":"2024-09-20T22:10:05.668447+0000","last_undegraded":"2024-09-20T22:10:05.668447+0000","last_fullsized":"2024-09-20T22:10:05.668447+0000","mapping_epoch":176,"log_start":"185'2032","ondisk_log_start":"185'2032","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"217'6588","last_scrub_stamp":"2024-09-20T22:09:11.017735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:09:11.017735+0000","objects_scrubbed":3372,"log_size":6197,"log_dups_size":0,"ondisk_log_size":6197,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:11.622652+0000","scrub_duration":565,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166002688,"num_objects":2533,"num_object_clones":0,"num_object_copies":7599,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1641,"num_write_kb":25664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,5],"acting":[4,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"71'11","reported_seq":400,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.445236+0000","last_change":"2024-09-20T22:07:13.511239+0000","last_active":"2024-09-20T22:10:01.445236+0000","last_peered":"2024-09-20T22:10:01.445236+0000","last_clean":"2024-09-20T22:10:01.445236+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:10:01.445236+0000","last_undegraded":"2024-09-20T22:10:01.445236+0000","last_fullsized":"2024-09-20T22:10:01.445236+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:07:13.511159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:13.511159+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.732955+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":384,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.442971+0000","last_change":"2024-09-20T22:08:40.493022+0000","last_active":"2024-09-20T22:10:01.442971+0000","last_peered":"2024-09-20T22:10:01.442971+0000","last_clean":"2024-09-20T22:10:01.442971+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:10:01.442971+0000","last_undegraded":"2024-09-20T22:10:01.442971+0000","last_fullsized":"2024-09-20T22:10:01.442971+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:40.492952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:40.492952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.733102+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":383,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.436431+0000","last_change":"2024-09-20T22:08:27.057946+0000","last_active":"2024-09-20T22:10:01.436431+0000","last_peered":"2024-09-20T22:10:01.436431+0000","last_clean":"2024-09-20T22:10:01.436431+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:10:01.436431+0000","last_undegraded":"2024-09-20T22:10:01.436431+0000","last_fullsized":"2024-09-20T22:10:01.436431+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:27.057849+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:27.057849+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":371,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.454072+0000","last_change":"2024-09-20T22:06:45.030549+0000","last_active":"2024-09-20T22:10:01.454072+0000","last_peered":"2024-09-20T22:10:01.454072+0000","last_clean":"2024-09-20T22:10:01.454072+0000","last_became_active":"2024-09-20T22:02:56.349376+0000","last_became_peered":"2024-09-20T22:02:56.349376+0000","last_unstale":"2024-09-20T22:10:01.454072+0000","last_undegraded":"2024-09-20T22:10:01.454072+0000","last_fullsized":"2024-09-20T22:10:01.454072+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.030289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.030289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":386,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.434956+0000","last_change":"2024-09-20T22:09:40.351194+0000","last_active":"2024-09-20T22:10:01.434956+0000","last_peered":"2024-09-20T22:10:01.434956+0000","last_clean":"2024-09-20T22:10:01.434956+0000","last_became_active":"2024-09-20T22:08:48.460964+0000","last_became_peered":"2024-09-20T22:08:48.460964+0000","last_unstale":"2024-09-20T22:10:01.434956+0000","last_undegraded":"2024-09-20T22:10:01.434956+0000","last_fullsized":"2024-09-20T22:10:01.434956+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:09:40.351083+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:09:40.351083+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:05.453781+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"241'8317","reported_seq":8506,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:07.824991+0000","last_change":"2024-09-20T22:09:10.957765+0000","last_active":"2024-09-20T22:10:07.824991+0000","last_peered":"2024-09-20T22:10:07.824991+0000","last_clean":"2024-09-20T22:10:07.824991+0000","last_became_active":"2024-09-20T22:09:09.551097+0000","last_became_peered":"2024-09-20T22:09:09.551097+0000","last_unstale":"2024-09-20T22:10:07.824991+0000","last_undegraded":"2024-09-20T22:10:07.824991+0000","last_fullsized":"2024-09-20T22:10:07.824991+0000","mapping_epoch":176,"log_start":"185'2185","ondisk_log_start":"185'2185","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"217'6597","last_scrub_stamp":"2024-09-20T22:09:10.957585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:09:10.957585+0000","objects_scrubbed":3328,"log_size":6132,"log_dups_size":0,"ondisk_log_size":6132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:28.897373+0000","scrub_duration":523,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159383552,"num_objects":2432,"num_object_clones":0,"num_object_copies":7296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1720,"num_write_kb":26368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,2],"acting":[1,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"71'17","reported_seq":417,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.444462+0000","last_change":"2024-09-20T22:07:54.528854+0000","last_active":"2024-09-20T22:10:01.444462+0000","last_peered":"2024-09-20T22:10:01.444462+0000","last_clean":"2024-09-20T22:10:01.444462+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:10:01.444462+0000","last_undegraded":"2024-09-20T22:10:01.444462+0000","last_fullsized":"2024-09-20T22:10:01.444462+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'17","last_scrub_stamp":"2024-09-20T22:07:54.528759+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:54.528759+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.733013+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":392,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.438660+0000","last_change":"2024-09-20T22:08:30.429880+0000","last_active":"2024-09-20T22:10:01.438660+0000","last_peered":"2024-09-20T22:10:01.438660+0000","last_clean":"2024-09-20T22:10:01.438660+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:10:01.438660+0000","last_undegraded":"2024-09-20T22:10:01.438660+0000","last_fullsized":"2024-09-20T22:10:01.438660+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:30.429704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:30.429704+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":221,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.432646+0000","last_change":"2024-09-20T22:07:19.376008+0000","last_active":"2024-09-20T22:10:01.432646+0000","last_peered":"2024-09-20T22:10:01.432646+0000","last_clean":"2024-09-20T22:10:01.432646+0000","last_became_active":"2024-09-20T22:06:54.848275+0000","last_became_peered":"2024-09-20T22:06:54.848275+0000","last_unstale":"2024-09-20T22:10:01.432646+0000","last_undegraded":"2024-09-20T22:10:01.432646+0000","last_fullsized":"2024-09-20T22:10:01.432646+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:19.375867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.375867+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":387,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.445612+0000","last_change":"2024-09-20T22:09:46.506921+0000","last_active":"2024-09-20T22:10:01.445612+0000","last_peered":"2024-09-20T22:10:01.445612+0000","last_clean":"2024-09-20T22:10:01.445612+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:10:01.445612+0000","last_undegraded":"2024-09-20T22:10:01.445612+0000","last_fullsized":"2024-09-20T22:10:01.445612+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:46.506818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:46.506818+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:04.198115+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":663,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.435445+0000","last_change":"2024-09-20T22:08:58.773488+0000","last_active":"2024-09-20T22:10:01.435445+0000","last_peered":"2024-09-20T22:10:01.435445+0000","last_clean":"2024-09-20T22:10:01.435445+0000","last_became_active":"2024-09-20T22:08:48.455288+0000","last_became_peered":"2024-09-20T22:08:48.455288+0000","last_unstale":"2024-09-20T22:10:01.435445+0000","last_undegraded":"2024-09-20T22:10:01.435445+0000","last_fullsized":"2024-09-20T22:10:01.435445+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:08:58.773367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:08:58.773367+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"241'8267","reported_seq":8458,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:06.214445+0000","last_change":"2024-09-20T22:09:10.725332+0000","last_active":"2024-09-20T22:10:06.214445+0000","last_peered":"2024-09-20T22:10:06.214445+0000","last_clean":"2024-09-20T22:10:06.214445+0000","last_became_active":"2024-09-20T22:09:09.591212+0000","last_became_peered":"2024-09-20T22:09:09.591212+0000","last_unstale":"2024-09-20T22:10:06.214445+0000","last_undegraded":"2024-09-20T22:10:06.214445+0000","last_fullsized":"2024-09-20T22:10:06.214445+0000","mapping_epoch":176,"log_start":"184'1794","ondisk_log_start":"184'1794","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"217'6680","last_scrub_stamp":"2024-09-20T22:09:10.725099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:09:10.725099+0000","objects_scrubbed":3317,"log_size":6473,"log_dups_size":0,"ondisk_log_size":6473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:40.073825+0000","scrub_duration":915,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":163643392,"num_objects":2497,"num_object_clones":0,"num_object_copies":7491,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1602,"num_write_kb":25024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"71'10","reported_seq":395,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.441977+0000","last_change":"2024-09-20T22:09:01.543757+0000","last_active":"2024-09-20T22:10:01.441977+0000","last_peered":"2024-09-20T22:10:01.441977+0000","last_clean":"2024-09-20T22:10:01.441977+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:10:01.441977+0000","last_undegraded":"2024-09-20T22:10:01.441977+0000","last_fullsized":"2024-09-20T22:10:01.441977+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:09:01.543678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:09:01.543678+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:02.970566+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":398,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.454633+0000","last_change":"2024-09-20T22:09:53.395968+0000","last_active":"2024-09-20T22:10:01.454633+0000","last_peered":"2024-09-20T22:10:01.454633+0000","last_clean":"2024-09-20T22:10:01.454633+0000","last_became_active":"2024-09-20T22:02:50.110446+0000","last_became_peered":"2024-09-20T22:02:50.110446+0000","last_unstale":"2024-09-20T22:10:01.454633+0000","last_undegraded":"2024-09-20T22:10:01.454633+0000","last_fullsized":"2024-09-20T22:10:01.454633+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:09:53.395833+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:14.305458+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"139'9","reported_seq":496,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.442368+0000","last_change":"2024-09-20T22:09:50.486852+0000","last_active":"2024-09-20T22:10:01.442368+0000","last_peered":"2024-09-20T22:10:01.442368+0000","last_clean":"2024-09-20T22:10:01.442368+0000","last_became_active":"2024-09-20T22:02:54.294458+0000","last_became_peered":"2024-09-20T22:02:54.294458+0000","last_unstale":"2024-09-20T22:10:01.442368+0000","last_undegraded":"2024-09-20T22:10:01.442368+0000","last_fullsized":"2024-09-20T22:10:01.442368+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'9","last_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:09:50.486676+0000","objects_scrubbed":1,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:58.723532+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":381,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.444993+0000","last_change":"2024-09-20T22:09:53.701152+0000","last_active":"2024-09-20T22:10:01.444993+0000","last_peered":"2024-09-20T22:10:01.444993+0000","last_clean":"2024-09-20T22:10:01.444993+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:10:01.444993+0000","last_undegraded":"2024-09-20T22:10:01.444993+0000","last_fullsized":"2024-09-20T22:10:01.444993+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:53.701054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:09:53.701054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:16.621355+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.7","version":"241'8369","reported_seq":8518,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:07.810647+0000","last_change":"2024-09-20T22:09:09.571096+0000","last_active":"2024-09-20T22:10:07.810647+0000","last_peered":"2024-09-20T22:10:07.810647+0000","last_clean":"2024-09-20T22:10:07.810647+0000","last_became_active":"2024-09-20T22:09:09.566984+0000","last_became_peered":"2024-09-20T22:09:09.566984+0000","last_unstale":"2024-09-20T22:10:07.810647+0000","last_undegraded":"2024-09-20T22:10:07.810647+0000","last_fullsized":"2024-09-20T22:10:07.810647+0000","mapping_epoch":176,"log_start":"185'2256","ondisk_log_start":"185'2256","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":6113,"log_dups_size":0,"ondisk_log_size":6113,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:08.461419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":162889728,"num_objects":2485,"num_object_clones":0,"num_object_copies":7455,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2485,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5063,"num_write_kb":241568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,6],"acting":[1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"71'9","reported_seq":404,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.431132+0000","last_change":"2024-09-20T22:07:59.740218+0000","last_active":"2024-09-20T22:10:01.431132+0000","last_peered":"2024-09-20T22:10:01.431132+0000","last_clean":"2024-09-20T22:10:01.431132+0000","last_became_active":"2024-09-20T22:06:54.847770+0000","last_became_peered":"2024-09-20T22:06:54.847770+0000","last_unstale":"2024-09-20T22:10:01.431132+0000","last_undegraded":"2024-09-20T22:10:01.431132+0000","last_fullsized":"2024-09-20T22:10:01.431132+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:59.740089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:59.740089+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":397,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.437790+0000","last_change":"2024-09-20T22:08:31.071694+0000","last_active":"2024-09-20T22:10:01.437790+0000","last_peered":"2024-09-20T22:10:01.437790+0000","last_clean":"2024-09-20T22:10:01.437790+0000","last_became_active":"2024-09-20T22:02:50.112682+0000","last_became_peered":"2024-09-20T22:02:50.112682+0000","last_unstale":"2024-09-20T22:10:01.437790+0000","last_undegraded":"2024-09-20T22:10:01.437790+0000","last_fullsized":"2024-09-20T22:10:01.437790+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:08:31.071610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:31.071610+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":391,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.445147+0000","last_change":"2024-09-20T22:09:06.528889+0000","last_active":"2024-09-20T22:10:01.445147+0000","last_peered":"2024-09-20T22:10:01.445147+0000","last_clean":"2024-09-20T22:10:01.445147+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:10:01.445147+0000","last_undegraded":"2024-09-20T22:10:01.445147+0000","last_fullsized":"2024-09-20T22:10:01.445147+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:09:06.528588+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:07.052667+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":381,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.434212+0000","last_change":"2024-09-20T22:08:27.817949+0000","last_active":"2024-09-20T22:10:01.434212+0000","last_peered":"2024-09-20T22:10:01.434212+0000","last_clean":"2024-09-20T22:10:01.434212+0000","last_became_active":"2024-09-20T22:06:55.824229+0000","last_became_peered":"2024-09-20T22:06:55.824229+0000","last_unstale":"2024-09-20T22:10:01.434212+0000","last_undegraded":"2024-09-20T22:10:01.434212+0000","last_fullsized":"2024-09-20T22:10:01.434212+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:27.817850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:27.817850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"241'8302","reported_seq":8451,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:06.564663+0000","last_change":"2024-09-20T22:09:09.567024+0000","last_active":"2024-09-20T22:10:06.564663+0000","last_peered":"2024-09-20T22:10:06.564663+0000","last_clean":"2024-09-20T22:10:06.564663+0000","last_became_active":"2024-09-20T22:09:09.564622+0000","last_became_peered":"2024-09-20T22:09:09.564622+0000","last_unstale":"2024-09-20T22:10:06.564663+0000","last_undegraded":"2024-09-20T22:10:06.564663+0000","last_fullsized":"2024-09-20T22:10:06.564663+0000","mapping_epoch":176,"log_start":"183'1626","ondisk_log_start":"183'1626","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":6676,"log_dups_size":0,"ondisk_log_size":6676,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:10.087821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":162758656,"num_objects":2483,"num_object_clones":0,"num_object_copies":7449,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5021,"num_write_kb":240160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"71'19","reported_seq":437,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.438737+0000","last_change":"2024-09-20T22:08:51.467188+0000","last_active":"2024-09-20T22:10:01.438737+0000","last_peered":"2024-09-20T22:10:01.438737+0000","last_clean":"2024-09-20T22:10:01.438737+0000","last_became_active":"2024-09-20T22:08:51.451319+0000","last_became_peered":"2024-09-20T22:08:51.451319+0000","last_unstale":"2024-09-20T22:10:01.438737+0000","last_undegraded":"2024-09-20T22:10:01.438737+0000","last_fullsized":"2024-09-20T22:10:01.438737+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:07:25.154919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:25.154919+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,5],"acting":[6,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":384,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.437788+0000","last_change":"2024-09-20T22:08:00.560208+0000","last_active":"2024-09-20T22:10:01.437788+0000","last_peered":"2024-09-20T22:10:01.437788+0000","last_clean":"2024-09-20T22:10:01.437788+0000","last_became_active":"2024-09-20T22:02:50.108712+0000","last_became_peered":"2024-09-20T22:02:50.108712+0000","last_unstale":"2024-09-20T22:10:01.437788+0000","last_undegraded":"2024-09-20T22:10:01.437788+0000","last_fullsized":"2024-09-20T22:10:01.437788+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:00.560098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:00.560098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":391,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.443700+0000","last_change":"2024-09-20T22:09:07.554566+0000","last_active":"2024-09-20T22:10:01.443700+0000","last_peered":"2024-09-20T22:10:01.443700+0000","last_clean":"2024-09-20T22:10:01.443700+0000","last_became_active":"2024-09-20T22:02:54.291134+0000","last_became_peered":"2024-09-20T22:02:54.291134+0000","last_unstale":"2024-09-20T22:10:01.443700+0000","last_undegraded":"2024-09-20T22:10:01.443700+0000","last_fullsized":"2024-09-20T22:10:01.443700+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:07.554455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:07.554455+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:21.068448+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":91,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.436601+0000","last_change":"2024-09-20T22:09:08.495817+0000","last_active":"2024-09-20T22:10:01.436601+0000","last_peered":"2024-09-20T22:10:01.436601+0000","last_clean":"2024-09-20T22:10:01.436601+0000","last_became_active":"2024-09-20T22:08:52.420555+0000","last_became_peered":"2024-09-20T22:08:52.420555+0000","last_unstale":"2024-09-20T22:10:01.436601+0000","last_undegraded":"2024-09-20T22:10:01.436601+0000","last_fullsized":"2024-09-20T22:10:01.436601+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:08.495599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:08.495599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:30.744649+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.5","version":"241'8410","reported_seq":8587,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:05.725299+0000","last_change":"2024-09-20T22:09:12.421515+0000","last_active":"2024-09-20T22:10:05.725299+0000","last_peered":"2024-09-20T22:10:05.725299+0000","last_clean":"2024-09-20T22:10:05.725299+0000","last_became_active":"2024-09-20T22:09:09.583278+0000","last_became_peered":"2024-09-20T22:09:09.583278+0000","last_unstale":"2024-09-20T22:10:05.725299+0000","last_undegraded":"2024-09-20T22:10:05.725299+0000","last_fullsized":"2024-09-20T22:10:05.725299+0000","mapping_epoch":176,"log_start":"182'1400","ondisk_log_start":"182'1400","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":2399,"log_size":7010,"log_dups_size":0,"ondisk_log_size":7010,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:06.256958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":166100992,"num_objects":2534,"num_object_clones":0,"num_object_copies":7602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5052,"num_write_kb":242784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"71'12","reported_seq":404,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.442632+0000","last_change":"2024-09-20T22:07:22.458421+0000","last_active":"2024-09-20T22:10:01.442632+0000","last_peered":"2024-09-20T22:10:01.442632+0000","last_clean":"2024-09-20T22:10:01.442632+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:10:01.442632+0000","last_undegraded":"2024-09-20T22:10:01.442632+0000","last_fullsized":"2024-09-20T22:10:01.442632+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:07:22.458279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:22.458279+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.732892+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":393,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.436365+0000","last_change":"2024-09-20T22:09:03.831996+0000","last_active":"2024-09-20T22:10:01.436365+0000","last_peered":"2024-09-20T22:10:01.436365+0000","last_clean":"2024-09-20T22:10:01.436365+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:10:01.436365+0000","last_undegraded":"2024-09-20T22:10:01.436365+0000","last_fullsized":"2024-09-20T22:10:01.436365+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:03.831774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:03.831774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:30.562595+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":392,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.436751+0000","last_change":"2024-09-20T22:09:01.509627+0000","last_active":"2024-09-20T22:10:01.436751+0000","last_peered":"2024-09-20T22:10:01.436751+0000","last_clean":"2024-09-20T22:10:01.436751+0000","last_became_active":"2024-09-20T22:06:54.843906+0000","last_became_peered":"2024-09-20T22:06:54.843906+0000","last_unstale":"2024-09-20T22:10:01.436751+0000","last_undegraded":"2024-09-20T22:10:01.436751+0000","last_fullsized":"2024-09-20T22:10:01.436751+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:01.509511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:01.509511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:02.825783+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":330,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.431443+0000","last_change":"2024-09-20T22:08:52.418883+0000","last_active":"2024-09-20T22:10:01.431443+0000","last_peered":"2024-09-20T22:10:01.431443+0000","last_clean":"2024-09-20T22:10:01.431443+0000","last_became_active":"2024-09-20T22:08:52.418544+0000","last_became_peered":"2024-09-20T22:08:52.418544+0000","last_unstale":"2024-09-20T22:10:01.431443+0000","last_undegraded":"2024-09-20T22:10:01.431443+0000","last_fullsized":"2024-09-20T22:10:01.431443+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:36.768146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:36.768146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6],"acting":[0,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"241'8240","reported_seq":8429,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:05.684788+0000","last_change":"2024-09-20T22:09:42.948704+0000","last_active":"2024-09-20T22:10:05.684788+0000","last_peered":"2024-09-20T22:10:05.684788+0000","last_clean":"2024-09-20T22:10:05.684788+0000","last_became_active":"2024-09-20T22:09:09.561556+0000","last_became_peered":"2024-09-20T22:09:09.561556+0000","last_unstale":"2024-09-20T22:10:05.684788+0000","last_undegraded":"2024-09-20T22:10:05.684788+0000","last_fullsized":"2024-09-20T22:10:05.684788+0000","mapping_epoch":176,"log_start":"183'1671","ondisk_log_start":"183'1671","created":176,"last_epoch_clean":216,"parent":"0.0","parent_split_bits":0,"last_scrub":"233'7473","last_scrub_stamp":"2024-09-20T22:09:42.948626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:09:42.948626+0000","objects_scrubbed":3205,"log_size":6569,"log_dups_size":0,"ondisk_log_size":6569,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:57.739863+0000","scrub_duration":620,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159252480,"num_objects":2430,"num_object_clones":0,"num_object_copies":7290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":775,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,2],"acting":[5,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"71'15","reported_seq":434,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.435035+0000","last_change":"2024-09-20T22:08:51.413659+0000","last_active":"2024-09-20T22:10:01.435035+0000","last_peered":"2024-09-20T22:10:01.435035+0000","last_clean":"2024-09-20T22:10:01.435035+0000","last_became_active":"2024-09-20T22:08:51.405122+0000","last_became_peered":"2024-09-20T22:08:51.405122+0000","last_unstale":"2024-09-20T22:10:01.435035+0000","last_undegraded":"2024-09-20T22:10:01.435035+0000","last_fullsized":"2024-09-20T22:10:01.435035+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:08:38.115162+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:38.115162+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,4],"acting":[5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":283,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.441678+0000","last_change":"2024-09-20T22:09:48.707056+0000","last_active":"2024-09-20T22:10:01.441678+0000","last_peered":"2024-09-20T22:10:01.441678+0000","last_clean":"2024-09-20T22:10:01.441678+0000","last_became_active":"2024-09-20T22:08:51.454410+0000","last_became_peered":"2024-09-20T22:08:51.454410+0000","last_unstale":"2024-09-20T22:10:01.441678+0000","last_undegraded":"2024-09-20T22:10:01.441678+0000","last_fullsized":"2024-09-20T22:10:01.441678+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":205,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:09:48.706938+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:09:48.706938+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:56.239495+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1],"acting":[4,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":472,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.435374+0000","last_change":"2024-09-20T22:08:32.824544+0000","last_active":"2024-09-20T22:10:01.435374+0000","last_peered":"2024-09-20T22:10:01.435374+0000","last_clean":"2024-09-20T22:10:01.435374+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:10:01.435374+0000","last_undegraded":"2024-09-20T22:10:01.435374+0000","last_fullsized":"2024-09-20T22:10:01.435374+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:08:32.824413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.824413+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":379,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.435250+0000","last_change":"2024-09-20T22:08:32.081639+0000","last_active":"2024-09-20T22:10:01.435250+0000","last_peered":"2024-09-20T22:10:01.435250+0000","last_clean":"2024-09-20T22:10:01.435250+0000","last_became_active":"2024-09-20T22:02:56.346990+0000","last_became_peered":"2024-09-20T22:02:56.346990+0000","last_unstale":"2024-09-20T22:10:01.435250+0000","last_undegraded":"2024-09-20T22:10:01.435250+0000","last_fullsized":"2024-09-20T22:10:01.435250+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:32.081527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.081527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:01.717616+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":391,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.437488+0000","last_change":"2024-09-20T22:08:01.261458+0000","last_active":"2024-09-20T22:10:01.437488+0000","last_peered":"2024-09-20T22:10:01.437488+0000","last_clean":"2024-09-20T22:10:01.437488+0000","last_became_active":"2024-09-20T22:02:50.111935+0000","last_became_peered":"2024-09-20T22:02:50.111935+0000","last_unstale":"2024-09-20T22:10:01.437488+0000","last_undegraded":"2024-09-20T22:10:01.437488+0000","last_fullsized":"2024-09-20T22:10:01.437488+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:01.261328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:01.261328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"71'6","reported_seq":402,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.454753+0000","last_change":"2024-09-20T22:08:52.431426+0000","last_active":"2024-09-20T22:10:01.454753+0000","last_peered":"2024-09-20T22:10:01.454753+0000","last_clean":"2024-09-20T22:10:01.454753+0000","last_became_active":"2024-09-20T22:08:52.419521+0000","last_became_peered":"2024-09-20T22:08:52.419521+0000","last_unstale":"2024-09-20T22:10:01.454753+0000","last_undegraded":"2024-09-20T22:10:01.454753+0000","last_fullsized":"2024-09-20T22:10:01.454753+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'6","last_scrub_stamp":"2024-09-20T22:08:33.574900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:33.574900+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:08.222255+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":375,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.436321+0000","last_change":"2024-09-20T22:07:19.865906+0000","last_active":"2024-09-20T22:10:01.436321+0000","last_peered":"2024-09-20T22:10:01.436321+0000","last_clean":"2024-09-20T22:10:01.436321+0000","last_became_active":"2024-09-20T22:06:54.836200+0000","last_became_peered":"2024-09-20T22:06:54.836200+0000","last_unstale":"2024-09-20T22:10:01.436321+0000","last_undegraded":"2024-09-20T22:10:01.436321+0000","last_fullsized":"2024-09-20T22:10:01.436321+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:19.865809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.865809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":371,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.435994+0000","last_change":"2024-09-20T22:09:00.737324+0000","last_active":"2024-09-20T22:10:01.435994+0000","last_peered":"2024-09-20T22:10:01.435994+0000","last_clean":"2024-09-20T22:10:01.435994+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:10:01.435994+0000","last_undegraded":"2024-09-20T22:10:01.435994+0000","last_fullsized":"2024-09-20T22:10:01.435994+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:00.737219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:00.737219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:04.114790+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"71'9","reported_seq":410,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.432601+0000","last_change":"2024-09-20T22:08:52.433690+0000","last_active":"2024-09-20T22:10:01.432601+0000","last_peered":"2024-09-20T22:10:01.432601+0000","last_clean":"2024-09-20T22:10:01.432601+0000","last_became_active":"2024-09-20T22:08:52.423437+0000","last_became_peered":"2024-09-20T22:08:52.423437+0000","last_unstale":"2024-09-20T22:10:01.432601+0000","last_undegraded":"2024-09-20T22:10:01.432601+0000","last_fullsized":"2024-09-20T22:10:01.432601+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:08:31.897253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:31.897253+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":384,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.435607+0000","last_change":"2024-09-20T22:08:28.755412+0000","last_active":"2024-09-20T22:10:01.435607+0000","last_peered":"2024-09-20T22:10:01.435607+0000","last_clean":"2024-09-20T22:10:01.435607+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:10:01.435607+0000","last_undegraded":"2024-09-20T22:10:01.435607+0000","last_fullsized":"2024-09-20T22:10:01.435607+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:28.755335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:28.755335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":375,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.454474+0000","last_change":"2024-09-20T22:08:36.681072+0000","last_active":"2024-09-20T22:10:01.454474+0000","last_peered":"2024-09-20T22:10:01.454474+0000","last_clean":"2024-09-20T22:10:01.454474+0000","last_became_active":"2024-09-20T22:06:54.845704+0000","last_became_peered":"2024-09-20T22:06:54.845704+0000","last_unstale":"2024-09-20T22:10:01.454474+0000","last_undegraded":"2024-09-20T22:10:01.454474+0000","last_fullsized":"2024-09-20T22:10:01.454474+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:36.680959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:36.680959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":387,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.444505+0000","last_change":"2024-09-20T22:09:52.541576+0000","last_active":"2024-09-20T22:10:01.444505+0000","last_peered":"2024-09-20T22:10:01.444505+0000","last_clean":"2024-09-20T22:10:01.444505+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:10:01.444505+0000","last_undegraded":"2024-09-20T22:10:01.444505+0000","last_fullsized":"2024-09-20T22:10:01.444505+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:52.541476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:52.541476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:03.092482+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"241'8300","reported_seq":15576,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:05.659298+0000","last_change":"2024-09-20T22:09:26.472135+0000","last_active":"2024-09-20T22:10:05.659298+0000","last_peered":"2024-09-20T22:10:05.659298+0000","last_clean":"2024-09-20T22:10:05.659298+0000","last_became_active":"2024-09-20T22:09:26.467651+0000","last_became_peered":"2024-09-20T22:09:26.467651+0000","last_unstale":"2024-09-20T22:10:05.659298+0000","last_undegraded":"2024-09-20T22:10:05.659298+0000","last_fullsized":"2024-09-20T22:10:05.659298+0000","mapping_epoch":223,"log_start":"183'1658","ondisk_log_start":"183'1658","created":215,"last_epoch_clean":224,"parent":"0.0","parent_split_bits":5,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:07:51.409224+0000","objects_scrubbed":0,"log_size":6642,"log_dups_size":0,"ondisk_log_size":6642,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:06.158473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":162070528,"num_objects":2473,"num_object_clones":0,"num_object_copies":7419,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5019,"num_write_kb":239744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3557,"num_bytes_recovered":233111552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"71'9","reported_seq":402,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.436210+0000","last_change":"2024-09-20T22:07:52.834986+0000","last_active":"2024-09-20T22:10:01.436210+0000","last_peered":"2024-09-20T22:10:01.436210+0000","last_clean":"2024-09-20T22:10:01.436210+0000","last_became_active":"2024-09-20T22:02:52.482141+0000","last_became_peered":"2024-09-20T22:02:52.482141+0000","last_unstale":"2024-09-20T22:10:01.436210+0000","last_undegraded":"2024-09-20T22:10:01.436210+0000","last_fullsized":"2024-09-20T22:10:01.436210+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:07:52.834898+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:52.834898+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":388,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.431679+0000","last_change":"2024-09-20T22:08:52.423987+0000","last_active":"2024-09-20T22:10:01.431679+0000","last_peered":"2024-09-20T22:10:01.431679+0000","last_clean":"2024-09-20T22:10:01.431679+0000","last_became_active":"2024-09-20T22:08:52.423263+0000","last_became_peered":"2024-09-20T22:08:52.423263+0000","last_unstale":"2024-09-20T22:10:01.431679+0000","last_undegraded":"2024-09-20T22:10:01.431679+0000","last_fullsized":"2024-09-20T22:10:01.431679+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:06:45.908616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:06:45.908616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"71'8","reported_seq":471,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.454323+0000","last_change":"2024-09-20T22:09:07.902620+0000","last_active":"2024-09-20T22:10:01.454323+0000","last_peered":"2024-09-20T22:10:01.454323+0000","last_clean":"2024-09-20T22:10:01.454323+0000","last_became_active":"2024-09-20T22:02:54.294381+0000","last_became_peered":"2024-09-20T22:02:54.294381+0000","last_unstale":"2024-09-20T22:10:01.454323+0000","last_undegraded":"2024-09-20T22:10:01.454323+0000","last_fullsized":"2024-09-20T22:10:01.454323+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:09:07.902502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:07.902502+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:25.791850+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":371,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.441437+0000","last_change":"2024-09-20T22:08:32.435371+0000","last_active":"2024-09-20T22:10:01.441437+0000","last_peered":"2024-09-20T22:10:01.441437+0000","last_clean":"2024-09-20T22:10:01.441437+0000","last_became_active":"2024-09-20T22:02:56.300041+0000","last_became_peered":"2024-09-20T22:02:56.300041+0000","last_unstale":"2024-09-20T22:10:01.441437+0000","last_undegraded":"2024-09-20T22:10:01.441437+0000","last_fullsized":"2024-09-20T22:10:01.441437+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:32.435297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.435297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.733179+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.18","version":"241'8365","reported_seq":15696,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:05.604801+0000","last_change":"2024-09-20T22:09:50.700716+0000","last_active":"2024-09-20T22:10:05.604801+0000","last_peered":"2024-09-20T22:10:05.604801+0000","last_clean":"2024-09-20T22:10:05.604801+0000","last_became_active":"2024-09-20T22:09:38.460218+0000","last_became_peered":"2024-09-20T22:09:38.460218+0000","last_unstale":"2024-09-20T22:10:05.604801+0000","last_undegraded":"2024-09-20T22:10:05.604801+0000","last_fullsized":"2024-09-20T22:10:05.604801+0000","mapping_epoch":228,"log_start":"184'1859","ondisk_log_start":"184'1859","created":215,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":5,"last_scrub":"236'7784","last_scrub_stamp":"2024-09-20T22:09:50.700492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:09:50.700492+0000","objects_scrubbed":3055,"log_size":6506,"log_dups_size":0,"ondisk_log_size":6506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:18.070886+0000","scrub_duration":984,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161218560,"num_objects":2460,"num_object_clones":0,"num_object_copies":7380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":595,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"71'10","reported_seq":395,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.454432+0000","last_change":"2024-09-20T22:08:38.681999+0000","last_active":"2024-09-20T22:10:01.454432+0000","last_peered":"2024-09-20T22:10:01.454432+0000","last_clean":"2024-09-20T22:10:01.454432+0000","last_became_active":"2024-09-20T22:06:54.862289+0000","last_became_peered":"2024-09-20T22:06:54.862289+0000","last_unstale":"2024-09-20T22:10:01.454432+0000","last_undegraded":"2024-09-20T22:10:01.454432+0000","last_fullsized":"2024-09-20T22:10:01.454432+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:08:38.681831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:38.681831+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":384,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.436031+0000","last_change":"2024-09-20T22:08:56.746747+0000","last_active":"2024-09-20T22:10:01.436031+0000","last_peered":"2024-09-20T22:10:01.436031+0000","last_clean":"2024-09-20T22:10:01.436031+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:10:01.436031+0000","last_undegraded":"2024-09-20T22:10:01.436031+0000","last_fullsized":"2024-09-20T22:10:01.436031+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:56.746651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:56.746651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:08.258731+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"146'10","reported_seq":473,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.436043+0000","last_change":"2024-09-20T22:07:15.870748+0000","last_active":"2024-09-20T22:10:01.436043+0000","last_peered":"2024-09-20T22:10:01.436043+0000","last_clean":"2024-09-20T22:10:01.436043+0000","last_became_active":"2024-09-20T22:06:54.843097+0000","last_became_peered":"2024-09-20T22:06:54.843097+0000","last_unstale":"2024-09-20T22:10:01.436043+0000","last_undegraded":"2024-09-20T22:10:01.436043+0000","last_fullsized":"2024-09-20T22:10:01.436043+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:07:15.870646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:07:15.870646+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":375,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.431675+0000","last_change":"2024-09-20T22:08:52.424184+0000","last_active":"2024-09-20T22:10:01.431675+0000","last_peered":"2024-09-20T22:10:01.431675+0000","last_clean":"2024-09-20T22:10:01.431675+0000","last_became_active":"2024-09-20T22:08:52.422795+0000","last_became_peered":"2024-09-20T22:08:52.422795+0000","last_unstale":"2024-09-20T22:10:01.431675+0000","last_undegraded":"2024-09-20T22:10:01.431675+0000","last_fullsized":"2024-09-20T22:10:01.431675+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:35.824973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:35.824973+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":29,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"71'11","reported_seq":426,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.442672+0000","last_change":"2024-09-20T22:09:54.730657+0000","last_active":"2024-09-20T22:10:01.442672+0000","last_peered":"2024-09-20T22:10:01.442672+0000","last_clean":"2024-09-20T22:10:01.442672+0000","last_became_active":"2024-09-20T22:08:22.139581+0000","last_became_peered":"2024-09-20T22:08:22.139581+0000","last_unstale":"2024-09-20T22:10:01.442672+0000","last_undegraded":"2024-09-20T22:10:01.442672+0000","last_fullsized":"2024-09-20T22:10:01.442672+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:09:54.730526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:09:54.730526+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:55.679780+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":384,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.445231+0000","last_change":"2024-09-20T22:08:29.474891+0000","last_active":"2024-09-20T22:10:01.445231+0000","last_peered":"2024-09-20T22:10:01.445231+0000","last_clean":"2024-09-20T22:10:01.445231+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:10:01.445231+0000","last_undegraded":"2024-09-20T22:10:01.445231+0000","last_fullsized":"2024-09-20T22:10:01.445231+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:29.474781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:29.474781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.733126+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":220,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.432999+0000","last_change":"2024-09-20T22:08:39.327541+0000","last_active":"2024-09-20T22:10:01.432999+0000","last_peered":"2024-09-20T22:10:01.432999+0000","last_clean":"2024-09-20T22:10:01.432999+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:10:01.432999+0000","last_undegraded":"2024-09-20T22:10:01.432999+0000","last_fullsized":"2024-09-20T22:10:01.432999+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:39.326984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:39.326984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.317069+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":199,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.443263+0000","last_change":"2024-09-20T22:06:55.835713+0000","last_active":"2024-09-20T22:10:01.443263+0000","last_peered":"2024-09-20T22:10:01.443263+0000","last_clean":"2024-09-20T22:10:01.443263+0000","last_became_active":"2024-09-20T22:06:55.835393+0000","last_became_peered":"2024-09-20T22:06:55.835393+0000","last_unstale":"2024-09-20T22:10:01.443263+0000","last_undegraded":"2024-09-20T22:10:01.443263+0000","last_fullsized":"2024-09-20T22:10:01.443263+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:05:33.623816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:05:33.623816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"71'9","reported_seq":284,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.433247+0000","last_change":"2024-09-20T22:06:54.844349+0000","last_active":"2024-09-20T22:10:01.433247+0000","last_peered":"2024-09-20T22:10:01.433247+0000","last_clean":"2024-09-20T22:10:01.433247+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:10:01.433247+0000","last_undegraded":"2024-09-20T22:10:01.433247+0000","last_fullsized":"2024-09-20T22:10:01.433247+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:04:12.196518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:04:12.196518+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":83,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.453703+0000","last_change":"2024-09-20T22:08:52.418861+0000","last_active":"2024-09-20T22:10:01.453703+0000","last_peered":"2024-09-20T22:10:01.453703+0000","last_clean":"2024-09-20T22:10:01.453703+0000","last_became_active":"2024-09-20T22:08:52.418520+0000","last_became_peered":"2024-09-20T22:08:52.418520+0000","last_unstale":"2024-09-20T22:10:01.453703+0000","last_undegraded":"2024-09-20T22:10:01.453703+0000","last_fullsized":"2024-09-20T22:10:01.453703+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:54.685751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:54.685751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,0],"acting":[3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":376,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.454486+0000","last_change":"2024-09-20T22:08:34.657517+0000","last_active":"2024-09-20T22:10:01.454486+0000","last_peered":"2024-09-20T22:10:01.454486+0000","last_clean":"2024-09-20T22:10:01.454486+0000","last_became_active":"2024-09-20T22:06:54.844271+0000","last_became_peered":"2024-09-20T22:06:54.844271+0000","last_unstale":"2024-09-20T22:10:01.454486+0000","last_undegraded":"2024-09-20T22:10:01.454486+0000","last_fullsized":"2024-09-20T22:10:01.454486+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:34.657278+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:34.657278+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:03.308836+0000","scrub_duration":53,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":372,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.430433+0000","last_change":"2024-09-20T22:07:23.495005+0000","last_active":"2024-09-20T22:10:01.430433+0000","last_peered":"2024-09-20T22:10:01.430433+0000","last_clean":"2024-09-20T22:10:01.430433+0000","last_became_active":"2024-09-20T22:06:54.848842+0000","last_became_peered":"2024-09-20T22:06:54.848842+0000","last_unstale":"2024-09-20T22:10:01.430433+0000","last_undegraded":"2024-09-20T22:10:01.430433+0000","last_fullsized":"2024-09-20T22:10:01.430433+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:23.494905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:07:23.494905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"71'11","reported_seq":423,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.453041+0000","last_change":"2024-09-20T22:09:55.321918+0000","last_active":"2024-09-20T22:10:01.453041+0000","last_peered":"2024-09-20T22:10:01.453041+0000","last_clean":"2024-09-20T22:10:01.453041+0000","last_became_active":"2024-09-20T22:08:52.417107+0000","last_became_peered":"2024-09-20T22:08:52.417107+0000","last_unstale":"2024-09-20T22:10:01.453041+0000","last_undegraded":"2024-09-20T22:10:01.453041+0000","last_fullsized":"2024-09-20T22:10:01.453041+0000","mapping_epoch":204,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":206,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:09:55.321800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:09:55.321800+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:10.406134+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":392,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.436391+0000","last_change":"2024-09-20T22:07:50.851948+0000","last_active":"2024-09-20T22:10:01.436391+0000","last_peered":"2024-09-20T22:10:01.436391+0000","last_clean":"2024-09-20T22:10:01.436391+0000","last_became_active":"2024-09-20T22:06:54.828826+0000","last_became_peered":"2024-09-20T22:06:54.828826+0000","last_unstale":"2024-09-20T22:10:01.436391+0000","last_undegraded":"2024-09-20T22:10:01.436391+0000","last_fullsized":"2024-09-20T22:10:01.436391+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:50.851839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:50.851839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":384,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.436287+0000","last_change":"2024-09-20T22:09:01.751444+0000","last_active":"2024-09-20T22:10:01.436287+0000","last_peered":"2024-09-20T22:10:01.436287+0000","last_clean":"2024-09-20T22:10:01.436287+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:10:01.436287+0000","last_undegraded":"2024-09-20T22:10:01.436287+0000","last_fullsized":"2024-09-20T22:10:01.436287+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:01.751321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:01.751321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:12.987039+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":34,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.431306+0000","last_change":"2024-09-20T22:09:39.492012+0000","last_active":"2024-09-20T22:10:01.431306+0000","last_peered":"2024-09-20T22:10:01.431306+0000","last_clean":"2024-09-20T22:10:01.431306+0000","last_became_active":"2024-09-20T22:09:39.491148+0000","last_became_peered":"2024-09-20T22:09:39.491148+0000","last_unstale":"2024-09-20T22:10:01.431306+0000","last_undegraded":"2024-09-20T22:10:01.431306+0000","last_fullsized":"2024-09-20T22:10:01.431306+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:10.873660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:10.873660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:18.554401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"71'4","reported_seq":386,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.453066+0000","last_change":"2024-09-20T22:07:19.309318+0000","last_active":"2024-09-20T22:10:01.453066+0000","last_peered":"2024-09-20T22:10:01.453066+0000","last_clean":"2024-09-20T22:10:01.453066+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:10:01.453066+0000","last_undegraded":"2024-09-20T22:10:01.453066+0000","last_fullsized":"2024-09-20T22:10:01.453066+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'4","last_scrub_stamp":"2024-09-20T22:07:19.309187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:07:19.309187+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":383,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.431689+0000","last_change":"2024-09-20T22:08:29.852608+0000","last_active":"2024-09-20T22:10:01.431689+0000","last_peered":"2024-09-20T22:10:01.431689+0000","last_clean":"2024-09-20T22:10:01.431689+0000","last_became_active":"2024-09-20T22:02:50.111142+0000","last_became_peered":"2024-09-20T22:02:50.111142+0000","last_unstale":"2024-09-20T22:10:01.431689+0000","last_undegraded":"2024-09-20T22:10:01.431689+0000","last_fullsized":"2024-09-20T22:10:01.431689+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:29.852502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:29.852502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":399,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.438225+0000","last_change":"2024-09-20T22:09:01.395413+0000","last_active":"2024-09-20T22:10:01.438225+0000","last_peered":"2024-09-20T22:10:01.438225+0000","last_clean":"2024-09-20T22:10:01.438225+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:10:01.438225+0000","last_undegraded":"2024-09-20T22:10:01.438225+0000","last_fullsized":"2024-09-20T22:10:01.438225+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:01.395328+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:01.395328+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:24.061199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":372,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.435677+0000","last_change":"2024-09-20T22:08:39.772520+0000","last_active":"2024-09-20T22:10:01.435677+0000","last_peered":"2024-09-20T22:10:01.435677+0000","last_clean":"2024-09-20T22:10:01.435677+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:10:01.435677+0000","last_undegraded":"2024-09-20T22:10:01.435677+0000","last_fullsized":"2024-09-20T22:10:01.435677+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:39.772379+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:39.772379+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.13","version":"241'8347","reported_seq":19008,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:06.261267+0000","last_change":"2024-09-20T22:09:49.602369+0000","last_active":"2024-09-20T22:10:06.261267+0000","last_peered":"2024-09-20T22:10:06.261267+0000","last_clean":"2024-09-20T22:10:06.261267+0000","last_became_active":"2024-09-20T22:09:47.430908+0000","last_became_peered":"2024-09-20T22:09:47.430908+0000","last_unstale":"2024-09-20T22:10:06.261267+0000","last_undegraded":"2024-09-20T22:10:06.261267+0000","last_fullsized":"2024-09-20T22:10:06.261267+0000","mapping_epoch":235,"log_start":"185'2186","ondisk_log_start":"185'2186","created":215,"last_epoch_clean":236,"parent":"0.0","parent_split_bits":5,"last_scrub":"236'7738","last_scrub_stamp":"2024-09-20T22:09:49.602139+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:51.409224+0000","last_clean_scrub_stamp":"2024-09-20T22:09:49.602139+0000","objects_scrubbed":3096,"log_size":6161,"log_dups_size":0,"ondisk_log_size":6161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:53.122023+0000","scrub_duration":1392,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161742848,"num_objects":2468,"num_object_clones":0,"num_object_copies":7404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":628,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,1],"acting":[3,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":261,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.433125+0000","last_change":"2024-09-20T22:06:54.828772+0000","last_active":"2024-09-20T22:10:01.433125+0000","last_peered":"2024-09-20T22:10:01.433125+0000","last_clean":"2024-09-20T22:10:01.433125+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:10:01.433125+0000","last_undegraded":"2024-09-20T22:10:01.433125+0000","last_fullsized":"2024-09-20T22:10:01.433125+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:04:15.303300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:04:15.303300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":384,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.453353+0000","last_change":"2024-09-20T22:07:57.428285+0000","last_active":"2024-09-20T22:10:01.453353+0000","last_peered":"2024-09-20T22:10:01.453353+0000","last_clean":"2024-09-20T22:10:01.453353+0000","last_became_active":"2024-09-20T22:06:54.848032+0000","last_became_peered":"2024-09-20T22:06:54.848032+0000","last_unstale":"2024-09-20T22:10:01.453353+0000","last_undegraded":"2024-09-20T22:10:01.453353+0000","last_fullsized":"2024-09-20T22:10:01.453353+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:07:57.428165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:07:57.428165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:10:08.222211+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"71'11","reported_seq":407,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.438798+0000","last_change":"2024-09-20T22:08:32.401623+0000","last_active":"2024-09-20T22:10:01.438798+0000","last_peered":"2024-09-20T22:10:01.438798+0000","last_clean":"2024-09-20T22:10:01.438798+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:10:01.438798+0000","last_undegraded":"2024-09-20T22:10:01.438798+0000","last_fullsized":"2024-09-20T22:10:01.438798+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:08:32.401501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.401501+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":375,"reported_epoch":241,"state":"active+clean","last_fresh":"2024-09-20T22:10:01.432005+0000","last_change":"2024-09-20T22:08:32.854748+0000","last_active":"2024-09-20T22:10:01.432005+0000","last_peered":"2024-09-20T22:10:01.432005+0000","last_clean":"2024-09-20T22:10:01.432005+0000","last_became_active":"2024-09-20T22:02:54.289078+0000","last_became_peered":"2024-09-20T22:02:54.289078+0000","last_unstale":"2024-09-20T22:10:01.432005+0000","last_undegraded":"2024-09-20T22:10:01.432005+0000","last_fullsized":"2024-09-20T22:10:01.432005+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:32.854221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.854221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":26,"stat_sum":{"num_bytes":5210538008,"num_objects":79508,"num_object_clones":0,"num_object_copies":238524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":10328,"num_objects_unfound":0,"num_objects_dirty":79508,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":125596,"num_write_kb":5631905,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22723,"num_bytes_recovered":1488977920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16933138432,"data_stored":16933126216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155975,"ondisk_log_size":155975,"up":78,"acting":78,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":693,"num_read_kb":450,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":55,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":402,"ondisk_log_size":402,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":24,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":9,"ondisk_log_size":9,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":422,"num_read_kb":417,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":697707,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":36,"num_read_kb":53,"num_write":33,"num_write_kb":439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":322019,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1408,"ondisk_log_size":1408,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135963,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2125908,"kb_used_data":1751812,"kb_used_omap":1,"kb_used_meta":374078,"kb_avail":91619244,"statfs":{"total":95995035648,"available":93818105856,"internally_reserved":0,"allocated":1793855488,"data_stored":1792785596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":383056330},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430303,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1932896,"kb_used_data":1608784,"kb_used_omap":1,"kb_used_meta":324094,"kb_avail":91812256,"statfs":{"total":95995035648,"available":94015750144,"internally_reserved":0,"allocated":1647394816,"data_stored":1646313272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":331872710},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724643,"num_pgs":71,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3423668,"kb_used_data":3071780,"kb_used_omap":1,"kb_used_meta":351870,"kb_avail":90321484,"statfs":{"total":95995035648,"available":92489199616,"internally_reserved":0,"allocated":3145502720,"data_stored":3144401940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":360315333},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986283,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2270844,"kb_used_data":1924524,"kb_used_omap":1,"kb_used_meta":346302,"kb_avail":91474308,"statfs":{"total":95995035648,"available":93669691392,"internally_reserved":0,"allocated":1970712576,"data_stored":1969602505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":354613701},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770257960,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2268636,"kb_used_data":2044620,"kb_used_omap":4,"kb_used_meta":223995,"kb_avail":91476516,"statfs":{"total":95995035648,"available":93671952384,"internally_reserved":0,"allocated":2093690880,"data_stored":2092548657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":229371560},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149808,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2323916,"kb_used_data":1913724,"kb_used_omap":1,"kb_used_meta":410174,"kb_avail":91421236,"statfs":{"total":95995035648,"available":93615345664,"internally_reserved":0,"allocated":1959653376,"data_stored":1958528647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":420018631},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411445,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2640104,"kb_used_data":2257688,"kb_used_omap":1,"kb_used_meta":382398,"kb_avail":91105048,"statfs":{"total":95995035648,"available":93291569152,"internally_reserved":0,"allocated":2311872512,"data_stored":2310782816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":391576005},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738496,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2494132,"kb_used_data":2065764,"kb_used_omap":1,"kb_used_meta":428222,"kb_avail":91251020,"statfs":{"total":95995035648,"available":93441044480,"internally_reserved":0,"allocated":2115342336,"data_stored":2114240536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":438499781},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":2101018624,"data_stored":2101018624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2081751040,"data_stored":2081751040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2291470336,"data_stored":2291466264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1955594240,"data_stored":1955594240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1967980544,"data_stored":1967980544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3118796800,"data_stored":3118792728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1634664448,"data_stored":1634664448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1781862400,"data_stored":1781858328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:10:11.548 INFO:tasks.thrashosds.thrasher:Setting 3.c to [3, 1, 6] 2024-09-20T22:10:11.548 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '3.c', '3', '1', '6'] 2024-09-20T22:10:11.548 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pg-upmap 3.c 3 1 6 2024-09-20T22:10:11.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:11.578 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:11.578 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:11.680 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:11.711 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:11.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:11.712 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:11.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:11 smithi067 ceph-mon[33511]: osdmap e244: 8 total, 8 up, 8 in 2024-09-20T22:10:11.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:11 smithi067 ceph-mon[33511]: osdmap e245: 8 total, 8 up, 8 in 2024-09-20T22:10:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:11 smithi060 ceph-mon[29694]: osdmap e244: 8 total, 8 up, 8 in 2024-09-20T22:10:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:11 smithi060 ceph-mon[29694]: osdmap e245: 8 total, 8 up, 8 in 2024-09-20T22:10:11.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:11 smithi060 ceph-mon[40290]: osdmap e244: 8 total, 8 up, 8 in 2024-09-20T22:10:11.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:11 smithi060 ceph-mon[40290]: osdmap e245: 8 total, 8 up, 8 in 2024-09-20T22:10:11.814 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:11.844 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:11.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:11.845 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:11.946 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:11.977 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:11.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:11.978 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:12.079 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:12.114 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:12.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:12.115 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:12.208 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T22:10:12.216 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:12.219 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:12.248 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:12.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:12.249 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:12.350 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:12.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:12.422 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:12.422 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:12.461 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:10:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:10:12.523 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:12.584 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:12.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:12.585 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:12.686 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:12.727 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:12 smithi060 ceph-mon[40290]: pgmap v381: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 935 op/s; 9245/227904 objects misplaced (4.057%); 3.6 MiB/s, 57 objects/s recovering 2024-09-20T22:10:12.727 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:12 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:10:12.727 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:12 smithi060 ceph-mon[29694]: pgmap v381: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 935 op/s; 9245/227904 objects misplaced (4.057%); 3.6 MiB/s, 57 objects/s recovering 2024-09-20T22:10:12.727 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:12 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:10:12.730 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:12.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:12.731 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:12.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:12 smithi067 ceph-mon[33511]: pgmap v381: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.6 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 935 op/s; 9245/227904 objects misplaced (4.057%); 3.6 MiB/s, 57 objects/s recovering 2024-09-20T22:10:12.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:12 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:10:12.832 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:12.854 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:12.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:12.892 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:12.892 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:12.993 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:13.029 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:13.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:13.030 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:13.066 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:10:13.130 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:13.157 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:13.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:13.158 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:13.259 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:13.287 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:13.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:13.288 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:13.388 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:13.426 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:13.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:13.427 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:13.482 INFO:teuthology.orchestra.run.smithi060.stderr:set 3.c pg_upmap mapping to [3,1,6] 2024-09-20T22:10:13.488 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:13.528 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:13.558 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:13.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:13.559 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:13.660 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:13.699 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:13.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:13.700 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:13.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:13 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:10:13.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:13 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:10:13.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:13 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3192235886' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [3, 1, 6]}]: dispatch 2024-09-20T22:10:13.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:13 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [3, 1, 6]}]: dispatch 2024-09-20T22:10:13.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:13 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:10:13.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:13 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:10:13.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:13 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3192235886' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [3, 1, 6]}]: dispatch 2024-09-20T22:10:13.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:13 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [3, 1, 6]}]: dispatch 2024-09-20T22:10:13.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:13 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:10:13.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:13 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:10:13.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:13 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3192235886' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [3, 1, 6]}]: dispatch 2024-09-20T22:10:13.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:13 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [3, 1, 6]}]: dispatch 2024-09-20T22:10:13.801 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:13.833 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:13.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:13.834 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:13.935 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:13.972 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:13.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:13.973 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:14.074 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:14.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:14.123 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:14.123 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:14.223 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:14.252 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:14.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:14.253 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:14.354 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:14.384 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:14.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:14.385 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:14.486 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:14.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:14.530 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:14.530 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:14.631 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:14.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:14.674 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:14.674 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:14 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [3, 1, 6]}]': finished 2024-09-20T22:10:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:14 smithi067 ceph-mon[33511]: osdmap e246: 8 total, 8 up, 8 in 2024-09-20T22:10:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:14 smithi067 ceph-mon[33511]: pgmap v383: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 979 op/s; 9245/225489 objects misplaced (4.100%); 3.1 MiB/s, 49 objects/s recovering 2024-09-20T22:10:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:14 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3843894593' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:10:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:14 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:10:14.775 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:14.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:14 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [3, 1, 6]}]': finished 2024-09-20T22:10:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:14 smithi060 ceph-mon[29694]: osdmap e246: 8 total, 8 up, 8 in 2024-09-20T22:10:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:14 smithi060 ceph-mon[29694]: pgmap v383: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 979 op/s; 9245/225489 objects misplaced (4.100%); 3.1 MiB/s, 49 objects/s recovering 2024-09-20T22:10:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:14 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3843894593' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:10:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:14 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:10:14.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:14 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "3.c", "id": [3, 1, 6]}]': finished 2024-09-20T22:10:14.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:14 smithi060 ceph-mon[40290]: osdmap e246: 8 total, 8 up, 8 in 2024-09-20T22:10:14.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:14 smithi060 ceph-mon[40290]: pgmap v383: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 979 op/s; 9245/225489 objects misplaced (4.100%); 3.1 MiB/s, 49 objects/s recovering 2024-09-20T22:10:14.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:14 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3843894593' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:10:14.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:14 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:10:14.808 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:14.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:14.809 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:14.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:14.949 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:14.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:14.950 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:14.986 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:10:14.986 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:10:14.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:10:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:10:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169959 7.19 7:9aa79aaf:::benchmark_data_smithi060_78800_object36926:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:10:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440410+0000", 2024-09-20T22:10:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 256.54188597199999, 2024-09-20T22:10:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0776166780000001, 2024-09-20T22:10:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:10:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169959 2024-09-20T22:10:14.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440410+0000", 2024-09-20T22:10:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440410+0000", 2024-09-20T22:10:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440410+0000", 2024-09-20T22:10:14.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9299999999999999e-07 2024-09-20T22:10:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440413+0000", 2024-09-20T22:10:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7219999999999999e-06 2024-09-20T22:10:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440419+0000", 2024-09-20T22:10:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0449999999999997e-06 2024-09-20T22:10:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440424+0000", 2024-09-20T22:10:14.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7090000000000001e-06 2024-09-20T22:10:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.651028+0000", 2024-09-20T22:10:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.21060456799999999 2024-09-20T22:10:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.651036+0000", 2024-09-20T22:10:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8720000000000002e-06 2024-09-20T22:10:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.517832+0000", 2024-09-20T22:10:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.86679538099999998 2024-09-20T22:10:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.518027+0000", 2024-09-20T22:10:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000194988 2024-09-20T22:10:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.991 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169973 7.19 7:9cb6a65c:::benchmark_data_smithi060_78800_object36940:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:10:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.452156+0000", 2024-09-20T22:10:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 256.53014021000001, 2024-09-20T22:10:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.079163852, 2024-09-20T22:10:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169973 2024-09-20T22:10:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:14.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.452156+0000", 2024-09-20T22:10:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.452156+0000", 2024-09-20T22:10:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.452158+0000", 2024-09-20T22:10:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0999999999999998e-06 2024-09-20T22:10:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:14.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.452165+0000", 2024-09-20T22:10:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5039999999999996e-06 2024-09-20T22:10:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.452181+0000", 2024-09-20T22:10:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5226e-05 2024-09-20T22:10:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.452190+0000", 2024-09-20T22:10:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0380000000000005e-06 2024-09-20T22:10:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.651057+0000", 2024-09-20T22:10:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.19886704599999999 2024-09-20T22:10:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.651062+0000", 2024-09-20T22:10:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7230000000000001e-06 2024-09-20T22:10:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.518364+0000", 2024-09-20T22:10:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.86730196400000004 2024-09-20T22:10:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.518458+0000", 2024-09-20T22:10:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3105999999999999e-05 2024-09-20T22:10:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4", 2024-09-20T22:10:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.518738+0000", 2024-09-20T22:10:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000280005 2024-09-20T22:10:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.521963+0000", 2024-09-20T22:10:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003225374 2024-09-20T22:10:14.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.531163+0000", 2024-09-20T22:10:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0091998740000000002 2024-09-20T22:10:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.531225+0000", 2024-09-20T22:10:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2119000000000006e-05 2024-09-20T22:10:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.531320+0000", 2024-09-20T22:10:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4772999999999995e-05 2024-09-20T22:10:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:14.997 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:197601 7.13 7:c8cc3338:::benchmark_data_smithi060_78800_object64568:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e132)", 2024-09-20T22:10:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:24.370429+0000", 2024-09-20T22:10:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 230.61186669, 2024-09-20T22:10:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0703220390000001, 2024-09-20T22:10:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:10:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:14.998 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 197601 2024-09-20T22:10:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370429+0000", 2024-09-20T22:10:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370429+0000", 2024-09-20T22:10:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:14.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370430+0000", 2024-09-20T22:10:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6200000000000006e-07 2024-09-20T22:10:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370435+0000", 2024-09-20T22:10:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5589999999999999e-06 2024-09-20T22:10:15.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370447+0000", 2024-09-20T22:10:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2058e-05 2024-09-20T22:10:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370454+0000", 2024-09-20T22:10:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4449999999999999e-06 2024-09-20T22:10:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370854+0000", 2024-09-20T22:10:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00039970400000000002 2024-09-20T22:10:15.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op must wait for map", 2024-09-20T22:10:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370861+0000", 2024-09-20T22:10:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8190000000000003e-06 2024-09-20T22:10:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.373710+0000", 2024-09-20T22:10:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0028495270000000001 2024-09-20T22:10:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.373724+0000", 2024-09-20T22:10:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.395e-05 2024-09-20T22:10:15.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.435399+0000", 2024-09-20T22:10:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0616748069999999 2024-09-20T22:10:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:10:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.435541+0000", 2024-09-20T22:10:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014141499999999999 2024-09-20T22:10:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.440680+0000", 2024-09-20T22:10:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0051391170000000003 2024-09-20T22:10:15.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.440751+0000", 2024-09-20T22:10:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1675999999999996e-05 2024-09-20T22:10:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:197609 7.13 7:c869088e:::benchmark_data_smithi060_78800_object64576:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e132)", 2024-09-20T22:10:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:24.370476+0000", 2024-09-20T22:10:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 230.61182040899999, 2024-09-20T22:10:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.075397427, 2024-09-20T22:10:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:15.004 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 197609 2024-09-20T22:10:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370476+0000", 2024-09-20T22:10:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370476+0000", 2024-09-20T22:10:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370476+0000", 2024-09-20T22:10:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4299999999999998e-07 2024-09-20T22:10:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370479+0000", 2024-09-20T22:10:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5299999999999999e-06 2024-09-20T22:10:15.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370486+0000", 2024-09-20T22:10:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.968e-06 2024-09-20T22:10:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370491+0000", 2024-09-20T22:10:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6359999999999999e-06 2024-09-20T22:10:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370914+0000", 2024-09-20T22:10:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00042270099999999999 2024-09-20T22:10:15.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting_for_map not empty", 2024-09-20T22:10:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.370919+0000", 2024-09-20T22:10:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6019999999999999e-06 2024-09-20T22:10:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.373986+0000", 2024-09-20T22:10:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0030664049999999999 2024-09-20T22:10:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:24.373993+0000", 2024-09-20T22:10:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5979999999999999e-06 2024-09-20T22:10:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.440778+0000", 2024-09-20T22:10:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0667841199999999 2024-09-20T22:10:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.440952+0000", 2024-09-20T22:10:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017486199999999999 2024-09-20T22:10:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:10:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.441236+0000", 2024-09-20T22:10:15.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00028399600000000001 2024-09-20T22:10:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445645+0000", 2024-09-20T22:10:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0044080669999999999 2024-09-20T22:10:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445661+0000", 2024-09-20T22:10:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6928000000000001e-05 2024-09-20T22:10:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445783+0000", 2024-09-20T22:10:15.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121438 2024-09-20T22:10:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445816+0000", 2024-09-20T22:10:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3413e-05 2024-09-20T22:10:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.445873+0000", 2024-09-20T22:10:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6719999999999999e-05 2024-09-20T22:10:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227609 7.4 7:26fdb431:::benchmark_data_smithi060_78800_object94576:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-20T22:10:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.802526+0000", 2024-09-20T22:10:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 201.179769609, 2024-09-20T22:10:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.327846133, 2024-09-20T22:10:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:10:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227609 2024-09-20T22:10:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:15.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802526+0000", 2024-09-20T22:10:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802526+0000", 2024-09-20T22:10:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802528+0000", 2024-09-20T22:10:15.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2550000000000001e-06 2024-09-20T22:10:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802533+0000", 2024-09-20T22:10:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5500000000000002e-06 2024-09-20T22:10:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802542+0000", 2024-09-20T22:10:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6950000000000006e-06 2024-09-20T22:10:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:15.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802548+0000", 2024-09-20T22:10:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8100000000000003e-06 2024-09-20T22:10:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802787+0000", 2024-09-20T22:10:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023914699999999999 2024-09-20T22:10:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802897+0000", 2024-09-20T22:10:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109645 2024-09-20T22:10:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4", 2024-09-20T22:10:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.803676+0000", 2024-09-20T22:10:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000779085 2024-09-20T22:10:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.814426+0000", 2024-09-20T22:10:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010750857000000001 2024-09-20T22:10:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.814439+0000", 2024-09-20T22:10:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2549e-05 2024-09-20T22:10:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.823484+0000", 2024-09-20T22:10:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0090446120000001 2024-09-20T22:10:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.823496+0000", 2024-09-20T22:10:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2413e-05 2024-09-20T22:10:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.115278+0000", 2024-09-20T22:10:15.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.291782427 2024-09-20T22:10:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:10:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.115476+0000", 2024-09-20T22:10:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019718599999999999 2024-09-20T22:10:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130062+0000", 2024-09-20T22:10:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014586541 2024-09-20T22:10:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:15.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130373+0000", 2024-09-20T22:10:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00031036099999999999 2024-09-20T22:10:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227614 7.4 7:22c1d0bc:::benchmark_data_smithi060_78800_object94581:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-20T22:10:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.804108+0000", 2024-09-20T22:10:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 201.178187611, 2024-09-20T22:10:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3264465649999999, 2024-09-20T22:10:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:15.019 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227614 2024-09-20T22:10:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.804108+0000", 2024-09-20T22:10:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.804108+0000", 2024-09-20T22:10:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.804110+0000", 2024-09-20T22:10:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1510000000000001e-06 2024-09-20T22:10:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.804115+0000", 2024-09-20T22:10:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0969999999999996e-06 2024-09-20T22:10:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.804128+0000", 2024-09-20T22:10:15.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2899e-05 2024-09-20T22:10:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.804136+0000", 2024-09-20T22:10:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0479999999999999e-06 2024-09-20T22:10:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.814469+0000", 2024-09-20T22:10:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010333187000000001 2024-09-20T22:10:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.814478+0000", 2024-09-20T22:10:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9530000000000005e-06 2024-09-20T22:10:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.823863+0000", 2024-09-20T22:10:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0093847810000001 2024-09-20T22:10:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.823872+0000", 2024-09-20T22:10:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.312e-06 2024-09-20T22:10:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.115762+0000", 2024-09-20T22:10:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.291889702 2024-09-20T22:10:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.115904+0000", 2024-09-20T22:10:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014230800000000001 2024-09-20T22:10:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:10:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.116318+0000", 2024-09-20T22:10:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00041407100000000001 2024-09-20T22:10:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.123684+0000", 2024-09-20T22:10:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0073657769999999996 2024-09-20T22:10:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.129612+0000", 2024-09-20T22:10:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0059283519999999996 2024-09-20T22:10:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130418+0000", 2024-09-20T22:10:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00080579299999999996 2024-09-20T22:10:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130473+0000", 2024-09-20T22:10:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4740000000000001e-05 2024-09-20T22:10:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130555+0000", 2024-09-20T22:10:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2393999999999998e-05 2024-09-20T22:10:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227652 7.4 7:213b8ad4:::benchmark_data_smithi060_78800_object94619:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:10:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.904523+0000", 2024-09-20T22:10:15.026 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 201.07777350200001, 2024-09-20T22:10:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2262305599999999, 2024-09-20T22:10:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:15.027 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227652 2024-09-20T22:10:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904523+0000", 2024-09-20T22:10:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904523+0000", 2024-09-20T22:10:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904524+0000", 2024-09-20T22:10:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0100000000000001e-06 2024-09-20T22:10:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904528+0000", 2024-09-20T22:10:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4710000000000001e-06 2024-09-20T22:10:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904537+0000", 2024-09-20T22:10:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2289999999999992e-06 2024-09-20T22:10:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904544+0000", 2024-09-20T22:10:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6900000000000003e-06 2024-09-20T22:10:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904608+0000", 2024-09-20T22:10:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3598999999999999e-05 2024-09-20T22:10:15.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904617+0000", 2024-09-20T22:10:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8179999999999999e-06 2024-09-20T22:10:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.823899+0000", 2024-09-20T22:10:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.919281338 2024-09-20T22:10:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.823907+0000", 2024-09-20T22:10:15.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9449999999999993e-06 2024-09-20T22:10:15.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.118500+0000", 2024-09-20T22:10:15.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2945934700000001 2024-09-20T22:10:15.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:15.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.118634+0000", 2024-09-20T22:10:15.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013352700000000001 2024-09-20T22:10:15.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:10:15.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.119027+0000", 2024-09-20T22:10:15.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000393533 2024-09-20T22:10:15.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.129598+0000", 2024-09-20T22:10:15.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010570941 2024-09-20T22:10:15.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:15.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.129619+0000", 2024-09-20T22:10:15.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0854000000000001e-05 2024-09-20T22:10:15.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130623+0000", 2024-09-20T22:10:15.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001003947 2024-09-20T22:10:15.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:15.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130663+0000", 2024-09-20T22:10:15.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0139000000000003e-05 2024-09-20T22:10:15.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:15.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130753+0000", 2024-09-20T22:10:15.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0049000000000005e-05 2024-09-20T22:10:15.034 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.034 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.034 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.034 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227664 7.4 7:264fd802:::benchmark_data_smithi060_78800_object94631:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:10:15.035 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.969131+0000", 2024-09-20T22:10:15.035 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 201.01316472600001, 2024-09-20T22:10:15.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1748220109999998, 2024-09-20T22:10:15.035 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:15.035 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:15.035 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:15.035 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:15.035 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:15.035 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:15.035 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227664 2024-09-20T22:10:15.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.035 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:15.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:15.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969131+0000", 2024-09-20T22:10:15.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:15.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969131+0000", 2024-09-20T22:10:15.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:15.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969133+0000", 2024-09-20T22:10:15.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.421e-06 2024-09-20T22:10:15.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:15.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969138+0000", 2024-09-20T22:10:15.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9239999999999998e-06 2024-09-20T22:10:15.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:15.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969148+0000", 2024-09-20T22:10:15.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0696000000000001e-05 2024-09-20T22:10:15.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:15.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969155+0000", 2024-09-20T22:10:15.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7780000000000002e-06 2024-09-20T22:10:15.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969217+0000", 2024-09-20T22:10:15.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2304000000000005e-05 2024-09-20T22:10:15.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969229+0000", 2024-09-20T22:10:15.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1100999999999999e-05 2024-09-20T22:10:15.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.824751+0000", 2024-09-20T22:10:15.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85552231099999998 2024-09-20T22:10:15.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.824761+0000", 2024-09-20T22:10:15.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9210000000000004e-06 2024-09-20T22:10:15.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.126989+0000", 2024-09-20T22:10:15.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.302228449 2024-09-20T22:10:15.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:15.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.127107+0000", 2024-09-20T22:10:15.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117712 2024-09-20T22:10:15.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:10:15.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.127530+0000", 2024-09-20T22:10:15.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000423056 2024-09-20T22:10:15.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:15.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130029+0000", 2024-09-20T22:10:15.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024987020000000002 2024-09-20T22:10:15.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.133706+0000", 2024-09-20T22:10:15.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0036771659999999999 2024-09-20T22:10:15.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.041 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:10:15.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.143463+0000", 2024-09-20T22:10:15.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0097571510000000004 2024-09-20T22:10:15.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:15.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.143714+0000", 2024-09-20T22:10:15.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00025083199999999999 2024-09-20T22:10:15.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:15.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.143953+0000", 2024-09-20T22:10:15.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023948700000000001 2024-09-20T22:10:15.042 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.042 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.042 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.042 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236412 7.9 7:953e1373:::benchmark_data_smithi060_78800_object103379:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-20T22:10:15.042 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.364320+0000", 2024-09-20T22:10:15.042 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 187.61797611099999, 2024-09-20T22:10:15.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1050446030000001, 2024-09-20T22:10:15.042 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:15.043 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:15.043 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:10:15.043 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:15.043 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:15.043 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:15.043 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236412 2024-09-20T22:10:15.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.043 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:15.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:15.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364320+0000", 2024-09-20T22:10:15.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:15.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364320+0000", 2024-09-20T22:10:15.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:15.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364321+0000", 2024-09-20T22:10:15.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.533e-06 2024-09-20T22:10:15.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:15.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364328+0000", 2024-09-20T22:10:15.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.241e-06 2024-09-20T22:10:15.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:15.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364346+0000", 2024-09-20T22:10:15.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8546000000000002e-05 2024-09-20T22:10:15.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:15.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364354+0000", 2024-09-20T22:10:15.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6839999999999996e-06 2024-09-20T22:10:15.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364429+0000", 2024-09-20T22:10:15.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4618999999999998e-05 2024-09-20T22:10:15.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:15.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364584+0000", 2024-09-20T22:10:15.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155725 2024-09-20T22:10:15.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:10:15.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.367359+0000", 2024-09-20T22:10:15.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002774421 2024-09-20T22:10:15.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.379883+0000", 2024-09-20T22:10:15.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.012524208 2024-09-20T22:10:15.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.379895+0000", 2024-09-20T22:10:15.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2268e-05 2024-09-20T22:10:15.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.421975+0000", 2024-09-20T22:10:15.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042079623 2024-09-20T22:10:15.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:10:15.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.422090+0000", 2024-09-20T22:10:15.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011518000000000001 2024-09-20T22:10:15.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.469296+0000", 2024-09-20T22:10:15.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.047205907999999998 2024-09-20T22:10:15.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:15.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.469365+0000", 2024-09-20T22:10:15.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8646999999999995e-05 2024-09-20T22:10:15.048 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.048 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.048 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.048 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236413 7.9 7:92cb2bc7:::benchmark_data_smithi060_78800_object103380:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-20T22:10:15.049 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.364373+0000", 2024-09-20T22:10:15.049 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 187.61792285199999, 2024-09-20T22:10:15.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.105379702, 2024-09-20T22:10:15.049 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:15.049 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:15.049 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:10:15.049 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:15.049 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:15.049 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:15.049 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236413 2024-09-20T22:10:15.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.049 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:15.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:15.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364373+0000", 2024-09-20T22:10:15.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:15.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364373+0000", 2024-09-20T22:10:15.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:15.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364374+0000", 2024-09-20T22:10:15.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2100000000000002e-07 2024-09-20T22:10:15.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:15.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364376+0000", 2024-09-20T22:10:15.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5440000000000001e-06 2024-09-20T22:10:15.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:15.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364383+0000", 2024-09-20T22:10:15.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7870000000000004e-06 2024-09-20T22:10:15.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:15.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364389+0000", 2024-09-20T22:10:15.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6540000000000004e-06 2024-09-20T22:10:15.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.374885+0000", 2024-09-20T22:10:15.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010496871 2024-09-20T22:10:15.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:15.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.375004+0000", 2024-09-20T22:10:15.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118956 2024-09-20T22:10:15.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:10:15.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.375466+0000", 2024-09-20T22:10:15.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00046160599999999998 2024-09-20T22:10:15.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.379926+0000", 2024-09-20T22:10:15.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0044598850000000002 2024-09-20T22:10:15.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.379935+0000", 2024-09-20T22:10:15.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8140000000000007e-06 2024-09-20T22:10:15.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.422377+0000", 2024-09-20T22:10:15.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042442028 2024-09-20T22:10:15.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:10:15.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.422482+0000", 2024-09-20T22:10:15.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010542 2024-09-20T22:10:15.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.469677+0000", 2024-09-20T22:10:15.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.047194699 2024-09-20T22:10:15.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:15.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.469753+0000", 2024-09-20T22:10:15.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6017000000000001e-05 2024-09-20T22:10:15.055 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.055 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.055 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.055 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236450 7.9 7:95eabd46:::benchmark_data_smithi060_78800_object103417:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:10:15.055 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.406110+0000", 2024-09-20T22:10:15.055 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 187.576186315, 2024-09-20T22:10:15.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0608129049999999, 2024-09-20T22:10:15.055 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:15.055 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:15.055 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:15.055 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:15.056 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:15.056 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:15.056 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236450 2024-09-20T22:10:15.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.056 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:15.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:15.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.406110+0000", 2024-09-20T22:10:15.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:15.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.406110+0000", 2024-09-20T22:10:15.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:15.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.406111+0000", 2024-09-20T22:10:15.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.248e-06 2024-09-20T22:10:15.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:15.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.406115+0000", 2024-09-20T22:10:15.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3830000000000002e-06 2024-09-20T22:10:15.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:15.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.406130+0000", 2024-09-20T22:10:15.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4456e-05 2024-09-20T22:10:15.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:15.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.406137+0000", 2024-09-20T22:10:15.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5329999999999997e-06 2024-09-20T22:10:15.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.406191+0000", 2024-09-20T22:10:15.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3764999999999998e-05 2024-09-20T22:10:15.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.058 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.406200+0000", 2024-09-20T22:10:15.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6710000000000005e-06 2024-09-20T22:10:15.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.422520+0000", 2024-09-20T22:10:15.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0163197589999999 2024-09-20T22:10:15.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:15.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.422622+0000", 2024-09-20T22:10:15.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010295 2024-09-20T22:10:15.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:10:15.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.423898+0000", 2024-09-20T22:10:15.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001275272 2024-09-20T22:10:15.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.447631+0000", 2024-09-20T22:10:15.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.023732896 2024-09-20T22:10:15.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:15.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.448017+0000", 2024-09-20T22:10:15.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000386723 2024-09-20T22:10:15.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.466783+0000", 2024-09-20T22:10:15.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.01876566 2024-09-20T22:10:15.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:15.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.466838+0000", 2024-09-20T22:10:15.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5241000000000002e-05 2024-09-20T22:10:15.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:15.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.466923+0000", 2024-09-20T22:10:15.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4347999999999999e-05 2024-09-20T22:10:15.061 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.061 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.062 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.062 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106944 8.12 8:4d0d0ddc:::benchmark_data_smithi060_111051_object106943:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:10:15.062 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.413909+0000", 2024-09-20T22:10:15.062 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 63.568386842000002, 2024-09-20T22:10:15.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.093576791, 2024-09-20T22:10:15.062 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:15.062 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:15.062 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:10:15.062 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:15.062 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:10:15.062 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:15.062 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106944 2024-09-20T22:10:15.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.063 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:15.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:15.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.413909+0000", 2024-09-20T22:10:15.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:15.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.413909+0000", 2024-09-20T22:10:15.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:15.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.413911+0000", 2024-09-20T22:10:15.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7129999999999999e-06 2024-09-20T22:10:15.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:15.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.413939+0000", 2024-09-20T22:10:15.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7881999999999998e-05 2024-09-20T22:10:15.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:15.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.413955+0000", 2024-09-20T22:10:15.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5835e-05 2024-09-20T22:10:15.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:15.065 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.413965+0000", 2024-09-20T22:10:15.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0832000000000001e-05 2024-09-20T22:10:15.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.065 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.065 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418483+0000", 2024-09-20T22:10:15.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0045172340000000002 2024-09-20T22:10:15.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.065 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.065 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418490+0000", 2024-09-20T22:10:15.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7000000000000008e-06 2024-09-20T22:10:15.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.065 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.507406+0000", 2024-09-20T22:10:15.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0889152179999999 2024-09-20T22:10:15.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:15.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.507486+0000", 2024-09-20T22:10:15.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0377000000000006e-05 2024-09-20T22:10:15.066 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.066 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.066 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.066 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106964 8.16 8:6977ad37:::benchmark_data_smithi060_111051_object106963:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:10:15.066 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418041+0000", 2024-09-20T22:10:15.067 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 63.564255342999999, 2024-09-20T22:10:15.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.094367187, 2024-09-20T22:10:15.067 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:15.067 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:15.067 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:10:15.067 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:15.067 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:10:15.067 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:15.067 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106964 2024-09-20T22:10:15.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.067 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:15.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:15.068 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:15.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418041+0000", 2024-09-20T22:10:15.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:15.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418041+0000", 2024-09-20T22:10:15.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:15.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418042+0000", 2024-09-20T22:10:15.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.302e-06 2024-09-20T22:10:15.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:15.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418104+0000", 2024-09-20T22:10:15.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1878999999999995e-05 2024-09-20T22:10:15.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:15.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418116+0000", 2024-09-20T22:10:15.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2177999999999999e-05 2024-09-20T22:10:15.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:15.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418124+0000", 2024-09-20T22:10:15.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9440000000000005e-06 2024-09-20T22:10:15.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418284+0000", 2024-09-20T22:10:15.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016023400000000001 2024-09-20T22:10:15.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418293+0000", 2024-09-20T22:10:15.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5720000000000003e-06 2024-09-20T22:10:15.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.512322+0000", 2024-09-20T22:10:15.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0940291790000001 2024-09-20T22:10:15.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:15.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.512408+0000", 2024-09-20T22:10:15.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5898999999999999e-05 2024-09-20T22:10:15.072 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.072 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.072 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.072 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106977 8.12 8:4ad51fd6:::benchmark_data_smithi060_111051_object106976:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:10:15.072 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418139+0000", 2024-09-20T22:10:15.072 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 63.564157475000002, 2024-09-20T22:10:15.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.107260704, 2024-09-20T22:10:15.073 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:15.073 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:15.073 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:15.073 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:15.073 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:10:15.073 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:15.073 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106977 2024-09-20T22:10:15.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.073 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:15.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:15.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418139+0000", 2024-09-20T22:10:15.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:15.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418139+0000", 2024-09-20T22:10:15.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:15.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418139+0000", 2024-09-20T22:10:15.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4300000000000003e-07 2024-09-20T22:10:15.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:15.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418194+0000", 2024-09-20T22:10:15.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4564999999999997e-05 2024-09-20T22:10:15.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:15.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418202+0000", 2024-09-20T22:10:15.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.755e-06 2024-09-20T22:10:15.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:15.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418211+0000", 2024-09-20T22:10:15.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5199999999999997e-06 2024-09-20T22:10:15.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418518+0000", 2024-09-20T22:10:15.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030665299999999999 2024-09-20T22:10:15.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418525+0000", 2024-09-20T22:10:15.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4930000000000001e-06 2024-09-20T22:10:15.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.508187+0000", 2024-09-20T22:10:15.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0896622950000001 2024-09-20T22:10:15.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:15.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.508457+0000", 2024-09-20T22:10:15.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00027002800000000001 2024-09-20T22:10:15.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,4,5,6,7", 2024-09-20T22:10:15.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.508645+0000", 2024-09-20T22:10:15.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018733000000000001 2024-09-20T22:10:15.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:15.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.514513+0000", 2024-09-20T22:10:15.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0058678259999999996 2024-09-20T22:10:15.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.514560+0000", 2024-09-20T22:10:15.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7314999999999997e-05 2024-09-20T22:10:15.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.514664+0000", 2024-09-20T22:10:15.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104125 2024-09-20T22:10:15.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515122+0000", 2024-09-20T22:10:15.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00045837200000000002 2024-09-20T22:10:15.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515704+0000", 2024-09-20T22:10:15.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00058129899999999997 2024-09-20T22:10:15.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525183+0000", 2024-09-20T22:10:15.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.009479019 2024-09-20T22:10:15.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:15.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525256+0000", 2024-09-20T22:10:15.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3410000000000004e-05 2024-09-20T22:10:15.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:15.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525399+0000", 2024-09-20T22:10:15.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000143156 2024-09-20T22:10:15.080 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.080 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.080 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.080 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106979 8.16 8:6c64d7d1:::benchmark_data_smithi060_111051_object106978:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:10:15.080 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418222+0000", 2024-09-20T22:10:15.080 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 63.564074400999999, 2024-09-20T22:10:15.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.107288856, 2024-09-20T22:10:15.080 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:15.080 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:15.080 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:15.081 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:15.081 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:10:15.081 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:15.081 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106979 2024-09-20T22:10:15.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.081 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:15.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:15.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418222+0000", 2024-09-20T22:10:15.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:15.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418222+0000", 2024-09-20T22:10:15.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:15.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418222+0000", 2024-09-20T22:10:15.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1900000000000003e-07 2024-09-20T22:10:15.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:15.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418273+0000", 2024-09-20T22:10:15.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0751000000000002e-05 2024-09-20T22:10:15.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:15.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418282+0000", 2024-09-20T22:10:15.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1439999999999992e-06 2024-09-20T22:10:15.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:15.083 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418289+0000", 2024-09-20T22:10:15.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4829999999999996e-06 2024-09-20T22:10:15.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.083 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418324+0000", 2024-09-20T22:10:15.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5508e-05 2024-09-20T22:10:15.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418331+0000", 2024-09-20T22:10:15.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2679999999999997e-06 2024-09-20T22:10:15.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.512703+0000", 2024-09-20T22:10:15.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.094371502 2024-09-20T22:10:15.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:15.084 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.512972+0000", 2024-09-20T22:10:15.084 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026964600000000001 2024-09-20T22:10:15.084 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.084 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.084 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5,6,7", 2024-09-20T22:10:15.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.513953+0000", 2024-09-20T22:10:15.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00098022600000000006 2024-09-20T22:10:15.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:15.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.519844+0000", 2024-09-20T22:10:15.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005891309 2024-09-20T22:10:15.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.520362+0000", 2024-09-20T22:10:15.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00051799699999999997 2024-09-20T22:10:15.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.520890+0000", 2024-09-20T22:10:15.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00052754400000000002 2024-09-20T22:10:15.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524659+0000", 2024-09-20T22:10:15.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037695160000000001 2024-09-20T22:10:15.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524693+0000", 2024-09-20T22:10:15.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3887999999999998e-05 2024-09-20T22:10:15.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525239+0000", 2024-09-20T22:10:15.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00054655600000000002 2024-09-20T22:10:15.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:15.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525313+0000", 2024-09-20T22:10:15.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3485999999999994e-05 2024-09-20T22:10:15.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:15.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525510+0000", 2024-09-20T22:10:15.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019751299999999999 2024-09-20T22:10:15.088 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.088 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.088 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.088 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106981 8.16 8:6961ec02:::benchmark_data_smithi060_111051_object106980:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:10:15.088 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418309+0000", 2024-09-20T22:10:15.088 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 63.563987271999999, 2024-09-20T22:10:15.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1092024140000001, 2024-09-20T22:10:15.088 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:15.088 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:15.088 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:15.088 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:15.089 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:10:15.089 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:15.089 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106981 2024-09-20T22:10:15.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.089 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:15.089 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.089 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:15.089 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418309+0000", 2024-09-20T22:10:15.089 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.089 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.089 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:15.089 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418309+0000", 2024-09-20T22:10:15.089 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:15.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418309+0000", 2024-09-20T22:10:15.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3000000000000001e-07 2024-09-20T22:10:15.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:15.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418370+0000", 2024-09-20T22:10:15.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0577e-05 2024-09-20T22:10:15.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:15.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418379+0000", 2024-09-20T22:10:15.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5279999999999992e-06 2024-09-20T22:10:15.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:15.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418385+0000", 2024-09-20T22:10:15.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.105e-06 2024-09-20T22:10:15.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418587+0000", 2024-09-20T22:10:15.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020189700000000001 2024-09-20T22:10:15.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418595+0000", 2024-09-20T22:10:15.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.187e-06 2024-09-20T22:10:15.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515314+0000", 2024-09-20T22:10:15.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0967195460000001 2024-09-20T22:10:15.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:15.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515442+0000", 2024-09-20T22:10:15.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012829599999999999 2024-09-20T22:10:15.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5,6,7", 2024-09-20T22:10:15.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515706+0000", 2024-09-20T22:10:15.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026385599999999998 2024-09-20T22:10:15.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.093 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:15.093 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.519902+0000", 2024-09-20T22:10:15.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.004195786 2024-09-20T22:10:15.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.093 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.093 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.520407+0000", 2024-09-20T22:10:15.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00050457400000000004 2024-09-20T22:10:15.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.093 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.093 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524729+0000", 2024-09-20T22:10:15.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0043228279999999999 2024-09-20T22:10:15.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524770+0000", 2024-09-20T22:10:15.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0528000000000002e-05 2024-09-20T22:10:15.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524965+0000", 2024-09-20T22:10:15.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019492000000000001 2024-09-20T22:10:15.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.527323+0000", 2024-09-20T22:10:15.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023577250000000002 2024-09-20T22:10:15.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:15.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.527382+0000", 2024-09-20T22:10:15.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9228999999999998e-05 2024-09-20T22:10:15.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:15.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.527511+0000", 2024-09-20T22:10:15.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129302 2024-09-20T22:10:15.096 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.096 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.096 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.096 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106997 8.16 8:6b0b3968:::benchmark_data_smithi060_111051_object106996:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:10:15.097 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418412+0000", 2024-09-20T22:10:15.097 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 63.563884336000001, 2024-09-20T22:10:15.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1094631589999999, 2024-09-20T22:10:15.097 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:15.097 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:15.097 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:15.097 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:15.097 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:10:15.097 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:15.097 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106997 2024-09-20T22:10:15.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.097 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:15.097 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.097 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:15.098 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418412+0000", 2024-09-20T22:10:15.098 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.098 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:15.098 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418412+0000", 2024-09-20T22:10:15.098 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.098 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:15.098 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418412+0000", 2024-09-20T22:10:15.098 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.13e-07 2024-09-20T22:10:15.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.099 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:15.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418461+0000", 2024-09-20T22:10:15.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8504999999999997e-05 2024-09-20T22:10:15.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.099 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:15.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418470+0000", 2024-09-20T22:10:15.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4660000000000005e-06 2024-09-20T22:10:15.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.099 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:15.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418476+0000", 2024-09-20T22:10:15.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2179999999999996e-06 2024-09-20T22:10:15.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418618+0000", 2024-09-20T22:10:15.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014114800000000001 2024-09-20T22:10:15.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418624+0000", 2024-09-20T22:10:15.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8009999999999998e-06 2024-09-20T22:10:15.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.517728+0000", 2024-09-20T22:10:15.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.099103513 2024-09-20T22:10:15.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:15.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.517868+0000", 2024-09-20T22:10:15.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014029400000000001 2024-09-20T22:10:15.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5,6,7", 2024-09-20T22:10:15.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.518178+0000", 2024-09-20T22:10:15.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030979800000000002 2024-09-20T22:10:15.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:15.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.519909+0000", 2024-09-20T22:10:15.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017308740000000001 2024-09-20T22:10:15.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.102 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.102 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.522366+0000", 2024-09-20T22:10:15.102 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024576609999999999 2024-09-20T22:10:15.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.102 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.102 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524823+0000", 2024-09-20T22:10:15.102 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024567830000000001 2024-09-20T22:10:15.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.102 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.102 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524862+0000", 2024-09-20T22:10:15.103 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9060000000000002e-05 2024-09-20T22:10:15.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.103 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.103 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.103 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525002+0000", 2024-09-20T22:10:15.103 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000139438 2024-09-20T22:10:15.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.103 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.103 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.103 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.527697+0000", 2024-09-20T22:10:15.103 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0026952149999999999 2024-09-20T22:10:15.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.103 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.103 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:15.104 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.527759+0000", 2024-09-20T22:10:15.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1612999999999994e-05 2024-09-20T22:10:15.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.104 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:15.104 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.527875+0000", 2024-09-20T22:10:15.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000116259 2024-09-20T22:10:15.104 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.104 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.104 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.104 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:107002 8.11 8:8e84af07:::benchmark_data_smithi060_111051_object107001:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:10:15.104 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418488+0000", 2024-09-20T22:10:15.104 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 63.563808252000001, 2024-09-20T22:10:15.105 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1327261360000001, 2024-09-20T22:10:15.105 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:15.105 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:15.105 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:15.105 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:15.105 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:10:15.105 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:15.105 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 107002 2024-09-20T22:10:15.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.105 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:15.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.105 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:15.105 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418488+0000", 2024-09-20T22:10:15.105 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:15.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418488+0000", 2024-09-20T22:10:15.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:15.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418488+0000", 2024-09-20T22:10:15.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4700000000000001e-07 2024-09-20T22:10:15.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:15.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418536+0000", 2024-09-20T22:10:15.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7899999999999999e-05 2024-09-20T22:10:15.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.107 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.107 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:15.107 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418545+0000", 2024-09-20T22:10:15.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5210000000000002e-06 2024-09-20T22:10:15.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.107 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.107 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:15.107 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418551+0000", 2024-09-20T22:10:15.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.409e-06 2024-09-20T22:10:15.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.107 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.107 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.107 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418608+0000", 2024-09-20T22:10:15.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7015999999999998e-05 2024-09-20T22:10:15.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.108 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.108 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.108 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418619+0000", 2024-09-20T22:10:15.108 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0900999999999999e-05 2024-09-20T22:10:15.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.108 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.108 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.108 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.538830+0000", 2024-09-20T22:10:15.108 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.120210792 2024-09-20T22:10:15.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.108 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.108 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:15.108 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.538979+0000", 2024-09-20T22:10:15.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014885199999999999 2024-09-20T22:10:15.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2,5,6,7", 2024-09-20T22:10:15.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.539152+0000", 2024-09-20T22:10:15.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000173134 2024-09-20T22:10:15.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.543638+0000", 2024-09-20T22:10:15.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0044860739999999996 2024-09-20T22:10:15.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.110 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.546226+0000", 2024-09-20T22:10:15.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00258807 2024-09-20T22:10:15.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.110 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.110 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.546267+0000", 2024-09-20T22:10:15.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0592e-05 2024-09-20T22:10:15.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.110 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.110 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.546317+0000", 2024-09-20T22:10:15.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0781000000000001e-05 2024-09-20T22:10:15.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.110 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:15.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.546370+0000", 2024-09-20T22:10:15.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2358999999999998e-05 2024-09-20T22:10:15.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.550100+0000", 2024-09-20T22:10:15.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037306309999999999 2024-09-20T22:10:15.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:15.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.551067+0000", 2024-09-20T22:10:15.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00096712000000000002 2024-09-20T22:10:15.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.112 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:15.112 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.551214+0000", 2024-09-20T22:10:15.112 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014643699999999999 2024-09-20T22:10:15.112 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.112 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.112 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.112 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.112 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:107007 8.12 8:4e9d1372:::benchmark_data_smithi060_111051_object107006:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:10:15.112 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418568+0000", 2024-09-20T22:10:15.112 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 63.563728249999997, 2024-09-20T22:10:15.112 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.108179131, 2024-09-20T22:10:15.112 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:15.112 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:15.112 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:15.113 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:15.113 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:10:15.113 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:15.113 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 107007 2024-09-20T22:10:15.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.113 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:15.113 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.113 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:15.113 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418568+0000", 2024-09-20T22:10:15.113 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.113 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.113 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:15.113 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418568+0000", 2024-09-20T22:10:15.114 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.114 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.114 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:15.114 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418568+0000", 2024-09-20T22:10:15.114 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3499999999999996e-07 2024-09-20T22:10:15.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.114 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.114 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:15.114 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418616+0000", 2024-09-20T22:10:15.114 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7386000000000003e-05 2024-09-20T22:10:15.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.114 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.114 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:15.115 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418624+0000", 2024-09-20T22:10:15.115 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1110000000000008e-06 2024-09-20T22:10:15.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.115 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.115 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:15.115 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418630+0000", 2024-09-20T22:10:15.115 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9390000000000002e-06 2024-09-20T22:10:15.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.115 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.115 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.115 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418675+0000", 2024-09-20T22:10:15.115 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5444000000000001e-05 2024-09-20T22:10:15.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.115 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.115 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.116 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418683+0000", 2024-09-20T22:10:15.116 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5140000000000004e-06 2024-09-20T22:10:15.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.116 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.116 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.116 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.510695+0000", 2024-09-20T22:10:15.116 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.092012032 2024-09-20T22:10:15.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.116 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.116 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:15.116 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.510853+0000", 2024-09-20T22:10:15.116 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015822899999999999 2024-09-20T22:10:15.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.116 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.116 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,4,5,6,7", 2024-09-20T22:10:15.117 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.511044+0000", 2024-09-20T22:10:15.117 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019054099999999999 2024-09-20T22:10:15.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.117 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.117 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:15.117 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.514574+0000", 2024-09-20T22:10:15.117 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003530858 2024-09-20T22:10:15.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.117 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.117 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.117 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515032+0000", 2024-09-20T22:10:15.117 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00045793899999999998 2024-09-20T22:10:15.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.117 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.118 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.118 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515080+0000", 2024-09-20T22:10:15.118 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8164999999999997e-05 2024-09-20T22:10:15.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.118 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.118 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.118 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515664+0000", 2024-09-20T22:10:15.118 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00058304799999999996 2024-09-20T22:10:15.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.118 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.118 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.118 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515792+0000", 2024-09-20T22:10:15.118 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012877 2024-09-20T22:10:15.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.119 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.119 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525673+0000", 2024-09-20T22:10:15.119 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0098810990000000008 2024-09-20T22:10:15.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.119 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:15.119 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.526607+0000", 2024-09-20T22:10:15.119 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00093347000000000005 2024-09-20T22:10:15.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.119 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:15.119 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.526747+0000", 2024-09-20T22:10:15.119 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140051 2024-09-20T22:10:15.119 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.120 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.120 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.120 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.120 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:107008 8.12 8:4cb7847c:::benchmark_data_smithi060_111051_object107007:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:10:15.120 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418644+0000", 2024-09-20T22:10:15.120 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 63.563652408999999, 2024-09-20T22:10:15.120 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1083103759999999, 2024-09-20T22:10:15.120 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:15.120 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:15.120 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:15.120 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:15.120 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:10:15.120 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:15.121 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 107008 2024-09-20T22:10:15.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.121 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:15.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.121 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:15.121 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418644+0000", 2024-09-20T22:10:15.121 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.121 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:15.121 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418644+0000", 2024-09-20T22:10:15.121 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:15.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.122 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:15.122 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418644+0000", 2024-09-20T22:10:15.122 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2e-07 2024-09-20T22:10:15.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.122 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.122 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:15.122 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418693+0000", 2024-09-20T22:10:15.122 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8569000000000002e-05 2024-09-20T22:10:15.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.122 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.122 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:15.122 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418702+0000", 2024-09-20T22:10:15.122 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8060000000000001e-06 2024-09-20T22:10:15.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.123 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:15.123 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418708+0000", 2024-09-20T22:10:15.123 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2500000000000003e-06 2024-09-20T22:10:15.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.123 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.123 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418797+0000", 2024-09-20T22:10:15.123 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9543999999999995e-05 2024-09-20T22:10:15.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.123 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:15.123 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418805+0000", 2024-09-20T22:10:15.123 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0290000000000005e-06 2024-09-20T22:10:15.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.124 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:15.124 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.512330+0000", 2024-09-20T22:10:15.124 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0935242919999999 2024-09-20T22:10:15.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.124 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:15.124 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.512473+0000", 2024-09-20T22:10:15.124 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000143695 2024-09-20T22:10:15.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.124 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,4,5,6,7", 2024-09-20T22:10:15.124 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.513190+0000", 2024-09-20T22:10:15.124 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00071678699999999998 2024-09-20T22:10:15.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.125 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.125 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.518925+0000", 2024-09-20T22:10:15.125 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0057347919999999998 2024-09-20T22:10:15.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.125 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.125 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.520073+0000", 2024-09-20T22:10:15.125 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0011479420000000001 2024-09-20T22:10:15.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.125 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.125 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:15.125 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.520313+0000", 2024-09-20T22:10:15.125 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023994399999999999 2024-09-20T22:10:15.125 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.126 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.126 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.523625+0000", 2024-09-20T22:10:15.126 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003312008 2024-09-20T22:10:15.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.126 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.126 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524809+0000", 2024-09-20T22:10:15.126 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001184456 2024-09-20T22:10:15.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.126 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:15.126 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.526788+0000", 2024-09-20T22:10:15.126 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001979218 2024-09-20T22:10:15.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.127 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.127 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:15.127 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.526838+0000", 2024-09-20T22:10:15.127 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9617999999999998e-05 2024-09-20T22:10:15.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:15.127 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:15.127 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:15.127 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.526954+0000", 2024-09-20T22:10:15.127 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115906 2024-09-20T22:10:15.127 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.127 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.127 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.127 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:15.127 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:15.128 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:10:15.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:15.131 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:15.131 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:15.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:15.268 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:15.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:15.269 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:15.325 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:10:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:10:15.370 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:15.417 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:15.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:15.418 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:15.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:15.549 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:15.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:15.550 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:15.651 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:15.685 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:15.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:15.686 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:15 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:10:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:15 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:10:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:15 smithi067 ceph-mon[33511]: osdmap e247: 8 total, 8 up, 8 in 2024-09-20T22:10:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:15 smithi067 ceph-mon[33511]: 5.e scrub starts 2024-09-20T22:10:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:15 smithi067 ceph-mon[33511]: 5.e scrub ok 2024-09-20T22:10:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:15 smithi067 ceph-mon[33511]: 5.1a scrub starts 2024-09-20T22:10:15.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:15 smithi067 ceph-mon[33511]: 5.1a scrub ok 2024-09-20T22:10:15.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:15 smithi067 ceph-mon[33511]: 4.a scrub starts 2024-09-20T22:10:15.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:15 smithi067 ceph-mon[33511]: 4.a scrub ok 2024-09-20T22:10:15.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:10:15.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:10:15.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-mon[29694]: osdmap e247: 8 total, 8 up, 8 in 2024-09-20T22:10:15.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-mon[29694]: 5.e scrub starts 2024-09-20T22:10:15.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-mon[29694]: 5.e scrub ok 2024-09-20T22:10:15.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-mon[29694]: 5.1a scrub starts 2024-09-20T22:10:15.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-mon[29694]: 5.1a scrub ok 2024-09-20T22:10:15.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-mon[29694]: 4.a scrub starts 2024-09-20T22:10:15.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-mon[29694]: 4.a scrub ok 2024-09-20T22:10:15.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:10:15.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:10:15.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-mon[40290]: osdmap e247: 8 total, 8 up, 8 in 2024-09-20T22:10:15.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-mon[40290]: 5.e scrub starts 2024-09-20T22:10:15.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-mon[40290]: 5.e scrub ok 2024-09-20T22:10:15.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-mon[40290]: 5.1a scrub starts 2024-09-20T22:10:15.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-mon[40290]: 5.1a scrub ok 2024-09-20T22:10:15.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-mon[40290]: 4.a scrub starts 2024-09-20T22:10:15.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:15 smithi060 ceph-mon[40290]: 4.a scrub ok 2024-09-20T22:10:15.788 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:15.831 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:15.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:15.832 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:15.932 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:15.963 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:15.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:15.964 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:16.065 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:16.100 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:16.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:16.101 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:16.201 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:16.295 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:16.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:16.296 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:16.341 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:16.397 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:16.433 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:16.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:16.434 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:16.535 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:16 smithi060 ceph-mon[29694]: 5.15 scrub starts 2024-09-20T22:10:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:16 smithi060 ceph-mon[29694]: 5.15 scrub ok 2024-09-20T22:10:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:16 smithi060 ceph-mon[29694]: 8.3 scrub starts 2024-09-20T22:10:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:16 smithi060 ceph-mon[29694]: 8.10 scrub starts 2024-09-20T22:10:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:16 smithi060 ceph-mon[29694]: 1.2 scrub starts 2024-09-20T22:10:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:16 smithi060 ceph-mon[29694]: 1.2 scrub ok 2024-09-20T22:10:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:16 smithi060 ceph-mon[29694]: pgmap v385: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 1.34k op/s; 9036/216411 objects misplaced (4.175%); 2.6 MiB/s, 40 objects/s recovering 2024-09-20T22:10:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:16 smithi060 ceph-mon[29694]: 8.12 scrub starts 2024-09-20T22:10:16.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:16 smithi060 ceph-mon[40290]: 5.15 scrub starts 2024-09-20T22:10:16.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:16 smithi060 ceph-mon[40290]: 5.15 scrub ok 2024-09-20T22:10:16.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:16 smithi060 ceph-mon[40290]: 8.3 scrub starts 2024-09-20T22:10:16.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:16 smithi060 ceph-mon[40290]: 8.10 scrub starts 2024-09-20T22:10:16.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:16 smithi060 ceph-mon[40290]: 1.2 scrub starts 2024-09-20T22:10:16.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:16 smithi060 ceph-mon[40290]: 1.2 scrub ok 2024-09-20T22:10:16.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:16 smithi060 ceph-mon[40290]: pgmap v385: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 1.34k op/s; 9036/216411 objects misplaced (4.175%); 2.6 MiB/s, 40 objects/s recovering 2024-09-20T22:10:16.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:16 smithi060 ceph-mon[40290]: 8.12 scrub starts 2024-09-20T22:10:16.577 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:16.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:16.578 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:16.678 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:16.713 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:16.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:16.714 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:16 smithi067 ceph-mon[33511]: 5.15 scrub starts 2024-09-20T22:10:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:16 smithi067 ceph-mon[33511]: 5.15 scrub ok 2024-09-20T22:10:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:16 smithi067 ceph-mon[33511]: 8.3 scrub starts 2024-09-20T22:10:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:16 smithi067 ceph-mon[33511]: 8.10 scrub starts 2024-09-20T22:10:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:16 smithi067 ceph-mon[33511]: 1.2 scrub starts 2024-09-20T22:10:16.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:16 smithi067 ceph-mon[33511]: 1.2 scrub ok 2024-09-20T22:10:16.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:16 smithi067 ceph-mon[33511]: pgmap v385: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.4 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 1.34k op/s; 9036/216411 objects misplaced (4.175%); 2.6 MiB/s, 40 objects/s recovering 2024-09-20T22:10:16.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:16 smithi067 ceph-mon[33511]: 8.12 scrub starts 2024-09-20T22:10:16.815 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:16.852 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:16.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:16.853 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:16.954 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:16.975 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T22:10:16.994 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:16.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:16.995 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:17.096 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:17.127 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:17.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:17.128 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:17.229 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:17.259 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:17.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:17.260 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:17.361 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:17.400 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:17.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:17.400 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:17.501 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:17.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[29694]: 8.3 scrub ok 2024-09-20T22:10:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[29694]: 8.19 scrub starts 2024-09-20T22:10:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[29694]: 8.7 scrub starts 2024-09-20T22:10:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[29694]: 4.1c scrub starts 2024-09-20T22:10:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[29694]: 4.1c scrub ok 2024-09-20T22:10:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[29694]: 8.10 scrub ok 2024-09-20T22:10:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[29694]: 8.12 scrub ok 2024-09-20T22:10:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[29694]: 8.17 scrub starts 2024-09-20T22:10:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[29694]: 1.0 scrub starts 2024-09-20T22:10:17.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[29694]: 1.0 scrub ok 2024-09-20T22:10:17.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[40290]: 8.3 scrub ok 2024-09-20T22:10:17.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[40290]: 8.19 scrub starts 2024-09-20T22:10:17.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[40290]: 8.7 scrub starts 2024-09-20T22:10:17.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[40290]: 4.1c scrub starts 2024-09-20T22:10:17.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[40290]: 4.1c scrub ok 2024-09-20T22:10:17.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[40290]: 8.10 scrub ok 2024-09-20T22:10:17.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[40290]: 8.12 scrub ok 2024-09-20T22:10:17.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[40290]: 8.17 scrub starts 2024-09-20T22:10:17.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[40290]: 1.0 scrub starts 2024-09-20T22:10:17.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:17 smithi060 ceph-mon[40290]: 1.0 scrub ok 2024-09-20T22:10:17.538 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:17.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:17.539 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:17.629 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:10:17.629 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:10:17.632 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:17.640 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:17.665 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:10:17.666 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:10:17.676 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:17.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:17.677 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:17.699 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:10:17.699 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:10:17.731 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:10:17.731 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:10:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:17 smithi067 ceph-mon[33511]: 8.3 scrub ok 2024-09-20T22:10:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:17 smithi067 ceph-mon[33511]: 8.19 scrub starts 2024-09-20T22:10:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:17 smithi067 ceph-mon[33511]: 8.7 scrub starts 2024-09-20T22:10:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:17 smithi067 ceph-mon[33511]: 4.1c scrub starts 2024-09-20T22:10:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:17 smithi067 ceph-mon[33511]: 4.1c scrub ok 2024-09-20T22:10:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:17 smithi067 ceph-mon[33511]: 8.10 scrub ok 2024-09-20T22:10:17.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:17 smithi067 ceph-mon[33511]: 8.12 scrub ok 2024-09-20T22:10:17.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:17 smithi067 ceph-mon[33511]: 8.17 scrub starts 2024-09-20T22:10:17.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:17 smithi067 ceph-mon[33511]: 1.0 scrub starts 2024-09-20T22:10:17.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:17 smithi067 ceph-mon[33511]: 1.0 scrub ok 2024-09-20T22:10:17.765 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:10:17.765 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:10:17.777 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:17.808 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:10:17.808 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:10:17.809 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:17.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:17.810 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:17.841 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:10:17.841 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:10:17.874 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:10:17.874 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:10:17.911 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:17.940 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:17.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:17.941 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:18.041 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:18.084 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:18.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:18.085 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:18.186 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:18.218 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:18.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:18.219 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:18.320 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:18.320 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:10:18.366 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:18.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:18.367 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:18.466 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:10:18.467 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:10:18.468 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:18.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:18.512 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:18.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:18.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[29694]: 8.19 scrub ok 2024-09-20T22:10:18.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[29694]: 8.7 scrub ok 2024-09-20T22:10:18.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[29694]: 6.c scrub starts 2024-09-20T22:10:18.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[29694]: 6.c scrub ok 2024-09-20T22:10:18.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[29694]: 8.11 scrub starts 2024-09-20T22:10:18.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[29694]: 8.17 scrub ok 2024-09-20T22:10:18.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[29694]: 2.1 scrub starts 2024-09-20T22:10:18.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[29694]: 2.1 scrub ok 2024-09-20T22:10:18.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1642303274' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:18.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[29694]: 8.6 scrub starts 2024-09-20T22:10:18.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[29694]: pgmap v386: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 8472/206121 objects misplaced (4.110%) 2024-09-20T22:10:18.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[29694]: 8.5 scrub starts 2024-09-20T22:10:18.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[40290]: 8.19 scrub ok 2024-09-20T22:10:18.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[40290]: 8.7 scrub ok 2024-09-20T22:10:18.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[40290]: 6.c scrub starts 2024-09-20T22:10:18.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[40290]: 6.c scrub ok 2024-09-20T22:10:18.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[40290]: 8.11 scrub starts 2024-09-20T22:10:18.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[40290]: 8.17 scrub ok 2024-09-20T22:10:18.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[40290]: 2.1 scrub starts 2024-09-20T22:10:18.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[40290]: 2.1 scrub ok 2024-09-20T22:10:18.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1642303274' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:18.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[40290]: 8.6 scrub starts 2024-09-20T22:10:18.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[40290]: pgmap v386: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 8472/206121 objects misplaced (4.110%) 2024-09-20T22:10:18.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:18 smithi060 ceph-mon[40290]: 8.5 scrub starts 2024-09-20T22:10:18.613 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:18.660 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:18.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:18.661 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:18.761 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:18 smithi067 ceph-mon[33511]: 8.19 scrub ok 2024-09-20T22:10:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:18 smithi067 ceph-mon[33511]: 8.7 scrub ok 2024-09-20T22:10:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:18 smithi067 ceph-mon[33511]: 6.c scrub starts 2024-09-20T22:10:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:18 smithi067 ceph-mon[33511]: 6.c scrub ok 2024-09-20T22:10:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:18 smithi067 ceph-mon[33511]: 8.11 scrub starts 2024-09-20T22:10:18.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:18 smithi067 ceph-mon[33511]: 8.17 scrub ok 2024-09-20T22:10:18.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:18 smithi067 ceph-mon[33511]: 2.1 scrub starts 2024-09-20T22:10:18.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:18 smithi067 ceph-mon[33511]: 2.1 scrub ok 2024-09-20T22:10:18.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:18 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1642303274' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:18.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:18 smithi067 ceph-mon[33511]: 8.6 scrub starts 2024-09-20T22:10:18.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:18 smithi067 ceph-mon[33511]: pgmap v386: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 8472/206121 objects misplaced (4.110%) 2024-09-20T22:10:18.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:18 smithi067 ceph-mon[33511]: 8.5 scrub starts 2024-09-20T22:10:18.801 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:18.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:18.802 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:18.902 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:18.938 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:18.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:18.939 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:19.039 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:19.075 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:19.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:19.076 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:19.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:19.202 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:19.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:19.203 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:19.303 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:19.332 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:19.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:19.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:19.434 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:19.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:19.471 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:19.471 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:19.572 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:19.603 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:19.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:19.605 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:19.705 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:19.707 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:10:19.708 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:10:19.708 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-20T22:10:19.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd test-reweight-by-utilization 1000 1.0 2024-09-20T22:10:19.736 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:19.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:19.737 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:19.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:19 smithi060 ceph-mon[29694]: 8.11 scrub ok 2024-09-20T22:10:19.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:19 smithi060 ceph-mon[29694]: 3.2 scrub starts 2024-09-20T22:10:19.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:19 smithi060 ceph-mon[29694]: 3.2 scrub ok 2024-09-20T22:10:19.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:19 smithi060 ceph-mon[29694]: 4.12 scrub starts 2024-09-20T22:10:19.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:19 smithi060 ceph-mon[29694]: 4.12 scrub ok 2024-09-20T22:10:19.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:19 smithi060 ceph-mon[29694]: 8.5 scrub ok 2024-09-20T22:10:19.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:19 smithi060 ceph-mon[29694]: 8.6 scrub ok 2024-09-20T22:10:19.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:19 smithi060 ceph-mon[29694]: 6.d scrub starts 2024-09-20T22:10:19.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:19 smithi060 ceph-mon[29694]: 6.d scrub ok 2024-09-20T22:10:19.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:19 smithi060 ceph-mon[40290]: 8.11 scrub ok 2024-09-20T22:10:19.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:19 smithi060 ceph-mon[40290]: 3.2 scrub starts 2024-09-20T22:10:19.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:19 smithi060 ceph-mon[40290]: 3.2 scrub ok 2024-09-20T22:10:19.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:19 smithi060 ceph-mon[40290]: 4.12 scrub starts 2024-09-20T22:10:19.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:19 smithi060 ceph-mon[40290]: 4.12 scrub ok 2024-09-20T22:10:19.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:19 smithi060 ceph-mon[40290]: 8.5 scrub ok 2024-09-20T22:10:19.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:19 smithi060 ceph-mon[40290]: 8.6 scrub ok 2024-09-20T22:10:19.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:19 smithi060 ceph-mon[40290]: 6.d scrub starts 2024-09-20T22:10:19.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:19 smithi060 ceph-mon[40290]: 6.d scrub ok 2024-09-20T22:10:19.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:19.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:19.880 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:19.880 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:19.980 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:20.005 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:20.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:20.006 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:19 smithi067 ceph-mon[33511]: 8.11 scrub ok 2024-09-20T22:10:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:19 smithi067 ceph-mon[33511]: 3.2 scrub starts 2024-09-20T22:10:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:19 smithi067 ceph-mon[33511]: 3.2 scrub ok 2024-09-20T22:10:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:19 smithi067 ceph-mon[33511]: 4.12 scrub starts 2024-09-20T22:10:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:19 smithi067 ceph-mon[33511]: 4.12 scrub ok 2024-09-20T22:10:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:19 smithi067 ceph-mon[33511]: 8.5 scrub ok 2024-09-20T22:10:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:19 smithi067 ceph-mon[33511]: 8.6 scrub ok 2024-09-20T22:10:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:19 smithi067 ceph-mon[33511]: 6.d scrub starts 2024-09-20T22:10:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:19 smithi067 ceph-mon[33511]: 6.d scrub ok 2024-09-20T22:10:20.107 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:20.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:20.155 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:20.156 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:20.256 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:20.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:20.278 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:20.278 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:20.379 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:20.421 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:20.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:20.422 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:20.522 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:20.553 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:20.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:20.554 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:20.655 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:20.692 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:20.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:20.694 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:20.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[29694]: 3.8 scrub starts 2024-09-20T22:10:20.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[29694]: 3.8 scrub ok 2024-09-20T22:10:20.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[29694]: 5.7 scrub starts 2024-09-20T22:10:20.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[29694]: 5.7 scrub ok 2024-09-20T22:10:20.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[29694]: 6.1d scrub starts 2024-09-20T22:10:20.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[29694]: 6.1d scrub ok 2024-09-20T22:10:20.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[29694]: 5.8 scrub starts 2024-09-20T22:10:20.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[29694]: 5.8 scrub ok 2024-09-20T22:10:20.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[29694]: pgmap v387: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 8472/206121 objects misplaced (4.110%) 2024-09-20T22:10:20.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:10:20.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[40290]: 3.8 scrub starts 2024-09-20T22:10:20.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[40290]: 3.8 scrub ok 2024-09-20T22:10:20.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[40290]: 5.7 scrub starts 2024-09-20T22:10:20.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[40290]: 5.7 scrub ok 2024-09-20T22:10:20.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[40290]: 6.1d scrub starts 2024-09-20T22:10:20.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[40290]: 6.1d scrub ok 2024-09-20T22:10:20.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[40290]: 5.8 scrub starts 2024-09-20T22:10:20.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[40290]: 5.8 scrub ok 2024-09-20T22:10:20.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[40290]: pgmap v387: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 8472/206121 objects misplaced (4.110%) 2024-09-20T22:10:20.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:10:20.795 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:20.835 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:20.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:20.836 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:20.937 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:20.976 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:20.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:20.977 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:20 smithi067 ceph-mon[33511]: 3.8 scrub starts 2024-09-20T22:10:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:20 smithi067 ceph-mon[33511]: 3.8 scrub ok 2024-09-20T22:10:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:20 smithi067 ceph-mon[33511]: 5.7 scrub starts 2024-09-20T22:10:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:20 smithi067 ceph-mon[33511]: 5.7 scrub ok 2024-09-20T22:10:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:20 smithi067 ceph-mon[33511]: 6.1d scrub starts 2024-09-20T22:10:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:20 smithi067 ceph-mon[33511]: 6.1d scrub ok 2024-09-20T22:10:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:20 smithi067 ceph-mon[33511]: 5.8 scrub starts 2024-09-20T22:10:21.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:20 smithi067 ceph-mon[33511]: 5.8 scrub ok 2024-09-20T22:10:21.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:20 smithi067 ceph-mon[33511]: pgmap v387: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 4.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 8472/206121 objects misplaced (4.110%) 2024-09-20T22:10:21.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:10:21.078 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:21.108 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:21.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:21.109 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:21.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:21.247 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:21.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:21.248 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:21.348 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:21.375 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:21.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:21.376 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:21.477 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:21.519 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:21.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:21.521 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:21.621 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:21.652 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:21.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:21.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:21.754 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:21.796 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:21.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:21.797 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:21.898 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:21.938 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:21.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:21.939 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:21 smithi067 ceph-mon[33511]: 2.0 scrub starts 2024-09-20T22:10:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:21 smithi067 ceph-mon[33511]: 2.0 scrub ok 2024-09-20T22:10:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:21 smithi067 ceph-mon[33511]: 5.16 scrub starts 2024-09-20T22:10:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:21 smithi067 ceph-mon[33511]: 5.16 scrub ok 2024-09-20T22:10:22.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:21 smithi067 ceph-mon[33511]: 4.10 scrub starts 2024-09-20T22:10:22.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:21 smithi067 ceph-mon[33511]: 4.10 scrub ok 2024-09-20T22:10:22.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 7]}]: dispatch 2024-09-20T22:10:22.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 0]}]: dispatch 2024-09-20T22:10:22.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:21 smithi060 ceph-mon[29694]: 2.0 scrub starts 2024-09-20T22:10:22.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:21 smithi060 ceph-mon[29694]: 2.0 scrub ok 2024-09-20T22:10:22.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:21 smithi060 ceph-mon[29694]: 5.16 scrub starts 2024-09-20T22:10:22.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:21 smithi060 ceph-mon[29694]: 5.16 scrub ok 2024-09-20T22:10:22.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:21 smithi060 ceph-mon[29694]: 4.10 scrub starts 2024-09-20T22:10:22.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:21 smithi060 ceph-mon[29694]: 4.10 scrub ok 2024-09-20T22:10:22.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 7]}]: dispatch 2024-09-20T22:10:22.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 0]}]: dispatch 2024-09-20T22:10:22.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:21 smithi060 ceph-mon[40290]: 2.0 scrub starts 2024-09-20T22:10:22.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:21 smithi060 ceph-mon[40290]: 2.0 scrub ok 2024-09-20T22:10:22.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:21 smithi060 ceph-mon[40290]: 5.16 scrub starts 2024-09-20T22:10:22.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:21 smithi060 ceph-mon[40290]: 5.16 scrub ok 2024-09-20T22:10:22.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:21 smithi060 ceph-mon[40290]: 4.10 scrub starts 2024-09-20T22:10:22.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:21 smithi060 ceph-mon[40290]: 4.10 scrub ok 2024-09-20T22:10:22.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 7]}]: dispatch 2024-09-20T22:10:22.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 0]}]: dispatch 2024-09-20T22:10:22.039 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:22.077 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:22.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:22.078 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:22.179 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:22.217 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T22:10:22.228 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:22.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:22.229 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:22.329 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:22.356 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:22.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:22.357 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:22.412 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:22.457 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:22.483 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:22.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:22.484 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:22.513 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:10:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:10:22.585 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:22.619 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:22.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:22.638 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:22.639 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:22.739 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:22.791 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:22.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:22.792 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:22.833 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[29694]: 2.2 scrub starts 2024-09-20T22:10:22.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[29694]: 2.2 scrub ok 2024-09-20T22:10:22.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 7]}]': finished 2024-09-20T22:10:22.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 0]}]': finished 2024-09-20T22:10:22.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[29694]: osdmap e248: 8 total, 8 up, 8 in 2024-09-20T22:10:22.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[29694]: 4.9 scrub starts 2024-09-20T22:10:22.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[29694]: 4.9 scrub ok 2024-09-20T22:10:22.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[29694]: pgmap v389: 168 pgs: 1 peering, 2 active+remapped+backfill_toofull, 165 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 7658/190866 objects misplaced (4.012%) 2024-09-20T22:10:22.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:10:22.834 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[29694]: osdmap e249: 8 total, 8 up, 8 in 2024-09-20T22:10:22.834 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[40290]: 2.2 scrub starts 2024-09-20T22:10:22.835 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[40290]: 2.2 scrub ok 2024-09-20T22:10:22.835 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 7]}]': finished 2024-09-20T22:10:22.835 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 0]}]': finished 2024-09-20T22:10:22.835 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[40290]: osdmap e248: 8 total, 8 up, 8 in 2024-09-20T22:10:22.835 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[40290]: 4.9 scrub starts 2024-09-20T22:10:22.835 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[40290]: 4.9 scrub ok 2024-09-20T22:10:22.835 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[40290]: pgmap v389: 168 pgs: 1 peering, 2 active+remapped+backfill_toofull, 165 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 7658/190866 objects misplaced (4.012%) 2024-09-20T22:10:22.835 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:10:22.835 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:22 smithi060 ceph-mon[40290]: osdmap e249: 8 total, 8 up, 8 in 2024-09-20T22:10:22.893 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:22.939 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:22.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:22.940 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:23.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:22 smithi067 ceph-mon[33511]: 2.2 scrub starts 2024-09-20T22:10:23.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:22 smithi067 ceph-mon[33511]: 2.2 scrub ok 2024-09-20T22:10:23.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:22 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.b", "id": [1, 7]}]': finished 2024-09-20T22:10:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:22 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [1, 0]}]': finished 2024-09-20T22:10:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:22 smithi067 ceph-mon[33511]: osdmap e248: 8 total, 8 up, 8 in 2024-09-20T22:10:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:22 smithi067 ceph-mon[33511]: 4.9 scrub starts 2024-09-20T22:10:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:22 smithi067 ceph-mon[33511]: 4.9 scrub ok 2024-09-20T22:10:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:22 smithi067 ceph-mon[33511]: pgmap v389: 168 pgs: 1 peering, 2 active+remapped+backfill_toofull, 165 active+clean; 3.9 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 7658/190866 objects misplaced (4.012%) 2024-09-20T22:10:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:22 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:10:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:22 smithi067 ceph-mon[33511]: osdmap e249: 8 total, 8 up, 8 in 2024-09-20T22:10:23.040 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:23.076 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:23.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:23.077 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:23.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:23.218 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:23.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:23.219 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:23.320 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:23.348 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:23.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:23.349 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:23.450 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:23.488 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:23.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:23.489 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:23.590 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:23.629 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:23.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:23.630 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:23.730 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:23.761 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:23.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:23.762 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:23.850 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:23.863 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:23.890 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:23.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:23.891 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:23.992 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:23.996 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[29694]: 6.3 scrub starts 2024-09-20T22:10:23.996 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[29694]: 6.3 scrub ok 2024-09-20T22:10:23.996 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[29694]: 5.19 scrub starts 2024-09-20T22:10:23.996 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[29694]: 5.19 scrub ok 2024-09-20T22:10:23.996 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[29694]: 5.a scrub starts 2024-09-20T22:10:23.996 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[29694]: 5.a scrub ok 2024-09-20T22:10:23.996 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[29694]: 5.0 scrub starts 2024-09-20T22:10:23.996 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[29694]: 5.0 scrub ok 2024-09-20T22:10:23.996 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[29694]: 3.10 scrub starts 2024-09-20T22:10:23.996 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[29694]: 3.10 scrub ok 2024-09-20T22:10:23.997 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[40290]: 6.3 scrub starts 2024-09-20T22:10:23.997 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[40290]: 6.3 scrub ok 2024-09-20T22:10:23.997 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[40290]: 5.19 scrub starts 2024-09-20T22:10:23.997 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[40290]: 5.19 scrub ok 2024-09-20T22:10:23.997 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[40290]: 5.a scrub starts 2024-09-20T22:10:23.997 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[40290]: 5.a scrub ok 2024-09-20T22:10:23.997 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[40290]: 5.0 scrub starts 2024-09-20T22:10:23.997 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[40290]: 5.0 scrub ok 2024-09-20T22:10:23.997 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[40290]: 3.10 scrub starts 2024-09-20T22:10:23.997 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:23 smithi060 ceph-mon[40290]: 3.10 scrub ok 2024-09-20T22:10:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:23 smithi067 ceph-mon[33511]: 6.3 scrub starts 2024-09-20T22:10:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:23 smithi067 ceph-mon[33511]: 6.3 scrub ok 2024-09-20T22:10:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:23 smithi067 ceph-mon[33511]: 5.19 scrub starts 2024-09-20T22:10:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:23 smithi067 ceph-mon[33511]: 5.19 scrub ok 2024-09-20T22:10:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:23 smithi067 ceph-mon[33511]: 5.a scrub starts 2024-09-20T22:10:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:23 smithi067 ceph-mon[33511]: 5.a scrub ok 2024-09-20T22:10:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:23 smithi067 ceph-mon[33511]: 5.0 scrub starts 2024-09-20T22:10:24.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:23 smithi067 ceph-mon[33511]: 5.0 scrub ok 2024-09-20T22:10:24.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:23 smithi067 ceph-mon[33511]: 3.10 scrub starts 2024-09-20T22:10:24.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:23 smithi067 ceph-mon[33511]: 3.10 scrub ok 2024-09-20T22:10:24.030 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:24.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:24.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:24.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:24.152 INFO:teuthology.orchestra.run.smithi060.stdout:moved 8 / 504 (1.5873%) 2024-09-20T22:10:24.152 INFO:teuthology.orchestra.run.smithi060.stdout:avg 63 2024-09-20T22:10:24.152 INFO:teuthology.orchestra.run.smithi060.stdout:stddev 7.26292 -> 4.71699 (expected baseline 7.42462) 2024-09-20T22:10:24.152 INFO:teuthology.orchestra.run.smithi060.stdout:min osd.7 with 47 -> 55 pgs (0.746032 -> 0.873016 * mean) 2024-09-20T22:10:24.152 INFO:teuthology.orchestra.run.smithi060.stdout:max osd.4 with 71 -> 70 pgs (1.12698 -> 1.11111 * mean) 2024-09-20T22:10:24.152 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:10:24.152 INFO:teuthology.orchestra.run.smithi060.stdout:oload 1000 2024-09-20T22:10:24.153 INFO:teuthology.orchestra.run.smithi060.stdout:max_change 1 2024-09-20T22:10:24.153 INFO:teuthology.orchestra.run.smithi060.stdout:max_change_osds 4 2024-09-20T22:10:24.153 INFO:teuthology.orchestra.run.smithi060.stdout:average_utilization 0.0209 2024-09-20T22:10:24.153 INFO:teuthology.orchestra.run.smithi060.stdout:overload_utilization 0.2090 2024-09-20T22:10:24.153 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 weight 0.7327 -> 0.8002 2024-09-20T22:10:24.153 INFO:teuthology.orchestra.run.smithi060.stderr:no change 2024-09-20T22:10:24.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:24.167 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:24.167 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:24.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:24.285 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:10:24.145+0000 7fa558495640 -1 mgr.server reply reply (1) Operation not permitted no change 2024-09-20T22:10:24.312 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:24.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:24.313 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:24.372 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:10:24.414 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:24.474 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:24.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:24.475 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:24.576 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:24.594 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:10:24.605 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[29694]: 6.11 scrub starts 2024-09-20T22:10:24.605 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[29694]: 6.11 scrub ok 2024-09-20T22:10:24.605 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[29694]: 6.1a scrub starts 2024-09-20T22:10:24.605 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[29694]: 6.1a scrub ok 2024-09-20T22:10:24.606 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1504827985' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:10:24.606 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:10:24.606 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[29694]: 6.0 scrub starts 2024-09-20T22:10:24.606 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[29694]: 6.0 scrub ok 2024-09-20T22:10:24.606 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:24.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:24.607 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:24.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:24.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:24.763 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:24.763 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:24.849 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:10:24.849 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:10:24.864 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:24.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[29694]: pgmap v391: 168 pgs: 1 peering, 2 active+remapped+backfill_toofull, 165 active+clean; 3.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 7658/187878 objects misplaced (4.076%) 2024-09-20T22:10:24.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[29694]: 4.5 scrub starts 2024-09-20T22:10:24.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[29694]: 4.5 scrub ok 2024-09-20T22:10:24.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[29694]: from='client.25168 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:10:24.865 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[40290]: 6.11 scrub starts 2024-09-20T22:10:24.865 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[40290]: 6.11 scrub ok 2024-09-20T22:10:24.865 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[40290]: 6.1a scrub starts 2024-09-20T22:10:24.865 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[40290]: 6.1a scrub ok 2024-09-20T22:10:24.865 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1504827985' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:10:24.865 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:10:24.865 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[40290]: 6.0 scrub starts 2024-09-20T22:10:24.866 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[40290]: 6.0 scrub ok 2024-09-20T22:10:24.866 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[40290]: pgmap v391: 168 pgs: 1 peering, 2 active+remapped+backfill_toofull, 165 active+clean; 3.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 7658/187878 objects misplaced (4.076%) 2024-09-20T22:10:24.866 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[40290]: 4.5 scrub starts 2024-09-20T22:10:24.866 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[40290]: 4.5 scrub ok 2024-09-20T22:10:24.866 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:24 smithi060 ceph-mon[40290]: from='client.25168 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:10:24.909 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:24.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:24.910 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:25.010 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:24 smithi067 ceph-mon[33511]: 6.11 scrub starts 2024-09-20T22:10:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:24 smithi067 ceph-mon[33511]: 6.11 scrub ok 2024-09-20T22:10:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:24 smithi067 ceph-mon[33511]: 6.1a scrub starts 2024-09-20T22:10:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:24 smithi067 ceph-mon[33511]: 6.1a scrub ok 2024-09-20T22:10:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:24 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1504827985' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:10:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:24 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:10:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:24 smithi067 ceph-mon[33511]: 6.0 scrub starts 2024-09-20T22:10:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:24 smithi067 ceph-mon[33511]: 6.0 scrub ok 2024-09-20T22:10:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:24 smithi067 ceph-mon[33511]: pgmap v391: 168 pgs: 1 peering, 2 active+remapped+backfill_toofull, 165 active+clean; 3.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 7658/187878 objects misplaced (4.076%) 2024-09-20T22:10:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:24 smithi067 ceph-mon[33511]: 4.5 scrub starts 2024-09-20T22:10:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:24 smithi067 ceph-mon[33511]: 4.5 scrub ok 2024-09-20T22:10:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:24 smithi067 ceph-mon[33511]: from='client.25168 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:10:25.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:25.055 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:25.055 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:25.142 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:10:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:10:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:10:25.155 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:25.209 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:25.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:25.210 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:25.311 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:25.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:25.348 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:25.348 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:25.449 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:25.490 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:25.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:25.491 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:25.592 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:25.624 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:25.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:25.625 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:25.726 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:25.769 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:25.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:25.770 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:25.870 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:25.900 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:25.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:25.901 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:25.962 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:25 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:10:25.962 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:25 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:10:25.962 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:25 smithi060 ceph-mon[29694]: osdmap e250: 8 total, 8 up, 8 in 2024-09-20T22:10:25.962 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:25 smithi060 ceph-mon[29694]: 4.e scrub starts 2024-09-20T22:10:25.962 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:25 smithi060 ceph-mon[29694]: 4.e scrub ok 2024-09-20T22:10:25.962 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:25 smithi060 ceph-mon[29694]: 8.f scrub starts 2024-09-20T22:10:25.963 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:25 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:10:25.963 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:25 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:10:25.963 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:25 smithi060 ceph-mon[40290]: osdmap e250: 8 total, 8 up, 8 in 2024-09-20T22:10:25.963 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:25 smithi060 ceph-mon[40290]: 4.e scrub starts 2024-09-20T22:10:25.963 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:25 smithi060 ceph-mon[40290]: 4.e scrub ok 2024-09-20T22:10:25.963 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:25 smithi060 ceph-mon[40290]: 8.f scrub starts 2024-09-20T22:10:26.002 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:26.009 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-20T22:10:26.010 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight-by-utilization 110 1.0 2024-09-20T22:10:26.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:25 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:10:26.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:25 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:10:26.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:25 smithi067 ceph-mon[33511]: osdmap e250: 8 total, 8 up, 8 in 2024-09-20T22:10:26.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:25 smithi067 ceph-mon[33511]: 4.e scrub starts 2024-09-20T22:10:26.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:25 smithi067 ceph-mon[33511]: 4.e scrub ok 2024-09-20T22:10:26.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:25 smithi067 ceph-mon[33511]: 8.f scrub starts 2024-09-20T22:10:26.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:26.035 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:26.035 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:26.135 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:26.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:26.182 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:26.282 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:26.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:26.315 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:26.316 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:26.416 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:26.450 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:26.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:26.452 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:26.552 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:26.590 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:26.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:26.591 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:26.692 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:26.732 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:26.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:26.733 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:26.834 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:26.860 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:26.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:26.861 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:26.962 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:26.997 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:26.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:26.998 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:27.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:26 smithi067 ceph-mon[33511]: pgmap v393: 168 pgs: 1 peering, 2 active+remapped+backfill_toofull, 165 active+clean; 3.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 7450/179022 objects misplaced (4.161%) 2024-09-20T22:10:27.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:26 smithi067 ceph-mon[33511]: 8.f scrub ok 2024-09-20T22:10:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:26 smithi060 ceph-mon[29694]: pgmap v393: 168 pgs: 1 peering, 2 active+remapped+backfill_toofull, 165 active+clean; 3.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 7450/179022 objects misplaced (4.161%) 2024-09-20T22:10:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:26 smithi060 ceph-mon[29694]: 8.f scrub ok 2024-09-20T22:10:27.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:26 smithi060 ceph-mon[40290]: pgmap v393: 168 pgs: 1 peering, 2 active+remapped+backfill_toofull, 165 active+clean; 3.6 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 7450/179022 objects misplaced (4.161%) 2024-09-20T22:10:27.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:26 smithi060 ceph-mon[40290]: 8.f scrub ok 2024-09-20T22:10:27.099 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:27.125 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:27.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:27.126 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:27.152 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T22:10:27.228 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:27.256 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:27.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:27.257 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:27.322 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:27.329 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:27.357 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:27.383 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:27.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:27.384 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:27.485 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:27.540 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:27.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:27.541 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:27.616 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:10:27.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:27.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:27.680 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:27.680 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:27.780 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:27 smithi060 ceph-mon[29694]: 5.b deep-scrub starts 2024-09-20T22:10:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:27 smithi060 ceph-mon[29694]: 5.b deep-scrub ok 2024-09-20T22:10:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:27 smithi060 ceph-mon[29694]: 5.18 scrub starts 2024-09-20T22:10:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:27 smithi060 ceph-mon[29694]: 5.18 scrub ok 2024-09-20T22:10:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:27 smithi060 ceph-mon[29694]: 4.1d deep-scrub starts 2024-09-20T22:10:27.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:27 smithi060 ceph-mon[29694]: 4.1d deep-scrub ok 2024-09-20T22:10:27.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:27 smithi060 ceph-mon[29694]: 3.a scrub starts 2024-09-20T22:10:27.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:27 smithi060 ceph-mon[29694]: 3.a scrub ok 2024-09-20T22:10:27.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:27 smithi060 ceph-mon[40290]: 5.b deep-scrub starts 2024-09-20T22:10:27.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:27 smithi060 ceph-mon[40290]: 5.b deep-scrub ok 2024-09-20T22:10:27.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:27 smithi060 ceph-mon[40290]: 5.18 scrub starts 2024-09-20T22:10:27.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:27 smithi060 ceph-mon[40290]: 5.18 scrub ok 2024-09-20T22:10:27.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:27 smithi060 ceph-mon[40290]: 4.1d deep-scrub starts 2024-09-20T22:10:27.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:27 smithi060 ceph-mon[40290]: 4.1d deep-scrub ok 2024-09-20T22:10:27.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:27 smithi060 ceph-mon[40290]: 3.a scrub starts 2024-09-20T22:10:27.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:27 smithi060 ceph-mon[40290]: 3.a scrub ok 2024-09-20T22:10:27.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:27.835 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:27.835 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:27.936 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:27.975 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:27.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:27.976 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:28.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:27 smithi067 ceph-mon[33511]: 5.b deep-scrub starts 2024-09-20T22:10:28.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:27 smithi067 ceph-mon[33511]: 5.b deep-scrub ok 2024-09-20T22:10:28.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:27 smithi067 ceph-mon[33511]: 5.18 scrub starts 2024-09-20T22:10:28.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:27 smithi067 ceph-mon[33511]: 5.18 scrub ok 2024-09-20T22:10:28.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:27 smithi067 ceph-mon[33511]: 4.1d deep-scrub starts 2024-09-20T22:10:28.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:27 smithi067 ceph-mon[33511]: 4.1d deep-scrub ok 2024-09-20T22:10:28.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:27 smithi067 ceph-mon[33511]: 3.a scrub starts 2024-09-20T22:10:28.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:27 smithi067 ceph-mon[33511]: 3.a scrub ok 2024-09-20T22:10:28.077 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:28.120 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:28.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:28.121 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:28.222 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:28.271 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:28.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:28.272 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:28.373 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:28.409 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:28.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:28.410 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:28.452 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:10:28.453 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:10:28.453 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:10:28.453 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:10:28.453 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:10:28.453 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:10:28.453 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:10:28.453 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:10:28.454 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:10:28.454 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:10:28.454 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:10:28.454 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:10:28.471 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:10:28.471 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:10:28.510 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:10:28.510 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:10:28.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:28.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:28.570 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:28.570 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:28.670 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:28.705 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:28.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:28.706 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:28 smithi060 ceph-mon[29694]: pgmap v394: 168 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 165 active+clean; 3.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 0 B/s wr, 460 op/s; 6865/168981 objects misplaced (4.063%); 0 B/s, 1 objects/s recovering 2024-09-20T22:10:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:28 smithi060 ceph-mon[29694]: 3.15 scrub starts 2024-09-20T22:10:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:28 smithi060 ceph-mon[29694]: 3.15 scrub ok 2024-09-20T22:10:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:28 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2172941066' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:28.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:28 smithi060 ceph-mon[40290]: pgmap v394: 168 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 165 active+clean; 3.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 0 B/s wr, 460 op/s; 6865/168981 objects misplaced (4.063%); 0 B/s, 1 objects/s recovering 2024-09-20T22:10:28.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:28 smithi060 ceph-mon[40290]: 3.15 scrub starts 2024-09-20T22:10:28.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:28 smithi060 ceph-mon[40290]: 3.15 scrub ok 2024-09-20T22:10:28.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:28 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2172941066' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:28.807 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:28.818 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:28.859 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:28.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:28.862 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:28.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:29.007 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:29.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:29.008 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:29.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:28 smithi067 ceph-mon[33511]: pgmap v394: 168 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 165 active+clean; 3.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 0 B/s wr, 460 op/s; 6865/168981 objects misplaced (4.063%); 0 B/s, 1 objects/s recovering 2024-09-20T22:10:29.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:28 smithi067 ceph-mon[33511]: 3.15 scrub starts 2024-09-20T22:10:29.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:28 smithi067 ceph-mon[33511]: 3.15 scrub ok 2024-09-20T22:10:29.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:28 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2172941066' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:29.109 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:29.145 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:29.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:29.146 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:29.246 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:29.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:29.279 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:29.279 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:29.380 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:29.411 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:29.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:29.412 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:29.513 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:29.548 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:29.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:29.549 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:29.629 INFO:teuthology.orchestra.run.smithi060.stdout:moved 31 / 504 (6.15079%) 2024-09-20T22:10:29.629 INFO:teuthology.orchestra.run.smithi060.stdout:avg 63 2024-09-20T22:10:29.629 INFO:teuthology.orchestra.run.smithi060.stdout:stddev 7.26292 -> 9.08295 (expected baseline 7.42462) 2024-09-20T22:10:29.629 INFO:teuthology.orchestra.run.smithi060.stdout:min osd.7 with 47 -> 56 pgs (0.746032 -> 0.888889 * mean) 2024-09-20T22:10:29.630 INFO:teuthology.orchestra.run.smithi060.stdout:max osd.4 with 71 -> 73 pgs (1.12698 -> 1.15873 * mean) 2024-09-20T22:10:29.630 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:10:29.630 INFO:teuthology.orchestra.run.smithi060.stdout:oload 110 2024-09-20T22:10:29.630 INFO:teuthology.orchestra.run.smithi060.stdout:max_change 1 2024-09-20T22:10:29.630 INFO:teuthology.orchestra.run.smithi060.stdout:max_change_osds 4 2024-09-20T22:10:29.630 INFO:teuthology.orchestra.run.smithi060.stdout:average_utilization 0.0192 2024-09-20T22:10:29.630 INFO:teuthology.orchestra.run.smithi060.stdout:overload_utilization 0.0211 2024-09-20T22:10:29.630 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 weight 1.0000 -> 0.6893 2024-09-20T22:10:29.630 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 weight 1.0000 -> 0.8900 2024-09-20T22:10:29.630 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 weight 0.7327 -> 0.8012 2024-09-20T22:10:29.645 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[29694]: 3.d scrub starts 2024-09-20T22:10:29.645 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[40290]: 3.d scrub starts 2024-09-20T22:10:29.650 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:29.687 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:29.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:29.688 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:29.789 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:29.811 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:29.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:29.812 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:29.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:29.935 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:29.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:29.936 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:30.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:29 smithi067 ceph-mon[33511]: 3.d scrub starts 2024-09-20T22:10:30.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:29 smithi067 ceph-mon[33511]: 3.d scrub ok 2024-09-20T22:10:30.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:29 smithi067 ceph-mon[33511]: 8.d scrub starts 2024-09-20T22:10:30.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:29 smithi067 ceph-mon[33511]: from='client.25277 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:10:30.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:29 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'2':'58325','5':'45172','7':'52507'}"}]: dispatch 2024-09-20T22:10:30.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:29 smithi067 ceph-mon[33511]: 3.12 scrub starts 2024-09-20T22:10:30.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:29 smithi067 ceph-mon[33511]: 3.12 scrub ok 2024-09-20T22:10:30.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:29 smithi067 ceph-mon[33511]: 3.11 scrub starts 2024-09-20T22:10:30.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:29 smithi067 ceph-mon[33511]: 3.11 scrub ok 2024-09-20T22:10:30.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:29 smithi067 ceph-mon[33511]: 8.b scrub starts 2024-09-20T22:10:30.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:29 smithi067 ceph-mon[33511]: 5.1f scrub starts 2024-09-20T22:10:30.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:29 smithi067 ceph-mon[33511]: 5.1f scrub ok 2024-09-20T22:10:30.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[29694]: 3.d scrub ok 2024-09-20T22:10:30.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[29694]: 8.d scrub starts 2024-09-20T22:10:30.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[29694]: from='client.25277 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:10:30.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'2':'58325','5':'45172','7':'52507'}"}]: dispatch 2024-09-20T22:10:30.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[29694]: 3.12 scrub starts 2024-09-20T22:10:30.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[29694]: 3.12 scrub ok 2024-09-20T22:10:30.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[29694]: 3.11 scrub starts 2024-09-20T22:10:30.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[29694]: 3.11 scrub ok 2024-09-20T22:10:30.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[29694]: 8.b scrub starts 2024-09-20T22:10:30.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[29694]: 5.1f scrub starts 2024-09-20T22:10:30.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[29694]: 5.1f scrub ok 2024-09-20T22:10:30.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[40290]: 3.d scrub ok 2024-09-20T22:10:30.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[40290]: 8.d scrub starts 2024-09-20T22:10:30.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[40290]: from='client.25277 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:10:30.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'2':'58325','5':'45172','7':'52507'}"}]: dispatch 2024-09-20T22:10:30.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[40290]: 3.12 scrub starts 2024-09-20T22:10:30.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[40290]: 3.12 scrub ok 2024-09-20T22:10:30.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[40290]: 3.11 scrub starts 2024-09-20T22:10:30.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[40290]: 3.11 scrub ok 2024-09-20T22:10:30.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[40290]: 8.b scrub starts 2024-09-20T22:10:30.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[40290]: 5.1f scrub starts 2024-09-20T22:10:30.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:29 smithi060 ceph-mon[40290]: 5.1f scrub ok 2024-09-20T22:10:30.038 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:30.060 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:30.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:30.061 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:30.161 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:30.183 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:30.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:30.184 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:30.285 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:30.307 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:30.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:30.308 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:30.409 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:30.434 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:30.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:30.435 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:30.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:30.563 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:30.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:30.564 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:30.664 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:30.696 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:30.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:30.697 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:30.798 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:30.820 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:30.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:30.821 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:30.922 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:30.948 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:30.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:30.949 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:31.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:30 smithi067 ceph-mon[33511]: 8.d scrub ok 2024-09-20T22:10:31.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:30 smithi067 ceph-mon[33511]: 4.d scrub starts 2024-09-20T22:10:31.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:30 smithi067 ceph-mon[33511]: 4.d scrub ok 2024-09-20T22:10:31.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:30 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'2':'58325','5':'45172','7':'52507'}"}]': finished 2024-09-20T22:10:31.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:30 smithi067 ceph-mon[33511]: osdmap e251: 8 total, 8 up, 8 in 2024-09-20T22:10:31.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:30 smithi067 ceph-mon[33511]: 8.b scrub ok 2024-09-20T22:10:31.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:30 smithi067 ceph-mon[33511]: pgmap v396: 168 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 165 active+clean; 3.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 678 B/s rd, 0 B/s wr, 968 op/s; 6865/168981 objects misplaced (4.063%); 14 B/s, 1 objects/s recovering 2024-09-20T22:10:31.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:30 smithi067 ceph-mon[33511]: 3.18 scrub starts 2024-09-20T22:10:31.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:30 smithi067 ceph-mon[33511]: 3.18 scrub ok 2024-09-20T22:10:31.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:30 smithi067 ceph-mon[33511]: osdmap e252: 8 total, 8 up, 8 in 2024-09-20T22:10:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[29694]: 8.d scrub ok 2024-09-20T22:10:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[29694]: 4.d scrub starts 2024-09-20T22:10:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[29694]: 4.d scrub ok 2024-09-20T22:10:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'2':'58325','5':'45172','7':'52507'}"}]': finished 2024-09-20T22:10:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[29694]: osdmap e251: 8 total, 8 up, 8 in 2024-09-20T22:10:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[29694]: 8.b scrub ok 2024-09-20T22:10:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[29694]: pgmap v396: 168 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 165 active+clean; 3.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 678 B/s rd, 0 B/s wr, 968 op/s; 6865/168981 objects misplaced (4.063%); 14 B/s, 1 objects/s recovering 2024-09-20T22:10:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[29694]: 3.18 scrub starts 2024-09-20T22:10:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[29694]: 3.18 scrub ok 2024-09-20T22:10:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[29694]: osdmap e252: 8 total, 8 up, 8 in 2024-09-20T22:10:31.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[40290]: 8.d scrub ok 2024-09-20T22:10:31.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[40290]: 4.d scrub starts 2024-09-20T22:10:31.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[40290]: 4.d scrub ok 2024-09-20T22:10:31.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'2':'58325','5':'45172','7':'52507'}"}]': finished 2024-09-20T22:10:31.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[40290]: osdmap e251: 8 total, 8 up, 8 in 2024-09-20T22:10:31.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[40290]: 8.b scrub ok 2024-09-20T22:10:31.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[40290]: pgmap v396: 168 pgs: 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 165 active+clean; 3.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 678 B/s rd, 0 B/s wr, 968 op/s; 6865/168981 objects misplaced (4.063%); 14 B/s, 1 objects/s recovering 2024-09-20T22:10:31.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[40290]: 3.18 scrub starts 2024-09-20T22:10:31.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[40290]: 3.18 scrub ok 2024-09-20T22:10:31.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:30 smithi060 ceph-mon[40290]: osdmap e252: 8 total, 8 up, 8 in 2024-09-20T22:10:31.051 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:31.073 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:31.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:31.074 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:31.175 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:31.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:31.198 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:31.198 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:31.299 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:31.326 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:31.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:31.327 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:31.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:31.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:31.458 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:31.458 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:31.558 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:31.594 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:31.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:31.595 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:31.695 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:31.718 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:31.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:31.719 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:31.820 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:31.845 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:31.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:31.846 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:31.947 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:31.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:31.979 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:31.980 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:31 smithi067 ceph-mon[33511]: 3.1c deep-scrub starts 2024-09-20T22:10:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:31 smithi067 ceph-mon[33511]: 3.1c deep-scrub ok 2024-09-20T22:10:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:31 smithi067 ceph-mon[33511]: 3.1 scrub starts 2024-09-20T22:10:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:31 smithi067 ceph-mon[33511]: 3.1 scrub ok 2024-09-20T22:10:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:31 smithi067 ceph-mon[33511]: 3.c scrub starts 2024-09-20T22:10:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:31 smithi067 ceph-mon[33511]: 3.c scrub ok 2024-09-20T22:10:32.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:31 smithi067 ceph-mon[33511]: osdmap e253: 8 total, 8 up, 8 in 2024-09-20T22:10:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:31 smithi060 ceph-mon[29694]: 3.1c deep-scrub starts 2024-09-20T22:10:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:31 smithi060 ceph-mon[29694]: 3.1c deep-scrub ok 2024-09-20T22:10:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:31 smithi060 ceph-mon[29694]: 3.1 scrub starts 2024-09-20T22:10:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:31 smithi060 ceph-mon[29694]: 3.1 scrub ok 2024-09-20T22:10:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:31 smithi060 ceph-mon[29694]: 3.c scrub starts 2024-09-20T22:10:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:31 smithi060 ceph-mon[29694]: 3.c scrub ok 2024-09-20T22:10:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:31 smithi060 ceph-mon[29694]: osdmap e253: 8 total, 8 up, 8 in 2024-09-20T22:10:32.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:31 smithi060 ceph-mon[40290]: 3.1c deep-scrub starts 2024-09-20T22:10:32.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:31 smithi060 ceph-mon[40290]: 3.1c deep-scrub ok 2024-09-20T22:10:32.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:31 smithi060 ceph-mon[40290]: 3.1 scrub starts 2024-09-20T22:10:32.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:31 smithi060 ceph-mon[40290]: 3.1 scrub ok 2024-09-20T22:10:32.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:31 smithi060 ceph-mon[40290]: 3.c scrub starts 2024-09-20T22:10:32.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:31 smithi060 ceph-mon[40290]: 3.c scrub ok 2024-09-20T22:10:32.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:31 smithi060 ceph-mon[40290]: osdmap e253: 8 total, 8 up, 8 in 2024-09-20T22:10:32.080 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:32.118 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:32.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:32.119 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:32.220 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:32.265 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:32.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:32.266 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:32.367 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:32.402 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:32.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:32.403 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:32.504 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:32.513 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:10:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:10:32.542 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:32.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:32.544 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:32.644 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:32.675 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:32.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:32.676 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:32.777 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:10:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:10:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[29694]: 5.4 scrub starts 2024-09-20T22:10:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[29694]: 5.4 scrub ok 2024-09-20T22:10:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[29694]: 4.18 deep-scrub starts 2024-09-20T22:10:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[29694]: 4.18 deep-scrub ok 2024-09-20T22:10:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[29694]: 8.9 starting backfill to osd.3 from (0'0,0'0] MAX to 250'9112 2024-09-20T22:10:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[29694]: 8.a starting backfill to osd.3 from (0'0,0'0] MAX to 250'11825 2024-09-20T22:10:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[29694]: 8.0 starting backfill to osd.4 from (0'0,0'0] MAX to 250'9116 2024-09-20T22:10:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[29694]: 8.c starting backfill to osd.4 from (0'0,0'0] MAX to 250'11725 2024-09-20T22:10:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[29694]: 3.1f scrub starts 2024-09-20T22:10:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[29694]: 3.1f scrub ok 2024-09-20T22:10:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[29694]: pgmap v399: 168 pgs: 1 activating+degraded+remapped, 1 active+undersized+remapped, 2 activating+degraded, 4 activating, 5 remapped+peering, 2 active+remapped+backfill_toofull, 153 active+clean; 3.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 5.0 KiB/s rd, 36 KiB/s wr, 1.17k op/s; 10/157980 objects degraded (0.006%); 9645/157980 objects misplaced (6.105%); 26 KiB/s, 3 objects/s recovering 2024-09-20T22:10:32.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:10:32.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[29694]: osdmap e254: 8 total, 8 up, 8 in 2024-09-20T22:10:32.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:10:32.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:10:32.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[40290]: 5.4 scrub starts 2024-09-20T22:10:32.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[40290]: 5.4 scrub ok 2024-09-20T22:10:32.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[40290]: 4.18 deep-scrub starts 2024-09-20T22:10:32.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[40290]: 4.18 deep-scrub ok 2024-09-20T22:10:32.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[40290]: 8.9 starting backfill to osd.3 from (0'0,0'0] MAX to 250'9112 2024-09-20T22:10:32.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[40290]: 8.a starting backfill to osd.3 from (0'0,0'0] MAX to 250'11825 2024-09-20T22:10:32.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[40290]: 8.0 starting backfill to osd.4 from (0'0,0'0] MAX to 250'9116 2024-09-20T22:10:32.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[40290]: 8.c starting backfill to osd.4 from (0'0,0'0] MAX to 250'11725 2024-09-20T22:10:32.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[40290]: 3.1f scrub starts 2024-09-20T22:10:32.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[40290]: 3.1f scrub ok 2024-09-20T22:10:32.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[40290]: pgmap v399: 168 pgs: 1 activating+degraded+remapped, 1 active+undersized+remapped, 2 activating+degraded, 4 activating, 5 remapped+peering, 2 active+remapped+backfill_toofull, 153 active+clean; 3.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 5.0 KiB/s rd, 36 KiB/s wr, 1.17k op/s; 10/157980 objects degraded (0.006%); 9645/157980 objects misplaced (6.105%); 26 KiB/s, 3 objects/s recovering 2024-09-20T22:10:32.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:10:32.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:32 smithi060 ceph-mon[40290]: osdmap e254: 8 total, 8 up, 8 in 2024-09-20T22:10:32.798 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:32.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:32.799 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:32.900 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:32.938 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:32.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:32.939 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:33.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:32 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:10:33.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:32 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:10:33.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:32 smithi067 ceph-mon[33511]: 5.4 scrub starts 2024-09-20T22:10:33.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:32 smithi067 ceph-mon[33511]: 5.4 scrub ok 2024-09-20T22:10:33.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:32 smithi067 ceph-mon[33511]: 4.18 deep-scrub starts 2024-09-20T22:10:33.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:32 smithi067 ceph-mon[33511]: 4.18 deep-scrub ok 2024-09-20T22:10:33.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:32 smithi067 ceph-mon[33511]: 8.9 starting backfill to osd.3 from (0'0,0'0] MAX to 250'9112 2024-09-20T22:10:33.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:32 smithi067 ceph-mon[33511]: 8.a starting backfill to osd.3 from (0'0,0'0] MAX to 250'11825 2024-09-20T22:10:33.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:32 smithi067 ceph-mon[33511]: 8.0 starting backfill to osd.4 from (0'0,0'0] MAX to 250'9116 2024-09-20T22:10:33.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:32 smithi067 ceph-mon[33511]: 8.c starting backfill to osd.4 from (0'0,0'0] MAX to 250'11725 2024-09-20T22:10:33.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:32 smithi067 ceph-mon[33511]: 3.1f scrub starts 2024-09-20T22:10:33.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:32 smithi067 ceph-mon[33511]: 3.1f scrub ok 2024-09-20T22:10:33.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:32 smithi067 ceph-mon[33511]: pgmap v399: 168 pgs: 1 activating+degraded+remapped, 1 active+undersized+remapped, 2 activating+degraded, 4 activating, 5 remapped+peering, 2 active+remapped+backfill_toofull, 153 active+clean; 3.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 5.0 KiB/s rd, 36 KiB/s wr, 1.17k op/s; 10/157980 objects degraded (0.006%); 9645/157980 objects misplaced (6.105%); 26 KiB/s, 3 objects/s recovering 2024-09-20T22:10:33.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:32 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:10:33.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:32 smithi067 ceph-mon[33511]: osdmap e254: 8 total, 8 up, 8 in 2024-09-20T22:10:33.040 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:33.072 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:33.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:33.073 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:33.174 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:33.213 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:33.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:33.215 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:33.315 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:33.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:33.355 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:33.355 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:33.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:33.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:33.496 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:33.496 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:33.597 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:33.640 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:33.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:33.641 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:33.742 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:33.771 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:33.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:33.772 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:33.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:33 smithi060 ceph-mon[29694]: Health check failed: Reduced data availability: 3 pgs inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:10:33.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:33 smithi060 ceph-mon[29694]: Health check failed: Degraded data redundancy: 10/157980 objects degraded (0.006%), 3 pgs degraded (PG_DEGRADED) 2024-09-20T22:10:33.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:33 smithi060 ceph-mon[29694]: 6.18 scrub starts 2024-09-20T22:10:33.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:33 smithi060 ceph-mon[29694]: 6.18 scrub ok 2024-09-20T22:10:33.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:33 smithi060 ceph-mon[29694]: 3.5 scrub starts 2024-09-20T22:10:33.783 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:33 smithi060 ceph-mon[29694]: 3.5 scrub ok 2024-09-20T22:10:33.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:33 smithi060 ceph-mon[29694]: 4.1f scrub starts 2024-09-20T22:10:33.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:33 smithi060 ceph-mon[29694]: 4.1f scrub ok 2024-09-20T22:10:33.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:33 smithi060 ceph-mon[29694]: osdmap e255: 8 total, 8 up, 8 in 2024-09-20T22:10:33.784 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:33 smithi060 ceph-mon[40290]: Health check failed: Reduced data availability: 3 pgs inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:10:33.784 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:33 smithi060 ceph-mon[40290]: Health check failed: Degraded data redundancy: 10/157980 objects degraded (0.006%), 3 pgs degraded (PG_DEGRADED) 2024-09-20T22:10:33.784 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:33 smithi060 ceph-mon[40290]: 6.18 scrub starts 2024-09-20T22:10:33.784 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:33 smithi060 ceph-mon[40290]: 6.18 scrub ok 2024-09-20T22:10:33.784 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:33 smithi060 ceph-mon[40290]: 3.5 scrub starts 2024-09-20T22:10:33.784 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:33 smithi060 ceph-mon[40290]: 3.5 scrub ok 2024-09-20T22:10:33.784 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:33 smithi060 ceph-mon[40290]: 4.1f scrub starts 2024-09-20T22:10:33.784 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:33 smithi060 ceph-mon[40290]: 4.1f scrub ok 2024-09-20T22:10:33.784 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:33 smithi060 ceph-mon[40290]: osdmap e255: 8 total, 8 up, 8 in 2024-09-20T22:10:33.872 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:33.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:33.915 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:33.916 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:33.999 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:10:33.999 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:10:33.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:10:33.999 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:10:33.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:33.999 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47598 7.3 7:c6c29109:::benchmark_data_smithi060_78800_object47597:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:10:33.999 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.238604+0000", 2024-09-20T22:10:34.000 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 373.75590045799999, 2024-09-20T22:10:34.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0797809089999999, 2024-09-20T22:10:34.000 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.000 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.000 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:34.000 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.000 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.000 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.000 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47598 2024-09-20T22:10:34.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.000 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238604+0000", 2024-09-20T22:10:34.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238604+0000", 2024-09-20T22:10:34.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238605+0000", 2024-09-20T22:10:34.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3099999999999996e-07 2024-09-20T22:10:34.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238626+0000", 2024-09-20T22:10:34.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1192999999999999e-05 2024-09-20T22:10:34.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238637+0000", 2024-09-20T22:10:34.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0516000000000001e-05 2024-09-20T22:10:34.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238644+0000", 2024-09-20T22:10:34.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3969999999999997e-06 2024-09-20T22:10:34.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.240757+0000", 2024-09-20T22:10:34.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021131100000000001 2024-09-20T22:10:34.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op must wait for map", 2024-09-20T22:10:34.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.240764+0000", 2024-09-20T22:10:34.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9550000000000003e-06 2024-09-20T22:10:34.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.246730+0000", 2024-09-20T22:10:34.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0059659020000000004 2024-09-20T22:10:34.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.246741+0000", 2024-09-20T22:10:34.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0855e-05 2024-09-20T22:10:34.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.235644+0000", 2024-09-20T22:10:34.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98890363400000003 2024-09-20T22:10:34.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.235657+0000", 2024-09-20T22:10:34.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3040000000000001e-05 2024-09-20T22:10:34.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.313762+0000", 2024-09-20T22:10:34.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.078104783999999997 2024-09-20T22:10:34.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:34.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.314529+0000", 2024-09-20T22:10:34.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00076653400000000003 2024-09-20T22:10:34.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:10:34.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.314632+0000", 2024-09-20T22:10:34.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103224 2024-09-20T22:10:34.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:34.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318215+0000", 2024-09-20T22:10:34.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0035831439999999999 2024-09-20T22:10:34.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318247+0000", 2024-09-20T22:10:34.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1804999999999998e-05 2024-09-20T22:10:34.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318292+0000", 2024-09-20T22:10:34.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4605999999999999e-05 2024-09-20T22:10:34.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:34.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318329+0000", 2024-09-20T22:10:34.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7175999999999998e-05 2024-09-20T22:10:34.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318385+0000", 2024-09-20T22:10:34.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6202999999999997e-05 2024-09-20T22:10:34.007 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.007 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.007 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.008 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47618 7.3 7:c8aeb189:::benchmark_data_smithi060_78800_object47617:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:10:34.008 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.257444+0000", 2024-09-20T22:10:34.008 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 373.73706047500002, 2024-09-20T22:10:34.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0612655090000001, 2024-09-20T22:10:34.008 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.008 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.008 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:34.008 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.008 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.008 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.008 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47618 2024-09-20T22:10:34.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.008 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257444+0000", 2024-09-20T22:10:34.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257444+0000", 2024-09-20T22:10:34.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257445+0000", 2024-09-20T22:10:34.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3790000000000001e-06 2024-09-20T22:10:34.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257468+0000", 2024-09-20T22:10:34.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2078000000000001e-05 2024-09-20T22:10:34.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257480+0000", 2024-09-20T22:10:34.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.253e-05 2024-09-20T22:10:34.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257488+0000", 2024-09-20T22:10:34.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4900000000000003e-06 2024-09-20T22:10:34.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257693+0000", 2024-09-20T22:10:34.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000205071 2024-09-20T22:10:34.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.011 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257702+0000", 2024-09-20T22:10:34.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6099999999999995e-06 2024-09-20T22:10:34.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.011 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236024+0000", 2024-09-20T22:10:34.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97832186300000001 2024-09-20T22:10:34.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236034+0000", 2024-09-20T22:10:34.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9189999999999994e-06 2024-09-20T22:10:34.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.316117+0000", 2024-09-20T22:10:34.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.080083484999999996 2024-09-20T22:10:34.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:34.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.316218+0000", 2024-09-20T22:10:34.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100846 2024-09-20T22:10:34.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:10:34.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.316363+0000", 2024-09-20T22:10:34.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014473099999999999 2024-09-20T22:10:34.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:34.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318472+0000", 2024-09-20T22:10:34.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021087039999999999 2024-09-20T22:10:34.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318532+0000", 2024-09-20T22:10:34.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0418999999999999e-05 2024-09-20T22:10:34.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318622+0000", 2024-09-20T22:10:34.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0074000000000005e-05 2024-09-20T22:10:34.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:34.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318659+0000", 2024-09-20T22:10:34.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6260999999999997e-05 2024-09-20T22:10:34.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318710+0000", 2024-09-20T22:10:34.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1048999999999999e-05 2024-09-20T22:10:34.016 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.016 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.016 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.016 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47683 7.3 7:cf1d2c6c:::benchmark_data_smithi060_78800_object47682:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:10:34.016 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.290834+0000", 2024-09-20T22:10:34.016 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 373.70367042200002, 2024-09-20T22:10:34.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.029069585, 2024-09-20T22:10:34.016 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.016 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.016 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:34.016 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.016 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.016 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.016 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47683 2024-09-20T22:10:34.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.017 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290834+0000", 2024-09-20T22:10:34.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290834+0000", 2024-09-20T22:10:34.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290835+0000", 2024-09-20T22:10:34.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2020000000000001e-06 2024-09-20T22:10:34.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290856+0000", 2024-09-20T22:10:34.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0585999999999999e-05 2024-09-20T22:10:34.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290868+0000", 2024-09-20T22:10:34.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2160999999999999e-05 2024-09-20T22:10:34.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290875+0000", 2024-09-20T22:10:34.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3200000000000002e-06 2024-09-20T22:10:34.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290955+0000", 2024-09-20T22:10:34.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9592999999999995e-05 2024-09-20T22:10:34.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290966+0000", 2024-09-20T22:10:34.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.119e-05 2024-09-20T22:10:34.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236104+0000", 2024-09-20T22:10:34.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94513756400000004 2024-09-20T22:10:34.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236112+0000", 2024-09-20T22:10:34.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2220000000000002e-06 2024-09-20T22:10:34.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.317361+0000", 2024-09-20T22:10:34.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.081249110999999999 2024-09-20T22:10:34.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:34.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.317444+0000", 2024-09-20T22:10:34.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2936999999999996e-05 2024-09-20T22:10:34.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:10:34.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.317551+0000", 2024-09-20T22:10:34.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010741100000000001 2024-09-20T22:10:34.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.021 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:34.021 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318732+0000", 2024-09-20T22:10:34.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0011802379999999999 2024-09-20T22:10:34.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319739+0000", 2024-09-20T22:10:34.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001007457 2024-09-20T22:10:34.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319786+0000", 2024-09-20T22:10:34.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6845999999999998e-05 2024-09-20T22:10:34.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:34.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319820+0000", 2024-09-20T22:10:34.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4103000000000002e-05 2024-09-20T22:10:34.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319904+0000", 2024-09-20T22:10:34.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3644000000000001e-05 2024-09-20T22:10:34.023 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.023 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.023 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.023 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47686 7.3 7:c4eaa487:::benchmark_data_smithi060_78800_object47685:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:10:34.023 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.291328+0000", 2024-09-20T22:10:34.023 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 373.70317684999998, 2024-09-20T22:10:34.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.03577915, 2024-09-20T22:10:34.023 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.024 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.024 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:34.024 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.024 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.024 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.024 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47686 2024-09-20T22:10:34.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.024 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291328+0000", 2024-09-20T22:10:34.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291328+0000", 2024-09-20T22:10:34.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291328+0000", 2024-09-20T22:10:34.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0699999999999996e-07 2024-09-20T22:10:34.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291353+0000", 2024-09-20T22:10:34.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4134999999999998e-05 2024-09-20T22:10:34.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291364+0000", 2024-09-20T22:10:34.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1501999999999999e-05 2024-09-20T22:10:34.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291371+0000", 2024-09-20T22:10:34.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1210000000000001e-06 2024-09-20T22:10:34.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291460+0000", 2024-09-20T22:10:34.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8869999999999994e-05 2024-09-20T22:10:34.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291473+0000", 2024-09-20T22:10:34.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2899e-05 2024-09-20T22:10:34.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236140+0000", 2024-09-20T22:10:34.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94466707500000002 2024-09-20T22:10:34.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236149+0000", 2024-09-20T22:10:34.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9509999999999995e-06 2024-09-20T22:10:34.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318750+0000", 2024-09-20T22:10:34.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.082601349000000004 2024-09-20T22:10:34.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:34.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318813+0000", 2024-09-20T22:10:34.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2306999999999998e-05 2024-09-20T22:10:34.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:10:34.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318890+0000", 2024-09-20T22:10:34.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7565999999999996e-05 2024-09-20T22:10:34.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323067+0000", 2024-09-20T22:10:34.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0041765129999999998 2024-09-20T22:10:34.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323117+0000", 2024-09-20T22:10:34.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0297000000000003e-05 2024-09-20T22:10:34.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:34.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.326959+0000", 2024-09-20T22:10:34.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003841957 2024-09-20T22:10:34.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:34.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327009+0000", 2024-09-20T22:10:34.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0328999999999999e-05 2024-09-20T22:10:34.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.030 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.030 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327107+0000", 2024-09-20T22:10:34.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7472000000000005e-05 2024-09-20T22:10:34.030 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.030 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.030 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.031 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47726 7.3 7:c33b26c6:::benchmark_data_smithi060_78800_object47725:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:10:34.031 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.311743+0000", 2024-09-20T22:10:34.031 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 373.68276104, 2024-09-20T22:10:34.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0155469880000001, 2024-09-20T22:10:34.031 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.031 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.031 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:34.031 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.031 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.031 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.031 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47726 2024-09-20T22:10:34.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.031 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311743+0000", 2024-09-20T22:10:34.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311743+0000", 2024-09-20T22:10:34.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311745+0000", 2024-09-20T22:10:34.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.333e-06 2024-09-20T22:10:34.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311769+0000", 2024-09-20T22:10:34.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3688000000000001e-05 2024-09-20T22:10:34.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311781+0000", 2024-09-20T22:10:34.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2099999999999999e-05 2024-09-20T22:10:34.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311788+0000", 2024-09-20T22:10:34.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2269999999999996e-06 2024-09-20T22:10:34.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311860+0000", 2024-09-20T22:10:34.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1823000000000007e-05 2024-09-20T22:10:34.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311870+0000", 2024-09-20T22:10:34.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0079e-05 2024-09-20T22:10:34.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.035 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.035 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236177+0000", 2024-09-20T22:10:34.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.92430713399999997 2024-09-20T22:10:34.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.035 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.035 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236185+0000", 2024-09-20T22:10:34.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.106e-06 2024-09-20T22:10:34.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.035 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.035 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319949+0000", 2024-09-20T22:10:34.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083764076000000007 2024-09-20T22:10:34.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:34.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.320088+0000", 2024-09-20T22:10:34.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013926300000000001 2024-09-20T22:10:34.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:10:34.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.320263+0000", 2024-09-20T22:10:34.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000174978 2024-09-20T22:10:34.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323166+0000", 2024-09-20T22:10:34.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0029027060000000001 2024-09-20T22:10:34.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323208+0000", 2024-09-20T22:10:34.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2373999999999998e-05 2024-09-20T22:10:34.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:34.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327156+0000", 2024-09-20T22:10:34.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0039471910000000001 2024-09-20T22:10:34.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:34.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327201+0000", 2024-09-20T22:10:34.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5062999999999999e-05 2024-09-20T22:10:34.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327290+0000", 2024-09-20T22:10:34.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9846999999999995e-05 2024-09-20T22:10:34.038 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.038 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.038 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.038 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47771 7.3 7:c7433b13:::benchmark_data_smithi060_78800_object47770:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:10:34.038 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.332925+0000", 2024-09-20T22:10:34.038 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 373.66157999400002, 2024-09-20T22:10:34.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99455883199999995, 2024-09-20T22:10:34.039 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.039 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.039 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:34.039 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.039 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.039 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.039 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:34.040 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:33 smithi067 ceph-mon[33511]: Health check failed: Reduced data availability: 3 pgs inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:10:34.040 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:33 smithi067 ceph-mon[33511]: Health check failed: Degraded data redundancy: 10/157980 objects degraded (0.006%), 3 pgs degraded (PG_DEGRADED) 2024-09-20T22:10:34.040 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:33 smithi067 ceph-mon[33511]: 6.18 scrub starts 2024-09-20T22:10:34.040 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:33 smithi067 ceph-mon[33511]: 6.18 scrub ok 2024-09-20T22:10:34.041 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:33 smithi067 ceph-mon[33511]: 3.5 scrub starts 2024-09-20T22:10:34.041 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:33 smithi067 ceph-mon[33511]: 3.5 scrub ok 2024-09-20T22:10:34.041 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:33 smithi067 ceph-mon[33511]: 4.1f scrub starts 2024-09-20T22:10:34.041 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:33 smithi067 ceph-mon[33511]: 4.1f scrub ok 2024-09-20T22:10:34.041 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:33 smithi067 ceph-mon[33511]: osdmap e255: 8 total, 8 up, 8 in 2024-09-20T22:10:34.041 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47771 2024-09-20T22:10:34.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.042 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332925+0000", 2024-09-20T22:10:34.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332925+0000", 2024-09-20T22:10:34.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332926+0000", 2024-09-20T22:10:34.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4959999999999999e-06 2024-09-20T22:10:34.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332954+0000", 2024-09-20T22:10:34.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7495e-05 2024-09-20T22:10:34.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.043 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.043 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332967+0000", 2024-09-20T22:10:34.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3005e-05 2024-09-20T22:10:34.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332974+0000", 2024-09-20T22:10:34.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9719999999999992e-06 2024-09-20T22:10:34.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333056+0000", 2024-09-20T22:10:34.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1390999999999994e-05 2024-09-20T22:10:34.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333067+0000", 2024-09-20T22:10:34.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0896999999999999e-05 2024-09-20T22:10:34.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236223+0000", 2024-09-20T22:10:34.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90315600299999999 2024-09-20T22:10:34.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236243+0000", 2024-09-20T22:10:34.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9928999999999998e-05 2024-09-20T22:10:34.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.321565+0000", 2024-09-20T22:10:34.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085322377000000005 2024-09-20T22:10:34.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:34.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.321707+0000", 2024-09-20T22:10:34.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014187100000000001 2024-09-20T22:10:34.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:10:34.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.321808+0000", 2024-09-20T22:10:34.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100536 2024-09-20T22:10:34.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.048 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323255+0000", 2024-09-20T22:10:34.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00144754 2024-09-20T22:10:34.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.326947+0000", 2024-09-20T22:10:34.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0036917360000000001 2024-09-20T22:10:34.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:34.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327341+0000", 2024-09-20T22:10:34.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00039394499999999998 2024-09-20T22:10:34.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:34.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327391+0000", 2024-09-20T22:10:34.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9836999999999997e-05 2024-09-20T22:10:34.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327483+0000", 2024-09-20T22:10:34.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2801999999999997e-05 2024-09-20T22:10:34.050 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.050 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.050 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.050 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47772 7.3 7:c5f9ddaf:::benchmark_data_smithi060_78800_object47771:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:10:34.050 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.333899+0000", 2024-09-20T22:10:34.050 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 373.66060547699999, 2024-09-20T22:10:34.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99376266499999999, 2024-09-20T22:10:34.050 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.050 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.050 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:34.050 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.050 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.051 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.051 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47772 2024-09-20T22:10:34.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.051 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333899+0000", 2024-09-20T22:10:34.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333899+0000", 2024-09-20T22:10:34.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333900+0000", 2024-09-20T22:10:34.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0049999999999999e-06 2024-09-20T22:10:34.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333927+0000", 2024-09-20T22:10:34.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.675e-05 2024-09-20T22:10:34.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333942+0000", 2024-09-20T22:10:34.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5678000000000001e-05 2024-09-20T22:10:34.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333950+0000", 2024-09-20T22:10:34.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7060000000000004e-06 2024-09-20T22:10:34.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.334006+0000", 2024-09-20T22:10:34.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5631000000000003e-05 2024-09-20T22:10:34.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.053 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.053 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.334031+0000", 2024-09-20T22:10:34.053 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5213999999999999e-05 2024-09-20T22:10:34.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236276+0000", 2024-09-20T22:10:34.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90224500100000005 2024-09-20T22:10:34.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236285+0000", 2024-09-20T22:10:34.054 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7139999999999999e-06 2024-09-20T22:10:34.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.054 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.054 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323289+0000", 2024-09-20T22:10:34.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087004215999999995 2024-09-20T22:10:34.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:34.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323404+0000", 2024-09-20T22:10:34.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011502000000000001 2024-09-20T22:10:34.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:10:34.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323545+0000", 2024-09-20T22:10:34.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014127700000000001 2024-09-20T22:10:34.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.055 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.055 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327330+0000", 2024-09-20T22:10:34.055 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037843400000000002 2024-09-20T22:10:34.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:34.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327491+0000", 2024-09-20T22:10:34.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016180099999999999 2024-09-20T22:10:34.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327529+0000", 2024-09-20T22:10:34.056 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8081999999999999e-05 2024-09-20T22:10:34.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.056 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:34.056 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327576+0000", 2024-09-20T22:10:34.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6881000000000001e-05 2024-09-20T22:10:34.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.057 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.057 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327662+0000", 2024-09-20T22:10:34.057 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5348999999999999e-05 2024-09-20T22:10:34.057 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.057 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.057 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.057 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47798 7.3 7:cc6426c7:::benchmark_data_smithi060_78800_object47797:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:10:34.057 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.346085+0000", 2024-09-20T22:10:34.058 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 373.64841908300002, 2024-09-20T22:10:34.058 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98393741899999998, 2024-09-20T22:10:34.058 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.058 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.058 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:34.058 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.058 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.058 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.058 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47798 2024-09-20T22:10:34.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.058 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.058 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346085+0000", 2024-09-20T22:10:34.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346085+0000", 2024-09-20T22:10:34.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.059 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.059 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346087+0000", 2024-09-20T22:10:34.059 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3289999999999999e-06 2024-09-20T22:10:34.059 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.059 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346109+0000", 2024-09-20T22:10:34.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2359999999999999e-05 2024-09-20T22:10:34.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346123+0000", 2024-09-20T22:10:34.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.367e-05 2024-09-20T22:10:34.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.060 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.060 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346133+0000", 2024-09-20T22:10:34.060 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0519999999999999e-05 2024-09-20T22:10:34.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346206+0000", 2024-09-20T22:10:34.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3011999999999998e-05 2024-09-20T22:10:34.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346218+0000", 2024-09-20T22:10:34.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2089000000000001e-05 2024-09-20T22:10:34.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.061 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.061 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236825+0000", 2024-09-20T22:10:34.061 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.89060705600000001 2024-09-20T22:10:34.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236839+0000", 2024-09-20T22:10:34.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3685e-05 2024-09-20T22:10:34.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.324961+0000", 2024-09-20T22:10:34.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088121685000000005 2024-09-20T22:10:34.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.062 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:34.062 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.325083+0000", 2024-09-20T22:10:34.062 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012199199999999999 2024-09-20T22:10:34.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:10:34.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.325216+0000", 2024-09-20T22:10:34.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000132862 2024-09-20T22:10:34.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:34.063 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327496+0000", 2024-09-20T22:10:34.063 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022805669999999998 2024-09-20T22:10:34.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.063 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327698+0000", 2024-09-20T22:10:34.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020205099999999999 2024-09-20T22:10:34.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.329834+0000", 2024-09-20T22:10:34.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021359249999999999 2024-09-20T22:10:34.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.064 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:34.064 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.329894+0000", 2024-09-20T22:10:34.064 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9929000000000002e-05 2024-09-20T22:10:34.064 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.065 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.065 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.330023+0000", 2024-09-20T22:10:34.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000128687 2024-09-20T22:10:34.065 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.065 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.065 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.065 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47809 7.3 7:caa74520:::benchmark_data_smithi060_78800_object47808:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:10:34.065 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.351924+0000", 2024-09-20T22:10:34.065 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 373.64258094500002, 2024-09-20T22:10:34.065 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.982167177, 2024-09-20T22:10:34.065 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.065 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.066 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:34.066 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.066 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.066 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.066 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47809 2024-09-20T22:10:34.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.066 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.066 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.066 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.351924+0000", 2024-09-20T22:10:34.066 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.351924+0000", 2024-09-20T22:10:34.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.351925+0000", 2024-09-20T22:10:34.067 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1680000000000001e-06 2024-09-20T22:10:34.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.067 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.067 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352100+0000", 2024-09-20T22:10:34.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017479600000000001 2024-09-20T22:10:34.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352112+0000", 2024-09-20T22:10:34.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.204e-05 2024-09-20T22:10:34.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.068 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352119+0000", 2024-09-20T22:10:34.068 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8960000000000003e-06 2024-09-20T22:10:34.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.068 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352199+0000", 2024-09-20T22:10:34.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9747999999999996e-05 2024-09-20T22:10:34.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352208+0000", 2024-09-20T22:10:34.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2709999999999998e-06 2024-09-20T22:10:34.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.069 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.069 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236880+0000", 2024-09-20T22:10:34.069 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.88467109200000005 2024-09-20T22:10:34.069 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236889+0000", 2024-09-20T22:10:34.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0580000000000004e-06 2024-09-20T22:10:34.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.070 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327738+0000", 2024-09-20T22:10:34.070 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.090849691999999996 2024-09-20T22:10:34.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.070 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:34.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327931+0000", 2024-09-20T22:10:34.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019231900000000001 2024-09-20T22:10:34.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:10:34.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.328060+0000", 2024-09-20T22:10:34.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012906699999999999 2024-09-20T22:10:34.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.071 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:34.071 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.331899+0000", 2024-09-20T22:10:34.071 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038394620000000001 2024-09-20T22:10:34.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.333872+0000", 2024-09-20T22:10:34.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019725540000000001 2024-09-20T22:10:34.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.333936+0000", 2024-09-20T22:10:34.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3849000000000005e-05 2024-09-20T22:10:34.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.072 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:34.072 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.333992+0000", 2024-09-20T22:10:34.072 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6730000000000001e-05 2024-09-20T22:10:34.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.073 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.073 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334091+0000", 2024-09-20T22:10:34.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8435000000000002e-05 2024-09-20T22:10:34.073 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.073 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.073 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.073 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47810 7.3 7:c6434cb9:::benchmark_data_smithi060_78800_object47809:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:10:34.073 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.352140+0000", 2024-09-20T22:10:34.073 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 373.64236495, 2024-09-20T22:10:34.073 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.985672357, 2024-09-20T22:10:34.073 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.073 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.073 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:34.073 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.074 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.074 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.074 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47810 2024-09-20T22:10:34.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.074 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352140+0000", 2024-09-20T22:10:34.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.074 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.074 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352140+0000", 2024-09-20T22:10:34.074 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352140+0000", 2024-09-20T22:10:34.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9299999999999997e-07 2024-09-20T22:10:34.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352187+0000", 2024-09-20T22:10:34.075 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6468000000000003e-05 2024-09-20T22:10:34.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.075 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.075 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352196+0000", 2024-09-20T22:10:34.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9670000000000007e-06 2024-09-20T22:10:34.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352202+0000", 2024-09-20T22:10:34.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0120000000000002e-06 2024-09-20T22:10:34.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352230+0000", 2024-09-20T22:10:34.076 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8289e-05 2024-09-20T22:10:34.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.076 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.076 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352236+0000", 2024-09-20T22:10:34.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.455e-06 2024-09-20T22:10:34.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236921+0000", 2024-09-20T22:10:34.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.88468433300000004 2024-09-20T22:10:34.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236931+0000", 2024-09-20T22:10:34.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0322e-05 2024-09-20T22:10:34.077 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.077 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.077 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.077 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.330063+0000", 2024-09-20T22:10:34.077 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.093132375000000003 2024-09-20T22:10:34.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:34.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.330188+0000", 2024-09-20T22:10:34.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000124664 2024-09-20T22:10:34.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:10:34.078 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.330329+0000", 2024-09-20T22:10:34.078 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014119299999999999 2024-09-20T22:10:34.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.078 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.078 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:34.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334105+0000", 2024-09-20T22:10:34.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037757799999999998 2024-09-20T22:10:34.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334140+0000", 2024-09-20T22:10:34.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5015000000000003e-05 2024-09-20T22:10:34.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337652+0000", 2024-09-20T22:10:34.079 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0035119819999999999 2024-09-20T22:10:34.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.079 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.079 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:34.079 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337709+0000", 2024-09-20T22:10:34.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6446000000000001e-05 2024-09-20T22:10:34.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.080 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.080 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337812+0000", 2024-09-20T22:10:34.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010336300000000001 2024-09-20T22:10:34.080 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.080 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.080 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.080 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.080 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.080 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47831 7.3 7:cf7c6d80:::benchmark_data_smithi060_78800_object47830:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:10:34.080 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.435232+0000", 2024-09-20T22:10:34.080 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 373.55927235799999, 2024-09-20T22:10:34.080 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90280988699999998, 2024-09-20T22:10:34.081 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.081 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.081 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:34.081 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.081 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.081 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.081 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47831 2024-09-20T22:10:34.081 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.081 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.081 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.081 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.081 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435232+0000", 2024-09-20T22:10:34.081 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435232+0000", 2024-09-20T22:10:34.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435233+0000", 2024-09-20T22:10:34.082 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0920000000000001e-06 2024-09-20T22:10:34.082 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.082 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.082 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.082 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435256+0000", 2024-09-20T22:10:34.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2840000000000002e-05 2024-09-20T22:10:34.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.083 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435270+0000", 2024-09-20T22:10:34.083 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3944000000000001e-05 2024-09-20T22:10:34.083 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.083 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.083 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435277+0000", 2024-09-20T22:10:34.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1420000000000004e-06 2024-09-20T22:10:34.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435341+0000", 2024-09-20T22:10:34.085 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3431999999999998e-05 2024-09-20T22:10:34.085 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.085 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.085 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.085 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435350+0000", 2024-09-20T22:10:34.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2920000000000001e-06 2024-09-20T22:10:34.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236961+0000", 2024-09-20T22:10:34.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.80161062500000002 2024-09-20T22:10:34.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.086 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.086 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236968+0000", 2024-09-20T22:10:34.086 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8529999999999992e-06 2024-09-20T22:10:34.086 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.086 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.331930+0000", 2024-09-20T22:10:34.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.094961952000000002 2024-09-20T22:10:34.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:34.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.332051+0000", 2024-09-20T22:10:34.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000120683 2024-09-20T22:10:34.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.087 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.087 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:10:34.087 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.332208+0000", 2024-09-20T22:10:34.087 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015661399999999999 2024-09-20T22:10:34.087 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.088 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.088 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334177+0000", 2024-09-20T22:10:34.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019690350000000001 2024-09-20T22:10:34.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.088 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:34.088 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334187+0000", 2024-09-20T22:10:34.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8870000000000004e-06 2024-09-20T22:10:34.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.088 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.088 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337890+0000", 2024-09-20T22:10:34.088 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037037379999999998 2024-09-20T22:10:34.088 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.088 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.089 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:34.089 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337940+0000", 2024-09-20T22:10:34.089 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0144e-05 2024-09-20T22:10:34.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.089 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.089 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.089 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.338042+0000", 2024-09-20T22:10:34.089 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010161400000000001 2024-09-20T22:10:34.089 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.089 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.089 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.089 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.089 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.089 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47833 7.3 7:c90762ed:::benchmark_data_smithi060_78800_object47832:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:10:34.089 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.437390+0000", 2024-09-20T22:10:34.090 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 373.55711439800001, 2024-09-20T22:10:34.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90566071400000003, 2024-09-20T22:10:34.090 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.090 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.090 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:34.090 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.090 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.090 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.090 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47833 2024-09-20T22:10:34.090 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.090 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.090 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.090 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.090 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437390+0000", 2024-09-20T22:10:34.090 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437390+0000", 2024-09-20T22:10:34.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437391+0000", 2024-09-20T22:10:34.091 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.085e-06 2024-09-20T22:10:34.091 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.091 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.091 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.091 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437415+0000", 2024-09-20T22:10:34.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3479e-05 2024-09-20T22:10:34.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437430+0000", 2024-09-20T22:10:34.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5485000000000002e-05 2024-09-20T22:10:34.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.092 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437439+0000", 2024-09-20T22:10:34.092 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3520000000000007e-06 2024-09-20T22:10:34.092 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.092 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.092 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.093 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437507+0000", 2024-09-20T22:10:34.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8011999999999999e-05 2024-09-20T22:10:34.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.093 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.093 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437520+0000", 2024-09-20T22:10:34.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3022e-05 2024-09-20T22:10:34.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.093 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.093 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.093 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237002+0000", 2024-09-20T22:10:34.093 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.79948219600000003 2024-09-20T22:10:34.093 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237011+0000", 2024-09-20T22:10:34.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8049999999999996e-06 2024-09-20T22:10:34.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334208+0000", 2024-09-20T22:10:34.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.097197622999999997 2024-09-20T22:10:34.094 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.094 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.094 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:34.094 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334306+0000", 2024-09-20T22:10:34.094 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8068999999999999e-05 2024-09-20T22:10:34.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:10:34.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334475+0000", 2024-09-20T22:10:34.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016830799999999999 2024-09-20T22:10:34.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337852+0000", 2024-09-20T22:10:34.095 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00337747 2024-09-20T22:10:34.095 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.095 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.095 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:34.095 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.340086+0000", 2024-09-20T22:10:34.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002234036 2024-09-20T22:10:34.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.342602+0000", 2024-09-20T22:10:34.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025159929999999998 2024-09-20T22:10:34.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:34.096 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.342808+0000", 2024-09-20T22:10:34.096 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020551099999999999 2024-09-20T22:10:34.096 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.096 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.096 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.097 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343051+0000", 2024-09-20T22:10:34.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00024326799999999999 2024-09-20T22:10:34.097 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.097 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.097 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.097 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.097 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.097 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47842 7.3 7:c97c3aed:::benchmark_data_smithi060_78800_object47841:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:10:34.097 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.442613+0000", 2024-09-20T22:10:34.097 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 373.551891791, 2024-09-20T22:10:34.097 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90079891400000001, 2024-09-20T22:10:34.097 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.097 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.097 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:34.098 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.098 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.098 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.098 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47842 2024-09-20T22:10:34.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.098 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.098 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.098 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442613+0000", 2024-09-20T22:10:34.098 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.098 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.098 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.098 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442613+0000", 2024-09-20T22:10:34.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.099 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442614+0000", 2024-09-20T22:10:34.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3e-06 2024-09-20T22:10:34.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.099 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.099 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442640+0000", 2024-09-20T22:10:34.099 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5544000000000001e-05 2024-09-20T22:10:34.099 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.099 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442651+0000", 2024-09-20T22:10:34.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1218e-05 2024-09-20T22:10:34.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442658+0000", 2024-09-20T22:10:34.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1980000000000004e-06 2024-09-20T22:10:34.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.100 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.100 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.100 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442734+0000", 2024-09-20T22:10:34.100 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6279999999999995e-05 2024-09-20T22:10:34.100 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442747+0000", 2024-09-20T22:10:34.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2393e-05 2024-09-20T22:10:34.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237041+0000", 2024-09-20T22:10:34.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.79429414499999995 2024-09-20T22:10:34.101 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.101 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.101 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.101 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237049+0000", 2024-09-20T22:10:34.101 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8150000000000006e-06 2024-09-20T22:10:34.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.102 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.102 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.335958+0000", 2024-09-20T22:10:34.102 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.098909056999999995 2024-09-20T22:10:34.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.102 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:34.102 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.336074+0000", 2024-09-20T22:10:34.102 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011582900000000001 2024-09-20T22:10:34.102 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.102 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.103 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:10:34.103 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.336216+0000", 2024-09-20T22:10:34.103 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014236899999999999 2024-09-20T22:10:34.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.103 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.103 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:34.103 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.342529+0000", 2024-09-20T22:10:34.103 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.006312799 2024-09-20T22:10:34.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.103 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.103 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.103 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343211+0000", 2024-09-20T22:10:34.103 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00068264900000000002 2024-09-20T22:10:34.103 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.103 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.103 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.104 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343252+0000", 2024-09-20T22:10:34.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1084000000000002e-05 2024-09-20T22:10:34.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.104 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:34.104 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343315+0000", 2024-09-20T22:10:34.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2989000000000003e-05 2024-09-20T22:10:34.104 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.104 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.104 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.104 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343412+0000", 2024-09-20T22:10:34.104 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6244999999999998e-05 2024-09-20T22:10:34.104 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.104 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.104 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.105 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.105 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47846 7.3 7:c61861ce:::benchmark_data_smithi060_78800_object47845:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:10:34.105 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.445377+0000", 2024-09-20T22:10:34.105 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 373.54912792200003, 2024-09-20T22:10:34.105 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91076886199999996, 2024-09-20T22:10:34.105 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.105 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.105 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:34.105 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.105 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.105 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.105 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47846 2024-09-20T22:10:34.105 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.105 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445377+0000", 2024-09-20T22:10:34.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445377+0000", 2024-09-20T22:10:34.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.106 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.106 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.106 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.106 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445378+0000", 2024-09-20T22:10:34.106 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.02e-06 2024-09-20T22:10:34.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.107 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.107 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.107 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445407+0000", 2024-09-20T22:10:34.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9654000000000001e-05 2024-09-20T22:10:34.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.107 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.107 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.107 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445423+0000", 2024-09-20T22:10:34.107 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6107999999999999e-05 2024-09-20T22:10:34.107 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.107 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.107 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.108 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445432+0000", 2024-09-20T22:10:34.108 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6770000000000001e-06 2024-09-20T22:10:34.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.108 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.108 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.108 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445522+0000", 2024-09-20T22:10:34.108 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0211999999999996e-05 2024-09-20T22:10:34.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.108 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.108 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.108 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445535+0000", 2024-09-20T22:10:34.108 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2279000000000001e-05 2024-09-20T22:10:34.108 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.108 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237076+0000", 2024-09-20T22:10:34.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.79154110300000002 2024-09-20T22:10:34.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237085+0000", 2024-09-20T22:10:34.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.611e-06 2024-09-20T22:10:34.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.109 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.338112+0000", 2024-09-20T22:10:34.109 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10102702700000001 2024-09-20T22:10:34.109 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.109 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.109 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:34.110 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.338234+0000", 2024-09-20T22:10:34.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121853 2024-09-20T22:10:34.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.110 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:10:34.110 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.338375+0000", 2024-09-20T22:10:34.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140428 2024-09-20T22:10:34.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.110 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.110 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343463+0000", 2024-09-20T22:10:34.110 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0050883710000000004 2024-09-20T22:10:34.110 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.110 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343504+0000", 2024-09-20T22:10:34.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1185e-05 2024-09-20T22:10:34.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:34.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.355992+0000", 2024-09-20T22:10:34.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.012487992 2024-09-20T22:10:34.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.111 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.111 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:34.111 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356046+0000", 2024-09-20T22:10:34.111 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4033000000000003e-05 2024-09-20T22:10:34.111 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.112 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.112 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.112 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356145+0000", 2024-09-20T22:10:34.112 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9308999999999994e-05 2024-09-20T22:10:34.112 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.112 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.112 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.112 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.112 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.112 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47847 7.3 7:c0c5cc37:::benchmark_data_smithi060_78800_object47846:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:10:34.112 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.445583+0000", 2024-09-20T22:10:34.112 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 373.54892164300003, 2024-09-20T22:10:34.112 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91077994900000003, 2024-09-20T22:10:34.112 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.112 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.113 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:34.113 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.113 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.113 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.113 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47847 2024-09-20T22:10:34.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.113 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.113 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.113 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.113 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445583+0000", 2024-09-20T22:10:34.113 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.113 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.113 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.113 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.113 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445583+0000", 2024-09-20T22:10:34.113 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.114 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.114 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.114 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445584+0000", 2024-09-20T22:10:34.114 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4900000000000005e-07 2024-09-20T22:10:34.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.114 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.114 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.114 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445607+0000", 2024-09-20T22:10:34.114 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3153e-05 2024-09-20T22:10:34.114 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.114 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.114 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.115 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445621+0000", 2024-09-20T22:10:34.115 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3638000000000001e-05 2024-09-20T22:10:34.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.115 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.115 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.115 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445628+0000", 2024-09-20T22:10:34.115 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.729e-06 2024-09-20T22:10:34.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.115 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.115 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.115 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445676+0000", 2024-09-20T22:10:34.115 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7911000000000003e-05 2024-09-20T22:10:34.115 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.115 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.116 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.116 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445685+0000", 2024-09-20T22:10:34.116 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8289999999999997e-06 2024-09-20T22:10:34.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.116 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.116 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.116 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237113+0000", 2024-09-20T22:10:34.116 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.79142811599999996 2024-09-20T22:10:34.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.116 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.116 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.116 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237121+0000", 2024-09-20T22:10:34.116 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8310000000000001e-06 2024-09-20T22:10:34.116 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.117 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.117 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.117 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.340131+0000", 2024-09-20T22:10:34.117 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103009795 2024-09-20T22:10:34.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.117 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.117 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:34.117 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.340324+0000", 2024-09-20T22:10:34.117 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000193034 2024-09-20T22:10:34.117 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.117 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.117 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:10:34.117 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.340466+0000", 2024-09-20T22:10:34.118 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000142032 2024-09-20T22:10:34.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.118 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.118 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.118 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343542+0000", 2024-09-20T22:10:34.118 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0030759860000000002 2024-09-20T22:10:34.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.118 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.118 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.118 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343584+0000", 2024-09-20T22:10:34.118 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1990999999999998e-05 2024-09-20T22:10:34.118 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.118 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.118 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:34.119 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356203+0000", 2024-09-20T22:10:34.119 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.012619361000000001 2024-09-20T22:10:34.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.119 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:34.119 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356268+0000", 2024-09-20T22:10:34.119 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4596000000000003e-05 2024-09-20T22:10:34.119 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.119 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.119 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.119 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356363+0000", 2024-09-20T22:10:34.119 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5098000000000003e-05 2024-09-20T22:10:34.119 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.119 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.120 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.120 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.120 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47853 7.3 7:c21485b4:::benchmark_data_smithi060_78800_object47852:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:10:34.120 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.457698+0000", 2024-09-20T22:10:34.120 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 373.53680634599999, 2024-09-20T22:10:34.120 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.89889923500000002, 2024-09-20T22:10:34.120 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.120 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.120 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:34.120 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.120 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.120 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.120 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47853 2024-09-20T22:10:34.120 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.121 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.121 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.121 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457698+0000", 2024-09-20T22:10:34.121 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.121 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.121 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457698+0000", 2024-09-20T22:10:34.121 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.121 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.121 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.121 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.121 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457699+0000", 2024-09-20T22:10:34.122 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.159e-06 2024-09-20T22:10:34.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.122 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.122 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.122 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457753+0000", 2024-09-20T22:10:34.122 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4067999999999998e-05 2024-09-20T22:10:34.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.122 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.122 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.122 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457765+0000", 2024-09-20T22:10:34.122 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1430000000000001e-05 2024-09-20T22:10:34.122 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.122 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.122 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.122 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457770+0000", 2024-09-20T22:10:34.123 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4410000000000001e-06 2024-09-20T22:10:34.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.123 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.123 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457821+0000", 2024-09-20T22:10:34.123 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1156999999999998e-05 2024-09-20T22:10:34.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.123 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.123 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457829+0000", 2024-09-20T22:10:34.123 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5259999999999996e-06 2024-09-20T22:10:34.123 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.123 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.123 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.124 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237712+0000", 2024-09-20T22:10:34.124 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.77988319699999997 2024-09-20T22:10:34.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.124 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.124 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237742+0000", 2024-09-20T22:10:34.124 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0224000000000001e-05 2024-09-20T22:10:34.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.124 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.124 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343621+0000", 2024-09-20T22:10:34.124 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10587846400000001 2024-09-20T22:10:34.124 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.124 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:34.125 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:34.125 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:34.126 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:34.126 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343759+0000", 2024-09-20T22:10:34.126 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000137947 2024-09-20T22:10:34.126 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.126 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.126 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:10:34.126 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343960+0000", 2024-09-20T22:10:34.127 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000201201 2024-09-20T22:10:34.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.127 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.127 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.127 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.349430+0000", 2024-09-20T22:10:34.127 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0054698710000000003 2024-09-20T22:10:34.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.127 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.127 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.127 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.349473+0000", 2024-09-20T22:10:34.127 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2865999999999999e-05 2024-09-20T22:10:34.127 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.127 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.128 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:34.128 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356374+0000", 2024-09-20T22:10:34.128 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0069012359999999998 2024-09-20T22:10:34.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.128 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.128 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:34.128 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356434+0000", 2024-09-20T22:10:34.128 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9669000000000001e-05 2024-09-20T22:10:34.128 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.128 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.128 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.128 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356597+0000", 2024-09-20T22:10:34.128 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016377899999999999 2024-09-20T22:10:34.128 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.128 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.128 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.129 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.129 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169951 7.3 7:c27f31c3:::benchmark_data_smithi060_78800_object36918:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:10:34.129 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440033+0000", 2024-09-20T22:10:34.129 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 275.55447115499999, 2024-09-20T22:10:34.129 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0308894289999999, 2024-09-20T22:10:34.129 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.129 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.129 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:10:34.129 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.129 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.129 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.129 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169951 2024-09-20T22:10:34.129 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.130 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.130 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.130 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440033+0000", 2024-09-20T22:10:34.130 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.130 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.130 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440033+0000", 2024-09-20T22:10:34.130 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.130 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.130 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.130 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.130 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440035+0000", 2024-09-20T22:10:34.131 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3170000000000001e-06 2024-09-20T22:10:34.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.131 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.131 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.131 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440041+0000", 2024-09-20T22:10:34.131 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8869999999999997e-06 2024-09-20T22:10:34.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.131 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.131 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.131 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440053+0000", 2024-09-20T22:10:34.131 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2146e-05 2024-09-20T22:10:34.131 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.131 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.131 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.132 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440061+0000", 2024-09-20T22:10:34.132 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3049999999999993e-06 2024-09-20T22:10:34.132 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.132 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.132 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.132 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440616+0000", 2024-09-20T22:10:34.132 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00055526700000000002 2024-09-20T22:10:34.132 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.132 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.132 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op must wait for map", 2024-09-20T22:10:34.132 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440623+0000", 2024-09-20T22:10:34.132 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9290000000000001e-06 2024-09-20T22:10:34.132 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.132 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.132 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.133 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.659494+0000", 2024-09-20T22:10:34.133 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.21887063800000001 2024-09-20T22:10:34.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.133 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.133 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.133 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.659506+0000", 2024-09-20T22:10:34.133 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2313000000000001e-05 2024-09-20T22:10:34.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.133 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.133 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.133 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.470588+0000", 2024-09-20T22:10:34.133 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.81108138900000004 2024-09-20T22:10:34.133 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.133 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.134 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.134 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.470923+0000", 2024-09-20T22:10:34.134 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033523800000000002 2024-09-20T22:10:34.134 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.134 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.134 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.134 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.134 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.134 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169962 7.8 7:1412370b:::benchmark_data_smithi060_78800_object36929:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:10:34.134 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440138+0000", 2024-09-20T22:10:34.134 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 275.554366125, 2024-09-20T22:10:34.134 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0713576439999999, 2024-09-20T22:10:34.134 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.134 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.134 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:10:34.134 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.135 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.135 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.135 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169962 2024-09-20T22:10:34.135 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.135 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.135 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.135 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.135 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440138+0000", 2024-09-20T22:10:34.135 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.135 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.135 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.135 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.135 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440138+0000", 2024-09-20T22:10:34.135 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.136 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.136 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.136 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.136 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440139+0000", 2024-09-20T22:10:34.136 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1399999999999996e-07 2024-09-20T22:10:34.136 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.136 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.136 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.136 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440143+0000", 2024-09-20T22:10:34.136 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8029999999999998e-06 2024-09-20T22:10:34.136 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.136 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.136 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.137 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440152+0000", 2024-09-20T22:10:34.137 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-20T22:10:34.137 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.137 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.137 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.137 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440158+0000", 2024-09-20T22:10:34.137 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3319999999999997e-06 2024-09-20T22:10:34.137 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.137 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.137 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.137 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440263+0000", 2024-09-20T22:10:34.137 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105258 2024-09-20T22:10:34.137 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.137 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.137 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op must wait for map", 2024-09-20T22:10:34.138 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440269+0000", 2024-09-20T22:10:34.138 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8549999999999998e-06 2024-09-20T22:10:34.138 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.138 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.138 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.138 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.647667+0000", 2024-09-20T22:10:34.138 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.207398358 2024-09-20T22:10:34.138 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.138 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.138 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.138 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.647675+0000", 2024-09-20T22:10:34.138 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5979999999999999e-06 2024-09-20T22:10:34.138 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.138 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.138 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.139 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.490134+0000", 2024-09-20T22:10:34.139 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.84245882699999997 2024-09-20T22:10:34.139 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.139 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.139 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:10:34.139 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.490253+0000", 2024-09-20T22:10:34.139 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011938999999999999 2024-09-20T22:10:34.139 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.139 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.139 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.139 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.511425+0000", 2024-09-20T22:10:34.139 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021171453999999999 2024-09-20T22:10:34.139 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.139 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.139 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.140 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.511496+0000", 2024-09-20T22:10:34.140 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1359000000000006e-05 2024-09-20T22:10:34.140 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.140 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.140 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.140 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.140 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.140 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169969 7.18 7:18661b50:::benchmark_data_smithi060_78800_object36936:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:10:34.140 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440181+0000", 2024-09-20T22:10:34.140 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 275.55432370699998, 2024-09-20T22:10:34.140 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.074892379, 2024-09-20T22:10:34.140 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.140 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.140 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:10:34.140 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.140 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.141 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.141 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169969 2024-09-20T22:10:34.141 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.141 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.141 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.141 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.141 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440181+0000", 2024-09-20T22:10:34.141 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.141 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.141 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.141 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.141 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440181+0000", 2024-09-20T22:10:34.141 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.141 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.142 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.142 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.142 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440181+0000", 2024-09-20T22:10:34.142 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7100000000000002e-07 2024-09-20T22:10:34.142 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.142 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.142 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.142 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440185+0000", 2024-09-20T22:10:34.142 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3840000000000001e-06 2024-09-20T22:10:34.142 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.142 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.142 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.142 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440193+0000", 2024-09-20T22:10:34.143 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5790000000000004e-06 2024-09-20T22:10:34.143 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.143 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.143 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.143 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440200+0000", 2024-09-20T22:10:34.143 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2310000000000001e-06 2024-09-20T22:10:34.143 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.143 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.143 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.143 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.647597+0000", 2024-09-20T22:10:34.143 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.20739776200000001 2024-09-20T22:10:34.143 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.143 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.143 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.143 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.647606+0000", 2024-09-20T22:10:34.144 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6580000000000008e-06 2024-09-20T22:10:34.144 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.144 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.144 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.144 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.510928+0000", 2024-09-20T22:10:34.144 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.86332191400000002 2024-09-20T22:10:34.144 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.144 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.144 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:10:34.144 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.511051+0000", 2024-09-20T22:10:34.144 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012358099999999999 2024-09-20T22:10:34.144 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.144 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.144 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.144 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.514989+0000", 2024-09-20T22:10:34.144 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003937744 2024-09-20T22:10:34.145 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.145 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.145 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.145 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.515073+0000", 2024-09-20T22:10:34.145 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3955000000000005e-05 2024-09-20T22:10:34.145 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.145 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.145 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.145 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.145 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.145 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227626 7.3 7:c62c9d8d:::benchmark_data_smithi060_78800_object94593:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:10:34.145 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.821701+0000", 2024-09-20T22:10:34.145 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 220.17280389800001, 2024-09-20T22:10:34.145 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1262373590000001, 2024-09-20T22:10:34.145 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:34.145 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:34.146 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:34.146 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:34.146 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:10:34.146 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:34.146 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227626 2024-09-20T22:10:34.146 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.146 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:34.146 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.146 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:34.146 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821701+0000", 2024-09-20T22:10:34.146 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.146 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.146 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.146 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:34.146 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821701+0000", 2024-09-20T22:10:34.147 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:34.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.147 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.147 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:34.147 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821702+0000", 2024-09-20T22:10:34.147 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.102e-06 2024-09-20T22:10:34.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.147 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.147 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:34.147 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821707+0000", 2024-09-20T22:10:34.147 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0579999999999997e-06 2024-09-20T22:10:34.147 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.147 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.147 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:34.148 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821733+0000", 2024-09-20T22:10:34.148 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6642999999999999e-05 2024-09-20T22:10:34.148 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.148 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.148 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:34.148 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821741+0000", 2024-09-20T22:10:34.148 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7009999999999996e-06 2024-09-20T22:10:34.148 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.148 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.148 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.148 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821806+0000", 2024-09-20T22:10:34.148 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4882000000000007e-05 2024-09-20T22:10:34.148 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.149 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.149 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:10:34.149 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821817+0000", 2024-09-20T22:10:34.149 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0852999999999999e-05 2024-09-20T22:10:34.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.149 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.149 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:34.149 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.941998+0000", 2024-09-20T22:10:34.149 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1201808070000001 2024-09-20T22:10:34.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.149 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.149 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:34.149 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.942120+0000", 2024-09-20T22:10:34.149 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012227900000000001 2024-09-20T22:10:34.149 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.150 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.150 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,3,4", 2024-09-20T22:10:34.150 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.942615+0000", 2024-09-20T22:10:34.150 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00049518700000000001 2024-09-20T22:10:34.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.150 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.150 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:34.150 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946370+0000", 2024-09-20T22:10:34.150 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003755141 2024-09-20T22:10:34.150 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.150 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.150 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.150 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946870+0000", 2024-09-20T22:10:34.151 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00049939899999999998 2024-09-20T22:10:34.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.151 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.151 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.151 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947042+0000", 2024-09-20T22:10:34.151 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000172199 2024-09-20T22:10:34.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.151 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.151 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:34.151 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947739+0000", 2024-09-20T22:10:34.151 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00069714899999999999 2024-09-20T22:10:34.151 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.151 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.151 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:10:34.151 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947809+0000", 2024-09-20T22:10:34.152 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9530999999999995e-05 2024-09-20T22:10:34.152 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:34.152 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:34.152 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:10:34.152 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947938+0000", 2024-09-20T22:10:34.152 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012942800000000001 2024-09-20T22:10:34.152 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.152 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.152 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.152 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:34.152 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:34.152 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:10:34.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:34.264 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:34.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:34.265 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:34.366 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:34.399 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:34.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:34.400 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:34.501 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:34.538 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:34.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:34.539 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:34.639 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:34.678 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:34.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:34.679 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:34.780 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:34.817 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:34.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:34.818 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:34.919 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:34.955 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:34.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:34.956 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:34 smithi067 ceph-mon[33511]: 5.5 scrub starts 2024-09-20T22:10:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:34 smithi067 ceph-mon[33511]: 5.5 scrub ok 2024-09-20T22:10:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:34 smithi067 ceph-mon[33511]: pgmap v402: 168 pgs: 1 active+remapped+backfilling, 1 activating+degraded+remapped, 1 active+undersized+remapped, 2 activating+degraded, 4 activating, 5 remapped+peering, 2 active+remapped+backfill_toofull, 152 active+clean; 3.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 7.5 KiB/s rd, 55 KiB/s wr, 1.06k op/s; 10/156297 objects degraded (0.006%); 12734/156297 objects misplaced (8.147%); 2.1 MiB/s, 38 objects/s recovering 2024-09-20T22:10:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:34 smithi067 ceph-mon[33511]: 5.f scrub starts 2024-09-20T22:10:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:34 smithi067 ceph-mon[33511]: 5.f scrub ok 2024-09-20T22:10:35.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:34 smithi060 ceph-mon[29694]: 5.5 scrub starts 2024-09-20T22:10:35.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:34 smithi060 ceph-mon[29694]: 5.5 scrub ok 2024-09-20T22:10:35.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:34 smithi060 ceph-mon[29694]: pgmap v402: 168 pgs: 1 active+remapped+backfilling, 1 activating+degraded+remapped, 1 active+undersized+remapped, 2 activating+degraded, 4 activating, 5 remapped+peering, 2 active+remapped+backfill_toofull, 152 active+clean; 3.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 7.5 KiB/s rd, 55 KiB/s wr, 1.06k op/s; 10/156297 objects degraded (0.006%); 12734/156297 objects misplaced (8.147%); 2.1 MiB/s, 38 objects/s recovering 2024-09-20T22:10:35.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:34 smithi060 ceph-mon[29694]: 5.f scrub starts 2024-09-20T22:10:35.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:34 smithi060 ceph-mon[29694]: 5.f scrub ok 2024-09-20T22:10:35.030 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:34 smithi060 ceph-mon[40290]: 5.5 scrub starts 2024-09-20T22:10:35.030 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:34 smithi060 ceph-mon[40290]: 5.5 scrub ok 2024-09-20T22:10:35.030 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:34 smithi060 ceph-mon[40290]: pgmap v402: 168 pgs: 1 active+remapped+backfilling, 1 activating+degraded+remapped, 1 active+undersized+remapped, 2 activating+degraded, 4 activating, 5 remapped+peering, 2 active+remapped+backfill_toofull, 152 active+clean; 3.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 7.5 KiB/s rd, 55 KiB/s wr, 1.06k op/s; 10/156297 objects degraded (0.006%); 12734/156297 objects misplaced (8.147%); 2.1 MiB/s, 38 objects/s recovering 2024-09-20T22:10:35.030 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:34 smithi060 ceph-mon[40290]: 5.f scrub starts 2024-09-20T22:10:35.030 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:34 smithi060 ceph-mon[40290]: 5.f scrub ok 2024-09-20T22:10:35.057 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:35.082 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-20T22:10:35.082 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd test-reweight-by-utilization 110 3.0 2024-09-20T22:10:35.113 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:35.113 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:35.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:35.114 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:35.216 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:35.278 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:35.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:35.279 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:35.285 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:10:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:10:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:10:35.380 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:35.416 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:35.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:35.417 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:35.518 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:35.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:35.576 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:35.576 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:35.608 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T22:10:35.677 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:35.752 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:35.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:35.754 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:35.786 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:10:35.839 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:35 smithi060 ceph-mon[29694]: 1.8 scrub starts 2024-09-20T22:10:35.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:35 smithi060 ceph-mon[29694]: 1.8 scrub ok 2024-09-20T22:10:35.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:35 smithi060 ceph-mon[29694]: 3.1d scrub starts 2024-09-20T22:10:35.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:35 smithi060 ceph-mon[29694]: 3.1d scrub ok 2024-09-20T22:10:35.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:35 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:10:35.840 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:35 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:10:35.840 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:35 smithi060 ceph-mon[40290]: 1.8 scrub starts 2024-09-20T22:10:35.840 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:35 smithi060 ceph-mon[40290]: 1.8 scrub ok 2024-09-20T22:10:35.841 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:35 smithi060 ceph-mon[40290]: 3.1d scrub starts 2024-09-20T22:10:35.841 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:35 smithi060 ceph-mon[40290]: 3.1d scrub ok 2024-09-20T22:10:35.841 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:35 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:10:35.841 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:35 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:10:35.854 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:35.881 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:35.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:35.882 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:35.982 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:36.012 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:36.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:36.013 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:35 smithi067 ceph-mon[33511]: 1.8 scrub starts 2024-09-20T22:10:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:35 smithi067 ceph-mon[33511]: 1.8 scrub ok 2024-09-20T22:10:36.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:35 smithi067 ceph-mon[33511]: 3.1d scrub starts 2024-09-20T22:10:36.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:35 smithi067 ceph-mon[33511]: 3.1d scrub ok 2024-09-20T22:10:36.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:35 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:10:36.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:35 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:10:36.114 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:36.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:36.179 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:36.179 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:36.280 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:36.352 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:36.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:36.375 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:36.376 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:36.414 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:36.430 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:36.476 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:36.536 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:36.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:36.537 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:36.637 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:36.668 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:36.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:36.669 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:36.708 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:10:36.756 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:36 smithi060 ceph-mon[40290]: 6.10 scrub starts 2024-09-20T22:10:36.756 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:36 smithi060 ceph-mon[40290]: 6.10 scrub ok 2024-09-20T22:10:36.756 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:36 smithi060 ceph-mon[40290]: pgmap v403: 168 pgs: 1 active+remapped+backfilling, 1 activating+degraded+remapped, 2 remapped+peering, 3 active+remapped+backfill_toofull, 161 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.19k op/s; 4/149403 objects degraded (0.003%); 14086/149403 objects misplaced (9.428%); 1.5 MiB/s, 29 objects/s recovering 2024-09-20T22:10:36.757 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:36 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3879884282' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:10:36.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:36 smithi060 ceph-mon[29694]: 6.10 scrub starts 2024-09-20T22:10:36.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:36 smithi060 ceph-mon[29694]: 6.10 scrub ok 2024-09-20T22:10:36.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:36 smithi060 ceph-mon[29694]: pgmap v403: 168 pgs: 1 active+remapped+backfilling, 1 activating+degraded+remapped, 2 remapped+peering, 3 active+remapped+backfill_toofull, 161 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.19k op/s; 4/149403 objects degraded (0.003%); 14086/149403 objects misplaced (9.428%); 1.5 MiB/s, 29 objects/s recovering 2024-09-20T22:10:36.757 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:36 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3879884282' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:10:36.770 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:36.806 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:36.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:36.807 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:36.908 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:36.940 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:36.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:36.941 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:36 smithi067 ceph-mon[33511]: 6.10 scrub starts 2024-09-20T22:10:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:36 smithi067 ceph-mon[33511]: 6.10 scrub ok 2024-09-20T22:10:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:36 smithi067 ceph-mon[33511]: pgmap v403: 168 pgs: 1 active+remapped+backfilling, 1 activating+degraded+remapped, 2 remapped+peering, 3 active+remapped+backfill_toofull, 161 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.19k op/s; 4/149403 objects degraded (0.003%); 14086/149403 objects misplaced (9.428%); 1.5 MiB/s, 29 objects/s recovering 2024-09-20T22:10:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:36 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3879884282' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:10:37.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:37.083 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:37.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:37.084 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:37.185 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:37.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:37.218 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:37.218 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:37.318 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:37.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:37.346 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:37.347 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:37.447 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:37.480 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:37.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:37.481 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:37.581 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:37.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:37.607 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:37.607 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:37.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:37.744 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:37.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:37.745 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:37.845 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:37.884 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:37.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:37.885 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:37.919 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:37 smithi060 ceph-mon[29694]: 3.16 deep-scrub starts 2024-09-20T22:10:37.919 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:37 smithi060 ceph-mon[29694]: 3.16 deep-scrub ok 2024-09-20T22:10:37.919 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:37 smithi060 ceph-mon[29694]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:10:37.919 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:37 smithi060 ceph-mon[29694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 1 pg peering) 2024-09-20T22:10:37.919 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:37 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:10:37.919 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:37 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3879884282' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:10:37.919 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:37 smithi060 ceph-mon[29694]: osdmap e256: 8 total, 8 up, 8 in 2024-09-20T22:10:37.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:37 smithi060 ceph-mon[40290]: 3.16 deep-scrub starts 2024-09-20T22:10:37.920 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:37 smithi060 ceph-mon[40290]: 3.16 deep-scrub ok 2024-09-20T22:10:37.920 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:37 smithi060 ceph-mon[40290]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:10:37.920 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:37 smithi060 ceph-mon[40290]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 1 pg peering) 2024-09-20T22:10:37.920 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:37 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:10:37.920 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:37 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3879884282' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:10:37.920 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:37 smithi060 ceph-mon[40290]: osdmap e256: 8 total, 8 up, 8 in 2024-09-20T22:10:37.987 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:37 smithi067 ceph-mon[33511]: 3.16 deep-scrub starts 2024-09-20T22:10:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:37 smithi067 ceph-mon[33511]: 3.16 deep-scrub ok 2024-09-20T22:10:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:37 smithi067 ceph-mon[33511]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:10:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:37 smithi067 ceph-mon[33511]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs inactive, 1 pg peering) 2024-09-20T22:10:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:37 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:10:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:37 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3879884282' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:10:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:37 smithi067 ceph-mon[33511]: osdmap e256: 8 total, 8 up, 8 in 2024-09-20T22:10:38.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:38.027 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:38.028 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:38.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:38.173 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:38.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:38.174 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:38.196 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:10:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:10:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.15192.0:195328 8.18 e256/228 8:1dd341a1:::benchmark_data_smithi060_111051_object75329:head v 256'9416, mlcod=256'9413)", 2024-09-20T22:10:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:38.185655+0000", 2024-09-20T22:10:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0084283510000000006, 2024-09-20T22:10:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0084568709999999995, 2024-09-20T22:10:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:10:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:38.197 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.185655+0000", 2024-09-20T22:10:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.185655+0000", 2024-09-20T22:10:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.185656+0000", 2024-09-20T22:10:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8500000000000001e-07 2024-09-20T22:10:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.198 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.185660+0000", 2024-09-20T22:10:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6550000000000003e-06 2024-09-20T22:10:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.185670+0000", 2024-09-20T22:10:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1279999999999997e-06 2024-09-20T22:10:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.185675+0000", 2024-09-20T22:10:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6269999999999997e-06 2024-09-20T22:10:38.199 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.186705+0000", 2024-09-20T22:10:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0010301450000000001 2024-09-20T22:10:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.186730+0000", 2024-09-20T22:10:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.459e-05 2024-09-20T22:10:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.200 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:195331 8.13 8:cfa1f2fd:::benchmark_data_smithi060_111051_object75332:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e256)", 2024-09-20T22:10:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:38.185879+0000", 2024-09-20T22:10:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0082048450000000005, 2024-09-20T22:10:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0083054240000000005, 2024-09-20T22:10:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:10:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:10:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 195331 2024-09-20T22:10:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.201 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.185879+0000", 2024-09-20T22:10:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.185879+0000", 2024-09-20T22:10:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:38.202 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.185893+0000", 2024-09-20T22:10:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4418e-05 2024-09-20T22:10:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.185959+0000", 2024-09-20T22:10:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6192000000000006e-05 2024-09-20T22:10:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.185972+0000", 2024-09-20T22:10:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2490000000000001e-05 2024-09-20T22:10:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.203 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.185980+0000", 2024-09-20T22:10:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7370000000000006e-06 2024-09-20T22:10:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.188633+0000", 2024-09-20T22:10:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0026535399999999998 2024-09-20T22:10:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.188758+0000", 2024-09-20T22:10:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000125122 2024-09-20T22:10:38.204 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:10:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.189663+0000", 2024-09-20T22:10:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00090499299999999999 2024-09-20T22:10:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:195335 8.0 8:07a7c4fa:::benchmark_data_smithi060_111051_object75336:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e256)", 2024-09-20T22:10:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:38.186849+0000", 2024-09-20T22:10:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0072341109999999997, 2024-09-20T22:10:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0074145929999999997, 2024-09-20T22:10:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:38.205 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:10:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:10:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 195335 2024-09-20T22:10:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.186849+0000", 2024-09-20T22:10:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.206 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.186849+0000", 2024-09-20T22:10:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.186851+0000", 2024-09-20T22:10:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0780000000000001e-06 2024-09-20T22:10:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.186856+0000", 2024-09-20T22:10:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9810000000000003e-06 2024-09-20T22:10:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.207 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.186869+0000", 2024-09-20T22:10:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3021e-05 2024-09-20T22:10:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.186876+0000", 2024-09-20T22:10:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7400000000000004e-06 2024-09-20T22:10:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.187960+0000", 2024-09-20T22:10:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0010838410000000001 2024-09-20T22:10:38.208 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.188077+0000", 2024-09-20T22:10:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000116834 2024-09-20T22:10:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4,5", 2024-09-20T22:10:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.188494+0000", 2024-09-20T22:10:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00041711 2024-09-20T22:10:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.193397+0000", 2024-09-20T22:10:38.209 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0049028250000000004 2024-09-20T22:10:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.193441+0000", 2024-09-20T22:10:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4443000000000001e-05 2024-09-20T22:10:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.210 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:195336 8.f 8:f2d60f95:::benchmark_data_smithi060_111051_object75337:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e256)", 2024-09-20T22:10:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:38.187231+0000", 2024-09-20T22:10:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0068526159999999997, 2024-09-20T22:10:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0071238619999999999, 2024-09-20T22:10:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:10:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:38.211 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:10:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 195336 2024-09-20T22:10:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.187231+0000", 2024-09-20T22:10:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.187231+0000", 2024-09-20T22:10:38.212 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.187232+0000", 2024-09-20T22:10:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.031e-06 2024-09-20T22:10:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.187237+0000", 2024-09-20T22:10:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9259999999999999e-06 2024-09-20T22:10:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:38.213 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.187249+0000", 2024-09-20T22:10:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2194e-05 2024-09-20T22:10:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.187257+0000", 2024-09-20T22:10:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.61e-06 2024-09-20T22:10:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.187872+0000", 2024-09-20T22:10:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00061488600000000003 2024-09-20T22:10:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.214 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.187994+0000", 2024-09-20T22:10:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121915 2024-09-20T22:10:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:10:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.188113+0000", 2024-09-20T22:10:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119379 2024-09-20T22:10:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:10:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.192617+0000", 2024-09-20T22:10:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0045040560000000002 2024-09-20T22:10:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.215 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.193732+0000", 2024-09-20T22:10:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0011151329999999999 2024-09-20T22:10:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:195338 8.b 8:d9951c89:::benchmark_data_smithi060_111051_object75339:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e256)", 2024-09-20T22:10:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:38.191235+0000", 2024-09-20T22:10:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.002848237, 2024-09-20T22:10:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0032073399999999999, 2024-09-20T22:10:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:38.216 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:10:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:10:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 195338 2024-09-20T22:10:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.191235+0000", 2024-09-20T22:10:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.217 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.191235+0000", 2024-09-20T22:10:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.191237+0000", 2024-09-20T22:10:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.19e-06 2024-09-20T22:10:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.191241+0000", 2024-09-20T22:10:38.218 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4209999999999997e-06 2024-09-20T22:10:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.191254+0000", 2024-09-20T22:10:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3159999999999999e-05 2024-09-20T22:10:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.191260+0000", 2024-09-20T22:10:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4469999999999997e-06 2024-09-20T22:10:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.219 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.192346+0000", 2024-09-20T22:10:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0010863559999999999 2024-09-20T22:10:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.192654+0000", 2024-09-20T22:10:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030776700000000002 2024-09-20T22:10:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:10:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.192773+0000", 2024-09-20T22:10:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011928 2024-09-20T22:10:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:38.220 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:195340 8.13 8:c8521d1c:::benchmark_data_smithi060_111051_object75341:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e256)", 2024-09-20T22:10:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:38.194037+0000", 2024-09-20T22:10:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4.6606000000000001e-05, 2024-09-20T22:10:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00048058600000000002, 2024-09-20T22:10:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:10:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:10:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 195340 2024-09-20T22:10:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.221 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194037+0000", 2024-09-20T22:10:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194037+0000", 2024-09-20T22:10:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194046+0000", 2024-09-20T22:10:38.222 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0890000000000006e-06 2024-09-20T22:10:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194051+0000", 2024-09-20T22:10:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1200000000000001e-06 2024-09-20T22:10:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194065+0000", 2024-09-20T22:10:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3475000000000001e-05 2024-09-20T22:10:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:38.223 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194072+0000", 2024-09-20T22:10:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6310000000000003e-06 2024-09-20T22:10:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194187+0000", 2024-09-20T22:10:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114522 2024-09-20T22:10:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194290+0000", 2024-09-20T22:10:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010303599999999999 2024-09-20T22:10:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:38.224 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:195341 8.13 8.ed7ffed3 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e256)", 2024-09-20T22:10:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:38.194091+0000", 2024-09-20T22:10:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.9999924, 2024-09-20T22:10:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00049391799999999998, 2024-09-20T22:10:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:10:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:10:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:10:38.225 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 195341 2024-09-20T22:10:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194091+0000", 2024-09-20T22:10:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194091+0000", 2024-09-20T22:10:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.226 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194092+0000", 2024-09-20T22:10:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6300000000000005e-07 2024-09-20T22:10:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194095+0000", 2024-09-20T22:10:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6669999999999998e-06 2024-09-20T22:10:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194106+0000", 2024-09-20T22:10:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.061e-05 2024-09-20T22:10:38.227 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194112+0000", 2024-09-20T22:10:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6500000000000001e-06 2024-09-20T22:10:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "MOSDPGPush(8.a 256/252 [PushOp(8:53b44155:::benchmark_data_smithi060_111051_object80115:head, version: 198'5030, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8:53b44155:::benchmark_data_smithi060_111051_object80115:head@198'5030, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-20T22:10:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:38.194138+0000", 2024-09-20T22:10:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.9999456, 2024-09-20T22:10:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00053525600000000001, 2024-09-20T22:10:38.228 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:10:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:10:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:10:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:10:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:10:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194138+0000", 2024-09-20T22:10:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:10:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194138+0000", 2024-09-20T22:10:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:10:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:10:38.229 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194139+0000", 2024-09-20T22:10:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4000000000000001e-07 2024-09-20T22:10:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:10:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194173+0000", 2024-09-20T22:10:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4328000000000001e-05 2024-09-20T22:10:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:10:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194190+0000", 2024-09-20T22:10:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6931000000000001e-05 2024-09-20T22:10:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:10:38.230 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194206+0000", 2024-09-20T22:10:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6308e-05 2024-09-20T22:10:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:10:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194624+0000", 2024-09-20T22:10:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00041781400000000002 2024-09-20T22:10:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:10:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:10:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:10:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:38.194638+0000", 2024-09-20T22:10:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4504999999999999e-05 2024-09-20T22:10:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:10:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:38.231 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:10:38.232 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:10:38.232 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 8 2024-09-20T22:10:38.232 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:10:38.275 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:38.297 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:38.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:38.298 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:38.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:38.436 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:38.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:38.437 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:38.538 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:38.578 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:38.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:38.579 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:38.679 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:38.719 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:38.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:38.720 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:38.820 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:38.857 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:38.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:38.858 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:38.959 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:38.990 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:38 smithi060 ceph-mon[29694]: pgmap v405: 168 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 162 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 957 op/s; 13278/140688 objects misplaced (9.438%); 13 MiB/s, 211 objects/s recovering 2024-09-20T22:10:38.990 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:38 smithi060 ceph-mon[40290]: pgmap v405: 168 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 162 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 957 op/s; 13278/140688 objects misplaced (9.438%); 13 MiB/s, 211 objects/s recovering 2024-09-20T22:10:39.004 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:39.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:39.005 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:38 smithi067 ceph-mon[33511]: pgmap v405: 168 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 162 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 957 op/s; 13278/140688 objects misplaced (9.438%); 13 MiB/s, 211 objects/s recovering 2024-09-20T22:10:39.105 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:39.133 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:39.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:39.134 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:39.235 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:39.265 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:39.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:39.266 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:39.367 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:39.407 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:39.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:39.407 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:39.508 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:39.544 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:39.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:39.545 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:39.646 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:39.688 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:39.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:39.689 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:39.790 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:39.827 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:39.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:39.828 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:39.884 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T22:10:39.929 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:39.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:39.971 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:39.971 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:39.973 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:39 smithi060 ceph-mon[29694]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/149403 objects degraded (0.003%), 1 pg degraded) 2024-09-20T22:10:39.973 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:39 smithi060 ceph-mon[40290]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/149403 objects degraded (0.003%), 1 pg degraded) 2024-09-20T22:10:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:39 smithi067 ceph-mon[33511]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4/149403 objects degraded (0.003%), 1 pg degraded) 2024-09-20T22:10:40.073 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:40.116 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:40.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:40.117 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:40.218 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:40.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:40.268 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:40.268 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:40.369 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:40.400 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:40.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:40.401 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:40.502 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:40.551 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:40.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:40.552 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:40.653 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:40.693 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:40.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:40.694 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:40.795 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:40.821 INFO:teuthology.orchestra.run.smithi060.stdout:moved 31 / 504 (6.15079%) 2024-09-20T22:10:40.821 INFO:teuthology.orchestra.run.smithi060.stdout:avg 63 2024-09-20T22:10:40.822 INFO:teuthology.orchestra.run.smithi060.stdout:stddev 9.08295 -> 13.6015 (expected baseline 7.42462) 2024-09-20T22:10:40.822 INFO:teuthology.orchestra.run.smithi060.stdout:min osd.5 with 46 -> 33 pgs (0.730159 -> 0.52381 * mean) 2024-09-20T22:10:40.822 INFO:teuthology.orchestra.run.smithi060.stdout:max osd.4 with 73 -> 77 pgs (1.15873 -> 1.22222 * mean) 2024-09-20T22:10:40.822 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:10:40.822 INFO:teuthology.orchestra.run.smithi060.stdout:oload 110 2024-09-20T22:10:40.822 INFO:teuthology.orchestra.run.smithi060.stdout:max_change 3 2024-09-20T22:10:40.822 INFO:teuthology.orchestra.run.smithi060.stdout:max_change_osds 4 2024-09-20T22:10:40.822 INFO:teuthology.orchestra.run.smithi060.stdout:average_utilization 0.0165 2024-09-20T22:10:40.822 INFO:teuthology.orchestra.run.smithi060.stdout:overload_utilization 0.0182 2024-09-20T22:10:40.822 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 weight 0.6893 -> 0.4743 2024-09-20T22:10:40.822 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 weight 0.8012 -> 0.9368 2024-09-20T22:10:40.822 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2 weight 0.8900 -> 0.7856 2024-09-20T22:10:40.823 INFO:teuthology.orchestra.run.smithi060.stderr:no change 2024-09-20T22:10:40.837 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:40.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:40.838 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:40.939 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:40.947 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:10:40.948 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:10:40.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:40 smithi060 ceph-mon[29694]: pgmap v406: 168 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 162 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 769 op/s; 13278/140688 objects misplaced (9.438%); 10 MiB/s, 170 objects/s recovering 2024-09-20T22:10:40.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:40 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:10:40.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:40 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:10:40.949 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:10:40 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:10:40.818+0000 7fa558495640 -1 mgr.server reply reply (3) No such process no change 2024-09-20T22:10:40.949 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:40 smithi060 ceph-mon[40290]: pgmap v406: 168 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 162 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 769 op/s; 13278/140688 objects misplaced (9.438%); 10 MiB/s, 170 objects/s recovering 2024-09-20T22:10:40.949 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:40 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:10:40.949 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:40 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:10:40.970 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:40.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:40.972 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:40.989 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:10:40.989 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:10:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:40 smithi067 ceph-mon[33511]: pgmap v406: 168 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 162 active+clean; 2.9 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 769 op/s; 13278/140688 objects misplaced (9.438%); 10 MiB/s, 170 objects/s recovering 2024-09-20T22:10:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:40 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:10:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:40 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:10:41.035 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:10:41.035 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:10:41.072 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:41.073 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:10:41.074 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:10:41.115 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:10:41.115 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:10:41.123 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:41.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:41.127 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:41.151 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:10:41.151 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:10:41.188 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:10:41.188 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:10:41.222 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:10:41.223 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:10:41.228 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:41.262 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:41.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:41.263 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:41.364 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:41.392 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:41.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:41.393 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:41.494 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:41.523 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:41.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:41.524 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:41.579 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-20T22:10:41.579 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight-by-utilization 110 3.0 2024-09-20T22:10:41.625 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:41.657 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:41.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:41.658 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:41.759 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:41.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:41.817 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:41.817 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:41.818 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:41.914 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:41 smithi067 ceph-mon[33511]: osdmap e257: 8 total, 8 up, 8 in 2024-09-20T22:10:41.914 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:41 smithi067 ceph-mon[33511]: from='client.25186 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:10:41.914 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:41 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3207090674' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:41.914 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:41 smithi067 ceph-mon[33511]: osdmap e258: 8 total, 8 up, 8 in 2024-09-20T22:10:41.918 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:41.950 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:41.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:41.951 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:42.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:41 smithi060 ceph-mon[29694]: osdmap e257: 8 total, 8 up, 8 in 2024-09-20T22:10:42.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:41 smithi060 ceph-mon[29694]: from='client.25186 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:10:42.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:41 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3207090674' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:42.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:41 smithi060 ceph-mon[29694]: osdmap e258: 8 total, 8 up, 8 in 2024-09-20T22:10:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:41 smithi060 ceph-mon[40290]: osdmap e257: 8 total, 8 up, 8 in 2024-09-20T22:10:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:41 smithi060 ceph-mon[40290]: from='client.25186 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:10:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:41 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3207090674' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:41 smithi060 ceph-mon[40290]: osdmap e258: 8 total, 8 up, 8 in 2024-09-20T22:10:42.052 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:42.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:42.097 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:42.098 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:42.198 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:42.234 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:42.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:42.236 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:42.336 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:42.379 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:42.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:42.380 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:42.481 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:42.513 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:10:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:10:42.514 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:42.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:42.515 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:42.616 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:42.652 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:42.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:42.653 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:42.662 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:10:42.754 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:42.815 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:42.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:42.816 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:42.917 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:42.958 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:42.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:42.960 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:42 smithi067 ceph-mon[33511]: pgmap v409: 168 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 162 active+clean; 2.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.28k op/s; 11346/126267 objects misplaced (8.986%); 18 MiB/s, 287 objects/s recovering 2024-09-20T22:10:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:42 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:10:43.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:42 smithi060 ceph-mon[29694]: pgmap v409: 168 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 162 active+clean; 2.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.28k op/s; 11346/126267 objects misplaced (8.986%); 18 MiB/s, 287 objects/s recovering 2024-09-20T22:10:43.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:42 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:10:43.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:42 smithi060 ceph-mon[40290]: pgmap v409: 168 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 162 active+clean; 2.6 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.28k op/s; 11346/126267 objects misplaced (8.986%); 18 MiB/s, 287 objects/s recovering 2024-09-20T22:10:43.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:42 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:10:43.061 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:43.096 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:43.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:43.097 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:43.197 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:43.223 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:43.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:43.224 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:43.324 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:43.369 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:43.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:43.370 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:43.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:43.498 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:43.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:43.499 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:43.599 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:43.636 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:43.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:43.637 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:43.738 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:43.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:43.779 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:43.779 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:43.880 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:43.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:43.918 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:43.919 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:44.019 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:44.059 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:44.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:44.060 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:44.161 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:44.187 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:44.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:44.188 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:44.289 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:44.312 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:44.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:44.313 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:44.414 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:44.439 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:44.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:44.440 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:44.540 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:44.563 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:44.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:44.564 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:44.665 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:44.687 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:44.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:44.688 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:44.788 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:44.810 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:44.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:44.811 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:44.912 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:44.993 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:44.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:44.995 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:45.095 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:45.128 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:45.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:45.129 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:45.230 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:45.259 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:45.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:45.260 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:45.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:44 smithi067 ceph-mon[33511]: osdmap e259: 8 total, 8 up, 8 in 2024-09-20T22:10:45.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:44 smithi067 ceph-mon[33511]: pgmap v411: 168 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 162 active+clean; 2.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 946 op/s; 10629/123666 objects misplaced (8.595%); 11 MiB/s, 182 objects/s recovering 2024-09-20T22:10:45.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:44 smithi060 ceph-mon[29694]: osdmap e259: 8 total, 8 up, 8 in 2024-09-20T22:10:45.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:44 smithi060 ceph-mon[29694]: pgmap v411: 168 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 162 active+clean; 2.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 946 op/s; 10629/123666 objects misplaced (8.595%); 11 MiB/s, 182 objects/s recovering 2024-09-20T22:10:45.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:10:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:10:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:10:45.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:44 smithi060 ceph-mon[40290]: osdmap e259: 8 total, 8 up, 8 in 2024-09-20T22:10:45.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:44 smithi060 ceph-mon[40290]: pgmap v411: 168 pgs: 2 active+remapped+backfilling, 4 active+remapped+backfill_toofull, 162 active+clean; 2.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 946 op/s; 10629/123666 objects misplaced (8.595%); 11 MiB/s, 182 objects/s recovering 2024-09-20T22:10:45.361 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:45.393 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:45.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:45.394 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:45.495 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:45.532 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:45.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:45.533 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:45.634 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:45.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:45.673 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:45.673 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:45.773 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:45.799 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:45.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:45.800 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:45.901 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:45.947 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:10:45.947 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:10:45.949 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:45.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:45.951 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:45.961 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:46.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:45 smithi060 ceph-mon[40290]: osdmap e260: 8 total, 8 up, 8 in 2024-09-20T22:10:46.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:45 smithi060 ceph-mon[29694]: osdmap e260: 8 total, 8 up, 8 in 2024-09-20T22:10:46.052 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:46.105 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:46.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:46.106 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:46.207 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:46.262 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:46.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:46.263 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:46.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:45 smithi067 ceph-mon[33511]: osdmap e260: 8 total, 8 up, 8 in 2024-09-20T22:10:46.364 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:46.398 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:46.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:46.400 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:46.500 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:46.539 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:46.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:46.540 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:46.640 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:46.680 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:46.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:46.683 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:46.784 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:46.822 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:46.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:46.823 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:46.914 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:10:46.989 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:47.020 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:47.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:47.021 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:47.033 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T22:10:47.122 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:47.151 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:46 smithi060 ceph-mon[29694]: pgmap v413: 168 pgs: 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 162 active+clean; 2.4 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 956 B/s rd, 0 B/s wr, 1.25k op/s; 9698/117426 objects misplaced (8.259%); 21 MiB/s, 333 objects/s recovering 2024-09-20T22:10:47.151 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:46 smithi060 ceph-mon[29694]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:10:47.151 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:46 smithi060 ceph-mon[29694]: osdmap e261: 8 total, 8 up, 8 in 2024-09-20T22:10:47.151 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:46 smithi060 ceph-mon[40290]: pgmap v413: 168 pgs: 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 162 active+clean; 2.4 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 956 B/s rd, 0 B/s wr, 1.25k op/s; 9698/117426 objects misplaced (8.259%); 21 MiB/s, 333 objects/s recovering 2024-09-20T22:10:47.151 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:46 smithi060 ceph-mon[40290]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:10:47.152 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:46 smithi060 ceph-mon[40290]: osdmap e261: 8 total, 8 up, 8 in 2024-09-20T22:10:47.155 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:47.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:47.156 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:47.257 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:47.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:46 smithi067 ceph-mon[33511]: pgmap v413: 168 pgs: 3 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 162 active+clean; 2.4 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 956 B/s rd, 0 B/s wr, 1.25k op/s; 9698/117426 objects misplaced (8.259%); 21 MiB/s, 333 objects/s recovering 2024-09-20T22:10:47.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:46 smithi067 ceph-mon[33511]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 4 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:10:47.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:46 smithi067 ceph-mon[33511]: osdmap e261: 8 total, 8 up, 8 in 2024-09-20T22:10:47.308 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:47.317 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:47.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:47.318 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:47.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:47.465 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:47.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:47.466 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:47.567 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:47.628 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:47.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:47.629 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:47.730 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:47.766 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:47.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:47.767 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:47.868 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:47.894 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:47.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:47.895 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:47.895 INFO:teuthology.orchestra.run.smithi060.stdout:moved 20 / 504 (3.96825%) 2024-09-20T22:10:47.895 INFO:teuthology.orchestra.run.smithi060.stdout:avg 63 2024-09-20T22:10:47.895 INFO:teuthology.orchestra.run.smithi060.stdout:stddev 9.08295 -> 12.1244 (expected baseline 7.42462) 2024-09-20T22:10:47.895 INFO:teuthology.orchestra.run.smithi060.stdout:min osd.5 with 46 -> 34 pgs (0.730159 -> 0.539683 * mean) 2024-09-20T22:10:47.896 INFO:teuthology.orchestra.run.smithi060.stdout:max osd.4 with 73 -> 74 pgs (1.15873 -> 1.1746 * mean) 2024-09-20T22:10:47.896 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:10:47.896 INFO:teuthology.orchestra.run.smithi060.stdout:oload 110 2024-09-20T22:10:47.896 INFO:teuthology.orchestra.run.smithi060.stdout:max_change 3 2024-09-20T22:10:47.896 INFO:teuthology.orchestra.run.smithi060.stdout:max_change_osds 4 2024-09-20T22:10:47.896 INFO:teuthology.orchestra.run.smithi060.stdout:average_utilization 0.0147 2024-09-20T22:10:47.896 INFO:teuthology.orchestra.run.smithi060.stdout:overload_utilization 0.0161 2024-09-20T22:10:47.896 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 weight 0.6893 -> 0.5011 2024-09-20T22:10:47.896 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 weight 0.8012 -> 0.8944 2024-09-20T22:10:47.996 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:48.024 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:48.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:48.025 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:48.125 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:48.156 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:48.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:48.157 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:48.257 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:48.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:47 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:10:48.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:47 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:10:48.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:47 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'5':'32839','7':'58614'}"}]: dispatch 2024-09-20T22:10:48.285 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:48.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:47 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:10:48.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:47 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:10:48.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:47 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'5':'32839','7':'58614'}"}]: dispatch 2024-09-20T22:10:48.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:48.287 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:48.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:47 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:10:48.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:47 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:10:48.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:47 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'5':'32839','7':'58614'}"}]: dispatch 2024-09-20T22:10:48.388 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:48.413 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:48.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:48.413 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:48.514 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:48.536 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:48.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:48.537 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:48.637 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:48.663 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:48.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:48.664 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:48.764 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:48.787 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:48.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:48.787 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:48.888 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:48.907 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:10:48.920 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:48.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:48.921 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:48.981 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-20T22:10:48.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd test-reweight-by-utilization 110 0.05 2024-09-20T22:10:49.022 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:49.046 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:49.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:49.047 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:49.147 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:49.170 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:49.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:49.171 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:49.172 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:49.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:48 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:10:49.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:48 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:10:49.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:48 smithi067 ceph-mon[33511]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:10:49.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:48 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'5':'32839','7':'58614'}"}]': finished 2024-09-20T22:10:49.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:48 smithi067 ceph-mon[33511]: osdmap e262: 8 total, 8 up, 8 in 2024-09-20T22:10:49.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:48 smithi067 ceph-mon[33511]: pgmap v416: 168 pgs: 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 163 active+clean; 2.2 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.07k op/s; 7608/110001 objects misplaced (6.916%); 29 MiB/s, 461 objects/s recovering 2024-09-20T22:10:49.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:48 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/998290173' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:10:49.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:48 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:10:49.265 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:48 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:10:49.265 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:48 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:10:49.266 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:48 smithi060 ceph-mon[29694]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:10:49.266 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:48 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'5':'32839','7':'58614'}"}]': finished 2024-09-20T22:10:49.266 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:48 smithi060 ceph-mon[29694]: osdmap e262: 8 total, 8 up, 8 in 2024-09-20T22:10:49.266 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:48 smithi060 ceph-mon[29694]: pgmap v416: 168 pgs: 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 163 active+clean; 2.2 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.07k op/s; 7608/110001 objects misplaced (6.916%); 29 MiB/s, 461 objects/s recovering 2024-09-20T22:10:49.266 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:48 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/998290173' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:10:49.266 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:48 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:10:49.266 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:48 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:10:49.266 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:48 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:10:49.266 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:48 smithi060 ceph-mon[40290]: from='client.15432 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:10:49.266 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:48 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'5':'32839','7':'58614'}"}]': finished 2024-09-20T22:10:49.266 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:48 smithi060 ceph-mon[40290]: osdmap e262: 8 total, 8 up, 8 in 2024-09-20T22:10:49.267 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:48 smithi060 ceph-mon[40290]: pgmap v416: 168 pgs: 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 163 active+clean; 2.2 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.07k op/s; 7608/110001 objects misplaced (6.916%); 29 MiB/s, 461 objects/s recovering 2024-09-20T22:10:49.267 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:48 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/998290173' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:10:49.267 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:48 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:10:49.272 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:49.294 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:49.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:49.295 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:49.396 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:49.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:49.419 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:49.419 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:49.520 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:49.543 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:49.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:49.544 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:49.645 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:49.672 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:49.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:49.673 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:49.773 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:49.796 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:49.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:49.797 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:49.881 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:49.898 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:49.944 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:49.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:49.945 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:50.045 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:50.074 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:50.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:50.075 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:50.175 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:50.205 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:50.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:50.206 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:50.212 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:49 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:10:50.212 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:49 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:10:50.212 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:49 smithi060 ceph-mon[29694]: osdmap e263: 8 total, 8 up, 8 in 2024-09-20T22:10:50.214 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:49 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:10:50.214 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:49 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:10:50.214 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:49 smithi060 ceph-mon[40290]: osdmap e263: 8 total, 8 up, 8 in 2024-09-20T22:10:50.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:49 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:10:50.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:49 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:10:50.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:49 smithi067 ceph-mon[33511]: osdmap e263: 8 total, 8 up, 8 in 2024-09-20T22:10:50.306 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:50.334 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:50.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:50.335 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:50.436 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:50.484 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:50.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:50.485 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:50.586 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:50.639 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:50.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:50.640 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:50.741 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:50.771 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:50.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:50.772 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:50.873 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:50.874 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:10:50.874 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:10:50.909 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:10:50.909 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:10:50.909 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:10:50.909 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:10:50.910 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:10:50.910 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:10:50.912 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:50.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:50.913 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:50.953 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:10:50.954 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:10:50.991 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:10:50.992 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:10:51.014 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:51.025 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:10:51.026 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:10:51.046 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:51.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:51.047 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:51.071 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:10:51.071 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:10:51.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:50 smithi060 ceph-mon[29694]: osdmap e264: 8 total, 8 up, 8 in 2024-09-20T22:10:51.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:50 smithi060 ceph-mon[29694]: 8.8 starting backfill to osd.0 from (0'0,0'0] MAX to 261'9803 2024-09-20T22:10:51.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:50 smithi060 ceph-mon[29694]: pgmap v419: 168 pgs: 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 163 active+clean; 2.2 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 0 B/s wr, 618 op/s; 7608/110001 objects misplaced (6.916%); 20 MiB/s, 318 objects/s recovering 2024-09-20T22:10:51.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:50 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:10:51.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:50 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:10:51.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:50 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3388787354' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:51.072 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:50 smithi060 ceph-mon[40290]: osdmap e264: 8 total, 8 up, 8 in 2024-09-20T22:10:51.072 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:50 smithi060 ceph-mon[40290]: 8.8 starting backfill to osd.0 from (0'0,0'0] MAX to 261'9803 2024-09-20T22:10:51.072 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:50 smithi060 ceph-mon[40290]: pgmap v419: 168 pgs: 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 163 active+clean; 2.2 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 0 B/s wr, 618 op/s; 7608/110001 objects misplaced (6.916%); 20 MiB/s, 318 objects/s recovering 2024-09-20T22:10:51.072 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:50 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:10:51.072 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:50 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:10:51.073 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:50 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3388787354' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:51.106 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:10:51.106 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:10:51.147 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:51.180 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:51.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:51.181 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:51.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:50 smithi067 ceph-mon[33511]: osdmap e264: 8 total, 8 up, 8 in 2024-09-20T22:10:51.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:50 smithi067 ceph-mon[33511]: 8.8 starting backfill to osd.0 from (0'0,0'0] MAX to 261'9803 2024-09-20T22:10:51.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:50 smithi067 ceph-mon[33511]: pgmap v419: 168 pgs: 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 163 active+clean; 2.2 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 0 B/s wr, 618 op/s; 7608/110001 objects misplaced (6.916%); 20 MiB/s, 318 objects/s recovering 2024-09-20T22:10:51.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:50 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:10:51.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:50 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:10:51.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:50 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3388787354' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:51.282 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:51.320 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:51.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:51.321 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:51.422 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:51.470 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:51.470 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:51.471 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:51.571 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:51.611 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:51.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:51.612 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:51.713 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:51.750 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:51.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:51.751 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:51.852 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:51.899 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:51.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:51.900 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:51.929 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:10:52.001 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:52.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:52.036 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:52.036 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:52.137 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:52.177 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:51 smithi067 ceph-mon[33511]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:10:52.177 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:51 smithi067 ceph-mon[33511]: osdmap e265: 8 total, 8 up, 8 in 2024-09-20T22:10:52.193 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:52.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:52.194 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:52.245 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:51 smithi060 ceph-mon[29694]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:10:52.245 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:51 smithi060 ceph-mon[29694]: osdmap e265: 8 total, 8 up, 8 in 2024-09-20T22:10:52.245 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:51 smithi060 ceph-mon[40290]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 3 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:10:52.245 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:51 smithi060 ceph-mon[40290]: osdmap e265: 8 total, 8 up, 8 in 2024-09-20T22:10:52.295 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:52.307 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T22:10:52.342 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:52.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:52.343 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:52.444 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:52.492 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:52.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:52.493 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:52.513 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:10:52 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:10:52.594 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:52.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:52.647 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:52.648 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:52.705 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:10:52.749 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:52.794 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:52.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:52.795 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:52.895 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:52.935 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:52.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:52.936 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:52.973 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:53.037 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:53.046 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:53.101 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:53.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:53.102 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:53.168 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:52 smithi060 ceph-mon[29694]: pgmap v421: 168 pgs: 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 163 active+clean; 2.0 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 926 op/s; 7333/100659 objects misplaced (7.285%); 24 MiB/s, 395 objects/s recovering 2024-09-20T22:10:53.168 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:52 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:10:53.168 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:52 smithi060 ceph-mon[40290]: pgmap v421: 168 pgs: 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 163 active+clean; 2.0 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 926 op/s; 7333/100659 objects misplaced (7.285%); 24 MiB/s, 395 objects/s recovering 2024-09-20T22:10:53.168 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:52 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:10:53.174 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:52 smithi067 ceph-mon[33511]: pgmap v421: 168 pgs: 2 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 163 active+clean; 2.0 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 926 op/s; 7333/100659 objects misplaced (7.285%); 24 MiB/s, 395 objects/s recovering 2024-09-20T22:10:53.174 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:52 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:10:53.203 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:53.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:53.248 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:53.248 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:53.341 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:10:53.342 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:10:53.342 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.342 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:209643 8.2 8:478acf5b:::benchmark_data_smithi060_111051_object89644:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e265)", 2024-09-20T22:10:53.342 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:53.329903+0000", 2024-09-20T22:10:53.342 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0088510359999999996, 2024-09-20T22:10:53.342 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00887895, 2024-09-20T22:10:53.342 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:53.342 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:53.342 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:10:53.342 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:10:53.342 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:10:53.342 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:53.342 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 209643 2024-09-20T22:10:53.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.343 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:53.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.343 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:53.343 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.329903+0000", 2024-09-20T22:10:53.343 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:53.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.343 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:53.343 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.329903+0000", 2024-09-20T22:10:53.343 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:53.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.344 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:53.344 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.329905+0000", 2024-09-20T22:10:53.344 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7039999999999999e-06 2024-09-20T22:10:53.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.344 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:53.344 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.329911+0000", 2024-09-20T22:10:53.344 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7450000000000001e-06 2024-09-20T22:10:53.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.344 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:53.344 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.329928+0000", 2024-09-20T22:10:53.345 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7368999999999999e-05 2024-09-20T22:10:53.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.345 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.345 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:53.345 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.329936+0000", 2024-09-20T22:10:53.345 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5979999999999999e-06 2024-09-20T22:10:53.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.345 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.345 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:53.345 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.329994+0000", 2024-09-20T22:10:53.345 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8142e-05 2024-09-20T22:10:53.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.345 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.345 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:10:53.346 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.330101+0000", 2024-09-20T22:10:53.346 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000107237 2024-09-20T22:10:53.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.346 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,5", 2024-09-20T22:10:53.346 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.330458+0000", 2024-09-20T22:10:53.346 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00035745800000000001 2024-09-20T22:10:53.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.346 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:10:53.346 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.332626+0000", 2024-09-20T22:10:53.346 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0021679410000000001 2024-09-20T22:10:53.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.347 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:53.347 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.334625+0000", 2024-09-20T22:10:53.347 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0019989209999999999 2024-09-20T22:10:53.347 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:53.347 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:53.347 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:53.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.347 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.15192.0:209649 8.b e265/176 8:da261830:::benchmark_data_smithi060_111051_object89650:head v 265'12823, mlcod=265'12821)", 2024-09-20T22:10:53.348 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:53.337793+0000", 2024-09-20T22:10:53.348 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.000961706, 2024-09-20T22:10:53.348 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0010846549999999999, 2024-09-20T22:10:53.348 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:53.348 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:53.348 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:10:53.348 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:53.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.348 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:53.348 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.337793+0000", 2024-09-20T22:10:53.348 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:53.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.349 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:53.349 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.337793+0000", 2024-09-20T22:10:53.349 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:53.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.349 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:53.349 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.337793+0000", 2024-09-20T22:10:53.349 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8599999999999997e-07 2024-09-20T22:10:53.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.349 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:53.349 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.337799+0000", 2024-09-20T22:10:53.350 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0299999999999999e-06 2024-09-20T22:10:53.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.350 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:53.350 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.337809+0000", 2024-09-20T22:10:53.350 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4420000000000004e-06 2024-09-20T22:10:53.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.350 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:53.350 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.337816+0000", 2024-09-20T22:10:53.350 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7159999999999997e-06 2024-09-20T22:10:53.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.350 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:53.351 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.337904+0000", 2024-09-20T22:10:53.351 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8844999999999994e-05 2024-09-20T22:10:53.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.351 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:10:53.351 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.337925+0000", 2024-09-20T22:10:53.351 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0554999999999999e-05 2024-09-20T22:10:53.351 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:53.351 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:53.351 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:53.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.351 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:209652 8.14 8.f7333a14 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e265)", 2024-09-20T22:10:53.351 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:53.338880+0000", 2024-09-20T22:10:53.351 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 4294967295.9998741, 2024-09-20T22:10:53.352 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0893000000000001e-05, 2024-09-20T22:10:53.352 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:53.352 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:53.352 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "queued for pg", 2024-09-20T22:10:53.352 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:10:53.352 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:10:53.352 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:53.352 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 209652 2024-09-20T22:10:53.352 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.352 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:53.352 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.352 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:53.352 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.338880+0000", 2024-09-20T22:10:53.352 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:53.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.353 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.353 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:53.353 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.338880+0000", 2024-09-20T22:10:53.353 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:53.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.353 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.353 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:53.353 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.338883+0000", 2024-09-20T22:10:53.353 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.204e-06 2024-09-20T22:10:53.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.353 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.353 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:53.353 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.338888+0000", 2024-09-20T22:10:53.353 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9030000000000001e-06 2024-09-20T22:10:53.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.354 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.354 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:53.354 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.338907+0000", 2024-09-20T22:10:53.354 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8943000000000001e-05 2024-09-20T22:10:53.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:53.354 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:53.354 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:53.354 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:53.338916+0000", 2024-09-20T22:10:53.354 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9060000000000008e-06 2024-09-20T22:10:53.354 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:53.354 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:53.354 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:53.354 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:53.354 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-20T22:10:53.355 INFO:teuthology.orchestra.run.smithi067.stdout: "num_ops": 3 2024-09-20T22:10:53.355 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:10:53.355 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:53.391 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:53.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:53.392 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:53.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:53.524 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:53.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:53.525 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:53.626 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:53.655 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:53.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:53.656 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:53.756 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:53.792 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:53.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:53.793 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:53.894 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:53.936 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:53.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:53.937 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:54.038 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:54.069 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:54.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:54.071 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:54.165 INFO:teuthology.orchestra.run.smithi060.stdout:moved 6 / 504 (1.19048%) 2024-09-20T22:10:54.166 INFO:teuthology.orchestra.run.smithi060.stdout:avg 63 2024-09-20T22:10:54.166 INFO:teuthology.orchestra.run.smithi060.stdout:stddev 12.1244 -> 13.2571 (expected baseline 7.42462) 2024-09-20T22:10:54.166 INFO:teuthology.orchestra.run.smithi060.stdout:min osd.5 with 34 -> 31 pgs (0.539683 -> 0.492063 * mean) 2024-09-20T22:10:54.166 INFO:teuthology.orchestra.run.smithi060.stdout:max osd.4 with 74 -> 75 pgs (1.1746 -> 1.19048 * mean) 2024-09-20T22:10:54.166 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:10:54.166 INFO:teuthology.orchestra.run.smithi060.stdout:oload 110 2024-09-20T22:10:54.166 INFO:teuthology.orchestra.run.smithi060.stdout:max_change 0.05 2024-09-20T22:10:54.166 INFO:teuthology.orchestra.run.smithi060.stdout:max_change_osds 4 2024-09-20T22:10:54.166 INFO:teuthology.orchestra.run.smithi060.stdout:average_utilization 0.0134 2024-09-20T22:10:54.166 INFO:teuthology.orchestra.run.smithi060.stdout:overload_utilization 0.0147 2024-09-20T22:10:54.167 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5 weight 0.5011 -> 0.4511 2024-09-20T22:10:54.167 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7 weight 0.8944 -> 0.9444 2024-09-20T22:10:54.167 INFO:teuthology.orchestra.run.smithi060.stderr:no change 2024-09-20T22:10:54.171 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:54.200 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:54.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:54.201 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:54.302 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:54.331 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:54.331 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:54.332 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:54.433 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:54.461 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:54.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:54.462 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:54.536 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:10:54 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:10:54.162+0000 7fa558495640 -1 mgr.server reply reply (2) No such file or directory no change 2024-09-20T22:10:54.563 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:54.607 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:54.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:54.608 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:54.708 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:54.740 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:54.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:54.741 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:54.841 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:54.887 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:54.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:54.888 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:54.974 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T22:10:54.988 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:55.030 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:10:55.064 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:55.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:55.065 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:55.165 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:55.167 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:10:55.188 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:55.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:55.189 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:55.191 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:10:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:10:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:10:55.191 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:55 smithi060 ceph-mon[29694]: pgmap v422: 168 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 164 active+clean; 2.0 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 623 op/s; 6954/98781 objects misplaced (7.040%); 11 MiB/s, 172 objects/s recovering 2024-09-20T22:10:55.192 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:55 smithi060 ceph-mon[40290]: pgmap v422: 168 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 164 active+clean; 2.0 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 623 op/s; 6954/98781 objects misplaced (7.040%); 11 MiB/s, 172 objects/s recovering 2024-09-20T22:10:55.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:55 smithi067 ceph-mon[33511]: pgmap v422: 168 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 164 active+clean; 2.0 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 623 op/s; 6954/98781 objects misplaced (7.040%); 11 MiB/s, 172 objects/s recovering 2024-09-20T22:10:55.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:55 smithi067 ceph-mon[33511]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:10:55.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1015554666' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:10:55.290 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:55.334 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:55.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:55.335 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:55.348 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:10:55.435 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:55.471 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:55.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:55.472 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:55.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:55 smithi060 ceph-mon[29694]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:10:55.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:55 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1015554666' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:10:55.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:55 smithi060 ceph-mon[40290]: from='client.15447 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:10:55.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:55 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1015554666' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:10:55.572 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:10:55.610 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:55.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:55.611 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:10:55.711 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:55.758 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:55.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:55.759 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:55.860 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:55.891 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:55.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:55.893 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:55.993 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:56.038 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:56.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:56.039 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:56.048 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:10:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:10:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "complaint_time": 30, 2024-09-20T22:10:56.049 INFO:teuthology.orchestra.run.smithi067.stdout: "num_blocked_ops": 0 2024-09-20T22:10:56.049 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:10:56.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:56.171 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:56.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:56.172 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:56.273 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:56.307 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:56.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:56.309 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:56.409 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:56.436 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:56 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:10:56.436 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:56 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1015554666' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:10:56.436 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:56 smithi067 ceph-mon[33511]: osdmap e266: 8 total, 8 up, 8 in 2024-09-20T22:10:56.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:56.443 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:56.443 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:56.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:56 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:10:56.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:56 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1015554666' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:10:56.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:56 smithi060 ceph-mon[29694]: osdmap e266: 8 total, 8 up, 8 in 2024-09-20T22:10:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:56 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:10:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:56 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1015554666' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:10:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:56 smithi060 ceph-mon[40290]: osdmap e266: 8 total, 8 up, 8 in 2024-09-20T22:10:56.544 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:56.565 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:56.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:56.566 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:56.667 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:56.710 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:56.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:56.711 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:56.808 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T22:10:56.812 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:56.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:56.890 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:56.891 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:56.992 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:57.013 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:57.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:57.014 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:57.115 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:57.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:57.216 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:57.216 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:57.216 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:10:57.317 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:57.352 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:57.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:57.353 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:57.454 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:57.487 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:57.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:57.488 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:57.581 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:57 smithi067 ceph-mon[33511]: pgmap v424: 168 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 164 active+clean; 1.8 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.02k op/s; 6772/91065 objects misplaced (7.436%); 10 MiB/s, 166 objects/s recovering 2024-09-20T22:10:57.581 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:57 smithi067 ceph-mon[33511]: 1.1 scrub starts 2024-09-20T22:10:57.581 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:57 smithi067 ceph-mon[33511]: 1.1 scrub ok 2024-09-20T22:10:57.582 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:57 smithi067 ceph-mon[33511]: 6.a scrub starts 2024-09-20T22:10:57.582 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:57 smithi067 ceph-mon[33511]: 6.a scrub ok 2024-09-20T22:10:57.582 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:57 smithi067 ceph-mon[33511]: 8.15 scrub starts 2024-09-20T22:10:57.582 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:57 smithi067 ceph-mon[33511]: osdmap e267: 8 total, 8 up, 8 in 2024-09-20T22:10:57.582 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:57 smithi067 ceph-mon[33511]: 8.15 scrub ok 2024-09-20T22:10:57.589 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:10:57.620 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:57.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:57.621 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:10:57.722 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:57.749 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:57.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:57.750 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:57 smithi060 ceph-mon[29694]: pgmap v424: 168 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 164 active+clean; 1.8 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.02k op/s; 6772/91065 objects misplaced (7.436%); 10 MiB/s, 166 objects/s recovering 2024-09-20T22:10:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:57 smithi060 ceph-mon[29694]: 1.1 scrub starts 2024-09-20T22:10:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:57 smithi060 ceph-mon[29694]: 1.1 scrub ok 2024-09-20T22:10:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:57 smithi060 ceph-mon[29694]: 6.a scrub starts 2024-09-20T22:10:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:57 smithi060 ceph-mon[29694]: 6.a scrub ok 2024-09-20T22:10:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:57 smithi060 ceph-mon[29694]: 8.15 scrub starts 2024-09-20T22:10:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:57 smithi060 ceph-mon[29694]: osdmap e267: 8 total, 8 up, 8 in 2024-09-20T22:10:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:57 smithi060 ceph-mon[29694]: 8.15 scrub ok 2024-09-20T22:10:57.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:57 smithi060 ceph-mon[40290]: pgmap v424: 168 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 164 active+clean; 1.8 GiB data, 9.0 GiB used, 706 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1.02k op/s; 6772/91065 objects misplaced (7.436%); 10 MiB/s, 166 objects/s recovering 2024-09-20T22:10:57.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:57 smithi060 ceph-mon[40290]: 1.1 scrub starts 2024-09-20T22:10:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:57 smithi060 ceph-mon[40290]: 1.1 scrub ok 2024-09-20T22:10:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:57 smithi060 ceph-mon[40290]: 6.a scrub starts 2024-09-20T22:10:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:57 smithi060 ceph-mon[40290]: 6.a scrub ok 2024-09-20T22:10:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:57 smithi060 ceph-mon[40290]: 8.15 scrub starts 2024-09-20T22:10:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:57 smithi060 ceph-mon[40290]: osdmap e267: 8 total, 8 up, 8 in 2024-09-20T22:10:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:57 smithi060 ceph-mon[40290]: 8.15 scrub ok 2024-09-20T22:10:57.851 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:57.853 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:10:57.853 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:10:57.853 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:10:57.853 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:10:57.853 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.853 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169956 7.1 7:8584ed68:::benchmark_data_smithi060_78800_object36923:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-20T22:10:57.853 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.419665+0000", 2024-09-20T22:10:57.853 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 299.43097654399998, 2024-09-20T22:10:57.854 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.18189190799999999, 2024-09-20T22:10:57.854 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.854 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.854 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:10:57.854 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:10:57.854 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:10:57.854 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:57.854 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169956 2024-09-20T22:10:57.854 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.854 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.855 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.855 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.855 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.419665+0000", 2024-09-20T22:10:57.855 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.855 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.855 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.855 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.855 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.419665+0000", 2024-09-20T22:10:57.855 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.855 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.855 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.855 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.856 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.419665+0000", 2024-09-20T22:10:57.856 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9200000000000001e-07 2024-09-20T22:10:57.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.856 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.856 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.856 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.419668+0000", 2024-09-20T22:10:57.856 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0419999999999998e-06 2024-09-20T22:10:57.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.856 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.856 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.856 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.419676+0000", 2024-09-20T22:10:57.856 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0080000000000002e-06 2024-09-20T22:10:57.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.857 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.857 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.857 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.419682+0000", 2024-09-20T22:10:57.857 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4009999999999996e-06 2024-09-20T22:10:57.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.857 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.857 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.857 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.423075+0000", 2024-09-20T22:10:57.857 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0033931629999999998 2024-09-20T22:10:57.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.857 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.857 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:10:57.857 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.423660+0000", 2024-09-20T22:10:57.858 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00058539699999999998 2024-09-20T22:10:57.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.858 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.858 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:10:57.858 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.424077+0000", 2024-09-20T22:10:57.858 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00041686199999999999 2024-09-20T22:10:57.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.858 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.858 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.858 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.429379+0000", 2024-09-20T22:10:57.858 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0053023300000000001 2024-09-20T22:10:57.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.858 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.858 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:10:57.859 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.538523+0000", 2024-09-20T22:10:57.859 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.10914377 2024-09-20T22:10:57.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.859 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.859 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.859 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601414+0000", 2024-09-20T22:10:57.859 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.062890450000000001 2024-09-20T22:10:57.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.859 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.859 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.859 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601556+0000", 2024-09-20T22:10:57.859 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000142893 2024-09-20T22:10:57.859 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.860 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.860 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.860 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.860 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.860 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.24667.0:169951 7.3 e120/95 7:c27f31c3:::benchmark_data_smithi060_78800_object36918:head v 120'10582, mlcod=120'10579)", 2024-09-20T22:10:57.860 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.423110+0000", 2024-09-20T22:10:57.860 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 299.42753075799999, 2024-09-20T22:10:57.860 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.14038883999999999, 2024-09-20T22:10:57.860 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.860 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.860 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:10:57.860 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.860 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.860 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.861 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.423110+0000", 2024-09-20T22:10:57.861 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.861 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.861 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.861 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.423110+0000", 2024-09-20T22:10:57.861 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.861 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.861 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.861 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.423111+0000", 2024-09-20T22:10:57.861 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7899999999999998e-07 2024-09-20T22:10:57.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.861 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.862 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.862 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.423117+0000", 2024-09-20T22:10:57.862 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1890000000000004e-06 2024-09-20T22:10:57.862 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.862 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.862 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.862 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.423127+0000", 2024-09-20T22:10:57.862 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9569999999999997e-06 2024-09-20T22:10:57.862 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.862 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.862 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.862 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.423134+0000", 2024-09-20T22:10:57.862 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7970000000000003e-06 2024-09-20T22:10:57.862 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.863 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.863 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.863 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.423485+0000", 2024-09-20T22:10:57.863 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00035125599999999999 2024-09-20T22:10:57.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.863 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.863 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:10:57.863 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.423508+0000", 2024-09-20T22:10:57.863 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2612999999999998e-05 2024-09-20T22:10:57.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.863 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.863 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.863 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.563499+0000", 2024-09-20T22:10:57.863 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.13999134899999999 2024-09-20T22:10:57.864 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.864 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.864 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.864 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.864 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.864 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.24667.0:169959 7.9 e120/91 7:9aa79aaf:::benchmark_data_smithi060_78800_object36926:head v 120'10552, mlcod=120'10549)", 2024-09-20T22:10:57.864 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.424269+0000", 2024-09-20T22:10:57.864 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 299.42637194899999, 2024-09-20T22:10:57.864 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.176774809, 2024-09-20T22:10:57.864 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.864 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.864 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:10:57.864 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.864 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.865 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.865 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.424269+0000", 2024-09-20T22:10:57.865 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.865 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.865 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.865 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.865 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.424269+0000", 2024-09-20T22:10:57.865 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.865 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.865 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.865 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.865 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.424270+0000", 2024-09-20T22:10:57.865 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8599999999999997e-07 2024-09-20T22:10:57.865 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.865 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.866 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.866 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.424277+0000", 2024-09-20T22:10:57.866 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1149999999999996e-06 2024-09-20T22:10:57.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.866 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.866 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.866 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.424292+0000", 2024-09-20T22:10:57.866 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4901e-05 2024-09-20T22:10:57.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.866 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.866 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.866 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.424300+0000", 2024-09-20T22:10:57.866 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6229999999999997e-06 2024-09-20T22:10:57.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.867 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.867 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.867 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.427901+0000", 2024-09-20T22:10:57.867 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0036014150000000002 2024-09-20T22:10:57.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.867 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.867 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:10:57.867 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.427924+0000", 2024-09-20T22:10:57.867 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2529000000000001e-05 2024-09-20T22:10:57.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.867 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.867 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.869 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601044+0000", 2024-09-20T22:10:57.870 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.17312043999999999 2024-09-20T22:10:57.870 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.870 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.870 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.870 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.870 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.870 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.24667.0:169961 7.3 e120/95 7:cf59cdb9:::benchmark_data_smithi060_78800_object36928:head v 120'10583, mlcod=120'10579)", 2024-09-20T22:10:57.870 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.428621+0000", 2024-09-20T22:10:57.870 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 299.422020116, 2024-09-20T22:10:57.870 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.13496223099999999, 2024-09-20T22:10:57.870 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.870 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.871 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:10:57.871 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.871 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.871 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.871 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.428621+0000", 2024-09-20T22:10:57.871 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.871 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.871 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.871 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.871 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.428621+0000", 2024-09-20T22:10:57.871 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.871 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.871 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.871 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.872 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.428622+0000", 2024-09-20T22:10:57.872 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4199999999999995e-07 2024-09-20T22:10:57.872 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.872 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.872 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.872 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.428628+0000", 2024-09-20T22:10:57.872 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2539999999999998e-06 2024-09-20T22:10:57.872 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.872 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.872 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.872 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.428641+0000", 2024-09-20T22:10:57.872 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2513e-05 2024-09-20T22:10:57.872 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.873 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.873 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.873 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.428658+0000", 2024-09-20T22:10:57.873 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7402e-05 2024-09-20T22:10:57.873 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.873 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.873 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.873 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.429707+0000", 2024-09-20T22:10:57.873 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0010490059999999999 2024-09-20T22:10:57.873 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.873 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.873 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:10:57.874 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.429725+0000", 2024-09-20T22:10:57.874 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8558000000000001e-05 2024-09-20T22:10:57.874 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.874 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.874 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.874 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.563583+0000", 2024-09-20T22:10:57.874 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.13385775599999999 2024-09-20T22:10:57.874 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.874 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.874 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.874 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.874 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.874 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop_reply(client.24667.0:169956 7.1 e120/80)", 2024-09-20T22:10:57.875 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.430313+0000", 2024-09-20T22:10:57.875 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 299.42032796900003, 2024-09-20T22:10:57.875 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.17075412600000001, 2024-09-20T22:10:57.875 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.875 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.875 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:10:57.875 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.875 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.875 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.875 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430313+0000", 2024-09-20T22:10:57.875 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.875 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.875 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.875 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.875 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430313+0000", 2024-09-20T22:10:57.876 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.876 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.876 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.876 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.876 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430314+0000", 2024-09-20T22:10:57.876 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2600000000000002e-07 2024-09-20T22:10:57.876 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.876 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.876 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.876 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430318+0000", 2024-09-20T22:10:57.876 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2799999999999997e-06 2024-09-20T22:10:57.876 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.876 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.876 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.877 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430327+0000", 2024-09-20T22:10:57.877 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6780000000000006e-06 2024-09-20T22:10:57.877 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.877 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.877 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.877 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430332+0000", 2024-09-20T22:10:57.877 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5759999999999996e-06 2024-09-20T22:10:57.877 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.877 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.877 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.877 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601046+0000", 2024-09-20T22:10:57.877 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.17071423699999999 2024-09-20T22:10:57.877 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.877 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.878 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.878 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601067+0000", 2024-09-20T22:10:57.878 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0829e-05 2024-09-20T22:10:57.878 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.878 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.878 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.878 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.878 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.878 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169965 7.1 7.31051ab1 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e120)", 2024-09-20T22:10:57.878 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.430602+0000", 2024-09-20T22:10:57.878 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 299.42003900499998, 2024-09-20T22:10:57.878 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.17100728100000001, 2024-09-20T22:10:57.878 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.879 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.879 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:10:57.879 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:10:57.879 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:10:57.879 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:57.879 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169965 2024-09-20T22:10:57.879 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.879 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.879 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.879 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.879 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430602+0000", 2024-09-20T22:10:57.879 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.879 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.879 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.879 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.880 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430602+0000", 2024-09-20T22:10:57.880 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.880 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.880 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.880 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.880 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430604+0000", 2024-09-20T22:10:57.880 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7069999999999999e-06 2024-09-20T22:10:57.880 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.880 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.880 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.880 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430609+0000", 2024-09-20T22:10:57.880 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3900000000000001e-06 2024-09-20T22:10:57.880 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.880 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.880 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.881 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430622+0000", 2024-09-20T22:10:57.881 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2407e-05 2024-09-20T22:10:57.881 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.881 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.881 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.881 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.430630+0000", 2024-09-20T22:10:57.881 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2260000000000005e-06 2024-09-20T22:10:57.881 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.881 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.881 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.881 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601598+0000", 2024-09-20T22:10:57.881 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.17096821800000001 2024-09-20T22:10:57.881 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.881 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.882 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.882 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601609+0000", 2024-09-20T22:10:57.882 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1333e-05 2024-09-20T22:10:57.882 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.882 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.882 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.882 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.882 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.882 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169955 7.1 7:85b7de5b:::benchmark_data_smithi060_78800_object36922:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:10:57.882 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.440021+0000", 2024-09-20T22:10:57.882 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 299.41062021300002, 2024-09-20T22:10:57.882 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.038197153, 2024-09-20T22:10:57.882 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.882 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.883 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:10:57.883 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:10:57.883 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:10:57.883 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:57.883 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169955 2024-09-20T22:10:57.883 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.883 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.883 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.883 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.883 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440021+0000", 2024-09-20T22:10:57.883 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.883 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.883 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.883 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.883 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440021+0000", 2024-09-20T22:10:57.884 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.884 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.884 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.884 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.884 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440025+0000", 2024-09-20T22:10:57.884 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4429999999999996e-06 2024-09-20T22:10:57.884 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.884 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.884 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.884 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440034+0000", 2024-09-20T22:10:57.884 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.7190000000000007e-06 2024-09-20T22:10:57.884 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.884 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.884 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.885 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440055+0000", 2024-09-20T22:10:57.885 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1146999999999999e-05 2024-09-20T22:10:57.885 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.885 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.885 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.885 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440081+0000", 2024-09-20T22:10:57.885 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.5432e-05 2024-09-20T22:10:57.885 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.885 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.885 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.885 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601707+0000", 2024-09-20T22:10:57.885 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.161626409 2024-09-20T22:10:57.885 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.885 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.886 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:10:57.886 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601721+0000", 2024-09-20T22:10:57.886 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.36e-05 2024-09-20T22:10:57.886 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.886 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.886 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.886 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.478152+0000", 2024-09-20T22:10:57.886 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87643165099999998 2024-09-20T22:10:57.886 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.886 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.886 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.886 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.478218+0000", 2024-09-20T22:10:57.886 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5752000000000003e-05 2024-09-20T22:10:57.886 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.887 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.887 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.887 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.887 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.887 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169956 7.1 7:8584ed68:::benchmark_data_smithi060_78800_object36923:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:10:57.887 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.440104+0000", 2024-09-20T22:10:57.887 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 299.41053699299999, 2024-09-20T22:10:57.887 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.038584881, 2024-09-20T22:10:57.887 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.887 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.887 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:10:57.887 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:10:57.887 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:10:57.887 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:57.887 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169956 2024-09-20T22:10:57.888 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.888 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.888 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.888 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.888 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440104+0000", 2024-09-20T22:10:57.888 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.888 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.888 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.888 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.888 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440104+0000", 2024-09-20T22:10:57.888 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.888 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.888 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.888 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.889 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440105+0000", 2024-09-20T22:10:57.889 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.68e-07 2024-09-20T22:10:57.889 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.889 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.889 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.889 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440108+0000", 2024-09-20T22:10:57.889 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.92e-06 2024-09-20T22:10:57.889 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.889 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.889 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.889 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440116+0000", 2024-09-20T22:10:57.889 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.276e-06 2024-09-20T22:10:57.889 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.889 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.889 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.890 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440121+0000", 2024-09-20T22:10:57.890 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5940000000000001e-06 2024-09-20T22:10:57.890 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.890 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.890 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.890 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601779+0000", 2024-09-20T22:10:57.890 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.16165766000000001 2024-09-20T22:10:57.890 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.890 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.890 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:10:57.890 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601788+0000", 2024-09-20T22:10:57.890 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.149e-06 2024-09-20T22:10:57.890 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.891 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.891 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.891 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.478621+0000", 2024-09-20T22:10:57.891 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87683237199999997 2024-09-20T22:10:57.891 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.891 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.891 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.891 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.478689+0000", 2024-09-20T22:10:57.891 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8342000000000004e-05 2024-09-20T22:10:57.891 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.891 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.891 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.891 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.891 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.891 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169965 7.11 7:8d58a08c:::benchmark_data_smithi060_78800_object36932:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:10:57.892 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.440131+0000", 2024-09-20T22:10:57.892 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 299.41051007599998, 2024-09-20T22:10:57.892 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.064937993, 2024-09-20T22:10:57.892 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.892 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.892 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:57.892 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:10:57.892 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:10:57.892 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:57.892 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169965 2024-09-20T22:10:57.892 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.892 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.892 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.892 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.893 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440131+0000", 2024-09-20T22:10:57.893 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.893 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.893 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.893 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.893 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440131+0000", 2024-09-20T22:10:57.893 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.893 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.893 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.893 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.893 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440131+0000", 2024-09-20T22:10:57.893 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9900000000000001e-07 2024-09-20T22:10:57.893 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.893 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.894 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.894 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440134+0000", 2024-09-20T22:10:57.894 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1100000000000001e-06 2024-09-20T22:10:57.894 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.894 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.894 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.894 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440144+0000", 2024-09-20T22:10:57.894 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0339999999999999e-05 2024-09-20T22:10:57.894 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.894 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.894 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.894 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440149+0000", 2024-09-20T22:10:57.894 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8339999999999998e-06 2024-09-20T22:10:57.894 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.894 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.895 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.895 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601764+0000", 2024-09-20T22:10:57.895 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.16161526700000001 2024-09-20T22:10:57.895 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.895 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.895 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:10:57.895 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601772+0000", 2024-09-20T22:10:57.895 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2220000000000002e-06 2024-09-20T22:10:57.895 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.895 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.895 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.895 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.501745+0000", 2024-09-20T22:10:57.895 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.89997252100000003 2024-09-20T22:10:57.895 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.896 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.896 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:10:57.896 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.501902+0000", 2024-09-20T22:10:57.896 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015688599999999999 2024-09-20T22:10:57.896 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.896 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.896 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:10:57.896 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502171+0000", 2024-09-20T22:10:57.896 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00026939500000000002 2024-09-20T22:10:57.896 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.896 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.896 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:10:57.896 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.503753+0000", 2024-09-20T22:10:57.896 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001582452 2024-09-20T22:10:57.896 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.896 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.897 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.897 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.504315+0000", 2024-09-20T22:10:57.897 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00056145700000000002 2024-09-20T22:10:57.897 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.897 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.897 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.897 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.504956+0000", 2024-09-20T22:10:57.897 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00064100400000000003 2024-09-20T22:10:57.897 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.897 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.897 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:10:57.897 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.505026+0000", 2024-09-20T22:10:57.897 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0556999999999995e-05 2024-09-20T22:10:57.897 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.898 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.898 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.898 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.505069+0000", 2024-09-20T22:10:57.898 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2549000000000002e-05 2024-09-20T22:10:57.898 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.898 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.898 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.898 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.898 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.898 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169972 7.1 7:8552298d:::benchmark_data_smithi060_78800_object36939:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:10:57.898 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.441015+0000", 2024-09-20T22:10:57.898 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 299.409626162, 2024-09-20T22:10:57.898 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0440233649999999, 2024-09-20T22:10:57.899 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.899 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.899 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:57.899 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:10:57.899 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:10:57.899 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:57.899 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169972 2024-09-20T22:10:57.899 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.899 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.899 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.899 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.899 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.441015+0000", 2024-09-20T22:10:57.899 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.899 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.900 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.900 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.900 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.441015+0000", 2024-09-20T22:10:57.900 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.900 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.900 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.900 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.441017+0000", 2024-09-20T22:10:57.900 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.604e-06 2024-09-20T22:10:57.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.900 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.900 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.900 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.441025+0000", 2024-09-20T22:10:57.900 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2949999999999994e-06 2024-09-20T22:10:57.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.900 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.901 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.901 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.441044+0000", 2024-09-20T22:10:57.901 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8695e-05 2024-09-20T22:10:57.901 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.901 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.901 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.901 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.441052+0000", 2024-09-20T22:10:57.901 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2109999999999998e-06 2024-09-20T22:10:57.901 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.901 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.902 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.902 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601834+0000", 2024-09-20T22:10:57.902 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.16078221000000001 2024-09-20T22:10:57.902 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.902 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.902 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:10:57.902 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.601844+0000", 2024-09-20T22:10:57.902 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9599999999999995e-06 2024-09-20T22:10:57.902 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.902 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.902 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.902 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.478719+0000", 2024-09-20T22:10:57.902 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87687500500000004 2024-09-20T22:10:57.902 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.903 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.903 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:10:57.903 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.478834+0000", 2024-09-20T22:10:57.903 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011486599999999999 2024-09-20T22:10:57.903 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.903 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.903 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:10:57.903 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.479121+0000", 2024-09-20T22:10:57.903 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000286908 2024-09-20T22:10:57.903 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.903 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.903 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:10:57.903 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.480496+0000", 2024-09-20T22:10:57.903 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0013755779999999999 2024-09-20T22:10:57.904 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.904 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.904 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.904 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.481174+0000", 2024-09-20T22:10:57.904 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000677518 2024-09-20T22:10:57.904 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.904 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.904 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.904 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.484740+0000", 2024-09-20T22:10:57.904 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0035661120000000002 2024-09-20T22:10:57.904 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.904 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.904 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:10:57.904 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.484828+0000", 2024-09-20T22:10:57.905 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8219000000000004e-05 2024-09-20T22:10:57.905 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.905 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.905 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.905 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.485038+0000", 2024-09-20T22:10:57.905 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00021018400000000001 2024-09-20T22:10:57.905 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.905 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.905 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.905 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.905 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.905 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227669 7.19 7:9d9b5c13:::benchmark_data_smithi060_78800_object94636:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-20T22:10:57.905 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:54.951888+0000", 2024-09-20T22:10:57.905 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 242.89875268599999, 2024-09-20T22:10:57.905 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.90692337199999995, 2024-09-20T22:10:57.906 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.906 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.906 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:57.906 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:10:57.906 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:10:57.906 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:57.906 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227669 2024-09-20T22:10:57.906 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.906 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.906 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.906 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.906 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.951888+0000", 2024-09-20T22:10:57.906 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.906 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.907 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.907 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.907 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.951888+0000", 2024-09-20T22:10:57.907 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.907 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.907 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.907 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.951890+0000", 2024-09-20T22:10:57.907 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.331e-06 2024-09-20T22:10:57.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.907 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.907 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.907 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.951896+0000", 2024-09-20T22:10:57.907 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9780000000000002e-06 2024-09-20T22:10:57.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.907 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.908 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.908 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.951909+0000", 2024-09-20T22:10:57.908 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3429e-05 2024-09-20T22:10:57.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.908 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.908 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.908 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.951915+0000", 2024-09-20T22:10:57.908 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3130000000000004e-06 2024-09-20T22:10:57.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.908 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.908 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.908 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.951968+0000", 2024-09-20T22:10:57.908 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2398999999999998e-05 2024-09-20T22:10:57.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.908 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.908 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:10:57.909 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.951976+0000", 2024-09-20T22:10:57.909 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6979999999999998e-06 2024-09-20T22:10:57.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.909 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.909 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.909 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.849419+0000", 2024-09-20T22:10:57.909 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.89744315100000005 2024-09-20T22:10:57.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.909 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.909 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:10:57.909 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.849567+0000", 2024-09-20T22:10:57.909 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014835599999999999 2024-09-20T22:10:57.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.909 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.909 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,2,6,7", 2024-09-20T22:10:57.910 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.849988+0000", 2024-09-20T22:10:57.910 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000420474 2024-09-20T22:10:57.910 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.910 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.910 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:10:57.910 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.852037+0000", 2024-09-20T22:10:57.910 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0020494839999999999 2024-09-20T22:10:57.910 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.910 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.910 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.910 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.852091+0000", 2024-09-20T22:10:57.910 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3730000000000002e-05 2024-09-20T22:10:57.910 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.910 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.910 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.911 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.852531+0000", 2024-09-20T22:10:57.911 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000440684 2024-09-20T22:10:57.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.911 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.912 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.912 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.852882+0000", 2024-09-20T22:10:57.912 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00035092000000000001 2024-09-20T22:10:57.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.912 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.912 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.912 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.858586+0000", 2024-09-20T22:10:57.912 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0057031319999999996 2024-09-20T22:10:57.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.912 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.912 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:10:57.912 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.858650+0000", 2024-09-20T22:10:57.912 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4193e-05 2024-09-20T22:10:57.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.913 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.913 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.913 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.858812+0000", 2024-09-20T22:10:57.913 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00016210000000000001 2024-09-20T22:10:57.913 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.913 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.913 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.913 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.913 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.913 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:228626 7.19 7:9d5cff51:::benchmark_data_smithi060_78800_object95593:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-20T22:10:57.913 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:57.413427+0000", 2024-09-20T22:10:57.913 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 240.43721381399999, 2024-09-20T22:10:57.913 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.168266368, 2024-09-20T22:10:57.914 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.914 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.914 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:57.914 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:10:57.914 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:10:57.914 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:57.914 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 228626 2024-09-20T22:10:57.914 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.914 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.914 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.914 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.914 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.413427+0000", 2024-09-20T22:10:57.914 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.914 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.915 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.915 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.915 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.413427+0000", 2024-09-20T22:10:57.915 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.915 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.915 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.915 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.413429+0000", 2024-09-20T22:10:57.915 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.655e-06 2024-09-20T22:10:57.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.915 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.915 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.915 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.413435+0000", 2024-09-20T22:10:57.916 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1199999999999999e-06 2024-09-20T22:10:57.916 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.916 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.916 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.916 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.413451+0000", 2024-09-20T22:10:57.916 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6226999999999999e-05 2024-09-20T22:10:57.916 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.916 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.916 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.916 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.413460+0000", 2024-09-20T22:10:57.916 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.337e-06 2024-09-20T22:10:57.916 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.916 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.916 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.916 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.413530+0000", 2024-09-20T22:10:57.916 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0208000000000003e-05 2024-09-20T22:10:57.917 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.917 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.917 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:10:57.917 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.413684+0000", 2024-09-20T22:10:57.917 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015458299999999999 2024-09-20T22:10:57.917 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.917 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.917 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,2,6,7", 2024-09-20T22:10:57.917 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.414028+0000", 2024-09-20T22:10:57.917 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00034360700000000001 2024-09-20T22:10:57.917 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.917 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.917 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.917 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.418106+0000", 2024-09-20T22:10:57.918 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0040777280000000001 2024-09-20T22:10:57.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.918 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.918 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:10:57.918 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.418125+0000", 2024-09-20T22:10:57.918 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9517e-05 2024-09-20T22:10:57.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.918 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.918 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.918 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.418166+0000", 2024-09-20T22:10:57.919 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0525000000000002e-05 2024-09-20T22:10:57.919 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.919 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.919 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.919 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.418233+0000", 2024-09-20T22:10:57.919 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7242000000000004e-05 2024-09-20T22:10:57.919 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.919 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.919 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.919 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.581489+0000", 2024-09-20T22:10:57.919 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.16325604299999999 2024-09-20T22:10:57.919 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.919 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.919 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:10:57.920 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.581550+0000", 2024-09-20T22:10:57.920 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0748000000000002e-05 2024-09-20T22:10:57.920 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.920 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.920 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.920 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.581694+0000", 2024-09-20T22:10:57.920 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014382800000000001 2024-09-20T22:10:57.920 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.920 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.920 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.920 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.920 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.920 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:228637 7.1 7:82d66d02:::benchmark_data_smithi060_78800_object95604:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-20T22:10:57.920 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:57.421356+0000", 2024-09-20T22:10:57.920 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 240.429284938, 2024-09-20T22:10:57.921 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.15607944700000001, 2024-09-20T22:10:57.921 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.921 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.921 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:57.921 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:10:57.921 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:10:57.921 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:57.921 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 228637 2024-09-20T22:10:57.921 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.921 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.921 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.921 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.921 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.421356+0000", 2024-09-20T22:10:57.921 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.921 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.921 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.922 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.922 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.421356+0000", 2024-09-20T22:10:57.922 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.922 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.922 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.922 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.922 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.421358+0000", 2024-09-20T22:10:57.922 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6199999999999999e-06 2024-09-20T22:10:57.922 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.922 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.922 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.922 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.421363+0000", 2024-09-20T22:10:57.922 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6690000000000003e-06 2024-09-20T22:10:57.923 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.923 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.923 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.923 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.421380+0000", 2024-09-20T22:10:57.923 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6526999999999999e-05 2024-09-20T22:10:57.923 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.923 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.923 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.923 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.421393+0000", 2024-09-20T22:10:57.923 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2842e-05 2024-09-20T22:10:57.923 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.923 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.923 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.923 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.421553+0000", 2024-09-20T22:10:57.923 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000160075 2024-09-20T22:10:57.924 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.924 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.924 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:10:57.924 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.421689+0000", 2024-09-20T22:10:57.924 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013628499999999999 2024-09-20T22:10:57.924 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.924 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.924 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:10:57.924 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.422046+0000", 2024-09-20T22:10:57.924 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00035689499999999998 2024-09-20T22:10:57.924 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.924 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.924 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.924 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.424709+0000", 2024-09-20T22:10:57.924 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0026629119999999999 2024-09-20T22:10:57.925 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.925 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.925 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:10:57.925 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.426217+0000", 2024-09-20T22:10:57.925 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0015080740000000001 2024-09-20T22:10:57.925 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.925 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.925 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.925 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.577303+0000", 2024-09-20T22:10:57.925 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.15108629600000001 2024-09-20T22:10:57.925 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.925 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.925 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:10:57.925 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.577391+0000", 2024-09-20T22:10:57.925 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.7684999999999998e-05 2024-09-20T22:10:57.926 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.926 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.926 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.926 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.577436+0000", 2024-09-20T22:10:57.926 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4567000000000002e-05 2024-09-20T22:10:57.926 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.926 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.926 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.926 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.926 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.926 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:228640 7.16 7:6e088e0d:::benchmark_data_smithi060_78800_object95607:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-20T22:10:57.926 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:57.423023+0000", 2024-09-20T22:10:57.926 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 240.42761771400001, 2024-09-20T22:10:57.926 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.155811536, 2024-09-20T22:10:57.927 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.927 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.927 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:57.927 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:10:57.927 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:10:57.927 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:57.927 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 228640 2024-09-20T22:10:57.927 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.927 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.927 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.927 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.927 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.423023+0000", 2024-09-20T22:10:57.927 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.927 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.928 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.928 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.928 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.423023+0000", 2024-09-20T22:10:57.928 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.928 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.928 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.928 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.423024+0000", 2024-09-20T22:10:57.928 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.47e-07 2024-09-20T22:10:57.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.928 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.928 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.928 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.423028+0000", 2024-09-20T22:10:57.928 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.659e-06 2024-09-20T22:10:57.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.423037+0000", 2024-09-20T22:10:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3430000000000002e-06 2024-09-20T22:10:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.423042+0000", 2024-09-20T22:10:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1780000000000002e-06 2024-09-20T22:10:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.423104+0000", 2024-09-20T22:10:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1307000000000001e-05 2024-09-20T22:10:57.929 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:10:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.423222+0000", 2024-09-20T22:10:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000118068 2024-09-20T22:10:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:10:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.423600+0000", 2024-09-20T22:10:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00037805400000000001 2024-09-20T22:10:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:10:57.930 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.425364+0000", 2024-09-20T22:10:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0017645720000000001 2024-09-20T22:10:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.426173+0000", 2024-09-20T22:10:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00080837499999999996 2024-09-20T22:10:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.578732+0000", 2024-09-20T22:10:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.15255918399999999 2024-09-20T22:10:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:10:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.578791+0000", 2024-09-20T22:10:57.931 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9478999999999998e-05 2024-09-20T22:10:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:57.578835+0000", 2024-09-20T22:10:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3470000000000002e-05 2024-09-20T22:10:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:230919 7.19 7:9fa8accd:::benchmark_data_smithi060_78800_object97886:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e147)", 2024-09-20T22:10:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:00.077530+0000", 2024-09-20T22:10:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 237.77311129700001, 2024-09-20T22:10:57.932 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.14338880000000001, 2024-09-20T22:10:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:10:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:10:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:10:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 230919 2024-09-20T22:10:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:00.077530+0000", 2024-09-20T22:10:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.933 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:00.077530+0000", 2024-09-20T22:10:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:00.077531+0000", 2024-09-20T22:10:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.578e-06 2024-09-20T22:10:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:00.077535+0000", 2024-09-20T22:10:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0509999999999998e-06 2024-09-20T22:10:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.934 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:00.077550+0000", 2024-09-20T22:10:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4275e-05 2024-09-20T22:10:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:00.077557+0000", 2024-09-20T22:10:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.25e-06 2024-09-20T22:10:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:00.077699+0000", 2024-09-20T22:10:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000141991 2024-09-20T22:10:57.935 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:10:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:00.077838+0000", 2024-09-20T22:10:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000138617 2024-09-20T22:10:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,2,6,7", 2024-09-20T22:10:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:00.078259+0000", 2024-09-20T22:10:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000420946 2024-09-20T22:10:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:00.082506+0000", 2024-09-20T22:10:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0042478079999999996 2024-09-20T22:10:57.936 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.937 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.937 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:10:57.937 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:00.082531+0000", 2024-09-20T22:10:57.937 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.4487000000000001e-05 2024-09-20T22:10:57.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.937 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.937 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.937 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:00.082944+0000", 2024-09-20T22:10:57.937 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00041335800000000001 2024-09-20T22:10:57.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.937 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.937 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.937 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:00.181106+0000", 2024-09-20T22:10:57.937 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.098161583999999996 2024-09-20T22:10:57.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.938 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.938 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.938 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:00.220722+0000", 2024-09-20T22:10:57.938 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.039616353999999999 2024-09-20T22:10:57.938 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.938 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:57.939 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:57.939 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:57.940 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:10:57.940 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:00.220784+0000", 2024-09-20T22:10:57.940 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2263999999999999e-05 2024-09-20T22:10:57.940 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.940 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.940 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.940 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:00.220919+0000", 2024-09-20T22:10:57.940 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000134237 2024-09-20T22:10:57.940 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.940 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.940 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.941 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.941 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.941 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:31026 8.2 8:4044a2fc:::benchmark_data_smithi060_111051_object31025:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e184)", 2024-09-20T22:10:57.941 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:08:09.463580+0000", 2024-09-20T22:10:57.941 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 168.38706151700001, 2024-09-20T22:10:57.941 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.14090234600000001, 2024-09-20T22:10:57.941 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.941 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.941 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:57.941 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:10:57.941 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:10:57.941 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:57.941 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 31026 2024-09-20T22:10:57.941 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.942 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.942 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.942 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.942 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.463580+0000", 2024-09-20T22:10:57.942 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.942 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.942 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.942 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.942 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.463580+0000", 2024-09-20T22:10:57.942 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.942 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.942 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.942 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.942 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.463581+0000", 2024-09-20T22:10:57.942 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.81e-06 2024-09-20T22:10:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.463624+0000", 2024-09-20T22:10:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.303e-05 2024-09-20T22:10:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.463634+0000", 2024-09-20T22:10:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6090000000000007e-06 2024-09-20T22:10:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.463641+0000", 2024-09-20T22:10:57.943 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7889999999999997e-06 2024-09-20T22:10:57.944 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.944 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.944 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.944 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.463691+0000", 2024-09-20T22:10:57.944 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9814000000000001e-05 2024-09-20T22:10:57.944 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.944 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.944 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:10:57.944 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.463766+0000", 2024-09-20T22:10:57.944 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5333000000000005e-05 2024-09-20T22:10:57.944 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.944 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.944 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,5", 2024-09-20T22:10:57.944 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.463844+0000", 2024-09-20T22:10:57.944 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7620999999999994e-05 2024-09-20T22:10:57.945 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.945 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.945 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:10:57.945 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.464809+0000", 2024-09-20T22:10:57.945 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00096550300000000004 2024-09-20T22:10:57.945 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.945 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.945 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.945 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.467370+0000", 2024-09-20T22:10:57.945 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0025604970000000001 2024-09-20T22:10:57.945 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.945 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.945 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.945 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.604365+0000", 2024-09-20T22:10:57.945 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.13699521100000001 2024-09-20T22:10:57.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.946 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.946 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:10:57.946 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.604438+0000", 2024-09-20T22:10:57.946 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2917000000000007e-05 2024-09-20T22:10:57.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.946 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.946 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.946 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:08:09.604482+0000", 2024-09-20T22:10:57.946 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4212000000000003e-05 2024-09-20T22:10:57.946 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.946 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.946 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.946 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.947 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105669 8.2 8:4082eb41:::benchmark_data_smithi060_111051_object105668:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:10:57.947 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.729510+0000", 2024-09-20T22:10:57.947 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 109.121130838, 2024-09-20T22:10:57.947 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.84946794800000003, 2024-09-20T22:10:57.947 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.947 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.947 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:57.947 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:10:57.947 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:10:57.947 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:57.947 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105669 2024-09-20T22:10:57.947 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.947 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.947 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.947 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.948 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.729510+0000", 2024-09-20T22:10:57.948 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.948 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.948 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.948 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.948 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.729510+0000", 2024-09-20T22:10:57.948 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.948 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.948 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.948 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.948 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.729513+0000", 2024-09-20T22:10:57.948 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.712e-06 2024-09-20T22:10:57.948 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.948 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.948 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.949 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.729668+0000", 2024-09-20T22:10:57.949 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015500400000000001 2024-09-20T22:10:57.949 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.949 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.949 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.949 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.729693+0000", 2024-09-20T22:10:57.949 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.4992000000000001e-05 2024-09-20T22:10:57.949 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.949 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.949 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.949 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.729704+0000", 2024-09-20T22:10:57.949 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1090000000000001e-05 2024-09-20T22:10:57.949 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.949 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.949 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.950 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.729795+0000", 2024-09-20T22:10:57.950 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1289e-05 2024-09-20T22:10:57.950 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.950 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.950 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:10:57.950 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.729808+0000", 2024-09-20T22:10:57.950 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2558000000000001e-05 2024-09-20T22:10:57.950 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.950 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.950 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.950 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.561105+0000", 2024-09-20T22:10:57.950 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.83129741499999998 2024-09-20T22:10:57.950 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.950 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.950 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:10:57.951 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.561285+0000", 2024-09-20T22:10:57.951 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00018009699999999999 2024-09-20T22:10:57.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.951 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.951 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,5", 2024-09-20T22:10:57.951 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.561641+0000", 2024-09-20T22:10:57.951 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000356032 2024-09-20T22:10:57.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.951 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.951 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.951 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.565957+0000", 2024-09-20T22:10:57.951 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0043153749999999998 2024-09-20T22:10:57.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.951 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.951 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:10:57.952 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.566871+0000", 2024-09-20T22:10:57.952 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00091387300000000005 2024-09-20T22:10:57.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.952 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.952 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.952 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.578871+0000", 2024-09-20T22:10:57.952 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.012000314 2024-09-20T22:10:57.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.952 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.952 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:10:57.952 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.578933+0000", 2024-09-20T22:10:57.952 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1736999999999995e-05 2024-09-20T22:10:57.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.952 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.953 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.953 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.578978+0000", 2024-09-20T22:10:57.953 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5460000000000002e-05 2024-09-20T22:10:57.953 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.953 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.953 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.953 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.953 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.953 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105670 8.12 8:49071780:::benchmark_data_smithi060_111051_object105669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:10:57.953 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.729735+0000", 2024-09-20T22:10:57.953 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 109.120906293, 2024-09-20T22:10:57.953 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.86556648800000002, 2024-09-20T22:10:57.953 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.953 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.953 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:57.954 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:10:57.954 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:10:57.954 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:57.954 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105670 2024-09-20T22:10:57.954 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.954 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.954 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.954 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.954 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.729735+0000", 2024-09-20T22:10:57.954 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.954 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.954 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.954 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.954 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.729735+0000", 2024-09-20T22:10:57.954 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.955 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.955 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.955 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.955 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.729736+0000", 2024-09-20T22:10:57.955 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6899999999999997e-07 2024-09-20T22:10:57.955 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.955 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.955 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.955 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.729764+0000", 2024-09-20T22:10:57.955 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.8932999999999999e-05 2024-09-20T22:10:57.955 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.955 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.955 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.955 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.729774+0000", 2024-09-20T22:10:57.955 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6069999999999997e-06 2024-09-20T22:10:57.956 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.956 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.956 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.956 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.729780+0000", 2024-09-20T22:10:57.956 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1330000000000004e-06 2024-09-20T22:10:57.956 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.956 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.957 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.957 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.729835+0000", 2024-09-20T22:10:57.958 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5266999999999998e-05 2024-09-20T22:10:57.958 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.958 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.958 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:10:57.958 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.729843+0000", 2024-09-20T22:10:57.958 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7239999999999992e-06 2024-09-20T22:10:57.958 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.958 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.958 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.958 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.581389+0000", 2024-09-20T22:10:57.958 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.85154538999999996 2024-09-20T22:10:57.958 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.958 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.958 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:10:57.959 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.581522+0000", 2024-09-20T22:10:57.959 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013327799999999999 2024-09-20T22:10:57.959 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.959 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.959 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,5", 2024-09-20T22:10:57.959 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.581760+0000", 2024-09-20T22:10:57.959 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00023805300000000001 2024-09-20T22:10:57.959 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.959 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.959 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.959 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584916+0000", 2024-09-20T22:10:57.959 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0031558990000000002 2024-09-20T22:10:57.959 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.959 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.959 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:10:57.959 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.585266+0000", 2024-09-20T22:10:57.960 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000349821 2024-09-20T22:10:57.960 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.960 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.960 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.960 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.595197+0000", 2024-09-20T22:10:57.960 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0099315949999999997 2024-09-20T22:10:57.960 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.960 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.960 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:10:57.960 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.595259+0000", 2024-09-20T22:10:57.960 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1297999999999995e-05 2024-09-20T22:10:57.960 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.960 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.960 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.960 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.595301+0000", 2024-09-20T22:10:57.961 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2821000000000002e-05 2024-09-20T22:10:57.961 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.961 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.961 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.961 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.961 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.961 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:106980 8.18 8.2839d438 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:10:57.961 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.578498+0000", 2024-09-20T22:10:57.961 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 107.272142872, 2024-09-20T22:10:57.961 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.83717385600000005, 2024-09-20T22:10:57.961 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.961 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.961 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:10:57.961 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:10:57.962 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:10:57.962 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:57.962 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 106980 2024-09-20T22:10:57.962 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.962 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.962 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.962 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.962 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.578498+0000", 2024-09-20T22:10:57.962 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.962 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.962 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.962 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.962 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.578498+0000", 2024-09-20T22:10:57.962 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.962 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.963 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.963 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.963 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.578508+0000", 2024-09-20T22:10:57.963 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4849999999999998e-06 2024-09-20T22:10:57.963 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.963 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.963 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.963 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.578539+0000", 2024-09-20T22:10:57.963 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1597999999999998e-05 2024-09-20T22:10:57.963 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.963 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.963 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.963 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.578553+0000", 2024-09-20T22:10:57.963 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4089e-05 2024-09-20T22:10:57.963 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.964 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.964 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.964 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.578562+0000", 2024-09-20T22:10:57.964 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.7369999999999995e-06 2024-09-20T22:10:57.964 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.964 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.964 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.964 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.578623+0000", 2024-09-20T22:10:57.964 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1123000000000004e-05 2024-09-20T22:10:57.964 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.964 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.964 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:10:57.964 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.578631+0000", 2024-09-20T22:10:57.964 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8820000000000001e-06 2024-09-20T22:10:57.964 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.965 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.965 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.965 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.415662+0000", 2024-09-20T22:10:57.965 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.83703112800000001 2024-09-20T22:10:57.965 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.965 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.965 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.965 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.415672+0000", 2024-09-20T22:10:57.965 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8139999999999995e-06 2024-09-20T22:10:57.965 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.965 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.965 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.965 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.965 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.965 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:106986 8.14 8:2e2da639:::benchmark_data_smithi060_111051_object106985:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:10:57.965 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.590055+0000", 2024-09-20T22:10:57.966 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 107.260585784, 2024-09-20T22:10:57.966 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.924907591, 2024-09-20T22:10:57.966 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:10:57.966 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:10:57.966 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:10:57.966 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:10:57.966 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:10:57.966 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:10:57.966 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 106986 2024-09-20T22:10:57.966 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.966 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:10:57.966 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.966 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:10:57.966 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.590055+0000", 2024-09-20T22:10:57.967 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.967 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.967 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.967 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:10:57.967 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.590055+0000", 2024-09-20T22:10:57.967 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:10:57.967 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.967 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.967 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:10:57.967 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.590057+0000", 2024-09-20T22:10:57.967 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8589999999999999e-06 2024-09-20T22:10:57.967 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.967 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.967 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:10:57.967 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.590088+0000", 2024-09-20T22:10:57.968 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1257000000000002e-05 2024-09-20T22:10:57.968 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.968 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.968 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:10:57.968 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.590102+0000", 2024-09-20T22:10:57.968 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3934000000000001e-05 2024-09-20T22:10:57.968 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.968 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.968 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:10:57.968 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.590111+0000", 2024-09-20T22:10:57.968 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2649999999999996e-06 2024-09-20T22:10:57.968 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.968 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.968 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.968 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.590180+0000", 2024-09-20T22:10:57.969 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9027000000000002e-05 2024-09-20T22:10:57.969 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.969 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.969 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:10:57.969 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.590192+0000", 2024-09-20T22:10:57.969 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2454e-05 2024-09-20T22:10:57.969 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.969 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.969 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.969 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.414403+0000", 2024-09-20T22:10:57.969 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.82421088499999995 2024-09-20T22:10:57.969 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.969 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.969 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:10:57.969 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.414415+0000", 2024-09-20T22:10:57.969 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2434e-05 2024-09-20T22:10:57.970 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.970 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.970 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:10:57.970 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.493605+0000", 2024-09-20T22:10:57.970 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.079189953 2024-09-20T22:10:57.970 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.970 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.970 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:10:57.970 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.493773+0000", 2024-09-20T22:10:57.970 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00016761799999999999 2024-09-20T22:10:57.970 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.970 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.970 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,2,5", 2024-09-20T22:10:57.970 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.494127+0000", 2024-09-20T22:10:57.971 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00035368200000000002 2024-09-20T22:10:57.971 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.971 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.971 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.971 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.496784+0000", 2024-09-20T22:10:57.971 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0026576030000000001 2024-09-20T22:10:57.971 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.971 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.971 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.971 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.505215+0000", 2024-09-20T22:10:57.971 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0084304830000000008 2024-09-20T22:10:57.971 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.971 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.971 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:10:57.971 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.505262+0000", 2024-09-20T22:10:57.972 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6798000000000002e-05 2024-09-20T22:10:57.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.972 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.972 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:10:57.972 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.514164+0000", 2024-09-20T22:10:57.972 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0089020360000000003 2024-09-20T22:10:57.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.972 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.972 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:10:57.972 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.514450+0000", 2024-09-20T22:10:57.972 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00028617999999999997 2024-09-20T22:10:57.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:10:57.972 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:10:57.972 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:10:57.972 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.514963+0000", 2024-09-20T22:10:57.972 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00051312299999999999 2024-09-20T22:10:57.973 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.973 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.973 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.973 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:10:57.973 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:10:57.973 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:10:58.041 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:58.062 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:58.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:58.063 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:58.164 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:10:58.175 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:10:58.186 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:58.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:58.187 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:10:58.287 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:58.310 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:58.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:58.311 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:58.412 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:58.433 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:58.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:58.434 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:58.535 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:58.593 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:58.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:58.594 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:58.671 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:58 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:10:58.672 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:58 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:10:58.672 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:58 smithi060 ceph-mon[29694]: 5.6 scrub starts 2024-09-20T22:10:58.672 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:58 smithi060 ceph-mon[29694]: 5.6 scrub ok 2024-09-20T22:10:58.672 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:58 smithi060 ceph-mon[29694]: osdmap e268: 8 total, 8 up, 8 in 2024-09-20T22:10:58.672 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:58 smithi060 ceph-mon[29694]: pgmap v427: 168 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 164 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.06k op/s; 5114/81477 objects misplaced (6.277%); 11 MiB/s, 171 objects/s recovering 2024-09-20T22:10:58.672 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:58 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:10:58.672 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:58 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:10:58.672 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:58 smithi060 ceph-mon[40290]: 5.6 scrub starts 2024-09-20T22:10:58.672 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:58 smithi060 ceph-mon[40290]: 5.6 scrub ok 2024-09-20T22:10:58.672 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:58 smithi060 ceph-mon[40290]: osdmap e268: 8 total, 8 up, 8 in 2024-09-20T22:10:58.673 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:58 smithi060 ceph-mon[40290]: pgmap v427: 168 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 164 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.06k op/s; 5114/81477 objects misplaced (6.277%); 11 MiB/s, 171 objects/s recovering 2024-09-20T22:10:58.695 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:58.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:58.734 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:58.734 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:58.734 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:10:58.734 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:10:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:58 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:10:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:58 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:10:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:58 smithi067 ceph-mon[33511]: 5.6 scrub starts 2024-09-20T22:10:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:58 smithi067 ceph-mon[33511]: 5.6 scrub ok 2024-09-20T22:10:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:58 smithi067 ceph-mon[33511]: osdmap e268: 8 total, 8 up, 8 in 2024-09-20T22:10:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:58 smithi067 ceph-mon[33511]: pgmap v427: 168 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 164 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 1.06k op/s; 5114/81477 objects misplaced (6.277%); 11 MiB/s, 171 objects/s recovering 2024-09-20T22:10:58.776 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:10:58.777 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:10:58.813 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:10:58.813 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:10:58.834 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:10:58.845 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:10:58.845 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:10:58.873 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:58.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:58.874 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:10:58.881 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:10:58.881 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:10:58.921 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:10:58.921 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:10:58.961 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:10:58.961 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:10:58.975 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:58.994 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:10:58.994 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:10:59.000 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T22:10:59.019 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:59.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:59.020 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:59.121 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:10:59.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:59.171 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:59.171 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:10:59.272 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:59.311 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:59.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:59.313 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:59.413 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:10:59.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:59.464 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:59.464 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:10:59.469 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:10:59.565 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:59.608 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:59.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:59.609 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:59.710 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:10:59.746 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:10:59.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:59.747 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:10:59.752 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:59 smithi060 ceph-mon[29694]: osdmap e269: 8 total, 8 up, 8 in 2024-09-20T22:10:59.752 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:59 smithi060 ceph-mon[29694]: 4.6 scrub starts 2024-09-20T22:10:59.752 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:59 smithi060 ceph-mon[29694]: 4.6 scrub ok 2024-09-20T22:10:59.752 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:59 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3394650219' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:59.752 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:10:59 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:10:59.752 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:59 smithi060 ceph-mon[40290]: osdmap e269: 8 total, 8 up, 8 in 2024-09-20T22:10:59.752 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:59 smithi060 ceph-mon[40290]: 4.6 scrub starts 2024-09-20T22:10:59.753 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:59 smithi060 ceph-mon[40290]: 4.6 scrub ok 2024-09-20T22:10:59.753 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:59 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3394650219' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:59.753 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:10:59 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:10:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:59 smithi067 ceph-mon[33511]: osdmap e269: 8 total, 8 up, 8 in 2024-09-20T22:10:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:59 smithi067 ceph-mon[33511]: 4.6 scrub starts 2024-09-20T22:10:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:59 smithi067 ceph-mon[33511]: 4.6 scrub ok 2024-09-20T22:10:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:59 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3394650219' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:10:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:10:59 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:10:59.848 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:10:59.891 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:10:59.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:10:59.892 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:10:59.990 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:10:59.990 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 0 1 2024-09-20T22:10:59.993 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:00.092 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:00.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:00.093 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:00.129 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:11:00.129 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:11:00.164 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:11:00.194 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:00.228 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:00.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:00.229 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:00.330 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:00.389 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:00.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:00.390 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:00.491 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[40290]: 3.1a scrub starts 2024-09-20T22:11:00.491 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[40290]: 3.1a scrub ok 2024-09-20T22:11:00.491 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[40290]: 3.4 scrub starts 2024-09-20T22:11:00.491 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:00.503 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:00.522 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:00.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:00.523 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:00.624 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:00.662 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:00.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:00.663 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:00 smithi067 ceph-mon[33511]: 3.1a scrub starts 2024-09-20T22:11:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:00 smithi067 ceph-mon[33511]: 3.1a scrub ok 2024-09-20T22:11:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:00 smithi067 ceph-mon[33511]: 3.4 scrub starts 2024-09-20T22:11:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:00 smithi067 ceph-mon[33511]: 3.4 scrub ok 2024-09-20T22:11:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:00 smithi067 ceph-mon[33511]: 4.15 scrub starts 2024-09-20T22:11:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:00 smithi067 ceph-mon[33511]: 4.15 scrub ok 2024-09-20T22:11:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:00 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:11:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:00 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:11:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:00 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:11:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:00 smithi067 ceph-mon[33511]: pgmap v429: 168 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 164 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.17k op/s; 5114/81477 objects misplaced (6.277%); 8.3 MiB/s, 132 objects/s recovering 2024-09-20T22:11:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:00 smithi067 ceph-mon[33511]: 5.2 scrub starts 2024-09-20T22:11:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:00 smithi067 ceph-mon[33511]: 5.2 scrub ok 2024-09-20T22:11:00.764 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[29694]: 3.1a scrub starts 2024-09-20T22:11:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[29694]: 3.1a scrub ok 2024-09-20T22:11:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[29694]: 3.4 scrub starts 2024-09-20T22:11:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[29694]: 3.4 scrub ok 2024-09-20T22:11:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[29694]: 4.15 scrub starts 2024-09-20T22:11:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[29694]: 4.15 scrub ok 2024-09-20T22:11:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:11:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:11:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:11:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[29694]: pgmap v429: 168 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 164 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.17k op/s; 5114/81477 objects misplaced (6.277%); 8.3 MiB/s, 132 objects/s recovering 2024-09-20T22:11:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[29694]: 5.2 scrub starts 2024-09-20T22:11:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[29694]: 5.2 scrub ok 2024-09-20T22:11:00.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[40290]: 3.4 scrub ok 2024-09-20T22:11:00.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[40290]: 4.15 scrub starts 2024-09-20T22:11:00.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[40290]: 4.15 scrub ok 2024-09-20T22:11:00.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:11:00.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:11:00.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:11:00.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[40290]: pgmap v429: 168 pgs: 1 active+remapped+backfilling, 3 active+remapped+backfill_toofull, 164 active+clean; 1.6 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.17k op/s; 5114/81477 objects misplaced (6.277%); 8.3 MiB/s, 132 objects/s recovering 2024-09-20T22:11:00.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[40290]: 5.2 scrub starts 2024-09-20T22:11:00.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:00 smithi060 ceph-mon[40290]: 5.2 scrub ok 2024-09-20T22:11:00.799 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:00.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:00.800 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:00.901 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:00.941 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:00.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:00.942 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:01.043 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:01.071 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:01.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:01.072 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:01.137 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T22:11:01.173 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:01.216 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:01.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:01.217 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:01.279 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:01.317 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:01.351 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:01.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:01.352 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:01.453 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:01.492 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:01 smithi067 ceph-mon[33511]: 6.7 scrub starts 2024-09-20T22:11:01.493 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:01 smithi067 ceph-mon[33511]: 6.7 scrub ok 2024-09-20T22:11:01.506 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:01.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:01.507 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:01.537 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:11:01.607 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:01.642 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:01.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:01.643 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:01.744 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:01.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:01 smithi060 ceph-mon[29694]: 6.7 scrub starts 2024-09-20T22:11:01.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:01 smithi060 ceph-mon[29694]: 6.7 scrub ok 2024-09-20T22:11:01.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:01 smithi060 ceph-mon[40290]: 6.7 scrub starts 2024-09-20T22:11:01.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:01 smithi060 ceph-mon[40290]: 6.7 scrub ok 2024-09-20T22:11:01.788 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:01.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:01.790 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:01.802 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:11:01.891 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:01.921 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:01.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:01.922 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:02.023 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:02.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:02.068 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:02.069 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:02.123 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:11:02.123 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:11:02.169 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:02.204 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:02.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:02.205 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:02.306 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:02.323 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:02.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:02.378 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:02.379 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:02.480 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:02.504 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:11:02 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:11:02.512 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:11:02.514 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:02.516 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.0 to 1 (10000) 2024-09-20T22:11:02.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:02.518 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:02.619 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:02.676 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:02.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:02.677 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:02.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:02 smithi067 ceph-mon[33511]: 3.0 scrub starts 2024-09-20T22:11:02.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:02 smithi067 ceph-mon[33511]: 3.0 scrub ok 2024-09-20T22:11:02.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:02 smithi067 ceph-mon[33511]: 8.e scrub starts 2024-09-20T22:11:02.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:02 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3626496492' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:11:02.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:02 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:11:02.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:02 smithi067 ceph-mon[33511]: pgmap v430: 168 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 164 active+clean; 1.4 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 118 op/s; 4034/68589 objects misplaced (5.881%) 2024-09-20T22:11:02.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:02 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:11:02.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:02 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1867423582' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:02.766 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:02 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:02.778 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:02.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:02 smithi060 ceph-mon[29694]: 3.0 scrub starts 2024-09-20T22:11:02.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:02 smithi060 ceph-mon[29694]: 3.0 scrub ok 2024-09-20T22:11:02.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:02 smithi060 ceph-mon[29694]: 8.e scrub starts 2024-09-20T22:11:02.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:02 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3626496492' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:11:02.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:02 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:11:02.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:02 smithi060 ceph-mon[29694]: pgmap v430: 168 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 164 active+clean; 1.4 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 118 op/s; 4034/68589 objects misplaced (5.881%) 2024-09-20T22:11:02.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:02 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:11:02.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:02 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1867423582' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:02.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:02 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:02.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:02 smithi060 ceph-mon[40290]: 3.0 scrub starts 2024-09-20T22:11:02.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:02 smithi060 ceph-mon[40290]: 3.0 scrub ok 2024-09-20T22:11:02.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:02 smithi060 ceph-mon[40290]: 8.e scrub starts 2024-09-20T22:11:02.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:02 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3626496492' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:11:02.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:02 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:11:02.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:02 smithi060 ceph-mon[40290]: pgmap v430: 168 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 164 active+clean; 1.4 GiB data, 7.7 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 118 op/s; 4034/68589 objects misplaced (5.881%) 2024-09-20T22:11:02.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:02 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:11:02.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:02 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1867423582' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:02.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:02 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:02.814 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:02.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:02.815 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:02.916 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:02.948 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:02.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:02.949 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:03.050 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:03.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:03.084 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:03.084 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:03.185 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:03.225 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:03.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:03.226 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:03.327 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:03.364 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:03.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:03.365 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:03.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 1 1 2024-09-20T22:11:03.466 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:03.495 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:03.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:03.496 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:03.597 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:03.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[29694]: 8.4 scrub starts 2024-09-20T22:11:03.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[29694]: 5.c scrub starts 2024-09-20T22:11:03.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[29694]: 5.c scrub ok 2024-09-20T22:11:03.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[29694]: 8.4 scrub ok 2024-09-20T22:11:03.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[29694]: 8.e scrub ok 2024-09-20T22:11:03.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[29694]: 4.8 scrub starts 2024-09-20T22:11:03.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[29694]: 4.8 scrub ok 2024-09-20T22:11:03.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:11:03.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:11:03.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:11:03.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[29694]: osdmap e270: 8 total, 8 up, 8 in 2024-09-20T22:11:03.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[29694]: 4.b scrub starts 2024-09-20T22:11:03.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[29694]: 4.b scrub ok 2024-09-20T22:11:03.626 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[40290]: 8.4 scrub starts 2024-09-20T22:11:03.626 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[40290]: 5.c scrub starts 2024-09-20T22:11:03.626 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[40290]: 5.c scrub ok 2024-09-20T22:11:03.626 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[40290]: 8.4 scrub ok 2024-09-20T22:11:03.626 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[40290]: 8.e scrub ok 2024-09-20T22:11:03.626 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[40290]: 4.8 scrub starts 2024-09-20T22:11:03.627 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[40290]: 4.8 scrub ok 2024-09-20T22:11:03.627 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:11:03.627 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:11:03.627 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:11:03.627 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[40290]: osdmap e270: 8 total, 8 up, 8 in 2024-09-20T22:11:03.627 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[40290]: 4.b scrub starts 2024-09-20T22:11:03.627 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:03 smithi060 ceph-mon[40290]: 4.b scrub ok 2024-09-20T22:11:03.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:03.646 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:03.646 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:03.686 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T22:11:03.748 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:03.757 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:11:03.758 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:11:03.800 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:11:03.800 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:11:03.801 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:03.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:03.803 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:03.836 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:11:03.837 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:11:03.880 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:11:03.880 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:11:03.904 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:03.913 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:11:03.914 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:11:03.935 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:03.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:03.936 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:03.950 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:11:03.950 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:11:03.988 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:11:03.988 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:11:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:03 smithi067 ceph-mon[33511]: 8.4 scrub starts 2024-09-20T22:11:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:03 smithi067 ceph-mon[33511]: 5.c scrub starts 2024-09-20T22:11:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:03 smithi067 ceph-mon[33511]: 5.c scrub ok 2024-09-20T22:11:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:03 smithi067 ceph-mon[33511]: 8.4 scrub ok 2024-09-20T22:11:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:03 smithi067 ceph-mon[33511]: 8.e scrub ok 2024-09-20T22:11:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:03 smithi067 ceph-mon[33511]: 4.8 scrub starts 2024-09-20T22:11:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:03 smithi067 ceph-mon[33511]: 4.8 scrub ok 2024-09-20T22:11:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:03 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:11:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:03 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:11:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:03 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:11:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:03 smithi067 ceph-mon[33511]: osdmap e270: 8 total, 8 up, 8 in 2024-09-20T22:11:04.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:03 smithi067 ceph-mon[33511]: 4.b scrub starts 2024-09-20T22:11:04.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:03 smithi067 ceph-mon[33511]: 4.b scrub ok 2024-09-20T22:11:04.025 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:11:04.025 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:11:04.037 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:04.039 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:11:04.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:04.071 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:04.071 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:04.172 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:04.201 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:04.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:04.202 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:04.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:04.343 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:04.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:04.344 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:04.445 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:04.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:04.492 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:04.492 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:04.593 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:04.634 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:04.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:04.635 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:04.645 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:11:04.645 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:11:04.645 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:11:04.645 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:11:04.645 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.645 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169974 7.10 7:0a082cf8:::benchmark_data_smithi060_78800_object36941:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:11:04.645 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:11:04.645 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 306.01704784499998, 2024-09-20T22:11:04.646 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87805208899999998, 2024-09-20T22:11:04.646 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.646 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.646 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:04.646 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:04.646 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:11:04.646 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:11:04.646 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169974 2024-09-20T22:11:04.646 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.646 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.646 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.646 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.647 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:11:04.647 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.647 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.647 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.647 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:11:04.647 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.647 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.647 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.647 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624616+0000", 2024-09-20T22:11:04.647 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1739999999999999e-06 2024-09-20T22:11:04.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.647 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.648 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.648 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624626+0000", 2024-09-20T22:11:04.648 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7909999999999999e-06 2024-09-20T22:11:04.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.648 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.648 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.648 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624654+0000", 2024-09-20T22:11:04.648 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.8524999999999999e-05 2024-09-20T22:11:04.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.648 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.648 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.648 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624665+0000", 2024-09-20T22:11:04.648 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0985000000000001e-05 2024-09-20T22:11:04.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.649 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.649 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.649 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.705237+0000", 2024-09-20T22:11:04.649 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.080571485999999998 2024-09-20T22:11:04.649 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.649 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.649 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.649 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.705245+0000", 2024-09-20T22:11:04.649 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8460000000000008e-06 2024-09-20T22:11:04.649 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.649 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.649 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.649 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498359+0000", 2024-09-20T22:11:04.649 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79311461999999999 2024-09-20T22:11:04.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.650 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.650 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:04.650 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498498+0000", 2024-09-20T22:11:04.650 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013918599999999999 2024-09-20T22:11:04.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.650 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.650 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:11:04.650 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498751+0000", 2024-09-20T22:11:04.650 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025235999999999998 2024-09-20T22:11:04.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.650 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.650 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.651 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.500697+0000", 2024-09-20T22:11:04.651 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0019460969999999999 2024-09-20T22:11:04.651 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.651 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.651 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:04.651 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502528+0000", 2024-09-20T22:11:04.651 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00183161 2024-09-20T22:11:04.651 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.651 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.651 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:04.651 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502597+0000", 2024-09-20T22:11:04.651 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8761000000000001e-05 2024-09-20T22:11:04.651 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.652 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.652 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.652 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502666+0000", 2024-09-20T22:11:04.652 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8647999999999997e-05 2024-09-20T22:11:04.652 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.652 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.652 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.652 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.652 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.652 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227618 7.0 7:0598f762:::benchmark_data_smithi060_78800_object94585:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-20T22:11:04.652 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:53.805369+0000", 2024-09-20T22:11:04.652 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 250.83629282000001, 2024-09-20T22:11:04.652 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2615457110000001, 2024-09-20T22:11:04.653 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.653 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.653 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:04.653 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:04.653 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:11:04.653 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:11:04.653 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227618 2024-09-20T22:11:04.653 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.653 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.653 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.653 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.653 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805369+0000", 2024-09-20T22:11:04.653 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.654 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.654 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.654 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.654 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805369+0000", 2024-09-20T22:11:04.654 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.654 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.654 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.654 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.654 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805371+0000", 2024-09-20T22:11:04.654 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.827e-06 2024-09-20T22:11:04.654 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.654 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.654 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.654 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805377+0000", 2024-09-20T22:11:04.655 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.161e-06 2024-09-20T22:11:04.655 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.655 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.655 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.655 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805398+0000", 2024-09-20T22:11:04.655 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1208000000000001e-05 2024-09-20T22:11:04.655 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.655 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.655 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.655 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805406+0000", 2024-09-20T22:11:04.655 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3820000000000004e-06 2024-09-20T22:11:04.655 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.655 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.655 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.656 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.813177+0000", 2024-09-20T22:11:04.656 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0077708919999999997 2024-09-20T22:11:04.656 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.656 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.656 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.656 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.813194+0000", 2024-09-20T22:11:04.656 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7107000000000001e-05 2024-09-20T22:11:04.656 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.656 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.656 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.656 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.813273+0000", 2024-09-20T22:11:04.656 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.000078609 2024-09-20T22:11:04.656 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.656 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.656 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.657 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.813290+0000", 2024-09-20T22:11:04.657 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7382999999999999e-05 2024-09-20T22:11:04.657 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.657 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.657 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.657 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.960476+0000", 2024-09-20T22:11:04.657 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1471854260000001 2024-09-20T22:11:04.657 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.657 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.657 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:04.657 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.960616+0000", 2024-09-20T22:11:04.657 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014035100000000001 2024-09-20T22:11:04.657 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.657 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.657 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,3", 2024-09-20T22:11:04.658 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.961013+0000", 2024-09-20T22:11:04.658 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00039649999999999999 2024-09-20T22:11:04.658 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.658 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.658 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:04.658 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.963511+0000", 2024-09-20T22:11:04.658 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0024986919999999998 2024-09-20T22:11:04.658 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.658 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.658 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.658 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.963553+0000", 2024-09-20T22:11:04.658 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1783999999999998e-05 2024-09-20T22:11:04.658 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.659 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.659 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.659 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.066713+0000", 2024-09-20T22:11:04.659 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.103160088 2024-09-20T22:11:04.659 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.659 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.660 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:04.660 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.066794+0000", 2024-09-20T22:11:04.660 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0924e-05 2024-09-20T22:11:04.660 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.660 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.660 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.660 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.066915+0000", 2024-09-20T22:11:04.660 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000120377 2024-09-20T22:11:04.660 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.660 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.661 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.661 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.661 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.661 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227635 7.b 7:da8bd73f:::benchmark_data_smithi060_78800_object94602:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e146)", 2024-09-20T22:11:04.661 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:54.820720+0000", 2024-09-20T22:11:04.661 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 249.820941944, 2024-09-20T22:11:04.661 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1621617440000001, 2024-09-20T22:11:04.661 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.661 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.661 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:04.661 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:04.661 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:11:04.661 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:11:04.661 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227635 2024-09-20T22:11:04.662 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.662 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.662 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.662 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.662 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820720+0000", 2024-09-20T22:11:04.662 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.662 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.662 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.662 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.662 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820720+0000", 2024-09-20T22:11:04.662 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.662 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.662 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.662 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.663 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820721+0000", 2024-09-20T22:11:04.663 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.593e-06 2024-09-20T22:11:04.663 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.663 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.663 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.663 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820728+0000", 2024-09-20T22:11:04.663 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3799999999999999e-06 2024-09-20T22:11:04.663 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.663 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.663 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.663 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820740+0000", 2024-09-20T22:11:04.663 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.279e-05 2024-09-20T22:11:04.663 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.663 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.664 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.664 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820749+0000", 2024-09-20T22:11:04.664 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8000000000000004e-06 2024-09-20T22:11:04.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.664 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.664 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.664 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820942+0000", 2024-09-20T22:11:04.664 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000192263 2024-09-20T22:11:04.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.664 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.664 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.664 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820953+0000", 2024-09-20T22:11:04.664 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1953000000000001e-05 2024-09-20T22:11:04.664 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.665 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.665 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.665 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.974865+0000", 2024-09-20T22:11:04.665 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.15391142 2024-09-20T22:11:04.665 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.665 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.665 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:04.665 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.974939+0000", 2024-09-20T22:11:04.665 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3798000000000001e-05 2024-09-20T22:11:04.665 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.665 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.665 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,6", 2024-09-20T22:11:04.665 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.975019+0000", 2024-09-20T22:11:04.666 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0692999999999995e-05 2024-09-20T22:11:04.666 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.666 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.666 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.666 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.977853+0000", 2024-09-20T22:11:04.666 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028332269999999998 2024-09-20T22:11:04.666 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.666 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.666 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.666 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982635+0000", 2024-09-20T22:11:04.666 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0047819840000000004 2024-09-20T22:11:04.666 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.666 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.667 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:04.667 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982688+0000", 2024-09-20T22:11:04.667 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.384e-05 2024-09-20T22:11:04.667 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.667 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.667 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:04.667 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982747+0000", 2024-09-20T22:11:04.667 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9004e-05 2024-09-20T22:11:04.667 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.667 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.667 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.667 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982881+0000", 2024-09-20T22:11:04.667 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013399899999999999 2024-09-20T22:11:04.667 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.668 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.668 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.668 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.668 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.668 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227641 7.b 7:d29062b1:::benchmark_data_smithi060_78800_object94608:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e146)", 2024-09-20T22:11:04.668 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:11:04.668 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 249.82089083299999, 2024-09-20T22:11:04.668 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2739681169999999, 2024-09-20T22:11:04.668 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.668 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.668 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:04.668 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:04.668 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:11:04.668 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:11:04.669 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227641 2024-09-20T22:11:04.669 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.669 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.669 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.669 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.669 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:11:04.669 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.669 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.669 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.669 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.669 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:11:04.669 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.669 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.670 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.670 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.670 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:11:04.670 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3900000000000005e-07 2024-09-20T22:11:04.670 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.670 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.670 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.670 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820775+0000", 2024-09-20T22:11:04.670 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4589999999999998e-06 2024-09-20T22:11:04.670 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.670 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.670 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.670 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820783+0000", 2024-09-20T22:11:04.671 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8949999999999998e-06 2024-09-20T22:11:04.671 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.671 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.671 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.671 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820788+0000", 2024-09-20T22:11:04.671 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5759999999999996e-06 2024-09-20T22:11:04.671 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.671 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.671 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.671 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820979+0000", 2024-09-20T22:11:04.671 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019122 2024-09-20T22:11:04.671 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.671 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.671 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.672 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820988+0000", 2024-09-20T22:11:04.672 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1480000000000006e-06 2024-09-20T22:11:04.672 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.672 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.672 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.672 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.977915+0000", 2024-09-20T22:11:04.672 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.156927834 2024-09-20T22:11:04.672 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.672 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.672 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:04.672 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.978045+0000", 2024-09-20T22:11:04.672 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012967 2024-09-20T22:11:04.672 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.672 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.673 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,6", 2024-09-20T22:11:04.673 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.978183+0000", 2024-09-20T22:11:04.673 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013767900000000001 2024-09-20T22:11:04.673 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.673 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.673 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.673 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982676+0000", 2024-09-20T22:11:04.673 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0044928210000000001 2024-09-20T22:11:04.673 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.673 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.673 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:04.673 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982891+0000", 2024-09-20T22:11:04.673 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00021542299999999999 2024-09-20T22:11:04.673 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.674 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.674 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.674 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.094638+0000", 2024-09-20T22:11:04.674 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.111746554 2024-09-20T22:11:04.674 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.674 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.674 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:04.674 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.094677+0000", 2024-09-20T22:11:04.674 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9069000000000002e-05 2024-09-20T22:11:04.674 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.674 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.674 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.674 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.094739+0000", 2024-09-20T22:11:04.675 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2230000000000006e-05 2024-09-20T22:11:04.675 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.675 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.675 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.675 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.675 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.675 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105650 8.4 8:29da2845:::benchmark_data_smithi060_111051_object105649:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e214)", 2024-09-20T22:11:04.675 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.523636+0000", 2024-09-20T22:11:04.675 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 116.11802583799999, 2024-09-20T22:11:04.675 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.17888942399999999, 2024-09-20T22:11:04.675 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.675 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.675 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:11:04.676 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:04.676 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:04.676 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:04.676 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105650 2024-09-20T22:11:04.676 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.676 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.676 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.676 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.676 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523636+0000", 2024-09-20T22:11:04.676 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.676 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.676 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.676 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.676 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523636+0000", 2024-09-20T22:11:04.677 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.677 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.677 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.677 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.677 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523637+0000", 2024-09-20T22:11:04.677 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.686e-06 2024-09-20T22:11:04.677 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.677 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.677 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.677 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523679+0000", 2024-09-20T22:11:04.677 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1081999999999997e-05 2024-09-20T22:11:04.677 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.677 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.678 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.678 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523690+0000", 2024-09-20T22:11:04.678 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1545e-05 2024-09-20T22:11:04.678 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.678 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.678 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.678 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523696+0000", 2024-09-20T22:11:04.678 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3879999999999999e-06 2024-09-20T22:11:04.678 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.678 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.678 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.678 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523738+0000", 2024-09-20T22:11:04.678 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2460999999999999e-05 2024-09-20T22:11:04.679 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.679 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.679 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:04.679 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523826+0000", 2024-09-20T22:11:04.679 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8088000000000001e-05 2024-09-20T22:11:04.679 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.679 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.679 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,4", 2024-09-20T22:11:04.679 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523912+0000", 2024-09-20T22:11:04.679 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6074000000000003e-05 2024-09-20T22:11:04.679 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.679 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.679 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.679 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.528900+0000", 2024-09-20T22:11:04.680 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0049876720000000003 2024-09-20T22:11:04.680 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.680 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.680 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:04.680 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.529517+0000", 2024-09-20T22:11:04.680 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00061690400000000004 2024-09-20T22:11:04.680 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.680 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.680 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.680 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.702506+0000", 2024-09-20T22:11:04.680 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.17298888600000001 2024-09-20T22:11:04.680 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.680 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.680 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.681 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.702525+0000", 2024-09-20T22:11:04.681 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9638000000000001e-05 2024-09-20T22:11:04.681 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.681 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.681 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.681 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.681 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.681 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105650 8.14 8:29da2845:::benchmark_data_smithi060_111051_object105649:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:11:04.681 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.545671+0000", 2024-09-20T22:11:04.681 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 116.09599060799999, 2024-09-20T22:11:04.681 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.038415869, 2024-09-20T22:11:04.681 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.681 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.682 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:11:04.682 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:04.682 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:04.682 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:04.682 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105650 2024-09-20T22:11:04.682 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.682 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.682 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.682 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.682 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545671+0000", 2024-09-20T22:11:04.682 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.682 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.682 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.682 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.683 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545671+0000", 2024-09-20T22:11:04.683 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.683 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.683 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.683 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.683 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545673+0000", 2024-09-20T22:11:04.683 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.17e-06 2024-09-20T22:11:04.683 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.683 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.683 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.683 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545723+0000", 2024-09-20T22:11:04.683 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9509000000000003e-05 2024-09-20T22:11:04.683 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.683 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.684 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.684 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545741+0000", 2024-09-20T22:11:04.684 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8634000000000002e-05 2024-09-20T22:11:04.684 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.684 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.684 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.684 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545751+0000", 2024-09-20T22:11:04.684 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4420000000000004e-06 2024-09-20T22:11:04.684 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.684 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.684 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.684 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.707696+0000", 2024-09-20T22:11:04.684 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.16194560099999999 2024-09-20T22:11:04.684 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.685 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.685 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.685 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.707706+0000", 2024-09-20T22:11:04.685 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6860000000000001e-06 2024-09-20T22:11:04.685 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.685 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.685 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.685 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584005+0000", 2024-09-20T22:11:04.685 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87629895400000002 2024-09-20T22:11:04.685 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.685 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.685 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.685 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584087+0000", 2024-09-20T22:11:04.685 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1872999999999994e-05 2024-09-20T22:11:04.686 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.686 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.686 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.686 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.686 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.686 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "MOSDPGPush(8.c 215/206 [PushOp(8:355e0c9a:::benchmark_data_smithi060_111051_object46967:head, version: 186'2869, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8:355e0c9a:::benchmark_data_smithi060_111051_object46967:head@186'2869, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-20T22:11:04.686 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.555562+0000", 2024-09-20T22:11:04.686 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 116.086099241, 2024-09-20T22:11:04.686 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.15072329400000001, 2024-09-20T22:11:04.686 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.686 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.686 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:11:04.686 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.687 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.687 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.687 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.555562+0000", 2024-09-20T22:11:04.687 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.687 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.687 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.687 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.555562+0000", 2024-09-20T22:11:04.687 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.687 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.687 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.687 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.687 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.555564+0000", 2024-09-20T22:11:04.687 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4139999999999999e-06 2024-09-20T22:11:04.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.688 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.688 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.688 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.555625+0000", 2024-09-20T22:11:04.688 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1282e-05 2024-09-20T22:11:04.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.688 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.688 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.688 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572138+0000", 2024-09-20T22:11:04.688 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.016512466999999999 2024-09-20T22:11:04.688 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.688 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.688 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.688 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572188+0000", 2024-09-20T22:11:04.689 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0773000000000003e-05 2024-09-20T22:11:04.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.689 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.689 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.689 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.706058+0000", 2024-09-20T22:11:04.689 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.13387001800000001 2024-09-20T22:11:04.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.689 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.689 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:04.689 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.706071+0000", 2024-09-20T22:11:04.689 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2876e-05 2024-09-20T22:11:04.689 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.689 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.689 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.689 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.706286+0000", 2024-09-20T22:11:04.690 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00021446400000000001 2024-09-20T22:11:04.690 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.690 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.690 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.690 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.690 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.690 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105665 8.18 8:1b9c2a37:::benchmark_data_smithi060_111051_object105664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:11:04.690 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.556635+0000", 2024-09-20T22:11:04.690 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 116.085026579, 2024-09-20T22:11:04.690 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.051209888, 2024-09-20T22:11:04.690 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.690 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.690 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:04.690 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:04.691 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:04.691 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:04.691 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105665 2024-09-20T22:11:04.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.691 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.691 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.691 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.691 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556635+0000", 2024-09-20T22:11:04.691 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.691 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.691 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.691 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556635+0000", 2024-09-20T22:11:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556638+0000", 2024-09-20T22:11:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.6400000000000001e-06 2024-09-20T22:11:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556714+0000", 2024-09-20T22:11:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6594999999999994e-05 2024-09-20T22:11:04.692 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556736+0000", 2024-09-20T22:11:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1662e-05 2024-09-20T22:11:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556751+0000", 2024-09-20T22:11:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4941e-05 2024-09-20T22:11:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.693 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.750414+0000", 2024-09-20T22:11:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.193663171 2024-09-20T22:11:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.750424+0000", 2024-09-20T22:11:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0397999999999999e-05 2024-09-20T22:11:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.596184+0000", 2024-09-20T22:11:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.84575938799999995 2024-09-20T22:11:04.694 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.596303+0000", 2024-09-20T22:11:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000119503 2024-09-20T22:11:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:11:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.596588+0000", 2024-09-20T22:11:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00028446100000000001 2024-09-20T22:11:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.603603+0000", 2024-09-20T22:11:04.695 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0070155649999999996 2024-09-20T22:11:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607589+0000", 2024-09-20T22:11:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0039854110000000003 2024-09-20T22:11:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607743+0000", 2024-09-20T22:11:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015391499999999999 2024-09-20T22:11:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.696 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607802+0000", 2024-09-20T22:11:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9472000000000003e-05 2024-09-20T22:11:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607845+0000", 2024-09-20T22:11:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2765999999999997e-05 2024-09-20T22:11:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.15192.0:105664 8.c e215/206 8:37c43236:::benchmark_data_smithi060_111051_object105663:head v 215'6587, mlcod=214'6584)", 2024-09-20T22:11:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.572293+0000", 2024-09-20T22:11:04.697 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 116.06936845600001, 2024-09-20T22:11:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.15892877499999999, 2024-09-20T22:11:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572293+0000", 2024-09-20T22:11:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572293+0000", 2024-09-20T22:11:04.698 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572294+0000", 2024-09-20T22:11:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.65e-07 2024-09-20T22:11:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572300+0000", 2024-09-20T22:11:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7599999999999997e-06 2024-09-20T22:11:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.699 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572308+0000", 2024-09-20T22:11:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2490000000000004e-06 2024-09-20T22:11:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572313+0000", 2024-09-20T22:11:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6590000000000004e-06 2024-09-20T22:11:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.703998+0000", 2024-09-20T22:11:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.13168523900000001 2024-09-20T22:11:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:04.700 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.704019+0000", 2024-09-20T22:11:04.701 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0267000000000001e-05 2024-09-20T22:11:04.701 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[29694]: 4.1b scrub starts 2024-09-20T22:11:04.701 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[29694]: 4.1b scrub ok 2024-09-20T22:11:04.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[29694]: 8.a deep-scrub starts 2024-09-20T22:11:04.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:11:04.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:11:04.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[29694]: 6.5 scrub starts 2024-09-20T22:11:04.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[29694]: 6.5 scrub ok 2024-09-20T22:11:04.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[29694]: osdmap e271: 8 total, 8 up, 8 in 2024-09-20T22:11:04.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/537807177' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:04.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[29694]: pgmap v433: 168 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 164 active+clean; 1.3 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 4034/66237 objects misplaced (6.090%) 2024-09-20T22:11:04.702 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[40290]: 4.1b scrub starts 2024-09-20T22:11:04.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[40290]: 4.1b scrub ok 2024-09-20T22:11:04.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[40290]: 8.a deep-scrub starts 2024-09-20T22:11:04.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:11:04.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:11:04.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[40290]: 6.5 scrub starts 2024-09-20T22:11:04.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[40290]: 6.5 scrub ok 2024-09-20T22:11:04.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[40290]: osdmap e271: 8 total, 8 up, 8 in 2024-09-20T22:11:04.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/537807177' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:04.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:04 smithi060 ceph-mon[40290]: pgmap v433: 168 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 164 active+clean; 1.3 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 4034/66237 objects misplaced (6.090%) 2024-09-20T22:11:04.703 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.703 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.703 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:04.704 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.731164+0000", 2024-09-20T22:11:04.704 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.027144784000000002 2024-09-20T22:11:04.704 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.704 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.704 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.704 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.731222+0000", 2024-09-20T22:11:04.704 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8452000000000002e-05 2024-09-20T22:11:04.704 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.704 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.704 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.704 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.704 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.704 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105673 8.14 8:2f269e39:::benchmark_data_smithi060_111051_object105672:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:11:04.705 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.761620+0000", 2024-09-20T22:11:04.705 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 115.880042026, 2024-09-20T22:11:04.705 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.83821859799999998, 2024-09-20T22:11:04.705 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.705 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.705 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:04.705 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:04.705 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:04.705 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:04.705 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105673 2024-09-20T22:11:04.705 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.705 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.705 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.705 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.706 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761620+0000", 2024-09-20T22:11:04.706 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.706 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.706 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.706 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.706 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761620+0000", 2024-09-20T22:11:04.706 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.706 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.706 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.706 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.706 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761621+0000", 2024-09-20T22:11:04.706 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.392e-06 2024-09-20T22:11:04.706 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.706 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.707 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.707 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761674+0000", 2024-09-20T22:11:04.707 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2677999999999999e-05 2024-09-20T22:11:04.707 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.707 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.707 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.707 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761686+0000", 2024-09-20T22:11:04.707 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1926000000000001e-05 2024-09-20T22:11:04.707 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.707 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.707 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.707 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761697+0000", 2024-09-20T22:11:04.707 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1364e-05 2024-09-20T22:11:04.707 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.708 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.708 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.708 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761770+0000", 2024-09-20T22:11:04.708 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2904999999999994e-05 2024-09-20T22:11:04.708 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.708 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.708 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.708 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761777+0000", 2024-09-20T22:11:04.708 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8029999999999999e-06 2024-09-20T22:11:04.708 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.708 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.708 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.708 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584113+0000", 2024-09-20T22:11:04.708 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.82233672499999999 2024-09-20T22:11:04.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.709 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.709 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:04.709 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584256+0000", 2024-09-20T22:11:04.709 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000142686 2024-09-20T22:11:04.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.709 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.709 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,4", 2024-09-20T22:11:04.709 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584528+0000", 2024-09-20T22:11:04.709 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00027199499999999997 2024-09-20T22:11:04.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.709 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.709 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.709 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.590292+0000", 2024-09-20T22:11:04.710 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.005764333 2024-09-20T22:11:04.710 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.710 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.710 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:04.710 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.590315+0000", 2024-09-20T22:11:04.710 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2768e-05 2024-09-20T22:11:04.710 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.710 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.710 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.710 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.599740+0000", 2024-09-20T22:11:04.710 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0094248200000000004 2024-09-20T22:11:04.710 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.710 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.710 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:04.711 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.599795+0000", 2024-09-20T22:11:04.711 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5108000000000002e-05 2024-09-20T22:11:04.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.711 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.711 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.711 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.599838+0000", 2024-09-20T22:11:04.711 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3095e-05 2024-09-20T22:11:04.711 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.711 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.711 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.711 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.711 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:106978 8.19 8:9e674fff:::benchmark_data_smithi060_111051_object106977:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:11:04.711 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.576265+0000", 2024-09-20T22:11:04.712 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 114.065396912, 2024-09-20T22:11:04.712 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9488434429999999, 2024-09-20T22:11:04.712 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.712 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.712 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:04.712 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:04.712 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:04.712 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:04.712 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 106978 2024-09-20T22:11:04.712 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.712 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.712 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.712 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.712 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576265+0000", 2024-09-20T22:11:04.712 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.713 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.713 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.713 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576265+0000", 2024-09-20T22:11:04.713 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.713 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.713 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.713 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576267+0000", 2024-09-20T22:11:04.713 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0509999999999999e-06 2024-09-20T22:11:04.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.713 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.713 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.714 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576299+0000", 2024-09-20T22:11:04.714 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1962000000000003e-05 2024-09-20T22:11:04.714 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.714 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.714 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.714 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576312+0000", 2024-09-20T22:11:04.714 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3423e-05 2024-09-20T22:11:04.714 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.714 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.714 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.714 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576322+0000", 2024-09-20T22:11:04.714 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0118e-05 2024-09-20T22:11:04.714 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.714 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.715 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.715 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576468+0000", 2024-09-20T22:11:04.715 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000145575 2024-09-20T22:11:04.715 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.715 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.715 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.715 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576481+0000", 2024-09-20T22:11:04.715 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3023999999999999e-05 2024-09-20T22:11:04.715 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.715 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.715 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.715 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410333+0000", 2024-09-20T22:11:04.715 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.83385187800000005 2024-09-20T22:11:04.716 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.716 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.716 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.716 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410345+0000", 2024-09-20T22:11:04.716 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.258e-05 2024-09-20T22:11:04.716 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.716 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.716 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.716 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.503436+0000", 2024-09-20T22:11:04.716 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.093091088 2024-09-20T22:11:04.716 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.716 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.717 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:04.717 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.503599+0000", 2024-09-20T22:11:04.717 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00016286999999999999 2024-09-20T22:11:04.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.717 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.717 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4,6", 2024-09-20T22:11:04.717 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.505264+0000", 2024-09-20T22:11:04.717 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001664591 2024-09-20T22:11:04.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.717 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.717 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.717 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.508504+0000", 2024-09-20T22:11:04.717 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0032404920000000002 2024-09-20T22:11:04.718 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.718 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.718 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.718 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.508547+0000", 2024-09-20T22:11:04.718 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2734000000000001e-05 2024-09-20T22:11:04.718 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.718 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.718 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.718 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.513180+0000", 2024-09-20T22:11:04.718 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046330490000000002 2024-09-20T22:11:04.718 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.718 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.718 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:04.719 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.524891+0000", 2024-09-20T22:11:04.719 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.011710379999999999 2024-09-20T22:11:04.719 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.719 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.719 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:04.719 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.524966+0000", 2024-09-20T22:11:04.719 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5290999999999995e-05 2024-09-20T22:11:04.719 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.719 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.719 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.719 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.525108+0000", 2024-09-20T22:11:04.719 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000142337 2024-09-20T22:11:04.719 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.719 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.720 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.720 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.720 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.720 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:107002 8.11 8.e0f52171 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:11:04.720 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.617682+0000", 2024-09-20T22:11:04.720 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 114.023979414, 2024-09-20T22:11:04.720 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79265608300000001, 2024-09-20T22:11:04.720 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.720 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.720 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:11:04.720 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:04.720 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:04.720 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:04.720 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 107002 2024-09-20T22:11:04.721 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.721 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.721 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.721 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.721 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617682+0000", 2024-09-20T22:11:04.721 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.721 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.721 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.721 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.721 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617682+0000", 2024-09-20T22:11:04.721 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.721 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.721 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.722 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.722 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617684+0000", 2024-09-20T22:11:04.722 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1299999999999999e-06 2024-09-20T22:11:04.722 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.722 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.722 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.722 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617725+0000", 2024-09-20T22:11:04.722 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0839e-05 2024-09-20T22:11:04.722 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.722 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.722 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.722 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617738+0000", 2024-09-20T22:11:04.722 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2310000000000001e-05 2024-09-20T22:11:04.723 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.723 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.723 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.723 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617746+0000", 2024-09-20T22:11:04.723 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3939999999999996e-06 2024-09-20T22:11:04.723 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.723 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.723 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.723 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617856+0000", 2024-09-20T22:11:04.723 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000109917 2024-09-20T22:11:04.723 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.723 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.723 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.723 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617878+0000", 2024-09-20T22:11:04.724 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2495999999999999e-05 2024-09-20T22:11:04.724 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.724 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.724 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.724 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410330+0000", 2024-09-20T22:11:04.724 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79245202299999995 2024-09-20T22:11:04.724 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.724 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.724 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.724 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410338+0000", 2024-09-20T22:11:04.724 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9740000000000002e-06 2024-09-20T22:11:04.724 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.724 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.724 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.725 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.725 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163158 8.11 8:8e0bce89:::benchmark_data_smithi060_111051_object43159:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:11:04.725 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:08.839869+0000", 2024-09-20T22:11:04.725 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 55.801792972999998, 2024-09-20T22:11:04.725 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.65193807500000001, 2024-09-20T22:11:04.725 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.725 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.725 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:04.725 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:04.725 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:04.725 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:04.725 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163158 2024-09-20T22:11:04.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.725 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.725 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.726 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.726 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839869+0000", 2024-09-20T22:11:04.726 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.726 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.726 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.726 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.726 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839869+0000", 2024-09-20T22:11:04.726 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.726 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.726 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.726 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.726 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839870+0000", 2024-09-20T22:11:04.726 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5710000000000001e-06 2024-09-20T22:11:04.726 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.727 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.727 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.727 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839876+0000", 2024-09-20T22:11:04.727 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3820000000000003e-06 2024-09-20T22:11:04.727 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.727 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.727 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.727 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839890+0000", 2024-09-20T22:11:04.727 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3952e-05 2024-09-20T22:11:04.727 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.727 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.727 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.727 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839897+0000", 2024-09-20T22:11:04.727 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4150000000000002e-06 2024-09-20T22:11:04.727 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.728 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.728 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.728 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839956+0000", 2024-09-20T22:11:04.728 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9157999999999998e-05 2024-09-20T22:11:04.728 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.728 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.728 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.728 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839967+0000", 2024-09-20T22:11:04.728 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1019000000000001e-05 2024-09-20T22:11:04.728 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.728 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.728 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.728 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477635+0000", 2024-09-20T22:11:04.728 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.63766765700000005 2024-09-20T22:11:04.728 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.729 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.729 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:04.729 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477737+0000", 2024-09-20T22:11:04.729 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010265899999999999 2024-09-20T22:11:04.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.729 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.729 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:11:04.729 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477939+0000", 2024-09-20T22:11:04.729 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020106900000000001 2024-09-20T22:11:04.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.729 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.729 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:04.729 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.482557+0000", 2024-09-20T22:11:04.729 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046186830000000002 2024-09-20T22:11:04.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.730 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.730 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.730 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491495+0000", 2024-09-20T22:11:04.730 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0089377699999999994 2024-09-20T22:11:04.730 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.730 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.730 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.730 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491673+0000", 2024-09-20T22:11:04.730 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017837500000000001 2024-09-20T22:11:04.730 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.730 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.730 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:04.730 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491762+0000", 2024-09-20T22:11:04.731 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8555999999999997e-05 2024-09-20T22:11:04.731 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.731 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.731 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.731 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491807+0000", 2024-09-20T22:11:04.731 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4808999999999997e-05 2024-09-20T22:11:04.731 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.731 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.731 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.731 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.731 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.731 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163289 8.11 8:8c47e2c8:::benchmark_data_smithi060_111051_object43290:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:11:04.731 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:08.954133+0000", 2024-09-20T22:11:04.731 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 55.687528338999996, 2024-09-20T22:11:04.732 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.53808307099999997, 2024-09-20T22:11:04.732 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.732 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.732 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:04.732 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:04.732 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:04.732 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:04.732 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163289 2024-09-20T22:11:04.732 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.732 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.732 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.732 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.732 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954133+0000", 2024-09-20T22:11:04.733 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.733 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.733 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.733 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954133+0000", 2024-09-20T22:11:04.733 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.733 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.733 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.733 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954134+0000", 2024-09-20T22:11:04.733 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2900000000000002e-07 2024-09-20T22:11:04.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.733 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.733 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.734 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954138+0000", 2024-09-20T22:11:04.734 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5260000000000002e-06 2024-09-20T22:11:04.734 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.734 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.734 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.734 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954147+0000", 2024-09-20T22:11:04.734 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2010000000000005e-06 2024-09-20T22:11:04.734 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.734 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.734 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.734 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954153+0000", 2024-09-20T22:11:04.734 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0610000000000001e-06 2024-09-20T22:11:04.734 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.734 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.735 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.735 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954209+0000", 2024-09-20T22:11:04.735 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5767999999999999e-05 2024-09-20T22:11:04.735 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.735 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.735 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.735 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954216+0000", 2024-09-20T22:11:04.735 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8099999999999998e-06 2024-09-20T22:11:04.735 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.735 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.735 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.735 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.478998+0000", 2024-09-20T22:11:04.735 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.52478124900000001 2024-09-20T22:11:04.735 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.736 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.736 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:04.736 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.479095+0000", 2024-09-20T22:11:04.736 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7125000000000003e-05 2024-09-20T22:11:04.736 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.736 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.736 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:11:04.736 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.479291+0000", 2024-09-20T22:11:04.736 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019642100000000001 2024-09-20T22:11:04.736 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.736 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.736 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:04.736 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.482759+0000", 2024-09-20T22:11:04.736 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0034672689999999998 2024-09-20T22:11:04.737 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.737 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.737 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.737 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491864+0000", 2024-09-20T22:11:04.737 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0091058630000000005 2024-09-20T22:11:04.737 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.737 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.737 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.737 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491961+0000", 2024-09-20T22:11:04.737 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6717000000000003e-05 2024-09-20T22:11:04.737 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.737 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.737 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:04.737 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492014+0000", 2024-09-20T22:11:04.738 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2877000000000002e-05 2024-09-20T22:11:04.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.738 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.738 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.738 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492216+0000", 2024-09-20T22:11:04.738 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020235500000000001 2024-09-20T22:11:04.738 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.738 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.738 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.738 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.738 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163324 8.11 8:8e312e9e:::benchmark_data_smithi060_111051_object43325:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:11:04.739 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:08.983848+0000", 2024-09-20T22:11:04.739 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 55.657813417, 2024-09-20T22:11:04.739 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.50861305999999995, 2024-09-20T22:11:04.739 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.739 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.739 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:04.739 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:04.739 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:04.739 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:04.739 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163324 2024-09-20T22:11:04.739 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.739 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.739 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.739 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.739 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983848+0000", 2024-09-20T22:11:04.740 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.740 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.740 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.740 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.740 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983848+0000", 2024-09-20T22:11:04.740 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.740 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.740 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.740 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.740 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983849+0000", 2024-09-20T22:11:04.740 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.11e-06 2024-09-20T22:11:04.740 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.740 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.740 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.741 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983853+0000", 2024-09-20T22:11:04.741 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0439999999999997e-06 2024-09-20T22:11:04.741 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.741 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.741 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.741 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983863+0000", 2024-09-20T22:11:04.741 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6430000000000007e-06 2024-09-20T22:11:04.741 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.741 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.741 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.741 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983868+0000", 2024-09-20T22:11:04.741 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1000000000000003e-06 2024-09-20T22:11:04.741 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.741 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.742 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.742 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983919+0000", 2024-09-20T22:11:04.742 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0433999999999999e-05 2024-09-20T22:11:04.742 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.742 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.742 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.742 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983928+0000", 2024-09-20T22:11:04.742 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3659999999999998e-06 2024-09-20T22:11:04.742 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.742 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.742 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.742 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.480646+0000", 2024-09-20T22:11:04.742 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.49671833999999998 2024-09-20T22:11:04.743 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.743 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.743 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:04.743 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.480768+0000", 2024-09-20T22:11:04.743 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012145 2024-09-20T22:11:04.743 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.743 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.743 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:11:04.743 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.481025+0000", 2024-09-20T22:11:04.743 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025713299999999999 2024-09-20T22:11:04.743 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.743 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.743 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:04.743 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.488496+0000", 2024-09-20T22:11:04.744 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0074709540000000001 2024-09-20T22:11:04.744 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.744 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.744 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.744 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492276+0000", 2024-09-20T22:11:04.744 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0037803670000000002 2024-09-20T22:11:04.744 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.744 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.744 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.744 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492350+0000", 2024-09-20T22:11:04.744 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3436999999999995e-05 2024-09-20T22:11:04.744 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.744 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.744 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:04.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:04.746 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492421+0000", 2024-09-20T22:11:04.746 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1303000000000005e-05 2024-09-20T22:11:04.746 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.746 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.746 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.746 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492461+0000", 2024-09-20T22:11:04.746 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0379000000000001e-05 2024-09-20T22:11:04.747 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.747 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.747 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.747 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.747 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.747 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163358 8.11 8:8b7ace29:::benchmark_data_smithi060_111051_object43359:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:11:04.747 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.014358+0000", 2024-09-20T22:11:04.747 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 55.627303431999998, 2024-09-20T22:11:04.747 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.47828052599999998, 2024-09-20T22:11:04.747 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.747 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.747 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:04.747 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:04.747 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:04.747 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:04.748 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163358 2024-09-20T22:11:04.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.748 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.748 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.748 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.748 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014358+0000", 2024-09-20T22:11:04.748 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.748 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.748 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.748 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014358+0000", 2024-09-20T22:11:04.748 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.748 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.749 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.749 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014359+0000", 2024-09-20T22:11:04.749 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0669999999999999e-06 2024-09-20T22:11:04.749 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.749 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.749 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.749 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014364+0000", 2024-09-20T22:11:04.749 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0220000000000004e-06 2024-09-20T22:11:04.749 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.749 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.749 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.749 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014374+0000", 2024-09-20T22:11:04.749 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9599999999999995e-06 2024-09-20T22:11:04.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.750 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.750 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.750 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014388+0000", 2024-09-20T22:11:04.750 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3413e-05 2024-09-20T22:11:04.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.750 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.750 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.750 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014447+0000", 2024-09-20T22:11:04.750 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9698999999999999e-05 2024-09-20T22:11:04.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.750 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.750 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.750 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014455+0000", 2024-09-20T22:11:04.751 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.072e-06 2024-09-20T22:11:04.751 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.751 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.751 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.751 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.482921+0000", 2024-09-20T22:11:04.751 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.46846513200000001 2024-09-20T22:11:04.751 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.751 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.751 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:04.751 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483020+0000", 2024-09-20T22:11:04.751 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9319000000000002e-05 2024-09-20T22:11:04.751 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.751 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.751 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:11:04.752 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483251+0000", 2024-09-20T22:11:04.752 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00023124300000000001 2024-09-20T22:11:04.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.752 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.752 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:04.752 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491608+0000", 2024-09-20T22:11:04.752 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0083564190000000003 2024-09-20T22:11:04.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.752 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.752 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.752 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492507+0000", 2024-09-20T22:11:04.752 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00089972900000000005 2024-09-20T22:11:04.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.752 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.753 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.753 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492546+0000", 2024-09-20T22:11:04.753 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9156999999999998e-05 2024-09-20T22:11:04.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.753 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.753 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:04.753 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492602+0000", 2024-09-20T22:11:04.753 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5114999999999997e-05 2024-09-20T22:11:04.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.753 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.753 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.753 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492639+0000", 2024-09-20T22:11:04.753 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7178999999999998e-05 2024-09-20T22:11:04.753 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.754 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.754 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.754 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.754 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.754 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163372 8.11 8:8b6ec81f:::benchmark_data_smithi060_111051_object43373:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:11:04.754 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.027195+0000", 2024-09-20T22:11:04.754 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 55.614466624000002, 2024-09-20T22:11:04.754 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.46568983899999999, 2024-09-20T22:11:04.754 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.754 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.754 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:04.754 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:04.754 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:04.754 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163372 2024-09-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027195+0000", 2024-09-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027195+0000", 2024-09-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.755 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027196+0000", 2024-09-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0599999999999999e-07 2024-09-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027200+0000", 2024-09-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9280000000000003e-06 2024-09-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027210+0000", 2024-09-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0035e-05 2024-09-20T22:11:04.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027217+0000", 2024-09-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6450000000000005e-06 2024-09-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027281+0000", 2024-09-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3281000000000006e-05 2024-09-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027291+0000", 2024-09-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0268999999999999e-05 2024-09-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.757 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.484647+0000", 2024-09-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.45735564499999998 2024-09-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.484745+0000", 2024-09-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8712e-05 2024-09-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.484940+0000", 2024-09-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019504900000000001 2024-09-20T22:11:04.758 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.759 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.759 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:04.759 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491619+0000", 2024-09-20T22:11:04.759 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0066782090000000001 2024-09-20T22:11:04.759 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.759 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.759 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.759 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492716+0000", 2024-09-20T22:11:04.759 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001097701 2024-09-20T22:11:04.759 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.759 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.759 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.759 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492778+0000", 2024-09-20T22:11:04.759 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1538000000000006e-05 2024-09-20T22:11:04.760 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.760 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.760 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:04.760 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492830+0000", 2024-09-20T22:11:04.760 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1997999999999999e-05 2024-09-20T22:11:04.760 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.760 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.760 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.760 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492885+0000", 2024-09-20T22:11:04.760 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5022999999999998e-05 2024-09-20T22:11:04.760 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.760 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.760 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163373 8.11 8:8d07a686:::benchmark_data_smithi060_111051_object43374:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.027847+0000", 2024-09-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 55.613814216000002, 2024-09-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.465200326, 2024-09-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163373 2024-09-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.761 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027847+0000", 2024-09-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027847+0000", 2024-09-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.762 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027848+0000", 2024-09-20T22:11:04.763 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3600000000000003e-07 2024-09-20T22:11:04.763 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.763 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.763 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.763 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027852+0000", 2024-09-20T22:11:04.763 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6330000000000002e-06 2024-09-20T22:11:04.763 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.763 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.763 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.763 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027860+0000", 2024-09-20T22:11:04.763 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2789999999999998e-06 2024-09-20T22:11:04.763 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.763 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.764 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.764 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027866+0000", 2024-09-20T22:11:04.764 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1070000000000001e-06 2024-09-20T22:11:04.764 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.764 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.764 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.764 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027897+0000", 2024-09-20T22:11:04.764 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0722000000000002e-05 2024-09-20T22:11:04.764 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.764 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.764 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.764 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027901+0000", 2024-09-20T22:11:04.765 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1649999999999999e-06 2024-09-20T22:11:04.765 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.765 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.765 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.765 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.486658+0000", 2024-09-20T22:11:04.765 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.45875733499999999 2024-09-20T22:11:04.765 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.765 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.765 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:04.765 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.486767+0000", 2024-09-20T22:11:04.765 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000108385 2024-09-20T22:11:04.765 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.765 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.765 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:11:04.765 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487120+0000", 2024-09-20T22:11:04.766 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00035312499999999998 2024-09-20T22:11:04.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.766 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.766 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:04.766 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491877+0000", 2024-09-20T22:11:04.766 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0047569170000000003 2024-09-20T22:11:04.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.766 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.766 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.766 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492928+0000", 2024-09-20T22:11:04.766 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001051633 2024-09-20T22:11:04.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.766 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.767 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.767 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492967+0000", 2024-09-20T22:11:04.767 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.8781e-05 2024-09-20T22:11:04.767 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.767 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.767 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:04.767 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493013+0000", 2024-09-20T22:11:04.767 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5555e-05 2024-09-20T22:11:04.767 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.767 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.767 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.767 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493048+0000", 2024-09-20T22:11:04.767 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5052999999999998e-05 2024-09-20T22:11:04.767 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.768 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.768 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.768 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.768 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163388 8.11 8:893f1034:::benchmark_data_smithi060_111051_object43389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:11:04.768 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.050114+0000", 2024-09-20T22:11:04.768 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 55.591547505999998, 2024-09-20T22:11:04.768 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.44308652799999998, 2024-09-20T22:11:04.768 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.768 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.768 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:04.768 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:04.768 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:04.768 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:04.769 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163388 2024-09-20T22:11:04.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.769 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.769 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.769 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.769 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050114+0000", 2024-09-20T22:11:04.769 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.769 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.769 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.769 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050114+0000", 2024-09-20T22:11:04.769 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.769 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.770 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.770 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050115+0000", 2024-09-20T22:11:04.770 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2500000000000005e-07 2024-09-20T22:11:04.770 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.770 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.770 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.770 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050119+0000", 2024-09-20T22:11:04.770 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.698e-06 2024-09-20T22:11:04.770 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.770 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.770 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.771 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050128+0000", 2024-09-20T22:11:04.771 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5820000000000008e-06 2024-09-20T22:11:04.771 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.771 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.771 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.771 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050133+0000", 2024-09-20T22:11:04.771 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9060000000000001e-06 2024-09-20T22:11:04.771 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.771 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.771 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.771 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050169+0000", 2024-09-20T22:11:04.771 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5984e-05 2024-09-20T22:11:04.771 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.772 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.772 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.772 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050174+0000", 2024-09-20T22:11:04.772 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0180000000000001e-06 2024-09-20T22:11:04.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.772 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.772 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.772 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.488568+0000", 2024-09-20T22:11:04.772 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.43839391 2024-09-20T22:11:04.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.772 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.772 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:04.772 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.488685+0000", 2024-09-20T22:11:04.773 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000117163 2024-09-20T22:11:04.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.773 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.773 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:11:04.773 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.489162+0000", 2024-09-20T22:11:04.773 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00047712600000000003 2024-09-20T22:11:04.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.773 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.773 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:04.773 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491884+0000", 2024-09-20T22:11:04.773 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0027215329999999999 2024-09-20T22:11:04.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.773 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.773 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.774 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493086+0000", 2024-09-20T22:11:04.774 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001201765 2024-09-20T22:11:04.774 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.774 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.774 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.774 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493122+0000", 2024-09-20T22:11:04.774 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6622999999999998e-05 2024-09-20T22:11:04.774 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.774 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.774 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:04.774 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493168+0000", 2024-09-20T22:11:04.775 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5757999999999998e-05 2024-09-20T22:11:04.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.775 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.775 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.775 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493201+0000", 2024-09-20T22:11:04.775 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2737000000000002e-05 2024-09-20T22:11:04.775 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.775 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.775 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.775 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.775 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163393 8.11 8:8bb06d42:::benchmark_data_smithi060_111051_object43394:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:11:04.775 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.064776+0000", 2024-09-20T22:11:04.776 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 55.576885582999999, 2024-09-20T22:11:04.776 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.43944195600000002, 2024-09-20T22:11:04.776 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:04.776 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:04.776 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:04.776 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:04.776 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:04.776 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:04.776 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163393 2024-09-20T22:11:04.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.776 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:04.776 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.776 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:04.776 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064776+0000", 2024-09-20T22:11:04.777 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.777 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.777 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.777 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:04.777 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064776+0000", 2024-09-20T22:11:04.777 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:04.777 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.777 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.777 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:04.777 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064777+0000", 2024-09-20T22:11:04.777 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9500000000000001e-07 2024-09-20T22:11:04.777 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.777 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.777 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:04.778 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064781+0000", 2024-09-20T22:11:04.778 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9870000000000001e-06 2024-09-20T22:11:04.778 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.778 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.778 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:04.778 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064791+0000", 2024-09-20T22:11:04.778 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0417999999999999e-05 2024-09-20T22:11:04.778 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.778 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.778 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:04.778 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064797+0000", 2024-09-20T22:11:04.778 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4040000000000003e-06 2024-09-20T22:11:04.778 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.779 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.779 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.779 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064838+0000", 2024-09-20T22:11:04.779 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1591000000000002e-05 2024-09-20T22:11:04.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.779 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.779 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:04.779 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064843+0000", 2024-09-20T22:11:04.779 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9470000000000002e-06 2024-09-20T22:11:04.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.779 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.779 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:04.779 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493235+0000", 2024-09-20T22:11:04.779 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.42839186899999998 2024-09-20T22:11:04.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.780 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.780 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:04.780 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493336+0000", 2024-09-20T22:11:04.780 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100923 2024-09-20T22:11:04.780 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.780 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.780 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:11:04.780 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493659+0000", 2024-09-20T22:11:04.780 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00032341599999999998 2024-09-20T22:11:04.780 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.780 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.780 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:04.780 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.499928+0000", 2024-09-20T22:11:04.780 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0062687730000000001 2024-09-20T22:11:04.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.781 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.781 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.781 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.503797+0000", 2024-09-20T22:11:04.781 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0038686020000000001 2024-09-20T22:11:04.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.781 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.781 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:04.781 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.504160+0000", 2024-09-20T22:11:04.781 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00036300900000000002 2024-09-20T22:11:04.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.781 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.781 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:04.781 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.504192+0000", 2024-09-20T22:11:04.782 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2592000000000002e-05 2024-09-20T22:11:04.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:04.782 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:04.782 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:04.782 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.504218+0000", 2024-09-20T22:11:04.782 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.563e-05 2024-09-20T22:11:04.782 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.782 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.782 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.782 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:04.782 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:04.782 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:11:04.783 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:04 smithi067 ceph-mon[33511]: 4.1b scrub starts 2024-09-20T22:11:04.783 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:04 smithi067 ceph-mon[33511]: 4.1b scrub ok 2024-09-20T22:11:04.783 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:04 smithi067 ceph-mon[33511]: 8.a deep-scrub starts 2024-09-20T22:11:04.783 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:04 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:11:04.783 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:04 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:11:04.783 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:04 smithi067 ceph-mon[33511]: 6.5 scrub starts 2024-09-20T22:11:04.783 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:04 smithi067 ceph-mon[33511]: 6.5 scrub ok 2024-09-20T22:11:04.783 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:04 smithi067 ceph-mon[33511]: osdmap e271: 8 total, 8 up, 8 in 2024-09-20T22:11:04.783 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:04 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/537807177' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:04.784 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:04 smithi067 ceph-mon[33511]: pgmap v433: 168 pgs: 1 active+clean+scrubbing, 3 active+remapped+backfill_toofull, 164 active+clean; 1.3 GiB data, 7.6 GiB used, 708 GiB / 715 GiB avail; 852 B/s rd, 0 op/s; 4034/66237 objects misplaced (6.090%) 2024-09-20T22:11:04.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:04.872 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:04.873 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:04.973 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:05.009 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:05.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:05.010 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:05.111 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:05.138 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:05.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:05.140 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:05.240 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:05.278 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:05.288 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:05.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:05.290 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:05.390 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:05.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:05.421 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:05.421 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:05.427 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:11:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:11:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:11:05.522 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:05.568 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:05.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:05.569 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:05.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:05.706 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:05.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:05.707 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:05.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:05 smithi060 ceph-mon[29694]: 3.6 scrub starts 2024-09-20T22:11:05.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:05 smithi060 ceph-mon[29694]: 3.6 scrub ok 2024-09-20T22:11:05.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:11:05.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:11:05.758 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:05 smithi060 ceph-mon[40290]: 3.6 scrub starts 2024-09-20T22:11:05.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:05 smithi060 ceph-mon[40290]: 3.6 scrub ok 2024-09-20T22:11:05.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:11:05.759 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:11:05.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:05.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:05.853 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:05.854 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:05.954 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:05.977 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:05.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:05.978 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:05 smithi067 ceph-mon[33511]: 3.6 scrub starts 2024-09-20T22:11:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:05 smithi067 ceph-mon[33511]: 3.6 scrub ok 2024-09-20T22:11:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:11:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:11:06.079 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:06.108 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:06.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:06.109 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:06.165 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T22:11:06.210 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:06.239 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:06.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:06.241 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:06.342 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:06.382 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:06.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:06.383 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:06.453 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.1 to 1 (10000) 2024-09-20T22:11:06.484 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:06.518 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:06.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:06.519 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:06.545 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:11:06.620 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:06.659 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:06.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:06.660 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:06.761 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:06.785 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:11:06.786 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:06.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:06.787 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:06.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[29694]: 6.17 scrub starts 2024-09-20T22:11:06.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[29694]: 6.17 scrub ok 2024-09-20T22:11:06.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[29694]: 4.4 deep-scrub starts 2024-09-20T22:11:06.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[29694]: 4.4 deep-scrub ok 2024-09-20T22:11:06.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[29694]: 6.12 deep-scrub starts 2024-09-20T22:11:06.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[29694]: 6.12 deep-scrub ok 2024-09-20T22:11:06.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[29694]: 6.14 scrub starts 2024-09-20T22:11:06.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[29694]: 6.14 scrub ok 2024-09-20T22:11:06.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[29694]: 6.16 scrub starts 2024-09-20T22:11:06.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[29694]: 6.16 scrub ok 2024-09-20T22:11:06.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[29694]: pgmap v434: 168 pgs: 1 active+clean+remapped, 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 164 active+clean; 1.2 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 72 op/s; 3320/59502 objects misplaced (5.580%) 2024-09-20T22:11:06.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3162495182' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:11:06.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3162495182' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T22:11:06.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[29694]: osdmap e272: 8 total, 8 up, 8 in 2024-09-20T22:11:06.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[40290]: 6.17 scrub starts 2024-09-20T22:11:06.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[40290]: 6.17 scrub ok 2024-09-20T22:11:06.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[40290]: 4.4 deep-scrub starts 2024-09-20T22:11:06.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[40290]: 4.4 deep-scrub ok 2024-09-20T22:11:06.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[40290]: 6.12 deep-scrub starts 2024-09-20T22:11:06.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[40290]: 6.12 deep-scrub ok 2024-09-20T22:11:06.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[40290]: 6.14 scrub starts 2024-09-20T22:11:06.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[40290]: 6.14 scrub ok 2024-09-20T22:11:06.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[40290]: 6.16 scrub starts 2024-09-20T22:11:06.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[40290]: 6.16 scrub ok 2024-09-20T22:11:06.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[40290]: pgmap v434: 168 pgs: 1 active+clean+remapped, 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 164 active+clean; 1.2 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 72 op/s; 3320/59502 objects misplaced (5.580%) 2024-09-20T22:11:06.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3162495182' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:11:06.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3162495182' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T22:11:06.790 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:06 smithi060 ceph-mon[40290]: osdmap e272: 8 total, 8 up, 8 in 2024-09-20T22:11:06.843 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:06 smithi067 ceph-mon[33511]: 6.17 scrub starts 2024-09-20T22:11:06.843 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:06 smithi067 ceph-mon[33511]: 6.17 scrub ok 2024-09-20T22:11:06.844 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:06 smithi067 ceph-mon[33511]: 4.4 deep-scrub starts 2024-09-20T22:11:06.844 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:06 smithi067 ceph-mon[33511]: 4.4 deep-scrub ok 2024-09-20T22:11:06.844 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:06 smithi067 ceph-mon[33511]: 6.12 deep-scrub starts 2024-09-20T22:11:06.844 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:06 smithi067 ceph-mon[33511]: 6.12 deep-scrub ok 2024-09-20T22:11:06.844 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:06 smithi067 ceph-mon[33511]: 6.14 scrub starts 2024-09-20T22:11:06.844 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:06 smithi067 ceph-mon[33511]: 6.14 scrub ok 2024-09-20T22:11:06.844 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:06 smithi067 ceph-mon[33511]: 6.16 scrub starts 2024-09-20T22:11:06.844 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:06 smithi067 ceph-mon[33511]: 6.16 scrub ok 2024-09-20T22:11:06.844 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:06 smithi067 ceph-mon[33511]: pgmap v434: 168 pgs: 1 active+clean+remapped, 1 active+clean+scrubbing, 2 active+remapped+backfill_toofull, 164 active+clean; 1.2 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 72 op/s; 3320/59502 objects misplaced (5.580%) 2024-09-20T22:11:06.844 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:06 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3162495182' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:11:06.844 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:06 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3162495182' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T22:11:06.844 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:06 smithi067 ceph-mon[33511]: osdmap e272: 8 total, 8 up, 8 in 2024-09-20T22:11:06.888 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:06.926 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:06.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:06.928 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:07.029 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:07.052 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:07.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:07.053 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:07.116 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:11:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:11:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223355 8.9 8.efdfed29 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:11:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.561400+0000", 2024-09-20T22:11:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.55313483200000002, 2024-09-20T22:11:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.55315461499999996, 2024-09-20T22:11:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:11:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:07.117 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223355 2024-09-20T22:11:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561400+0000", 2024-09-20T22:11:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561400+0000", 2024-09-20T22:11:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561401+0000", 2024-09-20T22:11:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3790000000000001e-06 2024-09-20T22:11:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561406+0000", 2024-09-20T22:11:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4559999999999999e-06 2024-09-20T22:11:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561419+0000", 2024-09-20T22:11:07.119 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2712e-05 2024-09-20T22:11:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561426+0000", 2024-09-20T22:11:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6080000000000001e-06 2024-09-20T22:11:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561475+0000", 2024-09-20T22:11:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9412000000000001e-05 2024-09-20T22:11:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:07.120 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561482+0000", 2024-09-20T22:11:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7789999999999998e-06 2024-09-20T22:11:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223360 8.9 8.77d47c69 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:11:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.566084+0000", 2024-09-20T22:11:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.54845015399999997, 2024-09-20T22:11:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.54851222700000002, 2024-09-20T22:11:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:11:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:07.121 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223360 2024-09-20T22:11:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566084+0000", 2024-09-20T22:11:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566084+0000", 2024-09-20T22:11:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.122 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566086+0000", 2024-09-20T22:11:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.175e-06 2024-09-20T22:11:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566090+0000", 2024-09-20T22:11:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2080000000000002e-06 2024-09-20T22:11:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566099+0000", 2024-09-20T22:11:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3559999999999999e-06 2024-09-20T22:11:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.123 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:07.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566105+0000", 2024-09-20T22:11:07.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8649999999999998e-06 2024-09-20T22:11:07.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.124 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:07.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566150+0000", 2024-09-20T22:11:07.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5252e-05 2024-09-20T22:11:07.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.124 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:07.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566156+0000", 2024-09-20T22:11:07.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7370000000000003e-06 2024-09-20T22:11:07.124 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.124 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:07.124 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.125 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223529 8.9 8.acbcc349 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:11:07.125 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.731511+0000", 2024-09-20T22:11:07.125 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.38302370000000002, 2024-09-20T22:11:07.125 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.38312259700000001, 2024-09-20T22:11:07.125 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:07.125 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:07.125 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:11:07.125 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:07.125 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:07.125 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:07.125 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223529 2024-09-20T22:11:07.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.125 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:07.126 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.126 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:07.126 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.731511+0000", 2024-09-20T22:11:07.126 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.126 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.126 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:07.126 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.731511+0000", 2024-09-20T22:11:07.126 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.126 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.126 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:07.126 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.731513+0000", 2024-09-20T22:11:07.126 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.21e-06 2024-09-20T22:11:07.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.127 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:07.127 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.731525+0000", 2024-09-20T22:11:07.127 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.207e-05 2024-09-20T22:11:07.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.127 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:07.127 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.731556+0000", 2024-09-20T22:11:07.127 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1343000000000001e-05 2024-09-20T22:11:07.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.127 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:07.127 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.731576+0000", 2024-09-20T22:11:07.127 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9179999999999999e-05 2024-09-20T22:11:07.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.128 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:07.128 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.731702+0000", 2024-09-20T22:11:07.128 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000126193 2024-09-20T22:11:07.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.128 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:07.128 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.731728+0000", 2024-09-20T22:11:07.128 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.6588999999999999e-05 2024-09-20T22:11:07.128 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.128 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:07.128 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.128 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223679 8.9 8.64659a9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:11:07.129 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:07.092501+0000", 2024-09-20T22:11:07.129 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.022033489999999999, 2024-09-20T22:11:07.129 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.022166219000000001, 2024-09-20T22:11:07.129 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:07.129 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:07.129 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:11:07.129 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:07.129 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:07.129 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:07.129 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223679 2024-09-20T22:11:07.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.129 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:07.129 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.129 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:07.129 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.092501+0000", 2024-09-20T22:11:07.129 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.130 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:07.130 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.092501+0000", 2024-09-20T22:11:07.130 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.131 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:07.131 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.092502+0000", 2024-09-20T22:11:07.131 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.204e-06 2024-09-20T22:11:07.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.131 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:07.131 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.092508+0000", 2024-09-20T22:11:07.131 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4620000000000004e-06 2024-09-20T22:11:07.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.132 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.132 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:07.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.092521+0000", 2024-09-20T22:11:07.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3154e-05 2024-09-20T22:11:07.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.132 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.132 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:07.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.092529+0000", 2024-09-20T22:11:07.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6829999999999997e-06 2024-09-20T22:11:07.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.132 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.132 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:07.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.092594+0000", 2024-09-20T22:11:07.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4455000000000005e-05 2024-09-20T22:11:07.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.133 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.133 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:07.133 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.092601+0000", 2024-09-20T22:11:07.133 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3560000000000004e-06 2024-09-20T22:11:07.133 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.133 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:07.133 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.133 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.133 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223604 8.9 8.a7f2f2a9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:11:07.133 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.825314+0000", 2024-09-20T22:11:07.133 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.28922003299999999, 2024-09-20T22:11:07.133 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.28938775700000002, 2024-09-20T22:11:07.134 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:07.134 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:07.134 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:11:07.134 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:07.134 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:07.134 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:07.134 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223604 2024-09-20T22:11:07.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.134 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:07.134 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.134 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:07.134 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.825314+0000", 2024-09-20T22:11:07.134 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.135 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:07.135 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.825314+0000", 2024-09-20T22:11:07.135 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.135 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:07.135 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.825316+0000", 2024-09-20T22:11:07.135 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1999999999999999e-06 2024-09-20T22:11:07.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.135 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:07.135 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.825321+0000", 2024-09-20T22:11:07.135 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8829999999999997e-06 2024-09-20T22:11:07.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.136 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:07.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.825333+0000", 2024-09-20T22:11:07.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2123000000000001e-05 2024-09-20T22:11:07.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.136 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:07.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.825339+0000", 2024-09-20T22:11:07.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.976e-06 2024-09-20T22:11:07.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.136 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:07.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.825448+0000", 2024-09-20T22:11:07.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010966 2024-09-20T22:11:07.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.137 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.137 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:07.137 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.825460+0000", 2024-09-20T22:11:07.137 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1515000000000001e-05 2024-09-20T22:11:07.137 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.137 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:07.137 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.137 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.137 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223616 8.9 8.10dd1329 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:11:07.137 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.850627+0000", 2024-09-20T22:11:07.137 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.26390735500000001, 2024-09-20T22:11:07.137 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.26410894099999999, 2024-09-20T22:11:07.137 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:07.137 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:07.138 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:11:07.138 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:07.138 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:07.138 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:07.138 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223616 2024-09-20T22:11:07.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.138 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:07.138 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.138 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:07.138 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.850627+0000", 2024-09-20T22:11:07.138 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.138 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.138 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:07.139 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.850627+0000", 2024-09-20T22:11:07.139 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.139 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.139 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:07.139 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.850629+0000", 2024-09-20T22:11:07.139 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4959999999999999e-06 2024-09-20T22:11:07.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.139 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.139 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:07.139 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.850635+0000", 2024-09-20T22:11:07.139 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1890000000000004e-06 2024-09-20T22:11:07.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.139 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.140 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:07.140 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.850652+0000", 2024-09-20T22:11:07.140 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7476999999999999e-05 2024-09-20T22:11:07.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.140 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.140 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:07.140 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.850661+0000", 2024-09-20T22:11:07.140 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8410000000000006e-06 2024-09-20T22:11:07.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.140 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.140 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:07.140 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.850712+0000", 2024-09-20T22:11:07.140 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1050000000000001e-05 2024-09-20T22:11:07.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.140 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.141 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:07.141 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.850718+0000", 2024-09-20T22:11:07.141 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7980000000000002e-06 2024-09-20T22:11:07.141 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.141 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:07.141 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.141 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223546 8.9 8.2e100f29 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:11:07.141 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.748360+0000", 2024-09-20T22:11:07.141 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.366174375, 2024-09-20T22:11:07.141 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.366409598, 2024-09-20T22:11:07.141 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:07.141 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:07.141 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:11:07.142 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:07.142 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:07.142 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:07.142 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223546 2024-09-20T22:11:07.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.142 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:07.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.142 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:07.142 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.748360+0000", 2024-09-20T22:11:07.142 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.142 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:07.142 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.748360+0000", 2024-09-20T22:11:07.143 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.143 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:07.143 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.748361+0000", 2024-09-20T22:11:07.143 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.108e-06 2024-09-20T22:11:07.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.143 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:07.143 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.748366+0000", 2024-09-20T22:11:07.143 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3420000000000001e-06 2024-09-20T22:11:07.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.143 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:07.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.748377+0000", 2024-09-20T22:11:07.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1877e-05 2024-09-20T22:11:07.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:07.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.748391+0000", 2024-09-20T22:11:07.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3781000000000001e-05 2024-09-20T22:11:07.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:07.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.748454+0000", 2024-09-20T22:11:07.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2561e-05 2024-09-20T22:11:07.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:07.145 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.748464+0000", 2024-09-20T22:11:07.145 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0409e-05 2024-09-20T22:11:07.145 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.145 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:07.145 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.145 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223308 8.9 8.908de7e9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:11:07.145 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.506871+0000", 2024-09-20T22:11:07.145 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.60766321000000001, 2024-09-20T22:11:07.145 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.60793293500000001, 2024-09-20T22:11:07.145 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:07.145 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:07.145 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:11:07.146 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:07.146 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:07.146 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:07.146 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223308 2024-09-20T22:11:07.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.146 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:07.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.146 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:07.146 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506871+0000", 2024-09-20T22:11:07.146 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.146 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:07.146 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506871+0000", 2024-09-20T22:11:07.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.147 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:07.147 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506873+0000", 2024-09-20T22:11:07.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0470000000000001e-06 2024-09-20T22:11:07.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.147 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:07.147 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506882+0000", 2024-09-20T22:11:07.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3310000000000004e-06 2024-09-20T22:11:07.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.147 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:07.148 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506904+0000", 2024-09-20T22:11:07.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.283e-05 2024-09-20T22:11:07.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.148 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:07.148 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506913+0000", 2024-09-20T22:11:07.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9530000000000005e-06 2024-09-20T22:11:07.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.148 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:07.148 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506975+0000", 2024-09-20T22:11:07.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1773000000000006e-05 2024-09-20T22:11:07.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.148 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:07.149 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506982+0000", 2024-09-20T22:11:07.149 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2869999999999999e-06 2024-09-20T22:11:07.149 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.149 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:07.149 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.149 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223310 8.9 8.8a4e4029 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:11:07.149 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.507858+0000", 2024-09-20T22:11:07.149 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.60667661699999997, 2024-09-20T22:11:07.149 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.60697928400000001, 2024-09-20T22:11:07.149 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:07.149 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:07.149 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:11:07.150 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:07.150 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:07.150 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:07.150 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223310 2024-09-20T22:11:07.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.150 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:07.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.150 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:07.150 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507858+0000", 2024-09-20T22:11:07.150 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.150 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:07.150 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507858+0000", 2024-09-20T22:11:07.151 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.151 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:07.151 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507859+0000", 2024-09-20T22:11:07.151 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.78e-07 2024-09-20T22:11:07.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.151 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:07.151 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507864+0000", 2024-09-20T22:11:07.151 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8969999999999999e-06 2024-09-20T22:11:07.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.151 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:07.152 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507879+0000", 2024-09-20T22:11:07.152 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4919999999999999e-05 2024-09-20T22:11:07.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.152 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:07.152 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507886+0000", 2024-09-20T22:11:07.152 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2119999999999997e-06 2024-09-20T22:11:07.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.152 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:07.152 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507948+0000", 2024-09-20T22:11:07.152 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1764e-05 2024-09-20T22:11:07.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.152 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:07.153 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507959+0000", 2024-09-20T22:11:07.153 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1613999999999999e-05 2024-09-20T22:11:07.153 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.153 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:07.153 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.153 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.153 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223451 8.9 8.25e85909 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:11:07.153 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.655987+0000", 2024-09-20T22:11:07.153 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.45854784199999998, 2024-09-20T22:11:07.153 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.45888413500000003, 2024-09-20T22:11:07.153 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:07.153 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:07.153 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:11:07.153 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:07.154 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:07.154 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:07.154 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223451 2024-09-20T22:11:07.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.154 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:07.154 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.154 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:07.154 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.655987+0000", 2024-09-20T22:11:07.154 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.154 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.154 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:07.154 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.655987+0000", 2024-09-20T22:11:07.154 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.155 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.155 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:07.155 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.655988+0000", 2024-09-20T22:11:07.155 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.229e-06 2024-09-20T22:11:07.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.155 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.155 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:07.155 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.655994+0000", 2024-09-20T22:11:07.155 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8860000000000001e-06 2024-09-20T22:11:07.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.155 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.155 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:07.155 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.656009+0000", 2024-09-20T22:11:07.155 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5709000000000002e-05 2024-09-20T22:11:07.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.156 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.156 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:07.156 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.656019+0000", 2024-09-20T22:11:07.156 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.149e-06 2024-09-20T22:11:07.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.156 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.156 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:07.156 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.656083+0000", 2024-09-20T22:11:07.156 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4739999999999993e-05 2024-09-20T22:11:07.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.156 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.156 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:07.156 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.656090+0000", 2024-09-20T22:11:07.156 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8510000000000001e-06 2024-09-20T22:11:07.157 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.157 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:07.157 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.157 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.157 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223557 8.9 8.29a0b749 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:11:07.157 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.760409+0000", 2024-09-20T22:11:07.157 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.35412564000000002, 2024-09-20T22:11:07.157 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.354495226, 2024-09-20T22:11:07.157 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:07.157 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:07.157 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:11:07.157 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:07.157 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:07.157 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:07.158 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223557 2024-09-20T22:11:07.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.158 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:07.158 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.158 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:07.158 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.760409+0000", 2024-09-20T22:11:07.158 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.158 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.158 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:07.158 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.760409+0000", 2024-09-20T22:11:07.158 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.159 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:07.159 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.760410+0000", 2024-09-20T22:11:07.159 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7100000000000001e-07 2024-09-20T22:11:07.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.159 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:07.159 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.760412+0000", 2024-09-20T22:11:07.159 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.9040000000000001e-06 2024-09-20T22:11:07.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.159 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:07.159 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.760423+0000", 2024-09-20T22:11:07.160 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0925e-05 2024-09-20T22:11:07.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.160 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.160 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:07.160 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.760428+0000", 2024-09-20T22:11:07.160 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5700000000000003e-06 2024-09-20T22:11:07.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.160 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.160 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:07.160 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.760480+0000", 2024-09-20T22:11:07.160 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1894999999999997e-05 2024-09-20T22:11:07.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.160 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.160 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:07.160 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.760486+0000", 2024-09-20T22:11:07.161 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7379999999999999e-06 2024-09-20T22:11:07.161 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.161 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:07.161 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.161 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.161 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223561 8.9 8.634d1189 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:11:07.161 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.768516+0000", 2024-09-20T22:11:07.161 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.34601872, 2024-09-20T22:11:07.161 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.34642274200000001, 2024-09-20T22:11:07.161 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:07.161 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:07.161 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:11:07.161 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:07.162 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:07.162 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:07.162 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223561 2024-09-20T22:11:07.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.162 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:07.162 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.162 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:07.162 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.768516+0000", 2024-09-20T22:11:07.162 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.162 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.162 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:07.162 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.768516+0000", 2024-09-20T22:11:07.162 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.163 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:07.163 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.768517+0000", 2024-09-20T22:11:07.163 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.35e-06 2024-09-20T22:11:07.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.163 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:07.163 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.768521+0000", 2024-09-20T22:11:07.163 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1980000000000003e-06 2024-09-20T22:11:07.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.163 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:07.163 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.768533+0000", 2024-09-20T22:11:07.163 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.188e-05 2024-09-20T22:11:07.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.164 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:07.164 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.768540+0000", 2024-09-20T22:11:07.164 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5969999999999997e-06 2024-09-20T22:11:07.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.164 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:07.164 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.768589+0000", 2024-09-20T22:11:07.164 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9075e-05 2024-09-20T22:11:07.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.164 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:07.164 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.768595+0000", 2024-09-20T22:11:07.164 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8370000000000002e-06 2024-09-20T22:11:07.164 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.165 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:07.165 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.165 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.165 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223467 8.9 8.f887af49 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:11:07.165 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.673500+0000", 2024-09-20T22:11:07.165 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.44103456200000002, 2024-09-20T22:11:07.165 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.44147220399999998, 2024-09-20T22:11:07.165 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:07.165 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:07.165 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:11:07.165 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:07.165 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:07.165 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:07.165 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223467 2024-09-20T22:11:07.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.166 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:07.166 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.166 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:07.166 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.673500+0000", 2024-09-20T22:11:07.166 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.166 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.166 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:07.166 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.673500+0000", 2024-09-20T22:11:07.166 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.166 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.166 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:07.167 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.673501+0000", 2024-09-20T22:11:07.167 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.212e-06 2024-09-20T22:11:07.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.167 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.167 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:07.167 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.673505+0000", 2024-09-20T22:11:07.167 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3270000000000002e-06 2024-09-20T22:11:07.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.167 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.167 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:07.167 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.673517+0000", 2024-09-20T22:11:07.167 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1545e-05 2024-09-20T22:11:07.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.167 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.167 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:07.168 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.673522+0000", 2024-09-20T22:11:07.168 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.101e-06 2024-09-20T22:11:07.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.168 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.168 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:07.168 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.673580+0000", 2024-09-20T22:11:07.168 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8102e-05 2024-09-20T22:11:07.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.168 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.168 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:07.168 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.673587+0000", 2024-09-20T22:11:07.168 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3060000000000003e-06 2024-09-20T22:11:07.168 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.168 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:07.168 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.169 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.169 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223643 8.9 8.8c3d60a9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:11:07.169 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.905858+0000", 2024-09-20T22:11:07.169 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.208676579, 2024-09-20T22:11:07.169 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.20915383000000001, 2024-09-20T22:11:07.169 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:07.169 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:07.169 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:11:07.169 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:07.169 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:07.169 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:07.169 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223643 2024-09-20T22:11:07.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.169 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:07.170 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.170 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:07.170 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.905858+0000", 2024-09-20T22:11:07.170 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.170 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.170 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.170 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:07.170 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.905858+0000", 2024-09-20T22:11:07.170 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.170 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.170 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.170 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:07.170 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.905859+0000", 2024-09-20T22:11:07.170 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.128e-06 2024-09-20T22:11:07.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.171 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.171 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:07.171 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.905864+0000", 2024-09-20T22:11:07.171 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.045e-06 2024-09-20T22:11:07.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.171 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.171 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:07.171 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.905876+0000", 2024-09-20T22:11:07.171 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1703e-05 2024-09-20T22:11:07.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.171 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.171 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:07.171 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.905883+0000", 2024-09-20T22:11:07.171 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8229999999999997e-06 2024-09-20T22:11:07.172 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.172 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.172 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:07.172 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.905933+0000", 2024-09-20T22:11:07.172 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0797999999999997e-05 2024-09-20T22:11:07.172 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:07.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.173 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.173 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:07.173 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.905938+0000", 2024-09-20T22:11:07.173 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9880000000000004e-06 2024-09-20T22:11:07.173 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.173 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:07.173 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.173 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.173 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223581 8.9 8.c3e0b569 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:11:07.174 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.791673+0000", 2024-09-20T22:11:07.174 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.32286160600000002, 2024-09-20T22:11:07.174 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.32337555400000001, 2024-09-20T22:11:07.174 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:07.174 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:07.174 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:11:07.174 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:07.174 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:07.174 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:07.174 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223581 2024-09-20T22:11:07.174 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.174 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:07.174 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.174 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:07.174 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.791673+0000", 2024-09-20T22:11:07.175 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.175 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.175 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:07.175 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.791673+0000", 2024-09-20T22:11:07.175 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.175 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.175 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:07.175 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.791674+0000", 2024-09-20T22:11:07.175 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5549999999999999e-06 2024-09-20T22:11:07.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.175 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.175 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.791680+0000", 2024-09-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1479999999999997e-06 2024-09-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.791692+0000", 2024-09-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2723e-05 2024-09-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.791698+0000", 2024-09-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0519999999999998e-06 2024-09-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:07.176 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.791746+0000", 2024-09-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.795e-05 2024-09-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.791752+0000", 2024-09-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1380000000000004e-06 2024-09-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223650 8.9 8.59457de9 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.923953+0000", 2024-09-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.190581367, 2024-09-20T22:11:07.177 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.19112849400000001, 2024-09-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for peered", 2024-09-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223650 2024-09-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.923953+0000", 2024-09-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.178 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.179 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.179 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:07.179 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.923953+0000", 2024-09-20T22:11:07.179 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:07.179 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.179 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.179 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:07.179 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.923954+0000", 2024-09-20T22:11:07.179 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2219999999999999e-06 2024-09-20T22:11:07.179 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.179 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.179 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:07.179 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.923962+0000", 2024-09-20T22:11:07.179 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3780000000000003e-06 2024-09-20T22:11:07.180 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.180 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.180 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:07.180 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.923973+0000", 2024-09-20T22:11:07.180 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1154e-05 2024-09-20T22:11:07.180 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.180 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.180 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:07.180 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.923978+0000", 2024-09-20T22:11:07.180 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0180000000000001e-06 2024-09-20T22:11:07.180 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.180 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.180 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:07.180 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.924024+0000", 2024-09-20T22:11:07.180 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6548999999999998e-05 2024-09-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.924030+0000", 2024-09-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7030000000000003e-06 2024-09-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi067.stdout: "num_ops": 16 2024-09-20T22:11:07.181 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:11:07.202 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:07.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:07.203 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:07.224 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:11:07.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:07.330 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:07.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:07.331 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:07.432 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:07.461 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:07.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:07.462 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:07.555 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 2 1 2024-09-20T22:11:07.563 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:07.590 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:07.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:07.591 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:07.687 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:07.692 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:07.716 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:07.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:07.717 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:07.769 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:07.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:07 smithi060 ceph-mon[29694]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:11:07.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:07 smithi060 ceph-mon[29694]: 1.7 scrub starts 2024-09-20T22:11:07.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:07 smithi060 ceph-mon[29694]: 1.7 scrub ok 2024-09-20T22:11:07.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:07 smithi060 ceph-mon[29694]: osdmap e273: 8 total, 8 up, 8 in 2024-09-20T22:11:07.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:07 smithi060 ceph-mon[40290]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:11:07.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:07 smithi060 ceph-mon[40290]: 1.7 scrub starts 2024-09-20T22:11:07.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:07 smithi060 ceph-mon[40290]: 1.7 scrub ok 2024-09-20T22:11:07.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:07 smithi060 ceph-mon[40290]: osdmap e273: 8 total, 8 up, 8 in 2024-09-20T22:11:07.818 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:07.844 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:07.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:07.845 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:07.946 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:08.013 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:08.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:07 smithi067 ceph-mon[33511]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:11:08.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:07 smithi067 ceph-mon[33511]: 1.7 scrub starts 2024-09-20T22:11:08.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:07 smithi067 ceph-mon[33511]: 1.7 scrub ok 2024-09-20T22:11:08.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:07 smithi067 ceph-mon[33511]: osdmap e273: 8 total, 8 up, 8 in 2024-09-20T22:11:08.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:08.014 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:08.056 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T22:11:08.115 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:08.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:08.162 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:08.163 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:08.263 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:08.300 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:08.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:08.301 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:08.401 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:08.434 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:11:08.443 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:08.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:08.444 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:08.545 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:08.596 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:08.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:08.597 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:08.682 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:11:08.682 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:11:08.697 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:08.717 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:11:08.717 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:11:08.727 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:08.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:08.728 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:08.753 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:11:08.753 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:11:08.753 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:08 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:11:08.753 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:08 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:11:08.753 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:08 smithi060 ceph-mon[29694]: 3.9 scrub starts 2024-09-20T22:11:08.753 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:08 smithi060 ceph-mon[29694]: 3.9 scrub ok 2024-09-20T22:11:08.753 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:08 smithi060 ceph-mon[29694]: 4.17 scrub starts 2024-09-20T22:11:08.753 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:08 smithi060 ceph-mon[29694]: 4.17 scrub ok 2024-09-20T22:11:08.754 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:08 smithi060 ceph-mon[29694]: pgmap v437: 168 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 2 active+remapped+backfill_toofull, 164 active+clean; 1.0 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 911 B/s rd, 0 B/s wr, 1.02k op/s; 2519/51330 objects misplaced (4.907%); 7.0 MiB/s, 111 objects/s recovering 2024-09-20T22:11:08.754 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:08 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:11:08.754 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:08 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:11:08.754 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:08 smithi060 ceph-mon[40290]: 3.9 scrub starts 2024-09-20T22:11:08.754 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:08 smithi060 ceph-mon[40290]: 3.9 scrub ok 2024-09-20T22:11:08.754 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:08 smithi060 ceph-mon[40290]: 4.17 scrub starts 2024-09-20T22:11:08.754 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:08 smithi060 ceph-mon[40290]: 4.17 scrub ok 2024-09-20T22:11:08.754 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:08 smithi060 ceph-mon[40290]: pgmap v437: 168 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 2 active+remapped+backfill_toofull, 164 active+clean; 1.0 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 911 B/s rd, 0 B/s wr, 1.02k op/s; 2519/51330 objects misplaced (4.907%); 7.0 MiB/s, 111 objects/s recovering 2024-09-20T22:11:08.793 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:11:08.793 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:11:08.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:08.834 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:11:08.834 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:11:08.870 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:08.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:08.872 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:08.876 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:11:08.876 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:11:08.912 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:11:08.912 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:11:08.954 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:11:08.955 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:11:08.973 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:08.986 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:09.002 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:09.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:09.003 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:09.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:08 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:11:09.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:08 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:11:09.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:08 smithi067 ceph-mon[33511]: 3.9 scrub starts 2024-09-20T22:11:09.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:08 smithi067 ceph-mon[33511]: 3.9 scrub ok 2024-09-20T22:11:09.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:08 smithi067 ceph-mon[33511]: 4.17 scrub starts 2024-09-20T22:11:09.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:08 smithi067 ceph-mon[33511]: 4.17 scrub ok 2024-09-20T22:11:09.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:08 smithi067 ceph-mon[33511]: pgmap v437: 168 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 2 active+remapped+backfill_toofull, 164 active+clean; 1.0 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 911 B/s rd, 0 B/s wr, 1.02k op/s; 2519/51330 objects misplaced (4.907%); 7.0 MiB/s, 111 objects/s recovering 2024-09-20T22:11:09.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:09.150 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:09.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:09.151 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:09.251 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:09.287 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:09.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:09.288 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:09.389 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:09.426 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:09.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:09.427 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:09.528 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:09.567 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:09.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:09.568 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:09.585 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:11:09.668 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:09.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:09.707 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:09.708 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:09.717 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:11:09.717 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:11:09.808 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:09.843 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:09.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:09.844 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:09.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:09 smithi060 ceph-mon[29694]: 1.a scrub starts 2024-09-20T22:11:09.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:09 smithi060 ceph-mon[29694]: 1.a scrub ok 2024-09-20T22:11:09.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:09 smithi060 ceph-mon[29694]: 8.a deep-scrub ok 2024-09-20T22:11:09.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:09 smithi060 ceph-mon[29694]: 5.d scrub starts 2024-09-20T22:11:09.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:09 smithi060 ceph-mon[29694]: 5.d scrub ok 2024-09-20T22:11:09.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:09 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/901212600' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:09.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:09 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4153853503' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:11:09.869 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:09 smithi060 ceph-mon[40290]: 1.a scrub starts 2024-09-20T22:11:09.869 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:09 smithi060 ceph-mon[40290]: 1.a scrub ok 2024-09-20T22:11:09.869 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:09 smithi060 ceph-mon[40290]: 8.a deep-scrub ok 2024-09-20T22:11:09.869 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:09 smithi060 ceph-mon[40290]: 5.d scrub starts 2024-09-20T22:11:09.869 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:09 smithi060 ceph-mon[40290]: 5.d scrub ok 2024-09-20T22:11:09.869 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:09 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/901212600' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:09.870 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:09 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4153853503' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:11:09.945 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:09.977 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:09.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:09.978 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:10.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:09 smithi067 ceph-mon[33511]: 1.a scrub starts 2024-09-20T22:11:10.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:09 smithi067 ceph-mon[33511]: 1.a scrub ok 2024-09-20T22:11:10.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:09 smithi067 ceph-mon[33511]: 8.a deep-scrub ok 2024-09-20T22:11:10.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:09 smithi067 ceph-mon[33511]: 5.d scrub starts 2024-09-20T22:11:10.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:09 smithi067 ceph-mon[33511]: 5.d scrub ok 2024-09-20T22:11:10.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:09 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/901212600' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:10.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:09 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4153853503' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:11:10.080 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:10.115 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:10.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:10.116 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:10.217 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:10.255 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:10.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:10.256 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:10.357 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:10.386 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:10.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:10.387 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:10.488 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:10.526 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:10.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:10.527 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:10.572 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T22:11:10.628 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:10.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:10.665 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:10.665 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:10.766 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:10.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:10.803 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:10.803 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:10.904 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:10.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:10.947 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:10.947 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:10.957 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:11:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:10 smithi067 ceph-mon[33511]: 5.1d scrub starts 2024-09-20T22:11:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:10 smithi067 ceph-mon[33511]: 5.1d scrub ok 2024-09-20T22:11:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:10 smithi067 ceph-mon[33511]: 5.17 deep-scrub starts 2024-09-20T22:11:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:10 smithi067 ceph-mon[33511]: 5.17 deep-scrub ok 2024-09-20T22:11:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:10 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:11:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:10 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4153853503' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:11:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:10 smithi067 ceph-mon[33511]: osdmap e274: 8 total, 8 up, 8 in 2024-09-20T22:11:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:10 smithi067 ceph-mon[33511]: pgmap v439: 168 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 2 active+remapped+backfill_toofull, 164 active+clean; 1.0 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 828 op/s; 2519/51330 objects misplaced (4.907%); 6.5 MiB/s, 104 objects/s recovering 2024-09-20T22:11:11.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:10 smithi060 ceph-mon[40290]: 5.1d scrub starts 2024-09-20T22:11:11.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:10 smithi060 ceph-mon[40290]: 5.1d scrub ok 2024-09-20T22:11:11.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:10 smithi060 ceph-mon[40290]: 5.17 deep-scrub starts 2024-09-20T22:11:11.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:10 smithi060 ceph-mon[40290]: 5.17 deep-scrub ok 2024-09-20T22:11:11.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:10 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:11:11.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:10 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4153853503' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:11:11.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:10 smithi060 ceph-mon[40290]: osdmap e274: 8 total, 8 up, 8 in 2024-09-20T22:11:11.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:10 smithi060 ceph-mon[40290]: pgmap v439: 168 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 2 active+remapped+backfill_toofull, 164 active+clean; 1.0 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 828 op/s; 2519/51330 objects misplaced (4.907%); 6.5 MiB/s, 104 objects/s recovering 2024-09-20T22:11:11.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:10 smithi060 ceph-mon[29694]: 5.1d scrub starts 2024-09-20T22:11:11.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:10 smithi060 ceph-mon[29694]: 5.1d scrub ok 2024-09-20T22:11:11.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:10 smithi060 ceph-mon[29694]: 5.17 deep-scrub starts 2024-09-20T22:11:11.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:10 smithi060 ceph-mon[29694]: 5.17 deep-scrub ok 2024-09-20T22:11:11.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:10 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:11:11.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:10 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4153853503' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:11:11.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:10 smithi060 ceph-mon[29694]: osdmap e274: 8 total, 8 up, 8 in 2024-09-20T22:11:11.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:10 smithi060 ceph-mon[29694]: pgmap v439: 168 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+remapped, 2 active+remapped+backfill_toofull, 164 active+clean; 1.0 GiB data, 6.6 GiB used, 709 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 828 op/s; 2519/51330 objects misplaced (4.907%); 6.5 MiB/s, 104 objects/s recovering 2024-09-20T22:11:11.048 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:11.086 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:11.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:11.087 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:11.188 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:11.221 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:11.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:11.222 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:11.323 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:11.357 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:11.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:11.358 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:11.459 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:11.461 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.2 to 1 (10000) 2024-09-20T22:11:11.489 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:11.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:11.490 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:11.591 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:11.615 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:11 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2266194387' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:11:11.615 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:11 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:11:11.615 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:11 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:11:11.615 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:11 smithi067 ceph-mon[33511]: osdmap e275: 8 total, 8 up, 8 in 2024-09-20T22:11:11.626 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:11.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:11.627 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:11.728 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:11.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:11.768 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:11.768 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:11 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2266194387' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:11:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:11 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:11:11.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:11 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:11:11.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:11 smithi060 ceph-mon[29694]: osdmap e275: 8 total, 8 up, 8 in 2024-09-20T22:11:11.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:11 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2266194387' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:11:11.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:11 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:11:11.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:11 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:11:11.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:11 smithi060 ceph-mon[40290]: osdmap e275: 8 total, 8 up, 8 in 2024-09-20T22:11:11.787 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:11:11.788 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:11:11.869 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:11.903 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:11.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:11.904 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:11.918 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:11:12.004 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:12.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:12.042 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:12.043 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:12.143 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:12.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:12.193 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:12.193 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:12.194 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 3 1 2024-09-20T22:11:12.295 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:12.321 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:12.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:12.322 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:12.395 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:12.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:12.500 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:12.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:12.501 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:12.513 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:11:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:11:12.602 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:12.639 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:12.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:12.641 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:12.654 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T22:11:12.742 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:12.765 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:12.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:12.766 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:12.867 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:12.905 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:12.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:12.906 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:13.006 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:13.042 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:13.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:13.043 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:13.061 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:11:13.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:11:13.144 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:13.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:13.234 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:13.234 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:13.335 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:13.370 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:13.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:13.371 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:13.472 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:13.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:13.494 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:13.494 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:13.595 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:13.636 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:13.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:13.637 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:13.666 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:11:13.666 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:11:13.737 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:13.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:13 smithi067 ceph-mon[33511]: pgmap v441: 168 pgs: 2 peering, 1 active+clean+scrubbing+deep, 2 active+remapped+backfill_toofull, 163 active+clean; 800 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 908 B/s rd, 0 B/s wr, 1.02k op/s; 1776/38955 objects misplaced (4.559%) 2024-09-20T22:11:13.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:13 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:11:13.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:13 smithi067 ceph-mon[33511]: osdmap e276: 8 total, 8 up, 8 in 2024-09-20T22:11:13.774 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:13.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:13.775 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:13.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:13 smithi060 ceph-mon[29694]: pgmap v441: 168 pgs: 2 peering, 1 active+clean+scrubbing+deep, 2 active+remapped+backfill_toofull, 163 active+clean; 800 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 908 B/s rd, 0 B/s wr, 1.02k op/s; 1776/38955 objects misplaced (4.559%) 2024-09-20T22:11:13.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:13 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:11:13.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:13 smithi060 ceph-mon[29694]: osdmap e276: 8 total, 8 up, 8 in 2024-09-20T22:11:13.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:13 smithi060 ceph-mon[40290]: pgmap v441: 168 pgs: 2 peering, 1 active+clean+scrubbing+deep, 2 active+remapped+backfill_toofull, 163 active+clean; 800 MiB data, 5.8 GiB used, 709 GiB / 715 GiB avail; 908 B/s rd, 0 B/s wr, 1.02k op/s; 1776/38955 objects misplaced (4.559%) 2024-09-20T22:11:13.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:13 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:11:13.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:13 smithi060 ceph-mon[40290]: osdmap e276: 8 total, 8 up, 8 in 2024-09-20T22:11:13.876 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:13.915 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:13.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:13.916 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:14.017 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:14.054 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:14.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:14.056 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:14.156 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:14.193 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:14.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:14.195 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:14.295 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:14.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:14.324 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:14.325 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:14.425 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:14.462 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:14.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:14.463 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:14.531 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T22:11:14.564 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:14.608 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:14.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:14.609 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:14.710 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:14.748 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:14.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:14.749 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:14 smithi067 ceph-mon[33511]: pgmap v443: 168 pgs: 2 peering, 2 active+remapped+backfill_toofull, 164 active+clean; 742 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 841 op/s; 1776/36192 objects misplaced (4.907%) 2024-09-20T22:11:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:14 smithi060 ceph-mon[29694]: pgmap v443: 168 pgs: 2 peering, 2 active+remapped+backfill_toofull, 164 active+clean; 742 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 841 op/s; 1776/36192 objects misplaced (4.907%) 2024-09-20T22:11:14.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:14 smithi060 ceph-mon[40290]: pgmap v443: 168 pgs: 2 peering, 2 active+remapped+backfill_toofull, 164 active+clean; 742 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 841 op/s; 1776/36192 objects misplaced (4.907%) 2024-09-20T22:11:14.850 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:14.876 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:14.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:14.877 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:14.900 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:11:14.978 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:15.003 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:15.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:15.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:15.105 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:15.135 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:15.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:15.136 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:15.237 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:15.266 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:15.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:15.267 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:15.368 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:15.390 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:15.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:15.391 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:15.460 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:11:15.461 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:11:15.492 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:15.516 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:11:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:11:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:11:15.530 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:15.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:15.531 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:15.631 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:15.664 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:15.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:15.665 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:15 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:11:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:15 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:11:15.765 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:15.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:15 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:11:15.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:15 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:11:15.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:15 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:11:15.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:15 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:11:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:15.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:15.798 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:15.898 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:15.942 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:15.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:15.943 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:16.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:16.074 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:16.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:16.075 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:16.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:16.216 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:16.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:16.217 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:16.219 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T22:11:16.318 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:16.358 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:16.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:16.359 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:16.460 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:16.510 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:16.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:16.511 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:16.585 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:11:16.611 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:16.649 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:16.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:16.650 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:16.750 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:16.779 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:16.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:16.780 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:16.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:16 smithi060 ceph-mon[29694]: osdmap e277: 8 total, 8 up, 8 in 2024-09-20T22:11:16.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:16 smithi060 ceph-mon[29694]: pgmap v445: 168 pgs: 2 peering, 2 active+remapped+backfill_toofull, 164 active+clean; 598 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.23k op/s; 1776/29277 objects misplaced (6.066%); 0 B/s, 0 objects/s recovering 2024-09-20T22:11:16.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:16 smithi060 ceph-mon[40290]: osdmap e277: 8 total, 8 up, 8 in 2024-09-20T22:11:16.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:16 smithi060 ceph-mon[40290]: pgmap v445: 168 pgs: 2 peering, 2 active+remapped+backfill_toofull, 164 active+clean; 598 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.23k op/s; 1776/29277 objects misplaced (6.066%); 0 B/s, 0 objects/s recovering 2024-09-20T22:11:16.881 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:16.915 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:16.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:16.916 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:17.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:16 smithi067 ceph-mon[33511]: osdmap e277: 8 total, 8 up, 8 in 2024-09-20T22:11:17.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:16 smithi067 ceph-mon[33511]: pgmap v445: 168 pgs: 2 peering, 2 active+remapped+backfill_toofull, 164 active+clean; 598 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.23k op/s; 1776/29277 objects misplaced (6.066%); 0 B/s, 0 objects/s recovering 2024-09-20T22:11:17.017 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:17.053 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:17.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:17.054 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:17.155 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:17.184 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:17.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:17.185 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:17.224 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:11:17.224 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:11:17.224 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:11:17.224 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:11:17.231 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:11:17.231 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:11:17.273 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:11:17.274 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:11:17.286 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:17.309 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:11:17.309 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:11:17.314 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:17.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:17.315 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:17.320 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:17.342 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:11:17.342 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:11:17.381 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:11:17.382 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:11:17.382 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:17.416 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:17.417 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:11:17.417 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:11:17.472 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:17.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:17.473 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:17.574 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:17.616 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:17.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:17.617 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:17.707 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:11:17.707 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:11:17.707 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:11:17.707 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:11:17.707 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.707 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169963 7.15 7:acd7b1d6:::benchmark_data_smithi060_78800_object36930:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:11:17.707 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.440422+0000", 2024-09-20T22:11:17.708 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 319.26208256500001, 2024-09-20T22:11:17.708 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0806133010000001, 2024-09-20T22:11:17.708 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.708 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.708 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:11:17.708 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.708 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:11:17.708 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:11:17.708 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169963 2024-09-20T22:11:17.708 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.708 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.708 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.708 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.708 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440422+0000", 2024-09-20T22:11:17.709 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.709 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.709 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.709 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440422+0000", 2024-09-20T22:11:17.709 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.709 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.709 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.709 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440424+0000", 2024-09-20T22:11:17.709 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.762e-06 2024-09-20T22:11:17.709 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.709 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.709 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.709 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440432+0000", 2024-09-20T22:11:17.710 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4530000000000002e-06 2024-09-20T22:11:17.710 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.710 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.710 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.710 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440448+0000", 2024-09-20T22:11:17.710 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5084e-05 2024-09-20T22:11:17.710 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.710 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.710 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.710 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440456+0000", 2024-09-20T22:11:17.711 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0020000000000006e-06 2024-09-20T22:11:17.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.711 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.711 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.711 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549826+0000", 2024-09-20T22:11:17.711 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.109370678 2024-09-20T22:11:17.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.711 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.711 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.711 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549832+0000", 2024-09-20T22:11:17.711 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3850000000000001e-06 2024-09-20T22:11:17.711 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.711 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.711 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.712 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.503733+0000", 2024-09-20T22:11:17.712 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.95390151700000003 2024-09-20T22:11:17.712 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.712 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.712 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for missing object", 2024-09-20T22:11:17.712 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.503868+0000", 2024-09-20T22:11:17.712 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013515399999999999 2024-09-20T22:11:17.712 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.712 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.712 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.712 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.520960+0000", 2024-09-20T22:11:17.713 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.017092006 2024-09-20T22:11:17.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.713 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.713 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.713 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.521036+0000", 2024-09-20T22:11:17.713 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5259999999999994e-05 2024-09-20T22:11:17.713 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.713 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.713 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.713 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.713 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.713 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169964 7.15 7:aca939f2:::benchmark_data_smithi060_78800_object36931:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:11:17.713 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.440473+0000", 2024-09-20T22:11:17.713 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 319.262032116, 2024-09-20T22:11:17.714 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.081014133, 2024-09-20T22:11:17.714 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.714 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.714 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:11:17.714 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.714 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:11:17.714 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:11:17.714 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169964 2024-09-20T22:11:17.714 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.714 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.714 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.714 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.714 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440473+0000", 2024-09-20T22:11:17.714 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.714 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.715 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.715 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.715 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440473+0000", 2024-09-20T22:11:17.715 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.715 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.715 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.715 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.715 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440473+0000", 2024-09-20T22:11:17.715 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0499999999999999e-07 2024-09-20T22:11:17.715 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.715 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.715 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.715 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440476+0000", 2024-09-20T22:11:17.715 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0750000000000002e-06 2024-09-20T22:11:17.716 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.716 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.716 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.716 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440487+0000", 2024-09-20T22:11:17.716 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1048e-05 2024-09-20T22:11:17.716 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.716 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.716 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.716 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.440493+0000", 2024-09-20T22:11:17.716 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.31e-06 2024-09-20T22:11:17.716 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.716 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.716 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.716 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549842+0000", 2024-09-20T22:11:17.716 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.109349718 2024-09-20T22:11:17.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.717 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.717 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.717 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.549845+0000", 2024-09-20T22:11:17.717 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0029999999999999e-06 2024-09-20T22:11:17.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.717 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.717 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.717 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.504130+0000", 2024-09-20T22:11:17.717 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.95428438100000001 2024-09-20T22:11:17.717 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.717 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.717 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for missing object", 2024-09-20T22:11:17.717 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.504241+0000", 2024-09-20T22:11:17.717 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011154099999999999 2024-09-20T22:11:17.718 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.718 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.718 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.718 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.521422+0000", 2024-09-20T22:11:17.718 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.017180540000000001 2024-09-20T22:11:17.718 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.718 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.718 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.718 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.521487+0000", 2024-09-20T22:11:17.718 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5111999999999996e-05 2024-09-20T22:11:17.718 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.718 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.718 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.719 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.719 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.719 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:236437 7.11 7:8eaecf20:::benchmark_data_smithi060_78800_object103404:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:11:17.719 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:07.392155+0000", 2024-09-20T22:11:17.719 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 250.31034966199999, 2024-09-20T22:11:17.719 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.019005036, 2024-09-20T22:11:17.719 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.719 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.719 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:17.719 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.719 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:11:17.719 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:11:17.719 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 236437 2024-09-20T22:11:17.719 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.719 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.720 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.720 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.720 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392155+0000", 2024-09-20T22:11:17.720 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.720 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.720 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.720 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.720 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392155+0000", 2024-09-20T22:11:17.720 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.720 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.720 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.720 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.720 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392157+0000", 2024-09-20T22:11:17.720 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9599999999999999e-06 2024-09-20T22:11:17.721 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.721 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.721 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.721 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392162+0000", 2024-09-20T22:11:17.721 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1719999999999998e-06 2024-09-20T22:11:17.721 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.721 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.721 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:17.722 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.722 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392180+0000", 2024-09-20T22:11:17.722 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8088e-05 2024-09-20T22:11:17.722 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.722 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.723 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.723 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392188+0000", 2024-09-20T22:11:17.723 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9929999999999995e-06 2024-09-20T22:11:17.723 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.723 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.723 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.723 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392254+0000", 2024-09-20T22:11:17.723 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6137999999999996e-05 2024-09-20T22:11:17.723 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.723 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.723 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.723 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392260+0000", 2024-09-20T22:11:17.723 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7089999999999999e-06 2024-09-20T22:11:17.723 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.724 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.724 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.724 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.406485+0000", 2024-09-20T22:11:17.724 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0142251280000001 2024-09-20T22:11:17.724 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.724 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.724 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:17.724 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.406593+0000", 2024-09-20T22:11:17.724 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000107672 2024-09-20T22:11:17.724 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.724 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.724 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:11:17.724 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.406954+0000", 2024-09-20T22:11:17.724 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000360973 2024-09-20T22:11:17.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.725 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.725 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:17.725 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411014+0000", 2024-09-20T22:11:17.725 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00406033 2024-09-20T22:11:17.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.725 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.725 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.725 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411059+0000", 2024-09-20T22:11:17.725 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4907999999999998e-05 2024-09-20T22:11:17.725 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.725 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.725 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.725 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411090+0000", 2024-09-20T22:11:17.725 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0874000000000003e-05 2024-09-20T22:11:17.726 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.726 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.726 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:17.726 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411121+0000", 2024-09-20T22:11:17.726 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0518999999999997e-05 2024-09-20T22:11:17.726 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.726 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.726 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.726 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411160+0000", 2024-09-20T22:11:17.726 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9572e-05 2024-09-20T22:11:17.726 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.726 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.726 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.727 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.727 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.727 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261924 7.17 7:ec34c39a:::benchmark_data_smithi060_78800_object128891:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:11:17.727 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.137969+0000", 2024-09-20T22:11:17.727 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 221.56453536199999, 2024-09-20T22:11:17.727 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0378232009999999, 2024-09-20T22:11:17.727 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.727 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.727 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:11:17.727 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.727 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:11:17.727 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:11:17.727 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261924 2024-09-20T22:11:17.728 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.728 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.728 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.728 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.728 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137969+0000", 2024-09-20T22:11:17.728 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.728 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.728 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.728 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.728 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137969+0000", 2024-09-20T22:11:17.728 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.728 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.728 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.728 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.729 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137971+0000", 2024-09-20T22:11:17.729 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.19e-06 2024-09-20T22:11:17.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.729 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.729 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.729 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137976+0000", 2024-09-20T22:11:17.729 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8429999999999998e-06 2024-09-20T22:11:17.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.729 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.729 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.729 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137987+0000", 2024-09-20T22:11:17.729 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0587e-05 2024-09-20T22:11:17.729 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.729 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.730 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.730 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137992+0000", 2024-09-20T22:11:17.730 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4090000000000002e-06 2024-09-20T22:11:17.730 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.730 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.730 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.730 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138048+0000", 2024-09-20T22:11:17.730 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5241999999999997e-05 2024-09-20T22:11:17.730 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.730 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.730 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.730 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138056+0000", 2024-09-20T22:11:17.730 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4649999999999994e-06 2024-09-20T22:11:17.730 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.730 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.731 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.731 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.175735+0000", 2024-09-20T22:11:17.731 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.037678871 2024-09-20T22:11:17.731 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.731 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.731 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.731 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.175793+0000", 2024-09-20T22:11:17.731 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7593999999999998e-05 2024-09-20T22:11:17.731 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.731 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.731 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.731 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.731 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.731 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261927 7.17 7:ed776cb1:::benchmark_data_smithi060_78800_object128894:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:11:17.732 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.138007+0000", 2024-09-20T22:11:17.732 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 221.56449793499999, 2024-09-20T22:11:17.732 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0471626510000001, 2024-09-20T22:11:17.732 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.732 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.732 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:11:17.732 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.732 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:11:17.732 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:11:17.732 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261927 2024-09-20T22:11:17.732 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.732 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.732 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.732 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.733 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138007+0000", 2024-09-20T22:11:17.733 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.733 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.733 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.733 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138007+0000", 2024-09-20T22:11:17.733 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.733 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.733 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.733 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138007+0000", 2024-09-20T22:11:17.733 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5600000000000001e-07 2024-09-20T22:11:17.733 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.733 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.734 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.734 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138009+0000", 2024-09-20T22:11:17.734 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6729999999999999e-06 2024-09-20T22:11:17.734 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.734 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.734 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.734 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138013+0000", 2024-09-20T22:11:17.734 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.16e-06 2024-09-20T22:11:17.734 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.734 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.734 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.734 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138016+0000", 2024-09-20T22:11:17.734 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1609999999999999e-06 2024-09-20T22:11:17.734 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.735 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.735 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.735 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138084+0000", 2024-09-20T22:11:17.735 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8044000000000001e-05 2024-09-20T22:11:17.735 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.735 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.735 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.735 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138093+0000", 2024-09-20T22:11:17.735 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.49e-06 2024-09-20T22:11:17.735 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.735 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.735 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.735 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177120+0000", 2024-09-20T22:11:17.735 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0390272819999999 2024-09-20T22:11:17.735 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.735 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.736 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for missing object", 2024-09-20T22:11:17.736 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177222+0000", 2024-09-20T22:11:17.736 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000102272 2024-09-20T22:11:17.736 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.736 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.736 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.736 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185103+0000", 2024-09-20T22:11:17.736 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0078803929999999994 2024-09-20T22:11:17.736 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.736 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.736 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.736 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185170+0000", 2024-09-20T22:11:17.736 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6820000000000001e-05 2024-09-20T22:11:17.736 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.737 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.737 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.737 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.737 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.737 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261940 7.17 7:e9ae652d:::benchmark_data_smithi060_78800_object128907:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:11:17.737 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.144358+0000", 2024-09-20T22:11:17.737 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 221.558147018, 2024-09-20T22:11:17.737 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.040411067, 2024-09-20T22:11:17.737 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.737 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.737 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:17.737 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.737 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:11:17.737 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:11:17.738 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261940 2024-09-20T22:11:17.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.738 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.738 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.738 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.738 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144358+0000", 2024-09-20T22:11:17.738 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.738 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.738 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.738 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144358+0000", 2024-09-20T22:11:17.738 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.738 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.738 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.738 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.739 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144359+0000", 2024-09-20T22:11:17.739 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9299999999999996e-07 2024-09-20T22:11:17.739 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.739 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.739 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.739 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144362+0000", 2024-09-20T22:11:17.739 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6799999999999999e-06 2024-09-20T22:11:17.739 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.739 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.739 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.739 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144369+0000", 2024-09-20T22:11:17.739 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8070000000000002e-06 2024-09-20T22:11:17.739 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.739 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.740 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.740 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144374+0000", 2024-09-20T22:11:17.740 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8999999999999997e-06 2024-09-20T22:11:17.740 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.740 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.740 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.740 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144416+0000", 2024-09-20T22:11:17.740 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2049999999999999e-05 2024-09-20T22:11:17.740 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.740 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.740 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.740 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144421+0000", 2024-09-20T22:11:17.740 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6550000000000003e-06 2024-09-20T22:11:17.740 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.740 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.741 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.741 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177283+0000", 2024-09-20T22:11:17.741 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0328618039999999 2024-09-20T22:11:17.741 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.741 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.741 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:17.741 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177468+0000", 2024-09-20T22:11:17.741 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00018551900000000001 2024-09-20T22:11:17.741 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.741 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.741 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:11:17.741 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177846+0000", 2024-09-20T22:11:17.741 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00037771700000000002 2024-09-20T22:11:17.741 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.741 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184561+0000", 2024-09-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0067147600000000002 2024-09-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184572+0000", 2024-09-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1433999999999999e-05 2024-09-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.742 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.743 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.743 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184635+0000", 2024-09-20T22:11:17.743 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3459000000000003e-05 2024-09-20T22:11:17.743 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.743 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.743 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:17.743 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184692+0000", 2024-09-20T22:11:17.743 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6759999999999999e-05 2024-09-20T22:11:17.743 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.743 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.743 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.743 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184769+0000", 2024-09-20T22:11:17.743 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6629000000000001e-05 2024-09-20T22:11:17.743 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.743 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.744 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.744 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.744 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.744 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261963 7.17 7:eacd681d:::benchmark_data_smithi060_78800_object128930:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:11:17.744 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.165356+0000", 2024-09-20T22:11:17.744 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 221.53714844000001, 2024-09-20T22:11:17.744 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.020053374, 2024-09-20T22:11:17.744 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.744 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.744 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:17.744 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.744 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:11:17.744 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:11:17.744 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261963 2024-09-20T22:11:17.744 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.745 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.745 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.745 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.745 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165356+0000", 2024-09-20T22:11:17.745 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.745 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.745 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.745 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.745 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165356+0000", 2024-09-20T22:11:17.745 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.745 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.745 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.745 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.745 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165357+0000", 2024-09-20T22:11:17.745 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.3299999999999998e-07 2024-09-20T22:11:17.746 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.746 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.746 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.746 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165359+0000", 2024-09-20T22:11:17.746 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.046e-06 2024-09-20T22:11:17.746 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.746 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.746 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.746 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165364+0000", 2024-09-20T22:11:17.746 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.502e-06 2024-09-20T22:11:17.746 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.746 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.746 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.746 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165369+0000", 2024-09-20T22:11:17.746 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6319999999999999e-06 2024-09-20T22:11:17.747 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.747 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.747 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.747 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.171567+0000", 2024-09-20T22:11:17.747 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061976460000000002 2024-09-20T22:11:17.747 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.747 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.747 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.747 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.171575+0000", 2024-09-20T22:11:17.747 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2390000000000002e-06 2024-09-20T22:11:17.747 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.747 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.747 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.747 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.180431+0000", 2024-09-20T22:11:17.748 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0088558569999999 2024-09-20T22:11:17.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.748 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.748 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:17.748 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.180570+0000", 2024-09-20T22:11:17.748 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013935799999999999 2024-09-20T22:11:17.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.748 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.748 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:11:17.748 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.180897+0000", 2024-09-20T22:11:17.748 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00032708599999999998 2024-09-20T22:11:17.748 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.748 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.748 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:17.748 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184575+0000", 2024-09-20T22:11:17.748 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0036776790000000001 2024-09-20T22:11:17.749 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.749 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.749 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.749 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185202+0000", 2024-09-20T22:11:17.749 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00062773099999999997 2024-09-20T22:11:17.749 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.749 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.749 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.749 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185253+0000", 2024-09-20T22:11:17.749 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0742999999999999e-05 2024-09-20T22:11:17.749 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.749 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.749 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:17.749 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185303+0000", 2024-09-20T22:11:17.750 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9339000000000003e-05 2024-09-20T22:11:17.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.750 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.750 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.750 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185410+0000", 2024-09-20T22:11:17.750 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010718299999999999 2024-09-20T22:11:17.750 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.750 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.750 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.750 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.750 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.750 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261986 7.17 7:ebad35fd:::benchmark_data_smithi060_78800_object128953:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:11:17.750 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.200930+0000", 2024-09-20T22:11:17.750 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 221.50157434799999, 2024-09-20T22:11:17.751 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98696583999999998, 2024-09-20T22:11:17.751 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.751 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.751 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:17.751 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.751 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:11:17.751 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:11:17.751 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261986 2024-09-20T22:11:17.751 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.751 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.751 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.751 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.751 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200930+0000", 2024-09-20T22:11:17.751 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.751 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.752 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.752 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.752 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200930+0000", 2024-09-20T22:11:17.752 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.752 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.752 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.752 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200931+0000", 2024-09-20T22:11:17.752 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8800000000000001e-07 2024-09-20T22:11:17.752 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.752 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.752 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.752 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200936+0000", 2024-09-20T22:11:17.752 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3939999999999998e-06 2024-09-20T22:11:17.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.753 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.753 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.753 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200946+0000", 2024-09-20T22:11:17.753 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0706999999999999e-05 2024-09-20T22:11:17.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.753 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.753 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.753 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200953+0000", 2024-09-20T22:11:17.753 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9260000000000003e-06 2024-09-20T22:11:17.753 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.753 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.753 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.753 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.201017+0000", 2024-09-20T22:11:17.753 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3793000000000004e-05 2024-09-20T22:11:17.754 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.754 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.754 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.754 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.201028+0000", 2024-09-20T22:11:17.754 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0577000000000001e-05 2024-09-20T22:11:17.754 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.754 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.754 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.754 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.183246+0000", 2024-09-20T22:11:17.754 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98221816100000003 2024-09-20T22:11:17.754 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.754 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.754 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:17.754 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.183318+0000", 2024-09-20T22:11:17.755 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1890000000000005e-05 2024-09-20T22:11:17.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.755 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.755 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:11:17.755 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.183503+0000", 2024-09-20T22:11:17.755 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00018529399999999999 2024-09-20T22:11:17.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.755 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.755 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:17.755 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185423+0000", 2024-09-20T22:11:17.755 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0019203390000000001 2024-09-20T22:11:17.755 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.755 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.755 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.756 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185478+0000", 2024-09-20T22:11:17.756 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5041999999999999e-05 2024-09-20T22:11:17.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.756 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.756 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.756 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.187745+0000", 2024-09-20T22:11:17.756 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0022662239999999998 2024-09-20T22:11:17.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.756 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.756 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:17.756 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.187795+0000", 2024-09-20T22:11:17.756 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0668999999999999e-05 2024-09-20T22:11:17.756 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.756 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.757 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.757 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.187896+0000", 2024-09-20T22:11:17.757 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100936 2024-09-20T22:11:17.757 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.757 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.757 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.757 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.757 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.757 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262055 7.17 7:eca2ebed:::benchmark_data_smithi060_78800_object129022:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:11:17.757 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.253316+0000", 2024-09-20T22:11:17.757 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 221.449189209, 2024-09-20T22:11:17.757 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93859176700000002, 2024-09-20T22:11:17.757 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.757 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.758 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:17.758 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.758 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:11:17.758 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:11:17.758 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262055 2024-09-20T22:11:17.758 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.758 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.758 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.758 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.758 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253316+0000", 2024-09-20T22:11:17.758 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.758 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.758 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.759 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.759 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253316+0000", 2024-09-20T22:11:17.759 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.759 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.759 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.759 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.759 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253316+0000", 2024-09-20T22:11:17.759 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2999999999999999e-07 2024-09-20T22:11:17.759 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.759 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.759 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.759 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253320+0000", 2024-09-20T22:11:17.759 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9129999999999996e-06 2024-09-20T22:11:17.759 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.760 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.760 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.760 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253333+0000", 2024-09-20T22:11:17.760 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2762000000000001e-05 2024-09-20T22:11:17.760 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.760 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.760 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.760 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253340+0000", 2024-09-20T22:11:17.760 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0180000000000004e-06 2024-09-20T22:11:17.760 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.760 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.760 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.760 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253402+0000", 2024-09-20T22:11:17.760 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2211000000000005e-05 2024-09-20T22:11:17.761 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.761 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.761 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.761 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253409+0000", 2024-09-20T22:11:17.761 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.985e-06 2024-09-20T22:11:17.761 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.761 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.761 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.761 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185521+0000", 2024-09-20T22:11:17.761 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93211197400000001 2024-09-20T22:11:17.761 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.761 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.761 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:17.761 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185620+0000", 2024-09-20T22:11:17.761 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9162999999999999e-05 2024-09-20T22:11:17.762 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.762 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.762 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:11:17.762 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185870+0000", 2024-09-20T22:11:17.762 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00024926499999999998 2024-09-20T22:11:17.762 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.762 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.762 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.762 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191631+0000", 2024-09-20T22:11:17.762 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.005761375 2024-09-20T22:11:17.763 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.763 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.763 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:17.763 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191650+0000", 2024-09-20T22:11:17.763 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8533999999999999e-05 2024-09-20T22:11:17.764 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.764 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.764 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.764 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191717+0000", 2024-09-20T22:11:17.764 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7731999999999995e-05 2024-09-20T22:11:17.764 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.764 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.764 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:17.764 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191780+0000", 2024-09-20T22:11:17.764 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2447999999999996e-05 2024-09-20T22:11:17.764 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.764 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.764 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.764 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191907+0000", 2024-09-20T22:11:17.765 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012755700000000001 2024-09-20T22:11:17.765 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.765 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.765 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.765 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.765 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.765 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:106968 8.15 8:a8961342:::benchmark_data_smithi060_111051_object106967:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:11:17.765 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.566651+0000", 2024-09-20T22:11:17.765 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 127.135853795, 2024-09-20T22:11:17.765 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9227000670000001, 2024-09-20T22:11:17.765 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.765 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.765 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:11:17.765 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.766 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:17.766 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:17.766 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 106968 2024-09-20T22:11:17.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.766 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.766 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.766 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.766 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566651+0000", 2024-09-20T22:11:17.766 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.766 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.766 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.766 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566651+0000", 2024-09-20T22:11:17.766 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.766 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.767 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.767 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.767 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566653+0000", 2024-09-20T22:11:17.767 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.674e-06 2024-09-20T22:11:17.767 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.767 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.767 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.767 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566809+0000", 2024-09-20T22:11:17.767 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015637600000000001 2024-09-20T22:11:17.767 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.767 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.767 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.767 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566820+0000", 2024-09-20T22:11:17.767 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.097e-05 2024-09-20T22:11:17.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.768 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.768 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.768 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566830+0000", 2024-09-20T22:11:17.768 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4690000000000003e-06 2024-09-20T22:11:17.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.768 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.768 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.768 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.567309+0000", 2024-09-20T22:11:17.768 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00047940599999999998 2024-09-20T22:11:17.768 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.768 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.768 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.768 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.567321+0000", 2024-09-20T22:11:17.769 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2451e-05 2024-09-20T22:11:17.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.769 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.769 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.769 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417364+0000", 2024-09-20T22:11:17.769 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.85004267200000005 2024-09-20T22:11:17.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.769 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.769 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.769 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417390+0000", 2024-09-20T22:11:17.769 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.5959999999999999e-05 2024-09-20T22:11:17.769 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.769 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.769 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.770 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489254+0000", 2024-09-20T22:11:17.770 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0718642620000001 2024-09-20T22:11:17.770 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.770 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.770 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.770 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489351+0000", 2024-09-20T22:11:17.770 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6827e-05 2024-09-20T22:11:17.770 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.770 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.770 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.770 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.770 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.770 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:106993 8.15 8:ae0aa858:::benchmark_data_smithi060_111051_object106992:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:11:17.770 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.600689+0000", 2024-09-20T22:11:17.771 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 127.10181613100001, 2024-09-20T22:11:17.771 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.911249717, 2024-09-20T22:11:17.771 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.771 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.771 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:17.771 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.771 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:17.771 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:17.771 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 106993 2024-09-20T22:11:17.771 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.771 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.771 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.771 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.771 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600689+0000", 2024-09-20T22:11:17.771 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.772 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.772 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.772 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600689+0000", 2024-09-20T22:11:17.772 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.772 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.772 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.772 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600691+0000", 2024-09-20T22:11:17.772 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.052e-06 2024-09-20T22:11:17.772 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.772 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.772 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.772 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600724+0000", 2024-09-20T22:11:17.772 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.3429000000000001e-05 2024-09-20T22:11:17.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.773 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.773 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.773 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600747+0000", 2024-09-20T22:11:17.773 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2663e-05 2024-09-20T22:11:17.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.773 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.773 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.773 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600756+0000", 2024-09-20T22:11:17.773 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4099999999999997e-06 2024-09-20T22:11:17.773 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.773 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.773 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.773 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600862+0000", 2024-09-20T22:11:17.774 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000106135 2024-09-20T22:11:17.774 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.774 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.774 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.774 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600873+0000", 2024-09-20T22:11:17.774 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0794e-05 2024-09-20T22:11:17.774 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.774 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.774 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.774 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417486+0000", 2024-09-20T22:11:17.774 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.81661262800000001 2024-09-20T22:11:17.774 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.774 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.774 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.774 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417495+0000", 2024-09-20T22:11:17.775 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1160000000000005e-06 2024-09-20T22:11:17.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.775 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.775 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.775 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489416+0000", 2024-09-20T22:11:17.775 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.071920784 2024-09-20T22:11:17.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.775 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.775 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:17.775 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489581+0000", 2024-09-20T22:11:17.775 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00016577399999999999 2024-09-20T22:11:17.775 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.775 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.775 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,4,6", 2024-09-20T22:11:17.776 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489742+0000", 2024-09-20T22:11:17.776 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000160204 2024-09-20T22:11:17.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.776 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.776 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:17.776 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.494442+0000", 2024-09-20T22:11:17.776 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0047004660000000004 2024-09-20T22:11:17.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.776 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.776 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.776 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.495057+0000", 2024-09-20T22:11:17.776 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000614677 2024-09-20T22:11:17.776 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.776 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.776 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.777 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.495091+0000", 2024-09-20T22:11:17.777 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4581e-05 2024-09-20T22:11:17.777 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.777 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.777 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.777 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.511743+0000", 2024-09-20T22:11:17.777 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.016651142000000001 2024-09-20T22:11:17.777 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.777 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.777 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:17.777 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.511810+0000", 2024-09-20T22:11:17.777 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7065999999999998e-05 2024-09-20T22:11:17.777 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.777 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.778 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.778 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.511938+0000", 2024-09-20T22:11:17.778 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000128796 2024-09-20T22:11:17.778 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.778 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.778 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.778 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.778 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.778 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:107013 8.15 8:ab48afac:::benchmark_data_smithi060_111051_object107012:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:11:17.778 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.648456+0000", 2024-09-20T22:11:17.778 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 127.05404842900001, 2024-09-20T22:11:17.778 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8646870719999999, 2024-09-20T22:11:17.778 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.778 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.778 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:17.779 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.779 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:17.779 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:17.779 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 107013 2024-09-20T22:11:17.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.779 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.779 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.779 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.779 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648456+0000", 2024-09-20T22:11:17.779 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.779 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.779 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.779 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.779 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648456+0000", 2024-09-20T22:11:17.780 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.780 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.780 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.780 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.780 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648459+0000", 2024-09-20T22:11:17.780 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2000000000000001e-06 2024-09-20T22:11:17.780 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.780 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.780 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.780 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648501+0000", 2024-09-20T22:11:17.780 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2453000000000002e-05 2024-09-20T22:11:17.780 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.780 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.780 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.781 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648525+0000", 2024-09-20T22:11:17.781 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.4034000000000001e-05 2024-09-20T22:11:17.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.781 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.781 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.781 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648535+0000", 2024-09-20T22:11:17.781 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0355e-05 2024-09-20T22:11:17.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.781 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.781 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.781 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648641+0000", 2024-09-20T22:11:17.781 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000105448 2024-09-20T22:11:17.781 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.781 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.781 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.782 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648654+0000", 2024-09-20T22:11:17.782 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2904999999999999e-05 2024-09-20T22:11:17.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.782 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.782 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.782 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417525+0000", 2024-09-20T22:11:17.782 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.76887095999999999 2024-09-20T22:11:17.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.782 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.782 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.782 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417959+0000", 2024-09-20T22:11:17.782 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00043430399999999999 2024-09-20T22:11:17.782 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.782 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.783 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.783 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.492147+0000", 2024-09-20T22:11:17.783 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0741883080000001 2024-09-20T22:11:17.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.783 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.783 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:17.783 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.492291+0000", 2024-09-20T22:11:17.783 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014341700000000001 2024-09-20T22:11:17.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.783 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.783 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,4,6", 2024-09-20T22:11:17.783 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.492463+0000", 2024-09-20T22:11:17.783 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017243400000000001 2024-09-20T22:11:17.783 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.784 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.784 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.784 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.499029+0000", 2024-09-20T22:11:17.784 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0065655460000000002 2024-09-20T22:11:17.784 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.784 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.784 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.784 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.499078+0000", 2024-09-20T22:11:17.784 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9527000000000002e-05 2024-09-20T22:11:17.784 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.784 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.784 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:17.784 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.499106+0000", 2024-09-20T22:11:17.784 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7280999999999999e-05 2024-09-20T22:11:17.784 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.785 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.785 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.785 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.512989+0000", 2024-09-20T22:11:17.785 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.013883929 2024-09-20T22:11:17.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.785 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.785 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:17.785 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.513048+0000", 2024-09-20T22:11:17.785 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8261999999999999e-05 2024-09-20T22:11:17.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.785 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.785 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.785 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.513143+0000", 2024-09-20T22:11:17.785 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5709000000000001e-05 2024-09-20T22:11:17.785 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.786 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.786 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.786 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.786 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.786 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163403 8.17 8:ebf82752:::benchmark_data_smithi060_111051_object43404:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:11:17.786 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.477864+0000", 2024-09-20T22:11:17.786 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 68.224641125999995, 2024-09-20T22:11:17.786 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0006944760000001, 2024-09-20T22:11:17.786 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.786 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.786 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:17.786 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.786 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:17.786 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:17.787 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163403 2024-09-20T22:11:17.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.787 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.787 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.787 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.787 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477864+0000", 2024-09-20T22:11:17.787 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.787 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.787 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.787 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477864+0000", 2024-09-20T22:11:17.787 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.787 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.787 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.788 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477865+0000", 2024-09-20T22:11:17.788 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4279999999999999e-06 2024-09-20T22:11:17.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.788 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.788 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.788 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477870+0000", 2024-09-20T22:11:17.788 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8899999999999998e-06 2024-09-20T22:11:17.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.788 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.788 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.788 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477881+0000", 2024-09-20T22:11:17.788 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1139e-05 2024-09-20T22:11:17.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.788 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477888+0000", 2024-09-20T22:11:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9779999999999999e-06 2024-09-20T22:11:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477943+0000", 2024-09-20T22:11:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5229000000000003e-05 2024-09-20T22:11:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477952+0000", 2024-09-20T22:11:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5420000000000005e-06 2024-09-20T22:11:17.789 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.473740+0000", 2024-09-20T22:11:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.995788063 2024-09-20T22:11:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.473826+0000", 2024-09-20T22:11:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6131000000000006e-05 2024-09-20T22:11:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:11:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.473905+0000", 2024-09-20T22:11:17.790 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8899000000000005e-05 2024-09-20T22:11:17.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.791 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.791 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:17.791 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478387+0000", 2024-09-20T22:11:17.791 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0044816680000000003 2024-09-20T22:11:17.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.791 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.791 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.791 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478428+0000", 2024-09-20T22:11:17.791 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1091999999999999e-05 2024-09-20T22:11:17.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.791 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.791 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.791 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478481+0000", 2024-09-20T22:11:17.792 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3628000000000002e-05 2024-09-20T22:11:17.792 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.792 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.792 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:17.792 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478514+0000", 2024-09-20T22:11:17.792 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2880999999999999e-05 2024-09-20T22:11:17.792 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.792 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.792 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.792 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478558+0000", 2024-09-20T22:11:17.792 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3908e-05 2024-09-20T22:11:17.792 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.792 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.792 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.792 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.793 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.793 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163409 8.17 8:ee9c487d:::benchmark_data_smithi060_111051_object43410:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:11:17.793 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.483034+0000", 2024-09-20T22:11:17.793 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 68.219470763999993, 2024-09-20T22:11:17.793 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.99573102899999999, 2024-09-20T22:11:17.793 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.793 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.793 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:17.793 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.793 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:17.793 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:17.793 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163409 2024-09-20T22:11:17.793 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.793 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.793 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.794 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.794 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483034+0000", 2024-09-20T22:11:17.794 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.794 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.794 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.794 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483034+0000", 2024-09-20T22:11:17.794 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.794 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.794 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.794 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483036+0000", 2024-09-20T22:11:17.794 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7940000000000001e-06 2024-09-20T22:11:17.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.795 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.795 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.795 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483041+0000", 2024-09-20T22:11:17.795 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.446e-06 2024-09-20T22:11:17.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.795 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.795 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.795 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483060+0000", 2024-09-20T22:11:17.795 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8258e-05 2024-09-20T22:11:17.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.795 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.795 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.795 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483068+0000", 2024-09-20T22:11:17.795 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5390000000000007e-06 2024-09-20T22:11:17.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.796 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.796 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.796 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483127+0000", 2024-09-20T22:11:17.796 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8542999999999998e-05 2024-09-20T22:11:17.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.796 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.796 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.796 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483135+0000", 2024-09-20T22:11:17.796 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.926e-06 2024-09-20T22:11:17.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.796 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.796 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.796 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.475590+0000", 2024-09-20T22:11:17.797 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.99245572400000004 2024-09-20T22:11:17.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.797 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.797 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:17.797 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.475685+0000", 2024-09-20T22:11:17.797 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4228999999999995e-05 2024-09-20T22:11:17.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.797 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.797 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:11:17.797 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.475757+0000", 2024-09-20T22:11:17.797 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2712000000000004e-05 2024-09-20T22:11:17.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.797 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.797 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:17.798 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478623+0000", 2024-09-20T22:11:17.798 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028660069999999998 2024-09-20T22:11:17.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.798 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.798 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478671+0000", 2024-09-20T22:11:17.798 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7890999999999999e-05 2024-09-20T22:11:17.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.798 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.798 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478701+0000", 2024-09-20T22:11:17.798 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.9986999999999999e-05 2024-09-20T22:11:17.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.799 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:17.799 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478727+0000", 2024-09-20T22:11:17.799 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.6211999999999999e-05 2024-09-20T22:11:17.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.799 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.799 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.799 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478765+0000", 2024-09-20T22:11:17.799 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7761e-05 2024-09-20T22:11:17.799 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.799 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.799 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.799 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.799 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163412 8.17 8:ed62347b:::benchmark_data_smithi060_111051_object43413:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:11:17.799 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.487680+0000", 2024-09-20T22:11:17.800 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 68.214824324999995, 2024-09-20T22:11:17.800 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.99125577600000003, 2024-09-20T22:11:17.800 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.800 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.800 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:17.800 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.800 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:17.800 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:17.800 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163412 2024-09-20T22:11:17.800 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.800 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.800 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.800 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487680+0000", 2024-09-20T22:11:17.801 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.801 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.801 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.801 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487680+0000", 2024-09-20T22:11:17.801 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.801 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.801 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.801 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487682+0000", 2024-09-20T22:11:17.801 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.759e-06 2024-09-20T22:11:17.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.801 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.801 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.802 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487688+0000", 2024-09-20T22:11:17.802 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5060000000000003e-06 2024-09-20T22:11:17.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.802 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.802 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.802 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487701+0000", 2024-09-20T22:11:17.802 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3225e-05 2024-09-20T22:11:17.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.802 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.802 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.802 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487709+0000", 2024-09-20T22:11:17.802 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5449999999999998e-06 2024-09-20T22:11:17.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.802 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.802 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.803 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487767+0000", 2024-09-20T22:11:17.803 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8387999999999997e-05 2024-09-20T22:11:17.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.804 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:17 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1335683978' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:17.805 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:17 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1335683978' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487774+0000", 2024-09-20T22:11:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.844e-06 2024-09-20T22:11:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.477116+0000", 2024-09-20T22:11:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98934203200000004 2024-09-20T22:11:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:17.805 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.477189+0000", 2024-09-20T22:11:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3220999999999996e-05 2024-09-20T22:11:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:11:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.477253+0000", 2024-09-20T22:11:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3770999999999996e-05 2024-09-20T22:11:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478774+0000", 2024-09-20T22:11:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0015216940000000001 2024-09-20T22:11:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.806 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478827+0000", 2024-09-20T22:11:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2262000000000002e-05 2024-09-20T22:11:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478872+0000", 2024-09-20T22:11:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5398999999999997e-05 2024-09-20T22:11:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478899+0000", 2024-09-20T22:11:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7146999999999999e-05 2024-09-20T22:11:17.807 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478936+0000", 2024-09-20T22:11:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6983000000000001e-05 2024-09-20T22:11:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163455 8.17 8:ef455ef1:::benchmark_data_smithi060_111051_object43456:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:11:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.524754+0000", 2024-09-20T22:11:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 68.177750564999997, 2024-09-20T22:11:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.96086008199999995, 2024-09-20T22:11:17.808 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163455 2024-09-20T22:11:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524754+0000", 2024-09-20T22:11:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.809 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524754+0000", 2024-09-20T22:11:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524756+0000", 2024-09-20T22:11:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5850000000000001e-06 2024-09-20T22:11:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524761+0000", 2024-09-20T22:11:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6699999999999999e-06 2024-09-20T22:11:17.810 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524776+0000", 2024-09-20T22:11:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4416000000000001e-05 2024-09-20T22:11:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524783+0000", 2024-09-20T22:11:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9009999999999997e-06 2024-09-20T22:11:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524851+0000", 2024-09-20T22:11:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8090999999999996e-05 2024-09-20T22:11:17.811 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524858+0000", 2024-09-20T22:11:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9299999999999997e-06 2024-09-20T22:11:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478989+0000", 2024-09-20T22:11:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.954130849 2024-09-20T22:11:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.479103+0000", 2024-09-20T22:11:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000114608 2024-09-20T22:11:17.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:11:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.479218+0000", 2024-09-20T22:11:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000115146 2024-09-20T22:11:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485313+0000", 2024-09-20T22:11:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0060945410000000002 2024-09-20T22:11:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485397+0000", 2024-09-20T22:11:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4055999999999996e-05 2024-09-20T22:11:17.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.814 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.814 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:17.814 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485483+0000", 2024-09-20T22:11:17.814 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5507000000000007e-05 2024-09-20T22:11:17.814 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.814 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.814 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:17.814 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485567+0000", 2024-09-20T22:11:17.814 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4388999999999995e-05 2024-09-20T22:11:17.814 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.814 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.814 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.814 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485614+0000", 2024-09-20T22:11:17.814 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7392999999999998e-05 2024-09-20T22:11:17.815 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.815 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.815 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.815 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.815 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163489 8.17 8:eb24224e:::benchmark_data_smithi060_111051_object43490:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:11:17.815 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.552125+0000", 2024-09-20T22:11:17.815 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 68.150379826000005, 2024-09-20T22:11:17.815 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93738651900000003, 2024-09-20T22:11:17.815 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.815 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.815 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:17.815 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.815 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:17.816 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:17.816 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163489 2024-09-20T22:11:17.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.816 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.816 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.816 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552125+0000", 2024-09-20T22:11:17.816 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.816 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.816 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552125+0000", 2024-09-20T22:11:17.816 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.816 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.817 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552126+0000", 2024-09-20T22:11:17.817 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2810000000000001e-06 2024-09-20T22:11:17.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.817 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.817 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.817 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552131+0000", 2024-09-20T22:11:17.817 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9450000000000001e-06 2024-09-20T22:11:17.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.817 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.817 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.817 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552143+0000", 2024-09-20T22:11:17.817 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1902999999999999e-05 2024-09-20T22:11:17.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.817 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552150+0000", 2024-09-20T22:11:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2609999999999996e-06 2024-09-20T22:11:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552210+0000", 2024-09-20T22:11:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9463999999999999e-05 2024-09-20T22:11:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552218+0000", 2024-09-20T22:11:17.818 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4460000000000001e-06 2024-09-20T22:11:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.480425+0000", 2024-09-20T22:11:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.928206859 2024-09-20T22:11:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.480526+0000", 2024-09-20T22:11:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100388 2024-09-20T22:11:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:11:17.819 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.480628+0000", 2024-09-20T22:11:17.820 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000102051 2024-09-20T22:11:17.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.820 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.820 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.820 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485464+0000", 2024-09-20T22:11:17.820 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0048367150000000001 2024-09-20T22:11:17.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.820 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.820 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:17.820 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485625+0000", 2024-09-20T22:11:17.820 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000160278 2024-09-20T22:11:17.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.820 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.820 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.820 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489410+0000", 2024-09-20T22:11:17.821 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0037854659999999999 2024-09-20T22:11:17.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.821 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:17.821 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489468+0000", 2024-09-20T22:11:17.821 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7883000000000001e-05 2024-09-20T22:11:17.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.821 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.821 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489512+0000", 2024-09-20T22:11:17.821 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3578999999999997e-05 2024-09-20T22:11:17.821 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.821 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.821 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.822 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.822 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163500 8.17 8:ef6c367b:::benchmark_data_smithi060_111051_object43501:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:11:17.822 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.560977+0000", 2024-09-20T22:11:17.822 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 68.141527998000001, 2024-09-20T22:11:17.822 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93477054299999995, 2024-09-20T22:11:17.822 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.822 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.822 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:17.822 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.822 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:17.822 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:17.822 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163500 2024-09-20T22:11:17.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.823 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.823 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.823 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.823 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.560977+0000", 2024-09-20T22:11:17.823 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.823 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.823 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.823 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.560977+0000", 2024-09-20T22:11:17.823 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.823 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.823 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.823 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.560978+0000", 2024-09-20T22:11:17.823 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.688e-06 2024-09-20T22:11:17.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.824 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.824 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.824 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.560984+0000", 2024-09-20T22:11:17.824 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7939999999999999e-06 2024-09-20T22:11:17.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.824 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.824 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.824 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.560997+0000", 2024-09-20T22:11:17.824 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.292e-05 2024-09-20T22:11:17.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.824 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.824 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.824 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.561006+0000", 2024-09-20T22:11:17.825 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2970000000000004e-06 2024-09-20T22:11:17.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.825 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.825 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.825 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.561071+0000", 2024-09-20T22:11:17.825 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5277000000000005e-05 2024-09-20T22:11:17.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.825 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.825 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.825 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.561080+0000", 2024-09-20T22:11:17.825 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6989999999999998e-06 2024-09-20T22:11:17.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.825 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.825 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.826 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.481917+0000", 2024-09-20T22:11:17.826 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.92083670500000003 2024-09-20T22:11:17.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.826 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.826 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:17.826 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.482017+0000", 2024-09-20T22:11:17.826 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9609000000000001e-05 2024-09-20T22:11:17.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.826 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.826 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:11:17.826 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.482117+0000", 2024-09-20T22:11:17.826 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9945999999999995e-05 2024-09-20T22:11:17.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.826 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.827 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:17.827 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485631+0000", 2024-09-20T22:11:17.827 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0035139049999999999 2024-09-20T22:11:17.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.827 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.827 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.827 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489611+0000", 2024-09-20T22:11:17.827 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0039801649999999999 2024-09-20T22:11:17.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.827 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.827 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.827 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495646+0000", 2024-09-20T22:11:17.827 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0060353660000000003 2024-09-20T22:11:17.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.827 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.828 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:17.828 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495704+0000", 2024-09-20T22:11:17.828 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7908999999999997e-05 2024-09-20T22:11:17.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.828 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.828 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.828 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495747+0000", 2024-09-20T22:11:17.828 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3263000000000002e-05 2024-09-20T22:11:17.828 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.828 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.828 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.828 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.828 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163521 8.17 8:edfbcf2c:::benchmark_data_smithi060_111051_object43522:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:11:17.828 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.577053+0000", 2024-09-20T22:11:17.829 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 68.125451549999994, 2024-09-20T22:11:17.829 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.91884361299999995, 2024-09-20T22:11:17.829 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.829 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.829 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:17.829 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.829 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:17.829 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:17.829 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163521 2024-09-20T22:11:17.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.829 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.829 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.829 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.829 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577053+0000", 2024-09-20T22:11:17.829 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.830 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.830 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.830 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577053+0000", 2024-09-20T22:11:17.830 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.830 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.830 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.830 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577055+0000", 2024-09-20T22:11:17.830 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3039999999999999e-06 2024-09-20T22:11:17.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.830 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.830 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.830 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577060+0000", 2024-09-20T22:11:17.831 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4319999999999998e-06 2024-09-20T22:11:17.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.831 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.831 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.831 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577071+0000", 2024-09-20T22:11:17.831 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1258e-05 2024-09-20T22:11:17.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.831 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.831 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.831 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577079+0000", 2024-09-20T22:11:17.831 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.374e-06 2024-09-20T22:11:17.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.831 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.831 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.831 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577177+0000", 2024-09-20T22:11:17.832 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7923000000000004e-05 2024-09-20T22:11:17.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.832 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.832 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.832 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577197+0000", 2024-09-20T22:11:17.832 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0001999999999999e-05 2024-09-20T22:11:17.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.832 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.832 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.832 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.483671+0000", 2024-09-20T22:11:17.832 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.90647459900000005 2024-09-20T22:11:17.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.832 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.832 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:17.833 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.483791+0000", 2024-09-20T22:11:17.833 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000119743 2024-09-20T22:11:17.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.833 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.833 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:11:17.833 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.483891+0000", 2024-09-20T22:11:17.833 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010016700000000001 2024-09-20T22:11:17.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.833 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.833 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:17.833 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489543+0000", 2024-09-20T22:11:17.833 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0056521109999999996 2024-09-20T22:11:17.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.833 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.833 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.834 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489657+0000", 2024-09-20T22:11:17.834 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000114249 2024-09-20T22:11:17.834 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.834 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.834 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.834 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495803+0000", 2024-09-20T22:11:17.834 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061460059999999999 2024-09-20T22:11:17.834 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.834 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.834 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:17.834 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495857+0000", 2024-09-20T22:11:17.834 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3408000000000001e-05 2024-09-20T22:11:17.834 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.834 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.835 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.835 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495897+0000", 2024-09-20T22:11:17.835 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0037000000000003e-05 2024-09-20T22:11:17.835 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.835 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.835 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.835 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.835 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163525 8.17 8:ea732579:::benchmark_data_smithi060_111051_object43526:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:11:17.835 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.577673+0000", 2024-09-20T22:11:17.835 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 68.124832244999993, 2024-09-20T22:11:17.835 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.91833821500000001, 2024-09-20T22:11:17.835 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:11:17.835 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:11:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:11:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:11:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:11:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:11:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163525 2024-09-20T22:11:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:11:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:11:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577673+0000", 2024-09-20T22:11:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:11:17.836 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577673+0000", 2024-09-20T22:11:17.837 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:11:17.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.837 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.837 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:11:17.837 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577673+0000", 2024-09-20T22:11:17.837 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1699999999999997e-07 2024-09-20T22:11:17.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.837 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.837 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:11:17.837 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577678+0000", 2024-09-20T22:11:17.837 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3200000000000001e-06 2024-09-20T22:11:17.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.837 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.837 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:11:17.838 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577689+0000", 2024-09-20T22:11:17.838 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0735e-05 2024-09-20T22:11:17.838 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.838 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.838 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:11:17.838 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577695+0000", 2024-09-20T22:11:17.838 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7549999999999997e-06 2024-09-20T22:11:17.838 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.838 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.838 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.838 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577745+0000", 2024-09-20T22:11:17.838 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9223e-05 2024-09-20T22:11:17.838 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.838 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.838 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:11:17.839 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577752+0000", 2024-09-20T22:11:17.839 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3300000000000001e-06 2024-09-20T22:11:17.839 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.839 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.839 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:11:17.839 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485694+0000", 2024-09-20T22:11:17.839 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.90794208499999995 2024-09-20T22:11:17.839 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.839 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.839 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:11:17.839 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485801+0000", 2024-09-20T22:11:17.839 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010699700000000001 2024-09-20T22:11:17.839 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.840 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.840 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:11:17.840 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485921+0000", 2024-09-20T22:11:17.840 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000120025 2024-09-20T22:11:17.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.840 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.840 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.840 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489730+0000", 2024-09-20T22:11:17.840 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003809046 2024-09-20T22:11:17.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.840 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.840 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:11:17.840 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489745+0000", 2024-09-20T22:11:17.840 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4711e-05 2024-09-20T22:11:17.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.841 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.841 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:11:17.841 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495935+0000", 2024-09-20T22:11:17.841 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061903419999999997 2024-09-20T22:11:17.841 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.841 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.841 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:11:17.841 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495975+0000", 2024-09-20T22:11:17.841 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0441000000000001e-05 2024-09-20T22:11:17.841 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:11:17.841 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:11:17.841 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:11:17.841 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.496011+0000", 2024-09-20T22:11:17.841 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5287999999999998e-05 2024-09-20T22:11:17.841 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.842 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.842 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.842 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:11:17.842 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:11:17.842 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:11:17.842 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:17 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1335683978' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:17.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:17.843 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:17.843 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:17.944 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:17.980 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:17.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:17.981 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:18.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:18.122 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:18.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:18.123 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:18.224 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:18.254 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:18.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:18.255 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:18.356 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:18.393 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:18.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:18.394 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:18.495 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:18.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:18.532 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:18.532 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:18.571 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.3 to 1 (10000) 2024-09-20T22:11:18.633 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:18.661 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:18.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:18.662 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:18.762 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:18 smithi067 ceph-mon[33511]: pgmap v446: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 415 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 768 B/s rd, 0 B/s wr, 1.06k op/s; 1014/20514 objects misplaced (4.943%); 0 B/s, 0 objects/s recovering 2024-09-20T22:11:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:18 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2495471414' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:11:18.792 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:18.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:18.793 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:18.894 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:18.929 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:18.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:18.931 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:19.028 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:18 smithi060 ceph-mon[29694]: pgmap v446: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 415 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 768 B/s rd, 0 B/s wr, 1.06k op/s; 1014/20514 objects misplaced (4.943%); 0 B/s, 0 objects/s recovering 2024-09-20T22:11:19.028 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:18 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2495471414' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:11:19.028 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:18 smithi060 ceph-mon[40290]: pgmap v446: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 415 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 768 B/s rd, 0 B/s wr, 1.06k op/s; 1014/20514 objects misplaced (4.943%); 0 B/s, 0 objects/s recovering 2024-09-20T22:11:19.028 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:18 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2495471414' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:11:19.032 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:19.057 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T22:11:19.077 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:19.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:19.079 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:19.179 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:19.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:19.217 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:19.217 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:19.318 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:19.354 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:19.354 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:19.355 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:19.455 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:19.497 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:19.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:19.498 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:19.586 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:11:19.600 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:19.628 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:19.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:19.629 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:19.730 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:19.759 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:19.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:19.760 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:19.861 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:19.902 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:19.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:19.903 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:20.003 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:19 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2495471414' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:11:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:19 smithi067 ceph-mon[33511]: osdmap e278: 8 total, 8 up, 8 in 2024-09-20T22:11:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:19 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2075867005' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:19 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:20.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:20.034 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:20.034 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:19 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2495471414' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:11:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:19 smithi060 ceph-mon[29694]: osdmap e278: 8 total, 8 up, 8 in 2024-09-20T22:11:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:19 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2075867005' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:19 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:20.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:19 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2495471414' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:11:20.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:19 smithi060 ceph-mon[40290]: osdmap e278: 8 total, 8 up, 8 in 2024-09-20T22:11:20.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:19 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2075867005' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:20.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:19 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:20.135 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:20.168 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:20.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:20.169 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:20.270 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:20.302 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:20.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:20.303 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:20.403 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:20.438 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:20.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:20.439 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:20.540 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:20.579 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:20.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:20.580 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:20.681 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:20.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:20.720 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:20.720 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:20.821 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:20.851 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:20.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:20.852 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:20.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:20 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:11:20.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:20 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:11:20.865 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:20 smithi060 ceph-mon[29694]: osdmap e279: 8 total, 8 up, 8 in 2024-09-20T22:11:20.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:20 smithi060 ceph-mon[29694]: pgmap v449: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 415 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 871 op/s; 1014/20514 objects misplaced (4.943%); 0 B/s, 0 objects/s recovering 2024-09-20T22:11:20.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:11:20.866 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:11:20.866 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:20 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:11:20.866 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:20 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:11:20.866 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:20 smithi060 ceph-mon[40290]: osdmap e279: 8 total, 8 up, 8 in 2024-09-20T22:11:20.866 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:20 smithi060 ceph-mon[40290]: pgmap v449: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 415 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 871 op/s; 1014/20514 objects misplaced (4.943%); 0 B/s, 0 objects/s recovering 2024-09-20T22:11:20.866 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:11:20.866 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:11:20.953 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:20.989 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:20.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:20.990 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:20 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:11:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:20 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:11:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:20 smithi067 ceph-mon[33511]: osdmap e279: 8 total, 8 up, 8 in 2024-09-20T22:11:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:20 smithi067 ceph-mon[33511]: pgmap v449: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 415 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 852 B/s rd, 0 B/s wr, 871 op/s; 1014/20514 objects misplaced (4.943%); 0 B/s, 0 objects/s recovering 2024-09-20T22:11:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:11:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:11:21.091 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:21.136 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:21.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:21.137 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:21.237 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:21.280 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:21.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:21.280 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:21.381 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:21.439 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 4 1 2024-09-20T22:11:21.452 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:21.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:21.453 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:21.554 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:21.607 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:21.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:21.608 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:21.709 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:21.765 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:21.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:21.766 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:21.867 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:21.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:21.901 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:21.902 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:22.002 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:22.044 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:22.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:22.045 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:22.137 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Removed 119996 objects 2024-09-20T22:11:22.137 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Clean up completed and total clean up time :114.982 2024-09-20T22:11:22.146 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:22.168 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:22.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:22.170 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:22.185 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:22.208 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:22.271 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:22.293 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:22.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:22.294 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:22.395 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:22.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:22.418 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:22.418 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:22.513 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:11:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:11:22.518 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:22.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:22.541 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:22.541 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:22.642 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:22.664 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:22.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:22.665 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:22.765 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:22.788 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:22.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:22.789 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:22.890 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:22.908 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:11:22.908 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:11:22.908 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-20T22:11:22.908 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:11:22.911 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:11:22.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:22.914 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:22.914 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:23.014 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:23.036 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:23.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:23.037 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:23.138 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:23.161 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:23.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:23.162 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:23.263 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:23.285 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:23.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:23.286 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:23.387 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:23.410 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:23.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:23.411 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:23.512 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:23.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:23 smithi060 ceph-mon[29694]: pgmap v450: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 115 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.52k op/s; 49/6093 objects misplaced (0.804%); 0 B/s, 0 objects/s recovering 2024-09-20T22:11:23.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:23 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:11:23.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:23 smithi060 ceph-mon[40290]: pgmap v450: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 115 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.52k op/s; 49/6093 objects misplaced (0.804%); 0 B/s, 0 objects/s recovering 2024-09-20T22:11:23.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:23 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:11:23.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:23.537 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:23.638 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:23.661 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:23.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:23.662 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:23.762 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:23.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:23 smithi067 ceph-mon[33511]: pgmap v450: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 115 MiB data, 3.7 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1.52k op/s; 49/6093 objects misplaced (0.804%); 0 B/s, 0 objects/s recovering 2024-09-20T22:11:23.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:23 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:11:23.769 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T22:11:23.785 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:23.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:23.786 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:23.833 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:23.886 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:23.912 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:23.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:23.913 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:24.014 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:24.036 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:24.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:24.037 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:24.138 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:24.160 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:24.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:24.161 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:24.191 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:11:24.262 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:24.288 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:24.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:24.289 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:24.308 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:11:24.308 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:11:24.329 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:11:24.329 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:11:24.335 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:24.347 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:11:24.347 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:11:24.366 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:11:24.366 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:11:24.384 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:11:24.384 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:11:24.390 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:24.402 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:11:24.402 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:11:24.413 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:24.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:24.414 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:24.420 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:11:24.420 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:11:24.437 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:11:24.438 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:11:24.475 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.4 to 1 (10000) 2024-09-20T22:11:24.514 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:24.518 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_1 2024-09-20T22:11:24.518 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool rm unique_pool_1 unique_pool_1 --yes-i-really-really-mean-it 2024-09-20T22:11:24.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:24.537 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:24.537 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:24.559 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:24 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3394740703' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:11:24.559 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:24 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:11:24.559 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:24 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/840353982' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:24.559 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:24 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3394740703' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:11:24.560 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:24 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:11:24.560 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:24 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/840353982' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:24.638 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:24.661 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:24.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:24.662 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:24.762 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:24.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:24 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3394740703' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:11:24.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:24 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:11:24.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:24 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/840353982' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:24.789 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:24.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:24.790 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:24.891 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:24.913 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:24.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:24.914 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:25.015 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:25.042 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:25.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:25.043 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:25.144 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:25.170 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:25.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:25.171 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:25.271 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:25.285 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:11:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:11:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:11:25.294 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:25.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:25.295 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:25.396 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:25.413 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:11:25.414 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:11:25.418 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:25.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:25.419 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:25.519 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:25.541 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:25.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:25.542 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:25.643 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:25.669 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:25.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:25.670 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:25 smithi067 ceph-mon[33511]: pgmap v451: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 67 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.06k op/s; 49/3771 objects misplaced (1.299%); 0 B/s, 0 objects/s recovering 2024-09-20T22:11:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:25 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T22:11:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:25 smithi067 ceph-mon[33511]: osdmap e280: 8 total, 8 up, 8 in 2024-09-20T22:11:25.771 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:25.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:25 smithi060 ceph-mon[29694]: pgmap v451: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 67 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.06k op/s; 49/3771 objects misplaced (1.299%); 0 B/s, 0 objects/s recovering 2024-09-20T22:11:25.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:25 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T22:11:25.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:25 smithi060 ceph-mon[29694]: osdmap e280: 8 total, 8 up, 8 in 2024-09-20T22:11:25.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:25 smithi060 ceph-mon[40290]: pgmap v451: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 67 MiB data, 3.6 GiB used, 712 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.06k op/s; 49/3771 objects misplaced (1.299%); 0 B/s, 0 objects/s recovering 2024-09-20T22:11:25.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:25 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T22:11:25.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:25 smithi060 ceph-mon[40290]: osdmap e280: 8 total, 8 up, 8 in 2024-09-20T22:11:25.796 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:25.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:25.797 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:25.898 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:25.920 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:25.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:25.921 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:26.022 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:26.048 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:26.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:26.049 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:26.150 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:26.177 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:26.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:26.178 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:26.278 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:26.305 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:26.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:26.306 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:26.407 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:26.432 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:26.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:26.433 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:26.511 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:26.514 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 5 1 2024-09-20T22:11:26.534 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:26.559 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:26.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:26.559 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:26.660 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:26.684 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:26.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:26.685 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:26.786 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:26.810 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:26.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:26.810 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:26.911 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:26.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T22:11:26.935 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:26.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:26.936 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:26.936 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:27.036 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:27.059 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:27.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:27.060 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:27.161 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:27.182 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:27.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:27.183 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:27.284 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:27.308 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:27.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:27.309 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:27.410 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:27.435 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:27.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:27.436 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:27.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:27 smithi060 ceph-mon[29694]: pgmap v453: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 19 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 840 op/s; 49/1506 objects misplaced (3.254%) 2024-09-20T22:11:27.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:27 smithi060 ceph-mon[40290]: pgmap v453: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 19 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 840 op/s; 49/1506 objects misplaced (3.254%) 2024-09-20T22:11:27.537 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:27.560 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:27.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:27.560 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:27.661 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:27.683 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:27.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:27.684 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:27.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:27 smithi067 ceph-mon[33511]: pgmap v453: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 19 MiB data, 3.3 GiB used, 712 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 840 op/s; 49/1506 objects misplaced (3.254%) 2024-09-20T22:11:27.785 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:27.812 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:27.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:27.813 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:27.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:27.936 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:27.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:27.937 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:27.987 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:28.037 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:28.059 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:28.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:28.060 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:28.138 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:11:28.161 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:28.187 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:28.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:28.188 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:28.260 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:28.289 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:28.315 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:28.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:28.316 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:28.416 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:28.440 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:28.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:28.441 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:28.468 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:11:28.481 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'unique_pool_1' removed 2024-09-20T22:11:28.542 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:28.576 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:28.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:28.577 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:28.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:28.701 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:28.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:28.701 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:28 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/321901977' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:11:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:28 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:11:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:28 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3737132950' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:11:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:28 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/321901977' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:11:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:28 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:11:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:28 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3737132950' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:11:28.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:28 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/321901977' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:11:28.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:28 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:11:28.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:28 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3737132950' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:11:28.802 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:28.824 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:28.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:28.826 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:28.926 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:28.948 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:28.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:28.949 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:29.050 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:29.071 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:29.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:29.072 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:29.173 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:29.199 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:29.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:29.200 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:29.300 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:29.323 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:29.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:29.325 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:29.390 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:11:29.390 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:11:29.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:29.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:29.450 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:29.450 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:29.474 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.5 to 1 (10000) 2024-09-20T22:11:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:29 smithi060 ceph-mon[29694]: pgmap v454: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 778 op/s 2024-09-20T22:11:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:29 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:11:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:29 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T22:11:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:29 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3737132950' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:11:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:29 smithi060 ceph-mon[29694]: osdmap e281: 8 total, 8 up, 8 in 2024-09-20T22:11:29.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:29 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/676715916' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:11:29.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:29 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:11:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:29 smithi060 ceph-mon[40290]: pgmap v454: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 778 op/s 2024-09-20T22:11:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:29 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:11:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:29 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T22:11:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:29 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3737132950' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:11:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:29 smithi060 ceph-mon[40290]: osdmap e281: 8 total, 8 up, 8 in 2024-09-20T22:11:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:29 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/676715916' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:11:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:29 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:11:29.551 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:29.573 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:29.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:29.574 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:29.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:29.697 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:29.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:29.698 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:29.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:29 smithi067 ceph-mon[33511]: pgmap v454: 168 pgs: 2 active+remapped+backfill_toofull, 166 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 778 op/s 2024-09-20T22:11:29.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:29 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:11:29.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:29 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_1", "pool2": "unique_pool_1", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T22:11:29.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:29 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3737132950' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:11:29.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:29 smithi067 ceph-mon[33511]: osdmap e281: 8 total, 8 up, 8 in 2024-09-20T22:11:29.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:29 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/676715916' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:11:29.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:29 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:11:29.799 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:29.822 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:29.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:29.823 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:29.924 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:29.949 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:29.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:29.950 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:30.051 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:30.078 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:30.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:30.079 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:30.179 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:30.206 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:30.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:30.207 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:30.307 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:30.332 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:30.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:30.333 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:30.346 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-20T22:11:30.346 INFO:tasks.radosbench:Beginning radosbench... 2024-09-20T22:11:30.346 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_2 2024-09-20T22:11:30.347 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool create unique_pool_2 16 2024-09-20T22:11:30.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:30.455 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:30.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:30.457 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:30.558 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:30.583 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:30.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:30.585 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:30.685 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:30.708 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:30.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:30.709 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:30 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:11:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:30 smithi067 ceph-mon[33511]: osdmap e282: 8 total, 8 up, 8 in 2024-09-20T22:11:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:30 smithi067 ceph-mon[33511]: pgmap v457: 142 pgs: 142 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T22:11:30.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:30 smithi067 ceph-mon[33511]: osdmap e283: 8 total, 8 up, 8 in 2024-09-20T22:11:30.767 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:30 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:11:30.767 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:30 smithi060 ceph-mon[29694]: osdmap e282: 8 total, 8 up, 8 in 2024-09-20T22:11:30.767 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:30 smithi060 ceph-mon[29694]: pgmap v457: 142 pgs: 142 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T22:11:30.767 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:30 smithi060 ceph-mon[29694]: osdmap e283: 8 total, 8 up, 8 in 2024-09-20T22:11:30.767 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:30 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:11:30.767 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:30 smithi060 ceph-mon[40290]: osdmap e282: 8 total, 8 up, 8 in 2024-09-20T22:11:30.767 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:30 smithi060 ceph-mon[40290]: pgmap v457: 142 pgs: 142 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T22:11:30.767 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:30 smithi060 ceph-mon[40290]: osdmap e283: 8 total, 8 up, 8 in 2024-09-20T22:11:30.809 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:30.830 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:30.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:30.831 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:30.932 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:30.959 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:30.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:30.960 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:31.061 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:31.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:31.090 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:31.090 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:31.190 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 6 1 2024-09-20T22:11:31.191 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:31.196 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:31.214 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:31.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:31.215 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:31.316 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:31.338 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:31.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:31.339 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:31.440 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:31.467 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:31.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:31.468 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:31.568 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:31.591 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:31.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:31.592 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:31.693 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:31.713 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:31.716 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:31.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:31.717 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:31.745 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:11:31.746 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:11:31.746 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:31 smithi060 ceph-mon[29694]: 6.4 scrub starts 2024-09-20T22:11:31.746 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:31 smithi060 ceph-mon[29694]: 6.4 scrub ok 2024-09-20T22:11:31.746 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:31 smithi060 ceph-mon[29694]: 4.0 scrub starts 2024-09-20T22:11:31.746 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:31 smithi060 ceph-mon[29694]: 4.0 scrub ok 2024-09-20T22:11:31.746 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:31 smithi060 ceph-mon[29694]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-20T22:11:31.747 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:31 smithi060 ceph-mon[29694]: osdmap e284: 8 total, 8 up, 8 in 2024-09-20T22:11:31.747 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:31 smithi060 ceph-mon[40290]: 6.4 scrub starts 2024-09-20T22:11:31.747 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:31 smithi060 ceph-mon[40290]: 6.4 scrub ok 2024-09-20T22:11:31.747 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:31 smithi060 ceph-mon[40290]: 4.0 scrub starts 2024-09-20T22:11:31.747 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:31 smithi060 ceph-mon[40290]: 4.0 scrub ok 2024-09-20T22:11:31.747 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:31 smithi060 ceph-mon[40290]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-20T22:11:31.747 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:31 smithi060 ceph-mon[40290]: osdmap e284: 8 total, 8 up, 8 in 2024-09-20T22:11:31.763 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:11:31.763 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:11:31.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:31 smithi067 ceph-mon[33511]: 6.4 scrub starts 2024-09-20T22:11:31.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:31 smithi067 ceph-mon[33511]: 6.4 scrub ok 2024-09-20T22:11:31.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:31 smithi067 ceph-mon[33511]: 4.0 scrub starts 2024-09-20T22:11:31.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:31 smithi067 ceph-mon[33511]: 4.0 scrub ok 2024-09-20T22:11:31.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:31 smithi067 ceph-mon[33511]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-20T22:11:31.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:31 smithi067 ceph-mon[33511]: osdmap e284: 8 total, 8 up, 8 in 2024-09-20T22:11:31.783 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:11:31.784 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:11:31.800 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:11:31.800 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:11:31.817 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:11:31.817 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:11:31.817 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:31.834 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:11:31.835 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:11:31.844 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:31.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:31.845 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:31.852 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:11:31.852 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:11:31.869 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:11:31.869 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:11:31.945 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:31.968 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:31.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:31.969 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:32.070 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:32.098 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:32.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:32.099 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:32.200 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:32.227 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:32.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:32.228 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:32.328 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:32.355 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:32.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:32.356 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:32.457 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:32.480 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:32.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:32.481 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:32.497 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:11:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:11:32.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[29694]: 4.2 scrub starts 2024-09-20T22:11:32.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[29694]: 4.2 scrub ok 2024-09-20T22:11:32.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[29694]: 5.1b scrub starts 2024-09-20T22:11:32.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[29694]: 5.1b scrub ok 2024-09-20T22:11:32.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[29694]: 5.13 scrub starts 2024-09-20T22:11:32.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[29694]: 5.13 scrub ok 2024-09-20T22:11:32.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[29694]: 6.f scrub starts 2024-09-20T22:11:32.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[29694]: 6.f scrub ok 2024-09-20T22:11:32.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1043545596' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:32.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[29694]: pgmap v460: 142 pgs: 2 remapped+peering, 140 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:11:32.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:11:32.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[40290]: 4.2 scrub starts 2024-09-20T22:11:32.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[40290]: 4.2 scrub ok 2024-09-20T22:11:32.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[40290]: 5.1b scrub starts 2024-09-20T22:11:32.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[40290]: 5.1b scrub ok 2024-09-20T22:11:32.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[40290]: 5.13 scrub starts 2024-09-20T22:11:32.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[40290]: 5.13 scrub ok 2024-09-20T22:11:32.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[40290]: 6.f scrub starts 2024-09-20T22:11:32.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[40290]: 6.f scrub ok 2024-09-20T22:11:32.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1043545596' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:32.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[40290]: pgmap v460: 142 pgs: 2 remapped+peering, 140 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:11:32.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:32 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:11:32.582 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:32.609 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:32.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:32.610 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:32.710 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:32.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:32.736 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:32.737 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:32.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:32 smithi067 ceph-mon[33511]: 4.2 scrub starts 2024-09-20T22:11:32.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:32 smithi067 ceph-mon[33511]: 4.2 scrub ok 2024-09-20T22:11:32.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:32 smithi067 ceph-mon[33511]: 5.1b scrub starts 2024-09-20T22:11:32.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:32 smithi067 ceph-mon[33511]: 5.1b scrub ok 2024-09-20T22:11:32.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:32 smithi067 ceph-mon[33511]: 5.13 scrub starts 2024-09-20T22:11:32.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:32 smithi067 ceph-mon[33511]: 5.13 scrub ok 2024-09-20T22:11:32.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:32 smithi067 ceph-mon[33511]: 6.f scrub starts 2024-09-20T22:11:32.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:32 smithi067 ceph-mon[33511]: 6.f scrub ok 2024-09-20T22:11:32.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:32 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1043545596' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:32.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:32 smithi067 ceph-mon[33511]: pgmap v460: 142 pgs: 2 remapped+peering, 140 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:11:32.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:32 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:11:32.800 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:11:32.837 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:32.866 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:32.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:32.867 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:32.968 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:32.990 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:32.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:32.991 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:33.092 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:33.118 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:33.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:33.119 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:33.220 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:33.242 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:33.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:33.243 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:33.344 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:33.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:33.367 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:33.367 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:33.468 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:33.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:33.491 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:33.491 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:33.552 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'unique_pool_2' created 2024-09-20T22:11:33.591 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T22:11:33.592 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:33.620 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:33.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:33.622 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:33.724 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:33.746 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:33.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:33.748 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:33.810 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:33.835 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:33.848 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:33.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:33.879 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:33.880 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:33.930 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:33 smithi060 ceph-mon[29694]: 5.4 scrub starts 2024-09-20T22:11:33.930 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:33 smithi060 ceph-mon[29694]: 5.4 scrub ok 2024-09-20T22:11:33.930 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:33 smithi060 ceph-mon[29694]: osdmap e285: 8 total, 8 up, 8 in 2024-09-20T22:11:33.930 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:33 smithi060 ceph-mon[40290]: 5.4 scrub starts 2024-09-20T22:11:33.930 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:33 smithi060 ceph-mon[40290]: 5.4 scrub ok 2024-09-20T22:11:33.930 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:33 smithi060 ceph-mon[40290]: osdmap e285: 8 total, 8 up, 8 in 2024-09-20T22:11:33.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:34.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:34.008 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:34.009 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:34.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:33 smithi067 ceph-mon[33511]: 5.4 scrub starts 2024-09-20T22:11:34.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:33 smithi067 ceph-mon[33511]: 5.4 scrub ok 2024-09-20T22:11:34.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:33 smithi067 ceph-mon[33511]: osdmap e285: 8 total, 8 up, 8 in 2024-09-20T22:11:34.110 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:34.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:34.142 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:34.142 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:34.243 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:34.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:34.276 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:34.276 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:34.377 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:34.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:34.404 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:34.404 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:34.505 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:34.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:34.534 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:34.535 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:34.636 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:34.662 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:34.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:34.663 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:34.719 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:11:34.764 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:34.790 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:34.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:34.791 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:34.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/358508614' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-20T22:11:34.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-20T22:11:34.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[29694]: 4.7 scrub starts 2024-09-20T22:11:34.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[29694]: 4.7 scrub ok 2024-09-20T22:11:34.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[29694]: 3.14 scrub starts 2024-09-20T22:11:34.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[29694]: 3.14 scrub ok 2024-09-20T22:11:34.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-20T22:11:34.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[29694]: osdmap e286: 8 total, 8 up, 8 in 2024-09-20T22:11:34.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[29694]: pgmap v463: 158 pgs: 16 unknown, 2 remapped+peering, 140 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 0 B/s, 1 objects/s recovering 2024-09-20T22:11:34.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[29694]: 5.11 scrub starts 2024-09-20T22:11:34.837 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[29694]: 5.11 scrub ok 2024-09-20T22:11:34.838 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[29694]: osdmap e287: 8 total, 8 up, 8 in 2024-09-20T22:11:34.838 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/358508614' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-20T22:11:34.838 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-20T22:11:34.838 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[40290]: 4.7 scrub starts 2024-09-20T22:11:34.838 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[40290]: 4.7 scrub ok 2024-09-20T22:11:34.838 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[40290]: 3.14 scrub starts 2024-09-20T22:11:34.838 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[40290]: 3.14 scrub ok 2024-09-20T22:11:34.838 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-20T22:11:34.839 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[40290]: osdmap e286: 8 total, 8 up, 8 in 2024-09-20T22:11:34.839 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[40290]: pgmap v463: 158 pgs: 16 unknown, 2 remapped+peering, 140 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 0 B/s, 1 objects/s recovering 2024-09-20T22:11:34.839 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[40290]: 5.11 scrub starts 2024-09-20T22:11:34.839 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[40290]: 5.11 scrub ok 2024-09-20T22:11:34.839 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:34 smithi060 ceph-mon[40290]: osdmap e287: 8 total, 8 up, 8 in 2024-09-20T22:11:34.843 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:34.892 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:34.918 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:34.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:34.919 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:34 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/358508614' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-20T22:11:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:34 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]: dispatch 2024-09-20T22:11:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:34 smithi067 ceph-mon[33511]: 4.7 scrub starts 2024-09-20T22:11:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:34 smithi067 ceph-mon[33511]: 4.7 scrub ok 2024-09-20T22:11:35.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:34 smithi067 ceph-mon[33511]: 3.14 scrub starts 2024-09-20T22:11:35.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:34 smithi067 ceph-mon[33511]: 3.14 scrub ok 2024-09-20T22:11:35.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:34 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_2", "pg_num": 16}]': finished 2024-09-20T22:11:35.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:34 smithi067 ceph-mon[33511]: osdmap e286: 8 total, 8 up, 8 in 2024-09-20T22:11:35.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:34 smithi067 ceph-mon[33511]: pgmap v463: 158 pgs: 16 unknown, 2 remapped+peering, 140 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s; 0 B/s, 1 objects/s recovering 2024-09-20T22:11:35.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:34 smithi067 ceph-mon[33511]: 5.11 scrub starts 2024-09-20T22:11:35.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:34 smithi067 ceph-mon[33511]: 5.11 scrub ok 2024-09-20T22:11:35.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:34 smithi067 ceph-mon[33511]: osdmap e287: 8 total, 8 up, 8 in 2024-09-20T22:11:35.019 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:35.043 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:35.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:35.044 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:35.089 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:11:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:11:35.145 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:35.167 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:35.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:35.168 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:35.269 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:35.293 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:35.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:35.294 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:35.294 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:11:35.294 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:11:35.295 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-20T22:11:35.295 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:11:35.394 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:35.418 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:35.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:35.419 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:35.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:35.543 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:35.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:35.544 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:35.551 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:11:35.555 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.6 to 1 (10000) 2024-09-20T22:11:35.645 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:35.667 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:35.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:35.668 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:35.769 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:35.794 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:35.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:35.795 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:35.895 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:35.918 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:35.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:35.919 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:36.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: 6.19 scrub starts 2024-09-20T22:11:36.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: 6.19 scrub ok 2024-09-20T22:11:36.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: 5.12 scrub starts 2024-09-20T22:11:36.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: 5.12 scrub ok 2024-09-20T22:11:36.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: 4.f scrub starts 2024-09-20T22:11:36.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: 4.f scrub ok 2024-09-20T22:11:36.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: 1.9 scrub starts 2024-09-20T22:11:36.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: 1.9 scrub ok 2024-09-20T22:11:36.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: 5.1c scrub starts 2024-09-20T22:11:36.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: 5.1c scrub ok 2024-09-20T22:11:36.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T22:11:36.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3630958122' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:11:36.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:11:36.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1041415652' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:36.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:11:36.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:11:36.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:11:36.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:11:36.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1041415652' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:11:36.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:35 smithi067 ceph-mon[33511]: osdmap e288: 8 total, 8 up, 8 in 2024-09-20T22:11:36.020 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: 6.19 scrub starts 2024-09-20T22:11:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: 6.19 scrub ok 2024-09-20T22:11:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: 5.12 scrub starts 2024-09-20T22:11:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: 5.12 scrub ok 2024-09-20T22:11:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: 4.f scrub starts 2024-09-20T22:11:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: 4.f scrub ok 2024-09-20T22:11:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: 1.9 scrub starts 2024-09-20T22:11:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: 1.9 scrub ok 2024-09-20T22:11:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: 5.1c scrub starts 2024-09-20T22:11:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: 5.1c scrub ok 2024-09-20T22:11:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T22:11:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3630958122' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:11:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:11:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1041415652' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:11:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:11:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:11:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:11:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1041415652' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:11:36.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[29694]: osdmap e288: 8 total, 8 up, 8 in 2024-09-20T22:11:36.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: 6.19 scrub starts 2024-09-20T22:11:36.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: 6.19 scrub ok 2024-09-20T22:11:36.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: 5.12 scrub starts 2024-09-20T22:11:36.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: 5.12 scrub ok 2024-09-20T22:11:36.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: 4.f scrub starts 2024-09-20T22:11:36.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: 4.f scrub ok 2024-09-20T22:11:36.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: 1.9 scrub starts 2024-09-20T22:11:36.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: 1.9 scrub ok 2024-09-20T22:11:36.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: 5.1c scrub starts 2024-09-20T22:11:36.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: 5.1c scrub ok 2024-09-20T22:11:36.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T22:11:36.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3630958122' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:11:36.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:11:36.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1041415652' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:36.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:11:36.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:11:36.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:11:36.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:11:36.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1041415652' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:11:36.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:35 smithi060 ceph-mon[40290]: osdmap e288: 8 total, 8 up, 8 in 2024-09-20T22:11:36.047 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:36.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:36.048 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:36.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:36.171 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:36.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:36.172 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:36.273 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:36.300 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:36.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:36.301 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:36.402 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:36.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:36.430 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:36.430 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:36.531 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:36.558 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:36.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:36.559 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:36.597 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 7 1 2024-09-20T22:11:36.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:36.685 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:36.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:36.686 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:36.787 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:36.809 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:36.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:36.810 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:36.911 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:36.939 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:36.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:36.940 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:36.960 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[40290]: 5.15 scrub starts 2024-09-20T22:11:36.960 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[40290]: 5.15 scrub ok 2024-09-20T22:11:36.960 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[40290]: 2.1 scrub starts 2024-09-20T22:11:36.960 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[40290]: 2.1 scrub ok 2024-09-20T22:11:36.960 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[40290]: 4.1e scrub starts 2024-09-20T22:11:36.960 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[40290]: 4.1e scrub ok 2024-09-20T22:11:36.960 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[40290]: pgmap v466: 158 pgs: 9 unknown, 1 remapped+peering, 148 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 64 KiB/s, 1 objects/s recovering 2024-09-20T22:11:36.960 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[40290]: 1.6 scrub starts 2024-09-20T22:11:36.960 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[40290]: 1.6 scrub ok 2024-09-20T22:11:36.960 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[40290]: osdmap e289: 8 total, 8 up, 8 in 2024-09-20T22:11:36.961 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[29694]: 5.15 scrub starts 2024-09-20T22:11:36.961 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[29694]: 5.15 scrub ok 2024-09-20T22:11:36.961 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[29694]: 2.1 scrub starts 2024-09-20T22:11:36.961 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[29694]: 2.1 scrub ok 2024-09-20T22:11:36.961 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[29694]: 4.1e scrub starts 2024-09-20T22:11:36.961 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[29694]: 4.1e scrub ok 2024-09-20T22:11:36.961 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[29694]: pgmap v466: 158 pgs: 9 unknown, 1 remapped+peering, 148 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 64 KiB/s, 1 objects/s recovering 2024-09-20T22:11:36.961 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[29694]: 1.6 scrub starts 2024-09-20T22:11:36.961 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[29694]: 1.6 scrub ok 2024-09-20T22:11:36.962 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:36 smithi060 ceph-mon[29694]: osdmap e289: 8 total, 8 up, 8 in 2024-09-20T22:11:36.988 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T22:11:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:36 smithi067 ceph-mon[33511]: 5.15 scrub starts 2024-09-20T22:11:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:36 smithi067 ceph-mon[33511]: 5.15 scrub ok 2024-09-20T22:11:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:36 smithi067 ceph-mon[33511]: 2.1 scrub starts 2024-09-20T22:11:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:36 smithi067 ceph-mon[33511]: 2.1 scrub ok 2024-09-20T22:11:37.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:36 smithi067 ceph-mon[33511]: 4.1e scrub starts 2024-09-20T22:11:37.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:36 smithi067 ceph-mon[33511]: 4.1e scrub ok 2024-09-20T22:11:37.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:36 smithi067 ceph-mon[33511]: pgmap v466: 158 pgs: 9 unknown, 1 remapped+peering, 148 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 64 KiB/s, 1 objects/s recovering 2024-09-20T22:11:37.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:36 smithi067 ceph-mon[33511]: 1.6 scrub starts 2024-09-20T22:11:37.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:36 smithi067 ceph-mon[33511]: 1.6 scrub ok 2024-09-20T22:11:37.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:36 smithi067 ceph-mon[33511]: osdmap e289: 8 total, 8 up, 8 in 2024-09-20T22:11:37.042 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:37.066 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:37.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:37.067 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:37.168 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:37.195 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:37.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:37.196 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:37.297 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:37.324 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:37.325 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:37.326 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:37.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:37.451 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:37.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:37.452 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:37.553 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:37.576 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:37.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:37.577 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:37.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:37.700 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:37.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:37.701 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:37.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:37 smithi060 ceph-mon[29694]: 1.3 scrub starts 2024-09-20T22:11:37.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:37 smithi060 ceph-mon[29694]: 1.3 scrub ok 2024-09-20T22:11:37.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:37 smithi060 ceph-mon[29694]: 5.8 scrub starts 2024-09-20T22:11:37.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:37 smithi060 ceph-mon[29694]: 5.8 scrub ok 2024-09-20T22:11:37.709 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:37 smithi060 ceph-mon[29694]: 3.1b scrub starts 2024-09-20T22:11:37.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:37 smithi060 ceph-mon[29694]: 3.1b scrub ok 2024-09-20T22:11:37.710 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:37 smithi060 ceph-mon[40290]: 1.3 scrub starts 2024-09-20T22:11:37.710 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:37 smithi060 ceph-mon[40290]: 1.3 scrub ok 2024-09-20T22:11:37.710 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:37 smithi060 ceph-mon[40290]: 5.8 scrub starts 2024-09-20T22:11:37.710 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:37 smithi060 ceph-mon[40290]: 5.8 scrub ok 2024-09-20T22:11:37.710 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:37 smithi060 ceph-mon[40290]: 3.1b scrub starts 2024-09-20T22:11:37.710 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:37 smithi060 ceph-mon[40290]: 3.1b scrub ok 2024-09-20T22:11:37.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool application enable unique_pool_2 rados --yes-i-really-mean-it || true 2024-09-20T22:11:37.802 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:37.829 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:37.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:37.830 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:37.916 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:37.930 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:37.957 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:37.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:37.958 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:37.959 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:37.961 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:37 smithi067 ceph-mon[33511]: 1.3 scrub starts 2024-09-20T22:11:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:37 smithi067 ceph-mon[33511]: 1.3 scrub ok 2024-09-20T22:11:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:37 smithi067 ceph-mon[33511]: 5.8 scrub starts 2024-09-20T22:11:38.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:37 smithi067 ceph-mon[33511]: 5.8 scrub ok 2024-09-20T22:11:38.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:37 smithi067 ceph-mon[33511]: 3.1b scrub starts 2024-09-20T22:11:38.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:37 smithi067 ceph-mon[33511]: 3.1b scrub ok 2024-09-20T22:11:38.059 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:38.081 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:38.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:38.082 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:38.183 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:38.210 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:38.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:38.211 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:38.311 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:38.338 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:38.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:38.339 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:38.440 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:38.463 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:38.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:38.464 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:38.565 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:38.593 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:38.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:38.594 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:38.694 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:38.721 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:38.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:38.722 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:38.822 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:38.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:38.847 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:38.847 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:38.892 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:38 smithi060 ceph-mon[29694]: 3.19 deep-scrub starts 2024-09-20T22:11:38.892 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:38 smithi060 ceph-mon[29694]: 3.19 deep-scrub ok 2024-09-20T22:11:38.892 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:38 smithi060 ceph-mon[29694]: 4.c deep-scrub starts 2024-09-20T22:11:38.892 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:38 smithi060 ceph-mon[29694]: 4.c deep-scrub ok 2024-09-20T22:11:38.892 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:38 smithi060 ceph-mon[29694]: pgmap v468: 158 pgs: 158 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 126 KiB/s, 0 objects/s recovering 2024-09-20T22:11:38.892 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:38 smithi060 ceph-mon[40290]: 3.19 deep-scrub starts 2024-09-20T22:11:38.892 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:38 smithi060 ceph-mon[40290]: 3.19 deep-scrub ok 2024-09-20T22:11:38.892 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:38 smithi060 ceph-mon[40290]: 4.c deep-scrub starts 2024-09-20T22:11:38.892 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:38 smithi060 ceph-mon[40290]: 4.c deep-scrub ok 2024-09-20T22:11:38.892 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:38 smithi060 ceph-mon[40290]: pgmap v468: 158 pgs: 158 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 126 KiB/s, 0 objects/s recovering 2024-09-20T22:11:38.898 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:11:38.898 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:11:38.898 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:11:38.898 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:11:38.898 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:11:38.948 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:38.973 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:38.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:38.974 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:38 smithi067 ceph-mon[33511]: 3.19 deep-scrub starts 2024-09-20T22:11:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:38 smithi067 ceph-mon[33511]: 3.19 deep-scrub ok 2024-09-20T22:11:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:38 smithi067 ceph-mon[33511]: 4.c deep-scrub starts 2024-09-20T22:11:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:38 smithi067 ceph-mon[33511]: 4.c deep-scrub ok 2024-09-20T22:11:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:38 smithi067 ceph-mon[33511]: pgmap v468: 158 pgs: 158 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 126 KiB/s, 0 objects/s recovering 2024-09-20T22:11:39.075 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:39.099 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:39.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:39.100 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:39.200 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:39.222 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:39.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:39.223 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:39.324 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:39.351 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:39.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:39.352 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:39.452 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:39.478 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:39.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:39.479 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:39.580 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:39.586 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.7 to 1 (10000) 2024-09-20T22:11:39.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:39 smithi060 ceph-mon[29694]: 2.2 scrub starts 2024-09-20T22:11:39.587 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:39 smithi060 ceph-mon[29694]: 2.2 scrub ok 2024-09-20T22:11:39.609 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:39.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:39.610 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:39.710 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:39.736 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:39.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:39.737 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:39.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:39.860 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:39.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:39.861 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:39.932 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:39 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3119276020' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:11:39.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:39 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:11:39.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:39 smithi060 ceph-mon[29694]: 3.10 scrub starts 2024-09-20T22:11:39.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:39 smithi060 ceph-mon[29694]: 3.10 scrub ok 2024-09-20T22:11:39.933 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:39 smithi060 ceph-mon[40290]: 2.2 scrub starts 2024-09-20T22:11:39.933 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:39 smithi060 ceph-mon[40290]: 2.2 scrub ok 2024-09-20T22:11:39.933 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:39 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3119276020' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:11:39.933 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:39 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:11:39.933 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:39 smithi060 ceph-mon[40290]: 3.10 scrub starts 2024-09-20T22:11:39.933 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:39 smithi060 ceph-mon[40290]: 3.10 scrub ok 2024-09-20T22:11:39.953 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:11:39.962 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:39 smithi067 ceph-mon[33511]: 2.2 scrub starts 2024-09-20T22:11:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:39 smithi067 ceph-mon[33511]: 2.2 scrub ok 2024-09-20T22:11:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:39 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3119276020' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:11:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:39 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:11:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:39 smithi067 ceph-mon[33511]: 3.10 scrub starts 2024-09-20T22:11:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:39 smithi067 ceph-mon[33511]: 3.10 scrub ok 2024-09-20T22:11:40.018 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:40.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:40.019 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:40.120 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:40.143 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:40.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:40.144 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:40.245 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:40.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T22:11:40.272 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:40.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:40.273 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:40.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:40.396 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:40.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:40.397 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:40.497 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:40.525 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:40.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:40.526 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:40.626 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:40.652 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:40.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:40.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:40.754 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:40.780 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:40.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:40.782 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:40.818 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-20T22:11:40.818 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:11:40.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[29694]: 5.7 scrub starts 2024-09-20T22:11:40.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[29694]: 5.7 scrub ok 2024-09-20T22:11:40.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[29694]: 6.1 scrub starts 2024-09-20T22:11:40.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[29694]: 6.1 scrub ok 2024-09-20T22:11:40.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[29694]: 6.1c scrub starts 2024-09-20T22:11:40.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[29694]: 6.1c scrub ok 2024-09-20T22:11:40.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[29694]: 1.4 scrub starts 2024-09-20T22:11:40.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[29694]: 1.4 scrub ok 2024-09-20T22:11:40.841 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:11:40.842 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[29694]: osdmap e290: 8 total, 8 up, 8 in 2024-09-20T22:11:40.842 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3522218138' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:40.842 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[29694]: pgmap v470: 158 pgs: 158 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 917 B/s rd, 0 op/s; 103 KiB/s, 0 objects/s recovering 2024-09-20T22:11:40.842 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[40290]: 5.7 scrub starts 2024-09-20T22:11:40.842 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[40290]: 5.7 scrub ok 2024-09-20T22:11:40.842 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[40290]: 6.1 scrub starts 2024-09-20T22:11:40.842 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[40290]: 6.1 scrub ok 2024-09-20T22:11:40.842 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[40290]: 6.1c scrub starts 2024-09-20T22:11:40.842 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[40290]: 6.1c scrub ok 2024-09-20T22:11:40.842 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[40290]: 1.4 scrub starts 2024-09-20T22:11:40.842 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[40290]: 1.4 scrub ok 2024-09-20T22:11:40.843 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:11:40.843 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[40290]: osdmap e290: 8 total, 8 up, 8 in 2024-09-20T22:11:40.843 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3522218138' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:40.843 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:40 smithi060 ceph-mon[40290]: pgmap v470: 158 pgs: 158 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 917 B/s rd, 0 op/s; 103 KiB/s, 0 objects/s recovering 2024-09-20T22:11:40.882 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:40.908 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:40.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:40.909 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:40.988 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:11:40.988 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:11:40.988 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:11:40.988 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:11:40.989 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:11:40.989 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:11:40.989 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:11:40.989 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:11:40.989 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:11:40.989 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:11:40.989 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:11:40.989 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:11:40.989 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:11:40.990 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:11:40.990 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:11:40.990 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:11:41.010 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:40 smithi067 ceph-mon[33511]: 5.7 scrub starts 2024-09-20T22:11:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:40 smithi067 ceph-mon[33511]: 5.7 scrub ok 2024-09-20T22:11:41.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:40 smithi067 ceph-mon[33511]: 6.1 scrub starts 2024-09-20T22:11:41.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:40 smithi067 ceph-mon[33511]: 6.1 scrub ok 2024-09-20T22:11:41.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:40 smithi067 ceph-mon[33511]: 6.1c scrub starts 2024-09-20T22:11:41.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:40 smithi067 ceph-mon[33511]: 6.1c scrub ok 2024-09-20T22:11:41.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:40 smithi067 ceph-mon[33511]: 1.4 scrub starts 2024-09-20T22:11:41.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:40 smithi067 ceph-mon[33511]: 1.4 scrub ok 2024-09-20T22:11:41.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:40 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:11:41.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:40 smithi067 ceph-mon[33511]: osdmap e290: 8 total, 8 up, 8 in 2024-09-20T22:11:41.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:40 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3522218138' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:41.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:40 smithi067 ceph-mon[33511]: pgmap v470: 158 pgs: 158 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 917 B/s rd, 0 op/s; 103 KiB/s, 0 objects/s recovering 2024-09-20T22:11:41.037 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:41.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:41.038 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:41.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:41.165 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:41.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:41.166 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:41.267 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:41.290 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:41.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:41.291 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:41.391 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:41.413 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:41.414 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:41.414 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:41.515 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:41.537 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:41.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:41.538 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:41.639 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:41.662 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:41.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:41.663 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:41.764 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:41.793 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:41.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:41.794 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:41.810 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:41.810 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:41.814 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:41.815 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:41.895 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:41.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:41 smithi067 ceph-mon[33511]: 3.1c scrub starts 2024-09-20T22:11:41.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:41 smithi067 ceph-mon[33511]: 3.1c scrub ok 2024-09-20T22:11:41.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:41 smithi067 ceph-mon[33511]: 6.2 scrub starts 2024-09-20T22:11:41.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:41 smithi067 ceph-mon[33511]: 6.2 scrub ok 2024-09-20T22:11:41.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:41 smithi067 ceph-mon[33511]: 4.14 scrub starts 2024-09-20T22:11:41.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:41 smithi067 ceph-mon[33511]: 4.14 scrub ok 2024-09-20T22:11:41.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:41 smithi067 ceph-mon[33511]: osdmap e291: 8 total, 8 up, 8 in 2024-09-20T22:11:41.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:41 smithi067 ceph-mon[33511]: 5.1f deep-scrub starts 2024-09-20T22:11:41.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:41 smithi067 ceph-mon[33511]: 5.1f deep-scrub ok 2024-09-20T22:11:41.918 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:41.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:41.919 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:41.940 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:41 smithi060 ceph-mon[29694]: 3.1c scrub starts 2024-09-20T22:11:41.940 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:41 smithi060 ceph-mon[29694]: 3.1c scrub ok 2024-09-20T22:11:41.940 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:41 smithi060 ceph-mon[29694]: 6.2 scrub starts 2024-09-20T22:11:41.940 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:41 smithi060 ceph-mon[29694]: 6.2 scrub ok 2024-09-20T22:11:41.940 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:41 smithi060 ceph-mon[29694]: 4.14 scrub starts 2024-09-20T22:11:41.940 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:41 smithi060 ceph-mon[29694]: 4.14 scrub ok 2024-09-20T22:11:41.940 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:41 smithi060 ceph-mon[29694]: osdmap e291: 8 total, 8 up, 8 in 2024-09-20T22:11:41.940 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:41 smithi060 ceph-mon[29694]: 5.1f deep-scrub starts 2024-09-20T22:11:41.940 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:41 smithi060 ceph-mon[29694]: 5.1f deep-scrub ok 2024-09-20T22:11:41.940 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:41 smithi060 ceph-mon[40290]: 3.1c scrub starts 2024-09-20T22:11:41.940 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:41 smithi060 ceph-mon[40290]: 3.1c scrub ok 2024-09-20T22:11:41.941 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:41 smithi060 ceph-mon[40290]: 6.2 scrub starts 2024-09-20T22:11:41.941 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:41 smithi060 ceph-mon[40290]: 6.2 scrub ok 2024-09-20T22:11:41.941 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:41 smithi060 ceph-mon[40290]: 4.14 scrub starts 2024-09-20T22:11:41.941 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:41 smithi060 ceph-mon[40290]: 4.14 scrub ok 2024-09-20T22:11:41.941 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:41 smithi060 ceph-mon[40290]: osdmap e291: 8 total, 8 up, 8 in 2024-09-20T22:11:41.941 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:41 smithi060 ceph-mon[40290]: 5.1f deep-scrub starts 2024-09-20T22:11:41.941 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:41 smithi060 ceph-mon[40290]: 5.1f deep-scrub ok 2024-09-20T22:11:42.020 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:42.042 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:42.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:42.043 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:42.144 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:42.166 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:42.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:42.167 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:42.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:42.293 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:42.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:42.294 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:42.395 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:42.418 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:42.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:42.419 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:42.513 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:11:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:11:42.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:42.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:42.543 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:42.543 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:42.625 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:11:42.644 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:42.667 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:42.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:42.668 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:42.768 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:42.791 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:42.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:42.792 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:42.893 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:42.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:42.916 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:42.916 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:42 smithi067 ceph-mon[33511]: 4.12 deep-scrub starts 2024-09-20T22:11:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:42 smithi067 ceph-mon[33511]: 4.12 deep-scrub ok 2024-09-20T22:11:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:42 smithi067 ceph-mon[33511]: osdmap e292: 8 total, 8 up, 8 in 2024-09-20T22:11:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:42 smithi067 ceph-mon[33511]: pgmap v473: 158 pgs: 1 activating+undersized+remapped, 157 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 903 B/s rd, 0 op/s; 0 B/s, 1 objects/s recovering 2024-09-20T22:11:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:42 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:11:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:42 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2472224821' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:11:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:42 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:11:43.017 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:43.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:42 smithi060 ceph-mon[29694]: 4.12 deep-scrub starts 2024-09-20T22:11:43.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:42 smithi060 ceph-mon[29694]: 4.12 deep-scrub ok 2024-09-20T22:11:43.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:42 smithi060 ceph-mon[29694]: osdmap e292: 8 total, 8 up, 8 in 2024-09-20T22:11:43.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:42 smithi060 ceph-mon[29694]: pgmap v473: 158 pgs: 1 activating+undersized+remapped, 157 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 903 B/s rd, 0 op/s; 0 B/s, 1 objects/s recovering 2024-09-20T22:11:43.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:42 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:11:43.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:42 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2472224821' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:11:43.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:42 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:11:43.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:42 smithi060 ceph-mon[40290]: 4.12 deep-scrub starts 2024-09-20T22:11:43.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:42 smithi060 ceph-mon[40290]: 4.12 deep-scrub ok 2024-09-20T22:11:43.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:42 smithi060 ceph-mon[40290]: osdmap e292: 8 total, 8 up, 8 in 2024-09-20T22:11:43.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:42 smithi060 ceph-mon[40290]: pgmap v473: 158 pgs: 1 activating+undersized+remapped, 157 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 903 B/s rd, 0 op/s; 0 B/s, 1 objects/s recovering 2024-09-20T22:11:43.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:42 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:11:43.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:42 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2472224821' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:11:43.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:42 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:11:43.038 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:43.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:43.040 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:43.140 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:43.162 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:43.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:43.163 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:43.264 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:43.290 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:43.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:43.291 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:43.391 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:43.414 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:43.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:43.415 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:43.516 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:43.539 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:43.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:43.540 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:43.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:11:43.640 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:43.695 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:43.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:43.697 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:43.797 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:43.820 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:43.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:43.821 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:43.922 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:43.944 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:43.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:43.945 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:43 smithi067 ceph-mon[33511]: 3.5 scrub starts 2024-09-20T22:11:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:43 smithi067 ceph-mon[33511]: 3.5 scrub ok 2024-09-20T22:11:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:43 smithi067 ceph-mon[33511]: 3.1e scrub starts 2024-09-20T22:11:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:43 smithi067 ceph-mon[33511]: 3.1e scrub ok 2024-09-20T22:11:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:43 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:11:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:43 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:11:44.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:43 smithi067 ceph-mon[33511]: osdmap e293: 8 total, 8 up, 8 in 2024-09-20T22:11:44.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:43 smithi060 ceph-mon[29694]: 3.5 scrub starts 2024-09-20T22:11:44.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:43 smithi060 ceph-mon[29694]: 3.5 scrub ok 2024-09-20T22:11:44.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:43 smithi060 ceph-mon[29694]: 3.1e scrub starts 2024-09-20T22:11:44.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:43 smithi060 ceph-mon[29694]: 3.1e scrub ok 2024-09-20T22:11:44.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:43 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:11:44.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:43 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:11:44.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:43 smithi060 ceph-mon[29694]: osdmap e293: 8 total, 8 up, 8 in 2024-09-20T22:11:44.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:43 smithi060 ceph-mon[40290]: 3.5 scrub starts 2024-09-20T22:11:44.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:43 smithi060 ceph-mon[40290]: 3.5 scrub ok 2024-09-20T22:11:44.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:43 smithi060 ceph-mon[40290]: 3.1e scrub starts 2024-09-20T22:11:44.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:43 smithi060 ceph-mon[40290]: 3.1e scrub ok 2024-09-20T22:11:44.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:43 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:11:44.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:43 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:11:44.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:43 smithi060 ceph-mon[40290]: osdmap e293: 8 total, 8 up, 8 in 2024-09-20T22:11:44.046 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:44.069 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:44.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:44.070 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:44.171 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:44.196 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:44.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:44.197 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:44.297 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:44.321 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:11:44.324 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:11:44.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:44.325 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:44.326 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:44.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:44.449 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:44.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:44.450 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:44.551 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:44.576 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:44.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:44.577 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:44.678 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:44.702 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:44.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:44.703 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:44.804 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:44.829 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:44.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:44.830 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:44.863 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:44 smithi060 ceph-mon[40290]: 3.16 scrub starts 2024-09-20T22:11:44.863 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:44 smithi060 ceph-mon[40290]: osdmap e294: 8 total, 8 up, 8 in 2024-09-20T22:11:44.863 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:44 smithi060 ceph-mon[40290]: from='client.15579 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:11:44.864 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:44 smithi060 ceph-mon[40290]: pgmap v476: 158 pgs: 1 activating+undersized+remapped, 157 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 1 objects/s recovering 2024-09-20T22:11:44.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:44 smithi060 ceph-mon[29694]: 3.16 scrub starts 2024-09-20T22:11:44.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:44 smithi060 ceph-mon[29694]: osdmap e294: 8 total, 8 up, 8 in 2024-09-20T22:11:44.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:44 smithi060 ceph-mon[29694]: from='client.15579 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:11:44.864 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:44 smithi060 ceph-mon[29694]: pgmap v476: 158 pgs: 1 activating+undersized+remapped, 157 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 1 objects/s recovering 2024-09-20T22:11:44.930 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:44.955 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:44.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:44.956 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:44 smithi067 ceph-mon[33511]: 3.16 scrub starts 2024-09-20T22:11:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:44 smithi067 ceph-mon[33511]: osdmap e294: 8 total, 8 up, 8 in 2024-09-20T22:11:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:44 smithi067 ceph-mon[33511]: from='client.15579 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:11:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:44 smithi067 ceph-mon[33511]: pgmap v476: 158 pgs: 1 activating+undersized+remapped, 157 active+clean; 583 KiB data, 3.2 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 0 B/s, 1 objects/s recovering 2024-09-20T22:11:45.057 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:45.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:45.081 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:45.081 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:45.182 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:45.204 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:45.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:45.205 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:45.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:11:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:11:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:11:45.306 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:45.333 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:45.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:45.334 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:45.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:45.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:45.458 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:45.458 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:45.559 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:45.562 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":475,"stamp":"2024-09-20T22:11:43.601939+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":1226,"num_read_kb":978,"num_write":436,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":921003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1943,"ondisk_log_size":1943,"up":474,"acting":473,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":473,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":3310796,"kb_used_data":54380,"kb_used_omap":15,"kb_used_meta":3256240,"kb_avail":746650420,"statfs":{"total":767960285184,"available":764570030080,"internally_reserved":0,"allocated":55685120,"data_stored":50824795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":3334390547},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.021940"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":506,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.593979+0000","last_change":"2024-09-20T22:11:12.498436+0000","last_active":"2024-09-20T22:11:41.593979+0000","last_peered":"2024-09-20T22:11:41.593979+0000","last_clean":"2024-09-20T22:11:41.593979+0000","last_became_active":"2024-09-20T22:11:12.498141+0000","last_became_peered":"2024-09-20T22:11:12.498141+0000","last_unstale":"2024-09-20T22:11:41.593979+0000","last_undegraded":"2024-09-20T22:11:41.593979+0000","last_fullsized":"2024-09-20T22:11:41.593979+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:31.859259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_clean_scrub_stamp":"2024-09-20T22:10:31.859259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"252'13","reported_seq":517,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.591511+0000","last_change":"2024-09-20T22:10:31.568218+0000","last_active":"2024-09-20T22:11:40.591511+0000","last_peered":"2024-09-20T22:11:40.591511+0000","last_clean":"2024-09-20T22:11:40.591511+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:11:40.591511+0000","last_undegraded":"2024-09-20T22:11:40.591511+0000","last_fullsized":"2024-09-20T22:11:40.591511+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"252'13","last_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_deep_scrub":"252'13","last_deep_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_clean_scrub_stamp":"2024-09-20T22:10:31.568098+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":376,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-09-20T22:11:42.603202+0000","last_change":"2024-09-20T22:10:22.311182+0000","last_active":"2024-09-20T22:11:42.603202+0000","last_peered":"2024-09-20T22:11:42.603202+0000","last_clean":"2024-09-20T22:11:42.603202+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:11:42.603202+0000","last_undegraded":"2024-09-20T22:11:42.603202+0000","last_fullsized":"2024-09-20T22:11:42.603202+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:22.311078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:22.311078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:51.019290+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":493,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.591857+0000","last_change":"2024-09-20T22:11:30.486243+0000","last_active":"2024-09-20T22:11:40.591857+0000","last_peered":"2024-09-20T22:11:40.591857+0000","last_clean":"2024-09-20T22:11:40.591857+0000","last_became_active":"2024-09-20T22:11:30.485735+0000","last_became_peered":"2024-09-20T22:11:30.485735+0000","last_unstale":"2024-09-20T22:11:40.591857+0000","last_undegraded":"2024-09-20T22:11:40.591857+0000","last_fullsized":"2024-09-20T22:11:40.591857+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:23.640516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:23.640516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":472,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597450+0000","last_change":"2024-09-20T22:08:41.770593+0000","last_active":"2024-09-20T22:11:40.597450+0000","last_peered":"2024-09-20T22:11:40.597450+0000","last_clean":"2024-09-20T22:11:40.597450+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:11:40.597450+0000","last_undegraded":"2024-09-20T22:11:40.597450+0000","last_fullsized":"2024-09-20T22:11:40.597450+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:08:41.770351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:41.770351+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":484,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597786+0000","last_change":"2024-09-20T22:09:00.839346+0000","last_active":"2024-09-20T22:11:40.597786+0000","last_peered":"2024-09-20T22:11:40.597786+0000","last_clean":"2024-09-20T22:11:40.597786+0000","last_became_active":"2024-09-20T22:02:50.107154+0000","last_became_peered":"2024-09-20T22:02:50.107154+0000","last_unstale":"2024-09-20T22:11:40.597786+0000","last_undegraded":"2024-09-20T22:11:40.597786+0000","last_fullsized":"2024-09-20T22:11:40.597786+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:00.839239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:00.839239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"71'15","reported_seq":504,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.598142+0000","last_change":"2024-09-20T22:08:39.705641+0000","last_active":"2024-09-20T22:11:40.598142+0000","last_peered":"2024-09-20T22:11:40.598142+0000","last_clean":"2024-09-20T22:11:40.598142+0000","last_became_active":"2024-09-20T22:02:52.282101+0000","last_became_peered":"2024-09-20T22:02:52.282101+0000","last_unstale":"2024-09-20T22:11:40.598142+0000","last_undegraded":"2024-09-20T22:11:40.598142+0000","last_fullsized":"2024-09-20T22:11:40.598142+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:08:39.705510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:39.705510+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":485,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.591510+0000","last_change":"2024-09-20T22:10:26.558035+0000","last_active":"2024-09-20T22:11:40.591510+0000","last_peered":"2024-09-20T22:11:40.591510+0000","last_clean":"2024-09-20T22:11:40.591510+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:11:40.591510+0000","last_undegraded":"2024-09-20T22:11:40.591510+0000","last_fullsized":"2024-09-20T22:11:40.591510+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:26.557925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:26.557925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":507,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.596592+0000","last_change":"2024-09-20T22:10:35.191206+0000","last_active":"2024-09-20T22:11:40.596592+0000","last_peered":"2024-09-20T22:11:40.596592+0000","last_clean":"2024-09-20T22:11:40.596592+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:11:40.596592+0000","last_undegraded":"2024-09-20T22:11:40.596592+0000","last_fullsized":"2024-09-20T22:11:40.596592+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:35.191108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:10:35.191108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:46.313768+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"71'9","reported_seq":504,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.593072+0000","last_change":"2024-09-20T22:09:47.733504+0000","last_active":"2024-09-20T22:11:40.593072+0000","last_peered":"2024-09-20T22:11:40.593072+0000","last_clean":"2024-09-20T22:11:40.593072+0000","last_became_active":"2024-09-20T22:02:52.279526+0000","last_became_peered":"2024-09-20T22:02:52.279526+0000","last_unstale":"2024-09-20T22:11:40.593072+0000","last_undegraded":"2024-09-20T22:11:40.593072+0000","last_fullsized":"2024-09-20T22:11:40.593072+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:09:47.733368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:09:47.733368+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":35,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.594475+0000","last_change":"2024-09-20T22:11:31.101654+0000","last_active":"2024-09-20T22:11:40.594475+0000","last_peered":"2024-09-20T22:11:40.594475+0000","last_clean":"2024-09-20T22:11:40.594475+0000","last_became_active":"2024-09-20T22:11:30.485036+0000","last_became_peered":"2024-09-20T22:11:30.485036+0000","last_unstale":"2024-09-20T22:11:40.594475+0000","last_undegraded":"2024-09-20T22:11:40.594475+0000","last_fullsized":"2024-09-20T22:11:40.594475+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:31.101573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:31.101573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:32.848267+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":482,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.592302+0000","last_change":"2024-09-20T22:10:32.855540+0000","last_active":"2024-09-20T22:11:41.592302+0000","last_peered":"2024-09-20T22:11:41.592302+0000","last_clean":"2024-09-20T22:11:41.592302+0000","last_became_active":"2024-09-20T22:06:54.841095+0000","last_became_peered":"2024-09-20T22:06:54.841095+0000","last_unstale":"2024-09-20T22:11:41.592302+0000","last_undegraded":"2024-09-20T22:11:41.592302+0000","last_fullsized":"2024-09-20T22:11:41.592302+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:32.855440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:32.855440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":520,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.594976+0000","last_change":"2024-09-20T22:11:40.127688+0000","last_active":"2024-09-20T22:11:40.594976+0000","last_peered":"2024-09-20T22:11:40.594976+0000","last_clean":"2024-09-20T22:11:40.594976+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:11:40.594976+0000","last_undegraded":"2024-09-20T22:11:40.594976+0000","last_fullsized":"2024-09-20T22:11:40.594976+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:40.127597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:30.215263+0000","last_clean_scrub_stamp":"2024-09-20T22:11:40.127597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:59.820229+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"71'5","reported_seq":519,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.591931+0000","last_change":"2024-09-20T22:11:03.163814+0000","last_active":"2024-09-20T22:11:40.591931+0000","last_peered":"2024-09-20T22:11:40.591931+0000","last_clean":"2024-09-20T22:11:40.591931+0000","last_became_active":"2024-09-20T22:08:22.144364+0000","last_became_peered":"2024-09-20T22:08:22.144364+0000","last_unstale":"2024-09-20T22:11:40.591931+0000","last_undegraded":"2024-09-20T22:11:40.591931+0000","last_fullsized":"2024-09-20T22:11:40.591931+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'5","last_scrub_stamp":"2024-09-20T22:11:03.163739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:03.163739+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:25.185084+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":493,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.598848+0000","last_change":"2024-09-20T22:10:15.024751+0000","last_active":"2024-09-20T22:11:40.598848+0000","last_peered":"2024-09-20T22:11:40.598848+0000","last_clean":"2024-09-20T22:11:40.598848+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:11:40.598848+0000","last_undegraded":"2024-09-20T22:11:40.598848+0000","last_fullsized":"2024-09-20T22:11:40.598848+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:15.024624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:15.024624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":35,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.594289+0000","last_change":"2024-09-20T22:11:34.036027+0000","last_active":"2024-09-20T22:11:40.594289+0000","last_peered":"2024-09-20T22:11:40.594289+0000","last_clean":"2024-09-20T22:11:40.594289+0000","last_became_active":"2024-09-20T22:11:30.487137+0000","last_became_peered":"2024-09-20T22:11:30.487137+0000","last_unstale":"2024-09-20T22:11:40.594289+0000","last_undegraded":"2024-09-20T22:11:40.594289+0000","last_fullsized":"2024-09-20T22:11:40.594289+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:34.035938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.035938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:57.022848+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":481,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.592257+0000","last_change":"2024-09-20T22:09:51.759271+0000","last_active":"2024-09-20T22:11:40.592257+0000","last_peered":"2024-09-20T22:11:40.592257+0000","last_clean":"2024-09-20T22:11:40.592257+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:11:40.592257+0000","last_undegraded":"2024-09-20T22:11:40.592257+0000","last_fullsized":"2024-09-20T22:11:40.592257+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:51.759042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:09:51.759042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":493,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.594471+0000","last_change":"2024-09-20T22:11:36.700618+0000","last_active":"2024-09-20T22:11:41.594471+0000","last_peered":"2024-09-20T22:11:41.594471+0000","last_clean":"2024-09-20T22:11:41.594471+0000","last_became_active":"2024-09-20T22:02:50.111517+0000","last_became_peered":"2024-09-20T22:02:50.111517+0000","last_unstale":"2024-09-20T22:11:41.594471+0000","last_undegraded":"2024-09-20T22:11:41.594471+0000","last_fullsized":"2024-09-20T22:11:41.594471+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:36.700507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:11:36.700507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:44.350271+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"195'22","reported_seq":57,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.591265+0000","last_change":"2024-09-20T22:11:12.545971+0000","last_active":"2024-09-20T22:11:40.591265+0000","last_peered":"2024-09-20T22:11:40.591265+0000","last_clean":"2024-09-20T22:11:40.591265+0000","last_became_active":"2024-09-20T22:11:12.507311+0000","last_became_peered":"2024-09-20T22:11:12.507311+0000","last_unstale":"2024-09-20T22:11:40.591265+0000","last_undegraded":"2024-09-20T22:11:40.591265+0000","last_fullsized":"2024-09-20T22:11:40.591265+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'22","last_scrub_stamp":"2024-09-20T22:10:16.538107+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:10:16.538107+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:42.147249+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":333,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-09-20T22:11:42.606266+0000","last_change":"2024-09-20T22:11:09.157162+0000","last_active":"2024-09-20T22:11:42.606266+0000","last_peered":"2024-09-20T22:11:42.606266+0000","last_clean":"2024-09-20T22:11:42.606266+0000","last_became_active":"2024-09-20T22:06:54.848505+0000","last_became_peered":"2024-09-20T22:06:54.848505+0000","last_unstale":"2024-09-20T22:11:42.606266+0000","last_undegraded":"2024-09-20T22:11:42.606266+0000","last_fullsized":"2024-09-20T22:11:42.606266+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:09.157055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:09.157055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:24.968380+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":492,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597243+0000","last_change":"2024-09-20T22:11:30.487554+0000","last_active":"2024-09-20T22:11:40.597243+0000","last_peered":"2024-09-20T22:11:40.597243+0000","last_clean":"2024-09-20T22:11:40.597243+0000","last_became_active":"2024-09-20T22:11:30.487199+0000","last_became_peered":"2024-09-20T22:11:30.487199+0000","last_unstale":"2024-09-20T22:11:40.597243+0000","last_undegraded":"2024-09-20T22:11:40.597243+0000","last_fullsized":"2024-09-20T22:11:40.597243+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:52.317931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:52.317931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":441,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.595891+0000","last_change":"2024-09-20T22:10:59.182372+0000","last_active":"2024-09-20T22:11:40.595891+0000","last_peered":"2024-09-20T22:11:40.595891+0000","last_clean":"2024-09-20T22:11:40.595891+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:11:40.595891+0000","last_undegraded":"2024-09-20T22:11:40.595891+0000","last_fullsized":"2024-09-20T22:11:40.595891+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:59.182274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:59.182274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:07.198779+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"250'17","reported_seq":516,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597055+0000","last_change":"2024-09-20T22:10:27.157652+0000","last_active":"2024-09-20T22:11:40.597055+0000","last_peered":"2024-09-20T22:11:40.597055+0000","last_clean":"2024-09-20T22:11:40.597055+0000","last_became_active":"2024-09-20T22:06:54.830946+0000","last_became_peered":"2024-09-20T22:06:54.830946+0000","last_unstale":"2024-09-20T22:11:40.597055+0000","last_undegraded":"2024-09-20T22:11:40.597055+0000","last_fullsized":"2024-09-20T22:11:40.597055+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"250'17","last_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_deep_scrub":"250'17","last_deep_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_clean_scrub_stamp":"2024-09-20T22:10:27.157501+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:45.358047+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":484,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.592480+0000","last_change":"2024-09-20T22:11:34.369112+0000","last_active":"2024-09-20T22:11:40.592480+0000","last_peered":"2024-09-20T22:11:40.592480+0000","last_clean":"2024-09-20T22:11:40.592480+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:11:40.592480+0000","last_undegraded":"2024-09-20T22:11:40.592480+0000","last_fullsized":"2024-09-20T22:11:40.592480+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:34.369006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.369006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:49.101366+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":482,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.600081+0000","last_change":"2024-09-20T22:11:39.286967+0000","last_active":"2024-09-20T22:11:40.600081+0000","last_peered":"2024-09-20T22:11:40.600081+0000","last_clean":"2024-09-20T22:11:40.600081+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:11:40.600081+0000","last_undegraded":"2024-09-20T22:11:40.600081+0000","last_fullsized":"2024-09-20T22:11:40.600081+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:11:39.286813+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:11:39.286813+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:43.641876+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"289'2","reported_seq":336,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-09-20T22:11:42.603295+0000","last_change":"2024-09-20T22:11:37.189066+0000","last_active":"2024-09-20T22:11:42.603295+0000","last_peered":"2024-09-20T22:11:42.603295+0000","last_clean":"2024-09-20T22:11:42.603295+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:11:42.603295+0000","last_undegraded":"2024-09-20T22:11:42.603295+0000","last_fullsized":"2024-09-20T22:11:42.603295+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_clean_scrub_stamp":"2024-09-20T22:11:37.188992+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:55.054428+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"71'10","reported_seq":540,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.594728+0000","last_change":"2024-09-20T22:11:40.608961+0000","last_active":"2024-09-20T22:11:41.594728+0000","last_peered":"2024-09-20T22:11:41.594728+0000","last_clean":"2024-09-20T22:11:41.594728+0000","last_became_active":"2024-09-20T22:11:40.602075+0000","last_became_peered":"2024-09-20T22:11:40.602075+0000","last_unstale":"2024-09-20T22:11:41.594728+0000","last_undegraded":"2024-09-20T22:11:41.594728+0000","last_fullsized":"2024-09-20T22:11:41.594728+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:11:35.700046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:35.700046+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:03.129277+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"291'10","reported_seq":630,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.605010+0000","last_change":"2024-09-20T22:11:40.604549+0000","last_active":"2024-09-20T22:11:40.605010+0000","last_peered":"2024-09-20T22:11:40.605010+0000","last_clean":"2024-09-20T22:11:40.605010+0000","last_became_active":"2024-09-20T22:11:40.601048+0000","last_became_peered":"2024-09-20T22:11:40.601048+0000","last_unstale":"2024-09-20T22:11:40.605010+0000","last_undegraded":"2024-09-20T22:11:40.605010+0000","last_fullsized":"2024-09-20T22:11:40.605010+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"208'9","last_scrub_stamp":"2024-09-20T22:10:29.287636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:29.287636+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":19,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597853+0000","last_change":"2024-09-20T22:11:35.304533+0000","last_active":"2024-09-20T22:11:40.597853+0000","last_peered":"2024-09-20T22:11:40.597853+0000","last_clean":"2024-09-20T22:11:40.597853+0000","last_became_active":"2024-09-20T22:11:35.304245+0000","last_became_peered":"2024-09-20T22:11:35.304245+0000","last_unstale":"2024-09-20T22:11:40.597853+0000","last_undegraded":"2024-09-20T22:11:40.597853+0000","last_fullsized":"2024-09-20T22:11:40.597853+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:57.958347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":495,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.595043+0000","last_change":"2024-09-20T22:11:08.187529+0000","last_active":"2024-09-20T22:11:40.595043+0000","last_peered":"2024-09-20T22:11:40.595043+0000","last_clean":"2024-09-20T22:11:40.595043+0000","last_became_active":"2024-09-20T22:08:48.452786+0000","last_became_peered":"2024-09-20T22:08:48.452786+0000","last_unstale":"2024-09-20T22:11:40.595043+0000","last_undegraded":"2024-09-20T22:11:40.595043+0000","last_fullsized":"2024-09-20T22:11:40.595043+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:11:08.187420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:11:08.187420+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:21.119364+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"71'15","reported_seq":258,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.592069+0000","last_change":"2024-09-20T22:11:34.283966+0000","last_active":"2024-09-20T22:11:40.592069+0000","last_peered":"2024-09-20T22:11:40.592069+0000","last_clean":"2024-09-20T22:11:40.592069+0000","last_became_active":"2024-09-20T22:08:22.145408+0000","last_became_peered":"2024-09-20T22:08:22.145408+0000","last_unstale":"2024-09-20T22:11:40.592069+0000","last_undegraded":"2024-09-20T22:11:40.592069+0000","last_fullsized":"2024-09-20T22:11:40.592069+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:11:34.283882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.283882+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:43.345533+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":493,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.596931+0000","last_change":"2024-09-20T22:10:19.208598+0000","last_active":"2024-09-20T22:11:40.596931+0000","last_peered":"2024-09-20T22:11:40.596931+0000","last_clean":"2024-09-20T22:11:40.596931+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:11:40.596931+0000","last_undegraded":"2024-09-20T22:11:40.596931+0000","last_fullsized":"2024-09-20T22:11:40.596931+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:19.208496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:19.208496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"283'11","reported_seq":640,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.592403+0000","last_change":"2024-09-20T22:11:30.493908+0000","last_active":"2024-09-20T22:11:40.592403+0000","last_peered":"2024-09-20T22:11:40.592403+0000","last_clean":"2024-09-20T22:11:40.592403+0000","last_became_active":"2024-09-20T22:11:30.484743+0000","last_became_peered":"2024-09-20T22:11:30.484743+0000","last_unstale":"2024-09-20T22:11:40.592403+0000","last_undegraded":"2024-09-20T22:11:40.592403+0000","last_fullsized":"2024-09-20T22:11:40.592403+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:10:14.680845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:14.680845+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":27,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.594738+0000","last_change":"2024-09-20T22:11:30.490404+0000","last_active":"2024-09-20T22:11:40.594738+0000","last_peered":"2024-09-20T22:11:40.594738+0000","last_clean":"2024-09-20T22:11:40.594738+0000","last_became_active":"2024-09-20T22:11:30.489416+0000","last_became_peered":"2024-09-20T22:11:30.489416+0000","last_unstale":"2024-09-20T22:11:40.594738+0000","last_undegraded":"2024-09-20T22:11:40.594738+0000","last_fullsized":"2024-09-20T22:11:40.594738+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:19.235368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:19.235368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:42.332031+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":19,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.599501+0000","last_change":"2024-09-20T22:11:34.561887+0000","last_active":"2024-09-20T22:11:40.599501+0000","last_peered":"2024-09-20T22:11:40.599501+0000","last_clean":"2024-09-20T22:11:40.599501+0000","last_became_active":"2024-09-20T22:11:34.561538+0000","last_became_peered":"2024-09-20T22:11:34.561538+0000","last_unstale":"2024-09-20T22:11:40.599501+0000","last_undegraded":"2024-09-20T22:11:40.599501+0000","last_fullsized":"2024-09-20T22:11:40.599501+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:56.980684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":487,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.593653+0000","last_change":"2024-09-20T22:08:59.563006+0000","last_active":"2024-09-20T22:11:40.593653+0000","last_peered":"2024-09-20T22:11:40.593653+0000","last_clean":"2024-09-20T22:11:40.593653+0000","last_became_active":"2024-09-20T22:08:48.452475+0000","last_became_peered":"2024-09-20T22:08:48.452475+0000","last_unstale":"2024-09-20T22:11:40.593653+0000","last_undegraded":"2024-09-20T22:11:40.593653+0000","last_fullsized":"2024-09-20T22:11:40.593653+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:08:59.562897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:08:59.562897+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"71'18","reported_seq":567,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.596759+0000","last_change":"2024-09-20T22:11:30.218845+0000","last_active":"2024-09-20T22:11:40.596759+0000","last_peered":"2024-09-20T22:11:40.596759+0000","last_clean":"2024-09-20T22:11:40.596759+0000","last_became_active":"2024-09-20T22:10:48.921997+0000","last_became_peered":"2024-09-20T22:10:48.921997+0000","last_unstale":"2024-09-20T22:11:40.596759+0000","last_undegraded":"2024-09-20T22:11:40.596759+0000","last_fullsized":"2024-09-20T22:11:40.596759+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'18","last_scrub_stamp":"2024-09-20T22:11:30.218749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:30.218749+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:45.626566+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":483,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597517+0000","last_change":"2024-09-20T22:09:01.889907+0000","last_active":"2024-09-20T22:11:40.597517+0000","last_peered":"2024-09-20T22:11:40.597517+0000","last_clean":"2024-09-20T22:11:40.597517+0000","last_became_active":"2024-09-20T22:02:50.113076+0000","last_became_peered":"2024-09-20T22:02:50.113076+0000","last_unstale":"2024-09-20T22:11:40.597517+0000","last_undegraded":"2024-09-20T22:11:40.597517+0000","last_fullsized":"2024-09-20T22:11:40.597517+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:01.889795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:01.889795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":497,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.591610+0000","last_change":"2024-09-20T22:10:48.920310+0000","last_active":"2024-09-20T22:11:40.591610+0000","last_peered":"2024-09-20T22:11:40.591610+0000","last_clean":"2024-09-20T22:11:40.591610+0000","last_became_active":"2024-09-20T22:10:48.919949+0000","last_became_peered":"2024-09-20T22:10:48.919949+0000","last_unstale":"2024-09-20T22:11:40.591610+0000","last_undegraded":"2024-09-20T22:11:40.591610+0000","last_fullsized":"2024-09-20T22:11:40.591610+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:05.535412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:05.535412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":489,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.592167+0000","last_change":"2024-09-20T22:11:40.260396+0000","last_active":"2024-09-20T22:11:40.592167+0000","last_peered":"2024-09-20T22:11:40.592167+0000","last_clean":"2024-09-20T22:11:40.592167+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:11:40.592167+0000","last_undegraded":"2024-09-20T22:11:40.592167+0000","last_fullsized":"2024-09-20T22:11:40.592167+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:40.260289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:40.260289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:49.182120+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":19,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597559+0000","last_change":"2024-09-20T22:11:34.559451+0000","last_active":"2024-09-20T22:11:40.597559+0000","last_peered":"2024-09-20T22:11:40.597559+0000","last_clean":"2024-09-20T22:11:40.597559+0000","last_became_active":"2024-09-20T22:11:34.558615+0000","last_became_peered":"2024-09-20T22:11:34.558615+0000","last_unstale":"2024-09-20T22:11:40.597559+0000","last_undegraded":"2024-09-20T22:11:40.597559+0000","last_fullsized":"2024-09-20T22:11:40.597559+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:39.452330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":521,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597685+0000","last_change":"2024-09-20T22:11:39.422439+0000","last_active":"2024-09-20T22:11:40.597685+0000","last_peered":"2024-09-20T22:11:40.597685+0000","last_clean":"2024-09-20T22:11:40.597685+0000","last_became_active":"2024-09-20T22:11:33.560632+0000","last_became_peered":"2024-09-20T22:11:33.560632+0000","last_unstale":"2024-09-20T22:11:40.597685+0000","last_undegraded":"2024-09-20T22:11:40.597685+0000","last_fullsized":"2024-09-20T22:11:40.597685+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:11:39.422355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:11:39.422355+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:08.591531+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"71'14","reported_seq":510,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.591774+0000","last_change":"2024-09-20T22:09:03.569008+0000","last_active":"2024-09-20T22:11:40.591774+0000","last_peered":"2024-09-20T22:11:40.591774+0000","last_clean":"2024-09-20T22:11:40.591774+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:11:40.591774+0000","last_undegraded":"2024-09-20T22:11:40.591774+0000","last_fullsized":"2024-09-20T22:11:40.591774+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'14","last_scrub_stamp":"2024-09-20T22:09:03.568900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:09:03.568900+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":496,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.594851+0000","last_change":"2024-09-20T22:11:04.592252+0000","last_active":"2024-09-20T22:11:41.594851+0000","last_peered":"2024-09-20T22:11:41.594851+0000","last_clean":"2024-09-20T22:11:41.594851+0000","last_became_active":"2024-09-20T22:06:54.841165+0000","last_became_peered":"2024-09-20T22:06:54.841165+0000","last_unstale":"2024-09-20T22:11:41.594851+0000","last_undegraded":"2024-09-20T22:11:41.594851+0000","last_fullsized":"2024-09-20T22:11:41.594851+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:11:04.590441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:11:04.590441+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:16.119865+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"146'10","reported_seq":602,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.596619+0000","last_change":"2024-09-20T22:10:23.115374+0000","last_active":"2024-09-20T22:11:40.596619+0000","last_peered":"2024-09-20T22:11:40.596619+0000","last_clean":"2024-09-20T22:11:40.596619+0000","last_became_active":"2024-09-20T22:06:54.847742+0000","last_became_peered":"2024-09-20T22:06:54.847742+0000","last_unstale":"2024-09-20T22:11:40.596619+0000","last_undegraded":"2024-09-20T22:11:40.596619+0000","last_fullsized":"2024-09-20T22:11:40.596619+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:10:23.114986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:23.114986+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:50.179566+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":481,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.599604+0000","last_change":"2024-09-20T22:10:22.075441+0000","last_active":"2024-09-20T22:11:40.599604+0000","last_peered":"2024-09-20T22:11:40.599604+0000","last_clean":"2024-09-20T22:11:40.599604+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:11:40.599604+0000","last_undegraded":"2024-09-20T22:11:40.599604+0000","last_fullsized":"2024-09-20T22:11:40.599604+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:22.075335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:22.075335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:45.954793+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":19,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.594564+0000","last_change":"2024-09-20T22:11:34.559393+0000","last_active":"2024-09-20T22:11:40.594564+0000","last_peered":"2024-09-20T22:11:40.594564+0000","last_clean":"2024-09-20T22:11:40.594564+0000","last_became_active":"2024-09-20T22:11:34.558898+0000","last_became_peered":"2024-09-20T22:11:34.558898+0000","last_unstale":"2024-09-20T22:11:40.594564+0000","last_undegraded":"2024-09-20T22:11:40.594564+0000","last_fullsized":"2024-09-20T22:11:40.594564+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:40.804978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":517,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.596456+0000","last_change":"2024-09-20T22:11:07.362346+0000","last_active":"2024-09-20T22:11:40.596456+0000","last_peered":"2024-09-20T22:11:40.596456+0000","last_clean":"2024-09-20T22:11:40.596456+0000","last_became_active":"2024-09-20T22:10:33.660948+0000","last_became_peered":"2024-09-20T22:10:33.660948+0000","last_unstale":"2024-09-20T22:11:40.596456+0000","last_undegraded":"2024-09-20T22:11:40.596456+0000","last_fullsized":"2024-09-20T22:11:40.596456+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:11:07.362277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:11:07.362277+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:23.879195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"71'10","reported_seq":396,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-09-20T22:11:42.604827+0000","last_change":"2024-09-20T22:11:31.064801+0000","last_active":"2024-09-20T22:11:42.604827+0000","last_peered":"2024-09-20T22:11:42.604827+0000","last_clean":"2024-09-20T22:11:42.604827+0000","last_became_active":"2024-09-20T22:11:30.486532+0000","last_became_peered":"2024-09-20T22:11:30.486532+0000","last_unstale":"2024-09-20T22:11:42.604827+0000","last_undegraded":"2024-09-20T22:11:42.604827+0000","last_fullsized":"2024-09-20T22:11:42.604827+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:11:31.064669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:31.064669+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:35.046612+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":508,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.596700+0000","last_change":"2024-09-20T22:10:33.228016+0000","last_active":"2024-09-20T22:11:40.596700+0000","last_peered":"2024-09-20T22:11:40.596700+0000","last_clean":"2024-09-20T22:11:40.596700+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:11:40.596700+0000","last_undegraded":"2024-09-20T22:11:40.596700+0000","last_fullsized":"2024-09-20T22:11:40.596700+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:33.227911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:10:33.227911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"283'10","reported_seq":613,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.593870+0000","last_change":"2024-09-20T22:11:30.496915+0000","last_active":"2024-09-20T22:11:41.593870+0000","last_peered":"2024-09-20T22:11:41.593870+0000","last_clean":"2024-09-20T22:11:41.593870+0000","last_became_active":"2024-09-20T22:11:30.486567+0000","last_became_peered":"2024-09-20T22:11:30.486567+0000","last_unstale":"2024-09-20T22:11:41.593870+0000","last_undegraded":"2024-09-20T22:11:41.593870+0000","last_fullsized":"2024-09-20T22:11:41.593870+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:08:28.860982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:28.860982+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":474,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.592440+0000","last_change":"2024-09-20T22:10:24.053258+0000","last_active":"2024-09-20T22:11:41.592440+0000","last_peered":"2024-09-20T22:11:41.592440+0000","last_clean":"2024-09-20T22:11:41.592440+0000","last_became_active":"2024-09-20T22:02:56.304274+0000","last_became_peered":"2024-09-20T22:02:56.304274+0000","last_unstale":"2024-09-20T22:11:41.592440+0000","last_undegraded":"2024-09-20T22:11:41.592440+0000","last_fullsized":"2024-09-20T22:11:41.592440+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:24.053159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:24.053159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":19,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.599282+0000","last_change":"2024-09-20T22:11:34.560812+0000","last_active":"2024-09-20T22:11:40.599282+0000","last_peered":"2024-09-20T22:11:40.599282+0000","last_clean":"2024-09-20T22:11:40.599282+0000","last_became_active":"2024-09-20T22:11:34.560514+0000","last_became_peered":"2024-09-20T22:11:34.560514+0000","last_unstale":"2024-09-20T22:11:40.599282+0000","last_undegraded":"2024-09-20T22:11:40.599282+0000","last_fullsized":"2024-09-20T22:11:40.599282+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:36.465015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":500,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.595286+0000","last_change":"2024-09-20T22:11:36.339782+0000","last_active":"2024-09-20T22:11:40.595286+0000","last_peered":"2024-09-20T22:11:40.595286+0000","last_clean":"2024-09-20T22:11:40.595286+0000","last_became_active":"2024-09-20T22:08:48.466328+0000","last_became_peered":"2024-09-20T22:08:48.466328+0000","last_unstale":"2024-09-20T22:11:40.595286+0000","last_undegraded":"2024-09-20T22:11:40.595286+0000","last_fullsized":"2024-09-20T22:11:40.595286+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:11:36.339687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:11:36.339687+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:56.138458+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"71'19","reported_seq":526,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.594615+0000","last_change":"2024-09-20T22:08:33.852966+0000","last_active":"2024-09-20T22:11:41.594615+0000","last_peered":"2024-09-20T22:11:41.594615+0000","last_clean":"2024-09-20T22:11:41.594615+0000","last_became_active":"2024-09-20T22:02:52.480077+0000","last_became_peered":"2024-09-20T22:02:52.480077+0000","last_unstale":"2024-09-20T22:11:41.594615+0000","last_undegraded":"2024-09-20T22:11:41.594615+0000","last_fullsized":"2024-09-20T22:11:41.594615+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:08:33.852806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:33.852806+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":347,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-09-20T22:11:42.605453+0000","last_change":"2024-09-20T22:11:30.486764+0000","last_active":"2024-09-20T22:11:42.605453+0000","last_peered":"2024-09-20T22:11:42.605453+0000","last_clean":"2024-09-20T22:11:42.605453+0000","last_became_active":"2024-09-20T22:11:30.486428+0000","last_became_peered":"2024-09-20T22:11:30.486428+0000","last_unstale":"2024-09-20T22:11:42.605453+0000","last_undegraded":"2024-09-20T22:11:42.605453+0000","last_fullsized":"2024-09-20T22:11:42.605453+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:59.221160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:59.221160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:17.826923+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":500,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.596635+0000","last_change":"2024-09-20T22:11:00.280237+0000","last_active":"2024-09-20T22:11:40.596635+0000","last_peered":"2024-09-20T22:11:40.596635+0000","last_clean":"2024-09-20T22:11:40.596635+0000","last_became_active":"2024-09-20T22:02:54.290787+0000","last_became_peered":"2024-09-20T22:02:54.290787+0000","last_unstale":"2024-09-20T22:11:40.596635+0000","last_undegraded":"2024-09-20T22:11:40.596635+0000","last_fullsized":"2024-09-20T22:11:40.596635+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:00.280165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:11:00.280165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:08.873300+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":333,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-09-20T22:11:42.603016+0000","last_change":"2024-09-20T22:11:39.264738+0000","last_active":"2024-09-20T22:11:42.603016+0000","last_peered":"2024-09-20T22:11:42.603016+0000","last_clean":"2024-09-20T22:11:42.603016+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:11:42.603016+0000","last_undegraded":"2024-09-20T22:11:42.603016+0000","last_fullsized":"2024-09-20T22:11:42.603016+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:39.264610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:39.264610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:55.038507+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":21,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.592863+0000","last_change":"2024-09-20T22:11:34.561687+0000","last_active":"2024-09-20T22:11:41.592863+0000","last_peered":"2024-09-20T22:11:41.592863+0000","last_clean":"2024-09-20T22:11:41.592863+0000","last_became_active":"2024-09-20T22:11:34.560531+0000","last_became_peered":"2024-09-20T22:11:34.560531+0000","last_unstale":"2024-09-20T22:11:41.592863+0000","last_undegraded":"2024-09-20T22:11:41.592863+0000","last_fullsized":"2024-09-20T22:11:41.592863+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:59.894258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":495,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.594212+0000","last_change":"2024-09-20T22:10:56.231802+0000","last_active":"2024-09-20T22:11:40.594212+0000","last_peered":"2024-09-20T22:11:40.594212+0000","last_clean":"2024-09-20T22:11:40.594212+0000","last_became_active":"2024-09-20T22:08:48.452234+0000","last_became_peered":"2024-09-20T22:08:48.452234+0000","last_unstale":"2024-09-20T22:11:40.594212+0000","last_undegraded":"2024-09-20T22:11:40.594212+0000","last_fullsized":"2024-09-20T22:11:40.594212+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:10:56.231679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:10:56.231679+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:02.751759+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"271'40","reported_seq":464,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-09-20T22:11:42.605743+0000","last_change":"2024-09-20T22:11:05.235955+0000","last_active":"2024-09-20T22:11:42.605743+0000","last_peered":"2024-09-20T22:11:42.605743+0000","last_clean":"2024-09-20T22:11:42.605743+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:11:42.605743+0000","last_undegraded":"2024-09-20T22:11:42.605743+0000","last_fullsized":"2024-09-20T22:11:42.605743+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"271'40","last_scrub_stamp":"2024-09-20T22:11:05.235779+0000","last_deep_scrub":"271'40","last_deep_scrub_stamp":"2024-09-20T22:11:05.235779+0000","last_clean_scrub_stamp":"2024-09-20T22:11:05.235779+0000","objects_scrubbed":10,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:31.478891+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":493,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.592961+0000","last_change":"2024-09-20T22:09:02.547888+0000","last_active":"2024-09-20T22:11:40.592961+0000","last_peered":"2024-09-20T22:11:40.592961+0000","last_clean":"2024-09-20T22:11:40.592961+0000","last_became_active":"2024-09-20T22:02:50.108457+0000","last_became_peered":"2024-09-20T22:02:50.108457+0000","last_unstale":"2024-09-20T22:11:40.592961+0000","last_undegraded":"2024-09-20T22:11:40.592961+0000","last_fullsized":"2024-09-20T22:11:40.592961+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:02.547780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:02.547780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":512,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.596096+0000","last_change":"2024-09-20T22:11:38.145140+0000","last_active":"2024-09-20T22:11:40.596096+0000","last_peered":"2024-09-20T22:11:40.596096+0000","last_clean":"2024-09-20T22:11:40.596096+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:11:40.596096+0000","last_undegraded":"2024-09-20T22:11:40.596096+0000","last_fullsized":"2024-09-20T22:11:40.596096+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:11:38.145006+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_clean_scrub_stamp":"2024-09-20T22:11:38.145006+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:42.494263+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":487,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.593581+0000","last_change":"2024-09-20T22:10:33.832638+0000","last_active":"2024-09-20T22:11:41.593581+0000","last_peered":"2024-09-20T22:11:41.593581+0000","last_clean":"2024-09-20T22:11:41.593581+0000","last_became_active":"2024-09-20T22:06:54.841419+0000","last_became_peered":"2024-09-20T22:06:54.841419+0000","last_unstale":"2024-09-20T22:11:41.593581+0000","last_undegraded":"2024-09-20T22:11:41.593581+0000","last_fullsized":"2024-09-20T22:11:41.593581+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:33.832519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:33.832519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:58.619938+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":472,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597189+0000","last_change":"2024-09-20T22:09:05.892350+0000","last_active":"2024-09-20T22:11:40.597189+0000","last_peered":"2024-09-20T22:11:40.597189+0000","last_clean":"2024-09-20T22:11:40.597189+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:11:40.597189+0000","last_undegraded":"2024-09-20T22:11:40.597189+0000","last_fullsized":"2024-09-20T22:11:40.597189+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:05.892235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:09:05.892235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":19,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.594748+0000","last_change":"2024-09-20T22:11:34.559139+0000","last_active":"2024-09-20T22:11:40.594748+0000","last_peered":"2024-09-20T22:11:40.594748+0000","last_clean":"2024-09-20T22:11:40.594748+0000","last_became_active":"2024-09-20T22:11:34.558703+0000","last_became_peered":"2024-09-20T22:11:34.558703+0000","last_unstale":"2024-09-20T22:11:40.594748+0000","last_undegraded":"2024-09-20T22:11:40.594748+0000","last_fullsized":"2024-09-20T22:11:40.594748+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:37.377130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":481,"reported_epoch":292,"state":"activating+undersized+remapped","last_fresh":"2024-09-20T22:11:41.592697+0000","last_change":"2024-09-20T22:11:41.592697+0000","last_active":"2024-09-20T22:11:40.594141+0000","last_peered":"2024-09-20T22:11:36.698684+0000","last_clean":"2024-09-20T22:11:36.698684+0000","last_became_active":"2024-09-20T22:08:55.444469+0000","last_became_peered":"2024-09-20T22:08:55.444469+0000","last_unstale":"2024-09-20T22:11:41.592697+0000","last_undegraded":"2024-09-20T22:11:41.592697+0000","last_fullsized":"2024-09-20T22:11:41.592485+0000","mapping_epoch":291,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":209,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:10:16.022854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:10:16.022854+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[0,4],"avail_no_missing":["0","2","4","7"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.7","version":"71'13","reported_seq":372,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-09-20T22:11:42.607781+0000","last_change":"2024-09-20T22:11:33.124675+0000","last_active":"2024-09-20T22:11:42.607781+0000","last_peered":"2024-09-20T22:11:42.607781+0000","last_clean":"2024-09-20T22:11:42.607781+0000","last_became_active":"2024-09-20T22:06:54.850696+0000","last_became_peered":"2024-09-20T22:06:54.850696+0000","last_unstale":"2024-09-20T22:11:42.607781+0000","last_undegraded":"2024-09-20T22:11:42.607781+0000","last_fullsized":"2024-09-20T22:11:42.607781+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'13","last_scrub_stamp":"2024-09-20T22:11:33.124595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.124595+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:33.149438+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":334,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-09-20T22:11:42.602840+0000","last_change":"2024-09-20T22:11:01.241914+0000","last_active":"2024-09-20T22:11:42.602840+0000","last_peered":"2024-09-20T22:11:42.602840+0000","last_clean":"2024-09-20T22:11:42.602840+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:11:42.602840+0000","last_undegraded":"2024-09-20T22:11:42.602840+0000","last_fullsized":"2024-09-20T22:11:42.602840+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:01.241803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:11:01.241803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:06.385257+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":526,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.591598+0000","last_change":"2024-09-20T22:11:35.306968+0000","last_active":"2024-09-20T22:11:40.591598+0000","last_peered":"2024-09-20T22:11:40.591598+0000","last_clean":"2024-09-20T22:11:40.591598+0000","last_became_active":"2024-09-20T22:02:49.099508+0000","last_became_peered":"2024-09-20T22:02:49.099508+0000","last_unstale":"2024-09-20T22:11:40.591598+0000","last_undegraded":"2024-09-20T22:11:40.591598+0000","last_fullsized":"2024-09-20T22:11:40.591598+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:11:35.306842+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:11:35.306842+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:44.238377+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":500,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.592012+0000","last_change":"2024-09-20T22:10:57.475677+0000","last_active":"2024-09-20T22:11:40.592012+0000","last_peered":"2024-09-20T22:11:40.592012+0000","last_clean":"2024-09-20T22:11:40.592012+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:11:40.592012+0000","last_undegraded":"2024-09-20T22:11:40.592012+0000","last_fullsized":"2024-09-20T22:11:40.592012+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:57.475078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:10:57.475078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:16.236114+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":481,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.599321+0000","last_change":"2024-09-20T22:11:03.448918+0000","last_active":"2024-09-20T22:11:40.599321+0000","last_peered":"2024-09-20T22:11:40.599321+0000","last_clean":"2024-09-20T22:11:40.599321+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:11:40.599321+0000","last_undegraded":"2024-09-20T22:11:40.599321+0000","last_fullsized":"2024-09-20T22:11:40.599321+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:03.448813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:03.448813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:25.942922+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":21,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.593967+0000","last_change":"2024-09-20T22:11:35.053290+0000","last_active":"2024-09-20T22:11:41.593967+0000","last_peered":"2024-09-20T22:11:41.593967+0000","last_clean":"2024-09-20T22:11:41.593967+0000","last_became_active":"2024-09-20T22:11:35.052922+0000","last_became_peered":"2024-09-20T22:11:35.052922+0000","last_unstale":"2024-09-20T22:11:41.593967+0000","last_undegraded":"2024-09-20T22:11:41.593967+0000","last_fullsized":"2024-09-20T22:11:41.593967+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:39.788504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":498,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-09-20T22:11:42.605273+0000","last_change":"2024-09-20T22:11:36.214541+0000","last_active":"2024-09-20T22:11:42.605273+0000","last_peered":"2024-09-20T22:11:42.605273+0000","last_clean":"2024-09-20T22:11:42.605273+0000","last_became_active":"2024-09-20T22:08:48.446840+0000","last_became_peered":"2024-09-20T22:08:48.446840+0000","last_unstale":"2024-09-20T22:11:42.605273+0000","last_undegraded":"2024-09-20T22:11:42.605273+0000","last_fullsized":"2024-09-20T22:11:42.605273+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:11:36.214466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:11:36.214466+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:41.066885+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"71'12","reported_seq":509,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.594756+0000","last_change":"2024-09-20T22:10:58.599478+0000","last_active":"2024-09-20T22:11:41.594756+0000","last_peered":"2024-09-20T22:11:41.594756+0000","last_clean":"2024-09-20T22:11:41.594756+0000","last_became_active":"2024-09-20T22:06:54.848670+0000","last_became_peered":"2024-09-20T22:06:54.848670+0000","last_unstale":"2024-09-20T22:11:41.594756+0000","last_undegraded":"2024-09-20T22:11:41.594756+0000","last_fullsized":"2024-09-20T22:11:41.594756+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:10:58.599362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:58.599362+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:24.582332+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":494,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.593275+0000","last_change":"2024-09-20T22:10:30.889564+0000","last_active":"2024-09-20T22:11:41.593275+0000","last_peered":"2024-09-20T22:11:41.593275+0000","last_clean":"2024-09-20T22:11:41.593275+0000","last_became_active":"2024-09-20T22:02:50.110448+0000","last_became_peered":"2024-09-20T22:02:50.110448+0000","last_unstale":"2024-09-20T22:11:41.593275+0000","last_undegraded":"2024-09-20T22:11:41.593275+0000","last_fullsized":"2024-09-20T22:11:41.593275+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:30.889484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:10:30.889484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":1028,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.599541+0000","last_change":"2024-09-20T22:10:20.084329+0000","last_active":"2024-09-20T22:11:40.599541+0000","last_peered":"2024-09-20T22:11:40.599541+0000","last_clean":"2024-09-20T22:11:40.599541+0000","last_became_active":"2024-09-20T22:06:54.844454+0000","last_became_peered":"2024-09-20T22:06:54.844454+0000","last_unstale":"2024-09-20T22:11:40.599541+0000","last_undegraded":"2024-09-20T22:11:40.599541+0000","last_fullsized":"2024-09-20T22:11:40.599541+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:10:20.084250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:10:20.084250+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:47.009552+0000","scrub_duration":61,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":517,"num_read_kb":512,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":35,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.595269+0000","last_change":"2024-09-20T22:11:39.171066+0000","last_active":"2024-09-20T22:11:40.595269+0000","last_peered":"2024-09-20T22:11:40.595269+0000","last_clean":"2024-09-20T22:11:40.595269+0000","last_became_active":"2024-09-20T22:11:30.489860+0000","last_became_peered":"2024-09-20T22:11:30.489860+0000","last_unstale":"2024-09-20T22:11:40.595269+0000","last_undegraded":"2024-09-20T22:11:40.595269+0000","last_fullsized":"2024-09-20T22:11:40.595269+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:39.170979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:39.170979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:44.884792+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":334,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-09-20T22:11:42.605325+0000","last_change":"2024-09-20T22:11:30.024842+0000","last_active":"2024-09-20T22:11:42.605325+0000","last_peered":"2024-09-20T22:11:42.605325+0000","last_clean":"2024-09-20T22:11:42.605325+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:11:42.605325+0000","last_undegraded":"2024-09-20T22:11:42.605325+0000","last_fullsized":"2024-09-20T22:11:42.605325+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:30.024770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.415468+0000","last_clean_scrub_stamp":"2024-09-20T22:11:30.024770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:34.524988+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":19,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.596761+0000","last_change":"2024-09-20T22:11:34.559269+0000","last_active":"2024-09-20T22:11:40.596761+0000","last_peered":"2024-09-20T22:11:40.596761+0000","last_clean":"2024-09-20T22:11:40.596761+0000","last_became_active":"2024-09-20T22:11:34.558981+0000","last_became_peered":"2024-09-20T22:11:34.558981+0000","last_unstale":"2024-09-20T22:11:40.596761+0000","last_undegraded":"2024-09-20T22:11:40.596761+0000","last_fullsized":"2024-09-20T22:11:40.596761+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:00.843953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"71'16","reported_seq":625,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.596196+0000","last_change":"2024-09-20T22:10:24.126185+0000","last_active":"2024-09-20T22:11:40.596196+0000","last_peered":"2024-09-20T22:11:40.596196+0000","last_clean":"2024-09-20T22:11:40.596196+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:11:40.596196+0000","last_undegraded":"2024-09-20T22:11:40.596196+0000","last_fullsized":"2024-09-20T22:11:40.596196+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'16","last_scrub_stamp":"2024-09-20T22:10:24.126065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:24.126065+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:46.151257+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":504,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597338+0000","last_change":"2024-09-20T22:11:30.487346+0000","last_active":"2024-09-20T22:11:40.597338+0000","last_peered":"2024-09-20T22:11:40.597338+0000","last_clean":"2024-09-20T22:11:40.597338+0000","last_became_active":"2024-09-20T22:11:30.486987+0000","last_became_peered":"2024-09-20T22:11:30.486987+0000","last_unstale":"2024-09-20T22:11:40.597338+0000","last_undegraded":"2024-09-20T22:11:40.597338+0000","last_fullsized":"2024-09-20T22:11:40.597338+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:18.235255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:18.235255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":484,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.598594+0000","last_change":"2024-09-20T22:10:33.656595+0000","last_active":"2024-09-20T22:11:40.598594+0000","last_peered":"2024-09-20T22:11:40.598594+0000","last_clean":"2024-09-20T22:11:40.598594+0000","last_became_active":"2024-09-20T22:10:33.656161+0000","last_became_peered":"2024-09-20T22:10:33.656161+0000","last_unstale":"2024-09-20T22:11:40.598594+0000","last_undegraded":"2024-09-20T22:11:40.598594+0000","last_fullsized":"2024-09-20T22:11:40.598594+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:10:17.014513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:10:17.014513+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":75,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.599608+0000","last_change":"2024-09-20T22:11:32.281684+0000","last_active":"2024-09-20T22:11:40.599608+0000","last_peered":"2024-09-20T22:11:40.599608+0000","last_clean":"2024-09-20T22:11:40.599608+0000","last_became_active":"2024-09-20T22:10:48.907271+0000","last_became_peered":"2024-09-20T22:10:48.907271+0000","last_unstale":"2024-09-20T22:11:40.599608+0000","last_undegraded":"2024-09-20T22:11:40.599608+0000","last_fullsized":"2024-09-20T22:11:40.599608+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:32.281594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:32.281594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:43.436523+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":502,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.595035+0000","last_change":"2024-09-20T22:11:00.215924+0000","last_active":"2024-09-20T22:11:40.595035+0000","last_peered":"2024-09-20T22:11:40.595035+0000","last_clean":"2024-09-20T22:11:40.595035+0000","last_became_active":"2024-09-20T22:04:21.237725+0000","last_became_peered":"2024-09-20T22:04:21.237725+0000","last_unstale":"2024-09-20T22:11:40.595035+0000","last_undegraded":"2024-09-20T22:11:40.595035+0000","last_fullsized":"2024-09-20T22:11:40.595035+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:00.215806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:11:00.215806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:25.875894+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":19,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.598998+0000","last_change":"2024-09-20T22:11:34.559883+0000","last_active":"2024-09-20T22:11:40.598998+0000","last_peered":"2024-09-20T22:11:40.598998+0000","last_clean":"2024-09-20T22:11:40.598998+0000","last_became_active":"2024-09-20T22:11:34.559353+0000","last_became_peered":"2024-09-20T22:11:34.559353+0000","last_unstale":"2024-09-20T22:11:40.598998+0000","last_undegraded":"2024-09-20T22:11:40.598998+0000","last_fullsized":"2024-09-20T22:11:40.598998+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:54.364720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"71'11","reported_seq":508,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.592478+0000","last_change":"2024-09-20T22:10:24.621114+0000","last_active":"2024-09-20T22:11:40.592478+0000","last_peered":"2024-09-20T22:11:40.592478+0000","last_clean":"2024-09-20T22:11:40.592478+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:11:40.592478+0000","last_undegraded":"2024-09-20T22:11:40.592478+0000","last_fullsized":"2024-09-20T22:11:40.592478+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:10:24.620997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:24.620997+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":493,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.592044+0000","last_change":"2024-09-20T22:11:07.211087+0000","last_active":"2024-09-20T22:11:40.592044+0000","last_peered":"2024-09-20T22:11:40.592044+0000","last_clean":"2024-09-20T22:11:40.592044+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:11:40.592044+0000","last_undegraded":"2024-09-20T22:11:40.592044+0000","last_fullsized":"2024-09-20T22:11:40.592044+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:07.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:11:07.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:16.920794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":491,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.594823+0000","last_change":"2024-09-20T22:10:34.232282+0000","last_active":"2024-09-20T22:11:40.594823+0000","last_peered":"2024-09-20T22:11:40.594823+0000","last_clean":"2024-09-20T22:11:40.594823+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:11:40.594823+0000","last_undegraded":"2024-09-20T22:11:40.594823+0000","last_fullsized":"2024-09-20T22:11:40.594823+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:34.232168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:34.232168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:42.928430+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":491,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597759+0000","last_change":"2024-09-20T22:11:30.487618+0000","last_active":"2024-09-20T22:11:40.597759+0000","last_peered":"2024-09-20T22:11:40.597759+0000","last_clean":"2024-09-20T22:11:40.597759+0000","last_became_active":"2024-09-20T22:11:30.487283+0000","last_became_peered":"2024-09-20T22:11:30.487283+0000","last_unstale":"2024-09-20T22:11:40.597759+0000","last_undegraded":"2024-09-20T22:11:40.597759+0000","last_fullsized":"2024-09-20T22:11:40.597759+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:17.209772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:17.209772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-09-20T22:11:42.602369+0000","last_change":"2024-09-20T22:11:35.304586+0000","last_active":"2024-09-20T22:11:42.602369+0000","last_peered":"2024-09-20T22:11:42.602369+0000","last_clean":"2024-09-20T22:11:42.602369+0000","last_became_active":"2024-09-20T22:11:35.304302+0000","last_became_peered":"2024-09-20T22:11:35.304302+0000","last_unstale":"2024-09-20T22:11:42.602369+0000","last_undegraded":"2024-09-20T22:11:42.602369+0000","last_fullsized":"2024-09-20T22:11:42.602369+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:45.993811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":398,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.594652+0000","last_change":"2024-09-20T22:11:33.557801+0000","last_active":"2024-09-20T22:11:40.594652+0000","last_peered":"2024-09-20T22:11:40.594652+0000","last_clean":"2024-09-20T22:11:40.594652+0000","last_became_active":"2024-09-20T22:11:33.557208+0000","last_became_peered":"2024-09-20T22:11:33.557208+0000","last_unstale":"2024-09-20T22:11:40.594652+0000","last_undegraded":"2024-09-20T22:11:40.594652+0000","last_fullsized":"2024-09-20T22:11:40.594652+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:10:34.490571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:10:34.490571+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:59.242488+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"71'17","reported_seq":525,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.593441+0000","last_change":"2024-09-20T22:10:29.580641+0000","last_active":"2024-09-20T22:11:40.593441+0000","last_peered":"2024-09-20T22:11:40.593441+0000","last_clean":"2024-09-20T22:11:40.593441+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:11:40.593441+0000","last_undegraded":"2024-09-20T22:11:40.593441+0000","last_fullsized":"2024-09-20T22:11:40.593441+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'17","last_scrub_stamp":"2024-09-20T22:10:29.579935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:29.579935+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:48.628798+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":501,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.596964+0000","last_change":"2024-09-20T22:10:27.228704+0000","last_active":"2024-09-20T22:11:40.596964+0000","last_peered":"2024-09-20T22:11:40.596964+0000","last_clean":"2024-09-20T22:11:40.596964+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:11:40.596964+0000","last_undegraded":"2024-09-20T22:11:40.596964+0000","last_fullsized":"2024-09-20T22:11:40.596964+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:27.228607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:27.228607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:51.592209+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":334,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-09-20T22:11:42.605434+0000","last_change":"2024-09-20T22:11:02.256320+0000","last_active":"2024-09-20T22:11:42.605434+0000","last_peered":"2024-09-20T22:11:42.605434+0000","last_clean":"2024-09-20T22:11:42.605434+0000","last_became_active":"2024-09-20T22:06:54.848275+0000","last_became_peered":"2024-09-20T22:06:54.848275+0000","last_unstale":"2024-09-20T22:11:42.605434+0000","last_undegraded":"2024-09-20T22:11:42.605434+0000","last_fullsized":"2024-09-20T22:11:42.605434+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:02.256217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:02.256217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:26.715522+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":495,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.592195+0000","last_change":"2024-09-20T22:11:31.333144+0000","last_active":"2024-09-20T22:11:40.592195+0000","last_peered":"2024-09-20T22:11:40.592195+0000","last_clean":"2024-09-20T22:11:40.592195+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:11:40.592195+0000","last_undegraded":"2024-09-20T22:11:40.592195+0000","last_fullsized":"2024-09-20T22:11:40.592195+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:31.333027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:31.333027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:53.330444+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":19,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597692+0000","last_change":"2024-09-20T22:11:35.053482+0000","last_active":"2024-09-20T22:11:40.597692+0000","last_peered":"2024-09-20T22:11:40.597692+0000","last_clean":"2024-09-20T22:11:40.597692+0000","last_became_active":"2024-09-20T22:11:35.053141+0000","last_became_peered":"2024-09-20T22:11:35.053141+0000","last_unstale":"2024-09-20T22:11:40.597692+0000","last_undegraded":"2024-09-20T22:11:40.597692+0000","last_fullsized":"2024-09-20T22:11:40.597692+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:49.844334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":772,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.599194+0000","last_change":"2024-09-20T22:11:34.332239+0000","last_active":"2024-09-20T22:11:40.599194+0000","last_peered":"2024-09-20T22:11:40.599194+0000","last_clean":"2024-09-20T22:11:40.599194+0000","last_became_active":"2024-09-20T22:08:48.455288+0000","last_became_peered":"2024-09-20T22:08:48.455288+0000","last_unstale":"2024-09-20T22:11:40.599194+0000","last_undegraded":"2024-09-20T22:11:40.599194+0000","last_fullsized":"2024-09-20T22:11:40.599194+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:11:34.332158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.332158+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:36.731335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"289'15","reported_seq":508,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.592657+0000","last_change":"2024-09-20T22:11:37.277758+0000","last_active":"2024-09-20T22:11:40.592657+0000","last_peered":"2024-09-20T22:11:40.592657+0000","last_clean":"2024-09-20T22:11:40.592657+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:11:40.592657+0000","last_undegraded":"2024-09-20T22:11:40.592657+0000","last_fullsized":"2024-09-20T22:11:40.592657+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'15","last_scrub_stamp":"2024-09-20T22:11:37.277648+0000","last_deep_scrub":"289'15","last_deep_scrub_stamp":"2024-09-20T22:11:37.277648+0000","last_clean_scrub_stamp":"2024-09-20T22:11:37.277648+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:05.356939+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":498,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597662+0000","last_change":"2024-09-20T22:09:53.395968+0000","last_active":"2024-09-20T22:11:40.597662+0000","last_peered":"2024-09-20T22:11:40.597662+0000","last_clean":"2024-09-20T22:11:40.597662+0000","last_became_active":"2024-09-20T22:02:50.110446+0000","last_became_peered":"2024-09-20T22:02:50.110446+0000","last_unstale":"2024-09-20T22:11:40.597662+0000","last_undegraded":"2024-09-20T22:11:40.597662+0000","last_fullsized":"2024-09-20T22:11:40.597662+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:09:53.395833+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"283'11","reported_seq":643,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.591800+0000","last_change":"2024-09-20T22:11:30.497599+0000","last_active":"2024-09-20T22:11:40.591800+0000","last_peered":"2024-09-20T22:11:40.591800+0000","last_clean":"2024-09-20T22:11:40.591800+0000","last_became_active":"2024-09-20T22:11:30.485356+0000","last_became_peered":"2024-09-20T22:11:30.485356+0000","last_unstale":"2024-09-20T22:11:40.591800+0000","last_undegraded":"2024-09-20T22:11:40.591800+0000","last_fullsized":"2024-09-20T22:11:40.591800+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'10","last_scrub_stamp":"2024-09-20T22:11:08.571767+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:11:08.571767+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:24.914453+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":481,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.592146+0000","last_change":"2024-09-20T22:09:53.701152+0000","last_active":"2024-09-20T22:11:40.592146+0000","last_peered":"2024-09-20T22:11:40.592146+0000","last_clean":"2024-09-20T22:11:40.592146+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:11:40.592146+0000","last_undegraded":"2024-09-20T22:11:40.592146+0000","last_fullsized":"2024-09-20T22:11:40.592146+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:53.701054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:09:53.701054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":19,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.594565+0000","last_change":"2024-09-20T22:11:35.304534+0000","last_active":"2024-09-20T22:11:40.594565+0000","last_peered":"2024-09-20T22:11:40.594565+0000","last_clean":"2024-09-20T22:11:40.594565+0000","last_became_active":"2024-09-20T22:11:35.304192+0000","last_became_peered":"2024-09-20T22:11:35.304192+0000","last_unstale":"2024-09-20T22:11:40.594565+0000","last_undegraded":"2024-09-20T22:11:40.594565+0000","last_fullsized":"2024-09-20T22:11:40.594565+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:35.149263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"71'9","reported_seq":530,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.594056+0000","last_change":"2024-09-20T22:11:02.575450+0000","last_active":"2024-09-20T22:11:41.594056+0000","last_peered":"2024-09-20T22:11:41.594056+0000","last_clean":"2024-09-20T22:11:41.594056+0000","last_became_active":"2024-09-20T22:10:22.565004+0000","last_became_peered":"2024-09-20T22:10:22.565004+0000","last_unstale":"2024-09-20T22:11:41.594056+0000","last_undegraded":"2024-09-20T22:11:41.594056+0000","last_fullsized":"2024-09-20T22:11:41.594056+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:11:02.575330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:02.575330+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:05.729253+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":120,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.598166+0000","last_change":"2024-09-20T22:10:31.161469+0000","last_active":"2024-09-20T22:11:40.598166+0000","last_peered":"2024-09-20T22:11:40.598166+0000","last_clean":"2024-09-20T22:11:40.598166+0000","last_became_active":"2024-09-20T22:10:14.536155+0000","last_became_peered":"2024-09-20T22:10:14.536155+0000","last_unstale":"2024-09-20T22:11:40.598166+0000","last_undegraded":"2024-09-20T22:11:40.598166+0000","last_fullsized":"2024-09-20T22:11:40.598166+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:10:31.161375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:31.161375+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:49.187429+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":500,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.591670+0000","last_change":"2024-09-20T22:10:22.522763+0000","last_active":"2024-09-20T22:11:40.591670+0000","last_peered":"2024-09-20T22:11:40.591670+0000","last_clean":"2024-09-20T22:11:40.591670+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:11:40.591670+0000","last_undegraded":"2024-09-20T22:11:40.591670+0000","last_fullsized":"2024-09-20T22:11:40.591670+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:22.522610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:10:22.522610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:40.748099+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":484,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.593234+0000","last_change":"2024-09-20T22:08:27.817949+0000","last_active":"2024-09-20T22:11:41.593234+0000","last_peered":"2024-09-20T22:11:41.593234+0000","last_clean":"2024-09-20T22:11:41.593234+0000","last_became_active":"2024-09-20T22:06:55.824229+0000","last_became_peered":"2024-09-20T22:06:55.824229+0000","last_unstale":"2024-09-20T22:11:41.593234+0000","last_undegraded":"2024-09-20T22:11:41.593234+0000","last_fullsized":"2024-09-20T22:11:41.593234+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:27.817850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:27.817850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":25,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.601503+0000","last_change":"2024-09-20T22:11:40.601503+0000","last_active":"2024-09-20T22:11:40.601503+0000","last_peered":"2024-09-20T22:11:40.601503+0000","last_clean":"2024-09-20T22:11:40.601503+0000","last_became_active":"2024-09-20T22:11:40.601030+0000","last_became_peered":"2024-09-20T22:11:40.601030+0000","last_unstale":"2024-09-20T22:11:40.601503+0000","last_undegraded":"2024-09-20T22:11:40.601503+0000","last_fullsized":"2024-09-20T22:11:40.601503+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:02.356649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"71'19","reported_seq":572,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.596785+0000","last_change":"2024-09-20T22:10:30.669468+0000","last_active":"2024-09-20T22:11:40.596785+0000","last_peered":"2024-09-20T22:11:40.596785+0000","last_clean":"2024-09-20T22:11:40.596785+0000","last_became_active":"2024-09-20T22:10:30.652967+0000","last_became_peered":"2024-09-20T22:10:30.652967+0000","last_unstale":"2024-09-20T22:11:40.596785+0000","last_undegraded":"2024-09-20T22:11:40.596785+0000","last_fullsized":"2024-09-20T22:11:40.596785+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:10:15.165817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:15.165817+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":504,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.599624+0000","last_change":"2024-09-20T22:11:30.489226+0000","last_active":"2024-09-20T22:11:40.599624+0000","last_peered":"2024-09-20T22:11:40.599624+0000","last_clean":"2024-09-20T22:11:40.599624+0000","last_became_active":"2024-09-20T22:11:30.488650+0000","last_became_peered":"2024-09-20T22:11:30.488650+0000","last_unstale":"2024-09-20T22:11:40.599624+0000","last_undegraded":"2024-09-20T22:11:40.599624+0000","last_fullsized":"2024-09-20T22:11:40.599624+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:28.095505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:28.095505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:43.091921+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":511,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.591629+0000","last_change":"2024-09-20T22:11:30.484978+0000","last_active":"2024-09-20T22:11:40.591629+0000","last_peered":"2024-09-20T22:11:40.591629+0000","last_clean":"2024-09-20T22:11:40.591629+0000","last_became_active":"2024-09-20T22:11:30.484597+0000","last_became_peered":"2024-09-20T22:11:30.484597+0000","last_unstale":"2024-09-20T22:11:40.591629+0000","last_undegraded":"2024-09-20T22:11:40.591629+0000","last_fullsized":"2024-09-20T22:11:40.591629+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_clean_scrub_stamp":"2024-09-20T22:10:26.501363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:46.983222+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":89,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.599680+0000","last_change":"2024-09-20T22:10:30.657488+0000","last_active":"2024-09-20T22:11:40.599680+0000","last_peered":"2024-09-20T22:11:40.599680+0000","last_clean":"2024-09-20T22:11:40.599680+0000","last_became_active":"2024-09-20T22:10:30.657116+0000","last_became_peered":"2024-09-20T22:10:30.657116+0000","last_unstale":"2024-09-20T22:11:40.599680+0000","last_undegraded":"2024-09-20T22:11:40.599680+0000","last_fullsized":"2024-09-20T22:11:40.599680+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:08.495599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:08.495599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":23,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-09-20T22:11:42.605583+0000","last_change":"2024-09-20T22:11:34.558254+0000","last_active":"2024-09-20T22:11:42.605583+0000","last_peered":"2024-09-20T22:11:42.605583+0000","last_clean":"2024-09-20T22:11:42.605583+0000","last_became_active":"2024-09-20T22:11:34.557976+0000","last_became_peered":"2024-09-20T22:11:34.557976+0000","last_unstale":"2024-09-20T22:11:42.605583+0000","last_undegraded":"2024-09-20T22:11:42.605583+0000","last_fullsized":"2024-09-20T22:11:42.605583+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:38.503099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"71'12","reported_seq":513,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.591642+0000","last_change":"2024-09-20T22:10:21.628457+0000","last_active":"2024-09-20T22:11:40.591642+0000","last_peered":"2024-09-20T22:11:40.591642+0000","last_clean":"2024-09-20T22:11:40.591642+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:11:40.591642+0000","last_undegraded":"2024-09-20T22:11:40.591642+0000","last_fullsized":"2024-09-20T22:11:40.591642+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:10:21.628338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:21.628338+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:42.485115+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":493,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.600078+0000","last_change":"2024-09-20T22:09:03.831996+0000","last_active":"2024-09-20T22:11:40.600078+0000","last_peered":"2024-09-20T22:11:40.600078+0000","last_clean":"2024-09-20T22:11:40.600078+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:11:40.600078+0000","last_undegraded":"2024-09-20T22:11:40.600078+0000","last_fullsized":"2024-09-20T22:11:40.600078+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:03.831774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:03.831774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":509,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.591104+0000","last_change":"2024-09-20T22:11:36.287314+0000","last_active":"2024-09-20T22:11:40.591104+0000","last_peered":"2024-09-20T22:11:40.591104+0000","last_clean":"2024-09-20T22:11:40.591104+0000","last_became_active":"2024-09-20T22:06:54.843906+0000","last_became_peered":"2024-09-20T22:06:54.843906+0000","last_unstale":"2024-09-20T22:11:40.591104+0000","last_undegraded":"2024-09-20T22:11:40.591104+0000","last_fullsized":"2024-09-20T22:11:40.591104+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:36.287220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:36.287220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:47.050640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":439,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.594320+0000","last_change":"2024-09-20T22:11:40.600795+0000","last_active":"2024-09-20T22:11:41.594320+0000","last_peered":"2024-09-20T22:11:41.594320+0000","last_clean":"2024-09-20T22:11:41.594320+0000","last_became_active":"2024-09-20T22:11:40.600444+0000","last_became_peered":"2024-09-20T22:11:40.600444+0000","last_unstale":"2024-09-20T22:11:41.594320+0000","last_undegraded":"2024-09-20T22:11:41.594320+0000","last_fullsized":"2024-09-20T22:11:41.594320+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:36.768146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:36.768146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":27,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.593345+0000","last_change":"2024-09-20T22:11:40.601169+0000","last_active":"2024-09-20T22:11:41.593345+0000","last_peered":"2024-09-20T22:11:41.593345+0000","last_clean":"2024-09-20T22:11:41.593345+0000","last_became_active":"2024-09-20T22:11:40.600788+0000","last_became_peered":"2024-09-20T22:11:40.600788+0000","last_unstale":"2024-09-20T22:11:41.593345+0000","last_undegraded":"2024-09-20T22:11:41.593345+0000","last_fullsized":"2024-09-20T22:11:41.593345+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:52.412361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"71'15","reported_seq":43,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.594280+0000","last_change":"2024-09-20T22:11:30.506603+0000","last_active":"2024-09-20T22:11:40.594280+0000","last_peered":"2024-09-20T22:11:40.594280+0000","last_clean":"2024-09-20T22:11:40.594280+0000","last_became_active":"2024-09-20T22:11:30.493275+0000","last_became_peered":"2024-09-20T22:11:30.493275+0000","last_unstale":"2024-09-20T22:11:40.594280+0000","last_undegraded":"2024-09-20T22:11:40.594280+0000","last_fullsized":"2024-09-20T22:11:40.594280+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:11:02.397374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:02.397374+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:28.185784+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":95,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.600297+0000","last_change":"2024-09-20T22:10:30.661904+0000","last_active":"2024-09-20T22:11:40.600297+0000","last_peered":"2024-09-20T22:11:40.600297+0000","last_clean":"2024-09-20T22:11:40.600297+0000","last_became_active":"2024-09-20T22:10:30.653330+0000","last_became_peered":"2024-09-20T22:10:30.653330+0000","last_unstale":"2024-09-20T22:11:40.600297+0000","last_undegraded":"2024-09-20T22:11:40.600297+0000","last_fullsized":"2024-09-20T22:11:40.600297+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:09:48.706938+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:09:48.706938+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":592,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.598906+0000","last_change":"2024-09-20T22:08:32.824544+0000","last_active":"2024-09-20T22:11:40.598906+0000","last_peered":"2024-09-20T22:11:40.598906+0000","last_clean":"2024-09-20T22:11:40.598906+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:11:40.598906+0000","last_undegraded":"2024-09-20T22:11:40.598906+0000","last_fullsized":"2024-09-20T22:11:40.598906+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:08:32.824413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.824413+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":27,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.595822+0000","last_change":"2024-09-20T22:11:30.492983+0000","last_active":"2024-09-20T22:11:40.595822+0000","last_peered":"2024-09-20T22:11:40.595822+0000","last_clean":"2024-09-20T22:11:40.595822+0000","last_became_active":"2024-09-20T22:11:30.492476+0000","last_became_peered":"2024-09-20T22:11:30.492476+0000","last_unstale":"2024-09-20T22:11:40.595822+0000","last_undegraded":"2024-09-20T22:11:40.595822+0000","last_fullsized":"2024-09-20T22:11:40.595822+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:56.334297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:56.334297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:21.467964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":519,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.596244+0000","last_change":"2024-09-20T22:11:39.375720+0000","last_active":"2024-09-20T22:11:40.596244+0000","last_peered":"2024-09-20T22:11:40.596244+0000","last_clean":"2024-09-20T22:11:40.596244+0000","last_became_active":"2024-09-20T22:11:30.486764+0000","last_became_peered":"2024-09-20T22:11:30.486764+0000","last_unstale":"2024-09-20T22:11:40.596244+0000","last_undegraded":"2024-09-20T22:11:40.596244+0000","last_fullsized":"2024-09-20T22:11:40.596244+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:39.375639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:11:39.375639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:44.853725+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"71'6","reported_seq":524,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597441+0000","last_change":"2024-09-20T22:11:07.315899+0000","last_active":"2024-09-20T22:11:40.597441+0000","last_peered":"2024-09-20T22:11:40.597441+0000","last_clean":"2024-09-20T22:11:40.597441+0000","last_became_active":"2024-09-20T22:10:48.922561+0000","last_became_peered":"2024-09-20T22:10:48.922561+0000","last_unstale":"2024-09-20T22:11:40.597441+0000","last_undegraded":"2024-09-20T22:11:40.597441+0000","last_fullsized":"2024-09-20T22:11:40.597441+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'6","last_scrub_stamp":"2024-09-20T22:11:07.315753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:07.315753+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:09.199775+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":27,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.594832+0000","last_change":"2024-09-20T22:11:30.487894+0000","last_active":"2024-09-20T22:11:40.594832+0000","last_peered":"2024-09-20T22:11:40.594832+0000","last_clean":"2024-09-20T22:11:40.594832+0000","last_became_active":"2024-09-20T22:11:30.487322+0000","last_became_peered":"2024-09-20T22:11:30.487322+0000","last_unstale":"2024-09-20T22:11:40.594832+0000","last_undegraded":"2024-09-20T22:11:40.594832+0000","last_fullsized":"2024-09-20T22:11:40.594832+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:20.293112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:20.293112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:47.777426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":472,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.599312+0000","last_change":"2024-09-20T22:09:00.737324+0000","last_active":"2024-09-20T22:11:40.599312+0000","last_peered":"2024-09-20T22:11:40.599312+0000","last_clean":"2024-09-20T22:11:40.599312+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:11:40.599312+0000","last_undegraded":"2024-09-20T22:11:40.599312+0000","last_fullsized":"2024-09-20T22:11:40.599312+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:00.737219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:00.737219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"71'9","reported_seq":528,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.594828+0000","last_change":"2024-09-20T22:10:48.930194+0000","last_active":"2024-09-20T22:11:41.594828+0000","last_peered":"2024-09-20T22:11:41.594828+0000","last_clean":"2024-09-20T22:11:41.594828+0000","last_became_active":"2024-09-20T22:10:48.924249+0000","last_became_peered":"2024-09-20T22:10:48.924249+0000","last_unstale":"2024-09-20T22:11:41.594828+0000","last_undegraded":"2024-09-20T22:11:41.594828+0000","last_fullsized":"2024-09-20T22:11:41.594828+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:08:31.897253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:31.897253+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":492,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.599346+0000","last_change":"2024-09-20T22:10:29.059396+0000","last_active":"2024-09-20T22:11:40.599346+0000","last_peered":"2024-09-20T22:11:40.599346+0000","last_clean":"2024-09-20T22:11:40.599346+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:11:40.599346+0000","last_undegraded":"2024-09-20T22:11:40.599346+0000","last_fullsized":"2024-09-20T22:11:40.599346+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:29.059287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:29.059287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":484,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597224+0000","last_change":"2024-09-20T22:11:09.360634+0000","last_active":"2024-09-20T22:11:40.597224+0000","last_peered":"2024-09-20T22:11:40.597224+0000","last_clean":"2024-09-20T22:11:40.597224+0000","last_became_active":"2024-09-20T22:06:54.845704+0000","last_became_peered":"2024-09-20T22:06:54.845704+0000","last_unstale":"2024-09-20T22:11:40.597224+0000","last_undegraded":"2024-09-20T22:11:40.597224+0000","last_fullsized":"2024-09-20T22:11:40.597224+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_clean_scrub_stamp":"2024-09-20T22:11:09.360439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:31.840825+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":495,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.591409+0000","last_change":"2024-09-20T22:11:05.544433+0000","last_active":"2024-09-20T22:11:40.591409+0000","last_peered":"2024-09-20T22:11:40.591409+0000","last_clean":"2024-09-20T22:11:40.591409+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:11:40.591409+0000","last_undegraded":"2024-09-20T22:11:40.591409+0000","last_fullsized":"2024-09-20T22:11:40.591409+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:05.544321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:05.544321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:20.221161+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"71'9","reported_seq":37,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.595587+0000","last_change":"2024-09-20T22:11:30.502818+0000","last_active":"2024-09-20T22:11:40.595587+0000","last_peered":"2024-09-20T22:11:40.595587+0000","last_clean":"2024-09-20T22:11:40.595587+0000","last_became_active":"2024-09-20T22:11:30.493335+0000","last_became_peered":"2024-09-20T22:11:30.493335+0000","last_unstale":"2024-09-20T22:11:40.595587+0000","last_undegraded":"2024-09-20T22:11:40.595587+0000","last_fullsized":"2024-09-20T22:11:40.595587+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:10:59.360044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:59.360044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:06.729574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":504,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.593109+0000","last_change":"2024-09-20T22:10:48.923031+0000","last_active":"2024-09-20T22:11:41.593109+0000","last_peered":"2024-09-20T22:11:41.593109+0000","last_clean":"2024-09-20T22:11:41.593109+0000","last_became_active":"2024-09-20T22:10:48.922707+0000","last_became_peered":"2024-09-20T22:10:48.922707+0000","last_unstale":"2024-09-20T22:11:41.593109+0000","last_undegraded":"2024-09-20T22:11:41.593109+0000","last_fullsized":"2024-09-20T22:11:41.593109+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:28.999559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:28.999559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:55.434360+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"276'10","reported_seq":610,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597896+0000","last_change":"2024-09-20T22:11:12.512544+0000","last_active":"2024-09-20T22:11:40.597896+0000","last_peered":"2024-09-20T22:11:40.597896+0000","last_clean":"2024-09-20T22:11:40.597896+0000","last_became_active":"2024-09-20T22:11:12.492297+0000","last_became_peered":"2024-09-20T22:11:12.492297+0000","last_unstale":"2024-09-20T22:11:40.597896+0000","last_undegraded":"2024-09-20T22:11:40.597896+0000","last_fullsized":"2024-09-20T22:11:40.597896+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:09:07.902502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:07.902502+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":492,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.592469+0000","last_change":"2024-09-20T22:11:30.485332+0000","last_active":"2024-09-20T22:11:40.592469+0000","last_peered":"2024-09-20T22:11:40.592469+0000","last_clean":"2024-09-20T22:11:40.592469+0000","last_became_active":"2024-09-20T22:11:30.484731+0000","last_became_peered":"2024-09-20T22:11:30.484731+0000","last_unstale":"2024-09-20T22:11:40.592469+0000","last_undegraded":"2024-09-20T22:11:40.592469+0000","last_fullsized":"2024-09-20T22:11:40.592469+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:05.183875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:05.183875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:32.622574+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"71'10","reported_seq":521,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.598016+0000","last_change":"2024-09-20T22:11:40.438463+0000","last_active":"2024-09-20T22:11:40.598016+0000","last_peered":"2024-09-20T22:11:40.598016+0000","last_clean":"2024-09-20T22:11:40.598016+0000","last_became_active":"2024-09-20T22:10:22.583028+0000","last_became_peered":"2024-09-20T22:10:22.583028+0000","last_unstale":"2024-09-20T22:11:40.598016+0000","last_undegraded":"2024-09-20T22:11:40.598016+0000","last_fullsized":"2024-09-20T22:11:40.598016+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:11:40.438345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:40.438345+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:02.299535+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":484,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.600002+0000","last_change":"2024-09-20T22:08:56.746747+0000","last_active":"2024-09-20T22:11:40.600002+0000","last_peered":"2024-09-20T22:11:40.600002+0000","last_clean":"2024-09-20T22:11:40.600002+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:11:40.600002+0000","last_undegraded":"2024-09-20T22:11:40.600002+0000","last_fullsized":"2024-09-20T22:11:40.600002+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:56.746651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:56.746651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"283'12","reported_seq":42,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.594873+0000","last_change":"2024-09-20T22:11:35.062361+0000","last_active":"2024-09-20T22:11:40.594873+0000","last_peered":"2024-09-20T22:11:40.594873+0000","last_clean":"2024-09-20T22:11:40.594873+0000","last_became_active":"2024-09-20T22:11:30.488787+0000","last_became_peered":"2024-09-20T22:11:30.488787+0000","last_unstale":"2024-09-20T22:11:40.594873+0000","last_undegraded":"2024-09-20T22:11:40.594873+0000","last_fullsized":"2024-09-20T22:11:40.594873+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'12","last_scrub_stamp":"2024-09-20T22:11:35.062247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:35.062247+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:39.696169+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":492,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.594374+0000","last_change":"2024-09-20T22:11:05.577252+0000","last_active":"2024-09-20T22:11:41.594374+0000","last_peered":"2024-09-20T22:11:41.594374+0000","last_clean":"2024-09-20T22:11:41.594374+0000","last_became_active":"2024-09-20T22:10:30.659586+0000","last_became_peered":"2024-09-20T22:10:30.659586+0000","last_unstale":"2024-09-20T22:11:41.594374+0000","last_undegraded":"2024-09-20T22:11:41.594374+0000","last_fullsized":"2024-09-20T22:11:41.594374+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:05.577043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:05.577043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:11.961511+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"71'11","reported_seq":526,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.592421+0000","last_change":"2024-09-20T22:09:54.730657+0000","last_active":"2024-09-20T22:11:40.592421+0000","last_peered":"2024-09-20T22:11:40.592421+0000","last_clean":"2024-09-20T22:11:40.592421+0000","last_became_active":"2024-09-20T22:08:22.139581+0000","last_became_peered":"2024-09-20T22:08:22.139581+0000","last_unstale":"2024-09-20T22:11:40.592421+0000","last_undegraded":"2024-09-20T22:11:40.592421+0000","last_fullsized":"2024-09-20T22:11:40.592421+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:09:54.730526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:09:54.730526+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":492,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.592897+0000","last_change":"2024-09-20T22:11:33.355268+0000","last_active":"2024-09-20T22:11:40.592897+0000","last_peered":"2024-09-20T22:11:40.592897+0000","last_clean":"2024-09-20T22:11:40.592897+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:11:40.592897+0000","last_undegraded":"2024-09-20T22:11:40.592897+0000","last_fullsized":"2024-09-20T22:11:40.592897+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.355142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.355142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:38.410766+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":333,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-09-20T22:11:42.604735+0000","last_change":"2024-09-20T22:11:34.160476+0000","last_active":"2024-09-20T22:11:42.604735+0000","last_peered":"2024-09-20T22:11:42.604735+0000","last_clean":"2024-09-20T22:11:42.604735+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:11:42.604735+0000","last_undegraded":"2024-09-20T22:11:42.604735+0000","last_fullsized":"2024-09-20T22:11:42.604735+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:34.160404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.160404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:49.059597+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":324,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-09-20T22:11:42.602915+0000","last_change":"2024-09-20T22:11:30.487252+0000","last_active":"2024-09-20T22:11:42.602915+0000","last_peered":"2024-09-20T22:11:42.602915+0000","last_clean":"2024-09-20T22:11:42.602915+0000","last_became_active":"2024-09-20T22:11:30.486423+0000","last_became_peered":"2024-09-20T22:11:30.486423+0000","last_unstale":"2024-09-20T22:11:42.602915+0000","last_undegraded":"2024-09-20T22:11:42.602915+0000","last_fullsized":"2024-09-20T22:11:42.602915+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:23.320831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:23.320831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:52.624823+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"291'13","reported_seq":409,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-09-20T22:11:42.604923+0000","last_change":"2024-09-20T22:11:41.362291+0000","last_active":"2024-09-20T22:11:42.604923+0000","last_peered":"2024-09-20T22:11:42.604923+0000","last_clean":"2024-09-20T22:11:42.604923+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:11:42.604923+0000","last_undegraded":"2024-09-20T22:11:42.604923+0000","last_fullsized":"2024-09-20T22:11:42.604923+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'13","last_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_deep_scrub":"291'13","last_deep_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_clean_scrub_stamp":"2024-09-20T22:11:41.362215+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:08.259331+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":89,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.599198+0000","last_change":"2024-09-20T22:10:30.656109+0000","last_active":"2024-09-20T22:11:40.599198+0000","last_peered":"2024-09-20T22:11:40.599198+0000","last_clean":"2024-09-20T22:11:40.599198+0000","last_became_active":"2024-09-20T22:10:30.655502+0000","last_became_peered":"2024-09-20T22:10:30.655502+0000","last_unstale":"2024-09-20T22:11:40.599198+0000","last_undegraded":"2024-09-20T22:11:40.599198+0000","last_fullsized":"2024-09-20T22:11:40.599198+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:28.129779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:28.129779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:55.022772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":484,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597504+0000","last_change":"2024-09-20T22:11:31.262221+0000","last_active":"2024-09-20T22:11:40.597504+0000","last_peered":"2024-09-20T22:11:40.597504+0000","last_clean":"2024-09-20T22:11:40.597504+0000","last_became_active":"2024-09-20T22:06:54.844271+0000","last_became_peered":"2024-09-20T22:06:54.844271+0000","last_unstale":"2024-09-20T22:11:40.597504+0000","last_undegraded":"2024-09-20T22:11:40.597504+0000","last_fullsized":"2024-09-20T22:11:40.597504+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:31.262095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:31.262095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:38.538398+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":494,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.592133+0000","last_change":"2024-09-20T22:11:30.487611+0000","last_active":"2024-09-20T22:11:41.592133+0000","last_peered":"2024-09-20T22:11:41.592133+0000","last_clean":"2024-09-20T22:11:41.592133+0000","last_became_active":"2024-09-20T22:11:30.487088+0000","last_became_peered":"2024-09-20T22:11:30.487088+0000","last_unstale":"2024-09-20T22:11:41.592133+0000","last_undegraded":"2024-09-20T22:11:41.592133+0000","last_fullsized":"2024-09-20T22:11:41.592133+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:35.783884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:35.783884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:03.527369+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"71'11","reported_seq":541,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597901+0000","last_change":"2024-09-20T22:10:30.672892+0000","last_active":"2024-09-20T22:11:40.597901+0000","last_peered":"2024-09-20T22:11:40.597901+0000","last_clean":"2024-09-20T22:11:40.597901+0000","last_became_active":"2024-09-20T22:10:30.651743+0000","last_became_peered":"2024-09-20T22:10:30.651743+0000","last_unstale":"2024-09-20T22:11:40.597901+0000","last_undegraded":"2024-09-20T22:11:40.597901+0000","last_fullsized":"2024-09-20T22:11:40.597901+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:09:55.321800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:09:55.321800+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":27,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.594696+0000","last_change":"2024-09-20T22:11:30.493936+0000","last_active":"2024-09-20T22:11:40.594696+0000","last_peered":"2024-09-20T22:11:40.594696+0000","last_clean":"2024-09-20T22:11:40.594696+0000","last_became_active":"2024-09-20T22:11:30.493598+0000","last_became_peered":"2024-09-20T22:11:30.493598+0000","last_unstale":"2024-09-20T22:11:40.594696+0000","last_undegraded":"2024-09-20T22:11:40.594696+0000","last_fullsized":"2024-09-20T22:11:40.594696+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_clean_scrub_stamp":"2024-09-20T22:10:36.098315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":484,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.598872+0000","last_change":"2024-09-20T22:09:01.751444+0000","last_active":"2024-09-20T22:11:40.598872+0000","last_peered":"2024-09-20T22:11:40.598872+0000","last_clean":"2024-09-20T22:11:40.598872+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:11:40.598872+0000","last_undegraded":"2024-09-20T22:11:40.598872+0000","last_fullsized":"2024-09-20T22:11:40.598872+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:01.751321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:01.751321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":136,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.594164+0000","last_change":"2024-09-20T22:09:39.492012+0000","last_active":"2024-09-20T22:11:41.594164+0000","last_peered":"2024-09-20T22:11:41.594164+0000","last_clean":"2024-09-20T22:11:41.594164+0000","last_became_active":"2024-09-20T22:09:39.491148+0000","last_became_peered":"2024-09-20T22:09:39.491148+0000","last_unstale":"2024-09-20T22:11:41.594164+0000","last_undegraded":"2024-09-20T22:11:41.594164+0000","last_fullsized":"2024-09-20T22:11:41.594164+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:10.873660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:10.873660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"71'4","reported_seq":494,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.596627+0000","last_change":"2024-09-20T22:10:21.165887+0000","last_active":"2024-09-20T22:11:40.596627+0000","last_peered":"2024-09-20T22:11:40.596627+0000","last_clean":"2024-09-20T22:11:40.596627+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:11:40.596627+0000","last_undegraded":"2024-09-20T22:11:40.596627+0000","last_fullsized":"2024-09-20T22:11:40.596627+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'4","last_scrub_stamp":"2024-09-20T22:10:21.165774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:21.165774+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":485,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.594061+0000","last_change":"2024-09-20T22:08:29.852608+0000","last_active":"2024-09-20T22:11:41.594061+0000","last_peered":"2024-09-20T22:11:41.594061+0000","last_clean":"2024-09-20T22:11:41.594061+0000","last_became_active":"2024-09-20T22:02:50.111142+0000","last_became_peered":"2024-09-20T22:02:50.111142+0000","last_unstale":"2024-09-20T22:11:41.594061+0000","last_undegraded":"2024-09-20T22:11:41.594061+0000","last_fullsized":"2024-09-20T22:11:41.594061+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:29.852502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:29.852502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":507,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.595754+0000","last_change":"2024-09-20T22:11:34.342938+0000","last_active":"2024-09-20T22:11:40.595754+0000","last_peered":"2024-09-20T22:11:40.595754+0000","last_clean":"2024-09-20T22:11:40.595754+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:11:40.595754+0000","last_undegraded":"2024-09-20T22:11:40.595754+0000","last_fullsized":"2024-09-20T22:11:40.595754+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:34.342862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.342862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:55.365769+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":480,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.598818+0000","last_change":"2024-09-20T22:11:05.468972+0000","last_active":"2024-09-20T22:11:40.598818+0000","last_peered":"2024-09-20T22:11:40.598818+0000","last_clean":"2024-09-20T22:11:40.598818+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:11:40.598818+0000","last_undegraded":"2024-09-20T22:11:40.598818+0000","last_fullsized":"2024-09-20T22:11:40.598818+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_clean_scrub_stamp":"2024-09-20T22:11:05.468863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:10.330169+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":373,"reported_epoch":293,"state":"active+clean","last_fresh":"2024-09-20T22:11:42.606050+0000","last_change":"2024-09-20T22:10:19.417244+0000","last_active":"2024-09-20T22:11:42.606050+0000","last_peered":"2024-09-20T22:11:42.606050+0000","last_clean":"2024-09-20T22:11:42.606050+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:11:42.606050+0000","last_undegraded":"2024-09-20T22:11:42.606050+0000","last_fullsized":"2024-09-20T22:11:42.606050+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:19.417142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:19.417142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:43.401093+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":492,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.597778+0000","last_change":"2024-09-20T22:10:30.179319+0000","last_active":"2024-09-20T22:11:40.597778+0000","last_peered":"2024-09-20T22:11:40.597778+0000","last_clean":"2024-09-20T22:11:40.597778+0000","last_became_active":"2024-09-20T22:06:54.848032+0000","last_became_peered":"2024-09-20T22:06:54.848032+0000","last_unstale":"2024-09-20T22:11:40.597778+0000","last_undegraded":"2024-09-20T22:11:40.597778+0000","last_fullsized":"2024-09-20T22:11:40.597778+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:30.179244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:30.179244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"71'11","reported_seq":516,"reported_epoch":291,"state":"active+clean","last_fresh":"2024-09-20T22:11:40.596731+0000","last_change":"2024-09-20T22:10:33.253189+0000","last_active":"2024-09-20T22:11:40.596731+0000","last_peered":"2024-09-20T22:11:40.596731+0000","last_clean":"2024-09-20T22:11:40.596731+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:11:40.596731+0000","last_undegraded":"2024-09-20T22:11:40.596731+0000","last_fullsized":"2024-09-20T22:11:40.596731+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:10:33.253077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:33.253077+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":490,"reported_epoch":292,"state":"active+clean","last_fresh":"2024-09-20T22:11:41.594546+0000","last_change":"2024-09-20T22:11:12.495944+0000","last_active":"2024-09-20T22:11:41.594546+0000","last_peered":"2024-09-20T22:11:41.594546+0000","last_clean":"2024-09-20T22:11:41.594546+0000","last_became_active":"2024-09-20T22:11:12.495261+0000","last_became_peered":"2024-09-20T22:11:12.495261+0000","last_unstale":"2024-09-20T22:11:41.594546+0000","last_undegraded":"2024-09-20T22:11:41.594546+0000","last_fullsized":"2024-09-20T22:11:41.594546+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:32.854221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.854221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82,"ondisk_log_size":82,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":692,"num_read_kb":449,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":1870,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":430,"ondisk_log_size":430,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1085,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":517,"num_read_kb":512,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":918048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1408,"ondisk_log_size":1408,"up":33,"acting":32,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135982,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":424220,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":422654,"kb_avail":93320932,"statfs":{"total":95995035648,"available":95560634368,"internally_reserved":0,"allocated":1581056,"data_stored":974677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":432798154},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430322,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":361416,"kb_used_data":1204,"kb_used_omap":1,"kb_used_meta":360190,"kb_avail":93383736,"statfs":{"total":95995035648,"available":95624945664,"internally_reserved":0,"allocated":1232896,"data_stored":646795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":368835014},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724662,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":468684,"kb_used_data":44600,"kb_used_omap":1,"kb_used_meta":424062,"kb_avail":93276468,"statfs":{"total":95995035648,"available":95515103232,"internally_reserved":0,"allocated":45670400,"data_stored":45080210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":434239941},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986302,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":393376,"kb_used_data":1228,"kb_used_omap":1,"kb_used_meta":392126,"kb_avail":93351776,"statfs":{"total":95995035648,"available":95592218624,"internally_reserved":0,"allocated":1257472,"data_stored":646830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":401537477},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770257979,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":261388,"kb_used_data":1272,"kb_used_omap":4,"kb_used_meta":260091,"kb_avail":93483764,"statfs":{"total":95995035648,"available":95727374336,"internally_reserved":0,"allocated":1302528,"data_stored":665028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":266333864},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149827,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":462004,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":460158,"kb_avail":93283148,"statfs":{"total":95995035648,"available":95521943552,"internally_reserved":0,"allocated":1867776,"data_stored":1238490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":471202247},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411464,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":438024,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":436542,"kb_avail":93307128,"statfs":{"total":95995035648,"available":95546499072,"internally_reserved":0,"allocated":1495040,"data_stored":908845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":447019461},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738515,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":501684,"kb_used_data":1248,"kb_used_omap":1,"kb_used_meta":500414,"kb_avail":93243468,"statfs":{"total":95995035648,"available":95481311232,"internally_reserved":0,"allocated":1277952,"data_stored":663920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":512424389},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:11:45.569 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:11:45.586 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:45.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:45.587 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:45.617 INFO:teuthology.orchestra.run.smithi060.stderr:enabled application 'rados' on pool 'unique_pool_2' 2024-09-20T22:11:45.687 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:45.713 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:45.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:45.714 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:45.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:45 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3019560850' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:11:45.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:45 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:11:45.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:45 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3019560850' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:11:45.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:45 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:11:45.815 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:45.838 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:45.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:45.839 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:45.939 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:45.966 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:45.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:45.967 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:45 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3019560850' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:11:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:45 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:11:46.068 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:46.095 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:46.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:46.096 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:46.196 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:46.214 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:11:46.224 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:46.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:46.225 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:46.325 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:46.352 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:46.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:46.353 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:46.454 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:46.476 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:46.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:46.477 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:46.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:46.601 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:46.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:46.602 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:46.702 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:46.725 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:46.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:46.726 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:46.827 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:46.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:46.854 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:46.855 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:46.956 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:46.983 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:46.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:46.984 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:46.991 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:46 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T22:11:46.991 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:46 smithi060 ceph-mon[29694]: osdmap e295: 8 total, 8 up, 8 in 2024-09-20T22:11:46.991 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:46 smithi060 ceph-mon[29694]: pgmap v478: 158 pgs: 158 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 0 B/s, 1 objects/s recovering 2024-09-20T22:11:46.991 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:46 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T22:11:46.991 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:46 smithi060 ceph-mon[40290]: osdmap e295: 8 total, 8 up, 8 in 2024-09-20T22:11:46.991 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:46 smithi060 ceph-mon[40290]: pgmap v478: 158 pgs: 158 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 0 B/s, 1 objects/s recovering 2024-09-20T22:11:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:46 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_2", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T22:11:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:46 smithi067 ceph-mon[33511]: osdmap e295: 8 total, 8 up, 8 in 2024-09-20T22:11:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:46 smithi067 ceph-mon[33511]: pgmap v478: 158 pgs: 158 active+clean; 583 KiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s; 0 B/s, 1 objects/s recovering 2024-09-20T22:11:47.085 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:47.109 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:11:47.109 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:11:47.111 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:47.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:47.112 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:47.118 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:47.122 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:47.213 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:47.240 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:47.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:47.241 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:47.322 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:47.342 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:47.368 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:47.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:47.369 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:47.470 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:47.492 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:47.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:47.493 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:47.594 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:47.616 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:47.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:47.617 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:47.719 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:47.740 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:47.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:47.742 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:47.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:47 smithi060 ceph-mon[29694]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T22:11:47.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:47 smithi060 ceph-mon[40290]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T22:11:47.833 DEBUG:teuthology.orchestra.run.smithi060:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_2 bench 90 write' 2024-09-20T22:11:47.836 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-20T22:11:47.836 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-20T22:11:47.842 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:47.864 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:47.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:47.865 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:47.887 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:hints = 1 2024-09-20T22:11:47.887 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-20T22:11:47.887 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Object prefix: benchmark_data_smithi060_140132 2024-09-20T22:11:47.888 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:11:47.888 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 0 0 0 0 0 0 - 0 2024-09-20T22:11:47.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:48.007 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:48.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:48.008 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:47 smithi067 ceph-mon[33511]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T22:11:48.048 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T22:11:48.109 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:48.141 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:48.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:48.142 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:48.243 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:48.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:48.275 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:48.276 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:48.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:48.417 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:11:48.417 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:11:48.419 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:48.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:48.420 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:48.456 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:11:48.457 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:11:48.494 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:11:48.494 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:11:48.521 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:48.544 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:48.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:48.545 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:48.586 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:11:48.586 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:11:48.618 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:11:48.619 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:11:48.646 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:48.654 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:11:48.654 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:11:48.681 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:48.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:48.683 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:48.691 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:11:48.691 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:11:48.728 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:11:48.728 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:11:48.784 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:48.816 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:48.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:48.817 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:48.888 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 1 16 3073 3057 191.052 191.062 0.00491065 0.00521605 2024-09-20T22:11:48.916 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:48 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:11:48.916 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:48 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:11:48.916 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:48 smithi060 ceph-mon[29694]: pgmap v479: 158 pgs: 158 active+clean; 4.6 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 682 KiB/s wr, 11 op/s 2024-09-20T22:11:48.916 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:48 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3481419373' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:48.916 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:48 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:11:48.916 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:48 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:11:48.917 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:48 smithi060 ceph-mon[40290]: pgmap v479: 158 pgs: 158 active+clean; 4.6 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 682 KiB/s wr, 11 op/s 2024-09-20T22:11:48.917 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:48 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3481419373' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:48.918 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:48.962 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:48.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:48.963 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:49.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:48 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:11:49.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:48 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:11:49.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:48 smithi067 ceph-mon[33511]: pgmap v479: 158 pgs: 158 active+clean; 4.6 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 853 B/s rd, 682 KiB/s wr, 11 op/s 2024-09-20T22:11:49.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:48 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3481419373' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:49.064 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:49.104 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:49.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:49.105 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:49.206 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:49.242 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:49.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:49.243 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:49.344 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:49.378 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:49.379 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:49.379 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:49.480 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:49.514 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:49.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:49.515 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:49.616 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:49.650 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:49.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:49.651 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:49.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:49.792 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:49.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:49.793 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:49.888 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 2 15 6253 6238 194.923 198.812 0.00514901 0.00512253 2024-09-20T22:11:49.894 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:49.903 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:49 smithi060 ceph-mon[29694]: osdmap e296: 8 total, 8 up, 8 in 2024-09-20T22:11:49.903 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:49 smithi060 ceph-mon[40290]: osdmap e296: 8 total, 8 up, 8 in 2024-09-20T22:11:49.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:49.943 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:49.943 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:50.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:49 smithi067 ceph-mon[33511]: osdmap e296: 8 total, 8 up, 8 in 2024-09-20T22:11:50.044 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:50.085 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:50.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:50.086 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:50.187 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:50.222 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:50.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:50.223 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:50.324 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:50.356 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:50.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:50.357 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:50.453 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:50.457 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:50.501 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:50.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:50.502 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:50.603 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:50.652 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:50.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:50.654 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:50.662 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:11:50.755 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:50.801 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:50.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:50.802 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:50.859 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:11:50.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:50 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2315729403' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:50.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:50 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:50.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:50 smithi060 ceph-mon[29694]: pgmap v481: 158 pgs: 158 active+clean; 4.6 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 784 B/s rd, 627 KiB/s wr, 10 op/s 2024-09-20T22:11:50.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:50 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:11:50.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:50 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:11:50.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:50 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2315729403' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:50.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:50 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:50.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:50 smithi060 ceph-mon[40290]: pgmap v481: 158 pgs: 158 active+clean; 4.6 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 784 B/s rd, 627 KiB/s wr, 10 op/s 2024-09-20T22:11:50.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:50 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:11:50.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:50 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:11:50.867 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:11:50.888 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 3 16 9061 9045 188.425 175.438 0.00498964 0.00529809 2024-09-20T22:11:50.903 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:50.940 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:50.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:50.941 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:50 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2315729403' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:50 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:11:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:50 smithi067 ceph-mon[33511]: pgmap v481: 158 pgs: 158 active+clean; 4.6 MiB data, 3.1 GiB used, 712 GiB / 715 GiB avail; 784 B/s rd, 627 KiB/s wr, 10 op/s 2024-09-20T22:11:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:50 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:11:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:50 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:11:51.042 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:51.081 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:51.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:51.082 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:51.183 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:51.227 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:51.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:51.228 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:51.329 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:51.371 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:51.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:51.372 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:51.473 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:51.495 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:51.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:51.496 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:51.597 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:51.635 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:51.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:51.637 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:51.737 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:51.771 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:51.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:51.773 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:51.873 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:51.888 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 4 16 11752 11736 183.362 168.188 0.00395547 0.00544609 2024-09-20T22:11:51.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:51.909 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:51.909 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:51.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:11:52.010 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:51 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:11:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:51 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:11:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:51 smithi067 ceph-mon[33511]: osdmap e297: 8 total, 8 up, 8 in 2024-09-20T22:11:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:51 smithi067 ceph-mon[33511]: from='client.25327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:11:52.014 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:51 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:11:52.014 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:51 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:11:52.014 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:51 smithi060 ceph-mon[29694]: osdmap e297: 8 total, 8 up, 8 in 2024-09-20T22:11:52.014 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:51 smithi060 ceph-mon[29694]: from='client.25327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:11:52.014 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:51 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:11:52.014 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:51 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:11:52.014 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:51 smithi060 ceph-mon[40290]: osdmap e297: 8 total, 8 up, 8 in 2024-09-20T22:11:52.014 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:51 smithi060 ceph-mon[40290]: from='client.25327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:11:52.040 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:52.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:52.041 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:52.142 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:52.185 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:52.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:52.186 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:52.287 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:52.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:52.322 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:52.322 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:52.423 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:52.458 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:52.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:52.460 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:52.513 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:11:52 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:11:52.561 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:52.599 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:52.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:52.601 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:52.702 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:52.744 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:52.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:52.745 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:52.772 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:11:52.773 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:11:52.773 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-20T22:11:52.773 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:11:52.845 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:52.885 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:52.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:52.886 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:52.888 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 5 15 14403 14388 179.837 165.75 0.00473532 0.00555495 2024-09-20T22:11:52.986 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:52 smithi067 ceph-mon[33511]: pgmap v483: 158 pgs: 158 active+clean; 540 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.5 KiB/s rd, 83 MiB/s wr, 1.32k op/s 2024-09-20T22:11:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:52 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:11:53.027 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:53.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:53.028 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:53.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:52 smithi060 ceph-mon[29694]: pgmap v483: 158 pgs: 158 active+clean; 540 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.5 KiB/s rd, 83 MiB/s wr, 1.32k op/s 2024-09-20T22:11:53.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:52 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:11:53.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:52 smithi060 ceph-mon[40290]: pgmap v483: 158 pgs: 158 active+clean; 540 MiB data, 4.4 GiB used, 711 GiB / 715 GiB avail; 1.5 KiB/s rd, 83 MiB/s wr, 1.32k op/s 2024-09-20T22:11:53.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:52 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:11:53.130 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:53.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:53.166 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:53.166 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:53.267 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:53.307 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:53.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:53.308 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:53.408 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:53.442 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:53.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:53.443 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:53.493 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":482,"stamp":"2024-09-20T22:11:50.649183+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":4790811,"num_objects":263,"num_object_clones":0,"num_object_copies":789,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":263,"num_whiteouts":0,"num_read":1231,"num_read_kb":983,"num_write":500,"num_write_kb":4141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":921003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2008,"ondisk_log_size":2008,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":475,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":3301580,"kb_used_data":45140,"kb_used_omap":15,"kb_used_meta":3256240,"kb_avail":746659636,"statfs":{"total":767960285184,"available":764579467264,"internally_reserved":0,"allocated":46223360,"data_stored":41247769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":3334390547},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":64,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.518913"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":512,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.574451+0000","last_change":"2024-09-20T22:11:12.498436+0000","last_active":"2024-09-20T22:11:46.574451+0000","last_peered":"2024-09-20T22:11:46.574451+0000","last_clean":"2024-09-20T22:11:46.574451+0000","last_became_active":"2024-09-20T22:11:12.498141+0000","last_became_peered":"2024-09-20T22:11:12.498141+0000","last_unstale":"2024-09-20T22:11:46.574451+0000","last_undegraded":"2024-09-20T22:11:46.574451+0000","last_fullsized":"2024-09-20T22:11:46.574451+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:31.859259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_clean_scrub_stamp":"2024-09-20T22:10:31.859259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"252'13","reported_seq":523,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.603980+0000","last_change":"2024-09-20T22:10:31.568218+0000","last_active":"2024-09-20T22:11:43.603980+0000","last_peered":"2024-09-20T22:11:43.603980+0000","last_clean":"2024-09-20T22:11:43.603980+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:11:43.603980+0000","last_undegraded":"2024-09-20T22:11:43.603980+0000","last_fullsized":"2024-09-20T22:11:43.603980+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"252'13","last_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_deep_scrub":"252'13","last_deep_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_clean_scrub_stamp":"2024-09-20T22:10:31.568098+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":380,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.615974+0000","last_change":"2024-09-20T22:10:22.311182+0000","last_active":"2024-09-20T22:11:45.615974+0000","last_peered":"2024-09-20T22:11:45.615974+0000","last_clean":"2024-09-20T22:11:45.615974+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:11:45.615974+0000","last_undegraded":"2024-09-20T22:11:45.615974+0000","last_fullsized":"2024-09-20T22:11:45.615974+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:22.311078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:22.311078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:51.019290+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":499,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.604206+0000","last_change":"2024-09-20T22:11:30.486243+0000","last_active":"2024-09-20T22:11:43.604206+0000","last_peered":"2024-09-20T22:11:43.604206+0000","last_clean":"2024-09-20T22:11:43.604206+0000","last_became_active":"2024-09-20T22:11:30.485735+0000","last_became_peered":"2024-09-20T22:11:30.485735+0000","last_unstale":"2024-09-20T22:11:43.604206+0000","last_undegraded":"2024-09-20T22:11:43.604206+0000","last_fullsized":"2024-09-20T22:11:43.604206+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:23.640516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:23.640516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":480,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175267+0000","last_change":"2024-09-20T22:08:41.770593+0000","last_active":"2024-09-20T22:11:46.175267+0000","last_peered":"2024-09-20T22:11:46.175267+0000","last_clean":"2024-09-20T22:11:46.175267+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:11:46.175267+0000","last_undegraded":"2024-09-20T22:11:46.175267+0000","last_fullsized":"2024-09-20T22:11:46.175267+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:08:41.770351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:41.770351+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":500,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175868+0000","last_change":"2024-09-20T22:11:42.438241+0000","last_active":"2024-09-20T22:11:46.175868+0000","last_peered":"2024-09-20T22:11:46.175868+0000","last_clean":"2024-09-20T22:11:46.175868+0000","last_became_active":"2024-09-20T22:02:50.107154+0000","last_became_peered":"2024-09-20T22:02:50.107154+0000","last_unstale":"2024-09-20T22:11:46.175868+0000","last_undegraded":"2024-09-20T22:11:46.175868+0000","last_fullsized":"2024-09-20T22:11:46.175868+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:42.438172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:11:42.438172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:57.591083+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"71'15","reported_seq":512,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175389+0000","last_change":"2024-09-20T22:08:39.705641+0000","last_active":"2024-09-20T22:11:46.175389+0000","last_peered":"2024-09-20T22:11:46.175389+0000","last_clean":"2024-09-20T22:11:46.175389+0000","last_became_active":"2024-09-20T22:02:52.282101+0000","last_became_peered":"2024-09-20T22:02:52.282101+0000","last_unstale":"2024-09-20T22:11:46.175389+0000","last_undegraded":"2024-09-20T22:11:46.175389+0000","last_fullsized":"2024-09-20T22:11:46.175389+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:08:39.705510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:39.705510+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":491,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.603964+0000","last_change":"2024-09-20T22:10:26.558035+0000","last_active":"2024-09-20T22:11:43.603964+0000","last_peered":"2024-09-20T22:11:43.603964+0000","last_clean":"2024-09-20T22:11:43.603964+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:11:43.603964+0000","last_undegraded":"2024-09-20T22:11:43.603964+0000","last_fullsized":"2024-09-20T22:11:43.603964+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:26.557925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:26.557925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":515,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.618367+0000","last_change":"2024-09-20T22:10:35.191206+0000","last_active":"2024-09-20T22:11:45.618367+0000","last_peered":"2024-09-20T22:11:45.618367+0000","last_clean":"2024-09-20T22:11:45.618367+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:11:45.618367+0000","last_undegraded":"2024-09-20T22:11:45.618367+0000","last_fullsized":"2024-09-20T22:11:45.618367+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:35.191108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:10:35.191108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:46.313768+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"71'9","reported_seq":510,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.604433+0000","last_change":"2024-09-20T22:09:47.733504+0000","last_active":"2024-09-20T22:11:43.604433+0000","last_peered":"2024-09-20T22:11:43.604433+0000","last_clean":"2024-09-20T22:11:43.604433+0000","last_became_active":"2024-09-20T22:02:52.279526+0000","last_became_peered":"2024-09-20T22:02:52.279526+0000","last_unstale":"2024-09-20T22:11:43.604433+0000","last_undegraded":"2024-09-20T22:11:43.604433+0000","last_fullsized":"2024-09-20T22:11:43.604433+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:09:47.733368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:09:47.733368+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":43,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.617440+0000","last_change":"2024-09-20T22:11:31.101654+0000","last_active":"2024-09-20T22:11:45.617440+0000","last_peered":"2024-09-20T22:11:45.617440+0000","last_clean":"2024-09-20T22:11:45.617440+0000","last_became_active":"2024-09-20T22:11:30.485036+0000","last_became_peered":"2024-09-20T22:11:30.485036+0000","last_unstale":"2024-09-20T22:11:45.617440+0000","last_undegraded":"2024-09-20T22:11:45.617440+0000","last_fullsized":"2024-09-20T22:11:45.617440+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:31.101573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:31.101573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:32.848267+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":488,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.573484+0000","last_change":"2024-09-20T22:10:32.855540+0000","last_active":"2024-09-20T22:11:46.573484+0000","last_peered":"2024-09-20T22:11:46.573484+0000","last_clean":"2024-09-20T22:11:46.573484+0000","last_became_active":"2024-09-20T22:06:54.841095+0000","last_became_peered":"2024-09-20T22:06:54.841095+0000","last_unstale":"2024-09-20T22:11:46.573484+0000","last_undegraded":"2024-09-20T22:11:46.573484+0000","last_fullsized":"2024-09-20T22:11:46.573484+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:32.855440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:32.855440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":528,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.617724+0000","last_change":"2024-09-20T22:11:40.127688+0000","last_active":"2024-09-20T22:11:45.617724+0000","last_peered":"2024-09-20T22:11:45.617724+0000","last_clean":"2024-09-20T22:11:45.617724+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:11:45.617724+0000","last_undegraded":"2024-09-20T22:11:45.617724+0000","last_fullsized":"2024-09-20T22:11:45.617724+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:40.127597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:30.215263+0000","last_clean_scrub_stamp":"2024-09-20T22:11:40.127597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:59.820229+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"71'5","reported_seq":525,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.604320+0000","last_change":"2024-09-20T22:11:03.163814+0000","last_active":"2024-09-20T22:11:43.604320+0000","last_peered":"2024-09-20T22:11:43.604320+0000","last_clean":"2024-09-20T22:11:43.604320+0000","last_became_active":"2024-09-20T22:08:22.144364+0000","last_became_peered":"2024-09-20T22:08:22.144364+0000","last_unstale":"2024-09-20T22:11:43.604320+0000","last_undegraded":"2024-09-20T22:11:43.604320+0000","last_fullsized":"2024-09-20T22:11:43.604320+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'5","last_scrub_stamp":"2024-09-20T22:11:03.163739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:03.163739+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:25.185084+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":499,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.606620+0000","last_change":"2024-09-20T22:10:15.024751+0000","last_active":"2024-09-20T22:11:43.606620+0000","last_peered":"2024-09-20T22:11:43.606620+0000","last_clean":"2024-09-20T22:11:43.606620+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:11:43.606620+0000","last_undegraded":"2024-09-20T22:11:43.606620+0000","last_fullsized":"2024-09-20T22:11:43.606620+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:15.024624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:15.024624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:48.339273+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":43,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.617328+0000","last_change":"2024-09-20T22:11:34.036027+0000","last_active":"2024-09-20T22:11:45.617328+0000","last_peered":"2024-09-20T22:11:45.617328+0000","last_clean":"2024-09-20T22:11:45.617328+0000","last_became_active":"2024-09-20T22:11:30.487137+0000","last_became_peered":"2024-09-20T22:11:30.487137+0000","last_unstale":"2024-09-20T22:11:45.617328+0000","last_undegraded":"2024-09-20T22:11:45.617328+0000","last_fullsized":"2024-09-20T22:11:45.617328+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:34.035938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.035938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:57.022848+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":487,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.605006+0000","last_change":"2024-09-20T22:09:51.759271+0000","last_active":"2024-09-20T22:11:43.605006+0000","last_peered":"2024-09-20T22:11:43.605006+0000","last_clean":"2024-09-20T22:11:43.605006+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:11:43.605006+0000","last_undegraded":"2024-09-20T22:11:43.605006+0000","last_fullsized":"2024-09-20T22:11:43.605006+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:51.759042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:09:51.759042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":499,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.573216+0000","last_change":"2024-09-20T22:11:36.700618+0000","last_active":"2024-09-20T22:11:46.573216+0000","last_peered":"2024-09-20T22:11:46.573216+0000","last_clean":"2024-09-20T22:11:46.573216+0000","last_became_active":"2024-09-20T22:02:50.111517+0000","last_became_peered":"2024-09-20T22:02:50.111517+0000","last_unstale":"2024-09-20T22:11:46.573216+0000","last_undegraded":"2024-09-20T22:11:46.573216+0000","last_fullsized":"2024-09-20T22:11:46.573216+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:36.700507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:11:36.700507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:44.350271+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"195'22","reported_seq":65,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.614641+0000","last_change":"2024-09-20T22:11:12.545971+0000","last_active":"2024-09-20T22:11:45.614641+0000","last_peered":"2024-09-20T22:11:45.614641+0000","last_clean":"2024-09-20T22:11:45.614641+0000","last_became_active":"2024-09-20T22:11:12.507311+0000","last_became_peered":"2024-09-20T22:11:12.507311+0000","last_unstale":"2024-09-20T22:11:45.614641+0000","last_undegraded":"2024-09-20T22:11:45.614641+0000","last_fullsized":"2024-09-20T22:11:45.614641+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'22","last_scrub_stamp":"2024-09-20T22:10:16.538107+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:10:16.538107+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:48.289469+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":337,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.616703+0000","last_change":"2024-09-20T22:11:09.157162+0000","last_active":"2024-09-20T22:11:45.616703+0000","last_peered":"2024-09-20T22:11:45.616703+0000","last_clean":"2024-09-20T22:11:45.616703+0000","last_became_active":"2024-09-20T22:06:54.848505+0000","last_became_peered":"2024-09-20T22:06:54.848505+0000","last_unstale":"2024-09-20T22:11:45.616703+0000","last_undegraded":"2024-09-20T22:11:45.616703+0000","last_fullsized":"2024-09-20T22:11:45.616703+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:09.157055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:09.157055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:24.968380+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":500,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175877+0000","last_change":"2024-09-20T22:11:30.487554+0000","last_active":"2024-09-20T22:11:46.175877+0000","last_peered":"2024-09-20T22:11:46.175877+0000","last_clean":"2024-09-20T22:11:46.175877+0000","last_became_active":"2024-09-20T22:11:30.487199+0000","last_became_peered":"2024-09-20T22:11:30.487199+0000","last_unstale":"2024-09-20T22:11:46.175877+0000","last_undegraded":"2024-09-20T22:11:46.175877+0000","last_fullsized":"2024-09-20T22:11:46.175877+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:52.317931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:52.317931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":449,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.617713+0000","last_change":"2024-09-20T22:10:59.182372+0000","last_active":"2024-09-20T22:11:45.617713+0000","last_peered":"2024-09-20T22:11:45.617713+0000","last_clean":"2024-09-20T22:11:45.617713+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:11:45.617713+0000","last_undegraded":"2024-09-20T22:11:45.617713+0000","last_fullsized":"2024-09-20T22:11:45.617713+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:59.182274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:59.182274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:07.198779+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"250'17","reported_seq":524,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175515+0000","last_change":"2024-09-20T22:10:27.157652+0000","last_active":"2024-09-20T22:11:46.175515+0000","last_peered":"2024-09-20T22:11:46.175515+0000","last_clean":"2024-09-20T22:11:46.175515+0000","last_became_active":"2024-09-20T22:06:54.830946+0000","last_became_peered":"2024-09-20T22:06:54.830946+0000","last_unstale":"2024-09-20T22:11:46.175515+0000","last_undegraded":"2024-09-20T22:11:46.175515+0000","last_fullsized":"2024-09-20T22:11:46.175515+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"250'17","last_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_deep_scrub":"250'17","last_deep_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_clean_scrub_stamp":"2024-09-20T22:10:27.157501+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":490,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.604329+0000","last_change":"2024-09-20T22:11:34.369112+0000","last_active":"2024-09-20T22:11:43.604329+0000","last_peered":"2024-09-20T22:11:43.604329+0000","last_clean":"2024-09-20T22:11:43.604329+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:11:43.604329+0000","last_undegraded":"2024-09-20T22:11:43.604329+0000","last_fullsized":"2024-09-20T22:11:43.604329+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:34.369006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.369006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:49.101366+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":488,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.606195+0000","last_change":"2024-09-20T22:11:39.286967+0000","last_active":"2024-09-20T22:11:43.606195+0000","last_peered":"2024-09-20T22:11:43.606195+0000","last_clean":"2024-09-20T22:11:43.606195+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:11:43.606195+0000","last_undegraded":"2024-09-20T22:11:43.606195+0000","last_fullsized":"2024-09-20T22:11:43.606195+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:11:39.286813+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:11:39.286813+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:43.641876+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"289'2","reported_seq":340,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.616064+0000","last_change":"2024-09-20T22:11:37.189066+0000","last_active":"2024-09-20T22:11:45.616064+0000","last_peered":"2024-09-20T22:11:45.616064+0000","last_clean":"2024-09-20T22:11:45.616064+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:11:45.616064+0000","last_undegraded":"2024-09-20T22:11:45.616064+0000","last_fullsized":"2024-09-20T22:11:45.616064+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_clean_scrub_stamp":"2024-09-20T22:11:37.188992+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:55.054428+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"71'10","reported_seq":546,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.574366+0000","last_change":"2024-09-20T22:11:40.608961+0000","last_active":"2024-09-20T22:11:46.574366+0000","last_peered":"2024-09-20T22:11:46.574366+0000","last_clean":"2024-09-20T22:11:46.574366+0000","last_became_active":"2024-09-20T22:11:40.602075+0000","last_became_peered":"2024-09-20T22:11:40.602075+0000","last_unstale":"2024-09-20T22:11:46.574366+0000","last_undegraded":"2024-09-20T22:11:46.574366+0000","last_fullsized":"2024-09-20T22:11:46.574366+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:11:35.700046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:35.700046+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:03.129277+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"291'11","reported_seq":646,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.636600+0000","last_change":"2024-09-20T22:11:41.372331+0000","last_active":"2024-09-20T22:11:43.636600+0000","last_peered":"2024-09-20T22:11:43.636600+0000","last_clean":"2024-09-20T22:11:43.636600+0000","last_became_active":"2024-09-20T22:11:40.601048+0000","last_became_peered":"2024-09-20T22:11:40.601048+0000","last_unstale":"2024-09-20T22:11:43.636600+0000","last_undegraded":"2024-09-20T22:11:43.636600+0000","last_fullsized":"2024-09-20T22:11:43.636600+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'11","last_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_deep_scrub":"291'11","last_deep_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_clean_scrub_stamp":"2024-09-20T22:11:41.372195+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:10.847568+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":28,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175309+0000","last_change":"2024-09-20T22:11:35.304533+0000","last_active":"2024-09-20T22:11:46.175309+0000","last_peered":"2024-09-20T22:11:46.175309+0000","last_clean":"2024-09-20T22:11:46.175309+0000","last_became_active":"2024-09-20T22:11:35.304245+0000","last_became_peered":"2024-09-20T22:11:35.304245+0000","last_unstale":"2024-09-20T22:11:46.175309+0000","last_undegraded":"2024-09-20T22:11:46.175309+0000","last_fullsized":"2024-09-20T22:11:46.175309+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:54.197789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":503,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.617487+0000","last_change":"2024-09-20T22:11:08.187529+0000","last_active":"2024-09-20T22:11:45.617487+0000","last_peered":"2024-09-20T22:11:45.617487+0000","last_clean":"2024-09-20T22:11:45.617487+0000","last_became_active":"2024-09-20T22:08:48.452786+0000","last_became_peered":"2024-09-20T22:08:48.452786+0000","last_unstale":"2024-09-20T22:11:45.617487+0000","last_undegraded":"2024-09-20T22:11:45.617487+0000","last_fullsized":"2024-09-20T22:11:45.617487+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:11:08.187420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:11:08.187420+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:21.119364+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"71'15","reported_seq":266,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.615139+0000","last_change":"2024-09-20T22:11:34.283966+0000","last_active":"2024-09-20T22:11:45.615139+0000","last_peered":"2024-09-20T22:11:45.615139+0000","last_clean":"2024-09-20T22:11:45.615139+0000","last_became_active":"2024-09-20T22:08:22.145408+0000","last_became_peered":"2024-09-20T22:08:22.145408+0000","last_unstale":"2024-09-20T22:11:45.615139+0000","last_undegraded":"2024-09-20T22:11:45.615139+0000","last_fullsized":"2024-09-20T22:11:45.615139+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:11:34.283882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.283882+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:43.345533+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":501,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175156+0000","last_change":"2024-09-20T22:10:19.208598+0000","last_active":"2024-09-20T22:11:46.175156+0000","last_peered":"2024-09-20T22:11:46.175156+0000","last_clean":"2024-09-20T22:11:46.175156+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:11:46.175156+0000","last_undegraded":"2024-09-20T22:11:46.175156+0000","last_fullsized":"2024-09-20T22:11:46.175156+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:19.208496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:19.208496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"283'11","reported_seq":647,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.605167+0000","last_change":"2024-09-20T22:11:30.493908+0000","last_active":"2024-09-20T22:11:43.605167+0000","last_peered":"2024-09-20T22:11:43.605167+0000","last_clean":"2024-09-20T22:11:43.605167+0000","last_became_active":"2024-09-20T22:11:30.484743+0000","last_became_peered":"2024-09-20T22:11:30.484743+0000","last_unstale":"2024-09-20T22:11:43.605167+0000","last_undegraded":"2024-09-20T22:11:43.605167+0000","last_fullsized":"2024-09-20T22:11:43.605167+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:10:14.680845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:14.680845+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":35,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.617993+0000","last_change":"2024-09-20T22:11:30.490404+0000","last_active":"2024-09-20T22:11:45.617993+0000","last_peered":"2024-09-20T22:11:45.617993+0000","last_clean":"2024-09-20T22:11:45.617993+0000","last_became_active":"2024-09-20T22:11:30.489416+0000","last_became_peered":"2024-09-20T22:11:30.489416+0000","last_unstale":"2024-09-20T22:11:45.617993+0000","last_undegraded":"2024-09-20T22:11:45.617993+0000","last_fullsized":"2024-09-20T22:11:45.617993+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:19.235368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:19.235368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:50.149182+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":25,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.606387+0000","last_change":"2024-09-20T22:11:34.561887+0000","last_active":"2024-09-20T22:11:43.606387+0000","last_peered":"2024-09-20T22:11:43.606387+0000","last_clean":"2024-09-20T22:11:43.606387+0000","last_became_active":"2024-09-20T22:11:34.561538+0000","last_became_peered":"2024-09-20T22:11:34.561538+0000","last_unstale":"2024-09-20T22:11:43.606387+0000","last_undegraded":"2024-09-20T22:11:43.606387+0000","last_fullsized":"2024-09-20T22:11:43.606387+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:56.980684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":493,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.604845+0000","last_change":"2024-09-20T22:08:59.563006+0000","last_active":"2024-09-20T22:11:43.604845+0000","last_peered":"2024-09-20T22:11:43.604845+0000","last_clean":"2024-09-20T22:11:43.604845+0000","last_became_active":"2024-09-20T22:08:48.452475+0000","last_became_peered":"2024-09-20T22:08:48.452475+0000","last_unstale":"2024-09-20T22:11:43.604845+0000","last_undegraded":"2024-09-20T22:11:43.604845+0000","last_fullsized":"2024-09-20T22:11:43.604845+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:08:59.562897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:08:59.562897+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"71'18","reported_seq":575,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.174960+0000","last_change":"2024-09-20T22:11:30.218845+0000","last_active":"2024-09-20T22:11:46.174960+0000","last_peered":"2024-09-20T22:11:46.174960+0000","last_clean":"2024-09-20T22:11:46.174960+0000","last_became_active":"2024-09-20T22:10:48.921997+0000","last_became_peered":"2024-09-20T22:10:48.921997+0000","last_unstale":"2024-09-20T22:11:46.174960+0000","last_undegraded":"2024-09-20T22:11:46.174960+0000","last_fullsized":"2024-09-20T22:11:46.174960+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'18","last_scrub_stamp":"2024-09-20T22:11:30.218749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:30.218749+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:45.626566+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":491,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.176090+0000","last_change":"2024-09-20T22:09:01.889907+0000","last_active":"2024-09-20T22:11:46.176090+0000","last_peered":"2024-09-20T22:11:46.176090+0000","last_clean":"2024-09-20T22:11:46.176090+0000","last_became_active":"2024-09-20T22:02:50.113076+0000","last_became_peered":"2024-09-20T22:02:50.113076+0000","last_unstale":"2024-09-20T22:11:46.176090+0000","last_undegraded":"2024-09-20T22:11:46.176090+0000","last_fullsized":"2024-09-20T22:11:46.176090+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:01.889795+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:01.889795+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":503,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.604053+0000","last_change":"2024-09-20T22:10:48.920310+0000","last_active":"2024-09-20T22:11:43.604053+0000","last_peered":"2024-09-20T22:11:43.604053+0000","last_clean":"2024-09-20T22:11:43.604053+0000","last_became_active":"2024-09-20T22:10:48.919949+0000","last_became_peered":"2024-09-20T22:10:48.919949+0000","last_unstale":"2024-09-20T22:11:43.604053+0000","last_undegraded":"2024-09-20T22:11:43.604053+0000","last_fullsized":"2024-09-20T22:11:43.604053+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:05.535412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:05.535412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":495,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.604344+0000","last_change":"2024-09-20T22:11:40.260396+0000","last_active":"2024-09-20T22:11:43.604344+0000","last_peered":"2024-09-20T22:11:43.604344+0000","last_clean":"2024-09-20T22:11:43.604344+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:11:43.604344+0000","last_undegraded":"2024-09-20T22:11:43.604344+0000","last_fullsized":"2024-09-20T22:11:43.604344+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:40.260289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:40.260289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:49.182120+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":28,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175453+0000","last_change":"2024-09-20T22:11:34.559451+0000","last_active":"2024-09-20T22:11:46.175453+0000","last_peered":"2024-09-20T22:11:46.175453+0000","last_clean":"2024-09-20T22:11:46.175453+0000","last_became_active":"2024-09-20T22:11:34.558615+0000","last_became_peered":"2024-09-20T22:11:34.558615+0000","last_unstale":"2024-09-20T22:11:46.175453+0000","last_undegraded":"2024-09-20T22:11:46.175453+0000","last_fullsized":"2024-09-20T22:11:46.175453+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:43.687599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":529,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175262+0000","last_change":"2024-09-20T22:11:39.422439+0000","last_active":"2024-09-20T22:11:46.175262+0000","last_peered":"2024-09-20T22:11:46.175262+0000","last_clean":"2024-09-20T22:11:46.175262+0000","last_became_active":"2024-09-20T22:11:33.560632+0000","last_became_peered":"2024-09-20T22:11:33.560632+0000","last_unstale":"2024-09-20T22:11:46.175262+0000","last_undegraded":"2024-09-20T22:11:46.175262+0000","last_fullsized":"2024-09-20T22:11:46.175262+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:11:39.422355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:11:39.422355+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:08.591531+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"71'14","reported_seq":516,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.604174+0000","last_change":"2024-09-20T22:09:03.569008+0000","last_active":"2024-09-20T22:11:43.604174+0000","last_peered":"2024-09-20T22:11:43.604174+0000","last_clean":"2024-09-20T22:11:43.604174+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:11:43.604174+0000","last_undegraded":"2024-09-20T22:11:43.604174+0000","last_fullsized":"2024-09-20T22:11:43.604174+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'14","last_scrub_stamp":"2024-09-20T22:09:03.568900+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:09:03.568900+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":502,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.574552+0000","last_change":"2024-09-20T22:11:04.592252+0000","last_active":"2024-09-20T22:11:46.574552+0000","last_peered":"2024-09-20T22:11:46.574552+0000","last_clean":"2024-09-20T22:11:46.574552+0000","last_became_active":"2024-09-20T22:06:54.841165+0000","last_became_peered":"2024-09-20T22:06:54.841165+0000","last_unstale":"2024-09-20T22:11:46.574552+0000","last_undegraded":"2024-09-20T22:11:46.574552+0000","last_fullsized":"2024-09-20T22:11:46.574552+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:11:04.590441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:11:04.590441+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:16.119865+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"146'10","reported_seq":611,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.174877+0000","last_change":"2024-09-20T22:10:23.115374+0000","last_active":"2024-09-20T22:11:46.174877+0000","last_peered":"2024-09-20T22:11:46.174877+0000","last_clean":"2024-09-20T22:11:46.174877+0000","last_became_active":"2024-09-20T22:06:54.847742+0000","last_became_peered":"2024-09-20T22:06:54.847742+0000","last_unstale":"2024-09-20T22:11:46.174877+0000","last_undegraded":"2024-09-20T22:11:46.174877+0000","last_fullsized":"2024-09-20T22:11:46.174877+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:10:23.114986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:23.114986+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:50.179566+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":487,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.606183+0000","last_change":"2024-09-20T22:10:22.075441+0000","last_active":"2024-09-20T22:11:43.606183+0000","last_peered":"2024-09-20T22:11:43.606183+0000","last_clean":"2024-09-20T22:11:43.606183+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:11:43.606183+0000","last_undegraded":"2024-09-20T22:11:43.606183+0000","last_fullsized":"2024-09-20T22:11:43.606183+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:22.075335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:22.075335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:45.954793+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":28,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.618151+0000","last_change":"2024-09-20T22:11:34.559393+0000","last_active":"2024-09-20T22:11:45.618151+0000","last_peered":"2024-09-20T22:11:45.618151+0000","last_clean":"2024-09-20T22:11:45.618151+0000","last_became_active":"2024-09-20T22:11:34.558898+0000","last_became_peered":"2024-09-20T22:11:34.558898+0000","last_unstale":"2024-09-20T22:11:45.618151+0000","last_undegraded":"2024-09-20T22:11:45.618151+0000","last_fullsized":"2024-09-20T22:11:45.618151+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:41.066557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":523,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.636329+0000","last_change":"2024-09-20T22:11:07.362346+0000","last_active":"2024-09-20T22:11:43.636329+0000","last_peered":"2024-09-20T22:11:43.636329+0000","last_clean":"2024-09-20T22:11:43.636329+0000","last_became_active":"2024-09-20T22:10:33.660948+0000","last_became_peered":"2024-09-20T22:10:33.660948+0000","last_unstale":"2024-09-20T22:11:43.636329+0000","last_undegraded":"2024-09-20T22:11:43.636329+0000","last_fullsized":"2024-09-20T22:11:43.636329+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:11:07.362277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:11:07.362277+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:23.879195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"71'10","reported_seq":400,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.616470+0000","last_change":"2024-09-20T22:11:31.064801+0000","last_active":"2024-09-20T22:11:45.616470+0000","last_peered":"2024-09-20T22:11:45.616470+0000","last_clean":"2024-09-20T22:11:45.616470+0000","last_became_active":"2024-09-20T22:11:30.486532+0000","last_became_peered":"2024-09-20T22:11:30.486532+0000","last_unstale":"2024-09-20T22:11:45.616470+0000","last_undegraded":"2024-09-20T22:11:45.616470+0000","last_fullsized":"2024-09-20T22:11:45.616470+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:11:31.064669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:31.064669+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:35.046612+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":524,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.618463+0000","last_change":"2024-09-20T22:11:42.173417+0000","last_active":"2024-09-20T22:11:45.618463+0000","last_peered":"2024-09-20T22:11:45.618463+0000","last_clean":"2024-09-20T22:11:45.618463+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:11:45.618463+0000","last_undegraded":"2024-09-20T22:11:45.618463+0000","last_fullsized":"2024-09-20T22:11:45.618463+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:42.173334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:11:42.173334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:44.551619+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"283'10","reported_seq":620,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.573340+0000","last_change":"2024-09-20T22:11:30.496915+0000","last_active":"2024-09-20T22:11:46.573340+0000","last_peered":"2024-09-20T22:11:46.573340+0000","last_clean":"2024-09-20T22:11:46.573340+0000","last_became_active":"2024-09-20T22:11:30.486567+0000","last_became_peered":"2024-09-20T22:11:30.486567+0000","last_unstale":"2024-09-20T22:11:46.573340+0000","last_undegraded":"2024-09-20T22:11:46.573340+0000","last_fullsized":"2024-09-20T22:11:46.573340+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:08:28.860982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:28.860982+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":480,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.573571+0000","last_change":"2024-09-20T22:10:24.053258+0000","last_active":"2024-09-20T22:11:46.573571+0000","last_peered":"2024-09-20T22:11:46.573571+0000","last_clean":"2024-09-20T22:11:46.573571+0000","last_became_active":"2024-09-20T22:02:56.304274+0000","last_became_peered":"2024-09-20T22:02:56.304274+0000","last_unstale":"2024-09-20T22:11:46.573571+0000","last_undegraded":"2024-09-20T22:11:46.573571+0000","last_fullsized":"2024-09-20T22:11:46.573571+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:24.053159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:24.053159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":25,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.605920+0000","last_change":"2024-09-20T22:11:34.560812+0000","last_active":"2024-09-20T22:11:43.605920+0000","last_peered":"2024-09-20T22:11:43.605920+0000","last_clean":"2024-09-20T22:11:43.605920+0000","last_became_active":"2024-09-20T22:11:34.560514+0000","last_became_peered":"2024-09-20T22:11:34.560514+0000","last_unstale":"2024-09-20T22:11:43.605920+0000","last_undegraded":"2024-09-20T22:11:43.605920+0000","last_fullsized":"2024-09-20T22:11:43.605920+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:36.465015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":506,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.636029+0000","last_change":"2024-09-20T22:11:36.339782+0000","last_active":"2024-09-20T22:11:43.636029+0000","last_peered":"2024-09-20T22:11:43.636029+0000","last_clean":"2024-09-20T22:11:43.636029+0000","last_became_active":"2024-09-20T22:08:48.466328+0000","last_became_peered":"2024-09-20T22:08:48.466328+0000","last_unstale":"2024-09-20T22:11:43.636029+0000","last_undegraded":"2024-09-20T22:11:43.636029+0000","last_fullsized":"2024-09-20T22:11:43.636029+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:11:36.339687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:11:36.339687+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:56.138458+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"71'19","reported_seq":532,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.574470+0000","last_change":"2024-09-20T22:08:33.852966+0000","last_active":"2024-09-20T22:11:46.574470+0000","last_peered":"2024-09-20T22:11:46.574470+0000","last_clean":"2024-09-20T22:11:46.574470+0000","last_became_active":"2024-09-20T22:02:52.480077+0000","last_became_peered":"2024-09-20T22:02:52.480077+0000","last_unstale":"2024-09-20T22:11:46.574470+0000","last_undegraded":"2024-09-20T22:11:46.574470+0000","last_fullsized":"2024-09-20T22:11:46.574470+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:08:33.852806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:33.852806+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":351,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.617175+0000","last_change":"2024-09-20T22:11:30.486764+0000","last_active":"2024-09-20T22:11:45.617175+0000","last_peered":"2024-09-20T22:11:45.617175+0000","last_clean":"2024-09-20T22:11:45.617175+0000","last_became_active":"2024-09-20T22:11:30.486428+0000","last_became_peered":"2024-09-20T22:11:30.486428+0000","last_unstale":"2024-09-20T22:11:45.617175+0000","last_undegraded":"2024-09-20T22:11:45.617175+0000","last_fullsized":"2024-09-20T22:11:45.617175+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:59.221160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:59.221160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:17.826923+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":506,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.636771+0000","last_change":"2024-09-20T22:11:00.280237+0000","last_active":"2024-09-20T22:11:43.636771+0000","last_peered":"2024-09-20T22:11:43.636771+0000","last_clean":"2024-09-20T22:11:43.636771+0000","last_became_active":"2024-09-20T22:02:54.290787+0000","last_became_peered":"2024-09-20T22:02:54.290787+0000","last_unstale":"2024-09-20T22:11:43.636771+0000","last_undegraded":"2024-09-20T22:11:43.636771+0000","last_fullsized":"2024-09-20T22:11:43.636771+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:00.280165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:11:00.280165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:08.873300+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":337,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.615811+0000","last_change":"2024-09-20T22:11:39.264738+0000","last_active":"2024-09-20T22:11:45.615811+0000","last_peered":"2024-09-20T22:11:45.615811+0000","last_clean":"2024-09-20T22:11:45.615811+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:11:45.615811+0000","last_undegraded":"2024-09-20T22:11:45.615811+0000","last_fullsized":"2024-09-20T22:11:45.615811+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:39.264610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:39.264610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:55.038507+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":28,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.573680+0000","last_change":"2024-09-20T22:11:34.561687+0000","last_active":"2024-09-20T22:11:46.573680+0000","last_peered":"2024-09-20T22:11:46.573680+0000","last_clean":"2024-09-20T22:11:46.573680+0000","last_became_active":"2024-09-20T22:11:34.560531+0000","last_became_peered":"2024-09-20T22:11:34.560531+0000","last_unstale":"2024-09-20T22:11:46.573680+0000","last_undegraded":"2024-09-20T22:11:46.573680+0000","last_fullsized":"2024-09-20T22:11:46.573680+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:47.491851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":503,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.617263+0000","last_change":"2024-09-20T22:10:56.231802+0000","last_active":"2024-09-20T22:11:45.617263+0000","last_peered":"2024-09-20T22:11:45.617263+0000","last_clean":"2024-09-20T22:11:45.617263+0000","last_became_active":"2024-09-20T22:08:48.452234+0000","last_became_peered":"2024-09-20T22:08:48.452234+0000","last_unstale":"2024-09-20T22:11:45.617263+0000","last_undegraded":"2024-09-20T22:11:45.617263+0000","last_fullsized":"2024-09-20T22:11:45.617263+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:10:56.231679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:10:56.231679+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:02.751759+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"271'40","reported_seq":469,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.616595+0000","last_change":"2024-09-20T22:11:05.235955+0000","last_active":"2024-09-20T22:11:45.616595+0000","last_peered":"2024-09-20T22:11:45.616595+0000","last_clean":"2024-09-20T22:11:45.616595+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:11:45.616595+0000","last_undegraded":"2024-09-20T22:11:45.616595+0000","last_fullsized":"2024-09-20T22:11:45.616595+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"271'40","last_scrub_stamp":"2024-09-20T22:11:05.235779+0000","last_deep_scrub":"271'40","last_deep_scrub_stamp":"2024-09-20T22:11:05.235779+0000","last_clean_scrub_stamp":"2024-09-20T22:11:05.235779+0000","objects_scrubbed":10,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:31.478891+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":499,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.604609+0000","last_change":"2024-09-20T22:09:02.547888+0000","last_active":"2024-09-20T22:11:43.604609+0000","last_peered":"2024-09-20T22:11:43.604609+0000","last_clean":"2024-09-20T22:11:43.604609+0000","last_became_active":"2024-09-20T22:02:50.108457+0000","last_became_peered":"2024-09-20T22:02:50.108457+0000","last_unstale":"2024-09-20T22:11:43.604609+0000","last_undegraded":"2024-09-20T22:11:43.604609+0000","last_fullsized":"2024-09-20T22:11:43.604609+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:02.547780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:02.547780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":520,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.617833+0000","last_change":"2024-09-20T22:11:38.145140+0000","last_active":"2024-09-20T22:11:45.617833+0000","last_peered":"2024-09-20T22:11:45.617833+0000","last_clean":"2024-09-20T22:11:45.617833+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:11:45.617833+0000","last_undegraded":"2024-09-20T22:11:45.617833+0000","last_fullsized":"2024-09-20T22:11:45.617833+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:11:38.145006+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_clean_scrub_stamp":"2024-09-20T22:11:38.145006+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:42.494263+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":493,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.574132+0000","last_change":"2024-09-20T22:10:33.832638+0000","last_active":"2024-09-20T22:11:46.574132+0000","last_peered":"2024-09-20T22:11:46.574132+0000","last_clean":"2024-09-20T22:11:46.574132+0000","last_became_active":"2024-09-20T22:06:54.841419+0000","last_became_peered":"2024-09-20T22:06:54.841419+0000","last_unstale":"2024-09-20T22:11:46.574132+0000","last_undegraded":"2024-09-20T22:11:46.574132+0000","last_fullsized":"2024-09-20T22:11:46.574132+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:33.832519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:33.832519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:58.619938+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":480,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175685+0000","last_change":"2024-09-20T22:09:05.892350+0000","last_active":"2024-09-20T22:11:46.175685+0000","last_peered":"2024-09-20T22:11:46.175685+0000","last_clean":"2024-09-20T22:11:46.175685+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:11:46.175685+0000","last_undegraded":"2024-09-20T22:11:46.175685+0000","last_fullsized":"2024-09-20T22:11:46.175685+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:05.892235+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:09:05.892235+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":28,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.617203+0000","last_change":"2024-09-20T22:11:34.559139+0000","last_active":"2024-09-20T22:11:45.617203+0000","last_peered":"2024-09-20T22:11:45.617203+0000","last_clean":"2024-09-20T22:11:45.617203+0000","last_became_active":"2024-09-20T22:11:34.558703+0000","last_became_peered":"2024-09-20T22:11:34.558703+0000","last_unstale":"2024-09-20T22:11:45.617203+0000","last_undegraded":"2024-09-20T22:11:45.617203+0000","last_fullsized":"2024-09-20T22:11:45.617203+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:38.441804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":491,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.609213+0000","last_change":"2024-09-20T22:11:43.608886+0000","last_active":"2024-09-20T22:11:43.609213+0000","last_peered":"2024-09-20T22:11:43.609213+0000","last_clean":"2024-09-20T22:11:43.609213+0000","last_became_active":"2024-09-20T22:11:43.608568+0000","last_became_peered":"2024-09-20T22:11:43.608568+0000","last_unstale":"2024-09-20T22:11:43.609213+0000","last_undegraded":"2024-09-20T22:11:43.609213+0000","last_fullsized":"2024-09-20T22:11:43.609213+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:10:16.022854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:10:16.022854+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:49.379118+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"71'13","reported_seq":376,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.617063+0000","last_change":"2024-09-20T22:11:33.124675+0000","last_active":"2024-09-20T22:11:45.617063+0000","last_peered":"2024-09-20T22:11:45.617063+0000","last_clean":"2024-09-20T22:11:45.617063+0000","last_became_active":"2024-09-20T22:06:54.850696+0000","last_became_peered":"2024-09-20T22:06:54.850696+0000","last_unstale":"2024-09-20T22:11:45.617063+0000","last_undegraded":"2024-09-20T22:11:45.617063+0000","last_fullsized":"2024-09-20T22:11:45.617063+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'13","last_scrub_stamp":"2024-09-20T22:11:33.124595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.124595+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:33.149438+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":338,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.615583+0000","last_change":"2024-09-20T22:11:01.241914+0000","last_active":"2024-09-20T22:11:45.615583+0000","last_peered":"2024-09-20T22:11:45.615583+0000","last_clean":"2024-09-20T22:11:45.615583+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:11:45.615583+0000","last_undegraded":"2024-09-20T22:11:45.615583+0000","last_fullsized":"2024-09-20T22:11:45.615583+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:01.241803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:11:01.241803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:06.385257+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":534,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.614931+0000","last_change":"2024-09-20T22:11:35.306968+0000","last_active":"2024-09-20T22:11:45.614931+0000","last_peered":"2024-09-20T22:11:45.614931+0000","last_clean":"2024-09-20T22:11:45.614931+0000","last_became_active":"2024-09-20T22:02:49.099508+0000","last_became_peered":"2024-09-20T22:02:49.099508+0000","last_unstale":"2024-09-20T22:11:45.614931+0000","last_undegraded":"2024-09-20T22:11:45.614931+0000","last_fullsized":"2024-09-20T22:11:45.614931+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:11:35.306842+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:11:35.306842+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:44.238377+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":508,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.614522+0000","last_change":"2024-09-20T22:10:57.475677+0000","last_active":"2024-09-20T22:11:45.614522+0000","last_peered":"2024-09-20T22:11:45.614522+0000","last_clean":"2024-09-20T22:11:45.614522+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:11:45.614522+0000","last_undegraded":"2024-09-20T22:11:45.614522+0000","last_fullsized":"2024-09-20T22:11:45.614522+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:57.475078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:10:57.475078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:16.236114+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":487,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.606292+0000","last_change":"2024-09-20T22:11:03.448918+0000","last_active":"2024-09-20T22:11:43.606292+0000","last_peered":"2024-09-20T22:11:43.606292+0000","last_clean":"2024-09-20T22:11:43.606292+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:11:43.606292+0000","last_undegraded":"2024-09-20T22:11:43.606292+0000","last_fullsized":"2024-09-20T22:11:43.606292+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:03.448813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:03.448813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:25.942922+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":28,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.574364+0000","last_change":"2024-09-20T22:11:35.053290+0000","last_active":"2024-09-20T22:11:46.574364+0000","last_peered":"2024-09-20T22:11:46.574364+0000","last_clean":"2024-09-20T22:11:46.574364+0000","last_became_active":"2024-09-20T22:11:35.052922+0000","last_became_peered":"2024-09-20T22:11:35.052922+0000","last_unstale":"2024-09-20T22:11:46.574364+0000","last_undegraded":"2024-09-20T22:11:46.574364+0000","last_fullsized":"2024-09-20T22:11:46.574364+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:50.247621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":502,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.615978+0000","last_change":"2024-09-20T22:11:36.214541+0000","last_active":"2024-09-20T22:11:45.615978+0000","last_peered":"2024-09-20T22:11:45.615978+0000","last_clean":"2024-09-20T22:11:45.615978+0000","last_became_active":"2024-09-20T22:08:48.446840+0000","last_became_peered":"2024-09-20T22:08:48.446840+0000","last_unstale":"2024-09-20T22:11:45.615978+0000","last_undegraded":"2024-09-20T22:11:45.615978+0000","last_fullsized":"2024-09-20T22:11:45.615978+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:11:36.214466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:11:36.214466+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:41.066885+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"71'12","reported_seq":515,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.574451+0000","last_change":"2024-09-20T22:10:58.599478+0000","last_active":"2024-09-20T22:11:46.574451+0000","last_peered":"2024-09-20T22:11:46.574451+0000","last_clean":"2024-09-20T22:11:46.574451+0000","last_became_active":"2024-09-20T22:06:54.848670+0000","last_became_peered":"2024-09-20T22:06:54.848670+0000","last_unstale":"2024-09-20T22:11:46.574451+0000","last_undegraded":"2024-09-20T22:11:46.574451+0000","last_fullsized":"2024-09-20T22:11:46.574451+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:10:58.599362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:58.599362+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:24.582332+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":500,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.573140+0000","last_change":"2024-09-20T22:10:30.889564+0000","last_active":"2024-09-20T22:11:46.573140+0000","last_peered":"2024-09-20T22:11:46.573140+0000","last_clean":"2024-09-20T22:11:46.573140+0000","last_became_active":"2024-09-20T22:02:50.110448+0000","last_became_peered":"2024-09-20T22:02:50.110448+0000","last_unstale":"2024-09-20T22:11:46.573140+0000","last_undegraded":"2024-09-20T22:11:46.573140+0000","last_fullsized":"2024-09-20T22:11:46.573140+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:30.889484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:10:30.889484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":1039,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:44.266341+0000","last_change":"2024-09-20T22:10:20.084329+0000","last_active":"2024-09-20T22:11:44.266341+0000","last_peered":"2024-09-20T22:11:44.266341+0000","last_clean":"2024-09-20T22:11:44.266341+0000","last_became_active":"2024-09-20T22:06:54.844454+0000","last_became_peered":"2024-09-20T22:06:54.844454+0000","last_unstale":"2024-09-20T22:11:44.266341+0000","last_undegraded":"2024-09-20T22:11:44.266341+0000","last_fullsized":"2024-09-20T22:11:44.266341+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:10:20.084250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:10:20.084250+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:47.009552+0000","scrub_duration":61,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":522,"num_read_kb":517,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":43,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.618584+0000","last_change":"2024-09-20T22:11:39.171066+0000","last_active":"2024-09-20T22:11:45.618584+0000","last_peered":"2024-09-20T22:11:45.618584+0000","last_clean":"2024-09-20T22:11:45.618584+0000","last_became_active":"2024-09-20T22:11:30.489860+0000","last_became_peered":"2024-09-20T22:11:30.489860+0000","last_unstale":"2024-09-20T22:11:45.618584+0000","last_undegraded":"2024-09-20T22:11:45.618584+0000","last_fullsized":"2024-09-20T22:11:45.618584+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:39.170979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:39.170979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:44.884792+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":338,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.615769+0000","last_change":"2024-09-20T22:11:30.024842+0000","last_active":"2024-09-20T22:11:45.615769+0000","last_peered":"2024-09-20T22:11:45.615769+0000","last_clean":"2024-09-20T22:11:45.615769+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:11:45.615769+0000","last_undegraded":"2024-09-20T22:11:45.615769+0000","last_fullsized":"2024-09-20T22:11:45.615769+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:30.024770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.415468+0000","last_clean_scrub_stamp":"2024-09-20T22:11:30.024770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:34.524988+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":28,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175062+0000","last_change":"2024-09-20T22:11:34.559269+0000","last_active":"2024-09-20T22:11:46.175062+0000","last_peered":"2024-09-20T22:11:46.175062+0000","last_clean":"2024-09-20T22:11:46.175062+0000","last_became_active":"2024-09-20T22:11:34.558981+0000","last_became_peered":"2024-09-20T22:11:34.558981+0000","last_unstale":"2024-09-20T22:11:46.175062+0000","last_undegraded":"2024-09-20T22:11:46.175062+0000","last_fullsized":"2024-09-20T22:11:46.175062+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:57.225697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"71'16","reported_seq":632,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.636092+0000","last_change":"2024-09-20T22:10:24.126185+0000","last_active":"2024-09-20T22:11:43.636092+0000","last_peered":"2024-09-20T22:11:43.636092+0000","last_clean":"2024-09-20T22:11:43.636092+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:11:43.636092+0000","last_undegraded":"2024-09-20T22:11:43.636092+0000","last_fullsized":"2024-09-20T22:11:43.636092+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'16","last_scrub_stamp":"2024-09-20T22:10:24.126065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:24.126065+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:46.151257+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":512,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175589+0000","last_change":"2024-09-20T22:11:30.487346+0000","last_active":"2024-09-20T22:11:46.175589+0000","last_peered":"2024-09-20T22:11:46.175589+0000","last_clean":"2024-09-20T22:11:46.175589+0000","last_became_active":"2024-09-20T22:11:30.486987+0000","last_became_peered":"2024-09-20T22:11:30.486987+0000","last_unstale":"2024-09-20T22:11:46.175589+0000","last_undegraded":"2024-09-20T22:11:46.175589+0000","last_fullsized":"2024-09-20T22:11:46.175589+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:18.235255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:18.235255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":490,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.605577+0000","last_change":"2024-09-20T22:10:33.656595+0000","last_active":"2024-09-20T22:11:43.605577+0000","last_peered":"2024-09-20T22:11:43.605577+0000","last_clean":"2024-09-20T22:11:43.605577+0000","last_became_active":"2024-09-20T22:10:33.656161+0000","last_became_peered":"2024-09-20T22:10:33.656161+0000","last_unstale":"2024-09-20T22:11:43.605577+0000","last_undegraded":"2024-09-20T22:11:43.605577+0000","last_fullsized":"2024-09-20T22:11:43.605577+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:10:17.014513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:10:17.014513+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:48.339290+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":81,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.605991+0000","last_change":"2024-09-20T22:11:32.281684+0000","last_active":"2024-09-20T22:11:43.605991+0000","last_peered":"2024-09-20T22:11:43.605991+0000","last_clean":"2024-09-20T22:11:43.605991+0000","last_became_active":"2024-09-20T22:10:48.907271+0000","last_became_peered":"2024-09-20T22:10:48.907271+0000","last_unstale":"2024-09-20T22:11:43.605991+0000","last_undegraded":"2024-09-20T22:11:43.605991+0000","last_fullsized":"2024-09-20T22:11:43.605991+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:32.281594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:32.281594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:43.436523+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":510,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.618407+0000","last_change":"2024-09-20T22:11:00.215924+0000","last_active":"2024-09-20T22:11:45.618407+0000","last_peered":"2024-09-20T22:11:45.618407+0000","last_clean":"2024-09-20T22:11:45.618407+0000","last_became_active":"2024-09-20T22:04:21.237725+0000","last_became_peered":"2024-09-20T22:04:21.237725+0000","last_unstale":"2024-09-20T22:11:45.618407+0000","last_undegraded":"2024-09-20T22:11:45.618407+0000","last_fullsized":"2024-09-20T22:11:45.618407+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:00.215806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:11:00.215806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:25.875894+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":25,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.606006+0000","last_change":"2024-09-20T22:11:34.559883+0000","last_active":"2024-09-20T22:11:43.606006+0000","last_peered":"2024-09-20T22:11:43.606006+0000","last_clean":"2024-09-20T22:11:43.606006+0000","last_became_active":"2024-09-20T22:11:34.559353+0000","last_became_peered":"2024-09-20T22:11:34.559353+0000","last_unstale":"2024-09-20T22:11:43.606006+0000","last_undegraded":"2024-09-20T22:11:43.606006+0000","last_fullsized":"2024-09-20T22:11:43.606006+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:54.364720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"71'11","reported_seq":514,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.605180+0000","last_change":"2024-09-20T22:10:24.621114+0000","last_active":"2024-09-20T22:11:43.605180+0000","last_peered":"2024-09-20T22:11:43.605180+0000","last_clean":"2024-09-20T22:11:43.605180+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:11:43.605180+0000","last_undegraded":"2024-09-20T22:11:43.605180+0000","last_fullsized":"2024-09-20T22:11:43.605180+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:10:24.620997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:24.620997+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":499,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.604826+0000","last_change":"2024-09-20T22:11:07.211087+0000","last_active":"2024-09-20T22:11:43.604826+0000","last_peered":"2024-09-20T22:11:43.604826+0000","last_clean":"2024-09-20T22:11:43.604826+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:11:43.604826+0000","last_undegraded":"2024-09-20T22:11:43.604826+0000","last_fullsized":"2024-09-20T22:11:43.604826+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:07.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:11:07.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:16.920794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":499,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.618258+0000","last_change":"2024-09-20T22:10:34.232282+0000","last_active":"2024-09-20T22:11:45.618258+0000","last_peered":"2024-09-20T22:11:45.618258+0000","last_clean":"2024-09-20T22:11:45.618258+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:11:45.618258+0000","last_undegraded":"2024-09-20T22:11:45.618258+0000","last_fullsized":"2024-09-20T22:11:45.618258+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:34.232168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:34.232168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:50.149200+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":499,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175595+0000","last_change":"2024-09-20T22:11:30.487618+0000","last_active":"2024-09-20T22:11:46.175595+0000","last_peered":"2024-09-20T22:11:46.175595+0000","last_clean":"2024-09-20T22:11:46.175595+0000","last_became_active":"2024-09-20T22:11:30.487283+0000","last_became_peered":"2024-09-20T22:11:30.487283+0000","last_unstale":"2024-09-20T22:11:46.175595+0000","last_undegraded":"2024-09-20T22:11:46.175595+0000","last_fullsized":"2024-09-20T22:11:46.175595+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:17.209772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:17.209772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.0","version":"295'28","reported_seq":56,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:48.021525+0000","last_change":"2024-09-20T22:11:35.304586+0000","last_active":"2024-09-20T22:11:48.021525+0000","last_peered":"2024-09-20T22:11:48.021525+0000","last_clean":"2024-09-20T22:11:48.021525+0000","last_became_active":"2024-09-20T22:11:35.304302+0000","last_became_peered":"2024-09-20T22:11:35.304302+0000","last_unstale":"2024-09-20T22:11:48.021525+0000","last_undegraded":"2024-09-20T22:11:48.021525+0000","last_fullsized":"2024-09-20T22:11:48.021525+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:35.122680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1835008,"num_objects":28,"num_object_clones":0,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":1792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":406,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.617277+0000","last_change":"2024-09-20T22:11:33.557801+0000","last_active":"2024-09-20T22:11:45.617277+0000","last_peered":"2024-09-20T22:11:45.617277+0000","last_clean":"2024-09-20T22:11:45.617277+0000","last_became_active":"2024-09-20T22:11:33.557208+0000","last_became_peered":"2024-09-20T22:11:33.557208+0000","last_unstale":"2024-09-20T22:11:45.617277+0000","last_undegraded":"2024-09-20T22:11:45.617277+0000","last_fullsized":"2024-09-20T22:11:45.617277+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:10:34.490571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:10:34.490571+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:59.242488+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"71'17","reported_seq":531,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.604682+0000","last_change":"2024-09-20T22:10:29.580641+0000","last_active":"2024-09-20T22:11:43.604682+0000","last_peered":"2024-09-20T22:11:43.604682+0000","last_clean":"2024-09-20T22:11:43.604682+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:11:43.604682+0000","last_undegraded":"2024-09-20T22:11:43.604682+0000","last_fullsized":"2024-09-20T22:11:43.604682+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'17","last_scrub_stamp":"2024-09-20T22:10:29.579935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:29.579935+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:48.628798+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":507,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.636949+0000","last_change":"2024-09-20T22:10:27.228704+0000","last_active":"2024-09-20T22:11:43.636949+0000","last_peered":"2024-09-20T22:11:43.636949+0000","last_clean":"2024-09-20T22:11:43.636949+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:11:43.636949+0000","last_undegraded":"2024-09-20T22:11:43.636949+0000","last_fullsized":"2024-09-20T22:11:43.636949+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:27.228607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:27.228607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:51.592209+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":338,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.616143+0000","last_change":"2024-09-20T22:11:02.256320+0000","last_active":"2024-09-20T22:11:45.616143+0000","last_peered":"2024-09-20T22:11:45.616143+0000","last_clean":"2024-09-20T22:11:45.616143+0000","last_became_active":"2024-09-20T22:06:54.848275+0000","last_became_peered":"2024-09-20T22:06:54.848275+0000","last_unstale":"2024-09-20T22:11:45.616143+0000","last_undegraded":"2024-09-20T22:11:45.616143+0000","last_fullsized":"2024-09-20T22:11:45.616143+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:02.256217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:02.256217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:26.715522+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":503,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.615235+0000","last_change":"2024-09-20T22:11:31.333144+0000","last_active":"2024-09-20T22:11:45.615235+0000","last_peered":"2024-09-20T22:11:45.615235+0000","last_clean":"2024-09-20T22:11:45.615235+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:11:45.615235+0000","last_undegraded":"2024-09-20T22:11:45.615235+0000","last_fullsized":"2024-09-20T22:11:45.615235+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:31.333027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:31.333027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:53.330444+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":28,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175023+0000","last_change":"2024-09-20T22:11:35.053482+0000","last_active":"2024-09-20T22:11:46.175023+0000","last_peered":"2024-09-20T22:11:46.175023+0000","last_clean":"2024-09-20T22:11:46.175023+0000","last_became_active":"2024-09-20T22:11:35.053141+0000","last_became_peered":"2024-09-20T22:11:35.053141+0000","last_unstale":"2024-09-20T22:11:46.175023+0000","last_undegraded":"2024-09-20T22:11:46.175023+0000","last_fullsized":"2024-09-20T22:11:46.175023+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:46.261380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":778,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.605900+0000","last_change":"2024-09-20T22:11:34.332239+0000","last_active":"2024-09-20T22:11:43.605900+0000","last_peered":"2024-09-20T22:11:43.605900+0000","last_clean":"2024-09-20T22:11:43.605900+0000","last_became_active":"2024-09-20T22:08:48.455288+0000","last_became_peered":"2024-09-20T22:08:48.455288+0000","last_unstale":"2024-09-20T22:11:43.605900+0000","last_undegraded":"2024-09-20T22:11:43.605900+0000","last_fullsized":"2024-09-20T22:11:43.605900+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:11:34.332158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.332158+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:36.731335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"289'15","reported_seq":514,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.604472+0000","last_change":"2024-09-20T22:11:37.277758+0000","last_active":"2024-09-20T22:11:43.604472+0000","last_peered":"2024-09-20T22:11:43.604472+0000","last_clean":"2024-09-20T22:11:43.604472+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:11:43.604472+0000","last_undegraded":"2024-09-20T22:11:43.604472+0000","last_fullsized":"2024-09-20T22:11:43.604472+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'15","last_scrub_stamp":"2024-09-20T22:11:37.277648+0000","last_deep_scrub":"289'15","last_deep_scrub_stamp":"2024-09-20T22:11:37.277648+0000","last_clean_scrub_stamp":"2024-09-20T22:11:37.277648+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:05.356939+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":506,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175405+0000","last_change":"2024-09-20T22:09:53.395968+0000","last_active":"2024-09-20T22:11:46.175405+0000","last_peered":"2024-09-20T22:11:46.175405+0000","last_clean":"2024-09-20T22:11:46.175405+0000","last_became_active":"2024-09-20T22:02:50.110446+0000","last_became_peered":"2024-09-20T22:02:50.110446+0000","last_unstale":"2024-09-20T22:11:46.175405+0000","last_undegraded":"2024-09-20T22:11:46.175405+0000","last_fullsized":"2024-09-20T22:11:46.175405+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:09:53.395833+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"283'11","reported_seq":652,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.614460+0000","last_change":"2024-09-20T22:11:30.497599+0000","last_active":"2024-09-20T22:11:45.614460+0000","last_peered":"2024-09-20T22:11:45.614460+0000","last_clean":"2024-09-20T22:11:45.614460+0000","last_became_active":"2024-09-20T22:11:30.485356+0000","last_became_peered":"2024-09-20T22:11:30.485356+0000","last_unstale":"2024-09-20T22:11:45.614460+0000","last_undegraded":"2024-09-20T22:11:45.614460+0000","last_fullsized":"2024-09-20T22:11:45.614460+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'10","last_scrub_stamp":"2024-09-20T22:11:08.571767+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:11:08.571767+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:24.914453+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":487,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.605005+0000","last_change":"2024-09-20T22:09:53.701152+0000","last_active":"2024-09-20T22:11:43.605005+0000","last_peered":"2024-09-20T22:11:43.605005+0000","last_clean":"2024-09-20T22:11:43.605005+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:11:43.605005+0000","last_undegraded":"2024-09-20T22:11:43.605005+0000","last_fullsized":"2024-09-20T22:11:43.605005+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:53.701054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:09:53.701054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":28,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.618064+0000","last_change":"2024-09-20T22:11:35.304534+0000","last_active":"2024-09-20T22:11:45.618064+0000","last_peered":"2024-09-20T22:11:45.618064+0000","last_clean":"2024-09-20T22:11:45.618064+0000","last_became_active":"2024-09-20T22:11:35.304192+0000","last_became_peered":"2024-09-20T22:11:35.304192+0000","last_unstale":"2024-09-20T22:11:45.618064+0000","last_undegraded":"2024-09-20T22:11:45.618064+0000","last_fullsized":"2024-09-20T22:11:45.618064+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:54.063980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"71'9","reported_seq":536,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.574335+0000","last_change":"2024-09-20T22:11:02.575450+0000","last_active":"2024-09-20T22:11:46.574335+0000","last_peered":"2024-09-20T22:11:46.574335+0000","last_clean":"2024-09-20T22:11:46.574335+0000","last_became_active":"2024-09-20T22:10:22.565004+0000","last_became_peered":"2024-09-20T22:10:22.565004+0000","last_unstale":"2024-09-20T22:11:46.574335+0000","last_undegraded":"2024-09-20T22:11:46.574335+0000","last_fullsized":"2024-09-20T22:11:46.574335+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:11:02.575330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:02.575330+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:05.729253+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":128,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.176113+0000","last_change":"2024-09-20T22:10:31.161469+0000","last_active":"2024-09-20T22:11:46.176113+0000","last_peered":"2024-09-20T22:11:46.176113+0000","last_clean":"2024-09-20T22:11:46.176113+0000","last_became_active":"2024-09-20T22:10:14.536155+0000","last_became_peered":"2024-09-20T22:10:14.536155+0000","last_unstale":"2024-09-20T22:11:46.176113+0000","last_undegraded":"2024-09-20T22:11:46.176113+0000","last_fullsized":"2024-09-20T22:11:46.176113+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:10:31.161375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:31.161375+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:49.187429+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":508,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.614404+0000","last_change":"2024-09-20T22:10:22.522763+0000","last_active":"2024-09-20T22:11:45.614404+0000","last_peered":"2024-09-20T22:11:45.614404+0000","last_clean":"2024-09-20T22:11:45.614404+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:11:45.614404+0000","last_undegraded":"2024-09-20T22:11:45.614404+0000","last_fullsized":"2024-09-20T22:11:45.614404+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:22.522610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:10:22.522610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:48.289448+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":490,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.573908+0000","last_change":"2024-09-20T22:08:27.817949+0000","last_active":"2024-09-20T22:11:46.573908+0000","last_peered":"2024-09-20T22:11:46.573908+0000","last_clean":"2024-09-20T22:11:46.573908+0000","last_became_active":"2024-09-20T22:06:55.824229+0000","last_became_peered":"2024-09-20T22:06:55.824229+0000","last_unstale":"2024-09-20T22:11:46.573908+0000","last_undegraded":"2024-09-20T22:11:46.573908+0000","last_fullsized":"2024-09-20T22:11:46.573908+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:27.817850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:27.817850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":34,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.618462+0000","last_change":"2024-09-20T22:11:40.601503+0000","last_active":"2024-09-20T22:11:45.618462+0000","last_peered":"2024-09-20T22:11:45.618462+0000","last_clean":"2024-09-20T22:11:45.618462+0000","last_became_active":"2024-09-20T22:11:40.601030+0000","last_became_peered":"2024-09-20T22:11:40.601030+0000","last_unstale":"2024-09-20T22:11:45.618462+0000","last_undegraded":"2024-09-20T22:11:45.618462+0000","last_fullsized":"2024-09-20T22:11:45.618462+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:44.031519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"71'19","reported_seq":578,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.636793+0000","last_change":"2024-09-20T22:10:30.669468+0000","last_active":"2024-09-20T22:11:43.636793+0000","last_peered":"2024-09-20T22:11:43.636793+0000","last_clean":"2024-09-20T22:11:43.636793+0000","last_became_active":"2024-09-20T22:10:30.652967+0000","last_became_peered":"2024-09-20T22:10:30.652967+0000","last_unstale":"2024-09-20T22:11:43.636793+0000","last_undegraded":"2024-09-20T22:11:43.636793+0000","last_fullsized":"2024-09-20T22:11:43.636793+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:10:15.165817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:15.165817+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":510,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.606628+0000","last_change":"2024-09-20T22:11:30.489226+0000","last_active":"2024-09-20T22:11:43.606628+0000","last_peered":"2024-09-20T22:11:43.606628+0000","last_clean":"2024-09-20T22:11:43.606628+0000","last_became_active":"2024-09-20T22:11:30.488650+0000","last_became_peered":"2024-09-20T22:11:30.488650+0000","last_unstale":"2024-09-20T22:11:43.606628+0000","last_undegraded":"2024-09-20T22:11:43.606628+0000","last_fullsized":"2024-09-20T22:11:43.606628+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:28.095505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:28.095505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:48.339321+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":519,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.614240+0000","last_change":"2024-09-20T22:11:30.484978+0000","last_active":"2024-09-20T22:11:45.614240+0000","last_peered":"2024-09-20T22:11:45.614240+0000","last_clean":"2024-09-20T22:11:45.614240+0000","last_became_active":"2024-09-20T22:11:30.484597+0000","last_became_peered":"2024-09-20T22:11:30.484597+0000","last_unstale":"2024-09-20T22:11:45.614240+0000","last_undegraded":"2024-09-20T22:11:45.614240+0000","last_fullsized":"2024-09-20T22:11:45.614240+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_clean_scrub_stamp":"2024-09-20T22:10:26.501363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:46.983222+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":95,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.605608+0000","last_change":"2024-09-20T22:10:30.657488+0000","last_active":"2024-09-20T22:11:43.605608+0000","last_peered":"2024-09-20T22:11:43.605608+0000","last_clean":"2024-09-20T22:11:43.605608+0000","last_became_active":"2024-09-20T22:10:30.657116+0000","last_became_peered":"2024-09-20T22:10:30.657116+0000","last_unstale":"2024-09-20T22:11:43.605608+0000","last_undegraded":"2024-09-20T22:11:43.605608+0000","last_fullsized":"2024-09-20T22:11:43.605608+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:08.495599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:08.495599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:48.339214+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"295'36","reported_seq":64,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:48.029452+0000","last_change":"2024-09-20T22:11:34.558254+0000","last_active":"2024-09-20T22:11:48.029452+0000","last_peered":"2024-09-20T22:11:48.029452+0000","last_clean":"2024-09-20T22:11:48.029452+0000","last_became_active":"2024-09-20T22:11:34.557976+0000","last_became_peered":"2024-09-20T22:11:34.557976+0000","last_unstale":"2024-09-20T22:11:48.029452+0000","last_undegraded":"2024-09-20T22:11:48.029452+0000","last_fullsized":"2024-09-20T22:11:48.029452+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:37.795228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2359296,"num_objects":36,"num_object_clones":0,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":36,"num_write_kb":2304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"71'12","reported_seq":519,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.604060+0000","last_change":"2024-09-20T22:10:21.628457+0000","last_active":"2024-09-20T22:11:43.604060+0000","last_peered":"2024-09-20T22:11:43.604060+0000","last_clean":"2024-09-20T22:11:43.604060+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:11:43.604060+0000","last_undegraded":"2024-09-20T22:11:43.604060+0000","last_fullsized":"2024-09-20T22:11:43.604060+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:10:21.628338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:21.628338+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":499,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.606684+0000","last_change":"2024-09-20T22:09:03.831996+0000","last_active":"2024-09-20T22:11:43.606684+0000","last_peered":"2024-09-20T22:11:43.606684+0000","last_clean":"2024-09-20T22:11:43.606684+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:11:43.606684+0000","last_undegraded":"2024-09-20T22:11:43.606684+0000","last_fullsized":"2024-09-20T22:11:43.606684+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:03.831774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:03.831774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:48.339242+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":517,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.614180+0000","last_change":"2024-09-20T22:11:36.287314+0000","last_active":"2024-09-20T22:11:45.614180+0000","last_peered":"2024-09-20T22:11:45.614180+0000","last_clean":"2024-09-20T22:11:45.614180+0000","last_became_active":"2024-09-20T22:06:54.843906+0000","last_became_peered":"2024-09-20T22:06:54.843906+0000","last_unstale":"2024-09-20T22:11:45.614180+0000","last_undegraded":"2024-09-20T22:11:45.614180+0000","last_fullsized":"2024-09-20T22:11:45.614180+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:36.287220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:36.287220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:47.050640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":445,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.574580+0000","last_change":"2024-09-20T22:11:40.600795+0000","last_active":"2024-09-20T22:11:46.574580+0000","last_peered":"2024-09-20T22:11:46.574580+0000","last_clean":"2024-09-20T22:11:46.574580+0000","last_became_active":"2024-09-20T22:11:40.600444+0000","last_became_peered":"2024-09-20T22:11:40.600444+0000","last_unstale":"2024-09-20T22:11:46.574580+0000","last_undegraded":"2024-09-20T22:11:46.574580+0000","last_fullsized":"2024-09-20T22:11:46.574580+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:36.768146+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:08:36.768146+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":34,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.574023+0000","last_change":"2024-09-20T22:11:40.601169+0000","last_active":"2024-09-20T22:11:46.574023+0000","last_peered":"2024-09-20T22:11:46.574023+0000","last_clean":"2024-09-20T22:11:46.574023+0000","last_became_active":"2024-09-20T22:11:40.600788+0000","last_became_peered":"2024-09-20T22:11:40.600788+0000","last_unstale":"2024-09-20T22:11:46.574023+0000","last_undegraded":"2024-09-20T22:11:46.574023+0000","last_fullsized":"2024-09-20T22:11:46.574023+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:37.359872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"71'15","reported_seq":51,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.617130+0000","last_change":"2024-09-20T22:11:30.506603+0000","last_active":"2024-09-20T22:11:45.617130+0000","last_peered":"2024-09-20T22:11:45.617130+0000","last_clean":"2024-09-20T22:11:45.617130+0000","last_became_active":"2024-09-20T22:11:30.493275+0000","last_became_peered":"2024-09-20T22:11:30.493275+0000","last_unstale":"2024-09-20T22:11:45.617130+0000","last_undegraded":"2024-09-20T22:11:45.617130+0000","last_fullsized":"2024-09-20T22:11:45.617130+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:11:02.397374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:02.397374+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:28.185784+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":101,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.606651+0000","last_change":"2024-09-20T22:10:30.661904+0000","last_active":"2024-09-20T22:11:43.606651+0000","last_peered":"2024-09-20T22:11:43.606651+0000","last_clean":"2024-09-20T22:11:43.606651+0000","last_became_active":"2024-09-20T22:10:30.653330+0000","last_became_peered":"2024-09-20T22:10:30.653330+0000","last_unstale":"2024-09-20T22:11:43.606651+0000","last_undegraded":"2024-09-20T22:11:43.606651+0000","last_fullsized":"2024-09-20T22:11:43.606651+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:09:48.706938+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:09:48.706938+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:48.339258+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":599,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.605797+0000","last_change":"2024-09-20T22:08:32.824544+0000","last_active":"2024-09-20T22:11:43.605797+0000","last_peered":"2024-09-20T22:11:43.605797+0000","last_clean":"2024-09-20T22:11:43.605797+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:11:43.605797+0000","last_undegraded":"2024-09-20T22:11:43.605797+0000","last_fullsized":"2024-09-20T22:11:43.605797+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:08:32.824413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.824413+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:48.339168+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":35,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.617646+0000","last_change":"2024-09-20T22:11:30.492983+0000","last_active":"2024-09-20T22:11:45.617646+0000","last_peered":"2024-09-20T22:11:45.617646+0000","last_clean":"2024-09-20T22:11:45.617646+0000","last_became_active":"2024-09-20T22:11:30.492476+0000","last_became_peered":"2024-09-20T22:11:30.492476+0000","last_unstale":"2024-09-20T22:11:45.617646+0000","last_undegraded":"2024-09-20T22:11:45.617646+0000","last_fullsized":"2024-09-20T22:11:45.617646+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:56.334297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:56.334297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:21.467964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":525,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.635498+0000","last_change":"2024-09-20T22:11:39.375720+0000","last_active":"2024-09-20T22:11:43.635498+0000","last_peered":"2024-09-20T22:11:43.635498+0000","last_clean":"2024-09-20T22:11:43.635498+0000","last_became_active":"2024-09-20T22:11:30.486764+0000","last_became_peered":"2024-09-20T22:11:30.486764+0000","last_unstale":"2024-09-20T22:11:43.635498+0000","last_undegraded":"2024-09-20T22:11:43.635498+0000","last_fullsized":"2024-09-20T22:11:43.635498+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:39.375639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:11:39.375639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:44.853725+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"71'6","reported_seq":532,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.176064+0000","last_change":"2024-09-20T22:11:07.315899+0000","last_active":"2024-09-20T22:11:46.176064+0000","last_peered":"2024-09-20T22:11:46.176064+0000","last_clean":"2024-09-20T22:11:46.176064+0000","last_became_active":"2024-09-20T22:10:48.922561+0000","last_became_peered":"2024-09-20T22:10:48.922561+0000","last_unstale":"2024-09-20T22:11:46.176064+0000","last_undegraded":"2024-09-20T22:11:46.176064+0000","last_fullsized":"2024-09-20T22:11:46.176064+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'6","last_scrub_stamp":"2024-09-20T22:11:07.315753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:07.315753+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:09.199775+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":35,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.618212+0000","last_change":"2024-09-20T22:11:30.487894+0000","last_active":"2024-09-20T22:11:45.618212+0000","last_peered":"2024-09-20T22:11:45.618212+0000","last_clean":"2024-09-20T22:11:45.618212+0000","last_became_active":"2024-09-20T22:11:30.487322+0000","last_became_peered":"2024-09-20T22:11:30.487322+0000","last_unstale":"2024-09-20T22:11:45.618212+0000","last_undegraded":"2024-09-20T22:11:45.618212+0000","last_fullsized":"2024-09-20T22:11:45.618212+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:20.293112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:20.293112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:47.777426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":478,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.606257+0000","last_change":"2024-09-20T22:09:00.737324+0000","last_active":"2024-09-20T22:11:43.606257+0000","last_peered":"2024-09-20T22:11:43.606257+0000","last_clean":"2024-09-20T22:11:43.606257+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:11:43.606257+0000","last_undegraded":"2024-09-20T22:11:43.606257+0000","last_fullsized":"2024-09-20T22:11:43.606257+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:00.737219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:00.737219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:48.339184+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"71'9","reported_seq":534,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.574477+0000","last_change":"2024-09-20T22:10:48.930194+0000","last_active":"2024-09-20T22:11:46.574477+0000","last_peered":"2024-09-20T22:11:46.574477+0000","last_clean":"2024-09-20T22:11:46.574477+0000","last_became_active":"2024-09-20T22:10:48.924249+0000","last_became_peered":"2024-09-20T22:10:48.924249+0000","last_unstale":"2024-09-20T22:11:46.574477+0000","last_undegraded":"2024-09-20T22:11:46.574477+0000","last_fullsized":"2024-09-20T22:11:46.574477+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:08:31.897253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:31.897253+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":498,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.606012+0000","last_change":"2024-09-20T22:10:29.059396+0000","last_active":"2024-09-20T22:11:43.606012+0000","last_peered":"2024-09-20T22:11:43.606012+0000","last_clean":"2024-09-20T22:11:43.606012+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:11:43.606012+0000","last_undegraded":"2024-09-20T22:11:43.606012+0000","last_fullsized":"2024-09-20T22:11:43.606012+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:29.059287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:29.059287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:48.339305+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":492,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175882+0000","last_change":"2024-09-20T22:11:09.360634+0000","last_active":"2024-09-20T22:11:46.175882+0000","last_peered":"2024-09-20T22:11:46.175882+0000","last_clean":"2024-09-20T22:11:46.175882+0000","last_became_active":"2024-09-20T22:06:54.845704+0000","last_became_peered":"2024-09-20T22:06:54.845704+0000","last_unstale":"2024-09-20T22:11:46.175882+0000","last_undegraded":"2024-09-20T22:11:46.175882+0000","last_fullsized":"2024-09-20T22:11:46.175882+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_clean_scrub_stamp":"2024-09-20T22:11:09.360439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:31.840825+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":503,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.614723+0000","last_change":"2024-09-20T22:11:05.544433+0000","last_active":"2024-09-20T22:11:45.614723+0000","last_peered":"2024-09-20T22:11:45.614723+0000","last_clean":"2024-09-20T22:11:45.614723+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:11:45.614723+0000","last_undegraded":"2024-09-20T22:11:45.614723+0000","last_fullsized":"2024-09-20T22:11:45.614723+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:05.544321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:05.544321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:20.221161+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"71'9","reported_seq":45,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.618331+0000","last_change":"2024-09-20T22:11:30.502818+0000","last_active":"2024-09-20T22:11:45.618331+0000","last_peered":"2024-09-20T22:11:45.618331+0000","last_clean":"2024-09-20T22:11:45.618331+0000","last_became_active":"2024-09-20T22:11:30.493335+0000","last_became_peered":"2024-09-20T22:11:30.493335+0000","last_unstale":"2024-09-20T22:11:45.618331+0000","last_undegraded":"2024-09-20T22:11:45.618331+0000","last_fullsized":"2024-09-20T22:11:45.618331+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:10:59.360044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:59.360044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:06.729574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":510,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.574047+0000","last_change":"2024-09-20T22:10:48.923031+0000","last_active":"2024-09-20T22:11:46.574047+0000","last_peered":"2024-09-20T22:11:46.574047+0000","last_clean":"2024-09-20T22:11:46.574047+0000","last_became_active":"2024-09-20T22:10:48.922707+0000","last_became_peered":"2024-09-20T22:10:48.922707+0000","last_unstale":"2024-09-20T22:11:46.574047+0000","last_undegraded":"2024-09-20T22:11:46.574047+0000","last_fullsized":"2024-09-20T22:11:46.574047+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:28.999559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:28.999559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:55.434360+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"276'10","reported_seq":619,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175816+0000","last_change":"2024-09-20T22:11:12.512544+0000","last_active":"2024-09-20T22:11:46.175816+0000","last_peered":"2024-09-20T22:11:46.175816+0000","last_clean":"2024-09-20T22:11:46.175816+0000","last_became_active":"2024-09-20T22:11:12.492297+0000","last_became_peered":"2024-09-20T22:11:12.492297+0000","last_unstale":"2024-09-20T22:11:46.175816+0000","last_undegraded":"2024-09-20T22:11:46.175816+0000","last_fullsized":"2024-09-20T22:11:46.175816+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:09:07.902502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:07.902502+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":498,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.605095+0000","last_change":"2024-09-20T22:11:30.485332+0000","last_active":"2024-09-20T22:11:43.605095+0000","last_peered":"2024-09-20T22:11:43.605095+0000","last_clean":"2024-09-20T22:11:43.605095+0000","last_became_active":"2024-09-20T22:11:30.484731+0000","last_became_peered":"2024-09-20T22:11:30.484731+0000","last_unstale":"2024-09-20T22:11:43.605095+0000","last_undegraded":"2024-09-20T22:11:43.605095+0000","last_fullsized":"2024-09-20T22:11:43.605095+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:05.183875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:05.183875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:32.622574+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"71'10","reported_seq":529,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175933+0000","last_change":"2024-09-20T22:11:40.438463+0000","last_active":"2024-09-20T22:11:46.175933+0000","last_peered":"2024-09-20T22:11:46.175933+0000","last_clean":"2024-09-20T22:11:46.175933+0000","last_became_active":"2024-09-20T22:10:22.583028+0000","last_became_peered":"2024-09-20T22:10:22.583028+0000","last_unstale":"2024-09-20T22:11:46.175933+0000","last_undegraded":"2024-09-20T22:11:46.175933+0000","last_fullsized":"2024-09-20T22:11:46.175933+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:11:40.438345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:40.438345+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:02.299535+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":490,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.606423+0000","last_change":"2024-09-20T22:08:56.746747+0000","last_active":"2024-09-20T22:11:43.606423+0000","last_peered":"2024-09-20T22:11:43.606423+0000","last_clean":"2024-09-20T22:11:43.606423+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:11:43.606423+0000","last_undegraded":"2024-09-20T22:11:43.606423+0000","last_fullsized":"2024-09-20T22:11:43.606423+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:56.746651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:56.746651+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:48.339199+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"283'12","reported_seq":51,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.618004+0000","last_change":"2024-09-20T22:11:35.062361+0000","last_active":"2024-09-20T22:11:45.618004+0000","last_peered":"2024-09-20T22:11:45.618004+0000","last_clean":"2024-09-20T22:11:45.618004+0000","last_became_active":"2024-09-20T22:11:30.488787+0000","last_became_peered":"2024-09-20T22:11:30.488787+0000","last_unstale":"2024-09-20T22:11:45.618004+0000","last_undegraded":"2024-09-20T22:11:45.618004+0000","last_fullsized":"2024-09-20T22:11:45.618004+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'12","last_scrub_stamp":"2024-09-20T22:11:35.062247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:35.062247+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:39.696169+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":498,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.574244+0000","last_change":"2024-09-20T22:11:05.577252+0000","last_active":"2024-09-20T22:11:46.574244+0000","last_peered":"2024-09-20T22:11:46.574244+0000","last_clean":"2024-09-20T22:11:46.574244+0000","last_became_active":"2024-09-20T22:10:30.659586+0000","last_became_peered":"2024-09-20T22:10:30.659586+0000","last_unstale":"2024-09-20T22:11:46.574244+0000","last_undegraded":"2024-09-20T22:11:46.574244+0000","last_fullsized":"2024-09-20T22:11:46.574244+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:05.577043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:05.577043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:11.961511+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"71'11","reported_seq":532,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.604720+0000","last_change":"2024-09-20T22:09:54.730657+0000","last_active":"2024-09-20T22:11:43.604720+0000","last_peered":"2024-09-20T22:11:43.604720+0000","last_clean":"2024-09-20T22:11:43.604720+0000","last_became_active":"2024-09-20T22:08:22.139581+0000","last_became_peered":"2024-09-20T22:08:22.139581+0000","last_unstale":"2024-09-20T22:11:43.604720+0000","last_undegraded":"2024-09-20T22:11:43.604720+0000","last_fullsized":"2024-09-20T22:11:43.604720+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:09:54.730526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:09:54.730526+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":498,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.604593+0000","last_change":"2024-09-20T22:11:33.355268+0000","last_active":"2024-09-20T22:11:43.604593+0000","last_peered":"2024-09-20T22:11:43.604593+0000","last_clean":"2024-09-20T22:11:43.604593+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:11:43.604593+0000","last_undegraded":"2024-09-20T22:11:43.604593+0000","last_fullsized":"2024-09-20T22:11:43.604593+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.355142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.355142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:38.410766+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":337,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.616334+0000","last_change":"2024-09-20T22:11:34.160476+0000","last_active":"2024-09-20T22:11:45.616334+0000","last_peered":"2024-09-20T22:11:45.616334+0000","last_clean":"2024-09-20T22:11:45.616334+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:11:45.616334+0000","last_undegraded":"2024-09-20T22:11:45.616334+0000","last_fullsized":"2024-09-20T22:11:45.616334+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:34.160404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.160404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:49.059597+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":328,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.615706+0000","last_change":"2024-09-20T22:11:30.487252+0000","last_active":"2024-09-20T22:11:45.615706+0000","last_peered":"2024-09-20T22:11:45.615706+0000","last_clean":"2024-09-20T22:11:45.615706+0000","last_became_active":"2024-09-20T22:11:30.486423+0000","last_became_peered":"2024-09-20T22:11:30.486423+0000","last_unstale":"2024-09-20T22:11:45.615706+0000","last_undegraded":"2024-09-20T22:11:45.615706+0000","last_fullsized":"2024-09-20T22:11:45.615706+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:23.320831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:23.320831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:52.624823+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"291'13","reported_seq":413,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.616610+0000","last_change":"2024-09-20T22:11:41.362291+0000","last_active":"2024-09-20T22:11:45.616610+0000","last_peered":"2024-09-20T22:11:45.616610+0000","last_clean":"2024-09-20T22:11:45.616610+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:11:45.616610+0000","last_undegraded":"2024-09-20T22:11:45.616610+0000","last_fullsized":"2024-09-20T22:11:45.616610+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'13","last_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_deep_scrub":"291'13","last_deep_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_clean_scrub_stamp":"2024-09-20T22:11:41.362215+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:08.259331+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":95,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.606161+0000","last_change":"2024-09-20T22:10:30.656109+0000","last_active":"2024-09-20T22:11:43.606161+0000","last_peered":"2024-09-20T22:11:43.606161+0000","last_clean":"2024-09-20T22:11:43.606161+0000","last_became_active":"2024-09-20T22:10:30.655502+0000","last_became_peered":"2024-09-20T22:10:30.655502+0000","last_unstale":"2024-09-20T22:11:43.606161+0000","last_undegraded":"2024-09-20T22:11:43.606161+0000","last_fullsized":"2024-09-20T22:11:43.606161+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:28.129779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:28.129779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:55.022772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":492,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175260+0000","last_change":"2024-09-20T22:11:31.262221+0000","last_active":"2024-09-20T22:11:46.175260+0000","last_peered":"2024-09-20T22:11:46.175260+0000","last_clean":"2024-09-20T22:11:46.175260+0000","last_became_active":"2024-09-20T22:06:54.844271+0000","last_became_peered":"2024-09-20T22:06:54.844271+0000","last_unstale":"2024-09-20T22:11:46.175260+0000","last_undegraded":"2024-09-20T22:11:46.175260+0000","last_fullsized":"2024-09-20T22:11:46.175260+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:31.262095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:31.262095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:38.538398+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":500,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.573140+0000","last_change":"2024-09-20T22:11:30.487611+0000","last_active":"2024-09-20T22:11:46.573140+0000","last_peered":"2024-09-20T22:11:46.573140+0000","last_clean":"2024-09-20T22:11:46.573140+0000","last_became_active":"2024-09-20T22:11:30.487088+0000","last_became_peered":"2024-09-20T22:11:30.487088+0000","last_unstale":"2024-09-20T22:11:46.573140+0000","last_undegraded":"2024-09-20T22:11:46.573140+0000","last_fullsized":"2024-09-20T22:11:46.573140+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:35.783884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:35.783884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:03.527369+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"71'11","reported_seq":549,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175031+0000","last_change":"2024-09-20T22:10:30.672892+0000","last_active":"2024-09-20T22:11:46.175031+0000","last_peered":"2024-09-20T22:11:46.175031+0000","last_clean":"2024-09-20T22:11:46.175031+0000","last_became_active":"2024-09-20T22:10:30.651743+0000","last_became_peered":"2024-09-20T22:10:30.651743+0000","last_unstale":"2024-09-20T22:11:46.175031+0000","last_undegraded":"2024-09-20T22:11:46.175031+0000","last_fullsized":"2024-09-20T22:11:46.175031+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:09:55.321800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:09:55.321800+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.618005+0000","last_change":"2024-09-20T22:11:43.636397+0000","last_active":"2024-09-20T22:11:45.618005+0000","last_peered":"2024-09-20T22:11:45.618005+0000","last_clean":"2024-09-20T22:11:45.618005+0000","last_became_active":"2024-09-20T22:11:30.493598+0000","last_became_peered":"2024-09-20T22:11:30.493598+0000","last_unstale":"2024-09-20T22:11:45.618005+0000","last_undegraded":"2024-09-20T22:11:45.618005+0000","last_fullsized":"2024-09-20T22:11:45.618005+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_clean_scrub_stamp":"2024-09-20T22:10:36.098315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:48.189476+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":490,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.605678+0000","last_change":"2024-09-20T22:09:01.751444+0000","last_active":"2024-09-20T22:11:43.605678+0000","last_peered":"2024-09-20T22:11:43.605678+0000","last_clean":"2024-09-20T22:11:43.605678+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:11:43.605678+0000","last_undegraded":"2024-09-20T22:11:43.605678+0000","last_fullsized":"2024-09-20T22:11:43.605678+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:01.751321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:01.751321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:48.339228+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":142,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.574472+0000","last_change":"2024-09-20T22:09:39.492012+0000","last_active":"2024-09-20T22:11:46.574472+0000","last_peered":"2024-09-20T22:11:46.574472+0000","last_clean":"2024-09-20T22:11:46.574472+0000","last_became_active":"2024-09-20T22:09:39.491148+0000","last_became_peered":"2024-09-20T22:09:39.491148+0000","last_unstale":"2024-09-20T22:11:46.574472+0000","last_undegraded":"2024-09-20T22:11:46.574472+0000","last_fullsized":"2024-09-20T22:11:46.574472+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:10.873660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:10.873660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"71'4","reported_seq":502,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.174874+0000","last_change":"2024-09-20T22:10:21.165887+0000","last_active":"2024-09-20T22:11:46.174874+0000","last_peered":"2024-09-20T22:11:46.174874+0000","last_clean":"2024-09-20T22:11:46.174874+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:11:46.174874+0000","last_undegraded":"2024-09-20T22:11:46.174874+0000","last_fullsized":"2024-09-20T22:11:46.174874+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'4","last_scrub_stamp":"2024-09-20T22:10:21.165774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:21.165774+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":491,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.573260+0000","last_change":"2024-09-20T22:08:29.852608+0000","last_active":"2024-09-20T22:11:46.573260+0000","last_peered":"2024-09-20T22:11:46.573260+0000","last_clean":"2024-09-20T22:11:46.573260+0000","last_became_active":"2024-09-20T22:02:50.111142+0000","last_became_peered":"2024-09-20T22:02:50.111142+0000","last_unstale":"2024-09-20T22:11:46.573260+0000","last_undegraded":"2024-09-20T22:11:46.573260+0000","last_fullsized":"2024-09-20T22:11:46.573260+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:29.852502+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:08:29.852502+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":513,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.635322+0000","last_change":"2024-09-20T22:11:34.342938+0000","last_active":"2024-09-20T22:11:43.635322+0000","last_peered":"2024-09-20T22:11:43.635322+0000","last_clean":"2024-09-20T22:11:43.635322+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:11:43.635322+0000","last_undegraded":"2024-09-20T22:11:43.635322+0000","last_fullsized":"2024-09-20T22:11:43.635322+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:34.342862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.342862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:55.365769+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":486,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.606559+0000","last_change":"2024-09-20T22:11:05.468972+0000","last_active":"2024-09-20T22:11:43.606559+0000","last_peered":"2024-09-20T22:11:43.606559+0000","last_clean":"2024-09-20T22:11:43.606559+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:11:43.606559+0000","last_undegraded":"2024-09-20T22:11:43.606559+0000","last_fullsized":"2024-09-20T22:11:43.606559+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_clean_scrub_stamp":"2024-09-20T22:11:05.468863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:10.330169+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":377,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:45.616603+0000","last_change":"2024-09-20T22:10:19.417244+0000","last_active":"2024-09-20T22:11:45.616603+0000","last_peered":"2024-09-20T22:11:45.616603+0000","last_clean":"2024-09-20T22:11:45.616603+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:11:45.616603+0000","last_undegraded":"2024-09-20T22:11:45.616603+0000","last_fullsized":"2024-09-20T22:11:45.616603+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:19.417142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:19.417142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":500,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.175120+0000","last_change":"2024-09-20T22:10:30.179319+0000","last_active":"2024-09-20T22:11:46.175120+0000","last_peered":"2024-09-20T22:11:46.175120+0000","last_clean":"2024-09-20T22:11:46.175120+0000","last_became_active":"2024-09-20T22:06:54.848032+0000","last_became_peered":"2024-09-20T22:06:54.848032+0000","last_unstale":"2024-09-20T22:11:46.175120+0000","last_undegraded":"2024-09-20T22:11:46.175120+0000","last_fullsized":"2024-09-20T22:11:46.175120+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:30.179244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:30.179244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"71'11","reported_seq":522,"reported_epoch":294,"state":"active+clean","last_fresh":"2024-09-20T22:11:43.636648+0000","last_change":"2024-09-20T22:10:33.253189+0000","last_active":"2024-09-20T22:11:43.636648+0000","last_peered":"2024-09-20T22:11:43.636648+0000","last_clean":"2024-09-20T22:11:43.636648+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:11:43.636648+0000","last_undegraded":"2024-09-20T22:11:43.636648+0000","last_fullsized":"2024-09-20T22:11:43.636648+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:10:33.253077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:33.253077+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":496,"reported_epoch":295,"state":"active+clean","last_fresh":"2024-09-20T22:11:46.574340+0000","last_change":"2024-09-20T22:11:12.495944+0000","last_active":"2024-09-20T22:11:46.574340+0000","last_peered":"2024-09-20T22:11:46.574340+0000","last_clean":"2024-09-20T22:11:46.574340+0000","last_became_active":"2024-09-20T22:11:12.495261+0000","last_became_peered":"2024-09-20T22:11:12.495261+0000","last_unstale":"2024-09-20T22:11:46.574340+0000","last_undegraded":"2024-09-20T22:11:46.574340+0000","last_fullsized":"2024-09-20T22:11:46.574340+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:32.854221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.854221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":4194304,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":64,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20709376,"data_stored":20709376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":692,"num_read_kb":449,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":1870,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":430,"ondisk_log_size":430,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1085,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":522,"num_read_kb":517,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":918048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1408,"ondisk_log_size":1408,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135983,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":424268,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":422654,"kb_avail":93320884,"statfs":{"total":95995035648,"available":95560585216,"internally_reserved":0,"allocated":1630208,"data_stored":1008389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":432798154},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430323,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":361472,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":360190,"kb_avail":93383680,"statfs":{"total":95995035648,"available":95624888320,"internally_reserved":0,"allocated":1282048,"data_stored":680507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":368835014},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724663,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":459140,"kb_used_data":35048,"kb_used_omap":1,"kb_used_meta":424062,"kb_avail":93286012,"statfs":{"total":95995035648,"available":95524876288,"internally_reserved":0,"allocated":35889152,"data_stored":35283522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":434239941},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986303,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":393432,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":392126,"kb_avail":93351720,"statfs":{"total":95995035648,"available":95592161280,"internally_reserved":0,"allocated":1306624,"data_stored":680542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":401537477},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770257980,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":261424,"kb_used_data":1308,"kb_used_omap":4,"kb_used_meta":260091,"kb_avail":93483728,"statfs":{"total":95995035648,"available":95727337472,"internally_reserved":0,"allocated":1339392,"data_stored":690579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":266333864},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149828,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":462052,"kb_used_data":1872,"kb_used_omap":1,"kb_used_meta":460158,"kb_avail":93283100,"statfs":{"total":95995035648,"available":95521894400,"internally_reserved":0,"allocated":1916928,"data_stored":1272202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":471202247},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411465,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":438072,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":436542,"kb_avail":93307080,"statfs":{"total":95995035648,"available":95546449920,"internally_reserved":0,"allocated":1544192,"data_stored":942557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":447019461},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738516,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":501720,"kb_used_data":1284,"kb_used_omap":1,"kb_used_meta":500414,"kb_avail":93243432,"statfs":{"total":95995035648,"available":95481274368,"internally_reserved":0,"allocated":1314816,"data_stored":689471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":512424389},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20709376,"data_stored":20709376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:11:53.500 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-20T22:11:53.500 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:11:53.500 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-20T22:11:53.500 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 1.000000 2024-09-20T22:11:53.500 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 0 1 2024-09-20T22:11:53.544 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:53.578 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:53.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:53.579 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:53.679 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:53.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:53.727 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:53.727 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:53.828 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:53.873 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:53.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:53.875 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:53.888 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 6 16 17078 17062 177.716 167.125 0.00606192 0.00561849 2024-09-20T22:11:53.946 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T22:11:53.976 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:54.033 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:54.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:54.047 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:54.047 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:54.085 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:54.148 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:54.197 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:54.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:54.199 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:54.299 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:54.335 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:54.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:54.337 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:54.437 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:54.471 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:54.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:54.471 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:54.572 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:54.609 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:54.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:54.610 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:54.700 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:11:54.710 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:54.744 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:54.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:54.746 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:54.846 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:54.877 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:54.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:54.878 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:54.888 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 7 16 19343 19327 172.549 141.562 0.00501101 0.00578597 2024-09-20T22:11:54.979 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:55.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:55.035 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:55.035 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:55.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:55.199 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:55.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:55.201 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:55.209 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:11:55.210 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:11:55.214 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:55.261 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:55 smithi060 ceph-mon[40290]: pgmap v484: 158 pgs: 158 active+clean; 652 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-20T22:11:55.261 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:55 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2378657841' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:55.261 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:11:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:11:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:11:55.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:55 smithi060 ceph-mon[29694]: pgmap v484: 158 pgs: 158 active+clean; 652 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-20T22:11:55.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:55 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2378657841' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:55.262 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:11:55.263 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:11:55.283 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:11:55.299 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:11:55.300 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:11:55.302 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:55.331 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:55.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:55.332 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:55.344 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:11:55.344 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:11:55.380 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:11:55.380 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:11:55.417 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:11:55.418 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:11:55.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:55.453 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:11:55.453 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:11:55.480 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:55.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:55.481 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:55.491 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:11:55.491 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:11:55.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:55 smithi067 ceph-mon[33511]: pgmap v484: 158 pgs: 158 active+clean; 652 MiB data, 4.8 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-20T22:11:55.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2378657841' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:11:55.581 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:55.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:55.628 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:55.628 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:55.729 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:55.762 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:55.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:55.763 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:55.863 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:55.889 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 8 16 21458 21442 167.499 132.188 0.00448108 0.00596333 2024-09-20T22:11:55.901 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:55.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:55.902 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:56.003 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:56.043 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:56.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:56.045 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:56.102 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:11:56.102 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:11:56.145 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:56.180 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:56.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:56.181 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:56.209 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-20T22:11:56.281 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:56.315 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:56.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:56.316 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:56.417 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:56.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:56.457 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:56.458 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:56.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:56 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3179306539' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:11:56.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:56 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:11:56.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:56 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3179306539' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:11:56.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:56 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:11:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:56 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3179306539' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:11:56.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:56 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:11:56.558 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:56.604 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:56.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:56.605 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:56.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:56.748 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:56.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:56.749 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:56.850 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:56.883 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:56.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:56.884 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:56.888 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 9 16 23831 23815 165.366 148.312 0.00352687 0.00604031 2024-09-20T22:11:56.985 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:57.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:57.020 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:57.020 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:57.121 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:57.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:57.158 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:57.158 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:57.259 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:57.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:57.293 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:57.293 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:57.394 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:11:57.428 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:57.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:57.429 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:11:57.473 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:57 smithi060 ceph-mon[29694]: pgmap v485: 158 pgs: 158 active+clean; 991 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 123 MiB/s wr, 1.97k op/s 2024-09-20T22:11:57.473 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:57 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:11:57.473 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:57 smithi060 ceph-mon[29694]: osdmap e298: 8 total, 8 up, 8 in 2024-09-20T22:11:57.473 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:57 smithi060 ceph-mon[29694]: osdmap e299: 8 total, 8 up, 8 in 2024-09-20T22:11:57.473 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:57 smithi060 ceph-mon[40290]: pgmap v485: 158 pgs: 158 active+clean; 991 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 123 MiB/s wr, 1.97k op/s 2024-09-20T22:11:57.473 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:57 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:11:57.473 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:57 smithi060 ceph-mon[40290]: osdmap e298: 8 total, 8 up, 8 in 2024-09-20T22:11:57.474 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:57 smithi060 ceph-mon[40290]: osdmap e299: 8 total, 8 up, 8 in 2024-09-20T22:11:57.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:57 smithi067 ceph-mon[33511]: pgmap v485: 158 pgs: 158 active+clean; 991 MiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 123 MiB/s wr, 1.97k op/s 2024-09-20T22:11:57.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:57 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:11:57.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:57 smithi067 ceph-mon[33511]: osdmap e298: 8 total, 8 up, 8 in 2024-09-20T22:11:57.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:57 smithi067 ceph-mon[33511]: osdmap e299: 8 total, 8 up, 8 in 2024-09-20T22:11:57.530 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:11:57.567 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:57.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:57.568 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:11:57.668 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:57.704 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:57.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:57.705 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:57.806 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:57.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:57.849 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:57.850 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:57.888 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 10 15 26274 26259 164.103 152.75 0.00646469 0.00608763 2024-09-20T22:11:57.951 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:57.982 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:57.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:57.983 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:58.084 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:58.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:58.117 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:58.117 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:58.218 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:58.258 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:58.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:58.259 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:58.360 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:58.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:58.406 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:58.406 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:58.507 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:11:58.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:58 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:11:58.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:58 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:11:58.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:58 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:11:58.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:58 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:11:58.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:58 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:11:58.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:58 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:11:58.542 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:58.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:58.543 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:11:58.630 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:11:58.644 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:58.675 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:58.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:58.676 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:58.777 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:11:58.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:58.817 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:58.818 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:11:58.889 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 11 16 28319 28303 160.797 127.75 0.00641876 0.00621272 2024-09-20T22:11:58.918 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:58.955 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T22:11:58.988 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:58.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:58.989 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:59.090 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:11:59.148 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:59.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:59.150 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:11:59.251 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:11:59.294 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:59.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:59.295 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:11:59.305 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:11:59.396 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:59.434 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:59.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:59.435 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:59.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:59 smithi067 ceph-mon[33511]: pgmap v488: 158 pgs: 158 active+clean; 1.4 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 188 MiB/s wr, 3.00k op/s 2024-09-20T22:11:59.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:59 smithi067 ceph-mon[33511]: osdmap e300: 8 total, 8 up, 8 in 2024-09-20T22:11:59.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:59 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3550821000' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:11:59.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:11:59 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:11:59.535 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:59.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:59 smithi060 ceph-mon[40290]: pgmap v488: 158 pgs: 158 active+clean; 1.4 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 188 MiB/s wr, 3.00k op/s 2024-09-20T22:11:59.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:59 smithi060 ceph-mon[40290]: osdmap e300: 8 total, 8 up, 8 in 2024-09-20T22:11:59.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:59 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3550821000' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:11:59.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:11:59 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:11:59.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:59 smithi060 ceph-mon[29694]: pgmap v488: 158 pgs: 158 active+clean; 1.4 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 188 MiB/s wr, 3.00k op/s 2024-09-20T22:11:59.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:59 smithi060 ceph-mon[29694]: osdmap e300: 8 total, 8 up, 8 in 2024-09-20T22:11:59.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:59 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3550821000' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:11:59.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:11:59 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:11:59.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:59.595 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:59.595 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:59.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:11:59.736 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:11:59.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:59.737 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:11:59.838 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:11:59.871 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:11:59.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:11:59.872 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:11:59.889 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 12 16 30226 30210 157.329 119.188 0.00742845 0.00635079 2024-09-20T22:11:59.972 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:00.009 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:00.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:00.011 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:00.111 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:00.112 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:00.145 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:00.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:00.146 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:00.192 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:00.246 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:00.282 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:00.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:00.283 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:00.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:00 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:12:00.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:00 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:12:00.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:00 smithi060 ceph-mon[29694]: osdmap e301: 8 total, 8 up, 8 in 2024-09-20T22:12:00.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:00 smithi060 ceph-mon[29694]: 6.1b scrub starts 2024-09-20T22:12:00.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:00 smithi060 ceph-mon[29694]: 6.1b scrub ok 2024-09-20T22:12:00.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:00 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:12:00.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:00 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:12:00.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:00 smithi060 ceph-mon[40290]: osdmap e301: 8 total, 8 up, 8 in 2024-09-20T22:12:00.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:00 smithi060 ceph-mon[40290]: 6.1b scrub starts 2024-09-20T22:12:00.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:00 smithi060 ceph-mon[40290]: 6.1b scrub ok 2024-09-20T22:12:00.384 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:00.416 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:00.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:00.417 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:00.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:00 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:12:00.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:00 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:12:00.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:00 smithi067 ceph-mon[33511]: osdmap e301: 8 total, 8 up, 8 in 2024-09-20T22:12:00.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:00 smithi067 ceph-mon[33511]: 6.1b scrub starts 2024-09-20T22:12:00.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:00 smithi067 ceph-mon[33511]: 6.1b scrub ok 2024-09-20T22:12:00.518 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:00.566 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:00.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:00.567 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:00.667 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:00.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:00.732 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:00.732 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:00.833 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:00.881 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:00.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:00.882 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:00.889 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 13 16 32097 32081 154.216 116.938 0.0129951 0.00647848 2024-09-20T22:12:00.946 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:12:00.946 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:12:00.983 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:01.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:01.031 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:01.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:01.131 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:01.162 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:01.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:01.163 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:01.220 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:12:01.220 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:12:01.255 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:12:01.256 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:12:01.263 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:01.293 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:12:01.293 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:12:01.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:01.306 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:01.307 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:01.343 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:12:01.343 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:12:01.380 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:12:01.380 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:12:01.407 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:01.416 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:12:01.416 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:12:01.416 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[29694]: 5.5 scrub starts 2024-09-20T22:12:01.417 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[29694]: 5.5 scrub ok 2024-09-20T22:12:01.417 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[29694]: pgmap v491: 158 pgs: 158 active+clean; 1.4 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 105 MiB/s wr, 1.67k op/s 2024-09-20T22:12:01.417 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:12:01.417 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[29694]: 4.a scrub starts 2024-09-20T22:12:01.417 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[29694]: 4.a scrub ok 2024-09-20T22:12:01.417 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[29694]: 4.19 scrub starts 2024-09-20T22:12:01.417 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[29694]: 4.19 scrub ok 2024-09-20T22:12:01.417 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:12:01.418 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:12:01.418 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:12:01.418 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3801676171' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:01.418 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[40290]: 5.5 scrub starts 2024-09-20T22:12:01.418 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[40290]: 5.5 scrub ok 2024-09-20T22:12:01.418 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[40290]: pgmap v491: 158 pgs: 158 active+clean; 1.4 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 105 MiB/s wr, 1.67k op/s 2024-09-20T22:12:01.418 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:12:01.419 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[40290]: 4.a scrub starts 2024-09-20T22:12:01.419 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[40290]: 4.a scrub ok 2024-09-20T22:12:01.419 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[40290]: 4.19 scrub starts 2024-09-20T22:12:01.419 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[40290]: 4.19 scrub ok 2024-09-20T22:12:01.419 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:12:01.419 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:12:01.419 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:12:01.419 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:01 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3801676171' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:01.441 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:01.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:01.443 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:01.448 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:12:01.448 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:12:01.485 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:12:01.485 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:12:01.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:01 smithi067 ceph-mon[33511]: 5.5 scrub starts 2024-09-20T22:12:01.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:01 smithi067 ceph-mon[33511]: 5.5 scrub ok 2024-09-20T22:12:01.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:01 smithi067 ceph-mon[33511]: pgmap v491: 158 pgs: 158 active+clean; 1.4 GiB data, 7.1 GiB used, 708 GiB / 715 GiB avail; 105 MiB/s wr, 1.67k op/s 2024-09-20T22:12:01.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:01 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:12:01.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:01 smithi067 ceph-mon[33511]: 4.a scrub starts 2024-09-20T22:12:01.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:01 smithi067 ceph-mon[33511]: 4.a scrub ok 2024-09-20T22:12:01.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:01 smithi067 ceph-mon[33511]: 4.19 scrub starts 2024-09-20T22:12:01.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:01 smithi067 ceph-mon[33511]: 4.19 scrub ok 2024-09-20T22:12:01.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:01 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:12:01.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:01 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:12:01.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:01 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:12:01.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:01 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3801676171' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:01.544 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:01.585 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:01.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:01.586 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:01.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:01.726 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:01.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:01.727 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:01.828 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:01.864 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:01.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:01.865 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:01.868 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T22:12:01.889 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 14 16 33924 33908 151.356 114.188 0.00805189 0.00660158 2024-09-20T22:12:01.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:02.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:02.007 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:02.008 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:02.081 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:12:02.108 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:02.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:02.179 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:02.179 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:02.280 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:02.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:02.333 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:02.333 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:02.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:02.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:02.477 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:02.477 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:02.484 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:02 smithi060 ceph-mon[40290]: 6.9 scrub starts 2024-09-20T22:12:02.484 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:02 smithi060 ceph-mon[40290]: 6.9 scrub ok 2024-09-20T22:12:02.484 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:02 smithi060 ceph-mon[40290]: 3.1d scrub starts 2024-09-20T22:12:02.484 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:02 smithi060 ceph-mon[40290]: 3.1d scrub ok 2024-09-20T22:12:02.484 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:02 smithi060 ceph-mon[29694]: 6.9 scrub starts 2024-09-20T22:12:02.485 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:02 smithi060 ceph-mon[29694]: 6.9 scrub ok 2024-09-20T22:12:02.485 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:02 smithi060 ceph-mon[29694]: 3.1d scrub starts 2024-09-20T22:12:02.485 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:02 smithi060 ceph-mon[29694]: 3.1d scrub ok 2024-09-20T22:12:02.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:02 smithi067 ceph-mon[33511]: 6.9 scrub starts 2024-09-20T22:12:02.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:02 smithi067 ceph-mon[33511]: 6.9 scrub ok 2024-09-20T22:12:02.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:02 smithi067 ceph-mon[33511]: 3.1d scrub starts 2024-09-20T22:12:02.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:02 smithi067 ceph-mon[33511]: 3.1d scrub ok 2024-09-20T22:12:02.514 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:12:02 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:12:02.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:02.625 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:02.638 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:02.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:02.639 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:02.739 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:02.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:02.776 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:02.776 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:02.835 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:12:02.836 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:12:02.836 INFO:tasks.thrashosds.thrasher:Shrinking pool .mgr 2024-09-20T22:12:02.836 INFO:tasks.cephadm.ceph_manager.ceph:contract_pool .mgr by 10 min 8 2024-09-20T22:12:02.836 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:12:02.873 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:02.877 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:02.892 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 15 16 35729 35713 148.757 112.812 0.0152395 0.00671575 2024-09-20T22:12:02.927 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:02.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:02.929 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:03.029 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:03.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:03.078 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:03.078 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:03.179 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:03.207 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:03.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:03.209 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:03.309 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:03.349 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:03.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:03.350 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:03.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:03.479 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:03.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:03.480 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:03.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:03 smithi067 ceph-mon[33511]: 3.17 scrub starts 2024-09-20T22:12:03.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:03 smithi067 ceph-mon[33511]: 3.17 scrub ok 2024-09-20T22:12:03.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:03 smithi067 ceph-mon[33511]: pgmap v492: 158 pgs: 158 active+clean; 1.9 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 863 B/s rd, 167 MiB/s wr, 2.68k op/s 2024-09-20T22:12:03.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:03 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:12:03.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:03 smithi060 ceph-mon[29694]: 3.17 scrub starts 2024-09-20T22:12:03.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:03 smithi060 ceph-mon[29694]: 3.17 scrub ok 2024-09-20T22:12:03.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:03 smithi060 ceph-mon[29694]: pgmap v492: 158 pgs: 158 active+clean; 1.9 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 863 B/s rd, 167 MiB/s wr, 2.68k op/s 2024-09-20T22:12:03.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:03 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:12:03.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:03 smithi060 ceph-mon[40290]: 3.17 scrub starts 2024-09-20T22:12:03.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:03 smithi060 ceph-mon[40290]: 3.17 scrub ok 2024-09-20T22:12:03.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:03 smithi060 ceph-mon[40290]: pgmap v492: 158 pgs: 158 active+clean; 1.9 GiB data, 8.7 GiB used, 706 GiB / 715 GiB avail; 863 B/s rd, 167 MiB/s wr, 2.68k op/s 2024-09-20T22:12:03.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:03 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:12:03.581 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:03.618 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:03.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:03.619 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:03.720 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:03.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:03.757 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:03.757 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:03.858 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:03.892 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 16 16 37568 37552 146.643 114.938 0.00305191 0.00681324 2024-09-20T22:12:03.897 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:03.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:03.899 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:03.999 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:04.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:04.033 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:04.033 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:04.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:04.157 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:04.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:04.158 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:04.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:04.296 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:04.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:04.298 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:04.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:04.436 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:04.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:04.438 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:04.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:04 smithi067 ceph-mon[33511]: 6.b scrub starts 2024-09-20T22:12:04.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:04 smithi067 ceph-mon[33511]: 6.b scrub ok 2024-09-20T22:12:04.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:04 smithi067 ceph-mon[33511]: 3.3 scrub starts 2024-09-20T22:12:04.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:04 smithi067 ceph-mon[33511]: 3.3 scrub ok 2024-09-20T22:12:04.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:04 smithi067 ceph-mon[33511]: 4.1f scrub starts 2024-09-20T22:12:04.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:04 smithi067 ceph-mon[33511]: 4.1f scrub ok 2024-09-20T22:12:04.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:04 smithi067 ceph-mon[33511]: 5.14 scrub starts 2024-09-20T22:12:04.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:04 smithi067 ceph-mon[33511]: 5.14 scrub ok 2024-09-20T22:12:04.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:04 smithi060 ceph-mon[29694]: 6.b scrub starts 2024-09-20T22:12:04.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:04 smithi060 ceph-mon[29694]: 6.b scrub ok 2024-09-20T22:12:04.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:04 smithi060 ceph-mon[29694]: 3.3 scrub starts 2024-09-20T22:12:04.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:04 smithi060 ceph-mon[29694]: 3.3 scrub ok 2024-09-20T22:12:04.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:04 smithi060 ceph-mon[29694]: 4.1f scrub starts 2024-09-20T22:12:04.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:04 smithi060 ceph-mon[29694]: 4.1f scrub ok 2024-09-20T22:12:04.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:04 smithi060 ceph-mon[29694]: 5.14 scrub starts 2024-09-20T22:12:04.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:04 smithi060 ceph-mon[29694]: 5.14 scrub ok 2024-09-20T22:12:04.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:04 smithi060 ceph-mon[40290]: 6.b scrub starts 2024-09-20T22:12:04.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:04 smithi060 ceph-mon[40290]: 6.b scrub ok 2024-09-20T22:12:04.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:04 smithi060 ceph-mon[40290]: 3.3 scrub starts 2024-09-20T22:12:04.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:04 smithi060 ceph-mon[40290]: 3.3 scrub ok 2024-09-20T22:12:04.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:04 smithi060 ceph-mon[40290]: 4.1f scrub starts 2024-09-20T22:12:04.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:04 smithi060 ceph-mon[40290]: 4.1f scrub ok 2024-09-20T22:12:04.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:04 smithi060 ceph-mon[40290]: 5.14 scrub starts 2024-09-20T22:12:04.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:04 smithi060 ceph-mon[40290]: 5.14 scrub ok 2024-09-20T22:12:04.538 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:04.552 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:12:04.573 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:04.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:04.574 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:04.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:04.717 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:04.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:04.719 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:04.819 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:04.877 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:04.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:04.878 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:04.893 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 17 16 39229 39213 144.121 103.812 0.0126654 0.00693184 2024-09-20T22:12:04.978 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:05.016 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:05.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:05.019 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:05.119 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:05.166 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:05.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:05.167 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:05.250 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:12:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:12:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:12:05.270 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:05.305 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:05.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:05.307 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:05.408 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:05.448 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:05.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:05.449 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:05.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:05 smithi067 ceph-mon[33511]: pgmap v493: 158 pgs: 158 active+clean; 2.0 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 667 B/s rd, 114 MiB/s wr, 1.83k op/s 2024-09-20T22:12:05.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:05 smithi067 ceph-mon[33511]: 5.9 scrub starts 2024-09-20T22:12:05.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:05 smithi067 ceph-mon[33511]: 5.9 scrub ok 2024-09-20T22:12:05.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:05 smithi067 ceph-mon[33511]: 3.7 scrub starts 2024-09-20T22:12:05.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:05 smithi067 ceph-mon[33511]: 3.7 scrub ok 2024-09-20T22:12:05.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:05 smithi060 ceph-mon[29694]: pgmap v493: 158 pgs: 158 active+clean; 2.0 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 667 B/s rd, 114 MiB/s wr, 1.83k op/s 2024-09-20T22:12:05.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:05 smithi060 ceph-mon[29694]: 5.9 scrub starts 2024-09-20T22:12:05.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:05 smithi060 ceph-mon[29694]: 5.9 scrub ok 2024-09-20T22:12:05.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:05 smithi060 ceph-mon[29694]: 3.7 scrub starts 2024-09-20T22:12:05.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:05 smithi060 ceph-mon[29694]: 3.7 scrub ok 2024-09-20T22:12:05.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:05 smithi060 ceph-mon[40290]: pgmap v493: 158 pgs: 158 active+clean; 2.0 GiB data, 9.1 GiB used, 706 GiB / 715 GiB avail; 667 B/s rd, 114 MiB/s wr, 1.83k op/s 2024-09-20T22:12:05.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:05 smithi060 ceph-mon[40290]: 5.9 scrub starts 2024-09-20T22:12:05.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:05 smithi060 ceph-mon[40290]: 5.9 scrub ok 2024-09-20T22:12:05.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:05 smithi060 ceph-mon[40290]: 3.7 scrub starts 2024-09-20T22:12:05.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:05 smithi060 ceph-mon[40290]: 3.7 scrub ok 2024-09-20T22:12:05.549 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:05.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:05.588 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:05.588 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:05.689 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:05.730 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:05.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:05.732 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:05.832 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:05.865 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:05.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:05.866 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:05.893 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 18 16 40964 40948 142.139 108.438 0.00981009 0.00702961 2024-09-20T22:12:05.966 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:05.999 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:06.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:06.000 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:06.101 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:06.123 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:06.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:06.124 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:06.225 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:06.265 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:06.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:06.266 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:06.367 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:06.402 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:06.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:06.403 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:06.504 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:06 smithi060 ceph-mon[29694]: 3.16 scrub starts 2024-09-20T22:12:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:06 smithi060 ceph-mon[29694]: 3.16 scrub ok 2024-09-20T22:12:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:06 smithi060 ceph-mon[29694]: 4.5 scrub starts 2024-09-20T22:12:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:06 smithi060 ceph-mon[29694]: 4.5 scrub ok 2024-09-20T22:12:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:12:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:06 smithi060 ceph-mon[29694]: 6.15 scrub starts 2024-09-20T22:12:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:06 smithi060 ceph-mon[29694]: 6.15 scrub ok 2024-09-20T22:12:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:06 smithi060 ceph-mon[29694]: 6.6 scrub starts 2024-09-20T22:12:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:06 smithi060 ceph-mon[29694]: 6.6 scrub ok 2024-09-20T22:12:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:06 smithi060 ceph-mon[40290]: 3.16 scrub starts 2024-09-20T22:12:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:06 smithi060 ceph-mon[40290]: 3.16 scrub ok 2024-09-20T22:12:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:06 smithi060 ceph-mon[40290]: 4.5 scrub starts 2024-09-20T22:12:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:06 smithi060 ceph-mon[40290]: 4.5 scrub ok 2024-09-20T22:12:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:12:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:06 smithi060 ceph-mon[40290]: 6.15 scrub starts 2024-09-20T22:12:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:06 smithi060 ceph-mon[40290]: 6.15 scrub ok 2024-09-20T22:12:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:06 smithi060 ceph-mon[40290]: 6.6 scrub starts 2024-09-20T22:12:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:06 smithi060 ceph-mon[40290]: 6.6 scrub ok 2024-09-20T22:12:06.540 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:06.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:06.541 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:06.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:06.673 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:06.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:06.675 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:06.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:06 smithi067 ceph-mon[33511]: 3.16 scrub starts 2024-09-20T22:12:06.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:06 smithi067 ceph-mon[33511]: 3.16 scrub ok 2024-09-20T22:12:06.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:06 smithi067 ceph-mon[33511]: 4.5 scrub starts 2024-09-20T22:12:06.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:06 smithi067 ceph-mon[33511]: 4.5 scrub ok 2024-09-20T22:12:06.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:12:06.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:06 smithi067 ceph-mon[33511]: 6.15 scrub starts 2024-09-20T22:12:06.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:06 smithi067 ceph-mon[33511]: 6.15 scrub ok 2024-09-20T22:12:06.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:06 smithi067 ceph-mon[33511]: 6.6 scrub starts 2024-09-20T22:12:06.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:06 smithi067 ceph-mon[33511]: 6.6 scrub ok 2024-09-20T22:12:06.775 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:06.815 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:06.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:06.816 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:06.893 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 19 16 42673 42657 140.279 106.812 0.00594909 0.00712343 2024-09-20T22:12:06.917 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:06.958 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:06.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:06.960 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:07.061 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:07.084 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:07.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:07.085 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:07.186 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:07.223 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:07.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:07.225 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:07.273 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:12:07.325 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:07.363 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:07.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:07.364 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:07.464 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:07.496 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:07.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:07.497 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:07.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:07 smithi060 ceph-mon[29694]: pgmap v494: 158 pgs: 158 active+clean; 2.3 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-20T22:12:07.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:07 smithi060 ceph-mon[29694]: 5.1 scrub starts 2024-09-20T22:12:07.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:07 smithi060 ceph-mon[29694]: 5.1 scrub ok 2024-09-20T22:12:07.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:07 smithi060 ceph-mon[29694]: 3.a scrub starts 2024-09-20T22:12:07.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:07 smithi060 ceph-mon[29694]: 3.a scrub ok 2024-09-20T22:12:07.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:07 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/678816756' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:12:07.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:07 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:12:07.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:07 smithi060 ceph-mon[29694]: 6.1f scrub starts 2024-09-20T22:12:07.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:07 smithi060 ceph-mon[29694]: 6.1f scrub ok 2024-09-20T22:12:07.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:07 smithi060 ceph-mon[40290]: pgmap v494: 158 pgs: 158 active+clean; 2.3 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-20T22:12:07.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:07 smithi060 ceph-mon[40290]: 5.1 scrub starts 2024-09-20T22:12:07.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:07 smithi060 ceph-mon[40290]: 5.1 scrub ok 2024-09-20T22:12:07.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:07 smithi060 ceph-mon[40290]: 3.a scrub starts 2024-09-20T22:12:07.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:07 smithi060 ceph-mon[40290]: 3.a scrub ok 2024-09-20T22:12:07.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:07 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/678816756' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:12:07.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:07 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:12:07.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:07 smithi060 ceph-mon[40290]: 6.1f scrub starts 2024-09-20T22:12:07.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:07 smithi060 ceph-mon[40290]: 6.1f scrub ok 2024-09-20T22:12:07.598 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:07.639 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:07.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:07.640 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:07.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:07.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:07 smithi067 ceph-mon[33511]: pgmap v494: 158 pgs: 158 active+clean; 2.3 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 112 MiB/s wr, 1.79k op/s 2024-09-20T22:12:07.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:07 smithi067 ceph-mon[33511]: 5.1 scrub starts 2024-09-20T22:12:07.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:07 smithi067 ceph-mon[33511]: 5.1 scrub ok 2024-09-20T22:12:07.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:07 smithi067 ceph-mon[33511]: 3.a scrub starts 2024-09-20T22:12:07.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:07 smithi067 ceph-mon[33511]: 3.a scrub ok 2024-09-20T22:12:07.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:07 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/678816756' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:12:07.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:07 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:12:07.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:07 smithi067 ceph-mon[33511]: 6.1f scrub starts 2024-09-20T22:12:07.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:07 smithi067 ceph-mon[33511]: 6.1f scrub ok 2024-09-20T22:12:07.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:07.764 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:07.764 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:07.865 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:07.893 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:12:07.893275+0000 min lat: 0.0012684 max lat: 0.100157 avg lat: 0.00722587 2024-09-20T22:12:07.893 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:12:07.893 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 20 16 44282 44266 138.292 100.562 0.00734127 0.00722587 2024-09-20T22:12:07.902 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:07.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:07.903 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:08.004 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:08.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:08.033 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:08.033 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:08.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:08.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:08.179 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:08.179 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:08.280 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:08.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:08.313 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:08.313 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:08.414 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:08.467 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:08.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:08.469 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:08.569 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:08.611 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:08.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:08.612 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:08.713 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:08.734 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[40290]: 6.d scrub starts 2024-09-20T22:12:08.734 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[40290]: 6.d scrub ok 2024-09-20T22:12:08.735 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[40290]: 1.5 scrub starts 2024-09-20T22:12:08.735 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[40290]: 1.5 scrub ok 2024-09-20T22:12:08.735 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:12:08.735 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:12:08.735 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:12:08.735 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[40290]: osdmap e302: 8 total, 8 up, 8 in 2024-09-20T22:12:08.735 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[40290]: 6.1d scrub starts 2024-09-20T22:12:08.735 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[40290]: 6.1d scrub ok 2024-09-20T22:12:08.735 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[40290]: 5.3 scrub starts 2024-09-20T22:12:08.735 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[40290]: 5.3 scrub ok 2024-09-20T22:12:08.735 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[29694]: 6.d scrub starts 2024-09-20T22:12:08.736 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[29694]: 6.d scrub ok 2024-09-20T22:12:08.736 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[29694]: 1.5 scrub starts 2024-09-20T22:12:08.736 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[29694]: 1.5 scrub ok 2024-09-20T22:12:08.736 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:12:08.736 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:12:08.736 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:12:08.736 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[29694]: osdmap e302: 8 total, 8 up, 8 in 2024-09-20T22:12:08.736 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[29694]: 6.1d scrub starts 2024-09-20T22:12:08.736 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[29694]: 6.1d scrub ok 2024-09-20T22:12:08.736 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[29694]: 5.3 scrub starts 2024-09-20T22:12:08.736 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:08 smithi060 ceph-mon[29694]: 5.3 scrub ok 2024-09-20T22:12:08.746 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:08.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:08.747 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:08 smithi067 ceph-mon[33511]: 6.d scrub starts 2024-09-20T22:12:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:08 smithi067 ceph-mon[33511]: 6.d scrub ok 2024-09-20T22:12:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:08 smithi067 ceph-mon[33511]: 1.5 scrub starts 2024-09-20T22:12:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:08 smithi067 ceph-mon[33511]: 1.5 scrub ok 2024-09-20T22:12:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:08 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:12:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:08 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:12:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:08 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:12:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:08 smithi067 ceph-mon[33511]: osdmap e302: 8 total, 8 up, 8 in 2024-09-20T22:12:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:08 smithi067 ceph-mon[33511]: 6.1d scrub starts 2024-09-20T22:12:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:08 smithi067 ceph-mon[33511]: 6.1d scrub ok 2024-09-20T22:12:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:08 smithi067 ceph-mon[33511]: 5.3 scrub starts 2024-09-20T22:12:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:08 smithi067 ceph-mon[33511]: 5.3 scrub ok 2024-09-20T22:12:08.848 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:08.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:08.881 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:08.893 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 21 16 45832 45816 136.319 96.875 0.0174508 0.00733008 2024-09-20T22:12:08.922 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:12:08.922 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:12:08.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.922 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:45851 9.1 9:89507b7a:::benchmark_data_smithi060_140132_object45850:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e302)", 2024-09-20T22:12:08.923 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:12:08.908481+0000", 2024-09-20T22:12:08.923 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0094502920000000008, 2024-09-20T22:12:08.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0094862130000000003, 2024-09-20T22:12:08.923 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:08.923 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:08.923 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:12:08.923 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:08.923 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:12:08.923 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:12:08.923 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 45851 2024-09-20T22:12:08.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.923 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:08.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:08.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.908481+0000", 2024-09-20T22:12:08.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:08.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:08.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.908481+0000", 2024-09-20T22:12:08.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:08.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:08.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.908482+0000", 2024-09-20T22:12:08.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1179999999999999e-06 2024-09-20T22:12:08.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:08.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.908506+0000", 2024-09-20T22:12:08.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4009999999999999e-05 2024-09-20T22:12:08.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:08.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.908517+0000", 2024-09-20T22:12:08.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0446e-05 2024-09-20T22:12:08.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:08.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.908524+0000", 2024-09-20T22:12:08.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6109999999999996e-06 2024-09-20T22:12:08.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:08.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.912829+0000", 2024-09-20T22:12:08.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0043041859999999998 2024-09-20T22:12:08.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:08.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.912970+0000", 2024-09-20T22:12:08.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014178800000000001 2024-09-20T22:12:08.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:12:08.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.913104+0000", 2024-09-20T22:12:08.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013314199999999999 2024-09-20T22:12:08.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:08.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.915300+0000", 2024-09-20T22:12:08.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021969110000000002 2024-09-20T22:12:08.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:08.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917783+0000", 2024-09-20T22:12:08.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024824529999999999 2024-09-20T22:12:08.928 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.928 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:08.928 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.928 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:45852 9.8 9:1c49c936:::benchmark_data_smithi060_140132_object45851:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e302)", 2024-09-20T22:12:08.928 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:12:08.908538+0000", 2024-09-20T22:12:08.928 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0093932259999999993, 2024-09-20T22:12:08.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0095241340000000001, 2024-09-20T22:12:08.928 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:08.928 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:08.928 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:08.928 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:08.928 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:12:08.929 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:12:08.929 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 45852 2024-09-20T22:12:08.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.929 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:08.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:08.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.908538+0000", 2024-09-20T22:12:08.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:08.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:08.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.908538+0000", 2024-09-20T22:12:08.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:08.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:08.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.908539+0000", 2024-09-20T22:12:08.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4700000000000001e-07 2024-09-20T22:12:08.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:08.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.908573+0000", 2024-09-20T22:12:08.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3701000000000001e-05 2024-09-20T22:12:08.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:08.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.908581+0000", 2024-09-20T22:12:08.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5739999999999996e-06 2024-09-20T22:12:08.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:08.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.908587+0000", 2024-09-20T22:12:08.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4990000000000002e-06 2024-09-20T22:12:08.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:08.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.908643+0000", 2024-09-20T22:12:08.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6406000000000002e-05 2024-09-20T22:12:08.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:08.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.908787+0000", 2024-09-20T22:12:08.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014360299999999999 2024-09-20T22:12:08.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,5", 2024-09-20T22:12:08.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.908930+0000", 2024-09-20T22:12:08.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014289399999999999 2024-09-20T22:12:08.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:08.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917510+0000", 2024-09-20T22:12:08.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0085808129999999996 2024-09-20T22:12:08.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:08.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917541+0000", 2024-09-20T22:12:08.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1007000000000003e-05 2024-09-20T22:12:08.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:08.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917981+0000", 2024-09-20T22:12:08.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00043919599999999999 2024-09-20T22:12:08.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:08.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.918032+0000", 2024-09-20T22:12:08.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1762999999999999e-05 2024-09-20T22:12:08.933 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.934 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:08.934 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.934 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:45854 9.2 9:44d9a27d:::benchmark_data_smithi060_140132_object45853:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e302)", 2024-09-20T22:12:08.934 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:12:08.910916+0000", 2024-09-20T22:12:08.934 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0070158750000000004, 2024-09-20T22:12:08.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0072558359999999999, 2024-09-20T22:12:08.934 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:08.934 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:08.934 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:12:08.934 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:08.934 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:12:08.934 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:12:08.935 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 45854 2024-09-20T22:12:08.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.935 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:08.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:08.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.910916+0000", 2024-09-20T22:12:08.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:08.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:08.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.910916+0000", 2024-09-20T22:12:08.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:08.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:08.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.910917+0000", 2024-09-20T22:12:08.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.099e-06 2024-09-20T22:12:08.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:08.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.910947+0000", 2024-09-20T22:12:08.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9978999999999999e-05 2024-09-20T22:12:08.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:08.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.910958+0000", 2024-09-20T22:12:08.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1119e-05 2024-09-20T22:12:08.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:08.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.910965+0000", 2024-09-20T22:12:08.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2830000000000004e-06 2024-09-20T22:12:08.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:08.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.911026+0000", 2024-09-20T22:12:08.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1371999999999994e-05 2024-09-20T22:12:08.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:08.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.911163+0000", 2024-09-20T22:12:08.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136878 2024-09-20T22:12:08.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:12:08.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.911300+0000", 2024-09-20T22:12:08.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013676699999999999 2024-09-20T22:12:08.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:08.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.913154+0000", 2024-09-20T22:12:08.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0018533969999999999 2024-09-20T22:12:08.938 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.938 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:08.938 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.939 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:45855 9.8 9:12215a1a:::benchmark_data_smithi060_140132_object45854:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e302)", 2024-09-20T22:12:08.939 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:12:08.913668+0000", 2024-09-20T22:12:08.939 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0042630639999999996, 2024-09-20T22:12:08.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0045941289999999997, 2024-09-20T22:12:08.939 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:08.939 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:08.939 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:12:08.939 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:08.939 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:12:08.939 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:12:08.940 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 45855 2024-09-20T22:12:08.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.940 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:08.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:08.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.913668+0000", 2024-09-20T22:12:08.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:08.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:08.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.913668+0000", 2024-09-20T22:12:08.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:08.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:08.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.913670+0000", 2024-09-20T22:12:08.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.226e-06 2024-09-20T22:12:08.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:08.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.913703+0000", 2024-09-20T22:12:08.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3770999999999999e-05 2024-09-20T22:12:08.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:08.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.913728+0000", 2024-09-20T22:12:08.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4811999999999999e-05 2024-09-20T22:12:08.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:08.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.913737+0000", 2024-09-20T22:12:08.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4889999999999995e-06 2024-09-20T22:12:08.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:08.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.913822+0000", 2024-09-20T22:12:08.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5650999999999997e-05 2024-09-20T22:12:08.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:08.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.913964+0000", 2024-09-20T22:12:08.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014199500000000001 2024-09-20T22:12:08.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,5", 2024-09-20T22:12:08.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.914121+0000", 2024-09-20T22:12:08.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015664100000000001 2024-09-20T22:12:08.943 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.943 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:08.943 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.943 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.15594.0:45856 9.4 e302/286 9:276423e9:::benchmark_data_smithi060_140132_object45855:head v 302'2838, mlcod=302'2836)", 2024-09-20T22:12:08.943 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:12:08.913859+0000", 2024-09-20T22:12:08.943 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0040729260000000002, 2024-09-20T22:12:08.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0044753630000000004, 2024-09-20T22:12:08.943 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:08.944 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:08.944 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:12:08.944 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:08.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:08.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.913859+0000", 2024-09-20T22:12:08.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:08.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:08.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.913859+0000", 2024-09-20T22:12:08.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:08.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:08.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.913859+0000", 2024-09-20T22:12:08.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1799999999999995e-07 2024-09-20T22:12:08.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:08.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.913883+0000", 2024-09-20T22:12:08.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3623999999999999e-05 2024-09-20T22:12:08.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:08.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.913892+0000", 2024-09-20T22:12:08.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2029999999999998e-06 2024-09-20T22:12:08.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:08.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.913898+0000", 2024-09-20T22:12:08.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0020000000000003e-06 2024-09-20T22:12:08.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:08.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.913964+0000", 2024-09-20T22:12:08.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5720000000000001e-05 2024-09-20T22:12:08.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:08.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.913983+0000", 2024-09-20T22:12:08.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9043e-05 2024-09-20T22:12:08.947 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.947 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:08.947 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.947 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.15594.0:45866 9.4 e302/286)", 2024-09-20T22:12:08.947 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:12:08.917046+0000", 2024-09-20T22:12:08.947 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00088546800000000004, 2024-09-20T22:12:08.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001341708, 2024-09-20T22:12:08.947 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:08.947 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:08.947 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:12:08.947 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:08.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:08.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917046+0000", 2024-09-20T22:12:08.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:08.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:08.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917046+0000", 2024-09-20T22:12:08.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:08.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:08.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917047+0000", 2024-09-20T22:12:08.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.63e-07 2024-09-20T22:12:08.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:08.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917068+0000", 2024-09-20T22:12:08.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1146e-05 2024-09-20T22:12:08.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:08.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917076+0000", 2024-09-20T22:12:08.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1599999999999998e-06 2024-09-20T22:12:08.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:08.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917081+0000", 2024-09-20T22:12:08.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3199999999999999e-06 2024-09-20T22:12:08.949 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.950 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:08.950 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.950 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:45860 9.c 9.1f40f00c (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e302)", 2024-09-20T22:12:08.950 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:12:08.917102+0000", 2024-09-20T22:12:08.950 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.000829889, 2024-09-20T22:12:08.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001336477, 2024-09-20T22:12:08.950 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:08.951 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:08.951 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:12:08.951 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:08.951 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:12:08.951 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:12:08.951 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 45860 2024-09-20T22:12:08.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.951 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:08.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:08.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917102+0000", 2024-09-20T22:12:08.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:08.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:08.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917102+0000", 2024-09-20T22:12:08.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:08.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:08.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917103+0000", 2024-09-20T22:12:08.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.483e-06 2024-09-20T22:12:08.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:08.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917128+0000", 2024-09-20T22:12:08.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5219999999999999e-05 2024-09-20T22:12:08.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:08.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917138+0000", 2024-09-20T22:12:08.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5440000000000004e-06 2024-09-20T22:12:08.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:08.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917144+0000", 2024-09-20T22:12:08.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1109999999999996e-06 2024-09-20T22:12:08.953 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.953 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:08.953 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.954 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:45864 9.2 9:477aa6aa:::benchmark_data_smithi060_140132_object45863:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e302)", 2024-09-20T22:12:08.954 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:12:08.917154+0000", 2024-09-20T22:12:08.954 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00077736500000000002, 2024-09-20T22:12:08.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001344589, 2024-09-20T22:12:08.954 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:08.954 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:08.954 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:12:08.954 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:08.954 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:12:08.954 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:12:08.954 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 45864 2024-09-20T22:12:08.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.955 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:08.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:08.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917154+0000", 2024-09-20T22:12:08.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:08.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:08.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917154+0000", 2024-09-20T22:12:08.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:08.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:08.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917155+0000", 2024-09-20T22:12:08.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1500000000000005e-07 2024-09-20T22:12:08.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:08.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917178+0000", 2024-09-20T22:12:08.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2884e-05 2024-09-20T22:12:08.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:08.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917184+0000", 2024-09-20T22:12:08.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6069999999999996e-06 2024-09-20T22:12:08.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:08.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917189+0000", 2024-09-20T22:12:08.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0699999999999997e-06 2024-09-20T22:12:08.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:08.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917251+0000", 2024-09-20T22:12:08.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2193999999999994e-05 2024-09-20T22:12:08.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:08.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917392+0000", 2024-09-20T22:12:08.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140903 2024-09-20T22:12:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:12:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.917530+0000", 2024-09-20T22:12:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013761700000000001 2024-09-20T22:12:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:45868 9.8 9:11159989:::benchmark_data_smithi060_140132_object45867:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e302)", 2024-09-20T22:12:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:12:08.918038+0000", 2024-09-20T22:12:08.958 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4294967295.9998932, 2024-09-20T22:12:08.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000541783, 2024-09-20T22:12:08.959 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:08.959 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:08.959 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:12:08.959 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:08.959 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:12:08.959 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:12:08.959 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 45868 2024-09-20T22:12:08.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.959 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:08.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:08.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.918038+0000", 2024-09-20T22:12:08.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:08.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:08.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.918038+0000", 2024-09-20T22:12:08.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:08.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:08.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.918040+0000", 2024-09-20T22:12:08.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.158e-06 2024-09-20T22:12:08.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:08.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.918065+0000", 2024-09-20T22:12:08.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5848000000000001e-05 2024-09-20T22:12:08.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:08.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.918077+0000", 2024-09-20T22:12:08.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1375999999999999e-05 2024-09-20T22:12:08.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:08.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.918085+0000", 2024-09-20T22:12:08.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.507e-06 2024-09-20T22:12:08.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:08.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.918259+0000", 2024-09-20T22:12:08.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017383500000000001 2024-09-20T22:12:08.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:08.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.918387+0000", 2024-09-20T22:12:08.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000127539 2024-09-20T22:12:08.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:08.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:08.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,5", 2024-09-20T22:12:08.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:08.918524+0000", 2024-09-20T22:12:08.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000137346 2024-09-20T22:12:08.962 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.962 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:08.962 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.962 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:08.963 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:12:08.963 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 9 2024-09-20T22:12:08.963 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:12:08.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:09.021 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:09.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:09.022 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:09.123 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:09.159 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:09.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:09.160 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:09.260 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:09.293 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:09.296 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:09.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:09.297 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:09.398 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:09.429 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:09.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:09.431 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:09.531 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:09.565 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:09.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:09.566 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:09.667 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:09.703 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:09.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:09.704 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:09 smithi067 ceph-mon[33511]: pgmap v496: 158 pgs: 158 active+clean; 2.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.1 KiB/s rd, 135 MiB/s wr, 2.15k op/s 2024-09-20T22:12:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:09 smithi067 ceph-mon[33511]: 4.1 scrub starts 2024-09-20T22:12:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:09 smithi067 ceph-mon[33511]: 4.1 scrub ok 2024-09-20T22:12:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:09 smithi067 ceph-mon[33511]: 5.a scrub starts 2024-09-20T22:12:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:09 smithi067 ceph-mon[33511]: 5.a scrub ok 2024-09-20T22:12:09.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:09 smithi067 ceph-mon[33511]: 3.13 scrub starts 2024-09-20T22:12:09.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:09 smithi067 ceph-mon[33511]: 3.13 scrub ok 2024-09-20T22:12:09.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:09 smithi067 ceph-mon[33511]: 4.16 scrub starts 2024-09-20T22:12:09.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:09 smithi067 ceph-mon[33511]: 4.16 scrub ok 2024-09-20T22:12:09.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:09 smithi067 ceph-mon[33511]: 3.b scrub starts 2024-09-20T22:12:09.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:09 smithi067 ceph-mon[33511]: 3.b scrub ok 2024-09-20T22:12:09.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[29694]: pgmap v496: 158 pgs: 158 active+clean; 2.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.1 KiB/s rd, 135 MiB/s wr, 2.15k op/s 2024-09-20T22:12:09.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[29694]: 4.1 scrub starts 2024-09-20T22:12:09.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[29694]: 4.1 scrub ok 2024-09-20T22:12:09.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[29694]: 5.a scrub starts 2024-09-20T22:12:09.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[29694]: 5.a scrub ok 2024-09-20T22:12:09.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[29694]: 3.13 scrub starts 2024-09-20T22:12:09.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[29694]: 3.13 scrub ok 2024-09-20T22:12:09.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[29694]: 4.16 scrub starts 2024-09-20T22:12:09.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[29694]: 4.16 scrub ok 2024-09-20T22:12:09.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[29694]: 3.b scrub starts 2024-09-20T22:12:09.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[29694]: 3.b scrub ok 2024-09-20T22:12:09.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[40290]: pgmap v496: 158 pgs: 158 active+clean; 2.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.1 KiB/s rd, 135 MiB/s wr, 2.15k op/s 2024-09-20T22:12:09.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[40290]: 4.1 scrub starts 2024-09-20T22:12:09.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[40290]: 4.1 scrub ok 2024-09-20T22:12:09.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[40290]: 5.a scrub starts 2024-09-20T22:12:09.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[40290]: 5.a scrub ok 2024-09-20T22:12:09.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[40290]: 3.13 scrub starts 2024-09-20T22:12:09.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[40290]: 3.13 scrub ok 2024-09-20T22:12:09.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[40290]: 4.16 scrub starts 2024-09-20T22:12:09.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[40290]: 4.16 scrub ok 2024-09-20T22:12:09.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[40290]: 3.b scrub starts 2024-09-20T22:12:09.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:09 smithi060 ceph-mon[40290]: 3.b scrub ok 2024-09-20T22:12:09.805 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:09.842 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:09.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:09.843 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:09.857 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:09.895 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 22 15 47564 47549 135.038 108.312 0.0100072 0.00740047 2024-09-20T22:12:09.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T22:12:09.944 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:09.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:09.998 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:09.998 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:10.099 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:10.136 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:10.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:10.138 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:10.239 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:10.296 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:10.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:10.297 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:10.398 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:10.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:10.443 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:10.443 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:10.544 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:10.550 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:10 smithi060 ceph-mon[29694]: 4.13 scrub starts 2024-09-20T22:12:10.550 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:10 smithi060 ceph-mon[29694]: 4.13 scrub ok 2024-09-20T22:12:10.551 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:10 smithi060 ceph-mon[40290]: 4.13 scrub starts 2024-09-20T22:12:10.552 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:10 smithi060 ceph-mon[40290]: 4.13 scrub ok 2024-09-20T22:12:10.576 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:10.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:10.577 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:10.678 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:10.710 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:10.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:10.711 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:10 smithi067 ceph-mon[33511]: 4.13 scrub starts 2024-09-20T22:12:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:10 smithi067 ceph-mon[33511]: 4.13 scrub ok 2024-09-20T22:12:10.812 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:10.856 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:10.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:10.857 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:10.895 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 23 16 49079 49063 133.281 94.625 0.00451393 0.00749728 2024-09-20T22:12:10.958 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:10.987 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:10.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:10.988 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:11.089 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:11.119 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:11.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:11.120 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:11.220 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:11.250 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:11.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:11.251 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:11.352 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:11.382 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:11.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:11.383 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:11.484 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:11.485 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:11 smithi060 ceph-mon[29694]: pgmap v497: 158 pgs: 158 active+clean; 2.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 120 MiB/s wr, 1.92k op/s 2024-09-20T22:12:11.485 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:11 smithi060 ceph-mon[29694]: 4.1a scrub starts 2024-09-20T22:12:11.485 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:11 smithi060 ceph-mon[29694]: 4.1a scrub ok 2024-09-20T22:12:11.485 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:11 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:12:11.485 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:11 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:12:11.485 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:11 smithi060 ceph-mon[40290]: pgmap v497: 158 pgs: 158 active+clean; 2.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 120 MiB/s wr, 1.92k op/s 2024-09-20T22:12:11.485 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:11 smithi060 ceph-mon[40290]: 4.1a scrub starts 2024-09-20T22:12:11.485 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:11 smithi060 ceph-mon[40290]: 4.1a scrub ok 2024-09-20T22:12:11.486 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:11 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:12:11.486 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:11 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:12:11.510 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:11.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:11.511 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:11.590 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:12:11.598 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:12:11.611 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:11.664 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:11 smithi067 ceph-mon[33511]: pgmap v497: 158 pgs: 158 active+clean; 2.5 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 120 MiB/s wr, 1.92k op/s 2024-09-20T22:12:11.664 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:11 smithi067 ceph-mon[33511]: 4.1a scrub starts 2024-09-20T22:12:11.664 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:11 smithi067 ceph-mon[33511]: 4.1a scrub ok 2024-09-20T22:12:11.664 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:11 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:12:11.664 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:11 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:12:11.668 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:11.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:11.669 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:11.698 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:12:11.698 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:12:11.738 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:12:11.738 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:12:11.770 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:11.780 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:12:11.780 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:12:11.808 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:11.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:11.809 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:11.821 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:12:11.821 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:12:11.857 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:12:11.858 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:12:11.893 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:12:11.893 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:12:11.895 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 24 16 50342 50326 131.015 78.9375 0.00781889 0.00762674 2024-09-20T22:12:11.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:11.928 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:12:11.928 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:12:11.951 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:11.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:11.952 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:11.968 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:12:11.968 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:12:12.053 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:12.090 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:12.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:12.091 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:12.192 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:12.228 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:12.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:12.229 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:12.330 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:12.363 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:12.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:12.364 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:12.465 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:12.472 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2024-09-20T22:12:11.357595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2965445147,"num_objects":45439,"num_object_clones":0,"num_object_copies":136317,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45439,"num_whiteouts":0,"num_read":1256,"num_read_kb":1008,"num_write":45676,"num_write_kb":2895405,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":137,"num_bytes_recovered":921003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":47184,"ondisk_log_size":47184,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":12214824,"kb_used_data":8730788,"kb_used_omap":15,"kb_used_meta":3483888,"kb_avail":737746392,"statfs":{"total":767960285184,"available":755452305408,"internally_reserved":0,"allocated":8940326912,"data_stored":8935122161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":3567502099},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[10,7,16,8,11],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":884539392,"num_objects":13497,"num_object_clones":0,"num_object_copies":40491,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13497,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":13497,"num_write_kb":863808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.221554"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":524,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.245305+0000","last_change":"2024-09-20T22:11:12.498436+0000","last_active":"2024-09-20T22:11:59.245305+0000","last_peered":"2024-09-20T22:11:59.245305+0000","last_clean":"2024-09-20T22:11:59.245305+0000","last_became_active":"2024-09-20T22:11:12.498141+0000","last_became_peered":"2024-09-20T22:11:12.498141+0000","last_unstale":"2024-09-20T22:11:59.245305+0000","last_undegraded":"2024-09-20T22:11:59.245305+0000","last_fullsized":"2024-09-20T22:11:59.245305+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:31.859259+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_clean_scrub_stamp":"2024-09-20T22:10:31.859259+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:59.578576+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"252'13","reported_seq":539,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.270747+0000","last_change":"2024-09-20T22:10:31.568218+0000","last_active":"2024-09-20T22:12:07.270747+0000","last_peered":"2024-09-20T22:12:07.270747+0000","last_clean":"2024-09-20T22:12:07.270747+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:12:07.270747+0000","last_undegraded":"2024-09-20T22:12:07.270747+0000","last_fullsized":"2024-09-20T22:12:07.270747+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"252'13","last_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_deep_scrub":"252'13","last_deep_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_clean_scrub_stamp":"2024-09-20T22:10:31.568098+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":394,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.290677+0000","last_change":"2024-09-20T22:10:22.311182+0000","last_active":"2024-09-20T22:12:07.290677+0000","last_peered":"2024-09-20T22:12:07.290677+0000","last_clean":"2024-09-20T22:12:07.290677+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:12:07.290677+0000","last_undegraded":"2024-09-20T22:12:07.290677+0000","last_fullsized":"2024-09-20T22:12:07.290677+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:22.311078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:22.311078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":515,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.270920+0000","last_change":"2024-09-20T22:11:30.486243+0000","last_active":"2024-09-20T22:12:07.270920+0000","last_peered":"2024-09-20T22:12:07.270920+0000","last_clean":"2024-09-20T22:12:07.270920+0000","last_became_active":"2024-09-20T22:11:30.485735+0000","last_became_peered":"2024-09-20T22:11:30.485735+0000","last_unstale":"2024-09-20T22:12:07.270920+0000","last_undegraded":"2024-09-20T22:12:07.270920+0000","last_fullsized":"2024-09-20T22:12:07.270920+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:23.640516+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:23.640516+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":500,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.594212+0000","last_change":"2024-09-20T22:11:59.594257+0000","last_active":"2024-09-20T22:11:59.594212+0000","last_peered":"2024-09-20T22:11:59.594212+0000","last_clean":"2024-09-20T22:11:59.594212+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:11:59.594212+0000","last_undegraded":"2024-09-20T22:11:59.594212+0000","last_fullsized":"2024-09-20T22:11:59.594212+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:11:59.594135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:59.594135+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:59.916366+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":512,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.249318+0000","last_change":"2024-09-20T22:11:42.438241+0000","last_active":"2024-09-20T22:11:59.249318+0000","last_peered":"2024-09-20T22:11:59.249318+0000","last_clean":"2024-09-20T22:11:59.249318+0000","last_became_active":"2024-09-20T22:02:50.107154+0000","last_became_peered":"2024-09-20T22:02:50.107154+0000","last_unstale":"2024-09-20T22:11:59.249318+0000","last_undegraded":"2024-09-20T22:11:59.249318+0000","last_fullsized":"2024-09-20T22:11:59.249318+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:42.438172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:11:42.438172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:57.591083+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"71'15","reported_seq":532,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:12:00.561977+0000","last_change":"2024-09-20T22:12:00.562094+0000","last_active":"2024-09-20T22:12:00.561977+0000","last_peered":"2024-09-20T22:12:00.561977+0000","last_clean":"2024-09-20T22:12:00.561977+0000","last_became_active":"2024-09-20T22:02:52.282101+0000","last_became_peered":"2024-09-20T22:02:52.282101+0000","last_unstale":"2024-09-20T22:12:00.561977+0000","last_undegraded":"2024-09-20T22:12:00.561977+0000","last_fullsized":"2024-09-20T22:12:00.561977+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:12:00.561897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:00.561897+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:20.644760+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":507,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.270742+0000","last_change":"2024-09-20T22:10:26.558035+0000","last_active":"2024-09-20T22:12:07.270742+0000","last_peered":"2024-09-20T22:12:07.270742+0000","last_clean":"2024-09-20T22:12:07.270742+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:12:07.270742+0000","last_undegraded":"2024-09-20T22:12:07.270742+0000","last_fullsized":"2024-09-20T22:12:07.270742+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:26.557925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:26.557925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":537,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.279167+0000","last_change":"2024-09-20T22:12:01.173031+0000","last_active":"2024-09-20T22:12:07.279167+0000","last_peered":"2024-09-20T22:12:07.279167+0000","last_clean":"2024-09-20T22:12:07.279167+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:12:07.279167+0000","last_undegraded":"2024-09-20T22:12:07.279167+0000","last_fullsized":"2024-09-20T22:12:07.279167+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:01.172926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:12:01.172926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:12.397406+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"71'9","reported_seq":534,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:10.172349+0000","last_change":"2024-09-20T22:12:10.172417+0000","last_active":"2024-09-20T22:12:10.172349+0000","last_peered":"2024-09-20T22:12:10.172349+0000","last_clean":"2024-09-20T22:12:10.172349+0000","last_became_active":"2024-09-20T22:02:52.279526+0000","last_became_peered":"2024-09-20T22:02:52.279526+0000","last_unstale":"2024-09-20T22:12:10.172349+0000","last_undegraded":"2024-09-20T22:12:10.172349+0000","last_fullsized":"2024-09-20T22:12:10.172349+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:12:10.172212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:10.172212+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:33.564324+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":57,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.278493+0000","last_change":"2024-09-20T22:11:31.101654+0000","last_active":"2024-09-20T22:12:07.278493+0000","last_peered":"2024-09-20T22:12:07.278493+0000","last_clean":"2024-09-20T22:12:07.278493+0000","last_became_active":"2024-09-20T22:11:30.485036+0000","last_became_peered":"2024-09-20T22:11:30.485036+0000","last_unstale":"2024-09-20T22:12:07.278493+0000","last_undegraded":"2024-09-20T22:12:07.278493+0000","last_fullsized":"2024-09-20T22:12:07.278493+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:31.101573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:31.101573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:32.848267+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":500,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.244904+0000","last_change":"2024-09-20T22:10:32.855540+0000","last_active":"2024-09-20T22:11:59.244904+0000","last_peered":"2024-09-20T22:11:59.244904+0000","last_clean":"2024-09-20T22:11:59.244904+0000","last_became_active":"2024-09-20T22:06:54.841095+0000","last_became_peered":"2024-09-20T22:06:54.841095+0000","last_unstale":"2024-09-20T22:11:59.244904+0000","last_undegraded":"2024-09-20T22:11:59.244904+0000","last_fullsized":"2024-09-20T22:11:59.244904+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:32.855440+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:32.855440+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:59.578648+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":542,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.278262+0000","last_change":"2024-09-20T22:11:40.127688+0000","last_active":"2024-09-20T22:12:07.278262+0000","last_peered":"2024-09-20T22:12:07.278262+0000","last_clean":"2024-09-20T22:12:07.278262+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:12:07.278262+0000","last_undegraded":"2024-09-20T22:12:07.278262+0000","last_fullsized":"2024-09-20T22:12:07.278262+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:40.127597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:30.215263+0000","last_clean_scrub_stamp":"2024-09-20T22:11:40.127597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:59.820229+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"71'5","reported_seq":541,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.271066+0000","last_change":"2024-09-20T22:11:03.163814+0000","last_active":"2024-09-20T22:12:07.271066+0000","last_peered":"2024-09-20T22:12:07.271066+0000","last_clean":"2024-09-20T22:12:07.271066+0000","last_became_active":"2024-09-20T22:08:22.144364+0000","last_became_peered":"2024-09-20T22:08:22.144364+0000","last_unstale":"2024-09-20T22:12:07.271066+0000","last_undegraded":"2024-09-20T22:12:07.271066+0000","last_fullsized":"2024-09-20T22:12:07.271066+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'5","last_scrub_stamp":"2024-09-20T22:11:03.163739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:03.163739+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:25.185084+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":515,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.274761+0000","last_change":"2024-09-20T22:10:15.024751+0000","last_active":"2024-09-20T22:12:07.274761+0000","last_peered":"2024-09-20T22:12:07.274761+0000","last_clean":"2024-09-20T22:12:07.274761+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:12:07.274761+0000","last_undegraded":"2024-09-20T22:12:07.274761+0000","last_fullsized":"2024-09-20T22:12:07.274761+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:15.024624+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:15.024624+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":57,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.277934+0000","last_change":"2024-09-20T22:11:34.036027+0000","last_active":"2024-09-20T22:12:07.277934+0000","last_peered":"2024-09-20T22:12:07.277934+0000","last_clean":"2024-09-20T22:12:07.277934+0000","last_became_active":"2024-09-20T22:11:30.487137+0000","last_became_peered":"2024-09-20T22:11:30.487137+0000","last_unstale":"2024-09-20T22:12:07.277934+0000","last_undegraded":"2024-09-20T22:12:07.277934+0000","last_fullsized":"2024-09-20T22:12:07.277934+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:34.035938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.035938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:57.022848+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":503,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.271768+0000","last_change":"2024-09-20T22:09:51.759271+0000","last_active":"2024-09-20T22:12:07.271768+0000","last_peered":"2024-09-20T22:12:07.271768+0000","last_clean":"2024-09-20T22:12:07.271768+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:12:07.271768+0000","last_undegraded":"2024-09-20T22:12:07.271768+0000","last_fullsized":"2024-09-20T22:12:07.271768+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:51.759042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:09:51.759042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":511,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.246766+0000","last_change":"2024-09-20T22:11:36.700618+0000","last_active":"2024-09-20T22:11:59.246766+0000","last_peered":"2024-09-20T22:11:59.246766+0000","last_clean":"2024-09-20T22:11:59.246766+0000","last_became_active":"2024-09-20T22:02:50.111517+0000","last_became_peered":"2024-09-20T22:02:50.111517+0000","last_unstale":"2024-09-20T22:11:59.246766+0000","last_undegraded":"2024-09-20T22:11:59.246766+0000","last_fullsized":"2024-09-20T22:11:59.246766+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:36.700507+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:11:36.700507+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:44.350271+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"195'22","reported_seq":79,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.277864+0000","last_change":"2024-09-20T22:11:12.545971+0000","last_active":"2024-09-20T22:12:07.277864+0000","last_peered":"2024-09-20T22:12:07.277864+0000","last_clean":"2024-09-20T22:12:07.277864+0000","last_became_active":"2024-09-20T22:11:12.507311+0000","last_became_peered":"2024-09-20T22:11:12.507311+0000","last_unstale":"2024-09-20T22:12:07.277864+0000","last_undegraded":"2024-09-20T22:12:07.277864+0000","last_fullsized":"2024-09-20T22:12:07.277864+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'22","last_scrub_stamp":"2024-09-20T22:10:16.538107+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:10:16.538107+0000","objects_scrubbed":7,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":352,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.291574+0000","last_change":"2024-09-20T22:11:09.157162+0000","last_active":"2024-09-20T22:12:07.291574+0000","last_peered":"2024-09-20T22:12:07.291574+0000","last_clean":"2024-09-20T22:12:07.291574+0000","last_became_active":"2024-09-20T22:06:54.848505+0000","last_became_peered":"2024-09-20T22:06:54.848505+0000","last_unstale":"2024-09-20T22:12:07.291574+0000","last_undegraded":"2024-09-20T22:12:07.291574+0000","last_fullsized":"2024-09-20T22:12:07.291574+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:09.157055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:09.157055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:24.968380+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":512,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.252599+0000","last_change":"2024-09-20T22:11:30.487554+0000","last_active":"2024-09-20T22:11:59.252599+0000","last_peered":"2024-09-20T22:11:59.252599+0000","last_clean":"2024-09-20T22:11:59.252599+0000","last_became_active":"2024-09-20T22:11:30.487199+0000","last_became_peered":"2024-09-20T22:11:30.487199+0000","last_unstale":"2024-09-20T22:11:59.252599+0000","last_undegraded":"2024-09-20T22:11:59.252599+0000","last_fullsized":"2024-09-20T22:11:59.252599+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:52.317931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:52.317931+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":464,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.278316+0000","last_change":"2024-09-20T22:10:59.182372+0000","last_active":"2024-09-20T22:12:07.278316+0000","last_peered":"2024-09-20T22:12:07.278316+0000","last_clean":"2024-09-20T22:12:07.278316+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:12:07.278316+0000","last_undegraded":"2024-09-20T22:12:07.278316+0000","last_fullsized":"2024-09-20T22:12:07.278316+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:59.182274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:59.182274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"250'17","reported_seq":536,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.249008+0000","last_change":"2024-09-20T22:10:27.157652+0000","last_active":"2024-09-20T22:11:59.249008+0000","last_peered":"2024-09-20T22:11:59.249008+0000","last_clean":"2024-09-20T22:11:59.249008+0000","last_became_active":"2024-09-20T22:06:54.830946+0000","last_became_peered":"2024-09-20T22:06:54.830946+0000","last_unstale":"2024-09-20T22:11:59.249008+0000","last_undegraded":"2024-09-20T22:11:59.249008+0000","last_fullsized":"2024-09-20T22:11:59.249008+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"250'17","last_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_deep_scrub":"250'17","last_deep_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_clean_scrub_stamp":"2024-09-20T22:10:27.157501+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":506,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.271004+0000","last_change":"2024-09-20T22:11:34.369112+0000","last_active":"2024-09-20T22:12:07.271004+0000","last_peered":"2024-09-20T22:12:07.271004+0000","last_clean":"2024-09-20T22:12:07.271004+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:12:07.271004+0000","last_undegraded":"2024-09-20T22:12:07.271004+0000","last_fullsized":"2024-09-20T22:12:07.271004+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:34.369006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.369006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:49.101366+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":504,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.274998+0000","last_change":"2024-09-20T22:11:39.286967+0000","last_active":"2024-09-20T22:12:07.274998+0000","last_peered":"2024-09-20T22:12:07.274998+0000","last_clean":"2024-09-20T22:12:07.274998+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:12:07.274998+0000","last_undegraded":"2024-09-20T22:12:07.274998+0000","last_fullsized":"2024-09-20T22:12:07.274998+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:11:39.286813+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:11:39.286813+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:43.641876+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"289'2","reported_seq":354,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.290736+0000","last_change":"2024-09-20T22:11:37.189066+0000","last_active":"2024-09-20T22:12:07.290736+0000","last_peered":"2024-09-20T22:12:07.290736+0000","last_clean":"2024-09-20T22:12:07.290736+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:12:07.290736+0000","last_undegraded":"2024-09-20T22:12:07.290736+0000","last_fullsized":"2024-09-20T22:12:07.290736+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_clean_scrub_stamp":"2024-09-20T22:11:37.188992+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:55.054428+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"71'10","reported_seq":558,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.244625+0000","last_change":"2024-09-20T22:11:40.608961+0000","last_active":"2024-09-20T22:11:59.244625+0000","last_peered":"2024-09-20T22:11:59.244625+0000","last_clean":"2024-09-20T22:11:59.244625+0000","last_became_active":"2024-09-20T22:11:40.602075+0000","last_became_peered":"2024-09-20T22:11:40.602075+0000","last_unstale":"2024-09-20T22:11:59.244625+0000","last_undegraded":"2024-09-20T22:11:59.244625+0000","last_fullsized":"2024-09-20T22:11:59.244625+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:11:35.700046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:35.700046+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:03.129277+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"291'11","reported_seq":667,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.286607+0000","last_change":"2024-09-20T22:11:41.372331+0000","last_active":"2024-09-20T22:12:07.286607+0000","last_peered":"2024-09-20T22:12:07.286607+0000","last_clean":"2024-09-20T22:12:07.286607+0000","last_became_active":"2024-09-20T22:11:40.601048+0000","last_became_peered":"2024-09-20T22:11:40.601048+0000","last_unstale":"2024-09-20T22:12:07.286607+0000","last_undegraded":"2024-09-20T22:12:07.286607+0000","last_fullsized":"2024-09-20T22:12:07.286607+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'11","last_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_deep_scrub":"291'11","last_deep_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_clean_scrub_stamp":"2024-09-20T22:11:41.372195+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:10.847568+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"301'2612","reported_seq":2652,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:12:06.371743+0000","last_change":"2024-09-20T22:11:35.304533+0000","last_active":"2024-09-20T22:12:06.371743+0000","last_peered":"2024-09-20T22:12:06.371743+0000","last_clean":"2024-09-20T22:12:06.371743+0000","last_became_active":"2024-09-20T22:11:35.304245+0000","last_became_peered":"2024-09-20T22:11:35.304245+0000","last_unstale":"2024-09-20T22:12:06.371743+0000","last_undegraded":"2024-09-20T22:12:06.371743+0000","last_fullsized":"2024-09-20T22:12:06.371743+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":2612,"log_dups_size":0,"ondisk_log_size":2612,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:54.197789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":171180032,"num_objects":2612,"num_object_clones":0,"num_object_copies":7836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2612,"num_write_kb":167168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,2],"acting":[3,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":517,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.278157+0000","last_change":"2024-09-20T22:11:08.187529+0000","last_active":"2024-09-20T22:12:07.278157+0000","last_peered":"2024-09-20T22:12:07.278157+0000","last_clean":"2024-09-20T22:12:07.278157+0000","last_became_active":"2024-09-20T22:08:48.452786+0000","last_became_peered":"2024-09-20T22:08:48.452786+0000","last_unstale":"2024-09-20T22:12:07.278157+0000","last_undegraded":"2024-09-20T22:12:07.278157+0000","last_fullsized":"2024-09-20T22:12:07.278157+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:11:08.187420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:11:08.187420+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:21.119364+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"71'15","reported_seq":280,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.278897+0000","last_change":"2024-09-20T22:11:34.283966+0000","last_active":"2024-09-20T22:12:07.278897+0000","last_peered":"2024-09-20T22:12:07.278897+0000","last_clean":"2024-09-20T22:12:07.278897+0000","last_became_active":"2024-09-20T22:08:22.145408+0000","last_became_peered":"2024-09-20T22:08:22.145408+0000","last_unstale":"2024-09-20T22:12:07.278897+0000","last_undegraded":"2024-09-20T22:12:07.278897+0000","last_fullsized":"2024-09-20T22:12:07.278897+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:11:34.283882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.283882+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:43.345533+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":513,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.255829+0000","last_change":"2024-09-20T22:10:19.208598+0000","last_active":"2024-09-20T22:11:59.255829+0000","last_peered":"2024-09-20T22:11:59.255829+0000","last_clean":"2024-09-20T22:11:59.255829+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:11:59.255829+0000","last_undegraded":"2024-09-20T22:11:59.255829+0000","last_fullsized":"2024-09-20T22:11:59.255829+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:19.208496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:19.208496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"283'11","reported_seq":669,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.272148+0000","last_change":"2024-09-20T22:11:30.493908+0000","last_active":"2024-09-20T22:12:07.272148+0000","last_peered":"2024-09-20T22:12:07.272148+0000","last_clean":"2024-09-20T22:12:07.272148+0000","last_became_active":"2024-09-20T22:11:30.484743+0000","last_became_peered":"2024-09-20T22:11:30.484743+0000","last_unstale":"2024-09-20T22:12:07.272148+0000","last_undegraded":"2024-09-20T22:12:07.272148+0000","last_fullsized":"2024-09-20T22:12:07.272148+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:10:14.680845+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:14.680845+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":57,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.278822+0000","last_change":"2024-09-20T22:12:07.114761+0000","last_active":"2024-09-20T22:12:07.278822+0000","last_peered":"2024-09-20T22:12:07.278822+0000","last_clean":"2024-09-20T22:12:07.278822+0000","last_became_active":"2024-09-20T22:11:30.489416+0000","last_became_peered":"2024-09-20T22:11:30.489416+0000","last_unstale":"2024-09-20T22:12:07.278822+0000","last_undegraded":"2024-09-20T22:12:07.278822+0000","last_fullsized":"2024-09-20T22:12:07.278822+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:07.114531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:07.114531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:22.972933+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"302'3058","reported_seq":3100,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:10.651540+0000","last_change":"2024-09-20T22:11:34.561887+0000","last_active":"2024-09-20T22:12:10.651540+0000","last_peered":"2024-09-20T22:12:10.651540+0000","last_clean":"2024-09-20T22:12:10.651540+0000","last_became_active":"2024-09-20T22:11:34.561538+0000","last_became_peered":"2024-09-20T22:11:34.561538+0000","last_unstale":"2024-09-20T22:12:10.651540+0000","last_undegraded":"2024-09-20T22:12:10.651540+0000","last_fullsized":"2024-09-20T22:12:10.651540+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":3058,"log_dups_size":0,"ondisk_log_size":3058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:50.616555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":200409088,"num_objects":3058,"num_object_clones":0,"num_object_copies":9174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3058,"num_write_kb":195712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":517,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.271965+0000","last_change":"2024-09-20T22:12:07.204327+0000","last_active":"2024-09-20T22:12:07.271965+0000","last_peered":"2024-09-20T22:12:07.271965+0000","last_clean":"2024-09-20T22:12:07.271965+0000","last_became_active":"2024-09-20T22:08:48.452475+0000","last_became_peered":"2024-09-20T22:08:48.452475+0000","last_unstale":"2024-09-20T22:12:07.271965+0000","last_undegraded":"2024-09-20T22:12:07.271965+0000","last_fullsized":"2024-09-20T22:12:07.271965+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:12:07.204213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:12:07.204213+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:17.525006+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"71'18","reported_seq":587,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.248445+0000","last_change":"2024-09-20T22:11:30.218845+0000","last_active":"2024-09-20T22:11:59.248445+0000","last_peered":"2024-09-20T22:11:59.248445+0000","last_clean":"2024-09-20T22:11:59.248445+0000","last_became_active":"2024-09-20T22:10:48.921997+0000","last_became_peered":"2024-09-20T22:10:48.921997+0000","last_unstale":"2024-09-20T22:11:59.248445+0000","last_undegraded":"2024-09-20T22:11:59.248445+0000","last_fullsized":"2024-09-20T22:11:59.248445+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'18","last_scrub_stamp":"2024-09-20T22:11:30.218749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:30.218749+0000","objects_scrubbed":8,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:45.626566+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":511,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:12:04.532295+0000","last_change":"2024-09-20T22:12:04.532330+0000","last_active":"2024-09-20T22:12:04.532295+0000","last_peered":"2024-09-20T22:12:04.532295+0000","last_clean":"2024-09-20T22:12:04.532295+0000","last_became_active":"2024-09-20T22:02:50.113076+0000","last_became_peered":"2024-09-20T22:02:50.113076+0000","last_unstale":"2024-09-20T22:12:04.532295+0000","last_undegraded":"2024-09-20T22:12:04.532295+0000","last_fullsized":"2024-09-20T22:12:04.532295+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:04.532220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:04.532220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:14.077324+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":527,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.270803+0000","last_change":"2024-09-20T22:12:06.219908+0000","last_active":"2024-09-20T22:12:07.270803+0000","last_peered":"2024-09-20T22:12:07.270803+0000","last_clean":"2024-09-20T22:12:07.270803+0000","last_became_active":"2024-09-20T22:10:48.919949+0000","last_became_peered":"2024-09-20T22:10:48.919949+0000","last_unstale":"2024-09-20T22:12:07.270803+0000","last_undegraded":"2024-09-20T22:12:07.270803+0000","last_fullsized":"2024-09-20T22:12:07.270803+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:06.219799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:06.219799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:08.272091+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":511,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.271159+0000","last_change":"2024-09-20T22:11:40.260396+0000","last_active":"2024-09-20T22:12:07.271159+0000","last_peered":"2024-09-20T22:12:07.271159+0000","last_clean":"2024-09-20T22:12:07.271159+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:12:07.271159+0000","last_undegraded":"2024-09-20T22:12:07.271159+0000","last_fullsized":"2024-09-20T22:12:07.271159+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:40.260289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:40.260289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:49.182120+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"301'2670","reported_seq":2710,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:12:06.381838+0000","last_change":"2024-09-20T22:11:34.559451+0000","last_active":"2024-09-20T22:12:06.381838+0000","last_peered":"2024-09-20T22:12:06.381838+0000","last_clean":"2024-09-20T22:12:06.381838+0000","last_became_active":"2024-09-20T22:11:34.558615+0000","last_became_peered":"2024-09-20T22:11:34.558615+0000","last_unstale":"2024-09-20T22:12:06.381838+0000","last_undegraded":"2024-09-20T22:12:06.381838+0000","last_fullsized":"2024-09-20T22:12:06.381838+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":2670,"log_dups_size":0,"ondisk_log_size":2670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:43.687599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174981120,"num_objects":2670,"num_object_clones":0,"num_object_copies":8010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2670,"num_write_kb":170880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":541,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.256195+0000","last_change":"2024-09-20T22:11:39.422439+0000","last_active":"2024-09-20T22:11:59.256195+0000","last_peered":"2024-09-20T22:11:59.256195+0000","last_clean":"2024-09-20T22:11:59.256195+0000","last_became_active":"2024-09-20T22:11:33.560632+0000","last_became_peered":"2024-09-20T22:11:33.560632+0000","last_unstale":"2024-09-20T22:11:59.256195+0000","last_undegraded":"2024-09-20T22:11:59.256195+0000","last_fullsized":"2024-09-20T22:11:59.256195+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:11:39.422355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:11:39.422355+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:08.591531+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"71'14","reported_seq":540,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:08.158662+0000","last_change":"2024-09-20T22:12:08.158731+0000","last_active":"2024-09-20T22:12:08.158662+0000","last_peered":"2024-09-20T22:12:08.158662+0000","last_clean":"2024-09-20T22:12:08.158662+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:12:08.158662+0000","last_undegraded":"2024-09-20T22:12:08.158662+0000","last_fullsized":"2024-09-20T22:12:08.158662+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'14","last_scrub_stamp":"2024-09-20T22:12:08.158574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:08.158574+0000","objects_scrubbed":7,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:20.252532+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":514,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.244929+0000","last_change":"2024-09-20T22:11:04.592252+0000","last_active":"2024-09-20T22:11:59.244929+0000","last_peered":"2024-09-20T22:11:59.244929+0000","last_clean":"2024-09-20T22:11:59.244929+0000","last_became_active":"2024-09-20T22:06:54.841165+0000","last_became_peered":"2024-09-20T22:06:54.841165+0000","last_unstale":"2024-09-20T22:11:59.244929+0000","last_undegraded":"2024-09-20T22:11:59.244929+0000","last_fullsized":"2024-09-20T22:11:59.244929+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:11:04.590441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:11:04.590441+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:16.119865+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"146'10","reported_seq":627,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:12:00.278244+0000","last_change":"2024-09-20T22:10:23.115374+0000","last_active":"2024-09-20T22:12:00.278244+0000","last_peered":"2024-09-20T22:12:00.278244+0000","last_clean":"2024-09-20T22:12:00.278244+0000","last_became_active":"2024-09-20T22:06:54.847742+0000","last_became_peered":"2024-09-20T22:06:54.847742+0000","last_unstale":"2024-09-20T22:12:00.278244+0000","last_undegraded":"2024-09-20T22:12:00.278244+0000","last_fullsized":"2024-09-20T22:12:00.278244+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"146'10","last_scrub_stamp":"2024-09-20T22:10:23.114986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:23.114986+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":503,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.274834+0000","last_change":"2024-09-20T22:10:22.075441+0000","last_active":"2024-09-20T22:12:07.274834+0000","last_peered":"2024-09-20T22:12:07.274834+0000","last_clean":"2024-09-20T22:12:07.274834+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:12:07.274834+0000","last_undegraded":"2024-09-20T22:12:07.274834+0000","last_fullsized":"2024-09-20T22:12:07.274834+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:22.075335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:22.075335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"302'3033","reported_seq":3075,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:10.834162+0000","last_change":"2024-09-20T22:11:34.559393+0000","last_active":"2024-09-20T22:12:10.834162+0000","last_peered":"2024-09-20T22:12:10.834162+0000","last_clean":"2024-09-20T22:12:10.834162+0000","last_became_active":"2024-09-20T22:11:34.558898+0000","last_became_peered":"2024-09-20T22:11:34.558898+0000","last_unstale":"2024-09-20T22:12:10.834162+0000","last_undegraded":"2024-09-20T22:12:10.834162+0000","last_fullsized":"2024-09-20T22:12:10.834162+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":3033,"log_dups_size":0,"ondisk_log_size":3033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:41.066557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198770688,"num_objects":3033,"num_object_clones":0,"num_object_copies":9099,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3033,"num_write_kb":194112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":539,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.286420+0000","last_change":"2024-09-20T22:11:07.362346+0000","last_active":"2024-09-20T22:12:07.286420+0000","last_peered":"2024-09-20T22:12:07.286420+0000","last_clean":"2024-09-20T22:12:07.286420+0000","last_became_active":"2024-09-20T22:10:33.660948+0000","last_became_peered":"2024-09-20T22:10:33.660948+0000","last_unstale":"2024-09-20T22:12:07.286420+0000","last_undegraded":"2024-09-20T22:12:07.286420+0000","last_fullsized":"2024-09-20T22:12:07.286420+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:11:07.362277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:11:07.362277+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:23.879195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"71'10","reported_seq":414,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.295147+0000","last_change":"2024-09-20T22:11:31.064801+0000","last_active":"2024-09-20T22:12:07.295147+0000","last_peered":"2024-09-20T22:12:07.295147+0000","last_clean":"2024-09-20T22:12:07.295147+0000","last_became_active":"2024-09-20T22:11:30.486532+0000","last_became_peered":"2024-09-20T22:11:30.486532+0000","last_unstale":"2024-09-20T22:12:07.295147+0000","last_undegraded":"2024-09-20T22:12:07.295147+0000","last_fullsized":"2024-09-20T22:12:07.295147+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:11:31.064669+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:31.064669+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:35.046612+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":14,"num_read_kb":9,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":538,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.279197+0000","last_change":"2024-09-20T22:11:42.173417+0000","last_active":"2024-09-20T22:12:07.279197+0000","last_peered":"2024-09-20T22:12:07.279197+0000","last_clean":"2024-09-20T22:12:07.279197+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:12:07.279197+0000","last_undegraded":"2024-09-20T22:12:07.279197+0000","last_fullsized":"2024-09-20T22:12:07.279197+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:42.173334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:11:42.173334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:44.551619+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"283'10","reported_seq":636,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:12:00.278312+0000","last_change":"2024-09-20T22:11:30.496915+0000","last_active":"2024-09-20T22:12:00.278312+0000","last_peered":"2024-09-20T22:12:00.278312+0000","last_clean":"2024-09-20T22:12:00.278312+0000","last_became_active":"2024-09-20T22:11:30.486567+0000","last_became_peered":"2024-09-20T22:11:30.486567+0000","last_unstale":"2024-09-20T22:12:00.278312+0000","last_undegraded":"2024-09-20T22:12:00.278312+0000","last_fullsized":"2024-09-20T22:12:00.278312+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:08:28.860982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:28.860982+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":492,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.245048+0000","last_change":"2024-09-20T22:10:24.053258+0000","last_active":"2024-09-20T22:11:59.245048+0000","last_peered":"2024-09-20T22:11:59.245048+0000","last_clean":"2024-09-20T22:11:59.245048+0000","last_became_active":"2024-09-20T22:02:56.304274+0000","last_became_peered":"2024-09-20T22:02:56.304274+0000","last_unstale":"2024-09-20T22:11:59.245048+0000","last_undegraded":"2024-09-20T22:11:59.245048+0000","last_fullsized":"2024-09-20T22:11:59.245048+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:24.053159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:24.053159+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:59.578590+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"302'3106","reported_seq":3148,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:10.680159+0000","last_change":"2024-09-20T22:11:34.560812+0000","last_active":"2024-09-20T22:12:10.680159+0000","last_peered":"2024-09-20T22:12:10.680159+0000","last_clean":"2024-09-20T22:12:10.680159+0000","last_became_active":"2024-09-20T22:11:34.560514+0000","last_became_peered":"2024-09-20T22:11:34.560514+0000","last_unstale":"2024-09-20T22:12:10.680159+0000","last_undegraded":"2024-09-20T22:12:10.680159+0000","last_fullsized":"2024-09-20T22:12:10.680159+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":3106,"log_dups_size":0,"ondisk_log_size":3106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:36.489562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":203554816,"num_objects":3106,"num_object_clones":0,"num_object_copies":9318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3106,"num_write_kb":198784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":522,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.286421+0000","last_change":"2024-09-20T22:11:36.339782+0000","last_active":"2024-09-20T22:12:07.286421+0000","last_peered":"2024-09-20T22:12:07.286421+0000","last_clean":"2024-09-20T22:12:07.286421+0000","last_became_active":"2024-09-20T22:08:48.466328+0000","last_became_peered":"2024-09-20T22:08:48.466328+0000","last_unstale":"2024-09-20T22:12:07.286421+0000","last_undegraded":"2024-09-20T22:12:07.286421+0000","last_fullsized":"2024-09-20T22:12:07.286421+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:11:36.339687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:11:36.339687+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:56.138458+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"71'19","reported_seq":544,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.246404+0000","last_change":"2024-09-20T22:08:33.852966+0000","last_active":"2024-09-20T22:11:59.246404+0000","last_peered":"2024-09-20T22:11:59.246404+0000","last_clean":"2024-09-20T22:11:59.246404+0000","last_became_active":"2024-09-20T22:02:52.480077+0000","last_became_peered":"2024-09-20T22:02:52.480077+0000","last_unstale":"2024-09-20T22:11:59.246404+0000","last_undegraded":"2024-09-20T22:11:59.246404+0000","last_fullsized":"2024-09-20T22:11:59.246404+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:08:33.852806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:33.852806+0000","objects_scrubbed":8,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:59.578495+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":365,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.290753+0000","last_change":"2024-09-20T22:11:30.486764+0000","last_active":"2024-09-20T22:12:07.290753+0000","last_peered":"2024-09-20T22:12:07.290753+0000","last_clean":"2024-09-20T22:12:07.290753+0000","last_became_active":"2024-09-20T22:11:30.486428+0000","last_became_peered":"2024-09-20T22:11:30.486428+0000","last_unstale":"2024-09-20T22:12:07.290753+0000","last_undegraded":"2024-09-20T22:12:07.290753+0000","last_fullsized":"2024-09-20T22:12:07.290753+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:59.221160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:59.221160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:17.826923+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":522,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.286241+0000","last_change":"2024-09-20T22:11:00.280237+0000","last_active":"2024-09-20T22:12:07.286241+0000","last_peered":"2024-09-20T22:12:07.286241+0000","last_clean":"2024-09-20T22:12:07.286241+0000","last_became_active":"2024-09-20T22:02:54.290787+0000","last_became_peered":"2024-09-20T22:02:54.290787+0000","last_unstale":"2024-09-20T22:12:07.286241+0000","last_undegraded":"2024-09-20T22:12:07.286241+0000","last_fullsized":"2024-09-20T22:12:07.286241+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:00.280165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:11:00.280165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:08.873300+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":351,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.290525+0000","last_change":"2024-09-20T22:11:39.264738+0000","last_active":"2024-09-20T22:12:07.290525+0000","last_peered":"2024-09-20T22:12:07.290525+0000","last_clean":"2024-09-20T22:12:07.290525+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:12:07.290525+0000","last_undegraded":"2024-09-20T22:12:07.290525+0000","last_fullsized":"2024-09-20T22:12:07.290525+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:39.264610+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:39.264610+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:55.038507+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"301'2670","reported_seq":2710,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:12:06.711303+0000","last_change":"2024-09-20T22:11:34.561687+0000","last_active":"2024-09-20T22:12:06.711303+0000","last_peered":"2024-09-20T22:12:06.711303+0000","last_clean":"2024-09-20T22:12:06.711303+0000","last_became_active":"2024-09-20T22:11:34.560531+0000","last_became_peered":"2024-09-20T22:11:34.560531+0000","last_unstale":"2024-09-20T22:12:06.711303+0000","last_undegraded":"2024-09-20T22:12:06.711303+0000","last_fullsized":"2024-09-20T22:12:06.711303+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":2670,"log_dups_size":0,"ondisk_log_size":2670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:47.491851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":174981120,"num_objects":2670,"num_object_clones":0,"num_object_copies":8010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2670,"num_write_kb":170880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":517,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.277870+0000","last_change":"2024-09-20T22:10:56.231802+0000","last_active":"2024-09-20T22:12:07.277870+0000","last_peered":"2024-09-20T22:12:07.277870+0000","last_clean":"2024-09-20T22:12:07.277870+0000","last_became_active":"2024-09-20T22:08:48.452234+0000","last_became_peered":"2024-09-20T22:08:48.452234+0000","last_unstale":"2024-09-20T22:12:07.277870+0000","last_undegraded":"2024-09-20T22:12:07.277870+0000","last_fullsized":"2024-09-20T22:12:07.277870+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:10:56.231679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:10:56.231679+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"271'40","reported_seq":487,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.291020+0000","last_change":"2024-09-20T22:11:05.235955+0000","last_active":"2024-09-20T22:12:07.291020+0000","last_peered":"2024-09-20T22:12:07.291020+0000","last_clean":"2024-09-20T22:12:07.291020+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:12:07.291020+0000","last_undegraded":"2024-09-20T22:12:07.291020+0000","last_fullsized":"2024-09-20T22:12:07.291020+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"271'40","last_scrub_stamp":"2024-09-20T22:11:05.235779+0000","last_deep_scrub":"271'40","last_deep_scrub_stamp":"2024-09-20T22:11:05.235779+0000","last_clean_scrub_stamp":"2024-09-20T22:11:05.235779+0000","objects_scrubbed":10,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:31.478891+0000","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":47,"num_read_kb":32,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":523,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.271277+0000","last_change":"2024-09-20T22:12:03.239324+0000","last_active":"2024-09-20T22:12:07.271277+0000","last_peered":"2024-09-20T22:12:07.271277+0000","last_clean":"2024-09-20T22:12:07.271277+0000","last_became_active":"2024-09-20T22:02:50.108457+0000","last_became_peered":"2024-09-20T22:02:50.108457+0000","last_unstale":"2024-09-20T22:12:07.271277+0000","last_undegraded":"2024-09-20T22:12:07.271277+0000","last_fullsized":"2024-09-20T22:12:07.271277+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:03.239256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:03.239256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:19.442026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":534,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.278540+0000","last_change":"2024-09-20T22:11:38.145140+0000","last_active":"2024-09-20T22:12:07.278540+0000","last_peered":"2024-09-20T22:12:07.278540+0000","last_clean":"2024-09-20T22:12:07.278540+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:12:07.278540+0000","last_undegraded":"2024-09-20T22:12:07.278540+0000","last_fullsized":"2024-09-20T22:12:07.278540+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:11:38.145006+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:07:21.886038+0000","last_clean_scrub_stamp":"2024-09-20T22:11:38.145006+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:42.494263+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":513,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.462565+0000","last_change":"2024-09-20T22:11:59.462610+0000","last_active":"2024-09-20T22:11:59.462565+0000","last_peered":"2024-09-20T22:11:59.462565+0000","last_clean":"2024-09-20T22:11:59.462565+0000","last_became_active":"2024-09-20T22:06:54.841419+0000","last_became_peered":"2024-09-20T22:06:54.841419+0000","last_unstale":"2024-09-20T22:11:59.462565+0000","last_undegraded":"2024-09-20T22:11:59.462565+0000","last_fullsized":"2024-09-20T22:11:59.462565+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:59.462479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:59.462479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:05.461546+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":500,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:12:05.559491+0000","last_change":"2024-09-20T22:12:05.559520+0000","last_active":"2024-09-20T22:12:05.559491+0000","last_peered":"2024-09-20T22:12:05.559491+0000","last_clean":"2024-09-20T22:12:05.559491+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:12:05.559491+0000","last_undegraded":"2024-09-20T22:12:05.559491+0000","last_fullsized":"2024-09-20T22:12:05.559491+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:05.559424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:12:05.559424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:11.651620+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"302'2946","reported_seq":2988,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:10.836437+0000","last_change":"2024-09-20T22:11:34.559139+0000","last_active":"2024-09-20T22:12:10.836437+0000","last_peered":"2024-09-20T22:12:10.836437+0000","last_clean":"2024-09-20T22:12:10.836437+0000","last_became_active":"2024-09-20T22:11:34.558703+0000","last_became_peered":"2024-09-20T22:11:34.558703+0000","last_unstale":"2024-09-20T22:12:10.836437+0000","last_undegraded":"2024-09-20T22:12:10.836437+0000","last_fullsized":"2024-09-20T22:12:10.836437+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":2946,"log_dups_size":0,"ondisk_log_size":2946,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:38.441804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":193069056,"num_objects":2946,"num_object_clones":0,"num_object_copies":8838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2946,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2946,"num_write_kb":188544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":507,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.274454+0000","last_change":"2024-09-20T22:11:43.608886+0000","last_active":"2024-09-20T22:12:07.274454+0000","last_peered":"2024-09-20T22:12:07.274454+0000","last_clean":"2024-09-20T22:12:07.274454+0000","last_became_active":"2024-09-20T22:11:43.608568+0000","last_became_peered":"2024-09-20T22:11:43.608568+0000","last_unstale":"2024-09-20T22:12:07.274454+0000","last_undegraded":"2024-09-20T22:12:07.274454+0000","last_fullsized":"2024-09-20T22:12:07.274454+0000","mapping_epoch":293,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":294,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:10:16.022854+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:10:16.022854+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"71'13","reported_seq":390,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.293853+0000","last_change":"2024-09-20T22:11:33.124675+0000","last_active":"2024-09-20T22:12:07.293853+0000","last_peered":"2024-09-20T22:12:07.293853+0000","last_clean":"2024-09-20T22:12:07.293853+0000","last_became_active":"2024-09-20T22:06:54.850696+0000","last_became_peered":"2024-09-20T22:06:54.850696+0000","last_unstale":"2024-09-20T22:12:07.293853+0000","last_undegraded":"2024-09-20T22:12:07.293853+0000","last_fullsized":"2024-09-20T22:12:07.293853+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'13","last_scrub_stamp":"2024-09-20T22:11:33.124595+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.124595+0000","objects_scrubbed":5,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:33.149438+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":18,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":352,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.291301+0000","last_change":"2024-09-20T22:11:01.241914+0000","last_active":"2024-09-20T22:12:07.291301+0000","last_peered":"2024-09-20T22:12:07.291301+0000","last_clean":"2024-09-20T22:12:07.291301+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:12:07.291301+0000","last_undegraded":"2024-09-20T22:12:07.291301+0000","last_fullsized":"2024-09-20T22:12:07.291301+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:01.241803+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:11:01.241803+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":548,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.277130+0000","last_change":"2024-09-20T22:11:35.306968+0000","last_active":"2024-09-20T22:12:07.277130+0000","last_peered":"2024-09-20T22:12:07.277130+0000","last_clean":"2024-09-20T22:12:07.277130+0000","last_became_active":"2024-09-20T22:02:49.099508+0000","last_became_peered":"2024-09-20T22:02:49.099508+0000","last_unstale":"2024-09-20T22:12:07.277130+0000","last_undegraded":"2024-09-20T22:12:07.277130+0000","last_fullsized":"2024-09-20T22:12:07.277130+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:11:35.306842+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:11:35.306842+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:44.238377+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":522,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.279197+0000","last_change":"2024-09-20T22:10:57.475677+0000","last_active":"2024-09-20T22:12:07.279197+0000","last_peered":"2024-09-20T22:12:07.279197+0000","last_clean":"2024-09-20T22:12:07.279197+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:12:07.279197+0000","last_undegraded":"2024-09-20T22:12:07.279197+0000","last_fullsized":"2024-09-20T22:12:07.279197+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:57.475078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:10:57.475078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:16.236114+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":503,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.275440+0000","last_change":"2024-09-20T22:11:03.448918+0000","last_active":"2024-09-20T22:12:07.275440+0000","last_peered":"2024-09-20T22:12:07.275440+0000","last_clean":"2024-09-20T22:12:07.275440+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:12:07.275440+0000","last_undegraded":"2024-09-20T22:12:07.275440+0000","last_fullsized":"2024-09-20T22:12:07.275440+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:03.448813+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:03.448813+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:25.942922+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"301'2530","reported_seq":2570,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:12:06.729183+0000","last_change":"2024-09-20T22:11:35.053290+0000","last_active":"2024-09-20T22:12:06.729183+0000","last_peered":"2024-09-20T22:12:06.729183+0000","last_clean":"2024-09-20T22:12:06.729183+0000","last_became_active":"2024-09-20T22:11:35.052922+0000","last_became_peered":"2024-09-20T22:11:35.052922+0000","last_unstale":"2024-09-20T22:12:06.729183+0000","last_undegraded":"2024-09-20T22:12:06.729183+0000","last_fullsized":"2024-09-20T22:12:06.729183+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":2530,"log_dups_size":0,"ondisk_log_size":2530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:50.247621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":165806080,"num_objects":2530,"num_object_clones":0,"num_object_copies":7590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2530,"num_write_kb":161920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":516,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.291090+0000","last_change":"2024-09-20T22:11:36.214541+0000","last_active":"2024-09-20T22:12:07.291090+0000","last_peered":"2024-09-20T22:12:07.291090+0000","last_clean":"2024-09-20T22:12:07.291090+0000","last_became_active":"2024-09-20T22:08:48.446840+0000","last_became_peered":"2024-09-20T22:08:48.446840+0000","last_unstale":"2024-09-20T22:12:07.291090+0000","last_undegraded":"2024-09-20T22:12:07.291090+0000","last_fullsized":"2024-09-20T22:12:07.291090+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:11:36.214466+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:11:36.214466+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:41.066885+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"71'12","reported_seq":527,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.244744+0000","last_change":"2024-09-20T22:10:58.599478+0000","last_active":"2024-09-20T22:11:59.244744+0000","last_peered":"2024-09-20T22:11:59.244744+0000","last_clean":"2024-09-20T22:11:59.244744+0000","last_became_active":"2024-09-20T22:06:54.848670+0000","last_became_peered":"2024-09-20T22:06:54.848670+0000","last_unstale":"2024-09-20T22:11:59.244744+0000","last_undegraded":"2024-09-20T22:11:59.244744+0000","last_fullsized":"2024-09-20T22:11:59.244744+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:10:58.599362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:58.599362+0000","objects_scrubbed":6,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:24.582332+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":512,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.244299+0000","last_change":"2024-09-20T22:10:30.889564+0000","last_active":"2024-09-20T22:11:59.244299+0000","last_peered":"2024-09-20T22:11:59.244299+0000","last_clean":"2024-09-20T22:11:59.244299+0000","last_became_active":"2024-09-20T22:02:50.110448+0000","last_became_peered":"2024-09-20T22:02:50.110448+0000","last_unstale":"2024-09-20T22:11:59.244299+0000","last_undegraded":"2024-09-20T22:11:59.244299+0000","last_fullsized":"2024-09-20T22:11:59.244299+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:30.889484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:10:30.889484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:59.578611+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":1080,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:09.298744+0000","last_change":"2024-09-20T22:10:20.084329+0000","last_active":"2024-09-20T22:12:09.298744+0000","last_peered":"2024-09-20T22:12:09.298744+0000","last_clean":"2024-09-20T22:12:09.298744+0000","last_became_active":"2024-09-20T22:06:54.844454+0000","last_became_peered":"2024-09-20T22:06:54.844454+0000","last_unstale":"2024-09-20T22:12:09.298744+0000","last_undegraded":"2024-09-20T22:12:09.298744+0000","last_fullsized":"2024-09-20T22:12:09.298744+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:10:20.084250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:10:20.084250+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":61,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":547,"num_read_kb":542,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":57,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.278724+0000","last_change":"2024-09-20T22:11:39.171066+0000","last_active":"2024-09-20T22:12:07.278724+0000","last_peered":"2024-09-20T22:12:07.278724+0000","last_clean":"2024-09-20T22:12:07.278724+0000","last_became_active":"2024-09-20T22:11:30.489860+0000","last_became_peered":"2024-09-20T22:11:30.489860+0000","last_unstale":"2024-09-20T22:12:07.278724+0000","last_undegraded":"2024-09-20T22:12:07.278724+0000","last_fullsized":"2024-09-20T22:12:07.278724+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:39.170979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:39.170979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:44.884792+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":352,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.290501+0000","last_change":"2024-09-20T22:11:30.024842+0000","last_active":"2024-09-20T22:12:07.290501+0000","last_peered":"2024-09-20T22:12:07.290501+0000","last_clean":"2024-09-20T22:12:07.290501+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:12:07.290501+0000","last_undegraded":"2024-09-20T22:12:07.290501+0000","last_fullsized":"2024-09-20T22:12:07.290501+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:30.024770+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.415468+0000","last_clean_scrub_stamp":"2024-09-20T22:11:30.024770+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:34.524988+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"301'2654","reported_seq":2694,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:12:06.363009+0000","last_change":"2024-09-20T22:11:34.559269+0000","last_active":"2024-09-20T22:12:06.363009+0000","last_peered":"2024-09-20T22:12:06.363009+0000","last_clean":"2024-09-20T22:12:06.363009+0000","last_became_active":"2024-09-20T22:11:34.558981+0000","last_became_peered":"2024-09-20T22:11:34.558981+0000","last_unstale":"2024-09-20T22:12:06.363009+0000","last_undegraded":"2024-09-20T22:12:06.363009+0000","last_fullsized":"2024-09-20T22:12:06.363009+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":2654,"log_dups_size":0,"ondisk_log_size":2654,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:57.225697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":173932544,"num_objects":2654,"num_object_clones":0,"num_object_copies":7962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2654,"num_write_kb":169856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"71'16","reported_seq":661,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.286472+0000","last_change":"2024-09-20T22:12:05.332528+0000","last_active":"2024-09-20T22:12:07.286472+0000","last_peered":"2024-09-20T22:12:07.286472+0000","last_clean":"2024-09-20T22:12:07.286472+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:12:07.286472+0000","last_undegraded":"2024-09-20T22:12:07.286472+0000","last_fullsized":"2024-09-20T22:12:07.286472+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'16","last_scrub_stamp":"2024-09-20T22:12:05.332362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:05.332362+0000","objects_scrubbed":8,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:23.592212+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":524,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.258887+0000","last_change":"2024-09-20T22:11:30.487346+0000","last_active":"2024-09-20T22:11:59.258887+0000","last_peered":"2024-09-20T22:11:59.258887+0000","last_clean":"2024-09-20T22:11:59.258887+0000","last_became_active":"2024-09-20T22:11:30.486987+0000","last_became_peered":"2024-09-20T22:11:30.486987+0000","last_unstale":"2024-09-20T22:11:59.258887+0000","last_undegraded":"2024-09-20T22:11:59.258887+0000","last_fullsized":"2024-09-20T22:11:59.258887+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:18.235255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:18.235255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":506,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.274087+0000","last_change":"2024-09-20T22:10:33.656595+0000","last_active":"2024-09-20T22:12:07.274087+0000","last_peered":"2024-09-20T22:12:07.274087+0000","last_clean":"2024-09-20T22:12:07.274087+0000","last_became_active":"2024-09-20T22:10:33.656161+0000","last_became_peered":"2024-09-20T22:10:33.656161+0000","last_unstale":"2024-09-20T22:12:07.274087+0000","last_undegraded":"2024-09-20T22:12:07.274087+0000","last_fullsized":"2024-09-20T22:12:07.274087+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:10:17.014513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:10:17.014513+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":97,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.274587+0000","last_change":"2024-09-20T22:11:32.281684+0000","last_active":"2024-09-20T22:12:07.274587+0000","last_peered":"2024-09-20T22:12:07.274587+0000","last_clean":"2024-09-20T22:12:07.274587+0000","last_became_active":"2024-09-20T22:10:48.907271+0000","last_became_peered":"2024-09-20T22:10:48.907271+0000","last_unstale":"2024-09-20T22:12:07.274587+0000","last_undegraded":"2024-09-20T22:12:07.274587+0000","last_fullsized":"2024-09-20T22:12:07.274587+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:32.281594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:32.281594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:43.436523+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":524,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.278577+0000","last_change":"2024-09-20T22:11:00.215924+0000","last_active":"2024-09-20T22:12:07.278577+0000","last_peered":"2024-09-20T22:12:07.278577+0000","last_clean":"2024-09-20T22:12:07.278577+0000","last_became_active":"2024-09-20T22:04:21.237725+0000","last_became_peered":"2024-09-20T22:04:21.237725+0000","last_unstale":"2024-09-20T22:12:07.278577+0000","last_undegraded":"2024-09-20T22:12:07.278577+0000","last_fullsized":"2024-09-20T22:12:07.278577+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:00.215806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:11:00.215806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:25.875894+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"302'3064","reported_seq":3105,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:10.680025+0000","last_change":"2024-09-20T22:11:34.559883+0000","last_active":"2024-09-20T22:12:10.680025+0000","last_peered":"2024-09-20T22:12:10.680025+0000","last_clean":"2024-09-20T22:12:10.680025+0000","last_became_active":"2024-09-20T22:11:34.559353+0000","last_became_peered":"2024-09-20T22:11:34.559353+0000","last_unstale":"2024-09-20T22:12:10.680025+0000","last_undegraded":"2024-09-20T22:12:10.680025+0000","last_fullsized":"2024-09-20T22:12:10.680025+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":3064,"log_dups_size":0,"ondisk_log_size":3064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:35.158173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":200802304,"num_objects":3064,"num_object_clones":0,"num_object_copies":9192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3064,"num_write_kb":196096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"71'11","reported_seq":531,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.272144+0000","last_change":"2024-09-20T22:10:24.621114+0000","last_active":"2024-09-20T22:12:07.272144+0000","last_peered":"2024-09-20T22:12:07.272144+0000","last_clean":"2024-09-20T22:12:07.272144+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:12:07.272144+0000","last_undegraded":"2024-09-20T22:12:07.272144+0000","last_fullsized":"2024-09-20T22:12:07.272144+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:10:24.620997+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:24.620997+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":515,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.271112+0000","last_change":"2024-09-20T22:11:07.211087+0000","last_active":"2024-09-20T22:12:07.271112+0000","last_peered":"2024-09-20T22:12:07.271112+0000","last_clean":"2024-09-20T22:12:07.271112+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:12:07.271112+0000","last_undegraded":"2024-09-20T22:12:07.271112+0000","last_fullsized":"2024-09-20T22:12:07.271112+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:07.211011+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:11:07.211011+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:16.920794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":514,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.278482+0000","last_change":"2024-09-20T22:10:34.232282+0000","last_active":"2024-09-20T22:12:07.278482+0000","last_peered":"2024-09-20T22:12:07.278482+0000","last_clean":"2024-09-20T22:12:07.278482+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:12:07.278482+0000","last_undegraded":"2024-09-20T22:12:07.278482+0000","last_fullsized":"2024-09-20T22:12:07.278482+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:34.232168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:34.232168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":512,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.256290+0000","last_change":"2024-09-20T22:11:30.487618+0000","last_active":"2024-09-20T22:11:59.256290+0000","last_peered":"2024-09-20T22:11:59.256290+0000","last_clean":"2024-09-20T22:11:59.256290+0000","last_became_active":"2024-09-20T22:11:30.487283+0000","last_became_peered":"2024-09-20T22:11:30.487283+0000","last_unstale":"2024-09-20T22:11:59.256290+0000","last_undegraded":"2024-09-20T22:11:59.256290+0000","last_fullsized":"2024-09-20T22:11:59.256290+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:17.209772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:17.209772+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.0","version":"302'2812","reported_seq":2853,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:08.041654+0000","last_change":"2024-09-20T22:11:35.304586+0000","last_active":"2024-09-20T22:12:08.041654+0000","last_peered":"2024-09-20T22:12:08.041654+0000","last_clean":"2024-09-20T22:12:08.041654+0000","last_became_active":"2024-09-20T22:11:35.304302+0000","last_became_peered":"2024-09-20T22:11:35.304302+0000","last_unstale":"2024-09-20T22:12:08.041654+0000","last_undegraded":"2024-09-20T22:12:08.041654+0000","last_fullsized":"2024-09-20T22:12:08.041654+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":2812,"log_dups_size":0,"ondisk_log_size":2812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:35.122680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":184287232,"num_objects":2812,"num_object_clones":0,"num_object_copies":8436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2812,"num_write_kb":179968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":420,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.277912+0000","last_change":"2024-09-20T22:11:33.557801+0000","last_active":"2024-09-20T22:12:07.277912+0000","last_peered":"2024-09-20T22:12:07.277912+0000","last_clean":"2024-09-20T22:12:07.277912+0000","last_became_active":"2024-09-20T22:11:33.557208+0000","last_became_peered":"2024-09-20T22:11:33.557208+0000","last_unstale":"2024-09-20T22:12:07.277912+0000","last_undegraded":"2024-09-20T22:12:07.277912+0000","last_fullsized":"2024-09-20T22:12:07.277912+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:10:34.490571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:10:34.490571+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"71'17","reported_seq":548,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.271835+0000","last_change":"2024-09-20T22:10:29.580641+0000","last_active":"2024-09-20T22:12:07.271835+0000","last_peered":"2024-09-20T22:12:07.271835+0000","last_clean":"2024-09-20T22:12:07.271835+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:12:07.271835+0000","last_undegraded":"2024-09-20T22:12:07.271835+0000","last_fullsized":"2024-09-20T22:12:07.271835+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'17","last_scrub_stamp":"2024-09-20T22:10:29.579935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:29.579935+0000","objects_scrubbed":8,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":531,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.286362+0000","last_change":"2024-09-20T22:12:06.305854+0000","last_active":"2024-09-20T22:12:07.286362+0000","last_peered":"2024-09-20T22:12:07.286362+0000","last_clean":"2024-09-20T22:12:07.286362+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:12:07.286362+0000","last_undegraded":"2024-09-20T22:12:07.286362+0000","last_fullsized":"2024-09-20T22:12:07.286362+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:06.305736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:06.305736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:13.385412+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":352,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.290602+0000","last_change":"2024-09-20T22:11:02.256320+0000","last_active":"2024-09-20T22:12:07.290602+0000","last_peered":"2024-09-20T22:12:07.290602+0000","last_clean":"2024-09-20T22:12:07.290602+0000","last_became_active":"2024-09-20T22:06:54.848275+0000","last_became_peered":"2024-09-20T22:06:54.848275+0000","last_unstale":"2024-09-20T22:12:07.290602+0000","last_undegraded":"2024-09-20T22:12:07.290602+0000","last_fullsized":"2024-09-20T22:12:07.290602+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:02.256217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:02.256217+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:26.715522+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":518,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.289888+0000","last_change":"2024-09-20T22:11:31.333144+0000","last_active":"2024-09-20T22:12:07.289888+0000","last_peered":"2024-09-20T22:12:07.289888+0000","last_clean":"2024-09-20T22:12:07.289888+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:12:07.289888+0000","last_undegraded":"2024-09-20T22:12:07.289888+0000","last_fullsized":"2024-09-20T22:12:07.289888+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:31.333027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:31.333027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:53.330444+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"301'2604","reported_seq":2644,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:12:06.370502+0000","last_change":"2024-09-20T22:11:35.053482+0000","last_active":"2024-09-20T22:12:06.370502+0000","last_peered":"2024-09-20T22:12:06.370502+0000","last_clean":"2024-09-20T22:12:06.370502+0000","last_became_active":"2024-09-20T22:11:35.053141+0000","last_became_peered":"2024-09-20T22:11:35.053141+0000","last_unstale":"2024-09-20T22:12:06.370502+0000","last_undegraded":"2024-09-20T22:12:06.370502+0000","last_fullsized":"2024-09-20T22:12:06.370502+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":2604,"log_dups_size":0,"ondisk_log_size":2604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:46.261380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170655744,"num_objects":2604,"num_object_clones":0,"num_object_copies":7812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2604,"num_write_kb":166656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":794,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.275113+0000","last_change":"2024-09-20T22:11:34.332239+0000","last_active":"2024-09-20T22:12:07.275113+0000","last_peered":"2024-09-20T22:12:07.275113+0000","last_clean":"2024-09-20T22:12:07.275113+0000","last_became_active":"2024-09-20T22:08:48.455288+0000","last_became_peered":"2024-09-20T22:08:48.455288+0000","last_unstale":"2024-09-20T22:12:07.275113+0000","last_undegraded":"2024-09-20T22:12:07.275113+0000","last_fullsized":"2024-09-20T22:12:07.275113+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:11:34.332158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.332158+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:36.731335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"289'15","reported_seq":531,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.271364+0000","last_change":"2024-09-20T22:11:37.277758+0000","last_active":"2024-09-20T22:12:07.271364+0000","last_peered":"2024-09-20T22:12:07.271364+0000","last_clean":"2024-09-20T22:12:07.271364+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:12:07.271364+0000","last_undegraded":"2024-09-20T22:12:07.271364+0000","last_fullsized":"2024-09-20T22:12:07.271364+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'15","last_scrub_stamp":"2024-09-20T22:11:37.277648+0000","last_deep_scrub":"289'15","last_deep_scrub_stamp":"2024-09-20T22:11:37.277648+0000","last_clean_scrub_stamp":"2024-09-20T22:11:37.277648+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:05.356939+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":519,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.249794+0000","last_change":"2024-09-20T22:09:53.395968+0000","last_active":"2024-09-20T22:11:59.249794+0000","last_peered":"2024-09-20T22:11:59.249794+0000","last_clean":"2024-09-20T22:11:59.249794+0000","last_became_active":"2024-09-20T22:02:50.110446+0000","last_became_peered":"2024-09-20T22:02:50.110446+0000","last_unstale":"2024-09-20T22:11:59.249794+0000","last_undegraded":"2024-09-20T22:11:59.249794+0000","last_fullsized":"2024-09-20T22:11:59.249794+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:09:53.395833+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"283'11","reported_seq":672,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.277667+0000","last_change":"2024-09-20T22:11:30.497599+0000","last_active":"2024-09-20T22:12:07.277667+0000","last_peered":"2024-09-20T22:12:07.277667+0000","last_clean":"2024-09-20T22:12:07.277667+0000","last_became_active":"2024-09-20T22:11:30.485356+0000","last_became_peered":"2024-09-20T22:11:30.485356+0000","last_unstale":"2024-09-20T22:12:07.277667+0000","last_undegraded":"2024-09-20T22:12:07.277667+0000","last_fullsized":"2024-09-20T22:12:07.277667+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"264'10","last_scrub_stamp":"2024-09-20T22:11:08.571767+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:11:08.571767+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:24.914453+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":504,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.271680+0000","last_change":"2024-09-20T22:09:53.701152+0000","last_active":"2024-09-20T22:12:07.271680+0000","last_peered":"2024-09-20T22:12:07.271680+0000","last_clean":"2024-09-20T22:12:07.271680+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:12:07.271680+0000","last_undegraded":"2024-09-20T22:12:07.271680+0000","last_fullsized":"2024-09-20T22:12:07.271680+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:53.701054+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:09:53.701054+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"302'3155","reported_seq":3197,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:10.817244+0000","last_change":"2024-09-20T22:11:35.304534+0000","last_active":"2024-09-20T22:12:10.817244+0000","last_peered":"2024-09-20T22:12:10.817244+0000","last_clean":"2024-09-20T22:12:10.817244+0000","last_became_active":"2024-09-20T22:11:35.304192+0000","last_became_peered":"2024-09-20T22:11:35.304192+0000","last_unstale":"2024-09-20T22:12:10.817244+0000","last_undegraded":"2024-09-20T22:12:10.817244+0000","last_fullsized":"2024-09-20T22:12:10.817244+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":3155,"log_dups_size":0,"ondisk_log_size":3155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:54.063980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":206766080,"num_objects":3155,"num_object_clones":0,"num_object_copies":9465,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3155,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3155,"num_write_kb":201920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"71'9","reported_seq":549,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.246312+0000","last_change":"2024-09-20T22:11:02.575450+0000","last_active":"2024-09-20T22:11:59.246312+0000","last_peered":"2024-09-20T22:11:59.246312+0000","last_clean":"2024-09-20T22:11:59.246312+0000","last_became_active":"2024-09-20T22:10:22.565004+0000","last_became_peered":"2024-09-20T22:10:22.565004+0000","last_unstale":"2024-09-20T22:11:59.246312+0000","last_undegraded":"2024-09-20T22:11:59.246312+0000","last_fullsized":"2024-09-20T22:11:59.246312+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:11:02.575330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:02.575330+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:05.729253+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":140,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.256889+0000","last_change":"2024-09-20T22:10:31.161469+0000","last_active":"2024-09-20T22:11:59.256889+0000","last_peered":"2024-09-20T22:11:59.256889+0000","last_clean":"2024-09-20T22:11:59.256889+0000","last_became_active":"2024-09-20T22:10:14.536155+0000","last_became_peered":"2024-09-20T22:10:14.536155+0000","last_unstale":"2024-09-20T22:11:59.256889+0000","last_undegraded":"2024-09-20T22:11:59.256889+0000","last_fullsized":"2024-09-20T22:11:59.256889+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:10:31.161375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:31.161375+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:00.554074+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":530,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:08.183897+0000","last_change":"2024-09-20T22:12:08.183939+0000","last_active":"2024-09-20T22:12:08.183897+0000","last_peered":"2024-09-20T22:12:08.183897+0000","last_clean":"2024-09-20T22:12:08.183897+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:12:08.183897+0000","last_undegraded":"2024-09-20T22:12:08.183897+0000","last_fullsized":"2024-09-20T22:12:08.183897+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:08.183800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:12:08.183800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:24.608235+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":510,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:12:00.473973+0000","last_change":"2024-09-20T22:12:00.474009+0000","last_active":"2024-09-20T22:12:00.473973+0000","last_peered":"2024-09-20T22:12:00.473973+0000","last_clean":"2024-09-20T22:12:00.473973+0000","last_became_active":"2024-09-20T22:06:55.824229+0000","last_became_peered":"2024-09-20T22:06:55.824229+0000","last_unstale":"2024-09-20T22:12:00.473973+0000","last_undegraded":"2024-09-20T22:12:00.473973+0000","last_fullsized":"2024-09-20T22:12:00.473973+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:00.473894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:00.473894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:27.516564+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"302'3023","reported_seq":3071,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:10.821221+0000","last_change":"2024-09-20T22:11:40.601503+0000","last_active":"2024-09-20T22:12:10.821221+0000","last_peered":"2024-09-20T22:12:10.821221+0000","last_clean":"2024-09-20T22:12:10.821221+0000","last_became_active":"2024-09-20T22:11:40.601030+0000","last_became_peered":"2024-09-20T22:11:40.601030+0000","last_unstale":"2024-09-20T22:12:10.821221+0000","last_undegraded":"2024-09-20T22:12:10.821221+0000","last_fullsized":"2024-09-20T22:12:10.821221+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":3023,"log_dups_size":0,"ondisk_log_size":3023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:44.031519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198115328,"num_objects":3023,"num_object_clones":0,"num_object_copies":9069,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3023,"num_write_kb":193472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"71'19","reported_seq":602,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.286093+0000","last_change":"2024-09-20T22:12:00.343719+0000","last_active":"2024-09-20T22:12:07.286093+0000","last_peered":"2024-09-20T22:12:07.286093+0000","last_clean":"2024-09-20T22:12:07.286093+0000","last_became_active":"2024-09-20T22:10:30.652967+0000","last_became_peered":"2024-09-20T22:10:30.652967+0000","last_unstale":"2024-09-20T22:12:07.286093+0000","last_undegraded":"2024-09-20T22:12:07.286093+0000","last_fullsized":"2024-09-20T22:12:07.286093+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:12:00.343588+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:00.343588+0000","objects_scrubbed":9,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:08.101831+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":527,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.275619+0000","last_change":"2024-09-20T22:11:30.489226+0000","last_active":"2024-09-20T22:12:07.275619+0000","last_peered":"2024-09-20T22:12:07.275619+0000","last_clean":"2024-09-20T22:12:07.275619+0000","last_became_active":"2024-09-20T22:11:30.488650+0000","last_became_peered":"2024-09-20T22:11:30.488650+0000","last_unstale":"2024-09-20T22:12:07.275619+0000","last_undegraded":"2024-09-20T22:12:07.275619+0000","last_fullsized":"2024-09-20T22:12:07.275619+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:28.095505+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:28.095505+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":534,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.278508+0000","last_change":"2024-09-20T22:11:30.484978+0000","last_active":"2024-09-20T22:12:07.278508+0000","last_peered":"2024-09-20T22:12:07.278508+0000","last_clean":"2024-09-20T22:12:07.278508+0000","last_became_active":"2024-09-20T22:11:30.484597+0000","last_became_peered":"2024-09-20T22:11:30.484597+0000","last_unstale":"2024-09-20T22:12:07.278508+0000","last_undegraded":"2024-09-20T22:12:07.278508+0000","last_fullsized":"2024-09-20T22:12:07.278508+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_clean_scrub_stamp":"2024-09-20T22:10:26.501363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":111,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.274157+0000","last_change":"2024-09-20T22:10:30.657488+0000","last_active":"2024-09-20T22:12:07.274157+0000","last_peered":"2024-09-20T22:12:07.274157+0000","last_clean":"2024-09-20T22:12:07.274157+0000","last_became_active":"2024-09-20T22:10:30.657116+0000","last_became_peered":"2024-09-20T22:10:30.657116+0000","last_unstale":"2024-09-20T22:12:07.274157+0000","last_undegraded":"2024-09-20T22:12:07.274157+0000","last_fullsized":"2024-09-20T22:12:07.274157+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:08.495599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:08.495599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"302'2742","reported_seq":2784,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:08.046557+0000","last_change":"2024-09-20T22:11:34.558254+0000","last_active":"2024-09-20T22:12:08.046557+0000","last_peered":"2024-09-20T22:12:08.046557+0000","last_clean":"2024-09-20T22:12:08.046557+0000","last_became_active":"2024-09-20T22:11:34.557976+0000","last_became_peered":"2024-09-20T22:11:34.557976+0000","last_unstale":"2024-09-20T22:12:08.046557+0000","last_undegraded":"2024-09-20T22:12:08.046557+0000","last_fullsized":"2024-09-20T22:12:08.046557+0000","mapping_epoch":286,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":2742,"log_dups_size":0,"ondisk_log_size":2742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:37.795228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":179699712,"num_objects":2742,"num_object_clones":0,"num_object_copies":8226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2742,"num_write_kb":175488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"71'12","reported_seq":535,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.270839+0000","last_change":"2024-09-20T22:10:21.628457+0000","last_active":"2024-09-20T22:12:07.270839+0000","last_peered":"2024-09-20T22:12:07.270839+0000","last_clean":"2024-09-20T22:12:07.270839+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:12:07.270839+0000","last_undegraded":"2024-09-20T22:12:07.270839+0000","last_fullsized":"2024-09-20T22:12:07.270839+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'12","last_scrub_stamp":"2024-09-20T22:10:21.628338+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:21.628338+0000","objects_scrubbed":5,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":516,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.275604+0000","last_change":"2024-09-20T22:09:03.831996+0000","last_active":"2024-09-20T22:12:07.275604+0000","last_peered":"2024-09-20T22:12:07.275604+0000","last_clean":"2024-09-20T22:12:07.275604+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:12:07.275604+0000","last_undegraded":"2024-09-20T22:12:07.275604+0000","last_fullsized":"2024-09-20T22:12:07.275604+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:03.831774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:09:03.831774+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":531,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.278538+0000","last_change":"2024-09-20T22:11:36.287314+0000","last_active":"2024-09-20T22:12:07.278538+0000","last_peered":"2024-09-20T22:12:07.278538+0000","last_clean":"2024-09-20T22:12:07.278538+0000","last_became_active":"2024-09-20T22:06:54.843906+0000","last_became_peered":"2024-09-20T22:06:54.843906+0000","last_unstale":"2024-09-20T22:12:07.278538+0000","last_undegraded":"2024-09-20T22:12:07.278538+0000","last_fullsized":"2024-09-20T22:12:07.278538+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:36.287220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:36.287220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:47.050640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":465,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:12:02.447493+0000","last_change":"2024-09-20T22:12:02.447527+0000","last_active":"2024-09-20T22:12:02.447493+0000","last_peered":"2024-09-20T22:12:02.447493+0000","last_clean":"2024-09-20T22:12:02.447493+0000","last_became_active":"2024-09-20T22:11:40.600444+0000","last_became_peered":"2024-09-20T22:11:40.600444+0000","last_unstale":"2024-09-20T22:12:02.447493+0000","last_undegraded":"2024-09-20T22:12:02.447493+0000","last_fullsized":"2024-09-20T22:12:02.447493+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:02.447415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:02.447415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:09.392488+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1],"acting":[0,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.5","version":"301'2561","reported_seq":2607,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:12:06.700829+0000","last_change":"2024-09-20T22:11:40.601169+0000","last_active":"2024-09-20T22:12:06.700829+0000","last_peered":"2024-09-20T22:12:06.700829+0000","last_clean":"2024-09-20T22:12:06.700829+0000","last_became_active":"2024-09-20T22:11:40.600788+0000","last_became_peered":"2024-09-20T22:11:40.600788+0000","last_unstale":"2024-09-20T22:12:06.700829+0000","last_undegraded":"2024-09-20T22:12:06.700829+0000","last_fullsized":"2024-09-20T22:12:06.700829+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":286,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":2561,"log_dups_size":0,"ondisk_log_size":2561,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:37.359872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167837696,"num_objects":2561,"num_object_clones":0,"num_object_copies":7683,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2561,"num_write_kb":163904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"71'15","reported_seq":65,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.277798+0000","last_change":"2024-09-20T22:11:30.506603+0000","last_active":"2024-09-20T22:12:07.277798+0000","last_peered":"2024-09-20T22:12:07.277798+0000","last_clean":"2024-09-20T22:12:07.277798+0000","last_became_active":"2024-09-20T22:11:30.493275+0000","last_became_peered":"2024-09-20T22:11:30.493275+0000","last_unstale":"2024-09-20T22:12:07.277798+0000","last_undegraded":"2024-09-20T22:12:07.277798+0000","last_fullsized":"2024-09-20T22:12:07.277798+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'15","last_scrub_stamp":"2024-09-20T22:11:02.397374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:02.397374+0000","objects_scrubbed":7,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:28.185784+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":117,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.275608+0000","last_change":"2024-09-20T22:10:30.661904+0000","last_active":"2024-09-20T22:12:07.275608+0000","last_peered":"2024-09-20T22:12:07.275608+0000","last_clean":"2024-09-20T22:12:07.275608+0000","last_became_active":"2024-09-20T22:10:30.653330+0000","last_became_peered":"2024-09-20T22:10:30.653330+0000","last_unstale":"2024-09-20T22:12:07.275608+0000","last_undegraded":"2024-09-20T22:12:07.275608+0000","last_fullsized":"2024-09-20T22:12:07.275608+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:09:48.706938+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:09:48.706938+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":628,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.275007+0000","last_change":"2024-09-20T22:12:04.528478+0000","last_active":"2024-09-20T22:12:07.275007+0000","last_peered":"2024-09-20T22:12:07.275007+0000","last_clean":"2024-09-20T22:12:07.275007+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:12:07.275007+0000","last_undegraded":"2024-09-20T22:12:07.275007+0000","last_fullsized":"2024-09-20T22:12:07.275007+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:12:04.528253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:04.528253+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:18.369058+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":49,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.278254+0000","last_change":"2024-09-20T22:11:30.492983+0000","last_active":"2024-09-20T22:12:07.278254+0000","last_peered":"2024-09-20T22:12:07.278254+0000","last_clean":"2024-09-20T22:12:07.278254+0000","last_became_active":"2024-09-20T22:11:30.492476+0000","last_became_peered":"2024-09-20T22:11:30.492476+0000","last_unstale":"2024-09-20T22:12:07.278254+0000","last_undegraded":"2024-09-20T22:12:07.278254+0000","last_fullsized":"2024-09-20T22:12:07.278254+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:56.334297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:56.334297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:21.467964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":542,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.286102+0000","last_change":"2024-09-20T22:11:39.375720+0000","last_active":"2024-09-20T22:12:07.286102+0000","last_peered":"2024-09-20T22:12:07.286102+0000","last_clean":"2024-09-20T22:12:07.286102+0000","last_became_active":"2024-09-20T22:11:30.486764+0000","last_became_peered":"2024-09-20T22:11:30.486764+0000","last_unstale":"2024-09-20T22:12:07.286102+0000","last_undegraded":"2024-09-20T22:12:07.286102+0000","last_fullsized":"2024-09-20T22:12:07.286102+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:39.375639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:11:39.375639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:44.853725+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"71'6","reported_seq":544,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.252965+0000","last_change":"2024-09-20T22:11:07.315899+0000","last_active":"2024-09-20T22:11:59.252965+0000","last_peered":"2024-09-20T22:11:59.252965+0000","last_clean":"2024-09-20T22:11:59.252965+0000","last_became_active":"2024-09-20T22:10:48.922561+0000","last_became_peered":"2024-09-20T22:10:48.922561+0000","last_unstale":"2024-09-20T22:11:59.252965+0000","last_undegraded":"2024-09-20T22:11:59.252965+0000","last_fullsized":"2024-09-20T22:11:59.252965+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'6","last_scrub_stamp":"2024-09-20T22:11:07.315753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:07.315753+0000","objects_scrubbed":3,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:09.199775+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":49,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.278363+0000","last_change":"2024-09-20T22:11:30.487894+0000","last_active":"2024-09-20T22:12:07.278363+0000","last_peered":"2024-09-20T22:12:07.278363+0000","last_clean":"2024-09-20T22:12:07.278363+0000","last_became_active":"2024-09-20T22:11:30.487322+0000","last_became_peered":"2024-09-20T22:11:30.487322+0000","last_unstale":"2024-09-20T22:12:07.278363+0000","last_undegraded":"2024-09-20T22:12:07.278363+0000","last_fullsized":"2024-09-20T22:12:07.278363+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:20.293112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:10:20.293112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":502,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.275315+0000","last_change":"2024-09-20T22:12:05.541834+0000","last_active":"2024-09-20T22:12:07.275315+0000","last_peered":"2024-09-20T22:12:07.275315+0000","last_clean":"2024-09-20T22:12:07.275315+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:12:07.275315+0000","last_undegraded":"2024-09-20T22:12:07.275315+0000","last_fullsized":"2024-09-20T22:12:07.275315+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:05.541367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:05.541367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:29.774109+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"71'9","reported_seq":546,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.244845+0000","last_change":"2024-09-20T22:10:48.930194+0000","last_active":"2024-09-20T22:11:59.244845+0000","last_peered":"2024-09-20T22:11:59.244845+0000","last_clean":"2024-09-20T22:11:59.244845+0000","last_became_active":"2024-09-20T22:10:48.924249+0000","last_became_peered":"2024-09-20T22:10:48.924249+0000","last_unstale":"2024-09-20T22:11:59.244845+0000","last_undegraded":"2024-09-20T22:11:59.244845+0000","last_fullsized":"2024-09-20T22:11:59.244845+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:08:31.897253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:08:31.897253+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:59.578462+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":515,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.275155+0000","last_change":"2024-09-20T22:10:29.059396+0000","last_active":"2024-09-20T22:12:07.275155+0000","last_peered":"2024-09-20T22:12:07.275155+0000","last_clean":"2024-09-20T22:12:07.275155+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:12:07.275155+0000","last_undegraded":"2024-09-20T22:12:07.275155+0000","last_fullsized":"2024-09-20T22:12:07.275155+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:29.059287+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:29.059287+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":504,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.252690+0000","last_change":"2024-09-20T22:11:09.360634+0000","last_active":"2024-09-20T22:11:59.252690+0000","last_peered":"2024-09-20T22:11:59.252690+0000","last_clean":"2024-09-20T22:11:59.252690+0000","last_became_active":"2024-09-20T22:06:54.845704+0000","last_became_peered":"2024-09-20T22:06:54.845704+0000","last_unstale":"2024-09-20T22:11:59.252690+0000","last_undegraded":"2024-09-20T22:11:59.252690+0000","last_fullsized":"2024-09-20T22:11:59.252690+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_clean_scrub_stamp":"2024-09-20T22:11:09.360439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:31.840825+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":517,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.278009+0000","last_change":"2024-09-20T22:11:05.544433+0000","last_active":"2024-09-20T22:12:07.278009+0000","last_peered":"2024-09-20T22:12:07.278009+0000","last_clean":"2024-09-20T22:12:07.278009+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:12:07.278009+0000","last_undegraded":"2024-09-20T22:12:07.278009+0000","last_fullsized":"2024-09-20T22:12:07.278009+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:05.544321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:05.544321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:20.221161+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"71'9","reported_seq":59,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.279098+0000","last_change":"2024-09-20T22:11:30.502818+0000","last_active":"2024-09-20T22:12:07.279098+0000","last_peered":"2024-09-20T22:12:07.279098+0000","last_clean":"2024-09-20T22:12:07.279098+0000","last_became_active":"2024-09-20T22:11:30.493335+0000","last_became_peered":"2024-09-20T22:11:30.493335+0000","last_unstale":"2024-09-20T22:12:07.279098+0000","last_undegraded":"2024-09-20T22:12:07.279098+0000","last_fullsized":"2024-09-20T22:12:07.279098+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:10:59.360044+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:59.360044+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":522,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.244114+0000","last_change":"2024-09-20T22:10:48.923031+0000","last_active":"2024-09-20T22:11:59.244114+0000","last_peered":"2024-09-20T22:11:59.244114+0000","last_clean":"2024-09-20T22:11:59.244114+0000","last_became_active":"2024-09-20T22:10:48.922707+0000","last_became_peered":"2024-09-20T22:10:48.922707+0000","last_unstale":"2024-09-20T22:11:59.244114+0000","last_undegraded":"2024-09-20T22:11:59.244114+0000","last_fullsized":"2024-09-20T22:11:59.244114+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:28.999559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:28.999559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:01.488607+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"276'10","reported_seq":643,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:12:03.510691+0000","last_change":"2024-09-20T22:12:03.510691+0000","last_active":"2024-09-20T22:12:03.510691+0000","last_peered":"2024-09-20T22:12:03.510691+0000","last_clean":"2024-09-20T22:12:03.510691+0000","last_became_active":"2024-09-20T22:11:12.492297+0000","last_became_peered":"2024-09-20T22:11:12.492297+0000","last_unstale":"2024-09-20T22:12:03.510691+0000","last_undegraded":"2024-09-20T22:12:03.510691+0000","last_fullsized":"2024-09-20T22:12:03.510691+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'10","last_scrub_stamp":"2024-09-20T22:12:03.510585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:03.510585+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:31.005661+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":514,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.272288+0000","last_change":"2024-09-20T22:11:30.485332+0000","last_active":"2024-09-20T22:12:07.272288+0000","last_peered":"2024-09-20T22:12:07.272288+0000","last_clean":"2024-09-20T22:12:07.272288+0000","last_became_active":"2024-09-20T22:11:30.484731+0000","last_became_peered":"2024-09-20T22:11:30.484731+0000","last_unstale":"2024-09-20T22:12:07.272288+0000","last_undegraded":"2024-09-20T22:12:07.272288+0000","last_fullsized":"2024-09-20T22:12:07.272288+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:05.183875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:05.183875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:32.622574+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"71'10","reported_seq":541,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.256608+0000","last_change":"2024-09-20T22:11:40.438463+0000","last_active":"2024-09-20T22:11:59.256608+0000","last_peered":"2024-09-20T22:11:59.256608+0000","last_clean":"2024-09-20T22:11:59.256608+0000","last_became_active":"2024-09-20T22:10:22.583028+0000","last_became_peered":"2024-09-20T22:10:22.583028+0000","last_unstale":"2024-09-20T22:11:59.256608+0000","last_undegraded":"2024-09-20T22:11:59.256608+0000","last_fullsized":"2024-09-20T22:11:59.256608+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:11:40.438345+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:40.438345+0000","objects_scrubbed":5,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:02.299535+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,7],"acting":[3,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":515,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:08.477834+0000","last_change":"2024-09-20T22:12:08.477834+0000","last_active":"2024-09-20T22:12:08.477834+0000","last_peered":"2024-09-20T22:12:08.477834+0000","last_clean":"2024-09-20T22:12:08.477834+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:12:08.477834+0000","last_undegraded":"2024-09-20T22:12:08.477834+0000","last_fullsized":"2024-09-20T22:12:08.477834+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:08.477741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:08.477741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:21.495162+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"283'12","reported_seq":70,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.278948+0000","last_change":"2024-09-20T22:11:35.062361+0000","last_active":"2024-09-20T22:12:07.278948+0000","last_peered":"2024-09-20T22:12:07.278948+0000","last_clean":"2024-09-20T22:12:07.278948+0000","last_became_active":"2024-09-20T22:11:30.488787+0000","last_became_peered":"2024-09-20T22:11:30.488787+0000","last_unstale":"2024-09-20T22:12:07.278948+0000","last_undegraded":"2024-09-20T22:12:07.278948+0000","last_fullsized":"2024-09-20T22:12:07.278948+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'12","last_scrub_stamp":"2024-09-20T22:11:35.062247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:35.062247+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:39.696169+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":510,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.244405+0000","last_change":"2024-09-20T22:11:05.577252+0000","last_active":"2024-09-20T22:11:59.244405+0000","last_peered":"2024-09-20T22:11:59.244405+0000","last_clean":"2024-09-20T22:11:59.244405+0000","last_became_active":"2024-09-20T22:10:30.659586+0000","last_became_peered":"2024-09-20T22:10:30.659586+0000","last_unstale":"2024-09-20T22:11:59.244405+0000","last_undegraded":"2024-09-20T22:11:59.244405+0000","last_fullsized":"2024-09-20T22:11:59.244405+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:05.577043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:05.577043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:11.961511+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"71'11","reported_seq":556,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:09.181590+0000","last_change":"2024-09-20T22:12:09.181635+0000","last_active":"2024-09-20T22:12:09.181590+0000","last_peered":"2024-09-20T22:12:09.181590+0000","last_clean":"2024-09-20T22:12:09.181590+0000","last_became_active":"2024-09-20T22:08:22.139581+0000","last_became_peered":"2024-09-20T22:08:22.139581+0000","last_unstale":"2024-09-20T22:12:09.181590+0000","last_undegraded":"2024-09-20T22:12:09.181590+0000","last_fullsized":"2024-09-20T22:12:09.181590+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:12:09.181513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:09.181513+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:28.642358+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":515,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.271537+0000","last_change":"2024-09-20T22:11:33.355268+0000","last_active":"2024-09-20T22:12:07.271537+0000","last_peered":"2024-09-20T22:12:07.271537+0000","last_clean":"2024-09-20T22:12:07.271537+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:12:07.271537+0000","last_undegraded":"2024-09-20T22:12:07.271537+0000","last_fullsized":"2024-09-20T22:12:07.271537+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.355142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.355142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:38.410766+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":351,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.294877+0000","last_change":"2024-09-20T22:11:34.160476+0000","last_active":"2024-09-20T22:12:07.294877+0000","last_peered":"2024-09-20T22:12:07.294877+0000","last_clean":"2024-09-20T22:12:07.294877+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:12:07.294877+0000","last_undegraded":"2024-09-20T22:12:07.294877+0000","last_fullsized":"2024-09-20T22:12:07.294877+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:34.160404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.160404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:49.059597+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":342,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.290403+0000","last_change":"2024-09-20T22:11:30.487252+0000","last_active":"2024-09-20T22:12:07.290403+0000","last_peered":"2024-09-20T22:12:07.290403+0000","last_clean":"2024-09-20T22:12:07.290403+0000","last_became_active":"2024-09-20T22:11:30.486423+0000","last_became_peered":"2024-09-20T22:11:30.486423+0000","last_unstale":"2024-09-20T22:12:07.290403+0000","last_undegraded":"2024-09-20T22:12:07.290403+0000","last_fullsized":"2024-09-20T22:12:07.290403+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:23.320831+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:23.320831+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"291'13","reported_seq":427,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.295280+0000","last_change":"2024-09-20T22:11:41.362291+0000","last_active":"2024-09-20T22:12:07.295280+0000","last_peered":"2024-09-20T22:12:07.295280+0000","last_clean":"2024-09-20T22:12:07.295280+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:12:07.295280+0000","last_undegraded":"2024-09-20T22:12:07.295280+0000","last_fullsized":"2024-09-20T22:12:07.295280+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'13","last_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_deep_scrub":"291'13","last_deep_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_clean_scrub_stamp":"2024-09-20T22:11:41.362215+0000","objects_scrubbed":4,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:08.259331+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16,"num_read_kb":10,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":111,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.275314+0000","last_change":"2024-09-20T22:10:30.656109+0000","last_active":"2024-09-20T22:12:07.275314+0000","last_peered":"2024-09-20T22:12:07.275314+0000","last_clean":"2024-09-20T22:12:07.275314+0000","last_became_active":"2024-09-20T22:10:30.655502+0000","last_became_peered":"2024-09-20T22:10:30.655502+0000","last_unstale":"2024-09-20T22:12:07.275314+0000","last_undegraded":"2024-09-20T22:12:07.275314+0000","last_fullsized":"2024-09-20T22:12:07.275314+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:28.129779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:28.129779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":504,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.249648+0000","last_change":"2024-09-20T22:11:31.262221+0000","last_active":"2024-09-20T22:11:59.249648+0000","last_peered":"2024-09-20T22:11:59.249648+0000","last_clean":"2024-09-20T22:11:59.249648+0000","last_became_active":"2024-09-20T22:06:54.844271+0000","last_became_peered":"2024-09-20T22:06:54.844271+0000","last_unstale":"2024-09-20T22:11:59.249648+0000","last_undegraded":"2024-09-20T22:11:59.249648+0000","last_fullsized":"2024-09-20T22:11:59.249648+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:31.262095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:31.262095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:38.538398+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":512,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.244745+0000","last_change":"2024-09-20T22:11:30.487611+0000","last_active":"2024-09-20T22:11:59.244745+0000","last_peered":"2024-09-20T22:11:59.244745+0000","last_clean":"2024-09-20T22:11:59.244745+0000","last_became_active":"2024-09-20T22:11:30.487088+0000","last_became_peered":"2024-09-20T22:11:30.487088+0000","last_unstale":"2024-09-20T22:11:59.244745+0000","last_undegraded":"2024-09-20T22:11:59.244745+0000","last_fullsized":"2024-09-20T22:11:59.244745+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:35.783884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:10:35.783884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:03.527369+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"71'11","reported_seq":561,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.255319+0000","last_change":"2024-09-20T22:10:30.672892+0000","last_active":"2024-09-20T22:11:59.255319+0000","last_peered":"2024-09-20T22:11:59.255319+0000","last_clean":"2024-09-20T22:11:59.255319+0000","last_became_active":"2024-09-20T22:10:30.651743+0000","last_became_peered":"2024-09-20T22:10:30.651743+0000","last_unstale":"2024-09-20T22:11:59.255319+0000","last_undegraded":"2024-09-20T22:11:59.255319+0000","last_fullsized":"2024-09-20T22:11:59.255319+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:09:55.321800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:09:55.321800+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":60,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.279054+0000","last_change":"2024-09-20T22:12:05.215296+0000","last_active":"2024-09-20T22:12:07.279054+0000","last_peered":"2024-09-20T22:12:07.279054+0000","last_clean":"2024-09-20T22:12:07.279054+0000","last_became_active":"2024-09-20T22:11:30.493598+0000","last_became_peered":"2024-09-20T22:11:30.493598+0000","last_unstale":"2024-09-20T22:12:07.279054+0000","last_undegraded":"2024-09-20T22:12:07.279054+0000","last_fullsized":"2024-09-20T22:12:07.279054+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:05.215188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_clean_scrub_stamp":"2024-09-20T22:12:05.215188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:17.841555+0000","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":506,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.274195+0000","last_change":"2024-09-20T22:09:01.751444+0000","last_active":"2024-09-20T22:12:07.274195+0000","last_peered":"2024-09-20T22:12:07.274195+0000","last_clean":"2024-09-20T22:12:07.274195+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:12:07.274195+0000","last_undegraded":"2024-09-20T22:12:07.274195+0000","last_fullsized":"2024-09-20T22:12:07.274195+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:01.751321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:09:01.751321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":155,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.246530+0000","last_change":"2024-09-20T22:09:39.492012+0000","last_active":"2024-09-20T22:11:59.246530+0000","last_peered":"2024-09-20T22:11:59.246530+0000","last_clean":"2024-09-20T22:11:59.246530+0000","last_became_active":"2024-09-20T22:09:39.491148+0000","last_became_peered":"2024-09-20T22:09:39.491148+0000","last_unstale":"2024-09-20T22:11:59.246530+0000","last_undegraded":"2024-09-20T22:11:59.246530+0000","last_fullsized":"2024-09-20T22:11:59.246530+0000","mapping_epoch":229,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":230,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:09:10.873660+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:09:10.873660+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"71'4","reported_seq":514,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.248344+0000","last_change":"2024-09-20T22:10:21.165887+0000","last_active":"2024-09-20T22:11:59.248344+0000","last_peered":"2024-09-20T22:11:59.248344+0000","last_clean":"2024-09-20T22:11:59.248344+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:11:59.248344+0000","last_undegraded":"2024-09-20T22:11:59.248344+0000","last_fullsized":"2024-09-20T22:11:59.248344+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'4","last_scrub_stamp":"2024-09-20T22:10:21.165774+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:10:21.165774+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":511,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:12:01.443534+0000","last_change":"2024-09-20T22:12:01.443569+0000","last_active":"2024-09-20T22:12:01.443534+0000","last_peered":"2024-09-20T22:12:01.443534+0000","last_clean":"2024-09-20T22:12:01.443534+0000","last_became_active":"2024-09-20T22:02:50.111142+0000","last_became_peered":"2024-09-20T22:02:50.111142+0000","last_unstale":"2024-09-20T22:12:01.443534+0000","last_undegraded":"2024-09-20T22:12:01.443534+0000","last_fullsized":"2024-09-20T22:12:01.443534+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:01.443465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:01.443465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:18.252756+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":529,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.286065+0000","last_change":"2024-09-20T22:11:34.342938+0000","last_active":"2024-09-20T22:12:07.286065+0000","last_peered":"2024-09-20T22:12:07.286065+0000","last_clean":"2024-09-20T22:12:07.286065+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:12:07.286065+0000","last_undegraded":"2024-09-20T22:12:07.286065+0000","last_fullsized":"2024-09-20T22:12:07.286065+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:34.342862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.342862+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:55.365769+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":502,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.274720+0000","last_change":"2024-09-20T22:11:05.468972+0000","last_active":"2024-09-20T22:12:07.274720+0000","last_peered":"2024-09-20T22:12:07.274720+0000","last_clean":"2024-09-20T22:12:07.274720+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:12:07.274720+0000","last_undegraded":"2024-09-20T22:12:07.274720+0000","last_fullsized":"2024-09-20T22:12:07.274720+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_clean_scrub_stamp":"2024-09-20T22:11:05.468863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:12:10.330169+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":400,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.349955+0000","last_change":"2024-09-20T22:12:07.349991+0000","last_active":"2024-09-20T22:12:07.349955+0000","last_peered":"2024-09-20T22:12:07.349955+0000","last_clean":"2024-09-20T22:12:07.349955+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:12:07.349955+0000","last_undegraded":"2024-09-20T22:12:07.349955+0000","last_fullsized":"2024-09-20T22:12:07.349955+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:07.349679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:07.349679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:29.418097+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":512,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.255732+0000","last_change":"2024-09-20T22:10:30.179319+0000","last_active":"2024-09-20T22:11:59.255732+0000","last_peered":"2024-09-20T22:11:59.255732+0000","last_clean":"2024-09-20T22:11:59.255732+0000","last_became_active":"2024-09-20T22:06:54.848032+0000","last_became_peered":"2024-09-20T22:06:54.848032+0000","last_unstale":"2024-09-20T22:11:59.255732+0000","last_undegraded":"2024-09-20T22:11:59.255732+0000","last_fullsized":"2024-09-20T22:11:59.255732+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:10:30.179244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:10:30.179244+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"71'11","reported_seq":546,"reported_epoch":302,"state":"active+clean","last_fresh":"2024-09-20T22:12:07.286611+0000","last_change":"2024-09-20T22:12:03.375105+0000","last_active":"2024-09-20T22:12:07.286611+0000","last_peered":"2024-09-20T22:12:07.286611+0000","last_clean":"2024-09-20T22:12:07.286611+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:12:07.286611+0000","last_undegraded":"2024-09-20T22:12:07.286611+0000","last_fullsized":"2024-09-20T22:12:07.286611+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:12:03.374976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:03.374976+0000","objects_scrubbed":5,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:13:15.507685+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":508,"reported_epoch":301,"state":"active+clean","last_fresh":"2024-09-20T22:11:59.244543+0000","last_change":"2024-09-20T22:11:12.495944+0000","last_active":"2024-09-20T22:11:59.244543+0000","last_peered":"2024-09-20T22:11:59.244543+0000","last_clean":"2024-09-20T22:11:59.244543+0000","last_became_active":"2024-09-20T22:11:12.495261+0000","last_became_peered":"2024-09-20T22:11:12.495261+0000","last_unstale":"2024-09-20T22:11:59.244543+0000","last_undegraded":"2024-09-20T22:11:59.244543+0000","last_fullsized":"2024-09-20T22:11:59.244543+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:08:32.854221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:08:32.854221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:11:59.578525+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":2964848640,"num_objects":45240,"num_object_clones":0,"num_object_copies":135720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45240,"num_write_kb":2895360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":9109831680,"data_stored":9109831680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":45240,"ondisk_log_size":45240,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":692,"num_read_kb":449,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":119,"num_bytes_recovered":1870,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":430,"ondisk_log_size":430,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1085,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":547,"num_read_kb":542,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":918048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1408,"ondisk_log_size":1408,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813135988,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2195752,"kb_used_data":1741144,"kb_used_omap":1,"kb_used_meta":454590,"kb_avail":91549400,"statfs":{"total":95995035648,"available":93746585600,"internally_reserved":0,"allocated":1782931456,"data_stored":1782279141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":465500618},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430328,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":941524,"kb_used_data":577156,"kb_used_omap":1,"kb_used_meta":364350,"kb_avail":92803628,"statfs":{"total":95995035648,"available":95030915072,"internally_reserved":0,"allocated":591007744,"data_stored":590375707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":373094854},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724668,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2185816,"kb_used_data":1713160,"kb_used_omap":1,"kb_used_meta":472638,"kb_avail":91559336,"statfs":{"total":95995035648,"available":93756760064,"internally_reserved":0,"allocated":1754275840,"data_stored":1753639714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":483981765},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,2,3,0,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986308,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":979308,"kb_used_data":583004,"kb_used_omap":1,"kb_used_meta":396286,"kb_avail":92765844,"statfs":{"total":95995035648,"available":94992224256,"internally_reserved":0,"allocated":596996096,"data_stored":596339518,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":405797317},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770257984,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1892996,"kb_used_data":1569008,"kb_used_omap":4,"kb_used_meta":323963,"kb_avail":91852156,"statfs":{"total":95995035648,"available":94056607744,"internally_reserved":0,"allocated":1606664192,"data_stored":1605988431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":331738792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149832,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1807604,"kb_used_data":1298852,"kb_used_omap":1,"kb_used_meta":508734,"kb_avail":91937548,"statfs":{"total":95995035648,"available":94144049152,"internally_reserved":0,"allocated":1330024448,"data_stored":1329353082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":520944071},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,5,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411470,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1045204,"kb_used_data":586436,"kb_used_omap":1,"kb_used_meta":458750,"kb_avail":92699948,"statfs":{"total":95995035648,"available":94924746752,"internally_reserved":0,"allocated":600510464,"data_stored":599878333,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":469760453},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738520,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1166620,"kb_used_data":662028,"kb_used_omap":1,"kb_used_meta":504574,"kb_avail":92578532,"statfs":{"total":95995035648,"available":94800416768,"internally_reserved":0,"allocated":677916672,"data_stored":677268235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":516684229},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":685506560,"data_stored":685506560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1641021440,"data_stored":1641021440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":615972864,"data_stored":615972864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1370423296,"data_stored":1370423296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":607059968,"data_stored":607059968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1796472832,"data_stored":1796472832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":597950464,"data_stored":597950464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1795424256,"data_stored":1795424256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:12:12.513 INFO:tasks.cephadm.ceph_manager.ceph:would drop below min_pgs, proj 1, currently 11 2024-09-20T22:12:12.513 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:12:12.514 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:12:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:12:12.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:12 smithi067 ceph-mon[33511]: 6.e scrub starts 2024-09-20T22:12:12.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:12 smithi067 ceph-mon[33511]: 4.1c scrub starts 2024-09-20T22:12:12.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:12 smithi067 ceph-mon[33511]: 6.e scrub ok 2024-09-20T22:12:12.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:12 smithi067 ceph-mon[33511]: 4.1c scrub ok 2024-09-20T22:12:12.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:12 smithi067 ceph-mon[33511]: osdmap e303: 8 total, 8 up, 8 in 2024-09-20T22:12:12.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:12 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:12:12.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:12 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:12:12.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:12 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/353296962' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:12.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:12.516 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:12.516 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:12.617 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:12.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:12.658 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:12.659 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:12.759 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:12 smithi060 ceph-mon[29694]: 6.e scrub starts 2024-09-20T22:12:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:12 smithi060 ceph-mon[29694]: 4.1c scrub starts 2024-09-20T22:12:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:12 smithi060 ceph-mon[29694]: 6.e scrub ok 2024-09-20T22:12:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:12 smithi060 ceph-mon[29694]: 4.1c scrub ok 2024-09-20T22:12:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:12 smithi060 ceph-mon[29694]: osdmap e303: 8 total, 8 up, 8 in 2024-09-20T22:12:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:12 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:12:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:12 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:12:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:12 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/353296962' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:12 smithi060 ceph-mon[40290]: 6.e scrub starts 2024-09-20T22:12:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:12 smithi060 ceph-mon[40290]: 4.1c scrub starts 2024-09-20T22:12:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:12 smithi060 ceph-mon[40290]: 6.e scrub ok 2024-09-20T22:12:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:12 smithi060 ceph-mon[40290]: 4.1c scrub ok 2024-09-20T22:12:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:12 smithi060 ceph-mon[40290]: osdmap e303: 8 total, 8 up, 8 in 2024-09-20T22:12:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:12 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:12:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:12 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:12:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:12 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/353296962' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:12.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:12.805 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:12.805 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:12.897 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 25 16 51802 51786 129.418 91.25 0.00907266 0.00772196 2024-09-20T22:12:12.906 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:12.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:12.947 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:12.948 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:13.048 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:13.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:13.094 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:13.094 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:13.195 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:13.231 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:13.239 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:13.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:13.240 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:13.313 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:13.341 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:13.381 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:13.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:13.382 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:13.483 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:13.512 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:13.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:13.513 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:13.614 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:13.642 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:13.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:13.643 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:13.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[29694]: from='client.15633 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:12:13.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[29694]: pgmap v499: 158 pgs: 158 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 123 MiB/s wr, 1.97k op/s 2024-09-20T22:12:13.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[29694]: 5.b scrub starts 2024-09-20T22:12:13.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[29694]: 5.b scrub ok 2024-09-20T22:12:13.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:12:13.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[29694]: osdmap e304: 8 total, 8 up, 8 in 2024-09-20T22:12:13.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[29694]: 6.8 scrub starts 2024-09-20T22:12:13.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[29694]: 6.8 scrub ok 2024-09-20T22:12:13.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[29694]: 4.3 scrub starts 2024-09-20T22:12:13.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[29694]: 4.3 scrub ok 2024-09-20T22:12:13.687 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[40290]: from='client.15633 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:12:13.687 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[40290]: pgmap v499: 158 pgs: 158 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 123 MiB/s wr, 1.97k op/s 2024-09-20T22:12:13.687 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[40290]: 5.b scrub starts 2024-09-20T22:12:13.687 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[40290]: 5.b scrub ok 2024-09-20T22:12:13.687 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:12:13.687 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[40290]: osdmap e304: 8 total, 8 up, 8 in 2024-09-20T22:12:13.687 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[40290]: 6.8 scrub starts 2024-09-20T22:12:13.687 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[40290]: 6.8 scrub ok 2024-09-20T22:12:13.687 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[40290]: 4.3 scrub starts 2024-09-20T22:12:13.687 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:13 smithi060 ceph-mon[40290]: 4.3 scrub ok 2024-09-20T22:12:13.744 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:13.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:13 smithi067 ceph-mon[33511]: from='client.15633 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:12:13.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:13 smithi067 ceph-mon[33511]: pgmap v499: 158 pgs: 158 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1.2 KiB/s rd, 123 MiB/s wr, 1.97k op/s 2024-09-20T22:12:13.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:13 smithi067 ceph-mon[33511]: 5.b scrub starts 2024-09-20T22:12:13.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:13 smithi067 ceph-mon[33511]: 5.b scrub ok 2024-09-20T22:12:13.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:13 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:12:13.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:13 smithi067 ceph-mon[33511]: osdmap e304: 8 total, 8 up, 8 in 2024-09-20T22:12:13.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:13 smithi067 ceph-mon[33511]: 6.8 scrub starts 2024-09-20T22:12:13.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:13 smithi067 ceph-mon[33511]: 6.8 scrub ok 2024-09-20T22:12:13.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:13 smithi067 ceph-mon[33511]: 4.3 scrub starts 2024-09-20T22:12:13.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:13 smithi067 ceph-mon[33511]: 4.3 scrub ok 2024-09-20T22:12:13.780 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:13.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:13.781 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:13.882 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:13.897 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 26 16 53206 53190 127.816 87.75 0.00946602 0.00781835 2024-09-20T22:12:13.921 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:13.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:13.922 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:14.023 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:14.052 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:14.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:14.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:14.154 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:14.190 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:14.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:14.191 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:14.292 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:14.325 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:14.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:14.326 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:14.427 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:14.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:14.466 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:14.466 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:14.567 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:14.588 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:14.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:14.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:14.690 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:14.724 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:14.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:14.725 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:14.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:14 smithi060 ceph-mon[29694]: 5.f scrub starts 2024-09-20T22:12:14.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:14 smithi060 ceph-mon[29694]: 5.f scrub ok 2024-09-20T22:12:14.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:14 smithi060 ceph-mon[29694]: 5.10 scrub starts 2024-09-20T22:12:14.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:14 smithi060 ceph-mon[29694]: 5.10 scrub ok 2024-09-20T22:12:14.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:14 smithi060 ceph-mon[29694]: 4.11 scrub starts 2024-09-20T22:12:14.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:14 smithi060 ceph-mon[29694]: 4.11 scrub ok 2024-09-20T22:12:14.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:14 smithi060 ceph-mon[29694]: 5.1e scrub starts 2024-09-20T22:12:14.744 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:14 smithi060 ceph-mon[29694]: 5.1e scrub ok 2024-09-20T22:12:14.744 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:14 smithi060 ceph-mon[40290]: 5.f scrub starts 2024-09-20T22:12:14.745 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:14 smithi060 ceph-mon[40290]: 5.f scrub ok 2024-09-20T22:12:14.745 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:14 smithi060 ceph-mon[40290]: 5.10 scrub starts 2024-09-20T22:12:14.745 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:14 smithi060 ceph-mon[40290]: 5.10 scrub ok 2024-09-20T22:12:14.745 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:14 smithi060 ceph-mon[40290]: 4.11 scrub starts 2024-09-20T22:12:14.745 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:14 smithi060 ceph-mon[40290]: 4.11 scrub ok 2024-09-20T22:12:14.745 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:14 smithi060 ceph-mon[40290]: 5.1e scrub starts 2024-09-20T22:12:14.745 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:14 smithi060 ceph-mon[40290]: 5.1e scrub ok 2024-09-20T22:12:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:14 smithi067 ceph-mon[33511]: 5.f scrub starts 2024-09-20T22:12:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:14 smithi067 ceph-mon[33511]: 5.f scrub ok 2024-09-20T22:12:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:14 smithi067 ceph-mon[33511]: 5.10 scrub starts 2024-09-20T22:12:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:14 smithi067 ceph-mon[33511]: 5.10 scrub ok 2024-09-20T22:12:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:14 smithi067 ceph-mon[33511]: 4.11 scrub starts 2024-09-20T22:12:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:14 smithi067 ceph-mon[33511]: 4.11 scrub ok 2024-09-20T22:12:14.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:14 smithi067 ceph-mon[33511]: 5.1e scrub starts 2024-09-20T22:12:14.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:14 smithi067 ceph-mon[33511]: 5.1e scrub ok 2024-09-20T22:12:14.826 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:14.861 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:14.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:14.863 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:14.897 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 27 16 54613 54597 126.339 87.9375 0.00696457 0.00791012 2024-09-20T22:12:14.963 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:14.992 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:14.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:14.993 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:15.094 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:15.140 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:15.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:15.141 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:15.189 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:12:15.242 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:15.289 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:15.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:15.290 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:15.391 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:15.407 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:12:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:12:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:12:15.421 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:15.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:15.422 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:15.523 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:15.552 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:15.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:15.553 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:15.654 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:15.681 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:15 smithi060 ceph-mon[29694]: pgmap v501: 158 pgs: 158 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 745 B/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-20T22:12:15.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:15 smithi060 ceph-mon[29694]: 5.16 scrub starts 2024-09-20T22:12:15.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:15 smithi060 ceph-mon[29694]: 5.16 scrub ok 2024-09-20T22:12:15.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:15 smithi060 ceph-mon[29694]: 5.2 scrub starts 2024-09-20T22:12:15.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:15 smithi060 ceph-mon[29694]: 5.2 scrub ok 2024-09-20T22:12:15.682 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:15 smithi060 ceph-mon[40290]: pgmap v501: 158 pgs: 158 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 745 B/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-20T22:12:15.682 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:15 smithi060 ceph-mon[40290]: 5.16 scrub starts 2024-09-20T22:12:15.682 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:15 smithi060 ceph-mon[40290]: 5.16 scrub ok 2024-09-20T22:12:15.682 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:15 smithi060 ceph-mon[40290]: 5.2 scrub starts 2024-09-20T22:12:15.682 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:15 smithi060 ceph-mon[40290]: 5.2 scrub ok 2024-09-20T22:12:15.685 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:15.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:15.686 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:15 smithi067 ceph-mon[33511]: pgmap v501: 158 pgs: 158 active+clean; 3.0 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 745 B/s rd, 82 MiB/s wr, 1.32k op/s 2024-09-20T22:12:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:15 smithi067 ceph-mon[33511]: 5.16 scrub starts 2024-09-20T22:12:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:15 smithi067 ceph-mon[33511]: 5.16 scrub ok 2024-09-20T22:12:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:15 smithi067 ceph-mon[33511]: 5.2 scrub starts 2024-09-20T22:12:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:15 smithi067 ceph-mon[33511]: 5.2 scrub ok 2024-09-20T22:12:15.787 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:15.809 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:15.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:15.810 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:15.897 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 28 16 55847 55831 124.581 77.125 0.00703335 0.00802207 2024-09-20T22:12:15.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:15.953 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:15.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:15.955 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:16.055 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:16.092 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:16.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:16.093 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:16.194 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:16.215 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:16.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:16.216 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:16.317 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:16.351 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:16.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:16.352 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:16.453 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:16.497 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:16.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:16.499 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:16.584 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:12:16.584 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:12:16.599 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:16.633 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:16.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:16.634 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:16.735 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:16 smithi067 ceph-mon[33511]: 1.8 scrub starts 2024-09-20T22:12:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:16 smithi067 ceph-mon[33511]: 1.8 scrub ok 2024-09-20T22:12:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:16 smithi067 ceph-mon[33511]: 3.e scrub starts 2024-09-20T22:12:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:16 smithi067 ceph-mon[33511]: 3.e scrub ok 2024-09-20T22:12:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:16 smithi067 ceph-mon[33511]: 6.13 scrub starts 2024-09-20T22:12:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:16 smithi067 ceph-mon[33511]: 6.13 scrub ok 2024-09-20T22:12:16.774 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:16.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:16.775 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:16.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:16 smithi060 ceph-mon[29694]: 1.8 scrub starts 2024-09-20T22:12:16.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:16 smithi060 ceph-mon[29694]: 1.8 scrub ok 2024-09-20T22:12:16.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:16 smithi060 ceph-mon[29694]: 3.e scrub starts 2024-09-20T22:12:16.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:16 smithi060 ceph-mon[29694]: 3.e scrub ok 2024-09-20T22:12:16.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:16 smithi060 ceph-mon[29694]: 6.13 scrub starts 2024-09-20T22:12:16.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:16 smithi060 ceph-mon[29694]: 6.13 scrub ok 2024-09-20T22:12:16.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:16 smithi060 ceph-mon[40290]: 1.8 scrub starts 2024-09-20T22:12:16.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:16 smithi060 ceph-mon[40290]: 1.8 scrub ok 2024-09-20T22:12:16.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:16 smithi060 ceph-mon[40290]: 3.e scrub starts 2024-09-20T22:12:16.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:16 smithi060 ceph-mon[40290]: 3.e scrub ok 2024-09-20T22:12:16.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:16 smithi060 ceph-mon[40290]: 6.13 scrub starts 2024-09-20T22:12:16.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:16 smithi060 ceph-mon[40290]: 6.13 scrub ok 2024-09-20T22:12:16.876 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:16.897 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 29 16 57288 57272 123.391 90.0625 0.0141847 0.0080989 2024-09-20T22:12:16.912 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:16.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:16.913 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:17.014 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:17.052 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:17.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:17.053 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:17.154 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:17.190 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:17.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:17.191 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:17.292 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:17.315 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:17.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:17.316 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:17.417 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:17.448 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:17.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:17.450 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:17.513 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:12:17.513 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:12:17.514 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-20T22:12:17.514 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 0.756419 2024-09-20T22:12:17.514 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 3 0.7564189639325687 2024-09-20T22:12:17.550 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:17.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:17.599 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:17.599 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:17.700 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:17.755 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:17.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:17.756 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:17 smithi067 ceph-mon[33511]: 1.1 scrub starts 2024-09-20T22:12:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:17 smithi067 ceph-mon[33511]: 1.1 scrub ok 2024-09-20T22:12:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:17 smithi067 ceph-mon[33511]: 5.e scrub starts 2024-09-20T22:12:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:17 smithi067 ceph-mon[33511]: 5.e scrub ok 2024-09-20T22:12:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:17 smithi067 ceph-mon[33511]: pgmap v502: 158 pgs: 158 active+clean; 3.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.37k op/s 2024-09-20T22:12:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:17 smithi067 ceph-mon[33511]: osdmap e305: 8 total, 8 up, 8 in 2024-09-20T22:12:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:17 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/205480378' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:12:17.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:17 smithi060 ceph-mon[29694]: 1.1 scrub starts 2024-09-20T22:12:17.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:17 smithi060 ceph-mon[29694]: 1.1 scrub ok 2024-09-20T22:12:17.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:17 smithi060 ceph-mon[29694]: 5.e scrub starts 2024-09-20T22:12:17.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:17 smithi060 ceph-mon[29694]: 5.e scrub ok 2024-09-20T22:12:17.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:17 smithi060 ceph-mon[29694]: pgmap v502: 158 pgs: 158 active+clean; 3.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.37k op/s 2024-09-20T22:12:17.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:17 smithi060 ceph-mon[29694]: osdmap e305: 8 total, 8 up, 8 in 2024-09-20T22:12:17.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:17 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/205480378' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:12:17.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:17 smithi060 ceph-mon[40290]: 1.1 scrub starts 2024-09-20T22:12:17.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:17 smithi060 ceph-mon[40290]: 1.1 scrub ok 2024-09-20T22:12:17.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:17 smithi060 ceph-mon[40290]: 5.e scrub starts 2024-09-20T22:12:17.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:17 smithi060 ceph-mon[40290]: 5.e scrub ok 2024-09-20T22:12:17.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:17 smithi060 ceph-mon[40290]: pgmap v502: 158 pgs: 158 active+clean; 3.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 85 MiB/s wr, 1.37k op/s 2024-09-20T22:12:17.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:17 smithi060 ceph-mon[40290]: osdmap e305: 8 total, 8 up, 8 in 2024-09-20T22:12:17.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:17 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/205480378' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:12:17.857 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:17.879 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:17.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:17.880 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:17.897 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 30 16 58561 58545 121.93 79.5625 0.00261322 0.0081673 2024-09-20T22:12:17.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:18.019 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:18.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:18.020 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:18.111 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T22:12:18.121 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:18.153 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:18.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:18.154 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:18.185 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:12:18.255 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:18.309 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:18.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:18.310 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:18.411 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:18.425 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:18 smithi060 ceph-mon[29694]: 5.19 scrub starts 2024-09-20T22:12:18.439 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:18.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:18.440 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:18.541 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:18.594 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:18.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:18.595 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:18.696 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:18.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:18.730 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:18.730 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:18 smithi067 ceph-mon[33511]: 5.19 scrub starts 2024-09-20T22:12:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:18 smithi067 ceph-mon[33511]: 5.19 scrub ok 2024-09-20T22:12:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:18 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:12:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:18 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/205480378' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:12:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:18 smithi067 ceph-mon[33511]: osdmap e306: 8 total, 8 up, 8 in 2024-09-20T22:12:18.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:18 smithi060 ceph-mon[29694]: 5.19 scrub ok 2024-09-20T22:12:18.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:18 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:12:18.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:18 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/205480378' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:12:18.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:18 smithi060 ceph-mon[29694]: osdmap e306: 8 total, 8 up, 8 in 2024-09-20T22:12:18.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:18 smithi060 ceph-mon[40290]: 5.19 scrub starts 2024-09-20T22:12:18.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:18 smithi060 ceph-mon[40290]: 5.19 scrub ok 2024-09-20T22:12:18.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:18 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:12:18.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:18 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/205480378' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:12:18.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:18 smithi060 ceph-mon[40290]: osdmap e306: 8 total, 8 up, 8 in 2024-09-20T22:12:18.831 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:18.871 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:18.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:18.872 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:18.897 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 31 16 60036 60020 120.97 92.1875 0.0186822 0.00826164 2024-09-20T22:12:18.973 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:18.981 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:19.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:19.035 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:19.035 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:19.136 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:19.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:19.171 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:19.171 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:19.272 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:19.304 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:19.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:19.305 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:19.305 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:19.406 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:19.440 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:19.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:19.441 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:19.542 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:19.585 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:19.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:19.586 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:19.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:19 smithi060 ceph-mon[29694]: pgmap v505: 158 pgs: 158 active+clean; 3.5 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-20T22:12:19.625 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:19 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:12:19.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:19 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:12:19.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:19 smithi060 ceph-mon[29694]: osdmap e307: 8 total, 8 up, 8 in 2024-09-20T22:12:19.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:19 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:12:19.626 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:19 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:12:19.626 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:19 smithi060 ceph-mon[40290]: pgmap v505: 158 pgs: 158 active+clean; 3.5 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-20T22:12:19.626 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:19 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:12:19.626 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:19 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:12:19.626 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:19 smithi060 ceph-mon[40290]: osdmap e307: 8 total, 8 up, 8 in 2024-09-20T22:12:19.626 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:19 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:12:19.627 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:19 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:12:19.686 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:19.727 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:19.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:19.729 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:19.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:19 smithi067 ceph-mon[33511]: pgmap v505: 158 pgs: 158 active+clean; 3.5 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 852 B/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-20T22:12:19.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:19 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:12:19.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:19 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:12:19.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:19 smithi067 ceph-mon[33511]: osdmap e307: 8 total, 8 up, 8 in 2024-09-20T22:12:19.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:19 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:12:19.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:19 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:12:19.829 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:19.853 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:19.858 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:19.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:19.859 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:19.899 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 32 16 61431 61415 119.91 87.1875 0.00727966 0.00833432 2024-09-20T22:12:19.941 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:12:19.942 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:12:19.960 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:19.979 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:12:19.979 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:12:20.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:20.016 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:20.016 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:20.024 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:12:20.024 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:12:20.076 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:12:20.077 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:12:20.116 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:12:20.116 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:12:20.117 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:20.154 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:12:20.154 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:12:20.159 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:20.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:20.160 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:20.199 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:12:20.199 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:12:20.234 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:12:20.234 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:12:20.261 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:20.289 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:20.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:20.290 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:20.390 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:20.434 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:20.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:20.435 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:20.516 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.3 primary-affinity to 0.756419 (8495722) 2024-09-20T22:12:20.537 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:20.571 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:20.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:20.572 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:20.672 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:20.695 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:20.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:20.696 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:20.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:20 smithi060 ceph-mon[29694]: osdmap e308: 8 total, 8 up, 8 in 2024-09-20T22:12:20.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:20 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/732030083' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:20.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:20 smithi060 ceph-mon[29694]: pgmap v508: 158 pgs: 158 active+clean; 3.5 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 60 MiB/s wr, 966 op/s 2024-09-20T22:12:20.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:12:20.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:20 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4103969019' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.7564189639325687}]: dispatch 2024-09-20T22:12:20.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:20 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.7564189639325687}]: dispatch 2024-09-20T22:12:20.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:20 smithi060 ceph-mon[40290]: osdmap e308: 8 total, 8 up, 8 in 2024-09-20T22:12:20.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:20 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/732030083' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:20.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:20 smithi060 ceph-mon[40290]: pgmap v508: 158 pgs: 158 active+clean; 3.5 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 60 MiB/s wr, 966 op/s 2024-09-20T22:12:20.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:12:20.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:20 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4103969019' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.7564189639325687}]: dispatch 2024-09-20T22:12:20.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:20 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.7564189639325687}]: dispatch 2024-09-20T22:12:20.797 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:20.820 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:20.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:20.821 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:20.899 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 33 16 62318 62302 117.956 55.4375 0.0030114 0.00839778 2024-09-20T22:12:20.921 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:20.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:20.944 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:20.945 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:20 smithi067 ceph-mon[33511]: osdmap e308: 8 total, 8 up, 8 in 2024-09-20T22:12:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:20 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/732030083' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:20 smithi067 ceph-mon[33511]: pgmap v508: 158 pgs: 158 active+clean; 3.5 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 60 MiB/s wr, 966 op/s 2024-09-20T22:12:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:12:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:20 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4103969019' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.7564189639325687}]: dispatch 2024-09-20T22:12:21.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:20 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.7564189639325687}]: dispatch 2024-09-20T22:12:21.045 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:12:21.045 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:21.069 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:21.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:21.070 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:21.170 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:21.197 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:21.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:21.198 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:21.298 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:21.321 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:21.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:21.322 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:21.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:21.445 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:21.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:21.446 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:21.522 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:21 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.7564189639325687}]': finished 2024-09-20T22:12:21.523 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:21 smithi060 ceph-mon[29694]: osdmap e309: 8 total, 8 up, 8 in 2024-09-20T22:12:21.523 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:21 smithi060 ceph-mon[29694]: osdmap e310: 8 total, 8 up, 8 in 2024-09-20T22:12:21.547 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:21.588 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:21.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:21.589 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:21.690 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:21.730 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:21.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:21.732 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:21.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:21 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.7564189639325687}]': finished 2024-09-20T22:12:21.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:21 smithi060 ceph-mon[40290]: osdmap e309: 8 total, 8 up, 8 in 2024-09-20T22:12:21.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:21 smithi060 ceph-mon[40290]: osdmap e310: 8 total, 8 up, 8 in 2024-09-20T22:12:21.833 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:21.866 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:21.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:21.867 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:21.899 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 34 16 62803 62787 115.379 30.3125 0.00494726 0.00866164 2024-09-20T22:12:21.968 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:22.006 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:22.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:22.007 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:21 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.7564189639325687}]': finished 2024-09-20T22:12:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:21 smithi067 ceph-mon[33511]: osdmap e309: 8 total, 8 up, 8 in 2024-09-20T22:12:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:21 smithi067 ceph-mon[33511]: osdmap e310: 8 total, 8 up, 8 in 2024-09-20T22:12:22.108 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:22.148 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:22.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:22.149 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:22.249 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:22.275 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:22.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:22.276 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:22.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:22.412 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:22.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:22.413 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:22.514 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:22.536 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:12:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:12:22.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:22.556 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:22.556 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:22.657 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:22.692 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:22.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:22.693 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:22.794 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:22.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:22.830 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:22.831 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:22.899 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 35 16 63907 63891 114.054 69 0.0101657 0.00876234 2024-09-20T22:12:22.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:22.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:22.970 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:22.970 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:23.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:22 smithi067 ceph-mon[33511]: pgmap v511: 158 pgs: 6 peering, 152 active+clean; 3.8 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 84 MiB/s wr, 1.34k op/s 2024-09-20T22:12:23.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:22 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:12:23.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:22 smithi060 ceph-mon[29694]: pgmap v511: 158 pgs: 6 peering, 152 active+clean; 3.8 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 84 MiB/s wr, 1.34k op/s 2024-09-20T22:12:23.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:22 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:12:23.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:22 smithi060 ceph-mon[40290]: pgmap v511: 158 pgs: 6 peering, 152 active+clean; 3.8 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1023 B/s rd, 84 MiB/s wr, 1.34k op/s 2024-09-20T22:12:23.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:22 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:12:23.071 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:23.106 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:23.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:23.107 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:23.208 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:23.234 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:23.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:23.235 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:23.336 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:23.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:23.378 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:23.379 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:23.479 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:23.512 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:23.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:23.513 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:23.614 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:23.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:23.645 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:23.646 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:23.745 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:12:23.746 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:12:23.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:12:23.746 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:12:23.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.746 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169961 7.13 7:cf59cdb9:::benchmark_data_smithi060_78800_object36928:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:12:23.746 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440329+0000", 2024-09-20T22:12:23.746 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 385.299378838, 2024-09-20T22:12:23.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.067711163, 2024-09-20T22:12:23.746 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.746 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.746 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:12:23.746 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.746 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:12:23.746 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:12:23.747 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169961 2024-09-20T22:12:23.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.747 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440329+0000", 2024-09-20T22:12:23.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440329+0000", 2024-09-20T22:12:23.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440330+0000", 2024-09-20T22:12:23.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2800000000000005e-07 2024-09-20T22:12:23.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440335+0000", 2024-09-20T22:12:23.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9520000000000002e-06 2024-09-20T22:12:23.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440347+0000", 2024-09-20T22:12:23.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1732999999999999e-05 2024-09-20T22:12:23.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440354+0000", 2024-09-20T22:12:23.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6420000000000007e-06 2024-09-20T22:12:23.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.698470+0000", 2024-09-20T22:12:23.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.25811523800000002 2024-09-20T22:12:23.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.698477+0000", 2024-09-20T22:12:23.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9999999999999999e-06 2024-09-20T22:12:23.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.507960+0000", 2024-09-20T22:12:23.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.809483493 2024-09-20T22:12:23.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.508040+0000", 2024-09-20T22:12:23.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0177000000000002e-05 2024-09-20T22:12:23.750 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.750 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.750 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.751 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169966 7.13 7:c9a28d28:::benchmark_data_smithi060_78800_object36933:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:12:23.751 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440368+0000", 2024-09-20T22:12:23.751 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 385.29933958200002, 2024-09-20T22:12:23.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0730672910000001, 2024-09-20T22:12:23.751 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.751 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.751 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:12:23.751 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.751 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:12:23.751 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:12:23.751 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169966 2024-09-20T22:12:23.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.752 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440368+0000", 2024-09-20T22:12:23.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440368+0000", 2024-09-20T22:12:23.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440369+0000", 2024-09-20T22:12:23.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9100000000000004e-07 2024-09-20T22:12:23.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440371+0000", 2024-09-20T22:12:23.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5280000000000002e-06 2024-09-20T22:12:23.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440378+0000", 2024-09-20T22:12:23.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7340000000000002e-06 2024-09-20T22:12:23.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440383+0000", 2024-09-20T22:12:23.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3750000000000002e-06 2024-09-20T22:12:23.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.698495+0000", 2024-09-20T22:12:23.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.258111427 2024-09-20T22:12:23.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.698499+0000", 2024-09-20T22:12:23.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.658e-06 2024-09-20T22:12:23.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.509530+0000", 2024-09-20T22:12:23.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.81103158399999997 2024-09-20T22:12:23.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:12:23.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.509656+0000", 2024-09-20T22:12:23.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012591300000000001 2024-09-20T22:12:23.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.513371+0000", 2024-09-20T22:12:23.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037147370000000001 2024-09-20T22:12:23.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.756 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.756 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.513436+0000", 2024-09-20T22:12:23.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4844000000000005e-05 2024-09-20T22:12:23.756 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.756 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.756 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.756 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198108 7.10 7:0b18f923:::benchmark_data_smithi060_78800_object65075:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:12:23.756 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.462286+0000", 2024-09-20T22:12:23.756 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 358.27742222799998, 2024-09-20T22:12:23.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97321715200000003, 2024-09-20T22:12:23.757 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.757 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.757 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:23.757 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.757 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:12:23.757 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:12:23.757 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198108 2024-09-20T22:12:23.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.757 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462286+0000", 2024-09-20T22:12:23.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462286+0000", 2024-09-20T22:12:23.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462287+0000", 2024-09-20T22:12:23.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.668e-06 2024-09-20T22:12:23.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462293+0000", 2024-09-20T22:12:23.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0020000000000003e-06 2024-09-20T22:12:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462315+0000", 2024-09-20T22:12:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1809000000000001e-05 2024-09-20T22:12:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462324+0000", 2024-09-20T22:12:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6209999999999993e-06 2024-09-20T22:12:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462389+0000", 2024-09-20T22:12:23.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5007999999999998e-05 2024-09-20T22:12:23.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462397+0000", 2024-09-20T22:12:23.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3690000000000007e-06 2024-09-20T22:12:23.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430729+0000", 2024-09-20T22:12:23.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.96833147100000005 2024-09-20T22:12:23.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:23.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430856+0000", 2024-09-20T22:12:23.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012690800000000001 2024-09-20T22:12:23.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:12:23.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.431205+0000", 2024-09-20T22:12:23.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00034923899999999999 2024-09-20T22:12:23.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435090+0000", 2024-09-20T22:12:23.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038847090000000001 2024-09-20T22:12:23.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:23.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435103+0000", 2024-09-20T22:12:23.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169999999999999e-05 2024-09-20T22:12:23.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435254+0000", 2024-09-20T22:12:23.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015143900000000001 2024-09-20T22:12:23.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:23.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435404+0000", 2024-09-20T22:12:23.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014957600000000001 2024-09-20T22:12:23.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435503+0000", 2024-09-20T22:12:23.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9162999999999999e-05 2024-09-20T22:12:23.763 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.763 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.763 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.763 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198131 7.10 7:0b2c8a99:::benchmark_data_smithi060_78800_object65098:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:12:23.763 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.479927+0000", 2024-09-20T22:12:23.763 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 358.25978089199998, 2024-09-20T22:12:23.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.96409556600000001, 2024-09-20T22:12:23.764 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.764 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.764 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:23.764 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.764 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:12:23.764 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:12:23.764 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198131 2024-09-20T22:12:23.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.764 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479927+0000", 2024-09-20T22:12:23.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479927+0000", 2024-09-20T22:12:23.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479929+0000", 2024-09-20T22:12:23.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5850000000000001e-06 2024-09-20T22:12:23.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479934+0000", 2024-09-20T22:12:23.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1869999999999996e-06 2024-09-20T22:12:23.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479955+0000", 2024-09-20T22:12:23.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1353e-05 2024-09-20T22:12:23.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479962+0000", 2024-09-20T22:12:23.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7549999999999997e-06 2024-09-20T22:12:23.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.480018+0000", 2024-09-20T22:12:23.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6342999999999999e-05 2024-09-20T22:12:23.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.480027+0000", 2024-09-20T22:12:23.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7469999999999994e-06 2024-09-20T22:12:23.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.433323+0000", 2024-09-20T22:12:23.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.95329631199999998 2024-09-20T22:12:23.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:23.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.433425+0000", 2024-09-20T22:12:23.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000101271 2024-09-20T22:12:23.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:12:23.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.433690+0000", 2024-09-20T22:12:23.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026574900000000002 2024-09-20T22:12:23.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:23.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.437320+0000", 2024-09-20T22:12:23.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003630075 2024-09-20T22:12:23.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.437363+0000", 2024-09-20T22:12:23.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2104000000000002e-05 2024-09-20T22:12:23.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.443794+0000", 2024-09-20T22:12:23.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0064310729999999998 2024-09-20T22:12:23.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:23.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.443902+0000", 2024-09-20T22:12:23.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010815 2024-09-20T22:12:23.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.444023+0000", 2024-09-20T22:12:23.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000120862 2024-09-20T22:12:23.771 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.771 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.771 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.771 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198163 7.10 7:086b2df9:::benchmark_data_smithi060_78800_object65130:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:12:23.771 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.501070+0000", 2024-09-20T22:12:23.771 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 358.23863751900001, 2024-09-20T22:12:23.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94904837799999997, 2024-09-20T22:12:23.771 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.771 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.772 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:23.772 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.772 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:12:23.772 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:12:23.772 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198163 2024-09-20T22:12:23.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.772 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501070+0000", 2024-09-20T22:12:23.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501070+0000", 2024-09-20T22:12:23.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501072+0000", 2024-09-20T22:12:23.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.742e-06 2024-09-20T22:12:23.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501078+0000", 2024-09-20T22:12:23.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3859999999999998e-06 2024-09-20T22:12:23.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501090+0000", 2024-09-20T22:12:23.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2003e-05 2024-09-20T22:12:23.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.774 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:23.775 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:23 smithi067 ceph-mon[33511]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-20T22:12:23.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501097+0000", 2024-09-20T22:12:23.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2089999999999999e-06 2024-09-20T22:12:23.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501154+0000", 2024-09-20T22:12:23.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6901000000000003e-05 2024-09-20T22:12:23.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501162+0000", 2024-09-20T22:12:23.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3440000000000001e-06 2024-09-20T22:12:23.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435556+0000", 2024-09-20T22:12:23.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.93439354699999999 2024-09-20T22:12:23.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:23.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435672+0000", 2024-09-20T22:12:23.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011694 2024-09-20T22:12:23.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:12:23.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435963+0000", 2024-09-20T22:12:23.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000290654 2024-09-20T22:12:23.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:23.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.440444+0000", 2024-09-20T22:12:23.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0044806450000000001 2024-09-20T22:12:23.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.440486+0000", 2024-09-20T22:12:23.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2036999999999997e-05 2024-09-20T22:12:23.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.449968+0000", 2024-09-20T22:12:23.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0094824560000000002 2024-09-20T22:12:23.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:23.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450024+0000", 2024-09-20T22:12:23.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5374000000000002e-05 2024-09-20T22:12:23.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450119+0000", 2024-09-20T22:12:23.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5140000000000001e-05 2024-09-20T22:12:23.780 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.780 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.780 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.780 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227634 7.10 7:0ea5dec3:::benchmark_data_smithi060_78800_object94601:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:12:23.780 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.846865+0000", 2024-09-20T22:12:23.780 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 329.89284269299998, 2024-09-20T22:12:23.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1004788329999999, 2024-09-20T22:12:23.780 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.781 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.781 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:23.781 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.781 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:12:23.781 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:12:23.781 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227634 2024-09-20T22:12:23.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.781 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846865+0000", 2024-09-20T22:12:23.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846865+0000", 2024-09-20T22:12:23.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846867+0000", 2024-09-20T22:12:23.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.641e-06 2024-09-20T22:12:23.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846873+0000", 2024-09-20T22:12:23.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1489999999999999e-06 2024-09-20T22:12:23.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846887+0000", 2024-09-20T22:12:23.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4113e-05 2024-09-20T22:12:23.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846895+0000", 2024-09-20T22:12:23.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6699999999999994e-06 2024-09-20T22:12:23.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.852299+0000", 2024-09-20T22:12:23.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0054038059999999997 2024-09-20T22:12:23.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.852311+0000", 2024-09-20T22:12:23.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2278e-05 2024-09-20T22:12:23.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.941457+0000", 2024-09-20T22:12:23.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0891458009999999 2024-09-20T22:12:23.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:23.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.941594+0000", 2024-09-20T22:12:23.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013708000000000001 2024-09-20T22:12:23.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2,5", 2024-09-20T22:12:23.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.941949+0000", 2024-09-20T22:12:23.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00035502599999999999 2024-09-20T22:12:23.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946263+0000", 2024-09-20T22:12:23.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0043141150000000003 2024-09-20T22:12:23.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:23.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946286+0000", 2024-09-20T22:12:23.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3419000000000001e-05 2024-09-20T22:12:23.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946351+0000", 2024-09-20T22:12:23.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4231000000000002e-05 2024-09-20T22:12:23.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947173+0000", 2024-09-20T22:12:23.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00082238199999999997 2024-09-20T22:12:23.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:23.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947237+0000", 2024-09-20T22:12:23.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4462999999999996e-05 2024-09-20T22:12:23.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947344+0000", 2024-09-20T22:12:23.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000106659 2024-09-20T22:12:23.789 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.789 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.789 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.789 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227639 7.10 7:0834985a:::benchmark_data_smithi060_78800_object94606:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:12:23.789 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.856347+0000", 2024-09-20T22:12:23.789 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 329.88336113000003, 2024-09-20T22:12:23.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0915877439999999, 2024-09-20T22:12:23.790 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.790 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.790 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:23.790 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.790 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:12:23.790 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:12:23.790 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227639 2024-09-20T22:12:23.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.790 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856347+0000", 2024-09-20T22:12:23.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856347+0000", 2024-09-20T22:12:23.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856349+0000", 2024-09-20T22:12:23.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7090000000000001e-06 2024-09-20T22:12:23.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856355+0000", 2024-09-20T22:12:23.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4729999999999997e-06 2024-09-20T22:12:23.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856369+0000", 2024-09-20T22:12:23.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3832000000000001e-05 2024-09-20T22:12:23.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856378+0000", 2024-09-20T22:12:23.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8689999999999993e-06 2024-09-20T22:12:23.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856556+0000", 2024-09-20T22:12:23.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000178529 2024-09-20T22:12:23.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856568+0000", 2024-09-20T22:12:23.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2281e-05 2024-09-20T22:12:23.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.944093+0000", 2024-09-20T22:12:23.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.087524438 2024-09-20T22:12:23.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:23.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.944232+0000", 2024-09-20T22:12:23.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000139178 2024-09-20T22:12:23.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2,5", 2024-09-20T22:12:23.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.944533+0000", 2024-09-20T22:12:23.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030114999999999997 2024-09-20T22:12:23.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946726+0000", 2024-09-20T22:12:23.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021925809999999999 2024-09-20T22:12:23.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:23.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946797+0000", 2024-09-20T22:12:23.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1570999999999996e-05 2024-09-20T22:12:23.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947394+0000", 2024-09-20T22:12:23.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00059673799999999998 2024-09-20T22:12:23.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947773+0000", 2024-09-20T22:12:23.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00037865499999999999 2024-09-20T22:12:23.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:23.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947831+0000", 2024-09-20T22:12:23.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8557000000000002e-05 2024-09-20T22:12:23.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947935+0000", 2024-09-20T22:12:23.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103183 2024-09-20T22:12:23.798 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.798 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.798 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.798 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227649 7.2 7:46c04214:::benchmark_data_smithi060_78800_object94616:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:12:23.798 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.900917+0000", 2024-09-20T22:12:23.798 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 329.83879108500003, 2024-09-20T22:12:23.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1912150110000002, 2024-09-20T22:12:23.798 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.798 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.798 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:23.799 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.799 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:12:23.799 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:12:23.799 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227649 2024-09-20T22:12:23.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.799 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900917+0000", 2024-09-20T22:12:23.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900917+0000", 2024-09-20T22:12:23.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900919+0000", 2024-09-20T22:12:23.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7400000000000001e-06 2024-09-20T22:12:23.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900924+0000", 2024-09-20T22:12:23.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6439999999999997e-06 2024-09-20T22:12:23.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900941+0000", 2024-09-20T22:12:23.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7244e-05 2024-09-20T22:12:23.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900949+0000", 2024-09-20T22:12:23.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0829999999999998e-06 2024-09-20T22:12:23.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.901025+0000", 2024-09-20T22:12:23.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6643999999999993e-05 2024-09-20T22:12:23.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.901037+0000", 2024-09-20T22:12:23.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1824999999999999e-05 2024-09-20T22:12:23.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.817605+0000", 2024-09-20T22:12:23.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91656772500000006 2024-09-20T22:12:23.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.817617+0000", 2024-09-20T22:12:23.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2320999999999999e-05 2024-09-20T22:12:23.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.803 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.803 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.067011+0000", 2024-09-20T22:12:23.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2493934259999999 2024-09-20T22:12:23.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.803 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:23.803 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.067109+0000", 2024-09-20T22:12:23.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8904e-05 2024-09-20T22:12:23.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.803 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:12:23.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.067424+0000", 2024-09-20T22:12:23.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00031460400000000001 2024-09-20T22:12:23.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:23.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.069092+0000", 2024-09-20T22:12:23.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001667836 2024-09-20T22:12:23.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.070743+0000", 2024-09-20T22:12:23.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001650763 2024-09-20T22:12:23.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.091949+0000", 2024-09-20T22:12:23.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021206434999999999 2024-09-20T22:12:23.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:23.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092012+0000", 2024-09-20T22:12:23.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2651999999999996e-05 2024-09-20T22:12:23.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092132+0000", 2024-09-20T22:12:23.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000120165 2024-09-20T22:12:23.805 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.805 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.806 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.806 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227675 7.2 7:45bb0f01:::benchmark_data_smithi060_78800_object94642:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-20T22:12:23.806 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:54.993128+0000", 2024-09-20T22:12:23.806 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 328.74657985300001, 2024-09-20T22:12:23.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.09956863, 2024-09-20T22:12:23.806 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.806 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.806 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:23.806 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.806 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:12:23.806 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:12:23.806 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227675 2024-09-20T22:12:23.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.807 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993128+0000", 2024-09-20T22:12:23.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993128+0000", 2024-09-20T22:12:23.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993130+0000", 2024-09-20T22:12:23.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5400000000000001e-06 2024-09-20T22:12:23.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993136+0000", 2024-09-20T22:12:23.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.669e-06 2024-09-20T22:12:23.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993153+0000", 2024-09-20T22:12:23.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6538999999999999e-05 2024-09-20T22:12:23.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993162+0000", 2024-09-20T22:12:23.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2529999999999993e-06 2024-09-20T22:12:23.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993239+0000", 2024-09-20T22:12:23.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6499000000000001e-05 2024-09-20T22:12:23.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993253+0000", 2024-09-20T22:12:23.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4278e-05 2024-09-20T22:12:23.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.069151+0000", 2024-09-20T22:12:23.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0758977059999999 2024-09-20T22:12:23.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:23.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.069260+0000", 2024-09-20T22:12:23.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109175 2024-09-20T22:12:23.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:12:23.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.069491+0000", 2024-09-20T22:12:23.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000231716 2024-09-20T22:12:23.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:23.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.070761+0000", 2024-09-20T22:12:23.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012690850000000001 2024-09-20T22:12:23.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.071295+0000", 2024-09-20T22:12:23.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00053413799999999997 2024-09-20T22:12:23.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092555+0000", 2024-09-20T22:12:23.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021260233 2024-09-20T22:12:23.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:23.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092609+0000", 2024-09-20T22:12:23.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4147999999999997e-05 2024-09-20T22:12:23.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.813 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.813 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092697+0000", 2024-09-20T22:12:23.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7651000000000005e-05 2024-09-20T22:12:23.813 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.813 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.813 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.813 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236464 7.2 7:41e7f20e:::benchmark_data_smithi060_78800_object103431:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:12:23.813 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.420278+0000", 2024-09-20T22:12:23.813 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 316.31942983499999, 2024-09-20T22:12:23.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99395603799999999, 2024-09-20T22:12:23.814 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.814 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.814 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:23.814 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.814 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:12:23.814 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:12:23.814 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236464 2024-09-20T22:12:23.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.814 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.814 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420278+0000", 2024-09-20T22:12:23.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420278+0000", 2024-09-20T22:12:23.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420280+0000", 2024-09-20T22:12:23.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0049999999999999e-06 2024-09-20T22:12:23.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420287+0000", 2024-09-20T22:12:23.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7429999999999996e-06 2024-09-20T22:12:23.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420305+0000", 2024-09-20T22:12:23.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.836e-05 2024-09-20T22:12:23.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420314+0000", 2024-09-20T22:12:23.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7329999999999992e-06 2024-09-20T22:12:23.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.421421+0000", 2024-09-20T22:12:23.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001107455 2024-09-20T22:12:23.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.421429+0000", 2024-09-20T22:12:23.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5669999999999997e-06 2024-09-20T22:12:23.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.405308+0000", 2024-09-20T22:12:23.818 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98387922900000002 2024-09-20T22:12:23.818 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.818 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.818 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:23.818 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.405428+0000", 2024-09-20T22:12:23.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012023699999999999 2024-09-20T22:12:23.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:12:23.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.405861+0000", 2024-09-20T22:12:23.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00043245600000000001 2024-09-20T22:12:23.819 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.819 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.819 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:23.819 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414023+0000", 2024-09-20T22:12:23.819 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0081622280000000005 2024-09-20T22:12:23.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414092+0000", 2024-09-20T22:12:23.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9129999999999997e-05 2024-09-20T22:12:23.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.820 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.820 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.820 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414141+0000", 2024-09-20T22:12:23.820 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8426e-05 2024-09-20T22:12:23.820 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:23.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414196+0000", 2024-09-20T22:12:23.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5226999999999998e-05 2024-09-20T22:12:23.821 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.821 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.821 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.821 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414234+0000", 2024-09-20T22:12:23.821 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8241999999999997e-05 2024-09-20T22:12:23.821 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.821 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.821 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.822 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.822 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.822 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236481 7.2 7:417f7fb8:::benchmark_data_smithi060_78800_object103448:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:12:23.822 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.436984+0000", 2024-09-20T22:12:23.822 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 316.30272361499999, 2024-09-20T22:12:23.822 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98983631900000002, 2024-09-20T22:12:23.822 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.822 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.823 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:23 smithi060 ceph-mon[29694]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-20T22:12:23.824 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:23 smithi060 ceph-mon[40290]: Health check failed: Reduced data availability: 1 pg inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-20T22:12:23.824 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:23.825 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.825 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:12:23.825 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:12:23.825 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236481 2024-09-20T22:12:23.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.825 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.825 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.825 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.825 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.436984+0000", 2024-09-20T22:12:23.825 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.825 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.436984+0000", 2024-09-20T22:12:23.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.436985+0000", 2024-09-20T22:12:23.826 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-07 2024-09-20T22:12:23.826 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.826 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.826 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.826 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.436990+0000", 2024-09-20T22:12:23.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7770000000000002e-06 2024-09-20T22:12:23.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.437004+0000", 2024-09-20T22:12:23.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3572e-05 2024-09-20T22:12:23.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.827 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.437010+0000", 2024-09-20T22:12:23.827 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9469999999999997e-06 2024-09-20T22:12:23.827 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.827 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.827 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.439285+0000", 2024-09-20T22:12:23.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022742539999999999 2024-09-20T22:12:23.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.439294+0000", 2024-09-20T22:12:23.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0720000000000006e-06 2024-09-20T22:12:23.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.828 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.828 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.828 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.408421+0000", 2024-09-20T22:12:23.828 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.96912711200000001 2024-09-20T22:12:23.828 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:23.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.408517+0000", 2024-09-20T22:12:23.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.624e-05 2024-09-20T22:12:23.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:12:23.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.408884+0000", 2024-09-20T22:12:23.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00036694800000000002 2024-09-20T22:12:23.829 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.829 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.829 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.829 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414277+0000", 2024-09-20T22:12:23.829 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0053932440000000002 2024-09-20T22:12:23.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.419733+0000", 2024-09-20T22:12:23.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0054556500000000003 2024-09-20T22:12:23.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:23.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426386+0000", 2024-09-20T22:12:23.830 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0066532680000000004 2024-09-20T22:12:23.830 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.830 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.830 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:23.830 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426769+0000", 2024-09-20T22:12:23.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000382347 2024-09-20T22:12:23.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.831 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.831 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426821+0000", 2024-09-20T22:12:23.831 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2008000000000001e-05 2024-09-20T22:12:23.831 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.831 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.831 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.831 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.831 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.831 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106980 8.18 8:1c2b9c14:::benchmark_data_smithi060_111051_object106979:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:12:23.831 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.414151+0000", 2024-09-20T22:12:23.831 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 192.325557334, 2024-09-20T22:12:23.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.111436294, 2024-09-20T22:12:23.832 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.832 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.832 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:23.832 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.832 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:23.832 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:23.832 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106980 2024-09-20T22:12:23.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.832 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.832 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.832 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.832 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414151+0000", 2024-09-20T22:12:23.832 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.832 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414151+0000", 2024-09-20T22:12:23.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414152+0000", 2024-09-20T22:12:23.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5179999999999999e-06 2024-09-20T22:12:23.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.833 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.833 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.833 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414179+0000", 2024-09-20T22:12:23.833 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6480999999999999e-05 2024-09-20T22:12:23.833 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414191+0000", 2024-09-20T22:12:23.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2833999999999999e-05 2024-09-20T22:12:23.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414200+0000", 2024-09-20T22:12:23.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1040000000000007e-06 2024-09-20T22:12:23.834 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.834 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.834 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.834 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418058+0000", 2024-09-20T22:12:23.834 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038580139999999999 2024-09-20T22:12:23.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418080+0000", 2024-09-20T22:12:23.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2438000000000001e-05 2024-09-20T22:12:23.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.501675+0000", 2024-09-20T22:12:23.835 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0835947560000001 2024-09-20T22:12:23.835 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.835 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.835 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:23.835 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.501814+0000", 2024-09-20T22:12:23.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013951799999999999 2024-09-20T22:12:23.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4,5,7", 2024-09-20T22:12:23.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.502081+0000", 2024-09-20T22:12:23.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026697100000000002 2024-09-20T22:12:23.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:23.836 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.503435+0000", 2024-09-20T22:12:23.836 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001354156 2024-09-20T22:12:23.836 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.836 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.836 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.514128+0000", 2024-09-20T22:12:23.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010692432 2024-09-20T22:12:23.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525290+0000", 2024-09-20T22:12:23.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.011161901 2024-09-20T22:12:23.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.837 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.837 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525399+0000", 2024-09-20T22:12:23.837 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010921699999999999 2024-09-20T22:12:23.837 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.837 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:23.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525468+0000", 2024-09-20T22:12:23.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8726999999999994e-05 2024-09-20T22:12:23.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.838 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.838 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525587+0000", 2024-09-20T22:12:23.838 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119227 2024-09-20T22:12:23.838 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.838 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.838 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.838 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.838 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.839 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187847 8.0 8:0387ec14:::benchmark_data_smithi060_111051_object67848:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:12:23.839 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.626135+0000", 2024-09-20T22:12:23.839 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 114.11357314, 2024-09-20T22:12:23.839 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2145214800000002, 2024-09-20T22:12:23.839 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.839 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.839 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:12:23.839 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.839 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:23.839 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:23.839 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187847 2024-09-20T22:12:23.839 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.839 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626135+0000", 2024-09-20T22:12:23.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626135+0000", 2024-09-20T22:12:23.840 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.840 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.840 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.840 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.840 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626137+0000", 2024-09-20T22:12:23.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0389999999999999e-06 2024-09-20T22:12:23.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626144+0000", 2024-09-20T22:12:23.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9299999999999997e-06 2024-09-20T22:12:23.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.841 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626285+0000", 2024-09-20T22:12:23.841 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000141445 2024-09-20T22:12:23.841 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.841 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.841 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626296+0000", 2024-09-20T22:12:23.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0873000000000001e-05 2024-09-20T22:12:23.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626432+0000", 2024-09-20T22:12:23.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000135537 2024-09-20T22:12:23.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.842 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.842 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:23.842 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626543+0000", 2024-09-20T22:12:23.842 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011132399999999999 2024-09-20T22:12:23.842 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,5", 2024-09-20T22:12:23.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627183+0000", 2024-09-20T22:12:23.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00064020400000000001 2024-09-20T22:12:23.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.656943+0000", 2024-09-20T22:12:23.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029759622999999999 2024-09-20T22:12:23.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.843 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.843 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.843 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.656954+0000", 2024-09-20T22:12:23.843 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0913999999999999e-05 2024-09-20T22:12:23.843 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651133+0000", 2024-09-20T22:12:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99417974399999998 2024-09-20T22:12:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651152+0000", 2024-09-20T22:12:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8108999999999999e-05 2024-09-20T22:12:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.840586+0000", 2024-09-20T22:12:23.844 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1894341669999999 2024-09-20T22:12:23.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.845 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.845 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.840656+0000", 2024-09-20T22:12:23.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0570999999999999e-05 2024-09-20T22:12:23.845 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.845 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.845 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.845 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.845 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.845 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187874 8.0 8:032cd1ec:::benchmark_data_smithi060_111051_object67875:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:12:23.845 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.678008+0000", 2024-09-20T22:12:23.845 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 114.061699776, 2024-09-20T22:12:23.845 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1758792680000001, 2024-09-20T22:12:23.846 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.846 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.846 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:23.846 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.846 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:23.846 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:23.846 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187874 2024-09-20T22:12:23.846 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.846 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.846 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.846 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.846 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678008+0000", 2024-09-20T22:12:23.846 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678008+0000", 2024-09-20T22:12:23.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678010+0000", 2024-09-20T22:12:23.847 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7430000000000001e-06 2024-09-20T22:12:23.847 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.847 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.847 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.847 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678016+0000", 2024-09-20T22:12:23.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4339999999999997e-06 2024-09-20T22:12:23.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678030+0000", 2024-09-20T22:12:23.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3971999999999999e-05 2024-09-20T22:12:23.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.848 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678038+0000", 2024-09-20T22:12:23.848 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5100000000000001e-06 2024-09-20T22:12:23.848 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.848 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.848 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.849 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678109+0000", 2024-09-20T22:12:23.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0708000000000001e-05 2024-09-20T22:12:23.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.849 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678119+0000", 2024-09-20T22:12:23.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0519e-05 2024-09-20T22:12:23.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.849 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.849 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651185+0000", 2024-09-20T22:12:23.849 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97306574199999996 2024-09-20T22:12:23.849 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.849 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651193+0000", 2024-09-20T22:12:23.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2609999999999993e-06 2024-09-20T22:12:23.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.840972+0000", 2024-09-20T22:12:23.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.189778644 2024-09-20T22:12:23.850 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.850 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.850 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:23.850 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.841244+0000", 2024-09-20T22:12:23.850 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00027224500000000001 2024-09-20T22:12:23.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4,5", 2024-09-20T22:12:23.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.841873+0000", 2024-09-20T22:12:23.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000629124 2024-09-20T22:12:23.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.851 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.852639+0000", 2024-09-20T22:12:23.851 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010765925000000001 2024-09-20T22:12:23.851 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.851 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.851 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.852704+0000", 2024-09-20T22:12:23.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5011000000000005e-05 2024-09-20T22:12:23.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:23.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.852729+0000", 2024-09-20T22:12:23.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4782000000000001e-05 2024-09-20T22:12:23.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.852 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.852 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.853614+0000", 2024-09-20T22:12:23.852 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00088560699999999995 2024-09-20T22:12:23.852 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.852 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:23.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.853671+0000", 2024-09-20T22:12:23.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6382999999999999e-05 2024-09-20T22:12:23.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.853 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.853 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.853887+0000", 2024-09-20T22:12:23.853 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000216658 2024-09-20T22:12:23.853 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.853 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.853 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.853 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.853 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.853 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187894 8.0 8:02b3e725:::benchmark_data_smithi060_111051_object67895:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:12:23.854 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.701140+0000", 2024-09-20T22:12:23.854 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 114.038568028, 2024-09-20T22:12:23.854 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1634560220000001, 2024-09-20T22:12:23.854 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.854 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.854 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:23.854 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.854 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:23.854 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:23.854 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187894 2024-09-20T22:12:23.854 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.854 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.854 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701140+0000", 2024-09-20T22:12:23.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701140+0000", 2024-09-20T22:12:23.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.855 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.855 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.855 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.855 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701141+0000", 2024-09-20T22:12:23.855 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2950000000000001e-06 2024-09-20T22:12:23.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701147+0000", 2024-09-20T22:12:23.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3890000000000004e-06 2024-09-20T22:12:23.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701168+0000", 2024-09-20T22:12:23.856 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1725999999999998e-05 2024-09-20T22:12:23.856 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.856 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.856 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.856 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701179+0000", 2024-09-20T22:12:23.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0393e-05 2024-09-20T22:12:23.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701245+0000", 2024-09-20T22:12:23.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6519999999999993e-05 2024-09-20T22:12:23.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.857 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.857 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701256+0000", 2024-09-20T22:12:23.857 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0722e-05 2024-09-20T22:12:23.857 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.857 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651217+0000", 2024-09-20T22:12:23.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94996075899999999 2024-09-20T22:12:23.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.654608+0000", 2024-09-20T22:12:23.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0033909819999999999 2024-09-20T22:12:23.858 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.858 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.858 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.858 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.849926+0000", 2024-09-20T22:12:23.858 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.195318353 2024-09-20T22:12:23.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:23.859 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.850057+0000", 2024-09-20T22:12:23.859 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013062999999999999 2024-09-20T22:12:23.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4,5", 2024-09-20T22:12:23.859 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.850462+0000", 2024-09-20T22:12:23.859 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00040517 2024-09-20T22:12:23.859 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.859 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.859 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:23.860 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.852996+0000", 2024-09-20T22:12:23.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025345179999999999 2024-09-20T22:12:23.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.860 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.860 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.854077+0000", 2024-09-20T22:12:23.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001080262 2024-09-20T22:12:23.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.860 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.860 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.864365+0000", 2024-09-20T22:12:23.860 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010287907000000001 2024-09-20T22:12:23.860 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.860 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.864413+0000", 2024-09-20T22:12:23.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8690999999999998e-05 2024-09-20T22:12:23.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:23.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.864473+0000", 2024-09-20T22:12:23.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9487000000000002e-05 2024-09-20T22:12:23.861 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.861 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.861 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.861 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.864596+0000", 2024-09-20T22:12:23.861 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000123218 2024-09-20T22:12:23.861 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.862 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.862 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.862 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.862 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.862 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214320 8.8 8:125eda0b:::benchmark_data_smithi060_111051_object94321:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:12:23.862 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:12:23.862 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.243326155000005, 2024-09-20T22:12:23.862 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0054397049999999, 2024-09-20T22:12:23.862 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.862 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.862 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:23.862 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.862 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:23.862 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:23.863 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214320 2024-09-20T22:12:23.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.863 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.863 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.863 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:12:23.863 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.863 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.863 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.863 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:12:23.863 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.863 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.864 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.864 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:12:23.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0999999999999999e-07 2024-09-20T22:12:23.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.864 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.864 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496384+0000", 2024-09-20T22:12:23.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1859999999999999e-06 2024-09-20T22:12:23.864 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.864 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.864 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.864 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496390+0000", 2024-09-20T22:12:23.864 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9920000000000004e-06 2024-09-20T22:12:23.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496395+0000", 2024-09-20T22:12:23.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.741e-06 2024-09-20T22:12:23.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496429+0000", 2024-09-20T22:12:23.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3736999999999999e-05 2024-09-20T22:12:23.865 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.865 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.865 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.865 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496436+0000", 2024-09-20T22:12:23.865 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2529999999999998e-06 2024-09-20T22:12:23.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.497957+0000", 2024-09-20T22:12:23.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.001521109 2024-09-20T22:12:23.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.866 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.866 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:23.866 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.498097+0000", 2024-09-20T22:12:23.866 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000139562 2024-09-20T22:12:23.866 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:23.867 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:23.867 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:23.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.868 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:12:23.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.498422+0000", 2024-09-20T22:12:23.868 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00032548199999999998 2024-09-20T22:12:23.868 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.868 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.868 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.868 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501661+0000", 2024-09-20T22:12:23.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0032386580000000002 2024-09-20T22:12:23.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:23.869 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501669+0000", 2024-09-20T22:12:23.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2500000000000006e-06 2024-09-20T22:12:23.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.869 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501709+0000", 2024-09-20T22:12:23.869 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9304999999999997e-05 2024-09-20T22:12:23.869 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.869 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.869 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:23.869 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501777+0000", 2024-09-20T22:12:23.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8285000000000001e-05 2024-09-20T22:12:23.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.870 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.870 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501821+0000", 2024-09-20T22:12:23.870 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4635000000000002e-05 2024-09-20T22:12:23.870 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.870 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.870 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.870 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.870 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.870 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214327 8.8 8:149a99e2:::benchmark_data_smithi060_111051_object94328:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:12:23.870 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.502392+0000", 2024-09-20T22:12:23.870 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.237315733000003, 2024-09-20T22:12:23.871 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.001539808, 2024-09-20T22:12:23.871 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.871 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.871 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:23.871 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.871 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:23.871 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:23.871 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214327 2024-09-20T22:12:23.871 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.871 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.871 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.871 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502392+0000", 2024-09-20T22:12:23.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.872 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502392+0000", 2024-09-20T22:12:23.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.872 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.872 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502393+0000", 2024-09-20T22:12:23.872 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2440000000000001e-06 2024-09-20T22:12:23.872 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.872 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.873 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502398+0000", 2024-09-20T22:12:23.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1749999999999998e-06 2024-09-20T22:12:23.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.873 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502406+0000", 2024-09-20T22:12:23.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4999999999999999e-06 2024-09-20T22:12:23.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.873 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.873 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.873 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502411+0000", 2024-09-20T22:12:23.873 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7119999999999999e-06 2024-09-20T22:12:23.873 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502465+0000", 2024-09-20T22:12:23.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4206000000000002e-05 2024-09-20T22:12:23.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502470+0000", 2024-09-20T22:12:23.874 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8069999999999999e-06 2024-09-20T22:12:23.874 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.874 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.874 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.874 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.500319+0000", 2024-09-20T22:12:23.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99784936499999999 2024-09-20T22:12:23.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:23.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.500435+0000", 2024-09-20T22:12:23.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115439 2024-09-20T22:12:23.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.875 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:12:23.875 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.500670+0000", 2024-09-20T22:12:23.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023555700000000001 2024-09-20T22:12:23.875 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503764+0000", 2024-09-20T22:12:23.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0030938839999999999 2024-09-20T22:12:23.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503827+0000", 2024-09-20T22:12:23.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2669999999999995e-05 2024-09-20T22:12:23.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:23.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503839+0000", 2024-09-20T22:12:23.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2301e-05 2024-09-20T22:12:23.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:23.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503887+0000", 2024-09-20T22:12:23.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8319999999999998e-05 2024-09-20T22:12:23.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503932+0000", 2024-09-20T22:12:23.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4628e-05 2024-09-20T22:12:23.877 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.877 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.877 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.878 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214339 8.8 8:166843c0:::benchmark_data_smithi060_111051_object94340:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:12:23.878 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.507919+0000", 2024-09-20T22:12:23.878 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.231788636999994, 2024-09-20T22:12:23.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99621294400000004, 2024-09-20T22:12:23.878 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.878 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.878 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:23.878 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.878 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:23.878 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:23.878 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214339 2024-09-20T22:12:23.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.879 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507919+0000", 2024-09-20T22:12:23.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507919+0000", 2024-09-20T22:12:23.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507921+0000", 2024-09-20T22:12:23.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.316e-06 2024-09-20T22:12:23.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507926+0000", 2024-09-20T22:12:23.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0350000000000001e-06 2024-09-20T22:12:23.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507937+0000", 2024-09-20T22:12:23.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1379999999999999e-05 2024-09-20T22:12:23.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507945+0000", 2024-09-20T22:12:23.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5780000000000001e-06 2024-09-20T22:12:23.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.508028+0000", 2024-09-20T22:12:23.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3641999999999996e-05 2024-09-20T22:12:23.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.508036+0000", 2024-09-20T22:12:23.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1780000000000003e-06 2024-09-20T22:12:23.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501861+0000", 2024-09-20T22:12:23.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99382438200000001 2024-09-20T22:12:23.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:23.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501935+0000", 2024-09-20T22:12:23.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4151000000000002e-05 2024-09-20T22:12:23.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:12:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.502143+0000", 2024-09-20T22:12:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020823700000000001 2024-09-20T22:12:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503984+0000", 2024-09-20T22:12:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001840602 2024-09-20T22:12:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504043+0000", 2024-09-20T22:12:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9061000000000003e-05 2024-09-20T22:12:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504055+0000", 2024-09-20T22:12:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1826e-05 2024-09-20T22:12:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504099+0000", 2024-09-20T22:12:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4801e-05 2024-09-20T22:12:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504132+0000", 2024-09-20T22:12:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2755000000000001e-05 2024-09-20T22:12:23.884 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214363 8.8 8:14aba4d5:::benchmark_data_smithi060_111051_object94364:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:12:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.530169+0000", 2024-09-20T22:12:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.209538702000003, 2024-09-20T22:12:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98026972499999998, 2024-09-20T22:12:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:23.885 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214363 2024-09-20T22:12:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530169+0000", 2024-09-20T22:12:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530169+0000", 2024-09-20T22:12:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530171+0000", 2024-09-20T22:12:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4300000000000001e-06 2024-09-20T22:12:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530177+0000", 2024-09-20T22:12:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8499999999999999e-06 2024-09-20T22:12:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530188+0000", 2024-09-20T22:12:23.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1669e-05 2024-09-20T22:12:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530196+0000", 2024-09-20T22:12:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0760000000000003e-06 2024-09-20T22:12:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530270+0000", 2024-09-20T22:12:23.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3709999999999997e-05 2024-09-20T22:12:23.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530278+0000", 2024-09-20T22:12:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5320000000000006e-06 2024-09-20T22:12:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504165+0000", 2024-09-20T22:12:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97388663600000003 2024-09-20T22:12:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504263+0000", 2024-09-20T22:12:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7412999999999997e-05 2024-09-20T22:12:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:12:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504540+0000", 2024-09-20T22:12:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00027761399999999997 2024-09-20T22:12:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510219+0000", 2024-09-20T22:12:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005678476 2024-09-20T22:12:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510260+0000", 2024-09-20T22:12:23.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1452000000000002e-05 2024-09-20T22:12:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510339+0000", 2024-09-20T22:12:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8669000000000002e-05 2024-09-20T22:12:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510393+0000", 2024-09-20T22:12:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4160999999999999e-05 2024-09-20T22:12:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510439+0000", 2024-09-20T22:12:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6037e-05 2024-09-20T22:12:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214367 8.8 8:15e42233:::benchmark_data_smithi060_111051_object94368:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:12:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.530758+0000", 2024-09-20T22:12:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 86.208949517999997, 2024-09-20T22:12:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98471673000000004, 2024-09-20T22:12:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:23.893 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214367 2024-09-20T22:12:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530758+0000", 2024-09-20T22:12:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530758+0000", 2024-09-20T22:12:23.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530759+0000", 2024-09-20T22:12:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-07 2024-09-20T22:12:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530764+0000", 2024-09-20T22:12:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5390000000000001e-06 2024-09-20T22:12:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530773+0000", 2024-09-20T22:12:23.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.986e-06 2024-09-20T22:12:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530780+0000", 2024-09-20T22:12:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8689999999999998e-06 2024-09-20T22:12:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530827+0000", 2024-09-20T22:12:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7314000000000001e-05 2024-09-20T22:12:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:23.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530832+0000", 2024-09-20T22:12:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.758e-06 2024-09-20T22:12:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.505913+0000", 2024-09-20T22:12:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97508115200000001 2024-09-20T22:12:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.506026+0000", 2024-09-20T22:12:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011298000000000001 2024-09-20T22:12:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:12:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.506308+0000", 2024-09-20T22:12:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000282477 2024-09-20T22:12:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510274+0000", 2024-09-20T22:12:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0039657160000000002 2024-09-20T22:12:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510478+0000", 2024-09-20T22:12:23.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204002 2024-09-20T22:12:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.515367+0000", 2024-09-20T22:12:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0048889099999999998 2024-09-20T22:12:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.515428+0000", 2024-09-20T22:12:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0826999999999999e-05 2024-09-20T22:12:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.515475+0000", 2024-09-20T22:12:23.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7388000000000001e-05 2024-09-20T22:12:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:23.900 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:23.900 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:12:23.900 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 36 16 65209 65193 113.146 81.375 0.023253 0.00883271 2024-09-20T22:12:23.968 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:24.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:24.007 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:24.007 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:24.108 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:24.145 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:24.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:24.146 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:24.247 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:24.282 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:24.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:24.283 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:24.384 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:24.424 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:24.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:24.425 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:24.526 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:24.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:24.561 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:24.561 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:24.662 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:24.700 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:24.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:24.701 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:24.802 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:24.840 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:24.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:24.841 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:24.894 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:24 smithi060 ceph-mon[29694]: pgmap v512: 158 pgs: 6 peering, 152 active+clean; 3.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 726 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-20T22:12:24.894 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:24 smithi060 ceph-mon[40290]: pgmap v512: 158 pgs: 6 peering, 152 active+clean; 3.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 726 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-20T22:12:24.899 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 37 16 66660 66644 112.539 90.6875 0.0143534 0.00888018 2024-09-20T22:12:24.942 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:24.978 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:24.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:24.979 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:24 smithi067 ceph-mon[33511]: pgmap v512: 158 pgs: 6 peering, 152 active+clean; 3.8 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 726 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-20T22:12:25.080 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:25.115 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:25.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:25.116 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:25.177 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:12:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:12:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:12:25.217 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:25.258 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:25.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:25.259 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:25.360 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:25.393 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:25.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:25.394 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:25.495 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:25.529 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:25.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:25.530 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:25.631 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:25.708 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:25.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:25.709 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:25.810 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:25.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:25.845 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:25.899 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 38 16 68097 68081 111.941 89.8125 0.00928761 0.00892777 2024-09-20T22:12:25.946 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:25.989 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:25.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:25.990 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:26.091 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:26.097 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T22:12:26.127 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:26.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:26.128 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:26.181 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:26.230 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:26.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:26.276 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:26.276 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:26.377 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:26.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:26.417 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:26.417 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:26.518 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:26.558 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:26.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:26.559 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:26.575 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:12:26.660 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:26.708 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:26.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:26.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:26.810 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:26.847 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:26.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:26.848 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:26.899 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 39 15 69418 69403 111.189 82.625 0.00442566 0.00898791 2024-09-20T22:12:26.948 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:12:26.949 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:27.000 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:27.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:27.001 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:27.102 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:27.152 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:27.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:27.154 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:27.244 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:12:27.245 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:12:27.245 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.245 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.15594.0:69776 9.c e311/286 9:39399269:::benchmark_data_smithi060_140132_object69775:head v 311'4376, mlcod=311'4372)", 2024-09-20T22:12:27.245 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:12:27.232132+0000", 2024-09-20T22:12:27.245 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0099670020000000008, 2024-09-20T22:12:27.245 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0099963090000000001, 2024-09-20T22:12:27.245 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:27.245 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:27.245 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:12:27.245 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:27.245 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.245 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:27.246 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.232132+0000", 2024-09-20T22:12:27.246 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:27.246 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.246 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.246 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:27.246 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.232132+0000", 2024-09-20T22:12:27.246 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:27.246 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.246 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.246 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:27.246 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.232132+0000", 2024-09-20T22:12:27.246 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1100000000000001e-07 2024-09-20T22:12:27.246 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.247 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.247 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:27.247 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.232159+0000", 2024-09-20T22:12:27.247 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.6516000000000002e-05 2024-09-20T22:12:27.247 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.247 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.247 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:27.247 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.232166+0000", 2024-09-20T22:12:27.247 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9940000000000003e-06 2024-09-20T22:12:27.247 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.247 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.247 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:27.247 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.232171+0000", 2024-09-20T22:12:27.247 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.028e-06 2024-09-20T22:12:27.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.248 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.248 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:27.248 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.235961+0000", 2024-09-20T22:12:27.248 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0037903009999999998 2024-09-20T22:12:27.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.248 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.248 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:27.248 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.235980+0000", 2024-09-20T22:12:27.248 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8729999999999999e-05 2024-09-20T22:12:27.248 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:27.248 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:27.248 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:27.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.249 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.249 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.15594.0:69781 9.4 e311/286 9:202de1e4:::benchmark_data_smithi060_140132_object69780:head v 311'4381, mlcod=311'4377)", 2024-09-20T22:12:27.249 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:12:27.236864+0000", 2024-09-20T22:12:27.249 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0052349170000000004, 2024-09-20T22:12:27.249 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.005327544, 2024-09-20T22:12:27.249 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:27.249 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:27.249 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:12:27.249 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:27.249 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.249 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:27.249 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.236864+0000", 2024-09-20T22:12:27.249 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:27.249 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.250 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.250 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:27.250 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.236864+0000", 2024-09-20T22:12:27.250 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:27.250 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.250 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.250 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:27.250 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.236865+0000", 2024-09-20T22:12:27.250 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5600000000000004e-07 2024-09-20T22:12:27.250 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.250 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.250 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:27.250 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.236919+0000", 2024-09-20T22:12:27.251 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4344e-05 2024-09-20T22:12:27.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.251 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.251 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:27.251 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.236935+0000", 2024-09-20T22:12:27.251 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5469e-05 2024-09-20T22:12:27.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.251 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.251 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:27.251 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.236942+0000", 2024-09-20T22:12:27.251 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7789999999999998e-06 2024-09-20T22:12:27.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.251 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.251 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:27.251 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.237543+0000", 2024-09-20T22:12:27.252 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00060092499999999998 2024-09-20T22:12:27.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.252 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.252 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:27.252 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.237559+0000", 2024-09-20T22:12:27.252 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6283999999999999e-05 2024-09-20T22:12:27.252 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:27.252 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:27.252 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:27.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.252 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.252 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.15594.0:69784 9.c e311/286 9:36d833ef:::benchmark_data_smithi060_140132_object69783:head v 311'4377, mlcod=311'4373)", 2024-09-20T22:12:27.252 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:12:27.237326+0000", 2024-09-20T22:12:27.252 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0047728659999999997, 2024-09-20T22:12:27.253 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0049216920000000001, 2024-09-20T22:12:27.253 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:27.253 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:27.253 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:12:27.253 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:27.253 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.253 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:27.253 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.237326+0000", 2024-09-20T22:12:27.253 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:27.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.253 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.253 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:27.253 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.237326+0000", 2024-09-20T22:12:27.253 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:27.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.254 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:27.254 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.237327+0000", 2024-09-20T22:12:27.254 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8700000000000005e-07 2024-09-20T22:12:27.254 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.254 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:27.254 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.237358+0000", 2024-09-20T22:12:27.254 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1432e-05 2024-09-20T22:12:27.254 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.254 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:27.254 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.237374+0000", 2024-09-20T22:12:27.254 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5392000000000001e-05 2024-09-20T22:12:27.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.255 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.255 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:27.255 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.237380+0000", 2024-09-20T22:12:27.255 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8240000000000002e-06 2024-09-20T22:12:27.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.255 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.255 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:27.255 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.238176+0000", 2024-09-20T22:12:27.255 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00079562800000000005 2024-09-20T22:12:27.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.255 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.255 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:27.255 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.238193+0000", 2024-09-20T22:12:27.255 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6487e-05 2024-09-20T22:12:27.256 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:27.256 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:27.256 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:27.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.256 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.15594.0:69791 9.c e311/286 9:388bcdfc:::benchmark_data_smithi060_140132_object69790:head v 311'4378, mlcod=311'4373)", 2024-09-20T22:12:27.256 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:12:27.240796+0000", 2024-09-20T22:12:27.256 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0013032009999999999, 2024-09-20T22:12:27.256 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0015058039999999999, 2024-09-20T22:12:27.256 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:27.256 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:27.256 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:12:27.256 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:27.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.256 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:27.257 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.240796+0000", 2024-09-20T22:12:27.257 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:27.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.257 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.257 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:27.257 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.240796+0000", 2024-09-20T22:12:27.257 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:27.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.257 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.257 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:27.257 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.240797+0000", 2024-09-20T22:12:27.257 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6499999999999998e-07 2024-09-20T22:12:27.257 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.257 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.257 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:27.258 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.240859+0000", 2024-09-20T22:12:27.258 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2048e-05 2024-09-20T22:12:27.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.258 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.258 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:27.258 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.240873+0000", 2024-09-20T22:12:27.258 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4426e-05 2024-09-20T22:12:27.258 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.258 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.258 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:27.258 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.240880+0000", 2024-09-20T22:12:27.258 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:27.259 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3749999999999997e-06 2024-09-20T22:12:27.259 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.260 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.260 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:27.260 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.240936+0000", 2024-09-20T22:12:27.260 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5458999999999999e-05 2024-09-20T22:12:27.260 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:27.260 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:27.260 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:27.260 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:27.240954+0000", 2024-09-20T22:12:27.260 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8366e-05 2024-09-20T22:12:27.260 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:27.260 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:27.260 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:27.260 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:27.261 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-20T22:12:27.261 INFO:teuthology.orchestra.run.smithi067.stdout: "num_ops": 4 2024-09-20T22:12:27.261 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:12:27.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:27 smithi060 ceph-mon[29694]: pgmap v513: 158 pgs: 1 peering, 157 active+clean; 4.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.4 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-20T22:12:27.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:27 smithi060 ceph-mon[29694]: osdmap e311: 8 total, 8 up, 8 in 2024-09-20T22:12:27.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:27 smithi060 ceph-mon[40290]: pgmap v513: 158 pgs: 1 peering, 157 active+clean; 4.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.4 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-20T22:12:27.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:27 smithi060 ceph-mon[40290]: osdmap e311: 8 total, 8 up, 8 in 2024-09-20T22:12:27.306 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:27.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:27.307 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:27.408 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:27.415 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:27.445 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:27.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:27.447 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:27.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:27 smithi067 ceph-mon[33511]: pgmap v513: 158 pgs: 1 peering, 157 active+clean; 4.0 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1.4 KiB/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-20T22:12:27.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:27 smithi067 ceph-mon[33511]: osdmap e311: 8 total, 8 up, 8 in 2024-09-20T22:12:27.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:27.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:27.584 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:27.584 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:27.685 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:27.723 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:27.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:27.724 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:27.825 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:27.866 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:27.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:27.867 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:27.899 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:12:27.899316+0000 min lat: 0.0012684 max lat: 1.0215 avg lat: 0.00905508 2024-09-20T22:12:27.899 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:12:27.899 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 40 16 70677 70661 110.375 78.625 0.0121308 0.00905508 2024-09-20T22:12:27.968 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:27.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:28.000 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:28.000 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:28.100 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:28.105 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:12:28.106 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:12:28.142 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:12:28.142 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:12:28.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:28.150 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:28.150 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:28.153 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T22:12:28.177 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:12:28.177 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:12:28.217 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:12:28.226 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:12:28.226 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:12:28.251 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:28.265 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:12:28.266 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:12:28.297 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:28.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:28.299 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:28.310 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:12:28.310 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:12:28.340 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:12:28.340 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:12:28.361 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:12:28.362 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:12:28.399 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:28.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:28.441 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:28.441 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:28.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:28 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3486298261' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:12:28.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:28 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1221652180' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:28.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:28 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3486298261' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:12:28.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:28 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1221652180' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:28.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:28 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3486298261' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:12:28.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:28 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1221652180' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:28.542 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:28.580 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:28.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:28.581 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:28.599 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:12:28.682 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:28.721 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:28.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:28.722 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:28.823 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:28.865 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:28.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:28.866 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:28.900 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 41 16 71945 71929 109.616 79.25 0.0072576 0.00911671 2024-09-20T22:12:28.966 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:28.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:28.997 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:28.998 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:29.099 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:29.137 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:29.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:29.138 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:29.239 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:29 smithi067 ceph-mon[33511]: pgmap v515: 158 pgs: 158 active+clean; 4.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-20T22:12:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:29 smithi067 ceph-mon[33511]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-20T22:12:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:29 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:12:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:29 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3486298261' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:12:29.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:29 smithi067 ceph-mon[33511]: osdmap e312: 8 total, 8 up, 8 in 2024-09-20T22:12:29.279 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:29.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:29.280 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:29.381 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:29.423 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:29.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:29.424 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:29.463 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:12:29.463 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:12:29.524 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:29 smithi060 ceph-mon[29694]: pgmap v515: 158 pgs: 158 active+clean; 4.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-20T22:12:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:29 smithi060 ceph-mon[29694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-20T22:12:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:29 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:12:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:29 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3486298261' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:12:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:29 smithi060 ceph-mon[29694]: osdmap e312: 8 total, 8 up, 8 in 2024-09-20T22:12:29.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:29 smithi060 ceph-mon[40290]: pgmap v515: 158 pgs: 158 active+clean; 4.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-20T22:12:29.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:29 smithi060 ceph-mon[40290]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-20T22:12:29.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:29 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:12:29.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:29 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3486298261' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:12:29.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:29 smithi060 ceph-mon[40290]: osdmap e312: 8 total, 8 up, 8 in 2024-09-20T22:12:29.561 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:29.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:29.562 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:29.663 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:29.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:29.700 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:29.701 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:29.802 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:29.836 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:29.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:29.837 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:29.900 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 42 15 73188 73173 108.857 77.75 0.0119788 0.00918132 2024-09-20T22:12:29.937 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:29.976 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:29.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:29.978 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:30.078 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:30.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:30.209 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:30.209 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:30.310 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:30.341 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T22:12:30.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:30.368 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:30.369 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:30.469 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:30.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:30 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:12:30.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:30 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:12:30.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:30 smithi067 ceph-mon[33511]: osdmap e313: 8 total, 8 up, 8 in 2024-09-20T22:12:30.516 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:30.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:30.517 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:30.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:30 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:12:30.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:30 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:12:30.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:30 smithi060 ceph-mon[29694]: osdmap e313: 8 total, 8 up, 8 in 2024-09-20T22:12:30.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:30 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:12:30.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:30 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:12:30.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:30 smithi060 ceph-mon[40290]: osdmap e313: 8 total, 8 up, 8 in 2024-09-20T22:12:30.618 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:30.633 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:12:30.633 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:12:30.633 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-09-20T22:12:30.633 INFO:tasks.thrashosds.thrasher:Setting osd 3 primary_affinity to 0.460940 2024-09-20T22:12:30.634 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 3 0.46093988163900423 2024-09-20T22:12:30.661 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:30.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:30.662 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:30.762 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:30.770 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:12:30.807 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:30.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:30.808 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:30.900 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 43 16 74491 74475 108.218 81.375 0.0120914 0.00923604 2024-09-20T22:12:30.909 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:30.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:30.963 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:30.963 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:31.064 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:31.111 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:31.112 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:31.112 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:31.213 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:31.246 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:31.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:31.247 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:31.348 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:31.400 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:31.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:31.401 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:31.424 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:31.434 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:12:31.434 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:12:31.501 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:31.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:31 smithi067 ceph-mon[33511]: pgmap v518: 158 pgs: 158 active+clean; 4.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-20T22:12:31.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:31 smithi060 ceph-mon[29694]: pgmap v518: 158 pgs: 158 active+clean; 4.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-20T22:12:31.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:31 smithi060 ceph-mon[40290]: pgmap v518: 158 pgs: 158 active+clean; 4.2 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 853 B/s rd, 64 MiB/s wr, 1.02k op/s 2024-09-20T22:12:31.545 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:31.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:31.546 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:31.647 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:31.684 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:31.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:31.685 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:31.786 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:31.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:31.831 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:31.900 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 44 16 75776 75760 107.584 80.3125 0.00494194 0.00928937 2024-09-20T22:12:31.932 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:31.967 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:31.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:31.968 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:32.069 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:32.110 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:32.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:32.111 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:32.212 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:32.247 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:32.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:32.248 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:32.320 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T22:12:32.349 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:32.367 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:12:32.394 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:32.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:32.396 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:32.497 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:32.519 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.3 primary-affinity to 0.46094 (8302082) 2024-09-20T22:12:32.535 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:32.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:32.536 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:32.637 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:32.658 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:32.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:32.659 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:32.674 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:12:32.760 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:32.763 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:12:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:12:32.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:32 smithi067 ceph-mon[33511]: pgmap v519: 158 pgs: 158 active+clean; 4.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 93 MiB/s wr, 1.49k op/s 2024-09-20T22:12:32.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:32 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:12:32.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:32 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2795321952' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.46093988163900423}]: dispatch 2024-09-20T22:12:32.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:32 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.46093988163900423}]: dispatch 2024-09-20T22:12:32.782 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:32.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:32.783 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:32 smithi060 ceph-mon[29694]: pgmap v519: 158 pgs: 158 active+clean; 4.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 93 MiB/s wr, 1.49k op/s 2024-09-20T22:12:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:32 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:12:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:32 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2795321952' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.46093988163900423}]: dispatch 2024-09-20T22:12:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:32 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.46093988163900423}]: dispatch 2024-09-20T22:12:32.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:32 smithi060 ceph-mon[40290]: pgmap v519: 158 pgs: 158 active+clean; 4.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 93 MiB/s wr, 1.49k op/s 2024-09-20T22:12:32.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:32 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:12:32.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:32 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2795321952' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.46093988163900423}]: dispatch 2024-09-20T22:12:32.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:32 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.46093988163900423}]: dispatch 2024-09-20T22:12:32.884 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:32.900 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 45 16 76587 76571 106.32 50.6875 0.00326515 0.00933371 2024-09-20T22:12:32.906 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:32.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:32.907 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:33.008 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:33.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:33.031 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:33.032 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:33.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:33.156 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:33.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:33.157 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:33.181 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:12:33.181 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:12:33.258 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:33.272 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:12:33.281 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:33.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:33.282 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:33.382 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:33.405 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:33.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:33.406 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:33.506 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:33.533 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:33.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:33.534 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:33.635 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:33.669 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:33.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:33.670 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:33.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:33 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.46093988163900423}]': finished 2024-09-20T22:12:33.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:33 smithi067 ceph-mon[33511]: osdmap e314: 8 total, 8 up, 8 in 2024-09-20T22:12:33.771 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:33.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:33 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.46093988163900423}]': finished 2024-09-20T22:12:33.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:33 smithi060 ceph-mon[29694]: osdmap e314: 8 total, 8 up, 8 in 2024-09-20T22:12:33.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:33 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 0.46093988163900423}]': finished 2024-09-20T22:12:33.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:33 smithi060 ceph-mon[40290]: osdmap e314: 8 total, 8 up, 8 in 2024-09-20T22:12:33.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:33.809 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:33.809 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:33.900 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 46 16 76971 76955 104.53 24 0.00294539 0.00955945 2024-09-20T22:12:33.910 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:33.933 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:33.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:33.934 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:34.035 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:34.077 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:34.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:34.078 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:34.178 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:34.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:34.220 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:34.220 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:34.321 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:34.358 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:34.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:34.359 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:34.460 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:34.498 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:34.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:34.499 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:34.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:34.617 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T22:12:34.650 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:34.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:34.651 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:34.669 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:34.678 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:34.751 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:34.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:34 smithi060 ceph-mon[29694]: osdmap e315: 8 total, 8 up, 8 in 2024-09-20T22:12:34.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:34 smithi060 ceph-mon[29694]: pgmap v522: 158 pgs: 3 peering, 155 active+clean; 4.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 862 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-20T22:12:34.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:34 smithi060 ceph-mon[40290]: osdmap e315: 8 total, 8 up, 8 in 2024-09-20T22:12:34.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:34 smithi060 ceph-mon[40290]: pgmap v522: 158 pgs: 3 peering, 155 active+clean; 4.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 862 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-20T22:12:34.798 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:34.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:34.799 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:34.900 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:34.901 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 47 16 78160 78144 103.887 74.3125 0.00784043 0.00962007 2024-09-20T22:12:34.951 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:34.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:34.952 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:35.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:34 smithi067 ceph-mon[33511]: osdmap e315: 8 total, 8 up, 8 in 2024-09-20T22:12:35.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:34 smithi067 ceph-mon[33511]: pgmap v522: 158 pgs: 3 peering, 155 active+clean; 4.6 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 862 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-20T22:12:35.053 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:35.057 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:12:35.077 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:12:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:12:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:12:35.101 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:35.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:35.102 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:35.203 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:35.247 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:35.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:35.248 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:35.349 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:35.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:35.390 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:35.390 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:35.491 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:35.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:35.536 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:35.536 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:35.637 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:35.664 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:35 smithi060 ceph-mon[40290]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-20T22:12:35.664 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:35 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:12:35.664 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:35 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:12:35.665 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:35 smithi060 ceph-mon[29694]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-20T22:12:35.665 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:35 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:12:35.665 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:35 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:12:35.681 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:35.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:35.682 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:35.700 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:12:35.701 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:12:35.725 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:12:35.725 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:12:35.743 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:12:35.744 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:12:35.778 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:12:35.779 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:12:35.783 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:35.817 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:35.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:35.818 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:35.820 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:12:35.820 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:12:35.858 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:12:35.858 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:12:35.897 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:12:35.898 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:12:35.900 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 48 16 79361 79345 103.287 75.0625 0.0113618 0.00967647 2024-09-20T22:12:35.918 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:35.938 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:12:35.938 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:12:35.954 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:35.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:35.955 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:35.988 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:12:35.989 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:12:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:35 smithi067 ceph-mon[33511]: Health check failed: Reduced data availability: 2 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-20T22:12:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:35 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:12:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:35 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:12:36.056 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:36.092 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:36.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:36.093 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:36.194 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:36.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:36.234 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:36.235 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:36.335 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:36.375 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:36.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:36.376 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:36.476 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:36.513 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:36.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:36.514 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:36.546 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:12:36.615 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:36.643 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:36.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:36.644 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:36.745 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:36.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:36 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1868904514' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:36.780 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:36 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2531776578' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:12:36.780 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:36 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:12:36.780 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:36 smithi060 ceph-mon[29694]: pgmap v523: 158 pgs: 3 peering, 155 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.4 KiB/s rd, 82 MiB/s wr, 1.31k op/s; 9.2 KiB/s, 0 objects/s recovering 2024-09-20T22:12:36.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:36 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1868904514' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:36.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:36 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2531776578' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:12:36.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:36 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:12:36.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:36 smithi060 ceph-mon[40290]: pgmap v523: 158 pgs: 3 peering, 155 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.4 KiB/s rd, 82 MiB/s wr, 1.31k op/s; 9.2 KiB/s, 0 objects/s recovering 2024-09-20T22:12:36.785 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:36.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:36.786 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:36.887 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:36.900 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 49 16 80586 80570 102.741 76.5625 0.0115192 0.00972788 2024-09-20T22:12:36.908 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:36 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1868904514' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:36.908 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:36 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2531776578' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:12:36.908 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:36 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:12:36.909 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:36 smithi067 ceph-mon[33511]: pgmap v523: 158 pgs: 3 peering, 155 active+clean; 4.7 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 1.4 KiB/s rd, 82 MiB/s wr, 1.31k op/s; 9.2 KiB/s, 0 objects/s recovering 2024-09-20T22:12:36.948 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:36.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:36.949 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:36.953 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T22:12:37.050 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:37.085 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:37.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:37.087 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:37.187 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:37.242 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:37.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:37.243 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:37.344 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:37.408 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:37.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:37.409 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:37.440 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:12:37.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:37.548 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:37.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:37.549 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:37.650 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:37.696 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:37.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:37.697 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:37.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:37 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:12:37.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:37 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:12:37.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:37 smithi060 ceph-mon[29694]: osdmap e316: 8 total, 8 up, 8 in 2024-09-20T22:12:37.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:37 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:12:37.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:37 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:12:37.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:37 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:12:37.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:37 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:12:37.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:37 smithi060 ceph-mon[40290]: osdmap e316: 8 total, 8 up, 8 in 2024-09-20T22:12:37.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:37 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:12:37.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:37 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:12:37.795 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:37 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:12:37.795 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:37 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:12:37.795 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:37 smithi067 ceph-mon[33511]: osdmap e316: 8 total, 8 up, 8 in 2024-09-20T22:12:37.795 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:37 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:12:37.796 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:37 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:12:37.798 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:37.835 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:37.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:37.837 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:37.900 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 50 16 81843 81827 102.258 78.5625 0.0142929 0.00977443 2024-09-20T22:12:37.939 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:37.972 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:37.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:37.973 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:38.074 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:38.105 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:12:38.105 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:12:38.105 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:12:38.105 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:12:38.105 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.105 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169974 7.10 7:0a082cf8:::benchmark_data_smithi060_78800_object36941:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:12:38.105 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:12:38.105 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 399.474212545, 2024-09-20T22:12:38.106 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87805208899999998, 2024-09-20T22:12:38.106 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.106 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.106 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:38.106 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:12:38.106 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:12:38.106 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:12:38.106 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169974 2024-09-20T22:12:38.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.106 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.106 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.106 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.106 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:12:38.107 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.107 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.107 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:12:38.107 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.107 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.107 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624616+0000", 2024-09-20T22:12:38.107 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1739999999999999e-06 2024-09-20T22:12:38.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.107 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.108 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624626+0000", 2024-09-20T22:12:38.108 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7909999999999999e-06 2024-09-20T22:12:38.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.108 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.108 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624654+0000", 2024-09-20T22:12:38.108 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.8524999999999999e-05 2024-09-20T22:12:38.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.108 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.108 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624665+0000", 2024-09-20T22:12:38.108 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0985000000000001e-05 2024-09-20T22:12:38.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.109 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.109 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.705237+0000", 2024-09-20T22:12:38.109 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.080571485999999998 2024-09-20T22:12:38.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.109 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.109 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.705245+0000", 2024-09-20T22:12:38.109 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8460000000000008e-06 2024-09-20T22:12:38.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.109 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.109 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498359+0000", 2024-09-20T22:12:38.109 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79311461999999999 2024-09-20T22:12:38.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.110 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:38.110 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498498+0000", 2024-09-20T22:12:38.110 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013918599999999999 2024-09-20T22:12:38.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.110 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:12:38.110 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498751+0000", 2024-09-20T22:12:38.110 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025235999999999998 2024-09-20T22:12:38.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.110 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.110 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.500697+0000", 2024-09-20T22:12:38.110 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0019460969999999999 2024-09-20T22:12:38.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.111 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.111 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:12:38.111 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502528+0000", 2024-09-20T22:12:38.111 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00183161 2024-09-20T22:12:38.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.111 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.111 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:12:38.111 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502597+0000", 2024-09-20T22:12:38.111 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8761000000000001e-05 2024-09-20T22:12:38.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.111 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.111 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.112 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502666+0000", 2024-09-20T22:12:38.112 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8647999999999997e-05 2024-09-20T22:12:38.112 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.112 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.112 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.112 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227618 7.0 7:0598f762:::benchmark_data_smithi060_78800_object94585:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-20T22:12:38.112 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:53.805369+0000", 2024-09-20T22:12:38.112 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 344.29345752, 2024-09-20T22:12:38.112 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2615457110000001, 2024-09-20T22:12:38.112 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.112 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.112 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:38.113 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:12:38.113 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:12:38.113 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:12:38.113 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227618 2024-09-20T22:12:38.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.113 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.113 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.113 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.113 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805369+0000", 2024-09-20T22:12:38.113 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.113 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.113 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.113 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805369+0000", 2024-09-20T22:12:38.114 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.114 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.114 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805371+0000", 2024-09-20T22:12:38.114 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.827e-06 2024-09-20T22:12:38.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.114 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.114 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805377+0000", 2024-09-20T22:12:38.114 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.161e-06 2024-09-20T22:12:38.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.114 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.115 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805398+0000", 2024-09-20T22:12:38.115 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1208000000000001e-05 2024-09-20T22:12:38.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.115 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.115 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805406+0000", 2024-09-20T22:12:38.115 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3820000000000004e-06 2024-09-20T22:12:38.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.115 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.115 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.813177+0000", 2024-09-20T22:12:38.115 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0077708919999999997 2024-09-20T22:12:38.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.116 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.116 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.813194+0000", 2024-09-20T22:12:38.116 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7107000000000001e-05 2024-09-20T22:12:38.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.116 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.116 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.116 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.813273+0000", 2024-09-20T22:12:38.116 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.000078609 2024-09-20T22:12:38.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.116 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.116 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.116 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.813290+0000", 2024-09-20T22:12:38.116 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7382999999999999e-05 2024-09-20T22:12:38.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.117 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.117 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.960476+0000", 2024-09-20T22:12:38.117 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1471854260000001 2024-09-20T22:12:38.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.117 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:38.117 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.960616+0000", 2024-09-20T22:12:38.117 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014035100000000001 2024-09-20T22:12:38.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.117 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,3", 2024-09-20T22:12:38.117 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.961013+0000", 2024-09-20T22:12:38.117 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00039649999999999999 2024-09-20T22:12:38.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.118 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:12:38.118 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.963511+0000", 2024-09-20T22:12:38.118 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0024986919999999998 2024-09-20T22:12:38.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.118 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.118 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.963553+0000", 2024-09-20T22:12:38.118 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1783999999999998e-05 2024-09-20T22:12:38.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.118 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.118 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.066713+0000", 2024-09-20T22:12:38.118 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.103160088 2024-09-20T22:12:38.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.119 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:12:38.119 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.066794+0000", 2024-09-20T22:12:38.119 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0924e-05 2024-09-20T22:12:38.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.119 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.119 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.066915+0000", 2024-09-20T22:12:38.119 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000120377 2024-09-20T22:12:38.119 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.119 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.119 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.120 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.120 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227635 7.b 7:da8bd73f:::benchmark_data_smithi060_78800_object94602:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e146)", 2024-09-20T22:12:38.120 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:54.820720+0000", 2024-09-20T22:12:38.120 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 343.27810664399999, 2024-09-20T22:12:38.120 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1621617440000001, 2024-09-20T22:12:38.120 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.120 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.120 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:38.120 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:12:38.120 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:12:38.120 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:12:38.120 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227635 2024-09-20T22:12:38.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.121 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.121 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.121 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820720+0000", 2024-09-20T22:12:38.121 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.121 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.121 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820720+0000", 2024-09-20T22:12:38.121 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.121 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.121 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820721+0000", 2024-09-20T22:12:38.122 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.593e-06 2024-09-20T22:12:38.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.122 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.122 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820728+0000", 2024-09-20T22:12:38.122 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3799999999999999e-06 2024-09-20T22:12:38.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.122 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.122 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820740+0000", 2024-09-20T22:12:38.122 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.279e-05 2024-09-20T22:12:38.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.122 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.123 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820749+0000", 2024-09-20T22:12:38.123 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8000000000000004e-06 2024-09-20T22:12:38.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.123 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.123 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820942+0000", 2024-09-20T22:12:38.123 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000192263 2024-09-20T22:12:38.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.123 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.123 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820953+0000", 2024-09-20T22:12:38.123 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1953000000000001e-05 2024-09-20T22:12:38.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.123 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.974865+0000", 2024-09-20T22:12:38.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.15391142 2024-09-20T22:12:38.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.124 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:38.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.974939+0000", 2024-09-20T22:12:38.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3798000000000001e-05 2024-09-20T22:12:38.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.124 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,6", 2024-09-20T22:12:38.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.975019+0000", 2024-09-20T22:12:38.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0692999999999995e-05 2024-09-20T22:12:38.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.125 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.125 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.977853+0000", 2024-09-20T22:12:38.125 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028332269999999998 2024-09-20T22:12:38.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.125 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.125 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982635+0000", 2024-09-20T22:12:38.125 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0047819840000000004 2024-09-20T22:12:38.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.125 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:12:38.125 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982688+0000", 2024-09-20T22:12:38.126 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.384e-05 2024-09-20T22:12:38.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.126 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.126 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:12:38.126 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982747+0000", 2024-09-20T22:12:38.126 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9004e-05 2024-09-20T22:12:38.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.126 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.126 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.126 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982881+0000", 2024-09-20T22:12:38.126 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013399899999999999 2024-09-20T22:12:38.126 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.126 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.126 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.127 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227641 7.b 7:d29062b1:::benchmark_data_smithi060_78800_object94608:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e146)", 2024-09-20T22:12:38.127 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:12:38.127 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 343.27805553299999, 2024-09-20T22:12:38.127 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2739681169999999, 2024-09-20T22:12:38.127 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.127 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.127 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:38.127 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:12:38.127 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:12:38.127 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:12:38.127 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227641 2024-09-20T22:12:38.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.128 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.128 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.128 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:12:38.128 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.128 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.128 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:12:38.128 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.128 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.129 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:12:38.129 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3900000000000005e-07 2024-09-20T22:12:38.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.129 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.129 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.129 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820775+0000", 2024-09-20T22:12:38.129 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4589999999999998e-06 2024-09-20T22:12:38.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.129 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.129 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.129 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820783+0000", 2024-09-20T22:12:38.129 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8949999999999998e-06 2024-09-20T22:12:38.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.130 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.130 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820788+0000", 2024-09-20T22:12:38.130 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5759999999999996e-06 2024-09-20T22:12:38.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.130 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.130 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820979+0000", 2024-09-20T22:12:38.130 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019122 2024-09-20T22:12:38.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.130 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.130 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820988+0000", 2024-09-20T22:12:38.130 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1480000000000006e-06 2024-09-20T22:12:38.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.131 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.131 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.977915+0000", 2024-09-20T22:12:38.131 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.156927834 2024-09-20T22:12:38.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.131 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:38.131 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.978045+0000", 2024-09-20T22:12:38.131 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012967 2024-09-20T22:12:38.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.131 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,6", 2024-09-20T22:12:38.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.978183+0000", 2024-09-20T22:12:38.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013767900000000001 2024-09-20T22:12:38.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.132 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.132 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982676+0000", 2024-09-20T22:12:38.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0044928210000000001 2024-09-20T22:12:38.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.132 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.132 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:12:38.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982891+0000", 2024-09-20T22:12:38.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00021542299999999999 2024-09-20T22:12:38.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.132 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.132 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.133 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.094638+0000", 2024-09-20T22:12:38.133 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.111746554 2024-09-20T22:12:38.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.133 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.133 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:12:38.133 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.094677+0000", 2024-09-20T22:12:38.133 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9069000000000002e-05 2024-09-20T22:12:38.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.134 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.134 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.134 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.094739+0000", 2024-09-20T22:12:38.134 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2230000000000006e-05 2024-09-20T22:12:38.134 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.134 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.134 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.134 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.134 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105650 8.4 8:29da2845:::benchmark_data_smithi060_111051_object105649:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e214)", 2024-09-20T22:12:38.134 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.523636+0000", 2024-09-20T22:12:38.134 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 209.57519053799999, 2024-09-20T22:12:38.135 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.17888942399999999, 2024-09-20T22:12:38.135 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.135 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.135 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:12:38.135 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:12:38.135 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:12:38.135 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:38.135 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105650 2024-09-20T22:12:38.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.135 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.135 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.135 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523636+0000", 2024-09-20T22:12:38.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.136 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523636+0000", 2024-09-20T22:12:38.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.136 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523637+0000", 2024-09-20T22:12:38.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.686e-06 2024-09-20T22:12:38.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.136 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523679+0000", 2024-09-20T22:12:38.137 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1081999999999997e-05 2024-09-20T22:12:38.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.137 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.137 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.137 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523690+0000", 2024-09-20T22:12:38.137 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1545e-05 2024-09-20T22:12:38.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.137 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.137 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.137 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523696+0000", 2024-09-20T22:12:38.137 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3879999999999999e-06 2024-09-20T22:12:38.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.137 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.137 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.137 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523738+0000", 2024-09-20T22:12:38.138 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2460999999999999e-05 2024-09-20T22:12:38.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.138 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.138 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:38.138 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523826+0000", 2024-09-20T22:12:38.138 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8088000000000001e-05 2024-09-20T22:12:38.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.138 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.138 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,4", 2024-09-20T22:12:38.138 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.523912+0000", 2024-09-20T22:12:38.138 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6074000000000003e-05 2024-09-20T22:12:38.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.138 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.138 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.139 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.528900+0000", 2024-09-20T22:12:38.139 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0049876720000000003 2024-09-20T22:12:38.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.139 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.139 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:12:38.139 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.529517+0000", 2024-09-20T22:12:38.139 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00061690400000000004 2024-09-20T22:12:38.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.139 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.139 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.139 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.702506+0000", 2024-09-20T22:12:38.139 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.17298888600000001 2024-09-20T22:12:38.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.139 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.139 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.139 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.702525+0000", 2024-09-20T22:12:38.140 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9638000000000001e-05 2024-09-20T22:12:38.140 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.140 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.140 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.140 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.140 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105650 8.14 8:29da2845:::benchmark_data_smithi060_111051_object105649:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:12:38.140 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.545671+0000", 2024-09-20T22:12:38.140 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 209.55315530799999, 2024-09-20T22:12:38.140 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.038415869, 2024-09-20T22:12:38.140 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.140 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.140 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:12:38.140 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:12:38.141 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:12:38.141 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:38.141 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105650 2024-09-20T22:12:38.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.141 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.141 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.141 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545671+0000", 2024-09-20T22:12:38.141 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.141 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.141 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545671+0000", 2024-09-20T22:12:38.141 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.142 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.142 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545673+0000", 2024-09-20T22:12:38.142 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.17e-06 2024-09-20T22:12:38.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.142 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.142 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545723+0000", 2024-09-20T22:12:38.142 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9509000000000003e-05 2024-09-20T22:12:38.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.142 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.142 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545741+0000", 2024-09-20T22:12:38.143 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8634000000000002e-05 2024-09-20T22:12:38.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.143 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.143 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545751+0000", 2024-09-20T22:12:38.143 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4420000000000004e-06 2024-09-20T22:12:38.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.143 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.143 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.707696+0000", 2024-09-20T22:12:38.143 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.16194560099999999 2024-09-20T22:12:38.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.143 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.143 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.707706+0000", 2024-09-20T22:12:38.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6860000000000001e-06 2024-09-20T22:12:38.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584005+0000", 2024-09-20T22:12:38.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87629895400000002 2024-09-20T22:12:38.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584087+0000", 2024-09-20T22:12:38.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1872999999999994e-05 2024-09-20T22:12:38.144 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.144 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.145 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.145 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "MOSDPGPush(8.c 215/206 [PushOp(8:355e0c9a:::benchmark_data_smithi060_111051_object46967:head, version: 186'2869, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(8:355e0c9a:::benchmark_data_smithi060_111051_object46967:head@186'2869, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-20T22:12:38.145 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.555562+0000", 2024-09-20T22:12:38.145 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 209.54326394099999, 2024-09-20T22:12:38.145 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.15072329400000001, 2024-09-20T22:12:38.145 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.145 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.145 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:12:38.145 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.145 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:38.146 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:38.147 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:38.147 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.555562+0000", 2024-09-20T22:12:38.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.147 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.147 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.555562+0000", 2024-09-20T22:12:38.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.148 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.148 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.555564+0000", 2024-09-20T22:12:38.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4139999999999999e-06 2024-09-20T22:12:38.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.148 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.148 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.555625+0000", 2024-09-20T22:12:38.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1282e-05 2024-09-20T22:12:38.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.148 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.148 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572138+0000", 2024-09-20T22:12:38.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.016512466999999999 2024-09-20T22:12:38.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.149 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.149 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572188+0000", 2024-09-20T22:12:38.149 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0773000000000003e-05 2024-09-20T22:12:38.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.149 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.149 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.706058+0000", 2024-09-20T22:12:38.149 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.13387001800000001 2024-09-20T22:12:38.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.149 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:38.149 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.706071+0000", 2024-09-20T22:12:38.149 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2876e-05 2024-09-20T22:12:38.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.150 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.150 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.706286+0000", 2024-09-20T22:12:38.150 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00021446400000000001 2024-09-20T22:12:38.150 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.150 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.150 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.150 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105665 8.18 8:1b9c2a37:::benchmark_data_smithi060_111051_object105664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:12:38.150 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.556635+0000", 2024-09-20T22:12:38.150 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 209.54219127900001, 2024-09-20T22:12:38.150 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.051209888, 2024-09-20T22:12:38.150 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.151 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.151 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:38.151 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:12:38.151 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:12:38.151 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:38.151 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105665 2024-09-20T22:12:38.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.151 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.151 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.151 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556635+0000", 2024-09-20T22:12:38.151 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.152 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.152 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556635+0000", 2024-09-20T22:12:38.152 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.152 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.152 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556638+0000", 2024-09-20T22:12:38.152 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.6400000000000001e-06 2024-09-20T22:12:38.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.152 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.152 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556714+0000", 2024-09-20T22:12:38.152 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6594999999999994e-05 2024-09-20T22:12:38.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.153 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.153 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556736+0000", 2024-09-20T22:12:38.153 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1662e-05 2024-09-20T22:12:38.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.153 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.153 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.153 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556751+0000", 2024-09-20T22:12:38.153 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4941e-05 2024-09-20T22:12:38.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.153 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.153 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.153 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.750414+0000", 2024-09-20T22:12:38.153 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.193663171 2024-09-20T22:12:38.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.154 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.154 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.154 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.750424+0000", 2024-09-20T22:12:38.154 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0397999999999999e-05 2024-09-20T22:12:38.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.154 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.154 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.154 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.596184+0000", 2024-09-20T22:12:38.154 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.84575938799999995 2024-09-20T22:12:38.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.154 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.154 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:38.154 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.596303+0000", 2024-09-20T22:12:38.154 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000119503 2024-09-20T22:12:38.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.155 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.155 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:12:38.155 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.596588+0000", 2024-09-20T22:12:38.155 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00028446100000000001 2024-09-20T22:12:38.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.155 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.155 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.155 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.603603+0000", 2024-09-20T22:12:38.155 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0070155649999999996 2024-09-20T22:12:38.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.155 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.155 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:12:38.155 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607589+0000", 2024-09-20T22:12:38.155 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0039854110000000003 2024-09-20T22:12:38.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.156 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.156 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.156 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607743+0000", 2024-09-20T22:12:38.156 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015391499999999999 2024-09-20T22:12:38.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.156 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.156 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:12:38.156 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607802+0000", 2024-09-20T22:12:38.156 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9472000000000003e-05 2024-09-20T22:12:38.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.156 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.156 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.156 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607845+0000", 2024-09-20T22:12:38.156 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2765999999999997e-05 2024-09-20T22:12:38.157 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.157 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.157 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.157 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.157 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.15192.0:105664 8.c e215/206 8:37c43236:::benchmark_data_smithi060_111051_object105663:head v 215'6587, mlcod=214'6584)", 2024-09-20T22:12:38.157 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.572293+0000", 2024-09-20T22:12:38.157 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 209.526533156, 2024-09-20T22:12:38.157 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.15892877499999999, 2024-09-20T22:12:38.157 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.157 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.157 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:38.157 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.157 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.158 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.158 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572293+0000", 2024-09-20T22:12:38.158 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.158 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.158 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.158 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572293+0000", 2024-09-20T22:12:38.158 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.158 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.158 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.158 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572294+0000", 2024-09-20T22:12:38.158 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.65e-07 2024-09-20T22:12:38.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.158 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.159 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.159 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572300+0000", 2024-09-20T22:12:38.159 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7599999999999997e-06 2024-09-20T22:12:38.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.159 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.159 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572308+0000", 2024-09-20T22:12:38.159 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2490000000000004e-06 2024-09-20T22:12:38.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.159 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.159 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.572313+0000", 2024-09-20T22:12:38.159 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6590000000000004e-06 2024-09-20T22:12:38.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.160 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.160 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.703998+0000", 2024-09-20T22:12:38.160 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.13168523900000001 2024-09-20T22:12:38.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.160 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.160 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:38.160 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.704019+0000", 2024-09-20T22:12:38.160 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0267000000000001e-05 2024-09-20T22:12:38.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.160 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.160 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:12:38.160 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.731164+0000", 2024-09-20T22:12:38.160 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.027144784000000002 2024-09-20T22:12:38.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.160 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.161 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.161 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.731222+0000", 2024-09-20T22:12:38.161 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8452000000000002e-05 2024-09-20T22:12:38.161 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.161 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.161 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.161 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.161 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105673 8.14 8:2f269e39:::benchmark_data_smithi060_111051_object105672:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:12:38.161 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.761620+0000", 2024-09-20T22:12:38.161 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 209.33720672600001, 2024-09-20T22:12:38.161 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.83821859799999998, 2024-09-20T22:12:38.161 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.161 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.162 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:38.162 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:12:38.162 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:12:38.162 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:38.162 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105673 2024-09-20T22:12:38.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.162 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.162 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.162 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.162 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761620+0000", 2024-09-20T22:12:38.162 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.163 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.163 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761620+0000", 2024-09-20T22:12:38.163 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.163 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.163 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761621+0000", 2024-09-20T22:12:38.163 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.392e-06 2024-09-20T22:12:38.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.163 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.164 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761674+0000", 2024-09-20T22:12:38.164 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2677999999999999e-05 2024-09-20T22:12:38.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.164 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.164 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761686+0000", 2024-09-20T22:12:38.164 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1926000000000001e-05 2024-09-20T22:12:38.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.164 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.164 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761697+0000", 2024-09-20T22:12:38.164 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1364e-05 2024-09-20T22:12:38.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.165 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.165 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761770+0000", 2024-09-20T22:12:38.165 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2904999999999994e-05 2024-09-20T22:12:38.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.165 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.165 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.165 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761777+0000", 2024-09-20T22:12:38.165 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8029999999999999e-06 2024-09-20T22:12:38.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.165 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.165 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.165 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584113+0000", 2024-09-20T22:12:38.165 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.82233672499999999 2024-09-20T22:12:38.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.166 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.166 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:38.166 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584256+0000", 2024-09-20T22:12:38.166 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000142686 2024-09-20T22:12:38.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.166 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.166 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,4", 2024-09-20T22:12:38.166 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584528+0000", 2024-09-20T22:12:38.166 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00027199499999999997 2024-09-20T22:12:38.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.166 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.166 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.166 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.590292+0000", 2024-09-20T22:12:38.167 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.005764333 2024-09-20T22:12:38.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.167 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.167 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:12:38.167 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.590315+0000", 2024-09-20T22:12:38.167 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2768e-05 2024-09-20T22:12:38.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.167 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.167 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.167 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.599740+0000", 2024-09-20T22:12:38.167 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0094248200000000004 2024-09-20T22:12:38.167 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.167 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.167 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:12:38.168 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.599795+0000", 2024-09-20T22:12:38.168 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5108000000000002e-05 2024-09-20T22:12:38.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.168 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.168 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.168 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.599838+0000", 2024-09-20T22:12:38.168 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3095e-05 2024-09-20T22:12:38.168 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.168 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.168 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.168 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.168 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.168 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:106978 8.19 8:9e674fff:::benchmark_data_smithi060_111051_object106977:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:12:38.168 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.576265+0000", 2024-09-20T22:12:38.169 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 207.522561612, 2024-09-20T22:12:38.169 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9488434429999999, 2024-09-20T22:12:38.169 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.169 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.169 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:38.169 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:12:38.169 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:12:38.169 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:38.169 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 106978 2024-09-20T22:12:38.169 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.169 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.169 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.169 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.169 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576265+0000", 2024-09-20T22:12:38.169 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.170 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.170 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.170 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.170 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576265+0000", 2024-09-20T22:12:38.170 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.170 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.170 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.170 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.170 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576267+0000", 2024-09-20T22:12:38.170 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0509999999999999e-06 2024-09-20T22:12:38.170 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.170 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.171 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.171 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576299+0000", 2024-09-20T22:12:38.171 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1962000000000003e-05 2024-09-20T22:12:38.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.171 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.171 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.171 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576312+0000", 2024-09-20T22:12:38.171 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3423e-05 2024-09-20T22:12:38.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.171 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.171 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.171 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576322+0000", 2024-09-20T22:12:38.171 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0118e-05 2024-09-20T22:12:38.171 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.172 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.172 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.172 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576468+0000", 2024-09-20T22:12:38.172 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000145575 2024-09-20T22:12:38.172 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.172 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.172 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.172 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576481+0000", 2024-09-20T22:12:38.172 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3023999999999999e-05 2024-09-20T22:12:38.172 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.172 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.172 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.172 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410333+0000", 2024-09-20T22:12:38.172 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.83385187800000005 2024-09-20T22:12:38.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.173 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.173 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.173 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410345+0000", 2024-09-20T22:12:38.173 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.258e-05 2024-09-20T22:12:38.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.173 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.173 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.173 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.503436+0000", 2024-09-20T22:12:38.173 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.093091088 2024-09-20T22:12:38.173 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.173 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.173 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:38.174 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.503599+0000", 2024-09-20T22:12:38.174 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00016286999999999999 2024-09-20T22:12:38.174 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.174 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.174 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4,6", 2024-09-20T22:12:38.174 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.505264+0000", 2024-09-20T22:12:38.174 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001664591 2024-09-20T22:12:38.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.175 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.175 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.175 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.508504+0000", 2024-09-20T22:12:38.175 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0032404920000000002 2024-09-20T22:12:38.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.175 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.175 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.175 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.508547+0000", 2024-09-20T22:12:38.175 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2734000000000001e-05 2024-09-20T22:12:38.175 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.175 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.176 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.176 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.513180+0000", 2024-09-20T22:12:38.176 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046330490000000002 2024-09-20T22:12:38.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.176 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.176 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:12:38.176 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.524891+0000", 2024-09-20T22:12:38.176 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.011710379999999999 2024-09-20T22:12:38.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.176 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.176 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:12:38.176 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.524966+0000", 2024-09-20T22:12:38.176 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5290999999999995e-05 2024-09-20T22:12:38.176 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.177 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.177 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.177 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.525108+0000", 2024-09-20T22:12:38.177 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000142337 2024-09-20T22:12:38.177 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.177 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.177 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.177 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.177 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.177 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:107002 8.11 8.e0f52171 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:12:38.177 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.617682+0000", 2024-09-20T22:12:38.177 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 207.48114411399999, 2024-09-20T22:12:38.177 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79265608300000001, 2024-09-20T22:12:38.177 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.178 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.178 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:12:38.178 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:12:38.178 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:12:38.178 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:38.178 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 107002 2024-09-20T22:12:38.178 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.178 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.178 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.178 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.178 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617682+0000", 2024-09-20T22:12:38.178 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.178 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.178 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.179 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.179 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617682+0000", 2024-09-20T22:12:38.179 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.179 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.179 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.179 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.179 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617684+0000", 2024-09-20T22:12:38.179 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1299999999999999e-06 2024-09-20T22:12:38.179 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.179 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.179 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.179 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617725+0000", 2024-09-20T22:12:38.179 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0839e-05 2024-09-20T22:12:38.179 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.180 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.180 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.180 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617738+0000", 2024-09-20T22:12:38.180 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2310000000000001e-05 2024-09-20T22:12:38.180 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.180 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.180 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.180 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617746+0000", 2024-09-20T22:12:38.180 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3939999999999996e-06 2024-09-20T22:12:38.180 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.180 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.180 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.180 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617856+0000", 2024-09-20T22:12:38.180 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000109917 2024-09-20T22:12:38.181 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.181 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.181 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.181 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617878+0000", 2024-09-20T22:12:38.181 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2495999999999999e-05 2024-09-20T22:12:38.181 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.181 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.181 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.181 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410330+0000", 2024-09-20T22:12:38.181 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79245202299999995 2024-09-20T22:12:38.181 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.181 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.181 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.181 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410338+0000", 2024-09-20T22:12:38.181 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9740000000000002e-06 2024-09-20T22:12:38.182 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.182 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.182 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.182 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.182 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.182 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163158 8.11 8:8e0bce89:::benchmark_data_smithi060_111051_object43159:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:12:38.182 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:08.839869+0000", 2024-09-20T22:12:38.182 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 149.258957673, 2024-09-20T22:12:38.182 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.65193807500000001, 2024-09-20T22:12:38.182 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.182 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.182 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:38.182 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:12:38.182 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:12:38.183 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:38.183 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163158 2024-09-20T22:12:38.183 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.183 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.183 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.183 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.183 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839869+0000", 2024-09-20T22:12:38.183 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.183 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.183 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.183 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.183 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839869+0000", 2024-09-20T22:12:38.183 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.183 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.184 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.184 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.184 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839870+0000", 2024-09-20T22:12:38.184 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5710000000000001e-06 2024-09-20T22:12:38.184 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.184 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.184 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.184 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839876+0000", 2024-09-20T22:12:38.184 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3820000000000003e-06 2024-09-20T22:12:38.184 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.184 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.184 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.184 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839890+0000", 2024-09-20T22:12:38.184 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3952e-05 2024-09-20T22:12:38.185 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.185 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.185 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.185 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839897+0000", 2024-09-20T22:12:38.185 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4150000000000002e-06 2024-09-20T22:12:38.185 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.185 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.185 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.185 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839956+0000", 2024-09-20T22:12:38.185 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9157999999999998e-05 2024-09-20T22:12:38.185 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.185 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.185 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.185 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839967+0000", 2024-09-20T22:12:38.186 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1019000000000001e-05 2024-09-20T22:12:38.186 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.186 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.186 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.186 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477635+0000", 2024-09-20T22:12:38.186 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.63766765700000005 2024-09-20T22:12:38.186 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.186 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.186 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:38.186 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477737+0000", 2024-09-20T22:12:38.186 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010265899999999999 2024-09-20T22:12:38.186 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.186 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.186 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:12:38.186 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477939+0000", 2024-09-20T22:12:38.187 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020106900000000001 2024-09-20T22:12:38.187 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.187 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.187 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:12:38.187 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.482557+0000", 2024-09-20T22:12:38.187 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046186830000000002 2024-09-20T22:12:38.187 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.187 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.187 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.187 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491495+0000", 2024-09-20T22:12:38.187 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0089377699999999994 2024-09-20T22:12:38.187 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.187 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.187 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.188 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491673+0000", 2024-09-20T22:12:38.188 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017837500000000001 2024-09-20T22:12:38.188 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.188 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.188 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:12:38.188 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491762+0000", 2024-09-20T22:12:38.188 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8555999999999997e-05 2024-09-20T22:12:38.188 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.188 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.188 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.188 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491807+0000", 2024-09-20T22:12:38.188 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4808999999999997e-05 2024-09-20T22:12:38.188 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.188 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.189 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.189 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.189 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.189 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163289 8.11 8:8c47e2c8:::benchmark_data_smithi060_111051_object43290:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:12:38.189 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:08.954133+0000", 2024-09-20T22:12:38.189 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 149.144693039, 2024-09-20T22:12:38.189 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.53808307099999997, 2024-09-20T22:12:38.189 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.189 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.189 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:38.189 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:12:38.189 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:12:38.189 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:38.190 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163289 2024-09-20T22:12:38.190 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.190 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.190 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.190 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.190 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954133+0000", 2024-09-20T22:12:38.190 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.190 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.190 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.190 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.190 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954133+0000", 2024-09-20T22:12:38.190 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.190 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.190 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954134+0000", 2024-09-20T22:12:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2900000000000002e-07 2024-09-20T22:12:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954138+0000", 2024-09-20T22:12:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5260000000000002e-06 2024-09-20T22:12:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954147+0000", 2024-09-20T22:12:38.191 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2010000000000005e-06 2024-09-20T22:12:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954153+0000", 2024-09-20T22:12:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0610000000000001e-06 2024-09-20T22:12:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954209+0000", 2024-09-20T22:12:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5767999999999999e-05 2024-09-20T22:12:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.192 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954216+0000", 2024-09-20T22:12:38.193 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8099999999999998e-06 2024-09-20T22:12:38.193 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.193 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.193 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.193 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.478998+0000", 2024-09-20T22:12:38.193 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.52478124900000001 2024-09-20T22:12:38.193 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.193 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.193 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:38.193 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.479095+0000", 2024-09-20T22:12:38.193 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7125000000000003e-05 2024-09-20T22:12:38.193 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.193 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.194 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:12:38.194 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.479291+0000", 2024-09-20T22:12:38.194 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019642100000000001 2024-09-20T22:12:38.194 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.194 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.194 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:12:38.194 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.482759+0000", 2024-09-20T22:12:38.194 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0034672689999999998 2024-09-20T22:12:38.194 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.194 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.194 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.194 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491864+0000", 2024-09-20T22:12:38.194 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0091058630000000005 2024-09-20T22:12:38.194 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.194 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.195 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.195 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491961+0000", 2024-09-20T22:12:38.195 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6717000000000003e-05 2024-09-20T22:12:38.195 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.195 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.195 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:12:38.195 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492014+0000", 2024-09-20T22:12:38.195 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2877000000000002e-05 2024-09-20T22:12:38.195 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.195 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.195 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.195 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492216+0000", 2024-09-20T22:12:38.195 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020235500000000001 2024-09-20T22:12:38.196 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.196 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.196 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.196 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.196 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.196 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163324 8.11 8:8e312e9e:::benchmark_data_smithi060_111051_object43325:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:12:38.196 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:08.983848+0000", 2024-09-20T22:12:38.196 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 149.11497811699999, 2024-09-20T22:12:38.196 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.50861305999999995, 2024-09-20T22:12:38.196 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.196 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.196 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:38.196 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:12:38.196 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:12:38.196 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:38.197 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163324 2024-09-20T22:12:38.197 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.197 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.197 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.197 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.197 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983848+0000", 2024-09-20T22:12:38.197 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.197 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.197 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.197 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.197 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983848+0000", 2024-09-20T22:12:38.197 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.197 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.197 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.197 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.198 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983849+0000", 2024-09-20T22:12:38.198 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.11e-06 2024-09-20T22:12:38.198 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.198 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.198 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.198 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983853+0000", 2024-09-20T22:12:38.198 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0439999999999997e-06 2024-09-20T22:12:38.198 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.198 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.198 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.198 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983863+0000", 2024-09-20T22:12:38.198 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6430000000000007e-06 2024-09-20T22:12:38.198 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.198 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.199 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.199 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983868+0000", 2024-09-20T22:12:38.199 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1000000000000003e-06 2024-09-20T22:12:38.199 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.199 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.199 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.199 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983919+0000", 2024-09-20T22:12:38.199 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0433999999999999e-05 2024-09-20T22:12:38.199 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.199 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.199 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.199 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983928+0000", 2024-09-20T22:12:38.199 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3659999999999998e-06 2024-09-20T22:12:38.199 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.200 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.200 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.200 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.480646+0000", 2024-09-20T22:12:38.200 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.49671833999999998 2024-09-20T22:12:38.200 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.200 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.200 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:38.200 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.480768+0000", 2024-09-20T22:12:38.200 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012145 2024-09-20T22:12:38.200 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.200 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.200 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:12:38.200 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.481025+0000", 2024-09-20T22:12:38.200 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025713299999999999 2024-09-20T22:12:38.201 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.201 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.201 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:12:38.201 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.488496+0000", 2024-09-20T22:12:38.201 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0074709540000000001 2024-09-20T22:12:38.201 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.201 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.201 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.201 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492276+0000", 2024-09-20T22:12:38.201 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0037803670000000002 2024-09-20T22:12:38.201 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.201 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.201 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.201 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492350+0000", 2024-09-20T22:12:38.202 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3436999999999995e-05 2024-09-20T22:12:38.202 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.202 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.202 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:12:38.202 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492421+0000", 2024-09-20T22:12:38.203 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1303000000000005e-05 2024-09-20T22:12:38.203 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.203 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.203 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.203 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492461+0000", 2024-09-20T22:12:38.203 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0379000000000001e-05 2024-09-20T22:12:38.203 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.203 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.203 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.203 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.203 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.204 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163358 8.11 8:8b7ace29:::benchmark_data_smithi060_111051_object43359:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:12:38.204 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.014358+0000", 2024-09-20T22:12:38.204 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 149.08446813200001, 2024-09-20T22:12:38.204 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.47828052599999998, 2024-09-20T22:12:38.204 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.204 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.204 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:38.204 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:12:38.204 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:12:38.204 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:38.204 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163358 2024-09-20T22:12:38.204 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.204 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.204 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.205 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.205 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014358+0000", 2024-09-20T22:12:38.205 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.205 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.205 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.205 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.205 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014358+0000", 2024-09-20T22:12:38.205 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.205 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.205 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.205 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.205 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014359+0000", 2024-09-20T22:12:38.205 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0669999999999999e-06 2024-09-20T22:12:38.206 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.206 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.206 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.206 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014364+0000", 2024-09-20T22:12:38.206 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0220000000000004e-06 2024-09-20T22:12:38.206 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.206 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.206 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.206 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014374+0000", 2024-09-20T22:12:38.206 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9599999999999995e-06 2024-09-20T22:12:38.206 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.206 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.206 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.206 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014388+0000", 2024-09-20T22:12:38.207 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3413e-05 2024-09-20T22:12:38.207 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.207 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.207 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.207 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014447+0000", 2024-09-20T22:12:38.207 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9698999999999999e-05 2024-09-20T22:12:38.207 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.207 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.207 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.207 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014455+0000", 2024-09-20T22:12:38.207 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.072e-06 2024-09-20T22:12:38.207 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.207 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.207 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.207 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.482921+0000", 2024-09-20T22:12:38.208 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.46846513200000001 2024-09-20T22:12:38.208 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.208 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.208 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:38.208 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483020+0000", 2024-09-20T22:12:38.208 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9319000000000002e-05 2024-09-20T22:12:38.208 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.208 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.208 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:12:38.208 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483251+0000", 2024-09-20T22:12:38.208 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00023124300000000001 2024-09-20T22:12:38.208 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.209 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.209 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:12:38.209 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491608+0000", 2024-09-20T22:12:38.209 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0083564190000000003 2024-09-20T22:12:38.209 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.209 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.209 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.209 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492507+0000", 2024-09-20T22:12:38.209 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00089972900000000005 2024-09-20T22:12:38.209 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.209 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.209 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.209 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492546+0000", 2024-09-20T22:12:38.209 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9156999999999998e-05 2024-09-20T22:12:38.210 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.210 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.210 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:12:38.210 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492602+0000", 2024-09-20T22:12:38.210 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5114999999999997e-05 2024-09-20T22:12:38.210 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.210 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.210 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.210 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492639+0000", 2024-09-20T22:12:38.210 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7178999999999998e-05 2024-09-20T22:12:38.210 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.210 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.210 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.211 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.211 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.211 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163372 8.11 8:8b6ec81f:::benchmark_data_smithi060_111051_object43373:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:12:38.211 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.027195+0000", 2024-09-20T22:12:38.211 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 149.07163132400001, 2024-09-20T22:12:38.211 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.46568983899999999, 2024-09-20T22:12:38.211 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.211 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.211 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:38.211 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:12:38.211 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:12:38.211 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:38.211 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163372 2024-09-20T22:12:38.212 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.212 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.212 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.212 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.212 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027195+0000", 2024-09-20T22:12:38.212 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.212 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.212 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.212 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.212 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027195+0000", 2024-09-20T22:12:38.212 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.212 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.212 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.212 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.213 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027196+0000", 2024-09-20T22:12:38.213 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0599999999999999e-07 2024-09-20T22:12:38.213 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.213 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.213 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.213 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027200+0000", 2024-09-20T22:12:38.213 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9280000000000003e-06 2024-09-20T22:12:38.213 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.213 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.213 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.213 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027210+0000", 2024-09-20T22:12:38.213 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0035e-05 2024-09-20T22:12:38.213 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.213 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.214 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.214 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027217+0000", 2024-09-20T22:12:38.214 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6450000000000005e-06 2024-09-20T22:12:38.214 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.214 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.214 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.214 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027281+0000", 2024-09-20T22:12:38.214 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3281000000000006e-05 2024-09-20T22:12:38.214 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.214 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.214 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.214 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027291+0000", 2024-09-20T22:12:38.214 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0268999999999999e-05 2024-09-20T22:12:38.214 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.215 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.215 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.215 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.484647+0000", 2024-09-20T22:12:38.215 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.45735564499999998 2024-09-20T22:12:38.215 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.215 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.215 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:38.215 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.484745+0000", 2024-09-20T22:12:38.215 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8712e-05 2024-09-20T22:12:38.215 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.215 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.215 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:12:38.215 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.484940+0000", 2024-09-20T22:12:38.215 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019504900000000001 2024-09-20T22:12:38.215 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.216 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.216 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:12:38.216 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491619+0000", 2024-09-20T22:12:38.216 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0066782090000000001 2024-09-20T22:12:38.216 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.216 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.216 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.216 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492716+0000", 2024-09-20T22:12:38.216 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001097701 2024-09-20T22:12:38.216 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.216 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.216 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.216 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492778+0000", 2024-09-20T22:12:38.216 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1538000000000006e-05 2024-09-20T22:12:38.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.217 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.217 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:12:38.217 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492830+0000", 2024-09-20T22:12:38.217 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1997999999999999e-05 2024-09-20T22:12:38.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.217 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.217 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.217 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492885+0000", 2024-09-20T22:12:38.217 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5022999999999998e-05 2024-09-20T22:12:38.217 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.217 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.217 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.217 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.218 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.218 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163373 8.11 8:8d07a686:::benchmark_data_smithi060_111051_object43374:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:12:38.218 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.027847+0000", 2024-09-20T22:12:38.218 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 149.070978916, 2024-09-20T22:12:38.218 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.465200326, 2024-09-20T22:12:38.218 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.218 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.218 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:38.218 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:12:38.218 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:12:38.218 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:38.218 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163373 2024-09-20T22:12:38.218 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.219 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.219 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.219 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.219 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027847+0000", 2024-09-20T22:12:38.219 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.219 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.219 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.219 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.219 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027847+0000", 2024-09-20T22:12:38.219 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.219 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.219 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.219 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.219 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027848+0000", 2024-09-20T22:12:38.220 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3600000000000003e-07 2024-09-20T22:12:38.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.220 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.220 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.220 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027852+0000", 2024-09-20T22:12:38.220 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6330000000000002e-06 2024-09-20T22:12:38.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.220 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.220 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.220 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027860+0000", 2024-09-20T22:12:38.220 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2789999999999998e-06 2024-09-20T22:12:38.220 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.220 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.220 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.220 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027866+0000", 2024-09-20T22:12:38.221 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1070000000000001e-06 2024-09-20T22:12:38.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.221 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.221 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.221 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027897+0000", 2024-09-20T22:12:38.221 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0722000000000002e-05 2024-09-20T22:12:38.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.221 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.221 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.221 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027901+0000", 2024-09-20T22:12:38.221 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1649999999999999e-06 2024-09-20T22:12:38.221 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.221 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.221 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.222 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.486658+0000", 2024-09-20T22:12:38.222 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.45875733499999999 2024-09-20T22:12:38.222 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.222 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.222 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:38.222 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.486767+0000", 2024-09-20T22:12:38.222 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000108385 2024-09-20T22:12:38.222 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.222 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.222 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:12:38.222 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487120+0000", 2024-09-20T22:12:38.222 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00035312499999999998 2024-09-20T22:12:38.222 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.222 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.223 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:12:38.223 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491877+0000", 2024-09-20T22:12:38.223 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0047569170000000003 2024-09-20T22:12:38.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.223 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.223 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.223 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492928+0000", 2024-09-20T22:12:38.223 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001051633 2024-09-20T22:12:38.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.223 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.223 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.223 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492967+0000", 2024-09-20T22:12:38.223 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.8781e-05 2024-09-20T22:12:38.223 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.223 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.224 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:12:38.224 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493013+0000", 2024-09-20T22:12:38.224 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5555e-05 2024-09-20T22:12:38.224 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.224 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.224 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.224 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493048+0000", 2024-09-20T22:12:38.224 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5052999999999998e-05 2024-09-20T22:12:38.224 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.224 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.224 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.224 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.224 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.224 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163388 8.11 8:893f1034:::benchmark_data_smithi060_111051_object43389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:12:38.225 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.050114+0000", 2024-09-20T22:12:38.225 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 149.048712206, 2024-09-20T22:12:38.225 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.44308652799999998, 2024-09-20T22:12:38.225 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.225 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.225 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:38.225 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:12:38.225 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:12:38.225 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:38.225 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163388 2024-09-20T22:12:38.225 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.225 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.225 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.225 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.225 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050114+0000", 2024-09-20T22:12:38.226 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.226 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.226 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.226 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.226 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050114+0000", 2024-09-20T22:12:38.226 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.226 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.226 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.226 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.226 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050115+0000", 2024-09-20T22:12:38.226 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2500000000000005e-07 2024-09-20T22:12:38.226 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.226 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.227 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.227 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050119+0000", 2024-09-20T22:12:38.227 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.698e-06 2024-09-20T22:12:38.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.227 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.227 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.227 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050128+0000", 2024-09-20T22:12:38.227 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5820000000000008e-06 2024-09-20T22:12:38.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.227 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.227 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.227 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050133+0000", 2024-09-20T22:12:38.227 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9060000000000001e-06 2024-09-20T22:12:38.227 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.228 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.228 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.228 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050169+0000", 2024-09-20T22:12:38.228 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5984e-05 2024-09-20T22:12:38.228 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.228 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.228 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.228 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050174+0000", 2024-09-20T22:12:38.228 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0180000000000001e-06 2024-09-20T22:12:38.228 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.228 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.228 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.228 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.488568+0000", 2024-09-20T22:12:38.228 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.43839391 2024-09-20T22:12:38.228 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.229 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.229 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:38.229 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.488685+0000", 2024-09-20T22:12:38.229 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000117163 2024-09-20T22:12:38.229 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.229 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.229 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:12:38.229 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.489162+0000", 2024-09-20T22:12:38.229 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00047712600000000003 2024-09-20T22:12:38.229 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.229 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.229 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:12:38.229 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491884+0000", 2024-09-20T22:12:38.229 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0027215329999999999 2024-09-20T22:12:38.229 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.230 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.230 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.230 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493086+0000", 2024-09-20T22:12:38.230 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001201765 2024-09-20T22:12:38.230 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.230 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.230 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.230 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493122+0000", 2024-09-20T22:12:38.230 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6622999999999998e-05 2024-09-20T22:12:38.230 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.230 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.230 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:12:38.230 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493168+0000", 2024-09-20T22:12:38.231 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5757999999999998e-05 2024-09-20T22:12:38.231 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.231 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.231 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.231 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493201+0000", 2024-09-20T22:12:38.231 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2737000000000002e-05 2024-09-20T22:12:38.231 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.231 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.231 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.231 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.231 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.231 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163393 8.11 8:8bb06d42:::benchmark_data_smithi060_111051_object43394:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:12:38.231 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.064776+0000", 2024-09-20T22:12:38.231 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 149.034050283, 2024-09-20T22:12:38.232 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.43944195600000002, 2024-09-20T22:12:38.232 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:38.232 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:38.232 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:38.232 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:12:38.232 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:12:38.232 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:38.232 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163393 2024-09-20T22:12:38.232 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.232 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:38.232 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.232 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:38.232 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064776+0000", 2024-09-20T22:12:38.232 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.233 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.233 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.233 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:38.233 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064776+0000", 2024-09-20T22:12:38.233 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:38.233 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.233 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.233 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:38.233 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064777+0000", 2024-09-20T22:12:38.233 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9500000000000001e-07 2024-09-20T22:12:38.233 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.233 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.233 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:38.233 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064781+0000", 2024-09-20T22:12:38.233 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9870000000000001e-06 2024-09-20T22:12:38.234 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.234 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.234 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:38.234 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064791+0000", 2024-09-20T22:12:38.234 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0417999999999999e-05 2024-09-20T22:12:38.234 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.234 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.234 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:38.234 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064797+0000", 2024-09-20T22:12:38.234 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4040000000000003e-06 2024-09-20T22:12:38.234 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.234 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.234 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.234 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064838+0000", 2024-09-20T22:12:38.235 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1591000000000002e-05 2024-09-20T22:12:38.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.235 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.235 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:12:38.235 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064843+0000", 2024-09-20T22:12:38.235 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9470000000000002e-06 2024-09-20T22:12:38.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.235 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.235 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:38.235 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493235+0000", 2024-09-20T22:12:38.235 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.42839186899999998 2024-09-20T22:12:38.235 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.235 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.235 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:38.235 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493336+0000", 2024-09-20T22:12:38.236 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100923 2024-09-20T22:12:38.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.236 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.236 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:12:38.236 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493659+0000", 2024-09-20T22:12:38.236 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00032341599999999998 2024-09-20T22:12:38.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.236 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.236 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:12:38.236 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.499928+0000", 2024-09-20T22:12:38.236 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0062687730000000001 2024-09-20T22:12:38.236 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.236 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.237 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.237 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.503797+0000", 2024-09-20T22:12:38.237 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0038686020000000001 2024-09-20T22:12:38.237 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.237 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.237 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:38.237 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.504160+0000", 2024-09-20T22:12:38.237 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00036300900000000002 2024-09-20T22:12:38.237 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.237 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.237 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:12:38.237 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.504192+0000", 2024-09-20T22:12:38.237 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2592000000000002e-05 2024-09-20T22:12:38.237 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:38.238 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:38.238 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:12:38.238 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.504218+0000", 2024-09-20T22:12:38.238 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.563e-05 2024-09-20T22:12:38.238 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.238 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.238 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.238 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:38.238 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:38.238 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:12:38.248 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:38.285 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:38.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:38.287 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:38.387 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:38.421 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:38.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:38.422 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:38.523 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:38.559 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:38.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:38.560 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:38.661 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:38.697 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:38.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:38.698 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:38.799 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:38.811 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:38 smithi067 ceph-mon[33511]: osdmap e317: 8 total, 8 up, 8 in 2024-09-20T22:12:38.811 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:38 smithi067 ceph-mon[33511]: 3.f scrub starts 2024-09-20T22:12:38.812 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:38 smithi067 ceph-mon[33511]: 3.f scrub ok 2024-09-20T22:12:38.812 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:38 smithi067 ceph-mon[33511]: pgmap v526: 158 pgs: 158 active+clean; 4.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 908 B/s rd, 63 MiB/s wr, 1.01k op/s; 11 KiB/s, 0 objects/s recovering 2024-09-20T22:12:38.834 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:12:38.843 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:38.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:38.845 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:38.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:38 smithi060 ceph-mon[29694]: osdmap e317: 8 total, 8 up, 8 in 2024-09-20T22:12:38.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:38 smithi060 ceph-mon[29694]: 3.f scrub starts 2024-09-20T22:12:38.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:38 smithi060 ceph-mon[29694]: 3.f scrub ok 2024-09-20T22:12:38.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:38 smithi060 ceph-mon[29694]: pgmap v526: 158 pgs: 158 active+clean; 4.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 908 B/s rd, 63 MiB/s wr, 1.01k op/s; 11 KiB/s, 0 objects/s recovering 2024-09-20T22:12:38.882 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:38 smithi060 ceph-mon[40290]: osdmap e317: 8 total, 8 up, 8 in 2024-09-20T22:12:38.882 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:38 smithi060 ceph-mon[40290]: 3.f scrub starts 2024-09-20T22:12:38.882 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:38 smithi060 ceph-mon[40290]: 3.f scrub ok 2024-09-20T22:12:38.882 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:38 smithi060 ceph-mon[40290]: pgmap v526: 158 pgs: 158 active+clean; 4.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 908 B/s rd, 63 MiB/s wr, 1.01k op/s; 11 KiB/s, 0 objects/s recovering 2024-09-20T22:12:38.900 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 51 15 83089 83074 101.781 77.9375 0.0205708 0.00981992 2024-09-20T22:12:38.946 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:38.978 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:38.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:38.979 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:39.080 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:39.109 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:39.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:39.110 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:39.211 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:39.242 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:39.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:39.244 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:39.281 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T22:12:39.310 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:39.345 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:39.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:39.396 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:39.396 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:39.473 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:12:39.496 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:39.518 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:39.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:39.519 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:39.620 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:39.628 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:12:39.628 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:12:39.628 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-09-20T22:12:39.628 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd out 0 2024-09-20T22:12:39.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:39.664 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:39.664 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:39.735 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:12:39.765 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:39 smithi060 ceph-mon[29694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-09-20T22:12:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:39 smithi060 ceph-mon[29694]: 3.1f scrub starts 2024-09-20T22:12:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:39 smithi060 ceph-mon[29694]: 5.1a scrub starts 2024-09-20T22:12:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:39 smithi060 ceph-mon[29694]: 5.1a scrub ok 2024-09-20T22:12:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:39 smithi060 ceph-mon[29694]: 3.1f scrub ok 2024-09-20T22:12:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:39 smithi060 ceph-mon[29694]: 9.4 scrub starts 2024-09-20T22:12:39.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:39 smithi060 ceph-mon[40290]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-09-20T22:12:39.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:39 smithi060 ceph-mon[40290]: 3.1f scrub starts 2024-09-20T22:12:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:39 smithi060 ceph-mon[40290]: 5.1a scrub starts 2024-09-20T22:12:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:39 smithi060 ceph-mon[40290]: 5.1a scrub ok 2024-09-20T22:12:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:39 smithi060 ceph-mon[40290]: 3.1f scrub ok 2024-09-20T22:12:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:39 smithi060 ceph-mon[40290]: 9.4 scrub starts 2024-09-20T22:12:39.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:39.811 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:39.812 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:39.867 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:39 smithi067 ceph-mon[33511]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs inactive, 2 pgs peering) 2024-09-20T22:12:39.867 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:39 smithi067 ceph-mon[33511]: 3.1f scrub starts 2024-09-20T22:12:39.867 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:39 smithi067 ceph-mon[33511]: 5.1a scrub starts 2024-09-20T22:12:39.867 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:39 smithi067 ceph-mon[33511]: 5.1a scrub ok 2024-09-20T22:12:39.867 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:39 smithi067 ceph-mon[33511]: 3.1f scrub ok 2024-09-20T22:12:39.867 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:39 smithi067 ceph-mon[33511]: 9.4 scrub starts 2024-09-20T22:12:39.900 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 52 16 84173 84157 101.125 67.6875 0.0118399 0.00988405 2024-09-20T22:12:39.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:39.988 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:39.988 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:39.989 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:40.090 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:40.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:40.159 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:40.159 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:40.260 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:40.298 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:40.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:40.299 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:40.399 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:40.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:40.436 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:40.436 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:40.478 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:12:40.478 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:12:40.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:40.478 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.15594.0:84750 9.0 e317/286 9:0f094282:::benchmark_data_smithi060_140132_object84749:head v 317'5277, mlcod=317'5275)", 2024-09-20T22:12:40.479 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:12:40.472812+0000", 2024-09-20T22:12:40.479 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.003380858, 2024-09-20T22:12:40.479 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0034092369999999999, 2024-09-20T22:12:40.479 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:12:40.479 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:12:40.479 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:12:40.479 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:12:40.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:40.479 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:12:40.479 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:40.472812+0000", 2024-09-20T22:12:40.479 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:40.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:40.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:40.479 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:12:40.480 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:40.472812+0000", 2024-09-20T22:12:40.480 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:12:40.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:40.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:40.480 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:12:40.480 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:40.472813+0000", 2024-09-20T22:12:40.480 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8400000000000003e-07 2024-09-20T22:12:40.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:40.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:40.480 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:12:40.480 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:40.472846+0000", 2024-09-20T22:12:40.480 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2527000000000002e-05 2024-09-20T22:12:40.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:40.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:40.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:12:40.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:40.472856+0000", 2024-09-20T22:12:40.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0713000000000001e-05 2024-09-20T22:12:40.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:40.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:40.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:12:40.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:40.472862+0000", 2024-09-20T22:12:40.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0750000000000003e-06 2024-09-20T22:12:40.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:40.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:40.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:12:40.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:40.472927+0000", 2024-09-20T22:12:40.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4187000000000001e-05 2024-09-20T22:12:40.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:12:40.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:12:40.482 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:12:40.482 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:40.472945+0000", 2024-09-20T22:12:40.482 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8210999999999999e-05 2024-09-20T22:12:40.482 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:40.482 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:12:40.482 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:40.482 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:12:40.482 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-20T22:12:40.482 INFO:teuthology.orchestra.run.smithi067.stdout: "num_ops": 1 2024-09-20T22:12:40.482 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:12:40.538 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:40.574 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:40.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:40.576 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:40.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:40.710 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:40.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:40.711 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:40 smithi067 ceph-mon[33511]: 4.10 scrub starts 2024-09-20T22:12:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:40 smithi067 ceph-mon[33511]: 4.10 scrub ok 2024-09-20T22:12:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:40 smithi067 ceph-mon[33511]: pgmap v527: 158 pgs: 158 active+clean; 4.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 770 B/s rd, 47 MiB/s wr, 747 op/s; 9.6 KiB/s, 0 objects/s recovering 2024-09-20T22:12:40.812 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:40.866 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:40.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:40.867 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:40.901 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 53 16 85219 85203 100.45 65.375 0.0216584 0.00995018 2024-09-20T22:12:40.915 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:40 smithi060 ceph-mon[29694]: 4.10 scrub starts 2024-09-20T22:12:40.915 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:40 smithi060 ceph-mon[29694]: 4.10 scrub ok 2024-09-20T22:12:40.915 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:40 smithi060 ceph-mon[29694]: pgmap v527: 158 pgs: 158 active+clean; 4.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 770 B/s rd, 47 MiB/s wr, 747 op/s; 9.6 KiB/s, 0 objects/s recovering 2024-09-20T22:12:40.915 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:40 smithi060 ceph-mon[40290]: 4.10 scrub starts 2024-09-20T22:12:40.916 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:40 smithi060 ceph-mon[40290]: 4.10 scrub ok 2024-09-20T22:12:40.916 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:40 smithi060 ceph-mon[40290]: pgmap v527: 158 pgs: 158 active+clean; 4.9 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 770 B/s rd, 47 MiB/s wr, 747 op/s; 9.6 KiB/s, 0 objects/s recovering 2024-09-20T22:12:40.968 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:40.994 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:40.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:40.995 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:41.023 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:41.036 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:12:41.036 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:12:41.040 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:41.072 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:12:41.073 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:12:41.095 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:41.108 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:12:41.108 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:12:41.152 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:41.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:41.154 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:41.158 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:12:41.158 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:12:41.192 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:12:41.192 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:12:41.229 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:12:41.229 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:12:41.254 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:41.265 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:12:41.266 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:12:41.301 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:12:41.302 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:12:41.305 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:41.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:41.307 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:41.407 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:41.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:41.442 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:41.442 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:41.543 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:41.566 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T22:12:41.585 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:41.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:41.586 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:41.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:41.730 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:41.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:41.731 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:41.832 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:41.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:41.872 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:41.872 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:41.901 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 54 16 86221 86205 99.7499 62.625 0.011655 0.0100199 2024-09-20T22:12:41.972 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:41.976 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:12:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:41 smithi067 ceph-mon[33511]: 3.14 scrub starts 2024-09-20T22:12:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:41 smithi067 ceph-mon[33511]: 3.14 scrub ok 2024-09-20T22:12:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:41 smithi067 ceph-mon[33511]: 6.c scrub starts 2024-09-20T22:12:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:41 smithi067 ceph-mon[33511]: 6.c scrub ok 2024-09-20T22:12:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:41 smithi067 ceph-mon[33511]: 1.0 scrub starts 2024-09-20T22:12:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:41 smithi067 ceph-mon[33511]: 1.0 scrub ok 2024-09-20T22:12:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:41 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1489090031' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:42.026 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:42.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:42.027 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:42.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:41 smithi060 ceph-mon[29694]: 3.14 scrub starts 2024-09-20T22:12:42.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:41 smithi060 ceph-mon[29694]: 3.14 scrub ok 2024-09-20T22:12:42.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:41 smithi060 ceph-mon[29694]: 6.c scrub starts 2024-09-20T22:12:42.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:41 smithi060 ceph-mon[29694]: 6.c scrub ok 2024-09-20T22:12:42.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:41 smithi060 ceph-mon[29694]: 1.0 scrub starts 2024-09-20T22:12:42.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:41 smithi060 ceph-mon[29694]: 1.0 scrub ok 2024-09-20T22:12:42.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:41 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1489090031' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:42.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:41 smithi060 ceph-mon[40290]: 3.14 scrub starts 2024-09-20T22:12:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:41 smithi060 ceph-mon[40290]: 3.14 scrub ok 2024-09-20T22:12:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:41 smithi060 ceph-mon[40290]: 6.c scrub starts 2024-09-20T22:12:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:41 smithi060 ceph-mon[40290]: 6.c scrub ok 2024-09-20T22:12:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:41 smithi060 ceph-mon[40290]: 1.0 scrub starts 2024-09-20T22:12:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:41 smithi060 ceph-mon[40290]: 1.0 scrub ok 2024-09-20T22:12:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:41 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1489090031' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:42.128 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:42.177 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:42.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:42.178 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:42.279 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:42.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:42.317 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:42.318 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:42.418 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:42.466 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:42.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:42.467 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:42.568 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:42.594 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:42.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:42.595 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:42.599 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:12:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:12:42.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:42 smithi060 ceph-mon[29694]: 9.4 scrub ok 2024-09-20T22:12:42.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:42 smithi060 ceph-mon[29694]: 6.1e scrub starts 2024-09-20T22:12:42.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:42 smithi060 ceph-mon[29694]: 6.1e scrub ok 2024-09-20T22:12:42.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:42 smithi060 ceph-mon[29694]: 6.0 scrub starts 2024-09-20T22:12:42.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:42 smithi060 ceph-mon[29694]: 6.0 scrub ok 2024-09-20T22:12:42.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:42 smithi060 ceph-mon[29694]: pgmap v528: 158 pgs: 158 active+clean; 5.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 76 MiB/s wr, 1.22k op/s; 8.0 KiB/s, 0 objects/s recovering 2024-09-20T22:12:42.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:42 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:12:42.682 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:42 smithi060 ceph-mon[40290]: 9.4 scrub ok 2024-09-20T22:12:42.683 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:42 smithi060 ceph-mon[40290]: 6.1e scrub starts 2024-09-20T22:12:42.683 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:42 smithi060 ceph-mon[40290]: 6.1e scrub ok 2024-09-20T22:12:42.683 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:42 smithi060 ceph-mon[40290]: 6.0 scrub starts 2024-09-20T22:12:42.683 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:42 smithi060 ceph-mon[40290]: 6.0 scrub ok 2024-09-20T22:12:42.683 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:42 smithi060 ceph-mon[40290]: pgmap v528: 158 pgs: 158 active+clean; 5.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 76 MiB/s wr, 1.22k op/s; 8.0 KiB/s, 0 objects/s recovering 2024-09-20T22:12:42.683 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:42 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:12:42.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:42.739 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:42.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:42.740 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:42.841 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:42.877 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:42.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:42.878 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:42.901 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 55 16 87207 87191 99.0568 61.625 0.0236272 0.0100902 2024-09-20T22:12:42.979 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:42 smithi067 ceph-mon[33511]: 9.4 scrub ok 2024-09-20T22:12:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:42 smithi067 ceph-mon[33511]: 6.1e scrub starts 2024-09-20T22:12:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:42 smithi067 ceph-mon[33511]: 6.1e scrub ok 2024-09-20T22:12:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:42 smithi067 ceph-mon[33511]: 6.0 scrub starts 2024-09-20T22:12:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:42 smithi067 ceph-mon[33511]: 6.0 scrub ok 2024-09-20T22:12:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:42 smithi067 ceph-mon[33511]: pgmap v528: 158 pgs: 158 active+clean; 5.2 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 76 MiB/s wr, 1.22k op/s; 8.0 KiB/s, 0 objects/s recovering 2024-09-20T22:12:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:42 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:12:43.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:43.015 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:43.015 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:43.016 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:42 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a[29671]: 2024-09-20T22:12:42.913+0000 7f01adcc6640 -1 mon.a@0(leader).osd e317 definitely_dead 0 2024-09-20T22:12:43.116 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:43.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:43.156 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:43.156 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:43.257 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:43.292 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:43.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:43.293 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:43.394 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:43.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:43.428 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:43.428 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:43.529 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:43.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:43.567 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:43.567 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:43.611 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:12:43.616 INFO:teuthology.orchestra.run.smithi060.stderr:marked out osd.0. 2024-09-20T22:12:43.667 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:43.695 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:43.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:43.696 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:43.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[29694]: 5.6 scrub starts 2024-09-20T22:12:43.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[29694]: 5.6 scrub ok 2024-09-20T22:12:43.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[29694]: 3.2 scrub starts 2024-09-20T22:12:43.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[29694]: 6.1a scrub starts 2024-09-20T22:12:43.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[29694]: 3.2 scrub ok 2024-09-20T22:12:43.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[29694]: 6.1a scrub ok 2024-09-20T22:12:43.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[29694]: 1.2 scrub starts 2024-09-20T22:12:43.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[29694]: 1.2 scrub ok 2024-09-20T22:12:43.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3417811093' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:12:43.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:12:43.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/5024206' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-09-20T22:12:43.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-09-20T22:12:43.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[29694]: Client client.admin marked osd.0 out, while it was still marked up 2024-09-20T22:12:43.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[29694]: 1.7 scrub starts 2024-09-20T22:12:43.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[29694]: 1.7 scrub ok 2024-09-20T22:12:43.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[40290]: 5.6 scrub starts 2024-09-20T22:12:43.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[40290]: 5.6 scrub ok 2024-09-20T22:12:43.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[40290]: 3.2 scrub starts 2024-09-20T22:12:43.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[40290]: 6.1a scrub starts 2024-09-20T22:12:43.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[40290]: 3.2 scrub ok 2024-09-20T22:12:43.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[40290]: 6.1a scrub ok 2024-09-20T22:12:43.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[40290]: 1.2 scrub starts 2024-09-20T22:12:43.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[40290]: 1.2 scrub ok 2024-09-20T22:12:43.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3417811093' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:12:43.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:12:43.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/5024206' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-09-20T22:12:43.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-09-20T22:12:43.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[40290]: Client client.admin marked osd.0 out, while it was still marked up 2024-09-20T22:12:43.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[40290]: 1.7 scrub starts 2024-09-20T22:12:43.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:43 smithi060 ceph-mon[40290]: 1.7 scrub ok 2024-09-20T22:12:43.797 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:43.819 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:43.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:43.820 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:43.901 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 56 16 88045 88029 98.2232 52.375 0.00312495 0.0101308 2024-09-20T22:12:43.908 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:43 smithi067 ceph-mon[33511]: 5.6 scrub starts 2024-09-20T22:12:43.908 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:43 smithi067 ceph-mon[33511]: 5.6 scrub ok 2024-09-20T22:12:43.908 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:43 smithi067 ceph-mon[33511]: 3.2 scrub starts 2024-09-20T22:12:43.908 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:43 smithi067 ceph-mon[33511]: 6.1a scrub starts 2024-09-20T22:12:43.908 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:43 smithi067 ceph-mon[33511]: 3.2 scrub ok 2024-09-20T22:12:43.908 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:43 smithi067 ceph-mon[33511]: 6.1a scrub ok 2024-09-20T22:12:43.908 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:43 smithi067 ceph-mon[33511]: 1.2 scrub starts 2024-09-20T22:12:43.908 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:43 smithi067 ceph-mon[33511]: 1.2 scrub ok 2024-09-20T22:12:43.908 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:43 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3417811093' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:12:43.908 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:43 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:12:43.909 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:43 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/5024206' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-09-20T22:12:43.909 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:43 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["0"]}]: dispatch 2024-09-20T22:12:43.909 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:43 smithi067 ceph-mon[33511]: Client client.admin marked osd.0 out, while it was still marked up 2024-09-20T22:12:43.909 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:43 smithi067 ceph-mon[33511]: 1.7 scrub starts 2024-09-20T22:12:43.909 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:43 smithi067 ceph-mon[33511]: 1.7 scrub ok 2024-09-20T22:12:43.920 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:43.944 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:43.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:43.945 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:44.016 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:12:44.016 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:12:44.046 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:44.069 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:44.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:44.070 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:44.171 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:44.195 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:44.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:44.196 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:44.296 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:44.319 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:44.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:44.320 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:44.421 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:44.444 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:44.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:44.446 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:44.546 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:44.570 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:44.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:44.571 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:44.672 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:44.695 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:44.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:44.696 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:44.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:44 smithi067 ceph-mon[33511]: 3.8 scrub starts 2024-09-20T22:12:44.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:44 smithi067 ceph-mon[33511]: 3.8 scrub ok 2024-09-20T22:12:44.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:44 smithi067 ceph-mon[33511]: 6.14 scrub starts 2024-09-20T22:12:44.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:44 smithi067 ceph-mon[33511]: 6.14 scrub ok 2024-09-20T22:12:44.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:44 smithi067 ceph-mon[33511]: 4.15 scrub starts 2024-09-20T22:12:44.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:44 smithi067 ceph-mon[33511]: 4.15 scrub ok 2024-09-20T22:12:44.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:44 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:12:44.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:44 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:12:44.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:44 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:12:44.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:44 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["0"]}]': finished 2024-09-20T22:12:44.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:44 smithi067 ceph-mon[33511]: osdmap e318: 8 total, 8 up, 7 in 2024-09-20T22:12:44.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:44 smithi067 ceph-mon[33511]: pgmap v530: 158 pgs: 158 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 671 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-20T22:12:44.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:44 smithi067 ceph-mon[33511]: osdmap e319: 8 total, 8 up, 7 in 2024-09-20T22:12:44.796 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:44.820 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:44.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:44.821 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:44.856 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T22:12:44.901 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 57 16 88045 88029 96.5001 0 - 0.0101308 2024-09-20T22:12:44.921 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:44.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[29694]: 3.8 scrub starts 2024-09-20T22:12:44.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[29694]: 3.8 scrub ok 2024-09-20T22:12:44.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[29694]: 6.14 scrub starts 2024-09-20T22:12:44.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[29694]: 6.14 scrub ok 2024-09-20T22:12:44.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[29694]: 4.15 scrub starts 2024-09-20T22:12:44.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[29694]: 4.15 scrub ok 2024-09-20T22:12:44.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:12:44.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:12:44.943 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:12:44.943 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["0"]}]': finished 2024-09-20T22:12:44.943 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[29694]: osdmap e318: 8 total, 8 up, 7 in 2024-09-20T22:12:44.943 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[29694]: pgmap v530: 158 pgs: 158 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 671 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-20T22:12:44.943 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[29694]: osdmap e319: 8 total, 8 up, 7 in 2024-09-20T22:12:44.943 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[40290]: 3.8 scrub starts 2024-09-20T22:12:44.943 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[40290]: 3.8 scrub ok 2024-09-20T22:12:44.943 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[40290]: 6.14 scrub starts 2024-09-20T22:12:44.943 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[40290]: 6.14 scrub ok 2024-09-20T22:12:44.943 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[40290]: 4.15 scrub starts 2024-09-20T22:12:44.943 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[40290]: 4.15 scrub ok 2024-09-20T22:12:44.943 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:12:44.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:12:44.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:12:44.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["0"]}]': finished 2024-09-20T22:12:44.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[40290]: osdmap e318: 8 total, 8 up, 7 in 2024-09-20T22:12:44.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[40290]: pgmap v530: 158 pgs: 158 active+clean; 5.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 671 B/s rd, 66 MiB/s wr, 1.05k op/s 2024-09-20T22:12:44.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:44 smithi060 ceph-mon[40290]: osdmap e319: 8 total, 8 up, 7 in 2024-09-20T22:12:44.945 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:44.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:44.946 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:45.047 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:45.070 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:45.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:45.071 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:45.110 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:12:45.172 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:45.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:45.195 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:45.196 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:45.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:12:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:12:45.297 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:45.319 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:45.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:45.320 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:45.421 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:45.443 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:45.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:45.444 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:45.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:12:45.545 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:45.574 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:45.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:45.575 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:45.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:45.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:45.708 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:45.708 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:45.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-mon[40290]: 3.1a scrub starts 2024-09-20T22:12:45.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-mon[40290]: 3.1a scrub ok 2024-09-20T22:12:45.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-mon[40290]: 5.d scrub starts 2024-09-20T22:12:45.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-mon[40290]: 5.d scrub ok 2024-09-20T22:12:45.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-mon[40290]: 3.11 scrub starts 2024-09-20T22:12:45.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-mon[40290]: 3.11 scrub ok 2024-09-20T22:12:45.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-mon[40290]: 6.11 scrub starts 2024-09-20T22:12:45.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-mon[40290]: 6.11 scrub ok 2024-09-20T22:12:45.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-mon[40290]: osdmap e320: 8 total, 8 up, 7 in 2024-09-20T22:12:45.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-mon[29694]: 3.1a scrub starts 2024-09-20T22:12:45.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-mon[29694]: 3.1a scrub ok 2024-09-20T22:12:45.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-mon[29694]: 5.d scrub starts 2024-09-20T22:12:45.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-mon[29694]: 5.d scrub ok 2024-09-20T22:12:45.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-mon[29694]: 3.11 scrub starts 2024-09-20T22:12:45.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-mon[29694]: 3.11 scrub ok 2024-09-20T22:12:45.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-mon[29694]: 6.11 scrub starts 2024-09-20T22:12:45.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-mon[29694]: 6.11 scrub ok 2024-09-20T22:12:45.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:45 smithi060 ceph-mon[29694]: osdmap e320: 8 total, 8 up, 7 in 2024-09-20T22:12:45.788 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:12:45.788 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:12:45.809 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:45.868 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:45.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:45.870 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:45.896 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:45.901 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 58 16 88218 88202 95.023 5.40625 0.00567301 0.010488 2024-09-20T22:12:45.971 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:46.001 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:46.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:46.002 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:45 smithi067 ceph-mon[33511]: 3.1a scrub starts 2024-09-20T22:12:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:45 smithi067 ceph-mon[33511]: 3.1a scrub ok 2024-09-20T22:12:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:45 smithi067 ceph-mon[33511]: 5.d scrub starts 2024-09-20T22:12:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:45 smithi067 ceph-mon[33511]: 5.d scrub ok 2024-09-20T22:12:46.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:45 smithi067 ceph-mon[33511]: 3.11 scrub starts 2024-09-20T22:12:46.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:45 smithi067 ceph-mon[33511]: 3.11 scrub ok 2024-09-20T22:12:46.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:45 smithi067 ceph-mon[33511]: 6.11 scrub starts 2024-09-20T22:12:46.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:45 smithi067 ceph-mon[33511]: 6.11 scrub ok 2024-09-20T22:12:46.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:45 smithi067 ceph-mon[33511]: osdmap e320: 8 total, 8 up, 7 in 2024-09-20T22:12:46.102 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:46.129 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:46.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:46.130 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:46.231 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:46.256 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:46.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:46.257 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:46.284 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:12:46.357 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:46.381 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:46.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:46.382 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:46.430 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:12:46.430 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:12:46.450 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:12:46.450 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:12:46.469 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:12:46.469 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:12:46.482 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:46.490 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:12:46.490 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:12:46.518 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:46.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:46.519 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:46.523 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:12:46.523 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:12:46.543 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:12:46.544 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:12:46.585 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:46.620 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:46.644 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:46.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:46.645 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:46.746 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:46.774 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:46.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:46.775 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:46.783 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T22:12:46.876 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:46.889 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:46 smithi060 ceph-mon[29694]: 2.1 scrub starts 2024-09-20T22:12:46.890 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:46 smithi060 ceph-mon[29694]: 2.1 scrub ok 2024-09-20T22:12:46.890 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:46 smithi060 ceph-mon[29694]: 9.8 starting backfill to osd.7 from (0'0,0'0] MAX to 317'5569 2024-09-20T22:12:46.890 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:46 smithi060 ceph-mon[29694]: 9.5 starting backfill to osd.1 from (0'0,0'0] MAX to 317'5467 2024-09-20T22:12:46.890 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:46 smithi060 ceph-mon[29694]: 9.9 starting backfill to osd.3 from (0'0,0'0] MAX to 317'5647 2024-09-20T22:12:46.890 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:46 smithi060 ceph-mon[29694]: pgmap v533: 158 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait+degraded+remapped, 3 active+undersized+remapped, 1 active+remapped+backfill_toofull, 152 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 682 B/s rd, 80 MiB/s wr, 1.28k op/s; 1/263685 objects degraded (0.000%); 16697/263685 objects misplaced (6.332%); 150 KiB/s, 6 objects/s recovering 2024-09-20T22:12:46.890 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:46 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/117321882' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:46.890 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:46 smithi060 ceph-mon[29694]: osdmap e321: 8 total, 8 up, 7 in 2024-09-20T22:12:46.890 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:46 smithi060 ceph-mon[40290]: 2.1 scrub starts 2024-09-20T22:12:46.890 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:46 smithi060 ceph-mon[40290]: 2.1 scrub ok 2024-09-20T22:12:46.891 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:46 smithi060 ceph-mon[40290]: 9.8 starting backfill to osd.7 from (0'0,0'0] MAX to 317'5569 2024-09-20T22:12:46.891 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:46 smithi060 ceph-mon[40290]: 9.5 starting backfill to osd.1 from (0'0,0'0] MAX to 317'5467 2024-09-20T22:12:46.891 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:46 smithi060 ceph-mon[40290]: 9.9 starting backfill to osd.3 from (0'0,0'0] MAX to 317'5647 2024-09-20T22:12:46.891 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:46 smithi060 ceph-mon[40290]: pgmap v533: 158 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait+degraded+remapped, 3 active+undersized+remapped, 1 active+remapped+backfill_toofull, 152 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 682 B/s rd, 80 MiB/s wr, 1.28k op/s; 1/263685 objects degraded (0.000%); 16697/263685 objects misplaced (6.332%); 150 KiB/s, 6 objects/s recovering 2024-09-20T22:12:46.891 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:46 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/117321882' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:46.891 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:46 smithi060 ceph-mon[40290]: osdmap e321: 8 total, 8 up, 7 in 2024-09-20T22:12:46.901 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:46.901 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 59 16 88458 88442 93.6668 15 0.00401334 0.0104764 2024-09-20T22:12:46.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:46.902 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:47.003 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:46 smithi067 ceph-mon[33511]: 2.1 scrub starts 2024-09-20T22:12:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:46 smithi067 ceph-mon[33511]: 2.1 scrub ok 2024-09-20T22:12:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:46 smithi067 ceph-mon[33511]: 9.8 starting backfill to osd.7 from (0'0,0'0] MAX to 317'5569 2024-09-20T22:12:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:46 smithi067 ceph-mon[33511]: 9.5 starting backfill to osd.1 from (0'0,0'0] MAX to 317'5467 2024-09-20T22:12:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:46 smithi067 ceph-mon[33511]: 9.9 starting backfill to osd.3 from (0'0,0'0] MAX to 317'5647 2024-09-20T22:12:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:46 smithi067 ceph-mon[33511]: pgmap v533: 158 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait+degraded+remapped, 3 active+undersized+remapped, 1 active+remapped+backfill_toofull, 152 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 682 B/s rd, 80 MiB/s wr, 1.28k op/s; 1/263685 objects degraded (0.000%); 16697/263685 objects misplaced (6.332%); 150 KiB/s, 6 objects/s recovering 2024-09-20T22:12:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:46 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/117321882' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:12:47.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:46 smithi067 ceph-mon[33511]: osdmap e321: 8 total, 8 up, 7 in 2024-09-20T22:12:47.025 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:47.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:47.026 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:47.104 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:12:47.127 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:47.150 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:47.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:47.151 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:47.254 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:47.277 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:47.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:47.278 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:47.379 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:47.402 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:47.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:47.403 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:47.493 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:12:47.504 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:47.530 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:47.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:47.531 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:47.632 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:47.659 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:47.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:47.660 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:47.710 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:12:47.710 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:12:47.761 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:47.784 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:47.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:47.786 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:47.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[29694]: 5.8 scrub starts 2024-09-20T22:12:47.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[29694]: 5.8 scrub ok 2024-09-20T22:12:47.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[29694]: 3.d deep-scrub starts 2024-09-20T22:12:47.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[29694]: 3.d deep-scrub ok 2024-09-20T22:12:47.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[29694]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:12:47.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[29694]: Health check failed: Degraded data redundancy: 1/263685 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-20T22:12:47.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/736399774' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:12:47.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:12:47.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/736399774' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:12:47.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[29694]: osdmap e322: 8 total, 8 up, 7 in 2024-09-20T22:12:47.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[40290]: 5.8 scrub starts 2024-09-20T22:12:47.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[40290]: 5.8 scrub ok 2024-09-20T22:12:47.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[40290]: 3.d deep-scrub starts 2024-09-20T22:12:47.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[40290]: 3.d deep-scrub ok 2024-09-20T22:12:47.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[40290]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:12:47.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[40290]: Health check failed: Degraded data redundancy: 1/263685 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-20T22:12:47.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/736399774' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:12:47.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:12:47.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/736399774' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:12:47.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:47 smithi060 ceph-mon[40290]: osdmap e322: 8 total, 8 up, 7 in 2024-09-20T22:12:47.887 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:47.901 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:12:47.901313+0000 min lat: 0.0012684 max lat: 2.11457 avg lat: 0.0104764 2024-09-20T22:12:47.901 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:12:47.901 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 60 16 88458 88442 92.1059 0 - 0.0104764 2024-09-20T22:12:47.909 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:47.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:47.910 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:48.011 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:47 smithi067 ceph-mon[33511]: 5.8 scrub starts 2024-09-20T22:12:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:47 smithi067 ceph-mon[33511]: 5.8 scrub ok 2024-09-20T22:12:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:47 smithi067 ceph-mon[33511]: 3.d deep-scrub starts 2024-09-20T22:12:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:47 smithi067 ceph-mon[33511]: 3.d deep-scrub ok 2024-09-20T22:12:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:47 smithi067 ceph-mon[33511]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:12:48.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:47 smithi067 ceph-mon[33511]: Health check failed: Degraded data redundancy: 1/263685 objects degraded (0.000%), 1 pg degraded (PG_DEGRADED) 2024-09-20T22:12:48.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:47 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/736399774' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:12:48.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:47 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:12:48.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:47 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/736399774' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:12:48.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:47 smithi067 ceph-mon[33511]: osdmap e322: 8 total, 8 up, 7 in 2024-09-20T22:12:48.033 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:48.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:48.034 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:48.135 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:48.160 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:48.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:48.161 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:48.262 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:48.286 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:48.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:48.287 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:48.388 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:48.411 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:48.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:48.412 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:48.512 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:48.534 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T22:12:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:48.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:48.536 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:48.637 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:48.659 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:48.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:48.660 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:48.761 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:48.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:48 smithi067 ceph-mon[33511]: 4.e scrub starts 2024-09-20T22:12:48.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:48 smithi067 ceph-mon[33511]: 4.e scrub ok 2024-09-20T22:12:48.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:48 smithi067 ceph-mon[33511]: pgmap v536: 158 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait+degraded+remapped, 3 active+undersized+remapped, 1 active+remapped+backfill_toofull, 152 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 33 MiB/s wr, 521 op/s; 1/264228 objects degraded (0.000%); 16697/264228 objects misplaced (6.319%); 224 KiB/s, 15 objects/s recovering 2024-09-20T22:12:48.788 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:48.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:48.789 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:48.817 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:12:48.890 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:48.901 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 61 16 88458 88442 90.5962 0 - 0.0104764 2024-09-20T22:12:48.915 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:48.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:48.916 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:49.017 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:49.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:48 smithi060 ceph-mon[29694]: 4.e scrub starts 2024-09-20T22:12:49.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:48 smithi060 ceph-mon[29694]: 4.e scrub ok 2024-09-20T22:12:49.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:48 smithi060 ceph-mon[29694]: pgmap v536: 158 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait+degraded+remapped, 3 active+undersized+remapped, 1 active+remapped+backfill_toofull, 152 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 33 MiB/s wr, 521 op/s; 1/264228 objects degraded (0.000%); 16697/264228 objects misplaced (6.319%); 224 KiB/s, 15 objects/s recovering 2024-09-20T22:12:49.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:48 smithi060 ceph-mon[40290]: 4.e scrub starts 2024-09-20T22:12:49.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:48 smithi060 ceph-mon[40290]: 4.e scrub ok 2024-09-20T22:12:49.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:48 smithi060 ceph-mon[40290]: pgmap v536: 158 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait+degraded+remapped, 3 active+undersized+remapped, 1 active+remapped+backfill_toofull, 152 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 33 MiB/s wr, 521 op/s; 1/264228 objects degraded (0.000%); 16697/264228 objects misplaced (6.319%); 224 KiB/s, 15 objects/s recovering 2024-09-20T22:12:49.040 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:49.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:49.041 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:49.142 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:49.169 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:49.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:49.169 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:49.270 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:49.294 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:49.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:49.296 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:49.397 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:49.407 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:12:49.407 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:12:49.422 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:49.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:49.423 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:49.524 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:49.550 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:49.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:49.551 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:49.651 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:49.675 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:49.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:49.676 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:49.777 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:49.801 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:49.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:49.802 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:49.901 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 62 16 88458 88442 89.1352 0 - 0.0104764 2024-09-20T22:12:49.903 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:49.925 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:49.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:49.926 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:49.972 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:12:49.972 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:12:49.972 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-09-20T22:12:49.972 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd in 0 2024-09-20T22:12:50.027 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:50.050 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:50.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:50.051 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:50.152 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:50.155 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T22:12:50.177 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:50.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:50.178 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:50.278 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:50.303 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:50.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:50.304 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:50.405 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:50.432 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:50.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:50.433 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:50.472 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:12:50.534 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:50.557 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:50.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:50.559 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:50.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:50.683 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:50.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:50.683 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:50.784 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:50.808 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:50.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:50.809 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:50.902 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 63 16 88458 88442 87.7205 0 - 0.0104764 2024-09-20T22:12:50.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:50.933 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:50.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:50.934 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:51.035 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:51.058 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:51.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:51.059 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:51.160 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:51.170 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:12:51.170 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:12:51.183 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:51.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:51.184 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:51.214 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:12:51.247 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:50 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a[29671]: 2024-09-20T22:12:50.971+0000 7f01adcc6640 -1 mon.a@0(leader).osd e322 definitely_dead 0 2024-09-20T22:12:51.247 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:51 smithi060 ceph-mon[29694]: pgmap v537: 158 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait+degraded+remapped, 3 active+undersized+remapped, 1 active+remapped+backfill_toofull, 152 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 23 MiB/s wr, 375 op/s; 1/264228 objects degraded (0.000%); 16697/264228 objects misplaced (6.319%); 162 KiB/s, 10 objects/s recovering 2024-09-20T22:12:51.247 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:12:51.247 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:12:51.247 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:51 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2126963000' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-09-20T22:12:51.248 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:51 smithi060 ceph-mon[40290]: pgmap v537: 158 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait+degraded+remapped, 3 active+undersized+remapped, 1 active+remapped+backfill_toofull, 152 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 23 MiB/s wr, 375 op/s; 1/264228 objects degraded (0.000%); 16697/264228 objects misplaced (6.319%); 162 KiB/s, 10 objects/s recovering 2024-09-20T22:12:51.248 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:12:51.248 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:12:51.248 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:51 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2126963000' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-09-20T22:12:51.284 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:51.308 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:51.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:51.309 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:51.410 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:51.436 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:51.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:51.437 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:51.494 INFO:teuthology.orchestra.run.smithi060.stderr:marked in osd.0. 2024-09-20T22:12:51.511 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:51 smithi067 ceph-mon[33511]: pgmap v537: 158 pgs: 1 active+remapped+backfilling, 1 active+recovery_wait+degraded+remapped, 3 active+undersized+remapped, 1 active+remapped+backfill_toofull, 152 active+clean; 5.4 GiB data, 18 GiB used, 608 GiB / 626 GiB avail; 23 MiB/s wr, 375 op/s; 1/264228 objects degraded (0.000%); 16697/264228 objects misplaced (6.319%); 162 KiB/s, 10 objects/s recovering 2024-09-20T22:12:51.511 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:12:51.511 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:12:51.511 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:51 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2126963000' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["0"]}]: dispatch 2024-09-20T22:12:51.521 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:51.538 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:51.561 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:51.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:51.562 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:51.662 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:51.685 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:51.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:51.686 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:51.786 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:51.809 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:51.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:51.810 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:51.902 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 64 16 88458 88442 86.35 0 - 0.0104764 2024-09-20T22:12:51.911 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:51.934 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:51.934 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:51.935 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:51.959 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T22:12:52.035 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:52.058 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:52.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:52.059 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:52.160 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:52.182 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:52.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:52.183 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:52.284 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:52.292 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-09-20T22:12:52.307 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:52.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:52.308 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:52.308 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:52.409 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:52.431 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:52.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:52.432 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:52.532 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:52.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:52 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2126963000' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["0"]}]': finished 2024-09-20T22:12:52.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:52 smithi060 ceph-mon[40290]: osdmap e323: 8 total, 8 up, 8 in 2024-09-20T22:12:52.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:52 smithi060 ceph-mon[40290]: pgmap v539: 158 pgs: 23 peering, 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 132 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 511 B/s rd, 4.6 MiB/s wr, 73 op/s; 13802/264993 objects misplaced (5.208%); 31 MiB/s, 491 objects/s recovering 2024-09-20T22:12:52.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:52 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:12:52.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:52 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2126963000' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["0"]}]': finished 2024-09-20T22:12:52.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:52 smithi060 ceph-mon[29694]: osdmap e323: 8 total, 8 up, 8 in 2024-09-20T22:12:52.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:52 smithi060 ceph-mon[29694]: pgmap v539: 158 pgs: 23 peering, 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 132 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 511 B/s rd, 4.6 MiB/s wr, 73 op/s; 13802/264993 objects misplaced (5.208%); 31 MiB/s, 491 objects/s recovering 2024-09-20T22:12:52.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:52 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:12:52.561 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:52.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:52.562 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:52.662 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:52.684 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:52.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:52.685 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:52.763 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:12:52 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:12:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:52 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2126963000' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["0"]}]': finished 2024-09-20T22:12:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:52 smithi067 ceph-mon[33511]: osdmap e323: 8 total, 8 up, 8 in 2024-09-20T22:12:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:52 smithi067 ceph-mon[33511]: pgmap v539: 158 pgs: 23 peering, 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 132 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 511 B/s rd, 4.6 MiB/s wr, 73 op/s; 13802/264993 objects misplaced (5.208%); 31 MiB/s, 491 objects/s recovering 2024-09-20T22:12:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:52 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:12:52.786 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:52.808 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:52.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:52.809 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:52.902 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 65 16 88458 88442 85.0217 0 - 0.0104764 2024-09-20T22:12:52.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:52.933 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:52.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:52.934 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:53.035 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:53.056 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:53.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:53.057 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:53.121 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:12:53.121 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:12:53.121 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-20T22:12:53.121 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:12:53.158 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:53.181 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:53.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:53.182 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:53.282 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:53.304 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:53.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:53.306 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:53.406 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:53.428 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:53.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:53.429 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:53.499 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:12:53.530 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:53.556 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:53.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:53.557 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:53.657 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:53.679 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:53.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:53.680 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:53 smithi067 ceph-mon[33511]: Health check failed: Reduced data availability: 10 pgs inactive, 13 pgs peering (PG_AVAILABILITY) 2024-09-20T22:12:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:53 smithi067 ceph-mon[33511]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:12:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:53 smithi067 ceph-mon[33511]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/264228 objects degraded (0.000%), 1 pg degraded) 2024-09-20T22:12:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:53 smithi067 ceph-mon[33511]: osdmap e324: 8 total, 8 up, 8 in 2024-09-20T22:12:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:53 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3713272456' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:12:53.782 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:53.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:53 smithi060 ceph-mon[29694]: Health check failed: Reduced data availability: 10 pgs inactive, 13 pgs peering (PG_AVAILABILITY) 2024-09-20T22:12:53.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:53 smithi060 ceph-mon[29694]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:12:53.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:53 smithi060 ceph-mon[29694]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/264228 objects degraded (0.000%), 1 pg degraded) 2024-09-20T22:12:53.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:53 smithi060 ceph-mon[29694]: osdmap e324: 8 total, 8 up, 8 in 2024-09-20T22:12:53.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:53 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3713272456' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:12:53.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:53 smithi060 ceph-mon[40290]: Health check failed: Reduced data availability: 10 pgs inactive, 13 pgs peering (PG_AVAILABILITY) 2024-09-20T22:12:53.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:53 smithi060 ceph-mon[40290]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:12:53.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:53 smithi060 ceph-mon[40290]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/264228 objects degraded (0.000%), 1 pg degraded) 2024-09-20T22:12:53.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:53 smithi060 ceph-mon[40290]: osdmap e324: 8 total, 8 up, 8 in 2024-09-20T22:12:53.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:53 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3713272456' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:12:53.804 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:53.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:53.805 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:53.843 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T22:12:53.902 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 66 16 88458 88442 83.7333 0 - 0.0104764 2024-09-20T22:12:53.905 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:53.927 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:53.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:53.928 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:54.029 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:54.051 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:54.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:54.052 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:54.152 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:54.175 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:54.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:54.176 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:54.277 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:54.299 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:54.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:54.300 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:54.401 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:54.424 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:54.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:54.425 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:54.525 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:54.547 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:54.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:54.548 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:54.649 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:54.666 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:54.673 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:54.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:54.674 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:54 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:12:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:54 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3713272456' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:12:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:54 smithi067 ceph-mon[33511]: osdmap e325: 8 total, 8 up, 8 in 2024-09-20T22:12:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:54 smithi067 ceph-mon[33511]: pgmap v542: 158 pgs: 23 peering, 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 132 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 2.7 MiB/s wr, 43 op/s; 13802/264993 objects misplaced (5.208%); 31 MiB/s, 488 objects/s recovering 2024-09-20T22:12:54.775 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:54.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:54 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:12:54.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:54 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3713272456' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:12:54.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:54 smithi060 ceph-mon[29694]: osdmap e325: 8 total, 8 up, 8 in 2024-09-20T22:12:54.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:54 smithi060 ceph-mon[29694]: pgmap v542: 158 pgs: 23 peering, 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 132 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 2.7 MiB/s wr, 43 op/s; 13802/264993 objects misplaced (5.208%); 31 MiB/s, 488 objects/s recovering 2024-09-20T22:12:54.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:54 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:12:54.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:54 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3713272456' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:12:54.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:54 smithi060 ceph-mon[40290]: osdmap e325: 8 total, 8 up, 8 in 2024-09-20T22:12:54.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:54 smithi060 ceph-mon[40290]: pgmap v542: 158 pgs: 23 peering, 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 132 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 2.7 MiB/s wr, 43 op/s; 13802/264993 objects misplaced (5.208%); 31 MiB/s, 488 objects/s recovering 2024-09-20T22:12:54.796 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:54.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:54.797 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:54.898 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:54.902 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 67 16 88458 88442 82.4837 0 - 0.0104764 2024-09-20T22:12:54.921 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:54.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:54.922 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:55.022 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:55.044 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:55.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:55.045 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:55.141 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:12:55.141 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:12:55.142 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:12:55.142 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:12:55.142 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:12:55.146 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:55.169 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:55.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:55.170 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:55.271 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:55.293 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:55.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:55.294 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:55.395 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:55.417 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:55.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:55.418 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:55.507 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:12:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:12:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:12:55.519 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:55.542 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:55.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:55.543 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:55.644 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:55.666 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:55.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:55.667 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:55.768 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:55.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:55 smithi060 ceph-mon[29694]: osdmap e326: 8 total, 8 up, 8 in 2024-09-20T22:12:55.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:55 smithi060 ceph-mon[40290]: osdmap e326: 8 total, 8 up, 8 in 2024-09-20T22:12:55.790 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:55.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:55.791 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:55.836 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T22:12:55.892 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:55.902 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 68 16 88458 88442 81.2709 0 - 0.0104764 2024-09-20T22:12:55.915 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:55.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:55.916 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:55 smithi067 ceph-mon[33511]: osdmap e326: 8 total, 8 up, 8 in 2024-09-20T22:12:56.017 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:56.040 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:56.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:56.041 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:56.091 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:56.142 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:56.164 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:56.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:56.165 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:56.266 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:56.288 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:56.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:56.289 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:56.389 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:56.411 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:56.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:56.413 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:56.513 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:56.537 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:56.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:56.538 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:56.605 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:12:56.605 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:12:56.605 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:12:56.605 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:12:56.605 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.605 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227635 7.b 7.fcebd15b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:12:56.606 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.849145+0000", 2024-09-20T22:12:56.606 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 362.733486648, 2024-09-20T22:12:56.606 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.958058509, 2024-09-20T22:12:56.606 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.606 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.606 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:12:56.606 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:56.606 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:12:56.606 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:12:56.606 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227635 2024-09-20T22:12:56.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.606 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.607 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.607 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849145+0000", 2024-09-20T22:12:56.607 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.607 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.607 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849145+0000", 2024-09-20T22:12:56.607 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.607 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.607 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849146+0000", 2024-09-20T22:12:56.608 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.562e-06 2024-09-20T22:12:56.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.608 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.608 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.608 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849152+0000", 2024-09-20T22:12:56.608 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5999999999999997e-06 2024-09-20T22:12:56.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.608 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.608 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.608 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849162+0000", 2024-09-20T22:12:56.608 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9620000000000005e-06 2024-09-20T22:12:56.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.608 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.608 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.608 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849171+0000", 2024-09-20T22:12:56.609 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4569999999999994e-06 2024-09-20T22:12:56.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.609 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.609 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849336+0000", 2024-09-20T22:12:56.609 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016478700000000001 2024-09-20T22:12:56.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.609 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.609 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849344+0000", 2024-09-20T22:12:56.609 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3950000000000001e-06 2024-09-20T22:12:56.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.609 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.610 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.807192+0000", 2024-09-20T22:12:56.610 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.95784742599999995 2024-09-20T22:12:56.610 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.610 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.610 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.610 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.807203+0000", 2024-09-20T22:12:56.610 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.132e-05 2024-09-20T22:12:56.610 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.610 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.610 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.610 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.610 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.611 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227641 7.b 7.8d46094b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:12:56.611 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.858720+0000", 2024-09-20T22:12:56.611 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 362.72391116, 2024-09-20T22:12:56.611 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.95058797399999995, 2024-09-20T22:12:56.611 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.611 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.611 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:12:56.611 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:56.611 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:12:56.611 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:12:56.611 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227641 2024-09-20T22:12:56.611 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.611 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.611 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.612 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.612 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858720+0000", 2024-09-20T22:12:56.612 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.612 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.612 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.612 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858720+0000", 2024-09-20T22:12:56.612 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.612 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.612 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.612 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858722+0000", 2024-09-20T22:12:56.612 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3969999999999999e-06 2024-09-20T22:12:56.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.613 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.613 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.613 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858726+0000", 2024-09-20T22:12:56.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7999999999999998e-06 2024-09-20T22:12:56.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.613 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.613 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.613 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858737+0000", 2024-09-20T22:12:56.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0403e-05 2024-09-20T22:12:56.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.613 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.613 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.613 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858746+0000", 2024-09-20T22:12:56.614 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2739999999999996e-06 2024-09-20T22:12:56.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.614 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.614 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.614 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858806+0000", 2024-09-20T22:12:56.614 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0315000000000001e-05 2024-09-20T22:12:56.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.614 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.614 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.614 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858818+0000", 2024-09-20T22:12:56.614 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2079000000000001e-05 2024-09-20T22:12:56.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.614 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.615 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.615 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.809293+0000", 2024-09-20T22:12:56.615 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.95047507899999994 2024-09-20T22:12:56.615 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.615 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.615 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.615 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.809308+0000", 2024-09-20T22:12:56.615 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4627000000000001e-05 2024-09-20T22:12:56.615 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.615 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.615 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.615 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.615 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.615 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105654 8.1 8:83678158:::benchmark_data_smithi060_111051_object105653:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e214)", 2024-09-20T22:12:56.616 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.527812+0000", 2024-09-20T22:12:56.616 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 228.05481929999999, 2024-09-20T22:12:56.616 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.17380899499999999, 2024-09-20T22:12:56.616 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.616 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.616 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:12:56.616 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:56.616 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:56.616 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:56.616 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105654 2024-09-20T22:12:56.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.616 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.616 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.616 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.527812+0000", 2024-09-20T22:12:56.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.617 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.617 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.527812+0000", 2024-09-20T22:12:56.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.617 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.617 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.527814+0000", 2024-09-20T22:12:56.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.564e-06 2024-09-20T22:12:56.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.617 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.617 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.618 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.527839+0000", 2024-09-20T22:12:56.618 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5341999999999999e-05 2024-09-20T22:12:56.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.618 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.618 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.618 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.527853+0000", 2024-09-20T22:12:56.618 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3719e-05 2024-09-20T22:12:56.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.618 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.618 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.618 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.527860+0000", 2024-09-20T22:12:56.618 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1890000000000001e-06 2024-09-20T22:12:56.619 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.619 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.619 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.619 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.529439+0000", 2024-09-20T22:12:56.619 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001579425 2024-09-20T22:12:56.619 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.619 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.619 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:56.619 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.529561+0000", 2024-09-20T22:12:56.619 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012155300000000001 2024-09-20T22:12:56.619 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.619 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.619 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:12:56.619 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.529705+0000", 2024-09-20T22:12:56.620 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014421900000000001 2024-09-20T22:12:56.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.620 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.620 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.701564+0000", 2024-09-20T22:12:56.620 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.171859179 2024-09-20T22:12:56.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.620 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.620 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.701621+0000", 2024-09-20T22:12:56.620 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6805000000000002e-05 2024-09-20T22:12:56.620 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.620 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.621 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.621 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.621 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105657 8.7 8.6bf50607 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e214)", 2024-09-20T22:12:56.621 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.533270+0000", 2024-09-20T22:12:56.621 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 228.049361081, 2024-09-20T22:12:56.621 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.18481678800000001, 2024-09-20T22:12:56.621 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.621 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.621 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:12:56.621 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:56.621 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:56.621 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:56.621 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105657 2024-09-20T22:12:56.622 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.622 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.622 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.622 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.622 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533270+0000", 2024-09-20T22:12:56.622 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.622 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.622 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.622 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.622 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533270+0000", 2024-09-20T22:12:56.622 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.622 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.622 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.623 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.623 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533272+0000", 2024-09-20T22:12:56.623 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.751e-06 2024-09-20T22:12:56.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.623 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.623 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.623 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533297+0000", 2024-09-20T22:12:56.623 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5105000000000001e-05 2024-09-20T22:12:56.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.623 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.623 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.623 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533424+0000", 2024-09-20T22:12:56.623 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000127181 2024-09-20T22:12:56.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.624 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.624 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.624 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533434+0000", 2024-09-20T22:12:56.624 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4229999999999994e-06 2024-09-20T22:12:56.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.624 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.624 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.624 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.718074+0000", 2024-09-20T22:12:56.624 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.18464032699999999 2024-09-20T22:12:56.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.624 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.624 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.624 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.718087+0000", 2024-09-20T22:12:56.624 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3001e-05 2024-09-20T22:12:56.625 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.625 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.625 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.625 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.15192.0:105656 8.b e214/176 8:dc90bde5:::benchmark_data_smithi060_111051_object105655:head v 214'6463, mlcod=214'6461)", 2024-09-20T22:12:56.625 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.533332+0000", 2024-09-20T22:12:56.625 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 228.049298814, 2024-09-20T22:12:56.625 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.16766616300000001, 2024-09-20T22:12:56.625 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.625 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.625 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:56.625 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.626 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.626 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533332+0000", 2024-09-20T22:12:56.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.626 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.626 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533332+0000", 2024-09-20T22:12:56.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.626 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.626 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533334+0000", 2024-09-20T22:12:56.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.012e-06 2024-09-20T22:12:56.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.627 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533357+0000", 2024-09-20T22:12:56.627 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3448999999999999e-05 2024-09-20T22:12:56.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.627 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.535888+0000", 2024-09-20T22:12:56.627 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025313219999999999 2024-09-20T22:12:56.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.627 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.535901+0000", 2024-09-20T22:12:56.628 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2496e-05 2024-09-20T22:12:56.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.628 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.628 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542308+0000", 2024-09-20T22:12:56.628 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0064067480000000003 2024-09-20T22:12:56.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.628 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:56.628 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542330+0000", 2024-09-20T22:12:56.628 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2860000000000001e-05 2024-09-20T22:12:56.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.629 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:56.629 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.700965+0000", 2024-09-20T22:12:56.629 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15863476900000001 2024-09-20T22:12:56.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.629 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.629 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.629 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.700999+0000", 2024-09-20T22:12:56.629 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3507000000000003e-05 2024-09-20T22:12:56.629 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.629 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.629 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.629 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.629 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105658 8.1 8.93657901 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e214)", 2024-09-20T22:12:56.630 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.533463+0000", 2024-09-20T22:12:56.630 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 228.049168181, 2024-09-20T22:12:56.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.168771484, 2024-09-20T22:12:56.630 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.630 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.630 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:12:56.630 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:56.630 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:56.630 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:56.630 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105658 2024-09-20T22:12:56.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.630 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533463+0000", 2024-09-20T22:12:56.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533463+0000", 2024-09-20T22:12:56.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533464+0000", 2024-09-20T22:12:56.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2600000000000002e-07 2024-09-20T22:12:56.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533489+0000", 2024-09-20T22:12:56.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4811e-05 2024-09-20T22:12:56.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533498+0000", 2024-09-20T22:12:56.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7019999999999996e-06 2024-09-20T22:12:56.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.533505+0000", 2024-09-20T22:12:56.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2369999999999997e-06 2024-09-20T22:12:56.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.633 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.633 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.633 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702223+0000", 2024-09-20T22:12:56.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.16871866599999999 2024-09-20T22:12:56.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.633 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.633 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.633 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702235+0000", 2024-09-20T22:12:56.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1342000000000001e-05 2024-09-20T22:12:56.633 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.633 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.633 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.634 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.634 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.15192.0:105653 8.f e214/176 8:f20d820f:::benchmark_data_smithi060_111051_object105652:head v 214'6689, mlcod=214'6685)", 2024-09-20T22:12:56.634 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.542169+0000", 2024-09-20T22:12:56.634 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 228.04046190099999, 2024-09-20T22:12:56.634 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.186805894, 2024-09-20T22:12:56.634 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.634 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.634 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:56.634 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.634 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.634 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.634 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542169+0000", 2024-09-20T22:12:56.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.635 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.635 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542169+0000", 2024-09-20T22:12:56.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.635 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.635 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542170+0000", 2024-09-20T22:12:56.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7800000000000001e-07 2024-09-20T22:12:56.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542205+0000", 2024-09-20T22:12:56.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4589999999999999e-05 2024-09-20T22:12:56.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542219+0000", 2024-09-20T22:12:56.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3879999999999999e-05 2024-09-20T22:12:56.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.542229+0000", 2024-09-20T22:12:56.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0797e-05 2024-09-20T22:12:56.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.637 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.637 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.718074+0000", 2024-09-20T22:12:56.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.17584509000000001 2024-09-20T22:12:56.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.637 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:56.637 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.718102+0000", 2024-09-20T22:12:56.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7668999999999999e-05 2024-09-20T22:12:56.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.637 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:56.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.728943+0000", 2024-09-20T22:12:56.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.01084106 2024-09-20T22:12:56.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.638 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.638 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.728975+0000", 2024-09-20T22:12:56.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2129999999999999e-05 2024-09-20T22:12:56.638 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.638 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.638 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.638 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.638 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105654 8.1 8:83678158:::benchmark_data_smithi060_111051_object105653:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:12:56.638 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.549939+0000", 2024-09-20T22:12:56.639 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 228.03269279200001, 2024-09-20T22:12:56.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.003362782, 2024-09-20T22:12:56.639 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.639 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.639 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:12:56.639 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:56.639 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:56.639 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:56.639 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105654 2024-09-20T22:12:56.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.639 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.640 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549939+0000", 2024-09-20T22:12:56.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.640 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.640 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549939+0000", 2024-09-20T22:12:56.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.640 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.640 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549940+0000", 2024-09-20T22:12:56.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.95e-06 2024-09-20T22:12:56.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.640 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.641 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549973+0000", 2024-09-20T22:12:56.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2798e-05 2024-09-20T22:12:56.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.641 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.641 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549997+0000", 2024-09-20T22:12:56.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3649e-05 2024-09-20T22:12:56.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.641 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.641 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550007+0000", 2024-09-20T22:12:56.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8190000000000003e-06 2024-09-20T22:12:56.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.642 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.642 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702300+0000", 2024-09-20T22:12:56.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.152292975 2024-09-20T22:12:56.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.642 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.642 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702310+0000", 2024-09-20T22:12:56.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0453e-05 2024-09-20T22:12:56.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.642 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.642 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.553223+0000", 2024-09-20T22:12:56.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85091275799999999 2024-09-20T22:12:56.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.553301+0000", 2024-09-20T22:12:56.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8380000000000005e-05 2024-09-20T22:12:56.643 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.643 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.643 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.643 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105657 8.7 8:e060afd6:::benchmark_data_smithi060_111051_object105656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:12:56.643 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.550025+0000", 2024-09-20T22:12:56.643 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 228.032606276, 2024-09-20T22:12:56.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.029495933, 2024-09-20T22:12:56.644 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.644 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.644 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:56.644 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:56.644 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:56.644 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:56.644 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105657 2024-09-20T22:12:56.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.644 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550025+0000", 2024-09-20T22:12:56.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.645 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550025+0000", 2024-09-20T22:12:56.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.645 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.645 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550026+0000", 2024-09-20T22:12:56.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1799999999999995e-07 2024-09-20T22:12:56.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.645 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.645 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550051+0000", 2024-09-20T22:12:56.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5554999999999998e-05 2024-09-20T22:12:56.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550062+0000", 2024-09-20T22:12:56.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.137e-05 2024-09-20T22:12:56.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550068+0000", 2024-09-20T22:12:56.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8289999999999996e-06 2024-09-20T22:12:56.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720577+0000", 2024-09-20T22:12:56.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.170508778 2024-09-20T22:12:56.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720588+0000", 2024-09-20T22:12:56.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0808000000000001e-05 2024-09-20T22:12:56.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.569210+0000", 2024-09-20T22:12:56.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.84862162900000004 2024-09-20T22:12:56.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:56.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.569360+0000", 2024-09-20T22:12:56.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015044799999999999 2024-09-20T22:12:56.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:12:56.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.569667+0000", 2024-09-20T22:12:56.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030655599999999998 2024-09-20T22:12:56.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:56.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.572342+0000", 2024-09-20T22:12:56.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0026757840000000001 2024-09-20T22:12:56.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.573025+0000", 2024-09-20T22:12:56.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00068279199999999999 2024-09-20T22:12:56.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579397+0000", 2024-09-20T22:12:56.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.006372071 2024-09-20T22:12:56.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:56.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579465+0000", 2024-09-20T22:12:56.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7520000000000004e-05 2024-09-20T22:12:56.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579521+0000", 2024-09-20T22:12:56.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6274999999999999e-05 2024-09-20T22:12:56.650 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.650 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.650 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.650 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105658 8.1 8:809ea6c9:::benchmark_data_smithi060_111051_object105657:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:12:56.650 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.550079+0000", 2024-09-20T22:12:56.651 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 228.03255269300001, 2024-09-20T22:12:56.651 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.010609463, 2024-09-20T22:12:56.651 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.651 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.651 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:56.651 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:56.651 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:56.651 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:56.651 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105658 2024-09-20T22:12:56.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.651 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.651 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.651 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550079+0000", 2024-09-20T22:12:56.651 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.652 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550079+0000", 2024-09-20T22:12:56.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.652 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550079+0000", 2024-09-20T22:12:56.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2599999999999998e-07 2024-09-20T22:12:56.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.652 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550101+0000", 2024-09-20T22:12:56.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1628e-05 2024-09-20T22:12:56.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550111+0000", 2024-09-20T22:12:56.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9620000000000005e-06 2024-09-20T22:12:56.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550116+0000", 2024-09-20T22:12:56.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9810000000000003e-06 2024-09-20T22:12:56.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702335+0000", 2024-09-20T22:12:56.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15221938700000001 2024-09-20T22:12:56.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702343+0000", 2024-09-20T22:12:56.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2120000000000003e-06 2024-09-20T22:12:56.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.553649+0000", 2024-09-20T22:12:56.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85130619600000002 2024-09-20T22:12:56.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:56.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.553784+0000", 2024-09-20T22:12:56.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000134499 2024-09-20T22:12:56.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:12:56.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.554060+0000", 2024-09-20T22:12:56.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000276579 2024-09-20T22:12:56.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:56.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.557338+0000", 2024-09-20T22:12:56.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003277608 2024-09-20T22:12:56.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.559599+0000", 2024-09-20T22:12:56.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002261304 2024-09-20T22:12:56.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560617+0000", 2024-09-20T22:12:56.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001017354 2024-09-20T22:12:56.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:56.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560660+0000", 2024-09-20T22:12:56.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3167999999999997e-05 2024-09-20T22:12:56.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560688+0000", 2024-09-20T22:12:56.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8158999999999999e-05 2024-09-20T22:12:56.657 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.657 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.657 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.657 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105659 8.1 8:80c09cff:::benchmark_data_smithi060_111051_object105658:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:12:56.657 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.550123+0000", 2024-09-20T22:12:56.657 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 228.03250795899999, 2024-09-20T22:12:56.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0107473760000001, 2024-09-20T22:12:56.658 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.658 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.658 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:56.658 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:56.658 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:56.658 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:56.658 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105659 2024-09-20T22:12:56.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.658 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.658 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.658 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550123+0000", 2024-09-20T22:12:56.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550123+0000", 2024-09-20T22:12:56.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550124+0000", 2024-09-20T22:12:56.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.39e-07 2024-09-20T22:12:56.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550148+0000", 2024-09-20T22:12:56.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3944e-05 2024-09-20T22:12:56.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550157+0000", 2024-09-20T22:12:56.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5410000000000006e-06 2024-09-20T22:12:56.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550162+0000", 2024-09-20T22:12:56.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8069999999999999e-06 2024-09-20T22:12:56.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702368+0000", 2024-09-20T22:12:56.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15220560899999999 2024-09-20T22:12:56.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702374+0000", 2024-09-20T22:12:56.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7939999999999997e-06 2024-09-20T22:12:56.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.555649+0000", 2024-09-20T22:12:56.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85327487000000002 2024-09-20T22:12:56.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:56.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.555819+0000", 2024-09-20T22:12:56.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016938200000000001 2024-09-20T22:12:56.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:12:56.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.556100+0000", 2024-09-20T22:12:56.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00028165699999999999 2024-09-20T22:12:56.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:56.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.558925+0000", 2024-09-20T22:12:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0028247290000000002 2024-09-20T22:12:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.559791+0000", 2024-09-20T22:12:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00086584599999999998 2024-09-20T22:12:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560771+0000", 2024-09-20T22:12:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00097968300000000007 2024-09-20T22:12:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560821+0000", 2024-09-20T22:12:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0460999999999997e-05 2024-09-20T22:12:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560871+0000", 2024-09-20T22:12:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9614000000000003e-05 2024-09-20T22:12:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.664 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105660 8.13 8:cf39f948:::benchmark_data_smithi060_111051_object105659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:12:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.550169+0000", 2024-09-20T22:12:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 228.03246222600001, 2024-09-20T22:12:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0659826960000001, 2024-09-20T22:12:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105660 2024-09-20T22:12:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.665 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550169+0000", 2024-09-20T22:12:56.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550169+0000", 2024-09-20T22:12:56.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550169+0000", 2024-09-20T22:12:56.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9099999999999999e-07 2024-09-20T22:12:56.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550221+0000", 2024-09-20T22:12:56.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1844e-05 2024-09-20T22:12:56.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550233+0000", 2024-09-20T22:12:56.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1871e-05 2024-09-20T22:12:56.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550238+0000", 2024-09-20T22:12:56.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6750000000000001e-06 2024-09-20T22:12:56.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.701152+0000", 2024-09-20T22:12:56.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15091443500000001 2024-09-20T22:12:56.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.701161+0000", 2024-09-20T22:12:56.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2679999999999994e-06 2024-09-20T22:12:56.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.605154+0000", 2024-09-20T22:12:56.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90399336100000005 2024-09-20T22:12:56.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:56.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.605243+0000", 2024-09-20T22:12:56.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8661999999999999e-05 2024-09-20T22:12:56.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 5,6", 2024-09-20T22:12:56.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.605424+0000", 2024-09-20T22:12:56.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018181800000000001 2024-09-20T22:12:56.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:56.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.606520+0000", 2024-09-20T22:12:56.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001095333 2024-09-20T22:12:56.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.612488+0000", 2024-09-20T22:12:56.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0059685019999999997 2024-09-20T22:12:56.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.616043+0000", 2024-09-20T22:12:56.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0035544249999999999 2024-09-20T22:12:56.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:56.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.616108+0000", 2024-09-20T22:12:56.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5506999999999995e-05 2024-09-20T22:12:56.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.616152+0000", 2024-09-20T22:12:56.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3603999999999998e-05 2024-09-20T22:12:56.671 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.671 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.671 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.671 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.15192.0:105655 8.f e215/176 8:ff82ed31:::benchmark_data_smithi060_111051_object105654:head v 215'6690, mlcod=214'6687)", 2024-09-20T22:12:56.671 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.562042+0000", 2024-09-20T22:12:56.672 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 228.02058927100001, 2024-09-20T22:12:56.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.16695152799999999, 2024-09-20T22:12:56.672 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.672 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.672 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:56.672 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.562042+0000", 2024-09-20T22:12:56.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.562042+0000", 2024-09-20T22:12:56.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.562044+0000", 2024-09-20T22:12:56.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.657e-06 2024-09-20T22:12:56.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.562073+0000", 2024-09-20T22:12:56.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9498000000000001e-05 2024-09-20T22:12:56.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.562091+0000", 2024-09-20T22:12:56.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.732e-05 2024-09-20T22:12:56.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.562101+0000", 2024-09-20T22:12:56.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0243e-05 2024-09-20T22:12:56.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720131+0000", 2024-09-20T22:12:56.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15803002899999999 2024-09-20T22:12:56.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:56.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720151+0000", 2024-09-20T22:12:56.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0579999999999999e-05 2024-09-20T22:12:56.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:56.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.728980+0000", 2024-09-20T22:12:56.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0088283089999999995 2024-09-20T22:12:56.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.728994+0000", 2024-09-20T22:12:56.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:12:56.676 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:56.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3892e-05 2024-09-20T22:12:56.677 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.677 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.677 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.678 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105668 8.7 8:e0173451:::benchmark_data_smithi060_111051_object105667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:12:56.678 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.706681+0000", 2024-09-20T22:12:56.678 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 227.87595042500001, 2024-09-20T22:12:56.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.87300058000000003, 2024-09-20T22:12:56.678 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.678 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.678 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:56.678 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:56.678 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:56.678 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:56.678 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105668 2024-09-20T22:12:56.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.678 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.679 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706681+0000", 2024-09-20T22:12:56.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.679 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706681+0000", 2024-09-20T22:12:56.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.679 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706683+0000", 2024-09-20T22:12:56.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.928e-06 2024-09-20T22:12:56.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.680 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706710+0000", 2024-09-20T22:12:56.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6911e-05 2024-09-20T22:12:56.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.680 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706734+0000", 2024-09-20T22:12:56.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3992999999999999e-05 2024-09-20T22:12:56.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706743+0000", 2024-09-20T22:12:56.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1209999999999996e-06 2024-09-20T22:12:56.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720612+0000", 2024-09-20T22:12:56.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.013869283 2024-09-20T22:12:56.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720619+0000", 2024-09-20T22:12:56.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7100000000000001e-06 2024-09-20T22:12:56.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.682 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.682 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.571247+0000", 2024-09-20T22:12:56.682 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85062784499999999 2024-09-20T22:12:56.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.682 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:56.682 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.571381+0000", 2024-09-20T22:12:56.682 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000134167 2024-09-20T22:12:56.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.682 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:12:56.682 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.571594+0000", 2024-09-20T22:12:56.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021272700000000001 2024-09-20T22:12:56.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:56.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.572500+0000", 2024-09-20T22:12:56.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00090607800000000005 2024-09-20T22:12:56.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.574997+0000", 2024-09-20T22:12:56.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024976759999999999 2024-09-20T22:12:56.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579580+0000", 2024-09-20T22:12:56.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0045830269999999999 2024-09-20T22:12:56.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:56.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579636+0000", 2024-09-20T22:12:56.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5413e-05 2024-09-20T22:12:56.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579681+0000", 2024-09-20T22:12:56.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5701000000000002e-05 2024-09-20T22:12:56.684 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.684 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.684 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.685 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106973 8.13 8:cbbe7a4a:::benchmark_data_smithi060_111051_object106972:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:12:56.685 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.414028+0000", 2024-09-20T22:12:56.685 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 225.16860298399999, 2024-09-20T22:12:56.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0848815410000001, 2024-09-20T22:12:56.685 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.685 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.685 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:56.685 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:56.685 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:56.685 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:56.685 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106973 2024-09-20T22:12:56.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.685 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414028+0000", 2024-09-20T22:12:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414028+0000", 2024-09-20T22:12:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414030+0000", 2024-09-20T22:12:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.866e-06 2024-09-20T22:12:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414059+0000", 2024-09-20T22:12:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9043999999999998e-05 2024-09-20T22:12:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414080+0000", 2024-09-20T22:12:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1254999999999999e-05 2024-09-20T22:12:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414090+0000", 2024-09-20T22:12:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2129999999999997e-06 2024-09-20T22:12:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.415798+0000", 2024-09-20T22:12:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017082340000000001 2024-09-20T22:12:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.415809+0000", 2024-09-20T22:12:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1433e-05 2024-09-20T22:12:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.479861+0000", 2024-09-20T22:12:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.064051522 2024-09-20T22:12:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.479994+0000", 2024-09-20T22:12:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013297800000000001 2024-09-20T22:12:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5,6", 2024-09-20T22:12:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.480283+0000", 2024-09-20T22:12:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00028927100000000002 2024-09-20T22:12:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.482942+0000", 2024-09-20T22:12:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002658737 2024-09-20T22:12:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.483991+0000", 2024-09-20T22:12:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001049267 2024-09-20T22:12:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.484017+0000", 2024-09-20T22:12:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6057000000000001e-05 2024-09-20T22:12:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.486044+0000", 2024-09-20T22:12:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020266189999999999 2024-09-20T22:12:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.498642+0000", 2024-09-20T22:12:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.012598417000000001 2024-09-20T22:12:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.498776+0000", 2024-09-20T22:12:56.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000133764 2024-09-20T22:12:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.498910+0000", 2024-09-20T22:12:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000133864 2024-09-20T22:12:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187845 8.a 8:5885b3a8:::benchmark_data_smithi060_111051_object67846:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:12:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.623815+0000", 2024-09-20T22:12:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 146.95881617500001, 2024-09-20T22:12:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1184221929999998, 2024-09-20T22:12:56.692 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:12:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187845 2024-09-20T22:12:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.623815+0000", 2024-09-20T22:12:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.623815+0000", 2024-09-20T22:12:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.623816+0000", 2024-09-20T22:12:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.114e-06 2024-09-20T22:12:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.623820+0000", 2024-09-20T22:12:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5360000000000001e-06 2024-09-20T22:12:56.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.623833+0000", 2024-09-20T22:12:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2874000000000001e-05 2024-09-20T22:12:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.623839+0000", 2024-09-20T22:12:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1319999999999999e-06 2024-09-20T22:12:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.624269+0000", 2024-09-20T22:12:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00043047400000000003 2024-09-20T22:12:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.624378+0000", 2024-09-20T22:12:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108738 2024-09-20T22:12:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:12:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.624481+0000", 2024-09-20T22:12:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103175 2024-09-20T22:12:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:56.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.629342+0000", 2024-09-20T22:12:56.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0048608089999999998 2024-09-20T22:12:56.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.631000+0000", 2024-09-20T22:12:56.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0016580049999999999 2024-09-20T22:12:56.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.640337+0000", 2024-09-20T22:12:56.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0093367550000000004 2024-09-20T22:12:56.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.651705+0000", 2024-09-20T22:12:56.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.011368038 2024-09-20T22:12:56.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650249+0000", 2024-09-20T22:12:56.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99854470299999998 2024-09-20T22:12:56.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650268+0000", 2024-09-20T22:12:56.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8981e-05 2024-09-20T22:12:56.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.742151+0000", 2024-09-20T22:12:56.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.09188247 2024-09-20T22:12:56.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.742237+0000", 2024-09-20T22:12:56.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6389000000000002e-05 2024-09-20T22:12:56.699 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.699 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.699 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.700 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187855 8.a 8:5492c8b1:::benchmark_data_smithi060_111051_object67856:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:12:56.700 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.636998+0000", 2024-09-20T22:12:56.700 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 146.94563358400001, 2024-09-20T22:12:56.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1143666080000001, 2024-09-20T22:12:56.700 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.700 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.700 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:56.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:56.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:56.700 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:56.700 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187855 2024-09-20T22:12:56.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.700 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.636998+0000", 2024-09-20T22:12:56.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.636998+0000", 2024-09-20T22:12:56.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.636999+0000", 2024-09-20T22:12:56.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6080000000000001e-06 2024-09-20T22:12:56.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.637004+0000", 2024-09-20T22:12:56.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5839999999999996e-06 2024-09-20T22:12:56.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.637014+0000", 2024-09-20T22:12:56.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0569e-05 2024-09-20T22:12:56.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.637021+0000", 2024-09-20T22:12:56.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9670000000000004e-06 2024-09-20T22:12:56.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.651901+0000", 2024-09-20T22:12:56.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014879966999999999 2024-09-20T22:12:56.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.651911+0000", 2024-09-20T22:12:56.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2040000000000003e-06 2024-09-20T22:12:56.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650383+0000", 2024-09-20T22:12:56.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99847266700000004 2024-09-20T22:12:56.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650393+0000", 2024-09-20T22:12:56.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6299999999999993e-06 2024-09-20T22:12:56.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.742483+0000", 2024-09-20T22:12:56.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0920898370000001 2024-09-20T22:12:56.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:56.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.742615+0000", 2024-09-20T22:12:56.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013252000000000001 2024-09-20T22:12:56.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5", 2024-09-20T22:12:56.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.743004+0000", 2024-09-20T22:12:56.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00038869300000000001 2024-09-20T22:12:56.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751051+0000", 2024-09-20T22:12:56.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0080469289999999995 2024-09-20T22:12:56.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:56.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751066+0000", 2024-09-20T22:12:56.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5245999999999999e-05 2024-09-20T22:12:56.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751111+0000", 2024-09-20T22:12:56.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4713999999999999e-05 2024-09-20T22:12:56.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751258+0000", 2024-09-20T22:12:56.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147479 2024-09-20T22:12:56.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:56.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751292+0000", 2024-09-20T22:12:56.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3330000000000001e-05 2024-09-20T22:12:56.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751364+0000", 2024-09-20T22:12:56.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2663999999999994e-05 2024-09-20T22:12:56.707 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.708 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.708 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.708 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187859 8.a 8:5b818400:::benchmark_data_smithi060_111051_object67860:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:12:56.708 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.649251+0000", 2024-09-20T22:12:56.708 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 146.93337985900001, 2024-09-20T22:12:56.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1040842400000002, 2024-09-20T22:12:56.708 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.708 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.708 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:56.708 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:56.708 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:56.708 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:56.708 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187859 2024-09-20T22:12:56.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.709 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.649251+0000", 2024-09-20T22:12:56.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.649251+0000", 2024-09-20T22:12:56.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.649253+0000", 2024-09-20T22:12:56.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6589999999999999e-06 2024-09-20T22:12:56.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.649258+0000", 2024-09-20T22:12:56.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7149999999999997e-06 2024-09-20T22:12:56.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.649268+0000", 2024-09-20T22:12:56.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0389e-05 2024-09-20T22:12:56.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.649278+0000", 2024-09-20T22:12:56.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0104e-05 2024-09-20T22:12:56.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.651941+0000", 2024-09-20T22:12:56.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0026624449999999998 2024-09-20T22:12:56.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.651948+0000", 2024-09-20T22:12:56.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5460000000000003e-06 2024-09-20T22:12:56.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650417+0000", 2024-09-20T22:12:56.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99846875400000001 2024-09-20T22:12:56.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650425+0000", 2024-09-20T22:12:56.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1300000000000001e-06 2024-09-20T22:12:56.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.746082+0000", 2024-09-20T22:12:56.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.095656484 2024-09-20T22:12:56.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:56.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.746198+0000", 2024-09-20T22:12:56.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000116168 2024-09-20T22:12:56.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5", 2024-09-20T22:12:56.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.746326+0000", 2024-09-20T22:12:56.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012862299999999999 2024-09-20T22:12:56.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:56.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751072+0000", 2024-09-20T22:12:56.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0047450720000000004 2024-09-20T22:12:56.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751157+0000", 2024-09-20T22:12:56.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5464999999999996e-05 2024-09-20T22:12:56.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751219+0000", 2024-09-20T22:12:56.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2420000000000002e-05 2024-09-20T22:12:56.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753223+0000", 2024-09-20T22:12:56.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020036950000000002 2024-09-20T22:12:56.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:56.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753255+0000", 2024-09-20T22:12:56.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2237999999999999e-05 2024-09-20T22:12:56.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753336+0000", 2024-09-20T22:12:56.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0333000000000005e-05 2024-09-20T22:12:56.716 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.716 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.716 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.716 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187868 8.a 8:5d2db7a2:::benchmark_data_smithi060_111051_object67869:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:12:56.716 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.665623+0000", 2024-09-20T22:12:56.716 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 146.91700832000001, 2024-09-20T22:12:56.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1683483180000001, 2024-09-20T22:12:56.716 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.716 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.716 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:56.716 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:56.717 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:56.717 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:56.717 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187868 2024-09-20T22:12:56.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.717 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665623+0000", 2024-09-20T22:12:56.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665623+0000", 2024-09-20T22:12:56.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665624+0000", 2024-09-20T22:12:56.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.850000000000001e-07 2024-09-20T22:12:56.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665629+0000", 2024-09-20T22:12:56.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8919999999999999e-06 2024-09-20T22:12:56.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665644+0000", 2024-09-20T22:12:56.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4808e-05 2024-09-20T22:12:56.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665652+0000", 2024-09-20T22:12:56.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0039999999999999e-06 2024-09-20T22:12:56.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665729+0000", 2024-09-20T22:12:56.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7385000000000005e-05 2024-09-20T22:12:56.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665739+0000", 2024-09-20T22:12:56.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6360000000000006e-06 2024-09-20T22:12:56.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650448+0000", 2024-09-20T22:12:56.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98470946199999998 2024-09-20T22:12:56.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650456+0000", 2024-09-20T22:12:56.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.413e-06 2024-09-20T22:12:56.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.749234+0000", 2024-09-20T22:12:56.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0987788570000001 2024-09-20T22:12:56.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:56.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.749337+0000", 2024-09-20T22:12:56.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000102441 2024-09-20T22:12:56.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5", 2024-09-20T22:12:56.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.749412+0000", 2024-09-20T22:12:56.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4808999999999995e-05 2024-09-20T22:12:56.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:56.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751122+0000", 2024-09-20T22:12:56.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017103159999999999 2024-09-20T22:12:56.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753047+0000", 2024-09-20T22:12:56.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019251750000000001 2024-09-20T22:12:56.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.723 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753196+0000", 2024-09-20T22:12:56.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000148441 2024-09-20T22:12:56.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.723 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.723 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.833608+0000", 2024-09-20T22:12:56.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.080412862000000002 2024-09-20T22:12:56.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.723 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:56.723 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.833835+0000", 2024-09-20T22:12:56.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022606199999999999 2024-09-20T22:12:56.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.833971+0000", 2024-09-20T22:12:56.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013677000000000001 2024-09-20T22:12:56.724 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.724 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.724 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.724 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187898 8.a 8:535037c1:::benchmark_data_smithi060_111051_object67899:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:12:56.724 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.705909+0000", 2024-09-20T22:12:56.724 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 146.87672257099999, 2024-09-20T22:12:56.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1284684729999999, 2024-09-20T22:12:56.724 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:12:56.724 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:12:56.725 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:12:56.725 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:12:56.725 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:12:56.725 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:12:56.725 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187898 2024-09-20T22:12:56.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.725 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:12:56.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:12:56.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.705909+0000", 2024-09-20T22:12:56.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:12:56.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.705909+0000", 2024-09-20T22:12:56.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:12:56.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:12:56.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.705910+0000", 2024-09-20T22:12:56.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.299e-06 2024-09-20T22:12:56.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:12:56.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.705915+0000", 2024-09-20T22:12:56.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.429e-06 2024-09-20T22:12:56.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:12:56.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.705930+0000", 2024-09-20T22:12:56.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4916000000000001e-05 2024-09-20T22:12:56.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:12:56.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.705938+0000", 2024-09-20T22:12:56.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0290000000000005e-06 2024-09-20T22:12:56.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.706003+0000", 2024-09-20T22:12:56.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4832999999999994e-05 2024-09-20T22:12:56.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.706014+0000", 2024-09-20T22:12:56.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0961000000000001e-05 2024-09-20T22:12:56.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650482+0000", 2024-09-20T22:12:56.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94446805099999998 2024-09-20T22:12:56.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:12:56.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650490+0000", 2024-09-20T22:12:56.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1589999999999993e-06 2024-09-20T22:12:56.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:12:56.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751402+0000", 2024-09-20T22:12:56.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1009116329999999 2024-09-20T22:12:56.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:12:56.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751502+0000", 2024-09-20T22:12:56.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9617000000000005e-05 2024-09-20T22:12:56.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5", 2024-09-20T22:12:56.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751574+0000", 2024-09-20T22:12:56.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2517999999999999e-05 2024-09-20T22:12:56.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:12:56.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753349+0000", 2024-09-20T22:12:56.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001774386 2024-09-20T22:12:56.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753383+0000", 2024-09-20T22:12:56.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4017999999999999e-05 2024-09-20T22:12:56.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753421+0000", 2024-09-20T22:12:56.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8603000000000003e-05 2024-09-20T22:12:56.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:12:56.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.834077+0000", 2024-09-20T22:12:56.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.080655317000000004 2024-09-20T22:12:56.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:12:56.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.834257+0000", 2024-09-20T22:12:56.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000180688 2024-09-20T22:12:56.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:12:56.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:12:56.732 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:12:56.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.834377+0000", 2024-09-20T22:12:56.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000120016 2024-09-20T22:12:56.732 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.732 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.732 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.732 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:12:56.732 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:12:56.732 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:12:56.755 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:56.756 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:56 smithi060 ceph-mon[29694]: osdmap e327: 8 total, 8 up, 8 in 2024-09-20T22:12:56.756 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:56 smithi060 ceph-mon[29694]: pgmap v545: 158 pgs: 24 peering, 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 131 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 13802/264993 objects misplaced (5.208%); 285 B/s, 5 objects/s recovering 2024-09-20T22:12:56.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:56.756 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:56.757 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:56 smithi060 ceph-mon[40290]: osdmap e327: 8 total, 8 up, 8 in 2024-09-20T22:12:56.757 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:56 smithi060 ceph-mon[40290]: pgmap v545: 158 pgs: 24 peering, 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 131 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 13802/264993 objects misplaced (5.208%); 285 B/s, 5 objects/s recovering 2024-09-20T22:12:56.857 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:56.880 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:56.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:56.880 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:56.902 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 69 16 88458 88442 80.0932 0 - 0.0104764 2024-09-20T22:12:56.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:57.005 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:57.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:57.006 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:57.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:56 smithi067 ceph-mon[33511]: osdmap e327: 8 total, 8 up, 8 in 2024-09-20T22:12:57.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:56 smithi067 ceph-mon[33511]: pgmap v545: 158 pgs: 24 peering, 1 active+remapped+backfilling, 2 active+remapped+backfill_toofull, 131 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 13802/264993 objects misplaced (5.208%); 285 B/s, 5 objects/s recovering 2024-09-20T22:12:57.106 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:57.128 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:57.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:57.129 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:57.230 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:57.252 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:57.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:57.253 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:57.293 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:12:57.293 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:12:57.293 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-09-20T22:12:57.294 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-20T22:12:57.294 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-20T22:12:57.294 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 config set filestore_inject_stall 3 2024-09-20T22:12:57.354 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:57.376 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:57.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:57.377 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:57.478 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:57.501 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:57.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:57.501 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:57.602 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:57.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T22:12:57.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:57.625 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:57.626 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:57.645 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:57.726 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:57.748 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:57.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:57.749 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:57.850 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:57.872 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:57.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:57.873 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:57.903 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 70 16 88458 88442 78.9491 0 - 0.0104764 2024-09-20T22:12:57.904 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:12:57.904 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:12:57.928 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:12:57.928 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:12:57.933 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:12:57.973 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:57.996 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:57.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:57.997 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:58.098 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:58.121 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:58.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:58.122 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:58.223 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:58.245 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:58.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:58.246 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:12:58.347 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:58.369 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:58.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:58.370 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:58.470 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:58.493 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:58.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:58.494 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:58.594 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:58.617 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:58.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:58.618 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:58.719 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:58.741 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:58.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:58.743 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:58.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:12:58.866 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:58.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:58.867 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:12:58.903 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 71 16 88458 88442 77.8373 0 - 0.0104764 2024-09-20T22:12:58.944 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:12:58.968 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:58.995 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:58.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:58.996 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:59.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:58 smithi060 ceph-mon[29694]: osdmap e328: 8 total, 8 up, 8 in 2024-09-20T22:12:59.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:58 smithi060 ceph-mon[29694]: pgmap v547: 158 pgs: 1 remapped+peering, 3 peering, 154 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 15 KiB/s rd, 4.2 MiB/s wr, 92 op/s 2024-09-20T22:12:59.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:58 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3009539702' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:12:59.089 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:58 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:12:59.089 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:58 smithi060 ceph-mon[40290]: osdmap e328: 8 total, 8 up, 8 in 2024-09-20T22:12:59.089 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:58 smithi060 ceph-mon[40290]: pgmap v547: 158 pgs: 1 remapped+peering, 3 peering, 154 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 15 KiB/s rd, 4.2 MiB/s wr, 92 op/s 2024-09-20T22:12:59.089 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:58 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3009539702' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:12:59.089 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:58 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:12:59.097 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:12:59.133 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:59.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:59.134 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:12:59.235 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:12:59.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:58 smithi067 ceph-mon[33511]: osdmap e328: 8 total, 8 up, 8 in 2024-09-20T22:12:59.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:58 smithi067 ceph-mon[33511]: pgmap v547: 158 pgs: 1 remapped+peering, 3 peering, 154 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 15 KiB/s rd, 4.2 MiB/s wr, 92 op/s 2024-09-20T22:12:59.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:58 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3009539702' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:12:59.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:58 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:12:59.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:59.280 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:59.280 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:12:59.381 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:12:59.421 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:59.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:59.422 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:12:59.522 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:12:59.556 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:59.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:59.557 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:12:59.658 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:12:59.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:59.698 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:59.698 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:12:59.799 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:12:59.838 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:12:59.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:59.839 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:12:59.903 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 72 16 89690 89674 77.8255 5.92308 0.0112396 0.0128446 2024-09-20T22:12:59.940 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:12:59.984 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:12:59.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:12:59.985 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:00.017 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:13:00.017 INFO:teuthology.orchestra.run.smithi060.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-20T22:13:00.017 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:13:00.017 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:59 smithi060 ceph-mon[29694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 10 pgs inactive, 13 pgs peering) 2024-09-20T22:13:00.018 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:59 smithi060 ceph-mon[29694]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-20T22:13:00.018 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:59 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:13:00.018 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:59 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:13:00.018 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:12:59 smithi060 ceph-mon[29694]: osdmap e329: 8 total, 8 up, 8 in 2024-09-20T22:13:00.018 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:59 smithi060 ceph-mon[40290]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 10 pgs inactive, 13 pgs peering) 2024-09-20T22:13:00.018 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:59 smithi060 ceph-mon[40290]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-20T22:13:00.018 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:59 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:13:00.018 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:59 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:13:00.018 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:12:59 smithi060 ceph-mon[40290]: osdmap e329: 8 total, 8 up, 8 in 2024-09-20T22:13:00.086 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:00.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:00.119 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:00.119 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:00.220 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:00.255 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:00.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:00.256 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:00.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:59 smithi067 ceph-mon[33511]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 10 pgs inactive, 13 pgs peering) 2024-09-20T22:13:00.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:59 smithi067 ceph-mon[33511]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull) 2024-09-20T22:13:00.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:59 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:13:00.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:59 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:13:00.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:12:59 smithi067 ceph-mon[33511]: osdmap e329: 8 total, 8 up, 8 in 2024-09-20T22:13:00.357 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:00.392 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:00.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:00.393 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:00.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:00.531 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:00.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:00.532 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:00.633 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:00.669 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:00.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:00.670 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:00.771 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:00.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:00.818 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:00.819 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:00.832 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:13:00.862 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:00.907 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 73 16 91026 91010 77.9031 83.5 0.00637186 0.0128318 2024-09-20T22:13:00.919 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:00.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:00.949 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:00.949 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:01.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:00 smithi060 ceph-mon[29694]: pgmap v549: 158 pgs: 1 remapped+peering, 3 peering, 154 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 13 KiB/s rd, 3.4 MiB/s wr, 76 op/s 2024-09-20T22:13:01.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:00 smithi060 ceph-mon[29694]: 6.a scrub starts 2024-09-20T22:13:01.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:00 smithi060 ceph-mon[29694]: 6.a scrub ok 2024-09-20T22:13:01.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:00 smithi060 ceph-mon[40290]: pgmap v549: 158 pgs: 1 remapped+peering, 3 peering, 154 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 13 KiB/s rd, 3.4 MiB/s wr, 76 op/s 2024-09-20T22:13:01.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:00 smithi060 ceph-mon[40290]: 6.a scrub starts 2024-09-20T22:13:01.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:00 smithi060 ceph-mon[40290]: 6.a scrub ok 2024-09-20T22:13:01.050 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:01.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:01.194 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:01.194 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:01.266 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:00 smithi067 ceph-mon[33511]: pgmap v549: 158 pgs: 1 remapped+peering, 3 peering, 154 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 13 KiB/s rd, 3.4 MiB/s wr, 76 op/s 2024-09-20T22:13:01.266 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:00 smithi067 ceph-mon[33511]: 6.a scrub starts 2024-09-20T22:13:01.266 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:00 smithi067 ceph-mon[33511]: 6.a scrub ok 2024-09-20T22:13:01.295 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:01.332 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:01.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:01.333 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:01.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:01.481 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:01.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:01.482 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:01.519 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:13:01.520 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:13:01.520 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.520 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:91717 9.8 9:190faab2:::benchmark_data_smithi060_140132_object91716:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-20T22:13:01.520 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:01.511759+0000", 2024-09-20T22:13:01.520 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0056251909999999999, 2024-09-20T22:13:01.520 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0056674100000000003, 2024-09-20T22:13:01.520 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:01.520 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:01.520 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:13:01.520 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:01.520 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:13:01.520 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:13:01.520 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 91717 2024-09-20T22:13:01.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.521 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:01.521 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.521 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:01.521 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.511759+0000", 2024-09-20T22:13:01.521 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:01.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.521 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.521 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:01.521 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.511759+0000", 2024-09-20T22:13:01.521 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:01.521 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.521 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.521 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:01.521 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.511761+0000", 2024-09-20T22:13:01.522 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2730000000000001e-06 2024-09-20T22:13:01.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.522 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:01.522 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.511799+0000", 2024-09-20T22:13:01.522 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7416999999999998e-05 2024-09-20T22:13:01.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.522 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:01.522 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.511814+0000", 2024-09-20T22:13:01.522 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5384e-05 2024-09-20T22:13:01.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.522 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:01.523 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.511824+0000", 2024-09-20T22:13:01.523 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0414000000000001e-05 2024-09-20T22:13:01.523 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.523 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.523 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:01.523 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.512117+0000", 2024-09-20T22:13:01.523 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029278799999999999 2024-09-20T22:13:01.523 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.523 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.523 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:01.523 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.512533+0000", 2024-09-20T22:13:01.523 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00041623700000000002 2024-09-20T22:13:01.523 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.524 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.524 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:13:01.524 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.512740+0000", 2024-09-20T22:13:01.524 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020710499999999999 2024-09-20T22:13:01.524 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:01.524 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:01.524 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:01.524 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.524 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.524 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:91718 9.8 9.5bab1db8 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-20T22:13:01.524 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:01.511848+0000", 2024-09-20T22:13:01.524 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0055357480000000001, 2024-09-20T22:13:01.524 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0056550230000000003, 2024-09-20T22:13:01.524 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:01.524 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:01.525 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:13:01.525 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:01.525 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:13:01.525 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:13:01.525 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 91718 2024-09-20T22:13:01.525 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.525 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:01.525 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.525 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:01.525 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.511848+0000", 2024-09-20T22:13:01.525 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:01.525 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.525 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.525 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:01.525 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.511848+0000", 2024-09-20T22:13:01.526 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:01.526 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.526 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.526 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:01.526 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.511849+0000", 2024-09-20T22:13:01.526 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8500000000000001e-07 2024-09-20T22:13:01.526 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.526 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.526 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:01.526 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.511871+0000", 2024-09-20T22:13:01.526 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2189999999999999e-05 2024-09-20T22:13:01.526 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.526 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.526 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:01.527 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.511882+0000", 2024-09-20T22:13:01.527 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0474e-05 2024-09-20T22:13:01.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.527 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.527 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:01.527 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.511889+0000", 2024-09-20T22:13:01.527 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5079999999999999e-06 2024-09-20T22:13:01.527 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:01.527 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:01.527 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:01.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.527 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.527 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:91724 9.9 9:998307c3:::benchmark_data_smithi060_140132_object91723:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-20T22:13:01.527 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:01.515728+0000", 2024-09-20T22:13:01.527 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0016563960000000001, 2024-09-20T22:13:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00183381, 2024-09-20T22:13:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:13:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:13:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:13:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 91724 2024-09-20T22:13:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.515728+0000", 2024-09-20T22:13:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:01.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.529 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:01.529 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.515728+0000", 2024-09-20T22:13:01.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:01.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.529 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:01.529 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.515729+0000", 2024-09-20T22:13:01.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.764e-06 2024-09-20T22:13:01.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.529 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:01.529 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.515777+0000", 2024-09-20T22:13:01.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7383999999999999e-05 2024-09-20T22:13:01.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.530 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.530 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:01.530 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.515796+0000", 2024-09-20T22:13:01.530 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9082000000000001e-05 2024-09-20T22:13:01.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.530 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.530 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:01.530 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.515806+0000", 2024-09-20T22:13:01.530 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0518e-05 2024-09-20T22:13:01.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.530 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.530 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:01.530 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.515999+0000", 2024-09-20T22:13:01.530 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019216599999999999 2024-09-20T22:13:01.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.531 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.531 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:01.531 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.516118+0000", 2024-09-20T22:13:01.531 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011977999999999999 2024-09-20T22:13:01.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.531 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.531 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 5,7", 2024-09-20T22:13:01.531 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.516252+0000", 2024-09-20T22:13:01.531 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013395 2024-09-20T22:13:01.531 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:01.531 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:01.531 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:01.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.531 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.531 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:91726 9.b 9:d69d984a:::benchmark_data_smithi060_140132_object91725:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e329)", 2024-09-20T22:13:01.532 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:01.515843+0000", 2024-09-20T22:13:01.532 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0015413709999999999, 2024-09-20T22:13:01.532 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017880890000000001, 2024-09-20T22:13:01.532 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:01.532 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:01.532 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:13:01.532 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:01.532 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:13:01.532 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:13:01.532 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 91726 2024-09-20T22:13:01.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.532 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:01.532 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.533 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:01.533 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.515843+0000", 2024-09-20T22:13:01.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:01.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.533 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.533 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:01.533 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.515843+0000", 2024-09-20T22:13:01.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:01.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.533 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.533 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:01.533 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.515843+0000", 2024-09-20T22:13:01.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1399999999999997e-07 2024-09-20T22:13:01.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.534 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:01.534 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.515868+0000", 2024-09-20T22:13:01.534 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4698999999999998e-05 2024-09-20T22:13:01.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.534 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:01.534 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.515878+0000", 2024-09-20T22:13:01.534 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5689999999999994e-06 2024-09-20T22:13:01.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.534 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:01.534 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.515884+0000", 2024-09-20T22:13:01.534 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4910000000000002e-06 2024-09-20T22:13:01.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.535 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:01.535 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.515953+0000", 2024-09-20T22:13:01.535 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8814999999999998e-05 2024-09-20T22:13:01.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.535 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:01.535 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.516081+0000", 2024-09-20T22:13:01.535 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000128441 2024-09-20T22:13:01.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:01.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:01.535 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 5,6", 2024-09-20T22:13:01.535 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:01.516213+0000", 2024-09-20T22:13:01.535 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013206700000000001 2024-09-20T22:13:01.535 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:01.536 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:01.536 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:01.536 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:01.536 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:13:01.536 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 4 2024-09-20T22:13:01.536 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:13:01.583 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:01.627 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:01.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:01.629 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:01.730 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:01.786 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:13:01.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:01.796 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:01.796 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:01.896 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:01.904 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 74 16 92158 92142 77.8062 70.75 0.00996029 0.0128477 2024-09-20T22:13:01.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:01.949 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:01.949 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:02.050 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:02.100 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:02.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:02.102 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:02.203 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:02.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:02.254 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:02.254 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:02.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:01 smithi067 ceph-mon[33511]: 6.f scrub starts 2024-09-20T22:13:02.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:01 smithi067 ceph-mon[33511]: 6.f scrub ok 2024-09-20T22:13:02.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:01 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:13:02.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:01 smithi060 ceph-mon[29694]: 6.f scrub starts 2024-09-20T22:13:02.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:01 smithi060 ceph-mon[29694]: 6.f scrub ok 2024-09-20T22:13:02.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:01 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:13:02.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:01 smithi060 ceph-mon[40290]: 6.f scrub starts 2024-09-20T22:13:02.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:01 smithi060 ceph-mon[40290]: 6.f scrub ok 2024-09-20T22:13:02.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:01 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:13:02.355 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:02.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:02.389 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:02.390 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:02.455 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T22:13:02.490 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:02.523 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:02.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:02.524 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:02.603 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:02.624 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:02.633 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:02.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:02.655 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:02.655 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:02.756 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:02.763 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:13:02 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:13:02.784 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:02.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:02.785 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:02.885 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:02.904 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 75 16 93333 93317 77.7474 73.4375 0.0224146 0.0128568 2024-09-20T22:13:02.918 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:02.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:02.919 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:03.020 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:03.028 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[29694]: 6.1 scrub starts 2024-09-20T22:13:03.028 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[29694]: 6.1 scrub ok 2024-09-20T22:13:03.028 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[29694]: 3.12 scrub starts 2024-09-20T22:13:03.028 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[29694]: 3.12 scrub ok 2024-09-20T22:13:03.028 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[29694]: 3.c scrub starts 2024-09-20T22:13:03.028 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[29694]: 3.c scrub ok 2024-09-20T22:13:03.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[29694]: pgmap v550: 158 pgs: 158 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 57 KiB/s rd, 23 MiB/s wr, 457 op/s 2024-09-20T22:13:03.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:13:03.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[29694]: 6.7 scrub starts 2024-09-20T22:13:03.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[29694]: 6.7 scrub ok 2024-09-20T22:13:03.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:13:03.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:13:03.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:13:03.029 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[40290]: 6.1 scrub starts 2024-09-20T22:13:03.029 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[40290]: 6.1 scrub ok 2024-09-20T22:13:03.029 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[40290]: 3.12 scrub starts 2024-09-20T22:13:03.029 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[40290]: 3.12 scrub ok 2024-09-20T22:13:03.029 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[40290]: 3.c scrub starts 2024-09-20T22:13:03.030 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[40290]: 3.c scrub ok 2024-09-20T22:13:03.030 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[40290]: pgmap v550: 158 pgs: 158 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 57 KiB/s rd, 23 MiB/s wr, 457 op/s 2024-09-20T22:13:03.030 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:13:03.030 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[40290]: 6.7 scrub starts 2024-09-20T22:13:03.030 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[40290]: 6.7 scrub ok 2024-09-20T22:13:03.030 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:13:03.030 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:13:03.030 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:02 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:13:03.051 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:03.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:03.052 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:03.153 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:03.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:03.179 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:03.179 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:03.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:02 smithi067 ceph-mon[33511]: 6.1 scrub starts 2024-09-20T22:13:03.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:02 smithi067 ceph-mon[33511]: 6.1 scrub ok 2024-09-20T22:13:03.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:02 smithi067 ceph-mon[33511]: 3.12 scrub starts 2024-09-20T22:13:03.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:02 smithi067 ceph-mon[33511]: 3.12 scrub ok 2024-09-20T22:13:03.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:02 smithi067 ceph-mon[33511]: 3.c scrub starts 2024-09-20T22:13:03.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:02 smithi067 ceph-mon[33511]: 3.c scrub ok 2024-09-20T22:13:03.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:02 smithi067 ceph-mon[33511]: pgmap v550: 158 pgs: 158 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 57 KiB/s rd, 23 MiB/s wr, 457 op/s 2024-09-20T22:13:03.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:02 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:13:03.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:02 smithi067 ceph-mon[33511]: 6.7 scrub starts 2024-09-20T22:13:03.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:02 smithi067 ceph-mon[33511]: 6.7 scrub ok 2024-09-20T22:13:03.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:02 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:13:03.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:02 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:13:03.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:02 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:13:03.280 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:03.318 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:03.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:03.319 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:03.420 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:03.457 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:03.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:03.458 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:03.559 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:03.598 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:03.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:03.599 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:03.700 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:03.738 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:03.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:03.739 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:03.840 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:03.882 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:03.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:03.883 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:03.904 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 76 16 94631 94615 77.7917 81.125 0.00736975 0.01285 2024-09-20T22:13:03.984 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:04.015 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:04.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:04.016 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:04.052 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:13:04.052 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:13:04.094 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:13:04.094 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:13:04.117 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:04.132 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:13:04.132 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:13:04.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:03 smithi060 ceph-mon[29694]: 3.19 scrub starts 2024-09-20T22:13:04.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:03 smithi060 ceph-mon[29694]: 3.19 scrub ok 2024-09-20T22:13:04.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:03 smithi060 ceph-mon[29694]: 6.18 scrub starts 2024-09-20T22:13:04.133 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:03 smithi060 ceph-mon[29694]: 6.18 scrub ok 2024-09-20T22:13:04.133 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:03 smithi060 ceph-mon[40290]: 3.19 scrub starts 2024-09-20T22:13:04.133 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:03 smithi060 ceph-mon[40290]: 3.19 scrub ok 2024-09-20T22:13:04.133 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:03 smithi060 ceph-mon[40290]: 6.18 scrub starts 2024-09-20T22:13:04.134 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:03 smithi060 ceph-mon[40290]: 6.18 scrub ok 2024-09-20T22:13:04.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:04.160 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:04.160 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:04.171 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:13:04.171 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:13:04.205 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:13:04.205 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:13:04.242 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:13:04.243 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:13:04.261 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:04.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:03 smithi067 ceph-mon[33511]: 3.19 scrub starts 2024-09-20T22:13:04.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:03 smithi067 ceph-mon[33511]: 3.19 scrub ok 2024-09-20T22:13:04.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:03 smithi067 ceph-mon[33511]: 6.18 scrub starts 2024-09-20T22:13:04.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:03 smithi067 ceph-mon[33511]: 6.18 scrub ok 2024-09-20T22:13:04.276 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:13:04.277 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:13:04.294 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:04.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:04.295 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:04.315 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:13:04.316 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:13:04.396 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:04.439 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:04.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:04.440 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:04.541 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:04.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:04.575 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:04.575 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:04.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:04.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:04.712 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:04.712 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:04.813 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:04.858 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:04.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:04.859 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:04.904 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 77 16 95838 95822 77.7611 75.4375 0.0168516 0.0128549 2024-09-20T22:13:04.959 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:04.990 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:13:05.002 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:05.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:05.004 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:05.105 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:05.146 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:05.146 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:04 smithi060 ceph-mon[29694]: 4.18 scrub starts 2024-09-20T22:13:05.146 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:04 smithi060 ceph-mon[29694]: 4.18 scrub ok 2024-09-20T22:13:05.146 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:04 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3956433279' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:05.147 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:04 smithi060 ceph-mon[29694]: pgmap v551: 158 pgs: 158 active+clean; 5.6 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 54 KiB/s rd, 29 MiB/s wr, 551 op/s 2024-09-20T22:13:05.147 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:04 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3919902567' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:13:05.147 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:13:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:13:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:13:05.147 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:04 smithi060 ceph-mon[40290]: 4.18 scrub starts 2024-09-20T22:13:05.147 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:04 smithi060 ceph-mon[40290]: 4.18 scrub ok 2024-09-20T22:13:05.147 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:04 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3956433279' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:05.147 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:04 smithi060 ceph-mon[40290]: pgmap v551: 158 pgs: 158 active+clean; 5.6 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 54 KiB/s rd, 29 MiB/s wr, 551 op/s 2024-09-20T22:13:05.147 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:04 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3919902567' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:13:05.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:05.148 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:05.249 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:05.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:04 smithi067 ceph-mon[33511]: 4.18 scrub starts 2024-09-20T22:13:05.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:04 smithi067 ceph-mon[33511]: 4.18 scrub ok 2024-09-20T22:13:05.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:04 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3956433279' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:05.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:04 smithi067 ceph-mon[33511]: pgmap v551: 158 pgs: 158 active+clean; 5.6 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 54 KiB/s rd, 29 MiB/s wr, 551 op/s 2024-09-20T22:13:05.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:04 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3919902567' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:13:05.292 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:05.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:05.293 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:05.393 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:05.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:05.454 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:05.454 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:05.554 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:05.598 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:05.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:05.599 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:05.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:05.744 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:05.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:05.745 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:05.845 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:13:05.845 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:13:05.846 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-09-20T22:13:05.846 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.1 reset_purged_snaps_last 2024-09-20T22:13:05.847 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:05.881 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:05.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:05.882 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:05.890 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:05.904 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 78 16 97121 97105 77.7916 80.1875 0.00589933 0.0128497 2024-09-20T22:13:05.983 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:06.013 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:06.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:06.014 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:06.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:05 smithi060 ceph-mon[29694]: 3.10 scrub starts 2024-09-20T22:13:06.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:05 smithi060 ceph-mon[29694]: 3.10 scrub ok 2024-09-20T22:13:06.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:05 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:13:06.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:05 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:13:06.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:05 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3919902567' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:13:06.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:05 smithi060 ceph-mon[29694]: osdmap e330: 8 total, 8 up, 8 in 2024-09-20T22:13:06.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:13:06.055 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:13:06.056 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:05 smithi060 ceph-mon[40290]: 3.10 scrub starts 2024-09-20T22:13:06.056 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:05 smithi060 ceph-mon[40290]: 3.10 scrub ok 2024-09-20T22:13:06.056 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:05 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:13:06.056 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:05 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:13:06.056 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:05 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3919902567' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:13:06.056 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:05 smithi060 ceph-mon[40290]: osdmap e330: 8 total, 8 up, 8 in 2024-09-20T22:13:06.056 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:13:06.056 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:13:06.114 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:06.164 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:06.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:06.165 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:06.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:05 smithi067 ceph-mon[33511]: 3.10 scrub starts 2024-09-20T22:13:06.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:05 smithi067 ceph-mon[33511]: 3.10 scrub ok 2024-09-20T22:13:06.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:05 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:13:06.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:05 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:13:06.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:05 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3919902567' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:13:06.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:05 smithi067 ceph-mon[33511]: osdmap e330: 8 total, 8 up, 8 in 2024-09-20T22:13:06.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:13:06.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:13:06.266 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:06.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:06.299 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:06.299 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:06.400 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:06.432 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:06.432 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:06.432 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:06.533 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:06.563 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:06.568 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:13:06.568 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:13:06.609 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:06.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:06.611 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:06.712 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:06.758 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:06.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:06.759 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:06.860 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:06.901 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:06.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:06.902 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:06.904 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 79 16 98310 98294 77.7475 74.3125 0.0141367 0.0128574 2024-09-20T22:13:07.003 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:07.042 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:07.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:07.043 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:07.144 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:07.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:07.183 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:07.183 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:07.201 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:13:07.208 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:06 smithi060 ceph-mon[29694]: 5.0 scrub starts 2024-09-20T22:13:07.208 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:06 smithi060 ceph-mon[29694]: 5.0 scrub ok 2024-09-20T22:13:07.208 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:06 smithi060 ceph-mon[29694]: pgmap v553: 158 pgs: 158 active+clean; 5.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 46 KiB/s rd, 50 MiB/s wr, 883 op/s; 8.0 KiB/s, 0 objects/s recovering 2024-09-20T22:13:07.208 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:06 smithi060 ceph-mon[29694]: osdmap e331: 8 total, 8 up, 8 in 2024-09-20T22:13:07.208 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:06 smithi060 ceph-mon[40290]: 5.0 scrub starts 2024-09-20T22:13:07.209 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:06 smithi060 ceph-mon[40290]: 5.0 scrub ok 2024-09-20T22:13:07.209 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:06 smithi060 ceph-mon[40290]: pgmap v553: 158 pgs: 158 active+clean; 5.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 46 KiB/s rd, 50 MiB/s wr, 883 op/s; 8.0 KiB/s, 0 objects/s recovering 2024-09-20T22:13:07.209 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:06 smithi060 ceph-mon[40290]: osdmap e331: 8 total, 8 up, 8 in 2024-09-20T22:13:07.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:06 smithi067 ceph-mon[33511]: 5.0 scrub starts 2024-09-20T22:13:07.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:06 smithi067 ceph-mon[33511]: 5.0 scrub ok 2024-09-20T22:13:07.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:06 smithi067 ceph-mon[33511]: pgmap v553: 158 pgs: 158 active+clean; 5.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 46 KiB/s rd, 50 MiB/s wr, 883 op/s; 8.0 KiB/s, 0 objects/s recovering 2024-09-20T22:13:07.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:06 smithi067 ceph-mon[33511]: osdmap e331: 8 total, 8 up, 8 in 2024-09-20T22:13:07.286 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:07.316 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:07.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:07.317 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:07.418 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:07.451 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:07.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:07.452 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:07.553 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:07.584 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:07.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:07.585 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:07.589 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T22:13:07.686 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:07.742 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:07.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:07.743 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:07.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:07.857 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:13:07.873 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:07.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:07.874 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:07.908 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:13:07.907727+0000 min lat: 0.0012684 max lat: 15.3765 avg lat: 0.0128685 2024-09-20T22:13:07.908 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:13:07.908 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 80 15 99465 99450 77.6757 72.25 0.014428 0.0128685 2024-09-20T22:13:07.939 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:07.975 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:08.008 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:08.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:08.009 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:08.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:07 smithi060 ceph-mon[29694]: 4.f scrub starts 2024-09-20T22:13:08.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:07 smithi060 ceph-mon[29694]: 4.f scrub ok 2024-09-20T22:13:08.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:07 smithi060 ceph-mon[29694]: 5.18 scrub starts 2024-09-20T22:13:08.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:07 smithi060 ceph-mon[29694]: 5.18 scrub ok 2024-09-20T22:13:08.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:07 smithi060 ceph-mon[29694]: 2.0 scrub starts 2024-09-20T22:13:08.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:08 smithi060 ceph-mon[29694]: 2.0 scrub ok 2024-09-20T22:13:08.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:08 smithi060 ceph-mon[40290]: 4.f scrub starts 2024-09-20T22:13:08.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:08 smithi060 ceph-mon[40290]: 4.f scrub ok 2024-09-20T22:13:08.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:08 smithi060 ceph-mon[40290]: 5.18 scrub starts 2024-09-20T22:13:08.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:08 smithi060 ceph-mon[40290]: 5.18 scrub ok 2024-09-20T22:13:08.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:08 smithi060 ceph-mon[40290]: 2.0 scrub starts 2024-09-20T22:13:08.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:08 smithi060 ceph-mon[40290]: 2.0 scrub ok 2024-09-20T22:13:08.110 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:08.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:08.147 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:08.148 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:08.248 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:08.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:08 smithi067 ceph-mon[33511]: 4.f scrub starts 2024-09-20T22:13:08.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:08 smithi067 ceph-mon[33511]: 4.f scrub ok 2024-09-20T22:13:08.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:08 smithi067 ceph-mon[33511]: 5.18 scrub starts 2024-09-20T22:13:08.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:08 smithi067 ceph-mon[33511]: 5.18 scrub ok 2024-09-20T22:13:08.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:08 smithi067 ceph-mon[33511]: 2.0 scrub starts 2024-09-20T22:13:08.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:08 smithi067 ceph-mon[33511]: 2.0 scrub ok 2024-09-20T22:13:08.285 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:08.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:08.286 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:08.387 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:08.429 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:08.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:08.430 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:08.531 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:08.566 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:08.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:08.567 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:08.668 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:08.709 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:08.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:08.710 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:08.811 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:08.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:08.852 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:08.853 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:08.908 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 81 16 100652 100636 77.6318 74.125 0.0182523 0.0128766 2024-09-20T22:13:08.953 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:08.994 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:08.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:08.996 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:09.096 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:09.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:09.150 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:09.150 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:09.250 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:09 smithi067 ceph-mon[33511]: 6.10 scrub starts 2024-09-20T22:13:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:09 smithi067 ceph-mon[33511]: 6.10 scrub ok 2024-09-20T22:13:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:09 smithi067 ceph-mon[33511]: pgmap v555: 158 pgs: 158 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 46 KiB/s rd, 74 MiB/s wr, 1.26k op/s; 8.0 KiB/s, 0 objects/s recovering 2024-09-20T22:13:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:09 smithi067 ceph-mon[33511]: 1.a scrub starts 2024-09-20T22:13:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:09 smithi067 ceph-mon[33511]: 1.a scrub ok 2024-09-20T22:13:09.272 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:09.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:09.273 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:09 smithi060 ceph-mon[29694]: 6.10 scrub starts 2024-09-20T22:13:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:09 smithi060 ceph-mon[29694]: 6.10 scrub ok 2024-09-20T22:13:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:09 smithi060 ceph-mon[29694]: pgmap v555: 158 pgs: 158 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 46 KiB/s rd, 74 MiB/s wr, 1.26k op/s; 8.0 KiB/s, 0 objects/s recovering 2024-09-20T22:13:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:09 smithi060 ceph-mon[29694]: 1.a scrub starts 2024-09-20T22:13:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:09 smithi060 ceph-mon[29694]: 1.a scrub ok 2024-09-20T22:13:09.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:09 smithi060 ceph-mon[40290]: 6.10 scrub starts 2024-09-20T22:13:09.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:09 smithi060 ceph-mon[40290]: 6.10 scrub ok 2024-09-20T22:13:09.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:09 smithi060 ceph-mon[40290]: pgmap v555: 158 pgs: 158 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 46 KiB/s rd, 74 MiB/s wr, 1.26k op/s; 8.0 KiB/s, 0 objects/s recovering 2024-09-20T22:13:09.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:09 smithi060 ceph-mon[40290]: 1.a scrub starts 2024-09-20T22:13:09.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:09 smithi060 ceph-mon[40290]: 1.a scrub ok 2024-09-20T22:13:09.374 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:09.408 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:09.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:09.409 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:09.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:09.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:09.552 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:09.552 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:09.653 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:09.689 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:09.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:09.691 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:09.791 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:09.824 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:09.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:09.825 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:09.908 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 82 15 101793 101778 77.5554 71.375 0.0192505 0.0128894 2024-09-20T22:13:09.926 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:09.959 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:09.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:09.960 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:10.061 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:10.093 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:10.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:10.094 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:10.195 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:10.220 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:10.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:10.221 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:10.229 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.2 reset_purged_snaps_last 2024-09-20T22:13:10.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:10 smithi060 ceph-mon[29694]: 4.1d scrub starts 2024-09-20T22:13:10.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:10 smithi060 ceph-mon[29694]: 4.1d scrub ok 2024-09-20T22:13:10.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:10 smithi060 ceph-mon[29694]: 1.6 scrub starts 2024-09-20T22:13:10.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:10 smithi060 ceph-mon[29694]: 1.6 scrub ok 2024-09-20T22:13:10.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:10 smithi060 ceph-mon[29694]: 3.0 scrub starts 2024-09-20T22:13:10.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:10 smithi060 ceph-mon[29694]: 3.0 scrub ok 2024-09-20T22:13:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:10 smithi060 ceph-mon[40290]: 4.1d scrub starts 2024-09-20T22:13:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:10 smithi060 ceph-mon[40290]: 4.1d scrub ok 2024-09-20T22:13:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:10 smithi060 ceph-mon[40290]: 1.6 scrub starts 2024-09-20T22:13:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:10 smithi060 ceph-mon[40290]: 1.6 scrub ok 2024-09-20T22:13:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:10 smithi060 ceph-mon[40290]: 3.0 scrub starts 2024-09-20T22:13:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:10 smithi060 ceph-mon[40290]: 3.0 scrub ok 2024-09-20T22:13:10.322 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:10.354 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:10.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:10.355 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:10.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:10.500 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:10.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:10.501 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:10.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:10 smithi067 ceph-mon[33511]: 4.1d scrub starts 2024-09-20T22:13:10.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:10 smithi067 ceph-mon[33511]: 4.1d scrub ok 2024-09-20T22:13:10.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:10 smithi067 ceph-mon[33511]: 1.6 scrub starts 2024-09-20T22:13:10.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:10 smithi067 ceph-mon[33511]: 1.6 scrub ok 2024-09-20T22:13:10.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:10 smithi067 ceph-mon[33511]: 3.0 scrub starts 2024-09-20T22:13:10.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:10 smithi067 ceph-mon[33511]: 3.0 scrub ok 2024-09-20T22:13:10.602 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:10.629 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:10.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:10.630 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:10.636 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:10.731 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:10.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:10.769 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:10.769 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:10.870 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:10.908 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 83 16 102811 102795 77.3868 63.5625 0.0177718 0.0129172 2024-09-20T22:13:10.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:10.909 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:10.909 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:11.010 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:11.055 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:11.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:11.056 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:11.156 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:11.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:11.202 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:11.202 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:11 smithi060 ceph-mon[29694]: 5.c scrub starts 2024-09-20T22:13:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:11 smithi060 ceph-mon[29694]: 5.c scrub ok 2024-09-20T22:13:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:11 smithi060 ceph-mon[29694]: pgmap v556: 158 pgs: 158 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 7.4 KiB/s rd, 58 MiB/s wr, 937 op/s 2024-09-20T22:13:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:11 smithi060 ceph-mon[29694]: 4.8 scrub starts 2024-09-20T22:13:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:11 smithi060 ceph-mon[29694]: 4.8 scrub ok 2024-09-20T22:13:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:11 smithi060 ceph-mon[29694]: 4.17 scrub starts 2024-09-20T22:13:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:11 smithi060 ceph-mon[29694]: 4.17 scrub ok 2024-09-20T22:13:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:11 smithi060 ceph-mon[40290]: 5.c scrub starts 2024-09-20T22:13:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:11 smithi060 ceph-mon[40290]: 5.c scrub ok 2024-09-20T22:13:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:11 smithi060 ceph-mon[40290]: pgmap v556: 158 pgs: 158 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 7.4 KiB/s rd, 58 MiB/s wr, 937 op/s 2024-09-20T22:13:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:11 smithi060 ceph-mon[40290]: 4.8 scrub starts 2024-09-20T22:13:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:11 smithi060 ceph-mon[40290]: 4.8 scrub ok 2024-09-20T22:13:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:11 smithi060 ceph-mon[40290]: 4.17 scrub starts 2024-09-20T22:13:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:11 smithi060 ceph-mon[40290]: 4.17 scrub ok 2024-09-20T22:13:11.302 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:11.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:11.336 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:11.414 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:11 smithi067 ceph-mon[33511]: 5.c scrub starts 2024-09-20T22:13:11.414 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:11 smithi067 ceph-mon[33511]: 5.c scrub ok 2024-09-20T22:13:11.414 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:11 smithi067 ceph-mon[33511]: pgmap v556: 158 pgs: 158 active+clean; 6.0 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 7.4 KiB/s rd, 58 MiB/s wr, 937 op/s 2024-09-20T22:13:11.414 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:11 smithi067 ceph-mon[33511]: 4.8 scrub starts 2024-09-20T22:13:11.415 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:11 smithi067 ceph-mon[33511]: 4.8 scrub ok 2024-09-20T22:13:11.415 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:11 smithi067 ceph-mon[33511]: 4.17 scrub starts 2024-09-20T22:13:11.415 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:11 smithi067 ceph-mon[33511]: 4.17 scrub ok 2024-09-20T22:13:11.437 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:11.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:11.477 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:11.477 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:11.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:11.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:11.614 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:11.614 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:11.714 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:11.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:11.749 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:11.749 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:11.850 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:11.886 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:11.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:11.887 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:11.908 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 84 16 103915 103899 77.2869 69 0.00942692 0.0129334 2024-09-20T22:13:11.988 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:12.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:12.022 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:12.022 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:12.123 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:12.158 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:12 smithi067 ceph-mon[33511]: 4.9 scrub starts 2024-09-20T22:13:12.158 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:12 smithi067 ceph-mon[33511]: 4.9 scrub ok 2024-09-20T22:13:12.158 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:12 smithi067 ceph-mon[33511]: 6.3 scrub starts 2024-09-20T22:13:12.159 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:12 smithi067 ceph-mon[33511]: 6.3 scrub ok 2024-09-20T22:13:12.159 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:12 smithi067 ceph-mon[33511]: 5.1d deep-scrub starts 2024-09-20T22:13:12.159 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:12 smithi067 ceph-mon[33511]: 5.1d deep-scrub ok 2024-09-20T22:13:12.159 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:12 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2494983580' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:12.159 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:12.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:12.160 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:12.184 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:13:12.184 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:13:12.184 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:13:12.184 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:13:12.184 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:13:12.184 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:13:12.184 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:13:12.184 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:13:12.185 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:13:12.185 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:13:12.185 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:13:12.185 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:13:12.185 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:13:12.185 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:13:12.185 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:13:12.185 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:13:12.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:12.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:12 smithi060 ceph-mon[29694]: 4.9 scrub starts 2024-09-20T22:13:12.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:12 smithi060 ceph-mon[29694]: 4.9 scrub ok 2024-09-20T22:13:12.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:12 smithi060 ceph-mon[29694]: 6.3 scrub starts 2024-09-20T22:13:12.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:12 smithi060 ceph-mon[29694]: 6.3 scrub ok 2024-09-20T22:13:12.288 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:12 smithi060 ceph-mon[29694]: 5.1d deep-scrub starts 2024-09-20T22:13:12.288 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:12 smithi060 ceph-mon[29694]: 5.1d deep-scrub ok 2024-09-20T22:13:12.288 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:12 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2494983580' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:12.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:12 smithi060 ceph-mon[40290]: 4.9 scrub starts 2024-09-20T22:13:12.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:12 smithi060 ceph-mon[40290]: 4.9 scrub ok 2024-09-20T22:13:12.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:12 smithi060 ceph-mon[40290]: 6.3 scrub starts 2024-09-20T22:13:12.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:12 smithi060 ceph-mon[40290]: 6.3 scrub ok 2024-09-20T22:13:12.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:12 smithi060 ceph-mon[40290]: 5.1d deep-scrub starts 2024-09-20T22:13:12.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:12 smithi060 ceph-mon[40290]: 5.1d deep-scrub ok 2024-09-20T22:13:12.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:12 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2494983580' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:12.299 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:12.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:12.301 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:12.401 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:12.449 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:12.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:12.450 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:12.513 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:13:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:13:12.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:12.584 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:12.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:12.585 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:12.686 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:12.727 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:12.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:12.728 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:12.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:12.869 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:12.876 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:12.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:12.877 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:12.911 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 85 16 105040 105024 77.202 70.3125 0.0147625 0.0129479 2024-09-20T22:13:12.978 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:13.022 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:13.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:13.023 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:13.124 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:13.169 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:13.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:13.170 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:13.189 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:13 smithi060 ceph-mon[29694]: 4.d scrub starts 2024-09-20T22:13:13.189 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:13 smithi060 ceph-mon[29694]: 4.d scrub ok 2024-09-20T22:13:13.189 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:13 smithi060 ceph-mon[29694]: 3.18 scrub starts 2024-09-20T22:13:13.189 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:13 smithi060 ceph-mon[29694]: 3.18 scrub ok 2024-09-20T22:13:13.189 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:13 smithi060 ceph-mon[29694]: 3.15 scrub starts 2024-09-20T22:13:13.189 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:13 smithi060 ceph-mon[29694]: 3.15 scrub ok 2024-09-20T22:13:13.189 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:13 smithi060 ceph-mon[29694]: pgmap v557: 158 pgs: 158 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-20T22:13:13.189 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:13 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:13:13.190 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:13 smithi060 ceph-mon[40290]: 4.d scrub starts 2024-09-20T22:13:13.190 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:13 smithi060 ceph-mon[40290]: 4.d scrub ok 2024-09-20T22:13:13.190 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:13 smithi060 ceph-mon[40290]: 3.18 scrub starts 2024-09-20T22:13:13.190 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:13 smithi060 ceph-mon[40290]: 3.18 scrub ok 2024-09-20T22:13:13.190 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:13 smithi060 ceph-mon[40290]: 3.15 scrub starts 2024-09-20T22:13:13.190 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:13 smithi060 ceph-mon[40290]: 3.15 scrub ok 2024-09-20T22:13:13.190 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:13 smithi060 ceph-mon[40290]: pgmap v557: 158 pgs: 158 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-20T22:13:13.190 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:13 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:13:13.271 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:13.316 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:13.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:13.318 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:13.386 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:13 smithi067 ceph-mon[33511]: 4.d scrub starts 2024-09-20T22:13:13.386 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:13 smithi067 ceph-mon[33511]: 4.d scrub ok 2024-09-20T22:13:13.386 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:13 smithi067 ceph-mon[33511]: 3.18 scrub starts 2024-09-20T22:13:13.386 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:13 smithi067 ceph-mon[33511]: 3.18 scrub ok 2024-09-20T22:13:13.386 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:13 smithi067 ceph-mon[33511]: 3.15 scrub starts 2024-09-20T22:13:13.387 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:13 smithi067 ceph-mon[33511]: 3.15 scrub ok 2024-09-20T22:13:13.387 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:13 smithi067 ceph-mon[33511]: pgmap v557: 158 pgs: 158 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.2 KiB/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-20T22:13:13.387 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:13 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:13:13.418 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:13.460 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:13.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:13.462 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:13.563 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:13.603 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:13.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:13.605 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:13.626 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:13:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:13:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:13:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:13:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227609 7.4 7:26fdb431:::benchmark_data_smithi060_78800_object94576:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-20T22:13:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.802526+0000", 2024-09-20T22:13:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 379.81784866499999, 2024-09-20T22:13:13.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.327846133, 2024-09-20T22:13:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:13:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227609 2024-09-20T22:13:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.627 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802526+0000", 2024-09-20T22:13:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802526+0000", 2024-09-20T22:13:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802528+0000", 2024-09-20T22:13:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2550000000000001e-06 2024-09-20T22:13:13.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802533+0000", 2024-09-20T22:13:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5500000000000002e-06 2024-09-20T22:13:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802542+0000", 2024-09-20T22:13:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6950000000000006e-06 2024-09-20T22:13:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.629 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802548+0000", 2024-09-20T22:13:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8100000000000003e-06 2024-09-20T22:13:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802787+0000", 2024-09-20T22:13:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023914699999999999 2024-09-20T22:13:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.802897+0000", 2024-09-20T22:13:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109645 2024-09-20T22:13:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4", 2024-09-20T22:13:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.803676+0000", 2024-09-20T22:13:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000779085 2024-09-20T22:13:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.814426+0000", 2024-09-20T22:13:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010750857000000001 2024-09-20T22:13:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.814439+0000", 2024-09-20T22:13:13.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2549e-05 2024-09-20T22:13:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.823484+0000", 2024-09-20T22:13:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0090446120000001 2024-09-20T22:13:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.823496+0000", 2024-09-20T22:13:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2413e-05 2024-09-20T22:13:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.115278+0000", 2024-09-20T22:13:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.291782427 2024-09-20T22:13:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:13:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.115476+0000", 2024-09-20T22:13:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019718599999999999 2024-09-20T22:13:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130062+0000", 2024-09-20T22:13:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014586541 2024-09-20T22:13:13.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130373+0000", 2024-09-20T22:13:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00031036099999999999 2024-09-20T22:13:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227614 7.4 7:22c1d0bc:::benchmark_data_smithi060_78800_object94581:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-20T22:13:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.804108+0000", 2024-09-20T22:13:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 379.81626666699998, 2024-09-20T22:13:13.634 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3264465649999999, 2024-09-20T22:13:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227614 2024-09-20T22:13:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.804108+0000", 2024-09-20T22:13:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.804108+0000", 2024-09-20T22:13:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.804110+0000", 2024-09-20T22:13:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1510000000000001e-06 2024-09-20T22:13:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.804115+0000", 2024-09-20T22:13:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0969999999999996e-06 2024-09-20T22:13:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.804128+0000", 2024-09-20T22:13:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2899e-05 2024-09-20T22:13:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.804136+0000", 2024-09-20T22:13:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0479999999999999e-06 2024-09-20T22:13:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.814469+0000", 2024-09-20T22:13:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010333187000000001 2024-09-20T22:13:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.814478+0000", 2024-09-20T22:13:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9530000000000005e-06 2024-09-20T22:13:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.823863+0000", 2024-09-20T22:13:13.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0093847810000001 2024-09-20T22:13:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.823872+0000", 2024-09-20T22:13:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.312e-06 2024-09-20T22:13:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.115762+0000", 2024-09-20T22:13:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.291889702 2024-09-20T22:13:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.115904+0000", 2024-09-20T22:13:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014230800000000001 2024-09-20T22:13:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:13:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.116318+0000", 2024-09-20T22:13:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00041407100000000001 2024-09-20T22:13:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.123684+0000", 2024-09-20T22:13:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0073657769999999996 2024-09-20T22:13:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.129612+0000", 2024-09-20T22:13:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0059283519999999996 2024-09-20T22:13:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130418+0000", 2024-09-20T22:13:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00080579299999999996 2024-09-20T22:13:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130473+0000", 2024-09-20T22:13:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4740000000000001e-05 2024-09-20T22:13:13.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130555+0000", 2024-09-20T22:13:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2393999999999998e-05 2024-09-20T22:13:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227652 7.4 7:213b8ad4:::benchmark_data_smithi060_78800_object94619:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:13:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.904523+0000", 2024-09-20T22:13:13.642 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 379.71585255799999, 2024-09-20T22:13:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2262305599999999, 2024-09-20T22:13:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227652 2024-09-20T22:13:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904523+0000", 2024-09-20T22:13:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904523+0000", 2024-09-20T22:13:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904524+0000", 2024-09-20T22:13:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0100000000000001e-06 2024-09-20T22:13:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904528+0000", 2024-09-20T22:13:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4710000000000001e-06 2024-09-20T22:13:13.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904537+0000", 2024-09-20T22:13:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2289999999999992e-06 2024-09-20T22:13:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904544+0000", 2024-09-20T22:13:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6900000000000003e-06 2024-09-20T22:13:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904608+0000", 2024-09-20T22:13:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3598999999999999e-05 2024-09-20T22:13:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.904617+0000", 2024-09-20T22:13:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8179999999999999e-06 2024-09-20T22:13:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.823899+0000", 2024-09-20T22:13:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.919281338 2024-09-20T22:13:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.823907+0000", 2024-09-20T22:13:13.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9449999999999993e-06 2024-09-20T22:13:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.118500+0000", 2024-09-20T22:13:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2945934700000001 2024-09-20T22:13:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.118634+0000", 2024-09-20T22:13:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013352700000000001 2024-09-20T22:13:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:13:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.119027+0000", 2024-09-20T22:13:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000393533 2024-09-20T22:13:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.129598+0000", 2024-09-20T22:13:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010570941 2024-09-20T22:13:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:13.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.129619+0000", 2024-09-20T22:13:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0854000000000001e-05 2024-09-20T22:13:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130623+0000", 2024-09-20T22:13:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001003947 2024-09-20T22:13:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130663+0000", 2024-09-20T22:13:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0139000000000003e-05 2024-09-20T22:13:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130753+0000", 2024-09-20T22:13:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0049000000000005e-05 2024-09-20T22:13:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227664 7.4 7:264fd802:::benchmark_data_smithi060_78800_object94631:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:13:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.969131+0000", 2024-09-20T22:13:13.650 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 379.65124378199999, 2024-09-20T22:13:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1748220109999998, 2024-09-20T22:13:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227664 2024-09-20T22:13:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.651 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969131+0000", 2024-09-20T22:13:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969131+0000", 2024-09-20T22:13:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969133+0000", 2024-09-20T22:13:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.421e-06 2024-09-20T22:13:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969138+0000", 2024-09-20T22:13:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9239999999999998e-06 2024-09-20T22:13:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969148+0000", 2024-09-20T22:13:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0696000000000001e-05 2024-09-20T22:13:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969155+0000", 2024-09-20T22:13:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7780000000000002e-06 2024-09-20T22:13:13.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969217+0000", 2024-09-20T22:13:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2304000000000005e-05 2024-09-20T22:13:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.969229+0000", 2024-09-20T22:13:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1100999999999999e-05 2024-09-20T22:13:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.824751+0000", 2024-09-20T22:13:13.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85552231099999998 2024-09-20T22:13:13.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.824761+0000", 2024-09-20T22:13:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9210000000000004e-06 2024-09-20T22:13:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.126989+0000", 2024-09-20T22:13:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.302228449 2024-09-20T22:13:13.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.127107+0000", 2024-09-20T22:13:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117712 2024-09-20T22:13:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:13:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.127530+0000", 2024-09-20T22:13:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000423056 2024-09-20T22:13:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.130029+0000", 2024-09-20T22:13:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024987020000000002 2024-09-20T22:13:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.133706+0000", 2024-09-20T22:13:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0036771659999999999 2024-09-20T22:13:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.658 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.143463+0000", 2024-09-20T22:13:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0097571510000000004 2024-09-20T22:13:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.143714+0000", 2024-09-20T22:13:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00025083199999999999 2024-09-20T22:13:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.143953+0000", 2024-09-20T22:13:13.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023948700000000001 2024-09-20T22:13:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236412 7.9 7:953e1373:::benchmark_data_smithi060_78800_object103379:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-20T22:13:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.364320+0000", 2024-09-20T22:13:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 366.256055167, 2024-09-20T22:13:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1050446030000001, 2024-09-20T22:13:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.660 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:13:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236412 2024-09-20T22:13:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364320+0000", 2024-09-20T22:13:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364320+0000", 2024-09-20T22:13:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364321+0000", 2024-09-20T22:13:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.533e-06 2024-09-20T22:13:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364328+0000", 2024-09-20T22:13:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.241e-06 2024-09-20T22:13:13.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364346+0000", 2024-09-20T22:13:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8546000000000002e-05 2024-09-20T22:13:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364354+0000", 2024-09-20T22:13:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6839999999999996e-06 2024-09-20T22:13:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364429+0000", 2024-09-20T22:13:13.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4618999999999998e-05 2024-09-20T22:13:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364584+0000", 2024-09-20T22:13:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155725 2024-09-20T22:13:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:13:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.367359+0000", 2024-09-20T22:13:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002774421 2024-09-20T22:13:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.379883+0000", 2024-09-20T22:13:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.012524208 2024-09-20T22:13:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.379895+0000", 2024-09-20T22:13:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2268e-05 2024-09-20T22:13:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.421975+0000", 2024-09-20T22:13:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042079623 2024-09-20T22:13:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:13:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.422090+0000", 2024-09-20T22:13:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011518000000000001 2024-09-20T22:13:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.469296+0000", 2024-09-20T22:13:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.047205907999999998 2024-09-20T22:13:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.469365+0000", 2024-09-20T22:13:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8646999999999995e-05 2024-09-20T22:13:13.666 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236413 7.9 7:92cb2bc7:::benchmark_data_smithi060_78800_object103380:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e151)", 2024-09-20T22:13:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.364373+0000", 2024-09-20T22:13:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 366.25600190799997, 2024-09-20T22:13:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.105379702, 2024-09-20T22:13:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:13:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.667 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236413 2024-09-20T22:13:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364373+0000", 2024-09-20T22:13:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364373+0000", 2024-09-20T22:13:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364374+0000", 2024-09-20T22:13:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2100000000000002e-07 2024-09-20T22:13:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364376+0000", 2024-09-20T22:13:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5440000000000001e-06 2024-09-20T22:13:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364383+0000", 2024-09-20T22:13:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7870000000000004e-06 2024-09-20T22:13:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.364389+0000", 2024-09-20T22:13:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6540000000000004e-06 2024-09-20T22:13:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.374885+0000", 2024-09-20T22:13:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010496871 2024-09-20T22:13:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.375004+0000", 2024-09-20T22:13:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118956 2024-09-20T22:13:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:13:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.375466+0000", 2024-09-20T22:13:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00046160599999999998 2024-09-20T22:13:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.379926+0000", 2024-09-20T22:13:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0044598850000000002 2024-09-20T22:13:13.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.379935+0000", 2024-09-20T22:13:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8140000000000007e-06 2024-09-20T22:13:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.422377+0000", 2024-09-20T22:13:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.042442028 2024-09-20T22:13:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:13:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.422482+0000", 2024-09-20T22:13:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010542 2024-09-20T22:13:13.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.469677+0000", 2024-09-20T22:13:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.047194699 2024-09-20T22:13:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.469753+0000", 2024-09-20T22:13:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6017000000000001e-05 2024-09-20T22:13:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.673 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106977 8.12 8:4ad51fd6:::benchmark_data_smithi060_111051_object106976:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:13:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418139+0000", 2024-09-20T22:13:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 242.20223653100001, 2024-09-20T22:13:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.107260704, 2024-09-20T22:13:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106977 2024-09-20T22:13:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.674 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418139+0000", 2024-09-20T22:13:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418139+0000", 2024-09-20T22:13:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418139+0000", 2024-09-20T22:13:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4300000000000003e-07 2024-09-20T22:13:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418194+0000", 2024-09-20T22:13:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4564999999999997e-05 2024-09-20T22:13:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418202+0000", 2024-09-20T22:13:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.755e-06 2024-09-20T22:13:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418211+0000", 2024-09-20T22:13:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5199999999999997e-06 2024-09-20T22:13:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418518+0000", 2024-09-20T22:13:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030665299999999999 2024-09-20T22:13:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418525+0000", 2024-09-20T22:13:13.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4930000000000001e-06 2024-09-20T22:13:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.508187+0000", 2024-09-20T22:13:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0896622950000001 2024-09-20T22:13:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.508457+0000", 2024-09-20T22:13:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00027002800000000001 2024-09-20T22:13:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,4,5,6,7", 2024-09-20T22:13:13.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.508645+0000", 2024-09-20T22:13:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018733000000000001 2024-09-20T22:13:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.514513+0000", 2024-09-20T22:13:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0058678259999999996 2024-09-20T22:13:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.514560+0000", 2024-09-20T22:13:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7314999999999997e-05 2024-09-20T22:13:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.514664+0000", 2024-09-20T22:13:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104125 2024-09-20T22:13:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515122+0000", 2024-09-20T22:13:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00045837200000000002 2024-09-20T22:13:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515704+0000", 2024-09-20T22:13:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00058129899999999997 2024-09-20T22:13:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525183+0000", 2024-09-20T22:13:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.009479019 2024-09-20T22:13:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525256+0000", 2024-09-20T22:13:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3410000000000004e-05 2024-09-20T22:13:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525399+0000", 2024-09-20T22:13:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000143156 2024-09-20T22:13:13.681 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106979 8.16 8:6c64d7d1:::benchmark_data_smithi060_111051_object106978:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:13:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418222+0000", 2024-09-20T22:13:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 242.20215345700001, 2024-09-20T22:13:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.107288856, 2024-09-20T22:13:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:13.682 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106979 2024-09-20T22:13:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418222+0000", 2024-09-20T22:13:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418222+0000", 2024-09-20T22:13:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418222+0000", 2024-09-20T22:13:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1900000000000003e-07 2024-09-20T22:13:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418273+0000", 2024-09-20T22:13:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0751000000000002e-05 2024-09-20T22:13:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418282+0000", 2024-09-20T22:13:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1439999999999992e-06 2024-09-20T22:13:13.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418289+0000", 2024-09-20T22:13:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4829999999999996e-06 2024-09-20T22:13:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418324+0000", 2024-09-20T22:13:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5508e-05 2024-09-20T22:13:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418331+0000", 2024-09-20T22:13:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2679999999999997e-06 2024-09-20T22:13:13.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.512703+0000", 2024-09-20T22:13:13.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.094371502 2024-09-20T22:13:13.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.512972+0000", 2024-09-20T22:13:13.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026964600000000001 2024-09-20T22:13:13.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5,6,7", 2024-09-20T22:13:13.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.513953+0000", 2024-09-20T22:13:13.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00098022600000000006 2024-09-20T22:13:13.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:13.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.519844+0000", 2024-09-20T22:13:13.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005891309 2024-09-20T22:13:13.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.520362+0000", 2024-09-20T22:13:13.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00051799699999999997 2024-09-20T22:13:13.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.520890+0000", 2024-09-20T22:13:13.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00052754400000000002 2024-09-20T22:13:13.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524659+0000", 2024-09-20T22:13:13.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037695160000000001 2024-09-20T22:13:13.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524693+0000", 2024-09-20T22:13:13.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3887999999999998e-05 2024-09-20T22:13:13.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525239+0000", 2024-09-20T22:13:13.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00054655600000000002 2024-09-20T22:13:13.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:13.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525313+0000", 2024-09-20T22:13:13.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3485999999999994e-05 2024-09-20T22:13:13.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525510+0000", 2024-09-20T22:13:13.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019751299999999999 2024-09-20T22:13:13.689 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.689 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.690 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.690 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106981 8.16 8:6961ec02:::benchmark_data_smithi060_111051_object106980:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:13:13.690 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418309+0000", 2024-09-20T22:13:13.690 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 242.202066328, 2024-09-20T22:13:13.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1092024140000001, 2024-09-20T22:13:13.690 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.690 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.690 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:13.690 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.690 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:13.690 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:13.691 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106981 2024-09-20T22:13:13.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.691 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418309+0000", 2024-09-20T22:13:13.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418309+0000", 2024-09-20T22:13:13.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418309+0000", 2024-09-20T22:13:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3000000000000001e-07 2024-09-20T22:13:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418370+0000", 2024-09-20T22:13:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0577e-05 2024-09-20T22:13:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418379+0000", 2024-09-20T22:13:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5279999999999992e-06 2024-09-20T22:13:13.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418385+0000", 2024-09-20T22:13:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.105e-06 2024-09-20T22:13:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418587+0000", 2024-09-20T22:13:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020189700000000001 2024-09-20T22:13:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418595+0000", 2024-09-20T22:13:13.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.187e-06 2024-09-20T22:13:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515314+0000", 2024-09-20T22:13:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0967195460000001 2024-09-20T22:13:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515442+0000", 2024-09-20T22:13:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012829599999999999 2024-09-20T22:13:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5,6,7", 2024-09-20T22:13:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515706+0000", 2024-09-20T22:13:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026385599999999998 2024-09-20T22:13:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.519902+0000", 2024-09-20T22:13:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.004195786 2024-09-20T22:13:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.520407+0000", 2024-09-20T22:13:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00050457400000000004 2024-09-20T22:13:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524729+0000", 2024-09-20T22:13:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0043228279999999999 2024-09-20T22:13:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524770+0000", 2024-09-20T22:13:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0528000000000002e-05 2024-09-20T22:13:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524965+0000", 2024-09-20T22:13:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019492000000000001 2024-09-20T22:13:13.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.527323+0000", 2024-09-20T22:13:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023577250000000002 2024-09-20T22:13:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.527382+0000", 2024-09-20T22:13:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9228999999999998e-05 2024-09-20T22:13:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.527511+0000", 2024-09-20T22:13:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129302 2024-09-20T22:13:13.697 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.698 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.698 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106997 8.16 8:6b0b3968:::benchmark_data_smithi060_111051_object106996:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:13:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418412+0000", 2024-09-20T22:13:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 242.20196339200001, 2024-09-20T22:13:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1094631589999999, 2024-09-20T22:13:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106997 2024-09-20T22:13:13.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418412+0000", 2024-09-20T22:13:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418412+0000", 2024-09-20T22:13:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418412+0000", 2024-09-20T22:13:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.13e-07 2024-09-20T22:13:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418461+0000", 2024-09-20T22:13:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8504999999999997e-05 2024-09-20T22:13:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418470+0000", 2024-09-20T22:13:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4660000000000005e-06 2024-09-20T22:13:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418476+0000", 2024-09-20T22:13:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2179999999999996e-06 2024-09-20T22:13:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418618+0000", 2024-09-20T22:13:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014114800000000001 2024-09-20T22:13:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418624+0000", 2024-09-20T22:13:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8009999999999998e-06 2024-09-20T22:13:13.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.517728+0000", 2024-09-20T22:13:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.099103513 2024-09-20T22:13:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.517868+0000", 2024-09-20T22:13:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014029400000000001 2024-09-20T22:13:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5,6,7", 2024-09-20T22:13:13.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.518178+0000", 2024-09-20T22:13:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030979800000000002 2024-09-20T22:13:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.519909+0000", 2024-09-20T22:13:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017308740000000001 2024-09-20T22:13:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.522366+0000", 2024-09-20T22:13:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024576609999999999 2024-09-20T22:13:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524823+0000", 2024-09-20T22:13:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024567830000000001 2024-09-20T22:13:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524862+0000", 2024-09-20T22:13:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9060000000000002e-05 2024-09-20T22:13:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525002+0000", 2024-09-20T22:13:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000139438 2024-09-20T22:13:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.527697+0000", 2024-09-20T22:13:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0026952149999999999 2024-09-20T22:13:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.527759+0000", 2024-09-20T22:13:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1612999999999994e-05 2024-09-20T22:13:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.527875+0000", 2024-09-20T22:13:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000116259 2024-09-20T22:13:13.706 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:107002 8.11 8:8e84af07:::benchmark_data_smithi060_111051_object107001:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:13:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418488+0000", 2024-09-20T22:13:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 242.20188730800001, 2024-09-20T22:13:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1327261360000001, 2024-09-20T22:13:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:13.707 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 107002 2024-09-20T22:13:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418488+0000", 2024-09-20T22:13:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418488+0000", 2024-09-20T22:13:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418488+0000", 2024-09-20T22:13:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4700000000000001e-07 2024-09-20T22:13:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418536+0000", 2024-09-20T22:13:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7899999999999999e-05 2024-09-20T22:13:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418545+0000", 2024-09-20T22:13:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5210000000000002e-06 2024-09-20T22:13:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418551+0000", 2024-09-20T22:13:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.409e-06 2024-09-20T22:13:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418608+0000", 2024-09-20T22:13:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7015999999999998e-05 2024-09-20T22:13:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418619+0000", 2024-09-20T22:13:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0900999999999999e-05 2024-09-20T22:13:13.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.538830+0000", 2024-09-20T22:13:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.120210792 2024-09-20T22:13:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.538979+0000", 2024-09-20T22:13:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014885199999999999 2024-09-20T22:13:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2,5,6,7", 2024-09-20T22:13:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.539152+0000", 2024-09-20T22:13:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000173134 2024-09-20T22:13:13.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.543638+0000", 2024-09-20T22:13:13.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0044860739999999996 2024-09-20T22:13:13.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.546226+0000", 2024-09-20T22:13:13.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00258807 2024-09-20T22:13:13.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.546267+0000", 2024-09-20T22:13:13.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0592e-05 2024-09-20T22:13:13.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.546317+0000", 2024-09-20T22:13:13.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0781000000000001e-05 2024-09-20T22:13:13.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:13.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.546370+0000", 2024-09-20T22:13:13.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2358999999999998e-05 2024-09-20T22:13:13.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.550100+0000", 2024-09-20T22:13:13.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037306309999999999 2024-09-20T22:13:13.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:13.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.551067+0000", 2024-09-20T22:13:13.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00096712000000000002 2024-09-20T22:13:13.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.551214+0000", 2024-09-20T22:13:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014643699999999999 2024-09-20T22:13:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:107007 8.12 8:4e9d1372:::benchmark_data_smithi060_111051_object107006:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:13:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418568+0000", 2024-09-20T22:13:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 242.20180730600001, 2024-09-20T22:13:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.108179131, 2024-09-20T22:13:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:13.715 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 107007 2024-09-20T22:13:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418568+0000", 2024-09-20T22:13:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418568+0000", 2024-09-20T22:13:13.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418568+0000", 2024-09-20T22:13:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3499999999999996e-07 2024-09-20T22:13:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418616+0000", 2024-09-20T22:13:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7386000000000003e-05 2024-09-20T22:13:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418624+0000", 2024-09-20T22:13:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1110000000000008e-06 2024-09-20T22:13:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418630+0000", 2024-09-20T22:13:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9390000000000002e-06 2024-09-20T22:13:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418675+0000", 2024-09-20T22:13:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5444000000000001e-05 2024-09-20T22:13:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418683+0000", 2024-09-20T22:13:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5140000000000004e-06 2024-09-20T22:13:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.510695+0000", 2024-09-20T22:13:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.092012032 2024-09-20T22:13:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.510853+0000", 2024-09-20T22:13:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015822899999999999 2024-09-20T22:13:13.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,4,5,6,7", 2024-09-20T22:13:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.511044+0000", 2024-09-20T22:13:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019054099999999999 2024-09-20T22:13:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.514574+0000", 2024-09-20T22:13:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003530858 2024-09-20T22:13:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515032+0000", 2024-09-20T22:13:13.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00045793899999999998 2024-09-20T22:13:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515080+0000", 2024-09-20T22:13:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8164999999999997e-05 2024-09-20T22:13:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515664+0000", 2024-09-20T22:13:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00058304799999999996 2024-09-20T22:13:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515792+0000", 2024-09-20T22:13:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012877 2024-09-20T22:13:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525673+0000", 2024-09-20T22:13:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0098810990000000008 2024-09-20T22:13:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.526607+0000", 2024-09-20T22:13:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00093347000000000005 2024-09-20T22:13:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.526747+0000", 2024-09-20T22:13:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140051 2024-09-20T22:13:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:107008 8.12 8:4cb7847c:::benchmark_data_smithi060_111051_object107007:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:13:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418644+0000", 2024-09-20T22:13:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 242.20173146499999, 2024-09-20T22:13:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1083103759999999, 2024-09-20T22:13:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.723 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 107008 2024-09-20T22:13:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418644+0000", 2024-09-20T22:13:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418644+0000", 2024-09-20T22:13:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418644+0000", 2024-09-20T22:13:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2e-07 2024-09-20T22:13:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418693+0000", 2024-09-20T22:13:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8569000000000002e-05 2024-09-20T22:13:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418702+0000", 2024-09-20T22:13:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8060000000000001e-06 2024-09-20T22:13:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418708+0000", 2024-09-20T22:13:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2500000000000003e-06 2024-09-20T22:13:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418797+0000", 2024-09-20T22:13:13.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9543999999999995e-05 2024-09-20T22:13:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418805+0000", 2024-09-20T22:13:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0290000000000005e-06 2024-09-20T22:13:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.512330+0000", 2024-09-20T22:13:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0935242919999999 2024-09-20T22:13:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.512473+0000", 2024-09-20T22:13:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000143695 2024-09-20T22:13:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,4,5,6,7", 2024-09-20T22:13:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.513190+0000", 2024-09-20T22:13:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00071678699999999998 2024-09-20T22:13:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.518925+0000", 2024-09-20T22:13:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0057347919999999998 2024-09-20T22:13:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.520073+0000", 2024-09-20T22:13:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0011479420000000001 2024-09-20T22:13:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.520313+0000", 2024-09-20T22:13:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023994399999999999 2024-09-20T22:13:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.523625+0000", 2024-09-20T22:13:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003312008 2024-09-20T22:13:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524809+0000", 2024-09-20T22:13:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001184456 2024-09-20T22:13:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.526788+0000", 2024-09-20T22:13:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001979218 2024-09-20T22:13:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.526838+0000", 2024-09-20T22:13:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9617999999999998e-05 2024-09-20T22:13:13.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.526954+0000", 2024-09-20T22:13:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115906 2024-09-20T22:13:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187850 8.c 8:31fd15bc:::benchmark_data_smithi060_111051_object67851:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:13:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.627306+0000", 2024-09-20T22:13:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 163.99306924300001, 2024-09-20T22:13:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2448928590000001, 2024-09-20T22:13:13.731 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:13:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187850 2024-09-20T22:13:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627306+0000", 2024-09-20T22:13:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627306+0000", 2024-09-20T22:13:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627307+0000", 2024-09-20T22:13:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.288e-06 2024-09-20T22:13:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627312+0000", 2024-09-20T22:13:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7990000000000001e-06 2024-09-20T22:13:13.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627328+0000", 2024-09-20T22:13:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5909e-05 2024-09-20T22:13:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627335+0000", 2024-09-20T22:13:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1250000000000004e-06 2024-09-20T22:13:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627384+0000", 2024-09-20T22:13:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9218000000000002e-05 2024-09-20T22:13:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.630537+0000", 2024-09-20T22:13:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0031527109999999999 2024-09-20T22:13:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:13:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.630770+0000", 2024-09-20T22:13:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023290799999999999 2024-09-20T22:13:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.642588+0000", 2024-09-20T22:13:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.011817831000000001 2024-09-20T22:13:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.666253+0000", 2024-09-20T22:13:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.023664916000000001 2024-09-20T22:13:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.666266+0000", 2024-09-20T22:13:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3818999999999999e-05 2024-09-20T22:13:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.658763+0000", 2024-09-20T22:13:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99249631000000005 2024-09-20T22:13:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.658780+0000", 2024-09-20T22:13:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7697e-05 2024-09-20T22:13:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.860783+0000", 2024-09-20T22:13:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.202002518 2024-09-20T22:13:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:13:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.860911+0000", 2024-09-20T22:13:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012827499999999999 2024-09-20T22:13:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.871844+0000", 2024-09-20T22:13:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010932667 2024-09-20T22:13:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.872199+0000", 2024-09-20T22:13:13.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00035486799999999999 2024-09-20T22:13:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187851 8.c 8:3eebc9c2:::benchmark_data_smithi060_111051_object67852:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:13:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.627552+0000", 2024-09-20T22:13:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 163.992822633, 2024-09-20T22:13:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2481179880000002, 2024-09-20T22:13:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.739 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.739 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:13.740 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:13:13.740 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187851 2024-09-20T22:13:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627552+0000", 2024-09-20T22:13:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627552+0000", 2024-09-20T22:13:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627553+0000", 2024-09-20T22:13:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7599999999999996e-07 2024-09-20T22:13:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627558+0000", 2024-09-20T22:13:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.369e-06 2024-09-20T22:13:13.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627567+0000", 2024-09-20T22:13:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8199999999999992e-06 2024-09-20T22:13:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627574+0000", 2024-09-20T22:13:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4880000000000004e-06 2024-09-20T22:13:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.635469+0000", 2024-09-20T22:13:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0078950770000000003 2024-09-20T22:13:13.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.635599+0000", 2024-09-20T22:13:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012998 2024-09-20T22:13:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:13:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.635730+0000", 2024-09-20T22:13:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013132000000000001 2024-09-20T22:13:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:13.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.642594+0000", 2024-09-20T22:13:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0068636829999999998 2024-09-20T22:13:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.666296+0000", 2024-09-20T22:13:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.023701567 2024-09-20T22:13:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.666303+0000", 2024-09-20T22:13:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.695e-06 2024-09-20T22:13:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.745 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.661330+0000", 2024-09-20T22:13:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99502645599999995 2024-09-20T22:13:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.661341+0000", 2024-09-20T22:13:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1564e-05 2024-09-20T22:13:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.861170+0000", 2024-09-20T22:13:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1998287999999999 2024-09-20T22:13:13.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:13:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.861243+0000", 2024-09-20T22:13:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3425999999999998e-05 2024-09-20T22:13:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.875594+0000", 2024-09-20T22:13:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014350577 2024-09-20T22:13:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.875670+0000", 2024-09-20T22:13:13.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6390000000000006e-05 2024-09-20T22:13:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187854 8.c 8:37fa2592:::benchmark_data_smithi060_111051_object67855:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:13:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.632485+0000", 2024-09-20T22:13:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 163.98789024300001, 2024-09-20T22:13:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2502292339999999, 2024-09-20T22:13:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:13:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.748 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187854 2024-09-20T22:13:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.632485+0000", 2024-09-20T22:13:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.632485+0000", 2024-09-20T22:13:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.632486+0000", 2024-09-20T22:13:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5099999999999999e-06 2024-09-20T22:13:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.632492+0000", 2024-09-20T22:13:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.054e-06 2024-09-20T22:13:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.632506+0000", 2024-09-20T22:13:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3801999999999999e-05 2024-09-20T22:13:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.632516+0000", 2024-09-20T22:13:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6630000000000005e-06 2024-09-20T22:13:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.639423+0000", 2024-09-20T22:13:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0069074949999999996 2024-09-20T22:13:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.639563+0000", 2024-09-20T22:13:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000139414 2024-09-20T22:13:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:13:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.639705+0000", 2024-09-20T22:13:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000142242 2024-09-20T22:13:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.666329+0000", 2024-09-20T22:13:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.026623709999999998 2024-09-20T22:13:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.666336+0000", 2024-09-20T22:13:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1099999999999997e-06 2024-09-20T22:13:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.661373+0000", 2024-09-20T22:13:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99503708499999999 2024-09-20T22:13:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.661382+0000", 2024-09-20T22:13:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4360000000000008e-06 2024-09-20T22:13:13.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.861335+0000", 2024-09-20T22:13:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1999522309999999 2024-09-20T22:13:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:13:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.861490+0000", 2024-09-20T22:13:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015551599999999999 2024-09-20T22:13:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.882638+0000", 2024-09-20T22:13:13.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021147702000000001 2024-09-20T22:13:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.882714+0000", 2024-09-20T22:13:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6264e-05 2024-09-20T22:13:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187861 8.c 8:3c873a3d:::benchmark_data_smithi060_111051_object67862:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:13:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.662679+0000", 2024-09-20T22:13:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 163.95769652000001, 2024-09-20T22:13:13.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2203969990000001, 2024-09-20T22:13:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187861 2024-09-20T22:13:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.662679+0000", 2024-09-20T22:13:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.662679+0000", 2024-09-20T22:13:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.662680+0000", 2024-09-20T22:13:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.471e-06 2024-09-20T22:13:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.662686+0000", 2024-09-20T22:13:13.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9889999999999997e-06 2024-09-20T22:13:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.662710+0000", 2024-09-20T22:13:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3574000000000002e-05 2024-09-20T22:13:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.663765+0000", 2024-09-20T22:13:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0010555689999999999 2024-09-20T22:13:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.666358+0000", 2024-09-20T22:13:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002593181 2024-09-20T22:13:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.666365+0000", 2024-09-20T22:13:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.319e-06 2024-09-20T22:13:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.661413+0000", 2024-09-20T22:13:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.995048826 2024-09-20T22:13:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.661422+0000", 2024-09-20T22:13:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7600000000000008e-06 2024-09-20T22:13:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.861522+0000", 2024-09-20T22:13:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.200099936 2024-09-20T22:13:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.861627+0000", 2024-09-20T22:13:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105011 2024-09-20T22:13:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4,5", 2024-09-20T22:13:13.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.862042+0000", 2024-09-20T22:13:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00041462200000000002 2024-09-20T22:13:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.871270+0000", 2024-09-20T22:13:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0092279939999999998 2024-09-20T22:13:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.871416+0000", 2024-09-20T22:13:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014590600000000001 2024-09-20T22:13:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.875258+0000", 2024-09-20T22:13:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038427219999999998 2024-09-20T22:13:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.882916+0000", 2024-09-20T22:13:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0076572790000000003 2024-09-20T22:13:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.882971+0000", 2024-09-20T22:13:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5485999999999997e-05 2024-09-20T22:13:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.883076+0000", 2024-09-20T22:13:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104354 2024-09-20T22:13:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187897 8.c 8:363b713e:::benchmark_data_smithi060_111051_object67898:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:13:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.704486+0000", 2024-09-20T22:13:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 163.91588886700001, 2024-09-20T22:13:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1787772589999999, 2024-09-20T22:13:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.763 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187897 2024-09-20T22:13:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.704486+0000", 2024-09-20T22:13:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.704486+0000", 2024-09-20T22:13:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.704487+0000", 2024-09-20T22:13:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.048e-06 2024-09-20T22:13:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.704492+0000", 2024-09-20T22:13:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5700000000000003e-06 2024-09-20T22:13:13.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.704503+0000", 2024-09-20T22:13:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0927000000000001e-05 2024-09-20T22:13:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.704508+0000", 2024-09-20T22:13:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3800000000000002e-06 2024-09-20T22:13:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.704566+0000", 2024-09-20T22:13:13.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7751000000000003e-05 2024-09-20T22:13:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.704573+0000", 2024-09-20T22:13:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.109e-06 2024-09-20T22:13:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.661456+0000", 2024-09-20T22:13:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.956882813 2024-09-20T22:13:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.661465+0000", 2024-09-20T22:13:13.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2210000000000003e-06 2024-09-20T22:13:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.865340+0000", 2024-09-20T22:13:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2038746760000001 2024-09-20T22:13:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.865540+0000", 2024-09-20T22:13:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000200609 2024-09-20T22:13:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4,5", 2024-09-20T22:13:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.865840+0000", 2024-09-20T22:13:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029952499999999998 2024-09-20T22:13:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.871278+0000", 2024-09-20T22:13:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0054385650000000002 2024-09-20T22:13:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.872251+0000", 2024-09-20T22:13:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00097270799999999997 2024-09-20T22:13:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.882180+0000", 2024-09-20T22:13:13.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0099287290000000007 2024-09-20T22:13:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.883112+0000", 2024-09-20T22:13:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00093256000000000003 2024-09-20T22:13:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.883162+0000", 2024-09-20T22:13:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9713999999999999e-05 2024-09-20T22:13:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.883263+0000", 2024-09-20T22:13:13.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010135400000000001 2024-09-20T22:13:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.14445.0:2207 5.3 5:c609908c:::notify.5:head [watch reconnect cookie 94424494156032 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e326)", 2024-09-20T22:13:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:12:55.289536+0000", 2024-09-20T22:13:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 18.330839294, 2024-09-20T22:13:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5980502149999998, 2024-09-20T22:13:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:13:13.772 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.14445", 2024-09-20T22:13:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1594620389", 2024-09-20T22:13:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 2207 2024-09-20T22:13:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:55.289536+0000", 2024-09-20T22:13:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:55.289536+0000", 2024-09-20T22:13:13.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:55.289538+0000", 2024-09-20T22:13:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2400000000000002e-06 2024-09-20T22:13:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:55.289546+0000", 2024-09-20T22:13:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1470000000000001e-06 2024-09-20T22:13:13.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:55.289566+0000", 2024-09-20T22:13:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9882e-05 2024-09-20T22:13:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:55.289577+0000", 2024-09-20T22:13:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0463e-05 2024-09-20T22:13:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:55.289630+0000", 2024-09-20T22:13:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3919999999999999e-05 2024-09-20T22:13:13.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:55.289638+0000", 2024-09-20T22:13:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4050000000000003e-06 2024-09-20T22:13:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.884644+0000", 2024-09-20T22:13:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5950061209999999 2024-09-20T22:13:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:13:13.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.884779+0000", 2024-09-20T22:13:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013511299999999999 2024-09-20T22:13:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.885627+0000", 2024-09-20T22:13:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00084780800000000003 2024-09-20T22:13:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for rw locks", 2024-09-20T22:13:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.885763+0000", 2024-09-20T22:13:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000135721 2024-09-20T22:13:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.886424+0000", 2024-09-20T22:13:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00066162600000000001 2024-09-20T22:13:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.886493+0000", 2024-09-20T22:13:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8477000000000002e-05 2024-09-20T22:13:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.887586+0000", 2024-09-20T22:13:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001093292 2024-09-20T22:13:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.778 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:88458 9.b 9:da866086:::benchmark_data_smithi060_140132_object88457:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e328)", 2024-09-20T22:13:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:12:57.940821+0000", 2024-09-20T22:13:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 15.679554197, 2024-09-20T22:13:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1030801809999999, 2024-09-20T22:13:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:13:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:13:13.779 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 88458 2024-09-20T22:13:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.940821+0000", 2024-09-20T22:13:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.940821+0000", 2024-09-20T22:13:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.940821+0000", 2024-09-20T22:13:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8999999999999997e-07 2024-09-20T22:13:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.940850+0000", 2024-09-20T22:13:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9002000000000001e-05 2024-09-20T22:13:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.940857+0000", 2024-09-20T22:13:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1239999999999999e-06 2024-09-20T22:13:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.940863+0000", 2024-09-20T22:13:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8730000000000004e-06 2024-09-20T22:13:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.943270+0000", 2024-09-20T22:13:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024065530000000001 2024-09-20T22:13:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.943277+0000", 2024-09-20T22:13:13.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1420000000000004e-06 2024-09-20T22:13:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:59.026589+0000", 2024-09-20T22:13:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.083311457 2024-09-20T22:13:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:59.026742+0000", 2024-09-20T22:13:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000153292 2024-09-20T22:13:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 5,6", 2024-09-20T22:13:13.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:59.026881+0000", 2024-09-20T22:13:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013902399999999999 2024-09-20T22:13:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:59.041125+0000", 2024-09-20T22:13:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014243811 2024-09-20T22:13:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:59.043675+0000", 2024-09-20T22:13:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025503710000000001 2024-09-20T22:13:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:59.043760+0000", 2024-09-20T22:13:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5439999999999995e-05 2024-09-20T22:13:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:59.043814+0000", 2024-09-20T22:13:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3199999999999999e-05 2024-09-20T22:13:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:59.043901+0000", 2024-09-20T22:13:13.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7402000000000001e-05 2024-09-20T22:13:13.786 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.786 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.786 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.786 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:13.786 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:13.786 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:13:13.798 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:13.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:13.799 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:13.900 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:13.911 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 86 16 106193 106177 77.1421 72.0625 0.00710949 0.0129582 2024-09-20T22:13:13.931 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:13.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:13.932 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:14.033 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:14.064 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:14.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:14.065 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:14.165 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:14.188 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:14.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:14.189 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:14.289 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:14.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:14.314 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:14.315 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:14.415 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:14.453 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:14.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:14.454 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:14.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:14 smithi067 ceph-mon[33511]: 3.4 scrub starts 2024-09-20T22:13:14.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:14 smithi060 ceph-mon[40290]: 3.4 scrub starts 2024-09-20T22:13:14.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:14 smithi060 ceph-mon[29694]: 3.4 scrub starts 2024-09-20T22:13:14.555 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:14.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:14.591 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:14.591 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:14.692 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:14.728 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:14.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:14.729 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:14.830 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:14.868 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:14.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:14.869 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:14.912 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 87 16 107136 107120 76.9329 58.9375 0.00743485 0.0129936 2024-09-20T22:13:14.970 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:15.003 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:15.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:15.004 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:15.071 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:13:15.105 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:15.148 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:15.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:15.149 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:15.250 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:15.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[29694]: 3.4 scrub ok 2024-09-20T22:13:15.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:13:15.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:13:15.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[29694]: 6.12 scrub starts 2024-09-20T22:13:15.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[29694]: 6.12 scrub ok 2024-09-20T22:13:15.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[29694]: 3.1e scrub starts 2024-09-20T22:13:15.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[29694]: 3.1e scrub ok 2024-09-20T22:13:15.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[29694]: osdmap e332: 8 total, 8 up, 8 in 2024-09-20T22:13:15.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3706857909' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:13:15.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:13:15.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[29694]: pgmap v559: 158 pgs: 158 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-20T22:13:15.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[29694]: 6.1b scrub starts 2024-09-20T22:13:15.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[29694]: 6.1b scrub ok 2024-09-20T22:13:15.287 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:13:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:13:15.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[40290]: 3.4 scrub ok 2024-09-20T22:13:15.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:13:15.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:13:15.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[40290]: 6.12 scrub starts 2024-09-20T22:13:15.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[40290]: 6.12 scrub ok 2024-09-20T22:13:15.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[40290]: 3.1e scrub starts 2024-09-20T22:13:15.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[40290]: 3.1e scrub ok 2024-09-20T22:13:15.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[40290]: osdmap e332: 8 total, 8 up, 8 in 2024-09-20T22:13:15.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3706857909' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:13:15.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:13:15.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[40290]: pgmap v559: 158 pgs: 158 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-20T22:13:15.289 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[40290]: 6.1b scrub starts 2024-09-20T22:13:15.289 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:15 smithi060 ceph-mon[40290]: 6.1b scrub ok 2024-09-20T22:13:15.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:15.290 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:15.290 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:15.391 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:15.432 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:15.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:15.433 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:15.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:15 smithi067 ceph-mon[33511]: 3.4 scrub ok 2024-09-20T22:13:15.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:15 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:13:15.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:15 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:13:15.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:15 smithi067 ceph-mon[33511]: 6.12 scrub starts 2024-09-20T22:13:15.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:15 smithi067 ceph-mon[33511]: 6.12 scrub ok 2024-09-20T22:13:15.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:15 smithi067 ceph-mon[33511]: 3.1e scrub starts 2024-09-20T22:13:15.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:15 smithi067 ceph-mon[33511]: 3.1e scrub ok 2024-09-20T22:13:15.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:15 smithi067 ceph-mon[33511]: osdmap e332: 8 total, 8 up, 8 in 2024-09-20T22:13:15.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:15 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3706857909' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:13:15.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:15 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:13:15.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:15 smithi067 ceph-mon[33511]: pgmap v559: 158 pgs: 158 active+clean; 6.3 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 68 MiB/s wr, 1.09k op/s 2024-09-20T22:13:15.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:15 smithi067 ceph-mon[33511]: 6.1b scrub starts 2024-09-20T22:13:15.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:15 smithi067 ceph-mon[33511]: 6.1b scrub ok 2024-09-20T22:13:15.534 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:15.565 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:15.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:15.566 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:15.667 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:15.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:15.702 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:15.703 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:15.803 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:15.843 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:15.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:15.844 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:15.912 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 88 16 108384 108368 76.9449 78 0.00766445 0.0129912 2024-09-20T22:13:15.943 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:15.945 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:15.978 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:15.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:15.979 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:16.080 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:16.124 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:16.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:16.125 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:16.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:16.265 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:16.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:16.266 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:16.366 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:16.407 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:16.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:16.408 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:16.509 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:16.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:16 smithi067 ceph-mon[33511]: 5.17 scrub starts 2024-09-20T22:13:16.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:16 smithi067 ceph-mon[33511]: 5.17 scrub ok 2024-09-20T22:13:16.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:16 smithi067 ceph-mon[33511]: 5.11 scrub starts 2024-09-20T22:13:16.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:16 smithi067 ceph-mon[33511]: 5.11 scrub ok 2024-09-20T22:13:16.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:16 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:13:16.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:16 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:13:16.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:16 smithi067 ceph-mon[33511]: osdmap e333: 8 total, 8 up, 8 in 2024-09-20T22:13:16.519 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:16 smithi060 ceph-mon[29694]: 5.17 scrub starts 2024-09-20T22:13:16.519 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:16 smithi060 ceph-mon[29694]: 5.17 scrub ok 2024-09-20T22:13:16.519 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:16 smithi060 ceph-mon[29694]: 5.11 scrub starts 2024-09-20T22:13:16.519 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:16 smithi060 ceph-mon[29694]: 5.11 scrub ok 2024-09-20T22:13:16.519 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:16 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:13:16.519 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:16 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:13:16.519 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:16 smithi060 ceph-mon[29694]: osdmap e333: 8 total, 8 up, 8 in 2024-09-20T22:13:16.519 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:16 smithi060 ceph-mon[40290]: 5.17 scrub starts 2024-09-20T22:13:16.519 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:16 smithi060 ceph-mon[40290]: 5.17 scrub ok 2024-09-20T22:13:16.519 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:16 smithi060 ceph-mon[40290]: 5.11 scrub starts 2024-09-20T22:13:16.520 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:16 smithi060 ceph-mon[40290]: 5.11 scrub ok 2024-09-20T22:13:16.520 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:16 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:13:16.520 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:16 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:13:16.520 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:16 smithi060 ceph-mon[40290]: osdmap e333: 8 total, 8 up, 8 in 2024-09-20T22:13:16.567 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:16.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:16.568 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:16.669 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:16.703 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:16.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:16.704 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:16.804 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:16.848 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:16.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:16.849 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:16.912 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 89 16 109653 109637 76.9714 79.3125 0.0102623 0.0129869 2024-09-20T22:13:16.950 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:16.990 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:16.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:16.992 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:17.092 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:17.126 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:17.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:17.127 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:17.228 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:17.261 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:17.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:17.262 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:17.362 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:17.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:17 smithi060 ceph-mon[29694]: pgmap v561: 158 pgs: 158 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 65 MiB/s wr, 1.05k op/s 2024-09-20T22:13:17.368 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:17 smithi060 ceph-mon[40290]: pgmap v561: 158 pgs: 158 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 65 MiB/s wr, 1.05k op/s 2024-09-20T22:13:17.406 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:17.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:17.407 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:17.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T22:13:17.508 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:17.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:17 smithi067 ceph-mon[33511]: pgmap v561: 158 pgs: 158 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 65 MiB/s wr, 1.05k op/s 2024-09-20T22:13:17.544 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:17.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:17.545 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:17.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:13:17.646 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:17.738 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:17.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:17.739 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:17.840 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:17.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:17.878 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:17.878 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:17.912 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Total time run: 90.0089 2024-09-20T22:13:17.912 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Total writes made: 110805 2024-09-20T22:13:17.912 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Write size: 65536 2024-09-20T22:13:17.912 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Object size: 65536 2024-09-20T22:13:17.912 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Bandwidth (MB/sec): 76.9403 2024-09-20T22:13:17.912 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Stddev Bandwidth: 47.1391 2024-09-20T22:13:17.912 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Max bandwidth (MB/sec): 198.812 2024-09-20T22:13:17.912 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Min bandwidth (MB/sec): 0 2024-09-20T22:13:17.913 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Average IOPS: 1231 2024-09-20T22:13:17.913 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Stddev IOPS: 754.246 2024-09-20T22:13:17.913 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Max IOPS: 3181 2024-09-20T22:13:17.913 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Min IOPS: 0 2024-09-20T22:13:17.913 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Average Latency(s): 0.0129929 2024-09-20T22:13:17.913 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Stddev Latency(s): 0.161526 2024-09-20T22:13:17.913 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Max latency(s): 15.3765 2024-09-20T22:13:17.913 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Min latency(s): 0.0012684 2024-09-20T22:13:17.918 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Cleaning up (deleting benchmark objects) 2024-09-20T22:13:17.979 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:18.010 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:18.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:18.011 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:18.112 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:18.147 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:18.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:18.148 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:18.249 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:18.282 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:18.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:18.283 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:18.384 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:18.418 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:18.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:18.419 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:18.520 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:18.554 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:18.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:18.556 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:18.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:13:18.657 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.3 reset_purged_snaps_last 2024-09-20T22:13:18.657 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:18.702 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:18.722 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:18.724 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:18.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:18.725 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:18.825 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:18.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:18.873 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:18.874 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:18.975 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:19.005 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:19.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:19.006 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:19.107 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:19.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:19.140 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:19.141 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:19.242 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:19.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:19.275 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:19.275 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:19.376 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:19.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:19.411 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:19.411 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:19.512 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:19.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:19 smithi067 ceph-mon[33511]: pgmap v562: 158 pgs: 158 active+clean; 6.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-20T22:13:19.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:19 smithi060 ceph-mon[29694]: pgmap v562: 158 pgs: 158 active+clean; 6.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-20T22:13:19.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:19 smithi060 ceph-mon[40290]: pgmap v562: 158 pgs: 158 active+clean; 6.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.2 KiB/s rd, 88 MiB/s wr, 1.41k op/s 2024-09-20T22:13:19.550 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:19.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:19.551 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:19.651 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:19.673 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:19.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:19.674 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:19.775 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:19.797 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:19.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:19.798 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:19.899 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:19.943 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:19.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:19.944 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:20.045 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:20.078 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:20.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:20.079 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:20.180 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:20.214 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:20.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:20.215 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:20.316 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:20.353 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:20.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:20.354 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:20.454 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:20.495 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:20.496 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:20.496 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:20.597 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:20.627 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:20.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:20.628 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:20.729 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:20.772 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:20.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:20.773 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:20.874 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:20.912 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:20.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:20.914 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:21.014 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:21.056 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:21.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:21.058 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:21.158 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:21.188 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:21.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:21.189 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:21.290 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:21.330 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:13:21.330 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:13:21.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.330 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.15594.0:114787 9.6 e334/286 9:6ea56b0e:::benchmark_data_smithi060_140132_object3979:head v 334'7299, mlcod=334'7297)", 2024-09-20T22:13:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:21.317927+0000", 2024-09-20T22:13:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0077619919999999997, 2024-09-20T22:13:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010083348000000001, 2024-09-20T22:13:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:21.317927+0000", 2024-09-20T22:13:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:21.317927+0000", 2024-09-20T22:13:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:21.317928+0000", 2024-09-20T22:13:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1620000000000001e-06 2024-09-20T22:13:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:21.317937+0000", 2024-09-20T22:13:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5140000000000001e-06 2024-09-20T22:13:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:21.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:21.317956+0000", 2024-09-20T22:13:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8576999999999998e-05 2024-09-20T22:13:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:21.317964+0000", 2024-09-20T22:13:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6929999999999996e-06 2024-09-20T22:13:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:21.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:21.318184+0000", 2024-09-20T22:13:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000219302 2024-09-20T22:13:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:21.318225+0000", 2024-09-20T22:13:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1853000000000001e-05 2024-09-20T22:13:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:21.327788+0000", 2024-09-20T22:13:21.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0095626540000000003 2024-09-20T22:13:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:21.328011+0000", 2024-09-20T22:13:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022259299999999999 2024-09-20T22:13:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.15594.0:114795 9.6 e334/286 9:62799a8b:::benchmark_data_smithi060_140132_object3987:head v 334'7300, mlcod=334'7297)", 2024-09-20T22:13:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:21.324591+0000", 2024-09-20T22:13:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0010978909999999999, 2024-09-20T22:13:21.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003538762, 2024-09-20T22:13:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:13:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:21.324591+0000", 2024-09-20T22:13:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:21.324591+0000", 2024-09-20T22:13:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:21.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:21.324593+0000", 2024-09-20T22:13:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3319999999999999e-06 2024-09-20T22:13:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:21.324603+0000", 2024-09-20T22:13:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0772e-05 2024-09-20T22:13:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:21.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:21.324625+0000", 2024-09-20T22:13:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1236000000000001e-05 2024-09-20T22:13:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:21.324634+0000", 2024-09-20T22:13:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5540000000000003e-06 2024-09-20T22:13:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:21.324859+0000", 2024-09-20T22:13:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022509299999999999 2024-09-20T22:13:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:21.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:21.324921+0000", 2024-09-20T22:13:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1123000000000004e-05 2024-09-20T22:13:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:13:21.339 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 2 2024-09-20T22:13:21.339 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:13:21.341 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:21.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:21.342 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:21.406 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:21 smithi060 ceph-mon[29694]: pgmap v563: 158 pgs: 158 active+clean; 6.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 54 MiB/s wr, 877 op/s 2024-09-20T22:13:21.406 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:21 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:13:21.406 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:21 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:13:21.406 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:13:21.406 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:13:21.411 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:21 smithi060 ceph-mon[40290]: pgmap v563: 158 pgs: 158 active+clean; 6.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 54 MiB/s wr, 877 op/s 2024-09-20T22:13:21.411 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:21 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:13:21.411 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:21 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:13:21.411 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:13:21.411 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:13:21.444 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:21.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:21.480 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:21.480 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:21.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:21 smithi067 ceph-mon[33511]: pgmap v563: 158 pgs: 158 active+clean; 6.7 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 54 MiB/s wr, 877 op/s 2024-09-20T22:13:21.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:21 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:13:21.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:21 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:13:21.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:13:21.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:13:21.582 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:21.606 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:13:21.607 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:13:21.624 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:21.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:21.626 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:21.649 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:13:21.649 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:13:21.685 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:13:21.686 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:13:21.717 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:13:21.718 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:13:21.726 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:21.749 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:13:21.749 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:13:21.760 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:21.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:21.761 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:21.782 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:13:21.783 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:13:21.817 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:13:21.818 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:13:21.852 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:13:21.852 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:13:21.862 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:21.894 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:21.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:21.895 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:21.996 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:22.027 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:22.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:22.028 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:22.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:22.169 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:22.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:22.170 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:22.271 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:22.307 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:22.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:22.308 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:22.386 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T22:13:22.408 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:22.445 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:22.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:22.446 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:22.513 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:13:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:13:22.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:22 smithi067 ceph-mon[33511]: osdmap e334: 8 total, 8 up, 8 in 2024-09-20T22:13:22.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:22 smithi067 ceph-mon[33511]: osdmap e335: 8 total, 8 up, 8 in 2024-09-20T22:13:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:22 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2047082009' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:22 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:13:22.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:22 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:13:22.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:22 smithi060 ceph-mon[29694]: osdmap e334: 8 total, 8 up, 8 in 2024-09-20T22:13:22.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:22 smithi060 ceph-mon[29694]: osdmap e335: 8 total, 8 up, 8 in 2024-09-20T22:13:22.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:22 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2047082009' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:22.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:22 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:13:22.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:22 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:13:22.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:22 smithi060 ceph-mon[40290]: osdmap e334: 8 total, 8 up, 8 in 2024-09-20T22:13:22.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:22 smithi060 ceph-mon[40290]: osdmap e335: 8 total, 8 up, 8 in 2024-09-20T22:13:22.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:22 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2047082009' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:22.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:22 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:13:22.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:22 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:13:22.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:22.588 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:22.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:22.589 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:22.634 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:22.690 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:22.749 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:22.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:22.750 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:22.851 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:22.883 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:22.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:22.884 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:22.985 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:23.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:23.007 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:23.008 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:23.108 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:23.142 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:23.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:23.143 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:23.174 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:23.246 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:23.293 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:23.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:23.295 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:23.396 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:23.433 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:23.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:23.434 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:23.478 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:23 smithi060 ceph-mon[29694]: pgmap v566: 158 pgs: 158 active+clean; 6.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.5 KiB/s rd, 63 MiB/s wr, 1.42k op/s 2024-09-20T22:13:23.478 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:23 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:13:23.478 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:23 smithi060 ceph-mon[29694]: osdmap e336: 8 total, 8 up, 8 in 2024-09-20T22:13:23.478 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:23 smithi060 ceph-mon[40290]: pgmap v566: 158 pgs: 158 active+clean; 6.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.5 KiB/s rd, 63 MiB/s wr, 1.42k op/s 2024-09-20T22:13:23.478 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:23 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:13:23.478 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:23 smithi060 ceph-mon[40290]: osdmap e336: 8 total, 8 up, 8 in 2024-09-20T22:13:23.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:23 smithi067 ceph-mon[33511]: pgmap v566: 158 pgs: 158 active+clean; 6.6 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.5 KiB/s rd, 63 MiB/s wr, 1.42k op/s 2024-09-20T22:13:23.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:23 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:13:23.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:23 smithi067 ceph-mon[33511]: osdmap e336: 8 total, 8 up, 8 in 2024-09-20T22:13:23.535 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:23.568 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:23.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:23.569 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:23.670 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:23.707 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:23.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:23.707 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:23.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:23.847 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:23.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:23.848 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:23.949 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:23.986 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:23.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:23.987 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:24.088 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:24.126 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:24.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:24.127 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:24.228 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:24.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:24.260 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:24.347 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:24.361 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:24.412 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:24.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:24.413 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:24.513 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:24.548 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:24.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:24.549 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:24.649 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:24.686 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:24.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:24.687 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:24.788 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:24.829 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:24.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:24.830 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:24.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:24.968 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:24.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:24.970 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:25.071 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:25.104 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:25.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:25.105 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:25.189 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:13:25.205 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:25.263 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:13:25.268 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:13:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:13:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:13:25.292 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:25.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:25.293 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:25.394 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:25.435 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:25.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:25.436 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:25.537 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:25.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:25 smithi060 ceph-mon[29694]: pgmap v568: 158 pgs: 158 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 17 MiB/s wr, 984 op/s 2024-09-20T22:13:25.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:25 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1529140115' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:13:25.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:25 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:13:25.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:25 smithi060 ceph-mon[40290]: pgmap v568: 158 pgs: 158 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 17 MiB/s wr, 984 op/s 2024-09-20T22:13:25.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:25 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1529140115' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:13:25.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:25 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:13:25.571 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:25.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:25.572 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:25.673 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:25.704 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:25.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:25.705 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:25 smithi067 ceph-mon[33511]: pgmap v568: 158 pgs: 158 active+clean; 6.5 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1023 B/s rd, 17 MiB/s wr, 984 op/s 2024-09-20T22:13:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:25 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1529140115' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:13:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:25 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:13:25.806 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:25.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:25.846 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:25.847 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:25.947 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:25.969 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:25.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:25.970 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:26.071 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:26.103 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:26.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:26.104 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:26.205 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:26.240 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:26.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:26.241 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:26.342 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:26.366 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:26.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:26.367 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:26.467 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:26.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:26.508 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:26.508 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:26 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:13:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:26 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:13:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:26 smithi060 ceph-mon[29694]: osdmap e337: 8 total, 8 up, 8 in 2024-09-20T22:13:26.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:26 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:13:26.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:26 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:13:26.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:26 smithi060 ceph-mon[40290]: osdmap e337: 8 total, 8 up, 8 in 2024-09-20T22:13:26.609 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:26.639 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:26.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:26.640 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:26.741 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:26 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:13:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:26 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:13:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:26 smithi067 ceph-mon[33511]: osdmap e337: 8 total, 8 up, 8 in 2024-09-20T22:13:26.778 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:26.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:26.779 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:26.879 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:26.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:26.914 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:26.914 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:27.015 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:27.055 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:27.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:27.056 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:27.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:27.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:27.193 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:27.194 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:27.269 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.4 reset_purged_snaps_last 2024-09-20T22:13:27.295 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:27.330 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:27.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:27.331 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:27.431 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:27.470 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:27.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:27.472 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:27.508 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:27 smithi060 ceph-mon[29694]: pgmap v570: 158 pgs: 158 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.0 KiB/s rd, 4.4 MiB/s wr, 1.23k op/s 2024-09-20T22:13:27.508 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:27 smithi060 ceph-mon[40290]: pgmap v570: 158 pgs: 158 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.0 KiB/s rd, 4.4 MiB/s wr, 1.23k op/s 2024-09-20T22:13:27.572 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:27.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:27.618 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:27.618 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:27.719 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:27.754 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:27.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:27.755 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:27.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:27 smithi067 ceph-mon[33511]: pgmap v570: 158 pgs: 158 active+clean; 6.3 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 1.0 KiB/s rd, 4.4 MiB/s wr, 1.23k op/s 2024-09-20T22:13:27.856 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:27.905 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:27.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:27.906 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:28.007 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:28.015 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:13:28.016 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:13:28.043 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:28.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:28.044 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:28.145 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:28.181 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:28.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:28.182 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:28.283 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:28.313 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:28.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:28.314 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:28.414 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:28.448 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:28.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:28.449 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:28.550 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:28.587 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:28.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:28.588 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:28.689 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:28.727 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:28.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:28.729 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:28.767 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:28.785 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T22:13:28.830 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:28.843 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:28.892 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:28.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:28.893 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:28.993 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:29.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:29.025 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:29.025 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:29.126 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:29.158 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:29.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:29.159 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:29.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:29.310 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:29.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:29.311 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:29.331 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:29 smithi060 ceph-mon[29694]: pgmap v571: 158 pgs: 158 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 768 B/s rd, 3.2 MiB/s wr, 1.20k op/s 2024-09-20T22:13:29.331 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:29 smithi060 ceph-mon[40290]: pgmap v571: 158 pgs: 158 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 768 B/s rd, 3.2 MiB/s wr, 1.20k op/s 2024-09-20T22:13:29.413 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:29.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:29.447 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:29.448 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:29.518 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:13:29.519 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:13:29.549 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:29.557 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:13:29.557 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:13:29.587 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:29.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:29.590 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:29.596 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:13:29.596 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:13:29.643 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:13:29.643 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:13:29.678 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:13:29.678 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:13:29.691 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:29.711 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:13:29.712 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:13:29.727 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:29.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:29.728 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:29.744 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:13:29.745 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:13:29.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:29 smithi067 ceph-mon[33511]: pgmap v571: 158 pgs: 158 active+clean; 6.2 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 768 B/s rd, 3.2 MiB/s wr, 1.20k op/s 2024-09-20T22:13:29.785 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:13:29.786 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:13:29.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:29.871 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:13:29.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:29.872 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:29.872 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:29.973 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:30.001 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:30.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:30.003 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:30.103 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:30.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:30.148 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:30.148 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:30.249 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:30.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:30.276 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:30.276 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:30.377 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:30.401 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:30.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:30.401 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:30.502 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:30.524 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:30.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:30.525 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:30.626 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:30.647 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:30.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:30.648 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:30.749 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:30 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3172042840' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:30.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:30 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3172042840' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:30.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:30 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3172042840' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:30.791 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:30.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:30.792 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:30.892 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:30.933 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:30.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:30.934 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:31.034 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:31.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:31.075 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:31.075 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:31.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:31.213 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:31.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:31.214 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:31.315 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:31.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:31.352 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:31.353 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:31.453 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:31.495 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:31.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:31.496 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:31.566 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:31 smithi060 ceph-mon[29694]: pgmap v572: 158 pgs: 158 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 24 KiB/s wr, 954 op/s 2024-09-20T22:13:31.567 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:31 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:13:31.567 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:31 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:13:31.567 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:31 smithi060 ceph-mon[29694]: osdmap e338: 8 total, 8 up, 8 in 2024-09-20T22:13:31.567 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:31 smithi060 ceph-mon[40290]: pgmap v572: 158 pgs: 158 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 24 KiB/s wr, 954 op/s 2024-09-20T22:13:31.567 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:31 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:13:31.567 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:31 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:13:31.567 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:31 smithi060 ceph-mon[40290]: osdmap e338: 8 total, 8 up, 8 in 2024-09-20T22:13:31.598 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:31.640 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:31.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:31.641 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:31.742 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:31.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:31 smithi067 ceph-mon[33511]: pgmap v572: 158 pgs: 158 active+clean; 6.1 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 639 B/s rd, 24 KiB/s wr, 954 op/s 2024-09-20T22:13:31.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:31 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:13:31.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:31 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:13:31.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:31 smithi067 ceph-mon[33511]: osdmap e338: 8 total, 8 up, 8 in 2024-09-20T22:13:31.777 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:31.777 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:31.778 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:31.879 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:31.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:31.917 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:31.917 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:32.018 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:32.108 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:32.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:32.109 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:32.210 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:32.247 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:32.247 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:32.248 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:32.349 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:32.383 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:32.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:32.384 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:32.485 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:32.525 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:32.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:32.526 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:32.573 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.5 reset_purged_snaps_last 2024-09-20T22:13:32.615 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:32.627 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:32.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:32.705 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:32.706 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:32.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:32 smithi067 ceph-mon[33511]: osdmap e339: 8 total, 8 up, 8 in 2024-09-20T22:13:32.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:32 smithi067 ceph-mon[33511]: pgmap v575: 158 pgs: 158 active+clean; 5.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.51k op/s 2024-09-20T22:13:32.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:32 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:13:32.764 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:13:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:13:32.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:32 smithi060 ceph-mon[29694]: osdmap e339: 8 total, 8 up, 8 in 2024-09-20T22:13:32.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:32 smithi060 ceph-mon[29694]: pgmap v575: 158 pgs: 158 active+clean; 5.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.51k op/s 2024-09-20T22:13:32.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:32 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:13:32.765 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:32 smithi060 ceph-mon[40290]: osdmap e339: 8 total, 8 up, 8 in 2024-09-20T22:13:32.765 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:32 smithi060 ceph-mon[40290]: pgmap v575: 158 pgs: 158 active+clean; 5.9 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1.51k op/s 2024-09-20T22:13:32.765 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:32 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:13:32.807 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:32.837 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:32.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:32.838 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:32.939 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:32.955 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:32.977 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:32.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:32.978 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:33.079 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:33.117 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:33.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:33.118 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:33.218 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:33.265 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:33.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:33.266 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:33.367 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:33.383 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:13:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:13:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:13:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:13:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47598 7.3 7:c6c29109:::benchmark_data_smithi060_78800_object47597:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:13:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.238604+0000", 2024-09-20T22:13:33.383 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 553.06083068600003, 2024-09-20T22:13:33.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0797809089999999, 2024-09-20T22:13:33.384 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.384 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.384 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:33.384 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.384 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.384 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.384 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47598 2024-09-20T22:13:33.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.384 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238604+0000", 2024-09-20T22:13:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238604+0000", 2024-09-20T22:13:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238605+0000", 2024-09-20T22:13:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3099999999999996e-07 2024-09-20T22:13:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238626+0000", 2024-09-20T22:13:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1192999999999999e-05 2024-09-20T22:13:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238637+0000", 2024-09-20T22:13:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0516000000000001e-05 2024-09-20T22:13:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.238644+0000", 2024-09-20T22:13:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3969999999999997e-06 2024-09-20T22:13:33.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.240757+0000", 2024-09-20T22:13:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021131100000000001 2024-09-20T22:13:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op must wait for map", 2024-09-20T22:13:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.240764+0000", 2024-09-20T22:13:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9550000000000003e-06 2024-09-20T22:13:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.246730+0000", 2024-09-20T22:13:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0059659020000000004 2024-09-20T22:13:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.246741+0000", 2024-09-20T22:13:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0855e-05 2024-09-20T22:13:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.235644+0000", 2024-09-20T22:13:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98890363400000003 2024-09-20T22:13:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.235657+0000", 2024-09-20T22:13:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3040000000000001e-05 2024-09-20T22:13:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.313762+0000", 2024-09-20T22:13:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.078104783999999997 2024-09-20T22:13:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.314529+0000", 2024-09-20T22:13:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00076653400000000003 2024-09-20T22:13:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:13:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.314632+0000", 2024-09-20T22:13:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103224 2024-09-20T22:13:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318215+0000", 2024-09-20T22:13:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0035831439999999999 2024-09-20T22:13:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318247+0000", 2024-09-20T22:13:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.1804999999999998e-05 2024-09-20T22:13:33.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318292+0000", 2024-09-20T22:13:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4605999999999999e-05 2024-09-20T22:13:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318329+0000", 2024-09-20T22:13:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7175999999999998e-05 2024-09-20T22:13:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.391 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318385+0000", 2024-09-20T22:13:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6202999999999997e-05 2024-09-20T22:13:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47618 7.3 7:c8aeb189:::benchmark_data_smithi060_78800_object47617:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:13:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.257444+0000", 2024-09-20T22:13:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 553.04199070300001, 2024-09-20T22:13:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0612655090000001, 2024-09-20T22:13:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.392 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47618 2024-09-20T22:13:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257444+0000", 2024-09-20T22:13:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257444+0000", 2024-09-20T22:13:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257445+0000", 2024-09-20T22:13:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3790000000000001e-06 2024-09-20T22:13:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257468+0000", 2024-09-20T22:13:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2078000000000001e-05 2024-09-20T22:13:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257480+0000", 2024-09-20T22:13:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.253e-05 2024-09-20T22:13:33.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257488+0000", 2024-09-20T22:13:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4900000000000003e-06 2024-09-20T22:13:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257693+0000", 2024-09-20T22:13:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000205071 2024-09-20T22:13:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.257702+0000", 2024-09-20T22:13:33.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6099999999999995e-06 2024-09-20T22:13:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236024+0000", 2024-09-20T22:13:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97832186300000001 2024-09-20T22:13:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236034+0000", 2024-09-20T22:13:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9189999999999994e-06 2024-09-20T22:13:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.396 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.316117+0000", 2024-09-20T22:13:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.080083484999999996 2024-09-20T22:13:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.316218+0000", 2024-09-20T22:13:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100846 2024-09-20T22:13:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:13:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.316363+0000", 2024-09-20T22:13:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014473099999999999 2024-09-20T22:13:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318472+0000", 2024-09-20T22:13:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021087039999999999 2024-09-20T22:13:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318532+0000", 2024-09-20T22:13:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0418999999999999e-05 2024-09-20T22:13:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318622+0000", 2024-09-20T22:13:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0074000000000005e-05 2024-09-20T22:13:33.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318659+0000", 2024-09-20T22:13:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6260999999999997e-05 2024-09-20T22:13:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318710+0000", 2024-09-20T22:13:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1048999999999999e-05 2024-09-20T22:13:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47683 7.3 7:cf1d2c6c:::benchmark_data_smithi060_78800_object47682:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:13:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.290834+0000", 2024-09-20T22:13:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 553.00860064999995, 2024-09-20T22:13:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.029069585, 2024-09-20T22:13:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47683 2024-09-20T22:13:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290834+0000", 2024-09-20T22:13:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290834+0000", 2024-09-20T22:13:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290835+0000", 2024-09-20T22:13:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2020000000000001e-06 2024-09-20T22:13:33.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290856+0000", 2024-09-20T22:13:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0585999999999999e-05 2024-09-20T22:13:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290868+0000", 2024-09-20T22:13:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2160999999999999e-05 2024-09-20T22:13:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290875+0000", 2024-09-20T22:13:33.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3200000000000002e-06 2024-09-20T22:13:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290955+0000", 2024-09-20T22:13:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9592999999999995e-05 2024-09-20T22:13:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.290966+0000", 2024-09-20T22:13:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.119e-05 2024-09-20T22:13:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236104+0000", 2024-09-20T22:13:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94513756400000004 2024-09-20T22:13:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236112+0000", 2024-09-20T22:13:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2220000000000002e-06 2024-09-20T22:13:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.317361+0000", 2024-09-20T22:13:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.081249110999999999 2024-09-20T22:13:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.317444+0000", 2024-09-20T22:13:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2936999999999996e-05 2024-09-20T22:13:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:13:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.317551+0000", 2024-09-20T22:13:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010741100000000001 2024-09-20T22:13:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318732+0000", 2024-09-20T22:13:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0011802379999999999 2024-09-20T22:13:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319739+0000", 2024-09-20T22:13:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001007457 2024-09-20T22:13:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319786+0000", 2024-09-20T22:13:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6845999999999998e-05 2024-09-20T22:13:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:33.406 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319820+0000", 2024-09-20T22:13:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4103000000000002e-05 2024-09-20T22:13:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319904+0000", 2024-09-20T22:13:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3644000000000001e-05 2024-09-20T22:13:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47686 7.3 7:c4eaa487:::benchmark_data_smithi060_78800_object47685:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:13:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.291328+0000", 2024-09-20T22:13:33.407 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 553.00810707799997, 2024-09-20T22:13:33.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.03577915, 2024-09-20T22:13:33.408 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.408 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.408 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:33.408 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.408 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.408 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.408 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47686 2024-09-20T22:13:33.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.408 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.408 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291328+0000", 2024-09-20T22:13:33.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291328+0000", 2024-09-20T22:13:33.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291328+0000", 2024-09-20T22:13:33.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0699999999999996e-07 2024-09-20T22:13:33.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291353+0000", 2024-09-20T22:13:33.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4134999999999998e-05 2024-09-20T22:13:33.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291364+0000", 2024-09-20T22:13:33.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1501999999999999e-05 2024-09-20T22:13:33.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291371+0000", 2024-09-20T22:13:33.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1210000000000001e-06 2024-09-20T22:13:33.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291460+0000", 2024-09-20T22:13:33.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8869999999999994e-05 2024-09-20T22:13:33.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.291473+0000", 2024-09-20T22:13:33.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2899e-05 2024-09-20T22:13:33.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236140+0000", 2024-09-20T22:13:33.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94466707500000002 2024-09-20T22:13:33.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236149+0000", 2024-09-20T22:13:33.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9509999999999995e-06 2024-09-20T22:13:33.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318750+0000", 2024-09-20T22:13:33.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.082601349000000004 2024-09-20T22:13:33.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:33.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318813+0000", 2024-09-20T22:13:33.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2306999999999998e-05 2024-09-20T22:13:33.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:13:33.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.318890+0000", 2024-09-20T22:13:33.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7565999999999996e-05 2024-09-20T22:13:33.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323067+0000", 2024-09-20T22:13:33.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0041765129999999998 2024-09-20T22:13:33.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323117+0000", 2024-09-20T22:13:33.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0297000000000003e-05 2024-09-20T22:13:33.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:33.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.326959+0000", 2024-09-20T22:13:33.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003841957 2024-09-20T22:13:33.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:33.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327009+0000", 2024-09-20T22:13:33.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0328999999999999e-05 2024-09-20T22:13:33.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327107+0000", 2024-09-20T22:13:33.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7472000000000005e-05 2024-09-20T22:13:33.415 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.415 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.415 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.415 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47726 7.3 7:c33b26c6:::benchmark_data_smithi060_78800_object47725:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:13:33.415 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.311743+0000", 2024-09-20T22:13:33.415 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 552.98769126800005, 2024-09-20T22:13:33.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0155469880000001, 2024-09-20T22:13:33.415 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.415 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.415 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:33.415 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.416 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.416 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.416 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47726 2024-09-20T22:13:33.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.416 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.416 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.416 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311743+0000", 2024-09-20T22:13:33.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.416 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.416 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311743+0000", 2024-09-20T22:13:33.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311745+0000", 2024-09-20T22:13:33.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.333e-06 2024-09-20T22:13:33.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311769+0000", 2024-09-20T22:13:33.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3688000000000001e-05 2024-09-20T22:13:33.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311781+0000", 2024-09-20T22:13:33.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2099999999999999e-05 2024-09-20T22:13:33.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311788+0000", 2024-09-20T22:13:33.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2269999999999996e-06 2024-09-20T22:13:33.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311860+0000", 2024-09-20T22:13:33.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1823000000000007e-05 2024-09-20T22:13:33.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.311870+0000", 2024-09-20T22:13:33.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0079e-05 2024-09-20T22:13:33.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236177+0000", 2024-09-20T22:13:33.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.92430713399999997 2024-09-20T22:13:33.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236185+0000", 2024-09-20T22:13:33.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.106e-06 2024-09-20T22:13:33.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.319949+0000", 2024-09-20T22:13:33.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.083764076000000007 2024-09-20T22:13:33.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:33.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.320088+0000", 2024-09-20T22:13:33.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013926300000000001 2024-09-20T22:13:33.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:13:33.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.320263+0000", 2024-09-20T22:13:33.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000174978 2024-09-20T22:13:33.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.421 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323166+0000", 2024-09-20T22:13:33.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0029027060000000001 2024-09-20T22:13:33.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.421 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323208+0000", 2024-09-20T22:13:33.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2373999999999998e-05 2024-09-20T22:13:33.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:33.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327156+0000", 2024-09-20T22:13:33.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0039471910000000001 2024-09-20T22:13:33.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:33.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327201+0000", 2024-09-20T22:13:33.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5062999999999999e-05 2024-09-20T22:13:33.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.422 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.422 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327290+0000", 2024-09-20T22:13:33.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9846999999999995e-05 2024-09-20T22:13:33.422 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.423 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.423 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.423 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47771 7.3 7:c7433b13:::benchmark_data_smithi060_78800_object47770:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:13:33.423 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.332925+0000", 2024-09-20T22:13:33.423 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 552.96651022200001, 2024-09-20T22:13:33.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99455883199999995, 2024-09-20T22:13:33.423 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.423 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.423 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:33.423 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.423 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.424 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.424 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47771 2024-09-20T22:13:33.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.424 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332925+0000", 2024-09-20T22:13:33.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332925+0000", 2024-09-20T22:13:33.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332926+0000", 2024-09-20T22:13:33.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4959999999999999e-06 2024-09-20T22:13:33.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332954+0000", 2024-09-20T22:13:33.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7495e-05 2024-09-20T22:13:33.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332967+0000", 2024-09-20T22:13:33.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3005e-05 2024-09-20T22:13:33.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.332974+0000", 2024-09-20T22:13:33.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9719999999999992e-06 2024-09-20T22:13:33.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333056+0000", 2024-09-20T22:13:33.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1390999999999994e-05 2024-09-20T22:13:33.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333067+0000", 2024-09-20T22:13:33.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0896999999999999e-05 2024-09-20T22:13:33.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236223+0000", 2024-09-20T22:13:33.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90315600299999999 2024-09-20T22:13:33.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236243+0000", 2024-09-20T22:13:33.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9928999999999998e-05 2024-09-20T22:13:33.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.321565+0000", 2024-09-20T22:13:33.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.085322377000000005 2024-09-20T22:13:33.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.428 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:33.428 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.321707+0000", 2024-09-20T22:13:33.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014187100000000001 2024-09-20T22:13:33.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:13:33.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.321808+0000", 2024-09-20T22:13:33.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000100536 2024-09-20T22:13:33.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323255+0000", 2024-09-20T22:13:33.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00144754 2024-09-20T22:13:33.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.429 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.429 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.326947+0000", 2024-09-20T22:13:33.429 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0036917360000000001 2024-09-20T22:13:33.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.430 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:33.430 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327341+0000", 2024-09-20T22:13:33.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00039394499999999998 2024-09-20T22:13:33.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.430 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:33.430 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327391+0000", 2024-09-20T22:13:33.430 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9836999999999997e-05 2024-09-20T22:13:33.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.430 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.430 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327483+0000", 2024-09-20T22:13:33.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2801999999999997e-05 2024-09-20T22:13:33.431 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.431 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.431 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.431 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47772 7.3 7:c5f9ddaf:::benchmark_data_smithi060_78800_object47771:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:13:33.431 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.333899+0000", 2024-09-20T22:13:33.431 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 552.96553570499998, 2024-09-20T22:13:33.431 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99376266499999999, 2024-09-20T22:13:33.431 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.431 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.431 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:33.432 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.432 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.432 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.432 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47772 2024-09-20T22:13:33.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.432 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333899+0000", 2024-09-20T22:13:33.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.432 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.432 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333899+0000", 2024-09-20T22:13:33.432 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333900+0000", 2024-09-20T22:13:33.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0049999999999999e-06 2024-09-20T22:13:33.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333927+0000", 2024-09-20T22:13:33.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.675e-05 2024-09-20T22:13:33.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.433 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.433 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.433 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333942+0000", 2024-09-20T22:13:33.433 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5678000000000001e-05 2024-09-20T22:13:33.433 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.333950+0000", 2024-09-20T22:13:33.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7060000000000004e-06 2024-09-20T22:13:33.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.334006+0000", 2024-09-20T22:13:33.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5631000000000003e-05 2024-09-20T22:13:33.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.434 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.434 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.334031+0000", 2024-09-20T22:13:33.434 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5213999999999999e-05 2024-09-20T22:13:33.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.435 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.435 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236276+0000", 2024-09-20T22:13:33.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90224500100000005 2024-09-20T22:13:33.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.435 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.435 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236285+0000", 2024-09-20T22:13:33.435 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7139999999999999e-06 2024-09-20T22:13:33.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.435 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.435 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323289+0000", 2024-09-20T22:13:33.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.087004215999999995 2024-09-20T22:13:33.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:33.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323404+0000", 2024-09-20T22:13:33.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011502000000000001 2024-09-20T22:13:33.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:13:33.436 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.323545+0000", 2024-09-20T22:13:33.436 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014127700000000001 2024-09-20T22:13:33.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.436 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327330+0000", 2024-09-20T22:13:33.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037843400000000002 2024-09-20T22:13:33.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:33.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327491+0000", 2024-09-20T22:13:33.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016180099999999999 2024-09-20T22:13:33.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.437 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.437 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327529+0000", 2024-09-20T22:13:33.437 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8081999999999999e-05 2024-09-20T22:13:33.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:33.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327576+0000", 2024-09-20T22:13:33.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6881000000000001e-05 2024-09-20T22:13:33.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.438 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.438 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327662+0000", 2024-09-20T22:13:33.438 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5348999999999999e-05 2024-09-20T22:13:33.438 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.438 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.438 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.439 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47798 7.3 7:cc6426c7:::benchmark_data_smithi060_78800_object47797:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:13:33.439 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.346085+0000", 2024-09-20T22:13:33.439 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 552.95334931100001, 2024-09-20T22:13:33.439 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98393741899999998, 2024-09-20T22:13:33.439 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.439 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.439 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:33.439 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.439 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.439 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.439 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47798 2024-09-20T22:13:33.439 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.439 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.439 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.440 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.440 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346085+0000", 2024-09-20T22:13:33.440 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.440 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.440 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346085+0000", 2024-09-20T22:13:33.440 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.440 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.440 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346087+0000", 2024-09-20T22:13:33.440 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3289999999999999e-06 2024-09-20T22:13:33.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346109+0000", 2024-09-20T22:13:33.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2359999999999999e-05 2024-09-20T22:13:33.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346123+0000", 2024-09-20T22:13:33.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.367e-05 2024-09-20T22:13:33.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.441 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.441 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346133+0000", 2024-09-20T22:13:33.441 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0519999999999999e-05 2024-09-20T22:13:33.441 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346206+0000", 2024-09-20T22:13:33.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3011999999999998e-05 2024-09-20T22:13:33.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.346218+0000", 2024-09-20T22:13:33.442 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2089000000000001e-05 2024-09-20T22:13:33.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.442 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.442 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236825+0000", 2024-09-20T22:13:33.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.89060705600000001 2024-09-20T22:13:33.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.443 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236839+0000", 2024-09-20T22:13:33.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3685e-05 2024-09-20T22:13:33.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.443 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.324961+0000", 2024-09-20T22:13:33.443 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.088121685000000005 2024-09-20T22:13:33.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.443 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:33.444 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.325083+0000", 2024-09-20T22:13:33.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012199199999999999 2024-09-20T22:13:33.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.444 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:13:33.444 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.325216+0000", 2024-09-20T22:13:33.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000132862 2024-09-20T22:13:33.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.444 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:33.444 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327496+0000", 2024-09-20T22:13:33.444 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022805669999999998 2024-09-20T22:13:33.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327698+0000", 2024-09-20T22:13:33.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020205099999999999 2024-09-20T22:13:33.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.329834+0000", 2024-09-20T22:13:33.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021359249999999999 2024-09-20T22:13:33.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.445 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:33.445 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.329894+0000", 2024-09-20T22:13:33.445 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9929000000000002e-05 2024-09-20T22:13:33.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.446 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.446 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.330023+0000", 2024-09-20T22:13:33.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000128687 2024-09-20T22:13:33.446 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.446 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.446 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.446 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47809 7.3 7:caa74520:::benchmark_data_smithi060_78800_object47808:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:13:33.446 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.351924+0000", 2024-09-20T22:13:33.446 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 552.94751117299995, 2024-09-20T22:13:33.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.982167177, 2024-09-20T22:13:33.446 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.447 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.447 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:33.447 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.447 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.447 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.447 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47809 2024-09-20T22:13:33.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.447 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.351924+0000", 2024-09-20T22:13:33.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.351924+0000", 2024-09-20T22:13:33.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.351925+0000", 2024-09-20T22:13:33.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1680000000000001e-06 2024-09-20T22:13:33.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352100+0000", 2024-09-20T22:13:33.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017479600000000001 2024-09-20T22:13:33.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352112+0000", 2024-09-20T22:13:33.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.204e-05 2024-09-20T22:13:33.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352119+0000", 2024-09-20T22:13:33.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8960000000000003e-06 2024-09-20T22:13:33.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352199+0000", 2024-09-20T22:13:33.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9747999999999996e-05 2024-09-20T22:13:33.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352208+0000", 2024-09-20T22:13:33.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2709999999999998e-06 2024-09-20T22:13:33.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236880+0000", 2024-09-20T22:13:33.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.88467109200000005 2024-09-20T22:13:33.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236889+0000", 2024-09-20T22:13:33.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0580000000000004e-06 2024-09-20T22:13:33.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327738+0000", 2024-09-20T22:13:33.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.090849691999999996 2024-09-20T22:13:33.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:33.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.327931+0000", 2024-09-20T22:13:33.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019231900000000001 2024-09-20T22:13:33.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:13:33.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.328060+0000", 2024-09-20T22:13:33.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012906699999999999 2024-09-20T22:13:33.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:33.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.331899+0000", 2024-09-20T22:13:33.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038394620000000001 2024-09-20T22:13:33.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.333872+0000", 2024-09-20T22:13:33.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019725540000000001 2024-09-20T22:13:33.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.333936+0000", 2024-09-20T22:13:33.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3849000000000005e-05 2024-09-20T22:13:33.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:33.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.333992+0000", 2024-09-20T22:13:33.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6730000000000001e-05 2024-09-20T22:13:33.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.454 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334091+0000", 2024-09-20T22:13:33.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8435000000000002e-05 2024-09-20T22:13:33.454 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.454 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.454 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.454 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47810 7.3 7:c6434cb9:::benchmark_data_smithi060_78800_object47809:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:13:33.454 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.352140+0000", 2024-09-20T22:13:33.454 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 552.94729517799999, 2024-09-20T22:13:33.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.985672357, 2024-09-20T22:13:33.454 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.454 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.455 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:33.455 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.455 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.455 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.455 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47810 2024-09-20T22:13:33.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.455 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352140+0000", 2024-09-20T22:13:33.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352140+0000", 2024-09-20T22:13:33.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352140+0000", 2024-09-20T22:13:33.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9299999999999997e-07 2024-09-20T22:13:33.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352187+0000", 2024-09-20T22:13:33.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6468000000000003e-05 2024-09-20T22:13:33.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352196+0000", 2024-09-20T22:13:33.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9670000000000007e-06 2024-09-20T22:13:33.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352202+0000", 2024-09-20T22:13:33.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0120000000000002e-06 2024-09-20T22:13:33.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352230+0000", 2024-09-20T22:13:33.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8289e-05 2024-09-20T22:13:33.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.352236+0000", 2024-09-20T22:13:33.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.455e-06 2024-09-20T22:13:33.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236921+0000", 2024-09-20T22:13:33.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.88468433300000004 2024-09-20T22:13:33.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236931+0000", 2024-09-20T22:13:33.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0322e-05 2024-09-20T22:13:33.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.330063+0000", 2024-09-20T22:13:33.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.093132375000000003 2024-09-20T22:13:33.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:33.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.330188+0000", 2024-09-20T22:13:33.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000124664 2024-09-20T22:13:33.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:13:33.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.330329+0000", 2024-09-20T22:13:33.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014119299999999999 2024-09-20T22:13:33.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:33.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334105+0000", 2024-09-20T22:13:33.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037757799999999998 2024-09-20T22:13:33.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334140+0000", 2024-09-20T22:13:33.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5015000000000003e-05 2024-09-20T22:13:33.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337652+0000", 2024-09-20T22:13:33.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0035119819999999999 2024-09-20T22:13:33.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:33.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337709+0000", 2024-09-20T22:13:33.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6446000000000001e-05 2024-09-20T22:13:33.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337812+0000", 2024-09-20T22:13:33.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010336300000000001 2024-09-20T22:13:33.462 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.462 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.462 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.462 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47831 7.3 7:cf7c6d80:::benchmark_data_smithi060_78800_object47830:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:13:33.462 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.435232+0000", 2024-09-20T22:13:33.462 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 552.86420258600003, 2024-09-20T22:13:33.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90280988699999998, 2024-09-20T22:13:33.462 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.463 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.463 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:33.463 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.463 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.463 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.463 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47831 2024-09-20T22:13:33.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.463 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.463 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.463 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435232+0000", 2024-09-20T22:13:33.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.464 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435232+0000", 2024-09-20T22:13:33.464 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.464 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435233+0000", 2024-09-20T22:13:33.464 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0920000000000001e-06 2024-09-20T22:13:33.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.464 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435256+0000", 2024-09-20T22:13:33.464 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2840000000000002e-05 2024-09-20T22:13:33.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435270+0000", 2024-09-20T22:13:33.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3944000000000001e-05 2024-09-20T22:13:33.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435277+0000", 2024-09-20T22:13:33.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1420000000000004e-06 2024-09-20T22:13:33.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435341+0000", 2024-09-20T22:13:33.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3431999999999998e-05 2024-09-20T22:13:33.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.466 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.435350+0000", 2024-09-20T22:13:33.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2920000000000001e-06 2024-09-20T22:13:33.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.466 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236961+0000", 2024-09-20T22:13:33.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.80161062500000002 2024-09-20T22:13:33.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.236968+0000", 2024-09-20T22:13:33.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8529999999999992e-06 2024-09-20T22:13:33.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.331930+0000", 2024-09-20T22:13:33.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.094961952000000002 2024-09-20T22:13:33.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:33.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.332051+0000", 2024-09-20T22:13:33.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000120683 2024-09-20T22:13:33.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:13:33.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.332208+0000", 2024-09-20T22:13:33.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015661399999999999 2024-09-20T22:13:33.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334177+0000", 2024-09-20T22:13:33.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019690350000000001 2024-09-20T22:13:33.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:33.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334187+0000", 2024-09-20T22:13:33.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8870000000000004e-06 2024-09-20T22:13:33.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337890+0000", 2024-09-20T22:13:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037037379999999998 2024-09-20T22:13:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337940+0000", 2024-09-20T22:13:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0144e-05 2024-09-20T22:13:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.338042+0000", 2024-09-20T22:13:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010161400000000001 2024-09-20T22:13:33.469 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47833 7.3 7:c90762ed:::benchmark_data_smithi060_78800_object47832:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:13:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.437390+0000", 2024-09-20T22:13:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 552.86204462600006, 2024-09-20T22:13:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90566071400000003, 2024-09-20T22:13:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.470 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47833 2024-09-20T22:13:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437390+0000", 2024-09-20T22:13:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437390+0000", 2024-09-20T22:13:33.471 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437391+0000", 2024-09-20T22:13:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.085e-06 2024-09-20T22:13:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437415+0000", 2024-09-20T22:13:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3479e-05 2024-09-20T22:13:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437430+0000", 2024-09-20T22:13:33.472 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5485000000000002e-05 2024-09-20T22:13:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437439+0000", 2024-09-20T22:13:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3520000000000007e-06 2024-09-20T22:13:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437507+0000", 2024-09-20T22:13:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8011999999999999e-05 2024-09-20T22:13:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.473 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.437520+0000", 2024-09-20T22:13:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3022e-05 2024-09-20T22:13:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237002+0000", 2024-09-20T22:13:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.79948219600000003 2024-09-20T22:13:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237011+0000", 2024-09-20T22:13:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8049999999999996e-06 2024-09-20T22:13:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.474 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334208+0000", 2024-09-20T22:13:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.097197622999999997 2024-09-20T22:13:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334306+0000", 2024-09-20T22:13:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8068999999999999e-05 2024-09-20T22:13:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:13:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.334475+0000", 2024-09-20T22:13:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016830799999999999 2024-09-20T22:13:33.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.337852+0000", 2024-09-20T22:13:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00337747 2024-09-20T22:13:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.340086+0000", 2024-09-20T22:13:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002234036 2024-09-20T22:13:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.476 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.342602+0000", 2024-09-20T22:13:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025159929999999998 2024-09-20T22:13:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.342808+0000", 2024-09-20T22:13:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020551099999999999 2024-09-20T22:13:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343051+0000", 2024-09-20T22:13:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00024326799999999999 2024-09-20T22:13:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.477 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47842 7.3 7:c97c3aed:::benchmark_data_smithi060_78800_object47841:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:13:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.442613+0000", 2024-09-20T22:13:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 552.85682201899999, 2024-09-20T22:13:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90079891400000001, 2024-09-20T22:13:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47842 2024-09-20T22:13:33.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442613+0000", 2024-09-20T22:13:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442613+0000", 2024-09-20T22:13:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.479 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442614+0000", 2024-09-20T22:13:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3e-06 2024-09-20T22:13:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442640+0000", 2024-09-20T22:13:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5544000000000001e-05 2024-09-20T22:13:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442651+0000", 2024-09-20T22:13:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1218e-05 2024-09-20T22:13:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442658+0000", 2024-09-20T22:13:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1980000000000004e-06 2024-09-20T22:13:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442734+0000", 2024-09-20T22:13:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6279999999999995e-05 2024-09-20T22:13:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.442747+0000", 2024-09-20T22:13:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2393e-05 2024-09-20T22:13:33.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237041+0000", 2024-09-20T22:13:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.79429414499999995 2024-09-20T22:13:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237049+0000", 2024-09-20T22:13:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8150000000000006e-06 2024-09-20T22:13:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.335958+0000", 2024-09-20T22:13:33.482 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.098909056999999995 2024-09-20T22:13:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.336074+0000", 2024-09-20T22:13:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011582900000000001 2024-09-20T22:13:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:13:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.336216+0000", 2024-09-20T22:13:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014236899999999999 2024-09-20T22:13:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:33.483 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.342529+0000", 2024-09-20T22:13:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.006312799 2024-09-20T22:13:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343211+0000", 2024-09-20T22:13:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00068264900000000002 2024-09-20T22:13:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343252+0000", 2024-09-20T22:13:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1084000000000002e-05 2024-09-20T22:13:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.484 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343315+0000", 2024-09-20T22:13:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2989000000000003e-05 2024-09-20T22:13:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343412+0000", 2024-09-20T22:13:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6244999999999998e-05 2024-09-20T22:13:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47846 7.3 7:c61861ce:::benchmark_data_smithi060_78800_object47845:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:13:33.485 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.445377+0000", 2024-09-20T22:13:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 552.85405815000001, 2024-09-20T22:13:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91076886199999996, 2024-09-20T22:13:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47846 2024-09-20T22:13:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.486 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445377+0000", 2024-09-20T22:13:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445377+0000", 2024-09-20T22:13:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445378+0000", 2024-09-20T22:13:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.02e-06 2024-09-20T22:13:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.487 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445407+0000", 2024-09-20T22:13:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9654000000000001e-05 2024-09-20T22:13:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445423+0000", 2024-09-20T22:13:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6107999999999999e-05 2024-09-20T22:13:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445432+0000", 2024-09-20T22:13:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6770000000000001e-06 2024-09-20T22:13:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.488 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445522+0000", 2024-09-20T22:13:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0211999999999996e-05 2024-09-20T22:13:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445535+0000", 2024-09-20T22:13:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2279000000000001e-05 2024-09-20T22:13:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237076+0000", 2024-09-20T22:13:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.79154110300000002 2024-09-20T22:13:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.489 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237085+0000", 2024-09-20T22:13:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.611e-06 2024-09-20T22:13:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.338112+0000", 2024-09-20T22:13:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10102702700000001 2024-09-20T22:13:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.338234+0000", 2024-09-20T22:13:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121853 2024-09-20T22:13:33.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:13:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.338375+0000", 2024-09-20T22:13:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140428 2024-09-20T22:13:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343463+0000", 2024-09-20T22:13:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0050883710000000004 2024-09-20T22:13:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343504+0000", 2024-09-20T22:13:33.491 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1185e-05 2024-09-20T22:13:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.355992+0000", 2024-09-20T22:13:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.012487992 2024-09-20T22:13:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.492 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356046+0000", 2024-09-20T22:13:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4033000000000003e-05 2024-09-20T22:13:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356145+0000", 2024-09-20T22:13:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9308999999999994e-05 2024-09-20T22:13:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.493 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47847 7.3 7:c0c5cc37:::benchmark_data_smithi060_78800_object47846:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:13:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.445583+0000", 2024-09-20T22:13:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 552.85385187099996, 2024-09-20T22:13:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91077994900000003, 2024-09-20T22:13:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47847 2024-09-20T22:13:33.494 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445583+0000", 2024-09-20T22:13:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445583+0000", 2024-09-20T22:13:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.495 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445584+0000", 2024-09-20T22:13:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4900000000000005e-07 2024-09-20T22:13:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445607+0000", 2024-09-20T22:13:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3153e-05 2024-09-20T22:13:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445621+0000", 2024-09-20T22:13:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3638000000000001e-05 2024-09-20T22:13:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.496 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445628+0000", 2024-09-20T22:13:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.729e-06 2024-09-20T22:13:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445676+0000", 2024-09-20T22:13:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7911000000000003e-05 2024-09-20T22:13:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.445685+0000", 2024-09-20T22:13:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8289999999999997e-06 2024-09-20T22:13:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.497 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237113+0000", 2024-09-20T22:13:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.79142811599999996 2024-09-20T22:13:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237121+0000", 2024-09-20T22:13:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8310000000000001e-06 2024-09-20T22:13:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.340131+0000", 2024-09-20T22:13:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103009795 2024-09-20T22:13:33.498 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.340324+0000", 2024-09-20T22:13:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000193034 2024-09-20T22:13:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:13:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.340466+0000", 2024-09-20T22:13:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000142032 2024-09-20T22:13:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343542+0000", 2024-09-20T22:13:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0030759860000000002 2024-09-20T22:13:33.499 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343584+0000", 2024-09-20T22:13:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1990999999999998e-05 2024-09-20T22:13:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356203+0000", 2024-09-20T22:13:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.012619361000000001 2024-09-20T22:13:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356268+0000", 2024-09-20T22:13:33.500 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4596000000000003e-05 2024-09-20T22:13:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356363+0000", 2024-09-20T22:13:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5098000000000003e-05 2024-09-20T22:13:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:47853 7.3 7:c21485b4:::benchmark_data_smithi060_78800_object47852:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-09-20T22:13:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:04:20.457698+0000", 2024-09-20T22:13:33.501 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 552.84173657400004, 2024-09-20T22:13:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.89889923500000002, 2024-09-20T22:13:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.502 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 47853 2024-09-20T22:13:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457698+0000", 2024-09-20T22:13:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457698+0000", 2024-09-20T22:13:33.503 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457699+0000", 2024-09-20T22:13:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.159e-06 2024-09-20T22:13:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457753+0000", 2024-09-20T22:13:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4067999999999998e-05 2024-09-20T22:13:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.504 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457765+0000", 2024-09-20T22:13:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1430000000000001e-05 2024-09-20T22:13:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457770+0000", 2024-09-20T22:13:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4410000000000001e-06 2024-09-20T22:13:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457821+0000", 2024-09-20T22:13:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1156999999999998e-05 2024-09-20T22:13:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.505 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:20.457829+0000", 2024-09-20T22:13:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5259999999999996e-06 2024-09-20T22:13:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237712+0000", 2024-09-20T22:13:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.77988319699999997 2024-09-20T22:13:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.237742+0000", 2024-09-20T22:13:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0224000000000001e-05 2024-09-20T22:13:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.506 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343621+0000", 2024-09-20T22:13:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10587846400000001 2024-09-20T22:13:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343759+0000", 2024-09-20T22:13:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000137947 2024-09-20T22:13:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:13:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.343960+0000", 2024-09-20T22:13:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000201201 2024-09-20T22:13:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.507 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.349430+0000", 2024-09-20T22:13:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0054698710000000003 2024-09-20T22:13:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.349473+0000", 2024-09-20T22:13:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2865999999999999e-05 2024-09-20T22:13:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356374+0000", 2024-09-20T22:13:33.508 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0069012359999999998 2024-09-20T22:13:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356434+0000", 2024-09-20T22:13:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9669000000000001e-05 2024-09-20T22:13:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:04:21.356597+0000", 2024-09-20T22:13:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016377899999999999 2024-09-20T22:13:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.509 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169951 7.3 7:c27f31c3:::benchmark_data_smithi060_78800_object36918:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:13:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440033+0000", 2024-09-20T22:13:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 454.85940138299998, 2024-09-20T22:13:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0308894289999999, 2024-09-20T22:13:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:13:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169951 2024-09-20T22:13:33.510 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440033+0000", 2024-09-20T22:13:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440033+0000", 2024-09-20T22:13:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.511 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440035+0000", 2024-09-20T22:13:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3170000000000001e-06 2024-09-20T22:13:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440041+0000", 2024-09-20T22:13:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8869999999999997e-06 2024-09-20T22:13:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440053+0000", 2024-09-20T22:13:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2146e-05 2024-09-20T22:13:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.512 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440061+0000", 2024-09-20T22:13:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3049999999999993e-06 2024-09-20T22:13:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440616+0000", 2024-09-20T22:13:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00055526700000000002 2024-09-20T22:13:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op must wait for map", 2024-09-20T22:13:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440623+0000", 2024-09-20T22:13:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9290000000000001e-06 2024-09-20T22:13:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.513 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.659494+0000", 2024-09-20T22:13:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.21887063800000001 2024-09-20T22:13:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.659506+0000", 2024-09-20T22:13:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2313000000000001e-05 2024-09-20T22:13:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.470588+0000", 2024-09-20T22:13:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.81108138900000004 2024-09-20T22:13:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.514 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.470923+0000", 2024-09-20T22:13:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00033523800000000002 2024-09-20T22:13:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169962 7.8 7:1412370b:::benchmark_data_smithi060_78800_object36929:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:13:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440138+0000", 2024-09-20T22:13:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 454.85929635299999, 2024-09-20T22:13:33.515 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0713576439999999, 2024-09-20T22:13:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:13:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169962 2024-09-20T22:13:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440138+0000", 2024-09-20T22:13:33.516 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440138+0000", 2024-09-20T22:13:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440139+0000", 2024-09-20T22:13:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1399999999999996e-07 2024-09-20T22:13:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440143+0000", 2024-09-20T22:13:33.517 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8029999999999998e-06 2024-09-20T22:13:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440152+0000", 2024-09-20T22:13:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6959999999999994e-06 2024-09-20T22:13:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440158+0000", 2024-09-20T22:13:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3319999999999997e-06 2024-09-20T22:13:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.518 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440263+0000", 2024-09-20T22:13:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105258 2024-09-20T22:13:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op must wait for map", 2024-09-20T22:13:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440269+0000", 2024-09-20T22:13:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8549999999999998e-06 2024-09-20T22:13:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.647667+0000", 2024-09-20T22:13:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.207398358 2024-09-20T22:13:33.519 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.647675+0000", 2024-09-20T22:13:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5979999999999999e-06 2024-09-20T22:13:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.520 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.521 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.521 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.490134+0000", 2024-09-20T22:13:33.521 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.84245882699999997 2024-09-20T22:13:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:13:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.490253+0000", 2024-09-20T22:13:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011938999999999999 2024-09-20T22:13:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.522 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:33.523 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:33.524 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.511425+0000", 2024-09-20T22:13:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021171453999999999 2024-09-20T22:13:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.511496+0000", 2024-09-20T22:13:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1359000000000006e-05 2024-09-20T22:13:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.524 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169969 7.18 7:18661b50:::benchmark_data_smithi060_78800_object36936:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:13:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440181+0000", 2024-09-20T22:13:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 454.85925393500003, 2024-09-20T22:13:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.074892379, 2024-09-20T22:13:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:13:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.525 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169969 2024-09-20T22:13:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440181+0000", 2024-09-20T22:13:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440181+0000", 2024-09-20T22:13:33.526 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440181+0000", 2024-09-20T22:13:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7100000000000002e-07 2024-09-20T22:13:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440185+0000", 2024-09-20T22:13:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3840000000000001e-06 2024-09-20T22:13:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.527 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440193+0000", 2024-09-20T22:13:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5790000000000004e-06 2024-09-20T22:13:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440200+0000", 2024-09-20T22:13:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2310000000000001e-06 2024-09-20T22:13:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.647597+0000", 2024-09-20T22:13:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.20739776200000001 2024-09-20T22:13:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.528 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.647606+0000", 2024-09-20T22:13:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6580000000000008e-06 2024-09-20T22:13:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.510928+0000", 2024-09-20T22:13:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.86332191400000002 2024-09-20T22:13:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:13:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.511051+0000", 2024-09-20T22:13:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012358099999999999 2024-09-20T22:13:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.529 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.514989+0000", 2024-09-20T22:13:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003937744 2024-09-20T22:13:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.515073+0000", 2024-09-20T22:13:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3955000000000005e-05 2024-09-20T22:13:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.530 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227626 7.3 7:c62c9d8d:::benchmark_data_smithi060_78800_object94593:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:13:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.821701+0000", 2024-09-20T22:13:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 399.47773412599997, 2024-09-20T22:13:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1262373590000001, 2024-09-20T22:13:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227626 2024-09-20T22:13:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:33.531 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821701+0000", 2024-09-20T22:13:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821701+0000", 2024-09-20T22:13:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:33.532 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821702+0000", 2024-09-20T22:13:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.102e-06 2024-09-20T22:13:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821707+0000", 2024-09-20T22:13:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0579999999999997e-06 2024-09-20T22:13:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821733+0000", 2024-09-20T22:13:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6642999999999999e-05 2024-09-20T22:13:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:33.533 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821741+0000", 2024-09-20T22:13:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7009999999999996e-06 2024-09-20T22:13:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821806+0000", 2024-09-20T22:13:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4882000000000007e-05 2024-09-20T22:13:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.821817+0000", 2024-09-20T22:13:33.534 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0852999999999999e-05 2024-09-20T22:13:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.941998+0000", 2024-09-20T22:13:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1201808070000001 2024-09-20T22:13:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.942120+0000", 2024-09-20T22:13:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012227900000000001 2024-09-20T22:13:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,3,4", 2024-09-20T22:13:33.535 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.942615+0000", 2024-09-20T22:13:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00049518700000000001 2024-09-20T22:13:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946370+0000", 2024-09-20T22:13:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003755141 2024-09-20T22:13:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946870+0000", 2024-09-20T22:13:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00049939899999999998 2024-09-20T22:13:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.536 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947042+0000", 2024-09-20T22:13:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000172199 2024-09-20T22:13:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947739+0000", 2024-09-20T22:13:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00069714899999999999 2024-09-20T22:13:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947809+0000", 2024-09-20T22:13:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9530999999999995e-05 2024-09-20T22:13:33.537 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947938+0000", 2024-09-20T22:13:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012942800000000001 2024-09-20T22:13:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:33.538 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:33.538 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:13:33.624 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:33.661 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:33.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:33.662 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:33.763 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:33.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:33.797 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:33.798 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:33.898 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:33.931 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:33.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:33.932 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:34.033 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:34.062 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:13:34.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:34.065 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:34.066 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:34.166 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:34.198 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:34.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:34.199 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:34.300 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:34.339 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:34.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:34.340 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:34.441 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:34.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:34.480 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:34.480 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:34.580 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:34.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:34.619 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:34.619 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:34.720 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:34.757 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:34.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:34.758 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:34.847 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T22:13:34.860 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:34.942 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:34.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:34.943 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:35.044 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:35.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:35.097 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:35.097 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:35.198 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:35.203 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:35.206 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:35.223 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:35.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:35.224 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:35.325 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:35.357 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:35 smithi060 ceph-mon[29694]: pgmap v576: 158 pgs: 158 active+clean; 5.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-20T22:13:35.357 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:13:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:13:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:13:35.358 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:35 smithi060 ceph-mon[40290]: pgmap v576: 158 pgs: 158 active+clean; 5.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-20T22:13:35.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:35.370 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:35.370 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:35.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:35.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:35 smithi067 ceph-mon[33511]: pgmap v576: 158 pgs: 158 active+clean; 5.8 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-20T22:13:35.514 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:35.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:35.515 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:35.616 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:35.657 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:35.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:35.659 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:35.760 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:35.801 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:35.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:35.802 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:35.903 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:35.943 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:35.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:35.944 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:36.045 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:36.080 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:36.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:36.081 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:36.182 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:36.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:36.219 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:36.219 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:36.246 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:13:36.320 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:36.342 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:36.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:36.343 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:36.404 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:36 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2858096714' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:13:36.404 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:36 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:13:36.405 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:36 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2858096714' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:13:36.405 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:36 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:13:36.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:36.488 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:36.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:36.489 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:36.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:36 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2858096714' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:13:36.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:36 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:13:36.590 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:36.622 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:13:36.622 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:13:36.622 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:13:36.622 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:13:36.622 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:13:36.622 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:13:36.622 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:13:36.623 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:13:36.623 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:13:36.623 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:13:36.623 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:13:36.623 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:13:36.623 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:13:36.623 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:13:36.627 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:36.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:36.628 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:36.658 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:13:36.658 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:13:36.730 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:36.766 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:36.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:36.767 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:36.868 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:36.904 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:36.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:36.905 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:37.006 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:37.043 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:37.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:37.044 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:37.144 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:37.179 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:37.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:37.180 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:37.281 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:37.315 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:37.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:37.316 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:37.417 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:37.452 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:37.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:37.453 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:37.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:37 smithi067 ceph-mon[33511]: pgmap v577: 158 pgs: 158 active+clean; 5.6 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-09-20T22:13:37.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:37 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:13:37.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:37 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2858096714' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:13:37.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:37 smithi067 ceph-mon[33511]: osdmap e340: 8 total, 8 up, 8 in 2024-09-20T22:13:37.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:37 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4091324531' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:37.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:37 smithi067 ceph-mon[33511]: 4.4 scrub starts 2024-09-20T22:13:37.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:37 smithi067 ceph-mon[33511]: 4.4 scrub ok 2024-09-20T22:13:37.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:37 smithi067 ceph-mon[33511]: osdmap e341: 8 total, 8 up, 8 in 2024-09-20T22:13:37.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:37 smithi067 ceph-mon[33511]: 1.4 scrub starts 2024-09-20T22:13:37.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:37 smithi067 ceph-mon[33511]: 1.4 scrub ok 2024-09-20T22:13:37.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:37 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:13:37.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:37 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:13:37.527 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[29694]: pgmap v577: 158 pgs: 158 active+clean; 5.6 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-09-20T22:13:37.527 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:13:37.527 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2858096714' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:13:37.527 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[29694]: osdmap e340: 8 total, 8 up, 8 in 2024-09-20T22:13:37.527 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4091324531' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:37.527 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[29694]: 4.4 scrub starts 2024-09-20T22:13:37.527 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[29694]: 4.4 scrub ok 2024-09-20T22:13:37.528 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[29694]: osdmap e341: 8 total, 8 up, 8 in 2024-09-20T22:13:37.528 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[29694]: 1.4 scrub starts 2024-09-20T22:13:37.528 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[29694]: 1.4 scrub ok 2024-09-20T22:13:37.528 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:13:37.528 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:13:37.528 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[40290]: pgmap v577: 158 pgs: 158 active+clean; 5.6 GiB data, 21 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.25k op/s 2024-09-20T22:13:37.528 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:13:37.528 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2858096714' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:13:37.528 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[40290]: osdmap e340: 8 total, 8 up, 8 in 2024-09-20T22:13:37.528 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4091324531' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:37.528 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[40290]: 4.4 scrub starts 2024-09-20T22:13:37.528 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[40290]: 4.4 scrub ok 2024-09-20T22:13:37.529 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[40290]: osdmap e341: 8 total, 8 up, 8 in 2024-09-20T22:13:37.529 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[40290]: 1.4 scrub starts 2024-09-20T22:13:37.529 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[40290]: 1.4 scrub ok 2024-09-20T22:13:37.529 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:13:37.529 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:37 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:13:37.553 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:37.588 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:37.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:37.589 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:37.689 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:37.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:37.723 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:37.723 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:37.824 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:37.857 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:37.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:37.858 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:37.959 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:37.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:37.994 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:37.994 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:38.095 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:38.123 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:38.128 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:38.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:38.129 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:38.230 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:38.262 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:38.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:38.263 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:38.364 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:38.396 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:38.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:38.397 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:38.497 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:38.536 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:38.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:38.537 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:38.638 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:38.669 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:38.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:38.670 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:38.771 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:38.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:38 smithi060 ceph-mon[29694]: osdmap e342: 8 total, 8 up, 8 in 2024-09-20T22:13:38.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:38 smithi060 ceph-mon[29694]: pgmap v581: 158 pgs: 158 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-20T22:13:38.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:38 smithi060 ceph-mon[40290]: osdmap e342: 8 total, 8 up, 8 in 2024-09-20T22:13:38.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:38 smithi060 ceph-mon[40290]: pgmap v581: 158 pgs: 158 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-20T22:13:38.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:38.817 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:38.817 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:38.918 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:38.956 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:38.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:38.957 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:38 smithi067 ceph-mon[33511]: osdmap e342: 8 total, 8 up, 8 in 2024-09-20T22:13:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:38 smithi067 ceph-mon[33511]: pgmap v581: 158 pgs: 158 active+clean; 5.5 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.09k op/s 2024-09-20T22:13:39.058 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:39.098 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:39.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:39.099 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:39.177 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.6 reset_purged_snaps_last 2024-09-20T22:13:39.199 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:39.239 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:13:39.279 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:39.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:39.282 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:39.383 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:39.410 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:39.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:39.411 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:39.471 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:13:39.471 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:13:39.512 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:39.543 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:39.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:39.544 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:39.644 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:39.678 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:39.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:39.679 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:39.780 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:39 smithi060 ceph-mon[29694]: 4.11 scrub starts 2024-09-20T22:13:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:39 smithi060 ceph-mon[29694]: 4.11 scrub ok 2024-09-20T22:13:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:39 smithi060 ceph-mon[29694]: 3.9 scrub starts 2024-09-20T22:13:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:39 smithi060 ceph-mon[29694]: 3.9 scrub ok 2024-09-20T22:13:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:39 smithi060 ceph-mon[40290]: 4.11 scrub starts 2024-09-20T22:13:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:39 smithi060 ceph-mon[40290]: 4.11 scrub ok 2024-09-20T22:13:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:39 smithi060 ceph-mon[40290]: 3.9 scrub starts 2024-09-20T22:13:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:39 smithi060 ceph-mon[40290]: 3.9 scrub ok 2024-09-20T22:13:39.819 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:39.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:39.820 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:39.911 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:13:39.920 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:40.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:40.003 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:40.003 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:39 smithi067 ceph-mon[33511]: 4.11 scrub starts 2024-09-20T22:13:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:39 smithi067 ceph-mon[33511]: 4.11 scrub ok 2024-09-20T22:13:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:39 smithi067 ceph-mon[33511]: 3.9 scrub starts 2024-09-20T22:13:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:39 smithi067 ceph-mon[33511]: 3.9 scrub ok 2024-09-20T22:13:40.104 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:40.144 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:40.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:40.145 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:40.239 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T22:13:40.245 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:40.250 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:40.272 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:40.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:40.273 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:40.309 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:40.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:40.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:40.433 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:40.433 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:40.534 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:40.567 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:40.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:40.568 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:40.669 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:40.700 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:40 smithi060 ceph-mon[29694]: 4.19 scrub starts 2024-09-20T22:13:40.700 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:40 smithi060 ceph-mon[29694]: 4.19 scrub ok 2024-09-20T22:13:40.700 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:40 smithi060 ceph-mon[29694]: 9.a scrub starts 2024-09-20T22:13:40.700 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:40 smithi060 ceph-mon[29694]: pgmap v582: 158 pgs: 158 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-20T22:13:40.700 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:40 smithi060 ceph-mon[40290]: 4.19 scrub starts 2024-09-20T22:13:40.700 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:40 smithi060 ceph-mon[40290]: 4.19 scrub ok 2024-09-20T22:13:40.700 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:40 smithi060 ceph-mon[40290]: 9.a scrub starts 2024-09-20T22:13:40.700 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:40 smithi060 ceph-mon[40290]: pgmap v582: 158 pgs: 158 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-20T22:13:40.704 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:40.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:40.705 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:40.806 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:40.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:40.849 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:40.849 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:40.950 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:41.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:41.006 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:41.006 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:40 smithi067 ceph-mon[33511]: 4.19 scrub starts 2024-09-20T22:13:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:40 smithi067 ceph-mon[33511]: 4.19 scrub ok 2024-09-20T22:13:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:40 smithi067 ceph-mon[33511]: 9.a scrub starts 2024-09-20T22:13:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:40 smithi067 ceph-mon[33511]: pgmap v582: 158 pgs: 158 active+clean; 5.4 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-20T22:13:41.107 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:41.149 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:41.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:41.150 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:41.250 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:41.278 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:41.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:41.279 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:41.380 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:41.408 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:41.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:41.409 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:41.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:41.549 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:41.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:41.551 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:41.586 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:13:41.652 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:41.686 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:41.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:41.687 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:41.787 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:41.817 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:41.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:41.818 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:41.914 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:41 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:13:41.914 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:41 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:13:41.914 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:41 smithi067 ceph-mon[33511]: 6.4 scrub starts 2024-09-20T22:13:41.914 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:41 smithi067 ceph-mon[33511]: 6.4 scrub ok 2024-09-20T22:13:41.914 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:41 smithi067 ceph-mon[33511]: osdmap e343: 8 total, 8 up, 8 in 2024-09-20T22:13:41.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:41 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/26343552' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:13:41.919 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:41.953 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:41.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:41.954 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:42.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:41 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:13:42.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:41 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:13:42.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:41 smithi060 ceph-mon[29694]: 6.4 scrub starts 2024-09-20T22:13:42.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:41 smithi060 ceph-mon[29694]: 6.4 scrub ok 2024-09-20T22:13:42.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:41 smithi060 ceph-mon[29694]: osdmap e343: 8 total, 8 up, 8 in 2024-09-20T22:13:42.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:41 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/26343552' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:13:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:41 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:13:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:41 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:13:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:41 smithi060 ceph-mon[40290]: 6.4 scrub starts 2024-09-20T22:13:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:41 smithi060 ceph-mon[40290]: 6.4 scrub ok 2024-09-20T22:13:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:41 smithi060 ceph-mon[40290]: osdmap e343: 8 total, 8 up, 8 in 2024-09-20T22:13:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:41 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/26343552' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:13:42.054 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:42.088 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:42.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:42.089 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:42.189 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:42.212 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:42.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:42.213 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:42.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:42.350 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:42.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:42.351 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:42.452 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:42.488 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:42.488 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:42.489 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:42.589 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:42.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:42.627 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:42.627 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:42.728 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:42.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:42.761 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:42.762 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:42 smithi067 ceph-mon[33511]: 9.a scrub ok 2024-09-20T22:13:42.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:42 smithi067 ceph-mon[33511]: 4.2 scrub starts 2024-09-20T22:13:42.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:42 smithi067 ceph-mon[33511]: 4.2 scrub ok 2024-09-20T22:13:42.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:42 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:13:42.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:42 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:13:42.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:42 smithi067 ceph-mon[33511]: 3.1 scrub starts 2024-09-20T22:13:42.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:42 smithi067 ceph-mon[33511]: 3.1 scrub ok 2024-09-20T22:13:42.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:42 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/26343552' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:13:42.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:42 smithi067 ceph-mon[33511]: osdmap e344: 8 total, 8 up, 8 in 2024-09-20T22:13:42.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:42 smithi067 ceph-mon[33511]: 6.5 scrub starts 2024-09-20T22:13:42.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:42 smithi067 ceph-mon[33511]: 6.5 scrub ok 2024-09-20T22:13:42.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:42 smithi067 ceph-mon[33511]: pgmap v585: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 5.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 904 B/s rd, 0 B/s wr, 1.38k op/s 2024-09-20T22:13:42.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:42 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:13:42.765 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:13:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:13:42.862 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:42.899 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:42.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[29694]: 9.a scrub ok 2024-09-20T22:13:42.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[29694]: 4.2 scrub starts 2024-09-20T22:13:42.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[29694]: 4.2 scrub ok 2024-09-20T22:13:42.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:13:42.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:13:42.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[29694]: 3.1 scrub starts 2024-09-20T22:13:42.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[29694]: 3.1 scrub ok 2024-09-20T22:13:42.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/26343552' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:13:42.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[29694]: osdmap e344: 8 total, 8 up, 8 in 2024-09-20T22:13:42.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[29694]: 6.5 scrub starts 2024-09-20T22:13:42.900 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[29694]: 6.5 scrub ok 2024-09-20T22:13:42.901 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[29694]: pgmap v585: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 5.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 904 B/s rd, 0 B/s wr, 1.38k op/s 2024-09-20T22:13:42.901 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:13:42.901 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[40290]: 9.a scrub ok 2024-09-20T22:13:42.901 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[40290]: 4.2 scrub starts 2024-09-20T22:13:42.901 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[40290]: 4.2 scrub ok 2024-09-20T22:13:42.901 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:13:42.901 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:13:42.901 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[40290]: 3.1 scrub starts 2024-09-20T22:13:42.901 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[40290]: 3.1 scrub ok 2024-09-20T22:13:42.901 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/26343552' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:13:42.901 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[40290]: osdmap e344: 8 total, 8 up, 8 in 2024-09-20T22:13:42.901 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[40290]: 6.5 scrub starts 2024-09-20T22:13:42.902 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[40290]: 6.5 scrub ok 2024-09-20T22:13:42.902 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[40290]: pgmap v585: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 5.1 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 904 B/s rd, 0 B/s wr, 1.38k op/s 2024-09-20T22:13:42.902 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:42 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:13:42.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:42.903 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:43.003 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:43.037 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:43.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:43.038 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:43.138 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:43.163 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:43.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:43.164 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:43.264 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:43.295 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:43.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:43.296 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:43.397 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:43.435 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:43.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:43.435 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:43.536 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:43.567 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:43.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:43.568 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:43.646 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:43.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:43.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:43.697 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:43.697 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:43.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:43 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:13:43.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:43 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:13:43.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:43 smithi060 ceph-mon[29694]: osdmap e345: 8 total, 8 up, 8 in 2024-09-20T22:13:43.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:43 smithi060 ceph-mon[29694]: 9.3 scrub starts 2024-09-20T22:13:43.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:43 smithi060 ceph-mon[29694]: 6.b scrub starts 2024-09-20T22:13:43.784 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:43 smithi060 ceph-mon[29694]: 6.b scrub ok 2024-09-20T22:13:43.784 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:43 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:13:43.784 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:43 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:13:43.784 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:43 smithi060 ceph-mon[40290]: osdmap e345: 8 total, 8 up, 8 in 2024-09-20T22:13:43.784 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:43 smithi060 ceph-mon[40290]: 9.3 scrub starts 2024-09-20T22:13:43.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:43 smithi060 ceph-mon[40290]: 6.b scrub starts 2024-09-20T22:13:43.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:43 smithi060 ceph-mon[40290]: 6.b scrub ok 2024-09-20T22:13:43.798 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:43.834 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:43.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:43.835 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:43.936 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:43.972 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:43.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:43.973 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:43 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:13:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:43 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:13:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:43 smithi067 ceph-mon[33511]: osdmap e345: 8 total, 8 up, 8 in 2024-09-20T22:13:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:43 smithi067 ceph-mon[33511]: 9.3 scrub starts 2024-09-20T22:13:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:43 smithi067 ceph-mon[33511]: 6.b scrub starts 2024-09-20T22:13:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:43 smithi067 ceph-mon[33511]: 6.b scrub ok 2024-09-20T22:13:44.074 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:44.115 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:44.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:44.116 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:44.216 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:44.273 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:44.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:44.274 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:44.374 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:44.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:44.414 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:44.414 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:44.515 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:44.553 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:44.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:44.554 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:44.655 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:44.687 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:44.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:44.688 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:44.784 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.7 reset_purged_snaps_last 2024-09-20T22:13:44.788 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:44.880 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:44.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:44.882 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:44.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[29694]: 5.a scrub starts 2024-09-20T22:13:44.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[29694]: 5.a scrub ok 2024-09-20T22:13:44.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[29694]: 9.0 scrub starts 2024-09-20T22:13:44.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[29694]: 9.f deep-scrub starts 2024-09-20T22:13:44.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[29694]: 6.16 scrub starts 2024-09-20T22:13:44.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[29694]: 6.16 scrub ok 2024-09-20T22:13:44.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[29694]: 9.e scrub starts 2024-09-20T22:13:44.882 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[29694]: 4.1b scrub starts 2024-09-20T22:13:44.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[29694]: 4.1b scrub ok 2024-09-20T22:13:44.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[29694]: pgmap v587: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 5.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-20T22:13:44.884 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[40290]: 5.a scrub starts 2024-09-20T22:13:44.884 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[40290]: 5.a scrub ok 2024-09-20T22:13:44.884 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[40290]: 9.0 scrub starts 2024-09-20T22:13:44.884 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[40290]: 9.f deep-scrub starts 2024-09-20T22:13:44.884 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[40290]: 6.16 scrub starts 2024-09-20T22:13:44.884 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[40290]: 6.16 scrub ok 2024-09-20T22:13:44.884 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[40290]: 9.e scrub starts 2024-09-20T22:13:44.884 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[40290]: 4.1b scrub starts 2024-09-20T22:13:44.884 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[40290]: 4.1b scrub ok 2024-09-20T22:13:44.884 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:44 smithi060 ceph-mon[40290]: pgmap v587: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 5.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-20T22:13:44.983 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:45.013 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:44 smithi067 ceph-mon[33511]: 5.a scrub starts 2024-09-20T22:13:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:44 smithi067 ceph-mon[33511]: 5.a scrub ok 2024-09-20T22:13:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:44 smithi067 ceph-mon[33511]: 9.0 scrub starts 2024-09-20T22:13:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:44 smithi067 ceph-mon[33511]: 9.f deep-scrub starts 2024-09-20T22:13:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:44 smithi067 ceph-mon[33511]: 6.16 scrub starts 2024-09-20T22:13:45.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:44 smithi067 ceph-mon[33511]: 6.16 scrub ok 2024-09-20T22:13:45.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:44 smithi067 ceph-mon[33511]: 9.e scrub starts 2024-09-20T22:13:45.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:44 smithi067 ceph-mon[33511]: 4.1b scrub starts 2024-09-20T22:13:45.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:44 smithi067 ceph-mon[33511]: 4.1b scrub ok 2024-09-20T22:13:45.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:44 smithi067 ceph-mon[33511]: pgmap v587: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 5.0 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-20T22:13:45.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:45.014 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:45.115 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:45.147 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:45.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:45.148 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:45.243 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:13:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:13:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:13:45.248 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:45.280 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:45.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:45.281 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:45.359 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:45.382 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:45.408 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:45.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:45.409 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:45.446 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:13:45.446 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:13:45.485 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:13:45.485 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:13:45.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:45.523 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:13:45.523 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:13:45.558 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:13:45.559 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:13:45.562 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:45.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:45.563 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:45.592 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:13:45.592 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:13:45.631 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:13:45.631 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:13:45.662 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:13:45.662 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:13:45.664 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:45.690 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:45.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:45.691 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:45.700 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:13:45.700 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:13:45.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:45 smithi060 ceph-mon[29694]: 9.3 scrub ok 2024-09-20T22:13:45.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:45 smithi060 ceph-mon[29694]: 4.6 scrub starts 2024-09-20T22:13:45.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:45 smithi060 ceph-mon[29694]: 4.6 scrub ok 2024-09-20T22:13:45.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:45 smithi060 ceph-mon[29694]: 9.e scrub ok 2024-09-20T22:13:45.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:45 smithi060 ceph-mon[29694]: 9.0 scrub ok 2024-09-20T22:13:45.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:45 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2652571678' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:45.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:45 smithi060 ceph-mon[40290]: 9.3 scrub ok 2024-09-20T22:13:45.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:45 smithi060 ceph-mon[40290]: 4.6 scrub starts 2024-09-20T22:13:45.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:45 smithi060 ceph-mon[40290]: 4.6 scrub ok 2024-09-20T22:13:45.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:45 smithi060 ceph-mon[40290]: 9.e scrub ok 2024-09-20T22:13:45.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:45 smithi060 ceph-mon[40290]: 9.0 scrub ok 2024-09-20T22:13:45.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:45 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2652571678' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:45.792 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:45.820 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:45.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:45.821 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:45.922 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:45.965 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:45.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:45.966 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:46.003 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:13:46.003 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:13:46.003 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:13:46.003 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:13:46.004 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:13:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:45 smithi067 ceph-mon[33511]: 9.3 scrub ok 2024-09-20T22:13:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:45 smithi067 ceph-mon[33511]: 4.6 scrub starts 2024-09-20T22:13:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:45 smithi067 ceph-mon[33511]: 4.6 scrub ok 2024-09-20T22:13:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:45 smithi067 ceph-mon[33511]: 9.e scrub ok 2024-09-20T22:13:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:45 smithi067 ceph-mon[33511]: 9.0 scrub ok 2024-09-20T22:13:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:45 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2652571678' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:46.067 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:46.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:46.108 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:46.108 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:46.209 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:46.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:46.239 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:46.240 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:46.293 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:13:46.340 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:46.367 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:46.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:46.368 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:46.449 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:46.469 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:46.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:46.538 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:46.538 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:46.639 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:46.668 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:46.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:46.669 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:46.770 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:46.808 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:46.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:46.809 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:46.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:46.949 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:46.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:46.950 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:46 smithi067 ceph-mon[33511]: pgmap v588: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 4.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.44k op/s 2024-09-20T22:13:47.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:46 smithi060 ceph-mon[29694]: pgmap v588: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 4.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.44k op/s 2024-09-20T22:13:47.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:46 smithi060 ceph-mon[40290]: pgmap v588: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 4.9 GiB data, 19 GiB used, 697 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.44k op/s 2024-09-20T22:13:47.051 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:47.072 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:47.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:47.073 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:47.174 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:47.210 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:47.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:47.211 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:47.311 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:47.340 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:47.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:47.341 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:47.442 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:47.468 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:47.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:47.469 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:47.569 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:47.598 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:47.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:47.599 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:47.700 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:47.733 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:47.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:47.734 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:47.835 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:47.857 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:47.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:47.857 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:47.908 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:47 smithi060 ceph-mon[29694]: 4.1c scrub starts 2024-09-20T22:13:47.908 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:47 smithi060 ceph-mon[29694]: 4.1c scrub ok 2024-09-20T22:13:47.908 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:47 smithi060 ceph-mon[29694]: 4.b scrub starts 2024-09-20T22:13:47.908 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:47 smithi060 ceph-mon[29694]: 4.b scrub ok 2024-09-20T22:13:47.908 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:47 smithi060 ceph-mon[29694]: 9.2 scrub starts 2024-09-20T22:13:47.908 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:47 smithi060 ceph-mon[40290]: 4.1c scrub starts 2024-09-20T22:13:47.908 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:47 smithi060 ceph-mon[40290]: 4.1c scrub ok 2024-09-20T22:13:47.908 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:47 smithi060 ceph-mon[40290]: 4.b scrub starts 2024-09-20T22:13:47.908 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:47 smithi060 ceph-mon[40290]: 4.b scrub ok 2024-09-20T22:13:47.909 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:47 smithi060 ceph-mon[40290]: 9.2 scrub starts 2024-09-20T22:13:47.958 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:47.986 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:47.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:47.987 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:47 smithi067 ceph-mon[33511]: 4.1c scrub starts 2024-09-20T22:13:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:47 smithi067 ceph-mon[33511]: 4.1c scrub ok 2024-09-20T22:13:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:47 smithi067 ceph-mon[33511]: 4.b scrub starts 2024-09-20T22:13:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:47 smithi067 ceph-mon[33511]: 4.b scrub ok 2024-09-20T22:13:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:47 smithi067 ceph-mon[33511]: 9.2 scrub starts 2024-09-20T22:13:48.088 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:48.115 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:48.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:48.116 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:48.217 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:48.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:48.255 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:48.255 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:48.356 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:48.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:48.390 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:48.391 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:48.491 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:48.535 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:48.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:48.536 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:48.637 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:48.664 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:48.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:48.665 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:48.766 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:48.803 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:48.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:48.805 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:48.905 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:48.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:48.944 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:48.944 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:49.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:48 smithi067 ceph-mon[33511]: 3.6 scrub starts 2024-09-20T22:13:49.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:48 smithi067 ceph-mon[33511]: 3.6 scrub ok 2024-09-20T22:13:49.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:48 smithi067 ceph-mon[33511]: pgmap v589: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 4.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-20T22:13:49.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:48 smithi060 ceph-mon[29694]: 3.6 scrub starts 2024-09-20T22:13:49.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:48 smithi060 ceph-mon[29694]: 3.6 scrub ok 2024-09-20T22:13:49.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:48 smithi060 ceph-mon[29694]: pgmap v589: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 4.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-20T22:13:49.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:48 smithi060 ceph-mon[40290]: 3.6 scrub starts 2024-09-20T22:13:49.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:48 smithi060 ceph-mon[40290]: 3.6 scrub ok 2024-09-20T22:13:49.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:48 smithi060 ceph-mon[40290]: pgmap v589: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 4.8 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-20T22:13:49.045 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:49.080 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:49.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:49.081 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:49.182 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:49.203 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:49.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:49.204 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:49.305 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:49.343 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:49.344 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:49.345 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:49.445 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:49.476 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:49.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:49.478 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:49.579 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:49.622 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:49.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:49.623 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:49.723 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:49.760 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:49.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:49.761 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:49.862 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:49.892 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:49.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:49.893 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:49.994 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:50.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:49 smithi067 ceph-mon[33511]: 6.17 scrub starts 2024-09-20T22:13:50.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:49 smithi067 ceph-mon[33511]: 6.17 scrub ok 2024-09-20T22:13:50.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:49 smithi067 ceph-mon[33511]: 9.2 scrub ok 2024-09-20T22:13:50.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:49 smithi067 ceph-mon[33511]: 9.8 scrub starts 2024-09-20T22:13:50.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:49 smithi067 ceph-mon[33511]: 1.3 deep-scrub starts 2024-09-20T22:13:50.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:49 smithi067 ceph-mon[33511]: 1.3 deep-scrub ok 2024-09-20T22:13:50.029 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:50.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:50.030 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:50.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:49 smithi060 ceph-mon[29694]: 6.17 scrub starts 2024-09-20T22:13:50.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:49 smithi060 ceph-mon[29694]: 6.17 scrub ok 2024-09-20T22:13:50.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:49 smithi060 ceph-mon[29694]: 9.2 scrub ok 2024-09-20T22:13:50.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:49 smithi060 ceph-mon[29694]: 9.8 scrub starts 2024-09-20T22:13:50.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:49 smithi060 ceph-mon[29694]: 1.3 deep-scrub starts 2024-09-20T22:13:50.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:49 smithi060 ceph-mon[29694]: 1.3 deep-scrub ok 2024-09-20T22:13:50.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:49 smithi060 ceph-mon[40290]: 6.17 scrub starts 2024-09-20T22:13:50.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:49 smithi060 ceph-mon[40290]: 6.17 scrub ok 2024-09-20T22:13:50.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:49 smithi060 ceph-mon[40290]: 9.2 scrub ok 2024-09-20T22:13:50.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:49 smithi060 ceph-mon[40290]: 9.8 scrub starts 2024-09-20T22:13:50.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:49 smithi060 ceph-mon[40290]: 1.3 deep-scrub starts 2024-09-20T22:13:50.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:49 smithi060 ceph-mon[40290]: 1.3 deep-scrub ok 2024-09-20T22:13:50.130 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:50.159 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:50.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:50.160 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:50.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:50.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:50.299 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:50.299 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:50.400 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:50.441 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:50.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:50.442 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:50.542 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:50.575 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:50.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:50.576 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:50.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T22:13:50.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:50.753 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:50.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:50.754 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:50.855 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:50.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:50.896 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:50.896 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:50.997 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:50 smithi067 ceph-mon[33511]: 5.b scrub starts 2024-09-20T22:13:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:50 smithi067 ceph-mon[33511]: 5.b scrub ok 2024-09-20T22:13:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:50 smithi067 ceph-mon[33511]: 9.c scrub starts 2024-09-20T22:13:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:50 smithi067 ceph-mon[33511]: 9.8 scrub ok 2024-09-20T22:13:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:50 smithi067 ceph-mon[33511]: 4.14 scrub starts 2024-09-20T22:13:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:50 smithi067 ceph-mon[33511]: 4.14 scrub ok 2024-09-20T22:13:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:50 smithi067 ceph-mon[33511]: 1.9 scrub starts 2024-09-20T22:13:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:50 smithi067 ceph-mon[33511]: 1.9 scrub ok 2024-09-20T22:13:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:50 smithi067 ceph-mon[33511]: pgmap v590: 158 pgs: 1 active+clean+scrubbing, 1 active+clean+scrubbing+deep, 156 active+clean; 4.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 595 B/s rd, 0 B/s wr, 912 op/s 2024-09-20T22:13:51.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:50 smithi067 ceph-mon[33511]: 9.1 scrub starts 2024-09-20T22:13:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[29694]: 5.b scrub starts 2024-09-20T22:13:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[29694]: 5.b scrub ok 2024-09-20T22:13:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[29694]: 9.c scrub starts 2024-09-20T22:13:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[29694]: 9.8 scrub ok 2024-09-20T22:13:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[29694]: 4.14 scrub starts 2024-09-20T22:13:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[29694]: 4.14 scrub ok 2024-09-20T22:13:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[29694]: 1.9 scrub starts 2024-09-20T22:13:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[29694]: 1.9 scrub ok 2024-09-20T22:13:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[29694]: pgmap v590: 158 pgs: 1 active+clean+scrubbing, 1 active+clean+scrubbing+deep, 156 active+clean; 4.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 595 B/s rd, 0 B/s wr, 912 op/s 2024-09-20T22:13:51.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[29694]: 9.1 scrub starts 2024-09-20T22:13:51.037 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:51.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[40290]: 5.b scrub starts 2024-09-20T22:13:51.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[40290]: 5.b scrub ok 2024-09-20T22:13:51.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[40290]: 9.c scrub starts 2024-09-20T22:13:51.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[40290]: 9.8 scrub ok 2024-09-20T22:13:51.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[40290]: 4.14 scrub starts 2024-09-20T22:13:51.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[40290]: 4.14 scrub ok 2024-09-20T22:13:51.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[40290]: 1.9 scrub starts 2024-09-20T22:13:51.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[40290]: 1.9 scrub ok 2024-09-20T22:13:51.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[40290]: pgmap v590: 158 pgs: 1 active+clean+scrubbing, 1 active+clean+scrubbing+deep, 156 active+clean; 4.7 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 595 B/s rd, 0 B/s wr, 912 op/s 2024-09-20T22:13:51.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:50 smithi060 ceph-mon[40290]: 9.1 scrub starts 2024-09-20T22:13:51.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:51.039 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:51.139 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:51.185 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:51.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:51.186 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:51.290 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:51.338 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:51.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:51.339 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:51.440 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:51.491 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:51.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:51.494 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:51.595 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:51.629 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:51.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:51.631 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:51.731 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:51.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:51 smithi060 ceph-mon[29694]: 9.c scrub ok 2024-09-20T22:13:51.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:51 smithi060 ceph-mon[29694]: 9.5 scrub starts 2024-09-20T22:13:51.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:13:51.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:13:51.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:51 smithi060 ceph-mon[29694]: 4.0 scrub starts 2024-09-20T22:13:51.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:51 smithi060 ceph-mon[29694]: 4.0 scrub ok 2024-09-20T22:13:51.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:51 smithi060 ceph-mon[29694]: 9.1 scrub ok 2024-09-20T22:13:51.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:51 smithi060 ceph-mon[29694]: 4.a scrub starts 2024-09-20T22:13:51.760 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:51 smithi060 ceph-mon[29694]: 4.a scrub ok 2024-09-20T22:13:51.760 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:51 smithi060 ceph-mon[40290]: 9.c scrub ok 2024-09-20T22:13:51.760 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:51 smithi060 ceph-mon[40290]: 9.5 scrub starts 2024-09-20T22:13:51.760 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:13:51.761 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:13:51.761 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:51 smithi060 ceph-mon[40290]: 4.0 scrub starts 2024-09-20T22:13:51.761 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:51 smithi060 ceph-mon[40290]: 4.0 scrub ok 2024-09-20T22:13:51.761 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:51 smithi060 ceph-mon[40290]: 9.1 scrub ok 2024-09-20T22:13:51.761 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:51 smithi060 ceph-mon[40290]: 4.a scrub starts 2024-09-20T22:13:51.761 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:51 smithi060 ceph-mon[40290]: 4.a scrub ok 2024-09-20T22:13:51.762 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:51.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:51.763 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:51.772 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:51.823 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:51.863 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:51.907 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:51.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:51.908 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:52.009 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:51 smithi067 ceph-mon[33511]: 9.c scrub ok 2024-09-20T22:13:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:51 smithi067 ceph-mon[33511]: 9.5 scrub starts 2024-09-20T22:13:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:13:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:13:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:51 smithi067 ceph-mon[33511]: 4.0 scrub starts 2024-09-20T22:13:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:51 smithi067 ceph-mon[33511]: 4.0 scrub ok 2024-09-20T22:13:52.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:51 smithi067 ceph-mon[33511]: 9.1 scrub ok 2024-09-20T22:13:52.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:51 smithi067 ceph-mon[33511]: 4.a scrub starts 2024-09-20T22:13:52.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:51 smithi067 ceph-mon[33511]: 4.a scrub ok 2024-09-20T22:13:52.040 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:52.040 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:52.040 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:52.141 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:52.178 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:52.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:52.179 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:52.280 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:52.320 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:52.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:52.322 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:52.422 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:52.454 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:52.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:52.454 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:52.555 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:52.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:52.596 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:52.596 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:52.658 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:13:52 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:13:52.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:52.732 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:52.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:52.734 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[29694]: 6.6 scrub starts 2024-09-20T22:13:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[29694]: 6.6 scrub ok 2024-09-20T22:13:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[29694]: 5.1c scrub starts 2024-09-20T22:13:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[29694]: 5.1c scrub ok 2024-09-20T22:13:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[29694]: 2.2 deep-scrub starts 2024-09-20T22:13:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[29694]: 2.2 deep-scrub ok 2024-09-20T22:13:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[29694]: 4.7 scrub starts 2024-09-20T22:13:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[29694]: 9.5 scrub ok 2024-09-20T22:13:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[29694]: 5.4 deep-scrub starts 2024-09-20T22:13:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[29694]: 5.4 deep-scrub ok 2024-09-20T22:13:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[29694]: 4.7 scrub ok 2024-09-20T22:13:52.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[29694]: pgmap v591: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 156 active+clean; 4.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 968 op/s 2024-09-20T22:13:52.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:13:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[40290]: 6.6 scrub starts 2024-09-20T22:13:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[40290]: 6.6 scrub ok 2024-09-20T22:13:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[40290]: 5.1c scrub starts 2024-09-20T22:13:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[40290]: 5.1c scrub ok 2024-09-20T22:13:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[40290]: 2.2 deep-scrub starts 2024-09-20T22:13:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[40290]: 2.2 deep-scrub ok 2024-09-20T22:13:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[40290]: 4.7 scrub starts 2024-09-20T22:13:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[40290]: 9.5 scrub ok 2024-09-20T22:13:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[40290]: 5.4 deep-scrub starts 2024-09-20T22:13:52.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[40290]: 5.4 deep-scrub ok 2024-09-20T22:13:52.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[40290]: 4.7 scrub ok 2024-09-20T22:13:52.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[40290]: pgmap v591: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 156 active+clean; 4.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 968 op/s 2024-09-20T22:13:52.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:52 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:13:52.834 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:52.869 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:52.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:52.870 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:52.971 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:53.004 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:53.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:53.005 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:52 smithi067 ceph-mon[33511]: 6.6 scrub starts 2024-09-20T22:13:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:52 smithi067 ceph-mon[33511]: 6.6 scrub ok 2024-09-20T22:13:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:52 smithi067 ceph-mon[33511]: 5.1c scrub starts 2024-09-20T22:13:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:52 smithi067 ceph-mon[33511]: 5.1c scrub ok 2024-09-20T22:13:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:52 smithi067 ceph-mon[33511]: 2.2 deep-scrub starts 2024-09-20T22:13:53.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:52 smithi067 ceph-mon[33511]: 2.2 deep-scrub ok 2024-09-20T22:13:53.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:52 smithi067 ceph-mon[33511]: 4.7 scrub starts 2024-09-20T22:13:53.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:52 smithi067 ceph-mon[33511]: 9.5 scrub ok 2024-09-20T22:13:53.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:52 smithi067 ceph-mon[33511]: 5.4 deep-scrub starts 2024-09-20T22:13:53.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:52 smithi067 ceph-mon[33511]: 5.4 deep-scrub ok 2024-09-20T22:13:53.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:52 smithi067 ceph-mon[33511]: 4.7 scrub ok 2024-09-20T22:13:53.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:52 smithi067 ceph-mon[33511]: pgmap v591: 158 pgs: 1 active+clean+scrubbing+deep, 1 active+clean+scrubbing, 156 active+clean; 4.5 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 968 op/s 2024-09-20T22:13:53.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:52 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:13:53.037 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.0 reset_purged_snaps_last 2024-09-20T22:13:53.107 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:53.136 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:53.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:53.137 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:53.238 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:53.245 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:13:53.265 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:53.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:53.266 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:53.367 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:53.434 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:53.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:53.435 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:53.536 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:53.562 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:53.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:53.563 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:53.646 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:13:53.646 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:13:53.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:13:53.646 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:13:53.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.646 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169961 7.13 7:cf59cdb9:::benchmark_data_smithi060_78800_object36928:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:13:53.646 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440329+0000", 2024-09-20T22:13:53.646 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 475.202480568, 2024-09-20T22:13:53.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.067711163, 2024-09-20T22:13:53.647 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.647 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.647 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:13:53.647 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.647 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:53.647 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:53.647 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169961 2024-09-20T22:13:53.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.647 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440329+0000", 2024-09-20T22:13:53.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440329+0000", 2024-09-20T22:13:53.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440330+0000", 2024-09-20T22:13:53.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2800000000000005e-07 2024-09-20T22:13:53.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440335+0000", 2024-09-20T22:13:53.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9520000000000002e-06 2024-09-20T22:13:53.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440347+0000", 2024-09-20T22:13:53.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1732999999999999e-05 2024-09-20T22:13:53.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440354+0000", 2024-09-20T22:13:53.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6420000000000007e-06 2024-09-20T22:13:53.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.698470+0000", 2024-09-20T22:13:53.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.25811523800000002 2024-09-20T22:13:53.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.698477+0000", 2024-09-20T22:13:53.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9999999999999999e-06 2024-09-20T22:13:53.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.507960+0000", 2024-09-20T22:13:53.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.809483493 2024-09-20T22:13:53.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.651 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.651 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.508040+0000", 2024-09-20T22:13:53.651 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0177000000000002e-05 2024-09-20T22:13:53.651 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.651 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.651 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.652 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:169966 7.13 7:c9a28d28:::benchmark_data_smithi060_78800_object36933:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:13:53.652 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:05:58.440368+0000", 2024-09-20T22:13:53.652 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 475.20244131200002, 2024-09-20T22:13:53.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0730672910000001, 2024-09-20T22:13:53.653 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.653 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.653 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:13:53.653 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.653 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:53.653 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:53.653 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 169966 2024-09-20T22:13:53.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.653 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440368+0000", 2024-09-20T22:13:53.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440368+0000", 2024-09-20T22:13:53.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440369+0000", 2024-09-20T22:13:53.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9100000000000004e-07 2024-09-20T22:13:53.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440371+0000", 2024-09-20T22:13:53.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5280000000000002e-06 2024-09-20T22:13:53.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440378+0000", 2024-09-20T22:13:53.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7340000000000002e-06 2024-09-20T22:13:53.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.440383+0000", 2024-09-20T22:13:53.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3750000000000002e-06 2024-09-20T22:13:53.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.698495+0000", 2024-09-20T22:13:53.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.258111427 2024-09-20T22:13:53.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:58.698499+0000", 2024-09-20T22:13:53.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.658e-06 2024-09-20T22:13:53.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.509530+0000", 2024-09-20T22:13:53.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.81103158399999997 2024-09-20T22:13:53.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:13:53.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.509656+0000", 2024-09-20T22:13:53.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012591300000000001 2024-09-20T22:13:53.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.513371+0000", 2024-09-20T22:13:53.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037147370000000001 2024-09-20T22:13:53.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:05:59.513436+0000", 2024-09-20T22:13:53.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4844000000000005e-05 2024-09-20T22:13:53.657 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.657 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.657 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.658 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198108 7.10 7:0b18f923:::benchmark_data_smithi060_78800_object65075:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:13:53.658 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.462286+0000", 2024-09-20T22:13:53.658 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 448.18052395799998, 2024-09-20T22:13:53.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97321715200000003, 2024-09-20T22:13:53.658 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.658 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.658 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:53.658 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.658 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:53.658 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:53.658 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198108 2024-09-20T22:13:53.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.658 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462286+0000", 2024-09-20T22:13:53.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462286+0000", 2024-09-20T22:13:53.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462287+0000", 2024-09-20T22:13:53.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.668e-06 2024-09-20T22:13:53.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462293+0000", 2024-09-20T22:13:53.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0020000000000003e-06 2024-09-20T22:13:53.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462315+0000", 2024-09-20T22:13:53.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1809000000000001e-05 2024-09-20T22:13:53.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462324+0000", 2024-09-20T22:13:53.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6209999999999993e-06 2024-09-20T22:13:53.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462389+0000", 2024-09-20T22:13:53.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5007999999999998e-05 2024-09-20T22:13:53.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.462397+0000", 2024-09-20T22:13:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3690000000000007e-06 2024-09-20T22:13:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430729+0000", 2024-09-20T22:13:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.96833147100000005 2024-09-20T22:13:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.430856+0000", 2024-09-20T22:13:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012690800000000001 2024-09-20T22:13:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:13:53.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.431205+0000", 2024-09-20T22:13:53.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00034923899999999999 2024-09-20T22:13:53.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435090+0000", 2024-09-20T22:13:53.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038847090000000001 2024-09-20T22:13:53.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:53.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435103+0000", 2024-09-20T22:13:53.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169999999999999e-05 2024-09-20T22:13:53.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435254+0000", 2024-09-20T22:13:53.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015143900000000001 2024-09-20T22:13:53.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:53.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435404+0000", 2024-09-20T22:13:53.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014957600000000001 2024-09-20T22:13:53.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435503+0000", 2024-09-20T22:13:53.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9162999999999999e-05 2024-09-20T22:13:53.664 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.665 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.665 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.665 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198131 7.10 7:0b2c8a99:::benchmark_data_smithi060_78800_object65098:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:13:53.665 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.479927+0000", 2024-09-20T22:13:53.665 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 448.16288262199998, 2024-09-20T22:13:53.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.96409556600000001, 2024-09-20T22:13:53.665 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.665 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.665 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:53.666 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.666 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:53.666 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:53.666 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198131 2024-09-20T22:13:53.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.667 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479927+0000", 2024-09-20T22:13:53.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479927+0000", 2024-09-20T22:13:53.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479929+0000", 2024-09-20T22:13:53.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5850000000000001e-06 2024-09-20T22:13:53.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479934+0000", 2024-09-20T22:13:53.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1869999999999996e-06 2024-09-20T22:13:53.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479955+0000", 2024-09-20T22:13:53.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1353e-05 2024-09-20T22:13:53.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.479962+0000", 2024-09-20T22:13:53.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7549999999999997e-06 2024-09-20T22:13:53.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.480018+0000", 2024-09-20T22:13:53.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6342999999999999e-05 2024-09-20T22:13:53.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.480027+0000", 2024-09-20T22:13:53.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7469999999999994e-06 2024-09-20T22:13:53.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.433323+0000", 2024-09-20T22:13:53.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.95329631199999998 2024-09-20T22:13:53.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:53.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.433425+0000", 2024-09-20T22:13:53.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000101271 2024-09-20T22:13:53.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:13:53.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.433690+0000", 2024-09-20T22:13:53.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026574900000000002 2024-09-20T22:13:53.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:53.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.437320+0000", 2024-09-20T22:13:53.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003630075 2024-09-20T22:13:53.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.437363+0000", 2024-09-20T22:13:53.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2104000000000002e-05 2024-09-20T22:13:53.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.443794+0000", 2024-09-20T22:13:53.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0064310729999999998 2024-09-20T22:13:53.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:53.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.443902+0000", 2024-09-20T22:13:53.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010815 2024-09-20T22:13:53.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.444023+0000", 2024-09-20T22:13:53.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000120862 2024-09-20T22:13:53.673 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.673 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.673 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.673 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:198163 7.10 7:086b2df9:::benchmark_data_smithi060_78800_object65130:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e133)", 2024-09-20T22:13:53.673 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:25.501070+0000", 2024-09-20T22:13:53.673 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 448.14173924900001, 2024-09-20T22:13:53.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94904837799999997, 2024-09-20T22:13:53.674 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.674 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.674 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:53.674 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.674 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:53.674 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:53.674 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 198163 2024-09-20T22:13:53.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.674 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501070+0000", 2024-09-20T22:13:53.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501070+0000", 2024-09-20T22:13:53.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501072+0000", 2024-09-20T22:13:53.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.742e-06 2024-09-20T22:13:53.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501078+0000", 2024-09-20T22:13:53.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3859999999999998e-06 2024-09-20T22:13:53.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501090+0000", 2024-09-20T22:13:53.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2003e-05 2024-09-20T22:13:53.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501097+0000", 2024-09-20T22:13:53.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2089999999999999e-06 2024-09-20T22:13:53.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501154+0000", 2024-09-20T22:13:53.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6901000000000003e-05 2024-09-20T22:13:53.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:25.501162+0000", 2024-09-20T22:13:53.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3440000000000001e-06 2024-09-20T22:13:53.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435556+0000", 2024-09-20T22:13:53.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.93439354699999999 2024-09-20T22:13:53.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:53.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435672+0000", 2024-09-20T22:13:53.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011694 2024-09-20T22:13:53.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:13:53.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.435963+0000", 2024-09-20T22:13:53.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000290654 2024-09-20T22:13:53.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:53.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.440444+0000", 2024-09-20T22:13:53.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0044806450000000001 2024-09-20T22:13:53.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.679 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.440486+0000", 2024-09-20T22:13:53.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2036999999999997e-05 2024-09-20T22:13:53.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.679 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.449968+0000", 2024-09-20T22:13:53.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0094824560000000002 2024-09-20T22:13:53.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:53.679 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450024+0000", 2024-09-20T22:13:53.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5374000000000002e-05 2024-09-20T22:13:53.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.680 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:26.450119+0000", 2024-09-20T22:13:53.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5140000000000001e-05 2024-09-20T22:13:53.680 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.680 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.680 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.680 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227634 7.10 7:0ea5dec3:::benchmark_data_smithi060_78800_object94601:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:13:53.680 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.846865+0000", 2024-09-20T22:13:53.680 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 419.79594442299998, 2024-09-20T22:13:53.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1004788329999999, 2024-09-20T22:13:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227634 2024-09-20T22:13:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.682 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846865+0000", 2024-09-20T22:13:53.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846865+0000", 2024-09-20T22:13:53.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846867+0000", 2024-09-20T22:13:53.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.641e-06 2024-09-20T22:13:53.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846873+0000", 2024-09-20T22:13:53.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1489999999999999e-06 2024-09-20T22:13:53.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846887+0000", 2024-09-20T22:13:53.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4113e-05 2024-09-20T22:13:53.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.846895+0000", 2024-09-20T22:13:53.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6699999999999994e-06 2024-09-20T22:13:53.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.852299+0000", 2024-09-20T22:13:53.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0054038059999999997 2024-09-20T22:13:53.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.852311+0000", 2024-09-20T22:13:53.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2278e-05 2024-09-20T22:13:53.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.941457+0000", 2024-09-20T22:13:53.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0891458009999999 2024-09-20T22:13:53.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:53.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.941594+0000", 2024-09-20T22:13:53.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013708000000000001 2024-09-20T22:13:53.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2,5", 2024-09-20T22:13:53.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.941949+0000", 2024-09-20T22:13:53.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00035502599999999999 2024-09-20T22:13:53.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946263+0000", 2024-09-20T22:13:53.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0043141150000000003 2024-09-20T22:13:53.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:53.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946286+0000", 2024-09-20T22:13:53.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3419000000000001e-05 2024-09-20T22:13:53.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946351+0000", 2024-09-20T22:13:53.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4231000000000002e-05 2024-09-20T22:13:53.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947173+0000", 2024-09-20T22:13:53.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00082238199999999997 2024-09-20T22:13:53.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:53.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947237+0000", 2024-09-20T22:13:53.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4462999999999996e-05 2024-09-20T22:13:53.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947344+0000", 2024-09-20T22:13:53.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000106659 2024-09-20T22:13:53.688 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.688 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.688 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.689 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227639 7.10 7:0834985a:::benchmark_data_smithi060_78800_object94606:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:13:53.689 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.856347+0000", 2024-09-20T22:13:53.689 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 419.78646285999997, 2024-09-20T22:13:53.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0915877439999999, 2024-09-20T22:13:53.689 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.689 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.689 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:53.689 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.689 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:53.689 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:53.689 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227639 2024-09-20T22:13:53.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.689 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856347+0000", 2024-09-20T22:13:53.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856347+0000", 2024-09-20T22:13:53.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856349+0000", 2024-09-20T22:13:53.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7090000000000001e-06 2024-09-20T22:13:53.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.691 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:53.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856355+0000", 2024-09-20T22:13:53.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4729999999999997e-06 2024-09-20T22:13:53.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856369+0000", 2024-09-20T22:13:53.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3832000000000001e-05 2024-09-20T22:13:53.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856378+0000", 2024-09-20T22:13:53.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8689999999999993e-06 2024-09-20T22:13:53.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856556+0000", 2024-09-20T22:13:53.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000178529 2024-09-20T22:13:53.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.856568+0000", 2024-09-20T22:13:53.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2281e-05 2024-09-20T22:13:53.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.944093+0000", 2024-09-20T22:13:53.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.087524438 2024-09-20T22:13:53.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:53.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.944232+0000", 2024-09-20T22:13:53.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000139178 2024-09-20T22:13:53.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2,5", 2024-09-20T22:13:53.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.944533+0000", 2024-09-20T22:13:53.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030114999999999997 2024-09-20T22:13:53.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946726+0000", 2024-09-20T22:13:53.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021925809999999999 2024-09-20T22:13:53.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:53.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.946797+0000", 2024-09-20T22:13:53.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1570999999999996e-05 2024-09-20T22:13:53.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947394+0000", 2024-09-20T22:13:53.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00059673799999999998 2024-09-20T22:13:53.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947773+0000", 2024-09-20T22:13:53.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00037865499999999999 2024-09-20T22:13:53.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:53.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947831+0000", 2024-09-20T22:13:53.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8557000000000002e-05 2024-09-20T22:13:53.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.947935+0000", 2024-09-20T22:13:53.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103183 2024-09-20T22:13:53.696 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.696 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.696 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.696 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227649 7.2 7:46c04214:::benchmark_data_smithi060_78800_object94616:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:13:53.697 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.900917+0000", 2024-09-20T22:13:53.697 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 419.74189281500003, 2024-09-20T22:13:53.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1912150110000002, 2024-09-20T22:13:53.697 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.697 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.697 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:53.697 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.697 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:53.697 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:53.697 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227649 2024-09-20T22:13:53.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.697 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900917+0000", 2024-09-20T22:13:53.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900917+0000", 2024-09-20T22:13:53.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900919+0000", 2024-09-20T22:13:53.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7400000000000001e-06 2024-09-20T22:13:53.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900924+0000", 2024-09-20T22:13:53.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6439999999999997e-06 2024-09-20T22:13:53.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900941+0000", 2024-09-20T22:13:53.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7244e-05 2024-09-20T22:13:53.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.900949+0000", 2024-09-20T22:13:53.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0829999999999998e-06 2024-09-20T22:13:53.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.901025+0000", 2024-09-20T22:13:53.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6643999999999993e-05 2024-09-20T22:13:53.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.700 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.901037+0000", 2024-09-20T22:13:53.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1824999999999999e-05 2024-09-20T22:13:53.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.700 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.817605+0000", 2024-09-20T22:13:53.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.91656772500000006 2024-09-20T22:13:53.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.700 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.817617+0000", 2024-09-20T22:13:53.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2320999999999999e-05 2024-09-20T22:13:53.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.067011+0000", 2024-09-20T22:13:53.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2493934259999999 2024-09-20T22:13:53.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:53.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.067109+0000", 2024-09-20T22:13:53.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8904e-05 2024-09-20T22:13:53.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:13:53.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.067424+0000", 2024-09-20T22:13:53.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00031460400000000001 2024-09-20T22:13:53.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:53.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.069092+0000", 2024-09-20T22:13:53.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001667836 2024-09-20T22:13:53.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.070743+0000", 2024-09-20T22:13:53.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001650763 2024-09-20T22:13:53.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.091949+0000", 2024-09-20T22:13:53.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021206434999999999 2024-09-20T22:13:53.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:53.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092012+0000", 2024-09-20T22:13:53.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2651999999999996e-05 2024-09-20T22:13:53.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092132+0000", 2024-09-20T22:13:53.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000120165 2024-09-20T22:13:53.704 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.704 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.704 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.704 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227675 7.2 7:45bb0f01:::benchmark_data_smithi060_78800_object94642:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-20T22:13:53.704 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:54.993128+0000", 2024-09-20T22:13:53.704 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 418.64968158300002, 2024-09-20T22:13:53.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.09956863, 2024-09-20T22:13:53.704 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.704 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.704 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:53.704 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.705 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:53.705 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:53.705 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:53.705 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:13:53.706 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227675 2024-09-20T22:13:53.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.706 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993128+0000", 2024-09-20T22:13:53.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993128+0000", 2024-09-20T22:13:53.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993130+0000", 2024-09-20T22:13:53.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5400000000000001e-06 2024-09-20T22:13:53.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993136+0000", 2024-09-20T22:13:53.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.669e-06 2024-09-20T22:13:53.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993153+0000", 2024-09-20T22:13:53.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6538999999999999e-05 2024-09-20T22:13:53.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993162+0000", 2024-09-20T22:13:53.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2529999999999993e-06 2024-09-20T22:13:53.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993239+0000", 2024-09-20T22:13:53.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6499000000000001e-05 2024-09-20T22:13:53.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.993253+0000", 2024-09-20T22:13:53.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4278e-05 2024-09-20T22:13:53.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.069151+0000", 2024-09-20T22:13:53.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0758977059999999 2024-09-20T22:13:53.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:53.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.069260+0000", 2024-09-20T22:13:53.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109175 2024-09-20T22:13:53.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:13:53.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.069491+0000", 2024-09-20T22:13:53.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000231716 2024-09-20T22:13:53.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:53.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.070761+0000", 2024-09-20T22:13:53.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012690850000000001 2024-09-20T22:13:53.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.071295+0000", 2024-09-20T22:13:53.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00053413799999999997 2024-09-20T22:13:53.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092555+0000", 2024-09-20T22:13:53.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021260233 2024-09-20T22:13:53.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:53.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092609+0000", 2024-09-20T22:13:53.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4147999999999997e-05 2024-09-20T22:13:53.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:56.092697+0000", 2024-09-20T22:13:53.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7651000000000005e-05 2024-09-20T22:13:53.712 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.712 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.712 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.712 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236464 7.2 7:41e7f20e:::benchmark_data_smithi060_78800_object103431:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:13:53.712 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.420278+0000", 2024-09-20T22:13:53.712 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 406.222531565, 2024-09-20T22:13:53.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99395603799999999, 2024-09-20T22:13:53.712 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.712 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.713 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:53.713 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.713 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:53.713 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:53.713 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236464 2024-09-20T22:13:53.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.713 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420278+0000", 2024-09-20T22:13:53.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420278+0000", 2024-09-20T22:13:53.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420280+0000", 2024-09-20T22:13:53.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0049999999999999e-06 2024-09-20T22:13:53.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420287+0000", 2024-09-20T22:13:53.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7429999999999996e-06 2024-09-20T22:13:53.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420305+0000", 2024-09-20T22:13:53.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.836e-05 2024-09-20T22:13:53.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.420314+0000", 2024-09-20T22:13:53.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7329999999999992e-06 2024-09-20T22:13:53.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.421421+0000", 2024-09-20T22:13:53.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001107455 2024-09-20T22:13:53.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.421429+0000", 2024-09-20T22:13:53.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5669999999999997e-06 2024-09-20T22:13:53.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.405308+0000", 2024-09-20T22:13:53.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98387922900000002 2024-09-20T22:13:53.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:53.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.405428+0000", 2024-09-20T22:13:53.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012023699999999999 2024-09-20T22:13:53.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:13:53.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.405861+0000", 2024-09-20T22:13:53.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00043245600000000001 2024-09-20T22:13:53.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:53.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414023+0000", 2024-09-20T22:13:53.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0081622280000000005 2024-09-20T22:13:53.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414092+0000", 2024-09-20T22:13:53.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9129999999999997e-05 2024-09-20T22:13:53.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414141+0000", 2024-09-20T22:13:53.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8426e-05 2024-09-20T22:13:53.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:53.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414196+0000", 2024-09-20T22:13:53.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5226999999999998e-05 2024-09-20T22:13:53.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414234+0000", 2024-09-20T22:13:53.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8241999999999997e-05 2024-09-20T22:13:53.718 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.718 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.718 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.718 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:236481 7.2 7:417f7fb8:::benchmark_data_smithi060_78800_object103448:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:13:53.719 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:07:07.436984+0000", 2024-09-20T22:13:53.719 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 406.20582534499999, 2024-09-20T22:13:53.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98983631900000002, 2024-09-20T22:13:53.719 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.719 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.719 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:53.719 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.719 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:13:53.719 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:13:53.719 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 236481 2024-09-20T22:13:53.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.719 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.436984+0000", 2024-09-20T22:13:53.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.436984+0000", 2024-09-20T22:13:53.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.436985+0000", 2024-09-20T22:13:53.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8000000000000004e-07 2024-09-20T22:13:53.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.436990+0000", 2024-09-20T22:13:53.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7770000000000002e-06 2024-09-20T22:13:53.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.437004+0000", 2024-09-20T22:13:53.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3572e-05 2024-09-20T22:13:53.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.437010+0000", 2024-09-20T22:13:53.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9469999999999997e-06 2024-09-20T22:13:53.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.439285+0000", 2024-09-20T22:13:53.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022742539999999999 2024-09-20T22:13:53.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:07.439294+0000", 2024-09-20T22:13:53.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0720000000000006e-06 2024-09-20T22:13:53.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.408421+0000", 2024-09-20T22:13:53.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.96912711200000001 2024-09-20T22:13:53.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:53.723 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.408517+0000", 2024-09-20T22:13:53.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.624e-05 2024-09-20T22:13:53.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.723 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:13:53.723 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.408884+0000", 2024-09-20T22:13:53.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00036694800000000002 2024-09-20T22:13:53.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.723 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.723 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.414277+0000", 2024-09-20T22:13:53.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0053932440000000002 2024-09-20T22:13:53.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.723 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.419733+0000", 2024-09-20T22:13:53.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0054556500000000003 2024-09-20T22:13:53.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:53.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426386+0000", 2024-09-20T22:13:53.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0066532680000000004 2024-09-20T22:13:53.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:53.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426769+0000", 2024-09-20T22:13:53.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000382347 2024-09-20T22:13:53.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:07:08.426821+0000", 2024-09-20T22:13:53.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2008000000000001e-05 2024-09-20T22:13:53.725 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.725 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.725 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.725 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106980 8.18 8:1c2b9c14:::benchmark_data_smithi060_111051_object106979:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:13:53.725 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.414151+0000", 2024-09-20T22:13:53.725 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 282.228659064, 2024-09-20T22:13:53.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.111436294, 2024-09-20T22:13:53.725 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.725 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.725 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:53.725 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.725 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:53.726 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:53.726 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106980 2024-09-20T22:13:53.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.726 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414151+0000", 2024-09-20T22:13:53.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414151+0000", 2024-09-20T22:13:53.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414152+0000", 2024-09-20T22:13:53.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5179999999999999e-06 2024-09-20T22:13:53.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414179+0000", 2024-09-20T22:13:53.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6480999999999999e-05 2024-09-20T22:13:53.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414191+0000", 2024-09-20T22:13:53.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2833999999999999e-05 2024-09-20T22:13:53.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414200+0000", 2024-09-20T22:13:53.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1040000000000007e-06 2024-09-20T22:13:53.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418058+0000", 2024-09-20T22:13:53.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038580139999999999 2024-09-20T22:13:53.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418080+0000", 2024-09-20T22:13:53.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2438000000000001e-05 2024-09-20T22:13:53.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.501675+0000", 2024-09-20T22:13:53.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0835947560000001 2024-09-20T22:13:53.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:53.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.501814+0000", 2024-09-20T22:13:53.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013951799999999999 2024-09-20T22:13:53.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4,5,7", 2024-09-20T22:13:53.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.502081+0000", 2024-09-20T22:13:53.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026697100000000002 2024-09-20T22:13:53.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:53.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.503435+0000", 2024-09-20T22:13:53.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001354156 2024-09-20T22:13:53.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.514128+0000", 2024-09-20T22:13:53.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010692432 2024-09-20T22:13:53.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525290+0000", 2024-09-20T22:13:53.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.011161901 2024-09-20T22:13:53.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525399+0000", 2024-09-20T22:13:53.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010921699999999999 2024-09-20T22:13:53.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:53.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525468+0000", 2024-09-20T22:13:53.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8726999999999994e-05 2024-09-20T22:13:53.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525587+0000", 2024-09-20T22:13:53.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119227 2024-09-20T22:13:53.732 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.732 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.732 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.732 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187847 8.0 8:0387ec14:::benchmark_data_smithi060_111051_object67848:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:13:53.732 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.626135+0000", 2024-09-20T22:13:53.732 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 204.01667487, 2024-09-20T22:13:53.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2145214800000002, 2024-09-20T22:13:53.732 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.732 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.732 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:13:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187847 2024-09-20T22:13:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626135+0000", 2024-09-20T22:13:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626135+0000", 2024-09-20T22:13:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626137+0000", 2024-09-20T22:13:53.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0389999999999999e-06 2024-09-20T22:13:53.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626144+0000", 2024-09-20T22:13:53.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9299999999999997e-06 2024-09-20T22:13:53.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626285+0000", 2024-09-20T22:13:53.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000141445 2024-09-20T22:13:53.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626296+0000", 2024-09-20T22:13:53.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0873000000000001e-05 2024-09-20T22:13:53.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626432+0000", 2024-09-20T22:13:53.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000135537 2024-09-20T22:13:53.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:53.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626543+0000", 2024-09-20T22:13:53.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011132399999999999 2024-09-20T22:13:53.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,5", 2024-09-20T22:13:53.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627183+0000", 2024-09-20T22:13:53.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00064020400000000001 2024-09-20T22:13:53.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.656943+0000", 2024-09-20T22:13:53.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029759622999999999 2024-09-20T22:13:53.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.656954+0000", 2024-09-20T22:13:53.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0913999999999999e-05 2024-09-20T22:13:53.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651133+0000", 2024-09-20T22:13:53.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99417974399999998 2024-09-20T22:13:53.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651152+0000", 2024-09-20T22:13:53.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8108999999999999e-05 2024-09-20T22:13:53.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.840586+0000", 2024-09-20T22:13:53.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1894341669999999 2024-09-20T22:13:53.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.840656+0000", 2024-09-20T22:13:53.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0570999999999999e-05 2024-09-20T22:13:53.738 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.738 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.738 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.739 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187874 8.0 8:032cd1ec:::benchmark_data_smithi060_111051_object67875:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:13:53.739 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.678008+0000", 2024-09-20T22:13:53.739 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 203.96480150599999, 2024-09-20T22:13:53.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1758792680000001, 2024-09-20T22:13:53.739 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.739 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.739 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:53.739 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.739 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:53.739 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:53.739 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187874 2024-09-20T22:13:53.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.740 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678008+0000", 2024-09-20T22:13:53.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678008+0000", 2024-09-20T22:13:53.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678010+0000", 2024-09-20T22:13:53.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7430000000000001e-06 2024-09-20T22:13:53.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678016+0000", 2024-09-20T22:13:53.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4339999999999997e-06 2024-09-20T22:13:53.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678030+0000", 2024-09-20T22:13:53.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3971999999999999e-05 2024-09-20T22:13:53.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678038+0000", 2024-09-20T22:13:53.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5100000000000001e-06 2024-09-20T22:13:53.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.742 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.742 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678109+0000", 2024-09-20T22:13:53.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0708000000000001e-05 2024-09-20T22:13:53.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.742 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.742 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678119+0000", 2024-09-20T22:13:53.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0519e-05 2024-09-20T22:13:53.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.742 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651185+0000", 2024-09-20T22:13:53.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97306574199999996 2024-09-20T22:13:53.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651193+0000", 2024-09-20T22:13:53.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2609999999999993e-06 2024-09-20T22:13:53.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.840972+0000", 2024-09-20T22:13:53.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.189778644 2024-09-20T22:13:53.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:53.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.841244+0000", 2024-09-20T22:13:53.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00027224500000000001 2024-09-20T22:13:53.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4,5", 2024-09-20T22:13:53.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.841873+0000", 2024-09-20T22:13:53.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000629124 2024-09-20T22:13:53.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.852639+0000", 2024-09-20T22:13:53.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010765925000000001 2024-09-20T22:13:53.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.745 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.852704+0000", 2024-09-20T22:13:53.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5011000000000005e-05 2024-09-20T22:13:53.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.745 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:53.745 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.852729+0000", 2024-09-20T22:13:53.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4782000000000001e-05 2024-09-20T22:13:53.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.745 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.853614+0000", 2024-09-20T22:13:53.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00088560699999999995 2024-09-20T22:13:53.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:53.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.853671+0000", 2024-09-20T22:13:53.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6382999999999999e-05 2024-09-20T22:13:53.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.853887+0000", 2024-09-20T22:13:53.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000216658 2024-09-20T22:13:53.747 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.747 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.747 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.747 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187894 8.0 8:02b3e725:::benchmark_data_smithi060_111051_object67895:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:13:53.748 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.701140+0000", 2024-09-20T22:13:53.748 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 203.94166975799999, 2024-09-20T22:13:53.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1634560220000001, 2024-09-20T22:13:53.748 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.748 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.748 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:53.748 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.748 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:53.748 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:53.748 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187894 2024-09-20T22:13:53.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.748 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701140+0000", 2024-09-20T22:13:53.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701140+0000", 2024-09-20T22:13:53.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701141+0000", 2024-09-20T22:13:53.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2950000000000001e-06 2024-09-20T22:13:53.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701147+0000", 2024-09-20T22:13:53.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3890000000000004e-06 2024-09-20T22:13:53.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701168+0000", 2024-09-20T22:13:53.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1725999999999998e-05 2024-09-20T22:13:53.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701179+0000", 2024-09-20T22:13:53.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0393e-05 2024-09-20T22:13:53.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701245+0000", 2024-09-20T22:13:53.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6519999999999993e-05 2024-09-20T22:13:53.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701256+0000", 2024-09-20T22:13:53.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0722e-05 2024-09-20T22:13:53.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651217+0000", 2024-09-20T22:13:53.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94996075899999999 2024-09-20T22:13:53.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.654608+0000", 2024-09-20T22:13:53.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0033909819999999999 2024-09-20T22:13:53.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.849926+0000", 2024-09-20T22:13:53.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.195318353 2024-09-20T22:13:53.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:53.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.850057+0000", 2024-09-20T22:13:53.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013062999999999999 2024-09-20T22:13:53.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4,5", 2024-09-20T22:13:53.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.850462+0000", 2024-09-20T22:13:53.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00040517 2024-09-20T22:13:53.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:53.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.852996+0000", 2024-09-20T22:13:53.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025345179999999999 2024-09-20T22:13:53.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.854077+0000", 2024-09-20T22:13:53.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001080262 2024-09-20T22:13:53.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.864365+0000", 2024-09-20T22:13:53.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010287907000000001 2024-09-20T22:13:53.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.864413+0000", 2024-09-20T22:13:53.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8690999999999998e-05 2024-09-20T22:13:53.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:53.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.864473+0000", 2024-09-20T22:13:53.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9487000000000002e-05 2024-09-20T22:13:53.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.864596+0000", 2024-09-20T22:13:53.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000123218 2024-09-20T22:13:53.755 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.755 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.755 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.755 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214320 8.8 8:125eda0b:::benchmark_data_smithi060_111051_object94321:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:13:53.755 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:13:53.756 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 176.14642788500001, 2024-09-20T22:13:53.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0054397049999999, 2024-09-20T22:13:53.756 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.756 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.756 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:53.756 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.756 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:53.756 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:53.756 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214320 2024-09-20T22:13:53.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.756 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.756 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.756 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:13:53.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:13:53.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:13:53.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0999999999999999e-07 2024-09-20T22:13:53.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496384+0000", 2024-09-20T22:13:53.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1859999999999999e-06 2024-09-20T22:13:53.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496390+0000", 2024-09-20T22:13:53.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9920000000000004e-06 2024-09-20T22:13:53.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496395+0000", 2024-09-20T22:13:53.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.741e-06 2024-09-20T22:13:53.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496429+0000", 2024-09-20T22:13:53.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3736999999999999e-05 2024-09-20T22:13:53.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496436+0000", 2024-09-20T22:13:53.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2529999999999998e-06 2024-09-20T22:13:53.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.497957+0000", 2024-09-20T22:13:53.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.001521109 2024-09-20T22:13:53.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:53.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.498097+0000", 2024-09-20T22:13:53.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000139562 2024-09-20T22:13:53.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:13:53.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.498422+0000", 2024-09-20T22:13:53.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00032548199999999998 2024-09-20T22:13:53.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501661+0000", 2024-09-20T22:13:53.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0032386580000000002 2024-09-20T22:13:53.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:53.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501669+0000", 2024-09-20T22:13:53.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2500000000000006e-06 2024-09-20T22:13:53.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501709+0000", 2024-09-20T22:13:53.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9304999999999997e-05 2024-09-20T22:13:53.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:53.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501777+0000", 2024-09-20T22:13:53.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8285000000000001e-05 2024-09-20T22:13:53.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501821+0000", 2024-09-20T22:13:53.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4635000000000002e-05 2024-09-20T22:13:53.762 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.762 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.762 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.763 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214327 8.8 8:149a99e2:::benchmark_data_smithi060_111051_object94328:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:13:53.763 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.502392+0000", 2024-09-20T22:13:53.763 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 176.14041746300001, 2024-09-20T22:13:53.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.001539808, 2024-09-20T22:13:53.763 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.763 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.763 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:53.763 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.763 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:53.763 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:53.763 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214327 2024-09-20T22:13:53.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.764 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502392+0000", 2024-09-20T22:13:53.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502392+0000", 2024-09-20T22:13:53.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502393+0000", 2024-09-20T22:13:53.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2440000000000001e-06 2024-09-20T22:13:53.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502398+0000", 2024-09-20T22:13:53.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1749999999999998e-06 2024-09-20T22:13:53.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502406+0000", 2024-09-20T22:13:53.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4999999999999999e-06 2024-09-20T22:13:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502411+0000", 2024-09-20T22:13:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7119999999999999e-06 2024-09-20T22:13:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502465+0000", 2024-09-20T22:13:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4206000000000002e-05 2024-09-20T22:13:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502470+0000", 2024-09-20T22:13:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8069999999999999e-06 2024-09-20T22:13:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.500319+0000", 2024-09-20T22:13:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99784936499999999 2024-09-20T22:13:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.500435+0000", 2024-09-20T22:13:53.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115439 2024-09-20T22:13:53.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:13:53.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.500670+0000", 2024-09-20T22:13:53.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023555700000000001 2024-09-20T22:13:53.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503764+0000", 2024-09-20T22:13:53.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0030938839999999999 2024-09-20T22:13:53.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503827+0000", 2024-09-20T22:13:53.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2669999999999995e-05 2024-09-20T22:13:53.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:53.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503839+0000", 2024-09-20T22:13:53.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2301e-05 2024-09-20T22:13:53.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:53.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503887+0000", 2024-09-20T22:13:53.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8319999999999998e-05 2024-09-20T22:13:53.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503932+0000", 2024-09-20T22:13:53.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4628e-05 2024-09-20T22:13:53.770 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.770 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.770 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.770 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214339 8.8 8:166843c0:::benchmark_data_smithi060_111051_object94340:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:13:53.770 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.507919+0000", 2024-09-20T22:13:53.770 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 176.134890367, 2024-09-20T22:13:53.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99621294400000004, 2024-09-20T22:13:53.770 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.771 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.771 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:53.771 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.771 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:53.771 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:53.771 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214339 2024-09-20T22:13:53.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.771 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507919+0000", 2024-09-20T22:13:53.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507919+0000", 2024-09-20T22:13:53.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507921+0000", 2024-09-20T22:13:53.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.316e-06 2024-09-20T22:13:53.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507926+0000", 2024-09-20T22:13:53.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0350000000000001e-06 2024-09-20T22:13:53.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507937+0000", 2024-09-20T22:13:53.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1379999999999999e-05 2024-09-20T22:13:53.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507945+0000", 2024-09-20T22:13:53.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5780000000000001e-06 2024-09-20T22:13:53.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.508028+0000", 2024-09-20T22:13:53.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3641999999999996e-05 2024-09-20T22:13:53.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.508036+0000", 2024-09-20T22:13:53.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1780000000000003e-06 2024-09-20T22:13:53.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501861+0000", 2024-09-20T22:13:53.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99382438200000001 2024-09-20T22:13:53.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:53.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501935+0000", 2024-09-20T22:13:53.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4151000000000002e-05 2024-09-20T22:13:53.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:13:53.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.502143+0000", 2024-09-20T22:13:53.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020823700000000001 2024-09-20T22:13:53.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503984+0000", 2024-09-20T22:13:53.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001840602 2024-09-20T22:13:53.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504043+0000", 2024-09-20T22:13:53.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9061000000000003e-05 2024-09-20T22:13:53.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:53.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504055+0000", 2024-09-20T22:13:53.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1826e-05 2024-09-20T22:13:53.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:53.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504099+0000", 2024-09-20T22:13:53.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4801e-05 2024-09-20T22:13:53.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504132+0000", 2024-09-20T22:13:53.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2755000000000001e-05 2024-09-20T22:13:53.777 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.777 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.777 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.777 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214363 8.8 8:14aba4d5:::benchmark_data_smithi060_111051_object94364:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:13:53.777 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.530169+0000", 2024-09-20T22:13:53.778 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 176.11264043200001, 2024-09-20T22:13:53.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98026972499999998, 2024-09-20T22:13:53.778 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.778 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.778 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:53.778 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.778 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:53.778 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:53.778 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214363 2024-09-20T22:13:53.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.778 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530169+0000", 2024-09-20T22:13:53.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530169+0000", 2024-09-20T22:13:53.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530171+0000", 2024-09-20T22:13:53.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4300000000000001e-06 2024-09-20T22:13:53.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530177+0000", 2024-09-20T22:13:53.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8499999999999999e-06 2024-09-20T22:13:53.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530188+0000", 2024-09-20T22:13:53.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1669e-05 2024-09-20T22:13:53.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530196+0000", 2024-09-20T22:13:53.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0760000000000003e-06 2024-09-20T22:13:53.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530270+0000", 2024-09-20T22:13:53.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3709999999999997e-05 2024-09-20T22:13:53.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530278+0000", 2024-09-20T22:13:53.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5320000000000006e-06 2024-09-20T22:13:53.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504165+0000", 2024-09-20T22:13:53.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97388663600000003 2024-09-20T22:13:53.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:53.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504263+0000", 2024-09-20T22:13:53.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7412999999999997e-05 2024-09-20T22:13:53.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:13:53.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504540+0000", 2024-09-20T22:13:53.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00027761399999999997 2024-09-20T22:13:53.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:53.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510219+0000", 2024-09-20T22:13:53.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005678476 2024-09-20T22:13:53.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510260+0000", 2024-09-20T22:13:53.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1452000000000002e-05 2024-09-20T22:13:53.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510339+0000", 2024-09-20T22:13:53.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8669000000000002e-05 2024-09-20T22:13:53.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:53.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510393+0000", 2024-09-20T22:13:53.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4160999999999999e-05 2024-09-20T22:13:53.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510439+0000", 2024-09-20T22:13:53.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6037e-05 2024-09-20T22:13:53.784 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.784 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.784 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.784 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214367 8.8 8:15e42233:::benchmark_data_smithi060_111051_object94368:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:13:53.784 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.530758+0000", 2024-09-20T22:13:53.784 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 176.112051248, 2024-09-20T22:13:53.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98471673000000004, 2024-09-20T22:13:53.784 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:13:53.784 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:13:53.784 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:13:53.784 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:13:53.785 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:13:53.785 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:13:53.785 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214367 2024-09-20T22:13:53.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.785 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:13:53.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:13:53.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530758+0000", 2024-09-20T22:13:53.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.785 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.785 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:13:53.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530758+0000", 2024-09-20T22:13:53.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:13:53.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:13:53.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530759+0000", 2024-09-20T22:13:53.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-07 2024-09-20T22:13:53.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:13:53.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530764+0000", 2024-09-20T22:13:53.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5390000000000001e-06 2024-09-20T22:13:53.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:13:53.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530773+0000", 2024-09-20T22:13:53.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.986e-06 2024-09-20T22:13:53.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:13:53.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530780+0000", 2024-09-20T22:13:53.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8689999999999998e-06 2024-09-20T22:13:53.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530827+0000", 2024-09-20T22:13:53.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7314000000000001e-05 2024-09-20T22:13:53.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:13:53.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530832+0000", 2024-09-20T22:13:53.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.758e-06 2024-09-20T22:13:53.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:13:53.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.505913+0000", 2024-09-20T22:13:53.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97508115200000001 2024-09-20T22:13:53.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:13:53.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.506026+0000", 2024-09-20T22:13:53.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011298000000000001 2024-09-20T22:13:53.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:13:53.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.506308+0000", 2024-09-20T22:13:53.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000282477 2024-09-20T22:13:53.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:13:53.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510274+0000", 2024-09-20T22:13:53.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0039657160000000002 2024-09-20T22:13:53.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510478+0000", 2024-09-20T22:13:53.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204002 2024-09-20T22:13:53.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:13:53.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.515367+0000", 2024-09-20T22:13:53.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0048889099999999998 2024-09-20T22:13:53.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:13:53.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.515428+0000", 2024-09-20T22:13:53.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0826999999999999e-05 2024-09-20T22:13:53.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:13:53.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:13:53.791 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:13:53.791 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.515475+0000", 2024-09-20T22:13:53.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7388000000000001e-05 2024-09-20T22:13:53.791 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.791 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.791 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.791 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:13:53.791 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:13:53.791 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:13:53.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:53.792 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:53.792 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:53.880 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[29694]: 6.14 scrub starts 2024-09-20T22:13:53.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[29694]: 6.14 scrub ok 2024-09-20T22:13:53.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[29694]: 3.1b scrub starts 2024-09-20T22:13:53.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[29694]: 3.1b scrub ok 2024-09-20T22:13:53.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[29694]: 9.7 scrub starts 2024-09-20T22:13:53.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[29694]: 5.12 scrub starts 2024-09-20T22:13:53.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[29694]: 5.12 scrub ok 2024-09-20T22:13:53.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[29694]: 6.1c scrub starts 2024-09-20T22:13:53.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[29694]: 6.1c scrub ok 2024-09-20T22:13:53.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2022091354' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:13:53.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:13:53.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[29694]: 5.1f scrub starts 2024-09-20T22:13:53.881 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[29694]: 5.1f scrub ok 2024-09-20T22:13:53.882 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[40290]: 6.14 scrub starts 2024-09-20T22:13:53.882 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[40290]: 6.14 scrub ok 2024-09-20T22:13:53.882 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[40290]: 3.1b scrub starts 2024-09-20T22:13:53.882 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[40290]: 3.1b scrub ok 2024-09-20T22:13:53.882 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[40290]: 9.7 scrub starts 2024-09-20T22:13:53.882 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[40290]: 5.12 scrub starts 2024-09-20T22:13:53.882 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[40290]: 5.12 scrub ok 2024-09-20T22:13:53.882 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[40290]: 6.1c scrub starts 2024-09-20T22:13:53.882 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[40290]: 6.1c scrub ok 2024-09-20T22:13:53.882 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2022091354' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:13:53.882 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:13:53.883 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[40290]: 5.1f scrub starts 2024-09-20T22:13:53.883 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:53 smithi060 ceph-mon[40290]: 5.1f scrub ok 2024-09-20T22:13:53.893 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:53.923 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:53.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:53.924 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:54.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:53 smithi067 ceph-mon[33511]: 6.14 scrub starts 2024-09-20T22:13:54.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:53 smithi067 ceph-mon[33511]: 6.14 scrub ok 2024-09-20T22:13:54.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:53 smithi067 ceph-mon[33511]: 3.1b scrub starts 2024-09-20T22:13:54.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:53 smithi067 ceph-mon[33511]: 3.1b scrub ok 2024-09-20T22:13:54.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:53 smithi067 ceph-mon[33511]: 9.7 scrub starts 2024-09-20T22:13:54.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:53 smithi067 ceph-mon[33511]: 5.12 scrub starts 2024-09-20T22:13:54.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:53 smithi067 ceph-mon[33511]: 5.12 scrub ok 2024-09-20T22:13:54.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:53 smithi067 ceph-mon[33511]: 6.1c scrub starts 2024-09-20T22:13:54.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:53 smithi067 ceph-mon[33511]: 6.1c scrub ok 2024-09-20T22:13:54.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:53 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2022091354' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:13:54.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:53 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:13:54.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:53 smithi067 ceph-mon[33511]: 5.1f scrub starts 2024-09-20T22:13:54.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:53 smithi067 ceph-mon[33511]: 5.1f scrub ok 2024-09-20T22:13:54.025 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:54.061 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:54.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:54.062 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:54.162 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:54.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:54.196 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:54.196 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:54.297 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:54.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:54.333 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:54.333 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:54.434 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:54.466 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:54.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:54.467 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:54.568 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:54.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:54.607 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:54.607 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:54.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:54.748 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:54.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:54.749 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:54.850 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:54.883 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:54.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:54.884 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:54.984 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:55.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:54 smithi067 ceph-mon[33511]: 3.8 scrub starts 2024-09-20T22:13:55.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:54 smithi067 ceph-mon[33511]: 3.8 scrub ok 2024-09-20T22:13:55.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:54 smithi067 ceph-mon[33511]: 5.8 scrub starts 2024-09-20T22:13:55.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:54 smithi067 ceph-mon[33511]: 5.8 scrub ok 2024-09-20T22:13:55.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:54 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:13:55.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:54 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:13:55.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:54 smithi067 ceph-mon[33511]: osdmap e346: 8 total, 8 up, 8 in 2024-09-20T22:13:55.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:54 smithi067 ceph-mon[33511]: pgmap v593: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 4.4 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 932 op/s 2024-09-20T22:13:55.023 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:55.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:55.024 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:55.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:54 smithi060 ceph-mon[29694]: 3.8 scrub starts 2024-09-20T22:13:55.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:54 smithi060 ceph-mon[29694]: 3.8 scrub ok 2024-09-20T22:13:55.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:54 smithi060 ceph-mon[29694]: 5.8 scrub starts 2024-09-20T22:13:55.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:54 smithi060 ceph-mon[29694]: 5.8 scrub ok 2024-09-20T22:13:55.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:54 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:13:55.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:54 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:13:55.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:54 smithi060 ceph-mon[29694]: osdmap e346: 8 total, 8 up, 8 in 2024-09-20T22:13:55.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:54 smithi060 ceph-mon[29694]: pgmap v593: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 4.4 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 932 op/s 2024-09-20T22:13:55.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:54 smithi060 ceph-mon[40290]: 3.8 scrub starts 2024-09-20T22:13:55.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:54 smithi060 ceph-mon[40290]: 3.8 scrub ok 2024-09-20T22:13:55.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:54 smithi060 ceph-mon[40290]: 5.8 scrub starts 2024-09-20T22:13:55.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:54 smithi060 ceph-mon[40290]: 5.8 scrub ok 2024-09-20T22:13:55.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:54 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:13:55.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:54 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:13:55.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:54 smithi060 ceph-mon[40290]: osdmap e346: 8 total, 8 up, 8 in 2024-09-20T22:13:55.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:54 smithi060 ceph-mon[40290]: pgmap v593: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 4.4 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 932 op/s 2024-09-20T22:13:55.125 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:55.150 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:55.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:55.151 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:55.251 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:55.287 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:55.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:55.288 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:55.388 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:55.421 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:55.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:55.422 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:55.523 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:55.535 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:13:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:13:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:13:55.546 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:13:55.550 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:55.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:55.551 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:55.652 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:55.682 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:55.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:55.683 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:55.784 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:55.824 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:55.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:55.825 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:55.926 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:55.967 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:55.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:55.968 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:55 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:13:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:55 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:13:56.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:55 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:13:56.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:55 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:13:56.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:55 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:13:56.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:55 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:13:56.070 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:56.097 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:56.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:56.098 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:56.199 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:56.230 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:56.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:56.231 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:56.332 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:56.359 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:56.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:56.360 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:56.461 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:56.493 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:56.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:56.495 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:56.595 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:56.626 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:56.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:56.627 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:56.728 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:56.758 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:56.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:56.759 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:56.860 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:56.890 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:56.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:56.892 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:56.907 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T22:13:56.972 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:56 smithi067 ceph-mon[33511]: osdmap e347: 8 total, 8 up, 8 in 2024-09-20T22:13:56.972 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:56 smithi067 ceph-mon[33511]: pgmap v595: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 976 op/s 2024-09-20T22:13:56.972 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:13:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=cleanup t=2024-09-20T22:13:56.803376706Z level=info msg="Completed cleanup jobs" duration=1.334455ms 2024-09-20T22:13:56.972 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:13:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=sqlstore.transactions t=2024-09-20T22:13:56.81286543Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-20T22:13:56.976 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:56 smithi060 ceph-mon[40290]: osdmap e347: 8 total, 8 up, 8 in 2024-09-20T22:13:56.976 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:56 smithi060 ceph-mon[40290]: pgmap v595: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 976 op/s 2024-09-20T22:13:56.977 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:56 smithi060 ceph-mon[29694]: osdmap e347: 8 total, 8 up, 8 in 2024-09-20T22:13:56.977 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:56 smithi060 ceph-mon[29694]: pgmap v595: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 4.3 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 976 op/s 2024-09-20T22:13:56.992 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:57.044 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:57.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:57.045 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:57.146 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:57.181 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:57.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:57.183 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:57.283 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:57.317 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:57.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:57.318 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:57.351 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:13:57.419 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:57.447 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:57.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:57.448 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:57.549 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:57.582 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:57.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:57.583 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:57.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:57.722 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:57.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:57.723 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:57.823 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:13:57.858 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:57.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:57.859 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:13:57.960 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:58.002 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:58.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:58.003 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:58.104 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:58.136 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:58.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:58.137 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:58.238 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:58.260 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:58.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:58.261 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:58.347 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:13:58.347 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:13:58.347 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:13:58.347 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:13:58.347 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:13:58.347 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:13:58.347 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:13:58.347 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:13:58.347 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:13:58.347 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:13:58.347 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:13:58.347 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:13:58.351 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:13:58.351 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:13:58.361 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:58.384 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:13:58.385 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:13:58.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:58.392 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:58.392 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:58.444 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:58 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/517531259' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:58.493 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:58.526 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:58.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:58.527 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:58.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:58 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/517531259' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:58.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:58 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/517531259' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:13:58.629 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:13:58.665 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:58.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:58.666 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:13:58.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:58.789 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:58.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:58.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:58.891 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:58.923 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:58.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:58.924 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:59.025 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:13:59.062 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:59.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:59.063 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:13:59.164 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:13:59.197 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:59.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:59.198 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:13:59.246 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:13:59 smithi060 ceph-mon[29694]: pgmap v596: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 4.2 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.06k op/s 2024-09-20T22:13:59.299 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:59.337 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:59.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:59.338 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:59.427 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:13:59.427 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:13:59.438 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:13:59.475 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:59.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:59.477 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:13:59.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:13:59 smithi067 ceph-mon[33511]: pgmap v596: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 4.2 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.06k op/s 2024-09-20T22:13:59.535 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:13:59 smithi060 ceph-mon[40290]: pgmap v596: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 4.2 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.06k op/s 2024-09-20T22:13:59.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:13:59.611 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:59.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:59.612 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:13:59.671 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:13:59.713 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:13:59.740 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:13:59.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:59.741 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:13:59.842 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:13:59.877 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:13:59.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:13:59.878 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:13:59.979 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:00.012 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:00.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:00.012 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:00.113 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:00.140 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:00.166 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:00.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:00.167 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:00.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:00.304 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:00.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:00.305 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:00.406 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:00.438 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:00.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:00.439 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:00.466 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T22:14:00.541 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:00.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:00.587 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:00.587 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:00.688 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:00.727 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:00.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:00.728 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:00.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:00.862 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:00.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:00.863 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:00.885 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:14:00.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:01.003 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:01.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:01.004 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:01.044 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:14:01.105 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:01.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:01.135 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:01.136 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:01.236 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:01.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:01 smithi067 ceph-mon[33511]: pgmap v597: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 4.1 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 762 op/s 2024-09-20T22:14:01.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:01 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2449770447' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:01.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:01 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:01.265 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:14:01.268 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:01.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:01.269 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:01.370 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:01.406 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:01.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:01.407 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:01.508 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:01.509 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:14:01.509 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:14:01.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:01 smithi060 ceph-mon[29694]: pgmap v597: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 4.1 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 762 op/s 2024-09-20T22:14:01.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:01 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2449770447' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:01.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:01 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:01.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:01 smithi060 ceph-mon[40290]: pgmap v597: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 4.1 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 762 op/s 2024-09-20T22:14:01.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:01 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2449770447' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:01.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:01 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:01.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:01.548 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:01.548 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:01.649 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:01.684 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:01.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:01.685 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:01.785 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:01.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:01.822 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:01.822 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:01.923 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:01.962 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:01.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:01.963 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:02.063 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:02.101 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:02.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:02.102 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:02.203 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:02.239 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:02.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:02.240 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:02.341 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:02.376 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:02.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:02.377 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:02.478 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:02.489 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T22:14:02.513 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:14:02 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:14:02.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:02 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:14:02.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:02 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:14:02.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:02 smithi067 ceph-mon[33511]: osdmap e348: 8 total, 8 up, 8 in 2024-09-20T22:14:02.516 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:02.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:02.517 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:02.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:02 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:14:02.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:02 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:14:02.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:02 smithi060 ceph-mon[29694]: osdmap e348: 8 total, 8 up, 8 in 2024-09-20T22:14:02.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:02 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:14:02.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:02 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:14:02.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:02 smithi060 ceph-mon[40290]: osdmap e348: 8 total, 8 up, 8 in 2024-09-20T22:14:02.618 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:02.646 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:02.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:02.647 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:02.748 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:02.778 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:02.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:02.779 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:02.872 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:14:02.880 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:02.917 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:02.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:02.918 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:03.019 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:03.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:03.098 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:03.098 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:03.199 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:03.236 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:03.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:03.237 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:03.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:03 smithi067 ceph-mon[33511]: pgmap v599: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.08k op/s 2024-09-20T22:14:03.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:03 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:14:03.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:03 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:14:03.338 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:03.377 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:03.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:03.378 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:03.478 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:03.520 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:03 smithi060 ceph-mon[29694]: pgmap v599: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.08k op/s 2024-09-20T22:14:03.520 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:03 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:14:03.520 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:03 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:14:03.521 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:03 smithi060 ceph-mon[40290]: pgmap v599: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.08k op/s 2024-09-20T22:14:03.521 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:03 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:14:03.521 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:03 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:14:03.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:03.527 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:03.528 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:03.556 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:14:03.556 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:14:03.628 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:03.639 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:03.674 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:03.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:03.675 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:03.776 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:03.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:03.836 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:03.837 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:03.937 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:03.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:03.975 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:03.975 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:04.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:04.125 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:04.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:04.126 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:04.227 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:04.268 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:04.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:04.269 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:04.270 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:14:04.270 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:14:04.311 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:14:04.311 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:14:04.345 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:14:04.345 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:14:04.345 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.346 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.346 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.346 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.346 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:14:04.346 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.346 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.346 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.346 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.346 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:14:04.346 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:14:04.346 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:14:04.346 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:14:04.346 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2302993797' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:04.347 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.347 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.347 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.347 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.347 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:14:04.347 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.347 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.347 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.347 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.347 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:14:04.347 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:14:04.347 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:14:04.347 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:14:04.347 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:04 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2302993797' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:04.370 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:04.389 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:14:04.389 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:14:04.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:04.409 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:04.409 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:04.423 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:14:04.423 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:14:04.459 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:14:04.459 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:14:04.494 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:14:04.495 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:14:04.510 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:04.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:04 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:04 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:04 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:04 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:04 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:14:04.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:04 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:04 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:04 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:04 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-20T22:14:04.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:04 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:14:04.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:04 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:14:04.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:04 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:14:04.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:04 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:14:04.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:04 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2302993797' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:04.535 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:14:04.535 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:14:04.543 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:04.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:04.544 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:04.645 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:04.652 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:14:04.652 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:14:04.652 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2024-09-20T22:14:04.652 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:14:04.684 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:04.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:04.685 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:04.756 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T22:14:04.786 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:04.824 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:04.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:04.825 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:04.926 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:04.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:04.969 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:04.969 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:05.070 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:05.122 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:05.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:05.123 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:05.172 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:14:05.224 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:05.268 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:05.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:05.270 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:05.343 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:14:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:14:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:14:05.343 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:05 smithi060 ceph-mon[29694]: Adjusting osd_memory_target on smithi067 to 3248M 2024-09-20T22:14:05.344 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:05 smithi060 ceph-mon[29694]: Adjusting osd_memory_target on smithi060 to 3249M 2024-09-20T22:14:05.344 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:05 smithi060 ceph-mon[29694]: pgmap v600: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 3.8 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-20T22:14:05.347 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:05 smithi060 ceph-mon[40290]: Adjusting osd_memory_target on smithi067 to 3248M 2024-09-20T22:14:05.347 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:05 smithi060 ceph-mon[40290]: Adjusting osd_memory_target on smithi060 to 3249M 2024-09-20T22:14:05.347 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:05 smithi060 ceph-mon[40290]: pgmap v600: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 3.8 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-20T22:14:05.370 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:05.417 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:05.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:05.419 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:05.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:05 smithi067 ceph-mon[33511]: Adjusting osd_memory_target on smithi067 to 3248M 2024-09-20T22:14:05.516 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:05 smithi067 ceph-mon[33511]: Adjusting osd_memory_target on smithi060 to 3249M 2024-09-20T22:14:05.516 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:05 smithi067 ceph-mon[33511]: pgmap v600: 158 pgs: 1 active+clean+scrubbing+deep, 157 active+clean; 3.8 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-20T22:14:05.520 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:05.565 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:05.570 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:14:05.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:05.584 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:05.584 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:05.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:05.733 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:05.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:05.734 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:05.807 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:14:05.807 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:14:05.807 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.807 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.15594.0:163591 9.b e348/328 9:d841d7fb:::benchmark_data_smithi060_140132_object52783:head v 348'10058, mlcod=348'10054)", 2024-09-20T22:14:05.807 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:14:05.796739+0000", 2024-09-20T22:14:05.807 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0062628559999999998, 2024-09-20T22:14:05.807 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.006283159, 2024-09-20T22:14:05.807 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:05.807 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:05.808 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:14:05.808 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:05.808 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.808 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:05.808 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.796739+0000", 2024-09-20T22:14:05.808 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:05.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.808 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.808 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:05.808 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.796739+0000", 2024-09-20T22:14:05.808 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:05.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.808 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.808 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:05.808 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.796739+0000", 2024-09-20T22:14:05.809 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7599999999999997e-07 2024-09-20T22:14:05.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.809 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.809 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:05.809 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.796745+0000", 2024-09-20T22:14:05.809 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6690000000000003e-06 2024-09-20T22:14:05.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.809 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.809 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:05.809 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.796759+0000", 2024-09-20T22:14:05.809 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3949e-05 2024-09-20T22:14:05.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.809 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.809 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:05.809 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.796765+0000", 2024-09-20T22:14:05.809 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6390000000000003e-06 2024-09-20T22:14:05.810 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.810 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.810 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:05.810 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.796826+0000", 2024-09-20T22:14:05.810 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0884000000000002e-05 2024-09-20T22:14:05.810 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.810 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.810 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:05.810 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.796845+0000", 2024-09-20T22:14:05.810 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8794999999999999e-05 2024-09-20T22:14:05.810 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:05.810 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:05.810 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:05.810 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.811 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.811 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:163601 9.7 9:e5fedf5c:::benchmark_data_smithi060_140132_object52793:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-20T22:14:05.811 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:14:05.796755+0000", 2024-09-20T22:14:05.811 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0062465259999999996, 2024-09-20T22:14:05.811 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0063092549999999997, 2024-09-20T22:14:05.811 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:05.811 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:05.811 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:14:05.811 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:05.811 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:14:05.811 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:14:05.811 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163601 2024-09-20T22:14:05.811 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.811 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:05.812 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.812 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:05.812 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.796755+0000", 2024-09-20T22:14:05.812 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:05.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.812 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.812 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:05.812 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.796755+0000", 2024-09-20T22:14:05.812 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:05.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.812 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.812 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:05.812 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.796756+0000", 2024-09-20T22:14:05.812 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.356e-06 2024-09-20T22:14:05.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.812 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.813 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:05.813 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.796761+0000", 2024-09-20T22:14:05.813 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9269999999999996e-06 2024-09-20T22:14:05.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.813 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.813 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:05.813 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.796817+0000", 2024-09-20T22:14:05.813 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6199000000000002e-05 2024-09-20T22:14:05.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.813 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.813 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:05.813 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.796903+0000", 2024-09-20T22:14:05.813 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5358999999999994e-05 2024-09-20T22:14:05.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.814 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.814 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:05.814 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.797207+0000", 2024-09-20T22:14:05.814 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00030475800000000003 2024-09-20T22:14:05.814 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.814 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.814 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:05.814 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.797312+0000", 2024-09-20T22:14:05.814 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000104139 2024-09-20T22:14:05.814 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.814 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.814 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:14:05.814 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.797427+0000", 2024-09-20T22:14:05.815 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000115114 2024-09-20T22:14:05.815 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:05.815 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:05.815 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:05.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.815 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.815 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.15594.0:163597 9.5 e348/324 9:a3cd9b58:::benchmark_data_smithi060_140132_object52789:head v 348'10073, mlcod=348'10071)", 2024-09-20T22:14:05.815 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:14:05.797086+0000", 2024-09-20T22:14:05.815 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0059153579999999999, 2024-09-20T22:14:05.815 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0060208320000000003, 2024-09-20T22:14:05.815 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:05.815 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:05.815 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:14:05.815 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:05.815 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.816 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:05.816 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.797086+0000", 2024-09-20T22:14:05.816 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:05.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.816 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:05.816 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.797086+0000", 2024-09-20T22:14:05.816 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:05.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.816 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:05.816 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.797087+0000", 2024-09-20T22:14:05.816 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6000000000000004e-07 2024-09-20T22:14:05.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.817 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:05.817 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.797092+0000", 2024-09-20T22:14:05.817 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2630000000000003e-06 2024-09-20T22:14:05.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.817 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.817 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:05.817 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.797100+0000", 2024-09-20T22:14:05.817 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4610000000000008e-06 2024-09-20T22:14:05.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.817 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.817 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:05.817 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.797106+0000", 2024-09-20T22:14:05.817 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5899999999999998e-06 2024-09-20T22:14:05.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.818 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.818 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:05.818 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.797604+0000", 2024-09-20T22:14:05.818 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00049782500000000003 2024-09-20T22:14:05.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.818 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.818 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:05.818 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.797622+0000", 2024-09-20T22:14:05.818 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8717999999999999e-05 2024-09-20T22:14:05.818 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:05.818 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:05.818 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:05.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.819 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:163603 9.7 9:e270d197:::benchmark_data_smithi060_140132_object52795:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-20T22:14:05.819 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:14:05.799874+0000", 2024-09-20T22:14:05.819 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0031270600000000001, 2024-09-20T22:14:05.819 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0032652419999999998, 2024-09-20T22:14:05.819 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:05.819 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:05.819 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:14:05.819 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:05.819 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:14:05.819 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:14:05.819 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163603 2024-09-20T22:14:05.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.819 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:05.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.820 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:05.820 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.799874+0000", 2024-09-20T22:14:05.820 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:05.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.820 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.820 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:05.820 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.799874+0000", 2024-09-20T22:14:05.820 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:05.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.820 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.820 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:05.820 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.799876+0000", 2024-09-20T22:14:05.820 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.24e-06 2024-09-20T22:14:05.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.820 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.821 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:05.821 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.799881+0000", 2024-09-20T22:14:05.821 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7200000000000003e-06 2024-09-20T22:14:05.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.821 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:05.821 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.799893+0000", 2024-09-20T22:14:05.821 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1306e-05 2024-09-20T22:14:05.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.821 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:05.821 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.799900+0000", 2024-09-20T22:14:05.821 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1890000000000001e-06 2024-09-20T22:14:05.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.822 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:05.822 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.800597+0000", 2024-09-20T22:14:05.822 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00069752699999999998 2024-09-20T22:14:05.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.822 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.822 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:05.822 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.800708+0000", 2024-09-20T22:14:05.822 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000110486 2024-09-20T22:14:05.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.822 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.822 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:14:05.822 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.800844+0000", 2024-09-20T22:14:05.822 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013622100000000001 2024-09-20T22:14:05.822 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:05.823 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:05.823 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:05.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.823 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.823 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.15594.0:163598 9.b e348/328 9:d1a0df1f:::benchmark_data_smithi060_140132_object52790:head v 348'10059, mlcod=348'10055)", 2024-09-20T22:14:05.823 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:14:05.800212+0000", 2024-09-20T22:14:05.823 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.002788989, 2024-09-20T22:14:05.823 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0029662299999999998, 2024-09-20T22:14:05.823 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:05.823 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:05.823 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:14:05.823 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:05.823 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.823 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:05.823 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.800212+0000", 2024-09-20T22:14:05.823 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:05.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.824 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.824 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:05.824 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.800212+0000", 2024-09-20T22:14:05.824 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:05.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.824 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.824 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:05.824 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.800213+0000", 2024-09-20T22:14:05.824 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1800000000000005e-07 2024-09-20T22:14:05.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.824 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.824 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:05.824 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.800218+0000", 2024-09-20T22:14:05.824 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9980000000000003e-06 2024-09-20T22:14:05.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.825 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.825 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:05.825 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.800228+0000", 2024-09-20T22:14:05.825 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.713e-06 2024-09-20T22:14:05.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.825 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.825 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:05.825 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.800234+0000", 2024-09-20T22:14:05.825 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1510000000000001e-06 2024-09-20T22:14:05.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.825 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.825 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:05.825 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.801362+0000", 2024-09-20T22:14:05.825 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001128055 2024-09-20T22:14:05.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.826 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.826 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:05.826 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.801380+0000", 2024-09-20T22:14:05.826 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.768e-05 2024-09-20T22:14:05.826 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:05.826 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:05.826 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:05.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.826 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.826 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop_reply(client.15594.0:163601 9.7 e348/290)", 2024-09-20T22:14:05.826 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:14:05.800870+0000", 2024-09-20T22:14:05.826 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0021314070000000001, 2024-09-20T22:14:05.826 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0023494290000000001, 2024-09-20T22:14:05.826 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:05.826 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:05.827 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "queued for pg", 2024-09-20T22:14:05.827 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:05.827 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.827 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:05.827 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.800870+0000", 2024-09-20T22:14:05.827 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:05.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.827 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.827 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:05.827 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.800870+0000", 2024-09-20T22:14:05.827 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:05.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.827 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.827 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:05.827 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.800871+0000", 2024-09-20T22:14:05.827 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1200000000000003e-07 2024-09-20T22:14:05.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.828 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.828 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:05.828 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.800875+0000", 2024-09-20T22:14:05.828 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2440000000000004e-06 2024-09-20T22:14:05.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.828 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.828 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:05.828 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.800884+0000", 2024-09-20T22:14:05.828 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1810000000000007e-06 2024-09-20T22:14:05.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.828 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.828 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:05.828 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.800890+0000", 2024-09-20T22:14:05.829 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2940000000000002e-06 2024-09-20T22:14:05.829 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:05.829 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:05.829 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:05.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.829 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.829 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:163607 9.7 9.de61ded7 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e348)", 2024-09-20T22:14:05.829 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:14:05.803211+0000", 2024-09-20T22:14:05.829 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 4294967295.9997902, 2024-09-20T22:14:05.829 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8063000000000003e-05, 2024-09-20T22:14:05.829 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:05.829 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:05.829 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "queued for pg", 2024-09-20T22:14:05.829 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:05.829 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:14:05.830 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:14:05.830 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163607 2024-09-20T22:14:05.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.830 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:05.830 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.830 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:05.830 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.803211+0000", 2024-09-20T22:14:05.830 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:05.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.830 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.830 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:05.830 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.803211+0000", 2024-09-20T22:14:05.830 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:05.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.830 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.830 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:05.831 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.803213+0000", 2024-09-20T22:14:05.831 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.282e-06 2024-09-20T22:14:05.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.831 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.831 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:05.831 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.803218+0000", 2024-09-20T22:14:05.831 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3789999999999997e-06 2024-09-20T22:14:05.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.831 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.831 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:05.831 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.803232+0000", 2024-09-20T22:14:05.831 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.455e-05 2024-09-20T22:14:05.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:05.831 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:05.831 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:05.832 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:14:05.803240+0000", 2024-09-20T22:14:05.832 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7069999999999992e-06 2024-09-20T22:14:05.832 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:05.832 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:05.832 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:05.832 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:05.832 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-20T22:14:05.832 INFO:teuthology.orchestra.run.smithi067.stdout: "num_ops": 7 2024-09-20T22:14:05.832 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:14:05.835 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:05.869 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:05.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:05.871 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:05.972 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:06.013 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:06.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:06.014 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:06.115 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:06.154 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:06.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:06.155 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:06.245 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:06.256 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:06.284 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:06.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:06.285 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:06.386 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:06.420 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:06.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:06.421 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:06.522 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:06.573 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:06.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:06.574 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:06.609 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:14:06.611 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:14:06.676 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:06.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:06.713 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:06.714 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:06.768 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T22:14:06.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:06.848 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:06.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:06.849 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:06.950 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:06.971 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:14:06.971 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:14:06.971 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:06 smithi060 ceph-mon[29694]: pgmap v601: 158 pgs: 158 active+clean; 3.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-20T22:14:06.972 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:14:06.972 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:14:06.972 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:06 smithi060 ceph-mon[40290]: pgmap v601: 158 pgs: 158 active+clean; 3.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-20T22:14:06.982 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:06.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:06.983 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:14:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:14:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:06 smithi067 ceph-mon[33511]: pgmap v601: 158 pgs: 158 active+clean; 3.7 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.04k op/s 2024-09-20T22:14:07.085 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:07.121 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:07.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:07.122 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:07.191 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:14:07.222 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:07.270 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:07.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:07.271 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:07.372 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:07.397 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:14:07.408 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:07.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:07.409 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:07.510 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:07.554 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:07.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:07.555 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:07.651 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":601,"stamp":"2024-09-20T22:14:06.189931+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3929872947,"num_objects":60156,"num_object_clones":0,"num_object_copies":180468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60156,"num_whiteouts":0,"num_read":1787,"num_read_kb":1539,"num_write":162377,"num_write_kb":7091566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":8460563,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82873,"ondisk_log_size":82873,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":15838180,"kb_used_data":11742560,"kb_used_omap":15,"kb_used_meta":4094768,"kb_avail":734123036,"statfs":{"total":767960285184,"available":751741988864,"internally_reserved":0,"allocated":12024381440,"data_stored":12018180947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":4193043219},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,4,1,1,7],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-678821888,"num_objects":-10358,"num_object_clones":0,"num_object_copies":-31074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-10358,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10358,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.005779"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":59,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269681+0000","last_change":"2024-09-20T22:12:52.516248+0000","last_active":"2024-09-20T22:14:01.269681+0000","last_peered":"2024-09-20T22:14:01.269681+0000","last_clean":"2024-09-20T22:14:01.269681+0000","last_became_active":"2024-09-20T22:12:52.515921+0000","last_became_peered":"2024-09-20T22:12:52.515921+0000","last_unstale":"2024-09-20T22:14:01.269681+0000","last_undegraded":"2024-09-20T22:14:01.269681+0000","last_fullsized":"2024-09-20T22:14:01.269681+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:38.640782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_clean_scrub_stamp":"2024-09-20T22:12:38.640782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:06.828631+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"327'19","reported_seq":649,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.275846+0000","last_change":"2024-09-20T22:13:03.108336+0000","last_active":"2024-09-20T22:14:01.275846+0000","last_peered":"2024-09-20T22:14:01.275846+0000","last_clean":"2024-09-20T22:14:01.275846+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:14:01.275846+0000","last_undegraded":"2024-09-20T22:14:01.275846+0000","last_fullsized":"2024-09-20T22:14:01.275846+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"327'19","last_scrub_stamp":"2024-09-20T22:13:03.108203+0000","last_deep_scrub":"252'13","last_deep_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_clean_scrub_stamp":"2024-09-20T22:13:03.108203+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:31.739841+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":494,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.263397+0000","last_change":"2024-09-20T22:12:17.148238+0000","last_active":"2024-09-20T22:14:01.263397+0000","last_peered":"2024-09-20T22:14:01.263397+0000","last_clean":"2024-09-20T22:14:01.263397+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:14:01.263397+0000","last_undegraded":"2024-09-20T22:14:01.263397+0000","last_fullsized":"2024-09-20T22:14:01.263397+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:17.148092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:17.148092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:02.558229+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":616,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.275950+0000","last_change":"2024-09-20T22:12:42.264702+0000","last_active":"2024-09-20T22:14:01.275950+0000","last_peered":"2024-09-20T22:14:01.275950+0000","last_clean":"2024-09-20T22:14:01.275950+0000","last_became_active":"2024-09-20T22:11:30.485735+0000","last_became_peered":"2024-09-20T22:11:30.485735+0000","last_unstale":"2024-09-20T22:14:01.275950+0000","last_undegraded":"2024-09-20T22:14:01.275950+0000","last_fullsized":"2024-09-20T22:14:01.275950+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:42.264587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:42.264587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:03.058221+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":603,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.278807+0000","last_change":"2024-09-20T22:13:14.774356+0000","last_active":"2024-09-20T22:14:01.278807+0000","last_peered":"2024-09-20T22:14:01.278807+0000","last_clean":"2024-09-20T22:14:01.278807+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:14:01.278807+0000","last_undegraded":"2024-09-20T22:14:01.278807+0000","last_fullsized":"2024-09-20T22:14:01.278807+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:13:14.774240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:14.774240+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:32.288584+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":86,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.258334+0000","last_change":"2024-09-20T22:13:13.991120+0000","last_active":"2024-09-20T22:14:01.258334+0000","last_peered":"2024-09-20T22:14:01.258334+0000","last_clean":"2024-09-20T22:14:01.258334+0000","last_became_active":"2024-09-20T22:12:33.545100+0000","last_became_peered":"2024-09-20T22:12:33.545100+0000","last_unstale":"2024-09-20T22:14:01.258334+0000","last_undegraded":"2024-09-20T22:14:01.258334+0000","last_fullsized":"2024-09-20T22:14:01.258334+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:13.991017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:13.991017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:25.946263+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"328'27","reported_seq":200,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-20T22:13:55.696528+0000","last_change":"2024-09-20T22:13:39.102555+0000","last_active":"2024-09-20T22:13:55.696528+0000","last_peered":"2024-09-20T22:13:55.696528+0000","last_clean":"2024-09-20T22:13:55.696528+0000","last_became_active":"2024-09-20T22:12:52.517296+0000","last_became_peered":"2024-09-20T22:12:52.517296+0000","last_unstale":"2024-09-20T22:13:55.696528+0000","last_undegraded":"2024-09-20T22:13:55.696528+0000","last_fullsized":"2024-09-20T22:13:55.696528+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:13:39.102397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:39.102397+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:04.682528+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":608,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.275768+0000","last_change":"2024-09-20T22:13:06.156767+0000","last_active":"2024-09-20T22:14:01.275768+0000","last_peered":"2024-09-20T22:14:01.275768+0000","last_clean":"2024-09-20T22:14:01.275768+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:14:01.275768+0000","last_undegraded":"2024-09-20T22:14:01.275768+0000","last_fullsized":"2024-09-20T22:14:01.275768+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:06.156656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:06.156656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:33.362926+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":630,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269834+0000","last_change":"2024-09-20T22:12:01.173031+0000","last_active":"2024-09-20T22:14:01.269834+0000","last_peered":"2024-09-20T22:14:01.269834+0000","last_clean":"2024-09-20T22:14:01.269834+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:14:01.269834+0000","last_undegraded":"2024-09-20T22:14:01.269834+0000","last_fullsized":"2024-09-20T22:14:01.269834+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:01.172926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:12:01.172926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"328'15","reported_seq":668,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.276167+0000","last_change":"2024-09-20T22:12:52.532952+0000","last_active":"2024-09-20T22:14:01.276167+0000","last_peered":"2024-09-20T22:14:01.276167+0000","last_clean":"2024-09-20T22:14:01.276167+0000","last_became_active":"2024-09-20T22:12:52.516894+0000","last_became_peered":"2024-09-20T22:12:52.516894+0000","last_unstale":"2024-09-20T22:14:01.276167+0000","last_undegraded":"2024-09-20T22:14:01.276167+0000","last_fullsized":"2024-09-20T22:14:01.276167+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:12:10.172212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:10.172212+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.099571+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":162,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.268522+0000","last_change":"2024-09-20T22:12:52.518888+0000","last_active":"2024-09-20T22:14:01.268522+0000","last_peered":"2024-09-20T22:14:01.268522+0000","last_clean":"2024-09-20T22:14:01.268522+0000","last_became_active":"2024-09-20T22:12:52.518549+0000","last_became_peered":"2024-09-20T22:12:52.518549+0000","last_unstale":"2024-09-20T22:14:01.268522+0000","last_undegraded":"2024-09-20T22:14:01.268522+0000","last_fullsized":"2024-09-20T22:14:01.268522+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:31.101573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:31.101573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":67,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.268026+0000","last_change":"2024-09-20T22:13:02.488382+0000","last_active":"2024-09-20T22:14:01.268026+0000","last_peered":"2024-09-20T22:14:01.268026+0000","last_clean":"2024-09-20T22:14:01.268026+0000","last_became_active":"2024-09-20T22:12:52.524631+0000","last_became_peered":"2024-09-20T22:12:52.524631+0000","last_unstale":"2024-09-20T22:14:01.268026+0000","last_undegraded":"2024-09-20T22:14:01.268026+0000","last_fullsized":"2024-09-20T22:14:01.268026+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:02.488267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:02.488267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:13.577267+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":635,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.268771+0000","last_change":"2024-09-20T22:11:40.127688+0000","last_active":"2024-09-20T22:14:01.268771+0000","last_peered":"2024-09-20T22:14:01.268771+0000","last_clean":"2024-09-20T22:14:01.268771+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:14:01.268771+0000","last_undegraded":"2024-09-20T22:14:01.268771+0000","last_fullsized":"2024-09-20T22:14:01.268771+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:40.127597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:30.215263+0000","last_clean_scrub_stamp":"2024-09-20T22:11:40.127597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"328'7","reported_seq":645,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.276306+0000","last_change":"2024-09-20T22:13:44.057396+0000","last_active":"2024-09-20T22:14:01.276306+0000","last_peered":"2024-09-20T22:14:01.276306+0000","last_clean":"2024-09-20T22:14:01.276306+0000","last_became_active":"2024-09-20T22:08:22.144364+0000","last_became_peered":"2024-09-20T22:08:22.144364+0000","last_unstale":"2024-09-20T22:14:01.276306+0000","last_undegraded":"2024-09-20T22:14:01.276306+0000","last_fullsized":"2024-09-20T22:14:01.276306+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'7","last_scrub_stamp":"2024-09-20T22:13:44.057245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:44.057245+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:09.952476+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":616,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.273191+0000","last_change":"2024-09-20T22:12:38.636611+0000","last_active":"2024-09-20T22:14:01.273191+0000","last_peered":"2024-09-20T22:14:01.273191+0000","last_clean":"2024-09-20T22:14:01.273191+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:14:01.273191+0000","last_undegraded":"2024-09-20T22:14:01.273191+0000","last_fullsized":"2024-09-20T22:14:01.273191+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:38.636478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:38.636478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.852661+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":150,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.268451+0000","last_change":"2024-09-20T22:11:34.036027+0000","last_active":"2024-09-20T22:14:01.268451+0000","last_peered":"2024-09-20T22:14:01.268451+0000","last_clean":"2024-09-20T22:14:01.268451+0000","last_became_active":"2024-09-20T22:11:30.487137+0000","last_became_peered":"2024-09-20T22:11:30.487137+0000","last_unstale":"2024-09-20T22:14:01.268451+0000","last_undegraded":"2024-09-20T22:14:01.268451+0000","last_fullsized":"2024-09-20T22:14:01.268451+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:34.035938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.035938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":604,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.276700+0000","last_change":"2024-09-20T22:12:41.309522+0000","last_active":"2024-09-20T22:14:01.276700+0000","last_peered":"2024-09-20T22:14:01.276700+0000","last_clean":"2024-09-20T22:14:01.276700+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:14:01.276700+0000","last_undegraded":"2024-09-20T22:14:01.276700+0000","last_fullsized":"2024-09-20T22:14:01.276700+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:41.309424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:12:41.309424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.099788+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":67,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269650+0000","last_change":"2024-09-20T22:13:52.481857+0000","last_active":"2024-09-20T22:14:01.269650+0000","last_peered":"2024-09-20T22:14:01.269650+0000","last_clean":"2024-09-20T22:14:01.269650+0000","last_became_active":"2024-09-20T22:12:52.526097+0000","last_became_peered":"2024-09-20T22:12:52.526097+0000","last_unstale":"2024-09-20T22:14:01.269650+0000","last_undegraded":"2024-09-20T22:14:01.269650+0000","last_fullsized":"2024-09-20T22:14:01.269650+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:52.481584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:13:52.481584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:54.068422+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"328'34","reported_seq":204,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-20T22:13:55.696847+0000","last_change":"2024-09-20T22:13:46.127942+0000","last_active":"2024-09-20T22:13:55.696847+0000","last_peered":"2024-09-20T22:13:55.696847+0000","last_clean":"2024-09-20T22:13:55.696847+0000","last_became_active":"2024-09-20T22:11:12.507311+0000","last_became_peered":"2024-09-20T22:11:12.507311+0000","last_unstale":"2024-09-20T22:13:55.696847+0000","last_undegraded":"2024-09-20T22:13:55.696847+0000","last_fullsized":"2024-09-20T22:13:55.696847+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'34","last_scrub_stamp":"2024-09-20T22:13:46.127812+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:13:46.127812+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:47.873800+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":464,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.267510+0000","last_change":"2024-09-20T22:13:11.053381+0000","last_active":"2024-09-20T22:14:01.267510+0000","last_peered":"2024-09-20T22:14:01.267510+0000","last_clean":"2024-09-20T22:14:01.267510+0000","last_became_active":"2024-09-20T22:12:52.520985+0000","last_became_peered":"2024-09-20T22:12:52.520985+0000","last_unstale":"2024-09-20T22:14:01.267510+0000","last_undegraded":"2024-09-20T22:14:01.267510+0000","last_fullsized":"2024-09-20T22:14:01.267510+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:11.053276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:11.053276+0000","last_clean_scrub_stamp":"2024-09-20T22:13:11.053276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:35.430970+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":555,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.258520+0000","last_change":"2024-09-20T22:12:21.521799+0000","last_active":"2024-09-20T22:14:01.258520+0000","last_peered":"2024-09-20T22:14:01.258520+0000","last_clean":"2024-09-20T22:14:01.258520+0000","last_became_active":"2024-09-20T22:12:21.521433+0000","last_became_peered":"2024-09-20T22:12:21.521433+0000","last_unstale":"2024-09-20T22:14:01.258520+0000","last_undegraded":"2024-09-20T22:14:01.258520+0000","last_fullsized":"2024-09-20T22:14:01.258520+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:06.585138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:06.585138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":564,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269282+0000","last_change":"2024-09-20T22:12:44.197752+0000","last_active":"2024-09-20T22:14:01.269282+0000","last_peered":"2024-09-20T22:14:01.269282+0000","last_clean":"2024-09-20T22:14:01.269282+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:14:01.269282+0000","last_undegraded":"2024-09-20T22:14:01.269282+0000","last_fullsized":"2024-09-20T22:14:01.269282+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:44.197654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:44.197654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"328'23","reported_seq":529,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.267806+0000","last_change":"2024-09-20T22:13:08.115776+0000","last_active":"2024-09-20T22:14:01.267806+0000","last_peered":"2024-09-20T22:14:01.267806+0000","last_clean":"2024-09-20T22:14:01.267806+0000","last_became_active":"2024-09-20T22:12:21.517147+0000","last_became_peered":"2024-09-20T22:12:21.517147+0000","last_unstale":"2024-09-20T22:14:01.267806+0000","last_undegraded":"2024-09-20T22:14:01.267806+0000","last_fullsized":"2024-09-20T22:14:01.267806+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'23","last_scrub_stamp":"2024-09-20T22:13:08.115352+0000","last_deep_scrub":"250'17","last_deep_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_clean_scrub_stamp":"2024-09-20T22:13:08.115352+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:16.402904+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":607,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.276404+0000","last_change":"2024-09-20T22:13:51.133410+0000","last_active":"2024-09-20T22:14:01.276404+0000","last_peered":"2024-09-20T22:14:01.276404+0000","last_clean":"2024-09-20T22:14:01.276404+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:14:01.276404+0000","last_undegraded":"2024-09-20T22:14:01.276404+0000","last_fullsized":"2024-09-20T22:14:01.276404+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:51.133308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.133308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:13.505445+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":605,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.273017+0000","last_change":"2024-09-20T22:13:52.688758+0000","last_active":"2024-09-20T22:14:01.273017+0000","last_peered":"2024-09-20T22:14:01.273017+0000","last_clean":"2024-09-20T22:14:01.273017+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:14:01.273017+0000","last_undegraded":"2024-09-20T22:14:01.273017+0000","last_fullsized":"2024-09-20T22:14:01.273017+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:13:52.688650+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:13:52.688650+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:55.995741+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"289'2","reported_seq":454,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.263483+0000","last_change":"2024-09-20T22:13:02.145576+0000","last_active":"2024-09-20T22:14:01.263483+0000","last_peered":"2024-09-20T22:14:01.263483+0000","last_clean":"2024-09-20T22:14:01.263483+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:14:01.263483+0000","last_undegraded":"2024-09-20T22:14:01.263483+0000","last_fullsized":"2024-09-20T22:14:01.263483+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-20T22:13:02.145448+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_clean_scrub_stamp":"2024-09-20T22:13:02.145448+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:02.302852+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"328'20","reported_seq":86,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.268916+0000","last_change":"2024-09-20T22:12:52.539465+0000","last_active":"2024-09-20T22:14:01.268916+0000","last_peered":"2024-09-20T22:14:01.268916+0000","last_clean":"2024-09-20T22:14:01.268916+0000","last_became_active":"2024-09-20T22:12:52.526803+0000","last_became_peered":"2024-09-20T22:12:52.526803+0000","last_unstale":"2024-09-20T22:14:01.268916+0000","last_undegraded":"2024-09-20T22:14:01.268916+0000","last_fullsized":"2024-09-20T22:14:01.268916+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'10","last_scrub_stamp":"2024-09-20T22:11:35.700046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:11:35.700046+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"291'11","reported_seq":790,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.258644+0000","last_change":"2024-09-20T22:13:53.376728+0000","last_active":"2024-09-20T22:14:01.258644+0000","last_peered":"2024-09-20T22:14:01.258644+0000","last_clean":"2024-09-20T22:14:01.258644+0000","last_became_active":"2024-09-20T22:11:40.601048+0000","last_became_peered":"2024-09-20T22:11:40.601048+0000","last_unstale":"2024-09-20T22:14:01.258644+0000","last_undegraded":"2024-09-20T22:14:01.258644+0000","last_fullsized":"2024-09-20T22:14:01.258644+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'11","last_scrub_stamp":"2024-09-20T22:13:53.376616+0000","last_deep_scrub":"291'11","last_deep_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_clean_scrub_stamp":"2024-09-20T22:13:53.376616+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:01.413994+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"348'10019","reported_seq":10214,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:03.254816+0000","last_change":"2024-09-20T22:13:48.324296+0000","last_active":"2024-09-20T22:14:03.254816+0000","last_peered":"2024-09-20T22:14:03.254816+0000","last_clean":"2024-09-20T22:14:03.254816+0000","last_became_active":"2024-09-20T22:12:33.545160+0000","last_became_peered":"2024-09-20T22:12:33.545160+0000","last_unstale":"2024-09-20T22:14:03.254816+0000","last_undegraded":"2024-09-20T22:14:03.254816+0000","last_fullsized":"2024-09-20T22:14:03.254816+0000","mapping_epoch":314,"log_start":"317'5464","ondisk_log_start":"317'5464","created":286,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"345'9130","last_scrub_stamp":"2024-09-20T22:13:48.324088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:13:48.324088+0000","objects_scrubbed":4747,"log_size":4555,"log_dups_size":0,"ondisk_log_size":4555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:56.742135+0000","scrub_duration":1847,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":249626624,"num_objects":3809,"num_object_clones":0,"num_object_copies":11427,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10019,"num_write_kb":442496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":618,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.268725+0000","last_change":"2024-09-20T22:13:08.407055+0000","last_active":"2024-09-20T22:14:01.268725+0000","last_peered":"2024-09-20T22:14:01.268725+0000","last_clean":"2024-09-20T22:14:01.268725+0000","last_became_active":"2024-09-20T22:08:48.452786+0000","last_became_peered":"2024-09-20T22:08:48.452786+0000","last_unstale":"2024-09-20T22:14:01.268725+0000","last_undegraded":"2024-09-20T22:14:01.268725+0000","last_fullsized":"2024-09-20T22:14:01.268725+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:08.406883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:13:08.406883+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:28.449151+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"328'27","reported_seq":397,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-20T22:13:55.698688+0000","last_change":"2024-09-20T22:13:06.060888+0000","last_active":"2024-09-20T22:13:55.698688+0000","last_peered":"2024-09-20T22:13:55.698688+0000","last_clean":"2024-09-20T22:13:55.698688+0000","last_became_active":"2024-09-20T22:08:22.145408+0000","last_became_peered":"2024-09-20T22:08:22.145408+0000","last_unstale":"2024-09-20T22:13:55.698688+0000","last_undegraded":"2024-09-20T22:13:55.698688+0000","last_fullsized":"2024-09-20T22:13:55.698688+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:13:06.060761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:06.060761+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:14.045591+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":624,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.288291+0000","last_change":"2024-09-20T22:13:53.012457+0000","last_active":"2024-09-20T22:14:01.288291+0000","last_peered":"2024-09-20T22:14:01.288291+0000","last_clean":"2024-09-20T22:14:01.288291+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:14:01.288291+0000","last_undegraded":"2024-09-20T22:14:01.288291+0000","last_fullsized":"2024-09-20T22:14:01.288291+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:53.012225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:53.012225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:05.749378+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"326'13","reported_seq":812,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.276810+0000","last_change":"2024-09-20T22:12:52.533157+0000","last_active":"2024-09-20T22:14:01.276810+0000","last_peered":"2024-09-20T22:14:01.276810+0000","last_clean":"2024-09-20T22:14:01.276810+0000","last_became_active":"2024-09-20T22:12:52.517588+0000","last_became_peered":"2024-09-20T22:12:52.517588+0000","last_unstale":"2024-09-20T22:14:01.276810+0000","last_undegraded":"2024-09-20T22:14:01.276810+0000","last_fullsized":"2024-09-20T22:14:01.276810+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'11","last_scrub_stamp":"2024-09-20T22:12:16.129029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:16.129029+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":162,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269481+0000","last_change":"2024-09-20T22:12:52.518570+0000","last_active":"2024-09-20T22:14:01.269481+0000","last_peered":"2024-09-20T22:14:01.269481+0000","last_clean":"2024-09-20T22:14:01.269481+0000","last_became_active":"2024-09-20T22:12:52.517826+0000","last_became_peered":"2024-09-20T22:12:52.517826+0000","last_unstale":"2024-09-20T22:14:01.269481+0000","last_undegraded":"2024-09-20T22:14:01.269481+0000","last_fullsized":"2024-09-20T22:14:01.269481+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:07.114531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:07.114531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"348'10190","reported_seq":10326,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:05.734939+0000","last_change":"2024-09-20T22:11:34.561887+0000","last_active":"2024-09-20T22:14:05.734939+0000","last_peered":"2024-09-20T22:14:05.734939+0000","last_clean":"2024-09-20T22:14:05.734939+0000","last_became_active":"2024-09-20T22:11:34.561538+0000","last_became_peered":"2024-09-20T22:11:34.561538+0000","last_unstale":"2024-09-20T22:14:05.734939+0000","last_undegraded":"2024-09-20T22:14:05.734939+0000","last_fullsized":"2024-09-20T22:14:05.734939+0000","mapping_epoch":286,"log_start":"329'5623","ondisk_log_start":"329'5623","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":4567,"log_dups_size":0,"ondisk_log_size":4567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.852334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237305880,"num_objects":3622,"num_object_clones":0,"num_object_copies":10866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3622,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10190,"num_write_kb":441921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":610,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.276991+0000","last_change":"2024-09-20T22:12:07.204327+0000","last_active":"2024-09-20T22:14:01.276991+0000","last_peered":"2024-09-20T22:14:01.276991+0000","last_clean":"2024-09-20T22:14:01.276991+0000","last_became_active":"2024-09-20T22:08:48.452475+0000","last_became_peered":"2024-09-20T22:08:48.452475+0000","last_unstale":"2024-09-20T22:14:01.276991+0000","last_undegraded":"2024-09-20T22:14:01.276991+0000","last_fullsized":"2024-09-20T22:14:01.276991+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:12:07.204213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:12:07.204213+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.099641+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"328'30","reported_seq":627,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.272547+0000","last_change":"2024-09-20T22:13:50.715007+0000","last_active":"2024-09-20T22:14:01.272547+0000","last_peered":"2024-09-20T22:14:01.272547+0000","last_clean":"2024-09-20T22:14:01.272547+0000","last_became_active":"2024-09-20T22:12:52.526867+0000","last_became_peered":"2024-09-20T22:12:52.526867+0000","last_unstale":"2024-09-20T22:14:01.272547+0000","last_undegraded":"2024-09-20T22:14:01.272547+0000","last_fullsized":"2024-09-20T22:14:01.272547+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'30","last_scrub_stamp":"2024-09-20T22:13:50.714867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:50.714867+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:05.605678+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":100,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.273420+0000","last_change":"2024-09-20T22:12:52.529301+0000","last_active":"2024-09-20T22:14:01.273420+0000","last_peered":"2024-09-20T22:14:01.273420+0000","last_clean":"2024-09-20T22:14:01.273420+0000","last_became_active":"2024-09-20T22:12:52.528947+0000","last_became_peered":"2024-09-20T22:12:52.528947+0000","last_unstale":"2024-09-20T22:14:01.273420+0000","last_undegraded":"2024-09-20T22:14:01.273420+0000","last_fullsized":"2024-09-20T22:14:01.273420+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:04.532220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:04.532220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.852410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":620,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.275491+0000","last_change":"2024-09-20T22:12:06.219908+0000","last_active":"2024-09-20T22:14:01.275491+0000","last_peered":"2024-09-20T22:14:01.275491+0000","last_clean":"2024-09-20T22:14:01.275491+0000","last_became_active":"2024-09-20T22:10:48.919949+0000","last_became_peered":"2024-09-20T22:10:48.919949+0000","last_unstale":"2024-09-20T22:14:01.275491+0000","last_undegraded":"2024-09-20T22:14:01.275491+0000","last_fullsized":"2024-09-20T22:14:01.275491+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:06.219799+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:06.219799+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.099538+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":604,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.276079+0000","last_change":"2024-09-20T22:11:40.260396+0000","last_active":"2024-09-20T22:14:01.276079+0000","last_peered":"2024-09-20T22:14:01.276079+0000","last_clean":"2024-09-20T22:14:01.276079+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:14:01.276079+0000","last_undegraded":"2024-09-20T22:14:01.276079+0000","last_fullsized":"2024-09-20T22:14:01.276079+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:40.260289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:11:40.260289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.099466+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"348'10381","reported_seq":10582,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:05.721777+0000","last_change":"2024-09-20T22:13:50.256083+0000","last_active":"2024-09-20T22:14:05.721777+0000","last_peered":"2024-09-20T22:14:05.721777+0000","last_clean":"2024-09-20T22:14:05.721777+0000","last_became_active":"2024-09-20T22:12:33.546173+0000","last_became_peered":"2024-09-20T22:12:33.546173+0000","last_unstale":"2024-09-20T22:14:05.721777+0000","last_undegraded":"2024-09-20T22:14:05.721777+0000","last_fullsized":"2024-09-20T22:14:05.721777+0000","mapping_epoch":314,"log_start":"317'5507","ondisk_log_start":"317'5507","created":286,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"345'9370","last_scrub_stamp":"2024-09-20T22:13:50.255922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:13:50.255922+0000","objects_scrubbed":4781,"log_size":4874,"log_dups_size":0,"ondisk_log_size":4874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:03.962179+0000","scrub_duration":1129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":244383744,"num_objects":3729,"num_object_clones":0,"num_object_copies":11187,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10381,"num_write_kb":451520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":644,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.278122+0000","last_change":"2024-09-20T22:13:36.804562+0000","last_active":"2024-09-20T22:14:01.278122+0000","last_peered":"2024-09-20T22:14:01.278122+0000","last_clean":"2024-09-20T22:14:01.278122+0000","last_became_active":"2024-09-20T22:11:33.560632+0000","last_became_peered":"2024-09-20T22:11:33.560632+0000","last_unstale":"2024-09-20T22:14:01.278122+0000","last_undegraded":"2024-09-20T22:14:01.278122+0000","last_fullsized":"2024-09-20T22:14:01.278122+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:36.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:13:36.803555+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:58.732994+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"328'28","reported_seq":654,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.275667+0000","last_change":"2024-09-20T22:12:08.158731+0000","last_active":"2024-09-20T22:14:01.275667+0000","last_peered":"2024-09-20T22:14:01.275667+0000","last_clean":"2024-09-20T22:14:01.275667+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:14:01.275667+0000","last_undegraded":"2024-09-20T22:14:01.275667+0000","last_fullsized":"2024-09-20T22:14:01.275667+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'14","last_scrub_stamp":"2024-09-20T22:12:08.158574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:08.158574+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.099702+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":71,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269102+0000","last_change":"2024-09-20T22:13:47.454245+0000","last_active":"2024-09-20T22:14:01.269102+0000","last_peered":"2024-09-20T22:14:01.269102+0000","last_clean":"2024-09-20T22:14:01.269102+0000","last_became_active":"2024-09-20T22:12:52.528068+0000","last_became_peered":"2024-09-20T22:12:52.528068+0000","last_unstale":"2024-09-20T22:14:01.269102+0000","last_undegraded":"2024-09-20T22:14:01.269102+0000","last_fullsized":"2024-09-20T22:14:01.269102+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:13:47.454137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:47.454137+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:15.159536+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"315'11","reported_seq":330,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.263075+0000","last_change":"2024-09-20T22:13:05.177108+0000","last_active":"2024-09-20T22:14:01.263075+0000","last_peered":"2024-09-20T22:14:01.263075+0000","last_clean":"2024-09-20T22:14:01.263075+0000","last_became_active":"2024-09-20T22:12:33.545942+0000","last_became_peered":"2024-09-20T22:12:33.545942+0000","last_unstale":"2024-09-20T22:14:01.263075+0000","last_undegraded":"2024-09-20T22:14:01.263075+0000","last_fullsized":"2024-09-20T22:14:01.263075+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:13:05.176973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:05.176973+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:23.809705+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":604,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.273483+0000","last_change":"2024-09-20T22:13:10.621055+0000","last_active":"2024-09-20T22:14:01.273483+0000","last_peered":"2024-09-20T22:14:01.273483+0000","last_clean":"2024-09-20T22:14:01.273483+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:14:01.273483+0000","last_undegraded":"2024-09-20T22:14:01.273483+0000","last_fullsized":"2024-09-20T22:14:01.273483+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:10.620925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.620925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:19.180309+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"348'13418","reported_seq":13688,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:06.017551+0000","last_change":"2024-09-20T22:14:01.375934+0000","last_active":"2024-09-20T22:14:06.017551+0000","last_peered":"2024-09-20T22:14:06.017551+0000","last_clean":"2024-09-20T22:14:06.017551+0000","last_became_active":"2024-09-20T22:11:34.558898+0000","last_became_peered":"2024-09-20T22:11:34.558898+0000","last_unstale":"2024-09-20T22:14:06.017551+0000","last_undegraded":"2024-09-20T22:14:06.017551+0000","last_fullsized":"2024-09-20T22:14:06.017551+0000","mapping_epoch":286,"log_start":"341'8428","ondisk_log_start":"341'8428","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":3177,"log_size":4990,"log_dups_size":0,"ondisk_log_size":4990,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:06.367838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242614272,"num_objects":3702,"num_object_clones":0,"num_object_copies":11106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10262,"num_write_kb":446848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":640,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.258435+0000","last_change":"2024-09-20T22:12:43.418176+0000","last_active":"2024-09-20T22:14:01.258435+0000","last_peered":"2024-09-20T22:14:01.258435+0000","last_clean":"2024-09-20T22:14:01.258435+0000","last_became_active":"2024-09-20T22:10:33.660948+0000","last_became_peered":"2024-09-20T22:10:33.660948+0000","last_unstale":"2024-09-20T22:14:01.258435+0000","last_undegraded":"2024-09-20T22:14:01.258435+0000","last_fullsized":"2024-09-20T22:14:01.258435+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:12:43.418052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:12:43.418052+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"328'20","reported_seq":530,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.264983+0000","last_change":"2024-09-20T22:13:41.421053+0000","last_active":"2024-09-20T22:14:01.264983+0000","last_peered":"2024-09-20T22:14:01.264983+0000","last_clean":"2024-09-20T22:14:01.264983+0000","last_became_active":"2024-09-20T22:11:30.486532+0000","last_became_peered":"2024-09-20T22:11:30.486532+0000","last_unstale":"2024-09-20T22:14:01.264983+0000","last_undegraded":"2024-09-20T22:14:01.264983+0000","last_fullsized":"2024-09-20T22:14:01.264983+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:13:41.420933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:41.420933+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:00.706107+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":631,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269911+0000","last_change":"2024-09-20T22:11:42.173417+0000","last_active":"2024-09-20T22:14:01.269911+0000","last_peered":"2024-09-20T22:14:01.269911+0000","last_clean":"2024-09-20T22:14:01.269911+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:14:01.269911+0000","last_undegraded":"2024-09-20T22:14:01.269911+0000","last_fullsized":"2024-09-20T22:14:01.269911+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:42.173334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:11:42.173334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"327'12","reported_seq":74,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.268667+0000","last_change":"2024-09-20T22:12:57.887268+0000","last_active":"2024-09-20T22:14:01.268667+0000","last_peered":"2024-09-20T22:14:01.268667+0000","last_clean":"2024-09-20T22:14:01.268667+0000","last_became_active":"2024-09-20T22:12:57.884415+0000","last_became_peered":"2024-09-20T22:12:57.884415+0000","last_unstale":"2024-09-20T22:14:01.268667+0000","last_undegraded":"2024-09-20T22:14:01.268667+0000","last_fullsized":"2024-09-20T22:14:01.268667+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'10","last_scrub_stamp":"2024-09-20T22:12:07.451943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:07.451943+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":59,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.267871+0000","last_change":"2024-09-20T22:12:52.519036+0000","last_active":"2024-09-20T22:14:01.267871+0000","last_peered":"2024-09-20T22:14:01.267871+0000","last_clean":"2024-09-20T22:14:01.267871+0000","last_became_active":"2024-09-20T22:12:52.518529+0000","last_became_peered":"2024-09-20T22:12:52.518529+0000","last_unstale":"2024-09-20T22:14:01.267871+0000","last_undegraded":"2024-09-20T22:14:01.267871+0000","last_fullsized":"2024-09-20T22:14:01.267871+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:41.689984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:41.689984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"348'10268","reported_seq":10460,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:05.742483+0000","last_change":"2024-09-20T22:13:44.886681+0000","last_active":"2024-09-20T22:14:05.742483+0000","last_peered":"2024-09-20T22:14:05.742483+0000","last_clean":"2024-09-20T22:14:05.742483+0000","last_became_active":"2024-09-20T22:11:34.560514+0000","last_became_peered":"2024-09-20T22:11:34.560514+0000","last_unstale":"2024-09-20T22:14:05.742483+0000","last_undegraded":"2024-09-20T22:14:05.742483+0000","last_fullsized":"2024-09-20T22:14:05.742483+0000","mapping_epoch":286,"log_start":"329'5766","ondisk_log_start":"329'5766","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"345'8947","last_scrub_stamp":"2024-09-20T22:13:44.886563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:13:44.886563+0000","objects_scrubbed":4900,"log_size":4502,"log_dups_size":0,"ondisk_log_size":4502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:45.294613+0000","scrub_duration":1137,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":232390656,"num_objects":3546,"num_object_clones":0,"num_object_copies":10638,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10268,"num_write_kb":442048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":641,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.258149+0000","last_change":"2024-09-20T22:13:08.929614+0000","last_active":"2024-09-20T22:14:01.258149+0000","last_peered":"2024-09-20T22:14:01.258149+0000","last_clean":"2024-09-20T22:14:01.258149+0000","last_became_active":"2024-09-20T22:12:55.523589+0000","last_became_peered":"2024-09-20T22:12:55.523589+0000","last_unstale":"2024-09-20T22:14:01.258149+0000","last_undegraded":"2024-09-20T22:14:01.258149+0000","last_fullsized":"2024-09-20T22:14:01.258149+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:08.929484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:13:08.929484+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:14.299669+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"328'29","reported_seq":92,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269556+0000","last_change":"2024-09-20T22:12:52.544145+0000","last_active":"2024-09-20T22:14:01.269556+0000","last_peered":"2024-09-20T22:14:01.269556+0000","last_clean":"2024-09-20T22:14:01.269556+0000","last_became_active":"2024-09-20T22:12:52.525538+0000","last_became_peered":"2024-09-20T22:12:52.525538+0000","last_unstale":"2024-09-20T22:14:01.269556+0000","last_undegraded":"2024-09-20T22:14:01.269556+0000","last_fullsized":"2024-09-20T22:14:01.269556+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'19","last_scrub_stamp":"2024-09-20T22:12:12.553949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:12.553949+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":466,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.266303+0000","last_change":"2024-09-20T22:13:13.091930+0000","last_active":"2024-09-20T22:14:01.266303+0000","last_peered":"2024-09-20T22:14:01.266303+0000","last_clean":"2024-09-20T22:14:01.266303+0000","last_became_active":"2024-09-20T22:11:30.486428+0000","last_became_peered":"2024-09-20T22:11:30.486428+0000","last_unstale":"2024-09-20T22:14:01.266303+0000","last_undegraded":"2024-09-20T22:14:01.266303+0000","last_fullsized":"2024-09-20T22:14:01.266303+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:13.091819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:13.091819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:34.138090+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":635,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.257170+0000","last_change":"2024-09-20T22:12:52.521078+0000","last_active":"2024-09-20T22:14:01.257170+0000","last_peered":"2024-09-20T22:14:01.257170+0000","last_clean":"2024-09-20T22:14:01.257170+0000","last_became_active":"2024-09-20T22:12:52.519826+0000","last_became_peered":"2024-09-20T22:12:52.519826+0000","last_unstale":"2024-09-20T22:14:01.257170+0000","last_undegraded":"2024-09-20T22:14:01.257170+0000","last_fullsized":"2024-09-20T22:14:01.257170+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:14.259029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:12:14.259029+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":452,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.263267+0000","last_change":"2024-09-20T22:13:01.117688+0000","last_active":"2024-09-20T22:14:01.263267+0000","last_peered":"2024-09-20T22:14:01.263267+0000","last_clean":"2024-09-20T22:14:01.263267+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:14:01.263267+0000","last_undegraded":"2024-09-20T22:14:01.263267+0000","last_fullsized":"2024-09-20T22:14:01.263267+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:01.117578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:01.117578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:23.322241+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"348'10139","reported_seq":10285,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.827563+0000","last_change":"2024-09-20T22:12:53.516854+0000","last_active":"2024-09-20T22:14:01.827563+0000","last_peered":"2024-09-20T22:14:01.827563+0000","last_clean":"2024-09-20T22:14:01.827563+0000","last_became_active":"2024-09-20T22:12:53.512687+0000","last_became_peered":"2024-09-20T22:12:53.512687+0000","last_unstale":"2024-09-20T22:14:01.827563+0000","last_undegraded":"2024-09-20T22:14:01.827563+0000","last_fullsized":"2024-09-20T22:14:01.827563+0000","mapping_epoch":324,"log_start":"311'4422","ondisk_log_start":"311'4422","created":286,"last_epoch_clean":325,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":5717,"log_dups_size":0,"ondisk_log_size":5717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262209536,"num_objects":4001,"num_object_clones":0,"num_object_copies":12003,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10139,"num_write_kb":452480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":636,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.268340+0000","last_change":"2024-09-20T22:12:55.522020+0000","last_active":"2024-09-20T22:14:01.268340+0000","last_peered":"2024-09-20T22:14:01.268340+0000","last_clean":"2024-09-20T22:14:01.268340+0000","last_became_active":"2024-09-20T22:12:55.521605+0000","last_became_peered":"2024-09-20T22:12:55.521605+0000","last_unstale":"2024-09-20T22:14:01.268340+0000","last_undegraded":"2024-09-20T22:14:01.268340+0000","last_fullsized":"2024-09-20T22:14:01.268340+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:12:16.121263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:12:16.121263+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"328'52","reported_seq":629,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.266528+0000","last_change":"2024-09-20T22:13:36.392213+0000","last_active":"2024-09-20T22:14:01.266528+0000","last_peered":"2024-09-20T22:14:01.266528+0000","last_clean":"2024-09-20T22:14:01.266528+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:14:01.266528+0000","last_undegraded":"2024-09-20T22:14:01.266528+0000","last_fullsized":"2024-09-20T22:14:01.266528+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'52","last_scrub_stamp":"2024-09-20T22:13:36.392127+0000","last_deep_scrub":"271'40","last_deep_scrub_stamp":"2024-09-20T22:11:05.235779+0000","last_clean_scrub_stamp":"2024-09-20T22:13:36.392127+0000","objects_scrubbed":10,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:41.333635+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":625,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.276521+0000","last_change":"2024-09-20T22:12:57.884648+0000","last_active":"2024-09-20T22:14:01.276521+0000","last_peered":"2024-09-20T22:14:01.276521+0000","last_clean":"2024-09-20T22:14:01.276521+0000","last_became_active":"2024-09-20T22:12:57.884257+0000","last_became_peered":"2024-09-20T22:12:57.884257+0000","last_unstale":"2024-09-20T22:14:01.276521+0000","last_undegraded":"2024-09-20T22:14:01.276521+0000","last_fullsized":"2024-09-20T22:14:01.276521+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:03.239256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:03.239256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.099730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":635,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269605+0000","last_change":"2024-09-20T22:13:51.454966+0000","last_active":"2024-09-20T22:14:01.269605+0000","last_peered":"2024-09-20T22:14:01.269605+0000","last_clean":"2024-09-20T22:14:01.269605+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:14:01.269605+0000","last_undegraded":"2024-09-20T22:14:01.269605+0000","last_fullsized":"2024-09-20T22:14:01.269605+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.454850+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:55.603862+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":59,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269300+0000","last_change":"2024-09-20T22:12:52.528059+0000","last_active":"2024-09-20T22:14:01.269300+0000","last_peered":"2024-09-20T22:14:01.269300+0000","last_clean":"2024-09-20T22:14:01.269300+0000","last_became_active":"2024-09-20T22:12:52.527547+0000","last_became_peered":"2024-09-20T22:12:52.527547+0000","last_unstale":"2024-09-20T22:14:01.269300+0000","last_undegraded":"2024-09-20T22:14:01.269300+0000","last_fullsized":"2024-09-20T22:14:01.269300+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:59.462479+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:59.462479+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":603,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.277802+0000","last_change":"2024-09-20T22:13:51.008308+0000","last_active":"2024-09-20T22:14:01.277802+0000","last_peered":"2024-09-20T22:14:01.277802+0000","last_clean":"2024-09-20T22:14:01.277802+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:14:01.277802+0000","last_undegraded":"2024-09-20T22:14:01.277802+0000","last_fullsized":"2024-09-20T22:14:01.277802+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:51.008202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.008202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:59.375373+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"348'9980","reported_seq":10175,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:06.047137+0000","last_change":"2024-09-20T22:13:40.982764+0000","last_active":"2024-09-20T22:14:06.047137+0000","last_peered":"2024-09-20T22:14:06.047137+0000","last_clean":"2024-09-20T22:14:06.047137+0000","last_became_active":"2024-09-20T22:11:34.558703+0000","last_became_peered":"2024-09-20T22:11:34.558703+0000","last_unstale":"2024-09-20T22:14:06.047137+0000","last_undegraded":"2024-09-20T22:14:06.047137+0000","last_fullsized":"2024-09-20T22:14:06.047137+0000","mapping_epoch":286,"log_start":"317'5061","ondisk_log_start":"317'5061","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"343'8471","last_scrub_stamp":"2024-09-20T22:13:40.982606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:13:40.982606+0000","objects_scrubbed":5133,"log_size":4919,"log_dups_size":0,"ondisk_log_size":4919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:52.855309+0000","scrub_duration":1519,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":234094592,"num_objects":3572,"num_object_clones":0,"num_object_copies":10716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9980,"num_write_kb":433664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":626,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.272789+0000","last_change":"2024-09-20T22:12:55.518461+0000","last_active":"2024-09-20T22:14:01.272789+0000","last_peered":"2024-09-20T22:14:01.272789+0000","last_clean":"2024-09-20T22:14:01.272789+0000","last_became_active":"2024-09-20T22:12:55.518061+0000","last_became_peered":"2024-09-20T22:12:55.518061+0000","last_unstale":"2024-09-20T22:14:01.272789+0000","last_undegraded":"2024-09-20T22:14:01.272789+0000","last_fullsized":"2024-09-20T22:14:01.272789+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:12:42.665244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:12:42.665244+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.852719+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"328'17","reported_seq":533,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.278010+0000","last_change":"2024-09-20T22:13:51.684806+0000","last_active":"2024-09-20T22:14:01.278010+0000","last_peered":"2024-09-20T22:14:01.278010+0000","last_clean":"2024-09-20T22:14:01.278010+0000","last_became_active":"2024-09-20T22:12:52.521073+0000","last_became_peered":"2024-09-20T22:12:52.521073+0000","last_unstale":"2024-09-20T22:14:01.278010+0000","last_undegraded":"2024-09-20T22:14:01.278010+0000","last_fullsized":"2024-09-20T22:14:01.278010+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'17","last_scrub_stamp":"2024-09-20T22:13:51.684667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.684667+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:19.648229+0000","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":453,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.263325+0000","last_change":"2024-09-20T22:13:09.056527+0000","last_active":"2024-09-20T22:14:01.263325+0000","last_peered":"2024-09-20T22:14:01.263325+0000","last_clean":"2024-09-20T22:14:01.263325+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:14:01.263325+0000","last_undegraded":"2024-09-20T22:14:01.263325+0000","last_fullsized":"2024-09-20T22:14:01.263325+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:09.056286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:09.056286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:09.464160+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":667,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-20T22:13:55.696558+0000","last_change":"2024-09-20T22:12:52.524326+0000","last_active":"2024-09-20T22:13:55.696558+0000","last_peered":"2024-09-20T22:13:55.696558+0000","last_clean":"2024-09-20T22:13:55.696558+0000","last_became_active":"2024-09-20T22:12:52.517255+0000","last_became_peered":"2024-09-20T22:12:52.517255+0000","last_unstale":"2024-09-20T22:13:55.696558+0000","last_undegraded":"2024-09-20T22:13:55.696558+0000","last_fullsized":"2024-09-20T22:13:55.696558+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:12:45.211188+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:12:45.211188+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:03.295252+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":621,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-20T22:13:55.710333+0000","last_change":"2024-09-20T22:12:42.213233+0000","last_active":"2024-09-20T22:13:55.710333+0000","last_peered":"2024-09-20T22:13:55.710333+0000","last_clean":"2024-09-20T22:13:55.710333+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:13:55.710333+0000","last_undegraded":"2024-09-20T22:13:55.710333+0000","last_fullsized":"2024-09-20T22:13:55.710333+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:42.212687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:12:42.212687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":604,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.273189+0000","last_change":"2024-09-20T22:13:41.730912+0000","last_active":"2024-09-20T22:14:01.273189+0000","last_peered":"2024-09-20T22:14:01.273189+0000","last_clean":"2024-09-20T22:14:01.273189+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:14:01.273189+0000","last_undegraded":"2024-09-20T22:14:01.273189+0000","last_fullsized":"2024-09-20T22:14:01.273189+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:41.730776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:41.730776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:10.936487+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"348'9772","reported_seq":9899,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.841807+0000","last_change":"2024-09-20T22:12:59.048311+0000","last_active":"2024-09-20T22:14:01.841807+0000","last_peered":"2024-09-20T22:14:01.841807+0000","last_clean":"2024-09-20T22:14:01.841807+0000","last_became_active":"2024-09-20T22:12:58.947913+0000","last_became_peered":"2024-09-20T22:12:58.947913+0000","last_unstale":"2024-09-20T22:14:01.841807+0000","last_undegraded":"2024-09-20T22:14:01.841807+0000","last_fullsized":"2024-09-20T22:14:01.841807+0000","mapping_epoch":328,"log_start":"310'4100","ondisk_log_start":"310'4100","created":286,"last_epoch_clean":329,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":5672,"log_dups_size":0,"ondisk_log_size":5672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":258867200,"num_objects":3950,"num_object_clones":0,"num_object_copies":11850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9772,"num_write_kb":439104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":617,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.264300+0000","last_change":"2024-09-20T22:13:48.490583+0000","last_active":"2024-09-20T22:14:01.264300+0000","last_peered":"2024-09-20T22:14:01.264300+0000","last_clean":"2024-09-20T22:14:01.264300+0000","last_became_active":"2024-09-20T22:08:48.446840+0000","last_became_peered":"2024-09-20T22:08:48.446840+0000","last_unstale":"2024-09-20T22:14:01.264300+0000","last_undegraded":"2024-09-20T22:14:01.264300+0000","last_fullsized":"2024-09-20T22:14:01.264300+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:13:48.490464+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:18.218793+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"328'24","reported_seq":99,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269011+0000","last_change":"2024-09-20T22:13:44.523683+0000","last_active":"2024-09-20T22:14:01.269011+0000","last_peered":"2024-09-20T22:14:01.269011+0000","last_clean":"2024-09-20T22:14:01.269011+0000","last_became_active":"2024-09-20T22:12:52.527919+0000","last_became_peered":"2024-09-20T22:12:52.527919+0000","last_unstale":"2024-09-20T22:14:01.269011+0000","last_undegraded":"2024-09-20T22:14:01.269011+0000","last_fullsized":"2024-09-20T22:14:01.269011+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'24","last_scrub_stamp":"2024-09-20T22:13:44.523540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:44.523540+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:00.404633+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":67,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.267427+0000","last_change":"2024-09-20T22:13:41.549500+0000","last_active":"2024-09-20T22:14:01.267427+0000","last_peered":"2024-09-20T22:14:01.267427+0000","last_clean":"2024-09-20T22:14:01.267427+0000","last_became_active":"2024-09-20T22:12:52.517432+0000","last_became_peered":"2024-09-20T22:12:52.517432+0000","last_unstale":"2024-09-20T22:14:01.267427+0000","last_undegraded":"2024-09-20T22:14:01.267427+0000","last_fullsized":"2024-09-20T22:14:01.267427+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:41.549376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:13:41.549376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:49.137072+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":1315,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:04.635709+0000","last_change":"2024-09-20T22:13:06.609600+0000","last_active":"2024-09-20T22:14:04.635709+0000","last_peered":"2024-09-20T22:14:04.635709+0000","last_clean":"2024-09-20T22:14:04.635709+0000","last_became_active":"2024-09-20T22:12:52.526622+0000","last_became_peered":"2024-09-20T22:12:52.526622+0000","last_unstale":"2024-09-20T22:14:04.635709+0000","last_undegraded":"2024-09-20T22:14:04.635709+0000","last_fullsized":"2024-09-20T22:14:04.635709+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:13:06.609525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:13:06.609525+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:33.606536+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":652,"num_read_kb":647,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":162,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.312587+0000","last_change":"2024-09-20T22:12:52.519217+0000","last_active":"2024-09-20T22:14:01.312587+0000","last_peered":"2024-09-20T22:14:01.312587+0000","last_clean":"2024-09-20T22:14:01.312587+0000","last_became_active":"2024-09-20T22:12:52.518823+0000","last_became_peered":"2024-09-20T22:12:52.518823+0000","last_unstale":"2024-09-20T22:14:01.312587+0000","last_undegraded":"2024-09-20T22:14:01.312587+0000","last_fullsized":"2024-09-20T22:14:01.312587+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:39.170979+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:39.170979+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":453,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.265477+0000","last_change":"2024-09-20T22:13:40.433856+0000","last_active":"2024-09-20T22:14:01.265477+0000","last_peered":"2024-09-20T22:14:01.265477+0000","last_clean":"2024-09-20T22:14:01.265477+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:14:01.265477+0000","last_undegraded":"2024-09-20T22:14:01.265477+0000","last_fullsized":"2024-09-20T22:14:01.265477+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:40.433606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.415468+0000","last_clean_scrub_stamp":"2024-09-20T22:13:40.433606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:55.124968+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"348'10045","reported_seq":10245,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.838345+0000","last_change":"2024-09-20T22:13:49.901852+0000","last_active":"2024-09-20T22:14:01.838345+0000","last_peered":"2024-09-20T22:14:01.838345+0000","last_clean":"2024-09-20T22:14:01.838345+0000","last_became_active":"2024-09-20T22:12:53.511966+0000","last_became_peered":"2024-09-20T22:12:53.511966+0000","last_unstale":"2024-09-20T22:14:01.838345+0000","last_undegraded":"2024-09-20T22:14:01.838345+0000","last_fullsized":"2024-09-20T22:14:01.838345+0000","mapping_epoch":324,"log_start":"311'4358","ondisk_log_start":"311'4358","created":286,"last_epoch_clean":325,"parent":"0.0","parent_split_bits":0,"last_scrub":"345'9333","last_scrub_stamp":"2024-09-20T22:13:49.901749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:13:49.901749+0000","objects_scrubbed":4713,"log_size":5687,"log_dups_size":0,"ondisk_log_size":5687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:04.124141+0000","scrub_duration":1462,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":259981312,"num_objects":3967,"num_object_clones":0,"num_object_copies":11901,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10045,"num_write_kb":448384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"328'28","reported_seq":794,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.257872+0000","last_change":"2024-09-20T22:12:05.332528+0000","last_active":"2024-09-20T22:14:01.257872+0000","last_peered":"2024-09-20T22:14:01.257872+0000","last_clean":"2024-09-20T22:14:01.257872+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:14:01.257872+0000","last_undegraded":"2024-09-20T22:14:01.257872+0000","last_fullsized":"2024-09-20T22:14:01.257872+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'16","last_scrub_stamp":"2024-09-20T22:12:05.332362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:05.332362+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":575,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.268574+0000","last_change":"2024-09-20T22:12:42.260316+0000","last_active":"2024-09-20T22:14:01.268574+0000","last_peered":"2024-09-20T22:14:01.268574+0000","last_clean":"2024-09-20T22:14:01.268574+0000","last_became_active":"2024-09-20T22:12:21.519717+0000","last_became_peered":"2024-09-20T22:12:21.519717+0000","last_unstale":"2024-09-20T22:14:01.268574+0000","last_undegraded":"2024-09-20T22:14:01.268574+0000","last_fullsized":"2024-09-20T22:14:01.268574+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:42.260197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:42.260197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:09.162390+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":625,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.272418+0000","last_change":"2024-09-20T22:12:55.522906+0000","last_active":"2024-09-20T22:14:01.272418+0000","last_peered":"2024-09-20T22:14:01.272418+0000","last_clean":"2024-09-20T22:14:01.272418+0000","last_became_active":"2024-09-20T22:12:55.522325+0000","last_became_peered":"2024-09-20T22:12:55.522325+0000","last_unstale":"2024-09-20T22:14:01.272418+0000","last_undegraded":"2024-09-20T22:14:01.272418+0000","last_fullsized":"2024-09-20T22:14:01.272418+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:12:40.635888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:12:40.635888+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.852634+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":198,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.273765+0000","last_change":"2024-09-20T22:13:51.675128+0000","last_active":"2024-09-20T22:14:01.273765+0000","last_peered":"2024-09-20T22:14:01.273765+0000","last_clean":"2024-09-20T22:14:01.273765+0000","last_became_active":"2024-09-20T22:10:48.907271+0000","last_became_peered":"2024-09-20T22:10:48.907271+0000","last_unstale":"2024-09-20T22:14:01.273765+0000","last_undegraded":"2024-09-20T22:14:01.273765+0000","last_fullsized":"2024-09-20T22:14:01.273765+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.674976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:02.287074+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":631,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.312444+0000","last_change":"2024-09-20T22:13:02.482655+0000","last_active":"2024-09-20T22:14:01.312444+0000","last_peered":"2024-09-20T22:14:01.312444+0000","last_clean":"2024-09-20T22:14:01.312444+0000","last_became_active":"2024-09-20T22:12:44.630044+0000","last_became_peered":"2024-09-20T22:12:44.630044+0000","last_unstale":"2024-09-20T22:14:01.312444+0000","last_undegraded":"2024-09-20T22:14:01.312444+0000","last_fullsized":"2024-09-20T22:14:01.312444+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:02.482546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:13:02.482546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:19.933347+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"348'10251","reported_seq":10445,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:05.757296+0000","last_change":"2024-09-20T22:13:44.166688+0000","last_active":"2024-09-20T22:14:05.757296+0000","last_peered":"2024-09-20T22:14:05.757296+0000","last_clean":"2024-09-20T22:14:05.757296+0000","last_became_active":"2024-09-20T22:11:34.559353+0000","last_became_peered":"2024-09-20T22:11:34.559353+0000","last_unstale":"2024-09-20T22:14:05.757296+0000","last_undegraded":"2024-09-20T22:14:05.757296+0000","last_fullsized":"2024-09-20T22:14:05.757296+0000","mapping_epoch":286,"log_start":"329'5771","ondisk_log_start":"329'5771","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"345'8915","last_scrub_stamp":"2024-09-20T22:13:44.166549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:13:44.166549+0000","objects_scrubbed":5006,"log_size":4480,"log_dups_size":0,"ondisk_log_size":4480,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:55.327841+0000","scrub_duration":1425,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237568000,"num_objects":3625,"num_object_clones":0,"num_object_copies":10875,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10251,"num_write_kb":444032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"328'19","reported_seq":643,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.276867+0000","last_change":"2024-09-20T22:12:47.150182+0000","last_active":"2024-09-20T22:14:01.276867+0000","last_peered":"2024-09-20T22:14:01.276867+0000","last_clean":"2024-09-20T22:14:01.276867+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:14:01.276867+0000","last_undegraded":"2024-09-20T22:14:01.276867+0000","last_fullsized":"2024-09-20T22:14:01.276867+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:12:47.150099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:47.150099+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:01.350054+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":616,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.275814+0000","last_change":"2024-09-20T22:13:39.055297+0000","last_active":"2024-09-20T22:14:01.275814+0000","last_peered":"2024-09-20T22:14:01.275814+0000","last_clean":"2024-09-20T22:14:01.275814+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:14:01.275814+0000","last_undegraded":"2024-09-20T22:14:01.275814+0000","last_fullsized":"2024-09-20T22:14:01.275814+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:39.055192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:39.055192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:54.441568+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":614,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.312185+0000","last_change":"2024-09-20T22:12:13.155581+0000","last_active":"2024-09-20T22:14:01.312185+0000","last_peered":"2024-09-20T22:14:01.312185+0000","last_clean":"2024-09-20T22:14:01.312185+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:14:01.312185+0000","last_undegraded":"2024-09-20T22:14:01.312185+0000","last_fullsized":"2024-09-20T22:14:01.312185+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:13.155346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:13.155346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":562,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.258452+0000","last_change":"2024-09-20T22:12:40.548181+0000","last_active":"2024-09-20T22:14:01.258452+0000","last_peered":"2024-09-20T22:14:01.258452+0000","last_clean":"2024-09-20T22:14:01.258452+0000","last_became_active":"2024-09-20T22:12:21.521195+0000","last_became_peered":"2024-09-20T22:12:21.521195+0000","last_unstale":"2024-09-20T22:14:01.258452+0000","last_undegraded":"2024-09-20T22:14:01.258452+0000","last_fullsized":"2024-09-20T22:14:01.258452+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:40.548065+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:40.548065+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:06.497433+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.0","version":"348'10063","reported_seq":10255,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:03.275085+0000","last_change":"2024-09-20T22:13:44.909535+0000","last_active":"2024-09-20T22:14:03.275085+0000","last_peered":"2024-09-20T22:14:03.275085+0000","last_clean":"2024-09-20T22:14:03.275085+0000","last_became_active":"2024-09-20T22:11:35.304302+0000","last_became_peered":"2024-09-20T22:11:35.304302+0000","last_unstale":"2024-09-20T22:14:03.275085+0000","last_undegraded":"2024-09-20T22:14:03.275085+0000","last_fullsized":"2024-09-20T22:14:03.275085+0000","mapping_epoch":286,"log_start":"317'5427","ondisk_log_start":"317'5427","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"345'8938","last_scrub_stamp":"2024-09-20T22:13:44.909161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:13:44.909161+0000","objects_scrubbed":4947,"log_size":4636,"log_dups_size":0,"ondisk_log_size":4636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:48.876107+0000","scrub_duration":1473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":248053760,"num_objects":3785,"num_object_clones":0,"num_object_copies":11355,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10063,"num_write_kb":443136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":521,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.268231+0000","last_change":"2024-09-20T22:12:15.134763+0000","last_active":"2024-09-20T22:14:01.268231+0000","last_peered":"2024-09-20T22:14:01.268231+0000","last_clean":"2024-09-20T22:14:01.268231+0000","last_became_active":"2024-09-20T22:11:33.557208+0000","last_became_peered":"2024-09-20T22:11:33.557208+0000","last_unstale":"2024-09-20T22:14:01.268231+0000","last_undegraded":"2024-09-20T22:14:01.268231+0000","last_fullsized":"2024-09-20T22:14:01.268231+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:12:15.134639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:12:15.134639+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"328'31","reported_seq":669,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.276849+0000","last_change":"2024-09-20T22:13:11.188414+0000","last_active":"2024-09-20T22:14:01.276849+0000","last_peered":"2024-09-20T22:14:01.276849+0000","last_clean":"2024-09-20T22:14:01.276849+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:14:01.276849+0000","last_undegraded":"2024-09-20T22:14:01.276849+0000","last_fullsized":"2024-09-20T22:14:01.276849+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'31","last_scrub_stamp":"2024-09-20T22:13:11.188264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:11.188264+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:28.710359+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":624,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.257648+0000","last_change":"2024-09-20T22:12:06.305854+0000","last_active":"2024-09-20T22:14:01.257648+0000","last_peered":"2024-09-20T22:14:01.257648+0000","last_clean":"2024-09-20T22:14:01.257648+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:14:01.257648+0000","last_undegraded":"2024-09-20T22:14:01.257648+0000","last_fullsized":"2024-09-20T22:14:01.257648+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:06.305736+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:06.305736+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":465,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.265561+0000","last_change":"2024-09-20T22:13:10.039183+0000","last_active":"2024-09-20T22:14:01.265561+0000","last_peered":"2024-09-20T22:14:01.265561+0000","last_clean":"2024-09-20T22:14:01.265561+0000","last_became_active":"2024-09-20T22:12:52.520612+0000","last_became_peered":"2024-09-20T22:12:52.520612+0000","last_unstale":"2024-09-20T22:14:01.265561+0000","last_undegraded":"2024-09-20T22:14:01.265561+0000","last_fullsized":"2024-09-20T22:14:01.265561+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:10.039074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.039074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:27.582640+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":616,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-20T22:13:55.698948+0000","last_change":"2024-09-20T22:13:01.042002+0000","last_active":"2024-09-20T22:13:55.698948+0000","last_peered":"2024-09-20T22:13:55.698948+0000","last_clean":"2024-09-20T22:13:55.698948+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:13:55.698948+0000","last_undegraded":"2024-09-20T22:13:55.698948+0000","last_fullsized":"2024-09-20T22:13:55.698948+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:01.041863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:01.041863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:14.980142+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"348'9952","reported_seq":10144,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:02.836950+0000","last_change":"2024-09-20T22:13:51.395292+0000","last_active":"2024-09-20T22:14:02.836950+0000","last_peered":"2024-09-20T22:14:02.836950+0000","last_clean":"2024-09-20T22:14:02.836950+0000","last_became_active":"2024-09-20T22:12:21.522567+0000","last_became_peered":"2024-09-20T22:12:21.522567+0000","last_unstale":"2024-09-20T22:14:02.836950+0000","last_undegraded":"2024-09-20T22:14:02.836950+0000","last_fullsized":"2024-09-20T22:14:02.836950+0000","mapping_epoch":309,"log_start":"310'4176","ondisk_log_start":"310'4176","created":286,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"345'9225","last_scrub_stamp":"2024-09-20T22:13:51.395052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.395052+0000","objects_scrubbed":4519,"log_size":5776,"log_dups_size":0,"ondisk_log_size":5776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:53.188292+0000","scrub_duration":1025,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":246546432,"num_objects":3762,"num_object_clones":0,"num_object_copies":11286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9952,"num_write_kb":438848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":895,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.273641+0000","last_change":"2024-09-20T22:13:49.912616+0000","last_active":"2024-09-20T22:14:01.273641+0000","last_peered":"2024-09-20T22:14:01.273641+0000","last_clean":"2024-09-20T22:14:01.273641+0000","last_became_active":"2024-09-20T22:08:48.455288+0000","last_became_peered":"2024-09-20T22:08:48.455288+0000","last_unstale":"2024-09-20T22:14:01.273641+0000","last_undegraded":"2024-09-20T22:14:01.273641+0000","last_fullsized":"2024-09-20T22:14:01.273641+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:49.912496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:13:49.912496+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:11.598929+0000","scrub_duration":227,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"328'25","reported_seq":638,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.276505+0000","last_change":"2024-09-20T22:11:37.277758+0000","last_active":"2024-09-20T22:14:01.276505+0000","last_peered":"2024-09-20T22:14:01.276505+0000","last_clean":"2024-09-20T22:14:01.276505+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:14:01.276505+0000","last_undegraded":"2024-09-20T22:14:01.276505+0000","last_fullsized":"2024-09-20T22:14:01.276505+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'15","last_scrub_stamp":"2024-09-20T22:11:37.277648+0000","last_deep_scrub":"289'15","last_deep_scrub_stamp":"2024-09-20T22:11:37.277648+0000","last_clean_scrub_stamp":"2024-09-20T22:11:37.277648+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.099508+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":63,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.268896+0000","last_change":"2024-09-20T22:12:52.547776+0000","last_active":"2024-09-20T22:14:01.268896+0000","last_peered":"2024-09-20T22:14:01.268896+0000","last_clean":"2024-09-20T22:14:01.268896+0000","last_became_active":"2024-09-20T22:12:52.528310+0000","last_became_peered":"2024-09-20T22:12:52.528310+0000","last_unstale":"2024-09-20T22:14:01.268896+0000","last_undegraded":"2024-09-20T22:14:01.268896+0000","last_fullsized":"2024-09-20T22:14:01.268896+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:12:08.663015+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:12:08.663015+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"283'11","reported_seq":792,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-20T22:13:55.697326+0000","last_change":"2024-09-20T22:12:44.208085+0000","last_active":"2024-09-20T22:13:55.697326+0000","last_peered":"2024-09-20T22:13:55.697326+0000","last_clean":"2024-09-20T22:13:55.697326+0000","last_became_active":"2024-09-20T22:11:30.485356+0000","last_became_peered":"2024-09-20T22:11:30.485356+0000","last_unstale":"2024-09-20T22:13:55.697326+0000","last_undegraded":"2024-09-20T22:13:55.697326+0000","last_fullsized":"2024-09-20T22:13:55.697326+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'11","last_scrub_stamp":"2024-09-20T22:12:44.208006+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:12:44.208006+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:13.803569+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":604,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.276646+0000","last_change":"2024-09-20T22:12:11.210404+0000","last_active":"2024-09-20T22:14:01.276646+0000","last_peered":"2024-09-20T22:14:01.276646+0000","last_clean":"2024-09-20T22:14:01.276646+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:14:01.276646+0000","last_undegraded":"2024-09-20T22:14:01.276646+0000","last_fullsized":"2024-09-20T22:14:01.276646+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:11.210279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:12:11.210279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.099669+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"348'10458","reported_seq":10593,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:06.034281+0000","last_change":"2024-09-20T22:11:35.304534+0000","last_active":"2024-09-20T22:14:06.034281+0000","last_peered":"2024-09-20T22:14:06.034281+0000","last_clean":"2024-09-20T22:14:06.034281+0000","last_became_active":"2024-09-20T22:11:35.304192+0000","last_became_peered":"2024-09-20T22:11:35.304192+0000","last_unstale":"2024-09-20T22:14:06.034281+0000","last_undegraded":"2024-09-20T22:14:06.034281+0000","last_fullsized":"2024-09-20T22:14:06.034281+0000","mapping_epoch":286,"log_start":"317'5515","ondisk_log_start":"317'5515","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":0,"log_size":4943,"log_dups_size":0,"ondisk_log_size":4943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:07.511750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":238157824,"num_objects":3634,"num_object_clones":0,"num_object_copies":10902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10458,"num_write_kb":450944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"328'15","reported_seq":86,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269790+0000","last_change":"2024-09-20T22:13:46.474322+0000","last_active":"2024-09-20T22:14:01.269790+0000","last_peered":"2024-09-20T22:14:01.269790+0000","last_clean":"2024-09-20T22:14:01.269790+0000","last_became_active":"2024-09-20T22:12:52.527531+0000","last_became_peered":"2024-09-20T22:12:52.527531+0000","last_unstale":"2024-09-20T22:14:01.269790+0000","last_undegraded":"2024-09-20T22:14:01.269790+0000","last_fullsized":"2024-09-20T22:14:01.269790+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:13:46.473818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:46.473818+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:59.832835+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":243,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.278787+0000","last_change":"2024-09-20T22:13:01.819308+0000","last_active":"2024-09-20T22:14:01.278787+0000","last_peered":"2024-09-20T22:14:01.278787+0000","last_clean":"2024-09-20T22:14:01.278787+0000","last_became_active":"2024-09-20T22:10:14.536155+0000","last_became_peered":"2024-09-20T22:10:14.536155+0000","last_unstale":"2024-09-20T22:14:01.278787+0000","last_undegraded":"2024-09-20T22:14:01.278787+0000","last_fullsized":"2024-09-20T22:14:01.278787+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:13:01.818638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:01.818638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:16.256044+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":629,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-20T22:13:55.696923+0000","last_change":"2024-09-20T22:13:43.106240+0000","last_active":"2024-09-20T22:13:55.696923+0000","last_peered":"2024-09-20T22:13:55.696923+0000","last_clean":"2024-09-20T22:13:55.696923+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:13:55.696923+0000","last_undegraded":"2024-09-20T22:13:55.696923+0000","last_fullsized":"2024-09-20T22:13:55.696923+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:43.106113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:13:43.106113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:00.182815+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":59,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.267448+0000","last_change":"2024-09-20T22:12:52.516597+0000","last_active":"2024-09-20T22:14:01.267448+0000","last_peered":"2024-09-20T22:14:01.267448+0000","last_clean":"2024-09-20T22:14:01.267448+0000","last_became_active":"2024-09-20T22:12:52.512924+0000","last_became_peered":"2024-09-20T22:12:52.512924+0000","last_unstale":"2024-09-20T22:14:01.267448+0000","last_undegraded":"2024-09-20T22:14:01.267448+0000","last_fullsized":"2024-09-20T22:14:01.267448+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:00.473894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:00.473894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"348'10155","reported_seq":10305,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:06.050553+0000","last_change":"2024-09-20T22:13:53.807609+0000","last_active":"2024-09-20T22:14:06.050553+0000","last_peered":"2024-09-20T22:14:06.050553+0000","last_clean":"2024-09-20T22:14:06.050553+0000","last_became_active":"2024-09-20T22:11:40.601030+0000","last_became_peered":"2024-09-20T22:11:40.601030+0000","last_unstale":"2024-09-20T22:14:06.050553+0000","last_undegraded":"2024-09-20T22:14:06.050553+0000","last_fullsized":"2024-09-20T22:14:06.050553+0000","mapping_epoch":290,"log_start":"317'5178","ondisk_log_start":"317'5178","created":286,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":424,"log_size":4977,"log_dups_size":0,"ondisk_log_size":4977,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:09.492012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":237830144,"num_objects":3629,"num_object_clones":0,"num_object_copies":10887,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10155,"num_write_kb":441088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"328'35","reported_seq":727,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.257288+0000","last_change":"2024-09-20T22:13:51.408635+0000","last_active":"2024-09-20T22:14:01.257288+0000","last_peered":"2024-09-20T22:14:01.257288+0000","last_clean":"2024-09-20T22:14:01.257288+0000","last_became_active":"2024-09-20T22:10:30.652967+0000","last_became_peered":"2024-09-20T22:10:30.652967+0000","last_unstale":"2024-09-20T22:14:01.257288+0000","last_undegraded":"2024-09-20T22:14:01.257288+0000","last_fullsized":"2024-09-20T22:14:01.257288+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'35","last_scrub_stamp":"2024-09-20T22:13:51.408543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.408543+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:14.979728+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":628,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.273474+0000","last_change":"2024-09-20T22:12:46.602166+0000","last_active":"2024-09-20T22:14:01.273474+0000","last_peered":"2024-09-20T22:14:01.273474+0000","last_clean":"2024-09-20T22:14:01.273474+0000","last_became_active":"2024-09-20T22:11:30.488650+0000","last_became_peered":"2024-09-20T22:11:30.488650+0000","last_unstale":"2024-09-20T22:14:01.273474+0000","last_undegraded":"2024-09-20T22:14:01.273474+0000","last_fullsized":"2024-09-20T22:14:01.273474+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:46.602035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:12:46.602035+0000","last_clean_scrub_stamp":"2024-09-20T22:12:46.602035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.852688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":652,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-20T22:13:55.696618+0000","last_change":"2024-09-20T22:13:49.085883+0000","last_active":"2024-09-20T22:13:55.696618+0000","last_peered":"2024-09-20T22:13:55.696618+0000","last_clean":"2024-09-20T22:13:55.696618+0000","last_became_active":"2024-09-20T22:12:52.515295+0000","last_became_peered":"2024-09-20T22:12:52.515295+0000","last_unstale":"2024-09-20T22:13:55.696618+0000","last_undegraded":"2024-09-20T22:13:55.696618+0000","last_fullsized":"2024-09-20T22:13:55.696618+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:49.085768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_clean_scrub_stamp":"2024-09-20T22:13:49.085768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:51.925476+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":212,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.272482+0000","last_change":"2024-09-20T22:12:12.517840+0000","last_active":"2024-09-20T22:14:01.272482+0000","last_peered":"2024-09-20T22:14:01.272482+0000","last_clean":"2024-09-20T22:14:01.272482+0000","last_became_active":"2024-09-20T22:10:30.657116+0000","last_became_peered":"2024-09-20T22:10:30.657116+0000","last_unstale":"2024-09-20T22:14:01.272482+0000","last_undegraded":"2024-09-20T22:14:01.272482+0000","last_fullsized":"2024-09-20T22:14:01.272482+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:12.517732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:12.517732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.852521+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"348'9950","reported_seq":10145,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:03.275640+0000","last_change":"2024-09-20T22:12:41.097772+0000","last_active":"2024-09-20T22:14:03.275640+0000","last_peered":"2024-09-20T22:14:03.275640+0000","last_clean":"2024-09-20T22:14:03.275640+0000","last_became_active":"2024-09-20T22:11:34.557976+0000","last_became_peered":"2024-09-20T22:11:34.557976+0000","last_unstale":"2024-09-20T22:14:03.275640+0000","last_undegraded":"2024-09-20T22:14:03.275640+0000","last_fullsized":"2024-09-20T22:14:03.275640+0000","mapping_epoch":286,"log_start":"317'5324","ondisk_log_start":"317'5324","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"317'5305","last_scrub_stamp":"2024-09-20T22:12:41.097557+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:12:41.097557+0000","objects_scrubbed":5232,"log_size":4626,"log_dups_size":0,"ondisk_log_size":4626,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:03.532257+0000","scrub_duration":2075,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":242876416,"num_objects":3706,"num_object_clones":0,"num_object_copies":11118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3706,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9950,"num_write_kb":436992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"328'18","reported_seq":645,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.275579+0000","last_change":"2024-09-20T22:13:10.164964+0000","last_active":"2024-09-20T22:14:01.275579+0000","last_peered":"2024-09-20T22:14:01.275579+0000","last_clean":"2024-09-20T22:14:01.275579+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:14:01.275579+0000","last_undegraded":"2024-09-20T22:14:01.275579+0000","last_fullsized":"2024-09-20T22:14:01.275579+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'18","last_scrub_stamp":"2024-09-20T22:13:10.164833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.164833+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:14.536117+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":617,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.273533+0000","last_change":"2024-09-20T22:12:15.557285+0000","last_active":"2024-09-20T22:14:01.273533+0000","last_peered":"2024-09-20T22:14:01.273533+0000","last_clean":"2024-09-20T22:14:01.273533+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:14:01.273533+0000","last_undegraded":"2024-09-20T22:14:01.273533+0000","last_fullsized":"2024-09-20T22:14:01.273533+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:15.557184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:15.557184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.852468+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":650,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-20T22:13:55.696367+0000","last_change":"2024-09-20T22:13:53.202635+0000","last_active":"2024-09-20T22:13:55.696367+0000","last_peered":"2024-09-20T22:13:55.696367+0000","last_clean":"2024-09-20T22:13:55.696367+0000","last_became_active":"2024-09-20T22:12:52.514202+0000","last_became_peered":"2024-09-20T22:12:52.514202+0000","last_unstale":"2024-09-20T22:13:55.696367+0000","last_undegraded":"2024-09-20T22:13:55.696367+0000","last_fullsized":"2024-09-20T22:13:55.696367+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:53.202518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:53.202518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:12.953838+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":77,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.277990+0000","last_change":"2024-09-20T22:13:42.822386+0000","last_active":"2024-09-20T22:14:01.277990+0000","last_peered":"2024-09-20T22:14:01.277990+0000","last_clean":"2024-09-20T22:14:01.277990+0000","last_became_active":"2024-09-20T22:12:44.638546+0000","last_became_peered":"2024-09-20T22:12:44.638546+0000","last_unstale":"2024-09-20T22:14:01.277990+0000","last_undegraded":"2024-09-20T22:14:01.277990+0000","last_fullsized":"2024-09-20T22:14:01.277990+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:42.822274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:42.822274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:59.413720+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"348'9770","reported_seq":10191,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.830408+0000","last_change":"2024-09-20T22:13:51.600919+0000","last_active":"2024-09-20T22:14:01.830408+0000","last_peered":"2024-09-20T22:14:01.830408+0000","last_clean":"2024-09-20T22:14:01.830408+0000","last_became_active":"2024-09-20T22:12:53.511729+0000","last_became_peered":"2024-09-20T22:12:53.511729+0000","last_unstale":"2024-09-20T22:14:01.830408+0000","last_undegraded":"2024-09-20T22:14:01.830408+0000","last_fullsized":"2024-09-20T22:14:01.830408+0000","mapping_epoch":324,"log_start":"310'4056","ondisk_log_start":"310'4056","created":286,"last_epoch_clean":325,"parent":"0.0","parent_split_bits":0,"last_scrub":"345'9142","last_scrub_stamp":"2024-09-20T22:13:51.600650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.600650+0000","objects_scrubbed":4571,"log_size":5714,"log_dups_size":0,"ondisk_log_size":5714,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:54.668893+0000","scrub_duration":1129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":256770048,"num_objects":3918,"num_object_clones":0,"num_object_copies":11754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9770,"num_write_kb":438016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":7340032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"328'27","reported_seq":184,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.268118+0000","last_change":"2024-09-20T22:13:10.353607+0000","last_active":"2024-09-20T22:14:01.268118+0000","last_peered":"2024-09-20T22:14:01.268118+0000","last_clean":"2024-09-20T22:14:01.268118+0000","last_became_active":"2024-09-20T22:11:30.493275+0000","last_became_peered":"2024-09-20T22:11:30.493275+0000","last_unstale":"2024-09-20T22:14:01.268118+0000","last_undegraded":"2024-09-20T22:14:01.268118+0000","last_fullsized":"2024-09-20T22:14:01.268118+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:13:10.353459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.353459+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:33.012468+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":242,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.273241+0000","last_change":"2024-09-20T22:12:52.541194+0000","last_active":"2024-09-20T22:14:01.273241+0000","last_peered":"2024-09-20T22:14:01.273241+0000","last_clean":"2024-09-20T22:14:01.273241+0000","last_became_active":"2024-09-20T22:12:52.526551+0000","last_became_peered":"2024-09-20T22:12:52.526551+0000","last_unstale":"2024-09-20T22:14:01.273241+0000","last_undegraded":"2024-09-20T22:14:01.273241+0000","last_fullsized":"2024-09-20T22:14:01.273241+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:12:37.642296+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:12:37.642296+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.852603+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":744,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.273451+0000","last_change":"2024-09-20T22:12:04.528478+0000","last_active":"2024-09-20T22:14:01.273451+0000","last_peered":"2024-09-20T22:14:01.273451+0000","last_clean":"2024-09-20T22:14:01.273451+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:14:01.273451+0000","last_undegraded":"2024-09-20T22:14:01.273451+0000","last_fullsized":"2024-09-20T22:14:01.273451+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:12:04.528253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:04.528253+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.852441+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":162,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269232+0000","last_change":"2024-09-20T22:13:00.454973+0000","last_active":"2024-09-20T22:14:01.269232+0000","last_peered":"2024-09-20T22:14:01.269232+0000","last_clean":"2024-09-20T22:14:01.269232+0000","last_became_active":"2024-09-20T22:12:52.517758+0000","last_became_peered":"2024-09-20T22:12:52.517758+0000","last_unstale":"2024-09-20T22:14:01.269232+0000","last_undegraded":"2024-09-20T22:14:01.269232+0000","last_fullsized":"2024-09-20T22:14:01.269232+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:00.454378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:00.454378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":654,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.257375+0000","last_change":"2024-09-20T22:13:04.905929+0000","last_active":"2024-09-20T22:14:01.257375+0000","last_peered":"2024-09-20T22:14:01.257375+0000","last_clean":"2024-09-20T22:14:01.257375+0000","last_became_active":"2024-09-20T22:12:52.516989+0000","last_became_peered":"2024-09-20T22:12:52.516989+0000","last_unstale":"2024-09-20T22:14:01.257375+0000","last_undegraded":"2024-09-20T22:14:01.257375+0000","last_fullsized":"2024-09-20T22:14:01.257375+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:04.905801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:04.905801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:08.217862+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"328'12","reported_seq":656,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.287308+0000","last_change":"2024-09-20T22:13:10.726072+0000","last_active":"2024-09-20T22:14:01.287308+0000","last_peered":"2024-09-20T22:14:01.287308+0000","last_clean":"2024-09-20T22:14:01.287308+0000","last_became_active":"2024-09-20T22:10:48.922561+0000","last_became_peered":"2024-09-20T22:10:48.922561+0000","last_unstale":"2024-09-20T22:14:01.287308+0000","last_undegraded":"2024-09-20T22:14:01.287308+0000","last_fullsized":"2024-09-20T22:14:01.287308+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'12","last_scrub_stamp":"2024-09-20T22:13:10.725057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.725057+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:36.040129+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":150,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269071+0000","last_change":"2024-09-20T22:12:14.148857+0000","last_active":"2024-09-20T22:14:01.269071+0000","last_peered":"2024-09-20T22:14:01.269071+0000","last_clean":"2024-09-20T22:14:01.269071+0000","last_became_active":"2024-09-20T22:11:30.487322+0000","last_became_peered":"2024-09-20T22:11:30.487322+0000","last_unstale":"2024-09-20T22:14:01.269071+0000","last_undegraded":"2024-09-20T22:14:01.269071+0000","last_fullsized":"2024-09-20T22:14:01.269071+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:14.148743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:14.148743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":595,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.273138+0000","last_change":"2024-09-20T22:12:05.541834+0000","last_active":"2024-09-20T22:14:01.273138+0000","last_peered":"2024-09-20T22:14:01.273138+0000","last_clean":"2024-09-20T22:14:01.273138+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:14:01.273138+0000","last_undegraded":"2024-09-20T22:14:01.273138+0000","last_fullsized":"2024-09-20T22:14:01.273138+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:05.541367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:05.541367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.852547+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"328'15","reported_seq":78,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269798+0000","last_change":"2024-09-20T22:12:52.541421+0000","last_active":"2024-09-20T22:14:01.269798+0000","last_peered":"2024-09-20T22:14:01.269798+0000","last_clean":"2024-09-20T22:14:01.269798+0000","last_became_active":"2024-09-20T22:12:52.527106+0000","last_became_peered":"2024-09-20T22:12:52.527106+0000","last_unstale":"2024-09-20T22:14:01.269798+0000","last_undegraded":"2024-09-20T22:14:01.269798+0000","last_fullsized":"2024-09-20T22:14:01.269798+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:12:08.497572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:08.497572+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":615,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.273740+0000","last_change":"2024-09-20T22:12:44.676449+0000","last_active":"2024-09-20T22:14:01.273740+0000","last_peered":"2024-09-20T22:14:01.273740+0000","last_clean":"2024-09-20T22:14:01.273740+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:14:01.273740+0000","last_undegraded":"2024-09-20T22:14:01.273740+0000","last_fullsized":"2024-09-20T22:14:01.273740+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:44.676334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:44.676334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.852744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":253,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.276387+0000","last_change":"2024-09-20T22:13:14.136545+0000","last_active":"2024-09-20T22:14:01.276387+0000","last_peered":"2024-09-20T22:14:01.276387+0000","last_clean":"2024-09-20T22:14:01.276387+0000","last_became_active":"2024-09-20T22:12:33.545506+0000","last_became_peered":"2024-09-20T22:12:33.545506+0000","last_unstale":"2024-09-20T22:14:01.276387+0000","last_undegraded":"2024-09-20T22:14:01.276387+0000","last_fullsized":"2024-09-20T22:14:01.276387+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:14.136472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_clean_scrub_stamp":"2024-09-20T22:13:14.136472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:33.179397+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":624,"reported_epoch":347,"state":"active+clean","last_fresh":"2024-09-20T22:13:55.697013+0000","last_change":"2024-09-20T22:13:52.203910+0000","last_active":"2024-09-20T22:13:55.697013+0000","last_peered":"2024-09-20T22:13:55.697013+0000","last_clean":"2024-09-20T22:13:55.697013+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:13:55.697013+0000","last_undegraded":"2024-09-20T22:13:55.697013+0000","last_fullsized":"2024-09-20T22:13:55.697013+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:52.203809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:52.203809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:58.016851+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"328'15","reported_seq":169,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269829+0000","last_change":"2024-09-20T22:12:43.218553+0000","last_active":"2024-09-20T22:14:01.269829+0000","last_peered":"2024-09-20T22:14:01.269829+0000","last_clean":"2024-09-20T22:14:01.269829+0000","last_became_active":"2024-09-20T22:11:30.493335+0000","last_became_peered":"2024-09-20T22:11:30.493335+0000","last_unstale":"2024-09-20T22:14:01.269829+0000","last_undegraded":"2024-09-20T22:14:01.269829+0000","last_fullsized":"2024-09-20T22:14:01.269829+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:12:43.218249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:43.218249+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":67,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.268937+0000","last_change":"2024-09-20T22:13:01.541472+0000","last_active":"2024-09-20T22:14:01.268937+0000","last_peered":"2024-09-20T22:14:01.268937+0000","last_clean":"2024-09-20T22:14:01.268937+0000","last_became_active":"2024-09-20T22:12:52.524149+0000","last_became_peered":"2024-09-20T22:12:52.524149+0000","last_unstale":"2024-09-20T22:14:01.268937+0000","last_undegraded":"2024-09-20T22:14:01.268937+0000","last_fullsized":"2024-09-20T22:14:01.268937+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:01.541330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:01.541330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:03.722364+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"315'11","reported_seq":122,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.273185+0000","last_change":"2024-09-20T22:12:33.544408+0000","last_active":"2024-09-20T22:14:01.273185+0000","last_peered":"2024-09-20T22:14:01.273185+0000","last_clean":"2024-09-20T22:14:01.273185+0000","last_became_active":"2024-09-20T22:12:33.544098+0000","last_became_peered":"2024-09-20T22:12:33.544098+0000","last_unstale":"2024-09-20T22:14:01.273185+0000","last_undegraded":"2024-09-20T22:14:01.273185+0000","last_fullsized":"2024-09-20T22:14:01.273185+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"276'10","last_scrub_stamp":"2024-09-20T22:12:03.510585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:03.510585+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.852368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":615,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.277171+0000","last_change":"2024-09-20T22:13:48.148947+0000","last_active":"2024-09-20T22:14:01.277171+0000","last_peered":"2024-09-20T22:14:01.277171+0000","last_clean":"2024-09-20T22:14:01.277171+0000","last_became_active":"2024-09-20T22:11:30.484731+0000","last_became_peered":"2024-09-20T22:11:30.484731+0000","last_unstale":"2024-09-20T22:14:01.277171+0000","last_undegraded":"2024-09-20T22:14:01.277171+0000","last_fullsized":"2024-09-20T22:14:01.277171+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:48.148819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:48.148819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:05.881048+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"328'20","reported_seq":104,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.264015+0000","last_change":"2024-09-20T22:13:49.905238+0000","last_active":"2024-09-20T22:14:01.264015+0000","last_peered":"2024-09-20T22:14:01.264015+0000","last_clean":"2024-09-20T22:14:01.264015+0000","last_became_active":"2024-09-20T22:12:44.631986+0000","last_became_peered":"2024-09-20T22:12:44.631986+0000","last_unstale":"2024-09-20T22:14:01.264015+0000","last_undegraded":"2024-09-20T22:14:01.264015+0000","last_fullsized":"2024-09-20T22:14:01.264015+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:13:49.905115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:49.905115+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:14.205691+0000","scrub_duration":405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":607,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.273758+0000","last_change":"2024-09-20T22:12:08.477834+0000","last_active":"2024-09-20T22:14:01.273758+0000","last_peered":"2024-09-20T22:14:01.273758+0000","last_clean":"2024-09-20T22:14:01.273758+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:14:01.273758+0000","last_undegraded":"2024-09-20T22:14:01.273758+0000","last_fullsized":"2024-09-20T22:14:01.273758+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:08.477741+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:08.477741+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.852493+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"326'14","reported_seq":206,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269609+0000","last_change":"2024-09-20T22:12:52.530641+0000","last_active":"2024-09-20T22:14:01.269609+0000","last_peered":"2024-09-20T22:14:01.269609+0000","last_clean":"2024-09-20T22:14:01.269609+0000","last_became_active":"2024-09-20T22:12:52.518770+0000","last_became_peered":"2024-09-20T22:12:52.518770+0000","last_unstale":"2024-09-20T22:14:01.269609+0000","last_undegraded":"2024-09-20T22:14:01.269609+0000","last_fullsized":"2024-09-20T22:14:01.269609+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'12","last_scrub_stamp":"2024-09-20T22:11:35.062247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:35.062247+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:07.511833+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":67,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.268810+0000","last_change":"2024-09-20T22:13:43.536313+0000","last_active":"2024-09-20T22:14:01.268810+0000","last_peered":"2024-09-20T22:14:01.268810+0000","last_clean":"2024-09-20T22:14:01.268810+0000","last_became_active":"2024-09-20T22:12:52.523781+0000","last_became_peered":"2024-09-20T22:12:52.523781+0000","last_unstale":"2024-09-20T22:14:01.268810+0000","last_undegraded":"2024-09-20T22:14:01.268810+0000","last_fullsized":"2024-09-20T22:14:01.268810+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:43.536210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:43.536210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:51.520380+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"328'19","reported_seq":663,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.276342+0000","last_change":"2024-09-20T22:12:57.881569+0000","last_active":"2024-09-20T22:14:01.276342+0000","last_peered":"2024-09-20T22:14:01.276342+0000","last_clean":"2024-09-20T22:14:01.276342+0000","last_became_active":"2024-09-20T22:08:22.139581+0000","last_became_peered":"2024-09-20T22:08:22.139581+0000","last_unstale":"2024-09-20T22:14:01.276342+0000","last_undegraded":"2024-09-20T22:14:01.276342+0000","last_fullsized":"2024-09-20T22:14:01.276342+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:12:09.181513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:09.181513+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.099762+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":615,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.276633+0000","last_change":"2024-09-20T22:12:40.292205+0000","last_active":"2024-09-20T22:14:01.276633+0000","last_peered":"2024-09-20T22:14:01.276633+0000","last_clean":"2024-09-20T22:14:01.276633+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:14:01.276633+0000","last_undegraded":"2024-09-20T22:14:01.276633+0000","last_fullsized":"2024-09-20T22:14:01.276633+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:40.292095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:40.292095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:04.488576+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":452,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.264859+0000","last_change":"2024-09-20T22:13:52.605374+0000","last_active":"2024-09-20T22:14:01.264859+0000","last_peered":"2024-09-20T22:14:01.264859+0000","last_clean":"2024-09-20T22:14:01.264859+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:14:01.264859+0000","last_undegraded":"2024-09-20T22:14:01.264859+0000","last_fullsized":"2024-09-20T22:14:01.264859+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:52.605277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:52.605277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:04.590848+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":455,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.263174+0000","last_change":"2024-09-20T22:12:52.521964+0000","last_active":"2024-09-20T22:14:01.263174+0000","last_peered":"2024-09-20T22:14:01.263174+0000","last_clean":"2024-09-20T22:14:01.263174+0000","last_became_active":"2024-09-20T22:12:52.521497+0000","last_became_peered":"2024-09-20T22:12:52.521497+0000","last_unstale":"2024-09-20T22:14:01.263174+0000","last_undegraded":"2024-09-20T22:14:01.263174+0000","last_fullsized":"2024-09-20T22:14:01.263174+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:45.053123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:45.053123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"328'19","reported_seq":529,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.265111+0000","last_change":"2024-09-20T22:11:41.362291+0000","last_active":"2024-09-20T22:14:01.265111+0000","last_peered":"2024-09-20T22:14:01.265111+0000","last_clean":"2024-09-20T22:14:01.265111+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:14:01.265111+0000","last_undegraded":"2024-09-20T22:14:01.265111+0000","last_fullsized":"2024-09-20T22:14:01.265111+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'13","last_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_deep_scrub":"291'13","last_deep_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_clean_scrub_stamp":"2024-09-20T22:11:41.362215+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:02.558175+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":212,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.273053+0000","last_change":"2024-09-20T22:13:11.651173+0000","last_active":"2024-09-20T22:14:01.273053+0000","last_peered":"2024-09-20T22:14:01.273053+0000","last_clean":"2024-09-20T22:14:01.273053+0000","last_became_active":"2024-09-20T22:10:30.655502+0000","last_became_peered":"2024-09-20T22:10:30.655502+0000","last_unstale":"2024-09-20T22:14:01.273053+0000","last_undegraded":"2024-09-20T22:14:01.273053+0000","last_fullsized":"2024-09-20T22:14:01.273053+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:11.650985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:11.650985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:31.355370+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":59,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.268580+0000","last_change":"2024-09-20T22:12:52.517342+0000","last_active":"2024-09-20T22:14:01.268580+0000","last_peered":"2024-09-20T22:14:01.268580+0000","last_clean":"2024-09-20T22:14:01.268580+0000","last_became_active":"2024-09-20T22:12:52.516953+0000","last_became_peered":"2024-09-20T22:12:52.516953+0000","last_unstale":"2024-09-20T22:14:01.268580+0000","last_undegraded":"2024-09-20T22:14:01.268580+0000","last_fullsized":"2024-09-20T22:14:01.268580+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:31.262095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:31.262095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":67,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.267729+0000","last_change":"2024-09-20T22:13:07.495192+0000","last_active":"2024-09-20T22:14:01.267729+0000","last_peered":"2024-09-20T22:14:01.267729+0000","last_clean":"2024-09-20T22:14:01.267729+0000","last_became_active":"2024-09-20T22:12:52.514247+0000","last_became_peered":"2024-09-20T22:12:52.514247+0000","last_unstale":"2024-09-20T22:14:01.267729+0000","last_undegraded":"2024-09-20T22:14:01.267729+0000","last_fullsized":"2024-09-20T22:14:01.267729+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:07.495088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:07.495088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:26.816441+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"328'19","reported_seq":540,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.273584+0000","last_change":"2024-09-20T22:13:38.718567+0000","last_active":"2024-09-20T22:14:01.273584+0000","last_peered":"2024-09-20T22:14:01.273584+0000","last_clean":"2024-09-20T22:14:01.273584+0000","last_became_active":"2024-09-20T22:12:33.545199+0000","last_became_peered":"2024-09-20T22:12:33.545199+0000","last_unstale":"2024-09-20T22:14:01.273584+0000","last_undegraded":"2024-09-20T22:14:01.273584+0000","last_fullsized":"2024-09-20T22:14:01.273584+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:13:38.718450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:38.718450+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:07.005555+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":153,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.268991+0000","last_change":"2024-09-20T22:12:05.215296+0000","last_active":"2024-09-20T22:14:01.268991+0000","last_peered":"2024-09-20T22:14:01.268991+0000","last_clean":"2024-09-20T22:14:01.268991+0000","last_became_active":"2024-09-20T22:11:30.493598+0000","last_became_peered":"2024-09-20T22:11:30.493598+0000","last_unstale":"2024-09-20T22:14:01.268991+0000","last_undegraded":"2024-09-20T22:14:01.268991+0000","last_fullsized":"2024-09-20T22:14:01.268991+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:05.215188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_clean_scrub_stamp":"2024-09-20T22:12:05.215188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":607,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.272531+0000","last_change":"2024-09-20T22:12:13.520411+0000","last_active":"2024-09-20T22:14:01.272531+0000","last_peered":"2024-09-20T22:14:01.272531+0000","last_clean":"2024-09-20T22:14:01.272531+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:14:01.272531+0000","last_undegraded":"2024-09-20T22:14:01.272531+0000","last_fullsized":"2024-09-20T22:14:01.272531+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:13.520285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:13.520285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:05.852573+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":56,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269671+0000","last_change":"2024-09-20T22:12:57.884265+0000","last_active":"2024-09-20T22:14:01.269671+0000","last_peered":"2024-09-20T22:14:01.269671+0000","last_clean":"2024-09-20T22:14:01.269671+0000","last_became_active":"2024-09-20T22:12:57.883998+0000","last_became_peered":"2024-09-20T22:12:57.883998+0000","last_unstale":"2024-09-20T22:14:01.269671+0000","last_undegraded":"2024-09-20T22:14:01.269671+0000","last_fullsized":"2024-09-20T22:14:01.269671+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:15.571001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:15.571001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"328'8","reported_seq":623,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.277197+0000","last_change":"2024-09-20T22:12:39.760971+0000","last_active":"2024-09-20T22:14:01.277197+0000","last_peered":"2024-09-20T22:14:01.277197+0000","last_clean":"2024-09-20T22:14:01.277197+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:14:01.277197+0000","last_undegraded":"2024-09-20T22:14:01.277197+0000","last_fullsized":"2024-09-20T22:14:01.277197+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'4","last_scrub_stamp":"2024-09-20T22:12:39.760526+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:39.760526+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:03.523992+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":59,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269532+0000","last_change":"2024-09-20T22:12:52.517833+0000","last_active":"2024-09-20T22:14:01.269532+0000","last_peered":"2024-09-20T22:14:01.269532+0000","last_clean":"2024-09-20T22:14:01.269532+0000","last_became_active":"2024-09-20T22:12:52.517510+0000","last_became_peered":"2024-09-20T22:12:52.517510+0000","last_unstale":"2024-09-20T22:14:01.269532+0000","last_undegraded":"2024-09-20T22:14:01.269532+0000","last_fullsized":"2024-09-20T22:14:01.269532+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:01.443465+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:01.443465+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":630,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.257667+0000","last_change":"2024-09-20T22:13:14.985915+0000","last_active":"2024-09-20T22:14:01.257667+0000","last_peered":"2024-09-20T22:14:01.257667+0000","last_clean":"2024-09-20T22:14:01.257667+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:14:01.257667+0000","last_undegraded":"2024-09-20T22:14:01.257667+0000","last_fullsized":"2024-09-20T22:14:01.257667+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:14.985633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:14.985633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:41.156494+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":603,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.273074+0000","last_change":"2024-09-20T22:13:13.656975+0000","last_active":"2024-09-20T22:14:01.273074+0000","last_peered":"2024-09-20T22:14:01.273074+0000","last_clean":"2024-09-20T22:14:01.273074+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:14:01.273074+0000","last_undegraded":"2024-09-20T22:14:01.273074+0000","last_fullsized":"2024-09-20T22:14:01.273074+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:13.656906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_clean_scrub_stamp":"2024-09-20T22:13:13.656906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:17.664150+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":492,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.267356+0000","last_change":"2024-09-20T22:12:07.349991+0000","last_active":"2024-09-20T22:14:01.267356+0000","last_peered":"2024-09-20T22:14:01.267356+0000","last_clean":"2024-09-20T22:14:01.267356+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:14:01.267356+0000","last_undegraded":"2024-09-20T22:14:01.267356+0000","last_fullsized":"2024-09-20T22:14:01.267356+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:07.349679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:07.349679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:02.558204+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":67,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.268100+0000","last_change":"2024-09-20T22:13:11.547507+0000","last_active":"2024-09-20T22:14:01.268100+0000","last_peered":"2024-09-20T22:14:01.268100+0000","last_clean":"2024-09-20T22:14:01.268100+0000","last_became_active":"2024-09-20T22:12:52.518076+0000","last_became_peered":"2024-09-20T22:12:52.518076+0000","last_unstale":"2024-09-20T22:14:01.268100+0000","last_undegraded":"2024-09-20T22:14:01.268100+0000","last_fullsized":"2024-09-20T22:14:01.268100+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:11.547394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:11.547394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:26.312708+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"328'19","reported_seq":651,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.258727+0000","last_change":"2024-09-20T22:12:03.375105+0000","last_active":"2024-09-20T22:14:01.258727+0000","last_peered":"2024-09-20T22:14:01.258727+0000","last_clean":"2024-09-20T22:14:01.258727+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:14:01.258727+0000","last_undegraded":"2024-09-20T22:14:01.258727+0000","last_fullsized":"2024-09-20T22:14:01.258727+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:12:03.374976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:03.374976+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":59,"reported_epoch":348,"state":"active+clean","last_fresh":"2024-09-20T22:14:01.269159+0000","last_change":"2024-09-20T22:12:52.525557+0000","last_active":"2024-09-20T22:14:01.269159+0000","last_peered":"2024-09-20T22:14:01.269159+0000","last_clean":"2024-09-20T22:14:01.269159+0000","last_became_active":"2024-09-20T22:12:52.524425+0000","last_became_peered":"2024-09-20T22:12:52.524425+0000","last_unstale":"2024-09-20T22:14:01.269159+0000","last_undegraded":"2024-09-20T22:14:01.269159+0000","last_fullsized":"2024-09-20T22:14:01.269159+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:13.583151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:13.583151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":3929276440,"num_objects":59957,"num_object_clones":0,"num_object_copies":179871,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59957,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161655,"num_write_kb":7091521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":7536640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11858227200,"data_stored":11858214984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80635,"ondisk_log_size":80635,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":3300,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":716,"ondisk_log_size":716,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":652,"num_read_kb":647,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":918048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1408,"ondisk_log_size":1408,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813136011,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2696008,"kb_used_data":2127544,"kb_used_omap":1,"kb_used_meta":568446,"kb_avail":91049144,"statfs":{"total":95995035648,"available":93234323456,"internally_reserved":0,"allocated":2178605056,"data_stored":2177852538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":582089162},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,0,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430348,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1143324,"kb_used_data":742624,"kb_used_omap":1,"kb_used_meta":400446,"kb_avail":92601828,"statfs":{"total":95995035648,"available":94824271872,"internally_reserved":0,"allocated":760446976,"data_stored":759718808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":410057158},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724691,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2719288,"kb_used_data":2116136,"kb_used_omap":1,"kb_used_meta":603134,"kb_avail":91025864,"statfs":{"total":95995035648,"available":93210484736,"internally_reserved":0,"allocated":2166923264,"data_stored":2166186935,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":617609669},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,1,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986331,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1138504,"kb_used_data":701944,"kb_used_omap":1,"kb_used_meta":436542,"kb_avail":92606648,"statfs":{"total":95995035648,"available":94829207552,"internally_reserved":0,"allocated":718790656,"data_stored":718037947,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":447019461},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770258007,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2645452,"kb_used_data":2217104,"kb_used_omap":4,"kb_used_meta":428091,"kb_avail":91099700,"statfs":{"total":95995035648,"available":93286092800,"internally_reserved":0,"allocated":2270314496,"data_stored":2269538532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":438365864},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149855,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2645176,"kb_used_data":2040616,"kb_used_omap":1,"kb_used_meta":604542,"kb_avail":91099976,"statfs":{"total":95995035648,"available":93286375424,"internally_reserved":0,"allocated":2089590784,"data_stored":2088716048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":619051463},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411492,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1267084,"kb_used_data":772220,"kb_used_omap":1,"kb_used_meta":494846,"kb_avail":92478068,"statfs":{"total":95995035648,"available":94697541632,"internally_reserved":0,"allocated":790753280,"data_stored":789921635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":506722757},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738543,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1583344,"kb_used_data":1024372,"kb_used_omap":1,"kb_used_meta":558718,"kb_avail":92161808,"statfs":{"total":95995035648,"available":94373691392,"internally_reserved":0,"allocated":1048956928,"data_stored":1048208504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":572127685},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1037828096,"data_stored":1037828096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2238058496,"data_stored":2238054424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":777322496,"data_stored":777322496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2069626880,"data_stored":2069626880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":707723264,"data_stored":707723264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2140147712,"data_stored":2140143640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":750518272,"data_stored":750518272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2137001984,"data_stored":2136997912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:14:07.658 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool set .mgr pgp_num 11 2024-09-20T22:14:07.658 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:07.687 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:14:07.730 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:07.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:07.731 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:07.795 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:14:07.795 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:14:07.795 INFO:teuthology.orchestra.run.smithi067.stdout: "complaint_time": 30, 2024-09-20T22:14:07.795 INFO:teuthology.orchestra.run.smithi067.stdout: "num_blocked_ops": 0 2024-09-20T22:14:07.795 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:14:07.831 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:07.869 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:07.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:07.870 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:07.971 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:08.003 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:08.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:08.004 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:08.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:07 smithi067 ceph-mon[33511]: from='client.25688 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:14:08.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:07 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4288150024' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:14:08.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:07 smithi060 ceph-mon[29694]: from='client.25688 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:14:08.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:07 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4288150024' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:14:08.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:07 smithi060 ceph-mon[40290]: from='client.25688 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:14:08.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:07 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4288150024' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:14:08.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:08.145 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:08.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:08.146 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:08.247 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:08.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:08.282 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:08.283 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:08.384 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:08.420 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:08.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:08.421 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:08.522 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:08.557 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T22:14:08.569 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:08.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:08.570 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:08.642 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:08.671 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:08.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:08.737 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:08.738 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:08 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:14:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:08 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4288150024' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:14:08.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:08 smithi067 ceph-mon[33511]: osdmap e349: 8 total, 8 up, 8 in 2024-09-20T22:14:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:08 smithi067 ceph-mon[33511]: 9.d scrub starts 2024-09-20T22:14:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:08 smithi067 ceph-mon[33511]: pgmap v603: 158 pgs: 158 active+clean; 3.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.22k op/s 2024-09-20T22:14:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:08 smithi067 ceph-mon[33511]: 6.c scrub starts 2024-09-20T22:14:08.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:08 smithi067 ceph-mon[33511]: 6.c scrub ok 2024-09-20T22:14:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:08 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:14:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:08 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4288150024' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:14:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:08 smithi060 ceph-mon[29694]: osdmap e349: 8 total, 8 up, 8 in 2024-09-20T22:14:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:08 smithi060 ceph-mon[29694]: 9.d scrub starts 2024-09-20T22:14:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:08 smithi060 ceph-mon[29694]: pgmap v603: 158 pgs: 158 active+clean; 3.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.22k op/s 2024-09-20T22:14:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:08 smithi060 ceph-mon[29694]: 6.c scrub starts 2024-09-20T22:14:08.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:08 smithi060 ceph-mon[29694]: 6.c scrub ok 2024-09-20T22:14:08.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:08 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:14:08.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:08 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4288150024' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:14:08.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:08 smithi060 ceph-mon[40290]: osdmap e349: 8 total, 8 up, 8 in 2024-09-20T22:14:08.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:08 smithi060 ceph-mon[40290]: 9.d scrub starts 2024-09-20T22:14:08.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:08 smithi060 ceph-mon[40290]: pgmap v603: 158 pgs: 158 active+clean; 3.5 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.22k op/s 2024-09-20T22:14:08.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:08 smithi060 ceph-mon[40290]: 6.c scrub starts 2024-09-20T22:14:08.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:08 smithi060 ceph-mon[40290]: 6.c scrub ok 2024-09-20T22:14:08.839 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:08.856 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:08.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:08.890 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:08.890 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:08.975 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:14:08.990 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:09.041 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:09.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:09.042 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:09.143 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:09.173 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:09.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:09.174 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:09.275 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:09.313 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:09.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:09.314 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:09.414 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:09.456 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:09.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:09.457 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:09.558 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:09.598 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:09.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:09.599 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:09.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:09.702 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:14:09.702 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:14:09.702 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:14:09.702 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:14:09.702 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:14:09.702 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:14:09.703 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:14:09.703 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:14:09.731 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:14:09.731 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:14:09.736 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:09.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:09.737 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:09.794 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:14:09.795 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:14:09.832 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:14:09.832 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:14:09.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:09.875 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:14:09.875 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:14:09.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:09 smithi060 ceph-mon[29694]: 6.2 scrub starts 2024-09-20T22:14:09.876 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:09 smithi060 ceph-mon[29694]: 6.2 scrub ok 2024-09-20T22:14:09.876 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:09 smithi060 ceph-mon[29694]: 9.b scrub starts 2024-09-20T22:14:09.876 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:09 smithi060 ceph-mon[29694]: 5.14 scrub starts 2024-09-20T22:14:09.876 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:09 smithi060 ceph-mon[29694]: 5.14 scrub ok 2024-09-20T22:14:09.876 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:09 smithi060 ceph-mon[29694]: 9.d scrub ok 2024-09-20T22:14:09.876 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:09 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3476132492' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:09.877 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:09 smithi060 ceph-mon[40290]: 6.2 scrub starts 2024-09-20T22:14:09.877 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:09 smithi060 ceph-mon[40290]: 6.2 scrub ok 2024-09-20T22:14:09.877 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:09 smithi060 ceph-mon[40290]: 9.b scrub starts 2024-09-20T22:14:09.877 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:09 smithi060 ceph-mon[40290]: 5.14 scrub starts 2024-09-20T22:14:09.877 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:09 smithi060 ceph-mon[40290]: 5.14 scrub ok 2024-09-20T22:14:09.877 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:09 smithi060 ceph-mon[40290]: 9.d scrub ok 2024-09-20T22:14:09.877 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:09 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3476132492' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:09.883 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:09.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:09.884 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:09.985 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:10.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:09 smithi067 ceph-mon[33511]: 6.2 scrub starts 2024-09-20T22:14:10.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:09 smithi067 ceph-mon[33511]: 6.2 scrub ok 2024-09-20T22:14:10.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:09 smithi067 ceph-mon[33511]: 9.b scrub starts 2024-09-20T22:14:10.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:09 smithi067 ceph-mon[33511]: 5.14 scrub starts 2024-09-20T22:14:10.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:09 smithi067 ceph-mon[33511]: 5.14 scrub ok 2024-09-20T22:14:10.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:09 smithi067 ceph-mon[33511]: 9.d scrub ok 2024-09-20T22:14:10.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:09 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3476132492' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:10.023 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:10.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:10.024 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:10.125 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:10.160 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:10.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:10.161 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:10.262 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:10.294 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:10.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:10.295 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:10.344 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:14:10.344 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:14:10.344 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:14:10.345 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:14:10.345 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.345 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169974 7.10 7:0a082cf8:::benchmark_data_smithi060_78800_object36941:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:14:10.345 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:14:10.345 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 490.992533428, 2024-09-20T22:14:10.345 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87805208899999998, 2024-09-20T22:14:10.345 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.345 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.345 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:10.345 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.345 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:14:10.345 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:14:10.345 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169974 2024-09-20T22:14:10.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.346 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.346 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.346 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:14:10.346 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.346 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.346 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:14:10.346 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.347 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.347 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624616+0000", 2024-09-20T22:14:10.347 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1739999999999999e-06 2024-09-20T22:14:10.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.347 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.347 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624626+0000", 2024-09-20T22:14:10.347 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7909999999999999e-06 2024-09-20T22:14:10.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.347 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.347 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624654+0000", 2024-09-20T22:14:10.347 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.8524999999999999e-05 2024-09-20T22:14:10.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.348 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.348 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624665+0000", 2024-09-20T22:14:10.348 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0985000000000001e-05 2024-09-20T22:14:10.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.348 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.348 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.705237+0000", 2024-09-20T22:14:10.348 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.080571485999999998 2024-09-20T22:14:10.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.348 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.348 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.705245+0000", 2024-09-20T22:14:10.348 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8460000000000008e-06 2024-09-20T22:14:10.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.349 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.349 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498359+0000", 2024-09-20T22:14:10.349 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79311461999999999 2024-09-20T22:14:10.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.349 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:10.349 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498498+0000", 2024-09-20T22:14:10.349 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013918599999999999 2024-09-20T22:14:10.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.349 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:14:10.349 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498751+0000", 2024-09-20T22:14:10.349 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025235999999999998 2024-09-20T22:14:10.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.350 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.350 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.500697+0000", 2024-09-20T22:14:10.350 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0019460969999999999 2024-09-20T22:14:10.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.350 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:10.350 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502528+0000", 2024-09-20T22:14:10.350 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00183161 2024-09-20T22:14:10.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.350 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:10.351 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502597+0000", 2024-09-20T22:14:10.351 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8761000000000001e-05 2024-09-20T22:14:10.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.351 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.351 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502666+0000", 2024-09-20T22:14:10.351 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8647999999999997e-05 2024-09-20T22:14:10.351 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.351 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.351 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.351 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227618 7.0 7:0598f762:::benchmark_data_smithi060_78800_object94585:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-20T22:14:10.352 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:53.805369+0000", 2024-09-20T22:14:10.352 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 435.81177840300001, 2024-09-20T22:14:10.352 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2615457110000001, 2024-09-20T22:14:10.352 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.352 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.352 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:10.352 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.352 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:14:10.352 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:14:10.352 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227618 2024-09-20T22:14:10.352 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.352 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.352 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.352 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.353 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805369+0000", 2024-09-20T22:14:10.353 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.353 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.353 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.353 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805369+0000", 2024-09-20T22:14:10.353 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.353 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.353 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.353 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805371+0000", 2024-09-20T22:14:10.353 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.827e-06 2024-09-20T22:14:10.353 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.353 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.353 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.354 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805377+0000", 2024-09-20T22:14:10.354 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.161e-06 2024-09-20T22:14:10.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.354 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.354 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.354 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805398+0000", 2024-09-20T22:14:10.354 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1208000000000001e-05 2024-09-20T22:14:10.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.354 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.354 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.354 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805406+0000", 2024-09-20T22:14:10.354 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3820000000000004e-06 2024-09-20T22:14:10.354 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.354 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.355 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.355 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.813177+0000", 2024-09-20T22:14:10.355 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0077708919999999997 2024-09-20T22:14:10.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.355 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.355 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.813194+0000", 2024-09-20T22:14:10.355 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7107000000000001e-05 2024-09-20T22:14:10.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.355 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.355 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.813273+0000", 2024-09-20T22:14:10.355 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.000078609 2024-09-20T22:14:10.355 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.355 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.356 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.356 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.813290+0000", 2024-09-20T22:14:10.356 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7382999999999999e-05 2024-09-20T22:14:10.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.356 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.356 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.356 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.960476+0000", 2024-09-20T22:14:10.356 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1471854260000001 2024-09-20T22:14:10.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.356 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.356 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:10.356 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.960616+0000", 2024-09-20T22:14:10.356 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014035100000000001 2024-09-20T22:14:10.356 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.357 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.357 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,3", 2024-09-20T22:14:10.357 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.961013+0000", 2024-09-20T22:14:10.357 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00039649999999999999 2024-09-20T22:14:10.357 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.357 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.357 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:10.357 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.963511+0000", 2024-09-20T22:14:10.357 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0024986919999999998 2024-09-20T22:14:10.357 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.357 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.357 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.357 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.963553+0000", 2024-09-20T22:14:10.358 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1783999999999998e-05 2024-09-20T22:14:10.358 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.358 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.358 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.358 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.066713+0000", 2024-09-20T22:14:10.358 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.103160088 2024-09-20T22:14:10.358 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.358 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.358 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:10.358 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.066794+0000", 2024-09-20T22:14:10.358 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0924e-05 2024-09-20T22:14:10.358 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.358 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.358 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.359 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.066915+0000", 2024-09-20T22:14:10.359 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000120377 2024-09-20T22:14:10.359 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.359 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.359 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.359 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.359 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.359 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227635 7.b 7:da8bd73f:::benchmark_data_smithi060_78800_object94602:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e146)", 2024-09-20T22:14:10.359 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:54.820720+0000", 2024-09-20T22:14:10.359 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 434.79642752699999, 2024-09-20T22:14:10.359 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1621617440000001, 2024-09-20T22:14:10.359 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.359 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.359 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:10.360 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.360 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:14:10.360 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:14:10.360 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227635 2024-09-20T22:14:10.360 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.360 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.360 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.360 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.360 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820720+0000", 2024-09-20T22:14:10.360 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.360 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.360 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.360 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.360 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820720+0000", 2024-09-20T22:14:10.360 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.361 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.361 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.361 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820721+0000", 2024-09-20T22:14:10.361 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.593e-06 2024-09-20T22:14:10.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.361 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.361 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.361 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820728+0000", 2024-09-20T22:14:10.361 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3799999999999999e-06 2024-09-20T22:14:10.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.361 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.361 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.362 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820740+0000", 2024-09-20T22:14:10.362 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.279e-05 2024-09-20T22:14:10.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.362 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.362 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820749+0000", 2024-09-20T22:14:10.362 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8000000000000004e-06 2024-09-20T22:14:10.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.362 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.362 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820942+0000", 2024-09-20T22:14:10.362 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000192263 2024-09-20T22:14:10.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.363 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.363 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820953+0000", 2024-09-20T22:14:10.363 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1953000000000001e-05 2024-09-20T22:14:10.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.363 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.363 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.974865+0000", 2024-09-20T22:14:10.363 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.15391142 2024-09-20T22:14:10.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.363 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:10.363 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.974939+0000", 2024-09-20T22:14:10.363 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3798000000000001e-05 2024-09-20T22:14:10.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.364 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,6", 2024-09-20T22:14:10.364 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.975019+0000", 2024-09-20T22:14:10.364 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0692999999999995e-05 2024-09-20T22:14:10.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.364 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.364 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.977853+0000", 2024-09-20T22:14:10.364 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028332269999999998 2024-09-20T22:14:10.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.364 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.364 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982635+0000", 2024-09-20T22:14:10.365 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0047819840000000004 2024-09-20T22:14:10.365 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.365 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.365 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:10.365 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982688+0000", 2024-09-20T22:14:10.365 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.384e-05 2024-09-20T22:14:10.365 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.365 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.365 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:10.365 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982747+0000", 2024-09-20T22:14:10.365 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9004e-05 2024-09-20T22:14:10.365 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.365 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.365 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.365 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982881+0000", 2024-09-20T22:14:10.366 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013399899999999999 2024-09-20T22:14:10.366 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.366 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.366 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.366 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.366 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227641 7.b 7:d29062b1:::benchmark_data_smithi060_78800_object94608:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e146)", 2024-09-20T22:14:10.366 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:14:10.366 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 434.79637641599999, 2024-09-20T22:14:10.366 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2739681169999999, 2024-09-20T22:14:10.366 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.366 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.367 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:10.367 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.367 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:14:10.367 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:14:10.367 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227641 2024-09-20T22:14:10.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.367 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.367 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.367 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.367 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:14:10.367 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.367 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.367 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.368 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:14:10.368 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.368 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.368 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.368 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.368 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:14:10.368 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3900000000000005e-07 2024-09-20T22:14:10.368 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.368 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.368 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.368 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820775+0000", 2024-09-20T22:14:10.368 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4589999999999998e-06 2024-09-20T22:14:10.368 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.368 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.369 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.369 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820783+0000", 2024-09-20T22:14:10.369 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8949999999999998e-06 2024-09-20T22:14:10.369 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.369 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.369 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.369 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820788+0000", 2024-09-20T22:14:10.369 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5759999999999996e-06 2024-09-20T22:14:10.369 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.369 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.369 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.369 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820979+0000", 2024-09-20T22:14:10.369 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019122 2024-09-20T22:14:10.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.370 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.370 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.370 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820988+0000", 2024-09-20T22:14:10.370 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1480000000000006e-06 2024-09-20T22:14:10.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.370 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.370 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.370 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.977915+0000", 2024-09-20T22:14:10.371 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.156927834 2024-09-20T22:14:10.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.371 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.371 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:10.371 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.978045+0000", 2024-09-20T22:14:10.371 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012967 2024-09-20T22:14:10.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.371 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.371 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,6", 2024-09-20T22:14:10.371 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.978183+0000", 2024-09-20T22:14:10.371 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013767900000000001 2024-09-20T22:14:10.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.372 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.372 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.372 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982676+0000", 2024-09-20T22:14:10.372 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0044928210000000001 2024-09-20T22:14:10.372 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.372 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.372 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:10.372 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982891+0000", 2024-09-20T22:14:10.372 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00021542299999999999 2024-09-20T22:14:10.372 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.372 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.372 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.372 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.094638+0000", 2024-09-20T22:14:10.372 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.111746554 2024-09-20T22:14:10.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.373 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.373 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:10.373 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.094677+0000", 2024-09-20T22:14:10.373 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9069000000000002e-05 2024-09-20T22:14:10.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.373 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.373 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.373 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.094739+0000", 2024-09-20T22:14:10.373 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2230000000000006e-05 2024-09-20T22:14:10.373 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.373 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.373 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.374 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.374 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105650 8.14 8:29da2845:::benchmark_data_smithi060_111051_object105649:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:14:10.374 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.545671+0000", 2024-09-20T22:14:10.374 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 301.07147619099999, 2024-09-20T22:14:10.374 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.038415869, 2024-09-20T22:14:10.374 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.374 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.374 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:14:10.374 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.374 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:10.374 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:10.374 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105650 2024-09-20T22:14:10.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.375 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.375 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.375 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.375 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545671+0000", 2024-09-20T22:14:10.375 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.375 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.375 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.375 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.375 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545671+0000", 2024-09-20T22:14:10.375 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.375 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.375 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.375 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.375 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545673+0000", 2024-09-20T22:14:10.376 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.17e-06 2024-09-20T22:14:10.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.376 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.376 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.376 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545723+0000", 2024-09-20T22:14:10.376 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9509000000000003e-05 2024-09-20T22:14:10.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.376 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.376 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.376 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545741+0000", 2024-09-20T22:14:10.376 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8634000000000002e-05 2024-09-20T22:14:10.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.376 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.376 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.377 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545751+0000", 2024-09-20T22:14:10.377 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4420000000000004e-06 2024-09-20T22:14:10.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.377 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.377 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.377 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.707696+0000", 2024-09-20T22:14:10.377 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.16194560099999999 2024-09-20T22:14:10.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.377 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.377 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.378 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.707706+0000", 2024-09-20T22:14:10.378 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6860000000000001e-06 2024-09-20T22:14:10.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.378 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.378 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584005+0000", 2024-09-20T22:14:10.378 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87629895400000002 2024-09-20T22:14:10.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.378 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.378 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584087+0000", 2024-09-20T22:14:10.378 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1872999999999994e-05 2024-09-20T22:14:10.378 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.379 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.379 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.379 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.379 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105665 8.18 8:1b9c2a37:::benchmark_data_smithi060_111051_object105664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:14:10.379 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.556635+0000", 2024-09-20T22:14:10.379 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 301.06051216200001, 2024-09-20T22:14:10.379 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.051209888, 2024-09-20T22:14:10.379 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.379 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.379 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:10.379 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.379 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:10.379 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:10.379 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105665 2024-09-20T22:14:10.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.380 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.380 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.380 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556635+0000", 2024-09-20T22:14:10.380 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.380 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.380 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556635+0000", 2024-09-20T22:14:10.380 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.380 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.380 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556638+0000", 2024-09-20T22:14:10.381 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.6400000000000001e-06 2024-09-20T22:14:10.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.381 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.381 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.381 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556714+0000", 2024-09-20T22:14:10.381 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6594999999999994e-05 2024-09-20T22:14:10.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.381 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.381 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.381 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556736+0000", 2024-09-20T22:14:10.381 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1662e-05 2024-09-20T22:14:10.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.381 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.381 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.382 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556751+0000", 2024-09-20T22:14:10.382 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4941e-05 2024-09-20T22:14:10.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.382 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.382 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.382 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.750414+0000", 2024-09-20T22:14:10.382 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.193663171 2024-09-20T22:14:10.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.382 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.382 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.382 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.750424+0000", 2024-09-20T22:14:10.382 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0397999999999999e-05 2024-09-20T22:14:10.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.382 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.383 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.383 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.596184+0000", 2024-09-20T22:14:10.383 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.84575938799999995 2024-09-20T22:14:10.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.383 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.383 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:10.383 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.596303+0000", 2024-09-20T22:14:10.383 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000119503 2024-09-20T22:14:10.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.383 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.383 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:14:10.383 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.596588+0000", 2024-09-20T22:14:10.383 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00028446100000000001 2024-09-20T22:14:10.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.384 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.384 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.384 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.603603+0000", 2024-09-20T22:14:10.384 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0070155649999999996 2024-09-20T22:14:10.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.384 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.384 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:10.384 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607589+0000", 2024-09-20T22:14:10.385 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0039854110000000003 2024-09-20T22:14:10.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.385 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.385 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.385 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607743+0000", 2024-09-20T22:14:10.385 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015391499999999999 2024-09-20T22:14:10.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.385 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.385 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:10.385 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607802+0000", 2024-09-20T22:14:10.385 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9472000000000003e-05 2024-09-20T22:14:10.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.385 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.386 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.386 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607845+0000", 2024-09-20T22:14:10.386 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2765999999999997e-05 2024-09-20T22:14:10.386 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.386 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.386 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.386 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.386 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105673 8.14 8:2f269e39:::benchmark_data_smithi060_111051_object105672:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:14:10.386 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.761620+0000", 2024-09-20T22:14:10.386 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 300.85552760899998, 2024-09-20T22:14:10.386 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.83821859799999998, 2024-09-20T22:14:10.386 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.387 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.387 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:10.387 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.387 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:10.387 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:10.387 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105673 2024-09-20T22:14:10.387 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.387 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.387 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.387 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.387 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761620+0000", 2024-09-20T22:14:10.387 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.387 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.387 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.388 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.388 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761620+0000", 2024-09-20T22:14:10.388 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.388 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.388 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.388 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.388 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761621+0000", 2024-09-20T22:14:10.388 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.392e-06 2024-09-20T22:14:10.388 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.388 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.388 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.388 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761674+0000", 2024-09-20T22:14:10.388 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2677999999999999e-05 2024-09-20T22:14:10.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.389 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.389 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.389 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761686+0000", 2024-09-20T22:14:10.389 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1926000000000001e-05 2024-09-20T22:14:10.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.389 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.389 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.389 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761697+0000", 2024-09-20T22:14:10.389 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1364e-05 2024-09-20T22:14:10.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.389 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.389 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.390 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761770+0000", 2024-09-20T22:14:10.390 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2904999999999994e-05 2024-09-20T22:14:10.390 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.390 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.390 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.390 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761777+0000", 2024-09-20T22:14:10.390 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8029999999999999e-06 2024-09-20T22:14:10.390 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.390 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.390 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.390 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584113+0000", 2024-09-20T22:14:10.390 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.82233672499999999 2024-09-20T22:14:10.390 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.391 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.391 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:10.391 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584256+0000", 2024-09-20T22:14:10.391 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000142686 2024-09-20T22:14:10.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.391 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.391 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,4", 2024-09-20T22:14:10.391 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584528+0000", 2024-09-20T22:14:10.391 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00027199499999999997 2024-09-20T22:14:10.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.392 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.392 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.392 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.590292+0000", 2024-09-20T22:14:10.392 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.005764333 2024-09-20T22:14:10.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.392 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.392 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:10.392 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.590315+0000", 2024-09-20T22:14:10.392 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2768e-05 2024-09-20T22:14:10.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.392 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.392 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.393 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.599740+0000", 2024-09-20T22:14:10.393 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0094248200000000004 2024-09-20T22:14:10.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.393 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.393 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:10.393 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.599795+0000", 2024-09-20T22:14:10.393 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5108000000000002e-05 2024-09-20T22:14:10.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.393 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.393 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.393 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.599838+0000", 2024-09-20T22:14:10.393 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3095e-05 2024-09-20T22:14:10.393 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.393 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.394 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.394 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.394 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:106978 8.19 8:9e674fff:::benchmark_data_smithi060_111051_object106977:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:14:10.394 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.576265+0000", 2024-09-20T22:14:10.394 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 299.04088249500001, 2024-09-20T22:14:10.394 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9488434429999999, 2024-09-20T22:14:10.394 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.394 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.394 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:10.394 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.394 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:10.394 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:10.394 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 106978 2024-09-20T22:14:10.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.395 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.395 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.395 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.395 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576265+0000", 2024-09-20T22:14:10.395 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.395 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.395 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.395 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576265+0000", 2024-09-20T22:14:10.395 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.395 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.395 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.396 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576267+0000", 2024-09-20T22:14:10.396 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0509999999999999e-06 2024-09-20T22:14:10.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.396 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.396 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.396 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576299+0000", 2024-09-20T22:14:10.396 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1962000000000003e-05 2024-09-20T22:14:10.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.396 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.396 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.396 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576312+0000", 2024-09-20T22:14:10.396 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3423e-05 2024-09-20T22:14:10.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.397 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.397 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.397 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576322+0000", 2024-09-20T22:14:10.397 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0118e-05 2024-09-20T22:14:10.397 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.397 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.397 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.397 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576468+0000", 2024-09-20T22:14:10.397 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000145575 2024-09-20T22:14:10.397 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.397 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.397 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.397 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576481+0000", 2024-09-20T22:14:10.397 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3023999999999999e-05 2024-09-20T22:14:10.398 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.398 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.398 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.398 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410333+0000", 2024-09-20T22:14:10.398 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.83385187800000005 2024-09-20T22:14:10.398 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.398 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.398 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.398 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410345+0000", 2024-09-20T22:14:10.399 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.258e-05 2024-09-20T22:14:10.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.399 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.399 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.399 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.503436+0000", 2024-09-20T22:14:10.399 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.093091088 2024-09-20T22:14:10.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.399 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.399 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:10.399 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.503599+0000", 2024-09-20T22:14:10.399 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00016286999999999999 2024-09-20T22:14:10.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.399 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.399 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4,6", 2024-09-20T22:14:10.400 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.505264+0000", 2024-09-20T22:14:10.400 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001664591 2024-09-20T22:14:10.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.400 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.400 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.400 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.508504+0000", 2024-09-20T22:14:10.400 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0032404920000000002 2024-09-20T22:14:10.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.400 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.400 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.400 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.508547+0000", 2024-09-20T22:14:10.400 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2734000000000001e-05 2024-09-20T22:14:10.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.400 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.401 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.401 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.513180+0000", 2024-09-20T22:14:10.401 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046330490000000002 2024-09-20T22:14:10.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.401 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.401 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:10.401 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.524891+0000", 2024-09-20T22:14:10.401 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.011710379999999999 2024-09-20T22:14:10.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.401 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.401 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:10.401 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.524966+0000", 2024-09-20T22:14:10.401 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5290999999999995e-05 2024-09-20T22:14:10.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.402 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.402 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.402 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.525108+0000", 2024-09-20T22:14:10.402 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000142337 2024-09-20T22:14:10.402 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.402 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.402 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.402 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.402 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.402 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:107002 8.11 8.e0f52171 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:14:10.402 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.617682+0000", 2024-09-20T22:14:10.402 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 298.99946499700002, 2024-09-20T22:14:10.403 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79265608300000001, 2024-09-20T22:14:10.403 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.403 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.403 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:14:10.403 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.403 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:10.403 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:10.403 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 107002 2024-09-20T22:14:10.403 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.403 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.403 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.403 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.403 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617682+0000", 2024-09-20T22:14:10.404 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.404 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.404 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.404 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.404 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617682+0000", 2024-09-20T22:14:10.404 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.404 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.404 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.404 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.404 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617684+0000", 2024-09-20T22:14:10.404 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1299999999999999e-06 2024-09-20T22:14:10.404 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.404 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.405 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.405 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617725+0000", 2024-09-20T22:14:10.405 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0839e-05 2024-09-20T22:14:10.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.405 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.405 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.405 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617738+0000", 2024-09-20T22:14:10.405 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2310000000000001e-05 2024-09-20T22:14:10.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.406 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.406 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.406 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617746+0000", 2024-09-20T22:14:10.406 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3939999999999996e-06 2024-09-20T22:14:10.406 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.406 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.406 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.406 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617856+0000", 2024-09-20T22:14:10.406 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000109917 2024-09-20T22:14:10.406 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.406 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.406 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.406 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617878+0000", 2024-09-20T22:14:10.407 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2495999999999999e-05 2024-09-20T22:14:10.407 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.407 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.407 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.407 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410330+0000", 2024-09-20T22:14:10.407 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79245202299999995 2024-09-20T22:14:10.407 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.407 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.407 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.407 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410338+0000", 2024-09-20T22:14:10.407 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9740000000000002e-06 2024-09-20T22:14:10.407 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.407 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.407 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163158 8.11 8:8e0bce89:::benchmark_data_smithi060_111051_object43159:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:14:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:08.839869+0000", 2024-09-20T22:14:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 240.777278556, 2024-09-20T22:14:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.65193807500000001, 2024-09-20T22:14:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:10.408 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163158 2024-09-20T22:14:10.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.409 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.409 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.409 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839869+0000", 2024-09-20T22:14:10.409 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.409 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.409 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839869+0000", 2024-09-20T22:14:10.409 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.410 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.410 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839870+0000", 2024-09-20T22:14:10.410 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5710000000000001e-06 2024-09-20T22:14:10.410 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.410 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.410 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.410 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839876+0000", 2024-09-20T22:14:10.410 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3820000000000003e-06 2024-09-20T22:14:10.410 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.410 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.410 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.410 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839890+0000", 2024-09-20T22:14:10.410 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3952e-05 2024-09-20T22:14:10.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.411 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.411 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.411 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839897+0000", 2024-09-20T22:14:10.411 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4150000000000002e-06 2024-09-20T22:14:10.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.411 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.411 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.411 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839956+0000", 2024-09-20T22:14:10.411 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9157999999999998e-05 2024-09-20T22:14:10.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.411 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.412 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.412 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839967+0000", 2024-09-20T22:14:10.412 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1019000000000001e-05 2024-09-20T22:14:10.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.412 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.412 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.412 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477635+0000", 2024-09-20T22:14:10.412 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.63766765700000005 2024-09-20T22:14:10.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.412 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477737+0000", 2024-09-20T22:14:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010265899999999999 2024-09-20T22:14:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:14:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477939+0000", 2024-09-20T22:14:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020106900000000001 2024-09-20T22:14:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.482557+0000", 2024-09-20T22:14:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046186830000000002 2024-09-20T22:14:10.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491495+0000", 2024-09-20T22:14:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0089377699999999994 2024-09-20T22:14:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491673+0000", 2024-09-20T22:14:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017837500000000001 2024-09-20T22:14:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491762+0000", 2024-09-20T22:14:10.414 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8555999999999997e-05 2024-09-20T22:14:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491807+0000", 2024-09-20T22:14:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4808999999999997e-05 2024-09-20T22:14:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163289 8.11 8:8c47e2c8:::benchmark_data_smithi060_111051_object43290:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:14:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:08.954133+0000", 2024-09-20T22:14:10.415 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 240.663013922, 2024-09-20T22:14:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.53808307099999997, 2024-09-20T22:14:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163289 2024-09-20T22:14:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954133+0000", 2024-09-20T22:14:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954133+0000", 2024-09-20T22:14:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954134+0000", 2024-09-20T22:14:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2900000000000002e-07 2024-09-20T22:14:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954138+0000", 2024-09-20T22:14:10.417 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5260000000000002e-06 2024-09-20T22:14:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954147+0000", 2024-09-20T22:14:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2010000000000005e-06 2024-09-20T22:14:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954153+0000", 2024-09-20T22:14:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0610000000000001e-06 2024-09-20T22:14:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.418 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954209+0000", 2024-09-20T22:14:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5767999999999999e-05 2024-09-20T22:14:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954216+0000", 2024-09-20T22:14:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8099999999999998e-06 2024-09-20T22:14:10.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.478998+0000", 2024-09-20T22:14:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.52478124900000001 2024-09-20T22:14:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.479095+0000", 2024-09-20T22:14:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7125000000000003e-05 2024-09-20T22:14:10.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:14:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.479291+0000", 2024-09-20T22:14:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019642100000000001 2024-09-20T22:14:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.482759+0000", 2024-09-20T22:14:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0034672689999999998 2024-09-20T22:14:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491864+0000", 2024-09-20T22:14:10.421 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0091058630000000005 2024-09-20T22:14:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491961+0000", 2024-09-20T22:14:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6717000000000003e-05 2024-09-20T22:14:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492014+0000", 2024-09-20T22:14:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2877000000000002e-05 2024-09-20T22:14:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.422 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492216+0000", 2024-09-20T22:14:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020235500000000001 2024-09-20T22:14:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163324 8.11 8:8e312e9e:::benchmark_data_smithi060_111051_object43325:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:14:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:08.983848+0000", 2024-09-20T22:14:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 240.63329899999999, 2024-09-20T22:14:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.50861305999999995, 2024-09-20T22:14:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:10.423 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163324 2024-09-20T22:14:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983848+0000", 2024-09-20T22:14:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.424 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983848+0000", 2024-09-20T22:14:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983849+0000", 2024-09-20T22:14:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.11e-06 2024-09-20T22:14:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983853+0000", 2024-09-20T22:14:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0439999999999997e-06 2024-09-20T22:14:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.425 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983863+0000", 2024-09-20T22:14:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6430000000000007e-06 2024-09-20T22:14:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983868+0000", 2024-09-20T22:14:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1000000000000003e-06 2024-09-20T22:14:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.426 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983919+0000", 2024-09-20T22:14:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0433999999999999e-05 2024-09-20T22:14:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983928+0000", 2024-09-20T22:14:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3659999999999998e-06 2024-09-20T22:14:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.427 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.480646+0000", 2024-09-20T22:14:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.49671833999999998 2024-09-20T22:14:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.480768+0000", 2024-09-20T22:14:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012145 2024-09-20T22:14:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:14:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.481025+0000", 2024-09-20T22:14:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025713299999999999 2024-09-20T22:14:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.428 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.488496+0000", 2024-09-20T22:14:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0074709540000000001 2024-09-20T22:14:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492276+0000", 2024-09-20T22:14:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0037803670000000002 2024-09-20T22:14:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492350+0000", 2024-09-20T22:14:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3436999999999995e-05 2024-09-20T22:14:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.429 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492421+0000", 2024-09-20T22:14:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1303000000000005e-05 2024-09-20T22:14:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492461+0000", 2024-09-20T22:14:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0379000000000001e-05 2024-09-20T22:14:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.430 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163358 8.11 8:8b7ace29:::benchmark_data_smithi060_111051_object43359:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:14:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.014358+0000", 2024-09-20T22:14:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 240.60278901500001, 2024-09-20T22:14:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.47828052599999998, 2024-09-20T22:14:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163358 2024-09-20T22:14:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.431 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.432 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014358+0000", 2024-09-20T22:14:10.432 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.432 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.432 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014358+0000", 2024-09-20T22:14:10.432 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.432 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.432 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014359+0000", 2024-09-20T22:14:10.432 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0669999999999999e-06 2024-09-20T22:14:10.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.432 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.433 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014364+0000", 2024-09-20T22:14:10.433 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0220000000000004e-06 2024-09-20T22:14:10.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.433 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.433 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.433 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014374+0000", 2024-09-20T22:14:10.433 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9599999999999995e-06 2024-09-20T22:14:10.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.433 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.434 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.434 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014388+0000", 2024-09-20T22:14:10.434 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3413e-05 2024-09-20T22:14:10.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.434 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.434 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014447+0000", 2024-09-20T22:14:10.434 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9698999999999999e-05 2024-09-20T22:14:10.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.434 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.434 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014455+0000", 2024-09-20T22:14:10.435 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.072e-06 2024-09-20T22:14:10.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.435 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.435 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.435 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.482921+0000", 2024-09-20T22:14:10.435 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.46846513200000001 2024-09-20T22:14:10.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.435 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.435 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:10.435 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483020+0000", 2024-09-20T22:14:10.435 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9319000000000002e-05 2024-09-20T22:14:10.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.435 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.436 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:14:10.436 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483251+0000", 2024-09-20T22:14:10.436 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00023124300000000001 2024-09-20T22:14:10.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.436 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:10.436 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491608+0000", 2024-09-20T22:14:10.436 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0083564190000000003 2024-09-20T22:14:10.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.436 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.436 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492507+0000", 2024-09-20T22:14:10.436 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00089972900000000005 2024-09-20T22:14:10.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.437 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.437 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.437 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492546+0000", 2024-09-20T22:14:10.437 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9156999999999998e-05 2024-09-20T22:14:10.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.437 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.437 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:10.437 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492602+0000", 2024-09-20T22:14:10.437 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5114999999999997e-05 2024-09-20T22:14:10.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.437 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.437 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.437 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492639+0000", 2024-09-20T22:14:10.437 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7178999999999998e-05 2024-09-20T22:14:10.438 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.438 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.438 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.438 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.438 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163372 8.11 8:8b6ec81f:::benchmark_data_smithi060_111051_object43373:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:14:10.438 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.027195+0000", 2024-09-20T22:14:10.438 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 240.58995220700001, 2024-09-20T22:14:10.438 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.46568983899999999, 2024-09-20T22:14:10.438 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.438 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.438 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:10.439 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.439 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:10.439 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:10.439 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163372 2024-09-20T22:14:10.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.439 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.439 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.439 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027195+0000", 2024-09-20T22:14:10.439 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.439 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.439 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027195+0000", 2024-09-20T22:14:10.439 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.440 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.440 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027196+0000", 2024-09-20T22:14:10.440 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0599999999999999e-07 2024-09-20T22:14:10.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.440 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.440 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027200+0000", 2024-09-20T22:14:10.441 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9280000000000003e-06 2024-09-20T22:14:10.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.441 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.441 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.441 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027210+0000", 2024-09-20T22:14:10.441 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0035e-05 2024-09-20T22:14:10.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.441 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.441 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.441 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027217+0000", 2024-09-20T22:14:10.441 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6450000000000005e-06 2024-09-20T22:14:10.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.441 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.442 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.442 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027281+0000", 2024-09-20T22:14:10.442 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3281000000000006e-05 2024-09-20T22:14:10.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.442 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.442 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027291+0000", 2024-09-20T22:14:10.442 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0268999999999999e-05 2024-09-20T22:14:10.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.442 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.442 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.484647+0000", 2024-09-20T22:14:10.443 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.45735564499999998 2024-09-20T22:14:10.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.443 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:10.443 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.484745+0000", 2024-09-20T22:14:10.443 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8712e-05 2024-09-20T22:14:10.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.443 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:14:10.443 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.484940+0000", 2024-09-20T22:14:10.443 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019504900000000001 2024-09-20T22:14:10.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.443 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:10.443 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491619+0000", 2024-09-20T22:14:10.444 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0066782090000000001 2024-09-20T22:14:10.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.444 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.444 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492716+0000", 2024-09-20T22:14:10.444 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001097701 2024-09-20T22:14:10.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.444 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.444 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492778+0000", 2024-09-20T22:14:10.444 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1538000000000006e-05 2024-09-20T22:14:10.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.444 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:10.444 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492830+0000", 2024-09-20T22:14:10.445 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1997999999999999e-05 2024-09-20T22:14:10.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.445 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.445 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492885+0000", 2024-09-20T22:14:10.445 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5022999999999998e-05 2024-09-20T22:14:10.445 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.445 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.445 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.445 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163373 8.11 8:8d07a686:::benchmark_data_smithi060_111051_object43374:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:14:10.445 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.027847+0000", 2024-09-20T22:14:10.445 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 240.589299799, 2024-09-20T22:14:10.445 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.465200326, 2024-09-20T22:14:10.446 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.446 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.446 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:10.446 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.446 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:10.446 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:10.446 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163373 2024-09-20T22:14:10.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.446 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.446 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.446 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027847+0000", 2024-09-20T22:14:10.446 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.447 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.447 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.447 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027847+0000", 2024-09-20T22:14:10.447 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.447 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.447 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.447 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027848+0000", 2024-09-20T22:14:10.447 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3600000000000003e-07 2024-09-20T22:14:10.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.448 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.448 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027852+0000", 2024-09-20T22:14:10.448 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6330000000000002e-06 2024-09-20T22:14:10.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.448 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.448 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027860+0000", 2024-09-20T22:14:10.448 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2789999999999998e-06 2024-09-20T22:14:10.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.448 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.448 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027866+0000", 2024-09-20T22:14:10.449 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1070000000000001e-06 2024-09-20T22:14:10.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.449 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.449 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027897+0000", 2024-09-20T22:14:10.449 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0722000000000002e-05 2024-09-20T22:14:10.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.449 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.449 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027901+0000", 2024-09-20T22:14:10.449 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1649999999999999e-06 2024-09-20T22:14:10.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.450 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.486658+0000", 2024-09-20T22:14:10.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.45875733499999999 2024-09-20T22:14:10.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:10.450 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.486767+0000", 2024-09-20T22:14:10.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000108385 2024-09-20T22:14:10.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:14:10.450 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487120+0000", 2024-09-20T22:14:10.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00035312499999999998 2024-09-20T22:14:10.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.451 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:10.451 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491877+0000", 2024-09-20T22:14:10.451 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0047569170000000003 2024-09-20T22:14:10.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.451 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.451 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492928+0000", 2024-09-20T22:14:10.451 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001051633 2024-09-20T22:14:10.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.451 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.452 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492967+0000", 2024-09-20T22:14:10.452 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.8781e-05 2024-09-20T22:14:10.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.452 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:10.452 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493013+0000", 2024-09-20T22:14:10.452 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5555e-05 2024-09-20T22:14:10.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.452 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.452 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493048+0000", 2024-09-20T22:14:10.452 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5052999999999998e-05 2024-09-20T22:14:10.452 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.452 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.452 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.453 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.453 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163388 8.11 8:893f1034:::benchmark_data_smithi060_111051_object43389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:14:10.453 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.050114+0000", 2024-09-20T22:14:10.453 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 240.56703308900001, 2024-09-20T22:14:10.453 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.44308652799999998, 2024-09-20T22:14:10.453 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.453 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.453 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:10.453 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.453 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:10.453 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:10.453 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163388 2024-09-20T22:14:10.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.453 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.453 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.454 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.454 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050114+0000", 2024-09-20T22:14:10.454 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.454 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.454 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.454 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050114+0000", 2024-09-20T22:14:10.454 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.455 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.455 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050115+0000", 2024-09-20T22:14:10.455 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2500000000000005e-07 2024-09-20T22:14:10.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.455 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.455 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050119+0000", 2024-09-20T22:14:10.455 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.698e-06 2024-09-20T22:14:10.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.455 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.455 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050128+0000", 2024-09-20T22:14:10.455 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5820000000000008e-06 2024-09-20T22:14:10.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.456 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.456 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050133+0000", 2024-09-20T22:14:10.456 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9060000000000001e-06 2024-09-20T22:14:10.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.456 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.456 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050169+0000", 2024-09-20T22:14:10.456 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5984e-05 2024-09-20T22:14:10.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.456 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.456 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050174+0000", 2024-09-20T22:14:10.456 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0180000000000001e-06 2024-09-20T22:14:10.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.457 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.488568+0000", 2024-09-20T22:14:10.457 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.43839391 2024-09-20T22:14:10.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.457 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:10.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.488685+0000", 2024-09-20T22:14:10.457 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000117163 2024-09-20T22:14:10.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.457 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:14:10.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.489162+0000", 2024-09-20T22:14:10.457 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00047712600000000003 2024-09-20T22:14:10.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.458 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:10.458 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491884+0000", 2024-09-20T22:14:10.458 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0027215329999999999 2024-09-20T22:14:10.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.458 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.458 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493086+0000", 2024-09-20T22:14:10.458 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001201765 2024-09-20T22:14:10.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.458 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.458 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493122+0000", 2024-09-20T22:14:10.458 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6622999999999998e-05 2024-09-20T22:14:10.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.459 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:10.459 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493168+0000", 2024-09-20T22:14:10.459 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5757999999999998e-05 2024-09-20T22:14:10.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.459 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.459 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493201+0000", 2024-09-20T22:14:10.459 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2737000000000002e-05 2024-09-20T22:14:10.459 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.459 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.459 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.459 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163393 8.11 8:8bb06d42:::benchmark_data_smithi060_111051_object43394:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:14:10.460 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.064776+0000", 2024-09-20T22:14:10.460 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 240.552371166, 2024-09-20T22:14:10.460 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.43944195600000002, 2024-09-20T22:14:10.460 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.460 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.460 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:10.460 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.460 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:10.460 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:10.460 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163393 2024-09-20T22:14:10.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.460 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.460 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.460 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064776+0000", 2024-09-20T22:14:10.461 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.461 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.461 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064776+0000", 2024-09-20T22:14:10.461 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.461 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.461 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064777+0000", 2024-09-20T22:14:10.462 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9500000000000001e-07 2024-09-20T22:14:10.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.462 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.462 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064781+0000", 2024-09-20T22:14:10.462 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9870000000000001e-06 2024-09-20T22:14:10.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.462 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.462 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064791+0000", 2024-09-20T22:14:10.462 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0417999999999999e-05 2024-09-20T22:14:10.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.463 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.463 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.463 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064797+0000", 2024-09-20T22:14:10.463 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4040000000000003e-06 2024-09-20T22:14:10.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.463 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.463 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.463 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064838+0000", 2024-09-20T22:14:10.463 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1591000000000002e-05 2024-09-20T22:14:10.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.463 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.463 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:10.463 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064843+0000", 2024-09-20T22:14:10.464 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9470000000000002e-06 2024-09-20T22:14:10.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.464 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.464 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493235+0000", 2024-09-20T22:14:10.464 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.42839186899999998 2024-09-20T22:14:10.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.464 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:10.464 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493336+0000", 2024-09-20T22:14:10.464 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100923 2024-09-20T22:14:10.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:14:10.465 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493659+0000", 2024-09-20T22:14:10.465 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00032341599999999998 2024-09-20T22:14:10.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:10.465 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.499928+0000", 2024-09-20T22:14:10.465 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0062687730000000001 2024-09-20T22:14:10.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.465 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.503797+0000", 2024-09-20T22:14:10.465 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0038686020000000001 2024-09-20T22:14:10.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.466 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.466 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.504160+0000", 2024-09-20T22:14:10.466 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00036300900000000002 2024-09-20T22:14:10.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.466 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:10.466 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.504192+0000", 2024-09-20T22:14:10.466 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2592000000000002e-05 2024-09-20T22:14:10.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.466 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.467 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.504218+0000", 2024-09-20T22:14:10.467 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.563e-05 2024-09-20T22:14:10.467 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.467 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.467 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.467 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.467 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:125848 9.6 9:604ae09f:::benchmark_data_smithi060_140132_object15040:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:14:10.467 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:13:30.323134+0000", 2024-09-20T22:14:10.467 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 39.294012895999998, 2024-09-20T22:14:10.467 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.39324260900000002, 2024-09-20T22:14:10.467 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.467 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.467 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:10.468 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.468 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:14:10.468 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:14:10.468 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 125848 2024-09-20T22:14:10.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.468 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.468 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.468 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.468 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323134+0000", 2024-09-20T22:14:10.468 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.469 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.469 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323134+0000", 2024-09-20T22:14:10.469 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.469 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.469 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323135+0000", 2024-09-20T22:14:10.469 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1399999999999997e-07 2024-09-20T22:14:10.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.469 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.469 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323137+0000", 2024-09-20T22:14:10.470 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1789999999999998e-06 2024-09-20T22:14:10.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.470 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.470 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.470 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323143+0000", 2024-09-20T22:14:10.470 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1469999999999998e-06 2024-09-20T22:14:10.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.470 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.470 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.470 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323149+0000", 2024-09-20T22:14:10.470 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2639999999999997e-06 2024-09-20T22:14:10.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.470 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.471 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.471 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323210+0000", 2024-09-20T22:14:10.471 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0537e-05 2024-09-20T22:14:10.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.471 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.471 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:10.471 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323333+0000", 2024-09-20T22:14:10.471 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012330500000000001 2024-09-20T22:14:10.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.471 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.471 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:14:10.471 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323444+0000", 2024-09-20T22:14:10.471 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000110562 2024-09-20T22:14:10.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.472 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:10.472 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.326284+0000", 2024-09-20T22:14:10.472 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028407409999999999 2024-09-20T22:14:10.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.472 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.472 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.328793+0000", 2024-09-20T22:14:10.472 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0025082680000000001 2024-09-20T22:14:10.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.472 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.472 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.716294+0000", 2024-09-20T22:14:10.472 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.38750077100000002 2024-09-20T22:14:10.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.473 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:10.473 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.716331+0000", 2024-09-20T22:14:10.473 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7419999999999997e-05 2024-09-20T22:14:10.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.473 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.473 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.716377+0000", 2024-09-20T22:14:10.473 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6001000000000002e-05 2024-09-20T22:14:10.473 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.473 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.473 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.473 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:125853 9.6 9:65bf9cb5:::benchmark_data_smithi060_140132_object15045:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:14:10.474 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:13:30.329387+0000", 2024-09-20T22:14:10.474 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 39.287760233, 2024-09-20T22:14:10.474 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.39097777700000003, 2024-09-20T22:14:10.474 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.474 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.474 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:10.474 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.474 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:14:10.474 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:14:10.474 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 125853 2024-09-20T22:14:10.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.474 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.474 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.475 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.475 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329387+0000", 2024-09-20T22:14:10.475 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.475 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.475 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329387+0000", 2024-09-20T22:14:10.475 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.476 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.476 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329388+0000", 2024-09-20T22:14:10.476 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3990000000000001e-06 2024-09-20T22:14:10.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.476 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.476 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329395+0000", 2024-09-20T22:14:10.476 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.161e-06 2024-09-20T22:14:10.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.476 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.476 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329407+0000", 2024-09-20T22:14:10.476 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2323999999999999e-05 2024-09-20T22:14:10.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.477 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.477 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329414+0000", 2024-09-20T22:14:10.477 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.289e-06 2024-09-20T22:14:10.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.477 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.477 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329463+0000", 2024-09-20T22:14:10.477 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8658000000000001e-05 2024-09-20T22:14:10.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.477 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:10.477 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329589+0000", 2024-09-20T22:14:10.477 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012579900000000001 2024-09-20T22:14:10.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.478 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:14:10.478 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329695+0000", 2024-09-20T22:14:10.478 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000105933 2024-09-20T22:14:10.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.478 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.478 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.335795+0000", 2024-09-20T22:14:10.478 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061008720000000002 2024-09-20T22:14:10.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.478 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:10.478 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.335858+0000", 2024-09-20T22:14:10.478 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2285000000000004e-05 2024-09-20T22:14:10.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.479 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.479 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720262+0000", 2024-09-20T22:14:10.479 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.38440416300000002 2024-09-20T22:14:10.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.479 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:10.479 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720315+0000", 2024-09-20T22:14:10.479 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2985000000000002e-05 2024-09-20T22:14:10.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.479 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.479 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720365+0000", 2024-09-20T22:14:10.479 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9908999999999999e-05 2024-09-20T22:14:10.480 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.480 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.480 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.480 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:125877 9.6 9:618bafd1:::benchmark_data_smithi060_140132_object15069:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:14:10.480 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:14:10.480 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 39.272058657000002, 2024-09-20T22:14:10.480 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.37546249399999998, 2024-09-20T22:14:10.480 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:10.480 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:10.480 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:10.480 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:10.481 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:14:10.481 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:14:10.481 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 125877 2024-09-20T22:14:10.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.481 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:10.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:10.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:14:10.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:10.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:14:10.482 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:10.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.482 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:10.482 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:14:10.482 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1999999999999999e-07 2024-09-20T22:14:10.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.482 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:10.483 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345092+0000", 2024-09-20T22:14:10.483 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2899999999999998e-06 2024-09-20T22:14:10.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.483 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:10.483 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345101+0000", 2024-09-20T22:14:10.483 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1550000000000007e-06 2024-09-20T22:14:10.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.483 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:10.483 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345107+0000", 2024-09-20T22:14:10.483 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2609999999999999e-06 2024-09-20T22:14:10.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.484 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:10.484 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345160+0000", 2024-09-20T22:14:10.484 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.287e-05 2024-09-20T22:14:10.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.484 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:10.484 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345269+0000", 2024-09-20T22:14:10.484 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000108795 2024-09-20T22:14:10.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.484 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:14:10.484 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345395+0000", 2024-09-20T22:14:10.484 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012654800000000001 2024-09-20T22:14:10.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.485 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:10.485 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.348668+0000", 2024-09-20T22:14:10.485 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0032731729999999999 2024-09-20T22:14:10.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.485 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.485 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.351790+0000", 2024-09-20T22:14:10.485 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003121567 2024-09-20T22:14:10.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.485 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:10.485 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720464+0000", 2024-09-20T22:14:10.485 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.36867422 2024-09-20T22:14:10.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.486 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:10.486 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720511+0000", 2024-09-20T22:14:10.486 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7076999999999997e-05 2024-09-20T22:14:10.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:10.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:10.486 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:10.486 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720551+0000", 2024-09-20T22:14:10.486 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9918e-05 2024-09-20T22:14:10.486 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.486 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.486 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.486 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:10.486 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:10.486 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:14:10.487 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:10.508 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:10.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:10.509 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:10.580 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:14:10.610 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:10.643 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:10.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:10.644 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:10.744 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:10.745 INFO:teuthology.orchestra.run.smithi060.stderr:set pool 1 pgp_num to 11 2024-09-20T22:14:10.773 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:10.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:10.774 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:10.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[29694]: 9.b scrub ok 2024-09-20T22:14:10.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[29694]: 5.13 scrub starts 2024-09-20T22:14:10.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[29694]: 5.13 scrub ok 2024-09-20T22:14:10.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[29694]: 3.7 scrub starts 2024-09-20T22:14:10.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[29694]: 3.7 scrub ok 2024-09-20T22:14:10.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/995776990' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]: dispatch 2024-09-20T22:14:10.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]: dispatch 2024-09-20T22:14:10.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[29694]: 4.c scrub starts 2024-09-20T22:14:10.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[29694]: 4.c scrub ok 2024-09-20T22:14:10.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[29694]: pgmap v604: 158 pgs: 158 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 572 B/s rd, 0 B/s wr, 952 op/s 2024-09-20T22:14:10.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[29694]: 9.7 scrub starts 2024-09-20T22:14:10.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[40290]: 9.b scrub ok 2024-09-20T22:14:10.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[40290]: 5.13 scrub starts 2024-09-20T22:14:10.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[40290]: 5.13 scrub ok 2024-09-20T22:14:10.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[40290]: 3.7 scrub starts 2024-09-20T22:14:10.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[40290]: 3.7 scrub ok 2024-09-20T22:14:10.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/995776990' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]: dispatch 2024-09-20T22:14:10.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]: dispatch 2024-09-20T22:14:10.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[40290]: 4.c scrub starts 2024-09-20T22:14:10.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[40290]: 4.c scrub ok 2024-09-20T22:14:10.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[40290]: pgmap v604: 158 pgs: 158 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 572 B/s rd, 0 B/s wr, 952 op/s 2024-09-20T22:14:10.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:10 smithi060 ceph-mon[40290]: 9.7 scrub starts 2024-09-20T22:14:10.875 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:10.920 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:10.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:10.921 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:10 smithi067 ceph-mon[33511]: 9.b scrub ok 2024-09-20T22:14:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:10 smithi067 ceph-mon[33511]: 5.13 scrub starts 2024-09-20T22:14:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:10 smithi067 ceph-mon[33511]: 5.13 scrub ok 2024-09-20T22:14:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:10 smithi067 ceph-mon[33511]: 3.7 scrub starts 2024-09-20T22:14:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:10 smithi067 ceph-mon[33511]: 3.7 scrub ok 2024-09-20T22:14:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:10 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/995776990' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]: dispatch 2024-09-20T22:14:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:10 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]: dispatch 2024-09-20T22:14:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:10 smithi067 ceph-mon[33511]: 4.c scrub starts 2024-09-20T22:14:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:10 smithi067 ceph-mon[33511]: 4.c scrub ok 2024-09-20T22:14:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:10 smithi067 ceph-mon[33511]: pgmap v604: 158 pgs: 158 active+clean; 3.5 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 572 B/s rd, 0 B/s wr, 952 op/s 2024-09-20T22:14:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:10 smithi067 ceph-mon[33511]: 9.7 scrub starts 2024-09-20T22:14:11.022 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:11.058 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:11.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:11.059 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:11.160 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:11.182 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:11.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:11.183 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:11.284 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:11.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:11.321 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:11.321 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:11.421 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:11.462 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:11.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:11.463 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:11.564 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:11.586 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:11.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:11.586 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:11.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:11.732 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:11.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:11.734 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:11.834 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:11.868 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:11.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:11.869 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:11.970 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:11.981 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:11.991 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:11.991 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:11.992 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:12.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:11 smithi067 ceph-mon[33511]: 5.6 scrub starts 2024-09-20T22:14:12.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:11 smithi067 ceph-mon[33511]: 5.6 scrub ok 2024-09-20T22:14:12.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:11 smithi067 ceph-mon[33511]: 9.9 scrub starts 2024-09-20T22:14:12.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:11 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]': finished 2024-09-20T22:14:12.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:11 smithi067 ceph-mon[33511]: osdmap e350: 8 total, 8 up, 8 in 2024-09-20T22:14:12.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:11 smithi067 ceph-mon[33511]: 3.5 scrub starts 2024-09-20T22:14:12.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:11 smithi067 ceph-mon[33511]: 3.5 scrub ok 2024-09-20T22:14:12.035 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:11 smithi060 ceph-mon[40290]: 5.6 scrub starts 2024-09-20T22:14:12.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:11 smithi060 ceph-mon[40290]: 5.6 scrub ok 2024-09-20T22:14:12.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:11 smithi060 ceph-mon[40290]: 9.9 scrub starts 2024-09-20T22:14:12.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:11 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]': finished 2024-09-20T22:14:12.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:11 smithi060 ceph-mon[40290]: osdmap e350: 8 total, 8 up, 8 in 2024-09-20T22:14:12.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:11 smithi060 ceph-mon[40290]: 3.5 scrub starts 2024-09-20T22:14:12.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:11 smithi060 ceph-mon[40290]: 3.5 scrub ok 2024-09-20T22:14:12.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:11 smithi060 ceph-mon[29694]: 5.6 scrub starts 2024-09-20T22:14:12.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:11 smithi060 ceph-mon[29694]: 5.6 scrub ok 2024-09-20T22:14:12.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:11 smithi060 ceph-mon[29694]: 9.9 scrub starts 2024-09-20T22:14:12.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:11 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "11"}]': finished 2024-09-20T22:14:12.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:11 smithi060 ceph-mon[29694]: osdmap e350: 8 total, 8 up, 8 in 2024-09-20T22:14:12.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:11 smithi060 ceph-mon[29694]: 3.5 scrub starts 2024-09-20T22:14:12.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:11 smithi060 ceph-mon[29694]: 3.5 scrub ok 2024-09-20T22:14:12.093 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:12.132 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:12.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:12.133 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:12.233 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:12.263 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:12.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:12.264 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:12.365 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:12.403 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:12.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:12.404 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:12.505 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:12.530 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:12.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:12.531 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:12.558 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:14:12.632 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:12.662 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:12.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:12.663 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:12.754 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:14:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:14:12.765 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:12 smithi060 ceph-mon[29694]: 4.1e scrub starts 2024-09-20T22:14:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:12 smithi060 ceph-mon[29694]: 4.1e scrub ok 2024-09-20T22:14:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:12 smithi060 ceph-mon[29694]: 9.9 scrub ok 2024-09-20T22:14:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:12 smithi060 ceph-mon[29694]: pgmap v606: 158 pgs: 2 active+clean+scrubbing, 156 active+clean; 3.2 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.24k op/s 2024-09-20T22:14:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:12 smithi060 ceph-mon[29694]: 3.a scrub starts 2024-09-20T22:14:12.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:12 smithi060 ceph-mon[29694]: 3.a scrub ok 2024-09-20T22:14:12.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:12 smithi060 ceph-mon[29694]: 9.7 scrub ok 2024-09-20T22:14:12.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:12 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:14:12.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:12 smithi060 ceph-mon[29694]: 9.f scrub starts 2024-09-20T22:14:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:12 smithi060 ceph-mon[40290]: 4.1e scrub starts 2024-09-20T22:14:12.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:12 smithi060 ceph-mon[40290]: 4.1e scrub ok 2024-09-20T22:14:12.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:12 smithi060 ceph-mon[40290]: 9.9 scrub ok 2024-09-20T22:14:12.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:12 smithi060 ceph-mon[40290]: pgmap v606: 158 pgs: 2 active+clean+scrubbing, 156 active+clean; 3.2 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.24k op/s 2024-09-20T22:14:12.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:12 smithi060 ceph-mon[40290]: 3.a scrub starts 2024-09-20T22:14:12.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:12 smithi060 ceph-mon[40290]: 3.a scrub ok 2024-09-20T22:14:12.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:12 smithi060 ceph-mon[40290]: 9.7 scrub ok 2024-09-20T22:14:12.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:12 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:14:12.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:12 smithi060 ceph-mon[40290]: 9.f scrub starts 2024-09-20T22:14:12.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:12.819 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:12.819 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:12.920 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:12.952 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:12.952 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:12.953 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:13.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:12 smithi067 ceph-mon[33511]: 4.1e scrub starts 2024-09-20T22:14:13.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:12 smithi067 ceph-mon[33511]: 4.1e scrub ok 2024-09-20T22:14:13.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:12 smithi067 ceph-mon[33511]: 9.9 scrub ok 2024-09-20T22:14:13.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:12 smithi067 ceph-mon[33511]: pgmap v606: 158 pgs: 2 active+clean+scrubbing, 156 active+clean; 3.2 GiB data, 14 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.24k op/s 2024-09-20T22:14:13.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:12 smithi067 ceph-mon[33511]: 3.a scrub starts 2024-09-20T22:14:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:12 smithi067 ceph-mon[33511]: 3.a scrub ok 2024-09-20T22:14:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:12 smithi067 ceph-mon[33511]: 9.7 scrub ok 2024-09-20T22:14:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:12 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:14:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:12 smithi067 ceph-mon[33511]: 9.f scrub starts 2024-09-20T22:14:13.034 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:13.053 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:13.110 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:13.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:13.111 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:13.212 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:13.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:13.240 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:13.240 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:13.341 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:13.371 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:13.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:13.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:13.473 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:13.495 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:13.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:13.496 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:13.597 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:13.637 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:13.638 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:13.638 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:13.690 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:14:13.691 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:14:13.732 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:14:13.732 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:14:13.739 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:13.756 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:14:13.760 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:14:13.761 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:14:13.761 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:13.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:13.762 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:13.778 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:14:13.779 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:14:13.830 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:14:13.830 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:14:13.863 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:13.879 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:14:13.879 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:14:13.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:13.896 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:13.896 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:13.909 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:14:13.910 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:14:13.940 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:14:13.941 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:14:13.996 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:13 smithi067 ceph-mon[33511]: 4.12 scrub starts 2024-09-20T22:14:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:13 smithi067 ceph-mon[33511]: 4.12 scrub ok 2024-09-20T22:14:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:13 smithi067 ceph-mon[33511]: 5.5 scrub starts 2024-09-20T22:14:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:13 smithi067 ceph-mon[33511]: 5.5 scrub ok 2024-09-20T22:14:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:13 smithi067 ceph-mon[33511]: 5.9 scrub starts 2024-09-20T22:14:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:13 smithi067 ceph-mon[33511]: 5.9 scrub ok 2024-09-20T22:14:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:13 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/573358602' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:13 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:13 smithi067 ceph-mon[33511]: 5.1 scrub starts 2024-09-20T22:14:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:13 smithi067 ceph-mon[33511]: 5.1 scrub ok 2024-09-20T22:14:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:13 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2581125147' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:14.030 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:14.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:14.031 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:14.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[29694]: 4.12 scrub starts 2024-09-20T22:14:14.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[29694]: 4.12 scrub ok 2024-09-20T22:14:14.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[29694]: 5.5 scrub starts 2024-09-20T22:14:14.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[29694]: 5.5 scrub ok 2024-09-20T22:14:14.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[29694]: 5.9 scrub starts 2024-09-20T22:14:14.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[29694]: 5.9 scrub ok 2024-09-20T22:14:14.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/573358602' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:14.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:14.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[29694]: 5.1 scrub starts 2024-09-20T22:14:14.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[29694]: 5.1 scrub ok 2024-09-20T22:14:14.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2581125147' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:14.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[40290]: 4.12 scrub starts 2024-09-20T22:14:14.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[40290]: 4.12 scrub ok 2024-09-20T22:14:14.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[40290]: 5.5 scrub starts 2024-09-20T22:14:14.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[40290]: 5.5 scrub ok 2024-09-20T22:14:14.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[40290]: 5.9 scrub starts 2024-09-20T22:14:14.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[40290]: 5.9 scrub ok 2024-09-20T22:14:14.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/573358602' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:14.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:14.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[40290]: 5.1 scrub starts 2024-09-20T22:14:14.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[40290]: 5.1 scrub ok 2024-09-20T22:14:14.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:13 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2581125147' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:14.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:14.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:14.175 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:14.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:14.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:14.315 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:14.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:14.315 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:14.416 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:14.446 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:14.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:14.447 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:14.547 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:14.581 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:14.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:14.582 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:14.682 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:14.722 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:14.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:14.723 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:14.824 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:14.856 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:14.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:14.857 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:14.958 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:14.995 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:14.996 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:14.996 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:14 smithi067 ceph-mon[33511]: 6.1d scrub starts 2024-09-20T22:14:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:14 smithi067 ceph-mon[33511]: 6.1d scrub ok 2024-09-20T22:14:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:14 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:14:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:14 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:14:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:14 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:14:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:14 smithi067 ceph-mon[33511]: osdmap e351: 8 total, 8 up, 8 in 2024-09-20T22:14:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:14 smithi067 ceph-mon[33511]: 9.f scrub ok 2024-09-20T22:14:15.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:14 smithi067 ceph-mon[33511]: pgmap v608: 158 pgs: 2 active+clean+scrubbing, 156 active+clean; 3.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 786 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-20T22:14:15.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:14 smithi067 ceph-mon[33511]: 5.7 scrub starts 2024-09-20T22:14:15.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:14 smithi067 ceph-mon[33511]: 5.7 scrub ok 2024-09-20T22:14:15.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[29694]: 6.1d scrub starts 2024-09-20T22:14:15.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[29694]: 6.1d scrub ok 2024-09-20T22:14:15.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:14:15.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:14:15.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:14:15.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[29694]: osdmap e351: 8 total, 8 up, 8 in 2024-09-20T22:14:15.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[29694]: 9.f scrub ok 2024-09-20T22:14:15.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[29694]: pgmap v608: 158 pgs: 2 active+clean+scrubbing, 156 active+clean; 3.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 786 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-20T22:14:15.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[29694]: 5.7 scrub starts 2024-09-20T22:14:15.040 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[29694]: 5.7 scrub ok 2024-09-20T22:14:15.040 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[40290]: 6.1d scrub starts 2024-09-20T22:14:15.040 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[40290]: 6.1d scrub ok 2024-09-20T22:14:15.040 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:14:15.041 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:14:15.041 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:14:15.041 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[40290]: osdmap e351: 8 total, 8 up, 8 in 2024-09-20T22:14:15.041 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[40290]: 9.f scrub ok 2024-09-20T22:14:15.041 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[40290]: pgmap v608: 158 pgs: 2 active+clean+scrubbing, 156 active+clean; 3.2 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 786 B/s rd, 0 B/s wr, 1.01k op/s 2024-09-20T22:14:15.041 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[40290]: 5.7 scrub starts 2024-09-20T22:14:15.041 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:14 smithi060 ceph-mon[40290]: 5.7 scrub ok 2024-09-20T22:14:15.096 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:15.127 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:15.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:15.128 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:15.228 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:15.260 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:15.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:15.261 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:15.361 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:15.380 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:14:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:14:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:14:15.394 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:15.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:15.395 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:15.495 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:15.532 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:15.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:15.533 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:15.634 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:15.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:15.672 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:15.672 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:15.773 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:15.802 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:15.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:15.803 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:15.882 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T22:14:15.903 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:15.937 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:15.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:15.938 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:16.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:15 smithi060 ceph-mon[29694]: 4.16 scrub starts 2024-09-20T22:14:16.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:15 smithi060 ceph-mon[29694]: 4.16 scrub ok 2024-09-20T22:14:16.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:15 smithi060 ceph-mon[29694]: 4.1f scrub starts 2024-09-20T22:14:16.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:15 smithi060 ceph-mon[29694]: 4.1f scrub ok 2024-09-20T22:14:16.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:15 smithi060 ceph-mon[40290]: 4.16 scrub starts 2024-09-20T22:14:16.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:15 smithi060 ceph-mon[40290]: 4.16 scrub ok 2024-09-20T22:14:16.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:15 smithi060 ceph-mon[40290]: 4.1f scrub starts 2024-09-20T22:14:16.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:15 smithi060 ceph-mon[40290]: 4.1f scrub ok 2024-09-20T22:14:16.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:16.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:16.073 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:16.073 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:16.174 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:16.223 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:16.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:16.224 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:16.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:15 smithi067 ceph-mon[33511]: 4.16 scrub starts 2024-09-20T22:14:16.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:15 smithi067 ceph-mon[33511]: 4.16 scrub ok 2024-09-20T22:14:16.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:15 smithi067 ceph-mon[33511]: 4.1f scrub starts 2024-09-20T22:14:16.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:15 smithi067 ceph-mon[33511]: 4.1f scrub ok 2024-09-20T22:14:16.326 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:16.328 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:14:16.361 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:16.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:16.361 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:16.462 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:16.497 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:16.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:16.499 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:16.599 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:16.634 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:16.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:16.635 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:16.735 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:16.769 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:14:16.777 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:16.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:16.778 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:16.879 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:16.912 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:16.913 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:16.913 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:16.950 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:14:16.950 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:14:16.950 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-09-20T22:14:16.950 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-20T22:14:17.014 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:17.069 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:17.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:17.070 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:17.105 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:16 smithi067 ceph-mon[33511]: 4.10 scrub starts 2024-09-20T22:14:17.105 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:16 smithi067 ceph-mon[33511]: 4.10 scrub ok 2024-09-20T22:14:17.105 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:16 smithi067 ceph-mon[33511]: 5.19 scrub starts 2024-09-20T22:14:17.105 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:16 smithi067 ceph-mon[33511]: 5.19 scrub ok 2024-09-20T22:14:17.105 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:16 smithi067 ceph-mon[33511]: 4.3 scrub starts 2024-09-20T22:14:17.105 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:16 smithi067 ceph-mon[33511]: 4.3 scrub ok 2024-09-20T22:14:17.105 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:16 smithi067 ceph-mon[33511]: 3.e scrub starts 2024-09-20T22:14:17.106 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:16 smithi067 ceph-mon[33511]: 3.e scrub ok 2024-09-20T22:14:17.106 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:16 smithi067 ceph-mon[33511]: pgmap v609: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s 2024-09-20T22:14:17.153 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:14:17.153 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:14:17.153 INFO:teuthology.orchestra.run.smithi067.stdout: "num_ops": 0 2024-09-20T22:14:17.153 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:14:17.171 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:17.231 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:17.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:17.232 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:17.258 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:17.272 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:14:17.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:16 smithi060 ceph-mon[29694]: 4.10 scrub starts 2024-09-20T22:14:17.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:16 smithi060 ceph-mon[29694]: 4.10 scrub ok 2024-09-20T22:14:17.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:16 smithi060 ceph-mon[29694]: 5.19 scrub starts 2024-09-20T22:14:17.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:16 smithi060 ceph-mon[29694]: 5.19 scrub ok 2024-09-20T22:14:17.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:16 smithi060 ceph-mon[29694]: 4.3 scrub starts 2024-09-20T22:14:17.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:16 smithi060 ceph-mon[29694]: 4.3 scrub ok 2024-09-20T22:14:17.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:16 smithi060 ceph-mon[29694]: 3.e scrub starts 2024-09-20T22:14:17.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:16 smithi060 ceph-mon[29694]: 3.e scrub ok 2024-09-20T22:14:17.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:16 smithi060 ceph-mon[29694]: pgmap v609: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s 2024-09-20T22:14:17.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:16 smithi060 ceph-mon[40290]: 4.10 scrub starts 2024-09-20T22:14:17.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:16 smithi060 ceph-mon[40290]: 4.10 scrub ok 2024-09-20T22:14:17.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:16 smithi060 ceph-mon[40290]: 5.19 scrub starts 2024-09-20T22:14:17.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:16 smithi060 ceph-mon[40290]: 5.19 scrub ok 2024-09-20T22:14:17.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:16 smithi060 ceph-mon[40290]: 4.3 scrub starts 2024-09-20T22:14:17.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:16 smithi060 ceph-mon[40290]: 4.3 scrub ok 2024-09-20T22:14:17.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:16 smithi060 ceph-mon[40290]: 3.e scrub starts 2024-09-20T22:14:17.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:16 smithi060 ceph-mon[40290]: 3.e scrub ok 2024-09-20T22:14:17.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:16 smithi060 ceph-mon[40290]: pgmap v609: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s 2024-09-20T22:14:17.333 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:17.364 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:17.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:17.365 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:17.466 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:17.496 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:17.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:17.497 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:17.598 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:17.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:17.653 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:17.653 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:17.753 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:17.793 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:17.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:17.794 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:17.881 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T22:14:17.896 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:17.932 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:17.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:17.933 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:18.034 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:18.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:18.070 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:18.071 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:18.171 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:18.207 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:18.208 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:18.208 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:18.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:17 smithi067 ceph-mon[33511]: 9.4 scrub starts 2024-09-20T22:14:18.276 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:14:18.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:17 smithi060 ceph-mon[29694]: 9.4 scrub starts 2024-09-20T22:14:18.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:17 smithi060 ceph-mon[40290]: 9.4 scrub starts 2024-09-20T22:14:18.309 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:18.345 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:18.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:18.347 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:18.448 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:18.483 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:18.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:18.484 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:18.586 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:18.626 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:18.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:18.627 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:18.728 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:18.761 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:18.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:18.762 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:18.863 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:18.887 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:18.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:18.888 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:18.919 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:14:18.919 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:14:18.919 INFO:teuthology.orchestra.run.smithi067.stdout: "complaint_time": 30, 2024-09-20T22:14:18.919 INFO:teuthology.orchestra.run.smithi067.stdout: "num_blocked_ops": 0 2024-09-20T22:14:18.919 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:14:18.990 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:19.024 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:19.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:19.025 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:19.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:18 smithi060 ceph-mon[29694]: 5.1e scrub starts 2024-09-20T22:14:19.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:18 smithi060 ceph-mon[29694]: 5.1e scrub ok 2024-09-20T22:14:19.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:18 smithi060 ceph-mon[29694]: 9.4 scrub ok 2024-09-20T22:14:19.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:18 smithi060 ceph-mon[29694]: pgmap v610: 158 pgs: 158 active+clean; 2.9 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-20T22:14:19.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:18 smithi060 ceph-mon[40290]: 5.1e scrub starts 2024-09-20T22:14:19.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:18 smithi060 ceph-mon[40290]: 5.1e scrub ok 2024-09-20T22:14:19.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:18 smithi060 ceph-mon[40290]: 9.4 scrub ok 2024-09-20T22:14:19.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:18 smithi060 ceph-mon[40290]: pgmap v610: 158 pgs: 158 active+clean; 2.9 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-20T22:14:19.126 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:19.158 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:19.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:19.160 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:19.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:19.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:18 smithi067 ceph-mon[33511]: 5.1e scrub starts 2024-09-20T22:14:19.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:18 smithi067 ceph-mon[33511]: 5.1e scrub ok 2024-09-20T22:14:19.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:18 smithi067 ceph-mon[33511]: 9.4 scrub ok 2024-09-20T22:14:19.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:18 smithi067 ceph-mon[33511]: pgmap v610: 158 pgs: 158 active+clean; 2.9 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-20T22:14:19.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:19.288 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:19.288 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:19.389 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:19.435 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:19.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:19.436 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:19.537 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:19.569 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:19.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:19.570 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:19.671 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:19.705 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:19.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:19.706 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:19.732 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T22:14:19.807 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:19.832 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:19.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:19.833 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:19.934 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:19.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:19.972 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:19.972 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:19 smithi067 ceph-mon[33511]: 2.1 scrub starts 2024-09-20T22:14:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:19 smithi067 ceph-mon[33511]: 2.1 scrub ok 2024-09-20T22:14:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:19 smithi067 ceph-mon[33511]: 6.11 scrub starts 2024-09-20T22:14:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:19 smithi067 ceph-mon[33511]: 6.11 scrub ok 2024-09-20T22:14:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:19 smithi067 ceph-mon[33511]: 4.1a scrub starts 2024-09-20T22:14:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:19 smithi067 ceph-mon[33511]: 4.1a scrub ok 2024-09-20T22:14:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:19 smithi067 ceph-mon[33511]: 6.1f scrub starts 2024-09-20T22:14:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:19 smithi067 ceph-mon[33511]: 6.1f scrub ok 2024-09-20T22:14:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:19 smithi060 ceph-mon[29694]: 2.1 scrub starts 2024-09-20T22:14:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:19 smithi060 ceph-mon[29694]: 2.1 scrub ok 2024-09-20T22:14:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:19 smithi060 ceph-mon[29694]: 6.11 scrub starts 2024-09-20T22:14:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:19 smithi060 ceph-mon[29694]: 6.11 scrub ok 2024-09-20T22:14:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:19 smithi060 ceph-mon[29694]: 4.1a scrub starts 2024-09-20T22:14:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:19 smithi060 ceph-mon[29694]: 4.1a scrub ok 2024-09-20T22:14:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:19 smithi060 ceph-mon[29694]: 6.1f scrub starts 2024-09-20T22:14:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:19 smithi060 ceph-mon[29694]: 6.1f scrub ok 2024-09-20T22:14:20.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:19 smithi060 ceph-mon[40290]: 2.1 scrub starts 2024-09-20T22:14:20.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:19 smithi060 ceph-mon[40290]: 2.1 scrub ok 2024-09-20T22:14:20.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:19 smithi060 ceph-mon[40290]: 6.11 scrub starts 2024-09-20T22:14:20.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:19 smithi060 ceph-mon[40290]: 6.11 scrub ok 2024-09-20T22:14:20.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:19 smithi060 ceph-mon[40290]: 4.1a scrub starts 2024-09-20T22:14:20.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:19 smithi060 ceph-mon[40290]: 4.1a scrub ok 2024-09-20T22:14:20.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:19 smithi060 ceph-mon[40290]: 6.1f scrub starts 2024-09-20T22:14:20.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:19 smithi060 ceph-mon[40290]: 6.1f scrub ok 2024-09-20T22:14:20.073 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:20.120 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:20.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:20.121 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:20.145 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:14:20.221 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:20.259 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:20.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:20.260 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:20.360 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:20.400 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:20.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:20.401 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:20.502 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:20.535 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:20.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:20.537 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:20.637 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:20.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:20.676 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:20.676 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:20.777 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:20.783 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:14:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:14:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:14:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:14:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227608 7.d 7:b27d2ce4:::benchmark_data_smithi060_78800_object94575:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-20T22:14:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:53.779361+0000", 2024-09-20T22:14:20.783 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 446.99765679900003, 2024-09-20T22:14:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1886413760000001, 2024-09-20T22:14:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:14:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:14:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:14:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227608 2024-09-20T22:14:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.779361+0000", 2024-09-20T22:14:20.784 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.779361+0000", 2024-09-20T22:14:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.779362+0000", 2024-09-20T22:14:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1459999999999999e-06 2024-09-20T22:14:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.785 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.779367+0000", 2024-09-20T22:14:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5419999999999999e-06 2024-09-20T22:14:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.779377+0000", 2024-09-20T22:14:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0079e-05 2024-09-20T22:14:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.779389+0000", 2024-09-20T22:14:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2194e-05 2024-09-20T22:14:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.786 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.779518+0000", 2024-09-20T22:14:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012875300000000001 2024-09-20T22:14:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.779646+0000", 2024-09-20T22:14:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012852100000000001 2024-09-20T22:14:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0", 2024-09-20T22:14:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.779799+0000", 2024-09-20T22:14:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015320199999999999 2024-09-20T22:14:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.787 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.782341+0000", 2024-09-20T22:14:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0025415059999999998 2024-09-20T22:14:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.811986+0000", 2024-09-20T22:14:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.029645448000000001 2024-09-20T22:14:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.811997+0000", 2024-09-20T22:14:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0920999999999999e-05 2024-09-20T22:14:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.788 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.817158+0000", 2024-09-20T22:14:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0051604279999999 2024-09-20T22:14:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.817175+0000", 2024-09-20T22:14:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7240000000000001e-05 2024-09-20T22:14:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.967946+0000", 2024-09-20T22:14:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1507709829999999 2024-09-20T22:14:20.789 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.968002+0000", 2024-09-20T22:14:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6412999999999997e-05 2024-09-20T22:14:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227637 7.d 7:b79d0cc9:::benchmark_data_smithi060_78800_object94604:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:14:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:53.852068+0000", 2024-09-20T22:14:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 446.92495026699999, 2024-09-20T22:14:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2539511239999999, 2024-09-20T22:14:20.790 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:14:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:14:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227637 2024-09-20T22:14:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.852068+0000", 2024-09-20T22:14:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.791 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.852068+0000", 2024-09-20T22:14:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.852069+0000", 2024-09-20T22:14:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.344e-06 2024-09-20T22:14:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.852074+0000", 2024-09-20T22:14:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4199999999999998e-06 2024-09-20T22:14:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.792 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.852088+0000", 2024-09-20T22:14:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4076999999999999e-05 2024-09-20T22:14:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.852103+0000", 2024-09-20T22:14:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4445e-05 2024-09-20T22:14:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.852156+0000", 2024-09-20T22:14:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2781999999999997e-05 2024-09-20T22:14:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.852163+0000", 2024-09-20T22:14:20.793 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4540000000000001e-06 2024-09-20T22:14:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.817498+0000", 2024-09-20T22:14:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.96533542000000006 2024-09-20T22:14:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.817508+0000", 2024-09-20T22:14:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3880000000000006e-06 2024-09-20T22:14:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.968187+0000", 2024-09-20T22:14:20.794 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.150679123 2024-09-20T22:14:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.968310+0000", 2024-09-20T22:14:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000122705 2024-09-20T22:14:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1", 2024-09-20T22:14:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.968443+0000", 2024-09-20T22:14:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013305200000000001 2024-09-20T22:14:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.973981+0000", 2024-09-20T22:14:20.795 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0055382620000000004 2024-09-20T22:14:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.974033+0000", 2024-09-20T22:14:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2258e-05 2024-09-20T22:14:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.105913+0000", 2024-09-20T22:14:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.13187998300000001 2024-09-20T22:14:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.105957+0000", 2024-09-20T22:14:20.796 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4206000000000003e-05 2024-09-20T22:14:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.106019+0000", 2024-09-20T22:14:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1204999999999994e-05 2024-09-20T22:14:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227638 7.d 7:b7ed5d18:::benchmark_data_smithi060_78800_object94605:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:14:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:53.852128+0000", 2024-09-20T22:14:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 446.92488935799997, 2024-09-20T22:14:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.254054215, 2024-09-20T22:14:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.797 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:14:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:14:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227638 2024-09-20T22:14:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.852128+0000", 2024-09-20T22:14:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.798 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.852128+0000", 2024-09-20T22:14:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.852129+0000", 2024-09-20T22:14:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9599999999999999e-07 2024-09-20T22:14:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.852132+0000", 2024-09-20T22:14:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1130000000000001e-06 2024-09-20T22:14:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.799 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.852140+0000", 2024-09-20T22:14:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5360000000000004e-06 2024-09-20T22:14:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.852145+0000", 2024-09-20T22:14:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5860000000000004e-06 2024-09-20T22:14:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.852197+0000", 2024-09-20T22:14:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2327999999999998e-05 2024-09-20T22:14:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.800 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.852209+0000", 2024-09-20T22:14:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1729000000000001e-05 2024-09-20T22:14:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.817544+0000", 2024-09-20T22:14:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.965334319 2024-09-20T22:14:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.817552+0000", 2024-09-20T22:14:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2470000000000008e-06 2024-09-20T22:14:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.801 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.971095+0000", 2024-09-20T22:14:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1535431389999999 2024-09-20T22:14:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.971214+0000", 2024-09-20T22:14:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000119005 2024-09-20T22:14:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1", 2024-09-20T22:14:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.971352+0000", 2024-09-20T22:14:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013839000000000001 2024-09-20T22:14:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.802 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.974108+0000", 2024-09-20T22:14:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0027554400000000001 2024-09-20T22:14:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.976872+0000", 2024-09-20T22:14:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002764318 2024-09-20T22:14:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.106063+0000", 2024-09-20T22:14:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.12919091899999999 2024-09-20T22:14:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:20.803 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.106112+0000", 2024-09-20T22:14:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8596e-05 2024-09-20T22:14:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.106183+0000", 2024-09-20T22:14:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1054000000000001e-05 2024-09-20T22:14:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227645 7.d 7:b26f0111:::benchmark_data_smithi060_78800_object94612:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:14:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:53.895174+0000", 2024-09-20T22:14:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 446.88184394400002, 2024-09-20T22:14:20.804 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2111581349999998, 2024-09-20T22:14:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:14:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:14:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227645 2024-09-20T22:14:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.895174+0000", 2024-09-20T22:14:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.805 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.895174+0000", 2024-09-20T22:14:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.895175+0000", 2024-09-20T22:14:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5499999999999996e-07 2024-09-20T22:14:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.895179+0000", 2024-09-20T22:14:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7919999999999999e-06 2024-09-20T22:14:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.806 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.895186+0000", 2024-09-20T22:14:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8529999999999992e-06 2024-09-20T22:14:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.895191+0000", 2024-09-20T22:14:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4839999999999998e-06 2024-09-20T22:14:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.895253+0000", 2024-09-20T22:14:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2370000000000001e-05 2024-09-20T22:14:20.807 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.895264+0000", 2024-09-20T22:14:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.078e-05 2024-09-20T22:14:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.817614+0000", 2024-09-20T22:14:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.92235028399999996 2024-09-20T22:14:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.817625+0000", 2024-09-20T22:14:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0519999999999999e-05 2024-09-20T22:14:20.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.974161+0000", 2024-09-20T22:14:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1565359749999999 2024-09-20T22:14:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.809 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.974294+0000", 2024-09-20T22:14:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013324100000000001 2024-09-20T22:14:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1", 2024-09-20T22:14:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.974426+0000", 2024-09-20T22:14:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013184399999999999 2024-09-20T22:14:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.980422+0000", 2024-09-20T22:14:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0059960159999999998 2024-09-20T22:14:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.810 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.985531+0000", 2024-09-20T22:14:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0051092289999999999 2024-09-20T22:14:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.106222+0000", 2024-09-20T22:14:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.12069086399999999 2024-09-20T22:14:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.106280+0000", 2024-09-20T22:14:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7961999999999998e-05 2024-09-20T22:14:20.811 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.106332+0000", 2024-09-20T22:14:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1965999999999997e-05 2024-09-20T22:14:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227661 7.d 7:b23bd03b:::benchmark_data_smithi060_78800_object94628:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:14:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:53.920877+0000", 2024-09-20T22:14:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 446.85614055500002, 2024-09-20T22:14:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1856612110000002, 2024-09-20T22:14:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.812 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:14:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:14:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227661 2024-09-20T22:14:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.920877+0000", 2024-09-20T22:14:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.813 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.920877+0000", 2024-09-20T22:14:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.920879+0000", 2024-09-20T22:14:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3659999999999999e-06 2024-09-20T22:14:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.920883+0000", 2024-09-20T22:14:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8300000000000003e-06 2024-09-20T22:14:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.814 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.920895+0000", 2024-09-20T22:14:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1851999999999999e-05 2024-09-20T22:14:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.920901+0000", 2024-09-20T22:14:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2789999999999998e-06 2024-09-20T22:14:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.920966+0000", 2024-09-20T22:14:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5219e-05 2024-09-20T22:14:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.815 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.920978+0000", 2024-09-20T22:14:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2198000000000001e-05 2024-09-20T22:14:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.817654+0000", 2024-09-20T22:14:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.896676009 2024-09-20T22:14:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.817662+0000", 2024-09-20T22:14:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1249999999999993e-06 2024-09-20T22:14:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.976903+0000", 2024-09-20T22:14:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1592406340000001 2024-09-20T22:14:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.977662+0000", 2024-09-20T22:14:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000759164 2024-09-20T22:14:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1", 2024-09-20T22:14:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.977806+0000", 2024-09-20T22:14:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014448299999999999 2024-09-20T22:14:20.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.992849+0000", 2024-09-20T22:14:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.015042157 2024-09-20T22:14:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.091664+0000", 2024-09-20T22:14:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.098815333000000005 2024-09-20T22:14:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.106374+0000", 2024-09-20T22:14:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.014709636 2024-09-20T22:14:20.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.106442+0000", 2024-09-20T22:14:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.86e-05 2024-09-20T22:14:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.106538+0000", 2024-09-20T22:14:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6325999999999999e-05 2024-09-20T22:14:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227670 7.d 7:b1d0b44c:::benchmark_data_smithi060_78800_object94637:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e146)", 2024-09-20T22:14:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:54.952219+0000", 2024-09-20T22:14:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 445.82479855899999, 2024-09-20T22:14:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.164760378, 2024-09-20T22:14:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:14:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:14:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227670 2024-09-20T22:14:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.820 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.952219+0000", 2024-09-20T22:14:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.952219+0000", 2024-09-20T22:14:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.952220+0000", 2024-09-20T22:14:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9800000000000002e-07 2024-09-20T22:14:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.952224+0000", 2024-09-20T22:14:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7910000000000002e-06 2024-09-20T22:14:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.952235+0000", 2024-09-20T22:14:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0842999999999999e-05 2024-09-20T22:14:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.952243+0000", 2024-09-20T22:14:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6750000000000002e-06 2024-09-20T22:14:20.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.952294+0000", 2024-09-20T22:14:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1752000000000002e-05 2024-09-20T22:14:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.952302+0000", 2024-09-20T22:14:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6750000000000002e-06 2024-09-20T22:14:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.980458+0000", 2024-09-20T22:14:20.823 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.028156262 2024-09-20T22:14:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.980572+0000", 2024-09-20T22:14:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011331 2024-09-20T22:14:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1", 2024-09-20T22:14:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.980704+0000", 2024-09-20T22:14:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013234600000000001 2024-09-20T22:14:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.824 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.992856+0000", 2024-09-20T22:14:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.012151832 2024-09-20T22:14:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.106580+0000", 2024-09-20T22:14:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.113724542 2024-09-20T22:14:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.116861+0000", 2024-09-20T22:14:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.010280777 2024-09-20T22:14:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:20.825 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.116898+0000", 2024-09-20T22:14:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7314999999999998e-05 2024-09-20T22:14:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.116980+0000", 2024-09-20T22:14:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1260000000000005e-05 2024-09-20T22:14:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:235116 7.19 7:99a4d54a:::benchmark_data_smithi060_78800_object102083:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e149)", 2024-09-20T22:14:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:05.148364+0000", 2024-09-20T22:14:20.826 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 435.62865344099998, 2024-09-20T22:14:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98949542000000001, 2024-09-20T22:14:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:14:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:14:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 235116 2024-09-20T22:14:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:05.148364+0000", 2024-09-20T22:14:20.827 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:05.148364+0000", 2024-09-20T22:14:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:05.148366+0000", 2024-09-20T22:14:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2270000000000001e-06 2024-09-20T22:14:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:05.148370+0000", 2024-09-20T22:14:20.828 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6430000000000003e-06 2024-09-20T22:14:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:05.148381+0000", 2024-09-20T22:14:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0708e-05 2024-09-20T22:14:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:05.148393+0000", 2024-09-20T22:14:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1775e-05 2024-09-20T22:14:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:05.148488+0000", 2024-09-20T22:14:20.829 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5346999999999994e-05 2024-09-20T22:14:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:05.148497+0000", 2024-09-20T22:14:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6240000000000008e-06 2024-09-20T22:14:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:06.131671+0000", 2024-09-20T22:14:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98317455499999995 2024-09-20T22:14:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.830 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:06.131801+0000", 2024-09-20T22:14:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012978299999999999 2024-09-20T22:14:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:14:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:06.132156+0000", 2024-09-20T22:14:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00035473699999999999 2024-09-20T22:14:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:06.134066+0000", 2024-09-20T22:14:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0019102909999999999 2024-09-20T22:14:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.831 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:06.137746+0000", 2024-09-20T22:14:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0036803500000000002 2024-09-20T22:14:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:06.137795+0000", 2024-09-20T22:14:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8365999999999997e-05 2024-09-20T22:14:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:06.137835+0000", 2024-09-20T22:14:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9910000000000002e-05 2024-09-20T22:14:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.832 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:06.137860+0000", 2024-09-20T22:14:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.5103999999999999e-05 2024-09-20T22:14:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105663 8.19 8:9a7b6ea1:::benchmark_data_smithi060_111051_object105662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:14:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.546306+0000", 2024-09-20T22:14:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 312.23071149100002, 2024-09-20T22:14:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.071300715, 2024-09-20T22:14:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.833 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105663 2024-09-20T22:14:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.546306+0000", 2024-09-20T22:14:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.834 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.546306+0000", 2024-09-20T22:14:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.546308+0000", 2024-09-20T22:14:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1579999999999999e-06 2024-09-20T22:14:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.546347+0000", 2024-09-20T22:14:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.8219000000000001e-05 2024-09-20T22:14:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.835 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.546371+0000", 2024-09-20T22:14:20.836 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.4712e-05 2024-09-20T22:14:20.836 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.836 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.836 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.836 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.546391+0000", 2024-09-20T22:14:20.836 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9898000000000002e-05 2024-09-20T22:14:20.836 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.836 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.836 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.760159+0000", 2024-09-20T22:14:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.21376741699999999 2024-09-20T22:14:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.760173+0000", 2024-09-20T22:14:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3869999999999999e-05 2024-09-20T22:14:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.611215+0000", 2024-09-20T22:14:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.85104213399999995 2024-09-20T22:14:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.837 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.611327+0000", 2024-09-20T22:14:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011246100000000001 2024-09-20T22:14:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,5", 2024-09-20T22:14:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.611571+0000", 2024-09-20T22:14:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000243706 2024-09-20T22:14:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.617336+0000", 2024-09-20T22:14:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0057650169999999999 2024-09-20T22:14:20.838 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.617408+0000", 2024-09-20T22:14:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2268999999999995e-05 2024-09-20T22:14:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.617520+0000", 2024-09-20T22:14:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000112067 2024-09-20T22:14:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.617571+0000", 2024-09-20T22:14:20.839 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0447e-05 2024-09-20T22:14:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.617607+0000", 2024-09-20T22:14:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6340000000000001e-05 2024-09-20T22:14:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105666 8.15 8:ad619d28:::benchmark_data_smithi060_111051_object105665:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:14:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.570331+0000", 2024-09-20T22:14:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 312.20668703899997, 2024-09-20T22:14:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0444536209999999, 2024-09-20T22:14:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.840 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105666 2024-09-20T22:14:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.570331+0000", 2024-09-20T22:14:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.570331+0000", 2024-09-20T22:14:20.841 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.570334+0000", 2024-09-20T22:14:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.8229999999999999e-06 2024-09-20T22:14:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.570387+0000", 2024-09-20T22:14:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3412000000000003e-05 2024-09-20T22:14:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.570419+0000", 2024-09-20T22:14:20.842 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2280000000000003e-05 2024-09-20T22:14:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.570435+0000", 2024-09-20T22:14:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5265e-05 2024-09-20T22:14:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.745782+0000", 2024-09-20T22:14:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.175347111 2024-09-20T22:14:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.843 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.745796+0000", 2024-09-20T22:14:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4178e-05 2024-09-20T22:14:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.600893+0000", 2024-09-20T22:14:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.855096992 2024-09-20T22:14:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.601413+0000", 2024-09-20T22:14:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00051982700000000005 2024-09-20T22:14:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.844 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 4,7", 2024-09-20T22:14:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.601756+0000", 2024-09-20T22:14:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00034325700000000002 2024-09-20T22:14:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.603923+0000", 2024-09-20T22:14:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002167329 2024-09-20T22:14:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.608479+0000", 2024-09-20T22:14:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0045553820000000002 2024-09-20T22:14:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.845 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.614677+0000", 2024-09-20T22:14:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061983100000000003 2024-09-20T22:14:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.614736+0000", 2024-09-20T22:14:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9598000000000001e-05 2024-09-20T22:14:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.614784+0000", 2024-09-20T22:14:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7856999999999999e-05 2024-09-20T22:14:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.846 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:187857 8.9 8:977b7bdd:::benchmark_data_smithi060_111051_object67858:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:14:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:29.640677+0000", 2024-09-20T22:14:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 231.13634035699999, 2024-09-20T22:14:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.102806486, 2024-09-20T22:14:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:20.847 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 187857 2024-09-20T22:14:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.640677+0000", 2024-09-20T22:14:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.640677+0000", 2024-09-20T22:14:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.848 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.640679+0000", 2024-09-20T22:14:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3430000000000001e-06 2024-09-20T22:14:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.640682+0000", 2024-09-20T22:14:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.557e-06 2024-09-20T22:14:20.849 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:20.850 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:20.850 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:20.850 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.640691+0000", 2024-09-20T22:14:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1710000000000008e-06 2024-09-20T22:14:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.640699+0000", 2024-09-20T22:14:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5059999999999998e-06 2024-09-20T22:14:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.643753+0000", 2024-09-20T22:14:20.851 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0030544809999999999 2024-09-20T22:14:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.643766+0000", 2024-09-20T22:14:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2227e-05 2024-09-20T22:14:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:30.635192+0000", 2024-09-20T22:14:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.991425899 2024-09-20T22:14:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:30.635203+0000", 2024-09-20T22:14:20.852 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0957e-05 2024-09-20T22:14:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.735750+0000", 2024-09-20T22:14:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.10054739 2024-09-20T22:14:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.735843+0000", 2024-09-20T22:14:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2980999999999996e-05 2024-09-20T22:14:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3,5", 2024-09-20T22:14:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.736139+0000", 2024-09-20T22:14:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000296104 2024-09-20T22:14:20.853 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.742910+0000", 2024-09-20T22:14:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0067707760000000001 2024-09-20T22:14:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.742932+0000", 2024-09-20T22:14:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2181e-05 2024-09-20T22:14:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.854 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743233+0000", 2024-09-20T22:14:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00030141400000000001 2024-09-20T22:14:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743293+0000", 2024-09-20T22:14:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9304e-05 2024-09-20T22:14:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743350+0000", 2024-09-20T22:14:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7015999999999998e-05 2024-09-20T22:14:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.855 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743484+0000", 2024-09-20T22:14:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000134179 2024-09-20T22:14:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:187866 8.9 8:90a8fa22:::benchmark_data_smithi060_111051_object67867:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:14:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:29.665081+0000", 2024-09-20T22:14:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 231.11193657199999, 2024-09-20T22:14:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0789447289999998, 2024-09-20T22:14:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:20.856 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 187866 2024-09-20T22:14:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665081+0000", 2024-09-20T22:14:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665081+0000", 2024-09-20T22:14:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.857 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665082+0000", 2024-09-20T22:14:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9800000000000002e-07 2024-09-20T22:14:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665086+0000", 2024-09-20T22:14:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.36e-06 2024-09-20T22:14:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665096+0000", 2024-09-20T22:14:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0755e-05 2024-09-20T22:14:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.858 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665104+0000", 2024-09-20T22:14:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6029999999999999e-06 2024-09-20T22:14:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665150+0000", 2024-09-20T22:14:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6409000000000002e-05 2024-09-20T22:14:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665158+0000", 2024-09-20T22:14:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9039999999999992e-06 2024-09-20T22:14:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.859 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:30.635228+0000", 2024-09-20T22:14:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.97006970000000003 2024-09-20T22:14:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:30.635238+0000", 2024-09-20T22:14:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0108999999999999e-05 2024-09-20T22:14:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.739124+0000", 2024-09-20T22:14:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1038857790000001 2024-09-20T22:14:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.860 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.739264+0000", 2024-09-20T22:14:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014064700000000001 2024-09-20T22:14:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3,5", 2024-09-20T22:14:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.739542+0000", 2024-09-20T22:14:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00027733800000000002 2024-09-20T22:14:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.742938+0000", 2024-09-20T22:14:20.861 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0033961389999999998 2024-09-20T22:14:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743151+0000", 2024-09-20T22:14:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000212592 2024-09-20T22:14:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743850+0000", 2024-09-20T22:14:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00069932800000000004 2024-09-20T22:14:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.862 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743933+0000", 2024-09-20T22:14:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2720999999999996e-05 2024-09-20T22:14:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743965+0000", 2024-09-20T22:14:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2230000000000001e-05 2024-09-20T22:14:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.744026+0000", 2024-09-20T22:14:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1117000000000005e-05 2024-09-20T22:14:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.863 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:187902 8.9 8:95e80f7a:::benchmark_data_smithi060_111051_object67903:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:14:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:29.720026+0000", 2024-09-20T22:14:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 231.05699203099999, 2024-09-20T22:14:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.024196764, 2024-09-20T22:14:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 187902 2024-09-20T22:14:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.864 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720026+0000", 2024-09-20T22:14:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720026+0000", 2024-09-20T22:14:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720027+0000", 2024-09-20T22:14:20.865 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.093e-06 2024-09-20T22:14:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720031+0000", 2024-09-20T22:14:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0720000000000001e-06 2024-09-20T22:14:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720042+0000", 2024-09-20T22:14:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1280999999999999e-05 2024-09-20T22:14:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.866 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720048+0000", 2024-09-20T22:14:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3870000000000003e-06 2024-09-20T22:14:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720098+0000", 2024-09-20T22:14:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0597999999999999e-05 2024-09-20T22:14:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720107+0000", 2024-09-20T22:14:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3510000000000002e-06 2024-09-20T22:14:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.867 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:30.635262+0000", 2024-09-20T22:14:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.91515579300000005 2024-09-20T22:14:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:30.635270+0000", 2024-09-20T22:14:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.994e-06 2024-09-20T22:14:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.740932+0000", 2024-09-20T22:14:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.105662157 2024-09-20T22:14:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.741060+0000", 2024-09-20T22:14:20.868 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012800800000000001 2024-09-20T22:14:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3,5", 2024-09-20T22:14:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.741319+0000", 2024-09-20T22:14:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025847700000000001 2024-09-20T22:14:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743179+0000", 2024-09-20T22:14:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0018601119999999999 2024-09-20T22:14:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.869 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743556+0000", 2024-09-20T22:14:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00037664000000000003 2024-09-20T22:14:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743893+0000", 2024-09-20T22:14:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00033714699999999999 2024-09-20T22:14:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.744064+0000", 2024-09-20T22:14:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017144400000000001 2024-09-20T22:14:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.870 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.744116+0000", 2024-09-20T22:14:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1873999999999998e-05 2024-09-20T22:14:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.744223+0000", 2024-09-20T22:14:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010633600000000001 2024-09-20T22:14:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.871 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223308 8.9 8:97e7b109:::benchmark_data_smithi060_111051_object103309:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:14:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.506871+0000", 2024-09-20T22:14:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 194.270146501, 2024-09-20T22:14:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.99735096099999998, 2024-09-20T22:14:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223308 2024-09-20T22:14:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.872 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506871+0000", 2024-09-20T22:14:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506871+0000", 2024-09-20T22:14:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506873+0000", 2024-09-20T22:14:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0470000000000001e-06 2024-09-20T22:14:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506882+0000", 2024-09-20T22:14:20.873 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3310000000000004e-06 2024-09-20T22:14:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506904+0000", 2024-09-20T22:14:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.283e-05 2024-09-20T22:14:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506913+0000", 2024-09-20T22:14:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9530000000000005e-06 2024-09-20T22:14:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506975+0000", 2024-09-20T22:14:20.874 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1773000000000006e-05 2024-09-20T22:14:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506982+0000", 2024-09-20T22:14:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2869999999999999e-06 2024-09-20T22:14:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.494954+0000", 2024-09-20T22:14:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98797138799999995 2024-09-20T22:14:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.495095+0000", 2024-09-20T22:14:20.875 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014108 2024-09-20T22:14:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:14:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.495506+0000", 2024-09-20T22:14:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00041153700000000001 2024-09-20T22:14:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.500203+0000", 2024-09-20T22:14:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046968380000000001 2024-09-20T22:14:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.876 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.500238+0000", 2024-09-20T22:14:20.877 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4195e-05 2024-09-20T22:14:20.877 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.877 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.877 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.504167+0000", 2024-09-20T22:14:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0039294819999999998 2024-09-20T22:14:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.504199+0000", 2024-09-20T22:14:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1853999999999997e-05 2024-09-20T22:14:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.504222+0000", 2024-09-20T22:14:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.3366e-05 2024-09-20T22:14:20.878 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223310 8.9 8:94027251:::benchmark_data_smithi060_111051_object103311:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:14:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.507858+0000", 2024-09-20T22:14:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 194.26915990800001, 2024-09-20T22:14:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0231674930000001, 2024-09-20T22:14:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:20.879 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223310 2024-09-20T22:14:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507858+0000", 2024-09-20T22:14:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507858+0000", 2024-09-20T22:14:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.880 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507859+0000", 2024-09-20T22:14:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.78e-07 2024-09-20T22:14:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507864+0000", 2024-09-20T22:14:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8969999999999999e-06 2024-09-20T22:14:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507879+0000", 2024-09-20T22:14:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4919999999999999e-05 2024-09-20T22:14:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.881 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507886+0000", 2024-09-20T22:14:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2119999999999997e-06 2024-09-20T22:14:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507948+0000", 2024-09-20T22:14:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1764e-05 2024-09-20T22:14:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507959+0000", 2024-09-20T22:14:20.882 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1613999999999999e-05 2024-09-20T22:14:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.497481+0000", 2024-09-20T22:14:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98952213899999997 2024-09-20T22:14:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.497555+0000", 2024-09-20T22:14:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4202000000000005e-05 2024-09-20T22:14:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:14:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.497785+0000", 2024-09-20T22:14:20.883 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00022984599999999999 2024-09-20T22:14:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.501438+0000", 2024-09-20T22:14:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0036526850000000001 2024-09-20T22:14:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.504265+0000", 2024-09-20T22:14:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002827319 2024-09-20T22:14:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.530930+0000", 2024-09-20T22:14:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.026664383999999999 2024-09-20T22:14:20.884 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.530981+0000", 2024-09-20T22:14:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1109999999999997e-05 2024-09-20T22:14:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531025+0000", 2024-09-20T22:14:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4523e-05 2024-09-20T22:14:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.885 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223355 8.9 8:94b7fbf7:::benchmark_data_smithi060_111051_object103356:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:14:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.561400+0000", 2024-09-20T22:14:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 194.21561812300001, 2024-09-20T22:14:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.96979188400000005, 2024-09-20T22:14:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223355 2024-09-20T22:14:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.886 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561400+0000", 2024-09-20T22:14:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561400+0000", 2024-09-20T22:14:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561401+0000", 2024-09-20T22:14:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3790000000000001e-06 2024-09-20T22:14:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.887 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561406+0000", 2024-09-20T22:14:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4559999999999999e-06 2024-09-20T22:14:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561419+0000", 2024-09-20T22:14:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2712e-05 2024-09-20T22:14:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561426+0000", 2024-09-20T22:14:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6080000000000001e-06 2024-09-20T22:14:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.888 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561475+0000", 2024-09-20T22:14:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9412000000000001e-05 2024-09-20T22:14:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561482+0000", 2024-09-20T22:14:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7789999999999998e-06 2024-09-20T22:14:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.499040+0000", 2024-09-20T22:14:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93755812199999999 2024-09-20T22:14:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.889 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.499105+0000", 2024-09-20T22:14:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5328999999999997e-05 2024-09-20T22:14:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:14:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.499270+0000", 2024-09-20T22:14:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000164678 2024-09-20T22:14:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.504133+0000", 2024-09-20T22:14:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0048630009999999996 2024-09-20T22:14:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.890 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.511137+0000", 2024-09-20T22:14:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.007004316 2024-09-20T22:14:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531092+0000", 2024-09-20T22:14:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.019954896999999999 2024-09-20T22:14:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531148+0000", 2024-09-20T22:14:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5828999999999997e-05 2024-09-20T22:14:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.891 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531192+0000", 2024-09-20T22:14:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3365999999999998e-05 2024-09-20T22:14:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223360 8.9 8:963e2bee:::benchmark_data_smithi060_111051_object103361:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:14:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.566084+0000", 2024-09-20T22:14:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 194.21093344499999, 2024-09-20T22:14:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.965237028, 2024-09-20T22:14:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.892 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:14:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:14:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223360 2024-09-20T22:14:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566084+0000", 2024-09-20T22:14:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566084+0000", 2024-09-20T22:14:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.893 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566086+0000", 2024-09-20T22:14:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.175e-06 2024-09-20T22:14:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566090+0000", 2024-09-20T22:14:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2080000000000002e-06 2024-09-20T22:14:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566099+0000", 2024-09-20T22:14:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3559999999999999e-06 2024-09-20T22:14:20.894 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566105+0000", 2024-09-20T22:14:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8649999999999998e-06 2024-09-20T22:14:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566150+0000", 2024-09-20T22:14:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5252e-05 2024-09-20T22:14:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566156+0000", 2024-09-20T22:14:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7370000000000003e-06 2024-09-20T22:14:20.895 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.500259+0000", 2024-09-20T22:14:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93410321200000002 2024-09-20T22:14:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.500327+0000", 2024-09-20T22:14:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8270999999999997e-05 2024-09-20T22:14:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:14:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.500502+0000", 2024-09-20T22:14:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000174797 2024-09-20T22:14:20.896 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.504273+0000", 2024-09-20T22:14:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003771019 2024-09-20T22:14:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.511209+0000", 2024-09-20T22:14:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0069359080000000002 2024-09-20T22:14:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531241+0000", 2024-09-20T22:14:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.020031694999999999 2024-09-20T22:14:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.897 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531287+0000", 2024-09-20T22:14:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6060000000000003e-05 2024-09-20T22:14:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531321+0000", 2024-09-20T22:14:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4473e-05 2024-09-20T22:14:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.898 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:62123 9.1 9:879a37c5:::benchmark_data_smithi060_140132_object62122:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e309)", 2024-09-20T22:14:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:12:20.517780+0000", 2024-09-20T22:14:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 120.25923808100001, 2024-09-20T22:14:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0093993530000001, 2024-09-20T22:14:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:14:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:14:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:14:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 62123 2024-09-20T22:14:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.899 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.517780+0000", 2024-09-20T22:14:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.517780+0000", 2024-09-20T22:14:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.517781+0000", 2024-09-20T22:14:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.486e-06 2024-09-20T22:14:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.517964+0000", 2024-09-20T22:14:20.900 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000182542 2024-09-20T22:14:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.517976+0000", 2024-09-20T22:14:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2031999999999999e-05 2024-09-20T22:14:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.517984+0000", 2024-09-20T22:14:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2859999999999999e-06 2024-09-20T22:14:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.518698+0000", 2024-09-20T22:14:20.901 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00071389999999999995 2024-09-20T22:14:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.518708+0000", 2024-09-20T22:14:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0176999999999999e-05 2024-09-20T22:14:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.526085+0000", 2024-09-20T22:14:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.007377027 2024-09-20T22:14:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.527179+0000", 2024-09-20T22:14:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001093903 2024-09-20T22:14:20.902 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.903 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:20 smithi067 ceph-mon[33511]: pgmap v611: 158 pgs: 158 active+clean; 2.8 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.10k op/s 2024-09-20T22:14:20.903 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:14:20.903 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:20 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:14:20.903 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:62141 9.1 9:8c9cf8d8:::benchmark_data_smithi060_140132_object62140:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e309)", 2024-09-20T22:14:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:12:20.531998+0000", 2024-09-20T22:14:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 120.245019755, 2024-09-20T22:14:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0013003810000001, 2024-09-20T22:14:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:14:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:14:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 62141 2024-09-20T22:14:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.904 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.531998+0000", 2024-09-20T22:14:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.531998+0000", 2024-09-20T22:14:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.532000+0000", 2024-09-20T22:14:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5719999999999999e-06 2024-09-20T22:14:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.532075+0000", 2024-09-20T22:14:20.905 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5104999999999993e-05 2024-09-20T22:14:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.532087+0000", 2024-09-20T22:14:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1990999999999999e-05 2024-09-20T22:14:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.532096+0000", 2024-09-20T22:14:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4110000000000002e-06 2024-09-20T22:14:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.906 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.532156+0000", 2024-09-20T22:14:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0000999999999997e-05 2024-09-20T22:14:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.532168+0000", 2024-09-20T22:14:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2194e-05 2024-09-20T22:14:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.527535+0000", 2024-09-20T22:14:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.99536664699999999 2024-09-20T22:14:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.527885+0000", 2024-09-20T22:14:20.907 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00034985700000000002 2024-09-20T22:14:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:14:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.527994+0000", 2024-09-20T22:14:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000109628 2024-09-20T22:14:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533024+0000", 2024-09-20T22:14:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.005029342 2024-09-20T22:14:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533086+0000", 2024-09-20T22:14:20.908 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2079e-05 2024-09-20T22:14:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533178+0000", 2024-09-20T22:14:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2559000000000006e-05 2024-09-20T22:14:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533243+0000", 2024-09-20T22:14:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4882999999999995e-05 2024-09-20T22:14:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533298+0000", 2024-09-20T22:14:20.909 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5111999999999997e-05 2024-09-20T22:14:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:62181 9.1 9:818888ea:::benchmark_data_smithi060_140132_object62180:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e309)", 2024-09-20T22:14:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:12:20.561567+0000", 2024-09-20T22:14:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 120.21545061400001, 2024-09-20T22:14:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.97555155000000005, 2024-09-20T22:14:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.910 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:14:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:14:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 62181 2024-09-20T22:14:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561567+0000", 2024-09-20T22:14:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561567+0000", 2024-09-20T22:14:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561569+0000", 2024-09-20T22:14:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9209999999999999e-06 2024-09-20T22:14:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561613+0000", 2024-09-20T22:14:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4295000000000002e-05 2024-09-20T22:14:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561639+0000", 2024-09-20T22:14:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.5177e-05 2024-09-20T22:14:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.912 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561648+0000", 2024-09-20T22:14:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8020000000000003e-06 2024-09-20T22:14:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561730+0000", 2024-09-20T22:14:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2032000000000003e-05 2024-09-20T22:14:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561743+0000", 2024-09-20T22:14:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2466999999999999e-05 2024-09-20T22:14:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.913 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.529714+0000", 2024-09-20T22:14:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.96797125799999995 2024-09-20T22:14:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.529848+0000", 2024-09-20T22:14:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013433100000000001 2024-09-20T22:14:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:14:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.530029+0000", 2024-09-20T22:14:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000180693 2024-09-20T22:14:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.914 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533139+0000", 2024-09-20T22:14:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0031102299999999999 2024-09-20T22:14:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533147+0000", 2024-09-20T22:14:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7330000000000003e-06 2024-09-20T22:14:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537004+0000", 2024-09-20T22:14:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0038571980000000001 2024-09-20T22:14:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.915 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537067+0000", 2024-09-20T22:14:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2385999999999995e-05 2024-09-20T22:14:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537119+0000", 2024-09-20T22:14:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2027000000000002e-05 2024-09-20T22:14:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:62188 9.1 9:83e4bb26:::benchmark_data_smithi060_140132_object62187:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e309)", 2024-09-20T22:14:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:12:20.566992+0000", 2024-09-20T22:14:20.916 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 120.210025333, 2024-09-20T22:14:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.97032768800000002, 2024-09-20T22:14:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:14:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:14:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:14:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:14:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:14:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:14:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 62188 2024-09-20T22:14:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:14:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:14:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.566992+0000", 2024-09-20T22:14:20.917 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:14:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.566992+0000", 2024-09-20T22:14:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:14:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:14:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.566994+0000", 2024-09-20T22:14:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9410000000000001e-06 2024-09-20T22:14:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:14:20.918 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.567080+0000", 2024-09-20T22:14:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5228999999999993e-05 2024-09-20T22:14:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:14:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.567104+0000", 2024-09-20T22:14:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.4309999999999999e-05 2024-09-20T22:14:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:14:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.567114+0000", 2024-09-20T22:14:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0487e-05 2024-09-20T22:14:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.919 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.567203+0000", 2024-09-20T22:14:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8109000000000006e-05 2024-09-20T22:14:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:14:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.567216+0000", 2024-09-20T22:14:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3040000000000001e-05 2024-09-20T22:14:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:14:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.531707+0000", 2024-09-20T22:14:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.96449152100000002 2024-09-20T22:14:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.920 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:14:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.531800+0000", 2024-09-20T22:14:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3015000000000003e-05 2024-09-20T22:14:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:14:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.531912+0000", 2024-09-20T22:14:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000111776 2024-09-20T22:14:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537181+0000", 2024-09-20T22:14:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0052689709999999999 2024-09-20T22:14:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.921 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:14:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537198+0000", 2024-09-20T22:14:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7142999999999998e-05 2024-09-20T22:14:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537256+0000", 2024-09-20T22:14:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8425999999999999e-05 2024-09-20T22:14:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:14:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537293+0000", 2024-09-20T22:14:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6313000000000003e-05 2024-09-20T22:14:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:14:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:14:20.922 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:14:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537320+0000", 2024-09-20T22:14:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7407e-05 2024-09-20T22:14:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:14:20.923 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:14:20.923 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:14:20.951 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:20.986 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:20.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:20.987 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:21.070 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:20 smithi060 ceph-mon[29694]: pgmap v611: 158 pgs: 158 active+clean; 2.8 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.10k op/s 2024-09-20T22:14:21.070 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:14:21.070 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:20 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:14:21.070 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:20 smithi060 ceph-mon[40290]: pgmap v611: 158 pgs: 158 active+clean; 2.8 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.10k op/s 2024-09-20T22:14:21.070 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:14:21.070 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:20 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:14:21.088 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:21.119 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:21.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:21.120 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:21.170 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:14:21.170 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:14:21.170 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:14:21.170 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:14:21.171 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:14:21.171 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:14:21.171 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:14:21.171 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:14:21.177 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:14:21.177 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:14:21.189 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:21.218 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:14:21.218 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:14:21.220 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:21.251 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:14:21.251 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:14:21.277 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:21.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:21.278 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:21.288 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:14:21.288 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:14:21.379 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:21.419 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:21.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:21.420 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:21.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:21.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:21.571 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:21.571 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:21.672 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:21.712 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:21.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:21.713 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:21.723 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:21.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:21.857 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:21.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:21.858 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:21.959 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:21.993 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:21.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:21.994 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:22.040 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T22:14:22.079 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:21 smithi067 ceph-mon[33511]: 5.d scrub starts 2024-09-20T22:14:22.079 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:21 smithi067 ceph-mon[33511]: 5.d scrub ok 2024-09-20T22:14:22.079 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:21 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/204076013' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:22.079 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:21 smithi067 ceph-mon[33511]: 1.5 deep-scrub starts 2024-09-20T22:14:22.080 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:21 smithi067 ceph-mon[33511]: 1.5 deep-scrub ok 2024-09-20T22:14:22.080 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:21 smithi067 ceph-mon[33511]: 4.5 scrub starts 2024-09-20T22:14:22.080 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:21 smithi067 ceph-mon[33511]: 4.5 scrub ok 2024-09-20T22:14:22.095 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:22.133 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:22.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:22.134 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:22.154 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:21 smithi060 ceph-mon[29694]: 5.d scrub starts 2024-09-20T22:14:22.155 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:21 smithi060 ceph-mon[29694]: 5.d scrub ok 2024-09-20T22:14:22.155 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:21 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/204076013' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:22.155 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:21 smithi060 ceph-mon[29694]: 1.5 deep-scrub starts 2024-09-20T22:14:22.155 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:21 smithi060 ceph-mon[29694]: 1.5 deep-scrub ok 2024-09-20T22:14:22.155 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:21 smithi060 ceph-mon[29694]: 4.5 scrub starts 2024-09-20T22:14:22.155 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:21 smithi060 ceph-mon[29694]: 4.5 scrub ok 2024-09-20T22:14:22.156 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:21 smithi060 ceph-mon[40290]: 5.d scrub starts 2024-09-20T22:14:22.156 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:21 smithi060 ceph-mon[40290]: 5.d scrub ok 2024-09-20T22:14:22.156 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:21 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/204076013' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:22.156 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:21 smithi060 ceph-mon[40290]: 1.5 deep-scrub starts 2024-09-20T22:14:22.156 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:21 smithi060 ceph-mon[40290]: 1.5 deep-scrub ok 2024-09-20T22:14:22.156 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:21 smithi060 ceph-mon[40290]: 4.5 scrub starts 2024-09-20T22:14:22.156 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:21 smithi060 ceph-mon[40290]: 4.5 scrub ok 2024-09-20T22:14:22.235 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:22.276 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:22.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:22.277 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:22.377 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:22.416 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:22.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:22.417 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:22.492 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:14:22.518 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:22.559 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:22.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:22.560 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:22.661 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:22.689 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:14:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:14:22.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:22.691 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:22.691 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:22.792 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:22.844 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:22.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:22.845 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:22.946 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:22.964 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:22 smithi060 ceph-mon[29694]: 3.17 scrub starts 2024-09-20T22:14:22.964 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:22 smithi060 ceph-mon[29694]: 3.17 scrub ok 2024-09-20T22:14:22.964 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:22 smithi060 ceph-mon[29694]: pgmap v612: 158 pgs: 158 active+clean; 2.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-20T22:14:22.964 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:22 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:14:22.964 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:22 smithi060 ceph-mon[29694]: 5.2 scrub starts 2024-09-20T22:14:22.964 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:22 smithi060 ceph-mon[29694]: 5.2 scrub ok 2024-09-20T22:14:22.965 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:22 smithi060 ceph-mon[40290]: 3.17 scrub starts 2024-09-20T22:14:22.965 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:22 smithi060 ceph-mon[40290]: 3.17 scrub ok 2024-09-20T22:14:22.965 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:22 smithi060 ceph-mon[40290]: pgmap v612: 158 pgs: 158 active+clean; 2.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-20T22:14:22.965 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:22 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:14:22.965 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:22 smithi060 ceph-mon[40290]: 5.2 scrub starts 2024-09-20T22:14:22.966 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:22 smithi060 ceph-mon[40290]: 5.2 scrub ok 2024-09-20T22:14:22.992 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:22 smithi067 ceph-mon[33511]: 3.17 scrub starts 2024-09-20T22:14:22.992 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:22 smithi067 ceph-mon[33511]: 3.17 scrub ok 2024-09-20T22:14:22.992 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:22 smithi067 ceph-mon[33511]: pgmap v612: 158 pgs: 158 active+clean; 2.6 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-20T22:14:22.992 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:22 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:14:22.992 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:22 smithi067 ceph-mon[33511]: 5.2 scrub starts 2024-09-20T22:14:22.992 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:22 smithi067 ceph-mon[33511]: 5.2 scrub ok 2024-09-20T22:14:22.993 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:22.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:22.994 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:23.095 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:23.128 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:23.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:23.129 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:23.151 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:14:23.152 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:14:23.230 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:23.264 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:23.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:23.266 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:23.364 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:14:23.365 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:14:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 351, 2024-09-20T22:14:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: "fsid": "3df9647e-779b-11ef-baf5-efdc52797490", 2024-09-20T22:14:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: "created": "2024-09-20T21:57:31.244934+0000", 2024-09-20T22:14:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: "modified": "2024-09-20T22:14:13.750637+0000", 2024-09-20T22:14:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: "last_up_change": "2024-09-20T22:06:53.796679+0000", 2024-09-20T22:14:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: "last_in_change": "2024-09-20T22:12:51.491976+0000", 2024-09-20T22:14:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-20T22:14:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_num": 5799936, 2024-09-20T22:14:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_set": [ 2024-09-20T22:14:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: "pglog_hardlimit", 2024-09-20T22:14:23.366 INFO:teuthology.orchestra.run.smithi060.stdout: "purged_snapdirs", 2024-09-20T22:14:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_deletes", 2024-09-20T22:14:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "sortbitwise" 2024-09-20T22:14:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:14:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_version": 19, 2024-09-20T22:14:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "full_ratio": 0.94999998807907104, 2024-09-20T22:14:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-20T22:14:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-20T22:14:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_snapshot": "", 2024-09-20T22:14:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_max": 9, 2024-09-20T22:14:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "max_osd": 8, 2024-09-20T22:14:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "require_min_compat_client": "luminous", 2024-09-20T22:14:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "min_compat_client": "luminous", 2024-09-20T22:14:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "require_osd_release": "squid", 2024-09-20T22:14:23.367 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_crimson": false, 2024-09-20T22:14:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "pools": [ 2024-09-20T22:14:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 1, 2024-09-20T22:14:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": ".mgr", 2024-09-20T22:14:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:00:35.131125+0000", 2024-09-20T22:14:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:14:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:14:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:14:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:14:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:14:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:14:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:14:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:14:23.368 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:14:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:14:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:14:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:14:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:14:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 11, 2024-09-20T22:14:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 11, 2024-09-20T22:14:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 11, 2024-09-20T22:14:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 11, 2024-09-20T22:14:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 11, 2024-09-20T22:14:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:14:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:14:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:14:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:14:23.369 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:14:23.370 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:14:23.370 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:14:23.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.370 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "350", 2024-09-20T22:14:23.370 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:14:23.370 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:14:23.370 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "197", 2024-09-20T22:14:23.370 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:14:23.370 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:14:23.370 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:14:23.370 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:14:23.370 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:14:23.370 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:14:23.371 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:14:23.371 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:14:23.371 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:14:23.371 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:14:23.371 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:14:23.371 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:14:23.371 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:14:23.371 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:14:23.371 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:14:23.371 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:14:23.371 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:14:23.371 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:14:23.372 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:14:23.372 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:14:23.372 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:14:23.372 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:14:23.372 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:14:23.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.372 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:14:23.372 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:14:23.372 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:14:23.372 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:14:23.373 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:14:23.373 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:14:23.373 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:14:23.373 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:14:23.373 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:14:23.373 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:14:23.373 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:14:23.373 INFO:teuthology.orchestra.run.smithi060.stdout: "options": { 2024-09-20T22:14:23.373 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_max": 32, 2024-09-20T22:14:23.373 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_min": 1 2024-09-20T22:14:23.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.374 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:14:23.374 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": {} 2024-09-20T22:14:23.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.374 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:14:23.374 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:14:23.374 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2.1800000667572021, 2024-09-20T22:14:23.374 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 2.1800000667572021, 2024-09-20T22:14:23.374 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:14:23.374 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2.1800000667572021, 2024-09-20T22:14:23.374 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 2.1800000667572021, 2024-09-20T22:14:23.374 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 0.9326171875, 2024-09-20T22:14:23.374 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-20T22:14:23.374 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-20T22:14:23.375 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.375 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 2, 2024-09-20T22:14:23.375 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "datapool", 2024-09-20T22:14:23.375 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:47.974480+0000", 2024-09-20T22:14:23.375 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 8193, 2024-09-20T22:14:23.375 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-20T22:14:23.375 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:14:23.375 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:14:23.375 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:14:23.375 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:14:23.375 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:14:23.375 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:14:23.376 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:14:23.376 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:14:23.376 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:14:23.376 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:14:23.376 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:14:23.376 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 3, 2024-09-20T22:14:23.376 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 3, 2024-09-20T22:14:23.376 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 3, 2024-09-20T22:14:23.376 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 3, 2024-09-20T22:14:23.376 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 3, 2024-09-20T22:14:23.376 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:14:23.376 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:14:23.376 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:14:23.376 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:14:23.377 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:14:23.377 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:14:23.377 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:14:23.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.377 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "66", 2024-09-20T22:14:23.377 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:14:23.377 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:14:23.377 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:14:23.377 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:14:23.377 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:14:23.377 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 2, 2024-09-20T22:14:23.377 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 66, 2024-09-20T22:14:23.377 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:14:23.377 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:14:23.378 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:14:23.378 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:14:23.378 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:14:23.378 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:14:23.378 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:14:23.378 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:14:23.378 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:14:23.378 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:14:23.378 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:14:23.378 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:14:23.378 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:14:23.378 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:14:23.378 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:14:23.378 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:14:23.379 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:14:23.379 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:14:23.379 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:14:23.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.379 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:14:23.379 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:14:23.379 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:14:23.379 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:14:23.379 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:14:23.379 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:14:23.379 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:14:23.379 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:14:23.380 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:14:23.380 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:14:23.380 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:14:23.380 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:14:23.380 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:14:23.380 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd": {} 2024-09-20T22:14:23.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.380 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:14:23.380 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:14:23.380 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2.6500000953674316, 2024-09-20T22:14:23.380 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 2.6500000953674316, 2024-09-20T22:14:23.381 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T22:14:23.381 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-20T22:14:23.381 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-20T22:14:23.381 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 0.9326171875, 2024-09-20T22:14:23.381 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-20T22:14:23.381 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-20T22:14:23.381 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.381 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 3, 2024-09-20T22:14:23.382 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": ".rgw.root", 2024-09-20T22:14:23.382 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:48.902465+0000", 2024-09-20T22:14:23.382 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:14:23.382 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:14:23.382 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:14:23.382 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:14:23.382 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:14:23.382 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:14:23.382 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:14:23.382 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:14:23.382 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:14:23.382 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:14:23.382 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:14:23.383 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:14:23.383 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:14:23.383 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:14:23.383 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:14:23.383 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:14:23.383 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:14:23.383 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:14:23.383 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:14:23.383 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:14:23.384 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:23.385 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:14:23.385 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:14:23.385 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:14:23.385 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:14:23.385 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:14:23.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.385 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "65", 2024-09-20T22:14:23.385 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:14:23.385 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:14:23.386 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:14:23.386 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:14:23.386 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:14:23.386 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:14:23.386 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:14:23.386 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:14:23.386 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:14:23.386 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:14:23.386 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:14:23.386 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:14:23.387 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:14:23.387 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:14:23.387 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:14:23.387 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:14:23.387 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:14:23.388 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:14:23.388 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:14:23.388 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:14:23.388 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:14:23.388 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:14:23.388 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:14:23.388 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:14:23.388 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:14:23.388 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:14:23.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.388 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:14:23.388 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:14:23.388 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:14:23.388 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:14:23.389 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:14:23.389 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:14:23.389 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:14:23.389 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:14:23.389 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:14:23.389 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:14:23.389 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:14:23.389 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:14:23.389 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:14:23.389 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:14:23.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.389 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:14:23.389 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:14:23.389 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2, 2024-09-20T22:14:23.390 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 2, 2024-09-20T22:14:23.390 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:14:23.390 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2, 2024-09-20T22:14:23.390 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 2, 2024-09-20T22:14:23.390 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 0.9326171875, 2024-09-20T22:14:23.390 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-20T22:14:23.390 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-20T22:14:23.390 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.391 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 4, 2024-09-20T22:14:23.391 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "default.rgw.log", 2024-09-20T22:14:23.391 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:50.620057+0000", 2024-09-20T22:14:23.391 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:14:23.391 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:14:23.392 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:14:23.392 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:14:23.392 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:14:23.392 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:14:23.392 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:14:23.392 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:14:23.392 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:14:23.392 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:14:23.392 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:14:23.392 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:14:23.392 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:14:23.392 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:14:23.392 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:14:23.393 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:14:23.393 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:14:23.393 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:14:23.393 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:14:23.393 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:14:23.393 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:14:23.393 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:14:23.394 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:14:23.394 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:14:23.394 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:14:23.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.394 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "67", 2024-09-20T22:14:23.394 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:14:23.395 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:14:23.395 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:14:23.395 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:14:23.395 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:14:23.395 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:14:23.395 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:14:23.396 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:14:23.396 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:14:23.396 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:14:23.396 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:14:23.396 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:14:23.396 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:14:23.396 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:14:23.396 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:14:23.396 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:14:23.396 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:14:23.396 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:14:23.396 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:14:23.397 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:14:23.397 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:14:23.397 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:14:23.397 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:14:23.397 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:14:23.397 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:14:23.397 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:14:23.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.397 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:14:23.397 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:14:23.397 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:14:23.397 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:14:23.397 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:14:23.397 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:14:23.398 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:14:23.398 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:14:23.398 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:14:23.398 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:14:23.398 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:14:23.398 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:14:23.398 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:14:23.398 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:14:23.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.398 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:14:23.398 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:14:23.398 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2.25, 2024-09-20T22:14:23.398 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 2.25, 2024-09-20T22:14:23.398 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:14:23.399 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2.25, 2024-09-20T22:14:23.399 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 2.25, 2024-09-20T22:14:23.399 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 0.9326171875, 2024-09-20T22:14:23.399 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-20T22:14:23.399 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-20T22:14:23.399 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.399 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 5, 2024-09-20T22:14:23.399 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "default.rgw.control", 2024-09-20T22:14:23.399 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:52.722912+0000", 2024-09-20T22:14:23.399 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:14:23.399 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:14:23.399 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:14:23.400 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:14:23.400 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:14:23.400 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:14:23.400 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:14:23.400 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:14:23.400 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:14:23.400 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:14:23.400 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:14:23.400 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:14:23.400 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:14:23.400 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:14:23.400 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:14:23.400 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:14:23.400 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:14:23.401 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:14:23.401 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:14:23.401 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:14:23.401 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:14:23.401 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:14:23.401 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:14:23.401 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:14:23.401 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:14:23.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.401 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "69", 2024-09-20T22:14:23.401 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:14:23.401 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:14:23.402 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:14:23.402 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:14:23.402 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:14:23.402 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:14:23.402 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:14:23.402 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:14:23.402 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:14:23.402 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:14:23.402 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:14:23.402 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:14:23.402 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:14:23.402 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:14:23.402 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:14:23.403 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:14:23.403 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:14:23.403 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:14:23.403 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:14:23.403 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:14:23.403 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:14:23.403 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:14:23.403 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:14:23.403 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:14:23.403 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:14:23.403 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:14:23.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.403 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:14:23.403 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:14:23.404 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:14:23.404 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:14:23.404 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:14:23.404 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:14:23.404 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:14:23.404 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:14:23.404 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:14:23.404 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:14:23.404 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:14:23.404 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:14:23.404 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:14:23.404 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:14:23.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.404 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:14:23.404 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:14:23.405 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.5, 2024-09-20T22:14:23.405 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.5, 2024-09-20T22:14:23.405 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:14:23.405 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.5, 2024-09-20T22:14:23.405 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.5, 2024-09-20T22:14:23.405 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 0.9326171875, 2024-09-20T22:14:23.405 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-20T22:14:23.405 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-20T22:14:23.405 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.405 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 6, 2024-09-20T22:14:23.406 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "default.rgw.meta", 2024-09-20T22:14:23.406 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:54.794248+0000", 2024-09-20T22:14:23.406 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:14:23.406 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:14:23.406 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:14:23.406 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:14:23.406 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:14:23.406 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:14:23.406 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:14:23.407 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:14:23.407 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:14:23.407 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:14:23.407 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:14:23.407 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:14:23.407 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:14:23.408 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:14:23.408 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:14:23.408 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:14:23.408 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:14:23.408 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:14:23.408 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:14:23.408 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:14:23.408 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:14:23.408 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:14:23.408 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:14:23.408 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:14:23.409 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:14:23.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.409 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "71", 2024-09-20T22:14:23.409 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:14:23.409 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:14:23.409 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:14:23.409 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:14:23.409 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:14:23.409 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:14:23.409 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:14:23.409 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:14:23.409 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:14:23.410 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:14:23.410 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:14:23.410 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:14:23.410 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:14:23.410 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:14:23.410 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:14:23.410 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:14:23.410 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:14:23.410 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:14:23.410 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:14:23.410 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:14:23.410 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:14:23.410 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:14:23.411 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:14:23.411 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:14:23.411 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:14:23.411 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:14:23.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.411 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:14:23.411 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:14:23.411 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:14:23.411 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:14:23.411 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:14:23.411 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:14:23.411 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:14:23.411 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:14:23.411 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:14:23.412 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:14:23.412 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:14:23.412 INFO:teuthology.orchestra.run.smithi060.stdout: "options": { 2024-09-20T22:14:23.412 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_bias": 4 2024-09-20T22:14:23.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.412 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:14:23.412 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:14:23.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.412 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:14:23.412 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:14:23.412 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.5, 2024-09-20T22:14:23.412 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.5, 2024-09-20T22:14:23.412 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:14:23.412 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.5, 2024-09-20T22:14:23.413 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.5, 2024-09-20T22:14:23.413 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 0.9326171875, 2024-09-20T22:14:23.413 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-20T22:14:23.413 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-20T22:14:23.413 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.413 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 9, 2024-09-20T22:14:23.413 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "unique_pool_2", 2024-09-20T22:14:23.413 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:11:33.546088+0000", 2024-09-20T22:14:23.413 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:14:23.413 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:14:23.414 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:14:23.414 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:14:23.414 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:14:23.414 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:14:23.414 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:14:23.414 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:14:23.414 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:14:23.414 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:14:23.414 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:14:23.414 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:14:23.414 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:14:23.414 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 16, 2024-09-20T22:14:23.414 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 16, 2024-09-20T22:14:23.415 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 16, 2024-09-20T22:14:23.415 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 16, 2024-09-20T22:14:23.415 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 16, 2024-09-20T22:14:23.415 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:14:23.415 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:14:23.415 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:14:23.415 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:14:23.415 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:14:23.415 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:14:23.415 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:14:23.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.415 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "295", 2024-09-20T22:14:23.415 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:14:23.415 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:14:23.416 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:14:23.416 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:14:23.416 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:14:23.416 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:14:23.416 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:14:23.416 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:14:23.416 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:14:23.416 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:14:23.416 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:14:23.416 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:14:23.416 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:14:23.416 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:14:23.416 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:14:23.416 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:14:23.417 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:14:23.417 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:14:23.417 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:14:23.417 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:14:23.417 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:14:23.417 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:14:23.417 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:14:23.417 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:14:23.417 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:14:23.417 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:14:23.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.418 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:14:23.418 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:14:23.418 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:14:23.418 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:14:23.418 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:14:23.418 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:14:23.418 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:14:23.418 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:14:23.418 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:14:23.418 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:14:23.418 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:14:23.418 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:14:23.418 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:14:23.418 INFO:teuthology.orchestra.run.smithi060.stdout: "rados": {} 2024-09-20T22:14:23.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.419 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:14:23.419 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:14:23.419 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2, 2024-09-20T22:14:23.419 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 2, 2024-09-20T22:14:23.419 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:14:23.419 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2, 2024-09-20T22:14:23.419 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 2, 2024-09-20T22:14:23.419 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 0.9326171875, 2024-09-20T22:14:23.419 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-20T22:14:23.419 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-20T22:14:23.419 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.419 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.420 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:14:23.420 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:14:23.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.420 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 0, 2024-09-20T22:14:23.420 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "0997c01a-a9bd-421e-abcc-fbcf7abe8512", 2024-09-20T22:14:23.420 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:14:23.420 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:14:23.420 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:14:23.420 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:14:23.420 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:14:23.420 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:14:23.420 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 8, 2024-09-20T22:14:23.420 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 328, 2024-09-20T22:14:23.420 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:14:23.420 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:14:23.421 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:14:23.421 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.421 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.421 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6802", 2024-09-20T22:14:23.421 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:14:23.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.421 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.421 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6803", 2024-09-20T22:14:23.421 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:14:23.421 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.421 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.421 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:14:23.422 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.422 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.422 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6804", 2024-09-20T22:14:23.422 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:14:23.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.422 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.422 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6805", 2024-09-20T22:14:23.422 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:14:23.422 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.422 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.423 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:14:23.423 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.423 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.423 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6808", 2024-09-20T22:14:23.423 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:14:23.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.423 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.423 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6809", 2024-09-20T22:14:23.423 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:14:23.423 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.423 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.424 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:14:23.424 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.425 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.425 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6806", 2024-09-20T22:14:23.425 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:14:23.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.425 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.425 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6807", 2024-09-20T22:14:23.425 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:14:23.425 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.425 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.426 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6803/1395849550", 2024-09-20T22:14:23.426 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6805/1395849550", 2024-09-20T22:14:23.426 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6809/1395849550", 2024-09-20T22:14:23.426 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6807/1395849550", 2024-09-20T22:14:23.426 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:14:23.426 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:14:23.426 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:14:23.426 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.426 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 1, 2024-09-20T22:14:23.427 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "c04bdcb1-d76b-47c8-a38b-c02043df6ba8", 2024-09-20T22:14:23.427 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:14:23.427 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:14:23.427 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:14:23.427 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:14:23.427 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 13, 2024-09-20T22:14:23.427 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 90, 2024-09-20T22:14:23.427 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 145, 2024-09-20T22:14:23.427 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 323, 2024-09-20T22:14:23.427 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 91, 2024-09-20T22:14:23.427 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:14:23.427 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:14:23.428 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.428 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.428 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6810", 2024-09-20T22:14:23.428 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:14:23.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.428 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.428 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.428 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6811", 2024-09-20T22:14:23.428 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:14:23.428 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.428 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.428 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.428 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:14:23.429 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.429 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.429 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6812", 2024-09-20T22:14:23.429 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:14:23.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.429 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.429 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.429 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6813", 2024-09-20T22:14:23.429 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:14:23.429 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.429 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.429 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.429 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:14:23.430 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.430 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.430 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6816", 2024-09-20T22:14:23.430 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:14:23.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.430 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.430 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.430 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6817", 2024-09-20T22:14:23.430 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:14:23.430 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.430 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.430 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.430 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:14:23.431 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.431 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.431 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6814", 2024-09-20T22:14:23.431 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:14:23.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.431 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.431 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.431 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6815", 2024-09-20T22:14:23.431 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:14:23.431 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.431 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.431 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.432 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6811/2970874921", 2024-09-20T22:14:23.432 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6813/2970874921", 2024-09-20T22:14:23.432 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6817/2970874921", 2024-09-20T22:14:23.432 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6815/2970874921", 2024-09-20T22:14:23.432 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:14:23.432 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:14:23.432 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:14:23.432 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.432 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.432 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.432 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 2, 2024-09-20T22:14:23.432 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "32c09d19-eb98-4926-a44e-e7ddee152956", 2024-09-20T22:14:23.432 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:14:23.433 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:14:23.433 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:14:23.433 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:14:23.433 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:14:23.433 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:14:23.433 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 18, 2024-09-20T22:14:23.433 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 323, 2024-09-20T22:14:23.433 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:14:23.433 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:14:23.433 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:14:23.433 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.434 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.434 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6818", 2024-09-20T22:14:23.434 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:14:23.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.434 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.434 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.434 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6819", 2024-09-20T22:14:23.434 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:14:23.434 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.434 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.434 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.435 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:14:23.435 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.435 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.435 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6820", 2024-09-20T22:14:23.435 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:14:23.435 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.435 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.436 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.436 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6821", 2024-09-20T22:14:23.436 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:14:23.436 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.436 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.436 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.436 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:14:23.436 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.436 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.436 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.436 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6824", 2024-09-20T22:14:23.436 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:14:23.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.437 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.437 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6825", 2024-09-20T22:14:23.437 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:14:23.437 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.437 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.437 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.437 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:14:23.437 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.437 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.437 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.438 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6822", 2024-09-20T22:14:23.438 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:14:23.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.438 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.438 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.438 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6823", 2024-09-20T22:14:23.438 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:14:23.438 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.438 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.438 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.439 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6819/3698562274", 2024-09-20T22:14:23.439 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6821/3698562274", 2024-09-20T22:14:23.439 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6825/3698562274", 2024-09-20T22:14:23.439 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6823/3698562274", 2024-09-20T22:14:23.439 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:14:23.439 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:14:23.439 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:14:23.440 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.440 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.440 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.440 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 3, 2024-09-20T22:14:23.440 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "b1c2cb69-8565-49ac-b854-7ea7615c6660", 2024-09-20T22:14:23.440 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:14:23.440 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:14:23.440 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:14:23.440 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 0.4609375, 2024-09-20T22:14:23.440 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:14:23.441 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:14:23.441 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 26, 2024-09-20T22:14:23.441 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 318, 2024-09-20T22:14:23.441 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:14:23.441 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:14:23.441 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:14:23.441 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.441 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.441 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.441 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6826", 2024-09-20T22:14:23.441 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:14:23.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.442 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.442 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6827", 2024-09-20T22:14:23.442 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:14:23.442 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.442 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.442 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.442 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:14:23.442 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.442 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.442 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.443 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6828", 2024-09-20T22:14:23.443 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:14:23.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.443 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.443 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.443 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6829", 2024-09-20T22:14:23.443 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:14:23.443 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.443 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.443 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.443 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:14:23.444 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.444 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.444 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6832", 2024-09-20T22:14:23.444 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:14:23.444 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.444 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.444 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.444 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6833", 2024-09-20T22:14:23.444 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:14:23.444 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.444 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.445 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:14:23.445 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.445 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.445 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6830", 2024-09-20T22:14:23.445 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:14:23.445 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.445 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.445 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.445 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6831", 2024-09-20T22:14:23.445 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:14:23.445 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.446 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.446 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6827/2592322449", 2024-09-20T22:14:23.446 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6829/2592322449", 2024-09-20T22:14:23.446 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6833/2592322449", 2024-09-20T22:14:23.446 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6831/2592322449", 2024-09-20T22:14:23.446 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:14:23.446 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:14:23.446 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:14:23.446 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.446 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.447 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 4, 2024-09-20T22:14:23.447 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "04a547d6-fc48-43b7-a8d7-18df8a83aa96", 2024-09-20T22:14:23.447 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:14:23.447 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:14:23.447 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:14:23.447 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:14:23.447 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:14:23.447 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:14:23.447 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 31, 2024-09-20T22:14:23.447 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 323, 2024-09-20T22:14:23.447 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:14:23.447 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:14:23.447 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:14:23.447 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.448 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.448 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6800", 2024-09-20T22:14:23.448 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:14:23.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.448 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.448 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6801", 2024-09-20T22:14:23.448 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:14:23.448 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.448 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.449 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:14:23.449 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.449 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.449 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6802", 2024-09-20T22:14:23.449 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:14:23.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.449 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.449 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6803", 2024-09-20T22:14:23.449 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:14:23.449 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.449 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.450 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:14:23.450 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.450 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.450 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6806", 2024-09-20T22:14:23.450 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:14:23.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.450 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.450 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6807", 2024-09-20T22:14:23.450 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:14:23.451 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.451 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.451 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:14:23.451 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.451 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.451 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6804", 2024-09-20T22:14:23.451 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:14:23.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.452 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.452 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6805", 2024-09-20T22:14:23.452 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:14:23.452 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.452 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.452 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6801/3243564239", 2024-09-20T22:14:23.452 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6803/3243564239", 2024-09-20T22:14:23.452 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6807/3243564239", 2024-09-20T22:14:23.452 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6805/3243564239", 2024-09-20T22:14:23.452 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:14:23.452 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:14:23.452 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:14:23.452 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.453 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 5, 2024-09-20T22:14:23.453 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "4763aa10-c367-485b-b525-e5241e642b86", 2024-09-20T22:14:23.453 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:14:23.453 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:14:23.453 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:14:23.453 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:14:23.453 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:14:23.453 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:14:23.453 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 39, 2024-09-20T22:14:23.453 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 326, 2024-09-20T22:14:23.453 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:14:23.454 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:14:23.454 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:14:23.454 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.454 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.454 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6808", 2024-09-20T22:14:23.454 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:14:23.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.454 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.454 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6809", 2024-09-20T22:14:23.454 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:14:23.454 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.454 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.455 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:14:23.455 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.455 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.455 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6810", 2024-09-20T22:14:23.455 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:14:23.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.455 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.455 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6811", 2024-09-20T22:14:23.455 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:14:23.455 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.455 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.456 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:14:23.456 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.456 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.456 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6814", 2024-09-20T22:14:23.456 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:14:23.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.456 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.456 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6815", 2024-09-20T22:14:23.456 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:14:23.456 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.456 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.456 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:14:23.456 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.457 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.457 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6812", 2024-09-20T22:14:23.457 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:14:23.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.457 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.457 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6813", 2024-09-20T22:14:23.457 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:14:23.457 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.457 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.457 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6809/1622258887", 2024-09-20T22:14:23.457 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6811/1622258887", 2024-09-20T22:14:23.458 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6815/1622258887", 2024-09-20T22:14:23.458 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6813/1622258887", 2024-09-20T22:14:23.458 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:14:23.458 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:14:23.458 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:14:23.458 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.458 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 6, 2024-09-20T22:14:23.458 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "a359116e-c013-4e2f-8b79-86219516ee16", 2024-09-20T22:14:23.458 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:14:23.458 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:14:23.458 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:14:23.459 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:14:23.459 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:14:23.459 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:14:23.459 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 48, 2024-09-20T22:14:23.459 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 327, 2024-09-20T22:14:23.459 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:14:23.459 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:14:23.459 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:14:23.459 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.459 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.459 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6816", 2024-09-20T22:14:23.459 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:14:23.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.460 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.460 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6817", 2024-09-20T22:14:23.460 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:14:23.460 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.460 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.460 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:14:23.460 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.460 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.460 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6818", 2024-09-20T22:14:23.460 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:14:23.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.461 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.461 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6819", 2024-09-20T22:14:23.461 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:14:23.461 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.461 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.461 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:14:23.461 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.461 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.461 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6822", 2024-09-20T22:14:23.461 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:14:23.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.462 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.462 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6823", 2024-09-20T22:14:23.462 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:14:23.462 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.462 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.463 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:14:23.463 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.463 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.463 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6820", 2024-09-20T22:14:23.463 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:14:23.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.463 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.463 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6821", 2024-09-20T22:14:23.463 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:14:23.463 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.464 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.464 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6817/1187114", 2024-09-20T22:14:23.464 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6819/1187114", 2024-09-20T22:14:23.464 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6823/1187114", 2024-09-20T22:14:23.464 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6821/1187114", 2024-09-20T22:14:23.464 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:14:23.464 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:14:23.464 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:14:23.464 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.464 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 7, 2024-09-20T22:14:23.465 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "a4bf4188-f133-4a31-8897-d7e8182248d7", 2024-09-20T22:14:23.465 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:14:23.465 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:14:23.465 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:14:23.465 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:14:23.465 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:14:23.465 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:14:23.465 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 57, 2024-09-20T22:14:23.465 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 326, 2024-09-20T22:14:23.465 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:14:23.466 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:14:23.466 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:14:23.467 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.467 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.467 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6824", 2024-09-20T22:14:23.467 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:14:23.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.467 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.467 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6825", 2024-09-20T22:14:23.467 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:14:23.467 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.467 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.468 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:14:23.468 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.468 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.468 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6826", 2024-09-20T22:14:23.468 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:14:23.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.468 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.468 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6827", 2024-09-20T22:14:23.468 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:14:23.468 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.468 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.469 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:14:23.469 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.469 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.469 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6830", 2024-09-20T22:14:23.469 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:14:23.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.469 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.469 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6831", 2024-09-20T22:14:23.469 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:14:23.469 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.469 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.469 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:14:23.470 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:14:23.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.470 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:14:23.470 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6828", 2024-09-20T22:14:23.470 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:14:23.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.470 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:14:23.470 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6829", 2024-09-20T22:14:23.470 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:14:23.470 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.470 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.471 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.471 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6825/3021842840", 2024-09-20T22:14:23.471 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6827/3021842840", 2024-09-20T22:14:23.471 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6831/3021842840", 2024-09-20T22:14:23.471 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6829/3021842840", 2024-09-20T22:14:23.471 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:14:23.471 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:14:23.471 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:14:23.471 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.471 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.471 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:14:23.472 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_xinfo": [ 2024-09-20T22:14:23.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.472 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 0, 2024-09-20T22:14:23.472 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:14:23.472 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:14:23.472 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:14:23.472 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:14:23.472 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:14:23.472 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:13:54.778606+0000", 2024-09-20T22:14:23.472 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:14:23.472 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.472 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.473 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 1, 2024-09-20T22:14:23.473 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:14:23.473 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:14:23.473 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:14:23.473 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:14:23.473 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:14:23.473 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:13:13.325543+0000", 2024-09-20T22:14:23.473 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 90 2024-09-20T22:14:23.473 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.473 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.473 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 2, 2024-09-20T22:14:23.473 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:14:23.473 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:14:23.474 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:14:23.474 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:14:23.474 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:14:23.474 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:13:21.848022+0000", 2024-09-20T22:14:23.474 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:14:23.474 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.474 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.474 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 3, 2024-09-20T22:14:23.474 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:14:23.475 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:14:23.475 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:14:23.475 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:14:23.475 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:14:23.475 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:13:41.851461+0000", 2024-09-20T22:14:23.475 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:14:23.475 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.475 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.475 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 4, 2024-09-20T22:14:23.476 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:14:23.476 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:14:23.476 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:14:23.476 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:14:23.476 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:14:23.476 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:13:37.033245+0000", 2024-09-20T22:14:23.476 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:14:23.476 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.476 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.476 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 5, 2024-09-20T22:14:23.476 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:14:23.476 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:14:23.476 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:14:23.477 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:14:23.477 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:14:23.477 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:13:40.296439+0000", 2024-09-20T22:14:23.477 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:14:23.477 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.477 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.477 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 6, 2024-09-20T22:14:23.477 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:14:23.477 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:14:23.477 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:14:23.477 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:14:23.477 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:14:23.477 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:13:20.436570+0000", 2024-09-20T22:14:23.478 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:14:23.478 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.478 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.478 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 7, 2024-09-20T22:14:23.478 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:14:23.478 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:14:23.478 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:14:23.478 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:14:23.478 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:14:23.478 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:13:30.251159+0000", 2024-09-20T22:14:23.478 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:14:23.478 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.478 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:14:23.479 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap": [ 2024-09-20T22:14:23.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.479 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "3.c", 2024-09-20T22:14:23.479 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:14:23.479 INFO:teuthology.orchestra.run.smithi060.stdout: 3, 2024-09-20T22:14:23.479 INFO:teuthology.orchestra.run.smithi060.stdout: 1, 2024-09-20T22:14:23.479 INFO:teuthology.orchestra.run.smithi060.stdout: 6 2024-09-20T22:14:23.479 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.479 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.479 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.479 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.13", 2024-09-20T22:14:23.479 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:14:23.479 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:14:23.479 INFO:teuthology.orchestra.run.smithi060.stdout: 7, 2024-09-20T22:14:23.479 INFO:teuthology.orchestra.run.smithi060.stdout: 6 2024-09-20T22:14:23.480 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.480 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.480 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:14:23.480 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap_items": [ 2024-09-20T22:14:23.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.480 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "3.1a", 2024-09-20T22:14:23.480 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:14:23.480 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.480 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 4, 2024-09-20T22:14:23.480 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 5 2024-09-20T22:14:23.480 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.480 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.481 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "3.1c", 2024-09-20T22:14:23.481 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:14:23.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.481 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 4, 2024-09-20T22:14:23.481 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 2 2024-09-20T22:14:23.481 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.481 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.481 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.481 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "4.b", 2024-09-20T22:14:23.481 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:14:23.481 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.481 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 1, 2024-09-20T22:14:23.482 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 7 2024-09-20T22:14:23.482 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.482 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.482 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "4.f", 2024-09-20T22:14:23.482 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:14:23.482 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.482 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 1, 2024-09-20T22:14:23.482 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 2 2024-09-20T22:14:23.482 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.482 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.482 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.483 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "4.13", 2024-09-20T22:14:23.483 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:14:23.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.483 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 1, 2024-09-20T22:14:23.483 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 2 2024-09-20T22:14:23.483 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.483 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.483 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.483 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "4.1b", 2024-09-20T22:14:23.483 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:14:23.483 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.483 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 1, 2024-09-20T22:14:23.484 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 5 2024-09-20T22:14:23.484 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.484 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.484 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.9", 2024-09-20T22:14:23.484 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:14:23.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.484 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 7, 2024-09-20T22:14:23.484 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 1 2024-09-20T22:14:23.484 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.484 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.484 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.484 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.485 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.11", 2024-09-20T22:14:23.485 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:14:23.485 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.485 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:14:23.485 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 1 2024-09-20T22:14:23.485 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.485 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.486 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.486 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.19", 2024-09-20T22:14:23.486 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:14:23.486 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:23.486 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:14:23.487 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 6 2024-09-20T22:14:23.487 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.487 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:23.487 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.487 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:14:23.487 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap_primaries": [], 2024-09-20T22:14:23.487 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_temp": [], 2024-09-20T22:14:23.487 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_temp": [], 2024-09-20T22:14:23.488 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklist": { 2024-09-20T22:14:23.488 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/890154710": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:14:23.488 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3386526350": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:14:23.488 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/604714621": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:14:23.488 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/604714621": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:14:23.488 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/3151907776": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:14:23.488 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/956264518": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:14:23.489 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/4150585713": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:14:23.489 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3311255314": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:14:23.489 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3965541528": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:14:23.489 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/3734574621": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:14:23.489 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1729862661": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:14:23.489 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/3734574621": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:14:23.489 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3927294026": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:14:23.489 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/3151907776": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:14:23.489 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1871351181": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:14:23.490 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1854455098": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:14:23.490 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/2221262203": "2024-09-21T22:03:19.518405+0000" 2024-09-20T22:14:23.490 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.490 INFO:teuthology.orchestra.run.smithi060.stdout: "range_blocklist": {}, 2024-09-20T22:14:23.490 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profiles": { 2024-09-20T22:14:23.490 INFO:teuthology.orchestra.run.smithi060.stdout: "default": { 2024-09-20T22:14:23.491 INFO:teuthology.orchestra.run.smithi060.stdout: "crush-failure-domain": "osd", 2024-09-20T22:14:23.491 INFO:teuthology.orchestra.run.smithi060.stdout: "k": "2", 2024-09-20T22:14:23.491 INFO:teuthology.orchestra.run.smithi060.stdout: "m": "1", 2024-09-20T22:14:23.491 INFO:teuthology.orchestra.run.smithi060.stdout: "plugin": "jerasure", 2024-09-20T22:14:23.491 INFO:teuthology.orchestra.run.smithi060.stdout: "technique": "reed_sol_van" 2024-09-20T22:14:23.491 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.491 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:23.492 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps_queue": [], 2024-09-20T22:14:23.492 INFO:teuthology.orchestra.run.smithi060.stdout: "new_removed_snaps": [], 2024-09-20T22:14:23.492 INFO:teuthology.orchestra.run.smithi060.stdout: "new_purged_snaps": [], 2024-09-20T22:14:23.492 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_node_flags": {}, 2024-09-20T22:14:23.492 INFO:teuthology.orchestra.run.smithi060.stdout: "device_class_flags": {}, 2024-09-20T22:14:23.492 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode": { 2024-09-20T22:14:23.492 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode_enabled": false, 2024-09-20T22:14:23.493 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_bucket_count": 0, 2024-09-20T22:14:23.493 INFO:teuthology.orchestra.run.smithi060.stdout: "degraded_stretch_mode": 0, 2024-09-20T22:14:23.493 INFO:teuthology.orchestra.run.smithi060.stdout: "recovering_stretch_mode": 0, 2024-09-20T22:14:23.493 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode_bucket": 0 2024-09-20T22:14:23.493 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:23.493 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:14:23.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:23.494 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:23.494 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:23.595 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:23.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:23.633 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:23.733 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:23.771 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:23.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:23.772 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:23.841 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:14:23.874 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:23.906 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T22:14:23.916 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:23.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:23.917 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:24.017 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:24.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:24.057 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:24.057 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:24.158 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:24.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:24.191 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:24.192 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:24.196 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:14:24.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[29694]: 3.19 scrub starts 2024-09-20T22:14:24.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[29694]: 3.19 scrub ok 2024-09-20T22:14:24.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[29694]: 3.13 scrub starts 2024-09-20T22:14:24.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[29694]: 3.13 scrub ok 2024-09-20T22:14:24.262 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/627122220' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:14:24.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:14:24.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1901614353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-20T22:14:24.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[29694]: 9.6 scrub starts 2024-09-20T22:14:24.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[29694]: 1.7 scrub starts 2024-09-20T22:14:24.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[29694]: 1.7 scrub ok 2024-09-20T22:14:24.263 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[40290]: 3.19 scrub starts 2024-09-20T22:14:24.263 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[40290]: 3.19 scrub ok 2024-09-20T22:14:24.263 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[40290]: 3.13 scrub starts 2024-09-20T22:14:24.263 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[40290]: 3.13 scrub ok 2024-09-20T22:14:24.263 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/627122220' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:14:24.263 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:14:24.263 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1901614353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-20T22:14:24.264 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[40290]: 9.6 scrub starts 2024-09-20T22:14:24.264 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[40290]: 1.7 scrub starts 2024-09-20T22:14:24.264 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:23 smithi060 ceph-mon[40290]: 1.7 scrub ok 2024-09-20T22:14:24.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:23 smithi067 ceph-mon[33511]: 3.19 scrub starts 2024-09-20T22:14:24.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:23 smithi067 ceph-mon[33511]: 3.19 scrub ok 2024-09-20T22:14:24.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:23 smithi067 ceph-mon[33511]: 3.13 scrub starts 2024-09-20T22:14:24.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:23 smithi067 ceph-mon[33511]: 3.13 scrub ok 2024-09-20T22:14:24.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:23 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/627122220' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:14:24.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:23 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:14:24.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:23 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1901614353' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-20T22:14:24.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:23 smithi067 ceph-mon[33511]: 9.6 scrub starts 2024-09-20T22:14:24.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:23 smithi067 ceph-mon[33511]: 1.7 scrub starts 2024-09-20T22:14:24.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:23 smithi067 ceph-mon[33511]: 1.7 scrub ok 2024-09-20T22:14:24.292 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:24.343 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:24.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:24.344 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:24.357 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:14:24.444 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:24.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:24.482 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:24.482 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:24.583 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:24.618 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:24.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:24.619 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:24.720 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:24.752 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:24.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:24.753 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:24.854 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:24.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:24.890 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:24.890 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:24.971 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:14:24.971 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:14:24.992 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:25.029 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:25.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:25.030 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:25.131 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:25.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:25.171 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:25.171 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:25.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:24 smithi067 ceph-mon[33511]: 3.b scrub starts 2024-09-20T22:14:25.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:24 smithi067 ceph-mon[33511]: 3.b scrub ok 2024-09-20T22:14:25.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:24 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:14:25.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:24 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:14:25.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:24 smithi067 ceph-mon[33511]: osdmap e352: 8 total, 8 up, 8 in 2024-09-20T22:14:25.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:24 smithi067 ceph-mon[33511]: pgmap v614: 158 pgs: 158 active+clean; 2.5 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-20T22:14:25.272 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:25.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:24 smithi060 ceph-mon[29694]: 3.b scrub starts 2024-09-20T22:14:25.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:24 smithi060 ceph-mon[29694]: 3.b scrub ok 2024-09-20T22:14:25.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:24 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:14:25.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:24 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:14:25.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:24 smithi060 ceph-mon[29694]: osdmap e352: 8 total, 8 up, 8 in 2024-09-20T22:14:25.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:24 smithi060 ceph-mon[29694]: pgmap v614: 158 pgs: 158 active+clean; 2.5 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-20T22:14:25.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:14:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:14:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:14:25.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:24 smithi060 ceph-mon[40290]: 3.b scrub starts 2024-09-20T22:14:25.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:24 smithi060 ceph-mon[40290]: 3.b scrub ok 2024-09-20T22:14:25.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:24 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:14:25.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:24 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:14:25.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:24 smithi060 ceph-mon[40290]: osdmap e352: 8 total, 8 up, 8 in 2024-09-20T22:14:25.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:24 smithi060 ceph-mon[40290]: pgmap v614: 158 pgs: 158 active+clean; 2.5 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.12k op/s 2024-09-20T22:14:25.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:25.312 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:25.312 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:25.413 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:25.448 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:25.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:25.449 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:25.550 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:25.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:25.573 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:25.673 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:25.711 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:25.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:25.712 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:25.800 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T22:14:25.813 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:25.846 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:25.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:25.847 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:25.948 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:25.975 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:25.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:25.976 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:26.077 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:26.133 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:26.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:26.135 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:26.216 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:14:26.236 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:26.284 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:26.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:26.285 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:26.386 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:26.422 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:26.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:26.423 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:26.524 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:26.576 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:26.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:26.580 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:26.681 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:26.684 INFO:tasks.thrashosds.thrasher:j is {'epoch': 351, 'fsid': '3df9647e-779b-11ef-baf5-efdc52797490', 'created': '2024-09-20T21:57:31.244934+0000', 'modified': '2024-09-20T22:14:13.750637+0000', 'last_up_change': '2024-09-20T22:06:53.796679+0000', 'last_in_change': '2024-09-20T22:12:51.491976+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 9, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-20T22:00:35.131125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 11, 'pg_placement_num': 11, 'pg_placement_num_target': 11, 'pg_num_target': 11, 'pg_num_pending': 11, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '350', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '197', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.180000066757202, 'score_stable': 2.180000066757202, 'optimal_score': 1, 'raw_score_acting': 2.180000066757202, 'raw_score_stable': 2.180000066757202, 'primary_affinity_weighted': 0.9326171875, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-20T22:02:47.974480+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6500000953674316, 'score_stable': 2.6500000953674316, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 0.9326171875, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-20T22:02:48.902465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9326171875, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-20T22:02:50.620057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9326171875, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-20T22:02:52.722912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9326171875, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-20T22:02:54.794248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9326171875, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 9, 'pool_name': 'unique_pool_2', 'create_time': '2024-09-20T22:11:33.546088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '295', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9326171875, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}], 'osds': [{'osd': 0, 'uuid': '0997c01a-a9bd-421e-abcc-fbcf7abe8512', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 328, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6802', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6803', 'nonce': 1395849550}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6804', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6805', 'nonce': 1395849550}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6808', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6809', 'nonce': 1395849550}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6806', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6807', 'nonce': 1395849550}]}, 'public_addr': '172.21.15.60:6803/1395849550', 'cluster_addr': '172.21.15.60:6805/1395849550', 'heartbeat_back_addr': '172.21.15.60:6809/1395849550', 'heartbeat_front_addr': '172.21.15.60:6807/1395849550', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c04bdcb1-d76b-47c8-a38b-c02043df6ba8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 90, 'up_from': 145, 'up_thru': 323, 'down_at': 91, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6810', 'nonce': 2970874921}, {'type': 'v1', 'addr': '172.21.15.60:6811', 'nonce': 2970874921}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6812', 'nonce': 2970874921}, {'type': 'v1', 'addr': '172.21.15.60:6813', 'nonce': 2970874921}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6816', 'nonce': 2970874921}, {'type': 'v1', 'addr': '172.21.15.60:6817', 'nonce': 2970874921}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6814', 'nonce': 2970874921}, {'type': 'v1', 'addr': '172.21.15.60:6815', 'nonce': 2970874921}]}, 'public_addr': '172.21.15.60:6811/2970874921', 'cluster_addr': '172.21.15.60:6813/2970874921', 'heartbeat_back_addr': '172.21.15.60:6817/2970874921', 'heartbeat_front_addr': '172.21.15.60:6815/2970874921', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '32c09d19-eb98-4926-a44e-e7ddee152956', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 323, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6818', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6819', 'nonce': 3698562274}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6820', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6821', 'nonce': 3698562274}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6824', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6825', 'nonce': 3698562274}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6822', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6823', 'nonce': 3698562274}]}, 'public_addr': '172.21.15.60:6819/3698562274', 'cluster_addr': '172.21.15.60:6821/3698562274', 'heartbeat_back_addr': '172.21.15.60:6825/3698562274', 'heartbeat_front_addr': '172.21.15.60:6823/3698562274', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'b1c2cb69-8565-49ac-b854-7ea7615c6660', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.4609375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 318, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6826', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6827', 'nonce': 2592322449}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6828', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6829', 'nonce': 2592322449}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6832', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6833', 'nonce': 2592322449}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6830', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6831', 'nonce': 2592322449}]}, 'public_addr': '172.21.15.60:6827/2592322449', 'cluster_addr': '172.21.15.60:6829/2592322449', 'heartbeat_back_addr': '172.21.15.60:6833/2592322449', 'heartbeat_front_addr': '172.21.15.60:6831/2592322449', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '04a547d6-fc48-43b7-a8d7-18df8a83aa96', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 323, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6800', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6801', 'nonce': 3243564239}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6802', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6803', 'nonce': 3243564239}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6806', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6807', 'nonce': 3243564239}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6804', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6805', 'nonce': 3243564239}]}, 'public_addr': '172.21.15.67:6801/3243564239', 'cluster_addr': '172.21.15.67:6803/3243564239', 'heartbeat_back_addr': '172.21.15.67:6807/3243564239', 'heartbeat_front_addr': '172.21.15.67:6805/3243564239', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4763aa10-c367-485b-b525-e5241e642b86', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 326, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6808', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6809', 'nonce': 1622258887}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6810', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6811', 'nonce': 1622258887}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6814', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6815', 'nonce': 1622258887}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6812', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6813', 'nonce': 1622258887}]}, 'public_addr': '172.21.15.67:6809/1622258887', 'cluster_addr': '172.21.15.67:6811/1622258887', 'heartbeat_back_addr': '172.21.15.67:6815/1622258887', 'heartbeat_front_addr': '172.21.15.67:6813/1622258887', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'a359116e-c013-4e2f-8b79-86219516ee16', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 48, 'up_thru': 327, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6816', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6817', 'nonce': 1187114}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6818', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6819', 'nonce': 1187114}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6822', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6823', 'nonce': 1187114}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6820', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6821', 'nonce': 1187114}]}, 'public_addr': '172.21.15.67:6817/1187114', 'cluster_addr': '172.21.15.67:6819/1187114', 'heartbeat_back_addr': '172.21.15.67:6823/1187114', 'heartbeat_front_addr': '172.21.15.67:6821/1187114', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a4bf4188-f133-4a31-8897-d7e8182248d7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 57, 'up_thru': 326, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6824', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6825', 'nonce': 3021842840}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6826', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6827', 'nonce': 3021842840}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6830', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6831', 'nonce': 3021842840}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6828', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6829', 'nonce': 3021842840}]}, 'public_addr': '172.21.15.67:6825/3021842840', 'cluster_addr': '172.21.15.67:6827/3021842840', 'heartbeat_back_addr': '172.21.15.67:6831/3021842840', 'heartbeat_front_addr': '172.21.15.67:6829/3021842840', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:13:54.778606+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:13:13.325543+0000', 'dead_epoch': 90}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:13:21.848022+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:13:41.851461+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:13:37.033245+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:13:40.296439+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:13:20.436570+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:13:30.251159+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '3.c', 'osds': [3, 1, 6]}, {'pgid': '6.13', 'osds': [0, 7, 6]}], 'pg_upmap_items': [{'pgid': '3.1a', 'mappings': [{'from': 4, 'to': 5}]}, {'pgid': '3.1c', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '4.b', 'mappings': [{'from': 1, 'to': 7}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.1b', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '6.9', 'mappings': [{'from': 7, 'to': 1}]}, {'pgid': '6.11', 'mappings': [{'from': 3, 'to': 1}]}, {'pgid': '6.19', 'mappings': [{'from': 3, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.60:0/890154710': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:0/3386526350': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:6801/604714621': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:6800/604714621': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:6800/3151907776': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/956264518': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/4150585713': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/3311255314': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:0/3965541528': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:6800/3734574621': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:0/1729862661': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:6801/3734574621': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:0/3927294026': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:6801/3151907776': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/1871351181': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:0/1854455098': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:0/2221262203': '2024-09-21T22:03:19.518405+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-20T22:14:26.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:14:26.763 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:26.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:26.764 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:26.864 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:26.886 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:14:26.886 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:14:26.907 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:26.907 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:26.908 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:27.008 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:27.056 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:27.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:27.057 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:27.157 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:27.196 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:27.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:27.197 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:27.297 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:27.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:27.353 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:27.353 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:27.374 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:27.439 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:27.454 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:27.497 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:27.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:27.498 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:27.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:27 smithi067 ceph-mon[33511]: pgmap v615: 158 pgs: 158 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-20T22:14:27.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:27 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:14:27.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:27 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:14:27.521 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:27 smithi060 ceph-mon[29694]: pgmap v615: 158 pgs: 158 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-20T22:14:27.522 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:27 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:14:27.522 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:27 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:14:27.523 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:27 smithi060 ceph-mon[40290]: pgmap v615: 158 pgs: 158 active+clean; 2.3 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1.17k op/s 2024-09-20T22:14:27.524 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:27 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:14:27.524 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:27 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:14:27.599 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:27.632 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:27.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:27.633 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:27.643 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T22:14:27.735 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:27.766 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:27.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:27.767 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:27.868 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:27.899 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:27.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:27.900 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:28.001 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:28.033 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:28.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:28.034 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:28.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:28.186 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:28.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:28.187 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:28.287 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:28.327 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:28.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:28.328 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:28.394 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:28 smithi060 ceph-mon[29694]: osdmap e353: 8 total, 8 up, 8 in 2024-09-20T22:14:28.394 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:28 smithi060 ceph-mon[40290]: osdmap e353: 8 total, 8 up, 8 in 2024-09-20T22:14:28.429 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:28.469 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:28.470 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:14:28.470 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:14:28.470 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:14:28.470 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:14:28.471 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:14:28.471 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:14:28.471 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:14:28.471 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:14:28.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:28.472 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:28.506 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:14:28.506 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:14:28.544 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:14:28.544 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:14:28.573 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:28.583 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:14:28.583 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:14:28.587 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:28.600 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:28.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:28.601 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:28.623 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:14:28.623 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:14:28.701 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:28.736 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:28.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:28.737 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:28 smithi067 ceph-mon[33511]: osdmap e353: 8 total, 8 up, 8 in 2024-09-20T22:14:28.795 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:14:28.798 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:14:28.838 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:28.867 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:28.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:28.868 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:28.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:29.003 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:29.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:29.004 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:29.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:29.145 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:29.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:29.146 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:29.246 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:29.289 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:29.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:29.290 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:29.296 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:14:29.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:29 smithi060 ceph-mon[29694]: pgmap v617: 158 pgs: 158 active+clean; 2.2 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-20T22:14:29.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:29 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1418087859' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:29.368 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:29 smithi060 ceph-mon[40290]: pgmap v617: 158 pgs: 158 active+clean; 2.2 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-20T22:14:29.368 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:29 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1418087859' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:29.391 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:29.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:29.434 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:29.435 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:29.535 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:29.582 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:29.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:29.583 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:29.684 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:29.695 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":617,"stamp":"2024-09-20T22:14:28.198934+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2347440691,"num_objects":36010,"num_object_clones":0,"num_object_copies":108030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36010,"num_whiteouts":0,"num_read":1807,"num_read_kb":1559,"num_write":186523,"num_write_kb":7091566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":315,"num_bytes_recovered":8460563,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83019,"ondisk_log_size":83019,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":11231904,"kb_used_data":7040776,"kb_used_omap":15,"kb_used_meta":4190576,"kb_avail":738729312,"statfs":{"total":767960285184,"available":756458815488,"internally_reserved":0,"allocated":7209754624,"data_stored":7206916285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":4291150611},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-704905216,"num_objects":-10756,"num_object_clones":0,"num_object_copies":-32268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-10756,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":10756,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.004464"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":67,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.839799+0000","last_change":"2024-09-20T22:12:52.516248+0000","last_active":"2024-09-20T22:14:23.839799+0000","last_peered":"2024-09-20T22:14:23.839799+0000","last_clean":"2024-09-20T22:14:23.839799+0000","last_became_active":"2024-09-20T22:12:52.515921+0000","last_became_peered":"2024-09-20T22:12:52.515921+0000","last_unstale":"2024-09-20T22:14:23.839799+0000","last_undegraded":"2024-09-20T22:14:23.839799+0000","last_fullsized":"2024-09-20T22:14:23.839799+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:38.640782+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_clean_scrub_stamp":"2024-09-20T22:12:38.640782+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"327'19","reported_seq":657,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.835687+0000","last_change":"2024-09-20T22:13:03.108336+0000","last_active":"2024-09-20T22:14:23.835687+0000","last_peered":"2024-09-20T22:14:23.835687+0000","last_clean":"2024-09-20T22:14:23.835687+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:14:23.835687+0000","last_undegraded":"2024-09-20T22:14:23.835687+0000","last_fullsized":"2024-09-20T22:14:23.835687+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"327'19","last_scrub_stamp":"2024-09-20T22:13:03.108203+0000","last_deep_scrub":"252'13","last_deep_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_clean_scrub_stamp":"2024-09-20T22:13:03.108203+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:31.739841+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":509,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:15.391117+0000","last_change":"2024-09-20T22:14:15.391149+0000","last_active":"2024-09-20T22:14:15.391117+0000","last_peered":"2024-09-20T22:14:15.391117+0000","last_clean":"2024-09-20T22:14:15.391117+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:14:15.391117+0000","last_undegraded":"2024-09-20T22:14:15.391117+0000","last_fullsized":"2024-09-20T22:14:15.391117+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:15.391049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:15.391049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:37.227878+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":624,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.836008+0000","last_change":"2024-09-20T22:12:42.264702+0000","last_active":"2024-09-20T22:14:23.836008+0000","last_peered":"2024-09-20T22:14:23.836008+0000","last_clean":"2024-09-20T22:14:23.836008+0000","last_became_active":"2024-09-20T22:11:30.485735+0000","last_became_peered":"2024-09-20T22:11:30.485735+0000","last_unstale":"2024-09-20T22:14:23.836008+0000","last_undegraded":"2024-09-20T22:14:23.836008+0000","last_fullsized":"2024-09-20T22:14:23.836008+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:42.264587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:42.264587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":611,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.856091+0000","last_change":"2024-09-20T22:13:14.774356+0000","last_active":"2024-09-20T22:14:23.856091+0000","last_peered":"2024-09-20T22:14:23.856091+0000","last_clean":"2024-09-20T22:14:23.856091+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:14:23.856091+0000","last_undegraded":"2024-09-20T22:14:23.856091+0000","last_fullsized":"2024-09-20T22:14:23.856091+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:13:14.774240+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:14.774240+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:32.288584+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":96,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-20T22:14:27.275660+0000","last_change":"2024-09-20T22:13:13.991120+0000","last_active":"2024-09-20T22:14:27.275660+0000","last_peered":"2024-09-20T22:14:27.275660+0000","last_clean":"2024-09-20T22:14:27.275660+0000","last_became_active":"2024-09-20T22:12:33.545100+0000","last_became_peered":"2024-09-20T22:12:33.545100+0000","last_unstale":"2024-09-20T22:14:27.275660+0000","last_undegraded":"2024-09-20T22:14:27.275660+0000","last_fullsized":"2024-09-20T22:14:27.275660+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:13.991017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:13.991017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:31.595811+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"328'27","reported_seq":210,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.850015+0000","last_change":"2024-09-20T22:13:39.102555+0000","last_active":"2024-09-20T22:14:23.850015+0000","last_peered":"2024-09-20T22:14:23.850015+0000","last_clean":"2024-09-20T22:14:23.850015+0000","last_became_active":"2024-09-20T22:12:52.517296+0000","last_became_peered":"2024-09-20T22:12:52.517296+0000","last_unstale":"2024-09-20T22:14:23.850015+0000","last_undegraded":"2024-09-20T22:14:23.850015+0000","last_fullsized":"2024-09-20T22:14:23.850015+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:13:39.102397+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:39.102397+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:04.682528+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":616,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.835630+0000","last_change":"2024-09-20T22:13:06.156767+0000","last_active":"2024-09-20T22:14:23.835630+0000","last_peered":"2024-09-20T22:14:23.835630+0000","last_clean":"2024-09-20T22:14:23.835630+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:14:23.835630+0000","last_undegraded":"2024-09-20T22:14:23.835630+0000","last_fullsized":"2024-09-20T22:14:23.835630+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:06.156656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:06.156656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:33.362926+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":638,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.849780+0000","last_change":"2024-09-20T22:12:01.173031+0000","last_active":"2024-09-20T22:14:23.849780+0000","last_peered":"2024-09-20T22:14:23.849780+0000","last_clean":"2024-09-20T22:14:23.849780+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:14:23.849780+0000","last_undegraded":"2024-09-20T22:14:23.849780+0000","last_fullsized":"2024-09-20T22:14:23.849780+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:01.172926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:12:01.172926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"328'15","reported_seq":684,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.836239+0000","last_change":"2024-09-20T22:14:19.076811+0000","last_active":"2024-09-20T22:14:23.836239+0000","last_peered":"2024-09-20T22:14:23.836239+0000","last_clean":"2024-09-20T22:14:23.836239+0000","last_became_active":"2024-09-20T22:12:52.516894+0000","last_became_peered":"2024-09-20T22:12:52.516894+0000","last_unstale":"2024-09-20T22:14:23.836239+0000","last_undegraded":"2024-09-20T22:14:23.836239+0000","last_fullsized":"2024-09-20T22:14:23.836239+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:14:19.076696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:19.076696+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:45.728095+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":170,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.849275+0000","last_change":"2024-09-20T22:12:52.518888+0000","last_active":"2024-09-20T22:14:23.849275+0000","last_peered":"2024-09-20T22:14:23.849275+0000","last_clean":"2024-09-20T22:14:23.849275+0000","last_became_active":"2024-09-20T22:12:52.518549+0000","last_became_peered":"2024-09-20T22:12:52.518549+0000","last_unstale":"2024-09-20T22:14:23.849275+0000","last_undegraded":"2024-09-20T22:14:23.849275+0000","last_fullsized":"2024-09-20T22:14:23.849275+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:31.101573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:31.101573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":75,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.843040+0000","last_change":"2024-09-20T22:13:02.488382+0000","last_active":"2024-09-20T22:14:23.843040+0000","last_peered":"2024-09-20T22:14:23.843040+0000","last_clean":"2024-09-20T22:14:23.843040+0000","last_became_active":"2024-09-20T22:12:52.524631+0000","last_became_peered":"2024-09-20T22:12:52.524631+0000","last_unstale":"2024-09-20T22:14:23.843040+0000","last_undegraded":"2024-09-20T22:14:23.843040+0000","last_fullsized":"2024-09-20T22:14:23.843040+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:02.488267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:02.488267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":643,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.847086+0000","last_change":"2024-09-20T22:11:40.127688+0000","last_active":"2024-09-20T22:14:23.847086+0000","last_peered":"2024-09-20T22:14:23.847086+0000","last_clean":"2024-09-20T22:14:23.847086+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:14:23.847086+0000","last_undegraded":"2024-09-20T22:14:23.847086+0000","last_fullsized":"2024-09-20T22:14:23.847086+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:40.127597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:30.215263+0000","last_clean_scrub_stamp":"2024-09-20T22:11:40.127597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"328'7","reported_seq":653,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.836273+0000","last_change":"2024-09-20T22:13:44.057396+0000","last_active":"2024-09-20T22:14:23.836273+0000","last_peered":"2024-09-20T22:14:23.836273+0000","last_clean":"2024-09-20T22:14:23.836273+0000","last_became_active":"2024-09-20T22:08:22.144364+0000","last_became_peered":"2024-09-20T22:08:22.144364+0000","last_unstale":"2024-09-20T22:14:23.836273+0000","last_undegraded":"2024-09-20T22:14:23.836273+0000","last_fullsized":"2024-09-20T22:14:23.836273+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'7","last_scrub_stamp":"2024-09-20T22:13:44.057245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:44.057245+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:09.952476+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":624,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.842868+0000","last_change":"2024-09-20T22:12:38.636611+0000","last_active":"2024-09-20T22:14:23.842868+0000","last_peered":"2024-09-20T22:14:23.842868+0000","last_clean":"2024-09-20T22:14:23.842868+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:14:23.842868+0000","last_undegraded":"2024-09-20T22:14:23.842868+0000","last_fullsized":"2024-09-20T22:14:23.842868+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:38.636478+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:38.636478+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":158,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.848558+0000","last_change":"2024-09-20T22:11:34.036027+0000","last_active":"2024-09-20T22:14:23.848558+0000","last_peered":"2024-09-20T22:14:23.848558+0000","last_clean":"2024-09-20T22:14:23.848558+0000","last_became_active":"2024-09-20T22:11:30.487137+0000","last_became_peered":"2024-09-20T22:11:30.487137+0000","last_unstale":"2024-09-20T22:14:23.848558+0000","last_undegraded":"2024-09-20T22:14:23.848558+0000","last_fullsized":"2024-09-20T22:14:23.848558+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:34.035938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:11:34.035938+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":612,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.836408+0000","last_change":"2024-09-20T22:12:41.309522+0000","last_active":"2024-09-20T22:14:23.836408+0000","last_peered":"2024-09-20T22:14:23.836408+0000","last_clean":"2024-09-20T22:14:23.836408+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:14:23.836408+0000","last_undegraded":"2024-09-20T22:14:23.836408+0000","last_fullsized":"2024-09-20T22:14:23.836408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:41.309424+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:12:41.309424+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":75,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.840753+0000","last_change":"2024-09-20T22:13:52.481857+0000","last_active":"2024-09-20T22:14:23.840753+0000","last_peered":"2024-09-20T22:14:23.840753+0000","last_clean":"2024-09-20T22:14:23.840753+0000","last_became_active":"2024-09-20T22:12:52.526097+0000","last_became_peered":"2024-09-20T22:12:52.526097+0000","last_unstale":"2024-09-20T22:14:23.840753+0000","last_undegraded":"2024-09-20T22:14:23.840753+0000","last_fullsized":"2024-09-20T22:14:23.840753+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:52.481584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:13:52.481584+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:54.068422+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"328'34","reported_seq":214,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.851413+0000","last_change":"2024-09-20T22:13:46.127942+0000","last_active":"2024-09-20T22:14:23.851413+0000","last_peered":"2024-09-20T22:14:23.851413+0000","last_clean":"2024-09-20T22:14:23.851413+0000","last_became_active":"2024-09-20T22:11:12.507311+0000","last_became_peered":"2024-09-20T22:11:12.507311+0000","last_unstale":"2024-09-20T22:14:23.851413+0000","last_undegraded":"2024-09-20T22:14:23.851413+0000","last_fullsized":"2024-09-20T22:14:23.851413+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'34","last_scrub_stamp":"2024-09-20T22:13:46.127812+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:13:46.127812+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:47.873800+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":471,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:13.755118+0000","last_change":"2024-09-20T22:13:11.053381+0000","last_active":"2024-09-20T22:14:13.755118+0000","last_peered":"2024-09-20T22:14:13.755118+0000","last_clean":"2024-09-20T22:14:13.755118+0000","last_became_active":"2024-09-20T22:12:52.520985+0000","last_became_peered":"2024-09-20T22:12:52.520985+0000","last_unstale":"2024-09-20T22:14:13.755118+0000","last_undegraded":"2024-09-20T22:14:13.755118+0000","last_fullsized":"2024-09-20T22:14:13.755118+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:11.053276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:11.053276+0000","last_clean_scrub_stamp":"2024-09-20T22:13:11.053276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:35.430970+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":574,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-20T22:14:27.275742+0000","last_change":"2024-09-20T22:14:19.451519+0000","last_active":"2024-09-20T22:14:27.275742+0000","last_peered":"2024-09-20T22:14:27.275742+0000","last_clean":"2024-09-20T22:14:27.275742+0000","last_became_active":"2024-09-20T22:12:21.521433+0000","last_became_peered":"2024-09-20T22:12:21.521433+0000","last_unstale":"2024-09-20T22:14:27.275742+0000","last_undegraded":"2024-09-20T22:14:27.275742+0000","last_fullsized":"2024-09-20T22:14:27.275742+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:19.451409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:19.451409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:42.318241+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":573,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.850437+0000","last_change":"2024-09-20T22:12:44.197752+0000","last_active":"2024-09-20T22:14:23.850437+0000","last_peered":"2024-09-20T22:14:23.850437+0000","last_clean":"2024-09-20T22:14:23.850437+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:14:23.850437+0000","last_undegraded":"2024-09-20T22:14:23.850437+0000","last_fullsized":"2024-09-20T22:14:23.850437+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:44.197654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:44.197654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"328'23","reported_seq":535,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:13.755281+0000","last_change":"2024-09-20T22:13:08.115776+0000","last_active":"2024-09-20T22:14:13.755281+0000","last_peered":"2024-09-20T22:14:13.755281+0000","last_clean":"2024-09-20T22:14:13.755281+0000","last_became_active":"2024-09-20T22:12:21.517147+0000","last_became_peered":"2024-09-20T22:12:21.517147+0000","last_unstale":"2024-09-20T22:14:13.755281+0000","last_undegraded":"2024-09-20T22:14:13.755281+0000","last_fullsized":"2024-09-20T22:14:13.755281+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'23","last_scrub_stamp":"2024-09-20T22:13:08.115352+0000","last_deep_scrub":"250'17","last_deep_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_clean_scrub_stamp":"2024-09-20T22:13:08.115352+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:16.402904+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":615,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.836133+0000","last_change":"2024-09-20T22:13:51.133410+0000","last_active":"2024-09-20T22:14:23.836133+0000","last_peered":"2024-09-20T22:14:23.836133+0000","last_clean":"2024-09-20T22:14:23.836133+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:14:23.836133+0000","last_undegraded":"2024-09-20T22:14:23.836133+0000","last_fullsized":"2024-09-20T22:14:23.836133+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:51.133308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.133308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:13.505445+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":613,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.843322+0000","last_change":"2024-09-20T22:13:52.688758+0000","last_active":"2024-09-20T22:14:23.843322+0000","last_peered":"2024-09-20T22:14:23.843322+0000","last_clean":"2024-09-20T22:14:23.843322+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:14:23.843322+0000","last_undegraded":"2024-09-20T22:14:23.843322+0000","last_fullsized":"2024-09-20T22:14:23.843322+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:13:52.688650+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:13:52.688650+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:55.995741+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"289'2","reported_seq":469,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:22.436404+0000","last_change":"2024-09-20T22:14:22.436454+0000","last_active":"2024-09-20T22:14:22.436404+0000","last_peered":"2024-09-20T22:14:22.436404+0000","last_clean":"2024-09-20T22:14:22.436404+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:14:22.436404+0000","last_undegraded":"2024-09-20T22:14:22.436404+0000","last_fullsized":"2024-09-20T22:14:22.436404+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-20T22:14:22.436308+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_clean_scrub_stamp":"2024-09-20T22:14:22.436308+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:51.653333+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"328'20","reported_seq":102,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.840007+0000","last_change":"2024-09-20T22:14:11.444573+0000","last_active":"2024-09-20T22:14:23.840007+0000","last_peered":"2024-09-20T22:14:23.840007+0000","last_clean":"2024-09-20T22:14:23.840007+0000","last_became_active":"2024-09-20T22:12:52.526803+0000","last_became_peered":"2024-09-20T22:12:52.526803+0000","last_unstale":"2024-09-20T22:14:23.840007+0000","last_undegraded":"2024-09-20T22:14:23.840007+0000","last_fullsized":"2024-09-20T22:14:23.840007+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:14:11.444444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:11.444444+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.306318+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"291'11","reported_seq":805,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-20T22:14:27.275907+0000","last_change":"2024-09-20T22:13:53.376728+0000","last_active":"2024-09-20T22:14:27.275907+0000","last_peered":"2024-09-20T22:14:27.275907+0000","last_clean":"2024-09-20T22:14:27.275907+0000","last_became_active":"2024-09-20T22:11:40.601048+0000","last_became_peered":"2024-09-20T22:11:40.601048+0000","last_unstale":"2024-09-20T22:14:27.275907+0000","last_undegraded":"2024-09-20T22:14:27.275907+0000","last_fullsized":"2024-09-20T22:14:27.275907+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'11","last_scrub_stamp":"2024-09-20T22:13:53.376616+0000","last_deep_scrub":"291'11","last_deep_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_clean_scrub_stamp":"2024-09-20T22:13:53.376616+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:01.413994+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2","version":"351'11429","reported_seq":11630,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.322510+0000","last_change":"2024-09-20T22:13:48.324296+0000","last_active":"2024-09-20T22:14:23.322510+0000","last_peered":"2024-09-20T22:14:23.322510+0000","last_clean":"2024-09-20T22:14:23.322510+0000","last_became_active":"2024-09-20T22:12:33.545160+0000","last_became_peered":"2024-09-20T22:12:33.545160+0000","last_unstale":"2024-09-20T22:14:23.322510+0000","last_undegraded":"2024-09-20T22:14:23.322510+0000","last_fullsized":"2024-09-20T22:14:23.322510+0000","mapping_epoch":314,"log_start":"333'6864","ondisk_log_start":"333'6864","created":286,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"345'9130","last_scrub_stamp":"2024-09-20T22:13:48.324088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:13:48.324088+0000","objects_scrubbed":4747,"log_size":4565,"log_dups_size":0,"ondisk_log_size":4565,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:56.742135+0000","scrub_duration":1847,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":157220864,"num_objects":2399,"num_object_clones":0,"num_object_copies":7197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11429,"num_write_kb":442496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,2],"acting":[1,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":627,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.850277+0000","last_change":"2024-09-20T22:13:08.407055+0000","last_active":"2024-09-20T22:14:23.850277+0000","last_peered":"2024-09-20T22:14:23.850277+0000","last_clean":"2024-09-20T22:14:23.850277+0000","last_became_active":"2024-09-20T22:08:48.452786+0000","last_became_peered":"2024-09-20T22:08:48.452786+0000","last_unstale":"2024-09-20T22:14:23.850277+0000","last_undegraded":"2024-09-20T22:14:23.850277+0000","last_fullsized":"2024-09-20T22:14:23.850277+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:08.406883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:13:08.406883+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"328'27","reported_seq":407,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.851068+0000","last_change":"2024-09-20T22:13:06.060888+0000","last_active":"2024-09-20T22:14:23.851068+0000","last_peered":"2024-09-20T22:14:23.851068+0000","last_clean":"2024-09-20T22:14:23.851068+0000","last_became_active":"2024-09-20T22:08:22.145408+0000","last_became_peered":"2024-09-20T22:08:22.145408+0000","last_unstale":"2024-09-20T22:14:23.851068+0000","last_undegraded":"2024-09-20T22:14:23.851068+0000","last_fullsized":"2024-09-20T22:14:23.851068+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:13:06.060761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:06.060761+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":632,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.853638+0000","last_change":"2024-09-20T22:13:53.012457+0000","last_active":"2024-09-20T22:14:23.853638+0000","last_peered":"2024-09-20T22:14:23.853638+0000","last_clean":"2024-09-20T22:14:23.853638+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:14:23.853638+0000","last_undegraded":"2024-09-20T22:14:23.853638+0000","last_fullsized":"2024-09-20T22:14:23.853638+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:53.012225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:53.012225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:05.749378+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"326'13","reported_seq":825,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.836563+0000","last_change":"2024-09-20T22:12:52.533157+0000","last_active":"2024-09-20T22:14:23.836563+0000","last_peered":"2024-09-20T22:14:23.836563+0000","last_clean":"2024-09-20T22:14:23.836563+0000","last_became_active":"2024-09-20T22:12:52.517588+0000","last_became_peered":"2024-09-20T22:12:52.517588+0000","last_unstale":"2024-09-20T22:14:23.836563+0000","last_undegraded":"2024-09-20T22:14:23.836563+0000","last_fullsized":"2024-09-20T22:14:23.836563+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'11","last_scrub_stamp":"2024-09-20T22:12:16.129029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:16.129029+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:29.028927+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":170,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.849371+0000","last_change":"2024-09-20T22:12:52.518570+0000","last_active":"2024-09-20T22:14:23.849371+0000","last_peered":"2024-09-20T22:14:23.849371+0000","last_clean":"2024-09-20T22:14:23.849371+0000","last_became_active":"2024-09-20T22:12:52.517826+0000","last_became_peered":"2024-09-20T22:12:52.517826+0000","last_unstale":"2024-09-20T22:14:23.849371+0000","last_undegraded":"2024-09-20T22:14:23.849371+0000","last_fullsized":"2024-09-20T22:14:23.849371+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:07.114531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:07.114531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"352'11522","reported_seq":11708,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:25.776317+0000","last_change":"2024-09-20T22:14:08.939841+0000","last_active":"2024-09-20T22:14:25.776317+0000","last_peered":"2024-09-20T22:14:25.776317+0000","last_clean":"2024-09-20T22:14:25.776317+0000","last_became_active":"2024-09-20T22:11:34.561538+0000","last_became_peered":"2024-09-20T22:11:34.561538+0000","last_unstale":"2024-09-20T22:14:25.776317+0000","last_undegraded":"2024-09-20T22:14:25.776317+0000","last_fullsized":"2024-09-20T22:14:25.776317+0000","mapping_epoch":286,"log_start":"333'7023","ondisk_log_start":"333'7023","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"349'10409","last_scrub_stamp":"2024-09-20T22:14:08.939348+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:14:08.939348+0000","objects_scrubbed":3444,"log_size":4499,"log_dups_size":0,"ondisk_log_size":4499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:27.670701+0000","scrub_duration":1082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":150011928,"num_objects":2290,"num_object_clones":0,"num_object_copies":6870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2290,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11522,"num_write_kb":441921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,5],"acting":[7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":627,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.836712+0000","last_change":"2024-09-20T22:14:21.059023+0000","last_active":"2024-09-20T22:14:23.836712+0000","last_peered":"2024-09-20T22:14:23.836712+0000","last_clean":"2024-09-20T22:14:23.836712+0000","last_became_active":"2024-09-20T22:08:48.452475+0000","last_became_peered":"2024-09-20T22:08:48.452475+0000","last_unstale":"2024-09-20T22:14:23.836712+0000","last_undegraded":"2024-09-20T22:14:23.836712+0000","last_fullsized":"2024-09-20T22:14:23.836712+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:14:21.058895+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:24.210483+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"328'30","reported_seq":635,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.842409+0000","last_change":"2024-09-20T22:13:50.715007+0000","last_active":"2024-09-20T22:14:23.842409+0000","last_peered":"2024-09-20T22:14:23.842409+0000","last_clean":"2024-09-20T22:14:23.842409+0000","last_became_active":"2024-09-20T22:12:52.526867+0000","last_became_peered":"2024-09-20T22:12:52.526867+0000","last_unstale":"2024-09-20T22:14:23.842409+0000","last_undegraded":"2024-09-20T22:14:23.842409+0000","last_fullsized":"2024-09-20T22:14:23.842409+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'30","last_scrub_stamp":"2024-09-20T22:13:50.714867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:50.714867+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:05.605678+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":116,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.843834+0000","last_change":"2024-09-20T22:14:09.922166+0000","last_active":"2024-09-20T22:14:23.843834+0000","last_peered":"2024-09-20T22:14:23.843834+0000","last_clean":"2024-09-20T22:14:23.843834+0000","last_became_active":"2024-09-20T22:12:52.528947+0000","last_became_peered":"2024-09-20T22:12:52.528947+0000","last_unstale":"2024-09-20T22:14:23.843834+0000","last_undegraded":"2024-09-20T22:14:23.843834+0000","last_fullsized":"2024-09-20T22:14:23.843834+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:09.922092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:09.922092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:27.839312+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":636,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.835759+0000","last_change":"2024-09-20T22:14:13.046284+0000","last_active":"2024-09-20T22:14:23.835759+0000","last_peered":"2024-09-20T22:14:23.835759+0000","last_clean":"2024-09-20T22:14:23.835759+0000","last_became_active":"2024-09-20T22:10:48.919949+0000","last_became_peered":"2024-09-20T22:10:48.919949+0000","last_unstale":"2024-09-20T22:14:23.835759+0000","last_undegraded":"2024-09-20T22:14:23.835759+0000","last_fullsized":"2024-09-20T22:14:23.835759+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:13.046189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:13.046189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:22.491976+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":620,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.836130+0000","last_change":"2024-09-20T22:14:08.158795+0000","last_active":"2024-09-20T22:14:23.836130+0000","last_peered":"2024-09-20T22:14:23.836130+0000","last_clean":"2024-09-20T22:14:23.836130+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:14:23.836130+0000","last_undegraded":"2024-09-20T22:14:23.836130+0000","last_fullsized":"2024-09-20T22:14:23.836130+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:08.158688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:08.158688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:25.401484+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"352'11759","reported_seq":11968,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:25.769329+0000","last_change":"2024-09-20T22:13:50.256083+0000","last_active":"2024-09-20T22:14:25.769329+0000","last_peered":"2024-09-20T22:14:25.769329+0000","last_clean":"2024-09-20T22:14:25.769329+0000","last_became_active":"2024-09-20T22:12:33.546173+0000","last_became_peered":"2024-09-20T22:12:33.546173+0000","last_unstale":"2024-09-20T22:14:25.769329+0000","last_undegraded":"2024-09-20T22:14:25.769329+0000","last_fullsized":"2024-09-20T22:14:25.769329+0000","mapping_epoch":314,"log_start":"333'6907","ondisk_log_start":"333'6907","created":286,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"345'9370","last_scrub_stamp":"2024-09-20T22:13:50.255922+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:13:50.255922+0000","objects_scrubbed":4781,"log_size":4852,"log_dups_size":0,"ondisk_log_size":4852,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:03.962179+0000","scrub_duration":1129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154075136,"num_objects":2351,"num_object_clones":0,"num_object_copies":7053,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2351,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11759,"num_write_kb":451520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":653,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.856000+0000","last_change":"2024-09-20T22:13:36.804562+0000","last_active":"2024-09-20T22:14:23.856000+0000","last_peered":"2024-09-20T22:14:23.856000+0000","last_clean":"2024-09-20T22:14:23.856000+0000","last_became_active":"2024-09-20T22:11:33.560632+0000","last_became_peered":"2024-09-20T22:11:33.560632+0000","last_unstale":"2024-09-20T22:14:23.856000+0000","last_undegraded":"2024-09-20T22:14:23.856000+0000","last_fullsized":"2024-09-20T22:14:23.856000+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:36.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:13:36.803555+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:01.676170+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"328'28","reported_seq":662,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.836114+0000","last_change":"2024-09-20T22:12:08.158731+0000","last_active":"2024-09-20T22:14:23.836114+0000","last_peered":"2024-09-20T22:14:23.836114+0000","last_clean":"2024-09-20T22:14:23.836114+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:14:23.836114+0000","last_undegraded":"2024-09-20T22:14:23.836114+0000","last_fullsized":"2024-09-20T22:14:23.836114+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'14","last_scrub_stamp":"2024-09-20T22:12:08.158574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:08.158574+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":79,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.840193+0000","last_change":"2024-09-20T22:13:47.454245+0000","last_active":"2024-09-20T22:14:23.840193+0000","last_peered":"2024-09-20T22:14:23.840193+0000","last_clean":"2024-09-20T22:14:23.840193+0000","last_became_active":"2024-09-20T22:12:52.528068+0000","last_became_peered":"2024-09-20T22:12:52.528068+0000","last_unstale":"2024-09-20T22:14:23.840193+0000","last_undegraded":"2024-09-20T22:14:23.840193+0000","last_fullsized":"2024-09-20T22:14:23.840193+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:13:47.454137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:47.454137+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:15.159536+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"315'11","reported_seq":340,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:15.311602+0000","last_change":"2024-09-20T22:13:05.177108+0000","last_active":"2024-09-20T22:14:15.311602+0000","last_peered":"2024-09-20T22:14:15.311602+0000","last_clean":"2024-09-20T22:14:15.311602+0000","last_became_active":"2024-09-20T22:12:33.545942+0000","last_became_peered":"2024-09-20T22:12:33.545942+0000","last_unstale":"2024-09-20T22:14:15.311602+0000","last_undegraded":"2024-09-20T22:14:15.311602+0000","last_fullsized":"2024-09-20T22:14:15.311602+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:13:05.176973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:05.176973+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:23.809705+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":612,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.843251+0000","last_change":"2024-09-20T22:13:10.621055+0000","last_active":"2024-09-20T22:14:23.843251+0000","last_peered":"2024-09-20T22:14:23.843251+0000","last_clean":"2024-09-20T22:14:23.843251+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:14:23.843251+0000","last_undegraded":"2024-09-20T22:14:23.843251+0000","last_fullsized":"2024-09-20T22:14:23.843251+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:10.620925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.620925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"352'14783","reported_seq":15101,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:26.126118+0000","last_change":"2024-09-20T22:14:14.108572+0000","last_active":"2024-09-20T22:14:26.126118+0000","last_peered":"2024-09-20T22:14:26.126118+0000","last_clean":"2024-09-20T22:14:26.126118+0000","last_became_active":"2024-09-20T22:11:34.558898+0000","last_became_peered":"2024-09-20T22:11:34.558898+0000","last_unstale":"2024-09-20T22:14:26.126118+0000","last_undegraded":"2024-09-20T22:14:26.126118+0000","last_fullsized":"2024-09-20T22:14:26.126118+0000","mapping_epoch":286,"log_start":"345'9828","ondisk_log_start":"345'9828","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"351'13903","last_scrub_stamp":"2024-09-20T22:14:14.108452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:14:14.108452+0000","objects_scrubbed":3261,"log_size":4955,"log_dups_size":0,"ondisk_log_size":4955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:26.667295+0000","scrub_duration":1671,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":153157632,"num_objects":2337,"num_object_clones":0,"num_object_copies":7011,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11627,"num_write_kb":446848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":659,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-20T22:14:27.275602+0000","last_change":"2024-09-20T22:14:23.503489+0000","last_active":"2024-09-20T22:14:27.275602+0000","last_peered":"2024-09-20T22:14:27.275602+0000","last_clean":"2024-09-20T22:14:27.275602+0000","last_became_active":"2024-09-20T22:10:33.660948+0000","last_became_peered":"2024-09-20T22:10:33.660948+0000","last_unstale":"2024-09-20T22:14:27.275602+0000","last_undegraded":"2024-09-20T22:14:27.275602+0000","last_fullsized":"2024-09-20T22:14:27.275602+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:14:23.503360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:14:23.503360+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:52.579727+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"328'20","reported_seq":536,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:13.756236+0000","last_change":"2024-09-20T22:13:41.421053+0000","last_active":"2024-09-20T22:14:13.756236+0000","last_peered":"2024-09-20T22:14:13.756236+0000","last_clean":"2024-09-20T22:14:13.756236+0000","last_became_active":"2024-09-20T22:11:30.486532+0000","last_became_peered":"2024-09-20T22:11:30.486532+0000","last_unstale":"2024-09-20T22:14:13.756236+0000","last_undegraded":"2024-09-20T22:14:13.756236+0000","last_fullsized":"2024-09-20T22:14:13.756236+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:13:41.420933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:41.420933+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:00.706107+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":647,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.849867+0000","last_change":"2024-09-20T22:14:11.402471+0000","last_active":"2024-09-20T22:14:23.849867+0000","last_peered":"2024-09-20T22:14:23.849867+0000","last_clean":"2024-09-20T22:14:23.849867+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:14:23.849867+0000","last_undegraded":"2024-09-20T22:14:23.849867+0000","last_fullsized":"2024-09-20T22:14:23.849867+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:11.402361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:14:11.402361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:27.694063+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"327'12","reported_seq":87,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.840583+0000","last_change":"2024-09-20T22:12:57.887268+0000","last_active":"2024-09-20T22:14:23.840583+0000","last_peered":"2024-09-20T22:14:23.840583+0000","last_clean":"2024-09-20T22:14:23.840583+0000","last_became_active":"2024-09-20T22:12:57.884415+0000","last_became_peered":"2024-09-20T22:12:57.884415+0000","last_unstale":"2024-09-20T22:14:23.840583+0000","last_undegraded":"2024-09-20T22:14:23.840583+0000","last_fullsized":"2024-09-20T22:14:23.840583+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'10","last_scrub_stamp":"2024-09-20T22:12:07.451943+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:07.451943+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":67,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.842901+0000","last_change":"2024-09-20T22:12:52.519036+0000","last_active":"2024-09-20T22:14:23.842901+0000","last_peered":"2024-09-20T22:14:23.842901+0000","last_clean":"2024-09-20T22:14:23.842901+0000","last_became_active":"2024-09-20T22:12:52.518529+0000","last_became_peered":"2024-09-20T22:12:52.518529+0000","last_unstale":"2024-09-20T22:14:23.842901+0000","last_undegraded":"2024-09-20T22:14:23.842901+0000","last_fullsized":"2024-09-20T22:14:23.842901+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:41.689984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:41.689984+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.e","version":"352'11720","reported_seq":11920,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:25.805908+0000","last_change":"2024-09-20T22:13:44.886681+0000","last_active":"2024-09-20T22:14:25.805908+0000","last_peered":"2024-09-20T22:14:25.805908+0000","last_clean":"2024-09-20T22:14:25.805908+0000","last_became_active":"2024-09-20T22:11:34.560514+0000","last_became_peered":"2024-09-20T22:11:34.560514+0000","last_unstale":"2024-09-20T22:14:25.805908+0000","last_undegraded":"2024-09-20T22:14:25.805908+0000","last_fullsized":"2024-09-20T22:14:25.805908+0000","mapping_epoch":286,"log_start":"333'7166","ondisk_log_start":"333'7166","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"345'8947","last_scrub_stamp":"2024-09-20T22:13:44.886563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:13:44.886563+0000","objects_scrubbed":4900,"log_size":4554,"log_dups_size":0,"ondisk_log_size":4554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:45.294613+0000","scrub_duration":1137,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":137232384,"num_objects":2094,"num_object_clones":0,"num_object_copies":6282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11720,"num_write_kb":442048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":652,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-20T22:14:27.275110+0000","last_change":"2024-09-20T22:13:08.929614+0000","last_active":"2024-09-20T22:14:27.275110+0000","last_peered":"2024-09-20T22:14:27.275110+0000","last_clean":"2024-09-20T22:14:27.275110+0000","last_became_active":"2024-09-20T22:12:55.523589+0000","last_became_peered":"2024-09-20T22:12:55.523589+0000","last_unstale":"2024-09-20T22:14:27.275110+0000","last_undegraded":"2024-09-20T22:14:27.275110+0000","last_fullsized":"2024-09-20T22:14:27.275110+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:08.929484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:13:08.929484+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:38.455770+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"328'29","reported_seq":108,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.840635+0000","last_change":"2024-09-20T22:14:15.427866+0000","last_active":"2024-09-20T22:14:23.840635+0000","last_peered":"2024-09-20T22:14:23.840635+0000","last_clean":"2024-09-20T22:14:23.840635+0000","last_became_active":"2024-09-20T22:12:52.525538+0000","last_became_peered":"2024-09-20T22:12:52.525538+0000","last_unstale":"2024-09-20T22:14:23.840635+0000","last_undegraded":"2024-09-20T22:14:23.840635+0000","last_fullsized":"2024-09-20T22:14:23.840635+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'29","last_scrub_stamp":"2024-09-20T22:14:15.427533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:15.427533+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.217474+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":472,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:13.754619+0000","last_change":"2024-09-20T22:13:13.091930+0000","last_active":"2024-09-20T22:14:13.754619+0000","last_peered":"2024-09-20T22:14:13.754619+0000","last_clean":"2024-09-20T22:14:13.754619+0000","last_became_active":"2024-09-20T22:11:30.486428+0000","last_became_peered":"2024-09-20T22:11:30.486428+0000","last_unstale":"2024-09-20T22:14:13.754619+0000","last_undegraded":"2024-09-20T22:14:13.754619+0000","last_fullsized":"2024-09-20T22:14:13.754619+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:13.091819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:13.091819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:34.138090+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":653,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-20T22:14:27.275076+0000","last_change":"2024-09-20T22:14:22.500268+0000","last_active":"2024-09-20T22:14:27.275076+0000","last_peered":"2024-09-20T22:14:27.275076+0000","last_clean":"2024-09-20T22:14:27.275076+0000","last_became_active":"2024-09-20T22:12:52.519826+0000","last_became_peered":"2024-09-20T22:12:52.519826+0000","last_unstale":"2024-09-20T22:14:27.275076+0000","last_undegraded":"2024-09-20T22:14:27.275076+0000","last_fullsized":"2024-09-20T22:14:27.275076+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:22.500119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:14:22.500119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:41.298289+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":458,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:13.754369+0000","last_change":"2024-09-20T22:13:01.117688+0000","last_active":"2024-09-20T22:14:13.754369+0000","last_peered":"2024-09-20T22:14:13.754369+0000","last_clean":"2024-09-20T22:14:13.754369+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:14:13.754369+0000","last_undegraded":"2024-09-20T22:14:13.754369+0000","last_fullsized":"2024-09-20T22:14:13.754369+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:01.117578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:01.117578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:23.322241+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"352'11947","reported_seq":12143,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:26.863593+0000","last_change":"2024-09-20T22:14:12.191593+0000","last_active":"2024-09-20T22:14:26.863593+0000","last_peered":"2024-09-20T22:14:26.863593+0000","last_clean":"2024-09-20T22:14:26.863593+0000","last_became_active":"2024-09-20T22:12:53.512687+0000","last_became_peered":"2024-09-20T22:12:53.512687+0000","last_unstale":"2024-09-20T22:14:26.863593+0000","last_undegraded":"2024-09-20T22:14:26.863593+0000","last_fullsized":"2024-09-20T22:14:26.863593+0000","mapping_epoch":324,"log_start":"331'6222","ondisk_log_start":"331'6222","created":286,"last_epoch_clean":325,"parent":"0.0","parent_split_bits":0,"last_scrub":"350'10879","last_scrub_stamp":"2024-09-20T22:14:12.191404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:14:12.191404+0000","objects_scrubbed":3304,"log_size":5725,"log_dups_size":0,"ondisk_log_size":5725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:27.496471+0000","scrub_duration":1746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143720448,"num_objects":2193,"num_object_clones":0,"num_object_copies":6579,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2193,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11947,"num_write_kb":452480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":645,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.848424+0000","last_change":"2024-09-20T22:12:55.522020+0000","last_active":"2024-09-20T22:14:23.848424+0000","last_peered":"2024-09-20T22:14:23.848424+0000","last_clean":"2024-09-20T22:14:23.848424+0000","last_became_active":"2024-09-20T22:12:55.521605+0000","last_became_peered":"2024-09-20T22:12:55.521605+0000","last_unstale":"2024-09-20T22:14:23.848424+0000","last_undegraded":"2024-09-20T22:14:23.848424+0000","last_fullsized":"2024-09-20T22:14:23.848424+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:12:16.121263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:12:16.121263+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"328'52","reported_seq":639,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:15.309591+0000","last_change":"2024-09-20T22:13:36.392213+0000","last_active":"2024-09-20T22:14:15.309591+0000","last_peered":"2024-09-20T22:14:15.309591+0000","last_clean":"2024-09-20T22:14:15.309591+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:14:15.309591+0000","last_undegraded":"2024-09-20T22:14:15.309591+0000","last_fullsized":"2024-09-20T22:14:15.309591+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'52","last_scrub_stamp":"2024-09-20T22:13:36.392127+0000","last_deep_scrub":"271'40","last_deep_scrub_stamp":"2024-09-20T22:11:05.235779+0000","last_clean_scrub_stamp":"2024-09-20T22:13:36.392127+0000","objects_scrubbed":10,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:41.333635+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":633,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.836622+0000","last_change":"2024-09-20T22:12:57.884648+0000","last_active":"2024-09-20T22:14:23.836622+0000","last_peered":"2024-09-20T22:14:23.836622+0000","last_clean":"2024-09-20T22:14:23.836622+0000","last_became_active":"2024-09-20T22:12:57.884257+0000","last_became_peered":"2024-09-20T22:12:57.884257+0000","last_unstale":"2024-09-20T22:14:23.836622+0000","last_undegraded":"2024-09-20T22:14:23.836622+0000","last_fullsized":"2024-09-20T22:14:23.836622+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:03.239256+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:03.239256+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":643,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.850523+0000","last_change":"2024-09-20T22:13:51.454966+0000","last_active":"2024-09-20T22:14:23.850523+0000","last_peered":"2024-09-20T22:14:23.850523+0000","last_clean":"2024-09-20T22:14:23.850523+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:14:23.850523+0000","last_undegraded":"2024-09-20T22:14:23.850523+0000","last_fullsized":"2024-09-20T22:14:23.850523+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.454850+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:55.603862+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":75,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.840880+0000","last_change":"2024-09-20T22:14:12.458599+0000","last_active":"2024-09-20T22:14:23.840880+0000","last_peered":"2024-09-20T22:14:23.840880+0000","last_clean":"2024-09-20T22:14:23.840880+0000","last_became_active":"2024-09-20T22:12:52.527547+0000","last_became_peered":"2024-09-20T22:12:52.527547+0000","last_unstale":"2024-09-20T22:14:23.840880+0000","last_undegraded":"2024-09-20T22:14:23.840880+0000","last_fullsized":"2024-09-20T22:14:23.840880+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:12.458492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:12.458492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:36.742957+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":611,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.858899+0000","last_change":"2024-09-20T22:13:51.008308+0000","last_active":"2024-09-20T22:14:23.858899+0000","last_peered":"2024-09-20T22:14:23.858899+0000","last_clean":"2024-09-20T22:14:23.858899+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:14:23.858899+0000","last_undegraded":"2024-09-20T22:14:23.858899+0000","last_fullsized":"2024-09-20T22:14:23.858899+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:51.008202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.008202+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:59.375373+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.a","version":"352'11342","reported_seq":11545,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:26.084899+0000","last_change":"2024-09-20T22:13:40.982764+0000","last_active":"2024-09-20T22:14:26.084899+0000","last_peered":"2024-09-20T22:14:26.084899+0000","last_clean":"2024-09-20T22:14:26.084899+0000","last_became_active":"2024-09-20T22:11:34.558703+0000","last_became_peered":"2024-09-20T22:11:34.558703+0000","last_unstale":"2024-09-20T22:14:26.084899+0000","last_undegraded":"2024-09-20T22:14:26.084899+0000","last_fullsized":"2024-09-20T22:14:26.084899+0000","mapping_epoch":286,"log_start":"331'6361","ondisk_log_start":"331'6361","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"343'8471","last_scrub_stamp":"2024-09-20T22:13:40.982606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:13:40.982606+0000","objects_scrubbed":5133,"log_size":4981,"log_dups_size":0,"ondisk_log_size":4981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:52.855309+0000","scrub_duration":1519,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":144834560,"num_objects":2210,"num_object_clones":0,"num_object_copies":6630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11342,"num_write_kb":433664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":635,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.842224+0000","last_change":"2024-09-20T22:12:55.518461+0000","last_active":"2024-09-20T22:14:23.842224+0000","last_peered":"2024-09-20T22:14:23.842224+0000","last_clean":"2024-09-20T22:14:23.842224+0000","last_became_active":"2024-09-20T22:12:55.518061+0000","last_became_peered":"2024-09-20T22:12:55.518061+0000","last_unstale":"2024-09-20T22:14:23.842224+0000","last_undegraded":"2024-09-20T22:14:23.842224+0000","last_fullsized":"2024-09-20T22:14:23.842224+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:12:42.665244+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:12:42.665244+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"328'17","reported_seq":539,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:13.754948+0000","last_change":"2024-09-20T22:13:51.684806+0000","last_active":"2024-09-20T22:14:13.754948+0000","last_peered":"2024-09-20T22:14:13.754948+0000","last_clean":"2024-09-20T22:14:13.754948+0000","last_became_active":"2024-09-20T22:12:52.521073+0000","last_became_peered":"2024-09-20T22:12:52.521073+0000","last_unstale":"2024-09-20T22:14:13.754948+0000","last_undegraded":"2024-09-20T22:14:13.754948+0000","last_fullsized":"2024-09-20T22:14:13.754948+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'17","last_scrub_stamp":"2024-09-20T22:13:51.684667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.684667+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:19.648229+0000","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":459,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:13.754416+0000","last_change":"2024-09-20T22:13:09.056527+0000","last_active":"2024-09-20T22:14:13.754416+0000","last_peered":"2024-09-20T22:14:13.754416+0000","last_clean":"2024-09-20T22:14:13.754416+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:14:13.754416+0000","last_undegraded":"2024-09-20T22:14:13.754416+0000","last_fullsized":"2024-09-20T22:14:13.754416+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:09.056286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:09.056286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":685,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.850123+0000","last_change":"2024-09-20T22:14:18.282947+0000","last_active":"2024-09-20T22:14:23.850123+0000","last_peered":"2024-09-20T22:14:23.850123+0000","last_clean":"2024-09-20T22:14:23.850123+0000","last_became_active":"2024-09-20T22:12:52.517255+0000","last_became_peered":"2024-09-20T22:12:52.517255+0000","last_unstale":"2024-09-20T22:14:23.850123+0000","last_undegraded":"2024-09-20T22:14:23.850123+0000","last_fullsized":"2024-09-20T22:14:23.850123+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:14:18.282814+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:14:18.282814+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:39.504754+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":639,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.851824+0000","last_change":"2024-09-20T22:14:10.238179+0000","last_active":"2024-09-20T22:14:23.851824+0000","last_peered":"2024-09-20T22:14:23.851824+0000","last_clean":"2024-09-20T22:14:23.851824+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:14:23.851824+0000","last_undegraded":"2024-09-20T22:14:23.851824+0000","last_fullsized":"2024-09-20T22:14:23.851824+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:10.237692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:14:10.237692+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:12.350855+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":612,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.845132+0000","last_change":"2024-09-20T22:13:41.730912+0000","last_active":"2024-09-20T22:14:23.845132+0000","last_peered":"2024-09-20T22:14:23.845132+0000","last_clean":"2024-09-20T22:14:23.845132+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:14:23.845132+0000","last_undegraded":"2024-09-20T22:14:23.845132+0000","last_fullsized":"2024-09-20T22:14:23.845132+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:41.730776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:41.730776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:10.936487+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.b","version":"352'11557","reported_seq":11734,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:26.906355+0000","last_change":"2024-09-20T22:14:09.472253+0000","last_active":"2024-09-20T22:14:26.906355+0000","last_peered":"2024-09-20T22:14:26.906355+0000","last_clean":"2024-09-20T22:14:26.906355+0000","last_became_active":"2024-09-20T22:12:58.947913+0000","last_became_peered":"2024-09-20T22:12:58.947913+0000","last_unstale":"2024-09-20T22:14:26.906355+0000","last_undegraded":"2024-09-20T22:14:26.906355+0000","last_fullsized":"2024-09-20T22:14:26.906355+0000","mapping_epoch":328,"log_start":"329'5800","ondisk_log_start":"329'5800","created":286,"last_epoch_clean":329,"parent":"0.0","parent_split_bits":0,"last_scrub":"349'10335","last_scrub_stamp":"2024-09-20T22:14:09.471976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:14:09.471976+0000","objects_scrubbed":3418,"log_size":5757,"log_dups_size":0,"ondisk_log_size":5757,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:31.788224+0000","scrub_duration":1034,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":141885440,"num_objects":2165,"num_object_clones":0,"num_object_copies":6495,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11557,"num_write_kb":439104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":65536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":624,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:13.755806+0000","last_change":"2024-09-20T22:13:48.490583+0000","last_active":"2024-09-20T22:14:13.755806+0000","last_peered":"2024-09-20T22:14:13.755806+0000","last_clean":"2024-09-20T22:14:13.755806+0000","last_became_active":"2024-09-20T22:08:48.446840+0000","last_became_peered":"2024-09-20T22:08:48.446840+0000","last_unstale":"2024-09-20T22:14:13.755806+0000","last_undegraded":"2024-09-20T22:14:13.755806+0000","last_fullsized":"2024-09-20T22:14:13.755806+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:13:48.490464+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:17.706581+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"328'24","reported_seq":107,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.840101+0000","last_change":"2024-09-20T22:13:44.523683+0000","last_active":"2024-09-20T22:14:23.840101+0000","last_peered":"2024-09-20T22:14:23.840101+0000","last_clean":"2024-09-20T22:14:23.840101+0000","last_became_active":"2024-09-20T22:12:52.527919+0000","last_became_peered":"2024-09-20T22:12:52.527919+0000","last_unstale":"2024-09-20T22:14:23.840101+0000","last_undegraded":"2024-09-20T22:14:23.840101+0000","last_fullsized":"2024-09-20T22:14:23.840101+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'24","last_scrub_stamp":"2024-09-20T22:13:44.523540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:44.523540+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:00.404633+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":75,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.839839+0000","last_change":"2024-09-20T22:13:41.549500+0000","last_active":"2024-09-20T22:14:23.839839+0000","last_peered":"2024-09-20T22:14:23.839839+0000","last_clean":"2024-09-20T22:14:23.839839+0000","last_became_active":"2024-09-20T22:12:52.517432+0000","last_became_peered":"2024-09-20T22:12:52.517432+0000","last_unstale":"2024-09-20T22:14:23.839839+0000","last_undegraded":"2024-09-20T22:14:23.839839+0000","last_fullsized":"2024-09-20T22:14:23.839839+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:41.549376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:13:41.549376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:49.137072+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":1343,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:24.682940+0000","last_change":"2024-09-20T22:13:06.609600+0000","last_active":"2024-09-20T22:14:24.682940+0000","last_peered":"2024-09-20T22:14:24.682940+0000","last_clean":"2024-09-20T22:14:24.682940+0000","last_became_active":"2024-09-20T22:12:52.526622+0000","last_became_peered":"2024-09-20T22:12:52.526622+0000","last_unstale":"2024-09-20T22:14:24.682940+0000","last_undegraded":"2024-09-20T22:14:24.682940+0000","last_fullsized":"2024-09-20T22:14:24.682940+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:13:06.609525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:13:06.609525+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:33.606536+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":672,"num_read_kb":667,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":178,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.849610+0000","last_change":"2024-09-20T22:14:14.443714+0000","last_active":"2024-09-20T22:14:23.849610+0000","last_peered":"2024-09-20T22:14:23.849610+0000","last_clean":"2024-09-20T22:14:23.849610+0000","last_became_active":"2024-09-20T22:12:52.518823+0000","last_became_peered":"2024-09-20T22:12:52.518823+0000","last_unstale":"2024-09-20T22:14:23.849610+0000","last_undegraded":"2024-09-20T22:14:23.849610+0000","last_fullsized":"2024-09-20T22:14:23.849610+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:14.443611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:14.443611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:22.004118+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":459,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:13.754447+0000","last_change":"2024-09-20T22:13:40.433856+0000","last_active":"2024-09-20T22:14:13.754447+0000","last_peered":"2024-09-20T22:14:13.754447+0000","last_clean":"2024-09-20T22:14:13.754447+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:14:13.754447+0000","last_undegraded":"2024-09-20T22:14:13.754447+0000","last_fullsized":"2024-09-20T22:14:13.754447+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:40.433606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.415468+0000","last_clean_scrub_stamp":"2024-09-20T22:13:40.433606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:55.124968+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.8","version":"352'11828","reported_seq":12036,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:26.899231+0000","last_change":"2024-09-20T22:13:49.901852+0000","last_active":"2024-09-20T22:14:26.899231+0000","last_peered":"2024-09-20T22:14:26.899231+0000","last_clean":"2024-09-20T22:14:26.899231+0000","last_became_active":"2024-09-20T22:12:53.511966+0000","last_became_peered":"2024-09-20T22:12:53.511966+0000","last_unstale":"2024-09-20T22:14:26.899231+0000","last_undegraded":"2024-09-20T22:14:26.899231+0000","last_fullsized":"2024-09-20T22:14:26.899231+0000","mapping_epoch":324,"log_start":"330'6158","ondisk_log_start":"330'6158","created":286,"last_epoch_clean":325,"parent":"0.0","parent_split_bits":0,"last_scrub":"345'9333","last_scrub_stamp":"2024-09-20T22:13:49.901749+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:13:49.901749+0000","objects_scrubbed":4713,"log_size":5670,"log_dups_size":0,"ondisk_log_size":5670,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:04.124141+0000","scrub_duration":1462,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143130624,"num_objects":2184,"num_object_clones":0,"num_object_copies":6552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11828,"num_write_kb":448384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"328'28","reported_seq":817,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-20T22:14:27.275231+0000","last_change":"2024-09-20T22:14:21.445235+0000","last_active":"2024-09-20T22:14:27.275231+0000","last_peered":"2024-09-20T22:14:27.275231+0000","last_clean":"2024-09-20T22:14:27.275231+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:14:27.275231+0000","last_undegraded":"2024-09-20T22:14:27.275231+0000","last_fullsized":"2024-09-20T22:14:27.275231+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:14:21.445103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:21.445103+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:22.472336+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":584,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.850233+0000","last_change":"2024-09-20T22:12:42.260316+0000","last_active":"2024-09-20T22:14:23.850233+0000","last_peered":"2024-09-20T22:14:23.850233+0000","last_clean":"2024-09-20T22:14:23.850233+0000","last_became_active":"2024-09-20T22:12:21.519717+0000","last_became_peered":"2024-09-20T22:12:21.519717+0000","last_unstale":"2024-09-20T22:14:23.850233+0000","last_undegraded":"2024-09-20T22:14:23.850233+0000","last_fullsized":"2024-09-20T22:14:23.850233+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:42.260197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:42.260197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":634,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.841999+0000","last_change":"2024-09-20T22:12:55.522906+0000","last_active":"2024-09-20T22:14:23.841999+0000","last_peered":"2024-09-20T22:14:23.841999+0000","last_clean":"2024-09-20T22:14:23.841999+0000","last_became_active":"2024-09-20T22:12:55.522325+0000","last_became_peered":"2024-09-20T22:12:55.522325+0000","last_unstale":"2024-09-20T22:14:23.841999+0000","last_undegraded":"2024-09-20T22:14:23.841999+0000","last_fullsized":"2024-09-20T22:14:23.841999+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:12:40.635888+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:12:40.635888+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":206,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.843060+0000","last_change":"2024-09-20T22:13:51.675128+0000","last_active":"2024-09-20T22:14:23.843060+0000","last_peered":"2024-09-20T22:14:23.843060+0000","last_clean":"2024-09-20T22:14:23.843060+0000","last_became_active":"2024-09-20T22:10:48.907271+0000","last_became_peered":"2024-09-20T22:10:48.907271+0000","last_unstale":"2024-09-20T22:14:23.843060+0000","last_undegraded":"2024-09-20T22:14:23.843060+0000","last_fullsized":"2024-09-20T22:14:23.843060+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.674976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:02.287074+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":639,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.849439+0000","last_change":"2024-09-20T22:13:02.482655+0000","last_active":"2024-09-20T22:14:23.849439+0000","last_peered":"2024-09-20T22:14:23.849439+0000","last_clean":"2024-09-20T22:14:23.849439+0000","last_became_active":"2024-09-20T22:12:44.630044+0000","last_became_peered":"2024-09-20T22:12:44.630044+0000","last_unstale":"2024-09-20T22:14:23.849439+0000","last_undegraded":"2024-09-20T22:14:23.849439+0000","last_fullsized":"2024-09-20T22:14:23.849439+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:02.482546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:13:02.482546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3","version":"352'11662","reported_seq":11864,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:25.796955+0000","last_change":"2024-09-20T22:13:44.166688+0000","last_active":"2024-09-20T22:14:25.796955+0000","last_peered":"2024-09-20T22:14:25.796955+0000","last_clean":"2024-09-20T22:14:25.796955+0000","last_became_active":"2024-09-20T22:11:34.559353+0000","last_became_peered":"2024-09-20T22:11:34.559353+0000","last_unstale":"2024-09-20T22:14:25.796955+0000","last_undegraded":"2024-09-20T22:14:25.796955+0000","last_fullsized":"2024-09-20T22:14:25.796955+0000","mapping_epoch":286,"log_start":"333'7171","ondisk_log_start":"333'7171","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"345'8915","last_scrub_stamp":"2024-09-20T22:13:44.166549+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:13:44.166549+0000","objects_scrubbed":5006,"log_size":4491,"log_dups_size":0,"ondisk_log_size":4491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:55.327841+0000","scrub_duration":1425,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145096704,"num_objects":2214,"num_object_clones":0,"num_object_copies":6642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11662,"num_write_kb":444032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,1],"acting":[7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"328'19","reported_seq":652,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.836623+0000","last_change":"2024-09-20T22:12:47.150182+0000","last_active":"2024-09-20T22:14:23.836623+0000","last_peered":"2024-09-20T22:14:23.836623+0000","last_clean":"2024-09-20T22:14:23.836623+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:14:23.836623+0000","last_undegraded":"2024-09-20T22:14:23.836623+0000","last_fullsized":"2024-09-20T22:14:23.836623+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:12:47.150099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:47.150099+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":624,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.836014+0000","last_change":"2024-09-20T22:13:39.055297+0000","last_active":"2024-09-20T22:14:23.836014+0000","last_peered":"2024-09-20T22:14:23.836014+0000","last_clean":"2024-09-20T22:14:23.836014+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:14:23.836014+0000","last_undegraded":"2024-09-20T22:14:23.836014+0000","last_fullsized":"2024-09-20T22:14:23.836014+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:39.055192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:39.055192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:54.441568+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":623,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.849324+0000","last_change":"2024-09-20T22:12:13.155581+0000","last_active":"2024-09-20T22:14:23.849324+0000","last_peered":"2024-09-20T22:14:23.849324+0000","last_clean":"2024-09-20T22:14:23.849324+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:14:23.849324+0000","last_undegraded":"2024-09-20T22:14:23.849324+0000","last_fullsized":"2024-09-20T22:14:23.849324+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:13.155346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:13.155346+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":580,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-20T22:14:27.275520+0000","last_change":"2024-09-20T22:14:08.297827+0000","last_active":"2024-09-20T22:14:27.275520+0000","last_peered":"2024-09-20T22:14:27.275520+0000","last_clean":"2024-09-20T22:14:27.275520+0000","last_became_active":"2024-09-20T22:12:21.521195+0000","last_became_peered":"2024-09-20T22:12:21.521195+0000","last_unstale":"2024-09-20T22:14:27.275520+0000","last_undegraded":"2024-09-20T22:14:27.275520+0000","last_fullsized":"2024-09-20T22:14:27.275520+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:08.297698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:08.297698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:18.738324+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.0","version":"351'11447","reported_seq":11645,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.303590+0000","last_change":"2024-09-20T22:13:44.909535+0000","last_active":"2024-09-20T22:14:23.303590+0000","last_peered":"2024-09-20T22:14:23.303590+0000","last_clean":"2024-09-20T22:14:23.303590+0000","last_became_active":"2024-09-20T22:11:35.304302+0000","last_became_peered":"2024-09-20T22:11:35.304302+0000","last_unstale":"2024-09-20T22:14:23.303590+0000","last_undegraded":"2024-09-20T22:14:23.303590+0000","last_fullsized":"2024-09-20T22:14:23.303590+0000","mapping_epoch":286,"log_start":"333'6827","ondisk_log_start":"333'6827","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"345'8938","last_scrub_stamp":"2024-09-20T22:13:44.909161+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:13:44.909161+0000","objects_scrubbed":4947,"log_size":4620,"log_dups_size":0,"ondisk_log_size":4620,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:48.876107+0000","scrub_duration":1473,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":157351936,"num_objects":2401,"num_object_clones":0,"num_object_copies":7203,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2401,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11447,"num_write_kb":443136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":530,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.848324+0000","last_change":"2024-09-20T22:12:15.134763+0000","last_active":"2024-09-20T22:14:23.848324+0000","last_peered":"2024-09-20T22:14:23.848324+0000","last_clean":"2024-09-20T22:14:23.848324+0000","last_became_active":"2024-09-20T22:11:33.557208+0000","last_became_peered":"2024-09-20T22:11:33.557208+0000","last_unstale":"2024-09-20T22:14:23.848324+0000","last_undegraded":"2024-09-20T22:14:23.848324+0000","last_fullsized":"2024-09-20T22:14:23.848324+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:12:15.134639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:12:15.134639+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,3],"acting":[5,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"328'31","reported_seq":678,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.836628+0000","last_change":"2024-09-20T22:13:11.188414+0000","last_active":"2024-09-20T22:14:23.836628+0000","last_peered":"2024-09-20T22:14:23.836628+0000","last_clean":"2024-09-20T22:14:23.836628+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:14:23.836628+0000","last_undegraded":"2024-09-20T22:14:23.836628+0000","last_fullsized":"2024-09-20T22:14:23.836628+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'31","last_scrub_stamp":"2024-09-20T22:13:11.188264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:11.188264+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:28.710359+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":642,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-20T22:14:27.275626+0000","last_change":"2024-09-20T22:14:12.293291+0000","last_active":"2024-09-20T22:14:27.275626+0000","last_peered":"2024-09-20T22:14:27.275626+0000","last_clean":"2024-09-20T22:14:27.275626+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:14:27.275626+0000","last_undegraded":"2024-09-20T22:14:27.275626+0000","last_fullsized":"2024-09-20T22:14:27.275626+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:12.293188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:12.293188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:18.483394+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":471,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:13.754539+0000","last_change":"2024-09-20T22:13:10.039183+0000","last_active":"2024-09-20T22:14:13.754539+0000","last_peered":"2024-09-20T22:14:13.754539+0000","last_clean":"2024-09-20T22:14:13.754539+0000","last_became_active":"2024-09-20T22:12:52.520612+0000","last_became_peered":"2024-09-20T22:12:52.520612+0000","last_unstale":"2024-09-20T22:14:13.754539+0000","last_undegraded":"2024-09-20T22:14:13.754539+0000","last_fullsized":"2024-09-20T22:14:13.754539+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:10.039074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.039074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:27.582640+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":627,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.851218+0000","last_change":"2024-09-20T22:13:01.042002+0000","last_active":"2024-09-20T22:14:23.851218+0000","last_peered":"2024-09-20T22:14:23.851218+0000","last_clean":"2024-09-20T22:14:23.851218+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:14:23.851218+0000","last_undegraded":"2024-09-20T22:14:23.851218+0000","last_fullsized":"2024-09-20T22:14:23.851218+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:01.041863+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:01.041863+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1","version":"353'11633","reported_seq":11836,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-20T22:14:27.897723+0000","last_change":"2024-09-20T22:13:51.395292+0000","last_active":"2024-09-20T22:14:27.897723+0000","last_peered":"2024-09-20T22:14:27.897723+0000","last_clean":"2024-09-20T22:14:27.897723+0000","last_became_active":"2024-09-20T22:12:21.522567+0000","last_became_peered":"2024-09-20T22:12:21.522567+0000","last_unstale":"2024-09-20T22:14:27.897723+0000","last_undegraded":"2024-09-20T22:14:27.897723+0000","last_fullsized":"2024-09-20T22:14:27.897723+0000","mapping_epoch":309,"log_start":"329'5776","ondisk_log_start":"329'5776","created":286,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"345'9225","last_scrub_stamp":"2024-09-20T22:13:51.395052+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.395052+0000","objects_scrubbed":4519,"log_size":5857,"log_dups_size":0,"ondisk_log_size":5857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:53.188292+0000","scrub_duration":1025,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":136380416,"num_objects":2081,"num_object_clones":0,"num_object_copies":6243,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2081,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11633,"num_write_kb":438848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":904,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.842924+0000","last_change":"2024-09-20T22:13:49.912616+0000","last_active":"2024-09-20T22:14:23.842924+0000","last_peered":"2024-09-20T22:14:23.842924+0000","last_clean":"2024-09-20T22:14:23.842924+0000","last_became_active":"2024-09-20T22:08:48.455288+0000","last_became_peered":"2024-09-20T22:08:48.455288+0000","last_unstale":"2024-09-20T22:14:23.842924+0000","last_undegraded":"2024-09-20T22:14:23.842924+0000","last_fullsized":"2024-09-20T22:14:23.842924+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:49.912496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:13:49.912496+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:12.232772+0000","scrub_duration":227,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"328'25","reported_seq":655,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.836274+0000","last_change":"2024-09-20T22:14:10.070008+0000","last_active":"2024-09-20T22:14:23.836274+0000","last_peered":"2024-09-20T22:14:23.836274+0000","last_clean":"2024-09-20T22:14:23.836274+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:14:23.836274+0000","last_undegraded":"2024-09-20T22:14:23.836274+0000","last_fullsized":"2024-09-20T22:14:23.836274+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'25","last_scrub_stamp":"2024-09-20T22:14:10.069812+0000","last_deep_scrub":"289'15","last_deep_scrub_stamp":"2024-09-20T22:11:37.277648+0000","last_clean_scrub_stamp":"2024-09-20T22:14:10.069812+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:20.381924+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":79,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.840477+0000","last_change":"2024-09-20T22:14:23.473613+0000","last_active":"2024-09-20T22:14:23.840477+0000","last_peered":"2024-09-20T22:14:23.840477+0000","last_clean":"2024-09-20T22:14:23.840477+0000","last_became_active":"2024-09-20T22:12:52.528310+0000","last_became_peered":"2024-09-20T22:12:52.528310+0000","last_unstale":"2024-09-20T22:14:23.840477+0000","last_undegraded":"2024-09-20T22:14:23.840477+0000","last_fullsized":"2024-09-20T22:14:23.840477+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:14:23.472792+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:14:23.472792+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:27.168827+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"283'11","reported_seq":816,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.850836+0000","last_change":"2024-09-20T22:14:20.265212+0000","last_active":"2024-09-20T22:14:23.850836+0000","last_peered":"2024-09-20T22:14:23.850836+0000","last_clean":"2024-09-20T22:14:23.850836+0000","last_became_active":"2024-09-20T22:11:30.485356+0000","last_became_peered":"2024-09-20T22:11:30.485356+0000","last_unstale":"2024-09-20T22:14:23.850836+0000","last_undegraded":"2024-09-20T22:14:23.850836+0000","last_fullsized":"2024-09-20T22:14:23.850836+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'11","last_scrub_stamp":"2024-09-20T22:14:20.265107+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:14:20.265107+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:43.795761+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":613,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.836299+0000","last_change":"2024-09-20T22:12:11.210404+0000","last_active":"2024-09-20T22:14:23.836299+0000","last_peered":"2024-09-20T22:14:23.836299+0000","last_clean":"2024-09-20T22:14:23.836299+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:14:23.836299+0000","last_undegraded":"2024-09-20T22:14:23.836299+0000","last_fullsized":"2024-09-20T22:14:23.836299+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:11.210279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:12:11.210279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.6","version":"352'11852","reported_seq":12015,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:26.079026+0000","last_change":"2024-09-20T22:14:23.871069+0000","last_active":"2024-09-20T22:14:26.079026+0000","last_peered":"2024-09-20T22:14:26.079026+0000","last_clean":"2024-09-20T22:14:26.079026+0000","last_became_active":"2024-09-20T22:11:35.304192+0000","last_became_peered":"2024-09-20T22:11:35.304192+0000","last_unstale":"2024-09-20T22:14:26.079026+0000","last_undegraded":"2024-09-20T22:14:26.079026+0000","last_fullsized":"2024-09-20T22:14:26.079026+0000","mapping_epoch":286,"log_start":"333'6915","ondisk_log_start":"333'6915","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:11:33.546173+0000","objects_scrubbed":1699,"log_size":4937,"log_dups_size":0,"ondisk_log_size":4937,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:28.868492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":146800640,"num_objects":2240,"num_object_clones":0,"num_object_copies":6720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11852,"num_write_kb":450944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"328'15","reported_seq":94,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.840862+0000","last_change":"2024-09-20T22:13:46.474322+0000","last_active":"2024-09-20T22:14:23.840862+0000","last_peered":"2024-09-20T22:14:23.840862+0000","last_clean":"2024-09-20T22:14:23.840862+0000","last_became_active":"2024-09-20T22:12:52.527531+0000","last_became_peered":"2024-09-20T22:12:52.527531+0000","last_unstale":"2024-09-20T22:14:23.840862+0000","last_undegraded":"2024-09-20T22:14:23.840862+0000","last_fullsized":"2024-09-20T22:14:23.840862+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:13:46.473818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:46.473818+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:59.832835+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":251,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.860480+0000","last_change":"2024-09-20T22:13:01.819308+0000","last_active":"2024-09-20T22:14:23.860480+0000","last_peered":"2024-09-20T22:14:23.860480+0000","last_clean":"2024-09-20T22:14:23.860480+0000","last_became_active":"2024-09-20T22:10:14.536155+0000","last_became_peered":"2024-09-20T22:10:14.536155+0000","last_unstale":"2024-09-20T22:14:23.860480+0000","last_undegraded":"2024-09-20T22:14:23.860480+0000","last_fullsized":"2024-09-20T22:14:23.860480+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:13:01.818638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:01.818638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":639,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.850693+0000","last_change":"2024-09-20T22:13:43.106240+0000","last_active":"2024-09-20T22:14:23.850693+0000","last_peered":"2024-09-20T22:14:23.850693+0000","last_clean":"2024-09-20T22:14:23.850693+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:14:23.850693+0000","last_undegraded":"2024-09-20T22:14:23.850693+0000","last_fullsized":"2024-09-20T22:14:23.850693+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:43.106113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:13:43.106113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:00.182815+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":67,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.839925+0000","last_change":"2024-09-20T22:12:52.516597+0000","last_active":"2024-09-20T22:14:23.839925+0000","last_peered":"2024-09-20T22:14:23.839925+0000","last_clean":"2024-09-20T22:14:23.839925+0000","last_became_active":"2024-09-20T22:12:52.512924+0000","last_became_peered":"2024-09-20T22:12:52.512924+0000","last_unstale":"2024-09-20T22:14:23.839925+0000","last_undegraded":"2024-09-20T22:14:23.839925+0000","last_fullsized":"2024-09-20T22:14:23.839925+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:00.473894+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:00.473894+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"352'11580","reported_seq":11780,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:26.077272+0000","last_change":"2024-09-20T22:14:12.327450+0000","last_active":"2024-09-20T22:14:26.077272+0000","last_peered":"2024-09-20T22:14:26.077272+0000","last_clean":"2024-09-20T22:14:26.077272+0000","last_became_active":"2024-09-20T22:11:40.601030+0000","last_became_peered":"2024-09-20T22:11:40.601030+0000","last_unstale":"2024-09-20T22:14:26.077272+0000","last_undegraded":"2024-09-20T22:14:26.077272+0000","last_fullsized":"2024-09-20T22:14:26.077272+0000","mapping_epoch":290,"log_start":"331'6578","ondisk_log_start":"331'6578","created":286,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"350'10594","last_scrub_stamp":"2024-09-20T22:14:12.327135+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:14:12.327135+0000","objects_scrubbed":3245,"log_size":5002,"log_dups_size":0,"ondisk_log_size":5002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.091574+0000","scrub_duration":1924,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":144441344,"num_objects":2204,"num_object_clones":0,"num_object_copies":6612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2204,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11580,"num_write_kb":441088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"328'35","reported_seq":737,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-20T22:14:27.275176+0000","last_change":"2024-09-20T22:13:51.408635+0000","last_active":"2024-09-20T22:14:27.275176+0000","last_peered":"2024-09-20T22:14:27.275176+0000","last_clean":"2024-09-20T22:14:27.275176+0000","last_became_active":"2024-09-20T22:10:30.652967+0000","last_became_peered":"2024-09-20T22:10:30.652967+0000","last_unstale":"2024-09-20T22:14:27.275176+0000","last_undegraded":"2024-09-20T22:14:27.275176+0000","last_fullsized":"2024-09-20T22:14:27.275176+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'35","last_scrub_stamp":"2024-09-20T22:13:51.408543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.408543+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:14.979728+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":636,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.845410+0000","last_change":"2024-09-20T22:12:46.602166+0000","last_active":"2024-09-20T22:14:23.845410+0000","last_peered":"2024-09-20T22:14:23.845410+0000","last_clean":"2024-09-20T22:14:23.845410+0000","last_became_active":"2024-09-20T22:11:30.488650+0000","last_became_peered":"2024-09-20T22:11:30.488650+0000","last_unstale":"2024-09-20T22:14:23.845410+0000","last_undegraded":"2024-09-20T22:14:23.845410+0000","last_fullsized":"2024-09-20T22:14:23.845410+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:46.602035+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:12:46.602035+0000","last_clean_scrub_stamp":"2024-09-20T22:12:46.602035+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":663,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.850568+0000","last_change":"2024-09-20T22:13:49.085883+0000","last_active":"2024-09-20T22:14:23.850568+0000","last_peered":"2024-09-20T22:14:23.850568+0000","last_clean":"2024-09-20T22:14:23.850568+0000","last_became_active":"2024-09-20T22:12:52.515295+0000","last_became_peered":"2024-09-20T22:12:52.515295+0000","last_unstale":"2024-09-20T22:14:23.850568+0000","last_undegraded":"2024-09-20T22:14:23.850568+0000","last_fullsized":"2024-09-20T22:14:23.850568+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:49.085768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_clean_scrub_stamp":"2024-09-20T22:13:49.085768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:51.925476+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":220,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.842216+0000","last_change":"2024-09-20T22:12:12.517840+0000","last_active":"2024-09-20T22:14:23.842216+0000","last_peered":"2024-09-20T22:14:23.842216+0000","last_clean":"2024-09-20T22:14:23.842216+0000","last_became_active":"2024-09-20T22:10:30.657116+0000","last_became_peered":"2024-09-20T22:10:30.657116+0000","last_unstale":"2024-09-20T22:14:23.842216+0000","last_undegraded":"2024-09-20T22:14:23.842216+0000","last_fullsized":"2024-09-20T22:14:23.842216+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:12.517732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:12.517732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.4","version":"351'11340","reported_seq":11578,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.325046+0000","last_change":"2024-09-20T22:14:17.505401+0000","last_active":"2024-09-20T22:14:23.325046+0000","last_peered":"2024-09-20T22:14:23.325046+0000","last_clean":"2024-09-20T22:14:23.325046+0000","last_became_active":"2024-09-20T22:11:34.557976+0000","last_became_peered":"2024-09-20T22:11:34.557976+0000","last_unstale":"2024-09-20T22:14:23.325046+0000","last_undegraded":"2024-09-20T22:14:23.325046+0000","last_fullsized":"2024-09-20T22:14:23.325046+0000","mapping_epoch":286,"log_start":"333'6724","ondisk_log_start":"333'6724","created":286,"last_epoch_clean":287,"parent":"0.0","parent_split_bits":0,"last_scrub":"351'10935","last_scrub_stamp":"2024-09-20T22:14:17.505268+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:14:17.505268+0000","objects_scrubbed":2756,"log_size":4616,"log_dups_size":0,"ondisk_log_size":4616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:29.000970+0000","scrub_duration":1080,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":151781376,"num_objects":2316,"num_object_clones":0,"num_object_copies":6948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11340,"num_write_kb":436992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.9","version":"328'18","reported_seq":653,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.835774+0000","last_change":"2024-09-20T22:13:10.164964+0000","last_active":"2024-09-20T22:14:23.835774+0000","last_peered":"2024-09-20T22:14:23.835774+0000","last_clean":"2024-09-20T22:14:23.835774+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:14:23.835774+0000","last_undegraded":"2024-09-20T22:14:23.835774+0000","last_fullsized":"2024-09-20T22:14:23.835774+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'18","last_scrub_stamp":"2024-09-20T22:13:10.164833+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.164833+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":633,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.843370+0000","last_change":"2024-09-20T22:14:15.965211+0000","last_active":"2024-09-20T22:14:23.843370+0000","last_peered":"2024-09-20T22:14:23.843370+0000","last_clean":"2024-09-20T22:14:23.843370+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:14:23.843370+0000","last_undegraded":"2024-09-20T22:14:23.843370+0000","last_fullsized":"2024-09-20T22:14:23.843370+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:15.965013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:15.965013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:17.753570+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":660,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.851135+0000","last_change":"2024-09-20T22:13:53.202635+0000","last_active":"2024-09-20T22:14:23.851135+0000","last_peered":"2024-09-20T22:14:23.851135+0000","last_clean":"2024-09-20T22:14:23.851135+0000","last_became_active":"2024-09-20T22:12:52.514202+0000","last_became_peered":"2024-09-20T22:12:52.514202+0000","last_unstale":"2024-09-20T22:14:23.851135+0000","last_undegraded":"2024-09-20T22:14:23.851135+0000","last_fullsized":"2024-09-20T22:14:23.851135+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:53.202518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:53.202518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:12.953838+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":85,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.855922+0000","last_change":"2024-09-20T22:13:42.822386+0000","last_active":"2024-09-20T22:14:23.855922+0000","last_peered":"2024-09-20T22:14:23.855922+0000","last_clean":"2024-09-20T22:14:23.855922+0000","last_became_active":"2024-09-20T22:12:44.638546+0000","last_became_peered":"2024-09-20T22:12:44.638546+0000","last_unstale":"2024-09-20T22:14:23.855922+0000","last_undegraded":"2024-09-20T22:14:23.855922+0000","last_fullsized":"2024-09-20T22:14:23.855922+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:42.822274+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:42.822274+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:59.413720+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.5","version":"352'11556","reported_seq":11985,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:26.889979+0000","last_change":"2024-09-20T22:13:51.600919+0000","last_active":"2024-09-20T22:14:26.889979+0000","last_peered":"2024-09-20T22:14:26.889979+0000","last_clean":"2024-09-20T22:14:26.889979+0000","last_became_active":"2024-09-20T22:12:53.511729+0000","last_became_peered":"2024-09-20T22:12:53.511729+0000","last_unstale":"2024-09-20T22:14:26.889979+0000","last_undegraded":"2024-09-20T22:14:26.889979+0000","last_fullsized":"2024-09-20T22:14:26.889979+0000","mapping_epoch":324,"log_start":"329'5856","ondisk_log_start":"329'5856","created":286,"last_epoch_clean":325,"parent":"0.0","parent_split_bits":0,"last_scrub":"345'9142","last_scrub_stamp":"2024-09-20T22:13:51.600650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:33.546173+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.600650+0000","objects_scrubbed":4571,"log_size":5700,"log_dups_size":0,"ondisk_log_size":5700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:54.668893+0000","scrub_duration":1129,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":139722752,"num_objects":2132,"num_object_clones":0,"num_object_copies":6396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2132,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11556,"num_write_kb":438016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":112,"num_bytes_recovered":7340032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"328'27","reported_seq":192,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.848179+0000","last_change":"2024-09-20T22:13:10.353607+0000","last_active":"2024-09-20T22:14:23.848179+0000","last_peered":"2024-09-20T22:14:23.848179+0000","last_clean":"2024-09-20T22:14:23.848179+0000","last_became_active":"2024-09-20T22:11:30.493275+0000","last_became_peered":"2024-09-20T22:11:30.493275+0000","last_unstale":"2024-09-20T22:14:23.848179+0000","last_undegraded":"2024-09-20T22:14:23.848179+0000","last_fullsized":"2024-09-20T22:14:23.848179+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:13:10.353459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.353459+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:33.012468+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":250,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.843779+0000","last_change":"2024-09-20T22:12:52.541194+0000","last_active":"2024-09-20T22:14:23.843779+0000","last_peered":"2024-09-20T22:14:23.843779+0000","last_clean":"2024-09-20T22:14:23.843779+0000","last_became_active":"2024-09-20T22:12:52.526551+0000","last_became_peered":"2024-09-20T22:12:52.526551+0000","last_unstale":"2024-09-20T22:14:23.843779+0000","last_undegraded":"2024-09-20T22:14:23.843779+0000","last_fullsized":"2024-09-20T22:14:23.843779+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:12:37.642296+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:12:37.642296+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":764,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.842732+0000","last_change":"2024-09-20T22:14:12.911852+0000","last_active":"2024-09-20T22:14:23.842732+0000","last_peered":"2024-09-20T22:14:23.842732+0000","last_clean":"2024-09-20T22:14:23.842732+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:14:23.842732+0000","last_undegraded":"2024-09-20T22:14:23.842732+0000","last_fullsized":"2024-09-20T22:14:23.842732+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:14:12.911520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:12.911520+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.190103+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":170,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.850351+0000","last_change":"2024-09-20T22:13:00.454973+0000","last_active":"2024-09-20T22:14:23.850351+0000","last_peered":"2024-09-20T22:14:23.850351+0000","last_clean":"2024-09-20T22:14:23.850351+0000","last_became_active":"2024-09-20T22:12:52.517758+0000","last_became_peered":"2024-09-20T22:12:52.517758+0000","last_unstale":"2024-09-20T22:14:23.850351+0000","last_undegraded":"2024-09-20T22:14:23.850351+0000","last_fullsized":"2024-09-20T22:14:23.850351+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:00.454378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:00.454378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":665,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-20T22:14:27.275813+0000","last_change":"2024-09-20T22:13:04.905929+0000","last_active":"2024-09-20T22:14:27.275813+0000","last_peered":"2024-09-20T22:14:27.275813+0000","last_clean":"2024-09-20T22:14:27.275813+0000","last_became_active":"2024-09-20T22:12:52.516989+0000","last_became_peered":"2024-09-20T22:12:52.516989+0000","last_unstale":"2024-09-20T22:14:27.275813+0000","last_undegraded":"2024-09-20T22:14:27.275813+0000","last_fullsized":"2024-09-20T22:14:27.275813+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:04.905801+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:04.905801+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:31.595778+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"328'12","reported_seq":664,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.858880+0000","last_change":"2024-09-20T22:13:10.726072+0000","last_active":"2024-09-20T22:14:23.858880+0000","last_peered":"2024-09-20T22:14:23.858880+0000","last_clean":"2024-09-20T22:14:23.858880+0000","last_became_active":"2024-09-20T22:10:48.922561+0000","last_became_peered":"2024-09-20T22:10:48.922561+0000","last_unstale":"2024-09-20T22:14:23.858880+0000","last_undegraded":"2024-09-20T22:14:23.858880+0000","last_fullsized":"2024-09-20T22:14:23.858880+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'12","last_scrub_stamp":"2024-09-20T22:13:10.725057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.725057+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:36.040129+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":158,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.860507+0000","last_change":"2024-09-20T22:12:14.148857+0000","last_active":"2024-09-20T22:14:23.860507+0000","last_peered":"2024-09-20T22:14:23.860507+0000","last_clean":"2024-09-20T22:14:23.860507+0000","last_became_active":"2024-09-20T22:11:30.487322+0000","last_became_peered":"2024-09-20T22:11:30.487322+0000","last_unstale":"2024-09-20T22:14:23.860507+0000","last_undegraded":"2024-09-20T22:14:23.860507+0000","last_fullsized":"2024-09-20T22:14:23.860507+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:14.148743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:14.148743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":603,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.845034+0000","last_change":"2024-09-20T22:12:05.541834+0000","last_active":"2024-09-20T22:14:23.845034+0000","last_peered":"2024-09-20T22:14:23.845034+0000","last_clean":"2024-09-20T22:14:23.845034+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:14:23.845034+0000","last_undegraded":"2024-09-20T22:14:23.845034+0000","last_fullsized":"2024-09-20T22:14:23.845034+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:05.541367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:05.541367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"328'15","reported_seq":94,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.840283+0000","last_change":"2024-09-20T22:14:14.397650+0000","last_active":"2024-09-20T22:14:23.840283+0000","last_peered":"2024-09-20T22:14:23.840283+0000","last_clean":"2024-09-20T22:14:23.840283+0000","last_became_active":"2024-09-20T22:12:52.527106+0000","last_became_peered":"2024-09-20T22:12:52.527106+0000","last_unstale":"2024-09-20T22:14:23.840283+0000","last_undegraded":"2024-09-20T22:14:23.840283+0000","last_fullsized":"2024-09-20T22:14:23.840283+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:14:14.397530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:14.397530+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:42.176603+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":623,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.843135+0000","last_change":"2024-09-20T22:12:44.676449+0000","last_active":"2024-09-20T22:14:23.843135+0000","last_peered":"2024-09-20T22:14:23.843135+0000","last_clean":"2024-09-20T22:14:23.843135+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:14:23.843135+0000","last_undegraded":"2024-09-20T22:14:23.843135+0000","last_fullsized":"2024-09-20T22:14:23.843135+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:44.676334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:44.676334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":259,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:13.754640+0000","last_change":"2024-09-20T22:13:14.136545+0000","last_active":"2024-09-20T22:14:13.754640+0000","last_peered":"2024-09-20T22:14:13.754640+0000","last_clean":"2024-09-20T22:14:13.754640+0000","last_became_active":"2024-09-20T22:12:33.545506+0000","last_became_peered":"2024-09-20T22:12:33.545506+0000","last_unstale":"2024-09-20T22:14:13.754640+0000","last_undegraded":"2024-09-20T22:14:13.754640+0000","last_fullsized":"2024-09-20T22:14:13.754640+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:14.136472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_clean_scrub_stamp":"2024-09-20T22:13:14.136472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:33.179397+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":634,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.851561+0000","last_change":"2024-09-20T22:13:52.203910+0000","last_active":"2024-09-20T22:14:23.851561+0000","last_peered":"2024-09-20T22:14:23.851561+0000","last_clean":"2024-09-20T22:14:23.851561+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:14:23.851561+0000","last_undegraded":"2024-09-20T22:14:23.851561+0000","last_fullsized":"2024-09-20T22:14:23.851561+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:52.203809+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:52.203809+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:58.016851+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"328'15","reported_seq":177,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.849740+0000","last_change":"2024-09-20T22:12:43.218553+0000","last_active":"2024-09-20T22:14:23.849740+0000","last_peered":"2024-09-20T22:14:23.849740+0000","last_clean":"2024-09-20T22:14:23.849740+0000","last_became_active":"2024-09-20T22:11:30.493335+0000","last_became_peered":"2024-09-20T22:11:30.493335+0000","last_unstale":"2024-09-20T22:14:23.849740+0000","last_undegraded":"2024-09-20T22:14:23.849740+0000","last_fullsized":"2024-09-20T22:14:23.849740+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:12:43.218249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:43.218249+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":75,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.839009+0000","last_change":"2024-09-20T22:13:01.541472+0000","last_active":"2024-09-20T22:14:23.839009+0000","last_peered":"2024-09-20T22:14:23.839009+0000","last_clean":"2024-09-20T22:14:23.839009+0000","last_became_active":"2024-09-20T22:12:52.524149+0000","last_became_peered":"2024-09-20T22:12:52.524149+0000","last_unstale":"2024-09-20T22:14:23.839009+0000","last_undegraded":"2024-09-20T22:14:23.839009+0000","last_fullsized":"2024-09-20T22:14:23.839009+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:01.541330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:01.541330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"315'11","reported_seq":142,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.843540+0000","last_change":"2024-09-20T22:14:08.896302+0000","last_active":"2024-09-20T22:14:23.843540+0000","last_peered":"2024-09-20T22:14:23.843540+0000","last_clean":"2024-09-20T22:14:23.843540+0000","last_became_active":"2024-09-20T22:12:33.544098+0000","last_became_peered":"2024-09-20T22:12:33.544098+0000","last_unstale":"2024-09-20T22:14:23.843540+0000","last_undegraded":"2024-09-20T22:14:23.843540+0000","last_fullsized":"2024-09-20T22:14:23.843540+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:14:08.896199+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:08.896199+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:14.130177+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":623,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.836794+0000","last_change":"2024-09-20T22:13:48.148947+0000","last_active":"2024-09-20T22:14:23.836794+0000","last_peered":"2024-09-20T22:14:23.836794+0000","last_clean":"2024-09-20T22:14:23.836794+0000","last_became_active":"2024-09-20T22:11:30.484731+0000","last_became_peered":"2024-09-20T22:11:30.484731+0000","last_unstale":"2024-09-20T22:14:23.836794+0000","last_undegraded":"2024-09-20T22:14:23.836794+0000","last_fullsized":"2024-09-20T22:14:23.836794+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:48.148819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:48.148819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:05.881048+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"328'20","reported_seq":110,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:13.754289+0000","last_change":"2024-09-20T22:13:49.905238+0000","last_active":"2024-09-20T22:14:13.754289+0000","last_peered":"2024-09-20T22:14:13.754289+0000","last_clean":"2024-09-20T22:14:13.754289+0000","last_became_active":"2024-09-20T22:12:44.631986+0000","last_became_peered":"2024-09-20T22:12:44.631986+0000","last_unstale":"2024-09-20T22:14:13.754289+0000","last_undegraded":"2024-09-20T22:14:13.754289+0000","last_fullsized":"2024-09-20T22:14:13.754289+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:13:49.905115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:49.905115+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:14.205691+0000","scrub_duration":405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":623,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.843575+0000","last_change":"2024-09-20T22:14:23.041828+0000","last_active":"2024-09-20T22:14:23.843575+0000","last_peered":"2024-09-20T22:14:23.843575+0000","last_clean":"2024-09-20T22:14:23.843575+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:14:23.843575+0000","last_undegraded":"2024-09-20T22:14:23.843575+0000","last_fullsized":"2024-09-20T22:14:23.843575+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:23.041722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:23.041722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:33.056700+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"326'14","reported_seq":218,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.849541+0000","last_change":"2024-09-20T22:12:52.530641+0000","last_active":"2024-09-20T22:14:23.849541+0000","last_peered":"2024-09-20T22:14:23.849541+0000","last_clean":"2024-09-20T22:14:23.849541+0000","last_became_active":"2024-09-20T22:12:52.518770+0000","last_became_peered":"2024-09-20T22:12:52.518770+0000","last_unstale":"2024-09-20T22:14:23.849541+0000","last_undegraded":"2024-09-20T22:14:23.849541+0000","last_fullsized":"2024-09-20T22:14:23.849541+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'12","last_scrub_stamp":"2024-09-20T22:11:35.062247+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:11:35.062247+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":75,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.839611+0000","last_change":"2024-09-20T22:13:43.536313+0000","last_active":"2024-09-20T22:14:23.839611+0000","last_peered":"2024-09-20T22:14:23.839611+0000","last_clean":"2024-09-20T22:14:23.839611+0000","last_became_active":"2024-09-20T22:12:52.523781+0000","last_became_peered":"2024-09-20T22:12:52.523781+0000","last_unstale":"2024-09-20T22:14:23.839611+0000","last_undegraded":"2024-09-20T22:14:23.839611+0000","last_fullsized":"2024-09-20T22:14:23.839611+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:43.536210+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:43.536210+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:51.520380+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"328'19","reported_seq":671,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.836316+0000","last_change":"2024-09-20T22:12:57.881569+0000","last_active":"2024-09-20T22:14:23.836316+0000","last_peered":"2024-09-20T22:14:23.836316+0000","last_clean":"2024-09-20T22:14:23.836316+0000","last_became_active":"2024-09-20T22:08:22.139581+0000","last_became_peered":"2024-09-20T22:08:22.139581+0000","last_unstale":"2024-09-20T22:14:23.836316+0000","last_undegraded":"2024-09-20T22:14:23.836316+0000","last_fullsized":"2024-09-20T22:14:23.836316+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'11","last_scrub_stamp":"2024-09-20T22:12:09.181513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:09.181513+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":623,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.836558+0000","last_change":"2024-09-20T22:12:40.292205+0000","last_active":"2024-09-20T22:14:23.836558+0000","last_peered":"2024-09-20T22:14:23.836558+0000","last_clean":"2024-09-20T22:14:23.836558+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:14:23.836558+0000","last_undegraded":"2024-09-20T22:14:23.836558+0000","last_fullsized":"2024-09-20T22:14:23.836558+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:40.292095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:40.292095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":458,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:13.756099+0000","last_change":"2024-09-20T22:13:52.605374+0000","last_active":"2024-09-20T22:14:13.756099+0000","last_peered":"2024-09-20T22:14:13.756099+0000","last_clean":"2024-09-20T22:14:13.756099+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:14:13.756099+0000","last_undegraded":"2024-09-20T22:14:13.756099+0000","last_fullsized":"2024-09-20T22:14:13.756099+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:52.605277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:52.605277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:04.590848+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":469,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:18.440922+0000","last_change":"2024-09-20T22:14:18.440962+0000","last_active":"2024-09-20T22:14:18.440922+0000","last_peered":"2024-09-20T22:14:18.440922+0000","last_clean":"2024-09-20T22:14:18.440922+0000","last_became_active":"2024-09-20T22:12:52.521497+0000","last_became_peered":"2024-09-20T22:12:52.521497+0000","last_unstale":"2024-09-20T22:14:18.440922+0000","last_undegraded":"2024-09-20T22:14:18.440922+0000","last_fullsized":"2024-09-20T22:14:18.440922+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:18.438451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:18.438451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:19.376731+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"328'19","reported_seq":543,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:13.756442+0000","last_change":"2024-09-20T22:14:12.410224+0000","last_active":"2024-09-20T22:14:13.756442+0000","last_peered":"2024-09-20T22:14:13.756442+0000","last_clean":"2024-09-20T22:14:13.756442+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:14:13.756442+0000","last_undegraded":"2024-09-20T22:14:13.756442+0000","last_fullsized":"2024-09-20T22:14:13.756442+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:14:12.410107+0000","last_deep_scrub":"291'13","last_deep_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_clean_scrub_stamp":"2024-09-20T22:14:12.410107+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:29.679127+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":220,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.844946+0000","last_change":"2024-09-20T22:13:11.651173+0000","last_active":"2024-09-20T22:14:23.844946+0000","last_peered":"2024-09-20T22:14:23.844946+0000","last_clean":"2024-09-20T22:14:23.844946+0000","last_became_active":"2024-09-20T22:10:30.655502+0000","last_became_peered":"2024-09-20T22:10:30.655502+0000","last_unstale":"2024-09-20T22:14:23.844946+0000","last_undegraded":"2024-09-20T22:14:23.844946+0000","last_fullsized":"2024-09-20T22:14:23.844946+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:11.650985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:11.650985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:31.355370+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":75,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.839280+0000","last_change":"2024-09-20T22:14:09.476493+0000","last_active":"2024-09-20T22:14:23.839280+0000","last_peered":"2024-09-20T22:14:23.839280+0000","last_clean":"2024-09-20T22:14:23.839280+0000","last_became_active":"2024-09-20T22:12:52.516953+0000","last_became_peered":"2024-09-20T22:12:52.516953+0000","last_unstale":"2024-09-20T22:14:23.839280+0000","last_undegraded":"2024-09-20T22:14:23.839280+0000","last_fullsized":"2024-09-20T22:14:23.839280+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:09.476386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:09.476386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:29.061649+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":75,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.842768+0000","last_change":"2024-09-20T22:13:07.495192+0000","last_active":"2024-09-20T22:14:23.842768+0000","last_peered":"2024-09-20T22:14:23.842768+0000","last_clean":"2024-09-20T22:14:23.842768+0000","last_became_active":"2024-09-20T22:12:52.514247+0000","last_became_peered":"2024-09-20T22:12:52.514247+0000","last_unstale":"2024-09-20T22:14:23.842768+0000","last_undegraded":"2024-09-20T22:14:23.842768+0000","last_fullsized":"2024-09-20T22:14:23.842768+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:07.495088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:07.495088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:26.816441+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"328'19","reported_seq":548,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.842833+0000","last_change":"2024-09-20T22:13:38.718567+0000","last_active":"2024-09-20T22:14:23.842833+0000","last_peered":"2024-09-20T22:14:23.842833+0000","last_clean":"2024-09-20T22:14:23.842833+0000","last_became_active":"2024-09-20T22:12:33.545199+0000","last_became_peered":"2024-09-20T22:12:33.545199+0000","last_unstale":"2024-09-20T22:14:23.842833+0000","last_undegraded":"2024-09-20T22:14:23.842833+0000","last_fullsized":"2024-09-20T22:14:23.842833+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:13:38.718450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:38.718450+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:07.005555+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":161,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.860427+0000","last_change":"2024-09-20T22:12:05.215296+0000","last_active":"2024-09-20T22:14:23.860427+0000","last_peered":"2024-09-20T22:14:23.860427+0000","last_clean":"2024-09-20T22:14:23.860427+0000","last_became_active":"2024-09-20T22:11:30.493598+0000","last_became_peered":"2024-09-20T22:11:30.493598+0000","last_unstale":"2024-09-20T22:14:23.860427+0000","last_undegraded":"2024-09-20T22:14:23.860427+0000","last_fullsized":"2024-09-20T22:14:23.860427+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:05.215188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_clean_scrub_stamp":"2024-09-20T22:12:05.215188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":615,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.842309+0000","last_change":"2024-09-20T22:12:13.520411+0000","last_active":"2024-09-20T22:14:23.842309+0000","last_peered":"2024-09-20T22:14:23.842309+0000","last_clean":"2024-09-20T22:14:23.842309+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:14:23.842309+0000","last_undegraded":"2024-09-20T22:14:23.842309+0000","last_fullsized":"2024-09-20T22:14:23.842309+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:13.520285+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:13.520285+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":64,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.840389+0000","last_change":"2024-09-20T22:12:57.884265+0000","last_active":"2024-09-20T22:14:23.840389+0000","last_peered":"2024-09-20T22:14:23.840389+0000","last_clean":"2024-09-20T22:14:23.840389+0000","last_became_active":"2024-09-20T22:12:57.883998+0000","last_became_peered":"2024-09-20T22:12:57.883998+0000","last_unstale":"2024-09-20T22:14:23.840389+0000","last_undegraded":"2024-09-20T22:14:23.840389+0000","last_fullsized":"2024-09-20T22:14:23.840389+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:15.571001+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:15.571001+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"328'8","reported_seq":639,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.851998+0000","last_change":"2024-09-20T22:14:14.979857+0000","last_active":"2024-09-20T22:14:23.851998+0000","last_peered":"2024-09-20T22:14:23.851998+0000","last_clean":"2024-09-20T22:14:23.851998+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:14:23.851998+0000","last_undegraded":"2024-09-20T22:14:23.851998+0000","last_fullsized":"2024-09-20T22:14:23.851998+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'8","last_scrub_stamp":"2024-09-20T22:14:14.979748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:14.979748+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:35.167435+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":75,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.839650+0000","last_change":"2024-09-20T22:14:21.540990+0000","last_active":"2024-09-20T22:14:23.839650+0000","last_peered":"2024-09-20T22:14:23.839650+0000","last_clean":"2024-09-20T22:14:23.839650+0000","last_became_active":"2024-09-20T22:12:52.517510+0000","last_became_peered":"2024-09-20T22:12:52.517510+0000","last_unstale":"2024-09-20T22:14:23.839650+0000","last_undegraded":"2024-09-20T22:14:23.839650+0000","last_fullsized":"2024-09-20T22:14:23.839650+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:21.540878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:21.540878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:31.469685+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":640,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-20T22:14:27.275307+0000","last_change":"2024-09-20T22:13:14.985915+0000","last_active":"2024-09-20T22:14:27.275307+0000","last_peered":"2024-09-20T22:14:27.275307+0000","last_clean":"2024-09-20T22:14:27.275307+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:14:27.275307+0000","last_undegraded":"2024-09-20T22:14:27.275307+0000","last_fullsized":"2024-09-20T22:14:27.275307+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:14.985633+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:14.985633+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:41.156494+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":611,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.842824+0000","last_change":"2024-09-20T22:13:13.656975+0000","last_active":"2024-09-20T22:14:23.842824+0000","last_peered":"2024-09-20T22:14:23.842824+0000","last_clean":"2024-09-20T22:14:23.842824+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:14:23.842824+0000","last_undegraded":"2024-09-20T22:14:23.842824+0000","last_fullsized":"2024-09-20T22:14:23.842824+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:13.656906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_clean_scrub_stamp":"2024-09-20T22:13:13.656906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":507,"reported_epoch":351,"state":"active+clean","last_fresh":"2024-09-20T22:14:13.754979+0000","last_change":"2024-09-20T22:14:13.408076+0000","last_active":"2024-09-20T22:14:13.754979+0000","last_peered":"2024-09-20T22:14:13.754979+0000","last_clean":"2024-09-20T22:14:13.754979+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:14:13.754979+0000","last_undegraded":"2024-09-20T22:14:13.754979+0000","last_fullsized":"2024-09-20T22:14:13.754979+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:13.407980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:13.407980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:41.846519+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":75,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.843217+0000","last_change":"2024-09-20T22:13:11.547507+0000","last_active":"2024-09-20T22:14:23.843217+0000","last_peered":"2024-09-20T22:14:23.843217+0000","last_clean":"2024-09-20T22:14:23.843217+0000","last_became_active":"2024-09-20T22:12:52.518076+0000","last_became_peered":"2024-09-20T22:12:52.518076+0000","last_unstale":"2024-09-20T22:14:23.843217+0000","last_undegraded":"2024-09-20T22:14:23.843217+0000","last_fullsized":"2024-09-20T22:14:23.843217+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:11.547394+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:11.547394+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:14:26.312708+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"328'19","reported_seq":669,"reported_epoch":353,"state":"active+clean","last_fresh":"2024-09-20T22:14:27.275997+0000","last_change":"2024-09-20T22:14:15.443443+0000","last_active":"2024-09-20T22:14:27.275997+0000","last_peered":"2024-09-20T22:14:27.275997+0000","last_clean":"2024-09-20T22:14:27.275997+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:14:27.275997+0000","last_undegraded":"2024-09-20T22:14:27.275997+0000","last_fullsized":"2024-09-20T22:14:27.275997+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:14:15.443321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:15.443321+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:24.219738+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":75,"reported_epoch":352,"state":"active+clean","last_fresh":"2024-09-20T22:14:23.839527+0000","last_change":"2024-09-20T22:14:17.434098+0000","last_active":"2024-09-20T22:14:23.839527+0000","last_peered":"2024-09-20T22:14:23.839527+0000","last_clean":"2024-09-20T22:14:23.839527+0000","last_became_active":"2024-09-20T22:12:52.524425+0000","last_became_peered":"2024-09-20T22:12:52.524425+0000","last_unstale":"2024-09-20T22:14:23.839527+0000","last_undegraded":"2024-09-20T22:14:23.839527+0000","last_fullsized":"2024-09-20T22:14:23.839527+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:17.433977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:17.433977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:20.208087+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":16,"stat_sum":{"num_bytes":2346844184,"num_objects":35811,"num_object_clones":0,"num_object_copies":107433,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35811,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":185801,"num_write_kb":7091521,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":7536640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7054569472,"data_stored":7054557256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80781,"ondisk_log_size":80781,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91,"ondisk_log_size":91,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":3300,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":716,"ondisk_log_size":716,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":672,"num_read_kb":667,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":918048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1408,"ondisk_log_size":1408,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813136015,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1919544,"kb_used_data":1320348,"kb_used_omap":1,"kb_used_meta":598974,"kb_avail":91825608,"statfs":{"total":95995035648,"available":94029422592,"internally_reserved":0,"allocated":1352036352,"data_stored":1351685200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":613349834},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430353,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":796440,"kb_used_data":404296,"kb_used_omap":1,"kb_used_meta":392126,"kb_avail":92948712,"statfs":{"total":95995035648,"available":95179481088,"internally_reserved":0,"allocated":413999104,"data_stored":413691235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":401537478},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724695,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1935772,"kb_used_data":1322892,"kb_used_omap":1,"kb_used_meta":612862,"kb_avail":91809380,"statfs":{"total":95995035648,"available":94012805120,"internally_reserved":0,"allocated":1354641408,"data_stored":1354306445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":627571141},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986335,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":868652,"kb_used_data":432092,"kb_used_omap":1,"kb_used_meta":436542,"kb_avail":92876500,"statfs":{"total":95995035648,"available":95105536000,"internally_reserved":0,"allocated":442462208,"data_stored":442110865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":447019461},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770258011,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1859392,"kb_used_data":1422960,"kb_used_omap":4,"kb_used_meta":436411,"kb_avail":91885760,"statfs":{"total":95995035648,"available":94091018240,"internally_reserved":0,"allocated":1457111040,"data_stored":1456661969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":446885544},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149860,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1792256,"kb_used_data":1147440,"kb_used_omap":1,"kb_used_meta":644798,"kb_avail":91952896,"statfs":{"total":95995035648,"available":94159765504,"internally_reserved":0,"allocated":1174978560,"data_stored":1174608573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":660273607},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411497,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":909216,"kb_used_data":426628,"kb_used_omap":1,"kb_used_meta":482366,"kb_avail":92835936,"statfs":{"total":95995035648,"available":95063998464,"internally_reserved":0,"allocated":436867072,"data_stored":436540176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":493943237},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738548,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1150632,"kb_used_data":564120,"kb_used_omap":1,"kb_used_meta":586494,"kb_avail":92594520,"statfs":{"total":95995035648,"available":94816788480,"internally_reserved":0,"allocated":577658880,"data_stored":577311822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":600570309},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":568328192,"data_stored":568328192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1417940992,"data_stored":1417936920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":435552256,"data_stored":435552256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1150681088,"data_stored":1150681088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":432078848,"data_stored":432078848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1322455040,"data_stored":1322450968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":408879104,"data_stored":408879104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1318653952,"data_stored":1318649880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:14:29.700 INFO:tasks.thrashosds.thrasher:Setting 1.8 to [6, 7, 2] 2024-09-20T22:14:29.700 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '1.8', '6', '7', '2'] 2024-09-20T22:14:29.701 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pg-upmap 1.8 6 7 2 2024-09-20T22:14:29.714 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:29.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:29.714 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:29.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:29 smithi067 ceph-mon[33511]: pgmap v617: 158 pgs: 158 active+clean; 2.2 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s 2024-09-20T22:14:29.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:29 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1418087859' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:29.815 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:29.846 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:29.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:29.847 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:29.948 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:29.983 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:29.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:29.984 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:30.085 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:30.121 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:30.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:30.123 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:30.223 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:30.261 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:30.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:30.262 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:30.363 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:30.399 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:30.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:30.401 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:30.501 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:30.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:30.544 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:30.545 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:30.558 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:30 smithi060 ceph-mon[29694]: from='client.25537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:14:30.558 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:30 smithi060 ceph-mon[40290]: from='client.25537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:14:30.646 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:30.678 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:30.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:30.679 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:30 smithi067 ceph-mon[33511]: from='client.25537 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:14:30.780 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:30.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:30.821 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:30.821 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:30.922 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:30.967 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:30.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:30.969 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:31.069 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:31.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:31.119 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:31.120 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:31.220 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:31.258 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:31.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:31.259 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:31.359 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:31.397 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:31.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:31.398 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:31.421 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:31.448 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:14:31.448 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:14:31.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.448 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:192629 9.2 9:4303b3fb:::benchmark_data_smithi060_140132_object81821:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e353)", 2024-09-20T22:14:31.448 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:14:31.422280+0000", 2024-09-20T22:14:31.448 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.022315549, 2024-09-20T22:14:31.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.022346135999999999, 2024-09-20T22:14:31.449 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:14:31.449 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:14:31.449 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:14:31.449 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:14:31.449 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:14:31.449 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:14:31.449 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 192629 2024-09-20T22:14:31.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.449 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:14:31.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:14:31.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.422280+0000", 2024-09-20T22:14:31.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:14:31.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:14:31.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.422280+0000", 2024-09-20T22:14:31.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:14:31.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:14:31.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.422281+0000", 2024-09-20T22:14:31.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0900000000000002e-07 2024-09-20T22:14:31.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:14:31.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.422283+0000", 2024-09-20T22:14:31.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4480000000000001e-06 2024-09-20T22:14:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:14:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.422289+0000", 2024-09-20T22:14:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2500000000000003e-06 2024-09-20T22:14:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:14:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.422294+0000", 2024-09-20T22:14:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9869999999999999e-06 2024-09-20T22:14:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:14:31.451 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.437050+0000", 2024-09-20T22:14:31.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014755218000000001 2024-09-20T22:14:31.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:14:31.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.437182+0000", 2024-09-20T22:14:31.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013220800000000001 2024-09-20T22:14:31.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3", 2024-09-20T22:14:31.452 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.437309+0000", 2024-09-20T22:14:31.452 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000127606 2024-09-20T22:14:31.452 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.452 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.452 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:31.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.441516+0000", 2024-09-20T22:14:31.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0042066120000000002 2024-09-20T22:14:31.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:14:31.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.441871+0000", 2024-09-20T22:14:31.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00035470199999999998 2024-09-20T22:14:31.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.453 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.453 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:14:31.453 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.444412+0000", 2024-09-20T22:14:31.453 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025412920000000001 2024-09-20T22:14:31.453 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:31.453 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:31.453 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:31.453 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.454 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:192645 9.4 9:28f79739:::benchmark_data_smithi060_140132_object81837:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e353)", 2024-09-20T22:14:31.454 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:14:31.443758+0000", 2024-09-20T22:14:31.454 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00083802400000000004, 2024-09-20T22:14:31.454 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00097631499999999995, 2024-09-20T22:14:31.454 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:14:31.454 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:14:31.454 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:14:31.454 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:14:31.454 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:14:31.454 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:14:31.454 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 192645 2024-09-20T22:14:31.454 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.455 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:14:31.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:14:31.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.443758+0000", 2024-09-20T22:14:31.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:14:31.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:14:31.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.443758+0000", 2024-09-20T22:14:31.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:14:31.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:14:31.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.443760+0000", 2024-09-20T22:14:31.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8920000000000001e-06 2024-09-20T22:14:31.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:14:31.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.443768+0000", 2024-09-20T22:14:31.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9459999999999998e-06 2024-09-20T22:14:31.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:14:31.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.443788+0000", 2024-09-20T22:14:31.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0633999999999999e-05 2024-09-20T22:14:31.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:14:31.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.443797+0000", 2024-09-20T22:14:31.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0599999999999997e-06 2024-09-20T22:14:31.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:14:31.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.444111+0000", 2024-09-20T22:14:31.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00031314500000000002 2024-09-20T22:14:31.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:14:31.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.444228+0000", 2024-09-20T22:14:31.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117285 2024-09-20T22:14:31.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4", 2024-09-20T22:14:31.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.444354+0000", 2024-09-20T22:14:31.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012598500000000001 2024-09-20T22:14:31.457 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:31.457 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:31.458 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:31.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.458 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:192649 9.4 9.3363ba54 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e353)", 2024-09-20T22:14:31.458 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:14:31.443822+0000", 2024-09-20T22:14:31.458 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00077387300000000001, 2024-09-20T22:14:31.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00098473000000000007, 2024-09-20T22:14:31.458 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:14:31.458 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:14:31.458 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:14:31.458 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:14:31.458 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:14:31.458 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:14:31.458 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 192649 2024-09-20T22:14:31.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.458 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:14:31.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:14:31.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.443822+0000", 2024-09-20T22:14:31.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:14:31.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:14:31.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.443822+0000", 2024-09-20T22:14:31.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:14:31.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:14:31.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.443822+0000", 2024-09-20T22:14:31.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9500000000000003e-07 2024-09-20T22:14:31.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:14:31.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.443829+0000", 2024-09-20T22:14:31.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6320000000000002e-06 2024-09-20T22:14:31.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:14:31.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.443838+0000", 2024-09-20T22:14:31.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7790000000000001e-06 2024-09-20T22:14:31.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:14:31.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:14:31.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:14:31.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:14:31.443844+0000", 2024-09-20T22:14:31.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8839999999999999e-06 2024-09-20T22:14:31.460 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:31.460 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:14:31.460 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:31.460 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:14:31.461 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:14:31.461 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 3 2024-09-20T22:14:31.461 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:14:31.499 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:31.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:31.538 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:31.538 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:31.639 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:31.678 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:31.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:31.679 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:31.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:31 smithi067 ceph-mon[33511]: pgmap v618: 158 pgs: 158 active+clean; 2.1 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 916 op/s 2024-09-20T22:14:31.779 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:31.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:31 smithi060 ceph-mon[29694]: pgmap v618: 158 pgs: 158 active+clean; 2.1 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 916 op/s 2024-09-20T22:14:31.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:31 smithi060 ceph-mon[40290]: pgmap v618: 158 pgs: 158 active+clean; 2.1 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 916 op/s 2024-09-20T22:14:31.815 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:31.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:31.816 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:31.917 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:31.955 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:31.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:31.957 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:32.057 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:32.096 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:32.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:32.097 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:32.179 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:32.198 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:32.229 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T22:14:32.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:32.260 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:32.261 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:32.361 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:32.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:32.412 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:32.413 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:32.514 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:32.548 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:32.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:32.550 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:32.651 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:32.689 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:32.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:32.691 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:32.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:32 smithi067 ceph-mon[33511]: pgmap v619: 158 pgs: 158 active+clean; 1.8 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-20T22:14:32.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:32 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:14:32.764 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:14:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:14:32.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:32 smithi060 ceph-mon[29694]: pgmap v619: 158 pgs: 158 active+clean; 1.8 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-20T22:14:32.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:32 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:14:32.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:32 smithi060 ceph-mon[40290]: pgmap v619: 158 pgs: 158 active+clean; 1.8 GiB data, 9.7 GiB used, 706 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.26k op/s 2024-09-20T22:14:32.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:32 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:14:32.791 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:32.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:32.830 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:32.831 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:32.931 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:32.966 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:32.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:32.967 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:33.068 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:33.098 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:14:33.105 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:33.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:33.107 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:33.142 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:33.207 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:33.234 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:33.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:33.235 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:33.336 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:33.392 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:33.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:33.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:33.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:33.529 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:33.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:33.530 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:33.600 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:14:33.631 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:33.658 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:33.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:33.659 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:33.760 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:33.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:33 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2826862998' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:33.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:33 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2826862998' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:33.790 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:33.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:33.791 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:33.892 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:33.920 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:33.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:33.921 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:34.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:33 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2826862998' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:34.021 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:34.050 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:34.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:34.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:34.152 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:34.190 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:34.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:34.190 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:34.291 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:34.324 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:34.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:34.325 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:34.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:34.467 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:34.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:34.468 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:34.569 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:34.602 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:34.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:34.603 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:34.618 INFO:teuthology.orchestra.run.smithi060.stderr:set 1.8 pg_upmap mapping to [6,7,2] 2024-09-20T22:14:34.704 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:34.741 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:34.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:34.742 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:34.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:34 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:14:34.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:34 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2826862998' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:14:34.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:34 smithi060 ceph-mon[29694]: osdmap e354: 8 total, 8 up, 8 in 2024-09-20T22:14:34.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:34 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3963503759' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "1.8", "id": [6, 7, 2]}]: dispatch 2024-09-20T22:14:34.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:34 smithi060 ceph-mon[29694]: pgmap v621: 158 pgs: 158 active+clean; 1.7 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-20T22:14:34.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:34 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:14:34.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:34 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2826862998' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:14:34.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:34 smithi060 ceph-mon[40290]: osdmap e354: 8 total, 8 up, 8 in 2024-09-20T22:14:34.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:34 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3963503759' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "1.8", "id": [6, 7, 2]}]: dispatch 2024-09-20T22:14:34.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:34 smithi060 ceph-mon[40290]: pgmap v621: 158 pgs: 158 active+clean; 1.7 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-20T22:14:34.843 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:34.882 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:34.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:34.883 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:34.984 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:34 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:14:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:34 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2826862998' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:14:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:34 smithi067 ceph-mon[33511]: osdmap e354: 8 total, 8 up, 8 in 2024-09-20T22:14:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:34 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3963503759' entity='client.admin' cmd=[{"prefix": "osd pg-upmap", "pgid": "1.8", "id": [6, 7, 2]}]: dispatch 2024-09-20T22:14:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:34 smithi067 ceph-mon[33511]: pgmap v621: 158 pgs: 158 active+clean; 1.7 GiB data, 9.4 GiB used, 706 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.14k op/s 2024-09-20T22:14:35.024 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:35.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:35.025 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:35.126 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:35.162 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:35.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:35.163 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:35.264 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:35.295 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:35.296 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:35.296 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:35.303 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:14:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:14:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:14:35.397 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:35.438 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:35.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:35.439 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:35.540 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:35.576 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:35.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:35.577 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:35.677 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:35.728 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:35.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:35.729 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:35.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:35.870 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:35.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:35.871 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:35.972 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:36.008 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:36.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:36.009 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:35 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3963503759' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "1.8", "id": [6, 7, 2]}]': finished 2024-09-20T22:14:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:35 smithi067 ceph-mon[33511]: osdmap e355: 8 total, 8 up, 8 in 2024-09-20T22:14:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:35 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3963503759' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "1.8", "id": [6, 7, 2]}]': finished 2024-09-20T22:14:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:35 smithi060 ceph-mon[29694]: osdmap e355: 8 total, 8 up, 8 in 2024-09-20T22:14:36.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:35 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3963503759' entity='client.admin' cmd='[{"prefix": "osd pg-upmap", "pgid": "1.8", "id": [6, 7, 2]}]': finished 2024-09-20T22:14:36.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:35 smithi060 ceph-mon[40290]: osdmap e355: 8 total, 8 up, 8 in 2024-09-20T22:14:36.109 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:36.147 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:36.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:36.148 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:36.249 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:36.280 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:36.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:36.280 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:36.382 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:36.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:36.420 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:36.420 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:36.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:36.526 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:14:36.526 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:14:36.526 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:14:36.526 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:14:36.526 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:14:36.561 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:36.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:36.563 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:36.663 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:36.702 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:36.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:36.704 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:36.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:36 smithi060 ceph-mon[29694]: osdmap e356: 8 total, 8 up, 8 in 2024-09-20T22:14:36.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:36 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:14:36.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:36 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:14:36.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:36 smithi060 ceph-mon[29694]: pgmap v624: 158 pgs: 158 active+clean; 1.6 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.53k op/s 2024-09-20T22:14:36.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:36 smithi060 ceph-mon[40290]: osdmap e356: 8 total, 8 up, 8 in 2024-09-20T22:14:36.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:36 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:14:36.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:36 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:14:36.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:36 smithi060 ceph-mon[40290]: pgmap v624: 158 pgs: 158 active+clean; 1.6 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.53k op/s 2024-09-20T22:14:36.805 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:36.844 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:36.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:36.845 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:36.946 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:36.981 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:36.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:36.982 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:36 smithi067 ceph-mon[33511]: osdmap e356: 8 total, 8 up, 8 in 2024-09-20T22:14:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:36 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:14:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:36 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:14:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:36 smithi067 ceph-mon[33511]: pgmap v624: 158 pgs: 158 active+clean; 1.6 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.53k op/s 2024-09-20T22:14:37.084 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:37.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:37.122 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:37.122 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:37.223 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:37.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:37.262 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:37.262 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:37.363 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:37.403 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:37.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:37.404 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:37.505 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:37.552 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:37.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:37.553 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:37.653 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:37.694 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:37.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:37.695 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:37.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:37 smithi060 ceph-mon[29694]: osdmap e357: 8 total, 8 up, 8 in 2024-09-20T22:14:37.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:37 smithi060 ceph-mon[40290]: osdmap e357: 8 total, 8 up, 8 in 2024-09-20T22:14:37.796 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:37.831 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:37.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:37.832 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:37.897 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:37.898 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T22:14:37.933 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:37.960 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:37.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:37.961 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:37 smithi067 ceph-mon[33511]: osdmap e357: 8 total, 8 up, 8 in 2024-09-20T22:14:38.062 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:38.095 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:38.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:38.096 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:38.197 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:38.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:38.231 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:38.231 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:38.332 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:38.367 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:38.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:38.368 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:38.468 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:38.494 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:38.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:38.495 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:38.550 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:14:38.551 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:14:38.582 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:38.595 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:14:38.595 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:14:38.595 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:38.641 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:14:38.642 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:14:38.650 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:38.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:38.652 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:38.680 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:14:38.681 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:14:38.715 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:14:38.715 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:14:38.747 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:14:38.747 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:14:38.747 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:38 smithi060 ceph-mon[29694]: osdmap e358: 8 total, 8 up, 8 in 2024-09-20T22:14:38.747 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:38 smithi060 ceph-mon[29694]: pgmap v627: 158 pgs: 1 peering, 157 active+clean; 1.4 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.21k op/s 2024-09-20T22:14:38.748 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:38 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/648920805' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:38.748 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:38 smithi060 ceph-mon[40290]: osdmap e358: 8 total, 8 up, 8 in 2024-09-20T22:14:38.748 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:38 smithi060 ceph-mon[40290]: pgmap v627: 158 pgs: 1 peering, 157 active+clean; 1.4 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.21k op/s 2024-09-20T22:14:38.748 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:38 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/648920805' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:38.753 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:38.782 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:14:38.782 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:14:38.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:38.786 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:38.787 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:38.822 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:14:38.822 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:14:38.887 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:38.920 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:38.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:38.921 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:38 smithi067 ceph-mon[33511]: osdmap e358: 8 total, 8 up, 8 in 2024-09-20T22:14:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:38 smithi067 ceph-mon[33511]: pgmap v627: 158 pgs: 1 peering, 157 active+clean; 1.4 GiB data, 8.5 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.21k op/s 2024-09-20T22:14:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:38 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/648920805' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:39.022 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:39.058 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:39.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:39.059 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:39.159 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:39.165 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:14:39.199 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:39.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:39.200 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:39.300 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:39.328 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:39.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:39.329 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:39.430 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:39.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:39.463 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:39.463 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:39.564 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:39.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:39.605 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:39.605 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:39.705 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:39.744 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:39.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:39.745 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:39.846 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:39.884 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:39.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:39.885 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:39.986 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:39 smithi067 ceph-mon[33511]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:14:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:39 smithi067 ceph-mon[33511]: osdmap e359: 8 total, 8 up, 8 in 2024-09-20T22:14:40.025 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:40.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:40.026 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:40.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:39 smithi060 ceph-mon[29694]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:14:40.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:39 smithi060 ceph-mon[29694]: osdmap e359: 8 total, 8 up, 8 in 2024-09-20T22:14:40.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:39 smithi060 ceph-mon[40290]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:14:40.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:39 smithi060 ceph-mon[40290]: osdmap e359: 8 total, 8 up, 8 in 2024-09-20T22:14:40.126 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:40.150 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:40.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:40.151 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:40.251 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:40.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:40.293 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:40.294 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:40.394 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:40.426 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:40.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:40.427 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:40.528 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:40.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:40.557 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:40.557 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:40.658 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:40.673 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:40 smithi060 ceph-mon[29694]: pgmap v629: 158 pgs: 1 peering, 157 active+clean; 1.4 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-09-20T22:14:40.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:40.691 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:40.691 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:40.792 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:40.830 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:40.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:40.831 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:40.932 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:40.962 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:40.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:40.963 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:40 smithi067 ceph-mon[33511]: pgmap v629: 158 pgs: 1 peering, 157 active+clean; 1.4 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-09-20T22:14:41.035 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:40 smithi060 ceph-mon[40290]: pgmap v629: 158 pgs: 1 peering, 157 active+clean; 1.4 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-09-20T22:14:41.064 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:41.096 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:41.097 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:41.097 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:41.198 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:41.239 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:41.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:41.240 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:41.294 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:14:41.295 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 1 1 2024-09-20T22:14:41.340 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:41.398 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:41.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:41.399 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:41.500 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:41.554 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:41.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:41.555 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:41.656 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:41.695 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:41.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:41.696 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:41.718 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:14:41.718 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:14:41.796 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:41.834 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:41.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:41.835 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:41.936 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:41.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:41.969 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:41.969 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:42.070 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:42.101 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:42.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:42.102 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:42.203 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:42.225 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:42.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:42.225 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:42.326 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:42.357 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:42.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:42.358 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:42.459 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:42.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:42.501 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:42.501 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:42.602 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:42.641 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:42.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:42.643 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:42.744 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:42.763 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:14:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:14:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:42 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:14:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:42 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:14:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:42 smithi067 ceph-mon[33511]: pgmap v630: 158 pgs: 1 peering, 157 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-20T22:14:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:42 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:14:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:42 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:14:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:42 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:14:42.779 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:42.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:42.780 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:42.881 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:42.907 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:42 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:14:42.907 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:42 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:14:42.907 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:42 smithi060 ceph-mon[29694]: pgmap v630: 158 pgs: 1 peering, 157 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-20T22:14:42.907 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:42 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:14:42.907 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:42 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:14:42.907 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:42 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:14:42.908 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:42 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:14:42.908 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:42 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:14:42.909 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:42 smithi060 ceph-mon[40290]: pgmap v630: 158 pgs: 1 peering, 157 active+clean; 1.1 GiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-20T22:14:42.909 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:42 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:14:42.909 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:42 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:14:42.909 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:42 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:14:42.923 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:42.926 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:42.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:42.927 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:42.960 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T22:14:43.028 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:43.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:43.067 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:43.168 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:43.207 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:43.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:43.208 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:43.250 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:43.309 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:43.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:43.365 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:43.365 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:43.438 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:14:43.465 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:43.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:43.525 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:43.525 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:43.626 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:43.666 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:43.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:43.667 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:43.768 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:43.817 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:43.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:43.818 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:43.919 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:43.927 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:43 smithi060 ceph-mon[40290]: osdmap e360: 8 total, 8 up, 8 in 2024-09-20T22:14:43.927 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:43 smithi060 ceph-mon[29694]: osdmap e360: 8 total, 8 up, 8 in 2024-09-20T22:14:43.951 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:43.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:43.952 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:43 smithi067 ceph-mon[33511]: osdmap e360: 8 total, 8 up, 8 in 2024-09-20T22:14:44.053 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:44.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:44.088 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:44.088 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:44.189 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:44.227 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:44.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:44.227 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:44.328 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:44.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:44.366 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:44.367 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:44.467 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:44.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:44.506 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:44.506 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:44.587 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:14:44.594 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.1 to 1 (10000) 2024-09-20T22:14:44.608 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:44.681 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:44.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:44.682 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:44.782 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:44.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:44 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1010021618' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:14:44.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:44 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:14:44.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:44 smithi060 ceph-mon[29694]: pgmap v632: 158 pgs: 158 active+clean; 1.0 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 780 B/s rd, 0 B/s wr, 1.15k op/s 2024-09-20T22:14:44.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:44 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4022478705' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:14:44.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:44 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1010021618' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:14:44.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:44 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:14:44.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:44 smithi060 ceph-mon[40290]: pgmap v632: 158 pgs: 158 active+clean; 1.0 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 780 B/s rd, 0 B/s wr, 1.15k op/s 2024-09-20T22:14:44.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:44 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4022478705' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:14:44.818 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:44.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:44.819 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:44.920 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:44.956 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:44.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:44.957 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:44 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1010021618' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:14:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:44 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:14:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:44 smithi067 ceph-mon[33511]: pgmap v632: 158 pgs: 158 active+clean; 1.0 GiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 780 B/s rd, 0 B/s wr, 1.15k op/s 2024-09-20T22:14:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:44 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4022478705' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:14:45.058 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:45.098 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:45.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:45.099 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:45.122 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:14:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:14:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:14:45.200 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:45.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:45.232 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:45.232 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:45.333 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:45.370 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:45.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:45.371 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:45.471 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:45.501 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:45.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:45.502 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:45.603 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:45.645 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:45.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:45.647 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:45.748 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:45.783 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:45.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:45.784 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:45.818 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:45 smithi060 ceph-mon[29694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-20T22:14:45.818 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:45 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:14:45.818 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:45 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T22:14:45.818 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:45 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4022478705' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:14:45.818 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:45 smithi060 ceph-mon[29694]: osdmap e361: 8 total, 8 up, 8 in 2024-09-20T22:14:45.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:45 smithi060 ceph-mon[40290]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-20T22:14:45.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:45 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:14:45.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:45 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T22:14:45.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:45 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4022478705' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:14:45.819 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:45 smithi060 ceph-mon[40290]: osdmap e361: 8 total, 8 up, 8 in 2024-09-20T22:14:45.884 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:45.923 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:45.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:45.924 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:45 smithi067 ceph-mon[33511]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-20T22:14:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:45 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:14:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:45 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T22:14:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:45 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4022478705' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:14:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:45 smithi067 ceph-mon[33511]: osdmap e361: 8 total, 8 up, 8 in 2024-09-20T22:14:46.025 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:46.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:46.066 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:46.067 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:46.123 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:46.160 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 2 1 2024-09-20T22:14:46.167 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:46.184 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:46.196 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:46.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:46.198 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:46.298 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:46.331 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:46.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:46.332 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:46.433 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:46.468 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:46.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:46.469 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:46.569 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:46.612 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:46.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:46.613 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:46.713 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:46.727 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:46 smithi060 ceph-mon[29694]: 6.9 scrub starts 2024-09-20T22:14:46.727 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:46 smithi060 ceph-mon[29694]: 6.9 scrub ok 2024-09-20T22:14:46.728 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:46 smithi060 ceph-mon[29694]: pgmap v634: 158 pgs: 158 active+clean; 863 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-20T22:14:46.728 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:46 smithi060 ceph-mon[29694]: 1.8 scrub starts 2024-09-20T22:14:46.728 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:46 smithi060 ceph-mon[29694]: 1.8 scrub ok 2024-09-20T22:14:46.728 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:46 smithi060 ceph-mon[40290]: 6.9 scrub starts 2024-09-20T22:14:46.728 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:46 smithi060 ceph-mon[40290]: 6.9 scrub ok 2024-09-20T22:14:46.728 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:46 smithi060 ceph-mon[40290]: pgmap v634: 158 pgs: 158 active+clean; 863 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-20T22:14:46.728 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:46 smithi060 ceph-mon[40290]: 1.8 scrub starts 2024-09-20T22:14:46.728 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:46 smithi060 ceph-mon[40290]: 1.8 scrub ok 2024-09-20T22:14:46.754 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:46.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:46.756 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:46.856 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:46.902 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:46.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:46.903 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:47.004 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:46 smithi067 ceph-mon[33511]: 6.9 scrub starts 2024-09-20T22:14:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:46 smithi067 ceph-mon[33511]: 6.9 scrub ok 2024-09-20T22:14:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:46 smithi067 ceph-mon[33511]: pgmap v634: 158 pgs: 158 active+clean; 863 MiB data, 6.8 GiB used, 708 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.17k op/s 2024-09-20T22:14:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:46 smithi067 ceph-mon[33511]: 1.8 scrub starts 2024-09-20T22:14:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:46 smithi067 ceph-mon[33511]: 1.8 scrub ok 2024-09-20T22:14:47.040 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:14:47.040 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:14:47.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:47.041 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:47.041 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:47.142 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:47.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:47.176 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:47.176 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:47.277 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:47.319 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:47.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:47.320 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:47.421 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:47.450 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:47.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:47.451 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:47.470 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:14:47.470 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:14:47.470 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:14:47.470 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:14:47.470 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:14:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:14:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:14:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:14:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:14:47.470 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:14:47.493 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:14:47.493 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:14:47.526 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:14:47.526 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:14:47.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:47.562 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:14:47.562 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:14:47.599 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:14:47.599 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:47.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:47.604 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:47.704 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:47.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:47.767 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:47.767 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:47.868 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:47.910 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:47.911 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:47.911 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:48.012 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:48.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:47 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:14:48.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:47 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:14:48.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:47 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:14:48.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:47 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:14:48.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:47 smithi067 ceph-mon[33511]: 4.4 scrub starts 2024-09-20T22:14:48.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:47 smithi067 ceph-mon[33511]: 4.4 scrub ok 2024-09-20T22:14:48.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:47 smithi067 ceph-mon[33511]: osdmap e362: 8 total, 8 up, 8 in 2024-09-20T22:14:48.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:47 smithi067 ceph-mon[33511]: 6.13 scrub starts 2024-09-20T22:14:48.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:47 smithi067 ceph-mon[33511]: 6.13 scrub ok 2024-09-20T22:14:48.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:47 smithi067 ceph-mon[33511]: 4.d scrub starts 2024-09-20T22:14:48.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:47 smithi067 ceph-mon[33511]: 4.d scrub ok 2024-09-20T22:14:48.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:47 smithi067 ceph-mon[33511]: 4.f scrub starts 2024-09-20T22:14:48.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:47 smithi067 ceph-mon[33511]: 4.f scrub ok 2024-09-20T22:14:48.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:47 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2377256694' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:48.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:14:48.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:14:48.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:14:48.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:14:48.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[40290]: 4.4 scrub starts 2024-09-20T22:14:48.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[40290]: 4.4 scrub ok 2024-09-20T22:14:48.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[40290]: osdmap e362: 8 total, 8 up, 8 in 2024-09-20T22:14:48.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[40290]: 6.13 scrub starts 2024-09-20T22:14:48.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[40290]: 6.13 scrub ok 2024-09-20T22:14:48.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[40290]: 4.d scrub starts 2024-09-20T22:14:48.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[40290]: 4.d scrub ok 2024-09-20T22:14:48.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[40290]: 4.f scrub starts 2024-09-20T22:14:48.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[40290]: 4.f scrub ok 2024-09-20T22:14:48.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2377256694' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:48.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:14:48.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:14:48.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:14:48.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:14:48.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[29694]: 4.4 scrub starts 2024-09-20T22:14:48.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[29694]: 4.4 scrub ok 2024-09-20T22:14:48.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[29694]: osdmap e362: 8 total, 8 up, 8 in 2024-09-20T22:14:48.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[29694]: 6.13 scrub starts 2024-09-20T22:14:48.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[29694]: 6.13 scrub ok 2024-09-20T22:14:48.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[29694]: 4.d scrub starts 2024-09-20T22:14:48.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[29694]: 4.d scrub ok 2024-09-20T22:14:48.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[29694]: 4.f scrub starts 2024-09-20T22:14:48.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[29694]: 4.f scrub ok 2024-09-20T22:14:48.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:47 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2377256694' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:48.055 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:48.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:48.056 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:48.157 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:48.191 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:48.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:48.192 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:48.293 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:48.330 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:48.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:48.331 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:48.432 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:48.461 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:48.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:48.462 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:48.563 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:48.602 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:48.602 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:48.603 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:48.703 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:48.737 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:48.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:48.738 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:48.838 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:48.879 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:48.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:48.880 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:48.981 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:49.004 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:49.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:49.005 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:49.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:48 smithi067 ceph-mon[33511]: 6.8 scrub starts 2024-09-20T22:14:49.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:48 smithi067 ceph-mon[33511]: 6.8 scrub ok 2024-09-20T22:14:49.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:48 smithi067 ceph-mon[33511]: pgmap v636: 158 pgs: 158 active+clean; 755 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 986 op/s 2024-09-20T22:14:49.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:48 smithi060 ceph-mon[40290]: 6.8 scrub starts 2024-09-20T22:14:49.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:48 smithi060 ceph-mon[40290]: 6.8 scrub ok 2024-09-20T22:14:49.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:48 smithi060 ceph-mon[40290]: pgmap v636: 158 pgs: 158 active+clean; 755 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 986 op/s 2024-09-20T22:14:49.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:48 smithi060 ceph-mon[29694]: 6.8 scrub starts 2024-09-20T22:14:49.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:48 smithi060 ceph-mon[29694]: 6.8 scrub ok 2024-09-20T22:14:49.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:48 smithi060 ceph-mon[29694]: pgmap v636: 158 pgs: 158 active+clean; 755 MiB data, 6.4 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 986 op/s 2024-09-20T22:14:49.106 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:49.146 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:49.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:49.147 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:49.248 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:49.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:49.282 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:49.282 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:49.383 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:49.425 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:49.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:49.426 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:49.527 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:49.564 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:49.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:49.565 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:49.667 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:49.705 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:49.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:49.706 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:49.806 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:49.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:49.847 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:49.847 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:49.916 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:49 smithi060 ceph-mon[29694]: 3.10 scrub starts 2024-09-20T22:14:49.916 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:49 smithi060 ceph-mon[29694]: 3.10 scrub ok 2024-09-20T22:14:49.917 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:49 smithi060 ceph-mon[40290]: 3.10 scrub starts 2024-09-20T22:14:49.917 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:49 smithi060 ceph-mon[40290]: 3.10 scrub ok 2024-09-20T22:14:49.947 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:49.970 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T22:14:49.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:49.996 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:49.996 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:50.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:49 smithi067 ceph-mon[33511]: 3.10 scrub starts 2024-09-20T22:14:50.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:49 smithi067 ceph-mon[33511]: 3.10 scrub ok 2024-09-20T22:14:50.025 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:50.097 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:50.107 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:50.130 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:50.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:50.131 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:50.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:50.279 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:50.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:50.280 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:50.381 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:50.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:50.415 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:50.415 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:50.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:50.562 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:50.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:50.563 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:50.664 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:50.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:50.698 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:50.698 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:50.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:50 smithi060 ceph-mon[29694]: 3.0 scrub starts 2024-09-20T22:14:50.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:50 smithi060 ceph-mon[29694]: 3.0 scrub ok 2024-09-20T22:14:50.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:50 smithi060 ceph-mon[29694]: 6.f scrub starts 2024-09-20T22:14:50.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:50 smithi060 ceph-mon[29694]: 6.f scrub ok 2024-09-20T22:14:50.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:50 smithi060 ceph-mon[29694]: 5.e scrub starts 2024-09-20T22:14:50.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:50 smithi060 ceph-mon[29694]: 5.e scrub ok 2024-09-20T22:14:50.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:50 smithi060 ceph-mon[29694]: pgmap v637: 158 pgs: 158 active+clean; 687 MiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 670 B/s rd, 0 B/s wr, 917 op/s 2024-09-20T22:14:50.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:50 smithi060 ceph-mon[40290]: 3.0 scrub starts 2024-09-20T22:14:50.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:50 smithi060 ceph-mon[40290]: 3.0 scrub ok 2024-09-20T22:14:50.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:50 smithi060 ceph-mon[40290]: 6.f scrub starts 2024-09-20T22:14:50.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:50 smithi060 ceph-mon[40290]: 6.f scrub ok 2024-09-20T22:14:50.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:50 smithi060 ceph-mon[40290]: 5.e scrub starts 2024-09-20T22:14:50.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:50 smithi060 ceph-mon[40290]: 5.e scrub ok 2024-09-20T22:14:50.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:50 smithi060 ceph-mon[40290]: pgmap v637: 158 pgs: 158 active+clean; 687 MiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 670 B/s rd, 0 B/s wr, 917 op/s 2024-09-20T22:14:50.799 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:50.834 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:50.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:50.835 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:50.935 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:50.975 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:50.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:50.977 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:50 smithi067 ceph-mon[33511]: 3.0 scrub starts 2024-09-20T22:14:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:50 smithi067 ceph-mon[33511]: 3.0 scrub ok 2024-09-20T22:14:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:50 smithi067 ceph-mon[33511]: 6.f scrub starts 2024-09-20T22:14:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:50 smithi067 ceph-mon[33511]: 6.f scrub ok 2024-09-20T22:14:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:50 smithi067 ceph-mon[33511]: 5.e scrub starts 2024-09-20T22:14:51.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:50 smithi067 ceph-mon[33511]: 5.e scrub ok 2024-09-20T22:14:51.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:50 smithi067 ceph-mon[33511]: pgmap v637: 158 pgs: 158 active+clean; 687 MiB data, 6.2 GiB used, 709 GiB / 715 GiB avail; 670 B/s rd, 0 B/s wr, 917 op/s 2024-09-20T22:14:51.077 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:51.109 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:51.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:51.110 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:51.192 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:51.211 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:51.242 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:51.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:51.243 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:51.344 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:51.389 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:51.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:51.391 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:51.491 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:51.535 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:51.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:51.536 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:51.636 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:51.645 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:14:51.646 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.2 to 1 (10000) 2024-09-20T22:14:51.648 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:14:51.672 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:51.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:51.673 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:51.773 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:14:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:51 smithi060 ceph-mon[29694]: 4.1c scrub starts 2024-09-20T22:14:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:51 smithi060 ceph-mon[29694]: 4.1c scrub ok 2024-09-20T22:14:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:51 smithi060 ceph-mon[29694]: 6.e scrub starts 2024-09-20T22:14:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:51 smithi060 ceph-mon[29694]: 6.e scrub ok 2024-09-20T22:14:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:51 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1767943896' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:14:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:51 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:14:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:51 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1522100160' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:51.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:51 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:51.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:14:51.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:51 smithi060 ceph-mon[40290]: 4.1c scrub starts 2024-09-20T22:14:51.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:51 smithi060 ceph-mon[40290]: 4.1c scrub ok 2024-09-20T22:14:51.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:51 smithi060 ceph-mon[40290]: 6.e scrub starts 2024-09-20T22:14:51.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:51 smithi060 ceph-mon[40290]: 6.e scrub ok 2024-09-20T22:14:51.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:51 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1767943896' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:14:51.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:51 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:14:51.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:51 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1522100160' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:51.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:51 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:51.815 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:51.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:51.816 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:51.916 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:51.956 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:51.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:51.957 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:14:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:51 smithi067 ceph-mon[33511]: 4.1c scrub starts 2024-09-20T22:14:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:51 smithi067 ceph-mon[33511]: 4.1c scrub ok 2024-09-20T22:14:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:51 smithi067 ceph-mon[33511]: 6.e scrub starts 2024-09-20T22:14:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:51 smithi067 ceph-mon[33511]: 6.e scrub ok 2024-09-20T22:14:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:51 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1767943896' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:14:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:51 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:14:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:51 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1522100160' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:52.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:51 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:14:52.058 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:52.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:52.110 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:52.110 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:52.211 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:52.244 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:52.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:52.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:52.345 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:52.377 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:52.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:52.378 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:52.478 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:52.509 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:52.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:52.510 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:52.604 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:14:52.604 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:14:52.604 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:14:52.604 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:14:52.604 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:14:52.611 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:52.648 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:52.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:52.649 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:52.750 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:52.763 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:14:52 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:14:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:52 smithi067 ceph-mon[33511]: 9.6 scrub starts 2024-09-20T22:14:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:52 smithi067 ceph-mon[33511]: 9.6 scrub ok 2024-09-20T22:14:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:52 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:14:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:52 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:14:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:52 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:14:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:52 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:14:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:52 smithi067 ceph-mon[33511]: osdmap e363: 8 total, 8 up, 8 in 2024-09-20T22:14:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:52 smithi067 ceph-mon[33511]: 5.b scrub starts 2024-09-20T22:14:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:52 smithi067 ceph-mon[33511]: 5.b scrub ok 2024-09-20T22:14:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:52 smithi067 ceph-mon[33511]: pgmap v639: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 401 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-20T22:14:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:52 smithi067 ceph-mon[33511]: 3.1e scrub starts 2024-09-20T22:14:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:52 smithi067 ceph-mon[33511]: 3.1e scrub ok 2024-09-20T22:14:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:52 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:14:52.783 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:52.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:52.784 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[29694]: 9.6 scrub starts 2024-09-20T22:14:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[29694]: 9.6 scrub ok 2024-09-20T22:14:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:14:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:14:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:14:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:14:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[29694]: osdmap e363: 8 total, 8 up, 8 in 2024-09-20T22:14:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[29694]: 5.b scrub starts 2024-09-20T22:14:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[29694]: 5.b scrub ok 2024-09-20T22:14:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[29694]: pgmap v639: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 401 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-20T22:14:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[29694]: 3.1e scrub starts 2024-09-20T22:14:52.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[29694]: 3.1e scrub ok 2024-09-20T22:14:52.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:14:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[40290]: 9.6 scrub starts 2024-09-20T22:14:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[40290]: 9.6 scrub ok 2024-09-20T22:14:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:14:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:14:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:14:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:14:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[40290]: osdmap e363: 8 total, 8 up, 8 in 2024-09-20T22:14:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[40290]: 5.b scrub starts 2024-09-20T22:14:52.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[40290]: 5.b scrub ok 2024-09-20T22:14:52.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[40290]: pgmap v639: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 401 MiB data, 5.3 GiB used, 710 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.33k op/s 2024-09-20T22:14:52.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[40290]: 3.1e scrub starts 2024-09-20T22:14:52.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[40290]: 3.1e scrub ok 2024-09-20T22:14:52.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:52 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:14:52.885 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:52.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:52.922 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:52.922 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:53.023 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:53.054 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:53.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:53.055 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:53.156 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:53.198 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:53.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:53.200 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:53.300 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:53.337 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:53.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:53.338 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:53.439 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:53.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:53.476 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:53.476 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:53.577 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:53.610 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:53.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:53.611 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:53.712 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:53.747 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:53.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:53.747 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:53.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:53 smithi060 ceph-mon[29694]: 5.3 deep-scrub starts 2024-09-20T22:14:53.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:53 smithi060 ceph-mon[29694]: 5.3 deep-scrub ok 2024-09-20T22:14:53.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:53 smithi060 ceph-mon[40290]: 5.3 deep-scrub starts 2024-09-20T22:14:53.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:53 smithi060 ceph-mon[40290]: 5.3 deep-scrub ok 2024-09-20T22:14:53.848 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:53.888 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:53.888 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:53.888 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:53.935 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:53 smithi067 ceph-mon[33511]: 5.3 deep-scrub starts 2024-09-20T22:14:53.935 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:53 smithi067 ceph-mon[33511]: 5.3 deep-scrub ok 2024-09-20T22:14:53.989 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:54.024 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:54.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:54.025 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:54.126 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:54.170 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:54.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:54.171 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:54.205 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 3 1 2024-09-20T22:14:54.272 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:54.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:54.334 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:54.334 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:54.435 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:54.470 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:54.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:54.471 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:54.572 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:54.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:54.607 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:54.607 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:54.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T22:14:54.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:54.722 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:54.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:54.800 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:54.800 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:54.899 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:54 smithi060 ceph-mon[29694]: pgmap v640: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 305 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.11k op/s 2024-09-20T22:14:54.899 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:54 smithi060 ceph-mon[40290]: pgmap v640: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 305 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.11k op/s 2024-09-20T22:14:54.901 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:54.951 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:54.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:54.952 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:55.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:54 smithi067 ceph-mon[33511]: pgmap v640: 158 pgs: 1 active+clean+scrubbing, 157 active+clean; 305 MiB data, 5.1 GiB used, 710 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.11k op/s 2024-09-20T22:14:55.014 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:55.053 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:55.104 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:55.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:55.105 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:55.163 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:14:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:14:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:14:55.206 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:55.238 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:55.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:55.240 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:55.340 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:55.396 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:55.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:55.397 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:55.497 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:55.531 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:55.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:55.532 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:55.540 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:14:55.632 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:55.661 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:55.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:55.662 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:55.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:55 smithi060 ceph-mon[29694]: 5.1b scrub starts 2024-09-20T22:14:55.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:55 smithi060 ceph-mon[29694]: 5.1b scrub ok 2024-09-20T22:14:55.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:55 smithi060 ceph-mon[29694]: 6.15 deep-scrub starts 2024-09-20T22:14:55.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:55 smithi060 ceph-mon[29694]: 6.15 deep-scrub ok 2024-09-20T22:14:55.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:55 smithi060 ceph-mon[29694]: 4.1 scrub starts 2024-09-20T22:14:55.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:55 smithi060 ceph-mon[29694]: 4.1 scrub ok 2024-09-20T22:14:55.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:55 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3017187334' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:55.710 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:55 smithi060 ceph-mon[40290]: 5.1b scrub starts 2024-09-20T22:14:55.710 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:55 smithi060 ceph-mon[40290]: 5.1b scrub ok 2024-09-20T22:14:55.710 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:55 smithi060 ceph-mon[40290]: 6.15 deep-scrub starts 2024-09-20T22:14:55.710 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:55 smithi060 ceph-mon[40290]: 6.15 deep-scrub ok 2024-09-20T22:14:55.710 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:55 smithi060 ceph-mon[40290]: 4.1 scrub starts 2024-09-20T22:14:55.711 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:55 smithi060 ceph-mon[40290]: 4.1 scrub ok 2024-09-20T22:14:55.711 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:55 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3017187334' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:55.762 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:55.785 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:14:55.785 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:14:55.785 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:14:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:14:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:14:55.786 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:14:55.791 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:55.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:55.791 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:55.802 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:14:55.803 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:14:55.806 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Removed 110805 objects 2024-09-20T22:14:55.806 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Clean up completed and total clean up time :97.8875 2024-09-20T22:14:55.824 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:14:55.824 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:14:55.842 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:55.843 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:14:55.843 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:14:55.868 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:14:55.868 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:14:55.891 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:14:55.891 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:14:55.892 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:55.919 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:55.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:55.920 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:55 smithi067 ceph-mon[33511]: 5.1b scrub starts 2024-09-20T22:14:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:55 smithi067 ceph-mon[33511]: 5.1b scrub ok 2024-09-20T22:14:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:55 smithi067 ceph-mon[33511]: 6.15 deep-scrub starts 2024-09-20T22:14:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:55 smithi067 ceph-mon[33511]: 6.15 deep-scrub ok 2024-09-20T22:14:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:55 smithi067 ceph-mon[33511]: 4.1 scrub starts 2024-09-20T22:14:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:55 smithi067 ceph-mon[33511]: 4.1 scrub ok 2024-09-20T22:14:56.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3017187334' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:14:56.021 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:56.042 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:56.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:56.043 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:56.144 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:56.167 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:56.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:56.167 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:56.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:56.290 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:56.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:56.291 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:56.392 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:56.414 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:56.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:56.415 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:56.447 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:14:56.448 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:14:56.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:56.541 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:56.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:56.542 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:56.642 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:56.665 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:56.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:56.666 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:56.679 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.3 to 1 (10000) 2024-09-20T22:14:56.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:56.789 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:56.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:56.790 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:56.890 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:56.914 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:56.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:56.915 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:57.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:56 smithi067 ceph-mon[33511]: 3.c scrub starts 2024-09-20T22:14:57.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:56 smithi067 ceph-mon[33511]: 3.c scrub ok 2024-09-20T22:14:57.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:56 smithi067 ceph-mon[33511]: 5.10 scrub starts 2024-09-20T22:14:57.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:56 smithi067 ceph-mon[33511]: 5.10 scrub ok 2024-09-20T22:14:57.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:56 smithi067 ceph-mon[33511]: 6.0 scrub starts 2024-09-20T22:14:57.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:56 smithi067 ceph-mon[33511]: 6.0 scrub ok 2024-09-20T22:14:57.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:56 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1866579531' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:14:57.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:56 smithi067 ceph-mon[33511]: pgmap v641: 158 pgs: 158 active+clean; 125 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-20T22:14:57.016 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:57.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:56 smithi060 ceph-mon[29694]: 3.c scrub starts 2024-09-20T22:14:57.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:56 smithi060 ceph-mon[29694]: 3.c scrub ok 2024-09-20T22:14:57.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:56 smithi060 ceph-mon[29694]: 5.10 scrub starts 2024-09-20T22:14:57.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:56 smithi060 ceph-mon[29694]: 5.10 scrub ok 2024-09-20T22:14:57.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:56 smithi060 ceph-mon[29694]: 6.0 scrub starts 2024-09-20T22:14:57.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:56 smithi060 ceph-mon[29694]: 6.0 scrub ok 2024-09-20T22:14:57.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:56 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1866579531' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:14:57.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:56 smithi060 ceph-mon[29694]: pgmap v641: 158 pgs: 158 active+clean; 125 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-20T22:14:57.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:56 smithi060 ceph-mon[40290]: 3.c scrub starts 2024-09-20T22:14:57.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:56 smithi060 ceph-mon[40290]: 3.c scrub ok 2024-09-20T22:14:57.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:56 smithi060 ceph-mon[40290]: 5.10 scrub starts 2024-09-20T22:14:57.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:56 smithi060 ceph-mon[40290]: 5.10 scrub ok 2024-09-20T22:14:57.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:56 smithi060 ceph-mon[40290]: 6.0 scrub starts 2024-09-20T22:14:57.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:56 smithi060 ceph-mon[40290]: 6.0 scrub ok 2024-09-20T22:14:57.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:56 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1866579531' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:14:57.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:56 smithi060 ceph-mon[40290]: pgmap v641: 158 pgs: 158 active+clean; 125 MiB data, 4.6 GiB used, 711 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.23k op/s 2024-09-20T22:14:57.038 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:57.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:57.038 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:57.052 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_2 2024-09-20T22:14:57.052 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool rm unique_pool_2 unique_pool_2 --yes-i-really-really-mean-it 2024-09-20T22:14:57.139 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:57.162 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:57.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:57.163 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:57.264 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:57.286 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:57.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:57.287 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:57.387 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:57.410 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:57.411 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:57.411 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:57.512 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:57.534 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:57.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:57.535 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:57.636 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:14:57.659 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:57.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:57.660 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:14:57.761 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:57.784 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:57.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:57.785 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:57.885 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:57.909 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:57.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:57.910 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:58.010 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:57 smithi067 ceph-mon[33511]: 6.1b scrub starts 2024-09-20T22:14:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:57 smithi067 ceph-mon[33511]: 6.1b scrub ok 2024-09-20T22:14:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:57 smithi067 ceph-mon[33511]: 3.f scrub starts 2024-09-20T22:14:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:57 smithi067 ceph-mon[33511]: 3.f scrub ok 2024-09-20T22:14:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:57 smithi067 ceph-mon[33511]: 3.12 deep-scrub starts 2024-09-20T22:14:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:57 smithi067 ceph-mon[33511]: 3.12 deep-scrub ok 2024-09-20T22:14:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:57 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1866579531' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:14:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:57 smithi067 ceph-mon[33511]: osdmap e364: 8 total, 8 up, 8 in 2024-09-20T22:14:58.034 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:58.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:58.035 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:58.035 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:57 smithi060 ceph-mon[40290]: 6.1b scrub starts 2024-09-20T22:14:58.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:57 smithi060 ceph-mon[40290]: 6.1b scrub ok 2024-09-20T22:14:58.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:57 smithi060 ceph-mon[40290]: 3.f scrub starts 2024-09-20T22:14:58.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:57 smithi060 ceph-mon[40290]: 3.f scrub ok 2024-09-20T22:14:58.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:57 smithi060 ceph-mon[40290]: 3.12 deep-scrub starts 2024-09-20T22:14:58.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:57 smithi060 ceph-mon[40290]: 3.12 deep-scrub ok 2024-09-20T22:14:58.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:57 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1866579531' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:14:58.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:57 smithi060 ceph-mon[40290]: osdmap e364: 8 total, 8 up, 8 in 2024-09-20T22:14:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:57 smithi060 ceph-mon[29694]: 6.1b scrub starts 2024-09-20T22:14:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:57 smithi060 ceph-mon[29694]: 6.1b scrub ok 2024-09-20T22:14:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:57 smithi060 ceph-mon[29694]: 3.f scrub starts 2024-09-20T22:14:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:57 smithi060 ceph-mon[29694]: 3.f scrub ok 2024-09-20T22:14:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:57 smithi060 ceph-mon[29694]: 3.12 deep-scrub starts 2024-09-20T22:14:58.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:57 smithi060 ceph-mon[29694]: 3.12 deep-scrub ok 2024-09-20T22:14:58.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:57 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1866579531' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:14:58.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:57 smithi060 ceph-mon[29694]: osdmap e364: 8 total, 8 up, 8 in 2024-09-20T22:14:58.135 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:58.159 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:58.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:58.160 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:58.261 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:58.285 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:58.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:58.286 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:58.364 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T22:14:58.386 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:58.410 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:58.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:58.411 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:58.511 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:58.535 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:58.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:58.536 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:58.636 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:58.661 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:58.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:58.662 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:58.763 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:14:58.787 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:58.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:58.788 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:14:58.832 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:14:58.889 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:58.913 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:58.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:58.914 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:58.955 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:58 smithi060 ceph-mon[29694]: 5.1a scrub starts 2024-09-20T22:14:58.955 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:58 smithi060 ceph-mon[29694]: 5.1a scrub ok 2024-09-20T22:14:58.955 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:58 smithi060 ceph-mon[29694]: 3.1f scrub starts 2024-09-20T22:14:58.955 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:58 smithi060 ceph-mon[29694]: 3.1f scrub ok 2024-09-20T22:14:58.955 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:58 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:14:58.955 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:58 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:14:58.955 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:58 smithi060 ceph-mon[29694]: 3.3 scrub starts 2024-09-20T22:14:58.955 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:58 smithi060 ceph-mon[29694]: 3.3 scrub ok 2024-09-20T22:14:58.955 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:58 smithi060 ceph-mon[29694]: pgmap v643: 158 pgs: 158 active+clean; 34 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-09-20T22:14:58.956 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:58 smithi060 ceph-mon[40290]: 5.1a scrub starts 2024-09-20T22:14:58.956 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:58 smithi060 ceph-mon[40290]: 5.1a scrub ok 2024-09-20T22:14:58.956 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:58 smithi060 ceph-mon[40290]: 3.1f scrub starts 2024-09-20T22:14:58.956 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:58 smithi060 ceph-mon[40290]: 3.1f scrub ok 2024-09-20T22:14:58.956 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:58 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:14:58.956 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:58 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:14:58.956 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:58 smithi060 ceph-mon[40290]: 3.3 scrub starts 2024-09-20T22:14:58.956 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:58 smithi060 ceph-mon[40290]: 3.3 scrub ok 2024-09-20T22:14:58.956 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:58 smithi060 ceph-mon[40290]: pgmap v643: 158 pgs: 158 active+clean; 34 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-09-20T22:14:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:58 smithi067 ceph-mon[33511]: 5.1a scrub starts 2024-09-20T22:14:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:58 smithi067 ceph-mon[33511]: 5.1a scrub ok 2024-09-20T22:14:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:58 smithi067 ceph-mon[33511]: 3.1f scrub starts 2024-09-20T22:14:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:58 smithi067 ceph-mon[33511]: 3.1f scrub ok 2024-09-20T22:14:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:58 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:14:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:58 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:14:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:58 smithi067 ceph-mon[33511]: 3.3 scrub starts 2024-09-20T22:14:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:58 smithi067 ceph-mon[33511]: 3.3 scrub ok 2024-09-20T22:14:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:58 smithi067 ceph-mon[33511]: pgmap v643: 158 pgs: 158 active+clean; 34 MiB data, 4.3 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.31k op/s 2024-09-20T22:14:59.015 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:14:59.038 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:59.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:59.039 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:14:59.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:14:59.165 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:59.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:59.166 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:14:59.252 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 4 1 2024-09-20T22:14:59.266 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:59.319 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:59.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:59.320 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:59.421 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:14:59.448 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:59.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:59.449 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:14:59.549 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:59.572 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:59.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:59.573 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:14:59.673 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:14:59.700 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:14:59.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:59.701 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:14:59.802 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:14:59.824 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:59.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:59.825 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:14:59.925 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:14:59.935 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:59 smithi060 ceph-mon[29694]: 4.17 scrub starts 2024-09-20T22:14:59.935 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:59 smithi060 ceph-mon[29694]: 4.17 scrub ok 2024-09-20T22:14:59.935 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:59 smithi060 ceph-mon[29694]: 6.18 deep-scrub starts 2024-09-20T22:14:59.935 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:59 smithi060 ceph-mon[29694]: 6.18 deep-scrub ok 2024-09-20T22:14:59.936 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:59 smithi060 ceph-mon[29694]: osdmap e365: 8 total, 8 up, 8 in 2024-09-20T22:14:59.936 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:59 smithi060 ceph-mon[29694]: 4.13 scrub starts 2024-09-20T22:14:59.936 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:14:59 smithi060 ceph-mon[29694]: 4.13 scrub ok 2024-09-20T22:14:59.936 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:59 smithi060 ceph-mon[40290]: 4.17 scrub starts 2024-09-20T22:14:59.936 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:59 smithi060 ceph-mon[40290]: 4.17 scrub ok 2024-09-20T22:14:59.936 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:59 smithi060 ceph-mon[40290]: 6.18 deep-scrub starts 2024-09-20T22:14:59.936 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:59 smithi060 ceph-mon[40290]: 6.18 deep-scrub ok 2024-09-20T22:14:59.936 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:59 smithi060 ceph-mon[40290]: osdmap e365: 8 total, 8 up, 8 in 2024-09-20T22:14:59.936 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:59 smithi060 ceph-mon[40290]: 4.13 scrub starts 2024-09-20T22:14:59.936 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:14:59 smithi060 ceph-mon[40290]: 4.13 scrub ok 2024-09-20T22:14:59.948 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:14:59.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:14:59.949 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:00.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:59 smithi067 ceph-mon[33511]: 4.17 scrub starts 2024-09-20T22:15:00.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:59 smithi067 ceph-mon[33511]: 4.17 scrub ok 2024-09-20T22:15:00.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:59 smithi067 ceph-mon[33511]: 6.18 deep-scrub starts 2024-09-20T22:15:00.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:59 smithi067 ceph-mon[33511]: 6.18 deep-scrub ok 2024-09-20T22:15:00.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:59 smithi067 ceph-mon[33511]: osdmap e365: 8 total, 8 up, 8 in 2024-09-20T22:15:00.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:59 smithi067 ceph-mon[33511]: 4.13 scrub starts 2024-09-20T22:15:00.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:14:59 smithi067 ceph-mon[33511]: 4.13 scrub ok 2024-09-20T22:15:00.049 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:00.076 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:00.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:00.077 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:00.178 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:00.200 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:00.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:00.201 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:00.302 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:00.320 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:00.320 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:00.329 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:00.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:00.330 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:00.357 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:00.430 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:00.452 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:00.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:00.453 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:00.554 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:00.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:00.577 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:00.577 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:00.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:00.697 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:15:00.705 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:00.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:00.706 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:00.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:00 smithi060 ceph-mon[29694]: 6.14 deep-scrub starts 2024-09-20T22:15:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:00 smithi060 ceph-mon[29694]: 6.14 deep-scrub ok 2024-09-20T22:15:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:00 smithi060 ceph-mon[29694]: pgmap v645: 158 pgs: 158 active+clean; 711 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 800 op/s 2024-09-20T22:15:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:00 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1453479688' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:15:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:00 smithi060 ceph-mon[40290]: 6.14 deep-scrub starts 2024-09-20T22:15:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:00 smithi060 ceph-mon[40290]: 6.14 deep-scrub ok 2024-09-20T22:15:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:00 smithi060 ceph-mon[40290]: pgmap v645: 158 pgs: 158 active+clean; 711 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 800 op/s 2024-09-20T22:15:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:00 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1453479688' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:15:00.807 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:00.834 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:00.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:00.835 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:00.936 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:00.963 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:00.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:00.964 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:00 smithi067 ceph-mon[33511]: 6.14 deep-scrub starts 2024-09-20T22:15:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:00 smithi067 ceph-mon[33511]: 6.14 deep-scrub ok 2024-09-20T22:15:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:00 smithi067 ceph-mon[33511]: pgmap v645: 158 pgs: 158 active+clean; 711 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 800 op/s 2024-09-20T22:15:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:00 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1453479688' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:15:01.065 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:01.086 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:01.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:01.087 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:01.188 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:01.210 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:01.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:01.211 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:01.311 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:01.339 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:01.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:01.340 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:01.440 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:01.466 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:01.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:01.467 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:01.567 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:01.592 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:01.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:01.593 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:01.694 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:01.699 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:15:01.699 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.4 to 1 (10000) 2024-09-20T22:15:01.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:01.720 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:01.720 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:01.821 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:01.843 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:01.843 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:01.844 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:01.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:01 smithi060 ceph-mon[29694]: 6.19 scrub starts 2024-09-20T22:15:01.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:01 smithi060 ceph-mon[29694]: 6.19 scrub ok 2024-09-20T22:15:01.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:01 smithi060 ceph-mon[29694]: 1.2 scrub starts 2024-09-20T22:15:01.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:01 smithi060 ceph-mon[29694]: 1.2 scrub ok 2024-09-20T22:15:01.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:01 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:15:01.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:01 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1453479688' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:15:01.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:01 smithi060 ceph-mon[29694]: osdmap e366: 8 total, 8 up, 8 in 2024-09-20T22:15:01.875 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:01 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4034203148' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:15:01.876 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:01 smithi060 ceph-mon[40290]: 6.19 scrub starts 2024-09-20T22:15:01.876 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:01 smithi060 ceph-mon[40290]: 6.19 scrub ok 2024-09-20T22:15:01.876 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:01 smithi060 ceph-mon[40290]: 1.2 scrub starts 2024-09-20T22:15:01.876 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:01 smithi060 ceph-mon[40290]: 1.2 scrub ok 2024-09-20T22:15:01.876 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:01 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:15:01.876 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:01 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1453479688' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:15:01.876 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:01 smithi060 ceph-mon[40290]: osdmap e366: 8 total, 8 up, 8 in 2024-09-20T22:15:01.876 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:01 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4034203148' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:15:01.945 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:01.969 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:01.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:01.970 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:01.988 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:15:01.988 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:15:02.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:01 smithi067 ceph-mon[33511]: 6.19 scrub starts 2024-09-20T22:15:02.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:01 smithi067 ceph-mon[33511]: 6.19 scrub ok 2024-09-20T22:15:02.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:01 smithi067 ceph-mon[33511]: 1.2 scrub starts 2024-09-20T22:15:02.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:01 smithi067 ceph-mon[33511]: 1.2 scrub ok 2024-09-20T22:15:02.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:01 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:15:02.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:01 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1453479688' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:15:02.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:01 smithi067 ceph-mon[33511]: osdmap e366: 8 total, 8 up, 8 in 2024-09-20T22:15:02.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:01 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4034203148' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:15:02.071 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:02.097 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:02.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:02.097 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:02.198 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:02.223 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:02.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:02.224 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:02.325 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:02.350 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:02.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:02.351 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:02.451 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:02.478 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:02.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:02.479 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:02.580 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:02.606 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:02.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:02.607 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:02.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:02.710 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'unique_pool_2' removed 2024-09-20T22:15:02.711 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:15:02 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:15:02.744 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:02.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:02.745 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:02.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:02 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4034203148' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T22:15:02.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:02 smithi060 ceph-mon[29694]: osdmap e367: 8 total, 8 up, 8 in 2024-09-20T22:15:02.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:02 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1331467498' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:15:02.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:02 smithi060 ceph-mon[29694]: pgmap v648: 158 pgs: 158 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 925 B/s rd, 0 B/s wr, 361 op/s 2024-09-20T22:15:02.766 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:02 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:15:02.766 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:02 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4034203148' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T22:15:02.766 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:02 smithi060 ceph-mon[40290]: osdmap e367: 8 total, 8 up, 8 in 2024-09-20T22:15:02.766 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:02 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1331467498' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:15:02.766 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:02 smithi060 ceph-mon[40290]: pgmap v648: 158 pgs: 158 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 925 B/s rd, 0 B/s wr, 361 op/s 2024-09-20T22:15:02.766 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:02 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:15:02.846 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:02.878 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:02.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:02.879 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:02.979 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:03.002 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:03.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:03.003 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:02 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4034203148' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T22:15:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:02 smithi067 ceph-mon[33511]: osdmap e367: 8 total, 8 up, 8 in 2024-09-20T22:15:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:02 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1331467498' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:15:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:02 smithi067 ceph-mon[33511]: pgmap v648: 158 pgs: 158 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 925 B/s rd, 0 B/s wr, 361 op/s 2024-09-20T22:15:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:02 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:15:03.104 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:03.126 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:03.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:03.127 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:03.228 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:03.250 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:03.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:03.251 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:03.351 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:03.371 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-20T22:15:03.371 INFO:tasks.radosbench:Beginning radosbench... 2024-09-20T22:15:03.371 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_3 2024-09-20T22:15:03.372 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool create unique_pool_3 16 2024-09-20T22:15:03.374 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:03.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:03.375 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:03.476 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:03.498 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:03.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:03.499 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:03.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:03.623 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:03.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:03.624 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:03.725 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:03.747 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:03.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:03.748 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:03.848 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:03.870 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:03.871 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:03.871 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:03.905 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:03 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:15:03.905 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:03 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:15:03.905 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:03 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1331467498' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T22:15:03.905 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:03 smithi060 ceph-mon[40290]: osdmap e368: 8 total, 8 up, 8 in 2024-09-20T22:15:03.906 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:03 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:15:03.906 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:03 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:15:03.906 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:03 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1331467498' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T22:15:03.906 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:03 smithi060 ceph-mon[29694]: osdmap e368: 8 total, 8 up, 8 in 2024-09-20T22:15:03.932 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 5 1 2024-09-20T22:15:03.972 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:03.998 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:03.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:03.999 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:03 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:15:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:03 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:15:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:03 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1331467498' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_2", "pool2": "unique_pool_2", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T22:15:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:03 smithi067 ceph-mon[33511]: osdmap e368: 8 total, 8 up, 8 in 2024-09-20T22:15:04.099 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:04.121 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:04.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:04.122 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:04.223 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:04.245 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:04.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:04.246 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:04.346 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:04.368 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:04.369 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:04.369 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:04.470 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:04.491 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:04.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:04.492 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:04.523 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T22:15:04.533 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:15:04.557 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:04.594 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:04.599 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:04.616 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:04.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:04.617 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:04.717 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:04.748 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:04.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:04.749 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:04.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:04 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:15:04.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:04 smithi060 ceph-mon[40290]: pgmap v650: 142 pgs: 142 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 742 B/s rd, 0 op/s 2024-09-20T22:15:04.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:04 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:15:04.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:04 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:15:04.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:04 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:15:04.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:04 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:15:04.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:04 smithi060 ceph-mon[29694]: pgmap v650: 142 pgs: 142 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 742 B/s rd, 0 op/s 2024-09-20T22:15:04.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:04 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:15:04.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:04 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:15:04.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:04 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:15:04.846 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:04.850 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:04.882 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:04.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:04.883 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:04.984 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:05.006 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:05.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:05.007 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:04 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:15:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:04 smithi067 ceph-mon[33511]: pgmap v650: 142 pgs: 142 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 742 B/s rd, 0 op/s 2024-09-20T22:15:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:04 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:15:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:04 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:15:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:04 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:15:05.108 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:05.130 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:05.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:05.131 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:05.164 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:15:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:15:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:15:05.232 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:05.248 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:15:05.248 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:15:05.248 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:15:05.248 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:15:05.248 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:15:05.248 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:15:05.248 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:15:05.248 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:15:05.248 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:15:05.248 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:15:05.256 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:15:05.257 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:15:05.258 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:05.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:05.259 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:05.273 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:15:05.274 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:15:05.291 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:15:05.291 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:15:05.360 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:05.387 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:05.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:05.388 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:05.488 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:05.513 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:05.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:05.514 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:05.614 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:05.641 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:05.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:05.642 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:05.743 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:05.764 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:05.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:05.765 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:05.861 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:05 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3718933011' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:05.861 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:05 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/86029841' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:15:05.862 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:05 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3718933011' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:05.862 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:05 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/86029841' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:15:05.866 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:05.894 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:05.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:05.895 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:05.970 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.5 to 1 (10000) 2024-09-20T22:15:05.995 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:05 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3718933011' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:05 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/86029841' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:15:06.017 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:06.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:06.018 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:06.118 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:06.140 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:06.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:06.141 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:06.241 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:06.268 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:06.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:06.269 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:06.370 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:06.392 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:06.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:06.393 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:06.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:06.516 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:06.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:06.517 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:06.618 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:06.644 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:06.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:06.644 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:06.745 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:06.754 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 6 1 2024-09-20T22:15:06.770 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:06.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:06.771 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:06.871 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:06.894 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:06.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:06.895 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:06.994 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:06 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/86029841' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:15:06.994 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:06 smithi060 ceph-mon[29694]: osdmap e369: 8 total, 8 up, 8 in 2024-09-20T22:15:06.994 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:15:06.994 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:06 smithi060 ceph-mon[29694]: pgmap v652: 142 pgs: 142 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-20T22:15:06.994 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:06 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/86029841' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:15:06.994 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:06 smithi060 ceph-mon[40290]: osdmap e369: 8 total, 8 up, 8 in 2024-09-20T22:15:06.994 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:15:06.994 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:06 smithi060 ceph-mon[40290]: pgmap v652: 142 pgs: 142 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-20T22:15:06.997 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:06 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/86029841' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:15:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:06 smithi067 ceph-mon[33511]: osdmap e369: 8 total, 8 up, 8 in 2024-09-20T22:15:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:15:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:06 smithi067 ceph-mon[33511]: pgmap v652: 142 pgs: 142 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-20T22:15:07.019 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:07.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:07.020 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:07.121 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:07.147 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:07.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:07.148 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:07.241 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:07.241 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:07.249 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:07.266 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:07.274 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:07.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:07.275 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:07.376 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:07.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:07.398 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:07.399 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:07.499 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:07.523 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:07.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:07.524 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:07.625 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:07.646 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:07.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:07.647 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:07.748 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:07.774 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:07.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:07.775 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:07.876 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:07.898 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:07.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:07.899 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:07.902 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:15:07.914 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'unique_pool_3' created 2024-09-20T22:15:07.939 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:15:07.939 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:15:07.939 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:15:07.939 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:15:07.940 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:15:08.000 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:08.022 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:08.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:08.023 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:08.124 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:08.147 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:08.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:08.148 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:08.248 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:08.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:07 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1191313819' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-20T22:15:08.275 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:08.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:08.276 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:08.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:07 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1191313819' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-20T22:15:08.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:07 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1191313819' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]: dispatch 2024-09-20T22:15:08.377 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:08.404 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:08.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:08.405 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:08.505 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:08.533 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:08.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:08.534 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:08.635 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:08.661 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:08.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:08.662 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:08.762 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:08.787 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:08.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:08.788 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:08.889 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:08.913 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:08.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:08.914 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:08.915 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:15:09.015 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:09.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:09.038 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:09.038 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:09.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:09.164 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:09.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:09.165 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:09.252 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:08 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1191313819' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-20T22:15:09.252 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:08 smithi060 ceph-mon[29694]: osdmap e370: 8 total, 8 up, 8 in 2024-09-20T22:15:09.252 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:08 smithi060 ceph-mon[29694]: pgmap v654: 158 pgs: 6 creating+peering, 10 unknown, 142 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T22:15:09.252 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:08 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3319760754' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:09.252 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:08 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:15:09.252 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:08 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3319760754' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:15:09.253 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:08 smithi060 ceph-mon[29694]: osdmap e371: 8 total, 8 up, 8 in 2024-09-20T22:15:09.253 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:08 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1191313819' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-20T22:15:09.253 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:08 smithi060 ceph-mon[40290]: osdmap e370: 8 total, 8 up, 8 in 2024-09-20T22:15:09.253 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:08 smithi060 ceph-mon[40290]: pgmap v654: 158 pgs: 6 creating+peering, 10 unknown, 142 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T22:15:09.253 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:08 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3319760754' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:09.253 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:08 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:15:09.253 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:08 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3319760754' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:15:09.253 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:08 smithi060 ceph-mon[40290]: osdmap e371: 8 total, 8 up, 8 in 2024-09-20T22:15:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:08 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1191313819' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_3", "pg_num": 16}]': finished 2024-09-20T22:15:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:08 smithi067 ceph-mon[33511]: osdmap e370: 8 total, 8 up, 8 in 2024-09-20T22:15:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:08 smithi067 ceph-mon[33511]: pgmap v654: 158 pgs: 6 creating+peering, 10 unknown, 142 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T22:15:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:08 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3319760754' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:08 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:15:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:08 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3319760754' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:15:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:08 smithi067 ceph-mon[33511]: osdmap e371: 8 total, 8 up, 8 in 2024-09-20T22:15:09.266 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:09.281 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool application enable unique_pool_3 rados --yes-i-really-mean-it || true 2024-09-20T22:15:09.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:09.288 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:09.289 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:09.390 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:09.416 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:09.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:09.417 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:09.518 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:09.544 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:09.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:09.544 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:09.645 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:09.672 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:09.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:09.673 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:09.773 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:09.796 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:09.796 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:09.796 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:09.897 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:09.912 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T22:15:09.921 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:09.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:09.922 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:10.023 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:10.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:10.046 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:10.046 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:10.147 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:10.174 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:10.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:10.175 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:10.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:09 smithi067 ceph-mon[33511]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T22:15:10.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:09 smithi067 ceph-mon[33511]: osdmap e372: 8 total, 8 up, 8 in 2024-09-20T22:15:10.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:10.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:09 smithi060 ceph-mon[29694]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T22:15:10.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:09 smithi060 ceph-mon[29694]: osdmap e372: 8 total, 8 up, 8 in 2024-09-20T22:15:10.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:09 smithi060 ceph-mon[40290]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T22:15:10.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:09 smithi060 ceph-mon[40290]: osdmap e372: 8 total, 8 up, 8 in 2024-09-20T22:15:10.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:10.300 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:10.300 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:10.401 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:10.428 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:10.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:10.429 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:10.529 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:10.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:10.554 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:10.554 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:10.655 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:10.679 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:10.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:10.681 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:10.724 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:10.763 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:10.782 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:10.810 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:10.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:10.811 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:10.912 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:10.939 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:10.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:10.940 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:10.941 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.6 to 1 (10000) 2024-09-20T22:15:11.041 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:11.067 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:11.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:11.068 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:11.169 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:11.196 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:11.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:11.196 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:11.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:10 smithi067 ceph-mon[33511]: pgmap v657: 158 pgs: 7 creating+peering, 9 unknown, 142 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T22:15:11.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:10 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2561779217' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:15:11.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:10 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:15:11.273 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:10 smithi060 ceph-mon[29694]: pgmap v657: 158 pgs: 7 creating+peering, 9 unknown, 142 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T22:15:11.274 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:10 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2561779217' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:15:11.274 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:10 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:15:11.274 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:10 smithi060 ceph-mon[40290]: pgmap v657: 158 pgs: 7 creating+peering, 9 unknown, 142 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T22:15:11.274 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:10 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2561779217' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:15:11.274 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:10 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:15:11.297 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:11.320 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:11.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:11.321 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:11.381 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:15:11.381 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:15:11.396 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:15:11.396 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:15:11.413 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:15:11.413 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:15:11.421 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:11.430 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:15:11.430 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:15:11.443 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:11.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:11.444 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:11.448 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:15:11.448 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:15:11.465 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:15:11.466 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:15:11.482 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:15:11.482 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:15:11.500 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:15:11.500 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:15:11.544 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:11.566 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:11.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:11.567 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:11.667 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:11.695 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:11.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:11.696 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:11.796 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:11.819 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:11.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:11.820 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:11.921 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:11.943 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:11.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:11.944 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:12.045 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:12.072 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:12.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:12.073 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:12.174 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:12.197 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:12.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:12.197 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:12.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:11 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:15:12.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:11 smithi067 ceph-mon[33511]: osdmap e373: 8 total, 8 up, 8 in 2024-09-20T22:15:12.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:11 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2446206242' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:12.279 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:11 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:15:12.279 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:11 smithi060 ceph-mon[29694]: osdmap e373: 8 total, 8 up, 8 in 2024-09-20T22:15:12.280 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:11 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2446206242' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:12.280 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:11 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:15:12.280 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:11 smithi060 ceph-mon[40290]: osdmap e373: 8 total, 8 up, 8 in 2024-09-20T22:15:12.280 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:11 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2446206242' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:12.298 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:12.326 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:12.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:12.327 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:12.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:12.448 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:15:12.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:12.451 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:12.451 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:12.553 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:12.576 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:12.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:12.577 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:12.585 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:15:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:15:12.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:12.700 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:12.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:12.701 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:12.802 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:12.820 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:15:12.820 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:15:12.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:12.825 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:12.825 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:12.925 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:12.947 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:12.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:12.948 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:13.049 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:13.071 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:13.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:13.072 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:13.160 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:12 smithi060 ceph-mon[29694]: pgmap v659: 158 pgs: 7 creating+peering, 151 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:15:13.160 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:12 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:15:13.160 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:12 smithi060 ceph-mon[40290]: pgmap v659: 158 pgs: 7 creating+peering, 151 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:15:13.161 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:12 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:15:13.173 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:13.189 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 7 1 2024-09-20T22:15:13.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:13.196 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:13.197 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:13.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:12 smithi067 ceph-mon[33511]: pgmap v659: 158 pgs: 7 creating+peering, 151 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:15:13.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:12 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:15:13.297 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:13.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:13.320 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:13.321 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:13.410 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:13.421 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:13.444 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:13.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:13.445 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:13.545 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:13.567 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:13.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:13.568 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:13.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:13.690 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:13.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:13.691 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:13.792 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:13.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:13.814 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:13.814 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:13.860 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T22:15:13.880 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:13.915 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:13.936 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:13.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:13.937 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:14.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:14.060 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:14.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:14.061 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:14.162 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:14.184 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:14.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:14.185 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:14.286 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:14.308 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:15:14.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:14.310 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:14.310 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:14.410 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:14.433 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:14.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:14.434 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:14.534 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:14.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:14.559 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:14.559 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:14.660 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:14.682 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:14.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:14.683 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:14.783 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:14.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:14.806 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:14.807 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:14.907 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:14.929 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:14.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:14.930 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:15.031 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:15.053 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:15.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:15.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:15.092 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:15.128 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:15.134 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:15.154 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:15.176 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:15.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:15.177 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:15.272 INFO:teuthology.orchestra.run.smithi060.stderr:enabled application 'rados' on pool 'unique_pool_3' 2024-09-20T22:15:15.272 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:15:15.278 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:15.303 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:15.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:15.304 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:15.320 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:15:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:15:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:15:15.320 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:15 smithi060 ceph-mon[29694]: pgmap v660: 158 pgs: 158 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:15:15.320 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:15 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2102047340' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:15:15.320 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:15 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:15:15.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:15 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1911915856' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:15:15.321 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:15 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:15:15.321 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:15 smithi060 ceph-mon[40290]: pgmap v660: 158 pgs: 158 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:15:15.321 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:15 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2102047340' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:15:15.321 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:15 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:15:15.321 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:15 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1911915856' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:15:15.321 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:15 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:15:15.404 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:15.428 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:15.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:15.429 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:15.530 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:15.552 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:15.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:15.553 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:15.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:15.676 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:15.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:15.677 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:15 smithi067 ceph-mon[33511]: pgmap v660: 158 pgs: 158 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:15:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:15 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2102047340' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:15:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:15 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:15:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:15 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1911915856' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:15:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:15 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:15:15.778 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:15.802 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:15.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:15.802 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:15.903 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:15.926 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:15.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:15.927 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:16.028 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:16.051 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:16.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:16.052 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:16.153 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:16.175 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:16.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:16.176 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:16.277 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:16.301 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:16.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:16.302 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:16.403 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:16.427 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:16.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:16.428 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:16.529 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:16.535 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:16 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:15:16.535 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:16 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T22:15:16.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:16 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:15:16.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:16 smithi060 ceph-mon[40290]: osdmap e374: 8 total, 8 up, 8 in 2024-09-20T22:15:16.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:16 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:15:16.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:16 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:15:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:16 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:15:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:16 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T22:15:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:16 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:15:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:16 smithi060 ceph-mon[29694]: osdmap e374: 8 total, 8 up, 8 in 2024-09-20T22:15:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:16 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:15:16.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:16 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:15:16.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:16.552 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:16.552 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:16.653 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:16.677 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:16.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:16.677 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:16 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:15:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:16 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_3", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T22:15:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:16 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:15:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:16 smithi067 ceph-mon[33511]: osdmap e374: 8 total, 8 up, 8 in 2024-09-20T22:15:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:16 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:15:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:16 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:15:16.779 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:16.802 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:16.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:16.803 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:16.903 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:16.926 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:16.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:16.927 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:17.028 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:17.050 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:17.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:17.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:17.152 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:17.176 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:17.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:17.177 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:17.277 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:17.284 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.7 to 1 (10000) 2024-09-20T22:15:17.302 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:17.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:17.303 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:17.374 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[40290]: 3.6 scrub starts 2024-09-20T22:15:17.374 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[40290]: 3.6 scrub ok 2024-09-20T22:15:17.374 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[40290]: pgmap v662: 158 pgs: 158 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-20T22:15:17.374 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[40290]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T22:15:17.374 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[40290]: osdmap e375: 8 total, 8 up, 8 in 2024-09-20T22:15:17.374 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/913677878' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:15:17.374 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:15:17.374 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[40290]: 5.14 scrub starts 2024-09-20T22:15:17.374 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[40290]: 5.14 scrub ok 2024-09-20T22:15:17.374 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[40290]: 6.1e scrub starts 2024-09-20T22:15:17.374 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[40290]: 6.1e scrub ok 2024-09-20T22:15:17.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[29694]: 3.6 scrub starts 2024-09-20T22:15:17.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[29694]: 3.6 scrub ok 2024-09-20T22:15:17.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[29694]: pgmap v662: 158 pgs: 158 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-20T22:15:17.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[29694]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T22:15:17.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[29694]: osdmap e375: 8 total, 8 up, 8 in 2024-09-20T22:15:17.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/913677878' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:15:17.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:15:17.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[29694]: 5.14 scrub starts 2024-09-20T22:15:17.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[29694]: 5.14 scrub ok 2024-09-20T22:15:17.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[29694]: 6.1e scrub starts 2024-09-20T22:15:17.375 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:17 smithi060 ceph-mon[29694]: 6.1e scrub ok 2024-09-20T22:15:17.404 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:17.426 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:17.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:17.427 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:17.490 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:15:17.491 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:15:17.491 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:15:17.491 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:15:17.491 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:15:17.491 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:15:17.491 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:15:17.491 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:15:17.491 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:15:17.491 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:15:17.492 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:15:17.492 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:15:17.492 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:15:17.492 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:15:17.503 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:15:17.504 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:15:17.528 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:17.551 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:17.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:17.552 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:17.608 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:15:17.609 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:15:17.609 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-20T22:15:17.609 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:15:17.652 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:17.675 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:17.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:17.676 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:17.712 DEBUG:teuthology.orchestra.run.smithi060:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_3 bench 90 write' 2024-09-20T22:15:17.718 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-20T22:15:17.719 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-20T22:15:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:17 smithi067 ceph-mon[33511]: 3.6 scrub starts 2024-09-20T22:15:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:17 smithi067 ceph-mon[33511]: 3.6 scrub ok 2024-09-20T22:15:17.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:17 smithi067 ceph-mon[33511]: pgmap v662: 158 pgs: 158 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-09-20T22:15:17.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:17 smithi067 ceph-mon[33511]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T22:15:17.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:17 smithi067 ceph-mon[33511]: osdmap e375: 8 total, 8 up, 8 in 2024-09-20T22:15:17.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:17 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/913677878' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:15:17.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:17 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:15:17.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:17 smithi067 ceph-mon[33511]: 5.14 scrub starts 2024-09-20T22:15:17.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:17 smithi067 ceph-mon[33511]: 5.14 scrub ok 2024-09-20T22:15:17.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:17 smithi067 ceph-mon[33511]: 6.1e scrub starts 2024-09-20T22:15:17.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:17 smithi067 ceph-mon[33511]: 6.1e scrub ok 2024-09-20T22:15:17.776 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:17.801 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:17.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:17.802 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:17.903 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:17.914 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:hints = 1 2024-09-20T22:15:17.914 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-20T22:15:17.914 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Object prefix: benchmark_data_smithi060_167766 2024-09-20T22:15:17.914 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:15:17.915 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 0 0 0 0 0 0 - 0 2024-09-20T22:15:17.932 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:17.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:17.933 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:18.020 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:15:18.034 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:18.070 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:18.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:18.071 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:18.172 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:18.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:18.209 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:18.209 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:18.309 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:18.350 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:18.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:18.352 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:18.445 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:18 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:15:18.445 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:18 smithi060 ceph-mon[29694]: osdmap e376: 8 total, 8 up, 8 in 2024-09-20T22:15:18.445 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:18 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1684038965' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:18.445 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:18 smithi060 ceph-mon[29694]: 1.6 scrub starts 2024-09-20T22:15:18.445 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:18 smithi060 ceph-mon[29694]: 1.6 scrub ok 2024-09-20T22:15:18.445 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:18 smithi060 ceph-mon[29694]: 6.1a scrub starts 2024-09-20T22:15:18.445 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:18 smithi060 ceph-mon[29694]: 6.1a scrub ok 2024-09-20T22:15:18.446 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:18 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:15:18.446 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:18 smithi060 ceph-mon[40290]: osdmap e376: 8 total, 8 up, 8 in 2024-09-20T22:15:18.446 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:18 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1684038965' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:18.446 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:18 smithi060 ceph-mon[40290]: 1.6 scrub starts 2024-09-20T22:15:18.446 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:18 smithi060 ceph-mon[40290]: 1.6 scrub ok 2024-09-20T22:15:18.446 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:18 smithi060 ceph-mon[40290]: 6.1a scrub starts 2024-09-20T22:15:18.446 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:18 smithi060 ceph-mon[40290]: 6.1a scrub ok 2024-09-20T22:15:18.452 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:18.490 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:18.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:18.490 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:18.591 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:18.630 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:18.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:18.631 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:18.731 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:18 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:15:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:18 smithi067 ceph-mon[33511]: osdmap e376: 8 total, 8 up, 8 in 2024-09-20T22:15:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:18 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1684038965' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:18 smithi067 ceph-mon[33511]: 1.6 scrub starts 2024-09-20T22:15:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:18 smithi067 ceph-mon[33511]: 1.6 scrub ok 2024-09-20T22:15:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:18 smithi067 ceph-mon[33511]: 6.1a scrub starts 2024-09-20T22:15:18.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:18 smithi067 ceph-mon[33511]: 6.1a scrub ok 2024-09-20T22:15:18.767 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:18.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:18.768 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:18.869 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:18.905 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:18.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:18.907 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:18.915 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 1 16 3608 3592 224.487 224.5 0.00505177 0.00443859 2024-09-20T22:15:18.955 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 0 1 2024-09-20T22:15:19.008 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:19.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:19.083 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:19.083 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:19.184 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:19.219 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:19.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:19.220 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:19.320 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:19.355 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:19.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:19.357 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:19.458 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:19.497 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:19.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:19.499 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:19.595 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:19 smithi060 ceph-mon[29694]: 4.1d scrub starts 2024-09-20T22:15:19.596 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:19 smithi060 ceph-mon[29694]: 4.1d scrub ok 2024-09-20T22:15:19.596 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:19 smithi060 ceph-mon[29694]: pgmap v665: 158 pgs: 158 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:15:19.596 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:19 smithi060 ceph-mon[29694]: 5.a scrub starts 2024-09-20T22:15:19.596 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:19 smithi060 ceph-mon[29694]: 5.a scrub ok 2024-09-20T22:15:19.596 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:19 smithi060 ceph-mon[29694]: 4.e scrub starts 2024-09-20T22:15:19.596 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:19 smithi060 ceph-mon[29694]: 4.e scrub ok 2024-09-20T22:15:19.597 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:19 smithi060 ceph-mon[40290]: 4.1d scrub starts 2024-09-20T22:15:19.597 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:19 smithi060 ceph-mon[40290]: 4.1d scrub ok 2024-09-20T22:15:19.597 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:19 smithi060 ceph-mon[40290]: pgmap v665: 158 pgs: 158 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:15:19.597 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:19 smithi060 ceph-mon[40290]: 5.a scrub starts 2024-09-20T22:15:19.597 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:19 smithi060 ceph-mon[40290]: 5.a scrub ok 2024-09-20T22:15:19.597 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:19 smithi060 ceph-mon[40290]: 4.e scrub starts 2024-09-20T22:15:19.597 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:19 smithi060 ceph-mon[40290]: 4.e scrub ok 2024-09-20T22:15:19.600 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:19.603 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:19.634 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:19.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:19.635 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:19.736 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:19.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:19 smithi067 ceph-mon[33511]: 4.1d scrub starts 2024-09-20T22:15:19.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:19 smithi067 ceph-mon[33511]: 4.1d scrub ok 2024-09-20T22:15:19.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:19 smithi067 ceph-mon[33511]: pgmap v665: 158 pgs: 158 active+clean; 583 KiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:15:19.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:19 smithi067 ceph-mon[33511]: 5.a scrub starts 2024-09-20T22:15:19.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:19 smithi067 ceph-mon[33511]: 5.a scrub ok 2024-09-20T22:15:19.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:19 smithi067 ceph-mon[33511]: 4.e scrub starts 2024-09-20T22:15:19.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:19 smithi067 ceph-mon[33511]: 4.e scrub ok 2024-09-20T22:15:19.774 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:19.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:19.775 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:19.876 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:19.915 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 2 15 6868 6853 214.115 203.812 0.00317586 0.0046623 2024-09-20T22:15:19.916 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:19.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:19.916 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:20.017 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:20.055 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:20.056 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:20.056 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:20.124 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T22:15:20.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:20.219 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:20.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:20.220 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:20.320 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:20.367 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:20.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:20.370 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:20.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:20.508 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:20.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:20.509 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:20.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:20 smithi060 ceph-mon[29694]: 3.18 scrub starts 2024-09-20T22:15:20.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:20 smithi060 ceph-mon[29694]: 3.18 scrub ok 2024-09-20T22:15:20.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:20 smithi060 ceph-mon[29694]: 5.15 scrub starts 2024-09-20T22:15:20.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:20 smithi060 ceph-mon[29694]: 5.15 scrub ok 2024-09-20T22:15:20.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:20 smithi060 ceph-mon[29694]: 3.14 scrub starts 2024-09-20T22:15:20.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:20 smithi060 ceph-mon[29694]: 3.14 scrub ok 2024-09-20T22:15:20.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:20 smithi060 ceph-mon[40290]: 3.18 scrub starts 2024-09-20T22:15:20.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:20 smithi060 ceph-mon[40290]: 3.18 scrub ok 2024-09-20T22:15:20.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:20 smithi060 ceph-mon[40290]: 5.15 scrub starts 2024-09-20T22:15:20.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:20 smithi060 ceph-mon[40290]: 5.15 scrub ok 2024-09-20T22:15:20.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:20 smithi060 ceph-mon[40290]: 3.14 scrub starts 2024-09-20T22:15:20.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:20 smithi060 ceph-mon[40290]: 3.14 scrub ok 2024-09-20T22:15:20.610 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:20.652 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:20.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:20.653 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:20.754 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:20.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:20 smithi067 ceph-mon[33511]: 3.18 scrub starts 2024-09-20T22:15:20.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:20 smithi067 ceph-mon[33511]: 3.18 scrub ok 2024-09-20T22:15:20.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:20 smithi067 ceph-mon[33511]: 5.15 scrub starts 2024-09-20T22:15:20.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:20 smithi067 ceph-mon[33511]: 5.15 scrub ok 2024-09-20T22:15:20.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:20 smithi067 ceph-mon[33511]: 3.14 scrub starts 2024-09-20T22:15:20.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:20 smithi067 ceph-mon[33511]: 3.14 scrub ok 2024-09-20T22:15:20.793 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:20.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:20.794 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:20.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:20.915 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 3 16 9934 9918 206.593 191.562 0.00475215 0.00483166 2024-09-20T22:15:20.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:20.930 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:20.931 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:21.031 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:21.066 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:21.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:21.067 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:21.069 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:21.167 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:21.204 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:21.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:21.205 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:21.306 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:21.349 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:21.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:21.350 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:21.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:21 smithi060 ceph-mon[29694]: 3.1c scrub starts 2024-09-20T22:15:21.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:21 smithi060 ceph-mon[29694]: 3.1c scrub ok 2024-09-20T22:15:21.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:21 smithi060 ceph-mon[29694]: pgmap v666: 158 pgs: 158 active+clean; 7.1 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 1.1 MiB/s wr, 18 op/s 2024-09-20T22:15:21.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:15:21.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:21 smithi060 ceph-mon[29694]: 4.19 scrub starts 2024-09-20T22:15:21.415 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:21 smithi060 ceph-mon[29694]: 4.19 scrub ok 2024-09-20T22:15:21.415 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:21 smithi060 ceph-mon[40290]: 3.1c scrub starts 2024-09-20T22:15:21.415 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:21 smithi060 ceph-mon[40290]: 3.1c scrub ok 2024-09-20T22:15:21.415 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:21 smithi060 ceph-mon[40290]: pgmap v666: 158 pgs: 158 active+clean; 7.1 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 1.1 MiB/s wr, 18 op/s 2024-09-20T22:15:21.416 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:15:21.416 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:21 smithi060 ceph-mon[40290]: 4.19 scrub starts 2024-09-20T22:15:21.416 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:21 smithi060 ceph-mon[40290]: 4.19 scrub ok 2024-09-20T22:15:21.451 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:21.479 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:21.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:21.480 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:21.581 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:21.598 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:15:21.616 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:21.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:21.617 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:21.718 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:21.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:21 smithi067 ceph-mon[33511]: 3.1c scrub starts 2024-09-20T22:15:21.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:21 smithi067 ceph-mon[33511]: 3.1c scrub ok 2024-09-20T22:15:21.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:21 smithi067 ceph-mon[33511]: pgmap v666: 158 pgs: 158 active+clean; 7.1 MiB data, 4.1 GiB used, 711 GiB / 715 GiB avail; 853 B/s rd, 1.1 MiB/s wr, 18 op/s 2024-09-20T22:15:21.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:15:21.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:21 smithi067 ceph-mon[33511]: 4.19 scrub starts 2024-09-20T22:15:21.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:21 smithi067 ceph-mon[33511]: 4.19 scrub ok 2024-09-20T22:15:21.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:21.767 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:21.767 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:21.868 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:21.902 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:21.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:21.903 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:21.915 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 4 16 12458 12442 194.373 157.75 0.0058091 0.00513493 2024-09-20T22:15:22.004 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:22.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:22.038 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:22.038 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:22.105 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:22.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:22.173 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:22.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:22.174 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:22.275 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:22.317 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:22.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:22.319 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:22.334 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:15:22.420 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:22.453 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:22.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:22.454 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:22.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[29694]: 5.11 scrub starts 2024-09-20T22:15:22.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[29694]: 5.11 scrub ok 2024-09-20T22:15:22.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[29694]: 6.10 scrub starts 2024-09-20T22:15:22.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[29694]: 6.10 scrub ok 2024-09-20T22:15:22.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[29694]: 3.1d scrub starts 2024-09-20T22:15:22.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[29694]: 3.1d scrub ok 2024-09-20T22:15:22.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4123655088' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:22.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:22.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[29694]: 4.9 scrub starts 2024-09-20T22:15:22.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[29694]: 4.9 scrub ok 2024-09-20T22:15:22.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[40290]: 5.11 scrub starts 2024-09-20T22:15:22.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[40290]: 5.11 scrub ok 2024-09-20T22:15:22.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[40290]: 6.10 scrub starts 2024-09-20T22:15:22.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[40290]: 6.10 scrub ok 2024-09-20T22:15:22.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[40290]: 3.1d scrub starts 2024-09-20T22:15:22.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[40290]: 3.1d scrub ok 2024-09-20T22:15:22.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4123655088' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:22.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:22.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[40290]: 4.9 scrub starts 2024-09-20T22:15:22.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:22 smithi060 ceph-mon[40290]: 4.9 scrub ok 2024-09-20T22:15:22.555 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:22.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:22.590 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:22.591 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:22.691 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:22.730 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:22.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:22.731 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:22.763 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:15:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:15:22.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:22 smithi067 ceph-mon[33511]: 5.11 scrub starts 2024-09-20T22:15:22.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:22 smithi067 ceph-mon[33511]: 5.11 scrub ok 2024-09-20T22:15:22.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:22 smithi067 ceph-mon[33511]: 6.10 scrub starts 2024-09-20T22:15:22.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:22 smithi067 ceph-mon[33511]: 6.10 scrub ok 2024-09-20T22:15:22.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:22 smithi067 ceph-mon[33511]: 3.1d scrub starts 2024-09-20T22:15:22.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:22 smithi067 ceph-mon[33511]: 3.1d scrub ok 2024-09-20T22:15:22.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:22 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4123655088' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:22.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:22 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:22.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:22 smithi067 ceph-mon[33511]: 4.9 scrub starts 2024-09-20T22:15:22.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:22 smithi067 ceph-mon[33511]: 4.9 scrub ok 2024-09-20T22:15:22.832 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:22.871 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:22.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:22.872 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:22.915 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 5 16 15147 15131 189.108 168.062 0.00382528 0.00528031 2024-09-20T22:15:22.967 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:15:22.967 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:15:22.968 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:15:22.968 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:15:22.968 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:15:22.973 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:23.017 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:23.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:23.018 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:23.119 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:23.156 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:23.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:23.157 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:23.258 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:23.294 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:23.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:23.295 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:23.396 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:23.399 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[40290]: 6.1 scrub starts 2024-09-20T22:15:23.399 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[40290]: 6.1 scrub ok 2024-09-20T22:15:23.399 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[40290]: 3.1 scrub starts 2024-09-20T22:15:23.399 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[40290]: 3.1 scrub ok 2024-09-20T22:15:23.399 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[40290]: pgmap v667: 158 pgs: 158 active+clean; 380 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.4 KiB/s rd, 55 MiB/s wr, 875 op/s 2024-09-20T22:15:23.399 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:15:23.399 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:15:23.399 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:15:23.399 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[40290]: osdmap e377: 8 total, 8 up, 8 in 2024-09-20T22:15:23.399 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:15:23.399 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3107529641' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:15:23.399 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:15:23.400 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[40290]: 4.18 scrub starts 2024-09-20T22:15:23.400 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[40290]: 4.18 scrub ok 2024-09-20T22:15:23.400 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[29694]: 6.1 scrub starts 2024-09-20T22:15:23.400 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[29694]: 6.1 scrub ok 2024-09-20T22:15:23.400 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[29694]: 3.1 scrub starts 2024-09-20T22:15:23.400 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[29694]: 3.1 scrub ok 2024-09-20T22:15:23.400 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[29694]: pgmap v667: 158 pgs: 158 active+clean; 380 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.4 KiB/s rd, 55 MiB/s wr, 875 op/s 2024-09-20T22:15:23.400 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:15:23.400 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:15:23.400 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:15:23.400 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[29694]: osdmap e377: 8 total, 8 up, 8 in 2024-09-20T22:15:23.400 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:15:23.400 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3107529641' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:15:23.401 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:15:23.401 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[29694]: 4.18 scrub starts 2024-09-20T22:15:23.401 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:23 smithi060 ceph-mon[29694]: 4.18 scrub ok 2024-09-20T22:15:23.440 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:23.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:23.442 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:23.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:23 smithi067 ceph-mon[33511]: 6.1 scrub starts 2024-09-20T22:15:23.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:23 smithi067 ceph-mon[33511]: 6.1 scrub ok 2024-09-20T22:15:23.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:23 smithi067 ceph-mon[33511]: 3.1 scrub starts 2024-09-20T22:15:23.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:23 smithi067 ceph-mon[33511]: 3.1 scrub ok 2024-09-20T22:15:23.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:23 smithi067 ceph-mon[33511]: pgmap v667: 158 pgs: 158 active+clean; 380 MiB data, 5.6 GiB used, 710 GiB / 715 GiB avail; 1.4 KiB/s rd, 55 MiB/s wr, 875 op/s 2024-09-20T22:15:23.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:23 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:15:23.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:23 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:15:23.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:23 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:15:23.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:23 smithi067 ceph-mon[33511]: osdmap e377: 8 total, 8 up, 8 in 2024-09-20T22:15:23.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:23 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:15:23.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:23 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3107529641' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:15:23.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:23 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:15:23.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:23 smithi067 ceph-mon[33511]: 4.18 scrub starts 2024-09-20T22:15:23.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:23 smithi067 ceph-mon[33511]: 4.18 scrub ok 2024-09-20T22:15:23.543 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:23.564 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.0 to 1 (10000) 2024-09-20T22:15:23.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:23.586 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:23.586 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:23.686 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:23.722 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:23.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:23.723 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:23.824 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:23.865 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:23.866 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:23.866 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:23.915 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 6 15 17823 17808 185.473 167.312 0.00351318 0.00538372 2024-09-20T22:15:23.967 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:24.001 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:24.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:24.002 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:24.103 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:24.140 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:24.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:24.142 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:24.242 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:24.281 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:24.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:24.282 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:24.382 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:24.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T22:15:24.433 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:24.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:24.435 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:24.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:24.584 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:24.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:24.585 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:24.686 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:24.733 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:24.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:24.735 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:24.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:24 smithi067 ceph-mon[33511]: 6.16 scrub starts 2024-09-20T22:15:24.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:24 smithi067 ceph-mon[33511]: 6.16 scrub ok 2024-09-20T22:15:24.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:24 smithi067 ceph-mon[33511]: 3.d scrub starts 2024-09-20T22:15:24.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:24 smithi067 ceph-mon[33511]: 3.d scrub ok 2024-09-20T22:15:24.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:24 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:15:24.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:24 smithi067 ceph-mon[33511]: osdmap e378: 8 total, 8 up, 8 in 2024-09-20T22:15:24.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:24 smithi060 ceph-mon[40290]: 6.16 scrub starts 2024-09-20T22:15:24.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:24 smithi060 ceph-mon[40290]: 6.16 scrub ok 2024-09-20T22:15:24.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:24 smithi060 ceph-mon[40290]: 3.d scrub starts 2024-09-20T22:15:24.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:24 smithi060 ceph-mon[40290]: 3.d scrub ok 2024-09-20T22:15:24.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:24 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:15:24.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:24 smithi060 ceph-mon[40290]: osdmap e378: 8 total, 8 up, 8 in 2024-09-20T22:15:24.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:24 smithi060 ceph-mon[29694]: 6.16 scrub starts 2024-09-20T22:15:24.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:24 smithi060 ceph-mon[29694]: 6.16 scrub ok 2024-09-20T22:15:24.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:24 smithi060 ceph-mon[29694]: 3.d scrub starts 2024-09-20T22:15:24.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:24 smithi060 ceph-mon[29694]: 3.d scrub ok 2024-09-20T22:15:24.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:24 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:15:24.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:24 smithi060 ceph-mon[29694]: osdmap e378: 8 total, 8 up, 8 in 2024-09-20T22:15:24.835 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:24.836 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:24.866 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:24.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:24.867 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:24.880 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-20T22:15:24.880 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:15:24.916 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 7 16 20336 20320 181.394 157 0.0102594 0.00550526 2024-09-20T22:15:24.967 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:25.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:25.013 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:25.013 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:25.114 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:25.157 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:25.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:25.159 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:25.166 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:15:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:15:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:15:25.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:25.283 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:25.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:25.283 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:25.384 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:25.419 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:25.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:25.420 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:25.458 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:15:25.496 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:25 smithi060 ceph-mon[29694]: 6.6 scrub starts 2024-09-20T22:15:25.496 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:25 smithi060 ceph-mon[29694]: 6.6 scrub ok 2024-09-20T22:15:25.497 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:25 smithi060 ceph-mon[29694]: 5.f scrub starts 2024-09-20T22:15:25.497 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:25 smithi060 ceph-mon[29694]: 5.f scrub ok 2024-09-20T22:15:25.497 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:25 smithi060 ceph-mon[29694]: pgmap v670: 158 pgs: 158 active+clean; 799 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 737 B/s rd, 115 MiB/s wr, 1.84k op/s 2024-09-20T22:15:25.497 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:25 smithi060 ceph-mon[29694]: 5.6 scrub starts 2024-09-20T22:15:25.497 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:25 smithi060 ceph-mon[29694]: 5.6 scrub ok 2024-09-20T22:15:25.497 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:25 smithi060 ceph-mon[40290]: 6.6 scrub starts 2024-09-20T22:15:25.497 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:25 smithi060 ceph-mon[40290]: 6.6 scrub ok 2024-09-20T22:15:25.497 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:25 smithi060 ceph-mon[40290]: 5.f scrub starts 2024-09-20T22:15:25.497 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:25 smithi060 ceph-mon[40290]: 5.f scrub ok 2024-09-20T22:15:25.497 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:25 smithi060 ceph-mon[40290]: pgmap v670: 158 pgs: 158 active+clean; 799 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 737 B/s rd, 115 MiB/s wr, 1.84k op/s 2024-09-20T22:15:25.498 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:25 smithi060 ceph-mon[40290]: 5.6 scrub starts 2024-09-20T22:15:25.498 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:25 smithi060 ceph-mon[40290]: 5.6 scrub ok 2024-09-20T22:15:25.521 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:25.555 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:25.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:25.556 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:25.657 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:25.700 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:25.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:25.701 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:25 smithi067 ceph-mon[33511]: 6.6 scrub starts 2024-09-20T22:15:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:25 smithi067 ceph-mon[33511]: 6.6 scrub ok 2024-09-20T22:15:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:25 smithi067 ceph-mon[33511]: 5.f scrub starts 2024-09-20T22:15:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:25 smithi067 ceph-mon[33511]: 5.f scrub ok 2024-09-20T22:15:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:25 smithi067 ceph-mon[33511]: pgmap v670: 158 pgs: 158 active+clean; 799 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 737 B/s rd, 115 MiB/s wr, 1.84k op/s 2024-09-20T22:15:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:25 smithi067 ceph-mon[33511]: 5.6 scrub starts 2024-09-20T22:15:25.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:25 smithi067 ceph-mon[33511]: 5.6 scrub ok 2024-09-20T22:15:25.801 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:25.823 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:25.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:25.823 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:25.916 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 8 16 22381 22365 174.696 127.812 0.00262688 0.00571802 2024-09-20T22:15:25.924 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:25.961 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:25.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:25.963 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:26.063 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:26.099 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:26.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:26.100 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:26.201 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:26.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:26.241 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:26.241 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:26.342 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:26.363 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:26.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:26.364 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:26.465 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:26.500 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:26.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:26.501 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:26.576 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:26 smithi060 ceph-mon[29694]: 3.1b scrub starts 2024-09-20T22:15:26.577 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:26 smithi060 ceph-mon[29694]: 3.1b scrub ok 2024-09-20T22:15:26.577 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:26 smithi060 ceph-mon[29694]: 1.0 scrub starts 2024-09-20T22:15:26.577 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:26 smithi060 ceph-mon[29694]: 1.0 scrub ok 2024-09-20T22:15:26.577 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:26 smithi060 ceph-mon[29694]: 5.18 scrub starts 2024-09-20T22:15:26.577 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:26 smithi060 ceph-mon[29694]: 5.18 scrub ok 2024-09-20T22:15:26.577 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:26 smithi060 ceph-mon[40290]: 3.1b scrub starts 2024-09-20T22:15:26.577 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:26 smithi060 ceph-mon[40290]: 3.1b scrub ok 2024-09-20T22:15:26.577 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:26 smithi060 ceph-mon[40290]: 1.0 scrub starts 2024-09-20T22:15:26.577 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:26 smithi060 ceph-mon[40290]: 1.0 scrub ok 2024-09-20T22:15:26.577 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:26 smithi060 ceph-mon[40290]: 5.18 scrub starts 2024-09-20T22:15:26.577 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:26 smithi060 ceph-mon[40290]: 5.18 scrub ok 2024-09-20T22:15:26.602 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:26.651 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:26.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:26.652 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:26.753 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:26 smithi067 ceph-mon[33511]: 3.1b scrub starts 2024-09-20T22:15:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:26 smithi067 ceph-mon[33511]: 3.1b scrub ok 2024-09-20T22:15:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:26 smithi067 ceph-mon[33511]: 1.0 scrub starts 2024-09-20T22:15:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:26 smithi067 ceph-mon[33511]: 1.0 scrub ok 2024-09-20T22:15:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:26 smithi067 ceph-mon[33511]: 5.18 scrub starts 2024-09-20T22:15:26.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:26 smithi067 ceph-mon[33511]: 5.18 scrub ok 2024-09-20T22:15:26.790 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:26.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:26.791 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:26.892 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:26.913 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:26.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:26.914 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:26.916 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 9 16 24372 24356 169.111 124.438 0.00283136 0.00585622 2024-09-20T22:15:27.015 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:27.050 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:27.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:27.051 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:27.152 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:27.173 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:27.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:27.174 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:27.275 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:27.297 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:27.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:27.298 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:27.399 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:27.429 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:27.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:27.431 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:27.531 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:27.570 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:27.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:27.571 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:27.593 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:15:27.593 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:15:27.593 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:15:27.593 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:15:27.593 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:15:27.593 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:15:27.593 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:15:27.593 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:15:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:15:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:15:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:15:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:15:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:15:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:15:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:15:27.594 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:15:27.672 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:27.705 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:27.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:27.706 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:27.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:27 smithi067 ceph-mon[33511]: 4.b scrub starts 2024-09-20T22:15:27.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:27 smithi067 ceph-mon[33511]: 4.b scrub ok 2024-09-20T22:15:27.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:27 smithi067 ceph-mon[33511]: 6.b scrub starts 2024-09-20T22:15:27.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:27 smithi067 ceph-mon[33511]: 6.b scrub ok 2024-09-20T22:15:27.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:27 smithi067 ceph-mon[33511]: pgmap v671: 158 pgs: 158 active+clean; 1.0 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 130 MiB/s wr, 2.08k op/s 2024-09-20T22:15:27.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:27 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2793740388' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:27.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:27 smithi067 ceph-mon[33511]: 3.9 scrub starts 2024-09-20T22:15:27.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:27 smithi067 ceph-mon[33511]: 3.9 scrub ok 2024-09-20T22:15:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:27 smithi060 ceph-mon[29694]: 4.b scrub starts 2024-09-20T22:15:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:27 smithi060 ceph-mon[29694]: 4.b scrub ok 2024-09-20T22:15:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:27 smithi060 ceph-mon[29694]: 6.b scrub starts 2024-09-20T22:15:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:27 smithi060 ceph-mon[29694]: 6.b scrub ok 2024-09-20T22:15:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:27 smithi060 ceph-mon[29694]: pgmap v671: 158 pgs: 158 active+clean; 1.0 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 130 MiB/s wr, 2.08k op/s 2024-09-20T22:15:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:27 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2793740388' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:27 smithi060 ceph-mon[29694]: 3.9 scrub starts 2024-09-20T22:15:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:27 smithi060 ceph-mon[29694]: 3.9 scrub ok 2024-09-20T22:15:27.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:27 smithi060 ceph-mon[40290]: 4.b scrub starts 2024-09-20T22:15:27.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:27 smithi060 ceph-mon[40290]: 4.b scrub ok 2024-09-20T22:15:27.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:27 smithi060 ceph-mon[40290]: 6.b scrub starts 2024-09-20T22:15:27.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:27 smithi060 ceph-mon[40290]: 6.b scrub ok 2024-09-20T22:15:27.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:27 smithi060 ceph-mon[40290]: pgmap v671: 158 pgs: 158 active+clean; 1.0 GiB data, 7.0 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 130 MiB/s wr, 2.08k op/s 2024-09-20T22:15:27.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:27 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2793740388' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:27.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:27 smithi060 ceph-mon[40290]: 3.9 scrub starts 2024-09-20T22:15:27.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:27 smithi060 ceph-mon[40290]: 3.9 scrub ok 2024-09-20T22:15:27.806 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:27.843 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:27.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:27.845 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:27.916 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 10 16 26125 26109 163.155 109.562 0.00975275 0.00612386 2024-09-20T22:15:27.946 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:27.967 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:27.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:27.968 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:28.068 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:28.090 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:28.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:28.091 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:28.192 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:28.214 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:28.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:28.215 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:28.315 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:28.351 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:28.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:28.352 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:28.453 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:28.489 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:28.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:28.490 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:28.591 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:28.613 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:28.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:28.614 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:28.715 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:28.750 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:28.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:28.751 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:28 smithi067 ceph-mon[33511]: 4.6 scrub starts 2024-09-20T22:15:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:28 smithi067 ceph-mon[33511]: 4.6 scrub ok 2024-09-20T22:15:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:28 smithi067 ceph-mon[33511]: 1.4 scrub starts 2024-09-20T22:15:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:28 smithi067 ceph-mon[33511]: 1.4 scrub ok 2024-09-20T22:15:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:28 smithi067 ceph-mon[33511]: 3.16 scrub starts 2024-09-20T22:15:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:28 smithi067 ceph-mon[33511]: 3.16 scrub ok 2024-09-20T22:15:28.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:28 smithi067 ceph-mon[33511]: 5.8 scrub starts 2024-09-20T22:15:28.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:28 smithi067 ceph-mon[33511]: 5.8 scrub ok 2024-09-20T22:15:28.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:28 smithi060 ceph-mon[29694]: 4.6 scrub starts 2024-09-20T22:15:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:28 smithi060 ceph-mon[29694]: 4.6 scrub ok 2024-09-20T22:15:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:28 smithi060 ceph-mon[29694]: 1.4 scrub starts 2024-09-20T22:15:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:28 smithi060 ceph-mon[29694]: 1.4 scrub ok 2024-09-20T22:15:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:28 smithi060 ceph-mon[29694]: 3.16 scrub starts 2024-09-20T22:15:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:28 smithi060 ceph-mon[29694]: 3.16 scrub ok 2024-09-20T22:15:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:28 smithi060 ceph-mon[29694]: 5.8 scrub starts 2024-09-20T22:15:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:28 smithi060 ceph-mon[29694]: 5.8 scrub ok 2024-09-20T22:15:28.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:28 smithi060 ceph-mon[40290]: 4.6 scrub starts 2024-09-20T22:15:28.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:28 smithi060 ceph-mon[40290]: 4.6 scrub ok 2024-09-20T22:15:28.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:28 smithi060 ceph-mon[40290]: 1.4 scrub starts 2024-09-20T22:15:28.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:28 smithi060 ceph-mon[40290]: 1.4 scrub ok 2024-09-20T22:15:28.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:28 smithi060 ceph-mon[40290]: 3.16 scrub starts 2024-09-20T22:15:28.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:28 smithi060 ceph-mon[40290]: 3.16 scrub ok 2024-09-20T22:15:28.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:28 smithi060 ceph-mon[40290]: 5.8 scrub starts 2024-09-20T22:15:28.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:28 smithi060 ceph-mon[40290]: 5.8 scrub ok 2024-09-20T22:15:28.852 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:28.889 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:28.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:28.890 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:28.916 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 11 16 27439 27423 155.789 82.125 0.0359114 0.00640012 2024-09-20T22:15:28.991 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:29.026 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:29.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:29.027 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:29.128 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:29.150 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:29.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:29.151 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:29.251 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:29.290 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:29.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:29.291 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:29.392 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:29.429 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:29.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:29.431 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:29.531 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:29.571 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:29.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:29.572 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:29.674 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:29.695 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:29.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:29.696 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:29.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:29 smithi067 ceph-mon[33511]: 5.1e scrub starts 2024-09-20T22:15:29.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:29 smithi067 ceph-mon[33511]: 5.1e scrub ok 2024-09-20T22:15:29.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:29 smithi067 ceph-mon[33511]: 3.8 deep-scrub starts 2024-09-20T22:15:29.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:29 smithi067 ceph-mon[33511]: 3.8 deep-scrub ok 2024-09-20T22:15:29.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:29 smithi067 ceph-mon[33511]: pgmap v672: 158 pgs: 158 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 185 MiB/s wr, 2.96k op/s 2024-09-20T22:15:29.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:29 smithi067 ceph-mon[33511]: 3.11 scrub starts 2024-09-20T22:15:29.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:29 smithi067 ceph-mon[33511]: 3.11 scrub ok 2024-09-20T22:15:29.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:29 smithi067 ceph-mon[33511]: 5.1f scrub starts 2024-09-20T22:15:29.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:29 smithi067 ceph-mon[33511]: 5.1f scrub ok 2024-09-20T22:15:29.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:29 smithi060 ceph-mon[40290]: 5.1e scrub starts 2024-09-20T22:15:29.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:29 smithi060 ceph-mon[40290]: 5.1e scrub ok 2024-09-20T22:15:29.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:29 smithi060 ceph-mon[40290]: 3.8 deep-scrub starts 2024-09-20T22:15:29.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:29 smithi060 ceph-mon[40290]: 3.8 deep-scrub ok 2024-09-20T22:15:29.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:29 smithi060 ceph-mon[40290]: pgmap v672: 158 pgs: 158 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 185 MiB/s wr, 2.96k op/s 2024-09-20T22:15:29.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:29 smithi060 ceph-mon[40290]: 3.11 scrub starts 2024-09-20T22:15:29.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:29 smithi060 ceph-mon[40290]: 3.11 scrub ok 2024-09-20T22:15:29.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:29 smithi060 ceph-mon[40290]: 5.1f scrub starts 2024-09-20T22:15:29.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:29 smithi060 ceph-mon[40290]: 5.1f scrub ok 2024-09-20T22:15:29.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:29 smithi060 ceph-mon[29694]: 5.1e scrub starts 2024-09-20T22:15:29.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:29 smithi060 ceph-mon[29694]: 5.1e scrub ok 2024-09-20T22:15:29.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:29 smithi060 ceph-mon[29694]: 3.8 deep-scrub starts 2024-09-20T22:15:29.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:29 smithi060 ceph-mon[29694]: 3.8 deep-scrub ok 2024-09-20T22:15:29.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:29 smithi060 ceph-mon[29694]: pgmap v672: 158 pgs: 158 active+clean; 1.5 GiB data, 8.3 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 185 MiB/s wr, 2.96k op/s 2024-09-20T22:15:29.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:29 smithi060 ceph-mon[29694]: 3.11 scrub starts 2024-09-20T22:15:29.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:29 smithi060 ceph-mon[29694]: 3.11 scrub ok 2024-09-20T22:15:29.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:29 smithi060 ceph-mon[29694]: 5.1f scrub starts 2024-09-20T22:15:29.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:29 smithi060 ceph-mon[29694]: 5.1f scrub ok 2024-09-20T22:15:29.796 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:29.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:29.831 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:29.832 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:29.847 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:29.916 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 12 16 29240 29224 152.185 112.562 0.00479635 0.00656526 2024-09-20T22:15:29.932 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:29.970 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:29.970 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:29.971 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:30.072 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:30.106 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:30.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:30.107 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:30.208 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:30.244 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:30.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:30.245 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:30.346 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:30.381 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:30.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:30.382 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:30.483 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:30.513 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:30.532 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:30.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:30.533 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:30.633 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:30.668 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:30 smithi060 ceph-mon[29694]: 6.12 scrub starts 2024-09-20T22:15:30.668 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:30 smithi060 ceph-mon[29694]: 6.12 scrub ok 2024-09-20T22:15:30.668 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:30 smithi060 ceph-mon[29694]: 6.17 deep-scrub starts 2024-09-20T22:15:30.668 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:30 smithi060 ceph-mon[29694]: 6.17 deep-scrub ok 2024-09-20T22:15:30.668 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:30 smithi060 ceph-mon[40290]: 6.12 scrub starts 2024-09-20T22:15:30.668 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:30 smithi060 ceph-mon[40290]: 6.12 scrub ok 2024-09-20T22:15:30.668 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:30 smithi060 ceph-mon[40290]: 6.17 deep-scrub starts 2024-09-20T22:15:30.668 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:30 smithi060 ceph-mon[40290]: 6.17 deep-scrub ok 2024-09-20T22:15:30.670 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:30.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:30.671 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:30 smithi067 ceph-mon[33511]: 6.12 scrub starts 2024-09-20T22:15:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:30 smithi067 ceph-mon[33511]: 6.12 scrub ok 2024-09-20T22:15:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:30 smithi067 ceph-mon[33511]: 6.17 deep-scrub starts 2024-09-20T22:15:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:30 smithi067 ceph-mon[33511]: 6.17 deep-scrub ok 2024-09-20T22:15:30.772 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:30.832 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:30.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:30.833 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:30.916 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 13 16 31433 31417 151.021 137.062 0.00513975 0.00661655 2024-09-20T22:15:30.934 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:30.970 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:30.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:30.972 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:31.072 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:31.113 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:31.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:31.114 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:31.214 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:31.255 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:31.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:31.256 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:31.327 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:15:31.328 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:15:31.357 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:31.388 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:31.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:31.390 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:31.490 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:31.528 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:31.529 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:31.529 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:31.530 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:31 smithi060 ceph-mon[40290]: pgmap v673: 158 pgs: 158 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 143 MiB/s wr, 2.29k op/s 2024-09-20T22:15:31.530 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:31 smithi060 ceph-mon[40290]: 4.1b scrub starts 2024-09-20T22:15:31.530 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:31 smithi060 ceph-mon[40290]: 4.1b scrub ok 2024-09-20T22:15:31.530 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:31 smithi060 ceph-mon[29694]: pgmap v673: 158 pgs: 158 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 143 MiB/s wr, 2.29k op/s 2024-09-20T22:15:31.530 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:31 smithi060 ceph-mon[29694]: 4.1b scrub starts 2024-09-20T22:15:31.530 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:31 smithi060 ceph-mon[29694]: 4.1b scrub ok 2024-09-20T22:15:31.629 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:31.668 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:31.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:31.668 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:31.758 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:31.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:31 smithi067 ceph-mon[33511]: pgmap v673: 158 pgs: 158 active+clean; 1.5 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 143 MiB/s wr, 2.29k op/s 2024-09-20T22:15:31.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:31 smithi067 ceph-mon[33511]: 4.1b scrub starts 2024-09-20T22:15:31.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:31 smithi067 ceph-mon[33511]: 4.1b scrub ok 2024-09-20T22:15:31.769 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:31.804 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:31.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:31.805 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:31.906 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:31.917 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 14 16 33410 33394 149.059 123.562 0.0096949 0.00670278 2024-09-20T22:15:31.934 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:15:31.937 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:15:31.940 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:31.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:31.941 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:32.042 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:32.064 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:32.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:32.064 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:32.165 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:32.197 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T22:15:32.219 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:32.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:32.220 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:32.321 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:32.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:32.364 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:32.364 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:32.465 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:32.488 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:32 smithi067 ceph-mon[33511]: 3.b scrub starts 2024-09-20T22:15:32.488 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:32 smithi067 ceph-mon[33511]: 3.b scrub ok 2024-09-20T22:15:32.488 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:32 smithi067 ceph-mon[33511]: 4.a scrub starts 2024-09-20T22:15:32.488 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:32 smithi067 ceph-mon[33511]: 4.a scrub ok 2024-09-20T22:15:32.489 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:15:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:15:32.502 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:15:32.509 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:32.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:32.510 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:32.611 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:32.641 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:15:32.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:32.648 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:32.648 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:32.749 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:32 smithi060 ceph-mon[29694]: 3.b scrub starts 2024-09-20T22:15:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:32 smithi060 ceph-mon[29694]: 3.b scrub ok 2024-09-20T22:15:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:32 smithi060 ceph-mon[29694]: 4.a scrub starts 2024-09-20T22:15:32.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:32 smithi060 ceph-mon[29694]: 4.a scrub ok 2024-09-20T22:15:32.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:32 smithi060 ceph-mon[40290]: 3.b scrub starts 2024-09-20T22:15:32.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:32 smithi060 ceph-mon[40290]: 3.b scrub ok 2024-09-20T22:15:32.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:32 smithi060 ceph-mon[40290]: 4.a scrub starts 2024-09-20T22:15:32.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:32 smithi060 ceph-mon[40290]: 4.a scrub ok 2024-09-20T22:15:32.792 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:32.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:32.793 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:32.894 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:32.917 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 15 16 35089 35073 146.111 104.938 0.00383806 0.00683786 2024-09-20T22:15:32.931 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:32.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:32.932 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:33.004 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":673,"stamp":"2024-09-20T22:15:30.219274+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1601116699,"num_objects":24621,"num_object_clones":0,"num_object_copies":73863,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24621,"num_whiteouts":0,"num_read":1866,"num_read_kb":1618,"num_write":25144,"num_write_kb":1563053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":923923,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26661,"ondisk_log_size":26661,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":8843612,"kb_used_data":4384988,"kb_used_omap":15,"kb_used_meta":4458480,"kb_avail":741117604,"statfs":{"total":767960285184,"available":758904426496,"internally_reserved":0,"allocated":4490227712,"data_stored":4487469377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":4565484307},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,5,6,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1202520064,"num_objects":18349,"num_object_clones":0,"num_object_copies":55047,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18349,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":18349,"num_write_kb":1174336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002693"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":128,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569482+0000","last_change":"2024-09-20T22:14:57.573764+0000","last_active":"2024-09-20T22:15:23.569482+0000","last_peered":"2024-09-20T22:15:23.569482+0000","last_clean":"2024-09-20T22:15:23.569482+0000","last_became_active":"2024-09-20T22:12:52.515921+0000","last_became_peered":"2024-09-20T22:12:52.515921+0000","last_unstale":"2024-09-20T22:15:23.569482+0000","last_undegraded":"2024-09-20T22:15:23.569482+0000","last_fullsized":"2024-09-20T22:15:23.569482+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:57.573679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_clean_scrub_stamp":"2024-09-20T22:14:57.573679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:20.753739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"327'19","reported_seq":718,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560653+0000","last_change":"2024-09-20T22:15:22.984401+0000","last_active":"2024-09-20T22:15:23.560653+0000","last_peered":"2024-09-20T22:15:23.560653+0000","last_clean":"2024-09-20T22:15:23.560653+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:15:23.560653+0000","last_undegraded":"2024-09-20T22:15:23.560653+0000","last_fullsized":"2024-09-20T22:15:23.560653+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"327'19","last_scrub_stamp":"2024-09-20T22:15:22.984271+0000","last_deep_scrub":"252'13","last_deep_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_clean_scrub_stamp":"2024-09-20T22:15:22.984271+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:25.020372+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":563,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559455+0000","last_change":"2024-09-20T22:14:15.391149+0000","last_active":"2024-09-20T22:15:23.559455+0000","last_peered":"2024-09-20T22:15:23.559455+0000","last_clean":"2024-09-20T22:15:23.559455+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:15:23.559455+0000","last_undegraded":"2024-09-20T22:15:23.559455+0000","last_fullsized":"2024-09-20T22:15:23.559455+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:15.391049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:15.391049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:37.227878+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":684,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561074+0000","last_change":"2024-09-20T22:15:18.062035+0000","last_active":"2024-09-20T22:15:23.561074+0000","last_peered":"2024-09-20T22:15:23.561074+0000","last_clean":"2024-09-20T22:15:23.561074+0000","last_became_active":"2024-09-20T22:11:30.485735+0000","last_became_peered":"2024-09-20T22:11:30.485735+0000","last_unstale":"2024-09-20T22:15:23.561074+0000","last_undegraded":"2024-09-20T22:15:23.561074+0000","last_fullsized":"2024-09-20T22:15:23.561074+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:18.061924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:18.061924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:19.499592+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":672,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.564130+0000","last_change":"2024-09-20T22:14:56.160729+0000","last_active":"2024-09-20T22:15:23.564130+0000","last_peered":"2024-09-20T22:15:23.564130+0000","last_clean":"2024-09-20T22:15:23.564130+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:15:23.564130+0000","last_undegraded":"2024-09-20T22:15:23.564130+0000","last_fullsized":"2024-09-20T22:15:23.564130+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:14:56.160638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:56.160638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:07.044469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":155,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565418+0000","last_change":"2024-09-20T22:14:52.402426+0000","last_active":"2024-09-20T22:15:23.565418+0000","last_peered":"2024-09-20T22:15:23.565418+0000","last_clean":"2024-09-20T22:15:23.565418+0000","last_became_active":"2024-09-20T22:12:33.545100+0000","last_became_peered":"2024-09-20T22:12:33.545100+0000","last_unstale":"2024-09-20T22:15:23.565418+0000","last_undegraded":"2024-09-20T22:15:23.565418+0000","last_fullsized":"2024-09-20T22:15:23.565418+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:52.402299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:52.402299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:13.318752+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"328'27","reported_seq":271,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565153+0000","last_change":"2024-09-20T22:15:20.902301+0000","last_active":"2024-09-20T22:15:23.565153+0000","last_peered":"2024-09-20T22:15:23.565153+0000","last_clean":"2024-09-20T22:15:23.565153+0000","last_became_active":"2024-09-20T22:12:52.517296+0000","last_became_peered":"2024-09-20T22:12:52.517296+0000","last_unstale":"2024-09-20T22:15:23.565153+0000","last_undegraded":"2024-09-20T22:15:23.565153+0000","last_fullsized":"2024-09-20T22:15:23.565153+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:15:20.902179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.902179+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:24.675476+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":669,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560556+0000","last_change":"2024-09-20T22:13:06.156767+0000","last_active":"2024-09-20T22:15:23.560556+0000","last_peered":"2024-09-20T22:15:23.560556+0000","last_clean":"2024-09-20T22:15:23.560556+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:15:23.560556+0000","last_undegraded":"2024-09-20T22:15:23.560556+0000","last_fullsized":"2024-09-20T22:15:23.560556+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:06.156656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:06.156656+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":699,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566268+0000","last_change":"2024-09-20T22:15:21.178920+0000","last_active":"2024-09-20T22:15:23.566268+0000","last_peered":"2024-09-20T22:15:23.566268+0000","last_clean":"2024-09-20T22:15:23.566268+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:15:23.566268+0000","last_undegraded":"2024-09-20T22:15:23.566268+0000","last_fullsized":"2024-09-20T22:15:23.566268+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:21.178806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:15:21.178806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:36.671122+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"328'15","reported_seq":737,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561282+0000","last_change":"2024-09-20T22:14:19.076811+0000","last_active":"2024-09-20T22:15:23.561282+0000","last_peered":"2024-09-20T22:15:23.561282+0000","last_clean":"2024-09-20T22:15:23.561282+0000","last_became_active":"2024-09-20T22:12:52.516894+0000","last_became_peered":"2024-09-20T22:12:52.516894+0000","last_unstale":"2024-09-20T22:15:23.561282+0000","last_undegraded":"2024-09-20T22:15:23.561282+0000","last_fullsized":"2024-09-20T22:15:23.561282+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:14:19.076696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:19.076696+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:45.728095+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":230,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566256+0000","last_change":"2024-09-20T22:14:54.153118+0000","last_active":"2024-09-20T22:15:23.566256+0000","last_peered":"2024-09-20T22:15:23.566256+0000","last_clean":"2024-09-20T22:15:23.566256+0000","last_became_active":"2024-09-20T22:12:52.518549+0000","last_became_peered":"2024-09-20T22:12:52.518549+0000","last_unstale":"2024-09-20T22:15:23.566256+0000","last_undegraded":"2024-09-20T22:15:23.566256+0000","last_fullsized":"2024-09-20T22:15:23.566256+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:54.152717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:54.152717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:03.431527+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":136,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569479+0000","last_change":"2024-09-20T22:14:58.570997+0000","last_active":"2024-09-20T22:15:23.569479+0000","last_peered":"2024-09-20T22:15:23.569479+0000","last_clean":"2024-09-20T22:15:23.569479+0000","last_became_active":"2024-09-20T22:12:52.524631+0000","last_became_peered":"2024-09-20T22:12:52.524631+0000","last_unstale":"2024-09-20T22:15:23.569479+0000","last_undegraded":"2024-09-20T22:15:23.569479+0000","last_fullsized":"2024-09-20T22:15:23.569479+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:58.570923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:58.570923+0000","last_clean_scrub_stamp":"2024-09-20T22:14:58.570923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:25.107938+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":704,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566051+0000","last_change":"2024-09-20T22:15:20.182058+0000","last_active":"2024-09-20T22:15:23.566051+0000","last_peered":"2024-09-20T22:15:23.566051+0000","last_clean":"2024-09-20T22:15:23.566051+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:15:23.566051+0000","last_undegraded":"2024-09-20T22:15:23.566051+0000","last_fullsized":"2024-09-20T22:15:23.566051+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.181942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:30.215263+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.181942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:46.935103+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"328'7","reported_seq":706,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561504+0000","last_change":"2024-09-20T22:13:44.057396+0000","last_active":"2024-09-20T22:15:23.561504+0000","last_peered":"2024-09-20T22:15:23.561504+0000","last_clean":"2024-09-20T22:15:23.561504+0000","last_became_active":"2024-09-20T22:08:22.144364+0000","last_became_peered":"2024-09-20T22:08:22.144364+0000","last_unstale":"2024-09-20T22:15:23.561504+0000","last_undegraded":"2024-09-20T22:15:23.561504+0000","last_fullsized":"2024-09-20T22:15:23.561504+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'7","last_scrub_stamp":"2024-09-20T22:13:44.057245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:44.057245+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":685,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566796+0000","last_change":"2024-09-20T22:14:57.354434+0000","last_active":"2024-09-20T22:15:23.566796+0000","last_peered":"2024-09-20T22:15:23.566796+0000","last_clean":"2024-09-20T22:15:23.566796+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:15:23.566796+0000","last_undegraded":"2024-09-20T22:15:23.566796+0000","last_fullsized":"2024-09-20T22:15:23.566796+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:57.354326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:57.354326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:14.925355+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":218,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.564678+0000","last_change":"2024-09-20T22:15:00.077644+0000","last_active":"2024-09-20T22:15:23.564678+0000","last_peered":"2024-09-20T22:15:23.564678+0000","last_clean":"2024-09-20T22:15:23.564678+0000","last_became_active":"2024-09-20T22:11:30.487137+0000","last_became_peered":"2024-09-20T22:11:30.487137+0000","last_unstale":"2024-09-20T22:15:23.564678+0000","last_undegraded":"2024-09-20T22:15:23.564678+0000","last_fullsized":"2024-09-20T22:15:23.564678+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:00.077519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:15:00.077519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:11.908799+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":673,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.562352+0000","last_change":"2024-09-20T22:15:17.029127+0000","last_active":"2024-09-20T22:15:23.562352+0000","last_peered":"2024-09-20T22:15:23.562352+0000","last_clean":"2024-09-20T22:15:23.562352+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:15:23.562352+0000","last_undegraded":"2024-09-20T22:15:23.562352+0000","last_fullsized":"2024-09-20T22:15:23.562352+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:17.029051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:15:17.029051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:29.616781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":136,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:24.747969+0000","last_change":"2024-09-20T22:15:24.748006+0000","last_active":"2024-09-20T22:15:24.747969+0000","last_peered":"2024-09-20T22:15:24.747969+0000","last_clean":"2024-09-20T22:15:24.747969+0000","last_became_active":"2024-09-20T22:12:52.526097+0000","last_became_peered":"2024-09-20T22:12:52.526097+0000","last_unstale":"2024-09-20T22:15:24.747969+0000","last_undegraded":"2024-09-20T22:15:24.747969+0000","last_fullsized":"2024-09-20T22:15:24.747969+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.747891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.747891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:29.188832+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"328'34","reported_seq":274,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559012+0000","last_change":"2024-09-20T22:14:50.993713+0000","last_active":"2024-09-20T22:15:23.559012+0000","last_peered":"2024-09-20T22:15:23.559012+0000","last_clean":"2024-09-20T22:15:23.559012+0000","last_became_active":"2024-09-20T22:11:12.507311+0000","last_became_peered":"2024-09-20T22:11:12.507311+0000","last_unstale":"2024-09-20T22:15:23.559012+0000","last_undegraded":"2024-09-20T22:15:23.559012+0000","last_fullsized":"2024-09-20T22:15:23.559012+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'34","last_scrub_stamp":"2024-09-20T22:14:50.993563+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:14:50.993563+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:53.748023+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":525,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560601+0000","last_change":"2024-09-20T22:13:11.053381+0000","last_active":"2024-09-20T22:15:23.560601+0000","last_peered":"2024-09-20T22:15:23.560601+0000","last_clean":"2024-09-20T22:15:23.560601+0000","last_became_active":"2024-09-20T22:12:52.520985+0000","last_became_peered":"2024-09-20T22:12:52.520985+0000","last_unstale":"2024-09-20T22:15:23.560601+0000","last_undegraded":"2024-09-20T22:15:23.560601+0000","last_fullsized":"2024-09-20T22:15:23.560601+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:11.053276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:11.053276+0000","last_clean_scrub_stamp":"2024-09-20T22:13:11.053276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":624,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566012+0000","last_change":"2024-09-20T22:14:19.451519+0000","last_active":"2024-09-20T22:15:23.566012+0000","last_peered":"2024-09-20T22:15:23.566012+0000","last_clean":"2024-09-20T22:15:23.566012+0000","last_became_active":"2024-09-20T22:12:21.521433+0000","last_became_peered":"2024-09-20T22:12:21.521433+0000","last_unstale":"2024-09-20T22:15:23.566012+0000","last_undegraded":"2024-09-20T22:15:23.566012+0000","last_fullsized":"2024-09-20T22:15:23.566012+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:19.451409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:19.451409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:42.318241+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":625,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565102+0000","last_change":"2024-09-20T22:12:44.197752+0000","last_active":"2024-09-20T22:15:23.565102+0000","last_peered":"2024-09-20T22:15:23.565102+0000","last_clean":"2024-09-20T22:15:23.565102+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:15:23.565102+0000","last_undegraded":"2024-09-20T22:15:23.565102+0000","last_fullsized":"2024-09-20T22:15:23.565102+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:44.197654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:44.197654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"328'23","reported_seq":598,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560790+0000","last_change":"2024-09-20T22:15:17.533607+0000","last_active":"2024-09-20T22:15:23.560790+0000","last_peered":"2024-09-20T22:15:23.560790+0000","last_clean":"2024-09-20T22:15:23.560790+0000","last_became_active":"2024-09-20T22:12:21.517147+0000","last_became_peered":"2024-09-20T22:12:21.517147+0000","last_unstale":"2024-09-20T22:15:23.560790+0000","last_undegraded":"2024-09-20T22:15:23.560790+0000","last_fullsized":"2024-09-20T22:15:23.560790+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'23","last_scrub_stamp":"2024-09-20T22:15:17.533453+0000","last_deep_scrub":"250'17","last_deep_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_clean_scrub_stamp":"2024-09-20T22:15:17.533453+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:38.348665+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":668,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561565+0000","last_change":"2024-09-20T22:13:51.133410+0000","last_active":"2024-09-20T22:15:23.561565+0000","last_peered":"2024-09-20T22:15:23.561565+0000","last_clean":"2024-09-20T22:15:23.561565+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:15:23.561565+0000","last_undegraded":"2024-09-20T22:15:23.561565+0000","last_fullsized":"2024-09-20T22:15:23.561565+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:51.133308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.133308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":666,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566888+0000","last_change":"2024-09-20T22:13:52.688758+0000","last_active":"2024-09-20T22:15:23.566888+0000","last_peered":"2024-09-20T22:15:23.566888+0000","last_clean":"2024-09-20T22:15:23.566888+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:15:23.566888+0000","last_undegraded":"2024-09-20T22:15:23.566888+0000","last_fullsized":"2024-09-20T22:15:23.566888+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:13:52.688650+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:13:52.688650+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"289'2","reported_seq":523,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559543+0000","last_change":"2024-09-20T22:14:22.436454+0000","last_active":"2024-09-20T22:15:23.559543+0000","last_peered":"2024-09-20T22:15:23.559543+0000","last_clean":"2024-09-20T22:15:23.559543+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:15:23.559543+0000","last_undegraded":"2024-09-20T22:15:23.559543+0000","last_fullsized":"2024-09-20T22:15:23.559543+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-20T22:14:22.436308+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_clean_scrub_stamp":"2024-09-20T22:14:22.436308+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:51.653333+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"328'20","reported_seq":155,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.570263+0000","last_change":"2024-09-20T22:14:11.444573+0000","last_active":"2024-09-20T22:15:23.570263+0000","last_peered":"2024-09-20T22:15:23.570263+0000","last_clean":"2024-09-20T22:15:23.570263+0000","last_became_active":"2024-09-20T22:12:52.526803+0000","last_became_peered":"2024-09-20T22:12:52.526803+0000","last_unstale":"2024-09-20T22:15:23.570263+0000","last_undegraded":"2024-09-20T22:15:23.570263+0000","last_fullsized":"2024-09-20T22:15:23.570263+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:14:11.444444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:11.444444+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.306318+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"291'11","reported_seq":868,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566288+0000","last_change":"2024-09-20T22:13:53.376728+0000","last_active":"2024-09-20T22:15:23.566288+0000","last_peered":"2024-09-20T22:15:23.566288+0000","last_clean":"2024-09-20T22:15:23.566288+0000","last_became_active":"2024-09-20T22:11:40.601048+0000","last_became_peered":"2024-09-20T22:11:40.601048+0000","last_unstale":"2024-09-20T22:15:23.566288+0000","last_undegraded":"2024-09-20T22:15:23.566288+0000","last_fullsized":"2024-09-20T22:15:23.566288+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'11","last_scrub_stamp":"2024-09-20T22:13:53.376616+0000","last_deep_scrub":"291'11","last_deep_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_clean_scrub_stamp":"2024-09-20T22:13:53.376616+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"378'1308","reported_seq":1334,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:25.765950+0000","last_change":"2024-09-20T22:15:08.927837+0000","last_active":"2024-09-20T22:15:25.765950+0000","last_peered":"2024-09-20T22:15:25.765950+0000","last_clean":"2024-09-20T22:15:25.765950+0000","last_became_active":"2024-09-20T22:15:08.927535+0000","last_became_peered":"2024-09-20T22:15:08.927535+0000","last_unstale":"2024-09-20T22:15:25.765950+0000","last_undegraded":"2024-09-20T22:15:25.765950+0000","last_fullsized":"2024-09-20T22:15:25.765950+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1308,"log_dups_size":0,"ondisk_log_size":1308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:17.347573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85721088,"num_objects":1308,"num_object_clones":0,"num_object_copies":3924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1308,"num_write_kb":83712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":680,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.564855+0000","last_change":"2024-09-20T22:13:08.407055+0000","last_active":"2024-09-20T22:15:23.564855+0000","last_peered":"2024-09-20T22:15:23.564855+0000","last_clean":"2024-09-20T22:15:23.564855+0000","last_became_active":"2024-09-20T22:08:48.452786+0000","last_became_peered":"2024-09-20T22:08:48.452786+0000","last_unstale":"2024-09-20T22:15:23.564855+0000","last_undegraded":"2024-09-20T22:15:23.564855+0000","last_fullsized":"2024-09-20T22:15:23.564855+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:08.406883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:13:08.406883+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"328'27","reported_seq":468,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559731+0000","last_change":"2024-09-20T22:14:46.998282+0000","last_active":"2024-09-20T22:15:23.559731+0000","last_peered":"2024-09-20T22:15:23.559731+0000","last_clean":"2024-09-20T22:15:23.559731+0000","last_became_active":"2024-09-20T22:08:22.145408+0000","last_became_peered":"2024-09-20T22:08:22.145408+0000","last_unstale":"2024-09-20T22:15:23.559731+0000","last_undegraded":"2024-09-20T22:15:23.559731+0000","last_fullsized":"2024-09-20T22:15:23.559731+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:14:46.998142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:46.998142+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:49.977452+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":684,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561145+0000","last_change":"2024-09-20T22:13:53.012457+0000","last_active":"2024-09-20T22:15:23.561145+0000","last_peered":"2024-09-20T22:15:23.561145+0000","last_clean":"2024-09-20T22:15:23.561145+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:15:23.561145+0000","last_undegraded":"2024-09-20T22:15:23.561145+0000","last_fullsized":"2024-09-20T22:15:23.561145+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:53.012225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:53.012225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"326'13","reported_seq":897,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.562511+0000","last_change":"2024-09-20T22:14:50.079897+0000","last_active":"2024-09-20T22:15:23.562511+0000","last_peered":"2024-09-20T22:15:23.562511+0000","last_clean":"2024-09-20T22:15:23.562511+0000","last_became_active":"2024-09-20T22:12:52.517588+0000","last_became_peered":"2024-09-20T22:12:52.517588+0000","last_unstale":"2024-09-20T22:15:23.562511+0000","last_undegraded":"2024-09-20T22:15:23.562511+0000","last_fullsized":"2024-09-20T22:15:23.562511+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'13","last_scrub_stamp":"2024-09-20T22:14:50.079818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:50.079818+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:57.638038+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":222,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566028+0000","last_change":"2024-09-20T22:12:52.518570+0000","last_active":"2024-09-20T22:15:23.566028+0000","last_peered":"2024-09-20T22:15:23.566028+0000","last_clean":"2024-09-20T22:15:23.566028+0000","last_became_active":"2024-09-20T22:12:52.517826+0000","last_became_peered":"2024-09-20T22:12:52.517826+0000","last_unstale":"2024-09-20T22:15:23.566028+0000","last_undegraded":"2024-09-20T22:15:23.566028+0000","last_fullsized":"2024-09-20T22:15:23.566028+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:07.114531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:07.114531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"378'1399","reported_seq":1425,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:25.793608+0000","last_change":"2024-09-20T22:15:08.928365+0000","last_active":"2024-09-20T22:15:25.793608+0000","last_peered":"2024-09-20T22:15:25.793608+0000","last_clean":"2024-09-20T22:15:25.793608+0000","last_became_active":"2024-09-20T22:15:08.928064+0000","last_became_peered":"2024-09-20T22:15:08.928064+0000","last_unstale":"2024-09-20T22:15:25.793608+0000","last_undegraded":"2024-09-20T22:15:25.793608+0000","last_fullsized":"2024-09-20T22:15:25.793608+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1399,"log_dups_size":0,"ondisk_log_size":1399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:27.570950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":91684864,"num_objects":1399,"num_object_clones":0,"num_object_copies":4197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1399,"num_write_kb":89536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":680,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.562456+0000","last_change":"2024-09-20T22:14:21.059023+0000","last_active":"2024-09-20T22:15:23.562456+0000","last_peered":"2024-09-20T22:15:23.562456+0000","last_clean":"2024-09-20T22:15:23.562456+0000","last_became_active":"2024-09-20T22:08:48.452475+0000","last_became_peered":"2024-09-20T22:08:48.452475+0000","last_unstale":"2024-09-20T22:15:23.562456+0000","last_undegraded":"2024-09-20T22:15:23.562456+0000","last_fullsized":"2024-09-20T22:15:23.562456+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:14:21.058895+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:24.210483+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"328'30","reported_seq":688,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566824+0000","last_change":"2024-09-20T22:13:50.715007+0000","last_active":"2024-09-20T22:15:23.566824+0000","last_peered":"2024-09-20T22:15:23.566824+0000","last_clean":"2024-09-20T22:15:23.566824+0000","last_became_active":"2024-09-20T22:12:52.526867+0000","last_became_peered":"2024-09-20T22:12:52.526867+0000","last_unstale":"2024-09-20T22:15:23.566824+0000","last_undegraded":"2024-09-20T22:15:23.566824+0000","last_fullsized":"2024-09-20T22:15:23.566824+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'30","last_scrub_stamp":"2024-09-20T22:13:50.714867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:50.714867+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":169,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567840+0000","last_change":"2024-09-20T22:14:09.922166+0000","last_active":"2024-09-20T22:15:23.567840+0000","last_peered":"2024-09-20T22:15:23.567840+0000","last_clean":"2024-09-20T22:15:23.567840+0000","last_became_active":"2024-09-20T22:12:52.528947+0000","last_became_peered":"2024-09-20T22:12:52.528947+0000","last_unstale":"2024-09-20T22:15:23.567840+0000","last_undegraded":"2024-09-20T22:15:23.567840+0000","last_fullsized":"2024-09-20T22:15:23.567840+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:09.922092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:09.922092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:27.839312+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":689,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560650+0000","last_change":"2024-09-20T22:14:13.046284+0000","last_active":"2024-09-20T22:15:23.560650+0000","last_peered":"2024-09-20T22:15:23.560650+0000","last_clean":"2024-09-20T22:15:23.560650+0000","last_became_active":"2024-09-20T22:10:48.919949+0000","last_became_peered":"2024-09-20T22:10:48.919949+0000","last_unstale":"2024-09-20T22:15:23.560650+0000","last_undegraded":"2024-09-20T22:15:23.560650+0000","last_fullsized":"2024-09-20T22:15:23.560650+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:13.046189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:13.046189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":672,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561179+0000","last_change":"2024-09-20T22:14:08.158795+0000","last_active":"2024-09-20T22:15:23.561179+0000","last_peered":"2024-09-20T22:15:23.561179+0000","last_clean":"2024-09-20T22:15:23.561179+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:15:23.561179+0000","last_undegraded":"2024-09-20T22:15:23.561179+0000","last_fullsized":"2024-09-20T22:15:23.561179+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:08.158688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:08.158688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:25.401484+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"378'1629","reported_seq":1655,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:27.920980+0000","last_change":"2024-09-20T22:15:09.088949+0000","last_active":"2024-09-20T22:15:27.920980+0000","last_peered":"2024-09-20T22:15:27.920980+0000","last_clean":"2024-09-20T22:15:27.920980+0000","last_became_active":"2024-09-20T22:15:09.088580+0000","last_became_peered":"2024-09-20T22:15:09.088580+0000","last_unstale":"2024-09-20T22:15:27.920980+0000","last_undegraded":"2024-09-20T22:15:27.920980+0000","last_fullsized":"2024-09-20T22:15:27.920980+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1629,"log_dups_size":0,"ondisk_log_size":1629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:33.130074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106758144,"num_objects":1629,"num_object_clones":0,"num_object_copies":4887,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1629,"num_write_kb":104256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":706,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561674+0000","last_change":"2024-09-20T22:13:36.804562+0000","last_active":"2024-09-20T22:15:23.561674+0000","last_peered":"2024-09-20T22:15:23.561674+0000","last_clean":"2024-09-20T22:15:23.561674+0000","last_became_active":"2024-09-20T22:11:33.560632+0000","last_became_peered":"2024-09-20T22:11:33.560632+0000","last_unstale":"2024-09-20T22:15:23.561674+0000","last_undegraded":"2024-09-20T22:15:23.561674+0000","last_fullsized":"2024-09-20T22:15:23.561674+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:36.803555+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:13:36.803555+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"328'28","reported_seq":723,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561101+0000","last_change":"2024-09-20T22:14:55.064788+0000","last_active":"2024-09-20T22:15:23.561101+0000","last_peered":"2024-09-20T22:15:23.561101+0000","last_clean":"2024-09-20T22:15:23.561101+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:15:23.561101+0000","last_undegraded":"2024-09-20T22:15:23.561101+0000","last_fullsized":"2024-09-20T22:15:23.561101+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:14:55.064689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:55.064689+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:59.495396+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":140,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.570560+0000","last_change":"2024-09-20T22:15:15.781305+0000","last_active":"2024-09-20T22:15:23.570560+0000","last_peered":"2024-09-20T22:15:23.570560+0000","last_clean":"2024-09-20T22:15:23.570560+0000","last_became_active":"2024-09-20T22:12:52.528068+0000","last_became_peered":"2024-09-20T22:12:52.528068+0000","last_unstale":"2024-09-20T22:15:23.570560+0000","last_undegraded":"2024-09-20T22:15:23.570560+0000","last_fullsized":"2024-09-20T22:15:23.570560+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:15:15.781222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:15.781222+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:24.974390+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"315'11","reported_seq":408,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559119+0000","last_change":"2024-09-20T22:13:05.177108+0000","last_active":"2024-09-20T22:15:23.559119+0000","last_peered":"2024-09-20T22:15:23.559119+0000","last_clean":"2024-09-20T22:15:23.559119+0000","last_became_active":"2024-09-20T22:12:33.545942+0000","last_became_peered":"2024-09-20T22:12:33.545942+0000","last_unstale":"2024-09-20T22:15:23.559119+0000","last_undegraded":"2024-09-20T22:15:23.559119+0000","last_fullsized":"2024-09-20T22:15:23.559119+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:13:05.176973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:05.176973+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":665,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567148+0000","last_change":"2024-09-20T22:13:10.621055+0000","last_active":"2024-09-20T22:15:23.567148+0000","last_peered":"2024-09-20T22:15:23.567148+0000","last_clean":"2024-09-20T22:15:23.567148+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:15:23.567148+0000","last_undegraded":"2024-09-20T22:15:23.567148+0000","last_fullsized":"2024-09-20T22:15:23.567148+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:10.620925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.620925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"378'1315","reported_seq":1341,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:25.772954+0000","last_change":"2024-09-20T22:15:08.931965+0000","last_active":"2024-09-20T22:15:25.772954+0000","last_peered":"2024-09-20T22:15:25.772954+0000","last_clean":"2024-09-20T22:15:25.772954+0000","last_became_active":"2024-09-20T22:15:08.931653+0000","last_became_peered":"2024-09-20T22:15:08.931653+0000","last_unstale":"2024-09-20T22:15:25.772954+0000","last_undegraded":"2024-09-20T22:15:25.772954+0000","last_fullsized":"2024-09-20T22:15:25.772954+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1315,"log_dups_size":0,"ondisk_log_size":1315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:32.473356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86179840,"num_objects":1315,"num_object_clones":0,"num_object_copies":3945,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1315,"num_write_kb":84160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":710,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565917+0000","last_change":"2024-09-20T22:14:23.503489+0000","last_active":"2024-09-20T22:15:23.565917+0000","last_peered":"2024-09-20T22:15:23.565917+0000","last_clean":"2024-09-20T22:15:23.565917+0000","last_became_active":"2024-09-20T22:10:33.660948+0000","last_became_peered":"2024-09-20T22:10:33.660948+0000","last_unstale":"2024-09-20T22:15:23.565917+0000","last_undegraded":"2024-09-20T22:15:23.565917+0000","last_fullsized":"2024-09-20T22:15:23.565917+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:14:23.503360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:14:23.503360+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:52.579727+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"328'20","reported_seq":591,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560075+0000","last_change":"2024-09-20T22:13:41.421053+0000","last_active":"2024-09-20T22:15:23.560075+0000","last_peered":"2024-09-20T22:15:23.560075+0000","last_clean":"2024-09-20T22:15:23.560075+0000","last_became_active":"2024-09-20T22:11:30.486532+0000","last_became_peered":"2024-09-20T22:11:30.486532+0000","last_unstale":"2024-09-20T22:15:23.560075+0000","last_undegraded":"2024-09-20T22:15:23.560075+0000","last_fullsized":"2024-09-20T22:15:23.560075+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:13:41.420933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:41.420933+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":700,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566377+0000","last_change":"2024-09-20T22:14:11.402471+0000","last_active":"2024-09-20T22:15:23.566377+0000","last_peered":"2024-09-20T22:15:23.566377+0000","last_clean":"2024-09-20T22:15:23.566377+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:15:23.566377+0000","last_undegraded":"2024-09-20T22:15:23.566377+0000","last_fullsized":"2024-09-20T22:15:23.566377+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:11.402361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:14:11.402361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:27.694063+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"363'13","reported_seq":161,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569784+0000","last_change":"2024-09-20T22:14:52.612861+0000","last_active":"2024-09-20T22:15:23.569784+0000","last_peered":"2024-09-20T22:15:23.569784+0000","last_clean":"2024-09-20T22:15:23.569784+0000","last_became_active":"2024-09-20T22:12:57.884415+0000","last_became_peered":"2024-09-20T22:12:57.884415+0000","last_unstale":"2024-09-20T22:15:23.569784+0000","last_undegraded":"2024-09-20T22:15:23.569784+0000","last_fullsized":"2024-09-20T22:15:23.569784+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"363'13","last_scrub_stamp":"2024-09-20T22:14:52.612615+0000","last_deep_scrub":"363'13","last_deep_scrub_stamp":"2024-09-20T22:14:52.612615+0000","last_clean_scrub_stamp":"2024-09-20T22:14:52.612615+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:56.113350+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":128,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569299+0000","last_change":"2024-09-20T22:14:55.614617+0000","last_active":"2024-09-20T22:15:23.569299+0000","last_peered":"2024-09-20T22:15:23.569299+0000","last_clean":"2024-09-20T22:15:23.569299+0000","last_became_active":"2024-09-20T22:12:52.518529+0000","last_became_peered":"2024-09-20T22:12:52.518529+0000","last_unstale":"2024-09-20T22:15:23.569299+0000","last_undegraded":"2024-09-20T22:15:23.569299+0000","last_fullsized":"2024-09-20T22:15:23.569299+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:55.614504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:55.614504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:18.700502+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"378'1615","reported_seq":1641,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:27.904406+0000","last_change":"2024-09-20T22:15:08.927958+0000","last_active":"2024-09-20T22:15:27.904406+0000","last_peered":"2024-09-20T22:15:27.904406+0000","last_clean":"2024-09-20T22:15:27.904406+0000","last_became_active":"2024-09-20T22:15:08.927661+0000","last_became_peered":"2024-09-20T22:15:08.927661+0000","last_unstale":"2024-09-20T22:15:27.904406+0000","last_undegraded":"2024-09-20T22:15:27.904406+0000","last_fullsized":"2024-09-20T22:15:27.904406+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1615,"log_dups_size":0,"ondisk_log_size":1615,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:29.356464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":105840640,"num_objects":1615,"num_object_clones":0,"num_object_copies":4845,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1615,"num_write_kb":103360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":711,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565207+0000","last_change":"2024-09-20T22:15:17.487480+0000","last_active":"2024-09-20T22:15:23.565207+0000","last_peered":"2024-09-20T22:15:23.565207+0000","last_clean":"2024-09-20T22:15:23.565207+0000","last_became_active":"2024-09-20T22:12:55.523589+0000","last_became_peered":"2024-09-20T22:12:55.523589+0000","last_unstale":"2024-09-20T22:15:23.565207+0000","last_undegraded":"2024-09-20T22:15:23.565207+0000","last_fullsized":"2024-09-20T22:15:23.565207+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:17.487408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:17.487408+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:28.563004+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"328'29","reported_seq":161,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569850+0000","last_change":"2024-09-20T22:14:15.427866+0000","last_active":"2024-09-20T22:15:23.569850+0000","last_peered":"2024-09-20T22:15:23.569850+0000","last_clean":"2024-09-20T22:15:23.569850+0000","last_became_active":"2024-09-20T22:12:52.525538+0000","last_became_peered":"2024-09-20T22:12:52.525538+0000","last_unstale":"2024-09-20T22:15:23.569850+0000","last_undegraded":"2024-09-20T22:15:23.569850+0000","last_fullsized":"2024-09-20T22:15:23.569850+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'29","last_scrub_stamp":"2024-09-20T22:14:15.427533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:15.427533+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.217474+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":527,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560143+0000","last_change":"2024-09-20T22:13:13.091930+0000","last_active":"2024-09-20T22:15:23.560143+0000","last_peered":"2024-09-20T22:15:23.560143+0000","last_clean":"2024-09-20T22:15:23.560143+0000","last_became_active":"2024-09-20T22:11:30.486428+0000","last_became_peered":"2024-09-20T22:11:30.486428+0000","last_unstale":"2024-09-20T22:15:23.560143+0000","last_undegraded":"2024-09-20T22:15:23.560143+0000","last_fullsized":"2024-09-20T22:15:23.560143+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:13.091819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:13.091819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":704,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.564920+0000","last_change":"2024-09-20T22:14:22.500268+0000","last_active":"2024-09-20T22:15:23.564920+0000","last_peered":"2024-09-20T22:15:23.564920+0000","last_clean":"2024-09-20T22:15:23.564920+0000","last_became_active":"2024-09-20T22:12:52.519826+0000","last_became_peered":"2024-09-20T22:12:52.519826+0000","last_unstale":"2024-09-20T22:15:23.564920+0000","last_undegraded":"2024-09-20T22:15:23.564920+0000","last_fullsized":"2024-09-20T22:15:23.564920+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:22.500119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:14:22.500119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:41.298289+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":521,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559302+0000","last_change":"2024-09-20T22:15:21.489682+0000","last_active":"2024-09-20T22:15:23.559302+0000","last_peered":"2024-09-20T22:15:23.559302+0000","last_clean":"2024-09-20T22:15:23.559302+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:15:23.559302+0000","last_undegraded":"2024-09-20T22:15:23.559302+0000","last_fullsized":"2024-09-20T22:15:23.559302+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:21.489619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:21.489619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:22.282365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"378'1659","reported_seq":1685,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:27.922142+0000","last_change":"2024-09-20T22:15:08.926557+0000","last_active":"2024-09-20T22:15:27.922142+0000","last_peered":"2024-09-20T22:15:27.922142+0000","last_clean":"2024-09-20T22:15:27.922142+0000","last_became_active":"2024-09-20T22:15:08.926245+0000","last_became_peered":"2024-09-20T22:15:08.926245+0000","last_unstale":"2024-09-20T22:15:27.922142+0000","last_undegraded":"2024-09-20T22:15:27.922142+0000","last_fullsized":"2024-09-20T22:15:27.922142+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1659,"log_dups_size":0,"ondisk_log_size":1659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:27.389588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":108724224,"num_objects":1659,"num_object_clones":0,"num_object_copies":4977,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1659,"num_write_kb":106176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":698,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.564541+0000","last_change":"2024-09-20T22:12:55.522020+0000","last_active":"2024-09-20T22:15:23.564541+0000","last_peered":"2024-09-20T22:15:23.564541+0000","last_clean":"2024-09-20T22:15:23.564541+0000","last_became_active":"2024-09-20T22:12:55.521605+0000","last_became_peered":"2024-09-20T22:12:55.521605+0000","last_unstale":"2024-09-20T22:15:23.564541+0000","last_undegraded":"2024-09-20T22:15:23.564541+0000","last_fullsized":"2024-09-20T22:15:23.564541+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:12:16.121263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:12:16.121263+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"328'52","reported_seq":715,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560333+0000","last_change":"2024-09-20T22:14:46.592348+0000","last_active":"2024-09-20T22:15:23.560333+0000","last_peered":"2024-09-20T22:15:23.560333+0000","last_clean":"2024-09-20T22:15:23.560333+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:15:23.560333+0000","last_undegraded":"2024-09-20T22:15:23.560333+0000","last_fullsized":"2024-09-20T22:15:23.560333+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'52","last_scrub_stamp":"2024-09-20T22:14:46.592203+0000","last_deep_scrub":"271'40","last_deep_scrub_stamp":"2024-09-20T22:11:05.235779+0000","last_clean_scrub_stamp":"2024-09-20T22:14:46.592203+0000","objects_scrubbed":10,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:01.016758+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":694,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561782+0000","last_change":"2024-09-20T22:14:58.031902+0000","last_active":"2024-09-20T22:15:23.561782+0000","last_peered":"2024-09-20T22:15:23.561782+0000","last_clean":"2024-09-20T22:15:23.561782+0000","last_became_active":"2024-09-20T22:12:57.884257+0000","last_became_peered":"2024-09-20T22:12:57.884257+0000","last_unstale":"2024-09-20T22:15:23.561782+0000","last_undegraded":"2024-09-20T22:15:23.561782+0000","last_fullsized":"2024-09-20T22:15:23.561782+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:58.031824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:58.031824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:04.750930+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":696,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565347+0000","last_change":"2024-09-20T22:13:51.454966+0000","last_active":"2024-09-20T22:15:23.565347+0000","last_peered":"2024-09-20T22:15:23.565347+0000","last_clean":"2024-09-20T22:15:23.565347+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:15:23.565347+0000","last_undegraded":"2024-09-20T22:15:23.565347+0000","last_fullsized":"2024-09-20T22:15:23.565347+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.454850+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":128,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.570512+0000","last_change":"2024-09-20T22:14:12.458599+0000","last_active":"2024-09-20T22:15:23.570512+0000","last_peered":"2024-09-20T22:15:23.570512+0000","last_clean":"2024-09-20T22:15:23.570512+0000","last_became_active":"2024-09-20T22:12:52.527547+0000","last_became_peered":"2024-09-20T22:12:52.527547+0000","last_unstale":"2024-09-20T22:15:23.570512+0000","last_undegraded":"2024-09-20T22:15:23.570512+0000","last_fullsized":"2024-09-20T22:15:23.570512+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:12.458492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:12.458492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:36.742957+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":671,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:24.071666+0000","last_change":"2024-09-20T22:15:24.071705+0000","last_active":"2024-09-20T22:15:24.071666+0000","last_peered":"2024-09-20T22:15:24.071666+0000","last_clean":"2024-09-20T22:15:24.071666+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:15:24.071666+0000","last_undegraded":"2024-09-20T22:15:24.071666+0000","last_fullsized":"2024-09-20T22:15:24.071666+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.071593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.071593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:51.612695+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"378'1465","reported_seq":1491,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:26.166906+0000","last_change":"2024-09-20T22:15:08.931328+0000","last_active":"2024-09-20T22:15:26.166906+0000","last_peered":"2024-09-20T22:15:26.166906+0000","last_clean":"2024-09-20T22:15:26.166906+0000","last_became_active":"2024-09-20T22:15:08.930968+0000","last_became_peered":"2024-09-20T22:15:08.930968+0000","last_unstale":"2024-09-20T22:15:26.166906+0000","last_undegraded":"2024-09-20T22:15:26.166906+0000","last_fullsized":"2024-09-20T22:15:26.166906+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1465,"log_dups_size":0,"ondisk_log_size":1465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:14.066582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96010240,"num_objects":1465,"num_object_clones":0,"num_object_copies":4395,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1465,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1465,"num_write_kb":93760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":695,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566437+0000","last_change":"2024-09-20T22:15:00.407717+0000","last_active":"2024-09-20T22:15:23.566437+0000","last_peered":"2024-09-20T22:15:23.566437+0000","last_clean":"2024-09-20T22:15:23.566437+0000","last_became_active":"2024-09-20T22:12:55.518061+0000","last_became_peered":"2024-09-20T22:12:55.518061+0000","last_unstale":"2024-09-20T22:15:23.566437+0000","last_undegraded":"2024-09-20T22:15:23.566437+0000","last_fullsized":"2024-09-20T22:15:23.566437+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:00.407627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:00.407627+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:02.154276+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"328'17","reported_seq":593,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561598+0000","last_change":"2024-09-20T22:13:51.684806+0000","last_active":"2024-09-20T22:15:23.561598+0000","last_peered":"2024-09-20T22:15:23.561598+0000","last_clean":"2024-09-20T22:15:23.561598+0000","last_became_active":"2024-09-20T22:12:52.521073+0000","last_became_peered":"2024-09-20T22:12:52.521073+0000","last_unstale":"2024-09-20T22:15:23.561598+0000","last_undegraded":"2024-09-20T22:15:23.561598+0000","last_fullsized":"2024-09-20T22:15:23.561598+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'17","last_scrub_stamp":"2024-09-20T22:13:51.684667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.684667+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":522,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559330+0000","last_change":"2024-09-20T22:14:49.575469+0000","last_active":"2024-09-20T22:15:23.559330+0000","last_peered":"2024-09-20T22:15:23.559330+0000","last_clean":"2024-09-20T22:15:23.559330+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:15:23.559330+0000","last_undegraded":"2024-09-20T22:15:23.559330+0000","last_fullsized":"2024-09-20T22:15:23.559330+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:49.575363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:49.575363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:08.882258+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":738,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565240+0000","last_change":"2024-09-20T22:14:18.282947+0000","last_active":"2024-09-20T22:15:23.565240+0000","last_peered":"2024-09-20T22:15:23.565240+0000","last_clean":"2024-09-20T22:15:23.565240+0000","last_became_active":"2024-09-20T22:12:52.517255+0000","last_became_peered":"2024-09-20T22:12:52.517255+0000","last_unstale":"2024-09-20T22:15:23.565240+0000","last_undegraded":"2024-09-20T22:15:23.565240+0000","last_fullsized":"2024-09-20T22:15:23.565240+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:14:18.282814+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:14:18.282814+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:39.504754+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":699,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:24.892517+0000","last_change":"2024-09-20T22:15:24.892562+0000","last_active":"2024-09-20T22:15:24.892517+0000","last_peered":"2024-09-20T22:15:24.892517+0000","last_clean":"2024-09-20T22:15:24.892517+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:15:24.892517+0000","last_undegraded":"2024-09-20T22:15:24.892517+0000","last_fullsized":"2024-09-20T22:15:24.892517+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.891955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.891955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:47.230621+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":664,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567341+0000","last_change":"2024-09-20T22:13:41.730912+0000","last_active":"2024-09-20T22:15:23.567341+0000","last_peered":"2024-09-20T22:15:23.567341+0000","last_clean":"2024-09-20T22:15:23.567341+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:15:23.567341+0000","last_undegraded":"2024-09-20T22:15:23.567341+0000","last_fullsized":"2024-09-20T22:15:23.567341+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:41.730776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:41.730776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"378'1535","reported_seq":1561,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:26.979728+0000","last_change":"2024-09-20T22:15:08.927019+0000","last_active":"2024-09-20T22:15:26.979728+0000","last_peered":"2024-09-20T22:15:26.979728+0000","last_clean":"2024-09-20T22:15:26.979728+0000","last_became_active":"2024-09-20T22:15:08.926702+0000","last_became_peered":"2024-09-20T22:15:08.926702+0000","last_unstale":"2024-09-20T22:15:26.979728+0000","last_undegraded":"2024-09-20T22:15:26.979728+0000","last_fullsized":"2024-09-20T22:15:26.979728+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1535,"log_dups_size":0,"ondisk_log_size":1535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:17.801635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":100597760,"num_objects":1535,"num_object_clones":0,"num_object_copies":4605,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1535,"num_write_kb":98240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":679,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559344+0000","last_change":"2024-09-20T22:13:48.490583+0000","last_active":"2024-09-20T22:15:23.559344+0000","last_peered":"2024-09-20T22:15:23.559344+0000","last_clean":"2024-09-20T22:15:23.559344+0000","last_became_active":"2024-09-20T22:08:48.446840+0000","last_became_peered":"2024-09-20T22:08:48.446840+0000","last_unstale":"2024-09-20T22:15:23.559344+0000","last_undegraded":"2024-09-20T22:15:23.559344+0000","last_fullsized":"2024-09-20T22:15:23.559344+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:13:48.490464+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"328'24","reported_seq":168,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:26.742385+0000","last_change":"2024-09-20T22:15:26.742439+0000","last_active":"2024-09-20T22:15:26.742385+0000","last_peered":"2024-09-20T22:15:26.742385+0000","last_clean":"2024-09-20T22:15:26.742385+0000","last_became_active":"2024-09-20T22:12:52.527919+0000","last_became_peered":"2024-09-20T22:12:52.527919+0000","last_unstale":"2024-09-20T22:15:26.742385+0000","last_undegraded":"2024-09-20T22:15:26.742385+0000","last_fullsized":"2024-09-20T22:15:26.742385+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'24","last_scrub_stamp":"2024-09-20T22:15:26.742317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:26.742317+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:32.350037+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":136,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569255+0000","last_change":"2024-09-20T22:15:21.774404+0000","last_active":"2024-09-20T22:15:23.569255+0000","last_peered":"2024-09-20T22:15:23.569255+0000","last_clean":"2024-09-20T22:15:23.569255+0000","last_became_active":"2024-09-20T22:12:52.517432+0000","last_became_peered":"2024-09-20T22:12:52.517432+0000","last_unstale":"2024-09-20T22:15:23.569255+0000","last_undegraded":"2024-09-20T22:15:23.569255+0000","last_fullsized":"2024-09-20T22:15:23.569255+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:21.774300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:15:21.774300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:29.628508+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":1456,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:24.769288+0000","last_change":"2024-09-20T22:13:06.609600+0000","last_active":"2024-09-20T22:15:24.769288+0000","last_peered":"2024-09-20T22:15:24.769288+0000","last_clean":"2024-09-20T22:15:24.769288+0000","last_became_active":"2024-09-20T22:12:52.526622+0000","last_became_peered":"2024-09-20T22:12:52.526622+0000","last_unstale":"2024-09-20T22:15:24.769288+0000","last_undegraded":"2024-09-20T22:15:24.769288+0000","last_fullsized":"2024-09-20T22:15:24.769288+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:13:06.609525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:13:06.609525+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":732,"num_read_kb":727,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":230,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567015+0000","last_change":"2024-09-20T22:14:14.443714+0000","last_active":"2024-09-20T22:15:23.567015+0000","last_peered":"2024-09-20T22:15:23.567015+0000","last_clean":"2024-09-20T22:15:23.567015+0000","last_became_active":"2024-09-20T22:12:52.518823+0000","last_became_peered":"2024-09-20T22:12:52.518823+0000","last_unstale":"2024-09-20T22:15:23.567015+0000","last_undegraded":"2024-09-20T22:15:23.567015+0000","last_fullsized":"2024-09-20T22:15:23.567015+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:14.443611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:14.443611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":514,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559961+0000","last_change":"2024-09-20T22:13:40.433856+0000","last_active":"2024-09-20T22:15:23.559961+0000","last_peered":"2024-09-20T22:15:23.559961+0000","last_clean":"2024-09-20T22:15:23.559961+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:15:23.559961+0000","last_undegraded":"2024-09-20T22:15:23.559961+0000","last_fullsized":"2024-09-20T22:15:23.559961+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:40.433606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.415468+0000","last_clean_scrub_stamp":"2024-09-20T22:13:40.433606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"378'1675","reported_seq":1701,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:27.921293+0000","last_change":"2024-09-20T22:15:08.925131+0000","last_active":"2024-09-20T22:15:27.921293+0000","last_peered":"2024-09-20T22:15:27.921293+0000","last_clean":"2024-09-20T22:15:27.921293+0000","last_became_active":"2024-09-20T22:15:08.924809+0000","last_became_peered":"2024-09-20T22:15:08.924809+0000","last_unstale":"2024-09-20T22:15:27.921293+0000","last_undegraded":"2024-09-20T22:15:27.921293+0000","last_fullsized":"2024-09-20T22:15:27.921293+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1675,"log_dups_size":0,"ondisk_log_size":1675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:36.576727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":109772800,"num_objects":1675,"num_object_clones":0,"num_object_copies":5025,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1675,"num_write_kb":107200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"328'28","reported_seq":879,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565409+0000","last_change":"2024-09-20T22:14:21.445235+0000","last_active":"2024-09-20T22:15:23.565409+0000","last_peered":"2024-09-20T22:15:23.565409+0000","last_clean":"2024-09-20T22:15:23.565409+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:15:23.565409+0000","last_undegraded":"2024-09-20T22:15:23.565409+0000","last_fullsized":"2024-09-20T22:15:23.565409+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:14:21.445103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:21.445103+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":636,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.564705+0000","last_change":"2024-09-20T22:12:42.260316+0000","last_active":"2024-09-20T22:15:23.564705+0000","last_peered":"2024-09-20T22:15:23.564705+0000","last_clean":"2024-09-20T22:15:23.564705+0000","last_became_active":"2024-09-20T22:12:21.519717+0000","last_became_peered":"2024-09-20T22:12:21.519717+0000","last_unstale":"2024-09-20T22:15:23.564705+0000","last_undegraded":"2024-09-20T22:15:23.564705+0000","last_fullsized":"2024-09-20T22:15:23.564705+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:42.260197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:42.260197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":695,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:25.365177+0000","last_change":"2024-09-20T22:15:25.365208+0000","last_active":"2024-09-20T22:15:25.365177+0000","last_peered":"2024-09-20T22:15:25.365177+0000","last_clean":"2024-09-20T22:15:25.365177+0000","last_became_active":"2024-09-20T22:12:55.522325+0000","last_became_peered":"2024-09-20T22:12:55.522325+0000","last_unstale":"2024-09-20T22:15:25.365177+0000","last_undegraded":"2024-09-20T22:15:25.365177+0000","last_fullsized":"2024-09-20T22:15:25.365177+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:25.365122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:25.365122+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:41.815575+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":259,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566610+0000","last_change":"2024-09-20T22:13:51.675128+0000","last_active":"2024-09-20T22:15:23.566610+0000","last_peered":"2024-09-20T22:15:23.566610+0000","last_clean":"2024-09-20T22:15:23.566610+0000","last_became_active":"2024-09-20T22:10:48.907271+0000","last_became_peered":"2024-09-20T22:10:48.907271+0000","last_unstale":"2024-09-20T22:15:23.566610+0000","last_undegraded":"2024-09-20T22:15:23.566610+0000","last_fullsized":"2024-09-20T22:15:23.566610+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.674976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":691,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566801+0000","last_change":"2024-09-20T22:13:02.482655+0000","last_active":"2024-09-20T22:15:23.566801+0000","last_peered":"2024-09-20T22:15:23.566801+0000","last_clean":"2024-09-20T22:15:23.566801+0000","last_became_active":"2024-09-20T22:12:44.630044+0000","last_became_peered":"2024-09-20T22:12:44.630044+0000","last_unstale":"2024-09-20T22:15:23.566801+0000","last_undegraded":"2024-09-20T22:15:23.566801+0000","last_fullsized":"2024-09-20T22:15:23.566801+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:02.482546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:13:02.482546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"378'1691","reported_seq":1717,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:27.905733+0000","last_change":"2024-09-20T22:15:08.925316+0000","last_active":"2024-09-20T22:15:27.905733+0000","last_peered":"2024-09-20T22:15:27.905733+0000","last_clean":"2024-09-20T22:15:27.905733+0000","last_became_active":"2024-09-20T22:15:08.925040+0000","last_became_peered":"2024-09-20T22:15:08.925040+0000","last_unstale":"2024-09-20T22:15:27.905733+0000","last_undegraded":"2024-09-20T22:15:27.905733+0000","last_fullsized":"2024-09-20T22:15:27.905733+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1691,"log_dups_size":0,"ondisk_log_size":1691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:30.887426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110821376,"num_objects":1691,"num_object_clones":0,"num_object_copies":5073,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1691,"num_write_kb":108224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"328'19","reported_seq":712,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.562573+0000","last_change":"2024-09-20T22:15:19.012731+0000","last_active":"2024-09-20T22:15:23.562573+0000","last_peered":"2024-09-20T22:15:23.562573+0000","last_clean":"2024-09-20T22:15:23.562573+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:15:23.562573+0000","last_undegraded":"2024-09-20T22:15:23.562573+0000","last_fullsized":"2024-09-20T22:15:23.562573+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:15:19.012609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:19.012609+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:20.138304+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":676,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561369+0000","last_change":"2024-09-20T22:13:39.055297+0000","last_active":"2024-09-20T22:15:23.561369+0000","last_peered":"2024-09-20T22:15:23.561369+0000","last_clean":"2024-09-20T22:15:23.561369+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:15:23.561369+0000","last_undegraded":"2024-09-20T22:15:23.561369+0000","last_fullsized":"2024-09-20T22:15:23.561369+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:39.055192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:39.055192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":683,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:24.150346+0000","last_change":"2024-09-20T22:15:24.150401+0000","last_active":"2024-09-20T22:15:24.150346+0000","last_peered":"2024-09-20T22:15:24.150346+0000","last_clean":"2024-09-20T22:15:24.150346+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:15:24.150346+0000","last_undegraded":"2024-09-20T22:15:24.150346+0000","last_fullsized":"2024-09-20T22:15:24.150346+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.150265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.150265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:32.448596+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":631,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565142+0000","last_change":"2024-09-20T22:14:08.297827+0000","last_active":"2024-09-20T22:15:23.565142+0000","last_peered":"2024-09-20T22:15:23.565142+0000","last_clean":"2024-09-20T22:15:23.565142+0000","last_became_active":"2024-09-20T22:12:21.521195+0000","last_became_peered":"2024-09-20T22:12:21.521195+0000","last_unstale":"2024-09-20T22:15:23.565142+0000","last_undegraded":"2024-09-20T22:15:23.565142+0000","last_fullsized":"2024-09-20T22:15:23.565142+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:08.297698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:08.297698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3","version":"378'1472","reported_seq":1498,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:26.237125+0000","last_change":"2024-09-20T22:15:09.087357+0000","last_active":"2024-09-20T22:15:26.237125+0000","last_peered":"2024-09-20T22:15:26.237125+0000","last_clean":"2024-09-20T22:15:26.237125+0000","last_became_active":"2024-09-20T22:15:09.087061+0000","last_became_peered":"2024-09-20T22:15:09.087061+0000","last_unstale":"2024-09-20T22:15:26.237125+0000","last_undegraded":"2024-09-20T22:15:26.237125+0000","last_fullsized":"2024-09-20T22:15:26.237125+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1472,"log_dups_size":0,"ondisk_log_size":1472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:23.286140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":96468992,"num_objects":1472,"num_object_clones":0,"num_object_copies":4416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1472,"num_write_kb":94208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":446,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.564473+0000","last_change":"2024-09-20T22:14:46.480442+0000","last_active":"2024-09-20T22:15:23.564473+0000","last_peered":"2024-09-20T22:15:23.564473+0000","last_clean":"2024-09-20T22:15:23.564473+0000","last_became_active":"2024-09-20T22:14:38.674758+0000","last_became_peered":"2024-09-20T22:14:38.674758+0000","last_unstale":"2024-09-20T22:15:23.564473+0000","last_undegraded":"2024-09-20T22:15:23.564473+0000","last_fullsized":"2024-09-20T22:15:23.564473+0000","mapping_epoch":358,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":359,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:14:46.480331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:14:46.480331+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:06.604863+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"328'31","reported_seq":738,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.562287+0000","last_change":"2024-09-20T22:14:46.989102+0000","last_active":"2024-09-20T22:15:23.562287+0000","last_peered":"2024-09-20T22:15:23.562287+0000","last_clean":"2024-09-20T22:15:23.562287+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:15:23.562287+0000","last_undegraded":"2024-09-20T22:15:23.562287+0000","last_fullsized":"2024-09-20T22:15:23.562287+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'31","last_scrub_stamp":"2024-09-20T22:14:46.988744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:46.988744+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:58.605093+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":692,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565751+0000","last_change":"2024-09-20T22:14:12.293291+0000","last_active":"2024-09-20T22:15:23.565751+0000","last_peered":"2024-09-20T22:15:23.565751+0000","last_clean":"2024-09-20T22:15:23.565751+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:15:23.565751+0000","last_undegraded":"2024-09-20T22:15:23.565751+0000","last_fullsized":"2024-09-20T22:15:23.565751+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:12.293188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:12.293188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":525,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560054+0000","last_change":"2024-09-20T22:13:10.039183+0000","last_active":"2024-09-20T22:15:23.560054+0000","last_peered":"2024-09-20T22:15:23.560054+0000","last_clean":"2024-09-20T22:15:23.560054+0000","last_became_active":"2024-09-20T22:12:52.520612+0000","last_became_peered":"2024-09-20T22:12:52.520612+0000","last_unstale":"2024-09-20T22:15:23.560054+0000","last_undegraded":"2024-09-20T22:15:23.560054+0000","last_fullsized":"2024-09-20T22:15:23.560054+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:10.039074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.039074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":687,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559895+0000","last_change":"2024-09-20T22:14:50.010648+0000","last_active":"2024-09-20T22:15:23.559895+0000","last_peered":"2024-09-20T22:15:23.559895+0000","last_clean":"2024-09-20T22:15:23.559895+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:15:23.559895+0000","last_undegraded":"2024-09-20T22:15:23.559895+0000","last_fullsized":"2024-09-20T22:15:23.559895+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:50.010536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:50.010536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:56.117583+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"378'1650","reported_seq":1676,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:28.381788+0000","last_change":"2024-09-20T22:15:08.926429+0000","last_active":"2024-09-20T22:15:28.381788+0000","last_peered":"2024-09-20T22:15:28.381788+0000","last_clean":"2024-09-20T22:15:28.381788+0000","last_became_active":"2024-09-20T22:15:08.926083+0000","last_became_peered":"2024-09-20T22:15:08.926083+0000","last_unstale":"2024-09-20T22:15:28.381788+0000","last_undegraded":"2024-09-20T22:15:28.381788+0000","last_fullsized":"2024-09-20T22:15:28.381788+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1650,"log_dups_size":0,"ondisk_log_size":1650,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:12.245813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":108134400,"num_objects":1650,"num_object_clones":0,"num_object_copies":4950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1650,"num_write_kb":105600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":957,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566946+0000","last_change":"2024-09-20T22:13:49.912616+0000","last_active":"2024-09-20T22:15:23.566946+0000","last_peered":"2024-09-20T22:15:23.566946+0000","last_clean":"2024-09-20T22:15:23.566946+0000","last_became_active":"2024-09-20T22:08:48.455288+0000","last_became_peered":"2024-09-20T22:08:48.455288+0000","last_unstale":"2024-09-20T22:15:23.566946+0000","last_undegraded":"2024-09-20T22:15:23.566946+0000","last_fullsized":"2024-09-20T22:15:23.566946+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:49.912496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:13:49.912496+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":227,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"328'25","reported_seq":707,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561681+0000","last_change":"2024-09-20T22:14:10.070008+0000","last_active":"2024-09-20T22:15:23.561681+0000","last_peered":"2024-09-20T22:15:23.561681+0000","last_clean":"2024-09-20T22:15:23.561681+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:15:23.561681+0000","last_undegraded":"2024-09-20T22:15:23.561681+0000","last_fullsized":"2024-09-20T22:15:23.561681+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'25","last_scrub_stamp":"2024-09-20T22:14:10.069812+0000","last_deep_scrub":"289'15","last_deep_scrub_stamp":"2024-09-20T22:11:37.277648+0000","last_clean_scrub_stamp":"2024-09-20T22:14:10.069812+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":132,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569840+0000","last_change":"2024-09-20T22:14:23.473613+0000","last_active":"2024-09-20T22:15:23.569840+0000","last_peered":"2024-09-20T22:15:23.569840+0000","last_clean":"2024-09-20T22:15:23.569840+0000","last_became_active":"2024-09-20T22:12:52.528310+0000","last_became_peered":"2024-09-20T22:12:52.528310+0000","last_unstale":"2024-09-20T22:15:23.569840+0000","last_undegraded":"2024-09-20T22:15:23.569840+0000","last_fullsized":"2024-09-20T22:15:23.569840+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:14:23.472792+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:14:23.472792+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:27.168827+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"283'11","reported_seq":880,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560636+0000","last_change":"2024-09-20T22:14:20.265212+0000","last_active":"2024-09-20T22:15:23.560636+0000","last_peered":"2024-09-20T22:15:23.560636+0000","last_clean":"2024-09-20T22:15:23.560636+0000","last_became_active":"2024-09-20T22:11:30.485356+0000","last_became_peered":"2024-09-20T22:11:30.485356+0000","last_unstale":"2024-09-20T22:15:23.560636+0000","last_undegraded":"2024-09-20T22:15:23.560636+0000","last_fullsized":"2024-09-20T22:15:23.560636+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'11","last_scrub_stamp":"2024-09-20T22:14:20.265107+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:14:20.265107+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:43.795761+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":673,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.562251+0000","last_change":"2024-09-20T22:14:51.059172+0000","last_active":"2024-09-20T22:15:23.562251+0000","last_peered":"2024-09-20T22:15:23.562251+0000","last_clean":"2024-09-20T22:15:23.562251+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:15:23.562251+0000","last_undegraded":"2024-09-20T22:15:23.562251+0000","last_fullsized":"2024-09-20T22:15:23.562251+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:51.059066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:14:51.059066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:20.407644+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"378'1418","reported_seq":1444,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:26.228678+0000","last_change":"2024-09-20T22:15:09.088109+0000","last_active":"2024-09-20T22:15:26.228678+0000","last_peered":"2024-09-20T22:15:26.228678+0000","last_clean":"2024-09-20T22:15:26.228678+0000","last_became_active":"2024-09-20T22:15:09.087830+0000","last_became_peered":"2024-09-20T22:15:09.087830+0000","last_unstale":"2024-09-20T22:15:26.228678+0000","last_undegraded":"2024-09-20T22:15:26.228678+0000","last_fullsized":"2024-09-20T22:15:26.228678+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1418,"log_dups_size":0,"ondisk_log_size":1418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:10.163377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92930048,"num_objects":1418,"num_object_clones":0,"num_object_copies":4254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1418,"num_write_kb":90752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"328'15","reported_seq":155,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:25.741727+0000","last_change":"2024-09-20T22:15:25.742250+0000","last_active":"2024-09-20T22:15:25.741727+0000","last_peered":"2024-09-20T22:15:25.741727+0000","last_clean":"2024-09-20T22:15:25.741727+0000","last_became_active":"2024-09-20T22:12:52.527531+0000","last_became_peered":"2024-09-20T22:12:52.527531+0000","last_unstale":"2024-09-20T22:15:25.741727+0000","last_undegraded":"2024-09-20T22:15:25.741727+0000","last_fullsized":"2024-09-20T22:15:25.741727+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:15:25.741631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:25.741631+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:28.256179+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":312,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.562235+0000","last_change":"2024-09-20T22:14:55.145177+0000","last_active":"2024-09-20T22:15:23.562235+0000","last_peered":"2024-09-20T22:15:23.562235+0000","last_clean":"2024-09-20T22:15:23.562235+0000","last_became_active":"2024-09-20T22:10:14.536155+0000","last_became_peered":"2024-09-20T22:10:14.536155+0000","last_unstale":"2024-09-20T22:15:23.562235+0000","last_undegraded":"2024-09-20T22:15:23.562235+0000","last_fullsized":"2024-09-20T22:15:23.562235+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:14:55.144682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:55.144682+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:11.541071+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":699,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.563569+0000","last_change":"2024-09-20T22:15:18.919125+0000","last_active":"2024-09-20T22:15:23.563569+0000","last_peered":"2024-09-20T22:15:23.563569+0000","last_clean":"2024-09-20T22:15:23.563569+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:15:23.563569+0000","last_undegraded":"2024-09-20T22:15:23.563569+0000","last_fullsized":"2024-09-20T22:15:23.563569+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:18.919024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:15:18.919024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:26.163742+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":128,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569534+0000","last_change":"2024-09-20T22:14:45.679385+0000","last_active":"2024-09-20T22:15:23.569534+0000","last_peered":"2024-09-20T22:15:23.569534+0000","last_clean":"2024-09-20T22:15:23.569534+0000","last_became_active":"2024-09-20T22:12:52.512924+0000","last_became_peered":"2024-09-20T22:12:52.512924+0000","last_unstale":"2024-09-20T22:15:23.569534+0000","last_undegraded":"2024-09-20T22:15:23.569534+0000","last_fullsized":"2024-09-20T22:15:23.569534+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:45.679291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:45.679291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:49.553806+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"378'1554","reported_seq":1579,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:26.983503+0000","last_change":"2024-09-20T22:15:09.087623+0000","last_active":"2024-09-20T22:15:26.983503+0000","last_peered":"2024-09-20T22:15:26.983503+0000","last_clean":"2024-09-20T22:15:26.983503+0000","last_became_active":"2024-09-20T22:15:09.087298+0000","last_became_peered":"2024-09-20T22:15:09.087298+0000","last_unstale":"2024-09-20T22:15:26.983503+0000","last_undegraded":"2024-09-20T22:15:26.983503+0000","last_fullsized":"2024-09-20T22:15:26.983503+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1554,"log_dups_size":0,"ondisk_log_size":1554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:13.382167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":101842944,"num_objects":1554,"num_object_clones":0,"num_object_copies":4662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1554,"num_write_kb":99456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"328'35","reported_seq":787,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565341+0000","last_change":"2024-09-20T22:13:51.408635+0000","last_active":"2024-09-20T22:15:23.565341+0000","last_peered":"2024-09-20T22:15:23.565341+0000","last_clean":"2024-09-20T22:15:23.565341+0000","last_became_active":"2024-09-20T22:10:30.652967+0000","last_became_peered":"2024-09-20T22:10:30.652967+0000","last_unstale":"2024-09-20T22:15:23.565341+0000","last_undegraded":"2024-09-20T22:15:23.565341+0000","last_fullsized":"2024-09-20T22:15:23.565341+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'35","last_scrub_stamp":"2024-09-20T22:13:51.408543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.408543+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":696,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567833+0000","last_change":"2024-09-20T22:15:23.351913+0000","last_active":"2024-09-20T22:15:23.567833+0000","last_peered":"2024-09-20T22:15:23.567833+0000","last_clean":"2024-09-20T22:15:23.567833+0000","last_became_active":"2024-09-20T22:11:30.488650+0000","last_became_peered":"2024-09-20T22:11:30.488650+0000","last_unstale":"2024-09-20T22:15:23.567833+0000","last_undegraded":"2024-09-20T22:15:23.567833+0000","last_fullsized":"2024-09-20T22:15:23.567833+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:23.351789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:12:46.602035+0000","last_clean_scrub_stamp":"2024-09-20T22:15:23.351789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:36.963085+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":723,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.563957+0000","last_change":"2024-09-20T22:14:51.988808+0000","last_active":"2024-09-20T22:15:23.563957+0000","last_peered":"2024-09-20T22:15:23.563957+0000","last_clean":"2024-09-20T22:15:23.563957+0000","last_became_active":"2024-09-20T22:12:52.515295+0000","last_became_peered":"2024-09-20T22:12:52.515295+0000","last_unstale":"2024-09-20T22:15:23.563957+0000","last_undegraded":"2024-09-20T22:15:23.563957+0000","last_fullsized":"2024-09-20T22:15:23.563957+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:51.988666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_clean_scrub_stamp":"2024-09-20T22:14:51.988666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:00.556168+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":281,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566618+0000","last_change":"2024-09-20T22:14:47.271963+0000","last_active":"2024-09-20T22:15:23.566618+0000","last_peered":"2024-09-20T22:15:23.566618+0000","last_clean":"2024-09-20T22:15:23.566618+0000","last_became_active":"2024-09-20T22:10:30.657116+0000","last_became_peered":"2024-09-20T22:10:30.657116+0000","last_unstale":"2024-09-20T22:15:23.566618+0000","last_undegraded":"2024-09-20T22:15:23.566618+0000","last_fullsized":"2024-09-20T22:15:23.566618+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:47.271872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:47.271872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:07.707877+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"378'1628","reported_seq":1654,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:27.906019+0000","last_change":"2024-09-20T22:15:09.089072+0000","last_active":"2024-09-20T22:15:27.906019+0000","last_peered":"2024-09-20T22:15:27.906019+0000","last_clean":"2024-09-20T22:15:27.906019+0000","last_became_active":"2024-09-20T22:15:09.088590+0000","last_became_peered":"2024-09-20T22:15:09.088590+0000","last_unstale":"2024-09-20T22:15:27.906019+0000","last_undegraded":"2024-09-20T22:15:27.906019+0000","last_fullsized":"2024-09-20T22:15:27.906019+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1628,"log_dups_size":0,"ondisk_log_size":1628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:25.980411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106692608,"num_objects":1628,"num_object_clones":0,"num_object_copies":4884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1628,"num_write_kb":104192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"328'18","reported_seq":713,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560618+0000","last_change":"2024-09-20T22:15:22.019818+0000","last_active":"2024-09-20T22:15:23.560618+0000","last_peered":"2024-09-20T22:15:23.560618+0000","last_clean":"2024-09-20T22:15:23.560618+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:15:23.560618+0000","last_undegraded":"2024-09-20T22:15:23.560618+0000","last_fullsized":"2024-09-20T22:15:23.560618+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'18","last_scrub_stamp":"2024-09-20T22:15:22.019700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:22.019700+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:42.439452+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":685,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567966+0000","last_change":"2024-09-20T22:14:15.965211+0000","last_active":"2024-09-20T22:15:23.567966+0000","last_peered":"2024-09-20T22:15:23.567966+0000","last_clean":"2024-09-20T22:15:23.567966+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:15:23.567966+0000","last_undegraded":"2024-09-20T22:15:23.567966+0000","last_fullsized":"2024-09-20T22:15:23.567966+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:15.965013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:15.965013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":712,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559040+0000","last_change":"2024-09-20T22:13:53.202635+0000","last_active":"2024-09-20T22:15:23.559040+0000","last_peered":"2024-09-20T22:15:23.559040+0000","last_clean":"2024-09-20T22:15:23.559040+0000","last_became_active":"2024-09-20T22:12:52.514202+0000","last_became_peered":"2024-09-20T22:12:52.514202+0000","last_unstale":"2024-09-20T22:15:23.559040+0000","last_undegraded":"2024-09-20T22:15:23.559040+0000","last_fullsized":"2024-09-20T22:15:23.559040+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:53.202518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:53.202518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":146,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:26.021714+0000","last_change":"2024-09-20T22:15:26.021754+0000","last_active":"2024-09-20T22:15:26.021714+0000","last_peered":"2024-09-20T22:15:26.021714+0000","last_clean":"2024-09-20T22:15:26.021714+0000","last_became_active":"2024-09-20T22:12:44.638546+0000","last_became_peered":"2024-09-20T22:12:44.638546+0000","last_unstale":"2024-09-20T22:15:26.021714+0000","last_undegraded":"2024-09-20T22:15:26.021714+0000","last_fullsized":"2024-09-20T22:15:26.021714+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:26.021636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:26.021636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:46.884542+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"378'1409","reported_seq":1435,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:26.162856+0000","last_change":"2024-09-20T22:15:08.928417+0000","last_active":"2024-09-20T22:15:26.162856+0000","last_peered":"2024-09-20T22:15:26.162856+0000","last_clean":"2024-09-20T22:15:26.162856+0000","last_became_active":"2024-09-20T22:15:08.928102+0000","last_became_peered":"2024-09-20T22:15:08.928102+0000","last_unstale":"2024-09-20T22:15:26.162856+0000","last_undegraded":"2024-09-20T22:15:26.162856+0000","last_fullsized":"2024-09-20T22:15:26.162856+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1409,"log_dups_size":0,"ondisk_log_size":1409,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:10.761616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92340224,"num_objects":1409,"num_object_clones":0,"num_object_copies":4227,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1409,"num_write_kb":90176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"328'27","reported_seq":244,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.564446+0000","last_change":"2024-09-20T22:13:10.353607+0000","last_active":"2024-09-20T22:15:23.564446+0000","last_peered":"2024-09-20T22:15:23.564446+0000","last_clean":"2024-09-20T22:15:23.564446+0000","last_became_active":"2024-09-20T22:11:30.493275+0000","last_became_peered":"2024-09-20T22:11:30.493275+0000","last_unstale":"2024-09-20T22:15:23.564446+0000","last_undegraded":"2024-09-20T22:15:23.564446+0000","last_fullsized":"2024-09-20T22:15:23.564446+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:13:10.353459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.353459+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":311,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567745+0000","last_change":"2024-09-20T22:14:56.339026+0000","last_active":"2024-09-20T22:15:23.567745+0000","last_peered":"2024-09-20T22:15:23.567745+0000","last_clean":"2024-09-20T22:15:23.567745+0000","last_became_active":"2024-09-20T22:12:52.526551+0000","last_became_peered":"2024-09-20T22:12:52.526551+0000","last_unstale":"2024-09-20T22:15:23.567745+0000","last_undegraded":"2024-09-20T22:15:23.567745+0000","last_fullsized":"2024-09-20T22:15:23.567745+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:14:56.338941+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:14:56.338941+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:09.361665+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":828,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566743+0000","last_change":"2024-09-20T22:14:12.911852+0000","last_active":"2024-09-20T22:15:23.566743+0000","last_peered":"2024-09-20T22:15:23.566743+0000","last_clean":"2024-09-20T22:15:23.566743+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:15:23.566743+0000","last_undegraded":"2024-09-20T22:15:23.566743+0000","last_fullsized":"2024-09-20T22:15:23.566743+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:14:12.911520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:12.911520+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.190103+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":222,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565005+0000","last_change":"2024-09-20T22:13:00.454973+0000","last_active":"2024-09-20T22:15:23.565005+0000","last_peered":"2024-09-20T22:15:23.565005+0000","last_clean":"2024-09-20T22:15:23.565005+0000","last_became_active":"2024-09-20T22:12:52.517758+0000","last_became_peered":"2024-09-20T22:12:52.517758+0000","last_unstale":"2024-09-20T22:15:23.565005+0000","last_undegraded":"2024-09-20T22:15:23.565005+0000","last_fullsized":"2024-09-20T22:15:23.565005+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:00.454378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:00.454378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":723,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565754+0000","last_change":"2024-09-20T22:14:49.417575+0000","last_active":"2024-09-20T22:15:23.565754+0000","last_peered":"2024-09-20T22:15:23.565754+0000","last_clean":"2024-09-20T22:15:23.565754+0000","last_became_active":"2024-09-20T22:12:52.516989+0000","last_became_peered":"2024-09-20T22:12:52.516989+0000","last_unstale":"2024-09-20T22:15:23.565754+0000","last_undegraded":"2024-09-20T22:15:23.565754+0000","last_fullsized":"2024-09-20T22:15:23.565754+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:49.417468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:49.417468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:19.052286+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"328'12","reported_seq":725,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565379+0000","last_change":"2024-09-20T22:14:58.150505+0000","last_active":"2024-09-20T22:15:23.565379+0000","last_peered":"2024-09-20T22:15:23.565379+0000","last_clean":"2024-09-20T22:15:23.565379+0000","last_became_active":"2024-09-20T22:10:48.922561+0000","last_became_peered":"2024-09-20T22:10:48.922561+0000","last_unstale":"2024-09-20T22:15:23.565379+0000","last_undegraded":"2024-09-20T22:15:23.565379+0000","last_fullsized":"2024-09-20T22:15:23.565379+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'12","last_scrub_stamp":"2024-09-20T22:14:58.150361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:58.150361+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:05.584490+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":210,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566854+0000","last_change":"2024-09-20T22:12:14.148857+0000","last_active":"2024-09-20T22:15:23.566854+0000","last_peered":"2024-09-20T22:15:23.566854+0000","last_clean":"2024-09-20T22:15:23.566854+0000","last_became_active":"2024-09-20T22:11:30.487322+0000","last_became_peered":"2024-09-20T22:11:30.487322+0000","last_unstale":"2024-09-20T22:15:23.566854+0000","last_undegraded":"2024-09-20T22:15:23.566854+0000","last_fullsized":"2024-09-20T22:15:23.566854+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:14.148743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:14.148743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":664,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567290+0000","last_change":"2024-09-20T22:14:54.359687+0000","last_active":"2024-09-20T22:15:23.567290+0000","last_peered":"2024-09-20T22:15:23.567290+0000","last_clean":"2024-09-20T22:15:23.567290+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:15:23.567290+0000","last_undegraded":"2024-09-20T22:15:23.567290+0000","last_fullsized":"2024-09-20T22:15:23.567290+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:54.359586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:54.359586+0000","last_clean_scrub_stamp":"2024-09-20T22:14:54.359586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:11.201500+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"328'15","reported_seq":147,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.570583+0000","last_change":"2024-09-20T22:14:14.397650+0000","last_active":"2024-09-20T22:15:23.570583+0000","last_peered":"2024-09-20T22:15:23.570583+0000","last_clean":"2024-09-20T22:15:23.570583+0000","last_became_active":"2024-09-20T22:12:52.527106+0000","last_became_peered":"2024-09-20T22:12:52.527106+0000","last_unstale":"2024-09-20T22:15:23.570583+0000","last_undegraded":"2024-09-20T22:15:23.570583+0000","last_fullsized":"2024-09-20T22:15:23.570583+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:14:14.397530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:14.397530+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:42.176603+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":676,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567106+0000","last_change":"2024-09-20T22:12:44.676449+0000","last_active":"2024-09-20T22:15:23.567106+0000","last_peered":"2024-09-20T22:15:23.567106+0000","last_clean":"2024-09-20T22:15:23.567106+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:15:23.567106+0000","last_undegraded":"2024-09-20T22:15:23.567106+0000","last_fullsized":"2024-09-20T22:15:23.567106+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:44.676334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:44.676334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":314,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561351+0000","last_change":"2024-09-20T22:13:14.136545+0000","last_active":"2024-09-20T22:15:23.561351+0000","last_peered":"2024-09-20T22:15:23.561351+0000","last_clean":"2024-09-20T22:15:23.561351+0000","last_became_active":"2024-09-20T22:12:33.545506+0000","last_became_peered":"2024-09-20T22:12:33.545506+0000","last_unstale":"2024-09-20T22:15:23.561351+0000","last_undegraded":"2024-09-20T22:15:23.561351+0000","last_fullsized":"2024-09-20T22:15:23.561351+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:14.136472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_clean_scrub_stamp":"2024-09-20T22:13:14.136472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":695,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559183+0000","last_change":"2024-09-20T22:15:00.011002+0000","last_active":"2024-09-20T22:15:23.559183+0000","last_peered":"2024-09-20T22:15:23.559183+0000","last_clean":"2024-09-20T22:15:23.559183+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:15:23.559183+0000","last_undegraded":"2024-09-20T22:15:23.559183+0000","last_fullsized":"2024-09-20T22:15:23.559183+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:00.010921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:00.010921+0000","last_clean_scrub_stamp":"2024-09-20T22:15:00.010921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:17.344827+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"328'15","reported_seq":229,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566265+0000","last_change":"2024-09-20T22:12:43.218553+0000","last_active":"2024-09-20T22:15:23.566265+0000","last_peered":"2024-09-20T22:15:23.566265+0000","last_clean":"2024-09-20T22:15:23.566265+0000","last_became_active":"2024-09-20T22:11:30.493335+0000","last_became_peered":"2024-09-20T22:11:30.493335+0000","last_unstale":"2024-09-20T22:15:23.566265+0000","last_undegraded":"2024-09-20T22:15:23.566265+0000","last_fullsized":"2024-09-20T22:15:23.566265+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:12:43.218249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:43.218249+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":136,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.571282+0000","last_change":"2024-09-20T22:14:56.538965+0000","last_active":"2024-09-20T22:15:23.571282+0000","last_peered":"2024-09-20T22:15:23.571282+0000","last_clean":"2024-09-20T22:15:23.571282+0000","last_became_active":"2024-09-20T22:12:52.524149+0000","last_became_peered":"2024-09-20T22:12:52.524149+0000","last_unstale":"2024-09-20T22:15:23.571282+0000","last_undegraded":"2024-09-20T22:15:23.571282+0000","last_fullsized":"2024-09-20T22:15:23.571282+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:56.538825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:56.538825+0000","last_clean_scrub_stamp":"2024-09-20T22:14:56.538825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:02.516641+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"315'11","reported_seq":215,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566989+0000","last_change":"2024-09-20T22:15:16.396532+0000","last_active":"2024-09-20T22:15:23.566989+0000","last_peered":"2024-09-20T22:15:23.566989+0000","last_clean":"2024-09-20T22:15:23.566989+0000","last_became_active":"2024-09-20T22:12:33.544098+0000","last_became_peered":"2024-09-20T22:12:33.544098+0000","last_unstale":"2024-09-20T22:15:23.566989+0000","last_undegraded":"2024-09-20T22:15:23.566989+0000","last_fullsized":"2024-09-20T22:15:23.566989+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:15:16.396454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:16.396454+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:18.359878+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":676,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.562682+0000","last_change":"2024-09-20T22:13:48.148947+0000","last_active":"2024-09-20T22:15:23.562682+0000","last_peered":"2024-09-20T22:15:23.562682+0000","last_clean":"2024-09-20T22:15:23.562682+0000","last_became_active":"2024-09-20T22:11:30.484731+0000","last_became_peered":"2024-09-20T22:11:30.484731+0000","last_unstale":"2024-09-20T22:15:23.562682+0000","last_undegraded":"2024-09-20T22:15:23.562682+0000","last_fullsized":"2024-09-20T22:15:23.562682+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:48.148819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:48.148819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"328'20","reported_seq":165,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559799+0000","last_change":"2024-09-20T22:13:49.905238+0000","last_active":"2024-09-20T22:15:23.559799+0000","last_peered":"2024-09-20T22:15:23.559799+0000","last_clean":"2024-09-20T22:15:23.559799+0000","last_became_active":"2024-09-20T22:12:44.631986+0000","last_became_peered":"2024-09-20T22:12:44.631986+0000","last_unstale":"2024-09-20T22:15:23.559799+0000","last_undegraded":"2024-09-20T22:15:23.559799+0000","last_fullsized":"2024-09-20T22:15:23.559799+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:13:49.905115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:49.905115+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":676,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567284+0000","last_change":"2024-09-20T22:14:23.041828+0000","last_active":"2024-09-20T22:15:23.567284+0000","last_peered":"2024-09-20T22:15:23.567284+0000","last_clean":"2024-09-20T22:15:23.567284+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:15:23.567284+0000","last_undegraded":"2024-09-20T22:15:23.567284+0000","last_fullsized":"2024-09-20T22:15:23.567284+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:23.041722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:23.041722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:33.056700+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"326'14","reported_seq":290,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566167+0000","last_change":"2024-09-20T22:15:19.186947+0000","last_active":"2024-09-20T22:15:23.566167+0000","last_peered":"2024-09-20T22:15:23.566167+0000","last_clean":"2024-09-20T22:15:23.566167+0000","last_became_active":"2024-09-20T22:12:52.518770+0000","last_became_peered":"2024-09-20T22:12:52.518770+0000","last_unstale":"2024-09-20T22:15:23.566167+0000","last_undegraded":"2024-09-20T22:15:23.566167+0000","last_fullsized":"2024-09-20T22:15:23.566167+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'14","last_scrub_stamp":"2024-09-20T22:15:19.186862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:19.186862+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:43.532138+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":136,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.570163+0000","last_change":"2024-09-20T22:15:22.741319+0000","last_active":"2024-09-20T22:15:23.570163+0000","last_peered":"2024-09-20T22:15:23.570163+0000","last_clean":"2024-09-20T22:15:23.570163+0000","last_became_active":"2024-09-20T22:12:52.523781+0000","last_became_peered":"2024-09-20T22:12:52.523781+0000","last_unstale":"2024-09-20T22:15:23.570163+0000","last_undegraded":"2024-09-20T22:15:23.570163+0000","last_fullsized":"2024-09-20T22:15:23.570163+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:22.741214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:22.741214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:34.096157+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"328'19","reported_seq":732,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561553+0000","last_change":"2024-09-20T22:14:59.020526+0000","last_active":"2024-09-20T22:15:23.561553+0000","last_peered":"2024-09-20T22:15:23.561553+0000","last_clean":"2024-09-20T22:15:23.561553+0000","last_became_active":"2024-09-20T22:08:22.139581+0000","last_became_peered":"2024-09-20T22:08:22.139581+0000","last_unstale":"2024-09-20T22:15:23.561553+0000","last_undegraded":"2024-09-20T22:15:23.561553+0000","last_fullsized":"2024-09-20T22:15:23.561553+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:14:59.020420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:59.020420+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:24.012397+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":684,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561836+0000","last_change":"2024-09-20T22:15:20.031722+0000","last_active":"2024-09-20T22:15:23.561836+0000","last_peered":"2024-09-20T22:15:23.561836+0000","last_clean":"2024-09-20T22:15:23.561836+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:15:23.561836+0000","last_undegraded":"2024-09-20T22:15:23.561836+0000","last_fullsized":"2024-09-20T22:15:23.561836+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.031625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.031625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:36.647532+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":512,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559942+0000","last_change":"2024-09-20T22:13:52.605374+0000","last_active":"2024-09-20T22:15:23.559942+0000","last_peered":"2024-09-20T22:15:23.559942+0000","last_clean":"2024-09-20T22:15:23.559942+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:15:23.559942+0000","last_undegraded":"2024-09-20T22:15:23.559942+0000","last_fullsized":"2024-09-20T22:15:23.559942+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:52.605277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:52.605277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":523,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559195+0000","last_change":"2024-09-20T22:14:18.440962+0000","last_active":"2024-09-20T22:15:23.559195+0000","last_peered":"2024-09-20T22:15:23.559195+0000","last_clean":"2024-09-20T22:15:23.559195+0000","last_became_active":"2024-09-20T22:12:52.521497+0000","last_became_peered":"2024-09-20T22:12:52.521497+0000","last_unstale":"2024-09-20T22:15:23.559195+0000","last_undegraded":"2024-09-20T22:15:23.559195+0000","last_fullsized":"2024-09-20T22:15:23.559195+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:18.438451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:18.438451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"328'19","reported_seq":597,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560215+0000","last_change":"2024-09-20T22:14:12.410224+0000","last_active":"2024-09-20T22:15:23.560215+0000","last_peered":"2024-09-20T22:15:23.560215+0000","last_clean":"2024-09-20T22:15:23.560215+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:15:23.560215+0000","last_undegraded":"2024-09-20T22:15:23.560215+0000","last_fullsized":"2024-09-20T22:15:23.560215+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:14:12.410107+0000","last_deep_scrub":"291'13","last_deep_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_clean_scrub_stamp":"2024-09-20T22:14:12.410107+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:29.679127+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":273,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567249+0000","last_change":"2024-09-20T22:13:11.651173+0000","last_active":"2024-09-20T22:15:23.567249+0000","last_peered":"2024-09-20T22:15:23.567249+0000","last_clean":"2024-09-20T22:15:23.567249+0000","last_became_active":"2024-09-20T22:10:30.655502+0000","last_became_peered":"2024-09-20T22:10:30.655502+0000","last_unstale":"2024-09-20T22:15:23.567249+0000","last_undegraded":"2024-09-20T22:15:23.567249+0000","last_fullsized":"2024-09-20T22:15:23.567249+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:11.650985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:11.650985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":128,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569343+0000","last_change":"2024-09-20T22:14:09.476493+0000","last_active":"2024-09-20T22:15:23.569343+0000","last_peered":"2024-09-20T22:15:23.569343+0000","last_clean":"2024-09-20T22:15:23.569343+0000","last_became_active":"2024-09-20T22:12:52.516953+0000","last_became_peered":"2024-09-20T22:12:52.516953+0000","last_unstale":"2024-09-20T22:15:23.569343+0000","last_undegraded":"2024-09-20T22:15:23.569343+0000","last_fullsized":"2024-09-20T22:15:23.569343+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:09.476386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:09.476386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:29.061649+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":136,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.568977+0000","last_change":"2024-09-20T22:15:20.820202+0000","last_active":"2024-09-20T22:15:23.568977+0000","last_peered":"2024-09-20T22:15:23.568977+0000","last_clean":"2024-09-20T22:15:23.568977+0000","last_became_active":"2024-09-20T22:12:52.514247+0000","last_became_peered":"2024-09-20T22:12:52.514247+0000","last_unstale":"2024-09-20T22:15:23.568977+0000","last_undegraded":"2024-09-20T22:15:23.568977+0000","last_fullsized":"2024-09-20T22:15:23.568977+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.819992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.819992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:39.289842+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"328'19","reported_seq":601,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566852+0000","last_change":"2024-09-20T22:13:38.718567+0000","last_active":"2024-09-20T22:15:23.566852+0000","last_peered":"2024-09-20T22:15:23.566852+0000","last_clean":"2024-09-20T22:15:23.566852+0000","last_became_active":"2024-09-20T22:12:33.545199+0000","last_became_peered":"2024-09-20T22:12:33.545199+0000","last_unstale":"2024-09-20T22:15:23.566852+0000","last_undegraded":"2024-09-20T22:15:23.566852+0000","last_fullsized":"2024-09-20T22:15:23.566852+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:13:38.718450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:38.718450+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":213,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566766+0000","last_change":"2024-09-20T22:12:05.215296+0000","last_active":"2024-09-20T22:15:23.566766+0000","last_peered":"2024-09-20T22:15:23.566766+0000","last_clean":"2024-09-20T22:15:23.566766+0000","last_became_active":"2024-09-20T22:11:30.493598+0000","last_became_peered":"2024-09-20T22:11:30.493598+0000","last_unstale":"2024-09-20T22:15:23.566766+0000","last_undegraded":"2024-09-20T22:15:23.566766+0000","last_fullsized":"2024-09-20T22:15:23.566766+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:05.215188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_clean_scrub_stamp":"2024-09-20T22:12:05.215188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":33,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":676,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566719+0000","last_change":"2024-09-20T22:14:55.338392+0000","last_active":"2024-09-20T22:15:23.566719+0000","last_peered":"2024-09-20T22:15:23.566719+0000","last_clean":"2024-09-20T22:15:23.566719+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:15:23.566719+0000","last_undegraded":"2024-09-20T22:15:23.566719+0000","last_fullsized":"2024-09-20T22:15:23.566719+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:55.338318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:55.338318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:19.629689+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":125,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569749+0000","last_change":"2024-09-20T22:14:46.684175+0000","last_active":"2024-09-20T22:15:23.569749+0000","last_peered":"2024-09-20T22:15:23.569749+0000","last_clean":"2024-09-20T22:15:23.569749+0000","last_became_active":"2024-09-20T22:12:57.883998+0000","last_became_peered":"2024-09-20T22:12:57.883998+0000","last_unstale":"2024-09-20T22:15:23.569749+0000","last_undegraded":"2024-09-20T22:15:23.569749+0000","last_fullsized":"2024-09-20T22:15:23.569749+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:46.683946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:46.683946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:46.807886+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"328'8","reported_seq":692,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560901+0000","last_change":"2024-09-20T22:14:14.979857+0000","last_active":"2024-09-20T22:15:23.560901+0000","last_peered":"2024-09-20T22:15:23.560901+0000","last_clean":"2024-09-20T22:15:23.560901+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:15:23.560901+0000","last_undegraded":"2024-09-20T22:15:23.560901+0000","last_fullsized":"2024-09-20T22:15:23.560901+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'8","last_scrub_stamp":"2024-09-20T22:14:14.979748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:14.979748+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:35.167435+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":128,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569383+0000","last_change":"2024-09-20T22:14:21.540990+0000","last_active":"2024-09-20T22:15:23.569383+0000","last_peered":"2024-09-20T22:15:23.569383+0000","last_clean":"2024-09-20T22:15:23.569383+0000","last_became_active":"2024-09-20T22:12:52.517510+0000","last_became_peered":"2024-09-20T22:12:52.517510+0000","last_unstale":"2024-09-20T22:15:23.569383+0000","last_undegraded":"2024-09-20T22:15:23.569383+0000","last_fullsized":"2024-09-20T22:15:23.569383+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:21.540878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:21.540878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:31.469685+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":699,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.564372+0000","last_change":"2024-09-20T22:15:20.569279+0000","last_active":"2024-09-20T22:15:23.564372+0000","last_peered":"2024-09-20T22:15:23.564372+0000","last_clean":"2024-09-20T22:15:23.564372+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:15:23.564372+0000","last_undegraded":"2024-09-20T22:15:23.564372+0000","last_fullsized":"2024-09-20T22:15:23.564372+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.569168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.569168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:31.279582+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":664,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566660+0000","last_change":"2024-09-20T22:13:13.656975+0000","last_active":"2024-09-20T22:15:23.566660+0000","last_peered":"2024-09-20T22:15:23.566660+0000","last_clean":"2024-09-20T22:15:23.566660+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:15:23.566660+0000","last_undegraded":"2024-09-20T22:15:23.566660+0000","last_fullsized":"2024-09-20T22:15:23.566660+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:13.656906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_clean_scrub_stamp":"2024-09-20T22:13:13.656906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":561,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560449+0000","last_change":"2024-09-20T22:14:13.408076+0000","last_active":"2024-09-20T22:15:23.560449+0000","last_peered":"2024-09-20T22:15:23.560449+0000","last_clean":"2024-09-20T22:15:23.560449+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:15:23.560449+0000","last_undegraded":"2024-09-20T22:15:23.560449+0000","last_fullsized":"2024-09-20T22:15:23.560449+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:13.407980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:13.407980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:41.846519+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":136,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.570731+0000","last_change":"2024-09-20T22:15:18.828980+0000","last_active":"2024-09-20T22:15:23.570731+0000","last_peered":"2024-09-20T22:15:23.570731+0000","last_clean":"2024-09-20T22:15:23.570731+0000","last_became_active":"2024-09-20T22:12:52.518076+0000","last_became_peered":"2024-09-20T22:12:52.518076+0000","last_unstale":"2024-09-20T22:15:23.570731+0000","last_undegraded":"2024-09-20T22:15:23.570731+0000","last_fullsized":"2024-09-20T22:15:23.570731+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:18.828903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:18.828903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:26.559172+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"328'19","reported_seq":720,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566340+0000","last_change":"2024-09-20T22:14:15.443443+0000","last_active":"2024-09-20T22:15:23.566340+0000","last_peered":"2024-09-20T22:15:23.566340+0000","last_clean":"2024-09-20T22:15:23.566340+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:15:23.566340+0000","last_undegraded":"2024-09-20T22:15:23.566340+0000","last_fullsized":"2024-09-20T22:15:23.566340+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:14:15.443321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:15.443321+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:24.219738+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":128,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569663+0000","last_change":"2024-09-20T22:14:17.434098+0000","last_active":"2024-09-20T22:15:23.569663+0000","last_peered":"2024-09-20T22:15:23.569663+0000","last_clean":"2024-09-20T22:15:23.569663+0000","last_became_active":"2024-09-20T22:12:52.524425+0000","last_became_peered":"2024-09-20T22:12:52.524425+0000","last_unstale":"2024-09-20T22:15:23.569663+0000","last_undegraded":"2024-09-20T22:15:23.569663+0000","last_fullsized":"2024-09-20T22:15:23.569663+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:17.433977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:17.433977+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":1600520192,"num_objects":24422,"num_object_clones":0,"num_object_copies":73266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24422,"num_write_kb":1563008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4697686016,"data_stored":4697686016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":24422,"ondisk_log_size":24422,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":3300,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":716,"ondisk_log_size":716,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":732,"num_read_kb":727,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":918048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1408,"ondisk_log_size":1408,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813136027,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":947800,"kb_used_data":334920,"kb_used_omap":1,"kb_used_meta":612862,"kb_avail":92797352,"statfs":{"total":95995035648,"available":95024488448,"internally_reserved":0,"allocated":342958080,"data_stored":342608979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":627571146},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430365,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1151108,"kb_used_data":710388,"kb_used_omap":1,"kb_used_meta":440702,"kb_avail":92594044,"statfs":{"total":95995035648,"available":94816301056,"internally_reserved":0,"allocated":727437312,"data_stored":727108489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":451279302},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724707,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1055552,"kb_used_data":420464,"kb_used_omap":1,"kb_used_meta":635070,"kb_avail":92689600,"statfs":{"total":95995035648,"available":94914150400,"internally_reserved":0,"allocated":430555136,"data_stored":430229328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":650312133},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986347,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1124888,"kb_used_data":656392,"kb_used_omap":1,"kb_used_meta":468478,"kb_avail":92620264,"statfs":{"total":95995035648,"available":94843150336,"internally_reserved":0,"allocated":672145408,"data_stored":671795596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":479721925},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770258024,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":763116,"kb_used_data":308636,"kb_used_omap":4,"kb_used_meta":454459,"kb_avail":92982036,"statfs":{"total":95995035648,"available":95213604864,"internally_reserved":0,"allocated":316043264,"data_stored":315674269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":465366696},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149872,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1127976,"kb_used_data":456792,"kb_used_omap":1,"kb_used_meta":671166,"kb_avail":92617176,"statfs":{"total":95995035648,"available":94839988224,"internally_reserved":0,"allocated":467755008,"data_stored":467390104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":687274439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411509,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1061568,"kb_used_data":538928,"kb_used_omap":1,"kb_used_meta":522622,"kb_avail":92683584,"statfs":{"total":95995035648,"available":94907990016,"internally_reserved":0,"allocated":551862272,"data_stored":551536907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":535165381},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738560,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1611604,"kb_used_data":958468,"kb_used_omap":1,"kb_used_meta":653118,"kb_avail":92133548,"statfs":{"total":95995035648,"available":94344753152,"internally_reserved":0,"allocated":981471232,"data_stored":981125705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":668793285},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,4,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":995098624,"data_stored":995098624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":331939840,"data_stored":331939840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":571932672,"data_stored":571932672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":490668032,"data_stored":490668032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":720306176,"data_stored":720306176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":474415104,"data_stored":474415104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":755564544,"data_stored":755564544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":357761024,"data_stored":357761024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:15:33.010 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:15:33.033 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:33.066 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:33.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:33.067 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:33.168 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:33.192 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:33.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:33.194 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:33.294 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:33.315 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:33.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:33.316 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:33.417 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:33.447 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:33.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:33.448 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:33.548 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:33.585 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:33.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:33.586 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:33.606 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:33 smithi060 ceph-mon[29694]: from='client.25877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:15:33.607 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:33 smithi060 ceph-mon[29694]: pgmap v674: 158 pgs: 158 active+clean; 1.8 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.0 KiB/s rd, 148 MiB/s wr, 2.37k op/s 2024-09-20T22:15:33.607 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:33 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:15:33.607 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:33 smithi060 ceph-mon[29694]: 3.a scrub starts 2024-09-20T22:15:33.607 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:33 smithi060 ceph-mon[29694]: 3.a scrub ok 2024-09-20T22:15:33.607 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:33 smithi060 ceph-mon[40290]: from='client.25877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:15:33.607 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:33 smithi060 ceph-mon[40290]: pgmap v674: 158 pgs: 158 active+clean; 1.8 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.0 KiB/s rd, 148 MiB/s wr, 2.37k op/s 2024-09-20T22:15:33.607 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:33 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:15:33.607 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:33 smithi060 ceph-mon[40290]: 3.a scrub starts 2024-09-20T22:15:33.607 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:33 smithi060 ceph-mon[40290]: 3.a scrub ok 2024-09-20T22:15:33.687 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:33.689 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:33.717 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:33.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:33.718 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:33.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:33 smithi067 ceph-mon[33511]: from='client.25877 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:15:33.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:33 smithi067 ceph-mon[33511]: pgmap v674: 158 pgs: 158 active+clean; 1.8 GiB data, 9.8 GiB used, 705 GiB / 715 GiB avail; 1.0 KiB/s rd, 148 MiB/s wr, 2.37k op/s 2024-09-20T22:15:33.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:33 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:15:33.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:33 smithi067 ceph-mon[33511]: 3.a scrub starts 2024-09-20T22:15:33.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:33 smithi067 ceph-mon[33511]: 3.a scrub ok 2024-09-20T22:15:33.774 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:33.820 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:33.854 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:33.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:33.855 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:33.918 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 16 16 36549 36533 142.68 91.25 0.00641032 0.00700343 2024-09-20T22:15:33.956 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:33.978 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:33.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:33.979 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:34.079 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:34.115 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:34.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:34.116 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:34.217 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:34.260 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:34.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:34.261 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:34.361 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:34.404 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:34.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:34.405 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:34.416 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:15:34.506 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:34.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:34 smithi067 ceph-mon[33511]: 6.d scrub starts 2024-09-20T22:15:34.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:34 smithi067 ceph-mon[33511]: 6.d scrub ok 2024-09-20T22:15:34.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:34 smithi067 ceph-mon[33511]: 5.1c scrub starts 2024-09-20T22:15:34.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:34 smithi067 ceph-mon[33511]: 5.1c scrub ok 2024-09-20T22:15:34.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:34 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2932905896' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:15:34.539 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:34.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:34.540 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:34.641 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:34.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:34 smithi060 ceph-mon[29694]: 6.d scrub starts 2024-09-20T22:15:34.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:34 smithi060 ceph-mon[29694]: 6.d scrub ok 2024-09-20T22:15:34.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:34 smithi060 ceph-mon[29694]: 5.1c scrub starts 2024-09-20T22:15:34.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:34 smithi060 ceph-mon[29694]: 5.1c scrub ok 2024-09-20T22:15:34.652 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:34 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2932905896' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:15:34.653 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:34 smithi060 ceph-mon[40290]: 6.d scrub starts 2024-09-20T22:15:34.653 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:34 smithi060 ceph-mon[40290]: 6.d scrub ok 2024-09-20T22:15:34.653 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:34 smithi060 ceph-mon[40290]: 5.1c scrub starts 2024-09-20T22:15:34.653 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:34 smithi060 ceph-mon[40290]: 5.1c scrub ok 2024-09-20T22:15:34.653 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:34 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2932905896' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:15:34.664 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:34.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:34.665 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:34.766 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:34.788 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:34.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:34.789 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:34.889 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:34.918 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 17 16 37897 37881 139.243 84.25 0.0069884 0.0071756 2024-09-20T22:15:34.926 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:15:34.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:34.931 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:15:34.931 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:34.931 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:35.032 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:35.066 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:35.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:35.067 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:35.168 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:35.190 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:35.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:35.191 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:35.285 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:15:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:15:35.292 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:35.332 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:35.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:35.333 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:35.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:35.477 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:35.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:35.478 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:35.518 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:15:35.518 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:15:35.518 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:15:35.518 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:15:35.518 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:15:35.518 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:15:35.518 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:15:35.518 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:15:35.519 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:15:35.519 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:15:35.519 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:15:35.519 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:15:35.519 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:15:35.519 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:15:35.519 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:15:35.519 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:15:35.578 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:15:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:15:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:38985 10.1 10:892bc34b:::benchmark_data_smithi060_167766_object38984:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e380)", 2024-09-20T22:15:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:15:35.561434+0000", 2024-09-20T22:15:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.014398948999999999, 2024-09-20T22:15:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.014423505999999999, 2024-09-20T22:15:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:15:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:15:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:15:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 38985 2024-09-20T22:15:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.561434+0000", 2024-09-20T22:15:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.561434+0000", 2024-09-20T22:15:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.561436+0000", 2024-09-20T22:15:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2139999999999999e-06 2024-09-20T22:15:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.561479+0000", 2024-09-20T22:15:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3682e-05 2024-09-20T22:15:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.561490+0000", 2024-09-20T22:15:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0356999999999999e-05 2024-09-20T22:15:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.561496+0000", 2024-09-20T22:15:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8329999999999999e-06 2024-09-20T22:15:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.561537+0000", 2024-09-20T22:15:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1239000000000003e-05 2024-09-20T22:15:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.561613+0000", 2024-09-20T22:15:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5767999999999997e-05 2024-09-20T22:15:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:15:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.561692+0000", 2024-09-20T22:15:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9907000000000006e-05 2024-09-20T22:15:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.562803+0000", 2024-09-20T22:15:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001110237 2024-09-20T22:15:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.563817+0000", 2024-09-20T22:15:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0010138160000000001 2024-09-20T22:15:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop_reply(client.16005.0:38985 10.1 e380/370)", 2024-09-20T22:15:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:15:35.564152+0000", 2024-09-20T22:15:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.011681498, 2024-09-20T22:15:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.011755923999999999, 2024-09-20T22:15:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "queued for pg", 2024-09-20T22:15:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.564152+0000", 2024-09-20T22:15:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.564152+0000", 2024-09-20T22:15:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.564152+0000", 2024-09-20T22:15:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4999999999999998e-07 2024-09-20T22:15:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.564155+0000", 2024-09-20T22:15:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.3800000000000001e-06 2024-09-20T22:15:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.564163+0000", 2024-09-20T22:15:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2500000000000006e-06 2024-09-20T22:15:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.564167+0000", 2024-09-20T22:15:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7409999999999998e-06 2024-09-20T22:15:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:38966 10.e 10:71103557:::benchmark_data_smithi060_167766_object38965:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e380)", 2024-09-20T22:15:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:15:35.551499+0000", 2024-09-20T22:15:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.024334597999999999, 2024-09-20T22:15:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.024440527, 2024-09-20T22:15:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:15:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:15:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:15:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 38966 2024-09-20T22:15:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.551499+0000", 2024-09-20T22:15:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.551499+0000", 2024-09-20T22:15:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.551500+0000", 2024-09-20T22:15:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5349999999999999e-06 2024-09-20T22:15:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.551530+0000", 2024-09-20T22:15:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.9652e-05 2024-09-20T22:15:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.551542+0000", 2024-09-20T22:15:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2048000000000001e-05 2024-09-20T22:15:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.551549+0000", 2024-09-20T22:15:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0339999999999999e-06 2024-09-20T22:15:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.551607+0000", 2024-09-20T22:15:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7466999999999997e-05 2024-09-20T22:15:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.551727+0000", 2024-09-20T22:15:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000120708 2024-09-20T22:15:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,6", 2024-09-20T22:15:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.551904+0000", 2024-09-20T22:15:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017647499999999999 2024-09-20T22:15:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.555050+0000", 2024-09-20T22:15:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003146228 2024-09-20T22:15:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:35.555512+0000", 2024-09-20T22:15:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00046243599999999999 2024-09-20T22:15:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-20T22:15:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: "num_ops": 3 2024-09-20T22:15:35.591 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:15:35.591 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:35.630 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:35.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:35.631 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:35.731 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:35.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:35 smithi067 ceph-mon[33511]: 5.13 scrub starts 2024-09-20T22:15:35.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:35 smithi067 ceph-mon[33511]: 5.13 scrub ok 2024-09-20T22:15:35.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:35 smithi067 ceph-mon[33511]: pgmap v675: 158 pgs: 158 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 960 B/s rd, 121 MiB/s wr, 1.94k op/s 2024-09-20T22:15:35.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:35 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:15:35.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:35 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2932905896' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:15:35.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:35 smithi067 ceph-mon[33511]: osdmap e379: 8 total, 8 up, 8 in 2024-09-20T22:15:35.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:35 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/851771477' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:35.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:35 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:15:35.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:35 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:15:35.772 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:35.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:35.774 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:35.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-mon[29694]: 5.13 scrub starts 2024-09-20T22:15:35.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-mon[29694]: 5.13 scrub ok 2024-09-20T22:15:35.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-mon[29694]: pgmap v675: 158 pgs: 158 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 960 B/s rd, 121 MiB/s wr, 1.94k op/s 2024-09-20T22:15:35.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:15:35.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2932905896' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:15:35.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-mon[29694]: osdmap e379: 8 total, 8 up, 8 in 2024-09-20T22:15:35.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/851771477' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:35.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:15:35.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:15:35.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-mon[40290]: 5.13 scrub starts 2024-09-20T22:15:35.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-mon[40290]: 5.13 scrub ok 2024-09-20T22:15:35.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-mon[40290]: pgmap v675: 158 pgs: 158 active+clean; 2.0 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 960 B/s rd, 121 MiB/s wr, 1.94k op/s 2024-09-20T22:15:35.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:15:35.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2932905896' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:15:35.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-mon[40290]: osdmap e379: 8 total, 8 up, 8 in 2024-09-20T22:15:35.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/851771477' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:35.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:15:35.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:35 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:15:35.875 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:35.912 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:35.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:35.913 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:35.918 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 18 16 39546 39530 137.233 103.062 0.00634747 0.00728171 2024-09-20T22:15:36.013 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:36.047 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:36.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:36.048 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:36.149 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:36.188 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:36.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:36.189 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:36.290 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:36.329 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:36.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:36.330 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:36.431 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:36.471 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:36.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:36.472 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:36.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:36 smithi060 ceph-mon[29694]: from='client.25889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:15:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:36 smithi060 ceph-mon[29694]: osdmap e380: 8 total, 8 up, 8 in 2024-09-20T22:15:36.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:36 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:15:36.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:36 smithi060 ceph-mon[40290]: from='client.25889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:15:36.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:36 smithi060 ceph-mon[40290]: osdmap e380: 8 total, 8 up, 8 in 2024-09-20T22:15:36.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:36 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:15:36.573 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:36.611 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:36.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:36.612 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:36.712 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:36.747 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:36.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:36.748 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:36.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:36 smithi067 ceph-mon[33511]: from='client.25889 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:15:36.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:36 smithi067 ceph-mon[33511]: osdmap e380: 8 total, 8 up, 8 in 2024-09-20T22:15:36.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:36 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:15:36.850 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:36.891 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:36.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:36.891 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:36.918 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 19 16 41473 41457 136.348 120.438 0.00830293 0.00732919 2024-09-20T22:15:36.992 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:37.028 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:37.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:37.029 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:37.069 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":676,"stamp":"2024-09-20T22:15:34.418640+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2195855899,"num_objects":33696,"num_object_clones":0,"num_object_copies":101088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33696,"num_whiteouts":0,"num_read":1871,"num_read_kb":1623,"num_write":34219,"num_write_kb":2143853,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":923923,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":35736,"ondisk_log_size":35736,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":10589084,"kb_used_data":6155356,"kb_used_omap":15,"kb_used_meta":4433520,"kb_avail":739372132,"statfs":{"total":767960285184,"available":757117063168,"internally_reserved":0,"allocated":6303084544,"data_stored":6300326209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":4539925267},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1357709312,"num_objects":20717,"num_object_clones":0,"num_object_copies":62151,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20717,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":20717,"num_write_kb":1325888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.201236"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":128,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569482+0000","last_change":"2024-09-20T22:14:57.573764+0000","last_active":"2024-09-20T22:15:23.569482+0000","last_peered":"2024-09-20T22:15:23.569482+0000","last_clean":"2024-09-20T22:15:23.569482+0000","last_became_active":"2024-09-20T22:12:52.515921+0000","last_became_peered":"2024-09-20T22:12:52.515921+0000","last_unstale":"2024-09-20T22:15:23.569482+0000","last_undegraded":"2024-09-20T22:15:23.569482+0000","last_fullsized":"2024-09-20T22:15:23.569482+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:57.573679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_clean_scrub_stamp":"2024-09-20T22:14:57.573679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:20.753739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"327'19","reported_seq":718,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560653+0000","last_change":"2024-09-20T22:15:22.984401+0000","last_active":"2024-09-20T22:15:23.560653+0000","last_peered":"2024-09-20T22:15:23.560653+0000","last_clean":"2024-09-20T22:15:23.560653+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:15:23.560653+0000","last_undegraded":"2024-09-20T22:15:23.560653+0000","last_fullsized":"2024-09-20T22:15:23.560653+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"327'19","last_scrub_stamp":"2024-09-20T22:15:22.984271+0000","last_deep_scrub":"252'13","last_deep_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_clean_scrub_stamp":"2024-09-20T22:15:22.984271+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:25.020372+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":563,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559455+0000","last_change":"2024-09-20T22:14:15.391149+0000","last_active":"2024-09-20T22:15:23.559455+0000","last_peered":"2024-09-20T22:15:23.559455+0000","last_clean":"2024-09-20T22:15:23.559455+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:15:23.559455+0000","last_undegraded":"2024-09-20T22:15:23.559455+0000","last_fullsized":"2024-09-20T22:15:23.559455+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:15.391049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:15.391049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:37.227878+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":685,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561123+0000","last_change":"2024-09-20T22:15:18.062035+0000","last_active":"2024-09-20T22:15:23.561123+0000","last_peered":"2024-09-20T22:15:23.561123+0000","last_clean":"2024-09-20T22:15:23.561123+0000","last_became_active":"2024-09-20T22:11:30.485735+0000","last_became_peered":"2024-09-20T22:11:30.485735+0000","last_unstale":"2024-09-20T22:15:23.561123+0000","last_undegraded":"2024-09-20T22:15:23.561123+0000","last_fullsized":"2024-09-20T22:15:23.561123+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:18.061924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:18.061924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:19.499592+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":672,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.564130+0000","last_change":"2024-09-20T22:14:56.160729+0000","last_active":"2024-09-20T22:15:23.564130+0000","last_peered":"2024-09-20T22:15:23.564130+0000","last_clean":"2024-09-20T22:15:23.564130+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:15:23.564130+0000","last_undegraded":"2024-09-20T22:15:23.564130+0000","last_fullsized":"2024-09-20T22:15:23.564130+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:14:56.160638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:56.160638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:07.044469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":155,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565418+0000","last_change":"2024-09-20T22:14:52.402426+0000","last_active":"2024-09-20T22:15:23.565418+0000","last_peered":"2024-09-20T22:15:23.565418+0000","last_clean":"2024-09-20T22:15:23.565418+0000","last_became_active":"2024-09-20T22:12:33.545100+0000","last_became_peered":"2024-09-20T22:12:33.545100+0000","last_unstale":"2024-09-20T22:15:23.565418+0000","last_undegraded":"2024-09-20T22:15:23.565418+0000","last_fullsized":"2024-09-20T22:15:23.565418+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:52.402299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:52.402299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:13.318752+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"328'27","reported_seq":271,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565153+0000","last_change":"2024-09-20T22:15:20.902301+0000","last_active":"2024-09-20T22:15:23.565153+0000","last_peered":"2024-09-20T22:15:23.565153+0000","last_clean":"2024-09-20T22:15:23.565153+0000","last_became_active":"2024-09-20T22:12:52.517296+0000","last_became_peered":"2024-09-20T22:12:52.517296+0000","last_unstale":"2024-09-20T22:15:23.565153+0000","last_undegraded":"2024-09-20T22:15:23.565153+0000","last_fullsized":"2024-09-20T22:15:23.565153+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:15:20.902179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.902179+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:24.675476+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":677,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:26.080107+0000","last_change":"2024-09-20T22:15:26.080144+0000","last_active":"2024-09-20T22:15:26.080107+0000","last_peered":"2024-09-20T22:15:26.080107+0000","last_clean":"2024-09-20T22:15:26.080107+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:15:26.080107+0000","last_undegraded":"2024-09-20T22:15:26.080107+0000","last_fullsized":"2024-09-20T22:15:26.080107+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:26.080040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:26.080040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:31.327638+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":699,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566268+0000","last_change":"2024-09-20T22:15:21.178920+0000","last_active":"2024-09-20T22:15:23.566268+0000","last_peered":"2024-09-20T22:15:23.566268+0000","last_clean":"2024-09-20T22:15:23.566268+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:15:23.566268+0000","last_undegraded":"2024-09-20T22:15:23.566268+0000","last_fullsized":"2024-09-20T22:15:23.566268+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:21.178806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:15:21.178806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:36.671122+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"328'15","reported_seq":737,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561282+0000","last_change":"2024-09-20T22:14:19.076811+0000","last_active":"2024-09-20T22:15:23.561282+0000","last_peered":"2024-09-20T22:15:23.561282+0000","last_clean":"2024-09-20T22:15:23.561282+0000","last_became_active":"2024-09-20T22:12:52.516894+0000","last_became_peered":"2024-09-20T22:12:52.516894+0000","last_unstale":"2024-09-20T22:15:23.561282+0000","last_undegraded":"2024-09-20T22:15:23.561282+0000","last_fullsized":"2024-09-20T22:15:23.561282+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:14:19.076696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:19.076696+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:45.728095+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":230,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566256+0000","last_change":"2024-09-20T22:14:54.153118+0000","last_active":"2024-09-20T22:15:23.566256+0000","last_peered":"2024-09-20T22:15:23.566256+0000","last_clean":"2024-09-20T22:15:23.566256+0000","last_became_active":"2024-09-20T22:12:52.518549+0000","last_became_peered":"2024-09-20T22:12:52.518549+0000","last_unstale":"2024-09-20T22:15:23.566256+0000","last_undegraded":"2024-09-20T22:15:23.566256+0000","last_fullsized":"2024-09-20T22:15:23.566256+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:54.152717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:54.152717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:03.431527+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":136,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569479+0000","last_change":"2024-09-20T22:14:58.570997+0000","last_active":"2024-09-20T22:15:23.569479+0000","last_peered":"2024-09-20T22:15:23.569479+0000","last_clean":"2024-09-20T22:15:23.569479+0000","last_became_active":"2024-09-20T22:12:52.524631+0000","last_became_peered":"2024-09-20T22:12:52.524631+0000","last_unstale":"2024-09-20T22:15:23.569479+0000","last_undegraded":"2024-09-20T22:15:23.569479+0000","last_fullsized":"2024-09-20T22:15:23.569479+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:58.570923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:58.570923+0000","last_clean_scrub_stamp":"2024-09-20T22:14:58.570923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:25.107938+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":704,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566051+0000","last_change":"2024-09-20T22:15:20.182058+0000","last_active":"2024-09-20T22:15:23.566051+0000","last_peered":"2024-09-20T22:15:23.566051+0000","last_clean":"2024-09-20T22:15:23.566051+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:15:23.566051+0000","last_undegraded":"2024-09-20T22:15:23.566051+0000","last_fullsized":"2024-09-20T22:15:23.566051+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.181942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:30.215263+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.181942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:46.935103+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"328'7","reported_seq":706,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561504+0000","last_change":"2024-09-20T22:13:44.057396+0000","last_active":"2024-09-20T22:15:23.561504+0000","last_peered":"2024-09-20T22:15:23.561504+0000","last_clean":"2024-09-20T22:15:23.561504+0000","last_became_active":"2024-09-20T22:08:22.144364+0000","last_became_peered":"2024-09-20T22:08:22.144364+0000","last_unstale":"2024-09-20T22:15:23.561504+0000","last_undegraded":"2024-09-20T22:15:23.561504+0000","last_fullsized":"2024-09-20T22:15:23.561504+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'7","last_scrub_stamp":"2024-09-20T22:13:44.057245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:44.057245+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":685,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566796+0000","last_change":"2024-09-20T22:14:57.354434+0000","last_active":"2024-09-20T22:15:23.566796+0000","last_peered":"2024-09-20T22:15:23.566796+0000","last_clean":"2024-09-20T22:15:23.566796+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:15:23.566796+0000","last_undegraded":"2024-09-20T22:15:23.566796+0000","last_fullsized":"2024-09-20T22:15:23.566796+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:57.354326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:57.354326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:14.925355+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":218,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.564678+0000","last_change":"2024-09-20T22:15:00.077644+0000","last_active":"2024-09-20T22:15:23.564678+0000","last_peered":"2024-09-20T22:15:23.564678+0000","last_clean":"2024-09-20T22:15:23.564678+0000","last_became_active":"2024-09-20T22:11:30.487137+0000","last_became_peered":"2024-09-20T22:11:30.487137+0000","last_unstale":"2024-09-20T22:15:23.564678+0000","last_undegraded":"2024-09-20T22:15:23.564678+0000","last_fullsized":"2024-09-20T22:15:23.564678+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:00.077519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:15:00.077519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:11.908799+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":673,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.562352+0000","last_change":"2024-09-20T22:15:17.029127+0000","last_active":"2024-09-20T22:15:23.562352+0000","last_peered":"2024-09-20T22:15:23.562352+0000","last_clean":"2024-09-20T22:15:23.562352+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:15:23.562352+0000","last_undegraded":"2024-09-20T22:15:23.562352+0000","last_fullsized":"2024-09-20T22:15:23.562352+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:17.029051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:15:17.029051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:29.616781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":136,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:24.747969+0000","last_change":"2024-09-20T22:15:24.748006+0000","last_active":"2024-09-20T22:15:24.747969+0000","last_peered":"2024-09-20T22:15:24.747969+0000","last_clean":"2024-09-20T22:15:24.747969+0000","last_became_active":"2024-09-20T22:12:52.526097+0000","last_became_peered":"2024-09-20T22:12:52.526097+0000","last_unstale":"2024-09-20T22:15:24.747969+0000","last_undegraded":"2024-09-20T22:15:24.747969+0000","last_fullsized":"2024-09-20T22:15:24.747969+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.747891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.747891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:29.188832+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"328'34","reported_seq":274,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559012+0000","last_change":"2024-09-20T22:14:50.993713+0000","last_active":"2024-09-20T22:15:23.559012+0000","last_peered":"2024-09-20T22:15:23.559012+0000","last_clean":"2024-09-20T22:15:23.559012+0000","last_became_active":"2024-09-20T22:11:12.507311+0000","last_became_peered":"2024-09-20T22:11:12.507311+0000","last_unstale":"2024-09-20T22:15:23.559012+0000","last_undegraded":"2024-09-20T22:15:23.559012+0000","last_fullsized":"2024-09-20T22:15:23.559012+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'34","last_scrub_stamp":"2024-09-20T22:14:50.993563+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:14:50.993563+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:53.748023+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":525,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560601+0000","last_change":"2024-09-20T22:13:11.053381+0000","last_active":"2024-09-20T22:15:23.560601+0000","last_peered":"2024-09-20T22:15:23.560601+0000","last_clean":"2024-09-20T22:15:23.560601+0000","last_became_active":"2024-09-20T22:12:52.520985+0000","last_became_peered":"2024-09-20T22:12:52.520985+0000","last_unstale":"2024-09-20T22:15:23.560601+0000","last_undegraded":"2024-09-20T22:15:23.560601+0000","last_fullsized":"2024-09-20T22:15:23.560601+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:11.053276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:11.053276+0000","last_clean_scrub_stamp":"2024-09-20T22:13:11.053276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":624,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566012+0000","last_change":"2024-09-20T22:14:19.451519+0000","last_active":"2024-09-20T22:15:23.566012+0000","last_peered":"2024-09-20T22:15:23.566012+0000","last_clean":"2024-09-20T22:15:23.566012+0000","last_became_active":"2024-09-20T22:12:21.521433+0000","last_became_peered":"2024-09-20T22:12:21.521433+0000","last_unstale":"2024-09-20T22:15:23.566012+0000","last_undegraded":"2024-09-20T22:15:23.566012+0000","last_fullsized":"2024-09-20T22:15:23.566012+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:19.451409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:19.451409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:42.318241+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":625,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565102+0000","last_change":"2024-09-20T22:12:44.197752+0000","last_active":"2024-09-20T22:15:23.565102+0000","last_peered":"2024-09-20T22:15:23.565102+0000","last_clean":"2024-09-20T22:15:23.565102+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:15:23.565102+0000","last_undegraded":"2024-09-20T22:15:23.565102+0000","last_fullsized":"2024-09-20T22:15:23.565102+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:44.197654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:44.197654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"328'23","reported_seq":598,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560790+0000","last_change":"2024-09-20T22:15:17.533607+0000","last_active":"2024-09-20T22:15:23.560790+0000","last_peered":"2024-09-20T22:15:23.560790+0000","last_clean":"2024-09-20T22:15:23.560790+0000","last_became_active":"2024-09-20T22:12:21.517147+0000","last_became_peered":"2024-09-20T22:12:21.517147+0000","last_unstale":"2024-09-20T22:15:23.560790+0000","last_undegraded":"2024-09-20T22:15:23.560790+0000","last_fullsized":"2024-09-20T22:15:23.560790+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'23","last_scrub_stamp":"2024-09-20T22:15:17.533453+0000","last_deep_scrub":"250'17","last_deep_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_clean_scrub_stamp":"2024-09-20T22:15:17.533453+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:38.348665+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":668,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561565+0000","last_change":"2024-09-20T22:13:51.133410+0000","last_active":"2024-09-20T22:15:23.561565+0000","last_peered":"2024-09-20T22:15:23.561565+0000","last_clean":"2024-09-20T22:15:23.561565+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:15:23.561565+0000","last_undegraded":"2024-09-20T22:15:23.561565+0000","last_fullsized":"2024-09-20T22:15:23.561565+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:51.133308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.133308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":666,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566888+0000","last_change":"2024-09-20T22:13:52.688758+0000","last_active":"2024-09-20T22:15:23.566888+0000","last_peered":"2024-09-20T22:15:23.566888+0000","last_clean":"2024-09-20T22:15:23.566888+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:15:23.566888+0000","last_undegraded":"2024-09-20T22:15:23.566888+0000","last_fullsized":"2024-09-20T22:15:23.566888+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:13:52.688650+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:13:52.688650+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"289'2","reported_seq":523,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559543+0000","last_change":"2024-09-20T22:14:22.436454+0000","last_active":"2024-09-20T22:15:23.559543+0000","last_peered":"2024-09-20T22:15:23.559543+0000","last_clean":"2024-09-20T22:15:23.559543+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:15:23.559543+0000","last_undegraded":"2024-09-20T22:15:23.559543+0000","last_fullsized":"2024-09-20T22:15:23.559543+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-20T22:14:22.436308+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_clean_scrub_stamp":"2024-09-20T22:14:22.436308+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:51.653333+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"328'20","reported_seq":155,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.570263+0000","last_change":"2024-09-20T22:14:11.444573+0000","last_active":"2024-09-20T22:15:23.570263+0000","last_peered":"2024-09-20T22:15:23.570263+0000","last_clean":"2024-09-20T22:15:23.570263+0000","last_became_active":"2024-09-20T22:12:52.526803+0000","last_became_peered":"2024-09-20T22:12:52.526803+0000","last_unstale":"2024-09-20T22:15:23.570263+0000","last_undegraded":"2024-09-20T22:15:23.570263+0000","last_fullsized":"2024-09-20T22:15:23.570263+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:14:11.444444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:11.444444+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.306318+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"291'11","reported_seq":877,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:28.500785+0000","last_change":"2024-09-20T22:15:28.500785+0000","last_active":"2024-09-20T22:15:28.500785+0000","last_peered":"2024-09-20T22:15:28.500785+0000","last_clean":"2024-09-20T22:15:28.500785+0000","last_became_active":"2024-09-20T22:11:40.601048+0000","last_became_peered":"2024-09-20T22:11:40.601048+0000","last_unstale":"2024-09-20T22:15:28.500785+0000","last_undegraded":"2024-09-20T22:15:28.500785+0000","last_fullsized":"2024-09-20T22:15:28.500785+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'11","last_scrub_stamp":"2024-09-20T22:15:28.500437+0000","last_deep_scrub":"291'11","last_deep_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_clean_scrub_stamp":"2024-09-20T22:15:28.500437+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:30.705651+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"378'1859","reported_seq":1885,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:30.789861+0000","last_change":"2024-09-20T22:15:08.927837+0000","last_active":"2024-09-20T22:15:30.789861+0000","last_peered":"2024-09-20T22:15:30.789861+0000","last_clean":"2024-09-20T22:15:30.789861+0000","last_became_active":"2024-09-20T22:15:08.927535+0000","last_became_peered":"2024-09-20T22:15:08.927535+0000","last_unstale":"2024-09-20T22:15:30.789861+0000","last_undegraded":"2024-09-20T22:15:30.789861+0000","last_fullsized":"2024-09-20T22:15:30.789861+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1859,"log_dups_size":0,"ondisk_log_size":1859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:17.347573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":121831424,"num_objects":1859,"num_object_clones":0,"num_object_copies":5577,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1859,"num_write_kb":118976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":680,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.564855+0000","last_change":"2024-09-20T22:13:08.407055+0000","last_active":"2024-09-20T22:15:23.564855+0000","last_peered":"2024-09-20T22:15:23.564855+0000","last_clean":"2024-09-20T22:15:23.564855+0000","last_became_active":"2024-09-20T22:08:48.452786+0000","last_became_peered":"2024-09-20T22:08:48.452786+0000","last_unstale":"2024-09-20T22:15:23.564855+0000","last_undegraded":"2024-09-20T22:15:23.564855+0000","last_fullsized":"2024-09-20T22:15:23.564855+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:08.406883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:13:08.406883+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"328'27","reported_seq":468,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559731+0000","last_change":"2024-09-20T22:14:46.998282+0000","last_active":"2024-09-20T22:15:23.559731+0000","last_peered":"2024-09-20T22:15:23.559731+0000","last_clean":"2024-09-20T22:15:23.559731+0000","last_became_active":"2024-09-20T22:08:22.145408+0000","last_became_peered":"2024-09-20T22:08:22.145408+0000","last_unstale":"2024-09-20T22:15:23.559731+0000","last_undegraded":"2024-09-20T22:15:23.559731+0000","last_fullsized":"2024-09-20T22:15:23.559731+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:14:46.998142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:46.998142+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:49.977452+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":692,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:28.020403+0000","last_change":"2024-09-20T22:15:28.020435+0000","last_active":"2024-09-20T22:15:28.020403+0000","last_peered":"2024-09-20T22:15:28.020403+0000","last_clean":"2024-09-20T22:15:28.020403+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:15:28.020403+0000","last_undegraded":"2024-09-20T22:15:28.020403+0000","last_fullsized":"2024-09-20T22:15:28.020403+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:28.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:28.020341+0000","last_clean_scrub_stamp":"2024-09-20T22:15:28.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:31.565898+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"326'13","reported_seq":898,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:25.323240+0000","last_change":"2024-09-20T22:14:50.079897+0000","last_active":"2024-09-20T22:15:25.323240+0000","last_peered":"2024-09-20T22:15:25.323240+0000","last_clean":"2024-09-20T22:15:25.323240+0000","last_became_active":"2024-09-20T22:12:52.517588+0000","last_became_peered":"2024-09-20T22:12:52.517588+0000","last_unstale":"2024-09-20T22:15:25.323240+0000","last_undegraded":"2024-09-20T22:15:25.323240+0000","last_fullsized":"2024-09-20T22:15:25.323240+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'13","last_scrub_stamp":"2024-09-20T22:14:50.079818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:50.079818+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:57.638038+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":222,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566028+0000","last_change":"2024-09-20T22:12:52.518570+0000","last_active":"2024-09-20T22:15:23.566028+0000","last_peered":"2024-09-20T22:15:23.566028+0000","last_clean":"2024-09-20T22:15:23.566028+0000","last_became_active":"2024-09-20T22:12:52.517826+0000","last_became_peered":"2024-09-20T22:12:52.517826+0000","last_unstale":"2024-09-20T22:15:23.566028+0000","last_undegraded":"2024-09-20T22:15:23.566028+0000","last_fullsized":"2024-09-20T22:15:23.566028+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:07.114531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:12:07.114531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"378'1924","reported_seq":1948,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:30.798161+0000","last_change":"2024-09-20T22:15:08.928365+0000","last_active":"2024-09-20T22:15:30.798161+0000","last_peered":"2024-09-20T22:15:30.798161+0000","last_clean":"2024-09-20T22:15:30.798161+0000","last_became_active":"2024-09-20T22:15:08.928064+0000","last_became_peered":"2024-09-20T22:15:08.928064+0000","last_unstale":"2024-09-20T22:15:30.798161+0000","last_undegraded":"2024-09-20T22:15:30.798161+0000","last_fullsized":"2024-09-20T22:15:30.798161+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1924,"log_dups_size":0,"ondisk_log_size":1924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:27.570950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":126091264,"num_objects":1924,"num_object_clones":0,"num_object_copies":5772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1924,"num_write_kb":123136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":680,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.562456+0000","last_change":"2024-09-20T22:14:21.059023+0000","last_active":"2024-09-20T22:15:23.562456+0000","last_peered":"2024-09-20T22:15:23.562456+0000","last_clean":"2024-09-20T22:15:23.562456+0000","last_became_active":"2024-09-20T22:08:48.452475+0000","last_became_peered":"2024-09-20T22:08:48.452475+0000","last_unstale":"2024-09-20T22:15:23.562456+0000","last_undegraded":"2024-09-20T22:15:23.562456+0000","last_fullsized":"2024-09-20T22:15:23.562456+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:14:21.058895+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:24.210483+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"328'30","reported_seq":688,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566824+0000","last_change":"2024-09-20T22:13:50.715007+0000","last_active":"2024-09-20T22:15:23.566824+0000","last_peered":"2024-09-20T22:15:23.566824+0000","last_clean":"2024-09-20T22:15:23.566824+0000","last_became_active":"2024-09-20T22:12:52.526867+0000","last_became_peered":"2024-09-20T22:12:52.526867+0000","last_unstale":"2024-09-20T22:15:23.566824+0000","last_undegraded":"2024-09-20T22:15:23.566824+0000","last_fullsized":"2024-09-20T22:15:23.566824+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'30","last_scrub_stamp":"2024-09-20T22:13:50.714867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:50.714867+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":169,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567840+0000","last_change":"2024-09-20T22:14:09.922166+0000","last_active":"2024-09-20T22:15:23.567840+0000","last_peered":"2024-09-20T22:15:23.567840+0000","last_clean":"2024-09-20T22:15:23.567840+0000","last_became_active":"2024-09-20T22:12:52.528947+0000","last_became_peered":"2024-09-20T22:12:52.528947+0000","last_unstale":"2024-09-20T22:15:23.567840+0000","last_undegraded":"2024-09-20T22:15:23.567840+0000","last_fullsized":"2024-09-20T22:15:23.567840+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:09.922092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:09.922092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:27.839312+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":689,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560650+0000","last_change":"2024-09-20T22:14:13.046284+0000","last_active":"2024-09-20T22:15:23.560650+0000","last_peered":"2024-09-20T22:15:23.560650+0000","last_clean":"2024-09-20T22:15:23.560650+0000","last_became_active":"2024-09-20T22:10:48.919949+0000","last_became_peered":"2024-09-20T22:10:48.919949+0000","last_unstale":"2024-09-20T22:15:23.560650+0000","last_undegraded":"2024-09-20T22:15:23.560650+0000","last_fullsized":"2024-09-20T22:15:23.560650+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:13.046189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:13.046189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":673,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561225+0000","last_change":"2024-09-20T22:14:08.158795+0000","last_active":"2024-09-20T22:15:23.561225+0000","last_peered":"2024-09-20T22:15:23.561225+0000","last_clean":"2024-09-20T22:15:23.561225+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:15:23.561225+0000","last_undegraded":"2024-09-20T22:15:23.561225+0000","last_fullsized":"2024-09-20T22:15:23.561225+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:08.158688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:08.158688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"378'2215","reported_seq":2241,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:32.848774+0000","last_change":"2024-09-20T22:15:09.088949+0000","last_active":"2024-09-20T22:15:32.848774+0000","last_peered":"2024-09-20T22:15:32.848774+0000","last_clean":"2024-09-20T22:15:32.848774+0000","last_became_active":"2024-09-20T22:15:09.088580+0000","last_became_peered":"2024-09-20T22:15:09.088580+0000","last_unstale":"2024-09-20T22:15:32.848774+0000","last_undegraded":"2024-09-20T22:15:32.848774+0000","last_fullsized":"2024-09-20T22:15:32.848774+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2215,"log_dups_size":0,"ondisk_log_size":2215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:33.130074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145162240,"num_objects":2215,"num_object_clones":0,"num_object_copies":6645,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2215,"num_write_kb":141760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":714,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:27.026044+0000","last_change":"2024-09-20T22:15:27.026088+0000","last_active":"2024-09-20T22:15:27.026044+0000","last_peered":"2024-09-20T22:15:27.026044+0000","last_clean":"2024-09-20T22:15:27.026044+0000","last_became_active":"2024-09-20T22:11:33.560632+0000","last_became_peered":"2024-09-20T22:11:33.560632+0000","last_unstale":"2024-09-20T22:15:27.026044+0000","last_undegraded":"2024-09-20T22:15:27.026044+0000","last_fullsized":"2024-09-20T22:15:27.026044+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:27.025812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.025812+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:39.819835+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"328'28","reported_seq":723,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561101+0000","last_change":"2024-09-20T22:14:55.064788+0000","last_active":"2024-09-20T22:15:23.561101+0000","last_peered":"2024-09-20T22:15:23.561101+0000","last_clean":"2024-09-20T22:15:23.561101+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:15:23.561101+0000","last_undegraded":"2024-09-20T22:15:23.561101+0000","last_fullsized":"2024-09-20T22:15:23.561101+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:14:55.064689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:55.064689+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:59.495396+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":140,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.570560+0000","last_change":"2024-09-20T22:15:15.781305+0000","last_active":"2024-09-20T22:15:23.570560+0000","last_peered":"2024-09-20T22:15:23.570560+0000","last_clean":"2024-09-20T22:15:23.570560+0000","last_became_active":"2024-09-20T22:12:52.528068+0000","last_became_peered":"2024-09-20T22:12:52.528068+0000","last_unstale":"2024-09-20T22:15:23.570560+0000","last_undegraded":"2024-09-20T22:15:23.570560+0000","last_fullsized":"2024-09-20T22:15:23.570560+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:15:15.781222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:15.781222+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:24.974390+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"315'11","reported_seq":409,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:25.323093+0000","last_change":"2024-09-20T22:13:05.177108+0000","last_active":"2024-09-20T22:15:25.323093+0000","last_peered":"2024-09-20T22:15:25.323093+0000","last_clean":"2024-09-20T22:15:25.323093+0000","last_became_active":"2024-09-20T22:12:33.545942+0000","last_became_peered":"2024-09-20T22:12:33.545942+0000","last_unstale":"2024-09-20T22:15:25.323093+0000","last_undegraded":"2024-09-20T22:15:25.323093+0000","last_fullsized":"2024-09-20T22:15:25.323093+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:13:05.176973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:05.176973+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":665,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567148+0000","last_change":"2024-09-20T22:13:10.621055+0000","last_active":"2024-09-20T22:15:23.567148+0000","last_peered":"2024-09-20T22:15:23.567148+0000","last_clean":"2024-09-20T22:15:23.567148+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:15:23.567148+0000","last_undegraded":"2024-09-20T22:15:23.567148+0000","last_fullsized":"2024-09-20T22:15:23.567148+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:10.620925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.620925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"378'1907","reported_seq":1933,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:30.798333+0000","last_change":"2024-09-20T22:15:08.931965+0000","last_active":"2024-09-20T22:15:30.798333+0000","last_peered":"2024-09-20T22:15:30.798333+0000","last_clean":"2024-09-20T22:15:30.798333+0000","last_became_active":"2024-09-20T22:15:08.931653+0000","last_became_peered":"2024-09-20T22:15:08.931653+0000","last_unstale":"2024-09-20T22:15:30.798333+0000","last_undegraded":"2024-09-20T22:15:30.798333+0000","last_fullsized":"2024-09-20T22:15:30.798333+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1907,"log_dups_size":0,"ondisk_log_size":1907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:32.473356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124977152,"num_objects":1907,"num_object_clones":0,"num_object_copies":5721,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1907,"num_write_kb":122048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":710,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565917+0000","last_change":"2024-09-20T22:14:23.503489+0000","last_active":"2024-09-20T22:15:23.565917+0000","last_peered":"2024-09-20T22:15:23.565917+0000","last_clean":"2024-09-20T22:15:23.565917+0000","last_became_active":"2024-09-20T22:10:33.660948+0000","last_became_peered":"2024-09-20T22:10:33.660948+0000","last_unstale":"2024-09-20T22:15:23.565917+0000","last_undegraded":"2024-09-20T22:15:23.565917+0000","last_fullsized":"2024-09-20T22:15:23.565917+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:14:23.503360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:14:23.503360+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:52.579727+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"328'20","reported_seq":591,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560075+0000","last_change":"2024-09-20T22:13:41.421053+0000","last_active":"2024-09-20T22:15:23.560075+0000","last_peered":"2024-09-20T22:15:23.560075+0000","last_clean":"2024-09-20T22:15:23.560075+0000","last_became_active":"2024-09-20T22:11:30.486532+0000","last_became_peered":"2024-09-20T22:11:30.486532+0000","last_unstale":"2024-09-20T22:15:23.560075+0000","last_undegraded":"2024-09-20T22:15:23.560075+0000","last_fullsized":"2024-09-20T22:15:23.560075+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:13:41.420933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:41.420933+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":700,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566377+0000","last_change":"2024-09-20T22:14:11.402471+0000","last_active":"2024-09-20T22:15:23.566377+0000","last_peered":"2024-09-20T22:15:23.566377+0000","last_clean":"2024-09-20T22:15:23.566377+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:15:23.566377+0000","last_undegraded":"2024-09-20T22:15:23.566377+0000","last_fullsized":"2024-09-20T22:15:23.566377+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:11.402361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:14:11.402361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:27.694063+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"363'13","reported_seq":162,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:25.354951+0000","last_change":"2024-09-20T22:14:52.612861+0000","last_active":"2024-09-20T22:15:25.354951+0000","last_peered":"2024-09-20T22:15:25.354951+0000","last_clean":"2024-09-20T22:15:25.354951+0000","last_became_active":"2024-09-20T22:12:57.884415+0000","last_became_peered":"2024-09-20T22:12:57.884415+0000","last_unstale":"2024-09-20T22:15:25.354951+0000","last_undegraded":"2024-09-20T22:15:25.354951+0000","last_fullsized":"2024-09-20T22:15:25.354951+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"363'13","last_scrub_stamp":"2024-09-20T22:14:52.612615+0000","last_deep_scrub":"363'13","last_deep_scrub_stamp":"2024-09-20T22:14:52.612615+0000","last_clean_scrub_stamp":"2024-09-20T22:14:52.612615+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:56.113350+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":128,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569299+0000","last_change":"2024-09-20T22:14:55.614617+0000","last_active":"2024-09-20T22:15:23.569299+0000","last_peered":"2024-09-20T22:15:23.569299+0000","last_clean":"2024-09-20T22:15:23.569299+0000","last_became_active":"2024-09-20T22:12:52.518529+0000","last_became_peered":"2024-09-20T22:12:52.518529+0000","last_unstale":"2024-09-20T22:15:23.569299+0000","last_undegraded":"2024-09-20T22:15:23.569299+0000","last_fullsized":"2024-09-20T22:15:23.569299+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:55.614504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:55.614504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:18.700502+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"378'2147","reported_seq":2173,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:32.908988+0000","last_change":"2024-09-20T22:15:08.927958+0000","last_active":"2024-09-20T22:15:32.908988+0000","last_peered":"2024-09-20T22:15:32.908988+0000","last_clean":"2024-09-20T22:15:32.908988+0000","last_became_active":"2024-09-20T22:15:08.927661+0000","last_became_peered":"2024-09-20T22:15:08.927661+0000","last_unstale":"2024-09-20T22:15:32.908988+0000","last_undegraded":"2024-09-20T22:15:32.908988+0000","last_fullsized":"2024-09-20T22:15:32.908988+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2147,"log_dups_size":0,"ondisk_log_size":2147,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:29.356464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140705792,"num_objects":2147,"num_object_clones":0,"num_object_copies":6441,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2147,"num_write_kb":137408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":711,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565207+0000","last_change":"2024-09-20T22:15:17.487480+0000","last_active":"2024-09-20T22:15:23.565207+0000","last_peered":"2024-09-20T22:15:23.565207+0000","last_clean":"2024-09-20T22:15:23.565207+0000","last_became_active":"2024-09-20T22:12:55.523589+0000","last_became_peered":"2024-09-20T22:12:55.523589+0000","last_unstale":"2024-09-20T22:15:23.565207+0000","last_undegraded":"2024-09-20T22:15:23.565207+0000","last_fullsized":"2024-09-20T22:15:23.565207+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:17.487408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:17.487408+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:28.563004+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"328'29","reported_seq":161,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569850+0000","last_change":"2024-09-20T22:14:15.427866+0000","last_active":"2024-09-20T22:15:23.569850+0000","last_peered":"2024-09-20T22:15:23.569850+0000","last_clean":"2024-09-20T22:15:23.569850+0000","last_became_active":"2024-09-20T22:12:52.525538+0000","last_became_peered":"2024-09-20T22:12:52.525538+0000","last_unstale":"2024-09-20T22:15:23.569850+0000","last_undegraded":"2024-09-20T22:15:23.569850+0000","last_fullsized":"2024-09-20T22:15:23.569850+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'29","last_scrub_stamp":"2024-09-20T22:14:15.427533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:15.427533+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.217474+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":527,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560143+0000","last_change":"2024-09-20T22:13:13.091930+0000","last_active":"2024-09-20T22:15:23.560143+0000","last_peered":"2024-09-20T22:15:23.560143+0000","last_clean":"2024-09-20T22:15:23.560143+0000","last_became_active":"2024-09-20T22:11:30.486428+0000","last_became_peered":"2024-09-20T22:11:30.486428+0000","last_unstale":"2024-09-20T22:15:23.560143+0000","last_undegraded":"2024-09-20T22:15:23.560143+0000","last_fullsized":"2024-09-20T22:15:23.560143+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:13.091819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:13.091819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":704,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.564920+0000","last_change":"2024-09-20T22:14:22.500268+0000","last_active":"2024-09-20T22:15:23.564920+0000","last_peered":"2024-09-20T22:15:23.564920+0000","last_clean":"2024-09-20T22:15:23.564920+0000","last_became_active":"2024-09-20T22:12:52.519826+0000","last_became_peered":"2024-09-20T22:12:52.519826+0000","last_unstale":"2024-09-20T22:15:23.564920+0000","last_undegraded":"2024-09-20T22:15:23.564920+0000","last_fullsized":"2024-09-20T22:15:23.564920+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:22.500119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:14:22.500119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:41.298289+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":521,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559302+0000","last_change":"2024-09-20T22:15:21.489682+0000","last_active":"2024-09-20T22:15:23.559302+0000","last_peered":"2024-09-20T22:15:23.559302+0000","last_clean":"2024-09-20T22:15:23.559302+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:15:23.559302+0000","last_undegraded":"2024-09-20T22:15:23.559302+0000","last_fullsized":"2024-09-20T22:15:23.559302+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:21.489619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:21.489619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:22.282365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"378'2223","reported_seq":2248,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:32.915615+0000","last_change":"2024-09-20T22:15:08.926557+0000","last_active":"2024-09-20T22:15:32.915615+0000","last_peered":"2024-09-20T22:15:32.915615+0000","last_clean":"2024-09-20T22:15:32.915615+0000","last_became_active":"2024-09-20T22:15:08.926245+0000","last_became_peered":"2024-09-20T22:15:08.926245+0000","last_unstale":"2024-09-20T22:15:32.915615+0000","last_undegraded":"2024-09-20T22:15:32.915615+0000","last_fullsized":"2024-09-20T22:15:32.915615+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2223,"log_dups_size":0,"ondisk_log_size":2223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:27.389588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145686528,"num_objects":2223,"num_object_clones":0,"num_object_copies":6669,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2223,"num_write_kb":142272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":698,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.564541+0000","last_change":"2024-09-20T22:12:55.522020+0000","last_active":"2024-09-20T22:15:23.564541+0000","last_peered":"2024-09-20T22:15:23.564541+0000","last_clean":"2024-09-20T22:15:23.564541+0000","last_became_active":"2024-09-20T22:12:55.521605+0000","last_became_peered":"2024-09-20T22:12:55.521605+0000","last_unstale":"2024-09-20T22:15:23.564541+0000","last_undegraded":"2024-09-20T22:15:23.564541+0000","last_fullsized":"2024-09-20T22:15:23.564541+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:12:16.121263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:12:16.121263+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"328'52","reported_seq":716,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:25.323106+0000","last_change":"2024-09-20T22:14:46.592348+0000","last_active":"2024-09-20T22:15:25.323106+0000","last_peered":"2024-09-20T22:15:25.323106+0000","last_clean":"2024-09-20T22:15:25.323106+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:15:25.323106+0000","last_undegraded":"2024-09-20T22:15:25.323106+0000","last_fullsized":"2024-09-20T22:15:25.323106+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'52","last_scrub_stamp":"2024-09-20T22:14:46.592203+0000","last_deep_scrub":"271'40","last_deep_scrub_stamp":"2024-09-20T22:11:05.235779+0000","last_clean_scrub_stamp":"2024-09-20T22:14:46.592203+0000","objects_scrubbed":10,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:01.016758+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":694,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561782+0000","last_change":"2024-09-20T22:14:58.031902+0000","last_active":"2024-09-20T22:15:23.561782+0000","last_peered":"2024-09-20T22:15:23.561782+0000","last_clean":"2024-09-20T22:15:23.561782+0000","last_became_active":"2024-09-20T22:12:57.884257+0000","last_became_peered":"2024-09-20T22:12:57.884257+0000","last_unstale":"2024-09-20T22:15:23.561782+0000","last_undegraded":"2024-09-20T22:15:23.561782+0000","last_fullsized":"2024-09-20T22:15:23.561782+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:58.031824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:58.031824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:04.750930+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":696,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565347+0000","last_change":"2024-09-20T22:13:51.454966+0000","last_active":"2024-09-20T22:15:23.565347+0000","last_peered":"2024-09-20T22:15:23.565347+0000","last_clean":"2024-09-20T22:15:23.565347+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:15:23.565347+0000","last_undegraded":"2024-09-20T22:15:23.565347+0000","last_fullsized":"2024-09-20T22:15:23.565347+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.454850+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":128,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.570512+0000","last_change":"2024-09-20T22:14:12.458599+0000","last_active":"2024-09-20T22:15:23.570512+0000","last_peered":"2024-09-20T22:15:23.570512+0000","last_clean":"2024-09-20T22:15:23.570512+0000","last_became_active":"2024-09-20T22:12:52.527547+0000","last_became_peered":"2024-09-20T22:12:52.527547+0000","last_unstale":"2024-09-20T22:15:23.570512+0000","last_undegraded":"2024-09-20T22:15:23.570512+0000","last_fullsized":"2024-09-20T22:15:23.570512+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:12.458492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:12.458492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:36.742957+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":671,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:24.071666+0000","last_change":"2024-09-20T22:15:24.071705+0000","last_active":"2024-09-20T22:15:24.071666+0000","last_peered":"2024-09-20T22:15:24.071666+0000","last_clean":"2024-09-20T22:15:24.071666+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:15:24.071666+0000","last_undegraded":"2024-09-20T22:15:24.071666+0000","last_fullsized":"2024-09-20T22:15:24.071666+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.071593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.071593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:51.612695+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"378'2055","reported_seq":2081,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:31.171944+0000","last_change":"2024-09-20T22:15:08.931328+0000","last_active":"2024-09-20T22:15:31.171944+0000","last_peered":"2024-09-20T22:15:31.171944+0000","last_clean":"2024-09-20T22:15:31.171944+0000","last_became_active":"2024-09-20T22:15:08.930968+0000","last_became_peered":"2024-09-20T22:15:08.930968+0000","last_unstale":"2024-09-20T22:15:31.171944+0000","last_undegraded":"2024-09-20T22:15:31.171944+0000","last_fullsized":"2024-09-20T22:15:31.171944+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2055,"log_dups_size":0,"ondisk_log_size":2055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:14.066582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134676480,"num_objects":2055,"num_object_clones":0,"num_object_copies":6165,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2055,"num_write_kb":131520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":695,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566437+0000","last_change":"2024-09-20T22:15:00.407717+0000","last_active":"2024-09-20T22:15:23.566437+0000","last_peered":"2024-09-20T22:15:23.566437+0000","last_clean":"2024-09-20T22:15:23.566437+0000","last_became_active":"2024-09-20T22:12:55.518061+0000","last_became_peered":"2024-09-20T22:12:55.518061+0000","last_unstale":"2024-09-20T22:15:23.566437+0000","last_undegraded":"2024-09-20T22:15:23.566437+0000","last_fullsized":"2024-09-20T22:15:23.566437+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:00.407627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:00.407627+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:02.154276+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"328'17","reported_seq":594,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561702+0000","last_change":"2024-09-20T22:13:51.684806+0000","last_active":"2024-09-20T22:15:23.561702+0000","last_peered":"2024-09-20T22:15:23.561702+0000","last_clean":"2024-09-20T22:15:23.561702+0000","last_became_active":"2024-09-20T22:12:52.521073+0000","last_became_peered":"2024-09-20T22:12:52.521073+0000","last_unstale":"2024-09-20T22:15:23.561702+0000","last_undegraded":"2024-09-20T22:15:23.561702+0000","last_fullsized":"2024-09-20T22:15:23.561702+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'17","last_scrub_stamp":"2024-09-20T22:13:51.684667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.684667+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":522,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559330+0000","last_change":"2024-09-20T22:14:49.575469+0000","last_active":"2024-09-20T22:15:23.559330+0000","last_peered":"2024-09-20T22:15:23.559330+0000","last_clean":"2024-09-20T22:15:23.559330+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:15:23.559330+0000","last_undegraded":"2024-09-20T22:15:23.559330+0000","last_fullsized":"2024-09-20T22:15:23.559330+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:49.575363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:49.575363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:08.882258+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":738,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565240+0000","last_change":"2024-09-20T22:14:18.282947+0000","last_active":"2024-09-20T22:15:23.565240+0000","last_peered":"2024-09-20T22:15:23.565240+0000","last_clean":"2024-09-20T22:15:23.565240+0000","last_became_active":"2024-09-20T22:12:52.517255+0000","last_became_peered":"2024-09-20T22:12:52.517255+0000","last_unstale":"2024-09-20T22:15:23.565240+0000","last_undegraded":"2024-09-20T22:15:23.565240+0000","last_fullsized":"2024-09-20T22:15:23.565240+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:14:18.282814+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:14:18.282814+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:39.504754+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":699,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:24.892517+0000","last_change":"2024-09-20T22:15:24.892562+0000","last_active":"2024-09-20T22:15:24.892517+0000","last_peered":"2024-09-20T22:15:24.892517+0000","last_clean":"2024-09-20T22:15:24.892517+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:15:24.892517+0000","last_undegraded":"2024-09-20T22:15:24.892517+0000","last_fullsized":"2024-09-20T22:15:24.892517+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.891955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.891955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:47.230621+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":665,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567608+0000","last_change":"2024-09-20T22:13:41.730912+0000","last_active":"2024-09-20T22:15:23.567608+0000","last_peered":"2024-09-20T22:15:23.567608+0000","last_clean":"2024-09-20T22:15:23.567608+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:15:23.567608+0000","last_undegraded":"2024-09-20T22:15:23.567608+0000","last_fullsized":"2024-09-20T22:15:23.567608+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:41.730776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:41.730776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"378'2095","reported_seq":2121,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:31.980291+0000","last_change":"2024-09-20T22:15:08.927019+0000","last_active":"2024-09-20T22:15:31.980291+0000","last_peered":"2024-09-20T22:15:31.980291+0000","last_clean":"2024-09-20T22:15:31.980291+0000","last_became_active":"2024-09-20T22:15:08.926702+0000","last_became_peered":"2024-09-20T22:15:08.926702+0000","last_unstale":"2024-09-20T22:15:31.980291+0000","last_undegraded":"2024-09-20T22:15:31.980291+0000","last_fullsized":"2024-09-20T22:15:31.980291+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2095,"log_dups_size":0,"ondisk_log_size":2095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:17.801635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":137297920,"num_objects":2095,"num_object_clones":0,"num_object_copies":6285,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2095,"num_write_kb":134080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":679,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559344+0000","last_change":"2024-09-20T22:13:48.490583+0000","last_active":"2024-09-20T22:15:23.559344+0000","last_peered":"2024-09-20T22:15:23.559344+0000","last_clean":"2024-09-20T22:15:23.559344+0000","last_became_active":"2024-09-20T22:08:48.446840+0000","last_became_peered":"2024-09-20T22:08:48.446840+0000","last_unstale":"2024-09-20T22:15:23.559344+0000","last_undegraded":"2024-09-20T22:15:23.559344+0000","last_fullsized":"2024-09-20T22:15:23.559344+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:13:48.490464+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"328'24","reported_seq":168,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:26.742385+0000","last_change":"2024-09-20T22:15:26.742439+0000","last_active":"2024-09-20T22:15:26.742385+0000","last_peered":"2024-09-20T22:15:26.742385+0000","last_clean":"2024-09-20T22:15:26.742385+0000","last_became_active":"2024-09-20T22:12:52.527919+0000","last_became_peered":"2024-09-20T22:12:52.527919+0000","last_unstale":"2024-09-20T22:15:26.742385+0000","last_undegraded":"2024-09-20T22:15:26.742385+0000","last_fullsized":"2024-09-20T22:15:26.742385+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'24","last_scrub_stamp":"2024-09-20T22:15:26.742317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:26.742317+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:32.350037+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":136,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569255+0000","last_change":"2024-09-20T22:15:21.774404+0000","last_active":"2024-09-20T22:15:23.569255+0000","last_peered":"2024-09-20T22:15:23.569255+0000","last_clean":"2024-09-20T22:15:23.569255+0000","last_became_active":"2024-09-20T22:12:52.517432+0000","last_became_peered":"2024-09-20T22:12:52.517432+0000","last_unstale":"2024-09-20T22:15:23.569255+0000","last_undegraded":"2024-09-20T22:15:23.569255+0000","last_fullsized":"2024-09-20T22:15:23.569255+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:21.774300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:15:21.774300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:29.628508+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":1461,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:29.776421+0000","last_change":"2024-09-20T22:13:06.609600+0000","last_active":"2024-09-20T22:15:29.776421+0000","last_peered":"2024-09-20T22:15:29.776421+0000","last_clean":"2024-09-20T22:15:29.776421+0000","last_became_active":"2024-09-20T22:12:52.526622+0000","last_became_peered":"2024-09-20T22:12:52.526622+0000","last_unstale":"2024-09-20T22:15:29.776421+0000","last_undegraded":"2024-09-20T22:15:29.776421+0000","last_fullsized":"2024-09-20T22:15:29.776421+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:13:06.609525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:13:06.609525+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":737,"num_read_kb":732,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":230,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567015+0000","last_change":"2024-09-20T22:14:14.443714+0000","last_active":"2024-09-20T22:15:23.567015+0000","last_peered":"2024-09-20T22:15:23.567015+0000","last_clean":"2024-09-20T22:15:23.567015+0000","last_became_active":"2024-09-20T22:12:52.518823+0000","last_became_peered":"2024-09-20T22:12:52.518823+0000","last_unstale":"2024-09-20T22:15:23.567015+0000","last_undegraded":"2024-09-20T22:15:23.567015+0000","last_fullsized":"2024-09-20T22:15:23.567015+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:14.443611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:14.443611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":514,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559961+0000","last_change":"2024-09-20T22:13:40.433856+0000","last_active":"2024-09-20T22:15:23.559961+0000","last_peered":"2024-09-20T22:15:23.559961+0000","last_clean":"2024-09-20T22:15:23.559961+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:15:23.559961+0000","last_undegraded":"2024-09-20T22:15:23.559961+0000","last_fullsized":"2024-09-20T22:15:23.559961+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:40.433606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.415468+0000","last_clean_scrub_stamp":"2024-09-20T22:13:40.433606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"378'2194","reported_seq":2220,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:32.922355+0000","last_change":"2024-09-20T22:15:08.925131+0000","last_active":"2024-09-20T22:15:32.922355+0000","last_peered":"2024-09-20T22:15:32.922355+0000","last_clean":"2024-09-20T22:15:32.922355+0000","last_became_active":"2024-09-20T22:15:08.924809+0000","last_became_peered":"2024-09-20T22:15:08.924809+0000","last_unstale":"2024-09-20T22:15:32.922355+0000","last_undegraded":"2024-09-20T22:15:32.922355+0000","last_fullsized":"2024-09-20T22:15:32.922355+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2194,"log_dups_size":0,"ondisk_log_size":2194,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:36.576727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":143785984,"num_objects":2194,"num_object_clones":0,"num_object_copies":6582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2194,"num_write_kb":140416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"328'28","reported_seq":881,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:30.324163+0000","last_change":"2024-09-20T22:14:21.445235+0000","last_active":"2024-09-20T22:15:30.324163+0000","last_peered":"2024-09-20T22:15:30.324163+0000","last_clean":"2024-09-20T22:15:30.324163+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:15:30.324163+0000","last_undegraded":"2024-09-20T22:15:30.324163+0000","last_fullsized":"2024-09-20T22:15:30.324163+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:14:21.445103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:21.445103+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":636,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.564705+0000","last_change":"2024-09-20T22:12:42.260316+0000","last_active":"2024-09-20T22:15:23.564705+0000","last_peered":"2024-09-20T22:15:23.564705+0000","last_clean":"2024-09-20T22:15:23.564705+0000","last_became_active":"2024-09-20T22:12:21.519717+0000","last_became_peered":"2024-09-20T22:12:21.519717+0000","last_unstale":"2024-09-20T22:15:23.564705+0000","last_undegraded":"2024-09-20T22:15:23.564705+0000","last_fullsized":"2024-09-20T22:15:23.564705+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:42.260197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:42.260197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":695,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:25.365177+0000","last_change":"2024-09-20T22:15:25.365208+0000","last_active":"2024-09-20T22:15:25.365177+0000","last_peered":"2024-09-20T22:15:25.365177+0000","last_clean":"2024-09-20T22:15:25.365177+0000","last_became_active":"2024-09-20T22:12:55.522325+0000","last_became_peered":"2024-09-20T22:12:55.522325+0000","last_unstale":"2024-09-20T22:15:25.365177+0000","last_undegraded":"2024-09-20T22:15:25.365177+0000","last_fullsized":"2024-09-20T22:15:25.365177+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:25.365122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:25.365122+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:41.815575+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":259,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566610+0000","last_change":"2024-09-20T22:13:51.675128+0000","last_active":"2024-09-20T22:15:23.566610+0000","last_peered":"2024-09-20T22:15:23.566610+0000","last_clean":"2024-09-20T22:15:23.566610+0000","last_became_active":"2024-09-20T22:10:48.907271+0000","last_became_peered":"2024-09-20T22:10:48.907271+0000","last_unstale":"2024-09-20T22:15:23.566610+0000","last_undegraded":"2024-09-20T22:15:23.566610+0000","last_fullsized":"2024-09-20T22:15:23.566610+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.674976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":691,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566801+0000","last_change":"2024-09-20T22:13:02.482655+0000","last_active":"2024-09-20T22:15:23.566801+0000","last_peered":"2024-09-20T22:15:23.566801+0000","last_clean":"2024-09-20T22:15:23.566801+0000","last_became_active":"2024-09-20T22:12:44.630044+0000","last_became_peered":"2024-09-20T22:12:44.630044+0000","last_unstale":"2024-09-20T22:15:23.566801+0000","last_undegraded":"2024-09-20T22:15:23.566801+0000","last_fullsized":"2024-09-20T22:15:23.566801+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:02.482546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:13:02.482546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"378'2270","reported_seq":2296,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:32.904782+0000","last_change":"2024-09-20T22:15:08.925316+0000","last_active":"2024-09-20T22:15:32.904782+0000","last_peered":"2024-09-20T22:15:32.904782+0000","last_clean":"2024-09-20T22:15:32.904782+0000","last_became_active":"2024-09-20T22:15:08.925040+0000","last_became_peered":"2024-09-20T22:15:08.925040+0000","last_unstale":"2024-09-20T22:15:32.904782+0000","last_undegraded":"2024-09-20T22:15:32.904782+0000","last_fullsized":"2024-09-20T22:15:32.904782+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2270,"log_dups_size":0,"ondisk_log_size":2270,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:30.887426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":148766720,"num_objects":2270,"num_object_clones":0,"num_object_copies":6810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2270,"num_write_kb":145280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"328'19","reported_seq":712,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.562573+0000","last_change":"2024-09-20T22:15:19.012731+0000","last_active":"2024-09-20T22:15:23.562573+0000","last_peered":"2024-09-20T22:15:23.562573+0000","last_clean":"2024-09-20T22:15:23.562573+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:15:23.562573+0000","last_undegraded":"2024-09-20T22:15:23.562573+0000","last_fullsized":"2024-09-20T22:15:23.562573+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:15:19.012609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:19.012609+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:20.138304+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":684,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:27.077365+0000","last_change":"2024-09-20T22:15:27.077414+0000","last_active":"2024-09-20T22:15:27.077365+0000","last_peered":"2024-09-20T22:15:27.077365+0000","last_clean":"2024-09-20T22:15:27.077365+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:15:27.077365+0000","last_undegraded":"2024-09-20T22:15:27.077365+0000","last_fullsized":"2024-09-20T22:15:27.077365+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:27.077295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.077295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:47.688033+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":683,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:24.150346+0000","last_change":"2024-09-20T22:15:24.150401+0000","last_active":"2024-09-20T22:15:24.150346+0000","last_peered":"2024-09-20T22:15:24.150346+0000","last_clean":"2024-09-20T22:15:24.150346+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:15:24.150346+0000","last_undegraded":"2024-09-20T22:15:24.150346+0000","last_fullsized":"2024-09-20T22:15:24.150346+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.150265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.150265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:32.448596+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":631,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565142+0000","last_change":"2024-09-20T22:14:08.297827+0000","last_active":"2024-09-20T22:15:23.565142+0000","last_peered":"2024-09-20T22:15:23.565142+0000","last_clean":"2024-09-20T22:15:23.565142+0000","last_became_active":"2024-09-20T22:12:21.521195+0000","last_became_peered":"2024-09-20T22:12:21.521195+0000","last_unstale":"2024-09-20T22:15:23.565142+0000","last_undegraded":"2024-09-20T22:15:23.565142+0000","last_fullsized":"2024-09-20T22:15:23.565142+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:08.297698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:08.297698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3","version":"378'2077","reported_seq":2102,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:31.237573+0000","last_change":"2024-09-20T22:15:09.087357+0000","last_active":"2024-09-20T22:15:31.237573+0000","last_peered":"2024-09-20T22:15:31.237573+0000","last_clean":"2024-09-20T22:15:31.237573+0000","last_became_active":"2024-09-20T22:15:09.087061+0000","last_became_peered":"2024-09-20T22:15:09.087061+0000","last_unstale":"2024-09-20T22:15:31.237573+0000","last_undegraded":"2024-09-20T22:15:31.237573+0000","last_fullsized":"2024-09-20T22:15:31.237573+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2077,"log_dups_size":0,"ondisk_log_size":2077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:23.286140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":136118272,"num_objects":2077,"num_object_clones":0,"num_object_copies":6231,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2077,"num_write_kb":132928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":446,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.564473+0000","last_change":"2024-09-20T22:14:46.480442+0000","last_active":"2024-09-20T22:15:23.564473+0000","last_peered":"2024-09-20T22:15:23.564473+0000","last_clean":"2024-09-20T22:15:23.564473+0000","last_became_active":"2024-09-20T22:14:38.674758+0000","last_became_peered":"2024-09-20T22:14:38.674758+0000","last_unstale":"2024-09-20T22:15:23.564473+0000","last_undegraded":"2024-09-20T22:15:23.564473+0000","last_fullsized":"2024-09-20T22:15:23.564473+0000","mapping_epoch":358,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":359,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:14:46.480331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:14:46.480331+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:06.604863+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"328'31","reported_seq":738,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.562287+0000","last_change":"2024-09-20T22:14:46.989102+0000","last_active":"2024-09-20T22:15:23.562287+0000","last_peered":"2024-09-20T22:15:23.562287+0000","last_clean":"2024-09-20T22:15:23.562287+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:15:23.562287+0000","last_undegraded":"2024-09-20T22:15:23.562287+0000","last_fullsized":"2024-09-20T22:15:23.562287+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'31","last_scrub_stamp":"2024-09-20T22:14:46.988744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:46.988744+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:58.605093+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":700,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:32.506611+0000","last_change":"2024-09-20T22:15:32.512124+0000","last_active":"2024-09-20T22:15:32.506611+0000","last_peered":"2024-09-20T22:15:32.506611+0000","last_clean":"2024-09-20T22:15:32.506611+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:15:32.506611+0000","last_undegraded":"2024-09-20T22:15:32.506611+0000","last_fullsized":"2024-09-20T22:15:32.506611+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:32.506512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:32.506512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:01.564420+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":525,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560054+0000","last_change":"2024-09-20T22:13:10.039183+0000","last_active":"2024-09-20T22:15:23.560054+0000","last_peered":"2024-09-20T22:15:23.560054+0000","last_clean":"2024-09-20T22:15:23.560054+0000","last_became_active":"2024-09-20T22:12:52.520612+0000","last_became_peered":"2024-09-20T22:12:52.520612+0000","last_unstale":"2024-09-20T22:15:23.560054+0000","last_undegraded":"2024-09-20T22:15:23.560054+0000","last_fullsized":"2024-09-20T22:15:23.560054+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:10.039074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.039074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":687,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559895+0000","last_change":"2024-09-20T22:14:50.010648+0000","last_active":"2024-09-20T22:15:23.559895+0000","last_peered":"2024-09-20T22:15:23.559895+0000","last_clean":"2024-09-20T22:15:23.559895+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:15:23.559895+0000","last_undegraded":"2024-09-20T22:15:23.559895+0000","last_fullsized":"2024-09-20T22:15:23.559895+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:50.010536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:50.010536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:56.117583+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"378'2252","reported_seq":2278,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:33.253827+0000","last_change":"2024-09-20T22:15:08.926429+0000","last_active":"2024-09-20T22:15:33.253827+0000","last_peered":"2024-09-20T22:15:33.253827+0000","last_clean":"2024-09-20T22:15:33.253827+0000","last_became_active":"2024-09-20T22:15:08.926083+0000","last_became_peered":"2024-09-20T22:15:08.926083+0000","last_unstale":"2024-09-20T22:15:33.253827+0000","last_undegraded":"2024-09-20T22:15:33.253827+0000","last_fullsized":"2024-09-20T22:15:33.253827+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2252,"log_dups_size":0,"ondisk_log_size":2252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:12.245813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147587072,"num_objects":2252,"num_object_clones":0,"num_object_copies":6756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2252,"num_write_kb":144128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":957,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566946+0000","last_change":"2024-09-20T22:13:49.912616+0000","last_active":"2024-09-20T22:15:23.566946+0000","last_peered":"2024-09-20T22:15:23.566946+0000","last_clean":"2024-09-20T22:15:23.566946+0000","last_became_active":"2024-09-20T22:08:48.455288+0000","last_became_peered":"2024-09-20T22:08:48.455288+0000","last_unstale":"2024-09-20T22:15:23.566946+0000","last_undegraded":"2024-09-20T22:15:23.566946+0000","last_fullsized":"2024-09-20T22:15:23.566946+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:49.912496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:13:49.912496+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":227,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"328'25","reported_seq":707,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561681+0000","last_change":"2024-09-20T22:14:10.070008+0000","last_active":"2024-09-20T22:15:23.561681+0000","last_peered":"2024-09-20T22:15:23.561681+0000","last_clean":"2024-09-20T22:15:23.561681+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:15:23.561681+0000","last_undegraded":"2024-09-20T22:15:23.561681+0000","last_fullsized":"2024-09-20T22:15:23.561681+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'25","last_scrub_stamp":"2024-09-20T22:14:10.069812+0000","last_deep_scrub":"289'15","last_deep_scrub_stamp":"2024-09-20T22:11:37.277648+0000","last_clean_scrub_stamp":"2024-09-20T22:14:10.069812+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":140,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:30.821212+0000","last_change":"2024-09-20T22:15:30.821251+0000","last_active":"2024-09-20T22:15:30.821212+0000","last_peered":"2024-09-20T22:15:30.821212+0000","last_clean":"2024-09-20T22:15:30.821212+0000","last_became_active":"2024-09-20T22:12:52.528310+0000","last_became_peered":"2024-09-20T22:12:52.528310+0000","last_unstale":"2024-09-20T22:15:30.821212+0000","last_undegraded":"2024-09-20T22:15:30.821212+0000","last_fullsized":"2024-09-20T22:15:30.821212+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:15:30.821142+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:15:30.821142+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:55.515561+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"283'11","reported_seq":881,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:25.355094+0000","last_change":"2024-09-20T22:14:20.265212+0000","last_active":"2024-09-20T22:15:25.355094+0000","last_peered":"2024-09-20T22:15:25.355094+0000","last_clean":"2024-09-20T22:15:25.355094+0000","last_became_active":"2024-09-20T22:11:30.485356+0000","last_became_peered":"2024-09-20T22:11:30.485356+0000","last_unstale":"2024-09-20T22:15:25.355094+0000","last_undegraded":"2024-09-20T22:15:25.355094+0000","last_fullsized":"2024-09-20T22:15:25.355094+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'11","last_scrub_stamp":"2024-09-20T22:14:20.265107+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:14:20.265107+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:43.795761+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":673,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.562251+0000","last_change":"2024-09-20T22:14:51.059172+0000","last_active":"2024-09-20T22:15:23.562251+0000","last_peered":"2024-09-20T22:15:23.562251+0000","last_clean":"2024-09-20T22:15:23.562251+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:15:23.562251+0000","last_undegraded":"2024-09-20T22:15:23.562251+0000","last_fullsized":"2024-09-20T22:15:23.562251+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:51.059066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:14:51.059066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:20.407644+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"378'1974","reported_seq":2000,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:31.211737+0000","last_change":"2024-09-20T22:15:09.088109+0000","last_active":"2024-09-20T22:15:31.211737+0000","last_peered":"2024-09-20T22:15:31.211737+0000","last_clean":"2024-09-20T22:15:31.211737+0000","last_became_active":"2024-09-20T22:15:09.087830+0000","last_became_peered":"2024-09-20T22:15:09.087830+0000","last_unstale":"2024-09-20T22:15:31.211737+0000","last_undegraded":"2024-09-20T22:15:31.211737+0000","last_fullsized":"2024-09-20T22:15:31.211737+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1974,"log_dups_size":0,"ondisk_log_size":1974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:10.163377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129368064,"num_objects":1974,"num_object_clones":0,"num_object_copies":5922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1974,"num_write_kb":126336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"328'15","reported_seq":155,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:25.741727+0000","last_change":"2024-09-20T22:15:25.742250+0000","last_active":"2024-09-20T22:15:25.741727+0000","last_peered":"2024-09-20T22:15:25.741727+0000","last_clean":"2024-09-20T22:15:25.741727+0000","last_became_active":"2024-09-20T22:12:52.527531+0000","last_became_peered":"2024-09-20T22:12:52.527531+0000","last_unstale":"2024-09-20T22:15:25.741727+0000","last_undegraded":"2024-09-20T22:15:25.741727+0000","last_fullsized":"2024-09-20T22:15:25.741727+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:15:25.741631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:25.741631+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:28.256179+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":312,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.562235+0000","last_change":"2024-09-20T22:14:55.145177+0000","last_active":"2024-09-20T22:15:23.562235+0000","last_peered":"2024-09-20T22:15:23.562235+0000","last_clean":"2024-09-20T22:15:23.562235+0000","last_became_active":"2024-09-20T22:10:14.536155+0000","last_became_peered":"2024-09-20T22:10:14.536155+0000","last_unstale":"2024-09-20T22:15:23.562235+0000","last_undegraded":"2024-09-20T22:15:23.562235+0000","last_fullsized":"2024-09-20T22:15:23.562235+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:14:55.144682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:55.144682+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:11.541071+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":699,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.563569+0000","last_change":"2024-09-20T22:15:18.919125+0000","last_active":"2024-09-20T22:15:23.563569+0000","last_peered":"2024-09-20T22:15:23.563569+0000","last_clean":"2024-09-20T22:15:23.563569+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:15:23.563569+0000","last_undegraded":"2024-09-20T22:15:23.563569+0000","last_fullsized":"2024-09-20T22:15:23.563569+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:18.919024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:15:18.919024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:26.163742+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":128,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569534+0000","last_change":"2024-09-20T22:14:45.679385+0000","last_active":"2024-09-20T22:15:23.569534+0000","last_peered":"2024-09-20T22:15:23.569534+0000","last_clean":"2024-09-20T22:15:23.569534+0000","last_became_active":"2024-09-20T22:12:52.512924+0000","last_became_peered":"2024-09-20T22:12:52.512924+0000","last_unstale":"2024-09-20T22:15:23.569534+0000","last_undegraded":"2024-09-20T22:15:23.569534+0000","last_fullsized":"2024-09-20T22:15:23.569534+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:45.679291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:45.679291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:49.553806+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"378'2151","reported_seq":2175,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:31.960786+0000","last_change":"2024-09-20T22:15:09.087623+0000","last_active":"2024-09-20T22:15:31.960786+0000","last_peered":"2024-09-20T22:15:31.960786+0000","last_clean":"2024-09-20T22:15:31.960786+0000","last_became_active":"2024-09-20T22:15:09.087298+0000","last_became_peered":"2024-09-20T22:15:09.087298+0000","last_unstale":"2024-09-20T22:15:31.960786+0000","last_undegraded":"2024-09-20T22:15:31.960786+0000","last_fullsized":"2024-09-20T22:15:31.960786+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2151,"log_dups_size":0,"ondisk_log_size":2151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:13.382167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140967936,"num_objects":2151,"num_object_clones":0,"num_object_copies":6453,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2151,"num_write_kb":137664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"328'35","reported_seq":795,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:31.492590+0000","last_change":"2024-09-20T22:15:31.492645+0000","last_active":"2024-09-20T22:15:31.492590+0000","last_peered":"2024-09-20T22:15:31.492590+0000","last_clean":"2024-09-20T22:15:31.492590+0000","last_became_active":"2024-09-20T22:10:30.652967+0000","last_became_peered":"2024-09-20T22:10:30.652967+0000","last_unstale":"2024-09-20T22:15:31.492590+0000","last_undegraded":"2024-09-20T22:15:31.492590+0000","last_fullsized":"2024-09-20T22:15:31.492590+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'35","last_scrub_stamp":"2024-09-20T22:15:31.492433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:31.492433+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:55.689722+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":696,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567833+0000","last_change":"2024-09-20T22:15:23.351913+0000","last_active":"2024-09-20T22:15:23.567833+0000","last_peered":"2024-09-20T22:15:23.567833+0000","last_clean":"2024-09-20T22:15:23.567833+0000","last_became_active":"2024-09-20T22:11:30.488650+0000","last_became_peered":"2024-09-20T22:11:30.488650+0000","last_unstale":"2024-09-20T22:15:23.567833+0000","last_undegraded":"2024-09-20T22:15:23.567833+0000","last_fullsized":"2024-09-20T22:15:23.567833+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:23.351789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:12:46.602035+0000","last_clean_scrub_stamp":"2024-09-20T22:15:23.351789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:36.963085+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":723,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.563957+0000","last_change":"2024-09-20T22:14:51.988808+0000","last_active":"2024-09-20T22:15:23.563957+0000","last_peered":"2024-09-20T22:15:23.563957+0000","last_clean":"2024-09-20T22:15:23.563957+0000","last_became_active":"2024-09-20T22:12:52.515295+0000","last_became_peered":"2024-09-20T22:12:52.515295+0000","last_unstale":"2024-09-20T22:15:23.563957+0000","last_undegraded":"2024-09-20T22:15:23.563957+0000","last_fullsized":"2024-09-20T22:15:23.563957+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:51.988666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_clean_scrub_stamp":"2024-09-20T22:14:51.988666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:00.556168+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":281,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566618+0000","last_change":"2024-09-20T22:14:47.271963+0000","last_active":"2024-09-20T22:15:23.566618+0000","last_peered":"2024-09-20T22:15:23.566618+0000","last_clean":"2024-09-20T22:15:23.566618+0000","last_became_active":"2024-09-20T22:10:30.657116+0000","last_became_peered":"2024-09-20T22:10:30.657116+0000","last_unstale":"2024-09-20T22:15:23.566618+0000","last_undegraded":"2024-09-20T22:15:23.566618+0000","last_fullsized":"2024-09-20T22:15:23.566618+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:47.271872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:47.271872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:07.707877+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"378'2180","reported_seq":2206,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:32.915266+0000","last_change":"2024-09-20T22:15:09.089072+0000","last_active":"2024-09-20T22:15:32.915266+0000","last_peered":"2024-09-20T22:15:32.915266+0000","last_clean":"2024-09-20T22:15:32.915266+0000","last_became_active":"2024-09-20T22:15:09.088590+0000","last_became_peered":"2024-09-20T22:15:09.088590+0000","last_unstale":"2024-09-20T22:15:32.915266+0000","last_undegraded":"2024-09-20T22:15:32.915266+0000","last_fullsized":"2024-09-20T22:15:32.915266+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2180,"log_dups_size":0,"ondisk_log_size":2180,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:25.980411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142868480,"num_objects":2180,"num_object_clones":0,"num_object_copies":6540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2180,"num_write_kb":139520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"328'18","reported_seq":713,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560618+0000","last_change":"2024-09-20T22:15:22.019818+0000","last_active":"2024-09-20T22:15:23.560618+0000","last_peered":"2024-09-20T22:15:23.560618+0000","last_clean":"2024-09-20T22:15:23.560618+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:15:23.560618+0000","last_undegraded":"2024-09-20T22:15:23.560618+0000","last_fullsized":"2024-09-20T22:15:23.560618+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'18","last_scrub_stamp":"2024-09-20T22:15:22.019700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:22.019700+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:42.439452+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":685,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567966+0000","last_change":"2024-09-20T22:14:15.965211+0000","last_active":"2024-09-20T22:15:23.567966+0000","last_peered":"2024-09-20T22:15:23.567966+0000","last_clean":"2024-09-20T22:15:23.567966+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:15:23.567966+0000","last_undegraded":"2024-09-20T22:15:23.567966+0000","last_fullsized":"2024-09-20T22:15:23.567966+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:15.965013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:15.965013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":720,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:27.924505+0000","last_change":"2024-09-20T22:15:27.924542+0000","last_active":"2024-09-20T22:15:27.924505+0000","last_peered":"2024-09-20T22:15:27.924505+0000","last_clean":"2024-09-20T22:15:27.924505+0000","last_became_active":"2024-09-20T22:12:52.514202+0000","last_became_peered":"2024-09-20T22:12:52.514202+0000","last_unstale":"2024-09-20T22:15:27.924505+0000","last_undegraded":"2024-09-20T22:15:27.924505+0000","last_fullsized":"2024-09-20T22:15:27.924505+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:27.924150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.924150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:41.071832+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":146,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:26.021714+0000","last_change":"2024-09-20T22:15:26.021754+0000","last_active":"2024-09-20T22:15:26.021714+0000","last_peered":"2024-09-20T22:15:26.021714+0000","last_clean":"2024-09-20T22:15:26.021714+0000","last_became_active":"2024-09-20T22:12:44.638546+0000","last_became_peered":"2024-09-20T22:12:44.638546+0000","last_unstale":"2024-09-20T22:15:26.021714+0000","last_undegraded":"2024-09-20T22:15:26.021714+0000","last_fullsized":"2024-09-20T22:15:26.021714+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:26.021636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:26.021636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:46.884542+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"378'1974","reported_seq":2000,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:31.175082+0000","last_change":"2024-09-20T22:15:08.928417+0000","last_active":"2024-09-20T22:15:31.175082+0000","last_peered":"2024-09-20T22:15:31.175082+0000","last_clean":"2024-09-20T22:15:31.175082+0000","last_became_active":"2024-09-20T22:15:08.928102+0000","last_became_peered":"2024-09-20T22:15:08.928102+0000","last_unstale":"2024-09-20T22:15:31.175082+0000","last_undegraded":"2024-09-20T22:15:31.175082+0000","last_fullsized":"2024-09-20T22:15:31.175082+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":1974,"log_dups_size":0,"ondisk_log_size":1974,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:10.761616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129368064,"num_objects":1974,"num_object_clones":0,"num_object_copies":5922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1974,"num_write_kb":126336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"328'27","reported_seq":244,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.564446+0000","last_change":"2024-09-20T22:13:10.353607+0000","last_active":"2024-09-20T22:15:23.564446+0000","last_peered":"2024-09-20T22:15:23.564446+0000","last_clean":"2024-09-20T22:15:23.564446+0000","last_became_active":"2024-09-20T22:11:30.493275+0000","last_became_peered":"2024-09-20T22:11:30.493275+0000","last_unstale":"2024-09-20T22:15:23.564446+0000","last_undegraded":"2024-09-20T22:15:23.564446+0000","last_fullsized":"2024-09-20T22:15:23.564446+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:13:10.353459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.353459+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":311,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567745+0000","last_change":"2024-09-20T22:14:56.339026+0000","last_active":"2024-09-20T22:15:23.567745+0000","last_peered":"2024-09-20T22:15:23.567745+0000","last_clean":"2024-09-20T22:15:23.567745+0000","last_became_active":"2024-09-20T22:12:52.526551+0000","last_became_peered":"2024-09-20T22:12:52.526551+0000","last_unstale":"2024-09-20T22:15:23.567745+0000","last_undegraded":"2024-09-20T22:15:23.567745+0000","last_fullsized":"2024-09-20T22:15:23.567745+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:14:56.338941+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:14:56.338941+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:09.361665+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":829,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:25.323186+0000","last_change":"2024-09-20T22:14:12.911852+0000","last_active":"2024-09-20T22:15:25.323186+0000","last_peered":"2024-09-20T22:15:25.323186+0000","last_clean":"2024-09-20T22:15:25.323186+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:15:25.323186+0000","last_undegraded":"2024-09-20T22:15:25.323186+0000","last_fullsized":"2024-09-20T22:15:25.323186+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:14:12.911520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:12.911520+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.190103+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":222,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565005+0000","last_change":"2024-09-20T22:13:00.454973+0000","last_active":"2024-09-20T22:15:23.565005+0000","last_peered":"2024-09-20T22:15:23.565005+0000","last_clean":"2024-09-20T22:15:23.565005+0000","last_became_active":"2024-09-20T22:12:52.517758+0000","last_became_peered":"2024-09-20T22:12:52.517758+0000","last_unstale":"2024-09-20T22:15:23.565005+0000","last_undegraded":"2024-09-20T22:15:23.565005+0000","last_fullsized":"2024-09-20T22:15:23.565005+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:00.454378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:00.454378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":723,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565754+0000","last_change":"2024-09-20T22:14:49.417575+0000","last_active":"2024-09-20T22:15:23.565754+0000","last_peered":"2024-09-20T22:15:23.565754+0000","last_clean":"2024-09-20T22:15:23.565754+0000","last_became_active":"2024-09-20T22:12:52.516989+0000","last_became_peered":"2024-09-20T22:12:52.516989+0000","last_unstale":"2024-09-20T22:15:23.565754+0000","last_undegraded":"2024-09-20T22:15:23.565754+0000","last_fullsized":"2024-09-20T22:15:23.565754+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:49.417468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:49.417468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:19.052286+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"328'12","reported_seq":725,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.565379+0000","last_change":"2024-09-20T22:14:58.150505+0000","last_active":"2024-09-20T22:15:23.565379+0000","last_peered":"2024-09-20T22:15:23.565379+0000","last_clean":"2024-09-20T22:15:23.565379+0000","last_became_active":"2024-09-20T22:10:48.922561+0000","last_became_peered":"2024-09-20T22:10:48.922561+0000","last_unstale":"2024-09-20T22:15:23.565379+0000","last_undegraded":"2024-09-20T22:15:23.565379+0000","last_fullsized":"2024-09-20T22:15:23.565379+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'12","last_scrub_stamp":"2024-09-20T22:14:58.150361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:58.150361+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:05.584490+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":210,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566854+0000","last_change":"2024-09-20T22:12:14.148857+0000","last_active":"2024-09-20T22:15:23.566854+0000","last_peered":"2024-09-20T22:15:23.566854+0000","last_clean":"2024-09-20T22:15:23.566854+0000","last_became_active":"2024-09-20T22:11:30.487322+0000","last_became_peered":"2024-09-20T22:11:30.487322+0000","last_unstale":"2024-09-20T22:15:23.566854+0000","last_undegraded":"2024-09-20T22:15:23.566854+0000","last_fullsized":"2024-09-20T22:15:23.566854+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:14.148743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:14.148743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":664,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567290+0000","last_change":"2024-09-20T22:14:54.359687+0000","last_active":"2024-09-20T22:15:23.567290+0000","last_peered":"2024-09-20T22:15:23.567290+0000","last_clean":"2024-09-20T22:15:23.567290+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:15:23.567290+0000","last_undegraded":"2024-09-20T22:15:23.567290+0000","last_fullsized":"2024-09-20T22:15:23.567290+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:54.359586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:54.359586+0000","last_clean_scrub_stamp":"2024-09-20T22:14:54.359586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:11.201500+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"328'15","reported_seq":147,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.570583+0000","last_change":"2024-09-20T22:14:14.397650+0000","last_active":"2024-09-20T22:15:23.570583+0000","last_peered":"2024-09-20T22:15:23.570583+0000","last_clean":"2024-09-20T22:15:23.570583+0000","last_became_active":"2024-09-20T22:12:52.527106+0000","last_became_peered":"2024-09-20T22:12:52.527106+0000","last_unstale":"2024-09-20T22:15:23.570583+0000","last_undegraded":"2024-09-20T22:15:23.570583+0000","last_fullsized":"2024-09-20T22:15:23.570583+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:14:14.397530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:14.397530+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:42.176603+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":684,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:28.374492+0000","last_change":"2024-09-20T22:15:28.374528+0000","last_active":"2024-09-20T22:15:28.374492+0000","last_peered":"2024-09-20T22:15:28.374492+0000","last_clean":"2024-09-20T22:15:28.374492+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:15:28.374492+0000","last_undegraded":"2024-09-20T22:15:28.374492+0000","last_fullsized":"2024-09-20T22:15:28.374492+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:28.374414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:28.374414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:30.191322+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":314,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561351+0000","last_change":"2024-09-20T22:13:14.136545+0000","last_active":"2024-09-20T22:15:23.561351+0000","last_peered":"2024-09-20T22:15:23.561351+0000","last_clean":"2024-09-20T22:15:23.561351+0000","last_became_active":"2024-09-20T22:12:33.545506+0000","last_became_peered":"2024-09-20T22:12:33.545506+0000","last_unstale":"2024-09-20T22:15:23.561351+0000","last_undegraded":"2024-09-20T22:15:23.561351+0000","last_fullsized":"2024-09-20T22:15:23.561351+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:14.136472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_clean_scrub_stamp":"2024-09-20T22:13:14.136472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":695,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559183+0000","last_change":"2024-09-20T22:15:00.011002+0000","last_active":"2024-09-20T22:15:23.559183+0000","last_peered":"2024-09-20T22:15:23.559183+0000","last_clean":"2024-09-20T22:15:23.559183+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:15:23.559183+0000","last_undegraded":"2024-09-20T22:15:23.559183+0000","last_fullsized":"2024-09-20T22:15:23.559183+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:00.010921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:00.010921+0000","last_clean_scrub_stamp":"2024-09-20T22:15:00.010921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:17.344827+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"328'15","reported_seq":229,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566265+0000","last_change":"2024-09-20T22:12:43.218553+0000","last_active":"2024-09-20T22:15:23.566265+0000","last_peered":"2024-09-20T22:15:23.566265+0000","last_clean":"2024-09-20T22:15:23.566265+0000","last_became_active":"2024-09-20T22:11:30.493335+0000","last_became_peered":"2024-09-20T22:11:30.493335+0000","last_unstale":"2024-09-20T22:15:23.566265+0000","last_undegraded":"2024-09-20T22:15:23.566265+0000","last_fullsized":"2024-09-20T22:15:23.566265+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:12:43.218249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:43.218249+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":136,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.571282+0000","last_change":"2024-09-20T22:14:56.538965+0000","last_active":"2024-09-20T22:15:23.571282+0000","last_peered":"2024-09-20T22:15:23.571282+0000","last_clean":"2024-09-20T22:15:23.571282+0000","last_became_active":"2024-09-20T22:12:52.524149+0000","last_became_peered":"2024-09-20T22:12:52.524149+0000","last_unstale":"2024-09-20T22:15:23.571282+0000","last_undegraded":"2024-09-20T22:15:23.571282+0000","last_fullsized":"2024-09-20T22:15:23.571282+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:56.538825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:56.538825+0000","last_clean_scrub_stamp":"2024-09-20T22:14:56.538825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:02.516641+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"315'11","reported_seq":216,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:25.323178+0000","last_change":"2024-09-20T22:15:16.396532+0000","last_active":"2024-09-20T22:15:25.323178+0000","last_peered":"2024-09-20T22:15:25.323178+0000","last_clean":"2024-09-20T22:15:25.323178+0000","last_became_active":"2024-09-20T22:12:33.544098+0000","last_became_peered":"2024-09-20T22:12:33.544098+0000","last_unstale":"2024-09-20T22:15:25.323178+0000","last_undegraded":"2024-09-20T22:15:25.323178+0000","last_fullsized":"2024-09-20T22:15:25.323178+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:15:16.396454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:16.396454+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:18.359878+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":684,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:30.093618+0000","last_change":"2024-09-20T22:15:30.093649+0000","last_active":"2024-09-20T22:15:30.093618+0000","last_peered":"2024-09-20T22:15:30.093618+0000","last_clean":"2024-09-20T22:15:30.093618+0000","last_became_active":"2024-09-20T22:11:30.484731+0000","last_became_peered":"2024-09-20T22:11:30.484731+0000","last_unstale":"2024-09-20T22:15:30.093618+0000","last_undegraded":"2024-09-20T22:15:30.093618+0000","last_fullsized":"2024-09-20T22:15:30.093618+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:30.093548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:30.093548+0000","last_clean_scrub_stamp":"2024-09-20T22:15:30.093548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:55.627047+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"328'20","reported_seq":165,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559799+0000","last_change":"2024-09-20T22:13:49.905238+0000","last_active":"2024-09-20T22:15:23.559799+0000","last_peered":"2024-09-20T22:15:23.559799+0000","last_clean":"2024-09-20T22:15:23.559799+0000","last_became_active":"2024-09-20T22:12:44.631986+0000","last_became_peered":"2024-09-20T22:12:44.631986+0000","last_unstale":"2024-09-20T22:15:23.559799+0000","last_undegraded":"2024-09-20T22:15:23.559799+0000","last_fullsized":"2024-09-20T22:15:23.559799+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:13:49.905115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:49.905115+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":676,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567284+0000","last_change":"2024-09-20T22:14:23.041828+0000","last_active":"2024-09-20T22:15:23.567284+0000","last_peered":"2024-09-20T22:15:23.567284+0000","last_clean":"2024-09-20T22:15:23.567284+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:15:23.567284+0000","last_undegraded":"2024-09-20T22:15:23.567284+0000","last_fullsized":"2024-09-20T22:15:23.567284+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:23.041722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:23.041722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:33.056700+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"326'14","reported_seq":291,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:25.323130+0000","last_change":"2024-09-20T22:15:19.186947+0000","last_active":"2024-09-20T22:15:25.323130+0000","last_peered":"2024-09-20T22:15:25.323130+0000","last_clean":"2024-09-20T22:15:25.323130+0000","last_became_active":"2024-09-20T22:12:52.518770+0000","last_became_peered":"2024-09-20T22:12:52.518770+0000","last_unstale":"2024-09-20T22:15:25.323130+0000","last_undegraded":"2024-09-20T22:15:25.323130+0000","last_fullsized":"2024-09-20T22:15:25.323130+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'14","last_scrub_stamp":"2024-09-20T22:15:19.186862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:19.186862+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:43.532138+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":136,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.570163+0000","last_change":"2024-09-20T22:15:22.741319+0000","last_active":"2024-09-20T22:15:23.570163+0000","last_peered":"2024-09-20T22:15:23.570163+0000","last_clean":"2024-09-20T22:15:23.570163+0000","last_became_active":"2024-09-20T22:12:52.523781+0000","last_became_peered":"2024-09-20T22:12:52.523781+0000","last_unstale":"2024-09-20T22:15:23.570163+0000","last_undegraded":"2024-09-20T22:15:23.570163+0000","last_fullsized":"2024-09-20T22:15:23.570163+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:22.741214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:22.741214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:34.096157+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"328'19","reported_seq":732,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561553+0000","last_change":"2024-09-20T22:14:59.020526+0000","last_active":"2024-09-20T22:15:23.561553+0000","last_peered":"2024-09-20T22:15:23.561553+0000","last_clean":"2024-09-20T22:15:23.561553+0000","last_became_active":"2024-09-20T22:08:22.139581+0000","last_became_peered":"2024-09-20T22:08:22.139581+0000","last_unstale":"2024-09-20T22:15:23.561553+0000","last_undegraded":"2024-09-20T22:15:23.561553+0000","last_fullsized":"2024-09-20T22:15:23.561553+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:14:59.020420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:59.020420+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:24.012397+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":684,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561836+0000","last_change":"2024-09-20T22:15:20.031722+0000","last_active":"2024-09-20T22:15:23.561836+0000","last_peered":"2024-09-20T22:15:23.561836+0000","last_clean":"2024-09-20T22:15:23.561836+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:15:23.561836+0000","last_undegraded":"2024-09-20T22:15:23.561836+0000","last_fullsized":"2024-09-20T22:15:23.561836+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.031625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.031625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:36.647532+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":512,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559942+0000","last_change":"2024-09-20T22:13:52.605374+0000","last_active":"2024-09-20T22:15:23.559942+0000","last_peered":"2024-09-20T22:15:23.559942+0000","last_clean":"2024-09-20T22:15:23.559942+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:15:23.559942+0000","last_undegraded":"2024-09-20T22:15:23.559942+0000","last_fullsized":"2024-09-20T22:15:23.559942+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:52.605277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:52.605277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":523,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559195+0000","last_change":"2024-09-20T22:14:18.440962+0000","last_active":"2024-09-20T22:15:23.559195+0000","last_peered":"2024-09-20T22:15:23.559195+0000","last_clean":"2024-09-20T22:15:23.559195+0000","last_became_active":"2024-09-20T22:12:52.521497+0000","last_became_peered":"2024-09-20T22:12:52.521497+0000","last_unstale":"2024-09-20T22:15:23.559195+0000","last_undegraded":"2024-09-20T22:15:23.559195+0000","last_fullsized":"2024-09-20T22:15:23.559195+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:18.438451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:18.438451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"328'19","reported_seq":597,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560215+0000","last_change":"2024-09-20T22:14:12.410224+0000","last_active":"2024-09-20T22:15:23.560215+0000","last_peered":"2024-09-20T22:15:23.560215+0000","last_clean":"2024-09-20T22:15:23.560215+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:15:23.560215+0000","last_undegraded":"2024-09-20T22:15:23.560215+0000","last_fullsized":"2024-09-20T22:15:23.560215+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:14:12.410107+0000","last_deep_scrub":"291'13","last_deep_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_clean_scrub_stamp":"2024-09-20T22:14:12.410107+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:29.679127+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":273,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.567249+0000","last_change":"2024-09-20T22:13:11.651173+0000","last_active":"2024-09-20T22:15:23.567249+0000","last_peered":"2024-09-20T22:15:23.567249+0000","last_clean":"2024-09-20T22:15:23.567249+0000","last_became_active":"2024-09-20T22:10:30.655502+0000","last_became_peered":"2024-09-20T22:10:30.655502+0000","last_unstale":"2024-09-20T22:15:23.567249+0000","last_undegraded":"2024-09-20T22:15:23.567249+0000","last_fullsized":"2024-09-20T22:15:23.567249+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:11.650985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:11.650985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":128,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569343+0000","last_change":"2024-09-20T22:14:09.476493+0000","last_active":"2024-09-20T22:15:23.569343+0000","last_peered":"2024-09-20T22:15:23.569343+0000","last_clean":"2024-09-20T22:15:23.569343+0000","last_became_active":"2024-09-20T22:12:52.516953+0000","last_became_peered":"2024-09-20T22:12:52.516953+0000","last_unstale":"2024-09-20T22:15:23.569343+0000","last_undegraded":"2024-09-20T22:15:23.569343+0000","last_fullsized":"2024-09-20T22:15:23.569343+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:09.476386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:09.476386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:29.061649+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":136,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.568977+0000","last_change":"2024-09-20T22:15:20.820202+0000","last_active":"2024-09-20T22:15:23.568977+0000","last_peered":"2024-09-20T22:15:23.568977+0000","last_clean":"2024-09-20T22:15:23.568977+0000","last_became_active":"2024-09-20T22:12:52.514247+0000","last_became_peered":"2024-09-20T22:12:52.514247+0000","last_unstale":"2024-09-20T22:15:23.568977+0000","last_undegraded":"2024-09-20T22:15:23.568977+0000","last_fullsized":"2024-09-20T22:15:23.568977+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.819992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.819992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:39.289842+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"328'19","reported_seq":601,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566852+0000","last_change":"2024-09-20T22:13:38.718567+0000","last_active":"2024-09-20T22:15:23.566852+0000","last_peered":"2024-09-20T22:15:23.566852+0000","last_clean":"2024-09-20T22:15:23.566852+0000","last_became_active":"2024-09-20T22:12:33.545199+0000","last_became_peered":"2024-09-20T22:12:33.545199+0000","last_unstale":"2024-09-20T22:15:23.566852+0000","last_undegraded":"2024-09-20T22:15:23.566852+0000","last_fullsized":"2024-09-20T22:15:23.566852+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:13:38.718450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:38.718450+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":221,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:27.149826+0000","last_change":"2024-09-20T22:15:27.149844+0000","last_active":"2024-09-20T22:15:27.149826+0000","last_peered":"2024-09-20T22:15:27.149826+0000","last_clean":"2024-09-20T22:15:27.149826+0000","last_became_active":"2024-09-20T22:11:30.493598+0000","last_became_peered":"2024-09-20T22:11:30.493598+0000","last_unstale":"2024-09-20T22:15:27.149826+0000","last_undegraded":"2024-09-20T22:15:27.149826+0000","last_fullsized":"2024-09-20T22:15:27.149826+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:27.149777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.149777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:39.146913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":676,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566719+0000","last_change":"2024-09-20T22:14:55.338392+0000","last_active":"2024-09-20T22:15:23.566719+0000","last_peered":"2024-09-20T22:15:23.566719+0000","last_clean":"2024-09-20T22:15:23.566719+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:15:23.566719+0000","last_undegraded":"2024-09-20T22:15:23.566719+0000","last_fullsized":"2024-09-20T22:15:23.566719+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:55.338318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:55.338318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:19.629689+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":125,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569749+0000","last_change":"2024-09-20T22:14:46.684175+0000","last_active":"2024-09-20T22:15:23.569749+0000","last_peered":"2024-09-20T22:15:23.569749+0000","last_clean":"2024-09-20T22:15:23.569749+0000","last_became_active":"2024-09-20T22:12:57.883998+0000","last_became_peered":"2024-09-20T22:12:57.883998+0000","last_unstale":"2024-09-20T22:15:23.569749+0000","last_undegraded":"2024-09-20T22:15:23.569749+0000","last_fullsized":"2024-09-20T22:15:23.569749+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:46.683946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:46.683946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:46.807886+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"328'8","reported_seq":692,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560901+0000","last_change":"2024-09-20T22:14:14.979857+0000","last_active":"2024-09-20T22:15:23.560901+0000","last_peered":"2024-09-20T22:15:23.560901+0000","last_clean":"2024-09-20T22:15:23.560901+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:15:23.560901+0000","last_undegraded":"2024-09-20T22:15:23.560901+0000","last_fullsized":"2024-09-20T22:15:23.560901+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'8","last_scrub_stamp":"2024-09-20T22:14:14.979748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:14.979748+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:35.167435+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":128,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.569383+0000","last_change":"2024-09-20T22:14:21.540990+0000","last_active":"2024-09-20T22:15:23.569383+0000","last_peered":"2024-09-20T22:15:23.569383+0000","last_clean":"2024-09-20T22:15:23.569383+0000","last_became_active":"2024-09-20T22:12:52.517510+0000","last_became_peered":"2024-09-20T22:12:52.517510+0000","last_unstale":"2024-09-20T22:15:23.569383+0000","last_undegraded":"2024-09-20T22:15:23.569383+0000","last_fullsized":"2024-09-20T22:15:23.569383+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:21.540878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:21.540878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:31.469685+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":699,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.564372+0000","last_change":"2024-09-20T22:15:20.569279+0000","last_active":"2024-09-20T22:15:23.564372+0000","last_peered":"2024-09-20T22:15:23.564372+0000","last_clean":"2024-09-20T22:15:23.564372+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:15:23.564372+0000","last_undegraded":"2024-09-20T22:15:23.564372+0000","last_fullsized":"2024-09-20T22:15:23.564372+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.569168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.569168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:31.279582+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":672,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:29.393416+0000","last_change":"2024-09-20T22:15:29.393451+0000","last_active":"2024-09-20T22:15:29.393416+0000","last_peered":"2024-09-20T22:15:29.393416+0000","last_clean":"2024-09-20T22:15:29.393416+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:15:29.393416+0000","last_undegraded":"2024-09-20T22:15:29.393416+0000","last_fullsized":"2024-09-20T22:15:29.393416+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:29.393342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_clean_scrub_stamp":"2024-09-20T22:15:29.393342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:34.744649+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":561,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560449+0000","last_change":"2024-09-20T22:14:13.408076+0000","last_active":"2024-09-20T22:15:23.560449+0000","last_peered":"2024-09-20T22:15:23.560449+0000","last_clean":"2024-09-20T22:15:23.560449+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:15:23.560449+0000","last_undegraded":"2024-09-20T22:15:23.560449+0000","last_fullsized":"2024-09-20T22:15:23.560449+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:13.407980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:13.407980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:41.846519+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":136,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.570731+0000","last_change":"2024-09-20T22:15:18.828980+0000","last_active":"2024-09-20T22:15:23.570731+0000","last_peered":"2024-09-20T22:15:23.570731+0000","last_clean":"2024-09-20T22:15:23.570731+0000","last_became_active":"2024-09-20T22:12:52.518076+0000","last_became_peered":"2024-09-20T22:12:52.518076+0000","last_unstale":"2024-09-20T22:15:23.570731+0000","last_undegraded":"2024-09-20T22:15:23.570731+0000","last_fullsized":"2024-09-20T22:15:23.570731+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:18.828903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:18.828903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:26.559172+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"328'19","reported_seq":720,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.566340+0000","last_change":"2024-09-20T22:14:15.443443+0000","last_active":"2024-09-20T22:15:23.566340+0000","last_peered":"2024-09-20T22:15:23.566340+0000","last_clean":"2024-09-20T22:15:23.566340+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:15:23.566340+0000","last_undegraded":"2024-09-20T22:15:23.566340+0000","last_fullsized":"2024-09-20T22:15:23.566340+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:14:15.443321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:15.443321+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:24.219738+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":136,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:27.761916+0000","last_change":"2024-09-20T22:15:27.761950+0000","last_active":"2024-09-20T22:15:27.761916+0000","last_peered":"2024-09-20T22:15:27.761916+0000","last_clean":"2024-09-20T22:15:27.761916+0000","last_became_active":"2024-09-20T22:12:52.524425+0000","last_became_peered":"2024-09-20T22:12:52.524425+0000","last_unstale":"2024-09-20T22:15:27.761916+0000","last_undegraded":"2024-09-20T22:15:27.761916+0000","last_fullsized":"2024-09-20T22:15:27.761916+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:27.761843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.761843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:46.763986+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":2195259392,"num_objects":33497,"num_object_clones":0,"num_object_copies":100491,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33497,"num_write_kb":2143808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6469844992,"data_stored":6469844992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33497,"ondisk_log_size":33497,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":3300,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":716,"ondisk_log_size":716,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":737,"num_read_kb":732,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":918048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1408,"ondisk_log_size":1408,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813136028,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1084632,"kb_used_data":480072,"kb_used_omap":1,"kb_used_meta":604542,"kb_avail":92660520,"statfs":{"total":95995035648,"available":94884372480,"internally_reserved":0,"allocated":491593728,"data_stored":491244627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":619051466},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430366,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1400900,"kb_used_data":964340,"kb_used_omap":1,"kb_used_meta":436542,"kb_avail":92344252,"statfs":{"total":95995035648,"available":94560514048,"internally_reserved":0,"allocated":987484160,"data_stored":987155337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":447019462},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724708,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1274240,"kb_used_data":643248,"kb_used_omap":1,"kb_used_meta":630910,"kb_avail":92470912,"statfs":{"total":95995035648,"available":94690213888,"internally_reserved":0,"allocated":658685952,"data_stored":658360144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":646052293},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986348,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1407192,"kb_used_data":938696,"kb_used_omap":1,"kb_used_meta":468478,"kb_avail":92337960,"statfs":{"total":95995035648,"available":94554071040,"internally_reserved":0,"allocated":961224704,"data_stored":960874892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":479721925},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770258025,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":877996,"kb_used_data":419356,"kb_used_omap":4,"kb_used_meta":458619,"kb_avail":92867156,"statfs":{"total":95995035648,"available":95095967744,"internally_reserved":0,"allocated":429420544,"data_stored":429051549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":469626536},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149873,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1299688,"kb_used_data":636824,"kb_used_omap":1,"kb_used_meta":662846,"kb_avail":92445464,"statfs":{"total":95995035648,"available":94664155136,"internally_reserved":0,"allocated":652107776,"data_stored":651742872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":678754759},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411510,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1274112,"kb_used_data":755632,"kb_used_omap":1,"kb_used_meta":518462,"kb_avail":92471040,"statfs":{"total":95995035648,"available":94690344960,"internally_reserved":0,"allocated":773767168,"data_stored":773441803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":530905541},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738561,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1970324,"kb_used_data":1317188,"kb_used_omap":1,"kb_used_meta":653118,"kb_avail":91774828,"statfs":{"total":95995035648,"available":93977423872,"internally_reserved":0,"allocated":1348800512,"data_stored":1348454985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":668793285},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1362886656,"data_stored":1362886656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":445054976,"data_stored":445054976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":794165248,"data_stored":794165248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":678821888,"data_stored":678821888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1012465664,"data_stored":1012465664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":662241280,"data_stored":662241280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1008664576,"data_stored":1008664576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":505544704,"data_stored":505544704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:15:37.075 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-20T22:15:37.075 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:15:37.076 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2024-09-20T22:15:37.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:15:37.130 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:37.196 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:37.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:37.197 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:37.277 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T22:15:37.299 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:37.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:37.368 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:37.368 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:37.458 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:15:37.469 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:37.503 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:37.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:37.504 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:37.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:37 smithi067 ceph-mon[33511]: pgmap v678: 158 pgs: 158 active+clean; 2.2 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-20T22:15:37.574 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:37.605 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:37.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:37.645 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:37.645 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:37.711 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:15:37.712 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:37 smithi060 ceph-mon[29694]: pgmap v678: 158 pgs: 158 active+clean; 2.2 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-20T22:15:37.712 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:37 smithi060 ceph-mon[40290]: pgmap v678: 158 pgs: 158 active+clean; 2.2 GiB data, 11 GiB used, 705 GiB / 715 GiB avail; 1.2 KiB/s rd, 95 MiB/s wr, 1.52k op/s 2024-09-20T22:15:37.746 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:37.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:37.790 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:37.790 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:37.890 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:37.920 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:15:37.920013+0000 min lat: 0.00137086 max lat: 0.272903 avg lat: 0.00741907 2024-09-20T22:15:37.920 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:15:37.920 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 20 15 43121 43106 134.67 103.062 0.0102303 0.00741907 2024-09-20T22:15:37.947 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:37.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:37.948 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:38.048 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:38.083 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:38.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:38.084 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:38.185 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:38.219 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:38.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:38.220 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:38.229 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:38.253 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:15:38.321 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:38.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:38.384 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:38.384 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:38.484 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:38.536 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:38.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:38.537 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:38.560 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:15:38.561 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:15:38.638 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:38.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:38.673 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:38.673 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:38.774 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:38.813 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:38.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:38.814 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:38.915 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:38.920 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 21 16 44542 44526 132.483 88.75 0.0154701 0.00754216 2024-09-20T22:15:38.950 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:38.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:38.952 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:39.052 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:39.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:39.091 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:39.091 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:39.192 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:39.225 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:39.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:39.227 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:39.327 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:39.363 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:39.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:39.363 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:39.464 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:39.467 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:15:39.509 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:39.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:39.511 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:39.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:39 smithi060 ceph-mon[29694]: pgmap v679: 158 pgs: 158 active+clean; 2.5 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 129 MiB/s wr, 2.07k op/s 2024-09-20T22:15:39.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:39 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2719443257' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:39.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:39 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:39.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:39 smithi060 ceph-mon[40290]: pgmap v679: 158 pgs: 158 active+clean; 2.5 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 129 MiB/s wr, 2.07k op/s 2024-09-20T22:15:39.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:39 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2719443257' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:39.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:39 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:39.612 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:39.648 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:39.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:39.649 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:39.686 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:39 smithi067 ceph-mon[33511]: pgmap v679: 158 pgs: 158 active+clean; 2.5 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 129 MiB/s wr, 2.07k op/s 2024-09-20T22:15:39.686 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:39 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2719443257' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:39.686 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:39 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:39.748 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:15:39.748 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:15:39.748 INFO:teuthology.orchestra.run.smithi067.stdout: "complaint_time": 30, 2024-09-20T22:15:39.748 INFO:teuthology.orchestra.run.smithi067.stdout: "num_blocked_ops": 0 2024-09-20T22:15:39.748 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:15:39.749 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:39.761 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":679,"stamp":"2024-09-20T22:15:38.222477+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2687834651,"num_objects":41203,"num_object_clones":0,"num_object_copies":123609,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41203,"num_whiteouts":0,"num_read":1876,"num_read_kb":1628,"num_write":41726,"num_write_kb":2624301,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":923923,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43243,"ondisk_log_size":43243,"up":474,"acting":474,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":474,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":12098192,"kb_used_data":7646380,"kb_used_omap":15,"kb_used_meta":4451568,"kb_avail":737863024,"statfs":{"total":767960285184,"available":755571736576,"internally_reserved":0,"allocated":7829893120,"data_stored":7827088115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":4558406419},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1086717952,"num_objects":16582,"num_object_clones":0,"num_object_copies":49746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16582,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":16582,"num_write_kb":1061248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003202"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":132,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.449270+0000","last_change":"2024-09-20T22:14:57.573764+0000","last_active":"2024-09-20T22:15:35.449270+0000","last_peered":"2024-09-20T22:15:35.449270+0000","last_clean":"2024-09-20T22:15:35.449270+0000","last_became_active":"2024-09-20T22:12:52.515921+0000","last_became_peered":"2024-09-20T22:12:52.515921+0000","last_unstale":"2024-09-20T22:15:35.449270+0000","last_undegraded":"2024-09-20T22:15:35.449270+0000","last_fullsized":"2024-09-20T22:15:35.449270+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:57.573679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:48.018157+0000","last_clean_scrub_stamp":"2024-09-20T22:14:57.573679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:20.753739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"327'19","reported_seq":722,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.442544+0000","last_change":"2024-09-20T22:15:22.984401+0000","last_active":"2024-09-20T22:15:35.442544+0000","last_peered":"2024-09-20T22:15:35.442544+0000","last_clean":"2024-09-20T22:15:35.442544+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:15:35.442544+0000","last_undegraded":"2024-09-20T22:15:35.442544+0000","last_fullsized":"2024-09-20T22:15:35.442544+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"327'19","last_scrub_stamp":"2024-09-20T22:15:22.984271+0000","last_deep_scrub":"252'13","last_deep_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_clean_scrub_stamp":"2024-09-20T22:15:22.984271+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:25.020372+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":563,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559455+0000","last_change":"2024-09-20T22:14:15.391149+0000","last_active":"2024-09-20T22:15:23.559455+0000","last_peered":"2024-09-20T22:15:23.559455+0000","last_clean":"2024-09-20T22:15:23.559455+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:15:23.559455+0000","last_undegraded":"2024-09-20T22:15:23.559455+0000","last_fullsized":"2024-09-20T22:15:23.559455+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:15.391049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:15.391049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:37.227878+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":689,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.442475+0000","last_change":"2024-09-20T22:15:18.062035+0000","last_active":"2024-09-20T22:15:35.442475+0000","last_peered":"2024-09-20T22:15:35.442475+0000","last_clean":"2024-09-20T22:15:35.442475+0000","last_became_active":"2024-09-20T22:11:30.485735+0000","last_became_peered":"2024-09-20T22:11:30.485735+0000","last_unstale":"2024-09-20T22:15:35.442475+0000","last_undegraded":"2024-09-20T22:15:35.442475+0000","last_fullsized":"2024-09-20T22:15:35.442475+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:18.061924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:18.061924+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:19.499592+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":676,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451227+0000","last_change":"2024-09-20T22:14:56.160729+0000","last_active":"2024-09-20T22:15:35.451227+0000","last_peered":"2024-09-20T22:15:35.451227+0000","last_clean":"2024-09-20T22:15:35.451227+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:15:35.451227+0000","last_undegraded":"2024-09-20T22:15:35.451227+0000","last_fullsized":"2024-09-20T22:15:35.451227+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:14:56.160638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:56.160638+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:07.044469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":159,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.447678+0000","last_change":"2024-09-20T22:14:52.402426+0000","last_active":"2024-09-20T22:15:35.447678+0000","last_peered":"2024-09-20T22:15:35.447678+0000","last_clean":"2024-09-20T22:15:35.447678+0000","last_became_active":"2024-09-20T22:12:33.545100+0000","last_became_peered":"2024-09-20T22:12:33.545100+0000","last_unstale":"2024-09-20T22:15:35.447678+0000","last_undegraded":"2024-09-20T22:15:35.447678+0000","last_fullsized":"2024-09-20T22:15:35.447678+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:52.402299+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:52.402299+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:13.318752+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"328'27","reported_seq":275,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.457161+0000","last_change":"2024-09-20T22:15:20.902301+0000","last_active":"2024-09-20T22:15:35.457161+0000","last_peered":"2024-09-20T22:15:35.457161+0000","last_clean":"2024-09-20T22:15:35.457161+0000","last_became_active":"2024-09-20T22:12:52.517296+0000","last_became_peered":"2024-09-20T22:12:52.517296+0000","last_unstale":"2024-09-20T22:15:35.457161+0000","last_undegraded":"2024-09-20T22:15:35.457161+0000","last_fullsized":"2024-09-20T22:15:35.457161+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:15:20.902179+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.902179+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:24.675476+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":681,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.442452+0000","last_change":"2024-09-20T22:15:26.080144+0000","last_active":"2024-09-20T22:15:35.442452+0000","last_peered":"2024-09-20T22:15:35.442452+0000","last_clean":"2024-09-20T22:15:35.442452+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:15:35.442452+0000","last_undegraded":"2024-09-20T22:15:35.442452+0000","last_fullsized":"2024-09-20T22:15:35.442452+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:26.080040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:26.080040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:31.327638+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":703,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.452895+0000","last_change":"2024-09-20T22:15:21.178920+0000","last_active":"2024-09-20T22:15:35.452895+0000","last_peered":"2024-09-20T22:15:35.452895+0000","last_clean":"2024-09-20T22:15:35.452895+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:15:35.452895+0000","last_undegraded":"2024-09-20T22:15:35.452895+0000","last_fullsized":"2024-09-20T22:15:35.452895+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:21.178806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:15:21.178806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:36.671122+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"328'15","reported_seq":741,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.442759+0000","last_change":"2024-09-20T22:14:19.076811+0000","last_active":"2024-09-20T22:15:35.442759+0000","last_peered":"2024-09-20T22:15:35.442759+0000","last_clean":"2024-09-20T22:15:35.442759+0000","last_became_active":"2024-09-20T22:12:52.516894+0000","last_became_peered":"2024-09-20T22:12:52.516894+0000","last_unstale":"2024-09-20T22:15:35.442759+0000","last_undegraded":"2024-09-20T22:15:35.442759+0000","last_fullsized":"2024-09-20T22:15:35.442759+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:14:19.076696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:19.076696+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:45.728095+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":234,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.452050+0000","last_change":"2024-09-20T22:14:54.153118+0000","last_active":"2024-09-20T22:15:35.452050+0000","last_peered":"2024-09-20T22:15:35.452050+0000","last_clean":"2024-09-20T22:15:35.452050+0000","last_became_active":"2024-09-20T22:12:52.518549+0000","last_became_peered":"2024-09-20T22:12:52.518549+0000","last_unstale":"2024-09-20T22:15:35.452050+0000","last_undegraded":"2024-09-20T22:15:35.452050+0000","last_fullsized":"2024-09-20T22:15:35.452050+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:54.152717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:54.152717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:03.431527+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":140,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453683+0000","last_change":"2024-09-20T22:14:58.570997+0000","last_active":"2024-09-20T22:15:35.453683+0000","last_peered":"2024-09-20T22:15:35.453683+0000","last_clean":"2024-09-20T22:15:35.453683+0000","last_became_active":"2024-09-20T22:12:52.524631+0000","last_became_peered":"2024-09-20T22:12:52.524631+0000","last_unstale":"2024-09-20T22:15:35.453683+0000","last_undegraded":"2024-09-20T22:15:35.453683+0000","last_fullsized":"2024-09-20T22:15:35.453683+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:58.570923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:58.570923+0000","last_clean_scrub_stamp":"2024-09-20T22:14:58.570923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:25.107938+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":708,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.452560+0000","last_change":"2024-09-20T22:15:20.182058+0000","last_active":"2024-09-20T22:15:35.452560+0000","last_peered":"2024-09-20T22:15:35.452560+0000","last_clean":"2024-09-20T22:15:35.452560+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:15:35.452560+0000","last_undegraded":"2024-09-20T22:15:35.452560+0000","last_fullsized":"2024-09-20T22:15:35.452560+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.181942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:30.215263+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.181942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:46.935103+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"328'7","reported_seq":718,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.443858+0000","last_change":"2024-09-20T22:15:31.119172+0000","last_active":"2024-09-20T22:15:35.443858+0000","last_peered":"2024-09-20T22:15:35.443858+0000","last_clean":"2024-09-20T22:15:35.443858+0000","last_became_active":"2024-09-20T22:08:22.144364+0000","last_became_peered":"2024-09-20T22:08:22.144364+0000","last_unstale":"2024-09-20T22:15:35.443858+0000","last_undegraded":"2024-09-20T22:15:35.443858+0000","last_fullsized":"2024-09-20T22:15:35.443858+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'7","last_scrub_stamp":"2024-09-20T22:15:31.119046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:31.119046+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.446818+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":689,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451240+0000","last_change":"2024-09-20T22:14:57.354434+0000","last_active":"2024-09-20T22:15:35.451240+0000","last_peered":"2024-09-20T22:15:35.451240+0000","last_clean":"2024-09-20T22:15:35.451240+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:15:35.451240+0000","last_undegraded":"2024-09-20T22:15:35.451240+0000","last_fullsized":"2024-09-20T22:15:35.451240+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:57.354326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:57.354326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:14.925355+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":222,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451166+0000","last_change":"2024-09-20T22:15:00.077644+0000","last_active":"2024-09-20T22:15:35.451166+0000","last_peered":"2024-09-20T22:15:35.451166+0000","last_clean":"2024-09-20T22:15:35.451166+0000","last_became_active":"2024-09-20T22:11:30.487137+0000","last_became_peered":"2024-09-20T22:11:30.487137+0000","last_unstale":"2024-09-20T22:15:35.451166+0000","last_undegraded":"2024-09-20T22:15:35.451166+0000","last_fullsized":"2024-09-20T22:15:35.451166+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:00.077519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:15:00.077519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:11.908799+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":677,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.443782+0000","last_change":"2024-09-20T22:15:17.029127+0000","last_active":"2024-09-20T22:15:35.443782+0000","last_peered":"2024-09-20T22:15:35.443782+0000","last_clean":"2024-09-20T22:15:35.443782+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:15:35.443782+0000","last_undegraded":"2024-09-20T22:15:35.443782+0000","last_fullsized":"2024-09-20T22:15:35.443782+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:17.029051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:15:17.029051+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:29.616781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":140,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.448811+0000","last_change":"2024-09-20T22:15:24.748006+0000","last_active":"2024-09-20T22:15:35.448811+0000","last_peered":"2024-09-20T22:15:35.448811+0000","last_clean":"2024-09-20T22:15:35.448811+0000","last_became_active":"2024-09-20T22:12:52.526097+0000","last_became_peered":"2024-09-20T22:12:52.526097+0000","last_unstale":"2024-09-20T22:15:35.448811+0000","last_undegraded":"2024-09-20T22:15:35.448811+0000","last_fullsized":"2024-09-20T22:15:35.448811+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.747891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.747891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:29.188832+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"328'34","reported_seq":278,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.457218+0000","last_change":"2024-09-20T22:14:50.993713+0000","last_active":"2024-09-20T22:15:35.457218+0000","last_peered":"2024-09-20T22:15:35.457218+0000","last_clean":"2024-09-20T22:15:35.457218+0000","last_became_active":"2024-09-20T22:11:12.507311+0000","last_became_peered":"2024-09-20T22:11:12.507311+0000","last_unstale":"2024-09-20T22:15:35.457218+0000","last_undegraded":"2024-09-20T22:15:35.457218+0000","last_fullsized":"2024-09-20T22:15:35.457218+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'34","last_scrub_stamp":"2024-09-20T22:14:50.993563+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:14:50.993563+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:53.748023+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":525,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560601+0000","last_change":"2024-09-20T22:13:11.053381+0000","last_active":"2024-09-20T22:15:23.560601+0000","last_peered":"2024-09-20T22:15:23.560601+0000","last_clean":"2024-09-20T22:15:23.560601+0000","last_became_active":"2024-09-20T22:12:52.520985+0000","last_became_peered":"2024-09-20T22:12:52.520985+0000","last_unstale":"2024-09-20T22:15:23.560601+0000","last_undegraded":"2024-09-20T22:15:23.560601+0000","last_fullsized":"2024-09-20T22:15:23.560601+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:11.053276+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:11.053276+0000","last_clean_scrub_stamp":"2024-09-20T22:13:11.053276+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":628,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.447753+0000","last_change":"2024-09-20T22:14:19.451519+0000","last_active":"2024-09-20T22:15:35.447753+0000","last_peered":"2024-09-20T22:15:35.447753+0000","last_clean":"2024-09-20T22:15:35.447753+0000","last_became_active":"2024-09-20T22:12:21.521433+0000","last_became_peered":"2024-09-20T22:12:21.521433+0000","last_unstale":"2024-09-20T22:15:35.447753+0000","last_undegraded":"2024-09-20T22:15:35.447753+0000","last_fullsized":"2024-09-20T22:15:35.447753+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:19.451409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:19.451409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:42.318241+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":629,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453102+0000","last_change":"2024-09-20T22:12:44.197752+0000","last_active":"2024-09-20T22:15:35.453102+0000","last_peered":"2024-09-20T22:15:35.453102+0000","last_clean":"2024-09-20T22:15:35.453102+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:15:35.453102+0000","last_undegraded":"2024-09-20T22:15:35.453102+0000","last_fullsized":"2024-09-20T22:15:35.453102+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:44.197654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:44.197654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"328'23","reported_seq":598,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560790+0000","last_change":"2024-09-20T22:15:17.533607+0000","last_active":"2024-09-20T22:15:23.560790+0000","last_peered":"2024-09-20T22:15:23.560790+0000","last_clean":"2024-09-20T22:15:23.560790+0000","last_became_active":"2024-09-20T22:12:21.517147+0000","last_became_peered":"2024-09-20T22:12:21.517147+0000","last_unstale":"2024-09-20T22:15:23.560790+0000","last_undegraded":"2024-09-20T22:15:23.560790+0000","last_fullsized":"2024-09-20T22:15:23.560790+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'23","last_scrub_stamp":"2024-09-20T22:15:17.533453+0000","last_deep_scrub":"250'17","last_deep_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_clean_scrub_stamp":"2024-09-20T22:15:17.533453+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:38.348665+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":680,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.443052+0000","last_change":"2024-09-20T22:15:33.188818+0000","last_active":"2024-09-20T22:15:35.443052+0000","last_peered":"2024-09-20T22:15:35.443052+0000","last_clean":"2024-09-20T22:15:35.443052+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:15:35.443052+0000","last_undegraded":"2024-09-20T22:15:35.443052+0000","last_fullsized":"2024-09-20T22:15:35.443052+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:33.188747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:33.188747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:40.715215+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":670,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451286+0000","last_change":"2024-09-20T22:13:52.688758+0000","last_active":"2024-09-20T22:15:35.451286+0000","last_peered":"2024-09-20T22:15:35.451286+0000","last_clean":"2024-09-20T22:15:35.451286+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:15:35.451286+0000","last_undegraded":"2024-09-20T22:15:35.451286+0000","last_fullsized":"2024-09-20T22:15:35.451286+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:13:52.688650+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:13:52.688650+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.321058+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"289'2","reported_seq":523,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559543+0000","last_change":"2024-09-20T22:14:22.436454+0000","last_active":"2024-09-20T22:15:23.559543+0000","last_peered":"2024-09-20T22:15:23.559543+0000","last_clean":"2024-09-20T22:15:23.559543+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:15:23.559543+0000","last_undegraded":"2024-09-20T22:15:23.559543+0000","last_fullsized":"2024-09-20T22:15:23.559543+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-20T22:14:22.436308+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_clean_scrub_stamp":"2024-09-20T22:14:22.436308+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:51.653333+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"328'20","reported_seq":159,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.448882+0000","last_change":"2024-09-20T22:14:11.444573+0000","last_active":"2024-09-20T22:15:35.448882+0000","last_peered":"2024-09-20T22:15:35.448882+0000","last_clean":"2024-09-20T22:15:35.448882+0000","last_became_active":"2024-09-20T22:12:52.526803+0000","last_became_peered":"2024-09-20T22:12:52.526803+0000","last_unstale":"2024-09-20T22:15:35.448882+0000","last_undegraded":"2024-09-20T22:15:35.448882+0000","last_fullsized":"2024-09-20T22:15:35.448882+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:14:11.444444+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:11.444444+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.306318+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"291'11","reported_seq":882,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.447811+0000","last_change":"2024-09-20T22:15:28.500785+0000","last_active":"2024-09-20T22:15:35.447811+0000","last_peered":"2024-09-20T22:15:35.447811+0000","last_clean":"2024-09-20T22:15:35.447811+0000","last_became_active":"2024-09-20T22:11:40.601048+0000","last_became_peered":"2024-09-20T22:11:40.601048+0000","last_unstale":"2024-09-20T22:15:35.447811+0000","last_undegraded":"2024-09-20T22:15:35.447811+0000","last_fullsized":"2024-09-20T22:15:35.447811+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'11","last_scrub_stamp":"2024-09-20T22:15:28.500437+0000","last_deep_scrub":"291'11","last_deep_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_clean_scrub_stamp":"2024-09-20T22:15:28.500437+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:30.705651+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"380'2361","reported_seq":2391,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.800702+0000","last_change":"2024-09-20T22:15:08.927837+0000","last_active":"2024-09-20T22:15:35.800702+0000","last_peered":"2024-09-20T22:15:35.800702+0000","last_clean":"2024-09-20T22:15:35.800702+0000","last_became_active":"2024-09-20T22:15:08.927535+0000","last_became_peered":"2024-09-20T22:15:08.927535+0000","last_unstale":"2024-09-20T22:15:35.800702+0000","last_undegraded":"2024-09-20T22:15:35.800702+0000","last_fullsized":"2024-09-20T22:15:35.800702+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2361,"log_dups_size":0,"ondisk_log_size":2361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:17.347573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154730496,"num_objects":2361,"num_object_clones":0,"num_object_copies":7083,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2361,"num_write_kb":151104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":684,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451719+0000","last_change":"2024-09-20T22:13:08.407055+0000","last_active":"2024-09-20T22:15:35.451719+0000","last_peered":"2024-09-20T22:15:35.451719+0000","last_clean":"2024-09-20T22:15:35.451719+0000","last_became_active":"2024-09-20T22:08:48.452786+0000","last_became_peered":"2024-09-20T22:08:48.452786+0000","last_unstale":"2024-09-20T22:15:35.451719+0000","last_undegraded":"2024-09-20T22:15:35.451719+0000","last_fullsized":"2024-09-20T22:15:35.451719+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:08.406883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:13:08.406883+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"328'27","reported_seq":472,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.458455+0000","last_change":"2024-09-20T22:14:46.998282+0000","last_active":"2024-09-20T22:15:35.458455+0000","last_peered":"2024-09-20T22:15:35.458455+0000","last_clean":"2024-09-20T22:15:35.458455+0000","last_became_active":"2024-09-20T22:08:22.145408+0000","last_became_peered":"2024-09-20T22:08:22.145408+0000","last_unstale":"2024-09-20T22:15:35.458455+0000","last_undegraded":"2024-09-20T22:15:35.458455+0000","last_fullsized":"2024-09-20T22:15:35.458455+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:14:46.998142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:46.998142+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:49.977452+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":696,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451126+0000","last_change":"2024-09-20T22:15:28.020435+0000","last_active":"2024-09-20T22:15:35.451126+0000","last_peered":"2024-09-20T22:15:35.451126+0000","last_clean":"2024-09-20T22:15:35.451126+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:15:35.451126+0000","last_undegraded":"2024-09-20T22:15:35.451126+0000","last_fullsized":"2024-09-20T22:15:35.451126+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:28.020341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:28.020341+0000","last_clean_scrub_stamp":"2024-09-20T22:15:28.020341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:31.565898+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"326'13","reported_seq":903,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.443791+0000","last_change":"2024-09-20T22:14:50.079897+0000","last_active":"2024-09-20T22:15:35.443791+0000","last_peered":"2024-09-20T22:15:35.443791+0000","last_clean":"2024-09-20T22:15:35.443791+0000","last_became_active":"2024-09-20T22:12:52.517588+0000","last_became_peered":"2024-09-20T22:12:52.517588+0000","last_unstale":"2024-09-20T22:15:35.443791+0000","last_undegraded":"2024-09-20T22:15:35.443791+0000","last_fullsized":"2024-09-20T22:15:35.443791+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'13","last_scrub_stamp":"2024-09-20T22:14:50.079818+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:50.079818+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:57.638038+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":234,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453307+0000","last_change":"2024-09-20T22:15:33.082489+0000","last_active":"2024-09-20T22:15:35.453307+0000","last_peered":"2024-09-20T22:15:35.453307+0000","last_clean":"2024-09-20T22:15:35.453307+0000","last_became_active":"2024-09-20T22:12:52.517826+0000","last_became_peered":"2024-09-20T22:12:52.517826+0000","last_unstale":"2024-09-20T22:15:35.453307+0000","last_undegraded":"2024-09-20T22:15:35.453307+0000","last_fullsized":"2024-09-20T22:15:35.453307+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:33.082368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:33.082368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:58.107873+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"380'2432","reported_seq":2462,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.793238+0000","last_change":"2024-09-20T22:15:08.928365+0000","last_active":"2024-09-20T22:15:35.793238+0000","last_peered":"2024-09-20T22:15:35.793238+0000","last_clean":"2024-09-20T22:15:35.793238+0000","last_became_active":"2024-09-20T22:15:08.928064+0000","last_became_peered":"2024-09-20T22:15:08.928064+0000","last_unstale":"2024-09-20T22:15:35.793238+0000","last_undegraded":"2024-09-20T22:15:35.793238+0000","last_fullsized":"2024-09-20T22:15:35.793238+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2432,"log_dups_size":0,"ondisk_log_size":2432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:27.570950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":159383552,"num_objects":2432,"num_object_clones":0,"num_object_copies":7296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2432,"num_write_kb":155648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":684,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.443860+0000","last_change":"2024-09-20T22:14:21.059023+0000","last_active":"2024-09-20T22:15:35.443860+0000","last_peered":"2024-09-20T22:15:35.443860+0000","last_clean":"2024-09-20T22:15:35.443860+0000","last_became_active":"2024-09-20T22:08:48.452475+0000","last_became_peered":"2024-09-20T22:08:48.452475+0000","last_unstale":"2024-09-20T22:15:35.443860+0000","last_undegraded":"2024-09-20T22:15:35.443860+0000","last_fullsized":"2024-09-20T22:15:35.443860+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:14:21.058895+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"328'30","reported_seq":692,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451400+0000","last_change":"2024-09-20T22:13:50.715007+0000","last_active":"2024-09-20T22:15:35.451400+0000","last_peered":"2024-09-20T22:15:35.451400+0000","last_clean":"2024-09-20T22:15:35.451400+0000","last_became_active":"2024-09-20T22:12:52.526867+0000","last_became_peered":"2024-09-20T22:12:52.526867+0000","last_unstale":"2024-09-20T22:15:35.451400+0000","last_undegraded":"2024-09-20T22:15:35.451400+0000","last_fullsized":"2024-09-20T22:15:35.451400+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'30","last_scrub_stamp":"2024-09-20T22:13:50.714867+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:50.714867+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.321114+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":173,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453650+0000","last_change":"2024-09-20T22:14:09.922166+0000","last_active":"2024-09-20T22:15:35.453650+0000","last_peered":"2024-09-20T22:15:35.453650+0000","last_clean":"2024-09-20T22:15:35.453650+0000","last_became_active":"2024-09-20T22:12:52.528947+0000","last_became_peered":"2024-09-20T22:12:52.528947+0000","last_unstale":"2024-09-20T22:15:35.453650+0000","last_undegraded":"2024-09-20T22:15:35.453650+0000","last_fullsized":"2024-09-20T22:15:35.453650+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:09.922092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:09.922092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.321257+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":693,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.442698+0000","last_change":"2024-09-20T22:14:13.046284+0000","last_active":"2024-09-20T22:15:35.442698+0000","last_peered":"2024-09-20T22:15:35.442698+0000","last_clean":"2024-09-20T22:15:35.442698+0000","last_became_active":"2024-09-20T22:10:48.919949+0000","last_became_peered":"2024-09-20T22:10:48.919949+0000","last_unstale":"2024-09-20T22:15:35.442698+0000","last_undegraded":"2024-09-20T22:15:35.442698+0000","last_fullsized":"2024-09-20T22:15:35.442698+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:13.046189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:13.046189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":677,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.442666+0000","last_change":"2024-09-20T22:14:08.158795+0000","last_active":"2024-09-20T22:15:35.442666+0000","last_peered":"2024-09-20T22:15:35.442666+0000","last_clean":"2024-09-20T22:15:35.442666+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:15:35.442666+0000","last_undegraded":"2024-09-20T22:15:35.442666+0000","last_fullsized":"2024-09-20T22:15:35.442666+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:08.158688+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:08.158688+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"380'2694","reported_seq":2724,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:37.917151+0000","last_change":"2024-09-20T22:15:09.088949+0000","last_active":"2024-09-20T22:15:37.917151+0000","last_peered":"2024-09-20T22:15:37.917151+0000","last_clean":"2024-09-20T22:15:37.917151+0000","last_became_active":"2024-09-20T22:15:09.088580+0000","last_became_peered":"2024-09-20T22:15:09.088580+0000","last_unstale":"2024-09-20T22:15:37.917151+0000","last_undegraded":"2024-09-20T22:15:37.917151+0000","last_fullsized":"2024-09-20T22:15:37.917151+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2694,"log_dups_size":0,"ondisk_log_size":2694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:33.130074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176553984,"num_objects":2694,"num_object_clones":0,"num_object_copies":8082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2694,"num_write_kb":172416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":718,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453201+0000","last_change":"2024-09-20T22:15:27.026088+0000","last_active":"2024-09-20T22:15:35.453201+0000","last_peered":"2024-09-20T22:15:35.453201+0000","last_clean":"2024-09-20T22:15:35.453201+0000","last_became_active":"2024-09-20T22:11:33.560632+0000","last_became_peered":"2024-09-20T22:11:33.560632+0000","last_unstale":"2024-09-20T22:15:35.453201+0000","last_undegraded":"2024-09-20T22:15:35.453201+0000","last_fullsized":"2024-09-20T22:15:35.453201+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":286,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:27.025812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.025812+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:39.819835+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"328'28","reported_seq":727,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.443230+0000","last_change":"2024-09-20T22:14:55.064788+0000","last_active":"2024-09-20T22:15:35.443230+0000","last_peered":"2024-09-20T22:15:35.443230+0000","last_clean":"2024-09-20T22:15:35.443230+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:15:35.443230+0000","last_undegraded":"2024-09-20T22:15:35.443230+0000","last_fullsized":"2024-09-20T22:15:35.443230+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:14:55.064689+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:55.064689+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:59.495396+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":144,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.449084+0000","last_change":"2024-09-20T22:15:15.781305+0000","last_active":"2024-09-20T22:15:35.449084+0000","last_peered":"2024-09-20T22:15:35.449084+0000","last_clean":"2024-09-20T22:15:35.449084+0000","last_became_active":"2024-09-20T22:12:52.528068+0000","last_became_peered":"2024-09-20T22:12:52.528068+0000","last_unstale":"2024-09-20T22:15:35.449084+0000","last_undegraded":"2024-09-20T22:15:35.449084+0000","last_fullsized":"2024-09-20T22:15:35.449084+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:15:15.781222+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:15.781222+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:24.974390+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"315'11","reported_seq":409,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:25.323093+0000","last_change":"2024-09-20T22:13:05.177108+0000","last_active":"2024-09-20T22:15:25.323093+0000","last_peered":"2024-09-20T22:15:25.323093+0000","last_clean":"2024-09-20T22:15:25.323093+0000","last_became_active":"2024-09-20T22:12:33.545942+0000","last_became_peered":"2024-09-20T22:12:33.545942+0000","last_unstale":"2024-09-20T22:15:25.323093+0000","last_undegraded":"2024-09-20T22:15:25.323093+0000","last_fullsized":"2024-09-20T22:15:25.323093+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:13:05.176973+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:05.176973+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":669,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451575+0000","last_change":"2024-09-20T22:13:10.621055+0000","last_active":"2024-09-20T22:15:35.451575+0000","last_peered":"2024-09-20T22:15:35.451575+0000","last_clean":"2024-09-20T22:15:35.451575+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:15:35.451575+0000","last_undegraded":"2024-09-20T22:15:35.451575+0000","last_fullsized":"2024-09-20T22:15:35.451575+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:10.620925+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.620925+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.320967+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"380'2412","reported_seq":2441,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.835698+0000","last_change":"2024-09-20T22:15:08.931965+0000","last_active":"2024-09-20T22:15:35.835698+0000","last_peered":"2024-09-20T22:15:35.835698+0000","last_clean":"2024-09-20T22:15:35.835698+0000","last_became_active":"2024-09-20T22:15:08.931653+0000","last_became_peered":"2024-09-20T22:15:08.931653+0000","last_unstale":"2024-09-20T22:15:35.835698+0000","last_undegraded":"2024-09-20T22:15:35.835698+0000","last_fullsized":"2024-09-20T22:15:35.835698+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2412,"log_dups_size":0,"ondisk_log_size":2412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:32.473356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":158072832,"num_objects":2412,"num_object_clones":0,"num_object_copies":7236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2412,"num_write_kb":154368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":714,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.447644+0000","last_change":"2024-09-20T22:14:23.503489+0000","last_active":"2024-09-20T22:15:35.447644+0000","last_peered":"2024-09-20T22:15:35.447644+0000","last_clean":"2024-09-20T22:15:35.447644+0000","last_became_active":"2024-09-20T22:10:33.660948+0000","last_became_peered":"2024-09-20T22:10:33.660948+0000","last_unstale":"2024-09-20T22:15:35.447644+0000","last_undegraded":"2024-09-20T22:15:35.447644+0000","last_fullsized":"2024-09-20T22:15:35.447644+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":255,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:14:23.503360+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:14:23.503360+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:52.579727+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"328'20","reported_seq":591,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560075+0000","last_change":"2024-09-20T22:13:41.421053+0000","last_active":"2024-09-20T22:15:23.560075+0000","last_peered":"2024-09-20T22:15:23.560075+0000","last_clean":"2024-09-20T22:15:23.560075+0000","last_became_active":"2024-09-20T22:11:30.486532+0000","last_became_peered":"2024-09-20T22:11:30.486532+0000","last_unstale":"2024-09-20T22:15:23.560075+0000","last_undegraded":"2024-09-20T22:15:23.560075+0000","last_fullsized":"2024-09-20T22:15:23.560075+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:13:41.420933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:41.420933+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":704,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.452969+0000","last_change":"2024-09-20T22:14:11.402471+0000","last_active":"2024-09-20T22:15:35.452969+0000","last_peered":"2024-09-20T22:15:35.452969+0000","last_clean":"2024-09-20T22:15:35.452969+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:15:35.452969+0000","last_undegraded":"2024-09-20T22:15:35.452969+0000","last_fullsized":"2024-09-20T22:15:35.452969+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:11.402361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:14:11.402361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"363'13","reported_seq":167,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.448612+0000","last_change":"2024-09-20T22:14:52.612861+0000","last_active":"2024-09-20T22:15:35.448612+0000","last_peered":"2024-09-20T22:15:35.448612+0000","last_clean":"2024-09-20T22:15:35.448612+0000","last_became_active":"2024-09-20T22:12:57.884415+0000","last_became_peered":"2024-09-20T22:12:57.884415+0000","last_unstale":"2024-09-20T22:15:35.448612+0000","last_undegraded":"2024-09-20T22:15:35.448612+0000","last_fullsized":"2024-09-20T22:15:35.448612+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"363'13","last_scrub_stamp":"2024-09-20T22:14:52.612615+0000","last_deep_scrub":"363'13","last_deep_scrub_stamp":"2024-09-20T22:14:52.612615+0000","last_clean_scrub_stamp":"2024-09-20T22:14:52.612615+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:56.113350+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":132,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453515+0000","last_change":"2024-09-20T22:14:55.614617+0000","last_active":"2024-09-20T22:15:35.453515+0000","last_peered":"2024-09-20T22:15:35.453515+0000","last_clean":"2024-09-20T22:15:35.453515+0000","last_became_active":"2024-09-20T22:12:52.518529+0000","last_became_peered":"2024-09-20T22:12:52.518529+0000","last_unstale":"2024-09-20T22:15:35.453515+0000","last_undegraded":"2024-09-20T22:15:35.453515+0000","last_fullsized":"2024-09-20T22:15:35.453515+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:55.614504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:55.614504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:18.700502+0000","scrub_duration":40,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"380'2629","reported_seq":2659,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:37.918919+0000","last_change":"2024-09-20T22:15:08.927958+0000","last_active":"2024-09-20T22:15:37.918919+0000","last_peered":"2024-09-20T22:15:37.918919+0000","last_clean":"2024-09-20T22:15:37.918919+0000","last_became_active":"2024-09-20T22:15:08.927661+0000","last_became_peered":"2024-09-20T22:15:08.927661+0000","last_unstale":"2024-09-20T22:15:37.918919+0000","last_undegraded":"2024-09-20T22:15:37.918919+0000","last_fullsized":"2024-09-20T22:15:37.918919+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2629,"log_dups_size":0,"ondisk_log_size":2629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:29.356464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":172294144,"num_objects":2629,"num_object_clones":0,"num_object_copies":7887,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2629,"num_write_kb":168256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":715,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.447565+0000","last_change":"2024-09-20T22:15:17.487480+0000","last_active":"2024-09-20T22:15:35.447565+0000","last_peered":"2024-09-20T22:15:35.447565+0000","last_clean":"2024-09-20T22:15:35.447565+0000","last_became_active":"2024-09-20T22:12:55.523589+0000","last_became_peered":"2024-09-20T22:12:55.523589+0000","last_unstale":"2024-09-20T22:15:35.447565+0000","last_undegraded":"2024-09-20T22:15:35.447565+0000","last_fullsized":"2024-09-20T22:15:35.447565+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:17.487408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:17.487408+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:28.563004+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"328'29","reported_seq":165,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.449361+0000","last_change":"2024-09-20T22:14:15.427866+0000","last_active":"2024-09-20T22:15:35.449361+0000","last_peered":"2024-09-20T22:15:35.449361+0000","last_clean":"2024-09-20T22:15:35.449361+0000","last_became_active":"2024-09-20T22:12:52.525538+0000","last_became_peered":"2024-09-20T22:12:52.525538+0000","last_unstale":"2024-09-20T22:15:35.449361+0000","last_undegraded":"2024-09-20T22:15:35.449361+0000","last_fullsized":"2024-09-20T22:15:35.449361+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'29","last_scrub_stamp":"2024-09-20T22:14:15.427533+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:15.427533+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.217474+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":527,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560143+0000","last_change":"2024-09-20T22:13:13.091930+0000","last_active":"2024-09-20T22:15:23.560143+0000","last_peered":"2024-09-20T22:15:23.560143+0000","last_clean":"2024-09-20T22:15:23.560143+0000","last_became_active":"2024-09-20T22:11:30.486428+0000","last_became_peered":"2024-09-20T22:11:30.486428+0000","last_unstale":"2024-09-20T22:15:23.560143+0000","last_undegraded":"2024-09-20T22:15:23.560143+0000","last_fullsized":"2024-09-20T22:15:23.560143+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:13.091819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:13.091819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":708,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.447458+0000","last_change":"2024-09-20T22:14:22.500268+0000","last_active":"2024-09-20T22:15:35.447458+0000","last_peered":"2024-09-20T22:15:35.447458+0000","last_clean":"2024-09-20T22:15:35.447458+0000","last_became_active":"2024-09-20T22:12:52.519826+0000","last_became_peered":"2024-09-20T22:12:52.519826+0000","last_unstale":"2024-09-20T22:15:35.447458+0000","last_undegraded":"2024-09-20T22:15:35.447458+0000","last_fullsized":"2024-09-20T22:15:35.447458+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:22.500119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:14:22.500119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:41.298289+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":521,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559302+0000","last_change":"2024-09-20T22:15:21.489682+0000","last_active":"2024-09-20T22:15:23.559302+0000","last_peered":"2024-09-20T22:15:23.559302+0000","last_clean":"2024-09-20T22:15:23.559302+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:15:23.559302+0000","last_undegraded":"2024-09-20T22:15:23.559302+0000","last_fullsized":"2024-09-20T22:15:23.559302+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:21.489619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:21.489619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:22.282365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"380'2730","reported_seq":2760,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:37.889018+0000","last_change":"2024-09-20T22:15:08.926557+0000","last_active":"2024-09-20T22:15:37.889018+0000","last_peered":"2024-09-20T22:15:37.889018+0000","last_clean":"2024-09-20T22:15:37.889018+0000","last_became_active":"2024-09-20T22:15:08.926245+0000","last_became_peered":"2024-09-20T22:15:08.926245+0000","last_unstale":"2024-09-20T22:15:37.889018+0000","last_undegraded":"2024-09-20T22:15:37.889018+0000","last_fullsized":"2024-09-20T22:15:37.889018+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2730,"log_dups_size":0,"ondisk_log_size":2730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:27.389588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":178913280,"num_objects":2730,"num_object_clones":0,"num_object_copies":8190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2730,"num_write_kb":174720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":702,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.450876+0000","last_change":"2024-09-20T22:12:55.522020+0000","last_active":"2024-09-20T22:15:35.450876+0000","last_peered":"2024-09-20T22:15:35.450876+0000","last_clean":"2024-09-20T22:15:35.450876+0000","last_became_active":"2024-09-20T22:12:55.521605+0000","last_became_peered":"2024-09-20T22:12:55.521605+0000","last_unstale":"2024-09-20T22:15:35.450876+0000","last_undegraded":"2024-09-20T22:15:35.450876+0000","last_fullsized":"2024-09-20T22:15:35.450876+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:12:16.121263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:12:16.121263+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"328'52","reported_seq":716,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:25.323106+0000","last_change":"2024-09-20T22:14:46.592348+0000","last_active":"2024-09-20T22:15:25.323106+0000","last_peered":"2024-09-20T22:15:25.323106+0000","last_clean":"2024-09-20T22:15:25.323106+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:15:25.323106+0000","last_undegraded":"2024-09-20T22:15:25.323106+0000","last_fullsized":"2024-09-20T22:15:25.323106+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'52","last_scrub_stamp":"2024-09-20T22:14:46.592203+0000","last_deep_scrub":"271'40","last_deep_scrub_stamp":"2024-09-20T22:11:05.235779+0000","last_clean_scrub_stamp":"2024-09-20T22:14:46.592203+0000","objects_scrubbed":10,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:01.016758+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":698,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.444237+0000","last_change":"2024-09-20T22:14:58.031902+0000","last_active":"2024-09-20T22:15:35.444237+0000","last_peered":"2024-09-20T22:15:35.444237+0000","last_clean":"2024-09-20T22:15:35.444237+0000","last_became_active":"2024-09-20T22:12:57.884257+0000","last_became_peered":"2024-09-20T22:12:57.884257+0000","last_unstale":"2024-09-20T22:15:35.444237+0000","last_undegraded":"2024-09-20T22:15:35.444237+0000","last_fullsized":"2024-09-20T22:15:35.444237+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:58.031824+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:58.031824+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:04.750930+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":700,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453351+0000","last_change":"2024-09-20T22:13:51.454966+0000","last_active":"2024-09-20T22:15:35.453351+0000","last_peered":"2024-09-20T22:15:35.453351+0000","last_clean":"2024-09-20T22:15:35.453351+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:15:35.453351+0000","last_undegraded":"2024-09-20T22:15:35.453351+0000","last_fullsized":"2024-09-20T22:15:35.453351+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.454850+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":132,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451838+0000","last_change":"2024-09-20T22:14:12.458599+0000","last_active":"2024-09-20T22:15:35.451838+0000","last_peered":"2024-09-20T22:15:35.451838+0000","last_clean":"2024-09-20T22:15:35.451838+0000","last_became_active":"2024-09-20T22:12:52.527547+0000","last_became_peered":"2024-09-20T22:12:52.527547+0000","last_unstale":"2024-09-20T22:15:35.451838+0000","last_undegraded":"2024-09-20T22:15:35.451838+0000","last_fullsized":"2024-09-20T22:15:35.451838+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:12.458492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:12.458492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:36.742957+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":675,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453726+0000","last_change":"2024-09-20T22:15:24.071705+0000","last_active":"2024-09-20T22:15:35.453726+0000","last_peered":"2024-09-20T22:15:35.453726+0000","last_clean":"2024-09-20T22:15:35.453726+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:15:35.453726+0000","last_undegraded":"2024-09-20T22:15:35.453726+0000","last_fullsized":"2024-09-20T22:15:35.453726+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.071593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.071593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:51.612695+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"380'2553","reported_seq":2583,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:36.154149+0000","last_change":"2024-09-20T22:15:08.931328+0000","last_active":"2024-09-20T22:15:36.154149+0000","last_peered":"2024-09-20T22:15:36.154149+0000","last_clean":"2024-09-20T22:15:36.154149+0000","last_became_active":"2024-09-20T22:15:08.930968+0000","last_became_peered":"2024-09-20T22:15:08.930968+0000","last_unstale":"2024-09-20T22:15:36.154149+0000","last_undegraded":"2024-09-20T22:15:36.154149+0000","last_fullsized":"2024-09-20T22:15:36.154149+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2553,"log_dups_size":0,"ondisk_log_size":2553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:14.066582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":167313408,"num_objects":2553,"num_object_clones":0,"num_object_copies":7659,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2553,"num_write_kb":163392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":699,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.450942+0000","last_change":"2024-09-20T22:15:00.407717+0000","last_active":"2024-09-20T22:15:35.450942+0000","last_peered":"2024-09-20T22:15:35.450942+0000","last_clean":"2024-09-20T22:15:35.450942+0000","last_became_active":"2024-09-20T22:12:55.518061+0000","last_became_peered":"2024-09-20T22:12:55.518061+0000","last_unstale":"2024-09-20T22:15:35.450942+0000","last_undegraded":"2024-09-20T22:15:35.450942+0000","last_fullsized":"2024-09-20T22:15:35.450942+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:00.407627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:00.407627+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:02.154276+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"328'17","reported_seq":594,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561702+0000","last_change":"2024-09-20T22:13:51.684806+0000","last_active":"2024-09-20T22:15:23.561702+0000","last_peered":"2024-09-20T22:15:23.561702+0000","last_clean":"2024-09-20T22:15:23.561702+0000","last_became_active":"2024-09-20T22:12:52.521073+0000","last_became_peered":"2024-09-20T22:12:52.521073+0000","last_unstale":"2024-09-20T22:15:23.561702+0000","last_undegraded":"2024-09-20T22:15:23.561702+0000","last_fullsized":"2024-09-20T22:15:23.561702+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'17","last_scrub_stamp":"2024-09-20T22:13:51.684667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.684667+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":522,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559330+0000","last_change":"2024-09-20T22:14:49.575469+0000","last_active":"2024-09-20T22:15:23.559330+0000","last_peered":"2024-09-20T22:15:23.559330+0000","last_clean":"2024-09-20T22:15:23.559330+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:15:23.559330+0000","last_undegraded":"2024-09-20T22:15:23.559330+0000","last_fullsized":"2024-09-20T22:15:23.559330+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:49.575363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:49.575363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:08.882258+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":742,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.457263+0000","last_change":"2024-09-20T22:14:18.282947+0000","last_active":"2024-09-20T22:15:35.457263+0000","last_peered":"2024-09-20T22:15:35.457263+0000","last_clean":"2024-09-20T22:15:35.457263+0000","last_became_active":"2024-09-20T22:12:52.517255+0000","last_became_peered":"2024-09-20T22:12:52.517255+0000","last_unstale":"2024-09-20T22:15:35.457263+0000","last_undegraded":"2024-09-20T22:15:35.457263+0000","last_fullsized":"2024-09-20T22:15:35.457263+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:14:18.282814+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:14:18.282814+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:39.504754+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":703,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.458198+0000","last_change":"2024-09-20T22:15:24.892562+0000","last_active":"2024-09-20T22:15:35.458198+0000","last_peered":"2024-09-20T22:15:35.458198+0000","last_clean":"2024-09-20T22:15:35.458198+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:15:35.458198+0000","last_undegraded":"2024-09-20T22:15:35.458198+0000","last_fullsized":"2024-09-20T22:15:35.458198+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.891955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.891955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:47.230621+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":669,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453699+0000","last_change":"2024-09-20T22:13:41.730912+0000","last_active":"2024-09-20T22:15:35.453699+0000","last_peered":"2024-09-20T22:15:35.453699+0000","last_clean":"2024-09-20T22:15:35.453699+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:15:35.453699+0000","last_undegraded":"2024-09-20T22:15:35.453699+0000","last_fullsized":"2024-09-20T22:15:35.453699+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:41.730776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:41.730776+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.321175+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"380'2601","reported_seq":2631,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:36.980533+0000","last_change":"2024-09-20T22:15:08.927019+0000","last_active":"2024-09-20T22:15:36.980533+0000","last_peered":"2024-09-20T22:15:36.980533+0000","last_clean":"2024-09-20T22:15:36.980533+0000","last_became_active":"2024-09-20T22:15:08.926702+0000","last_became_peered":"2024-09-20T22:15:08.926702+0000","last_unstale":"2024-09-20T22:15:36.980533+0000","last_undegraded":"2024-09-20T22:15:36.980533+0000","last_fullsized":"2024-09-20T22:15:36.980533+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2601,"log_dups_size":0,"ondisk_log_size":2601,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:17.801635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170459136,"num_objects":2601,"num_object_clones":0,"num_object_copies":7803,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2601,"num_write_kb":166464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":679,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559344+0000","last_change":"2024-09-20T22:13:48.490583+0000","last_active":"2024-09-20T22:15:23.559344+0000","last_peered":"2024-09-20T22:15:23.559344+0000","last_clean":"2024-09-20T22:15:23.559344+0000","last_became_active":"2024-09-20T22:08:48.446840+0000","last_became_peered":"2024-09-20T22:08:48.446840+0000","last_unstale":"2024-09-20T22:15:23.559344+0000","last_undegraded":"2024-09-20T22:15:23.559344+0000","last_fullsized":"2024-09-20T22:15:23.559344+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:13:48.490464+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"328'24","reported_seq":172,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.448982+0000","last_change":"2024-09-20T22:15:26.742439+0000","last_active":"2024-09-20T22:15:35.448982+0000","last_peered":"2024-09-20T22:15:35.448982+0000","last_clean":"2024-09-20T22:15:35.448982+0000","last_became_active":"2024-09-20T22:12:52.527919+0000","last_became_peered":"2024-09-20T22:12:52.527919+0000","last_unstale":"2024-09-20T22:15:35.448982+0000","last_undegraded":"2024-09-20T22:15:35.448982+0000","last_fullsized":"2024-09-20T22:15:35.448982+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'24","last_scrub_stamp":"2024-09-20T22:15:26.742317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:26.742317+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:32.350037+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":140,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.450042+0000","last_change":"2024-09-20T22:15:21.774404+0000","last_active":"2024-09-20T22:15:35.450042+0000","last_peered":"2024-09-20T22:15:35.450042+0000","last_clean":"2024-09-20T22:15:35.450042+0000","last_became_active":"2024-09-20T22:12:52.517432+0000","last_became_peered":"2024-09-20T22:12:52.517432+0000","last_unstale":"2024-09-20T22:15:35.450042+0000","last_undegraded":"2024-09-20T22:15:35.450042+0000","last_fullsized":"2024-09-20T22:15:35.450042+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:21.774300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:15:21.774300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:29.628508+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":1470,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451691+0000","last_change":"2024-09-20T22:13:06.609600+0000","last_active":"2024-09-20T22:15:35.451691+0000","last_peered":"2024-09-20T22:15:35.451691+0000","last_clean":"2024-09-20T22:15:35.451691+0000","last_became_active":"2024-09-20T22:12:52.526622+0000","last_became_peered":"2024-09-20T22:12:52.526622+0000","last_unstale":"2024-09-20T22:15:35.451691+0000","last_undegraded":"2024-09-20T22:15:35.451691+0000","last_fullsized":"2024-09-20T22:15:35.451691+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:13:06.609525+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:13:06.609525+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.321027+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":742,"num_read_kb":737,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":234,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.452730+0000","last_change":"2024-09-20T22:14:14.443714+0000","last_active":"2024-09-20T22:15:35.452730+0000","last_peered":"2024-09-20T22:15:35.452730+0000","last_clean":"2024-09-20T22:15:35.452730+0000","last_became_active":"2024-09-20T22:12:52.518823+0000","last_became_peered":"2024-09-20T22:12:52.518823+0000","last_unstale":"2024-09-20T22:15:35.452730+0000","last_undegraded":"2024-09-20T22:15:35.452730+0000","last_fullsized":"2024-09-20T22:15:35.452730+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:14.443611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:14.443611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":514,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559961+0000","last_change":"2024-09-20T22:13:40.433856+0000","last_active":"2024-09-20T22:15:23.559961+0000","last_peered":"2024-09-20T22:15:23.559961+0000","last_clean":"2024-09-20T22:15:23.559961+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:15:23.559961+0000","last_undegraded":"2024-09-20T22:15:23.559961+0000","last_fullsized":"2024-09-20T22:15:23.559961+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:40.433606+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.415468+0000","last_clean_scrub_stamp":"2024-09-20T22:13:40.433606+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"380'2735","reported_seq":2763,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:37.926648+0000","last_change":"2024-09-20T22:15:08.925131+0000","last_active":"2024-09-20T22:15:37.926648+0000","last_peered":"2024-09-20T22:15:37.926648+0000","last_clean":"2024-09-20T22:15:37.926648+0000","last_became_active":"2024-09-20T22:15:08.924809+0000","last_became_peered":"2024-09-20T22:15:08.924809+0000","last_unstale":"2024-09-20T22:15:37.926648+0000","last_undegraded":"2024-09-20T22:15:37.926648+0000","last_fullsized":"2024-09-20T22:15:37.926648+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2735,"log_dups_size":0,"ondisk_log_size":2735,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:36.576727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":179240960,"num_objects":2735,"num_object_clones":0,"num_object_copies":8205,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2735,"num_write_kb":175040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"328'28","reported_seq":886,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.447418+0000","last_change":"2024-09-20T22:14:21.445235+0000","last_active":"2024-09-20T22:15:35.447418+0000","last_peered":"2024-09-20T22:15:35.447418+0000","last_clean":"2024-09-20T22:15:35.447418+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:15:35.447418+0000","last_undegraded":"2024-09-20T22:15:35.447418+0000","last_fullsized":"2024-09-20T22:15:35.447418+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:14:21.445103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:21.445103+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":640,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451818+0000","last_change":"2024-09-20T22:12:42.260316+0000","last_active":"2024-09-20T22:15:35.451818+0000","last_peered":"2024-09-20T22:15:35.451818+0000","last_clean":"2024-09-20T22:15:35.451818+0000","last_became_active":"2024-09-20T22:12:21.519717+0000","last_became_peered":"2024-09-20T22:12:21.519717+0000","last_unstale":"2024-09-20T22:15:35.451818+0000","last_undegraded":"2024-09-20T22:15:35.451818+0000","last_fullsized":"2024-09-20T22:15:35.451818+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:42.260197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:42.260197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":699,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451175+0000","last_change":"2024-09-20T22:15:25.365208+0000","last_active":"2024-09-20T22:15:35.451175+0000","last_peered":"2024-09-20T22:15:35.451175+0000","last_clean":"2024-09-20T22:15:35.451175+0000","last_became_active":"2024-09-20T22:12:55.522325+0000","last_became_peered":"2024-09-20T22:12:55.522325+0000","last_unstale":"2024-09-20T22:15:35.451175+0000","last_undegraded":"2024-09-20T22:15:35.451175+0000","last_fullsized":"2024-09-20T22:15:35.451175+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:25.365122+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:25.365122+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:41.815575+0000","scrub_duration":34,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":263,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.450964+0000","last_change":"2024-09-20T22:13:51.675128+0000","last_active":"2024-09-20T22:15:35.450964+0000","last_peered":"2024-09-20T22:15:35.450964+0000","last_clean":"2024-09-20T22:15:35.450964+0000","last_became_active":"2024-09-20T22:10:48.907271+0000","last_became_peered":"2024-09-20T22:10:48.907271+0000","last_unstale":"2024-09-20T22:15:35.450964+0000","last_undegraded":"2024-09-20T22:15:35.450964+0000","last_fullsized":"2024-09-20T22:15:35.450964+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.674976+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.321083+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":695,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.452434+0000","last_change":"2024-09-20T22:13:02.482655+0000","last_active":"2024-09-20T22:15:35.452434+0000","last_peered":"2024-09-20T22:15:35.452434+0000","last_clean":"2024-09-20T22:15:35.452434+0000","last_became_active":"2024-09-20T22:12:44.630044+0000","last_became_peered":"2024-09-20T22:12:44.630044+0000","last_unstale":"2024-09-20T22:15:35.452434+0000","last_undegraded":"2024-09-20T22:15:35.452434+0000","last_fullsized":"2024-09-20T22:15:35.452434+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:02.482546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:13:02.482546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.0","version":"380'2797","reported_seq":2827,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:37.900910+0000","last_change":"2024-09-20T22:15:08.925316+0000","last_active":"2024-09-20T22:15:37.900910+0000","last_peered":"2024-09-20T22:15:37.900910+0000","last_clean":"2024-09-20T22:15:37.900910+0000","last_became_active":"2024-09-20T22:15:08.925040+0000","last_became_peered":"2024-09-20T22:15:08.925040+0000","last_unstale":"2024-09-20T22:15:37.900910+0000","last_undegraded":"2024-09-20T22:15:37.900910+0000","last_fullsized":"2024-09-20T22:15:37.900910+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2797,"log_dups_size":0,"ondisk_log_size":2797,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:30.887426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":183304192,"num_objects":2797,"num_object_clones":0,"num_object_copies":8391,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2797,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2797,"num_write_kb":179008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"328'19","reported_seq":716,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.443886+0000","last_change":"2024-09-20T22:15:19.012731+0000","last_active":"2024-09-20T22:15:35.443886+0000","last_peered":"2024-09-20T22:15:35.443886+0000","last_clean":"2024-09-20T22:15:35.443886+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:15:35.443886+0000","last_undegraded":"2024-09-20T22:15:35.443886+0000","last_fullsized":"2024-09-20T22:15:35.443886+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:15:19.012609+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:19.012609+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:20.138304+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":688,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.443034+0000","last_change":"2024-09-20T22:15:27.077414+0000","last_active":"2024-09-20T22:15:35.443034+0000","last_peered":"2024-09-20T22:15:35.443034+0000","last_clean":"2024-09-20T22:15:35.443034+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:15:35.443034+0000","last_undegraded":"2024-09-20T22:15:35.443034+0000","last_fullsized":"2024-09-20T22:15:35.443034+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:27.077295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.077295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:47.688033+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":687,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.452262+0000","last_change":"2024-09-20T22:15:24.150401+0000","last_active":"2024-09-20T22:15:35.452262+0000","last_peered":"2024-09-20T22:15:35.452262+0000","last_clean":"2024-09-20T22:15:35.452262+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:15:35.452262+0000","last_undegraded":"2024-09-20T22:15:35.452262+0000","last_fullsized":"2024-09-20T22:15:35.452262+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.150265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.150265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:32.448596+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":635,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.447170+0000","last_change":"2024-09-20T22:14:08.297827+0000","last_active":"2024-09-20T22:15:35.447170+0000","last_peered":"2024-09-20T22:15:35.447170+0000","last_clean":"2024-09-20T22:15:35.447170+0000","last_became_active":"2024-09-20T22:12:21.521195+0000","last_became_peered":"2024-09-20T22:12:21.521195+0000","last_unstale":"2024-09-20T22:15:35.447170+0000","last_undegraded":"2024-09-20T22:15:35.447170+0000","last_fullsized":"2024-09-20T22:15:35.447170+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:08.297698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:08.297698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3","version":"380'2575","reported_seq":2605,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:36.229176+0000","last_change":"2024-09-20T22:15:09.087357+0000","last_active":"2024-09-20T22:15:36.229176+0000","last_peered":"2024-09-20T22:15:36.229176+0000","last_clean":"2024-09-20T22:15:36.229176+0000","last_became_active":"2024-09-20T22:15:09.087061+0000","last_became_peered":"2024-09-20T22:15:09.087061+0000","last_unstale":"2024-09-20T22:15:36.229176+0000","last_undegraded":"2024-09-20T22:15:36.229176+0000","last_fullsized":"2024-09-20T22:15:36.229176+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2575,"log_dups_size":0,"ondisk_log_size":2575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:23.286140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":168755200,"num_objects":2575,"num_object_clones":0,"num_object_copies":7725,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2575,"num_write_kb":164800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":450,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.446636+0000","last_change":"2024-09-20T22:14:46.480442+0000","last_active":"2024-09-20T22:15:35.446636+0000","last_peered":"2024-09-20T22:15:35.446636+0000","last_clean":"2024-09-20T22:15:35.446636+0000","last_became_active":"2024-09-20T22:14:38.674758+0000","last_became_peered":"2024-09-20T22:14:38.674758+0000","last_unstale":"2024-09-20T22:15:35.446636+0000","last_undegraded":"2024-09-20T22:15:35.446636+0000","last_fullsized":"2024-09-20T22:15:35.446636+0000","mapping_epoch":358,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":359,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:14:46.480331+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:14:46.480331+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:06.604863+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"328'31","reported_seq":742,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.443709+0000","last_change":"2024-09-20T22:14:46.989102+0000","last_active":"2024-09-20T22:15:35.443709+0000","last_peered":"2024-09-20T22:15:35.443709+0000","last_clean":"2024-09-20T22:15:35.443709+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:15:35.443709+0000","last_undegraded":"2024-09-20T22:15:35.443709+0000","last_fullsized":"2024-09-20T22:15:35.443709+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'31","last_scrub_stamp":"2024-09-20T22:14:46.988744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:46.988744+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:58.605093+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":704,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.447724+0000","last_change":"2024-09-20T22:15:32.512124+0000","last_active":"2024-09-20T22:15:35.447724+0000","last_peered":"2024-09-20T22:15:35.447724+0000","last_clean":"2024-09-20T22:15:35.447724+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:15:35.447724+0000","last_undegraded":"2024-09-20T22:15:35.447724+0000","last_fullsized":"2024-09-20T22:15:35.447724+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:32.506512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:32.506512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:01.564420+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":525,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560054+0000","last_change":"2024-09-20T22:13:10.039183+0000","last_active":"2024-09-20T22:15:23.560054+0000","last_peered":"2024-09-20T22:15:23.560054+0000","last_clean":"2024-09-20T22:15:23.560054+0000","last_became_active":"2024-09-20T22:12:52.520612+0000","last_became_peered":"2024-09-20T22:12:52.520612+0000","last_unstale":"2024-09-20T22:15:23.560054+0000","last_undegraded":"2024-09-20T22:15:23.560054+0000","last_fullsized":"2024-09-20T22:15:23.560054+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:10.039074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.039074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":691,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.458541+0000","last_change":"2024-09-20T22:14:50.010648+0000","last_active":"2024-09-20T22:15:35.458541+0000","last_peered":"2024-09-20T22:15:35.458541+0000","last_clean":"2024-09-20T22:15:35.458541+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:15:35.458541+0000","last_undegraded":"2024-09-20T22:15:35.458541+0000","last_fullsized":"2024-09-20T22:15:35.458541+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:50.010536+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:50.010536+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:56.117583+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"378'2252","reported_seq":2278,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:33.253827+0000","last_change":"2024-09-20T22:15:08.926429+0000","last_active":"2024-09-20T22:15:33.253827+0000","last_peered":"2024-09-20T22:15:33.253827+0000","last_clean":"2024-09-20T22:15:33.253827+0000","last_became_active":"2024-09-20T22:15:08.926083+0000","last_became_peered":"2024-09-20T22:15:08.926083+0000","last_unstale":"2024-09-20T22:15:33.253827+0000","last_undegraded":"2024-09-20T22:15:33.253827+0000","last_fullsized":"2024-09-20T22:15:33.253827+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2252,"log_dups_size":0,"ondisk_log_size":2252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:12.245813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147587072,"num_objects":2252,"num_object_clones":0,"num_object_copies":6756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2252,"num_write_kb":144128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":961,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451328+0000","last_change":"2024-09-20T22:13:49.912616+0000","last_active":"2024-09-20T22:15:35.451328+0000","last_peered":"2024-09-20T22:15:35.451328+0000","last_clean":"2024-09-20T22:15:35.451328+0000","last_became_active":"2024-09-20T22:08:48.455288+0000","last_became_peered":"2024-09-20T22:08:48.455288+0000","last_unstale":"2024-09-20T22:15:35.451328+0000","last_undegraded":"2024-09-20T22:15:35.451328+0000","last_fullsized":"2024-09-20T22:15:35.451328+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:49.912496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:13:49.912496+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.321205+0000","scrub_duration":227,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"328'25","reported_seq":711,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.443178+0000","last_change":"2024-09-20T22:14:10.070008+0000","last_active":"2024-09-20T22:15:35.443178+0000","last_peered":"2024-09-20T22:15:35.443178+0000","last_clean":"2024-09-20T22:15:35.443178+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:15:35.443178+0000","last_undegraded":"2024-09-20T22:15:35.443178+0000","last_fullsized":"2024-09-20T22:15:35.443178+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'25","last_scrub_stamp":"2024-09-20T22:14:10.069812+0000","last_deep_scrub":"289'15","last_deep_scrub_stamp":"2024-09-20T22:11:37.277648+0000","last_clean_scrub_stamp":"2024-09-20T22:14:10.069812+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":144,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.448703+0000","last_change":"2024-09-20T22:15:30.821251+0000","last_active":"2024-09-20T22:15:35.448703+0000","last_peered":"2024-09-20T22:15:35.448703+0000","last_clean":"2024-09-20T22:15:35.448703+0000","last_became_active":"2024-09-20T22:12:52.528310+0000","last_became_peered":"2024-09-20T22:12:52.528310+0000","last_unstale":"2024-09-20T22:15:35.448703+0000","last_undegraded":"2024-09-20T22:15:35.448703+0000","last_fullsized":"2024-09-20T22:15:35.448703+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:15:30.821142+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:15:30.821142+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:55.515561+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"283'11","reported_seq":886,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.457816+0000","last_change":"2024-09-20T22:14:20.265212+0000","last_active":"2024-09-20T22:15:35.457816+0000","last_peered":"2024-09-20T22:15:35.457816+0000","last_clean":"2024-09-20T22:15:35.457816+0000","last_became_active":"2024-09-20T22:11:30.485356+0000","last_became_peered":"2024-09-20T22:11:30.485356+0000","last_unstale":"2024-09-20T22:15:35.457816+0000","last_undegraded":"2024-09-20T22:15:35.457816+0000","last_fullsized":"2024-09-20T22:15:35.457816+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'11","last_scrub_stamp":"2024-09-20T22:14:20.265107+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:14:20.265107+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:43.795761+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":677,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.443686+0000","last_change":"2024-09-20T22:14:51.059172+0000","last_active":"2024-09-20T22:15:35.443686+0000","last_peered":"2024-09-20T22:15:35.443686+0000","last_clean":"2024-09-20T22:15:35.443686+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:15:35.443686+0000","last_undegraded":"2024-09-20T22:15:35.443686+0000","last_fullsized":"2024-09-20T22:15:35.443686+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:51.059066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:14:51.059066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:20.407644+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"380'2477","reported_seq":2507,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:36.247382+0000","last_change":"2024-09-20T22:15:09.088109+0000","last_active":"2024-09-20T22:15:36.247382+0000","last_peered":"2024-09-20T22:15:36.247382+0000","last_clean":"2024-09-20T22:15:36.247382+0000","last_became_active":"2024-09-20T22:15:09.087830+0000","last_became_peered":"2024-09-20T22:15:09.087830+0000","last_unstale":"2024-09-20T22:15:36.247382+0000","last_undegraded":"2024-09-20T22:15:36.247382+0000","last_fullsized":"2024-09-20T22:15:36.247382+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2477,"log_dups_size":0,"ondisk_log_size":2477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:10.163377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":162332672,"num_objects":2477,"num_object_clones":0,"num_object_copies":7431,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2477,"num_write_kb":158528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"328'15","reported_seq":159,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.449462+0000","last_change":"2024-09-20T22:15:25.742250+0000","last_active":"2024-09-20T22:15:35.449462+0000","last_peered":"2024-09-20T22:15:35.449462+0000","last_clean":"2024-09-20T22:15:35.449462+0000","last_became_active":"2024-09-20T22:12:52.527531+0000","last_became_peered":"2024-09-20T22:12:52.527531+0000","last_unstale":"2024-09-20T22:15:35.449462+0000","last_undegraded":"2024-09-20T22:15:35.449462+0000","last_fullsized":"2024-09-20T22:15:35.449462+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:15:25.741631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:25.741631+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:28.256179+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":316,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453769+0000","last_change":"2024-09-20T22:14:55.145177+0000","last_active":"2024-09-20T22:15:35.453769+0000","last_peered":"2024-09-20T22:15:35.453769+0000","last_clean":"2024-09-20T22:15:35.453769+0000","last_became_active":"2024-09-20T22:10:14.536155+0000","last_became_peered":"2024-09-20T22:10:14.536155+0000","last_unstale":"2024-09-20T22:15:35.453769+0000","last_undegraded":"2024-09-20T22:15:35.453769+0000","last_fullsized":"2024-09-20T22:15:35.453769+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:14:55.144682+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:55.144682+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:11.541071+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":703,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.456915+0000","last_change":"2024-09-20T22:15:18.919125+0000","last_active":"2024-09-20T22:15:35.456915+0000","last_peered":"2024-09-20T22:15:35.456915+0000","last_clean":"2024-09-20T22:15:35.456915+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:15:35.456915+0000","last_undegraded":"2024-09-20T22:15:35.456915+0000","last_fullsized":"2024-09-20T22:15:35.456915+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:18.919024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:15:18.919024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:26.163742+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":132,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.450127+0000","last_change":"2024-09-20T22:14:45.679385+0000","last_active":"2024-09-20T22:15:35.450127+0000","last_peered":"2024-09-20T22:15:35.450127+0000","last_clean":"2024-09-20T22:15:35.450127+0000","last_became_active":"2024-09-20T22:12:52.512924+0000","last_became_peered":"2024-09-20T22:12:52.512924+0000","last_unstale":"2024-09-20T22:15:35.450127+0000","last_undegraded":"2024-09-20T22:15:35.450127+0000","last_fullsized":"2024-09-20T22:15:35.450127+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:45.679291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:45.679291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:49.553806+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"380'2603","reported_seq":2633,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:36.964967+0000","last_change":"2024-09-20T22:15:09.087623+0000","last_active":"2024-09-20T22:15:36.964967+0000","last_peered":"2024-09-20T22:15:36.964967+0000","last_clean":"2024-09-20T22:15:36.964967+0000","last_became_active":"2024-09-20T22:15:09.087298+0000","last_became_peered":"2024-09-20T22:15:09.087298+0000","last_unstale":"2024-09-20T22:15:36.964967+0000","last_undegraded":"2024-09-20T22:15:36.964967+0000","last_fullsized":"2024-09-20T22:15:36.964967+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2603,"log_dups_size":0,"ondisk_log_size":2603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:13.382167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170590208,"num_objects":2603,"num_object_clones":0,"num_object_copies":7809,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2603,"num_write_kb":166592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"328'35","reported_seq":799,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.447502+0000","last_change":"2024-09-20T22:15:31.492645+0000","last_active":"2024-09-20T22:15:35.447502+0000","last_peered":"2024-09-20T22:15:35.447502+0000","last_clean":"2024-09-20T22:15:35.447502+0000","last_became_active":"2024-09-20T22:10:30.652967+0000","last_became_peered":"2024-09-20T22:10:30.652967+0000","last_unstale":"2024-09-20T22:15:35.447502+0000","last_undegraded":"2024-09-20T22:15:35.447502+0000","last_fullsized":"2024-09-20T22:15:35.447502+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'35","last_scrub_stamp":"2024-09-20T22:15:31.492433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:31.492433+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:55.689722+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":700,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453846+0000","last_change":"2024-09-20T22:15:23.351913+0000","last_active":"2024-09-20T22:15:35.453846+0000","last_peered":"2024-09-20T22:15:35.453846+0000","last_clean":"2024-09-20T22:15:35.453846+0000","last_became_active":"2024-09-20T22:11:30.488650+0000","last_became_peered":"2024-09-20T22:11:30.488650+0000","last_unstale":"2024-09-20T22:15:35.453846+0000","last_undegraded":"2024-09-20T22:15:35.453846+0000","last_fullsized":"2024-09-20T22:15:35.453846+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:23.351789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:12:46.602035+0000","last_clean_scrub_stamp":"2024-09-20T22:15:23.351789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:36.963085+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":727,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.457067+0000","last_change":"2024-09-20T22:14:51.988808+0000","last_active":"2024-09-20T22:15:35.457067+0000","last_peered":"2024-09-20T22:15:35.457067+0000","last_clean":"2024-09-20T22:15:35.457067+0000","last_became_active":"2024-09-20T22:12:52.515295+0000","last_became_peered":"2024-09-20T22:12:52.515295+0000","last_unstale":"2024-09-20T22:15:35.457067+0000","last_undegraded":"2024-09-20T22:15:35.457067+0000","last_fullsized":"2024-09-20T22:15:35.457067+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:51.988666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_clean_scrub_stamp":"2024-09-20T22:14:51.988666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:00.556168+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":285,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451230+0000","last_change":"2024-09-20T22:14:47.271963+0000","last_active":"2024-09-20T22:15:35.451230+0000","last_peered":"2024-09-20T22:15:35.451230+0000","last_clean":"2024-09-20T22:15:35.451230+0000","last_became_active":"2024-09-20T22:10:30.657116+0000","last_became_peered":"2024-09-20T22:10:30.657116+0000","last_unstale":"2024-09-20T22:15:35.451230+0000","last_undegraded":"2024-09-20T22:15:35.451230+0000","last_fullsized":"2024-09-20T22:15:35.451230+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:47.271872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:47.271872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:07.707877+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.7","version":"380'2687","reported_seq":2717,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:37.914867+0000","last_change":"2024-09-20T22:15:09.089072+0000","last_active":"2024-09-20T22:15:37.914867+0000","last_peered":"2024-09-20T22:15:37.914867+0000","last_clean":"2024-09-20T22:15:37.914867+0000","last_became_active":"2024-09-20T22:15:09.088590+0000","last_became_peered":"2024-09-20T22:15:09.088590+0000","last_unstale":"2024-09-20T22:15:37.914867+0000","last_undegraded":"2024-09-20T22:15:37.914867+0000","last_fullsized":"2024-09-20T22:15:37.914867+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2687,"log_dups_size":0,"ondisk_log_size":2687,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:25.980411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":176095232,"num_objects":2687,"num_object_clones":0,"num_object_copies":8061,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2687,"num_write_kb":171968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"328'18","reported_seq":717,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.442779+0000","last_change":"2024-09-20T22:15:22.019818+0000","last_active":"2024-09-20T22:15:35.442779+0000","last_peered":"2024-09-20T22:15:35.442779+0000","last_clean":"2024-09-20T22:15:35.442779+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:15:35.442779+0000","last_undegraded":"2024-09-20T22:15:35.442779+0000","last_fullsized":"2024-09-20T22:15:35.442779+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'18","last_scrub_stamp":"2024-09-20T22:15:22.019700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:22.019700+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:42.439452+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":689,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453948+0000","last_change":"2024-09-20T22:14:15.965211+0000","last_active":"2024-09-20T22:15:35.453948+0000","last_peered":"2024-09-20T22:15:35.453948+0000","last_clean":"2024-09-20T22:15:35.453948+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:15:35.453948+0000","last_undegraded":"2024-09-20T22:15:35.453948+0000","last_fullsized":"2024-09-20T22:15:35.453948+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:15.965013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:15.965013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.321231+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":724,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.456759+0000","last_change":"2024-09-20T22:15:27.924542+0000","last_active":"2024-09-20T22:15:35.456759+0000","last_peered":"2024-09-20T22:15:35.456759+0000","last_clean":"2024-09-20T22:15:35.456759+0000","last_became_active":"2024-09-20T22:12:52.514202+0000","last_became_peered":"2024-09-20T22:12:52.514202+0000","last_unstale":"2024-09-20T22:15:35.456759+0000","last_undegraded":"2024-09-20T22:15:35.456759+0000","last_fullsized":"2024-09-20T22:15:35.456759+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:27.924150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.924150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:41.071832+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":150,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451130+0000","last_change":"2024-09-20T22:15:26.021754+0000","last_active":"2024-09-20T22:15:35.451130+0000","last_peered":"2024-09-20T22:15:35.451130+0000","last_clean":"2024-09-20T22:15:35.451130+0000","last_became_active":"2024-09-20T22:12:44.638546+0000","last_became_peered":"2024-09-20T22:12:44.638546+0000","last_unstale":"2024-09-20T22:15:35.451130+0000","last_undegraded":"2024-09-20T22:15:35.451130+0000","last_fullsized":"2024-09-20T22:15:35.451130+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:26.021636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:26.021636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:46.884542+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.6","version":"380'2466","reported_seq":2496,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:36.177111+0000","last_change":"2024-09-20T22:15:08.928417+0000","last_active":"2024-09-20T22:15:36.177111+0000","last_peered":"2024-09-20T22:15:36.177111+0000","last_clean":"2024-09-20T22:15:36.177111+0000","last_became_active":"2024-09-20T22:15:08.928102+0000","last_became_peered":"2024-09-20T22:15:08.928102+0000","last_unstale":"2024-09-20T22:15:36.177111+0000","last_undegraded":"2024-09-20T22:15:36.177111+0000","last_fullsized":"2024-09-20T22:15:36.177111+0000","mapping_epoch":370,"log_start":"0'0","ondisk_log_start":"0'0","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":2466,"log_dups_size":0,"ondisk_log_size":2466,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:10.761616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":161611776,"num_objects":2466,"num_object_clones":0,"num_object_copies":7398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2466,"num_write_kb":157824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"328'27","reported_seq":248,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.450606+0000","last_change":"2024-09-20T22:13:10.353607+0000","last_active":"2024-09-20T22:15:35.450606+0000","last_peered":"2024-09-20T22:15:35.450606+0000","last_clean":"2024-09-20T22:15:35.450606+0000","last_became_active":"2024-09-20T22:11:30.493275+0000","last_became_peered":"2024-09-20T22:11:30.493275+0000","last_unstale":"2024-09-20T22:15:35.450606+0000","last_undegraded":"2024-09-20T22:15:35.450606+0000","last_fullsized":"2024-09-20T22:15:35.450606+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:13:10.353459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.353459+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":315,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453568+0000","last_change":"2024-09-20T22:14:56.339026+0000","last_active":"2024-09-20T22:15:35.453568+0000","last_peered":"2024-09-20T22:15:35.453568+0000","last_clean":"2024-09-20T22:15:35.453568+0000","last_became_active":"2024-09-20T22:12:52.526551+0000","last_became_peered":"2024-09-20T22:12:52.526551+0000","last_unstale":"2024-09-20T22:15:35.453568+0000","last_undegraded":"2024-09-20T22:15:35.453568+0000","last_fullsized":"2024-09-20T22:15:35.453568+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:14:56.338941+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:14:56.338941+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:09.361665+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":834,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.450916+0000","last_change":"2024-09-20T22:14:12.911852+0000","last_active":"2024-09-20T22:15:35.450916+0000","last_peered":"2024-09-20T22:15:35.450916+0000","last_clean":"2024-09-20T22:15:35.450916+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:15:35.450916+0000","last_undegraded":"2024-09-20T22:15:35.450916+0000","last_fullsized":"2024-09-20T22:15:35.450916+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:14:12.911520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:12.911520+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.190103+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":226,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453298+0000","last_change":"2024-09-20T22:13:00.454973+0000","last_active":"2024-09-20T22:15:35.453298+0000","last_peered":"2024-09-20T22:15:35.453298+0000","last_clean":"2024-09-20T22:15:35.453298+0000","last_became_active":"2024-09-20T22:12:52.517758+0000","last_became_peered":"2024-09-20T22:12:52.517758+0000","last_unstale":"2024-09-20T22:15:35.453298+0000","last_undegraded":"2024-09-20T22:15:35.453298+0000","last_fullsized":"2024-09-20T22:15:35.453298+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:00.454378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:13:00.454378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":727,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.447129+0000","last_change":"2024-09-20T22:14:49.417575+0000","last_active":"2024-09-20T22:15:35.447129+0000","last_peered":"2024-09-20T22:15:35.447129+0000","last_clean":"2024-09-20T22:15:35.447129+0000","last_became_active":"2024-09-20T22:12:52.516989+0000","last_became_peered":"2024-09-20T22:12:52.516989+0000","last_unstale":"2024-09-20T22:15:35.447129+0000","last_undegraded":"2024-09-20T22:15:35.447129+0000","last_fullsized":"2024-09-20T22:15:35.447129+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:49.417468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:49.417468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:19.052286+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"328'12","reported_seq":729,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.454977+0000","last_change":"2024-09-20T22:14:58.150505+0000","last_active":"2024-09-20T22:15:35.454977+0000","last_peered":"2024-09-20T22:15:35.454977+0000","last_clean":"2024-09-20T22:15:35.454977+0000","last_became_active":"2024-09-20T22:10:48.922561+0000","last_became_peered":"2024-09-20T22:10:48.922561+0000","last_unstale":"2024-09-20T22:15:35.454977+0000","last_undegraded":"2024-09-20T22:15:35.454977+0000","last_fullsized":"2024-09-20T22:15:35.454977+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'12","last_scrub_stamp":"2024-09-20T22:14:58.150361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:58.150361+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:05.584490+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":214,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.452167+0000","last_change":"2024-09-20T22:12:14.148857+0000","last_active":"2024-09-20T22:15:35.452167+0000","last_peered":"2024-09-20T22:15:35.452167+0000","last_clean":"2024-09-20T22:15:35.452167+0000","last_became_active":"2024-09-20T22:11:30.487322+0000","last_became_peered":"2024-09-20T22:11:30.487322+0000","last_unstale":"2024-09-20T22:15:35.452167+0000","last_undegraded":"2024-09-20T22:15:35.452167+0000","last_fullsized":"2024-09-20T22:15:35.452167+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:14.148743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:12:14.148743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":668,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453625+0000","last_change":"2024-09-20T22:14:54.359687+0000","last_active":"2024-09-20T22:15:35.453625+0000","last_peered":"2024-09-20T22:15:35.453625+0000","last_clean":"2024-09-20T22:15:35.453625+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:15:35.453625+0000","last_undegraded":"2024-09-20T22:15:35.453625+0000","last_fullsized":"2024-09-20T22:15:35.453625+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:54.359586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:54.359586+0000","last_clean_scrub_stamp":"2024-09-20T22:14:54.359586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:11.201500+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"328'15","reported_seq":151,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.449578+0000","last_change":"2024-09-20T22:14:14.397650+0000","last_active":"2024-09-20T22:15:35.449578+0000","last_peered":"2024-09-20T22:15:35.449578+0000","last_clean":"2024-09-20T22:15:35.449578+0000","last_became_active":"2024-09-20T22:12:52.527106+0000","last_became_peered":"2024-09-20T22:12:52.527106+0000","last_unstale":"2024-09-20T22:15:35.449578+0000","last_undegraded":"2024-09-20T22:15:35.449578+0000","last_fullsized":"2024-09-20T22:15:35.449578+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:14:14.397530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:14.397530+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:42.176603+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":688,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451464+0000","last_change":"2024-09-20T22:15:28.374528+0000","last_active":"2024-09-20T22:15:35.451464+0000","last_peered":"2024-09-20T22:15:35.451464+0000","last_clean":"2024-09-20T22:15:35.451464+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:15:35.451464+0000","last_undegraded":"2024-09-20T22:15:35.451464+0000","last_fullsized":"2024-09-20T22:15:35.451464+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:28.374414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:28.374414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:30.191322+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":314,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.561351+0000","last_change":"2024-09-20T22:13:14.136545+0000","last_active":"2024-09-20T22:15:23.561351+0000","last_peered":"2024-09-20T22:15:23.561351+0000","last_clean":"2024-09-20T22:15:23.561351+0000","last_became_active":"2024-09-20T22:12:33.545506+0000","last_became_peered":"2024-09-20T22:12:33.545506+0000","last_unstale":"2024-09-20T22:15:23.561351+0000","last_undegraded":"2024-09-20T22:15:23.561351+0000","last_fullsized":"2024-09-20T22:15:23.561351+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:14.136472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_clean_scrub_stamp":"2024-09-20T22:13:14.136472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":699,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.457368+0000","last_change":"2024-09-20T22:15:00.011002+0000","last_active":"2024-09-20T22:15:35.457368+0000","last_peered":"2024-09-20T22:15:35.457368+0000","last_clean":"2024-09-20T22:15:35.457368+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:15:35.457368+0000","last_undegraded":"2024-09-20T22:15:35.457368+0000","last_fullsized":"2024-09-20T22:15:35.457368+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:00.010921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:00.010921+0000","last_clean_scrub_stamp":"2024-09-20T22:15:00.010921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:17.344827+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"328'15","reported_seq":233,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.452809+0000","last_change":"2024-09-20T22:12:43.218553+0000","last_active":"2024-09-20T22:15:35.452809+0000","last_peered":"2024-09-20T22:15:35.452809+0000","last_clean":"2024-09-20T22:15:35.452809+0000","last_became_active":"2024-09-20T22:11:30.493335+0000","last_became_peered":"2024-09-20T22:11:30.493335+0000","last_unstale":"2024-09-20T22:15:35.452809+0000","last_undegraded":"2024-09-20T22:15:35.452809+0000","last_fullsized":"2024-09-20T22:15:35.452809+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'9","last_scrub_stamp":"2024-09-20T22:12:43.218249+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:12:43.218249+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":140,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451946+0000","last_change":"2024-09-20T22:14:56.538965+0000","last_active":"2024-09-20T22:15:35.451946+0000","last_peered":"2024-09-20T22:15:35.451946+0000","last_clean":"2024-09-20T22:15:35.451946+0000","last_became_active":"2024-09-20T22:12:52.524149+0000","last_became_peered":"2024-09-20T22:12:52.524149+0000","last_unstale":"2024-09-20T22:15:35.451946+0000","last_undegraded":"2024-09-20T22:15:35.451946+0000","last_fullsized":"2024-09-20T22:15:35.451946+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:56.538825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:56.538825+0000","last_clean_scrub_stamp":"2024-09-20T22:14:56.538825+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:02.516641+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"315'11","reported_seq":221,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453332+0000","last_change":"2024-09-20T22:15:16.396532+0000","last_active":"2024-09-20T22:15:35.453332+0000","last_peered":"2024-09-20T22:15:35.453332+0000","last_clean":"2024-09-20T22:15:35.453332+0000","last_became_active":"2024-09-20T22:12:33.544098+0000","last_became_peered":"2024-09-20T22:12:33.544098+0000","last_unstale":"2024-09-20T22:15:35.453332+0000","last_undegraded":"2024-09-20T22:15:35.453332+0000","last_fullsized":"2024-09-20T22:15:35.453332+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:15:16.396454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:16.396454+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:18.359878+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":688,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.443193+0000","last_change":"2024-09-20T22:15:30.093649+0000","last_active":"2024-09-20T22:15:35.443193+0000","last_peered":"2024-09-20T22:15:35.443193+0000","last_clean":"2024-09-20T22:15:35.443193+0000","last_became_active":"2024-09-20T22:11:30.484731+0000","last_became_peered":"2024-09-20T22:11:30.484731+0000","last_unstale":"2024-09-20T22:15:35.443193+0000","last_undegraded":"2024-09-20T22:15:35.443193+0000","last_fullsized":"2024-09-20T22:15:35.443193+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:30.093548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:30.093548+0000","last_clean_scrub_stamp":"2024-09-20T22:15:30.093548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:55.627047+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"328'20","reported_seq":165,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559799+0000","last_change":"2024-09-20T22:13:49.905238+0000","last_active":"2024-09-20T22:15:23.559799+0000","last_peered":"2024-09-20T22:15:23.559799+0000","last_clean":"2024-09-20T22:15:23.559799+0000","last_became_active":"2024-09-20T22:12:44.631986+0000","last_became_peered":"2024-09-20T22:12:44.631986+0000","last_unstale":"2024-09-20T22:15:23.559799+0000","last_undegraded":"2024-09-20T22:15:23.559799+0000","last_fullsized":"2024-09-20T22:15:23.559799+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:13:49.905115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:49.905115+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":680,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453023+0000","last_change":"2024-09-20T22:14:23.041828+0000","last_active":"2024-09-20T22:15:35.453023+0000","last_peered":"2024-09-20T22:15:35.453023+0000","last_clean":"2024-09-20T22:15:35.453023+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:15:35.453023+0000","last_undegraded":"2024-09-20T22:15:35.453023+0000","last_fullsized":"2024-09-20T22:15:35.453023+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:23.041722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:23.041722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.321283+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"326'14","reported_seq":296,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.452619+0000","last_change":"2024-09-20T22:15:19.186947+0000","last_active":"2024-09-20T22:15:35.452619+0000","last_peered":"2024-09-20T22:15:35.452619+0000","last_clean":"2024-09-20T22:15:35.452619+0000","last_became_active":"2024-09-20T22:12:52.518770+0000","last_became_peered":"2024-09-20T22:12:52.518770+0000","last_unstale":"2024-09-20T22:15:35.452619+0000","last_undegraded":"2024-09-20T22:15:35.452619+0000","last_fullsized":"2024-09-20T22:15:35.452619+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'14","last_scrub_stamp":"2024-09-20T22:15:19.186862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:19.186862+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:43.532138+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":140,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.448769+0000","last_change":"2024-09-20T22:15:22.741319+0000","last_active":"2024-09-20T22:15:35.448769+0000","last_peered":"2024-09-20T22:15:35.448769+0000","last_clean":"2024-09-20T22:15:35.448769+0000","last_became_active":"2024-09-20T22:12:52.523781+0000","last_became_peered":"2024-09-20T22:12:52.523781+0000","last_unstale":"2024-09-20T22:15:35.448769+0000","last_undegraded":"2024-09-20T22:15:35.448769+0000","last_fullsized":"2024-09-20T22:15:35.448769+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:22.741214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:22.741214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:34.096157+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"328'19","reported_seq":736,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.443979+0000","last_change":"2024-09-20T22:14:59.020526+0000","last_active":"2024-09-20T22:15:35.443979+0000","last_peered":"2024-09-20T22:15:35.443979+0000","last_clean":"2024-09-20T22:15:35.443979+0000","last_became_active":"2024-09-20T22:08:22.139581+0000","last_became_peered":"2024-09-20T22:08:22.139581+0000","last_unstale":"2024-09-20T22:15:35.443979+0000","last_undegraded":"2024-09-20T22:15:35.443979+0000","last_fullsized":"2024-09-20T22:15:35.443979+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:14:59.020420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:59.020420+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:24.012397+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":688,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.443467+0000","last_change":"2024-09-20T22:15:20.031722+0000","last_active":"2024-09-20T22:15:35.443467+0000","last_peered":"2024-09-20T22:15:35.443467+0000","last_clean":"2024-09-20T22:15:35.443467+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:15:35.443467+0000","last_undegraded":"2024-09-20T22:15:35.443467+0000","last_fullsized":"2024-09-20T22:15:35.443467+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.031625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.031625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:36.647532+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":512,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559942+0000","last_change":"2024-09-20T22:13:52.605374+0000","last_active":"2024-09-20T22:15:23.559942+0000","last_peered":"2024-09-20T22:15:23.559942+0000","last_clean":"2024-09-20T22:15:23.559942+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:15:23.559942+0000","last_undegraded":"2024-09-20T22:15:23.559942+0000","last_fullsized":"2024-09-20T22:15:23.559942+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:52.605277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:13:52.605277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":523,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.559195+0000","last_change":"2024-09-20T22:14:18.440962+0000","last_active":"2024-09-20T22:15:23.559195+0000","last_peered":"2024-09-20T22:15:23.559195+0000","last_clean":"2024-09-20T22:15:23.559195+0000","last_became_active":"2024-09-20T22:12:52.521497+0000","last_became_peered":"2024-09-20T22:12:52.521497+0000","last_unstale":"2024-09-20T22:15:23.559195+0000","last_undegraded":"2024-09-20T22:15:23.559195+0000","last_fullsized":"2024-09-20T22:15:23.559195+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:18.438451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:18.438451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"328'19","reported_seq":597,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560215+0000","last_change":"2024-09-20T22:14:12.410224+0000","last_active":"2024-09-20T22:15:23.560215+0000","last_peered":"2024-09-20T22:15:23.560215+0000","last_clean":"2024-09-20T22:15:23.560215+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:15:23.560215+0000","last_undegraded":"2024-09-20T22:15:23.560215+0000","last_fullsized":"2024-09-20T22:15:23.560215+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:14:12.410107+0000","last_deep_scrub":"291'13","last_deep_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_clean_scrub_stamp":"2024-09-20T22:14:12.410107+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:29.679127+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":277,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453531+0000","last_change":"2024-09-20T22:13:11.651173+0000","last_active":"2024-09-20T22:15:35.453531+0000","last_peered":"2024-09-20T22:15:35.453531+0000","last_clean":"2024-09-20T22:15:35.453531+0000","last_became_active":"2024-09-20T22:10:30.655502+0000","last_became_peered":"2024-09-20T22:10:30.655502+0000","last_unstale":"2024-09-20T22:15:35.453531+0000","last_undegraded":"2024-09-20T22:15:35.453531+0000","last_fullsized":"2024-09-20T22:15:35.453531+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:11.650985+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:13:11.650985+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.320996+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":140,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.448499+0000","last_change":"2024-09-20T22:15:33.868714+0000","last_active":"2024-09-20T22:15:35.448499+0000","last_peered":"2024-09-20T22:15:35.448499+0000","last_clean":"2024-09-20T22:15:35.448499+0000","last_became_active":"2024-09-20T22:12:52.516953+0000","last_became_peered":"2024-09-20T22:12:52.516953+0000","last_unstale":"2024-09-20T22:15:35.448499+0000","last_undegraded":"2024-09-20T22:15:35.448499+0000","last_fullsized":"2024-09-20T22:15:35.448499+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:33.868591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:33.868591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:54.347076+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":140,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453349+0000","last_change":"2024-09-20T22:15:20.820202+0000","last_active":"2024-09-20T22:15:35.453349+0000","last_peered":"2024-09-20T22:15:35.453349+0000","last_clean":"2024-09-20T22:15:35.453349+0000","last_became_active":"2024-09-20T22:12:52.514247+0000","last_became_peered":"2024-09-20T22:12:52.514247+0000","last_unstale":"2024-09-20T22:15:35.453349+0000","last_undegraded":"2024-09-20T22:15:35.453349+0000","last_fullsized":"2024-09-20T22:15:35.453349+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.819992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.819992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:39.289842+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"328'19","reported_seq":605,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451035+0000","last_change":"2024-09-20T22:13:38.718567+0000","last_active":"2024-09-20T22:15:35.451035+0000","last_peered":"2024-09-20T22:15:35.451035+0000","last_clean":"2024-09-20T22:15:35.451035+0000","last_became_active":"2024-09-20T22:12:33.545199+0000","last_became_peered":"2024-09-20T22:12:33.545199+0000","last_unstale":"2024-09-20T22:15:35.451035+0000","last_undegraded":"2024-09-20T22:15:35.451035+0000","last_fullsized":"2024-09-20T22:15:35.451035+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:13:38.718450+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:38.718450+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:40.321148+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":225,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.452079+0000","last_change":"2024-09-20T22:15:27.149844+0000","last_active":"2024-09-20T22:15:35.452079+0000","last_peered":"2024-09-20T22:15:35.452079+0000","last_clean":"2024-09-20T22:15:35.452079+0000","last_became_active":"2024-09-20T22:11:30.493598+0000","last_became_peered":"2024-09-20T22:11:30.493598+0000","last_unstale":"2024-09-20T22:15:35.452079+0000","last_undegraded":"2024-09-20T22:15:35.452079+0000","last_fullsized":"2024-09-20T22:15:35.452079+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:27.149777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.149777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:39.146913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":680,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451276+0000","last_change":"2024-09-20T22:14:55.338392+0000","last_active":"2024-09-20T22:15:35.451276+0000","last_peered":"2024-09-20T22:15:35.451276+0000","last_clean":"2024-09-20T22:15:35.451276+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:15:35.451276+0000","last_undegraded":"2024-09-20T22:15:35.451276+0000","last_fullsized":"2024-09-20T22:15:35.451276+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:55.338318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:55.338318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:19.629689+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":129,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.448679+0000","last_change":"2024-09-20T22:14:46.684175+0000","last_active":"2024-09-20T22:15:35.448679+0000","last_peered":"2024-09-20T22:15:35.448679+0000","last_clean":"2024-09-20T22:15:35.448679+0000","last_became_active":"2024-09-20T22:12:57.883998+0000","last_became_peered":"2024-09-20T22:12:57.883998+0000","last_unstale":"2024-09-20T22:15:35.448679+0000","last_undegraded":"2024-09-20T22:15:35.448679+0000","last_fullsized":"2024-09-20T22:15:35.448679+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:46.683946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:46.683946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:46.807886+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"328'8","reported_seq":696,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.450876+0000","last_change":"2024-09-20T22:14:14.979857+0000","last_active":"2024-09-20T22:15:35.450876+0000","last_peered":"2024-09-20T22:15:35.450876+0000","last_clean":"2024-09-20T22:15:35.450876+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:15:35.450876+0000","last_undegraded":"2024-09-20T22:15:35.450876+0000","last_fullsized":"2024-09-20T22:15:35.450876+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'8","last_scrub_stamp":"2024-09-20T22:14:14.979748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:14.979748+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":132,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.449127+0000","last_change":"2024-09-20T22:14:21.540990+0000","last_active":"2024-09-20T22:15:35.449127+0000","last_peered":"2024-09-20T22:15:35.449127+0000","last_clean":"2024-09-20T22:15:35.449127+0000","last_became_active":"2024-09-20T22:12:52.517510+0000","last_became_peered":"2024-09-20T22:12:52.517510+0000","last_unstale":"2024-09-20T22:15:35.449127+0000","last_undegraded":"2024-09-20T22:15:35.449127+0000","last_fullsized":"2024-09-20T22:15:35.449127+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:21.540878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:21.540878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":703,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.446768+0000","last_change":"2024-09-20T22:15:20.569279+0000","last_active":"2024-09-20T22:15:35.446768+0000","last_peered":"2024-09-20T22:15:35.446768+0000","last_clean":"2024-09-20T22:15:35.446768+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:15:35.446768+0000","last_undegraded":"2024-09-20T22:15:35.446768+0000","last_fullsized":"2024-09-20T22:15:35.446768+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.569168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.569168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:31.279582+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":676,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.451144+0000","last_change":"2024-09-20T22:15:29.393451+0000","last_active":"2024-09-20T22:15:35.451144+0000","last_peered":"2024-09-20T22:15:35.451144+0000","last_clean":"2024-09-20T22:15:35.451144+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:15:35.451144+0000","last_undegraded":"2024-09-20T22:15:35.451144+0000","last_fullsized":"2024-09-20T22:15:35.451144+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:29.393342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_clean_scrub_stamp":"2024-09-20T22:15:29.393342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:34.744649+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":561,"reported_epoch":378,"state":"active+clean","last_fresh":"2024-09-20T22:15:23.560449+0000","last_change":"2024-09-20T22:14:13.408076+0000","last_active":"2024-09-20T22:15:23.560449+0000","last_peered":"2024-09-20T22:15:23.560449+0000","last_clean":"2024-09-20T22:15:23.560449+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:15:23.560449+0000","last_undegraded":"2024-09-20T22:15:23.560449+0000","last_fullsized":"2024-09-20T22:15:23.560449+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:13.407980+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:13.407980+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:15:41.846519+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":140,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.453864+0000","last_change":"2024-09-20T22:15:18.828980+0000","last_active":"2024-09-20T22:15:35.453864+0000","last_peered":"2024-09-20T22:15:35.453864+0000","last_clean":"2024-09-20T22:15:35.453864+0000","last_became_active":"2024-09-20T22:12:52.518076+0000","last_became_peered":"2024-09-20T22:12:52.518076+0000","last_unstale":"2024-09-20T22:15:35.453864+0000","last_undegraded":"2024-09-20T22:15:35.453864+0000","last_fullsized":"2024-09-20T22:15:35.453864+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:18.828903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:18.828903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:26.559172+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"328'19","reported_seq":724,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.447862+0000","last_change":"2024-09-20T22:14:15.443443+0000","last_active":"2024-09-20T22:15:35.447862+0000","last_peered":"2024-09-20T22:15:35.447862+0000","last_clean":"2024-09-20T22:15:35.447862+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:15:35.447862+0000","last_undegraded":"2024-09-20T22:15:35.447862+0000","last_fullsized":"2024-09-20T22:15:35.447862+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:14:15.443321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:14:15.443321+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":140,"reported_epoch":380,"state":"active+clean","last_fresh":"2024-09-20T22:15:35.448517+0000","last_change":"2024-09-20T22:15:27.761950+0000","last_active":"2024-09-20T22:15:35.448517+0000","last_peered":"2024-09-20T22:15:35.448517+0000","last_clean":"2024-09-20T22:15:35.448517+0000","last_became_active":"2024-09-20T22:12:52.524425+0000","last_became_peered":"2024-09-20T22:12:52.524425+0000","last_unstale":"2024-09-20T22:15:35.448517+0000","last_undegraded":"2024-09-20T22:15:35.448517+0000","last_fullsized":"2024-09-20T22:15:35.448517+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:27.761843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.761843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:46.763986+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":2687238144,"num_objects":41004,"num_object_clones":0,"num_object_copies":123012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41004,"num_write_kb":2624256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":7956791296,"data_stored":7956791296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41004,"ondisk_log_size":41004,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":3300,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":716,"ondisk_log_size":716,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":742,"num_read_kb":737,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":11,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":918048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1408,"ondisk_log_size":1408,"up":33,"acting":33,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813136029,"num_pgs":63,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1223592,"kb_used_data":610708,"kb_used_omap":1,"kb_used_meta":612862,"kb_avail":92521560,"statfs":{"total":95995035648,"available":94742077440,"internally_reserved":0,"allocated":625364992,"data_stored":625011981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":627571146},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430367,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1649244,"kb_used_data":1198796,"kb_used_omap":1,"kb_used_meta":450430,"kb_avail":92095908,"statfs":{"total":95995035648,"available":94306209792,"internally_reserved":0,"allocated":1227567104,"data_stored":1227230511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":461240774},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724709,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1429664,"kb_used_data":802888,"kb_used_omap":1,"kb_used_meta":626750,"kb_avail":92315488,"statfs":{"total":95995035648,"available":94531059712,"internally_reserved":0,"allocated":822157312,"data_stored":821823734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":641792453},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986349,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1683752,"kb_used_data":1219412,"kb_used_omap":1,"kb_used_meta":464318,"kb_avail":92061400,"statfs":{"total":95995035648,"available":94270873600,"internally_reserved":0,"allocated":1248677888,"data_stored":1248324166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":475462085},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770258025,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":877996,"kb_used_data":419356,"kb_used_omap":4,"kb_used_meta":458619,"kb_avail":92867156,"statfs":{"total":95995035648,"available":95095967744,"internally_reserved":0,"allocated":429420544,"data_stored":429051549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":469626536},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149874,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1467912,"kb_used_data":796720,"kb_used_omap":1,"kb_used_meta":671166,"kb_avail":92277240,"statfs":{"total":95995035648,"available":94491893760,"internally_reserved":0,"allocated":815841280,"data_stored":815468606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":687274439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411511,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1464928,"kb_used_data":950600,"kb_used_omap":1,"kb_used_meta":514302,"kb_avail":92280224,"statfs":{"total":95995035648,"available":94494949376,"internally_reserved":0,"allocated":973414400,"data_stored":973081265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":526645701},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738562,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2301104,"kb_used_data":1647900,"kb_used_omap":1,"kb_used_meta":653118,"kb_avail":91444048,"statfs":{"total":95995035648,"available":93638705152,"internally_reserved":0,"allocated":1687449600,"data_stored":1687096303,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":668793285},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1692073984,"data_stored":1692073984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":445054976,"data_stored":445054976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":991100928,"data_stored":991100928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":840630272,"data_stored":840630272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1280770048,"data_stored":1280770048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":826408960,"data_stored":826408960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1242628096,"data_stored":1242628096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":638124032,"data_stored":638124032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:15:39.766 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-20T22:15:39.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool set .mgr pg_num 21 2024-09-20T22:15:39.789 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:39.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:39.789 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:39.890 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:39.917 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:39.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:39.918 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:39.921 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 22 16 46093 46077 130.867 96.9375 0.00495223 0.00763508 2024-09-20T22:15:40.019 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:40.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:40.065 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:40.066 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:40.166 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:40.209 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:40.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:40.211 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:40.311 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:40.334 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:40.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:40.335 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:40.436 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:40.458 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:40.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:40.459 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:40.521 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:40.526 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:40.561 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:40.598 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:40.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:40.599 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:40.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:40.707 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:40 smithi060 ceph-mon[29694]: from='client.16044 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:15:40.708 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:40 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:15:40.708 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:40 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:15:40.708 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:40 smithi060 ceph-mon[29694]: osdmap e381: 8 total, 8 up, 8 in 2024-09-20T22:15:40.708 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:40 smithi060 ceph-mon[40290]: from='client.16044 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:15:40.708 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:40 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:15:40.708 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:40 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:15:40.708 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:40 smithi060 ceph-mon[40290]: osdmap e381: 8 total, 8 up, 8 in 2024-09-20T22:15:40.737 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:40.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:40.738 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:40 smithi067 ceph-mon[33511]: from='client.16044 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:15:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:40 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:15:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:40 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:15:40.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:40 smithi067 ceph-mon[33511]: osdmap e381: 8 total, 8 up, 8 in 2024-09-20T22:15:40.839 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:40.890 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:40.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:40.891 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:40.920 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 23 16 47464 47448 128.903 85.6875 0.010239 0.00775293 2024-09-20T22:15:40.992 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:40.997 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T22:15:41.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:41.029 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:41.030 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:41.130 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:41.176 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:41.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:41.177 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:41.278 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:41.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:41.318 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:41.318 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:41.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:41.459 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:41.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:41.460 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:41.472 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:15:41.473 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:15:41.477 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:15:41.511 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:15:41.511 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:15:41.561 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:41.563 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:15:41.563 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:15:41.589 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:41.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:41.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:41.611 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:15:41.611 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:15:41.611 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:41 smithi060 ceph-mon[29694]: pgmap v681: 158 pgs: 158 active+clean; 2.5 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-20T22:15:41.611 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:41 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/87357960' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:41.612 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:41 smithi060 ceph-mon[40290]: pgmap v681: 158 pgs: 158 active+clean; 2.5 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-20T22:15:41.612 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:41 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/87357960' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:41.650 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:15:41.650 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:15:41.690 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:41.694 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:15:41.694 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:15:41.723 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:41.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:41.724 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:41.733 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:15:41.733 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:15:41.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:41 smithi067 ceph-mon[33511]: pgmap v681: 158 pgs: 158 active+clean; 2.5 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 852 B/s rd, 83 MiB/s wr, 1.33k op/s 2024-09-20T22:15:41.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:41 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/87357960' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:41.768 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:15:41.768 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:15:41.825 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:41.864 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:41.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:41.866 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:41.920 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 24 16 48819 48803 127.061 84.6875 0.00488838 0.00786447 2024-09-20T22:15:41.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:42.001 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:42.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:42.002 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:42.102 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:42.130 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:15:42.130 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:15:42.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:42.141 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:42.142 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:42.243 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:42.280 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:42.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:42.281 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:42.382 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:42.421 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:42.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:42.422 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:42.478 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:15:42.523 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:42.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:42.555 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:42.555 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:42.567 INFO:teuthology.orchestra.run.smithi060.stderr:set pool 1 pg_num to 21 2024-09-20T22:15:42.656 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:42.688 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:42.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:42.689 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:42 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1780573195' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]: dispatch 2024-09-20T22:15:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:42 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]: dispatch 2024-09-20T22:15:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:42 smithi067 ceph-mon[33511]: pgmap v682: 158 pgs: 158 active+clean; 2.8 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.3 KiB/s rd, 99 MiB/s wr, 1.59k op/s 2024-09-20T22:15:42.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:42 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:15:42.764 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:15:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:15:42.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:42 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1780573195' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]: dispatch 2024-09-20T22:15:42.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:42 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]: dispatch 2024-09-20T22:15:42.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:42 smithi060 ceph-mon[29694]: pgmap v682: 158 pgs: 158 active+clean; 2.8 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.3 KiB/s rd, 99 MiB/s wr, 1.59k op/s 2024-09-20T22:15:42.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:42 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:15:42.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:42 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1780573195' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]: dispatch 2024-09-20T22:15:42.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:42 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]: dispatch 2024-09-20T22:15:42.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:42 smithi060 ceph-mon[40290]: pgmap v682: 158 pgs: 158 active+clean; 2.8 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 1.3 KiB/s rd, 99 MiB/s wr, 1.59k op/s 2024-09-20T22:15:42.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:42 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:15:42.791 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:42.832 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:42.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:42.833 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:42.877 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T22:15:42.920 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 25 16 50301 50285 125.683 92.625 0.00454071 0.00794924 2024-09-20T22:15:42.934 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:42.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:42.984 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:42.984 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:43.085 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:43.114 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:43.115 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:43.115 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:43.216 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:43.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:43.275 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:43.275 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:43.336 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:15:43.377 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:43.417 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:43.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:43.418 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:43.519 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:43.557 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:43.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:43.558 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:43.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:43.683 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:43.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:43.684 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:43.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:43 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]': finished 2024-09-20T22:15:43.763 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:43 smithi060 ceph-mon[29694]: osdmap e382: 8 total, 8 up, 8 in 2024-09-20T22:15:43.764 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:43 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]': finished 2024-09-20T22:15:43.764 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:43 smithi060 ceph-mon[40290]: osdmap e382: 8 total, 8 up, 8 in 2024-09-20T22:15:43.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:43 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "21"}]': finished 2024-09-20T22:15:43.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:43 smithi067 ceph-mon[33511]: osdmap e382: 8 total, 8 up, 8 in 2024-09-20T22:15:43.785 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:43.824 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:43.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:43.825 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:43.921 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 26 16 51622 51606 124.025 82.5625 0.00786756 0.00805642 2024-09-20T22:15:43.925 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:43.951 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:43.963 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:43.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:43.964 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:44.064 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:44.088 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:44.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:44.089 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:44.190 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:44.222 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:44.222 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:44.223 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:44.323 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:44.352 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:44.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:44.353 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:44.454 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:44.488 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:44.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:44.489 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:44.590 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:44.629 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:44.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:44.630 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:44.691 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:15:44.713 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:44 smithi060 ceph-mon[29694]: pgmap v684: 158 pgs: 158 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 101 MiB/s wr, 1.62k op/s 2024-09-20T22:15:44.713 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:44 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-09-20T22:15:44.713 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:44 smithi060 ceph-mon[40290]: pgmap v684: 158 pgs: 158 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 101 MiB/s wr, 1.62k op/s 2024-09-20T22:15:44.713 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:44 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-09-20T22:15:44.731 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:44.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:44.754 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:44.839 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:44 smithi067 ceph-mon[33511]: pgmap v684: 158 pgs: 158 active+clean; 3.0 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 639 B/s rd, 101 MiB/s wr, 1.62k op/s 2024-09-20T22:15:44.839 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:44 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]: dispatch 2024-09-20T22:15:44.854 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:44.900 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:44.901 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:44.902 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:44.922 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 27 16 52658 52642 121.824 64.75 0.00997873 0.00820238 2024-09-20T22:15:45.002 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:45.042 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:45.047 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:45.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:45.049 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:45.150 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:45.191 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:45.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:45.192 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:45.293 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:45.333 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:45.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:45.334 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:45.346 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:15:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:15:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:15:45.435 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:45.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:45.473 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:45.474 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:45.574 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:45.603 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:15:45.613 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:45.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:45.614 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:45.649 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:15:45.857 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:15:45.857 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:45.858 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:45 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]': finished 2024-09-20T22:15:45.859 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:45 smithi067 ceph-mon[33511]: osdmap e383: 8 total, 8 up, 8 in 2024-09-20T22:15:45.859 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:45 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2778210286' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:15:45.859 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:15:45.859 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:15:45.859 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:15:45.860 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:15:45.860 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:15:45.860 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:15:45.861 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:15:45.861 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:15:45.876 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:15:45.877 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:15:45.877 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:45 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]': finished 2024-09-20T22:15:45.877 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:45 smithi060 ceph-mon[29694]: osdmap e383: 8 total, 8 up, 8 in 2024-09-20T22:15:45.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:45 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2778210286' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:15:45.878 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:45 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "21"}]': finished 2024-09-20T22:15:45.878 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:45 smithi060 ceph-mon[40290]: osdmap e383: 8 total, 8 up, 8 in 2024-09-20T22:15:45.878 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:45 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2778210286' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:15:45.893 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:45.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:45.894 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:45.915 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:15:45.915 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:15:45.922 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 28 16 54102 54086 120.696 90.25 0.011496 0.00827995 2024-09-20T22:15:45.952 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:15:45.952 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:15:45.991 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:15:45.991 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:15:45.995 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:46.025 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:46.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:46.026 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:46.127 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:46.164 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:46.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:46.165 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:46.265 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:46.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:46.305 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:46.305 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:46.406 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:46.411 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T22:15:46.444 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:46.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:46.445 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:46.546 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:46.590 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:46.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:46.592 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:46.692 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:46.728 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:46.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:46.729 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:46.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:15:46.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2778210286' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:15:46.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[29694]: osdmap e384: 8 total, 8 up, 8 in 2024-09-20T22:15:46.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3428670010' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:46.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[29694]: 1.14 scrub starts 2024-09-20T22:15:46.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[29694]: 1.14 scrub ok 2024-09-20T22:15:46.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[29694]: pgmap v687: 168 pgs: 6 unknown, 162 active+clean; 3.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.7 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-20T22:15:46.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]: dispatch 2024-09-20T22:15:46.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[29694]: 1.3 scrub starts 2024-09-20T22:15:46.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[29694]: 1.3 scrub ok 2024-09-20T22:15:46.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]': finished 2024-09-20T22:15:46.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[29694]: osdmap e385: 8 total, 8 up, 8 in 2024-09-20T22:15:46.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:15:46.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2778210286' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:15:46.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[40290]: osdmap e384: 8 total, 8 up, 8 in 2024-09-20T22:15:46.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3428670010' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:46.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[40290]: 1.14 scrub starts 2024-09-20T22:15:46.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[40290]: 1.14 scrub ok 2024-09-20T22:15:46.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[40290]: pgmap v687: 168 pgs: 6 unknown, 162 active+clean; 3.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.7 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-20T22:15:46.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]: dispatch 2024-09-20T22:15:46.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[40290]: 1.3 scrub starts 2024-09-20T22:15:46.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[40290]: 1.3 scrub ok 2024-09-20T22:15:46.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]': finished 2024-09-20T22:15:46.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:46 smithi060 ceph-mon[40290]: osdmap e385: 8 total, 8 up, 8 in 2024-09-20T22:15:46.830 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:46.884 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:15:46.884 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:46.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:46.886 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:46.922 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 29 16 55629 55613 119.825 95.4375 0.00690215 0.00834057 2024-09-20T22:15:46.986 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:46 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:15:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:46 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2778210286' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:15:47.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:46 smithi067 ceph-mon[33511]: osdmap e384: 8 total, 8 up, 8 in 2024-09-20T22:15:47.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:46 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3428670010' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:47.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:46 smithi067 ceph-mon[33511]: 1.14 scrub starts 2024-09-20T22:15:47.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:46 smithi067 ceph-mon[33511]: 1.14 scrub ok 2024-09-20T22:15:47.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:46 smithi067 ceph-mon[33511]: pgmap v687: 168 pgs: 6 unknown, 162 active+clean; 3.1 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.7 KiB/s rd, 100 MiB/s wr, 1.60k op/s 2024-09-20T22:15:47.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:46 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]: dispatch 2024-09-20T22:15:47.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:46 smithi067 ceph-mon[33511]: 1.3 scrub starts 2024-09-20T22:15:47.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:46 smithi067 ceph-mon[33511]: 1.3 scrub ok 2024-09-20T22:15:47.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:46 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "21"}]': finished 2024-09-20T22:15:47.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:46 smithi067 ceph-mon[33511]: osdmap e385: 8 total, 8 up, 8 in 2024-09-20T22:15:47.021 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:47.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:47.022 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:47.123 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:47.158 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:47.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:47.159 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:47.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:47.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:47.299 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:47.300 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:47.400 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:47.443 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:47.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:47.444 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:47.522 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:15:47.522 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:15:47.523 INFO:teuthology.orchestra.run.smithi067.stdout: "complaint_time": 30, 2024-09-20T22:15:47.523 INFO:teuthology.orchestra.run.smithi067.stdout: "num_blocked_ops": 0 2024-09-20T22:15:47.523 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:15:47.545 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:47.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:47.588 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:47.588 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:47.689 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:47.727 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:47.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:47.728 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:47.828 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:47.868 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:47.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:47.869 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:47.877 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:47 smithi067 ceph-mon[33511]: 1.1 scrub starts 2024-09-20T22:15:47.877 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:47 smithi067 ceph-mon[33511]: 1.1 scrub ok 2024-09-20T22:15:47.877 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:47 smithi067 ceph-mon[33511]: 1.5 scrub starts 2024-09-20T22:15:47.877 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:47 smithi067 ceph-mon[33511]: 1.5 scrub ok 2024-09-20T22:15:47.877 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:47 smithi067 ceph-mon[33511]: 1.2 scrub starts 2024-09-20T22:15:47.877 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:47 smithi067 ceph-mon[33511]: 1.2 scrub ok 2024-09-20T22:15:47.878 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:47 smithi067 ceph-mon[33511]: 1.7 scrub starts 2024-09-20T22:15:47.878 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:47 smithi067 ceph-mon[33511]: 1.7 scrub ok 2024-09-20T22:15:47.878 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:47 smithi067 ceph-mon[33511]: 1.4 scrub starts 2024-09-20T22:15:47.878 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:47 smithi067 ceph-mon[33511]: 1.4 scrub ok 2024-09-20T22:15:47.878 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:47 smithi067 ceph-mon[33511]: osdmap e386: 8 total, 8 up, 8 in 2024-09-20T22:15:47.922 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 30 16 57212 57196 119.129 98.9375 0.00540379 0.00838905 2024-09-20T22:15:47.970 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:48.011 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:48.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:48.014 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:48.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[29694]: 1.1 scrub starts 2024-09-20T22:15:48.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[29694]: 1.1 scrub ok 2024-09-20T22:15:48.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[29694]: 1.5 scrub starts 2024-09-20T22:15:48.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[29694]: 1.5 scrub ok 2024-09-20T22:15:48.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[29694]: 1.2 scrub starts 2024-09-20T22:15:48.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[29694]: 1.2 scrub ok 2024-09-20T22:15:48.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[29694]: 1.7 scrub starts 2024-09-20T22:15:48.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[29694]: 1.7 scrub ok 2024-09-20T22:15:48.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[29694]: 1.4 scrub starts 2024-09-20T22:15:48.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[29694]: 1.4 scrub ok 2024-09-20T22:15:48.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[29694]: osdmap e386: 8 total, 8 up, 8 in 2024-09-20T22:15:48.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[40290]: 1.1 scrub starts 2024-09-20T22:15:48.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[40290]: 1.1 scrub ok 2024-09-20T22:15:48.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[40290]: 1.5 scrub starts 2024-09-20T22:15:48.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[40290]: 1.5 scrub ok 2024-09-20T22:15:48.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[40290]: 1.2 scrub starts 2024-09-20T22:15:48.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[40290]: 1.2 scrub ok 2024-09-20T22:15:48.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[40290]: 1.7 scrub starts 2024-09-20T22:15:48.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[40290]: 1.7 scrub ok 2024-09-20T22:15:48.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[40290]: 1.4 scrub starts 2024-09-20T22:15:48.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[40290]: 1.4 scrub ok 2024-09-20T22:15:48.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:47 smithi060 ceph-mon[40290]: osdmap e386: 8 total, 8 up, 8 in 2024-09-20T22:15:48.115 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:48.157 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:48.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:48.158 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:48.259 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:48.303 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T22:15:48.304 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:48.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:48.305 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:48.405 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:48.440 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:48.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:48.441 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:48.539 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:15:48.542 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:48.577 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:48.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:48.578 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:48.679 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:48.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:48.733 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:48.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:48.793 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:15:48.817 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:15:48.817 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 1 1 2024-09-20T22:15:48.834 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:48.857 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[29694]: 1.0 scrub starts 2024-09-20T22:15:48.857 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[29694]: 1.0 scrub ok 2024-09-20T22:15:48.857 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:15:48.857 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:15:48.857 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[29694]: 1.6 scrub starts 2024-09-20T22:15:48.857 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[29694]: 1.6 scrub ok 2024-09-20T22:15:48.857 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[29694]: 4.10 scrub starts 2024-09-20T22:15:48.857 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[29694]: 4.10 scrub ok 2024-09-20T22:15:48.857 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[29694]: pgmap v690: 168 pgs: 1 remapped+peering, 6 unknown, 161 active+clean; 3.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-20T22:15:48.857 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[29694]: osdmap e387: 8 total, 8 up, 8 in 2024-09-20T22:15:48.858 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[40290]: 1.0 scrub starts 2024-09-20T22:15:48.858 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[40290]: 1.0 scrub ok 2024-09-20T22:15:48.858 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:15:48.858 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:15:48.858 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[40290]: 1.6 scrub starts 2024-09-20T22:15:48.858 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[40290]: 1.6 scrub ok 2024-09-20T22:15:48.858 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[40290]: 4.10 scrub starts 2024-09-20T22:15:48.858 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[40290]: 4.10 scrub ok 2024-09-20T22:15:48.858 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[40290]: pgmap v690: 168 pgs: 1 remapped+peering, 6 unknown, 161 active+clean; 3.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-20T22:15:48.858 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:48 smithi060 ceph-mon[40290]: osdmap e387: 8 total, 8 up, 8 in 2024-09-20T22:15:48.873 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:48.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:48.874 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:48.922 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 31 16 58673 58657 118.231 91.3125 0.00785783 0.00845303 2024-09-20T22:15:48.927 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:48 smithi067 ceph-mon[33511]: 1.0 scrub starts 2024-09-20T22:15:48.928 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:48 smithi067 ceph-mon[33511]: 1.0 scrub ok 2024-09-20T22:15:48.928 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:48 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:15:48.928 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:48 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:15:48.928 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:48 smithi067 ceph-mon[33511]: 1.6 scrub starts 2024-09-20T22:15:48.928 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:48 smithi067 ceph-mon[33511]: 1.6 scrub ok 2024-09-20T22:15:48.928 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:48 smithi067 ceph-mon[33511]: 4.10 scrub starts 2024-09-20T22:15:48.928 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:48 smithi067 ceph-mon[33511]: 4.10 scrub ok 2024-09-20T22:15:48.928 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:48 smithi067 ceph-mon[33511]: pgmap v690: 168 pgs: 1 remapped+peering, 6 unknown, 161 active+clean; 3.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.2 KiB/s rd, 101 MiB/s wr, 1.61k op/s 2024-09-20T22:15:48.928 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:48 smithi067 ceph-mon[33511]: osdmap e387: 8 total, 8 up, 8 in 2024-09-20T22:15:48.975 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:49.022 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:49.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:49.023 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:49.124 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:49.157 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:49.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:49.158 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:49.171 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:49.259 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:49.296 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:49.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:49.297 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:49.381 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:15:49.393 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:49.397 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:49.421 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:49.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:49.422 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:49.523 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:49.545 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:49.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:49.546 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:49.646 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:49.692 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:49.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:49.693 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:49.794 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:49.833 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:49.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:49.834 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:49.922 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 32 16 59716 59700 116.574 65.1875 0.00667957 0.0085724 2024-09-20T22:15:49.935 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:49.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:49.971 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:49.971 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:50.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:49 smithi067 ceph-mon[33511]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-20T22:15:50.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:49 smithi067 ceph-mon[33511]: 2.1 scrub starts 2024-09-20T22:15:50.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:49 smithi067 ceph-mon[33511]: 2.1 scrub ok 2024-09-20T22:15:50.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:49 smithi067 ceph-mon[33511]: 6.1d scrub starts 2024-09-20T22:15:50.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:49 smithi067 ceph-mon[33511]: 6.1d scrub ok 2024-09-20T22:15:50.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:49 smithi067 ceph-mon[33511]: osdmap e388: 8 total, 8 up, 8 in 2024-09-20T22:15:50.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:49 smithi060 ceph-mon[29694]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-20T22:15:50.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:49 smithi060 ceph-mon[29694]: 2.1 scrub starts 2024-09-20T22:15:50.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:49 smithi060 ceph-mon[29694]: 2.1 scrub ok 2024-09-20T22:15:50.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:49 smithi060 ceph-mon[29694]: 6.1d scrub starts 2024-09-20T22:15:50.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:49 smithi060 ceph-mon[29694]: 6.1d scrub ok 2024-09-20T22:15:50.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:49 smithi060 ceph-mon[29694]: osdmap e388: 8 total, 8 up, 8 in 2024-09-20T22:15:50.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:49 smithi060 ceph-mon[40290]: Health check failed: Reduced data availability: 1 pg inactive (PG_AVAILABILITY) 2024-09-20T22:15:50.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:49 smithi060 ceph-mon[40290]: 2.1 scrub starts 2024-09-20T22:15:50.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:49 smithi060 ceph-mon[40290]: 2.1 scrub ok 2024-09-20T22:15:50.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:49 smithi060 ceph-mon[40290]: 6.1d scrub starts 2024-09-20T22:15:50.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:49 smithi060 ceph-mon[40290]: 6.1d scrub ok 2024-09-20T22:15:50.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:49 smithi060 ceph-mon[40290]: osdmap e388: 8 total, 8 up, 8 in 2024-09-20T22:15:50.072 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:50.108 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:50.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:50.109 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:50.210 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:50.235 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:50.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:50.236 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:50.336 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:50.358 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:50.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:50.359 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:50.459 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:50.481 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:50.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:50.482 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:50.583 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:50.609 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:50.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:50.610 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:50.711 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:50.739 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:50.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:50.740 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:50.840 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:50.877 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:50.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:50.879 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:50.911 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:50 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:15:50.911 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:50 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:15:50.911 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:50 smithi067 ceph-mon[33511]: 5.d scrub starts 2024-09-20T22:15:50.911 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:50 smithi067 ceph-mon[33511]: 5.d scrub ok 2024-09-20T22:15:50.911 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:50 smithi067 ceph-mon[33511]: 3.17 scrub starts 2024-09-20T22:15:50.911 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:50 smithi067 ceph-mon[33511]: 3.17 scrub ok 2024-09-20T22:15:50.911 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:50 smithi067 ceph-mon[33511]: pgmap v693: 168 pgs: 3 remapped+peering, 4 unknown, 161 active+clean; 3.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 75 MiB/s wr, 1.19k op/s 2024-09-20T22:15:50.911 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:50 smithi067 ceph-mon[33511]: osdmap e389: 8 total, 8 up, 8 in 2024-09-20T22:15:50.923 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 33 16 60664 60648 114.837 59.25 0.013698 0.00870239 2024-09-20T22:15:50.980 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:51.027 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:51.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:51.028 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:51.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:50 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:15:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:50 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:15:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:50 smithi060 ceph-mon[29694]: 5.d scrub starts 2024-09-20T22:15:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:50 smithi060 ceph-mon[29694]: 5.d scrub ok 2024-09-20T22:15:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:50 smithi060 ceph-mon[29694]: 3.17 scrub starts 2024-09-20T22:15:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:50 smithi060 ceph-mon[29694]: 3.17 scrub ok 2024-09-20T22:15:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:50 smithi060 ceph-mon[29694]: pgmap v693: 168 pgs: 3 remapped+peering, 4 unknown, 161 active+clean; 3.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 75 MiB/s wr, 1.19k op/s 2024-09-20T22:15:51.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:50 smithi060 ceph-mon[29694]: osdmap e389: 8 total, 8 up, 8 in 2024-09-20T22:15:51.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:50 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:15:51.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:50 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:15:51.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:50 smithi060 ceph-mon[40290]: 5.d scrub starts 2024-09-20T22:15:51.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:50 smithi060 ceph-mon[40290]: 5.d scrub ok 2024-09-20T22:15:51.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:50 smithi060 ceph-mon[40290]: 3.17 scrub starts 2024-09-20T22:15:51.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:50 smithi060 ceph-mon[40290]: 3.17 scrub ok 2024-09-20T22:15:51.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:50 smithi060 ceph-mon[40290]: pgmap v693: 168 pgs: 3 remapped+peering, 4 unknown, 161 active+clean; 3.4 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 75 MiB/s wr, 1.19k op/s 2024-09-20T22:15:51.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:50 smithi060 ceph-mon[40290]: osdmap e389: 8 total, 8 up, 8 in 2024-09-20T22:15:51.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:51.167 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:51.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:51.168 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:51.269 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:51.307 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:51.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:51.308 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:51.408 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:51.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:51.445 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:51.446 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:51.517 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:15:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:15:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:15:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:15:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:169974 7.10 7:0a082cf8:::benchmark_data_smithi060_78800_object36941:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e121)", 2024-09-20T22:15:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:15:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 592.45736164899995, 2024-09-20T22:15:51.517 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87805208899999998, 2024-09-20T22:15:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:15:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:15:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:15:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 169974 2024-09-20T22:15:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:15:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.518 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624614+0000", 2024-09-20T22:15:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624616+0000", 2024-09-20T22:15:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1739999999999999e-06 2024-09-20T22:15:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624626+0000", 2024-09-20T22:15:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7909999999999999e-06 2024-09-20T22:15:51.519 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624654+0000", 2024-09-20T22:15:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.8524999999999999e-05 2024-09-20T22:15:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.624665+0000", 2024-09-20T22:15:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0985000000000001e-05 2024-09-20T22:15:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.705237+0000", 2024-09-20T22:15:51.520 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.080571485999999998 2024-09-20T22:15:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:58.705245+0000", 2024-09-20T22:15:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8460000000000008e-06 2024-09-20T22:15:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498359+0000", 2024-09-20T22:15:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79311461999999999 2024-09-20T22:15:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.521 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498498+0000", 2024-09-20T22:15:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013918599999999999 2024-09-20T22:15:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3", 2024-09-20T22:15:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.498751+0000", 2024-09-20T22:15:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025235999999999998 2024-09-20T22:15:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.500697+0000", 2024-09-20T22:15:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0019460969999999999 2024-09-20T22:15:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.522 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502528+0000", 2024-09-20T22:15:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00183161 2024-09-20T22:15:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:15:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502597+0000", 2024-09-20T22:15:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8761000000000001e-05 2024-09-20T22:15:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:05:59.502666+0000", 2024-09-20T22:15:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8647999999999997e-05 2024-09-20T22:15:51.523 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.524 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.524 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227618 7.0 7:0598f762:::benchmark_data_smithi060_78800_object94585:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e144)", 2024-09-20T22:15:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:53.805369+0000", 2024-09-20T22:15:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 537.27660662400001, 2024-09-20T22:15:51.525 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2615457110000001, 2024-09-20T22:15:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:15:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:15:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:15:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227618 2024-09-20T22:15:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805369+0000", 2024-09-20T22:15:51.526 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805369+0000", 2024-09-20T22:15:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805371+0000", 2024-09-20T22:15:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.827e-06 2024-09-20T22:15:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805377+0000", 2024-09-20T22:15:51.527 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.161e-06 2024-09-20T22:15:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805398+0000", 2024-09-20T22:15:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1208000000000001e-05 2024-09-20T22:15:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.805406+0000", 2024-09-20T22:15:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3820000000000004e-06 2024-09-20T22:15:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.813177+0000", 2024-09-20T22:15:51.528 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0077708919999999997 2024-09-20T22:15:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:53.813194+0000", 2024-09-20T22:15:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7107000000000001e-05 2024-09-20T22:15:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.813273+0000", 2024-09-20T22:15:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.000078609 2024-09-20T22:15:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.529 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.813290+0000", 2024-09-20T22:15:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7382999999999999e-05 2024-09-20T22:15:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.960476+0000", 2024-09-20T22:15:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1471854260000001 2024-09-20T22:15:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.960616+0000", 2024-09-20T22:15:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014035100000000001 2024-09-20T22:15:51.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,3", 2024-09-20T22:15:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.961013+0000", 2024-09-20T22:15:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00039649999999999999 2024-09-20T22:15:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.963511+0000", 2024-09-20T22:15:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0024986919999999998 2024-09-20T22:15:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.963553+0000", 2024-09-20T22:15:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1783999999999998e-05 2024-09-20T22:15:51.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.066713+0000", 2024-09-20T22:15:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.103160088 2024-09-20T22:15:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:15:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.066794+0000", 2024-09-20T22:15:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0924e-05 2024-09-20T22:15:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.066915+0000", 2024-09-20T22:15:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000120377 2024-09-20T22:15:51.532 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227635 7.b 7:da8bd73f:::benchmark_data_smithi060_78800_object94602:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e146)", 2024-09-20T22:15:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:54.820720+0000", 2024-09-20T22:15:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 536.26125574800005, 2024-09-20T22:15:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1621617440000001, 2024-09-20T22:15:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:15:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:15:51.533 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:15:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227635 2024-09-20T22:15:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820720+0000", 2024-09-20T22:15:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820720+0000", 2024-09-20T22:15:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.534 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820721+0000", 2024-09-20T22:15:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.593e-06 2024-09-20T22:15:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820728+0000", 2024-09-20T22:15:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3799999999999999e-06 2024-09-20T22:15:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820740+0000", 2024-09-20T22:15:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.279e-05 2024-09-20T22:15:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.535 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820749+0000", 2024-09-20T22:15:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8000000000000004e-06 2024-09-20T22:15:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820942+0000", 2024-09-20T22:15:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000192263 2024-09-20T22:15:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820953+0000", 2024-09-20T22:15:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1953000000000001e-05 2024-09-20T22:15:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.536 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.974865+0000", 2024-09-20T22:15:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.15391142 2024-09-20T22:15:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.974939+0000", 2024-09-20T22:15:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3798000000000001e-05 2024-09-20T22:15:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,6", 2024-09-20T22:15:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.975019+0000", 2024-09-20T22:15:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0692999999999995e-05 2024-09-20T22:15:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.977853+0000", 2024-09-20T22:15:51.537 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028332269999999998 2024-09-20T22:15:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982635+0000", 2024-09-20T22:15:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0047819840000000004 2024-09-20T22:15:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982688+0000", 2024-09-20T22:15:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.384e-05 2024-09-20T22:15:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:15:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982747+0000", 2024-09-20T22:15:51.538 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9004e-05 2024-09-20T22:15:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982881+0000", 2024-09-20T22:15:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013399899999999999 2024-09-20T22:15:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:227641 7.b 7:d29062b1:::benchmark_data_smithi060_78800_object94608:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e146)", 2024-09-20T22:15:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:15:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 536.26120463699999, 2024-09-20T22:15:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2739681169999999, 2024-09-20T22:15:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.539 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:15:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:15:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:15:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 227641 2024-09-20T22:15:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:15:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:15:51.540 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820771+0000", 2024-09-20T22:15:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3900000000000005e-07 2024-09-20T22:15:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820775+0000", 2024-09-20T22:15:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4589999999999998e-06 2024-09-20T22:15:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.541 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820783+0000", 2024-09-20T22:15:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8949999999999998e-06 2024-09-20T22:15:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820788+0000", 2024-09-20T22:15:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5759999999999996e-06 2024-09-20T22:15:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820979+0000", 2024-09-20T22:15:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019122 2024-09-20T22:15:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.542 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:54.820988+0000", 2024-09-20T22:15:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1480000000000006e-06 2024-09-20T22:15:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.977915+0000", 2024-09-20T22:15:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.156927834 2024-09-20T22:15:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.978045+0000", 2024-09-20T22:15:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012967 2024-09-20T22:15:51.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,6", 2024-09-20T22:15:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.978183+0000", 2024-09-20T22:15:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013767900000000001 2024-09-20T22:15:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982676+0000", 2024-09-20T22:15:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0044928210000000001 2024-09-20T22:15:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:55.982891+0000", 2024-09-20T22:15:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00021542299999999999 2024-09-20T22:15:51.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.094638+0000", 2024-09-20T22:15:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.111746554 2024-09-20T22:15:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:15:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.094677+0000", 2024-09-20T22:15:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9069000000000002e-05 2024-09-20T22:15:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:06:56.094739+0000", 2024-09-20T22:15:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2230000000000006e-05 2024-09-20T22:15:51.545 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105650 8.14 8:29da2845:::benchmark_data_smithi060_111051_object105649:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:15:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.545671+0000", 2024-09-20T22:15:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 402.53630441199999, 2024-09-20T22:15:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.038415869, 2024-09-20T22:15:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:15:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:15:51.546 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:15:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105650 2024-09-20T22:15:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545671+0000", 2024-09-20T22:15:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545671+0000", 2024-09-20T22:15:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.547 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545673+0000", 2024-09-20T22:15:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.17e-06 2024-09-20T22:15:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545723+0000", 2024-09-20T22:15:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9509000000000003e-05 2024-09-20T22:15:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545741+0000", 2024-09-20T22:15:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8634000000000002e-05 2024-09-20T22:15:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545751+0000", 2024-09-20T22:15:51.548 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4420000000000004e-06 2024-09-20T22:15:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.707696+0000", 2024-09-20T22:15:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.16194560099999999 2024-09-20T22:15:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.707706+0000", 2024-09-20T22:15:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6860000000000001e-06 2024-09-20T22:15:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584005+0000", 2024-09-20T22:15:51.549 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87629895400000002 2024-09-20T22:15:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584087+0000", 2024-09-20T22:15:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1872999999999994e-05 2024-09-20T22:15:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105665 8.18 8:1b9c2a37:::benchmark_data_smithi060_111051_object105664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:15:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.556635+0000", 2024-09-20T22:15:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 402.52534038300001, 2024-09-20T22:15:51.550 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.051209888, 2024-09-20T22:15:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:15:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:15:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:15:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105665 2024-09-20T22:15:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556635+0000", 2024-09-20T22:15:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556635+0000", 2024-09-20T22:15:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556638+0000", 2024-09-20T22:15:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.6400000000000001e-06 2024-09-20T22:15:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556714+0000", 2024-09-20T22:15:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6594999999999994e-05 2024-09-20T22:15:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.552 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556736+0000", 2024-09-20T22:15:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1662e-05 2024-09-20T22:15:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556751+0000", 2024-09-20T22:15:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4941e-05 2024-09-20T22:15:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.750414+0000", 2024-09-20T22:15:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.193663171 2024-09-20T22:15:51.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.750424+0000", 2024-09-20T22:15:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0397999999999999e-05 2024-09-20T22:15:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.596184+0000", 2024-09-20T22:15:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.84575938799999995 2024-09-20T22:15:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.596303+0000", 2024-09-20T22:15:51.554 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000119503 2024-09-20T22:15:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:15:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.596588+0000", 2024-09-20T22:15:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00028446100000000001 2024-09-20T22:15:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.603603+0000", 2024-09-20T22:15:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0070155649999999996 2024-09-20T22:15:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607589+0000", 2024-09-20T22:15:51.555 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0039854110000000003 2024-09-20T22:15:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607743+0000", 2024-09-20T22:15:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015391499999999999 2024-09-20T22:15:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:15:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607802+0000", 2024-09-20T22:15:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9472000000000003e-05 2024-09-20T22:15:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607845+0000", 2024-09-20T22:15:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2765999999999997e-05 2024-09-20T22:15:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.556 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105673 8.14 8:2f269e39:::benchmark_data_smithi060_111051_object105672:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:15:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.761620+0000", 2024-09-20T22:15:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 402.32035582999998, 2024-09-20T22:15:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.83821859799999998, 2024-09-20T22:15:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:15:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:15:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:15:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105673 2024-09-20T22:15:51.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761620+0000", 2024-09-20T22:15:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761620+0000", 2024-09-20T22:15:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.558 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761621+0000", 2024-09-20T22:15:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.392e-06 2024-09-20T22:15:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761674+0000", 2024-09-20T22:15:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2677999999999999e-05 2024-09-20T22:15:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761686+0000", 2024-09-20T22:15:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1926000000000001e-05 2024-09-20T22:15:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761697+0000", 2024-09-20T22:15:51.559 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1364e-05 2024-09-20T22:15:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761770+0000", 2024-09-20T22:15:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2904999999999994e-05 2024-09-20T22:15:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761777+0000", 2024-09-20T22:15:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8029999999999999e-06 2024-09-20T22:15:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584113+0000", 2024-09-20T22:15:51.560 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.82233672499999999 2024-09-20T22:15:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584256+0000", 2024-09-20T22:15:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000142686 2024-09-20T22:15:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,4", 2024-09-20T22:15:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584528+0000", 2024-09-20T22:15:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00027199499999999997 2024-09-20T22:15:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.590292+0000", 2024-09-20T22:15:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.005764333 2024-09-20T22:15:51.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.590315+0000", 2024-09-20T22:15:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2768e-05 2024-09-20T22:15:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.599740+0000", 2024-09-20T22:15:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0094248200000000004 2024-09-20T22:15:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:15:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.599795+0000", 2024-09-20T22:15:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5108000000000002e-05 2024-09-20T22:15:51.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.599838+0000", 2024-09-20T22:15:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3095e-05 2024-09-20T22:15:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:106978 8.19 8:9e674fff:::benchmark_data_smithi060_111051_object106977:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:15:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.576265+0000", 2024-09-20T22:15:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 400.50571071600001, 2024-09-20T22:15:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9488434429999999, 2024-09-20T22:15:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.563 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:15:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:15:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:15:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 106978 2024-09-20T22:15:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576265+0000", 2024-09-20T22:15:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576265+0000", 2024-09-20T22:15:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576267+0000", 2024-09-20T22:15:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0509999999999999e-06 2024-09-20T22:15:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576299+0000", 2024-09-20T22:15:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1962000000000003e-05 2024-09-20T22:15:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576312+0000", 2024-09-20T22:15:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3423e-05 2024-09-20T22:15:51.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576322+0000", 2024-09-20T22:15:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0118e-05 2024-09-20T22:15:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576468+0000", 2024-09-20T22:15:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000145575 2024-09-20T22:15:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576481+0000", 2024-09-20T22:15:51.566 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3023999999999999e-05 2024-09-20T22:15:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410333+0000", 2024-09-20T22:15:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.83385187800000005 2024-09-20T22:15:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410345+0000", 2024-09-20T22:15:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.258e-05 2024-09-20T22:15:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.503436+0000", 2024-09-20T22:15:51.567 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.093091088 2024-09-20T22:15:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.503599+0000", 2024-09-20T22:15:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00016286999999999999 2024-09-20T22:15:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4,6", 2024-09-20T22:15:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.505264+0000", 2024-09-20T22:15:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001664591 2024-09-20T22:15:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.508504+0000", 2024-09-20T22:15:51.568 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0032404920000000002 2024-09-20T22:15:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.508547+0000", 2024-09-20T22:15:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2734000000000001e-05 2024-09-20T22:15:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.513180+0000", 2024-09-20T22:15:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046330490000000002 2024-09-20T22:15:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.524891+0000", 2024-09-20T22:15:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.011710379999999999 2024-09-20T22:15:51.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:15:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.524966+0000", 2024-09-20T22:15:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5290999999999995e-05 2024-09-20T22:15:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.525108+0000", 2024-09-20T22:15:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000142337 2024-09-20T22:15:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.570 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:107002 8.11 8.e0f52171 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:15:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.617682+0000", 2024-09-20T22:15:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 400.46429321800002, 2024-09-20T22:15:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79265608300000001, 2024-09-20T22:15:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:15:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:15:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:15:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 107002 2024-09-20T22:15:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.571 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617682+0000", 2024-09-20T22:15:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617682+0000", 2024-09-20T22:15:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617684+0000", 2024-09-20T22:15:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1299999999999999e-06 2024-09-20T22:15:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617725+0000", 2024-09-20T22:15:51.572 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0839e-05 2024-09-20T22:15:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617738+0000", 2024-09-20T22:15:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2310000000000001e-05 2024-09-20T22:15:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617746+0000", 2024-09-20T22:15:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3939999999999996e-06 2024-09-20T22:15:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617856+0000", 2024-09-20T22:15:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000109917 2024-09-20T22:15:51.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617878+0000", 2024-09-20T22:15:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2495999999999999e-05 2024-09-20T22:15:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410330+0000", 2024-09-20T22:15:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79245202299999995 2024-09-20T22:15:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410338+0000", 2024-09-20T22:15:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9740000000000002e-06 2024-09-20T22:15:51.574 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163158 8.11 8:8e0bce89:::benchmark_data_smithi060_111051_object43159:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:15:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:08.839869+0000", 2024-09-20T22:15:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 342.242106777, 2024-09-20T22:15:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.65193807500000001, 2024-09-20T22:15:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:15:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:15:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:15:51.575 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163158 2024-09-20T22:15:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839869+0000", 2024-09-20T22:15:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839869+0000", 2024-09-20T22:15:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.576 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839870+0000", 2024-09-20T22:15:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5710000000000001e-06 2024-09-20T22:15:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839876+0000", 2024-09-20T22:15:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3820000000000003e-06 2024-09-20T22:15:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839890+0000", 2024-09-20T22:15:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3952e-05 2024-09-20T22:15:51.577 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839897+0000", 2024-09-20T22:15:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4150000000000002e-06 2024-09-20T22:15:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839956+0000", 2024-09-20T22:15:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9157999999999998e-05 2024-09-20T22:15:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839967+0000", 2024-09-20T22:15:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1019000000000001e-05 2024-09-20T22:15:51.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477635+0000", 2024-09-20T22:15:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.63766765700000005 2024-09-20T22:15:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477737+0000", 2024-09-20T22:15:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010265899999999999 2024-09-20T22:15:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:15:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477939+0000", 2024-09-20T22:15:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020106900000000001 2024-09-20T22:15:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.579 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.482557+0000", 2024-09-20T22:15:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046186830000000002 2024-09-20T22:15:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491495+0000", 2024-09-20T22:15:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0089377699999999994 2024-09-20T22:15:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491673+0000", 2024-09-20T22:15:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017837500000000001 2024-09-20T22:15:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.580 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:15:51.581 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491762+0000", 2024-09-20T22:15:51.581 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8555999999999997e-05 2024-09-20T22:15:51.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.581 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.581 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.581 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491807+0000", 2024-09-20T22:15:51.581 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4808999999999997e-05 2024-09-20T22:15:51.581 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.581 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.581 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.581 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.581 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163289 8.11 8:8c47e2c8:::benchmark_data_smithi060_111051_object43290:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:15:51.581 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:08.954133+0000", 2024-09-20T22:15:51.581 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 342.12784214300001, 2024-09-20T22:15:51.581 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.53808307099999997, 2024-09-20T22:15:51.582 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.582 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.582 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:15:51.582 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.582 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:15:51.582 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:15:51.582 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163289 2024-09-20T22:15:51.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.582 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.582 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.582 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.582 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954133+0000", 2024-09-20T22:15:51.582 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.582 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.583 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.583 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954133+0000", 2024-09-20T22:15:51.583 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.583 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.583 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.583 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954134+0000", 2024-09-20T22:15:51.583 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2900000000000002e-07 2024-09-20T22:15:51.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.583 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.583 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.583 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954138+0000", 2024-09-20T22:15:51.583 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5260000000000002e-06 2024-09-20T22:15:51.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.583 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.583 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.584 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954147+0000", 2024-09-20T22:15:51.584 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2010000000000005e-06 2024-09-20T22:15:51.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.584 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.584 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.584 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954153+0000", 2024-09-20T22:15:51.584 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0610000000000001e-06 2024-09-20T22:15:51.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.584 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.584 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.584 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954209+0000", 2024-09-20T22:15:51.584 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5767999999999999e-05 2024-09-20T22:15:51.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.584 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.584 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.585 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954216+0000", 2024-09-20T22:15:51.585 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8099999999999998e-06 2024-09-20T22:15:51.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.585 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.585 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.585 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.478998+0000", 2024-09-20T22:15:51.585 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.52478124900000001 2024-09-20T22:15:51.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.585 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.585 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:51.585 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.479095+0000", 2024-09-20T22:15:51.585 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7125000000000003e-05 2024-09-20T22:15:51.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.585 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.585 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:15:51.585 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.479291+0000", 2024-09-20T22:15:51.586 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019642100000000001 2024-09-20T22:15:51.586 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.586 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.586 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:51.586 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.482759+0000", 2024-09-20T22:15:51.586 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0034672689999999998 2024-09-20T22:15:51.586 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.586 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.586 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.586 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491864+0000", 2024-09-20T22:15:51.586 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0091058630000000005 2024-09-20T22:15:51.586 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.586 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.586 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.586 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491961+0000", 2024-09-20T22:15:51.586 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6717000000000003e-05 2024-09-20T22:15:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:15:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492014+0000", 2024-09-20T22:15:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2877000000000002e-05 2024-09-20T22:15:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492216+0000", 2024-09-20T22:15:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020235500000000001 2024-09-20T22:15:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.587 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163324 8.11 8:8e312e9e:::benchmark_data_smithi060_111051_object43325:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:15:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:08.983848+0000", 2024-09-20T22:15:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 342.09812722100003, 2024-09-20T22:15:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.50861305999999995, 2024-09-20T22:15:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:15:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:15:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:15:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163324 2024-09-20T22:15:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.588 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983848+0000", 2024-09-20T22:15:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983848+0000", 2024-09-20T22:15:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983849+0000", 2024-09-20T22:15:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.11e-06 2024-09-20T22:15:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983853+0000", 2024-09-20T22:15:51.589 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0439999999999997e-06 2024-09-20T22:15:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983863+0000", 2024-09-20T22:15:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6430000000000007e-06 2024-09-20T22:15:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983868+0000", 2024-09-20T22:15:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1000000000000003e-06 2024-09-20T22:15:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983919+0000", 2024-09-20T22:15:51.590 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0433999999999999e-05 2024-09-20T22:15:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983928+0000", 2024-09-20T22:15:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3659999999999998e-06 2024-09-20T22:15:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.480646+0000", 2024-09-20T22:15:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.49671833999999998 2024-09-20T22:15:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.480768+0000", 2024-09-20T22:15:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012145 2024-09-20T22:15:51.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:15:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.481025+0000", 2024-09-20T22:15:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025713299999999999 2024-09-20T22:15:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.488496+0000", 2024-09-20T22:15:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0074709540000000001 2024-09-20T22:15:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492276+0000", 2024-09-20T22:15:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0037803670000000002 2024-09-20T22:15:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.592 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492350+0000", 2024-09-20T22:15:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3436999999999995e-05 2024-09-20T22:15:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:15:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492421+0000", 2024-09-20T22:15:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1303000000000005e-05 2024-09-20T22:15:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492461+0000", 2024-09-20T22:15:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0379000000000001e-05 2024-09-20T22:15:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163358 8.11 8:8b7ace29:::benchmark_data_smithi060_111051_object43359:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:15:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.014358+0000", 2024-09-20T22:15:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 342.06761723599999, 2024-09-20T22:15:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.47828052599999998, 2024-09-20T22:15:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:15:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:15:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:15:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163358 2024-09-20T22:15:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.594 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014358+0000", 2024-09-20T22:15:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014358+0000", 2024-09-20T22:15:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014359+0000", 2024-09-20T22:15:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0669999999999999e-06 2024-09-20T22:15:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.595 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.596 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014364+0000", 2024-09-20T22:15:51.596 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0220000000000004e-06 2024-09-20T22:15:51.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.596 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.596 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.596 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014374+0000", 2024-09-20T22:15:51.596 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9599999999999995e-06 2024-09-20T22:15:51.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.596 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.596 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.596 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014388+0000", 2024-09-20T22:15:51.596 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3413e-05 2024-09-20T22:15:51.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.596 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.596 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.596 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014447+0000", 2024-09-20T22:15:51.596 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9698999999999999e-05 2024-09-20T22:15:51.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.597 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.597 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.597 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014455+0000", 2024-09-20T22:15:51.597 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.072e-06 2024-09-20T22:15:51.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.597 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.597 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.597 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.482921+0000", 2024-09-20T22:15:51.597 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.46846513200000001 2024-09-20T22:15:51.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.597 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.597 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:51.597 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483020+0000", 2024-09-20T22:15:51.597 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9319000000000002e-05 2024-09-20T22:15:51.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.598 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.598 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:15:51.598 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483251+0000", 2024-09-20T22:15:51.598 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00023124300000000001 2024-09-20T22:15:51.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.598 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.598 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:51.598 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491608+0000", 2024-09-20T22:15:51.598 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0083564190000000003 2024-09-20T22:15:51.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.598 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.598 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.598 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492507+0000", 2024-09-20T22:15:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00089972900000000005 2024-09-20T22:15:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492546+0000", 2024-09-20T22:15:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9156999999999998e-05 2024-09-20T22:15:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:15:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492602+0000", 2024-09-20T22:15:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5114999999999997e-05 2024-09-20T22:15:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.599 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492639+0000", 2024-09-20T22:15:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7178999999999998e-05 2024-09-20T22:15:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163372 8.11 8:8b6ec81f:::benchmark_data_smithi060_111051_object43373:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:15:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.027195+0000", 2024-09-20T22:15:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 342.05478042800002, 2024-09-20T22:15:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.46568983899999999, 2024-09-20T22:15:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:15:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:15:51.600 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:15:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163372 2024-09-20T22:15:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027195+0000", 2024-09-20T22:15:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027195+0000", 2024-09-20T22:15:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.601 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027196+0000", 2024-09-20T22:15:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0599999999999999e-07 2024-09-20T22:15:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027200+0000", 2024-09-20T22:15:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9280000000000003e-06 2024-09-20T22:15:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027210+0000", 2024-09-20T22:15:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0035e-05 2024-09-20T22:15:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027217+0000", 2024-09-20T22:15:51.602 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6450000000000005e-06 2024-09-20T22:15:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027281+0000", 2024-09-20T22:15:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3281000000000006e-05 2024-09-20T22:15:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027291+0000", 2024-09-20T22:15:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0268999999999999e-05 2024-09-20T22:15:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.484647+0000", 2024-09-20T22:15:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.45735564499999998 2024-09-20T22:15:51.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.484745+0000", 2024-09-20T22:15:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8712e-05 2024-09-20T22:15:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:15:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.484940+0000", 2024-09-20T22:15:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019504900000000001 2024-09-20T22:15:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491619+0000", 2024-09-20T22:15:51.604 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0066782090000000001 2024-09-20T22:15:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492716+0000", 2024-09-20T22:15:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001097701 2024-09-20T22:15:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492778+0000", 2024-09-20T22:15:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1538000000000006e-05 2024-09-20T22:15:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:15:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492830+0000", 2024-09-20T22:15:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1997999999999999e-05 2024-09-20T22:15:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.605 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492885+0000", 2024-09-20T22:15:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5022999999999998e-05 2024-09-20T22:15:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163373 8.11 8:8d07a686:::benchmark_data_smithi060_111051_object43374:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:15:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.027847+0000", 2024-09-20T22:15:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 342.05412802000001, 2024-09-20T22:15:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.465200326, 2024-09-20T22:15:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.606 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:15:51.607 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.607 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:15:51.607 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:15:51.607 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163373 2024-09-20T22:15:51.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.607 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.607 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.607 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.607 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027847+0000", 2024-09-20T22:15:51.607 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.607 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.607 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.607 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027847+0000", 2024-09-20T22:15:51.607 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.608 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.608 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.608 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027848+0000", 2024-09-20T22:15:51.608 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3600000000000003e-07 2024-09-20T22:15:51.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.608 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.608 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.608 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027852+0000", 2024-09-20T22:15:51.608 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6330000000000002e-06 2024-09-20T22:15:51.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.608 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.608 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.608 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027860+0000", 2024-09-20T22:15:51.608 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2789999999999998e-06 2024-09-20T22:15:51.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.609 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.609 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.609 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027866+0000", 2024-09-20T22:15:51.609 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1070000000000001e-06 2024-09-20T22:15:51.609 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.609 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.609 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.609 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027897+0000", 2024-09-20T22:15:51.609 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0722000000000002e-05 2024-09-20T22:15:51.609 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.609 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.609 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.609 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027901+0000", 2024-09-20T22:15:51.609 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1649999999999999e-06 2024-09-20T22:15:51.609 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.609 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.610 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.610 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.486658+0000", 2024-09-20T22:15:51.610 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.45875733499999999 2024-09-20T22:15:51.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.610 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.610 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:51.610 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.486767+0000", 2024-09-20T22:15:51.610 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000108385 2024-09-20T22:15:51.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.610 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.610 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:15:51.610 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487120+0000", 2024-09-20T22:15:51.610 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00035312499999999998 2024-09-20T22:15:51.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.610 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.611 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:51.611 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491877+0000", 2024-09-20T22:15:51.611 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0047569170000000003 2024-09-20T22:15:51.611 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.611 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.611 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.611 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492928+0000", 2024-09-20T22:15:51.611 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001051633 2024-09-20T22:15:51.611 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.611 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.611 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.611 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492967+0000", 2024-09-20T22:15:51.611 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.8781e-05 2024-09-20T22:15:51.611 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.611 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.611 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:15:51.612 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493013+0000", 2024-09-20T22:15:51.612 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5555e-05 2024-09-20T22:15:51.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.612 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.612 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.612 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493048+0000", 2024-09-20T22:15:51.612 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5052999999999998e-05 2024-09-20T22:15:51.612 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.612 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.612 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.612 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.612 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163388 8.11 8:893f1034:::benchmark_data_smithi060_111051_object43389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:15:51.612 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.050114+0000", 2024-09-20T22:15:51.612 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 342.03186131000001, 2024-09-20T22:15:51.612 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.44308652799999998, 2024-09-20T22:15:51.613 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.613 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.613 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:15:51.613 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.613 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:15:51.613 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:15:51.613 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163388 2024-09-20T22:15:51.613 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.613 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.613 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.613 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.613 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050114+0000", 2024-09-20T22:15:51.613 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.613 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.613 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.613 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.614 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050114+0000", 2024-09-20T22:15:51.614 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.614 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.614 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.614 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050115+0000", 2024-09-20T22:15:51.614 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2500000000000005e-07 2024-09-20T22:15:51.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.614 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.614 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.614 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050119+0000", 2024-09-20T22:15:51.614 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.698e-06 2024-09-20T22:15:51.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.614 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.614 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.615 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050128+0000", 2024-09-20T22:15:51.615 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5820000000000008e-06 2024-09-20T22:15:51.615 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.615 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.615 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.615 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050133+0000", 2024-09-20T22:15:51.615 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9060000000000001e-06 2024-09-20T22:15:51.615 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.615 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.615 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.615 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050169+0000", 2024-09-20T22:15:51.615 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5984e-05 2024-09-20T22:15:51.615 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.615 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.615 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.615 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050174+0000", 2024-09-20T22:15:51.616 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0180000000000001e-06 2024-09-20T22:15:51.616 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.616 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.616 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.616 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.488568+0000", 2024-09-20T22:15:51.616 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.43839391 2024-09-20T22:15:51.616 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.616 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.616 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:51.616 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.488685+0000", 2024-09-20T22:15:51.616 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000117163 2024-09-20T22:15:51.616 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.616 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.616 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:15:51.616 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.489162+0000", 2024-09-20T22:15:51.616 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00047712600000000003 2024-09-20T22:15:51.617 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.617 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.617 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:51.617 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491884+0000", 2024-09-20T22:15:51.617 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0027215329999999999 2024-09-20T22:15:51.617 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.617 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.617 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.617 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493086+0000", 2024-09-20T22:15:51.617 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001201765 2024-09-20T22:15:51.617 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.617 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.617 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.617 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493122+0000", 2024-09-20T22:15:51.617 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6622999999999998e-05 2024-09-20T22:15:51.617 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.618 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.618 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:15:51.618 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493168+0000", 2024-09-20T22:15:51.618 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5757999999999998e-05 2024-09-20T22:15:51.618 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.618 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.618 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.618 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493201+0000", 2024-09-20T22:15:51.618 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2737000000000002e-05 2024-09-20T22:15:51.618 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.618 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.618 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.618 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.618 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.618 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163393 8.11 8:8bb06d42:::benchmark_data_smithi060_111051_object43394:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:15:51.618 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.064776+0000", 2024-09-20T22:15:51.619 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 342.01719938700001, 2024-09-20T22:15:51.619 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.43944195600000002, 2024-09-20T22:15:51.619 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.619 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.619 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:15:51.619 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.619 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:15:51.619 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:15:51.619 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163393 2024-09-20T22:15:51.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.619 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.619 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.619 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.619 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064776+0000", 2024-09-20T22:15:51.619 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.620 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.620 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.620 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064776+0000", 2024-09-20T22:15:51.620 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.620 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.620 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.620 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.620 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064777+0000", 2024-09-20T22:15:51.620 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9500000000000001e-07 2024-09-20T22:15:51.620 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.620 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.620 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.620 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064781+0000", 2024-09-20T22:15:51.620 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9870000000000001e-06 2024-09-20T22:15:51.620 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.620 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.621 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.621 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064791+0000", 2024-09-20T22:15:51.621 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0417999999999999e-05 2024-09-20T22:15:51.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.621 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.621 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.621 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064797+0000", 2024-09-20T22:15:51.621 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4040000000000003e-06 2024-09-20T22:15:51.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.621 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.621 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.621 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064838+0000", 2024-09-20T22:15:51.621 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1591000000000002e-05 2024-09-20T22:15:51.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.621 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.621 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:15:51.622 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064843+0000", 2024-09-20T22:15:51.622 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9470000000000002e-06 2024-09-20T22:15:51.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.622 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.622 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.622 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493235+0000", 2024-09-20T22:15:51.622 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.42839186899999998 2024-09-20T22:15:51.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.622 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.622 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:51.622 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493336+0000", 2024-09-20T22:15:51.622 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100923 2024-09-20T22:15:51.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.622 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.622 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:15:51.623 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493659+0000", 2024-09-20T22:15:51.623 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00032341599999999998 2024-09-20T22:15:51.623 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.623 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.623 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:51.623 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.499928+0000", 2024-09-20T22:15:51.623 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0062687730000000001 2024-09-20T22:15:51.623 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.623 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.623 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.623 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.503797+0000", 2024-09-20T22:15:51.623 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0038686020000000001 2024-09-20T22:15:51.623 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.623 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.623 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.623 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.504160+0000", 2024-09-20T22:15:51.624 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00036300900000000002 2024-09-20T22:15:51.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.624 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.624 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:15:51.624 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.504192+0000", 2024-09-20T22:15:51.624 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2592000000000002e-05 2024-09-20T22:15:51.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.624 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.624 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.624 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.504218+0000", 2024-09-20T22:15:51.624 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.563e-05 2024-09-20T22:15:51.624 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.624 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.624 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.624 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.625 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:125848 9.6 9:604ae09f:::benchmark_data_smithi060_140132_object15040:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:15:51.625 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:13:30.323134+0000", 2024-09-20T22:15:51.625 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 140.758841117, 2024-09-20T22:15:51.625 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.39324260900000002, 2024-09-20T22:15:51.625 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.625 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.625 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:15:51.625 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.625 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:15:51.625 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:15:51.625 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 125848 2024-09-20T22:15:51.625 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.625 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.625 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.625 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.625 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323134+0000", 2024-09-20T22:15:51.626 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.626 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.626 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.626 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.626 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323134+0000", 2024-09-20T22:15:51.626 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.626 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.626 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.626 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.626 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323135+0000", 2024-09-20T22:15:51.626 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1399999999999997e-07 2024-09-20T22:15:51.626 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.626 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.626 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.627 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323137+0000", 2024-09-20T22:15:51.627 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1789999999999998e-06 2024-09-20T22:15:51.627 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.627 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.627 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.627 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323143+0000", 2024-09-20T22:15:51.627 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1469999999999998e-06 2024-09-20T22:15:51.627 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.627 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.627 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.627 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323149+0000", 2024-09-20T22:15:51.627 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2639999999999997e-06 2024-09-20T22:15:51.627 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.627 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.627 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.628 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323210+0000", 2024-09-20T22:15:51.628 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0537e-05 2024-09-20T22:15:51.628 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.628 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.628 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:51.628 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323333+0000", 2024-09-20T22:15:51.628 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012330500000000001 2024-09-20T22:15:51.628 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.628 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.628 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:15:51.628 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323444+0000", 2024-09-20T22:15:51.628 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000110562 2024-09-20T22:15:51.628 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.628 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.628 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:51.628 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.326284+0000", 2024-09-20T22:15:51.629 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028407409999999999 2024-09-20T22:15:51.629 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.629 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.629 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.629 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.328793+0000", 2024-09-20T22:15:51.629 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0025082680000000001 2024-09-20T22:15:51.629 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.629 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.629 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.629 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.716294+0000", 2024-09-20T22:15:51.629 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.38750077100000002 2024-09-20T22:15:51.629 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.629 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.629 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:15:51.629 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.716331+0000", 2024-09-20T22:15:51.629 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7419999999999997e-05 2024-09-20T22:15:51.630 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.630 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.630 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.630 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.716377+0000", 2024-09-20T22:15:51.630 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6001000000000002e-05 2024-09-20T22:15:51.630 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.630 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.630 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.630 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.630 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.630 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:125853 9.6 9:65bf9cb5:::benchmark_data_smithi060_140132_object15045:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:15:51.630 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:13:30.329387+0000", 2024-09-20T22:15:51.630 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 140.752588454, 2024-09-20T22:15:51.630 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.39097777700000003, 2024-09-20T22:15:51.630 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.630 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.631 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:15:51.631 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.631 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:15:51.631 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:15:51.631 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 125853 2024-09-20T22:15:51.631 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.631 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.631 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.631 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.631 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329387+0000", 2024-09-20T22:15:51.631 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.631 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.631 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.631 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.631 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329387+0000", 2024-09-20T22:15:51.632 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.632 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.632 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.632 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.632 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329388+0000", 2024-09-20T22:15:51.632 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3990000000000001e-06 2024-09-20T22:15:51.632 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.632 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.632 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.632 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329395+0000", 2024-09-20T22:15:51.632 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.161e-06 2024-09-20T22:15:51.632 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.632 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.632 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.632 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329407+0000", 2024-09-20T22:15:51.632 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2323999999999999e-05 2024-09-20T22:15:51.633 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.633 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.633 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.633 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329414+0000", 2024-09-20T22:15:51.633 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.289e-06 2024-09-20T22:15:51.633 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.633 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.633 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.633 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329463+0000", 2024-09-20T22:15:51.633 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8658000000000001e-05 2024-09-20T22:15:51.633 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.633 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.633 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:51.633 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329589+0000", 2024-09-20T22:15:51.633 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012579900000000001 2024-09-20T22:15:51.634 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.634 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.634 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:15:51.634 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329695+0000", 2024-09-20T22:15:51.634 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000105933 2024-09-20T22:15:51.634 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.634 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.634 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.634 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.335795+0000", 2024-09-20T22:15:51.634 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061008720000000002 2024-09-20T22:15:51.634 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.634 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.634 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:51.634 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.335858+0000", 2024-09-20T22:15:51.634 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2285000000000004e-05 2024-09-20T22:15:51.635 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.635 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.635 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.635 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720262+0000", 2024-09-20T22:15:51.635 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.38440416300000002 2024-09-20T22:15:51.635 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.635 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.635 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:15:51.635 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720315+0000", 2024-09-20T22:15:51.635 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2985000000000002e-05 2024-09-20T22:15:51.635 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.635 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.635 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.635 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720365+0000", 2024-09-20T22:15:51.635 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9908999999999999e-05 2024-09-20T22:15:51.635 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.636 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.636 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.636 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.636 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.636 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:125877 9.6 9:618bafd1:::benchmark_data_smithi060_140132_object15069:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:15:51.636 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:15:51.636 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 140.73688687800001, 2024-09-20T22:15:51.636 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.37546249399999998, 2024-09-20T22:15:51.636 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:51.636 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:51.636 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:15:51.636 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:51.636 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:15:51.636 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:15:51.636 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 125877 2024-09-20T22:15:51.636 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.636 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:51.637 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.637 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:51.637 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:15:51.637 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.637 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.637 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.637 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:51.637 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:15:51.637 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:51.637 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.637 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.637 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:51.637 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:15:51.637 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1999999999999999e-07 2024-09-20T22:15:51.637 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.637 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.638 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:51.638 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345092+0000", 2024-09-20T22:15:51.638 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2899999999999998e-06 2024-09-20T22:15:51.638 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.638 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.638 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:51.638 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345101+0000", 2024-09-20T22:15:51.638 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1550000000000007e-06 2024-09-20T22:15:51.638 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.638 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.638 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:51.638 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345107+0000", 2024-09-20T22:15:51.638 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2609999999999999e-06 2024-09-20T22:15:51.638 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.639 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.639 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:51.639 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345160+0000", 2024-09-20T22:15:51.639 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.287e-05 2024-09-20T22:15:51.639 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.639 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.639 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:51.639 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345269+0000", 2024-09-20T22:15:51.639 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000108795 2024-09-20T22:15:51.639 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.639 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.639 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:15:51.639 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345395+0000", 2024-09-20T22:15:51.639 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012654800000000001 2024-09-20T22:15:51.639 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.640 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.640 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:15:51.640 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.348668+0000", 2024-09-20T22:15:51.640 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0032731729999999999 2024-09-20T22:15:51.640 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.640 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.640 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.640 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.351790+0000", 2024-09-20T22:15:51.640 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003121567 2024-09-20T22:15:51.640 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.640 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.640 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:51.640 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720464+0000", 2024-09-20T22:15:51.640 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.36867422 2024-09-20T22:15:51.640 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.640 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.641 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:15:51.641 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720511+0000", 2024-09-20T22:15:51.641 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7076999999999997e-05 2024-09-20T22:15:51.641 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:51.641 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:51.641 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:15:51.641 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720551+0000", 2024-09-20T22:15:51.641 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9918e-05 2024-09-20T22:15:51.641 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.641 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.641 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.641 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:51.641 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:51.641 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:15:51.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:51.678 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:51.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:51.679 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:51.780 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:51.814 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:51.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:51.815 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:51.916 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:51.922 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 34 16 61914 61898 113.757 78.125 0.0167943 0.0087856 2024-09-20T22:15:51.958 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:51.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:51.959 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:51 smithi067 ceph-mon[33511]: 4.c scrub starts 2024-09-20T22:15:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:51 smithi067 ceph-mon[33511]: 4.c scrub ok 2024-09-20T22:15:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:51 smithi067 ceph-mon[33511]: 1.f scrub starts 2024-09-20T22:15:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:51 smithi067 ceph-mon[33511]: 1.f scrub ok 2024-09-20T22:15:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:15:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:15:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:51 smithi067 ceph-mon[33511]: 4.f scrub starts 2024-09-20T22:15:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:51 smithi067 ceph-mon[33511]: 1.c scrub starts 2024-09-20T22:15:52.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:51 smithi067 ceph-mon[33511]: 1.c scrub ok 2024-09-20T22:15:52.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:51 smithi067 ceph-mon[33511]: 4.f scrub ok 2024-09-20T22:15:52.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:51 smithi067 ceph-mon[33511]: 1.11 scrub starts 2024-09-20T22:15:52.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:51 smithi067 ceph-mon[33511]: 1.11 scrub ok 2024-09-20T22:15:52.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:51 smithi067 ceph-mon[33511]: 1.b scrub starts 2024-09-20T22:15:52.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:51 smithi067 ceph-mon[33511]: 1.b scrub ok 2024-09-20T22:15:52.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:51 smithi067 ceph-mon[33511]: osdmap e390: 8 total, 8 up, 8 in 2024-09-20T22:15:52.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[29694]: 4.c scrub starts 2024-09-20T22:15:52.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[29694]: 4.c scrub ok 2024-09-20T22:15:52.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[29694]: 1.f scrub starts 2024-09-20T22:15:52.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[29694]: 1.f scrub ok 2024-09-20T22:15:52.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:15:52.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:15:52.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[29694]: 4.f scrub starts 2024-09-20T22:15:52.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[29694]: 1.c scrub starts 2024-09-20T22:15:52.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[29694]: 1.c scrub ok 2024-09-20T22:15:52.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[29694]: 4.f scrub ok 2024-09-20T22:15:52.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[29694]: 1.11 scrub starts 2024-09-20T22:15:52.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[29694]: 1.11 scrub ok 2024-09-20T22:15:52.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[29694]: 1.b scrub starts 2024-09-20T22:15:52.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[29694]: 1.b scrub ok 2024-09-20T22:15:52.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[29694]: osdmap e390: 8 total, 8 up, 8 in 2024-09-20T22:15:52.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[40290]: 4.c scrub starts 2024-09-20T22:15:52.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[40290]: 4.c scrub ok 2024-09-20T22:15:52.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[40290]: 1.f scrub starts 2024-09-20T22:15:52.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[40290]: 1.f scrub ok 2024-09-20T22:15:52.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:15:52.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:15:52.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[40290]: 4.f scrub starts 2024-09-20T22:15:52.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[40290]: 1.c scrub starts 2024-09-20T22:15:52.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[40290]: 1.c scrub ok 2024-09-20T22:15:52.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[40290]: 4.f scrub ok 2024-09-20T22:15:52.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[40290]: 1.11 scrub starts 2024-09-20T22:15:52.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[40290]: 1.11 scrub ok 2024-09-20T22:15:52.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[40290]: 1.b scrub starts 2024-09-20T22:15:52.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[40290]: 1.b scrub ok 2024-09-20T22:15:52.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:51 smithi060 ceph-mon[40290]: osdmap e390: 8 total, 8 up, 8 in 2024-09-20T22:15:52.060 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:52.099 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:52.100 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:52.100 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:52.201 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:52.231 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:52.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:52.231 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:52.332 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:52.370 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:52.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:52.371 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:52.472 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:52.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:52.502 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:52.503 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:52.603 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:52.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:52.640 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:52.640 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:52.741 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:52.763 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:15:52 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:15:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:52 smithi067 ceph-mon[33511]: 1.10 scrub starts 2024-09-20T22:15:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:52 smithi067 ceph-mon[33511]: 1.10 scrub ok 2024-09-20T22:15:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:52 smithi067 ceph-mon[33511]: 1.e scrub starts 2024-09-20T22:15:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:52 smithi067 ceph-mon[33511]: 1.e scrub ok 2024-09-20T22:15:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:52 smithi067 ceph-mon[33511]: 1.d scrub starts 2024-09-20T22:15:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:52 smithi067 ceph-mon[33511]: 1.d scrub ok 2024-09-20T22:15:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:52 smithi067 ceph-mon[33511]: pgmap v696: 168 pgs: 3 remapped+peering, 165 active+clean; 3.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 4.7 KiB/s rd, 63 MiB/s wr, 1.02k op/s 2024-09-20T22:15:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:52 smithi067 ceph-mon[33511]: 1.13 scrub starts 2024-09-20T22:15:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:52 smithi067 ceph-mon[33511]: 1.13 scrub ok 2024-09-20T22:15:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:52 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:15:52.777 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:52.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:52.778 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:52.879 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:52.915 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:52.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:52.916 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:52.922 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 35 16 63031 63015 112.501 69.8125 0.00467387 0.00888354 2024-09-20T22:15:52.989 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T22:15:53.018 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:53.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[29694]: 1.10 scrub starts 2024-09-20T22:15:53.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[29694]: 1.10 scrub ok 2024-09-20T22:15:53.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[29694]: 1.e scrub starts 2024-09-20T22:15:53.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[29694]: 1.e scrub ok 2024-09-20T22:15:53.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[29694]: 1.d scrub starts 2024-09-20T22:15:53.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[29694]: 1.d scrub ok 2024-09-20T22:15:53.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[29694]: pgmap v696: 168 pgs: 3 remapped+peering, 165 active+clean; 3.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 4.7 KiB/s rd, 63 MiB/s wr, 1.02k op/s 2024-09-20T22:15:53.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[29694]: 1.13 scrub starts 2024-09-20T22:15:53.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[29694]: 1.13 scrub ok 2024-09-20T22:15:53.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:15:53.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[40290]: 1.10 scrub starts 2024-09-20T22:15:53.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[40290]: 1.10 scrub ok 2024-09-20T22:15:53.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[40290]: 1.e scrub starts 2024-09-20T22:15:53.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[40290]: 1.e scrub ok 2024-09-20T22:15:53.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[40290]: 1.d scrub starts 2024-09-20T22:15:53.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[40290]: 1.d scrub ok 2024-09-20T22:15:53.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[40290]: pgmap v696: 168 pgs: 3 remapped+peering, 165 active+clean; 3.6 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 4.7 KiB/s rd, 63 MiB/s wr, 1.02k op/s 2024-09-20T22:15:53.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[40290]: 1.13 scrub starts 2024-09-20T22:15:53.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[40290]: 1.13 scrub ok 2024-09-20T22:15:53.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:52 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:15:53.056 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:53.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:53.058 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:53.159 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:53.191 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:53.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:53.192 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:53.292 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:53.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:53.336 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:53.336 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:53.437 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:53.467 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:15:53.479 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:53.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:53.480 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:53.581 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:53.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:53.617 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:53.618 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:53.718 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:53.760 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:53.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:53.761 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:53.799 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:53 smithi067 ceph-mon[33511]: 1.12 scrub starts 2024-09-20T22:15:53.800 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:53 smithi067 ceph-mon[33511]: 1.12 scrub ok 2024-09-20T22:15:53.800 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:53 smithi067 ceph-mon[33511]: 6.3 scrub starts 2024-09-20T22:15:53.800 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:53 smithi067 ceph-mon[33511]: 6.3 scrub ok 2024-09-20T22:15:53.800 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:53 smithi067 ceph-mon[33511]: 6.c scrub starts 2024-09-20T22:15:53.800 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:53 smithi067 ceph-mon[33511]: 6.c scrub ok 2024-09-20T22:15:53.800 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:53 smithi067 ceph-mon[33511]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-20T22:15:53.861 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:53.899 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:53.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:53.900 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:53.923 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 36 16 64341 64325 111.651 81.875 0.0110522 0.0089498 2024-09-20T22:15:54.001 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:54.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:53 smithi060 ceph-mon[29694]: 1.12 scrub starts 2024-09-20T22:15:54.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:53 smithi060 ceph-mon[29694]: 1.12 scrub ok 2024-09-20T22:15:54.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:53 smithi060 ceph-mon[29694]: 6.3 scrub starts 2024-09-20T22:15:54.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:53 smithi060 ceph-mon[29694]: 6.3 scrub ok 2024-09-20T22:15:54.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:53 smithi060 ceph-mon[29694]: 6.c scrub starts 2024-09-20T22:15:54.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:53 smithi060 ceph-mon[29694]: 6.c scrub ok 2024-09-20T22:15:54.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:53 smithi060 ceph-mon[29694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-20T22:15:54.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:53 smithi060 ceph-mon[40290]: 1.12 scrub starts 2024-09-20T22:15:54.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:53 smithi060 ceph-mon[40290]: 1.12 scrub ok 2024-09-20T22:15:54.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:53 smithi060 ceph-mon[40290]: 6.3 scrub starts 2024-09-20T22:15:54.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:53 smithi060 ceph-mon[40290]: 6.3 scrub ok 2024-09-20T22:15:54.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:53 smithi060 ceph-mon[40290]: 6.c scrub starts 2024-09-20T22:15:54.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:53 smithi060 ceph-mon[40290]: 6.c scrub ok 2024-09-20T22:15:54.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:53 smithi060 ceph-mon[40290]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive) 2024-09-20T22:15:54.042 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:54.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:54.044 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:54.120 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:15:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:15:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:64585 10.a 10:5bb83716:::benchmark_data_smithi060_167766_object64584:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-20T22:15:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:15:54.110781+0000", 2024-09-20T22:15:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0063246630000000003, 2024-09-20T22:15:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0063599859999999998, 2024-09-20T22:15:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:15:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:15:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:15:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 64585 2024-09-20T22:15:54.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110781+0000", 2024-09-20T22:15:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110781+0000", 2024-09-20T22:15:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110782+0000", 2024-09-20T22:15:54.122 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.004e-06 2024-09-20T22:15:54.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.123 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:54.123 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110846+0000", 2024-09-20T22:15:54.123 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3738999999999994e-05 2024-09-20T22:15:54.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.123 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:54.123 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110932+0000", 2024-09-20T22:15:54.123 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5857000000000001e-05 2024-09-20T22:15:54.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.123 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:54.123 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110944+0000", 2024-09-20T22:15:54.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1827e-05 2024-09-20T22:15:54.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.124 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:54.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.113074+0000", 2024-09-20T22:15:54.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0021304509999999998 2024-09-20T22:15:54.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.124 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:54.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.113194+0000", 2024-09-20T22:15:54.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000119461 2024-09-20T22:15:54.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.124 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,5", 2024-09-20T22:15:54.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.113320+0000", 2024-09-20T22:15:54.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012598000000000001 2024-09-20T22:15:54.125 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:54.125 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:54.125 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:54.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.125 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:64586 10.d 10:bd1219ac:::benchmark_data_smithi060_167766_object64585:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-20T22:15:54.125 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:15:54.112087+0000", 2024-09-20T22:15:54.125 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0050194599999999999, 2024-09-20T22:15:54.125 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0051352580000000002, 2024-09-20T22:15:54.125 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:54.125 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:54.125 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:15:54.125 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:54.126 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:15:54.126 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:15:54.126 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 64586 2024-09-20T22:15:54.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.126 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:54.126 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.126 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:54.126 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.112087+0000", 2024-09-20T22:15:54.126 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:54.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.126 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.126 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:54.126 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.112087+0000", 2024-09-20T22:15:54.126 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:54.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.127 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:54.127 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.112088+0000", 2024-09-20T22:15:54.127 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9099999999999991e-07 2024-09-20T22:15:54.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.127 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:54.127 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.112148+0000", 2024-09-20T22:15:54.127 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0117000000000001e-05 2024-09-20T22:15:54.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.127 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:54.128 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.112160+0000", 2024-09-20T22:15:54.128 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2490000000000001e-05 2024-09-20T22:15:54.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.128 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:54.128 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.112169+0000", 2024-09-20T22:15:54.128 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3890000000000005e-06 2024-09-20T22:15:54.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.128 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:54.128 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.112233+0000", 2024-09-20T22:15:54.128 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3888999999999998e-05 2024-09-20T22:15:54.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.129 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:54.129 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.112363+0000", 2024-09-20T22:15:54.129 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013051100000000001 2024-09-20T22:15:54.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.129 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.129 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:15:54.129 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.112509+0000", 2024-09-20T22:15:54.129 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014578599999999999 2024-09-20T22:15:54.129 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:54.129 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:54.129 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:54.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.129 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.129 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop_reply(client.16005.0:64579 10.7 e390/370)", 2024-09-20T22:15:54.129 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:15:54.114151+0000", 2024-09-20T22:15:54.130 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0029553040000000002, 2024-09-20T22:15:54.130 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0031380969999999998, 2024-09-20T22:15:54.130 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:54.130 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:54.130 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "queued for pg", 2024-09-20T22:15:54.130 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:54.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.130 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:54.130 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114151+0000", 2024-09-20T22:15:54.130 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:54.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.130 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:54.131 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114151+0000", 2024-09-20T22:15:54.131 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:54.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.131 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:54.131 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114152+0000", 2024-09-20T22:15:54.131 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6499999999999998e-07 2024-09-20T22:15:54.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.131 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:54.131 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114156+0000", 2024-09-20T22:15:54.131 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1489999999999996e-06 2024-09-20T22:15:54.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.131 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:54.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114168+0000", 2024-09-20T22:15:54.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2126999999999999e-05 2024-09-20T22:15:54.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.132 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.132 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:54.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114175+0000", 2024-09-20T22:15:54.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6789999999999999e-06 2024-09-20T22:15:54.132 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:54.132 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:54.132 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:54.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.132 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.132 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:64588 10.d 10:bf257d2a:::benchmark_data_smithi060_167766_object64587:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-20T22:15:54.132 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:15:54.114557+0000", 2024-09-20T22:15:54.133 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0025491979999999999, 2024-09-20T22:15:54.133 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0027821959999999998, 2024-09-20T22:15:54.133 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:54.133 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:54.133 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:15:54.133 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:54.133 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:15:54.133 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:15:54.133 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 64588 2024-09-20T22:15:54.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.133 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:54.133 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.133 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:54.133 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114557+0000", 2024-09-20T22:15:54.133 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:54.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.134 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.134 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:54.134 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114557+0000", 2024-09-20T22:15:54.134 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:54.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.134 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.134 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:54.135 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114558+0000", 2024-09-20T22:15:54.135 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2279999999999999e-06 2024-09-20T22:15:54.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.135 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:54.135 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114619+0000", 2024-09-20T22:15:54.135 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1057999999999997e-05 2024-09-20T22:15:54.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.135 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:54.135 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114635+0000", 2024-09-20T22:15:54.135 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5585000000000001e-05 2024-09-20T22:15:54.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:54.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114642+0000", 2024-09-20T22:15:54.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4680000000000003e-06 2024-09-20T22:15:54.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.136 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:54.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114763+0000", 2024-09-20T22:15:54.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000120316 2024-09-20T22:15:54.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.136 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:54.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114893+0000", 2024-09-20T22:15:54.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013076 2024-09-20T22:15:54.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.136 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.137 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:15:54.137 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.115037+0000", 2024-09-20T22:15:54.137 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000143318 2024-09-20T22:15:54.137 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:54.137 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:54.137 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:54.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.137 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.137 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:64590 10.d 10.84f56dbd (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-20T22:15:54.137 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:15:54.114817+0000", 2024-09-20T22:15:54.137 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0022892749999999999, 2024-09-20T22:15:54.137 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0025970500000000001, 2024-09-20T22:15:54.137 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:54.137 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:54.138 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "queued for pg", 2024-09-20T22:15:54.138 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:54.138 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:15:54.138 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:15:54.138 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 64590 2024-09-20T22:15:54.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.138 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:54.138 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.138 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:54.138 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114817+0000", 2024-09-20T22:15:54.138 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:54.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.138 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.139 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:54.139 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114817+0000", 2024-09-20T22:15:54.139 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:54.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.139 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.139 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:54.139 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114818+0000", 2024-09-20T22:15:54.139 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4900000000000005e-07 2024-09-20T22:15:54.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.139 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.139 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:54.139 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114876+0000", 2024-09-20T22:15:54.139 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8300000000000001e-05 2024-09-20T22:15:54.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.139 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.140 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:54.140 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114886+0000", 2024-09-20T22:15:54.140 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7180000000000008e-06 2024-09-20T22:15:54.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.140 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.140 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:54.140 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114894+0000", 2024-09-20T22:15:54.140 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4249999999999998e-06 2024-09-20T22:15:54.140 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:54.140 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:54.140 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:54.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.140 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.140 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:64591 10.0 10:0214b7ff:::benchmark_data_smithi060_167766_object64590:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-20T22:15:54.141 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:15:54.114986+0000", 2024-09-20T22:15:54.141 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.00212014, 2024-09-20T22:15:54.141 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0024833590000000001, 2024-09-20T22:15:54.141 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:54.141 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:54.141 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:15:54.141 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:54.141 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:15:54.141 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:15:54.141 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 64591 2024-09-20T22:15:54.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.141 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:54.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.142 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:54.142 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114986+0000", 2024-09-20T22:15:54.142 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:54.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.142 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:54.142 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114986+0000", 2024-09-20T22:15:54.142 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:54.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.142 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:54.142 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.114987+0000", 2024-09-20T22:15:54.142 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7000000000000004e-07 2024-09-20T22:15:54.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.143 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:54.143 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.115083+0000", 2024-09-20T22:15:54.143 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6152999999999999e-05 2024-09-20T22:15:54.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.143 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:54.143 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.115095+0000", 2024-09-20T22:15:54.143 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.184e-05 2024-09-20T22:15:54.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.143 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:54.143 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.115101+0000", 2024-09-20T22:15:54.143 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5679999999999996e-06 2024-09-20T22:15:54.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:54.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.115160+0000", 2024-09-20T22:15:54.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9069e-05 2024-09-20T22:15:54.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:54.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.115294+0000", 2024-09-20T22:15:54.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013405599999999999 2024-09-20T22:15:54.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:15:54.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.115533+0000", 2024-09-20T22:15:54.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00023866299999999999 2024-09-20T22:15:54.145 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:54.145 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:54.145 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:54.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.145 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:64579 10.7 10:e218ab53:::benchmark_data_smithi060_167766_object64578:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-20T22:15:54.145 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:15:54.109965+0000", 2024-09-20T22:15:54.145 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0071415259999999996, 2024-09-20T22:15:54.145 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0075765040000000004, 2024-09-20T22:15:54.145 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:54.145 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:54.145 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:15:54.145 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:54.145 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:15:54.145 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:15:54.146 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 64579 2024-09-20T22:15:54.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.146 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:54.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.146 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:54.146 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.109965+0000", 2024-09-20T22:15:54.146 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:54.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.146 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:54.146 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.109965+0000", 2024-09-20T22:15:54.146 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:54.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.146 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:54.147 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.109966+0000", 2024-09-20T22:15:54.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.217e-06 2024-09-20T22:15:54.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.147 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:54.147 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110048+0000", 2024-09-20T22:15:54.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1981e-05 2024-09-20T22:15:54.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.147 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:54.147 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110062+0000", 2024-09-20T22:15:54.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4470999999999999e-05 2024-09-20T22:15:54.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.147 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:54.148 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110070+0000", 2024-09-20T22:15:54.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.875e-06 2024-09-20T22:15:54.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.148 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:54.148 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110133+0000", 2024-09-20T22:15:54.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2984000000000005e-05 2024-09-20T22:15:54.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.148 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:54.148 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110281+0000", 2024-09-20T22:15:54.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000148257 2024-09-20T22:15:54.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.148 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:15:54.149 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110509+0000", 2024-09-20T22:15:54.149 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00022776299999999999 2024-09-20T22:15:54.149 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:54.149 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:54.149 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:54.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.149 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:64580 10.a 10:51a8bf89:::benchmark_data_smithi060_167766_object64579:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e390)", 2024-09-20T22:15:54.149 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:15:54.110092+0000", 2024-09-20T22:15:54.149 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0070144489999999999, 2024-09-20T22:15:54.149 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0075282200000000004, 2024-09-20T22:15:54.149 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:15:54.149 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:15:54.149 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:15:54.150 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:15:54.150 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:15:54.150 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:15:54.150 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 64580 2024-09-20T22:15:54.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.150 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:15:54.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.150 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:15:54.150 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110092+0000", 2024-09-20T22:15:54.150 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:54.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.150 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:15:54.151 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110092+0000", 2024-09-20T22:15:54.151 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:15:54.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.151 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:15:54.151 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110092+0000", 2024-09-20T22:15:54.151 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3799999999999997e-07 2024-09-20T22:15:54.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.151 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:15:54.151 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110163+0000", 2024-09-20T22:15:54.151 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0282000000000002e-05 2024-09-20T22:15:54.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.152 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:15:54.152 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110176+0000", 2024-09-20T22:15:54.152 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3302000000000001e-05 2024-09-20T22:15:54.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.152 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:15:54.152 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110183+0000", 2024-09-20T22:15:54.152 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.357e-06 2024-09-20T22:15:54.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.152 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:15:54.152 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110237+0000", 2024-09-20T22:15:54.152 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3829000000000002e-05 2024-09-20T22:15:54.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.153 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.153 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:15:54.153 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110371+0000", 2024-09-20T22:15:54.153 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000134108 2024-09-20T22:15:54.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.153 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.153 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,5", 2024-09-20T22:15:54.153 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.110592+0000", 2024-09-20T22:15:54.153 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00022038200000000001 2024-09-20T22:15:54.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:15:54.153 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:15:54.153 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:15:54.153 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:15:54.117279+0000", 2024-09-20T22:15:54.153 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0066876890000000001 2024-09-20T22:15:54.154 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:54.154 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:15:54.154 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:54.154 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:15:54.154 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-20T22:15:54.154 INFO:teuthology.orchestra.run.smithi067.stdout: "num_ops": 8 2024-09-20T22:15:54.154 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:15:54.154 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:54.196 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:54.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:54.197 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:54.297 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:54.319 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:54.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:54.320 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:54.421 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:54.459 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:54.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:54.460 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:54.561 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:54.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:54.602 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:54.602 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:54.673 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:15:54.703 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:54.739 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:54.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:54.740 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:54.841 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:54.882 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:54.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:54.883 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:54.923 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 37 16 65635 65619 110.819 80.875 0.0327339 0.00901853 2024-09-20T22:15:54.943 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:54 smithi060 ceph-mon[29694]: 5.5 scrub starts 2024-09-20T22:15:54.944 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:54 smithi060 ceph-mon[29694]: 5.5 scrub ok 2024-09-20T22:15:54.944 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:54 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3647713547' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:54.944 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:54 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:54.944 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:54 smithi060 ceph-mon[29694]: pgmap v697: 168 pgs: 168 active+clean; 3.8 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 3.4 KiB/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-20T22:15:54.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:54 smithi060 ceph-mon[40290]: 5.5 scrub starts 2024-09-20T22:15:54.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:54 smithi060 ceph-mon[40290]: 5.5 scrub ok 2024-09-20T22:15:54.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:54 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3647713547' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:54.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:54 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:54.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:54 smithi060 ceph-mon[40290]: pgmap v697: 168 pgs: 168 active+clean; 3.8 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 3.4 KiB/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-20T22:15:54.984 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:55.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:54 smithi067 ceph-mon[33511]: 5.5 scrub starts 2024-09-20T22:15:55.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:54 smithi067 ceph-mon[33511]: 5.5 scrub ok 2024-09-20T22:15:55.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:54 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3647713547' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:55.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:54 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:15:55.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:54 smithi067 ceph-mon[33511]: pgmap v697: 168 pgs: 168 active+clean; 3.8 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 3.4 KiB/s rd, 68 MiB/s wr, 1.08k op/s 2024-09-20T22:15:55.020 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:55.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:55.022 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:55.122 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:55.160 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:55.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:55.161 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:55.172 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T22:15:55.262 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:55.285 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:15:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:15:55.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:55.317 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:55.317 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:55.418 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:55.476 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:55.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:55.477 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:55.578 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:55.610 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:15:55.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:55.618 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:55.618 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:55.689 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.1 to 1 (10000) 2024-09-20T22:15:55.720 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:55.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:55.754 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:55.754 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:55.855 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:55.889 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:55.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:55.891 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:55.923 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 38 16 66968 66952 110.095 83.3125 0.00826087 0.00907817 2024-09-20T22:15:55.955 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:55 smithi067 ceph-mon[33511]: 5.16 scrub starts 2024-09-20T22:15:55.956 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:55 smithi067 ceph-mon[33511]: 5.16 scrub ok 2024-09-20T22:15:55.956 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:55 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:15:55.956 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:55 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:15:55.956 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:55 smithi067 ceph-mon[33511]: 3.15 scrub starts 2024-09-20T22:15:55.956 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:55 smithi067 ceph-mon[33511]: 3.15 scrub ok 2024-09-20T22:15:55.956 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:55 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:15:55.956 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:55 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:15:55.956 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:55 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:15:55.956 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:55 smithi067 ceph-mon[33511]: osdmap e391: 8 total, 8 up, 8 in 2024-09-20T22:15:55.956 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2358697043' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:15:55.956 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:55 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:15:55.992 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:56.029 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:56.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:56.030 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:56.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[29694]: 5.16 scrub starts 2024-09-20T22:15:56.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[29694]: 5.16 scrub ok 2024-09-20T22:15:56.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:15:56.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:15:56.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[29694]: 3.15 scrub starts 2024-09-20T22:15:56.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[29694]: 3.15 scrub ok 2024-09-20T22:15:56.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:15:56.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:15:56.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:15:56.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[29694]: osdmap e391: 8 total, 8 up, 8 in 2024-09-20T22:15:56.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2358697043' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:15:56.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:15:56.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[40290]: 5.16 scrub starts 2024-09-20T22:15:56.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[40290]: 5.16 scrub ok 2024-09-20T22:15:56.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:15:56.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:15:56.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[40290]: 3.15 scrub starts 2024-09-20T22:15:56.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[40290]: 3.15 scrub ok 2024-09-20T22:15:56.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:15:56.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:15:56.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:15:56.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[40290]: osdmap e391: 8 total, 8 up, 8 in 2024-09-20T22:15:56.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2358697043' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:15:56.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:55 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:15:56.062 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:56.131 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:56.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:56.185 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:56.186 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:56.278 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:15:56.278 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:15:56.278 INFO:teuthology.orchestra.run.smithi067.stdout: "complaint_time": 30, 2024-09-20T22:15:56.278 INFO:teuthology.orchestra.run.smithi067.stdout: "num_blocked_ops": 0 2024-09-20T22:15:56.278 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:15:56.286 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:56.319 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:56.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:56.320 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:56.420 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:56.458 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:56.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:56.459 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:56.560 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:56.572 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 2 1 2024-09-20T22:15:56.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:56.604 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:56.604 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:56.705 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:56.735 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:56.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:56.736 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[40290]: 2.0 scrub starts 2024-09-20T22:15:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[40290]: 2.0 scrub ok 2024-09-20T22:15:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T22:15:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[40290]: osdmap e392: 8 total, 8 up, 8 in 2024-09-20T22:15:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[40290]: 4.5 scrub starts 2024-09-20T22:15:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[40290]: 4.5 scrub ok 2024-09-20T22:15:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[40290]: 4.1c scrub starts 2024-09-20T22:15:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[40290]: 4.1c scrub ok 2024-09-20T22:15:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[40290]: 4.3 deep-scrub starts 2024-09-20T22:15:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[40290]: 4.3 deep-scrub ok 2024-09-20T22:15:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[40290]: pgmap v700: 168 pgs: 168 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.8 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-20T22:15:56.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[29694]: 2.0 scrub starts 2024-09-20T22:15:56.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[29694]: 2.0 scrub ok 2024-09-20T22:15:56.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T22:15:56.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[29694]: osdmap e392: 8 total, 8 up, 8 in 2024-09-20T22:15:56.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[29694]: 4.5 scrub starts 2024-09-20T22:15:56.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[29694]: 4.5 scrub ok 2024-09-20T22:15:56.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[29694]: 4.1c scrub starts 2024-09-20T22:15:56.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[29694]: 4.1c scrub ok 2024-09-20T22:15:56.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[29694]: 4.3 deep-scrub starts 2024-09-20T22:15:56.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[29694]: 4.3 deep-scrub ok 2024-09-20T22:15:56.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:56 smithi060 ceph-mon[29694]: pgmap v700: 168 pgs: 168 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.8 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-20T22:15:56.837 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:56.878 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:56.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:56.879 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:56.923 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 39 16 68267 68251 109.353 81.1875 0.0103288 0.00913926 2024-09-20T22:15:56.980 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:57.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:56 smithi067 ceph-mon[33511]: 2.0 scrub starts 2024-09-20T22:15:57.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:56 smithi067 ceph-mon[33511]: 2.0 scrub ok 2024-09-20T22:15:57.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:56 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T22:15:57.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:56 smithi067 ceph-mon[33511]: osdmap e392: 8 total, 8 up, 8 in 2024-09-20T22:15:57.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:56 smithi067 ceph-mon[33511]: 4.5 scrub starts 2024-09-20T22:15:57.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:56 smithi067 ceph-mon[33511]: 4.5 scrub ok 2024-09-20T22:15:57.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:56 smithi067 ceph-mon[33511]: 4.1c scrub starts 2024-09-20T22:15:57.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:56 smithi067 ceph-mon[33511]: 4.1c scrub ok 2024-09-20T22:15:57.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:56 smithi067 ceph-mon[33511]: 4.3 deep-scrub starts 2024-09-20T22:15:57.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:56 smithi067 ceph-mon[33511]: 4.3 deep-scrub ok 2024-09-20T22:15:57.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:56 smithi067 ceph-mon[33511]: pgmap v700: 168 pgs: 168 active+clean; 3.9 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 1.8 KiB/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-20T22:15:57.021 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:57.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:57.023 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:57.123 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:57.148 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:15:57.148 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:15:57.149 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:57.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:57.150 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:57.167 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:15:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:15:57.168 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:57.186 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:15:57.186 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:15:57.226 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:15:57.226 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:15:57.250 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:15:57.261 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:15:57.261 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:15:57.286 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:57.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:57.287 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:15:57.294 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:15:57.294 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:15:57.337 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:15:57.338 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:15:57.374 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:15:57.374 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:15:57.388 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:57.424 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:57.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:57.425 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:57.526 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:57.562 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:57.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:57.563 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:57.628 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T22:15:57.664 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:57.697 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:57.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:57.698 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:57.798 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:57.813 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:15:57.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:57.851 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:57.852 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:57.923 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:15:57.923178+0000 min lat: 0.00137086 max lat: 0.30872 avg lat: 0.00921066 2024-09-20T22:15:57.923 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:15:57.923 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 40 16 69474 69458 108.505 75.4375 0.0129874 0.00921066 2024-09-20T22:15:57.952 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:57 smithi067 ceph-mon[33511]: 5.1 scrub starts 2024-09-20T22:15:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:57 smithi067 ceph-mon[33511]: 5.1 scrub ok 2024-09-20T22:15:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:57 smithi067 ceph-mon[33511]: 5.0 scrub starts 2024-09-20T22:15:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:57 smithi067 ceph-mon[33511]: 5.0 scrub ok 2024-09-20T22:15:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:57 smithi067 ceph-mon[33511]: 1.9 scrub starts 2024-09-20T22:15:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:57 smithi067 ceph-mon[33511]: 1.9 scrub ok 2024-09-20T22:15:58.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:57 smithi067 ceph-mon[33511]: 6.f scrub starts 2024-09-20T22:15:58.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:57 smithi067 ceph-mon[33511]: 6.f scrub ok 2024-09-20T22:15:58.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:57 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2354455363' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:58.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:57 smithi067 ceph-mon[33511]: 4.1e scrub starts 2024-09-20T22:15:58.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:57 smithi067 ceph-mon[33511]: 4.1e scrub ok 2024-09-20T22:15:58.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:57 smithi067 ceph-mon[33511]: 4.1f scrub starts 2024-09-20T22:15:58.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:57 smithi067 ceph-mon[33511]: 4.1f scrub ok 2024-09-20T22:15:58.014 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:58.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:58.015 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[29694]: 5.1 scrub starts 2024-09-20T22:15:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[29694]: 5.1 scrub ok 2024-09-20T22:15:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[29694]: 5.0 scrub starts 2024-09-20T22:15:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[29694]: 5.0 scrub ok 2024-09-20T22:15:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[29694]: 1.9 scrub starts 2024-09-20T22:15:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[29694]: 1.9 scrub ok 2024-09-20T22:15:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[29694]: 6.f scrub starts 2024-09-20T22:15:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[29694]: 6.f scrub ok 2024-09-20T22:15:58.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2354455363' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:58.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[29694]: 4.1e scrub starts 2024-09-20T22:15:58.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[29694]: 4.1e scrub ok 2024-09-20T22:15:58.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[29694]: 4.1f scrub starts 2024-09-20T22:15:58.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[29694]: 4.1f scrub ok 2024-09-20T22:15:58.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[40290]: 5.1 scrub starts 2024-09-20T22:15:58.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[40290]: 5.1 scrub ok 2024-09-20T22:15:58.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[40290]: 5.0 scrub starts 2024-09-20T22:15:58.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[40290]: 5.0 scrub ok 2024-09-20T22:15:58.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[40290]: 1.9 scrub starts 2024-09-20T22:15:58.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[40290]: 1.9 scrub ok 2024-09-20T22:15:58.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[40290]: 6.f scrub starts 2024-09-20T22:15:58.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[40290]: 6.f scrub ok 2024-09-20T22:15:58.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2354455363' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:15:58.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[40290]: 4.1e scrub starts 2024-09-20T22:15:58.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[40290]: 4.1e scrub ok 2024-09-20T22:15:58.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[40290]: 4.1f scrub starts 2024-09-20T22:15:58.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:57 smithi060 ceph-mon[40290]: 4.1f scrub ok 2024-09-20T22:15:58.058 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:15:58.116 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:15:58.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:58.182 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:58.182 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:15:58.283 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:58.309 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:58.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:58.310 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:58.411 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:15:58.468 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:58.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:58.469 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:15:58.521 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:15:58.570 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:58.615 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:58.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:58.616 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:58.717 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:15:58.729 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.2 to 1 (10000) 2024-09-20T22:15:58.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:58.761 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:58.761 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:15:58.861 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:58.897 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:58.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:58.898 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:58.923 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 41 16 70702 70686 107.73 76.75 0.00667903 0.0092763 2024-09-20T22:15:58.999 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:15:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:58 smithi067 ceph-mon[33511]: 4.15 scrub starts 2024-09-20T22:15:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:58 smithi067 ceph-mon[33511]: 4.15 scrub ok 2024-09-20T22:15:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:58 smithi067 ceph-mon[33511]: 5.c scrub starts 2024-09-20T22:15:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:58 smithi067 ceph-mon[33511]: 5.c scrub ok 2024-09-20T22:15:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:58 smithi067 ceph-mon[33511]: 6.1c scrub starts 2024-09-20T22:15:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:58 smithi067 ceph-mon[33511]: 6.1c scrub ok 2024-09-20T22:15:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:58 smithi067 ceph-mon[33511]: pgmap v701: 168 pgs: 168 active+clean; 4.2 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 87 MiB/s wr, 1.40k op/s 2024-09-20T22:15:59.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:58 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1556280624' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:15:59.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:58 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:15:59.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:58 smithi067 ceph-mon[33511]: 5.2 scrub starts 2024-09-20T22:15:59.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:58 smithi067 ceph-mon[33511]: 5.2 scrub ok 2024-09-20T22:15:59.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[29694]: 4.15 scrub starts 2024-09-20T22:15:59.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[29694]: 4.15 scrub ok 2024-09-20T22:15:59.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[29694]: 5.c scrub starts 2024-09-20T22:15:59.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[29694]: 5.c scrub ok 2024-09-20T22:15:59.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[29694]: 6.1c scrub starts 2024-09-20T22:15:59.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[29694]: 6.1c scrub ok 2024-09-20T22:15:59.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[29694]: pgmap v701: 168 pgs: 168 active+clean; 4.2 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 87 MiB/s wr, 1.40k op/s 2024-09-20T22:15:59.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1556280624' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:15:59.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:15:59.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[29694]: 5.2 scrub starts 2024-09-20T22:15:59.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[29694]: 5.2 scrub ok 2024-09-20T22:15:59.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[40290]: 4.15 scrub starts 2024-09-20T22:15:59.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[40290]: 4.15 scrub ok 2024-09-20T22:15:59.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[40290]: 5.c scrub starts 2024-09-20T22:15:59.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[40290]: 5.c scrub ok 2024-09-20T22:15:59.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[40290]: 6.1c scrub starts 2024-09-20T22:15:59.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[40290]: 6.1c scrub ok 2024-09-20T22:15:59.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[40290]: pgmap v701: 168 pgs: 168 active+clean; 4.2 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 87 MiB/s wr, 1.40k op/s 2024-09-20T22:15:59.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1556280624' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:15:59.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:15:59.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[40290]: 5.2 scrub starts 2024-09-20T22:15:59.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:58 smithi060 ceph-mon[40290]: 5.2 scrub ok 2024-09-20T22:15:59.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:59.038 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:59.038 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:15:59.139 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:59.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:59.181 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:59.181 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:59.282 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:15:59.320 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:59.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:59.321 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:15:59.421 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:59.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:59.455 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:59.456 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:59.556 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:59.562 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:15:59.562 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:15:59.597 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:59.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:59.598 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:59.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:15:59.730 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:15:59.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:59.731 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:15:59.735 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 3 1 2024-09-20T22:15:59.831 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:15:59.832 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:15:59.867 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:15:59.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:15:59.868 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:15:59.923 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 42 16 71859 71843 106.887 72.3125 0.00469114 0.00934264 2024-09-20T22:15:59.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:00.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:00.011 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:00.011 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:00.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:59 smithi067 ceph-mon[33511]: 3.1a scrub starts 2024-09-20T22:16:00.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:59 smithi067 ceph-mon[33511]: 3.1a scrub ok 2024-09-20T22:16:00.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:59 smithi067 ceph-mon[33511]: 6.2 scrub starts 2024-09-20T22:16:00.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:59 smithi067 ceph-mon[33511]: 6.2 scrub ok 2024-09-20T22:16:00.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:59 smithi067 ceph-mon[33511]: 5.4 scrub starts 2024-09-20T22:16:00.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:59 smithi067 ceph-mon[33511]: 5.4 scrub ok 2024-09-20T22:16:00.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:59 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:16:00.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:59 smithi067 ceph-mon[33511]: osdmap e393: 8 total, 8 up, 8 in 2024-09-20T22:16:00.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:59 smithi067 ceph-mon[33511]: 4.16 scrub starts 2024-09-20T22:16:00.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:59 smithi067 ceph-mon[33511]: 4.16 scrub ok 2024-09-20T22:16:00.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:59 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4018168253' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:16:00.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:15:59 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:16:00.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[29694]: 3.1a scrub starts 2024-09-20T22:16:00.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[29694]: 3.1a scrub ok 2024-09-20T22:16:00.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[29694]: 6.2 scrub starts 2024-09-20T22:16:00.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[29694]: 6.2 scrub ok 2024-09-20T22:16:00.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[29694]: 5.4 scrub starts 2024-09-20T22:16:00.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[29694]: 5.4 scrub ok 2024-09-20T22:16:00.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:16:00.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[29694]: osdmap e393: 8 total, 8 up, 8 in 2024-09-20T22:16:00.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[29694]: 4.16 scrub starts 2024-09-20T22:16:00.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[29694]: 4.16 scrub ok 2024-09-20T22:16:00.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4018168253' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:16:00.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:16:00.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[40290]: 3.1a scrub starts 2024-09-20T22:16:00.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[40290]: 3.1a scrub ok 2024-09-20T22:16:00.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[40290]: 6.2 scrub starts 2024-09-20T22:16:00.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[40290]: 6.2 scrub ok 2024-09-20T22:16:00.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[40290]: 5.4 scrub starts 2024-09-20T22:16:00.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[40290]: 5.4 scrub ok 2024-09-20T22:16:00.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:16:00.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[40290]: osdmap e393: 8 total, 8 up, 8 in 2024-09-20T22:16:00.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[40290]: 4.16 scrub starts 2024-09-20T22:16:00.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[40290]: 4.16 scrub ok 2024-09-20T22:16:00.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4018168253' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:16:00.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:15:59 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:16:00.113 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:00.146 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:00.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:00.147 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:00.248 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:00.285 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:00.286 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:00.286 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:00.387 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:00.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:00.422 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:00.422 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:00.466 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:16:00.523 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:00.593 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:00.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:00.594 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:00.613 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:00.695 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:00.727 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:00.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:00.728 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:00.748 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:00 smithi060 ceph-mon[29694]: 4.1a scrub starts 2024-09-20T22:16:00.748 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:00 smithi060 ceph-mon[29694]: 4.1a scrub ok 2024-09-20T22:16:00.748 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:00 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:16:00.748 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:00 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:16:00.749 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:00 smithi060 ceph-mon[29694]: osdmap e394: 8 total, 8 up, 8 in 2024-09-20T22:16:00.749 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:00 smithi060 ceph-mon[29694]: pgmap v704: 168 pgs: 168 active+clean; 4.2 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 72 MiB/s wr, 1.15k op/s 2024-09-20T22:16:00.749 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:00 smithi060 ceph-mon[40290]: 4.1a scrub starts 2024-09-20T22:16:00.749 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:00 smithi060 ceph-mon[40290]: 4.1a scrub ok 2024-09-20T22:16:00.749 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:00 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:16:00.749 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:00 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:16:00.749 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:00 smithi060 ceph-mon[40290]: osdmap e394: 8 total, 8 up, 8 in 2024-09-20T22:16:00.749 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:00 smithi060 ceph-mon[40290]: pgmap v704: 168 pgs: 168 active+clean; 4.2 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 72 MiB/s wr, 1.15k op/s 2024-09-20T22:16:00.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:00.873 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:00.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:00.875 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:00.898 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T22:16:00.923 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 43 15 73113 73098 106.225 78.4375 0.0174857 0.00940873 2024-09-20T22:16:00.976 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:00 smithi067 ceph-mon[33511]: 4.1a scrub starts 2024-09-20T22:16:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:00 smithi067 ceph-mon[33511]: 4.1a scrub ok 2024-09-20T22:16:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:00 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:16:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:00 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:16:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:00 smithi067 ceph-mon[33511]: osdmap e394: 8 total, 8 up, 8 in 2024-09-20T22:16:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:00 smithi067 ceph-mon[33511]: pgmap v704: 168 pgs: 168 active+clean; 4.2 GiB data, 17 GiB used, 699 GiB / 715 GiB avail; 72 MiB/s wr, 1.15k op/s 2024-09-20T22:16:01.017 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:01.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:01.019 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:01.119 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:01.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:01.158 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:01.158 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:01.259 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:01.276 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:01.311 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:01.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:01.312 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:01.345 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:16:01.413 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:01.452 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:01.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:01.453 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:01.554 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:01.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:01.600 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:01.600 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:01.700 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:01.741 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:01.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:01.742 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:01.767 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.3 to 1 (10000) 2024-09-20T22:16:01.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:01.888 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:01.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:01.889 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:01.923 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 44 16 74268 74252 105.45 72.125 0.00548611 0.00947741 2024-09-20T22:16:01.990 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:01.992 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:16:01.993 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:16:02.034 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:02.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:01 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:16:02.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:01 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:16:02.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:01 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3916071020' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:16:02.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:01 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:16:02.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:01 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:16:02.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:01 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:16:02.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:01 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3916071020' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:16:02.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:01 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:16:02.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:02.037 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:02.137 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:02.176 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:02.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:02.177 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:02.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:01 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:16:02.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:01 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:16:02.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:01 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3916071020' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:16:02.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:01 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:16:02.278 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:02.313 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:02.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:02.314 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:02.415 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:02.455 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:02.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:02.456 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:02.554 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:16:02.556 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:02.588 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:02.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:02.589 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:02.680 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:16:02.680 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:16:02.690 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:02.719 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:16:02.719 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:16:02.755 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:02.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:02.757 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:02.757 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:16:02.757 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:16:02.763 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:16:02 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:16:02.791 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:16:02.791 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:16:02.829 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:16:02.829 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:16:02.856 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T22:16:02.858 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:02.865 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:16:02.865 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:16:02.908 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:02.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:02.910 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:02.910 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:16:02.910 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:16:02.923 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 45 16 75506 75490 104.826 77.375 0.00712098 0.00953421 2024-09-20T22:16:02.944 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:16:02.944 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:16:03.011 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:03.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:02 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:16:03.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:02 smithi060 ceph-mon[29694]: osdmap e395: 8 total, 8 up, 8 in 2024-09-20T22:16:03.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:02 smithi060 ceph-mon[29694]: pgmap v706: 168 pgs: 168 active+clean; 4.4 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-20T22:16:03.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:02 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:16:03.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:02 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2512284126' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:02 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:16:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:02 smithi060 ceph-mon[40290]: osdmap e395: 8 total, 8 up, 8 in 2024-09-20T22:16:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:02 smithi060 ceph-mon[40290]: pgmap v706: 168 pgs: 168 active+clean; 4.4 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-20T22:16:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:02 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:16:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:02 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2512284126' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:03.046 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:03.047 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:03.047 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:03.148 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:03.180 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:03.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:03.181 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:03.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:02 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:16:03.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:02 smithi067 ceph-mon[33511]: osdmap e395: 8 total, 8 up, 8 in 2024-09-20T22:16:03.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:02 smithi067 ceph-mon[33511]: pgmap v706: 168 pgs: 168 active+clean; 4.4 GiB data, 18 GiB used, 698 GiB / 715 GiB avail; 853 B/s rd, 81 MiB/s wr, 1.30k op/s 2024-09-20T22:16:03.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:02 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:16:03.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:02 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2512284126' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:03.282 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:03.284 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:16:03.322 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:03.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:03.323 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:03.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:03.462 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 4 1 2024-09-20T22:16:03.485 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:03.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:03.487 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:03.588 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:03.621 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:03.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:03.622 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:03.723 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:03.744 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:03.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:03.745 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:03.846 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:03.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:03.883 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:03.883 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:03.924 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 46 16 76636 76620 104.083 70.625 0.00889774 0.00960324 2024-09-20T22:16:03.984 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:04.019 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:04.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:04.020 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:04.121 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:04.183 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:04.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:04.184 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:04.204 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:04.231 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:04.285 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:04.307 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:04.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:04.308 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:04.408 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:04.430 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:04.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:04.432 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:04.532 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:04.568 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:04.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:04.569 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:04.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:04.713 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:04.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:04.714 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:04.815 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:04.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:04.842 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:04.842 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:04.924 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 47 16 77490 77474 103.004 53.375 0.00382947 0.00969586 2024-09-20T22:16:04.942 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:04.964 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:04.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:04.965 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:05.065 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:05.104 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:05.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:05.105 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:05.205 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:05.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:05.245 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:05.246 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:05.285 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:16:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:16:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:16:05.289 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:16:05.347 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:05.386 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:05.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:05.387 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:05.487 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:05.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:05 smithi067 ceph-mon[33511]: pgmap v707: 168 pgs: 168 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-20T22:16:05.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:05 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:16:05.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:05 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:16:05.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:05 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:16:05.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:05 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3424187412' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:16:05.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:05.526 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:05.526 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:05.571 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:16:05.571 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:16:05.571 INFO:teuthology.orchestra.run.smithi067.stdout: "complaint_time": 30, 2024-09-20T22:16:05.571 INFO:teuthology.orchestra.run.smithi067.stdout: "num_blocked_ops": 0 2024-09-20T22:16:05.571 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:16:05.627 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:05.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:05 smithi060 ceph-mon[29694]: pgmap v707: 168 pgs: 168 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-20T22:16:05.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:05 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:16:05.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:05 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:16:05.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:05 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:16:05.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:05 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3424187412' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:16:05.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:05 smithi060 ceph-mon[40290]: pgmap v707: 168 pgs: 168 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 853 B/s rd, 67 MiB/s wr, 1.07k op/s 2024-09-20T22:16:05.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:05 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:16:05.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:05 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:16:05.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:05 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:16:05.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:05 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3424187412' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:16:05.664 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:05.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:05.665 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:05.766 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:05.792 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:05.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:05.793 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:05.894 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:05.925 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 48 16 78479 78463 102.146 61.8125 0.0138843 0.00978477 2024-09-20T22:16:05.937 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:05.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:05.938 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:06.039 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:06.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:06.083 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:06.083 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:06.127 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:16:06.183 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:06.256 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:06.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:06.257 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:06.323 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.4 to 1 (10000) 2024-09-20T22:16:06.358 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:06.400 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:06.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:06.401 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:06.452 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T22:16:06.501 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:06 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:16:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:06 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3424187412' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:16:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:06 smithi060 ceph-mon[29694]: osdmap e396: 8 total, 8 up, 8 in 2024-09-20T22:16:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:06 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3497864418' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:16:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:16:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:16:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:16:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:16:06.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:16:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:06 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:16:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:06 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3424187412' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:16:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:06 smithi060 ceph-mon[40290]: osdmap e396: 8 total, 8 up, 8 in 2024-09-20T22:16:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:06 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3497864418' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:16:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:16:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:16:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:16:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:16:06.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:16:06.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:06.538 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:06.538 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:06.639 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:06.677 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:06.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:06.678 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:06.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:06 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:16:06.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:06 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3424187412' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:16:06.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:06 smithi067 ceph-mon[33511]: osdmap e396: 8 total, 8 up, 8 in 2024-09-20T22:16:06.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:06 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3497864418' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:16:06.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:16:06.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:16:06.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:16:06.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:16:06.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:16:06.779 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:06.817 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:06.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:06.818 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:06.886 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:16:06.919 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:06.924 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 49 16 79560 79544 101.44 67.5625 0.00985999 0.00985229 2024-09-20T22:16:06.970 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:06.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:06.971 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:07.072 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:07.105 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:07.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:07.106 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:07.207 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:07.243 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:07.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:07.244 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:07.345 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:07.386 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:07.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:07.386 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:07.487 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:07.524 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:07.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:07.525 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:07.590 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:07 smithi067 ceph-mon[33511]: pgmap v709: 168 pgs: 168 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.5 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-20T22:16:07.590 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:07 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3497864418' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T22:16:07.590 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:07 smithi067 ceph-mon[33511]: osdmap e397: 8 total, 8 up, 8 in 2024-09-20T22:16:07.590 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:07 smithi067 ceph-mon[33511]: osdmap e398: 8 total, 8 up, 8 in 2024-09-20T22:16:07.625 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:07.664 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:07.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:07.665 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:07.766 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:07.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:07 smithi060 ceph-mon[40290]: pgmap v709: 168 pgs: 168 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.5 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-20T22:16:07.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:07 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3497864418' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T22:16:07.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:07 smithi060 ceph-mon[40290]: osdmap e397: 8 total, 8 up, 8 in 2024-09-20T22:16:07.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:07 smithi060 ceph-mon[40290]: osdmap e398: 8 total, 8 up, 8 in 2024-09-20T22:16:07.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:07 smithi060 ceph-mon[29694]: pgmap v709: 168 pgs: 168 active+clean; 4.6 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.5 KiB/s rd, 75 MiB/s wr, 1.20k op/s 2024-09-20T22:16:07.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:07 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3497864418' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T22:16:07.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:07 smithi060 ceph-mon[29694]: osdmap e397: 8 total, 8 up, 8 in 2024-09-20T22:16:07.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:07 smithi060 ceph-mon[29694]: osdmap e398: 8 total, 8 up, 8 in 2024-09-20T22:16:07.812 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:07.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:07.813 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:07.914 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:07.923 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:16:07.923 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:16:07.923 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:16:07.923 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:16:07.923 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.923 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:236437 7.11 7:8eaecf20:::benchmark_data_smithi060_78800_object103404:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e152)", 2024-09-20T22:16:07.923 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:07.392155+0000", 2024-09-20T22:16:07.923 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 540.52792644500005, 2024-09-20T22:16:07.923 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.019005036, 2024-09-20T22:16:07.923 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:07.923 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:07.923 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:07.924 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:16:07.924 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:16:07.924 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:16:07.924 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 236437 2024-09-20T22:16:07.924 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.924 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:07.924 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.924 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:07.924 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392155+0000", 2024-09-20T22:16:07.924 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.924 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.924 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.924 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:07.925 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392155+0000", 2024-09-20T22:16:07.925 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.925 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.925 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.925 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:07.925 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392157+0000", 2024-09-20T22:16:07.925 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9599999999999999e-06 2024-09-20T22:16:07.925 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.925 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.925 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:07.925 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392162+0000", 2024-09-20T22:16:07.925 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1719999999999998e-06 2024-09-20T22:16:07.925 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.926 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.926 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:07.926 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392180+0000", 2024-09-20T22:16:07.926 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8088e-05 2024-09-20T22:16:07.926 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.926 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.926 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:07.926 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392188+0000", 2024-09-20T22:16:07.926 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9929999999999995e-06 2024-09-20T22:16:07.926 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.926 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.926 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.926 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392254+0000", 2024-09-20T22:16:07.927 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6137999999999996e-05 2024-09-20T22:16:07.927 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.927 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.927 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:07.927 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:07.392260+0000", 2024-09-20T22:16:07.927 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7089999999999999e-06 2024-09-20T22:16:07.927 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.927 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.927 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.927 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.406485+0000", 2024-09-20T22:16:07.927 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0142251280000001 2024-09-20T22:16:07.927 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.927 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.927 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:16:07.928 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.406593+0000", 2024-09-20T22:16:07.928 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000107672 2024-09-20T22:16:07.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.928 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.928 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:16:07.928 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.406954+0000", 2024-09-20T22:16:07.928 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000360973 2024-09-20T22:16:07.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.928 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.928 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:16:07.928 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411014+0000", 2024-09-20T22:16:07.928 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00406033 2024-09-20T22:16:07.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.929 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.929 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:07.929 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411059+0000", 2024-09-20T22:16:07.929 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4907999999999998e-05 2024-09-20T22:16:07.929 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.929 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.929 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:07.929 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411090+0000", 2024-09-20T22:16:07.929 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0874000000000003e-05 2024-09-20T22:16:07.929 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.929 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.929 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:16:07.930 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411121+0000", 2024-09-20T22:16:07.930 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0518999999999997e-05 2024-09-20T22:16:07.930 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.930 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.930 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:07.930 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:08.411160+0000", 2024-09-20T22:16:07.930 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9572e-05 2024-09-20T22:16:07.930 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.930 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:07.930 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.930 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.930 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.930 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261924 7.17 7:ec34c39a:::benchmark_data_smithi060_78800_object128891:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:16:07.930 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.137969+0000", 2024-09-20T22:16:07.931 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 511.78211214499999, 2024-09-20T22:16:07.931 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0378232009999999, 2024-09-20T22:16:07.931 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:07.931 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:07.931 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:16:07.931 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:16:07.931 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:16:07.931 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:16:07.931 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261924 2024-09-20T22:16:07.931 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.931 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:07.931 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.932 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:07.932 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137969+0000", 2024-09-20T22:16:07.932 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.932 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.932 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.932 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:07.932 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137969+0000", 2024-09-20T22:16:07.932 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.932 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.932 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.932 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:07.932 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137971+0000", 2024-09-20T22:16:07.932 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.19e-06 2024-09-20T22:16:07.932 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.932 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.933 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:07.933 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137976+0000", 2024-09-20T22:16:07.933 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8429999999999998e-06 2024-09-20T22:16:07.933 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.933 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.933 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:07.933 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137987+0000", 2024-09-20T22:16:07.933 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0587e-05 2024-09-20T22:16:07.933 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.933 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.933 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:07.933 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.137992+0000", 2024-09-20T22:16:07.933 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4090000000000002e-06 2024-09-20T22:16:07.933 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.934 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.934 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.934 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138048+0000", 2024-09-20T22:16:07.934 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5241999999999997e-05 2024-09-20T22:16:07.934 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.934 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.934 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:07.934 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138056+0000", 2024-09-20T22:16:07.934 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4649999999999994e-06 2024-09-20T22:16:07.934 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.934 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.934 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.934 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.175735+0000", 2024-09-20T22:16:07.934 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.037678871 2024-09-20T22:16:07.935 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.935 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.935 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:07.935 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.175793+0000", 2024-09-20T22:16:07.935 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7593999999999998e-05 2024-09-20T22:16:07.935 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.935 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:07.935 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.935 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.935 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.935 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261927 7.17 7:ed776cb1:::benchmark_data_smithi060_78800_object128894:head [delete] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:16:07.935 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.138007+0000", 2024-09-20T22:16:07.935 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 511.78207471799999, 2024-09-20T22:16:07.935 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0471626510000001, 2024-09-20T22:16:07.936 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:07.936 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:07.936 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:16:07.936 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:16:07.936 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:16:07.936 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:16:07.936 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261927 2024-09-20T22:16:07.936 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.936 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:07.936 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.936 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:07.936 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138007+0000", 2024-09-20T22:16:07.936 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.936 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.937 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.937 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:07.937 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138007+0000", 2024-09-20T22:16:07.937 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.937 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.938 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 50 16 80688 80672 100.821 70.5 0.0169453 0.00990588 2024-09-20T22:16:07.938 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:07.938 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138007+0000", 2024-09-20T22:16:07.938 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5600000000000001e-07 2024-09-20T22:16:07.938 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.938 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.938 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:07.938 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138009+0000", 2024-09-20T22:16:07.939 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6729999999999999e-06 2024-09-20T22:16:07.939 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.939 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.939 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:07.939 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138013+0000", 2024-09-20T22:16:07.939 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.16e-06 2024-09-20T22:16:07.939 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.939 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.939 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:07.939 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138016+0000", 2024-09-20T22:16:07.939 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1609999999999999e-06 2024-09-20T22:16:07.939 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.939 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.940 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.940 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138084+0000", 2024-09-20T22:16:07.940 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8044000000000001e-05 2024-09-20T22:16:07.940 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.940 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.940 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:07.940 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.138093+0000", 2024-09-20T22:16:07.940 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.49e-06 2024-09-20T22:16:07.940 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.940 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.940 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.940 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177120+0000", 2024-09-20T22:16:07.940 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0390272819999999 2024-09-20T22:16:07.940 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.941 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.941 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for missing object", 2024-09-20T22:16:07.941 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177222+0000", 2024-09-20T22:16:07.941 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000102272 2024-09-20T22:16:07.941 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.941 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.941 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.941 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185103+0000", 2024-09-20T22:16:07.941 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0078803929999999994 2024-09-20T22:16:07.941 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.941 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.941 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:07.941 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185170+0000", 2024-09-20T22:16:07.942 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6820000000000001e-05 2024-09-20T22:16:07.942 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.942 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:07.942 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.942 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.942 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.942 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261940 7.17 7:e9ae652d:::benchmark_data_smithi060_78800_object128907:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:16:07.942 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.144358+0000", 2024-09-20T22:16:07.942 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 511.77572380100003, 2024-09-20T22:16:07.942 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.040411067, 2024-09-20T22:16:07.942 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:07.942 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:07.942 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:07.942 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:16:07.943 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:16:07.943 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:16:07.943 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261940 2024-09-20T22:16:07.943 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.943 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:07.943 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.943 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:07.943 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144358+0000", 2024-09-20T22:16:07.943 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.943 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.943 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.943 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:07.943 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144358+0000", 2024-09-20T22:16:07.943 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.944 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.944 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.944 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:07.944 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144359+0000", 2024-09-20T22:16:07.944 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9299999999999996e-07 2024-09-20T22:16:07.944 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.944 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.944 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:07.944 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144362+0000", 2024-09-20T22:16:07.944 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6799999999999999e-06 2024-09-20T22:16:07.944 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.944 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.944 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:07.944 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144369+0000", 2024-09-20T22:16:07.945 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8070000000000002e-06 2024-09-20T22:16:07.945 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.945 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.945 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:07.945 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144374+0000", 2024-09-20T22:16:07.945 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8999999999999997e-06 2024-09-20T22:16:07.945 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.945 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.945 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.945 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144416+0000", 2024-09-20T22:16:07.945 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2049999999999999e-05 2024-09-20T22:16:07.945 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.945 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.945 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:07.945 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.144421+0000", 2024-09-20T22:16:07.946 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6550000000000003e-06 2024-09-20T22:16:07.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.946 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.946 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.946 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177283+0000", 2024-09-20T22:16:07.946 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0328618039999999 2024-09-20T22:16:07.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.946 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.946 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:16:07.946 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177468+0000", 2024-09-20T22:16:07.946 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00018551900000000001 2024-09-20T22:16:07.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.946 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.947 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:16:07.947 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.177846+0000", 2024-09-20T22:16:07.947 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00037771700000000002 2024-09-20T22:16:07.947 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.947 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.947 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:07.947 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184561+0000", 2024-09-20T22:16:07.947 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0067147600000000002 2024-09-20T22:16:07.947 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.947 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.947 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:16:07.947 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184572+0000", 2024-09-20T22:16:07.947 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1433999999999999e-05 2024-09-20T22:16:07.947 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.948 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.948 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:07.948 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184635+0000", 2024-09-20T22:16:07.948 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3459000000000003e-05 2024-09-20T22:16:07.948 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.948 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.948 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:16:07.948 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184692+0000", 2024-09-20T22:16:07.948 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6759999999999999e-05 2024-09-20T22:16:07.948 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.948 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.948 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:07.948 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184769+0000", 2024-09-20T22:16:07.948 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6629000000000001e-05 2024-09-20T22:16:07.949 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.949 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:07.949 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.949 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.949 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.949 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261963 7.17 7:eacd681d:::benchmark_data_smithi060_78800_object128930:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:16:07.949 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.165356+0000", 2024-09-20T22:16:07.949 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 511.75472522299998, 2024-09-20T22:16:07.949 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.020053374, 2024-09-20T22:16:07.949 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:07.949 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:07.949 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:07.949 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:16:07.950 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:16:07.950 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:16:07.950 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261963 2024-09-20T22:16:07.950 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.950 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:07.950 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.950 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:07.950 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165356+0000", 2024-09-20T22:16:07.950 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.950 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.950 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.950 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:07.950 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165356+0000", 2024-09-20T22:16:07.950 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.951 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.951 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:07.951 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165357+0000", 2024-09-20T22:16:07.951 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.3299999999999998e-07 2024-09-20T22:16:07.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.951 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.951 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:07.951 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165359+0000", 2024-09-20T22:16:07.951 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.046e-06 2024-09-20T22:16:07.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.951 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.951 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:07.952 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165364+0000", 2024-09-20T22:16:07.952 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.502e-06 2024-09-20T22:16:07.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.952 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.952 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:07.952 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.165369+0000", 2024-09-20T22:16:07.952 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6319999999999999e-06 2024-09-20T22:16:07.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.952 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.952 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.952 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.171567+0000", 2024-09-20T22:16:07.952 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061976460000000002 2024-09-20T22:16:07.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.953 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.953 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:07.953 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.171575+0000", 2024-09-20T22:16:07.953 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2390000000000002e-06 2024-09-20T22:16:07.953 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.953 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.953 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.953 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.180431+0000", 2024-09-20T22:16:07.953 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0088558569999999 2024-09-20T22:16:07.953 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.953 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.953 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:16:07.953 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.180570+0000", 2024-09-20T22:16:07.953 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013935799999999999 2024-09-20T22:16:07.953 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.954 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.954 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:16:07.954 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.180897+0000", 2024-09-20T22:16:07.954 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00032708599999999998 2024-09-20T22:16:07.954 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.954 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.954 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:16:07.954 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.184575+0000", 2024-09-20T22:16:07.954 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0036776790000000001 2024-09-20T22:16:07.954 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.954 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.954 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:07.954 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185202+0000", 2024-09-20T22:16:07.954 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00062773099999999997 2024-09-20T22:16:07.955 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.955 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.955 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:07.955 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185253+0000", 2024-09-20T22:16:07.955 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0742999999999999e-05 2024-09-20T22:16:07.955 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.955 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.955 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:16:07.955 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185303+0000", 2024-09-20T22:16:07.955 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9339000000000003e-05 2024-09-20T22:16:07.955 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.955 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.955 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:07.955 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185410+0000", 2024-09-20T22:16:07.956 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010718299999999999 2024-09-20T22:16:07.956 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.956 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:07.956 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.956 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.956 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.956 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:261986 7.17 7:ebad35fd:::benchmark_data_smithi060_78800_object128953:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:16:07.956 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.200930+0000", 2024-09-20T22:16:07.956 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 511.71915113099999, 2024-09-20T22:16:07.956 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98696583999999998, 2024-09-20T22:16:07.956 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:07.956 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:07.957 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:07.957 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:16:07.957 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:16:07.957 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:16:07.957 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 261986 2024-09-20T22:16:07.957 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.957 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:07.957 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.957 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:07.957 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200930+0000", 2024-09-20T22:16:07.957 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.957 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.957 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.957 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:07.958 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200930+0000", 2024-09-20T22:16:07.958 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.958 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.958 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.958 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:07.958 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200931+0000", 2024-09-20T22:16:07.958 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8800000000000001e-07 2024-09-20T22:16:07.958 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.958 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.958 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:07.958 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200936+0000", 2024-09-20T22:16:07.958 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3939999999999998e-06 2024-09-20T22:16:07.958 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.958 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.959 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:07.959 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200946+0000", 2024-09-20T22:16:07.959 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0706999999999999e-05 2024-09-20T22:16:07.959 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.959 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.959 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:07.959 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.200953+0000", 2024-09-20T22:16:07.959 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9260000000000003e-06 2024-09-20T22:16:07.959 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.959 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.959 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.959 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.201017+0000", 2024-09-20T22:16:07.959 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3793000000000004e-05 2024-09-20T22:16:07.960 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.960 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.960 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:07.960 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.201028+0000", 2024-09-20T22:16:07.960 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0577000000000001e-05 2024-09-20T22:16:07.960 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.960 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.960 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.960 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.183246+0000", 2024-09-20T22:16:07.960 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98221816100000003 2024-09-20T22:16:07.960 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.960 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.960 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:16:07.960 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.183318+0000", 2024-09-20T22:16:07.961 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1890000000000005e-05 2024-09-20T22:16:07.961 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.961 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.961 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:16:07.961 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.183503+0000", 2024-09-20T22:16:07.961 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00018529399999999999 2024-09-20T22:16:07.961 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.961 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.961 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:16:07.961 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185423+0000", 2024-09-20T22:16:07.961 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0019203390000000001 2024-09-20T22:16:07.962 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.962 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.962 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:07.962 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185478+0000", 2024-09-20T22:16:07.962 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5041999999999999e-05 2024-09-20T22:16:07.962 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.962 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.962 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:07.962 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.187745+0000", 2024-09-20T22:16:07.962 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0022662239999999998 2024-09-20T22:16:07.962 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.962 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.962 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:16:07.962 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.187795+0000", 2024-09-20T22:16:07.963 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0668999999999999e-05 2024-09-20T22:16:07.963 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.963 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.963 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:07.963 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.187896+0000", 2024-09-20T22:16:07.963 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100936 2024-09-20T22:16:07.963 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.963 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:07.963 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.963 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.963 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.963 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.24667.0:262055 7.17 7:eca2ebed:::benchmark_data_smithi060_78800_object129022:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e169)", 2024-09-20T22:16:07.963 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:07:36.253316+0000", 2024-09-20T22:16:07.963 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 511.66676599200002, 2024-09-20T22:16:07.964 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93859176700000002, 2024-09-20T22:16:07.964 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:07.964 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:07.964 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:07.964 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:16:07.964 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.24667", 2024-09-20T22:16:07.964 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:16:07.964 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 262055 2024-09-20T22:16:07.964 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.964 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:07.964 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.964 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:07.964 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253316+0000", 2024-09-20T22:16:07.964 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.965 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.965 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.965 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:07.965 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253316+0000", 2024-09-20T22:16:07.965 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.965 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.965 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.965 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:07.965 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253316+0000", 2024-09-20T22:16:07.965 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2999999999999999e-07 2024-09-20T22:16:07.965 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.965 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.965 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:07.965 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253320+0000", 2024-09-20T22:16:07.965 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9129999999999996e-06 2024-09-20T22:16:07.966 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.966 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.966 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:07.966 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253333+0000", 2024-09-20T22:16:07.966 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2762000000000001e-05 2024-09-20T22:16:07.966 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.966 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.966 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:07.966 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253340+0000", 2024-09-20T22:16:07.966 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0180000000000004e-06 2024-09-20T22:16:07.966 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.966 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.966 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.966 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253402+0000", 2024-09-20T22:16:07.967 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2211000000000005e-05 2024-09-20T22:16:07.967 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.967 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.967 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:07.967 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:36.253409+0000", 2024-09-20T22:16:07.967 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.985e-06 2024-09-20T22:16:07.967 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.967 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.967 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.967 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185521+0000", 2024-09-20T22:16:07.967 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93211197400000001 2024-09-20T22:16:07.967 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.968 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.968 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:16:07.968 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185620+0000", 2024-09-20T22:16:07.968 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9162999999999999e-05 2024-09-20T22:16:07.968 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.968 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.968 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:16:07.968 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.185870+0000", 2024-09-20T22:16:07.968 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00024926499999999998 2024-09-20T22:16:07.968 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.968 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.968 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:07.968 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191631+0000", 2024-09-20T22:16:07.969 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.005761375 2024-09-20T22:16:07.969 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.969 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.969 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:16:07.969 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191650+0000", 2024-09-20T22:16:07.969 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8533999999999999e-05 2024-09-20T22:16:07.969 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.969 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.969 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:07.969 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191717+0000", 2024-09-20T22:16:07.969 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7731999999999995e-05 2024-09-20T22:16:07.969 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.969 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.969 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:16:07.969 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191780+0000", 2024-09-20T22:16:07.970 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2447999999999996e-05 2024-09-20T22:16:07.970 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.970 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.970 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:07.970 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:07:37.191907+0000", 2024-09-20T22:16:07.970 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012755700000000001 2024-09-20T22:16:07.970 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.970 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:07.970 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.970 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.970 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.970 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:106968 8.15 8:a8961342:::benchmark_data_smithi060_111051_object106967:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:16:07.970 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.566651+0000", 2024-09-20T22:16:07.971 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 417.35343057799997, 2024-09-20T22:16:07.971 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9227000670000001, 2024-09-20T22:16:07.971 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:07.971 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:07.971 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:16:07.971 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:16:07.971 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:16:07.971 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:07.971 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 106968 2024-09-20T22:16:07.971 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.971 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:07.971 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.971 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:07.971 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566651+0000", 2024-09-20T22:16:07.972 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.972 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.972 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:07.972 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566651+0000", 2024-09-20T22:16:07.972 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.972 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.972 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:07.972 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566653+0000", 2024-09-20T22:16:07.972 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.674e-06 2024-09-20T22:16:07.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.972 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.973 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:07.973 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566809+0000", 2024-09-20T22:16:07.973 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015637600000000001 2024-09-20T22:16:07.973 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.973 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.973 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:07.973 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566820+0000", 2024-09-20T22:16:07.973 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.097e-05 2024-09-20T22:16:07.973 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.973 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.973 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:07.973 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566830+0000", 2024-09-20T22:16:07.973 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4690000000000003e-06 2024-09-20T22:16:07.973 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.974 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.974 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.974 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.567309+0000", 2024-09-20T22:16:07.974 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00047940599999999998 2024-09-20T22:16:07.974 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.974 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.974 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:07.974 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.567321+0000", 2024-09-20T22:16:07.974 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2451e-05 2024-09-20T22:16:07.974 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.974 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.974 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.974 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417364+0000", 2024-09-20T22:16:07.974 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.85004267200000005 2024-09-20T22:16:07.975 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.975 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.975 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:07.975 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417390+0000", 2024-09-20T22:16:07.975 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.5959999999999999e-05 2024-09-20T22:16:07.975 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.975 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.975 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.975 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489254+0000", 2024-09-20T22:16:07.975 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0718642620000001 2024-09-20T22:16:07.975 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.975 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.975 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:07.976 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489351+0000", 2024-09-20T22:16:07.976 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6827e-05 2024-09-20T22:16:07.976 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.976 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:07.976 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.976 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.976 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.976 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:106993 8.15 8:ae0aa858:::benchmark_data_smithi060_111051_object106992:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:16:07.976 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.600689+0000", 2024-09-20T22:16:07.976 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 417.31939291399999, 2024-09-20T22:16:07.976 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.911249717, 2024-09-20T22:16:07.976 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:07.976 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:07.976 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:07.977 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:16:07.977 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:16:07.977 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:07.977 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 106993 2024-09-20T22:16:07.977 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.977 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:07.977 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.977 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:07.977 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600689+0000", 2024-09-20T22:16:07.977 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.977 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.977 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.977 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:07.977 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600689+0000", 2024-09-20T22:16:07.978 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.978 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.978 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.978 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:07.978 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600691+0000", 2024-09-20T22:16:07.978 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.052e-06 2024-09-20T22:16:07.978 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.978 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.978 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:07.978 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600724+0000", 2024-09-20T22:16:07.978 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.3429000000000001e-05 2024-09-20T22:16:07.978 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.978 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.978 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:07.979 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600747+0000", 2024-09-20T22:16:07.979 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2663e-05 2024-09-20T22:16:07.979 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.979 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.979 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:07.979 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600756+0000", 2024-09-20T22:16:07.979 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4099999999999997e-06 2024-09-20T22:16:07.979 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.979 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.979 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.979 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600862+0000", 2024-09-20T22:16:07.979 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000106135 2024-09-20T22:16:07.979 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.979 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.980 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:07.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:07.981 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600873+0000", 2024-09-20T22:16:07.981 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0794e-05 2024-09-20T22:16:07.981 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.981 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.981 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.981 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417486+0000", 2024-09-20T22:16:07.982 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.81661262800000001 2024-09-20T22:16:07.982 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.982 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.982 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:07.982 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417495+0000", 2024-09-20T22:16:07.982 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1160000000000005e-06 2024-09-20T22:16:07.982 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.982 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.982 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.982 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489416+0000", 2024-09-20T22:16:07.982 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.071920784 2024-09-20T22:16:07.982 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.982 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.982 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:16:07.983 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489581+0000", 2024-09-20T22:16:07.983 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00016577399999999999 2024-09-20T22:16:07.983 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.983 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.983 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,4,6", 2024-09-20T22:16:07.983 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489742+0000", 2024-09-20T22:16:07.983 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000160204 2024-09-20T22:16:07.983 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.983 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.983 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:16:07.983 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.494442+0000", 2024-09-20T22:16:07.983 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0047004660000000004 2024-09-20T22:16:07.983 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.983 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.984 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:07.984 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.495057+0000", 2024-09-20T22:16:07.984 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000614677 2024-09-20T22:16:07.984 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.984 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.984 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:07.984 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.495091+0000", 2024-09-20T22:16:07.984 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4581e-05 2024-09-20T22:16:07.984 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.984 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.984 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:07.984 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.511743+0000", 2024-09-20T22:16:07.984 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.016651142000000001 2024-09-20T22:16:07.984 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.985 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.985 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:16:07.985 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.511810+0000", 2024-09-20T22:16:07.985 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7065999999999998e-05 2024-09-20T22:16:07.985 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.985 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.985 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:07.985 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.511938+0000", 2024-09-20T22:16:07.985 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000128796 2024-09-20T22:16:07.985 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.985 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:07.985 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.985 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.985 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.986 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:107013 8.15 8:ab48afac:::benchmark_data_smithi060_111051_object107012:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:16:07.986 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.648456+0000", 2024-09-20T22:16:07.986 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 417.271625212, 2024-09-20T22:16:07.986 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8646870719999999, 2024-09-20T22:16:07.986 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:07.986 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:07.986 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:07.986 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:16:07.986 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:16:07.986 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:07.986 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 107013 2024-09-20T22:16:07.986 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.986 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:07.987 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.987 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:07.987 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648456+0000", 2024-09-20T22:16:07.987 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.987 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.987 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:07.987 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648456+0000", 2024-09-20T22:16:07.987 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.987 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.987 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:07.987 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648459+0000", 2024-09-20T22:16:07.987 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2000000000000001e-06 2024-09-20T22:16:07.988 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.988 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.988 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:07.988 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648501+0000", 2024-09-20T22:16:07.988 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2453000000000002e-05 2024-09-20T22:16:07.988 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.988 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.988 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:07.988 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648525+0000", 2024-09-20T22:16:07.988 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.4034000000000001e-05 2024-09-20T22:16:07.988 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.988 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.988 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:07.988 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648535+0000", 2024-09-20T22:16:07.989 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0355e-05 2024-09-20T22:16:07.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.989 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.989 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.989 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648641+0000", 2024-09-20T22:16:07.989 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000105448 2024-09-20T22:16:07.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.989 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.989 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:07.989 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648654+0000", 2024-09-20T22:16:07.989 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2904999999999999e-05 2024-09-20T22:16:07.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.989 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.990 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.990 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417525+0000", 2024-09-20T22:16:07.990 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.76887095999999999 2024-09-20T22:16:07.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.990 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.990 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:07.990 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417959+0000", 2024-09-20T22:16:07.990 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00043430399999999999 2024-09-20T22:16:07.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.990 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.990 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.990 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.492147+0000", 2024-09-20T22:16:07.990 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0741883080000001 2024-09-20T22:16:07.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.991 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.991 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:16:07.991 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.492291+0000", 2024-09-20T22:16:07.991 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014341700000000001 2024-09-20T22:16:07.991 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.991 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.991 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,4,6", 2024-09-20T22:16:07.991 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.492463+0000", 2024-09-20T22:16:07.991 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017243400000000001 2024-09-20T22:16:07.991 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.991 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.991 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:07.991 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.499029+0000", 2024-09-20T22:16:07.991 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0065655460000000002 2024-09-20T22:16:07.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.992 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.992 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:07.992 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.499078+0000", 2024-09-20T22:16:07.992 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9527000000000002e-05 2024-09-20T22:16:07.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.992 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.992 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:16:07.992 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.499106+0000", 2024-09-20T22:16:07.992 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7280999999999999e-05 2024-09-20T22:16:07.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.992 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.992 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:07.992 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.512989+0000", 2024-09-20T22:16:07.993 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.013883929 2024-09-20T22:16:07.993 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.993 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.993 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:16:07.993 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.513048+0000", 2024-09-20T22:16:07.993 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8261999999999999e-05 2024-09-20T22:16:07.993 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.993 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.993 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:07.993 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.513143+0000", 2024-09-20T22:16:07.993 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5709000000000001e-05 2024-09-20T22:16:07.993 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.993 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:07.993 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:07.993 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.994 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.994 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163403 8.17 8:ebf82752:::benchmark_data_smithi060_111051_object43404:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:16:07.994 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.477864+0000", 2024-09-20T22:16:07.994 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 358.44221790900002, 2024-09-20T22:16:07.994 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0006944760000001, 2024-09-20T22:16:07.994 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:07.994 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:07.994 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:07.994 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:16:07.994 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:16:07.994 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:07.994 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163403 2024-09-20T22:16:07.994 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.995 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:07.995 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.995 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:07.995 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477864+0000", 2024-09-20T22:16:07.995 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.995 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.995 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.995 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:07.995 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477864+0000", 2024-09-20T22:16:07.995 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:07.995 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.995 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.995 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:07.995 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477865+0000", 2024-09-20T22:16:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4279999999999999e-06 2024-09-20T22:16:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477870+0000", 2024-09-20T22:16:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8899999999999998e-06 2024-09-20T22:16:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477881+0000", 2024-09-20T22:16:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1139e-05 2024-09-20T22:16:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.996 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477888+0000", 2024-09-20T22:16:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9779999999999999e-06 2024-09-20T22:16:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477943+0000", 2024-09-20T22:16:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5229000000000003e-05 2024-09-20T22:16:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477952+0000", 2024-09-20T22:16:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5420000000000005e-06 2024-09-20T22:16:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.997 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.473740+0000", 2024-09-20T22:16:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.995788063 2024-09-20T22:16:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:16:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.473826+0000", 2024-09-20T22:16:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6131000000000006e-05 2024-09-20T22:16:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:16:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.473905+0000", 2024-09-20T22:16:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8899000000000005e-05 2024-09-20T22:16:07.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:16:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478387+0000", 2024-09-20T22:16:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0044816680000000003 2024-09-20T22:16:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478428+0000", 2024-09-20T22:16:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1091999999999999e-05 2024-09-20T22:16:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:07.999 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478481+0000", 2024-09-20T22:16:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3628000000000002e-05 2024-09-20T22:16:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:16:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478514+0000", 2024-09-20T22:16:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2880999999999999e-05 2024-09-20T22:16:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478558+0000", 2024-09-20T22:16:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3908e-05 2024-09-20T22:16:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:08.000 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163409 8.17 8:ee9c487d:::benchmark_data_smithi060_111051_object43410:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:16:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.483034+0000", 2024-09-20T22:16:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 358.43704754700002, 2024-09-20T22:16:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.99573102899999999, 2024-09-20T22:16:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:16:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:16:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163409 2024-09-20T22:16:08.001 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483034+0000", 2024-09-20T22:16:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483034+0000", 2024-09-20T22:16:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.002 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483036+0000", 2024-09-20T22:16:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7940000000000001e-06 2024-09-20T22:16:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483041+0000", 2024-09-20T22:16:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.446e-06 2024-09-20T22:16:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483060+0000", 2024-09-20T22:16:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8258e-05 2024-09-20T22:16:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.003 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483068+0000", 2024-09-20T22:16:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5390000000000007e-06 2024-09-20T22:16:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483127+0000", 2024-09-20T22:16:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8542999999999998e-05 2024-09-20T22:16:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483135+0000", 2024-09-20T22:16:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.926e-06 2024-09-20T22:16:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.004 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.475590+0000", 2024-09-20T22:16:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.99245572400000004 2024-09-20T22:16:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:16:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.475685+0000", 2024-09-20T22:16:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4228999999999995e-05 2024-09-20T22:16:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:16:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.475757+0000", 2024-09-20T22:16:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2712000000000004e-05 2024-09-20T22:16:08.005 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:16:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478623+0000", 2024-09-20T22:16:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028660069999999998 2024-09-20T22:16:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478671+0000", 2024-09-20T22:16:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7890999999999999e-05 2024-09-20T22:16:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:08.006 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478701+0000", 2024-09-20T22:16:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.9986999999999999e-05 2024-09-20T22:16:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:16:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478727+0000", 2024-09-20T22:16:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.6211999999999999e-05 2024-09-20T22:16:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478765+0000", 2024-09-20T22:16:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7761e-05 2024-09-20T22:16:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.007 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163412 8.17 8:ed62347b:::benchmark_data_smithi060_111051_object43413:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:16:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.487680+0000", 2024-09-20T22:16:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 358.43240110800002, 2024-09-20T22:16:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.99125577600000003, 2024-09-20T22:16:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:16:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:16:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163412 2024-09-20T22:16:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.008 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487680+0000", 2024-09-20T22:16:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487680+0000", 2024-09-20T22:16:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487682+0000", 2024-09-20T22:16:08.009 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.759e-06 2024-09-20T22:16:08.010 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.010 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.010 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:08.010 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487688+0000", 2024-09-20T22:16:08.010 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5060000000000003e-06 2024-09-20T22:16:08.010 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.010 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.010 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:08.010 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487701+0000", 2024-09-20T22:16:08.010 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3225e-05 2024-09-20T22:16:08.010 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.010 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.010 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:08.010 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487709+0000", 2024-09-20T22:16:08.011 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5449999999999998e-06 2024-09-20T22:16:08.011 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.011 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.011 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:08.011 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487767+0000", 2024-09-20T22:16:08.011 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8387999999999997e-05 2024-09-20T22:16:08.011 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.011 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.011 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:08.011 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487774+0000", 2024-09-20T22:16:08.011 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.844e-06 2024-09-20T22:16:08.011 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.011 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.011 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:08.012 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.477116+0000", 2024-09-20T22:16:08.012 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98934203200000004 2024-09-20T22:16:08.012 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.012 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.012 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:16:08.012 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.477189+0000", 2024-09-20T22:16:08.012 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3220999999999996e-05 2024-09-20T22:16:08.012 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.012 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.012 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:16:08.012 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.477253+0000", 2024-09-20T22:16:08.012 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3770999999999996e-05 2024-09-20T22:16:08.012 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.012 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.013 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:16:08.013 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478774+0000", 2024-09-20T22:16:08.013 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0015216940000000001 2024-09-20T22:16:08.013 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.013 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.013 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:08.013 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478827+0000", 2024-09-20T22:16:08.013 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2262000000000002e-05 2024-09-20T22:16:08.013 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.013 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.013 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:08.013 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478872+0000", 2024-09-20T22:16:08.013 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5398999999999997e-05 2024-09-20T22:16:08.013 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.014 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.014 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:16:08.014 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478899+0000", 2024-09-20T22:16:08.014 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7146999999999999e-05 2024-09-20T22:16:08.014 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.014 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.014 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:08.014 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478936+0000", 2024-09-20T22:16:08.014 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6983000000000001e-05 2024-09-20T22:16:08.014 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.014 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:08.014 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.014 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.014 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.015 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163455 8.17 8:ef455ef1:::benchmark_data_smithi060_111051_object43456:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:16:08.015 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.524754+0000", 2024-09-20T22:16:08.015 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 358.39532734800002, 2024-09-20T22:16:08.015 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.96086008199999995, 2024-09-20T22:16:08.015 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:08.015 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:08.015 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:08.015 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:16:08.015 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:16:08.015 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:08.015 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163455 2024-09-20T22:16:08.015 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.015 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:08.015 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.015 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:08.016 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524754+0000", 2024-09-20T22:16:08.016 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:08.016 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.016 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.016 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:08.016 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524754+0000", 2024-09-20T22:16:08.016 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:08.016 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.016 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.016 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:08.016 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524756+0000", 2024-09-20T22:16:08.016 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5850000000000001e-06 2024-09-20T22:16:08.016 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.016 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.017 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:08.017 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524761+0000", 2024-09-20T22:16:08.017 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6699999999999999e-06 2024-09-20T22:16:08.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.017 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.017 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:08.017 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524776+0000", 2024-09-20T22:16:08.017 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4416000000000001e-05 2024-09-20T22:16:08.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.017 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.017 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:08.017 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524783+0000", 2024-09-20T22:16:08.017 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9009999999999997e-06 2024-09-20T22:16:08.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.018 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.018 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:08.018 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524851+0000", 2024-09-20T22:16:08.018 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8090999999999996e-05 2024-09-20T22:16:08.018 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.018 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.018 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:08.018 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524858+0000", 2024-09-20T22:16:08.018 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9299999999999997e-06 2024-09-20T22:16:08.018 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.018 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.018 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:08.018 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478989+0000", 2024-09-20T22:16:08.018 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.954130849 2024-09-20T22:16:08.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.019 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.019 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:16:08.019 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.479103+0000", 2024-09-20T22:16:08.019 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000114608 2024-09-20T22:16:08.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.019 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.019 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:16:08.019 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.479218+0000", 2024-09-20T22:16:08.019 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000115146 2024-09-20T22:16:08.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.019 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.019 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:08.019 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485313+0000", 2024-09-20T22:16:08.020 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0060945410000000002 2024-09-20T22:16:08.020 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.020 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.020 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:08.020 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485397+0000", 2024-09-20T22:16:08.020 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4055999999999996e-05 2024-09-20T22:16:08.020 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.020 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.020 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:16:08.020 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485483+0000", 2024-09-20T22:16:08.020 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5507000000000007e-05 2024-09-20T22:16:08.020 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.020 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.020 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:16:08.020 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485567+0000", 2024-09-20T22:16:08.021 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4388999999999995e-05 2024-09-20T22:16:08.021 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.021 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.021 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:08.021 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485614+0000", 2024-09-20T22:16:08.021 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7392999999999998e-05 2024-09-20T22:16:08.021 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.021 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:08.021 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.021 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.021 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.021 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163489 8.17 8:eb24224e:::benchmark_data_smithi060_111051_object43490:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:16:08.021 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.552125+0000", 2024-09-20T22:16:08.021 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 358.36795660899998, 2024-09-20T22:16:08.022 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93738651900000003, 2024-09-20T22:16:08.022 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:08.022 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:08.022 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:08.022 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:16:08.022 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:16:08.022 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:08.022 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163489 2024-09-20T22:16:08.022 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.022 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:08.022 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.022 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:08.022 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552125+0000", 2024-09-20T22:16:08.023 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:08.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.023 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.023 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:08.023 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552125+0000", 2024-09-20T22:16:08.023 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:08.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.023 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.023 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:08.023 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552126+0000", 2024-09-20T22:16:08.023 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2810000000000001e-06 2024-09-20T22:16:08.023 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.023 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.023 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:08.024 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552131+0000", 2024-09-20T22:16:08.024 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9450000000000001e-06 2024-09-20T22:16:08.024 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.024 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.024 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:08.024 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552143+0000", 2024-09-20T22:16:08.024 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1902999999999999e-05 2024-09-20T22:16:08.024 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.024 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.024 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:08.024 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552150+0000", 2024-09-20T22:16:08.024 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2609999999999996e-06 2024-09-20T22:16:08.024 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.024 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.025 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:08.025 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552210+0000", 2024-09-20T22:16:08.025 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9463999999999999e-05 2024-09-20T22:16:08.025 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.025 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.025 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:08.025 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552218+0000", 2024-09-20T22:16:08.025 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4460000000000001e-06 2024-09-20T22:16:08.025 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.025 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.025 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:08.025 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.480425+0000", 2024-09-20T22:16:08.025 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.928206859 2024-09-20T22:16:08.025 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.026 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.026 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:16:08.026 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.480526+0000", 2024-09-20T22:16:08.026 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100388 2024-09-20T22:16:08.026 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.026 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.026 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:16:08.026 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.480628+0000", 2024-09-20T22:16:08.026 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000102051 2024-09-20T22:16:08.026 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.026 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.026 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:08.026 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485464+0000", 2024-09-20T22:16:08.026 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0048367150000000001 2024-09-20T22:16:08.027 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.027 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.027 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:16:08.027 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485625+0000", 2024-09-20T22:16:08.027 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000160278 2024-09-20T22:16:08.027 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.027 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.027 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:08.027 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489410+0000", 2024-09-20T22:16:08.027 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0037854659999999999 2024-09-20T22:16:08.027 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.027 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.027 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:16:08.028 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489468+0000", 2024-09-20T22:16:08.028 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7883000000000001e-05 2024-09-20T22:16:08.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.028 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.028 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:08.028 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489512+0000", 2024-09-20T22:16:08.028 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3578999999999997e-05 2024-09-20T22:16:08.028 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.028 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:08.028 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.028 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.028 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.028 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163500 8.17 8:ef6c367b:::benchmark_data_smithi060_111051_object43501:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:16:08.028 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.560977+0000", 2024-09-20T22:16:08.028 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 358.35910478099998, 2024-09-20T22:16:08.029 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93477054299999995, 2024-09-20T22:16:08.029 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:08.029 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:08.029 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:08.029 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:16:08.029 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:16:08.029 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:08.029 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163500 2024-09-20T22:16:08.029 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.029 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:08.029 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.029 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:08.029 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.560977+0000", 2024-09-20T22:16:08.029 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:08.030 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.030 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.030 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:08.030 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.560977+0000", 2024-09-20T22:16:08.030 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:08.030 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.030 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.030 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:08.030 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.560978+0000", 2024-09-20T22:16:08.030 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.688e-06 2024-09-20T22:16:08.030 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.030 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.030 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:08.031 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.560984+0000", 2024-09-20T22:16:08.031 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7939999999999999e-06 2024-09-20T22:16:08.031 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.031 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.031 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:08.031 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.560997+0000", 2024-09-20T22:16:08.031 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.292e-05 2024-09-20T22:16:08.031 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.031 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.031 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:08.031 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.561006+0000", 2024-09-20T22:16:08.031 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2970000000000004e-06 2024-09-20T22:16:08.031 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.031 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.032 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:08.032 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.561071+0000", 2024-09-20T22:16:08.032 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5277000000000005e-05 2024-09-20T22:16:08.032 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.032 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.032 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:08.032 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.561080+0000", 2024-09-20T22:16:08.032 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6989999999999998e-06 2024-09-20T22:16:08.032 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.032 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.032 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:08.032 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.481917+0000", 2024-09-20T22:16:08.032 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.92083670500000003 2024-09-20T22:16:08.032 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.033 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.033 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:16:08.033 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.482017+0000", 2024-09-20T22:16:08.033 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9609000000000001e-05 2024-09-20T22:16:08.033 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.033 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.033 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:16:08.033 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.482117+0000", 2024-09-20T22:16:08.033 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9945999999999995e-05 2024-09-20T22:16:08.033 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.033 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.033 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:16:08.033 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485631+0000", 2024-09-20T22:16:08.033 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0035139049999999999 2024-09-20T22:16:08.033 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.034 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.034 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:08.034 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489611+0000", 2024-09-20T22:16:08.034 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0039801649999999999 2024-09-20T22:16:08.034 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.034 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.034 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:08.034 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495646+0000", 2024-09-20T22:16:08.034 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0060353660000000003 2024-09-20T22:16:08.034 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.034 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.034 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:16:08.034 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495704+0000", 2024-09-20T22:16:08.035 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7908999999999997e-05 2024-09-20T22:16:08.035 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.035 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.035 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:08.035 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495747+0000", 2024-09-20T22:16:08.035 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3263000000000002e-05 2024-09-20T22:16:08.035 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.035 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:08.035 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.035 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.035 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.035 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163521 8.17 8:edfbcf2c:::benchmark_data_smithi060_111051_object43522:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:16:08.036 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:08.036 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:08.036 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.577053+0000", 2024-09-20T22:16:08.037 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 358.34302833300001, 2024-09-20T22:16:08.037 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.91884361299999995, 2024-09-20T22:16:08.037 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:08.037 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:08.037 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:08.037 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:16:08.037 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:16:08.037 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:08.037 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163521 2024-09-20T22:16:08.037 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.037 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:08.037 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.037 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:08.037 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577053+0000", 2024-09-20T22:16:08.038 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:08.038 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.038 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.038 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:08.038 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577053+0000", 2024-09-20T22:16:08.038 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:08.038 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.038 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.038 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:08.038 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577055+0000", 2024-09-20T22:16:08.038 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3039999999999999e-06 2024-09-20T22:16:08.038 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.038 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.039 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:08.039 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577060+0000", 2024-09-20T22:16:08.039 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4319999999999998e-06 2024-09-20T22:16:08.039 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.039 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.039 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:08.039 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577071+0000", 2024-09-20T22:16:08.039 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1258e-05 2024-09-20T22:16:08.039 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.039 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.039 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:08.039 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577079+0000", 2024-09-20T22:16:08.039 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.374e-06 2024-09-20T22:16:08.039 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.039 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.040 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:08.040 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577177+0000", 2024-09-20T22:16:08.040 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7923000000000004e-05 2024-09-20T22:16:08.040 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.040 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.040 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:08.040 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577197+0000", 2024-09-20T22:16:08.040 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0001999999999999e-05 2024-09-20T22:16:08.040 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.040 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.040 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:08.040 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.483671+0000", 2024-09-20T22:16:08.040 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.90647459900000005 2024-09-20T22:16:08.041 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.041 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.041 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:16:08.041 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.483791+0000", 2024-09-20T22:16:08.041 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000119743 2024-09-20T22:16:08.041 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.041 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.041 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:16:08.041 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.483891+0000", 2024-09-20T22:16:08.041 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010016700000000001 2024-09-20T22:16:08.041 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.041 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.041 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:16:08.041 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489543+0000", 2024-09-20T22:16:08.042 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0056521109999999996 2024-09-20T22:16:08.042 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.042 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.042 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:08.042 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489657+0000", 2024-09-20T22:16:08.042 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000114249 2024-09-20T22:16:08.042 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.042 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.042 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:08.042 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495803+0000", 2024-09-20T22:16:08.042 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061460059999999999 2024-09-20T22:16:08.042 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.042 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.043 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:16:08.043 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495857+0000", 2024-09-20T22:16:08.043 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3408000000000001e-05 2024-09-20T22:16:08.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.043 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.043 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:08.043 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495897+0000", 2024-09-20T22:16:08.043 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0037000000000003e-05 2024-09-20T22:16:08.043 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.043 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:08.043 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.043 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.043 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.043 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163525 8.17 8:ea732579:::benchmark_data_smithi060_111051_object43526:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:16:08.044 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.577673+0000", 2024-09-20T22:16:08.044 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 358.34240902800002, 2024-09-20T22:16:08.044 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.91833821500000001, 2024-09-20T22:16:08.044 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:08.044 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:08.044 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:08.044 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:16:08.044 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:16:08.044 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:08.044 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163525 2024-09-20T22:16:08.044 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.044 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:08.045 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.045 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:08.045 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577673+0000", 2024-09-20T22:16:08.045 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:08.045 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.045 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.045 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:08.045 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577673+0000", 2024-09-20T22:16:08.045 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:08.045 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.045 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.045 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:08.045 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577673+0000", 2024-09-20T22:16:08.045 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1699999999999997e-07 2024-09-20T22:16:08.046 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.046 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.046 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:08.046 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577678+0000", 2024-09-20T22:16:08.046 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3200000000000001e-06 2024-09-20T22:16:08.046 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.046 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.046 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:08.046 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577689+0000", 2024-09-20T22:16:08.046 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0735e-05 2024-09-20T22:16:08.046 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.046 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.046 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:08.047 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577695+0000", 2024-09-20T22:16:08.047 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7549999999999997e-06 2024-09-20T22:16:08.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.047 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.047 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:08.047 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577745+0000", 2024-09-20T22:16:08.047 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9223e-05 2024-09-20T22:16:08.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.047 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.047 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:16:08.047 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577752+0000", 2024-09-20T22:16:08.047 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3300000000000001e-06 2024-09-20T22:16:08.047 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.047 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.048 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:08.048 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485694+0000", 2024-09-20T22:16:08.048 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.90794208499999995 2024-09-20T22:16:08.048 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.048 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.048 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:16:08.048 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485801+0000", 2024-09-20T22:16:08.048 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010699700000000001 2024-09-20T22:16:08.048 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.048 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.048 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:16:08.048 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485921+0000", 2024-09-20T22:16:08.048 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000120025 2024-09-20T22:16:08.048 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.049 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.049 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:08.049 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489730+0000", 2024-09-20T22:16:08.049 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003809046 2024-09-20T22:16:08.049 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.049 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.049 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:16:08.049 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489745+0000", 2024-09-20T22:16:08.049 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4711e-05 2024-09-20T22:16:08.049 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.049 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.049 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:08.049 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495935+0000", 2024-09-20T22:16:08.049 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061903419999999997 2024-09-20T22:16:08.049 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.050 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.050 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:16:08.050 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495975+0000", 2024-09-20T22:16:08.050 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0441000000000001e-05 2024-09-20T22:16:08.050 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.050 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.050 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:08.050 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.496011+0000", 2024-09-20T22:16:08.050 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5287999999999998e-05 2024-09-20T22:16:08.050 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.050 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:08.050 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.050 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.050 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.051 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.16005.0:77005 10.9 e395/370 10:91259046:::benchmark_data_smithi060_167766_object77004:head v 395'4865, mlcod=395'4862)", 2024-09-20T22:16:08.051 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:16:04.242729+0000", 2024-09-20T22:16:08.051 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 3.677352639, 2024-09-20T22:16:08.051 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.23358954400000001, 2024-09-20T22:16:08.051 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:08.051 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:08.051 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:08.051 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:08.051 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.051 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:08.051 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.242729+0000", 2024-09-20T22:16:08.051 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:08.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.052 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:08.052 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.242729+0000", 2024-09-20T22:16:08.052 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:08.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.052 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:08.052 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.242730+0000", 2024-09-20T22:16:08.052 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7199999999999998e-07 2024-09-20T22:16:08.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.052 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:08.052 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.242749+0000", 2024-09-20T22:16:08.052 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9083999999999999e-05 2024-09-20T22:16:08.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.053 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.053 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:08.053 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.242759+0000", 2024-09-20T22:16:08.053 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0208e-05 2024-09-20T22:16:08.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.053 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.053 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:08.053 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.242766+0000", 2024-09-20T22:16:08.053 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8510000000000001e-06 2024-09-20T22:16:08.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.053 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.053 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:08.053 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.467751+0000", 2024-09-20T22:16:08.053 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.22498507400000001 2024-09-20T22:16:08.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.054 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.054 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:16:08.054 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.467770+0000", 2024-09-20T22:16:08.054 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9318999999999999e-05 2024-09-20T22:16:08.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.054 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.054 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:16:08.054 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.476274+0000", 2024-09-20T22:16:08.054 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0085032600000000003 2024-09-20T22:16:08.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.054 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.054 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:08.055 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.476318+0000", 2024-09-20T22:16:08.055 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4975999999999998e-05 2024-09-20T22:16:08.055 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.055 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:08.055 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.055 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.16005.0:77013 10.9 e395/370 10:9a8ed282:::benchmark_data_smithi060_167766_object77012:head v 395'4866, mlcod=395'4862)", 2024-09-20T22:16:08.055 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:16:04.247828+0000", 2024-09-20T22:16:08.055 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 3.6722533510000002, 2024-09-20T22:16:08.055 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.238548656, 2024-09-20T22:16:08.055 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:16:08.055 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:16:08.055 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:08.056 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:16:08.056 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.056 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:16:08.056 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.247828+0000", 2024-09-20T22:16:08.056 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:08.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.056 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.056 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:16:08.056 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.247828+0000", 2024-09-20T22:16:08.056 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:16:08.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.056 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.056 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:16:08.056 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.247829+0000", 2024-09-20T22:16:08.057 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.23e-07 2024-09-20T22:16:08.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.057 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:16:08.057 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.247852+0000", 2024-09-20T22:16:08.057 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2705999999999999e-05 2024-09-20T22:16:08.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.057 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:16:08.057 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.467712+0000", 2024-09-20T22:16:08.057 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.21986037 2024-09-20T22:16:08.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.057 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:16:08.057 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.467726+0000", 2024-09-20T22:16:08.058 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3495999999999999e-05 2024-09-20T22:16:08.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.058 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.058 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:16:08.058 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.471324+0000", 2024-09-20T22:16:08.058 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0035982060000000001 2024-09-20T22:16:08.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.058 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.058 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:16:08.058 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.471346+0000", 2024-09-20T22:16:08.058 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.211e-05 2024-09-20T22:16:08.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.058 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.059 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:16:08.059 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.486238+0000", 2024-09-20T22:16:08.059 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.014892036000000001 2024-09-20T22:16:08.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:16:08.059 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:16:08.059 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:16:08.059 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.486377+0000", 2024-09-20T22:16:08.059 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000139009 2024-09-20T22:16:08.059 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.059 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:08.059 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.059 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:16:08.059 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:16:08.059 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:16:08.137 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:08.158 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:08.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:08.159 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:08.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:08.301 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:08.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:08.302 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:08.403 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:08.441 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:08.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:08.442 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:08.543 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:08.577 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:08.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:08.578 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:08.678 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:08.715 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:08.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:08.716 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:08.817 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:08.859 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:08.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:08.860 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:08.924 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 51 16 81900 81884 100.329 75.75 0.0062149 0.00996197 2024-09-20T22:16:08.961 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:09.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:09.010 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:09.010 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:09.111 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:09.153 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T22:16:09.164 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:09.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:09.165 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:09.266 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:09.313 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:09.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:09.314 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:09.415 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:09.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:09.453 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:09.453 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:09.504 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:09 smithi060 ceph-mon[29694]: pgmap v712: 168 pgs: 168 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-20T22:16:09.506 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:09 smithi060 ceph-mon[40290]: pgmap v712: 168 pgs: 168 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-20T22:16:09.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:09 smithi067 ceph-mon[33511]: pgmap v712: 168 pgs: 168 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-20T22:16:09.554 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:09.562 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 5 1 2024-09-20T22:16:09.583 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:09.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:09.618 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:09.619 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:09.719 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:09.754 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:09.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:09.755 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:09.856 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:09.888 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:09.889 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:09.889 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:09.926 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 52 16 83039 83023 99.767 71.1875 0.0199091 0.0100178 2024-09-20T22:16:09.934 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:09.990 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:10.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:10.038 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:10.039 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:10.139 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:10.174 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:10.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:10.175 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:10.275 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:10.300 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:16:10.300 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:16:10.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:10.319 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:10.319 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:10.336 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:16:10.336 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:16:10.378 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:16:10.378 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:16:10.412 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:16:10.412 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:16:10.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:10.447 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:16:10.448 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:16:10.461 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:10.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:10.462 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:10.481 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:16:10.481 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:16:10.515 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:16:10.515 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:16:10.557 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:16:10.557 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:16:10.563 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:10.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:10.609 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:10.610 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:10.711 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:10.736 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:16:10.748 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:10.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:10.750 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:10.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:10 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2362819016' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:10.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:10 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2362819016' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:10.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:10 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2362819016' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:10.851 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:10.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:10.879 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:10.879 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:10.927 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 53 16 84192 84176 99.2415 72.0625 0.0116968 0.0100715 2024-09-20T22:16:10.980 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:11.012 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:11.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:11.013 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:11.114 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:11.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:11.163 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:11.163 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:11.264 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:11.301 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:11.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:11.302 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:11.402 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:11.442 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:11.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:11.444 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:11.545 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:11.609 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:11.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:11.610 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:11.711 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:11.711 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:11 smithi060 ceph-mon[29694]: pgmap v713: 168 pgs: 168 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 54 MiB/s wr, 871 op/s 2024-09-20T22:16:11.712 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:11 smithi060 ceph-mon[40290]: pgmap v713: 168 pgs: 168 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 54 MiB/s wr, 871 op/s 2024-09-20T22:16:11.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:11.753 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:11.753 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:11.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:11 smithi067 ceph-mon[33511]: pgmap v713: 168 pgs: 168 active+clean; 4.9 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 852 B/s rd, 54 MiB/s wr, 871 op/s 2024-09-20T22:16:11.855 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:11.859 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:11.886 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:16:11.886 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:16:11.886 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:11.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:11.887 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:11.927 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 54 16 85410 85394 98.8131 76.125 0.0221727 0.0101147 2024-09-20T22:16:11.988 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:12.026 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:12.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:12.027 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:12.127 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:12.149 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:12.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:12.150 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:12.250 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:12.291 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:12.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:12.292 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:12.393 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:12.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:12.427 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:12.427 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:12.528 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:12.568 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:12.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:12.569 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:12.627 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T22:16:12.671 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:12.739 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:12.740 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:12.740 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:12.841 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:12.866 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:12.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:12.867 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:12.894 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:12 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:16:12.894 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:12 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:16:12.894 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:12 smithi060 ceph-mon[40290]: pgmap v714: 168 pgs: 168 active+clean; 5.1 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.4 KiB/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-20T22:16:12.894 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:12 smithi060 ceph-mon[40290]: osdmap e399: 8 total, 8 up, 8 in 2024-09-20T22:16:12.894 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:12 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:16:12.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:12 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:16:12.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:12 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:16:12.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:12 smithi060 ceph-mon[29694]: pgmap v714: 168 pgs: 168 active+clean; 5.1 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.4 KiB/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-20T22:16:12.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:12 smithi060 ceph-mon[29694]: osdmap e399: 8 total, 8 up, 8 in 2024-09-20T22:16:12.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:12 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:16:12.928 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 55 16 86410 86394 98.1513 62.5 0.0049794 0.0101821 2024-09-20T22:16:12.969 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:12.999 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:13.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:13.000 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:13.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:16:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:16:13.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:12 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:16:13.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:12 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:16:13.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:12 smithi067 ceph-mon[33511]: pgmap v714: 168 pgs: 168 active+clean; 5.1 GiB data, 20 GiB used, 696 GiB / 715 GiB avail; 1.4 KiB/s rd, 77 MiB/s wr, 1.23k op/s 2024-09-20T22:16:13.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:12 smithi067 ceph-mon[33511]: osdmap e399: 8 total, 8 up, 8 in 2024-09-20T22:16:13.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:12 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:16:13.101 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:13.134 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:13.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:13.135 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:13.195 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:13.235 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:13.292 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:13.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:13.308 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:13.308 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:13.409 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:13.458 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:13.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:13.459 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:13.505 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:16:13.560 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:13.587 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:13.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:13.588 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:13.676 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.5 to 1 (10000) 2024-09-20T22:16:13.689 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:13.721 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:13.721 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:13.722 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:13.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:13 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2795733433' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:16:13.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:13 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2795733433' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:16:13.822 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:13.852 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:13.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:13.853 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:13.928 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 56 16 87486 87470 97.5994 67.25 0.0143292 0.0102406 2024-09-20T22:16:13.954 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:14.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:14.013 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:14.013 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:13 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2795733433' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:16:14.114 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:14.152 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:14.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:14.153 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:14.254 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:14.292 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:14.293 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:14.293 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:14.394 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:14.433 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:14.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:14.434 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:14.535 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:14.537 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 6 1 2024-09-20T22:16:14.567 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:14.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:14.568 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:14.668 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:14.707 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:16:14.725 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:14.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:14.726 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:14.739 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:14 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2795733433' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:16:14.739 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:14 smithi060 ceph-mon[40290]: osdmap e400: 8 total, 8 up, 8 in 2024-09-20T22:16:14.739 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:14 smithi060 ceph-mon[40290]: pgmap v717: 168 pgs: 168 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 666 B/s rd, 71 MiB/s wr, 1.13k op/s 2024-09-20T22:16:14.739 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:14 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/955928101' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:16:14.739 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:14 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2795733433' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:16:14.739 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:14 smithi060 ceph-mon[29694]: osdmap e400: 8 total, 8 up, 8 in 2024-09-20T22:16:14.739 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:14 smithi060 ceph-mon[29694]: pgmap v717: 168 pgs: 168 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 666 B/s rd, 71 MiB/s wr, 1.13k op/s 2024-09-20T22:16:14.739 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:14 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/955928101' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:16:14.827 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:14.877 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:14.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:14.878 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:14.928 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 57 16 88637 88621 97.1491 71.9375 0.0164638 0.0102883 2024-09-20T22:16:14.979 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:14 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2795733433' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:16:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:14 smithi067 ceph-mon[33511]: osdmap e400: 8 total, 8 up, 8 in 2024-09-20T22:16:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:14 smithi067 ceph-mon[33511]: pgmap v717: 168 pgs: 168 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 666 B/s rd, 71 MiB/s wr, 1.13k op/s 2024-09-20T22:16:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:14 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/955928101' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:16:15.027 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:15.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:15.028 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:15.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:15.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:15.169 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:15.170 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:15.243 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:16:15.243 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:16:15.270 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:15.317 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:15.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:15.318 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:15.418 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:15.455 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:15.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:15.456 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:15.535 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:16:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:16:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:16:15.557 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:15.599 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:15.599 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:15.600 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:15.700 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:15.735 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:15.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:15.736 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:15.837 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:15.878 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:15.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:15.879 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:15.930 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 58 16 89802 89786 96.7286 72.8125 0.0285568 0.0103328 2024-09-20T22:16:15.980 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:16.010 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:15 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:16:16.010 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:15 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/955928101' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:16:16.010 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:15 smithi060 ceph-mon[40290]: osdmap e401: 8 total, 8 up, 8 in 2024-09-20T22:16:16.011 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:15 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:16:16.011 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:15 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/955928101' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:16:16.012 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:15 smithi060 ceph-mon[29694]: osdmap e401: 8 total, 8 up, 8 in 2024-09-20T22:16:16.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:15 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:16:16.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:15 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/955928101' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:16:16.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:15 smithi067 ceph-mon[33511]: osdmap e401: 8 total, 8 up, 8 in 2024-09-20T22:16:16.014 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:16.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:16.015 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:16.116 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:16.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:16.156 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:16.156 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:16.257 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:16.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:16.295 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:16.295 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:16.386 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T22:16:16.395 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:16.398 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:16.403 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:16.427 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:16.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:16.428 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:16.529 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:16.596 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:16.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:16.597 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:16.698 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:16.708 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:16 smithi060 ceph-mon[29694]: 5.e scrub starts 2024-09-20T22:16:16.708 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:16 smithi060 ceph-mon[29694]: 5.e scrub ok 2024-09-20T22:16:16.708 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:16 smithi060 ceph-mon[29694]: pgmap v719: 168 pgs: 168 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-20T22:16:16.708 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:16 smithi060 ceph-mon[29694]: osdmap e402: 8 total, 8 up, 8 in 2024-09-20T22:16:16.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:16.747 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:16.748 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:16.848 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:16.882 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:16.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:16.883 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:16.929 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 59 16 90995 90979 96.3529 74.5625 0.00961507 0.0103734 2024-09-20T22:16:16.969 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:16 smithi060 ceph-mon[40290]: 5.e scrub starts 2024-09-20T22:16:16.969 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:16 smithi060 ceph-mon[40290]: 5.e scrub ok 2024-09-20T22:16:16.969 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:16 smithi060 ceph-mon[40290]: pgmap v719: 168 pgs: 168 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-20T22:16:16.969 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:16 smithi060 ceph-mon[40290]: osdmap e402: 8 total, 8 up, 8 in 2024-09-20T22:16:16.984 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:17.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:16 smithi067 ceph-mon[33511]: 5.e scrub starts 2024-09-20T22:16:17.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:16 smithi067 ceph-mon[33511]: 5.e scrub ok 2024-09-20T22:16:17.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:16 smithi067 ceph-mon[33511]: pgmap v719: 168 pgs: 168 active+clean; 5.3 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.7 KiB/s rd, 79 MiB/s wr, 1.27k op/s 2024-09-20T22:16:17.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:16 smithi067 ceph-mon[33511]: osdmap e402: 8 total, 8 up, 8 in 2024-09-20T22:16:17.034 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:17.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:17.035 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:17.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:17.176 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:17.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:17.177 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:17.278 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:17.315 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:16:17.315 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:16:17.315 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:16:17.315 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:16:17.315 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:16:17.315 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:16:17.316 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:16:17.316 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:16:17.318 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:17.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:17.319 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:17.334 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:16:17.335 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:16:17.367 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:16:17.368 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:16:17.404 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:16:17.404 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:16:17.420 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:17.440 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:16:17.441 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:16:17.450 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:17.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:17.451 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:17.552 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:17.594 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:17.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:17.596 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:17.653 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:16:17.698 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:17.728 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:17.729 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:17.729 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:17.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:17 smithi060 ceph-mon[40290]: 4.0 scrub starts 2024-09-20T22:16:17.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:17 smithi060 ceph-mon[40290]: 4.0 scrub ok 2024-09-20T22:16:17.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:17 smithi060 ceph-mon[40290]: 6.1f scrub starts 2024-09-20T22:16:17.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:17 smithi060 ceph-mon[40290]: 6.1f scrub ok 2024-09-20T22:16:17.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:17 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/500854772' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:17.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:17 smithi060 ceph-mon[40290]: 5.17 scrub starts 2024-09-20T22:16:17.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:17 smithi060 ceph-mon[40290]: 5.17 scrub ok 2024-09-20T22:16:17.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:17 smithi060 ceph-mon[29694]: 4.0 scrub starts 2024-09-20T22:16:17.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:17 smithi060 ceph-mon[29694]: 4.0 scrub ok 2024-09-20T22:16:17.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:17 smithi060 ceph-mon[29694]: 6.1f scrub starts 2024-09-20T22:16:17.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:17 smithi060 ceph-mon[29694]: 6.1f scrub ok 2024-09-20T22:16:17.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:17 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/500854772' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:17.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:17 smithi060 ceph-mon[29694]: 5.17 scrub starts 2024-09-20T22:16:17.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:17 smithi060 ceph-mon[29694]: 5.17 scrub ok 2024-09-20T22:16:17.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:17.883 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:17.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:17.884 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:17.929 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:16:17.929329+0000 min lat: 0.00137086 max lat: 0.30872 avg lat: 0.0104259 2024-09-20T22:16:17.929 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:16:17.929 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 60 16 92063 92047 95.8588 66.75 0.0125876 0.0104259 2024-09-20T22:16:17.985 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:17 smithi067 ceph-mon[33511]: 4.0 scrub starts 2024-09-20T22:16:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:17 smithi067 ceph-mon[33511]: 4.0 scrub ok 2024-09-20T22:16:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:17 smithi067 ceph-mon[33511]: 6.1f scrub starts 2024-09-20T22:16:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:17 smithi067 ceph-mon[33511]: 6.1f scrub ok 2024-09-20T22:16:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:17 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/500854772' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:17 smithi067 ceph-mon[33511]: 5.17 scrub starts 2024-09-20T22:16:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:17 smithi067 ceph-mon[33511]: 5.17 scrub ok 2024-09-20T22:16:18.028 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:18.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:18.029 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:18.130 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:18.161 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:18.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:18.162 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:18.263 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:18.297 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:18.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:18.298 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:18.399 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:18.429 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:18.445 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:18.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:18.446 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:18.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:18.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:18.597 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:18.598 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:18.698 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:18.725 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.6 to 1 (10000) 2024-09-20T22:16:18.753 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:18.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:18.754 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:18.855 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:18.894 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:18.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:18.895 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:18.929 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 61 16 93258 93242 95.5116 74.6875 0.00627399 0.0104652 2024-09-20T22:16:18.996 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:19.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:18 smithi067 ceph-mon[33511]: 4.d scrub starts 2024-09-20T22:16:19.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:18 smithi067 ceph-mon[33511]: 4.d scrub ok 2024-09-20T22:16:19.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:18 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3944708281' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:16:19.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:18 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:16:19.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:18 smithi067 ceph-mon[33511]: pgmap v721: 168 pgs: 168 active+clean; 5.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 867 B/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-20T22:16:19.031 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:19.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:19.033 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:19.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:18 smithi060 ceph-mon[29694]: 4.d scrub starts 2024-09-20T22:16:19.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:18 smithi060 ceph-mon[29694]: 4.d scrub ok 2024-09-20T22:16:19.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:18 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3944708281' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:16:19.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:18 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:16:19.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:18 smithi060 ceph-mon[29694]: pgmap v721: 168 pgs: 168 active+clean; 5.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 867 B/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-20T22:16:19.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:18 smithi060 ceph-mon[40290]: 4.d scrub starts 2024-09-20T22:16:19.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:18 smithi060 ceph-mon[40290]: 4.d scrub ok 2024-09-20T22:16:19.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:18 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3944708281' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:16:19.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:18 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:16:19.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:18 smithi060 ceph-mon[40290]: pgmap v721: 168 pgs: 168 active+clean; 5.5 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 867 B/s rd, 82 MiB/s wr, 1.31k op/s 2024-09-20T22:16:19.133 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:19.168 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:19.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:19.170 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:19.248 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:16:19.248 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:16:19.248 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:16:19.248 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:16:19.248 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.248 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227635 7.b 7.fcebd15b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:16:19.248 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.849145+0000", 2024-09-20T22:16:19.248 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 565.39453732699997, 2024-09-20T22:16:19.249 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.958058509, 2024-09-20T22:16:19.249 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.249 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.249 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:16:19.249 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.249 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:16:19.249 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:16:19.249 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227635 2024-09-20T22:16:19.249 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.249 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.249 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.249 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.249 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849145+0000", 2024-09-20T22:16:19.250 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.250 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.250 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.250 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849145+0000", 2024-09-20T22:16:19.250 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.250 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.250 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.250 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849146+0000", 2024-09-20T22:16:19.250 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.562e-06 2024-09-20T22:16:19.250 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.250 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.250 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.251 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849152+0000", 2024-09-20T22:16:19.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5999999999999997e-06 2024-09-20T22:16:19.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.251 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.251 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.251 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849162+0000", 2024-09-20T22:16:19.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9620000000000005e-06 2024-09-20T22:16:19.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.251 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.251 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.251 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849171+0000", 2024-09-20T22:16:19.251 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4569999999999994e-06 2024-09-20T22:16:19.251 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.252 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.252 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.252 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849336+0000", 2024-09-20T22:16:19.252 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016478700000000001 2024-09-20T22:16:19.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.252 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.252 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.252 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.849344+0000", 2024-09-20T22:16:19.252 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3950000000000001e-06 2024-09-20T22:16:19.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.252 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.252 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.252 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.807192+0000", 2024-09-20T22:16:19.252 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.95784742599999995 2024-09-20T22:16:19.252 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.253 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.253 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.807203+0000", 2024-09-20T22:16:19.253 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.132e-05 2024-09-20T22:16:19.253 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.253 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.253 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.253 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.253 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.253 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.24667.0:227641 7.b 7.8d46094b (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e145)", 2024-09-20T22:16:19.253 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:06:53.858720+0000", 2024-09-20T22:16:19.253 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 565.38496183899997, 2024-09-20T22:16:19.253 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.95058797399999995, 2024-09-20T22:16:19.253 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.254 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.254 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:16:19.254 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.254 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.24667", 2024-09-20T22:16:19.254 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2247701693", 2024-09-20T22:16:19.254 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 227641 2024-09-20T22:16:19.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.254 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.254 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.254 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858720+0000", 2024-09-20T22:16:19.254 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.254 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.254 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.254 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858720+0000", 2024-09-20T22:16:19.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858722+0000", 2024-09-20T22:16:19.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3969999999999999e-06 2024-09-20T22:16:19.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.255 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858726+0000", 2024-09-20T22:16:19.255 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7999999999999998e-06 2024-09-20T22:16:19.255 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.255 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.255 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.256 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858737+0000", 2024-09-20T22:16:19.256 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0403e-05 2024-09-20T22:16:19.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.256 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.256 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.256 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858746+0000", 2024-09-20T22:16:19.256 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2739999999999996e-06 2024-09-20T22:16:19.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.256 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.256 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.256 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858806+0000", 2024-09-20T22:16:19.256 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0315000000000001e-05 2024-09-20T22:16:19.256 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.256 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.257 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.257 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:53.858818+0000", 2024-09-20T22:16:19.257 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2079000000000001e-05 2024-09-20T22:16:19.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.257 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.257 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.257 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.809293+0000", 2024-09-20T22:16:19.257 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.95047507899999994 2024-09-20T22:16:19.257 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.257 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.257 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.257 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:06:54.809308+0000", 2024-09-20T22:16:19.257 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4627000000000001e-05 2024-09-20T22:16:19.257 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.257 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.258 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.258 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.258 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105654 8.1 8:83678158:::benchmark_data_smithi060_111051_object105653:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:16:19.258 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.549939+0000", 2024-09-20T22:16:19.258 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 430.693743471, 2024-09-20T22:16:19.258 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.003362782, 2024-09-20T22:16:19.258 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.258 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.258 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:16:19.258 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.258 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:16:19.258 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:19.258 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105654 2024-09-20T22:16:19.258 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.259 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549939+0000", 2024-09-20T22:16:19.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549939+0000", 2024-09-20T22:16:19.259 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.259 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.259 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.259 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.259 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549940+0000", 2024-09-20T22:16:19.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.95e-06 2024-09-20T22:16:19.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549973+0000", 2024-09-20T22:16:19.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2798e-05 2024-09-20T22:16:19.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.260 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549997+0000", 2024-09-20T22:16:19.260 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3649e-05 2024-09-20T22:16:19.260 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.260 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.260 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.261 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550007+0000", 2024-09-20T22:16:19.261 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8190000000000003e-06 2024-09-20T22:16:19.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.261 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.261 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702300+0000", 2024-09-20T22:16:19.261 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.152292975 2024-09-20T22:16:19.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.261 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.261 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702310+0000", 2024-09-20T22:16:19.261 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0453e-05 2024-09-20T22:16:19.261 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.261 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.262 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.262 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.553223+0000", 2024-09-20T22:16:19.262 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85091275799999999 2024-09-20T22:16:19.262 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.262 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.262 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.263 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.553301+0000", 2024-09-20T22:16:19.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8380000000000005e-05 2024-09-20T22:16:19.263 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.263 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.263 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.263 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.263 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.263 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105657 8.7 8:e060afd6:::benchmark_data_smithi060_111051_object105656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:16:19.263 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.550025+0000", 2024-09-20T22:16:19.263 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 430.69365695499999, 2024-09-20T22:16:19.263 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.029495933, 2024-09-20T22:16:19.263 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.263 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.263 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:19.264 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.264 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:16:19.264 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:19.264 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105657 2024-09-20T22:16:19.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.264 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550025+0000", 2024-09-20T22:16:19.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.264 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.264 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.264 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550025+0000", 2024-09-20T22:16:19.264 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.264 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550026+0000", 2024-09-20T22:16:19.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1799999999999995e-07 2024-09-20T22:16:19.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550051+0000", 2024-09-20T22:16:19.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5554999999999998e-05 2024-09-20T22:16:19.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.265 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.265 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.265 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550062+0000", 2024-09-20T22:16:19.265 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.137e-05 2024-09-20T22:16:19.265 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.266 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.266 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550068+0000", 2024-09-20T22:16:19.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8289999999999996e-06 2024-09-20T22:16:19.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.266 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.266 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720577+0000", 2024-09-20T22:16:19.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.170508778 2024-09-20T22:16:19.266 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.266 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.266 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.266 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720588+0000", 2024-09-20T22:16:19.266 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0808000000000001e-05 2024-09-20T22:16:19.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.569210+0000", 2024-09-20T22:16:19.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.84862162900000004 2024-09-20T22:16:19.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:16:19.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.569360+0000", 2024-09-20T22:16:19.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015044799999999999 2024-09-20T22:16:19.267 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.267 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.267 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:16:19.267 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.569667+0000", 2024-09-20T22:16:19.267 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030655599999999998 2024-09-20T22:16:19.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:16:19.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.572342+0000", 2024-09-20T22:16:19.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0026757840000000001 2024-09-20T22:16:19.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.573025+0000", 2024-09-20T22:16:19.268 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00068279199999999999 2024-09-20T22:16:19.268 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.268 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.268 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.268 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579397+0000", 2024-09-20T22:16:19.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.006372071 2024-09-20T22:16:19.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:16:19.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579465+0000", 2024-09-20T22:16:19.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7520000000000004e-05 2024-09-20T22:16:19.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.269 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.269 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.269 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579521+0000", 2024-09-20T22:16:19.269 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6274999999999999e-05 2024-09-20T22:16:19.269 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.269 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.269 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.269 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.270 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.270 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105658 8.1 8:809ea6c9:::benchmark_data_smithi060_111051_object105657:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:16:19.270 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.550079+0000", 2024-09-20T22:16:19.270 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 430.69360337199998, 2024-09-20T22:16:19.270 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.010609463, 2024-09-20T22:16:19.270 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.270 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.270 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:19.270 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.270 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:16:19.270 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:19.270 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105658 2024-09-20T22:16:19.270 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.271 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550079+0000", 2024-09-20T22:16:19.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550079+0000", 2024-09-20T22:16:19.271 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.271 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.271 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.271 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.271 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550079+0000", 2024-09-20T22:16:19.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2599999999999998e-07 2024-09-20T22:16:19.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550101+0000", 2024-09-20T22:16:19.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1628e-05 2024-09-20T22:16:19.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550111+0000", 2024-09-20T22:16:19.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9620000000000005e-06 2024-09-20T22:16:19.272 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.272 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.272 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550116+0000", 2024-09-20T22:16:19.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9810000000000003e-06 2024-09-20T22:16:19.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702335+0000", 2024-09-20T22:16:19.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15221938700000001 2024-09-20T22:16:19.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702343+0000", 2024-09-20T22:16:19.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2120000000000003e-06 2024-09-20T22:16:19.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.553649+0000", 2024-09-20T22:16:19.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85130619600000002 2024-09-20T22:16:19.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:16:19.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.553784+0000", 2024-09-20T22:16:19.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000134499 2024-09-20T22:16:19.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:16:19.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.554060+0000", 2024-09-20T22:16:19.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000276579 2024-09-20T22:16:19.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:16:19.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.557338+0000", 2024-09-20T22:16:19.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003277608 2024-09-20T22:16:19.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.559599+0000", 2024-09-20T22:16:19.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002261304 2024-09-20T22:16:19.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560617+0000", 2024-09-20T22:16:19.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001017354 2024-09-20T22:16:19.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:16:19.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560660+0000", 2024-09-20T22:16:19.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3167999999999997e-05 2024-09-20T22:16:19.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560688+0000", 2024-09-20T22:16:19.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8158999999999999e-05 2024-09-20T22:16:19.276 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.276 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.276 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.276 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105659 8.1 8:80c09cff:::benchmark_data_smithi060_111051_object105658:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:16:19.277 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.550123+0000", 2024-09-20T22:16:19.277 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 430.69355863800001, 2024-09-20T22:16:19.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0107473760000001, 2024-09-20T22:16:19.277 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.277 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.277 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:19.277 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.277 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:16:19.277 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:19.277 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105659 2024-09-20T22:16:19.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.277 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550123+0000", 2024-09-20T22:16:19.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550123+0000", 2024-09-20T22:16:19.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550124+0000", 2024-09-20T22:16:19.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.39e-07 2024-09-20T22:16:19.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550148+0000", 2024-09-20T22:16:19.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3944e-05 2024-09-20T22:16:19.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550157+0000", 2024-09-20T22:16:19.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5410000000000006e-06 2024-09-20T22:16:19.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.279 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550162+0000", 2024-09-20T22:16:19.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8069999999999999e-06 2024-09-20T22:16:19.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702368+0000", 2024-09-20T22:16:19.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15220560899999999 2024-09-20T22:16:19.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702374+0000", 2024-09-20T22:16:19.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7939999999999997e-06 2024-09-20T22:16:19.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.555649+0000", 2024-09-20T22:16:19.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85327487000000002 2024-09-20T22:16:19.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:16:19.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.555819+0000", 2024-09-20T22:16:19.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016938200000000001 2024-09-20T22:16:19.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:16:19.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.556100+0000", 2024-09-20T22:16:19.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00028165699999999999 2024-09-20T22:16:19.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:16:19.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.558925+0000", 2024-09-20T22:16:19.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0028247290000000002 2024-09-20T22:16:19.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.559791+0000", 2024-09-20T22:16:19.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00086584599999999998 2024-09-20T22:16:19.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560771+0000", 2024-09-20T22:16:19.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00097968300000000007 2024-09-20T22:16:19.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:16:19.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560821+0000", 2024-09-20T22:16:19.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0460999999999997e-05 2024-09-20T22:16:19.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560871+0000", 2024-09-20T22:16:19.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9614000000000003e-05 2024-09-20T22:16:19.283 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.283 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.283 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.283 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105660 8.13 8:cf39f948:::benchmark_data_smithi060_111051_object105659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:16:19.284 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.550169+0000", 2024-09-20T22:16:19.284 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 430.69351290499998, 2024-09-20T22:16:19.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0659826960000001, 2024-09-20T22:16:19.284 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.284 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.284 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:19.284 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.284 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:16:19.284 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:19.284 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105660 2024-09-20T22:16:19.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.284 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550169+0000", 2024-09-20T22:16:19.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550169+0000", 2024-09-20T22:16:19.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550169+0000", 2024-09-20T22:16:19.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9099999999999999e-07 2024-09-20T22:16:19.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550221+0000", 2024-09-20T22:16:19.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1844e-05 2024-09-20T22:16:19.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550233+0000", 2024-09-20T22:16:19.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1871e-05 2024-09-20T22:16:19.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550238+0000", 2024-09-20T22:16:19.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6750000000000001e-06 2024-09-20T22:16:19.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.701152+0000", 2024-09-20T22:16:19.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15091443500000001 2024-09-20T22:16:19.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.701161+0000", 2024-09-20T22:16:19.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2679999999999994e-06 2024-09-20T22:16:19.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.605154+0000", 2024-09-20T22:16:19.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90399336100000005 2024-09-20T22:16:19.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:16:19.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.605243+0000", 2024-09-20T22:16:19.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8661999999999999e-05 2024-09-20T22:16:19.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 5,6", 2024-09-20T22:16:19.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.605424+0000", 2024-09-20T22:16:19.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018181800000000001 2024-09-20T22:16:19.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:16:19.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.606520+0000", 2024-09-20T22:16:19.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001095333 2024-09-20T22:16:19.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.289 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:19.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.612488+0000", 2024-09-20T22:16:19.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0059685019999999997 2024-09-20T22:16:19.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.616043+0000", 2024-09-20T22:16:19.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0035544249999999999 2024-09-20T22:16:19.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:16:19.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.616108+0000", 2024-09-20T22:16:19.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5506999999999995e-05 2024-09-20T22:16:19.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.616152+0000", 2024-09-20T22:16:19.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3603999999999998e-05 2024-09-20T22:16:19.291 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.291 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.291 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.292 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105668 8.7 8:e0173451:::benchmark_data_smithi060_111051_object105667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:16:19.292 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.706681+0000", 2024-09-20T22:16:19.292 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 430.53700110400001, 2024-09-20T22:16:19.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.87300058000000003, 2024-09-20T22:16:19.292 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.292 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.292 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:19.292 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.292 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:16:19.292 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:19.292 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105668 2024-09-20T22:16:19.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.292 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706681+0000", 2024-09-20T22:16:19.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706681+0000", 2024-09-20T22:16:19.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706683+0000", 2024-09-20T22:16:19.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.928e-06 2024-09-20T22:16:19.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706710+0000", 2024-09-20T22:16:19.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6911e-05 2024-09-20T22:16:19.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706734+0000", 2024-09-20T22:16:19.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3992999999999999e-05 2024-09-20T22:16:19.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706743+0000", 2024-09-20T22:16:19.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1209999999999996e-06 2024-09-20T22:16:19.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720612+0000", 2024-09-20T22:16:19.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.013869283 2024-09-20T22:16:19.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720619+0000", 2024-09-20T22:16:19.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7100000000000001e-06 2024-09-20T22:16:19.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.571247+0000", 2024-09-20T22:16:19.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85062784499999999 2024-09-20T22:16:19.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:16:19.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.571381+0000", 2024-09-20T22:16:19.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000134167 2024-09-20T22:16:19.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:16:19.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.571594+0000", 2024-09-20T22:16:19.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021272700000000001 2024-09-20T22:16:19.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:16:19.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.572500+0000", 2024-09-20T22:16:19.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00090607800000000005 2024-09-20T22:16:19.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.574997+0000", 2024-09-20T22:16:19.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024976759999999999 2024-09-20T22:16:19.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579580+0000", 2024-09-20T22:16:19.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0045830269999999999 2024-09-20T22:16:19.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:16:19.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579636+0000", 2024-09-20T22:16:19.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5413e-05 2024-09-20T22:16:19.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579681+0000", 2024-09-20T22:16:19.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5701000000000002e-05 2024-09-20T22:16:19.298 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.298 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.298 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.299 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106973 8.13 8:cbbe7a4a:::benchmark_data_smithi060_111051_object106972:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:16:19.299 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.414028+0000", 2024-09-20T22:16:19.299 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 427.82965366299999, 2024-09-20T22:16:19.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0848815410000001, 2024-09-20T22:16:19.299 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.299 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.299 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:19.299 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.299 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:16:19.299 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:19.299 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106973 2024-09-20T22:16:19.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.299 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.300 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414028+0000", 2024-09-20T22:16:19.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.300 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414028+0000", 2024-09-20T22:16:19.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.300 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414030+0000", 2024-09-20T22:16:19.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.866e-06 2024-09-20T22:16:19.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.300 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414059+0000", 2024-09-20T22:16:19.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9043999999999998e-05 2024-09-20T22:16:19.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414080+0000", 2024-09-20T22:16:19.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1254999999999999e-05 2024-09-20T22:16:19.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414090+0000", 2024-09-20T22:16:19.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2129999999999997e-06 2024-09-20T22:16:19.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.415798+0000", 2024-09-20T22:16:19.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017082340000000001 2024-09-20T22:16:19.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.415809+0000", 2024-09-20T22:16:19.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1433e-05 2024-09-20T22:16:19.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.479861+0000", 2024-09-20T22:16:19.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.064051522 2024-09-20T22:16:19.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:16:19.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.479994+0000", 2024-09-20T22:16:19.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013297800000000001 2024-09-20T22:16:19.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5,6", 2024-09-20T22:16:19.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.480283+0000", 2024-09-20T22:16:19.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00028927100000000002 2024-09-20T22:16:19.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.482942+0000", 2024-09-20T22:16:19.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002658737 2024-09-20T22:16:19.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.304 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.304 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.483991+0000", 2024-09-20T22:16:19.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001049267 2024-09-20T22:16:19.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.304 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:16:19.304 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.484017+0000", 2024-09-20T22:16:19.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6057000000000001e-05 2024-09-20T22:16:19.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.304 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.304 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.486044+0000", 2024-09-20T22:16:19.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020266189999999999 2024-09-20T22:16:19.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.498642+0000", 2024-09-20T22:16:19.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.012598417000000001 2024-09-20T22:16:19.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:16:19.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.498776+0000", 2024-09-20T22:16:19.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000133764 2024-09-20T22:16:19.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.498910+0000", 2024-09-20T22:16:19.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000133864 2024-09-20T22:16:19.306 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.306 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.306 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.306 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187845 8.a 8:5885b3a8:::benchmark_data_smithi060_111051_object67846:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:16:19.306 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.623815+0000", 2024-09-20T22:16:19.306 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 349.61986685400001, 2024-09-20T22:16:19.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1184221929999998, 2024-09-20T22:16:19.306 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.306 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.306 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:16:19.307 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.307 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:16:19.307 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:19.307 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187845 2024-09-20T22:16:19.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.307 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.623815+0000", 2024-09-20T22:16:19.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.623815+0000", 2024-09-20T22:16:19.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.623816+0000", 2024-09-20T22:16:19.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.114e-06 2024-09-20T22:16:19.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.623820+0000", 2024-09-20T22:16:19.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5360000000000001e-06 2024-09-20T22:16:19.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.623833+0000", 2024-09-20T22:16:19.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2874000000000001e-05 2024-09-20T22:16:19.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.623839+0000", 2024-09-20T22:16:19.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1319999999999999e-06 2024-09-20T22:16:19.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.624269+0000", 2024-09-20T22:16:19.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00043047400000000003 2024-09-20T22:16:19.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:16:19.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.624378+0000", 2024-09-20T22:16:19.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108738 2024-09-20T22:16:19.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:16:19.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.624481+0000", 2024-09-20T22:16:19.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103175 2024-09-20T22:16:19.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:16:19.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.629342+0000", 2024-09-20T22:16:19.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0048608089999999998 2024-09-20T22:16:19.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.631000+0000", 2024-09-20T22:16:19.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0016580049999999999 2024-09-20T22:16:19.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.640337+0000", 2024-09-20T22:16:19.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0093367550000000004 2024-09-20T22:16:19.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.651705+0000", 2024-09-20T22:16:19.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.011368038 2024-09-20T22:16:19.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650249+0000", 2024-09-20T22:16:19.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99854470299999998 2024-09-20T22:16:19.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650268+0000", 2024-09-20T22:16:19.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8981e-05 2024-09-20T22:16:19.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.742151+0000", 2024-09-20T22:16:19.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.09188247 2024-09-20T22:16:19.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.742237+0000", 2024-09-20T22:16:19.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6389000000000002e-05 2024-09-20T22:16:19.313 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.313 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.313 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.313 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187855 8.a 8:5492c8b1:::benchmark_data_smithi060_111051_object67856:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:16:19.313 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.636998+0000", 2024-09-20T22:16:19.314 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 349.60668426299998, 2024-09-20T22:16:19.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1143666080000001, 2024-09-20T22:16:19.314 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.314 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.314 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:19.314 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.314 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:16:19.314 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:19.314 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187855 2024-09-20T22:16:19.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.314 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.314 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.636998+0000", 2024-09-20T22:16:19.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.636998+0000", 2024-09-20T22:16:19.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.636999+0000", 2024-09-20T22:16:19.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6080000000000001e-06 2024-09-20T22:16:19.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.637004+0000", 2024-09-20T22:16:19.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5839999999999996e-06 2024-09-20T22:16:19.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.637014+0000", 2024-09-20T22:16:19.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0569e-05 2024-09-20T22:16:19.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.637021+0000", 2024-09-20T22:16:19.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9670000000000004e-06 2024-09-20T22:16:19.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.651901+0000", 2024-09-20T22:16:19.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014879966999999999 2024-09-20T22:16:19.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.651911+0000", 2024-09-20T22:16:19.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2040000000000003e-06 2024-09-20T22:16:19.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650383+0000", 2024-09-20T22:16:19.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99847266700000004 2024-09-20T22:16:19.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650393+0000", 2024-09-20T22:16:19.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6299999999999993e-06 2024-09-20T22:16:19.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.742483+0000", 2024-09-20T22:16:19.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0920898370000001 2024-09-20T22:16:19.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:16:19.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.742615+0000", 2024-09-20T22:16:19.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013252000000000001 2024-09-20T22:16:19.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5", 2024-09-20T22:16:19.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.743004+0000", 2024-09-20T22:16:19.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00038869300000000001 2024-09-20T22:16:19.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751051+0000", 2024-09-20T22:16:19.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0080469289999999995 2024-09-20T22:16:19.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:16:19.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751066+0000", 2024-09-20T22:16:19.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5245999999999999e-05 2024-09-20T22:16:19.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751111+0000", 2024-09-20T22:16:19.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4713999999999999e-05 2024-09-20T22:16:19.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751258+0000", 2024-09-20T22:16:19.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147479 2024-09-20T22:16:19.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:16:19.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751292+0000", 2024-09-20T22:16:19.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3330000000000001e-05 2024-09-20T22:16:19.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751364+0000", 2024-09-20T22:16:19.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2663999999999994e-05 2024-09-20T22:16:19.321 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.321 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.321 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.321 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187859 8.a 8:5b818400:::benchmark_data_smithi060_111051_object67860:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:16:19.321 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.649251+0000", 2024-09-20T22:16:19.321 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 349.59443053799998, 2024-09-20T22:16:19.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1040842400000002, 2024-09-20T22:16:19.322 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.322 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.322 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:19.322 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.322 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:16:19.322 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:19.322 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187859 2024-09-20T22:16:19.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.322 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.322 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.322 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.649251+0000", 2024-09-20T22:16:19.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.649251+0000", 2024-09-20T22:16:19.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.649253+0000", 2024-09-20T22:16:19.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6589999999999999e-06 2024-09-20T22:16:19.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.649258+0000", 2024-09-20T22:16:19.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7149999999999997e-06 2024-09-20T22:16:19.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.649268+0000", 2024-09-20T22:16:19.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0389e-05 2024-09-20T22:16:19.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.649278+0000", 2024-09-20T22:16:19.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0104e-05 2024-09-20T22:16:19.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.651941+0000", 2024-09-20T22:16:19.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0026624449999999998 2024-09-20T22:16:19.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.651948+0000", 2024-09-20T22:16:19.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5460000000000003e-06 2024-09-20T22:16:19.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650417+0000", 2024-09-20T22:16:19.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99846875400000001 2024-09-20T22:16:19.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650425+0000", 2024-09-20T22:16:19.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1300000000000001e-06 2024-09-20T22:16:19.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.746082+0000", 2024-09-20T22:16:19.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.095656484 2024-09-20T22:16:19.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:16:19.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.746198+0000", 2024-09-20T22:16:19.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000116168 2024-09-20T22:16:19.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5", 2024-09-20T22:16:19.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.746326+0000", 2024-09-20T22:16:19.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012862299999999999 2024-09-20T22:16:19.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:16:19.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751072+0000", 2024-09-20T22:16:19.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0047450720000000004 2024-09-20T22:16:19.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751157+0000", 2024-09-20T22:16:19.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5464999999999996e-05 2024-09-20T22:16:19.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751219+0000", 2024-09-20T22:16:19.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2420000000000002e-05 2024-09-20T22:16:19.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753223+0000", 2024-09-20T22:16:19.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020036950000000002 2024-09-20T22:16:19.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:16:19.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753255+0000", 2024-09-20T22:16:19.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2237999999999999e-05 2024-09-20T22:16:19.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753336+0000", 2024-09-20T22:16:19.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0333000000000005e-05 2024-09-20T22:16:19.329 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.329 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.329 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.329 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187868 8.a 8:5d2db7a2:::benchmark_data_smithi060_111051_object67869:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:16:19.329 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.665623+0000", 2024-09-20T22:16:19.329 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 349.57805899900001, 2024-09-20T22:16:19.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1683483180000001, 2024-09-20T22:16:19.330 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.330 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.330 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:19.330 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.330 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:16:19.330 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:19.330 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187868 2024-09-20T22:16:19.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.330 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665623+0000", 2024-09-20T22:16:19.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665623+0000", 2024-09-20T22:16:19.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665624+0000", 2024-09-20T22:16:19.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.850000000000001e-07 2024-09-20T22:16:19.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665629+0000", 2024-09-20T22:16:19.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8919999999999999e-06 2024-09-20T22:16:19.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665644+0000", 2024-09-20T22:16:19.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4808e-05 2024-09-20T22:16:19.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665652+0000", 2024-09-20T22:16:19.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0039999999999999e-06 2024-09-20T22:16:19.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665729+0000", 2024-09-20T22:16:19.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7385000000000005e-05 2024-09-20T22:16:19.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665739+0000", 2024-09-20T22:16:19.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6360000000000006e-06 2024-09-20T22:16:19.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650448+0000", 2024-09-20T22:16:19.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98470946199999998 2024-09-20T22:16:19.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650456+0000", 2024-09-20T22:16:19.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.413e-06 2024-09-20T22:16:19.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.749234+0000", 2024-09-20T22:16:19.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0987788570000001 2024-09-20T22:16:19.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:16:19.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.749337+0000", 2024-09-20T22:16:19.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000102441 2024-09-20T22:16:19.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5", 2024-09-20T22:16:19.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.749412+0000", 2024-09-20T22:16:19.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4808999999999995e-05 2024-09-20T22:16:19.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:16:19.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751122+0000", 2024-09-20T22:16:19.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017103159999999999 2024-09-20T22:16:19.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753047+0000", 2024-09-20T22:16:19.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019251750000000001 2024-09-20T22:16:19.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753196+0000", 2024-09-20T22:16:19.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000148441 2024-09-20T22:16:19.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.833608+0000", 2024-09-20T22:16:19.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.080412862000000002 2024-09-20T22:16:19.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:16:19.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.833835+0000", 2024-09-20T22:16:19.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022606199999999999 2024-09-20T22:16:19.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.833971+0000", 2024-09-20T22:16:19.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013677000000000001 2024-09-20T22:16:19.338 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.338 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.338 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.339 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187898 8.a 8:535037c1:::benchmark_data_smithi060_111051_object67899:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:16:19.339 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.705909+0000", 2024-09-20T22:16:19.339 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 349.53777324999999, 2024-09-20T22:16:19.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1284684729999999, 2024-09-20T22:16:19.339 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.339 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.339 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:19.339 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.339 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:16:19.339 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:16:19.339 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187898 2024-09-20T22:16:19.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.340 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.705909+0000", 2024-09-20T22:16:19.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.705909+0000", 2024-09-20T22:16:19.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.705910+0000", 2024-09-20T22:16:19.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.299e-06 2024-09-20T22:16:19.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.705915+0000", 2024-09-20T22:16:19.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.429e-06 2024-09-20T22:16:19.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.705930+0000", 2024-09-20T22:16:19.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4916000000000001e-05 2024-09-20T22:16:19.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.705938+0000", 2024-09-20T22:16:19.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0290000000000005e-06 2024-09-20T22:16:19.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.706003+0000", 2024-09-20T22:16:19.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4832999999999994e-05 2024-09-20T22:16:19.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.706014+0000", 2024-09-20T22:16:19.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0961000000000001e-05 2024-09-20T22:16:19.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650482+0000", 2024-09-20T22:16:19.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94446805099999998 2024-09-20T22:16:19.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:16:19.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650490+0000", 2024-09-20T22:16:19.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1589999999999993e-06 2024-09-20T22:16:19.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751402+0000", 2024-09-20T22:16:19.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1009116329999999 2024-09-20T22:16:19.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:16:19.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751502+0000", 2024-09-20T22:16:19.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9617000000000005e-05 2024-09-20T22:16:19.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5", 2024-09-20T22:16:19.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751574+0000", 2024-09-20T22:16:19.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2517999999999999e-05 2024-09-20T22:16:19.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:16:19.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753349+0000", 2024-09-20T22:16:19.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001774386 2024-09-20T22:16:19.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753383+0000", 2024-09-20T22:16:19.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4017999999999999e-05 2024-09-20T22:16:19.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753421+0000", 2024-09-20T22:16:19.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8603000000000003e-05 2024-09-20T22:16:19.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.834077+0000", 2024-09-20T22:16:19.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.080655317000000004 2024-09-20T22:16:19.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:16:19.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.834257+0000", 2024-09-20T22:16:19.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000180688 2024-09-20T22:16:19.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.834377+0000", 2024-09-20T22:16:19.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000120016 2024-09-20T22:16:19.346 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.346 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.347 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.347 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:125840 9.2 9:4a8ba3ac:::benchmark_data_smithi060_140132_object15032:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:16:19.347 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:30.317157+0000", 2024-09-20T22:16:19.347 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 168.926524769, 2024-09-20T22:16:19.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.39984702300000002, 2024-09-20T22:16:19.347 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.347 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.347 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:19.347 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.347 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:16:19.347 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:16:19.348 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 125840 2024-09-20T22:16:19.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.348 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317157+0000", 2024-09-20T22:16:19.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317157+0000", 2024-09-20T22:16:19.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317159+0000", 2024-09-20T22:16:19.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4929999999999999e-06 2024-09-20T22:16:19.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317165+0000", 2024-09-20T22:16:19.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8610000000000003e-06 2024-09-20T22:16:19.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317181+0000", 2024-09-20T22:16:19.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6133999999999998e-05 2024-09-20T22:16:19.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317188+0000", 2024-09-20T22:16:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2840000000000001e-06 2024-09-20T22:16:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317259+0000", 2024-09-20T22:16:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0816999999999996e-05 2024-09-20T22:16:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:16:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317363+0000", 2024-09-20T22:16:19.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103702 2024-09-20T22:16:19.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3", 2024-09-20T22:16:19.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317465+0000", 2024-09-20T22:16:19.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010202 2024-09-20T22:16:19.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:16:19.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.320561+0000", 2024-09-20T22:16:19.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0030965540000000001 2024-09-20T22:16:19.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.321130+0000", 2024-09-20T22:16:19.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00056882299999999998 2024-09-20T22:16:19.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.716883+0000", 2024-09-20T22:16:19.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.39575325 2024-09-20T22:16:19.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:16:19.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.716956+0000", 2024-09-20T22:16:19.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2352000000000001e-05 2024-09-20T22:16:19.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.717004+0000", 2024-09-20T22:16:19.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8733000000000002e-05 2024-09-20T22:16:19.353 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.353 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.353 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.353 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:125859 9.0 9:05dacd16:::benchmark_data_smithi060_140132_object15051:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:16:19.353 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:30.331372+0000", 2024-09-20T22:16:19.353 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 168.912309706, 2024-09-20T22:16:19.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.38946761499999999, 2024-09-20T22:16:19.353 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.353 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.353 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:19.354 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.354 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:16:19.354 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:16:19.354 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 125859 2024-09-20T22:16:19.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.354 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331372+0000", 2024-09-20T22:16:19.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331372+0000", 2024-09-20T22:16:19.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331374+0000", 2024-09-20T22:16:19.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5290000000000001e-06 2024-09-20T22:16:19.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331379+0000", 2024-09-20T22:16:19.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.502e-06 2024-09-20T22:16:19.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331397+0000", 2024-09-20T22:16:19.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7249e-05 2024-09-20T22:16:19.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331406+0000", 2024-09-20T22:16:19.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0659999999999993e-06 2024-09-20T22:16:19.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331474+0000", 2024-09-20T22:16:19.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8396e-05 2024-09-20T22:16:19.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:16:19.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331609+0000", 2024-09-20T22:16:19.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013487800000000001 2024-09-20T22:16:19.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,6", 2024-09-20T22:16:19.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331730+0000", 2024-09-20T22:16:19.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121408 2024-09-20T22:16:19.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:16:19.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.334028+0000", 2024-09-20T22:16:19.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022976429999999998 2024-09-20T22:16:19.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.334978+0000", 2024-09-20T22:16:19.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00095049999999999996 2024-09-20T22:16:19.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720566+0000", 2024-09-20T22:16:19.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.38558782699999999 2024-09-20T22:16:19.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:16:19.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720619+0000", 2024-09-20T22:16:19.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2404000000000002e-05 2024-09-20T22:16:19.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720840+0000", 2024-09-20T22:16:19.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000221213 2024-09-20T22:16:19.359 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.359 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.359 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:19.360 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:19.361 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:19.361 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:125862 9.2 9:4de341df:::benchmark_data_smithi060_140132_object15054:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:16:19.361 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:30.336602+0000", 2024-09-20T22:16:19.361 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 168.907080157, 2024-09-20T22:16:19.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.38056002500000002, 2024-09-20T22:16:19.361 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.361 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.361 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:19.361 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.361 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:16:19.362 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:16:19.362 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 125862 2024-09-20T22:16:19.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.362 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336602+0000", 2024-09-20T22:16:19.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336602+0000", 2024-09-20T22:16:19.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336603+0000", 2024-09-20T22:16:19.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3799999999999996e-07 2024-09-20T22:16:19.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336608+0000", 2024-09-20T22:16:19.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9350000000000002e-06 2024-09-20T22:16:19.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336624+0000", 2024-09-20T22:16:19.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5849e-05 2024-09-20T22:16:19.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336645+0000", 2024-09-20T22:16:19.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1067999999999999e-05 2024-09-20T22:16:19.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336704+0000", 2024-09-20T22:16:19.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9525999999999999e-05 2024-09-20T22:16:19.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:16:19.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336824+0000", 2024-09-20T22:16:19.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011957 2024-09-20T22:16:19.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3", 2024-09-20T22:16:19.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336942+0000", 2024-09-20T22:16:19.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117819 2024-09-20T22:16:19.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.339146+0000", 2024-09-20T22:16:19.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022047680000000002 2024-09-20T22:16:19.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.366 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:16:19.366 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.339234+0000", 2024-09-20T22:16:19.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7968999999999998e-05 2024-09-20T22:16:19.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.366 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.366 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.717065+0000", 2024-09-20T22:16:19.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.37783098799999998 2024-09-20T22:16:19.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.366 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:16:19.366 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.717120+0000", 2024-09-20T22:16:19.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5133999999999998e-05 2024-09-20T22:16:19.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.717162+0000", 2024-09-20T22:16:19.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1461000000000002e-05 2024-09-20T22:16:19.367 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.367 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.367 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.367 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:125869 9.0 9:0c9f6e22:::benchmark_data_smithi060_140132_object15061:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:16:19.367 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:30.340895+0000", 2024-09-20T22:16:19.367 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 168.902786819, 2024-09-20T22:16:19.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.38006649199999998, 2024-09-20T22:16:19.368 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.368 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.368 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:19.368 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.368 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:16:19.368 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:16:19.368 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 125869 2024-09-20T22:16:19.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.368 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340895+0000", 2024-09-20T22:16:19.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340895+0000", 2024-09-20T22:16:19.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340896+0000", 2024-09-20T22:16:19.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.31e-06 2024-09-20T22:16:19.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340902+0000", 2024-09-20T22:16:19.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9370000000000001e-06 2024-09-20T22:16:19.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340917+0000", 2024-09-20T22:16:19.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4181e-05 2024-09-20T22:16:19.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340924+0000", 2024-09-20T22:16:19.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3499999999999999e-06 2024-09-20T22:16:19.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340992+0000", 2024-09-20T22:16:19.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7887999999999998e-05 2024-09-20T22:16:19.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:16:19.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.341093+0000", 2024-09-20T22:16:19.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000101074 2024-09-20T22:16:19.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,6", 2024-09-20T22:16:19.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.341159+0000", 2024-09-20T22:16:19.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6167999999999994e-05 2024-09-20T22:16:19.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:16:19.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343748+0000", 2024-09-20T22:16:19.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025892060000000001 2024-09-20T22:16:19.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.348961+0000", 2024-09-20T22:16:19.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005213082 2024-09-20T22:16:19.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720878+0000", 2024-09-20T22:16:19.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.37191645499999998 2024-09-20T22:16:19.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:16:19.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720928+0000", 2024-09-20T22:16:19.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9888000000000001e-05 2024-09-20T22:16:19.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720962+0000", 2024-09-20T22:16:19.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3952999999999998e-05 2024-09-20T22:16:19.373 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.373 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.373 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.373 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:125872 9.0 9:0ae55481:::benchmark_data_smithi060_140132_object15064:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:16:19.373 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:30.343972+0000", 2024-09-20T22:16:19.373 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 168.899710024, 2024-09-20T22:16:19.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.37716829000000002, 2024-09-20T22:16:19.374 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.374 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.374 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:19.374 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.374 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:16:19.374 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:16:19.374 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 125872 2024-09-20T22:16:19.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.374 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343972+0000", 2024-09-20T22:16:19.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343972+0000", 2024-09-20T22:16:19.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343973+0000", 2024-09-20T22:16:19.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2640000000000001e-06 2024-09-20T22:16:19.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343978+0000", 2024-09-20T22:16:19.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1569999999999999e-06 2024-09-20T22:16:19.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343991+0000", 2024-09-20T22:16:19.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2313000000000001e-05 2024-09-20T22:16:19.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343998+0000", 2024-09-20T22:16:19.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7379999999999994e-06 2024-09-20T22:16:19.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344068+0000", 2024-09-20T22:16:19.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9962000000000005e-05 2024-09-20T22:16:19.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:16:19.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344184+0000", 2024-09-20T22:16:19.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000116063 2024-09-20T22:16:19.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,6", 2024-09-20T22:16:19.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344284+0000", 2024-09-20T22:16:19.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9779999999999997e-05 2024-09-20T22:16:19.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.378 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:16:19.378 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.348919+0000", 2024-09-20T22:16:19.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0046344170000000001 2024-09-20T22:16:19.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.378 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.378 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.349022+0000", 2024-09-20T22:16:19.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103521 2024-09-20T22:16:19.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.378 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.378 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720993+0000", 2024-09-20T22:16:19.379 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.37197118600000001 2024-09-20T22:16:19.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.379 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:16:19.379 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.721105+0000", 2024-09-20T22:16:19.379 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111161 2024-09-20T22:16:19.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.379 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.379 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.721140+0000", 2024-09-20T22:16:19.379 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5728000000000001e-05 2024-09-20T22:16:19.379 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.379 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.379 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.380 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:125874 9.0 9:01e41a72:::benchmark_data_smithi060_140132_object15066:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:16:19.380 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:30.344246+0000", 2024-09-20T22:16:19.380 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 168.89943586300001, 2024-09-20T22:16:19.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.37701091199999998, 2024-09-20T22:16:19.380 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:16:19.380 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:16:19.380 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:16:19.380 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:16:19.380 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:16:19.380 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:16:19.380 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 125874 2024-09-20T22:16:19.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.381 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:16:19.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:16:19.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344246+0000", 2024-09-20T22:16:19.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:16:19.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344246+0000", 2024-09-20T22:16:19.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:16:19.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:16:19.382 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344247+0000", 2024-09-20T22:16:19.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6099999999999999e-07 2024-09-20T22:16:19.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.382 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:16:19.382 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344252+0000", 2024-09-20T22:16:19.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5129999999999998e-06 2024-09-20T22:16:19.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.382 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:16:19.382 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344263+0000", 2024-09-20T22:16:19.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1744e-05 2024-09-20T22:16:19.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:16:19.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344271+0000", 2024-09-20T22:16:19.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8709999999999997e-06 2024-09-20T22:16:19.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:16:19.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.346350+0000", 2024-09-20T22:16:19.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002079072 2024-09-20T22:16:19.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:16:19.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.346467+0000", 2024-09-20T22:16:19.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117207 2024-09-20T22:16:19.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,6", 2024-09-20T22:16:19.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.346581+0000", 2024-09-20T22:16:19.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113241 2024-09-20T22:16:19.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:16:19.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.348991+0000", 2024-09-20T22:16:19.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024100020000000001 2024-09-20T22:16:19.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.349689+0000", 2024-09-20T22:16:19.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00069810599999999999 2024-09-20T22:16:19.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:16:19.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.721177+0000", 2024-09-20T22:16:19.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.37148772800000002 2024-09-20T22:16:19.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:16:19.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.721218+0000", 2024-09-20T22:16:19.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1366999999999999e-05 2024-09-20T22:16:19.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:16:19.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:16:19.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:16:19.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.721257+0000", 2024-09-20T22:16:19.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9100000000000002e-05 2024-09-20T22:16:19.386 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.386 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.386 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.386 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:16:19.386 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:16:19.386 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:16:19.462 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:19.504 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:19.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:19.505 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:19.606 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:19.646 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:19.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:19.647 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:19.748 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:19.784 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:19.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:19.785 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:19.886 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:19.924 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:19.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:19.925 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:19.929 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 62 16 94495 94479 95.218 77.3125 0.00874232 0.0104962 2024-09-20T22:16:20.013 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 7 1 2024-09-20T22:16:20.013 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:19 smithi060 ceph-mon[29694]: 4.1 scrub starts 2024-09-20T22:16:20.013 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:19 smithi060 ceph-mon[29694]: 4.1 scrub ok 2024-09-20T22:16:20.013 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:19 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:16:20.013 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:19 smithi060 ceph-mon[29694]: osdmap e403: 8 total, 8 up, 8 in 2024-09-20T22:16:20.013 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:19 smithi060 ceph-mon[29694]: 1.8 scrub starts 2024-09-20T22:16:20.013 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:19 smithi060 ceph-mon[29694]: 1.8 scrub ok 2024-09-20T22:16:20.014 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:19 smithi060 ceph-mon[29694]: 5.b scrub starts 2024-09-20T22:16:20.014 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:19 smithi060 ceph-mon[29694]: 5.b scrub ok 2024-09-20T22:16:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:19 smithi067 ceph-mon[33511]: 4.1 scrub starts 2024-09-20T22:16:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:19 smithi067 ceph-mon[33511]: 4.1 scrub ok 2024-09-20T22:16:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:19 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:16:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:19 smithi067 ceph-mon[33511]: osdmap e403: 8 total, 8 up, 8 in 2024-09-20T22:16:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:19 smithi067 ceph-mon[33511]: 1.8 scrub starts 2024-09-20T22:16:20.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:19 smithi067 ceph-mon[33511]: 1.8 scrub ok 2024-09-20T22:16:20.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:19 smithi067 ceph-mon[33511]: 5.b scrub starts 2024-09-20T22:16:20.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:19 smithi067 ceph-mon[33511]: 5.b scrub ok 2024-09-20T22:16:20.015 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:19 smithi060 ceph-mon[40290]: 4.1 scrub starts 2024-09-20T22:16:20.015 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:19 smithi060 ceph-mon[40290]: 4.1 scrub ok 2024-09-20T22:16:20.015 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:19 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:16:20.015 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:19 smithi060 ceph-mon[40290]: osdmap e403: 8 total, 8 up, 8 in 2024-09-20T22:16:20.015 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:19 smithi060 ceph-mon[40290]: 1.8 scrub starts 2024-09-20T22:16:20.015 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:19 smithi060 ceph-mon[40290]: 1.8 scrub ok 2024-09-20T22:16:20.015 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:19 smithi060 ceph-mon[40290]: 5.b scrub starts 2024-09-20T22:16:20.015 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:19 smithi060 ceph-mon[40290]: 5.b scrub ok 2024-09-20T22:16:20.025 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:20.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:20.107 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:20.107 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:20.208 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:20.237 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:20.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:20.238 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:20.339 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:20.396 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:20.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:20.398 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:20.469 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:16:20.498 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:20.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:20.563 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:20.564 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:20.664 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:20.702 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:20.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:20.703 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:20.803 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:20.807 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:20.832 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:20.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:20.833 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:20.854 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T22:16:20.930 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 63 16 95519 95503 94.7216 64 0.014299 0.0105515 2024-09-20T22:16:20.934 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:20.963 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:20.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:20.964 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:20 smithi067 ceph-mon[33511]: 10.5 scrub starts 2024-09-20T22:16:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:20 smithi067 ceph-mon[33511]: 6.13 scrub starts 2024-09-20T22:16:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:20 smithi067 ceph-mon[33511]: 6.13 scrub ok 2024-09-20T22:16:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:20 smithi067 ceph-mon[33511]: pgmap v723: 168 pgs: 168 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 58 MiB/s wr, 932 op/s 2024-09-20T22:16:21.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:20 smithi060 ceph-mon[29694]: 10.5 scrub starts 2024-09-20T22:16:21.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:20 smithi060 ceph-mon[29694]: 6.13 scrub starts 2024-09-20T22:16:21.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:20 smithi060 ceph-mon[29694]: 6.13 scrub ok 2024-09-20T22:16:21.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:20 smithi060 ceph-mon[29694]: pgmap v723: 168 pgs: 168 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 58 MiB/s wr, 932 op/s 2024-09-20T22:16:21.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:20 smithi060 ceph-mon[40290]: 10.5 scrub starts 2024-09-20T22:16:21.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:20 smithi060 ceph-mon[40290]: 6.13 scrub starts 2024-09-20T22:16:21.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:20 smithi060 ceph-mon[40290]: 6.13 scrub ok 2024-09-20T22:16:21.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:20 smithi060 ceph-mon[40290]: pgmap v723: 168 pgs: 168 active+clean; 5.6 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 852 B/s rd, 58 MiB/s wr, 932 op/s 2024-09-20T22:16:21.066 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:21.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:21.112 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:21.112 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:21.213 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:21.240 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:21.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:21.241 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:21.342 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:21.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:21.382 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:21.382 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:21.483 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:21.506 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:21.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:21.507 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:21.608 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:21.633 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:21.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:21.634 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:21.735 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:21.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:21.780 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:21.781 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:21.881 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:21.923 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:21.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:21.924 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:21.930 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 64 16 96402 96386 94.1038 55.1875 0.0139893 0.0106208 2024-09-20T22:16:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:21 smithi067 ceph-mon[33511]: 3.3 scrub starts 2024-09-20T22:16:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:21 smithi067 ceph-mon[33511]: 3.3 scrub ok 2024-09-20T22:16:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:16:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:21 smithi067 ceph-mon[33511]: osdmap e404: 8 total, 8 up, 8 in 2024-09-20T22:16:22.025 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:22.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:21 smithi060 ceph-mon[29694]: 3.3 scrub starts 2024-09-20T22:16:22.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:21 smithi060 ceph-mon[29694]: 3.3 scrub ok 2024-09-20T22:16:22.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:16:22.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:21 smithi060 ceph-mon[29694]: osdmap e404: 8 total, 8 up, 8 in 2024-09-20T22:16:22.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:21 smithi060 ceph-mon[40290]: 3.3 scrub starts 2024-09-20T22:16:22.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:21 smithi060 ceph-mon[40290]: 3.3 scrub ok 2024-09-20T22:16:22.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:16:22.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:21 smithi060 ceph-mon[40290]: osdmap e404: 8 total, 8 up, 8 in 2024-09-20T22:16:22.064 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:22.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:22.065 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:22.166 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:22.209 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:22.209 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:22.216 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:22.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:22.218 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:22.319 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:22.354 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:22.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:22.355 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:22.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:22.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:22.505 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:22.505 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:22.606 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:22.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:22.650 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:22.650 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:22.751 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:22.762 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:16:22.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:22.785 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:22.786 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:22.887 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:22.914 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:16:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:16:22.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:22 smithi067 ceph-mon[33511]: 4.11 scrub starts 2024-09-20T22:16:22.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:22 smithi067 ceph-mon[33511]: 4.11 scrub ok 2024-09-20T22:16:22.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:22 smithi067 ceph-mon[33511]: 10.5 scrub ok 2024-09-20T22:16:22.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:22 smithi067 ceph-mon[33511]: pgmap v725: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 682 B/s rd, 73 MiB/s wr, 1.16k op/s 2024-09-20T22:16:22.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:22 smithi067 ceph-mon[33511]: 3.4 scrub starts 2024-09-20T22:16:22.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:22 smithi067 ceph-mon[33511]: 3.4 scrub ok 2024-09-20T22:16:22.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:22 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:16:22.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:22 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3565489550' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:16:22.924 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:22.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:22.925 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:22.930 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 65 16 97498 97482 93.7099 68.5 0.00848256 0.0106666 2024-09-20T22:16:22.947 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:16:22.947 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:16:22.984 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:16:22.984 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:16:23.018 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:16:23.018 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:16:23.018 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:22 smithi060 ceph-mon[29694]: 4.11 scrub starts 2024-09-20T22:16:23.018 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:22 smithi060 ceph-mon[29694]: 4.11 scrub ok 2024-09-20T22:16:23.018 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:22 smithi060 ceph-mon[29694]: 10.5 scrub ok 2024-09-20T22:16:23.019 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:22 smithi060 ceph-mon[29694]: pgmap v725: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 682 B/s rd, 73 MiB/s wr, 1.16k op/s 2024-09-20T22:16:23.019 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:22 smithi060 ceph-mon[29694]: 3.4 scrub starts 2024-09-20T22:16:23.019 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:22 smithi060 ceph-mon[29694]: 3.4 scrub ok 2024-09-20T22:16:23.019 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:22 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:16:23.019 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:22 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3565489550' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:16:23.020 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:22 smithi060 ceph-mon[40290]: 4.11 scrub starts 2024-09-20T22:16:23.020 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:22 smithi060 ceph-mon[40290]: 4.11 scrub ok 2024-09-20T22:16:23.020 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:22 smithi060 ceph-mon[40290]: 10.5 scrub ok 2024-09-20T22:16:23.020 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:22 smithi060 ceph-mon[40290]: pgmap v725: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 682 B/s rd, 73 MiB/s wr, 1.16k op/s 2024-09-20T22:16:23.020 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:22 smithi060 ceph-mon[40290]: 3.4 scrub starts 2024-09-20T22:16:23.020 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:22 smithi060 ceph-mon[40290]: 3.4 scrub ok 2024-09-20T22:16:23.020 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:22 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:16:23.020 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:22 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3565489550' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:16:23.026 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:23.050 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:16:23.050 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:16:23.063 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:23.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:23.064 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:23.085 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:16:23.085 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:16:23.123 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:16:23.123 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:16:23.156 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:16:23.156 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:16:23.165 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:23.191 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:16:23.191 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:16:23.205 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:23.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:23.206 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:23.307 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:23.345 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:23.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:23.346 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:23.447 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:23.483 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:23.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:23.484 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:23.585 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:23.623 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:23.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:23.625 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:23.726 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:23.767 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:23.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:23.768 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[29694]: 6.a scrub starts 2024-09-20T22:16:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[29694]: 6.a scrub ok 2024-09-20T22:16:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[29694]: 10.1 scrub starts 2024-09-20T22:16:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[29694]: 4.17 scrub starts 2024-09-20T22:16:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[29694]: 4.17 scrub ok 2024-09-20T22:16:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:16:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:16:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3565489550' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:16:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[29694]: osdmap e405: 8 total, 8 up, 8 in 2024-09-20T22:16:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[29694]: 3.1e scrub starts 2024-09-20T22:16:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[29694]: 3.1e scrub ok 2024-09-20T22:16:23.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2543994504' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:23.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[29694]: 5.1d scrub starts 2024-09-20T22:16:23.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[29694]: 5.1d scrub ok 2024-09-20T22:16:23.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[40290]: 6.a scrub starts 2024-09-20T22:16:23.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[40290]: 6.a scrub ok 2024-09-20T22:16:23.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[40290]: 10.1 scrub starts 2024-09-20T22:16:23.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[40290]: 4.17 scrub starts 2024-09-20T22:16:23.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[40290]: 4.17 scrub ok 2024-09-20T22:16:23.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:16:23.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:16:23.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3565489550' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:16:23.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[40290]: osdmap e405: 8 total, 8 up, 8 in 2024-09-20T22:16:23.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[40290]: 3.1e scrub starts 2024-09-20T22:16:23.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[40290]: 3.1e scrub ok 2024-09-20T22:16:23.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2543994504' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:23.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[40290]: 5.1d scrub starts 2024-09-20T22:16:23.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:23 smithi060 ceph-mon[40290]: 5.1d scrub ok 2024-09-20T22:16:23.868 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:23.909 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:23.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:23.910 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:23.930 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 66 16 98595 98579 93.3289 68.5625 0.0075657 0.0107094 2024-09-20T22:16:24.011 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:24.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:23 smithi067 ceph-mon[33511]: 6.a scrub starts 2024-09-20T22:16:24.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:23 smithi067 ceph-mon[33511]: 6.a scrub ok 2024-09-20T22:16:24.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:23 smithi067 ceph-mon[33511]: 10.1 scrub starts 2024-09-20T22:16:24.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:23 smithi067 ceph-mon[33511]: 4.17 scrub starts 2024-09-20T22:16:24.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:23 smithi067 ceph-mon[33511]: 4.17 scrub ok 2024-09-20T22:16:24.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:23 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:16:24.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:23 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:16:24.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:23 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3565489550' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:16:24.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:23 smithi067 ceph-mon[33511]: osdmap e405: 8 total, 8 up, 8 in 2024-09-20T22:16:24.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:23 smithi067 ceph-mon[33511]: 3.1e scrub starts 2024-09-20T22:16:24.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:23 smithi067 ceph-mon[33511]: 3.1e scrub ok 2024-09-20T22:16:24.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:23 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2543994504' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:24.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:23 smithi067 ceph-mon[33511]: 5.1d scrub starts 2024-09-20T22:16:24.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:23 smithi067 ceph-mon[33511]: 5.1d scrub ok 2024-09-20T22:16:24.054 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:24.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:24.055 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:24.156 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:24.194 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:24.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:24.195 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:24.295 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:24.332 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:24.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:24.333 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:24.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:24.473 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:24.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:24.474 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:24.575 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:24.605 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:24.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:24.606 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:24.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:24.743 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:24.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:24.744 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:24.846 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:24.890 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:24.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:24.891 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:24.930 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 67 16 99846 99830 93.1028 78.1875 0.0139625 0.0107362 2024-09-20T22:16:24.992 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:24.994 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:24 smithi067 ceph-mon[33511]: 6.1a scrub starts 2024-09-20T22:16:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:24 smithi067 ceph-mon[33511]: 6.1a scrub ok 2024-09-20T22:16:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:24 smithi067 ceph-mon[33511]: 6.5 scrub starts 2024-09-20T22:16:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:24 smithi067 ceph-mon[33511]: 6.5 scrub ok 2024-09-20T22:16:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:24 smithi067 ceph-mon[33511]: 3.10 scrub starts 2024-09-20T22:16:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:24 smithi067 ceph-mon[33511]: 3.10 scrub ok 2024-09-20T22:16:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:24 smithi067 ceph-mon[33511]: 6.9 scrub starts 2024-09-20T22:16:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:24 smithi067 ceph-mon[33511]: 6.9 scrub ok 2024-09-20T22:16:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:24 smithi067 ceph-mon[33511]: 10.1 scrub ok 2024-09-20T22:16:25.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:24 smithi067 ceph-mon[33511]: pgmap v727: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 5.9 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 682 B/s rd, 62 MiB/s wr, 989 op/s 2024-09-20T22:16:25.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[29694]: 6.1a scrub starts 2024-09-20T22:16:25.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[29694]: 6.1a scrub ok 2024-09-20T22:16:25.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[29694]: 6.5 scrub starts 2024-09-20T22:16:25.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[29694]: 6.5 scrub ok 2024-09-20T22:16:25.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[29694]: 3.10 scrub starts 2024-09-20T22:16:25.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[29694]: 3.10 scrub ok 2024-09-20T22:16:25.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[29694]: 6.9 scrub starts 2024-09-20T22:16:25.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[29694]: 6.9 scrub ok 2024-09-20T22:16:25.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[29694]: 10.1 scrub ok 2024-09-20T22:16:25.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[29694]: pgmap v727: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 5.9 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 682 B/s rd, 62 MiB/s wr, 989 op/s 2024-09-20T22:16:25.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[40290]: 6.1a scrub starts 2024-09-20T22:16:25.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[40290]: 6.1a scrub ok 2024-09-20T22:16:25.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[40290]: 6.5 scrub starts 2024-09-20T22:16:25.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[40290]: 6.5 scrub ok 2024-09-20T22:16:25.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[40290]: 3.10 scrub starts 2024-09-20T22:16:25.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[40290]: 3.10 scrub ok 2024-09-20T22:16:25.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[40290]: 6.9 scrub starts 2024-09-20T22:16:25.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[40290]: 6.9 scrub ok 2024-09-20T22:16:25.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[40290]: 10.1 scrub ok 2024-09-20T22:16:25.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:24 smithi060 ceph-mon[40290]: pgmap v727: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 5.9 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 682 B/s rd, 62 MiB/s wr, 989 op/s 2024-09-20T22:16:25.059 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:25.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:25.061 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:25.161 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:25.188 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:25.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:25.189 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:25.289 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:25.311 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:16:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:16:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:16:25.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:25.319 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:25.319 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:25.420 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:25.455 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:25.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:25.456 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:25.557 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:25.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:25.604 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:25.605 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:25.665 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:16:25.665 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:16:25.705 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:25.736 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:25.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:25.737 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:25.772 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.7 to 1 (10000) 2024-09-20T22:16:25.839 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:25.875 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:25.876 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:25.876 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:25.898 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:25 smithi060 ceph-mon[29694]: 4.e scrub starts 2024-09-20T22:16:25.898 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:25 smithi060 ceph-mon[29694]: 4.e scrub ok 2024-09-20T22:16:25.898 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:25 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1498651803' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:16:25.899 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:25 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:16:25.899 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:25 smithi060 ceph-mon[40290]: 4.e scrub starts 2024-09-20T22:16:25.899 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:25 smithi060 ceph-mon[40290]: 4.e scrub ok 2024-09-20T22:16:25.899 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:25 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1498651803' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:16:25.899 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:25 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:16:25.931 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 68 15 100994 100979 92.7897 71.8125 0.0115829 0.0107723 2024-09-20T22:16:25.977 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:26.013 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:26.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:26.014 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:26.028 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:16:26.115 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:26.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:26.154 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:26.155 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:26.204 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:25 smithi067 ceph-mon[33511]: 4.e scrub starts 2024-09-20T22:16:26.204 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:25 smithi067 ceph-mon[33511]: 4.e scrub ok 2024-09-20T22:16:26.204 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:25 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1498651803' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:16:26.204 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:25 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:16:26.255 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:26.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:26.287 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:26.288 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:26.388 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:26.423 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:26.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:26.424 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:26.525 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:26.561 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:26.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:26.562 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:26.662 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:26.699 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:26.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:26.700 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:26.742 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 0 1 2024-09-20T22:16:26.787 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:16:26.801 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:26.892 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:26.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:26.893 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:26.931 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 69 16 102171 102155 92.5094 73.5 0.00788538 0.0108046 2024-09-20T22:16:26.994 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:27.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:27.025 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:27.025 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:26 smithi060 ceph-mon[29694]: 3.e scrub starts 2024-09-20T22:16:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:26 smithi060 ceph-mon[29694]: 3.e scrub ok 2024-09-20T22:16:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:26 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:16:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:26 smithi060 ceph-mon[29694]: osdmap e406: 8 total, 8 up, 8 in 2024-09-20T22:16:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:26 smithi060 ceph-mon[29694]: 5.3 scrub starts 2024-09-20T22:16:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:26 smithi060 ceph-mon[29694]: 5.3 scrub ok 2024-09-20T22:16:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:26 smithi060 ceph-mon[29694]: pgmap v729: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.5 KiB/s rd, 75 MiB/s wr, 1.21k op/s 2024-09-20T22:16:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:26 smithi060 ceph-mon[29694]: 6.4 deep-scrub starts 2024-09-20T22:16:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:26 smithi060 ceph-mon[29694]: 6.4 deep-scrub ok 2024-09-20T22:16:27.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:26 smithi060 ceph-mon[40290]: 3.e scrub starts 2024-09-20T22:16:27.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:26 smithi060 ceph-mon[40290]: 3.e scrub ok 2024-09-20T22:16:27.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:26 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:16:27.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:26 smithi060 ceph-mon[40290]: osdmap e406: 8 total, 8 up, 8 in 2024-09-20T22:16:27.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:26 smithi060 ceph-mon[40290]: 5.3 scrub starts 2024-09-20T22:16:27.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:26 smithi060 ceph-mon[40290]: 5.3 scrub ok 2024-09-20T22:16:27.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:26 smithi060 ceph-mon[40290]: pgmap v729: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.5 KiB/s rd, 75 MiB/s wr, 1.21k op/s 2024-09-20T22:16:27.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:26 smithi060 ceph-mon[40290]: 6.4 deep-scrub starts 2024-09-20T22:16:27.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:26 smithi060 ceph-mon[40290]: 6.4 deep-scrub ok 2024-09-20T22:16:27.126 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:27.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:27.167 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:27.167 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:27.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:26 smithi067 ceph-mon[33511]: 3.e scrub starts 2024-09-20T22:16:27.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:26 smithi067 ceph-mon[33511]: 3.e scrub ok 2024-09-20T22:16:27.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:26 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:16:27.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:26 smithi067 ceph-mon[33511]: osdmap e406: 8 total, 8 up, 8 in 2024-09-20T22:16:27.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:26 smithi067 ceph-mon[33511]: 5.3 scrub starts 2024-09-20T22:16:27.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:26 smithi067 ceph-mon[33511]: 5.3 scrub ok 2024-09-20T22:16:27.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:26 smithi067 ceph-mon[33511]: pgmap v729: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.0 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 1.5 KiB/s rd, 75 MiB/s wr, 1.21k op/s 2024-09-20T22:16:27.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:26 smithi067 ceph-mon[33511]: 6.4 deep-scrub starts 2024-09-20T22:16:27.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:26 smithi067 ceph-mon[33511]: 6.4 deep-scrub ok 2024-09-20T22:16:27.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:27.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:27.291 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:27.291 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:27.392 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:27.427 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:27.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:27.428 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:27.529 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:27.568 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:27.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:27.569 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:27.670 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:27.702 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:27.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:27.703 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:27.804 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:27.838 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:27.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:27.839 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:27.931 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 70 16 103236 103220 92.1387 66.5625 0.00670148 0.0108483 2024-09-20T22:16:27.940 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:27.974 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:27.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:27.975 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:28.075 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:28.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:28.110 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:28.110 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:28.211 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:28.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:28.247 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:28.247 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:28.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:27 smithi067 ceph-mon[33511]: 6.14 deep-scrub starts 2024-09-20T22:16:28.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:27 smithi067 ceph-mon[33511]: 6.14 deep-scrub ok 2024-09-20T22:16:28.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:27 smithi060 ceph-mon[29694]: 6.14 deep-scrub starts 2024-09-20T22:16:28.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:27 smithi060 ceph-mon[29694]: 6.14 deep-scrub ok 2024-09-20T22:16:28.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:27 smithi060 ceph-mon[40290]: 6.14 deep-scrub starts 2024-09-20T22:16:28.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:27 smithi060 ceph-mon[40290]: 6.14 deep-scrub ok 2024-09-20T22:16:28.347 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:28.391 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:28.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:28.392 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:28.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:28.536 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:28.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:28.537 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:28.638 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:28.668 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:28.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:28.669 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:28.770 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:28.814 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:28.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:28.815 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:28.916 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:28.931 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 71 16 104250 104234 91.7336 63.375 0.0148299 0.0108961 2024-09-20T22:16:28.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:28.954 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:28.954 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:29.055 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:29.090 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:29.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:29.091 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:29.192 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:29.225 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:29.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:29.225 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:28 smithi067 ceph-mon[33511]: 6.e scrub starts 2024-09-20T22:16:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:28 smithi067 ceph-mon[33511]: 6.e scrub ok 2024-09-20T22:16:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:28 smithi067 ceph-mon[33511]: 6.1b scrub starts 2024-09-20T22:16:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:28 smithi067 ceph-mon[33511]: 6.1b scrub ok 2024-09-20T22:16:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:28 smithi067 ceph-mon[33511]: 10.7 scrub starts 2024-09-20T22:16:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:28 smithi067 ceph-mon[33511]: 3.12 scrub starts 2024-09-20T22:16:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:28 smithi067 ceph-mon[33511]: 3.12 scrub ok 2024-09-20T22:16:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:28 smithi067 ceph-mon[33511]: pgmap v730: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 765 B/s rd, 79 MiB/s wr, 1.26k op/s 2024-09-20T22:16:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:28 smithi067 ceph-mon[33511]: 4.2 scrub starts 2024-09-20T22:16:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:28 smithi067 ceph-mon[33511]: 4.2 scrub ok 2024-09-20T22:16:29.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[29694]: 6.e scrub starts 2024-09-20T22:16:29.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[29694]: 6.e scrub ok 2024-09-20T22:16:29.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[29694]: 6.1b scrub starts 2024-09-20T22:16:29.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[29694]: 6.1b scrub ok 2024-09-20T22:16:29.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[29694]: 10.7 scrub starts 2024-09-20T22:16:29.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[29694]: 3.12 scrub starts 2024-09-20T22:16:29.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[29694]: 3.12 scrub ok 2024-09-20T22:16:29.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[29694]: pgmap v730: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 765 B/s rd, 79 MiB/s wr, 1.26k op/s 2024-09-20T22:16:29.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[29694]: 4.2 scrub starts 2024-09-20T22:16:29.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[29694]: 4.2 scrub ok 2024-09-20T22:16:29.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[40290]: 6.e scrub starts 2024-09-20T22:16:29.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[40290]: 6.e scrub ok 2024-09-20T22:16:29.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[40290]: 6.1b scrub starts 2024-09-20T22:16:29.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[40290]: 6.1b scrub ok 2024-09-20T22:16:29.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[40290]: 10.7 scrub starts 2024-09-20T22:16:29.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[40290]: 3.12 scrub starts 2024-09-20T22:16:29.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[40290]: 3.12 scrub ok 2024-09-20T22:16:29.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[40290]: pgmap v730: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 765 B/s rd, 79 MiB/s wr, 1.26k op/s 2024-09-20T22:16:29.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[40290]: 4.2 scrub starts 2024-09-20T22:16:29.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:28 smithi060 ceph-mon[40290]: 4.2 scrub ok 2024-09-20T22:16:29.326 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:29.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:29.362 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:29.362 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:29.463 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:29.501 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:29.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:29.502 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:29.603 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:29.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:29.637 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:29.638 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:29.738 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:29.782 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:29.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:29.783 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:29.884 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:29.920 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:29.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:29.921 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:29.931 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 72 15 105231 105216 91.3119 61.375 0.0163009 0.010947 2024-09-20T22:16:30.022 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:30.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:30.067 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:30.067 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:30.168 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:30.204 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:30.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:30.205 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:30.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:29 smithi067 ceph-mon[33511]: 4.13 scrub starts 2024-09-20T22:16:30.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:29 smithi067 ceph-mon[33511]: 4.13 scrub ok 2024-09-20T22:16:30.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:29 smithi067 ceph-mon[33511]: 3.7 scrub starts 2024-09-20T22:16:30.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:29 smithi067 ceph-mon[33511]: 3.7 scrub ok 2024-09-20T22:16:30.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:29 smithi067 ceph-mon[33511]: 10.4 scrub starts 2024-09-20T22:16:30.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:29 smithi060 ceph-mon[29694]: 4.13 scrub starts 2024-09-20T22:16:30.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:29 smithi060 ceph-mon[29694]: 4.13 scrub ok 2024-09-20T22:16:30.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:29 smithi060 ceph-mon[29694]: 3.7 scrub starts 2024-09-20T22:16:30.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:29 smithi060 ceph-mon[29694]: 3.7 scrub ok 2024-09-20T22:16:30.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:29 smithi060 ceph-mon[29694]: 10.4 scrub starts 2024-09-20T22:16:30.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:29 smithi060 ceph-mon[40290]: 4.13 scrub starts 2024-09-20T22:16:30.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:29 smithi060 ceph-mon[40290]: 4.13 scrub ok 2024-09-20T22:16:30.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:29 smithi060 ceph-mon[40290]: 3.7 scrub starts 2024-09-20T22:16:30.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:29 smithi060 ceph-mon[40290]: 3.7 scrub ok 2024-09-20T22:16:30.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:29 smithi060 ceph-mon[40290]: 10.4 scrub starts 2024-09-20T22:16:30.306 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:30.345 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:30.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:30.346 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:30.446 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:30.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:30.504 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:30.504 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:30.605 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:30.652 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:30.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:30.653 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:30.753 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:30.792 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:30.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:30.793 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:30.894 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:30.915 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:30.916 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:30.916 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:30.932 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 73 16 106120 106104 90.8212 55.5 0.0113105 0.0109988 2024-09-20T22:16:31.017 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:31.049 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:31.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:31.051 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:31.151 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:31.187 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:31.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:31.188 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:31.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:30 smithi067 ceph-mon[33511]: 4.18 scrub starts 2024-09-20T22:16:31.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:30 smithi067 ceph-mon[33511]: 4.18 scrub ok 2024-09-20T22:16:31.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:30 smithi067 ceph-mon[33511]: 3.c scrub starts 2024-09-20T22:16:31.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:30 smithi067 ceph-mon[33511]: 3.c scrub ok 2024-09-20T22:16:31.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:30 smithi067 ceph-mon[33511]: 10.3 scrub starts 2024-09-20T22:16:31.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:30 smithi067 ceph-mon[33511]: 10.8 scrub starts 2024-09-20T22:16:31.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:30 smithi067 ceph-mon[33511]: pgmap v731: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 63 MiB/s wr, 1.01k op/s 2024-09-20T22:16:31.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:30 smithi060 ceph-mon[29694]: 4.18 scrub starts 2024-09-20T22:16:31.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:30 smithi060 ceph-mon[29694]: 4.18 scrub ok 2024-09-20T22:16:31.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:30 smithi060 ceph-mon[29694]: 3.c scrub starts 2024-09-20T22:16:31.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:30 smithi060 ceph-mon[29694]: 3.c scrub ok 2024-09-20T22:16:31.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:30 smithi060 ceph-mon[29694]: 10.3 scrub starts 2024-09-20T22:16:31.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:30 smithi060 ceph-mon[29694]: 10.8 scrub starts 2024-09-20T22:16:31.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:30 smithi060 ceph-mon[29694]: pgmap v731: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 63 MiB/s wr, 1.01k op/s 2024-09-20T22:16:31.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:30 smithi060 ceph-mon[40290]: 4.18 scrub starts 2024-09-20T22:16:31.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:30 smithi060 ceph-mon[40290]: 4.18 scrub ok 2024-09-20T22:16:31.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:30 smithi060 ceph-mon[40290]: 3.c scrub starts 2024-09-20T22:16:31.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:30 smithi060 ceph-mon[40290]: 3.c scrub ok 2024-09-20T22:16:31.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:30 smithi060 ceph-mon[40290]: 10.3 scrub starts 2024-09-20T22:16:31.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:30 smithi060 ceph-mon[40290]: 10.8 scrub starts 2024-09-20T22:16:31.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:30 smithi060 ceph-mon[40290]: pgmap v731: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 639 B/s rd, 63 MiB/s wr, 1.01k op/s 2024-09-20T22:16:31.289 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:31.334 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:31.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:31.334 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:31.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:31.470 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:31.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:31.471 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:31.542 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T22:16:31.572 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:31.596 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:31.638 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:31.656 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:31.659 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:31.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:31.661 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:31.761 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:31.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:31.831 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:31.831 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:31.932 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:31.935 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 74 16 106940 106924 90.2866 51.25 0.00954438 0.0110702 2024-09-20T22:16:31.992 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:31.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:31.993 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:32.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:31 smithi060 ceph-mon[40290]: 10.7 scrub ok 2024-09-20T22:16:32.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:31 smithi060 ceph-mon[40290]: 10.a scrub starts 2024-09-20T22:16:32.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:31 smithi060 ceph-mon[40290]: 4.19 deep-scrub starts 2024-09-20T22:16:32.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:31 smithi060 ceph-mon[40290]: 4.19 deep-scrub ok 2024-09-20T22:16:32.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:31 smithi060 ceph-mon[40290]: 6.0 deep-scrub starts 2024-09-20T22:16:32.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:31 smithi060 ceph-mon[40290]: 6.0 deep-scrub ok 2024-09-20T22:16:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:31 smithi060 ceph-mon[29694]: 10.7 scrub ok 2024-09-20T22:16:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:31 smithi060 ceph-mon[29694]: 10.a scrub starts 2024-09-20T22:16:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:31 smithi060 ceph-mon[29694]: 4.19 deep-scrub starts 2024-09-20T22:16:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:31 smithi060 ceph-mon[29694]: 4.19 deep-scrub ok 2024-09-20T22:16:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:31 smithi060 ceph-mon[29694]: 6.0 deep-scrub starts 2024-09-20T22:16:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:31 smithi060 ceph-mon[29694]: 6.0 deep-scrub ok 2024-09-20T22:16:32.093 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:32.129 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:32.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:32.131 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:32.231 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:32.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:32.260 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:32.260 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:32.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:31 smithi067 ceph-mon[33511]: 10.7 scrub ok 2024-09-20T22:16:32.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:31 smithi067 ceph-mon[33511]: 10.a scrub starts 2024-09-20T22:16:32.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:31 smithi067 ceph-mon[33511]: 4.19 deep-scrub starts 2024-09-20T22:16:32.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:31 smithi067 ceph-mon[33511]: 4.19 deep-scrub ok 2024-09-20T22:16:32.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:31 smithi067 ceph-mon[33511]: 6.0 deep-scrub starts 2024-09-20T22:16:32.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:31 smithi067 ceph-mon[33511]: 6.0 deep-scrub ok 2024-09-20T22:16:32.361 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:32.403 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:32.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:32.404 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:32.505 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:32.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:32.546 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:32.546 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:32.647 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:32.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:32.681 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:32.682 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:32.782 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:32.822 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:32.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:32.823 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:32.828 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:16:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:16:32.924 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:32.932 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 75 16 107966 107950 89.9377 64.125 0.00771835 0.0111123 2024-09-20T22:16:32.960 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:32.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:32.961 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:33.063 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:33.087 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[40290]: 3.8 scrub starts 2024-09-20T22:16:33.087 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[40290]: 3.8 scrub ok 2024-09-20T22:16:33.087 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[40290]: 3.13 scrub starts 2024-09-20T22:16:33.087 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[40290]: 3.13 scrub ok 2024-09-20T22:16:33.087 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[40290]: 10.4 scrub ok 2024-09-20T22:16:33.087 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[40290]: 3.1f deep-scrub starts 2024-09-20T22:16:33.087 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[40290]: 3.1f deep-scrub ok 2024-09-20T22:16:33.087 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[40290]: 5.12 scrub starts 2024-09-20T22:16:33.087 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[40290]: 5.12 scrub ok 2024-09-20T22:16:33.087 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[40290]: pgmap v732: 168 pgs: 3 active+clean+scrubbing, 165 active+clean; 6.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-20T22:16:33.088 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:16:33.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[29694]: 3.8 scrub starts 2024-09-20T22:16:33.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[29694]: 3.8 scrub ok 2024-09-20T22:16:33.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[29694]: 3.13 scrub starts 2024-09-20T22:16:33.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[29694]: 3.13 scrub ok 2024-09-20T22:16:33.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[29694]: 10.4 scrub ok 2024-09-20T22:16:33.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[29694]: 3.1f deep-scrub starts 2024-09-20T22:16:33.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[29694]: 3.1f deep-scrub ok 2024-09-20T22:16:33.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[29694]: 5.12 scrub starts 2024-09-20T22:16:33.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[29694]: 5.12 scrub ok 2024-09-20T22:16:33.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[29694]: pgmap v732: 168 pgs: 3 active+clean+scrubbing, 165 active+clean; 6.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-20T22:16:33.088 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:32 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:16:33.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:33.091 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:33.091 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:33.192 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:33.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:33.224 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:33.225 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:33.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:32 smithi067 ceph-mon[33511]: 3.8 scrub starts 2024-09-20T22:16:33.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:32 smithi067 ceph-mon[33511]: 3.8 scrub ok 2024-09-20T22:16:33.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:32 smithi067 ceph-mon[33511]: 3.13 scrub starts 2024-09-20T22:16:33.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:32 smithi067 ceph-mon[33511]: 3.13 scrub ok 2024-09-20T22:16:33.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:32 smithi067 ceph-mon[33511]: 10.4 scrub ok 2024-09-20T22:16:33.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:32 smithi067 ceph-mon[33511]: 3.1f deep-scrub starts 2024-09-20T22:16:33.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:32 smithi067 ceph-mon[33511]: 3.1f deep-scrub ok 2024-09-20T22:16:33.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:32 smithi067 ceph-mon[33511]: 5.12 scrub starts 2024-09-20T22:16:33.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:32 smithi067 ceph-mon[33511]: 5.12 scrub ok 2024-09-20T22:16:33.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:32 smithi067 ceph-mon[33511]: pgmap v732: 168 pgs: 3 active+clean+scrubbing, 165 active+clean; 6.4 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.1 KiB/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-20T22:16:33.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:32 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:16:33.326 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:33.363 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:33.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:33.364 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:33.464 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:33.502 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:33.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:33.503 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:33.604 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:33.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:33.644 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:33.644 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:33.745 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:33.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:33.781 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:33.781 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:33.882 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:33.926 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:33.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:33.927 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:33.932 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 76 16 108915 108899 89.5347 59.3125 0.0165475 0.0111628 2024-09-20T22:16:34.027 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:34.072 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:34.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:34.073 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:34.174 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:34.211 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:34.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:34.212 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:34.220 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:16:34.220 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:16:34.220 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:16:34.220 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:16:34.220 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:16:34.220 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:16:34.220 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:16:34.220 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:16:34.220 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:16:34.220 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:16:34.220 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:16:34.220 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:16:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:16:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:16:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:16:34.221 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:16:34.221 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:33 smithi060 ceph-mon[29694]: 10.e scrub starts 2024-09-20T22:16:34.221 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:33 smithi060 ceph-mon[29694]: 10.3 scrub ok 2024-09-20T22:16:34.221 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:33 smithi060 ceph-mon[29694]: 10.8 scrub ok 2024-09-20T22:16:34.221 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:33 smithi060 ceph-mon[29694]: 10.a scrub ok 2024-09-20T22:16:34.221 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:33 smithi060 ceph-mon[29694]: 5.9 scrub starts 2024-09-20T22:16:34.221 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:33 smithi060 ceph-mon[29694]: 5.9 scrub ok 2024-09-20T22:16:34.221 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:33 smithi060 ceph-mon[29694]: 5.a scrub starts 2024-09-20T22:16:34.222 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:33 smithi060 ceph-mon[29694]: 5.a scrub ok 2024-09-20T22:16:34.222 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:33 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3021949549' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:34.222 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:33 smithi060 ceph-mon[40290]: 10.e scrub starts 2024-09-20T22:16:34.222 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:33 smithi060 ceph-mon[40290]: 10.3 scrub ok 2024-09-20T22:16:34.222 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:33 smithi060 ceph-mon[40290]: 10.8 scrub ok 2024-09-20T22:16:34.222 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:33 smithi060 ceph-mon[40290]: 10.a scrub ok 2024-09-20T22:16:34.222 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:33 smithi060 ceph-mon[40290]: 5.9 scrub starts 2024-09-20T22:16:34.222 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:33 smithi060 ceph-mon[40290]: 5.9 scrub ok 2024-09-20T22:16:34.222 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:33 smithi060 ceph-mon[40290]: 5.a scrub starts 2024-09-20T22:16:34.222 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:33 smithi060 ceph-mon[40290]: 5.a scrub ok 2024-09-20T22:16:34.222 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:33 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3021949549' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:34.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:33 smithi067 ceph-mon[33511]: 10.e scrub starts 2024-09-20T22:16:34.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:33 smithi067 ceph-mon[33511]: 10.3 scrub ok 2024-09-20T22:16:34.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:33 smithi067 ceph-mon[33511]: 10.8 scrub ok 2024-09-20T22:16:34.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:33 smithi067 ceph-mon[33511]: 10.a scrub ok 2024-09-20T22:16:34.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:33 smithi067 ceph-mon[33511]: 5.9 scrub starts 2024-09-20T22:16:34.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:33 smithi067 ceph-mon[33511]: 5.9 scrub ok 2024-09-20T22:16:34.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:33 smithi067 ceph-mon[33511]: 5.a scrub starts 2024-09-20T22:16:34.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:33 smithi067 ceph-mon[33511]: 5.a scrub ok 2024-09-20T22:16:34.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:33 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3021949549' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:34.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:34.335 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:34.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:34.336 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:34.436 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:34.485 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:34.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:34.486 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:34.586 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:34.623 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:34.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:34.624 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:34.724 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:34.760 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:34.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:34.761 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:34.844 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.0 to 1 (10000) 2024-09-20T22:16:34.862 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:34.902 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:34.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:34.903 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:34.932 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 77 16 109957 109941 89.2177 65.125 0.00756957 0.0112039 2024-09-20T22:16:35.004 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:35.042 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:35.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:35.044 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:35.144 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:35.181 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:35.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:35.182 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:35.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:34 smithi067 ceph-mon[33511]: 10.d scrub starts 2024-09-20T22:16:35.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:34 smithi067 ceph-mon[33511]: pgmap v733: 168 pgs: 2 active+clean+scrubbing, 166 active+clean; 6.5 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 65 MiB/s wr, 1.05k op/s 2024-09-20T22:16:35.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:34 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2160550257' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:16:35.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:34 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:16:35.283 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:35.288 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:34 smithi060 ceph-mon[29694]: 10.d scrub starts 2024-09-20T22:16:35.288 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:34 smithi060 ceph-mon[29694]: pgmap v733: 168 pgs: 2 active+clean+scrubbing, 166 active+clean; 6.5 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 65 MiB/s wr, 1.05k op/s 2024-09-20T22:16:35.288 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:34 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2160550257' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:16:35.288 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:34 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:16:35.288 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:16:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:16:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:16:35.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:34 smithi060 ceph-mon[40290]: 10.d scrub starts 2024-09-20T22:16:35.289 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:34 smithi060 ceph-mon[40290]: pgmap v733: 168 pgs: 2 active+clean+scrubbing, 166 active+clean; 6.5 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 65 MiB/s wr, 1.05k op/s 2024-09-20T22:16:35.289 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:34 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2160550257' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:16:35.289 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:34 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:16:35.321 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:35.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:35.322 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:35.422 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:35.461 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:35.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:35.463 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:35.563 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:35.593 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:35.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:35.594 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:35.695 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:35.734 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:35.734 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:35.735 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:35.835 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:35.868 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:35.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:35.869 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:35.932 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 78 16 111175 111159 89.0498 76.125 0.0107174 0.0112247 2024-09-20T22:16:35.969 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:36.011 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:36.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:36.013 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:36.113 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:36.150 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:36.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:36.151 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:36.252 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:36.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:35 smithi067 ceph-mon[33511]: 10.e scrub ok 2024-09-20T22:16:36.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:35 smithi067 ceph-mon[33511]: 6.1e scrub starts 2024-09-20T22:16:36.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:35 smithi067 ceph-mon[33511]: 6.1e scrub ok 2024-09-20T22:16:36.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:35 smithi067 ceph-mon[33511]: 5.1f scrub starts 2024-09-20T22:16:36.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:35 smithi067 ceph-mon[33511]: 5.1f scrub ok 2024-09-20T22:16:36.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:35 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:16:36.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:35 smithi067 ceph-mon[33511]: osdmap e407: 8 total, 8 up, 8 in 2024-09-20T22:16:36.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:35 smithi060 ceph-mon[29694]: 10.e scrub ok 2024-09-20T22:16:36.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:35 smithi060 ceph-mon[29694]: 6.1e scrub starts 2024-09-20T22:16:36.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:35 smithi060 ceph-mon[29694]: 6.1e scrub ok 2024-09-20T22:16:36.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:35 smithi060 ceph-mon[29694]: 5.1f scrub starts 2024-09-20T22:16:36.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:35 smithi060 ceph-mon[29694]: 5.1f scrub ok 2024-09-20T22:16:36.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:35 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:16:36.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:35 smithi060 ceph-mon[29694]: osdmap e407: 8 total, 8 up, 8 in 2024-09-20T22:16:36.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:35 smithi060 ceph-mon[40290]: 10.e scrub ok 2024-09-20T22:16:36.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:35 smithi060 ceph-mon[40290]: 6.1e scrub starts 2024-09-20T22:16:36.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:35 smithi060 ceph-mon[40290]: 6.1e scrub ok 2024-09-20T22:16:36.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:35 smithi060 ceph-mon[40290]: 5.1f scrub starts 2024-09-20T22:16:36.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:35 smithi060 ceph-mon[40290]: 5.1f scrub ok 2024-09-20T22:16:36.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:35 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:16:36.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:35 smithi060 ceph-mon[40290]: osdmap e407: 8 total, 8 up, 8 in 2024-09-20T22:16:36.290 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:36.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:36.291 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:36.320 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:36.348 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-20T22:16:36.348 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:16:36.392 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:36.428 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:36.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:36.429 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:36.530 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:36.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:36.569 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:36.569 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:36.670 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:36.704 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:36.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:36.705 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:36.806 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:36.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:36.849 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:36.849 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:36.932 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 79 16 112301 112285 88.8133 70.375 0.00827967 0.0112549 2024-09-20T22:16:36.950 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:36.993 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:36.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:36.994 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:37.095 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:37.119 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:16:37.119 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:16:37.119 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:16:37.119 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:16:37.119 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:16:37.141 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:37.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:37.143 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:37.212 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[29694]: 6.8 scrub starts 2024-09-20T22:16:37.213 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[29694]: 6.8 scrub ok 2024-09-20T22:16:37.213 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:16:37.213 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:16:37.213 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[29694]: 3.6 scrub starts 2024-09-20T22:16:37.213 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[29694]: 3.6 scrub ok 2024-09-20T22:16:37.213 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[29694]: 10.d scrub ok 2024-09-20T22:16:37.213 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[29694]: pgmap v735: 168 pgs: 2 active+clean+scrubbing, 166 active+clean; 6.6 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-20T22:16:37.213 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[29694]: osdmap e408: 8 total, 8 up, 8 in 2024-09-20T22:16:37.213 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[29694]: 10.0 scrub starts 2024-09-20T22:16:37.213 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[40290]: 6.8 scrub starts 2024-09-20T22:16:37.213 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[40290]: 6.8 scrub ok 2024-09-20T22:16:37.213 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:16:37.214 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:16:37.214 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[40290]: 3.6 scrub starts 2024-09-20T22:16:37.214 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[40290]: 3.6 scrub ok 2024-09-20T22:16:37.214 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[40290]: 10.d scrub ok 2024-09-20T22:16:37.214 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[40290]: pgmap v735: 168 pgs: 2 active+clean+scrubbing, 166 active+clean; 6.6 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-20T22:16:37.214 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[40290]: osdmap e408: 8 total, 8 up, 8 in 2024-09-20T22:16:37.214 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:36 smithi060 ceph-mon[40290]: 10.0 scrub starts 2024-09-20T22:16:37.243 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:37.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:36 smithi067 ceph-mon[33511]: 6.8 scrub starts 2024-09-20T22:16:37.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:36 smithi067 ceph-mon[33511]: 6.8 scrub ok 2024-09-20T22:16:37.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:36 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:16:37.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:36 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:16:37.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:36 smithi067 ceph-mon[33511]: 3.6 scrub starts 2024-09-20T22:16:37.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:36 smithi067 ceph-mon[33511]: 3.6 scrub ok 2024-09-20T22:16:37.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:36 smithi067 ceph-mon[33511]: 10.d scrub ok 2024-09-20T22:16:37.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:36 smithi067 ceph-mon[33511]: pgmap v735: 168 pgs: 2 active+clean+scrubbing, 166 active+clean; 6.6 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1023 B/s rd, 65 MiB/s wr, 1.04k op/s 2024-09-20T22:16:37.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:36 smithi067 ceph-mon[33511]: osdmap e408: 8 total, 8 up, 8 in 2024-09-20T22:16:37.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:36 smithi067 ceph-mon[33511]: 10.0 scrub starts 2024-09-20T22:16:37.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:37.286 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:37.286 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:37.386 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:37.425 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:37.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:37.426 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:37.527 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:37.572 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:37.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:37.573 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:37.632 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:37.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:37.721 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:37.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:37.722 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:37.823 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:37.858 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:37.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:37.859 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:37.877 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:16:37.921 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:16:37.932 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:16:37.932258+0000 min lat: 0.00137086 max lat: 0.320133 avg lat: 0.0112869 2024-09-20T22:16:37.932 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:16:37.932 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 80 16 113396 113380 88.5586 68.4375 0.0176365 0.0112869 2024-09-20T22:16:37.960 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:38.010 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:38.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:38.011 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:38.112 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:38.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:38.146 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:38.146 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:38.247 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:38.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:37 smithi067 ceph-mon[33511]: 5.18 scrub starts 2024-09-20T22:16:38.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:37 smithi067 ceph-mon[33511]: 5.18 scrub ok 2024-09-20T22:16:38.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:37 smithi067 ceph-mon[33511]: 3.2 deep-scrub starts 2024-09-20T22:16:38.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:37 smithi067 ceph-mon[33511]: 3.2 deep-scrub ok 2024-09-20T22:16:38.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:37 smithi067 ceph-mon[33511]: 4.14 scrub starts 2024-09-20T22:16:38.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:37 smithi067 ceph-mon[33511]: 4.14 scrub ok 2024-09-20T22:16:38.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:37 smithi067 ceph-mon[33511]: 5.11 scrub starts 2024-09-20T22:16:38.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:37 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2445085236' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:16:38.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:37 smithi067 ceph-mon[33511]: 5.11 scrub ok 2024-09-20T22:16:38.268 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:38.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:38.269 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:38.277 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:37 smithi060 ceph-mon[29694]: 5.18 scrub starts 2024-09-20T22:16:38.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:37 smithi060 ceph-mon[29694]: 5.18 scrub ok 2024-09-20T22:16:38.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:37 smithi060 ceph-mon[29694]: 3.2 deep-scrub starts 2024-09-20T22:16:38.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:37 smithi060 ceph-mon[29694]: 3.2 deep-scrub ok 2024-09-20T22:16:38.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:37 smithi060 ceph-mon[29694]: 4.14 scrub starts 2024-09-20T22:16:38.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:37 smithi060 ceph-mon[29694]: 4.14 scrub ok 2024-09-20T22:16:38.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:37 smithi060 ceph-mon[29694]: 5.11 scrub starts 2024-09-20T22:16:38.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:37 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2445085236' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:16:38.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:37 smithi060 ceph-mon[29694]: 5.11 scrub ok 2024-09-20T22:16:38.278 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:37 smithi060 ceph-mon[40290]: 5.18 scrub starts 2024-09-20T22:16:38.278 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:37 smithi060 ceph-mon[40290]: 5.18 scrub ok 2024-09-20T22:16:38.278 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:37 smithi060 ceph-mon[40290]: 3.2 deep-scrub starts 2024-09-20T22:16:38.279 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:37 smithi060 ceph-mon[40290]: 3.2 deep-scrub ok 2024-09-20T22:16:38.279 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:37 smithi060 ceph-mon[40290]: 4.14 scrub starts 2024-09-20T22:16:38.279 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:37 smithi060 ceph-mon[40290]: 4.14 scrub ok 2024-09-20T22:16:38.279 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:37 smithi060 ceph-mon[40290]: 5.11 scrub starts 2024-09-20T22:16:38.279 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:37 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2445085236' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:16:38.279 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:37 smithi060 ceph-mon[40290]: 5.11 scrub ok 2024-09-20T22:16:38.305 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T22:16:38.370 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:38.443 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:38.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:38.444 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:38.545 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:38.576 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:38.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:38.577 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:38.677 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:38.705 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:38.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:38.706 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:38.807 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:38.845 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:38.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:38.846 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:38.932 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 81 16 114375 114359 88.2206 61.1875 0.0100992 0.0113296 2024-09-20T22:16:38.947 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:38.977 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:38.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:38.978 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:39.079 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:39.118 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:39.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:39.119 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:39.220 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:39.257 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:39.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:39.258 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:39.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:38 smithi067 ceph-mon[33511]: 3.f scrub starts 2024-09-20T22:16:39.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:38 smithi067 ceph-mon[33511]: 3.f scrub ok 2024-09-20T22:16:39.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:38 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:16:39.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:38 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2445085236' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:16:39.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:38 smithi067 ceph-mon[33511]: osdmap e409: 8 total, 8 up, 8 in 2024-09-20T22:16:39.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:38 smithi067 ceph-mon[33511]: 6.18 deep-scrub starts 2024-09-20T22:16:39.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:38 smithi067 ceph-mon[33511]: 6.18 deep-scrub ok 2024-09-20T22:16:39.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:38 smithi067 ceph-mon[33511]: pgmap v738: 168 pgs: 168 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 852 B/s rd, 73 MiB/s wr, 1.18k op/s 2024-09-20T22:16:39.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:38 smithi060 ceph-mon[29694]: 3.f scrub starts 2024-09-20T22:16:39.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:38 smithi060 ceph-mon[29694]: 3.f scrub ok 2024-09-20T22:16:39.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:38 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:16:39.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:38 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2445085236' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:16:39.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:38 smithi060 ceph-mon[29694]: osdmap e409: 8 total, 8 up, 8 in 2024-09-20T22:16:39.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:38 smithi060 ceph-mon[29694]: 6.18 deep-scrub starts 2024-09-20T22:16:39.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:38 smithi060 ceph-mon[29694]: 6.18 deep-scrub ok 2024-09-20T22:16:39.278 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:38 smithi060 ceph-mon[29694]: pgmap v738: 168 pgs: 168 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 852 B/s rd, 73 MiB/s wr, 1.18k op/s 2024-09-20T22:16:39.279 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:38 smithi060 ceph-mon[40290]: 3.f scrub starts 2024-09-20T22:16:39.279 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:38 smithi060 ceph-mon[40290]: 3.f scrub ok 2024-09-20T22:16:39.279 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:38 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:16:39.279 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:38 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2445085236' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:16:39.279 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:38 smithi060 ceph-mon[40290]: osdmap e409: 8 total, 8 up, 8 in 2024-09-20T22:16:39.279 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:38 smithi060 ceph-mon[40290]: 6.18 deep-scrub starts 2024-09-20T22:16:39.279 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:38 smithi060 ceph-mon[40290]: 6.18 deep-scrub ok 2024-09-20T22:16:39.279 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:38 smithi060 ceph-mon[40290]: pgmap v738: 168 pgs: 168 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 852 B/s rd, 73 MiB/s wr, 1.18k op/s 2024-09-20T22:16:39.359 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:39.391 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:39.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:39.392 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:39.480 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:39.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:39.524 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:39.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:39.525 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:39.560 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:39.625 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:39.679 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:39.679 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:39.680 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:39.780 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:39.815 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:39.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:39.816 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:39.835 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:16:39.840 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:16:39.917 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:39.932 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 82 16 115496 115480 87.9992 70.0625 0.0198381 0.0113582 2024-09-20T22:16:39.946 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:39.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:39.947 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:40.048 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:40.085 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:40.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:40.087 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:40.187 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:40.195 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:39 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:16:40.195 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:39 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:16:40.196 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:39 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:16:40.196 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:39 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:16:40.233 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:40.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:40.234 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:40.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:39 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:16:40.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:39 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:16:40.334 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:40.360 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:40.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:40.361 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:40.461 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:40.500 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:40.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:40.501 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:40.602 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:40.636 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:40.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:40.637 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:40.737 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:40.754 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:16:40.755 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:16:40.755 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:16:40.755 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:16:40.755 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:16:40.755 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:16:40.755 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:16:40.755 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:16:40.756 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:16:40.756 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:16:40.756 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:16:40.756 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:16:40.756 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:16:40.756 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:16:40.756 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:16:40.756 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:16:40.758 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:40.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:40.759 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:40.860 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:40.893 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:40.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:40.894 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:40.932 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 83 16 116622 116606 87.7868 70.375 0.0199187 0.0113866 2024-09-20T22:16:40.995 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:41.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:40 smithi060 ceph-mon[29694]: from='client.26006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:16:41.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:40 smithi060 ceph-mon[29694]: osdmap e410: 8 total, 8 up, 8 in 2024-09-20T22:16:41.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:40 smithi060 ceph-mon[29694]: pgmap v740: 168 pgs: 168 active+clean; 6.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 945 B/s rd, 62 MiB/s wr, 990 op/s 2024-09-20T22:16:41.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:40 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1291066548' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:41.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:40 smithi060 ceph-mon[40290]: from='client.26006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:16:41.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:40 smithi060 ceph-mon[40290]: osdmap e410: 8 total, 8 up, 8 in 2024-09-20T22:16:41.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:40 smithi060 ceph-mon[40290]: pgmap v740: 168 pgs: 168 active+clean; 6.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 945 B/s rd, 62 MiB/s wr, 990 op/s 2024-09-20T22:16:41.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:40 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1291066548' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:41.039 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:41.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:41.040 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:41.090 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:16:41.140 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:41.190 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:41.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:41.191 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:41.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:40 smithi067 ceph-mon[33511]: from='client.26006 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:16:41.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:40 smithi067 ceph-mon[33511]: osdmap e410: 8 total, 8 up, 8 in 2024-09-20T22:16:41.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:40 smithi067 ceph-mon[33511]: pgmap v740: 168 pgs: 168 active+clean; 6.9 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 945 B/s rd, 62 MiB/s wr, 990 op/s 2024-09-20T22:16:41.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:40 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1291066548' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:41.266 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:16:41.266 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:16:41.293 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:41.336 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:41.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:41.337 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:41.437 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:41.461 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:41.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:41.462 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:41.562 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:41.611 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:41.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:41.612 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:41.713 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:41.749 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:41.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:41.749 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:41.850 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:41.882 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:41.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:41.883 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:41.932 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 84 16 117781 117765 87.604 72.4375 0.0114619 0.0114105 2024-09-20T22:16:41.984 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:42.014 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:42.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:42.015 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:42.116 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:42.153 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:42.154 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:42.154 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:42.255 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:42.294 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:42.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:42.295 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:42.396 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:42.431 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:42.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:42.432 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:42.532 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:42.578 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:42.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:42.579 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:42.679 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:42.714 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:42.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:42.715 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:42.717 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":738,"stamp":"2024-09-20T22:16:38.254365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7350917659,"num_objects":112356,"num_object_clones":0,"num_object_copies":337068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112356,"num_whiteouts":0,"num_read":1935,"num_read_kb":1687,"num_write":112879,"num_write_kb":7178093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":268563,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82591,"ondisk_log_size":82591,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":504,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":26273904,"kb_used_data":21411156,"kb_used_omap":15,"kb_used_meta":4862512,"kb_avail":723687312,"statfs":{"total":767960285184,"available":741055807488,"internally_reserved":0,"allocated":21925023744,"data_stored":21921307965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":4979213075},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,6,11,4,1,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":462422016,"num_objects":7056,"num_object_clones":0,"num_object_copies":21168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7056,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":7056,"num_write_kb":451584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002364"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":196,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.588629+0000","last_change":"2024-09-20T22:16:31.993232+0000","last_active":"2024-09-20T22:16:36.588629+0000","last_peered":"2024-09-20T22:16:36.588629+0000","last_clean":"2024-09-20T22:16:36.588629+0000","last_became_active":"2024-09-20T22:12:52.515921+0000","last_became_peered":"2024-09-20T22:12:52.515921+0000","last_unstale":"2024-09-20T22:16:36.588629+0000","last_undegraded":"2024-09-20T22:16:36.588629+0000","last_fullsized":"2024-09-20T22:16:36.588629+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:31.993104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:31.993104+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.993104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:36.968800+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"327'19","reported_seq":784,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.840942+0000","last_change":"2024-09-20T22:16:29.508726+0000","last_active":"2024-09-20T22:16:34.840942+0000","last_peered":"2024-09-20T22:16:34.840942+0000","last_clean":"2024-09-20T22:16:34.840942+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:16:34.840942+0000","last_undegraded":"2024-09-20T22:16:34.840942+0000","last_fullsized":"2024-09-20T22:16:34.840942+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"327'19","last_scrub_stamp":"2024-09-20T22:16:29.508607+0000","last_deep_scrub":"252'13","last_deep_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_clean_scrub_stamp":"2024-09-20T22:16:29.508607+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:49.145180+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":620,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:25.774242+0000","last_change":"2024-09-20T22:14:15.391149+0000","last_active":"2024-09-20T22:16:25.774242+0000","last_peered":"2024-09-20T22:16:25.774242+0000","last_clean":"2024-09-20T22:16:25.774242+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:16:25.774242+0000","last_undegraded":"2024-09-20T22:16:25.774242+0000","last_fullsized":"2024-09-20T22:16:25.774242+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:15.391049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:15.391049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":751,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.841297+0000","last_change":"2024-09-20T22:16:23.438091+0000","last_active":"2024-09-20T22:16:34.841297+0000","last_peered":"2024-09-20T22:16:34.841297+0000","last_clean":"2024-09-20T22:16:34.841297+0000","last_became_active":"2024-09-20T22:11:30.485735+0000","last_became_peered":"2024-09-20T22:11:30.485735+0000","last_unstale":"2024-09-20T22:16:34.841297+0000","last_undegraded":"2024-09-20T22:16:34.841297+0000","last_fullsized":"2024-09-20T22:16:34.841297+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:23.437959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:23.437959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:36.548769+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":740,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.609155+0000","last_change":"2024-09-20T22:16:27.694458+0000","last_active":"2024-09-20T22:16:36.609155+0000","last_peered":"2024-09-20T22:16:36.609155+0000","last_clean":"2024-09-20T22:16:36.609155+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:16:36.609155+0000","last_undegraded":"2024-09-20T22:16:36.609155+0000","last_fullsized":"2024-09-20T22:16:36.609155+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:16:27.694309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:27.694309+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:50.010230+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":226,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:37.937651+0000","last_change":"2024-09-20T22:16:22.789638+0000","last_active":"2024-09-20T22:16:37.937651+0000","last_peered":"2024-09-20T22:16:37.937651+0000","last_clean":"2024-09-20T22:16:37.937651+0000","last_became_active":"2024-09-20T22:12:33.545100+0000","last_became_peered":"2024-09-20T22:12:33.545100+0000","last_unstale":"2024-09-20T22:16:37.937651+0000","last_undegraded":"2024-09-20T22:16:37.937651+0000","last_fullsized":"2024-09-20T22:16:37.937651+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:22.789569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:22.789569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:52.149562+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"406'34","reported_seq":344,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.846812+0000","last_change":"2024-09-20T22:16:31.033633+0000","last_active":"2024-09-20T22:16:34.846812+0000","last_peered":"2024-09-20T22:16:34.846812+0000","last_clean":"2024-09-20T22:16:34.846812+0000","last_became_active":"2024-09-20T22:12:52.517296+0000","last_became_peered":"2024-09-20T22:12:52.517296+0000","last_unstale":"2024-09-20T22:16:34.846812+0000","last_undegraded":"2024-09-20T22:16:34.846812+0000","last_fullsized":"2024-09-20T22:16:34.846812+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'34","last_scrub_stamp":"2024-09-20T22:16:31.033354+0000","last_deep_scrub":"406'34","last_deep_scrub_stamp":"2024-09-20T22:16:31.033354+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.033354+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:54.712855+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":735,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.840913+0000","last_change":"2024-09-20T22:15:26.080144+0000","last_active":"2024-09-20T22:16:34.840913+0000","last_peered":"2024-09-20T22:16:34.840913+0000","last_clean":"2024-09-20T22:16:34.840913+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:16:34.840913+0000","last_undegraded":"2024-09-20T22:16:34.840913+0000","last_fullsized":"2024-09-20T22:16:34.840913+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:26.080040+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:26.080040+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":757,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.846360+0000","last_change":"2024-09-20T22:15:21.178920+0000","last_active":"2024-09-20T22:16:34.846360+0000","last_peered":"2024-09-20T22:16:34.846360+0000","last_clean":"2024-09-20T22:16:34.846360+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:16:34.846360+0000","last_undegraded":"2024-09-20T22:16:34.846360+0000","last_fullsized":"2024-09-20T22:16:34.846360+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:21.178806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:15:21.178806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:36.671122+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"328'15","reported_seq":803,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.841550+0000","last_change":"2024-09-20T22:15:59.331710+0000","last_active":"2024-09-20T22:16:34.841550+0000","last_peered":"2024-09-20T22:16:34.841550+0000","last_clean":"2024-09-20T22:16:34.841550+0000","last_became_active":"2024-09-20T22:12:52.516894+0000","last_became_peered":"2024-09-20T22:12:52.516894+0000","last_unstale":"2024-09-20T22:16:34.841550+0000","last_undegraded":"2024-09-20T22:16:34.841550+0000","last_fullsized":"2024-09-20T22:16:34.841550+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:15:59.331593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:59.331593+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:13.858540+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":289,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.846320+0000","last_change":"2024-09-20T22:14:54.153118+0000","last_active":"2024-09-20T22:16:34.846320+0000","last_peered":"2024-09-20T22:16:34.846320+0000","last_clean":"2024-09-20T22:16:34.846320+0000","last_became_active":"2024-09-20T22:12:52.518549+0000","last_became_peered":"2024-09-20T22:12:52.518549+0000","last_unstale":"2024-09-20T22:16:34.846320+0000","last_undegraded":"2024-09-20T22:16:34.846320+0000","last_fullsized":"2024-09-20T22:16:34.846320+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:54.152717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:54.152717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":196,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.590220+0000","last_change":"2024-09-20T22:14:58.570997+0000","last_active":"2024-09-20T22:16:36.590220+0000","last_peered":"2024-09-20T22:16:36.590220+0000","last_clean":"2024-09-20T22:16:36.590220+0000","last_became_active":"2024-09-20T22:12:52.524631+0000","last_became_peered":"2024-09-20T22:12:52.524631+0000","last_unstale":"2024-09-20T22:16:36.590220+0000","last_undegraded":"2024-09-20T22:16:36.590220+0000","last_fullsized":"2024-09-20T22:16:36.590220+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:58.570923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:58.570923+0000","last_clean_scrub_stamp":"2024-09-20T22:14:58.570923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":762,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.845837+0000","last_change":"2024-09-20T22:15:20.182058+0000","last_active":"2024-09-20T22:16:34.845837+0000","last_peered":"2024-09-20T22:16:34.845837+0000","last_clean":"2024-09-20T22:16:34.845837+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:16:34.845837+0000","last_undegraded":"2024-09-20T22:16:34.845837+0000","last_fullsized":"2024-09-20T22:16:34.845837+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.181942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:30.215263+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.181942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:46.935103+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"328'7","reported_seq":772,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.841008+0000","last_change":"2024-09-20T22:15:31.119172+0000","last_active":"2024-09-20T22:16:34.841008+0000","last_peered":"2024-09-20T22:16:34.841008+0000","last_clean":"2024-09-20T22:16:34.841008+0000","last_became_active":"2024-09-20T22:08:22.144364+0000","last_became_peered":"2024-09-20T22:08:22.144364+0000","last_unstale":"2024-09-20T22:16:34.841008+0000","last_undegraded":"2024-09-20T22:16:34.841008+0000","last_fullsized":"2024-09-20T22:16:34.841008+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'7","last_scrub_stamp":"2024-09-20T22:15:31.119046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:31.119046+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.446818+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":743,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842137+0000","last_change":"2024-09-20T22:14:57.354434+0000","last_active":"2024-09-20T22:16:34.842137+0000","last_peered":"2024-09-20T22:16:34.842137+0000","last_clean":"2024-09-20T22:16:34.842137+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:16:34.842137+0000","last_undegraded":"2024-09-20T22:16:34.842137+0000","last_fullsized":"2024-09-20T22:16:34.842137+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:57.354326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:57.354326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":277,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.846819+0000","last_change":"2024-09-20T22:15:00.077644+0000","last_active":"2024-09-20T22:16:34.846819+0000","last_peered":"2024-09-20T22:16:34.846819+0000","last_clean":"2024-09-20T22:16:34.846819+0000","last_became_active":"2024-09-20T22:11:30.487137+0000","last_became_peered":"2024-09-20T22:11:30.487137+0000","last_unstale":"2024-09-20T22:16:34.846819+0000","last_undegraded":"2024-09-20T22:16:34.846819+0000","last_fullsized":"2024-09-20T22:16:34.846819+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:00.077519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:15:00.077519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":739,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.840774+0000","last_change":"2024-09-20T22:16:34.472967+0000","last_active":"2024-09-20T22:16:34.840774+0000","last_peered":"2024-09-20T22:16:34.840774+0000","last_clean":"2024-09-20T22:16:34.840774+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:16:34.840774+0000","last_undegraded":"2024-09-20T22:16:34.840774+0000","last_fullsized":"2024-09-20T22:16:34.840774+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:34.472700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:16:34.472700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:51.839167+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":196,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.587944+0000","last_change":"2024-09-20T22:15:24.748006+0000","last_active":"2024-09-20T22:16:36.587944+0000","last_peered":"2024-09-20T22:16:36.587944+0000","last_clean":"2024-09-20T22:16:36.587944+0000","last_became_active":"2024-09-20T22:12:52.526097+0000","last_became_peered":"2024-09-20T22:12:52.526097+0000","last_unstale":"2024-09-20T22:16:36.587944+0000","last_undegraded":"2024-09-20T22:16:36.587944+0000","last_fullsized":"2024-09-20T22:16:36.587944+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.747891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.747891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"328'34","reported_seq":341,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842004+0000","last_change":"2024-09-20T22:15:55.987603+0000","last_active":"2024-09-20T22:16:34.842004+0000","last_peered":"2024-09-20T22:16:34.842004+0000","last_clean":"2024-09-20T22:16:34.842004+0000","last_became_active":"2024-09-20T22:11:12.507311+0000","last_became_peered":"2024-09-20T22:11:12.507311+0000","last_unstale":"2024-09-20T22:16:34.842004+0000","last_undegraded":"2024-09-20T22:16:34.842004+0000","last_fullsized":"2024-09-20T22:16:34.842004+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'34","last_scrub_stamp":"2024-09-20T22:15:55.987386+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:15:55.987386+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:17.614237+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":589,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:25.773429+0000","last_change":"2024-09-20T22:16:23.258053+0000","last_active":"2024-09-20T22:16:25.773429+0000","last_peered":"2024-09-20T22:16:25.773429+0000","last_clean":"2024-09-20T22:16:25.773429+0000","last_became_active":"2024-09-20T22:12:52.520985+0000","last_became_peered":"2024-09-20T22:12:52.520985+0000","last_unstale":"2024-09-20T22:16:25.773429+0000","last_undegraded":"2024-09-20T22:16:25.773429+0000","last_fullsized":"2024-09-20T22:16:25.773429+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:23.257937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:11.053276+0000","last_clean_scrub_stamp":"2024-09-20T22:16:23.257937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:33.735295+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":695,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:37.937816+0000","last_change":"2024-09-20T22:16:16.771843+0000","last_active":"2024-09-20T22:16:37.937816+0000","last_peered":"2024-09-20T22:16:37.937816+0000","last_clean":"2024-09-20T22:16:37.937816+0000","last_became_active":"2024-09-20T22:12:21.521433+0000","last_became_peered":"2024-09-20T22:12:21.521433+0000","last_unstale":"2024-09-20T22:16:37.937816+0000","last_undegraded":"2024-09-20T22:16:37.937816+0000","last_fullsized":"2024-09-20T22:16:37.937816+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:16.771744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:16.771744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:43.671506+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":691,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.845759+0000","last_change":"2024-09-20T22:15:58.078205+0000","last_active":"2024-09-20T22:16:34.845759+0000","last_peered":"2024-09-20T22:16:34.845759+0000","last_clean":"2024-09-20T22:16:34.845759+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:16:34.845759+0000","last_undegraded":"2024-09-20T22:16:34.845759+0000","last_fullsized":"2024-09-20T22:16:34.845759+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:58.078099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:58.078099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:15.885682+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"328'23","reported_seq":654,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:25.774852+0000","last_change":"2024-09-20T22:15:17.533607+0000","last_active":"2024-09-20T22:16:25.774852+0000","last_peered":"2024-09-20T22:16:25.774852+0000","last_clean":"2024-09-20T22:16:25.774852+0000","last_became_active":"2024-09-20T22:12:21.517147+0000","last_became_peered":"2024-09-20T22:12:21.517147+0000","last_unstale":"2024-09-20T22:16:25.774852+0000","last_undegraded":"2024-09-20T22:16:25.774852+0000","last_fullsized":"2024-09-20T22:16:25.774852+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'23","last_scrub_stamp":"2024-09-20T22:15:17.533453+0000","last_deep_scrub":"250'17","last_deep_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_clean_scrub_stamp":"2024-09-20T22:15:17.533453+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:38.348665+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":734,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.843085+0000","last_change":"2024-09-20T22:15:33.188818+0000","last_active":"2024-09-20T22:16:34.843085+0000","last_peered":"2024-09-20T22:16:34.843085+0000","last_clean":"2024-09-20T22:16:34.843085+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:16:34.843085+0000","last_undegraded":"2024-09-20T22:16:34.843085+0000","last_fullsized":"2024-09-20T22:16:34.843085+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:33.188747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:33.188747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:40.715215+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":732,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842892+0000","last_change":"2024-09-20T22:15:57.632871+0000","last_active":"2024-09-20T22:16:34.842892+0000","last_peered":"2024-09-20T22:16:34.842892+0000","last_clean":"2024-09-20T22:16:34.842892+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:16:34.842892+0000","last_undegraded":"2024-09-20T22:16:34.842892+0000","last_fullsized":"2024-09-20T22:16:34.842892+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:15:57.632600+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.632600+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:14.194144+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"289'2","reported_seq":580,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:25.774257+0000","last_change":"2024-09-20T22:14:22.436454+0000","last_active":"2024-09-20T22:16:25.774257+0000","last_peered":"2024-09-20T22:16:25.774257+0000","last_clean":"2024-09-20T22:16:25.774257+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:16:25.774257+0000","last_undegraded":"2024-09-20T22:16:25.774257+0000","last_fullsized":"2024-09-20T22:16:25.774257+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-20T22:14:22.436308+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_clean_scrub_stamp":"2024-09-20T22:14:22.436308+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"328'20","reported_seq":223,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.588282+0000","last_change":"2024-09-20T22:15:57.132570+0000","last_active":"2024-09-20T22:16:36.588282+0000","last_peered":"2024-09-20T22:16:36.588282+0000","last_clean":"2024-09-20T22:16:36.588282+0000","last_became_active":"2024-09-20T22:12:52.526803+0000","last_became_peered":"2024-09-20T22:12:52.526803+0000","last_unstale":"2024-09-20T22:16:36.588282+0000","last_undegraded":"2024-09-20T22:16:36.588282+0000","last_fullsized":"2024-09-20T22:16:36.588282+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:15:57.132435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.132435+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:18.800340+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"291'11","reported_seq":960,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:37.938069+0000","last_change":"2024-09-20T22:16:34.733980+0000","last_active":"2024-09-20T22:16:37.938069+0000","last_peered":"2024-09-20T22:16:37.938069+0000","last_clean":"2024-09-20T22:16:37.938069+0000","last_became_active":"2024-09-20T22:11:40.601048+0000","last_became_peered":"2024-09-20T22:11:40.601048+0000","last_unstale":"2024-09-20T22:16:37.938069+0000","last_undegraded":"2024-09-20T22:16:37.938069+0000","last_fullsized":"2024-09-20T22:16:37.938069+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'11","last_scrub_stamp":"2024-09-20T22:16:34.733870+0000","last_deep_scrub":"291'11","last_deep_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_clean_scrub_stamp":"2024-09-20T22:16:34.733870+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:58.907961+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"407'6903","reported_seq":7055,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:35.871243+0000","last_change":"2024-09-20T22:16:24.149762+0000","last_active":"2024-09-20T22:16:35.871243+0000","last_peered":"2024-09-20T22:16:35.871243+0000","last_clean":"2024-09-20T22:16:35.871243+0000","last_became_active":"2024-09-20T22:15:08.927535+0000","last_became_peered":"2024-09-20T22:15:08.927535+0000","last_unstale":"2024-09-20T22:16:35.871243+0000","last_undegraded":"2024-09-20T22:16:35.871243+0000","last_fullsized":"2024-09-20T22:16:35.871243+0000","mapping_epoch":370,"log_start":"380'2700","ondisk_log_start":"380'2700","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"405'6096","last_scrub_stamp":"2024-09-20T22:16:24.146029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:24.146029+0000","objects_scrubbed":6025,"log_size":4203,"log_dups_size":0,"ondisk_log_size":4203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.643132+0000","scrub_duration":1743,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":452395008,"num_objects":6903,"num_object_clones":0,"num_object_copies":20709,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6903,"num_write_kb":441792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":741,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.845513+0000","last_change":"2024-09-20T22:13:08.407055+0000","last_active":"2024-09-20T22:16:34.845513+0000","last_peered":"2024-09-20T22:16:34.845513+0000","last_clean":"2024-09-20T22:16:34.845513+0000","last_became_active":"2024-09-20T22:08:48.452786+0000","last_became_peered":"2024-09-20T22:08:48.452786+0000","last_unstale":"2024-09-20T22:16:34.845513+0000","last_undegraded":"2024-09-20T22:16:34.845513+0000","last_fullsized":"2024-09-20T22:16:34.845513+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:08.406883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:13:08.406883+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"328'27","reported_seq":534,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842244+0000","last_change":"2024-09-20T22:15:50.925776+0000","last_active":"2024-09-20T22:16:34.842244+0000","last_peered":"2024-09-20T22:16:34.842244+0000","last_clean":"2024-09-20T22:16:34.842244+0000","last_became_active":"2024-09-20T22:08:22.145408+0000","last_became_peered":"2024-09-20T22:08:22.145408+0000","last_unstale":"2024-09-20T22:16:34.842244+0000","last_undegraded":"2024-09-20T22:16:34.842244+0000","last_fullsized":"2024-09-20T22:16:34.842244+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:15:50.925654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:50.925654+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:54.745209+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":761,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.606655+0000","last_change":"2024-09-20T22:16:31.637874+0000","last_active":"2024-09-20T22:16:36.606655+0000","last_peered":"2024-09-20T22:16:36.606655+0000","last_clean":"2024-09-20T22:16:36.606655+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:16:36.606655+0000","last_undegraded":"2024-09-20T22:16:36.606655+0000","last_fullsized":"2024-09-20T22:16:36.606655+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:31.637766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:28.020341+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.637766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:44.980512+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"326'13","reported_seq":978,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.840922+0000","last_change":"2024-09-20T22:16:15.367687+0000","last_active":"2024-09-20T22:16:34.840922+0000","last_peered":"2024-09-20T22:16:34.840922+0000","last_clean":"2024-09-20T22:16:34.840922+0000","last_became_active":"2024-09-20T22:12:52.517588+0000","last_became_peered":"2024-09-20T22:12:52.517588+0000","last_unstale":"2024-09-20T22:16:34.840922+0000","last_undegraded":"2024-09-20T22:16:34.840922+0000","last_fullsized":"2024-09-20T22:16:34.840922+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'13","last_scrub_stamp":"2024-09-20T22:16:15.367449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:15.367449+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:32.822489+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":289,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.846040+0000","last_change":"2024-09-20T22:15:33.082489+0000","last_active":"2024-09-20T22:16:34.846040+0000","last_peered":"2024-09-20T22:16:34.846040+0000","last_clean":"2024-09-20T22:16:34.846040+0000","last_became_active":"2024-09-20T22:12:52.517826+0000","last_became_peered":"2024-09-20T22:12:52.517826+0000","last_unstale":"2024-09-20T22:16:34.846040+0000","last_undegraded":"2024-09-20T22:16:34.846040+0000","last_fullsized":"2024-09-20T22:16:34.846040+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:33.082368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:33.082368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:58.107873+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"407'6804","reported_seq":6962,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:35.867271+0000","last_change":"2024-09-20T22:16:34.386928+0000","last_active":"2024-09-20T22:16:35.867271+0000","last_peered":"2024-09-20T22:16:35.867271+0000","last_clean":"2024-09-20T22:16:35.867271+0000","last_became_active":"2024-09-20T22:15:08.928064+0000","last_became_peered":"2024-09-20T22:15:08.928064+0000","last_unstale":"2024-09-20T22:16:35.867271+0000","last_undegraded":"2024-09-20T22:16:35.867271+0000","last_fullsized":"2024-09-20T22:16:35.867271+0000","mapping_epoch":370,"log_start":"380'2600","ondisk_log_start":"380'2600","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6697","last_scrub_stamp":"2024-09-20T22:16:34.386672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:34.386672+0000","objects_scrubbed":6630,"log_size":4204,"log_dups_size":0,"ondisk_log_size":4204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:49.633430+0000","scrub_duration":1869,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":445906944,"num_objects":6804,"num_object_clones":0,"num_object_copies":20412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6804,"num_write_kb":435456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":754,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.841680+0000","last_change":"2024-09-20T22:15:46.295600+0000","last_active":"2024-09-20T22:16:34.841680+0000","last_peered":"2024-09-20T22:16:34.841680+0000","last_clean":"2024-09-20T22:16:34.841680+0000","last_became_active":"2024-09-20T22:15:45.627770+0000","last_became_peered":"2024-09-20T22:15:45.627770+0000","last_unstale":"2024-09-20T22:16:34.841680+0000","last_undegraded":"2024-09-20T22:16:34.841680+0000","last_fullsized":"2024-09-20T22:16:34.841680+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:46.295352+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:15:46.295352+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:11.775763+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"328'30","reported_seq":755,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842396+0000","last_change":"2024-09-20T22:16:16.609501+0000","last_active":"2024-09-20T22:16:34.842396+0000","last_peered":"2024-09-20T22:16:34.842396+0000","last_clean":"2024-09-20T22:16:34.842396+0000","last_became_active":"2024-09-20T22:12:52.526867+0000","last_became_peered":"2024-09-20T22:12:52.526867+0000","last_unstale":"2024-09-20T22:16:34.842396+0000","last_undegraded":"2024-09-20T22:16:34.842396+0000","last_fullsized":"2024-09-20T22:16:34.842396+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'30","last_scrub_stamp":"2024-09-20T22:16:16.609377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:16.609377+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:45.896136+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":235,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.843184+0000","last_change":"2024-09-20T22:16:28.701889+0000","last_active":"2024-09-20T22:16:34.843184+0000","last_peered":"2024-09-20T22:16:34.843184+0000","last_clean":"2024-09-20T22:16:34.843184+0000","last_became_active":"2024-09-20T22:12:52.528947+0000","last_became_peered":"2024-09-20T22:12:52.528947+0000","last_unstale":"2024-09-20T22:16:34.843184+0000","last_undegraded":"2024-09-20T22:16:34.843184+0000","last_fullsized":"2024-09-20T22:16:34.843184+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:28.701780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:28.701780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:46.729610+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":755,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.840324+0000","last_change":"2024-09-20T22:15:56.322206+0000","last_active":"2024-09-20T22:16:34.840324+0000","last_peered":"2024-09-20T22:16:34.840324+0000","last_clean":"2024-09-20T22:16:34.840324+0000","last_became_active":"2024-09-20T22:10:48.919949+0000","last_became_peered":"2024-09-20T22:10:48.919949+0000","last_unstale":"2024-09-20T22:16:34.840324+0000","last_undegraded":"2024-09-20T22:16:34.840324+0000","last_fullsized":"2024-09-20T22:16:34.840324+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:56.321409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:56.321409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:24.041893+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":739,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.841503+0000","last_change":"2024-09-20T22:15:58.309888+0000","last_active":"2024-09-20T22:16:34.841503+0000","last_peered":"2024-09-20T22:16:34.841503+0000","last_clean":"2024-09-20T22:16:34.841503+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:16:34.841503+0000","last_undegraded":"2024-09-20T22:16:34.841503+0000","last_fullsized":"2024-09-20T22:16:34.841503+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:58.309764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:58.309764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:14.798510+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"409'7010","reported_seq":7098,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:38.039796+0000","last_change":"2024-09-20T22:15:09.088949+0000","last_active":"2024-09-20T22:16:38.039796+0000","last_peered":"2024-09-20T22:16:38.039796+0000","last_clean":"2024-09-20T22:16:38.039796+0000","last_became_active":"2024-09-20T22:15:09.088580+0000","last_became_peered":"2024-09-20T22:15:09.088580+0000","last_unstale":"2024-09-20T22:16:38.039796+0000","last_undegraded":"2024-09-20T22:16:38.039796+0000","last_fullsized":"2024-09-20T22:16:38.039796+0000","mapping_epoch":370,"log_start":"378'2000","ondisk_log_start":"378'2000","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":5010,"log_dups_size":0,"ondisk_log_size":5010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459407360,"num_objects":7010,"num_object_clones":0,"num_object_copies":21030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7010,"num_write_kb":448640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":790,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.616795+0000","last_change":"2024-09-20T22:15:46.874158+0000","last_active":"2024-09-20T22:16:36.616795+0000","last_peered":"2024-09-20T22:16:36.616795+0000","last_clean":"2024-09-20T22:16:36.616795+0000","last_became_active":"2024-09-20T22:15:45.642671+0000","last_became_peered":"2024-09-20T22:15:45.642671+0000","last_unstale":"2024-09-20T22:16:36.616795+0000","last_undegraded":"2024-09-20T22:16:36.616795+0000","last_fullsized":"2024-09-20T22:16:36.616795+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:46.874000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:46.874000+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:16.327234+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"328'28","reported_seq":789,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.840502+0000","last_change":"2024-09-20T22:16:18.404188+0000","last_active":"2024-09-20T22:16:34.840502+0000","last_peered":"2024-09-20T22:16:34.840502+0000","last_clean":"2024-09-20T22:16:34.840502+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:16:34.840502+0000","last_undegraded":"2024-09-20T22:16:34.840502+0000","last_fullsized":"2024-09-20T22:16:34.840502+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:16:18.403972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:18.403972+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:43.385480+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":208,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.588372+0000","last_change":"2024-09-20T22:16:35.952285+0000","last_active":"2024-09-20T22:16:36.588372+0000","last_peered":"2024-09-20T22:16:36.588372+0000","last_clean":"2024-09-20T22:16:36.588372+0000","last_became_active":"2024-09-20T22:12:52.528068+0000","last_became_peered":"2024-09-20T22:12:52.528068+0000","last_unstale":"2024-09-20T22:16:36.588372+0000","last_undegraded":"2024-09-20T22:16:36.588372+0000","last_fullsized":"2024-09-20T22:16:36.588372+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:16:35.952049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:35.952049+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:45.642636+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"315'11","reported_seq":486,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:30.338338+0000","last_change":"2024-09-20T22:15:56.451915+0000","last_active":"2024-09-20T22:16:30.338338+0000","last_peered":"2024-09-20T22:16:30.338338+0000","last_clean":"2024-09-20T22:16:30.338338+0000","last_became_active":"2024-09-20T22:12:33.545942+0000","last_became_peered":"2024-09-20T22:12:33.545942+0000","last_unstale":"2024-09-20T22:16:30.338338+0000","last_undegraded":"2024-09-20T22:16:30.338338+0000","last_fullsized":"2024-09-20T22:16:30.338338+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:15:56.451808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:56.451808+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:22.777565+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":731,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842408+0000","last_change":"2024-09-20T22:15:52.564511+0000","last_active":"2024-09-20T22:16:34.842408+0000","last_peered":"2024-09-20T22:16:34.842408+0000","last_clean":"2024-09-20T22:16:34.842408+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:16:34.842408+0000","last_undegraded":"2024-09-20T22:16:34.842408+0000","last_fullsized":"2024-09-20T22:16:34.842408+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:52.564398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:52.564398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:17.677910+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"407'6914","reported_seq":6998,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:35.824367+0000","last_change":"2024-09-20T22:15:08.931965+0000","last_active":"2024-09-20T22:16:35.824367+0000","last_peered":"2024-09-20T22:16:35.824367+0000","last_clean":"2024-09-20T22:16:35.824367+0000","last_became_active":"2024-09-20T22:15:08.931653+0000","last_became_peered":"2024-09-20T22:15:08.931653+0000","last_unstale":"2024-09-20T22:16:35.824367+0000","last_undegraded":"2024-09-20T22:16:35.824367+0000","last_fullsized":"2024-09-20T22:16:35.824367+0000","mapping_epoch":370,"log_start":"380'2400","ondisk_log_start":"380'2400","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":4514,"log_dups_size":0,"ondisk_log_size":4514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":453115904,"num_objects":6914,"num_object_clones":0,"num_object_copies":20742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6914,"num_write_kb":442496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":788,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:37.937397+0000","last_change":"2024-09-20T22:15:46.585672+0000","last_active":"2024-09-20T22:16:37.937397+0000","last_peered":"2024-09-20T22:16:37.937397+0000","last_clean":"2024-09-20T22:16:37.937397+0000","last_became_active":"2024-09-20T22:15:45.635509+0000","last_became_peered":"2024-09-20T22:15:45.635509+0000","last_unstale":"2024-09-20T22:16:37.937397+0000","last_undegraded":"2024-09-20T22:16:37.937397+0000","last_fullsized":"2024-09-20T22:16:37.937397+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:46.584651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:46.584651+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:55.115974+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"328'20","reported_seq":655,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:28.261634+0000","last_change":"2024-09-20T22:16:28.261680+0000","last_active":"2024-09-20T22:16:28.261634+0000","last_peered":"2024-09-20T22:16:28.261634+0000","last_clean":"2024-09-20T22:16:28.261634+0000","last_became_active":"2024-09-20T22:11:30.486532+0000","last_became_peered":"2024-09-20T22:11:30.486532+0000","last_unstale":"2024-09-20T22:16:28.261634+0000","last_undegraded":"2024-09-20T22:16:28.261634+0000","last_fullsized":"2024-09-20T22:16:28.261634+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:16:28.261547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:28.261547+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:47.319640+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":758,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.846448+0000","last_change":"2024-09-20T22:14:11.402471+0000","last_active":"2024-09-20T22:16:34.846448+0000","last_peered":"2024-09-20T22:16:34.846448+0000","last_clean":"2024-09-20T22:16:34.846448+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:16:34.846448+0000","last_undegraded":"2024-09-20T22:16:34.846448+0000","last_fullsized":"2024-09-20T22:16:34.846448+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:11.402361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:14:11.402361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"363'13","reported_seq":243,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.587547+0000","last_change":"2024-09-20T22:16:25.963851+0000","last_active":"2024-09-20T22:16:36.587547+0000","last_peered":"2024-09-20T22:16:36.587547+0000","last_clean":"2024-09-20T22:16:36.587547+0000","last_became_active":"2024-09-20T22:12:57.884415+0000","last_became_peered":"2024-09-20T22:12:57.884415+0000","last_unstale":"2024-09-20T22:16:36.587547+0000","last_undegraded":"2024-09-20T22:16:36.587547+0000","last_fullsized":"2024-09-20T22:16:36.587547+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"363'13","last_scrub_stamp":"2024-09-20T22:16:25.963742+0000","last_deep_scrub":"363'13","last_deep_scrub_stamp":"2024-09-20T22:14:52.612615+0000","last_clean_scrub_stamp":"2024-09-20T22:16:25.963742+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:50.917231+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":196,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.590067+0000","last_change":"2024-09-20T22:16:31.044840+0000","last_active":"2024-09-20T22:16:36.590067+0000","last_peered":"2024-09-20T22:16:36.590067+0000","last_clean":"2024-09-20T22:16:36.590067+0000","last_became_active":"2024-09-20T22:12:52.518529+0000","last_became_peered":"2024-09-20T22:12:52.518529+0000","last_unstale":"2024-09-20T22:16:36.590067+0000","last_undegraded":"2024-09-20T22:16:36.590067+0000","last_fullsized":"2024-09-20T22:16:36.590067+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:31.044564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:31.044564+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.044564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:00.508338+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"409'7008","reported_seq":7172,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:38.019286+0000","last_change":"2024-09-20T22:16:35.962440+0000","last_active":"2024-09-20T22:16:38.019286+0000","last_peered":"2024-09-20T22:16:38.019286+0000","last_clean":"2024-09-20T22:16:38.019286+0000","last_became_active":"2024-09-20T22:15:08.927661+0000","last_became_peered":"2024-09-20T22:15:08.927661+0000","last_unstale":"2024-09-20T22:16:38.019286+0000","last_undegraded":"2024-09-20T22:16:38.019286+0000","last_fullsized":"2024-09-20T22:16:38.019286+0000","mapping_epoch":370,"log_start":"378'2000","ondisk_log_start":"378'2000","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"407'6856","last_scrub_stamp":"2024-09-20T22:16:35.962220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:35.962220+0000","objects_scrubbed":6781,"log_size":5008,"log_dups_size":0,"ondisk_log_size":5008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:00.411848+0000","scrub_duration":2297,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459276288,"num_objects":7008,"num_object_clones":0,"num_object_copies":21024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7008,"num_write_kb":448512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":789,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:37.937368+0000","last_change":"2024-09-20T22:15:47.629135+0000","last_active":"2024-09-20T22:16:37.937368+0000","last_peered":"2024-09-20T22:16:37.937368+0000","last_clean":"2024-09-20T22:16:37.937368+0000","last_became_active":"2024-09-20T22:15:45.634237+0000","last_became_peered":"2024-09-20T22:15:45.634237+0000","last_unstale":"2024-09-20T22:16:37.937368+0000","last_undegraded":"2024-09-20T22:16:37.937368+0000","last_fullsized":"2024-09-20T22:16:37.937368+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:47.629023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:47.629023+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:49.579004+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"392'37","reported_seq":237,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.587845+0000","last_change":"2024-09-20T22:15:56.128146+0000","last_active":"2024-09-20T22:16:36.587845+0000","last_peered":"2024-09-20T22:16:36.587845+0000","last_clean":"2024-09-20T22:16:36.587845+0000","last_became_active":"2024-09-20T22:12:52.525538+0000","last_became_peered":"2024-09-20T22:12:52.525538+0000","last_unstale":"2024-09-20T22:16:36.587845+0000","last_undegraded":"2024-09-20T22:16:36.587845+0000","last_fullsized":"2024-09-20T22:16:36.587845+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"392'37","last_scrub_stamp":"2024-09-20T22:15:56.127540+0000","last_deep_scrub":"392'37","last_deep_scrub_stamp":"2024-09-20T22:15:56.127540+0000","last_clean_scrub_stamp":"2024-09-20T22:15:56.127540+0000","objects_scrubbed":8,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:01.639462+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":591,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:25.773695+0000","last_change":"2024-09-20T22:16:22.267306+0000","last_active":"2024-09-20T22:16:25.773695+0000","last_peered":"2024-09-20T22:16:25.773695+0000","last_clean":"2024-09-20T22:16:25.773695+0000","last_became_active":"2024-09-20T22:11:30.486428+0000","last_became_peered":"2024-09-20T22:11:30.486428+0000","last_unstale":"2024-09-20T22:16:25.773695+0000","last_undegraded":"2024-09-20T22:16:25.773695+0000","last_fullsized":"2024-09-20T22:16:25.773695+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:22.267191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:22.267191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:32.461862+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":774,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:37.936909+0000","last_change":"2024-09-20T22:15:58.616337+0000","last_active":"2024-09-20T22:16:37.936909+0000","last_peered":"2024-09-20T22:16:37.936909+0000","last_clean":"2024-09-20T22:16:37.936909+0000","last_became_active":"2024-09-20T22:12:52.519826+0000","last_became_peered":"2024-09-20T22:12:52.519826+0000","last_unstale":"2024-09-20T22:16:37.936909+0000","last_undegraded":"2024-09-20T22:16:37.936909+0000","last_fullsized":"2024-09-20T22:16:37.936909+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:58.616150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:15:58.616150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:15.336586+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":577,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:25.774081+0000","last_change":"2024-09-20T22:15:21.489682+0000","last_active":"2024-09-20T22:16:25.774081+0000","last_peered":"2024-09-20T22:16:25.774081+0000","last_clean":"2024-09-20T22:16:25.774081+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:16:25.774081+0000","last_undegraded":"2024-09-20T22:16:25.774081+0000","last_fullsized":"2024-09-20T22:16:25.774081+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:21.489619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:21.489619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"409'7162","reported_seq":7327,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:38.027182+0000","last_change":"2024-09-20T22:16:32.715147+0000","last_active":"2024-09-20T22:16:38.027182+0000","last_peered":"2024-09-20T22:16:38.027182+0000","last_clean":"2024-09-20T22:16:38.027182+0000","last_became_active":"2024-09-20T22:15:08.926245+0000","last_became_peered":"2024-09-20T22:15:08.926245+0000","last_unstale":"2024-09-20T22:16:38.027182+0000","last_undegraded":"2024-09-20T22:16:38.027182+0000","last_fullsized":"2024-09-20T22:16:38.027182+0000","mapping_epoch":370,"log_start":"378'2200","ondisk_log_start":"378'2200","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6800","last_scrub_stamp":"2024-09-20T22:16:32.714946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.714946+0000","objects_scrubbed":6743,"log_size":4962,"log_dups_size":0,"ondisk_log_size":4962,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:39.872460+0000","scrub_duration":2056,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":469368832,"num_objects":7162,"num_object_clones":0,"num_object_copies":21486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7162,"num_write_kb":458368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":772,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.845102+0000","last_change":"2024-09-20T22:15:46.095461+0000","last_active":"2024-09-20T22:16:34.845102+0000","last_peered":"2024-09-20T22:16:34.845102+0000","last_clean":"2024-09-20T22:16:34.845102+0000","last_became_active":"2024-09-20T22:15:45.627135+0000","last_became_peered":"2024-09-20T22:15:45.627135+0000","last_unstale":"2024-09-20T22:16:34.845102+0000","last_undegraded":"2024-09-20T22:16:34.845102+0000","last_fullsized":"2024-09-20T22:16:34.845102+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:46.095188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:46.095188+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:01.343209+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"328'52","reported_seq":784,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:25.773874+0000","last_change":"2024-09-20T22:14:46.592348+0000","last_active":"2024-09-20T22:16:25.773874+0000","last_peered":"2024-09-20T22:16:25.773874+0000","last_clean":"2024-09-20T22:16:25.773874+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:16:25.773874+0000","last_undegraded":"2024-09-20T22:16:25.773874+0000","last_fullsized":"2024-09-20T22:16:25.773874+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'52","last_scrub_stamp":"2024-09-20T22:14:46.592203+0000","last_deep_scrub":"271'40","last_deep_scrub_stamp":"2024-09-20T22:11:05.235779+0000","last_clean_scrub_stamp":"2024-09-20T22:14:46.592203+0000","objects_scrubbed":10,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":760,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.841367+0000","last_change":"2024-09-20T22:16:20.370000+0000","last_active":"2024-09-20T22:16:34.841367+0000","last_peered":"2024-09-20T22:16:34.841367+0000","last_clean":"2024-09-20T22:16:34.841367+0000","last_became_active":"2024-09-20T22:12:57.884257+0000","last_became_peered":"2024-09-20T22:12:57.884257+0000","last_unstale":"2024-09-20T22:16:34.841367+0000","last_undegraded":"2024-09-20T22:16:34.841367+0000","last_fullsized":"2024-09-20T22:16:34.841367+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:20.369864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:20.369864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:22.410427+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":754,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.846023+0000","last_change":"2024-09-20T22:13:51.454966+0000","last_active":"2024-09-20T22:16:34.846023+0000","last_peered":"2024-09-20T22:16:34.846023+0000","last_clean":"2024-09-20T22:16:34.846023+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:16:34.846023+0000","last_undegraded":"2024-09-20T22:16:34.846023+0000","last_fullsized":"2024-09-20T22:16:34.846023+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.454850+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":196,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.593733+0000","last_change":"2024-09-20T22:15:54.083256+0000","last_active":"2024-09-20T22:16:36.593733+0000","last_peered":"2024-09-20T22:16:36.593733+0000","last_clean":"2024-09-20T22:16:36.593733+0000","last_became_active":"2024-09-20T22:12:52.527547+0000","last_became_peered":"2024-09-20T22:12:52.527547+0000","last_unstale":"2024-09-20T22:16:36.593733+0000","last_undegraded":"2024-09-20T22:16:36.593733+0000","last_fullsized":"2024-09-20T22:16:36.593733+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:54.082968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:54.082968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:06.402225+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":732,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.611277+0000","last_change":"2024-09-20T22:15:24.071705+0000","last_active":"2024-09-20T22:16:36.611277+0000","last_peered":"2024-09-20T22:16:36.611277+0000","last_clean":"2024-09-20T22:16:36.611277+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:16:36.611277+0000","last_undegraded":"2024-09-20T22:16:36.611277+0000","last_fullsized":"2024-09-20T22:16:36.611277+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.071593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.071593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:51.612695+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"407'7030","reported_seq":7112,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.219540+0000","last_change":"2024-09-20T22:15:08.931328+0000","last_active":"2024-09-20T22:16:36.219540+0000","last_peered":"2024-09-20T22:16:36.219540+0000","last_clean":"2024-09-20T22:16:36.219540+0000","last_became_active":"2024-09-20T22:15:08.930968+0000","last_became_peered":"2024-09-20T22:15:08.930968+0000","last_unstale":"2024-09-20T22:16:36.219540+0000","last_undegraded":"2024-09-20T22:16:36.219540+0000","last_fullsized":"2024-09-20T22:16:36.219540+0000","mapping_epoch":370,"log_start":"378'2100","ondisk_log_start":"378'2100","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":4930,"log_dups_size":0,"ondisk_log_size":4930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":460718080,"num_objects":7030,"num_object_clones":0,"num_object_copies":21090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7030,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7030,"num_write_kb":449920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":770,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.841965+0000","last_change":"2024-09-20T22:15:46.548031+0000","last_active":"2024-09-20T22:16:34.841965+0000","last_peered":"2024-09-20T22:16:34.841965+0000","last_clean":"2024-09-20T22:16:34.841965+0000","last_became_active":"2024-09-20T22:15:45.620376+0000","last_became_peered":"2024-09-20T22:15:45.620376+0000","last_unstale":"2024-09-20T22:16:34.841965+0000","last_undegraded":"2024-09-20T22:16:34.841965+0000","last_fullsized":"2024-09-20T22:16:34.841965+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:46.547907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:46.547907+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:55.630577+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"328'17","reported_seq":650,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:25.775743+0000","last_change":"2024-09-20T22:13:51.684806+0000","last_active":"2024-09-20T22:16:25.775743+0000","last_peered":"2024-09-20T22:16:25.775743+0000","last_clean":"2024-09-20T22:16:25.775743+0000","last_became_active":"2024-09-20T22:12:52.521073+0000","last_became_peered":"2024-09-20T22:12:52.521073+0000","last_unstale":"2024-09-20T22:16:25.775743+0000","last_undegraded":"2024-09-20T22:16:25.775743+0000","last_fullsized":"2024-09-20T22:16:25.775743+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'17","last_scrub_stamp":"2024-09-20T22:13:51.684667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.684667+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":578,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:25.774143+0000","last_change":"2024-09-20T22:14:49.575469+0000","last_active":"2024-09-20T22:16:25.774143+0000","last_peered":"2024-09-20T22:16:25.774143+0000","last_clean":"2024-09-20T22:16:25.774143+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:16:25.774143+0000","last_undegraded":"2024-09-20T22:16:25.774143+0000","last_fullsized":"2024-09-20T22:16:25.774143+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:49.575363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:49.575363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":804,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.846797+0000","last_change":"2024-09-20T22:15:48.955294+0000","last_active":"2024-09-20T22:16:34.846797+0000","last_peered":"2024-09-20T22:16:34.846797+0000","last_clean":"2024-09-20T22:16:34.846797+0000","last_became_active":"2024-09-20T22:12:52.517255+0000","last_became_peered":"2024-09-20T22:12:52.517255+0000","last_unstale":"2024-09-20T22:16:34.846797+0000","last_undegraded":"2024-09-20T22:16:34.846797+0000","last_fullsized":"2024-09-20T22:16:34.846797+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:15:48.955042+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:15:48.955042+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:16.989469+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":758,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.845172+0000","last_change":"2024-09-20T22:15:24.892562+0000","last_active":"2024-09-20T22:16:34.845172+0000","last_peered":"2024-09-20T22:16:34.845172+0000","last_clean":"2024-09-20T22:16:34.845172+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:16:34.845172+0000","last_undegraded":"2024-09-20T22:16:34.845172+0000","last_fullsized":"2024-09-20T22:16:34.845172+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.891955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.891955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:47.230621+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":731,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.843030+0000","last_change":"2024-09-20T22:16:23.711184+0000","last_active":"2024-09-20T22:16:34.843030+0000","last_peered":"2024-09-20T22:16:34.843030+0000","last_clean":"2024-09-20T22:16:34.843030+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:16:34.843030+0000","last_undegraded":"2024-09-20T22:16:34.843030+0000","last_fullsized":"2024-09-20T22:16:34.843030+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:23.711072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:23.711072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:36.786537+0000","scrub_duration":87,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"408'7004","reported_seq":7165,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:37.172699+0000","last_change":"2024-09-20T22:16:32.694961+0000","last_active":"2024-09-20T22:16:37.172699+0000","last_peered":"2024-09-20T22:16:37.172699+0000","last_clean":"2024-09-20T22:16:37.172699+0000","last_became_active":"2024-09-20T22:15:08.926702+0000","last_became_peered":"2024-09-20T22:15:08.926702+0000","last_unstale":"2024-09-20T22:16:37.172699+0000","last_undegraded":"2024-09-20T22:16:37.172699+0000","last_fullsized":"2024-09-20T22:16:37.172699+0000","mapping_epoch":370,"log_start":"378'2100","ondisk_log_start":"378'2100","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6715","last_scrub_stamp":"2024-09-20T22:16:32.694684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.694684+0000","objects_scrubbed":6639,"log_size":4904,"log_dups_size":0,"ondisk_log_size":4904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:00.250998+0000","scrub_duration":2750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459014144,"num_objects":7004,"num_object_clones":0,"num_object_copies":21012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7004,"num_write_kb":448256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":751,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:25.773816+0000","last_change":"2024-09-20T22:15:46.338840+0000","last_active":"2024-09-20T22:16:25.773816+0000","last_peered":"2024-09-20T22:16:25.773816+0000","last_clean":"2024-09-20T22:16:25.773816+0000","last_became_active":"2024-09-20T22:15:45.630804+0000","last_became_peered":"2024-09-20T22:15:45.630804+0000","last_unstale":"2024-09-20T22:16:25.773816+0000","last_undegraded":"2024-09-20T22:16:25.773816+0000","last_fullsized":"2024-09-20T22:16:25.773816+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:46.338700+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:15:46.338700+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:06.684506+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"328'24","reported_seq":228,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.588345+0000","last_change":"2024-09-20T22:15:26.742439+0000","last_active":"2024-09-20T22:16:36.588345+0000","last_peered":"2024-09-20T22:16:36.588345+0000","last_clean":"2024-09-20T22:16:36.588345+0000","last_became_active":"2024-09-20T22:12:52.527919+0000","last_became_peered":"2024-09-20T22:12:52.527919+0000","last_unstale":"2024-09-20T22:16:36.588345+0000","last_undegraded":"2024-09-20T22:16:36.588345+0000","last_fullsized":"2024-09-20T22:16:36.588345+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'24","last_scrub_stamp":"2024-09-20T22:15:26.742317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:26.742317+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":196,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.588475+0000","last_change":"2024-09-20T22:15:21.774404+0000","last_active":"2024-09-20T22:16:36.588475+0000","last_peered":"2024-09-20T22:16:36.588475+0000","last_clean":"2024-09-20T22:16:36.588475+0000","last_became_active":"2024-09-20T22:12:52.517432+0000","last_became_peered":"2024-09-20T22:12:52.517432+0000","last_unstale":"2024-09-20T22:16:36.588475+0000","last_undegraded":"2024-09-20T22:16:36.588475+0000","last_fullsized":"2024-09-20T22:16:36.588475+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:21.774300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:15:21.774300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":1591,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842676+0000","last_change":"2024-09-20T22:15:55.652986+0000","last_active":"2024-09-20T22:16:34.842676+0000","last_peered":"2024-09-20T22:16:34.842676+0000","last_clean":"2024-09-20T22:16:34.842676+0000","last_became_active":"2024-09-20T22:12:52.526622+0000","last_became_peered":"2024-09-20T22:12:52.526622+0000","last_unstale":"2024-09-20T22:16:34.842676+0000","last_undegraded":"2024-09-20T22:16:34.842676+0000","last_fullsized":"2024-09-20T22:16:34.842676+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:15:55.652860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:15:55.652860+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:58.127886+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":801,"num_read_kb":796,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":289,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.847183+0000","last_change":"2024-09-20T22:14:14.443714+0000","last_active":"2024-09-20T22:16:34.847183+0000","last_peered":"2024-09-20T22:16:34.847183+0000","last_clean":"2024-09-20T22:16:34.847183+0000","last_became_active":"2024-09-20T22:12:52.518823+0000","last_became_peered":"2024-09-20T22:12:52.518823+0000","last_unstale":"2024-09-20T22:16:34.847183+0000","last_undegraded":"2024-09-20T22:16:34.847183+0000","last_fullsized":"2024-09-20T22:16:34.847183+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:14.443611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:14.443611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":578,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:26.330369+0000","last_change":"2024-09-20T22:16:26.330409+0000","last_active":"2024-09-20T22:16:26.330369+0000","last_peered":"2024-09-20T22:16:26.330369+0000","last_clean":"2024-09-20T22:16:26.330369+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:16:26.330369+0000","last_undegraded":"2024-09-20T22:16:26.330369+0000","last_fullsized":"2024-09-20T22:16:26.330369+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:26.330270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:26.330270+0000","last_clean_scrub_stamp":"2024-09-20T22:16:26.330270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:27.069295+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"409'7163","reported_seq":7250,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:38.003599+0000","last_change":"2024-09-20T22:15:08.925131+0000","last_active":"2024-09-20T22:16:38.003599+0000","last_peered":"2024-09-20T22:16:38.003599+0000","last_clean":"2024-09-20T22:16:38.003599+0000","last_became_active":"2024-09-20T22:15:08.924809+0000","last_became_peered":"2024-09-20T22:15:08.924809+0000","last_unstale":"2024-09-20T22:16:38.003599+0000","last_undegraded":"2024-09-20T22:16:38.003599+0000","last_fullsized":"2024-09-20T22:16:38.003599+0000","mapping_epoch":370,"log_start":"378'2200","ondisk_log_start":"378'2200","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":4963,"log_dups_size":0,"ondisk_log_size":4963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":469434368,"num_objects":7163,"num_object_clones":0,"num_object_copies":21489,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7163,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7163,"num_write_kb":458432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"328'28","reported_seq":964,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:37.938461+0000","last_change":"2024-09-20T22:15:55.704801+0000","last_active":"2024-09-20T22:16:37.938461+0000","last_peered":"2024-09-20T22:16:37.938461+0000","last_clean":"2024-09-20T22:16:37.938461+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:16:37.938461+0000","last_undegraded":"2024-09-20T22:16:37.938461+0000","last_fullsized":"2024-09-20T22:16:37.938461+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:15:55.704666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:55.704666+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:23.796840+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":695,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.845237+0000","last_change":"2024-09-20T22:12:42.260316+0000","last_active":"2024-09-20T22:16:34.845237+0000","last_peered":"2024-09-20T22:16:34.845237+0000","last_clean":"2024-09-20T22:16:34.845237+0000","last_became_active":"2024-09-20T22:12:21.519717+0000","last_became_peered":"2024-09-20T22:12:21.519717+0000","last_unstale":"2024-09-20T22:16:34.845237+0000","last_undegraded":"2024-09-20T22:16:34.845237+0000","last_fullsized":"2024-09-20T22:16:34.845237+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:12:42.260197+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:12:42.260197+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":770,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.841985+0000","last_change":"2024-09-20T22:15:47.554095+0000","last_active":"2024-09-20T22:16:34.841985+0000","last_peered":"2024-09-20T22:16:34.841985+0000","last_clean":"2024-09-20T22:16:34.841985+0000","last_became_active":"2024-09-20T22:15:45.620972+0000","last_became_peered":"2024-09-20T22:15:45.620972+0000","last_unstale":"2024-09-20T22:16:34.841985+0000","last_undegraded":"2024-09-20T22:16:34.841985+0000","last_fullsized":"2024-09-20T22:16:34.841985+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:47.553962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:47.553962+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:48.105550+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":325,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842588+0000","last_change":"2024-09-20T22:15:58.617583+0000","last_active":"2024-09-20T22:16:34.842588+0000","last_peered":"2024-09-20T22:16:34.842588+0000","last_clean":"2024-09-20T22:16:34.842588+0000","last_became_active":"2024-09-20T22:10:48.907271+0000","last_became_peered":"2024-09-20T22:10:48.907271+0000","last_unstale":"2024-09-20T22:16:34.842588+0000","last_undegraded":"2024-09-20T22:16:34.842588+0000","last_fullsized":"2024-09-20T22:16:34.842588+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:58.617484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_clean_scrub_stamp":"2024-09-20T22:15:58.617484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:05.365475+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":750,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.846915+0000","last_change":"2024-09-20T22:13:02.482655+0000","last_active":"2024-09-20T22:16:34.846915+0000","last_peered":"2024-09-20T22:16:34.846915+0000","last_clean":"2024-09-20T22:16:34.846915+0000","last_became_active":"2024-09-20T22:12:44.630044+0000","last_became_peered":"2024-09-20T22:12:44.630044+0000","last_unstale":"2024-09-20T22:16:34.846915+0000","last_undegraded":"2024-09-20T22:16:34.846915+0000","last_fullsized":"2024-09-20T22:16:34.846915+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:02.482546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:13:02.482546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"75'128","reported_seq":759,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:25.773604+0000","last_change":"2024-09-20T22:15:51.314327+0000","last_active":"2024-09-20T22:16:25.773604+0000","last_peered":"2024-09-20T22:16:25.773604+0000","last_clean":"2024-09-20T22:16:25.773604+0000","last_became_active":"2024-09-20T22:15:50.615076+0000","last_became_peered":"2024-09-20T22:15:50.615076+0000","last_unstale":"2024-09-20T22:16:25.773604+0000","last_undegraded":"2024-09-20T22:16:25.773604+0000","last_fullsized":"2024-09-20T22:16:25.773604+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:51.314232+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:15:51.314232+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:19.808113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"409'7241","reported_seq":7379,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:38.001165+0000","last_change":"2024-09-20T22:16:37.959295+0000","last_active":"2024-09-20T22:16:38.001165+0000","last_peered":"2024-09-20T22:16:38.001165+0000","last_clean":"2024-09-20T22:16:38.001165+0000","last_became_active":"2024-09-20T22:15:08.925040+0000","last_became_peered":"2024-09-20T22:15:08.925040+0000","last_unstale":"2024-09-20T22:16:38.001165+0000","last_undegraded":"2024-09-20T22:16:38.001165+0000","last_fullsized":"2024-09-20T22:16:38.001165+0000","mapping_epoch":370,"log_start":"378'2300","ondisk_log_start":"378'2300","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":4649,"log_size":4941,"log_dups_size":0,"ondisk_log_size":4941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:42.943921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":474546176,"num_objects":7241,"num_object_clones":0,"num_object_copies":21723,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7241,"num_write_kb":463424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"328'19","reported_seq":779,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.840977+0000","last_change":"2024-09-20T22:16:24.435555+0000","last_active":"2024-09-20T22:16:34.840977+0000","last_peered":"2024-09-20T22:16:34.840977+0000","last_clean":"2024-09-20T22:16:34.840977+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:16:34.840977+0000","last_undegraded":"2024-09-20T22:16:34.840977+0000","last_fullsized":"2024-09-20T22:16:34.840977+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:16:24.435260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:24.435260+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:26.710702+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":743,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.840709+0000","last_change":"2024-09-20T22:15:27.077414+0000","last_active":"2024-09-20T22:16:34.840709+0000","last_peered":"2024-09-20T22:16:34.840709+0000","last_clean":"2024-09-20T22:16:34.840709+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:16:34.840709+0000","last_undegraded":"2024-09-20T22:16:34.840709+0000","last_fullsized":"2024-09-20T22:16:34.840709+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:27.077295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.077295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:47.688033+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":742,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.846727+0000","last_change":"2024-09-20T22:15:24.150401+0000","last_active":"2024-09-20T22:16:34.846727+0000","last_peered":"2024-09-20T22:16:34.846727+0000","last_clean":"2024-09-20T22:16:34.846727+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:16:34.846727+0000","last_undegraded":"2024-09-20T22:16:34.846727+0000","last_fullsized":"2024-09-20T22:16:34.846727+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.150265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.150265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":701,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:37.937214+0000","last_change":"2024-09-20T22:15:52.643084+0000","last_active":"2024-09-20T22:16:37.937214+0000","last_peered":"2024-09-20T22:16:37.937214+0000","last_clean":"2024-09-20T22:16:37.937214+0000","last_became_active":"2024-09-20T22:12:21.521195+0000","last_became_peered":"2024-09-20T22:12:21.521195+0000","last_unstale":"2024-09-20T22:16:37.937214+0000","last_undegraded":"2024-09-20T22:16:37.937214+0000","last_fullsized":"2024-09-20T22:16:37.937214+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:52.642911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:52.642911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:16.037925+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3","version":"407'6972","reported_seq":7133,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.283648+0000","last_change":"2024-09-20T22:16:32.632993+0000","last_active":"2024-09-20T22:16:36.283648+0000","last_peered":"2024-09-20T22:16:36.283648+0000","last_clean":"2024-09-20T22:16:36.283648+0000","last_became_active":"2024-09-20T22:15:09.087061+0000","last_became_peered":"2024-09-20T22:15:09.087061+0000","last_unstale":"2024-09-20T22:16:36.283648+0000","last_undegraded":"2024-09-20T22:16:36.283648+0000","last_fullsized":"2024-09-20T22:16:36.283648+0000","mapping_epoch":370,"log_start":"377'1000","ondisk_log_start":"377'1000","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6740","last_scrub_stamp":"2024-09-20T22:16:32.632756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.632756+0000","objects_scrubbed":6652,"log_size":5972,"log_dups_size":0,"ondisk_log_size":5972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:35.823889+0000","scrub_duration":2683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":456916992,"num_objects":6972,"num_object_clones":0,"num_object_copies":20916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6972,"num_write_kb":446208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":519,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:37.936786+0000","last_change":"2024-09-20T22:16:18.773405+0000","last_active":"2024-09-20T22:16:37.936786+0000","last_peered":"2024-09-20T22:16:37.936786+0000","last_clean":"2024-09-20T22:16:37.936786+0000","last_became_active":"2024-09-20T22:14:38.674758+0000","last_became_peered":"2024-09-20T22:14:38.674758+0000","last_unstale":"2024-09-20T22:16:37.936786+0000","last_undegraded":"2024-09-20T22:16:37.936786+0000","last_fullsized":"2024-09-20T22:16:37.936786+0000","mapping_epoch":358,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":359,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:16:18.773201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:16:18.773201+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:39.013289+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"328'31","reported_seq":805,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.841543+0000","last_change":"2024-09-20T22:16:17.388362+0000","last_active":"2024-09-20T22:16:34.841543+0000","last_peered":"2024-09-20T22:16:34.841543+0000","last_clean":"2024-09-20T22:16:34.841543+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:16:34.841543+0000","last_undegraded":"2024-09-20T22:16:34.841543+0000","last_fullsized":"2024-09-20T22:16:34.841543+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'31","last_scrub_stamp":"2024-09-20T22:16:17.387895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:17.387895+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:42.903740+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":763,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:37.937367+0000","last_change":"2024-09-20T22:15:32.512124+0000","last_active":"2024-09-20T22:16:37.937367+0000","last_peered":"2024-09-20T22:16:37.937367+0000","last_clean":"2024-09-20T22:16:37.937367+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:16:37.937367+0000","last_undegraded":"2024-09-20T22:16:37.937367+0000","last_fullsized":"2024-09-20T22:16:37.937367+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:32.506512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:32.506512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:01.564420+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":590,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:25.773618+0000","last_change":"2024-09-20T22:15:57.442451+0000","last_active":"2024-09-20T22:16:25.773618+0000","last_peered":"2024-09-20T22:16:25.773618+0000","last_clean":"2024-09-20T22:16:25.773618+0000","last_became_active":"2024-09-20T22:12:52.520612+0000","last_became_peered":"2024-09-20T22:12:52.520612+0000","last_unstale":"2024-09-20T22:16:25.773618+0000","last_undegraded":"2024-09-20T22:16:25.773618+0000","last_fullsized":"2024-09-20T22:16:25.773618+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:57.442341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.442341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:10.741669+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":754,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842403+0000","last_change":"2024-09-20T22:15:56.941775+0000","last_active":"2024-09-20T22:16:34.842403+0000","last_peered":"2024-09-20T22:16:34.842403+0000","last_clean":"2024-09-20T22:16:34.842403+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:16:34.842403+0000","last_undegraded":"2024-09-20T22:16:34.842403+0000","last_fullsized":"2024-09-20T22:16:34.842403+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:56.941427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:56.941427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:04.318727+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"406'6767","reported_seq":6849,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:33.401034+0000","last_change":"2024-09-20T22:15:08.926429+0000","last_active":"2024-09-20T22:16:33.401034+0000","last_peered":"2024-09-20T22:16:33.401034+0000","last_clean":"2024-09-20T22:16:33.401034+0000","last_became_active":"2024-09-20T22:15:08.926083+0000","last_became_peered":"2024-09-20T22:15:08.926083+0000","last_unstale":"2024-09-20T22:16:33.401034+0000","last_undegraded":"2024-09-20T22:16:33.401034+0000","last_fullsized":"2024-09-20T22:16:33.401034+0000","mapping_epoch":370,"log_start":"378'2000","ondisk_log_start":"378'2000","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":4767,"log_dups_size":0,"ondisk_log_size":4767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":443482112,"num_objects":6767,"num_object_clones":0,"num_object_copies":20301,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6767,"num_write_kb":433088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":1026,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842719+0000","last_change":"2024-09-20T22:15:56.638628+0000","last_active":"2024-09-20T22:16:34.842719+0000","last_peered":"2024-09-20T22:16:34.842719+0000","last_clean":"2024-09-20T22:16:34.842719+0000","last_became_active":"2024-09-20T22:08:48.455288+0000","last_became_peered":"2024-09-20T22:08:48.455288+0000","last_unstale":"2024-09-20T22:16:34.842719+0000","last_undegraded":"2024-09-20T22:16:34.842719+0000","last_fullsized":"2024-09-20T22:16:34.842719+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:56.638414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:56.638414+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:22.223454+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"328'25","reported_seq":774,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.843292+0000","last_change":"2024-09-20T22:15:50.223866+0000","last_active":"2024-09-20T22:16:34.843292+0000","last_peered":"2024-09-20T22:16:34.843292+0000","last_clean":"2024-09-20T22:16:34.843292+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:16:34.843292+0000","last_undegraded":"2024-09-20T22:16:34.843292+0000","last_fullsized":"2024-09-20T22:16:34.843292+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'25","last_scrub_stamp":"2024-09-20T22:15:50.223785+0000","last_deep_scrub":"289'15","last_deep_scrub_stamp":"2024-09-20T22:11:37.277648+0000","last_clean_scrub_stamp":"2024-09-20T22:15:50.223785+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:16.829889+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":200,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.587749+0000","last_change":"2024-09-20T22:15:30.821251+0000","last_active":"2024-09-20T22:16:36.587749+0000","last_peered":"2024-09-20T22:16:36.587749+0000","last_clean":"2024-09-20T22:16:36.587749+0000","last_became_active":"2024-09-20T22:12:52.528310+0000","last_became_peered":"2024-09-20T22:12:52.528310+0000","last_unstale":"2024-09-20T22:16:36.587749+0000","last_undegraded":"2024-09-20T22:16:36.587749+0000","last_fullsized":"2024-09-20T22:16:36.587749+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:15:30.821142+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:15:30.821142+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:55.515561+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"283'11","reported_seq":961,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.846852+0000","last_change":"2024-09-20T22:15:49.973513+0000","last_active":"2024-09-20T22:16:34.846852+0000","last_peered":"2024-09-20T22:16:34.846852+0000","last_clean":"2024-09-20T22:16:34.846852+0000","last_became_active":"2024-09-20T22:11:30.485356+0000","last_became_peered":"2024-09-20T22:11:30.485356+0000","last_unstale":"2024-09-20T22:16:34.846852+0000","last_undegraded":"2024-09-20T22:16:34.846852+0000","last_fullsized":"2024-09-20T22:16:34.846852+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'11","last_scrub_stamp":"2024-09-20T22:15:49.973229+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:15:49.973229+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:52.728072+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":740,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.840682+0000","last_change":"2024-09-20T22:16:27.486633+0000","last_active":"2024-09-20T22:16:34.840682+0000","last_peered":"2024-09-20T22:16:34.840682+0000","last_clean":"2024-09-20T22:16:34.840682+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:16:34.840682+0000","last_undegraded":"2024-09-20T22:16:34.840682+0000","last_fullsized":"2024-09-20T22:16:34.840682+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:27.486520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:16:27.486520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:41.356713+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"75'128","reported_seq":632,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.840419+0000","last_change":"2024-09-20T22:15:51.252861+0000","last_active":"2024-09-20T22:16:34.840419+0000","last_peered":"2024-09-20T22:16:34.840419+0000","last_clean":"2024-09-20T22:16:34.840419+0000","last_became_active":"2024-09-20T22:15:50.594320+0000","last_became_peered":"2024-09-20T22:15:50.594320+0000","last_unstale":"2024-09-20T22:16:34.840419+0000","last_undegraded":"2024-09-20T22:16:34.840419+0000","last_fullsized":"2024-09-20T22:16:34.840419+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:51.252685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:51.252685+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:04.804456+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"407'6984","reported_seq":7137,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.269726+0000","last_change":"2024-09-20T22:16:21.665425+0000","last_active":"2024-09-20T22:16:36.269726+0000","last_peered":"2024-09-20T22:16:36.269726+0000","last_clean":"2024-09-20T22:16:36.269726+0000","last_became_active":"2024-09-20T22:15:09.087830+0000","last_became_peered":"2024-09-20T22:15:09.087830+0000","last_unstale":"2024-09-20T22:16:36.269726+0000","last_undegraded":"2024-09-20T22:16:36.269726+0000","last_fullsized":"2024-09-20T22:16:36.269726+0000","mapping_epoch":370,"log_start":"378'1100","ondisk_log_start":"378'1100","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"403'6042","last_scrub_stamp":"2024-09-20T22:16:21.665289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:21.665289+0000","objects_scrubbed":6017,"log_size":5884,"log_dups_size":0,"ondisk_log_size":5884,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:21.711237+0000","scrub_duration":1845,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":457703424,"num_objects":6984,"num_object_clones":0,"num_object_copies":20952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6984,"num_write_kb":446976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"328'15","reported_seq":215,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.588034+0000","last_change":"2024-09-20T22:15:25.742250+0000","last_active":"2024-09-20T22:16:36.588034+0000","last_peered":"2024-09-20T22:16:36.588034+0000","last_clean":"2024-09-20T22:16:36.588034+0000","last_became_active":"2024-09-20T22:12:52.527531+0000","last_became_peered":"2024-09-20T22:12:52.527531+0000","last_unstale":"2024-09-20T22:16:36.588034+0000","last_undegraded":"2024-09-20T22:16:36.588034+0000","last_fullsized":"2024-09-20T22:16:36.588034+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:15:25.741631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:25.741631+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":380,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.619959+0000","last_change":"2024-09-20T22:16:29.707733+0000","last_active":"2024-09-20T22:16:36.619959+0000","last_peered":"2024-09-20T22:16:36.619959+0000","last_clean":"2024-09-20T22:16:36.619959+0000","last_became_active":"2024-09-20T22:10:14.536155+0000","last_became_peered":"2024-09-20T22:10:14.536155+0000","last_unstale":"2024-09-20T22:16:36.619959+0000","last_undegraded":"2024-09-20T22:16:36.619959+0000","last_fullsized":"2024-09-20T22:16:36.619959+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:16:29.707441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:29.707441+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:45.157085+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":766,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.841613+0000","last_change":"2024-09-20T22:16:33.008048+0000","last_active":"2024-09-20T22:16:34.841613+0000","last_peered":"2024-09-20T22:16:34.841613+0000","last_clean":"2024-09-20T22:16:34.841613+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:16:34.841613+0000","last_undegraded":"2024-09-20T22:16:34.841613+0000","last_fullsized":"2024-09-20T22:16:34.841613+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:33.007962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:16:33.007962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:56.423160+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":196,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.589088+0000","last_change":"2024-09-20T22:16:24.020680+0000","last_active":"2024-09-20T22:16:36.589088+0000","last_peered":"2024-09-20T22:16:36.589088+0000","last_clean":"2024-09-20T22:16:36.589088+0000","last_became_active":"2024-09-20T22:12:52.512924+0000","last_became_peered":"2024-09-20T22:12:52.512924+0000","last_unstale":"2024-09-20T22:16:36.589088+0000","last_undegraded":"2024-09-20T22:16:36.589088+0000","last_fullsized":"2024-09-20T22:16:36.589088+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:24.020573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:24.020573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:44.565343+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"75'128","reported_seq":796,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:37.938165+0000","last_change":"2024-09-20T22:15:50.666033+0000","last_active":"2024-09-20T22:16:37.938165+0000","last_peered":"2024-09-20T22:16:37.938165+0000","last_clean":"2024-09-20T22:16:37.938165+0000","last_became_active":"2024-09-20T22:15:50.576540+0000","last_became_peered":"2024-09-20T22:15:50.576540+0000","last_unstale":"2024-09-20T22:16:37.938165+0000","last_undegraded":"2024-09-20T22:16:37.938165+0000","last_fullsized":"2024-09-20T22:16:37.938165+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:50.665724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:50.665724+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:57.862343+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.4","version":"408'6978","reported_seq":7138,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:37.169583+0000","last_change":"2024-09-20T22:16:31.859517+0000","last_active":"2024-09-20T22:16:37.169583+0000","last_peered":"2024-09-20T22:16:37.169583+0000","last_clean":"2024-09-20T22:16:37.169583+0000","last_became_active":"2024-09-20T22:15:09.087298+0000","last_became_peered":"2024-09-20T22:15:09.087298+0000","last_unstale":"2024-09-20T22:16:37.169583+0000","last_undegraded":"2024-09-20T22:16:37.169583+0000","last_fullsized":"2024-09-20T22:16:37.169583+0000","mapping_epoch":370,"log_start":"378'2100","ondisk_log_start":"378'2100","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6582","last_scrub_stamp":"2024-09-20T22:16:31.859305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.859305+0000","objects_scrubbed":6495,"log_size":4878,"log_dups_size":0,"ondisk_log_size":4878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:58.626512+0000","scrub_duration":2899,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":457310208,"num_objects":6978,"num_object_clones":0,"num_object_copies":20934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6978,"num_write_kb":446592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"328'35","reported_seq":858,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:37.937028+0000","last_change":"2024-09-20T22:15:31.492645+0000","last_active":"2024-09-20T22:16:37.937028+0000","last_peered":"2024-09-20T22:16:37.937028+0000","last_clean":"2024-09-20T22:16:37.937028+0000","last_became_active":"2024-09-20T22:10:30.652967+0000","last_became_peered":"2024-09-20T22:10:30.652967+0000","last_unstale":"2024-09-20T22:16:37.937028+0000","last_undegraded":"2024-09-20T22:16:37.937028+0000","last_fullsized":"2024-09-20T22:16:37.937028+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'35","last_scrub_stamp":"2024-09-20T22:15:31.492433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:31.492433+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:55.689722+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":755,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.843304+0000","last_change":"2024-09-20T22:15:23.351913+0000","last_active":"2024-09-20T22:16:34.843304+0000","last_peered":"2024-09-20T22:16:34.843304+0000","last_clean":"2024-09-20T22:16:34.843304+0000","last_became_active":"2024-09-20T22:11:30.488650+0000","last_became_peered":"2024-09-20T22:11:30.488650+0000","last_unstale":"2024-09-20T22:16:34.843304+0000","last_undegraded":"2024-09-20T22:16:34.843304+0000","last_fullsized":"2024-09-20T22:16:34.843304+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:23.351789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:12:46.602035+0000","last_clean_scrub_stamp":"2024-09-20T22:15:23.351789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:36.963085+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":790,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.856877+0000","last_change":"2024-09-20T22:16:18.795310+0000","last_active":"2024-09-20T22:16:34.856877+0000","last_peered":"2024-09-20T22:16:34.856877+0000","last_clean":"2024-09-20T22:16:34.856877+0000","last_became_active":"2024-09-20T22:12:52.515295+0000","last_became_peered":"2024-09-20T22:12:52.515295+0000","last_unstale":"2024-09-20T22:16:34.856877+0000","last_undegraded":"2024-09-20T22:16:34.856877+0000","last_fullsized":"2024-09-20T22:16:34.856877+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:18.795079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_clean_scrub_stamp":"2024-09-20T22:16:18.795079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:36.635490+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":347,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:35.739367+0000","last_change":"2024-09-20T22:16:35.739421+0000","last_active":"2024-09-20T22:16:35.739367+0000","last_peered":"2024-09-20T22:16:35.739367+0000","last_clean":"2024-09-20T22:16:35.739367+0000","last_became_active":"2024-09-20T22:10:30.657116+0000","last_became_peered":"2024-09-20T22:10:30.657116+0000","last_unstale":"2024-09-20T22:16:35.739367+0000","last_undegraded":"2024-09-20T22:16:35.739367+0000","last_fullsized":"2024-09-20T22:16:35.739367+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:35.739168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:35.739168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:45.935247+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"75'128","reported_seq":804,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.618475+0000","last_change":"2024-09-20T22:15:50.922162+0000","last_active":"2024-09-20T22:16:36.618475+0000","last_peered":"2024-09-20T22:16:36.618475+0000","last_clean":"2024-09-20T22:16:36.618475+0000","last_became_active":"2024-09-20T22:15:50.619935+0000","last_became_peered":"2024-09-20T22:15:50.619935+0000","last_unstale":"2024-09-20T22:16:36.618475+0000","last_undegraded":"2024-09-20T22:16:36.618475+0000","last_fullsized":"2024-09-20T22:16:36.618475+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:50.922028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:50.922028+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:16.552077+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.7","version":"409'7123","reported_seq":7287,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:38.005747+0000","last_change":"2024-09-20T22:16:29.896220+0000","last_active":"2024-09-20T22:16:38.005747+0000","last_peered":"2024-09-20T22:16:38.005747+0000","last_clean":"2024-09-20T22:16:38.005747+0000","last_became_active":"2024-09-20T22:15:09.088590+0000","last_became_peered":"2024-09-20T22:15:09.088590+0000","last_unstale":"2024-09-20T22:16:38.005747+0000","last_undegraded":"2024-09-20T22:16:38.005747+0000","last_fullsized":"2024-09-20T22:16:38.005747+0000","mapping_epoch":370,"log_start":"378'2200","ondisk_log_start":"378'2200","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6589","last_scrub_stamp":"2024-09-20T22:16:29.896110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:29.896110+0000","objects_scrubbed":6527,"log_size":4923,"log_dups_size":0,"ondisk_log_size":4923,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:39.515203+0000","scrub_duration":2191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":466812928,"num_objects":7123,"num_object_clones":0,"num_object_copies":21369,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7123,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7123,"num_write_kb":455872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"328'18","reported_seq":772,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.840364+0000","last_change":"2024-09-20T22:15:22.019818+0000","last_active":"2024-09-20T22:16:34.840364+0000","last_peered":"2024-09-20T22:16:34.840364+0000","last_clean":"2024-09-20T22:16:34.840364+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:16:34.840364+0000","last_undegraded":"2024-09-20T22:16:34.840364+0000","last_fullsized":"2024-09-20T22:16:34.840364+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'18","last_scrub_stamp":"2024-09-20T22:15:22.019700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:22.019700+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:42.439452+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":752,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.843102+0000","last_change":"2024-09-20T22:16:25.649600+0000","last_active":"2024-09-20T22:16:34.843102+0000","last_peered":"2024-09-20T22:16:34.843102+0000","last_clean":"2024-09-20T22:16:34.843102+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:16:34.843102+0000","last_undegraded":"2024-09-20T22:16:34.843102+0000","last_fullsized":"2024-09-20T22:16:34.843102+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:25.649511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:25.649511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:42.623417+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":779,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.854616+0000","last_change":"2024-09-20T22:15:27.924542+0000","last_active":"2024-09-20T22:16:34.854616+0000","last_peered":"2024-09-20T22:16:34.854616+0000","last_clean":"2024-09-20T22:16:34.854616+0000","last_became_active":"2024-09-20T22:12:52.514202+0000","last_became_peered":"2024-09-20T22:12:52.514202+0000","last_unstale":"2024-09-20T22:16:34.854616+0000","last_undegraded":"2024-09-20T22:16:34.854616+0000","last_fullsized":"2024-09-20T22:16:34.854616+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:27.924150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.924150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:41.071832+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":206,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.608614+0000","last_change":"2024-09-20T22:15:26.021754+0000","last_active":"2024-09-20T22:16:36.608614+0000","last_peered":"2024-09-20T22:16:36.608614+0000","last_clean":"2024-09-20T22:16:36.608614+0000","last_became_active":"2024-09-20T22:12:44.638546+0000","last_became_peered":"2024-09-20T22:12:44.638546+0000","last_unstale":"2024-09-20T22:16:36.608614+0000","last_undegraded":"2024-09-20T22:16:36.608614+0000","last_fullsized":"2024-09-20T22:16:36.608614+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:26.021636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:26.021636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:46.884542+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'128","reported_seq":348,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842795+0000","last_change":"2024-09-20T22:15:51.552197+0000","last_active":"2024-09-20T22:16:34.842795+0000","last_peered":"2024-09-20T22:16:34.842795+0000","last_clean":"2024-09-20T22:16:34.842795+0000","last_became_active":"2024-09-20T22:15:50.594241+0000","last_became_peered":"2024-09-20T22:15:50.594241+0000","last_unstale":"2024-09-20T22:16:34.842795+0000","last_undegraded":"2024-09-20T22:16:34.842795+0000","last_fullsized":"2024-09-20T22:16:34.842795+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:51.552071+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:15:51.552071+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:19.449851+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.6","version":"407'7094","reported_seq":7177,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.219630+0000","last_change":"2024-09-20T22:15:08.928417+0000","last_active":"2024-09-20T22:16:36.219630+0000","last_peered":"2024-09-20T22:16:36.219630+0000","last_clean":"2024-09-20T22:16:36.219630+0000","last_became_active":"2024-09-20T22:15:08.928102+0000","last_became_peered":"2024-09-20T22:15:08.928102+0000","last_unstale":"2024-09-20T22:16:36.219630+0000","last_undegraded":"2024-09-20T22:16:36.219630+0000","last_fullsized":"2024-09-20T22:16:36.219630+0000","mapping_epoch":370,"log_start":"378'2100","ondisk_log_start":"378'2100","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":4994,"log_dups_size":0,"ondisk_log_size":4994,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":464912384,"num_objects":7094,"num_object_clones":0,"num_object_copies":21282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7094,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7094,"num_write_kb":454016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"328'27","reported_seq":303,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.844879+0000","last_change":"2024-09-20T22:13:10.353607+0000","last_active":"2024-09-20T22:16:34.844879+0000","last_peered":"2024-09-20T22:16:34.844879+0000","last_clean":"2024-09-20T22:16:34.844879+0000","last_became_active":"2024-09-20T22:11:30.493275+0000","last_became_peered":"2024-09-20T22:11:30.493275+0000","last_unstale":"2024-09-20T22:16:34.844879+0000","last_undegraded":"2024-09-20T22:16:34.844879+0000","last_fullsized":"2024-09-20T22:16:34.844879+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:13:10.353459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.353459+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":369,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842633+0000","last_change":"2024-09-20T22:14:56.339026+0000","last_active":"2024-09-20T22:16:34.842633+0000","last_peered":"2024-09-20T22:16:34.842633+0000","last_clean":"2024-09-20T22:16:34.842633+0000","last_became_active":"2024-09-20T22:12:52.526551+0000","last_became_peered":"2024-09-20T22:12:52.526551+0000","last_unstale":"2024-09-20T22:16:34.842633+0000","last_undegraded":"2024-09-20T22:16:34.842633+0000","last_fullsized":"2024-09-20T22:16:34.842633+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:14:56.338941+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:14:56.338941+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":909,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842514+0000","last_change":"2024-09-20T22:16:32.744150+0000","last_active":"2024-09-20T22:16:34.842514+0000","last_peered":"2024-09-20T22:16:34.842514+0000","last_clean":"2024-09-20T22:16:34.842514+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:16:34.842514+0000","last_undegraded":"2024-09-20T22:16:34.842514+0000","last_fullsized":"2024-09-20T22:16:34.842514+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:16:32.744031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.744031+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:47.355070+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":289,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.845668+0000","last_change":"2024-09-20T22:16:21.904789+0000","last_active":"2024-09-20T22:16:34.845668+0000","last_peered":"2024-09-20T22:16:34.845668+0000","last_clean":"2024-09-20T22:16:34.845668+0000","last_became_active":"2024-09-20T22:12:52.517758+0000","last_became_peered":"2024-09-20T22:12:52.517758+0000","last_unstale":"2024-09-20T22:16:34.845668+0000","last_undegraded":"2024-09-20T22:16:34.845668+0000","last_fullsized":"2024-09-20T22:16:34.845668+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:21.904548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:21.904548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:36.763352+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"75'128","reported_seq":625,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.845331+0000","last_change":"2024-09-20T22:15:52.087940+0000","last_active":"2024-09-20T22:16:34.845331+0000","last_peered":"2024-09-20T22:16:34.845331+0000","last_clean":"2024-09-20T22:16:34.845331+0000","last_became_active":"2024-09-20T22:15:50.622954+0000","last_became_peered":"2024-09-20T22:15:50.622954+0000","last_unstale":"2024-09-20T22:16:34.845331+0000","last_undegraded":"2024-09-20T22:16:34.845331+0000","last_fullsized":"2024-09-20T22:16:34.845331+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:52.087707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:52.087707+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:07.776471+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":793,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:37.937521+0000","last_change":"2024-09-20T22:16:23.783338+0000","last_active":"2024-09-20T22:16:37.937521+0000","last_peered":"2024-09-20T22:16:37.937521+0000","last_clean":"2024-09-20T22:16:37.937521+0000","last_became_active":"2024-09-20T22:12:52.516989+0000","last_became_peered":"2024-09-20T22:12:52.516989+0000","last_unstale":"2024-09-20T22:16:37.937521+0000","last_undegraded":"2024-09-20T22:16:37.937521+0000","last_fullsized":"2024-09-20T22:16:37.937521+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:23.783127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:23.783127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:27.538492+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"328'12","reported_seq":793,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.613304+0000","last_change":"2024-09-20T22:16:22.641814+0000","last_active":"2024-09-20T22:16:36.613304+0000","last_peered":"2024-09-20T22:16:36.613304+0000","last_clean":"2024-09-20T22:16:36.613304+0000","last_became_active":"2024-09-20T22:10:48.922561+0000","last_became_peered":"2024-09-20T22:10:48.922561+0000","last_unstale":"2024-09-20T22:16:36.613304+0000","last_undegraded":"2024-09-20T22:16:36.613304+0000","last_fullsized":"2024-09-20T22:16:36.613304+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'12","last_scrub_stamp":"2024-09-20T22:16:22.641558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:22.641558+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:50.501207+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":277,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.847693+0000","last_change":"2024-09-20T22:15:54.065141+0000","last_active":"2024-09-20T22:16:34.847693+0000","last_peered":"2024-09-20T22:16:34.847693+0000","last_clean":"2024-09-20T22:16:34.847693+0000","last_became_active":"2024-09-20T22:11:30.487322+0000","last_became_peered":"2024-09-20T22:11:30.487322+0000","last_unstale":"2024-09-20T22:16:34.847693+0000","last_undegraded":"2024-09-20T22:16:34.847693+0000","last_fullsized":"2024-09-20T22:16:34.847693+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:54.065026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:54.065026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:01.465493+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":722,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842936+0000","last_change":"2024-09-20T22:14:54.359687+0000","last_active":"2024-09-20T22:16:34.842936+0000","last_peered":"2024-09-20T22:16:34.842936+0000","last_clean":"2024-09-20T22:16:34.842936+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:16:34.842936+0000","last_undegraded":"2024-09-20T22:16:34.842936+0000","last_fullsized":"2024-09-20T22:16:34.842936+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:54.359586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:54.359586+0000","last_clean_scrub_stamp":"2024-09-20T22:14:54.359586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"75'128","reported_seq":759,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:25.773504+0000","last_change":"2024-09-20T22:15:52.362195+0000","last_active":"2024-09-20T22:16:25.773504+0000","last_peered":"2024-09-20T22:16:25.773504+0000","last_clean":"2024-09-20T22:16:25.773504+0000","last_became_active":"2024-09-20T22:15:50.615607+0000","last_became_peered":"2024-09-20T22:15:50.615607+0000","last_unstale":"2024-09-20T22:16:25.773504+0000","last_undegraded":"2024-09-20T22:16:25.773504+0000","last_fullsized":"2024-09-20T22:16:25.773504+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:52.362111+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:15:52.362111+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:59.380973+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"328'15","reported_seq":215,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.588440+0000","last_change":"2024-09-20T22:15:59.144712+0000","last_active":"2024-09-20T22:16:36.588440+0000","last_peered":"2024-09-20T22:16:36.588440+0000","last_clean":"2024-09-20T22:16:36.588440+0000","last_became_active":"2024-09-20T22:12:52.527106+0000","last_became_peered":"2024-09-20T22:12:52.527106+0000","last_unstale":"2024-09-20T22:16:36.588440+0000","last_undegraded":"2024-09-20T22:16:36.588440+0000","last_fullsized":"2024-09-20T22:16:36.588440+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:15:59.144584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:59.144584+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:02.167431+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":742,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842831+0000","last_change":"2024-09-20T22:15:28.374528+0000","last_active":"2024-09-20T22:16:34.842831+0000","last_peered":"2024-09-20T22:16:34.842831+0000","last_clean":"2024-09-20T22:16:34.842831+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:16:34.842831+0000","last_undegraded":"2024-09-20T22:16:34.842831+0000","last_fullsized":"2024-09-20T22:16:34.842831+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:28.374414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:28.374414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":378,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:25.775268+0000","last_change":"2024-09-20T22:16:17.270484+0000","last_active":"2024-09-20T22:16:25.775268+0000","last_peered":"2024-09-20T22:16:25.775268+0000","last_clean":"2024-09-20T22:16:25.775268+0000","last_became_active":"2024-09-20T22:12:33.545506+0000","last_became_peered":"2024-09-20T22:12:33.545506+0000","last_unstale":"2024-09-20T22:16:25.775268+0000","last_undegraded":"2024-09-20T22:16:25.775268+0000","last_fullsized":"2024-09-20T22:16:25.775268+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:17.270376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_clean_scrub_stamp":"2024-09-20T22:16:17.270376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:17.773205+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":761,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842159+0000","last_change":"2024-09-20T22:16:26.862388+0000","last_active":"2024-09-20T22:16:34.842159+0000","last_peered":"2024-09-20T22:16:34.842159+0000","last_clean":"2024-09-20T22:16:34.842159+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:16:34.842159+0000","last_undegraded":"2024-09-20T22:16:34.842159+0000","last_fullsized":"2024-09-20T22:16:34.842159+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:26.862279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:26.862279+0000","last_clean_scrub_stamp":"2024-09-20T22:16:26.862279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:33.809107+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"75'128","reported_seq":625,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.844764+0000","last_change":"2024-09-20T22:15:51.068931+0000","last_active":"2024-09-20T22:16:34.844764+0000","last_peered":"2024-09-20T22:16:34.844764+0000","last_clean":"2024-09-20T22:16:34.844764+0000","last_became_active":"2024-09-20T22:15:50.574865+0000","last_became_peered":"2024-09-20T22:15:50.574865+0000","last_unstale":"2024-09-20T22:16:34.844764+0000","last_undegraded":"2024-09-20T22:16:34.844764+0000","last_fullsized":"2024-09-20T22:16:34.844764+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:51.068807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:51.068807+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:04.679167+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"328'15","reported_seq":296,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.846319+0000","last_change":"2024-09-20T22:15:57.097701+0000","last_active":"2024-09-20T22:16:34.846319+0000","last_peered":"2024-09-20T22:16:34.846319+0000","last_clean":"2024-09-20T22:16:34.846319+0000","last_became_active":"2024-09-20T22:11:30.493335+0000","last_became_peered":"2024-09-20T22:11:30.493335+0000","last_unstale":"2024-09-20T22:16:34.846319+0000","last_undegraded":"2024-09-20T22:16:34.846319+0000","last_fullsized":"2024-09-20T22:16:34.846319+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:15:57.097563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.097563+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:04.507802+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":204,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.587499+0000","last_change":"2024-09-20T22:16:27.938805+0000","last_active":"2024-09-20T22:16:36.587499+0000","last_peered":"2024-09-20T22:16:36.587499+0000","last_clean":"2024-09-20T22:16:36.587499+0000","last_became_active":"2024-09-20T22:12:52.524149+0000","last_became_peered":"2024-09-20T22:12:52.524149+0000","last_unstale":"2024-09-20T22:16:36.587499+0000","last_undegraded":"2024-09-20T22:16:36.587499+0000","last_fullsized":"2024-09-20T22:16:36.587499+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:27.938677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:56.538825+0000","last_clean_scrub_stamp":"2024-09-20T22:16:27.938677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:44.662705+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"315'11","reported_seq":287,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.843054+0000","last_change":"2024-09-20T22:15:16.396532+0000","last_active":"2024-09-20T22:16:34.843054+0000","last_peered":"2024-09-20T22:16:34.843054+0000","last_clean":"2024-09-20T22:16:34.843054+0000","last_became_active":"2024-09-20T22:12:33.544098+0000","last_became_peered":"2024-09-20T22:12:33.544098+0000","last_unstale":"2024-09-20T22:16:34.843054+0000","last_undegraded":"2024-09-20T22:16:34.843054+0000","last_fullsized":"2024-09-20T22:16:34.843054+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:15:16.396454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:16.396454+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":742,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.840557+0000","last_change":"2024-09-20T22:15:30.093649+0000","last_active":"2024-09-20T22:16:34.840557+0000","last_peered":"2024-09-20T22:16:34.840557+0000","last_clean":"2024-09-20T22:16:34.840557+0000","last_became_active":"2024-09-20T22:11:30.484731+0000","last_became_peered":"2024-09-20T22:11:30.484731+0000","last_unstale":"2024-09-20T22:16:34.840557+0000","last_undegraded":"2024-09-20T22:16:34.840557+0000","last_fullsized":"2024-09-20T22:16:34.840557+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:30.093548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:30.093548+0000","last_clean_scrub_stamp":"2024-09-20T22:15:30.093548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:55.627047+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"75'128","reported_seq":643,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.587927+0000","last_change":"2024-09-20T22:15:51.077313+0000","last_active":"2024-09-20T22:16:36.587927+0000","last_peered":"2024-09-20T22:16:36.587927+0000","last_clean":"2024-09-20T22:16:36.587927+0000","last_became_active":"2024-09-20T22:15:50.612544+0000","last_became_peered":"2024-09-20T22:15:50.612544+0000","last_unstale":"2024-09-20T22:16:36.587927+0000","last_undegraded":"2024-09-20T22:16:36.587927+0000","last_fullsized":"2024-09-20T22:16:36.587927+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:51.077187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:51.077187+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:18.801109+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"328'20","reported_seq":221,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:25.773268+0000","last_change":"2024-09-20T22:13:49.905238+0000","last_active":"2024-09-20T22:16:25.773268+0000","last_peered":"2024-09-20T22:16:25.773268+0000","last_clean":"2024-09-20T22:16:25.773268+0000","last_became_active":"2024-09-20T22:12:44.631986+0000","last_became_peered":"2024-09-20T22:12:44.631986+0000","last_unstale":"2024-09-20T22:16:25.773268+0000","last_undegraded":"2024-09-20T22:16:25.773268+0000","last_fullsized":"2024-09-20T22:16:25.773268+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:13:49.905115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:49.905115+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":405,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":742,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842686+0000","last_change":"2024-09-20T22:16:31.755794+0000","last_active":"2024-09-20T22:16:34.842686+0000","last_peered":"2024-09-20T22:16:34.842686+0000","last_clean":"2024-09-20T22:16:34.842686+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:16:34.842686+0000","last_undegraded":"2024-09-20T22:16:34.842686+0000","last_fullsized":"2024-09-20T22:16:34.842686+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:31.755589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.755589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:47.093311+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"326'14","reported_seq":363,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.846167+0000","last_change":"2024-09-20T22:15:19.186947+0000","last_active":"2024-09-20T22:16:34.846167+0000","last_peered":"2024-09-20T22:16:34.846167+0000","last_clean":"2024-09-20T22:16:34.846167+0000","last_became_active":"2024-09-20T22:12:52.518770+0000","last_became_peered":"2024-09-20T22:12:52.518770+0000","last_unstale":"2024-09-20T22:16:34.846167+0000","last_undegraded":"2024-09-20T22:16:34.846167+0000","last_fullsized":"2024-09-20T22:16:34.846167+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'14","last_scrub_stamp":"2024-09-20T22:15:19.186862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:19.186862+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:43.532138+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":196,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.588248+0000","last_change":"2024-09-20T22:15:22.741319+0000","last_active":"2024-09-20T22:16:36.588248+0000","last_peered":"2024-09-20T22:16:36.588248+0000","last_clean":"2024-09-20T22:16:36.588248+0000","last_became_active":"2024-09-20T22:12:52.523781+0000","last_became_peered":"2024-09-20T22:12:52.523781+0000","last_unstale":"2024-09-20T22:16:36.588248+0000","last_undegraded":"2024-09-20T22:16:36.588248+0000","last_fullsized":"2024-09-20T22:16:36.588248+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:22.741214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:22.741214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"328'19","reported_seq":798,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.841112+0000","last_change":"2024-09-20T22:16:28.533330+0000","last_active":"2024-09-20T22:16:34.841112+0000","last_peered":"2024-09-20T22:16:34.841112+0000","last_clean":"2024-09-20T22:16:34.841112+0000","last_became_active":"2024-09-20T22:08:22.139581+0000","last_became_peered":"2024-09-20T22:08:22.139581+0000","last_unstale":"2024-09-20T22:16:34.841112+0000","last_undegraded":"2024-09-20T22:16:34.841112+0000","last_fullsized":"2024-09-20T22:16:34.841112+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:16:28.533201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:28.533201+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:50.375092+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":742,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.843452+0000","last_change":"2024-09-20T22:15:20.031722+0000","last_active":"2024-09-20T22:16:34.843452+0000","last_peered":"2024-09-20T22:16:34.843452+0000","last_clean":"2024-09-20T22:16:34.843452+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:16:34.843452+0000","last_undegraded":"2024-09-20T22:16:34.843452+0000","last_fullsized":"2024-09-20T22:16:34.843452+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.031625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.031625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:36.647532+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":577,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:32.216034+0000","last_change":"2024-09-20T22:16:32.216071+0000","last_active":"2024-09-20T22:16:32.216034+0000","last_peered":"2024-09-20T22:16:32.216034+0000","last_clean":"2024-09-20T22:16:32.216034+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:16:32.216034+0000","last_undegraded":"2024-09-20T22:16:32.216034+0000","last_fullsized":"2024-09-20T22:16:32.216034+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:32.215758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.215758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:00.249999+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":580,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:25.773995+0000","last_change":"2024-09-20T22:14:18.440962+0000","last_active":"2024-09-20T22:16:25.773995+0000","last_peered":"2024-09-20T22:16:25.773995+0000","last_clean":"2024-09-20T22:16:25.773995+0000","last_became_active":"2024-09-20T22:12:52.521497+0000","last_became_peered":"2024-09-20T22:12:52.521497+0000","last_unstale":"2024-09-20T22:16:25.773995+0000","last_undegraded":"2024-09-20T22:16:25.773995+0000","last_fullsized":"2024-09-20T22:16:25.773995+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:18.438451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:18.438451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"328'19","reported_seq":654,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:25.778107+0000","last_change":"2024-09-20T22:14:12.410224+0000","last_active":"2024-09-20T22:16:25.778107+0000","last_peered":"2024-09-20T22:16:25.778107+0000","last_clean":"2024-09-20T22:16:25.778107+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:16:25.778107+0000","last_undegraded":"2024-09-20T22:16:25.778107+0000","last_fullsized":"2024-09-20T22:16:25.778107+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:14:12.410107+0000","last_deep_scrub":"291'13","last_deep_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_clean_scrub_stamp":"2024-09-20T22:14:12.410107+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":339,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842894+0000","last_change":"2024-09-20T22:15:54.609470+0000","last_active":"2024-09-20T22:16:34.842894+0000","last_peered":"2024-09-20T22:16:34.842894+0000","last_clean":"2024-09-20T22:16:34.842894+0000","last_became_active":"2024-09-20T22:10:30.655502+0000","last_became_peered":"2024-09-20T22:10:30.655502+0000","last_unstale":"2024-09-20T22:16:34.842894+0000","last_undegraded":"2024-09-20T22:16:34.842894+0000","last_fullsized":"2024-09-20T22:16:34.842894+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:54.609342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:54.609342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:54.717401+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":196,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.587440+0000","last_change":"2024-09-20T22:15:33.868714+0000","last_active":"2024-09-20T22:16:36.587440+0000","last_peered":"2024-09-20T22:16:36.587440+0000","last_clean":"2024-09-20T22:16:36.587440+0000","last_became_active":"2024-09-20T22:12:52.516953+0000","last_became_peered":"2024-09-20T22:12:52.516953+0000","last_unstale":"2024-09-20T22:16:36.587440+0000","last_undegraded":"2024-09-20T22:16:36.587440+0000","last_fullsized":"2024-09-20T22:16:36.587440+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:33.868591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:33.868591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:54.347076+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":196,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.589917+0000","last_change":"2024-09-20T22:15:20.820202+0000","last_active":"2024-09-20T22:16:36.589917+0000","last_peered":"2024-09-20T22:16:36.589917+0000","last_clean":"2024-09-20T22:16:36.589917+0000","last_became_active":"2024-09-20T22:12:52.514247+0000","last_became_peered":"2024-09-20T22:12:52.514247+0000","last_unstale":"2024-09-20T22:16:36.589917+0000","last_undegraded":"2024-09-20T22:16:36.589917+0000","last_fullsized":"2024-09-20T22:16:36.589917+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.819992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.819992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:39.289842+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"75'128","reported_seq":790,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.841445+0000","last_change":"2024-09-20T22:15:50.611775+0000","last_active":"2024-09-20T22:16:34.841445+0000","last_peered":"2024-09-20T22:16:34.841445+0000","last_clean":"2024-09-20T22:16:34.841445+0000","last_became_active":"2024-09-20T22:15:50.611421+0000","last_became_peered":"2024-09-20T22:15:50.611421+0000","last_unstale":"2024-09-20T22:16:34.841445+0000","last_undegraded":"2024-09-20T22:16:34.841445+0000","last_fullsized":"2024-09-20T22:16:34.841445+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:45.920357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:45.920357+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:46.078893+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"328'19","reported_seq":667,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842625+0000","last_change":"2024-09-20T22:16:21.625991+0000","last_active":"2024-09-20T22:16:34.842625+0000","last_peered":"2024-09-20T22:16:34.842625+0000","last_clean":"2024-09-20T22:16:34.842625+0000","last_became_active":"2024-09-20T22:12:33.545199+0000","last_became_peered":"2024-09-20T22:12:33.545199+0000","last_unstale":"2024-09-20T22:16:34.842625+0000","last_undegraded":"2024-09-20T22:16:34.842625+0000","last_fullsized":"2024-09-20T22:16:34.842625+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:16:21.625915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:21.625915+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:47.759718+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":280,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.847615+0000","last_change":"2024-09-20T22:15:27.149844+0000","last_active":"2024-09-20T22:16:34.847615+0000","last_peered":"2024-09-20T22:16:34.847615+0000","last_clean":"2024-09-20T22:16:34.847615+0000","last_became_active":"2024-09-20T22:11:30.493598+0000","last_became_peered":"2024-09-20T22:11:30.493598+0000","last_unstale":"2024-09-20T22:16:34.847615+0000","last_undegraded":"2024-09-20T22:16:34.847615+0000","last_fullsized":"2024-09-20T22:16:34.847615+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:27.149777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.149777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:39.146913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":734,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.842137+0000","last_change":"2024-09-20T22:14:55.338392+0000","last_active":"2024-09-20T22:16:34.842137+0000","last_peered":"2024-09-20T22:16:34.842137+0000","last_clean":"2024-09-20T22:16:34.842137+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:16:34.842137+0000","last_undegraded":"2024-09-20T22:16:34.842137+0000","last_fullsized":"2024-09-20T22:16:34.842137+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:55.338318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:55.338318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":193,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.587647+0000","last_change":"2024-09-20T22:16:20.113927+0000","last_active":"2024-09-20T22:16:36.587647+0000","last_peered":"2024-09-20T22:16:36.587647+0000","last_clean":"2024-09-20T22:16:36.587647+0000","last_became_active":"2024-09-20T22:12:57.883998+0000","last_became_peered":"2024-09-20T22:12:57.883998+0000","last_unstale":"2024-09-20T22:16:36.587647+0000","last_undegraded":"2024-09-20T22:16:36.587647+0000","last_fullsized":"2024-09-20T22:16:36.587647+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:20.113814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:20.113814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:26.993977+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"328'8","reported_seq":760,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.606392+0000","last_change":"2024-09-20T22:15:47.882115+0000","last_active":"2024-09-20T22:16:36.606392+0000","last_peered":"2024-09-20T22:16:36.606392+0000","last_clean":"2024-09-20T22:16:36.606392+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:16:36.606392+0000","last_undegraded":"2024-09-20T22:16:36.606392+0000","last_fullsized":"2024-09-20T22:16:36.606392+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'8","last_scrub_stamp":"2024-09-20T22:15:47.881989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:47.881989+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:00.408284+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":196,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.588478+0000","last_change":"2024-09-20T22:15:50.069823+0000","last_active":"2024-09-20T22:16:36.588478+0000","last_peered":"2024-09-20T22:16:36.588478+0000","last_clean":"2024-09-20T22:16:36.588478+0000","last_became_active":"2024-09-20T22:12:52.517510+0000","last_became_peered":"2024-09-20T22:12:52.517510+0000","last_unstale":"2024-09-20T22:16:36.588478+0000","last_undegraded":"2024-09-20T22:16:36.588478+0000","last_fullsized":"2024-09-20T22:16:36.588478+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:50.069659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:50.069659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:18.119042+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":769,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:37.937174+0000","last_change":"2024-09-20T22:16:37.743248+0000","last_active":"2024-09-20T22:16:37.937174+0000","last_peered":"2024-09-20T22:16:37.937174+0000","last_clean":"2024-09-20T22:16:37.937174+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:16:37.937174+0000","last_undegraded":"2024-09-20T22:16:37.937174+0000","last_fullsized":"2024-09-20T22:16:37.937174+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:37.743137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:37.743137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:04.711878+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":730,"reported_epoch":407,"state":"active+clean","last_fresh":"2024-09-20T22:16:34.843396+0000","last_change":"2024-09-20T22:15:29.393451+0000","last_active":"2024-09-20T22:16:34.843396+0000","last_peered":"2024-09-20T22:16:34.843396+0000","last_clean":"2024-09-20T22:16:34.843396+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:16:34.843396+0000","last_undegraded":"2024-09-20T22:16:34.843396+0000","last_fullsized":"2024-09-20T22:16:34.843396+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:29.393342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_clean_scrub_stamp":"2024-09-20T22:15:29.393342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":625,"reported_epoch":406,"state":"active+clean","last_fresh":"2024-09-20T22:16:25.773268+0000","last_change":"2024-09-20T22:15:49.314143+0000","last_active":"2024-09-20T22:16:25.773268+0000","last_peered":"2024-09-20T22:16:25.773268+0000","last_clean":"2024-09-20T22:16:25.773268+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:16:25.773268+0000","last_undegraded":"2024-09-20T22:16:25.773268+0000","last_fullsized":"2024-09-20T22:16:25.773268+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:49.314024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:49.314024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:18.675580+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":196,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.590354+0000","last_change":"2024-09-20T22:15:18.828980+0000","last_active":"2024-09-20T22:16:36.590354+0000","last_peered":"2024-09-20T22:16:36.590354+0000","last_clean":"2024-09-20T22:16:36.590354+0000","last_became_active":"2024-09-20T22:12:52.518076+0000","last_became_peered":"2024-09-20T22:12:52.518076+0000","last_unstale":"2024-09-20T22:16:36.590354+0000","last_undegraded":"2024-09-20T22:16:36.590354+0000","last_fullsized":"2024-09-20T22:16:36.590354+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:18.828903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:18.828903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"328'19","reported_seq":790,"reported_epoch":409,"state":"active+clean","last_fresh":"2024-09-20T22:16:37.938260+0000","last_change":"2024-09-20T22:15:57.649394+0000","last_active":"2024-09-20T22:16:37.938260+0000","last_peered":"2024-09-20T22:16:37.938260+0000","last_clean":"2024-09-20T22:16:37.938260+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:16:37.938260+0000","last_undegraded":"2024-09-20T22:16:37.938260+0000","last_fullsized":"2024-09-20T22:16:37.938260+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:15:57.649159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.649159+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:08.717893+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":196,"reported_epoch":408,"state":"active+clean","last_fresh":"2024-09-20T22:16:36.588199+0000","last_change":"2024-09-20T22:15:27.761950+0000","last_active":"2024-09-20T22:16:36.588199+0000","last_peered":"2024-09-20T22:16:36.588199+0000","last_clean":"2024-09-20T22:16:36.588199+0000","last_became_active":"2024-09-20T22:12:52.524425+0000","last_became_peered":"2024-09-20T22:12:52.524425+0000","last_unstale":"2024-09-20T22:16:36.588199+0000","last_undegraded":"2024-09-20T22:16:36.588199+0000","last_fullsized":"2024-09-20T22:16:36.588199+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:27.761843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.761843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:46.763986+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":7350321152,"num_objects":112157,"num_object_clones":0,"num_object_copies":336471,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":112157,"num_write_kb":7178048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":21980577792,"data_stored":21980577792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79057,"ondisk_log_size":79057,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":3300,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":731,"ondisk_log_size":731,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":801,"num_read_kb":796,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2688,"ondisk_log_size":2688,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813136041,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2426368,"kb_used_data":1773228,"kb_used_omap":1,"kb_used_meta":653118,"kb_avail":91318784,"statfs":{"total":95995035648,"available":93510434816,"internally_reserved":0,"allocated":1815785472,"data_stored":1815317669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":668793290},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430379,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3652660,"kb_used_data":3170276,"kb_used_omap":1,"kb_used_meta":482366,"kb_avail":90092492,"statfs":{"total":95995035648,"available":92254711808,"internally_reserved":0,"allocated":3246362624,"data_stored":3245911189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":493943238},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724721,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2901416,"kb_used_data":2238548,"kb_used_omap":1,"kb_used_meta":662846,"kb_avail":90843736,"statfs":{"total":95995035648,"available":93023985664,"internally_reserved":0,"allocated":2292273152,"data_stored":2291828642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":678754757},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986361,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4086464,"kb_used_data":3531820,"kb_used_omap":1,"kb_used_meta":554558,"kb_avail":89658688,"statfs":{"total":95995035648,"available":91810496512,"internally_reserved":0,"allocated":3616583680,"data_stored":3616115166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":567867845},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,3],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":145,"seq":622770258037,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1799748,"kb_used_data":1313332,"kb_used_omap":4,"kb_used_meta":486395,"kb_avail":91945404,"statfs":{"total":95995035648,"available":94152093696,"internally_reserved":0,"allocated":1344851968,"data_stored":1344368173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":498069160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149886,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2927904,"kb_used_data":2233096,"kb_used_omap":1,"kb_used_meta":694782,"kb_avail":90817248,"statfs":{"total":95995035648,"available":92996861952,"internally_reserved":0,"allocated":2286690304,"data_stored":2286202788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":711457223},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411523,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3279080,"kb_used_data":2657876,"kb_used_omap":1,"kb_used_meta":621182,"kb_avail":90466072,"statfs":{"total":95995035648,"available":92637257728,"internally_reserved":0,"allocated":2721665024,"data_stored":2721220957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":636090821},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,3,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738574,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5200264,"kb_used_data":4492980,"kb_used_omap":1,"kb_used_meta":707262,"kb_avail":88544888,"statfs":{"total":95995035648,"available":90669965312,"internally_reserved":0,"allocated":4600811520,"data_stored":4600343381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":724236741},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4610981888,"data_stored":4610981888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1346830336,"data_stored":1346830336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2730426368,"data_stored":2730426368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2285895680,"data_stored":2285895680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3632005120,"data_stored":3632005120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2302541824,"data_stored":2302541824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3255173120,"data_stored":3255173120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1816723456,"data_stored":1816723456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:16:42.725 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:16:42.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:42.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:42.862 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:42.863 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:42.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:42 smithi060 ceph-mon[29694]: pgmap v741: 168 pgs: 168 active+clean; 7.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 853 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-20T22:16:42.883 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:42 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:16:42.884 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:42 smithi060 ceph-mon[40290]: pgmap v741: 168 pgs: 168 active+clean; 7.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 853 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-20T22:16:42.884 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:42 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:16:42.933 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 85 16 119014 118998 87.4794 77.0625 0.0119185 0.0114264 2024-09-20T22:16:42.963 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:42.999 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:42.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:42.999 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:43.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:16:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:16:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:42 smithi067 ceph-mon[33511]: pgmap v741: 168 pgs: 168 active+clean; 7.1 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 853 B/s rd, 71 MiB/s wr, 1.14k op/s 2024-09-20T22:16:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:42 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:16:43.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:43.140 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:43.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:43.141 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:43.242 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:43.254 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T22:16:43.270 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:43.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:43.271 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:43.351 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:43.372 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:43.405 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:43.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:43.454 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:43.455 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:43.555 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:43.590 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:43.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:43.591 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:43.691 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:43.722 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:43.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:43.723 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:43.824 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:43.857 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:43.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:43.858 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:43.933 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 86 16 120171 120155 87.303 72.3125 0.00816532 0.0114492 2024-09-20T22:16:43.958 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:43.989 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:43.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:43.991 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:44.045 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:16:44.091 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:44.124 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:44.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:44.125 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:44.225 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:44.254 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:44.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:44.255 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:44.355 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:44.405 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:44.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:44.406 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:44.506 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:44.545 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:44.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:44.546 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:44.646 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:44.683 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:44.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:44.684 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:44.785 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:44.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:44.835 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:44.835 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:44.933 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 87 16 121233 121217 87.0622 66.375 0.023248 0.0114808 2024-09-20T22:16:44.936 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:44.950 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:44.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:44.993 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:44.993 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:45.094 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:45.124 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:45.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:45.125 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:45.225 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:45.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:45.279 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:45.279 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:45.297 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:16:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:16:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:16:45.320 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:16:45.323 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:16:45.337 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:16:45.380 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:45.419 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:45.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:45.420 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:45.520 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:45.557 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:45.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:45.558 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:45.628 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:45 smithi060 ceph-mon[40290]: pgmap v742: 168 pgs: 168 active+clean; 7.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 667 B/s rd, 70 MiB/s wr, 1.12k op/s 2024-09-20T22:16:45.628 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:45 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2620442161' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:16:45.628 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:45 smithi060 ceph-mon[29694]: pgmap v742: 168 pgs: 168 active+clean; 7.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 667 B/s rd, 70 MiB/s wr, 1.12k op/s 2024-09-20T22:16:45.628 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:45 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2620442161' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:16:45.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:45.692 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:45.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:45.693 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:45.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:45 smithi067 ceph-mon[33511]: pgmap v742: 168 pgs: 168 active+clean; 7.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 667 B/s rd, 70 MiB/s wr, 1.12k op/s 2024-09-20T22:16:45.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:45 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2620442161' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:16:45.794 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:45.833 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:45.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:45.834 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:45.933 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 88 16 122337 122321 86.8569 69 0.0095477 0.0115084 2024-09-20T22:16:45.935 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:45.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:45.974 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:45.974 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:46.075 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:46.117 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:46.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:46.119 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:46.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:46.262 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:46.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:46.263 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:46.364 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:46.404 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:46.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:46.406 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:46.506 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:46.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:46 smithi060 ceph-mon[29694]: from='client.16173 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:16:46.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:46 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:16:46.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:46 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2620442161' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:16:46.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:46 smithi060 ceph-mon[29694]: osdmap e411: 8 total, 8 up, 8 in 2024-09-20T22:16:46.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:46 smithi060 ceph-mon[40290]: from='client.16173 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:16:46.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:46 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:16:46.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:46 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2620442161' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:16:46.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:46 smithi060 ceph-mon[40290]: osdmap e411: 8 total, 8 up, 8 in 2024-09-20T22:16:46.543 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:46.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:46.544 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:46.645 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:46.684 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:46.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:46.685 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:46.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:46 smithi067 ceph-mon[33511]: from='client.16173 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:16:46.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:46 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:16:46.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:46 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2620442161' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:16:46.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:46 smithi067 ceph-mon[33511]: osdmap e411: 8 total, 8 up, 8 in 2024-09-20T22:16:46.786 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:46.825 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:46.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:46.826 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:46.926 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:46.934 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 89 16 123611 123595 86.7756 79.625 0.0104954 0.0115194 2024-09-20T22:16:46.964 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:46.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:46.965 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:47.016 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":742,"stamp":"2024-09-20T22:16:44.256103+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":7735220763,"num_objects":118220,"num_object_clones":0,"num_object_copies":354660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118220,"num_whiteouts":0,"num_read":1940,"num_read_kb":1692,"num_write":118743,"num_write_kb":7553389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":268563,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82655,"ondisk_log_size":82655,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":504,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":27411428,"kb_used_data":22526492,"kb_used_omap":15,"kb_used_meta":4884720,"kb_avail":722549788,"statfs":{"total":767960285184,"available":739890982912,"internally_reserved":0,"allocated":23067127808,"data_stored":23063326417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":5001954067},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,3,6,5],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":564264960,"num_objects":8610,"num_object_clones":0,"num_object_copies":25830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8610,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":8610,"num_write_kb":551040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.675415"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":200,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.933757+0000","last_change":"2024-09-20T22:16:31.993232+0000","last_active":"2024-09-20T22:16:39.933757+0000","last_peered":"2024-09-20T22:16:39.933757+0000","last_clean":"2024-09-20T22:16:39.933757+0000","last_became_active":"2024-09-20T22:12:52.515921+0000","last_became_peered":"2024-09-20T22:12:52.515921+0000","last_unstale":"2024-09-20T22:16:39.933757+0000","last_undegraded":"2024-09-20T22:16:39.933757+0000","last_fullsized":"2024-09-20T22:16:39.933757+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:31.993104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:31.993104+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.993104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:36.968800+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"327'19","reported_seq":790,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.930195+0000","last_change":"2024-09-20T22:16:29.508726+0000","last_active":"2024-09-20T22:16:39.930195+0000","last_peered":"2024-09-20T22:16:39.930195+0000","last_clean":"2024-09-20T22:16:39.930195+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:16:39.930195+0000","last_undegraded":"2024-09-20T22:16:39.930195+0000","last_fullsized":"2024-09-20T22:16:39.930195+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"327'19","last_scrub_stamp":"2024-09-20T22:16:29.508607+0000","last_deep_scrub":"252'13","last_deep_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_clean_scrub_stamp":"2024-09-20T22:16:29.508607+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:49.145180+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":628,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.933956+0000","last_change":"2024-09-20T22:14:15.391149+0000","last_active":"2024-09-20T22:16:39.933956+0000","last_peered":"2024-09-20T22:16:39.933956+0000","last_clean":"2024-09-20T22:16:39.933956+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:16:39.933956+0000","last_undegraded":"2024-09-20T22:16:39.933956+0000","last_fullsized":"2024-09-20T22:16:39.933956+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:15.391049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:15.391049+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:43.233796+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":757,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.930815+0000","last_change":"2024-09-20T22:16:23.438091+0000","last_active":"2024-09-20T22:16:39.930815+0000","last_peered":"2024-09-20T22:16:39.930815+0000","last_clean":"2024-09-20T22:16:39.930815+0000","last_became_active":"2024-09-20T22:11:30.485735+0000","last_became_peered":"2024-09-20T22:11:30.485735+0000","last_unstale":"2024-09-20T22:16:39.930815+0000","last_undegraded":"2024-09-20T22:16:39.930815+0000","last_fullsized":"2024-09-20T22:16:39.930815+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:23.437959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:23.437959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:36.548769+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":744,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.942115+0000","last_change":"2024-09-20T22:16:27.694458+0000","last_active":"2024-09-20T22:16:39.942115+0000","last_peered":"2024-09-20T22:16:39.942115+0000","last_clean":"2024-09-20T22:16:39.942115+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:16:39.942115+0000","last_undegraded":"2024-09-20T22:16:39.942115+0000","last_fullsized":"2024-09-20T22:16:39.942115+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:16:27.694309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:27.694309+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:50.010230+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":228,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.940094+0000","last_change":"2024-09-20T22:16:22.789638+0000","last_active":"2024-09-20T22:16:39.940094+0000","last_peered":"2024-09-20T22:16:39.940094+0000","last_clean":"2024-09-20T22:16:39.940094+0000","last_became_active":"2024-09-20T22:12:33.545100+0000","last_became_peered":"2024-09-20T22:12:33.545100+0000","last_unstale":"2024-09-20T22:16:39.940094+0000","last_undegraded":"2024-09-20T22:16:39.940094+0000","last_fullsized":"2024-09-20T22:16:39.940094+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:22.789569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:22.789569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:52.149562+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"406'34","reported_seq":350,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.941327+0000","last_change":"2024-09-20T22:16:31.033633+0000","last_active":"2024-09-20T22:16:39.941327+0000","last_peered":"2024-09-20T22:16:39.941327+0000","last_clean":"2024-09-20T22:16:39.941327+0000","last_became_active":"2024-09-20T22:12:52.517296+0000","last_became_peered":"2024-09-20T22:12:52.517296+0000","last_unstale":"2024-09-20T22:16:39.941327+0000","last_undegraded":"2024-09-20T22:16:39.941327+0000","last_fullsized":"2024-09-20T22:16:39.941327+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'34","last_scrub_stamp":"2024-09-20T22:16:31.033354+0000","last_deep_scrub":"406'34","last_deep_scrub_stamp":"2024-09-20T22:16:31.033354+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.033354+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:54.712855+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":749,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.930105+0000","last_change":"2024-09-20T22:16:36.461743+0000","last_active":"2024-09-20T22:16:39.930105+0000","last_peered":"2024-09-20T22:16:39.930105+0000","last_clean":"2024-09-20T22:16:39.930105+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:16:39.930105+0000","last_undegraded":"2024-09-20T22:16:39.930105+0000","last_fullsized":"2024-09-20T22:16:39.930105+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:36.461517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:36.461517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:36.828806+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":763,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.939504+0000","last_change":"2024-09-20T22:15:21.178920+0000","last_active":"2024-09-20T22:16:39.939504+0000","last_peered":"2024-09-20T22:16:39.939504+0000","last_clean":"2024-09-20T22:16:39.939504+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:16:39.939504+0000","last_undegraded":"2024-09-20T22:16:39.939504+0000","last_fullsized":"2024-09-20T22:16:39.939504+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:21.178806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:15:21.178806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.752004+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"328'15","reported_seq":809,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.931078+0000","last_change":"2024-09-20T22:15:59.331710+0000","last_active":"2024-09-20T22:16:39.931078+0000","last_peered":"2024-09-20T22:16:39.931078+0000","last_clean":"2024-09-20T22:16:39.931078+0000","last_became_active":"2024-09-20T22:12:52.516894+0000","last_became_peered":"2024-09-20T22:12:52.516894+0000","last_unstale":"2024-09-20T22:16:39.931078+0000","last_undegraded":"2024-09-20T22:16:39.931078+0000","last_fullsized":"2024-09-20T22:16:39.931078+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:15:59.331593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:59.331593+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:13.858540+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":295,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.938986+0000","last_change":"2024-09-20T22:14:54.153118+0000","last_active":"2024-09-20T22:16:39.938986+0000","last_peered":"2024-09-20T22:16:39.938986+0000","last_clean":"2024-09-20T22:16:39.938986+0000","last_became_active":"2024-09-20T22:12:52.518549+0000","last_became_peered":"2024-09-20T22:12:52.518549+0000","last_unstale":"2024-09-20T22:16:39.938986+0000","last_undegraded":"2024-09-20T22:16:39.938986+0000","last_fullsized":"2024-09-20T22:16:39.938986+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:54.152717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:54.152717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.751929+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":208,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.932911+0000","last_change":"2024-09-20T22:16:37.935027+0000","last_active":"2024-09-20T22:16:39.932911+0000","last_peered":"2024-09-20T22:16:39.932911+0000","last_clean":"2024-09-20T22:16:39.932911+0000","last_became_active":"2024-09-20T22:12:52.524631+0000","last_became_peered":"2024-09-20T22:12:52.524631+0000","last_unstale":"2024-09-20T22:16:39.932911+0000","last_undegraded":"2024-09-20T22:16:39.932911+0000","last_fullsized":"2024-09-20T22:16:39.932911+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:37.934760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:37.934760+0000","last_clean_scrub_stamp":"2024-09-20T22:16:37.934760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:56.296354+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":768,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.939508+0000","last_change":"2024-09-20T22:15:20.182058+0000","last_active":"2024-09-20T22:16:39.939508+0000","last_peered":"2024-09-20T22:16:39.939508+0000","last_clean":"2024-09-20T22:16:39.939508+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:16:39.939508+0000","last_undegraded":"2024-09-20T22:16:39.939508+0000","last_fullsized":"2024-09-20T22:16:39.939508+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.181942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:30.215263+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.181942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:46.935103+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"328'7","reported_seq":778,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.930530+0000","last_change":"2024-09-20T22:15:31.119172+0000","last_active":"2024-09-20T22:16:39.930530+0000","last_peered":"2024-09-20T22:16:39.930530+0000","last_clean":"2024-09-20T22:16:39.930530+0000","last_became_active":"2024-09-20T22:08:22.144364+0000","last_became_peered":"2024-09-20T22:08:22.144364+0000","last_unstale":"2024-09-20T22:16:39.930530+0000","last_undegraded":"2024-09-20T22:16:39.930530+0000","last_fullsized":"2024-09-20T22:16:39.930530+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'7","last_scrub_stamp":"2024-09-20T22:15:31.119046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:31.119046+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.446818+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":749,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935637+0000","last_change":"2024-09-20T22:14:57.354434+0000","last_active":"2024-09-20T22:16:39.935637+0000","last_peered":"2024-09-20T22:16:39.935637+0000","last_clean":"2024-09-20T22:16:39.935637+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:16:39.935637+0000","last_undegraded":"2024-09-20T22:16:39.935637+0000","last_fullsized":"2024-09-20T22:16:39.935637+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:57.354326+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:57.354326+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":283,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.939561+0000","last_change":"2024-09-20T22:15:00.077644+0000","last_active":"2024-09-20T22:16:39.939561+0000","last_peered":"2024-09-20T22:16:39.939561+0000","last_clean":"2024-09-20T22:16:39.939561+0000","last_became_active":"2024-09-20T22:11:30.487137+0000","last_became_peered":"2024-09-20T22:11:30.487137+0000","last_unstale":"2024-09-20T22:16:39.939561+0000","last_undegraded":"2024-09-20T22:16:39.939561+0000","last_fullsized":"2024-09-20T22:16:39.939561+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:00.077519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:15:00.077519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.751960+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":745,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.931308+0000","last_change":"2024-09-20T22:16:34.472967+0000","last_active":"2024-09-20T22:16:39.931308+0000","last_peered":"2024-09-20T22:16:39.931308+0000","last_clean":"2024-09-20T22:16:39.931308+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:16:39.931308+0000","last_undegraded":"2024-09-20T22:16:39.931308+0000","last_fullsized":"2024-09-20T22:16:39.931308+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:34.472700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:16:34.472700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:51.839167+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":200,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.934823+0000","last_change":"2024-09-20T22:15:24.748006+0000","last_active":"2024-09-20T22:16:39.934823+0000","last_peered":"2024-09-20T22:16:39.934823+0000","last_clean":"2024-09-20T22:16:39.934823+0000","last_became_active":"2024-09-20T22:12:52.526097+0000","last_became_peered":"2024-09-20T22:12:52.526097+0000","last_unstale":"2024-09-20T22:16:39.934823+0000","last_undegraded":"2024-09-20T22:16:39.934823+0000","last_fullsized":"2024-09-20T22:16:39.934823+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.747891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.747891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.914047+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"328'34","reported_seq":347,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.939065+0000","last_change":"2024-09-20T22:15:55.987603+0000","last_active":"2024-09-20T22:16:39.939065+0000","last_peered":"2024-09-20T22:16:39.939065+0000","last_clean":"2024-09-20T22:16:39.939065+0000","last_became_active":"2024-09-20T22:11:12.507311+0000","last_became_peered":"2024-09-20T22:11:12.507311+0000","last_unstale":"2024-09-20T22:16:39.939065+0000","last_undegraded":"2024-09-20T22:16:39.939065+0000","last_fullsized":"2024-09-20T22:16:39.939065+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'34","last_scrub_stamp":"2024-09-20T22:15:55.987386+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:15:55.987386+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:17.614237+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":597,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935953+0000","last_change":"2024-09-20T22:16:23.258053+0000","last_active":"2024-09-20T22:16:39.935953+0000","last_peered":"2024-09-20T22:16:39.935953+0000","last_clean":"2024-09-20T22:16:39.935953+0000","last_became_active":"2024-09-20T22:12:52.520985+0000","last_became_peered":"2024-09-20T22:12:52.520985+0000","last_unstale":"2024-09-20T22:16:39.935953+0000","last_undegraded":"2024-09-20T22:16:39.935953+0000","last_fullsized":"2024-09-20T22:16:39.935953+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:23.257937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:11.053276+0000","last_clean_scrub_stamp":"2024-09-20T22:16:23.257937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:33.735295+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":697,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.938522+0000","last_change":"2024-09-20T22:16:16.771843+0000","last_active":"2024-09-20T22:16:39.938522+0000","last_peered":"2024-09-20T22:16:39.938522+0000","last_clean":"2024-09-20T22:16:39.938522+0000","last_became_active":"2024-09-20T22:12:21.521433+0000","last_became_peered":"2024-09-20T22:12:21.521433+0000","last_unstale":"2024-09-20T22:16:39.938522+0000","last_undegraded":"2024-09-20T22:16:39.938522+0000","last_fullsized":"2024-09-20T22:16:39.938522+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:16.771744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:16.771744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:43.671506+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":697,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.940791+0000","last_change":"2024-09-20T22:15:58.078205+0000","last_active":"2024-09-20T22:16:39.940791+0000","last_peered":"2024-09-20T22:16:39.940791+0000","last_clean":"2024-09-20T22:16:39.940791+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:16:39.940791+0000","last_undegraded":"2024-09-20T22:16:39.940791+0000","last_fullsized":"2024-09-20T22:16:39.940791+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:58.078099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:58.078099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:15.885682+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"328'23","reported_seq":662,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.936214+0000","last_change":"2024-09-20T22:15:17.533607+0000","last_active":"2024-09-20T22:16:39.936214+0000","last_peered":"2024-09-20T22:16:39.936214+0000","last_clean":"2024-09-20T22:16:39.936214+0000","last_became_active":"2024-09-20T22:12:21.517147+0000","last_became_peered":"2024-09-20T22:12:21.517147+0000","last_unstale":"2024-09-20T22:16:39.936214+0000","last_undegraded":"2024-09-20T22:16:39.936214+0000","last_fullsized":"2024-09-20T22:16:39.936214+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'23","last_scrub_stamp":"2024-09-20T22:15:17.533453+0000","last_deep_scrub":"250'17","last_deep_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_clean_scrub_stamp":"2024-09-20T22:15:17.533453+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":740,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.931885+0000","last_change":"2024-09-20T22:15:33.188818+0000","last_active":"2024-09-20T22:16:39.931885+0000","last_peered":"2024-09-20T22:16:39.931885+0000","last_clean":"2024-09-20T22:16:39.931885+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:16:39.931885+0000","last_undegraded":"2024-09-20T22:16:39.931885+0000","last_fullsized":"2024-09-20T22:16:39.931885+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:33.188747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:33.188747+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:40.715215+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":738,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.938162+0000","last_change":"2024-09-20T22:15:57.632871+0000","last_active":"2024-09-20T22:16:39.938162+0000","last_peered":"2024-09-20T22:16:39.938162+0000","last_clean":"2024-09-20T22:16:39.938162+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:16:39.938162+0000","last_undegraded":"2024-09-20T22:16:39.938162+0000","last_fullsized":"2024-09-20T22:16:39.938162+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:15:57.632600+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.632600+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:14.194144+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"289'2","reported_seq":588,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.934042+0000","last_change":"2024-09-20T22:14:22.436454+0000","last_active":"2024-09-20T22:16:39.934042+0000","last_peered":"2024-09-20T22:16:39.934042+0000","last_clean":"2024-09-20T22:16:39.934042+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:16:39.934042+0000","last_undegraded":"2024-09-20T22:16:39.934042+0000","last_fullsized":"2024-09-20T22:16:39.934042+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-20T22:14:22.436308+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_clean_scrub_stamp":"2024-09-20T22:14:22.436308+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:43.233826+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"328'20","reported_seq":227,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.934806+0000","last_change":"2024-09-20T22:15:57.132570+0000","last_active":"2024-09-20T22:16:39.934806+0000","last_peered":"2024-09-20T22:16:39.934806+0000","last_clean":"2024-09-20T22:16:39.934806+0000","last_became_active":"2024-09-20T22:12:52.526803+0000","last_became_peered":"2024-09-20T22:12:52.526803+0000","last_unstale":"2024-09-20T22:16:39.934806+0000","last_undegraded":"2024-09-20T22:16:39.934806+0000","last_fullsized":"2024-09-20T22:16:39.934806+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:15:57.132435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.132435+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:18.800340+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"291'11","reported_seq":963,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.938784+0000","last_change":"2024-09-20T22:16:34.733980+0000","last_active":"2024-09-20T22:16:39.938784+0000","last_peered":"2024-09-20T22:16:39.938784+0000","last_clean":"2024-09-20T22:16:39.938784+0000","last_became_active":"2024-09-20T22:11:40.601048+0000","last_became_peered":"2024-09-20T22:11:40.601048+0000","last_unstale":"2024-09-20T22:16:39.938784+0000","last_undegraded":"2024-09-20T22:16:39.938784+0000","last_fullsized":"2024-09-20T22:16:39.938784+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'11","last_scrub_stamp":"2024-09-20T22:16:34.733870+0000","last_deep_scrub":"291'11","last_deep_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_clean_scrub_stamp":"2024-09-20T22:16:34.733870+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:58.907961+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"410'7242","reported_seq":7398,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:40.879715+0000","last_change":"2024-09-20T22:16:24.149762+0000","last_active":"2024-09-20T22:16:40.879715+0000","last_peered":"2024-09-20T22:16:40.879715+0000","last_clean":"2024-09-20T22:16:40.879715+0000","last_became_active":"2024-09-20T22:15:08.927535+0000","last_became_peered":"2024-09-20T22:15:08.927535+0000","last_unstale":"2024-09-20T22:16:40.879715+0000","last_undegraded":"2024-09-20T22:16:40.879715+0000","last_fullsized":"2024-09-20T22:16:40.879715+0000","mapping_epoch":370,"log_start":"381'3000","ondisk_log_start":"381'3000","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"405'6096","last_scrub_stamp":"2024-09-20T22:16:24.146029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:24.146029+0000","objects_scrubbed":6025,"log_size":4242,"log_dups_size":0,"ondisk_log_size":4242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.643132+0000","scrub_duration":1743,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":474611712,"num_objects":7242,"num_object_clones":0,"num_object_copies":21726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7242,"num_write_kb":463488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":747,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.940258+0000","last_change":"2024-09-20T22:13:08.407055+0000","last_active":"2024-09-20T22:16:39.940258+0000","last_peered":"2024-09-20T22:16:39.940258+0000","last_clean":"2024-09-20T22:16:39.940258+0000","last_became_active":"2024-09-20T22:08:48.452786+0000","last_became_peered":"2024-09-20T22:08:48.452786+0000","last_unstale":"2024-09-20T22:16:39.940258+0000","last_undegraded":"2024-09-20T22:16:39.940258+0000","last_fullsized":"2024-09-20T22:16:39.940258+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:13:08.406883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:13:08.406883+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.751778+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"328'27","reported_seq":540,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.940064+0000","last_change":"2024-09-20T22:15:50.925776+0000","last_active":"2024-09-20T22:16:39.940064+0000","last_peered":"2024-09-20T22:16:39.940064+0000","last_clean":"2024-09-20T22:16:39.940064+0000","last_became_active":"2024-09-20T22:08:22.145408+0000","last_became_peered":"2024-09-20T22:08:22.145408+0000","last_unstale":"2024-09-20T22:16:39.940064+0000","last_undegraded":"2024-09-20T22:16:39.940064+0000","last_fullsized":"2024-09-20T22:16:39.940064+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:15:50.925654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:50.925654+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:54.745209+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":765,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.945039+0000","last_change":"2024-09-20T22:16:31.637874+0000","last_active":"2024-09-20T22:16:39.945039+0000","last_peered":"2024-09-20T22:16:39.945039+0000","last_clean":"2024-09-20T22:16:39.945039+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:16:39.945039+0000","last_undegraded":"2024-09-20T22:16:39.945039+0000","last_fullsized":"2024-09-20T22:16:39.945039+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:31.637766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:28.020341+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.637766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:44.980512+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"326'13","reported_seq":985,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.931391+0000","last_change":"2024-09-20T22:16:15.367687+0000","last_active":"2024-09-20T22:16:39.931391+0000","last_peered":"2024-09-20T22:16:39.931391+0000","last_clean":"2024-09-20T22:16:39.931391+0000","last_became_active":"2024-09-20T22:12:52.517588+0000","last_became_peered":"2024-09-20T22:12:52.517588+0000","last_unstale":"2024-09-20T22:16:39.931391+0000","last_undegraded":"2024-09-20T22:16:39.931391+0000","last_fullsized":"2024-09-20T22:16:39.931391+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'13","last_scrub_stamp":"2024-09-20T22:16:15.367449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:15.367449+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:32.822489+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":295,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.939014+0000","last_change":"2024-09-20T22:15:33.082489+0000","last_active":"2024-09-20T22:16:39.939014+0000","last_peered":"2024-09-20T22:16:39.939014+0000","last_clean":"2024-09-20T22:16:39.939014+0000","last_became_active":"2024-09-20T22:12:52.517826+0000","last_became_peered":"2024-09-20T22:12:52.517826+0000","last_unstale":"2024-09-20T22:16:39.939014+0000","last_undegraded":"2024-09-20T22:16:39.939014+0000","last_fullsized":"2024-09-20T22:16:39.939014+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:33.082368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:33.082368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:58.107873+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"410'7148","reported_seq":7312,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:40.862939+0000","last_change":"2024-09-20T22:16:34.386928+0000","last_active":"2024-09-20T22:16:40.862939+0000","last_peered":"2024-09-20T22:16:40.862939+0000","last_clean":"2024-09-20T22:16:40.862939+0000","last_became_active":"2024-09-20T22:15:08.928064+0000","last_became_peered":"2024-09-20T22:15:08.928064+0000","last_unstale":"2024-09-20T22:16:40.862939+0000","last_undegraded":"2024-09-20T22:16:40.862939+0000","last_fullsized":"2024-09-20T22:16:40.862939+0000","mapping_epoch":370,"log_start":"381'2900","ondisk_log_start":"381'2900","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6697","last_scrub_stamp":"2024-09-20T22:16:34.386672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:34.386672+0000","objects_scrubbed":6630,"log_size":4248,"log_dups_size":0,"ondisk_log_size":4248,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:49.633430+0000","scrub_duration":1869,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":468451328,"num_objects":7148,"num_object_clones":0,"num_object_copies":21444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7148,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7148,"num_write_kb":457472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":760,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.931639+0000","last_change":"2024-09-20T22:15:46.295600+0000","last_active":"2024-09-20T22:16:39.931639+0000","last_peered":"2024-09-20T22:16:39.931639+0000","last_clean":"2024-09-20T22:16:39.931639+0000","last_became_active":"2024-09-20T22:15:45.627770+0000","last_became_peered":"2024-09-20T22:15:45.627770+0000","last_unstale":"2024-09-20T22:16:39.931639+0000","last_undegraded":"2024-09-20T22:16:39.931639+0000","last_fullsized":"2024-09-20T22:16:39.931639+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:46.295352+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:15:46.295352+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:11.775763+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"328'30","reported_seq":761,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.940373+0000","last_change":"2024-09-20T22:16:16.609501+0000","last_active":"2024-09-20T22:16:39.940373+0000","last_peered":"2024-09-20T22:16:39.940373+0000","last_clean":"2024-09-20T22:16:39.940373+0000","last_became_active":"2024-09-20T22:12:52.526867+0000","last_became_peered":"2024-09-20T22:12:52.526867+0000","last_unstale":"2024-09-20T22:16:39.940373+0000","last_undegraded":"2024-09-20T22:16:39.940373+0000","last_fullsized":"2024-09-20T22:16:39.940373+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'30","last_scrub_stamp":"2024-09-20T22:16:16.609377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:16.609377+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:45.896136+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":242,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.937990+0000","last_change":"2024-09-20T22:16:28.701889+0000","last_active":"2024-09-20T22:16:39.937990+0000","last_peered":"2024-09-20T22:16:39.937990+0000","last_clean":"2024-09-20T22:16:39.937990+0000","last_became_active":"2024-09-20T22:12:52.528947+0000","last_became_peered":"2024-09-20T22:12:52.528947+0000","last_unstale":"2024-09-20T22:16:39.937990+0000","last_undegraded":"2024-09-20T22:16:39.937990+0000","last_fullsized":"2024-09-20T22:16:39.937990+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:28.701780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:28.701780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:46.729610+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":761,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.930193+0000","last_change":"2024-09-20T22:15:56.322206+0000","last_active":"2024-09-20T22:16:39.930193+0000","last_peered":"2024-09-20T22:16:39.930193+0000","last_clean":"2024-09-20T22:16:39.930193+0000","last_became_active":"2024-09-20T22:10:48.919949+0000","last_became_peered":"2024-09-20T22:10:48.919949+0000","last_unstale":"2024-09-20T22:16:39.930193+0000","last_undegraded":"2024-09-20T22:16:39.930193+0000","last_fullsized":"2024-09-20T22:16:39.930193+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:56.321409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:56.321409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:24.041893+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":745,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.930996+0000","last_change":"2024-09-20T22:15:58.309888+0000","last_active":"2024-09-20T22:16:39.930996+0000","last_peered":"2024-09-20T22:16:39.930996+0000","last_clean":"2024-09-20T22:16:39.930996+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:16:39.930996+0000","last_undegraded":"2024-09-20T22:16:39.930996+0000","last_fullsized":"2024-09-20T22:16:39.930996+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:58.309764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:58.309764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:14.798510+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"410'7335","reported_seq":7425,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:43.024527+0000","last_change":"2024-09-20T22:15:09.088949+0000","last_active":"2024-09-20T22:16:43.024527+0000","last_peered":"2024-09-20T22:16:43.024527+0000","last_clean":"2024-09-20T22:16:43.024527+0000","last_became_active":"2024-09-20T22:15:09.088580+0000","last_became_peered":"2024-09-20T22:15:09.088580+0000","last_unstale":"2024-09-20T22:16:43.024527+0000","last_undegraded":"2024-09-20T22:16:43.024527+0000","last_fullsized":"2024-09-20T22:16:43.024527+0000","mapping_epoch":370,"log_start":"379'2400","ondisk_log_start":"379'2400","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":4935,"log_dups_size":0,"ondisk_log_size":4935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:45.655811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":480706560,"num_objects":7335,"num_object_clones":0,"num_object_copies":22005,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7335,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7335,"num_write_kb":469440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":794,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.942417+0000","last_change":"2024-09-20T22:15:46.874158+0000","last_active":"2024-09-20T22:16:39.942417+0000","last_peered":"2024-09-20T22:16:39.942417+0000","last_clean":"2024-09-20T22:16:39.942417+0000","last_became_active":"2024-09-20T22:15:45.642671+0000","last_became_peered":"2024-09-20T22:15:45.642671+0000","last_unstale":"2024-09-20T22:16:39.942417+0000","last_undegraded":"2024-09-20T22:16:39.942417+0000","last_fullsized":"2024-09-20T22:16:39.942417+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:46.874000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:46.874000+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:16.327234+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"328'28","reported_seq":795,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.930511+0000","last_change":"2024-09-20T22:16:18.404188+0000","last_active":"2024-09-20T22:16:39.930511+0000","last_peered":"2024-09-20T22:16:39.930511+0000","last_clean":"2024-09-20T22:16:39.930511+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:16:39.930511+0000","last_undegraded":"2024-09-20T22:16:39.930511+0000","last_fullsized":"2024-09-20T22:16:39.930511+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:16:18.403972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:18.403972+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:43.385480+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":212,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935003+0000","last_change":"2024-09-20T22:16:35.952285+0000","last_active":"2024-09-20T22:16:39.935003+0000","last_peered":"2024-09-20T22:16:39.935003+0000","last_clean":"2024-09-20T22:16:39.935003+0000","last_became_active":"2024-09-20T22:12:52.528068+0000","last_became_peered":"2024-09-20T22:12:52.528068+0000","last_unstale":"2024-09-20T22:16:39.935003+0000","last_undegraded":"2024-09-20T22:16:39.935003+0000","last_fullsized":"2024-09-20T22:16:39.935003+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:16:35.952049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:35.952049+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:45.642636+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"315'11","reported_seq":496,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.933817+0000","last_change":"2024-09-20T22:15:56.451915+0000","last_active":"2024-09-20T22:16:39.933817+0000","last_peered":"2024-09-20T22:16:39.933817+0000","last_clean":"2024-09-20T22:16:39.933817+0000","last_became_active":"2024-09-20T22:12:33.545942+0000","last_became_peered":"2024-09-20T22:12:33.545942+0000","last_unstale":"2024-09-20T22:16:39.933817+0000","last_undegraded":"2024-09-20T22:16:39.933817+0000","last_fullsized":"2024-09-20T22:16:39.933817+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:15:56.451808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:56.451808+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:22.777565+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":737,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935970+0000","last_change":"2024-09-20T22:15:52.564511+0000","last_active":"2024-09-20T22:16:39.935970+0000","last_peered":"2024-09-20T22:16:39.935970+0000","last_clean":"2024-09-20T22:16:39.935970+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:16:39.935970+0000","last_undegraded":"2024-09-20T22:16:39.935970+0000","last_fullsized":"2024-09-20T22:16:39.935970+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:52.564398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:52.564398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:17.677910+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"410'7247","reported_seq":7336,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:40.858665+0000","last_change":"2024-09-20T22:15:08.931965+0000","last_active":"2024-09-20T22:16:40.858665+0000","last_peered":"2024-09-20T22:16:40.858665+0000","last_clean":"2024-09-20T22:16:40.858665+0000","last_became_active":"2024-09-20T22:15:08.931653+0000","last_became_peered":"2024-09-20T22:15:08.931653+0000","last_unstale":"2024-09-20T22:16:40.858665+0000","last_undegraded":"2024-09-20T22:16:40.858665+0000","last_fullsized":"2024-09-20T22:16:40.858665+0000","mapping_epoch":370,"log_start":"380'2700","ondisk_log_start":"380'2700","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":4547,"log_dups_size":0,"ondisk_log_size":4547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":474939392,"num_objects":7247,"num_object_clones":0,"num_object_copies":21741,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7247,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7247,"num_write_kb":463808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":790,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.938429+0000","last_change":"2024-09-20T22:15:46.585672+0000","last_active":"2024-09-20T22:16:39.938429+0000","last_peered":"2024-09-20T22:16:39.938429+0000","last_clean":"2024-09-20T22:16:39.938429+0000","last_became_active":"2024-09-20T22:15:45.635509+0000","last_became_peered":"2024-09-20T22:15:45.635509+0000","last_unstale":"2024-09-20T22:16:39.938429+0000","last_undegraded":"2024-09-20T22:16:39.938429+0000","last_fullsized":"2024-09-20T22:16:39.938429+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:46.584651+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:46.584651+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:55.115974+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"328'20","reported_seq":663,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.934469+0000","last_change":"2024-09-20T22:16:28.261680+0000","last_active":"2024-09-20T22:16:39.934469+0000","last_peered":"2024-09-20T22:16:39.934469+0000","last_clean":"2024-09-20T22:16:39.934469+0000","last_became_active":"2024-09-20T22:11:30.486532+0000","last_became_peered":"2024-09-20T22:11:30.486532+0000","last_unstale":"2024-09-20T22:16:39.934469+0000","last_undegraded":"2024-09-20T22:16:39.934469+0000","last_fullsized":"2024-09-20T22:16:39.934469+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:16:28.261547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:28.261547+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:47.319640+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":764,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.939688+0000","last_change":"2024-09-20T22:14:11.402471+0000","last_active":"2024-09-20T22:16:39.939688+0000","last_peered":"2024-09-20T22:16:39.939688+0000","last_clean":"2024-09-20T22:16:39.939688+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:16:39.939688+0000","last_undegraded":"2024-09-20T22:16:39.939688+0000","last_fullsized":"2024-09-20T22:16:39.939688+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:11.402361+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:14:11.402361+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.751915+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"363'13","reported_seq":248,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.932647+0000","last_change":"2024-09-20T22:16:25.963851+0000","last_active":"2024-09-20T22:16:39.932647+0000","last_peered":"2024-09-20T22:16:39.932647+0000","last_clean":"2024-09-20T22:16:39.932647+0000","last_became_active":"2024-09-20T22:12:57.884415+0000","last_became_peered":"2024-09-20T22:12:57.884415+0000","last_unstale":"2024-09-20T22:16:39.932647+0000","last_undegraded":"2024-09-20T22:16:39.932647+0000","last_fullsized":"2024-09-20T22:16:39.932647+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"363'13","last_scrub_stamp":"2024-09-20T22:16:25.963742+0000","last_deep_scrub":"363'13","last_deep_scrub_stamp":"2024-09-20T22:14:52.612615+0000","last_clean_scrub_stamp":"2024-09-20T22:16:25.963742+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:50.917231+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":200,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.932746+0000","last_change":"2024-09-20T22:16:31.044840+0000","last_active":"2024-09-20T22:16:39.932746+0000","last_peered":"2024-09-20T22:16:39.932746+0000","last_clean":"2024-09-20T22:16:39.932746+0000","last_became_active":"2024-09-20T22:12:52.518529+0000","last_became_peered":"2024-09-20T22:12:52.518529+0000","last_unstale":"2024-09-20T22:16:39.932746+0000","last_undegraded":"2024-09-20T22:16:39.932746+0000","last_fullsized":"2024-09-20T22:16:39.932746+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:31.044564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:31.044564+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.044564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:00.508338+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"410'7399","reported_seq":7565,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:43.024638+0000","last_change":"2024-09-20T22:16:35.962440+0000","last_active":"2024-09-20T22:16:43.024638+0000","last_peered":"2024-09-20T22:16:43.024638+0000","last_clean":"2024-09-20T22:16:43.024638+0000","last_became_active":"2024-09-20T22:15:08.927661+0000","last_became_peered":"2024-09-20T22:15:08.927661+0000","last_unstale":"2024-09-20T22:16:43.024638+0000","last_undegraded":"2024-09-20T22:16:43.024638+0000","last_fullsized":"2024-09-20T22:16:43.024638+0000","mapping_epoch":370,"log_start":"380'2400","ondisk_log_start":"380'2400","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"407'6856","last_scrub_stamp":"2024-09-20T22:16:35.962220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:35.962220+0000","objects_scrubbed":6781,"log_size":4999,"log_dups_size":0,"ondisk_log_size":4999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:00.411848+0000","scrub_duration":2297,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":484900864,"num_objects":7399,"num_object_clones":0,"num_object_copies":22197,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7399,"num_write_kb":473536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":791,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.936283+0000","last_change":"2024-09-20T22:15:47.629135+0000","last_active":"2024-09-20T22:16:39.936283+0000","last_peered":"2024-09-20T22:16:39.936283+0000","last_clean":"2024-09-20T22:16:39.936283+0000","last_became_active":"2024-09-20T22:15:45.634237+0000","last_became_peered":"2024-09-20T22:15:45.634237+0000","last_unstale":"2024-09-20T22:16:39.936283+0000","last_undegraded":"2024-09-20T22:16:39.936283+0000","last_fullsized":"2024-09-20T22:16:39.936283+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:47.629023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:47.629023+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:49.579004+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"392'37","reported_seq":241,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.933118+0000","last_change":"2024-09-20T22:15:56.128146+0000","last_active":"2024-09-20T22:16:39.933118+0000","last_peered":"2024-09-20T22:16:39.933118+0000","last_clean":"2024-09-20T22:16:39.933118+0000","last_became_active":"2024-09-20T22:12:52.525538+0000","last_became_peered":"2024-09-20T22:12:52.525538+0000","last_unstale":"2024-09-20T22:16:39.933118+0000","last_undegraded":"2024-09-20T22:16:39.933118+0000","last_fullsized":"2024-09-20T22:16:39.933118+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"392'37","last_scrub_stamp":"2024-09-20T22:15:56.127540+0000","last_deep_scrub":"392'37","last_deep_scrub_stamp":"2024-09-20T22:15:56.127540+0000","last_clean_scrub_stamp":"2024-09-20T22:15:56.127540+0000","objects_scrubbed":8,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:01.639462+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":599,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935477+0000","last_change":"2024-09-20T22:16:22.267306+0000","last_active":"2024-09-20T22:16:39.935477+0000","last_peered":"2024-09-20T22:16:39.935477+0000","last_clean":"2024-09-20T22:16:39.935477+0000","last_became_active":"2024-09-20T22:11:30.486428+0000","last_became_peered":"2024-09-20T22:11:30.486428+0000","last_unstale":"2024-09-20T22:16:39.935477+0000","last_undegraded":"2024-09-20T22:16:39.935477+0000","last_fullsized":"2024-09-20T22:16:39.935477+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:22.267191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:22.267191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:32.461862+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":776,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.933960+0000","last_change":"2024-09-20T22:15:58.616337+0000","last_active":"2024-09-20T22:16:39.933960+0000","last_peered":"2024-09-20T22:16:39.933960+0000","last_clean":"2024-09-20T22:16:39.933960+0000","last_became_active":"2024-09-20T22:12:52.519826+0000","last_became_peered":"2024-09-20T22:12:52.519826+0000","last_unstale":"2024-09-20T22:16:39.933960+0000","last_undegraded":"2024-09-20T22:16:39.933960+0000","last_fullsized":"2024-09-20T22:16:39.933960+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:58.616150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:15:58.616150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:15.336586+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":585,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.933827+0000","last_change":"2024-09-20T22:15:21.489682+0000","last_active":"2024-09-20T22:16:39.933827+0000","last_peered":"2024-09-20T22:16:39.933827+0000","last_clean":"2024-09-20T22:16:39.933827+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:16:39.933827+0000","last_undegraded":"2024-09-20T22:16:39.933827+0000","last_fullsized":"2024-09-20T22:16:39.933827+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:21.489619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:21.489619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:43.233932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"410'7512","reported_seq":7677,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:42.979794+0000","last_change":"2024-09-20T22:16:32.715147+0000","last_active":"2024-09-20T22:16:42.979794+0000","last_peered":"2024-09-20T22:16:42.979794+0000","last_clean":"2024-09-20T22:16:42.979794+0000","last_became_active":"2024-09-20T22:15:08.926245+0000","last_became_peered":"2024-09-20T22:15:08.926245+0000","last_unstale":"2024-09-20T22:16:42.979794+0000","last_undegraded":"2024-09-20T22:16:42.979794+0000","last_fullsized":"2024-09-20T22:16:42.979794+0000","mapping_epoch":370,"log_start":"380'2500","ondisk_log_start":"380'2500","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6800","last_scrub_stamp":"2024-09-20T22:16:32.714946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.714946+0000","objects_scrubbed":6743,"log_size":5012,"log_dups_size":0,"ondisk_log_size":5012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:39.872460+0000","scrub_duration":2056,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":492306432,"num_objects":7512,"num_object_clones":0,"num_object_copies":22536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7512,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7512,"num_write_kb":480768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":778,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.939362+0000","last_change":"2024-09-20T22:15:46.095461+0000","last_active":"2024-09-20T22:16:39.939362+0000","last_peered":"2024-09-20T22:16:39.939362+0000","last_clean":"2024-09-20T22:16:39.939362+0000","last_became_active":"2024-09-20T22:15:45.627135+0000","last_became_peered":"2024-09-20T22:15:45.627135+0000","last_unstale":"2024-09-20T22:16:39.939362+0000","last_undegraded":"2024-09-20T22:16:39.939362+0000","last_fullsized":"2024-09-20T22:16:39.939362+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:46.095188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:46.095188+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:01.343209+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"328'52","reported_seq":794,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935665+0000","last_change":"2024-09-20T22:14:46.592348+0000","last_active":"2024-09-20T22:16:39.935665+0000","last_peered":"2024-09-20T22:16:39.935665+0000","last_clean":"2024-09-20T22:16:39.935665+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:16:39.935665+0000","last_undegraded":"2024-09-20T22:16:39.935665+0000","last_fullsized":"2024-09-20T22:16:39.935665+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'52","last_scrub_stamp":"2024-09-20T22:14:46.592203+0000","last_deep_scrub":"271'40","last_deep_scrub_stamp":"2024-09-20T22:11:05.235779+0000","last_clean_scrub_stamp":"2024-09-20T22:14:46.592203+0000","objects_scrubbed":10,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:43.233855+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":766,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.930884+0000","last_change":"2024-09-20T22:16:20.370000+0000","last_active":"2024-09-20T22:16:39.930884+0000","last_peered":"2024-09-20T22:16:39.930884+0000","last_clean":"2024-09-20T22:16:39.930884+0000","last_became_active":"2024-09-20T22:12:57.884257+0000","last_became_peered":"2024-09-20T22:12:57.884257+0000","last_unstale":"2024-09-20T22:16:39.930884+0000","last_undegraded":"2024-09-20T22:16:39.930884+0000","last_fullsized":"2024-09-20T22:16:39.930884+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:20.369864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:20.369864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:22.410427+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":760,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.940698+0000","last_change":"2024-09-20T22:13:51.454966+0000","last_active":"2024-09-20T22:16:39.940698+0000","last_peered":"2024-09-20T22:16:39.940698+0000","last_clean":"2024-09-20T22:16:39.940698+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:16:39.940698+0000","last_undegraded":"2024-09-20T22:16:39.940698+0000","last_fullsized":"2024-09-20T22:16:39.940698+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.454850+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.751881+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":200,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935350+0000","last_change":"2024-09-20T22:15:54.083256+0000","last_active":"2024-09-20T22:16:39.935350+0000","last_peered":"2024-09-20T22:16:39.935350+0000","last_clean":"2024-09-20T22:16:39.935350+0000","last_became_active":"2024-09-20T22:12:52.527547+0000","last_became_peered":"2024-09-20T22:12:52.527547+0000","last_unstale":"2024-09-20T22:16:39.935350+0000","last_undegraded":"2024-09-20T22:16:39.935350+0000","last_fullsized":"2024-09-20T22:16:39.935350+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:54.082968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:54.082968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:06.402225+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":736,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.944984+0000","last_change":"2024-09-20T22:15:24.071705+0000","last_active":"2024-09-20T22:16:39.944984+0000","last_peered":"2024-09-20T22:16:39.944984+0000","last_clean":"2024-09-20T22:16:39.944984+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:16:39.944984+0000","last_undegraded":"2024-09-20T22:16:39.944984+0000","last_fullsized":"2024-09-20T22:16:39.944984+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.071593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.071593+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:51.612695+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"410'7353","reported_seq":7443,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:41.213129+0000","last_change":"2024-09-20T22:15:08.931328+0000","last_active":"2024-09-20T22:16:41.213129+0000","last_peered":"2024-09-20T22:16:41.213129+0000","last_clean":"2024-09-20T22:16:41.213129+0000","last_became_active":"2024-09-20T22:15:08.930968+0000","last_became_peered":"2024-09-20T22:15:08.930968+0000","last_unstale":"2024-09-20T22:16:41.213129+0000","last_undegraded":"2024-09-20T22:16:41.213129+0000","last_fullsized":"2024-09-20T22:16:41.213129+0000","mapping_epoch":370,"log_start":"379'2400","ondisk_log_start":"379'2400","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":4953,"log_dups_size":0,"ondisk_log_size":4953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.751976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":481886208,"num_objects":7353,"num_object_clones":0,"num_object_copies":22059,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7353,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7353,"num_write_kb":470592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":776,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935303+0000","last_change":"2024-09-20T22:15:46.548031+0000","last_active":"2024-09-20T22:16:39.935303+0000","last_peered":"2024-09-20T22:16:39.935303+0000","last_clean":"2024-09-20T22:16:39.935303+0000","last_became_active":"2024-09-20T22:15:45.620376+0000","last_became_peered":"2024-09-20T22:15:45.620376+0000","last_unstale":"2024-09-20T22:16:39.935303+0000","last_undegraded":"2024-09-20T22:16:39.935303+0000","last_fullsized":"2024-09-20T22:16:39.935303+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:46.547907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:46.547907+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:55.630577+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"328'17","reported_seq":658,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.937038+0000","last_change":"2024-09-20T22:13:51.684806+0000","last_active":"2024-09-20T22:16:39.937038+0000","last_peered":"2024-09-20T22:16:39.937038+0000","last_clean":"2024-09-20T22:16:39.937038+0000","last_became_active":"2024-09-20T22:12:52.521073+0000","last_became_peered":"2024-09-20T22:12:52.521073+0000","last_unstale":"2024-09-20T22:16:39.937038+0000","last_undegraded":"2024-09-20T22:16:39.937038+0000","last_fullsized":"2024-09-20T22:16:39.937038+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'17","last_scrub_stamp":"2024-09-20T22:13:51.684667+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:51.684667+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:43.233709+0000","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":586,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.934252+0000","last_change":"2024-09-20T22:14:49.575469+0000","last_active":"2024-09-20T22:16:39.934252+0000","last_peered":"2024-09-20T22:16:39.934252+0000","last_clean":"2024-09-20T22:16:39.934252+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:16:39.934252+0000","last_undegraded":"2024-09-20T22:16:39.934252+0000","last_fullsized":"2024-09-20T22:16:39.934252+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:49.575363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:14:49.575363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:43.233879+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":810,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.941323+0000","last_change":"2024-09-20T22:15:48.955294+0000","last_active":"2024-09-20T22:16:39.941323+0000","last_peered":"2024-09-20T22:16:39.941323+0000","last_clean":"2024-09-20T22:16:39.941323+0000","last_became_active":"2024-09-20T22:12:52.517255+0000","last_became_peered":"2024-09-20T22:12:52.517255+0000","last_unstale":"2024-09-20T22:16:39.941323+0000","last_undegraded":"2024-09-20T22:16:39.941323+0000","last_fullsized":"2024-09-20T22:16:39.941323+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:15:48.955042+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:15:48.955042+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:16.989469+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":764,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.939338+0000","last_change":"2024-09-20T22:15:24.892562+0000","last_active":"2024-09-20T22:16:39.939338+0000","last_peered":"2024-09-20T22:16:39.939338+0000","last_clean":"2024-09-20T22:16:39.939338+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:16:39.939338+0000","last_undegraded":"2024-09-20T22:16:39.939338+0000","last_fullsized":"2024-09-20T22:16:39.939338+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.891955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.891955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:47.230621+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":737,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.936777+0000","last_change":"2024-09-20T22:16:23.711184+0000","last_active":"2024-09-20T22:16:39.936777+0000","last_peered":"2024-09-20T22:16:39.936777+0000","last_clean":"2024-09-20T22:16:39.936777+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:16:39.936777+0000","last_undegraded":"2024-09-20T22:16:39.936777+0000","last_fullsized":"2024-09-20T22:16:39.936777+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:23.711072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:23.711072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:36.786537+0000","scrub_duration":87,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"410'7375","reported_seq":7541,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:42.201125+0000","last_change":"2024-09-20T22:16:32.694961+0000","last_active":"2024-09-20T22:16:42.201125+0000","last_peered":"2024-09-20T22:16:42.201125+0000","last_clean":"2024-09-20T22:16:42.201125+0000","last_became_active":"2024-09-20T22:15:08.926702+0000","last_became_peered":"2024-09-20T22:15:08.926702+0000","last_unstale":"2024-09-20T22:16:42.201125+0000","last_undegraded":"2024-09-20T22:16:42.201125+0000","last_fullsized":"2024-09-20T22:16:42.201125+0000","mapping_epoch":370,"log_start":"380'2500","ondisk_log_start":"380'2500","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6715","last_scrub_stamp":"2024-09-20T22:16:32.694684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.694684+0000","objects_scrubbed":6639,"log_size":4875,"log_dups_size":0,"ondisk_log_size":4875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:00.250998+0000","scrub_duration":2750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":483328000,"num_objects":7375,"num_object_clones":0,"num_object_copies":22125,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7375,"num_write_kb":472000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":759,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935572+0000","last_change":"2024-09-20T22:15:46.338840+0000","last_active":"2024-09-20T22:16:39.935572+0000","last_peered":"2024-09-20T22:16:39.935572+0000","last_clean":"2024-09-20T22:16:39.935572+0000","last_became_active":"2024-09-20T22:15:45.630804+0000","last_became_peered":"2024-09-20T22:15:45.630804+0000","last_unstale":"2024-09-20T22:16:39.935572+0000","last_undegraded":"2024-09-20T22:16:39.935572+0000","last_fullsized":"2024-09-20T22:16:39.935572+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:46.338700+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:15:46.338700+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:06.684506+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"328'24","reported_seq":232,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.934906+0000","last_change":"2024-09-20T22:15:26.742439+0000","last_active":"2024-09-20T22:16:39.934906+0000","last_peered":"2024-09-20T22:16:39.934906+0000","last_clean":"2024-09-20T22:16:39.934906+0000","last_became_active":"2024-09-20T22:12:52.527919+0000","last_became_peered":"2024-09-20T22:12:52.527919+0000","last_unstale":"2024-09-20T22:16:39.934906+0000","last_undegraded":"2024-09-20T22:16:39.934906+0000","last_fullsized":"2024-09-20T22:16:39.934906+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'24","last_scrub_stamp":"2024-09-20T22:15:26.742317+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:26.742317+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.914101+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":200,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.940487+0000","last_change":"2024-09-20T22:15:21.774404+0000","last_active":"2024-09-20T22:16:39.940487+0000","last_peered":"2024-09-20T22:16:39.940487+0000","last_clean":"2024-09-20T22:16:39.940487+0000","last_became_active":"2024-09-20T22:12:52.517432+0000","last_became_peered":"2024-09-20T22:12:52.517432+0000","last_unstale":"2024-09-20T22:16:39.940487+0000","last_undegraded":"2024-09-20T22:16:39.940487+0000","last_fullsized":"2024-09-20T22:16:39.940487+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:21.774300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:15:21.774300+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.914072+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":1602,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.936342+0000","last_change":"2024-09-20T22:15:55.652986+0000","last_active":"2024-09-20T22:16:39.936342+0000","last_peered":"2024-09-20T22:16:39.936342+0000","last_clean":"2024-09-20T22:16:39.936342+0000","last_became_active":"2024-09-20T22:12:52.526622+0000","last_became_peered":"2024-09-20T22:12:52.526622+0000","last_unstale":"2024-09-20T22:16:39.936342+0000","last_undegraded":"2024-09-20T22:16:39.936342+0000","last_fullsized":"2024-09-20T22:16:39.936342+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:15:55.652860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:15:55.652860+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:58.127886+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":806,"num_read_kb":801,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":295,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.940576+0000","last_change":"2024-09-20T22:14:14.443714+0000","last_active":"2024-09-20T22:16:39.940576+0000","last_peered":"2024-09-20T22:16:39.940576+0000","last_clean":"2024-09-20T22:16:39.940576+0000","last_became_active":"2024-09-20T22:12:52.518823+0000","last_became_peered":"2024-09-20T22:12:52.518823+0000","last_unstale":"2024-09-20T22:16:39.940576+0000","last_undegraded":"2024-09-20T22:16:39.940576+0000","last_fullsized":"2024-09-20T22:16:39.940576+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:14.443611+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:14.443611+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.751900+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":586,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935303+0000","last_change":"2024-09-20T22:16:26.330409+0000","last_active":"2024-09-20T22:16:39.935303+0000","last_peered":"2024-09-20T22:16:39.935303+0000","last_clean":"2024-09-20T22:16:39.935303+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:16:39.935303+0000","last_undegraded":"2024-09-20T22:16:39.935303+0000","last_fullsized":"2024-09-20T22:16:39.935303+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:26.330270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:26.330270+0000","last_clean_scrub_stamp":"2024-09-20T22:16:26.330270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:27.069295+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"410'7491","reported_seq":7581,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:43.024679+0000","last_change":"2024-09-20T22:15:08.925131+0000","last_active":"2024-09-20T22:16:43.024679+0000","last_peered":"2024-09-20T22:16:43.024679+0000","last_clean":"2024-09-20T22:16:43.024679+0000","last_became_active":"2024-09-20T22:15:08.924809+0000","last_became_peered":"2024-09-20T22:15:08.924809+0000","last_unstale":"2024-09-20T22:16:43.024679+0000","last_undegraded":"2024-09-20T22:16:43.024679+0000","last_fullsized":"2024-09-20T22:16:43.024679+0000","mapping_epoch":370,"log_start":"380'2500","ondisk_log_start":"380'2500","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":4991,"log_dups_size":0,"ondisk_log_size":4991,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:45.656059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":490930176,"num_objects":7491,"num_object_clones":0,"num_object_copies":22473,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7491,"num_write_kb":479424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"328'28","reported_seq":967,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.934847+0000","last_change":"2024-09-20T22:15:55.704801+0000","last_active":"2024-09-20T22:16:39.934847+0000","last_peered":"2024-09-20T22:16:39.934847+0000","last_clean":"2024-09-20T22:16:39.934847+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:16:39.934847+0000","last_undegraded":"2024-09-20T22:16:39.934847+0000","last_fullsized":"2024-09-20T22:16:39.934847+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:15:55.704666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:55.704666+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:23.796840+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":709,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.939983+0000","last_change":"2024-09-20T22:16:36.729250+0000","last_active":"2024-09-20T22:16:39.939983+0000","last_peered":"2024-09-20T22:16:39.939983+0000","last_clean":"2024-09-20T22:16:39.939983+0000","last_became_active":"2024-09-20T22:12:21.519717+0000","last_became_peered":"2024-09-20T22:12:21.519717+0000","last_unstale":"2024-09-20T22:16:39.939983+0000","last_undegraded":"2024-09-20T22:16:39.939983+0000","last_fullsized":"2024-09-20T22:16:39.939983+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:36.729158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:36.729158+0000","last_clean_scrub_stamp":"2024-09-20T22:16:36.729158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:53.476434+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":776,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935912+0000","last_change":"2024-09-20T22:15:47.554095+0000","last_active":"2024-09-20T22:16:39.935912+0000","last_peered":"2024-09-20T22:16:39.935912+0000","last_clean":"2024-09-20T22:16:39.935912+0000","last_became_active":"2024-09-20T22:15:45.620972+0000","last_became_peered":"2024-09-20T22:15:45.620972+0000","last_unstale":"2024-09-20T22:16:39.935912+0000","last_undegraded":"2024-09-20T22:16:39.935912+0000","last_fullsized":"2024-09-20T22:16:39.935912+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:47.553962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:47.553962+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:48.105550+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":331,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935496+0000","last_change":"2024-09-20T22:15:58.617583+0000","last_active":"2024-09-20T22:16:39.935496+0000","last_peered":"2024-09-20T22:16:39.935496+0000","last_clean":"2024-09-20T22:16:39.935496+0000","last_became_active":"2024-09-20T22:10:48.907271+0000","last_became_peered":"2024-09-20T22:10:48.907271+0000","last_unstale":"2024-09-20T22:16:39.935496+0000","last_undegraded":"2024-09-20T22:16:39.935496+0000","last_fullsized":"2024-09-20T22:16:39.935496+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:58.617484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_clean_scrub_stamp":"2024-09-20T22:15:58.617484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:05.365475+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":756,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.940375+0000","last_change":"2024-09-20T22:13:02.482655+0000","last_active":"2024-09-20T22:16:39.940375+0000","last_peered":"2024-09-20T22:16:39.940375+0000","last_clean":"2024-09-20T22:16:39.940375+0000","last_became_active":"2024-09-20T22:12:44.630044+0000","last_became_peered":"2024-09-20T22:12:44.630044+0000","last_unstale":"2024-09-20T22:16:39.940375+0000","last_undegraded":"2024-09-20T22:16:39.940375+0000","last_fullsized":"2024-09-20T22:16:39.940375+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:13:02.482546+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:13:02.482546+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.751815+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"75'128","reported_seq":767,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935370+0000","last_change":"2024-09-20T22:15:51.314327+0000","last_active":"2024-09-20T22:16:39.935370+0000","last_peered":"2024-09-20T22:16:39.935370+0000","last_clean":"2024-09-20T22:16:39.935370+0000","last_became_active":"2024-09-20T22:15:50.615076+0000","last_became_peered":"2024-09-20T22:15:50.615076+0000","last_unstale":"2024-09-20T22:16:39.935370+0000","last_undegraded":"2024-09-20T22:16:39.935370+0000","last_fullsized":"2024-09-20T22:16:39.935370+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:51.314232+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:15:51.314232+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:19.808113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"410'7561","reported_seq":7701,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:43.000183+0000","last_change":"2024-09-20T22:16:37.959295+0000","last_active":"2024-09-20T22:16:43.000183+0000","last_peered":"2024-09-20T22:16:43.000183+0000","last_clean":"2024-09-20T22:16:43.000183+0000","last_became_active":"2024-09-20T22:15:08.925040+0000","last_became_peered":"2024-09-20T22:15:08.925040+0000","last_unstale":"2024-09-20T22:16:43.000183+0000","last_undegraded":"2024-09-20T22:16:43.000183+0000","last_fullsized":"2024-09-20T22:16:43.000183+0000","mapping_epoch":370,"log_start":"380'2600","ondisk_log_start":"380'2600","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":4649,"log_size":4961,"log_dups_size":0,"ondisk_log_size":4961,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":495517696,"num_objects":7561,"num_object_clones":0,"num_object_copies":22683,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7561,"num_write_kb":483904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"328'19","reported_seq":785,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.931414+0000","last_change":"2024-09-20T22:16:24.435555+0000","last_active":"2024-09-20T22:16:39.931414+0000","last_peered":"2024-09-20T22:16:39.931414+0000","last_clean":"2024-09-20T22:16:39.931414+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:16:39.931414+0000","last_undegraded":"2024-09-20T22:16:39.931414+0000","last_fullsized":"2024-09-20T22:16:39.931414+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:16:24.435260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:24.435260+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:26.710702+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":749,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.930506+0000","last_change":"2024-09-20T22:15:27.077414+0000","last_active":"2024-09-20T22:16:39.930506+0000","last_peered":"2024-09-20T22:16:39.930506+0000","last_clean":"2024-09-20T22:16:39.930506+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:16:39.930506+0000","last_undegraded":"2024-09-20T22:16:39.930506+0000","last_fullsized":"2024-09-20T22:16:39.930506+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:27.077295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.077295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:47.688033+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":748,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.940160+0000","last_change":"2024-09-20T22:15:24.150401+0000","last_active":"2024-09-20T22:16:39.940160+0000","last_peered":"2024-09-20T22:16:39.940160+0000","last_clean":"2024-09-20T22:16:39.940160+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:16:39.940160+0000","last_undegraded":"2024-09-20T22:16:39.940160+0000","last_fullsized":"2024-09-20T22:16:39.940160+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.150265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.150265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.751990+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":703,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935702+0000","last_change":"2024-09-20T22:15:52.643084+0000","last_active":"2024-09-20T22:16:39.935702+0000","last_peered":"2024-09-20T22:16:39.935702+0000","last_clean":"2024-09-20T22:16:39.935702+0000","last_became_active":"2024-09-20T22:12:21.521195+0000","last_became_peered":"2024-09-20T22:12:21.521195+0000","last_unstale":"2024-09-20T22:16:39.935702+0000","last_undegraded":"2024-09-20T22:16:39.935702+0000","last_fullsized":"2024-09-20T22:16:39.935702+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:52.642911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:52.642911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:16.037925+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3","version":"410'7310","reported_seq":7477,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:41.256524+0000","last_change":"2024-09-20T22:16:32.632993+0000","last_active":"2024-09-20T22:16:41.256524+0000","last_peered":"2024-09-20T22:16:41.256524+0000","last_clean":"2024-09-20T22:16:41.256524+0000","last_became_active":"2024-09-20T22:15:09.087061+0000","last_became_peered":"2024-09-20T22:15:09.087061+0000","last_unstale":"2024-09-20T22:16:41.256524+0000","last_undegraded":"2024-09-20T22:16:41.256524+0000","last_fullsized":"2024-09-20T22:16:41.256524+0000","mapping_epoch":370,"log_start":"378'1400","ondisk_log_start":"378'1400","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6740","last_scrub_stamp":"2024-09-20T22:16:32.632756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.632756+0000","objects_scrubbed":6652,"log_size":5910,"log_dups_size":0,"ondisk_log_size":5910,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:35.823889+0000","scrub_duration":2683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":479068160,"num_objects":7310,"num_object_clones":0,"num_object_copies":21930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7310,"num_write_kb":467840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":521,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.933236+0000","last_change":"2024-09-20T22:16:18.773405+0000","last_active":"2024-09-20T22:16:39.933236+0000","last_peered":"2024-09-20T22:16:39.933236+0000","last_clean":"2024-09-20T22:16:39.933236+0000","last_became_active":"2024-09-20T22:14:38.674758+0000","last_became_peered":"2024-09-20T22:14:38.674758+0000","last_unstale":"2024-09-20T22:16:39.933236+0000","last_undegraded":"2024-09-20T22:16:39.933236+0000","last_fullsized":"2024-09-20T22:16:39.933236+0000","mapping_epoch":358,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":359,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:16:18.773201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:16:18.773201+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:39.013289+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"328'31","reported_seq":811,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.931524+0000","last_change":"2024-09-20T22:16:17.388362+0000","last_active":"2024-09-20T22:16:39.931524+0000","last_peered":"2024-09-20T22:16:39.931524+0000","last_clean":"2024-09-20T22:16:39.931524+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:16:39.931524+0000","last_undegraded":"2024-09-20T22:16:39.931524+0000","last_fullsized":"2024-09-20T22:16:39.931524+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'31","last_scrub_stamp":"2024-09-20T22:16:17.387895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:17.387895+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:42.903740+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":765,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.934182+0000","last_change":"2024-09-20T22:15:32.512124+0000","last_active":"2024-09-20T22:16:39.934182+0000","last_peered":"2024-09-20T22:16:39.934182+0000","last_clean":"2024-09-20T22:16:39.934182+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:16:39.934182+0000","last_undegraded":"2024-09-20T22:16:39.934182+0000","last_fullsized":"2024-09-20T22:16:39.934182+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:32.506512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:32.506512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:01.564420+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":598,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935391+0000","last_change":"2024-09-20T22:15:57.442451+0000","last_active":"2024-09-20T22:16:39.935391+0000","last_peered":"2024-09-20T22:16:39.935391+0000","last_clean":"2024-09-20T22:16:39.935391+0000","last_became_active":"2024-09-20T22:12:52.520612+0000","last_became_peered":"2024-09-20T22:12:52.520612+0000","last_unstale":"2024-09-20T22:16:39.935391+0000","last_undegraded":"2024-09-20T22:16:39.935391+0000","last_fullsized":"2024-09-20T22:16:39.935391+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:57.442341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.442341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:10.741669+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":760,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.940204+0000","last_change":"2024-09-20T22:15:56.941775+0000","last_active":"2024-09-20T22:16:39.940204+0000","last_peered":"2024-09-20T22:16:39.940204+0000","last_clean":"2024-09-20T22:16:39.940204+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:16:39.940204+0000","last_undegraded":"2024-09-20T22:16:39.940204+0000","last_fullsized":"2024-09-20T22:16:39.940204+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:56.941427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:56.941427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:04.318727+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"410'7436","reported_seq":7525,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:43.432162+0000","last_change":"2024-09-20T22:15:08.926429+0000","last_active":"2024-09-20T22:16:43.432162+0000","last_peered":"2024-09-20T22:16:43.432162+0000","last_clean":"2024-09-20T22:16:43.432162+0000","last_became_active":"2024-09-20T22:15:08.926083+0000","last_became_peered":"2024-09-20T22:15:08.926083+0000","last_unstale":"2024-09-20T22:16:43.432162+0000","last_undegraded":"2024-09-20T22:16:43.432162+0000","last_fullsized":"2024-09-20T22:16:43.432162+0000","mapping_epoch":370,"log_start":"380'2700","ondisk_log_start":"380'2700","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":4736,"log_dups_size":0,"ondisk_log_size":4736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":487325696,"num_objects":7436,"num_object_clones":0,"num_object_copies":22308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7436,"num_write_kb":475904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":1032,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.937804+0000","last_change":"2024-09-20T22:15:56.638628+0000","last_active":"2024-09-20T22:16:39.937804+0000","last_peered":"2024-09-20T22:16:39.937804+0000","last_clean":"2024-09-20T22:16:39.937804+0000","last_became_active":"2024-09-20T22:08:48.455288+0000","last_became_peered":"2024-09-20T22:08:48.455288+0000","last_unstale":"2024-09-20T22:16:39.937804+0000","last_undegraded":"2024-09-20T22:16:39.937804+0000","last_fullsized":"2024-09-20T22:16:39.937804+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:56.638414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:56.638414+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:22.223454+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"328'25","reported_seq":780,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.932038+0000","last_change":"2024-09-20T22:15:50.223866+0000","last_active":"2024-09-20T22:16:39.932038+0000","last_peered":"2024-09-20T22:16:39.932038+0000","last_clean":"2024-09-20T22:16:39.932038+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:16:39.932038+0000","last_undegraded":"2024-09-20T22:16:39.932038+0000","last_fullsized":"2024-09-20T22:16:39.932038+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'25","last_scrub_stamp":"2024-09-20T22:15:50.223785+0000","last_deep_scrub":"289'15","last_deep_scrub_stamp":"2024-09-20T22:11:37.277648+0000","last_clean_scrub_stamp":"2024-09-20T22:15:50.223785+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:16.829889+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":204,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.932814+0000","last_change":"2024-09-20T22:15:30.821251+0000","last_active":"2024-09-20T22:16:39.932814+0000","last_peered":"2024-09-20T22:16:39.932814+0000","last_clean":"2024-09-20T22:16:39.932814+0000","last_became_active":"2024-09-20T22:12:52.528310+0000","last_became_peered":"2024-09-20T22:12:52.528310+0000","last_unstale":"2024-09-20T22:16:39.932814+0000","last_undegraded":"2024-09-20T22:16:39.932814+0000","last_fullsized":"2024-09-20T22:16:39.932814+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:15:30.821142+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:15:30.821142+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:55.515561+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"283'11","reported_seq":968,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.939031+0000","last_change":"2024-09-20T22:15:49.973513+0000","last_active":"2024-09-20T22:16:39.939031+0000","last_peered":"2024-09-20T22:16:39.939031+0000","last_clean":"2024-09-20T22:16:39.939031+0000","last_became_active":"2024-09-20T22:11:30.485356+0000","last_became_peered":"2024-09-20T22:11:30.485356+0000","last_unstale":"2024-09-20T22:16:39.939031+0000","last_undegraded":"2024-09-20T22:16:39.939031+0000","last_fullsized":"2024-09-20T22:16:39.939031+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'11","last_scrub_stamp":"2024-09-20T22:15:49.973229+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:15:49.973229+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:52.728072+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":746,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.931233+0000","last_change":"2024-09-20T22:16:27.486633+0000","last_active":"2024-09-20T22:16:39.931233+0000","last_peered":"2024-09-20T22:16:39.931233+0000","last_clean":"2024-09-20T22:16:39.931233+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:16:39.931233+0000","last_undegraded":"2024-09-20T22:16:39.931233+0000","last_fullsized":"2024-09-20T22:16:39.931233+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:27.486520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:16:27.486520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:41.356713+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"75'128","reported_seq":638,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.931031+0000","last_change":"2024-09-20T22:15:51.252861+0000","last_active":"2024-09-20T22:16:39.931031+0000","last_peered":"2024-09-20T22:16:39.931031+0000","last_clean":"2024-09-20T22:16:39.931031+0000","last_became_active":"2024-09-20T22:15:50.594320+0000","last_became_peered":"2024-09-20T22:15:50.594320+0000","last_unstale":"2024-09-20T22:16:39.931031+0000","last_undegraded":"2024-09-20T22:16:39.931031+0000","last_fullsized":"2024-09-20T22:16:39.931031+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:51.252685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:51.252685+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:04.804456+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"410'7343","reported_seq":7502,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:41.256410+0000","last_change":"2024-09-20T22:16:21.665425+0000","last_active":"2024-09-20T22:16:41.256410+0000","last_peered":"2024-09-20T22:16:41.256410+0000","last_clean":"2024-09-20T22:16:41.256410+0000","last_became_active":"2024-09-20T22:15:09.087830+0000","last_became_peered":"2024-09-20T22:15:09.087830+0000","last_unstale":"2024-09-20T22:16:41.256410+0000","last_undegraded":"2024-09-20T22:16:41.256410+0000","last_fullsized":"2024-09-20T22:16:41.256410+0000","mapping_epoch":370,"log_start":"378'1400","ondisk_log_start":"378'1400","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"403'6042","last_scrub_stamp":"2024-09-20T22:16:21.665289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:21.665289+0000","objects_scrubbed":6017,"log_size":5943,"log_dups_size":0,"ondisk_log_size":5943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:21.711237+0000","scrub_duration":1845,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":481230848,"num_objects":7343,"num_object_clones":0,"num_object_copies":22029,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7343,"num_write_kb":469952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"328'15","reported_seq":219,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.934913+0000","last_change":"2024-09-20T22:15:25.742250+0000","last_active":"2024-09-20T22:16:39.934913+0000","last_peered":"2024-09-20T22:16:39.934913+0000","last_clean":"2024-09-20T22:16:39.934913+0000","last_became_active":"2024-09-20T22:12:52.527531+0000","last_became_peered":"2024-09-20T22:12:52.527531+0000","last_unstale":"2024-09-20T22:16:39.934913+0000","last_undegraded":"2024-09-20T22:16:39.934913+0000","last_fullsized":"2024-09-20T22:16:39.934913+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:15:25.741631+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:25.741631+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.914019+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":384,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.945746+0000","last_change":"2024-09-20T22:16:29.707733+0000","last_active":"2024-09-20T22:16:39.945746+0000","last_peered":"2024-09-20T22:16:39.945746+0000","last_clean":"2024-09-20T22:16:39.945746+0000","last_became_active":"2024-09-20T22:10:14.536155+0000","last_became_peered":"2024-09-20T22:10:14.536155+0000","last_unstale":"2024-09-20T22:16:39.945746+0000","last_undegraded":"2024-09-20T22:16:39.945746+0000","last_fullsized":"2024-09-20T22:16:39.945746+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:16:29.707441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:29.707441+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:45.157085+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":772,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.938756+0000","last_change":"2024-09-20T22:16:33.008048+0000","last_active":"2024-09-20T22:16:39.938756+0000","last_peered":"2024-09-20T22:16:39.938756+0000","last_clean":"2024-09-20T22:16:39.938756+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:16:39.938756+0000","last_undegraded":"2024-09-20T22:16:39.938756+0000","last_fullsized":"2024-09-20T22:16:39.938756+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:33.007962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:16:33.007962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:56.423160+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":200,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.940644+0000","last_change":"2024-09-20T22:16:24.020680+0000","last_active":"2024-09-20T22:16:39.940644+0000","last_peered":"2024-09-20T22:16:39.940644+0000","last_clean":"2024-09-20T22:16:39.940644+0000","last_became_active":"2024-09-20T22:12:52.512924+0000","last_became_peered":"2024-09-20T22:12:52.512924+0000","last_unstale":"2024-09-20T22:16:39.940644+0000","last_undegraded":"2024-09-20T22:16:39.940644+0000","last_fullsized":"2024-09-20T22:16:39.940644+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:24.020573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:24.020573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:44.565343+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"75'128","reported_seq":798,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.938831+0000","last_change":"2024-09-20T22:15:50.666033+0000","last_active":"2024-09-20T22:16:39.938831+0000","last_peered":"2024-09-20T22:16:39.938831+0000","last_clean":"2024-09-20T22:16:39.938831+0000","last_became_active":"2024-09-20T22:15:50.576540+0000","last_became_peered":"2024-09-20T22:15:50.576540+0000","last_unstale":"2024-09-20T22:16:39.938831+0000","last_undegraded":"2024-09-20T22:16:39.938831+0000","last_fullsized":"2024-09-20T22:16:39.938831+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:50.665724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:50.665724+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:57.862343+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.4","version":"410'7348","reported_seq":7511,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:42.205213+0000","last_change":"2024-09-20T22:16:31.859517+0000","last_active":"2024-09-20T22:16:42.205213+0000","last_peered":"2024-09-20T22:16:42.205213+0000","last_clean":"2024-09-20T22:16:42.205213+0000","last_became_active":"2024-09-20T22:15:09.087298+0000","last_became_peered":"2024-09-20T22:15:09.087298+0000","last_unstale":"2024-09-20T22:16:42.205213+0000","last_undegraded":"2024-09-20T22:16:42.205213+0000","last_fullsized":"2024-09-20T22:16:42.205213+0000","mapping_epoch":370,"log_start":"380'2500","ondisk_log_start":"380'2500","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6582","last_scrub_stamp":"2024-09-20T22:16:31.859305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.859305+0000","objects_scrubbed":6495,"log_size":4848,"log_dups_size":0,"ondisk_log_size":4848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:58.626512+0000","scrub_duration":2899,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":481558528,"num_objects":7348,"num_object_clones":0,"num_object_copies":22044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7348,"num_write_kb":470272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"328'35","reported_seq":860,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.933968+0000","last_change":"2024-09-20T22:15:31.492645+0000","last_active":"2024-09-20T22:16:39.933968+0000","last_peered":"2024-09-20T22:16:39.933968+0000","last_clean":"2024-09-20T22:16:39.933968+0000","last_became_active":"2024-09-20T22:10:30.652967+0000","last_became_peered":"2024-09-20T22:10:30.652967+0000","last_unstale":"2024-09-20T22:16:39.933968+0000","last_undegraded":"2024-09-20T22:16:39.933968+0000","last_fullsized":"2024-09-20T22:16:39.933968+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'35","last_scrub_stamp":"2024-09-20T22:15:31.492433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:31.492433+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:55.689722+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":761,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.937039+0000","last_change":"2024-09-20T22:15:23.351913+0000","last_active":"2024-09-20T22:16:39.937039+0000","last_peered":"2024-09-20T22:16:39.937039+0000","last_clean":"2024-09-20T22:16:39.937039+0000","last_became_active":"2024-09-20T22:11:30.488650+0000","last_became_peered":"2024-09-20T22:11:30.488650+0000","last_unstale":"2024-09-20T22:16:39.937039+0000","last_undegraded":"2024-09-20T22:16:39.937039+0000","last_fullsized":"2024-09-20T22:16:39.937039+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:23.351789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:12:46.602035+0000","last_clean_scrub_stamp":"2024-09-20T22:15:23.351789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":796,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.943259+0000","last_change":"2024-09-20T22:16:18.795310+0000","last_active":"2024-09-20T22:16:39.943259+0000","last_peered":"2024-09-20T22:16:39.943259+0000","last_clean":"2024-09-20T22:16:39.943259+0000","last_became_active":"2024-09-20T22:12:52.515295+0000","last_became_peered":"2024-09-20T22:12:52.515295+0000","last_unstale":"2024-09-20T22:16:39.943259+0000","last_undegraded":"2024-09-20T22:16:39.943259+0000","last_fullsized":"2024-09-20T22:16:39.943259+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:18.795079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_clean_scrub_stamp":"2024-09-20T22:16:18.795079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:36.635490+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":353,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.936013+0000","last_change":"2024-09-20T22:16:35.739421+0000","last_active":"2024-09-20T22:16:39.936013+0000","last_peered":"2024-09-20T22:16:39.936013+0000","last_clean":"2024-09-20T22:16:39.936013+0000","last_became_active":"2024-09-20T22:10:30.657116+0000","last_became_peered":"2024-09-20T22:10:30.657116+0000","last_unstale":"2024-09-20T22:16:39.936013+0000","last_undegraded":"2024-09-20T22:16:39.936013+0000","last_fullsized":"2024-09-20T22:16:39.936013+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:35.739168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:35.739168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:45.935247+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"75'128","reported_seq":808,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.945350+0000","last_change":"2024-09-20T22:15:50.922162+0000","last_active":"2024-09-20T22:16:39.945350+0000","last_peered":"2024-09-20T22:16:39.945350+0000","last_clean":"2024-09-20T22:16:39.945350+0000","last_became_active":"2024-09-20T22:15:50.619935+0000","last_became_peered":"2024-09-20T22:15:50.619935+0000","last_unstale":"2024-09-20T22:16:39.945350+0000","last_undegraded":"2024-09-20T22:16:39.945350+0000","last_fullsized":"2024-09-20T22:16:39.945350+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:50.922028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:50.922028+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:16.552077+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.7","version":"410'7480","reported_seq":7646,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:43.015044+0000","last_change":"2024-09-20T22:16:29.896220+0000","last_active":"2024-09-20T22:16:43.015044+0000","last_peered":"2024-09-20T22:16:43.015044+0000","last_clean":"2024-09-20T22:16:43.015044+0000","last_became_active":"2024-09-20T22:15:09.088590+0000","last_became_peered":"2024-09-20T22:15:09.088590+0000","last_unstale":"2024-09-20T22:16:43.015044+0000","last_undegraded":"2024-09-20T22:16:43.015044+0000","last_fullsized":"2024-09-20T22:16:43.015044+0000","mapping_epoch":370,"log_start":"380'2500","ondisk_log_start":"380'2500","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6589","last_scrub_stamp":"2024-09-20T22:16:29.896110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:29.896110+0000","objects_scrubbed":6527,"log_size":4980,"log_dups_size":0,"ondisk_log_size":4980,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:39.515203+0000","scrub_duration":2191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":490209280,"num_objects":7480,"num_object_clones":0,"num_object_copies":22440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7480,"num_write_kb":478720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"328'18","reported_seq":778,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.930219+0000","last_change":"2024-09-20T22:15:22.019818+0000","last_active":"2024-09-20T22:16:39.930219+0000","last_peered":"2024-09-20T22:16:39.930219+0000","last_clean":"2024-09-20T22:16:39.930219+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:16:39.930219+0000","last_undegraded":"2024-09-20T22:16:39.930219+0000","last_fullsized":"2024-09-20T22:16:39.930219+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'18","last_scrub_stamp":"2024-09-20T22:15:22.019700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:22.019700+0000","objects_scrubbed":5,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:42.439452+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":758,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.937540+0000","last_change":"2024-09-20T22:16:25.649600+0000","last_active":"2024-09-20T22:16:39.937540+0000","last_peered":"2024-09-20T22:16:39.937540+0000","last_clean":"2024-09-20T22:16:39.937540+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:16:39.937540+0000","last_undegraded":"2024-09-20T22:16:39.937540+0000","last_fullsized":"2024-09-20T22:16:39.937540+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:25.649511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:25.649511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:42.623417+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":785,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.938605+0000","last_change":"2024-09-20T22:15:27.924542+0000","last_active":"2024-09-20T22:16:39.938605+0000","last_peered":"2024-09-20T22:16:39.938605+0000","last_clean":"2024-09-20T22:16:39.938605+0000","last_became_active":"2024-09-20T22:12:52.514202+0000","last_became_peered":"2024-09-20T22:12:52.514202+0000","last_unstale":"2024-09-20T22:16:39.938605+0000","last_undegraded":"2024-09-20T22:16:39.938605+0000","last_fullsized":"2024-09-20T22:16:39.938605+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:27.924150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.924150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:41.071832+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":210,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.942011+0000","last_change":"2024-09-20T22:15:26.021754+0000","last_active":"2024-09-20T22:16:39.942011+0000","last_peered":"2024-09-20T22:16:39.942011+0000","last_clean":"2024-09-20T22:16:39.942011+0000","last_became_active":"2024-09-20T22:12:44.638546+0000","last_became_peered":"2024-09-20T22:12:44.638546+0000","last_unstale":"2024-09-20T22:16:39.942011+0000","last_undegraded":"2024-09-20T22:16:39.942011+0000","last_fullsized":"2024-09-20T22:16:39.942011+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:26.021636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:26.021636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:46.884542+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'128","reported_seq":354,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.936505+0000","last_change":"2024-09-20T22:15:51.552197+0000","last_active":"2024-09-20T22:16:39.936505+0000","last_peered":"2024-09-20T22:16:39.936505+0000","last_clean":"2024-09-20T22:16:39.936505+0000","last_became_active":"2024-09-20T22:15:50.594241+0000","last_became_peered":"2024-09-20T22:15:50.594241+0000","last_unstale":"2024-09-20T22:16:39.936505+0000","last_undegraded":"2024-09-20T22:16:39.936505+0000","last_fullsized":"2024-09-20T22:16:39.936505+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:51.552071+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:15:51.552071+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:19.449851+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.6","version":"410'7441","reported_seq":7527,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:41.197973+0000","last_change":"2024-09-20T22:15:08.928417+0000","last_active":"2024-09-20T22:16:41.197973+0000","last_peered":"2024-09-20T22:16:41.197973+0000","last_clean":"2024-09-20T22:16:41.197973+0000","last_became_active":"2024-09-20T22:15:08.928102+0000","last_became_peered":"2024-09-20T22:15:08.928102+0000","last_unstale":"2024-09-20T22:16:41.197973+0000","last_undegraded":"2024-09-20T22:16:41.197973+0000","last_fullsized":"2024-09-20T22:16:41.197973+0000","mapping_epoch":370,"log_start":"380'2500","ondisk_log_start":"380'2500","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":4941,"log_dups_size":0,"ondisk_log_size":4941,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.751946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":487653376,"num_objects":7441,"num_object_clones":0,"num_object_copies":22323,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7441,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7441,"num_write_kb":476224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"328'27","reported_seq":309,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.939017+0000","last_change":"2024-09-20T22:13:10.353607+0000","last_active":"2024-09-20T22:16:39.939017+0000","last_peered":"2024-09-20T22:16:39.939017+0000","last_clean":"2024-09-20T22:16:39.939017+0000","last_became_active":"2024-09-20T22:11:30.493275+0000","last_became_peered":"2024-09-20T22:11:30.493275+0000","last_unstale":"2024-09-20T22:16:39.939017+0000","last_undegraded":"2024-09-20T22:16:39.939017+0000","last_fullsized":"2024-09-20T22:16:39.939017+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:13:10.353459+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:13:10.353459+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.751850+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":383,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.937821+0000","last_change":"2024-09-20T22:16:37.713822+0000","last_active":"2024-09-20T22:16:39.937821+0000","last_peered":"2024-09-20T22:16:39.937821+0000","last_clean":"2024-09-20T22:16:39.937821+0000","last_became_active":"2024-09-20T22:12:52.526551+0000","last_became_peered":"2024-09-20T22:12:52.526551+0000","last_unstale":"2024-09-20T22:16:39.937821+0000","last_undegraded":"2024-09-20T22:16:39.937821+0000","last_fullsized":"2024-09-20T22:16:39.937821+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:16:37.713713+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:16:37.713713+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:58.859262+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":916,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.937549+0000","last_change":"2024-09-20T22:16:32.744150+0000","last_active":"2024-09-20T22:16:39.937549+0000","last_peered":"2024-09-20T22:16:39.937549+0000","last_clean":"2024-09-20T22:16:39.937549+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:16:39.937549+0000","last_undegraded":"2024-09-20T22:16:39.937549+0000","last_fullsized":"2024-09-20T22:16:39.937549+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:16:32.744031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.744031+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:47.355070+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":295,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.940366+0000","last_change":"2024-09-20T22:16:21.904789+0000","last_active":"2024-09-20T22:16:39.940366+0000","last_peered":"2024-09-20T22:16:39.940366+0000","last_clean":"2024-09-20T22:16:39.940366+0000","last_became_active":"2024-09-20T22:12:52.517758+0000","last_became_peered":"2024-09-20T22:12:52.517758+0000","last_unstale":"2024-09-20T22:16:39.940366+0000","last_undegraded":"2024-09-20T22:16:39.940366+0000","last_fullsized":"2024-09-20T22:16:39.940366+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:21.904548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:21.904548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:36.763352+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"75'128","reported_seq":631,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.940088+0000","last_change":"2024-09-20T22:15:52.087940+0000","last_active":"2024-09-20T22:16:39.940088+0000","last_peered":"2024-09-20T22:16:39.940088+0000","last_clean":"2024-09-20T22:16:39.940088+0000","last_became_active":"2024-09-20T22:15:50.622954+0000","last_became_peered":"2024-09-20T22:15:50.622954+0000","last_unstale":"2024-09-20T22:16:39.940088+0000","last_undegraded":"2024-09-20T22:16:39.940088+0000","last_fullsized":"2024-09-20T22:16:39.940088+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:52.087707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:52.087707+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:07.776471+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":795,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.933463+0000","last_change":"2024-09-20T22:16:23.783338+0000","last_active":"2024-09-20T22:16:39.933463+0000","last_peered":"2024-09-20T22:16:39.933463+0000","last_clean":"2024-09-20T22:16:39.933463+0000","last_became_active":"2024-09-20T22:12:52.516989+0000","last_became_peered":"2024-09-20T22:12:52.516989+0000","last_unstale":"2024-09-20T22:16:39.933463+0000","last_undegraded":"2024-09-20T22:16:39.933463+0000","last_fullsized":"2024-09-20T22:16:39.933463+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:23.783127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:23.783127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:27.538492+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"328'12","reported_seq":797,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.944505+0000","last_change":"2024-09-20T22:16:22.641814+0000","last_active":"2024-09-20T22:16:39.944505+0000","last_peered":"2024-09-20T22:16:39.944505+0000","last_clean":"2024-09-20T22:16:39.944505+0000","last_became_active":"2024-09-20T22:10:48.922561+0000","last_became_peered":"2024-09-20T22:10:48.922561+0000","last_unstale":"2024-09-20T22:16:39.944505+0000","last_undegraded":"2024-09-20T22:16:39.944505+0000","last_fullsized":"2024-09-20T22:16:39.944505+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'12","last_scrub_stamp":"2024-09-20T22:16:22.641558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:22.641558+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:50.501207+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":283,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.939536+0000","last_change":"2024-09-20T22:15:54.065141+0000","last_active":"2024-09-20T22:16:39.939536+0000","last_peered":"2024-09-20T22:16:39.939536+0000","last_clean":"2024-09-20T22:16:39.939536+0000","last_became_active":"2024-09-20T22:11:30.487322+0000","last_became_peered":"2024-09-20T22:11:30.487322+0000","last_unstale":"2024-09-20T22:16:39.939536+0000","last_undegraded":"2024-09-20T22:16:39.939536+0000","last_fullsized":"2024-09-20T22:16:39.939536+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:54.065026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:54.065026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:01.465493+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":728,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.936686+0000","last_change":"2024-09-20T22:14:54.359687+0000","last_active":"2024-09-20T22:16:39.936686+0000","last_peered":"2024-09-20T22:16:39.936686+0000","last_clean":"2024-09-20T22:16:39.936686+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:16:39.936686+0000","last_undegraded":"2024-09-20T22:16:39.936686+0000","last_fullsized":"2024-09-20T22:16:39.936686+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:54.359586+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:54.359586+0000","last_clean_scrub_stamp":"2024-09-20T22:14:54.359586+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"75'128","reported_seq":767,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935275+0000","last_change":"2024-09-20T22:15:52.362195+0000","last_active":"2024-09-20T22:16:39.935275+0000","last_peered":"2024-09-20T22:16:39.935275+0000","last_clean":"2024-09-20T22:16:39.935275+0000","last_became_active":"2024-09-20T22:15:50.615607+0000","last_became_peered":"2024-09-20T22:15:50.615607+0000","last_unstale":"2024-09-20T22:16:39.935275+0000","last_undegraded":"2024-09-20T22:16:39.935275+0000","last_fullsized":"2024-09-20T22:16:39.935275+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:52.362111+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:15:52.362111+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:59.380973+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"328'15","reported_seq":219,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935101+0000","last_change":"2024-09-20T22:15:59.144712+0000","last_active":"2024-09-20T22:16:39.935101+0000","last_peered":"2024-09-20T22:16:39.935101+0000","last_clean":"2024-09-20T22:16:39.935101+0000","last_became_active":"2024-09-20T22:12:52.527106+0000","last_became_peered":"2024-09-20T22:12:52.527106+0000","last_unstale":"2024-09-20T22:16:39.935101+0000","last_undegraded":"2024-09-20T22:16:39.935101+0000","last_fullsized":"2024-09-20T22:16:39.935101+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:15:59.144584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:59.144584+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:02.167431+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":748,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.937958+0000","last_change":"2024-09-20T22:15:28.374528+0000","last_active":"2024-09-20T22:16:39.937958+0000","last_peered":"2024-09-20T22:16:39.937958+0000","last_clean":"2024-09-20T22:16:39.937958+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:16:39.937958+0000","last_undegraded":"2024-09-20T22:16:39.937958+0000","last_fullsized":"2024-09-20T22:16:39.937958+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:28.374414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:28.374414+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":386,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.936774+0000","last_change":"2024-09-20T22:16:17.270484+0000","last_active":"2024-09-20T22:16:39.936774+0000","last_peered":"2024-09-20T22:16:39.936774+0000","last_clean":"2024-09-20T22:16:39.936774+0000","last_became_active":"2024-09-20T22:12:33.545506+0000","last_became_peered":"2024-09-20T22:12:33.545506+0000","last_unstale":"2024-09-20T22:16:39.936774+0000","last_undegraded":"2024-09-20T22:16:39.936774+0000","last_fullsized":"2024-09-20T22:16:39.936774+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:17.270376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_clean_scrub_stamp":"2024-09-20T22:16:17.270376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:17.773205+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":767,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.939228+0000","last_change":"2024-09-20T22:16:26.862388+0000","last_active":"2024-09-20T22:16:39.939228+0000","last_peered":"2024-09-20T22:16:39.939228+0000","last_clean":"2024-09-20T22:16:39.939228+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:16:39.939228+0000","last_undegraded":"2024-09-20T22:16:39.939228+0000","last_fullsized":"2024-09-20T22:16:39.939228+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:26.862279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:26.862279+0000","last_clean_scrub_stamp":"2024-09-20T22:16:26.862279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:33.809107+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"75'128","reported_seq":631,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.938842+0000","last_change":"2024-09-20T22:15:51.068931+0000","last_active":"2024-09-20T22:16:39.938842+0000","last_peered":"2024-09-20T22:16:39.938842+0000","last_clean":"2024-09-20T22:16:39.938842+0000","last_became_active":"2024-09-20T22:15:50.574865+0000","last_became_peered":"2024-09-20T22:15:50.574865+0000","last_unstale":"2024-09-20T22:16:39.938842+0000","last_undegraded":"2024-09-20T22:16:39.938842+0000","last_fullsized":"2024-09-20T22:16:39.938842+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:51.068807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:51.068807+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:04.679167+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"328'15","reported_seq":302,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.939340+0000","last_change":"2024-09-20T22:15:57.097701+0000","last_active":"2024-09-20T22:16:39.939340+0000","last_peered":"2024-09-20T22:16:39.939340+0000","last_clean":"2024-09-20T22:16:39.939340+0000","last_became_active":"2024-09-20T22:11:30.493335+0000","last_became_peered":"2024-09-20T22:11:30.493335+0000","last_unstale":"2024-09-20T22:16:39.939340+0000","last_undegraded":"2024-09-20T22:16:39.939340+0000","last_fullsized":"2024-09-20T22:16:39.939340+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:15:57.097563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.097563+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:04.507802+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":208,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935163+0000","last_change":"2024-09-20T22:16:27.938805+0000","last_active":"2024-09-20T22:16:39.935163+0000","last_peered":"2024-09-20T22:16:39.935163+0000","last_clean":"2024-09-20T22:16:39.935163+0000","last_became_active":"2024-09-20T22:12:52.524149+0000","last_became_peered":"2024-09-20T22:12:52.524149+0000","last_unstale":"2024-09-20T22:16:39.935163+0000","last_undegraded":"2024-09-20T22:16:39.935163+0000","last_fullsized":"2024-09-20T22:16:39.935163+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:27.938677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:56.538825+0000","last_clean_scrub_stamp":"2024-09-20T22:16:27.938677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:44.662705+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"315'11","reported_seq":294,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.938299+0000","last_change":"2024-09-20T22:15:16.396532+0000","last_active":"2024-09-20T22:16:39.938299+0000","last_peered":"2024-09-20T22:16:39.938299+0000","last_clean":"2024-09-20T22:16:39.938299+0000","last_became_active":"2024-09-20T22:12:33.544098+0000","last_became_peered":"2024-09-20T22:12:33.544098+0000","last_unstale":"2024-09-20T22:16:39.938299+0000","last_undegraded":"2024-09-20T22:16:39.938299+0000","last_fullsized":"2024-09-20T22:16:39.938299+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:15:16.396454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:16.396454+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":748,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.931204+0000","last_change":"2024-09-20T22:15:30.093649+0000","last_active":"2024-09-20T22:16:39.931204+0000","last_peered":"2024-09-20T22:16:39.931204+0000","last_clean":"2024-09-20T22:16:39.931204+0000","last_became_active":"2024-09-20T22:11:30.484731+0000","last_became_peered":"2024-09-20T22:11:30.484731+0000","last_unstale":"2024-09-20T22:16:39.931204+0000","last_undegraded":"2024-09-20T22:16:39.931204+0000","last_fullsized":"2024-09-20T22:16:39.931204+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:30.093548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:30.093548+0000","last_clean_scrub_stamp":"2024-09-20T22:15:30.093548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:55.627047+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"75'128","reported_seq":647,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.938692+0000","last_change":"2024-09-20T22:15:51.077313+0000","last_active":"2024-09-20T22:16:39.938692+0000","last_peered":"2024-09-20T22:16:39.938692+0000","last_clean":"2024-09-20T22:16:39.938692+0000","last_became_active":"2024-09-20T22:15:50.612544+0000","last_became_peered":"2024-09-20T22:15:50.612544+0000","last_unstale":"2024-09-20T22:16:39.938692+0000","last_undegraded":"2024-09-20T22:16:39.938692+0000","last_fullsized":"2024-09-20T22:16:39.938692+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:51.077187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:51.077187+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:18.801109+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"328'20","reported_seq":237,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.937139+0000","last_change":"2024-09-20T22:16:37.280708+0000","last_active":"2024-09-20T22:16:39.937139+0000","last_peered":"2024-09-20T22:16:39.937139+0000","last_clean":"2024-09-20T22:16:39.937139+0000","last_became_active":"2024-09-20T22:12:44.631986+0000","last_became_peered":"2024-09-20T22:12:44.631986+0000","last_unstale":"2024-09-20T22:16:39.937139+0000","last_undegraded":"2024-09-20T22:16:39.937139+0000","last_fullsized":"2024-09-20T22:16:39.937139+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:16:37.280566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:37.280566+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:04.676364+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":748,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.936234+0000","last_change":"2024-09-20T22:16:31.755794+0000","last_active":"2024-09-20T22:16:39.936234+0000","last_peered":"2024-09-20T22:16:39.936234+0000","last_clean":"2024-09-20T22:16:39.936234+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:16:39.936234+0000","last_undegraded":"2024-09-20T22:16:39.936234+0000","last_fullsized":"2024-09-20T22:16:39.936234+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:31.755589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.755589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:47.093311+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"326'14","reported_seq":370,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.939171+0000","last_change":"2024-09-20T22:15:19.186947+0000","last_active":"2024-09-20T22:16:39.939171+0000","last_peered":"2024-09-20T22:16:39.939171+0000","last_clean":"2024-09-20T22:16:39.939171+0000","last_became_active":"2024-09-20T22:12:52.518770+0000","last_became_peered":"2024-09-20T22:12:52.518770+0000","last_unstale":"2024-09-20T22:16:39.939171+0000","last_undegraded":"2024-09-20T22:16:39.939171+0000","last_fullsized":"2024-09-20T22:16:39.939171+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'14","last_scrub_stamp":"2024-09-20T22:15:19.186862+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:19.186862+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:43.532138+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":200,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.934702+0000","last_change":"2024-09-20T22:15:22.741319+0000","last_active":"2024-09-20T22:16:39.934702+0000","last_peered":"2024-09-20T22:16:39.934702+0000","last_clean":"2024-09-20T22:16:39.934702+0000","last_became_active":"2024-09-20T22:12:52.523781+0000","last_became_peered":"2024-09-20T22:12:52.523781+0000","last_unstale":"2024-09-20T22:16:39.934702+0000","last_undegraded":"2024-09-20T22:16:39.934702+0000","last_fullsized":"2024-09-20T22:16:39.934702+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:22.741214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:22.741214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.914127+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"328'19","reported_seq":804,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.930633+0000","last_change":"2024-09-20T22:16:28.533330+0000","last_active":"2024-09-20T22:16:39.930633+0000","last_peered":"2024-09-20T22:16:39.930633+0000","last_clean":"2024-09-20T22:16:39.930633+0000","last_became_active":"2024-09-20T22:08:22.139581+0000","last_became_peered":"2024-09-20T22:08:22.139581+0000","last_unstale":"2024-09-20T22:16:39.930633+0000","last_undegraded":"2024-09-20T22:16:39.930633+0000","last_fullsized":"2024-09-20T22:16:39.930633+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:16:28.533201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:28.533201+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:50.375092+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":748,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.932096+0000","last_change":"2024-09-20T22:15:20.031722+0000","last_active":"2024-09-20T22:16:39.932096+0000","last_peered":"2024-09-20T22:16:39.932096+0000","last_clean":"2024-09-20T22:16:39.932096+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:16:39.932096+0000","last_undegraded":"2024-09-20T22:16:39.932096+0000","last_fullsized":"2024-09-20T22:16:39.932096+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.031625+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.031625+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.566217+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":585,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.934374+0000","last_change":"2024-09-20T22:16:32.216071+0000","last_active":"2024-09-20T22:16:39.934374+0000","last_peered":"2024-09-20T22:16:39.934374+0000","last_clean":"2024-09-20T22:16:39.934374+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:16:39.934374+0000","last_undegraded":"2024-09-20T22:16:39.934374+0000","last_fullsized":"2024-09-20T22:16:39.934374+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:32.215758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.215758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:00.249999+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":588,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.933729+0000","last_change":"2024-09-20T22:14:18.440962+0000","last_active":"2024-09-20T22:16:39.933729+0000","last_peered":"2024-09-20T22:16:39.933729+0000","last_clean":"2024-09-20T22:16:39.933729+0000","last_became_active":"2024-09-20T22:12:52.521497+0000","last_became_peered":"2024-09-20T22:12:52.521497+0000","last_unstale":"2024-09-20T22:16:39.933729+0000","last_undegraded":"2024-09-20T22:16:39.933729+0000","last_fullsized":"2024-09-20T22:16:39.933729+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:18.438451+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:14:18.438451+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:43.233673+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"328'19","reported_seq":662,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.934611+0000","last_change":"2024-09-20T22:14:12.410224+0000","last_active":"2024-09-20T22:16:39.934611+0000","last_peered":"2024-09-20T22:16:39.934611+0000","last_clean":"2024-09-20T22:16:39.934611+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:16:39.934611+0000","last_undegraded":"2024-09-20T22:16:39.934611+0000","last_fullsized":"2024-09-20T22:16:39.934611+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:14:12.410107+0000","last_deep_scrub":"291'13","last_deep_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_clean_scrub_stamp":"2024-09-20T22:14:12.410107+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:43.233768+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":345,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.936599+0000","last_change":"2024-09-20T22:15:54.609470+0000","last_active":"2024-09-20T22:16:39.936599+0000","last_peered":"2024-09-20T22:16:39.936599+0000","last_clean":"2024-09-20T22:16:39.936599+0000","last_became_active":"2024-09-20T22:10:30.655502+0000","last_became_peered":"2024-09-20T22:10:30.655502+0000","last_unstale":"2024-09-20T22:16:39.936599+0000","last_undegraded":"2024-09-20T22:16:39.936599+0000","last_fullsized":"2024-09-20T22:16:39.936599+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:54.609342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:54.609342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:54.717401+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":200,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.932506+0000","last_change":"2024-09-20T22:15:33.868714+0000","last_active":"2024-09-20T22:16:39.932506+0000","last_peered":"2024-09-20T22:16:39.932506+0000","last_clean":"2024-09-20T22:16:39.932506+0000","last_became_active":"2024-09-20T22:12:52.516953+0000","last_became_peered":"2024-09-20T22:12:52.516953+0000","last_unstale":"2024-09-20T22:16:39.932506+0000","last_undegraded":"2024-09-20T22:16:39.932506+0000","last_fullsized":"2024-09-20T22:16:39.932506+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:33.868591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:33.868591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:54.347076+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":200,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.932590+0000","last_change":"2024-09-20T22:15:20.820202+0000","last_active":"2024-09-20T22:16:39.932590+0000","last_peered":"2024-09-20T22:16:39.932590+0000","last_clean":"2024-09-20T22:16:39.932590+0000","last_became_active":"2024-09-20T22:12:52.514247+0000","last_became_peered":"2024-09-20T22:12:52.514247+0000","last_unstale":"2024-09-20T22:16:39.932590+0000","last_undegraded":"2024-09-20T22:16:39.932590+0000","last_fullsized":"2024-09-20T22:16:39.932590+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.819992+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.819992+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.914152+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"75'128","reported_seq":796,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.938979+0000","last_change":"2024-09-20T22:15:50.611775+0000","last_active":"2024-09-20T22:16:39.938979+0000","last_peered":"2024-09-20T22:16:39.938979+0000","last_clean":"2024-09-20T22:16:39.938979+0000","last_became_active":"2024-09-20T22:15:50.611421+0000","last_became_peered":"2024-09-20T22:15:50.611421+0000","last_unstale":"2024-09-20T22:16:39.938979+0000","last_undegraded":"2024-09-20T22:16:39.938979+0000","last_fullsized":"2024-09-20T22:16:39.938979+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:45.920357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:45.920357+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:46.078893+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"328'19","reported_seq":673,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.937709+0000","last_change":"2024-09-20T22:16:21.625991+0000","last_active":"2024-09-20T22:16:39.937709+0000","last_peered":"2024-09-20T22:16:39.937709+0000","last_clean":"2024-09-20T22:16:39.937709+0000","last_became_active":"2024-09-20T22:12:33.545199+0000","last_became_peered":"2024-09-20T22:12:33.545199+0000","last_unstale":"2024-09-20T22:16:39.937709+0000","last_undegraded":"2024-09-20T22:16:39.937709+0000","last_fullsized":"2024-09-20T22:16:39.937709+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:16:21.625915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:21.625915+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:47.759718+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":286,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.939450+0000","last_change":"2024-09-20T22:15:27.149844+0000","last_active":"2024-09-20T22:16:39.939450+0000","last_peered":"2024-09-20T22:16:39.939450+0000","last_clean":"2024-09-20T22:16:39.939450+0000","last_became_active":"2024-09-20T22:11:30.493598+0000","last_became_peered":"2024-09-20T22:11:30.493598+0000","last_unstale":"2024-09-20T22:16:39.939450+0000","last_undegraded":"2024-09-20T22:16:39.939450+0000","last_fullsized":"2024-09-20T22:16:39.939450+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:27.149777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.149777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.752018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":740,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.936077+0000","last_change":"2024-09-20T22:14:55.338392+0000","last_active":"2024-09-20T22:16:39.936077+0000","last_peered":"2024-09-20T22:16:39.936077+0000","last_clean":"2024-09-20T22:16:39.936077+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:16:39.936077+0000","last_undegraded":"2024-09-20T22:16:39.936077+0000","last_fullsized":"2024-09-20T22:16:39.936077+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:55.338318+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:55.338318+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":197,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.932704+0000","last_change":"2024-09-20T22:16:20.113927+0000","last_active":"2024-09-20T22:16:39.932704+0000","last_peered":"2024-09-20T22:16:39.932704+0000","last_clean":"2024-09-20T22:16:39.932704+0000","last_became_active":"2024-09-20T22:12:57.883998+0000","last_became_peered":"2024-09-20T22:12:57.883998+0000","last_unstale":"2024-09-20T22:16:39.932704+0000","last_undegraded":"2024-09-20T22:16:39.932704+0000","last_fullsized":"2024-09-20T22:16:39.932704+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:20.113814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:20.113814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:26.993977+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"328'8","reported_seq":764,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.944747+0000","last_change":"2024-09-20T22:15:47.882115+0000","last_active":"2024-09-20T22:16:39.944747+0000","last_peered":"2024-09-20T22:16:39.944747+0000","last_clean":"2024-09-20T22:16:39.944747+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:16:39.944747+0000","last_undegraded":"2024-09-20T22:16:39.944747+0000","last_fullsized":"2024-09-20T22:16:39.944747+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'8","last_scrub_stamp":"2024-09-20T22:15:47.881989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:47.881989+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:00.408284+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":200,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.933616+0000","last_change":"2024-09-20T22:15:50.069823+0000","last_active":"2024-09-20T22:16:39.933616+0000","last_peered":"2024-09-20T22:16:39.933616+0000","last_clean":"2024-09-20T22:16:39.933616+0000","last_became_active":"2024-09-20T22:12:52.517510+0000","last_became_peered":"2024-09-20T22:12:52.517510+0000","last_unstale":"2024-09-20T22:16:39.933616+0000","last_undegraded":"2024-09-20T22:16:39.933616+0000","last_fullsized":"2024-09-20T22:16:39.933616+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:50.069659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:50.069659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:18.119042+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":771,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.933429+0000","last_change":"2024-09-20T22:16:37.743248+0000","last_active":"2024-09-20T22:16:39.933429+0000","last_peered":"2024-09-20T22:16:39.933429+0000","last_clean":"2024-09-20T22:16:39.933429+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:16:39.933429+0000","last_undegraded":"2024-09-20T22:16:39.933429+0000","last_fullsized":"2024-09-20T22:16:39.933429+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:37.743137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:37.743137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:04.711878+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":736,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935528+0000","last_change":"2024-09-20T22:15:29.393451+0000","last_active":"2024-09-20T22:16:39.935528+0000","last_peered":"2024-09-20T22:16:39.935528+0000","last_clean":"2024-09-20T22:16:39.935528+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:16:39.935528+0000","last_undegraded":"2024-09-20T22:16:39.935528+0000","last_fullsized":"2024-09-20T22:16:39.935528+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:29.393342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_clean_scrub_stamp":"2024-09-20T22:15:29.393342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":633,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935797+0000","last_change":"2024-09-20T22:15:49.314143+0000","last_active":"2024-09-20T22:16:39.935797+0000","last_peered":"2024-09-20T22:16:39.935797+0000","last_clean":"2024-09-20T22:16:39.935797+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:16:39.935797+0000","last_undegraded":"2024-09-20T22:16:39.935797+0000","last_fullsized":"2024-09-20T22:16:39.935797+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:49.314024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:49.314024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:18.675580+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":200,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.935544+0000","last_change":"2024-09-20T22:15:18.828980+0000","last_active":"2024-09-20T22:16:39.935544+0000","last_peered":"2024-09-20T22:16:39.935544+0000","last_clean":"2024-09-20T22:16:39.935544+0000","last_became_active":"2024-09-20T22:12:52.518076+0000","last_became_peered":"2024-09-20T22:12:52.518076+0000","last_unstale":"2024-09-20T22:16:39.935544+0000","last_undegraded":"2024-09-20T22:16:39.935544+0000","last_fullsized":"2024-09-20T22:16:39.935544+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:18.828903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:18.828903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:44.913983+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"328'19","reported_seq":792,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.938935+0000","last_change":"2024-09-20T22:15:57.649394+0000","last_active":"2024-09-20T22:16:39.938935+0000","last_peered":"2024-09-20T22:16:39.938935+0000","last_clean":"2024-09-20T22:16:39.938935+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:16:39.938935+0000","last_undegraded":"2024-09-20T22:16:39.938935+0000","last_fullsized":"2024-09-20T22:16:39.938935+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:15:57.649159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.649159+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:08.717893+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":200,"reported_epoch":410,"state":"active+clean","last_fresh":"2024-09-20T22:16:39.934607+0000","last_change":"2024-09-20T22:15:27.761950+0000","last_active":"2024-09-20T22:16:39.934607+0000","last_peered":"2024-09-20T22:16:39.934607+0000","last_clean":"2024-09-20T22:16:39.934607+0000","last_became_active":"2024-09-20T22:12:52.524425+0000","last_became_peered":"2024-09-20T22:12:52.524425+0000","last_unstale":"2024-09-20T22:16:39.934607+0000","last_undegraded":"2024-09-20T22:16:39.934607+0000","last_fullsized":"2024-09-20T22:16:39.934607+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:27.761843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.761843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:16:46.763986+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":7734624256,"num_objects":118021,"num_object_clones":0,"num_object_copies":354063,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":118021,"num_write_kb":7553344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":23126867968,"data_stored":23126867968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79121,"ondisk_log_size":79121,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1117,"num_read_kb":874,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":170,"num_bytes_recovered":3300,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":731,"ondisk_log_size":731,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":806,"num_read_kb":801,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2688,"ondisk_log_size":2688,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813136042,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2503332,"kb_used_data":1858512,"kb_used_omap":1,"kb_used_meta":644798,"kb_avail":91241820,"statfs":{"total":95995035648,"available":93431623680,"internally_reserved":0,"allocated":1903116288,"data_stored":1902636805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":660273610},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430380,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3814796,"kb_used_data":3324092,"kb_used_omap":1,"kb_used_meta":490686,"kb_avail":89930356,"statfs":{"total":95995035648,"available":92088684544,"internally_reserved":0,"allocated":3403870208,"data_stored":3403411003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":502462918},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724722,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3013576,"kb_used_data":2346552,"kb_used_omap":1,"kb_used_meta":667006,"kb_avail":90731576,"statfs":{"total":95995035648,"available":92909133824,"internally_reserved":0,"allocated":2402869248,"data_stored":2402413058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":683014597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986362,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":4275296,"kb_used_data":3712400,"kb_used_omap":1,"kb_used_meta":562878,"kb_avail":89469856,"statfs":{"total":95995035648,"available":91617132544,"internally_reserved":0,"allocated":3801497600,"data_stored":3801017406,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":576387525},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770258039,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1930356,"kb_used_data":1439780,"kb_used_omap":4,"kb_used_meta":490555,"kb_avail":91814796,"statfs":{"total":95995035648,"available":94018351104,"internally_reserved":0,"allocated":1474334720,"data_stored":1473835343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":502329000},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149887,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3041328,"kb_used_data":2342368,"kb_used_omap":1,"kb_used_meta":698942,"kb_avail":90703824,"statfs":{"total":95995035648,"available":92880715776,"internally_reserved":0,"allocated":2398584832,"data_stored":2398089546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":715717063},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411524,"num_pgs":51,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3397260,"kb_used_data":2788536,"kb_used_omap":1,"kb_used_meta":608702,"kb_avail":90347892,"statfs":{"total":95995035648,"available":92516241408,"internally_reserved":0,"allocated":2855460864,"data_stored":2855005117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":623311301},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738575,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5435484,"kb_used_data":4714252,"kb_used_omap":1,"kb_used_meta":721150,"kb_avail":88309668,"statfs":{"total":95995035648,"available":90429100032,"internally_reserved":0,"allocated":4827394048,"data_stored":4826918139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":738458053},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4841799680,"data_stored":4841799680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1476919296,"data_stored":1476919296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2863529984,"data_stored":2863529984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":2400059392,"data_stored":2400059392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":3812425728,"data_stored":3812425728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2413559808,"data_stored":2413559808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3413245952,"data_stored":3413245952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1905328128,"data_stored":1905328128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:16:47.023 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-20T22:16:47.023 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:16:47.023 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-20T22:16:47.024 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-20T22:16:47.024 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-20T22:16:47.024 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 config set filestore_inject_stall 3 2024-09-20T22:16:47.066 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:47.079 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:16:47.079 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:16:47.105 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:47.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:47.187 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:47.288 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:47.293 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:47.332 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:47.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:47.334 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:47.434 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:47.474 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:47.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:47.474 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:47.499 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:16:47.575 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:47.617 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:47.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:47.620 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:47.720 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:47.762 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:47.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:47.763 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:47.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:47 smithi060 ceph-mon[29694]: pgmap v744: 168 pgs: 168 active+clean; 7.3 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 59 MiB/s wr, 951 op/s 2024-09-20T22:16:47.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:47 smithi060 ceph-mon[29694]: osdmap e412: 8 total, 8 up, 8 in 2024-09-20T22:16:47.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:47 smithi060 ceph-mon[40290]: pgmap v744: 168 pgs: 168 active+clean; 7.3 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 59 MiB/s wr, 951 op/s 2024-09-20T22:16:47.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:47 smithi060 ceph-mon[40290]: osdmap e412: 8 total, 8 up, 8 in 2024-09-20T22:16:47.848 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T22:16:47.863 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:47.900 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:47.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:47.901 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:47.934 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Total time run: 90.0081 2024-09-20T22:16:47.934 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Total writes made: 124789 2024-09-20T22:16:47.934 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Write size: 65536 2024-09-20T22:16:47.934 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Object size: 65536 2024-09-20T22:16:47.935 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Bandwidth (MB/sec): 86.6512 2024-09-20T22:16:47.935 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Stddev Bandwidth: 33.2468 2024-09-20T22:16:47.935 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Max bandwidth (MB/sec): 224.5 2024-09-20T22:16:47.935 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Min bandwidth (MB/sec): 51.25 2024-09-20T22:16:47.935 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Average IOPS: 1386 2024-09-20T22:16:47.935 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Stddev IOPS: 531.949 2024-09-20T22:16:47.935 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Max IOPS: 3592 2024-09-20T22:16:47.935 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Min IOPS: 820 2024-09-20T22:16:47.935 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Average Latency(s): 0.0115364 2024-09-20T22:16:47.935 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Stddev Latency(s): 0.0114901 2024-09-20T22:16:47.935 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Max latency(s): 0.320133 2024-09-20T22:16:47.935 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Min latency(s): 0.00137086 2024-09-20T22:16:47.936 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:47 smithi067 ceph-mon[33511]: pgmap v744: 168 pgs: 168 active+clean; 7.3 GiB data, 27 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 59 MiB/s wr, 951 op/s 2024-09-20T22:16:47.936 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:47 smithi067 ceph-mon[33511]: osdmap e412: 8 total, 8 up, 8 in 2024-09-20T22:16:47.940 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Cleaning up (deleting benchmark objects) 2024-09-20T22:16:48.002 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:48.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:48.054 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:48.054 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:48.155 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:48.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:48.203 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:48.204 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:48.270 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:16:48.271 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-09-20T22:16:48.271 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:16:48.304 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:48.306 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:16:48.341 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:48.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:48.343 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:48.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:48.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:48.500 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:48.500 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:48.515 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:48.522 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:16:48.522 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:16:48.562 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:16:48.562 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:16:48.601 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:48.605 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:16:48.606 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:16:48.628 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:48.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:48.629 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:48.650 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:16:48.650 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:16:48.683 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:16:48.683 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:48 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1137617907' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:48.683 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:48 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1137617907' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:48.683 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:16:48.716 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:16:48.716 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:16:48.730 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:48.750 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:16:48.750 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:16:48.766 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:48.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:48.767 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:48.784 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:16:48.784 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:16:48.867 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:48.913 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:48.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:48.914 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:49.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:48 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1137617907' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:49.015 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:49.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:49.052 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:49.052 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:49.153 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:49.199 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:49.199 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:49.200 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:49.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:49.320 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:16:49.321 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:16:49.321 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:16:49.321 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:16:49.321 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:16:49.335 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:49.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:49.336 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:49.437 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:49.471 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:49.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:49.472 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:49.573 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:49.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:49.618 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:49.618 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:49.718 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:49.753 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:49.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:49.754 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:49.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:49 smithi060 ceph-mon[29694]: pgmap v746: 168 pgs: 168 active+clean; 7.5 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.4 KiB/s rd, 85 MiB/s wr, 1.37k op/s 2024-09-20T22:16:49.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:49 smithi060 ceph-mon[40290]: pgmap v746: 168 pgs: 168 active+clean; 7.5 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.4 KiB/s rd, 85 MiB/s wr, 1.37k op/s 2024-09-20T22:16:49.854 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:49.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:49.891 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:49.891 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:49.992 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:50.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:49 smithi067 ceph-mon[33511]: pgmap v746: 168 pgs: 168 active+clean; 7.5 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.4 KiB/s rd, 85 MiB/s wr, 1.37k op/s 2024-09-20T22:16:50.036 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:50.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:50.038 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:50.138 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:50.172 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:50.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:50.173 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:50.273 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:50.295 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:50.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:50.296 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:50.396 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:50.429 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:50.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:50.430 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:50.531 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:50.561 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:50.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:50.562 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:50.663 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:50.701 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:50.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:50.702 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:50.803 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:50.837 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:50.838 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:50.838 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:50.939 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:50.960 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:50.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:50.961 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:51.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:50 smithi067 ceph-mon[33511]: pgmap v747: 168 pgs: 168 active+clean; 7.5 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 767 B/s rd, 63 MiB/s wr, 1.02k op/s 2024-09-20T22:16:51.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:50 smithi060 ceph-mon[29694]: pgmap v747: 168 pgs: 168 active+clean; 7.5 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 767 B/s rd, 63 MiB/s wr, 1.02k op/s 2024-09-20T22:16:51.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:50 smithi060 ceph-mon[40290]: pgmap v747: 168 pgs: 168 active+clean; 7.5 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 767 B/s rd, 63 MiB/s wr, 1.02k op/s 2024-09-20T22:16:51.062 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:51.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:51.099 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:51.100 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:51.200 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:51.237 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:51.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:51.238 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:51.339 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:51.360 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:51.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:51.361 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:51.462 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:51.498 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:51.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:51.499 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:51.599 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:51.632 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:51.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:51.633 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:51.734 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:51.766 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:51.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:51.768 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:51.868 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:51.900 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:16:51.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:51.904 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:51.904 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:52.005 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:52.030 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:52.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:52.031 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:52.131 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:52.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:52.165 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:52.166 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:52.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:16:52.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:16:52.266 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:52.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:16:52.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:16:52.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:16:52.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:16:52.306 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:52.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:52.307 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:52.408 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:52.454 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:52.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:52.456 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:52.556 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:52.592 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:52.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:52.594 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:52.694 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:52.729 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:52.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:52.731 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:52.831 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:52.861 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:52.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:52.862 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:52.942 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:16:52 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:16:52.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:53.007 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:53.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:53.008 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:53.109 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:53.139 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:53.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:53.140 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:53.207 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:52 smithi060 ceph-mon[29694]: pgmap v748: 168 pgs: 168 active+clean; 7.5 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.4 KiB/s rd, 52 MiB/s wr, 1.06k op/s 2024-09-20T22:16:53.207 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:52 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:16:53.207 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:52 smithi060 ceph-mon[40290]: pgmap v748: 168 pgs: 168 active+clean; 7.5 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.4 KiB/s rd, 52 MiB/s wr, 1.06k op/s 2024-09-20T22:16:53.208 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:52 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:16:53.241 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:53.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:52 smithi067 ceph-mon[33511]: pgmap v748: 168 pgs: 168 active+clean; 7.5 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.4 KiB/s rd, 52 MiB/s wr, 1.06k op/s 2024-09-20T22:16:53.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:52 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:16:53.280 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:53.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:53.281 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:53.382 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:53.419 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:53.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:53.421 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:53.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:53.559 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:53.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:53.560 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:53.661 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:53.692 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:53.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:53.693 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:53.709 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T22:16:53.793 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:53.826 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:53.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:53.827 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:53.863 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:53.869 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:53.927 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:53.957 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:53.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:53.958 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:54.059 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:54.101 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:16:54.101 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:16:54.102 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-20T22:16:54.102 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd test-reweight-by-utilization 1000 0.05 2024-09-20T22:16:54.113 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:54.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:54.115 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:54.215 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:54.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:54.271 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:54.271 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:54.372 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:16:54.417 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:54.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:54.417 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:16:54.518 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:54.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:54.557 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:54.557 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:54.658 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:54.690 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:54.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:54.691 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:54.788 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:16:54.788 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:16:54.792 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:54.836 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:54.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:54.837 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:54.842 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:16:54.842 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:16:54.882 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:16:54.882 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:16:54.925 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:16:54.925 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:16:54.938 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:54.964 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:54.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:54.965 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:16:54.965 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:16:54.966 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:54.997 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:55.003 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:16:55.003 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:16:55.046 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:16:55.046 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:16:55.057 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:55.066 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:55.089 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:16:55.090 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:16:55.128 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:16:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:16:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:16:55.134 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:55.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:55.135 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:55.235 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:55.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:55.274 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:55.275 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:55.333 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:16:55.375 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:55.390 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:55 smithi060 ceph-mon[29694]: pgmap v749: 168 pgs: 168 active+clean; 7.3 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.2 KiB/s rd, 47 MiB/s wr, 1.34k op/s 2024-09-20T22:16:55.390 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:55 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4153148320' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:55.390 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:55 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1107624616' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:16:55.390 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:55 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:16:55.390 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:55 smithi060 ceph-mon[40290]: pgmap v749: 168 pgs: 168 active+clean; 7.3 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.2 KiB/s rd, 47 MiB/s wr, 1.34k op/s 2024-09-20T22:16:55.390 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:55 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4153148320' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:55.390 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:55 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1107624616' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:16:55.390 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:55 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:16:55.407 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:55.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:55.408 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:55.508 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:55.542 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:55.543 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:55.543 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:55.644 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:55.680 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:55.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:55.681 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:55 smithi067 ceph-mon[33511]: pgmap v749: 168 pgs: 168 active+clean; 7.3 GiB data, 27 GiB used, 688 GiB / 715 GiB avail; 1.2 KiB/s rd, 47 MiB/s wr, 1.34k op/s 2024-09-20T22:16:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4153148320' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:16:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1107624616' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:16:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:55 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:16:55.782 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:55.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:55.823 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:55.824 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:55.924 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:55.962 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:55.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:55.963 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:56.064 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:56.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:56.098 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:56.099 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:56.199 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:56.239 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:56.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:56.240 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:56.341 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:56.373 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:56.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:56.374 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:56.474 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:56.507 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:56.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:56.508 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:56.524 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:56 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:16:56.524 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:56 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:16:56.524 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:56 smithi060 ceph-mon[29694]: osdmap e413: 8 total, 8 up, 8 in 2024-09-20T22:16:56.524 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:56 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:16:56.524 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:56 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:16:56.524 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:56 smithi060 ceph-mon[40290]: osdmap e413: 8 total, 8 up, 8 in 2024-09-20T22:16:56.609 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:56.644 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:56.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:56.645 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:56.746 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:56 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:16:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:56 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:16:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:56 smithi067 ceph-mon[33511]: osdmap e413: 8 total, 8 up, 8 in 2024-09-20T22:16:56.779 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:56.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:56.780 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:56.881 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:56.920 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:56.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:56.922 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:57.022 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:57.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:57.058 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:57.059 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:57.082 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:16:57.082 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:16:57.160 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:57.196 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:57.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:57.197 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:57.297 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:57.329 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:57.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:57.330 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:57.431 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:57.470 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:57.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:57.471 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:57.572 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:16:57.607 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:57.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:57.608 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:16:57.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:57 smithi060 ceph-mon[29694]: 5.15 scrub starts 2024-09-20T22:16:57.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:57 smithi060 ceph-mon[29694]: 5.15 scrub ok 2024-09-20T22:16:57.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:57 smithi060 ceph-mon[29694]: pgmap v751: 168 pgs: 168 active+clean; 7.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 28 MiB/s wr, 1.17k op/s 2024-09-20T22:16:57.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:57 smithi060 ceph-mon[29694]: 3.14 scrub starts 2024-09-20T22:16:57.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:57 smithi060 ceph-mon[29694]: 3.14 scrub ok 2024-09-20T22:16:57.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:57 smithi060 ceph-mon[29694]: osdmap e414: 8 total, 8 up, 8 in 2024-09-20T22:16:57.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:57 smithi060 ceph-mon[29694]: 10.0 scrub starts 2024-09-20T22:16:57.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:57 smithi060 ceph-mon[29694]: 3.18 scrub starts 2024-09-20T22:16:57.698 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:57 smithi060 ceph-mon[29694]: 3.18 scrub ok 2024-09-20T22:16:57.699 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:57 smithi060 ceph-mon[40290]: 5.15 scrub starts 2024-09-20T22:16:57.699 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:57 smithi060 ceph-mon[40290]: 5.15 scrub ok 2024-09-20T22:16:57.699 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:57 smithi060 ceph-mon[40290]: pgmap v751: 168 pgs: 168 active+clean; 7.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 28 MiB/s wr, 1.17k op/s 2024-09-20T22:16:57.699 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:57 smithi060 ceph-mon[40290]: 3.14 scrub starts 2024-09-20T22:16:57.699 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:57 smithi060 ceph-mon[40290]: 3.14 scrub ok 2024-09-20T22:16:57.699 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:57 smithi060 ceph-mon[40290]: osdmap e414: 8 total, 8 up, 8 in 2024-09-20T22:16:57.699 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:57 smithi060 ceph-mon[40290]: 10.0 scrub starts 2024-09-20T22:16:57.699 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:57 smithi060 ceph-mon[40290]: 3.18 scrub starts 2024-09-20T22:16:57.699 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:57 smithi060 ceph-mon[40290]: 3.18 scrub ok 2024-09-20T22:16:57.708 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:57.745 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:57.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:57.746 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:57.748 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T22:16:57.847 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:57.885 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:57.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:57.886 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:57.987 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:57 smithi067 ceph-mon[33511]: 5.15 scrub starts 2024-09-20T22:16:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:57 smithi067 ceph-mon[33511]: 5.15 scrub ok 2024-09-20T22:16:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:57 smithi067 ceph-mon[33511]: pgmap v751: 168 pgs: 168 active+clean; 7.2 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.1 KiB/s rd, 28 MiB/s wr, 1.17k op/s 2024-09-20T22:16:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:57 smithi067 ceph-mon[33511]: 3.14 scrub starts 2024-09-20T22:16:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:57 smithi067 ceph-mon[33511]: 3.14 scrub ok 2024-09-20T22:16:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:57 smithi067 ceph-mon[33511]: osdmap e414: 8 total, 8 up, 8 in 2024-09-20T22:16:58.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:57 smithi067 ceph-mon[33511]: 10.0 scrub starts 2024-09-20T22:16:58.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:57 smithi067 ceph-mon[33511]: 3.18 scrub starts 2024-09-20T22:16:58.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:57 smithi067 ceph-mon[33511]: 3.18 scrub ok 2024-09-20T22:16:58.032 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:58.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:58.033 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:58.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:58.169 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:58.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:58.170 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:58.271 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:58.306 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:58.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:58.307 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:16:58.319 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:16:58.408 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:58.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:58.439 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:58.439 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:58.445 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:16:58.539 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:58.571 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:58.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:58.571 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:58.637 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:58 smithi060 ceph-mon[29694]: 6.b scrub starts 2024-09-20T22:16:58.637 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:58 smithi060 ceph-mon[29694]: 6.b scrub ok 2024-09-20T22:16:58.637 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:58 smithi060 ceph-mon[29694]: 1.a scrub starts 2024-09-20T22:16:58.637 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:58 smithi060 ceph-mon[29694]: 1.a scrub ok 2024-09-20T22:16:58.637 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:58 smithi060 ceph-mon[40290]: 6.b scrub starts 2024-09-20T22:16:58.638 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:58 smithi060 ceph-mon[40290]: 6.b scrub ok 2024-09-20T22:16:58.638 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:58 smithi060 ceph-mon[40290]: 1.a scrub starts 2024-09-20T22:16:58.638 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:58 smithi060 ceph-mon[40290]: 1.a scrub ok 2024-09-20T22:16:58.672 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:58.726 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:58.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:58.727 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:moved 0 / 504 (0%) 2024-09-20T22:16:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:avg 63 2024-09-20T22:16:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:stddev 5.70088 -> 5.70088 (expected baseline 7.42462) 2024-09-20T22:16:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:min osd.2 with 51 -> 51 pgs (0.809524 -> 0.809524 * mean) 2024-09-20T22:16:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:max osd.4 with 72 -> 72 pgs (1.14286 -> 1.14286 * mean) 2024-09-20T22:16:58.763 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:16:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:oload 1000 2024-09-20T22:16:58.763 INFO:teuthology.orchestra.run.smithi060.stdout:max_change 0.05 2024-09-20T22:16:58.764 INFO:teuthology.orchestra.run.smithi060.stdout:max_change_osds 4 2024-09-20T22:16:58.764 INFO:teuthology.orchestra.run.smithi060.stdout:average_utilization 0.0360 2024-09-20T22:16:58.764 INFO:teuthology.orchestra.run.smithi060.stdout:overload_utilization 0.3600 2024-09-20T22:16:58.764 INFO:teuthology.orchestra.run.smithi060.stderr:no change 2024-09-20T22:16:58.827 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:16:58.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:58.877 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:58.878 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:16:58.978 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:59.001 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:16:59.012 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:58 smithi067 ceph-mon[33511]: 6.b scrub starts 2024-09-20T22:16:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:58 smithi067 ceph-mon[33511]: 6.b scrub ok 2024-09-20T22:16:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:58 smithi067 ceph-mon[33511]: 1.a scrub starts 2024-09-20T22:16:59.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:58 smithi067 ceph-mon[33511]: 1.a scrub ok 2024-09-20T22:16:59.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:59.014 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:59.115 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:59.152 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:59.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:59.153 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:59.253 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:59.292 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:59.293 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:59.293 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:59.313 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:16:59.314 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:16:59.394 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:16:59.437 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:59.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:59.438 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:16:59.539 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:16:59.575 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:59.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:59.576 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:16:59.677 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:16:59.714 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:59.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:59.715 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:16:59.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:16:59.853 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:16:59.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:59.854 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:16:59.867 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[40290]: pgmap v753: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.9 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.3 MiB/s wr, 1.50k op/s 2024-09-20T22:16:59.867 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[40290]: 10.f scrub starts 2024-09-20T22:16:59.867 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[40290]: from='client.26042 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:16:59.867 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[40290]: 6.6 scrub starts 2024-09-20T22:16:59.867 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[40290]: 6.6 scrub ok 2024-09-20T22:16:59.867 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[40290]: 10.0 scrub ok 2024-09-20T22:16:59.867 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[40290]: 5.8 scrub starts 2024-09-20T22:16:59.867 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[40290]: 5.8 scrub ok 2024-09-20T22:16:59.867 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[40290]: 6.11 scrub starts 2024-09-20T22:16:59.867 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[40290]: 6.11 scrub ok 2024-09-20T22:16:59.867 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[29694]: pgmap v753: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.9 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.3 MiB/s wr, 1.50k op/s 2024-09-20T22:16:59.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[29694]: 10.f scrub starts 2024-09-20T22:16:59.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[29694]: from='client.26042 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:16:59.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[29694]: 6.6 scrub starts 2024-09-20T22:16:59.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[29694]: 6.6 scrub ok 2024-09-20T22:16:59.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[29694]: 10.0 scrub ok 2024-09-20T22:16:59.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[29694]: 5.8 scrub starts 2024-09-20T22:16:59.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[29694]: 5.8 scrub ok 2024-09-20T22:16:59.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[29694]: 6.11 scrub starts 2024-09-20T22:16:59.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:16:59 smithi060 ceph-mon[29694]: 6.11 scrub ok 2024-09-20T22:16:59.869 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-09-20T22:16:59.869 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight-by-utilization 110 0.05 2024-09-20T22:16:59.955 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:16:59.991 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:16:59.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:16:59.992 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:00.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:59 smithi067 ceph-mon[33511]: pgmap v753: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.9 GiB data, 26 GiB used, 689 GiB / 715 GiB avail; 1.2 KiB/s rd, 8.3 MiB/s wr, 1.50k op/s 2024-09-20T22:17:00.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:59 smithi067 ceph-mon[33511]: 10.f scrub starts 2024-09-20T22:17:00.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:59 smithi067 ceph-mon[33511]: from='client.26042 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:17:00.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:59 smithi067 ceph-mon[33511]: 6.6 scrub starts 2024-09-20T22:17:00.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:59 smithi067 ceph-mon[33511]: 6.6 scrub ok 2024-09-20T22:17:00.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:59 smithi067 ceph-mon[33511]: 10.0 scrub ok 2024-09-20T22:17:00.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:59 smithi067 ceph-mon[33511]: 5.8 scrub starts 2024-09-20T22:17:00.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:59 smithi067 ceph-mon[33511]: 5.8 scrub ok 2024-09-20T22:17:00.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:59 smithi067 ceph-mon[33511]: 6.11 scrub starts 2024-09-20T22:17:00.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:16:59 smithi067 ceph-mon[33511]: 6.11 scrub ok 2024-09-20T22:17:00.093 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:00.126 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:00.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:00.128 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:00.228 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:00.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:00.265 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:00.265 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:00.366 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:00.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:00.399 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:00.399 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:00.499 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:00.530 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:00.531 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:00.532 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:00.568 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:00.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T22:17:00.632 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:00.667 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:00.668 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:00.668 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:00.769 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:00.825 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:00.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:00.826 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:00.917 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:00.922 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:00 smithi060 ceph-mon[29694]: 4.b scrub starts 2024-09-20T22:17:00.922 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:00 smithi060 ceph-mon[29694]: 4.b scrub ok 2024-09-20T22:17:00.922 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:00 smithi060 ceph-mon[29694]: 10.b scrub starts 2024-09-20T22:17:00.922 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:00 smithi060 ceph-mon[29694]: pgmap v754: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.9 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 639 B/s rd, 720 KiB/s wr, 1.21k op/s 2024-09-20T22:17:00.923 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:00 smithi060 ceph-mon[40290]: 4.b scrub starts 2024-09-20T22:17:00.923 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:00 smithi060 ceph-mon[40290]: 4.b scrub ok 2024-09-20T22:17:00.923 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:00 smithi060 ceph-mon[40290]: 10.b scrub starts 2024-09-20T22:17:00.923 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:00 smithi060 ceph-mon[40290]: pgmap v754: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.9 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 639 B/s rd, 720 KiB/s wr, 1.21k op/s 2024-09-20T22:17:00.926 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:00.988 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:00.989 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:00.990 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:00 smithi067 ceph-mon[33511]: 4.b scrub starts 2024-09-20T22:17:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:00 smithi067 ceph-mon[33511]: 4.b scrub ok 2024-09-20T22:17:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:00 smithi067 ceph-mon[33511]: 10.b scrub starts 2024-09-20T22:17:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:00 smithi067 ceph-mon[33511]: pgmap v754: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.9 GiB data, 26 GiB used, 690 GiB / 715 GiB avail; 639 B/s rd, 720 KiB/s wr, 1.21k op/s 2024-09-20T22:17:01.091 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:01.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:01.147 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:01.147 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:01.220 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:01.248 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:01.290 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:01.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:01.291 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:01.392 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:01.446 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:01.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:01.447 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:01.548 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:01.600 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:01.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:01.602 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:01.702 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:01.737 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:01.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:01.738 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:01 smithi067 ceph-mon[33511]: 1.6 scrub starts 2024-09-20T22:17:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:01 smithi067 ceph-mon[33511]: 1.6 scrub ok 2024-09-20T22:17:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:01 smithi067 ceph-mon[33511]: 4.7 scrub starts 2024-09-20T22:17:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:01 smithi067 ceph-mon[33511]: 4.7 scrub ok 2024-09-20T22:17:01.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:01 smithi060 ceph-mon[40290]: 1.6 scrub starts 2024-09-20T22:17:01.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:01 smithi060 ceph-mon[40290]: 1.6 scrub ok 2024-09-20T22:17:01.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:01 smithi060 ceph-mon[40290]: 4.7 scrub starts 2024-09-20T22:17:01.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:01 smithi060 ceph-mon[40290]: 4.7 scrub ok 2024-09-20T22:17:01.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:01 smithi060 ceph-mon[29694]: 1.6 scrub starts 2024-09-20T22:17:01.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:01 smithi060 ceph-mon[29694]: 1.6 scrub ok 2024-09-20T22:17:01.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:01 smithi060 ceph-mon[29694]: 4.7 scrub starts 2024-09-20T22:17:01.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:01 smithi060 ceph-mon[29694]: 4.7 scrub ok 2024-09-20T22:17:01.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:01.884 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:01.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:01.886 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:01.986 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:02.017 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:02.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:02.018 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:02.119 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:02.149 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:02.150 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:02.150 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:02.251 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:02.288 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:02.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:02.289 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:02.389 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:02.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:02.420 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:02.420 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:02.465 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:02.481 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:17:02.481 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:17:02.515 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:17:02.515 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:17:02.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:02.554 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:17:02.554 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:17:02.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:02.572 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:02.573 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:02.591 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:17:02.591 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:17:02.627 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:17:02.627 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:17:02.643 INFO:teuthology.orchestra.run.smithi060.stdout:moved 4 / 504 (0.793651%) 2024-09-20T22:17:02.643 INFO:teuthology.orchestra.run.smithi060.stdout:avg 63 2024-09-20T22:17:02.643 INFO:teuthology.orchestra.run.smithi060.stdout:stddev 5.70088 -> 4.89898 (expected baseline 7.42462) 2024-09-20T22:17:02.643 INFO:teuthology.orchestra.run.smithi060.stdout:min osd.2 with 51 -> 53 pgs (0.809524 -> 0.84127 * mean) 2024-09-20T22:17:02.643 INFO:teuthology.orchestra.run.smithi060.stdout:max osd.4 with 72 -> 70 pgs (1.14286 -> 1.11111 * mean) 2024-09-20T22:17:02.643 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:17:02.643 INFO:teuthology.orchestra.run.smithi060.stdout:oload 110 2024-09-20T22:17:02.643 INFO:teuthology.orchestra.run.smithi060.stdout:max_change 0.05 2024-09-20T22:17:02.643 INFO:teuthology.orchestra.run.smithi060.stdout:max_change_osds 4 2024-09-20T22:17:02.643 INFO:teuthology.orchestra.run.smithi060.stdout:average_utilization 0.0350 2024-09-20T22:17:02.644 INFO:teuthology.orchestra.run.smithi060.stdout:overload_utilization 0.0385 2024-09-20T22:17:02.644 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 weight 1.0000 -> 0.9500 2024-09-20T22:17:02.644 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 weight 1.0000 -> 0.9500 2024-09-20T22:17:02.644 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 weight 1.0000 -> 0.9500 2024-09-20T22:17:02.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[29694]: 10.f scrub ok 2024-09-20T22:17:02.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1453054986' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:02.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:02.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[29694]: 10.b scrub ok 2024-09-20T22:17:02.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:17:02.644 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:17:02.645 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[29694]: pgmap v755: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s 2024-09-20T22:17:02.645 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[29694]: from='client.26054 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:17:02.645 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'0':'62260','4':'62260','6':'62260'}"}]: dispatch 2024-09-20T22:17:02.645 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3657118827' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:02.645 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:17:02.645 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[40290]: 10.f scrub ok 2024-09-20T22:17:02.645 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1453054986' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:02.645 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:02.645 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[40290]: 10.b scrub ok 2024-09-20T22:17:02.645 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:17:02.645 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:17:02.645 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[40290]: pgmap v755: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s 2024-09-20T22:17:02.646 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[40290]: from='client.26054 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:17:02.646 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'0':'62260','4':'62260','6':'62260'}"}]: dispatch 2024-09-20T22:17:02.646 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3657118827' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:02.646 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:02 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:17:02.646 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:17:02.671 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:17:02.671 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:17:02.673 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:02.705 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:02.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:02.706 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:02.714 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:17:02.715 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:17:02.747 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:17:02.748 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:17:02.807 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:02.840 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:02.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:02.842 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:02.943 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:02.980 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:02.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:02.981 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:03.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:17:02 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:17:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:02 smithi067 ceph-mon[33511]: 10.f scrub ok 2024-09-20T22:17:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:02 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1453054986' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:02 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:02 smithi067 ceph-mon[33511]: 10.b scrub ok 2024-09-20T22:17:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:02 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:17:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:02 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:17:03.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:02 smithi067 ceph-mon[33511]: pgmap v755: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 6.8 GiB data, 25 GiB used, 690 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.09k op/s 2024-09-20T22:17:03.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:02 smithi067 ceph-mon[33511]: from='client.26054 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:17:03.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:02 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd reweightn", "weights": "{'0':'62260','4':'62260','6':'62260'}"}]: dispatch 2024-09-20T22:17:03.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:02 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3657118827' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:03.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:02 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:17:03.082 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:03.120 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:03.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:03.120 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:03.221 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:03.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:03.265 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:03.266 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:03.366 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:03.400 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:03.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:03.401 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:03.502 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:03.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:03.538 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:03.538 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:03.639 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:03.672 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:03.672 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:03.672 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:03.773 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:03.814 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:03.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:03.815 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:03.915 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:03.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:03.954 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:03.955 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:03 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:17:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:03 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:17:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:03 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:17:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:03 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'0':'62260','4':'62260','6':'62260'}"}]': finished 2024-09-20T22:17:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:03 smithi067 ceph-mon[33511]: osdmap e415: 8 total, 8 up, 8 in 2024-09-20T22:17:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:03 smithi067 ceph-mon[33511]: 6.15 scrub starts 2024-09-20T22:17:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:03 smithi067 ceph-mon[33511]: 6.15 scrub ok 2024-09-20T22:17:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:03 smithi067 ceph-mon[33511]: 1.7 scrub starts 2024-09-20T22:17:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:03 smithi067 ceph-mon[33511]: 1.7 scrub ok 2024-09-20T22:17:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:03 smithi067 ceph-mon[33511]: 6.7 scrub starts 2024-09-20T22:17:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:03 smithi067 ceph-mon[33511]: 6.7 scrub ok 2024-09-20T22:17:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:03 smithi067 ceph-mon[33511]: 4.12 scrub starts 2024-09-20T22:17:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:03 smithi067 ceph-mon[33511]: 4.12 scrub ok 2024-09-20T22:17:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:17:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:17:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:17:04.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'0':'62260','4':'62260','6':'62260'}"}]': finished 2024-09-20T22:17:04.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[29694]: osdmap e415: 8 total, 8 up, 8 in 2024-09-20T22:17:04.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[29694]: 6.15 scrub starts 2024-09-20T22:17:04.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[29694]: 6.15 scrub ok 2024-09-20T22:17:04.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[29694]: 1.7 scrub starts 2024-09-20T22:17:04.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[29694]: 1.7 scrub ok 2024-09-20T22:17:04.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[29694]: 6.7 scrub starts 2024-09-20T22:17:04.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[29694]: 6.7 scrub ok 2024-09-20T22:17:04.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[29694]: 4.12 scrub starts 2024-09-20T22:17:04.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[29694]: 4.12 scrub ok 2024-09-20T22:17:04.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:17:04.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:17:04.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:17:04.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd reweightn", "weights": "{'0':'62260','4':'62260','6':'62260'}"}]': finished 2024-09-20T22:17:04.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[40290]: osdmap e415: 8 total, 8 up, 8 in 2024-09-20T22:17:04.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[40290]: 6.15 scrub starts 2024-09-20T22:17:04.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[40290]: 6.15 scrub ok 2024-09-20T22:17:04.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[40290]: 1.7 scrub starts 2024-09-20T22:17:04.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[40290]: 1.7 scrub ok 2024-09-20T22:17:04.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[40290]: 6.7 scrub starts 2024-09-20T22:17:04.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[40290]: 6.7 scrub ok 2024-09-20T22:17:04.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[40290]: 4.12 scrub starts 2024-09-20T22:17:04.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:03 smithi060 ceph-mon[40290]: 4.12 scrub ok 2024-09-20T22:17:04.055 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:04.089 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:04.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:04.090 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:04.190 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:04.225 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:04.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:04.226 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:04.230 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-09-20T22:17:04.231 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight-by-utilization 1000 3.0 2024-09-20T22:17:04.327 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:04.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:04.361 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:04.361 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:04.455 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:17:04.455 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:17:04.455 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:17:04.455 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:17:04.456 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:17:04.462 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:04.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:04.501 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:04.501 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:04.602 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:04.650 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:04.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:04.651 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:04.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[29694]: osdmap e416: 8 total, 8 up, 8 in 2024-09-20T22:17:04.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[29694]: 4.10 scrub starts 2024-09-20T22:17:04.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[29694]: 4.10 scrub ok 2024-09-20T22:17:04.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[29694]: 5.1a scrub starts 2024-09-20T22:17:04.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[29694]: 5.1a scrub ok 2024-09-20T22:17:04.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[29694]: 4.a scrub starts 2024-09-20T22:17:04.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[29694]: 4.a scrub ok 2024-09-20T22:17:04.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[29694]: pgmap v758: 168 pgs: 1 peering, 167 active+clean; 6.6 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 666 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-20T22:17:04.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[29694]: 5.19 scrub starts 2024-09-20T22:17:04.737 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[29694]: 5.19 scrub ok 2024-09-20T22:17:04.738 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[40290]: osdmap e416: 8 total, 8 up, 8 in 2024-09-20T22:17:04.738 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[40290]: 4.10 scrub starts 2024-09-20T22:17:04.738 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[40290]: 4.10 scrub ok 2024-09-20T22:17:04.738 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[40290]: 5.1a scrub starts 2024-09-20T22:17:04.738 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[40290]: 5.1a scrub ok 2024-09-20T22:17:04.738 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[40290]: 4.a scrub starts 2024-09-20T22:17:04.738 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[40290]: 4.a scrub ok 2024-09-20T22:17:04.738 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[40290]: pgmap v758: 168 pgs: 1 peering, 167 active+clean; 6.6 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 666 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-20T22:17:04.738 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[40290]: 5.19 scrub starts 2024-09-20T22:17:04.738 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:04 smithi060 ceph-mon[40290]: 5.19 scrub ok 2024-09-20T22:17:04.752 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:04.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:04.787 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:04.787 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:04.888 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:04.924 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:04.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:04.925 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:04 smithi067 ceph-mon[33511]: osdmap e416: 8 total, 8 up, 8 in 2024-09-20T22:17:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:04 smithi067 ceph-mon[33511]: 4.10 scrub starts 2024-09-20T22:17:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:04 smithi067 ceph-mon[33511]: 4.10 scrub ok 2024-09-20T22:17:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:04 smithi067 ceph-mon[33511]: 5.1a scrub starts 2024-09-20T22:17:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:04 smithi067 ceph-mon[33511]: 5.1a scrub ok 2024-09-20T22:17:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:04 smithi067 ceph-mon[33511]: 4.a scrub starts 2024-09-20T22:17:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:04 smithi067 ceph-mon[33511]: 4.a scrub ok 2024-09-20T22:17:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:04 smithi067 ceph-mon[33511]: pgmap v758: 168 pgs: 1 peering, 167 active+clean; 6.6 GiB data, 25 GiB used, 691 GiB / 715 GiB avail; 666 B/s rd, 0 B/s wr, 1.22k op/s 2024-09-20T22:17:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:04 smithi067 ceph-mon[33511]: 5.19 scrub starts 2024-09-20T22:17:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:04 smithi067 ceph-mon[33511]: 5.19 scrub ok 2024-09-20T22:17:05.026 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:05.064 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:05.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:05.065 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:05.166 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:05.198 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:05.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:05.199 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:05.299 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:05.334 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:05.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:05.335 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:05.427 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:17:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:17:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:17:05.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:05.473 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:05.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:05.474 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:05.575 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:05.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:05.614 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:05.614 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:05.715 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:05.754 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:05.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:05.755 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:05.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:05 smithi060 ceph-mon[29694]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:17:05.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:05 smithi060 ceph-mon[29694]: 1.14 scrub starts 2024-09-20T22:17:05.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:05 smithi060 ceph-mon[29694]: 1.14 scrub ok 2024-09-20T22:17:05.769 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:05 smithi060 ceph-mon[40290]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:17:05.769 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:05 smithi060 ceph-mon[40290]: 1.14 scrub starts 2024-09-20T22:17:05.769 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:05 smithi060 ceph-mon[40290]: 1.14 scrub ok 2024-09-20T22:17:05.811 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T22:17:05.850 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:05.856 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:05.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:05.954 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:05.955 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:05 smithi067 ceph-mon[33511]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:17:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:05 smithi067 ceph-mon[33511]: 1.14 scrub starts 2024-09-20T22:17:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:05 smithi067 ceph-mon[33511]: 1.14 scrub ok 2024-09-20T22:17:06.055 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:06.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:06.098 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:06.099 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:06.199 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:06.218 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:17:06.240 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:06.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:06.241 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:06.342 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:06.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:06.405 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:06.406 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:06.506 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:06.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:06.584 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:06.584 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:06.589 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:06.662 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[29694]: 4.8 deep-scrub starts 2024-09-20T22:17:06.663 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[29694]: 4.8 deep-scrub ok 2024-09-20T22:17:06.663 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[29694]: 5.14 scrub starts 2024-09-20T22:17:06.663 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[29694]: 5.14 scrub ok 2024-09-20T22:17:06.663 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[29694]: 4.6 scrub starts 2024-09-20T22:17:06.663 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[29694]: 4.6 scrub ok 2024-09-20T22:17:06.663 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:06.663 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:17:06.663 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[29694]: pgmap v759: 168 pgs: 1 peering, 167 active+clean; 6.5 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 907 op/s 2024-09-20T22:17:06.663 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:17:06.685 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:06.717 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:06.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:06.718 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:06.819 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:06.861 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:06.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:06.862 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:06.895 INFO:teuthology.orchestra.run.smithi060.stdout:moved 0 / 504 (0%) 2024-09-20T22:17:06.896 INFO:teuthology.orchestra.run.smithi060.stdout:avg 63 2024-09-20T22:17:06.896 INFO:teuthology.orchestra.run.smithi060.stdout:stddev 4.89898 -> 4.89898 (expected baseline 7.42462) 2024-09-20T22:17:06.896 INFO:teuthology.orchestra.run.smithi060.stdout:min osd.2 with 53 -> 53 pgs (0.84127 -> 0.84127 * mean) 2024-09-20T22:17:06.896 INFO:teuthology.orchestra.run.smithi060.stdout:max osd.4 with 70 -> 70 pgs (1.11111 -> 1.11111 * mean) 2024-09-20T22:17:06.896 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:17:06.896 INFO:teuthology.orchestra.run.smithi060.stdout:oload 1000 2024-09-20T22:17:06.896 INFO:teuthology.orchestra.run.smithi060.stdout:max_change 3 2024-09-20T22:17:06.896 INFO:teuthology.orchestra.run.smithi060.stdout:max_change_osds 4 2024-09-20T22:17:06.896 INFO:teuthology.orchestra.run.smithi060.stdout:average_utilization 0.0340 2024-09-20T22:17:06.896 INFO:teuthology.orchestra.run.smithi060.stdout:overload_utilization 0.3400 2024-09-20T22:17:06.896 INFO:teuthology.orchestra.run.smithi060.stderr:no change 2024-09-20T22:17:06.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[40290]: 4.8 deep-scrub starts 2024-09-20T22:17:06.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[40290]: 4.8 deep-scrub ok 2024-09-20T22:17:06.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[40290]: 5.14 scrub starts 2024-09-20T22:17:06.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[40290]: 5.14 scrub ok 2024-09-20T22:17:06.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[40290]: 4.6 scrub starts 2024-09-20T22:17:06.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[40290]: 4.6 scrub ok 2024-09-20T22:17:06.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:06.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:17:06.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[40290]: pgmap v759: 168 pgs: 1 peering, 167 active+clean; 6.5 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 907 op/s 2024-09-20T22:17:06.920 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:17:06.963 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:07.007 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:07.007 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:07.007 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:06 smithi067 ceph-mon[33511]: 4.8 deep-scrub starts 2024-09-20T22:17:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:06 smithi067 ceph-mon[33511]: 4.8 deep-scrub ok 2024-09-20T22:17:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:06 smithi067 ceph-mon[33511]: 5.14 scrub starts 2024-09-20T22:17:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:06 smithi067 ceph-mon[33511]: 5.14 scrub ok 2024-09-20T22:17:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:06 smithi067 ceph-mon[33511]: 4.6 scrub starts 2024-09-20T22:17:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:06 smithi067 ceph-mon[33511]: 4.6 scrub ok 2024-09-20T22:17:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:17:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:06 smithi067 ceph-mon[33511]: pgmap v759: 168 pgs: 1 peering, 167 active+clean; 6.5 GiB data, 24 GiB used, 691 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 907 op/s 2024-09-20T22:17:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:17:07.103 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:17:07.108 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:07.193 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:07.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:07.194 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:07.295 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:07.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:07.341 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:07.342 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:07.442 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:07.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:07.480 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:07.480 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:07.581 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:07.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:07.626 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:07.626 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:07.701 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:17:07.701 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:17:07.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:17:07.701 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:17:07.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.701 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106980 8.18 8:1c2b9c14:::benchmark_data_smithi060_111051_object106979:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:17:07.701 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.414151+0000", 2024-09-20T22:17:07.702 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 476.273134868, 2024-09-20T22:17:07.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.111436294, 2024-09-20T22:17:07.702 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.702 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.702 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.702 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:17:07.702 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:17:07.702 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:07.702 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106980 2024-09-20T22:17:07.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.702 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414151+0000", 2024-09-20T22:17:07.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414151+0000", 2024-09-20T22:17:07.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414152+0000", 2024-09-20T22:17:07.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5179999999999999e-06 2024-09-20T22:17:07.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414179+0000", 2024-09-20T22:17:07.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6480999999999999e-05 2024-09-20T22:17:07.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414191+0000", 2024-09-20T22:17:07.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2833999999999999e-05 2024-09-20T22:17:07.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414200+0000", 2024-09-20T22:17:07.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1040000000000007e-06 2024-09-20T22:17:07.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418058+0000", 2024-09-20T22:17:07.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038580139999999999 2024-09-20T22:17:07.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:17:07.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418080+0000", 2024-09-20T22:17:07.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2438000000000001e-05 2024-09-20T22:17:07.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.501675+0000", 2024-09-20T22:17:07.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0835947560000001 2024-09-20T22:17:07.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.501814+0000", 2024-09-20T22:17:07.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013951799999999999 2024-09-20T22:17:07.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4,5,7", 2024-09-20T22:17:07.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.502081+0000", 2024-09-20T22:17:07.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026697100000000002 2024-09-20T22:17:07.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:17:07.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.503435+0000", 2024-09-20T22:17:07.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001354156 2024-09-20T22:17:07.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.514128+0000", 2024-09-20T22:17:07.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010692432 2024-09-20T22:17:07.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525290+0000", 2024-09-20T22:17:07.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.011161901 2024-09-20T22:17:07.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525399+0000", 2024-09-20T22:17:07.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010921699999999999 2024-09-20T22:17:07.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525468+0000", 2024-09-20T22:17:07.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8726999999999994e-05 2024-09-20T22:17:07.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525587+0000", 2024-09-20T22:17:07.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119227 2024-09-20T22:17:07.708 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.708 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.709 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.709 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187847 8.0 8:0387ec14:::benchmark_data_smithi060_111051_object67848:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:17:07.709 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.626135+0000", 2024-09-20T22:17:07.709 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 398.06115067399998, 2024-09-20T22:17:07.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2145214800000002, 2024-09-20T22:17:07.709 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.709 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.709 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:17:07.709 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:17:07.709 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:17:07.709 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:07.709 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187847 2024-09-20T22:17:07.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.710 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626135+0000", 2024-09-20T22:17:07.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626135+0000", 2024-09-20T22:17:07.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626137+0000", 2024-09-20T22:17:07.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0389999999999999e-06 2024-09-20T22:17:07.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626144+0000", 2024-09-20T22:17:07.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9299999999999997e-06 2024-09-20T22:17:07.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626285+0000", 2024-09-20T22:17:07.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000141445 2024-09-20T22:17:07.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626296+0000", 2024-09-20T22:17:07.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0873000000000001e-05 2024-09-20T22:17:07.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626432+0000", 2024-09-20T22:17:07.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000135537 2024-09-20T22:17:07.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626543+0000", 2024-09-20T22:17:07.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011132399999999999 2024-09-20T22:17:07.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,5", 2024-09-20T22:17:07.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627183+0000", 2024-09-20T22:17:07.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00064020400000000001 2024-09-20T22:17:07.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.656943+0000", 2024-09-20T22:17:07.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029759622999999999 2024-09-20T22:17:07.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:17:07.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.656954+0000", 2024-09-20T22:17:07.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0913999999999999e-05 2024-09-20T22:17:07.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651133+0000", 2024-09-20T22:17:07.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99417974399999998 2024-09-20T22:17:07.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:17:07.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651152+0000", 2024-09-20T22:17:07.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8108999999999999e-05 2024-09-20T22:17:07.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.714 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.840586+0000", 2024-09-20T22:17:07.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1894341669999999 2024-09-20T22:17:07.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.840656+0000", 2024-09-20T22:17:07.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0570999999999999e-05 2024-09-20T22:17:07.715 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.715 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.715 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.715 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187874 8.0 8:032cd1ec:::benchmark_data_smithi060_111051_object67875:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:17:07.715 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.678008+0000", 2024-09-20T22:17:07.715 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 398.00927731000002, 2024-09-20T22:17:07.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1758792680000001, 2024-09-20T22:17:07.715 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.715 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.716 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.716 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:17:07.716 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:17:07.716 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:07.716 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187874 2024-09-20T22:17:07.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.716 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.716 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678008+0000", 2024-09-20T22:17:07.716 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.716 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.716 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678008+0000", 2024-09-20T22:17:07.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678010+0000", 2024-09-20T22:17:07.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7430000000000001e-06 2024-09-20T22:17:07.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678016+0000", 2024-09-20T22:17:07.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4339999999999997e-06 2024-09-20T22:17:07.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678030+0000", 2024-09-20T22:17:07.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3971999999999999e-05 2024-09-20T22:17:07.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678038+0000", 2024-09-20T22:17:07.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5100000000000001e-06 2024-09-20T22:17:07.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678109+0000", 2024-09-20T22:17:07.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0708000000000001e-05 2024-09-20T22:17:07.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:17:07.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678119+0000", 2024-09-20T22:17:07.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0519e-05 2024-09-20T22:17:07.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651185+0000", 2024-09-20T22:17:07.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97306574199999996 2024-09-20T22:17:07.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:17:07.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651193+0000", 2024-09-20T22:17:07.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2609999999999993e-06 2024-09-20T22:17:07.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.840972+0000", 2024-09-20T22:17:07.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.189778644 2024-09-20T22:17:07.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.841244+0000", 2024-09-20T22:17:07.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00027224500000000001 2024-09-20T22:17:07.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4,5", 2024-09-20T22:17:07.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.841873+0000", 2024-09-20T22:17:07.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000629124 2024-09-20T22:17:07.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.852639+0000", 2024-09-20T22:17:07.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010765925000000001 2024-09-20T22:17:07.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.852704+0000", 2024-09-20T22:17:07.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5011000000000005e-05 2024-09-20T22:17:07.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:17:07.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.852729+0000", 2024-09-20T22:17:07.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4782000000000001e-05 2024-09-20T22:17:07.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.853614+0000", 2024-09-20T22:17:07.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00088560699999999995 2024-09-20T22:17:07.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.853671+0000", 2024-09-20T22:17:07.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6382999999999999e-05 2024-09-20T22:17:07.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.853887+0000", 2024-09-20T22:17:07.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000216658 2024-09-20T22:17:07.722 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.722 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.723 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.723 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187894 8.0 8:02b3e725:::benchmark_data_smithi060_111051_object67895:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:17:07.723 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.701140+0000", 2024-09-20T22:17:07.723 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 397.98614556199999, 2024-09-20T22:17:07.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1634560220000001, 2024-09-20T22:17:07.723 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.723 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.723 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.723 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:17:07.723 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:17:07.723 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:07.723 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187894 2024-09-20T22:17:07.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.723 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701140+0000", 2024-09-20T22:17:07.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701140+0000", 2024-09-20T22:17:07.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701141+0000", 2024-09-20T22:17:07.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2950000000000001e-06 2024-09-20T22:17:07.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701147+0000", 2024-09-20T22:17:07.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3890000000000004e-06 2024-09-20T22:17:07.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701168+0000", 2024-09-20T22:17:07.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1725999999999998e-05 2024-09-20T22:17:07.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701179+0000", 2024-09-20T22:17:07.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0393e-05 2024-09-20T22:17:07.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701245+0000", 2024-09-20T22:17:07.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6519999999999993e-05 2024-09-20T22:17:07.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:17:07.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701256+0000", 2024-09-20T22:17:07.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0722e-05 2024-09-20T22:17:07.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651217+0000", 2024-09-20T22:17:07.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94996075899999999 2024-09-20T22:17:07.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:17:07.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.654608+0000", 2024-09-20T22:17:07.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0033909819999999999 2024-09-20T22:17:07.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.849926+0000", 2024-09-20T22:17:07.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.195318353 2024-09-20T22:17:07.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.728 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:07.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.850057+0000", 2024-09-20T22:17:07.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013062999999999999 2024-09-20T22:17:07.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4,5", 2024-09-20T22:17:07.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.850462+0000", 2024-09-20T22:17:07.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00040517 2024-09-20T22:17:07.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:17:07.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.852996+0000", 2024-09-20T22:17:07.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025345179999999999 2024-09-20T22:17:07.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.854077+0000", 2024-09-20T22:17:07.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001080262 2024-09-20T22:17:07.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.864365+0000", 2024-09-20T22:17:07.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010287907000000001 2024-09-20T22:17:07.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.864413+0000", 2024-09-20T22:17:07.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8690999999999998e-05 2024-09-20T22:17:07.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.864473+0000", 2024-09-20T22:17:07.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9487000000000002e-05 2024-09-20T22:17:07.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.731 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.731 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.864596+0000", 2024-09-20T22:17:07.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000123218 2024-09-20T22:17:07.731 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.731 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.731 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.731 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214320 8.8 8:125eda0b:::benchmark_data_smithi060_111051_object94321:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:17:07.731 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:17:07.731 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 370.19090368899998, 2024-09-20T22:17:07.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0054397049999999, 2024-09-20T22:17:07.732 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.732 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.732 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.732 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:17:07.732 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:17:07.732 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:07.732 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214320 2024-09-20T22:17:07.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.732 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.732 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:17:07.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:17:07.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:17:07.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0999999999999999e-07 2024-09-20T22:17:07.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496384+0000", 2024-09-20T22:17:07.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1859999999999999e-06 2024-09-20T22:17:07.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496390+0000", 2024-09-20T22:17:07.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9920000000000004e-06 2024-09-20T22:17:07.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496395+0000", 2024-09-20T22:17:07.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.741e-06 2024-09-20T22:17:07.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496429+0000", 2024-09-20T22:17:07.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3736999999999999e-05 2024-09-20T22:17:07.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:17:07.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496436+0000", 2024-09-20T22:17:07.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2529999999999998e-06 2024-09-20T22:17:07.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.497957+0000", 2024-09-20T22:17:07.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.001521109 2024-09-20T22:17:07.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.498097+0000", 2024-09-20T22:17:07.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000139562 2024-09-20T22:17:07.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:17:07.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.498422+0000", 2024-09-20T22:17:07.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00032548199999999998 2024-09-20T22:17:07.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501661+0000", 2024-09-20T22:17:07.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0032386580000000002 2024-09-20T22:17:07.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:17:07.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501669+0000", 2024-09-20T22:17:07.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2500000000000006e-06 2024-09-20T22:17:07.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501709+0000", 2024-09-20T22:17:07.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9304999999999997e-05 2024-09-20T22:17:07.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501777+0000", 2024-09-20T22:17:07.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8285000000000001e-05 2024-09-20T22:17:07.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.737 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.737 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501821+0000", 2024-09-20T22:17:07.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4635000000000002e-05 2024-09-20T22:17:07.737 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.737 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.738 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.738 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214327 8.8 8:149a99e2:::benchmark_data_smithi060_111051_object94328:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:17:07.738 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.502392+0000", 2024-09-20T22:17:07.738 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 370.18489326700001, 2024-09-20T22:17:07.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.001539808, 2024-09-20T22:17:07.738 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.738 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.738 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.738 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:17:07.738 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:17:07.738 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:07.738 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214327 2024-09-20T22:17:07.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.739 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502392+0000", 2024-09-20T22:17:07.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502392+0000", 2024-09-20T22:17:07.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502393+0000", 2024-09-20T22:17:07.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2440000000000001e-06 2024-09-20T22:17:07.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502398+0000", 2024-09-20T22:17:07.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1749999999999998e-06 2024-09-20T22:17:07.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502406+0000", 2024-09-20T22:17:07.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4999999999999999e-06 2024-09-20T22:17:07.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502411+0000", 2024-09-20T22:17:07.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7119999999999999e-06 2024-09-20T22:17:07.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502465+0000", 2024-09-20T22:17:07.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4206000000000002e-05 2024-09-20T22:17:07.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:17:07.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502470+0000", 2024-09-20T22:17:07.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8069999999999999e-06 2024-09-20T22:17:07.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.500319+0000", 2024-09-20T22:17:07.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99784936499999999 2024-09-20T22:17:07.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.742 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.742 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.500435+0000", 2024-09-20T22:17:07.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115439 2024-09-20T22:17:07.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.742 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:17:07.742 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.500670+0000", 2024-09-20T22:17:07.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023555700000000001 2024-09-20T22:17:07.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.742 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.742 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503764+0000", 2024-09-20T22:17:07.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0030938839999999999 2024-09-20T22:17:07.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503827+0000", 2024-09-20T22:17:07.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2669999999999995e-05 2024-09-20T22:17:07.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:17:07.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503839+0000", 2024-09-20T22:17:07.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2301e-05 2024-09-20T22:17:07.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503887+0000", 2024-09-20T22:17:07.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8319999999999998e-05 2024-09-20T22:17:07.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503932+0000", 2024-09-20T22:17:07.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4628e-05 2024-09-20T22:17:07.744 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.744 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.744 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.744 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214339 8.8 8:166843c0:::benchmark_data_smithi060_111051_object94340:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:17:07.744 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.507919+0000", 2024-09-20T22:17:07.745 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 370.17936617100003, 2024-09-20T22:17:07.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99621294400000004, 2024-09-20T22:17:07.745 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.745 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.745 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.745 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:17:07.745 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:17:07.745 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:07.745 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214339 2024-09-20T22:17:07.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.745 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.745 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.745 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507919+0000", 2024-09-20T22:17:07.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.746 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507919+0000", 2024-09-20T22:17:07.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.746 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507921+0000", 2024-09-20T22:17:07.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.316e-06 2024-09-20T22:17:07.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.746 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507926+0000", 2024-09-20T22:17:07.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0350000000000001e-06 2024-09-20T22:17:07.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507937+0000", 2024-09-20T22:17:07.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1379999999999999e-05 2024-09-20T22:17:07.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507945+0000", 2024-09-20T22:17:07.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5780000000000001e-06 2024-09-20T22:17:07.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.508028+0000", 2024-09-20T22:17:07.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3641999999999996e-05 2024-09-20T22:17:07.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:17:07.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.508036+0000", 2024-09-20T22:17:07.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1780000000000003e-06 2024-09-20T22:17:07.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501861+0000", 2024-09-20T22:17:07.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99382438200000001 2024-09-20T22:17:07.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501935+0000", 2024-09-20T22:17:07.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4151000000000002e-05 2024-09-20T22:17:07.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:17:07.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.502143+0000", 2024-09-20T22:17:07.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020823700000000001 2024-09-20T22:17:07.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503984+0000", 2024-09-20T22:17:07.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001840602 2024-09-20T22:17:07.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504043+0000", 2024-09-20T22:17:07.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9061000000000003e-05 2024-09-20T22:17:07.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:17:07.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504055+0000", 2024-09-20T22:17:07.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1826e-05 2024-09-20T22:17:07.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504099+0000", 2024-09-20T22:17:07.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4801e-05 2024-09-20T22:17:07.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504132+0000", 2024-09-20T22:17:07.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2755000000000001e-05 2024-09-20T22:17:07.751 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.751 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.751 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.751 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214363 8.8 8:14aba4d5:::benchmark_data_smithi060_111051_object94364:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:17:07.751 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.530169+0000", 2024-09-20T22:17:07.751 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 370.15711623599998, 2024-09-20T22:17:07.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98026972499999998, 2024-09-20T22:17:07.752 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.752 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.752 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.752 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:17:07.752 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:17:07.752 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:07.752 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214363 2024-09-20T22:17:07.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.752 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530169+0000", 2024-09-20T22:17:07.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530169+0000", 2024-09-20T22:17:07.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530171+0000", 2024-09-20T22:17:07.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4300000000000001e-06 2024-09-20T22:17:07.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530177+0000", 2024-09-20T22:17:07.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8499999999999999e-06 2024-09-20T22:17:07.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530188+0000", 2024-09-20T22:17:07.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1669e-05 2024-09-20T22:17:07.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530196+0000", 2024-09-20T22:17:07.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0760000000000003e-06 2024-09-20T22:17:07.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530270+0000", 2024-09-20T22:17:07.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3709999999999997e-05 2024-09-20T22:17:07.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:17:07.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530278+0000", 2024-09-20T22:17:07.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5320000000000006e-06 2024-09-20T22:17:07.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504165+0000", 2024-09-20T22:17:07.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97388663600000003 2024-09-20T22:17:07.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.757 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.757 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.757 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504263+0000", 2024-09-20T22:17:07.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7412999999999997e-05 2024-09-20T22:17:07.757 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:17:07.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504540+0000", 2024-09-20T22:17:07.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00027761399999999997 2024-09-20T22:17:07.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:17:07.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510219+0000", 2024-09-20T22:17:07.758 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005678476 2024-09-20T22:17:07.758 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510260+0000", 2024-09-20T22:17:07.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1452000000000002e-05 2024-09-20T22:17:07.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510339+0000", 2024-09-20T22:17:07.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8669000000000002e-05 2024-09-20T22:17:07.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510393+0000", 2024-09-20T22:17:07.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4160999999999999e-05 2024-09-20T22:17:07.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510439+0000", 2024-09-20T22:17:07.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6037e-05 2024-09-20T22:17:07.760 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.760 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.760 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.760 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214367 8.8 8:15e42233:::benchmark_data_smithi060_111051_object94368:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:17:07.760 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.530758+0000", 2024-09-20T22:17:07.760 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 370.156527052, 2024-09-20T22:17:07.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98471673000000004, 2024-09-20T22:17:07.760 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.760 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.761 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.761 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:17:07.761 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:17:07.761 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:07.761 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214367 2024-09-20T22:17:07.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.761 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530758+0000", 2024-09-20T22:17:07.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530758+0000", 2024-09-20T22:17:07.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530759+0000", 2024-09-20T22:17:07.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-07 2024-09-20T22:17:07.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.762 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.762 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530764+0000", 2024-09-20T22:17:07.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5390000000000001e-06 2024-09-20T22:17:07.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530773+0000", 2024-09-20T22:17:07.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.986e-06 2024-09-20T22:17:07.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530780+0000", 2024-09-20T22:17:07.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8689999999999998e-06 2024-09-20T22:17:07.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530827+0000", 2024-09-20T22:17:07.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7314000000000001e-05 2024-09-20T22:17:07.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:17:07.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530832+0000", 2024-09-20T22:17:07.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.758e-06 2024-09-20T22:17:07.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.505913+0000", 2024-09-20T22:17:07.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97508115200000001 2024-09-20T22:17:07.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.506026+0000", 2024-09-20T22:17:07.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011298000000000001 2024-09-20T22:17:07.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:17:07.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.506308+0000", 2024-09-20T22:17:07.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000282477 2024-09-20T22:17:07.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:17:07.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510274+0000", 2024-09-20T22:17:07.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0039657160000000002 2024-09-20T22:17:07.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510478+0000", 2024-09-20T22:17:07.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204002 2024-09-20T22:17:07.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:07.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.515367+0000", 2024-09-20T22:17:07.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0048889099999999998 2024-09-20T22:17:07.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.515428+0000", 2024-09-20T22:17:07.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0826999999999999e-05 2024-09-20T22:17:07.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.515475+0000", 2024-09-20T22:17:07.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7388000000000001e-05 2024-09-20T22:17:07.767 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.767 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.767 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.767 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:95467 10.3 e403/370 10:cb61ccb2:::benchmark_data_smithi060_167766_object95466:head v 403'5960, mlcod=403'5953)", 2024-09-20T22:17:07.767 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:16:20.860836+0000", 2024-09-20T22:17:07.767 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 46.826449799999999, 2024-09-20T22:17:07.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.036401876999999999, 2024-09-20T22:17:07.767 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.768 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.768 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.768 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:20.860836+0000", 2024-09-20T22:17:07.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:20.860836+0000", 2024-09-20T22:17:07.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:20.860837+0000", 2024-09-20T22:17:07.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.054e-06 2024-09-20T22:17:07.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:20.860860+0000", 2024-09-20T22:17:07.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3312e-05 2024-09-20T22:17:07.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:20.860876+0000", 2024-09-20T22:17:07.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6184e-05 2024-09-20T22:17:07.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:20.860884+0000", 2024-09-20T22:17:07.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4730000000000003e-06 2024-09-20T22:17:07.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:20.871732+0000", 2024-09-20T22:17:07.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010847841 2024-09-20T22:17:07.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:20.871818+0000", 2024-09-20T22:17:07.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6682999999999997e-05 2024-09-20T22:17:07.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:20.897051+0000", 2024-09-20T22:17:07.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.025232827999999999 2024-09-20T22:17:07.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:20.897238+0000", 2024-09-20T22:17:07.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018650200000000001 2024-09-20T22:17:07.771 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.771 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.771 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.772 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:115927 10.5 e410/370 10:ae1020f4:::benchmark_data_smithi060_167766_object115926:head v 410'7262, mlcod=410'7258)", 2024-09-20T22:17:07.772 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:16:40.299201+0000", 2024-09-20T22:17:07.772 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 27.388084937999999, 2024-09-20T22:17:07.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061993108999999998, 2024-09-20T22:17:07.772 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.772 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.772 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.772 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.299201+0000", 2024-09-20T22:17:07.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.299201+0000", 2024-09-20T22:17:07.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.299202+0000", 2024-09-20T22:17:07.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7300000000000004e-07 2024-09-20T22:17:07.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.299225+0000", 2024-09-20T22:17:07.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3261e-05 2024-09-20T22:17:07.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.299236+0000", 2024-09-20T22:17:07.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1590000000000001e-05 2024-09-20T22:17:07.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.299244+0000", 2024-09-20T22:17:07.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8790000000000003e-06 2024-09-20T22:17:07.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.299362+0000", 2024-09-20T22:17:07.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118255 2024-09-20T22:17:07.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.299382+0000", 2024-09-20T22:17:07.775 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9936999999999999e-05 2024-09-20T22:17:07.775 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.775 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.775 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.775 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.361087+0000", 2024-09-20T22:17:07.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061704145000000002 2024-09-20T22:17:07.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.776 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.776 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.361194+0000", 2024-09-20T22:17:07.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000107069 2024-09-20T22:17:07.776 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.776 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.776 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.776 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.776 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.776 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:115931 10.5 e410/370 10:a27edb24:::benchmark_data_smithi060_167766_object115930:head v 410'7263, mlcod=410'7259)", 2024-09-20T22:17:07.776 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:16:40.302367+0000", 2024-09-20T22:17:07.776 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 27.384918655, 2024-09-20T22:17:07.776 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.063059085000000001, 2024-09-20T22:17:07.776 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.777 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.777 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.777 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.302367+0000", 2024-09-20T22:17:07.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.302367+0000", 2024-09-20T22:17:07.777 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.777 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.777 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.777 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.777 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.302368+0000", 2024-09-20T22:17:07.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1820000000000001e-06 2024-09-20T22:17:07.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.302391+0000", 2024-09-20T22:17:07.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3479999999999999e-05 2024-09-20T22:17:07.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.778 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.778 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.778 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.302406+0000", 2024-09-20T22:17:07.778 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4287000000000001e-05 2024-09-20T22:17:07.778 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.302414+0000", 2024-09-20T22:17:07.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2640000000000008e-06 2024-09-20T22:17:07.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.303184+0000", 2024-09-20T22:17:07.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00076964899999999996 2024-09-20T22:17:07.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.779 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.779 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.303203+0000", 2024-09-20T22:17:07.779 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9057999999999999e-05 2024-09-20T22:17:07.779 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.779 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.364948+0000", 2024-09-20T22:17:07.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061744868000000001 2024-09-20T22:17:07.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.780 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.780 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.365426+0000", 2024-09-20T22:17:07.780 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00047829699999999998 2024-09-20T22:17:07.780 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.780 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.780 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.780 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.780 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.780 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:115938 10.f e410/370 10:f921f01a:::benchmark_data_smithi060_167766_object115937:head v 410'7143, mlcod=410'7141)", 2024-09-20T22:17:07.780 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:16:40.305071+0000", 2024-09-20T22:17:07.780 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 27.382214038000001, 2024-09-20T22:17:07.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.056598686000000002, 2024-09-20T22:17:07.781 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.781 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.781 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.781 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.781 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.781 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.305071+0000", 2024-09-20T22:17:07.781 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.781 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.781 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.305071+0000", 2024-09-20T22:17:07.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.305073+0000", 2024-09-20T22:17:07.782 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1990000000000001e-06 2024-09-20T22:17:07.782 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.782 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.782 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.782 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.305140+0000", 2024-09-20T22:17:07.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7525000000000001e-05 2024-09-20T22:17:07.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.354568+0000", 2024-09-20T22:17:07.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.049427472 2024-09-20T22:17:07.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.354588+0000", 2024-09-20T22:17:07.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0207000000000002e-05 2024-09-20T22:17:07.783 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.783 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.783 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.783 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.355247+0000", 2024-09-20T22:17:07.783 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00065898300000000003 2024-09-20T22:17:07.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.784 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.355266+0000", 2024-09-20T22:17:07.784 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9015e-05 2024-09-20T22:17:07.784 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.784 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.784 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.785 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.361130+0000", 2024-09-20T22:17:07.785 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005864073 2024-09-20T22:17:07.785 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.786 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.786 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.361670+0000", 2024-09-20T22:17:07.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00054021199999999998 2024-09-20T22:17:07.786 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.786 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.786 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.786 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.786 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.786 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:115942 10.4 e410/370 10:21687d53:::benchmark_data_smithi060_167766_object115941:head v 410'7206, mlcod=410'7204)", 2024-09-20T22:17:07.786 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:16:40.308513+0000", 2024-09-20T22:17:07.786 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 27.378772112, 2024-09-20T22:17:07.786 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.053038452, 2024-09-20T22:17:07.787 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.787 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.787 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.787 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.308513+0000", 2024-09-20T22:17:07.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.787 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.787 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.787 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.308513+0000", 2024-09-20T22:17:07.787 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.787 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.308515+0000", 2024-09-20T22:17:07.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5409999999999999e-06 2024-09-20T22:17:07.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.308544+0000", 2024-09-20T22:17:07.788 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9045000000000001e-05 2024-09-20T22:17:07.788 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.788 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.788 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.788 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.354799+0000", 2024-09-20T22:17:07.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.046254847000000002 2024-09-20T22:17:07.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.354811+0000", 2024-09-20T22:17:07.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1768e-05 2024-09-20T22:17:07.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.789 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.354973+0000", 2024-09-20T22:17:07.789 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000162244 2024-09-20T22:17:07.789 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.789 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.789 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.354999+0000", 2024-09-20T22:17:07.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6478e-05 2024-09-20T22:17:07.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.361296+0000", 2024-09-20T22:17:07.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0062968590000000001 2024-09-20T22:17:07.790 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.790 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.790 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.790 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.361552+0000", 2024-09-20T22:17:07.790 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00025567 2024-09-20T22:17:07.790 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.791 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.791 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.791 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.791 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:133321 10.f e412/370 10:f07f2f51:::benchmark_data_smithi060_167766_object8529:head v 412'8250, mlcod=412'8247)", 2024-09-20T22:17:07.791 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:16:54.888494+0000", 2024-09-20T22:17:07.791 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 12.798791696, 2024-09-20T22:17:07.791 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028167217000000001, 2024-09-20T22:17:07.791 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.791 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.791 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.791 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:54.888494+0000", 2024-09-20T22:17:07.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:54.888494+0000", 2024-09-20T22:17:07.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.792 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:54.888494+0000", 2024-09-20T22:17:07.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9400000000000005e-07 2024-09-20T22:17:07.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:54.888499+0000", 2024-09-20T22:17:07.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7160000000000002e-06 2024-09-20T22:17:07.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:54.888509+0000", 2024-09-20T22:17:07.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.022e-05 2024-09-20T22:17:07.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:54.888516+0000", 2024-09-20T22:17:07.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2639999999999997e-06 2024-09-20T22:17:07.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:54.892394+0000", 2024-09-20T22:17:07.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038787259999999999 2024-09-20T22:17:07.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:54.892413+0000", 2024-09-20T22:17:07.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8722000000000001e-05 2024-09-20T22:17:07.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:54.916616+0000", 2024-09-20T22:17:07.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.024203395999999999 2024-09-20T22:17:07.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:54.916661+0000", 2024-09-20T22:17:07.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4579000000000001e-05 2024-09-20T22:17:07.795 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.795 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.795 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.795 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:142779 10.5 e415/370 10:af8f14ec:::benchmark_data_smithi060_167766_object17987:head v 415'8947, mlcod=415'8944)", 2024-09-20T22:17:07.796 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:17:02.930659+0000", 2024-09-20T22:17:07.796 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4.7566265620000001, 2024-09-20T22:17:07.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028483886, 2024-09-20T22:17:07.796 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.796 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.796 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.796 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:02.930659+0000", 2024-09-20T22:17:07.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:02.930659+0000", 2024-09-20T22:17:07.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:02.930659+0000", 2024-09-20T22:17:07.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7399999999999999e-07 2024-09-20T22:17:07.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:02.930663+0000", 2024-09-20T22:17:07.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2509999999999996e-06 2024-09-20T22:17:07.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.798 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:02.930670+0000", 2024-09-20T22:17:07.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3679999999999998e-06 2024-09-20T22:17:07.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.798 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:02.930675+0000", 2024-09-20T22:17:07.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.809e-06 2024-09-20T22:17:07.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.798 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:02.933365+0000", 2024-09-20T22:17:07.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002690067 2024-09-20T22:17:07.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:02.933383+0000", 2024-09-20T22:17:07.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8720999999999999e-05 2024-09-20T22:17:07.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:02.959104+0000", 2024-09-20T22:17:07.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.025720476999999999 2024-09-20T22:17:07.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:02.959143+0000", 2024-09-20T22:17:07.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8918999999999998e-05 2024-09-20T22:17:07.800 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.800 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.800 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.800 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:142892 10.5 e415/370 10:a29c9490:::benchmark_data_smithi060_167766_object18100:head v 415'8954, mlcod=415'8952)", 2024-09-20T22:17:07.800 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:17:03.003595+0000", 2024-09-20T22:17:07.800 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 4.6836907989999998, 2024-09-20T22:17:07.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.028772421999999999, 2024-09-20T22:17:07.800 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.800 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.800 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.801 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:03.003595+0000", 2024-09-20T22:17:07.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:03.003595+0000", 2024-09-20T22:17:07.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:03.003595+0000", 2024-09-20T22:17:07.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.6399999999999998e-07 2024-09-20T22:17:07.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:03.003599+0000", 2024-09-20T22:17:07.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4170000000000002e-06 2024-09-20T22:17:07.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:03.003606+0000", 2024-09-20T22:17:07.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2829999999999998e-06 2024-09-20T22:17:07.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:03.003611+0000", 2024-09-20T22:17:07.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9030000000000003e-06 2024-09-20T22:17:07.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.803 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.803 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:03.006321+0000", 2024-09-20T22:17:07.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00271024 2024-09-20T22:17:07.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.803 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.803 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:03.006372+0000", 2024-09-20T22:17:07.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0762e-05 2024-09-20T22:17:07.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.803 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.803 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:03.032179+0000", 2024-09-20T22:17:07.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.025807026 2024-09-20T22:17:07.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:03.032367+0000", 2024-09-20T22:17:07.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018842699999999999 2024-09-20T22:17:07.804 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.804 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.804 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.804 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:147137 10.3 e416/370 10:cb5096ec:::benchmark_data_smithi060_167766_object22345:head v 416'9220, mlcod=416'9217)", 2024-09-20T22:17:07.804 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:17:06.402769+0000", 2024-09-20T22:17:07.804 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.2845160789999999, 2024-09-20T22:17:07.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.026307166999999999, 2024-09-20T22:17:07.804 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.805 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.805 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.805 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:06.402769+0000", 2024-09-20T22:17:07.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:06.402769+0000", 2024-09-20T22:17:07.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:06.402770+0000", 2024-09-20T22:17:07.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4600000000000003e-07 2024-09-20T22:17:07.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:06.402777+0000", 2024-09-20T22:17:07.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3929999999999996e-06 2024-09-20T22:17:07.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:06.402790+0000", 2024-09-20T22:17:07.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3399e-05 2024-09-20T22:17:07.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:06.402798+0000", 2024-09-20T22:17:07.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5440000000000001e-06 2024-09-20T22:17:07.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:06.408699+0000", 2024-09-20T22:17:07.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005901313 2024-09-20T22:17:07.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:06.409379+0000", 2024-09-20T22:17:07.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00067979100000000001 2024-09-20T22:17:07.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:06.429026+0000", 2024-09-20T22:17:07.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.019647319 2024-09-20T22:17:07.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:06.429077+0000", 2024-09-20T22:17:07.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0562000000000001e-05 2024-09-20T22:17:07.808 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.808 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.808 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.809 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:148481 10.f e416/370 10:ff1cb986:::benchmark_data_smithi060_167766_object23689:head v 416'9181, mlcod=416'9177)", 2024-09-20T22:17:07.809 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:17:07.618060+0000", 2024-09-20T22:17:07.809 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.069225657999999995, 2024-09-20T22:17:07.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021080068, 2024-09-20T22:17:07.809 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.809 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.809 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.809 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.618060+0000", 2024-09-20T22:17:07.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.618060+0000", 2024-09-20T22:17:07.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.618061+0000", 2024-09-20T22:17:07.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.879999999999999e-07 2024-09-20T22:17:07.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.618068+0000", 2024-09-20T22:17:07.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3810000000000001e-06 2024-09-20T22:17:07.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.618078+0000", 2024-09-20T22:17:07.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7450000000000007e-06 2024-09-20T22:17:07.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.618085+0000", 2024-09-20T22:17:07.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4050000000000003e-06 2024-09-20T22:17:07.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.621344+0000", 2024-09-20T22:17:07.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0032582290000000001 2024-09-20T22:17:07.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.621407+0000", 2024-09-20T22:17:07.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3304000000000002e-05 2024-09-20T22:17:07.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.639087+0000", 2024-09-20T22:17:07.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.017680224000000001 2024-09-20T22:17:07.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.639140+0000", 2024-09-20T22:17:07.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2791999999999999e-05 2024-09-20T22:17:07.813 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.813 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.813 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.813 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.813 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.813 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:148482 10.f e416/370 10:f4622df9:::benchmark_data_smithi060_167766_object23690:head v 416'9182, mlcod=416'9177)", 2024-09-20T22:17:07.813 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:17:07.621515+0000", 2024-09-20T22:17:07.813 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.065770506000000006, 2024-09-20T22:17:07.813 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.017658165, 2024-09-20T22:17:07.813 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:17:07.813 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:17:07.813 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:17:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:17:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.621515+0000", 2024-09-20T22:17:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:17:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.621515+0000", 2024-09-20T22:17:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:17:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.814 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:17:07.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.621516+0000", 2024-09-20T22:17:07.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.206e-06 2024-09-20T22:17:07.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:17:07.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.621524+0000", 2024-09-20T22:17:07.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7349999999999996e-06 2024-09-20T22:17:07.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.815 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.815 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:17:07.815 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.622182+0000", 2024-09-20T22:17:07.815 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00065776599999999995 2024-09-20T22:17:07.815 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:17:07.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.622192+0000", 2024-09-20T22:17:07.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0315e-05 2024-09-20T22:17:07.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:17:07.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.624260+0000", 2024-09-20T22:17:07.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020680870000000001 2024-09-20T22:17:07.816 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.816 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.816 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:17:07.816 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.624278+0000", 2024-09-20T22:17:07.816 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8037e-05 2024-09-20T22:17:07.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:17:07.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.639145+0000", 2024-09-20T22:17:07.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014866667 2024-09-20T22:17:07.817 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:07.817 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:07.817 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:17:07.817 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:07.639173+0000", 2024-09-20T22:17:07.817 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8351999999999999e-05 2024-09-20T22:17:07.817 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.817 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.817 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.817 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:07.817 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:07.818 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:17:07.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:07.819 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:07.819 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:07.920 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:07.965 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:07.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:07.965 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:08.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:07 smithi060 ceph-mon[29694]: 6.16 scrub starts 2024-09-20T22:17:08.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:07 smithi060 ceph-mon[29694]: 6.16 scrub ok 2024-09-20T22:17:08.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:07 smithi060 ceph-mon[29694]: from='client.16215 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:17:08.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:07 smithi060 ceph-mon[29694]: 2.2 scrub starts 2024-09-20T22:17:08.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:07 smithi060 ceph-mon[29694]: 2.2 scrub ok 2024-09-20T22:17:08.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:07 smithi060 ceph-mon[29694]: 3.19 scrub starts 2024-09-20T22:17:08.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:07 smithi060 ceph-mon[29694]: 3.19 scrub ok 2024-09-20T22:17:08.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:07 smithi060 ceph-mon[40290]: 6.16 scrub starts 2024-09-20T22:17:08.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:07 smithi060 ceph-mon[40290]: 6.16 scrub ok 2024-09-20T22:17:08.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:07 smithi060 ceph-mon[40290]: from='client.16215 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:17:08.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:07 smithi060 ceph-mon[40290]: 2.2 scrub starts 2024-09-20T22:17:08.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:07 smithi060 ceph-mon[40290]: 2.2 scrub ok 2024-09-20T22:17:08.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:07 smithi060 ceph-mon[40290]: 3.19 scrub starts 2024-09-20T22:17:08.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:07 smithi060 ceph-mon[40290]: 3.19 scrub ok 2024-09-20T22:17:08.066 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:08.100 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:08.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:08.101 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:08.202 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:08.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:08.235 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:08.235 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:08.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:07 smithi067 ceph-mon[33511]: 6.16 scrub starts 2024-09-20T22:17:08.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:07 smithi067 ceph-mon[33511]: 6.16 scrub ok 2024-09-20T22:17:08.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:07 smithi067 ceph-mon[33511]: from='client.16215 -' entity='client.admin' cmd=[{"prefix": "osd reweight-by-utilization", "oload": 1000, "max_change": 3.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:17:08.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:07 smithi067 ceph-mon[33511]: 2.2 scrub starts 2024-09-20T22:17:08.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:07 smithi067 ceph-mon[33511]: 2.2 scrub ok 2024-09-20T22:17:08.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:07 smithi067 ceph-mon[33511]: 3.19 scrub starts 2024-09-20T22:17:08.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:07 smithi067 ceph-mon[33511]: 3.19 scrub ok 2024-09-20T22:17:08.335 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:08.381 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:08.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:08.382 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:08.483 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:08.515 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:08.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:08.516 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:08.617 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:08.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:08.663 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:08.663 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:08.764 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:08.817 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:08.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:08.818 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:08.919 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:08.956 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:08.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:08.957 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:09.058 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:09.092 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:09.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:09.093 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:09.193 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:09.226 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:09.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:09.227 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:08 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:17:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:08 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:17:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:08 smithi067 ceph-mon[33511]: 3.1b scrub starts 2024-09-20T22:17:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:08 smithi067 ceph-mon[33511]: 3.1b scrub ok 2024-09-20T22:17:09.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:08 smithi067 ceph-mon[33511]: 1.f deep-scrub starts 2024-09-20T22:17:09.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:08 smithi067 ceph-mon[33511]: 1.f deep-scrub ok 2024-09-20T22:17:09.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:08 smithi067 ceph-mon[33511]: osdmap e417: 8 total, 8 up, 8 in 2024-09-20T22:17:09.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:08 smithi067 ceph-mon[33511]: pgmap v761: 168 pgs: 1 peering, 167 active+clean; 6.2 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 1.49k op/s; 18 B/s, 0 objects/s recovering 2024-09-20T22:17:09.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:08 smithi067 ceph-mon[33511]: 4.4 deep-scrub starts 2024-09-20T22:17:09.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:08 smithi067 ceph-mon[33511]: 4.4 deep-scrub ok 2024-09-20T22:17:09.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:08 smithi067 ceph-mon[33511]: 3.a deep-scrub starts 2024-09-20T22:17:09.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:08 smithi067 ceph-mon[33511]: 3.a deep-scrub ok 2024-09-20T22:17:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:17:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:17:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[29694]: 3.1b scrub starts 2024-09-20T22:17:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[29694]: 3.1b scrub ok 2024-09-20T22:17:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[29694]: 1.f deep-scrub starts 2024-09-20T22:17:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[29694]: 1.f deep-scrub ok 2024-09-20T22:17:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[29694]: osdmap e417: 8 total, 8 up, 8 in 2024-09-20T22:17:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[29694]: pgmap v761: 168 pgs: 1 peering, 167 active+clean; 6.2 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 1.49k op/s; 18 B/s, 0 objects/s recovering 2024-09-20T22:17:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[29694]: 4.4 deep-scrub starts 2024-09-20T22:17:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[29694]: 4.4 deep-scrub ok 2024-09-20T22:17:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[29694]: 3.a deep-scrub starts 2024-09-20T22:17:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[29694]: 3.a deep-scrub ok 2024-09-20T22:17:09.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:17:09.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:17:09.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[40290]: 3.1b scrub starts 2024-09-20T22:17:09.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[40290]: 3.1b scrub ok 2024-09-20T22:17:09.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[40290]: 1.f deep-scrub starts 2024-09-20T22:17:09.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[40290]: 1.f deep-scrub ok 2024-09-20T22:17:09.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[40290]: osdmap e417: 8 total, 8 up, 8 in 2024-09-20T22:17:09.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[40290]: pgmap v761: 168 pgs: 1 peering, 167 active+clean; 6.2 GiB data, 24 GiB used, 692 GiB / 715 GiB avail; 682 B/s rd, 0 B/s wr, 1.49k op/s; 18 B/s, 0 objects/s recovering 2024-09-20T22:17:09.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[40290]: 4.4 deep-scrub starts 2024-09-20T22:17:09.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[40290]: 4.4 deep-scrub ok 2024-09-20T22:17:09.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[40290]: 3.a deep-scrub starts 2024-09-20T22:17:09.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:08 smithi060 ceph-mon[40290]: 3.a deep-scrub ok 2024-09-20T22:17:09.328 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:09.356 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:09.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:09.357 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:09.458 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:09.486 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:09.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:09.487 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:09.588 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:09.631 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:09.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:09.632 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:09.733 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:09.765 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:09.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:09.766 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:09.867 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:09.909 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:09.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:09.910 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:09.993 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-09-20T22:17:09.993 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd test-reweight-by-utilization 110 0.05 2024-09-20T22:17:10.011 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:10.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:10.041 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:10.041 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:10.142 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:10.173 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:10.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:10.174 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:10.246 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:10.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:09 smithi067 ceph-mon[33511]: 5.10 scrub starts 2024-09-20T22:17:10.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:09 smithi067 ceph-mon[33511]: 5.10 scrub ok 2024-09-20T22:17:10.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:09 smithi067 ceph-mon[33511]: 5.6 scrub starts 2024-09-20T22:17:10.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:09 smithi067 ceph-mon[33511]: 5.6 scrub ok 2024-09-20T22:17:10.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:09 smithi067 ceph-mon[33511]: 5.7 deep-scrub starts 2024-09-20T22:17:10.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:09 smithi067 ceph-mon[33511]: 5.7 deep-scrub ok 2024-09-20T22:17:10.275 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:10.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:09 smithi060 ceph-mon[29694]: 5.10 scrub starts 2024-09-20T22:17:10.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:09 smithi060 ceph-mon[29694]: 5.10 scrub ok 2024-09-20T22:17:10.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:09 smithi060 ceph-mon[29694]: 5.6 scrub starts 2024-09-20T22:17:10.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:09 smithi060 ceph-mon[29694]: 5.6 scrub ok 2024-09-20T22:17:10.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:09 smithi060 ceph-mon[29694]: 5.7 deep-scrub starts 2024-09-20T22:17:10.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:09 smithi060 ceph-mon[29694]: 5.7 deep-scrub ok 2024-09-20T22:17:10.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:09 smithi060 ceph-mon[40290]: 5.10 scrub starts 2024-09-20T22:17:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:09 smithi060 ceph-mon[40290]: 5.10 scrub ok 2024-09-20T22:17:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:09 smithi060 ceph-mon[40290]: 5.6 scrub starts 2024-09-20T22:17:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:09 smithi060 ceph-mon[40290]: 5.6 scrub ok 2024-09-20T22:17:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:09 smithi060 ceph-mon[40290]: 5.7 deep-scrub starts 2024-09-20T22:17:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:09 smithi060 ceph-mon[40290]: 5.7 deep-scrub ok 2024-09-20T22:17:10.311 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:10.312 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:10.312 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:10.413 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:10.461 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:10.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:10.462 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:10.563 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:10.597 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:10.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:10.598 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:10.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:10.733 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T22:17:10.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:10.735 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:10.735 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:10.836 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:10.879 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:10.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:10.880 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:10.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:11.034 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:11.035 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:11.035 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:11.072 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:10 smithi067 ceph-mon[33511]: 5.1c scrub starts 2024-09-20T22:17:11.072 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:10 smithi067 ceph-mon[33511]: 5.1c scrub ok 2024-09-20T22:17:11.072 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:10 smithi067 ceph-mon[33511]: 3.0 scrub starts 2024-09-20T22:17:11.073 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:10 smithi067 ceph-mon[33511]: 3.0 scrub ok 2024-09-20T22:17:11.073 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:10 smithi067 ceph-mon[33511]: 6.10 scrub starts 2024-09-20T22:17:11.073 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:10 smithi067 ceph-mon[33511]: 6.10 scrub ok 2024-09-20T22:17:11.073 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:10 smithi067 ceph-mon[33511]: 3.5 scrub starts 2024-09-20T22:17:11.073 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:10 smithi067 ceph-mon[33511]: 3.5 scrub ok 2024-09-20T22:17:11.073 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:10 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:17:11.073 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:10 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:17:11.073 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:10 smithi067 ceph-mon[33511]: pgmap v762: 168 pgs: 168 active+clean; 6.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 536 B/s rd, 0 B/s wr, 1.12k op/s; 14 B/s, 0 objects/s recovering 2024-09-20T22:17:11.136 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:11.178 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:11.178 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:11.178 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:11.279 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[29694]: 5.1c scrub starts 2024-09-20T22:17:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[29694]: 5.1c scrub ok 2024-09-20T22:17:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[29694]: 3.0 scrub starts 2024-09-20T22:17:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[29694]: 3.0 scrub ok 2024-09-20T22:17:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[29694]: 6.10 scrub starts 2024-09-20T22:17:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[29694]: 6.10 scrub ok 2024-09-20T22:17:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[29694]: 3.5 scrub starts 2024-09-20T22:17:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[29694]: 3.5 scrub ok 2024-09-20T22:17:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:17:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:17:11.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[29694]: pgmap v762: 168 pgs: 168 active+clean; 6.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 536 B/s rd, 0 B/s wr, 1.12k op/s; 14 B/s, 0 objects/s recovering 2024-09-20T22:17:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[40290]: 5.1c scrub starts 2024-09-20T22:17:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[40290]: 5.1c scrub ok 2024-09-20T22:17:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[40290]: 3.0 scrub starts 2024-09-20T22:17:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[40290]: 3.0 scrub ok 2024-09-20T22:17:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[40290]: 6.10 scrub starts 2024-09-20T22:17:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[40290]: 6.10 scrub ok 2024-09-20T22:17:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[40290]: 3.5 scrub starts 2024-09-20T22:17:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[40290]: 3.5 scrub ok 2024-09-20T22:17:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:17:11.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:17:11.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:10 smithi060 ceph-mon[40290]: pgmap v762: 168 pgs: 168 active+clean; 6.2 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 536 B/s rd, 0 B/s wr, 1.12k op/s; 14 B/s, 0 objects/s recovering 2024-09-20T22:17:11.294 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:17:11.323 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:11.324 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:11.335 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:11.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:11.336 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:11.437 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:11.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:11.479 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:11.479 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:11.579 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:11.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:11.618 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:11.618 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:11.719 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:11.760 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:11.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:11.761 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:11.861 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:11.895 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:11.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:11.896 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:11.996 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:12.003 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:17:12.031 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:12.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:12.033 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:12.133 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:12.164 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:12.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:12.165 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:12.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:12 smithi067 ceph-mon[33511]: 10.2 scrub starts 2024-09-20T22:17:12.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:12 smithi067 ceph-mon[33511]: 3.11 scrub starts 2024-09-20T22:17:12.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:12 smithi067 ceph-mon[33511]: 3.11 scrub ok 2024-09-20T22:17:12.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:12 smithi067 ceph-mon[33511]: 5.1e scrub starts 2024-09-20T22:17:12.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:12 smithi067 ceph-mon[33511]: 5.1e scrub ok 2024-09-20T22:17:12.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:12 smithi067 ceph-mon[33511]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-20T22:17:12.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:12 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:17:12.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:12 smithi067 ceph-mon[33511]: osdmap e418: 8 total, 8 up, 8 in 2024-09-20T22:17:12.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:12 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2227442300' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:17:12.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:12 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:17:12.265 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:12.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[29694]: 10.2 scrub starts 2024-09-20T22:17:12.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[29694]: 3.11 scrub starts 2024-09-20T22:17:12.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[29694]: 3.11 scrub ok 2024-09-20T22:17:12.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[29694]: 5.1e scrub starts 2024-09-20T22:17:12.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[29694]: 5.1e scrub ok 2024-09-20T22:17:12.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[29694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-20T22:17:12.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:17:12.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[29694]: osdmap e418: 8 total, 8 up, 8 in 2024-09-20T22:17:12.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2227442300' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:17:12.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:17:12.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[40290]: 10.2 scrub starts 2024-09-20T22:17:12.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[40290]: 3.11 scrub starts 2024-09-20T22:17:12.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[40290]: 3.11 scrub ok 2024-09-20T22:17:12.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[40290]: 5.1e scrub starts 2024-09-20T22:17:12.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[40290]: 5.1e scrub ok 2024-09-20T22:17:12.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[40290]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-20T22:17:12.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:17:12.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[40290]: osdmap e418: 8 total, 8 up, 8 in 2024-09-20T22:17:12.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2227442300' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:17:12.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:17:12.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:12.309 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:12.309 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:12.337 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:17:12.337 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:17:12.337 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.338 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:153948 10.1 10:894dc0f2:::benchmark_data_smithi060_167766_object29156:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e419)", 2024-09-20T22:17:12.338 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:12.315919+0000", 2024-09-20T22:17:12.338 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.019118546, 2024-09-20T22:17:12.338 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.019155103999999999, 2024-09-20T22:17:12.338 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:12.338 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:12.338 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:17:12.338 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:12.338 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:12.338 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:12.338 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 153948 2024-09-20T22:17:12.338 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.338 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:12.339 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.339 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:12.339 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.315919+0000", 2024-09-20T22:17:12.339 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:12.339 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.339 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.339 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:12.339 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.315919+0000", 2024-09-20T22:17:12.339 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:12.339 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.339 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.339 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:12.339 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.315921+0000", 2024-09-20T22:17:12.339 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.632e-06 2024-09-20T22:17:12.339 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.339 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.340 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:12.340 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.315928+0000", 2024-09-20T22:17:12.340 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1559999999999998e-06 2024-09-20T22:17:12.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.340 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.340 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:12.340 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.315943+0000", 2024-09-20T22:17:12.340 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5438e-05 2024-09-20T22:17:12.340 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.340 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.340 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:12.340 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.315951+0000", 2024-09-20T22:17:12.341 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5229999999999998e-06 2024-09-20T22:17:12.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.341 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.341 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:12.341 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.316016+0000", 2024-09-20T22:17:12.341 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5437000000000004e-05 2024-09-20T22:17:12.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.341 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.341 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:12.341 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.316155+0000", 2024-09-20T22:17:12.341 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013863600000000001 2024-09-20T22:17:12.341 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.341 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.341 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:17:12.341 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.316299+0000", 2024-09-20T22:17:12.341 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014428999999999999 2024-09-20T22:17:12.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.342 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.342 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:12.342 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.318693+0000", 2024-09-20T22:17:12.342 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0023941119999999999 2024-09-20T22:17:12.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.342 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.342 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:12.342 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.319623+0000", 2024-09-20T22:17:12.342 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00092931399999999999 2024-09-20T22:17:12.342 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:12.342 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:12.342 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:12.342 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.342 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.343 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:153934 10.1 10:8cc4957a:::benchmark_data_smithi060_167766_object29142:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e419)", 2024-09-20T22:17:12.343 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:12.303177+0000", 2024-09-20T22:17:12.343 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.031860239999999998, 2024-09-20T22:17:12.343 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.031996292000000003, 2024-09-20T22:17:12.343 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:12.343 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:12.343 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:17:12.343 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:12.343 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:12.343 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:12.343 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 153934 2024-09-20T22:17:12.343 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.343 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:12.343 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.343 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:12.344 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.303177+0000", 2024-09-20T22:17:12.344 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:12.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.344 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:12.344 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.303177+0000", 2024-09-20T22:17:12.344 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:12.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.344 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:12.344 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.303178+0000", 2024-09-20T22:17:12.344 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.102e-06 2024-09-20T22:17:12.344 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.344 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.344 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:12.344 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.303183+0000", 2024-09-20T22:17:12.345 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6800000000000001e-06 2024-09-20T22:17:12.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.345 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.345 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:12.345 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.303194+0000", 2024-09-20T22:17:12.345 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0889e-05 2024-09-20T22:17:12.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.345 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.345 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:12.345 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.303200+0000", 2024-09-20T22:17:12.345 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6439999999999997e-06 2024-09-20T22:17:12.345 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.345 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.345 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:12.345 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.303254+0000", 2024-09-20T22:17:12.345 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4511e-05 2024-09-20T22:17:12.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.346 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:12.346 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.303381+0000", 2024-09-20T22:17:12.346 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000126692 2024-09-20T22:17:12.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.346 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:17:12.346 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.303530+0000", 2024-09-20T22:17:12.346 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014886200000000001 2024-09-20T22:17:12.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.346 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.346 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:12.346 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.308842+0000", 2024-09-20T22:17:12.346 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0053119559999999996 2024-09-20T22:17:12.346 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.347 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:12.347 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.308996+0000", 2024-09-20T22:17:12.347 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015396400000000001 2024-09-20T22:17:12.347 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:12.347 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:12.347 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:12.347 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.347 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.347 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:153936 10.1 10:8800029a:::benchmark_data_smithi060_167766_object29144:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e419)", 2024-09-20T22:17:12.347 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:12.305125+0000", 2024-09-20T22:17:12.347 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.029912199, 2024-09-20T22:17:12.347 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.030141068, 2024-09-20T22:17:12.347 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:12.347 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:12.347 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:17:12.347 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:12.348 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:12.348 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:12.348 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 153936 2024-09-20T22:17:12.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.348 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:12.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.348 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:12.348 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.305125+0000", 2024-09-20T22:17:12.348 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:12.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.348 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:12.348 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.305125+0000", 2024-09-20T22:17:12.348 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:12.348 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.348 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.349 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:12.349 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.305126+0000", 2024-09-20T22:17:12.349 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8599999999999997e-07 2024-09-20T22:17:12.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.349 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:12.349 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.305131+0000", 2024-09-20T22:17:12.349 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7949999999999998e-06 2024-09-20T22:17:12.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.349 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:12.349 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.305145+0000", 2024-09-20T22:17:12.349 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3575e-05 2024-09-20T22:17:12.349 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.349 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.349 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:12.349 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.305150+0000", 2024-09-20T22:17:12.350 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3929999999999999e-06 2024-09-20T22:17:12.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.350 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:12.350 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.305394+0000", 2024-09-20T22:17:12.350 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000243627 2024-09-20T22:17:12.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.350 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:12.350 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.305521+0000", 2024-09-20T22:17:12.350 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012687700000000001 2024-09-20T22:17:12.350 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.350 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.350 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:17:12.350 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.305657+0000", 2024-09-20T22:17:12.351 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013607499999999999 2024-09-20T22:17:12.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.351 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:12.351 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.309126+0000", 2024-09-20T22:17:12.351 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0034693789999999999 2024-09-20T22:17:12.351 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:12.351 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:12.351 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:12.351 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:12.310443+0000", 2024-09-20T22:17:12.351 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0013166250000000001 2024-09-20T22:17:12.351 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:12.351 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:12.351 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:12.351 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:12.352 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-20T22:17:12.352 INFO:teuthology.orchestra.run.smithi067.stdout: "num_ops": 3 2024-09-20T22:17:12.352 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:17:12.410 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:12.445 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:12.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:12.445 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:12.546 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:12.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:12.575 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:12.575 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:12.610 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:17:12 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:17:12.525+0000 7fa558495640 -1 mgr.server reply reply (3) No such process no change 2024-09-20T22:17:12.676 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:12.710 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:12.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:12.742 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:12.843 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:12.875 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:17:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:17:12.877 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:12.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:12.879 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:12.979 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:13.007 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:13.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:13.008 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:13.109 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:13.146 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:13.146 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:13.146 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:13.247 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:13.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:13 smithi067 ceph-mon[33511]: 4.9 deep-scrub starts 2024-09-20T22:17:13.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:13 smithi067 ceph-mon[33511]: 4.9 deep-scrub ok 2024-09-20T22:17:13.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:13 smithi067 ceph-mon[33511]: 3.1 scrub starts 2024-09-20T22:17:13.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:13 smithi067 ceph-mon[33511]: 3.1 scrub ok 2024-09-20T22:17:13.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:13 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:17:13.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:13 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:17:13.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:13 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:17:13.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:13 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:17:13.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:13 smithi067 ceph-mon[33511]: osdmap e419: 8 total, 8 up, 8 in 2024-09-20T22:17:13.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:13 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3830384410' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:13.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:13 smithi067 ceph-mon[33511]: pgmap v765: 168 pgs: 168 active+clean; 6.0 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 681 B/s rd, 0 B/s wr, 1.26k op/s; 18 B/s, 0 objects/s recovering 2024-09-20T22:17:13.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:13 smithi067 ceph-mon[33511]: from='client.25786 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:17:13.273 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:13.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:13.274 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:13.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[29694]: 4.9 deep-scrub starts 2024-09-20T22:17:13.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[29694]: 4.9 deep-scrub ok 2024-09-20T22:17:13.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[29694]: 3.1 scrub starts 2024-09-20T22:17:13.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[29694]: 3.1 scrub ok 2024-09-20T22:17:13.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:17:13.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:17:13.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:17:13.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:17:13.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[29694]: osdmap e419: 8 total, 8 up, 8 in 2024-09-20T22:17:13.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3830384410' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:13.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[29694]: pgmap v765: 168 pgs: 168 active+clean; 6.0 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 681 B/s rd, 0 B/s wr, 1.26k op/s; 18 B/s, 0 objects/s recovering 2024-09-20T22:17:13.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[29694]: from='client.25786 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:17:13.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[40290]: 4.9 deep-scrub starts 2024-09-20T22:17:13.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[40290]: 4.9 deep-scrub ok 2024-09-20T22:17:13.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[40290]: 3.1 scrub starts 2024-09-20T22:17:13.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[40290]: 3.1 scrub ok 2024-09-20T22:17:13.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:17:13.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:17:13.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:17:13.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:17:13.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[40290]: osdmap e419: 8 total, 8 up, 8 in 2024-09-20T22:17:13.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3830384410' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:13.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[40290]: pgmap v765: 168 pgs: 168 active+clean; 6.0 GiB data, 23 GiB used, 692 GiB / 715 GiB avail; 681 B/s rd, 0 B/s wr, 1.26k op/s; 18 B/s, 0 objects/s recovering 2024-09-20T22:17:13.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:13 smithi060 ceph-mon[40290]: from='client.25786 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 110, "max_change": 0.05, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:17:13.375 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:13.396 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:13.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:13.397 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:13.498 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:13.541 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:13.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:13.542 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:13.642 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:13.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:13.684 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:13.684 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:13.785 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:13.811 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T22:17:13.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:13.831 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:13.831 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:13.932 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:13.965 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:13.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:13.966 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:14.067 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:14.090 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:14.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:14.091 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:14.192 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:14.237 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:14.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:14.238 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:14.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:14 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:17:14.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:14 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:17:14.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:14 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:17:14.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:14 smithi060 ceph-mon[29694]: osdmap e420: 8 total, 8 up, 8 in 2024-09-20T22:17:14.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:14 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:17:14.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:14 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:17:14.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:14 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:17:14.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:14 smithi060 ceph-mon[40290]: osdmap e420: 8 total, 8 up, 8 in 2024-09-20T22:17:14.338 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:14.360 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:14.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:14.361 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:14.462 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:14.483 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:14.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:14.484 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:14.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:14 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:17:14.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:14 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:17:14.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:14 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:17:14.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:14 smithi067 ceph-mon[33511]: osdmap e420: 8 total, 8 up, 8 in 2024-09-20T22:17:14.585 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:14.615 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:14.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:14.616 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:14.716 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:14.745 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:14.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:14.746 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:14.846 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:14.868 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:14.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:14.869 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:14.921 INFO:teuthology.orchestra.run.smithi060.stdout:moved 13 / 504 (2.57937%) 2024-09-20T22:17:14.921 INFO:teuthology.orchestra.run.smithi060.stdout:avg 63 2024-09-20T22:17:14.921 INFO:teuthology.orchestra.run.smithi060.stdout:stddev 4.89898 -> 3.67423 (expected baseline 7.42462) 2024-09-20T22:17:14.921 INFO:teuthology.orchestra.run.smithi060.stdout:min osd.2 with 53 -> 57 pgs (0.84127 -> 0.904762 * mean) 2024-09-20T22:17:14.922 INFO:teuthology.orchestra.run.smithi060.stdout:max osd.4 with 70 -> 64 pgs (1.11111 -> 1.01587 * mean) 2024-09-20T22:17:14.922 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:17:14.922 INFO:teuthology.orchestra.run.smithi060.stdout:oload 110 2024-09-20T22:17:14.922 INFO:teuthology.orchestra.run.smithi060.stdout:max_change 0.05 2024-09-20T22:17:14.922 INFO:teuthology.orchestra.run.smithi060.stdout:max_change_osds 4 2024-09-20T22:17:14.922 INFO:teuthology.orchestra.run.smithi060.stdout:average_utilization 0.0320 2024-09-20T22:17:14.922 INFO:teuthology.orchestra.run.smithi060.stdout:overload_utilization 0.0351 2024-09-20T22:17:14.922 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0 weight 0.9500 -> 0.9000 2024-09-20T22:17:14.922 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4 weight 0.9500 -> 0.9000 2024-09-20T22:17:14.922 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6 weight 0.9500 -> 0.9000 2024-09-20T22:17:14.923 INFO:teuthology.orchestra.run.smithi060.stderr:no change 2024-09-20T22:17:14.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:14.996 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:14.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:14.997 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:15.098 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:15.125 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:15.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:15.125 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:15.226 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:15.248 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:15.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:15.249 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:15.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:15 smithi067 ceph-mon[33511]: pgmap v767: 168 pgs: 168 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 851 B/s rd, 0 B/s wr, 1.02k op/s 2024-09-20T22:17:15.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:15 smithi060 ceph-mon[29694]: pgmap v767: 168 pgs: 168 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 851 B/s rd, 0 B/s wr, 1.02k op/s 2024-09-20T22:17:15.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:17:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:17:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:17:15.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:15 smithi060 ceph-mon[40290]: pgmap v767: 168 pgs: 168 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 851 B/s rd, 0 B/s wr, 1.02k op/s 2024-09-20T22:17:15.351 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:15.383 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:15.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:15.384 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:15.485 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:15.520 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:15.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:15.521 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:15.621 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:15.660 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:15.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:15.660 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:15.761 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:15.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:15.808 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:15.808 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:15.908 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:15.911 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:17:15.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:15.958 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:15.958 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:16.059 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:16.097 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:16.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:16.102 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:16.203 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:16.238 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:16.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:16.239 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:16.339 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:16.379 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:16.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:16.380 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:16.481 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:16.514 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:16.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:16.515 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:16.562 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:17:16.562 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:17:16.615 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:16.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:16.654 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:16.654 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:16.755 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:16.803 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:16.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:16.804 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:16.866 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:17:16.866 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:17:16.866 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:17:16.866 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:17:16.866 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:17:16.866 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:17:16.866 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:17:16.866 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:17:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:17:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:17:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:17:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:17:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:17:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:17:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:17:16.867 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:17:16.905 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:16.949 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:16.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:16.950 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:17.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:16 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:17:17.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:16 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:17:17.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:16 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:17.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:16 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:17.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:16 smithi060 ceph-mon[29694]: osdmap e421: 8 total, 8 up, 8 in 2024-09-20T22:17:17.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:16 smithi060 ceph-mon[29694]: pgmap v769: 168 pgs: 168 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 969 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-20T22:17:17.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:16 smithi060 ceph-mon[29694]: osdmap e422: 8 total, 8 up, 8 in 2024-09-20T22:17:17.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:16 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:17:17.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:16 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:17:17.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:16 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:17.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:16 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:17.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:16 smithi060 ceph-mon[40290]: osdmap e421: 8 total, 8 up, 8 in 2024-09-20T22:17:17.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:16 smithi060 ceph-mon[40290]: pgmap v769: 168 pgs: 168 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 969 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-20T22:17:17.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:16 smithi060 ceph-mon[40290]: osdmap e422: 8 total, 8 up, 8 in 2024-09-20T22:17:17.051 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:17.080 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:17.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:17.081 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:17.182 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:17.221 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:17.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:17.222 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:17.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:16 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:17:17.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:16 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:17:17.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:16 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:17.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:16 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:17.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:16 smithi067 ceph-mon[33511]: osdmap e421: 8 total, 8 up, 8 in 2024-09-20T22:17:17.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:16 smithi067 ceph-mon[33511]: pgmap v769: 168 pgs: 168 active+clean; 5.8 GiB data, 22 GiB used, 693 GiB / 715 GiB avail; 969 B/s rd, 0 B/s wr, 1.08k op/s 2024-09-20T22:17:17.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:16 smithi067 ceph-mon[33511]: osdmap e422: 8 total, 8 up, 8 in 2024-09-20T22:17:17.322 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:17.362 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:17.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:17.363 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:17.464 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:17.502 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:17.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:17.503 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:17.604 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:17.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:17.645 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:17.646 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:17.746 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:17.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:17.785 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:17.786 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:17.817 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T22:17:17.887 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:17.924 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:17.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:17.925 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:17.996 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '1.0', 'overage': '1000', 'type': 'test-reweight-by-utilization'} 2024-09-20T22:17:17.996 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd test-reweight-by-utilization 1000 1.0 2024-09-20T22:17:18.026 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:18.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:18.084 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:18.084 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:18.185 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:18.242 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:18.243 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:18.244 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:18.305 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:17:18.344 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:18.380 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:18.380 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:18.380 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:18.481 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:18.518 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:18.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:18.519 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:18.620 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:18.663 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:18.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:18.665 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:18.732 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:17:18.766 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:18.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:18.842 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:18.843 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:18.943 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:18.981 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:17:18.981 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:17:18.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:18.988 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:18.988 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:19.089 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:19.122 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:19.123 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:19.123 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:19.224 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:19.260 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:19.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:19.261 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:19.328 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:19.362 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:19.395 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:19.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:19.396 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:19.497 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:19.529 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:17:19.551 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:19.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:19.553 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:19.589 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:19 smithi060 ceph-mon[29694]: pgmap v771: 168 pgs: 168 active+clean; 5.5 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.39k op/s 2024-09-20T22:17:19.591 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:19 smithi060 ceph-mon[40290]: pgmap v771: 168 pgs: 168 active+clean; 5.5 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.39k op/s 2024-09-20T22:17:19.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:19.667 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:19 smithi067 ceph-mon[33511]: pgmap v771: 168 pgs: 168 active+clean; 5.5 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.39k op/s 2024-09-20T22:17:19.703 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T22:17:19.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:19.728 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:19.728 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:19.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:19.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:19.881 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:19.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:19.982 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:20.015 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:20.016 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:20.017 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:20.083 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:17:20.117 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:20.165 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:20.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:20.166 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:20.267 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:20.306 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:20.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:20.307 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:20.408 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:20.435 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:20.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:20.436 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:20.536 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:20.571 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:20.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:20.572 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:20.672 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:20.710 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:20.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:20.711 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:20.764 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:17:20.764 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:17:20.812 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:20.862 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:20.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:20.863 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:20.869 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:20.964 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:21.028 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:21.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:21.029 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:21.129 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:21.166 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:21.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:21.167 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:21.224 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:21.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:21.335 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:21.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:21.336 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:21.437 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:21.445 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:21 smithi067 ceph-mon[33511]: pgmap v772: 168 pgs: 168 active+clean; 5.5 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 705 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-20T22:17:21.445 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:21.445 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:17:21.445 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:21 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/288695483' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:21.445 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:21 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:21.471 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:21.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:21.472 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:21.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T22:17:21.573 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:21.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:21 smithi060 ceph-mon[29694]: pgmap v772: 168 pgs: 168 active+clean; 5.5 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 705 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-20T22:17:21.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:21.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:17:21.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:21 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/288695483' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:21.619 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:21 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:21.619 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:21 smithi060 ceph-mon[40290]: pgmap v772: 168 pgs: 168 active+clean; 5.5 GiB data, 22 GiB used, 694 GiB / 715 GiB avail; 705 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-20T22:17:21.620 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:21.620 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:17:21.620 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:21 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/288695483' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:21.620 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:21 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:21.636 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:21.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:21.637 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:21.738 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:21.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:21.781 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:21.781 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:21.881 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:21.935 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:21.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:21.936 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:21.937 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:17:21.986 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:17:22.037 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:22.070 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:22.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:22.071 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:22.172 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:22.204 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:22.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:22.206 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:22.306 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:22.346 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:22.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:22.347 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:22.388 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:17:22.388 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:17:22.388 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:17:22.388 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:17:22.388 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:17:22.388 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:17:22.388 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:17:22.389 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:17:22.408 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:17:22.408 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:17:22.445 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:17:22.446 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:17:22.448 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:22.480 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:17:22.480 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:17:22.490 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:22.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:22.491 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:22.514 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:17:22.514 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:17:22.592 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:22.614 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:22.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:22.615 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:22.717 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:22.750 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:22.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:22.751 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:22.763 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:17:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:17:22.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:22 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:17:22.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:22 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:17:22.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:22 smithi067 ceph-mon[33511]: osdmap e423: 8 total, 8 up, 8 in 2024-09-20T22:17:22.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:22 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1698018172' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:22.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:22 smithi067 ceph-mon[33511]: pgmap v774: 168 pgs: 168 active+clean; 5.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 818 B/s rd, 0 B/s wr, 1.13k op/s 2024-09-20T22:17:22.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:22 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:17:22.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:22 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:17:22.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:22 smithi060 ceph-mon[29694]: osdmap e423: 8 total, 8 up, 8 in 2024-09-20T22:17:22.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:22 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1698018172' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:22.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:22 smithi060 ceph-mon[29694]: pgmap v774: 168 pgs: 168 active+clean; 5.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 818 B/s rd, 0 B/s wr, 1.13k op/s 2024-09-20T22:17:22.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:22 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:17:22.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:22 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:17:22.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:22 smithi060 ceph-mon[40290]: osdmap e423: 8 total, 8 up, 8 in 2024-09-20T22:17:22.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:22 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1698018172' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:22.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:22 smithi060 ceph-mon[40290]: pgmap v774: 168 pgs: 168 active+clean; 5.4 GiB data, 21 GiB used, 694 GiB / 715 GiB avail; 818 B/s rd, 0 B/s wr, 1.13k op/s 2024-09-20T22:17:22.853 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:22.893 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:22.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:22.894 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:22.932 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:17:22.932 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:17:22.932 INFO:teuthology.orchestra.run.smithi067.stdout: "complaint_time": 30, 2024-09-20T22:17:22.932 INFO:teuthology.orchestra.run.smithi067.stdout: "num_blocked_ops": 0 2024-09-20T22:17:22.932 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:17:22.995 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:23.021 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:23.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:23.021 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:23.122 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:23.165 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:23.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:23.166 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:23.267 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:23.299 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:23.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:23.300 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:23.401 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:23.439 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:23.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:23.439 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:23.540 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:23.562 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:23.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:23.563 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:23.663 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:23.698 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:23.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:23.699 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:23.800 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:23.835 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:23.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:23.836 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:23.936 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:23.968 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:23.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:23.970 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:24.070 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:24.116 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:24.116 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:24.117 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:24.217 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:24.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:24.254 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:24.254 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:24.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:23 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:17:24.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:23 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:17:24.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:23 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:17:24.355 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:24.386 INFO:teuthology.orchestra.run.smithi060.stdout:moved 0 / 504 (0%) 2024-09-20T22:17:24.387 INFO:teuthology.orchestra.run.smithi060.stdout:avg 63 2024-09-20T22:17:24.387 INFO:teuthology.orchestra.run.smithi060.stdout:stddev 4.89898 -> 4.89898 (expected baseline 7.42462) 2024-09-20T22:17:24.387 INFO:teuthology.orchestra.run.smithi060.stdout:min osd.2 with 53 -> 53 pgs (0.84127 -> 0.84127 * mean) 2024-09-20T22:17:24.387 INFO:teuthology.orchestra.run.smithi060.stdout:max osd.4 with 70 -> 70 pgs (1.11111 -> 1.11111 * mean) 2024-09-20T22:17:24.387 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:17:24.387 INFO:teuthology.orchestra.run.smithi060.stdout:oload 1000 2024-09-20T22:17:24.387 INFO:teuthology.orchestra.run.smithi060.stdout:max_change 1 2024-09-20T22:17:24.387 INFO:teuthology.orchestra.run.smithi060.stdout:max_change_osds 4 2024-09-20T22:17:24.387 INFO:teuthology.orchestra.run.smithi060.stdout:average_utilization 0.0286 2024-09-20T22:17:24.387 INFO:teuthology.orchestra.run.smithi060.stdout:overload_utilization 0.2855 2024-09-20T22:17:24.387 INFO:teuthology.orchestra.run.smithi060.stderr:no change 2024-09-20T22:17:24.396 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:24.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:24.398 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:24.498 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:24.533 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:24.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:24.534 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:24.635 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:24.670 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:24.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:24.672 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:24.772 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:24.807 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:24.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:24.808 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:24.908 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:24.945 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:24.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:24.946 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:25.003 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:17:25.046 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:25.104 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:25.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:25.106 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:25.206 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:25.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:25.260 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:25.260 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:25.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:24 smithi067 ceph-mon[33511]: pgmap v775: 168 pgs: 168 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.29k op/s 2024-09-20T22:17:25.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:24 smithi067 ceph-mon[33511]: from='client.26093 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:17:25.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:24 smithi060 ceph-mon[29694]: pgmap v775: 168 pgs: 168 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.29k op/s 2024-09-20T22:17:25.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:24 smithi060 ceph-mon[29694]: from='client.26093 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:17:25.287 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:17:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:17:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:17:25.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:24 smithi060 ceph-mon[40290]: pgmap v775: 168 pgs: 168 active+clean; 5.2 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.29k op/s 2024-09-20T22:17:25.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:24 smithi060 ceph-mon[40290]: from='client.26093 -' entity='client.admin' cmd=[{"prefix": "osd test-reweight-by-utilization", "oload": 1000, "max_change": 1.0, "target": ["mon-mgr", ""]}]: dispatch 2024-09-20T22:17:25.361 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:25.398 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:25.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:25.399 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:25.499 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:25.547 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:25.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:25.548 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:25.650 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:25.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:25.683 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:25.683 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:25.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:17:25.783 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:25.824 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:25.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:25.825 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:25.926 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:25.996 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:25.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:25.998 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:26.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:25 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:17:26.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:25 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:17:26.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:26 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:17:26.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:26 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:17:26.098 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:26.132 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:26.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:26.133 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:26.233 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:26.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:25 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:17:26.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:25 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:17:26.266 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:26.267 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:26.267 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:26.364 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T22:17:26.368 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:26.406 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:26.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:26.421 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:26.422 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:26.522 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:26.580 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:26.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:26.582 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:26.683 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:26.728 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:26.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:26.729 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:26.753 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:17:26.829 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:26.866 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:26.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:26.868 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:26.968 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:27.015 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:27.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:27.018 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:27.100 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:27.119 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:27.150 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:27.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:27.151 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:27.202 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:27 smithi067 ceph-mon[33511]: osdmap e424: 8 total, 8 up, 8 in 2024-09-20T22:17:27.202 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:27 smithi067 ceph-mon[33511]: pgmap v777: 168 pgs: 168 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 922 op/s 2024-09-20T22:17:27.202 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:27 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:27.202 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:27 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:27.251 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:27 smithi060 ceph-mon[29694]: osdmap e424: 8 total, 8 up, 8 in 2024-09-20T22:17:27.251 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:27 smithi060 ceph-mon[29694]: pgmap v777: 168 pgs: 168 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 922 op/s 2024-09-20T22:17:27.251 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:27 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:27.251 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:27 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:27.252 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:27 smithi060 ceph-mon[40290]: osdmap e424: 8 total, 8 up, 8 in 2024-09-20T22:17:27.252 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:27 smithi060 ceph-mon[40290]: pgmap v777: 168 pgs: 168 active+clean; 5.1 GiB data, 20 GiB used, 695 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 922 op/s 2024-09-20T22:17:27.252 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:27 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:27.252 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:27 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:27.252 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:27.281 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:27.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:27.282 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:27.359 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:17:27.359 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:17:27.359 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:17:27.359 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:17:27.359 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.359 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105650 8.14 8:29da2845:::benchmark_data_smithi060_111051_object105649:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:17:27.359 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.545671+0000", 2024-09-20T22:17:27.359 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 498.80834517599999, 2024-09-20T22:17:27.359 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.038415869, 2024-09-20T22:17:27.359 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.360 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.360 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:17:27.360 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:27.360 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:27.360 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:27.360 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105650 2024-09-20T22:17:27.360 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.360 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.360 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.360 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.360 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545671+0000", 2024-09-20T22:17:27.360 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.360 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.360 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.361 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.361 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545671+0000", 2024-09-20T22:17:27.361 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.361 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.361 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.361 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545673+0000", 2024-09-20T22:17:27.361 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.17e-06 2024-09-20T22:17:27.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.361 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.361 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.361 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545723+0000", 2024-09-20T22:17:27.361 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9509000000000003e-05 2024-09-20T22:17:27.361 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.361 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.361 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.362 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545741+0000", 2024-09-20T22:17:27.362 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8634000000000002e-05 2024-09-20T22:17:27.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.362 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.362 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.545751+0000", 2024-09-20T22:17:27.362 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4420000000000004e-06 2024-09-20T22:17:27.362 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.362 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.362 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.363 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.707696+0000", 2024-09-20T22:17:27.363 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.16194560099999999 2024-09-20T22:17:27.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.363 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:27.363 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.707706+0000", 2024-09-20T22:17:27.363 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6860000000000001e-06 2024-09-20T22:17:27.363 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.363 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.363 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.363 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584005+0000", 2024-09-20T22:17:27.364 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.87629895400000002 2024-09-20T22:17:27.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.364 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.364 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584087+0000", 2024-09-20T22:17:27.364 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1872999999999994e-05 2024-09-20T22:17:27.364 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.364 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.364 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.364 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.364 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.364 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105665 8.18 8:1b9c2a37:::benchmark_data_smithi060_111051_object105664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:17:27.365 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.556635+0000", 2024-09-20T22:17:27.365 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 498.79738114700001, 2024-09-20T22:17:27.365 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.051209888, 2024-09-20T22:17:27.365 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.365 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.365 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:27.365 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:27.365 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:27.365 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:27.365 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105665 2024-09-20T22:17:27.365 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.365 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.366 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.366 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.366 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556635+0000", 2024-09-20T22:17:27.366 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.366 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.366 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.366 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556635+0000", 2024-09-20T22:17:27.366 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.366 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.366 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.366 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.366 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556638+0000", 2024-09-20T22:17:27.367 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.6400000000000001e-06 2024-09-20T22:17:27.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.367 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.367 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.367 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556714+0000", 2024-09-20T22:17:27.367 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6594999999999994e-05 2024-09-20T22:17:27.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.367 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.367 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.367 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556736+0000", 2024-09-20T22:17:27.367 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1662e-05 2024-09-20T22:17:27.367 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.368 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.368 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.368 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.556751+0000", 2024-09-20T22:17:27.368 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4941e-05 2024-09-20T22:17:27.368 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.369 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.369 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.369 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.750414+0000", 2024-09-20T22:17:27.369 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.193663171 2024-09-20T22:17:27.369 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.369 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.369 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:27.370 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.750424+0000", 2024-09-20T22:17:27.370 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0397999999999999e-05 2024-09-20T22:17:27.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.370 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.370 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.370 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.596184+0000", 2024-09-20T22:17:27.370 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.84575938799999995 2024-09-20T22:17:27.370 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.370 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.370 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:27.370 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.596303+0000", 2024-09-20T22:17:27.371 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000119503 2024-09-20T22:17:27.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.371 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.371 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:17:27.371 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.596588+0000", 2024-09-20T22:17:27.371 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00028446100000000001 2024-09-20T22:17:27.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.371 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.371 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.371 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.603603+0000", 2024-09-20T22:17:27.371 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0070155649999999996 2024-09-20T22:17:27.371 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.371 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.371 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:27.372 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607589+0000", 2024-09-20T22:17:27.372 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0039854110000000003 2024-09-20T22:17:27.372 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.372 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.372 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.372 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607743+0000", 2024-09-20T22:17:27.372 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015391499999999999 2024-09-20T22:17:27.372 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.372 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.372 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:27.373 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607802+0000", 2024-09-20T22:17:27.373 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9472000000000003e-05 2024-09-20T22:17:27.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.373 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.373 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.373 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.607845+0000", 2024-09-20T22:17:27.373 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2765999999999997e-05 2024-09-20T22:17:27.373 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.373 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.373 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.373 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.373 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.374 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105673 8.14 8:2f269e39:::benchmark_data_smithi060_111051_object105672:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:17:27.374 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.761620+0000", 2024-09-20T22:17:27.374 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 498.59239659399998, 2024-09-20T22:17:27.374 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.83821859799999998, 2024-09-20T22:17:27.374 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.374 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.374 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:27.374 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:27.374 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:27.374 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:27.374 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105673 2024-09-20T22:17:27.374 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.375 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.375 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.375 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.375 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761620+0000", 2024-09-20T22:17:27.375 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.375 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.375 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.376 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.376 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761620+0000", 2024-09-20T22:17:27.376 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.376 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.376 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.376 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761621+0000", 2024-09-20T22:17:27.376 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.392e-06 2024-09-20T22:17:27.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.376 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.376 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.377 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761674+0000", 2024-09-20T22:17:27.377 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2677999999999999e-05 2024-09-20T22:17:27.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.377 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.377 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.377 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761686+0000", 2024-09-20T22:17:27.377 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1926000000000001e-05 2024-09-20T22:17:27.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.377 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.377 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.377 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761697+0000", 2024-09-20T22:17:27.377 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1364e-05 2024-09-20T22:17:27.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.377 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.377 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.378 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761770+0000", 2024-09-20T22:17:27.378 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2904999999999994e-05 2024-09-20T22:17:27.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.378 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:27.378 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.761777+0000", 2024-09-20T22:17:27.378 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8029999999999999e-06 2024-09-20T22:17:27.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.378 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.378 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584113+0000", 2024-09-20T22:17:27.379 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.82233672499999999 2024-09-20T22:17:27.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.379 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.379 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:27.379 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584256+0000", 2024-09-20T22:17:27.379 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000142686 2024-09-20T22:17:27.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.379 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.379 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,4", 2024-09-20T22:17:27.379 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.584528+0000", 2024-09-20T22:17:27.379 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00027199499999999997 2024-09-20T22:17:27.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.379 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.379 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.380 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.590292+0000", 2024-09-20T22:17:27.380 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.005764333 2024-09-20T22:17:27.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.380 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:27.380 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.590315+0000", 2024-09-20T22:17:27.380 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2768e-05 2024-09-20T22:17:27.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.380 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.380 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.599740+0000", 2024-09-20T22:17:27.380 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0094248200000000004 2024-09-20T22:17:27.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.380 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:27.381 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.599795+0000", 2024-09-20T22:17:27.381 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5108000000000002e-05 2024-09-20T22:17:27.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.381 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.381 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.381 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.599838+0000", 2024-09-20T22:17:27.381 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3095e-05 2024-09-20T22:17:27.381 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.381 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.381 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.381 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.381 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:106978 8.19 8:9e674fff:::benchmark_data_smithi060_111051_object106977:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:17:27.381 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.576265+0000", 2024-09-20T22:17:27.381 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 496.77775148000001, 2024-09-20T22:17:27.381 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9488434429999999, 2024-09-20T22:17:27.382 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.382 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.382 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:27.382 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:27.382 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:27.382 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:27.382 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 106978 2024-09-20T22:17:27.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.382 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.382 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.383 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.383 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576265+0000", 2024-09-20T22:17:27.383 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.383 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.383 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.383 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576265+0000", 2024-09-20T22:17:27.383 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.383 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.383 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.383 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576267+0000", 2024-09-20T22:17:27.383 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0509999999999999e-06 2024-09-20T22:17:27.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.384 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.384 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.384 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576299+0000", 2024-09-20T22:17:27.384 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1962000000000003e-05 2024-09-20T22:17:27.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.384 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.384 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.384 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576312+0000", 2024-09-20T22:17:27.384 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3423e-05 2024-09-20T22:17:27.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.384 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.384 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.384 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576322+0000", 2024-09-20T22:17:27.384 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0118e-05 2024-09-20T22:17:27.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.385 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.385 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.385 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576468+0000", 2024-09-20T22:17:27.385 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000145575 2024-09-20T22:17:27.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.385 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.385 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:27.385 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.576481+0000", 2024-09-20T22:17:27.385 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3023999999999999e-05 2024-09-20T22:17:27.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.385 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.385 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.385 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410333+0000", 2024-09-20T22:17:27.386 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.83385187800000005 2024-09-20T22:17:27.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.386 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.386 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:27.386 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410345+0000", 2024-09-20T22:17:27.386 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.258e-05 2024-09-20T22:17:27.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.386 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.386 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.386 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.503436+0000", 2024-09-20T22:17:27.387 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.093091088 2024-09-20T22:17:27.387 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.387 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.387 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:27.387 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.503599+0000", 2024-09-20T22:17:27.387 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00016286999999999999 2024-09-20T22:17:27.387 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.387 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.387 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4,6", 2024-09-20T22:17:27.387 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.505264+0000", 2024-09-20T22:17:27.387 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001664591 2024-09-20T22:17:27.387 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.387 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.387 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.387 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.508504+0000", 2024-09-20T22:17:27.387 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0032404920000000002 2024-09-20T22:17:27.388 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.388 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.388 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.388 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.508547+0000", 2024-09-20T22:17:27.388 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2734000000000001e-05 2024-09-20T22:17:27.388 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.388 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.388 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.388 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.513180+0000", 2024-09-20T22:17:27.388 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046330490000000002 2024-09-20T22:17:27.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.389 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.389 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:27.389 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.524891+0000", 2024-09-20T22:17:27.389 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.011710379999999999 2024-09-20T22:17:27.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.389 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.389 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:27.389 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.524966+0000", 2024-09-20T22:17:27.389 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5290999999999995e-05 2024-09-20T22:17:27.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.389 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.389 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.389 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.525108+0000", 2024-09-20T22:17:27.390 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000142337 2024-09-20T22:17:27.390 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.390 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.390 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.390 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.390 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.390 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:107002 8.11 8.e0f52171 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:17:27.390 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.617682+0000", 2024-09-20T22:17:27.390 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 496.73633398200002, 2024-09-20T22:17:27.390 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79265608300000001, 2024-09-20T22:17:27.390 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.390 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.391 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:17:27.391 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:27.391 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:27.391 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:27.391 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 107002 2024-09-20T22:17:27.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.391 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.391 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.391 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.391 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617682+0000", 2024-09-20T22:17:27.391 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.392 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.392 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.392 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617682+0000", 2024-09-20T22:17:27.392 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.392 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.392 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.392 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617684+0000", 2024-09-20T22:17:27.392 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1299999999999999e-06 2024-09-20T22:17:27.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.393 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.393 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.393 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617725+0000", 2024-09-20T22:17:27.393 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0839e-05 2024-09-20T22:17:27.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.393 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.394 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.394 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617738+0000", 2024-09-20T22:17:27.394 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2310000000000001e-05 2024-09-20T22:17:27.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.394 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.394 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.394 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617746+0000", 2024-09-20T22:17:27.394 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3939999999999996e-06 2024-09-20T22:17:27.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.395 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.395 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.395 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617856+0000", 2024-09-20T22:17:27.395 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000109917 2024-09-20T22:17:27.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.395 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.395 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:27.395 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.617878+0000", 2024-09-20T22:17:27.395 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2495999999999999e-05 2024-09-20T22:17:27.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.395 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.395 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.395 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410330+0000", 2024-09-20T22:17:27.396 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.79245202299999995 2024-09-20T22:17:27.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.396 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.396 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.396 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.410338+0000", 2024-09-20T22:17:27.396 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9740000000000002e-06 2024-09-20T22:17:27.396 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.396 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.396 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.397 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.397 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163158 8.11 8:8e0bce89:::benchmark_data_smithi060_111051_object43159:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:17:27.397 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:08.839869+0000", 2024-09-20T22:17:27.397 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 438.514147541, 2024-09-20T22:17:27.397 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.65193807500000001, 2024-09-20T22:17:27.397 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.397 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.397 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:27.397 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:27.397 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:27.397 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:27.397 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163158 2024-09-20T22:17:27.397 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.398 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.398 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.398 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.398 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839869+0000", 2024-09-20T22:17:27.398 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.398 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.398 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.398 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.398 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839869+0000", 2024-09-20T22:17:27.398 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.398 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.398 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.398 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.399 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839870+0000", 2024-09-20T22:17:27.399 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5710000000000001e-06 2024-09-20T22:17:27.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.399 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.399 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.399 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839876+0000", 2024-09-20T22:17:27.399 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3820000000000003e-06 2024-09-20T22:17:27.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.399 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.399 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.399 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839890+0000", 2024-09-20T22:17:27.399 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3952e-05 2024-09-20T22:17:27.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.399 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.399 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.400 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839897+0000", 2024-09-20T22:17:27.400 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4150000000000002e-06 2024-09-20T22:17:27.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.400 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.400 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.400 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839956+0000", 2024-09-20T22:17:27.400 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9157999999999998e-05 2024-09-20T22:17:27.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.400 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.400 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:27.400 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839967+0000", 2024-09-20T22:17:27.400 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1019000000000001e-05 2024-09-20T22:17:27.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.400 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.400 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.400 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477635+0000", 2024-09-20T22:17:27.401 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.63766765700000005 2024-09-20T22:17:27.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.401 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.401 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:27.401 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477737+0000", 2024-09-20T22:17:27.401 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010265899999999999 2024-09-20T22:17:27.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.401 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.401 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:17:27.401 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477939+0000", 2024-09-20T22:17:27.401 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020106900000000001 2024-09-20T22:17:27.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.402 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.402 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:27.402 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.482557+0000", 2024-09-20T22:17:27.402 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046186830000000002 2024-09-20T22:17:27.402 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.402 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.402 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.402 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491495+0000", 2024-09-20T22:17:27.402 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0089377699999999994 2024-09-20T22:17:27.402 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.402 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.402 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.402 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491673+0000", 2024-09-20T22:17:27.402 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017837500000000001 2024-09-20T22:17:27.402 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.403 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.403 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:27.403 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491762+0000", 2024-09-20T22:17:27.403 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8555999999999997e-05 2024-09-20T22:17:27.403 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.403 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.403 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.403 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491807+0000", 2024-09-20T22:17:27.403 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4808999999999997e-05 2024-09-20T22:17:27.403 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.403 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.403 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.404 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.404 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.404 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163289 8.11 8:8c47e2c8:::benchmark_data_smithi060_111051_object43290:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:17:27.404 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:08.954133+0000", 2024-09-20T22:17:27.404 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 438.39988290700001, 2024-09-20T22:17:27.404 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.53808307099999997, 2024-09-20T22:17:27.404 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.404 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.404 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:27.404 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:27.404 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:27.404 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:27.405 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163289 2024-09-20T22:17:27.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.405 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.405 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.405 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.405 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954133+0000", 2024-09-20T22:17:27.405 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.405 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.405 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.405 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954133+0000", 2024-09-20T22:17:27.405 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.405 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.405 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.405 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954134+0000", 2024-09-20T22:17:27.406 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2900000000000002e-07 2024-09-20T22:17:27.406 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.406 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.406 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.406 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954138+0000", 2024-09-20T22:17:27.406 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5260000000000002e-06 2024-09-20T22:17:27.406 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.406 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.406 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.406 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954147+0000", 2024-09-20T22:17:27.406 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2010000000000005e-06 2024-09-20T22:17:27.406 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.406 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.407 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.407 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954153+0000", 2024-09-20T22:17:27.407 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0610000000000001e-06 2024-09-20T22:17:27.407 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.407 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.407 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.407 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954209+0000", 2024-09-20T22:17:27.407 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5767999999999999e-05 2024-09-20T22:17:27.407 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.407 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.407 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:27.407 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954216+0000", 2024-09-20T22:17:27.407 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8099999999999998e-06 2024-09-20T22:17:27.407 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.407 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.408 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.408 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.478998+0000", 2024-09-20T22:17:27.408 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.52478124900000001 2024-09-20T22:17:27.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.408 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:27.408 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.479095+0000", 2024-09-20T22:17:27.408 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7125000000000003e-05 2024-09-20T22:17:27.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.408 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:17:27.408 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.479291+0000", 2024-09-20T22:17:27.408 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019642100000000001 2024-09-20T22:17:27.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.409 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:27.409 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.482759+0000", 2024-09-20T22:17:27.409 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0034672689999999998 2024-09-20T22:17:27.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.409 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.409 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491864+0000", 2024-09-20T22:17:27.409 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0091058630000000005 2024-09-20T22:17:27.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.409 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.409 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491961+0000", 2024-09-20T22:17:27.409 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6717000000000003e-05 2024-09-20T22:17:27.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.409 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:27.410 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492014+0000", 2024-09-20T22:17:27.410 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2877000000000002e-05 2024-09-20T22:17:27.410 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.410 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.410 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.410 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492216+0000", 2024-09-20T22:17:27.410 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020235500000000001 2024-09-20T22:17:27.410 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.410 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.410 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.410 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.410 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.410 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163324 8.11 8:8e312e9e:::benchmark_data_smithi060_111051_object43325:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:17:27.410 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:08.983848+0000", 2024-09-20T22:17:27.410 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 438.37016798500002, 2024-09-20T22:17:27.411 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.50861305999999995, 2024-09-20T22:17:27.411 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.411 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.411 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:27.411 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:27.411 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:27.411 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:27.411 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163324 2024-09-20T22:17:27.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.411 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.411 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.411 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.411 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983848+0000", 2024-09-20T22:17:27.411 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.412 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.412 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.412 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983848+0000", 2024-09-20T22:17:27.412 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.412 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.412 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.412 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983849+0000", 2024-09-20T22:17:27.412 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.11e-06 2024-09-20T22:17:27.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.412 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.412 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.412 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983853+0000", 2024-09-20T22:17:27.412 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0439999999999997e-06 2024-09-20T22:17:27.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.413 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.413 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.413 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983863+0000", 2024-09-20T22:17:27.413 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6430000000000007e-06 2024-09-20T22:17:27.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.413 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.413 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.413 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983868+0000", 2024-09-20T22:17:27.413 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1000000000000003e-06 2024-09-20T22:17:27.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.413 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.413 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.413 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983919+0000", 2024-09-20T22:17:27.413 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0433999999999999e-05 2024-09-20T22:17:27.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.413 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.414 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:27.414 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983928+0000", 2024-09-20T22:17:27.414 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3659999999999998e-06 2024-09-20T22:17:27.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.414 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.414 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.414 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.480646+0000", 2024-09-20T22:17:27.414 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.49671833999999998 2024-09-20T22:17:27.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.414 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.414 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:27.414 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.480768+0000", 2024-09-20T22:17:27.414 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012145 2024-09-20T22:17:27.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.415 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.415 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:17:27.415 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.481025+0000", 2024-09-20T22:17:27.415 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025713299999999999 2024-09-20T22:17:27.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.415 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.415 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:27.415 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.488496+0000", 2024-09-20T22:17:27.415 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0074709540000000001 2024-09-20T22:17:27.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.415 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.415 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.415 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492276+0000", 2024-09-20T22:17:27.415 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0037803670000000002 2024-09-20T22:17:27.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.416 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.416 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.416 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492350+0000", 2024-09-20T22:17:27.416 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3436999999999995e-05 2024-09-20T22:17:27.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.416 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.416 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:27.416 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492421+0000", 2024-09-20T22:17:27.416 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1303000000000005e-05 2024-09-20T22:17:27.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.416 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.416 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.416 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492461+0000", 2024-09-20T22:17:27.416 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0379000000000001e-05 2024-09-20T22:17:27.416 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.417 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.417 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.417 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.417 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.417 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163358 8.11 8:8b7ace29:::benchmark_data_smithi060_111051_object43359:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:17:27.417 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.014358+0000", 2024-09-20T22:17:27.417 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 438.33965799999999, 2024-09-20T22:17:27.417 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.47828052599999998, 2024-09-20T22:17:27.417 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.417 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.417 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:27.417 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:27.417 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:27.417 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:27.417 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163358 2024-09-20T22:17:27.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.418 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.418 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.418 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.418 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014358+0000", 2024-09-20T22:17:27.418 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.418 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.418 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.418 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014358+0000", 2024-09-20T22:17:27.418 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:27.420 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.420 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.420 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014359+0000", 2024-09-20T22:17:27.420 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0669999999999999e-06 2024-09-20T22:17:27.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.420 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.420 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.420 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014364+0000", 2024-09-20T22:17:27.420 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0220000000000004e-06 2024-09-20T22:17:27.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.421 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.421 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014374+0000", 2024-09-20T22:17:27.421 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9599999999999995e-06 2024-09-20T22:17:27.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.421 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.421 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014388+0000", 2024-09-20T22:17:27.421 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3413e-05 2024-09-20T22:17:27.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.421 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.422 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014447+0000", 2024-09-20T22:17:27.422 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9698999999999999e-05 2024-09-20T22:17:27.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.422 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:27.422 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014455+0000", 2024-09-20T22:17:27.422 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.072e-06 2024-09-20T22:17:27.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.422 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.422 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.482921+0000", 2024-09-20T22:17:27.422 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.46846513200000001 2024-09-20T22:17:27.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.423 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.423 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:27.423 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483020+0000", 2024-09-20T22:17:27.423 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9319000000000002e-05 2024-09-20T22:17:27.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.423 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.423 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:17:27.423 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483251+0000", 2024-09-20T22:17:27.423 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00023124300000000001 2024-09-20T22:17:27.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.423 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.423 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:27.424 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491608+0000", 2024-09-20T22:17:27.424 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0083564190000000003 2024-09-20T22:17:27.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.424 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.424 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.424 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492507+0000", 2024-09-20T22:17:27.424 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00089972900000000005 2024-09-20T22:17:27.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.424 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.424 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.424 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492546+0000", 2024-09-20T22:17:27.424 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9156999999999998e-05 2024-09-20T22:17:27.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.424 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.424 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:27.425 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492602+0000", 2024-09-20T22:17:27.425 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5114999999999997e-05 2024-09-20T22:17:27.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.425 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.425 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492639+0000", 2024-09-20T22:17:27.425 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7178999999999998e-05 2024-09-20T22:17:27.425 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.425 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.425 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.425 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163372 8.11 8:8b6ec81f:::benchmark_data_smithi060_111051_object43373:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:17:27.425 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.027195+0000", 2024-09-20T22:17:27.426 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 438.32682119200001, 2024-09-20T22:17:27.426 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.46568983899999999, 2024-09-20T22:17:27.426 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.426 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.426 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:27.426 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:27.426 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:27.426 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:27.426 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163372 2024-09-20T22:17:27.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.426 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.426 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.427 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.427 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027195+0000", 2024-09-20T22:17:27.427 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.427 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.427 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.427 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027195+0000", 2024-09-20T22:17:27.427 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.428 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.428 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.428 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027196+0000", 2024-09-20T22:17:27.428 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0599999999999999e-07 2024-09-20T22:17:27.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.428 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.428 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.428 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027200+0000", 2024-09-20T22:17:27.428 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9280000000000003e-06 2024-09-20T22:17:27.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.430 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.430 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.430 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027210+0000", 2024-09-20T22:17:27.430 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0035e-05 2024-09-20T22:17:27.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.431 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.431 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027217+0000", 2024-09-20T22:17:27.431 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6450000000000005e-06 2024-09-20T22:17:27.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.431 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.431 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027281+0000", 2024-09-20T22:17:27.431 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3281000000000006e-05 2024-09-20T22:17:27.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.431 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:27.431 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027291+0000", 2024-09-20T22:17:27.432 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0268999999999999e-05 2024-09-20T22:17:27.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.432 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.432 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.484647+0000", 2024-09-20T22:17:27.432 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.45735564499999998 2024-09-20T22:17:27.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.432 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:27.432 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.484745+0000", 2024-09-20T22:17:27.432 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8712e-05 2024-09-20T22:17:27.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.433 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:17:27.433 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.484940+0000", 2024-09-20T22:17:27.433 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019504900000000001 2024-09-20T22:17:27.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.433 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.433 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:27.433 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491619+0000", 2024-09-20T22:17:27.433 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0066782090000000001 2024-09-20T22:17:27.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.433 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.433 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.433 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492716+0000", 2024-09-20T22:17:27.433 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001097701 2024-09-20T22:17:27.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.434 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.434 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492778+0000", 2024-09-20T22:17:27.434 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1538000000000006e-05 2024-09-20T22:17:27.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.434 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:27.434 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492830+0000", 2024-09-20T22:17:27.434 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1997999999999999e-05 2024-09-20T22:17:27.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.434 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.434 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492885+0000", 2024-09-20T22:17:27.434 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5022999999999998e-05 2024-09-20T22:17:27.435 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.435 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.435 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.435 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.435 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163373 8.11 8:8d07a686:::benchmark_data_smithi060_111051_object43374:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:17:27.435 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.027847+0000", 2024-09-20T22:17:27.435 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 438.326168784, 2024-09-20T22:17:27.435 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.465200326, 2024-09-20T22:17:27.435 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.435 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.435 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:27.435 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:27.435 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:27.436 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:27.436 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163373 2024-09-20T22:17:27.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.436 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.436 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.436 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027847+0000", 2024-09-20T22:17:27.436 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.436 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.436 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027847+0000", 2024-09-20T22:17:27.436 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.437 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.437 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.437 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027848+0000", 2024-09-20T22:17:27.437 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3600000000000003e-07 2024-09-20T22:17:27.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.437 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.437 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.437 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027852+0000", 2024-09-20T22:17:27.437 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6330000000000002e-06 2024-09-20T22:17:27.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.437 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.437 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.437 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027860+0000", 2024-09-20T22:17:27.437 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2789999999999998e-06 2024-09-20T22:17:27.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.438 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.438 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.438 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027866+0000", 2024-09-20T22:17:27.438 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1070000000000001e-06 2024-09-20T22:17:27.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.438 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.438 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.438 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027897+0000", 2024-09-20T22:17:27.438 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0722000000000002e-05 2024-09-20T22:17:27.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.438 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.438 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:27.438 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027901+0000", 2024-09-20T22:17:27.439 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1649999999999999e-06 2024-09-20T22:17:27.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.439 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.439 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.486658+0000", 2024-09-20T22:17:27.439 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.45875733499999999 2024-09-20T22:17:27.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.439 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:27.439 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.486767+0000", 2024-09-20T22:17:27.439 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000108385 2024-09-20T22:17:27.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.439 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:17:27.439 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487120+0000", 2024-09-20T22:17:27.440 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00035312499999999998 2024-09-20T22:17:27.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.440 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:27.440 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491877+0000", 2024-09-20T22:17:27.440 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0047569170000000003 2024-09-20T22:17:27.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.440 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.440 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492928+0000", 2024-09-20T22:17:27.440 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001051633 2024-09-20T22:17:27.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.440 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.441 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492967+0000", 2024-09-20T22:17:27.441 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.8781e-05 2024-09-20T22:17:27.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.441 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.441 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:27.441 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493013+0000", 2024-09-20T22:17:27.441 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5555e-05 2024-09-20T22:17:27.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.441 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.441 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.441 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493048+0000", 2024-09-20T22:17:27.441 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5052999999999998e-05 2024-09-20T22:17:27.441 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.441 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.442 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.442 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163388 8.11 8:893f1034:::benchmark_data_smithi060_111051_object43389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:17:27.442 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.050114+0000", 2024-09-20T22:17:27.442 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 438.30390207400001, 2024-09-20T22:17:27.442 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.44308652799999998, 2024-09-20T22:17:27.442 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.442 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.442 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:27.442 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:27.442 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:27.442 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:27.442 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163388 2024-09-20T22:17:27.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.443 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.443 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.443 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050114+0000", 2024-09-20T22:17:27.443 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.443 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.443 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050114+0000", 2024-09-20T22:17:27.443 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.443 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.443 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050115+0000", 2024-09-20T22:17:27.443 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2500000000000005e-07 2024-09-20T22:17:27.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.444 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.444 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050119+0000", 2024-09-20T22:17:27.444 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.698e-06 2024-09-20T22:17:27.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.444 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.444 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050128+0000", 2024-09-20T22:17:27.444 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5820000000000008e-06 2024-09-20T22:17:27.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.444 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.444 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050133+0000", 2024-09-20T22:17:27.444 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9060000000000001e-06 2024-09-20T22:17:27.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.445 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.445 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050169+0000", 2024-09-20T22:17:27.445 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5984e-05 2024-09-20T22:17:27.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.445 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:27.445 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050174+0000", 2024-09-20T22:17:27.445 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0180000000000001e-06 2024-09-20T22:17:27.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.445 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.445 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.488568+0000", 2024-09-20T22:17:27.445 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.43839391 2024-09-20T22:17:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.488685+0000", 2024-09-20T22:17:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000117163 2024-09-20T22:17:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:17:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.489162+0000", 2024-09-20T22:17:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00047712600000000003 2024-09-20T22:17:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491884+0000", 2024-09-20T22:17:27.446 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0027215329999999999 2024-09-20T22:17:27.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.447 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.447 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.447 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493086+0000", 2024-09-20T22:17:27.447 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001201765 2024-09-20T22:17:27.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.447 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.447 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.447 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493122+0000", 2024-09-20T22:17:27.447 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6622999999999998e-05 2024-09-20T22:17:27.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.447 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.447 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:27.447 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493168+0000", 2024-09-20T22:17:27.447 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5757999999999998e-05 2024-09-20T22:17:27.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.447 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.448 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.448 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493201+0000", 2024-09-20T22:17:27.448 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2737000000000002e-05 2024-09-20T22:17:27.448 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.448 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.448 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.448 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163393 8.11 8:8bb06d42:::benchmark_data_smithi060_111051_object43394:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:17:27.448 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.064776+0000", 2024-09-20T22:17:27.448 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 438.289240151, 2024-09-20T22:17:27.448 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.43944195600000002, 2024-09-20T22:17:27.448 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.448 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.448 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:27.448 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:27.449 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:27.449 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:27.449 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163393 2024-09-20T22:17:27.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.449 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.449 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.449 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064776+0000", 2024-09-20T22:17:27.449 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.449 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.449 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064776+0000", 2024-09-20T22:17:27.449 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.450 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064777+0000", 2024-09-20T22:17:27.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9500000000000001e-07 2024-09-20T22:17:27.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.450 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064781+0000", 2024-09-20T22:17:27.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9870000000000001e-06 2024-09-20T22:17:27.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.450 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064791+0000", 2024-09-20T22:17:27.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0417999999999999e-05 2024-09-20T22:17:27.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.451 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064797+0000", 2024-09-20T22:17:27.451 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4040000000000003e-06 2024-09-20T22:17:27.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.451 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.451 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064838+0000", 2024-09-20T22:17:27.451 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1591000000000002e-05 2024-09-20T22:17:27.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.451 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:27.451 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064843+0000", 2024-09-20T22:17:27.451 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9470000000000002e-06 2024-09-20T22:17:27.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.452 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.452 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493235+0000", 2024-09-20T22:17:27.452 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.42839186899999998 2024-09-20T22:17:27.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.452 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:27.452 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493336+0000", 2024-09-20T22:17:27.452 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100923 2024-09-20T22:17:27.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.452 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:17:27.452 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493659+0000", 2024-09-20T22:17:27.453 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00032341599999999998 2024-09-20T22:17:27.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.453 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.453 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:27.453 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.499928+0000", 2024-09-20T22:17:27.453 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0062687730000000001 2024-09-20T22:17:27.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.453 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.453 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.453 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.503797+0000", 2024-09-20T22:17:27.453 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0038686020000000001 2024-09-20T22:17:27.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.453 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.453 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.453 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.504160+0000", 2024-09-20T22:17:27.454 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00036300900000000002 2024-09-20T22:17:27.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.454 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.454 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:27.454 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.504192+0000", 2024-09-20T22:17:27.454 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2592000000000002e-05 2024-09-20T22:17:27.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.454 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.454 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.454 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.504218+0000", 2024-09-20T22:17:27.454 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.563e-05 2024-09-20T22:17:27.454 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.454 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.454 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.455 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:125848 9.6 9:604ae09f:::benchmark_data_smithi060_140132_object15040:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:17:27.455 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:13:30.323134+0000", 2024-09-20T22:17:27.455 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 237.030881881, 2024-09-20T22:17:27.455 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.39324260900000002, 2024-09-20T22:17:27.455 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.455 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.455 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:27.455 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:27.455 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:17:27.455 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:17:27.455 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 125848 2024-09-20T22:17:27.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.456 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.456 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.456 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323134+0000", 2024-09-20T22:17:27.456 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.456 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.456 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323134+0000", 2024-09-20T22:17:27.456 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.456 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.456 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323135+0000", 2024-09-20T22:17:27.456 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1399999999999997e-07 2024-09-20T22:17:27.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.457 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323137+0000", 2024-09-20T22:17:27.457 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1789999999999998e-06 2024-09-20T22:17:27.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.457 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323143+0000", 2024-09-20T22:17:27.457 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1469999999999998e-06 2024-09-20T22:17:27.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.457 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323149+0000", 2024-09-20T22:17:27.457 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2639999999999997e-06 2024-09-20T22:17:27.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.458 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.458 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323210+0000", 2024-09-20T22:17:27.458 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0537e-05 2024-09-20T22:17:27.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.458 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:27.458 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323333+0000", 2024-09-20T22:17:27.458 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012330500000000001 2024-09-20T22:17:27.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.458 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:17:27.459 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323444+0000", 2024-09-20T22:17:27.459 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000110562 2024-09-20T22:17:27.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.459 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:27.459 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.326284+0000", 2024-09-20T22:17:27.459 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028407409999999999 2024-09-20T22:17:27.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.459 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.459 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.328793+0000", 2024-09-20T22:17:27.459 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0025082680000000001 2024-09-20T22:17:27.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.459 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.459 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.716294+0000", 2024-09-20T22:17:27.460 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.38750077100000002 2024-09-20T22:17:27.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.460 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:27.460 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.716331+0000", 2024-09-20T22:17:27.460 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7419999999999997e-05 2024-09-20T22:17:27.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.460 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.460 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.716377+0000", 2024-09-20T22:17:27.460 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6001000000000002e-05 2024-09-20T22:17:27.460 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.460 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.460 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.461 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:125853 9.6 9:65bf9cb5:::benchmark_data_smithi060_140132_object15045:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:17:27.461 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:13:30.329387+0000", 2024-09-20T22:17:27.461 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 237.024629218, 2024-09-20T22:17:27.461 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.39097777700000003, 2024-09-20T22:17:27.461 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.461 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.461 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:27.461 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:27.461 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:17:27.461 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:17:27.461 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 125853 2024-09-20T22:17:27.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.462 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.462 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.462 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329387+0000", 2024-09-20T22:17:27.462 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.462 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.463 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329387+0000", 2024-09-20T22:17:27.463 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.464 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.464 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329388+0000", 2024-09-20T22:17:27.464 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3990000000000001e-06 2024-09-20T22:17:27.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.464 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.464 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329395+0000", 2024-09-20T22:17:27.464 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.161e-06 2024-09-20T22:17:27.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.464 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.464 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329407+0000", 2024-09-20T22:17:27.464 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2323999999999999e-05 2024-09-20T22:17:27.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.465 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329414+0000", 2024-09-20T22:17:27.465 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.289e-06 2024-09-20T22:17:27.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.465 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329463+0000", 2024-09-20T22:17:27.465 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8658000000000001e-05 2024-09-20T22:17:27.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:27.465 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329589+0000", 2024-09-20T22:17:27.465 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012579900000000001 2024-09-20T22:17:27.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.466 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:17:27.466 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329695+0000", 2024-09-20T22:17:27.466 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000105933 2024-09-20T22:17:27.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.466 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.466 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.335795+0000", 2024-09-20T22:17:27.466 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061008720000000002 2024-09-20T22:17:27.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.466 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:27.466 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.335858+0000", 2024-09-20T22:17:27.466 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2285000000000004e-05 2024-09-20T22:17:27.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.467 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.467 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720262+0000", 2024-09-20T22:17:27.467 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.38440416300000002 2024-09-20T22:17:27.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.467 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.467 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:27.467 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720315+0000", 2024-09-20T22:17:27.467 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2985000000000002e-05 2024-09-20T22:17:27.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.467 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.467 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.467 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720365+0000", 2024-09-20T22:17:27.467 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9908999999999999e-05 2024-09-20T22:17:27.467 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.467 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.468 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.468 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.468 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:125877 9.6 9:618bafd1:::benchmark_data_smithi060_140132_object15069:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:17:27.468 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:17:27.468 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 237.008927642, 2024-09-20T22:17:27.468 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.37546249399999998, 2024-09-20T22:17:27.468 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.468 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.468 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:27.468 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:27.468 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:17:27.468 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:17:27.468 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 125877 2024-09-20T22:17:27.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.469 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.469 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.469 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:17:27.469 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.469 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.469 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:17:27.469 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.469 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.469 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:17:27.469 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1999999999999999e-07 2024-09-20T22:17:27.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.470 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.470 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.470 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345092+0000", 2024-09-20T22:17:27.470 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2899999999999998e-06 2024-09-20T22:17:27.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.470 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.470 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.470 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345101+0000", 2024-09-20T22:17:27.470 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1550000000000007e-06 2024-09-20T22:17:27.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.470 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.470 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.470 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345107+0000", 2024-09-20T22:17:27.470 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2609999999999999e-06 2024-09-20T22:17:27.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.471 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.471 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.471 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345160+0000", 2024-09-20T22:17:27.471 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.287e-05 2024-09-20T22:17:27.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.471 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.471 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:27.471 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345269+0000", 2024-09-20T22:17:27.471 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000108795 2024-09-20T22:17:27.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.471 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.471 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:17:27.471 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345395+0000", 2024-09-20T22:17:27.471 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012654800000000001 2024-09-20T22:17:27.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.472 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:27.472 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.348668+0000", 2024-09-20T22:17:27.472 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0032731729999999999 2024-09-20T22:17:27.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.472 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.472 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.351790+0000", 2024-09-20T22:17:27.472 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003121567 2024-09-20T22:17:27.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.472 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.473 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720464+0000", 2024-09-20T22:17:27.473 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.36867422 2024-09-20T22:17:27.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.473 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:27.473 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720511+0000", 2024-09-20T22:17:27.473 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7076999999999997e-05 2024-09-20T22:17:27.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.473 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.473 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720551+0000", 2024-09-20T22:17:27.473 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9918e-05 2024-09-20T22:17:27.473 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.473 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.473 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.474 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.474 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:77013 10.9 10:9a8ed282:::benchmark_data_smithi060_167766_object77012:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e395)", 2024-09-20T22:17:27.474 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:16:04.243329+0000", 2024-09-20T22:17:27.474 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 83.110686814000005, 2024-09-20T22:17:27.474 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.24326910600000001, 2024-09-20T22:17:27.474 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.474 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.474 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:27.474 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:27.474 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:27.474 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:27.474 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 77013 2024-09-20T22:17:27.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.475 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.475 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.475 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.243329+0000", 2024-09-20T22:17:27.475 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.475 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.475 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.243329+0000", 2024-09-20T22:17:27.475 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.475 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.475 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.243331+0000", 2024-09-20T22:17:27.476 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4479999999999999e-06 2024-09-20T22:17:27.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.476 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.476 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.243402+0000", 2024-09-20T22:17:27.476 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1604000000000001e-05 2024-09-20T22:17:27.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.476 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.476 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.243418+0000", 2024-09-20T22:17:27.476 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5934e-05 2024-09-20T22:17:27.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.476 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.477 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.243426+0000", 2024-09-20T22:17:27.477 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.165e-06 2024-09-20T22:17:27.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.477 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.477 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.247494+0000", 2024-09-20T22:17:27.477 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0040683840000000004 2024-09-20T22:17:27.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.477 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:27.477 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.247586+0000", 2024-09-20T22:17:27.477 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2409000000000002e-05 2024-09-20T22:17:27.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:17:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.247708+0000", 2024-09-20T22:17:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000121508 2024-09-20T22:17:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.250076+0000", 2024-09-20T22:17:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0023685999999999998 2024-09-20T22:17:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.259850+0000", 2024-09-20T22:17:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0097730779999999993 2024-09-20T22:17:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.479 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.479 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.486491+0000", 2024-09-20T22:17:27.479 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.22664168800000001 2024-09-20T22:17:27.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.479 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:27.479 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.486548+0000", 2024-09-20T22:17:27.479 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7215e-05 2024-09-20T22:17:27.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.479 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.479 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.486599+0000", 2024-09-20T22:17:27.479 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0073e-05 2024-09-20T22:17:27.479 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.479 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.480 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.480 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:146986 10.6 10:676af665:::benchmark_data_smithi060_167766_object22194:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e416)", 2024-09-20T22:17:27.480 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:06.200699+0000", 2024-09-20T22:17:27.480 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 21.153316827000001, 2024-09-20T22:17:27.480 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.076551260999999995, 2024-09-20T22:17:27.480 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.480 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.480 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:27.480 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:27.480 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:27.480 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:27.480 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 146986 2024-09-20T22:17:27.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.480 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.200699+0000", 2024-09-20T22:17:27.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.200699+0000", 2024-09-20T22:17:27.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.200701+0000", 2024-09-20T22:17:27.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3039999999999999e-06 2024-09-20T22:17:27.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.482 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.482 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.200706+0000", 2024-09-20T22:17:27.482 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8559999999999998e-06 2024-09-20T22:17:27.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.482 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.482 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.200716+0000", 2024-09-20T22:17:27.482 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0729e-05 2024-09-20T22:17:27.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.482 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.482 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.200723+0000", 2024-09-20T22:17:27.482 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1790000000000002e-06 2024-09-20T22:17:27.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.483 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.483 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.200780+0000", 2024-09-20T22:17:27.483 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6147999999999998e-05 2024-09-20T22:17:27.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.483 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:27.483 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.200887+0000", 2024-09-20T22:17:27.483 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010781999999999999 2024-09-20T22:17:27.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.483 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:17:27.483 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.200990+0000", 2024-09-20T22:17:27.483 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000102988 2024-09-20T22:17:27.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.483 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:27.484 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.206925+0000", 2024-09-20T22:17:27.484 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0059342919999999999 2024-09-20T22:17:27.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.484 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.484 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.277030+0000", 2024-09-20T22:17:27.484 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.070105405999999995 2024-09-20T22:17:27.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.484 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:27.484 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.277166+0000", 2024-09-20T22:17:27.484 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013634500000000001 2024-09-20T22:17:27.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.484 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:27.484 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.277211+0000", 2024-09-20T22:17:27.485 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4302999999999999e-05 2024-09-20T22:17:27.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.485 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.485 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.277251+0000", 2024-09-20T22:17:27.485 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9891000000000001e-05 2024-09-20T22:17:27.485 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.485 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.485 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.485 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.16005.0:146988 10.7 e416/370 10:e31fe6a5:::benchmark_data_smithi060_167766_object22196:head v 416'9216, mlcod=416'9213)", 2024-09-20T22:17:27.485 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:06.203557+0000", 2024-09-20T22:17:27.485 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 21.150459597000001, 2024-09-20T22:17:27.486 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.080685705999999996, 2024-09-20T22:17:27.486 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.486 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.486 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:27.486 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.486 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.486 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.203557+0000", 2024-09-20T22:17:27.486 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.486 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.486 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.203557+0000", 2024-09-20T22:17:27.487 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.487 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.487 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.487 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.203557+0000", 2024-09-20T22:17:27.487 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3200000000000005e-07 2024-09-20T22:17:27.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.487 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.487 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.487 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.203563+0000", 2024-09-20T22:17:27.487 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5400000000000003e-06 2024-09-20T22:17:27.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.487 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.487 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.487 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.203572+0000", 2024-09-20T22:17:27.488 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3600000000000002e-06 2024-09-20T22:17:27.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.488 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.488 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.488 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.203578+0000", 2024-09-20T22:17:27.488 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0950000000000001e-06 2024-09-20T22:17:27.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.488 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.488 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.488 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.205536+0000", 2024-09-20T22:17:27.488 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0019576160000000001 2024-09-20T22:17:27.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.488 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.489 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:27.489 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.205555+0000", 2024-09-20T22:17:27.489 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9443e-05 2024-09-20T22:17:27.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.489 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:27.489 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.284157+0000", 2024-09-20T22:17:27.489 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.078602105000000005 2024-09-20T22:17:27.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.489 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.489 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.284242+0000", 2024-09-20T22:17:27.489 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4914999999999996e-05 2024-09-20T22:17:27.489 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.489 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.490 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.490 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.16005.0:146991 10.7 e416/370 10:e894efc8:::benchmark_data_smithi060_167766_object22199:head v 416'9217, mlcod=416'9213)", 2024-09-20T22:17:27.490 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:06.206682+0000", 2024-09-20T22:17:27.490 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 21.147334618999999, 2024-09-20T22:17:27.490 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.077626945000000003, 2024-09-20T22:17:27.490 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:27.490 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:27.490 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:27.490 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:27.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.490 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:27.491 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.206682+0000", 2024-09-20T22:17:27.491 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.491 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.491 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:27.491 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.206682+0000", 2024-09-20T22:17:27.491 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:27.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.491 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.491 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:27.491 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.206682+0000", 2024-09-20T22:17:27.491 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2500000000000005e-07 2024-09-20T22:17:27.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.491 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.491 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:27.492 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.206688+0000", 2024-09-20T22:17:27.492 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3750000000000002e-06 2024-09-20T22:17:27.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.492 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.492 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:27.492 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.206698+0000", 2024-09-20T22:17:27.492 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0199e-05 2024-09-20T22:17:27.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.492 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.492 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:27.492 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.206704+0000", 2024-09-20T22:17:27.492 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3470000000000004e-06 2024-09-20T22:17:27.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.492 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.493 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:27.493 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.208032+0000", 2024-09-20T22:17:27.493 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001328131 2024-09-20T22:17:27.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.493 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.493 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:27.493 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.208049+0000", 2024-09-20T22:17:27.493 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6725999999999999e-05 2024-09-20T22:17:27.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.493 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.493 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:27.493 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.284248+0000", 2024-09-20T22:17:27.493 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.076198629000000004 2024-09-20T22:17:27.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:27.493 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:27.493 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:27.494 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:06.284309+0000", 2024-09-20T22:17:27.494 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0813000000000002e-05 2024-09-20T22:17:27.494 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.494 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.494 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.494 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:27.494 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:27.494 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:17:27.503 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:27.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:27.504 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:27.604 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:27.635 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:27.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:27.636 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:27.737 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:27.775 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:27.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:27.776 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:27.877 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:27.913 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:27.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:27.914 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:28.015 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:28.053 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:28.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:28.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:28.154 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:28.189 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:28.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:28.191 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:28.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:28 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:17:28.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:28 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:17:28.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:28 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:28.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:28 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1577965160' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:17:28.285 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:28 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:17:28.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:28 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:17:28.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:28 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:28.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:28 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1577965160' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:17:28.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:28 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:17:28.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:28 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:17:28.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:28 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:17:28.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:28 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1577965160' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:17:28.292 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:28.326 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:17:28.332 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:28.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:28.333 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:28.433 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:28.457 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:28.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:28.458 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:28.559 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:28.591 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:28.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:28.592 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:28.654 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T22:17:28.693 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:28.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:28.734 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:28.734 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:28.835 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:28.873 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:28.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:28.874 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:28.975 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:29.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:29.019 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:29.019 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:29.089 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:17:29.121 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:29.159 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:29.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:29.160 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:29.180 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:17:29.180 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:17:29.220 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:17:29.220 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:17:29.257 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:29 smithi060 ceph-mon[29694]: pgmap v778: 168 pgs: 168 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-20T22:17:29.257 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:29 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:17:29.257 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:29 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1577965160' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:17:29.257 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:29 smithi060 ceph-mon[29694]: osdmap e425: 8 total, 8 up, 8 in 2024-09-20T22:17:29.257 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:29 smithi060 ceph-mon[40290]: pgmap v778: 168 pgs: 168 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-20T22:17:29.257 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:29 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:17:29.257 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:29 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1577965160' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:17:29.257 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:29 smithi060 ceph-mon[40290]: osdmap e425: 8 total, 8 up, 8 in 2024-09-20T22:17:29.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:29.265 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:17:29.265 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:17:29.299 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:29.299 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:17:29.299 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:17:29.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:29.300 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:29.331 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:17:29.331 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:17:29.367 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:17:29.367 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:17:29.390 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:29 smithi067 ceph-mon[33511]: pgmap v778: 168 pgs: 168 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.43k op/s 2024-09-20T22:17:29.390 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:29 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:17:29.390 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:29 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1577965160' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:17:29.390 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:29 smithi067 ceph-mon[33511]: osdmap e425: 8 total, 8 up, 8 in 2024-09-20T22:17:29.401 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:29.409 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:17:29.409 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:17:29.439 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:29.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:29.441 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:29.442 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:17:29.442 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:17:29.542 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:29.584 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:29.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:29.585 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:29.686 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:29.719 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:29.719 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:29.719 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:29.820 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:29.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:29.854 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:29.854 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:29.955 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:29.986 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:29.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:29.987 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:30.087 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:30.110 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:30.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:30.111 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:30.211 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:30.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:30.236 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:30.236 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:30.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:30 smithi060 ceph-mon[29694]: 3.3 scrub starts 2024-09-20T22:17:30.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:30 smithi060 ceph-mon[29694]: 3.3 scrub ok 2024-09-20T22:17:30.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:30 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2754662582' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:30.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:30 smithi060 ceph-mon[29694]: 1.4 scrub starts 2024-09-20T22:17:30.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:30 smithi060 ceph-mon[29694]: 1.4 scrub ok 2024-09-20T22:17:30.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:30 smithi060 ceph-mon[29694]: 4.1f scrub starts 2024-09-20T22:17:30.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:30 smithi060 ceph-mon[29694]: 4.1f scrub ok 2024-09-20T22:17:30.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:30 smithi060 ceph-mon[40290]: 3.3 scrub starts 2024-09-20T22:17:30.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:30 smithi060 ceph-mon[40290]: 3.3 scrub ok 2024-09-20T22:17:30.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:30 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2754662582' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:30.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:30 smithi060 ceph-mon[40290]: 1.4 scrub starts 2024-09-20T22:17:30.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:30 smithi060 ceph-mon[40290]: 1.4 scrub ok 2024-09-20T22:17:30.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:30 smithi060 ceph-mon[40290]: 4.1f scrub starts 2024-09-20T22:17:30.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:30 smithi060 ceph-mon[40290]: 4.1f scrub ok 2024-09-20T22:17:30.337 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:30.373 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:30.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:30.374 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:30.475 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:30.497 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:30.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:30.498 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:30 smithi067 ceph-mon[33511]: 3.3 scrub starts 2024-09-20T22:17:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:30 smithi067 ceph-mon[33511]: 3.3 scrub ok 2024-09-20T22:17:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:30 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2754662582' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:30 smithi067 ceph-mon[33511]: 1.4 scrub starts 2024-09-20T22:17:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:30 smithi067 ceph-mon[33511]: 1.4 scrub ok 2024-09-20T22:17:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:30 smithi067 ceph-mon[33511]: 4.1f scrub starts 2024-09-20T22:17:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:30 smithi067 ceph-mon[33511]: 4.1f scrub ok 2024-09-20T22:17:30.550 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:17:30.551 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:17:30.551 INFO:tasks.thrashosds.thrasher:Growing pool .mgr 2024-09-20T22:17:30.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:17:30.599 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:30.640 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:30.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:30.641 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:30.742 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:30.785 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:30.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:30.786 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:30.887 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:30.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:30.936 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:30.936 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:31.018 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:31.037 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:31.073 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:31.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:31.074 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:31.175 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:31.204 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:31.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:31.205 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:31.302 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:17:31.302 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:17:31.305 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:31.333 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:31 smithi060 ceph-mon[29694]: 5.1b scrub starts 2024-09-20T22:17:31.333 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:31 smithi060 ceph-mon[29694]: 5.1b scrub ok 2024-09-20T22:17:31.334 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:31 smithi060 ceph-mon[29694]: pgmap v780: 168 pgs: 168 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-20T22:17:31.334 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:31 smithi060 ceph-mon[29694]: 10.2 scrub starts 2024-09-20T22:17:31.334 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:31 smithi060 ceph-mon[29694]: 1.c scrub starts 2024-09-20T22:17:31.334 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:31 smithi060 ceph-mon[29694]: 1.c scrub ok 2024-09-20T22:17:31.334 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:31 smithi060 ceph-mon[40290]: 5.1b scrub starts 2024-09-20T22:17:31.334 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:31 smithi060 ceph-mon[40290]: 5.1b scrub ok 2024-09-20T22:17:31.334 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:31 smithi060 ceph-mon[40290]: pgmap v780: 168 pgs: 168 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-20T22:17:31.334 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:31 smithi060 ceph-mon[40290]: 10.2 scrub starts 2024-09-20T22:17:31.334 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:31 smithi060 ceph-mon[40290]: 1.c scrub starts 2024-09-20T22:17:31.335 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:31 smithi060 ceph-mon[40290]: 1.c scrub ok 2024-09-20T22:17:31.356 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:31.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:31.357 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:31.458 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:31.496 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:31.497 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:31.497 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:31.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:31 smithi067 ceph-mon[33511]: 5.1b scrub starts 2024-09-20T22:17:31.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:31 smithi067 ceph-mon[33511]: 5.1b scrub ok 2024-09-20T22:17:31.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:31 smithi067 ceph-mon[33511]: pgmap v780: 168 pgs: 168 active+clean; 4.8 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.16k op/s 2024-09-20T22:17:31.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:31 smithi067 ceph-mon[33511]: 10.2 scrub starts 2024-09-20T22:17:31.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:31 smithi067 ceph-mon[33511]: 1.c scrub starts 2024-09-20T22:17:31.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:31 smithi067 ceph-mon[33511]: 1.c scrub ok 2024-09-20T22:17:31.598 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:31.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:31.637 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:31.637 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:31.718 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:17:31.737 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:31.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:31.806 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:31.806 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:31.907 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:31.941 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:31.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:31.942 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:32.042 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:32.056 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:17:32.059 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:17:32.075 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:32.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:32.076 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:32.177 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:32.182 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T22:17:32.212 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:32.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:32.213 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:32.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:32.354 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:32.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:32.355 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:32.456 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:32.465 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:32 smithi060 ceph-mon[29694]: 4.1b scrub starts 2024-09-20T22:17:32.465 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:32 smithi060 ceph-mon[29694]: 4.1b scrub ok 2024-09-20T22:17:32.466 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:32 smithi060 ceph-mon[29694]: 10.6 scrub starts 2024-09-20T22:17:32.466 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:32 smithi060 ceph-mon[40290]: 4.1b scrub starts 2024-09-20T22:17:32.466 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:32 smithi060 ceph-mon[40290]: 4.1b scrub ok 2024-09-20T22:17:32.466 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:32 smithi060 ceph-mon[40290]: 10.6 scrub starts 2024-09-20T22:17:32.480 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:17:32.504 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:32.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:32.505 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:32.516 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:32 smithi067 ceph-mon[33511]: 4.1b scrub starts 2024-09-20T22:17:32.516 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:32 smithi067 ceph-mon[33511]: 4.1b scrub ok 2024-09-20T22:17:32.516 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:32 smithi067 ceph-mon[33511]: 10.6 scrub starts 2024-09-20T22:17:32.606 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:32.639 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:32.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:32.640 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:32.678 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:17:32.740 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:32.778 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:32.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:32.779 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:32.879 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:32.921 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:32.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:32.922 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:32.985 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":780,"stamp":"2024-09-20T22:17:30.282005+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5143730739,"num_objects":78678,"num_object_clones":0,"num_object_copies":236034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78678,"num_whiteouts":0,"num_read":1985,"num_read_kb":1737,"num_write":171823,"num_write_kb":7986542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":202,"num_bytes_recovered":268673,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":82665,"ondisk_log_size":82665,"up":504,"acting":504,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":504,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":20277816,"kb_used_data":15225432,"kb_used_omap":15,"kb_used_meta":5051312,"kb_avail":729683400,"statfs":{"total":767960285184,"available":747195801600,"internally_reserved":0,"allocated":15590842368,"data_stored":15587630925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":5172544275},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,7,3,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-605814784,"num_objects":-9244,"num_object_clones":0,"num_object_copies":-27732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-9244,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":9244,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001981"},"pg_stats":[{"pgid":"3.1f","version":"0'0","reported_seq":229,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.026399+0000","last_change":"2024-09-20T22:16:31.993232+0000","last_active":"2024-09-20T22:17:26.026399+0000","last_peered":"2024-09-20T22:17:26.026399+0000","last_clean":"2024-09-20T22:17:26.026399+0000","last_became_active":"2024-09-20T22:12:52.515921+0000","last_became_peered":"2024-09-20T22:12:52.515921+0000","last_unstale":"2024-09-20T22:17:26.026399+0000","last_undegraded":"2024-09-20T22:17:26.026399+0000","last_fullsized":"2024-09-20T22:17:26.026399+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:31.993104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:31.993104+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.993104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:36.968800+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"327'19","reported_seq":817,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.982920+0000","last_change":"2024-09-20T22:16:29.508726+0000","last_active":"2024-09-20T22:17:21.982920+0000","last_peered":"2024-09-20T22:17:21.982920+0000","last_clean":"2024-09-20T22:17:21.982920+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:17:21.982920+0000","last_undegraded":"2024-09-20T22:17:21.982920+0000","last_fullsized":"2024-09-20T22:17:21.982920+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"327'19","last_scrub_stamp":"2024-09-20T22:16:29.508607+0000","last_deep_scrub":"252'13","last_deep_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_clean_scrub_stamp":"2024-09-20T22:16:29.508607+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:49.145180+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":666,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.336311+0000","last_change":"2024-09-20T22:17:04.319541+0000","last_active":"2024-09-20T22:17:28.336311+0000","last_peered":"2024-09-20T22:17:28.336311+0000","last_clean":"2024-09-20T22:17:28.336311+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:17:28.336311+0000","last_undegraded":"2024-09-20T22:17:28.336311+0000","last_fullsized":"2024-09-20T22:17:28.336311+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:04.319429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:04.319429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:24.998542+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":783,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.980727+0000","last_change":"2024-09-20T22:16:23.438091+0000","last_active":"2024-09-20T22:17:21.980727+0000","last_peered":"2024-09-20T22:17:21.980727+0000","last_clean":"2024-09-20T22:17:21.980727+0000","last_became_active":"2024-09-20T22:11:30.485735+0000","last_became_peered":"2024-09-20T22:11:30.485735+0000","last_unstale":"2024-09-20T22:17:21.980727+0000","last_undegraded":"2024-09-20T22:17:21.980727+0000","last_fullsized":"2024-09-20T22:17:21.980727+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:23.437959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:23.437959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:36.548769+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":772,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.014436+0000","last_change":"2024-09-20T22:16:27.694458+0000","last_active":"2024-09-20T22:17:26.014436+0000","last_peered":"2024-09-20T22:17:26.014436+0000","last_clean":"2024-09-20T22:17:26.014436+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:17:26.014436+0000","last_undegraded":"2024-09-20T22:17:26.014436+0000","last_fullsized":"2024-09-20T22:17:26.014436+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:16:27.694309+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:27.694309+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:50.010230+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":256,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.021147+0000","last_change":"2024-09-20T22:16:22.789638+0000","last_active":"2024-09-20T22:17:26.021147+0000","last_peered":"2024-09-20T22:17:26.021147+0000","last_clean":"2024-09-20T22:17:26.021147+0000","last_became_active":"2024-09-20T22:12:33.545100+0000","last_became_peered":"2024-09-20T22:12:33.545100+0000","last_unstale":"2024-09-20T22:17:26.021147+0000","last_undegraded":"2024-09-20T22:17:26.021147+0000","last_fullsized":"2024-09-20T22:17:26.021147+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:22.789569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:22.789569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:52.149562+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"406'34","reported_seq":379,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.025789+0000","last_change":"2024-09-20T22:16:31.033633+0000","last_active":"2024-09-20T22:17:26.025789+0000","last_peered":"2024-09-20T22:17:26.025789+0000","last_clean":"2024-09-20T22:17:26.025789+0000","last_became_active":"2024-09-20T22:12:52.517296+0000","last_became_peered":"2024-09-20T22:12:52.517296+0000","last_unstale":"2024-09-20T22:17:26.025789+0000","last_undegraded":"2024-09-20T22:17:26.025789+0000","last_fullsized":"2024-09-20T22:17:26.025789+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'34","last_scrub_stamp":"2024-09-20T22:16:31.033354+0000","last_deep_scrub":"406'34","last_deep_scrub_stamp":"2024-09-20T22:16:31.033354+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.033354+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:54.712855+0000","scrub_duration":30,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":776,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.982837+0000","last_change":"2024-09-20T22:16:36.461743+0000","last_active":"2024-09-20T22:17:21.982837+0000","last_peered":"2024-09-20T22:17:21.982837+0000","last_clean":"2024-09-20T22:17:21.982837+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:17:21.982837+0000","last_undegraded":"2024-09-20T22:17:21.982837+0000","last_fullsized":"2024-09-20T22:17:21.982837+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:36.461517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:36.461517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:36.828806+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":791,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.023397+0000","last_change":"2024-09-20T22:15:21.178920+0000","last_active":"2024-09-20T22:17:26.023397+0000","last_peered":"2024-09-20T22:17:26.023397+0000","last_clean":"2024-09-20T22:17:26.023397+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:17:26.023397+0000","last_undegraded":"2024-09-20T22:17:26.023397+0000","last_fullsized":"2024-09-20T22:17:26.023397+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:21.178806+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:15:21.178806+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.047474+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"328'15","reported_seq":664,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.025702+0000","last_change":"2024-09-20T22:17:03.697131+0000","last_active":"2024-09-20T22:17:26.025702+0000","last_peered":"2024-09-20T22:17:26.025702+0000","last_clean":"2024-09-20T22:17:26.025702+0000","last_became_active":"2024-09-20T22:17:03.663697+0000","last_became_peered":"2024-09-20T22:17:03.663697+0000","last_unstale":"2024-09-20T22:17:26.025702+0000","last_undegraded":"2024-09-20T22:17:26.025702+0000","last_fullsized":"2024-09-20T22:17:26.025702+0000","mapping_epoch":415,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":416,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:15:59.331593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:59.331593+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:28.247001+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":323,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.023044+0000","last_change":"2024-09-20T22:14:54.153118+0000","last_active":"2024-09-20T22:17:26.023044+0000","last_peered":"2024-09-20T22:17:26.023044+0000","last_clean":"2024-09-20T22:17:26.023044+0000","last_became_active":"2024-09-20T22:12:52.518549+0000","last_became_peered":"2024-09-20T22:12:52.518549+0000","last_unstale":"2024-09-20T22:17:26.023044+0000","last_undegraded":"2024-09-20T22:17:26.023044+0000","last_fullsized":"2024-09-20T22:17:26.023044+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:14:54.152717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:14:54.152717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.047350+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":210,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.337427+0000","last_change":"2024-09-20T22:17:03.665051+0000","last_active":"2024-09-20T22:17:28.337427+0000","last_peered":"2024-09-20T22:17:28.337427+0000","last_clean":"2024-09-20T22:17:28.337427+0000","last_became_active":"2024-09-20T22:17:03.664619+0000","last_became_peered":"2024-09-20T22:17:03.664619+0000","last_unstale":"2024-09-20T22:17:28.337427+0000","last_undegraded":"2024-09-20T22:17:28.337427+0000","last_fullsized":"2024-09-20T22:17:28.337427+0000","mapping_epoch":415,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":416,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:37.934760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:37.934760+0000","last_clean_scrub_stamp":"2024-09-20T22:16:37.934760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:01.638632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":796,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.022962+0000","last_change":"2024-09-20T22:15:20.182058+0000","last_active":"2024-09-20T22:17:26.022962+0000","last_peered":"2024-09-20T22:17:26.022962+0000","last_clean":"2024-09-20T22:17:26.022962+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:17:26.022962+0000","last_undegraded":"2024-09-20T22:17:26.022962+0000","last_fullsized":"2024-09-20T22:17:26.022962+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:20.181942+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:30.215263+0000","last_clean_scrub_stamp":"2024-09-20T22:15:20.181942+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.047525+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"328'7","reported_seq":804,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.980883+0000","last_change":"2024-09-20T22:15:31.119172+0000","last_active":"2024-09-20T22:17:21.980883+0000","last_peered":"2024-09-20T22:17:21.980883+0000","last_clean":"2024-09-20T22:17:21.980883+0000","last_became_active":"2024-09-20T22:08:22.144364+0000","last_became_peered":"2024-09-20T22:08:22.144364+0000","last_unstale":"2024-09-20T22:17:21.980883+0000","last_undegraded":"2024-09-20T22:17:21.980883+0000","last_fullsized":"2024-09-20T22:17:21.980883+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'7","last_scrub_stamp":"2024-09-20T22:15:31.119046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:31.119046+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:23.492105+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":783,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.988359+0000","last_change":"2024-09-20T22:17:03.783046+0000","last_active":"2024-09-20T22:17:21.988359+0000","last_peered":"2024-09-20T22:17:21.988359+0000","last_clean":"2024-09-20T22:17:21.988359+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:17:21.988359+0000","last_undegraded":"2024-09-20T22:17:21.988359+0000","last_fullsized":"2024-09-20T22:17:21.988359+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:03.782934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:03.782934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:09.870686+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":311,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.023800+0000","last_change":"2024-09-20T22:15:00.077644+0000","last_active":"2024-09-20T22:17:26.023800+0000","last_peered":"2024-09-20T22:17:26.023800+0000","last_clean":"2024-09-20T22:17:26.023800+0000","last_became_active":"2024-09-20T22:11:30.487137+0000","last_became_peered":"2024-09-20T22:11:30.487137+0000","last_unstale":"2024-09-20T22:17:26.023800+0000","last_undegraded":"2024-09-20T22:17:26.023800+0000","last_fullsized":"2024-09-20T22:17:26.023800+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:00.077519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:15:00.077519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.047391+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":772,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.981480+0000","last_change":"2024-09-20T22:16:34.472967+0000","last_active":"2024-09-20T22:17:21.981480+0000","last_peered":"2024-09-20T22:17:21.981480+0000","last_clean":"2024-09-20T22:17:21.981480+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:17:21.981480+0000","last_undegraded":"2024-09-20T22:17:21.981480+0000","last_fullsized":"2024-09-20T22:17:21.981480+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:34.472700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:16:34.472700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:51.839167+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":51,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.981114+0000","last_change":"2024-09-20T22:17:07.404010+0000","last_active":"2024-09-20T22:17:21.981114+0000","last_peered":"2024-09-20T22:17:21.981114+0000","last_clean":"2024-09-20T22:17:21.981114+0000","last_became_active":"2024-09-20T22:17:03.652857+0000","last_became_peered":"2024-09-20T22:17:03.652857+0000","last_unstale":"2024-09-20T22:17:21.981114+0000","last_undegraded":"2024-09-20T22:17:21.981114+0000","last_fullsized":"2024-09-20T22:17:21.981114+0000","mapping_epoch":415,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":416,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:07.403912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:17:07.403912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:35.691657+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"328'34","reported_seq":375,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.024568+0000","last_change":"2024-09-20T22:15:55.987603+0000","last_active":"2024-09-20T22:17:26.024568+0000","last_peered":"2024-09-20T22:17:26.024568+0000","last_clean":"2024-09-20T22:17:26.024568+0000","last_became_active":"2024-09-20T22:11:12.507311+0000","last_became_peered":"2024-09-20T22:11:12.507311+0000","last_unstale":"2024-09-20T22:17:26.024568+0000","last_undegraded":"2024-09-20T22:17:26.024568+0000","last_fullsized":"2024-09-20T22:17:26.024568+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'34","last_scrub_stamp":"2024-09-20T22:15:55.987386+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:15:55.987386+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":628,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.337073+0000","last_change":"2024-09-20T22:16:23.258053+0000","last_active":"2024-09-20T22:17:28.337073+0000","last_peered":"2024-09-20T22:17:28.337073+0000","last_clean":"2024-09-20T22:17:28.337073+0000","last_became_active":"2024-09-20T22:12:52.520985+0000","last_became_peered":"2024-09-20T22:12:52.520985+0000","last_unstale":"2024-09-20T22:17:28.337073+0000","last_undegraded":"2024-09-20T22:17:28.337073+0000","last_fullsized":"2024-09-20T22:17:28.337073+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:23.257937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:11.053276+0000","last_clean_scrub_stamp":"2024-09-20T22:16:23.257937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:33.735295+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":725,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.021187+0000","last_change":"2024-09-20T22:16:16.771843+0000","last_active":"2024-09-20T22:17:26.021187+0000","last_peered":"2024-09-20T22:17:26.021187+0000","last_clean":"2024-09-20T22:17:26.021187+0000","last_became_active":"2024-09-20T22:12:21.521433+0000","last_became_peered":"2024-09-20T22:12:21.521433+0000","last_unstale":"2024-09-20T22:17:26.021187+0000","last_undegraded":"2024-09-20T22:17:26.021187+0000","last_fullsized":"2024-09-20T22:17:26.021187+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:16.771744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:16.771744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:43.671506+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":726,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.024008+0000","last_change":"2024-09-20T22:15:58.078205+0000","last_active":"2024-09-20T22:17:26.024008+0000","last_peered":"2024-09-20T22:17:26.024008+0000","last_clean":"2024-09-20T22:17:26.024008+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:17:26.024008+0000","last_undegraded":"2024-09-20T22:17:26.024008+0000","last_fullsized":"2024-09-20T22:17:26.024008+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:58.078099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:58.078099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.047727+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"328'23","reported_seq":693,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.337993+0000","last_change":"2024-09-20T22:15:17.533607+0000","last_active":"2024-09-20T22:17:28.337993+0000","last_peered":"2024-09-20T22:17:28.337993+0000","last_clean":"2024-09-20T22:17:28.337993+0000","last_became_active":"2024-09-20T22:12:21.517147+0000","last_became_peered":"2024-09-20T22:12:21.517147+0000","last_unstale":"2024-09-20T22:17:28.337993+0000","last_undegraded":"2024-09-20T22:17:28.337993+0000","last_fullsized":"2024-09-20T22:17:28.337993+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'23","last_scrub_stamp":"2024-09-20T22:15:17.533453+0000","last_deep_scrub":"250'17","last_deep_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_clean_scrub_stamp":"2024-09-20T22:15:17.533453+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.477098+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":774,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.982136+0000","last_change":"2024-09-20T22:17:09.392111+0000","last_active":"2024-09-20T22:17:21.982136+0000","last_peered":"2024-09-20T22:17:21.982136+0000","last_clean":"2024-09-20T22:17:21.982136+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:17:21.982136+0000","last_undegraded":"2024-09-20T22:17:21.982136+0000","last_fullsized":"2024-09-20T22:17:21.982136+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:09.391995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:09.391995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:31.332781+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":764,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.989885+0000","last_change":"2024-09-20T22:15:57.632871+0000","last_active":"2024-09-20T22:17:21.989885+0000","last_peered":"2024-09-20T22:17:21.989885+0000","last_clean":"2024-09-20T22:17:21.989885+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:17:21.989885+0000","last_undegraded":"2024-09-20T22:17:21.989885+0000","last_fullsized":"2024-09-20T22:17:21.989885+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:15:57.632600+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.632600+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:22.715542+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"289'2","reported_seq":626,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.336402+0000","last_change":"2024-09-20T22:17:07.356345+0000","last_active":"2024-09-20T22:17:28.336402+0000","last_peered":"2024-09-20T22:17:28.336402+0000","last_clean":"2024-09-20T22:17:28.336402+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:17:28.336402+0000","last_undegraded":"2024-09-20T22:17:28.336402+0000","last_fullsized":"2024-09-20T22:17:28.336402+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-20T22:17:07.356220+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_clean_scrub_stamp":"2024-09-20T22:17:07.356220+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:31.683638+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"328'20","reported_seq":256,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.028471+0000","last_change":"2024-09-20T22:15:57.132570+0000","last_active":"2024-09-20T22:17:26.028471+0000","last_peered":"2024-09-20T22:17:26.028471+0000","last_clean":"2024-09-20T22:17:26.028471+0000","last_became_active":"2024-09-20T22:12:52.526803+0000","last_became_peered":"2024-09-20T22:12:52.526803+0000","last_unstale":"2024-09-20T22:17:26.028471+0000","last_undegraded":"2024-09-20T22:17:26.028471+0000","last_fullsized":"2024-09-20T22:17:26.028471+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:15:57.132435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.132435+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:26.984119+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"291'11","reported_seq":1001,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.021406+0000","last_change":"2024-09-20T22:16:34.733980+0000","last_active":"2024-09-20T22:17:26.021406+0000","last_peered":"2024-09-20T22:17:26.021406+0000","last_clean":"2024-09-20T22:17:26.021406+0000","last_became_active":"2024-09-20T22:11:40.601048+0000","last_became_peered":"2024-09-20T22:11:40.601048+0000","last_unstale":"2024-09-20T22:17:26.021406+0000","last_undegraded":"2024-09-20T22:17:26.021406+0000","last_fullsized":"2024-09-20T22:17:26.021406+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'11","last_scrub_stamp":"2024-09-20T22:16:34.733870+0000","last_deep_scrub":"291'11","last_deep_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_clean_scrub_stamp":"2024-09-20T22:16:34.733870+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:58.907961+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"423'10466","reported_seq":10651,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:25.877436+0000","last_change":"2024-09-20T22:16:24.149762+0000","last_active":"2024-09-20T22:17:25.877436+0000","last_peered":"2024-09-20T22:17:25.877436+0000","last_clean":"2024-09-20T22:17:25.877436+0000","last_became_active":"2024-09-20T22:15:08.927535+0000","last_became_peered":"2024-09-20T22:15:08.927535+0000","last_unstale":"2024-09-20T22:17:25.877436+0000","last_undegraded":"2024-09-20T22:17:25.877436+0000","last_fullsized":"2024-09-20T22:17:25.877436+0000","mapping_epoch":370,"log_start":"405'6100","ondisk_log_start":"405'6100","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"405'6096","last_scrub_stamp":"2024-09-20T22:16:24.146029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:24.146029+0000","objects_scrubbed":6025,"log_size":4366,"log_dups_size":0,"ondisk_log_size":4366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.643132+0000","scrub_duration":1743,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330170368,"num_objects":5038,"num_object_clones":0,"num_object_copies":15114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10466,"num_write_kb":496128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":783,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.023821+0000","last_change":"2024-09-20T22:16:57.867797+0000","last_active":"2024-09-20T22:17:26.023821+0000","last_peered":"2024-09-20T22:17:26.023821+0000","last_clean":"2024-09-20T22:17:26.023821+0000","last_became_active":"2024-09-20T22:08:48.452786+0000","last_became_peered":"2024-09-20T22:08:48.452786+0000","last_unstale":"2024-09-20T22:17:26.023821+0000","last_undegraded":"2024-09-20T22:17:26.023821+0000","last_fullsized":"2024-09-20T22:17:26.023821+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:16:57.867690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:16:57.867690+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:15.633464+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"328'27","reported_seq":569,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.024293+0000","last_change":"2024-09-20T22:15:50.925776+0000","last_active":"2024-09-20T22:17:26.024293+0000","last_peered":"2024-09-20T22:17:26.024293+0000","last_clean":"2024-09-20T22:17:26.024293+0000","last_became_active":"2024-09-20T22:08:22.145408+0000","last_became_peered":"2024-09-20T22:08:22.145408+0000","last_unstale":"2024-09-20T22:17:26.024293+0000","last_undegraded":"2024-09-20T22:17:26.024293+0000","last_fullsized":"2024-09-20T22:17:26.024293+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:15:50.925654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:50.925654+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":793,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.015228+0000","last_change":"2024-09-20T22:16:31.637874+0000","last_active":"2024-09-20T22:17:26.015228+0000","last_peered":"2024-09-20T22:17:26.015228+0000","last_clean":"2024-09-20T22:17:26.015228+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:17:26.015228+0000","last_undegraded":"2024-09-20T22:17:26.015228+0000","last_fullsized":"2024-09-20T22:17:26.015228+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:31.637766+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:28.020341+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.637766+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:44.980512+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"326'13","reported_seq":1020,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.981628+0000","last_change":"2024-09-20T22:16:15.367687+0000","last_active":"2024-09-20T22:17:21.981628+0000","last_peered":"2024-09-20T22:17:21.981628+0000","last_clean":"2024-09-20T22:17:21.981628+0000","last_became_active":"2024-09-20T22:12:52.517588+0000","last_became_peered":"2024-09-20T22:12:52.517588+0000","last_unstale":"2024-09-20T22:17:21.981628+0000","last_undegraded":"2024-09-20T22:17:21.981628+0000","last_fullsized":"2024-09-20T22:17:21.981628+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'13","last_scrub_stamp":"2024-09-20T22:16:15.367449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:15.367449+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:32.822489+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":323,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.022995+0000","last_change":"2024-09-20T22:15:33.082489+0000","last_active":"2024-09-20T22:17:26.022995+0000","last_peered":"2024-09-20T22:17:26.022995+0000","last_clean":"2024-09-20T22:17:26.022995+0000","last_became_active":"2024-09-20T22:12:52.517826+0000","last_became_peered":"2024-09-20T22:12:52.517826+0000","last_unstale":"2024-09-20T22:17:26.022995+0000","last_undegraded":"2024-09-20T22:17:26.022995+0000","last_fullsized":"2024-09-20T22:17:26.022995+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:33.082368+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:33.082368+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.047552+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"423'10474","reported_seq":10665,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:25.923107+0000","last_change":"2024-09-20T22:16:34.386928+0000","last_active":"2024-09-20T22:17:25.923107+0000","last_peered":"2024-09-20T22:17:25.923107+0000","last_clean":"2024-09-20T22:17:25.923107+0000","last_became_active":"2024-09-20T22:15:08.928064+0000","last_became_peered":"2024-09-20T22:15:08.928064+0000","last_unstale":"2024-09-20T22:17:25.923107+0000","last_undegraded":"2024-09-20T22:17:25.923107+0000","last_fullsized":"2024-09-20T22:17:25.923107+0000","mapping_epoch":370,"log_start":"405'6100","ondisk_log_start":"405'6100","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6697","last_scrub_stamp":"2024-09-20T22:16:34.386672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:34.386672+0000","objects_scrubbed":6630,"log_size":4374,"log_dups_size":0,"ondisk_log_size":4374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:49.633430+0000","scrub_duration":1869,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319029248,"num_objects":4868,"num_object_clones":0,"num_object_copies":14604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10474,"num_write_kb":490944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":787,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.984403+0000","last_change":"2024-09-20T22:15:46.295600+0000","last_active":"2024-09-20T22:17:21.984403+0000","last_peered":"2024-09-20T22:17:21.984403+0000","last_clean":"2024-09-20T22:17:21.984403+0000","last_became_active":"2024-09-20T22:15:45.627770+0000","last_became_peered":"2024-09-20T22:15:45.627770+0000","last_unstale":"2024-09-20T22:17:21.984403+0000","last_undegraded":"2024-09-20T22:17:21.984403+0000","last_fullsized":"2024-09-20T22:17:21.984403+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:46.295352+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:15:46.295352+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:23.492253+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"328'30","reported_seq":787,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.988423+0000","last_change":"2024-09-20T22:16:16.609501+0000","last_active":"2024-09-20T22:17:21.988423+0000","last_peered":"2024-09-20T22:17:21.988423+0000","last_clean":"2024-09-20T22:17:21.988423+0000","last_became_active":"2024-09-20T22:12:52.526867+0000","last_became_peered":"2024-09-20T22:12:52.526867+0000","last_unstale":"2024-09-20T22:17:21.988423+0000","last_undegraded":"2024-09-20T22:17:21.988423+0000","last_fullsized":"2024-09-20T22:17:21.988423+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'30","last_scrub_stamp":"2024-09-20T22:16:16.609377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:16.609377+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:45.896136+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":268,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.988890+0000","last_change":"2024-09-20T22:16:28.701889+0000","last_active":"2024-09-20T22:17:21.988890+0000","last_peered":"2024-09-20T22:17:21.988890+0000","last_clean":"2024-09-20T22:17:21.988890+0000","last_became_active":"2024-09-20T22:12:52.528947+0000","last_became_peered":"2024-09-20T22:12:52.528947+0000","last_unstale":"2024-09-20T22:17:21.988890+0000","last_undegraded":"2024-09-20T22:17:21.988890+0000","last_fullsized":"2024-09-20T22:17:21.988890+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:28.701780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:28.701780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:46.729610+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":787,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.982649+0000","last_change":"2024-09-20T22:15:56.322206+0000","last_active":"2024-09-20T22:17:21.982649+0000","last_peered":"2024-09-20T22:17:21.982649+0000","last_clean":"2024-09-20T22:17:21.982649+0000","last_became_active":"2024-09-20T22:10:48.919949+0000","last_became_peered":"2024-09-20T22:10:48.919949+0000","last_unstale":"2024-09-20T22:17:21.982649+0000","last_undegraded":"2024-09-20T22:17:21.982649+0000","last_fullsized":"2024-09-20T22:17:21.982649+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:56.321409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:56.321409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:24.041893+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":771,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.980941+0000","last_change":"2024-09-20T22:15:58.309888+0000","last_active":"2024-09-20T22:17:21.980941+0000","last_peered":"2024-09-20T22:17:21.980941+0000","last_clean":"2024-09-20T22:17:21.980941+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:17:21.980941+0000","last_undegraded":"2024-09-20T22:17:21.980941+0000","last_fullsized":"2024-09-20T22:17:21.980941+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:58.309764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:58.309764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:23.492280+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"424'10668","reported_seq":10864,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.134556+0000","last_change":"2024-09-20T22:17:01.077916+0000","last_active":"2024-09-20T22:17:28.134556+0000","last_peered":"2024-09-20T22:17:28.134556+0000","last_clean":"2024-09-20T22:17:28.134556+0000","last_became_active":"2024-09-20T22:15:09.088580+0000","last_became_peered":"2024-09-20T22:15:09.088580+0000","last_unstale":"2024-09-20T22:17:28.134556+0000","last_undegraded":"2024-09-20T22:17:28.134556+0000","last_fullsized":"2024-09-20T22:17:28.134556+0000","mapping_epoch":370,"log_start":"402'5700","ondisk_log_start":"402'5700","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"414'8713","last_scrub_stamp":"2024-09-20T22:17:01.077656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:17:01.077656+0000","objects_scrubbed":6782,"log_size":4968,"log_dups_size":0,"ondisk_log_size":4968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:02.245999+0000","scrub_duration":2367,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":310378496,"num_objects":4736,"num_object_clones":0,"num_object_copies":14208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10668,"num_write_kb":492928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":823,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.019467+0000","last_change":"2024-09-20T22:15:46.874158+0000","last_active":"2024-09-20T22:17:26.019467+0000","last_peered":"2024-09-20T22:17:26.019467+0000","last_clean":"2024-09-20T22:17:26.019467+0000","last_became_active":"2024-09-20T22:15:45.642671+0000","last_became_peered":"2024-09-20T22:15:45.642671+0000","last_unstale":"2024-09-20T22:17:26.019467+0000","last_undegraded":"2024-09-20T22:17:26.019467+0000","last_fullsized":"2024-09-20T22:17:26.019467+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:46.874000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:46.874000+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:28.714608+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"328'28","reported_seq":821,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.983024+0000","last_change":"2024-09-20T22:16:18.404188+0000","last_active":"2024-09-20T22:17:21.983024+0000","last_peered":"2024-09-20T22:17:21.983024+0000","last_clean":"2024-09-20T22:17:21.983024+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:17:21.983024+0000","last_undegraded":"2024-09-20T22:17:21.983024+0000","last_fullsized":"2024-09-20T22:17:21.983024+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:16:18.403972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:18.403972+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:43.385480+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":241,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.032002+0000","last_change":"2024-09-20T22:16:35.952285+0000","last_active":"2024-09-20T22:17:26.032002+0000","last_peered":"2024-09-20T22:17:26.032002+0000","last_clean":"2024-09-20T22:17:26.032002+0000","last_became_active":"2024-09-20T22:12:52.528068+0000","last_became_peered":"2024-09-20T22:12:52.528068+0000","last_unstale":"2024-09-20T22:17:26.032002+0000","last_undegraded":"2024-09-20T22:17:26.032002+0000","last_fullsized":"2024-09-20T22:17:26.032002+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:16:35.952049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:35.952049+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:45.642636+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"315'11","reported_seq":535,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.336790+0000","last_change":"2024-09-20T22:15:56.451915+0000","last_active":"2024-09-20T22:17:28.336790+0000","last_peered":"2024-09-20T22:17:28.336790+0000","last_clean":"2024-09-20T22:17:28.336790+0000","last_became_active":"2024-09-20T22:12:33.545942+0000","last_became_peered":"2024-09-20T22:12:33.545942+0000","last_unstale":"2024-09-20T22:17:28.336790+0000","last_undegraded":"2024-09-20T22:17:28.336790+0000","last_fullsized":"2024-09-20T22:17:28.336790+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:15:56.451808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:56.451808+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.477285+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":763,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.988731+0000","last_change":"2024-09-20T22:15:52.564511+0000","last_active":"2024-09-20T22:17:21.988731+0000","last_peered":"2024-09-20T22:17:21.988731+0000","last_clean":"2024-09-20T22:17:21.988731+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:17:21.988731+0000","last_undegraded":"2024-09-20T22:17:21.988731+0000","last_fullsized":"2024-09-20T22:17:21.988731+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:52.564398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:52.564398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:22.715573+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"423'10519","reported_seq":10636,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:25.870576+0000","last_change":"2024-09-20T22:15:08.931965+0000","last_active":"2024-09-20T22:17:25.870576+0000","last_peered":"2024-09-20T22:17:25.870576+0000","last_clean":"2024-09-20T22:17:25.870576+0000","last_became_active":"2024-09-20T22:15:08.931653+0000","last_became_peered":"2024-09-20T22:15:08.931653+0000","last_unstale":"2024-09-20T22:17:25.870576+0000","last_undegraded":"2024-09-20T22:17:25.870576+0000","last_fullsized":"2024-09-20T22:17:25.870576+0000","mapping_epoch":370,"log_start":"404'6000","ondisk_log_start":"404'6000","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":4519,"log_dups_size":0,"ondisk_log_size":4519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.795732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327352320,"num_objects":4995,"num_object_clones":0,"num_object_copies":14985,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4995,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10519,"num_write_kb":496448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":827,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.021072+0000","last_change":"2024-09-20T22:17:02.809282+0000","last_active":"2024-09-20T22:17:26.021072+0000","last_peered":"2024-09-20T22:17:26.021072+0000","last_clean":"2024-09-20T22:17:26.021072+0000","last_became_active":"2024-09-20T22:15:45.635509+0000","last_became_peered":"2024-09-20T22:15:45.635509+0000","last_unstale":"2024-09-20T22:17:26.021072+0000","last_undegraded":"2024-09-20T22:17:26.021072+0000","last_fullsized":"2024-09-20T22:17:26.021072+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:02.808924+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:02.808924+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:29.934937+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"328'20","reported_seq":694,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.336811+0000","last_change":"2024-09-20T22:16:28.261680+0000","last_active":"2024-09-20T22:17:28.336811+0000","last_peered":"2024-09-20T22:17:28.336811+0000","last_clean":"2024-09-20T22:17:28.336811+0000","last_became_active":"2024-09-20T22:11:30.486532+0000","last_became_peered":"2024-09-20T22:11:30.486532+0000","last_unstale":"2024-09-20T22:17:28.336811+0000","last_undegraded":"2024-09-20T22:17:28.336811+0000","last_fullsized":"2024-09-20T22:17:28.336811+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:16:28.261547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:28.261547+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:47.319640+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":801,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.023515+0000","last_change":"2024-09-20T22:17:09.988639+0000","last_active":"2024-09-20T22:17:26.023515+0000","last_peered":"2024-09-20T22:17:26.023515+0000","last_clean":"2024-09-20T22:17:26.023515+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:17:26.023515+0000","last_undegraded":"2024-09-20T22:17:26.023515+0000","last_fullsized":"2024-09-20T22:17:26.023515+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:09.988552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:17:09.988552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:18.243010+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"363'13","reported_seq":286,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.031103+0000","last_change":"2024-09-20T22:16:25.963851+0000","last_active":"2024-09-20T22:17:26.031103+0000","last_peered":"2024-09-20T22:17:26.031103+0000","last_clean":"2024-09-20T22:17:26.031103+0000","last_became_active":"2024-09-20T22:12:57.884415+0000","last_became_peered":"2024-09-20T22:12:57.884415+0000","last_unstale":"2024-09-20T22:17:26.031103+0000","last_undegraded":"2024-09-20T22:17:26.031103+0000","last_fullsized":"2024-09-20T22:17:26.031103+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"363'13","last_scrub_stamp":"2024-09-20T22:16:25.963742+0000","last_deep_scrub":"363'13","last_deep_scrub_stamp":"2024-09-20T22:14:52.612615+0000","last_clean_scrub_stamp":"2024-09-20T22:16:25.963742+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:50.917231+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":229,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.025130+0000","last_change":"2024-09-20T22:16:31.044840+0000","last_active":"2024-09-20T22:17:26.025130+0000","last_peered":"2024-09-20T22:17:26.025130+0000","last_clean":"2024-09-20T22:17:26.025130+0000","last_became_active":"2024-09-20T22:12:52.518529+0000","last_became_peered":"2024-09-20T22:12:52.518529+0000","last_unstale":"2024-09-20T22:17:26.025130+0000","last_undegraded":"2024-09-20T22:17:26.025130+0000","last_fullsized":"2024-09-20T22:17:26.025130+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:31.044564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:31.044564+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.044564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:00.508338+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"424'10655","reported_seq":10851,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.116996+0000","last_change":"2024-09-20T22:16:35.962440+0000","last_active":"2024-09-20T22:17:28.116996+0000","last_peered":"2024-09-20T22:17:28.116996+0000","last_clean":"2024-09-20T22:17:28.116996+0000","last_became_active":"2024-09-20T22:15:08.927661+0000","last_became_peered":"2024-09-20T22:15:08.927661+0000","last_unstale":"2024-09-20T22:17:28.116996+0000","last_undegraded":"2024-09-20T22:17:28.116996+0000","last_fullsized":"2024-09-20T22:17:28.116996+0000","mapping_epoch":370,"log_start":"403'5700","ondisk_log_start":"403'5700","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"407'6856","last_scrub_stamp":"2024-09-20T22:16:35.962220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:35.962220+0000","objects_scrubbed":6781,"log_size":4955,"log_dups_size":0,"ondisk_log_size":4955,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:00.411848+0000","scrub_duration":2297,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":319815704,"num_objects":4881,"num_object_clones":0,"num_object_copies":14643,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4881,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10655,"num_write_kb":497089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":828,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.020992+0000","last_change":"2024-09-20T22:17:00.783988+0000","last_active":"2024-09-20T22:17:26.020992+0000","last_peered":"2024-09-20T22:17:26.020992+0000","last_clean":"2024-09-20T22:17:26.020992+0000","last_became_active":"2024-09-20T22:15:45.634237+0000","last_became_peered":"2024-09-20T22:15:45.634237+0000","last_unstale":"2024-09-20T22:17:26.020992+0000","last_undegraded":"2024-09-20T22:17:26.020992+0000","last_fullsized":"2024-09-20T22:17:26.020992+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:00.783879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:00.783879+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:16.123982+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"392'37","reported_seq":270,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.031359+0000","last_change":"2024-09-20T22:15:56.128146+0000","last_active":"2024-09-20T22:17:26.031359+0000","last_peered":"2024-09-20T22:17:26.031359+0000","last_clean":"2024-09-20T22:17:26.031359+0000","last_became_active":"2024-09-20T22:12:52.525538+0000","last_became_peered":"2024-09-20T22:12:52.525538+0000","last_unstale":"2024-09-20T22:17:26.031359+0000","last_undegraded":"2024-09-20T22:17:26.031359+0000","last_fullsized":"2024-09-20T22:17:26.031359+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"392'37","last_scrub_stamp":"2024-09-20T22:15:56.127540+0000","last_deep_scrub":"392'37","last_deep_scrub_stamp":"2024-09-20T22:15:56.127540+0000","last_clean_scrub_stamp":"2024-09-20T22:15:56.127540+0000","objects_scrubbed":8,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:26.984014+0000","scrub_duration":21,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":630,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.337521+0000","last_change":"2024-09-20T22:16:22.267306+0000","last_active":"2024-09-20T22:17:28.337521+0000","last_peered":"2024-09-20T22:17:28.337521+0000","last_clean":"2024-09-20T22:17:28.337521+0000","last_became_active":"2024-09-20T22:11:30.486428+0000","last_became_peered":"2024-09-20T22:11:30.486428+0000","last_unstale":"2024-09-20T22:17:28.337521+0000","last_undegraded":"2024-09-20T22:17:28.337521+0000","last_fullsized":"2024-09-20T22:17:28.337521+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:22.267191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:22.267191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:32.461862+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":805,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.020486+0000","last_change":"2024-09-20T22:15:58.616337+0000","last_active":"2024-09-20T22:17:26.020486+0000","last_peered":"2024-09-20T22:17:26.020486+0000","last_clean":"2024-09-20T22:17:26.020486+0000","last_became_active":"2024-09-20T22:12:52.519826+0000","last_became_peered":"2024-09-20T22:12:52.519826+0000","last_unstale":"2024-09-20T22:17:26.020486+0000","last_undegraded":"2024-09-20T22:17:26.020486+0000","last_fullsized":"2024-09-20T22:17:26.020486+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:58.616150+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:15:58.616150+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:28.855155+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":616,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.336164+0000","last_change":"2024-09-20T22:15:21.489682+0000","last_active":"2024-09-20T22:17:28.336164+0000","last_peered":"2024-09-20T22:17:28.336164+0000","last_clean":"2024-09-20T22:17:28.336164+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:17:28.336164+0000","last_undegraded":"2024-09-20T22:17:28.336164+0000","last_fullsized":"2024-09-20T22:17:28.336164+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:21.489619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:21.489619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.477069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"424'10882","reported_seq":11076,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.113695+0000","last_change":"2024-09-20T22:16:32.715147+0000","last_active":"2024-09-20T22:17:28.113695+0000","last_peered":"2024-09-20T22:17:28.113695+0000","last_clean":"2024-09-20T22:17:28.113695+0000","last_became_active":"2024-09-20T22:15:08.926245+0000","last_became_peered":"2024-09-20T22:15:08.926245+0000","last_unstale":"2024-09-20T22:17:28.113695+0000","last_undegraded":"2024-09-20T22:17:28.113695+0000","last_fullsized":"2024-09-20T22:17:28.113695+0000","mapping_epoch":370,"log_start":"403'5900","ondisk_log_start":"403'5900","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6800","last_scrub_stamp":"2024-09-20T22:16:32.714946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.714946+0000","objects_scrubbed":6743,"log_size":4982,"log_dups_size":0,"ondisk_log_size":4982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:39.872460+0000","scrub_duration":2056,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318767104,"num_objects":4864,"num_object_clones":0,"num_object_copies":14592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10882,"num_write_kb":503872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":807,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.023609+0000","last_change":"2024-09-20T22:15:46.095461+0000","last_active":"2024-09-20T22:17:26.023609+0000","last_peered":"2024-09-20T22:17:26.023609+0000","last_clean":"2024-09-20T22:17:26.023609+0000","last_became_active":"2024-09-20T22:15:45.627135+0000","last_became_peered":"2024-09-20T22:15:45.627135+0000","last_unstale":"2024-09-20T22:17:26.023609+0000","last_undegraded":"2024-09-20T22:17:26.023609+0000","last_fullsized":"2024-09-20T22:17:26.023609+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:46.095188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:46.095188+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.047582+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"328'52","reported_seq":842,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.337660+0000","last_change":"2024-09-20T22:17:08.395721+0000","last_active":"2024-09-20T22:17:28.337660+0000","last_peered":"2024-09-20T22:17:28.337660+0000","last_clean":"2024-09-20T22:17:28.337660+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:17:28.337660+0000","last_undegraded":"2024-09-20T22:17:28.337660+0000","last_fullsized":"2024-09-20T22:17:28.337660+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'52","last_scrub_stamp":"2024-09-20T22:17:08.395578+0000","last_deep_scrub":"328'52","last_deep_scrub_stamp":"2024-09-20T22:17:08.395578+0000","last_clean_scrub_stamp":"2024-09-20T22:17:08.395578+0000","objects_scrubbed":10,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:33.462021+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":793,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.981281+0000","last_change":"2024-09-20T22:16:20.370000+0000","last_active":"2024-09-20T22:17:21.981281+0000","last_peered":"2024-09-20T22:17:21.981281+0000","last_clean":"2024-09-20T22:17:21.981281+0000","last_became_active":"2024-09-20T22:12:57.884257+0000","last_became_peered":"2024-09-20T22:12:57.884257+0000","last_unstale":"2024-09-20T22:17:21.981281+0000","last_undegraded":"2024-09-20T22:17:21.981281+0000","last_fullsized":"2024-09-20T22:17:21.981281+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:20.369864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:20.369864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:27.414575+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":797,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.024198+0000","last_change":"2024-09-20T22:17:06.944648+0000","last_active":"2024-09-20T22:17:26.024198+0000","last_peered":"2024-09-20T22:17:26.024198+0000","last_clean":"2024-09-20T22:17:26.024198+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:17:26.024198+0000","last_undegraded":"2024-09-20T22:17:26.024198+0000","last_fullsized":"2024-09-20T22:17:26.024198+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:17:06.944406+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_clean_scrub_stamp":"2024-09-20T22:17:06.944406+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:33.207021+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":229,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.025914+0000","last_change":"2024-09-20T22:15:54.083256+0000","last_active":"2024-09-20T22:17:26.025914+0000","last_peered":"2024-09-20T22:17:26.025914+0000","last_clean":"2024-09-20T22:17:26.025914+0000","last_became_active":"2024-09-20T22:12:52.527547+0000","last_became_peered":"2024-09-20T22:12:52.527547+0000","last_unstale":"2024-09-20T22:17:26.025914+0000","last_undegraded":"2024-09-20T22:17:26.025914+0000","last_fullsized":"2024-09-20T22:17:26.025914+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:54.082968+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:54.082968+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:26.984066+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":772,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.019704+0000","last_change":"2024-09-20T22:16:58.792938+0000","last_active":"2024-09-20T22:17:26.019704+0000","last_peered":"2024-09-20T22:17:26.019704+0000","last_clean":"2024-09-20T22:17:26.019704+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:17:26.019704+0000","last_undegraded":"2024-09-20T22:17:26.019704+0000","last_fullsized":"2024-09-20T22:17:26.019704+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:58.792800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:16:58.792800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:10.710603+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"424'10704","reported_seq":10823,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.298352+0000","last_change":"2024-09-20T22:15:08.931328+0000","last_active":"2024-09-20T22:17:26.298352+0000","last_peered":"2024-09-20T22:17:26.298352+0000","last_clean":"2024-09-20T22:17:26.298352+0000","last_became_active":"2024-09-20T22:15:08.930968+0000","last_became_peered":"2024-09-20T22:15:08.930968+0000","last_unstale":"2024-09-20T22:17:26.298352+0000","last_undegraded":"2024-09-20T22:17:26.298352+0000","last_fullsized":"2024-09-20T22:17:26.298352+0000","mapping_epoch":370,"log_start":"402'5800","ondisk_log_start":"402'5800","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":4904,"log_dups_size":0,"ondisk_log_size":4904,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.047422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":324665344,"num_objects":4954,"num_object_clones":0,"num_object_copies":14862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10704,"num_write_kb":501056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":802,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.988119+0000","last_change":"2024-09-20T22:15:46.548031+0000","last_active":"2024-09-20T22:17:21.988119+0000","last_peered":"2024-09-20T22:17:21.988119+0000","last_clean":"2024-09-20T22:17:21.988119+0000","last_became_active":"2024-09-20T22:15:45.620376+0000","last_became_peered":"2024-09-20T22:15:45.620376+0000","last_unstale":"2024-09-20T22:17:21.988119+0000","last_undegraded":"2024-09-20T22:17:21.988119+0000","last_fullsized":"2024-09-20T22:17:21.988119+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:46.547907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:46.547907+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"328'17","reported_seq":696,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.337394+0000","last_change":"2024-09-20T22:17:01.307957+0000","last_active":"2024-09-20T22:17:28.337394+0000","last_peered":"2024-09-20T22:17:28.337394+0000","last_clean":"2024-09-20T22:17:28.337394+0000","last_became_active":"2024-09-20T22:12:52.521073+0000","last_became_peered":"2024-09-20T22:12:52.521073+0000","last_unstale":"2024-09-20T22:17:28.337394+0000","last_undegraded":"2024-09-20T22:17:28.337394+0000","last_fullsized":"2024-09-20T22:17:28.337394+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'17","last_scrub_stamp":"2024-09-20T22:17:01.307755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:01.307755+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:19.260372+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":625,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.337194+0000","last_change":"2024-09-20T22:17:09.478913+0000","last_active":"2024-09-20T22:17:28.337194+0000","last_peered":"2024-09-20T22:17:28.337194+0000","last_clean":"2024-09-20T22:17:28.337194+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:17:28.337194+0000","last_undegraded":"2024-09-20T22:17:28.337194+0000","last_fullsized":"2024-09-20T22:17:28.337194+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:09.478390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:09.478390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:15.007402+0000","scrub_duration":51,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":839,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.025921+0000","last_change":"2024-09-20T22:15:48.955294+0000","last_active":"2024-09-20T22:17:26.025921+0000","last_peered":"2024-09-20T22:17:26.025921+0000","last_clean":"2024-09-20T22:17:26.025921+0000","last_became_active":"2024-09-20T22:12:52.517255+0000","last_became_peered":"2024-09-20T22:12:52.517255+0000","last_unstale":"2024-09-20T22:17:26.025921+0000","last_undegraded":"2024-09-20T22:17:26.025921+0000","last_fullsized":"2024-09-20T22:17:26.025921+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:15:48.955042+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:15:48.955042+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":800,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.026091+0000","last_change":"2024-09-20T22:17:08.832699+0000","last_active":"2024-09-20T22:17:26.026091+0000","last_peered":"2024-09-20T22:17:26.026091+0000","last_clean":"2024-09-20T22:17:26.026091+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:17:26.026091+0000","last_undegraded":"2024-09-20T22:17:26.026091+0000","last_fullsized":"2024-09-20T22:17:26.026091+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:08.832589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:17:08.832589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:14.550051+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":763,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.989299+0000","last_change":"2024-09-20T22:16:23.711184+0000","last_active":"2024-09-20T22:17:21.989299+0000","last_peered":"2024-09-20T22:17:21.989299+0000","last_clean":"2024-09-20T22:17:21.989299+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:17:21.989299+0000","last_undegraded":"2024-09-20T22:17:21.989299+0000","last_fullsized":"2024-09-20T22:17:21.989299+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:23.711072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:23.711072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:36.786537+0000","scrub_duration":87,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"424'10657","reported_seq":10851,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:27.300328+0000","last_change":"2024-09-20T22:16:32.694961+0000","last_active":"2024-09-20T22:17:27.300328+0000","last_peered":"2024-09-20T22:17:27.300328+0000","last_clean":"2024-09-20T22:17:27.300328+0000","last_became_active":"2024-09-20T22:15:08.926702+0000","last_became_peered":"2024-09-20T22:15:08.926702+0000","last_unstale":"2024-09-20T22:17:27.300328+0000","last_undegraded":"2024-09-20T22:17:27.300328+0000","last_fullsized":"2024-09-20T22:17:27.300328+0000","mapping_epoch":370,"log_start":"401'5600","ondisk_log_start":"401'5600","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6715","last_scrub_stamp":"2024-09-20T22:16:32.694684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.694684+0000","objects_scrubbed":6639,"log_size":5057,"log_dups_size":0,"ondisk_log_size":5057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:00.250998+0000","scrub_duration":2750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":317521920,"num_objects":4845,"num_object_clones":0,"num_object_copies":14535,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10657,"num_write_kb":496064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":790,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.341068+0000","last_change":"2024-09-20T22:15:46.338840+0000","last_active":"2024-09-20T22:17:28.341068+0000","last_peered":"2024-09-20T22:17:28.341068+0000","last_clean":"2024-09-20T22:17:28.341068+0000","last_became_active":"2024-09-20T22:15:45.630804+0000","last_became_peered":"2024-09-20T22:15:45.630804+0000","last_unstale":"2024-09-20T22:17:28.341068+0000","last_undegraded":"2024-09-20T22:17:28.341068+0000","last_fullsized":"2024-09-20T22:17:28.341068+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:46.338700+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:15:46.338700+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.477153+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"328'24","reported_seq":269,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.028574+0000","last_change":"2024-09-20T22:17:05.822751+0000","last_active":"2024-09-20T22:17:26.028574+0000","last_peered":"2024-09-20T22:17:26.028574+0000","last_clean":"2024-09-20T22:17:26.028574+0000","last_became_active":"2024-09-20T22:12:52.527919+0000","last_became_peered":"2024-09-20T22:12:52.527919+0000","last_unstale":"2024-09-20T22:17:26.028574+0000","last_undegraded":"2024-09-20T22:17:26.028574+0000","last_fullsized":"2024-09-20T22:17:26.028574+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'24","last_scrub_stamp":"2024-09-20T22:17:05.822650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:05.822650+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:27.748220+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":243,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.024900+0000","last_change":"2024-09-20T22:17:11.814549+0000","last_active":"2024-09-20T22:17:26.024900+0000","last_peered":"2024-09-20T22:17:26.024900+0000","last_clean":"2024-09-20T22:17:26.024900+0000","last_became_active":"2024-09-20T22:17:03.663870+0000","last_became_peered":"2024-09-20T22:17:03.663870+0000","last_unstale":"2024-09-20T22:17:26.024900+0000","last_undegraded":"2024-09-20T22:17:26.024900+0000","last_fullsized":"2024-09-20T22:17:26.024900+0000","mapping_epoch":415,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":416,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:11.814439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:17:11.814439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:27.478502+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":1674,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:23.987753+0000","last_change":"2024-09-20T22:15:55.652986+0000","last_active":"2024-09-20T22:17:23.987753+0000","last_peered":"2024-09-20T22:17:23.987753+0000","last_clean":"2024-09-20T22:17:23.987753+0000","last_became_active":"2024-09-20T22:12:52.526622+0000","last_became_peered":"2024-09-20T22:12:52.526622+0000","last_unstale":"2024-09-20T22:17:23.987753+0000","last_undegraded":"2024-09-20T22:17:23.987753+0000","last_fullsized":"2024-09-20T22:17:23.987753+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:15:55.652860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:15:55.652860+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.795821+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":851,"num_read_kb":846,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":331,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.023699+0000","last_change":"2024-09-20T22:17:08.955894+0000","last_active":"2024-09-20T22:17:26.023699+0000","last_peered":"2024-09-20T22:17:26.023699+0000","last_clean":"2024-09-20T22:17:26.023699+0000","last_became_active":"2024-09-20T22:12:52.518823+0000","last_became_peered":"2024-09-20T22:12:52.518823+0000","last_unstale":"2024-09-20T22:17:26.023699+0000","last_undegraded":"2024-09-20T22:17:26.023699+0000","last_fullsized":"2024-09-20T22:17:26.023699+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:08.955790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:17:08.955790+0000","last_clean_scrub_stamp":"2024-09-20T22:17:08.955790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:18.057717+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":617,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.337340+0000","last_change":"2024-09-20T22:16:26.330409+0000","last_active":"2024-09-20T22:17:28.337340+0000","last_peered":"2024-09-20T22:17:28.337340+0000","last_clean":"2024-09-20T22:17:28.337340+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:17:28.337340+0000","last_undegraded":"2024-09-20T22:17:28.337340+0000","last_fullsized":"2024-09-20T22:17:28.337340+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:26.330270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:26.330270+0000","last_clean_scrub_stamp":"2024-09-20T22:16:26.330270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:32.399118+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"424'10848","reported_seq":11044,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.112258+0000","last_change":"2024-09-20T22:17:01.821597+0000","last_active":"2024-09-20T22:17:28.112258+0000","last_peered":"2024-09-20T22:17:28.112258+0000","last_clean":"2024-09-20T22:17:28.112258+0000","last_became_active":"2024-09-20T22:15:08.924809+0000","last_became_peered":"2024-09-20T22:15:08.924809+0000","last_unstale":"2024-09-20T22:17:28.112258+0000","last_undegraded":"2024-09-20T22:17:28.112258+0000","last_fullsized":"2024-09-20T22:17:28.112258+0000","mapping_epoch":370,"log_start":"403'5900","ondisk_log_start":"403'5900","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"414'8932","last_scrub_stamp":"2024-09-20T22:17:01.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:17:01.821270+0000","objects_scrubbed":6888,"log_size":4948,"log_dups_size":0,"ondisk_log_size":4948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:07.401461+0000","scrub_duration":2077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":321519616,"num_objects":4906,"num_object_clones":0,"num_object_copies":14718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4906,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10848,"num_write_kb":504128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"328'28","reported_seq":1004,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.020916+0000","last_change":"2024-09-20T22:15:55.704801+0000","last_active":"2024-09-20T22:17:26.020916+0000","last_peered":"2024-09-20T22:17:26.020916+0000","last_clean":"2024-09-20T22:17:26.020916+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:17:26.020916+0000","last_undegraded":"2024-09-20T22:17:26.020916+0000","last_fullsized":"2024-09-20T22:17:26.020916+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:15:55.704666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:55.704666+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:28.855209+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":737,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.023548+0000","last_change":"2024-09-20T22:16:36.729250+0000","last_active":"2024-09-20T22:17:26.023548+0000","last_peered":"2024-09-20T22:17:26.023548+0000","last_clean":"2024-09-20T22:17:26.023548+0000","last_became_active":"2024-09-20T22:12:21.519717+0000","last_became_peered":"2024-09-20T22:12:21.519717+0000","last_unstale":"2024-09-20T22:17:26.023548+0000","last_undegraded":"2024-09-20T22:17:26.023548+0000","last_fullsized":"2024-09-20T22:17:26.023548+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:36.729158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:36.729158+0000","last_clean_scrub_stamp":"2024-09-20T22:16:36.729158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:53.476434+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":802,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.988116+0000","last_change":"2024-09-20T22:15:47.554095+0000","last_active":"2024-09-20T22:17:21.988116+0000","last_peered":"2024-09-20T22:17:21.988116+0000","last_clean":"2024-09-20T22:17:21.988116+0000","last_became_active":"2024-09-20T22:15:45.620972+0000","last_became_peered":"2024-09-20T22:15:45.620972+0000","last_unstale":"2024-09-20T22:17:21.988116+0000","last_undegraded":"2024-09-20T22:17:21.988116+0000","last_fullsized":"2024-09-20T22:17:21.988116+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:47.553962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:47.553962+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":358,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.989623+0000","last_change":"2024-09-20T22:15:58.617583+0000","last_active":"2024-09-20T22:17:21.989623+0000","last_peered":"2024-09-20T22:17:21.989623+0000","last_clean":"2024-09-20T22:17:21.989623+0000","last_became_active":"2024-09-20T22:10:48.907271+0000","last_became_peered":"2024-09-20T22:10:48.907271+0000","last_unstale":"2024-09-20T22:17:21.989623+0000","last_undegraded":"2024-09-20T22:17:21.989623+0000","last_fullsized":"2024-09-20T22:17:21.989623+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:58.617484+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_clean_scrub_stamp":"2024-09-20T22:15:58.617484+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":792,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.023311+0000","last_change":"2024-09-20T22:17:02.907969+0000","last_active":"2024-09-20T22:17:26.023311+0000","last_peered":"2024-09-20T22:17:26.023311+0000","last_clean":"2024-09-20T22:17:26.023311+0000","last_became_active":"2024-09-20T22:12:44.630044+0000","last_became_peered":"2024-09-20T22:12:44.630044+0000","last_unstale":"2024-09-20T22:17:26.023311+0000","last_undegraded":"2024-09-20T22:17:26.023311+0000","last_fullsized":"2024-09-20T22:17:26.023311+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:02.907851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:17:02.907851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:09.766067+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"75'128","reported_seq":797,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.337146+0000","last_change":"2024-09-20T22:15:51.314327+0000","last_active":"2024-09-20T22:17:28.337146+0000","last_peered":"2024-09-20T22:17:28.337146+0000","last_clean":"2024-09-20T22:17:28.337146+0000","last_became_active":"2024-09-20T22:15:50.615076+0000","last_became_peered":"2024-09-20T22:15:50.615076+0000","last_unstale":"2024-09-20T22:17:28.337146+0000","last_undegraded":"2024-09-20T22:17:28.337146+0000","last_fullsized":"2024-09-20T22:17:28.337146+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:51.314232+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:15:51.314232+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.477257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"424'10989","reported_seq":11238,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.094671+0000","last_change":"2024-09-20T22:16:58.807828+0000","last_active":"2024-09-20T22:17:28.094671+0000","last_peered":"2024-09-20T22:17:28.094671+0000","last_clean":"2024-09-20T22:17:28.094671+0000","last_became_active":"2024-09-20T22:15:08.925040+0000","last_became_peered":"2024-09-20T22:15:08.925040+0000","last_unstale":"2024-09-20T22:17:28.094671+0000","last_undegraded":"2024-09-20T22:17:28.094671+0000","last_fullsized":"2024-09-20T22:17:28.094671+0000","mapping_epoch":370,"log_start":"403'6000","ondisk_log_start":"403'6000","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"414'8778","last_scrub_stamp":"2024-09-20T22:16:58.807519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:58.807519+0000","objects_scrubbed":7101,"log_size":4989,"log_dups_size":0,"ondisk_log_size":4989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:03.076101+0000","scrub_duration":2051,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":315686912,"num_objects":4817,"num_object_clones":0,"num_object_copies":14451,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10989,"num_write_kb":505792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"328'19","reported_seq":811,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.981686+0000","last_change":"2024-09-20T22:16:24.435555+0000","last_active":"2024-09-20T22:17:21.981686+0000","last_peered":"2024-09-20T22:17:21.981686+0000","last_clean":"2024-09-20T22:17:21.981686+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:17:21.981686+0000","last_undegraded":"2024-09-20T22:17:21.981686+0000","last_fullsized":"2024-09-20T22:17:21.981686+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:16:24.435260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:24.435260+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:26.710702+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":775,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.983178+0000","last_change":"2024-09-20T22:15:27.077414+0000","last_active":"2024-09-20T22:17:21.983178+0000","last_peered":"2024-09-20T22:17:21.983178+0000","last_clean":"2024-09-20T22:17:21.983178+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:17:21.983178+0000","last_undegraded":"2024-09-20T22:17:21.983178+0000","last_fullsized":"2024-09-20T22:17:21.983178+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:27.077295+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.077295+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:23.492139+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":776,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.023108+0000","last_change":"2024-09-20T22:15:24.150401+0000","last_active":"2024-09-20T22:17:26.023108+0000","last_peered":"2024-09-20T22:17:26.023108+0000","last_clean":"2024-09-20T22:17:26.023108+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:17:26.023108+0000","last_undegraded":"2024-09-20T22:17:26.023108+0000","last_fullsized":"2024-09-20T22:17:26.023108+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:24.150265+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:24.150265+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.047448+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":732,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.020485+0000","last_change":"2024-09-20T22:15:52.643084+0000","last_active":"2024-09-20T22:17:26.020485+0000","last_peered":"2024-09-20T22:17:26.020485+0000","last_clean":"2024-09-20T22:17:26.020485+0000","last_became_active":"2024-09-20T22:12:21.521195+0000","last_became_peered":"2024-09-20T22:12:21.521195+0000","last_unstale":"2024-09-20T22:17:26.020485+0000","last_undegraded":"2024-09-20T22:17:26.020485+0000","last_fullsized":"2024-09-20T22:17:26.020485+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:52.642911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:52.642911+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:28.855181+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3","version":"424'10679","reported_seq":10874,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.396492+0000","last_change":"2024-09-20T22:16:32.632993+0000","last_active":"2024-09-20T22:17:26.396492+0000","last_peered":"2024-09-20T22:17:26.396492+0000","last_clean":"2024-09-20T22:17:26.396492+0000","last_became_active":"2024-09-20T22:15:09.087061+0000","last_became_peered":"2024-09-20T22:15:09.087061+0000","last_unstale":"2024-09-20T22:17:26.396492+0000","last_undegraded":"2024-09-20T22:17:26.396492+0000","last_fullsized":"2024-09-20T22:17:26.396492+0000","mapping_epoch":370,"log_start":"396'4950","ondisk_log_start":"396'4950","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6740","last_scrub_stamp":"2024-09-20T22:16:32.632756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.632756+0000","objects_scrubbed":6652,"log_size":5729,"log_dups_size":0,"ondisk_log_size":5729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:35.823889+0000","scrub_duration":2683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":322633728,"num_objects":4923,"num_object_clones":0,"num_object_copies":14769,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4923,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10679,"num_write_kb":499264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":550,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.019998+0000","last_change":"2024-09-20T22:16:18.773405+0000","last_active":"2024-09-20T22:17:26.019998+0000","last_peered":"2024-09-20T22:17:26.019998+0000","last_clean":"2024-09-20T22:17:26.019998+0000","last_became_active":"2024-09-20T22:14:38.674758+0000","last_became_peered":"2024-09-20T22:14:38.674758+0000","last_unstale":"2024-09-20T22:17:26.019998+0000","last_undegraded":"2024-09-20T22:17:26.019998+0000","last_fullsized":"2024-09-20T22:17:26.019998+0000","mapping_epoch":358,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":359,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:16:18.773201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:16:18.773201+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:39.013289+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"328'31","reported_seq":837,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.984241+0000","last_change":"2024-09-20T22:16:17.388362+0000","last_active":"2024-09-20T22:17:21.984241+0000","last_peered":"2024-09-20T22:17:21.984241+0000","last_clean":"2024-09-20T22:17:21.984241+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:17:21.984241+0000","last_undegraded":"2024-09-20T22:17:21.984241+0000","last_fullsized":"2024-09-20T22:17:21.984241+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'31","last_scrub_stamp":"2024-09-20T22:16:17.387895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:17.387895+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:42.903740+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":801,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.020605+0000","last_change":"2024-09-20T22:17:08.847011+0000","last_active":"2024-09-20T22:17:26.020605+0000","last_peered":"2024-09-20T22:17:26.020605+0000","last_clean":"2024-09-20T22:17:26.020605+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:17:26.020605+0000","last_undegraded":"2024-09-20T22:17:26.020605+0000","last_fullsized":"2024-09-20T22:17:26.020605+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:08.846904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:17:08.846904+0000","last_clean_scrub_stamp":"2024-09-20T22:17:08.846904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:23.342530+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":628,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.337439+0000","last_change":"2024-09-20T22:15:57.442451+0000","last_active":"2024-09-20T22:17:28.337439+0000","last_peered":"2024-09-20T22:17:28.337439+0000","last_clean":"2024-09-20T22:17:28.337439+0000","last_became_active":"2024-09-20T22:12:52.520612+0000","last_became_peered":"2024-09-20T22:12:52.520612+0000","last_unstale":"2024-09-20T22:17:28.337439+0000","last_undegraded":"2024-09-20T22:17:28.337439+0000","last_fullsized":"2024-09-20T22:17:28.337439+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:57.442341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.442341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.477183+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":788,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.024419+0000","last_change":"2024-09-20T22:15:56.941775+0000","last_active":"2024-09-20T22:17:26.024419+0000","last_peered":"2024-09-20T22:17:26.024419+0000","last_clean":"2024-09-20T22:17:26.024419+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:17:26.024419+0000","last_undegraded":"2024-09-20T22:17:26.024419+0000","last_fullsized":"2024-09-20T22:17:26.024419+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:56.941427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:56.941427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"425'10750","reported_seq":10925,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.435700+0000","last_change":"2024-09-20T22:17:12.039317+0000","last_active":"2024-09-20T22:17:28.435700+0000","last_peered":"2024-09-20T22:17:28.435700+0000","last_clean":"2024-09-20T22:17:28.435700+0000","last_became_active":"2024-09-20T22:15:08.926083+0000","last_became_peered":"2024-09-20T22:15:08.926083+0000","last_unstale":"2024-09-20T22:17:28.435700+0000","last_undegraded":"2024-09-20T22:17:28.435700+0000","last_fullsized":"2024-09-20T22:17:28.435700+0000","mapping_epoch":370,"log_start":"404'6054","ondisk_log_start":"404'6054","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":4949,"log_size":4696,"log_dups_size":0,"ondisk_log_size":4696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.477043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":313917440,"num_objects":4790,"num_object_clones":0,"num_object_copies":14370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10750,"num_write_kb":497280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":1059,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.991334+0000","last_change":"2024-09-20T22:15:56.638628+0000","last_active":"2024-09-20T22:17:21.991334+0000","last_peered":"2024-09-20T22:17:21.991334+0000","last_clean":"2024-09-20T22:17:21.991334+0000","last_became_active":"2024-09-20T22:08:48.455288+0000","last_became_peered":"2024-09-20T22:08:48.455288+0000","last_unstale":"2024-09-20T22:17:21.991334+0000","last_undegraded":"2024-09-20T22:17:21.991334+0000","last_fullsized":"2024-09-20T22:17:21.991334+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":203,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:56.638414+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:56.638414+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.795893+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"328'25","reported_seq":806,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.982347+0000","last_change":"2024-09-20T22:15:50.223866+0000","last_active":"2024-09-20T22:17:21.982347+0000","last_peered":"2024-09-20T22:17:21.982347+0000","last_clean":"2024-09-20T22:17:21.982347+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:17:21.982347+0000","last_undegraded":"2024-09-20T22:17:21.982347+0000","last_fullsized":"2024-09-20T22:17:21.982347+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'25","last_scrub_stamp":"2024-09-20T22:15:50.223785+0000","last_deep_scrub":"289'15","last_deep_scrub_stamp":"2024-09-20T22:11:37.277648+0000","last_clean_scrub_stamp":"2024-09-20T22:15:50.223785+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:23.492310+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":233,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.031274+0000","last_change":"2024-09-20T22:15:30.821251+0000","last_active":"2024-09-20T22:17:26.031274+0000","last_peered":"2024-09-20T22:17:26.031274+0000","last_clean":"2024-09-20T22:17:26.031274+0000","last_became_active":"2024-09-20T22:12:52.528310+0000","last_became_peered":"2024-09-20T22:12:52.528310+0000","last_unstale":"2024-09-20T22:17:26.031274+0000","last_undegraded":"2024-09-20T22:17:26.031274+0000","last_fullsized":"2024-09-20T22:17:26.031274+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:15:30.821142+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:15:30.821142+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:26.983985+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"283'11","reported_seq":1005,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.024802+0000","last_change":"2024-09-20T22:15:49.973513+0000","last_active":"2024-09-20T22:17:26.024802+0000","last_peered":"2024-09-20T22:17:26.024802+0000","last_clean":"2024-09-20T22:17:26.024802+0000","last_became_active":"2024-09-20T22:11:30.485356+0000","last_became_peered":"2024-09-20T22:11:30.485356+0000","last_unstale":"2024-09-20T22:17:26.024802+0000","last_undegraded":"2024-09-20T22:17:26.024802+0000","last_fullsized":"2024-09-20T22:17:26.024802+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'11","last_scrub_stamp":"2024-09-20T22:15:49.973229+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:15:49.973229+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":772,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.981376+0000","last_change":"2024-09-20T22:16:27.486633+0000","last_active":"2024-09-20T22:17:21.981376+0000","last_peered":"2024-09-20T22:17:21.981376+0000","last_clean":"2024-09-20T22:17:21.981376+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:17:21.981376+0000","last_undegraded":"2024-09-20T22:17:21.981376+0000","last_fullsized":"2024-09-20T22:17:21.981376+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:27.486520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:16:27.486520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:41.356713+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"75'128","reported_seq":664,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.981106+0000","last_change":"2024-09-20T22:15:51.252861+0000","last_active":"2024-09-20T22:17:21.981106+0000","last_peered":"2024-09-20T22:17:21.981106+0000","last_clean":"2024-09-20T22:17:21.981106+0000","last_became_active":"2024-09-20T22:15:50.594320+0000","last_became_peered":"2024-09-20T22:15:50.594320+0000","last_unstale":"2024-09-20T22:17:21.981106+0000","last_undegraded":"2024-09-20T22:17:21.981106+0000","last_fullsized":"2024-09-20T22:17:21.981106+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:51.252685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:51.252685+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:23.492198+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"424'10651","reported_seq":10838,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.392059+0000","last_change":"2024-09-20T22:16:21.665425+0000","last_active":"2024-09-20T22:17:26.392059+0000","last_peered":"2024-09-20T22:17:26.392059+0000","last_clean":"2024-09-20T22:17:26.392059+0000","last_became_active":"2024-09-20T22:15:09.087830+0000","last_became_peered":"2024-09-20T22:15:09.087830+0000","last_unstale":"2024-09-20T22:17:26.392059+0000","last_undegraded":"2024-09-20T22:17:26.392059+0000","last_fullsized":"2024-09-20T22:17:26.392059+0000","mapping_epoch":370,"log_start":"396'4948","ondisk_log_start":"396'4948","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"403'6042","last_scrub_stamp":"2024-09-20T22:16:21.665289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:21.665289+0000","objects_scrubbed":6017,"log_size":5703,"log_dups_size":0,"ondisk_log_size":5703,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1845,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330629120,"num_objects":5045,"num_object_clones":0,"num_object_copies":15135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10651,"num_write_kb":502272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"328'15","reported_seq":256,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.031446+0000","last_change":"2024-09-20T22:16:59.741092+0000","last_active":"2024-09-20T22:17:26.031446+0000","last_peered":"2024-09-20T22:17:26.031446+0000","last_clean":"2024-09-20T22:17:26.031446+0000","last_became_active":"2024-09-20T22:12:52.527531+0000","last_became_peered":"2024-09-20T22:12:52.527531+0000","last_unstale":"2024-09-20T22:17:26.031446+0000","last_undegraded":"2024-09-20T22:17:26.031446+0000","last_fullsized":"2024-09-20T22:17:26.031446+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:16:59.740962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:59.740962+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:24.664014+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":413,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.020277+0000","last_change":"2024-09-20T22:16:29.707733+0000","last_active":"2024-09-20T22:17:26.020277+0000","last_peered":"2024-09-20T22:17:26.020277+0000","last_clean":"2024-09-20T22:17:26.020277+0000","last_became_active":"2024-09-20T22:10:14.536155+0000","last_became_peered":"2024-09-20T22:10:14.536155+0000","last_unstale":"2024-09-20T22:17:26.020277+0000","last_undegraded":"2024-09-20T22:17:26.020277+0000","last_fullsized":"2024-09-20T22:17:26.020277+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:16:29.707441+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:29.707441+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:45.157085+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":800,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.024235+0000","last_change":"2024-09-20T22:16:33.008048+0000","last_active":"2024-09-20T22:17:26.024235+0000","last_peered":"2024-09-20T22:17:26.024235+0000","last_clean":"2024-09-20T22:17:26.024235+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:17:26.024235+0000","last_undegraded":"2024-09-20T22:17:26.024235+0000","last_fullsized":"2024-09-20T22:17:26.024235+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:33.007962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:16:33.007962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:56.423160+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":229,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.024991+0000","last_change":"2024-09-20T22:16:24.020680+0000","last_active":"2024-09-20T22:17:26.024991+0000","last_peered":"2024-09-20T22:17:26.024991+0000","last_clean":"2024-09-20T22:17:26.024991+0000","last_became_active":"2024-09-20T22:12:52.512924+0000","last_became_peered":"2024-09-20T22:12:52.512924+0000","last_unstale":"2024-09-20T22:17:26.024991+0000","last_undegraded":"2024-09-20T22:17:26.024991+0000","last_fullsized":"2024-09-20T22:17:26.024991+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:24.020573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:24.020573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:44.565343+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"75'128","reported_seq":834,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.021413+0000","last_change":"2024-09-20T22:17:07.885447+0000","last_active":"2024-09-20T22:17:26.021413+0000","last_peered":"2024-09-20T22:17:26.021413+0000","last_clean":"2024-09-20T22:17:26.021413+0000","last_became_active":"2024-09-20T22:15:50.576540+0000","last_became_peered":"2024-09-20T22:15:50.576540+0000","last_unstale":"2024-09-20T22:17:26.021413+0000","last_undegraded":"2024-09-20T22:17:26.021413+0000","last_fullsized":"2024-09-20T22:17:26.021413+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:07.885346+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:17:07.885346+0000","last_clean_scrub_stamp":"2024-09-20T22:17:07.885346+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:17.857744+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.4","version":"424'10719","reported_seq":10912,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:27.295829+0000","last_change":"2024-09-20T22:16:31.859517+0000","last_active":"2024-09-20T22:17:27.295829+0000","last_peered":"2024-09-20T22:17:27.295829+0000","last_clean":"2024-09-20T22:17:27.295829+0000","last_became_active":"2024-09-20T22:15:09.087298+0000","last_became_peered":"2024-09-20T22:15:09.087298+0000","last_unstale":"2024-09-20T22:17:27.295829+0000","last_undegraded":"2024-09-20T22:17:27.295829+0000","last_fullsized":"2024-09-20T22:17:27.295829+0000","mapping_epoch":370,"log_start":"402'5700","ondisk_log_start":"402'5700","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6582","last_scrub_stamp":"2024-09-20T22:16:31.859305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.859305+0000","objects_scrubbed":6495,"log_size":5019,"log_dups_size":0,"ondisk_log_size":5019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:58.626512+0000","scrub_duration":2899,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":318832640,"num_objects":4865,"num_object_clones":0,"num_object_copies":14595,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10719,"num_write_kb":498688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"328'35","reported_seq":896,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.020349+0000","last_change":"2024-09-20T22:17:03.808976+0000","last_active":"2024-09-20T22:17:26.020349+0000","last_peered":"2024-09-20T22:17:26.020349+0000","last_clean":"2024-09-20T22:17:26.020349+0000","last_became_active":"2024-09-20T22:10:30.652967+0000","last_became_peered":"2024-09-20T22:10:30.652967+0000","last_unstale":"2024-09-20T22:17:26.020349+0000","last_undegraded":"2024-09-20T22:17:26.020349+0000","last_fullsized":"2024-09-20T22:17:26.020349+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'35","last_scrub_stamp":"2024-09-20T22:17:03.808887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:03.808887+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:27.160343+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":787,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.989566+0000","last_change":"2024-09-20T22:15:23.351913+0000","last_active":"2024-09-20T22:17:21.989566+0000","last_peered":"2024-09-20T22:17:21.989566+0000","last_clean":"2024-09-20T22:17:21.989566+0000","last_became_active":"2024-09-20T22:11:30.488650+0000","last_became_peered":"2024-09-20T22:11:30.488650+0000","last_unstale":"2024-09-20T22:17:21.989566+0000","last_undegraded":"2024-09-20T22:17:21.989566+0000","last_fullsized":"2024-09-20T22:17:21.989566+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:23.351789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:12:46.602035+0000","last_clean_scrub_stamp":"2024-09-20T22:15:23.351789+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":824,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.024174+0000","last_change":"2024-09-20T22:16:18.795310+0000","last_active":"2024-09-20T22:17:26.024174+0000","last_peered":"2024-09-20T22:17:26.024174+0000","last_clean":"2024-09-20T22:17:26.024174+0000","last_became_active":"2024-09-20T22:12:52.515295+0000","last_became_peered":"2024-09-20T22:12:52.515295+0000","last_unstale":"2024-09-20T22:17:26.024174+0000","last_undegraded":"2024-09-20T22:17:26.024174+0000","last_fullsized":"2024-09-20T22:17:26.024174+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:18.795079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_clean_scrub_stamp":"2024-09-20T22:16:18.795079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:36.635490+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":380,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.988164+0000","last_change":"2024-09-20T22:16:35.739421+0000","last_active":"2024-09-20T22:17:21.988164+0000","last_peered":"2024-09-20T22:17:21.988164+0000","last_clean":"2024-09-20T22:17:21.988164+0000","last_became_active":"2024-09-20T22:10:30.657116+0000","last_became_peered":"2024-09-20T22:10:30.657116+0000","last_unstale":"2024-09-20T22:17:21.988164+0000","last_undegraded":"2024-09-20T22:17:21.988164+0000","last_fullsized":"2024-09-20T22:17:21.988164+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:35.739168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:35.739168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:45.935247+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"75'128","reported_seq":836,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.019930+0000","last_change":"2024-09-20T22:15:50.922162+0000","last_active":"2024-09-20T22:17:26.019930+0000","last_peered":"2024-09-20T22:17:26.019930+0000","last_clean":"2024-09-20T22:17:26.019930+0000","last_became_active":"2024-09-20T22:15:50.619935+0000","last_became_peered":"2024-09-20T22:15:50.619935+0000","last_unstale":"2024-09-20T22:17:26.019930+0000","last_undegraded":"2024-09-20T22:17:26.019930+0000","last_fullsized":"2024-09-20T22:17:26.019930+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:50.922028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:50.922028+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:28.714639+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.7","version":"424'10778","reported_seq":10972,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.128992+0000","last_change":"2024-09-20T22:16:29.896220+0000","last_active":"2024-09-20T22:17:28.128992+0000","last_peered":"2024-09-20T22:17:28.128992+0000","last_clean":"2024-09-20T22:17:28.128992+0000","last_became_active":"2024-09-20T22:15:09.088590+0000","last_became_peered":"2024-09-20T22:15:09.088590+0000","last_unstale":"2024-09-20T22:17:28.128992+0000","last_undegraded":"2024-09-20T22:17:28.128992+0000","last_fullsized":"2024-09-20T22:17:28.128992+0000","mapping_epoch":370,"log_start":"402'5800","ondisk_log_start":"402'5800","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6589","last_scrub_stamp":"2024-09-20T22:16:29.896110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:29.896110+0000","objects_scrubbed":6527,"log_size":4978,"log_dups_size":0,"ondisk_log_size":4978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:39.515203+0000","scrub_duration":2191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":316801024,"num_objects":4834,"num_object_clones":0,"num_object_copies":14502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10778,"num_write_kb":499584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"418'23","reported_seq":817,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.982744+0000","last_change":"2024-09-20T22:17:11.339243+0000","last_active":"2024-09-20T22:17:21.982744+0000","last_peered":"2024-09-20T22:17:21.982744+0000","last_clean":"2024-09-20T22:17:21.982744+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:17:21.982744+0000","last_undegraded":"2024-09-20T22:17:21.982744+0000","last_fullsized":"2024-09-20T22:17:21.982744+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"418'23","last_scrub_stamp":"2024-09-20T22:17:11.338821+0000","last_deep_scrub":"418'23","last_deep_scrub_stamp":"2024-09-20T22:17:11.338821+0000","last_clean_scrub_stamp":"2024-09-20T22:17:11.338821+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:25.121498+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":784,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.989626+0000","last_change":"2024-09-20T22:16:25.649600+0000","last_active":"2024-09-20T22:17:21.989626+0000","last_peered":"2024-09-20T22:17:21.989626+0000","last_clean":"2024-09-20T22:17:21.989626+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:17:21.989626+0000","last_undegraded":"2024-09-20T22:17:21.989626+0000","last_fullsized":"2024-09-20T22:17:21.989626+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:25.649511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:25.649511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:42.623417+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":821,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.027459+0000","last_change":"2024-09-20T22:16:59.037833+0000","last_active":"2024-09-20T22:17:26.027459+0000","last_peered":"2024-09-20T22:17:26.027459+0000","last_clean":"2024-09-20T22:17:26.027459+0000","last_became_active":"2024-09-20T22:12:52.514202+0000","last_became_peered":"2024-09-20T22:12:52.514202+0000","last_unstale":"2024-09-20T22:17:26.027459+0000","last_undegraded":"2024-09-20T22:17:26.027459+0000","last_fullsized":"2024-09-20T22:17:26.027459+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:59.036713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:59.036713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:10.054093+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":247,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.014321+0000","last_change":"2024-09-20T22:16:57.795001+0000","last_active":"2024-09-20T22:17:26.014321+0000","last_peered":"2024-09-20T22:17:26.014321+0000","last_clean":"2024-09-20T22:17:26.014321+0000","last_became_active":"2024-09-20T22:12:44.638546+0000","last_became_peered":"2024-09-20T22:12:44.638546+0000","last_unstale":"2024-09-20T22:17:26.014321+0000","last_undegraded":"2024-09-20T22:17:26.014321+0000","last_fullsized":"2024-09-20T22:17:26.014321+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:57.794891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:57.794891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:26.253847+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'128","reported_seq":380,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.989093+0000","last_change":"2024-09-20T22:15:51.552197+0000","last_active":"2024-09-20T22:17:21.989093+0000","last_peered":"2024-09-20T22:17:21.989093+0000","last_clean":"2024-09-20T22:17:21.989093+0000","last_became_active":"2024-09-20T22:15:50.594241+0000","last_became_peered":"2024-09-20T22:15:50.594241+0000","last_unstale":"2024-09-20T22:17:21.989093+0000","last_undegraded":"2024-09-20T22:17:21.989093+0000","last_fullsized":"2024-09-20T22:17:21.989093+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:51.552071+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:15:51.552071+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.6","version":"424'10662","reported_seq":10781,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.296295+0000","last_change":"2024-09-20T22:15:08.928417+0000","last_active":"2024-09-20T22:17:26.296295+0000","last_peered":"2024-09-20T22:17:26.296295+0000","last_clean":"2024-09-20T22:17:26.296295+0000","last_became_active":"2024-09-20T22:15:08.928102+0000","last_became_peered":"2024-09-20T22:15:08.928102+0000","last_unstale":"2024-09-20T22:17:26.296295+0000","last_undegraded":"2024-09-20T22:17:26.296295+0000","last_fullsized":"2024-09-20T22:17:26.296295+0000","mapping_epoch":370,"log_start":"401'5730","ondisk_log_start":"401'5730","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:15:07.911431+0000","objects_scrubbed":0,"log_size":4932,"log_dups_size":0,"ondisk_log_size":4932,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.047367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":335413248,"num_objects":5118,"num_object_clones":0,"num_object_copies":15354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10662,"num_write_kb":504960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"416'34","reported_seq":352,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.022762+0000","last_change":"2024-09-20T22:17:04.960544+0000","last_active":"2024-09-20T22:17:26.022762+0000","last_peered":"2024-09-20T22:17:26.022762+0000","last_clean":"2024-09-20T22:17:26.022762+0000","last_became_active":"2024-09-20T22:11:30.493275+0000","last_became_peered":"2024-09-20T22:11:30.493275+0000","last_unstale":"2024-09-20T22:17:26.022762+0000","last_undegraded":"2024-09-20T22:17:26.022762+0000","last_fullsized":"2024-09-20T22:17:26.022762+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"416'34","last_scrub_stamp":"2024-09-20T22:17:04.960380+0000","last_deep_scrub":"416'34","last_deep_scrub_stamp":"2024-09-20T22:17:04.960380+0000","last_clean_scrub_stamp":"2024-09-20T22:17:04.960380+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:30.318376+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":410,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.988749+0000","last_change":"2024-09-20T22:16:37.713822+0000","last_active":"2024-09-20T22:17:21.988749+0000","last_peered":"2024-09-20T22:17:21.988749+0000","last_clean":"2024-09-20T22:17:21.988749+0000","last_became_active":"2024-09-20T22:12:52.526551+0000","last_became_peered":"2024-09-20T22:12:52.526551+0000","last_unstale":"2024-09-20T22:17:21.988749+0000","last_undegraded":"2024-09-20T22:17:21.988749+0000","last_fullsized":"2024-09-20T22:17:21.988749+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:16:37.713713+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:16:37.713713+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:58.859262+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":951,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.991077+0000","last_change":"2024-09-20T22:16:32.744150+0000","last_active":"2024-09-20T22:17:21.991077+0000","last_peered":"2024-09-20T22:17:21.991077+0000","last_clean":"2024-09-20T22:17:21.991077+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:17:21.991077+0000","last_undegraded":"2024-09-20T22:17:21.991077+0000","last_fullsized":"2024-09-20T22:17:21.991077+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:16:32.744031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.744031+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:47.355070+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":323,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.023981+0000","last_change":"2024-09-20T22:16:21.904789+0000","last_active":"2024-09-20T22:17:26.023981+0000","last_peered":"2024-09-20T22:17:26.023981+0000","last_clean":"2024-09-20T22:17:26.023981+0000","last_became_active":"2024-09-20T22:12:52.517758+0000","last_became_peered":"2024-09-20T22:12:52.517758+0000","last_unstale":"2024-09-20T22:17:26.023981+0000","last_undegraded":"2024-09-20T22:17:26.023981+0000","last_fullsized":"2024-09-20T22:17:26.023981+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:21.904548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:21.904548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:36.763352+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"75'128","reported_seq":659,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.023663+0000","last_change":"2024-09-20T22:15:52.087940+0000","last_active":"2024-09-20T22:17:26.023663+0000","last_peered":"2024-09-20T22:17:26.023663+0000","last_clean":"2024-09-20T22:17:26.023663+0000","last_became_active":"2024-09-20T22:15:50.622954+0000","last_became_peered":"2024-09-20T22:15:50.622954+0000","last_unstale":"2024-09-20T22:17:26.023663+0000","last_undegraded":"2024-09-20T22:17:26.023663+0000","last_fullsized":"2024-09-20T22:17:26.023663+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:52.087707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:52.087707+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.047698+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":824,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.020326+0000","last_change":"2024-09-20T22:16:23.783338+0000","last_active":"2024-09-20T22:17:26.020326+0000","last_peered":"2024-09-20T22:17:26.020326+0000","last_clean":"2024-09-20T22:17:26.020326+0000","last_became_active":"2024-09-20T22:12:52.516989+0000","last_became_peered":"2024-09-20T22:12:52.516989+0000","last_unstale":"2024-09-20T22:17:26.020326+0000","last_undegraded":"2024-09-20T22:17:26.020326+0000","last_fullsized":"2024-09-20T22:17:26.020326+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:23.783127+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:23.783127+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:27.538492+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"328'12","reported_seq":826,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.021580+0000","last_change":"2024-09-20T22:16:22.641814+0000","last_active":"2024-09-20T22:17:26.021580+0000","last_peered":"2024-09-20T22:17:26.021580+0000","last_clean":"2024-09-20T22:17:26.021580+0000","last_became_active":"2024-09-20T22:10:48.922561+0000","last_became_peered":"2024-09-20T22:10:48.922561+0000","last_unstale":"2024-09-20T22:17:26.021580+0000","last_undegraded":"2024-09-20T22:17:26.021580+0000","last_fullsized":"2024-09-20T22:17:26.021580+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'12","last_scrub_stamp":"2024-09-20T22:16:22.641558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:22.641558+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:50.501207+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":311,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.024281+0000","last_change":"2024-09-20T22:15:54.065141+0000","last_active":"2024-09-20T22:17:26.024281+0000","last_peered":"2024-09-20T22:17:26.024281+0000","last_clean":"2024-09-20T22:17:26.024281+0000","last_became_active":"2024-09-20T22:11:30.487322+0000","last_became_peered":"2024-09-20T22:11:30.487322+0000","last_unstale":"2024-09-20T22:17:26.024281+0000","last_undegraded":"2024-09-20T22:17:26.024281+0000","last_fullsized":"2024-09-20T22:17:26.024281+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:54.065026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:54.065026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.047608+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":763,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.989254+0000","last_change":"2024-09-20T22:17:02.792431+0000","last_active":"2024-09-20T22:17:21.989254+0000","last_peered":"2024-09-20T22:17:21.989254+0000","last_clean":"2024-09-20T22:17:21.989254+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:17:21.989254+0000","last_undegraded":"2024-09-20T22:17:21.989254+0000","last_fullsized":"2024-09-20T22:17:21.989254+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:02.792356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:54.359586+0000","last_clean_scrub_stamp":"2024-09-20T22:17:02.792356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:03.522124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"75'128","reported_seq":797,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.337031+0000","last_change":"2024-09-20T22:15:52.362195+0000","last_active":"2024-09-20T22:17:28.337031+0000","last_peered":"2024-09-20T22:17:28.337031+0000","last_clean":"2024-09-20T22:17:28.337031+0000","last_became_active":"2024-09-20T22:15:50.615607+0000","last_became_peered":"2024-09-20T22:15:50.615607+0000","last_unstale":"2024-09-20T22:17:28.337031+0000","last_undegraded":"2024-09-20T22:17:28.337031+0000","last_fullsized":"2024-09-20T22:17:28.337031+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:52.362111+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:15:52.362111+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.477126+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"328'15","reported_seq":247,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.032112+0000","last_change":"2024-09-20T22:15:59.144712+0000","last_active":"2024-09-20T22:17:26.032112+0000","last_peered":"2024-09-20T22:17:26.032112+0000","last_clean":"2024-09-20T22:17:26.032112+0000","last_became_active":"2024-09-20T22:12:52.527106+0000","last_became_peered":"2024-09-20T22:12:52.527106+0000","last_unstale":"2024-09-20T22:17:26.032112+0000","last_undegraded":"2024-09-20T22:17:26.032112+0000","last_fullsized":"2024-09-20T22:17:26.032112+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:15:59.144584+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:59.144584+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:26.984042+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":783,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.991566+0000","last_change":"2024-09-20T22:17:10.768641+0000","last_active":"2024-09-20T22:17:21.991566+0000","last_peered":"2024-09-20T22:17:21.991566+0000","last_clean":"2024-09-20T22:17:21.991566+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:17:21.991566+0000","last_undegraded":"2024-09-20T22:17:21.991566+0000","last_fullsized":"2024-09-20T22:17:21.991566+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:10.768513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:10.768513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:36.496048+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":417,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.337128+0000","last_change":"2024-09-20T22:16:17.270484+0000","last_active":"2024-09-20T22:17:28.337128+0000","last_peered":"2024-09-20T22:17:28.337128+0000","last_clean":"2024-09-20T22:17:28.337128+0000","last_became_active":"2024-09-20T22:12:33.545506+0000","last_became_peered":"2024-09-20T22:12:33.545506+0000","last_unstale":"2024-09-20T22:17:28.337128+0000","last_undegraded":"2024-09-20T22:17:28.337128+0000","last_fullsized":"2024-09-20T22:17:28.337128+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:17.270376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_clean_scrub_stamp":"2024-09-20T22:16:17.270376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.477207+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":796,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.024726+0000","last_change":"2024-09-20T22:16:26.862388+0000","last_active":"2024-09-20T22:17:26.024726+0000","last_peered":"2024-09-20T22:17:26.024726+0000","last_clean":"2024-09-20T22:17:26.024726+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:17:26.024726+0000","last_undegraded":"2024-09-20T22:17:26.024726+0000","last_fullsized":"2024-09-20T22:17:26.024726+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:26.862279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:26.862279+0000","last_clean_scrub_stamp":"2024-09-20T22:16:26.862279+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:33.809107+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"75'128","reported_seq":659,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.022601+0000","last_change":"2024-09-20T22:15:51.068931+0000","last_active":"2024-09-20T22:17:26.022601+0000","last_peered":"2024-09-20T22:17:26.022601+0000","last_clean":"2024-09-20T22:17:26.022601+0000","last_became_active":"2024-09-20T22:15:50.574865+0000","last_became_peered":"2024-09-20T22:15:50.574865+0000","last_unstale":"2024-09-20T22:17:26.022601+0000","last_undegraded":"2024-09-20T22:17:26.022601+0000","last_fullsized":"2024-09-20T22:17:26.022601+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:51.068807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:51.068807+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.047668+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"328'15","reported_seq":330,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.023298+0000","last_change":"2024-09-20T22:15:57.097701+0000","last_active":"2024-09-20T22:17:26.023298+0000","last_peered":"2024-09-20T22:17:26.023298+0000","last_clean":"2024-09-20T22:17:26.023298+0000","last_became_active":"2024-09-20T22:11:30.493335+0000","last_became_peered":"2024-09-20T22:11:30.493335+0000","last_unstale":"2024-09-20T22:17:26.023298+0000","last_undegraded":"2024-09-20T22:17:26.023298+0000","last_fullsized":"2024-09-20T22:17:26.023298+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:15:57.097563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.097563+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.047638+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":236,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.025814+0000","last_change":"2024-09-20T22:16:27.938805+0000","last_active":"2024-09-20T22:17:26.025814+0000","last_peered":"2024-09-20T22:17:26.025814+0000","last_clean":"2024-09-20T22:17:26.025814+0000","last_became_active":"2024-09-20T22:12:52.524149+0000","last_became_peered":"2024-09-20T22:12:52.524149+0000","last_unstale":"2024-09-20T22:17:26.025814+0000","last_undegraded":"2024-09-20T22:17:26.025814+0000","last_fullsized":"2024-09-20T22:17:26.025814+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:27.938677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:56.538825+0000","last_clean_scrub_stamp":"2024-09-20T22:16:27.938677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:44.662705+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"315'11","reported_seq":338,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.989963+0000","last_change":"2024-09-20T22:17:05.784312+0000","last_active":"2024-09-20T22:17:21.989963+0000","last_peered":"2024-09-20T22:17:21.989963+0000","last_clean":"2024-09-20T22:17:21.989963+0000","last_became_active":"2024-09-20T22:12:33.544098+0000","last_became_peered":"2024-09-20T22:12:33.544098+0000","last_unstale":"2024-09-20T22:17:21.989963+0000","last_undegraded":"2024-09-20T22:17:21.989963+0000","last_fullsized":"2024-09-20T22:17:21.989963+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:17:05.784212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:05.784212+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:31.462376+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":774,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.981513+0000","last_change":"2024-09-20T22:15:30.093649+0000","last_active":"2024-09-20T22:17:21.981513+0000","last_peered":"2024-09-20T22:17:21.981513+0000","last_clean":"2024-09-20T22:17:21.981513+0000","last_became_active":"2024-09-20T22:11:30.484731+0000","last_became_peered":"2024-09-20T22:11:30.484731+0000","last_unstale":"2024-09-20T22:17:21.981513+0000","last_undegraded":"2024-09-20T22:17:21.981513+0000","last_fullsized":"2024-09-20T22:17:21.981513+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:30.093548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:30.093548+0000","last_clean_scrub_stamp":"2024-09-20T22:15:30.093548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:23.492166+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"75'128","reported_seq":675,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.024603+0000","last_change":"2024-09-20T22:15:51.077313+0000","last_active":"2024-09-20T22:17:26.024603+0000","last_peered":"2024-09-20T22:17:26.024603+0000","last_clean":"2024-09-20T22:17:26.024603+0000","last_became_active":"2024-09-20T22:15:50.612544+0000","last_became_peered":"2024-09-20T22:15:50.612544+0000","last_unstale":"2024-09-20T22:17:26.024603+0000","last_undegraded":"2024-09-20T22:17:26.024603+0000","last_fullsized":"2024-09-20T22:17:26.024603+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:51.077187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:51.077187+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:26.984146+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"328'20","reported_seq":268,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.337940+0000","last_change":"2024-09-20T22:16:37.280708+0000","last_active":"2024-09-20T22:17:28.337940+0000","last_peered":"2024-09-20T22:17:28.337940+0000","last_clean":"2024-09-20T22:17:28.337940+0000","last_became_active":"2024-09-20T22:12:44.631986+0000","last_became_peered":"2024-09-20T22:12:44.631986+0000","last_unstale":"2024-09-20T22:17:28.337940+0000","last_undegraded":"2024-09-20T22:17:28.337940+0000","last_fullsized":"2024-09-20T22:17:28.337940+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:16:37.280566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:37.280566+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:04.676364+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":775,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.988985+0000","last_change":"2024-09-20T22:16:31.755794+0000","last_active":"2024-09-20T22:17:21.988985+0000","last_peered":"2024-09-20T22:17:21.988985+0000","last_clean":"2024-09-20T22:17:21.988985+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:17:21.988985+0000","last_undegraded":"2024-09-20T22:17:21.988985+0000","last_fullsized":"2024-09-20T22:17:21.988985+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:31.755589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.755589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:47.093311+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"326'14","reported_seq":415,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.023093+0000","last_change":"2024-09-20T22:16:55.917269+0000","last_active":"2024-09-20T22:17:26.023093+0000","last_peered":"2024-09-20T22:17:26.023093+0000","last_clean":"2024-09-20T22:17:26.023093+0000","last_became_active":"2024-09-20T22:12:52.518770+0000","last_became_peered":"2024-09-20T22:12:52.518770+0000","last_unstale":"2024-09-20T22:17:26.023093+0000","last_undegraded":"2024-09-20T22:17:26.023093+0000","last_fullsized":"2024-09-20T22:17:26.023093+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'14","last_scrub_stamp":"2024-09-20T22:16:55.917159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:55.917159+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:20.333421+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":236,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.028366+0000","last_change":"2024-09-20T22:17:06.828671+0000","last_active":"2024-09-20T22:17:26.028366+0000","last_peered":"2024-09-20T22:17:26.028366+0000","last_clean":"2024-09-20T22:17:26.028366+0000","last_became_active":"2024-09-20T22:12:52.523781+0000","last_became_peered":"2024-09-20T22:12:52.523781+0000","last_unstale":"2024-09-20T22:17:26.028366+0000","last_undegraded":"2024-09-20T22:17:26.028366+0000","last_fullsized":"2024-09-20T22:17:26.028366+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:06.828504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:06.828504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:22.792860+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"328'19","reported_seq":831,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.980979+0000","last_change":"2024-09-20T22:16:28.533330+0000","last_active":"2024-09-20T22:17:21.980979+0000","last_peered":"2024-09-20T22:17:21.980979+0000","last_clean":"2024-09-20T22:17:21.980979+0000","last_became_active":"2024-09-20T22:08:22.139581+0000","last_became_peered":"2024-09-20T22:08:22.139581+0000","last_unstale":"2024-09-20T22:17:21.980979+0000","last_undegraded":"2024-09-20T22:17:21.980979+0000","last_fullsized":"2024-09-20T22:17:21.980979+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:16:28.533201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:28.533201+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:50.375092+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":783,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.982506+0000","last_change":"2024-09-20T22:16:56.523856+0000","last_active":"2024-09-20T22:17:21.982506+0000","last_peered":"2024-09-20T22:17:21.982506+0000","last_clean":"2024-09-20T22:17:21.982506+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:17:21.982506+0000","last_undegraded":"2024-09-20T22:17:21.982506+0000","last_fullsized":"2024-09-20T22:17:21.982506+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:56.523743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:56.523743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:00.924603+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":615,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.336250+0000","last_change":"2024-09-20T22:16:32.216071+0000","last_active":"2024-09-20T22:17:28.336250+0000","last_peered":"2024-09-20T22:17:28.336250+0000","last_clean":"2024-09-20T22:17:28.336250+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:17:28.336250+0000","last_undegraded":"2024-09-20T22:17:28.336250+0000","last_fullsized":"2024-09-20T22:17:28.336250+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:32.215758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.215758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:00.249999+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":626,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.336064+0000","last_change":"2024-09-20T22:16:59.256702+0000","last_active":"2024-09-20T22:17:28.336064+0000","last_peered":"2024-09-20T22:17:28.336064+0000","last_clean":"2024-09-20T22:17:28.336064+0000","last_became_active":"2024-09-20T22:12:52.521497+0000","last_became_peered":"2024-09-20T22:12:52.521497+0000","last_unstale":"2024-09-20T22:17:28.336064+0000","last_undegraded":"2024-09-20T22:17:28.336064+0000","last_fullsized":"2024-09-20T22:17:28.336064+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:59.256587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:59.256587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:18.828736+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"328'19","reported_seq":700,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.336966+0000","last_change":"2024-09-20T22:17:03.305508+0000","last_active":"2024-09-20T22:17:28.336966+0000","last_peered":"2024-09-20T22:17:28.336966+0000","last_clean":"2024-09-20T22:17:28.336966+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:17:28.336966+0000","last_undegraded":"2024-09-20T22:17:28.336966+0000","last_fullsized":"2024-09-20T22:17:28.336966+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:17:03.305256+0000","last_deep_scrub":"291'13","last_deep_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_clean_scrub_stamp":"2024-09-20T22:17:03.305256+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:32.849170+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":371,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.989452+0000","last_change":"2024-09-20T22:15:54.609470+0000","last_active":"2024-09-20T22:17:21.989452+0000","last_peered":"2024-09-20T22:17:21.989452+0000","last_clean":"2024-09-20T22:17:21.989452+0000","last_became_active":"2024-09-20T22:10:30.655502+0000","last_became_peered":"2024-09-20T22:10:30.655502+0000","last_unstale":"2024-09-20T22:17:21.989452+0000","last_undegraded":"2024-09-20T22:17:21.989452+0000","last_fullsized":"2024-09-20T22:17:21.989452+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:54.609342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:54.609342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":228,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.031019+0000","last_change":"2024-09-20T22:15:33.868714+0000","last_active":"2024-09-20T22:17:26.031019+0000","last_peered":"2024-09-20T22:17:26.031019+0000","last_clean":"2024-09-20T22:17:26.031019+0000","last_became_active":"2024-09-20T22:12:52.516953+0000","last_became_peered":"2024-09-20T22:12:52.516953+0000","last_unstale":"2024-09-20T22:17:26.031019+0000","last_undegraded":"2024-09-20T22:17:26.031019+0000","last_fullsized":"2024-09-20T22:17:26.031019+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:33.868591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:33.868591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:26.983955+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":236,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.024994+0000","last_change":"2024-09-20T22:17:09.839341+0000","last_active":"2024-09-20T22:17:26.024994+0000","last_peered":"2024-09-20T22:17:26.024994+0000","last_clean":"2024-09-20T22:17:26.024994+0000","last_became_active":"2024-09-20T22:12:52.514247+0000","last_became_peered":"2024-09-20T22:12:52.514247+0000","last_unstale":"2024-09-20T22:17:26.024994+0000","last_undegraded":"2024-09-20T22:17:26.024994+0000","last_fullsized":"2024-09-20T22:17:26.024994+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:09.839241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:09.839241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:29.081256+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"75'128","reported_seq":832,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.024408+0000","last_change":"2024-09-20T22:17:04.909914+0000","last_active":"2024-09-20T22:17:26.024408+0000","last_peered":"2024-09-20T22:17:26.024408+0000","last_clean":"2024-09-20T22:17:26.024408+0000","last_became_active":"2024-09-20T22:15:50.611421+0000","last_became_peered":"2024-09-20T22:15:50.611421+0000","last_unstale":"2024-09-20T22:17:26.024408+0000","last_undegraded":"2024-09-20T22:17:26.024408+0000","last_fullsized":"2024-09-20T22:17:26.024408+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:04.909798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:04.909798+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:09.247622+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"328'19","reported_seq":700,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.991183+0000","last_change":"2024-09-20T22:16:21.625991+0000","last_active":"2024-09-20T22:17:21.991183+0000","last_peered":"2024-09-20T22:17:21.991183+0000","last_clean":"2024-09-20T22:17:21.991183+0000","last_became_active":"2024-09-20T22:12:33.545199+0000","last_became_peered":"2024-09-20T22:12:33.545199+0000","last_unstale":"2024-09-20T22:17:21.991183+0000","last_undegraded":"2024-09-20T22:17:21.991183+0000","last_fullsized":"2024-09-20T22:17:21.991183+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:16:21.625915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:21.625915+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:47.759718+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":314,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.024193+0000","last_change":"2024-09-20T22:15:27.149844+0000","last_active":"2024-09-20T22:17:26.024193+0000","last_peered":"2024-09-20T22:17:26.024193+0000","last_clean":"2024-09-20T22:17:26.024193+0000","last_became_active":"2024-09-20T22:11:30.493598+0000","last_became_peered":"2024-09-20T22:11:30.493598+0000","last_unstale":"2024-09-20T22:17:26.024193+0000","last_undegraded":"2024-09-20T22:17:26.024193+0000","last_fullsized":"2024-09-20T22:17:26.024193+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:27.149777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_clean_scrub_stamp":"2024-09-20T22:15:27.149777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.047500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":775,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.988257+0000","last_change":"2024-09-20T22:17:08.799282+0000","last_active":"2024-09-20T22:17:21.988257+0000","last_peered":"2024-09-20T22:17:21.988257+0000","last_clean":"2024-09-20T22:17:21.988257+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:17:21.988257+0000","last_undegraded":"2024-09-20T22:17:21.988257+0000","last_fullsized":"2024-09-20T22:17:21.988257+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:08.798598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:08.798598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:17.552228+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":225,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.031188+0000","last_change":"2024-09-20T22:16:20.113927+0000","last_active":"2024-09-20T22:17:26.031188+0000","last_peered":"2024-09-20T22:17:26.031188+0000","last_clean":"2024-09-20T22:17:26.031188+0000","last_became_active":"2024-09-20T22:12:57.883998+0000","last_became_peered":"2024-09-20T22:12:57.883998+0000","last_unstale":"2024-09-20T22:17:26.031188+0000","last_undegraded":"2024-09-20T22:17:26.031188+0000","last_fullsized":"2024-09-20T22:17:26.031188+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:20.113814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:20.113814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:26.993977+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"328'8","reported_seq":801,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.014950+0000","last_change":"2024-09-20T22:17:03.661899+0000","last_active":"2024-09-20T22:17:26.014950+0000","last_peered":"2024-09-20T22:17:26.014950+0000","last_clean":"2024-09-20T22:17:26.014950+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:17:26.014950+0000","last_undegraded":"2024-09-20T22:17:26.014950+0000","last_fullsized":"2024-09-20T22:17:26.014950+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'8","last_scrub_stamp":"2024-09-20T22:17:03.661772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:03.661772+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:19.537624+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":228,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.026276+0000","last_change":"2024-09-20T22:15:50.069823+0000","last_active":"2024-09-20T22:17:26.026276+0000","last_peered":"2024-09-20T22:17:26.026276+0000","last_clean":"2024-09-20T22:17:26.026276+0000","last_became_active":"2024-09-20T22:12:52.517510+0000","last_became_peered":"2024-09-20T22:12:52.517510+0000","last_unstale":"2024-09-20T22:17:26.026276+0000","last_undegraded":"2024-09-20T22:17:26.026276+0000","last_fullsized":"2024-09-20T22:17:26.026276+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:50.069659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:50.069659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:26.984090+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":800,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.020088+0000","last_change":"2024-09-20T22:16:37.743248+0000","last_active":"2024-09-20T22:17:26.020088+0000","last_peered":"2024-09-20T22:17:26.020088+0000","last_clean":"2024-09-20T22:17:26.020088+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:17:26.020088+0000","last_undegraded":"2024-09-20T22:17:26.020088+0000","last_fullsized":"2024-09-20T22:17:26.020088+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:37.743137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:37.743137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:04.711878+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":763,"reported_epoch":423,"state":"active+clean","last_fresh":"2024-09-20T22:17:21.988262+0000","last_change":"2024-09-20T22:15:29.393451+0000","last_active":"2024-09-20T22:17:21.988262+0000","last_peered":"2024-09-20T22:17:21.988262+0000","last_clean":"2024-09-20T22:17:21.988262+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:17:21.988262+0000","last_undegraded":"2024-09-20T22:17:21.988262+0000","last_fullsized":"2024-09-20T22:17:21.988262+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:29.393342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_clean_scrub_stamp":"2024-09-20T22:15:29.393342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":664,"reported_epoch":425,"state":"active+clean","last_fresh":"2024-09-20T22:17:28.336898+0000","last_change":"2024-09-20T22:15:49.314143+0000","last_active":"2024-09-20T22:17:28.336898+0000","last_peered":"2024-09-20T22:17:28.336898+0000","last_clean":"2024-09-20T22:17:28.336898+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:17:28.336898+0000","last_undegraded":"2024-09-20T22:17:28.336898+0000","last_fullsized":"2024-09-20T22:17:28.336898+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:49.314024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:49.314024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:29.477230+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":236,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.029087+0000","last_change":"2024-09-20T22:16:56.775739+0000","last_active":"2024-09-20T22:17:26.029087+0000","last_peered":"2024-09-20T22:17:26.029087+0000","last_clean":"2024-09-20T22:17:26.029087+0000","last_became_active":"2024-09-20T22:12:52.518076+0000","last_became_peered":"2024-09-20T22:12:52.518076+0000","last_unstale":"2024-09-20T22:17:26.029087+0000","last_undegraded":"2024-09-20T22:17:26.029087+0000","last_fullsized":"2024-09-20T22:17:26.029087+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:56.775388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:56.775388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:17:59.279895+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"328'19","reported_seq":821,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.021503+0000","last_change":"2024-09-20T22:15:57.649394+0000","last_active":"2024-09-20T22:17:26.021503+0000","last_peered":"2024-09-20T22:17:26.021503+0000","last_clean":"2024-09-20T22:17:26.021503+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:17:26.021503+0000","last_undegraded":"2024-09-20T22:17:26.021503+0000","last_fullsized":"2024-09-20T22:17:26.021503+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:15:57.649159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.649159+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":237,"reported_epoch":424,"state":"active+clean","last_fresh":"2024-09-20T22:17:26.028275+0000","last_change":"2024-09-20T22:17:10.840330+0000","last_active":"2024-09-20T22:17:26.028275+0000","last_peered":"2024-09-20T22:17:26.028275+0000","last_clean":"2024-09-20T22:17:26.028275+0000","last_became_active":"2024-09-20T22:12:52.524425+0000","last_became_peered":"2024-09-20T22:12:52.524425+0000","last_unstale":"2024-09-20T22:17:26.028275+0000","last_undegraded":"2024-09-20T22:17:26.028275+0000","last_fullsized":"2024-09-20T22:17:26.028275+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:10.839899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:10.839899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:35.398127+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":5143134232,"num_objects":78479,"num_object_clones":0,"num_object_copies":235437,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78479,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":171101,"num_write_kb":7986497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":15508320256,"data_stored":15508308040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79119,"ondisk_log_size":79119,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1116,"num_read_kb":873,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":3410,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":851,"num_read_kb":846,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":21,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":262688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2688,"ondisk_log_size":2688,"up":63,"acting":63,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813136051,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1926380,"kb_used_data":1281564,"kb_used_omap":1,"kb_used_meta":644798,"kb_avail":91818772,"statfs":{"total":95995035648,"available":94022422528,"internally_reserved":0,"allocated":1312321536,"data_stored":1311876223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":660273610},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430389,"num_pgs":61,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2742252,"kb_used_data":2175012,"kb_used_omap":1,"kb_used_meta":567038,"kb_avail":91002900,"statfs":{"total":95995035648,"available":93186969600,"internally_reserved":0,"allocated":2227212288,"data_stored":2226838116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":580647366},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724731,"num_pgs":62,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2291236,"kb_used_data":1605980,"kb_used_omap":1,"kb_used_meta":685054,"kb_avail":91453916,"statfs":{"total":95995035648,"available":93648809984,"internally_reserved":0,"allocated":1644523520,"data_stored":1644156435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":701495749},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986371,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3160236,"kb_used_data":2579292,"kb_used_omap":1,"kb_used_meta":580926,"kb_avail":90584916,"statfs":{"total":95995035648,"available":92758953984,"internally_reserved":0,"allocated":2641195008,"data_stored":2640748898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":594868677},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770258048,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1410984,"kb_used_data":928544,"kb_used_omap":4,"kb_used_meta":482235,"kb_avail":92334168,"statfs":{"total":95995035648,"available":94550188032,"internally_reserved":0,"allocated":950829056,"data_stored":950472913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":493809320},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149896,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2276692,"kb_used_data":1577732,"kb_used_omap":1,"kb_used_meta":698942,"kb_avail":91468460,"statfs":{"total":95995035648,"available":93663703040,"internally_reserved":0,"allocated":1615597568,"data_stored":1615191387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":715717063},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411533,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2543576,"kb_used_data":1908488,"kb_used_omap":1,"kb_used_meta":635070,"kb_avail":91201576,"statfs":{"total":95995035648,"available":93390413824,"internally_reserved":0,"allocated":1954291712,"data_stored":1953866149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":650312133},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738584,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":3926460,"kb_used_data":3168820,"kb_used_omap":1,"kb_used_meta":757246,"kb_avail":89818692,"statfs":{"total":95995035648,"available":91974340608,"internally_reserved":0,"allocated":3244871680,"data_stored":3244480804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":775420357},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":3231322112,"data_stored":3231318040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":948568064,"data_stored":948568064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1934295040,"data_stored":1934295040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1613561856,"data_stored":1613561856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2621968384,"data_stored":2621964312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1637154816,"data_stored":1637154816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2212106240,"data_stored":2212102168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1309343744,"data_stored":1309343744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:17:32.992 INFO:tasks.cephadm.ceph_manager.ceph:increase pool size by 10 2024-09-20T22:17:32.992 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool set .mgr pg_num 31 2024-09-20T22:17:33.012 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:33.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:17:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:17:33.023 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:33.052 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:33.054 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:33.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:33.055 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:33.156 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:33.185 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:33.217 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:33 smithi060 ceph-mon[29694]: 10.2 scrub ok 2024-09-20T22:17:33.217 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:33 smithi060 ceph-mon[29694]: 5.d scrub starts 2024-09-20T22:17:33.217 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:33 smithi060 ceph-mon[29694]: 5.d scrub ok 2024-09-20T22:17:33.218 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:33 smithi060 ceph-mon[29694]: 10.c scrub starts 2024-09-20T22:17:33.218 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:33 smithi060 ceph-mon[29694]: from='client.26111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:17:33.218 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:33 smithi060 ceph-mon[29694]: 5.13 scrub starts 2024-09-20T22:17:33.218 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:33 smithi060 ceph-mon[29694]: 5.13 scrub ok 2024-09-20T22:17:33.218 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:33 smithi060 ceph-mon[29694]: pgmap v781: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 4.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-20T22:17:33.218 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:33 smithi060 ceph-mon[29694]: 10.6 scrub ok 2024-09-20T22:17:33.218 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:33 smithi060 ceph-mon[40290]: 10.2 scrub ok 2024-09-20T22:17:33.218 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:33 smithi060 ceph-mon[40290]: 5.d scrub starts 2024-09-20T22:17:33.219 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:33 smithi060 ceph-mon[40290]: 5.d scrub ok 2024-09-20T22:17:33.219 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:33 smithi060 ceph-mon[40290]: 10.c scrub starts 2024-09-20T22:17:33.219 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:33 smithi060 ceph-mon[40290]: from='client.26111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:17:33.219 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:33 smithi060 ceph-mon[40290]: 5.13 scrub starts 2024-09-20T22:17:33.219 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:33 smithi060 ceph-mon[40290]: 5.13 scrub ok 2024-09-20T22:17:33.219 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:33 smithi060 ceph-mon[40290]: pgmap v781: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 4.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-20T22:17:33.219 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:33 smithi060 ceph-mon[40290]: 10.6 scrub ok 2024-09-20T22:17:33.219 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:33.220 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:33.320 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:33.375 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:33.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:33.376 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:33.413 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:33 smithi067 ceph-mon[33511]: 10.2 scrub ok 2024-09-20T22:17:33.413 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:33 smithi067 ceph-mon[33511]: 5.d scrub starts 2024-09-20T22:17:33.413 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:33 smithi067 ceph-mon[33511]: 5.d scrub ok 2024-09-20T22:17:33.413 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:33 smithi067 ceph-mon[33511]: 10.c scrub starts 2024-09-20T22:17:33.414 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:33 smithi067 ceph-mon[33511]: from='client.26111 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:17:33.414 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:33 smithi067 ceph-mon[33511]: 5.13 scrub starts 2024-09-20T22:17:33.414 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:33 smithi067 ceph-mon[33511]: 5.13 scrub ok 2024-09-20T22:17:33.414 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:33 smithi067 ceph-mon[33511]: pgmap v781: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 4.6 GiB data, 19 GiB used, 696 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-20T22:17:33.414 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:33 smithi067 ceph-mon[33511]: 10.6 scrub ok 2024-09-20T22:17:33.477 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:33.507 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:33.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:33.508 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:33.567 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:17:33.568 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:17:33.609 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:33.643 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:33.644 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:33.644 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:33.745 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:33.782 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:33.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:33.783 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:33.883 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:33.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:33.923 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:33.923 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:34.024 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:34.060 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:34.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:34.061 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:34.162 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:34.201 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:34.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:34.202 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:34.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:34 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:17:34.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:34 smithi067 ceph-mon[33511]: 6.19 scrub starts 2024-09-20T22:17:34.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:34 smithi067 ceph-mon[33511]: 6.19 scrub ok 2024-09-20T22:17:34.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:34 smithi067 ceph-mon[33511]: 10.c scrub ok 2024-09-20T22:17:34.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:34.336 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:34.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:34.338 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:34.432 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T22:17:34.438 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:34.526 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:34.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:34.527 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:34.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:34 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:17:34.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:34 smithi060 ceph-mon[29694]: 6.19 scrub starts 2024-09-20T22:17:34.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:34 smithi060 ceph-mon[29694]: 6.19 scrub ok 2024-09-20T22:17:34.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:34 smithi060 ceph-mon[29694]: 10.c scrub ok 2024-09-20T22:17:34.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:34 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:17:34.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:34 smithi060 ceph-mon[40290]: 6.19 scrub starts 2024-09-20T22:17:34.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:34 smithi060 ceph-mon[40290]: 6.19 scrub ok 2024-09-20T22:17:34.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:34 smithi060 ceph-mon[40290]: 10.c scrub ok 2024-09-20T22:17:34.627 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:34.673 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:34.674 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:34.674 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:34.775 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:34.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:34.815 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:34.815 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:34.869 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:17:34.916 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:34.944 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:34.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:34.945 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:35.046 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:35.068 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:17:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:17:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:17:35.082 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:35.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:35.083 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:35.184 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:35.185 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:17:35.185 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:17:35.185 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:17:35.185 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:17:35.223 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:17:35.223 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:17:35.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:35.226 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:35.226 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:35.262 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:17:35.262 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:17:35.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:35 smithi067 ceph-mon[33511]: 6.12 scrub starts 2024-09-20T22:17:35.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:35 smithi067 ceph-mon[33511]: 6.12 scrub ok 2024-09-20T22:17:35.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:35 smithi067 ceph-mon[33511]: 10.9 scrub starts 2024-09-20T22:17:35.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:35 smithi067 ceph-mon[33511]: pgmap v782: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 619 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-20T22:17:35.306 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:17:35.307 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:17:35.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:35.353 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:35 smithi060 ceph-mon[29694]: 6.12 scrub starts 2024-09-20T22:17:35.353 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:35 smithi060 ceph-mon[29694]: 6.12 scrub ok 2024-09-20T22:17:35.353 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:35 smithi060 ceph-mon[29694]: 10.9 scrub starts 2024-09-20T22:17:35.353 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:35 smithi060 ceph-mon[29694]: pgmap v782: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 619 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-20T22:17:35.353 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:17:35.354 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:17:35.354 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:35 smithi060 ceph-mon[40290]: 6.12 scrub starts 2024-09-20T22:17:35.354 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:35 smithi060 ceph-mon[40290]: 6.12 scrub ok 2024-09-20T22:17:35.354 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:35 smithi060 ceph-mon[40290]: 10.9 scrub starts 2024-09-20T22:17:35.354 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:35 smithi060 ceph-mon[40290]: pgmap v782: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 4.5 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 619 B/s rd, 0 B/s wr, 1.24k op/s 2024-09-20T22:17:35.364 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:35.377 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:35.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:35.379 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:35.394 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:17:35.394 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:17:35.436 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:17:35.436 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:17:35.480 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:35.516 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:35.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:35.517 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:35.518 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:17:35.518 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:17:35.519 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:17:35.519 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:17:35.519 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.519 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105663 8.19 8:9a7b6ea1:::benchmark_data_smithi060_111051_object105662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:17:35.519 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.546306+0000", 2024-09-20T22:17:35.519 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 506.96818586299997, 2024-09-20T22:17:35.519 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.071300715, 2024-09-20T22:17:35.519 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.519 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.519 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.519 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.519 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:35.520 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:35.520 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105663 2024-09-20T22:17:35.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.520 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.520 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.520 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.520 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.546306+0000", 2024-09-20T22:17:35.520 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.520 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.520 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.520 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.546306+0000", 2024-09-20T22:17:35.520 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.521 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.521 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.546308+0000", 2024-09-20T22:17:35.521 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1579999999999999e-06 2024-09-20T22:17:35.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.521 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.521 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.546347+0000", 2024-09-20T22:17:35.521 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.8219000000000001e-05 2024-09-20T22:17:35.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.521 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.521 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.546371+0000", 2024-09-20T22:17:35.522 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.4712e-05 2024-09-20T22:17:35.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.522 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.522 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.522 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.546391+0000", 2024-09-20T22:17:35.522 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9898000000000002e-05 2024-09-20T22:17:35.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.522 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.522 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.522 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.760159+0000", 2024-09-20T22:17:35.522 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.21376741699999999 2024-09-20T22:17:35.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.522 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.522 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:35.522 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.760173+0000", 2024-09-20T22:17:35.523 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3869999999999999e-05 2024-09-20T22:17:35.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.523 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.523 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.523 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.611215+0000", 2024-09-20T22:17:35.523 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.85104213399999995 2024-09-20T22:17:35.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.523 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.523 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.523 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.611327+0000", 2024-09-20T22:17:35.523 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011246100000000001 2024-09-20T22:17:35.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.523 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.523 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,5", 2024-09-20T22:17:35.523 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.611571+0000", 2024-09-20T22:17:35.523 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000243706 2024-09-20T22:17:35.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.524 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.524 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.524 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.617336+0000", 2024-09-20T22:17:35.524 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0057650169999999999 2024-09-20T22:17:35.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.524 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.524 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.524 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.617408+0000", 2024-09-20T22:17:35.524 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2268999999999995e-05 2024-09-20T22:17:35.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.524 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.524 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.524 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.617520+0000", 2024-09-20T22:17:35.524 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000112067 2024-09-20T22:17:35.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.525 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.525 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.525 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.617571+0000", 2024-09-20T22:17:35.525 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0447e-05 2024-09-20T22:17:35.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.525 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.525 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.525 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.617607+0000", 2024-09-20T22:17:35.525 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6340000000000001e-05 2024-09-20T22:17:35.525 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.525 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.525 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.525 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.526 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:105666 8.15 8:ad619d28:::benchmark_data_smithi060_111051_object105665:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:17:35.526 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:08.570331+0000", 2024-09-20T22:17:35.526 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 506.94416141099998, 2024-09-20T22:17:35.526 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0444536209999999, 2024-09-20T22:17:35.526 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.526 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.526 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.526 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.526 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:35.526 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:35.526 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 105666 2024-09-20T22:17:35.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.526 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.526 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.526 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.526 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.570331+0000", 2024-09-20T22:17:35.527 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.527 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.527 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.570331+0000", 2024-09-20T22:17:35.527 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.527 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.527 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.570334+0000", 2024-09-20T22:17:35.527 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.8229999999999999e-06 2024-09-20T22:17:35.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.527 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.528 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.570387+0000", 2024-09-20T22:17:35.528 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3412000000000003e-05 2024-09-20T22:17:35.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.528 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.528 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.528 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.570419+0000", 2024-09-20T22:17:35.528 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2280000000000003e-05 2024-09-20T22:17:35.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.528 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.528 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.528 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.570435+0000", 2024-09-20T22:17:35.528 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5265e-05 2024-09-20T22:17:35.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.528 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.528 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.528 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.745782+0000", 2024-09-20T22:17:35.528 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.175347111 2024-09-20T22:17:35.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.529 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.529 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:35.529 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:08.745796+0000", 2024-09-20T22:17:35.529 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4178e-05 2024-09-20T22:17:35.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.529 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.529 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.529 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.600893+0000", 2024-09-20T22:17:35.529 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.855096992 2024-09-20T22:17:35.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.529 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.529 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.529 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.601413+0000", 2024-09-20T22:17:35.529 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00051982700000000005 2024-09-20T22:17:35.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.530 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.530 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 4,7", 2024-09-20T22:17:35.530 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.601756+0000", 2024-09-20T22:17:35.530 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00034325700000000002 2024-09-20T22:17:35.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.530 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.530 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.530 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.603923+0000", 2024-09-20T22:17:35.530 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002167329 2024-09-20T22:17:35.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.530 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.530 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.530 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.608479+0000", 2024-09-20T22:17:35.530 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0045553820000000002 2024-09-20T22:17:35.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.531 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.531 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.531 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.614677+0000", 2024-09-20T22:17:35.531 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061983100000000003 2024-09-20T22:17:35.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.531 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.531 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.531 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.614736+0000", 2024-09-20T22:17:35.531 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9598000000000001e-05 2024-09-20T22:17:35.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.532 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.532 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.532 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:09.614784+0000", 2024-09-20T22:17:35.532 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7856999999999999e-05 2024-09-20T22:17:35.532 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.532 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.532 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.532 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.532 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:187857 8.9 8:977b7bdd:::benchmark_data_smithi060_111051_object67858:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:17:35.532 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:29.640677+0000", 2024-09-20T22:17:35.532 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 425.873814729, 2024-09-20T22:17:35.532 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.102806486, 2024-09-20T22:17:35.532 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.533 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.533 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.533 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.533 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:35.533 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:35.533 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 187857 2024-09-20T22:17:35.533 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.533 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.533 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.533 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.533 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.640677+0000", 2024-09-20T22:17:35.533 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.533 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.533 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.534 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.534 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.640677+0000", 2024-09-20T22:17:35.534 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.534 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.534 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.534 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.640679+0000", 2024-09-20T22:17:35.534 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3430000000000001e-06 2024-09-20T22:17:35.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.534 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.534 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.534 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.640682+0000", 2024-09-20T22:17:35.535 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.557e-06 2024-09-20T22:17:35.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.535 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.535 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.640691+0000", 2024-09-20T22:17:35.535 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1710000000000008e-06 2024-09-20T22:17:35.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.535 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.535 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.640699+0000", 2024-09-20T22:17:35.535 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5059999999999998e-06 2024-09-20T22:17:35.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.535 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.535 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.643753+0000", 2024-09-20T22:17:35.536 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0030544809999999999 2024-09-20T22:17:35.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.536 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:35.536 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.643766+0000", 2024-09-20T22:17:35.536 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2227e-05 2024-09-20T22:17:35.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.536 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.536 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:30.635192+0000", 2024-09-20T22:17:35.536 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.991425899 2024-09-20T22:17:35.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.536 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:35.536 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:30.635203+0000", 2024-09-20T22:17:35.537 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0957e-05 2024-09-20T22:17:35.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.537 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.537 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.537 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.735750+0000", 2024-09-20T22:17:35.537 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.10054739 2024-09-20T22:17:35.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.537 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.537 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.537 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.735843+0000", 2024-09-20T22:17:35.537 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2980999999999996e-05 2024-09-20T22:17:35.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.537 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.537 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3,5", 2024-09-20T22:17:35.537 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.736139+0000", 2024-09-20T22:17:35.538 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000296104 2024-09-20T22:17:35.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.538 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.538 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.538 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.742910+0000", 2024-09-20T22:17:35.538 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0067707760000000001 2024-09-20T22:17:35.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.538 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.538 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.538 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.742932+0000", 2024-09-20T22:17:35.538 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2181e-05 2024-09-20T22:17:35.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.538 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.538 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.539 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743233+0000", 2024-09-20T22:17:35.539 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00030141400000000001 2024-09-20T22:17:35.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.539 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.539 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.539 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743293+0000", 2024-09-20T22:17:35.539 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9304e-05 2024-09-20T22:17:35.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.539 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.539 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.539 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743350+0000", 2024-09-20T22:17:35.539 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7015999999999998e-05 2024-09-20T22:17:35.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.539 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.539 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.540 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743484+0000", 2024-09-20T22:17:35.540 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000134179 2024-09-20T22:17:35.540 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.540 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.540 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.540 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.540 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:187866 8.9 8:90a8fa22:::benchmark_data_smithi060_111051_object67867:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:17:35.540 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:29.665081+0000", 2024-09-20T22:17:35.540 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 425.849410944, 2024-09-20T22:17:35.540 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0789447289999998, 2024-09-20T22:17:35.540 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.540 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.540 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.540 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.540 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:35.541 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:35.541 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 187866 2024-09-20T22:17:35.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.541 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.541 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.541 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665081+0000", 2024-09-20T22:17:35.541 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.541 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.541 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665081+0000", 2024-09-20T22:17:35.541 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.541 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.542 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665082+0000", 2024-09-20T22:17:35.542 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9800000000000002e-07 2024-09-20T22:17:35.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.542 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.542 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.542 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665086+0000", 2024-09-20T22:17:35.542 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.36e-06 2024-09-20T22:17:35.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.542 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.542 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.542 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665096+0000", 2024-09-20T22:17:35.542 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0755e-05 2024-09-20T22:17:35.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.542 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.543 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.543 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665104+0000", 2024-09-20T22:17:35.543 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6029999999999999e-06 2024-09-20T22:17:35.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.543 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.543 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.543 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665150+0000", 2024-09-20T22:17:35.543 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6409000000000002e-05 2024-09-20T22:17:35.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.543 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.543 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:35.543 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665158+0000", 2024-09-20T22:17:35.543 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9039999999999992e-06 2024-09-20T22:17:35.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.544 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.544 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:30.635228+0000", 2024-09-20T22:17:35.544 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.97006970000000003 2024-09-20T22:17:35.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.544 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:35.544 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:30.635238+0000", 2024-09-20T22:17:35.544 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0108999999999999e-05 2024-09-20T22:17:35.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.544 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.544 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.739124+0000", 2024-09-20T22:17:35.544 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1038857790000001 2024-09-20T22:17:35.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.545 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.545 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.739264+0000", 2024-09-20T22:17:35.545 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014064700000000001 2024-09-20T22:17:35.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.545 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3,5", 2024-09-20T22:17:35.545 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.739542+0000", 2024-09-20T22:17:35.545 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00027733800000000002 2024-09-20T22:17:35.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.545 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.545 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.742938+0000", 2024-09-20T22:17:35.545 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0033961389999999998 2024-09-20T22:17:35.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.546 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.546 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.546 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743151+0000", 2024-09-20T22:17:35.546 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000212592 2024-09-20T22:17:35.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.546 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.546 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.546 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743850+0000", 2024-09-20T22:17:35.546 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00069932800000000004 2024-09-20T22:17:35.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.546 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.546 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.546 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743933+0000", 2024-09-20T22:17:35.547 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2720999999999996e-05 2024-09-20T22:17:35.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.547 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.547 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.547 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743965+0000", 2024-09-20T22:17:35.547 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2230000000000001e-05 2024-09-20T22:17:35.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.547 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.547 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.547 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.744026+0000", 2024-09-20T22:17:35.547 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1117000000000005e-05 2024-09-20T22:17:35.547 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.547 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.547 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.548 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.548 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:187902 8.9 8:95e80f7a:::benchmark_data_smithi060_111051_object67903:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:17:35.548 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:29.720026+0000", 2024-09-20T22:17:35.548 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 425.794466403, 2024-09-20T22:17:35.548 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.024196764, 2024-09-20T22:17:35.548 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.548 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.548 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.548 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.548 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:35.548 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:35.548 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 187902 2024-09-20T22:17:35.548 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.548 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.548 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.549 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.549 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720026+0000", 2024-09-20T22:17:35.549 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.549 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.549 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.549 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720026+0000", 2024-09-20T22:17:35.549 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.549 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.549 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.549 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720027+0000", 2024-09-20T22:17:35.549 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.093e-06 2024-09-20T22:17:35.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.550 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.550 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.550 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720031+0000", 2024-09-20T22:17:35.550 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0720000000000001e-06 2024-09-20T22:17:35.550 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.550 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.550 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.550 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720042+0000", 2024-09-20T22:17:35.550 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1280999999999999e-05 2024-09-20T22:17:35.550 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.550 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.550 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.550 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720048+0000", 2024-09-20T22:17:35.551 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3870000000000003e-06 2024-09-20T22:17:35.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.551 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.551 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.551 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720098+0000", 2024-09-20T22:17:35.551 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0597999999999999e-05 2024-09-20T22:17:35.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.551 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.551 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:35.551 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720107+0000", 2024-09-20T22:17:35.551 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3510000000000002e-06 2024-09-20T22:17:35.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.551 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.551 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.552 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:30.635262+0000", 2024-09-20T22:17:35.552 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.91515579300000005 2024-09-20T22:17:35.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.552 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.552 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:35.552 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:30.635270+0000", 2024-09-20T22:17:35.552 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.994e-06 2024-09-20T22:17:35.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.552 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.552 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.552 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.740932+0000", 2024-09-20T22:17:35.552 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.105662157 2024-09-20T22:17:35.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.552 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.552 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.553 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.741060+0000", 2024-09-20T22:17:35.553 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012800800000000001 2024-09-20T22:17:35.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.553 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.553 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3,5", 2024-09-20T22:17:35.553 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.741319+0000", 2024-09-20T22:17:35.553 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025847700000000001 2024-09-20T22:17:35.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.553 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.553 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.553 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743179+0000", 2024-09-20T22:17:35.553 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0018601119999999999 2024-09-20T22:17:35.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.553 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.553 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.553 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743556+0000", 2024-09-20T22:17:35.554 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00037664000000000003 2024-09-20T22:17:35.554 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.554 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.554 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.554 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743893+0000", 2024-09-20T22:17:35.554 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00033714699999999999 2024-09-20T22:17:35.554 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.554 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.554 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.554 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.744064+0000", 2024-09-20T22:17:35.554 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017144400000000001 2024-09-20T22:17:35.554 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.554 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.554 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.554 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.744116+0000", 2024-09-20T22:17:35.555 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1873999999999998e-05 2024-09-20T22:17:35.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.555 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.555 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.555 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.744223+0000", 2024-09-20T22:17:35.555 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010633600000000001 2024-09-20T22:17:35.555 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.555 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.555 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.555 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.555 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223308 8.9 8:97e7b109:::benchmark_data_smithi060_111051_object103309:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:17:35.555 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.506871+0000", 2024-09-20T22:17:35.556 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 389.00762087300001, 2024-09-20T22:17:35.556 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.99735096099999998, 2024-09-20T22:17:35.556 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.556 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.556 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.556 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.556 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:35.556 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:35.556 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223308 2024-09-20T22:17:35.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.556 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.556 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.556 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.556 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506871+0000", 2024-09-20T22:17:35.556 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.557 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.557 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.557 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506871+0000", 2024-09-20T22:17:35.557 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.557 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.557 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.557 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506873+0000", 2024-09-20T22:17:35.557 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0470000000000001e-06 2024-09-20T22:17:35.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.557 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.557 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.557 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506882+0000", 2024-09-20T22:17:35.558 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3310000000000004e-06 2024-09-20T22:17:35.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.558 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.558 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.558 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506904+0000", 2024-09-20T22:17:35.558 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.283e-05 2024-09-20T22:17:35.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.559 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.559 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.559 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506913+0000", 2024-09-20T22:17:35.559 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9530000000000005e-06 2024-09-20T22:17:35.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.559 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.559 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.559 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506975+0000", 2024-09-20T22:17:35.559 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1773000000000006e-05 2024-09-20T22:17:35.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.559 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.559 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:35.559 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506982+0000", 2024-09-20T22:17:35.559 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2869999999999999e-06 2024-09-20T22:17:35.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.560 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.560 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.560 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.494954+0000", 2024-09-20T22:17:35.560 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98797138799999995 2024-09-20T22:17:35.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.560 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.560 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.560 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.495095+0000", 2024-09-20T22:17:35.560 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014108 2024-09-20T22:17:35.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.560 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.560 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:17:35.560 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.495506+0000", 2024-09-20T22:17:35.561 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00041153700000000001 2024-09-20T22:17:35.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.561 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.561 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.500203+0000", 2024-09-20T22:17:35.561 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046968380000000001 2024-09-20T22:17:35.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.561 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.561 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.500238+0000", 2024-09-20T22:17:35.561 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4195e-05 2024-09-20T22:17:35.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.562 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.562 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.504167+0000", 2024-09-20T22:17:35.562 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0039294819999999998 2024-09-20T22:17:35.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.562 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.562 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.562 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.504199+0000", 2024-09-20T22:17:35.562 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1853999999999997e-05 2024-09-20T22:17:35.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.562 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.562 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.562 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.504222+0000", 2024-09-20T22:17:35.562 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.3366e-05 2024-09-20T22:17:35.563 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.563 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.563 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.563 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223310 8.9 8:94027251:::benchmark_data_smithi060_111051_object103311:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:17:35.563 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.507858+0000", 2024-09-20T22:17:35.563 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 389.00663428000001, 2024-09-20T22:17:35.563 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0231674930000001, 2024-09-20T22:17:35.563 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.563 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.563 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.563 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.564 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:35.564 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:35.564 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223310 2024-09-20T22:17:35.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.564 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.564 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.564 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.564 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507858+0000", 2024-09-20T22:17:35.564 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.564 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.564 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.564 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507858+0000", 2024-09-20T22:17:35.564 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.565 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.565 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.565 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507859+0000", 2024-09-20T22:17:35.565 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.78e-07 2024-09-20T22:17:35.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.565 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.565 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.565 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507864+0000", 2024-09-20T22:17:35.565 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8969999999999999e-06 2024-09-20T22:17:35.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.565 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.565 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.565 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507879+0000", 2024-09-20T22:17:35.565 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4919999999999999e-05 2024-09-20T22:17:35.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.565 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.565 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.566 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507886+0000", 2024-09-20T22:17:35.566 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2119999999999997e-06 2024-09-20T22:17:35.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.566 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.566 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.566 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507948+0000", 2024-09-20T22:17:35.566 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1764e-05 2024-09-20T22:17:35.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.566 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.566 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:35.566 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507959+0000", 2024-09-20T22:17:35.566 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1613999999999999e-05 2024-09-20T22:17:35.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.567 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.567 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.497481+0000", 2024-09-20T22:17:35.567 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98952213899999997 2024-09-20T22:17:35.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.567 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.567 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.497555+0000", 2024-09-20T22:17:35.567 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4202000000000005e-05 2024-09-20T22:17:35.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.567 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:17:35.567 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.497785+0000", 2024-09-20T22:17:35.567 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00022984599999999999 2024-09-20T22:17:35.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.567 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.568 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.501438+0000", 2024-09-20T22:17:35.568 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0036526850000000001 2024-09-20T22:17:35.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.568 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.568 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.568 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.504265+0000", 2024-09-20T22:17:35.568 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002827319 2024-09-20T22:17:35.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.568 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.568 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.568 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.530930+0000", 2024-09-20T22:17:35.568 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.026664383999999999 2024-09-20T22:17:35.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.568 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.568 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.569 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.530981+0000", 2024-09-20T22:17:35.569 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1109999999999997e-05 2024-09-20T22:17:35.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.569 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.569 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.569 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531025+0000", 2024-09-20T22:17:35.569 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4523e-05 2024-09-20T22:17:35.569 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.569 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.569 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.569 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.569 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223355 8.9 8:94b7fbf7:::benchmark_data_smithi060_111051_object103356:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:17:35.569 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.561400+0000", 2024-09-20T22:17:35.569 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 388.95309249500002, 2024-09-20T22:17:35.570 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.96979188400000005, 2024-09-20T22:17:35.570 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.570 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.570 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.570 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.570 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:35.570 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:35.570 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223355 2024-09-20T22:17:35.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.570 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.570 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.570 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.570 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561400+0000", 2024-09-20T22:17:35.571 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.571 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.571 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.571 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561400+0000", 2024-09-20T22:17:35.571 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.571 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.571 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.571 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561401+0000", 2024-09-20T22:17:35.571 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3790000000000001e-06 2024-09-20T22:17:35.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.571 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.571 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.572 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561406+0000", 2024-09-20T22:17:35.572 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4559999999999999e-06 2024-09-20T22:17:35.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.572 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.572 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.572 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561419+0000", 2024-09-20T22:17:35.572 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2712e-05 2024-09-20T22:17:35.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.572 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.572 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.572 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561426+0000", 2024-09-20T22:17:35.572 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6080000000000001e-06 2024-09-20T22:17:35.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.572 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.573 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.573 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561475+0000", 2024-09-20T22:17:35.573 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9412000000000001e-05 2024-09-20T22:17:35.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.573 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.573 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:35.573 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561482+0000", 2024-09-20T22:17:35.573 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7789999999999998e-06 2024-09-20T22:17:35.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.573 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.573 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.573 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.499040+0000", 2024-09-20T22:17:35.573 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93755812199999999 2024-09-20T22:17:35.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.573 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.574 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.574 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.499105+0000", 2024-09-20T22:17:35.574 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5328999999999997e-05 2024-09-20T22:17:35.574 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.574 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.574 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:17:35.574 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.499270+0000", 2024-09-20T22:17:35.574 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000164678 2024-09-20T22:17:35.574 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.574 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.574 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.574 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.504133+0000", 2024-09-20T22:17:35.574 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0048630009999999996 2024-09-20T22:17:35.574 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.575 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.575 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.575 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.511137+0000", 2024-09-20T22:17:35.575 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.007004316 2024-09-20T22:17:35.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.575 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.575 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.575 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531092+0000", 2024-09-20T22:17:35.575 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.019954896999999999 2024-09-20T22:17:35.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.575 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.575 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.575 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531148+0000", 2024-09-20T22:17:35.575 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5828999999999997e-05 2024-09-20T22:17:35.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.576 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.576 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.576 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531192+0000", 2024-09-20T22:17:35.576 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3365999999999998e-05 2024-09-20T22:17:35.576 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.576 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.576 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.576 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.576 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223360 8.9 8:963e2bee:::benchmark_data_smithi060_111051_object103361:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:17:35.576 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.566084+0000", 2024-09-20T22:17:35.576 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 388.94840781699997, 2024-09-20T22:17:35.576 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.965237028, 2024-09-20T22:17:35.576 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.577 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.577 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.577 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.577 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:35.577 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:35.577 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223360 2024-09-20T22:17:35.577 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.577 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.577 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.577 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.577 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566084+0000", 2024-09-20T22:17:35.577 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.577 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566084+0000", 2024-09-20T22:17:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566086+0000", 2024-09-20T22:17:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.175e-06 2024-09-20T22:17:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566090+0000", 2024-09-20T22:17:35.578 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2080000000000002e-06 2024-09-20T22:17:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566099+0000", 2024-09-20T22:17:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3559999999999999e-06 2024-09-20T22:17:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566105+0000", 2024-09-20T22:17:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8649999999999998e-06 2024-09-20T22:17:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566150+0000", 2024-09-20T22:17:35.579 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5252e-05 2024-09-20T22:17:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566156+0000", 2024-09-20T22:17:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7370000000000003e-06 2024-09-20T22:17:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.500259+0000", 2024-09-20T22:17:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93410321200000002 2024-09-20T22:17:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.500327+0000", 2024-09-20T22:17:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8270999999999997e-05 2024-09-20T22:17:35.580 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:17:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.500502+0000", 2024-09-20T22:17:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000174797 2024-09-20T22:17:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.504273+0000", 2024-09-20T22:17:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003771019 2024-09-20T22:17:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.511209+0000", 2024-09-20T22:17:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0069359080000000002 2024-09-20T22:17:35.581 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531241+0000", 2024-09-20T22:17:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.020031694999999999 2024-09-20T22:17:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531287+0000", 2024-09-20T22:17:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6060000000000003e-05 2024-09-20T22:17:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531321+0000", 2024-09-20T22:17:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4473e-05 2024-09-20T22:17:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.582 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:62123 9.1 9:879a37c5:::benchmark_data_smithi060_140132_object62122:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e309)", 2024-09-20T22:17:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:12:20.517780+0000", 2024-09-20T22:17:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 314.99671245299999, 2024-09-20T22:17:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0093993530000001, 2024-09-20T22:17:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:17:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:17:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:17:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 62123 2024-09-20T22:17:35.583 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.517780+0000", 2024-09-20T22:17:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.517780+0000", 2024-09-20T22:17:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.584 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.517781+0000", 2024-09-20T22:17:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.486e-06 2024-09-20T22:17:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.517964+0000", 2024-09-20T22:17:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000182542 2024-09-20T22:17:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.585 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.517976+0000", 2024-09-20T22:17:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2031999999999999e-05 2024-09-20T22:17:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.517984+0000", 2024-09-20T22:17:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2859999999999999e-06 2024-09-20T22:17:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.518698+0000", 2024-09-20T22:17:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00071389999999999995 2024-09-20T22:17:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.586 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.518708+0000", 2024-09-20T22:17:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0176999999999999e-05 2024-09-20T22:17:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.526085+0000", 2024-09-20T22:17:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.007377027 2024-09-20T22:17:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.527179+0000", 2024-09-20T22:17:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001093903 2024-09-20T22:17:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.587 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:62141 9.1 9:8c9cf8d8:::benchmark_data_smithi060_140132_object62140:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e309)", 2024-09-20T22:17:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:12:20.531998+0000", 2024-09-20T22:17:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 314.982494127, 2024-09-20T22:17:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0013003810000001, 2024-09-20T22:17:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:17:35.588 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:17:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 62141 2024-09-20T22:17:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.531998+0000", 2024-09-20T22:17:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.531998+0000", 2024-09-20T22:17:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.589 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.532000+0000", 2024-09-20T22:17:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5719999999999999e-06 2024-09-20T22:17:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.532075+0000", 2024-09-20T22:17:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5104999999999993e-05 2024-09-20T22:17:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.532087+0000", 2024-09-20T22:17:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1990999999999999e-05 2024-09-20T22:17:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.590 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.532096+0000", 2024-09-20T22:17:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4110000000000002e-06 2024-09-20T22:17:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.532156+0000", 2024-09-20T22:17:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0000999999999997e-05 2024-09-20T22:17:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.532168+0000", 2024-09-20T22:17:35.591 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2194e-05 2024-09-20T22:17:35.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.592 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.592 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.592 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.527535+0000", 2024-09-20T22:17:35.592 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.99536664699999999 2024-09-20T22:17:35.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.592 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.592 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.592 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.527885+0000", 2024-09-20T22:17:35.592 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00034985700000000002 2024-09-20T22:17:35.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.592 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.592 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:17:35.592 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.527994+0000", 2024-09-20T22:17:35.592 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000109628 2024-09-20T22:17:35.592 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.593 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.593 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.593 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533024+0000", 2024-09-20T22:17:35.593 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.005029342 2024-09-20T22:17:35.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.593 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.593 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.593 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533086+0000", 2024-09-20T22:17:35.593 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2079e-05 2024-09-20T22:17:35.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.593 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.593 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.593 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533178+0000", 2024-09-20T22:17:35.593 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2559000000000006e-05 2024-09-20T22:17:35.593 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.593 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.594 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.594 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533243+0000", 2024-09-20T22:17:35.594 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4882999999999995e-05 2024-09-20T22:17:35.594 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.594 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.594 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.594 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533298+0000", 2024-09-20T22:17:35.594 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5111999999999997e-05 2024-09-20T22:17:35.594 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.594 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.594 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.594 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.594 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.594 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:62181 9.1 9:818888ea:::benchmark_data_smithi060_140132_object62180:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e309)", 2024-09-20T22:17:35.594 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:12:20.561567+0000", 2024-09-20T22:17:35.595 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 314.95292498600003, 2024-09-20T22:17:35.595 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.97555155000000005, 2024-09-20T22:17:35.595 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.595 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.595 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.595 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.595 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:17:35.595 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:17:35.595 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 62181 2024-09-20T22:17:35.595 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.595 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.595 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.595 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.595 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561567+0000", 2024-09-20T22:17:35.595 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.595 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.596 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.596 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.596 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561567+0000", 2024-09-20T22:17:35.596 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.596 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.596 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.596 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561569+0000", 2024-09-20T22:17:35.596 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9209999999999999e-06 2024-09-20T22:17:35.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.596 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.596 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.596 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561613+0000", 2024-09-20T22:17:35.596 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4295000000000002e-05 2024-09-20T22:17:35.596 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.596 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.597 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.597 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561639+0000", 2024-09-20T22:17:35.597 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.5177e-05 2024-09-20T22:17:35.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.597 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.597 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.597 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561648+0000", 2024-09-20T22:17:35.597 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8020000000000003e-06 2024-09-20T22:17:35.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.597 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.597 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.597 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561730+0000", 2024-09-20T22:17:35.597 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2032000000000003e-05 2024-09-20T22:17:35.597 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.597 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.597 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:35.597 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561743+0000", 2024-09-20T22:17:35.598 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2466999999999999e-05 2024-09-20T22:17:35.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.598 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.598 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.598 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.529714+0000", 2024-09-20T22:17:35.598 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.96797125799999995 2024-09-20T22:17:35.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.598 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.598 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.598 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.529848+0000", 2024-09-20T22:17:35.598 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013433100000000001 2024-09-20T22:17:35.598 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.598 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.598 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:17:35.599 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.530029+0000", 2024-09-20T22:17:35.599 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000180693 2024-09-20T22:17:35.599 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.599 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.599 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.599 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533139+0000", 2024-09-20T22:17:35.599 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0031102299999999999 2024-09-20T22:17:35.599 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.599 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.599 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.599 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533147+0000", 2024-09-20T22:17:35.599 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7330000000000003e-06 2024-09-20T22:17:35.599 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.599 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.600 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.600 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537004+0000", 2024-09-20T22:17:35.600 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0038571980000000001 2024-09-20T22:17:35.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.600 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.600 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.600 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537067+0000", 2024-09-20T22:17:35.600 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2385999999999995e-05 2024-09-20T22:17:35.600 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.600 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.600 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.600 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537119+0000", 2024-09-20T22:17:35.600 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2027000000000002e-05 2024-09-20T22:17:35.600 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.601 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.601 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.601 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.601 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.601 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:62188 9.1 9:83e4bb26:::benchmark_data_smithi060_140132_object62187:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e309)", 2024-09-20T22:17:35.601 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:12:20.566992+0000", 2024-09-20T22:17:35.601 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 314.94749970499998, 2024-09-20T22:17:35.601 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.97032768800000002, 2024-09-20T22:17:35.601 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.601 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.601 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.601 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.601 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:17:35.601 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:17:35.602 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 62188 2024-09-20T22:17:35.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.602 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.602 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.602 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.602 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.566992+0000", 2024-09-20T22:17:35.602 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.602 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.602 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.602 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.566992+0000", 2024-09-20T22:17:35.602 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.602 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.603 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.603 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.603 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.566994+0000", 2024-09-20T22:17:35.603 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9410000000000001e-06 2024-09-20T22:17:35.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.603 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.603 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.603 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.567080+0000", 2024-09-20T22:17:35.603 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5228999999999993e-05 2024-09-20T22:17:35.603 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.603 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.603 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.603 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.567104+0000", 2024-09-20T22:17:35.603 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.4309999999999999e-05 2024-09-20T22:17:35.604 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.604 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.604 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.604 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.567114+0000", 2024-09-20T22:17:35.604 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0487e-05 2024-09-20T22:17:35.604 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.604 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.604 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.604 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.567203+0000", 2024-09-20T22:17:35.604 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8109000000000006e-05 2024-09-20T22:17:35.604 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.604 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.605 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:35.605 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.567216+0000", 2024-09-20T22:17:35.605 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3040000000000001e-05 2024-09-20T22:17:35.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.605 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.605 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.605 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.531707+0000", 2024-09-20T22:17:35.605 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.96449152100000002 2024-09-20T22:17:35.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.605 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.605 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.605 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.531800+0000", 2024-09-20T22:17:35.605 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3015000000000003e-05 2024-09-20T22:17:35.605 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.606 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.606 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:17:35.606 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.531912+0000", 2024-09-20T22:17:35.606 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000111776 2024-09-20T22:17:35.606 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.606 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.606 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.606 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537181+0000", 2024-09-20T22:17:35.606 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0052689709999999999 2024-09-20T22:17:35.606 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.606 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.606 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.606 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537198+0000", 2024-09-20T22:17:35.606 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7142999999999998e-05 2024-09-20T22:17:35.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.607 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.607 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.607 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537256+0000", 2024-09-20T22:17:35.607 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8425999999999999e-05 2024-09-20T22:17:35.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.607 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.607 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.607 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537293+0000", 2024-09-20T22:17:35.607 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6313000000000003e-05 2024-09-20T22:17:35.607 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.607 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.607 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.607 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537320+0000", 2024-09-20T22:17:35.608 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7407e-05 2024-09-20T22:17:35.608 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.608 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.608 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.608 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.608 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.608 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:150704 10.f 10:fb8a8281:::benchmark_data_smithi060_167766_object25912:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e417)", 2024-09-20T22:17:35.608 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:09.399845+0000", 2024-09-20T22:17:35.608 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 26.114647187999999, 2024-09-20T22:17:35.608 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.082665131000000003, 2024-09-20T22:17:35.608 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.608 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.608 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.609 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.609 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:35.609 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:35.609 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 150704 2024-09-20T22:17:35.609 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.609 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.609 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.609 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.609 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.399845+0000", 2024-09-20T22:17:35.609 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.609 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.609 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.609 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.609 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.399845+0000", 2024-09-20T22:17:35.610 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.610 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.610 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.610 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.399846+0000", 2024-09-20T22:17:35.610 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.096e-06 2024-09-20T22:17:35.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.610 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.610 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.610 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.399851+0000", 2024-09-20T22:17:35.610 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6949999999999999e-06 2024-09-20T22:17:35.610 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.610 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.610 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.610 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.399861+0000", 2024-09-20T22:17:35.611 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0436e-05 2024-09-20T22:17:35.611 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.611 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.611 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.611 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.399869+0000", 2024-09-20T22:17:35.611 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6799999999999993e-06 2024-09-20T22:17:35.611 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.611 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.611 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.611 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.399912+0000", 2024-09-20T22:17:35.612 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2660000000000002e-05 2024-09-20T22:17:35.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.612 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.612 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.612 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.400020+0000", 2024-09-20T22:17:35.612 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010819099999999999 2024-09-20T22:17:35.612 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.612 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.612 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,3", 2024-09-20T22:17:35.612 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.400432+0000", 2024-09-20T22:17:35.612 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000412679 2024-09-20T22:17:35.613 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.613 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.613 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.613 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.407260+0000", 2024-09-20T22:17:35.613 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.006828036 2024-09-20T22:17:35.613 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.613 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.613 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.613 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.407275+0000", 2024-09-20T22:17:35.613 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4115e-05 2024-09-20T22:17:35.613 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.613 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.613 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.614 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.482445+0000", 2024-09-20T22:17:35.614 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.075170735000000002 2024-09-20T22:17:35.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.614 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.614 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.614 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.482478+0000", 2024-09-20T22:17:35.614 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.3130999999999998e-05 2024-09-20T22:17:35.614 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.614 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.614 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.614 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.482510+0000", 2024-09-20T22:17:35.614 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1677000000000002e-05 2024-09-20T22:17:35.614 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.614 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.615 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.615 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.615 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.615 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:150706 10.f 10:f976a74d:::benchmark_data_smithi060_167766_object25914:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e417)", 2024-09-20T22:17:35.615 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:09.400162+0000", 2024-09-20T22:17:35.615 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 26.114330106000001, 2024-09-20T22:17:35.615 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.082527416000000006, 2024-09-20T22:17:35.615 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.615 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.615 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.615 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.615 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:35.615 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:35.615 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 150706 2024-09-20T22:17:35.616 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.616 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.616 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.616 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.616 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.400162+0000", 2024-09-20T22:17:35.616 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.616 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.616 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.616 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.616 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.400162+0000", 2024-09-20T22:17:35.616 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.616 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.616 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.616 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.616 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.400163+0000", 2024-09-20T22:17:35.617 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4e-07 2024-09-20T22:17:35.617 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.617 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.617 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.617 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.400168+0000", 2024-09-20T22:17:35.617 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8620000000000002e-06 2024-09-20T22:17:35.617 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.617 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.617 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.617 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.400181+0000", 2024-09-20T22:17:35.617 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2935999999999999e-05 2024-09-20T22:17:35.617 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.617 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.618 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.618 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.400276+0000", 2024-09-20T22:17:35.618 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4901000000000005e-05 2024-09-20T22:17:35.618 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.618 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.618 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.618 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.404172+0000", 2024-09-20T22:17:35.618 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0038963280000000001 2024-09-20T22:17:35.618 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.618 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.618 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.618 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.404291+0000", 2024-09-20T22:17:35.618 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011941400000000001 2024-09-20T22:17:35.618 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.619 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.619 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,3", 2024-09-20T22:17:35.619 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.404416+0000", 2024-09-20T22:17:35.619 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012511400000000001 2024-09-20T22:17:35.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.619 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.619 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.619 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.407279+0000", 2024-09-20T22:17:35.619 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028625759999999999 2024-09-20T22:17:35.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.619 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.619 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.619 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.408071+0000", 2024-09-20T22:17:35.619 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00079175700000000003 2024-09-20T22:17:35.619 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.619 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.620 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.620 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.482567+0000", 2024-09-20T22:17:35.620 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.074496725 2024-09-20T22:17:35.620 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.620 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.620 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.620 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.482645+0000", 2024-09-20T22:17:35.620 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7537e-05 2024-09-20T22:17:35.620 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.620 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.620 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.620 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:09.482689+0000", 2024-09-20T22:17:35.620 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4425999999999998e-05 2024-09-20T22:17:35.620 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.620 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.621 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.621 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.621 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:165542 10.a 10:585114ef:::benchmark_data_smithi060_167766_object40750:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e423)", 2024-09-20T22:17:35.621 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:22.534727+0000", 2024-09-20T22:17:35.621 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 12.979765506, 2024-09-20T22:17:35.621 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.095584924000000002, 2024-09-20T22:17:35.621 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.621 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.621 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.621 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.621 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:35.621 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:35.621 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 165542 2024-09-20T22:17:35.621 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.622 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.622 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.622 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.622 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.534727+0000", 2024-09-20T22:17:35.622 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.622 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.622 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.622 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.534727+0000", 2024-09-20T22:17:35.622 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.622 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.622 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.622 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.622 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.534728+0000", 2024-09-20T22:17:35.623 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1399999999999995e-07 2024-09-20T22:17:35.623 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.623 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.623 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.623 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.534732+0000", 2024-09-20T22:17:35.623 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7010000000000004e-06 2024-09-20T22:17:35.623 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.623 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.623 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.623 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.534744+0000", 2024-09-20T22:17:35.623 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1761e-05 2024-09-20T22:17:35.623 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.623 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.623 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.623 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.534751+0000", 2024-09-20T22:17:35.623 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9179999999999997e-06 2024-09-20T22:17:35.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.624 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.624 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.624 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.534804+0000", 2024-09-20T22:17:35.624 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3186999999999997e-05 2024-09-20T22:17:35.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.624 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.624 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.624 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.534907+0000", 2024-09-20T22:17:35.624 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010288 2024-09-20T22:17:35.624 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.624 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.624 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,5", 2024-09-20T22:17:35.624 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.535007+0000", 2024-09-20T22:17:35.624 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9933999999999996e-05 2024-09-20T22:17:35.625 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.625 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.625 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.625 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.543853+0000", 2024-09-20T22:17:35.625 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0088461789999999992 2024-09-20T22:17:35.625 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.625 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.625 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.625 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.543997+0000", 2024-09-20T22:17:35.625 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014426200000000001 2024-09-20T22:17:35.625 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.625 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.625 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.625 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.630202+0000", 2024-09-20T22:17:35.625 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.086204728999999994 2024-09-20T22:17:35.625 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.626 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.626 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.626 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.630258+0000", 2024-09-20T22:17:35.626 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6039999999999999e-05 2024-09-20T22:17:35.626 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.626 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.626 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.626 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.630312+0000", 2024-09-20T22:17:35.626 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3418999999999998e-05 2024-09-20T22:17:35.626 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.626 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.626 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.626 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.626 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.627 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:165543 10.a 10:5904f3af:::benchmark_data_smithi060_167766_object40751:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e423)", 2024-09-20T22:17:35.627 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:22.536965+0000", 2024-09-20T22:17:35.627 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 12.97752751, 2024-09-20T22:17:35.627 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.093470565000000005, 2024-09-20T22:17:35.627 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.627 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.627 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.627 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.627 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:35.627 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:35.627 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 165543 2024-09-20T22:17:35.627 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.627 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.627 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.627 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.628 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.536965+0000", 2024-09-20T22:17:35.628 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.628 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.628 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.628 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.628 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.536965+0000", 2024-09-20T22:17:35.628 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.628 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.628 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.628 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.628 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.536966+0000", 2024-09-20T22:17:35.628 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1540000000000001e-06 2024-09-20T22:17:35.628 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.628 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.628 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.628 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.536971+0000", 2024-09-20T22:17:35.628 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.186e-06 2024-09-20T22:17:35.629 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.629 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.629 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.629 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.537020+0000", 2024-09-20T22:17:35.629 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9478000000000003e-05 2024-09-20T22:17:35.629 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.629 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.629 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.629 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.537030+0000", 2024-09-20T22:17:35.629 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1600000000000004e-06 2024-09-20T22:17:35.629 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.629 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.629 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.629 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.537702+0000", 2024-09-20T22:17:35.629 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00067253799999999998 2024-09-20T22:17:35.630 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.630 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.630 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.630 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.537799+0000", 2024-09-20T22:17:35.630 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6871000000000001e-05 2024-09-20T22:17:35.630 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.630 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.630 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,5", 2024-09-20T22:17:35.630 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.537903+0000", 2024-09-20T22:17:35.630 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000103483 2024-09-20T22:17:35.630 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.630 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.630 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.630 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.543985+0000", 2024-09-20T22:17:35.630 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0060824199999999998 2024-09-20T22:17:35.630 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.631 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.631 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.631 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.544005+0000", 2024-09-20T22:17:35.631 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9658999999999999e-05 2024-09-20T22:17:35.631 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.631 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.631 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.631 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.630349+0000", 2024-09-20T22:17:35.631 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.086344827999999998 2024-09-20T22:17:35.631 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.631 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.631 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.631 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.630400+0000", 2024-09-20T22:17:35.631 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0649000000000002e-05 2024-09-20T22:17:35.631 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.632 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.632 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.632 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.630435+0000", 2024-09-20T22:17:35.632 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5138999999999997e-05 2024-09-20T22:17:35.632 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.632 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.632 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.632 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.632 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.632 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:165554 10.7 10:e5cd7cf0:::benchmark_data_smithi060_167766_object40762:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e423)", 2024-09-20T22:17:35.632 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:22.546697+0000", 2024-09-20T22:17:35.632 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 12.967794703999999, 2024-09-20T22:17:35.632 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.085029096999999998, 2024-09-20T22:17:35.632 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.632 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.632 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.632 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.633 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:35.633 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:35.633 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 165554 2024-09-20T22:17:35.633 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.633 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.633 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.633 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.633 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.546697+0000", 2024-09-20T22:17:35.633 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.633 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.633 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.633 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.633 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.546697+0000", 2024-09-20T22:17:35.633 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.633 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.633 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.634 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.634 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.546699+0000", 2024-09-20T22:17:35.634 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.063e-06 2024-09-20T22:17:35.634 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.634 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.634 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.634 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.546703+0000", 2024-09-20T22:17:35.634 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3810000000000001e-06 2024-09-20T22:17:35.634 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.634 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.634 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.634 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.546712+0000", 2024-09-20T22:17:35.634 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1889999999999996e-06 2024-09-20T22:17:35.634 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.634 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.634 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.635 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.546717+0000", 2024-09-20T22:17:35.635 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9860000000000002e-06 2024-09-20T22:17:35.635 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.635 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.635 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.635 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.546777+0000", 2024-09-20T22:17:35.635 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9500999999999998e-05 2024-09-20T22:17:35.635 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.635 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.635 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.635 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.546893+0000", 2024-09-20T22:17:35.635 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000116069 2024-09-20T22:17:35.635 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.635 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.635 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:17:35.636 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.547010+0000", 2024-09-20T22:17:35.636 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000117699 2024-09-20T22:17:35.636 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.636 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.636 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.636 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.549860+0000", 2024-09-20T22:17:35.636 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028497290000000001 2024-09-20T22:17:35.636 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.636 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.636 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.636 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.550048+0000", 2024-09-20T22:17:35.636 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00018781099999999999 2024-09-20T22:17:35.636 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.636 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.636 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.636 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.631642+0000", 2024-09-20T22:17:35.637 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.081593691999999995 2024-09-20T22:17:35.637 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.637 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.637 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.637 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.631691+0000", 2024-09-20T22:17:35.637 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9731999999999998e-05 2024-09-20T22:17:35.637 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.637 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.637 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:35.638 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.638 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.631727+0000", 2024-09-20T22:17:35.638 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5244999999999999e-05 2024-09-20T22:17:35.638 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.638 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.638 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.639 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.639 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.639 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:165555 10.f 10:fe3720c5:::benchmark_data_smithi060_167766_object40763:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e423)", 2024-09-20T22:17:35.639 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:22.546747+0000", 2024-09-20T22:17:35.639 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 12.96774557, 2024-09-20T22:17:35.639 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.085118726000000006, 2024-09-20T22:17:35.639 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.639 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.639 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.639 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.639 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:35.639 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:35.639 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 165555 2024-09-20T22:17:35.640 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.640 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.640 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.640 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.640 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.546747+0000", 2024-09-20T22:17:35.640 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.640 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.640 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.640 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.640 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.546747+0000", 2024-09-20T22:17:35.640 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.640 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.640 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.640 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.640 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.546747+0000", 2024-09-20T22:17:35.640 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2499999999999995e-07 2024-09-20T22:17:35.641 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.641 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.641 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.641 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.546750+0000", 2024-09-20T22:17:35.641 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.3119999999999998e-06 2024-09-20T22:17:35.641 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.641 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.641 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.641 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.546759+0000", 2024-09-20T22:17:35.641 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4929999999999998e-06 2024-09-20T22:17:35.641 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.641 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.641 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.642 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.546765+0000", 2024-09-20T22:17:35.642 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6389999999999997e-06 2024-09-20T22:17:35.642 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.642 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.642 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.642 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.546799+0000", 2024-09-20T22:17:35.642 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4730999999999997e-05 2024-09-20T22:17:35.642 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.642 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.642 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.642 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.546869+0000", 2024-09-20T22:17:35.642 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9654999999999996e-05 2024-09-20T22:17:35.642 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.643 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.643 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,3", 2024-09-20T22:17:35.643 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.547035+0000", 2024-09-20T22:17:35.643 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00016569499999999999 2024-09-20T22:17:35.643 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.643 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.643 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.643 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.549980+0000", 2024-09-20T22:17:35.643 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0029451519999999999 2024-09-20T22:17:35.643 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.643 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.643 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.643 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.631763+0000", 2024-09-20T22:17:35.643 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.081782808999999998 2024-09-20T22:17:35.644 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.644 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.644 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.644 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.631794+0000", 2024-09-20T22:17:35.644 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1606999999999997e-05 2024-09-20T22:17:35.644 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.644 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.644 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.644 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.631834+0000", 2024-09-20T22:17:35.644 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0161999999999999e-05 2024-09-20T22:17:35.644 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.644 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.644 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.644 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.631865+0000", 2024-09-20T22:17:35.645 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0945999999999998e-05 2024-09-20T22:17:35.645 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.645 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.645 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.645 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.645 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.645 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:165557 10.b 10:dac8cff9:::benchmark_data_smithi060_167766_object40765:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e423)", 2024-09-20T22:17:35.645 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:22.547651+0000", 2024-09-20T22:17:35.645 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 12.966841500999999, 2024-09-20T22:17:35.645 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.082942911999999994, 2024-09-20T22:17:35.645 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:35.645 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:35.645 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:35.645 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:35.646 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:35.646 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:35.646 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 165557 2024-09-20T22:17:35.646 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.646 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:35.646 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.646 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:35.646 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.547651+0000", 2024-09-20T22:17:35.646 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.646 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.646 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.646 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:35.646 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.547651+0000", 2024-09-20T22:17:35.646 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:35.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.647 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.647 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:35.647 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.547651+0000", 2024-09-20T22:17:35.647 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4899999999999995e-07 2024-09-20T22:17:35.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.647 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.647 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:35.647 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.547655+0000", 2024-09-20T22:17:35.647 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5719999999999999e-06 2024-09-20T22:17:35.647 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.647 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.647 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:35.647 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.547663+0000", 2024-09-20T22:17:35.647 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5990000000000002e-06 2024-09-20T22:17:35.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.648 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.648 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:35.648 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.547669+0000", 2024-09-20T22:17:35.648 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1789999999999999e-06 2024-09-20T22:17:35.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.648 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.648 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:35.648 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.547712+0000", 2024-09-20T22:17:35.648 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3066999999999999e-05 2024-09-20T22:17:35.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.648 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.648 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:35.648 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.547782+0000", 2024-09-20T22:17:35.648 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0652000000000001e-05 2024-09-20T22:17:35.648 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.648 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.649 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1", 2024-09-20T22:17:35.649 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.547890+0000", 2024-09-20T22:17:35.649 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010798600000000001 2024-09-20T22:17:35.649 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.649 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.649 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.649 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.551227+0000", 2024-09-20T22:17:35.649 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003336866 2024-09-20T22:17:35.649 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.649 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.649 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:35.649 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.630505+0000", 2024-09-20T22:17:35.649 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.079278055 2024-09-20T22:17:35.649 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.649 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.649 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:35.650 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.630518+0000", 2024-09-20T22:17:35.650 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2425e-05 2024-09-20T22:17:35.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.650 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.650 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:35.650 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.630558+0000", 2024-09-20T22:17:35.650 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0686000000000003e-05 2024-09-20T22:17:35.650 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:35.650 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:35.650 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:35.650 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:22.630594+0000", 2024-09-20T22:17:35.650 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5176000000000003e-05 2024-09-20T22:17:35.650 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.650 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.650 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.650 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:35.651 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:35.651 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:17:35.693 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:35.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:35.694 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:35.795 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:35.832 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:35.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:35.833 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:35.934 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:35.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:35.977 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:35.977 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:36.077 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:36.109 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:17:36.124 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:36.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:36.125 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:36.226 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:36.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:36.269 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:36.269 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:36.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[29694]: 4.f scrub starts 2024-09-20T22:17:36.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[29694]: 4.f scrub ok 2024-09-20T22:17:36.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[29694]: 3.b scrub starts 2024-09-20T22:17:36.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[29694]: 3.b scrub ok 2024-09-20T22:17:36.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2408055865' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:36.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[29694]: 5.f scrub starts 2024-09-20T22:17:36.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[29694]: 5.f scrub ok 2024-09-20T22:17:36.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[29694]: 10.9 scrub ok 2024-09-20T22:17:36.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4215419447' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:36.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:36.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:17:36.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[40290]: 4.f scrub starts 2024-09-20T22:17:36.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[40290]: 4.f scrub ok 2024-09-20T22:17:36.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[40290]: 3.b scrub starts 2024-09-20T22:17:36.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[40290]: 3.b scrub ok 2024-09-20T22:17:36.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2408055865' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:36.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[40290]: 5.f scrub starts 2024-09-20T22:17:36.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[40290]: 5.f scrub ok 2024-09-20T22:17:36.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[40290]: 10.9 scrub ok 2024-09-20T22:17:36.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4215419447' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:36.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:36.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:36 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:17:36.370 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:36.414 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:36.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:36.416 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:36.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:36 smithi067 ceph-mon[33511]: 4.f scrub starts 2024-09-20T22:17:36.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:36 smithi067 ceph-mon[33511]: 4.f scrub ok 2024-09-20T22:17:36.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:36 smithi067 ceph-mon[33511]: 3.b scrub starts 2024-09-20T22:17:36.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:36 smithi067 ceph-mon[33511]: 3.b scrub ok 2024-09-20T22:17:36.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:36 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2408055865' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:36.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:36 smithi067 ceph-mon[33511]: 5.f scrub starts 2024-09-20T22:17:36.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:36 smithi067 ceph-mon[33511]: 5.f scrub ok 2024-09-20T22:17:36.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:36 smithi067 ceph-mon[33511]: 10.9 scrub ok 2024-09-20T22:17:36.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:36 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4215419447' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:36.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:36 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:36.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:36 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:17:36.516 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:36.549 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:36.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:36.551 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:36.651 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:36.685 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:36.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:36.686 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:36.787 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:36.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:36.834 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:36.835 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:36.835 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T22:17:36.935 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:36.970 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:36.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:36.971 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:37.072 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:37.113 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:37.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:37.114 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:37.214 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:37.254 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:37.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:37.255 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:37.263 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:17:37.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:37 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:17:37.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:37 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:17:37.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:37 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:17:37.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:37 smithi060 ceph-mon[29694]: osdmap e426: 8 total, 8 up, 8 in 2024-09-20T22:17:37.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:37 smithi060 ceph-mon[29694]: pgmap v784: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 4.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 854 op/s 2024-09-20T22:17:37.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:37 smithi060 ceph-mon[29694]: 6.1 scrub starts 2024-09-20T22:17:37.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:37 smithi060 ceph-mon[29694]: 6.1 scrub ok 2024-09-20T22:17:37.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:37 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/242124101' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "31"}]: dispatch 2024-09-20T22:17:37.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:37 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:17:37.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:37 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:17:37.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:37 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:17:37.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:37 smithi060 ceph-mon[40290]: osdmap e426: 8 total, 8 up, 8 in 2024-09-20T22:17:37.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:37 smithi060 ceph-mon[40290]: pgmap v784: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 4.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 854 op/s 2024-09-20T22:17:37.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:37 smithi060 ceph-mon[40290]: 6.1 scrub starts 2024-09-20T22:17:37.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:37 smithi060 ceph-mon[40290]: 6.1 scrub ok 2024-09-20T22:17:37.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:37 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/242124101' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "31"}]: dispatch 2024-09-20T22:17:37.356 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:37.400 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:37 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:17:37.400 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:37 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:17:37.400 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:37 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:17:37.400 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:37 smithi067 ceph-mon[33511]: osdmap e426: 8 total, 8 up, 8 in 2024-09-20T22:17:37.400 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:37 smithi067 ceph-mon[33511]: pgmap v784: 168 pgs: 1 active+clean+scrubbing, 167 active+clean; 4.4 GiB data, 18 GiB used, 697 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 854 op/s 2024-09-20T22:17:37.400 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:37 smithi067 ceph-mon[33511]: 6.1 scrub starts 2024-09-20T22:17:37.400 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:37 smithi067 ceph-mon[33511]: 6.1 scrub ok 2024-09-20T22:17:37.400 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:37 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/242124101' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "31"}]: dispatch 2024-09-20T22:17:37.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:37.402 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:37.402 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:37.503 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:37.528 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:37.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:37.529 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:37.599 INFO:teuthology.orchestra.run.smithi060.stderr:set pool 1 pg_num to 31 2024-09-20T22:17:37.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:37.674 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:37.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:37.676 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:37.776 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:37.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:37.815 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:37.815 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:37.915 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:37.957 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:37.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:37.958 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:38.059 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:38.091 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:38.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:38.092 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:38.193 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:38.232 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:38.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:38.233 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:38.334 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:38.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:38 smithi060 ceph-mon[29694]: 3.9 scrub starts 2024-09-20T22:17:38.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:38 smithi060 ceph-mon[29694]: 3.9 scrub ok 2024-09-20T22:17:38.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:38 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/242124101' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "31"}]': finished 2024-09-20T22:17:38.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:38 smithi060 ceph-mon[29694]: osdmap e427: 8 total, 8 up, 8 in 2024-09-20T22:17:38.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:38 smithi060 ceph-mon[29694]: 5.2 scrub starts 2024-09-20T22:17:38.363 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:38 smithi060 ceph-mon[29694]: 5.2 scrub ok 2024-09-20T22:17:38.367 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:38 smithi060 ceph-mon[40290]: 3.9 scrub starts 2024-09-20T22:17:38.367 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:38 smithi060 ceph-mon[40290]: 3.9 scrub ok 2024-09-20T22:17:38.367 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:38 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/242124101' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "31"}]': finished 2024-09-20T22:17:38.367 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:38 smithi060 ceph-mon[40290]: osdmap e427: 8 total, 8 up, 8 in 2024-09-20T22:17:38.367 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:38 smithi060 ceph-mon[40290]: 5.2 scrub starts 2024-09-20T22:17:38.367 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:38 smithi060 ceph-mon[40290]: 5.2 scrub ok 2024-09-20T22:17:38.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:38.368 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:38.368 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:38.469 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:38.487 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:17:38.511 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:38.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:38.512 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:38.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:38 smithi067 ceph-mon[33511]: 3.9 scrub starts 2024-09-20T22:17:38.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:38 smithi067 ceph-mon[33511]: 3.9 scrub ok 2024-09-20T22:17:38.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:38 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/242124101' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num", "val": "31"}]': finished 2024-09-20T22:17:38.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:38 smithi067 ceph-mon[33511]: osdmap e427: 8 total, 8 up, 8 in 2024-09-20T22:17:38.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:38 smithi067 ceph-mon[33511]: 5.2 scrub starts 2024-09-20T22:17:38.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:38 smithi067 ceph-mon[33511]: 5.2 scrub ok 2024-09-20T22:17:38.613 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:38.653 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:38.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:38.654 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:38.755 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:38.792 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:38.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:38.793 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:38.894 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:38.947 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:38.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:38.950 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:38.950 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:39.051 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:39.087 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:39.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:39.088 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:39.189 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:39.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:39.214 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:39.214 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:39.315 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:39.337 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:39.338 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:39.338 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:39.362 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:17:39.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:39 smithi060 ceph-mon[29694]: 3.d scrub starts 2024-09-20T22:17:39.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:39 smithi060 ceph-mon[29694]: 3.d scrub ok 2024-09-20T22:17:39.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:39 smithi060 ceph-mon[29694]: 4.3 scrub starts 2024-09-20T22:17:39.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:39 smithi060 ceph-mon[29694]: 4.3 scrub ok 2024-09-20T22:17:39.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:39 smithi060 ceph-mon[29694]: 4.1d scrub starts 2024-09-20T22:17:39.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:39 smithi060 ceph-mon[29694]: 4.1d scrub ok 2024-09-20T22:17:39.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:39 smithi060 ceph-mon[29694]: pgmap v786: 168 pgs: 168 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.37k op/s 2024-09-20T22:17:39.391 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:39 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-20T22:17:39.392 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:39 smithi060 ceph-mon[40290]: 3.d scrub starts 2024-09-20T22:17:39.392 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:39 smithi060 ceph-mon[40290]: 3.d scrub ok 2024-09-20T22:17:39.392 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:39 smithi060 ceph-mon[40290]: 4.3 scrub starts 2024-09-20T22:17:39.392 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:39 smithi060 ceph-mon[40290]: 4.3 scrub ok 2024-09-20T22:17:39.392 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:39 smithi060 ceph-mon[40290]: 4.1d scrub starts 2024-09-20T22:17:39.392 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:39 smithi060 ceph-mon[40290]: 4.1d scrub ok 2024-09-20T22:17:39.392 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:39 smithi060 ceph-mon[40290]: pgmap v786: 168 pgs: 168 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.37k op/s 2024-09-20T22:17:39.392 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:39 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-20T22:17:39.438 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:39.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:39.477 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:39.478 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:39.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:39 smithi067 ceph-mon[33511]: 3.d scrub starts 2024-09-20T22:17:39.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:39 smithi067 ceph-mon[33511]: 3.d scrub ok 2024-09-20T22:17:39.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:39 smithi067 ceph-mon[33511]: 4.3 scrub starts 2024-09-20T22:17:39.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:39 smithi067 ceph-mon[33511]: 4.3 scrub ok 2024-09-20T22:17:39.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:39 smithi067 ceph-mon[33511]: 4.1d scrub starts 2024-09-20T22:17:39.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:39 smithi067 ceph-mon[33511]: 4.1d scrub ok 2024-09-20T22:17:39.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:39 smithi067 ceph-mon[33511]: pgmap v786: 168 pgs: 168 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.37k op/s 2024-09-20T22:17:39.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:39 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-20T22:17:39.578 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:39.604 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:39.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:39.605 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:39.705 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:39.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:39.742 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:39.742 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:39.843 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:39.869 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:39.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:39.871 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:39.972 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:40.004 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:40.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:40.005 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:40.106 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:40.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:40.147 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:40.147 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:40.248 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:40.279 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:40.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:40.280 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:40.350 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:40 smithi060 ceph-mon[40290]: 1.2 scrub starts 2024-09-20T22:17:40.350 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:40 smithi060 ceph-mon[40290]: 1.2 scrub ok 2024-09-20T22:17:40.350 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:40 smithi060 ceph-mon[40290]: 4.16 scrub starts 2024-09-20T22:17:40.350 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:40 smithi060 ceph-mon[40290]: 4.16 scrub ok 2024-09-20T22:17:40.350 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:40 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-20T22:17:40.350 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:40 smithi060 ceph-mon[40290]: osdmap e428: 8 total, 8 up, 8 in 2024-09-20T22:17:40.350 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:40 smithi060 ceph-mon[29694]: 1.2 scrub starts 2024-09-20T22:17:40.350 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:40 smithi060 ceph-mon[29694]: 1.2 scrub ok 2024-09-20T22:17:40.351 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:40 smithi060 ceph-mon[29694]: 4.16 scrub starts 2024-09-20T22:17:40.351 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:40 smithi060 ceph-mon[29694]: 4.16 scrub ok 2024-09-20T22:17:40.351 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:40 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-20T22:17:40.351 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:40 smithi060 ceph-mon[29694]: osdmap e428: 8 total, 8 up, 8 in 2024-09-20T22:17:40.381 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:40.414 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:40.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:40.417 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:40.430 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:40 smithi067 ceph-mon[33511]: 1.2 scrub starts 2024-09-20T22:17:40.431 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:40 smithi067 ceph-mon[33511]: 1.2 scrub ok 2024-09-20T22:17:40.431 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:40 smithi067 ceph-mon[33511]: 4.16 scrub starts 2024-09-20T22:17:40.431 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:40 smithi067 ceph-mon[33511]: 4.16 scrub ok 2024-09-20T22:17:40.431 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:40 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-20T22:17:40.431 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:40 smithi067 ceph-mon[33511]: osdmap e428: 8 total, 8 up, 8 in 2024-09-20T22:17:40.518 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:40.544 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:40.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:40.545 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:40.607 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:17:40.608 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:17:40.617 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:40.629 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:17:40.629 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:17:40.646 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:40.652 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:17:40.652 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:17:40.668 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:40.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:40.669 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:40.669 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:17:40.670 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:17:40.686 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:17:40.687 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:17:40.703 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:17:40.703 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:17:40.769 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:40.771 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:17:40.771 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:17:40.797 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:40.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:40.798 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:40.800 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:17:40.800 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:17:40.898 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:40.939 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:40.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:40.940 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:41.041 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:41.086 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:41.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:41.087 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:41.096 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:17:41.096 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:17:41.188 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:41.232 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:41.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:41.233 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:41.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:41 smithi067 ceph-mon[33511]: 6.f scrub starts 2024-09-20T22:17:41.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:41 smithi067 ceph-mon[33511]: 6.f scrub ok 2024-09-20T22:17:41.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:41 smithi067 ceph-mon[33511]: osdmap e429: 8 total, 8 up, 8 in 2024-09-20T22:17:41.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:41 smithi067 ceph-mon[33511]: 1.b scrub starts 2024-09-20T22:17:41.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:41 smithi067 ceph-mon[33511]: 1.b scrub ok 2024-09-20T22:17:41.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:41 smithi067 ceph-mon[33511]: pgmap v789: 178 pgs: 10 unknown, 168 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 0 B/s wr, 1.16k op/s 2024-09-20T22:17:41.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:41 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-20T22:17:41.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:41 smithi067 ceph-mon[33511]: 1.a scrub starts 2024-09-20T22:17:41.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:41 smithi067 ceph-mon[33511]: 1.a scrub ok 2024-09-20T22:17:41.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:41 smithi067 ceph-mon[33511]: 1.1e deep-scrub starts 2024-09-20T22:17:41.266 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:41 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1439106602' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:41.266 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:41 smithi067 ceph-mon[33511]: 1.c scrub starts 2024-09-20T22:17:41.266 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:41 smithi067 ceph-mon[33511]: 1.c scrub ok 2024-09-20T22:17:41.266 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:41 smithi067 ceph-mon[33511]: 1.6 scrub starts 2024-09-20T22:17:41.266 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:41 smithi067 ceph-mon[33511]: 1.6 scrub ok 2024-09-20T22:17:41.334 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:41.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:41.375 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:41.375 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:41.476 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:41.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:41.519 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:41.520 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:41.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[29694]: 6.f scrub starts 2024-09-20T22:17:41.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[29694]: 6.f scrub ok 2024-09-20T22:17:41.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[29694]: osdmap e429: 8 total, 8 up, 8 in 2024-09-20T22:17:41.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[29694]: 1.b scrub starts 2024-09-20T22:17:41.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[29694]: 1.b scrub ok 2024-09-20T22:17:41.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[29694]: pgmap v789: 178 pgs: 10 unknown, 168 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 0 B/s wr, 1.16k op/s 2024-09-20T22:17:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-20T22:17:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[29694]: 1.a scrub starts 2024-09-20T22:17:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[29694]: 1.a scrub ok 2024-09-20T22:17:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[29694]: 1.1e deep-scrub starts 2024-09-20T22:17:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1439106602' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:41.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[29694]: 1.c scrub starts 2024-09-20T22:17:41.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[29694]: 1.c scrub ok 2024-09-20T22:17:41.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[29694]: 1.6 scrub starts 2024-09-20T22:17:41.538 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[29694]: 1.6 scrub ok 2024-09-20T22:17:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[40290]: 6.f scrub starts 2024-09-20T22:17:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[40290]: 6.f scrub ok 2024-09-20T22:17:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[40290]: osdmap e429: 8 total, 8 up, 8 in 2024-09-20T22:17:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[40290]: 1.b scrub starts 2024-09-20T22:17:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[40290]: 1.b scrub ok 2024-09-20T22:17:41.539 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[40290]: pgmap v789: 178 pgs: 10 unknown, 168 active+clean; 4.1 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 0 B/s wr, 1.16k op/s 2024-09-20T22:17:41.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-20T22:17:41.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[40290]: 1.a scrub starts 2024-09-20T22:17:41.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[40290]: 1.a scrub ok 2024-09-20T22:17:41.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[40290]: 1.1e deep-scrub starts 2024-09-20T22:17:41.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1439106602' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:41.540 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[40290]: 1.c scrub starts 2024-09-20T22:17:41.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[40290]: 1.c scrub ok 2024-09-20T22:17:41.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[40290]: 1.6 scrub starts 2024-09-20T22:17:41.541 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:41 smithi060 ceph-mon[40290]: 1.6 scrub ok 2024-09-20T22:17:41.620 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:41.661 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:41.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:41.663 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:41.764 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:41.808 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:41.809 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:41.809 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:41.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:41.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:41.950 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:41.950 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:41.952 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T22:17:42.051 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:42.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:42.088 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:42.089 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:42.189 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:42.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:42.229 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:42.229 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:42.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:42 smithi067 ceph-mon[33511]: 1.1e deep-scrub ok 2024-09-20T22:17:42.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:42 smithi067 ceph-mon[33511]: 2.1 scrub starts 2024-09-20T22:17:42.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:42 smithi067 ceph-mon[33511]: 2.1 scrub ok 2024-09-20T22:17:42.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:42 smithi067 ceph-mon[33511]: 1.d scrub starts 2024-09-20T22:17:42.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:42 smithi067 ceph-mon[33511]: 1.d scrub ok 2024-09-20T22:17:42.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:42 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-20T22:17:42.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:42 smithi067 ceph-mon[33511]: osdmap e430: 8 total, 8 up, 8 in 2024-09-20T22:17:42.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:42 smithi067 ceph-mon[33511]: 3.1d scrub starts 2024-09-20T22:17:42.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:42 smithi067 ceph-mon[33511]: 3.1d scrub ok 2024-09-20T22:17:42.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:42 smithi067 ceph-mon[33511]: osdmap e431: 8 total, 8 up, 8 in 2024-09-20T22:17:42.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:42 smithi067 ceph-mon[33511]: 1.7 scrub starts 2024-09-20T22:17:42.265 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:42 smithi067 ceph-mon[33511]: 1.7 scrub ok 2024-09-20T22:17:42.331 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:42.364 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:17:42.375 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:42.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:42.377 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:42.477 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:42.517 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:42.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:42.518 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:42.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[29694]: 1.1e deep-scrub ok 2024-09-20T22:17:42.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[29694]: 2.1 scrub starts 2024-09-20T22:17:42.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[29694]: 2.1 scrub ok 2024-09-20T22:17:42.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[29694]: 1.d scrub starts 2024-09-20T22:17:42.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[29694]: 1.d scrub ok 2024-09-20T22:17:42.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-20T22:17:42.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[29694]: osdmap e430: 8 total, 8 up, 8 in 2024-09-20T22:17:42.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[29694]: 3.1d scrub starts 2024-09-20T22:17:42.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[29694]: 3.1d scrub ok 2024-09-20T22:17:42.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[29694]: osdmap e431: 8 total, 8 up, 8 in 2024-09-20T22:17:42.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[29694]: 1.7 scrub starts 2024-09-20T22:17:42.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[29694]: 1.7 scrub ok 2024-09-20T22:17:42.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[40290]: 1.1e deep-scrub ok 2024-09-20T22:17:42.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[40290]: 2.1 scrub starts 2024-09-20T22:17:42.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[40290]: 2.1 scrub ok 2024-09-20T22:17:42.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[40290]: 1.d scrub starts 2024-09-20T22:17:42.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[40290]: 1.d scrub ok 2024-09-20T22:17:42.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-20T22:17:42.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[40290]: osdmap e430: 8 total, 8 up, 8 in 2024-09-20T22:17:42.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[40290]: 3.1d scrub starts 2024-09-20T22:17:42.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[40290]: 3.1d scrub ok 2024-09-20T22:17:42.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[40290]: osdmap e431: 8 total, 8 up, 8 in 2024-09-20T22:17:42.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[40290]: 1.7 scrub starts 2024-09-20T22:17:42.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:42 smithi060 ceph-mon[40290]: 1.7 scrub ok 2024-09-20T22:17:42.598 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:17:42.619 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:42.649 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:42.650 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:42.650 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:42.751 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:42.763 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:17:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:17:42.788 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:42.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:42.789 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:42.890 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:42.932 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:42.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:42.933 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:43.015 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:17:43.015 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:17:43.033 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:43.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:43.074 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:43.074 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:43.175 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:43.214 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:43.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:43.215 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:43.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[29694]: 1.e scrub starts 2024-09-20T22:17:43.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[29694]: 1.e scrub ok 2024-09-20T22:17:43.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[29694]: 4.1c scrub starts 2024-09-20T22:17:43.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[29694]: 4.1c scrub ok 2024-09-20T22:17:43.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[29694]: 1.9 scrub starts 2024-09-20T22:17:43.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[29694]: 1.9 scrub ok 2024-09-20T22:17:43.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[29694]: pgmap v792: 178 pgs: 1 remapped, 2 remapped+peering, 4 unknown, 171 active+clean; 4.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 4.7 KiB/s rd, 27 KiB/s wr, 698 op/s; 84 KiB/s, 0 objects/s recovering 2024-09-20T22:17:43.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1129087206' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:17:43.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:17:43.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1129087206' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:17:43.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[29694]: osdmap e432: 8 total, 8 up, 8 in 2024-09-20T22:17:43.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[40290]: 1.e scrub starts 2024-09-20T22:17:43.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[40290]: 1.e scrub ok 2024-09-20T22:17:43.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[40290]: 4.1c scrub starts 2024-09-20T22:17:43.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[40290]: 4.1c scrub ok 2024-09-20T22:17:43.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[40290]: 1.9 scrub starts 2024-09-20T22:17:43.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[40290]: 1.9 scrub ok 2024-09-20T22:17:43.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[40290]: pgmap v792: 178 pgs: 1 remapped, 2 remapped+peering, 4 unknown, 171 active+clean; 4.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 4.7 KiB/s rd, 27 KiB/s wr, 698 op/s; 84 KiB/s, 0 objects/s recovering 2024-09-20T22:17:43.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1129087206' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:17:43.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:17:43.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1129087206' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:17:43.288 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:43 smithi060 ceph-mon[40290]: osdmap e432: 8 total, 8 up, 8 in 2024-09-20T22:17:43.316 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:43.351 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:43.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:43.352 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:43.416 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:17:43.416 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:17:43.416 INFO:tasks.thrashosds.thrasher:inject_pause on osd.6 2024-09-20T22:17:43.417 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-09-20T22:17:43.417 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-09-20T22:17:43.417 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 config set filestore_inject_stall 3 2024-09-20T22:17:43.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:43 smithi067 ceph-mon[33511]: 1.e scrub starts 2024-09-20T22:17:43.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:43 smithi067 ceph-mon[33511]: 1.e scrub ok 2024-09-20T22:17:43.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:43 smithi067 ceph-mon[33511]: 4.1c scrub starts 2024-09-20T22:17:43.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:43 smithi067 ceph-mon[33511]: 4.1c scrub ok 2024-09-20T22:17:43.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:43 smithi067 ceph-mon[33511]: 1.9 scrub starts 2024-09-20T22:17:43.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:43 smithi067 ceph-mon[33511]: 1.9 scrub ok 2024-09-20T22:17:43.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:43 smithi067 ceph-mon[33511]: pgmap v792: 178 pgs: 1 remapped, 2 remapped+peering, 4 unknown, 171 active+clean; 4.0 GiB data, 17 GiB used, 698 GiB / 715 GiB avail; 4.7 KiB/s rd, 27 KiB/s wr, 698 op/s; 84 KiB/s, 0 objects/s recovering 2024-09-20T22:17:43.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:43 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1129087206' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:17:43.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:43 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:17:43.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:43 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1129087206' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:17:43.449 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:43 smithi067 ceph-mon[33511]: osdmap e432: 8 total, 8 up, 8 in 2024-09-20T22:17:43.453 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:43.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:43.491 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:43.492 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:43.592 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:43.645 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:43.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:43.646 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:43.699 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:17:43.746 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:43.767 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T22:17:43.792 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:43.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:43.793 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:43.873 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:17:43.894 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:43.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:43.955 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:43.955 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:44.055 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:44.105 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:44.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:44.106 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:44.121 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:44.206 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:44.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:44.247 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:44.247 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:44.289 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:44 smithi067 ceph-mon[33511]: 1.5 scrub starts 2024-09-20T22:17:44.289 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:44 smithi067 ceph-mon[33511]: 1.5 scrub ok 2024-09-20T22:17:44.289 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:44 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:17:44.289 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:44 smithi067 ceph-mon[33511]: Health check failed: Reduced data availability: 3 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-20T22:17:44.289 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:44 smithi067 ceph-mon[33511]: osdmap e433: 8 total, 8 up, 8 in 2024-09-20T22:17:44.347 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:44.387 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:44.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:44.388 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:44.428 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:44 smithi060 ceph-mon[29694]: 1.5 scrub starts 2024-09-20T22:17:44.428 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:44 smithi060 ceph-mon[29694]: 1.5 scrub ok 2024-09-20T22:17:44.429 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:44 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:17:44.429 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:44 smithi060 ceph-mon[29694]: Health check failed: Reduced data availability: 3 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-20T22:17:44.429 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:44 smithi060 ceph-mon[29694]: osdmap e433: 8 total, 8 up, 8 in 2024-09-20T22:17:44.429 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:44 smithi060 ceph-mon[40290]: 1.5 scrub starts 2024-09-20T22:17:44.429 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:44 smithi060 ceph-mon[40290]: 1.5 scrub ok 2024-09-20T22:17:44.429 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:44 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:17:44.429 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:44 smithi060 ceph-mon[40290]: Health check failed: Reduced data availability: 3 pgs inactive, 2 pgs peering (PG_AVAILABILITY) 2024-09-20T22:17:44.429 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:44 smithi060 ceph-mon[40290]: osdmap e433: 8 total, 8 up, 8 in 2024-09-20T22:17:44.488 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:44.538 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:44.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:44.539 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:44.571 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:17:44.571 INFO:teuthology.orchestra.run.smithi067.stdout: "success": "" 2024-09-20T22:17:44.571 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:17:44.640 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:44.663 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:44.663 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:44.663 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:44.763 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:17:44.763 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:17:44.765 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:44.802 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:17:44.802 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:17:44.805 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:44.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:44.806 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:44.838 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:17:44.838 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:17:44.872 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:17:44.872 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:17:44.906 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:44.907 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:17:44.907 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:17:44.946 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:17:44.946 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:17:44.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:44.949 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:44.950 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:44.979 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:17:44.979 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:17:45.013 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:17:45.013 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:17:45.050 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:45.092 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:45.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:45.093 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:45.194 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:45.232 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:45.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:45.232 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:45.285 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:17:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:17:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:17:45.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:45 smithi060 ceph-mon[29694]: pgmap v795: 178 pgs: 1 active+undersized+remapped, 1 remapped, 4 remapped+peering, 2 unknown, 170 active+clean; 3.8 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 6.5 KiB/s rd, 41 KiB/s wr, 1.42k op/s; 88 KiB/s, 0 objects/s recovering 2024-09-20T22:17:45.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:45 smithi060 ceph-mon[29694]: osdmap e434: 8 total, 8 up, 8 in 2024-09-20T22:17:45.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:45 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/118015413' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:45.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:45 smithi060 ceph-mon[40290]: pgmap v795: 178 pgs: 1 active+undersized+remapped, 1 remapped, 4 remapped+peering, 2 unknown, 170 active+clean; 3.8 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 6.5 KiB/s rd, 41 KiB/s wr, 1.42k op/s; 88 KiB/s, 0 objects/s recovering 2024-09-20T22:17:45.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:45 smithi060 ceph-mon[40290]: osdmap e434: 8 total, 8 up, 8 in 2024-09-20T22:17:45.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:45 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/118015413' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:45.333 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:45.361 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:45.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:45.362 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:45.462 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:45.466 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:17:45.501 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:45.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:45.502 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:45.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:45 smithi067 ceph-mon[33511]: pgmap v795: 178 pgs: 1 active+undersized+remapped, 1 remapped, 4 remapped+peering, 2 unknown, 170 active+clean; 3.8 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 6.5 KiB/s rd, 41 KiB/s wr, 1.42k op/s; 88 KiB/s, 0 objects/s recovering 2024-09-20T22:17:45.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:45 smithi067 ceph-mon[33511]: osdmap e434: 8 total, 8 up, 8 in 2024-09-20T22:17:45.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:45 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/118015413' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:45.603 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:45.640 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:45.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:45.641 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:45.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:45.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:45.794 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:45.794 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:45.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:45.922 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:45.928 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:45.929 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:45.929 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:46.030 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:46.063 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:46.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:46.064 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:46.165 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:46.173 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:17:46.211 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:46.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:46.212 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:46.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:46.348 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:46.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:46.349 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:46.449 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:46.492 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:46.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:46.493 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:46.593 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:46.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:46.643 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:46.643 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:46.744 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:46.780 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:46.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:46.781 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:46.882 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:46.916 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:46.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:46.917 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:47.018 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:47.052 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:47.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:47.053 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:47.154 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:47.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:47.193 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:47.193 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:47.294 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:47.331 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:47.332 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:47.332 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:47.366 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:17:47.375 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:17:47.375 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:17:47.375 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:17:47.375 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:17:47.375 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.375 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:106968 8.15 8:a8961342:::benchmark_data_smithi060_111051_object106967:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:17:47.375 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.566651+0000", 2024-09-20T22:17:47.375 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 516.80467816800001, 2024-09-20T22:17:47.376 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9227000670000001, 2024-09-20T22:17:47.376 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.376 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.376 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:17:47.376 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:47.376 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:47.376 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:47.376 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 106968 2024-09-20T22:17:47.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.376 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.376 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.376 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.376 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566651+0000", 2024-09-20T22:17:47.376 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.376 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.377 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.377 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.377 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566651+0000", 2024-09-20T22:17:47.377 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.377 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.377 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.377 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566653+0000", 2024-09-20T22:17:47.377 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.674e-06 2024-09-20T22:17:47.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.377 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.377 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.377 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566809+0000", 2024-09-20T22:17:47.377 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015637600000000001 2024-09-20T22:17:47.377 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.378 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.378 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566820+0000", 2024-09-20T22:17:47.378 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.097e-05 2024-09-20T22:17:47.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.378 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.378 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.566830+0000", 2024-09-20T22:17:47.378 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4690000000000003e-06 2024-09-20T22:17:47.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.378 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.378 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.378 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.567309+0000", 2024-09-20T22:17:47.378 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00047940599999999998 2024-09-20T22:17:47.378 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.379 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.379 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:47.379 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.567321+0000", 2024-09-20T22:17:47.379 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2451e-05 2024-09-20T22:17:47.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.379 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.379 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.379 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417364+0000", 2024-09-20T22:17:47.379 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.85004267200000005 2024-09-20T22:17:47.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.379 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.379 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:47.379 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417390+0000", 2024-09-20T22:17:47.379 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.5959999999999999e-05 2024-09-20T22:17:47.379 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.380 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.380 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489254+0000", 2024-09-20T22:17:47.380 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0718642620000001 2024-09-20T22:17:47.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.380 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.380 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489351+0000", 2024-09-20T22:17:47.380 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6827e-05 2024-09-20T22:17:47.380 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.380 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.380 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.380 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.380 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.380 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:106993 8.15 8:ae0aa858:::benchmark_data_smithi060_111051_object106992:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:17:47.381 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.600689+0000", 2024-09-20T22:17:47.381 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 516.77064050399997, 2024-09-20T22:17:47.381 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.911249717, 2024-09-20T22:17:47.381 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.381 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.381 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.381 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:47.381 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:47.381 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:47.381 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 106993 2024-09-20T22:17:47.381 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.381 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.381 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.381 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.381 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600689+0000", 2024-09-20T22:17:47.382 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.382 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.382 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.382 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600689+0000", 2024-09-20T22:17:47.382 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.382 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.382 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.382 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600691+0000", 2024-09-20T22:17:47.382 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.052e-06 2024-09-20T22:17:47.382 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.382 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.383 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.383 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600724+0000", 2024-09-20T22:17:47.383 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.3429000000000001e-05 2024-09-20T22:17:47.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.383 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.383 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.383 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600747+0000", 2024-09-20T22:17:47.383 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2663e-05 2024-09-20T22:17:47.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.383 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.383 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.383 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600756+0000", 2024-09-20T22:17:47.383 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4099999999999997e-06 2024-09-20T22:17:47.383 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.383 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.384 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.384 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600862+0000", 2024-09-20T22:17:47.384 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000106135 2024-09-20T22:17:47.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.384 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.384 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:47.384 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.600873+0000", 2024-09-20T22:17:47.384 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0794e-05 2024-09-20T22:17:47.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.384 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.384 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.384 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417486+0000", 2024-09-20T22:17:47.384 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.81661262800000001 2024-09-20T22:17:47.384 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.384 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.384 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:47.385 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417495+0000", 2024-09-20T22:17:47.385 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1160000000000005e-06 2024-09-20T22:17:47.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.385 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.385 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.385 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489416+0000", 2024-09-20T22:17:47.385 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.071920784 2024-09-20T22:17:47.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.385 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.385 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.385 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489581+0000", 2024-09-20T22:17:47.385 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00016577399999999999 2024-09-20T22:17:47.385 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.385 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.385 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,4,6", 2024-09-20T22:17:47.386 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.489742+0000", 2024-09-20T22:17:47.386 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000160204 2024-09-20T22:17:47.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.386 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.386 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:47.386 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.494442+0000", 2024-09-20T22:17:47.386 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0047004660000000004 2024-09-20T22:17:47.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.386 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.386 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.386 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.495057+0000", 2024-09-20T22:17:47.386 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000614677 2024-09-20T22:17:47.386 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.387 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.387 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.387 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.495091+0000", 2024-09-20T22:17:47.387 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4581e-05 2024-09-20T22:17:47.387 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.387 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.387 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.387 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.511743+0000", 2024-09-20T22:17:47.387 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.016651142000000001 2024-09-20T22:17:47.387 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.387 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.387 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.387 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.511810+0000", 2024-09-20T22:17:47.387 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7065999999999998e-05 2024-09-20T22:17:47.388 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.388 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.388 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.388 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.511938+0000", 2024-09-20T22:17:47.388 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000128796 2024-09-20T22:17:47.388 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.388 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.388 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.389 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.389 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:107013 8.15 8:ab48afac:::benchmark_data_smithi060_111051_object107012:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e217)", 2024-09-20T22:17:47.389 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:09:10.648456+0000", 2024-09-20T22:17:47.389 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 516.72287280199998, 2024-09-20T22:17:47.389 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8646870719999999, 2024-09-20T22:17:47.389 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.389 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.389 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.389 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:47.389 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:47.389 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:47.389 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 107013 2024-09-20T22:17:47.389 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.390 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.390 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.390 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.390 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648456+0000", 2024-09-20T22:17:47.390 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.390 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.390 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.390 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.390 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648456+0000", 2024-09-20T22:17:47.390 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.390 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.390 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.390 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.390 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648459+0000", 2024-09-20T22:17:47.391 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2000000000000001e-06 2024-09-20T22:17:47.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.391 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.391 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.391 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648501+0000", 2024-09-20T22:17:47.391 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2453000000000002e-05 2024-09-20T22:17:47.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.391 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.391 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.391 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648525+0000", 2024-09-20T22:17:47.391 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.4034000000000001e-05 2024-09-20T22:17:47.391 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.391 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.391 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.392 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648535+0000", 2024-09-20T22:17:47.392 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0355e-05 2024-09-20T22:17:47.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.392 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.392 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.392 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648641+0000", 2024-09-20T22:17:47.392 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000105448 2024-09-20T22:17:47.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.392 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.392 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:47.392 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:10.648654+0000", 2024-09-20T22:17:47.392 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2904999999999999e-05 2024-09-20T22:17:47.392 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.392 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.393 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.393 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417525+0000", 2024-09-20T22:17:47.393 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.76887095999999999 2024-09-20T22:17:47.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.393 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.393 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:47.393 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:11.417959+0000", 2024-09-20T22:17:47.393 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00043430399999999999 2024-09-20T22:17:47.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.393 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.393 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.393 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.492147+0000", 2024-09-20T22:17:47.393 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0741883080000001 2024-09-20T22:17:47.393 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.393 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.394 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.394 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.492291+0000", 2024-09-20T22:17:47.394 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014341700000000001 2024-09-20T22:17:47.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.394 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.394 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,4,6", 2024-09-20T22:17:47.394 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.492463+0000", 2024-09-20T22:17:47.394 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017243400000000001 2024-09-20T22:17:47.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.394 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.394 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.394 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.499029+0000", 2024-09-20T22:17:47.394 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0065655460000000002 2024-09-20T22:17:47.394 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.395 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.395 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.395 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.499078+0000", 2024-09-20T22:17:47.395 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9527000000000002e-05 2024-09-20T22:17:47.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.395 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.395 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:47.395 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.499106+0000", 2024-09-20T22:17:47.395 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7280999999999999e-05 2024-09-20T22:17:47.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.395 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.395 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.395 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.512989+0000", 2024-09-20T22:17:47.395 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.013883929 2024-09-20T22:17:47.395 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.396 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.396 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.396 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.513048+0000", 2024-09-20T22:17:47.396 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8261999999999999e-05 2024-09-20T22:17:47.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.396 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.396 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.396 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:09:12.513143+0000", 2024-09-20T22:17:47.396 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5709000000000001e-05 2024-09-20T22:17:47.396 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.396 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.396 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.396 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.396 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.396 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163403 8.17 8:ebf82752:::benchmark_data_smithi060_111051_object43404:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:17:47.396 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.477864+0000", 2024-09-20T22:17:47.397 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 457.893465499, 2024-09-20T22:17:47.397 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0006944760000001, 2024-09-20T22:17:47.397 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.397 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.397 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.397 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:47.397 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:47.397 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:47.397 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163403 2024-09-20T22:17:47.397 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.397 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.397 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.397 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.397 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477864+0000", 2024-09-20T22:17:47.398 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.398 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.398 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.398 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.398 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477864+0000", 2024-09-20T22:17:47.398 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.398 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.398 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.398 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.398 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477865+0000", 2024-09-20T22:17:47.398 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4279999999999999e-06 2024-09-20T22:17:47.398 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.398 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.398 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.398 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477870+0000", 2024-09-20T22:17:47.399 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8899999999999998e-06 2024-09-20T22:17:47.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.399 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.399 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.399 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477881+0000", 2024-09-20T22:17:47.399 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1139e-05 2024-09-20T22:17:47.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.399 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.399 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.399 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477888+0000", 2024-09-20T22:17:47.399 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9779999999999999e-06 2024-09-20T22:17:47.399 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.399 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.399 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.400 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477943+0000", 2024-09-20T22:17:47.400 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5229000000000003e-05 2024-09-20T22:17:47.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.400 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.400 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:47.400 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477952+0000", 2024-09-20T22:17:47.400 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5420000000000005e-06 2024-09-20T22:17:47.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.400 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.400 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.400 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.473740+0000", 2024-09-20T22:17:47.400 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.995788063 2024-09-20T22:17:47.400 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.400 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.400 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.400 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.473826+0000", 2024-09-20T22:17:47.401 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6131000000000006e-05 2024-09-20T22:17:47.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.401 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.401 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:17:47.401 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.473905+0000", 2024-09-20T22:17:47.401 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8899000000000005e-05 2024-09-20T22:17:47.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.401 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.401 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:47.401 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478387+0000", 2024-09-20T22:17:47.401 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0044816680000000003 2024-09-20T22:17:47.401 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.401 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.401 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.401 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478428+0000", 2024-09-20T22:17:47.402 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1091999999999999e-05 2024-09-20T22:17:47.402 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.402 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.402 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.402 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478481+0000", 2024-09-20T22:17:47.402 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3628000000000002e-05 2024-09-20T22:17:47.402 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.402 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.402 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.402 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478514+0000", 2024-09-20T22:17:47.402 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2880999999999999e-05 2024-09-20T22:17:47.402 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.402 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.402 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.403 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478558+0000", 2024-09-20T22:17:47.403 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3908e-05 2024-09-20T22:17:47.403 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.403 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.403 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.403 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.403 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.403 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163409 8.17 8:ee9c487d:::benchmark_data_smithi060_111051_object43410:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:17:47.403 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.483034+0000", 2024-09-20T22:17:47.403 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 457.888295137, 2024-09-20T22:17:47.403 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.99573102899999999, 2024-09-20T22:17:47.403 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.403 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.403 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.403 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:47.404 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:47.404 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:47.404 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163409 2024-09-20T22:17:47.404 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.404 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.404 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.404 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.404 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483034+0000", 2024-09-20T22:17:47.404 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.404 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.404 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.404 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.404 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483034+0000", 2024-09-20T22:17:47.404 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.404 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.404 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.405 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.405 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483036+0000", 2024-09-20T22:17:47.405 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7940000000000001e-06 2024-09-20T22:17:47.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.405 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.405 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.405 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483041+0000", 2024-09-20T22:17:47.405 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.446e-06 2024-09-20T22:17:47.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.405 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.405 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.405 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483060+0000", 2024-09-20T22:17:47.405 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8258e-05 2024-09-20T22:17:47.405 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.405 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.406 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.406 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483068+0000", 2024-09-20T22:17:47.406 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5390000000000007e-06 2024-09-20T22:17:47.406 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.406 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.406 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.406 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483127+0000", 2024-09-20T22:17:47.406 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8542999999999998e-05 2024-09-20T22:17:47.406 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.406 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.406 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:47.406 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483135+0000", 2024-09-20T22:17:47.406 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.926e-06 2024-09-20T22:17:47.406 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.475590+0000", 2024-09-20T22:17:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.99245572400000004 2024-09-20T22:17:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.475685+0000", 2024-09-20T22:17:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4228999999999995e-05 2024-09-20T22:17:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:17:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.475757+0000", 2024-09-20T22:17:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2712000000000004e-05 2024-09-20T22:17:47.407 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478623+0000", 2024-09-20T22:17:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028660069999999998 2024-09-20T22:17:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478671+0000", 2024-09-20T22:17:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7890999999999999e-05 2024-09-20T22:17:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478701+0000", 2024-09-20T22:17:47.408 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.9986999999999999e-05 2024-09-20T22:17:47.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.409 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.409 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478727+0000", 2024-09-20T22:17:47.409 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.6211999999999999e-05 2024-09-20T22:17:47.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.409 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.409 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478765+0000", 2024-09-20T22:17:47.409 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7761e-05 2024-09-20T22:17:47.409 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.409 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.409 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.410 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163412 8.17 8:ed62347b:::benchmark_data_smithi060_111051_object43413:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:17:47.410 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.487680+0000", 2024-09-20T22:17:47.410 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 457.883648698, 2024-09-20T22:17:47.410 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.99125577600000003, 2024-09-20T22:17:47.410 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.410 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.410 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.410 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:47.410 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:47.410 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:47.410 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163412 2024-09-20T22:17:47.410 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.410 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.410 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.411 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.411 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487680+0000", 2024-09-20T22:17:47.411 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.411 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.411 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.411 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487680+0000", 2024-09-20T22:17:47.411 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.411 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.411 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.411 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487682+0000", 2024-09-20T22:17:47.411 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.759e-06 2024-09-20T22:17:47.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.411 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.412 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.412 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487688+0000", 2024-09-20T22:17:47.412 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5060000000000003e-06 2024-09-20T22:17:47.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.412 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.412 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.412 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487701+0000", 2024-09-20T22:17:47.412 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3225e-05 2024-09-20T22:17:47.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.412 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.412 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.412 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487709+0000", 2024-09-20T22:17:47.412 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5449999999999998e-06 2024-09-20T22:17:47.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.412 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.413 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.413 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487767+0000", 2024-09-20T22:17:47.413 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8387999999999997e-05 2024-09-20T22:17:47.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.413 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.413 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:47.413 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487774+0000", 2024-09-20T22:17:47.413 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.844e-06 2024-09-20T22:17:47.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.413 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.413 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.413 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.477116+0000", 2024-09-20T22:17:47.413 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98934203200000004 2024-09-20T22:17:47.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.414 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.414 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.414 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.477189+0000", 2024-09-20T22:17:47.414 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3220999999999996e-05 2024-09-20T22:17:47.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.414 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.414 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:17:47.414 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.477253+0000", 2024-09-20T22:17:47.414 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3770999999999996e-05 2024-09-20T22:17:47.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.414 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.415 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:47.415 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478774+0000", 2024-09-20T22:17:47.415 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0015216940000000001 2024-09-20T22:17:47.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.415 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.415 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.415 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478827+0000", 2024-09-20T22:17:47.415 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2262000000000002e-05 2024-09-20T22:17:47.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.416 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.416 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.416 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478872+0000", 2024-09-20T22:17:47.416 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5398999999999997e-05 2024-09-20T22:17:47.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.416 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.416 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.416 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478899+0000", 2024-09-20T22:17:47.416 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7146999999999999e-05 2024-09-20T22:17:47.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.416 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.416 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.416 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478936+0000", 2024-09-20T22:17:47.417 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6983000000000001e-05 2024-09-20T22:17:47.417 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.417 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.417 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.417 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.417 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.417 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163455 8.17 8:ef455ef1:::benchmark_data_smithi060_111051_object43456:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:17:47.417 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.524754+0000", 2024-09-20T22:17:47.417 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 457.846574938, 2024-09-20T22:17:47.417 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.96086008199999995, 2024-09-20T22:17:47.417 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.417 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.417 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.418 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:47.418 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:47.418 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:47.418 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163455 2024-09-20T22:17:47.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.418 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.418 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.418 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.418 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524754+0000", 2024-09-20T22:17:47.418 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.418 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.418 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.419 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524754+0000", 2024-09-20T22:17:47.419 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.419 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.419 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.419 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524756+0000", 2024-09-20T22:17:47.419 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5850000000000001e-06 2024-09-20T22:17:47.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.419 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.419 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.419 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524761+0000", 2024-09-20T22:17:47.419 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6699999999999999e-06 2024-09-20T22:17:47.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.419 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.419 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.420 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524776+0000", 2024-09-20T22:17:47.420 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4416000000000001e-05 2024-09-20T22:17:47.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.420 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.420 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.420 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524783+0000", 2024-09-20T22:17:47.420 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9009999999999997e-06 2024-09-20T22:17:47.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.420 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.420 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.420 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524851+0000", 2024-09-20T22:17:47.420 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8090999999999996e-05 2024-09-20T22:17:47.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.420 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.420 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:47.421 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.524858+0000", 2024-09-20T22:17:47.421 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9299999999999997e-06 2024-09-20T22:17:47.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.421 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.421 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.478989+0000", 2024-09-20T22:17:47.421 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.954130849 2024-09-20T22:17:47.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.421 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.421 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.479103+0000", 2024-09-20T22:17:47.421 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000114608 2024-09-20T22:17:47.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.421 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:17:47.422 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.479218+0000", 2024-09-20T22:17:47.422 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000115146 2024-09-20T22:17:47.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.422 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.422 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485313+0000", 2024-09-20T22:17:47.422 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0060945410000000002 2024-09-20T22:17:47.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.422 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.422 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485397+0000", 2024-09-20T22:17:47.422 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4055999999999996e-05 2024-09-20T22:17:47.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.423 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:47.423 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485483+0000", 2024-09-20T22:17:47.423 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5507000000000007e-05 2024-09-20T22:17:47.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.423 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.423 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.423 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485567+0000", 2024-09-20T22:17:47.423 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4388999999999995e-05 2024-09-20T22:17:47.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.423 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.423 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.423 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485614+0000", 2024-09-20T22:17:47.423 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7392999999999998e-05 2024-09-20T22:17:47.423 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.424 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.424 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.424 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.424 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163489 8.17 8:eb24224e:::benchmark_data_smithi060_111051_object43490:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:17:47.424 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.552125+0000", 2024-09-20T22:17:47.424 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 457.81920419900001, 2024-09-20T22:17:47.424 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93738651900000003, 2024-09-20T22:17:47.424 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.424 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.424 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.424 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:47.424 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:47.424 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:47.424 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163489 2024-09-20T22:17:47.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.425 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.425 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.425 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552125+0000", 2024-09-20T22:17:47.425 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.425 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.425 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552125+0000", 2024-09-20T22:17:47.425 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.425 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.425 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552126+0000", 2024-09-20T22:17:47.425 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2810000000000001e-06 2024-09-20T22:17:47.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.426 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.426 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.426 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552131+0000", 2024-09-20T22:17:47.426 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9450000000000001e-06 2024-09-20T22:17:47.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.426 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.426 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.426 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552143+0000", 2024-09-20T22:17:47.426 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1902999999999999e-05 2024-09-20T22:17:47.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.426 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.426 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.426 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552150+0000", 2024-09-20T22:17:47.427 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2609999999999996e-06 2024-09-20T22:17:47.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.427 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.427 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.427 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552210+0000", 2024-09-20T22:17:47.427 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9463999999999999e-05 2024-09-20T22:17:47.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.427 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.427 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:47.427 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.552218+0000", 2024-09-20T22:17:47.427 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4460000000000001e-06 2024-09-20T22:17:47.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.427 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.427 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.427 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.480425+0000", 2024-09-20T22:17:47.427 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.928206859 2024-09-20T22:17:47.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.428 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.428 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.428 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.480526+0000", 2024-09-20T22:17:47.428 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100388 2024-09-20T22:17:47.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.428 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.428 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:17:47.428 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.480628+0000", 2024-09-20T22:17:47.428 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000102051 2024-09-20T22:17:47.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.428 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.428 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.428 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485464+0000", 2024-09-20T22:17:47.428 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0048367150000000001 2024-09-20T22:17:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485625+0000", 2024-09-20T22:17:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000160278 2024-09-20T22:17:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489410+0000", 2024-09-20T22:17:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0037854659999999999 2024-09-20T22:17:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489468+0000", 2024-09-20T22:17:47.429 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7883000000000001e-05 2024-09-20T22:17:47.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.430 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.430 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.430 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489512+0000", 2024-09-20T22:17:47.430 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3578999999999997e-05 2024-09-20T22:17:47.430 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.430 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.430 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.430 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.430 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163500 8.17 8:ef6c367b:::benchmark_data_smithi060_111051_object43501:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:17:47.430 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.560977+0000", 2024-09-20T22:17:47.430 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 457.81035237100002, 2024-09-20T22:17:47.430 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93477054299999995, 2024-09-20T22:17:47.430 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.431 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.431 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.431 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:47.431 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:47.431 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:47.431 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163500 2024-09-20T22:17:47.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.431 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.431 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.431 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.560977+0000", 2024-09-20T22:17:47.431 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.431 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.432 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.560977+0000", 2024-09-20T22:17:47.432 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.432 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.432 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.560978+0000", 2024-09-20T22:17:47.432 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.688e-06 2024-09-20T22:17:47.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.432 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.432 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.560984+0000", 2024-09-20T22:17:47.432 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7939999999999999e-06 2024-09-20T22:17:47.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.432 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.432 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.560997+0000", 2024-09-20T22:17:47.432 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.292e-05 2024-09-20T22:17:47.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.433 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.433 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.433 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.561006+0000", 2024-09-20T22:17:47.433 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2970000000000004e-06 2024-09-20T22:17:47.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.433 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.433 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.433 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.561071+0000", 2024-09-20T22:17:47.433 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5277000000000005e-05 2024-09-20T22:17:47.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.433 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.433 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:47.433 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.561080+0000", 2024-09-20T22:17:47.433 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6989999999999998e-06 2024-09-20T22:17:47.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.434 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.434 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.481917+0000", 2024-09-20T22:17:47.434 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.92083670500000003 2024-09-20T22:17:47.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.434 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.434 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.482017+0000", 2024-09-20T22:17:47.434 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9609000000000001e-05 2024-09-20T22:17:47.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.434 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:17:47.434 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.482117+0000", 2024-09-20T22:17:47.434 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9945999999999995e-05 2024-09-20T22:17:47.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.435 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:47.435 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485631+0000", 2024-09-20T22:17:47.435 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0035139049999999999 2024-09-20T22:17:47.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.435 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.435 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.435 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489611+0000", 2024-09-20T22:17:47.435 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0039801649999999999 2024-09-20T22:17:47.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.435 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.435 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.435 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495646+0000", 2024-09-20T22:17:47.435 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0060353660000000003 2024-09-20T22:17:47.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.435 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.436 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.436 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495704+0000", 2024-09-20T22:17:47.436 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7908999999999997e-05 2024-09-20T22:17:47.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.436 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.436 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495747+0000", 2024-09-20T22:17:47.436 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3263000000000002e-05 2024-09-20T22:17:47.436 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.436 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.436 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.436 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163521 8.17 8:edfbcf2c:::benchmark_data_smithi060_111051_object43522:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:17:47.437 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.577053+0000", 2024-09-20T22:17:47.437 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 457.79427592299999, 2024-09-20T22:17:47.437 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.91884361299999995, 2024-09-20T22:17:47.437 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.437 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.437 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.437 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:47.437 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:47.437 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:47.437 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163521 2024-09-20T22:17:47.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.437 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.437 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.437 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.437 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577053+0000", 2024-09-20T22:17:47.438 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.438 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.438 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.438 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577053+0000", 2024-09-20T22:17:47.438 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.438 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.438 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.438 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577055+0000", 2024-09-20T22:17:47.438 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3039999999999999e-06 2024-09-20T22:17:47.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.438 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.438 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.438 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577060+0000", 2024-09-20T22:17:47.439 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4319999999999998e-06 2024-09-20T22:17:47.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.439 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.439 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577071+0000", 2024-09-20T22:17:47.439 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1258e-05 2024-09-20T22:17:47.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.439 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.439 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577079+0000", 2024-09-20T22:17:47.439 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.374e-06 2024-09-20T22:17:47.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.439 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.439 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577177+0000", 2024-09-20T22:17:47.439 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7923000000000004e-05 2024-09-20T22:17:47.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.440 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:47.440 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577197+0000", 2024-09-20T22:17:47.440 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0001999999999999e-05 2024-09-20T22:17:47.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.440 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.440 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.483671+0000", 2024-09-20T22:17:47.440 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.90647459900000005 2024-09-20T22:17:47.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.440 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.440 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.483791+0000", 2024-09-20T22:17:47.440 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:47.441 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000119743 2024-09-20T22:17:47.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.441 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.442 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:17:47.442 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.483891+0000", 2024-09-20T22:17:47.442 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010016700000000001 2024-09-20T22:17:47.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.442 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:47.442 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489543+0000", 2024-09-20T22:17:47.442 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0056521109999999996 2024-09-20T22:17:47.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.442 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.442 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489657+0000", 2024-09-20T22:17:47.442 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000114249 2024-09-20T22:17:47.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.443 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.443 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495803+0000", 2024-09-20T22:17:47.443 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061460059999999999 2024-09-20T22:17:47.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.443 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.443 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495857+0000", 2024-09-20T22:17:47.443 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3408000000000001e-05 2024-09-20T22:17:47.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.443 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.443 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495897+0000", 2024-09-20T22:17:47.443 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0037000000000003e-05 2024-09-20T22:17:47.443 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.443 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.444 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.444 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163525 8.17 8:ea732579:::benchmark_data_smithi060_111051_object43526:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e243)", 2024-09-20T22:17:47.444 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.577673+0000", 2024-09-20T22:17:47.444 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 457.793656618, 2024-09-20T22:17:47.444 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.91833821500000001, 2024-09-20T22:17:47.444 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.444 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.444 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.444 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:47.444 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:17:47.444 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:17:47.444 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163525 2024-09-20T22:17:47.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.444 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.445 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.445 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577673+0000", 2024-09-20T22:17:47.445 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.445 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.445 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577673+0000", 2024-09-20T22:17:47.445 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.445 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.445 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577673+0000", 2024-09-20T22:17:47.445 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1699999999999997e-07 2024-09-20T22:17:47.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.446 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.446 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577678+0000", 2024-09-20T22:17:47.446 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3200000000000001e-06 2024-09-20T22:17:47.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.446 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.446 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577689+0000", 2024-09-20T22:17:47.446 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0735e-05 2024-09-20T22:17:47.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.446 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.446 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577695+0000", 2024-09-20T22:17:47.446 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7549999999999997e-06 2024-09-20T22:17:47.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.447 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.447 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577745+0000", 2024-09-20T22:17:47.447 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9223e-05 2024-09-20T22:17:47.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.447 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.447 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:17:47.447 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.577752+0000", 2024-09-20T22:17:47.447 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3300000000000001e-06 2024-09-20T22:17:47.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.447 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.447 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.447 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485694+0000", 2024-09-20T22:17:47.447 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.90794208499999995 2024-09-20T22:17:47.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.448 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.448 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485801+0000", 2024-09-20T22:17:47.448 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010699700000000001 2024-09-20T22:17:47.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.448 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:17:47.448 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.485921+0000", 2024-09-20T22:17:47.448 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000120025 2024-09-20T22:17:47.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.448 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.448 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489730+0000", 2024-09-20T22:17:47.448 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003809046 2024-09-20T22:17:47.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.449 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:47.449 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.489745+0000", 2024-09-20T22:17:47.449 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4711e-05 2024-09-20T22:17:47.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.449 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.449 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495935+0000", 2024-09-20T22:17:47.449 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061903419999999997 2024-09-20T22:17:47.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.449 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.449 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.495975+0000", 2024-09-20T22:17:47.449 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0441000000000001e-05 2024-09-20T22:17:47.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.450 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:10.496011+0000", 2024-09-20T22:17:47.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5287999999999998e-05 2024-09-20T22:17:47.450 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.450 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.450 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.450 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.16005.0:77005 10.9 e395/370 10:91259046:::benchmark_data_smithi060_167766_object77004:head v 395'4865, mlcod=395'4862)", 2024-09-20T22:17:47.450 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:16:04.242729+0000", 2024-09-20T22:17:47.450 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 103.128600229, 2024-09-20T22:17:47.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.23358954400000001, 2024-09-20T22:17:47.450 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.451 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.451 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.451 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.451 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.451 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.242729+0000", 2024-09-20T22:17:47.451 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.451 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.451 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.242729+0000", 2024-09-20T22:17:47.451 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.451 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.452 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.242730+0000", 2024-09-20T22:17:47.452 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7199999999999998e-07 2024-09-20T22:17:47.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.452 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.452 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.242749+0000", 2024-09-20T22:17:47.452 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9083999999999999e-05 2024-09-20T22:17:47.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.452 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.452 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.242759+0000", 2024-09-20T22:17:47.452 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0208e-05 2024-09-20T22:17:47.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.452 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.453 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.242766+0000", 2024-09-20T22:17:47.453 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8510000000000001e-06 2024-09-20T22:17:47.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.453 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.453 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.453 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.467751+0000", 2024-09-20T22:17:47.453 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.22498507400000001 2024-09-20T22:17:47.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.453 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.453 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.453 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.467770+0000", 2024-09-20T22:17:47.453 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9318999999999999e-05 2024-09-20T22:17:47.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.453 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.453 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.453 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.476274+0000", 2024-09-20T22:17:47.454 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0085032600000000003 2024-09-20T22:17:47.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.454 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.454 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.454 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.476318+0000", 2024-09-20T22:17:47.454 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4975999999999998e-05 2024-09-20T22:17:47.454 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.454 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.454 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.454 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.454 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.16005.0:77013 10.9 e395/370 10:9a8ed282:::benchmark_data_smithi060_167766_object77012:head v 395'4866, mlcod=395'4862)", 2024-09-20T22:17:47.454 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:16:04.247828+0000", 2024-09-20T22:17:47.454 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 103.123500941, 2024-09-20T22:17:47.454 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.238548656, 2024-09-20T22:17:47.455 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.455 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.455 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.455 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.455 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.455 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.247828+0000", 2024-09-20T22:17:47.455 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.455 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.455 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.247828+0000", 2024-09-20T22:17:47.455 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.455 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.456 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.247829+0000", 2024-09-20T22:17:47.456 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.23e-07 2024-09-20T22:17:47.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.456 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.456 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.247852+0000", 2024-09-20T22:17:47.456 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2705999999999999e-05 2024-09-20T22:17:47.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.456 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.456 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.467712+0000", 2024-09-20T22:17:47.456 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.21986037 2024-09-20T22:17:47.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.456 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.467726+0000", 2024-09-20T22:17:47.457 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3495999999999999e-05 2024-09-20T22:17:47.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.457 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.471324+0000", 2024-09-20T22:17:47.457 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0035982060000000001 2024-09-20T22:17:47.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.457 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.471346+0000", 2024-09-20T22:17:47.457 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.211e-05 2024-09-20T22:17:47.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.457 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.486238+0000", 2024-09-20T22:17:47.458 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.014892036000000001 2024-09-20T22:17:47.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.458 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.458 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.486377+0000", 2024-09-20T22:17:47.458 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000139009 2024-09-20T22:17:47.458 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.458 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.458 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.458 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:172315 10.c 10:39190bf8:::benchmark_data_smithi060_167766_object47523:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e424)", 2024-09-20T22:17:47.458 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:28.233910+0000", 2024-09-20T22:17:47.458 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 19.137419250000001, 2024-09-20T22:17:47.458 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.060791816999999998, 2024-09-20T22:17:47.458 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.459 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.459 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.459 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:47.459 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:47.459 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:47.459 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 172315 2024-09-20T22:17:47.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.459 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.459 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.459 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:28.233910+0000", 2024-09-20T22:17:47.459 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.459 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.460 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:28.233910+0000", 2024-09-20T22:17:47.460 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.460 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.460 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:28.233912+0000", 2024-09-20T22:17:47.460 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1399999999999998e-06 2024-09-20T22:17:47.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.460 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.460 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:28.233918+0000", 2024-09-20T22:17:47.460 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9270000000000002e-06 2024-09-20T22:17:47.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.460 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.460 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:28.233937+0000", 2024-09-20T22:17:47.461 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8559999999999998e-05 2024-09-20T22:17:47.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.461 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.461 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:28.233945+0000", 2024-09-20T22:17:47.461 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9729999999999997e-06 2024-09-20T22:17:47.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.461 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.461 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:28.234073+0000", 2024-09-20T22:17:47.461 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012848800000000001 2024-09-20T22:17:47.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.461 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.461 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:28.234192+0000", 2024-09-20T22:17:47.462 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000118917 2024-09-20T22:17:47.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.462 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:17:47.462 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:28.234610+0000", 2024-09-20T22:17:47.462 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00041796100000000001 2024-09-20T22:17:47.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.462 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:47.462 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:28.237547+0000", 2024-09-20T22:17:47.462 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0029367690000000001 2024-09-20T22:17:47.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.462 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.462 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:28.237670+0000", 2024-09-20T22:17:47.463 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012285200000000001 2024-09-20T22:17:47.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.463 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.463 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.463 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:28.294611+0000", 2024-09-20T22:17:47.463 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.056941283000000002 2024-09-20T22:17:47.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.463 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.463 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.463 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:28.294661+0000", 2024-09-20T22:17:47.463 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9975000000000002e-05 2024-09-20T22:17:47.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.463 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.463 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.463 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:28.294702+0000", 2024-09-20T22:17:47.463 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0972e-05 2024-09-20T22:17:47.464 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.464 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.464 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.464 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:182757 10.c 10:3ce41db7:::benchmark_data_smithi060_167766_object57965:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e426)", 2024-09-20T22:17:47.464 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:37.449993+0000", 2024-09-20T22:17:47.464 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 9.921335697, 2024-09-20T22:17:47.464 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.084986775000000001, 2024-09-20T22:17:47.464 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.464 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.464 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.464 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:47.464 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:47.464 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:47.464 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 182757 2024-09-20T22:17:47.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.465 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.465 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.449993+0000", 2024-09-20T22:17:47.465 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.465 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.449993+0000", 2024-09-20T22:17:47.465 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.465 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.449995+0000", 2024-09-20T22:17:47.465 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2780000000000001e-06 2024-09-20T22:17:47.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.466 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.466 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.449999+0000", 2024-09-20T22:17:47.466 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0629999999999999e-06 2024-09-20T22:17:47.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.466 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.466 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.450008+0000", 2024-09-20T22:17:47.466 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4450000000000002e-06 2024-09-20T22:17:47.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.466 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.466 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.450015+0000", 2024-09-20T22:17:47.467 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5169999999999996e-06 2024-09-20T22:17:47.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.467 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.467 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.467 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.450052+0000", 2024-09-20T22:17:47.467 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7024999999999999e-05 2024-09-20T22:17:47.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.467 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.467 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.467 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.450115+0000", 2024-09-20T22:17:47.467 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3149000000000001e-05 2024-09-20T22:17:47.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.467 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.467 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:17:47.467 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.450177+0000", 2024-09-20T22:17:47.468 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1659e-05 2024-09-20T22:17:47.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.468 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.468 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:47.468 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.452695+0000", 2024-09-20T22:17:47.468 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0025188430000000002 2024-09-20T22:17:47.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.468 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.468 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.468 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.453077+0000", 2024-09-20T22:17:47.468 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00038192199999999998 2024-09-20T22:17:47.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.468 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.468 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.468 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.534868+0000", 2024-09-20T22:17:47.468 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.081790427999999998 2024-09-20T22:17:47.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.469 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.469 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.534933+0000", 2024-09-20T22:17:47.469 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5454e-05 2024-09-20T22:17:47.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.469 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.469 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.534980+0000", 2024-09-20T22:17:47.469 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6992e-05 2024-09-20T22:17:47.469 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.469 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.469 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.470 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:182765 10.c 10:36595850:::benchmark_data_smithi060_167766_object57973:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e426)", 2024-09-20T22:17:47.470 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:37.500452+0000", 2024-09-20T22:17:47.470 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 9.8708772469999992, 2024-09-20T22:17:47.470 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.048632121, 2024-09-20T22:17:47.470 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.470 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.470 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.470 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:47.470 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:47.470 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:47.470 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 182765 2024-09-20T22:17:47.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.470 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.470 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.470 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.470 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.500452+0000", 2024-09-20T22:17:47.471 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.471 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.471 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.471 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.500452+0000", 2024-09-20T22:17:47.471 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.471 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.471 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.471 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.500454+0000", 2024-09-20T22:17:47.471 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6449999999999999e-06 2024-09-20T22:17:47.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.471 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.471 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.472 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.500460+0000", 2024-09-20T22:17:47.472 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9549999999999997e-06 2024-09-20T22:17:47.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.472 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.472 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.500473+0000", 2024-09-20T22:17:47.472 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3055e-05 2024-09-20T22:17:47.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.472 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.472 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.500480+0000", 2024-09-20T22:17:47.472 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1350000000000003e-06 2024-09-20T22:17:47.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.472 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.473 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.500539+0000", 2024-09-20T22:17:47.473 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9391000000000001e-05 2024-09-20T22:17:47.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.473 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.473 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.500649+0000", 2024-09-20T22:17:47.473 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000109418 2024-09-20T22:17:47.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.473 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:17:47.473 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.500729+0000", 2024-09-20T22:17:47.473 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0717000000000007e-05 2024-09-20T22:17:47.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.473 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:47.473 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.502836+0000", 2024-09-20T22:17:47.474 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002106948 2024-09-20T22:17:47.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.474 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.474 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.474 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.502893+0000", 2024-09-20T22:17:47.474 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.6502999999999998e-05 2024-09-20T22:17:47.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.474 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.474 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.474 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.548994+0000", 2024-09-20T22:17:47.474 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.046101408000000003 2024-09-20T22:17:47.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.474 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.474 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.474 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.549045+0000", 2024-09-20T22:17:47.475 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0798999999999999e-05 2024-09-20T22:17:47.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.475 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.475 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:37.549084+0000", 2024-09-20T22:17:47.475 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9147000000000003e-05 2024-09-20T22:17:47.475 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.475 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.475 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.475 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:184796 10.c 10:3e8287b8:::benchmark_data_smithi060_167766_object60004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e428)", 2024-09-20T22:17:47.475 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:39.195723+0000", 2024-09-20T22:17:47.475 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 8.1756057240000004, 2024-09-20T22:17:47.475 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.172269858, 2024-09-20T22:17:47.475 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.476 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.476 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.476 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:47.476 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:47.476 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:47.476 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 184796 2024-09-20T22:17:47.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.476 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.476 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.476 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.195723+0000", 2024-09-20T22:17:47.476 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.476 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.476 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.195723+0000", 2024-09-20T22:17:47.477 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.477 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.477 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.195725+0000", 2024-09-20T22:17:47.477 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5060000000000001e-06 2024-09-20T22:17:47.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.477 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.477 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.195730+0000", 2024-09-20T22:17:47.477 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7389999999999999e-06 2024-09-20T22:17:47.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.477 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.477 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.195740+0000", 2024-09-20T22:17:47.477 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.049e-05 2024-09-20T22:17:47.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.478 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.478 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.195747+0000", 2024-09-20T22:17:47.478 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1080000000000004e-06 2024-09-20T22:17:47.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.478 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.478 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.195797+0000", 2024-09-20T22:17:47.478 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9291e-05 2024-09-20T22:17:47.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.478 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.478 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.195864+0000", 2024-09-20T22:17:47.478 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7749999999999993e-05 2024-09-20T22:17:47.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.479 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:17:47.479 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.195928+0000", 2024-09-20T22:17:47.479 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4156e-05 2024-09-20T22:17:47.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.480 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:47.480 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.197927+0000", 2024-09-20T22:17:47.480 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0019980319999999998 2024-09-20T22:17:47.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.480 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.480 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.198535+0000", 2024-09-20T22:17:47.480 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00060831199999999996 2024-09-20T22:17:47.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.480 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.480 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.367755+0000", 2024-09-20T22:17:47.480 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.16921968900000001 2024-09-20T22:17:47.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.367819+0000", 2024-09-20T22:17:47.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4624999999999999e-05 2024-09-20T22:17:47.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.367993+0000", 2024-09-20T22:17:47.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017416 2024-09-20T22:17:47.481 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.481 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.481 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.481 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:184805 10.c 10:356458e3:::benchmark_data_smithi060_167766_object60013:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e428)", 2024-09-20T22:17:47.481 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:39.202058+0000", 2024-09-20T22:17:47.481 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 8.169271148, 2024-09-20T22:17:47.482 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.172117927, 2024-09-20T22:17:47.482 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.482 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.482 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.482 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:47.482 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:47.482 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:47.482 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 184805 2024-09-20T22:17:47.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.482 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.482 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.482 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.202058+0000", 2024-09-20T22:17:47.482 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.483 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.483 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.202058+0000", 2024-09-20T22:17:47.483 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.483 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.483 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.202059+0000", 2024-09-20T22:17:47.483 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1909999999999999e-06 2024-09-20T22:17:47.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.483 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.483 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.202063+0000", 2024-09-20T22:17:47.483 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0690000000000003e-06 2024-09-20T22:17:47.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.484 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.484 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.202073+0000", 2024-09-20T22:17:47.484 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3870000000000001e-06 2024-09-20T22:17:47.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.484 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.484 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.202078+0000", 2024-09-20T22:17:47.484 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1320000000000002e-06 2024-09-20T22:17:47.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.484 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.484 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.202119+0000", 2024-09-20T22:17:47.484 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1029000000000003e-05 2024-09-20T22:17:47.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.485 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.485 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.202191+0000", 2024-09-20T22:17:47.485 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2058999999999996e-05 2024-09-20T22:17:47.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.485 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:17:47.485 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.202257+0000", 2024-09-20T22:17:47.485 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5738e-05 2024-09-20T22:17:47.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.485 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:47.485 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.204025+0000", 2024-09-20T22:17:47.486 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0017679760000000001 2024-09-20T22:17:47.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.486 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.486 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.204217+0000", 2024-09-20T22:17:47.486 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019245300000000001 2024-09-20T22:17:47.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.486 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.486 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.374056+0000", 2024-09-20T22:17:47.486 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.169838712 2024-09-20T22:17:47.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.486 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.486 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.374129+0000", 2024-09-20T22:17:47.487 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2818000000000007e-05 2024-09-20T22:17:47.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.487 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.487 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.487 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.374176+0000", 2024-09-20T22:17:47.487 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7363e-05 2024-09-20T22:17:47.487 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.487 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.487 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.487 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.487 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:185339 10.c 10:3a63c6bd:::benchmark_data_smithi060_167766_object60547:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e428)", 2024-09-20T22:17:47.487 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:39.853059+0000", 2024-09-20T22:17:47.487 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 7.5182703789999996, 2024-09-20T22:17:47.487 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.036684028, 2024-09-20T22:17:47.487 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.488 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.488 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.488 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:47.488 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:47.488 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:47.488 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 185339 2024-09-20T22:17:47.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.488 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.488 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.488 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.488 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.853059+0000", 2024-09-20T22:17:47.488 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.488 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.488 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.488 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.853059+0000", 2024-09-20T22:17:47.489 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.489 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.489 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.853061+0000", 2024-09-20T22:17:47.489 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.835e-06 2024-09-20T22:17:47.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.489 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.489 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.853066+0000", 2024-09-20T22:17:47.489 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0220000000000004e-06 2024-09-20T22:17:47.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.489 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.489 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.853083+0000", 2024-09-20T22:17:47.489 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7257000000000001e-05 2024-09-20T22:17:47.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.490 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.490 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.853091+0000", 2024-09-20T22:17:47.490 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0809999999999994e-06 2024-09-20T22:17:47.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.490 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.490 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.853150+0000", 2024-09-20T22:17:47.490 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8594000000000002e-05 2024-09-20T22:17:47.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.490 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.490 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.853255+0000", 2024-09-20T22:17:47.490 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000105828 2024-09-20T22:17:47.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.491 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.491 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:17:47.491 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.853359+0000", 2024-09-20T22:17:47.491 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000103513 2024-09-20T22:17:47.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.491 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.491 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:47.491 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.858281+0000", 2024-09-20T22:17:47.491 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0049223330000000001 2024-09-20T22:17:47.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.491 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.491 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.491 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.862875+0000", 2024-09-20T22:17:47.491 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0045942209999999999 2024-09-20T22:17:47.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.492 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.492 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.492 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.889650+0000", 2024-09-20T22:17:47.492 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.026774743 2024-09-20T22:17:47.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.492 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.492 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.492 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.889700+0000", 2024-09-20T22:17:47.492 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0071000000000002e-05 2024-09-20T22:17:47.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.492 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.492 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.492 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.889743+0000", 2024-09-20T22:17:47.492 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2530000000000001e-05 2024-09-20T22:17:47.493 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.493 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.493 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.493 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.493 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:185380 10.c 10:36811c91:::benchmark_data_smithi060_167766_object60588:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e428)", 2024-09-20T22:17:47.493 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:39.907666+0000", 2024-09-20T22:17:47.493 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 7.4636635379999996, 2024-09-20T22:17:47.493 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.064579575, 2024-09-20T22:17:47.493 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:17:47.493 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:17:47.493 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:17:47.493 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:17:47.493 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:17:47.493 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:17:47.494 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 185380 2024-09-20T22:17:47.494 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.494 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:17:47.494 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.494 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:17:47.494 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.907666+0000", 2024-09-20T22:17:47.494 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.494 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.494 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.494 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:17:47.494 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.907666+0000", 2024-09-20T22:17:47.494 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:17:47.494 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.494 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.494 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:17:47.495 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.907667+0000", 2024-09-20T22:17:47.495 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6360000000000001e-06 2024-09-20T22:17:47.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.495 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.495 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:17:47.495 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.907671+0000", 2024-09-20T22:17:47.495 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2039999999999999e-06 2024-09-20T22:17:47.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.495 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.495 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:17:47.495 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.907683+0000", 2024-09-20T22:17:47.495 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1362999999999999e-05 2024-09-20T22:17:47.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.495 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.495 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:17:47.495 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.907690+0000", 2024-09-20T22:17:47.496 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8650000000000003e-06 2024-09-20T22:17:47.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.496 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.496 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:17:47.496 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.907742+0000", 2024-09-20T22:17:47.496 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1879000000000003e-05 2024-09-20T22:17:47.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.496 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.496 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:17:47.496 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.907845+0000", 2024-09-20T22:17:47.496 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010344900000000001 2024-09-20T22:17:47.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.496 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.496 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:17:47.496 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.907947+0000", 2024-09-20T22:17:47.496 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010148400000000001 2024-09-20T22:17:47.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.497 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.497 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:17:47.497 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.910550+0000", 2024-09-20T22:17:47.497 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0026033829999999999 2024-09-20T22:17:47.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.497 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.497 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.497 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.913896+0000", 2024-09-20T22:17:47.497 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0033456380000000002 2024-09-20T22:17:47.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.497 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.497 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:17:47.497 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.972133+0000", 2024-09-20T22:17:47.497 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.058237832000000003 2024-09-20T22:17:47.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.498 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.498 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:17:47.498 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.972199+0000", 2024-09-20T22:17:47.498 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5168999999999999e-05 2024-09-20T22:17:47.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:17:47.498 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:17:47.498 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:17:47.498 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:39.972245+0000", 2024-09-20T22:17:47.498 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6672999999999999e-05 2024-09-20T22:17:47.498 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.498 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.498 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.498 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:17:47.498 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:17:47.499 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:17:47.516 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:47.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:47.517 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:47.618 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:47.652 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:47.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:47.653 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:47.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:47 smithi060 ceph-mon[29694]: pgmap v797: 178 pgs: 1 active+undersized+remapped, 1 remapped, 3 remapped+peering, 1 unknown, 172 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 9.8 KiB/s rd, 70 KiB/s wr, 954 op/s; 46 KiB/s, 0 objects/s recovering 2024-09-20T22:17:47.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:47 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4168861900' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:47.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:47 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:47.711 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:47 smithi060 ceph-mon[40290]: pgmap v797: 178 pgs: 1 active+undersized+remapped, 1 remapped, 3 remapped+peering, 1 unknown, 172 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 9.8 KiB/s rd, 70 KiB/s wr, 954 op/s; 46 KiB/s, 0 objects/s recovering 2024-09-20T22:17:47.711 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:47 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4168861900' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:47.711 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:47 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:47.753 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:47.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:47 smithi067 ceph-mon[33511]: pgmap v797: 178 pgs: 1 active+undersized+remapped, 1 remapped, 3 remapped+peering, 1 unknown, 172 active+clean; 3.7 GiB data, 16 GiB used, 699 GiB / 715 GiB avail; 9.8 KiB/s rd, 70 KiB/s wr, 954 op/s; 46 KiB/s, 0 objects/s recovering 2024-09-20T22:17:47.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:47 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4168861900' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:47.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:47 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:17:47.787 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:47.787 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:47.787 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:47.810 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:17:47.889 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:47.929 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:47.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:47.930 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:48.031 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:48.080 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:48.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:48.081 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:48.181 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:48.227 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:48.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:48.228 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:48.269 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:48.329 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:48.362 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:48.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:48.363 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:48.464 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:48.502 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T22:17:48.509 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:48.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:48.509 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:48.610 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:48.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:48 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:17:48.630 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:48 smithi060 ceph-mon[29694]: osdmap e435: 8 total, 8 up, 8 in 2024-09-20T22:17:48.634 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:48 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:17:48.634 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:48 smithi060 ceph-mon[40290]: osdmap e435: 8 total, 8 up, 8 in 2024-09-20T22:17:48.639 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:48.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:48.640 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:48.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:48.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:48 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:17:48.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:48 smithi067 ceph-mon[33511]: osdmap e435: 8 total, 8 up, 8 in 2024-09-20T22:17:48.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:48.790 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:48.790 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:48.890 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:48.924 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:48.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:48.925 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:48.978 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:17:48.978 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:17:49.014 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:17:49.014 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:17:49.026 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:49.037 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:49.047 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:17:49.048 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:17:49.061 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:49.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:49.062 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:49.086 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:17:49.086 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:17:49.123 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:17:49.123 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:17:49.154 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:17:49.155 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:17:49.162 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:49.194 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:17:49.194 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:17:49.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:49.203 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:49.203 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:49.228 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:17:49.228 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:17:49.305 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:49.340 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:49.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:49.341 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:49.441 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:49.478 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:49.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:49.479 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:49.580 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:49.615 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:49.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:49.616 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:49.642 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:49 smithi060 ceph-mon[29694]: pgmap v799: 178 pgs: 1 active+undersized+remapped, 1 remapped+peering, 176 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 8.1 KiB/s rd, 58 KiB/s wr, 1.49k op/s; 64 KiB/s, 0 objects/s recovering 2024-09-20T22:17:49.642 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:49 smithi060 ceph-mon[29694]: Health check update: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:17:49.642 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:49 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1310519609' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:49.643 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:49 smithi060 ceph-mon[40290]: pgmap v799: 178 pgs: 1 active+undersized+remapped, 1 remapped+peering, 176 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 8.1 KiB/s rd, 58 KiB/s wr, 1.49k op/s; 64 KiB/s, 0 objects/s recovering 2024-09-20T22:17:49.643 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:49 smithi060 ceph-mon[40290]: Health check update: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:17:49.643 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:49 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1310519609' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:49.717 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:49.756 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:49.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:49.757 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:49.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:49 smithi067 ceph-mon[33511]: pgmap v799: 178 pgs: 1 active+undersized+remapped, 1 remapped+peering, 176 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 8.1 KiB/s rd, 58 KiB/s wr, 1.49k op/s; 64 KiB/s, 0 objects/s recovering 2024-09-20T22:17:49.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:49 smithi067 ceph-mon[33511]: Health check update: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:17:49.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:49 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1310519609' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:49.857 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:49.887 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:49.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:49.888 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:49.988 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:50.021 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:50.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:50.022 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:50.123 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:50.158 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:50.158 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:17:50.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:50.159 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:50.260 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:50.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:50.295 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:50.295 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:50.396 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:50.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:50.439 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:50.440 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:50.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:50 smithi060 ceph-mon[29694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-20T22:17:50.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:50 smithi060 ceph-mon[40290]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-20T22:17:50.540 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:50.585 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:50.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:50.586 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:50.601 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:17:50.601 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:17:50.688 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:50.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:50.725 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:50.725 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:50.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:50 smithi067 ceph-mon[33511]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-20T22:17:50.825 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:50.868 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:50.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:50.869 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:50.970 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:51.009 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:51.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:51.010 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:51.111 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:51.143 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:51.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:51.144 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:51.245 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:51.281 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:51.282 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:51.282 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:51.371 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T22:17:51.382 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:51.410 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:51.411 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:51.411 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:51.434 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:51.512 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:51.542 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:51.542 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:51.543 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:51.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:51 smithi060 ceph-mon[29694]: pgmap v800: 178 pgs: 178 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 5.8 KiB/s rd, 41 KiB/s wr, 940 op/s; 68 KiB/s, 0 objects/s recovering 2024-09-20T22:17:51.571 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:17:51.571 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:51 smithi060 ceph-mon[40290]: pgmap v800: 178 pgs: 178 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 5.8 KiB/s rd, 41 KiB/s wr, 940 op/s; 68 KiB/s, 0 objects/s recovering 2024-09-20T22:17:51.571 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:17:51.643 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:51.690 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:51.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:51.691 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:51.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:51 smithi067 ceph-mon[33511]: pgmap v800: 178 pgs: 178 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 5.8 KiB/s rd, 41 KiB/s wr, 940 op/s; 68 KiB/s, 0 objects/s recovering 2024-09-20T22:17:51.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:17:51.792 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:51.835 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:51.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:51.836 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:51.937 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:51.975 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:51.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:51.976 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:52.068 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:17:52.068 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:17:52.068 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-09-20T22:17:52.069 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd dump -f json-pretty 2024-09-20T22:17:52.077 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:52.106 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:52.106 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:52.107 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:52.121 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:52.207 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:52.240 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:17:52.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:52.245 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:52.246 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:52.346 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:52.434 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:52.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:52.435 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:52.525 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:52 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:17:52.527 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:52 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:17:52.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:52.593 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:52.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:52.594 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:52.695 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:52.729 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:52.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:52.731 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:52.763 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:17:52 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:17:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:52 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:17:52.816 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:17:52.816 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:17:52.821 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:52.831 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:52.881 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:52.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:52.882 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:52.983 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:53.017 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:53.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:53.018 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:53.119 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:53.159 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:53.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:53.160 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:53.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:53.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:53.299 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:53.299 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:53.400 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:53.434 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:53.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:53.435 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:53.440 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:17:53.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:53.570 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:53.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:53.571 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:53.672 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:53.738 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:53.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:53.739 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:53 smithi067 ceph-mon[33511]: pgmap v801: 178 pgs: 178 active+clean; 3.2 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 5.5 KiB/s rd, 34 KiB/s wr, 1.15k op/s; 57 KiB/s, 0 objects/s recovering 2024-09-20T22:17:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:53 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/429588131' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:17:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:53 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:17:53.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:53 smithi060 ceph-mon[29694]: pgmap v801: 178 pgs: 178 active+clean; 3.2 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 5.5 KiB/s rd, 34 KiB/s wr, 1.15k op/s; 57 KiB/s, 0 objects/s recovering 2024-09-20T22:17:53.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:53 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/429588131' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:17:53.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:53 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:17:53.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:53 smithi060 ceph-mon[40290]: pgmap v801: 178 pgs: 178 active+clean; 3.2 GiB data, 15 GiB used, 701 GiB / 715 GiB avail; 5.5 KiB/s rd, 34 KiB/s wr, 1.15k op/s; 57 KiB/s, 0 objects/s recovering 2024-09-20T22:17:53.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:53 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/429588131' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:17:53.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:53 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:17:53.829 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:17:53.840 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:53.882 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T22:17:53.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:53.897 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:53.897 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:53.998 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:54.031 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:54.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:54.032 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:54.133 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:54.162 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:54.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:54.163 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:54.264 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:54.307 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:54.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:54.308 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:54.409 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:54.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:54.456 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:54.456 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:54.557 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:54.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:54.587 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:54.587 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:54.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:54 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:17:54.616 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:54 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:17:54.617 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:54 smithi060 ceph-mon[29694]: osdmap e436: 8 total, 8 up, 8 in 2024-09-20T22:17:54.617 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:54 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:17:54.617 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:54 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:17:54.617 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:54 smithi060 ceph-mon[40290]: osdmap e436: 8 total, 8 up, 8 in 2024-09-20T22:17:54.688 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:54.723 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:54.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:54.724 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:54 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:17:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:54 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:17:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:54 smithi067 ceph-mon[33511]: osdmap e436: 8 total, 8 up, 8 in 2024-09-20T22:17:54.825 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:54.861 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:54.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:54.862 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:54.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:54.998 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:54.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:55.000 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:55.100 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:55.137 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:55.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:55.138 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:55.139 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:17:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:17:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:17:55.239 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:17:55.269 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:55.269 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:55.270 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:17:55.329 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:17:55.329 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:17:55.370 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:55.372 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:17:55.372 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:17:55.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:55.410 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:55.410 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:55.411 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:17:55.411 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:17:55.451 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:17:55.451 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:17:55.490 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:17:55.490 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:17:55.511 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:55.531 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:17:55.532 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:17:55.553 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:55.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:55.554 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:55.566 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:17:55.566 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:17:55.607 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:17:55.607 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:17:55.655 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:55.685 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:55.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:55.686 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:55 smithi067 ceph-mon[33511]: pgmap v803: 178 pgs: 178 active+clean; 3.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 2.4 KiB/s rd, 14 KiB/s wr, 1.43k op/s; 42 KiB/s, 0 objects/s recovering 2024-09-20T22:17:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:55 smithi067 ceph-mon[33511]: 1.8 scrub starts 2024-09-20T22:17:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:55 smithi067 ceph-mon[33511]: 1.8 scrub ok 2024-09-20T22:17:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4186087743' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:55.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:55 smithi060 ceph-mon[29694]: pgmap v803: 178 pgs: 178 active+clean; 3.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 2.4 KiB/s rd, 14 KiB/s wr, 1.43k op/s; 42 KiB/s, 0 objects/s recovering 2024-09-20T22:17:55.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:55 smithi060 ceph-mon[29694]: 1.8 scrub starts 2024-09-20T22:17:55.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:55 smithi060 ceph-mon[29694]: 1.8 scrub ok 2024-09-20T22:17:55.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:55 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4186087743' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:55.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:55 smithi060 ceph-mon[40290]: pgmap v803: 178 pgs: 178 active+clean; 3.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 2.4 KiB/s rd, 14 KiB/s wr, 1.43k op/s; 42 KiB/s, 0 objects/s recovering 2024-09-20T22:17:55.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:55 smithi060 ceph-mon[40290]: 1.8 scrub starts 2024-09-20T22:17:55.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:55 smithi060 ceph-mon[40290]: 1.8 scrub ok 2024-09-20T22:17:55.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:55 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4186087743' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:17:55.788 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:55.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:55.825 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:55.925 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:55.970 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:55.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:55.972 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:55.974 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:17:55.975 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:17:55.975 INFO:teuthology.orchestra.run.smithi060.stdout: "epoch": 436, 2024-09-20T22:17:55.975 INFO:teuthology.orchestra.run.smithi060.stdout: "fsid": "3df9647e-779b-11ef-baf5-efdc52797490", 2024-09-20T22:17:55.975 INFO:teuthology.orchestra.run.smithi060.stdout: "created": "2024-09-20T21:57:31.244934+0000", 2024-09-20T22:17:55.975 INFO:teuthology.orchestra.run.smithi060.stdout: "modified": "2024-09-20T22:17:53.426581+0000", 2024-09-20T22:17:55.975 INFO:teuthology.orchestra.run.smithi060.stdout: "last_up_change": "2024-09-20T22:06:53.796679+0000", 2024-09-20T22:17:55.975 INFO:teuthology.orchestra.run.smithi060.stdout: "last_in_change": "2024-09-20T22:12:51.491976+0000", 2024-09-20T22:17:55.975 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": "nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-09-20T22:17:55.975 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_num": 5804032, 2024-09-20T22:17:55.975 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_set": [ 2024-09-20T22:17:55.975 INFO:teuthology.orchestra.run.smithi060.stdout: "nodeep-scrub", 2024-09-20T22:17:55.976 INFO:teuthology.orchestra.run.smithi060.stdout: "pglog_hardlimit", 2024-09-20T22:17:55.976 INFO:teuthology.orchestra.run.smithi060.stdout: "purged_snapdirs", 2024-09-20T22:17:55.976 INFO:teuthology.orchestra.run.smithi060.stdout: "recovery_deletes", 2024-09-20T22:17:55.976 INFO:teuthology.orchestra.run.smithi060.stdout: "sortbitwise" 2024-09-20T22:17:55.976 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:17:55.976 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_version": 19, 2024-09-20T22:17:55.976 INFO:teuthology.orchestra.run.smithi060.stdout: "full_ratio": 0.94999998807907104, 2024-09-20T22:17:55.976 INFO:teuthology.orchestra.run.smithi060.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-09-20T22:17:55.976 INFO:teuthology.orchestra.run.smithi060.stdout: "nearfull_ratio": 0.85000002384185791, 2024-09-20T22:17:55.976 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_snapshot": "", 2024-09-20T22:17:55.976 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_max": 10, 2024-09-20T22:17:55.976 INFO:teuthology.orchestra.run.smithi060.stdout: "max_osd": 8, 2024-09-20T22:17:55.976 INFO:teuthology.orchestra.run.smithi060.stdout: "require_min_compat_client": "luminous", 2024-09-20T22:17:55.977 INFO:teuthology.orchestra.run.smithi060.stdout: "min_compat_client": "luminous", 2024-09-20T22:17:55.977 INFO:teuthology.orchestra.run.smithi060.stdout: "require_osd_release": "squid", 2024-09-20T22:17:55.977 INFO:teuthology.orchestra.run.smithi060.stdout: "allow_crimson": false, 2024-09-20T22:17:55.977 INFO:teuthology.orchestra.run.smithi060.stdout: "pools": [ 2024-09-20T22:17:55.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:55.977 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 1, 2024-09-20T22:17:55.977 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": ".mgr", 2024-09-20T22:17:55.977 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:00:35.131125+0000", 2024-09-20T22:17:55.977 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:17:55.977 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:17:55.977 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:17:55.977 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:17:55.977 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:17:55.978 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:17:55.978 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:17:55.978 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:17:55.978 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:17:55.978 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:17:55.978 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:17:55.978 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:17:55.978 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:17:55.978 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 31, 2024-09-20T22:17:55.978 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 31, 2024-09-20T22:17:55.978 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 31, 2024-09-20T22:17:55.978 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 31, 2024-09-20T22:17:55.978 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 31, 2024-09-20T22:17:55.979 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:17:55.979 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:17:55.979 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:17:55.979 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:17:55.979 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:17:55.979 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:17:55.979 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:17:55.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:55.979 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "430", 2024-09-20T22:17:55.979 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:17:55.979 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:17:55.979 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "428", 2024-09-20T22:17:55.979 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:17:55.980 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:17:55.980 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:17:55.980 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:17:55.980 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:17:55.980 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:17:55.980 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:17:55.980 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:17:55.980 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:17:55.980 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:17:55.980 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:17:55.980 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:17:55.980 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:17:55.980 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:17:55.980 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:17:55.981 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:17:55.981 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:17:55.981 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:17:55.981 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:17:55.981 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:17:55.981 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:17:55.981 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:17:55.981 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:17:55.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:55.981 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:17:55.981 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:17:55.981 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:17:55.981 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:17:55.982 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:17:55.982 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:17:55.982 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:17:55.982 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:17:55.982 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:17:55.982 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:17:55.982 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:17:55.982 INFO:teuthology.orchestra.run.smithi060.stdout: "options": { 2024-09-20T22:17:55.982 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_max": 32, 2024-09-20T22:17:55.982 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_min": 1 2024-09-20T22:17:55.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:55.982 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:17:55.982 INFO:teuthology.orchestra.run.smithi060.stdout: "mgr": {} 2024-09-20T22:17:55.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:55.983 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:17:55.983 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:17:55.983 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.809999942779541, 2024-09-20T22:17:55.983 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.809999942779541, 2024-09-20T22:17:55.983 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:17:55.983 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.809999942779541, 2024-09-20T22:17:55.983 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.809999942779541, 2024-09-20T22:17:55.983 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 0.9326171875, 2024-09-20T22:17:55.983 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-20T22:17:55.983 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-20T22:17:55.983 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:55.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:55.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:55.984 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 2, 2024-09-20T22:17:55.984 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "datapool", 2024-09-20T22:17:55.984 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:47.974480+0000", 2024-09-20T22:17:55.984 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 8193, 2024-09-20T22:17:55.984 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-09-20T22:17:55.984 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:17:55.984 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:17:55.984 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:17:55.984 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:17:55.984 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:17:55.984 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:17:55.984 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:17:55.985 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:17:55.985 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:17:55.985 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:17:55.985 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:17:55.985 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 3, 2024-09-20T22:17:55.985 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 3, 2024-09-20T22:17:55.985 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 3, 2024-09-20T22:17:55.985 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 3, 2024-09-20T22:17:55.985 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 3, 2024-09-20T22:17:55.985 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:17:55.985 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:17:55.985 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:17:55.985 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:17:55.985 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:17:55.986 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:17:55.986 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:17:55.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:55.986 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "66", 2024-09-20T22:17:55.986 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:17:55.986 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:17:55.986 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:17:55.986 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:17:55.986 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:17:55.986 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 2, 2024-09-20T22:17:55.986 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 66, 2024-09-20T22:17:55.986 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:17:55.986 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:17:55.986 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:17:55.987 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:17:55.987 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:17:55.987 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:17:55.987 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:17:55.987 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:17:55.987 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:17:55.987 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:17:55.987 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:17:55.987 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:17:55.987 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:17:55.987 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:17:55.987 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:17:55.987 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:17:55.988 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:17:55.988 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:17:55.988 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:17:55.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:55.988 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:17:55.988 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:17:55.988 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:17:55.988 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:17:55.988 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:17:55.988 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:17:55.988 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:17:55.988 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:17:55.988 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:17:55.988 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:17:55.988 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:17:55.989 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:17:55.989 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:17:55.989 INFO:teuthology.orchestra.run.smithi060.stdout: "rbd": {} 2024-09-20T22:17:55.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:55.989 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:17:55.989 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:17:55.989 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2.6500000953674316, 2024-09-20T22:17:55.989 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 2.6500000953674316, 2024-09-20T22:17:55.989 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 0.87999999523162842, 2024-09-20T22:17:55.989 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2.3299999237060547, 2024-09-20T22:17:55.989 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 2.3299999237060547, 2024-09-20T22:17:55.989 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 0.9326171875, 2024-09-20T22:17:55.989 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-20T22:17:55.989 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-20T22:17:55.990 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:55.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:55.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:55.990 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 3, 2024-09-20T22:17:55.990 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": ".rgw.root", 2024-09-20T22:17:55.990 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:48.902465+0000", 2024-09-20T22:17:55.990 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:17:55.990 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:17:55.990 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:17:55.990 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:17:55.990 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:17:55.990 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:17:55.990 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:17:55.991 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:17:55.991 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:17:55.991 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:17:55.991 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:17:55.991 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:17:55.991 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:17:55.991 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:17:55.991 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:17:55.991 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:17:55.991 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:17:55.991 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:17:55.991 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:17:55.992 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:17:55.992 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:17:55.992 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:17:55.992 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:17:55.992 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:17:55.992 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:17:55.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:55.993 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "65", 2024-09-20T22:17:55.993 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:17:55.993 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:17:55.993 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:17:55.993 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:17:55.993 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:17:55.993 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:17:55.993 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:17:55.993 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:17:55.993 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:17:55.993 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:17:55.993 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:17:55.994 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:17:55.994 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:17:55.994 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:17:55.994 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:17:55.994 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:17:55.994 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:17:55.994 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:17:55.994 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:17:55.994 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:17:55.994 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:17:55.994 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:17:55.994 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:17:55.994 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:17:55.994 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:17:55.995 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:17:55.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:55.995 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:17:55.995 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:17:55.995 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:17:55.995 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:17:55.995 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:17:55.995 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:17:55.995 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:17:55.995 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:17:55.995 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:17:55.995 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:17:55.995 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:17:55.995 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:17:55.996 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:17:55.996 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:17:55.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:55.996 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:17:55.996 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:17:55.996 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.75, 2024-09-20T22:17:55.996 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.75, 2024-09-20T22:17:55.996 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:17:55.996 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.75, 2024-09-20T22:17:55.996 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.75, 2024-09-20T22:17:55.996 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 0.9326171875, 2024-09-20T22:17:55.996 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-20T22:17:55.996 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-20T22:17:55.996 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:55.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:55.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:55.997 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 4, 2024-09-20T22:17:55.997 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "default.rgw.log", 2024-09-20T22:17:55.997 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:50.620057+0000", 2024-09-20T22:17:55.997 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:17:55.997 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:17:55.997 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:17:55.997 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:17:55.997 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:17:55.997 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:17:55.997 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:17:55.997 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:17:55.997 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:17:55.997 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:17:55.998 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:17:55.998 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:17:55.998 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:17:55.998 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:17:55.998 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:17:55.998 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:17:55.998 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:17:55.998 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:17:55.998 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:17:55.998 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:17:55.998 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:17:55.998 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:17:55.998 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:17:55.999 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:17:55.999 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:17:55.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:55.999 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "67", 2024-09-20T22:17:55.999 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:17:55.999 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:17:55.999 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:17:55.999 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:17:55.999 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:17:55.999 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:17:55.999 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:17:55.999 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:17:55.999 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:17:56.000 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:17:56.000 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:17:56.000 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:17:56.000 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:17:56.000 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:17:56.000 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:17:56.000 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:17:56.000 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:17:56.000 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:17:56.000 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:17:56.000 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:17:56.000 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:17:56.000 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:17:56.000 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:17:56.000 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:17:56.001 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:17:56.001 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:17:56.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.001 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:17:56.001 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:17:56.001 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:17:56.001 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:17:56.001 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:17:56.001 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:17:56.001 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:17:56.001 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:17:56.001 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:17:56.001 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:17:56.001 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:17:56.002 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:17:56.002 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:17:56.002 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:17:56.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.002 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:17:56.002 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:17:56.002 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 2, 2024-09-20T22:17:56.002 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 2, 2024-09-20T22:17:56.002 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:17:56.002 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 2, 2024-09-20T22:17:56.002 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 2, 2024-09-20T22:17:56.002 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 0.9326171875, 2024-09-20T22:17:56.002 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-20T22:17:56.003 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-20T22:17:56.003 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.003 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 5, 2024-09-20T22:17:56.003 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "default.rgw.control", 2024-09-20T22:17:56.003 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:52.722912+0000", 2024-09-20T22:17:56.003 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:17:56.003 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:17:56.003 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:17:56.003 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:17:56.003 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:17:56.003 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:17:56.003 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:17:56.003 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:17:56.004 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:17:56.004 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:17:56.004 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:17:56.004 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:17:56.004 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:17:56.004 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:17:56.004 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:17:56.004 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:17:56.004 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:17:56.004 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:17:56.004 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:17:56.004 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:17:56.004 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:17:56.004 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:17:56.004 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:17:56.005 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:17:56.005 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:17:56.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.005 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "69", 2024-09-20T22:17:56.005 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:17:56.005 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:17:56.005 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:17:56.005 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:17:56.005 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:17:56.005 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:17:56.005 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:17:56.005 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:17:56.005 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:17:56.005 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:17:56.005 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:17:56.006 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:17:56.006 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:17:56.006 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:17:56.006 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:17:56.006 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:17:56.006 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:17:56.006 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:17:56.006 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:17:56.006 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:17:56.006 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:17:56.006 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:17:56.006 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:17:56.006 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:17:56.006 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:17:56.007 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:17:56.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.007 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:17:56.007 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:17:56.007 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:17:56.007 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:17:56.007 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:17:56.007 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:17:56.007 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:17:56.007 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:17:56.007 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:17:56.007 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:17:56.007 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:17:56.007 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:17:56.007 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:17:56.007 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:17:56.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.008 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:17:56.008 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:17:56.008 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.5, 2024-09-20T22:17:56.008 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.5, 2024-09-20T22:17:56.008 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:17:56.008 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.5, 2024-09-20T22:17:56.008 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.5, 2024-09-20T22:17:56.008 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 0.9326171875, 2024-09-20T22:17:56.008 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-20T22:17:56.008 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-20T22:17:56.008 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.008 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 6, 2024-09-20T22:17:56.009 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "default.rgw.meta", 2024-09-20T22:17:56.009 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:02:54.794248+0000", 2024-09-20T22:17:56.009 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:17:56.009 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:17:56.009 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:17:56.009 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:17:56.009 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:17:56.009 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:17:56.009 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:17:56.009 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:17:56.009 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:17:56.009 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:17:56.009 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:17:56.009 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:17:56.010 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:17:56.010 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 32, 2024-09-20T22:17:56.010 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 32, 2024-09-20T22:17:56.010 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 32, 2024-09-20T22:17:56.010 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 32, 2024-09-20T22:17:56.010 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 32, 2024-09-20T22:17:56.010 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:17:56.010 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:17:56.010 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:17:56.010 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:17:56.010 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:17:56.010 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:17:56.010 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:17:56.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.011 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "71", 2024-09-20T22:17:56.011 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:17:56.011 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:17:56.011 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:17:56.011 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:17:56.011 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:17:56.011 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:17:56.011 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:17:56.011 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:17:56.011 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:17:56.011 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:17:56.011 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:17:56.011 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:17:56.012 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:17:56.012 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:17:56.012 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:17:56.012 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:17:56.012 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:17:56.012 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:17:56.012 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:17:56.012 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:17:56.012 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:17:56.012 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:17:56.012 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:17:56.012 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:17:56.012 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:17:56.012 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:17:56.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.013 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:17:56.013 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:17:56.013 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:17:56.013 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:17:56.013 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:17:56.013 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:17:56.013 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:17:56.013 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:17:56.013 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:17:56.013 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:17:56.013 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:17:56.013 INFO:teuthology.orchestra.run.smithi060.stdout: "options": { 2024-09-20T22:17:56.013 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_bias": 4 2024-09-20T22:17:56.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.014 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:17:56.014 INFO:teuthology.orchestra.run.smithi060.stdout: "rgw": {} 2024-09-20T22:17:56.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.014 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:17:56.014 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:17:56.014 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 1.5, 2024-09-20T22:17:56.014 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 1.5, 2024-09-20T22:17:56.014 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:17:56.014 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 1.5, 2024-09-20T22:17:56.014 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 1.5, 2024-09-20T22:17:56.014 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 0.9326171875, 2024-09-20T22:17:56.014 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-20T22:17:56.014 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-20T22:17:56.015 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.015 INFO:teuthology.orchestra.run.smithi060.stdout: "pool": 10, 2024-09-20T22:17:56.015 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_name": "unique_pool_3", 2024-09-20T22:17:56.015 INFO:teuthology.orchestra.run.smithi060.stdout: "create_time": "2024-09-20T22:15:07.861275+0000", 2024-09-20T22:17:56.015 INFO:teuthology.orchestra.run.smithi060.stdout: "flags": 1, 2024-09-20T22:17:56.015 INFO:teuthology.orchestra.run.smithi060.stdout: "flags_names": "hashpspool", 2024-09-20T22:17:56.015 INFO:teuthology.orchestra.run.smithi060.stdout: "type": 1, 2024-09-20T22:17:56.015 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 3, 2024-09-20T22:17:56.015 INFO:teuthology.orchestra.run.smithi060.stdout: "min_size": 2, 2024-09-20T22:17:56.015 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_rule": 0, 2024-09-20T22:17:56.015 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_count": 0, 2024-09-20T22:17:56.015 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_target": 0, 2024-09-20T22:17:56.016 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_barrier": 0, 2024-09-20T22:17:56.016 INFO:teuthology.orchestra.run.smithi060.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-09-20T22:17:56.016 INFO:teuthology.orchestra.run.smithi060.stdout: "is_stretch_pool": false, 2024-09-20T22:17:56.016 INFO:teuthology.orchestra.run.smithi060.stdout: "object_hash": 2, 2024-09-20T22:17:56.016 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_autoscale_mode": "off", 2024-09-20T22:17:56.016 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num": 16, 2024-09-20T22:17:56.016 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num": 16, 2024-09-20T22:17:56.016 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_placement_num_target": 16, 2024-09-20T22:17:56.016 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_target": 16, 2024-09-20T22:17:56.016 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_num_pending": 16, 2024-09-20T22:17:56.016 INFO:teuthology.orchestra.run.smithi060.stdout: "last_pg_merge_meta": { 2024-09-20T22:17:56.016 INFO:teuthology.orchestra.run.smithi060.stdout: "source_pgid": "0.0", 2024-09-20T22:17:56.016 INFO:teuthology.orchestra.run.smithi060.stdout: "ready_epoch": 0, 2024-09-20T22:17:56.016 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_started": 0, 2024-09-20T22:17:56.016 INFO:teuthology.orchestra.run.smithi060.stdout: "last_epoch_clean": 0, 2024-09-20T22:17:56.017 INFO:teuthology.orchestra.run.smithi060.stdout: "source_version": "0'0", 2024-09-20T22:17:56.017 INFO:teuthology.orchestra.run.smithi060.stdout: "target_version": "0'0" 2024-09-20T22:17:56.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.017 INFO:teuthology.orchestra.run.smithi060.stdout: "last_change": "374", 2024-09-20T22:17:56.017 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend": "0", 2024-09-20T22:17:56.017 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_prenautilus": "0", 2024-09-20T22:17:56.017 INFO:teuthology.orchestra.run.smithi060.stdout: "last_force_op_resend_preluminous": "0", 2024-09-20T22:17:56.017 INFO:teuthology.orchestra.run.smithi060.stdout: "auid": 0, 2024-09-20T22:17:56.017 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_mode": "selfmanaged", 2024-09-20T22:17:56.017 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_seq": 0, 2024-09-20T22:17:56.017 INFO:teuthology.orchestra.run.smithi060.stdout: "snap_epoch": 0, 2024-09-20T22:17:56.017 INFO:teuthology.orchestra.run.smithi060.stdout: "pool_snaps": [], 2024-09-20T22:17:56.017 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps": "[]", 2024-09-20T22:17:56.017 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_bytes": 0, 2024-09-20T22:17:56.018 INFO:teuthology.orchestra.run.smithi060.stdout: "quota_max_objects": 0, 2024-09-20T22:17:56.018 INFO:teuthology.orchestra.run.smithi060.stdout: "tiers": [], 2024-09-20T22:17:56.018 INFO:teuthology.orchestra.run.smithi060.stdout: "tier_of": -1, 2024-09-20T22:17:56.018 INFO:teuthology.orchestra.run.smithi060.stdout: "read_tier": -1, 2024-09-20T22:17:56.018 INFO:teuthology.orchestra.run.smithi060.stdout: "write_tier": -1, 2024-09-20T22:17:56.018 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_mode": "none", 2024-09-20T22:17:56.018 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_bytes": 0, 2024-09-20T22:17:56.018 INFO:teuthology.orchestra.run.smithi060.stdout: "target_max_objects": 0, 2024-09-20T22:17:56.018 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-09-20T22:17:56.018 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-09-20T22:17:56.018 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_target_full_ratio_micro": 800000, 2024-09-20T22:17:56.018 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_flush_age": 0, 2024-09-20T22:17:56.018 INFO:teuthology.orchestra.run.smithi060.stdout: "cache_min_evict_age": 0, 2024-09-20T22:17:56.019 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profile": "", 2024-09-20T22:17:56.019 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_params": { 2024-09-20T22:17:56.019 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "none" 2024-09-20T22:17:56.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.019 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_period": 0, 2024-09-20T22:17:56.019 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_count": 0, 2024-09-20T22:17:56.019 INFO:teuthology.orchestra.run.smithi060.stdout: "use_gmt_hitset": true, 2024-09-20T22:17:56.019 INFO:teuthology.orchestra.run.smithi060.stdout: "min_read_recency_for_promote": 0, 2024-09-20T22:17:56.019 INFO:teuthology.orchestra.run.smithi060.stdout: "min_write_recency_for_promote": 0, 2024-09-20T22:17:56.019 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_grade_decay_rate": 0, 2024-09-20T22:17:56.019 INFO:teuthology.orchestra.run.smithi060.stdout: "hit_set_search_last_n": 0, 2024-09-20T22:17:56.019 INFO:teuthology.orchestra.run.smithi060.stdout: "grade_table": [], 2024-09-20T22:17:56.019 INFO:teuthology.orchestra.run.smithi060.stdout: "stripe_width": 0, 2024-09-20T22:17:56.019 INFO:teuthology.orchestra.run.smithi060.stdout: "expected_num_objects": 0, 2024-09-20T22:17:56.020 INFO:teuthology.orchestra.run.smithi060.stdout: "fast_read": false, 2024-09-20T22:17:56.020 INFO:teuthology.orchestra.run.smithi060.stdout: "options": {}, 2024-09-20T22:17:56.020 INFO:teuthology.orchestra.run.smithi060.stdout: "application_metadata": { 2024-09-20T22:17:56.020 INFO:teuthology.orchestra.run.smithi060.stdout: "rados": {} 2024-09-20T22:17:56.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.020 INFO:teuthology.orchestra.run.smithi060.stdout: "read_balance": { 2024-09-20T22:17:56.020 INFO:teuthology.orchestra.run.smithi060.stdout: "score_type": "Fair distribution", 2024-09-20T22:17:56.020 INFO:teuthology.orchestra.run.smithi060.stdout: "score_acting": 3, 2024-09-20T22:17:56.020 INFO:teuthology.orchestra.run.smithi060.stdout: "score_stable": 3, 2024-09-20T22:17:56.020 INFO:teuthology.orchestra.run.smithi060.stdout: "optimal_score": 1, 2024-09-20T22:17:56.020 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_acting": 3, 2024-09-20T22:17:56.020 INFO:teuthology.orchestra.run.smithi060.stdout: "raw_score_stable": 3, 2024-09-20T22:17:56.020 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity_weighted": 0.9326171875, 2024-09-20T22:17:56.020 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity": 0.93000000715255737, 2024-09-20T22:17:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: "average_primary_affinity_weighted": 0.93000000715255737 2024-09-20T22:17:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:17:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:17:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 0, 2024-09-20T22:17:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "0997c01a-a9bd-421e-abcc-fbcf7abe8512", 2024-09-20T22:17:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:17:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:17:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 0.95001220703125, 2024-09-20T22:17:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:17:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:17:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:17:56.021 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 8, 2024-09-20T22:17:56.022 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 433, 2024-09-20T22:17:56.022 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:17:56.022 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:17:56.022 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:17:56.022 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.022 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.022 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6802", 2024-09-20T22:17:56.022 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:17:56.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.022 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.022 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6803", 2024-09-20T22:17:56.022 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:17:56.023 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.023 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.023 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:17:56.023 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.023 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.023 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6804", 2024-09-20T22:17:56.023 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:17:56.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.023 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.023 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6805", 2024-09-20T22:17:56.023 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:17:56.024 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.024 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.024 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:17:56.024 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.024 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.024 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6808", 2024-09-20T22:17:56.024 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:17:56.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.024 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.024 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6809", 2024-09-20T22:17:56.024 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:17:56.025 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.025 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.025 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:17:56.025 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.025 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.025 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6806", 2024-09-20T22:17:56.025 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:17:56.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.025 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.025 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6807", 2024-09-20T22:17:56.025 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1395849550 2024-09-20T22:17:56.025 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.026 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.026 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6803/1395849550", 2024-09-20T22:17:56.026 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6805/1395849550", 2024-09-20T22:17:56.026 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6809/1395849550", 2024-09-20T22:17:56.026 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6807/1395849550", 2024-09-20T22:17:56.026 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:17:56.026 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:17:56.026 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:17:56.026 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.026 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 1, 2024-09-20T22:17:56.026 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "c04bdcb1-d76b-47c8-a38b-c02043df6ba8", 2024-09-20T22:17:56.026 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:17:56.027 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:17:56.027 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:17:56.027 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:17:56.027 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 13, 2024-09-20T22:17:56.027 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 90, 2024-09-20T22:17:56.027 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 145, 2024-09-20T22:17:56.027 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 433, 2024-09-20T22:17:56.027 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 91, 2024-09-20T22:17:56.027 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:17:56.027 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:17:56.027 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.027 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.027 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6810", 2024-09-20T22:17:56.027 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:17:56.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.028 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.028 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6811", 2024-09-20T22:17:56.028 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:17:56.028 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.028 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.028 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:17:56.028 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.028 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.028 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6812", 2024-09-20T22:17:56.028 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:17:56.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.029 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.029 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6813", 2024-09-20T22:17:56.029 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:17:56.029 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.029 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.029 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:17:56.029 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.029 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.029 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6816", 2024-09-20T22:17:56.029 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:17:56.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.030 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.030 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6817", 2024-09-20T22:17:56.030 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:17:56.030 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.030 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.030 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:17:56.030 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.030 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.030 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6814", 2024-09-20T22:17:56.030 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:17:56.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.030 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.031 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6815", 2024-09-20T22:17:56.031 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2970874921 2024-09-20T22:17:56.031 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.031 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.031 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6811/2970874921", 2024-09-20T22:17:56.031 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6813/2970874921", 2024-09-20T22:17:56.031 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6817/2970874921", 2024-09-20T22:17:56.031 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6815/2970874921", 2024-09-20T22:17:56.031 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:17:56.031 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:17:56.031 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:17:56.031 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.032 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 2, 2024-09-20T22:17:56.032 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "32c09d19-eb98-4926-a44e-e7ddee152956", 2024-09-20T22:17:56.032 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:17:56.032 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:17:56.032 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:17:56.032 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:17:56.032 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:17:56.032 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:17:56.032 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 18, 2024-09-20T22:17:56.032 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 432, 2024-09-20T22:17:56.032 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:17:56.032 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:17:56.032 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:17:56.032 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.033 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.033 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6818", 2024-09-20T22:17:56.033 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:17:56.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.033 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.033 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6819", 2024-09-20T22:17:56.033 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:17:56.033 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.033 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.033 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:17:56.033 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.033 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.033 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6820", 2024-09-20T22:17:56.034 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:17:56.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.034 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.034 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6821", 2024-09-20T22:17:56.034 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:17:56.034 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.034 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.034 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:17:56.034 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.034 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.034 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6824", 2024-09-20T22:17:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:17:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6825", 2024-09-20T22:17:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:17:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:17:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6822", 2024-09-20T22:17:56.035 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:17:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6823", 2024-09-20T22:17:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3698562274 2024-09-20T22:17:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6819/3698562274", 2024-09-20T22:17:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6821/3698562274", 2024-09-20T22:17:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6825/3698562274", 2024-09-20T22:17:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6823/3698562274", 2024-09-20T22:17:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:17:56.036 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:17:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:17:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 3, 2024-09-20T22:17:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "b1c2cb69-8565-49ac-b854-7ea7615c6660", 2024-09-20T22:17:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:17:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:17:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:17:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 0.4609375, 2024-09-20T22:17:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:17:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:17:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 26, 2024-09-20T22:17:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 433, 2024-09-20T22:17:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:17:56.037 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:17:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:17:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6826", 2024-09-20T22:17:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:17:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6827", 2024-09-20T22:17:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:17:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.038 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:17:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6828", 2024-09-20T22:17:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:17:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6829", 2024-09-20T22:17:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:17:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:17:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6832", 2024-09-20T22:17:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:17:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6833", 2024-09-20T22:17:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:17:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.040 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:17:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6830", 2024-09-20T22:17:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:17:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.60:6831", 2024-09-20T22:17:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 2592322449 2024-09-20T22:17:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.60:6827/2592322449", 2024-09-20T22:17:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.60:6829/2592322449", 2024-09-20T22:17:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.60:6833/2592322449", 2024-09-20T22:17:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.60:6831/2592322449", 2024-09-20T22:17:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:17:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:17:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:17:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 4, 2024-09-20T22:17:56.042 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "04a547d6-fc48-43b7-a8d7-18df8a83aa96", 2024-09-20T22:17:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:17:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:17:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 0.95001220703125, 2024-09-20T22:17:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:17:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:17:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:17:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 31, 2024-09-20T22:17:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 433, 2024-09-20T22:17:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:17:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:17:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:17:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.043 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6800", 2024-09-20T22:17:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:17:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6801", 2024-09-20T22:17:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:17:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:17:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.044 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6802", 2024-09-20T22:17:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:17:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6803", 2024-09-20T22:17:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:17:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:17:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6806", 2024-09-20T22:17:56.045 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:17:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6807", 2024-09-20T22:17:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:17:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:17:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6804", 2024-09-20T22:17:56.046 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:17:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6805", 2024-09-20T22:17:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3243564239 2024-09-20T22:17:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6801/3243564239", 2024-09-20T22:17:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6803/3243564239", 2024-09-20T22:17:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6807/3243564239", 2024-09-20T22:17:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6805/3243564239", 2024-09-20T22:17:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:17:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:17:56.047 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:17:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 5, 2024-09-20T22:17:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "4763aa10-c367-485b-b525-e5241e642b86", 2024-09-20T22:17:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:17:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:17:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:17:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:17:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:17:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:17:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 39, 2024-09-20T22:17:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 433, 2024-09-20T22:17:56.048 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:17:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:17:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:17:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6808", 2024-09-20T22:17:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:17:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6809", 2024-09-20T22:17:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:17:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.049 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:17:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6810", 2024-09-20T22:17:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:17:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6811", 2024-09-20T22:17:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:17:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:17:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6814", 2024-09-20T22:17:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:17:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6815", 2024-09-20T22:17:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:17:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:17:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6812", 2024-09-20T22:17:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:17:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6813", 2024-09-20T22:17:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1622258887 2024-09-20T22:17:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6809/1622258887", 2024-09-20T22:17:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6811/1622258887", 2024-09-20T22:17:56.052 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6815/1622258887", 2024-09-20T22:17:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6813/1622258887", 2024-09-20T22:17:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:17:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:17:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:17:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 6, 2024-09-20T22:17:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "a359116e-c013-4e2f-8b79-86219516ee16", 2024-09-20T22:17:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:17:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:17:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 0.95001220703125, 2024-09-20T22:17:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:17:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:17:56.053 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:17:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 48, 2024-09-20T22:17:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 428, 2024-09-20T22:17:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:17:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:17:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:17:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6816", 2024-09-20T22:17:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:17:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6817", 2024-09-20T22:17:56.054 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:17:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:17:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6818", 2024-09-20T22:17:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:17:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6819", 2024-09-20T22:17:56.055 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:17:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:17:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6822", 2024-09-20T22:17:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:17:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6823", 2024-09-20T22:17:56.056 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:17:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:17:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6820", 2024-09-20T22:17:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:17:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.057 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6821", 2024-09-20T22:17:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 1187114 2024-09-20T22:17:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6817/1187114", 2024-09-20T22:17:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6819/1187114", 2024-09-20T22:17:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6823/1187114", 2024-09-20T22:17:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6821/1187114", 2024-09-20T22:17:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:17:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:17:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:17:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.058 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 7, 2024-09-20T22:17:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "uuid": "a4bf4188-f133-4a31-8897-d7e8182248d7", 2024-09-20T22:17:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "up": 1, 2024-09-20T22:17:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "in": 1, 2024-09-20T22:17:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "weight": 1, 2024-09-20T22:17:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_affinity": 1, 2024-09-20T22:17:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_begin": 0, 2024-09-20T22:17:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "last_clean_end": 0, 2024-09-20T22:17:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "up_from": 57, 2024-09-20T22:17:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "up_thru": 433, 2024-09-20T22:17:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "down_at": 0, 2024-09-20T22:17:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "lost_at": 0, 2024-09-20T22:17:56.059 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addrs": { 2024-09-20T22:17:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6824", 2024-09-20T22:17:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:17:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6825", 2024-09-20T22:17:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:17:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addrs": { 2024-09-20T22:17:56.060 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6826", 2024-09-20T22:17:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:17:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6827", 2024-09-20T22:17:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:17:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addrs": { 2024-09-20T22:17:56.061 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6830", 2024-09-20T22:17:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:17:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6831", 2024-09-20T22:17:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:17:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addrs": { 2024-09-20T22:17:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: "addrvec": [ 2024-09-20T22:17:56.062 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v2", 2024-09-20T22:17:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6828", 2024-09-20T22:17:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:17:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: "type": "v1", 2024-09-20T22:17:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: "addr": "172.21.15.67:6829", 2024-09-20T22:17:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: "nonce": 3021842840 2024-09-20T22:17:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: "public_addr": "172.21.15.67:6825/3021842840", 2024-09-20T22:17:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: "cluster_addr": "172.21.15.67:6827/3021842840", 2024-09-20T22:17:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_back_addr": "172.21.15.67:6831/3021842840", 2024-09-20T22:17:56.063 INFO:teuthology.orchestra.run.smithi060.stdout: "heartbeat_front_addr": "172.21.15.67:6829/3021842840", 2024-09-20T22:17:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "state": [ 2024-09-20T22:17:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "exists", 2024-09-20T22:17:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "up" 2024-09-20T22:17:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:17:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "osd_xinfo": [ 2024-09-20T22:17:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 0, 2024-09-20T22:17:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:17:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:17:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:17:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:17:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:17:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:16:39.012311+0000", 2024-09-20T22:17:56.064 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:17:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 1, 2024-09-20T22:17:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:17:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:17:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:17:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:17:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:17:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:17:02.041351+0000", 2024-09-20T22:17:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 90 2024-09-20T22:17:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 2, 2024-09-20T22:17:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:17:56.065 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:17:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:17:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:17:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:17:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:17:12.682915+0000", 2024-09-20T22:17:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:17:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 3, 2024-09-20T22:17:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:17:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:17:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:17:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:17:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:17:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:17:15.514516+0000", 2024-09-20T22:17:56.066 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:17:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 4, 2024-09-20T22:17:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:17:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:17:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:17:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:17:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:17:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:17:07.357814+0000", 2024-09-20T22:17:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:17:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 5, 2024-09-20T22:17:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:17:56.067 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:17:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:17:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:17:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:17:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:17:11.911243+0000", 2024-09-20T22:17:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:17:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 6, 2024-09-20T22:17:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:17:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:17:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:17:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:17:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:17:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:17:10.226518+0000", 2024-09-20T22:17:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:17:56.068 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "osd": 7, 2024-09-20T22:17:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "down_stamp": "0.000000", 2024-09-20T22:17:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_probability": 0, 2024-09-20T22:17:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "laggy_interval": 0, 2024-09-20T22:17:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "features": 4540701547738038271, 2024-09-20T22:17:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "old_weight": 0, 2024-09-20T22:17:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "last_purged_snaps_scrub": "2024-09-20T22:17:25.435787+0000", 2024-09-20T22:17:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "dead_epoch": 0 2024-09-20T22:17:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:17:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap": [ 2024-09-20T22:17:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "1.8", 2024-09-20T22:17:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:17:56.069 INFO:teuthology.orchestra.run.smithi060.stdout: 6, 2024-09-20T22:17:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: 7, 2024-09-20T22:17:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: 2 2024-09-20T22:17:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "3.c", 2024-09-20T22:17:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:17:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: 3, 2024-09-20T22:17:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: 1, 2024-09-20T22:17:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: 6 2024-09-20T22:17:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.13", 2024-09-20T22:17:56.070 INFO:teuthology.orchestra.run.smithi060.stdout: "osds": [ 2024-09-20T22:17:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: 0, 2024-09-20T22:17:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: 7, 2024-09-20T22:17:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: 6 2024-09-20T22:17:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:17:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap_items": [ 2024-09-20T22:17:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "3.1a", 2024-09-20T22:17:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:17:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 4, 2024-09-20T22:17:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 5 2024-09-20T22:17:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.071 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "3.1c", 2024-09-20T22:17:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:17:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 4, 2024-09-20T22:17:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 2 2024-09-20T22:17:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "4.b", 2024-09-20T22:17:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:17:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.072 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 1, 2024-09-20T22:17:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 7 2024-09-20T22:17:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "4.f", 2024-09-20T22:17:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:17:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 1, 2024-09-20T22:17:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 2 2024-09-20T22:17:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.073 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "4.13", 2024-09-20T22:17:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:17:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 1, 2024-09-20T22:17:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 2 2024-09-20T22:17:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "4.1b", 2024-09-20T22:17:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:17:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 1, 2024-09-20T22:17:56.074 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 5 2024-09-20T22:17:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.9", 2024-09-20T22:17:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:17:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 7, 2024-09-20T22:17:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 1 2024-09-20T22:17:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.075 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.11", 2024-09-20T22:17:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:17:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:17:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 1 2024-09-20T22:17:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: "pgid": "6.19", 2024-09-20T22:17:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: "mappings": [ 2024-09-20T22:17:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:17:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: "from": 3, 2024-09-20T22:17:56.076 INFO:teuthology.orchestra.run.smithi060.stdout: "to": 6 2024-09-20T22:17:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:17:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:17:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_upmap_primaries": [], 2024-09-20T22:17:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "pg_temp": [], 2024-09-20T22:17:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "primary_temp": [], 2024-09-20T22:17:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "blocklist": { 2024-09-20T22:17:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/890154710": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:17:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3386526350": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:17:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/604714621": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:17:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/604714621": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:17:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/3151907776": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:17:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/956264518": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:17:56.077 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/4150585713": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:17:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3311255314": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:17:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3965541528": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:17:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6800/3734574621": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:17:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1729862661": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:17:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/3734574621": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:17:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/3927294026": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:17:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:6801/3151907776": "2024-09-21T21:58:02.105557+0000", 2024-09-20T22:17:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1871351181": "2024-09-21T21:58:26.613414+0000", 2024-09-20T22:17:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/1854455098": "2024-09-21T22:03:19.518405+0000", 2024-09-20T22:17:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "172.21.15.60:0/2221262203": "2024-09-21T22:03:19.518405+0000" 2024-09-20T22:17:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "range_blocklist": {}, 2024-09-20T22:17:56.078 INFO:teuthology.orchestra.run.smithi060.stdout: "erasure_code_profiles": { 2024-09-20T22:17:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "default": { 2024-09-20T22:17:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "crush-failure-domain": "osd", 2024-09-20T22:17:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "k": "2", 2024-09-20T22:17:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "m": "1", 2024-09-20T22:17:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "plugin": "jerasure", 2024-09-20T22:17:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "technique": "reed_sol_van" 2024-09-20T22:17:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:17:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "removed_snaps_queue": [], 2024-09-20T22:17:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "new_removed_snaps": [], 2024-09-20T22:17:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "new_purged_snaps": [], 2024-09-20T22:17:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "crush_node_flags": {}, 2024-09-20T22:17:56.079 INFO:teuthology.orchestra.run.smithi060.stdout: "device_class_flags": {}, 2024-09-20T22:17:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode": { 2024-09-20T22:17:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode_enabled": false, 2024-09-20T22:17:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_bucket_count": 0, 2024-09-20T22:17:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: "degraded_stretch_mode": 0, 2024-09-20T22:17:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: "recovering_stretch_mode": 0, 2024-09-20T22:17:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: "stretch_mode_bucket": 0 2024-09-20T22:17:56.080 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:17:56.080 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:17:56.080 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:56.108 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:56.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:56.109 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:56.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:56.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:56.248 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:56.248 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:56.349 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:56.383 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:56.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:56.384 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:56.394 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:17:56.485 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:56.536 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:56.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:56.537 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:56.638 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:56.670 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:56.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:56.671 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:56 smithi067 ceph-mon[33511]: 3.15 scrub starts 2024-09-20T22:17:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:56 smithi067 ceph-mon[33511]: 3.15 scrub ok 2024-09-20T22:17:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:56 smithi067 ceph-mon[33511]: 1.11 scrub starts 2024-09-20T22:17:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:56 smithi067 ceph-mon[33511]: 1.11 scrub ok 2024-09-20T22:17:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:56 smithi067 ceph-mon[33511]: 3.16 scrub starts 2024-09-20T22:17:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:56 smithi067 ceph-mon[33511]: 3.16 scrub ok 2024-09-20T22:17:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:56 smithi067 ceph-mon[33511]: 6.c scrub starts 2024-09-20T22:17:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:56 smithi067 ceph-mon[33511]: 6.c scrub ok 2024-09-20T22:17:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:56 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2920434401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-20T22:17:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:56 smithi067 ceph-mon[33511]: 4.19 scrub starts 2024-09-20T22:17:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:56 smithi067 ceph-mon[33511]: 4.19 scrub ok 2024-09-20T22:17:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:56 smithi067 ceph-mon[33511]: 1.16 scrub starts 2024-09-20T22:17:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:56 smithi067 ceph-mon[33511]: 1.16 scrub ok 2024-09-20T22:17:56.772 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[29694]: 3.15 scrub starts 2024-09-20T22:17:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[29694]: 3.15 scrub ok 2024-09-20T22:17:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[29694]: 1.11 scrub starts 2024-09-20T22:17:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[29694]: 1.11 scrub ok 2024-09-20T22:17:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[29694]: 3.16 scrub starts 2024-09-20T22:17:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[29694]: 3.16 scrub ok 2024-09-20T22:17:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[29694]: 6.c scrub starts 2024-09-20T22:17:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[29694]: 6.c scrub ok 2024-09-20T22:17:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2920434401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-20T22:17:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[29694]: 4.19 scrub starts 2024-09-20T22:17:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[29694]: 4.19 scrub ok 2024-09-20T22:17:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[29694]: 1.16 scrub starts 2024-09-20T22:17:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[29694]: 1.16 scrub ok 2024-09-20T22:17:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[40290]: 3.15 scrub starts 2024-09-20T22:17:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[40290]: 3.15 scrub ok 2024-09-20T22:17:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[40290]: 1.11 scrub starts 2024-09-20T22:17:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[40290]: 1.11 scrub ok 2024-09-20T22:17:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[40290]: 3.16 scrub starts 2024-09-20T22:17:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[40290]: 3.16 scrub ok 2024-09-20T22:17:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[40290]: 6.c scrub starts 2024-09-20T22:17:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[40290]: 6.c scrub ok 2024-09-20T22:17:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2920434401' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json-pretty"}]: dispatch 2024-09-20T22:17:56.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[40290]: 4.19 scrub starts 2024-09-20T22:17:56.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[40290]: 4.19 scrub ok 2024-09-20T22:17:56.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[40290]: 1.16 scrub starts 2024-09-20T22:17:56.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:56 smithi060 ceph-mon[40290]: 1.16 scrub ok 2024-09-20T22:17:56.805 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:56.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:56.806 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:56.907 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:56.943 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:56.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:56.944 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:57.044 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:57.080 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:57.081 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:57.081 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:57.182 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:57.205 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:57.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:57.206 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:57.307 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:57.346 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:57.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:57.347 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:57.447 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:57.484 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:57.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:57.485 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:57.586 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:57.608 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:57.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:57.609 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:57.709 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:57.732 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:57.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:57.733 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:57 smithi067 ceph-mon[33511]: 1.19 scrub starts 2024-09-20T22:17:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:57 smithi067 ceph-mon[33511]: 1.19 scrub ok 2024-09-20T22:17:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:57 smithi067 ceph-mon[33511]: pgmap v804: 178 pgs: 178 active+clean; 2.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-20T22:17:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:57 smithi067 ceph-mon[33511]: 1.18 scrub starts 2024-09-20T22:17:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:57 smithi067 ceph-mon[33511]: 1.18 scrub ok 2024-09-20T22:17:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:57 smithi067 ceph-mon[33511]: 1.1a scrub starts 2024-09-20T22:17:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:57 smithi067 ceph-mon[33511]: 1.1a scrub ok 2024-09-20T22:17:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:57 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:17:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:57 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:17:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:57 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:17:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:57 smithi067 ceph-mon[33511]: 1.1e scrub starts 2024-09-20T22:17:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:57 smithi067 ceph-mon[33511]: 1.1e scrub ok 2024-09-20T22:17:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:57 smithi067 ceph-mon[33511]: 1.1d scrub starts 2024-09-20T22:17:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:57 smithi067 ceph-mon[33511]: 1.1d scrub ok 2024-09-20T22:17:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[29694]: 1.19 scrub starts 2024-09-20T22:17:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[29694]: 1.19 scrub ok 2024-09-20T22:17:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[29694]: pgmap v804: 178 pgs: 178 active+clean; 2.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-20T22:17:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[29694]: 1.18 scrub starts 2024-09-20T22:17:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[29694]: 1.18 scrub ok 2024-09-20T22:17:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[29694]: 1.1a scrub starts 2024-09-20T22:17:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[29694]: 1.1a scrub ok 2024-09-20T22:17:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:17:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:17:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:17:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[29694]: 1.1e scrub starts 2024-09-20T22:17:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[29694]: 1.1e scrub ok 2024-09-20T22:17:57.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[29694]: 1.1d scrub starts 2024-09-20T22:17:57.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[29694]: 1.1d scrub ok 2024-09-20T22:17:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[40290]: 1.19 scrub starts 2024-09-20T22:17:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[40290]: 1.19 scrub ok 2024-09-20T22:17:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[40290]: pgmap v804: 178 pgs: 178 active+clean; 2.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.16k op/s 2024-09-20T22:17:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[40290]: 1.18 scrub starts 2024-09-20T22:17:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[40290]: 1.18 scrub ok 2024-09-20T22:17:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[40290]: 1.1a scrub starts 2024-09-20T22:17:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[40290]: 1.1a scrub ok 2024-09-20T22:17:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:17:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:17:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:17:57.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[40290]: 1.1e scrub starts 2024-09-20T22:17:57.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[40290]: 1.1e scrub ok 2024-09-20T22:17:57.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[40290]: 1.1d scrub starts 2024-09-20T22:17:57.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:57 smithi060 ceph-mon[40290]: 1.1d scrub ok 2024-09-20T22:17:57.834 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:57.872 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:57.873 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:57.873 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:57.974 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:58.002 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:58.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:58.003 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:58.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:17:58.126 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:58.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:58.127 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:17:58.227 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:17:58.260 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:58.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:58.261 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:17:58.362 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:58.394 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:58.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:58.396 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:58.497 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:58.533 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:58.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:58.534 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:58.634 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:58.668 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:58.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:58.669 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:58 smithi067 ceph-mon[33511]: 1.15 scrub starts 2024-09-20T22:17:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:58 smithi067 ceph-mon[33511]: 1.15 scrub ok 2024-09-20T22:17:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:58 smithi067 ceph-mon[33511]: 5.5 scrub starts 2024-09-20T22:17:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:58 smithi067 ceph-mon[33511]: 5.5 scrub ok 2024-09-20T22:17:58.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:58 smithi067 ceph-mon[33511]: 1.17 scrub starts 2024-09-20T22:17:58.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:58 smithi067 ceph-mon[33511]: 1.17 scrub ok 2024-09-20T22:17:58.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:58 smithi067 ceph-mon[33511]: osdmap e437: 8 total, 8 up, 8 in 2024-09-20T22:17:58.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:58 smithi067 ceph-mon[33511]: 4.5 scrub starts 2024-09-20T22:17:58.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:58 smithi067 ceph-mon[33511]: 4.5 scrub ok 2024-09-20T22:17:58.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:58 smithi067 ceph-mon[33511]: 3.8 scrub starts 2024-09-20T22:17:58.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:58 smithi067 ceph-mon[33511]: 3.8 scrub ok 2024-09-20T22:17:58.769 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:58.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[29694]: 1.15 scrub starts 2024-09-20T22:17:58.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[29694]: 1.15 scrub ok 2024-09-20T22:17:58.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[29694]: 5.5 scrub starts 2024-09-20T22:17:58.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[29694]: 5.5 scrub ok 2024-09-20T22:17:58.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[29694]: 1.17 scrub starts 2024-09-20T22:17:58.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[29694]: 1.17 scrub ok 2024-09-20T22:17:58.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[29694]: osdmap e437: 8 total, 8 up, 8 in 2024-09-20T22:17:58.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[29694]: 4.5 scrub starts 2024-09-20T22:17:58.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[29694]: 4.5 scrub ok 2024-09-20T22:17:58.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[29694]: 3.8 scrub starts 2024-09-20T22:17:58.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[29694]: 3.8 scrub ok 2024-09-20T22:17:58.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[40290]: 1.15 scrub starts 2024-09-20T22:17:58.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[40290]: 1.15 scrub ok 2024-09-20T22:17:58.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[40290]: 5.5 scrub starts 2024-09-20T22:17:58.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[40290]: 5.5 scrub ok 2024-09-20T22:17:58.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[40290]: 1.17 scrub starts 2024-09-20T22:17:58.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[40290]: 1.17 scrub ok 2024-09-20T22:17:58.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[40290]: osdmap e437: 8 total, 8 up, 8 in 2024-09-20T22:17:58.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[40290]: 4.5 scrub starts 2024-09-20T22:17:58.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[40290]: 4.5 scrub ok 2024-09-20T22:17:58.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[40290]: 3.8 scrub starts 2024-09-20T22:17:58.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:58 smithi060 ceph-mon[40290]: 3.8 scrub ok 2024-09-20T22:17:58.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:58.809 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:58.809 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:58.910 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:17:58.937 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:58.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:58.938 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:17:59.039 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:59.073 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:59.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:59.074 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:59.174 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:59.217 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:59.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:59.218 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:59.319 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:59.340 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:59.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:59.341 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:59.442 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:17:59.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:59.481 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:59.481 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:17:59.582 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:17:59.621 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:59.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:59.622 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:17:59.722 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:17:59.755 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:17:59.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:59.756 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:17:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:59 smithi067 ceph-mon[33511]: 1.1c scrub starts 2024-09-20T22:17:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:59 smithi067 ceph-mon[33511]: 1.1c scrub ok 2024-09-20T22:17:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:59 smithi067 ceph-mon[33511]: pgmap v806: 178 pgs: 178 active+clean; 2.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 0 B/s wr, 1.53k op/s 2024-09-20T22:17:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:59 smithi067 ceph-mon[33511]: 1.1b scrub starts 2024-09-20T22:17:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:17:59 smithi067 ceph-mon[33511]: 1.1b scrub ok 2024-09-20T22:17:59.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:59 smithi060 ceph-mon[40290]: 1.1c scrub starts 2024-09-20T22:17:59.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:59 smithi060 ceph-mon[40290]: 1.1c scrub ok 2024-09-20T22:17:59.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:59 smithi060 ceph-mon[40290]: pgmap v806: 178 pgs: 178 active+clean; 2.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 0 B/s wr, 1.53k op/s 2024-09-20T22:17:59.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:59 smithi060 ceph-mon[40290]: 1.1b scrub starts 2024-09-20T22:17:59.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:17:59 smithi060 ceph-mon[40290]: 1.1b scrub ok 2024-09-20T22:17:59.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:59 smithi060 ceph-mon[29694]: 1.1c scrub starts 2024-09-20T22:17:59.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:59 smithi060 ceph-mon[29694]: 1.1c scrub ok 2024-09-20T22:17:59.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:59 smithi060 ceph-mon[29694]: pgmap v806: 178 pgs: 178 active+clean; 2.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 0 B/s wr, 1.53k op/s 2024-09-20T22:17:59.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:59 smithi060 ceph-mon[29694]: 1.1b scrub starts 2024-09-20T22:17:59.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:17:59 smithi060 ceph-mon[29694]: 1.1b scrub ok 2024-09-20T22:17:59.857 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:17:59.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:17:59.896 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:17:59.896 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:17:59.996 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:00.038 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:00.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:00.039 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:00.097 INFO:tasks.thrashosds.thrasher:j is {'epoch': 436, 'fsid': '3df9647e-779b-11ef-baf5-efdc52797490', 'created': '2024-09-20T21:57:31.244934+0000', 'modified': '2024-09-20T22:17:53.426581+0000', 'last_up_change': '2024-09-20T22:06:53.796679+0000', 'last_in_change': '2024-09-20T22:12:51.491976+0000', 'flags': 'nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5804032, 'flags_set': ['nodeep-scrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 19, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 10, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-20T22:00:35.131125+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 31, 'pg_placement_num': 31, 'pg_placement_num_target': 31, 'pg_num_target': 31, 'pg_num_pending': 31, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '430', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '428', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.809999942779541, 'score_stable': 1.809999942779541, 'optimal_score': 1, 'raw_score_acting': 1.809999942779541, 'raw_score_stable': 1.809999942779541, 'primary_affinity_weighted': 0.9326171875, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 2, 'pool_name': 'datapool', 'create_time': '2024-09-20T22:02:47.974480+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.6500000953674316, 'score_stable': 2.6500000953674316, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 2.3299999237060547, 'raw_score_stable': 2.3299999237060547, 'primary_affinity_weighted': 0.9326171875, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-09-20T22:02:48.902465+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 0.9326171875, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-09-20T22:02:50.620057+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.9326171875, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-09-20T22:02:52.722912+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '69', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9326171875, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-09-20T22:02:54.794248+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '71', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9326171875, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}, {'pool': 10, 'pool_name': 'unique_pool_3', 'create_time': '2024-09-20T22:15:07.861275+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 16, 'pg_placement_num': 16, 'pg_placement_num_target': 16, 'pg_num_target': 16, 'pg_num_pending': 16, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '374', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rados': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 3, 'score_stable': 3, 'optimal_score': 1, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9326171875, 'average_primary_affinity': 0.9300000071525574, 'average_primary_affinity_weighted': 0.9300000071525574}}], 'osds': [{'osd': 0, 'uuid': '0997c01a-a9bd-421e-abcc-fbcf7abe8512', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 8, 'up_thru': 433, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6802', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6803', 'nonce': 1395849550}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6804', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6805', 'nonce': 1395849550}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6808', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6809', 'nonce': 1395849550}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6806', 'nonce': 1395849550}, {'type': 'v1', 'addr': '172.21.15.60:6807', 'nonce': 1395849550}]}, 'public_addr': '172.21.15.60:6803/1395849550', 'cluster_addr': '172.21.15.60:6805/1395849550', 'heartbeat_back_addr': '172.21.15.60:6809/1395849550', 'heartbeat_front_addr': '172.21.15.60:6807/1395849550', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'c04bdcb1-d76b-47c8-a38b-c02043df6ba8', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 90, 'up_from': 145, 'up_thru': 433, 'down_at': 91, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6810', 'nonce': 2970874921}, {'type': 'v1', 'addr': '172.21.15.60:6811', 'nonce': 2970874921}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6812', 'nonce': 2970874921}, {'type': 'v1', 'addr': '172.21.15.60:6813', 'nonce': 2970874921}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6816', 'nonce': 2970874921}, {'type': 'v1', 'addr': '172.21.15.60:6817', 'nonce': 2970874921}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6814', 'nonce': 2970874921}, {'type': 'v1', 'addr': '172.21.15.60:6815', 'nonce': 2970874921}]}, 'public_addr': '172.21.15.60:6811/2970874921', 'cluster_addr': '172.21.15.60:6813/2970874921', 'heartbeat_back_addr': '172.21.15.60:6817/2970874921', 'heartbeat_front_addr': '172.21.15.60:6815/2970874921', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '32c09d19-eb98-4926-a44e-e7ddee152956', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 18, 'up_thru': 432, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6818', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6819', 'nonce': 3698562274}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6820', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6821', 'nonce': 3698562274}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6824', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6825', 'nonce': 3698562274}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6822', 'nonce': 3698562274}, {'type': 'v1', 'addr': '172.21.15.60:6823', 'nonce': 3698562274}]}, 'public_addr': '172.21.15.60:6819/3698562274', 'cluster_addr': '172.21.15.60:6821/3698562274', 'heartbeat_back_addr': '172.21.15.60:6825/3698562274', 'heartbeat_front_addr': '172.21.15.60:6823/3698562274', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': 'b1c2cb69-8565-49ac-b854-7ea7615c6660', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.4609375, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 26, 'up_thru': 433, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6826', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6827', 'nonce': 2592322449}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6828', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6829', 'nonce': 2592322449}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6832', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6833', 'nonce': 2592322449}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.60:6830', 'nonce': 2592322449}, {'type': 'v1', 'addr': '172.21.15.60:6831', 'nonce': 2592322449}]}, 'public_addr': '172.21.15.60:6827/2592322449', 'cluster_addr': '172.21.15.60:6829/2592322449', 'heartbeat_back_addr': '172.21.15.60:6833/2592322449', 'heartbeat_front_addr': '172.21.15.60:6831/2592322449', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '04a547d6-fc48-43b7-a8d7-18df8a83aa96', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 31, 'up_thru': 433, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6800', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6801', 'nonce': 3243564239}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6802', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6803', 'nonce': 3243564239}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6806', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6807', 'nonce': 3243564239}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6804', 'nonce': 3243564239}, {'type': 'v1', 'addr': '172.21.15.67:6805', 'nonce': 3243564239}]}, 'public_addr': '172.21.15.67:6801/3243564239', 'cluster_addr': '172.21.15.67:6803/3243564239', 'heartbeat_back_addr': '172.21.15.67:6807/3243564239', 'heartbeat_front_addr': '172.21.15.67:6805/3243564239', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '4763aa10-c367-485b-b525-e5241e642b86', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 39, 'up_thru': 433, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6808', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6809', 'nonce': 1622258887}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6810', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6811', 'nonce': 1622258887}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6814', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6815', 'nonce': 1622258887}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6812', 'nonce': 1622258887}, {'type': 'v1', 'addr': '172.21.15.67:6813', 'nonce': 1622258887}]}, 'public_addr': '172.21.15.67:6809/1622258887', 'cluster_addr': '172.21.15.67:6811/1622258887', 'heartbeat_back_addr': '172.21.15.67:6815/1622258887', 'heartbeat_front_addr': '172.21.15.67:6813/1622258887', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': 'a359116e-c013-4e2f-8b79-86219516ee16', 'up': 1, 'in': 1, 'weight': 0.95001220703125, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 48, 'up_thru': 428, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6816', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6817', 'nonce': 1187114}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6818', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6819', 'nonce': 1187114}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6822', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6823', 'nonce': 1187114}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6820', 'nonce': 1187114}, {'type': 'v1', 'addr': '172.21.15.67:6821', 'nonce': 1187114}]}, 'public_addr': '172.21.15.67:6817/1187114', 'cluster_addr': '172.21.15.67:6819/1187114', 'heartbeat_back_addr': '172.21.15.67:6823/1187114', 'heartbeat_front_addr': '172.21.15.67:6821/1187114', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': 'a4bf4188-f133-4a31-8897-d7e8182248d7', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 57, 'up_thru': 433, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6824', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6825', 'nonce': 3021842840}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6826', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6827', 'nonce': 3021842840}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6830', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6831', 'nonce': 3021842840}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.67:6828', 'nonce': 3021842840}, {'type': 'v1', 'addr': '172.21.15.67:6829', 'nonce': 3021842840}]}, 'public_addr': '172.21.15.67:6825/3021842840', 'cluster_addr': '172.21.15.67:6827/3021842840', 'heartbeat_back_addr': '172.21.15.67:6831/3021842840', 'heartbeat_front_addr': '172.21.15.67:6829/3021842840', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:16:39.012311+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:17:02.041351+0000', 'dead_epoch': 90}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:17:12.682915+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:17:15.514516+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:17:07.357814+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:17:11.911243+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:17:10.226518+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-09-20T22:17:25.435787+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '1.8', 'osds': [6, 7, 2]}, {'pgid': '3.c', 'osds': [3, 1, 6]}, {'pgid': '6.13', 'osds': [0, 7, 6]}], 'pg_upmap_items': [{'pgid': '3.1a', 'mappings': [{'from': 4, 'to': 5}]}, {'pgid': '3.1c', 'mappings': [{'from': 4, 'to': 2}]}, {'pgid': '4.b', 'mappings': [{'from': 1, 'to': 7}]}, {'pgid': '4.f', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.13', 'mappings': [{'from': 1, 'to': 2}]}, {'pgid': '4.1b', 'mappings': [{'from': 1, 'to': 5}]}, {'pgid': '6.9', 'mappings': [{'from': 7, 'to': 1}]}, {'pgid': '6.11', 'mappings': [{'from': 3, 'to': 1}]}, {'pgid': '6.19', 'mappings': [{'from': 3, 'to': 6}]}], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {'172.21.15.60:0/890154710': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:0/3386526350': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:6801/604714621': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:6800/604714621': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:6800/3151907776': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/956264518': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/4150585713': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/3311255314': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:0/3965541528': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:6800/3734574621': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:0/1729862661': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:6801/3734574621': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:0/3927294026': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:6801/3151907776': '2024-09-21T21:58:02.105557+0000', '172.21.15.60:0/1871351181': '2024-09-21T21:58:26.613414+0000', '172.21.15.60:0/1854455098': '2024-09-21T22:03:19.518405+0000', '172.21.15.60:0/2221262203': '2024-09-21T22:03:19.518405+0000'}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-09-20T22:18:00.097 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:18:00.140 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:00.192 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:00.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:00.194 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:00.294 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:00.340 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:00.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:00.341 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:00.370 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:00.442 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:00.483 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:00.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:00.484 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:00.585 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:00.615 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:00.616 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:00.616 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:00.717 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:00.752 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:00.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:00.753 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:00 smithi067 ceph-mon[33511]: 3.c scrub starts 2024-09-20T22:18:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:00 smithi067 ceph-mon[33511]: 3.c scrub ok 2024-09-20T22:18:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:00 smithi067 ceph-mon[33511]: 10.5 scrub starts 2024-09-20T22:18:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:00 smithi060 ceph-mon[29694]: 3.c scrub starts 2024-09-20T22:18:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:00 smithi060 ceph-mon[29694]: 3.c scrub ok 2024-09-20T22:18:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:00 smithi060 ceph-mon[29694]: 10.5 scrub starts 2024-09-20T22:18:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:00 smithi060 ceph-mon[40290]: 3.c scrub starts 2024-09-20T22:18:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:00 smithi060 ceph-mon[40290]: 3.c scrub ok 2024-09-20T22:18:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:00 smithi060 ceph-mon[40290]: 10.5 scrub starts 2024-09-20T22:18:00.853 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:00.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:00.915 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:00.915 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:00.978 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:01.016 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:01.048 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:01.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:01.049 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:01.149 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:01.187 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:01.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:01.188 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:01.289 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:01.329 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:01.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:01.330 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:01.431 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:01.464 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:01.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:01.465 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:01.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:01 smithi060 ceph-mon[29694]: 2.0 scrub starts 2024-09-20T22:18:01.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:01 smithi060 ceph-mon[29694]: 2.0 scrub ok 2024-09-20T22:18:01.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:01 smithi060 ceph-mon[29694]: pgmap v807: 178 pgs: 178 active+clean; 2.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 0 B/s wr, 1.22k op/s 2024-09-20T22:18:01.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:01 smithi060 ceph-mon[29694]: 3.10 scrub starts 2024-09-20T22:18:01.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:01 smithi060 ceph-mon[29694]: 3.10 scrub ok 2024-09-20T22:18:01.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:01 smithi060 ceph-mon[29694]: 10.5 scrub ok 2024-09-20T22:18:01.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:01 smithi060 ceph-mon[29694]: 1.3 scrub starts 2024-09-20T22:18:01.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:01 smithi060 ceph-mon[29694]: 1.3 scrub ok 2024-09-20T22:18:01.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:01 smithi060 ceph-mon[40290]: 2.0 scrub starts 2024-09-20T22:18:01.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:01 smithi060 ceph-mon[40290]: 2.0 scrub ok 2024-09-20T22:18:01.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:01 smithi060 ceph-mon[40290]: pgmap v807: 178 pgs: 178 active+clean; 2.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 0 B/s wr, 1.22k op/s 2024-09-20T22:18:01.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:01 smithi060 ceph-mon[40290]: 3.10 scrub starts 2024-09-20T22:18:01.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:01 smithi060 ceph-mon[40290]: 3.10 scrub ok 2024-09-20T22:18:01.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:01 smithi060 ceph-mon[40290]: 10.5 scrub ok 2024-09-20T22:18:01.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:01 smithi060 ceph-mon[40290]: 1.3 scrub starts 2024-09-20T22:18:01.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:01 smithi060 ceph-mon[40290]: 1.3 scrub ok 2024-09-20T22:18:01.565 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:01.605 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:01.606 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:01.606 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:01.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:01.742 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:01.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:01.743 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:01 smithi067 ceph-mon[33511]: 2.0 scrub starts 2024-09-20T22:18:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:01 smithi067 ceph-mon[33511]: 2.0 scrub ok 2024-09-20T22:18:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:01 smithi067 ceph-mon[33511]: pgmap v807: 178 pgs: 178 active+clean; 2.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 0 B/s wr, 1.22k op/s 2024-09-20T22:18:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:01 smithi067 ceph-mon[33511]: 3.10 scrub starts 2024-09-20T22:18:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:01 smithi067 ceph-mon[33511]: 3.10 scrub ok 2024-09-20T22:18:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:01 smithi067 ceph-mon[33511]: 10.5 scrub ok 2024-09-20T22:18:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:01 smithi067 ceph-mon[33511]: 1.3 scrub starts 2024-09-20T22:18:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:01 smithi067 ceph-mon[33511]: 1.3 scrub ok 2024-09-20T22:18:01.818 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:01.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:01.874 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:18:01.874 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:18:01.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:18:01.875 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:18:01.875 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.875 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105654 8.1 8:83678158:::benchmark_data_smithi060_111051_object105653:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:18:01.875 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.549939+0000", 2024-09-20T22:18:01.875 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 533.31815341100003, 2024-09-20T22:18:01.875 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.003362782, 2024-09-20T22:18:01.875 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.875 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.875 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:18:01.875 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.875 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:01.875 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:01.875 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105654 2024-09-20T22:18:01.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.876 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549939+0000", 2024-09-20T22:18:01.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549939+0000", 2024-09-20T22:18:01.876 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.876 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.876 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.876 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:01.876 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549940+0000", 2024-09-20T22:18:01.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.95e-06 2024-09-20T22:18:01.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:01.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549973+0000", 2024-09-20T22:18:01.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2798e-05 2024-09-20T22:18:01.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:01.877 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.549997+0000", 2024-09-20T22:18:01.877 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3649e-05 2024-09-20T22:18:01.877 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.877 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.877 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:01.878 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550007+0000", 2024-09-20T22:18:01.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8190000000000003e-06 2024-09-20T22:18:01.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.878 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702300+0000", 2024-09-20T22:18:01.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.152292975 2024-09-20T22:18:01.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.878 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:01.878 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702310+0000", 2024-09-20T22:18:01.878 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0453e-05 2024-09-20T22:18:01.878 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.878 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.553223+0000", 2024-09-20T22:18:01.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85091275799999999 2024-09-20T22:18:01.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.879 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:01.879 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.553301+0000", 2024-09-20T22:18:01.879 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8380000000000005e-05 2024-09-20T22:18:01.879 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.879 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:01.879 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.879 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.879 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.879 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105657 8.7 8:e060afd6:::benchmark_data_smithi060_111051_object105656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:18:01.879 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.550025+0000", 2024-09-20T22:18:01.880 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 533.31806689500002, 2024-09-20T22:18:01.880 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.029495933, 2024-09-20T22:18:01.880 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.880 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.880 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:01.880 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.880 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:01.880 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:01.880 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105657 2024-09-20T22:18:01.880 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.880 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.880 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.880 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.880 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550025+0000", 2024-09-20T22:18:01.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550025+0000", 2024-09-20T22:18:01.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:01.881 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550026+0000", 2024-09-20T22:18:01.881 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1799999999999995e-07 2024-09-20T22:18:01.881 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.881 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.881 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:01.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550051+0000", 2024-09-20T22:18:01.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5554999999999998e-05 2024-09-20T22:18:01.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:01.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550062+0000", 2024-09-20T22:18:01.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.137e-05 2024-09-20T22:18:01.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:01.882 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550068+0000", 2024-09-20T22:18:01.882 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8289999999999996e-06 2024-09-20T22:18:01.882 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.882 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.882 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720577+0000", 2024-09-20T22:18:01.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.170508778 2024-09-20T22:18:01.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.883 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:01.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720588+0000", 2024-09-20T22:18:01.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0808000000000001e-05 2024-09-20T22:18:01.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.883 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.883 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.883 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.569210+0000", 2024-09-20T22:18:01.883 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.84862162900000004 2024-09-20T22:18:01.883 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:01.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.569360+0000", 2024-09-20T22:18:01.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015044799999999999 2024-09-20T22:18:01.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:18:01.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.569667+0000", 2024-09-20T22:18:01.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030655599999999998 2024-09-20T22:18:01.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.884 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.884 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:01.884 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.572342+0000", 2024-09-20T22:18:01.884 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0026757840000000001 2024-09-20T22:18:01.884 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.573025+0000", 2024-09-20T22:18:01.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00068279199999999999 2024-09-20T22:18:01.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579397+0000", 2024-09-20T22:18:01.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.006372071 2024-09-20T22:18:01.885 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.885 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.885 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:01.885 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579465+0000", 2024-09-20T22:18:01.885 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7520000000000004e-05 2024-09-20T22:18:01.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.886 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:01.886 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579521+0000", 2024-09-20T22:18:01.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6274999999999999e-05 2024-09-20T22:18:01.886 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.886 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:01.886 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.886 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.886 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.886 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105658 8.1 8:809ea6c9:::benchmark_data_smithi060_111051_object105657:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:18:01.886 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.550079+0000", 2024-09-20T22:18:01.886 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 533.31801331199995, 2024-09-20T22:18:01.886 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.010609463, 2024-09-20T22:18:01.887 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.887 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.887 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:01.887 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.887 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:01.887 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:01.887 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105658 2024-09-20T22:18:01.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.887 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.887 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.887 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550079+0000", 2024-09-20T22:18:01.887 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.887 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.887 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550079+0000", 2024-09-20T22:18:01.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:01.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550079+0000", 2024-09-20T22:18:01.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2599999999999998e-07 2024-09-20T22:18:01.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.888 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.888 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:01.888 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550101+0000", 2024-09-20T22:18:01.888 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1628e-05 2024-09-20T22:18:01.888 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:01.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550111+0000", 2024-09-20T22:18:01.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9620000000000005e-06 2024-09-20T22:18:01.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:01.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550116+0000", 2024-09-20T22:18:01.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9810000000000003e-06 2024-09-20T22:18:01.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.889 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.889 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.889 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702335+0000", 2024-09-20T22:18:01.889 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15221938700000001 2024-09-20T22:18:01.889 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:01.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702343+0000", 2024-09-20T22:18:01.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2120000000000003e-06 2024-09-20T22:18:01.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.553649+0000", 2024-09-20T22:18:01.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85130619600000002 2024-09-20T22:18:01.890 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.890 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.890 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:01.890 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.553784+0000", 2024-09-20T22:18:01.890 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000134499 2024-09-20T22:18:01.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:18:01.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.554060+0000", 2024-09-20T22:18:01.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000276579 2024-09-20T22:18:01.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:01.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.557338+0000", 2024-09-20T22:18:01.891 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003277608 2024-09-20T22:18:01.891 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.891 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.891 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.891 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.559599+0000", 2024-09-20T22:18:01.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002261304 2024-09-20T22:18:01.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560617+0000", 2024-09-20T22:18:01.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001017354 2024-09-20T22:18:01.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:01.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560660+0000", 2024-09-20T22:18:01.892 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3167999999999997e-05 2024-09-20T22:18:01.892 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.892 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.892 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:01.892 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560688+0000", 2024-09-20T22:18:01.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8158999999999999e-05 2024-09-20T22:18:01.893 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.893 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:01.893 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.893 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.893 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.893 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105659 8.1 8:80c09cff:::benchmark_data_smithi060_111051_object105658:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:18:01.893 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.550123+0000", 2024-09-20T22:18:01.893 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 533.31796857799998, 2024-09-20T22:18:01.893 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0107473760000001, 2024-09-20T22:18:01.893 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.893 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.893 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:01.893 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.894 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:01.894 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:01.894 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105659 2024-09-20T22:18:01.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.894 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550123+0000", 2024-09-20T22:18:01.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.894 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.894 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.894 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.894 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550123+0000", 2024-09-20T22:18:01.894 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:01.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550124+0000", 2024-09-20T22:18:01.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.39e-07 2024-09-20T22:18:01.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:01.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550148+0000", 2024-09-20T22:18:01.895 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3944e-05 2024-09-20T22:18:01.895 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.895 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.895 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:01.895 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550157+0000", 2024-09-20T22:18:01.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5410000000000006e-06 2024-09-20T22:18:01.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:01.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550162+0000", 2024-09-20T22:18:01.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8069999999999999e-06 2024-09-20T22:18:01.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702368+0000", 2024-09-20T22:18:01.896 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15220560899999999 2024-09-20T22:18:01.896 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.896 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.896 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:01.896 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.702374+0000", 2024-09-20T22:18:01.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7939999999999997e-06 2024-09-20T22:18:01.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.555649+0000", 2024-09-20T22:18:01.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85327487000000002 2024-09-20T22:18:01.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:01.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.555819+0000", 2024-09-20T22:18:01.897 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016938200000000001 2024-09-20T22:18:01.897 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.897 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.897 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,2", 2024-09-20T22:18:01.897 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.556100+0000", 2024-09-20T22:18:01.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00028165699999999999 2024-09-20T22:18:01.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:01.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.558925+0000", 2024-09-20T22:18:01.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0028247290000000002 2024-09-20T22:18:01.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.898 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.559791+0000", 2024-09-20T22:18:01.898 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00086584599999999998 2024-09-20T22:18:01.898 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.898 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.898 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560771+0000", 2024-09-20T22:18:01.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00097968300000000007 2024-09-20T22:18:01.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:01.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560821+0000", 2024-09-20T22:18:01.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0460999999999997e-05 2024-09-20T22:18:01.899 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.899 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.899 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:01.899 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.560871+0000", 2024-09-20T22:18:01.899 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9614000000000003e-05 2024-09-20T22:18:01.899 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.899 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:01.899 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.900 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.900 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105660 8.13 8:cf39f948:::benchmark_data_smithi060_111051_object105659:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:18:01.900 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.550169+0000", 2024-09-20T22:18:01.900 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 533.317922845, 2024-09-20T22:18:01.900 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0659826960000001, 2024-09-20T22:18:01.900 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.900 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.900 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:01.900 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.900 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:01.900 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:01.900 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105660 2024-09-20T22:18:01.900 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.901 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550169+0000", 2024-09-20T22:18:01.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550169+0000", 2024-09-20T22:18:01.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.901 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.901 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.901 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:01.901 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550169+0000", 2024-09-20T22:18:01.901 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9099999999999999e-07 2024-09-20T22:18:01.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:01.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550221+0000", 2024-09-20T22:18:01.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1844e-05 2024-09-20T22:18:01.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:01.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550233+0000", 2024-09-20T22:18:01.902 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1871e-05 2024-09-20T22:18:01.902 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.902 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.902 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:01.902 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.550238+0000", 2024-09-20T22:18:01.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6750000000000001e-06 2024-09-20T22:18:01.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.701152+0000", 2024-09-20T22:18:01.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15091443500000001 2024-09-20T22:18:01.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:01.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.701161+0000", 2024-09-20T22:18:01.903 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2679999999999994e-06 2024-09-20T22:18:01.903 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.903 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.903 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.903 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.605154+0000", 2024-09-20T22:18:01.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.90399336100000005 2024-09-20T22:18:01.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:01.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.605243+0000", 2024-09-20T22:18:01.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8661999999999999e-05 2024-09-20T22:18:01.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 5,6", 2024-09-20T22:18:01.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.605424+0000", 2024-09-20T22:18:01.904 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018181800000000001 2024-09-20T22:18:01.904 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.904 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.904 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:01.904 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.606520+0000", 2024-09-20T22:18:01.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001095333 2024-09-20T22:18:01.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.905 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.612488+0000", 2024-09-20T22:18:01.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0059685019999999997 2024-09-20T22:18:01.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.905 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.905 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.616043+0000", 2024-09-20T22:18:01.905 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0035544249999999999 2024-09-20T22:18:01.905 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.905 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.905 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:01.906 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.616108+0000", 2024-09-20T22:18:01.906 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5506999999999995e-05 2024-09-20T22:18:01.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.906 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:01.906 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.616152+0000", 2024-09-20T22:18:01.906 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3603999999999998e-05 2024-09-20T22:18:01.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.906 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:01.906 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.906 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.906 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:105668 8.7 8:e0173451:::benchmark_data_smithi060_111051_object105667:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e215)", 2024-09-20T22:18:01.906 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:08.706681+0000", 2024-09-20T22:18:01.907 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 533.16141104400003, 2024-09-20T22:18:01.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.87300058000000003, 2024-09-20T22:18:01.907 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.907 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.907 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:01.907 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.907 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:01.907 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:01.907 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 105668 2024-09-20T22:18:01.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.907 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706681+0000", 2024-09-20T22:18:01.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706681+0000", 2024-09-20T22:18:01.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:01.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706683+0000", 2024-09-20T22:18:01.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.928e-06 2024-09-20T22:18:01.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:01.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706710+0000", 2024-09-20T22:18:01.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6911e-05 2024-09-20T22:18:01.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:01.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706734+0000", 2024-09-20T22:18:01.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3992999999999999e-05 2024-09-20T22:18:01.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:01.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.706743+0000", 2024-09-20T22:18:01.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1209999999999996e-06 2024-09-20T22:18:01.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720612+0000", 2024-09-20T22:18:01.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.013869283 2024-09-20T22:18:01.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:01.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:08.720619+0000", 2024-09-20T22:18:01.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7100000000000001e-06 2024-09-20T22:18:01.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.571247+0000", 2024-09-20T22:18:01.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.85062784499999999 2024-09-20T22:18:01.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:01.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.571381+0000", 2024-09-20T22:18:01.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000134167 2024-09-20T22:18:01.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,6", 2024-09-20T22:18:01.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.571594+0000", 2024-09-20T22:18:01.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021272700000000001 2024-09-20T22:18:01.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:01.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.572500+0000", 2024-09-20T22:18:01.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00090607800000000005 2024-09-20T22:18:01.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.574997+0000", 2024-09-20T22:18:01.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024976759999999999 2024-09-20T22:18:01.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579580+0000", 2024-09-20T22:18:01.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0045830269999999999 2024-09-20T22:18:01.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:01.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579636+0000", 2024-09-20T22:18:01.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5413e-05 2024-09-20T22:18:01.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:01.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:09.579681+0000", 2024-09-20T22:18:01.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5701000000000002e-05 2024-09-20T22:18:01.913 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.913 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:01.913 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.913 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106973 8.13 8:cbbe7a4a:::benchmark_data_smithi060_111051_object106972:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=2 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:18:01.913 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.414028+0000", 2024-09-20T22:18:01.913 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 530.45406360300001, 2024-09-20T22:18:01.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0848815410000001, 2024-09-20T22:18:01.914 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.914 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.914 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:01.914 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.915 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:01.915 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:01.915 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106973 2024-09-20T22:18:01.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.915 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414028+0000", 2024-09-20T22:18:01.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.915 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.915 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414028+0000", 2024-09-20T22:18:01.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:01.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414030+0000", 2024-09-20T22:18:01.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.866e-06 2024-09-20T22:18:01.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:01.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414059+0000", 2024-09-20T22:18:01.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9043999999999998e-05 2024-09-20T22:18:01.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:01.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414080+0000", 2024-09-20T22:18:01.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1254999999999999e-05 2024-09-20T22:18:01.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:01.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414090+0000", 2024-09-20T22:18:01.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2129999999999997e-06 2024-09-20T22:18:01.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.415798+0000", 2024-09-20T22:18:01.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017082340000000001 2024-09-20T22:18:01.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:01.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.415809+0000", 2024-09-20T22:18:01.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1433e-05 2024-09-20T22:18:01.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.479861+0000", 2024-09-20T22:18:01.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.064051522 2024-09-20T22:18:01.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:01.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.479994+0000", 2024-09-20T22:18:01.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013297800000000001 2024-09-20T22:18:01.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5,6", 2024-09-20T22:18:01.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.480283+0000", 2024-09-20T22:18:01.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00028927100000000002 2024-09-20T22:18:01.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.482942+0000", 2024-09-20T22:18:01.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002658737 2024-09-20T22:18:01.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.483991+0000", 2024-09-20T22:18:01.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001049267 2024-09-20T22:18:01.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.919 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:01.919 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.484017+0000", 2024-09-20T22:18:01.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6057000000000001e-05 2024-09-20T22:18:01.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.486044+0000", 2024-09-20T22:18:01.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020266189999999999 2024-09-20T22:18:01.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.498642+0000", 2024-09-20T22:18:01.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.012598417000000001 2024-09-20T22:18:01.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:01.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.498776+0000", 2024-09-20T22:18:01.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000133764 2024-09-20T22:18:01.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:01.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.498910+0000", 2024-09-20T22:18:01.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000133864 2024-09-20T22:18:01.921 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.921 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:01.921 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.921 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187845 8.a 8:5885b3a8:::benchmark_data_smithi060_111051_object67846:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:18:01.921 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.623815+0000", 2024-09-20T22:18:01.921 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 452.24427679399997, 2024-09-20T22:18:01.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1184221929999998, 2024-09-20T22:18:01.922 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.922 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.922 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:18:01.922 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.922 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:01.922 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:01.922 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187845 2024-09-20T22:18:01.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.922 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.623815+0000", 2024-09-20T22:18:01.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.623815+0000", 2024-09-20T22:18:01.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:01.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.623816+0000", 2024-09-20T22:18:01.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.114e-06 2024-09-20T22:18:01.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:01.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.623820+0000", 2024-09-20T22:18:01.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5360000000000001e-06 2024-09-20T22:18:01.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:01.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.623833+0000", 2024-09-20T22:18:01.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2874000000000001e-05 2024-09-20T22:18:01.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:01.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.623839+0000", 2024-09-20T22:18:01.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1319999999999999e-06 2024-09-20T22:18:01.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.624269+0000", 2024-09-20T22:18:01.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00043047400000000003 2024-09-20T22:18:01.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:01.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.624378+0000", 2024-09-20T22:18:01.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108738 2024-09-20T22:18:01.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:18:01.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.624481+0000", 2024-09-20T22:18:01.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103175 2024-09-20T22:18:01.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:01.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.629342+0000", 2024-09-20T22:18:01.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0048608089999999998 2024-09-20T22:18:01.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.631000+0000", 2024-09-20T22:18:01.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0016580049999999999 2024-09-20T22:18:01.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.640337+0000", 2024-09-20T22:18:01.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0093367550000000004 2024-09-20T22:18:01.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:01.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.651705+0000", 2024-09-20T22:18:01.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.011368038 2024-09-20T22:18:01.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650249+0000", 2024-09-20T22:18:01.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99854470299999998 2024-09-20T22:18:01.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:01.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650268+0000", 2024-09-20T22:18:01.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8981e-05 2024-09-20T22:18:01.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.742151+0000", 2024-09-20T22:18:01.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.09188247 2024-09-20T22:18:01.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:01.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.742237+0000", 2024-09-20T22:18:01.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6389000000000002e-05 2024-09-20T22:18:01.928 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.928 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:01.928 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.928 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187855 8.a 8:5492c8b1:::benchmark_data_smithi060_111051_object67856:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:18:01.928 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.636998+0000", 2024-09-20T22:18:01.928 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 452.231094203, 2024-09-20T22:18:01.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1143666080000001, 2024-09-20T22:18:01.928 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.929 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.929 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:01.929 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.929 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:01.929 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:01.929 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187855 2024-09-20T22:18:01.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.929 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.636998+0000", 2024-09-20T22:18:01.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.636998+0000", 2024-09-20T22:18:01.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:01.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.636999+0000", 2024-09-20T22:18:01.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6080000000000001e-06 2024-09-20T22:18:01.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:01.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.637004+0000", 2024-09-20T22:18:01.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5839999999999996e-06 2024-09-20T22:18:01.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.930 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:01.930 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.637014+0000", 2024-09-20T22:18:01.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0569e-05 2024-09-20T22:18:01.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:01.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.637021+0000", 2024-09-20T22:18:01.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9670000000000004e-06 2024-09-20T22:18:01.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.651901+0000", 2024-09-20T22:18:01.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014879966999999999 2024-09-20T22:18:01.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:01.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.651911+0000", 2024-09-20T22:18:01.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2040000000000003e-06 2024-09-20T22:18:01.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650383+0000", 2024-09-20T22:18:01.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99847266700000004 2024-09-20T22:18:01.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:01.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650393+0000", 2024-09-20T22:18:01.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6299999999999993e-06 2024-09-20T22:18:01.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.742483+0000", 2024-09-20T22:18:01.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0920898370000001 2024-09-20T22:18:01.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:01.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.742615+0000", 2024-09-20T22:18:01.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013252000000000001 2024-09-20T22:18:01.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5", 2024-09-20T22:18:01.933 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.743004+0000", 2024-09-20T22:18:01.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00038869300000000001 2024-09-20T22:18:01.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.933 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751051+0000", 2024-09-20T22:18:01.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0080469289999999995 2024-09-20T22:18:01.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:01.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751066+0000", 2024-09-20T22:18:01.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5245999999999999e-05 2024-09-20T22:18:01.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751111+0000", 2024-09-20T22:18:01.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4713999999999999e-05 2024-09-20T22:18:01.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751258+0000", 2024-09-20T22:18:01.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000147479 2024-09-20T22:18:01.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:01.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751292+0000", 2024-09-20T22:18:01.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3330000000000001e-05 2024-09-20T22:18:01.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:01.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751364+0000", 2024-09-20T22:18:01.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2663999999999994e-05 2024-09-20T22:18:01.935 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.935 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:01.935 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.936 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187859 8.a 8:5b818400:::benchmark_data_smithi060_111051_object67860:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:18:01.936 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.649251+0000", 2024-09-20T22:18:01.936 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 452.218840478, 2024-09-20T22:18:01.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1040842400000002, 2024-09-20T22:18:01.936 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.936 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.936 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:01.936 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.936 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:01.936 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:01.936 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187859 2024-09-20T22:18:01.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.937 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.649251+0000", 2024-09-20T22:18:01.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.649251+0000", 2024-09-20T22:18:01.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.937 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:01.937 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.649253+0000", 2024-09-20T22:18:01.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6589999999999999e-06 2024-09-20T22:18:01.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:01.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.649258+0000", 2024-09-20T22:18:01.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7149999999999997e-06 2024-09-20T22:18:01.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:01.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.649268+0000", 2024-09-20T22:18:01.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0389e-05 2024-09-20T22:18:01.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:01.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.649278+0000", 2024-09-20T22:18:01.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0104e-05 2024-09-20T22:18:01.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.651941+0000", 2024-09-20T22:18:01.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0026624449999999998 2024-09-20T22:18:01.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:01.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.651948+0000", 2024-09-20T22:18:01.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5460000000000003e-06 2024-09-20T22:18:01.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650417+0000", 2024-09-20T22:18:01.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99846875400000001 2024-09-20T22:18:01.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:01.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650425+0000", 2024-09-20T22:18:01.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1300000000000001e-06 2024-09-20T22:18:01.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.746082+0000", 2024-09-20T22:18:01.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:01.940 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:01.941 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:01.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.095656484 2024-09-20T22:18:01.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:01.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.746198+0000", 2024-09-20T22:18:01.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000116168 2024-09-20T22:18:01.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5", 2024-09-20T22:18:01.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.746326+0000", 2024-09-20T22:18:01.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012862299999999999 2024-09-20T22:18:01.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:01.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751072+0000", 2024-09-20T22:18:01.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0047450720000000004 2024-09-20T22:18:01.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751157+0000", 2024-09-20T22:18:01.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5464999999999996e-05 2024-09-20T22:18:01.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751219+0000", 2024-09-20T22:18:01.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2420000000000002e-05 2024-09-20T22:18:01.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753223+0000", 2024-09-20T22:18:01.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020036950000000002 2024-09-20T22:18:01.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:01.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753255+0000", 2024-09-20T22:18:01.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2237999999999999e-05 2024-09-20T22:18:01.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:01.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753336+0000", 2024-09-20T22:18:01.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0333000000000005e-05 2024-09-20T22:18:01.944 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.944 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:01.944 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.944 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187868 8.a 8:5d2db7a2:::benchmark_data_smithi060_111051_object67869:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:18:01.944 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.665623+0000", 2024-09-20T22:18:01.945 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 452.20246893900003, 2024-09-20T22:18:01.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1683483180000001, 2024-09-20T22:18:01.945 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.945 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.945 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:01.945 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.945 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:01.945 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:01.945 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187868 2024-09-20T22:18:01.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.945 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665623+0000", 2024-09-20T22:18:01.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665623+0000", 2024-09-20T22:18:01.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:01.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665624+0000", 2024-09-20T22:18:01.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.850000000000001e-07 2024-09-20T22:18:01.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:01.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665629+0000", 2024-09-20T22:18:01.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8919999999999999e-06 2024-09-20T22:18:01.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:01.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665644+0000", 2024-09-20T22:18:01.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4808e-05 2024-09-20T22:18:01.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:01.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665652+0000", 2024-09-20T22:18:01.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0039999999999999e-06 2024-09-20T22:18:01.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665729+0000", 2024-09-20T22:18:01.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7385000000000005e-05 2024-09-20T22:18:01.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:01.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.665739+0000", 2024-09-20T22:18:01.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6360000000000006e-06 2024-09-20T22:18:01.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650448+0000", 2024-09-20T22:18:01.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98470946199999998 2024-09-20T22:18:01.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:01.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650456+0000", 2024-09-20T22:18:01.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.413e-06 2024-09-20T22:18:01.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.749234+0000", 2024-09-20T22:18:01.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0987788570000001 2024-09-20T22:18:01.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:01.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.749337+0000", 2024-09-20T22:18:01.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000102441 2024-09-20T22:18:01.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5", 2024-09-20T22:18:01.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.749412+0000", 2024-09-20T22:18:01.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4808999999999995e-05 2024-09-20T22:18:01.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:01.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751122+0000", 2024-09-20T22:18:01.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017103159999999999 2024-09-20T22:18:01.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753047+0000", 2024-09-20T22:18:01.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019251750000000001 2024-09-20T22:18:01.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753196+0000", 2024-09-20T22:18:01.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000148441 2024-09-20T22:18:01.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.833608+0000", 2024-09-20T22:18:01.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.080412862000000002 2024-09-20T22:18:01.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:01.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.833835+0000", 2024-09-20T22:18:01.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022606199999999999 2024-09-20T22:18:01.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:01.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.833971+0000", 2024-09-20T22:18:01.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013677000000000001 2024-09-20T22:18:01.952 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.952 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:01.952 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.952 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187898 8.a 8:535037c1:::benchmark_data_smithi060_111051_object67899:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:18:01.952 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.705909+0000", 2024-09-20T22:18:01.952 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 452.16218319000001, 2024-09-20T22:18:01.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1284684729999999, 2024-09-20T22:18:01.952 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.952 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.952 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:01.952 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.953 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:01.953 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:01.953 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187898 2024-09-20T22:18:01.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.953 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.705909+0000", 2024-09-20T22:18:01.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.705909+0000", 2024-09-20T22:18:01.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:01.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.705910+0000", 2024-09-20T22:18:01.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.299e-06 2024-09-20T22:18:01.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:01.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.705915+0000", 2024-09-20T22:18:01.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.429e-06 2024-09-20T22:18:01.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:01.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.705930+0000", 2024-09-20T22:18:01.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4916000000000001e-05 2024-09-20T22:18:01.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:01.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.705938+0000", 2024-09-20T22:18:01.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0290000000000005e-06 2024-09-20T22:18:01.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.706003+0000", 2024-09-20T22:18:01.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4832999999999994e-05 2024-09-20T22:18:01.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:01.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.706014+0000", 2024-09-20T22:18:01.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0961000000000001e-05 2024-09-20T22:18:01.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650482+0000", 2024-09-20T22:18:01.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94446805099999998 2024-09-20T22:18:01.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:01.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.650490+0000", 2024-09-20T22:18:01.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1589999999999993e-06 2024-09-20T22:18:01.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751402+0000", 2024-09-20T22:18:01.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1009116329999999 2024-09-20T22:18:01.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:01.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751502+0000", 2024-09-20T22:18:01.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9617000000000005e-05 2024-09-20T22:18:01.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5", 2024-09-20T22:18:01.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.751574+0000", 2024-09-20T22:18:01.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2517999999999999e-05 2024-09-20T22:18:01.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:01.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753349+0000", 2024-09-20T22:18:01.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001774386 2024-09-20T22:18:01.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753383+0000", 2024-09-20T22:18:01.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4017999999999999e-05 2024-09-20T22:18:01.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.753421+0000", 2024-09-20T22:18:01.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8603000000000003e-05 2024-09-20T22:18:01.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.834077+0000", 2024-09-20T22:18:01.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.080655317000000004 2024-09-20T22:18:01.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:01.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.834257+0000", 2024-09-20T22:18:01.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000180688 2024-09-20T22:18:01.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.959 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:01.959 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.834377+0000", 2024-09-20T22:18:01.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000120016 2024-09-20T22:18:01.959 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.959 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:01.960 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.960 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:125840 9.2 9:4a8ba3ac:::benchmark_data_smithi060_140132_object15032:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:18:01.960 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:30.317157+0000", 2024-09-20T22:18:01.960 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 271.55093470899999, 2024-09-20T22:18:01.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.39984702300000002, 2024-09-20T22:18:01.960 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.960 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.960 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:01.960 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.960 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:18:01.960 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:18:01.960 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 125840 2024-09-20T22:18:01.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.961 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317157+0000", 2024-09-20T22:18:01.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317157+0000", 2024-09-20T22:18:01.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:01.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317159+0000", 2024-09-20T22:18:01.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4929999999999999e-06 2024-09-20T22:18:01.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:01.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317165+0000", 2024-09-20T22:18:01.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8610000000000003e-06 2024-09-20T22:18:01.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:01.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317181+0000", 2024-09-20T22:18:01.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6133999999999998e-05 2024-09-20T22:18:01.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:01.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317188+0000", 2024-09-20T22:18:01.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2840000000000001e-06 2024-09-20T22:18:01.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317259+0000", 2024-09-20T22:18:01.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0816999999999996e-05 2024-09-20T22:18:01.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:01.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317363+0000", 2024-09-20T22:18:01.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103702 2024-09-20T22:18:01.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3", 2024-09-20T22:18:01.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317465+0000", 2024-09-20T22:18:01.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010202 2024-09-20T22:18:01.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:01.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.320561+0000", 2024-09-20T22:18:01.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0030965540000000001 2024-09-20T22:18:01.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.321130+0000", 2024-09-20T22:18:01.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00056882299999999998 2024-09-20T22:18:01.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.716883+0000", 2024-09-20T22:18:01.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.39575325 2024-09-20T22:18:01.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:01.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.716956+0000", 2024-09-20T22:18:01.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2352000000000001e-05 2024-09-20T22:18:01.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:01.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.717004+0000", 2024-09-20T22:18:01.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8733000000000002e-05 2024-09-20T22:18:01.965 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.965 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:01.965 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.966 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:125859 9.0 9:05dacd16:::benchmark_data_smithi060_140132_object15051:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:18:01.966 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:30.331372+0000", 2024-09-20T22:18:01.966 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 271.53671964599999, 2024-09-20T22:18:01.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.38946761499999999, 2024-09-20T22:18:01.966 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.966 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.966 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:01.966 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.966 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:18:01.966 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:18:01.966 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 125859 2024-09-20T22:18:01.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.966 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331372+0000", 2024-09-20T22:18:01.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.967 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.967 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331372+0000", 2024-09-20T22:18:01.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:01.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331374+0000", 2024-09-20T22:18:01.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5290000000000001e-06 2024-09-20T22:18:01.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:01.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331379+0000", 2024-09-20T22:18:01.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.502e-06 2024-09-20T22:18:01.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:01.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331397+0000", 2024-09-20T22:18:01.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7249e-05 2024-09-20T22:18:01.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:01.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331406+0000", 2024-09-20T22:18:01.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0659999999999993e-06 2024-09-20T22:18:01.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331474+0000", 2024-09-20T22:18:01.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8396e-05 2024-09-20T22:18:01.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:01.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331609+0000", 2024-09-20T22:18:01.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013487800000000001 2024-09-20T22:18:01.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,6", 2024-09-20T22:18:01.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331730+0000", 2024-09-20T22:18:01.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121408 2024-09-20T22:18:01.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:01.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.334028+0000", 2024-09-20T22:18:01.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022976429999999998 2024-09-20T22:18:01.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.334978+0000", 2024-09-20T22:18:01.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00095049999999999996 2024-09-20T22:18:01.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720566+0000", 2024-09-20T22:18:01.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.38558782699999999 2024-09-20T22:18:01.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:01.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720619+0000", 2024-09-20T22:18:01.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2404000000000002e-05 2024-09-20T22:18:01.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:01.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720840+0000", 2024-09-20T22:18:01.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000221213 2024-09-20T22:18:01.973 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.973 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:01.973 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.973 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:125862 9.2 9:4de341df:::benchmark_data_smithi060_140132_object15054:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:18:01.973 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:30.336602+0000", 2024-09-20T22:18:01.973 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 271.53149009700002, 2024-09-20T22:18:01.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.38056002500000002, 2024-09-20T22:18:01.974 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.974 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.974 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:01.974 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.974 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:18:01.974 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:18:01.974 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 125862 2024-09-20T22:18:01.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.974 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336602+0000", 2024-09-20T22:18:01.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336602+0000", 2024-09-20T22:18:01.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:01.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336603+0000", 2024-09-20T22:18:01.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3799999999999996e-07 2024-09-20T22:18:01.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:01.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336608+0000", 2024-09-20T22:18:01.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9350000000000002e-06 2024-09-20T22:18:01.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:01.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336624+0000", 2024-09-20T22:18:01.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5849e-05 2024-09-20T22:18:01.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:01.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336645+0000", 2024-09-20T22:18:01.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1067999999999999e-05 2024-09-20T22:18:01.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336704+0000", 2024-09-20T22:18:01.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9525999999999999e-05 2024-09-20T22:18:01.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:01.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336824+0000", 2024-09-20T22:18:01.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011957 2024-09-20T22:18:01.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3", 2024-09-20T22:18:01.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336942+0000", 2024-09-20T22:18:01.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117819 2024-09-20T22:18:01.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.339146+0000", 2024-09-20T22:18:01.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022047680000000002 2024-09-20T22:18:01.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:01.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.339234+0000", 2024-09-20T22:18:01.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7968999999999998e-05 2024-09-20T22:18:01.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.717065+0000", 2024-09-20T22:18:01.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.37783098799999998 2024-09-20T22:18:01.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:01.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.717120+0000", 2024-09-20T22:18:01.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5133999999999998e-05 2024-09-20T22:18:01.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:01.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.717162+0000", 2024-09-20T22:18:01.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1461000000000002e-05 2024-09-20T22:18:01.979 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.979 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:01.979 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.979 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:125869 9.0 9:0c9f6e22:::benchmark_data_smithi060_140132_object15061:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:18:01.979 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:30.340895+0000", 2024-09-20T22:18:01.979 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 271.52719675899999, 2024-09-20T22:18:01.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.38006649199999998, 2024-09-20T22:18:01.980 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.980 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.980 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:01.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:18:01.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:18:01.980 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 125869 2024-09-20T22:18:01.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.980 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.980 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.980 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340895+0000", 2024-09-20T22:18:01.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340895+0000", 2024-09-20T22:18:01.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:01.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340896+0000", 2024-09-20T22:18:01.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.31e-06 2024-09-20T22:18:01.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:01.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340902+0000", 2024-09-20T22:18:01.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9370000000000001e-06 2024-09-20T22:18:01.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:01.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340917+0000", 2024-09-20T22:18:01.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4181e-05 2024-09-20T22:18:01.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:01.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340924+0000", 2024-09-20T22:18:01.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3499999999999999e-06 2024-09-20T22:18:01.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340992+0000", 2024-09-20T22:18:01.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7887999999999998e-05 2024-09-20T22:18:01.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:01.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.341093+0000", 2024-09-20T22:18:01.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000101074 2024-09-20T22:18:01.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,6", 2024-09-20T22:18:01.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.341159+0000", 2024-09-20T22:18:01.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6167999999999994e-05 2024-09-20T22:18:01.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:01.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343748+0000", 2024-09-20T22:18:01.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025892060000000001 2024-09-20T22:18:01.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.348961+0000", 2024-09-20T22:18:01.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005213082 2024-09-20T22:18:01.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720878+0000", 2024-09-20T22:18:01.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.37191645499999998 2024-09-20T22:18:01.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:01.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720928+0000", 2024-09-20T22:18:01.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9888000000000001e-05 2024-09-20T22:18:01.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:01.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720962+0000", 2024-09-20T22:18:01.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3952999999999998e-05 2024-09-20T22:18:01.985 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.985 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:01.985 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.985 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:125872 9.0 9:0ae55481:::benchmark_data_smithi060_140132_object15064:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:18:01.985 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:30.343972+0000", 2024-09-20T22:18:01.985 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 271.52411996400002, 2024-09-20T22:18:01.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.37716829000000002, 2024-09-20T22:18:01.986 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.986 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.986 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:01.986 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.986 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:18:01.986 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:18:01.986 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 125872 2024-09-20T22:18:01.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.986 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343972+0000", 2024-09-20T22:18:01.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343972+0000", 2024-09-20T22:18:01.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:01.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343973+0000", 2024-09-20T22:18:01.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2640000000000001e-06 2024-09-20T22:18:01.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.987 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:01.987 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343978+0000", 2024-09-20T22:18:01.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1569999999999999e-06 2024-09-20T22:18:01.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:01.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343991+0000", 2024-09-20T22:18:01.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2313000000000001e-05 2024-09-20T22:18:01.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:01.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343998+0000", 2024-09-20T22:18:01.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7379999999999994e-06 2024-09-20T22:18:01.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344068+0000", 2024-09-20T22:18:01.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9962000000000005e-05 2024-09-20T22:18:01.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:01.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344184+0000", 2024-09-20T22:18:01.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000116063 2024-09-20T22:18:01.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,6", 2024-09-20T22:18:01.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344284+0000", 2024-09-20T22:18:01.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9779999999999997e-05 2024-09-20T22:18:01.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:01.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.348919+0000", 2024-09-20T22:18:01.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0046344170000000001 2024-09-20T22:18:01.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.349022+0000", 2024-09-20T22:18:01.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103521 2024-09-20T22:18:01.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720993+0000", 2024-09-20T22:18:01.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.37197118600000001 2024-09-20T22:18:01.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:01.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.721105+0000", 2024-09-20T22:18:01.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111161 2024-09-20T22:18:01.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.991 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:01.991 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.721140+0000", 2024-09-20T22:18:01.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5728000000000001e-05 2024-09-20T22:18:01.991 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.991 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:01.991 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.991 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:125874 9.0 9:01e41a72:::benchmark_data_smithi060_140132_object15066:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:18:01.991 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:30.344246+0000", 2024-09-20T22:18:01.991 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 271.52384580299997, 2024-09-20T22:18:01.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.37701091199999998, 2024-09-20T22:18:01.992 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.992 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.992 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:01.992 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.992 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:18:01.992 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:18:01.992 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 125874 2024-09-20T22:18:01.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.992 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344246+0000", 2024-09-20T22:18:01.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344246+0000", 2024-09-20T22:18:01.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:01.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344247+0000", 2024-09-20T22:18:01.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6099999999999999e-07 2024-09-20T22:18:01.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:01.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344252+0000", 2024-09-20T22:18:01.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5129999999999998e-06 2024-09-20T22:18:01.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:01.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344263+0000", 2024-09-20T22:18:01.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1744e-05 2024-09-20T22:18:01.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:01.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344271+0000", 2024-09-20T22:18:01.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8709999999999997e-06 2024-09-20T22:18:01.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:01.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.346350+0000", 2024-09-20T22:18:01.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002079072 2024-09-20T22:18:01.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:01.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.346467+0000", 2024-09-20T22:18:01.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117207 2024-09-20T22:18:01.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.995 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,6", 2024-09-20T22:18:01.995 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.346581+0000", 2024-09-20T22:18:01.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113241 2024-09-20T22:18:01.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:01.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.348991+0000", 2024-09-20T22:18:01.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024100020000000001 2024-09-20T22:18:01.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.349689+0000", 2024-09-20T22:18:01.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00069810599999999999 2024-09-20T22:18:01.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:01.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.721177+0000", 2024-09-20T22:18:01.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.37148772800000002 2024-09-20T22:18:01.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:01.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.721218+0000", 2024-09-20T22:18:01.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1366999999999999e-05 2024-09-20T22:18:01.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:01.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.721257+0000", 2024-09-20T22:18:01.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9100000000000002e-05 2024-09-20T22:18:01.997 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.997 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:01.998 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:01.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.998 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:184794 10.2 10:4057a2e4:::benchmark_data_smithi060_167766_object60002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e428)", 2024-09-20T22:18:01.998 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:17:39.192306+0000", 2024-09-20T22:18:01.998 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 22.675785568999999, 2024-09-20T22:18:01.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.17511643700000001, 2024-09-20T22:18:01.998 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:01.998 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:01.998 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:01.998 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:01.998 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:01.998 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:01.999 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 184794 2024-09-20T22:18:01.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.999 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:01.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:01.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.192306+0000", 2024-09-20T22:18:01.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:01.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.192306+0000", 2024-09-20T22:18:01.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:01.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:01.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:01.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:02.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.192308+0000", 2024-09-20T22:18:02.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.688e-06 2024-09-20T22:18:02.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:02.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.192315+0000", 2024-09-20T22:18:02.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9920000000000001e-06 2024-09-20T22:18:02.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:02.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.192328+0000", 2024-09-20T22:18:02.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2490000000000001e-05 2024-09-20T22:18:02.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:02.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.192335+0000", 2024-09-20T22:18:02.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2740000000000002e-06 2024-09-20T22:18:02.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:02.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.192394+0000", 2024-09-20T22:18:02.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9206000000000002e-05 2024-09-20T22:18:02.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:02.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.192511+0000", 2024-09-20T22:18:02.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000116587 2024-09-20T22:18:02.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:18:02.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.192620+0000", 2024-09-20T22:18:02.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109103 2024-09-20T22:18:02.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:02.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.195232+0000", 2024-09-20T22:18:02.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0026118309999999998 2024-09-20T22:18:02.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.002 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:02.002 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.197584+0000", 2024-09-20T22:18:02.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023521319999999998 2024-09-20T22:18:02.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:02.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.367340+0000", 2024-09-20T22:18:02.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.16975664100000001 2024-09-20T22:18:02.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:02.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.367382+0000", 2024-09-20T22:18:02.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2079000000000001e-05 2024-09-20T22:18:02.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:02.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.367423+0000", 2024-09-20T22:18:02.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0414000000000003e-05 2024-09-20T22:18:02.004 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:02.004 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:02.004 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:02.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.004 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:185110 10.2 10:466aa4b8:::benchmark_data_smithi060_167766_object60318:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e428)", 2024-09-20T22:18:02.004 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:17:39.596327+0000", 2024-09-20T22:18:02.004 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 22.271764682000001, 2024-09-20T22:18:02.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.103334831, 2024-09-20T22:18:02.004 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:02.004 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:02.004 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:02.004 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:02.004 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:02.004 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:02.005 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 185110 2024-09-20T22:18:02.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.005 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:02.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:02.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.596327+0000", 2024-09-20T22:18:02.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:02.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:02.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.596327+0000", 2024-09-20T22:18:02.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:02.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:02.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.596329+0000", 2024-09-20T22:18:02.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.584e-06 2024-09-20T22:18:02.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:02.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.596336+0000", 2024-09-20T22:18:02.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4850000000000003e-06 2024-09-20T22:18:02.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:02.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.596353+0000", 2024-09-20T22:18:02.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6925000000000001e-05 2024-09-20T22:18:02.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:02.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.596361+0000", 2024-09-20T22:18:02.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3100000000000003e-06 2024-09-20T22:18:02.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:02.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.596431+0000", 2024-09-20T22:18:02.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0593000000000007e-05 2024-09-20T22:18:02.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:02.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.596574+0000", 2024-09-20T22:18:02.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014275799999999999 2024-09-20T22:18:02.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:18:02.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.596912+0000", 2024-09-20T22:18:02.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000337695 2024-09-20T22:18:02.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:02.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.600016+0000", 2024-09-20T22:18:02.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0031044390000000001 2024-09-20T22:18:02.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:02.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.600604+0000", 2024-09-20T22:18:02.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000587946 2024-09-20T22:18:02.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:02.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.699576+0000", 2024-09-20T22:18:02.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.098972215000000002 2024-09-20T22:18:02.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:02.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.699618+0000", 2024-09-20T22:18:02.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1529999999999997e-05 2024-09-20T22:18:02.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:02.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:02.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:02.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:39.699662+0000", 2024-09-20T22:18:02.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4351000000000003e-05 2024-09-20T22:18:02.009 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:02.010 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:02.010 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:02.010 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:02.010 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:02.010 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:18:02.041 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:02.078 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:02.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:02.079 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:02.180 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:02.211 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:02.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:02.212 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:02.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:02.342 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:02.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:02.343 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:02.443 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:02.472 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:02.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:02.473 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:02.574 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:02.607 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:02.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:02.608 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:02.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:02.730 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:02.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:02.731 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:02.813 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:18:02.832 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:02.864 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:02.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:02.865 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:02.934 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:18:02.965 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:02.973 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:02 smithi060 ceph-mon[29694]: osdmap e438: 8 total, 8 up, 8 in 2024-09-20T22:18:02.973 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:02 smithi060 ceph-mon[29694]: 6.1b scrub starts 2024-09-20T22:18:02.973 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:02 smithi060 ceph-mon[29694]: 6.1b scrub ok 2024-09-20T22:18:02.973 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:02 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3043300988' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:18:02.974 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:02 smithi060 ceph-mon[40290]: osdmap e438: 8 total, 8 up, 8 in 2024-09-20T22:18:02.974 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:02 smithi060 ceph-mon[40290]: 6.1b scrub starts 2024-09-20T22:18:02.974 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:02 smithi060 ceph-mon[40290]: 6.1b scrub ok 2024-09-20T22:18:02.974 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:02 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3043300988' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:18:02.994 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:02.994 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:02.995 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:03.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:18:02 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:18:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:02 smithi067 ceph-mon[33511]: osdmap e438: 8 total, 8 up, 8 in 2024-09-20T22:18:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:02 smithi067 ceph-mon[33511]: 6.1b scrub starts 2024-09-20T22:18:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:02 smithi067 ceph-mon[33511]: 6.1b scrub ok 2024-09-20T22:18:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:02 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3043300988' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:18:03.096 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:03.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:03.132 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:03.132 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:03.233 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:03.258 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:03.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:03.259 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:03.360 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:03.399 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:03.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:03.400 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:03.500 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:03.537 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:03.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:03.538 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:03.639 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:03.674 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:03.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:03.675 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:03.776 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:03.815 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:03.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:03.817 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:03.917 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:03.954 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:03.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:03.955 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:03 smithi067 ceph-mon[33511]: pgmap v809: 178 pgs: 178 active+clean; 2.4 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 1.10k op/s 2024-09-20T22:18:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:03 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:18:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:03 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:18:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:03 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3043300988' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:18:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:03 smithi067 ceph-mon[33511]: osdmap e439: 8 total, 8 up, 8 in 2024-09-20T22:18:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:03 smithi067 ceph-mon[33511]: 6.17 deep-scrub starts 2024-09-20T22:18:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:03 smithi067 ceph-mon[33511]: 6.17 deep-scrub ok 2024-09-20T22:18:04.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:03 smithi060 ceph-mon[29694]: pgmap v809: 178 pgs: 178 active+clean; 2.4 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 1.10k op/s 2024-09-20T22:18:04.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:03 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:18:04.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:03 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:18:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:03 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3043300988' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:18:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:03 smithi060 ceph-mon[29694]: osdmap e439: 8 total, 8 up, 8 in 2024-09-20T22:18:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:03 smithi060 ceph-mon[29694]: 6.17 deep-scrub starts 2024-09-20T22:18:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:03 smithi060 ceph-mon[29694]: 6.17 deep-scrub ok 2024-09-20T22:18:04.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:03 smithi060 ceph-mon[40290]: pgmap v809: 178 pgs: 178 active+clean; 2.4 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 1.10k op/s 2024-09-20T22:18:04.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:03 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:18:04.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:03 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:18:04.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:03 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3043300988' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:18:04.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:03 smithi060 ceph-mon[40290]: osdmap e439: 8 total, 8 up, 8 in 2024-09-20T22:18:04.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:03 smithi060 ceph-mon[40290]: 6.17 deep-scrub starts 2024-09-20T22:18:04.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:03 smithi060 ceph-mon[40290]: 6.17 deep-scrub ok 2024-09-20T22:18:04.056 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:04.077 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:04.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:04.078 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:04.179 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:04.204 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:04.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:04.204 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:04.305 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:04.329 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:04.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:04.330 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:04.430 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:04.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:04.457 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:04.458 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:04.558 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:04.586 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:04.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:04.587 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:04.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:04.723 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:04.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:04.724 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:04.825 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:04.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:04.859 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:04.859 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:04.868 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:04 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:18:04.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:04 smithi060 ceph-mon[29694]: 3.1c scrub starts 2024-09-20T22:18:04.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:04 smithi060 ceph-mon[29694]: 3.1c scrub ok 2024-09-20T22:18:04.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:04 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:18:04.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:04 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:18:04.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:04 smithi060 ceph-mon[29694]: 6.14 scrub starts 2024-09-20T22:18:04.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:04 smithi060 ceph-mon[29694]: 6.14 scrub ok 2024-09-20T22:18:04.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:04 smithi060 ceph-mon[29694]: 6.2 scrub starts 2024-09-20T22:18:04.869 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:04 smithi060 ceph-mon[29694]: 6.2 scrub ok 2024-09-20T22:18:04.869 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:04 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:18:04.869 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:04 smithi060 ceph-mon[40290]: 3.1c scrub starts 2024-09-20T22:18:04.869 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:04 smithi060 ceph-mon[40290]: 3.1c scrub ok 2024-09-20T22:18:04.869 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:04 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:18:04.870 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:04 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:18:04.870 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:04 smithi060 ceph-mon[40290]: 6.14 scrub starts 2024-09-20T22:18:04.870 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:04 smithi060 ceph-mon[40290]: 6.14 scrub ok 2024-09-20T22:18:04.870 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:04 smithi060 ceph-mon[40290]: 6.2 scrub starts 2024-09-20T22:18:04.870 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:04 smithi060 ceph-mon[40290]: 6.2 scrub ok 2024-09-20T22:18:04.959 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:04.998 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:04.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:04.999 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:04 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:18:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:04 smithi067 ceph-mon[33511]: 3.1c scrub starts 2024-09-20T22:18:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:04 smithi067 ceph-mon[33511]: 3.1c scrub ok 2024-09-20T22:18:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:04 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:18:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:04 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:18:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:04 smithi067 ceph-mon[33511]: 6.14 scrub starts 2024-09-20T22:18:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:04 smithi067 ceph-mon[33511]: 6.14 scrub ok 2024-09-20T22:18:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:04 smithi067 ceph-mon[33511]: 6.2 scrub starts 2024-09-20T22:18:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:04 smithi067 ceph-mon[33511]: 6.2 scrub ok 2024-09-20T22:18:05.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:05.138 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:05.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:05.139 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:05.240 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:05.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:05.272 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:05.272 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:05.285 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:18:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:18:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:18:05.373 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:05.408 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:05.409 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:05.409 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:05.510 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:05.545 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:05.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:05.546 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:05.647 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:05.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:05.681 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:05.782 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:05.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:05 smithi060 ceph-mon[29694]: pgmap v811: 178 pgs: 178 active+clean; 2.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 1.26k op/s 2024-09-20T22:18:05.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:05 smithi060 ceph-mon[29694]: 6.d scrub starts 2024-09-20T22:18:05.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:05 smithi060 ceph-mon[29694]: 6.d scrub ok 2024-09-20T22:18:05.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:05 smithi060 ceph-mon[29694]: osdmap e440: 8 total, 8 up, 8 in 2024-09-20T22:18:05.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:05 smithi060 ceph-mon[29694]: 5.4 scrub starts 2024-09-20T22:18:05.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:05 smithi060 ceph-mon[29694]: 5.4 scrub ok 2024-09-20T22:18:05.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:05 smithi060 ceph-mon[29694]: 4.c deep-scrub starts 2024-09-20T22:18:05.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:05 smithi060 ceph-mon[29694]: 4.c deep-scrub ok 2024-09-20T22:18:05.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:05 smithi060 ceph-mon[40290]: pgmap v811: 178 pgs: 178 active+clean; 2.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 1.26k op/s 2024-09-20T22:18:05.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:05 smithi060 ceph-mon[40290]: 6.d scrub starts 2024-09-20T22:18:05.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:05 smithi060 ceph-mon[40290]: 6.d scrub ok 2024-09-20T22:18:05.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:05 smithi060 ceph-mon[40290]: osdmap e440: 8 total, 8 up, 8 in 2024-09-20T22:18:05.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:05 smithi060 ceph-mon[40290]: 5.4 scrub starts 2024-09-20T22:18:05.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:05 smithi060 ceph-mon[40290]: 5.4 scrub ok 2024-09-20T22:18:05.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:05 smithi060 ceph-mon[40290]: 4.c deep-scrub starts 2024-09-20T22:18:05.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:05 smithi060 ceph-mon[40290]: 4.c deep-scrub ok 2024-09-20T22:18:05.822 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:05.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:05.823 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:05.880 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T22:18:05.923 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:05.967 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:05.968 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:05.968 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:05 smithi067 ceph-mon[33511]: pgmap v811: 178 pgs: 178 active+clean; 2.2 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 0 B/s wr, 1.26k op/s 2024-09-20T22:18:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:05 smithi067 ceph-mon[33511]: 6.d scrub starts 2024-09-20T22:18:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:05 smithi067 ceph-mon[33511]: 6.d scrub ok 2024-09-20T22:18:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:05 smithi067 ceph-mon[33511]: osdmap e440: 8 total, 8 up, 8 in 2024-09-20T22:18:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:05 smithi067 ceph-mon[33511]: 5.4 scrub starts 2024-09-20T22:18:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:05 smithi067 ceph-mon[33511]: 5.4 scrub ok 2024-09-20T22:18:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:05 smithi067 ceph-mon[33511]: 4.c deep-scrub starts 2024-09-20T22:18:06.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:05 smithi067 ceph-mon[33511]: 4.c deep-scrub ok 2024-09-20T22:18:06.069 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:06.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:06.102 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:06.102 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:06.203 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:06.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:06.251 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:06.252 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:06.352 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:06.388 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:06.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:06.389 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:06.490 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:06.524 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:06.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:06.525 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:06.626 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:06.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:06.654 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:06.654 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:06.755 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:06.792 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:06.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:06.793 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:06.893 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:06.913 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:06 smithi060 ceph-mon[29694]: 4.17 scrub starts 2024-09-20T22:18:06.913 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:06 smithi060 ceph-mon[29694]: 4.17 scrub ok 2024-09-20T22:18:06.913 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:18:06.913 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:18:06.913 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:06 smithi060 ceph-mon[29694]: pgmap v813: 178 pgs: 178 active+clean; 2.2 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 0 B/s wr, 1.16k op/s 2024-09-20T22:18:06.913 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:06 smithi060 ceph-mon[29694]: osdmap e441: 8 total, 8 up, 8 in 2024-09-20T22:18:06.913 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:06 smithi060 ceph-mon[40290]: 4.17 scrub starts 2024-09-20T22:18:06.914 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:06 smithi060 ceph-mon[40290]: 4.17 scrub ok 2024-09-20T22:18:06.914 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:18:06.914 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:18:06.914 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:06 smithi060 ceph-mon[40290]: pgmap v813: 178 pgs: 178 active+clean; 2.2 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 0 B/s wr, 1.16k op/s 2024-09-20T22:18:06.914 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:06 smithi060 ceph-mon[40290]: osdmap e441: 8 total, 8 up, 8 in 2024-09-20T22:18:06.923 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:06.924 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:06.924 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:06 smithi067 ceph-mon[33511]: 4.17 scrub starts 2024-09-20T22:18:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:06 smithi067 ceph-mon[33511]: 4.17 scrub ok 2024-09-20T22:18:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:18:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:18:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:06 smithi067 ceph-mon[33511]: pgmap v813: 178 pgs: 178 active+clean; 2.2 GiB data, 12 GiB used, 704 GiB / 715 GiB avail; 0 B/s wr, 1.16k op/s 2024-09-20T22:18:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:06 smithi067 ceph-mon[33511]: osdmap e441: 8 total, 8 up, 8 in 2024-09-20T22:18:07.025 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:07.054 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:07.055 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:07.055 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:07.081 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:07.120 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:07.157 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:07.200 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:07.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:07.201 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:07.302 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:07.326 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:18:07.331 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:18:07.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:07.340 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:07.340 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:07.441 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:07.475 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:07.476 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:07.476 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:07.577 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:07.620 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:07.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:07.621 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:07.721 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:07.761 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:07.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:07.762 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:07.863 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:07.899 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:07.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:07.900 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:07.906 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:18:07.906 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:18:07.906 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.907 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:219452 10.5 e441/370 10:a4169d4b:::benchmark_data_smithi060_167766_object94660:head v 441'13806, mlcod=441'13804)", 2024-09-20T22:18:07.907 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:07.886320+0000", 2024-09-20T22:18:07.907 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016562554, 2024-09-20T22:18:07.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.016591113000000001, 2024-09-20T22:18:07.907 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:07.907 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:07.907 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:18:07.907 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:07.907 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.907 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:07.907 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.886320+0000", 2024-09-20T22:18:07.907 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.907 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:07.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.886320+0000", 2024-09-20T22:18:07.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:07.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.886320+0000", 2024-09-20T22:18:07.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.61e-07 2024-09-20T22:18:07.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.908 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.908 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:07.908 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.886327+0000", 2024-09-20T22:18:07.908 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0099999999999998e-06 2024-09-20T22:18:07.908 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:07.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.886336+0000", 2024-09-20T22:18:07.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9830000000000002e-06 2024-09-20T22:18:07.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:07.909 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.886343+0000", 2024-09-20T22:18:07.909 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9160000000000004e-06 2024-09-20T22:18:07.909 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.909 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.909 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:07.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.886425+0000", 2024-09-20T22:18:07.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1341000000000006e-05 2024-09-20T22:18:07.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.910 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:07.910 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.886504+0000", 2024-09-20T22:18:07.910 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8831999999999993e-05 2024-09-20T22:18:07.910 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.910 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:07.910 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.910 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.910 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.910 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:219453 10.8 10:147fe67f:::benchmark_data_smithi060_167766_object94661:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e441)", 2024-09-20T22:18:07.911 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:07.886563+0000", 2024-09-20T22:18:07.911 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.016319485000000002, 2024-09-20T22:18:07.911 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.016421716999999999, 2024-09-20T22:18:07.911 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:07.911 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:07.911 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:18:07.911 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:07.911 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:07.911 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:07.911 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 219453 2024-09-20T22:18:07.911 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.911 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:07.911 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.911 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:07.911 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.886563+0000", 2024-09-20T22:18:07.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:07.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.886563+0000", 2024-09-20T22:18:07.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:07.912 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.886564+0000", 2024-09-20T22:18:07.912 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.581e-06 2024-09-20T22:18:07.912 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.912 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.912 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:07.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.886570+0000", 2024-09-20T22:18:07.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2800000000000003e-06 2024-09-20T22:18:07.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:07.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.886589+0000", 2024-09-20T22:18:07.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9415999999999998e-05 2024-09-20T22:18:07.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:07.913 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.886597+0000", 2024-09-20T22:18:07.913 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8509999999999999e-06 2024-09-20T22:18:07.913 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.913 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.913 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:07.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.886736+0000", 2024-09-20T22:18:07.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000139445 2024-09-20T22:18:07.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:07.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.886857+0000", 2024-09-20T22:18:07.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000120658 2024-09-20T22:18:07.914 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.914 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.914 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:18:07.914 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.886982+0000", 2024-09-20T22:18:07.914 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000125096 2024-09-20T22:18:07.914 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.914 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:07.914 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.915 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.915 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:219460 10.8 10:17390374:::benchmark_data_smithi060_167766_object94668:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e441)", 2024-09-20T22:18:07.915 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:07.892906+0000", 2024-09-20T22:18:07.915 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.009976469, 2024-09-20T22:18:07.915 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010173659, 2024-09-20T22:18:07.915 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:07.915 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:07.915 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:18:07.915 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:07.915 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:07.915 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:07.915 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 219460 2024-09-20T22:18:07.915 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.916 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:07.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:07.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.892906+0000", 2024-09-20T22:18:07.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:07.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.892906+0000", 2024-09-20T22:18:07.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.916 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.916 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.916 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:07.916 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.892907+0000", 2024-09-20T22:18:07.916 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2449999999999999e-06 2024-09-20T22:18:07.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:07.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.892912+0000", 2024-09-20T22:18:07.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3789999999999997e-06 2024-09-20T22:18:07.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:07.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.892926+0000", 2024-09-20T22:18:07.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.358e-05 2024-09-20T22:18:07.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.917 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.917 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:07.917 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.892933+0000", 2024-09-20T22:18:07.917 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3130000000000001e-06 2024-09-20T22:18:07.917 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:07.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.896345+0000", 2024-09-20T22:18:07.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003412062 2024-09-20T22:18:07.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:07.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.896523+0000", 2024-09-20T22:18:07.918 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017796000000000001 2024-09-20T22:18:07.918 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.918 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.918 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:18:07.918 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.896643+0000", 2024-09-20T22:18:07.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119428 2024-09-20T22:18:07.919 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.919 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:07.919 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.919 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.919 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.919 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:219459 10.d e441/370 10:bc98eb92:::benchmark_data_smithi060_167766_object94667:head v 441'13546, mlcod=441'13543)", 2024-09-20T22:18:07.919 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:07.893076+0000", 2024-09-20T22:18:07.919 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0098065019999999999, 2024-09-20T22:18:07.919 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010089994, 2024-09-20T22:18:07.919 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:07.919 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:07.919 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:18:07.919 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:07.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:07.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.893076+0000", 2024-09-20T22:18:07.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:07.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.893076+0000", 2024-09-20T22:18:07.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.920 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.920 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.920 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:07.920 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.893077+0000", 2024-09-20T22:18:07.920 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.47e-07 2024-09-20T22:18:07.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:07.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.893099+0000", 2024-09-20T22:18:07.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1895e-05 2024-09-20T22:18:07.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:07.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.893108+0000", 2024-09-20T22:18:07.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3689999999999996e-06 2024-09-20T22:18:07.921 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.921 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.921 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:07.921 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.893115+0000", 2024-09-20T22:18:07.921 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5699999999999998e-06 2024-09-20T22:18:07.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:07.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.893203+0000", 2024-09-20T22:18:07.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8745999999999994e-05 2024-09-20T22:18:07.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.922 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:07.922 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.893223+0000", 2024-09-20T22:18:07.922 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9851e-05 2024-09-20T22:18:07.922 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.922 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:07.922 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.922 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.922 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.923 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop_reply(client.16005.0:219453 10.8 e441/370)", 2024-09-20T22:18:07.923 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:07.893251+0000", 2024-09-20T22:18:07.923 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0096309340000000007, 2024-09-20T22:18:07.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0099735629999999995, 2024-09-20T22:18:07.923 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:07.923 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:07.923 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:18:07.923 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:07.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.923 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:07.923 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.893251+0000", 2024-09-20T22:18:07.923 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.923 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.923 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:07.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.893251+0000", 2024-09-20T22:18:07.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:07.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.893252+0000", 2024-09-20T22:18:07.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4700000000000001e-07 2024-09-20T22:18:07.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.924 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:07.924 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.893256+0000", 2024-09-20T22:18:07.924 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5190000000000001e-06 2024-09-20T22:18:07.924 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.924 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:07.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.893264+0000", 2024-09-20T22:18:07.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0649999999999999e-06 2024-09-20T22:18:07.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.925 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:07.925 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.893270+0000", 2024-09-20T22:18:07.925 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9510000000000003e-06 2024-09-20T22:18:07.925 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.925 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:07.925 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.925 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.925 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.925 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:219461 10.5 e441/370 10:a25c1e4b:::benchmark_data_smithi060_167766_object94669:head v 441'13807, mlcod=441'13804)", 2024-09-20T22:18:07.925 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:07.894029+0000", 2024-09-20T22:18:07.926 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0088531470000000004, 2024-09-20T22:18:07.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0092444569999999993, 2024-09-20T22:18:07.926 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:07.926 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:07.926 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:18:07.926 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:07.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:07.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.894029+0000", 2024-09-20T22:18:07.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.926 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.926 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.926 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:07.926 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.894029+0000", 2024-09-20T22:18:07.926 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:07.927 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.894030+0000", 2024-09-20T22:18:07.927 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5000000000000003e-07 2024-09-20T22:18:07.927 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.927 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.927 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:07.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.894074+0000", 2024-09-20T22:18:07.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4338999999999997e-05 2024-09-20T22:18:07.928 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.928 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.928 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:07.928 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.894108+0000", 2024-09-20T22:18:07.928 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3957e-05 2024-09-20T22:18:07.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:07.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.894114+0000", 2024-09-20T22:18:07.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3799999999999999e-06 2024-09-20T22:18:07.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:07.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.899758+0000", 2024-09-20T22:18:07.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0056437889999999998 2024-09-20T22:18:07.929 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.929 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.929 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:07.929 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.899778+0000", 2024-09-20T22:18:07.929 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9868e-05 2024-09-20T22:18:07.930 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.930 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:07.930 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.930 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.930 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop_reply(client.16005.0:219453 10.8 e441/370)", 2024-09-20T22:18:07.930 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:07.897366+0000", 2024-09-20T22:18:07.930 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0055166080000000001, 2024-09-20T22:18:07.930 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0059637750000000002, 2024-09-20T22:18:07.930 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:07.930 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:07.930 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:18:07.930 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:07.930 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:07.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.897366+0000", 2024-09-20T22:18:07.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:07.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.897366+0000", 2024-09-20T22:18:07.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.931 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.931 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.931 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:07.931 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.897366+0000", 2024-09-20T22:18:07.931 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0500000000000003e-07 2024-09-20T22:18:07.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:07.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.897370+0000", 2024-09-20T22:18:07.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4189999999999998e-06 2024-09-20T22:18:07.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:07.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.897381+0000", 2024-09-20T22:18:07.932 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.117e-05 2024-09-20T22:18:07.932 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.932 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.932 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:07.932 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.897388+0000", 2024-09-20T22:18:07.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3259999999999998e-06 2024-09-20T22:18:07.933 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.933 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:07.933 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.933 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.933 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.933 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:219468 10.5 e441/370 10:a407ceba:::benchmark_data_smithi060_167766_object94676:head v 441'13808, mlcod=441'13804)", 2024-09-20T22:18:07.933 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:07.899610+0000", 2024-09-20T22:18:07.933 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0032727030000000001, 2024-09-20T22:18:07.933 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003769821, 2024-09-20T22:18:07.933 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:07.933 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:07.933 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:18:07.933 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:07.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:07.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.899610+0000", 2024-09-20T22:18:07.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:07.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.899610+0000", 2024-09-20T22:18:07.934 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.934 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.934 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.934 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:07.934 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.899610+0000", 2024-09-20T22:18:07.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6300000000000005e-07 2024-09-20T22:18:07.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:07.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.899614+0000", 2024-09-20T22:18:07.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3470000000000001e-06 2024-09-20T22:18:07.935 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.935 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.935 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:07.935 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.899628+0000", 2024-09-20T22:18:07.935 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3059e-05 2024-09-20T22:18:07.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:07.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.899634+0000", 2024-09-20T22:18:07.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7819999999999996e-06 2024-09-20T22:18:07.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:07.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.903250+0000", 2024-09-20T22:18:07.936 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0036158029999999999 2024-09-20T22:18:07.936 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.936 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.936 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:07.936 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.903271+0000", 2024-09-20T22:18:07.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0948e-05 2024-09-20T22:18:07.937 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.937 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:07.937 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.937 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.937 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:219467 10.d e441/370)", 2024-09-20T22:18:07.937 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:07.899653+0000", 2024-09-20T22:18:07.937 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.003229464, 2024-09-20T22:18:07.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003783429, 2024-09-20T22:18:07.937 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:07.937 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:07.937 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:18:07.937 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:07.937 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:07.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.899653+0000", 2024-09-20T22:18:07.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:07.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.899653+0000", 2024-09-20T22:18:07.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.938 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:07.938 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.899653+0000", 2024-09-20T22:18:07.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3000000000000002e-07 2024-09-20T22:18:07.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:07.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.899657+0000", 2024-09-20T22:18:07.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8e-06 2024-09-20T22:18:07.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:07.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.899665+0000", 2024-09-20T22:18:07.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0790000000000001e-06 2024-09-20T22:18:07.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:07.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.899670+0000", 2024-09-20T22:18:07.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.493e-06 2024-09-20T22:18:07.940 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.940 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:07.940 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.940 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop_reply(client.16005.0:219460 10.8 e441/370)", 2024-09-20T22:18:07.940 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:07.900555+0000", 2024-09-20T22:18:07.940 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0023271770000000002, 2024-09-20T22:18:07.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002926854, 2024-09-20T22:18:07.940 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:07.940 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:07.940 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:18:07.940 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:07.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:07.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.900555+0000", 2024-09-20T22:18:07.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:07.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.900555+0000", 2024-09-20T22:18:07.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:07.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.900556+0000", 2024-09-20T22:18:07.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4000000000000001e-07 2024-09-20T22:18:07.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:07.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.900560+0000", 2024-09-20T22:18:07.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5600000000000004e-06 2024-09-20T22:18:07.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:07.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.900572+0000", 2024-09-20T22:18:07.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2077e-05 2024-09-20T22:18:07.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:07.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.900578+0000", 2024-09-20T22:18:07.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.699e-06 2024-09-20T22:18:07.942 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.942 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:07.942 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.943 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop_reply(client.16005.0:219460 10.8 e441/370)", 2024-09-20T22:18:07.943 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:07.901208+0000", 2024-09-20T22:18:07.943 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0016739140000000001, 2024-09-20T22:18:07.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023182820000000001, 2024-09-20T22:18:07.943 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:07.943 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:07.943 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:18:07.943 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:07.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:07.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.901208+0000", 2024-09-20T22:18:07.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:07.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.901208+0000", 2024-09-20T22:18:07.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:07.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:07.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.901209+0000", 2024-09-20T22:18:07.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5199999999999996e-07 2024-09-20T22:18:07.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:07.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.901213+0000", 2024-09-20T22:18:07.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.895e-06 2024-09-20T22:18:07.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:07.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.901226+0000", 2024-09-20T22:18:07.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2661e-05 2024-09-20T22:18:07.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:07.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:07.945 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:07.945 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:07.901232+0000", 2024-09-20T22:18:07.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4489999999999996e-06 2024-09-20T22:18:07.945 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.945 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:07.945 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.945 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:07.945 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:18:07.945 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 11 2024-09-20T22:18:07.946 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:18:08.001 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:08.027 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:08.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:08.028 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:08.129 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:08.167 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:08.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:08.168 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:08.269 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:08.290 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:08.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:08.291 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:08.349 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:08 smithi060 ceph-mon[29694]: 1.0 deep-scrub starts 2024-09-20T22:18:08.350 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:08 smithi060 ceph-mon[29694]: 1.0 deep-scrub ok 2024-09-20T22:18:08.350 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:08 smithi060 ceph-mon[29694]: 1.13 scrub starts 2024-09-20T22:18:08.350 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:08 smithi060 ceph-mon[29694]: 1.13 scrub ok 2024-09-20T22:18:08.350 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:08 smithi060 ceph-mon[29694]: from='client.25846 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:18:08.350 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:08 smithi060 ceph-mon[29694]: 1.10 scrub starts 2024-09-20T22:18:08.350 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:08 smithi060 ceph-mon[29694]: 1.10 scrub ok 2024-09-20T22:18:08.350 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:08 smithi060 ceph-mon[29694]: 10.7 deep-scrub starts 2024-09-20T22:18:08.350 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:08 smithi060 ceph-mon[40290]: 1.0 deep-scrub starts 2024-09-20T22:18:08.350 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:08 smithi060 ceph-mon[40290]: 1.0 deep-scrub ok 2024-09-20T22:18:08.350 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:08 smithi060 ceph-mon[40290]: 1.13 scrub starts 2024-09-20T22:18:08.350 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:08 smithi060 ceph-mon[40290]: 1.13 scrub ok 2024-09-20T22:18:08.350 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:08 smithi060 ceph-mon[40290]: from='client.25846 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:18:08.351 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:08 smithi060 ceph-mon[40290]: 1.10 scrub starts 2024-09-20T22:18:08.351 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:08 smithi060 ceph-mon[40290]: 1.10 scrub ok 2024-09-20T22:18:08.351 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:08 smithi060 ceph-mon[40290]: 10.7 deep-scrub starts 2024-09-20T22:18:08.392 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:08.428 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:08.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:08.429 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:08.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:08 smithi067 ceph-mon[33511]: 1.0 deep-scrub starts 2024-09-20T22:18:08.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:08 smithi067 ceph-mon[33511]: 1.0 deep-scrub ok 2024-09-20T22:18:08.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:08 smithi067 ceph-mon[33511]: 1.13 scrub starts 2024-09-20T22:18:08.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:08 smithi067 ceph-mon[33511]: 1.13 scrub ok 2024-09-20T22:18:08.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:08 smithi067 ceph-mon[33511]: from='client.25846 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:18:08.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:08 smithi067 ceph-mon[33511]: 1.10 scrub starts 2024-09-20T22:18:08.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:08 smithi067 ceph-mon[33511]: 1.10 scrub ok 2024-09-20T22:18:08.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:08 smithi067 ceph-mon[33511]: 10.7 deep-scrub starts 2024-09-20T22:18:08.529 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:08.559 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:08.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:08.560 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:08.661 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:08.701 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:08.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:08.702 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:08.721 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:18:08.802 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:08.827 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:08.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:08.828 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:08.929 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:08.962 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:08.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:08.964 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:09.064 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:09.100 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:09.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:09.102 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:09.159 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":814,"stamp":"2024-09-20T22:18:06.602180+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2241665587,"num_objects":34396,"num_object_clones":0,"num_object_copies":103188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34396,"num_whiteouts":0,"num_read":2025,"num_read_kb":1777,"num_write":216105,"num_write_kb":7986542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":5985,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79511,"ondisk_log_size":79511,"up":534,"acting":534,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":534,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":11918568,"kb_used_data":6701652,"kb_used_omap":15,"kb_used_meta":5216560,"kb_avail":738042648,"statfs":{"total":767960285184,"available":755755671552,"internally_reserved":0,"allocated":6862491648,"data_stored":6859588827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":5341758227},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,4,6,2,6],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-382337024,"num_objects":-5834,"num_object_clones":0,"num_object_copies":-17502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5834,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5834,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.994631"},"pg_stats":[{"pgid":"1.1d","version":"75'128","reported_seq":330,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.822761+0000","last_change":"2024-09-20T22:17:57.129002+0000","last_active":"2024-09-20T22:18:02.822761+0000","last_peered":"2024-09-20T22:18:02.822761+0000","last_clean":"2024-09-20T22:18:02.822761+0000","last_became_active":"2024-09-20T22:17:44.720436+0000","last_became_peered":"2024-09-20T22:17:44.720436+0000","last_unstale":"2024-09-20T22:18:02.822761+0000","last_undegraded":"2024-09-20T22:18:02.822761+0000","last_fullsized":"2024-09-20T22:18:02.822761+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:57.128894+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.128894+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:12.360328+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":257,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.616701+0000","last_change":"2024-09-20T22:16:31.993232+0000","last_active":"2024-09-20T22:18:01.616701+0000","last_peered":"2024-09-20T22:18:01.616701+0000","last_clean":"2024-09-20T22:18:01.616701+0000","last_became_active":"2024-09-20T22:12:52.515921+0000","last_became_peered":"2024-09-20T22:12:52.515921+0000","last_unstale":"2024-09-20T22:18:01.616701+0000","last_undegraded":"2024-09-20T22:18:01.616701+0000","last_fullsized":"2024-09-20T22:18:01.616701+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:31.993104+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:31.993104+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.993104+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"327'19","reported_seq":851,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.629805+0000","last_change":"2024-09-20T22:16:29.508726+0000","last_active":"2024-09-20T22:18:04.629805+0000","last_peered":"2024-09-20T22:18:04.629805+0000","last_clean":"2024-09-20T22:18:04.629805+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:18:04.629805+0000","last_undegraded":"2024-09-20T22:18:04.629805+0000","last_fullsized":"2024-09-20T22:18:04.629805+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"327'19","last_scrub_stamp":"2024-09-20T22:16:29.508607+0000","last_deep_scrub":"252'13","last_deep_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_clean_scrub_stamp":"2024-09-20T22:16:29.508607+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":694,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.815901+0000","last_change":"2024-09-20T22:17:04.319541+0000","last_active":"2024-09-20T22:18:02.815901+0000","last_peered":"2024-09-20T22:18:02.815901+0000","last_clean":"2024-09-20T22:18:02.815901+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:18:02.815901+0000","last_undegraded":"2024-09-20T22:18:02.815901+0000","last_fullsized":"2024-09-20T22:18:02.815901+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:04.319429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:04.319429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:24.998542+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":818,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.629952+0000","last_change":"2024-09-20T22:16:23.438091+0000","last_active":"2024-09-20T22:18:04.629952+0000","last_peered":"2024-09-20T22:18:04.629952+0000","last_clean":"2024-09-20T22:18:04.629952+0000","last_became_active":"2024-09-20T22:11:30.485735+0000","last_became_peered":"2024-09-20T22:11:30.485735+0000","last_unstale":"2024-09-20T22:18:04.629952+0000","last_undegraded":"2024-09-20T22:18:04.629952+0000","last_fullsized":"2024-09-20T22:18:04.629952+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:23.437959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:23.437959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1c","version":"75'128","reported_seq":772,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.633378+0000","last_change":"2024-09-20T22:17:57.852321+0000","last_active":"2024-09-20T22:18:04.633378+0000","last_peered":"2024-09-20T22:18:04.633378+0000","last_clean":"2024-09-20T22:18:04.633378+0000","last_became_active":"2024-09-20T22:17:44.724322+0000","last_became_peered":"2024-09-20T22:17:44.724322+0000","last_unstale":"2024-09-20T22:18:04.633378+0000","last_undegraded":"2024-09-20T22:18:04.633378+0000","last_fullsized":"2024-09-20T22:18:04.633378+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:57.852213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.852213+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:58.987600+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":809,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.667464+0000","last_change":"2024-09-20T22:18:01.667508+0000","last_active":"2024-09-20T22:18:01.667464+0000","last_peered":"2024-09-20T22:18:01.667464+0000","last_clean":"2024-09-20T22:18:01.667464+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:18:01.667464+0000","last_undegraded":"2024-09-20T22:18:01.667464+0000","last_fullsized":"2024-09-20T22:18:01.667464+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:18:01.667298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:01.667298+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:25.121768+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":286,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.823159+0000","last_change":"2024-09-20T22:16:22.789638+0000","last_active":"2024-09-20T22:18:02.823159+0000","last_peered":"2024-09-20T22:18:02.823159+0000","last_clean":"2024-09-20T22:18:02.823159+0000","last_became_active":"2024-09-20T22:12:33.545100+0000","last_became_peered":"2024-09-20T22:12:33.545100+0000","last_unstale":"2024-09-20T22:18:02.823159+0000","last_undegraded":"2024-09-20T22:18:02.823159+0000","last_fullsized":"2024-09-20T22:18:02.823159+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:22.789569+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:22.789569+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"406'34","reported_seq":419,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.623840+0000","last_change":"2024-09-20T22:17:55.941665+0000","last_active":"2024-09-20T22:18:04.623840+0000","last_peered":"2024-09-20T22:18:04.623840+0000","last_clean":"2024-09-20T22:18:04.623840+0000","last_became_active":"2024-09-20T22:12:52.517296+0000","last_became_peered":"2024-09-20T22:12:52.517296+0000","last_unstale":"2024-09-20T22:18:04.623840+0000","last_undegraded":"2024-09-20T22:18:04.623840+0000","last_fullsized":"2024-09-20T22:18:04.623840+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'34","last_scrub_stamp":"2024-09-20T22:17:55.940057+0000","last_deep_scrub":"406'34","last_deep_scrub_stamp":"2024-09-20T22:16:31.033354+0000","last_clean_scrub_stamp":"2024-09-20T22:17:55.940057+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:57.162777+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":810,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.629709+0000","last_change":"2024-09-20T22:16:36.461743+0000","last_active":"2024-09-20T22:18:04.629709+0000","last_peered":"2024-09-20T22:18:04.629709+0000","last_clean":"2024-09-20T22:18:04.629709+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:18:04.629709+0000","last_undegraded":"2024-09-20T22:18:04.629709+0000","last_fullsized":"2024-09-20T22:18:04.629709+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:36.461517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:36.461517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":832,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.621288+0000","last_change":"2024-09-20T22:17:41.350439+0000","last_active":"2024-09-20T22:18:04.621288+0000","last_peered":"2024-09-20T22:18:04.621288+0000","last_clean":"2024-09-20T22:18:04.621288+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:18:04.621288+0000","last_undegraded":"2024-09-20T22:18:04.621288+0000","last_fullsized":"2024-09-20T22:18:04.621288+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:41.350325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:17:41.350325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:44.553517+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"328'15","reported_seq":692,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.623093+0000","last_change":"2024-09-20T22:17:03.697131+0000","last_active":"2024-09-20T22:18:01.623093+0000","last_peered":"2024-09-20T22:18:01.623093+0000","last_clean":"2024-09-20T22:18:01.623093+0000","last_became_active":"2024-09-20T22:17:03.663697+0000","last_became_peered":"2024-09-20T22:17:03.663697+0000","last_unstale":"2024-09-20T22:18:01.623093+0000","last_undegraded":"2024-09-20T22:18:01.623093+0000","last_fullsized":"2024-09-20T22:18:01.623093+0000","mapping_epoch":415,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":416,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:15:59.331593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:59.331593+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":363,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.621470+0000","last_change":"2024-09-20T22:17:30.235268+0000","last_active":"2024-09-20T22:18:04.621470+0000","last_peered":"2024-09-20T22:18:04.621470+0000","last_clean":"2024-09-20T22:18:04.621470+0000","last_became_active":"2024-09-20T22:12:52.518549+0000","last_became_peered":"2024-09-20T22:12:52.518549+0000","last_unstale":"2024-09-20T22:18:04.621470+0000","last_undegraded":"2024-09-20T22:18:04.621470+0000","last_fullsized":"2024-09-20T22:18:04.621470+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:30.235059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:30.235059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:50.727250+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":238,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.816033+0000","last_change":"2024-09-20T22:17:03.665051+0000","last_active":"2024-09-20T22:18:02.816033+0000","last_peered":"2024-09-20T22:18:02.816033+0000","last_clean":"2024-09-20T22:18:02.816033+0000","last_became_active":"2024-09-20T22:17:03.664619+0000","last_became_peered":"2024-09-20T22:17:03.664619+0000","last_unstale":"2024-09-20T22:18:02.816033+0000","last_undegraded":"2024-09-20T22:18:02.816033+0000","last_fullsized":"2024-09-20T22:18:02.816033+0000","mapping_epoch":415,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":416,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:37.934760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:37.934760+0000","last_clean_scrub_stamp":"2024-09-20T22:16:37.934760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1e","version":"75'128","reported_seq":327,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.618589+0000","last_change":"2024-09-20T22:17:56.699899+0000","last_active":"2024-09-20T22:18:01.618589+0000","last_peered":"2024-09-20T22:18:01.618589+0000","last_clean":"2024-09-20T22:18:01.618589+0000","last_became_active":"2024-09-20T22:17:44.708330+0000","last_became_peered":"2024-09-20T22:17:44.708330+0000","last_unstale":"2024-09-20T22:18:01.618589+0000","last_undegraded":"2024-09-20T22:18:01.618589+0000","last_fullsized":"2024-09-20T22:18:01.618589+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.699777+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:17:40.367940+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.699777+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:11.112545+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":837,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.620866+0000","last_change":"2024-09-20T22:18:03.470422+0000","last_active":"2024-09-20T22:18:04.620866+0000","last_peered":"2024-09-20T22:18:04.620866+0000","last_clean":"2024-09-20T22:18:04.620866+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:18:04.620866+0000","last_undegraded":"2024-09-20T22:18:04.620866+0000","last_fullsized":"2024-09-20T22:18:04.620866+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:03.470297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:30.215263+0000","last_clean_scrub_stamp":"2024-09-20T22:18:03.470297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:14.814301+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"328'7","reported_seq":847,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.630063+0000","last_change":"2024-09-20T22:17:30.369631+0000","last_active":"2024-09-20T22:18:04.630063+0000","last_peered":"2024-09-20T22:18:04.630063+0000","last_clean":"2024-09-20T22:18:04.630063+0000","last_became_active":"2024-09-20T22:08:22.144364+0000","last_became_peered":"2024-09-20T22:08:22.144364+0000","last_unstale":"2024-09-20T22:18:04.630063+0000","last_undegraded":"2024-09-20T22:18:04.630063+0000","last_fullsized":"2024-09-20T22:18:04.630063+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'7","last_scrub_stamp":"2024-09-20T22:17:30.369518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:30.369518+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:31.277150+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":818,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.632570+0000","last_change":"2024-09-20T22:17:03.783046+0000","last_active":"2024-09-20T22:18:04.632570+0000","last_peered":"2024-09-20T22:18:04.632570+0000","last_clean":"2024-09-20T22:18:04.632570+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:18:04.632570+0000","last_undegraded":"2024-09-20T22:18:04.632570+0000","last_fullsized":"2024-09-20T22:18:04.632570+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:03.782934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:03.782934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:09.870686+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":351,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.620492+0000","last_change":"2024-09-20T22:17:33.243978+0000","last_active":"2024-09-20T22:18:04.620492+0000","last_peered":"2024-09-20T22:18:04.620492+0000","last_clean":"2024-09-20T22:18:04.620492+0000","last_became_active":"2024-09-20T22:11:30.487137+0000","last_became_peered":"2024-09-20T22:11:30.487137+0000","last_unstale":"2024-09-20T22:18:04.620492+0000","last_undegraded":"2024-09-20T22:18:04.620492+0000","last_fullsized":"2024-09-20T22:18:04.620492+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:33.243552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:17:33.243552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:53.049349+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.19","version":"75'128","reported_seq":593,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.616090+0000","last_change":"2024-09-20T22:17:56.103865+0000","last_active":"2024-09-20T22:18:01.616090+0000","last_peered":"2024-09-20T22:18:01.616090+0000","last_clean":"2024-09-20T22:18:01.616090+0000","last_became_active":"2024-09-20T22:17:44.716139+0000","last_became_peered":"2024-09-20T22:17:44.716139+0000","last_unstale":"2024-09-20T22:18:01.616090+0000","last_undegraded":"2024-09-20T22:18:01.616090+0000","last_fullsized":"2024-09-20T22:18:01.616090+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.103744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.103744+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:23.437505+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":806,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.631624+0000","last_change":"2024-09-20T22:16:34.472967+0000","last_active":"2024-09-20T22:18:04.631624+0000","last_peered":"2024-09-20T22:18:04.631624+0000","last_clean":"2024-09-20T22:18:04.631624+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:18:04.631624+0000","last_undegraded":"2024-09-20T22:18:04.631624+0000","last_fullsized":"2024-09-20T22:18:04.631624+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:34.472700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:16:34.472700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":85,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.630519+0000","last_change":"2024-09-20T22:17:07.404010+0000","last_active":"2024-09-20T22:18:04.630519+0000","last_peered":"2024-09-20T22:18:04.630519+0000","last_clean":"2024-09-20T22:18:04.630519+0000","last_became_active":"2024-09-20T22:17:03.652857+0000","last_became_peered":"2024-09-20T22:17:03.652857+0000","last_unstale":"2024-09-20T22:18:04.630519+0000","last_undegraded":"2024-09-20T22:18:04.630519+0000","last_fullsized":"2024-09-20T22:18:04.630519+0000","mapping_epoch":415,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":416,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:07.403912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:17:07.403912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:35.691657+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"328'34","reported_seq":415,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.624405+0000","last_change":"2024-09-20T22:17:41.808122+0000","last_active":"2024-09-20T22:18:04.624405+0000","last_peered":"2024-09-20T22:18:04.624405+0000","last_clean":"2024-09-20T22:18:04.624405+0000","last_became_active":"2024-09-20T22:11:12.507311+0000","last_became_peered":"2024-09-20T22:11:12.507311+0000","last_unstale":"2024-09-20T22:18:04.624405+0000","last_undegraded":"2024-09-20T22:18:04.624405+0000","last_fullsized":"2024-09-20T22:18:04.624405+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'34","last_scrub_stamp":"2024-09-20T22:17:41.807990+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:17:41.807990+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:56.934080+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":656,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.823006+0000","last_change":"2024-09-20T22:16:23.258053+0000","last_active":"2024-09-20T22:18:02.823006+0000","last_peered":"2024-09-20T22:18:02.823006+0000","last_clean":"2024-09-20T22:18:02.823006+0000","last_became_active":"2024-09-20T22:12:52.520985+0000","last_became_peered":"2024-09-20T22:12:52.520985+0000","last_unstale":"2024-09-20T22:18:02.823006+0000","last_undegraded":"2024-09-20T22:18:02.823006+0000","last_fullsized":"2024-09-20T22:18:02.823006+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:23.257937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:11.053276+0000","last_clean_scrub_stamp":"2024-09-20T22:16:23.257937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.18","version":"75'128","reported_seq":437,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.620141+0000","last_change":"2024-09-20T22:17:56.357255+0000","last_active":"2024-09-20T22:18:04.620141+0000","last_peered":"2024-09-20T22:18:04.620141+0000","last_clean":"2024-09-20T22:18:04.620141+0000","last_became_active":"2024-09-20T22:17:44.717551+0000","last_became_peered":"2024-09-20T22:17:44.717551+0000","last_unstale":"2024-09-20T22:18:04.620141+0000","last_undegraded":"2024-09-20T22:18:04.620141+0000","last_fullsized":"2024-09-20T22:18:04.620141+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.357167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.357167+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.867832+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":755,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.822108+0000","last_change":"2024-09-20T22:16:16.771843+0000","last_active":"2024-09-20T22:18:02.822108+0000","last_peered":"2024-09-20T22:18:02.822108+0000","last_clean":"2024-09-20T22:18:02.822108+0000","last_became_active":"2024-09-20T22:12:21.521433+0000","last_became_peered":"2024-09-20T22:12:21.521433+0000","last_unstale":"2024-09-20T22:18:02.822108+0000","last_undegraded":"2024-09-20T22:18:02.822108+0000","last_fullsized":"2024-09-20T22:18:02.822108+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:16.771744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:16.771744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":758,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.620865+0000","last_change":"2024-09-20T22:15:58.078205+0000","last_active":"2024-09-20T22:18:04.620865+0000","last_peered":"2024-09-20T22:18:04.620865+0000","last_clean":"2024-09-20T22:18:04.620865+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:18:04.620865+0000","last_undegraded":"2024-09-20T22:18:04.620865+0000","last_fullsized":"2024-09-20T22:18:04.620865+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:58.078099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:58.078099+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"328'23","reported_seq":729,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.823269+0000","last_change":"2024-09-20T22:17:38.263995+0000","last_active":"2024-09-20T22:18:02.823269+0000","last_peered":"2024-09-20T22:18:02.823269+0000","last_clean":"2024-09-20T22:18:02.823269+0000","last_became_active":"2024-09-20T22:12:21.517147+0000","last_became_peered":"2024-09-20T22:12:21.517147+0000","last_unstale":"2024-09-20T22:18:02.823269+0000","last_undegraded":"2024-09-20T22:18:02.823269+0000","last_fullsized":"2024-09-20T22:18:02.823269+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'23","last_scrub_stamp":"2024-09-20T22:17:38.263865+0000","last_deep_scrub":"250'17","last_deep_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_clean_scrub_stamp":"2024-09-20T22:17:38.263865+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:01.181325+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":809,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.633220+0000","last_change":"2024-09-20T22:17:09.392111+0000","last_active":"2024-09-20T22:18:04.633220+0000","last_peered":"2024-09-20T22:18:04.633220+0000","last_clean":"2024-09-20T22:18:04.633220+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:18:04.633220+0000","last_undegraded":"2024-09-20T22:18:04.633220+0000","last_fullsized":"2024-09-20T22:18:04.633220+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:09.391995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:09.391995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:31.332781+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1b","version":"75'128","reported_seq":741,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.619953+0000","last_change":"2024-09-20T22:17:58.426160+0000","last_active":"2024-09-20T22:18:04.619953+0000","last_peered":"2024-09-20T22:18:04.619953+0000","last_clean":"2024-09-20T22:18:04.619953+0000","last_became_active":"2024-09-20T22:17:44.725866+0000","last_became_peered":"2024-09-20T22:17:44.725866+0000","last_unstale":"2024-09-20T22:18:04.619953+0000","last_undegraded":"2024-09-20T22:18:04.619953+0000","last_fullsized":"2024-09-20T22:18:04.619953+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:58.426044+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:17:58.426044+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:20.577220+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":799,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.633235+0000","last_change":"2024-09-20T22:15:57.632871+0000","last_active":"2024-09-20T22:18:04.633235+0000","last_peered":"2024-09-20T22:18:04.633235+0000","last_clean":"2024-09-20T22:18:04.633235+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:18:04.633235+0000","last_undegraded":"2024-09-20T22:18:04.633235+0000","last_fullsized":"2024-09-20T22:18:04.633235+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:15:57.632600+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.632600+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"289'2","reported_seq":654,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.816003+0000","last_change":"2024-09-20T22:17:07.356345+0000","last_active":"2024-09-20T22:18:02.816003+0000","last_peered":"2024-09-20T22:18:02.816003+0000","last_clean":"2024-09-20T22:18:02.816003+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:18:02.816003+0000","last_undegraded":"2024-09-20T22:18:02.816003+0000","last_fullsized":"2024-09-20T22:18:02.816003+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-20T22:17:07.356220+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_clean_scrub_stamp":"2024-09-20T22:17:07.356220+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:31.683638+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"328'20","reported_seq":284,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.617271+0000","last_change":"2024-09-20T22:15:57.132570+0000","last_active":"2024-09-20T22:18:01.617271+0000","last_peered":"2024-09-20T22:18:01.617271+0000","last_clean":"2024-09-20T22:18:01.617271+0000","last_became_active":"2024-09-20T22:12:52.526803+0000","last_became_peered":"2024-09-20T22:12:52.526803+0000","last_unstale":"2024-09-20T22:18:01.617271+0000","last_undegraded":"2024-09-20T22:18:01.617271+0000","last_fullsized":"2024-09-20T22:18:01.617271+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:15:57.132435+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.132435+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"291'11","reported_seq":1038,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.822389+0000","last_change":"2024-09-20T22:16:34.733980+0000","last_active":"2024-09-20T22:18:02.822389+0000","last_peered":"2024-09-20T22:18:02.822389+0000","last_clean":"2024-09-20T22:18:02.822389+0000","last_became_active":"2024-09-20T22:11:40.601048+0000","last_became_peered":"2024-09-20T22:11:40.601048+0000","last_unstale":"2024-09-20T22:18:02.822389+0000","last_undegraded":"2024-09-20T22:18:02.822389+0000","last_fullsized":"2024-09-20T22:18:02.822389+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'11","last_scrub_stamp":"2024-09-20T22:16:34.733870+0000","last_deep_scrub":"291'11","last_deep_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_clean_scrub_stamp":"2024-09-20T22:16:34.733870+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"440'13451","reported_seq":13670,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:05.945166+0000","last_change":"2024-09-20T22:16:24.149762+0000","last_active":"2024-09-20T22:18:05.945166+0000","last_peered":"2024-09-20T22:18:05.945166+0000","last_clean":"2024-09-20T22:18:05.945166+0000","last_became_active":"2024-09-20T22:15:08.927535+0000","last_became_peered":"2024-09-20T22:15:08.927535+0000","last_unstale":"2024-09-20T22:18:05.945166+0000","last_undegraded":"2024-09-20T22:18:05.945166+0000","last_fullsized":"2024-09-20T22:18:05.945166+0000","mapping_epoch":370,"log_start":"417'9264","ondisk_log_start":"417'9264","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"405'6096","last_scrub_stamp":"2024-09-20T22:16:24.146029+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:24.146029+0000","objects_scrubbed":6025,"log_size":4187,"log_dups_size":0,"ondisk_log_size":4187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1743,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134545408,"num_objects":2053,"num_object_clones":0,"num_object_copies":6159,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13451,"num_write_kb":496128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,0],"acting":[4,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":832,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.620705+0000","last_change":"2024-09-20T22:17:40.331506+0000","last_active":"2024-09-20T22:18:04.620705+0000","last_peered":"2024-09-20T22:18:04.620705+0000","last_clean":"2024-09-20T22:18:04.620705+0000","last_became_active":"2024-09-20T22:17:40.139526+0000","last_became_peered":"2024-09-20T22:17:40.139526+0000","last_unstale":"2024-09-20T22:18:04.620705+0000","last_undegraded":"2024-09-20T22:18:04.620705+0000","last_fullsized":"2024-09-20T22:18:04.620705+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:40.331395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:40.331395+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:57.977510+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"328'27","reported_seq":609,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.624650+0000","last_change":"2024-09-20T22:17:34.818518+0000","last_active":"2024-09-20T22:18:04.624650+0000","last_peered":"2024-09-20T22:18:04.624650+0000","last_clean":"2024-09-20T22:18:04.624650+0000","last_became_active":"2024-09-20T22:08:22.145408+0000","last_became_peered":"2024-09-20T22:08:22.145408+0000","last_unstale":"2024-09-20T22:18:04.624650+0000","last_undegraded":"2024-09-20T22:18:04.624650+0000","last_fullsized":"2024-09-20T22:18:04.624650+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:17:34.818375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:34.818375+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:01.304955+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":830,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.617483+0000","last_change":"2024-09-20T22:17:57.726107+0000","last_active":"2024-09-20T22:18:01.617483+0000","last_peered":"2024-09-20T22:18:01.617483+0000","last_clean":"2024-09-20T22:18:01.617483+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:18:01.617483+0000","last_undegraded":"2024-09-20T22:18:01.617483+0000","last_fullsized":"2024-09-20T22:18:01.617483+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:57.726043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:28.020341+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.726043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:07.260365+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"326'13","reported_seq":1062,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.631934+0000","last_change":"2024-09-20T22:16:15.367687+0000","last_active":"2024-09-20T22:18:04.631934+0000","last_peered":"2024-09-20T22:18:04.631934+0000","last_clean":"2024-09-20T22:18:04.631934+0000","last_became_active":"2024-09-20T22:12:52.517588+0000","last_became_peered":"2024-09-20T22:12:52.517588+0000","last_unstale":"2024-09-20T22:18:04.631934+0000","last_undegraded":"2024-09-20T22:18:04.631934+0000","last_fullsized":"2024-09-20T22:18:04.631934+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'13","last_scrub_stamp":"2024-09-20T22:16:15.367449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:15.367449+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":363,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.620805+0000","last_change":"2024-09-20T22:18:04.453544+0000","last_active":"2024-09-20T22:18:04.620805+0000","last_peered":"2024-09-20T22:18:04.620805+0000","last_clean":"2024-09-20T22:18:04.620805+0000","last_became_active":"2024-09-20T22:12:52.517826+0000","last_became_peered":"2024-09-20T22:12:52.517826+0000","last_unstale":"2024-09-20T22:18:04.620805+0000","last_undegraded":"2024-09-20T22:18:04.620805+0000","last_fullsized":"2024-09-20T22:18:04.620805+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:04.453445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:04.453445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:10.865173+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.e","version":"440'13283","reported_seq":13508,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:05.932697+0000","last_change":"2024-09-20T22:16:34.386928+0000","last_active":"2024-09-20T22:18:05.932697+0000","last_peered":"2024-09-20T22:18:05.932697+0000","last_clean":"2024-09-20T22:18:05.932697+0000","last_became_active":"2024-09-20T22:15:08.928064+0000","last_became_peered":"2024-09-20T22:15:08.928064+0000","last_unstale":"2024-09-20T22:18:05.932697+0000","last_undegraded":"2024-09-20T22:18:05.932697+0000","last_fullsized":"2024-09-20T22:18:05.932697+0000","mapping_epoch":370,"log_start":"416'9127","ondisk_log_start":"416'9127","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6697","last_scrub_stamp":"2024-09-20T22:16:34.386672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:34.386672+0000","objects_scrubbed":6630,"log_size":4156,"log_dups_size":0,"ondisk_log_size":4156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"queued for scrub","scrub_duration":1869,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134938624,"num_objects":2059,"num_object_clones":0,"num_object_copies":6177,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13283,"num_write_kb":490944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":837,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.630779+0000","last_change":"2024-09-20T22:17:42.412312+0000","last_active":"2024-09-20T22:18:04.630779+0000","last_peered":"2024-09-20T22:18:04.630779+0000","last_clean":"2024-09-20T22:18:04.630779+0000","last_became_active":"2024-09-20T22:17:40.147065+0000","last_became_peered":"2024-09-20T22:17:40.147065+0000","last_unstale":"2024-09-20T22:18:04.630779+0000","last_undegraded":"2024-09-20T22:18:04.630779+0000","last_fullsized":"2024-09-20T22:18:04.630779+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:42.412207+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:17:42.412207+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:52.569394+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"328'30","reported_seq":821,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.635674+0000","last_change":"2024-09-20T22:16:16.609501+0000","last_active":"2024-09-20T22:18:04.635674+0000","last_peered":"2024-09-20T22:18:04.635674+0000","last_clean":"2024-09-20T22:18:04.635674+0000","last_became_active":"2024-09-20T22:12:52.526867+0000","last_became_peered":"2024-09-20T22:12:52.526867+0000","last_unstale":"2024-09-20T22:18:04.635674+0000","last_undegraded":"2024-09-20T22:18:04.635674+0000","last_fullsized":"2024-09-20T22:18:04.635674+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'30","last_scrub_stamp":"2024-09-20T22:16:16.609377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:16.609377+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":302,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.634573+0000","last_change":"2024-09-20T22:16:28.701889+0000","last_active":"2024-09-20T22:18:04.634573+0000","last_peered":"2024-09-20T22:18:04.634573+0000","last_clean":"2024-09-20T22:18:04.634573+0000","last_became_active":"2024-09-20T22:12:52.528947+0000","last_became_peered":"2024-09-20T22:12:52.528947+0000","last_unstale":"2024-09-20T22:18:04.634573+0000","last_undegraded":"2024-09-20T22:18:04.634573+0000","last_fullsized":"2024-09-20T22:18:04.634573+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:28.701780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:28.701780+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":822,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.630010+0000","last_change":"2024-09-20T22:15:56.322206+0000","last_active":"2024-09-20T22:18:04.630010+0000","last_peered":"2024-09-20T22:18:04.630010+0000","last_clean":"2024-09-20T22:18:04.630010+0000","last_became_active":"2024-09-20T22:10:48.919949+0000","last_became_peered":"2024-09-20T22:10:48.919949+0000","last_unstale":"2024-09-20T22:18:04.630010+0000","last_undegraded":"2024-09-20T22:18:04.630010+0000","last_fullsized":"2024-09-20T22:18:04.630010+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:56.321409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:56.321409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":814,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.630211+0000","last_change":"2024-09-20T22:18:04.327361+0000","last_active":"2024-09-20T22:18:04.630211+0000","last_peered":"2024-09-20T22:18:04.630211+0000","last_clean":"2024-09-20T22:18:04.630211+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:18:04.630211+0000","last_undegraded":"2024-09-20T22:18:04.630211+0000","last_fullsized":"2024-09-20T22:18:04.630211+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:04.327270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:04.327270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:33.395263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"439'13234","reported_seq":13460,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:03.187781+0000","last_change":"2024-09-20T22:17:01.077916+0000","last_active":"2024-09-20T22:18:03.187781+0000","last_peered":"2024-09-20T22:18:03.187781+0000","last_clean":"2024-09-20T22:18:03.187781+0000","last_became_active":"2024-09-20T22:15:09.088580+0000","last_became_peered":"2024-09-20T22:15:09.088580+0000","last_unstale":"2024-09-20T22:18:03.187781+0000","last_undegraded":"2024-09-20T22:18:03.187781+0000","last_fullsized":"2024-09-20T22:18:03.187781+0000","mapping_epoch":370,"log_start":"414'8601","ondisk_log_start":"414'8601","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"414'8713","last_scrub_stamp":"2024-09-20T22:17:01.077656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:17:01.077656+0000","objects_scrubbed":6782,"log_size":4633,"log_dups_size":0,"ondisk_log_size":4633,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2367,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142213120,"num_objects":2170,"num_object_clones":0,"num_object_copies":6510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13234,"num_write_kb":492928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":862,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.618425+0000","last_change":"2024-09-20T22:17:29.719066+0000","last_active":"2024-09-20T22:18:01.618425+0000","last_peered":"2024-09-20T22:18:01.618425+0000","last_clean":"2024-09-20T22:18:01.618425+0000","last_became_active":"2024-09-20T22:15:45.642671+0000","last_became_peered":"2024-09-20T22:15:45.642671+0000","last_unstale":"2024-09-20T22:18:01.618425+0000","last_undegraded":"2024-09-20T22:18:01.618425+0000","last_fullsized":"2024-09-20T22:18:01.618425+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:29.718808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:29.718808+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:42.680208+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"328'28","reported_seq":856,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.630424+0000","last_change":"2024-09-20T22:16:18.404188+0000","last_active":"2024-09-20T22:18:04.630424+0000","last_peered":"2024-09-20T22:18:04.630424+0000","last_clean":"2024-09-20T22:18:04.630424+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:18:04.630424+0000","last_undegraded":"2024-09-20T22:18:04.630424+0000","last_fullsized":"2024-09-20T22:18:04.630424+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:16:18.403972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:18.403972+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":269,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.617475+0000","last_change":"2024-09-20T22:16:35.952285+0000","last_active":"2024-09-20T22:18:01.617475+0000","last_peered":"2024-09-20T22:18:01.617475+0000","last_clean":"2024-09-20T22:18:01.617475+0000","last_became_active":"2024-09-20T22:12:52.528068+0000","last_became_peered":"2024-09-20T22:12:52.528068+0000","last_unstale":"2024-09-20T22:18:01.617475+0000","last_undegraded":"2024-09-20T22:18:01.617475+0000","last_fullsized":"2024-09-20T22:18:01.617475+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:16:35.952049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:35.952049+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"315'11","reported_seq":570,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.815600+0000","last_change":"2024-09-20T22:15:56.451915+0000","last_active":"2024-09-20T22:18:02.815600+0000","last_peered":"2024-09-20T22:18:02.815600+0000","last_clean":"2024-09-20T22:18:02.815600+0000","last_became_active":"2024-09-20T22:12:33.545942+0000","last_became_peered":"2024-09-20T22:12:33.545942+0000","last_unstale":"2024-09-20T22:18:02.815600+0000","last_undegraded":"2024-09-20T22:18:02.815600+0000","last_fullsized":"2024-09-20T22:18:02.815600+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:15:56.451808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:56.451808+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":798,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.632646+0000","last_change":"2024-09-20T22:15:52.564511+0000","last_active":"2024-09-20T22:18:04.632646+0000","last_peered":"2024-09-20T22:18:04.632646+0000","last_clean":"2024-09-20T22:18:04.632646+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:18:04.632646+0000","last_undegraded":"2024-09-20T22:18:04.632646+0000","last_fullsized":"2024-09-20T22:18:04.632646+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:52.564398+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:52.564398+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"440'13493","reported_seq":13697,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:05.867872+0000","last_change":"2024-09-20T22:17:33.338870+0000","last_active":"2024-09-20T22:18:05.867872+0000","last_peered":"2024-09-20T22:18:05.867872+0000","last_clean":"2024-09-20T22:18:05.867872+0000","last_became_active":"2024-09-20T22:15:08.931653+0000","last_became_peered":"2024-09-20T22:15:08.931653+0000","last_unstale":"2024-09-20T22:18:05.867872+0000","last_undegraded":"2024-09-20T22:18:05.867872+0000","last_fullsized":"2024-09-20T22:18:05.867872+0000","mapping_epoch":370,"log_start":"417'9309","ondisk_log_start":"417'9309","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"425'11022","last_scrub_stamp":"2024-09-20T22:17:33.338531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:17:33.338531+0000","objects_scrubbed":4556,"log_size":4184,"log_dups_size":0,"ondisk_log_size":4184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:59.425004+0000","scrub_duration":1442,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":132448256,"num_objects":2021,"num_object_clones":0,"num_object_copies":6063,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13493,"num_write_kb":496448,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":873,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.821998+0000","last_change":"2024-09-20T22:17:41.753132+0000","last_active":"2024-09-20T22:18:02.821998+0000","last_peered":"2024-09-20T22:18:02.821998+0000","last_clean":"2024-09-20T22:18:02.821998+0000","last_became_active":"2024-09-20T22:17:40.142028+0000","last_became_peered":"2024-09-20T22:17:40.142028+0000","last_unstale":"2024-09-20T22:18:02.821998+0000","last_undegraded":"2024-09-20T22:18:02.821998+0000","last_fullsized":"2024-09-20T22:18:02.821998+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:41.752931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:41.752931+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:47.146105+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"328'20","reported_seq":722,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.832063+0000","last_change":"2024-09-20T22:16:28.261680+0000","last_active":"2024-09-20T22:18:02.832063+0000","last_peered":"2024-09-20T22:18:02.832063+0000","last_clean":"2024-09-20T22:18:02.832063+0000","last_became_active":"2024-09-20T22:11:30.486532+0000","last_became_peered":"2024-09-20T22:11:30.486532+0000","last_unstale":"2024-09-20T22:18:02.832063+0000","last_undegraded":"2024-09-20T22:18:02.832063+0000","last_fullsized":"2024-09-20T22:18:02.832063+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:16:28.261547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:28.261547+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":833,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.621375+0000","last_change":"2024-09-20T22:17:09.988639+0000","last_active":"2024-09-20T22:18:04.621375+0000","last_peered":"2024-09-20T22:18:04.621375+0000","last_clean":"2024-09-20T22:18:04.621375+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:18:04.621375+0000","last_undegraded":"2024-09-20T22:18:04.621375+0000","last_fullsized":"2024-09-20T22:18:04.621375+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:09.988552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:17:09.988552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:18.243010+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"363'13","reported_seq":321,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.620463+0000","last_change":"2024-09-20T22:16:25.963851+0000","last_active":"2024-09-20T22:18:01.620463+0000","last_peered":"2024-09-20T22:18:01.620463+0000","last_clean":"2024-09-20T22:18:01.620463+0000","last_became_active":"2024-09-20T22:12:57.884415+0000","last_became_peered":"2024-09-20T22:12:57.884415+0000","last_unstale":"2024-09-20T22:18:01.620463+0000","last_undegraded":"2024-09-20T22:18:01.620463+0000","last_fullsized":"2024-09-20T22:18:01.620463+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"363'13","last_scrub_stamp":"2024-09-20T22:16:25.963742+0000","last_deep_scrub":"363'13","last_deep_scrub_stamp":"2024-09-20T22:14:52.612615+0000","last_clean_scrub_stamp":"2024-09-20T22:16:25.963742+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":257,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.615398+0000","last_change":"2024-09-20T22:16:31.044840+0000","last_active":"2024-09-20T22:18:01.615398+0000","last_peered":"2024-09-20T22:18:01.615398+0000","last_clean":"2024-09-20T22:18:01.615398+0000","last_became_active":"2024-09-20T22:12:52.518529+0000","last_became_peered":"2024-09-20T22:12:52.518529+0000","last_unstale":"2024-09-20T22:18:01.615398+0000","last_undegraded":"2024-09-20T22:18:01.615398+0000","last_fullsized":"2024-09-20T22:18:01.615398+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:31.044564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:31.044564+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.044564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.d","version":"439'13227","reported_seq":13453,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:03.175144+0000","last_change":"2024-09-20T22:16:35.962440+0000","last_active":"2024-09-20T22:18:03.175144+0000","last_peered":"2024-09-20T22:18:03.175144+0000","last_clean":"2024-09-20T22:18:03.175144+0000","last_became_active":"2024-09-20T22:15:08.927661+0000","last_became_peered":"2024-09-20T22:15:08.927661+0000","last_unstale":"2024-09-20T22:18:03.175144+0000","last_undegraded":"2024-09-20T22:18:03.175144+0000","last_fullsized":"2024-09-20T22:18:03.175144+0000","mapping_epoch":370,"log_start":"414'8538","ondisk_log_start":"414'8538","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"407'6856","last_scrub_stamp":"2024-09-20T22:16:35.962220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:35.962220+0000","objects_scrubbed":6781,"log_size":4689,"log_dups_size":0,"ondisk_log_size":4689,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2297,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":151257112,"num_objects":2309,"num_object_clones":0,"num_object_copies":6927,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2309,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13227,"num_write_kb":497089,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":874,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.822909+0000","last_change":"2024-09-20T22:17:40.744592+0000","last_active":"2024-09-20T22:18:02.822909+0000","last_peered":"2024-09-20T22:18:02.822909+0000","last_clean":"2024-09-20T22:18:02.822909+0000","last_became_active":"2024-09-20T22:17:40.139979+0000","last_became_peered":"2024-09-20T22:17:40.139979+0000","last_unstale":"2024-09-20T22:18:02.822909+0000","last_undegraded":"2024-09-20T22:18:02.822909+0000","last_fullsized":"2024-09-20T22:18:02.822909+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:40.744482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:40.744482+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:59.156193+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"392'37","reported_seq":306,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.620753+0000","last_change":"2024-09-20T22:17:38.045665+0000","last_active":"2024-09-20T22:18:01.620753+0000","last_peered":"2024-09-20T22:18:01.620753+0000","last_clean":"2024-09-20T22:18:01.620753+0000","last_became_active":"2024-09-20T22:12:52.525538+0000","last_became_peered":"2024-09-20T22:12:52.525538+0000","last_unstale":"2024-09-20T22:18:01.620753+0000","last_undegraded":"2024-09-20T22:18:01.620753+0000","last_fullsized":"2024-09-20T22:18:01.620753+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"392'37","last_scrub_stamp":"2024-09-20T22:17:38.043643+0000","last_deep_scrub":"392'37","last_deep_scrub_stamp":"2024-09-20T22:15:56.127540+0000","last_clean_scrub_stamp":"2024-09-20T22:17:38.043643+0000","objects_scrubbed":8,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:56.236091+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":658,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.822806+0000","last_change":"2024-09-20T22:16:22.267306+0000","last_active":"2024-09-20T22:18:02.822806+0000","last_peered":"2024-09-20T22:18:02.822806+0000","last_clean":"2024-09-20T22:18:02.822806+0000","last_became_active":"2024-09-20T22:11:30.486428+0000","last_became_peered":"2024-09-20T22:11:30.486428+0000","last_unstale":"2024-09-20T22:18:02.822806+0000","last_undegraded":"2024-09-20T22:18:02.822806+0000","last_fullsized":"2024-09-20T22:18:02.822806+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:22.267191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:22.267191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":843,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.823024+0000","last_change":"2024-09-20T22:17:37.755759+0000","last_active":"2024-09-20T22:18:02.823024+0000","last_peered":"2024-09-20T22:18:02.823024+0000","last_clean":"2024-09-20T22:18:02.823024+0000","last_became_active":"2024-09-20T22:12:52.519826+0000","last_became_peered":"2024-09-20T22:12:52.519826+0000","last_unstale":"2024-09-20T22:18:02.823024+0000","last_undegraded":"2024-09-20T22:18:02.823024+0000","last_fullsized":"2024-09-20T22:18:02.823024+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:37.755521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:17:37.755521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:40.969469+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":652,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.815738+0000","last_change":"2024-09-20T22:17:36.301819+0000","last_active":"2024-09-20T22:18:02.815738+0000","last_peered":"2024-09-20T22:18:02.815738+0000","last_clean":"2024-09-20T22:18:02.815738+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:18:02.815738+0000","last_undegraded":"2024-09-20T22:18:02.815738+0000","last_fullsized":"2024-09-20T22:18:02.815738+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:36.301648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:36.301648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:42.314194+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"439'13540","reported_seq":13766,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:03.192215+0000","last_change":"2024-09-20T22:16:32.715147+0000","last_active":"2024-09-20T22:18:03.192215+0000","last_peered":"2024-09-20T22:18:03.192215+0000","last_clean":"2024-09-20T22:18:03.192215+0000","last_became_active":"2024-09-20T22:15:08.926245+0000","last_became_peered":"2024-09-20T22:15:08.926245+0000","last_unstale":"2024-09-20T22:18:03.192215+0000","last_undegraded":"2024-09-20T22:18:03.192215+0000","last_fullsized":"2024-09-20T22:18:03.192215+0000","mapping_epoch":370,"log_start":"414'8880","ondisk_log_start":"414'8880","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6800","last_scrub_stamp":"2024-09-20T22:16:32.714946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.714946+0000","objects_scrubbed":6743,"log_size":4660,"log_dups_size":0,"ondisk_log_size":4660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2056,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":144572416,"num_objects":2206,"num_object_clones":0,"num_object_copies":6618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2206,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13540,"num_write_kb":503872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":842,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.620280+0000","last_change":"2024-09-20T22:15:46.095461+0000","last_active":"2024-09-20T22:18:04.620280+0000","last_peered":"2024-09-20T22:18:04.620280+0000","last_clean":"2024-09-20T22:18:04.620280+0000","last_became_active":"2024-09-20T22:15:45.627135+0000","last_became_peered":"2024-09-20T22:15:45.627135+0000","last_unstale":"2024-09-20T22:18:04.620280+0000","last_undegraded":"2024-09-20T22:18:04.620280+0000","last_fullsized":"2024-09-20T22:18:04.620280+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:46.095188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:46.095188+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"328'52","reported_seq":877,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.822949+0000","last_change":"2024-09-20T22:17:08.395721+0000","last_active":"2024-09-20T22:18:02.822949+0000","last_peered":"2024-09-20T22:18:02.822949+0000","last_clean":"2024-09-20T22:18:02.822949+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:18:02.822949+0000","last_undegraded":"2024-09-20T22:18:02.822949+0000","last_fullsized":"2024-09-20T22:18:02.822949+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'52","last_scrub_stamp":"2024-09-20T22:17:08.395578+0000","last_deep_scrub":"328'52","last_deep_scrub_stamp":"2024-09-20T22:17:08.395578+0000","last_clean_scrub_stamp":"2024-09-20T22:17:08.395578+0000","objects_scrubbed":10,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:33.462021+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":835,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.630721+0000","last_change":"2024-09-20T22:17:28.386631+0000","last_active":"2024-09-20T22:18:04.630721+0000","last_peered":"2024-09-20T22:18:04.630721+0000","last_clean":"2024-09-20T22:18:04.630721+0000","last_became_active":"2024-09-20T22:12:57.884257+0000","last_became_peered":"2024-09-20T22:12:57.884257+0000","last_unstale":"2024-09-20T22:18:04.630721+0000","last_undegraded":"2024-09-20T22:18:04.630721+0000","last_fullsized":"2024-09-20T22:18:04.630721+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:28.386523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:28.386523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:37.601752+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":829,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.621121+0000","last_change":"2024-09-20T22:17:06.944648+0000","last_active":"2024-09-20T22:18:04.621121+0000","last_peered":"2024-09-20T22:18:04.621121+0000","last_clean":"2024-09-20T22:18:04.621121+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:18:04.621121+0000","last_undegraded":"2024-09-20T22:18:04.621121+0000","last_fullsized":"2024-09-20T22:18:04.621121+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:17:06.944406+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_clean_scrub_stamp":"2024-09-20T22:17:06.944406+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:33.207021+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":265,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.618566+0000","last_change":"2024-09-20T22:17:57.210932+0000","last_active":"2024-09-20T22:18:01.618566+0000","last_peered":"2024-09-20T22:18:01.618566+0000","last_clean":"2024-09-20T22:18:01.618566+0000","last_became_active":"2024-09-20T22:12:52.527547+0000","last_became_peered":"2024-09-20T22:12:52.527547+0000","last_unstale":"2024-09-20T22:18:01.618566+0000","last_undegraded":"2024-09-20T22:18:01.618566+0000","last_fullsized":"2024-09-20T22:18:01.618566+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:57.210706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.210706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:15.145091+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":800,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.618434+0000","last_change":"2024-09-20T22:16:58.792938+0000","last_active":"2024-09-20T22:18:01.618434+0000","last_peered":"2024-09-20T22:18:01.618434+0000","last_clean":"2024-09-20T22:18:01.618434+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:18:01.618434+0000","last_undegraded":"2024-09-20T22:18:01.618434+0000","last_fullsized":"2024-09-20T22:18:01.618434+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:58.792800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:16:58.792800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:10.710603+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"440'13679","reported_seq":13881,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:06.332665+0000","last_change":"2024-09-20T22:17:35.558661+0000","last_active":"2024-09-20T22:18:06.332665+0000","last_peered":"2024-09-20T22:18:06.332665+0000","last_clean":"2024-09-20T22:18:06.332665+0000","last_became_active":"2024-09-20T22:15:08.930968+0000","last_became_peered":"2024-09-20T22:15:08.930968+0000","last_unstale":"2024-09-20T22:18:06.332665+0000","last_undegraded":"2024-09-20T22:18:06.332665+0000","last_fullsized":"2024-09-20T22:18:06.332665+0000","mapping_epoch":370,"log_start":"416'9094","ondisk_log_start":"416'9094","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"425'11355","last_scrub_stamp":"2024-09-20T22:17:35.558330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:17:35.558330+0000","objects_scrubbed":4337,"log_size":4585,"log_dups_size":0,"ondisk_log_size":4585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:52.648027+0000","scrub_duration":1298,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129695744,"num_objects":1979,"num_object_clones":0,"num_object_copies":5937,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13679,"num_write_kb":501056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":848,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.632449+0000","last_change":"2024-09-20T22:17:38.946417+0000","last_active":"2024-09-20T22:18:04.632449+0000","last_peered":"2024-09-20T22:18:04.632449+0000","last_clean":"2024-09-20T22:18:04.632449+0000","last_became_active":"2024-09-20T22:15:45.620376+0000","last_became_peered":"2024-09-20T22:15:45.620376+0000","last_unstale":"2024-09-20T22:18:04.632449+0000","last_undegraded":"2024-09-20T22:18:04.632449+0000","last_fullsized":"2024-09-20T22:18:04.632449+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:38.946300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:38.946300+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:55.193938+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"328'17","reported_seq":725,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.824336+0000","last_change":"2024-09-20T22:17:01.307957+0000","last_active":"2024-09-20T22:18:02.824336+0000","last_peered":"2024-09-20T22:18:02.824336+0000","last_clean":"2024-09-20T22:18:02.824336+0000","last_became_active":"2024-09-20T22:12:52.521073+0000","last_became_peered":"2024-09-20T22:12:52.521073+0000","last_unstale":"2024-09-20T22:18:02.824336+0000","last_undegraded":"2024-09-20T22:18:02.824336+0000","last_fullsized":"2024-09-20T22:18:02.824336+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'17","last_scrub_stamp":"2024-09-20T22:17:01.307755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:01.307755+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:19.260372+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":653,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.815766+0000","last_change":"2024-09-20T22:17:09.478913+0000","last_active":"2024-09-20T22:18:02.815766+0000","last_peered":"2024-09-20T22:18:02.815766+0000","last_clean":"2024-09-20T22:18:02.815766+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:18:02.815766+0000","last_undegraded":"2024-09-20T22:18:02.815766+0000","last_fullsized":"2024-09-20T22:18:02.815766+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:09.478390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:09.478390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:15.007402+0000","scrub_duration":51,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":879,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.623935+0000","last_change":"2024-09-20T22:17:40.827298+0000","last_active":"2024-09-20T22:18:04.623935+0000","last_peered":"2024-09-20T22:18:04.623935+0000","last_clean":"2024-09-20T22:18:04.623935+0000","last_became_active":"2024-09-20T22:12:52.517255+0000","last_became_peered":"2024-09-20T22:12:52.517255+0000","last_unstale":"2024-09-20T22:18:04.623935+0000","last_undegraded":"2024-09-20T22:18:04.623935+0000","last_fullsized":"2024-09-20T22:18:04.623935+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:17:40.827184+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:17:40.827184+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:42.469590+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":833,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.625146+0000","last_change":"2024-09-20T22:17:08.832699+0000","last_active":"2024-09-20T22:18:04.625146+0000","last_peered":"2024-09-20T22:18:04.625146+0000","last_clean":"2024-09-20T22:18:04.625146+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:18:04.625146+0000","last_undegraded":"2024-09-20T22:18:04.625146+0000","last_fullsized":"2024-09-20T22:18:04.625146+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:08.832589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:17:08.832589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:14.550051+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":798,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.633889+0000","last_change":"2024-09-20T22:16:23.711184+0000","last_active":"2024-09-20T22:18:04.633889+0000","last_peered":"2024-09-20T22:18:04.633889+0000","last_clean":"2024-09-20T22:18:04.633889+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:18:04.633889+0000","last_undegraded":"2024-09-20T22:18:04.633889+0000","last_fullsized":"2024-09-20T22:18:04.633889+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:23.711072+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:23.711072+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":87,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"438'13244","reported_seq":13466,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.328298+0000","last_change":"2024-09-20T22:16:32.694961+0000","last_active":"2024-09-20T22:18:02.328298+0000","last_peered":"2024-09-20T22:18:02.328298+0000","last_clean":"2024-09-20T22:18:02.328298+0000","last_became_active":"2024-09-20T22:15:08.926702+0000","last_became_peered":"2024-09-20T22:15:08.926702+0000","last_unstale":"2024-09-20T22:18:02.328298+0000","last_undegraded":"2024-09-20T22:18:02.328298+0000","last_fullsized":"2024-09-20T22:18:02.328298+0000","mapping_epoch":370,"log_start":"414'8473","ondisk_log_start":"414'8473","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6715","last_scrub_stamp":"2024-09-20T22:16:32.694684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.694684+0000","objects_scrubbed":6639,"log_size":4771,"log_dups_size":0,"ondisk_log_size":4771,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2750,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147980288,"num_objects":2258,"num_object_clones":0,"num_object_copies":6774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13244,"num_write_kb":496064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":829,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.824836+0000","last_change":"2024-09-20T22:18:01.139847+0000","last_active":"2024-09-20T22:18:02.824836+0000","last_peered":"2024-09-20T22:18:02.824836+0000","last_clean":"2024-09-20T22:18:02.824836+0000","last_became_active":"2024-09-20T22:15:45.630804+0000","last_became_peered":"2024-09-20T22:15:45.630804+0000","last_unstale":"2024-09-20T22:18:02.824836+0000","last_undegraded":"2024-09-20T22:18:02.824836+0000","last_fullsized":"2024-09-20T22:18:02.824836+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:18:01.139707+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:18:01.139707+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:17.176581+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"328'24","reported_seq":297,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.617374+0000","last_change":"2024-09-20T22:17:05.822751+0000","last_active":"2024-09-20T22:18:01.617374+0000","last_peered":"2024-09-20T22:18:01.617374+0000","last_clean":"2024-09-20T22:18:01.617374+0000","last_became_active":"2024-09-20T22:12:52.527919+0000","last_became_peered":"2024-09-20T22:12:52.527919+0000","last_unstale":"2024-09-20T22:18:01.617374+0000","last_undegraded":"2024-09-20T22:18:01.617374+0000","last_fullsized":"2024-09-20T22:18:01.617374+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'24","last_scrub_stamp":"2024-09-20T22:17:05.822650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:05.822650+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:27.748220+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":271,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.616667+0000","last_change":"2024-09-20T22:17:11.814549+0000","last_active":"2024-09-20T22:18:01.616667+0000","last_peered":"2024-09-20T22:18:01.616667+0000","last_clean":"2024-09-20T22:18:01.616667+0000","last_became_active":"2024-09-20T22:17:03.663870+0000","last_became_peered":"2024-09-20T22:17:03.663870+0000","last_unstale":"2024-09-20T22:18:01.616667+0000","last_undegraded":"2024-09-20T22:18:01.616667+0000","last_fullsized":"2024-09-20T22:18:01.616667+0000","mapping_epoch":415,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":416,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:11.814439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:17:11.814439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:27.478502+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":1756,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.636067+0000","last_change":"2024-09-20T22:17:59.765338+0000","last_active":"2024-09-20T22:18:04.636067+0000","last_peered":"2024-09-20T22:18:04.636067+0000","last_clean":"2024-09-20T22:18:04.636067+0000","last_became_active":"2024-09-20T22:12:52.526622+0000","last_became_peered":"2024-09-20T22:12:52.526622+0000","last_unstale":"2024-09-20T22:18:04.636067+0000","last_undegraded":"2024-09-20T22:18:04.636067+0000","last_fullsized":"2024-09-20T22:18:04.636067+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:17:59.765217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:17:59.765217+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:23.072429+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":891,"num_read_kb":886,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":364,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.622393+0000","last_change":"2024-09-20T22:17:08.955894+0000","last_active":"2024-09-20T22:18:04.622393+0000","last_peered":"2024-09-20T22:18:04.622393+0000","last_clean":"2024-09-20T22:18:04.622393+0000","last_became_active":"2024-09-20T22:12:52.518823+0000","last_became_peered":"2024-09-20T22:12:52.518823+0000","last_unstale":"2024-09-20T22:18:04.622393+0000","last_undegraded":"2024-09-20T22:18:04.622393+0000","last_fullsized":"2024-09-20T22:18:04.622393+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:08.955790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:17:08.955790+0000","last_clean_scrub_stamp":"2024-09-20T22:17:08.955790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:18.057717+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":645,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.822561+0000","last_change":"2024-09-20T22:16:26.330409+0000","last_active":"2024-09-20T22:18:02.822561+0000","last_peered":"2024-09-20T22:18:02.822561+0000","last_clean":"2024-09-20T22:18:02.822561+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:18:02.822561+0000","last_undegraded":"2024-09-20T22:18:02.822561+0000","last_fullsized":"2024-09-20T22:18:02.822561+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:26.330270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:26.330270+0000","last_clean_scrub_stamp":"2024-09-20T22:16:26.330270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"439'13501","reported_seq":13725,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:03.200120+0000","last_change":"2024-09-20T22:17:01.821597+0000","last_active":"2024-09-20T22:18:03.200120+0000","last_peered":"2024-09-20T22:18:03.200120+0000","last_clean":"2024-09-20T22:18:03.200120+0000","last_became_active":"2024-09-20T22:15:08.924809+0000","last_became_peered":"2024-09-20T22:15:08.924809+0000","last_unstale":"2024-09-20T22:18:03.200120+0000","last_undegraded":"2024-09-20T22:18:03.200120+0000","last_fullsized":"2024-09-20T22:18:03.200120+0000","mapping_epoch":370,"log_start":"414'8838","ondisk_log_start":"414'8838","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"414'8932","last_scrub_stamp":"2024-09-20T22:17:01.821270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:17:01.821270+0000","objects_scrubbed":6888,"log_size":4663,"log_dups_size":0,"ondisk_log_size":4663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:07.401461+0000","scrub_duration":2077,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147652608,"num_objects":2253,"num_object_clones":0,"num_object_copies":6759,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13501,"num_write_kb":504128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,0],"acting":[6,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"328'28","reported_seq":1050,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.823708+0000","last_change":"2024-09-20T22:17:57.685174+0000","last_active":"2024-09-20T22:18:02.823708+0000","last_peered":"2024-09-20T22:18:02.823708+0000","last_clean":"2024-09-20T22:18:02.823708+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:18:02.823708+0000","last_undegraded":"2024-09-20T22:18:02.823708+0000","last_fullsized":"2024-09-20T22:18:02.823708+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:17:57.685068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.685068+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:23.761739+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":769,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.619998+0000","last_change":"2024-09-20T22:16:36.729250+0000","last_active":"2024-09-20T22:18:04.619998+0000","last_peered":"2024-09-20T22:18:04.619998+0000","last_clean":"2024-09-20T22:18:04.619998+0000","last_became_active":"2024-09-20T22:12:21.519717+0000","last_became_peered":"2024-09-20T22:12:21.519717+0000","last_unstale":"2024-09-20T22:18:04.619998+0000","last_undegraded":"2024-09-20T22:18:04.619998+0000","last_fullsized":"2024-09-20T22:18:04.619998+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:36.729158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:36.729158+0000","last_clean_scrub_stamp":"2024-09-20T22:16:36.729158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":839,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.635354+0000","last_change":"2024-09-20T22:15:47.554095+0000","last_active":"2024-09-20T22:18:04.635354+0000","last_peered":"2024-09-20T22:18:04.635354+0000","last_clean":"2024-09-20T22:18:04.635354+0000","last_became_active":"2024-09-20T22:15:45.620972+0000","last_became_peered":"2024-09-20T22:15:45.620972+0000","last_unstale":"2024-09-20T22:18:04.635354+0000","last_undegraded":"2024-09-20T22:18:04.635354+0000","last_fullsized":"2024-09-20T22:18:04.635354+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:47.553962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:47.553962+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":400,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.809816+0000","last_change":"2024-09-20T22:18:04.809849+0000","last_active":"2024-09-20T22:18:04.809816+0000","last_peered":"2024-09-20T22:18:04.809816+0000","last_clean":"2024-09-20T22:18:04.809816+0000","last_became_active":"2024-09-20T22:10:48.907271+0000","last_became_peered":"2024-09-20T22:10:48.907271+0000","last_unstale":"2024-09-20T22:18:04.809816+0000","last_undegraded":"2024-09-20T22:18:04.809816+0000","last_fullsized":"2024-09-20T22:18:04.809816+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:04.809565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_clean_scrub_stamp":"2024-09-20T22:18:04.809565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:30.143079+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":825,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.622336+0000","last_change":"2024-09-20T22:17:02.907969+0000","last_active":"2024-09-20T22:18:04.622336+0000","last_peered":"2024-09-20T22:18:04.622336+0000","last_clean":"2024-09-20T22:18:04.622336+0000","last_became_active":"2024-09-20T22:12:44.630044+0000","last_became_peered":"2024-09-20T22:12:44.630044+0000","last_unstale":"2024-09-20T22:18:04.622336+0000","last_undegraded":"2024-09-20T22:18:04.622336+0000","last_fullsized":"2024-09-20T22:18:04.622336+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:02.907851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:17:02.907851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:09.766067+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"75'128","reported_seq":842,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.816102+0000","last_change":"2024-09-20T22:17:40.268249+0000","last_active":"2024-09-20T22:18:02.816102+0000","last_peered":"2024-09-20T22:18:02.816102+0000","last_clean":"2024-09-20T22:18:02.816102+0000","last_became_active":"2024-09-20T22:17:40.147603+0000","last_became_peered":"2024-09-20T22:17:40.147603+0000","last_unstale":"2024-09-20T22:18:02.816102+0000","last_undegraded":"2024-09-20T22:18:02.816102+0000","last_fullsized":"2024-09-20T22:18:02.816102+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:40.268008+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:17:40.268008+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:50.466996+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"439'13624","reported_seq":13903,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:03.174218+0000","last_change":"2024-09-20T22:16:58.807828+0000","last_active":"2024-09-20T22:18:03.174218+0000","last_peered":"2024-09-20T22:18:03.174218+0000","last_clean":"2024-09-20T22:18:03.174218+0000","last_became_active":"2024-09-20T22:15:08.925040+0000","last_became_peered":"2024-09-20T22:15:08.925040+0000","last_unstale":"2024-09-20T22:18:03.174218+0000","last_undegraded":"2024-09-20T22:18:03.174218+0000","last_fullsized":"2024-09-20T22:18:03.174218+0000","mapping_epoch":370,"log_start":"414'8959","ondisk_log_start":"414'8959","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"414'8778","last_scrub_stamp":"2024-09-20T22:16:58.807519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:58.807519+0000","objects_scrubbed":7101,"log_size":4665,"log_dups_size":0,"ondisk_log_size":4665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2051,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142999552,"num_objects":2182,"num_object_clones":0,"num_object_copies":6546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13624,"num_write_kb":505792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,4],"acting":[6,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"328'19","reported_seq":845,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.631960+0000","last_change":"2024-09-20T22:16:24.435555+0000","last_active":"2024-09-20T22:18:04.631960+0000","last_peered":"2024-09-20T22:18:04.631960+0000","last_clean":"2024-09-20T22:18:04.631960+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:18:04.631960+0000","last_undegraded":"2024-09-20T22:18:04.631960+0000","last_fullsized":"2024-09-20T22:18:04.631960+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:16:24.435260+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:24.435260+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":818,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.630577+0000","last_change":"2024-09-20T22:17:36.374494+0000","last_active":"2024-09-20T22:18:04.630577+0000","last_peered":"2024-09-20T22:18:04.630577+0000","last_clean":"2024-09-20T22:18:04.630577+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:18:04.630577+0000","last_undegraded":"2024-09-20T22:18:04.630577+0000","last_fullsized":"2024-09-20T22:18:04.630577+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:36.374371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:36.374371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:05.201143+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":816,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.622074+0000","last_change":"2024-09-20T22:17:35.268781+0000","last_active":"2024-09-20T22:18:04.622074+0000","last_peered":"2024-09-20T22:18:04.622074+0000","last_clean":"2024-09-20T22:18:04.622074+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:18:04.622074+0000","last_undegraded":"2024-09-20T22:18:04.622074+0000","last_fullsized":"2024-09-20T22:18:04.622074+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:35.268679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:35.268679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:04.175881+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":770,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.822609+0000","last_change":"2024-09-20T22:17:55.630149+0000","last_active":"2024-09-20T22:18:02.822609+0000","last_peered":"2024-09-20T22:18:02.822609+0000","last_clean":"2024-09-20T22:18:02.822609+0000","last_became_active":"2024-09-20T22:12:21.521195+0000","last_became_peered":"2024-09-20T22:12:21.521195+0000","last_unstale":"2024-09-20T22:18:02.822609+0000","last_undegraded":"2024-09-20T22:18:02.822609+0000","last_fullsized":"2024-09-20T22:18:02.822609+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:55.630084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:55.630084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:15.908970+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3","version":"440'13604","reported_seq":13829,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:06.441216+0000","last_change":"2024-09-20T22:16:32.632993+0000","last_active":"2024-09-20T22:18:06.441216+0000","last_peered":"2024-09-20T22:18:06.441216+0000","last_clean":"2024-09-20T22:18:06.441216+0000","last_became_active":"2024-09-20T22:15:09.087061+0000","last_became_peered":"2024-09-20T22:15:09.087061+0000","last_unstale":"2024-09-20T22:18:06.441216+0000","last_undegraded":"2024-09-20T22:18:06.441216+0000","last_fullsized":"2024-09-20T22:18:06.441216+0000","mapping_epoch":370,"log_start":"412'8025","ondisk_log_start":"412'8025","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6740","last_scrub_stamp":"2024-09-20T22:16:32.632756+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.632756+0000","objects_scrubbed":6652,"log_size":5579,"log_dups_size":0,"ondisk_log_size":5579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2683,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130940928,"num_objects":1998,"num_object_clones":0,"num_object_copies":5994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13604,"num_write_kb":499264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,3],"acting":[2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":596,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.822063+0000","last_change":"2024-09-20T22:17:54.588617+0000","last_active":"2024-09-20T22:18:02.822063+0000","last_peered":"2024-09-20T22:18:02.822063+0000","last_clean":"2024-09-20T22:18:02.822063+0000","last_became_active":"2024-09-20T22:17:40.139875+0000","last_became_peered":"2024-09-20T22:17:40.139875+0000","last_unstale":"2024-09-20T22:18:02.822063+0000","last_undegraded":"2024-09-20T22:18:02.822063+0000","last_fullsized":"2024-09-20T22:18:02.822063+0000","mapping_epoch":358,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:54.588504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:54.588504+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:10.848347+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"328'31","reported_seq":871,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.630970+0000","last_change":"2024-09-20T22:16:17.388362+0000","last_active":"2024-09-20T22:18:04.630970+0000","last_peered":"2024-09-20T22:18:04.630970+0000","last_clean":"2024-09-20T22:18:04.630970+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:18:04.630970+0000","last_undegraded":"2024-09-20T22:18:04.630970+0000","last_fullsized":"2024-09-20T22:18:04.630970+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'31","last_scrub_stamp":"2024-09-20T22:16:17.387895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:17.387895+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":831,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.823494+0000","last_change":"2024-09-20T22:17:08.847011+0000","last_active":"2024-09-20T22:18:02.823494+0000","last_peered":"2024-09-20T22:18:02.823494+0000","last_clean":"2024-09-20T22:18:02.823494+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:18:02.823494+0000","last_undegraded":"2024-09-20T22:18:02.823494+0000","last_fullsized":"2024-09-20T22:18:02.823494+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:08.846904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:17:08.846904+0000","last_clean_scrub_stamp":"2024-09-20T22:17:08.846904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:23.342530+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":656,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.822661+0000","last_change":"2024-09-20T22:15:57.442451+0000","last_active":"2024-09-20T22:18:02.822661+0000","last_peered":"2024-09-20T22:18:02.822661+0000","last_clean":"2024-09-20T22:18:02.822661+0000","last_became_active":"2024-09-20T22:12:52.520612+0000","last_became_peered":"2024-09-20T22:12:52.520612+0000","last_unstale":"2024-09-20T22:18:02.822661+0000","last_undegraded":"2024-09-20T22:18:02.822661+0000","last_fullsized":"2024-09-20T22:18:02.822661+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:57.442341+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.442341+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":828,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.624850+0000","last_change":"2024-09-20T22:17:39.808194+0000","last_active":"2024-09-20T22:18:04.624850+0000","last_peered":"2024-09-20T22:18:04.624850+0000","last_clean":"2024-09-20T22:18:04.624850+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:18:04.624850+0000","last_undegraded":"2024-09-20T22:18:04.624850+0000","last_fullsized":"2024-09-20T22:18:04.624850+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:39.807978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:39.807978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:04.536384+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2","version":"439'13360","reported_seq":13617,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:03.449093+0000","last_change":"2024-09-20T22:17:31.558631+0000","last_active":"2024-09-20T22:18:03.449093+0000","last_peered":"2024-09-20T22:18:03.449093+0000","last_clean":"2024-09-20T22:18:03.449093+0000","last_became_active":"2024-09-20T22:15:08.926083+0000","last_became_peered":"2024-09-20T22:15:08.926083+0000","last_unstale":"2024-09-20T22:18:03.449093+0000","last_undegraded":"2024-09-20T22:18:03.449093+0000","last_fullsized":"2024-09-20T22:18:03.449093+0000","mapping_epoch":370,"log_start":"416'9026","ondisk_log_start":"416'9026","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"425'10955","last_scrub_stamp":"2024-09-20T22:17:31.558438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:17:31.558438+0000","objects_scrubbed":4625,"log_size":4334,"log_dups_size":0,"ondisk_log_size":4334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:57.748074+0000","scrub_duration":1227,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142868480,"num_objects":2180,"num_object_clones":0,"num_object_copies":6540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13360,"num_write_kb":497280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":1109,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.634983+0000","last_change":"2024-09-20T22:17:42.025688+0000","last_active":"2024-09-20T22:18:04.634983+0000","last_peered":"2024-09-20T22:18:04.634983+0000","last_clean":"2024-09-20T22:18:04.634983+0000","last_became_active":"2024-09-20T22:17:40.146794+0000","last_became_peered":"2024-09-20T22:17:40.146794+0000","last_unstale":"2024-09-20T22:18:04.634983+0000","last_undegraded":"2024-09-20T22:18:04.634983+0000","last_fullsized":"2024-09-20T22:18:04.634983+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:42.025576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:42.025576+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:43.844172+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"328'25","reported_seq":848,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:05.300918+0000","last_change":"2024-09-20T22:18:05.300953+0000","last_active":"2024-09-20T22:18:05.300918+0000","last_peered":"2024-09-20T22:18:05.300918+0000","last_clean":"2024-09-20T22:18:05.300918+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:18:05.300918+0000","last_undegraded":"2024-09-20T22:18:05.300918+0000","last_fullsized":"2024-09-20T22:18:05.300918+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'25","last_scrub_stamp":"2024-09-20T22:18:05.300862+0000","last_deep_scrub":"328'25","last_deep_scrub_stamp":"2024-09-20T22:18:05.300862+0000","last_clean_scrub_stamp":"2024-09-20T22:18:05.300862+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:35.276486+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":269,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.620642+0000","last_change":"2024-09-20T22:17:35.054489+0000","last_active":"2024-09-20T22:18:01.620642+0000","last_peered":"2024-09-20T22:18:01.620642+0000","last_clean":"2024-09-20T22:18:01.620642+0000","last_became_active":"2024-09-20T22:12:52.528310+0000","last_became_peered":"2024-09-20T22:12:52.528310+0000","last_unstale":"2024-09-20T22:18:01.620642+0000","last_undegraded":"2024-09-20T22:18:01.620642+0000","last_fullsized":"2024-09-20T22:18:01.620642+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:17:35.054116+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:17:35.054116+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:54.869547+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"283'11","reported_seq":1053,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.624790+0000","last_change":"2024-09-20T22:17:31.773328+0000","last_active":"2024-09-20T22:18:04.624790+0000","last_peered":"2024-09-20T22:18:04.624790+0000","last_clean":"2024-09-20T22:18:04.624790+0000","last_became_active":"2024-09-20T22:11:30.485356+0000","last_became_peered":"2024-09-20T22:11:30.485356+0000","last_unstale":"2024-09-20T22:18:04.624790+0000","last_undegraded":"2024-09-20T22:18:04.624790+0000","last_fullsized":"2024-09-20T22:18:04.624790+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'11","last_scrub_stamp":"2024-09-20T22:17:31.773028+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:17:31.773028+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:32.243713+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":806,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.631544+0000","last_change":"2024-09-20T22:16:27.486633+0000","last_active":"2024-09-20T22:18:04.631544+0000","last_peered":"2024-09-20T22:18:04.631544+0000","last_clean":"2024-09-20T22:18:04.631544+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:18:04.631544+0000","last_undegraded":"2024-09-20T22:18:04.631544+0000","last_fullsized":"2024-09-20T22:18:04.631544+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:27.486520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:16:27.486520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"75'128","reported_seq":715,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.631901+0000","last_change":"2024-09-20T22:17:41.370648+0000","last_active":"2024-09-20T22:18:04.631901+0000","last_peered":"2024-09-20T22:18:04.631901+0000","last_clean":"2024-09-20T22:18:04.631901+0000","last_became_active":"2024-09-20T22:17:40.140779+0000","last_became_peered":"2024-09-20T22:17:40.140779+0000","last_unstale":"2024-09-20T22:18:04.631901+0000","last_undegraded":"2024-09-20T22:18:04.631901+0000","last_fullsized":"2024-09-20T22:18:04.631901+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:41.370226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:41.370226+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:41.629450+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.5","version":"440'13710","reported_seq":13961,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:06.466835+0000","last_change":"2024-09-20T22:18:00.864106+0000","last_active":"2024-09-20T22:18:06.466835+0000","last_peered":"2024-09-20T22:18:06.466835+0000","last_clean":"2024-09-20T22:18:06.466835+0000","last_became_active":"2024-09-20T22:15:09.087830+0000","last_became_peered":"2024-09-20T22:15:09.087830+0000","last_unstale":"2024-09-20T22:18:06.466835+0000","last_undegraded":"2024-09-20T22:18:06.466835+0000","last_fullsized":"2024-09-20T22:18:06.466835+0000","mapping_epoch":370,"log_start":"412'8111","ondisk_log_start":"412'8111","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"437'13291","last_scrub_stamp":"2024-09-20T22:18:00.863455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:18:00.863455+0000","objects_scrubbed":2432,"log_size":5599,"log_dups_size":0,"ondisk_log_size":5599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:01.295587+0000","scrub_duration":902,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130154496,"num_objects":1986,"num_object_clones":0,"num_object_copies":5958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13710,"num_write_kb":502272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,3],"acting":[2,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"328'15","reported_seq":284,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.620846+0000","last_change":"2024-09-20T22:16:59.741092+0000","last_active":"2024-09-20T22:18:01.620846+0000","last_peered":"2024-09-20T22:18:01.620846+0000","last_clean":"2024-09-20T22:18:01.620846+0000","last_became_active":"2024-09-20T22:12:52.527531+0000","last_became_peered":"2024-09-20T22:12:52.527531+0000","last_unstale":"2024-09-20T22:18:01.620846+0000","last_undegraded":"2024-09-20T22:18:01.620846+0000","last_fullsized":"2024-09-20T22:18:01.620846+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:16:59.740962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:59.740962+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:24.664014+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":449,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.619920+0000","last_change":"2024-09-20T22:17:59.659397+0000","last_active":"2024-09-20T22:18:01.619920+0000","last_peered":"2024-09-20T22:18:01.619920+0000","last_clean":"2024-09-20T22:18:01.619920+0000","last_became_active":"2024-09-20T22:10:14.536155+0000","last_became_peered":"2024-09-20T22:10:14.536155+0000","last_unstale":"2024-09-20T22:18:01.619920+0000","last_undegraded":"2024-09-20T22:18:01.619920+0000","last_fullsized":"2024-09-20T22:18:01.619920+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:17:59.659279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:59.659279+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:21.111403+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":833,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.623914+0000","last_change":"2024-09-20T22:16:33.008048+0000","last_active":"2024-09-20T22:18:04.623914+0000","last_peered":"2024-09-20T22:18:04.623914+0000","last_clean":"2024-09-20T22:18:04.623914+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:18:04.623914+0000","last_undegraded":"2024-09-20T22:18:04.623914+0000","last_fullsized":"2024-09-20T22:18:04.623914+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:33.007962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:16:33.007962+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":257,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.618109+0000","last_change":"2024-09-20T22:16:24.020680+0000","last_active":"2024-09-20T22:18:01.618109+0000","last_peered":"2024-09-20T22:18:01.618109+0000","last_clean":"2024-09-20T22:18:01.618109+0000","last_became_active":"2024-09-20T22:12:52.512924+0000","last_became_peered":"2024-09-20T22:12:52.512924+0000","last_unstale":"2024-09-20T22:18:01.618109+0000","last_undegraded":"2024-09-20T22:18:01.618109+0000","last_fullsized":"2024-09-20T22:18:01.618109+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:24.020573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:24.020573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"75'128","reported_seq":868,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.822550+0000","last_change":"2024-09-20T22:17:07.885447+0000","last_active":"2024-09-20T22:18:02.822550+0000","last_peered":"2024-09-20T22:18:02.822550+0000","last_clean":"2024-09-20T22:18:02.822550+0000","last_became_active":"2024-09-20T22:15:50.576540+0000","last_became_peered":"2024-09-20T22:15:50.576540+0000","last_unstale":"2024-09-20T22:18:02.822550+0000","last_undegraded":"2024-09-20T22:18:02.822550+0000","last_fullsized":"2024-09-20T22:18:02.822550+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:07.885346+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:17:07.885346+0000","last_clean_scrub_stamp":"2024-09-20T22:17:07.885346+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:19.927064+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.4","version":"438'13232","reported_seq":13453,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.354505+0000","last_change":"2024-09-20T22:16:31.859517+0000","last_active":"2024-09-20T22:18:02.354505+0000","last_peered":"2024-09-20T22:18:02.354505+0000","last_clean":"2024-09-20T22:18:02.354505+0000","last_became_active":"2024-09-20T22:15:09.087298+0000","last_became_peered":"2024-09-20T22:15:09.087298+0000","last_unstale":"2024-09-20T22:18:02.354505+0000","last_undegraded":"2024-09-20T22:18:02.354505+0000","last_fullsized":"2024-09-20T22:18:02.354505+0000","mapping_epoch":370,"log_start":"413'8470","ondisk_log_start":"413'8470","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6582","last_scrub_stamp":"2024-09-20T22:16:31.859305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.859305+0000","objects_scrubbed":6495,"log_size":4762,"log_dups_size":0,"ondisk_log_size":4762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2899,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154140672,"num_objects":2352,"num_object_clones":0,"num_object_copies":7056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13232,"num_write_kb":498688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.a","version":"328'35","reported_seq":926,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.823065+0000","last_change":"2024-09-20T22:17:03.808976+0000","last_active":"2024-09-20T22:18:02.823065+0000","last_peered":"2024-09-20T22:18:02.823065+0000","last_clean":"2024-09-20T22:18:02.823065+0000","last_became_active":"2024-09-20T22:10:30.652967+0000","last_became_peered":"2024-09-20T22:10:30.652967+0000","last_unstale":"2024-09-20T22:18:02.823065+0000","last_undegraded":"2024-09-20T22:18:02.823065+0000","last_fullsized":"2024-09-20T22:18:02.823065+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'35","last_scrub_stamp":"2024-09-20T22:17:03.808887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:03.808887+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:27.160343+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":829,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.634146+0000","last_change":"2024-09-20T22:17:37.950739+0000","last_active":"2024-09-20T22:18:04.634146+0000","last_peered":"2024-09-20T22:18:04.634146+0000","last_clean":"2024-09-20T22:18:04.634146+0000","last_became_active":"2024-09-20T22:11:30.488650+0000","last_became_peered":"2024-09-20T22:11:30.488650+0000","last_unstale":"2024-09-20T22:18:04.634146+0000","last_undegraded":"2024-09-20T22:18:04.634146+0000","last_fullsized":"2024-09-20T22:18:04.634146+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:37.950674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:12:46.602035+0000","last_clean_scrub_stamp":"2024-09-20T22:17:37.950674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:59.016065+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":856,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.624275+0000","last_change":"2024-09-20T22:16:18.795310+0000","last_active":"2024-09-20T22:18:04.624275+0000","last_peered":"2024-09-20T22:18:04.624275+0000","last_clean":"2024-09-20T22:18:04.624275+0000","last_became_active":"2024-09-20T22:12:52.515295+0000","last_became_peered":"2024-09-20T22:12:52.515295+0000","last_unstale":"2024-09-20T22:18:04.624275+0000","last_undegraded":"2024-09-20T22:18:04.624275+0000","last_fullsized":"2024-09-20T22:18:04.624275+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:18.795079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_clean_scrub_stamp":"2024-09-20T22:16:18.795079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":414,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.635454+0000","last_change":"2024-09-20T22:16:35.739421+0000","last_active":"2024-09-20T22:18:04.635454+0000","last_peered":"2024-09-20T22:18:04.635454+0000","last_clean":"2024-09-20T22:18:04.635454+0000","last_became_active":"2024-09-20T22:10:30.657116+0000","last_became_peered":"2024-09-20T22:10:30.657116+0000","last_unstale":"2024-09-20T22:18:04.635454+0000","last_undegraded":"2024-09-20T22:18:04.635454+0000","last_fullsized":"2024-09-20T22:18:04.635454+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:35.739168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:35.739168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"75'128","reported_seq":890,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.619486+0000","last_change":"2024-09-20T22:17:40.737208+0000","last_active":"2024-09-20T22:18:01.619486+0000","last_peered":"2024-09-20T22:18:01.619486+0000","last_clean":"2024-09-20T22:18:01.619486+0000","last_became_active":"2024-09-20T22:17:40.152364+0000","last_became_peered":"2024-09-20T22:17:40.152364+0000","last_unstale":"2024-09-20T22:18:01.619486+0000","last_undegraded":"2024-09-20T22:18:01.619486+0000","last_fullsized":"2024-09-20T22:18:01.619486+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:40.737088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:40.737088+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:46.563168+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.7","version":"439'13388","reported_seq":13612,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:03.175274+0000","last_change":"2024-09-20T22:16:29.896220+0000","last_active":"2024-09-20T22:18:03.175274+0000","last_peered":"2024-09-20T22:18:03.175274+0000","last_clean":"2024-09-20T22:18:03.175274+0000","last_became_active":"2024-09-20T22:15:09.088590+0000","last_became_peered":"2024-09-20T22:15:09.088590+0000","last_unstale":"2024-09-20T22:18:03.175274+0000","last_undegraded":"2024-09-20T22:18:03.175274+0000","last_fullsized":"2024-09-20T22:18:03.175274+0000","mapping_epoch":370,"log_start":"414'8745","ondisk_log_start":"414'8745","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'6589","last_scrub_stamp":"2024-09-20T22:16:29.896110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:16:29.896110+0000","objects_scrubbed":6527,"log_size":4643,"log_dups_size":0,"ondisk_log_size":4643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"queued for scrub","scrub_duration":2191,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145752064,"num_objects":2224,"num_object_clones":0,"num_object_copies":6672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13388,"num_write_kb":499584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"418'23","reported_seq":852,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.632097+0000","last_change":"2024-09-20T22:17:11.339243+0000","last_active":"2024-09-20T22:18:04.632097+0000","last_peered":"2024-09-20T22:18:04.632097+0000","last_clean":"2024-09-20T22:18:04.632097+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:18:04.632097+0000","last_undegraded":"2024-09-20T22:18:04.632097+0000","last_fullsized":"2024-09-20T22:18:04.632097+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"418'23","last_scrub_stamp":"2024-09-20T22:17:11.338821+0000","last_deep_scrub":"418'23","last_deep_scrub_stamp":"2024-09-20T22:17:11.338821+0000","last_clean_scrub_stamp":"2024-09-20T22:17:11.338821+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:25.121498+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":818,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.634288+0000","last_change":"2024-09-20T22:16:25.649600+0000","last_active":"2024-09-20T22:18:04.634288+0000","last_peered":"2024-09-20T22:18:04.634288+0000","last_clean":"2024-09-20T22:18:04.634288+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:18:04.634288+0000","last_undegraded":"2024-09-20T22:18:04.634288+0000","last_fullsized":"2024-09-20T22:18:04.634288+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:25.649511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:25.649511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":854,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.624758+0000","last_change":"2024-09-20T22:16:59.037833+0000","last_active":"2024-09-20T22:18:04.624758+0000","last_peered":"2024-09-20T22:18:04.624758+0000","last_clean":"2024-09-20T22:18:04.624758+0000","last_became_active":"2024-09-20T22:12:52.514202+0000","last_became_peered":"2024-09-20T22:12:52.514202+0000","last_unstale":"2024-09-20T22:18:04.624758+0000","last_undegraded":"2024-09-20T22:18:04.624758+0000","last_fullsized":"2024-09-20T22:18:04.624758+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:59.036713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:59.036713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:10.054093+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":275,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.617030+0000","last_change":"2024-09-20T22:16:57.795001+0000","last_active":"2024-09-20T22:18:01.617030+0000","last_peered":"2024-09-20T22:18:01.617030+0000","last_clean":"2024-09-20T22:18:01.617030+0000","last_became_active":"2024-09-20T22:12:44.638546+0000","last_became_peered":"2024-09-20T22:12:44.638546+0000","last_unstale":"2024-09-20T22:18:01.617030+0000","last_undegraded":"2024-09-20T22:18:01.617030+0000","last_fullsized":"2024-09-20T22:18:01.617030+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:57.794891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:57.794891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:26.253847+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'128","reported_seq":431,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.633379+0000","last_change":"2024-09-20T22:17:40.996913+0000","last_active":"2024-09-20T22:18:04.633379+0000","last_peered":"2024-09-20T22:18:04.633379+0000","last_clean":"2024-09-20T22:18:04.633379+0000","last_became_active":"2024-09-20T22:17:40.136632+0000","last_became_peered":"2024-09-20T22:17:40.136632+0000","last_unstale":"2024-09-20T22:18:04.633379+0000","last_undegraded":"2024-09-20T22:18:04.633379+0000","last_fullsized":"2024-09-20T22:18:04.633379+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:40.996791+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:17:40.996791+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:10.524269+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.6","version":"440'13813","reported_seq":14019,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:06.318180+0000","last_change":"2024-09-20T22:17:32.677989+0000","last_active":"2024-09-20T22:18:06.318180+0000","last_peered":"2024-09-20T22:18:06.318180+0000","last_clean":"2024-09-20T22:18:06.318180+0000","last_became_active":"2024-09-20T22:15:08.928102+0000","last_became_peered":"2024-09-20T22:15:08.928102+0000","last_unstale":"2024-09-20T22:18:06.318180+0000","last_undegraded":"2024-09-20T22:18:06.318180+0000","last_fullsized":"2024-09-20T22:18:06.318180+0000","mapping_epoch":370,"log_start":"416'9238","ondisk_log_start":"416'9238","created":370,"last_epoch_clean":371,"parent":"0.0","parent_split_bits":0,"last_scrub":"425'11140","last_scrub_stamp":"2024-09-20T22:17:32.677655+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:07.911431+0000","last_clean_scrub_stamp":"2024-09-20T22:17:32.677655+0000","objects_scrubbed":4689,"log_size":4575,"log_dups_size":0,"ondisk_log_size":4575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:32.759353+0000","scrub_duration":1485,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128909312,"num_objects":1967,"num_object_clones":0,"num_object_copies":5901,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13813,"num_write_kb":504960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,4],"acting":[5,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.8","version":"416'34","reported_seq":385,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.620243+0000","last_change":"2024-09-20T22:17:04.960544+0000","last_active":"2024-09-20T22:18:04.620243+0000","last_peered":"2024-09-20T22:18:04.620243+0000","last_clean":"2024-09-20T22:18:04.620243+0000","last_became_active":"2024-09-20T22:11:30.493275+0000","last_became_peered":"2024-09-20T22:11:30.493275+0000","last_unstale":"2024-09-20T22:18:04.620243+0000","last_undegraded":"2024-09-20T22:18:04.620243+0000","last_fullsized":"2024-09-20T22:18:04.620243+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"416'34","last_scrub_stamp":"2024-09-20T22:17:04.960380+0000","last_deep_scrub":"416'34","last_deep_scrub_stamp":"2024-09-20T22:17:04.960380+0000","last_clean_scrub_stamp":"2024-09-20T22:17:04.960380+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:30.318376+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":444,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.634439+0000","last_change":"2024-09-20T22:16:37.713822+0000","last_active":"2024-09-20T22:18:04.634439+0000","last_peered":"2024-09-20T22:18:04.634439+0000","last_clean":"2024-09-20T22:18:04.634439+0000","last_became_active":"2024-09-20T22:12:52.526551+0000","last_became_peered":"2024-09-20T22:12:52.526551+0000","last_unstale":"2024-09-20T22:18:04.634439+0000","last_undegraded":"2024-09-20T22:18:04.634439+0000","last_fullsized":"2024-09-20T22:18:04.634439+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:16:37.713713+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:16:37.713713+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":994,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:05.361240+0000","last_change":"2024-09-20T22:16:32.744150+0000","last_active":"2024-09-20T22:18:05.361240+0000","last_peered":"2024-09-20T22:18:05.361240+0000","last_clean":"2024-09-20T22:18:05.361240+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:18:05.361240+0000","last_undegraded":"2024-09-20T22:18:05.361240+0000","last_fullsized":"2024-09-20T22:18:05.361240+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:16:32.744031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.744031+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":356,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.621154+0000","last_change":"2024-09-20T22:16:21.904789+0000","last_active":"2024-09-20T22:18:04.621154+0000","last_peered":"2024-09-20T22:18:04.621154+0000","last_clean":"2024-09-20T22:18:04.621154+0000","last_became_active":"2024-09-20T22:12:52.517758+0000","last_became_peered":"2024-09-20T22:12:52.517758+0000","last_unstale":"2024-09-20T22:18:04.621154+0000","last_undegraded":"2024-09-20T22:18:04.621154+0000","last_fullsized":"2024-09-20T22:18:04.621154+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:21.904548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:21.904548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"75'128","reported_seq":694,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.620100+0000","last_change":"2024-09-20T22:15:52.087940+0000","last_active":"2024-09-20T22:18:04.620100+0000","last_peered":"2024-09-20T22:18:04.620100+0000","last_clean":"2024-09-20T22:18:04.620100+0000","last_became_active":"2024-09-20T22:15:50.622954+0000","last_became_peered":"2024-09-20T22:15:50.622954+0000","last_unstale":"2024-09-20T22:18:04.620100+0000","last_undegraded":"2024-09-20T22:18:04.620100+0000","last_fullsized":"2024-09-20T22:18:04.620100+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:52.087707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:52.087707+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":862,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.823628+0000","last_change":"2024-09-20T22:18:00.786710+0000","last_active":"2024-09-20T22:18:02.823628+0000","last_peered":"2024-09-20T22:18:02.823628+0000","last_clean":"2024-09-20T22:18:02.823628+0000","last_became_active":"2024-09-20T22:12:52.516989+0000","last_became_peered":"2024-09-20T22:12:52.516989+0000","last_unstale":"2024-09-20T22:18:02.823628+0000","last_undegraded":"2024-09-20T22:18:02.823628+0000","last_fullsized":"2024-09-20T22:18:02.823628+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:00.786604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:00.786604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:16.514903+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"328'12","reported_seq":854,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.621629+0000","last_change":"2024-09-20T22:16:22.641814+0000","last_active":"2024-09-20T22:18:01.621629+0000","last_peered":"2024-09-20T22:18:01.621629+0000","last_clean":"2024-09-20T22:18:01.621629+0000","last_became_active":"2024-09-20T22:10:48.922561+0000","last_became_peered":"2024-09-20T22:10:48.922561+0000","last_unstale":"2024-09-20T22:18:01.621629+0000","last_undegraded":"2024-09-20T22:18:01.621629+0000","last_fullsized":"2024-09-20T22:18:01.621629+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'12","last_scrub_stamp":"2024-09-20T22:16:22.641558+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:22.641558+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":343,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.621796+0000","last_change":"2024-09-20T22:15:54.065141+0000","last_active":"2024-09-20T22:18:04.621796+0000","last_peered":"2024-09-20T22:18:04.621796+0000","last_clean":"2024-09-20T22:18:04.621796+0000","last_became_active":"2024-09-20T22:11:30.487322+0000","last_became_peered":"2024-09-20T22:11:30.487322+0000","last_unstale":"2024-09-20T22:18:04.621796+0000","last_undegraded":"2024-09-20T22:18:04.621796+0000","last_fullsized":"2024-09-20T22:18:04.621796+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:54.065026+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:15:54.065026+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":797,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.633789+0000","last_change":"2024-09-20T22:17:02.792431+0000","last_active":"2024-09-20T22:18:04.633789+0000","last_peered":"2024-09-20T22:18:04.633789+0000","last_clean":"2024-09-20T22:18:04.633789+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:18:04.633789+0000","last_undegraded":"2024-09-20T22:18:04.633789+0000","last_fullsized":"2024-09-20T22:18:04.633789+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:02.792356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:54.359586+0000","last_clean_scrub_stamp":"2024-09-20T22:17:02.792356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"75'128","reported_seq":828,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.815939+0000","last_change":"2024-09-20T22:15:52.362195+0000","last_active":"2024-09-20T22:18:02.815939+0000","last_peered":"2024-09-20T22:18:02.815939+0000","last_clean":"2024-09-20T22:18:02.815939+0000","last_became_active":"2024-09-20T22:15:50.615607+0000","last_became_peered":"2024-09-20T22:15:50.615607+0000","last_unstale":"2024-09-20T22:18:02.815939+0000","last_undegraded":"2024-09-20T22:18:02.815939+0000","last_fullsized":"2024-09-20T22:18:02.815939+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:52.362111+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:15:52.362111+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"328'15","reported_seq":284,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.617574+0000","last_change":"2024-09-20T22:17:39.086388+0000","last_active":"2024-09-20T22:18:01.617574+0000","last_peered":"2024-09-20T22:18:01.617574+0000","last_clean":"2024-09-20T22:18:01.617574+0000","last_became_active":"2024-09-20T22:12:52.527106+0000","last_became_peered":"2024-09-20T22:12:52.527106+0000","last_unstale":"2024-09-20T22:18:01.617574+0000","last_undegraded":"2024-09-20T22:18:01.617574+0000","last_fullsized":"2024-09-20T22:18:01.617574+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:17:39.086307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:39.086307+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:39.449906+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":817,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.635111+0000","last_change":"2024-09-20T22:17:10.768641+0000","last_active":"2024-09-20T22:18:04.635111+0000","last_peered":"2024-09-20T22:18:04.635111+0000","last_clean":"2024-09-20T22:18:04.635111+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:18:04.635111+0000","last_undegraded":"2024-09-20T22:18:04.635111+0000","last_fullsized":"2024-09-20T22:18:04.635111+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:10.768513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:10.768513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:36.496048+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":445,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.824420+0000","last_change":"2024-09-20T22:16:17.270484+0000","last_active":"2024-09-20T22:18:02.824420+0000","last_peered":"2024-09-20T22:18:02.824420+0000","last_clean":"2024-09-20T22:18:02.824420+0000","last_became_active":"2024-09-20T22:12:33.545506+0000","last_became_peered":"2024-09-20T22:12:33.545506+0000","last_unstale":"2024-09-20T22:18:02.824420+0000","last_undegraded":"2024-09-20T22:18:02.824420+0000","last_fullsized":"2024-09-20T22:18:02.824420+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:17.270376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_clean_scrub_stamp":"2024-09-20T22:16:17.270376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":836,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.624626+0000","last_change":"2024-09-20T22:18:03.970869+0000","last_active":"2024-09-20T22:18:04.624626+0000","last_peered":"2024-09-20T22:18:04.624626+0000","last_clean":"2024-09-20T22:18:04.624626+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:18:04.624626+0000","last_undegraded":"2024-09-20T22:18:04.624626+0000","last_fullsized":"2024-09-20T22:18:04.624626+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:03.970735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:26.862279+0000","last_clean_scrub_stamp":"2024-09-20T22:18:03.970735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:26.723590+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"75'128","reported_seq":694,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.619949+0000","last_change":"2024-09-20T22:15:51.068931+0000","last_active":"2024-09-20T22:18:04.619949+0000","last_peered":"2024-09-20T22:18:04.619949+0000","last_clean":"2024-09-20T22:18:04.619949+0000","last_became_active":"2024-09-20T22:15:50.574865+0000","last_became_peered":"2024-09-20T22:15:50.574865+0000","last_unstale":"2024-09-20T22:18:04.619949+0000","last_undegraded":"2024-09-20T22:18:04.619949+0000","last_fullsized":"2024-09-20T22:18:04.619949+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:51.068807+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:51.068807+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"328'15","reported_seq":362,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.621079+0000","last_change":"2024-09-20T22:15:57.097701+0000","last_active":"2024-09-20T22:18:04.621079+0000","last_peered":"2024-09-20T22:18:04.621079+0000","last_clean":"2024-09-20T22:18:04.621079+0000","last_became_active":"2024-09-20T22:11:30.493335+0000","last_became_peered":"2024-09-20T22:11:30.493335+0000","last_unstale":"2024-09-20T22:18:04.621079+0000","last_undegraded":"2024-09-20T22:18:04.621079+0000","last_fullsized":"2024-09-20T22:18:04.621079+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:15:57.097563+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:15:57.097563+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":265,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.622447+0000","last_change":"2024-09-20T22:16:27.938805+0000","last_active":"2024-09-20T22:18:01.622447+0000","last_peered":"2024-09-20T22:18:01.622447+0000","last_clean":"2024-09-20T22:18:01.622447+0000","last_became_active":"2024-09-20T22:12:52.524149+0000","last_became_peered":"2024-09-20T22:12:52.524149+0000","last_unstale":"2024-09-20T22:18:01.622447+0000","last_undegraded":"2024-09-20T22:18:01.622447+0000","last_fullsized":"2024-09-20T22:18:01.622447+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:27.938677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:56.538825+0000","last_clean_scrub_stamp":"2024-09-20T22:16:27.938677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"315'11","reported_seq":380,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.633356+0000","last_change":"2024-09-20T22:17:05.784312+0000","last_active":"2024-09-20T22:18:04.633356+0000","last_peered":"2024-09-20T22:18:04.633356+0000","last_clean":"2024-09-20T22:18:04.633356+0000","last_became_active":"2024-09-20T22:12:33.544098+0000","last_became_peered":"2024-09-20T22:12:33.544098+0000","last_unstale":"2024-09-20T22:18:04.633356+0000","last_undegraded":"2024-09-20T22:18:04.633356+0000","last_fullsized":"2024-09-20T22:18:04.633356+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:17:05.784212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:05.784212+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:31.462376+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":817,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.630408+0000","last_change":"2024-09-20T22:18:03.323718+0000","last_active":"2024-09-20T22:18:04.630408+0000","last_peered":"2024-09-20T22:18:04.630408+0000","last_clean":"2024-09-20T22:18:04.630408+0000","last_became_active":"2024-09-20T22:11:30.484731+0000","last_became_peered":"2024-09-20T22:11:30.484731+0000","last_unstale":"2024-09-20T22:18:04.630408+0000","last_undegraded":"2024-09-20T22:18:04.630408+0000","last_fullsized":"2024-09-20T22:18:04.630408+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:03.323607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:03.323607+0000","last_clean_scrub_stamp":"2024-09-20T22:18:03.323607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:15.835890+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"75'128","reported_seq":714,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.616380+0000","last_change":"2024-09-20T22:17:55.057664+0000","last_active":"2024-09-20T22:18:01.616380+0000","last_peered":"2024-09-20T22:18:01.616380+0000","last_clean":"2024-09-20T22:18:01.616380+0000","last_became_active":"2024-09-20T22:15:50.612544+0000","last_became_peered":"2024-09-20T22:15:50.612544+0000","last_unstale":"2024-09-20T22:18:01.616380+0000","last_undegraded":"2024-09-20T22:18:01.616380+0000","last_fullsized":"2024-09-20T22:18:01.616380+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:55.057543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:55.057543+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:58.102108+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"328'20","reported_seq":296,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.816105+0000","last_change":"2024-09-20T22:16:37.280708+0000","last_active":"2024-09-20T22:18:02.816105+0000","last_peered":"2024-09-20T22:18:02.816105+0000","last_clean":"2024-09-20T22:18:02.816105+0000","last_became_active":"2024-09-20T22:12:44.631986+0000","last_became_peered":"2024-09-20T22:12:44.631986+0000","last_unstale":"2024-09-20T22:18:02.816105+0000","last_undegraded":"2024-09-20T22:18:02.816105+0000","last_fullsized":"2024-09-20T22:18:02.816105+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:16:37.280566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:37.280566+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:04.676364+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":809,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.632843+0000","last_change":"2024-09-20T22:16:31.755794+0000","last_active":"2024-09-20T22:18:04.632843+0000","last_peered":"2024-09-20T22:18:04.632843+0000","last_clean":"2024-09-20T22:18:04.632843+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:18:04.632843+0000","last_undegraded":"2024-09-20T22:18:04.632843+0000","last_fullsized":"2024-09-20T22:18:04.632843+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:31.755589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.755589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"326'14","reported_seq":455,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.621040+0000","last_change":"2024-09-20T22:16:55.917269+0000","last_active":"2024-09-20T22:18:04.621040+0000","last_peered":"2024-09-20T22:18:04.621040+0000","last_clean":"2024-09-20T22:18:04.621040+0000","last_became_active":"2024-09-20T22:12:52.518770+0000","last_became_peered":"2024-09-20T22:12:52.518770+0000","last_unstale":"2024-09-20T22:18:04.621040+0000","last_undegraded":"2024-09-20T22:18:04.621040+0000","last_fullsized":"2024-09-20T22:18:04.621040+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'14","last_scrub_stamp":"2024-09-20T22:16:55.917159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:55.917159+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:20.333421+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":265,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.617173+0000","last_change":"2024-09-20T22:17:06.828671+0000","last_active":"2024-09-20T22:18:01.617173+0000","last_peered":"2024-09-20T22:18:01.617173+0000","last_clean":"2024-09-20T22:18:01.617173+0000","last_became_active":"2024-09-20T22:12:52.523781+0000","last_became_peered":"2024-09-20T22:12:52.523781+0000","last_unstale":"2024-09-20T22:18:01.617173+0000","last_undegraded":"2024-09-20T22:18:01.617173+0000","last_fullsized":"2024-09-20T22:18:01.617173+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:06.828504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:06.828504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:22.792860+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.16","version":"75'128","reported_seq":607,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.822383+0000","last_change":"2024-09-20T22:17:56.147181+0000","last_active":"2024-09-20T22:18:02.822383+0000","last_peered":"2024-09-20T22:18:02.822383+0000","last_clean":"2024-09-20T22:18:02.822383+0000","last_became_active":"2024-09-20T22:17:44.621596+0000","last_became_peered":"2024-09-20T22:17:44.621596+0000","last_unstale":"2024-09-20T22:18:02.822383+0000","last_undegraded":"2024-09-20T22:18:02.822383+0000","last_fullsized":"2024-09-20T22:18:02.822383+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.147063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.147063+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:02.629362+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"328'19","reported_seq":865,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.630372+0000","last_change":"2024-09-20T22:16:28.533330+0000","last_active":"2024-09-20T22:18:04.630372+0000","last_peered":"2024-09-20T22:18:04.630372+0000","last_clean":"2024-09-20T22:18:04.630372+0000","last_became_active":"2024-09-20T22:08:22.139581+0000","last_became_peered":"2024-09-20T22:08:22.139581+0000","last_unstale":"2024-09-20T22:18:04.630372+0000","last_undegraded":"2024-09-20T22:18:04.630372+0000","last_fullsized":"2024-09-20T22:18:04.630372+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:16:28.533201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:28.533201+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":817,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.634521+0000","last_change":"2024-09-20T22:16:56.523856+0000","last_active":"2024-09-20T22:18:04.634521+0000","last_peered":"2024-09-20T22:18:04.634521+0000","last_clean":"2024-09-20T22:18:04.634521+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:18:04.634521+0000","last_undegraded":"2024-09-20T22:18:04.634521+0000","last_fullsized":"2024-09-20T22:18:04.634521+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:56.523743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:56.523743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":643,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.823128+0000","last_change":"2024-09-20T22:16:32.216071+0000","last_active":"2024-09-20T22:18:02.823128+0000","last_peered":"2024-09-20T22:18:02.823128+0000","last_clean":"2024-09-20T22:18:02.823128+0000","last_became_active":"2024-09-20T22:06:54.853551+0000","last_became_peered":"2024-09-20T22:06:54.853551+0000","last_unstale":"2024-09-20T22:18:02.823128+0000","last_undegraded":"2024-09-20T22:18:02.823128+0000","last_fullsized":"2024-09-20T22:18:02.823128+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:32.215758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.215758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":654,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.815617+0000","last_change":"2024-09-20T22:16:59.256702+0000","last_active":"2024-09-20T22:18:02.815617+0000","last_peered":"2024-09-20T22:18:02.815617+0000","last_clean":"2024-09-20T22:18:02.815617+0000","last_became_active":"2024-09-20T22:12:52.521497+0000","last_became_peered":"2024-09-20T22:12:52.521497+0000","last_unstale":"2024-09-20T22:18:02.815617+0000","last_undegraded":"2024-09-20T22:18:02.815617+0000","last_fullsized":"2024-09-20T22:18:02.815617+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:59.256587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:59.256587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:18.828736+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.17","version":"75'128","reported_seq":325,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.621888+0000","last_change":"2024-09-20T22:17:57.421052+0000","last_active":"2024-09-20T22:18:04.621888+0000","last_peered":"2024-09-20T22:18:04.621888+0000","last_clean":"2024-09-20T22:18:04.621888+0000","last_became_active":"2024-09-20T22:17:44.721755+0000","last_became_peered":"2024-09-20T22:17:44.721755+0000","last_unstale":"2024-09-20T22:18:04.621888+0000","last_undegraded":"2024-09-20T22:18:04.621888+0000","last_fullsized":"2024-09-20T22:18:04.621888+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:57.420935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.420935+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:14.038998+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"328'19","reported_seq":728,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.827278+0000","last_change":"2024-09-20T22:17:03.305508+0000","last_active":"2024-09-20T22:18:02.827278+0000","last_peered":"2024-09-20T22:18:02.827278+0000","last_clean":"2024-09-20T22:18:02.827278+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:18:02.827278+0000","last_undegraded":"2024-09-20T22:18:02.827278+0000","last_fullsized":"2024-09-20T22:18:02.827278+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:17:03.305256+0000","last_deep_scrub":"291'13","last_deep_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_clean_scrub_stamp":"2024-09-20T22:17:03.305256+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:32.849170+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":414,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.633458+0000","last_change":"2024-09-20T22:17:54.900226+0000","last_active":"2024-09-20T22:18:04.633458+0000","last_peered":"2024-09-20T22:18:04.633458+0000","last_clean":"2024-09-20T22:18:04.633458+0000","last_became_active":"2024-09-20T22:10:30.655502+0000","last_became_peered":"2024-09-20T22:10:30.655502+0000","last_unstale":"2024-09-20T22:18:04.633458+0000","last_undegraded":"2024-09-20T22:18:04.633458+0000","last_fullsized":"2024-09-20T22:18:04.633458+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:54.900125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:54.900125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:14.560078+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":265,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.620376+0000","last_change":"2024-09-20T22:17:32.074051+0000","last_active":"2024-09-20T22:18:01.620376+0000","last_peered":"2024-09-20T22:18:01.620376+0000","last_clean":"2024-09-20T22:18:01.620376+0000","last_became_active":"2024-09-20T22:12:52.516953+0000","last_became_peered":"2024-09-20T22:12:52.516953+0000","last_unstale":"2024-09-20T22:18:01.620376+0000","last_undegraded":"2024-09-20T22:18:01.620376+0000","last_fullsized":"2024-09-20T22:18:01.620376+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:32.073941+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:32.073941+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:56.963638+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":265,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.615252+0000","last_change":"2024-09-20T22:17:09.839341+0000","last_active":"2024-09-20T22:18:01.615252+0000","last_peered":"2024-09-20T22:18:01.615252+0000","last_clean":"2024-09-20T22:18:01.615252+0000","last_became_active":"2024-09-20T22:12:52.514247+0000","last_became_peered":"2024-09-20T22:12:52.514247+0000","last_unstale":"2024-09-20T22:18:01.615252+0000","last_undegraded":"2024-09-20T22:18:01.615252+0000","last_fullsized":"2024-09-20T22:18:01.615252+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:09.839241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:09.839241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:29.081256+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"75'128","reported_seq":867,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.624222+0000","last_change":"2024-09-20T22:17:04.909914+0000","last_active":"2024-09-20T22:18:04.624222+0000","last_peered":"2024-09-20T22:18:04.624222+0000","last_clean":"2024-09-20T22:18:04.624222+0000","last_became_active":"2024-09-20T22:15:50.611421+0000","last_became_peered":"2024-09-20T22:15:50.611421+0000","last_unstale":"2024-09-20T22:18:04.624222+0000","last_undegraded":"2024-09-20T22:18:04.624222+0000","last_fullsized":"2024-09-20T22:18:04.624222+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:04.909798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:04.909798+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:18.648668+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"328'19","reported_seq":734,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.634896+0000","last_change":"2024-09-20T22:16:21.625991+0000","last_active":"2024-09-20T22:18:04.634896+0000","last_peered":"2024-09-20T22:18:04.634896+0000","last_clean":"2024-09-20T22:18:04.634896+0000","last_became_active":"2024-09-20T22:12:33.545199+0000","last_became_peered":"2024-09-20T22:12:33.545199+0000","last_unstale":"2024-09-20T22:18:04.634896+0000","last_undegraded":"2024-09-20T22:18:04.634896+0000","last_fullsized":"2024-09-20T22:18:04.634896+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:16:21.625915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:21.625915+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":354,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.621781+0000","last_change":"2024-09-20T22:17:55.398271+0000","last_active":"2024-09-20T22:18:04.621781+0000","last_peered":"2024-09-20T22:18:04.621781+0000","last_clean":"2024-09-20T22:18:04.621781+0000","last_became_active":"2024-09-20T22:11:30.493598+0000","last_became_peered":"2024-09-20T22:11:30.493598+0000","last_unstale":"2024-09-20T22:18:04.621781+0000","last_undegraded":"2024-09-20T22:18:04.621781+0000","last_fullsized":"2024-09-20T22:18:04.621781+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:55.398175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_clean_scrub_stamp":"2024-09-20T22:17:55.398175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:13.951899+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":809,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.635539+0000","last_change":"2024-09-20T22:17:08.799282+0000","last_active":"2024-09-20T22:18:04.635539+0000","last_peered":"2024-09-20T22:18:04.635539+0000","last_clean":"2024-09-20T22:18:04.635539+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:18:04.635539+0000","last_undegraded":"2024-09-20T22:18:04.635539+0000","last_fullsized":"2024-09-20T22:18:04.635539+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:08.798598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:08.798598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:17.552228+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":254,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.620555+0000","last_change":"2024-09-20T22:16:20.113927+0000","last_active":"2024-09-20T22:18:01.620555+0000","last_peered":"2024-09-20T22:18:01.620555+0000","last_clean":"2024-09-20T22:18:01.620555+0000","last_became_active":"2024-09-20T22:12:57.883998+0000","last_became_peered":"2024-09-20T22:12:57.883998+0000","last_unstale":"2024-09-20T22:18:01.620555+0000","last_undegraded":"2024-09-20T22:18:01.620555+0000","last_fullsized":"2024-09-20T22:18:01.620555+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:20.113814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:20.113814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.15","version":"75'128","reported_seq":324,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.633203+0000","last_change":"2024-09-20T22:17:56.859515+0000","last_active":"2024-09-20T22:18:04.633203+0000","last_peered":"2024-09-20T22:18:04.633203+0000","last_clean":"2024-09-20T22:18:04.633203+0000","last_became_active":"2024-09-20T22:17:44.719101+0000","last_became_peered":"2024-09-20T22:17:44.719101+0000","last_unstale":"2024-09-20T22:18:04.633203+0000","last_undegraded":"2024-09-20T22:18:04.633203+0000","last_fullsized":"2024-09-20T22:18:04.633203+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.859373+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.859373+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:07.658533+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"328'8","reported_seq":829,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.617172+0000","last_change":"2024-09-20T22:17:03.661899+0000","last_active":"2024-09-20T22:18:01.617172+0000","last_peered":"2024-09-20T22:18:01.617172+0000","last_clean":"2024-09-20T22:18:01.617172+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:18:01.617172+0000","last_undegraded":"2024-09-20T22:18:01.617172+0000","last_fullsized":"2024-09-20T22:18:01.617172+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'8","last_scrub_stamp":"2024-09-20T22:17:03.661772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:03.661772+0000","objects_scrubbed":2,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:19.537624+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":257,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.616611+0000","last_change":"2024-09-20T22:15:50.069823+0000","last_active":"2024-09-20T22:18:01.616611+0000","last_peered":"2024-09-20T22:18:01.616611+0000","last_clean":"2024-09-20T22:18:01.616611+0000","last_became_active":"2024-09-20T22:12:52.517510+0000","last_became_peered":"2024-09-20T22:12:52.517510+0000","last_unstale":"2024-09-20T22:18:01.616611+0000","last_undegraded":"2024-09-20T22:18:01.616611+0000","last_fullsized":"2024-09-20T22:18:01.616611+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:50.069659+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:15:50.069659+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":830,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.821964+0000","last_change":"2024-09-20T22:16:37.743248+0000","last_active":"2024-09-20T22:18:02.821964+0000","last_peered":"2024-09-20T22:18:02.821964+0000","last_clean":"2024-09-20T22:18:02.821964+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:18:02.821964+0000","last_undegraded":"2024-09-20T22:18:02.821964+0000","last_fullsized":"2024-09-20T22:18:02.821964+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:37.743137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:37.743137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:04.711878+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":805,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.632748+0000","last_change":"2024-09-20T22:17:33.935280+0000","last_active":"2024-09-20T22:18:04.632748+0000","last_peered":"2024-09-20T22:18:04.632748+0000","last_clean":"2024-09-20T22:18:04.632748+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:18:04.632748+0000","last_undegraded":"2024-09-20T22:18:04.632748+0000","last_fullsized":"2024-09-20T22:18:04.632748+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:33.935154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_clean_scrub_stamp":"2024-09-20T22:17:33.935154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:44.941220+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1a","version":"75'128","reported_seq":324,"reported_epoch":440,"state":"active+clean","last_fresh":"2024-09-20T22:18:04.629597+0000","last_change":"2024-09-20T22:17:56.396410+0000","last_active":"2024-09-20T22:18:04.629597+0000","last_peered":"2024-09-20T22:18:04.629597+0000","last_clean":"2024-09-20T22:18:04.629597+0000","last_became_active":"2024-09-20T22:17:44.723076+0000","last_became_peered":"2024-09-20T22:17:44.723076+0000","last_unstale":"2024-09-20T22:18:04.629597+0000","last_undegraded":"2024-09-20T22:18:04.629597+0000","last_fullsized":"2024-09-20T22:18:04.629597+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.396173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.396173+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:16.999042+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":692,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.822856+0000","last_change":"2024-09-20T22:15:49.314143+0000","last_active":"2024-09-20T22:18:02.822856+0000","last_peered":"2024-09-20T22:18:02.822856+0000","last_clean":"2024-09-20T22:18:02.822856+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:18:02.822856+0000","last_undegraded":"2024-09-20T22:18:02.822856+0000","last_fullsized":"2024-09-20T22:18:02.822856+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:15:49.314024+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:15:49.314024+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":265,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.615618+0000","last_change":"2024-09-20T22:16:56.775739+0000","last_active":"2024-09-20T22:18:01.615618+0000","last_peered":"2024-09-20T22:18:01.615618+0000","last_clean":"2024-09-20T22:18:01.615618+0000","last_became_active":"2024-09-20T22:12:52.518076+0000","last_became_peered":"2024-09-20T22:12:52.518076+0000","last_unstale":"2024-09-20T22:18:01.615618+0000","last_undegraded":"2024-09-20T22:18:01.615618+0000","last_fullsized":"2024-09-20T22:18:01.615618+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:56.775388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:56.775388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"328'19","reported_seq":859,"reported_epoch":439,"state":"active+clean","last_fresh":"2024-09-20T22:18:02.822604+0000","last_change":"2024-09-20T22:17:29.804302+0000","last_active":"2024-09-20T22:18:02.822604+0000","last_peered":"2024-09-20T22:18:02.822604+0000","last_clean":"2024-09-20T22:18:02.822604+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:18:02.822604+0000","last_undegraded":"2024-09-20T22:18:02.822604+0000","last_fullsized":"2024-09-20T22:18:02.822604+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:17:29.804188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:29.804188+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:41.153608+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":265,"reported_epoch":438,"state":"active+clean","last_fresh":"2024-09-20T22:18:01.617080+0000","last_change":"2024-09-20T22:17:10.840330+0000","last_active":"2024-09-20T22:18:01.617080+0000","last_peered":"2024-09-20T22:18:01.617080+0000","last_clean":"2024-09-20T22:18:01.617080+0000","last_became_active":"2024-09-20T22:12:52.524425+0000","last_became_peered":"2024-09-20T22:12:52.524425+0000","last_unstale":"2024-09-20T22:18:01.617080+0000","last_undegraded":"2024-09-20T22:18:01.617080+0000","last_fullsized":"2024-09-20T22:18:01.617080+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:10.839899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:10.839899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:18:35.398127+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":16,"stat_sum":{"num_bytes":2241069080,"num_objects":34197,"num_object_clones":0,"num_object_copies":102591,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34197,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":215383,"num_write_kb":7986497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6745108480,"data_stored":6745096264,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74685,"ondisk_log_size":74685,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1116,"num_read_kb":873,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":3410,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":743,"ondisk_log_size":743,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":891,"num_read_kb":886,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":31,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3968,"ondisk_log_size":3968,"up":93,"acting":93,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813136059,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1200764,"kb_used_data":524008,"kb_used_omap":1,"kb_used_meta":676734,"kb_avail":92544388,"statfs":{"total":95995035648,"available":94765453312,"internally_reserved":0,"allocated":536584192,"data_stored":536215161,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":692976074},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430396,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1580300,"kb_used_data":1011836,"kb_used_omap":1,"kb_used_meta":568446,"kb_avail":92164852,"statfs":{"total":95995035648,"available":94376808448,"internally_reserved":0,"allocated":1036120064,"data_stored":1035778561,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":582089158},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724739,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1357284,"kb_used_data":662420,"kb_used_omap":1,"kb_used_meta":694782,"kb_avail":92387868,"statfs":{"total":95995035648,"available":94605176832,"internally_reserved":0,"allocated":678318080,"data_stored":677968790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":711457221},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986379,"num_pgs":73,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1663480,"kb_used_data":1076968,"kb_used_omap":1,"kb_used_meta":586494,"kb_avail":92081672,"statfs":{"total":95995035648,"available":94291632128,"internally_reserved":0,"allocated":1102815232,"data_stored":1102445404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":600570309},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770258055,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":961928,"kb_used_data":426864,"kb_used_omap":4,"kb_used_meta":534971,"kb_avail":92783224,"statfs":{"total":95995035648,"available":95010021376,"internally_reserved":0,"allocated":437108736,"data_stored":436723671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":547810984},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149903,"num_pgs":70,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1481716,"kb_used_data":746656,"kb_used_omap":1,"kb_used_meta":735038,"kb_avail":92263436,"statfs":{"total":95995035648,"available":94477758464,"internally_reserved":0,"allocated":764575744,"data_stored":764198680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":752679367},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411541,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1432548,"kb_used_data":779348,"kb_used_omap":1,"kb_used_meta":653118,"kb_avail":92312604,"statfs":{"total":95995035648,"available":94528106496,"internally_reserved":0,"allocated":798052352,"data_stored":797703071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":668793285},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738591,"num_pgs":64,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2240548,"kb_used_data":1473552,"kb_used_omap":1,"kb_used_meta":766974,"kb_avail":91504604,"statfs":{"total":95995035648,"available":93700714496,"internally_reserved":0,"allocated":1508917248,"data_stored":1508555489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":785381829},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,1,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1496322048,"data_stored":1496317976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":430702592,"data_stored":430702592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":778502144,"data_stored":778502144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":759824384,"data_stored":759824384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1071648768,"data_stored":1071644696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":654180352,"data_stored":654180352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1020858368,"data_stored":1020854296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":533069824,"data_stored":533069824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:18:09.166 INFO:tasks.thrashosds.thrasher:Setting 5.12 to [2, 5, 4, 7, 1, 0] 2024-09-20T22:18:09.167 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap-items', '5.12', '2', '5', '4', '7', '1', '0'] 2024-09-20T22:18:09.167 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pg-upmap-items 5.12 2 5 4 7 1 0 2024-09-20T22:18:09.198 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:18:09.199 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:18:09.199 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:18:09.199 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:18:09.199 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:18:09.199 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:18:09.199 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:18:09.199 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:18:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:18:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:18:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:18:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:18:09.199 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:18:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:18:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:18:09.200 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:18:09.203 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:09.227 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:09.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:09.228 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:09.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:09 smithi060 ceph-mon[29694]: 10.3 scrub starts 2024-09-20T22:18:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:09 smithi060 ceph-mon[29694]: pgmap v815: 178 pgs: 1 active+clean+scrubbing+deep, 177 active+clean; 1.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.44k op/s 2024-09-20T22:18:09.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:09 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1256360632' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:09.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:09 smithi060 ceph-mon[40290]: 10.3 scrub starts 2024-09-20T22:18:09.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:09 smithi060 ceph-mon[40290]: pgmap v815: 178 pgs: 1 active+clean+scrubbing+deep, 177 active+clean; 1.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.44k op/s 2024-09-20T22:18:09.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:09 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1256360632' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:09.328 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:09.364 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:09.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:09.365 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:09.466 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:09.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:09 smithi067 ceph-mon[33511]: 10.3 scrub starts 2024-09-20T22:18:09.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:09 smithi067 ceph-mon[33511]: pgmap v815: 178 pgs: 1 active+clean+scrubbing+deep, 177 active+clean; 1.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.44k op/s 2024-09-20T22:18:09.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:09 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1256360632' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:09.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:09.534 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:09.535 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:09.635 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:09.674 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:09.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:09.675 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:09.776 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:09.808 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:09.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:09.809 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:09.910 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:09.949 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:09.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:09.950 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:10.003 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T22:18:10.050 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:10.097 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:10.097 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:10.098 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:10.198 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:10.233 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:10.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:10.235 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:10.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:10 smithi060 ceph-mon[29694]: 10.3 scrub ok 2024-09-20T22:18:10.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:10 smithi060 ceph-mon[29694]: 3.17 scrub starts 2024-09-20T22:18:10.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:10 smithi060 ceph-mon[29694]: 3.17 scrub ok 2024-09-20T22:18:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:10 smithi060 ceph-mon[40290]: 10.3 scrub ok 2024-09-20T22:18:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:10 smithi060 ceph-mon[40290]: 3.17 scrub starts 2024-09-20T22:18:10.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:10 smithi060 ceph-mon[40290]: 3.17 scrub ok 2024-09-20T22:18:10.336 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:10.367 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:10.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:10.368 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:10.469 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:10.504 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:10.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:10.505 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:10.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:10 smithi067 ceph-mon[33511]: 10.3 scrub ok 2024-09-20T22:18:10.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:10 smithi067 ceph-mon[33511]: 3.17 scrub starts 2024-09-20T22:18:10.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:10 smithi067 ceph-mon[33511]: 3.17 scrub ok 2024-09-20T22:18:10.605 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:10.639 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:10.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:10.640 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:10.741 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:10.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:10.783 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:10.784 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:10.884 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:10.912 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:10.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:10.913 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:10.938 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:10.946 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:11.014 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:11.045 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:11.046 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:11.046 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:11.147 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:11.182 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:11.183 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:11.183 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:11.232 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[29694]: 6.1c scrub starts 2024-09-20T22:18:11.233 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[29694]: 6.1c scrub ok 2024-09-20T22:18:11.233 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[29694]: 10.a scrub starts 2024-09-20T22:18:11.233 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[29694]: 5.c deep-scrub starts 2024-09-20T22:18:11.233 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[29694]: 5.c deep-scrub ok 2024-09-20T22:18:11.233 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[29694]: pgmap v816: 178 pgs: 1 active+clean+scrubbing+deep, 177 active+clean; 1.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 683 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-20T22:18:11.233 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[29694]: 4.1e scrub starts 2024-09-20T22:18:11.233 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[29694]: 4.1e scrub ok 2024-09-20T22:18:11.233 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[29694]: 5.1 scrub starts 2024-09-20T22:18:11.233 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[29694]: 5.1 scrub ok 2024-09-20T22:18:11.233 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[29694]: 10.a scrub ok 2024-09-20T22:18:11.233 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[40290]: 6.1c scrub starts 2024-09-20T22:18:11.233 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[40290]: 6.1c scrub ok 2024-09-20T22:18:11.233 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[40290]: 10.a scrub starts 2024-09-20T22:18:11.234 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[40290]: 5.c deep-scrub starts 2024-09-20T22:18:11.234 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[40290]: 5.c deep-scrub ok 2024-09-20T22:18:11.234 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[40290]: pgmap v816: 178 pgs: 1 active+clean+scrubbing+deep, 177 active+clean; 1.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 683 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-20T22:18:11.234 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[40290]: 4.1e scrub starts 2024-09-20T22:18:11.234 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[40290]: 4.1e scrub ok 2024-09-20T22:18:11.234 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[40290]: 5.1 scrub starts 2024-09-20T22:18:11.234 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[40290]: 5.1 scrub ok 2024-09-20T22:18:11.234 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:11 smithi060 ceph-mon[40290]: 10.a scrub ok 2024-09-20T22:18:11.285 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:11.333 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:11.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:11.334 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:11.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:11.463 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:11.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:11.464 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:11.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:11 smithi067 ceph-mon[33511]: 6.1c scrub starts 2024-09-20T22:18:11.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:11 smithi067 ceph-mon[33511]: 6.1c scrub ok 2024-09-20T22:18:11.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:11 smithi067 ceph-mon[33511]: 10.a scrub starts 2024-09-20T22:18:11.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:11 smithi067 ceph-mon[33511]: 5.c deep-scrub starts 2024-09-20T22:18:11.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:11 smithi067 ceph-mon[33511]: 5.c deep-scrub ok 2024-09-20T22:18:11.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:11 smithi067 ceph-mon[33511]: pgmap v816: 178 pgs: 1 active+clean+scrubbing+deep, 177 active+clean; 1.9 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 683 B/s rd, 0 B/s wr, 1.10k op/s 2024-09-20T22:18:11.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:11 smithi067 ceph-mon[33511]: 4.1e scrub starts 2024-09-20T22:18:11.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:11 smithi067 ceph-mon[33511]: 4.1e scrub ok 2024-09-20T22:18:11.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:11 smithi067 ceph-mon[33511]: 5.1 scrub starts 2024-09-20T22:18:11.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:11 smithi067 ceph-mon[33511]: 5.1 scrub ok 2024-09-20T22:18:11.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:11 smithi067 ceph-mon[33511]: 10.a scrub ok 2024-09-20T22:18:11.565 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:11.599 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:11.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:11.600 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:11.701 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:11.730 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:11.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:11.731 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:11.831 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:11.860 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:11.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:11.861 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:11.962 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:11.991 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:11.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:11.992 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:12.093 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:12.133 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:12.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:12.135 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:12.236 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:12.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:12.274 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:12.275 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:12.375 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:12.391 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:12.415 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:12.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:12.416 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:12.517 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:12.550 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:12.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:12.551 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:12.623 INFO:teuthology.orchestra.run.smithi060.stderr:set 5.12 pg_upmap_items mapping to [2->5,4->7,1->0] 2024-09-20T22:18:12.626 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:18:12.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[29694]: 6.6 scrub starts 2024-09-20T22:18:12.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[29694]: 6.6 scrub ok 2024-09-20T22:18:12.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[29694]: osdmap e442: 8 total, 8 up, 8 in 2024-09-20T22:18:12.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[29694]: 6.3 scrub starts 2024-09-20T22:18:12.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[29694]: 6.3 scrub ok 2024-09-20T22:18:12.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[29694]: 5.b scrub starts 2024-09-20T22:18:12.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[29694]: 5.b scrub ok 2024-09-20T22:18:12.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3850714762' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "5.12", "id": [2, 5, 4, 7, 1, 0]}]: dispatch 2024-09-20T22:18:12.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "5.12", "id": [2, 5, 4, 7, 1, 0]}]: dispatch 2024-09-20T22:18:12.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3815001747' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:18:12.647 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:18:12.647 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[40290]: 6.6 scrub starts 2024-09-20T22:18:12.648 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[40290]: 6.6 scrub ok 2024-09-20T22:18:12.648 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[40290]: osdmap e442: 8 total, 8 up, 8 in 2024-09-20T22:18:12.648 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[40290]: 6.3 scrub starts 2024-09-20T22:18:12.648 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[40290]: 6.3 scrub ok 2024-09-20T22:18:12.648 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[40290]: 5.b scrub starts 2024-09-20T22:18:12.648 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[40290]: 5.b scrub ok 2024-09-20T22:18:12.648 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3850714762' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "5.12", "id": [2, 5, 4, 7, 1, 0]}]: dispatch 2024-09-20T22:18:12.648 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "5.12", "id": [2, 5, 4, 7, 1, 0]}]: dispatch 2024-09-20T22:18:12.648 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3815001747' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:18:12.648 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:12 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:18:12.652 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:12.696 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:12.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:12.697 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:12.798 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:12.833 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:12.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:12.835 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:12.923 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:18:12.935 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:13.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:18:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:18:13.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:12 smithi067 ceph-mon[33511]: 6.6 scrub starts 2024-09-20T22:18:13.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:12 smithi067 ceph-mon[33511]: 6.6 scrub ok 2024-09-20T22:18:13.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:12 smithi067 ceph-mon[33511]: osdmap e442: 8 total, 8 up, 8 in 2024-09-20T22:18:13.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:12 smithi067 ceph-mon[33511]: 6.3 scrub starts 2024-09-20T22:18:13.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:12 smithi067 ceph-mon[33511]: 6.3 scrub ok 2024-09-20T22:18:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:12 smithi067 ceph-mon[33511]: 5.b scrub starts 2024-09-20T22:18:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:12 smithi067 ceph-mon[33511]: 5.b scrub ok 2024-09-20T22:18:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:12 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3850714762' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "5.12", "id": [2, 5, 4, 7, 1, 0]}]: dispatch 2024-09-20T22:18:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:12 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pg-upmap-items", "pgid": "5.12", "id": [2, 5, 4, 7, 1, 0]}]: dispatch 2024-09-20T22:18:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:12 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3815001747' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:18:13.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:12 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:18:13.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:13.015 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:13.015 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:13.116 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:13.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:13.152 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:13.152 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:13.183 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:18:13.183 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:18:13.252 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:13.290 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:13.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:13.291 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:13.392 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:13.425 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:13.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:13.426 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:13.527 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:13.556 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:13.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:13.557 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:13.658 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:13.687 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:13.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:13.687 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:13.788 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:13.823 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:13.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:13.824 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:13.891 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:13 smithi060 ceph-mon[29694]: pgmap v818: 178 pgs: 1 active+clean+scrubbing+deep, 177 active+clean; 1.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.04k op/s 2024-09-20T22:18:13.891 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:13 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:18:13.891 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:13 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "5.12", "id": [2, 5, 4, 7, 1, 0]}]': finished 2024-09-20T22:18:13.891 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:13 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:18:13.891 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:13 smithi060 ceph-mon[29694]: osdmap e443: 8 total, 8 up, 8 in 2024-09-20T22:18:13.891 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:13 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:18:13.892 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:13 smithi060 ceph-mon[40290]: pgmap v818: 178 pgs: 1 active+clean+scrubbing+deep, 177 active+clean; 1.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.04k op/s 2024-09-20T22:18:13.892 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:13 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:18:13.892 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:13 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "5.12", "id": [2, 5, 4, 7, 1, 0]}]': finished 2024-09-20T22:18:13.892 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:13 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:18:13.892 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:13 smithi060 ceph-mon[40290]: osdmap e443: 8 total, 8 up, 8 in 2024-09-20T22:18:13.892 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:13 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:18:13.925 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:13.963 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:13.964 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:13.964 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:13 smithi067 ceph-mon[33511]: pgmap v818: 178 pgs: 1 active+clean+scrubbing+deep, 177 active+clean; 1.7 GiB data, 10 GiB used, 705 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.04k op/s 2024-09-20T22:18:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:13 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:18:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:13 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pg-upmap-items", "pgid": "5.12", "id": [2, 5, 4, 7, 1, 0]}]': finished 2024-09-20T22:18:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:13 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:18:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:13 smithi067 ceph-mon[33511]: osdmap e443: 8 total, 8 up, 8 in 2024-09-20T22:18:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:13 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:18:14.065 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:14.099 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:14.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:14.100 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:14.201 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:14.234 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:14.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:14.235 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:14.336 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:14.369 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:14.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:14.370 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:14.471 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:14.502 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:14.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:14.503 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:14.603 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:14.641 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:14.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:14.642 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:14.743 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:14.774 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:14.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:14.775 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:14.875 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:14.905 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:14.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:14.906 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:15.007 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:14 smithi067 ceph-mon[33511]: osdmap e444: 8 total, 8 up, 8 in 2024-09-20T22:18:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:14 smithi067 ceph-mon[33511]: pgmap v821: 178 pgs: 1 active+clean+scrubbing+deep, 177 active+clean; 1.6 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 946 op/s 2024-09-20T22:18:15.033 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:15.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:15.034 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:15.035 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:14 smithi060 ceph-mon[40290]: osdmap e444: 8 total, 8 up, 8 in 2024-09-20T22:18:15.035 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:14 smithi060 ceph-mon[40290]: pgmap v821: 178 pgs: 1 active+clean+scrubbing+deep, 177 active+clean; 1.6 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 946 op/s 2024-09-20T22:18:15.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:14 smithi060 ceph-mon[29694]: osdmap e444: 8 total, 8 up, 8 in 2024-09-20T22:18:15.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:14 smithi060 ceph-mon[29694]: pgmap v821: 178 pgs: 1 active+clean+scrubbing+deep, 177 active+clean; 1.6 GiB data, 9.9 GiB used, 705 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 946 op/s 2024-09-20T22:18:15.096 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T22:18:15.116 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:15.134 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:15.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:15.167 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:15.167 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:15.268 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:15.295 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:18:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:18:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:18:15.298 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:15.299 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:15.299 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:15.400 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:15.434 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:15.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:15.435 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:15.535 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:15.569 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:15.570 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:15.570 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:15.671 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:15.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:15.715 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:15.715 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:15.805 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:18:15.805 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:18:15.815 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:15.816 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:15.843 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:18:15.843 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:18:15.856 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:15.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:15.857 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:15.875 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:18:15.875 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:18:15.897 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:18:15.898 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:18:15.898 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:15 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2635845873' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:15.898 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:15 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2635845873' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:15.922 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:18:15.922 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:18:15.951 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:18:15.952 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:18:15.958 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:15.977 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:18:15.977 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:18:15.986 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:15.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:15.987 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:16.007 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:18:16.008 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:18:16.090 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:16.130 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:16.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:16.131 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:16.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:16.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:15 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2635845873' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:16.265 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:16.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:16.266 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:16.367 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:16.401 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:16.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:16.402 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:16.503 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:16.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:16.546 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:16.546 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:16.632 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:18:16.632 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:18:16.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:18:16.632 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:18:16.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.632 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106977 8.12 8:4ad51fd6:::benchmark_data_smithi060_111051_object106976:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:18:16.632 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418139+0000", 2024-09-20T22:18:16.633 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 545.20896798499996, 2024-09-20T22:18:16.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.107260704, 2024-09-20T22:18:16.633 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.633 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.633 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:16.633 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:16.633 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:16.633 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:16.633 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106977 2024-09-20T22:18:16.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.633 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.633 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.633 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.633 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418139+0000", 2024-09-20T22:18:16.634 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.634 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.634 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.634 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418139+0000", 2024-09-20T22:18:16.634 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.634 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.634 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.634 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418139+0000", 2024-09-20T22:18:16.634 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4300000000000003e-07 2024-09-20T22:18:16.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.634 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.634 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.635 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418194+0000", 2024-09-20T22:18:16.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4564999999999997e-05 2024-09-20T22:18:16.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.635 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.635 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418202+0000", 2024-09-20T22:18:16.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.755e-06 2024-09-20T22:18:16.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.635 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.635 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418211+0000", 2024-09-20T22:18:16.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5199999999999997e-06 2024-09-20T22:18:16.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418518+0000", 2024-09-20T22:18:16.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030665299999999999 2024-09-20T22:18:16.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418525+0000", 2024-09-20T22:18:16.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4930000000000001e-06 2024-09-20T22:18:16.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.508187+0000", 2024-09-20T22:18:16.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0896622950000001 2024-09-20T22:18:16.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.637 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.637 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.508457+0000", 2024-09-20T22:18:16.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00027002800000000001 2024-09-20T22:18:16.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.637 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,4,5,6,7", 2024-09-20T22:18:16.637 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.508645+0000", 2024-09-20T22:18:16.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018733000000000001 2024-09-20T22:18:16.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.637 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:16.637 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.514513+0000", 2024-09-20T22:18:16.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0058678259999999996 2024-09-20T22:18:16.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.638 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.638 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.514560+0000", 2024-09-20T22:18:16.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7314999999999997e-05 2024-09-20T22:18:16.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.638 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.638 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.514664+0000", 2024-09-20T22:18:16.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104125 2024-09-20T22:18:16.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.638 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.638 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515122+0000", 2024-09-20T22:18:16.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00045837200000000002 2024-09-20T22:18:16.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515704+0000", 2024-09-20T22:18:16.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00058129899999999997 2024-09-20T22:18:16.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525183+0000", 2024-09-20T22:18:16.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.009479019 2024-09-20T22:18:16.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:16.640 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525256+0000", 2024-09-20T22:18:16.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3410000000000004e-05 2024-09-20T22:18:16.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.640 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.640 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525399+0000", 2024-09-20T22:18:16.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000143156 2024-09-20T22:18:16.640 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.640 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.640 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.640 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106979 8.16 8:6c64d7d1:::benchmark_data_smithi060_111051_object106978:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:18:16.640 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418222+0000", 2024-09-20T22:18:16.641 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 545.20888491100004, 2024-09-20T22:18:16.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.107288856, 2024-09-20T22:18:16.641 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.641 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.641 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:16.641 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:16.641 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:16.641 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:16.641 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106979 2024-09-20T22:18:16.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.641 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.641 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.641 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418222+0000", 2024-09-20T22:18:16.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.642 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.642 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418222+0000", 2024-09-20T22:18:16.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.642 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.642 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418222+0000", 2024-09-20T22:18:16.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1900000000000003e-07 2024-09-20T22:18:16.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.642 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418273+0000", 2024-09-20T22:18:16.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0751000000000002e-05 2024-09-20T22:18:16.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418282+0000", 2024-09-20T22:18:16.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1439999999999992e-06 2024-09-20T22:18:16.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418289+0000", 2024-09-20T22:18:16.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4829999999999996e-06 2024-09-20T22:18:16.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418324+0000", 2024-09-20T22:18:16.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5508e-05 2024-09-20T22:18:16.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418331+0000", 2024-09-20T22:18:16.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2679999999999997e-06 2024-09-20T22:18:16.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.512703+0000", 2024-09-20T22:18:16.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.094371502 2024-09-20T22:18:16.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.645 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.645 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.512972+0000", 2024-09-20T22:18:16.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026964600000000001 2024-09-20T22:18:16.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.645 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5,6,7", 2024-09-20T22:18:16.645 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.513953+0000", 2024-09-20T22:18:16.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00098022600000000006 2024-09-20T22:18:16.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.645 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:16.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.519844+0000", 2024-09-20T22:18:16.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005891309 2024-09-20T22:18:16.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.520362+0000", 2024-09-20T22:18:16.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00051799699999999997 2024-09-20T22:18:16.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.520890+0000", 2024-09-20T22:18:16.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00052754400000000002 2024-09-20T22:18:16.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524659+0000", 2024-09-20T22:18:16.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037695160000000001 2024-09-20T22:18:16.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524693+0000", 2024-09-20T22:18:16.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3887999999999998e-05 2024-09-20T22:18:16.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525239+0000", 2024-09-20T22:18:16.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00054655600000000002 2024-09-20T22:18:16.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:16.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525313+0000", 2024-09-20T22:18:16.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3485999999999994e-05 2024-09-20T22:18:16.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525510+0000", 2024-09-20T22:18:16.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019751299999999999 2024-09-20T22:18:16.649 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.649 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.649 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.649 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106981 8.16 8:6961ec02:::benchmark_data_smithi060_111051_object106980:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:18:16.649 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418309+0000", 2024-09-20T22:18:16.649 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 545.20879778200003, 2024-09-20T22:18:16.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1092024140000001, 2024-09-20T22:18:16.650 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.650 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.650 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:16.650 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:16.650 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:16.650 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:16.650 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106981 2024-09-20T22:18:16.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.650 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418309+0000", 2024-09-20T22:18:16.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.651 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.651 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418309+0000", 2024-09-20T22:18:16.651 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.651 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.651 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418309+0000", 2024-09-20T22:18:16.651 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3000000000000001e-07 2024-09-20T22:18:16.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.651 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.651 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418370+0000", 2024-09-20T22:18:16.651 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0577e-05 2024-09-20T22:18:16.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.652 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418379+0000", 2024-09-20T22:18:16.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5279999999999992e-06 2024-09-20T22:18:16.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.652 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418385+0000", 2024-09-20T22:18:16.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.105e-06 2024-09-20T22:18:16.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.652 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418587+0000", 2024-09-20T22:18:16.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020189700000000001 2024-09-20T22:18:16.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418595+0000", 2024-09-20T22:18:16.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.187e-06 2024-09-20T22:18:16.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515314+0000", 2024-09-20T22:18:16.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0967195460000001 2024-09-20T22:18:16.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515442+0000", 2024-09-20T22:18:16.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012829599999999999 2024-09-20T22:18:16.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5,6,7", 2024-09-20T22:18:16.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515706+0000", 2024-09-20T22:18:16.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026385599999999998 2024-09-20T22:18:16.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:16.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.519902+0000", 2024-09-20T22:18:16.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.004195786 2024-09-20T22:18:16.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.520407+0000", 2024-09-20T22:18:16.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00050457400000000004 2024-09-20T22:18:16.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524729+0000", 2024-09-20T22:18:16.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0043228279999999999 2024-09-20T22:18:16.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524770+0000", 2024-09-20T22:18:16.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0528000000000002e-05 2024-09-20T22:18:16.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524965+0000", 2024-09-20T22:18:16.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019492000000000001 2024-09-20T22:18:16.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.527323+0000", 2024-09-20T22:18:16.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0023577250000000002 2024-09-20T22:18:16.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:16.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.527382+0000", 2024-09-20T22:18:16.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9228999999999998e-05 2024-09-20T22:18:16.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.527511+0000", 2024-09-20T22:18:16.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000129302 2024-09-20T22:18:16.657 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.657 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.657 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.657 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106997 8.16 8:6b0b3968:::benchmark_data_smithi060_111051_object106996:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:18:16.657 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418412+0000", 2024-09-20T22:18:16.657 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 545.20869484599996, 2024-09-20T22:18:16.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1094631589999999, 2024-09-20T22:18:16.657 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.658 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.658 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:16.658 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:16.658 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:16.658 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:16.658 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106997 2024-09-20T22:18:16.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.658 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.658 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.658 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418412+0000", 2024-09-20T22:18:16.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.658 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418412+0000", 2024-09-20T22:18:16.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418412+0000", 2024-09-20T22:18:16.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.13e-07 2024-09-20T22:18:16.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418461+0000", 2024-09-20T22:18:16.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8504999999999997e-05 2024-09-20T22:18:16.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418470+0000", 2024-09-20T22:18:16.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4660000000000005e-06 2024-09-20T22:18:16.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418476+0000", 2024-09-20T22:18:16.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2179999999999996e-06 2024-09-20T22:18:16.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418618+0000", 2024-09-20T22:18:16.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014114800000000001 2024-09-20T22:18:16.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418624+0000", 2024-09-20T22:18:16.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8009999999999998e-06 2024-09-20T22:18:16.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.517728+0000", 2024-09-20T22:18:16.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.099103513 2024-09-20T22:18:16.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.517868+0000", 2024-09-20T22:18:16.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014029400000000001 2024-09-20T22:18:16.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3,5,6,7", 2024-09-20T22:18:16.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.518178+0000", 2024-09-20T22:18:16.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030979800000000002 2024-09-20T22:18:16.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:16.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.519909+0000", 2024-09-20T22:18:16.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017308740000000001 2024-09-20T22:18:16.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.522366+0000", 2024-09-20T22:18:16.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024576609999999999 2024-09-20T22:18:16.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524823+0000", 2024-09-20T22:18:16.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024567830000000001 2024-09-20T22:18:16.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524862+0000", 2024-09-20T22:18:16.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9060000000000002e-05 2024-09-20T22:18:16.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525002+0000", 2024-09-20T22:18:16.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000139438 2024-09-20T22:18:16.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.527697+0000", 2024-09-20T22:18:16.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0026952149999999999 2024-09-20T22:18:16.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:16.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.527759+0000", 2024-09-20T22:18:16.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1612999999999994e-05 2024-09-20T22:18:16.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.527875+0000", 2024-09-20T22:18:16.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000116259 2024-09-20T22:18:16.665 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.665 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.665 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.665 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:107002 8.11 8:8e84af07:::benchmark_data_smithi060_111051_object107001:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:18:16.665 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418488+0000", 2024-09-20T22:18:16.665 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 545.20861876200001, 2024-09-20T22:18:16.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1327261360000001, 2024-09-20T22:18:16.665 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.665 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.665 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:16.665 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:16.665 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:16.666 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:16.666 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 107002 2024-09-20T22:18:16.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.666 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418488+0000", 2024-09-20T22:18:16.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418488+0000", 2024-09-20T22:18:16.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418488+0000", 2024-09-20T22:18:16.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4700000000000001e-07 2024-09-20T22:18:16.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418536+0000", 2024-09-20T22:18:16.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7899999999999999e-05 2024-09-20T22:18:16.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418545+0000", 2024-09-20T22:18:16.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5210000000000002e-06 2024-09-20T22:18:16.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418551+0000", 2024-09-20T22:18:16.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.409e-06 2024-09-20T22:18:16.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418608+0000", 2024-09-20T22:18:16.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7015999999999998e-05 2024-09-20T22:18:16.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.668 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.668 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418619+0000", 2024-09-20T22:18:16.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0900999999999999e-05 2024-09-20T22:18:16.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.538830+0000", 2024-09-20T22:18:16.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.120210792 2024-09-20T22:18:16.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.538979+0000", 2024-09-20T22:18:16.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014885199999999999 2024-09-20T22:18:16.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2,5,6,7", 2024-09-20T22:18:16.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.539152+0000", 2024-09-20T22:18:16.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000173134 2024-09-20T22:18:16.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.543638+0000", 2024-09-20T22:18:16.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0044860739999999996 2024-09-20T22:18:16.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.546226+0000", 2024-09-20T22:18:16.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00258807 2024-09-20T22:18:16.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.546267+0000", 2024-09-20T22:18:16.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0592e-05 2024-09-20T22:18:16.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.546317+0000", 2024-09-20T22:18:16.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0781000000000001e-05 2024-09-20T22:18:16.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:16.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.546370+0000", 2024-09-20T22:18:16.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2358999999999998e-05 2024-09-20T22:18:16.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.550100+0000", 2024-09-20T22:18:16.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0037306309999999999 2024-09-20T22:18:16.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:16.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.551067+0000", 2024-09-20T22:18:16.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00096712000000000002 2024-09-20T22:18:16.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.551214+0000", 2024-09-20T22:18:16.673 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:16.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014643699999999999 2024-09-20T22:18:16.674 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.674 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.674 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.674 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:107007 8.12 8:4e9d1372:::benchmark_data_smithi060_111051_object107006:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:18:16.674 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418568+0000", 2024-09-20T22:18:16.674 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 545.20853876000001, 2024-09-20T22:18:16.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.108179131, 2024-09-20T22:18:16.674 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.675 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.675 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:16.675 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:16.675 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:16.675 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:16.675 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 107007 2024-09-20T22:18:16.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.675 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418568+0000", 2024-09-20T22:18:16.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418568+0000", 2024-09-20T22:18:16.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418568+0000", 2024-09-20T22:18:16.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3499999999999996e-07 2024-09-20T22:18:16.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418616+0000", 2024-09-20T22:18:16.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7386000000000003e-05 2024-09-20T22:18:16.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418624+0000", 2024-09-20T22:18:16.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1110000000000008e-06 2024-09-20T22:18:16.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418630+0000", 2024-09-20T22:18:16.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9390000000000002e-06 2024-09-20T22:18:16.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418675+0000", 2024-09-20T22:18:16.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5444000000000001e-05 2024-09-20T22:18:16.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418683+0000", 2024-09-20T22:18:16.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5140000000000004e-06 2024-09-20T22:18:16.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.510695+0000", 2024-09-20T22:18:16.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.092012032 2024-09-20T22:18:16.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.510853+0000", 2024-09-20T22:18:16.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015822899999999999 2024-09-20T22:18:16.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,4,5,6,7", 2024-09-20T22:18:16.679 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.511044+0000", 2024-09-20T22:18:16.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019054099999999999 2024-09-20T22:18:16.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:16.679 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.514574+0000", 2024-09-20T22:18:16.679 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003530858 2024-09-20T22:18:16.679 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.679 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.679 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515032+0000", 2024-09-20T22:18:16.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00045793899999999998 2024-09-20T22:18:16.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.680 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515080+0000", 2024-09-20T22:18:16.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8164999999999997e-05 2024-09-20T22:18:16.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.680 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515664+0000", 2024-09-20T22:18:16.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00058304799999999996 2024-09-20T22:18:16.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.680 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.515792+0000", 2024-09-20T22:18:16.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012877 2024-09-20T22:18:16.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.682 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525673+0000", 2024-09-20T22:18:16.682 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0098810990000000008 2024-09-20T22:18:16.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.682 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:16.682 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.526607+0000", 2024-09-20T22:18:16.682 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00093347000000000005 2024-09-20T22:18:16.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.682 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.526747+0000", 2024-09-20T22:18:16.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000140051 2024-09-20T22:18:16.683 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.683 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.683 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.683 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:107008 8.12 8:4cb7847c:::benchmark_data_smithi060_111051_object107007:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:18:16.683 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.418644+0000", 2024-09-20T22:18:16.683 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 545.208462919, 2024-09-20T22:18:16.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1083103759999999, 2024-09-20T22:18:16.683 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.683 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.684 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:16.684 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:16.684 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:16.684 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:16.684 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 107008 2024-09-20T22:18:16.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.684 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418644+0000", 2024-09-20T22:18:16.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418644+0000", 2024-09-20T22:18:16.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418644+0000", 2024-09-20T22:18:16.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2e-07 2024-09-20T22:18:16.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418693+0000", 2024-09-20T22:18:16.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8569000000000002e-05 2024-09-20T22:18:16.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418702+0000", 2024-09-20T22:18:16.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8060000000000001e-06 2024-09-20T22:18:16.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418708+0000", 2024-09-20T22:18:16.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2500000000000003e-06 2024-09-20T22:18:16.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418797+0000", 2024-09-20T22:18:16.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9543999999999995e-05 2024-09-20T22:18:16.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418805+0000", 2024-09-20T22:18:16.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0290000000000005e-06 2024-09-20T22:18:16.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.512330+0000", 2024-09-20T22:18:16.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0935242919999999 2024-09-20T22:18:16.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.512473+0000", 2024-09-20T22:18:16.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000143695 2024-09-20T22:18:16.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,4,5,6,7", 2024-09-20T22:18:16.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.513190+0000", 2024-09-20T22:18:16.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00071678699999999998 2024-09-20T22:18:16.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.518925+0000", 2024-09-20T22:18:16.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0057347919999999998 2024-09-20T22:18:16.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.520073+0000", 2024-09-20T22:18:16.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0011479420000000001 2024-09-20T22:18:16.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:16.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.520313+0000", 2024-09-20T22:18:16.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023994399999999999 2024-09-20T22:18:16.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.523625+0000", 2024-09-20T22:18:16.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003312008 2024-09-20T22:18:16.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.524809+0000", 2024-09-20T22:18:16.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001184456 2024-09-20T22:18:16.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.526788+0000", 2024-09-20T22:18:16.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001979218 2024-09-20T22:18:16.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:16.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.526838+0000", 2024-09-20T22:18:16.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9617999999999998e-05 2024-09-20T22:18:16.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.526954+0000", 2024-09-20T22:18:16.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115906 2024-09-20T22:18:16.691 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.691 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.691 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.691 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187850 8.c 8:31fd15bc:::benchmark_data_smithi060_111051_object67851:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:18:16.691 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.627306+0000", 2024-09-20T22:18:16.691 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 466.99980069700001, 2024-09-20T22:18:16.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2448928590000001, 2024-09-20T22:18:16.691 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.691 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.691 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:18:16.691 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:16.691 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:16.692 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:16.692 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187850 2024-09-20T22:18:16.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.692 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627306+0000", 2024-09-20T22:18:16.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627306+0000", 2024-09-20T22:18:16.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627307+0000", 2024-09-20T22:18:16.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.288e-06 2024-09-20T22:18:16.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627312+0000", 2024-09-20T22:18:16.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7990000000000001e-06 2024-09-20T22:18:16.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627328+0000", 2024-09-20T22:18:16.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5909e-05 2024-09-20T22:18:16.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627335+0000", 2024-09-20T22:18:16.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1250000000000004e-06 2024-09-20T22:18:16.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627384+0000", 2024-09-20T22:18:16.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9218000000000002e-05 2024-09-20T22:18:16.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.630537+0000", 2024-09-20T22:18:16.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0031527109999999999 2024-09-20T22:18:16.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:18:16.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.630770+0000", 2024-09-20T22:18:16.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023290799999999999 2024-09-20T22:18:16.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:16.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.642588+0000", 2024-09-20T22:18:16.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.011817831000000001 2024-09-20T22:18:16.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.695 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.666253+0000", 2024-09-20T22:18:16.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.023664916000000001 2024-09-20T22:18:16.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.666266+0000", 2024-09-20T22:18:16.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3818999999999999e-05 2024-09-20T22:18:16.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.658763+0000", 2024-09-20T22:18:16.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99249631000000005 2024-09-20T22:18:16.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.658780+0000", 2024-09-20T22:18:16.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7697e-05 2024-09-20T22:18:16.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.860783+0000", 2024-09-20T22:18:16.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.202002518 2024-09-20T22:18:16.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:18:16.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.860911+0000", 2024-09-20T22:18:16.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012827499999999999 2024-09-20T22:18:16.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.871844+0000", 2024-09-20T22:18:16.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010932667 2024-09-20T22:18:16.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.872199+0000", 2024-09-20T22:18:16.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00035486799999999999 2024-09-20T22:18:16.698 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.698 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.698 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.699 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187851 8.c 8:3eebc9c2:::benchmark_data_smithi060_111051_object67852:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:18:16.699 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.627552+0000", 2024-09-20T22:18:16.699 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 466.99955408699998, 2024-09-20T22:18:16.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2481179880000002, 2024-09-20T22:18:16.699 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.699 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.699 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:18:16.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:16.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:16.699 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:16.699 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187851 2024-09-20T22:18:16.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.699 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627552+0000", 2024-09-20T22:18:16.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.700 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627552+0000", 2024-09-20T22:18:16.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.700 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627553+0000", 2024-09-20T22:18:16.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7599999999999996e-07 2024-09-20T22:18:16.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627558+0000", 2024-09-20T22:18:16.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.369e-06 2024-09-20T22:18:16.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627567+0000", 2024-09-20T22:18:16.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8199999999999992e-06 2024-09-20T22:18:16.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627574+0000", 2024-09-20T22:18:16.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4880000000000004e-06 2024-09-20T22:18:16.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.635469+0000", 2024-09-20T22:18:16.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0078950770000000003 2024-09-20T22:18:16.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.635599+0000", 2024-09-20T22:18:16.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012998 2024-09-20T22:18:16.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:18:16.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.635730+0000", 2024-09-20T22:18:16.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013132000000000001 2024-09-20T22:18:16.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:16.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.642594+0000", 2024-09-20T22:18:16.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0068636829999999998 2024-09-20T22:18:16.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.666296+0000", 2024-09-20T22:18:16.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.023701567 2024-09-20T22:18:16.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.666303+0000", 2024-09-20T22:18:16.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.695e-06 2024-09-20T22:18:16.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.661330+0000", 2024-09-20T22:18:16.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99502645599999995 2024-09-20T22:18:16.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.704 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.704 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.704 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.661341+0000", 2024-09-20T22:18:16.704 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1564e-05 2024-09-20T22:18:16.704 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.861170+0000", 2024-09-20T22:18:16.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1998287999999999 2024-09-20T22:18:16.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:18:16.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.861243+0000", 2024-09-20T22:18:16.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3425999999999998e-05 2024-09-20T22:18:16.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.705 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.705 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.705 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.875594+0000", 2024-09-20T22:18:16.705 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014350577 2024-09-20T22:18:16.705 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.706 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.706 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.875670+0000", 2024-09-20T22:18:16.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6390000000000006e-05 2024-09-20T22:18:16.706 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.706 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.706 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.706 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.706 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.706 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187854 8.c 8:37fa2592:::benchmark_data_smithi060_111051_object67855:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:18:16.706 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.632485+0000", 2024-09-20T22:18:16.706 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 466.99462169700001, 2024-09-20T22:18:16.706 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2502292339999999, 2024-09-20T22:18:16.706 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.707 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.707 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:18:16.707 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:16.707 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:16.707 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:16.707 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187854 2024-09-20T22:18:16.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.707 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.707 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.707 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.632485+0000", 2024-09-20T22:18:16.707 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.707 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.707 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.632485+0000", 2024-09-20T22:18:16.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.632486+0000", 2024-09-20T22:18:16.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5099999999999999e-06 2024-09-20T22:18:16.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.708 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.708 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.708 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.632492+0000", 2024-09-20T22:18:16.708 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.054e-06 2024-09-20T22:18:16.708 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.632506+0000", 2024-09-20T22:18:16.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3801999999999999e-05 2024-09-20T22:18:16.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.632516+0000", 2024-09-20T22:18:16.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6630000000000005e-06 2024-09-20T22:18:16.709 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.709 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.709 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.709 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.639423+0000", 2024-09-20T22:18:16.709 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0069074949999999996 2024-09-20T22:18:16.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.639563+0000", 2024-09-20T22:18:16.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000139414 2024-09-20T22:18:16.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:18:16.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.639705+0000", 2024-09-20T22:18:16.710 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000142242 2024-09-20T22:18:16.710 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.710 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.710 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.710 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.666329+0000", 2024-09-20T22:18:16.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.026623709999999998 2024-09-20T22:18:16.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.666336+0000", 2024-09-20T22:18:16.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1099999999999997e-06 2024-09-20T22:18:16.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.711 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.711 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.661373+0000", 2024-09-20T22:18:16.711 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99503708499999999 2024-09-20T22:18:16.711 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.711 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.661382+0000", 2024-09-20T22:18:16.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4360000000000008e-06 2024-09-20T22:18:16.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.861335+0000", 2024-09-20T22:18:16.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1999522309999999 2024-09-20T22:18:16.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.712 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.712 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:18:16.712 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.861490+0000", 2024-09-20T22:18:16.712 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015551599999999999 2024-09-20T22:18:16.712 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.882638+0000", 2024-09-20T22:18:16.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.021147702000000001 2024-09-20T22:18:16.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.713 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.713 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.882714+0000", 2024-09-20T22:18:16.713 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6264e-05 2024-09-20T22:18:16.713 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.713 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.713 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.713 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.713 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.713 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187861 8.c 8:3c873a3d:::benchmark_data_smithi060_111051_object67862:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:18:16.714 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.662679+0000", 2024-09-20T22:18:16.714 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 466.96442797399999, 2024-09-20T22:18:16.714 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2203969990000001, 2024-09-20T22:18:16.714 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.714 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.714 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:16.714 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:16.714 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:16.714 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:16.714 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187861 2024-09-20T22:18:16.714 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.714 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.714 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.714 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.662679+0000", 2024-09-20T22:18:16.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.715 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.715 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.662679+0000", 2024-09-20T22:18:16.715 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.715 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.715 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.716 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:18:16.716 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.662680+0000", 2024-09-20T22:18:16.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.471e-06 2024-09-20T22:18:16.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.662686+0000", 2024-09-20T22:18:16.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9889999999999997e-06 2024-09-20T22:18:16.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.717 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.717 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.717 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.662710+0000", 2024-09-20T22:18:16.717 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3574000000000002e-05 2024-09-20T22:18:16.717 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.663765+0000", 2024-09-20T22:18:16.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0010555689999999999 2024-09-20T22:18:16.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.666358+0000", 2024-09-20T22:18:16.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002593181 2024-09-20T22:18:16.718 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.718 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.718 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.718 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.666365+0000", 2024-09-20T22:18:16.718 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.319e-06 2024-09-20T22:18:16.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.661413+0000", 2024-09-20T22:18:16.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.995048826 2024-09-20T22:18:16.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.661422+0000", 2024-09-20T22:18:16.719 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7600000000000008e-06 2024-09-20T22:18:16.719 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.719 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.719 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.719 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.861522+0000", 2024-09-20T22:18:16.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.200099936 2024-09-20T22:18:16.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.861627+0000", 2024-09-20T22:18:16.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000105011 2024-09-20T22:18:16.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4,5", 2024-09-20T22:18:16.720 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.862042+0000", 2024-09-20T22:18:16.720 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00041462200000000002 2024-09-20T22:18:16.720 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.720 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.720 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:16.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.871270+0000", 2024-09-20T22:18:16.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0092279939999999998 2024-09-20T22:18:16.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.871416+0000", 2024-09-20T22:18:16.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014590600000000001 2024-09-20T22:18:16.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.721 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.875258+0000", 2024-09-20T22:18:16.721 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038427219999999998 2024-09-20T22:18:16.721 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.721 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.721 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.882916+0000", 2024-09-20T22:18:16.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0076572790000000003 2024-09-20T22:18:16.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:16.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.882971+0000", 2024-09-20T22:18:16.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5485999999999997e-05 2024-09-20T22:18:16.722 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.722 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.722 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.722 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.883076+0000", 2024-09-20T22:18:16.722 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000104354 2024-09-20T22:18:16.722 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.723 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.723 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.723 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.723 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.723 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187897 8.c 8:363b713e:::benchmark_data_smithi060_111051_object67898:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:18:16.723 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.704486+0000", 2024-09-20T22:18:16.723 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 466.92262032100001, 2024-09-20T22:18:16.723 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1787772589999999, 2024-09-20T22:18:16.723 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.723 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.723 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:16.723 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:16.723 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:16.723 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:16.724 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187897 2024-09-20T22:18:16.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.724 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.704486+0000", 2024-09-20T22:18:16.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.724 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.704486+0000", 2024-09-20T22:18:16.724 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.724 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.724 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.724 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.704487+0000", 2024-09-20T22:18:16.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.048e-06 2024-09-20T22:18:16.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.704492+0000", 2024-09-20T22:18:16.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5700000000000003e-06 2024-09-20T22:18:16.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.725 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.725 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.704503+0000", 2024-09-20T22:18:16.725 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0927000000000001e-05 2024-09-20T22:18:16.725 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.725 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.704508+0000", 2024-09-20T22:18:16.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3800000000000002e-06 2024-09-20T22:18:16.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.704566+0000", 2024-09-20T22:18:16.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7751000000000003e-05 2024-09-20T22:18:16.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.726 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.726 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.726 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.704573+0000", 2024-09-20T22:18:16.726 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.109e-06 2024-09-20T22:18:16.726 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.661456+0000", 2024-09-20T22:18:16.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.956882813 2024-09-20T22:18:16.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.661465+0000", 2024-09-20T22:18:16.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2210000000000003e-06 2024-09-20T22:18:16.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.727 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.727 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.727 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.865340+0000", 2024-09-20T22:18:16.727 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2038746760000001 2024-09-20T22:18:16.727 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.865540+0000", 2024-09-20T22:18:16.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000200609 2024-09-20T22:18:16.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,4,5", 2024-09-20T22:18:16.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.865840+0000", 2024-09-20T22:18:16.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00029952499999999998 2024-09-20T22:18:16.728 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.728 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.728 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:16.728 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.871278+0000", 2024-09-20T22:18:16.728 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0054385650000000002 2024-09-20T22:18:16.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.872251+0000", 2024-09-20T22:18:16.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00097270799999999997 2024-09-20T22:18:16.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.882180+0000", 2024-09-20T22:18:16.729 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0099287290000000007 2024-09-20T22:18:16.729 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.729 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.729 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.729 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.883112+0000", 2024-09-20T22:18:16.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00093256000000000003 2024-09-20T22:18:16.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:16.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.883162+0000", 2024-09-20T22:18:16.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9713999999999999e-05 2024-09-20T22:18:16.730 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.730 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.730 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.730 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.883263+0000", 2024-09-20T22:18:16.730 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010135400000000001 2024-09-20T22:18:16.730 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.730 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.731 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.731 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.731 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.14445.0:2207 5.3 5:c609908c:::notify.5:head [watch reconnect cookie 94424494156032 gen 4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e326)", 2024-09-20T22:18:16.731 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:12:55.289536+0000", 2024-09-20T22:18:16.731 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 321.33757074800002, 2024-09-20T22:18:16.731 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5980502149999998, 2024-09-20T22:18:16.731 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.731 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.731 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:18:16.731 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:16.731 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.14445", 2024-09-20T22:18:16.731 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1594620389", 2024-09-20T22:18:16.731 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 2207 2024-09-20T22:18:16.731 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.732 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.732 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:55.289536+0000", 2024-09-20T22:18:16.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.732 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:55.289536+0000", 2024-09-20T22:18:16.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.732 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.732 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.732 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.732 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:55.289538+0000", 2024-09-20T22:18:16.732 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2400000000000002e-06 2024-09-20T22:18:16.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:55.289546+0000", 2024-09-20T22:18:16.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1470000000000001e-06 2024-09-20T22:18:16.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:55.289566+0000", 2024-09-20T22:18:16.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9882e-05 2024-09-20T22:18:16.733 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.733 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.733 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.733 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:55.289577+0000", 2024-09-20T22:18:16.733 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0463e-05 2024-09-20T22:18:16.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:55.289630+0000", 2024-09-20T22:18:16.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3919999999999999e-05 2024-09-20T22:18:16.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:55.289638+0000", 2024-09-20T22:18:16.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4050000000000003e-06 2024-09-20T22:18:16.734 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.734 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.734 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.734 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.884644+0000", 2024-09-20T22:18:16.734 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5950061209999999 2024-09-20T22:18:16.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for missing object", 2024-09-20T22:18:16.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.884779+0000", 2024-09-20T22:18:16.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013511299999999999 2024-09-20T22:18:16.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.885627+0000", 2024-09-20T22:18:16.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00084780800000000003 2024-09-20T22:18:16.735 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.735 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.735 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for rw locks", 2024-09-20T22:18:16.735 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.885763+0000", 2024-09-20T22:18:16.735 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000135721 2024-09-20T22:18:16.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.886424+0000", 2024-09-20T22:18:16.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00066162600000000001 2024-09-20T22:18:16.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.886493+0000", 2024-09-20T22:18:16.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8477000000000002e-05 2024-09-20T22:18:16.736 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.736 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.736 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.736 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.887586+0000", 2024-09-20T22:18:16.736 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001093292 2024-09-20T22:18:16.737 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.737 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.737 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.737 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.737 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:88458 9.b 9:da866086:::benchmark_data_smithi060_140132_object88457:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e328)", 2024-09-20T22:18:16.737 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:12:57.940821+0000", 2024-09-20T22:18:16.737 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 318.68628565099999, 2024-09-20T22:18:16.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1030801809999999, 2024-09-20T22:18:16.737 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.737 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.737 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:16.737 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:16.737 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:18:16.737 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:18:16.738 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 88458 2024-09-20T22:18:16.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.738 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.940821+0000", 2024-09-20T22:18:16.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.940821+0000", 2024-09-20T22:18:16.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.940821+0000", 2024-09-20T22:18:16.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8999999999999997e-07 2024-09-20T22:18:16.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.940850+0000", 2024-09-20T22:18:16.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9002000000000001e-05 2024-09-20T22:18:16.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.940857+0000", 2024-09-20T22:18:16.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1239999999999999e-06 2024-09-20T22:18:16.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.940863+0000", 2024-09-20T22:18:16.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8730000000000004e-06 2024-09-20T22:18:16.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.943270+0000", 2024-09-20T22:18:16.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024065530000000001 2024-09-20T22:18:16.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:16.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:57.943277+0000", 2024-09-20T22:18:16.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1420000000000004e-06 2024-09-20T22:18:16.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:59.026589+0000", 2024-09-20T22:18:16.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.083311457 2024-09-20T22:18:16.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:59.026742+0000", 2024-09-20T22:18:16.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000153292 2024-09-20T22:18:16.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.742 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 5,6", 2024-09-20T22:18:16.742 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:59.026881+0000", 2024-09-20T22:18:16.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013902399999999999 2024-09-20T22:18:16.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.742 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:16.742 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:59.041125+0000", 2024-09-20T22:18:16.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014243811 2024-09-20T22:18:16.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.742 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:59.043675+0000", 2024-09-20T22:18:16.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025503710000000001 2024-09-20T22:18:16.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:59.043760+0000", 2024-09-20T22:18:16.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5439999999999995e-05 2024-09-20T22:18:16.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:16.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:59.043814+0000", 2024-09-20T22:18:16.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3199999999999999e-05 2024-09-20T22:18:16.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:12:59.043901+0000", 2024-09-20T22:18:16.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7402000000000001e-05 2024-09-20T22:18:16.744 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.744 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.744 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.744 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:215003 10.8 10:1007f822:::benchmark_data_smithi060_167766_object90211:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e439)", 2024-09-20T22:18:16.744 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:04.023071+0000", 2024-09-20T22:18:16.744 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 12.604035720000001, 2024-09-20T22:18:16.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.361784681, 2024-09-20T22:18:16.744 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.745 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.745 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:16.745 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:16.745 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:16.745 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:16.745 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 215003 2024-09-20T22:18:16.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.745 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.745 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.745 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.023071+0000", 2024-09-20T22:18:16.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.745 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.023071+0000", 2024-09-20T22:18:16.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.746 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.023072+0000", 2024-09-20T22:18:16.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.553e-06 2024-09-20T22:18:16.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.746 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.023078+0000", 2024-09-20T22:18:16.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6069999999999998e-06 2024-09-20T22:18:16.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.023091+0000", 2024-09-20T22:18:16.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2945000000000001e-05 2024-09-20T22:18:16.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.023099+0000", 2024-09-20T22:18:16.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2239999999999995e-06 2024-09-20T22:18:16.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.023174+0000", 2024-09-20T22:18:16.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5165000000000003e-05 2024-09-20T22:18:16.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.023708+0000", 2024-09-20T22:18:16.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000534078 2024-09-20T22:18:16.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:18:16.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.023843+0000", 2024-09-20T22:18:16.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013443799999999999 2024-09-20T22:18:16.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:16.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.029685+0000", 2024-09-20T22:18:16.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0058418360000000004 2024-09-20T22:18:16.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.029855+0000", 2024-09-20T22:18:16.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017054599999999999 2024-09-20T22:18:16.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.384745+0000", 2024-09-20T22:18:16.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.354890233 2024-09-20T22:18:16.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:16.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.384800+0000", 2024-09-20T22:18:16.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4688000000000002e-05 2024-09-20T22:18:16.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.384855+0000", 2024-09-20T22:18:16.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5368000000000003e-05 2024-09-20T22:18:16.750 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.750 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.750 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.750 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:215007 10.8 10:133aadaf:::benchmark_data_smithi060_167766_object90215:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e439)", 2024-09-20T22:18:16.750 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:04.024616+0000", 2024-09-20T22:18:16.750 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 12.602490206000001, 2024-09-20T22:18:16.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.36035629699999999, 2024-09-20T22:18:16.751 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.751 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.751 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:16.751 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:16.751 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:16.751 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:16.751 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 215007 2024-09-20T22:18:16.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.751 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.024616+0000", 2024-09-20T22:18:16.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.024616+0000", 2024-09-20T22:18:16.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.024617+0000", 2024-09-20T22:18:16.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0049999999999999e-06 2024-09-20T22:18:16.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.024623+0000", 2024-09-20T22:18:16.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3689999999999998e-06 2024-09-20T22:18:16.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.024634+0000", 2024-09-20T22:18:16.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1235e-05 2024-09-20T22:18:16.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.024641+0000", 2024-09-20T22:18:16.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8009999999999998e-06 2024-09-20T22:18:16.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.026522+0000", 2024-09-20T22:18:16.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0018807800000000001 2024-09-20T22:18:16.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.026630+0000", 2024-09-20T22:18:16.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108774 2024-09-20T22:18:16.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:18:16.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.026757+0000", 2024-09-20T22:18:16.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000126888 2024-09-20T22:18:16.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:16.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.029878+0000", 2024-09-20T22:18:16.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003120556 2024-09-20T22:18:16.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.030546+0000", 2024-09-20T22:18:16.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00066847300000000005 2024-09-20T22:18:16.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:16.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.384896+0000", 2024-09-20T22:18:16.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.35434960900000001 2024-09-20T22:18:16.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.756 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:16.756 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.384940+0000", 2024-09-20T22:18:16.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4345999999999999e-05 2024-09-20T22:18:16.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.756 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.756 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.384973+0000", 2024-09-20T22:18:16.756 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2461e-05 2024-09-20T22:18:16.756 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.756 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.756 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.756 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.756 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.756 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:215005 10.d e439/370 10:b25cdf3e:::benchmark_data_smithi060_167766_object90213:head v 439'13286, mlcod=439'13284)", 2024-09-20T22:18:16.756 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:04.024661+0000", 2024-09-20T22:18:16.757 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 12.602445366, 2024-09-20T22:18:16.757 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.35999785899999998, 2024-09-20T22:18:16.757 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.757 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.758 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:16.758 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.758 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.758 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.758 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.024661+0000", 2024-09-20T22:18:16.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.024661+0000", 2024-09-20T22:18:16.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.024662+0000", 2024-09-20T22:18:16.759 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8400000000000005e-07 2024-09-20T22:18:16.759 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.759 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.759 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.759 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.024666+0000", 2024-09-20T22:18:16.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4610000000000001e-06 2024-09-20T22:18:16.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.024673+0000", 2024-09-20T22:18:16.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.759e-06 2024-09-20T22:18:16.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.024678+0000", 2024-09-20T22:18:16.760 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.673e-06 2024-09-20T22:18:16.760 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.760 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.760 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.760 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.026291+0000", 2024-09-20T22:18:16.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001613176 2024-09-20T22:18:16.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.026309+0000", 2024-09-20T22:18:16.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8569000000000001e-05 2024-09-20T22:18:16.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:16.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.031321+0000", 2024-09-20T22:18:16.761 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0050118819999999996 2024-09-20T22:18:16.761 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.761 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.761 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.761 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.384659+0000", 2024-09-20T22:18:16.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.35333785499999998 2024-09-20T22:18:16.762 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.762 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.762 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.762 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.762 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.762 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:215008 10.d e439/370 10:b219a629:::benchmark_data_smithi060_167766_object90216:head v 439'13287, mlcod=439'13284)", 2024-09-20T22:18:16.762 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:04.026525+0000", 2024-09-20T22:18:16.762 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 12.600581980999999, 2024-09-20T22:18:16.762 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.35817082, 2024-09-20T22:18:16.762 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.762 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.762 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:16.762 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.026525+0000", 2024-09-20T22:18:16.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.026525+0000", 2024-09-20T22:18:16.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.763 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.763 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.763 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.763 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.026525+0000", 2024-09-20T22:18:16.763 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4600000000000004e-07 2024-09-20T22:18:16.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.026531+0000", 2024-09-20T22:18:16.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4240000000000001e-06 2024-09-20T22:18:16.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.026540+0000", 2024-09-20T22:18:16.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5070000000000006e-06 2024-09-20T22:18:16.764 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.764 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.764 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.764 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.026546+0000", 2024-09-20T22:18:16.764 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1890000000000004e-06 2024-09-20T22:18:16.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.029140+0000", 2024-09-20T22:18:16.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025939209999999999 2024-09-20T22:18:16.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.029158+0000", 2024-09-20T22:18:16.765 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8184000000000001e-05 2024-09-20T22:18:16.765 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.765 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.765 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:16.765 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.384667+0000", 2024-09-20T22:18:16.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.35550825899999999 2024-09-20T22:18:16.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.766 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.766 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.384695+0000", 2024-09-20T22:18:16.766 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8690000000000001e-05 2024-09-20T22:18:16.766 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.766 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.766 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.766 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.766 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.766 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:215015 10.1 e439/370 10:8b206a59:::benchmark_data_smithi060_167766_object90223:head v 439'13347, mlcod=439'13345)", 2024-09-20T22:18:16.766 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:04.030580+0000", 2024-09-20T22:18:16.766 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 12.596526424, 2024-09-20T22:18:16.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.35517752200000002, 2024-09-20T22:18:16.767 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.767 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.767 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:16.767 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.030580+0000", 2024-09-20T22:18:16.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.767 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.767 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.030580+0000", 2024-09-20T22:18:16.767 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.767 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.767 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.030581+0000", 2024-09-20T22:18:16.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.5999999999999991e-07 2024-09-20T22:18:16.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.030587+0000", 2024-09-20T22:18:16.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2709999999999998e-06 2024-09-20T22:18:16.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.768 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.768 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.768 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.030596+0000", 2024-09-20T22:18:16.768 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1899999999999995e-06 2024-09-20T22:18:16.768 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.030601+0000", 2024-09-20T22:18:16.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5280000000000003e-06 2024-09-20T22:18:16.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.030677+0000", 2024-09-20T22:18:16.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5951000000000005e-05 2024-09-20T22:18:16.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.769 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.769 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.769 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.030695+0000", 2024-09-20T22:18:16.769 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8479e-05 2024-09-20T22:18:16.769 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:16.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.385120+0000", 2024-09-20T22:18:16.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.35442443800000001 2024-09-20T22:18:16.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.770 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.770 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.385758+0000", 2024-09-20T22:18:16.770 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00063770500000000002 2024-09-20T22:18:16.770 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.770 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.770 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.770 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.770 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.771 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:215021 10.c e439/370 10:38a5b8fe:::benchmark_data_smithi060_167766_object90229:head v 439'13381, mlcod=439'13379)", 2024-09-20T22:18:16.771 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:04.034182+0000", 2024-09-20T22:18:16.771 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 12.592924941, 2024-09-20T22:18:16.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.35997258599999998, 2024-09-20T22:18:16.771 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:16.771 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:16.771 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:16.771 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:16.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:16.771 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.034182+0000", 2024-09-20T22:18:16.771 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.771 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.771 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.771 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:16.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.034182+0000", 2024-09-20T22:18:16.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:16.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:16.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.034182+0000", 2024-09-20T22:18:16.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5899999999999996e-07 2024-09-20T22:18:16.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:16.772 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.034188+0000", 2024-09-20T22:18:16.772 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8610000000000003e-06 2024-09-20T22:18:16.772 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.772 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.772 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:16.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.384606+0000", 2024-09-20T22:18:16.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.35041786600000002 2024-09-20T22:18:16.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:16.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.384622+0000", 2024-09-20T22:18:16.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6436999999999999e-05 2024-09-20T22:18:16.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.773 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:16.773 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.384713+0000", 2024-09-20T22:18:16.773 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0331e-05 2024-09-20T22:18:16.773 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.773 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:16.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.384747+0000", 2024-09-20T22:18:16.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4133e-05 2024-09-20T22:18:16.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:16.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.394119+0000", 2024-09-20T22:18:16.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0093716579999999997 2024-09-20T22:18:16.774 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:16.774 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:16.774 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:16.774 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.394154+0000", 2024-09-20T22:18:16.774 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5641e-05 2024-09-20T22:18:16.774 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.774 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.775 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.775 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:16.775 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:16.775 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:18:16.775 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:16.776 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:16.776 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:16.877 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:16.922 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:16.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:16.923 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:17.024 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:17.057 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:17.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:17.058 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:17.159 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:17.197 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:17.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:17.198 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:17.299 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:17.332 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:17.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:17.333 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:17.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:17.472 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:17.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:17.473 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:17.574 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:17.608 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:17.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:17.609 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:17.710 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:17.743 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:17.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:17.744 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:17.845 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:17.881 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:17.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:17.882 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:17.983 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:18.005 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:18.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:18.006 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:17 smithi067 ceph-mon[33511]: pgmap v822: 178 pgs: 1 active+clean+scrubbing+deep, 177 active+clean; 1.5 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-20T22:18:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:17 smithi067 ceph-mon[33511]: osdmap e445: 8 total, 8 up, 8 in 2024-09-20T22:18:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:17 smithi067 ceph-mon[33511]: 10.7 deep-scrub ok 2024-09-20T22:18:18.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:17 smithi060 ceph-mon[29694]: pgmap v822: 178 pgs: 1 active+clean+scrubbing+deep, 177 active+clean; 1.5 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-20T22:18:18.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:17 smithi060 ceph-mon[29694]: osdmap e445: 8 total, 8 up, 8 in 2024-09-20T22:18:18.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:17 smithi060 ceph-mon[29694]: 10.7 deep-scrub ok 2024-09-20T22:18:18.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:17 smithi060 ceph-mon[40290]: pgmap v822: 178 pgs: 1 active+clean+scrubbing+deep, 177 active+clean; 1.5 GiB data, 9.6 GiB used, 706 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.05k op/s 2024-09-20T22:18:18.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:17 smithi060 ceph-mon[40290]: osdmap e445: 8 total, 8 up, 8 in 2024-09-20T22:18:18.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:17 smithi060 ceph-mon[40290]: 10.7 deep-scrub ok 2024-09-20T22:18:18.106 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:18.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:18.129 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:18.129 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:18.230 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:18.269 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:18.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:18.270 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:18.371 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:18.406 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:18.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:18.407 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:18.508 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:18.540 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:18.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:18.541 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:18.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:18.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:18.677 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:18.677 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:18.778 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:18.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:18.808 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:18.808 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:18.909 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:18.942 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:18:18.942 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:18:18.942 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-09-20T22:18:18.942 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.1 'trim stale osdmaps' 2024-09-20T22:18:18.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:18.943 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:18.944 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:19.044 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:19.081 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:19.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:19.082 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:19.183 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:19.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:19.219 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:19.219 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:19.319 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:19.363 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:19.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:19.364 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:19.465 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:19.498 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:19.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:19.499 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:19.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:19.634 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:19.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:19.635 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:19.736 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:19.771 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:19.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:19.773 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:19.874 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:19.909 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:19.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:19.910 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:20.011 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:19 smithi067 ceph-mon[33511]: pgmap v824: 178 pgs: 178 active+clean; 1.3 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.34k op/s 2024-09-20T22:18:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:19 smithi060 ceph-mon[29694]: pgmap v824: 178 pgs: 178 active+clean; 1.3 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.34k op/s 2024-09-20T22:18:20.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:19 smithi060 ceph-mon[40290]: pgmap v824: 178 pgs: 178 active+clean; 1.3 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.34k op/s 2024-09-20T22:18:20.049 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:20.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:20.050 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:20.150 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:20.181 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:20.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:20.182 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:20.283 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:20.317 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:20.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:20.317 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:20.418 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:20.457 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:20.457 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:20.458 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:20.559 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:20.596 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:20.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:20.597 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:20.698 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:20.719 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:20.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:20.720 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:20.821 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:20.844 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:20.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:20.845 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:20.946 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:20.981 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:20.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:20.982 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:20 smithi067 ceph-mon[33511]: pgmap v825: 178 pgs: 178 active+clean; 1.2 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 666 B/s rd, 0 B/s wr, 1.00k op/s 2024-09-20T22:18:21.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:20 smithi060 ceph-mon[29694]: pgmap v825: 178 pgs: 178 active+clean; 1.2 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 666 B/s rd, 0 B/s wr, 1.00k op/s 2024-09-20T22:18:21.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:20 smithi060 ceph-mon[40290]: pgmap v825: 178 pgs: 178 active+clean; 1.2 GiB data, 8.9 GiB used, 706 GiB / 715 GiB avail; 666 B/s rd, 0 B/s wr, 1.00k op/s 2024-09-20T22:18:21.083 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:21.116 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:21.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:21.117 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:21.218 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:21.251 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:21.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:21.252 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:21.352 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:21.389 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:21.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:21.390 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:21.491 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:21.501 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T22:18:21.522 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:21.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:21.524 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:21.624 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:21.650 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:21.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:21.651 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:21.659 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:21.707 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:21.752 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:21.778 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:21.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:21.779 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:21.880 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:21.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:21.924 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:21.924 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:22.025 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:22.059 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:22.060 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:22.060 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:22.161 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:22.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:22.200 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:22.200 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:22.301 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:22.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:22.335 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:22.336 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:22.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:22 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:18:22.368 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:22 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:18:22.369 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:22 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:18:22.369 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:22 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:18:22.436 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:22.471 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:22.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:22.472 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:22.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:22 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:18:22.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:22 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:18:22.573 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:22.608 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:22.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:22.609 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:22.709 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:22.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:22.758 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:22.758 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:22.815 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:22.856 INFO:teuthology.orchestra.run.smithi060.stderr:no valid command found; 10 closest matches: 2024-09-20T22:18:22.856 INFO:teuthology.orchestra.run.smithi060.stderr:0 2024-09-20T22:18:22.856 INFO:teuthology.orchestra.run.smithi060.stderr:1 2024-09-20T22:18:22.856 INFO:teuthology.orchestra.run.smithi060.stderr:2 2024-09-20T22:18:22.856 INFO:teuthology.orchestra.run.smithi060.stderr:abort 2024-09-20T22:18:22.856 INFO:teuthology.orchestra.run.smithi060.stderr:assert 2024-09-20T22:18:22.857 INFO:teuthology.orchestra.run.smithi060.stderr:bench [] [] [] [] 2024-09-20T22:18:22.857 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs debug_inject_read_zeros 2024-09-20T22:18:22.857 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs files list 2024-09-20T22:18:22.857 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs stats 2024-09-20T22:18:22.857 INFO:teuthology.orchestra.run.smithi060.stderr:bluestore allocator dump block 2024-09-20T22:18:22.857 INFO:teuthology.orchestra.run.smithi060.stderr:Error EINVAL: invalid command 2024-09-20T22:18:22.859 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:22.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:22.893 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:22.994 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:23.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:18:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:18:23.029 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:23.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:23.031 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:23.075 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:18:23.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:23.169 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:23.170 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:23.170 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:23.271 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:23.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:23 smithi060 ceph-mon[29694]: pgmap v826: 178 pgs: 178 active+clean; 1.1 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 921 op/s 2024-09-20T22:18:23.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:23 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2775789599' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:18:23.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:23 smithi060 ceph-mon[40290]: pgmap v826: 178 pgs: 178 active+clean; 1.1 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 921 op/s 2024-09-20T22:18:23.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:23 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2775789599' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:18:23.308 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:23.309 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:23.309 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:23.410 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:23.449 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:23.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:23.450 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:23.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:23 smithi067 ceph-mon[33511]: pgmap v826: 178 pgs: 178 active+clean; 1.1 GiB data, 8.2 GiB used, 707 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 921 op/s 2024-09-20T22:18:23.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:23 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2775789599' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:18:23.551 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:23.593 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:23.594 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:23.595 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:23.658 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:18:23.695 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:23.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:23.744 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:23.744 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:23.845 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:23.893 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:23.894 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:23.894 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:23.995 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:24.027 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:24.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:24.028 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:24.129 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:24.160 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:24.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:24.161 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:24.261 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:24.291 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:24.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:24.292 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:24.358 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:24 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:18:24.358 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:24 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:18:24.359 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:24 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2775789599' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:18:24.359 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:24 smithi060 ceph-mon[29694]: osdmap e446: 8 total, 8 up, 8 in 2024-09-20T22:18:24.359 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:24 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:18:24.359 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:24 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:18:24.359 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:24 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2775789599' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:18:24.359 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:24 smithi060 ceph-mon[40290]: osdmap e446: 8 total, 8 up, 8 in 2024-09-20T22:18:24.393 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:24.454 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:18:24.454 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:18:24.454 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.454 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:238061 10.3 10:c617bc97:::benchmark_data_smithi060_167766_object113269:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e446)", 2024-09-20T22:18:24.454 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:24.419189+0000", 2024-09-20T22:18:24.454 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.010091934, 2024-09-20T22:18:24.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010123607999999999, 2024-09-20T22:18:24.455 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:24.455 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:24.455 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:18:24.455 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:24.455 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:24.455 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:24.455 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 238061 2024-09-20T22:18:24.455 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.455 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:24.455 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.455 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:24.455 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.419189+0000", 2024-09-20T22:18:24.455 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:24.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:24.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.419189+0000", 2024-09-20T22:18:24.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:24.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:24.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.419191+0000", 2024-09-20T22:18:24.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2529999999999999e-06 2024-09-20T22:18:24.456 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.456 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.456 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:24.456 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.419196+0000", 2024-09-20T22:18:24.456 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2650000000000004e-06 2024-09-20T22:18:24.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:24.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.419209+0000", 2024-09-20T22:18:24.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3416e-05 2024-09-20T22:18:24.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:24.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.419218+0000", 2024-09-20T22:18:24.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6740000000000003e-06 2024-09-20T22:18:24.457 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.457 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.457 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:24.457 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.420082+0000", 2024-09-20T22:18:24.457 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00086430199999999995 2024-09-20T22:18:24.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:24.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.420208+0000", 2024-09-20T22:18:24.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012529199999999999 2024-09-20T22:18:24.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:18:24.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.420333+0000", 2024-09-20T22:18:24.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000124926 2024-09-20T22:18:24.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.458 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.458 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:24.458 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.425077+0000", 2024-09-20T22:18:24.458 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0047447959999999999 2024-09-20T22:18:24.458 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.459 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:24.459 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.425092+0000", 2024-09-20T22:18:24.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.417e-05 2024-09-20T22:18:24.459 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:24.459 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:24.459 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:24.459 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.459 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.459 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:238069 10.3 10:c9e1812a:::benchmark_data_smithi060_167766_object113277:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e446)", 2024-09-20T22:18:24.459 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:24.425095+0000", 2024-09-20T22:18:24.459 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0041860090000000001, 2024-09-20T22:18:24.459 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0043070870000000002, 2024-09-20T22:18:24.459 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:24.459 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:24.460 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:18:24.460 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:24.460 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:24.460 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:24.460 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 238069 2024-09-20T22:18:24.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.460 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:24.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:24.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.425095+0000", 2024-09-20T22:18:24.460 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:24.460 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.460 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.460 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:24.460 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.425095+0000", 2024-09-20T22:18:24.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:24.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:24.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.425097+0000", 2024-09-20T22:18:24.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2529999999999999e-06 2024-09-20T22:18:24.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:24.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.425102+0000", 2024-09-20T22:18:24.461 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9779999999999996e-06 2024-09-20T22:18:24.461 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.461 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.461 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:24.461 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.425114+0000", 2024-09-20T22:18:24.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2821e-05 2024-09-20T22:18:24.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:24.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.425122+0000", 2024-09-20T22:18:24.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3100000000000003e-06 2024-09-20T22:18:24.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:24.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.425234+0000", 2024-09-20T22:18:24.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011254100000000001 2024-09-20T22:18:24.462 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.462 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.462 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:24.462 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.425342+0000", 2024-09-20T22:18:24.462 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010771 2024-09-20T22:18:24.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.463 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:18:24.463 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.425469+0000", 2024-09-20T22:18:24.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012710900000000001 2024-09-20T22:18:24.463 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:24.463 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:24.463 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:24.463 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.463 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.463 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:238070 10.3 10:ccde9e79:::benchmark_data_smithi060_167766_object113278:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e446)", 2024-09-20T22:18:24.463 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:24.428067+0000", 2024-09-20T22:18:24.463 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0012145470000000001, 2024-09-20T22:18:24.463 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0014062829999999999, 2024-09-20T22:18:24.463 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:24.464 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:24.464 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:18:24.464 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:24.464 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:24.464 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:24.464 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 238070 2024-09-20T22:18:24.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.464 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:24.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:24.464 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.428067+0000", 2024-09-20T22:18:24.464 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:24.464 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.464 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.464 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:24.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.428067+0000", 2024-09-20T22:18:24.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:24.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:24.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.428068+0000", 2024-09-20T22:18:24.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.528e-06 2024-09-20T22:18:24.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:24.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.428074+0000", 2024-09-20T22:18:24.465 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5289999999999999e-06 2024-09-20T22:18:24.465 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.465 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.465 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:24.465 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.428091+0000", 2024-09-20T22:18:24.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7271000000000001e-05 2024-09-20T22:18:24.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:24.466 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.428099+0000", 2024-09-20T22:18:24.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8019999999999992e-06 2024-09-20T22:18:24.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:24.466 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.428666+0000", 2024-09-20T22:18:24.466 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00056723199999999998 2024-09-20T22:18:24.466 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.466 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.466 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:24.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.428807+0000", 2024-09-20T22:18:24.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014060900000000001 2024-09-20T22:18:24.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.467 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:18:24.467 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.429430+0000", 2024-09-20T22:18:24.467 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00062331699999999997 2024-09-20T22:18:24.467 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:24.467 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:24.467 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:24.467 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.467 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.467 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:238077 10.3 10.7052333 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e446)", 2024-09-20T22:18:24.467 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:24.429074+0000", 2024-09-20T22:18:24.467 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00020735, 2024-09-20T22:18:24.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00046817899999999999, 2024-09-20T22:18:24.468 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:24.468 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:24.468 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:18:24.468 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:24.468 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:24.468 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:24.468 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 238077 2024-09-20T22:18:24.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.468 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:24.468 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.468 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:24.468 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.429074+0000", 2024-09-20T22:18:24.468 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:24.468 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:24.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.429074+0000", 2024-09-20T22:18:24.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:24.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:24.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.429075+0000", 2024-09-20T22:18:24.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0330000000000001e-06 2024-09-20T22:18:24.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.469 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:24.469 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.429080+0000", 2024-09-20T22:18:24.469 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.622e-06 2024-09-20T22:18:24.469 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.469 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.470 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:24.470 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.429091+0000", 2024-09-20T22:18:24.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0886e-05 2024-09-20T22:18:24.470 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:24.470 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:24.470 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:24.470 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:24.429098+0000", 2024-09-20T22:18:24.470 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6890000000000004e-06 2024-09-20T22:18:24.470 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:24.470 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:24.470 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:24.470 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:24.470 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:18:24.470 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 4 2024-09-20T22:18:24.470 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:18:24.498 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:24.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:24.499 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:24.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:24 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:18:24.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:24 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:18:24.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:24 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2775789599' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:18:24.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:24 smithi067 ceph-mon[33511]: osdmap e446: 8 total, 8 up, 8 in 2024-09-20T22:18:24.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:24.643 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:24.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:24.644 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:24.744 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:24.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:24.784 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:24.784 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:24.884 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:24.915 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:24.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:24.916 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:24.941 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T22:18:24.941 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-20T22:18:24.942 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.2 'trim stale osdmaps' 2024-09-20T22:18:25.016 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:25.048 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:25.048 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:25.149 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:25.184 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:25.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:25.185 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:25.285 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:18:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:18:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:18:25.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:25 smithi060 ceph-mon[29694]: pgmap v828: 178 pgs: 178 active+clean; 897 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.30k op/s 2024-09-20T22:18:25.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:25 smithi060 ceph-mon[40290]: pgmap v828: 178 pgs: 178 active+clean; 897 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.30k op/s 2024-09-20T22:18:25.286 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:25.314 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:25.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:25.315 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:25.416 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:25.453 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:25.453 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:25.453 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:25.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:25 smithi067 ceph-mon[33511]: pgmap v828: 178 pgs: 178 active+clean; 897 MiB data, 7.8 GiB used, 707 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.30k op/s 2024-09-20T22:18:25.554 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:25.597 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:25.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:25.598 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:25.639 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T22:18:25.682 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:25.698 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:25.746 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:25.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:25.779 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:25.779 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:25.880 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:25.932 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:25.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:25.933 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:26.033 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:26.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:26.102 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:26.102 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:26.151 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:18:26.203 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:26.229 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:26.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:26.229 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:26.330 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:26.352 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:26.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:26.353 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:26.454 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:26.486 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:26.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:26.487 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:26.588 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:26.630 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:26.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:26.631 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:26.732 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:26.754 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:26.755 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:26.755 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:26.856 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:26.897 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:26.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:26.898 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:26.999 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:27.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:27.043 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:27.043 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:27.144 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:27.183 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:27.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:27.184 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:27.285 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:27.325 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:27.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:27.327 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:27.427 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:27.459 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:27.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:27.460 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:27.561 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:27.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:27.610 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:27.610 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:27.670 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:18:27.670 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:18:27.710 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:27.724 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:18:27.724 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:18:27.752 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:27.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:27.753 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:27.772 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:18:27.772 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:18:27.795 INFO:teuthology.orchestra.run.smithi060.stderr:no valid command found; 10 closest matches: 2024-09-20T22:18:27.795 INFO:teuthology.orchestra.run.smithi060.stderr:0 2024-09-20T22:18:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:1 2024-09-20T22:18:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:2 2024-09-20T22:18:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:abort 2024-09-20T22:18:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:assert 2024-09-20T22:18:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:bench [] [] [] [] 2024-09-20T22:18:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs debug_inject_read_zeros 2024-09-20T22:18:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs files list 2024-09-20T22:18:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs stats 2024-09-20T22:18:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:bluestore allocator dump block 2024-09-20T22:18:27.796 INFO:teuthology.orchestra.run.smithi060.stderr:Error EINVAL: invalid command 2024-09-20T22:18:27.819 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:18:27.819 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:18:27.853 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:27.855 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:18:27.856 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:18:27.887 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:18:27.887 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:18:27.888 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:27 smithi060 ceph-mon[29694]: pgmap v829: 178 pgs: 178 active+clean; 821 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.06k op/s 2024-09-20T22:18:27.888 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:27 smithi060 ceph-mon[29694]: osdmap e447: 8 total, 8 up, 8 in 2024-09-20T22:18:27.888 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:27 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:18:27.888 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:27 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1940302087' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:27.888 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:27 smithi060 ceph-mon[40290]: pgmap v829: 178 pgs: 178 active+clean; 821 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.06k op/s 2024-09-20T22:18:27.888 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:27 smithi060 ceph-mon[40290]: osdmap e447: 8 total, 8 up, 8 in 2024-09-20T22:18:27.888 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:27 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:18:27.888 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:27 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1940302087' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:27.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:27.900 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:27.900 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:27.922 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:18:27.922 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:18:27.958 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:18:27.958 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:18:28.001 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:28.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:27 smithi067 ceph-mon[33511]: pgmap v829: 178 pgs: 178 active+clean; 821 MiB data, 7.5 GiB used, 708 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1.06k op/s 2024-09-20T22:18:28.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:27 smithi067 ceph-mon[33511]: osdmap e447: 8 total, 8 up, 8 in 2024-09-20T22:18:28.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:27 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:18:28.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:27 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1940302087' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:28.038 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:28.039 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:28.040 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:28.140 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:28.174 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:28.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:28.175 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:28.225 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:28.276 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:28.322 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:28.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:28.323 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:28.424 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:28.459 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:28.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:28.460 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:28.561 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:28.597 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:28.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:28.598 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:28.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:28.730 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:28.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:28.731 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:28.831 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:28.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:28.865 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:28.866 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:28.966 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:28.993 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:28 smithi060 ceph-mon[29694]: pgmap v831: 178 pgs: 178 active+clean; 534 MiB data, 6.7 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s 2024-09-20T22:18:28.993 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:28 smithi060 ceph-mon[40290]: pgmap v831: 178 pgs: 178 active+clean; 534 MiB data, 6.7 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s 2024-09-20T22:18:28.996 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:28.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:28.997 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:29.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:28 smithi067 ceph-mon[33511]: pgmap v831: 178 pgs: 178 active+clean; 534 MiB data, 6.7 GiB used, 708 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.47k op/s 2024-09-20T22:18:29.099 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:29.127 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:29.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:29.128 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:29.229 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:29.261 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:29.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:29.262 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:29.363 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:29.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:29.402 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:29.402 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:29.503 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:29.536 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:29.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:29.537 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:29.637 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:29.645 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T22:18:29.646 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-20T22:18:29.646 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.3 'trim stale osdmaps' 2024-09-20T22:18:29.665 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:29.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:29.666 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:29.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:29.810 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:29.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:29.811 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:29.876 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:29 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:18:29.876 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:29 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:18:29.876 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:29 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:18:29.877 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:29 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:18:29.877 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:29 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:18:29.877 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:29 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:18:29.911 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:29.947 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:29.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:29.948 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:30.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:29 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:18:30.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:29 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:18:30.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:29 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:18:30.048 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:30.082 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:30.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:30.083 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:30.184 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:30.193 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:30.213 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:30.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:30.214 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:30.315 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:30.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:30.355 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:30.355 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:30.456 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:30.485 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:30.486 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:30.486 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:30.587 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:30.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:30.629 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:30.629 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:30.651 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:18:30.731 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:30.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:30.765 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:30.765 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:30.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:30 smithi060 ceph-mon[29694]: pgmap v832: 178 pgs: 178 active+clean; 511 MiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.19k op/s 2024-09-20T22:18:30.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:30 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1259191470' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:18:30.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:30 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:18:30.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:30 smithi060 ceph-mon[40290]: pgmap v832: 178 pgs: 178 active+clean; 511 MiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.19k op/s 2024-09-20T22:18:30.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:30 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1259191470' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:18:30.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:30 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:18:30.866 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:30.871 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:18:30.896 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:30.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:30.897 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:30.998 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:31.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:30 smithi067 ceph-mon[33511]: pgmap v832: 178 pgs: 178 active+clean; 511 MiB data, 6.7 GiB used, 709 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 1.19k op/s 2024-09-20T22:18:31.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:30 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1259191470' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:18:31.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:30 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:18:31.022 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:18:31.022 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:18:31.024 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:31.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:31.025 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:31.126 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:31.155 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:31.156 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:31.156 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:31.242 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:31.257 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:31.284 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:31.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:31.285 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:31.385 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:31.423 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:31.424 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:31.424 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:31.525 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:31.562 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:31.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:31.563 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:31.664 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:31.699 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:31.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:31.700 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:31.794 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T22:18:31.801 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:31.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:31.879 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:31.879 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:31.909 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:31 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:18:31.909 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:31 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:18:31.909 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:31 smithi060 ceph-mon[40290]: osdmap e448: 8 total, 8 up, 8 in 2024-09-20T22:18:31.909 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:31 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:18:31.909 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:31 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:18:31.909 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:31 smithi060 ceph-mon[40290]: osdmap e449: 8 total, 8 up, 8 in 2024-09-20T22:18:31.910 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:31 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:18:31.911 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:31 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:18:31.911 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:31 smithi060 ceph-mon[29694]: osdmap e448: 8 total, 8 up, 8 in 2024-09-20T22:18:31.911 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:31 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:18:31.911 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:31 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:18:31.911 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:31 smithi060 ceph-mon[29694]: osdmap e449: 8 total, 8 up, 8 in 2024-09-20T22:18:31.980 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:32.008 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:32.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:32.009 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:31 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:18:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:31 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:18:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:31 smithi067 ceph-mon[33511]: osdmap e448: 8 total, 8 up, 8 in 2024-09-20T22:18:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:31 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:18:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:31 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:18:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:31 smithi067 ceph-mon[33511]: osdmap e449: 8 total, 8 up, 8 in 2024-09-20T22:18:32.111 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:32.142 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:32.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:32.143 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:32.244 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:32.273 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:32.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:32.274 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:32.374 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:32.396 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:32.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:32.397 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:32.498 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:32.538 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:32.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:32.539 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:32.640 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:32.662 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:32.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:32.663 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:32.763 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:32.797 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:32.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:32.798 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:32.899 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:32.929 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:32.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:32.930 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:33.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:18:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:18:33.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:32 smithi067 ceph-mon[33511]: 6.1f scrub starts 2024-09-20T22:18:33.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:32 smithi067 ceph-mon[33511]: 6.1f scrub ok 2024-09-20T22:18:33.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:32 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:18:33.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:32 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:18:33.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:32 smithi067 ceph-mon[33511]: pgmap v835: 178 pgs: 178 active+clean; 344 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-20T22:18:33.031 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:33.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:32 smithi060 ceph-mon[29694]: 6.1f scrub starts 2024-09-20T22:18:33.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:32 smithi060 ceph-mon[29694]: 6.1f scrub ok 2024-09-20T22:18:33.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:32 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:18:33.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:32 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:18:33.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:32 smithi060 ceph-mon[29694]: pgmap v835: 178 pgs: 178 active+clean; 344 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-20T22:18:33.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:32 smithi060 ceph-mon[40290]: 6.1f scrub starts 2024-09-20T22:18:33.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:32 smithi060 ceph-mon[40290]: 6.1f scrub ok 2024-09-20T22:18:33.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:32 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:18:33.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:32 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:18:33.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:32 smithi060 ceph-mon[40290]: pgmap v835: 178 pgs: 178 active+clean; 344 MiB data, 6.1 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.27k op/s 2024-09-20T22:18:33.066 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:33.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:33.067 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:33.168 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:33.203 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:33.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:33.204 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:33.304 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:33.343 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:33.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:33.344 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:33.444 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:33.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:33.483 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:33.483 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:33.584 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:33.621 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:33.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:33.622 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:33.662 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:33.703 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:33.723 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:33.732 INFO:teuthology.orchestra.run.smithi060.stderr:no valid command found; 10 closest matches: 2024-09-20T22:18:33.732 INFO:teuthology.orchestra.run.smithi060.stderr:0 2024-09-20T22:18:33.732 INFO:teuthology.orchestra.run.smithi060.stderr:1 2024-09-20T22:18:33.732 INFO:teuthology.orchestra.run.smithi060.stderr:2 2024-09-20T22:18:33.732 INFO:teuthology.orchestra.run.smithi060.stderr:abort 2024-09-20T22:18:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:assert 2024-09-20T22:18:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:bench [] [] [] [] 2024-09-20T22:18:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs debug_inject_read_zeros 2024-09-20T22:18:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs files list 2024-09-20T22:18:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs stats 2024-09-20T22:18:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:bluestore allocator dump block 2024-09-20T22:18:33.733 INFO:teuthology.orchestra.run.smithi060.stderr:Error EINVAL: invalid command 2024-09-20T22:18:33.785 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:33.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:33.787 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:33.821 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Removed 124789 objects 2024-09-20T22:18:33.822 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Clean up completed and total clean up time :105.882 2024-09-20T22:18:33.887 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:33.909 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:33.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:33.910 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:34.010 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:34.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:33 smithi067 ceph-mon[33511]: 3.1e scrub starts 2024-09-20T22:18:34.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:33 smithi067 ceph-mon[33511]: 3.1e scrub ok 2024-09-20T22:18:34.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:33 smithi067 ceph-mon[33511]: 6.13 scrub starts 2024-09-20T22:18:34.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:33 smithi067 ceph-mon[33511]: 6.13 scrub ok 2024-09-20T22:18:34.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:33 smithi067 ceph-mon[33511]: osdmap e450: 8 total, 8 up, 8 in 2024-09-20T22:18:34.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:33 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:18:34.032 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:34.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:34.033 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:34.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:33 smithi060 ceph-mon[29694]: 3.1e scrub starts 2024-09-20T22:18:34.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:33 smithi060 ceph-mon[29694]: 3.1e scrub ok 2024-09-20T22:18:34.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:33 smithi060 ceph-mon[29694]: 6.13 scrub starts 2024-09-20T22:18:34.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:33 smithi060 ceph-mon[29694]: 6.13 scrub ok 2024-09-20T22:18:34.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:33 smithi060 ceph-mon[29694]: osdmap e450: 8 total, 8 up, 8 in 2024-09-20T22:18:34.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:33 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:18:34.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:33 smithi060 ceph-mon[40290]: 3.1e scrub starts 2024-09-20T22:18:34.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:33 smithi060 ceph-mon[40290]: 3.1e scrub ok 2024-09-20T22:18:34.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:33 smithi060 ceph-mon[40290]: 6.13 scrub starts 2024-09-20T22:18:34.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:33 smithi060 ceph-mon[40290]: 6.13 scrub ok 2024-09-20T22:18:34.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:33 smithi060 ceph-mon[40290]: osdmap e450: 8 total, 8 up, 8 in 2024-09-20T22:18:34.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:33 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:18:34.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:34.156 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:34.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:34.157 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:34.258 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:34.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:34.281 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:34.281 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:34.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:18:34.381 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:34.403 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:34.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:34.404 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:34.505 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:34.527 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:34.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:34.528 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:34.629 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:34.651 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:34.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:34.652 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:34.752 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:34.775 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:34.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:34.776 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:34.877 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:34.899 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:34.900 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T22:18:34.900 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:34.901 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-20T22:18:34.901 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.4 'trim stale osdmaps' 2024-09-20T22:18:34.901 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:34.924 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:34 smithi060 ceph-mon[29694]: 10.6 scrub starts 2024-09-20T22:18:34.924 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:34 smithi060 ceph-mon[29694]: 10.6 scrub ok 2024-09-20T22:18:34.924 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:34 smithi060 ceph-mon[29694]: pgmap v837: 178 pgs: 178 active+clean; 119 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.11k op/s 2024-09-20T22:18:34.924 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:34 smithi060 ceph-mon[40290]: 10.6 scrub starts 2024-09-20T22:18:34.924 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:34 smithi060 ceph-mon[40290]: 10.6 scrub ok 2024-09-20T22:18:34.924 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:34 smithi060 ceph-mon[40290]: pgmap v837: 178 pgs: 178 active+clean; 119 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.11k op/s 2024-09-20T22:18:35.002 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:34 smithi067 ceph-mon[33511]: 10.6 scrub starts 2024-09-20T22:18:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:34 smithi067 ceph-mon[33511]: 10.6 scrub ok 2024-09-20T22:18:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:34 smithi067 ceph-mon[33511]: pgmap v837: 178 pgs: 178 active+clean; 119 MiB data, 5.7 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1.11k op/s 2024-09-20T22:18:35.024 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:35.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:35.025 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:35.095 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:18:35.095 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:18:35.113 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:18:35.113 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:18:35.125 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:35.132 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:18:35.132 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:18:35.148 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:35.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:35.149 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:35.149 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:18:35.150 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:18:35.166 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:18:35.167 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:18:35.183 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:18:35.184 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:18:35.184 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:18:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:18:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:18:35.201 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:18:35.201 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:18:35.218 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:18:35.218 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:18:35.249 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:35.272 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:35.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:35.273 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:35.373 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:35.395 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:35.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:35.396 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:35.497 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:35.519 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:35.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:35.520 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:35.620 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:35.642 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:35.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:35.643 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:35.743 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:35.767 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:35.767 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:35.768 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:35.869 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:35.891 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:35.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:35.892 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:35.992 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:35 smithi067 ceph-mon[33511]: 5.1f scrub starts 2024-09-20T22:18:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:35 smithi067 ceph-mon[33511]: 5.1f scrub ok 2024-09-20T22:18:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:35 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3270806636' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:36.014 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:36.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:36.015 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:35 smithi060 ceph-mon[29694]: 5.1f scrub starts 2024-09-20T22:18:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:35 smithi060 ceph-mon[29694]: 5.1f scrub ok 2024-09-20T22:18:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:35 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3270806636' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:36.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:35 smithi060 ceph-mon[40290]: 5.1f scrub starts 2024-09-20T22:18:36.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:35 smithi060 ceph-mon[40290]: 5.1f scrub ok 2024-09-20T22:18:36.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:35 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3270806636' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:36.116 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:36.137 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:36.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:36.138 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:36.239 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:36.261 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:36.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:36.261 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:36.272 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:18:36.272 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:18:36.272 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:18:36.272 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:18:36.272 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.272 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:113682 10.3 10:c0482019:::benchmark_data_smithi060_167766_object113681:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e409)", 2024-09-20T22:18:36.272 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:16:38.196367+0000", 2024-09-20T22:18:36.273 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 117.16392323700001, 2024-09-20T22:18:36.273 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.177617993, 2024-09-20T22:18:36.273 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.273 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.273 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:36.273 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:36.273 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:36.273 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:36.273 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 113682 2024-09-20T22:18:36.273 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.273 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.273 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.273 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.273 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.196367+0000", 2024-09-20T22:18:36.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.196367+0000", 2024-09-20T22:18:36.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.274 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.196368+0000", 2024-09-20T22:18:36.274 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.421e-06 2024-09-20T22:18:36.274 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.274 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.274 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.196391+0000", 2024-09-20T22:18:36.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2946999999999999e-05 2024-09-20T22:18:36.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.196401+0000", 2024-09-20T22:18:36.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0097e-05 2024-09-20T22:18:36.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.275 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.196409+0000", 2024-09-20T22:18:36.275 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6939999999999995e-06 2024-09-20T22:18:36.275 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.275 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.275 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.197335+0000", 2024-09-20T22:18:36.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00092567599999999995 2024-09-20T22:18:36.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.197470+0000", 2024-09-20T22:18:36.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013541000000000001 2024-09-20T22:18:36.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:18:36.276 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.197793+0000", 2024-09-20T22:18:36.276 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00032344900000000001 2024-09-20T22:18:36.276 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.276 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.276 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:36.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.203026+0000", 2024-09-20T22:18:36.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005232647 2024-09-20T22:18:36.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.205159+0000", 2024-09-20T22:18:36.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021324640000000001 2024-09-20T22:18:36.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.277 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.277 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.373909+0000", 2024-09-20T22:18:36.277 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.16875030999999999 2024-09-20T22:18:36.277 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.277 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:36.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.373951+0000", 2024-09-20T22:18:36.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1859e-05 2024-09-20T22:18:36.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.278 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.278 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.373985+0000", 2024-09-20T22:18:36.278 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4019000000000001e-05 2024-09-20T22:18:36.278 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.278 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.278 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.278 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.278 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.278 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:113713 10.3 10:ca46d9e6:::benchmark_data_smithi060_167766_object113712:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e409)", 2024-09-20T22:18:36.279 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:16:38.219403+0000", 2024-09-20T22:18:36.279 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 117.140886763, 2024-09-20T22:18:36.279 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15901236899999999, 2024-09-20T22:18:36.279 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.279 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.279 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:36.279 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:36.279 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:36.279 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:36.279 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 113713 2024-09-20T22:18:36.279 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.279 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.279 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.279 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.219403+0000", 2024-09-20T22:18:36.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.219403+0000", 2024-09-20T22:18:36.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.280 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.219405+0000", 2024-09-20T22:18:36.280 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.694e-06 2024-09-20T22:18:36.280 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.280 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.280 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.219430+0000", 2024-09-20T22:18:36.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4777e-05 2024-09-20T22:18:36.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.219446+0000", 2024-09-20T22:18:36.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6359e-05 2024-09-20T22:18:36.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.219455+0000", 2024-09-20T22:18:36.281 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6340000000000007e-06 2024-09-20T22:18:36.281 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.281 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.281 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.281 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.219524+0000", 2024-09-20T22:18:36.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9128000000000006e-05 2024-09-20T22:18:36.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.219648+0000", 2024-09-20T22:18:36.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012458700000000001 2024-09-20T22:18:36.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:18:36.282 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.219802+0000", 2024-09-20T22:18:36.282 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015337099999999999 2024-09-20T22:18:36.282 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.282 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.282 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:36.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.223756+0000", 2024-09-20T22:18:36.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0039542939999999997 2024-09-20T22:18:36.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.225279+0000", 2024-09-20T22:18:36.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001523389 2024-09-20T22:18:36.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.283 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.378304+0000", 2024-09-20T22:18:36.283 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15302411399999999 2024-09-20T22:18:36.283 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.283 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.283 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:36.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.378366+0000", 2024-09-20T22:18:36.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2356999999999999e-05 2024-09-20T22:18:36.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.284 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.284 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.378416+0000", 2024-09-20T22:18:36.284 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9665e-05 2024-09-20T22:18:36.284 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.284 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.284 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.284 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.284 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.284 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:113723 10.3 10:c905c004:::benchmark_data_smithi060_167766_object113722:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e409)", 2024-09-20T22:18:36.284 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:16:38.229131+0000", 2024-09-20T22:18:36.285 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 117.1311587, 2024-09-20T22:18:36.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.14943024899999999, 2024-09-20T22:18:36.285 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.285 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.285 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:36.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:36.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:36.285 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:36.285 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 113723 2024-09-20T22:18:36.285 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.285 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.285 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.285 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.285 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.229131+0000", 2024-09-20T22:18:36.285 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.229131+0000", 2024-09-20T22:18:36.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.229133+0000", 2024-09-20T22:18:36.286 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7519999999999999e-06 2024-09-20T22:18:36.286 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.286 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.286 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.286 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.229157+0000", 2024-09-20T22:18:36.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3980000000000001e-05 2024-09-20T22:18:36.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.229176+0000", 2024-09-20T22:18:36.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9471999999999999e-05 2024-09-20T22:18:36.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.229184+0000", 2024-09-20T22:18:36.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9170000000000006e-06 2024-09-20T22:18:36.287 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.287 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.287 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.287 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.229251+0000", 2024-09-20T22:18:36.287 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6672000000000001e-05 2024-09-20T22:18:36.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.229416+0000", 2024-09-20T22:18:36.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016509900000000001 2024-09-20T22:18:36.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:18:36.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.229586+0000", 2024-09-20T22:18:36.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000169815 2024-09-20T22:18:36.288 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.288 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.288 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:36.288 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.232848+0000", 2024-09-20T22:18:36.288 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0032623800000000001 2024-09-20T22:18:36.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.235292+0000", 2024-09-20T22:18:36.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024431790000000002 2024-09-20T22:18:36.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.378462+0000", 2024-09-20T22:18:36.289 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.14317053699999999 2024-09-20T22:18:36.289 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.289 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.289 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:36.289 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.378528+0000", 2024-09-20T22:18:36.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6388000000000002e-05 2024-09-20T22:18:36.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.290 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.290 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:38.378562+0000", 2024-09-20T22:18:36.290 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3058000000000001e-05 2024-09-20T22:18:36.290 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.290 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.290 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.290 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.290 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.290 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:207366 10.3 10:c9a07d0f:::benchmark_data_smithi060_167766_object82574:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e437)", 2024-09-20T22:18:36.290 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:17:57.495797+0000", 2024-09-20T22:18:36.291 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 37.864492783999999, 2024-09-20T22:18:36.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.189135052, 2024-09-20T22:18:36.291 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.291 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.291 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:36.291 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:36.291 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:36.291 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:36.291 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 207366 2024-09-20T22:18:36.291 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.291 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.291 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.291 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.291 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.495797+0000", 2024-09-20T22:18:36.291 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.495797+0000", 2024-09-20T22:18:36.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.495798+0000", 2024-09-20T22:18:36.292 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.159e-06 2024-09-20T22:18:36.292 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.292 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.292 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.292 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.495803+0000", 2024-09-20T22:18:36.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.101e-06 2024-09-20T22:18:36.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.495815+0000", 2024-09-20T22:18:36.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1476e-05 2024-09-20T22:18:36.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.495822+0000", 2024-09-20T22:18:36.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0539999999999997e-06 2024-09-20T22:18:36.293 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.293 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.293 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.293 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.495873+0000", 2024-09-20T22:18:36.293 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0522999999999997e-05 2024-09-20T22:18:36.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.495990+0000", 2024-09-20T22:18:36.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117471 2024-09-20T22:18:36.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:18:36.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.496102+0000", 2024-09-20T22:18:36.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112244 2024-09-20T22:18:36.294 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.294 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.294 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.294 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.502425+0000", 2024-09-20T22:18:36.294 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0063225080000000001 2024-09-20T22:18:36.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.684657+0000", 2024-09-20T22:18:36.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.182231798 2024-09-20T22:18:36.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:36.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.684668+0000", 2024-09-20T22:18:36.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1276e-05 2024-09-20T22:18:36.295 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.295 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.295 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:36.295 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.684721+0000", 2024-09-20T22:18:36.295 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3690999999999998e-05 2024-09-20T22:18:36.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.296 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.296 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.684932+0000", 2024-09-20T22:18:36.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00021075099999999999 2024-09-20T22:18:36.296 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.296 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.296 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.296 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.296 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.296 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop_reply(client.16005.0:207366 10.3 e437/370 ondisk, result = 0)", 2024-09-20T22:18:36.296 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:17:57.502476+0000", 2024-09-20T22:18:36.296 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 37.857814466999997, 2024-09-20T22:18:36.296 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.18246290000000001, 2024-09-20T22:18:36.297 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.297 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.297 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:18:36.297 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.502476+0000", 2024-09-20T22:18:36.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.502476+0000", 2024-09-20T22:18:36.297 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.297 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.297 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.297 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.297 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.502476+0000", 2024-09-20T22:18:36.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2200000000000003e-07 2024-09-20T22:18:36.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.502482+0000", 2024-09-20T22:18:36.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6180000000000003e-06 2024-09-20T22:18:36.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.502493+0000", 2024-09-20T22:18:36.298 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1347e-05 2024-09-20T22:18:36.298 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.298 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.298 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.298 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.502738+0000", 2024-09-20T22:18:36.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000244794 2024-09-20T22:18:36.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.299 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.299 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.684643+0000", 2024-09-20T22:18:36.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.181904868 2024-09-20T22:18:36.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.299 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.299 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.684650+0000", 2024-09-20T22:18:36.299 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0559999999999999e-06 2024-09-20T22:18:36.299 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.299 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.299 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.299 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.684938+0000", 2024-09-20T22:18:36.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00028839499999999999 2024-09-20T22:18:36.300 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.300 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.300 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.300 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.300 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:207377 10.3 10:c7179adc:::benchmark_data_smithi060_167766_object82585:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e437)", 2024-09-20T22:18:36.300 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:17:57.502648+0000", 2024-09-20T22:18:36.300 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 37.857641545, 2024-09-20T22:18:36.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.18264470599999999, 2024-09-20T22:18:36.300 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.300 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.300 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:36.300 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:36.301 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:36.301 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:36.301 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 207377 2024-09-20T22:18:36.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.301 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.502648+0000", 2024-09-20T22:18:36.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.301 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.301 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.502648+0000", 2024-09-20T22:18:36.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.301 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.502650+0000", 2024-09-20T22:18:36.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5680000000000001e-06 2024-09-20T22:18:36.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.502656+0000", 2024-09-20T22:18:36.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9680000000000003e-06 2024-09-20T22:18:36.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.502668+0000", 2024-09-20T22:18:36.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2497000000000001e-05 2024-09-20T22:18:36.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.502676+0000", 2024-09-20T22:18:36.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0840000000000003e-06 2024-09-20T22:18:36.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.502733+0000", 2024-09-20T22:18:36.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7355000000000002e-05 2024-09-20T22:18:36.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.502838+0000", 2024-09-20T22:18:36.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010550100000000001 2024-09-20T22:18:36.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.304 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:18:36.304 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.502946+0000", 2024-09-20T22:18:36.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000107198 2024-09-20T22:18:36.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.685138+0000", 2024-09-20T22:18:36.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.18219234100000001 2024-09-20T22:18:36.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:36.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.685152+0000", 2024-09-20T22:18:36.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3705e-05 2024-09-20T22:18:36.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.685208+0000", 2024-09-20T22:18:36.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5923e-05 2024-09-20T22:18:36.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:36.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.685255+0000", 2024-09-20T22:18:36.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7496000000000001e-05 2024-09-20T22:18:36.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.685293+0000", 2024-09-20T22:18:36.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.807e-05 2024-09-20T22:18:36.307 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.307 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.307 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.307 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:207380 10.9 e437/370 10:94c19669:::benchmark_data_smithi060_167766_object82588:head v 437'13040, mlcod=436'13038)", 2024-09-20T22:18:36.307 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:17:57.505921+0000", 2024-09-20T22:18:36.307 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 37.854368995999998, 2024-09-20T22:18:36.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.248125243, 2024-09-20T22:18:36.307 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.308 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.308 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:36.308 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.505921+0000", 2024-09-20T22:18:36.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.505921+0000", 2024-09-20T22:18:36.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.505922+0000", 2024-09-20T22:18:36.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.08e-06 2024-09-20T22:18:36.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.505930+0000", 2024-09-20T22:18:36.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8820000000000001e-06 2024-09-20T22:18:36.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.681643+0000", 2024-09-20T22:18:36.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.17571337400000001 2024-09-20T22:18:36.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.681657+0000", 2024-09-20T22:18:36.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3771000000000001e-05 2024-09-20T22:18:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.681814+0000", 2024-09-20T22:18:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015706800000000001 2024-09-20T22:18:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.681835+0000", 2024-09-20T22:18:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0364999999999999e-05 2024-09-20T22:18:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:36.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.753959+0000", 2024-09-20T22:18:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.072124634000000007 2024-09-20T22:18:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.754046+0000", 2024-09-20T22:18:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7069000000000003e-05 2024-09-20T22:18:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:207381 10.e e437/370 10:7538be57:::benchmark_data_smithi060_167766_object82589:head v 437'12739, mlcod=436'12737)", 2024-09-20T22:18:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:17:57.507026+0000", 2024-09-20T22:18:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 37.853263493, 2024-09-20T22:18:36.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.17859319700000001, 2024-09-20T22:18:36.312 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.312 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.312 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:36.312 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.507026+0000", 2024-09-20T22:18:36.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.507026+0000", 2024-09-20T22:18:36.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.507027+0000", 2024-09-20T22:18:36.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4299999999999999e-07 2024-09-20T22:18:36.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.507032+0000", 2024-09-20T22:18:36.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8690000000000003e-06 2024-09-20T22:18:36.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.681648+0000", 2024-09-20T22:18:36.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.17461615899999999 2024-09-20T22:18:36.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.314 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.681657+0000", 2024-09-20T22:18:36.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.4509999999999998e-06 2024-09-20T22:18:36.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.314 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.681731+0000", 2024-09-20T22:18:36.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3310000000000001e-05 2024-09-20T22:18:36.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.314 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.681754+0000", 2024-09-20T22:18:36.314 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3672999999999998e-05 2024-09-20T22:18:36.314 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.314 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.314 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:36.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.685535+0000", 2024-09-20T22:18:36.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003781244 2024-09-20T22:18:36.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.685620+0000", 2024-09-20T22:18:36.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4247999999999997e-05 2024-09-20T22:18:36.315 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.315 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.315 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.315 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop_reply(client.16005.0:207377 10.3 e437/370 ondisk, result = 0)", 2024-09-20T22:18:36.315 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:17:57.508271+0000", 2024-09-20T22:18:36.315 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 37.852018882000003, 2024-09-20T22:18:36.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.176882497, 2024-09-20T22:18:36.316 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.316 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.316 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:18:36.316 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.508271+0000", 2024-09-20T22:18:36.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.508271+0000", 2024-09-20T22:18:36.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.508272+0000", 2024-09-20T22:18:36.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5799999999999998e-07 2024-09-20T22:18:36.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.508278+0000", 2024-09-20T22:18:36.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1430000000000003e-06 2024-09-20T22:18:36.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.681647+0000", 2024-09-20T22:18:36.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.173368733 2024-09-20T22:18:36.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.681657+0000", 2024-09-20T22:18:36.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9410000000000002e-06 2024-09-20T22:18:36.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.685124+0000", 2024-09-20T22:18:36.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0034670199999999999 2024-09-20T22:18:36.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.685132+0000", 2024-09-20T22:18:36.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4239999999999993e-06 2024-09-20T22:18:36.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:57.685154+0000", 2024-09-20T22:18:36.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1478e-05 2024-09-20T22:18:36.319 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.319 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.319 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.319 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:209425 10.5 10:a8219775:::benchmark_data_smithi060_167766_object84633:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e437)", 2024-09-20T22:18:36.319 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:17:59.255827+0000", 2024-09-20T22:18:36.319 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 36.104462640000001, 2024-09-20T22:18:36.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.13759189199999999, 2024-09-20T22:18:36.319 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.319 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.319 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:36.320 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:36.320 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:36.320 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:36.320 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 209425 2024-09-20T22:18:36.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.320 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:59.255827+0000", 2024-09-20T22:18:36.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:59.255827+0000", 2024-09-20T22:18:36.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:59.255829+0000", 2024-09-20T22:18:36.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.297e-06 2024-09-20T22:18:36.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:59.255833+0000", 2024-09-20T22:18:36.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.848e-06 2024-09-20T22:18:36.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:59.255844+0000", 2024-09-20T22:18:36.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.022e-05 2024-09-20T22:18:36.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.322 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.322 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:59.255851+0000", 2024-09-20T22:18:36.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8020000000000003e-06 2024-09-20T22:18:36.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.322 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.322 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:59.255903+0000", 2024-09-20T22:18:36.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2856999999999999e-05 2024-09-20T22:18:36.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.322 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.322 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:59.256020+0000", 2024-09-20T22:18:36.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000116742 2024-09-20T22:18:36.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:18:36.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:59.256225+0000", 2024-09-20T22:18:36.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020452 2024-09-20T22:18:36.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:36.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:59.259910+0000", 2024-09-20T22:18:36.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0036852220000000002 2024-09-20T22:18:36.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:59.261874+0000", 2024-09-20T22:18:36.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019638569999999998 2024-09-20T22:18:36.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:59.393337+0000", 2024-09-20T22:18:36.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.131463466 2024-09-20T22:18:36.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:36.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:59.393371+0000", 2024-09-20T22:18:36.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.4010999999999997e-05 2024-09-20T22:18:36.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:17:59.393419+0000", 2024-09-20T22:18:36.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8050000000000002e-05 2024-09-20T22:18:36.325 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.325 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.325 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.325 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:215026 10.5 10:a33075d0:::benchmark_data_smithi060_167766_object90234:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e439)", 2024-09-20T22:18:36.325 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:04.039598+0000", 2024-09-20T22:18:36.325 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 31.320691812, 2024-09-20T22:18:36.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.36301854300000003, 2024-09-20T22:18:36.325 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.325 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.325 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:36.325 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:36.325 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:36.325 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:36.325 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 215026 2024-09-20T22:18:36.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.326 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.039598+0000", 2024-09-20T22:18:36.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.039598+0000", 2024-09-20T22:18:36.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.039599+0000", 2024-09-20T22:18:36.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1069999999999999e-06 2024-09-20T22:18:36.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.039603+0000", 2024-09-20T22:18:36.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8999999999999999e-06 2024-09-20T22:18:36.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.039618+0000", 2024-09-20T22:18:36.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4909000000000001e-05 2024-09-20T22:18:36.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.039625+0000", 2024-09-20T22:18:36.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7259999999999996e-06 2024-09-20T22:18:36.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.039682+0000", 2024-09-20T22:18:36.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6762000000000003e-05 2024-09-20T22:18:36.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.039815+0000", 2024-09-20T22:18:36.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013338099999999999 2024-09-20T22:18:36.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:18:36.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.039896+0000", 2024-09-20T22:18:36.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0711999999999996e-05 2024-09-20T22:18:36.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.042863+0000", 2024-09-20T22:18:36.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0029671649999999999 2024-09-20T22:18:36.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:36.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.042905+0000", 2024-09-20T22:18:36.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2035e-05 2024-09-20T22:18:36.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.402510+0000", 2024-09-20T22:18:36.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.35960535199999999 2024-09-20T22:18:36.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:36.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.402569+0000", 2024-09-20T22:18:36.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8480000000000003e-05 2024-09-20T22:18:36.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.402617+0000", 2024-09-20T22:18:36.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8013999999999998e-05 2024-09-20T22:18:36.330 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.330 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.330 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.331 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:215035 10.5 10:aefada6b:::benchmark_data_smithi060_167766_object90243:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e439)", 2024-09-20T22:18:36.331 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:04.052366+0000", 2024-09-20T22:18:36.331 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 31.307924051000001, 2024-09-20T22:18:36.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.35043473800000002, 2024-09-20T22:18:36.331 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.331 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.331 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:36.331 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:36.331 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:36.331 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:36.331 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 215035 2024-09-20T22:18:36.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.331 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.052366+0000", 2024-09-20T22:18:36.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.052366+0000", 2024-09-20T22:18:36.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.052367+0000", 2024-09-20T22:18:36.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9700000000000005e-07 2024-09-20T22:18:36.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.052371+0000", 2024-09-20T22:18:36.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.924e-06 2024-09-20T22:18:36.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.052382+0000", 2024-09-20T22:18:36.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0939e-05 2024-09-20T22:18:36.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.052387+0000", 2024-09-20T22:18:36.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4929999999999998e-06 2024-09-20T22:18:36.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.052427+0000", 2024-09-20T22:18:36.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0083999999999998e-05 2024-09-20T22:18:36.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.052508+0000", 2024-09-20T22:18:36.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0334999999999996e-05 2024-09-20T22:18:36.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:18:36.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.052579+0000", 2024-09-20T22:18:36.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1225999999999999e-05 2024-09-20T22:18:36.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:36.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.054839+0000", 2024-09-20T22:18:36.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002260626 2024-09-20T22:18:36.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.055105+0000", 2024-09-20T22:18:36.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026582999999999999 2024-09-20T22:18:36.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.402669+0000", 2024-09-20T22:18:36.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.34756410599999998 2024-09-20T22:18:36.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.335 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:36.335 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.402748+0000", 2024-09-20T22:18:36.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8604999999999997e-05 2024-09-20T22:18:36.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.402801+0000", 2024-09-20T22:18:36.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2673000000000002e-05 2024-09-20T22:18:36.336 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.336 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.336 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.336 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:215068 10.3 10:c0c0e10d:::benchmark_data_smithi060_167766_object90276:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e439)", 2024-09-20T22:18:36.336 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:04.409293+0000", 2024-09-20T22:18:36.336 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 30.950996815, 2024-09-20T22:18:36.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.16057969899999999, 2024-09-20T22:18:36.337 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.337 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.337 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:36.337 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:36.337 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:36.337 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:36.337 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 215068 2024-09-20T22:18:36.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.337 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.409293+0000", 2024-09-20T22:18:36.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.409293+0000", 2024-09-20T22:18:36.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.409294+0000", 2024-09-20T22:18:36.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8899999999999998e-07 2024-09-20T22:18:36.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.409299+0000", 2024-09-20T22:18:36.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9030000000000003e-06 2024-09-20T22:18:36.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.409310+0000", 2024-09-20T22:18:36.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0932e-05 2024-09-20T22:18:36.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.409318+0000", 2024-09-20T22:18:36.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7020000000000001e-06 2024-09-20T22:18:36.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.409378+0000", 2024-09-20T22:18:36.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0216000000000001e-05 2024-09-20T22:18:36.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.409492+0000", 2024-09-20T22:18:36.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114061 2024-09-20T22:18:36.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:18:36.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.409556+0000", 2024-09-20T22:18:36.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4486000000000006e-05 2024-09-20T22:18:36.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:36.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.412893+0000", 2024-09-20T22:18:36.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0033366540000000001 2024-09-20T22:18:36.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.569692+0000", 2024-09-20T22:18:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15679864299999999 2024-09-20T22:18:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.569760+0000", 2024-09-20T22:18:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8408e-05 2024-09-20T22:18:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.569824+0000", 2024-09-20T22:18:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3396000000000001e-05 2024-09-20T22:18:36.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.569873+0000", 2024-09-20T22:18:36.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9308999999999998e-05 2024-09-20T22:18:36.342 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.342 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.342 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.342 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop_reply(client.16005.0:215068 10.3 e439/370 ondisk, result = 0)", 2024-09-20T22:18:36.342 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:04.413267+0000", 2024-09-20T22:18:36.342 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 30.947022504, 2024-09-20T22:18:36.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15644408500000001, 2024-09-20T22:18:36.342 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.342 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.343 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:18:36.343 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.413267+0000", 2024-09-20T22:18:36.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.413267+0000", 2024-09-20T22:18:36.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.413268+0000", 2024-09-20T22:18:36.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-07 2024-09-20T22:18:36.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.413273+0000", 2024-09-20T22:18:36.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1080000000000001e-06 2024-09-20T22:18:36.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.413285+0000", 2024-09-20T22:18:36.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1263e-05 2024-09-20T22:18:36.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.413292+0000", 2024-09-20T22:18:36.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0010000000000004e-06 2024-09-20T22:18:36.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.413349+0000", 2024-09-20T22:18:36.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7450000000000001e-05 2024-09-20T22:18:36.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.413357+0000", 2024-09-20T22:18:36.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7610000000000007e-06 2024-09-20T22:18:36.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.569712+0000", 2024-09-20T22:18:36.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15635468999999999 2024-09-20T22:18:36.345 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.345 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.346 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.346 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:215072 10.f e439/370 10:f34ae14c:::benchmark_data_smithi060_167766_object90280:head v 439'13299, mlcod=439'13297)", 2024-09-20T22:18:36.346 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:04.413346+0000", 2024-09-20T22:18:36.346 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 30.946944326000001, 2024-09-20T22:18:36.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.16378674700000001, 2024-09-20T22:18:36.346 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.346 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.346 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:36.346 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.413346+0000", 2024-09-20T22:18:36.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.413346+0000", 2024-09-20T22:18:36.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.413346+0000", 2024-09-20T22:18:36.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8699999999999995e-07 2024-09-20T22:18:36.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.413351+0000", 2024-09-20T22:18:36.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0250000000000002e-06 2024-09-20T22:18:36.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.569941+0000", 2024-09-20T22:18:36.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15658952400000001 2024-09-20T22:18:36.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.569947+0000", 2024-09-20T22:18:36.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1e-06 2024-09-20T22:18:36.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.570041+0000", 2024-09-20T22:18:36.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3702999999999994e-05 2024-09-20T22:18:36.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.570059+0000", 2024-09-20T22:18:36.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8342000000000002e-05 2024-09-20T22:18:36.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:36.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.577091+0000", 2024-09-20T22:18:36.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0070320959999999998 2024-09-20T22:18:36.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.577132+0000", 2024-09-20T22:18:36.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1369999999999999e-05 2024-09-20T22:18:36.349 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.350 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.350 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.350 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop_reply(client.16005.0:215068 10.3 e439/370 ondisk, result = 0)", 2024-09-20T22:18:36.350 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:04.413473+0000", 2024-09-20T22:18:36.350 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 30.946816552000001, 2024-09-20T22:18:36.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.156405089, 2024-09-20T22:18:36.350 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.350 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.350 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:18:36.350 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.413473+0000", 2024-09-20T22:18:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.413473+0000", 2024-09-20T22:18:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.413474+0000", 2024-09-20T22:18:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0999999999999998e-07 2024-09-20T22:18:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.413478+0000", 2024-09-20T22:18:36.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.206e-06 2024-09-20T22:18:36.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.569684+0000", 2024-09-20T22:18:36.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.15620608699999999 2024-09-20T22:18:36.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.569698+0000", 2024-09-20T22:18:36.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3390000000000001e-05 2024-09-20T22:18:36.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.569747+0000", 2024-09-20T22:18:36.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8906999999999998e-05 2024-09-20T22:18:36.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.569755+0000", 2024-09-20T22:18:36.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9589999999999995e-06 2024-09-20T22:18:36.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.353 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.569879+0000", 2024-09-20T22:18:36.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012392999999999999 2024-09-20T22:18:36.353 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.353 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.353 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.354 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:233489 10.5 10:ae485314:::benchmark_data_smithi060_167766_object108697:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e445)", 2024-09-20T22:18:36.354 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:20.594889+0000", 2024-09-20T22:18:36.354 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.765400608, 2024-09-20T22:18:36.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.30263380299999998, 2024-09-20T22:18:36.354 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.354 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.354 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:36.354 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:36.354 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:36.354 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:36.354 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 233489 2024-09-20T22:18:36.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.354 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.594889+0000", 2024-09-20T22:18:36.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.594889+0000", 2024-09-20T22:18:36.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.594891+0000", 2024-09-20T22:18:36.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8059999999999999e-06 2024-09-20T22:18:36.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.594897+0000", 2024-09-20T22:18:36.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4589999999999997e-06 2024-09-20T22:18:36.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.594907+0000", 2024-09-20T22:18:36.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0464e-05 2024-09-20T22:18:36.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.594914+0000", 2024-09-20T22:18:36.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9059999999999996e-06 2024-09-20T22:18:36.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.595004+0000", 2024-09-20T22:18:36.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0039999999999999e-05 2024-09-20T22:18:36.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.595431+0000", 2024-09-20T22:18:36.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00042740999999999998 2024-09-20T22:18:36.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:18:36.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.595546+0000", 2024-09-20T22:18:36.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000114324 2024-09-20T22:18:36.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:36.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.602410+0000", 2024-09-20T22:18:36.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0068646899999999997 2024-09-20T22:18:36.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.603255+0000", 2024-09-20T22:18:36.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00084478500000000002 2024-09-20T22:18:36.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.897410+0000", 2024-09-20T22:18:36.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.29415432000000002 2024-09-20T22:18:36.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:36.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.897469+0000", 2024-09-20T22:18:36.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9462000000000001e-05 2024-09-20T22:18:36.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.897523+0000", 2024-09-20T22:18:36.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4137e-05 2024-09-20T22:18:36.359 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.359 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.359 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.359 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:233509 10.5 10:a2aeeffa:::benchmark_data_smithi060_167766_object108717:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e445)", 2024-09-20T22:18:36.360 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:20.611744+0000", 2024-09-20T22:18:36.360 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 14.748545947, 2024-09-20T22:18:36.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.28642404500000002, 2024-09-20T22:18:36.360 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.360 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.360 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:36.360 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:36.360 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:36.360 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:36.360 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 233509 2024-09-20T22:18:36.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.360 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.611744+0000", 2024-09-20T22:18:36.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.611744+0000", 2024-09-20T22:18:36.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.611745+0000", 2024-09-20T22:18:36.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1090000000000001e-06 2024-09-20T22:18:36.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.611750+0000", 2024-09-20T22:18:36.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2000000000000002e-06 2024-09-20T22:18:36.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.611761+0000", 2024-09-20T22:18:36.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0851e-05 2024-09-20T22:18:36.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.611769+0000", 2024-09-20T22:18:36.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7389999999999999e-06 2024-09-20T22:18:36.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.611827+0000", 2024-09-20T22:18:36.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8396000000000001e-05 2024-09-20T22:18:36.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.611914+0000", 2024-09-20T22:18:36.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7174000000000003e-05 2024-09-20T22:18:36.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:18:36.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.611987+0000", 2024-09-20T22:18:36.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2663999999999994e-05 2024-09-20T22:18:36.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:36.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.614403+0000", 2024-09-20T22:18:36.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024154770000000001 2024-09-20T22:18:36.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.614810+0000", 2024-09-20T22:18:36.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00040694700000000001 2024-09-20T22:18:36.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.898017+0000", 2024-09-20T22:18:36.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.283207659 2024-09-20T22:18:36.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:36.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.898122+0000", 2024-09-20T22:18:36.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010460599999999999 2024-09-20T22:18:36.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:20.898168+0000", 2024-09-20T22:18:36.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6223000000000001e-05 2024-09-20T22:18:36.365 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.365 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.365 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.365 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:240327 10.5 10:a6c7a20c:::benchmark_data_smithi060_167766_object115535:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e446)", 2024-09-20T22:18:36.365 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:26.273877+0000", 2024-09-20T22:18:36.365 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 9.0864126079999998, 2024-09-20T22:18:36.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.14098437699999999, 2024-09-20T22:18:36.366 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.366 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.366 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:36.366 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:36.366 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:36.366 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:36.366 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 240327 2024-09-20T22:18:36.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.366 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.366 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.366 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.273877+0000", 2024-09-20T22:18:36.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.273877+0000", 2024-09-20T22:18:36.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.273878+0000", 2024-09-20T22:18:36.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4700000000000001e-07 2024-09-20T22:18:36.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.273882+0000", 2024-09-20T22:18:36.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0890000000000002e-06 2024-09-20T22:18:36.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.273893+0000", 2024-09-20T22:18:36.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0353000000000001e-05 2024-09-20T22:18:36.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.273899+0000", 2024-09-20T22:18:36.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7410000000000003e-06 2024-09-20T22:18:36.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.281502+0000", 2024-09-20T22:18:36.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0076025889999999999 2024-09-20T22:18:36.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.281631+0000", 2024-09-20T22:18:36.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012875000000000001 2024-09-20T22:18:36.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:18:36.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.281781+0000", 2024-09-20T22:18:36.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015022200000000001 2024-09-20T22:18:36.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:36.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.288975+0000", 2024-09-20T22:18:36.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0071936980000000001 2024-09-20T22:18:36.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.303682+0000", 2024-09-20T22:18:36.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.014707039 2024-09-20T22:18:36.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.414683+0000", 2024-09-20T22:18:36.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.111000917 2024-09-20T22:18:36.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:36.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.414808+0000", 2024-09-20T22:18:36.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012597899999999999 2024-09-20T22:18:36.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.414862+0000", 2024-09-20T22:18:36.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3253e-05 2024-09-20T22:18:36.371 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.371 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.371 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.371 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:240330 10.5 10:a5626458:::benchmark_data_smithi060_167766_object115538:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e446)", 2024-09-20T22:18:36.371 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:26.273910+0000", 2024-09-20T22:18:36.371 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 9.0863797999999996, 2024-09-20T22:18:36.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.14456706899999999, 2024-09-20T22:18:36.371 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:36.371 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:36.371 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:36.372 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:36.372 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:18:36.372 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:18:36.372 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 240330 2024-09-20T22:18:36.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.372 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:36.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:36.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.273910+0000", 2024-09-20T22:18:36.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:36.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.273910+0000", 2024-09-20T22:18:36.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:36.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:36.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.273911+0000", 2024-09-20T22:18:36.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5400000000000002e-07 2024-09-20T22:18:36.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:36.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.273913+0000", 2024-09-20T22:18:36.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4839999999999998e-06 2024-09-20T22:18:36.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:36.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.273920+0000", 2024-09-20T22:18:36.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8040000000000004e-06 2024-09-20T22:18:36.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:36.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.273925+0000", 2024-09-20T22:18:36.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8199999999999996e-06 2024-09-20T22:18:36.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:36.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.285185+0000", 2024-09-20T22:18:36.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.011259918000000001 2024-09-20T22:18:36.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:36.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.285304+0000", 2024-09-20T22:18:36.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119423 2024-09-20T22:18:36.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:18:36.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.285754+0000", 2024-09-20T22:18:36.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000450087 2024-09-20T22:18:36.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:36.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.292755+0000", 2024-09-20T22:18:36.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0070007300000000001 2024-09-20T22:18:36.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.304022+0000", 2024-09-20T22:18:36.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.011267396000000001 2024-09-20T22:18:36.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:36.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.418382+0000", 2024-09-20T22:18:36.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.114360013 2024-09-20T22:18:36.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:36.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.418434+0000", 2024-09-20T22:18:36.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2033000000000001e-05 2024-09-20T22:18:36.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:36.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:36.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:36.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.418477+0000", 2024-09-20T22:18:36.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2907000000000001e-05 2024-09-20T22:18:36.377 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.377 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.377 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.377 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:36.377 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:36.377 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:18:36.377 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:36.404 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:36.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:36.405 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:36.506 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:36.533 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:36.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:36.533 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:36.634 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:36.657 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:36.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:36.658 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:36.759 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:36.781 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:36.781 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:36.782 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:36.882 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:36.909 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:36.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:36.910 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:36.985 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[29694]: 4.1a scrub starts 2024-09-20T22:18:36.986 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[29694]: 4.1a scrub ok 2024-09-20T22:18:36.986 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[29694]: 6.5 scrub starts 2024-09-20T22:18:36.986 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[29694]: 6.5 scrub ok 2024-09-20T22:18:36.986 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:18:36.986 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[29694]: 5.17 scrub starts 2024-09-20T22:18:36.986 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[29694]: 5.17 scrub ok 2024-09-20T22:18:36.986 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[29694]: pgmap v838: 178 pgs: 178 active+clean; 78 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-09-20T22:18:36.986 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[29694]: 4.e scrub starts 2024-09-20T22:18:36.986 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[29694]: 4.e scrub ok 2024-09-20T22:18:36.986 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[29694]: osdmap e451: 8 total, 8 up, 8 in 2024-09-20T22:18:36.986 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[40290]: 4.1a scrub starts 2024-09-20T22:18:36.986 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[40290]: 4.1a scrub ok 2024-09-20T22:18:36.986 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[40290]: 6.5 scrub starts 2024-09-20T22:18:36.987 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[40290]: 6.5 scrub ok 2024-09-20T22:18:36.987 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:18:36.987 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[40290]: 5.17 scrub starts 2024-09-20T22:18:36.987 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[40290]: 5.17 scrub ok 2024-09-20T22:18:36.987 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[40290]: pgmap v838: 178 pgs: 178 active+clean; 78 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-09-20T22:18:36.987 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[40290]: 4.e scrub starts 2024-09-20T22:18:36.987 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[40290]: 4.e scrub ok 2024-09-20T22:18:36.987 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:36 smithi060 ceph-mon[40290]: osdmap e451: 8 total, 8 up, 8 in 2024-09-20T22:18:37.011 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:36 smithi067 ceph-mon[33511]: 4.1a scrub starts 2024-09-20T22:18:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:36 smithi067 ceph-mon[33511]: 4.1a scrub ok 2024-09-20T22:18:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:36 smithi067 ceph-mon[33511]: 6.5 scrub starts 2024-09-20T22:18:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:36 smithi067 ceph-mon[33511]: 6.5 scrub ok 2024-09-20T22:18:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:36 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:18:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:36 smithi067 ceph-mon[33511]: 5.17 scrub starts 2024-09-20T22:18:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:36 smithi067 ceph-mon[33511]: 5.17 scrub ok 2024-09-20T22:18:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:36 smithi067 ceph-mon[33511]: pgmap v838: 178 pgs: 178 active+clean; 78 MiB data, 5.5 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.15k op/s 2024-09-20T22:18:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:36 smithi067 ceph-mon[33511]: 4.e scrub starts 2024-09-20T22:18:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:36 smithi067 ceph-mon[33511]: 4.e scrub ok 2024-09-20T22:18:37.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:36 smithi067 ceph-mon[33511]: osdmap e451: 8 total, 8 up, 8 in 2024-09-20T22:18:37.019 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T22:18:37.035 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:37.035 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:37.038 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:37.038 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:37.038 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:37.139 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:37.162 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:37.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:37.163 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:37.263 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:37.290 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:37.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:37.291 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:37.392 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:37.418 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:37.419 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:37.419 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:37.520 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:37.547 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:37.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:37.548 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:37.648 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:37.677 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:37.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:37.678 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:37.779 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:37.801 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:37.802 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:37.802 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:37.903 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:37.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:37.925 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:37.925 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:37 smithi067 ceph-mon[33511]: 5.16 scrub starts 2024-09-20T22:18:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:37 smithi067 ceph-mon[33511]: 5.16 scrub ok 2024-09-20T22:18:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:37 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:18:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:37 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:18:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:37 smithi067 ceph-mon[33511]: 6.1d scrub starts 2024-09-20T22:18:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:37 smithi067 ceph-mon[33511]: 6.1d scrub ok 2024-09-20T22:18:38.017 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:37 smithi060 ceph-mon[29694]: 5.16 scrub starts 2024-09-20T22:18:38.017 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:37 smithi060 ceph-mon[29694]: 5.16 scrub ok 2024-09-20T22:18:38.017 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:37 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:18:38.017 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:37 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:18:38.017 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:37 smithi060 ceph-mon[29694]: 6.1d scrub starts 2024-09-20T22:18:38.017 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:37 smithi060 ceph-mon[29694]: 6.1d scrub ok 2024-09-20T22:18:38.018 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:37 smithi060 ceph-mon[40290]: 5.16 scrub starts 2024-09-20T22:18:38.018 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:37 smithi060 ceph-mon[40290]: 5.16 scrub ok 2024-09-20T22:18:38.018 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:37 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:18:38.018 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:37 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:18:38.018 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:37 smithi060 ceph-mon[40290]: 6.1d scrub starts 2024-09-20T22:18:38.018 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:37 smithi060 ceph-mon[40290]: 6.1d scrub ok 2024-09-20T22:18:38.018 INFO:teuthology.orchestra.run.smithi060.stderr:no valid command found; 10 closest matches: 2024-09-20T22:18:38.018 INFO:teuthology.orchestra.run.smithi060.stderr:0 2024-09-20T22:18:38.018 INFO:teuthology.orchestra.run.smithi060.stderr:1 2024-09-20T22:18:38.018 INFO:teuthology.orchestra.run.smithi060.stderr:2 2024-09-20T22:18:38.018 INFO:teuthology.orchestra.run.smithi060.stderr:abort 2024-09-20T22:18:38.018 INFO:teuthology.orchestra.run.smithi060.stderr:assert 2024-09-20T22:18:38.019 INFO:teuthology.orchestra.run.smithi060.stderr:bench [] [] [] [] 2024-09-20T22:18:38.019 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs debug_inject_read_zeros 2024-09-20T22:18:38.019 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs files list 2024-09-20T22:18:38.019 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs stats 2024-09-20T22:18:38.019 INFO:teuthology.orchestra.run.smithi060.stderr:bluestore allocator dump block 2024-09-20T22:18:38.019 INFO:teuthology.orchestra.run.smithi060.stderr:Error EINVAL: invalid command 2024-09-20T22:18:38.026 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:38.053 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:38.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:38.054 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:38.123 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:18:38.155 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:38.181 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:38.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:38.182 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:38.265 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:38.283 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:38.305 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:38.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:38.306 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:38.406 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:38.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:38.430 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:38.430 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:38.531 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:38.553 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:38.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:38.553 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:38.654 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:38.674 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:18:38.683 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:38.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:38.684 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:38.785 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:38.812 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:38.812 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T22:18:38.813 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-20T22:18:38.813 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.5 'trim stale osdmaps' 2024-09-20T22:18:38.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:38.814 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:38.915 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:38.942 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:38.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:38.943 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:38 smithi067 ceph-mon[33511]: osdmap e452: 8 total, 8 up, 8 in 2024-09-20T22:18:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:38 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:18:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:38 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:18:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:38 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1469862109' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:18:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:38 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:18:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:38 smithi067 ceph-mon[33511]: pgmap v841: 178 pgs: 178 active+clean; 2.5 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 911 op/s 2024-09-20T22:18:39.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:38 smithi067 ceph-mon[33511]: 10.1 scrub starts 2024-09-20T22:18:39.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:38 smithi067 ceph-mon[33511]: 10.1 scrub ok 2024-09-20T22:18:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:38 smithi060 ceph-mon[29694]: osdmap e452: 8 total, 8 up, 8 in 2024-09-20T22:18:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:38 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:18:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:38 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:18:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:38 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1469862109' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:18:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:38 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:18:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:38 smithi060 ceph-mon[29694]: pgmap v841: 178 pgs: 178 active+clean; 2.5 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 911 op/s 2024-09-20T22:18:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:38 smithi060 ceph-mon[29694]: 10.1 scrub starts 2024-09-20T22:18:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:38 smithi060 ceph-mon[29694]: 10.1 scrub ok 2024-09-20T22:18:39.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:38 smithi060 ceph-mon[40290]: osdmap e452: 8 total, 8 up, 8 in 2024-09-20T22:18:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:38 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:18:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:38 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:18:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:38 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1469862109' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:18:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:38 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:18:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:38 smithi060 ceph-mon[40290]: pgmap v841: 178 pgs: 178 active+clean; 2.5 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 911 op/s 2024-09-20T22:18:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:38 smithi060 ceph-mon[40290]: 10.1 scrub starts 2024-09-20T22:18:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:38 smithi060 ceph-mon[40290]: 10.1 scrub ok 2024-09-20T22:18:39.044 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:39.061 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_3 2024-09-20T22:18:39.061 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool rm unique_pool_3 unique_pool_3 --yes-i-really-really-mean-it 2024-09-20T22:18:39.066 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:39.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:39.067 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:39.168 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:39.195 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:39.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:39.195 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:39.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:39.322 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:39.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:39.323 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:39.424 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:39.446 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:39.447 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:39.447 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:39.548 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:39.575 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:39.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:39.576 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:39.672 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:39 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:18:39.672 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:39 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:18:39.672 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:39 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:18:39.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:39.700 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:39.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:39.701 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:39.754 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:39 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:18:39.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:39 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:18:39.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:39 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:18:39.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:39 smithi060 ceph-mon[29694]: osdmap e453: 8 total, 8 up, 8 in 2024-09-20T22:18:39.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:39 smithi060 ceph-mon[29694]: 3.e scrub starts 2024-09-20T22:18:39.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:39 smithi060 ceph-mon[29694]: 3.e scrub ok 2024-09-20T22:18:39.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:39 smithi060 ceph-mon[29694]: 5.0 scrub starts 2024-09-20T22:18:39.755 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:39 smithi060 ceph-mon[29694]: 5.0 scrub ok 2024-09-20T22:18:39.755 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:39 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:18:39.755 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:39 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:18:39.755 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:39 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:18:39.755 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:39 smithi060 ceph-mon[40290]: osdmap e453: 8 total, 8 up, 8 in 2024-09-20T22:18:39.756 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:39 smithi060 ceph-mon[40290]: 3.e scrub starts 2024-09-20T22:18:39.756 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:39 smithi060 ceph-mon[40290]: 3.e scrub ok 2024-09-20T22:18:39.756 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:39 smithi060 ceph-mon[40290]: 5.0 scrub starts 2024-09-20T22:18:39.756 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:39 smithi060 ceph-mon[40290]: 5.0 scrub ok 2024-09-20T22:18:39.801 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:39.823 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:39.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:39.824 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:39.925 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:39.947 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:39.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:39.948 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:39 smithi067 ceph-mon[33511]: osdmap e453: 8 total, 8 up, 8 in 2024-09-20T22:18:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:39 smithi067 ceph-mon[33511]: 3.e scrub starts 2024-09-20T22:18:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:39 smithi067 ceph-mon[33511]: 3.e scrub ok 2024-09-20T22:18:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:39 smithi067 ceph-mon[33511]: 5.0 scrub starts 2024-09-20T22:18:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:39 smithi067 ceph-mon[33511]: 5.0 scrub ok 2024-09-20T22:18:40.028 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:18:40.029 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:18:40.029 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 0 2024-09-20T22:18:40.029 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:18:40.030 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:40.030 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:40.049 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:40.073 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:40.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:40.074 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:40.175 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:40.201 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:40.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:40.202 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:40.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:40.306 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:40.327 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:40.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:40.328 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:40.429 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:40.451 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:40.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:40.452 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:40.553 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:40.575 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:40.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:40.576 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:40.677 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:40.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:40.701 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:40.701 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:40.801 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:40.824 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:40.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:40.825 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:40.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[29694]: 10.d scrub starts 2024-09-20T22:18:40.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[29694]: 10.d scrub ok 2024-09-20T22:18:40.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[29694]: 4.15 scrub starts 2024-09-20T22:18:40.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[29694]: 4.15 scrub ok 2024-09-20T22:18:40.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[29694]: 4.10 deep-scrub starts 2024-09-20T22:18:40.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[29694]: 4.10 deep-scrub ok 2024-09-20T22:18:40.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[29694]: 5.a scrub starts 2024-09-20T22:18:40.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[29694]: 5.a scrub ok 2024-09-20T22:18:40.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[29694]: pgmap v843: 178 pgs: 178 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 317 op/s 2024-09-20T22:18:40.898 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[29694]: 6.4 deep-scrub starts 2024-09-20T22:18:40.898 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[29694]: 6.4 deep-scrub ok 2024-09-20T22:18:40.898 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[40290]: 10.d scrub starts 2024-09-20T22:18:40.898 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[40290]: 10.d scrub ok 2024-09-20T22:18:40.898 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[40290]: 4.15 scrub starts 2024-09-20T22:18:40.898 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[40290]: 4.15 scrub ok 2024-09-20T22:18:40.898 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[40290]: 4.10 deep-scrub starts 2024-09-20T22:18:40.898 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[40290]: 4.10 deep-scrub ok 2024-09-20T22:18:40.898 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[40290]: 5.a scrub starts 2024-09-20T22:18:40.898 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[40290]: 5.a scrub ok 2024-09-20T22:18:40.898 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[40290]: pgmap v843: 178 pgs: 178 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 317 op/s 2024-09-20T22:18:40.899 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[40290]: 6.4 deep-scrub starts 2024-09-20T22:18:40.899 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:40 smithi060 ceph-mon[40290]: 6.4 deep-scrub ok 2024-09-20T22:18:40.925 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:40.947 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:40.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:40.948 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:40 smithi067 ceph-mon[33511]: 10.d scrub starts 2024-09-20T22:18:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:40 smithi067 ceph-mon[33511]: 10.d scrub ok 2024-09-20T22:18:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:40 smithi067 ceph-mon[33511]: 4.15 scrub starts 2024-09-20T22:18:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:40 smithi067 ceph-mon[33511]: 4.15 scrub ok 2024-09-20T22:18:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:40 smithi067 ceph-mon[33511]: 4.10 deep-scrub starts 2024-09-20T22:18:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:40 smithi067 ceph-mon[33511]: 4.10 deep-scrub ok 2024-09-20T22:18:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:40 smithi067 ceph-mon[33511]: 5.a scrub starts 2024-09-20T22:18:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:40 smithi067 ceph-mon[33511]: 5.a scrub ok 2024-09-20T22:18:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:40 smithi067 ceph-mon[33511]: pgmap v843: 178 pgs: 178 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 317 op/s 2024-09-20T22:18:41.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:40 smithi067 ceph-mon[33511]: 6.4 deep-scrub starts 2024-09-20T22:18:41.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:40 smithi067 ceph-mon[33511]: 6.4 deep-scrub ok 2024-09-20T22:18:41.049 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:41.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:41.077 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:41.178 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:41.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:41.201 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:41.202 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:41.302 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:41.325 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:41.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:41.326 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:41.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:41.433 INFO:teuthology.orchestra.run.smithi060.stderr:no valid command found; 10 closest matches: 2024-09-20T22:18:41.433 INFO:teuthology.orchestra.run.smithi060.stderr:0 2024-09-20T22:18:41.433 INFO:teuthology.orchestra.run.smithi060.stderr:1 2024-09-20T22:18:41.434 INFO:teuthology.orchestra.run.smithi060.stderr:2 2024-09-20T22:18:41.434 INFO:teuthology.orchestra.run.smithi060.stderr:abort 2024-09-20T22:18:41.434 INFO:teuthology.orchestra.run.smithi060.stderr:assert 2024-09-20T22:18:41.434 INFO:teuthology.orchestra.run.smithi060.stderr:bench [] [] [] [] 2024-09-20T22:18:41.434 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs debug_inject_read_zeros 2024-09-20T22:18:41.434 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs files list 2024-09-20T22:18:41.434 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs stats 2024-09-20T22:18:41.434 INFO:teuthology.orchestra.run.smithi060.stderr:bluestore allocator dump block 2024-09-20T22:18:41.434 INFO:teuthology.orchestra.run.smithi060.stderr:Error EINVAL: invalid command 2024-09-20T22:18:41.449 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:41.450 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:41.450 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:41.511 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:18:41.551 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:41.579 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:41.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:41.580 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:41.616 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'unique_pool_3' removed 2024-09-20T22:18:41.681 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:41.725 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:41.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:41.726 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:41.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:18:41.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:18:41.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[40290]: 5.e scrub starts 2024-09-20T22:18:41.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[40290]: 5.e scrub ok 2024-09-20T22:18:41.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[40290]: 5.12 scrub starts 2024-09-20T22:18:41.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[40290]: 5.12 scrub ok 2024-09-20T22:18:41.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[40290]: osdmap e454: 8 total, 8 up, 8 in 2024-09-20T22:18:41.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3209152076' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:18:41.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3209152076' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T22:18:41.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[40290]: osdmap e455: 8 total, 8 up, 8 in 2024-09-20T22:18:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:18:41.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:18:41.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[29694]: 5.e scrub starts 2024-09-20T22:18:41.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[29694]: 5.e scrub ok 2024-09-20T22:18:41.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[29694]: 5.12 scrub starts 2024-09-20T22:18:41.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[29694]: 5.12 scrub ok 2024-09-20T22:18:41.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[29694]: osdmap e454: 8 total, 8 up, 8 in 2024-09-20T22:18:41.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3209152076' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:18:41.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3209152076' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T22:18:41.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:41 smithi060 ceph-mon[29694]: osdmap e455: 8 total, 8 up, 8 in 2024-09-20T22:18:41.827 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:41.858 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:41.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:41.859 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:41.960 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:41.982 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:41.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:41.983 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:41 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:18:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:41 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:18:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:41 smithi067 ceph-mon[33511]: 5.e scrub starts 2024-09-20T22:18:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:41 smithi067 ceph-mon[33511]: 5.e scrub ok 2024-09-20T22:18:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:41 smithi067 ceph-mon[33511]: 5.12 scrub starts 2024-09-20T22:18:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:41 smithi067 ceph-mon[33511]: 5.12 scrub ok 2024-09-20T22:18:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:41 smithi067 ceph-mon[33511]: osdmap e454: 8 total, 8 up, 8 in 2024-09-20T22:18:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:41 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3209152076' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:18:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:41 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3209152076' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_3", "pool2": "unique_pool_3", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T22:18:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:41 smithi067 ceph-mon[33511]: osdmap e455: 8 total, 8 up, 8 in 2024-09-20T22:18:42.084 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:42.106 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:42.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:42.107 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:42.208 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:42.230 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:42.231 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:42.231 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:42.332 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:42.351 INFO:teuthology.task.full_sequential:In full_sequential, running task radosbench... 2024-09-20T22:18:42.351 INFO:tasks.radosbench:Beginning radosbench... 2024-09-20T22:18:42.352 INFO:tasks.cephadm.ceph_manager.ceph:creating pool_name unique_pool_4 2024-09-20T22:18:42.352 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool create unique_pool_4 16 2024-09-20T22:18:42.354 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:42.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:42.355 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:42.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:42.479 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:42.479 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:42.480 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:42.580 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:42.602 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:42.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:42.603 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:42.704 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:42.726 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:42.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:42.727 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:42.828 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:42.850 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:42.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:42.851 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:42.952 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:42.974 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:42.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:42.975 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:43.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:18:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:18:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:42 smithi067 ceph-mon[33511]: 1.1 scrub starts 2024-09-20T22:18:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:42 smithi067 ceph-mon[33511]: 1.1 scrub ok 2024-09-20T22:18:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:42 smithi067 ceph-mon[33511]: pgmap v846: 162 pgs: 162 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 880 B/s rd, 0 op/s 2024-09-20T22:18:43.018 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:42 smithi060 ceph-mon[40290]: 1.1 scrub starts 2024-09-20T22:18:43.018 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:42 smithi060 ceph-mon[40290]: 1.1 scrub ok 2024-09-20T22:18:43.018 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:42 smithi060 ceph-mon[40290]: pgmap v846: 162 pgs: 162 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 880 B/s rd, 0 op/s 2024-09-20T22:18:43.019 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:42 smithi060 ceph-mon[29694]: 1.1 scrub starts 2024-09-20T22:18:43.019 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:42 smithi060 ceph-mon[29694]: 1.1 scrub ok 2024-09-20T22:18:43.019 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:42 smithi060 ceph-mon[29694]: pgmap v846: 162 pgs: 162 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 880 B/s rd, 0 op/s 2024-09-20T22:18:43.046 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T22:18:43.046 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-20T22:18:43.047 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.6 'trim stale osdmaps' 2024-09-20T22:18:43.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:43.113 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:43.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:43.115 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:43.215 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:43.239 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:43.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:43.240 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:43.319 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:18:43.320 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:18:43.339 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:18:43.339 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:18:43.341 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:43.357 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:18:43.357 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:18:43.364 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:43.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:43.365 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:43.373 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:18:43.373 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:18:43.390 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:18:43.391 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:18:43.409 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:18:43.409 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:18:43.431 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:18:43.432 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:18:43.451 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:18:43.451 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:18:43.465 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:43.488 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:43.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:43.489 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:43.590 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:43.612 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:43.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:43.613 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:43.714 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:43.736 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:43.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:43.737 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:43.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:43.860 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:43.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:43.861 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:43.950 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:43 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:18:43.950 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:43 smithi060 ceph-mon[29694]: 3.4 scrub starts 2024-09-20T22:18:43.950 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:43 smithi060 ceph-mon[29694]: 3.4 scrub ok 2024-09-20T22:18:43.950 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:43 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2707781185' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:43.950 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:43 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:18:43.951 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:43 smithi060 ceph-mon[40290]: 3.4 scrub starts 2024-09-20T22:18:43.951 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:43 smithi060 ceph-mon[40290]: 3.4 scrub ok 2024-09-20T22:18:43.951 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:43 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2707781185' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:43.952 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T22:18:43.962 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:43.986 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:43.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:43.987 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:43 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:18:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:43 smithi067 ceph-mon[33511]: 3.4 scrub starts 2024-09-20T22:18:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:43 smithi067 ceph-mon[33511]: 3.4 scrub ok 2024-09-20T22:18:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:43 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2707781185' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:44.089 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:44.110 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:44.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:44.111 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:44.200 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:44.200 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:44.200 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:44.213 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:44.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:44.239 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:44.239 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:44.340 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:44.366 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:44.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:44.367 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:44.468 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:44.493 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:44.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:44.493 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:44.594 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:44.618 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:44.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:44.619 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:44.720 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:44.742 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:44.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:44.743 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:44.844 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:44.869 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:44.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:44.870 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:44.877 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:44 smithi060 ceph-mon[29694]: 3.1f deep-scrub starts 2024-09-20T22:18:44.877 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:44 smithi060 ceph-mon[29694]: 3.1f deep-scrub ok 2024-09-20T22:18:44.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:44 smithi060 ceph-mon[29694]: 4.0 scrub starts 2024-09-20T22:18:44.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:44 smithi060 ceph-mon[29694]: 4.0 scrub ok 2024-09-20T22:18:44.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:44 smithi060 ceph-mon[29694]: 5.1d deep-scrub starts 2024-09-20T22:18:44.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:44 smithi060 ceph-mon[29694]: 5.1d deep-scrub ok 2024-09-20T22:18:44.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:44 smithi060 ceph-mon[29694]: pgmap v847: 162 pgs: 162 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T22:18:44.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:44 smithi060 ceph-mon[29694]: 6.1a scrub starts 2024-09-20T22:18:44.878 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:44 smithi060 ceph-mon[29694]: 6.1a scrub ok 2024-09-20T22:18:44.878 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:44 smithi060 ceph-mon[40290]: 3.1f deep-scrub starts 2024-09-20T22:18:44.878 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:44 smithi060 ceph-mon[40290]: 3.1f deep-scrub ok 2024-09-20T22:18:44.878 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:44 smithi060 ceph-mon[40290]: 4.0 scrub starts 2024-09-20T22:18:44.879 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:44 smithi060 ceph-mon[40290]: 4.0 scrub ok 2024-09-20T22:18:44.879 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:44 smithi060 ceph-mon[40290]: 5.1d deep-scrub starts 2024-09-20T22:18:44.879 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:44 smithi060 ceph-mon[40290]: 5.1d deep-scrub ok 2024-09-20T22:18:44.879 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:44 smithi060 ceph-mon[40290]: pgmap v847: 162 pgs: 162 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T22:18:44.879 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:44 smithi060 ceph-mon[40290]: 6.1a scrub starts 2024-09-20T22:18:44.879 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:44 smithi060 ceph-mon[40290]: 6.1a scrub ok 2024-09-20T22:18:44.971 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:44.992 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:44.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:44.993 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:44 smithi067 ceph-mon[33511]: 3.1f deep-scrub starts 2024-09-20T22:18:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:44 smithi067 ceph-mon[33511]: 3.1f deep-scrub ok 2024-09-20T22:18:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:44 smithi067 ceph-mon[33511]: 4.0 scrub starts 2024-09-20T22:18:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:44 smithi067 ceph-mon[33511]: 4.0 scrub ok 2024-09-20T22:18:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:44 smithi067 ceph-mon[33511]: 5.1d deep-scrub starts 2024-09-20T22:18:45.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:44 smithi067 ceph-mon[33511]: 5.1d deep-scrub ok 2024-09-20T22:18:45.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:44 smithi067 ceph-mon[33511]: pgmap v847: 162 pgs: 162 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T22:18:45.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:44 smithi067 ceph-mon[33511]: 6.1a scrub starts 2024-09-20T22:18:45.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:44 smithi067 ceph-mon[33511]: 6.1a scrub ok 2024-09-20T22:18:45.094 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:45.121 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:45.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:45.122 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:45.222 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:45.226 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:18:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:18:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:18:45.248 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:45.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:45.249 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:45.350 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:45.375 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:45.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:45.375 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:45.476 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:45.501 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:45.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:45.502 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:45.603 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:45.627 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:45.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:45.628 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:45.728 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:45.751 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:45.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:45.752 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:45.853 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:45.862 INFO:teuthology.orchestra.run.smithi060.stderr:no valid command found; 10 closest matches: 2024-09-20T22:18:45.862 INFO:teuthology.orchestra.run.smithi060.stderr:0 2024-09-20T22:18:45.863 INFO:teuthology.orchestra.run.smithi060.stderr:1 2024-09-20T22:18:45.863 INFO:teuthology.orchestra.run.smithi060.stderr:2 2024-09-20T22:18:45.863 INFO:teuthology.orchestra.run.smithi060.stderr:abort 2024-09-20T22:18:45.863 INFO:teuthology.orchestra.run.smithi060.stderr:assert 2024-09-20T22:18:45.863 INFO:teuthology.orchestra.run.smithi060.stderr:bench [] [] [] [] 2024-09-20T22:18:45.863 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs debug_inject_read_zeros 2024-09-20T22:18:45.863 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs files list 2024-09-20T22:18:45.863 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs stats 2024-09-20T22:18:45.863 INFO:teuthology.orchestra.run.smithi060.stderr:bluestore allocator dump block 2024-09-20T22:18:45.863 INFO:teuthology.orchestra.run.smithi060.stderr:Error EINVAL: invalid command 2024-09-20T22:18:45.878 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:45.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:45.879 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:45.980 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:45.999 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'unique_pool_4' created 2024-09-20T22:18:46.004 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:46.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:46.005 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:46.106 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:46.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:46.129 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:46.129 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:46.140 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:18:46.140 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:45 smithi060 ceph-mon[29694]: 5.11 scrub starts 2024-09-20T22:18:46.140 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:45 smithi060 ceph-mon[29694]: 5.11 scrub ok 2024-09-20T22:18:46.140 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:45 smithi060 ceph-mon[29694]: 3.1a deep-scrub starts 2024-09-20T22:18:46.140 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:45 smithi060 ceph-mon[29694]: 3.1a deep-scrub ok 2024-09-20T22:18:46.140 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:45 smithi060 ceph-mon[40290]: 5.11 scrub starts 2024-09-20T22:18:46.141 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:45 smithi060 ceph-mon[40290]: 5.11 scrub ok 2024-09-20T22:18:46.141 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:45 smithi060 ceph-mon[40290]: 3.1a deep-scrub starts 2024-09-20T22:18:46.141 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:45 smithi060 ceph-mon[40290]: 3.1a deep-scrub ok 2024-09-20T22:18:46.230 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:46.257 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:46.258 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:46.258 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:46.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:45 smithi067 ceph-mon[33511]: 5.11 scrub starts 2024-09-20T22:18:46.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:45 smithi067 ceph-mon[33511]: 5.11 scrub ok 2024-09-20T22:18:46.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:45 smithi067 ceph-mon[33511]: 3.1a deep-scrub starts 2024-09-20T22:18:46.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:45 smithi067 ceph-mon[33511]: 3.1a deep-scrub ok 2024-09-20T22:18:46.360 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:46.386 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:46.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:46.387 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:46.488 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:46.511 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:46.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:46.512 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:46.607 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:18:46.612 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:46.672 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:46.673 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:46.673 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:46.774 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:46.796 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:46.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:46.797 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:46.898 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:46.920 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:46.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:46.921 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:47.022 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:47.044 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:47.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:47.045 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:47.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[29694]: 6.8 scrub starts 2024-09-20T22:18:47.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[29694]: 6.8 scrub ok 2024-09-20T22:18:47.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1094196030' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-20T22:18:47.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[29694]: 3.7 scrub starts 2024-09-20T22:18:47.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[29694]: 3.7 scrub ok 2024-09-20T22:18:47.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/138538705' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:18:47.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:18:47.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1094196030' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-20T22:18:47.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/138538705' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:18:47.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[29694]: osdmap e456: 8 total, 8 up, 8 in 2024-09-20T22:18:47.099 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[29694]: pgmap v849: 178 pgs: 16 unknown, 162 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-20T22:18:47.100 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[29694]: osdmap e457: 8 total, 8 up, 8 in 2024-09-20T22:18:47.100 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[40290]: 6.8 scrub starts 2024-09-20T22:18:47.100 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[40290]: 6.8 scrub ok 2024-09-20T22:18:47.100 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1094196030' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-20T22:18:47.100 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[40290]: 3.7 scrub starts 2024-09-20T22:18:47.100 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[40290]: 3.7 scrub ok 2024-09-20T22:18:47.100 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/138538705' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:18:47.100 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:18:47.100 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1094196030' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-20T22:18:47.100 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/138538705' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:18:47.100 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[40290]: osdmap e456: 8 total, 8 up, 8 in 2024-09-20T22:18:47.100 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[40290]: pgmap v849: 178 pgs: 16 unknown, 162 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-20T22:18:47.101 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:46 smithi060 ceph-mon[40290]: osdmap e457: 8 total, 8 up, 8 in 2024-09-20T22:18:47.146 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:47.173 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:47.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:47.174 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:47.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:46 smithi067 ceph-mon[33511]: 6.8 scrub starts 2024-09-20T22:18:47.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:46 smithi067 ceph-mon[33511]: 6.8 scrub ok 2024-09-20T22:18:47.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:46 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1094196030' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]: dispatch 2024-09-20T22:18:47.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:46 smithi067 ceph-mon[33511]: 3.7 scrub starts 2024-09-20T22:18:47.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:46 smithi067 ceph-mon[33511]: 3.7 scrub ok 2024-09-20T22:18:47.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:46 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/138538705' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:18:47.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:46 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:18:47.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:46 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1094196030' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "unique_pool_4", "pg_num": 16}]': finished 2024-09-20T22:18:47.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:46 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/138538705' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:18:47.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:46 smithi067 ceph-mon[33511]: osdmap e456: 8 total, 8 up, 8 in 2024-09-20T22:18:47.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:46 smithi067 ceph-mon[33511]: pgmap v849: 178 pgs: 16 unknown, 162 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-20T22:18:47.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:46 smithi067 ceph-mon[33511]: osdmap e457: 8 total, 8 up, 8 in 2024-09-20T22:18:47.274 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:47.302 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:47.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:47.303 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:47.403 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:47.429 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:47.430 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:47.430 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:47.531 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:47.557 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:47.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:47.558 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:47.559 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool application enable unique_pool_4 rados --yes-i-really-mean-it || true 2024-09-20T22:18:47.572 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:47.594 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:47.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:47.685 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:47.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:47.686 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:47.787 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:47.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:47.810 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:47.810 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:47.911 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:47.939 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:47.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:47.940 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:48.041 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:48.064 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:48.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:48.065 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:48.165 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:48.169 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:47 smithi060 ceph-mon[29694]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T22:18:48.169 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:47 smithi060 ceph-mon[29694]: osdmap e458: 8 total, 8 up, 8 in 2024-09-20T22:18:48.169 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:47 smithi060 ceph-mon[40290]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T22:18:48.169 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:47 smithi060 ceph-mon[40290]: osdmap e458: 8 total, 8 up, 8 in 2024-09-20T22:18:48.192 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:48.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:48.193 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:48.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:47 smithi067 ceph-mon[33511]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-20T22:18:48.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:47 smithi067 ceph-mon[33511]: osdmap e458: 8 total, 8 up, 8 in 2024-09-20T22:18:48.274 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:18:48.274 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:18:48.294 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:48.321 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:48.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:48.322 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:48.423 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:48.445 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:48.446 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:48.446 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:48.547 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:48.574 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:48.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:48.575 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:48.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:48.687 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T22:18:48.687 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-20T22:18:48.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.7 'trim stale osdmaps' 2024-09-20T22:18:48.698 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:48.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:48.699 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:48.800 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:48.827 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:48.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:48.827 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:48.868 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:18:48.928 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:48.952 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:48.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:48.953 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:49.054 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:49.076 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:49.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:49.077 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:49.178 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:49.205 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:49.206 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:49.206 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:49.227 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:48 smithi060 ceph-mon[40290]: pgmap v852: 178 pgs: 3 creating+peering, 6 unknown, 169 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:18:49.227 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:48 smithi060 ceph-mon[29694]: pgmap v852: 178 pgs: 3 creating+peering, 6 unknown, 169 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:18:49.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:48 smithi067 ceph-mon[33511]: pgmap v852: 178 pgs: 3 creating+peering, 6 unknown, 169 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:18:49.307 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:49.331 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:18:49.331 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:18:49.334 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:49.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:49.335 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:49.351 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:18:49.351 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:18:49.367 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:18:49.368 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:18:49.383 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:18:49.384 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:18:49.402 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:18:49.402 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:18:49.419 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:18:49.420 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:18:49.436 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:49.437 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:18:49.437 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:18:49.455 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:18:49.455 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:18:49.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:49.463 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:49.463 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:49.564 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:49.590 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:49.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:49.591 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:49.692 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:49.718 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:49.718 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:49.719 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:49.775 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T22:18:49.820 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:49.846 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:49.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:49.848 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:49.948 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:49.975 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:49.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:49.976 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:50.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:49 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2360283193' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:50.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:49 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2360283193' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:50.037 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:50.077 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:50.103 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:50.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:50.104 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:50.206 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:50.233 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:50.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:50.233 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:50.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:49 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2360283193' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:50.334 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:50.361 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:50.361 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:50.361 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:50.462 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:50.489 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:50.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:50.490 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:50.590 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:50.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:50.615 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:50.615 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:50.716 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:50.743 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:50.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:50.744 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:50.845 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:50.867 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:50.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:50.868 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:50.896 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:50.896 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:50.937 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:50.966 INFO:teuthology.orchestra.run.smithi060.stderr:enabled application 'rados' on pool 'unique_pool_4' 2024-09-20T22:18:50.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:50.992 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:50.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:50.994 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:51.095 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:51.122 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:51.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:51.123 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:51.149 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:50 smithi060 ceph-mon[29694]: pgmap v853: 178 pgs: 3 creating+peering, 4 unknown, 171 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:18:51.149 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:50 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/743357497' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:18:51.149 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:50 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:18:51.149 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:50 smithi060 ceph-mon[40290]: pgmap v853: 178 pgs: 3 creating+peering, 4 unknown, 171 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:18:51.150 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:50 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/743357497' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:18:51.150 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:50 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:18:51.223 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:51.250 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:51.251 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:51.251 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:51.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:50 smithi067 ceph-mon[33511]: pgmap v853: 178 pgs: 3 creating+peering, 4 unknown, 171 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:18:51.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:50 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/743357497' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:18:51.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:50 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]: dispatch 2024-09-20T22:18:51.351 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:51.378 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:51.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:51.379 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:51.479 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:51.505 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:51.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:51.507 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:51.607 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:51.632 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:51.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:51.633 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:51.733 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:51.755 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:51.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:51.756 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:51.857 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:51.884 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:51.885 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:51.885 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:51.986 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:52.014 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:52.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:52.015 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:52.116 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:52.138 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:52.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:52.139 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:52.223 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:51 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T22:18:52.223 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:51 smithi060 ceph-mon[29694]: osdmap e459: 8 total, 8 up, 8 in 2024-09-20T22:18:52.223 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:18:52.223 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:18:52.223 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:51 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/685674326' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:18:52.224 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:51 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T22:18:52.224 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:51 smithi060 ceph-mon[40290]: osdmap e459: 8 total, 8 up, 8 in 2024-09-20T22:18:52.224 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:18:52.224 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:18:52.224 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:51 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/685674326' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:18:52.240 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:52.263 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:52.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:51 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "unique_pool_4", "app": "rados", "yes_i_really_mean_it": true}]': finished 2024-09-20T22:18:52.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:51 smithi067 ceph-mon[33511]: osdmap e459: 8 total, 8 up, 8 in 2024-09-20T22:18:52.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:18:52.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:18:52.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:51 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/685674326' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:18:52.264 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:52.264 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:52.366 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:52.392 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:52.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:52.393 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:52.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:52.518 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:52.518 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:52.519 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:52.619 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:52.620 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:18:52.629 INFO:teuthology.orchestra.run.smithi060.stderr:no valid command found; 10 closest matches: 2024-09-20T22:18:52.629 INFO:teuthology.orchestra.run.smithi060.stderr:0 2024-09-20T22:18:52.629 INFO:teuthology.orchestra.run.smithi060.stderr:1 2024-09-20T22:18:52.629 INFO:teuthology.orchestra.run.smithi060.stderr:2 2024-09-20T22:18:52.629 INFO:teuthology.orchestra.run.smithi060.stderr:abort 2024-09-20T22:18:52.629 INFO:teuthology.orchestra.run.smithi060.stderr:assert 2024-09-20T22:18:52.630 INFO:teuthology.orchestra.run.smithi060.stderr:bench [] [] [] [] 2024-09-20T22:18:52.630 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs debug_inject_read_zeros 2024-09-20T22:18:52.630 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs files list 2024-09-20T22:18:52.630 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs stats 2024-09-20T22:18:52.630 INFO:teuthology.orchestra.run.smithi060.stderr:bluestore allocator dump block 2024-09-20T22:18:52.630 INFO:teuthology.orchestra.run.smithi060.stderr:Error EINVAL: invalid command 2024-09-20T22:18:52.645 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:52.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:52.646 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:52.746 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:52.774 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:52.774 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:52.775 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:52.875 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:52.904 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:52.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:52.904 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:52.932 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:18:52.970 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:18:52 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:18:53.005 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:53.018 DEBUG:teuthology.orchestra.run.smithi060:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -b 65536 --object-size 65536 -p unique_pool_4 bench 90 write' 2024-09-20T22:18:53.027 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:53.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:53.028 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:53.029 INFO:tasks.radosbench:joining radosbench (timing out after 3000s) 2024-09-20T22:18:53.029 INFO:teuthology.orchestra.run:waiting for 3000 2024-09-20T22:18:53.093 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:hints = 1 2024-09-20T22:18:53.093 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Maintaining 16 concurrent writes of 65536 bytes to objects of size 65536 for up to 90 seconds or 0 objects 2024-09-20T22:18:53.093 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Object prefix: benchmark_data_smithi060_196073 2024-09-20T22:18:53.094 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:18:53.094 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 0 0 0 0 0 0 - 0 2024-09-20T22:18:53.129 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:53.171 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:53.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:53.172 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:53.179 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:52 smithi060 ceph-mon[29694]: pgmap v855: 178 pgs: 178 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:18:53.179 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:52 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:18:53.179 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:52 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/685674326' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:18:53.179 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:52 smithi060 ceph-mon[29694]: osdmap e460: 8 total, 8 up, 8 in 2024-09-20T22:18:53.179 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:52 smithi060 ceph-mon[40290]: pgmap v855: 178 pgs: 178 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:18:53.179 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:52 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:18:53.179 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:52 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/685674326' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:18:53.180 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:52 smithi060 ceph-mon[40290]: osdmap e460: 8 total, 8 up, 8 in 2024-09-20T22:18:53.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:52 smithi067 ceph-mon[33511]: pgmap v855: 178 pgs: 178 active+clean; 583 KiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:18:53.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:52 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:18:53.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:52 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/685674326' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:18:53.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:52 smithi067 ceph-mon[33511]: osdmap e460: 8 total, 8 up, 8 in 2024-09-20T22:18:53.274 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:53.315 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:53.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:53.317 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:53.418 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:53.455 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:53.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:53.456 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:53.557 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:53.586 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:53.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:53.587 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:53.688 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:53.726 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:53.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:53.727 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:53.827 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:53.867 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:53.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:53.868 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:53.937 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:18:53.937 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:18:53.937 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:18:53.937 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:18:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:106980 8.18 8:1c2b9c14:::benchmark_data_smithi060_111051_object106979:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e218)", 2024-09-20T22:18:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:09:11.414151+0000", 2024-09-20T22:18:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 582.51804102400001, 2024-09-20T22:18:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.111436294, 2024-09-20T22:18:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 106980 2024-09-20T22:18:53.938 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414151+0000", 2024-09-20T22:18:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414151+0000", 2024-09-20T22:18:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:53.939 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414152+0000", 2024-09-20T22:18:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5179999999999999e-06 2024-09-20T22:18:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414179+0000", 2024-09-20T22:18:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6480999999999999e-05 2024-09-20T22:18:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414191+0000", 2024-09-20T22:18:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2833999999999999e-05 2024-09-20T22:18:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.940 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.414200+0000", 2024-09-20T22:18:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1040000000000007e-06 2024-09-20T22:18:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418058+0000", 2024-09-20T22:18:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0038580139999999999 2024-09-20T22:18:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:11.418080+0000", 2024-09-20T22:18:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2438000000000001e-05 2024-09-20T22:18:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.941 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.501675+0000", 2024-09-20T22:18:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0835947560000001 2024-09-20T22:18:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.501814+0000", 2024-09-20T22:18:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013951799999999999 2024-09-20T22:18:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 4,5,7", 2024-09-20T22:18:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.502081+0000", 2024-09-20T22:18:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00026697100000000002 2024-09-20T22:18:53.942 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.503435+0000", 2024-09-20T22:18:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001354156 2024-09-20T22:18:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.514128+0000", 2024-09-20T22:18:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010692432 2024-09-20T22:18:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525290+0000", 2024-09-20T22:18:53.943 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.011161901 2024-09-20T22:18:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525399+0000", 2024-09-20T22:18:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010921699999999999 2024-09-20T22:18:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525468+0000", 2024-09-20T22:18:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8726999999999994e-05 2024-09-20T22:18:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:09:12.525587+0000", 2024-09-20T22:18:53.944 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000119227 2024-09-20T22:18:53.945 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:53.945 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:53.945 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:53.945 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.945 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.945 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187847 8.0 8:0387ec14:::benchmark_data_smithi060_111051_object67848:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:18:53.945 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.626135+0000", 2024-09-20T22:18:53.945 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 504.30605682999999, 2024-09-20T22:18:53.945 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2145214800000002, 2024-09-20T22:18:53.945 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:53.945 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:53.946 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:18:53.946 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:53.946 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:53.946 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:53.946 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187847 2024-09-20T22:18:53.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.946 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:53.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:53.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626135+0000", 2024-09-20T22:18:53.946 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:53.946 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.946 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.946 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:53.946 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626135+0000", 2024-09-20T22:18:53.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:53.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:53.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626137+0000", 2024-09-20T22:18:53.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0389999999999999e-06 2024-09-20T22:18:53.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:53.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626144+0000", 2024-09-20T22:18:53.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9299999999999997e-06 2024-09-20T22:18:53.947 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.947 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.947 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:53.947 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626285+0000", 2024-09-20T22:18:53.947 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000141445 2024-09-20T22:18:53.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:53.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626296+0000", 2024-09-20T22:18:53.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0873000000000001e-05 2024-09-20T22:18:53.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626432+0000", 2024-09-20T22:18:53.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000135537 2024-09-20T22:18:53.948 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.948 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.948 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:53.948 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.626543+0000", 2024-09-20T22:18:53.948 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011132399999999999 2024-09-20T22:18:53.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,5", 2024-09-20T22:18:53.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.627183+0000", 2024-09-20T22:18:53.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00064020400000000001 2024-09-20T22:18:53.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.656943+0000", 2024-09-20T22:18:53.949 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.029759622999999999 2024-09-20T22:18:53.949 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.949 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.949 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:53.949 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.656954+0000", 2024-09-20T22:18:53.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0913999999999999e-05 2024-09-20T22:18:53.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651133+0000", 2024-09-20T22:18:53.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99417974399999998 2024-09-20T22:18:53.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:53.950 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651152+0000", 2024-09-20T22:18:53.950 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8108999999999999e-05 2024-09-20T22:18:53.950 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.950 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.950 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.840586+0000", 2024-09-20T22:18:53.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1894341669999999 2024-09-20T22:18:53.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.951 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:53.951 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.840656+0000", 2024-09-20T22:18:53.951 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0570999999999999e-05 2024-09-20T22:18:53.951 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:53.951 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:53.951 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:53.951 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.951 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.951 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187874 8.0 8:032cd1ec:::benchmark_data_smithi060_111051_object67875:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:18:53.951 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.678008+0000", 2024-09-20T22:18:53.951 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 504.25418346599997, 2024-09-20T22:18:53.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1758792680000001, 2024-09-20T22:18:53.952 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:53.952 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:53.952 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:53.952 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:53.952 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:53.952 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:53.952 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187874 2024-09-20T22:18:53.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.952 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:53.952 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.952 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:53.952 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678008+0000", 2024-09-20T22:18:53.952 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:53.952 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:53.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678008+0000", 2024-09-20T22:18:53.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:53.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:53.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678010+0000", 2024-09-20T22:18:53.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7430000000000001e-06 2024-09-20T22:18:53.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.953 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:53.953 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678016+0000", 2024-09-20T22:18:53.953 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4339999999999997e-06 2024-09-20T22:18:53.953 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.953 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:53.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678030+0000", 2024-09-20T22:18:53.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3971999999999999e-05 2024-09-20T22:18:53.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:53.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678038+0000", 2024-09-20T22:18:53.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5100000000000001e-06 2024-09-20T22:18:53.954 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.954 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.954 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.954 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678109+0000", 2024-09-20T22:18:53.954 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0708000000000001e-05 2024-09-20T22:18:53.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:53.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.678119+0000", 2024-09-20T22:18:53.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0519e-05 2024-09-20T22:18:53.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651185+0000", 2024-09-20T22:18:53.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97306574199999996 2024-09-20T22:18:53.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.955 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.955 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:53.955 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651193+0000", 2024-09-20T22:18:53.955 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2609999999999993e-06 2024-09-20T22:18:53.955 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.840972+0000", 2024-09-20T22:18:53.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.189778644 2024-09-20T22:18:53.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:53.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.841244+0000", 2024-09-20T22:18:53.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00027224500000000001 2024-09-20T22:18:53.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.956 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4,5", 2024-09-20T22:18:53.956 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.841873+0000", 2024-09-20T22:18:53.956 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000629124 2024-09-20T22:18:53.956 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.956 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:53.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.852639+0000", 2024-09-20T22:18:53.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010765925000000001 2024-09-20T22:18:53.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:53.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.852704+0000", 2024-09-20T22:18:53.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5011000000000005e-05 2024-09-20T22:18:53.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:53.957 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.852729+0000", 2024-09-20T22:18:53.957 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4782000000000001e-05 2024-09-20T22:18:53.957 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.957 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.957 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:53.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.853614+0000", 2024-09-20T22:18:53.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00088560699999999995 2024-09-20T22:18:53.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:53.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.853671+0000", 2024-09-20T22:18:53.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6382999999999999e-05 2024-09-20T22:18:53.958 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.958 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.958 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:53.958 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.853887+0000", 2024-09-20T22:18:53.958 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000216658 2024-09-20T22:18:53.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:53.958 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:53.958 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:53.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.959 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.959 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:187894 8.0 8:02b3e725:::benchmark_data_smithi060_111051_object67895:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:18:53.959 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:29.701140+0000", 2024-09-20T22:18:53.959 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 504.231051718, 2024-09-20T22:18:53.959 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1634560220000001, 2024-09-20T22:18:53.959 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:53.959 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:53.959 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:53.959 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:53.959 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:53.959 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:53.959 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 187894 2024-09-20T22:18:53.959 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.960 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:53.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:53.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701140+0000", 2024-09-20T22:18:53.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:53.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:53.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701140+0000", 2024-09-20T22:18:53.960 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:53.960 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.960 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.960 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:53.960 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701141+0000", 2024-09-20T22:18:53.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2950000000000001e-06 2024-09-20T22:18:53.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:53.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701147+0000", 2024-09-20T22:18:53.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3890000000000004e-06 2024-09-20T22:18:53.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:53.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701168+0000", 2024-09-20T22:18:53.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1725999999999998e-05 2024-09-20T22:18:53.961 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.961 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.961 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:53.961 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701179+0000", 2024-09-20T22:18:53.961 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0393e-05 2024-09-20T22:18:53.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701245+0000", 2024-09-20T22:18:53.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6519999999999993e-05 2024-09-20T22:18:53.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:53.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:29.701256+0000", 2024-09-20T22:18:53.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0722e-05 2024-09-20T22:18:53.962 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.962 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.962 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.962 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.651217+0000", 2024-09-20T22:18:53.962 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.94996075899999999 2024-09-20T22:18:53.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:53.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:30.654608+0000", 2024-09-20T22:18:53.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0033909819999999999 2024-09-20T22:18:53.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.963 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.963 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.849926+0000", 2024-09-20T22:18:53.963 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.195318353 2024-09-20T22:18:53.963 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.963 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:53.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.850057+0000", 2024-09-20T22:18:53.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013062999999999999 2024-09-20T22:18:53.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.964 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.964 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,4,5", 2024-09-20T22:18:53.964 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.850462+0000", 2024-09-20T22:18:53.964 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00040517 2024-09-20T22:18:53.964 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:53.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.852996+0000", 2024-09-20T22:18:53.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025345179999999999 2024-09-20T22:18:53.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:53.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.854077+0000", 2024-09-20T22:18:53.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001080262 2024-09-20T22:18:53.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.965 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:53.965 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.864365+0000", 2024-09-20T22:18:53.965 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010287907000000001 2024-09-20T22:18:53.965 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.965 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:53.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.864413+0000", 2024-09-20T22:18:53.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8690999999999998e-05 2024-09-20T22:18:53.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:53.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.864473+0000", 2024-09-20T22:18:53.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9487000000000002e-05 2024-09-20T22:18:53.966 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.966 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.966 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:53.966 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:31.864596+0000", 2024-09-20T22:18:53.966 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000123218 2024-09-20T22:18:53.966 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:53.967 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:53.967 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:53.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.967 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.967 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214320 8.8 8:125eda0b:::benchmark_data_smithi060_111051_object94321:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:18:53.967 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:18:53.967 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 476.43580984499999, 2024-09-20T22:18:53.967 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0054397049999999, 2024-09-20T22:18:53.967 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:53.967 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:53.967 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:53.967 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:53.967 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:53.967 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:53.967 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214320 2024-09-20T22:18:53.967 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.968 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:53.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:53.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:18:53.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:53.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:53.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:18:53.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:53.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.968 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.968 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:53.968 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:18:53.968 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0999999999999999e-07 2024-09-20T22:18:53.968 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:53.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496384+0000", 2024-09-20T22:18:53.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1859999999999999e-06 2024-09-20T22:18:53.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:53.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496390+0000", 2024-09-20T22:18:53.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9920000000000004e-06 2024-09-20T22:18:53.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.969 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:53.969 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496395+0000", 2024-09-20T22:18:53.969 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.741e-06 2024-09-20T22:18:53.969 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.969 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496429+0000", 2024-09-20T22:18:53.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3736999999999999e-05 2024-09-20T22:18:53.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:53.970 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496436+0000", 2024-09-20T22:18:53.970 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2529999999999998e-06 2024-09-20T22:18:53.970 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.970 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.970 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.497957+0000", 2024-09-20T22:18:53.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.001521109 2024-09-20T22:18:53.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:53.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.498097+0000", 2024-09-20T22:18:53.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000139562 2024-09-20T22:18:53.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.971 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:18:53.971 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.498422+0000", 2024-09-20T22:18:53.971 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00032548199999999998 2024-09-20T22:18:53.971 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.971 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:53.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501661+0000", 2024-09-20T22:18:53.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0032386580000000002 2024-09-20T22:18:53.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:53.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501669+0000", 2024-09-20T22:18:53.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2500000000000006e-06 2024-09-20T22:18:53.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:53.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501709+0000", 2024-09-20T22:18:53.972 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9304999999999997e-05 2024-09-20T22:18:53.972 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.972 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.972 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:53.972 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501777+0000", 2024-09-20T22:18:53.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8285000000000001e-05 2024-09-20T22:18:53.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.973 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:53.973 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501821+0000", 2024-09-20T22:18:53.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4635000000000002e-05 2024-09-20T22:18:53.973 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:53.973 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:53.973 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:53.973 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.973 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.973 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214327 8.8 8:149a99e2:::benchmark_data_smithi060_111051_object94328:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:18:53.973 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.502392+0000", 2024-09-20T22:18:53.973 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 476.42979942300002, 2024-09-20T22:18:53.973 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.001539808, 2024-09-20T22:18:53.973 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:53.974 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:53.974 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:53.974 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:53.974 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:53.974 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:53.974 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214327 2024-09-20T22:18:53.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.974 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:53.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:53.974 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502392+0000", 2024-09-20T22:18:53.974 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:53.974 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.974 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.974 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:53.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502392+0000", 2024-09-20T22:18:53.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:53.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:53.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502393+0000", 2024-09-20T22:18:53.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2440000000000001e-06 2024-09-20T22:18:53.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.975 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:53.975 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502398+0000", 2024-09-20T22:18:53.975 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1749999999999998e-06 2024-09-20T22:18:53.975 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.975 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:53.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502406+0000", 2024-09-20T22:18:53.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4999999999999999e-06 2024-09-20T22:18:53.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:53.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502411+0000", 2024-09-20T22:18:53.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7119999999999999e-06 2024-09-20T22:18:53.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.976 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502465+0000", 2024-09-20T22:18:53.976 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4206000000000002e-05 2024-09-20T22:18:53.976 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.976 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.976 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:53.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502470+0000", 2024-09-20T22:18:53.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8069999999999999e-06 2024-09-20T22:18:53.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.500319+0000", 2024-09-20T22:18:53.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99784936499999999 2024-09-20T22:18:53.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:53.977 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.500435+0000", 2024-09-20T22:18:53.977 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115439 2024-09-20T22:18:53.977 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.977 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.977 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:18:53.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.500670+0000", 2024-09-20T22:18:53.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023555700000000001 2024-09-20T22:18:53.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:53.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503764+0000", 2024-09-20T22:18:53.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0030938839999999999 2024-09-20T22:18:53.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:53.978 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503827+0000", 2024-09-20T22:18:53.978 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2669999999999995e-05 2024-09-20T22:18:53.978 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.978 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.978 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:53.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503839+0000", 2024-09-20T22:18:53.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2301e-05 2024-09-20T22:18:53.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:53.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503887+0000", 2024-09-20T22:18:53.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8319999999999998e-05 2024-09-20T22:18:53.979 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.979 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.979 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:53.979 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503932+0000", 2024-09-20T22:18:53.979 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4628e-05 2024-09-20T22:18:53.979 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:53.979 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:53.980 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:53.980 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.980 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.980 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214339 8.8 8:166843c0:::benchmark_data_smithi060_111051_object94340:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:18:53.980 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.507919+0000", 2024-09-20T22:18:53.980 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 476.42427232699998, 2024-09-20T22:18:53.980 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99621294400000004, 2024-09-20T22:18:53.980 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:53.980 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:53.980 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:53.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:53.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:53.980 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:53.980 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214339 2024-09-20T22:18:53.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.981 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:53.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:53.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507919+0000", 2024-09-20T22:18:53.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:53.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:53.981 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507919+0000", 2024-09-20T22:18:53.981 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:53.981 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.981 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.981 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:53.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507921+0000", 2024-09-20T22:18:53.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.316e-06 2024-09-20T22:18:53.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:53.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507926+0000", 2024-09-20T22:18:53.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0350000000000001e-06 2024-09-20T22:18:53.982 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.982 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.982 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:53.982 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507937+0000", 2024-09-20T22:18:53.982 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1379999999999999e-05 2024-09-20T22:18:53.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:53.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507945+0000", 2024-09-20T22:18:53.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5780000000000001e-06 2024-09-20T22:18:53.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.508028+0000", 2024-09-20T22:18:53.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3641999999999996e-05 2024-09-20T22:18:53.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.983 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.983 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:53.983 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.508036+0000", 2024-09-20T22:18:53.983 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1780000000000003e-06 2024-09-20T22:18:53.983 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501861+0000", 2024-09-20T22:18:53.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99382438200000001 2024-09-20T22:18:53.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:53.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501935+0000", 2024-09-20T22:18:53.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4151000000000002e-05 2024-09-20T22:18:53.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.984 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.984 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:18:53.984 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.502143+0000", 2024-09-20T22:18:53.984 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020823700000000001 2024-09-20T22:18:53.984 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:53.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503984+0000", 2024-09-20T22:18:53.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001840602 2024-09-20T22:18:53.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:53.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504043+0000", 2024-09-20T22:18:53.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9061000000000003e-05 2024-09-20T22:18:53.985 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.985 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.985 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:53.985 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504055+0000", 2024-09-20T22:18:53.985 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1826e-05 2024-09-20T22:18:53.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:53.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504099+0000", 2024-09-20T22:18:53.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4801e-05 2024-09-20T22:18:53.986 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.986 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.986 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:53.986 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504132+0000", 2024-09-20T22:18:53.986 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2755000000000001e-05 2024-09-20T22:18:53.986 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:53.986 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:53.987 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:53.987 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.987 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.987 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214363 8.8 8:14aba4d5:::benchmark_data_smithi060_111051_object94364:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:18:53.987 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.530169+0000", 2024-09-20T22:18:53.987 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 476.40202239199999, 2024-09-20T22:18:53.987 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98026972499999998, 2024-09-20T22:18:53.987 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:53.987 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:53.987 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:53.987 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:53.987 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:53.987 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:53.987 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214363 2024-09-20T22:18:53.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.988 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:53.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:53.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530169+0000", 2024-09-20T22:18:53.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:53.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:53.988 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530169+0000", 2024-09-20T22:18:53.988 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:53.988 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.988 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.988 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:53.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530171+0000", 2024-09-20T22:18:53.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4300000000000001e-06 2024-09-20T22:18:53.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:53.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530177+0000", 2024-09-20T22:18:53.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8499999999999999e-06 2024-09-20T22:18:53.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.989 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:53.989 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530188+0000", 2024-09-20T22:18:53.989 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1669e-05 2024-09-20T22:18:53.989 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.989 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:53.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530196+0000", 2024-09-20T22:18:53.990 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0760000000000003e-06 2024-09-20T22:18:53.990 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.990 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.990 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.990 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530270+0000", 2024-09-20T22:18:53.990 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:53.991 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3709999999999997e-05 2024-09-20T22:18:53.991 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.991 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:53.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530278+0000", 2024-09-20T22:18:53.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5320000000000006e-06 2024-09-20T22:18:53.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504165+0000", 2024-09-20T22:18:53.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97388663600000003 2024-09-20T22:18:53.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.992 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.992 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:53.992 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504263+0000", 2024-09-20T22:18:53.992 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7412999999999997e-05 2024-09-20T22:18:53.992 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:18:53.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504540+0000", 2024-09-20T22:18:53.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00027761399999999997 2024-09-20T22:18:53.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:53.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510219+0000", 2024-09-20T22:18:53.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005678476 2024-09-20T22:18:53.993 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.993 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.993 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:53.993 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510260+0000", 2024-09-20T22:18:53.993 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1452000000000002e-05 2024-09-20T22:18:53.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:53.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510339+0000", 2024-09-20T22:18:53.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8669000000000002e-05 2024-09-20T22:18:53.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:53.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510393+0000", 2024-09-20T22:18:53.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4160999999999999e-05 2024-09-20T22:18:53.994 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.994 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.994 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:53.994 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510439+0000", 2024-09-20T22:18:53.994 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6037e-05 2024-09-20T22:18:53.995 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:53.995 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:53.995 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:53.995 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.995 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.995 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214367 8.8 8:15e42233:::benchmark_data_smithi060_111051_object94368:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:18:53.995 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.530758+0000", 2024-09-20T22:18:53.995 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 476.40143320800001, 2024-09-20T22:18:53.995 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98471673000000004, 2024-09-20T22:18:53.995 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:53.995 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:53.995 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:53.995 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:18:53.996 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:18:53.996 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:18:53.996 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214367 2024-09-20T22:18:53.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.996 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:53.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:53.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530758+0000", 2024-09-20T22:18:53.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:53.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.996 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.996 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:53.996 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530758+0000", 2024-09-20T22:18:53.996 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:53.996 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:53.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530759+0000", 2024-09-20T22:18:53.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-07 2024-09-20T22:18:53.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:53.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530764+0000", 2024-09-20T22:18:53.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5390000000000001e-06 2024-09-20T22:18:53.997 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.997 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.997 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:53.997 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530773+0000", 2024-09-20T22:18:53.997 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.986e-06 2024-09-20T22:18:53.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:53.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530780+0000", 2024-09-20T22:18:53.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8689999999999998e-06 2024-09-20T22:18:53.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.998 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530827+0000", 2024-09-20T22:18:53.998 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7314000000000001e-05 2024-09-20T22:18:53.998 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.998 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.998 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:18:53.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530832+0000", 2024-09-20T22:18:53.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.758e-06 2024-09-20T22:18:53.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:53.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.505913+0000", 2024-09-20T22:18:53.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97508115200000001 2024-09-20T22:18:53.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:53.999 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:53.999 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.506026+0000", 2024-09-20T22:18:53.999 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011298000000000001 2024-09-20T22:18:53.999 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:53.999 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:18:54.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.506308+0000", 2024-09-20T22:18:54.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000282477 2024-09-20T22:18:54.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:18:54.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510274+0000", 2024-09-20T22:18:54.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0039657160000000002 2024-09-20T22:18:54.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.000 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:54.000 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510478+0000", 2024-09-20T22:18:54.000 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204002 2024-09-20T22:18:54.000 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.000 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:18:54.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.515367+0000", 2024-09-20T22:18:54.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0048889099999999998 2024-09-20T22:18:54.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:54.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.515428+0000", 2024-09-20T22:18:54.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0826999999999999e-05 2024-09-20T22:18:54.001 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.001 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.001 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:54.001 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.515475+0000", 2024-09-20T22:18:54.001 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7388000000000001e-05 2024-09-20T22:18:54.001 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.002 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:54.002 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.002 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.002 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:115927 10.5 e410/370 10:ae1020f4:::benchmark_data_smithi060_167766_object115926:head v 410'7262, mlcod=410'7258)", 2024-09-20T22:18:54.002 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:16:40.299201+0000", 2024-09-20T22:18:54.002 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.632991094, 2024-09-20T22:18:54.002 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061993108999999998, 2024-09-20T22:18:54.002 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:54.002 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:54.002 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:54.002 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:54.002 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:54.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.299201+0000", 2024-09-20T22:18:54.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:54.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.299201+0000", 2024-09-20T22:18:54.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:54.003 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.299202+0000", 2024-09-20T22:18:54.003 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7300000000000004e-07 2024-09-20T22:18:54.003 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.003 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.003 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:54.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.299225+0000", 2024-09-20T22:18:54.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3261e-05 2024-09-20T22:18:54.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:54.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.299236+0000", 2024-09-20T22:18:54.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1590000000000001e-05 2024-09-20T22:18:54.004 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.004 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.004 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:54.004 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.299244+0000", 2024-09-20T22:18:54.004 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8790000000000003e-06 2024-09-20T22:18:54.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:54.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.299362+0000", 2024-09-20T22:18:54.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000118255 2024-09-20T22:18:54.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:54.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.299382+0000", 2024-09-20T22:18:54.005 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9936999999999999e-05 2024-09-20T22:18:54.005 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.005 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.005 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:54.005 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.361087+0000", 2024-09-20T22:18:54.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061704145000000002 2024-09-20T22:18:54.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.006 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:54.006 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.361194+0000", 2024-09-20T22:18:54.006 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000107069 2024-09-20T22:18:54.006 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.006 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:54.006 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.006 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.006 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.006 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:115931 10.5 e410/370 10:a27edb24:::benchmark_data_smithi060_167766_object115930:head v 410'7263, mlcod=410'7259)", 2024-09-20T22:18:54.006 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:16:40.302367+0000", 2024-09-20T22:18:54.007 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.62982481099999, 2024-09-20T22:18:54.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.063059085000000001, 2024-09-20T22:18:54.007 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:54.007 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:54.007 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:54.007 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:54.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.007 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:54.007 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.302367+0000", 2024-09-20T22:18:54.007 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.007 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.007 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:54.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.302367+0000", 2024-09-20T22:18:54.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:54.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.302368+0000", 2024-09-20T22:18:54.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1820000000000001e-06 2024-09-20T22:18:54.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.008 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:54.008 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.302391+0000", 2024-09-20T22:18:54.008 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3479999999999999e-05 2024-09-20T22:18:54.008 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.008 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:54.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.302406+0000", 2024-09-20T22:18:54.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4287000000000001e-05 2024-09-20T22:18:54.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:54.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.302414+0000", 2024-09-20T22:18:54.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2640000000000008e-06 2024-09-20T22:18:54.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.009 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.009 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:54.009 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.303184+0000", 2024-09-20T22:18:54.009 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00076964899999999996 2024-09-20T22:18:54.009 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:54.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.303203+0000", 2024-09-20T22:18:54.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9057999999999999e-05 2024-09-20T22:18:54.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:54.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.364948+0000", 2024-09-20T22:18:54.010 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.061744868000000001 2024-09-20T22:18:54.010 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.010 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.010 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:54.010 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.365426+0000", 2024-09-20T22:18:54.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00047829699999999998 2024-09-20T22:18:54.011 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.011 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:54.011 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.011 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.011 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.011 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:115938 10.f e410/370 10:f921f01a:::benchmark_data_smithi060_167766_object115937:head v 410'7143, mlcod=410'7141)", 2024-09-20T22:18:54.011 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:16:40.305071+0000", 2024-09-20T22:18:54.011 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.62712019400001, 2024-09-20T22:18:54.011 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.056598686000000002, 2024-09-20T22:18:54.011 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:54.011 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:54.011 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:54.011 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:54.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:54.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.305071+0000", 2024-09-20T22:18:54.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:54.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.305071+0000", 2024-09-20T22:18:54.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.012 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.012 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.012 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:54.012 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.305073+0000", 2024-09-20T22:18:54.012 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1990000000000001e-06 2024-09-20T22:18:54.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:54.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.305140+0000", 2024-09-20T22:18:54.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7525000000000001e-05 2024-09-20T22:18:54.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:54.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.354568+0000", 2024-09-20T22:18:54.013 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.049427472 2024-09-20T22:18:54.013 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.013 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.013 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:54.013 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.354588+0000", 2024-09-20T22:18:54.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0207000000000002e-05 2024-09-20T22:18:54.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:54.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.355247+0000", 2024-09-20T22:18:54.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00065898300000000003 2024-09-20T22:18:54.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:54.014 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.355266+0000", 2024-09-20T22:18:54.014 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9015e-05 2024-09-20T22:18:54.014 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.014 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.014 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:54.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.361130+0000", 2024-09-20T22:18:54.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005864073 2024-09-20T22:18:54.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.015 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:54.015 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.361670+0000", 2024-09-20T22:18:54.015 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00054021199999999998 2024-09-20T22:18:54.015 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.015 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:54.015 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.015 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.015 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.015 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:115942 10.4 e410/370 10:21687d53:::benchmark_data_smithi060_167766_object115941:head v 410'7206, mlcod=410'7204)", 2024-09-20T22:18:54.015 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:16:40.308513+0000", 2024-09-20T22:18:54.016 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 133.62367826799999, 2024-09-20T22:18:54.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.053038452, 2024-09-20T22:18:54.016 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:54.016 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:54.016 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:54.016 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:54.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:54.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.308513+0000", 2024-09-20T22:18:54.016 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.016 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.016 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.016 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:54.016 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.308513+0000", 2024-09-20T22:18:54.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:54.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.308515+0000", 2024-09-20T22:18:54.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5409999999999999e-06 2024-09-20T22:18:54.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:54.017 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.308544+0000", 2024-09-20T22:18:54.017 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9045000000000001e-05 2024-09-20T22:18:54.017 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.017 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.017 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:54.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.354799+0000", 2024-09-20T22:18:54.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.046254847000000002 2024-09-20T22:18:54.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:54.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.354811+0000", 2024-09-20T22:18:54.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1768e-05 2024-09-20T22:18:54.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.018 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.018 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:54.018 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.354973+0000", 2024-09-20T22:18:54.018 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000162244 2024-09-20T22:18:54.018 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:54.019 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.354999+0000", 2024-09-20T22:18:54.019 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6478e-05 2024-09-20T22:18:54.019 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.019 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.019 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:54.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.361296+0000", 2024-09-20T22:18:54.020 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0062968590000000001 2024-09-20T22:18:54.020 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.020 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.020 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:54.020 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:16:40.361552+0000", 2024-09-20T22:18:54.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00025567 2024-09-20T22:18:54.021 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.021 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:54.021 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.021 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.021 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.021 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:227868 10.7 e444/370 10:e1e34614:::benchmark_data_smithi060_167766_object103076:head v 444'15700, mlcod=444'15684)", 2024-09-20T22:18:54.021 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:15.902702+0000", 2024-09-20T22:18:54.021 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 38.029489699999999, 2024-09-20T22:18:54.021 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.046320947000000001, 2024-09-20T22:18:54.021 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:54.021 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:54.021 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:54.022 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:54.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:54.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:15.902702+0000", 2024-09-20T22:18:54.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:54.022 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:15.902702+0000", 2024-09-20T22:18:54.022 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.022 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.022 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.022 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:54.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:15.902703+0000", 2024-09-20T22:18:54.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3799999999999996e-07 2024-09-20T22:18:54.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:54.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:15.902709+0000", 2024-09-20T22:18:54.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5880000000000003e-06 2024-09-20T22:18:54.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.023 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:54.023 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:15.902731+0000", 2024-09-20T22:18:54.023 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1610000000000001e-05 2024-09-20T22:18:54.023 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.023 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:54.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:15.902737+0000", 2024-09-20T22:18:54.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6380000000000001e-06 2024-09-20T22:18:54.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:54.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:15.922444+0000", 2024-09-20T22:18:54.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.019707572999999999 2024-09-20T22:18:54.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.024 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:54.024 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:15.922503+0000", 2024-09-20T22:18:54.024 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8671000000000001e-05 2024-09-20T22:18:54.024 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.024 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:54.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:15.948984+0000", 2024-09-20T22:18:54.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.026480980000000001 2024-09-20T22:18:54.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.025 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:54.025 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:15.949023+0000", 2024-09-20T22:18:54.025 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9048999999999998e-05 2024-09-20T22:18:54.025 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.025 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:54.025 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.025 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.025 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.025 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:240321 10.5 e446/370 10:ac24d118:::benchmark_data_smithi060_167766_object115529:head v 446'15077, mlcod=446'15074)", 2024-09-20T22:18:54.026 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:26.284160+0000", 2024-09-20T22:18:54.026 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 27.648031976999999, 2024-09-20T22:18:54.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.121285268, 2024-09-20T22:18:54.026 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:54.026 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:54.026 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:54.026 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:54.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.026 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:54.026 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.284160+0000", 2024-09-20T22:18:54.026 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.026 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.026 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:54.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.284160+0000", 2024-09-20T22:18:54.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:54.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.284161+0000", 2024-09-20T22:18:54.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.124e-06 2024-09-20T22:18:54.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.027 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.027 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:54.027 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.284167+0000", 2024-09-20T22:18:54.027 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6030000000000001e-06 2024-09-20T22:18:54.027 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:54.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.284178+0000", 2024-09-20T22:18:54.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0463e-05 2024-09-20T22:18:54.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:54.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.284184+0000", 2024-09-20T22:18:54.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6529999999999997e-06 2024-09-20T22:18:54.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.028 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.028 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:54.028 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.284284+0000", 2024-09-20T22:18:54.028 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9844000000000002e-05 2024-09-20T22:18:54.028 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:54.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.284317+0000", 2024-09-20T22:18:54.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2551000000000001e-05 2024-09-20T22:18:54.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:54.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.405399+0000", 2024-09-20T22:18:54.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.121082444 2024-09-20T22:18:54.029 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.029 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.029 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:54.029 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.405445+0000", 2024-09-20T22:18:54.029 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5586e-05 2024-09-20T22:18:54.029 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.030 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:54.030 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.030 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.030 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:240327 10.5 e446/370 10:a6c7a20c:::benchmark_data_smithi060_167766_object115535:head v 446'15078, mlcod=446'15074)", 2024-09-20T22:18:54.030 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:26.284206+0000", 2024-09-20T22:18:54.030 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 27.64798592, 2024-09-20T22:18:54.030 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.13031405900000001, 2024-09-20T22:18:54.030 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:54.030 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:54.030 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:54.030 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:54.030 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:54.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.284206+0000", 2024-09-20T22:18:54.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:54.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.284206+0000", 2024-09-20T22:18:54.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.031 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:54.031 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.284206+0000", 2024-09-20T22:18:54.031 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8500000000000002e-07 2024-09-20T22:18:54.031 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.031 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:54.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.284210+0000", 2024-09-20T22:18:54.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1710000000000004e-06 2024-09-20T22:18:54.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:54.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.284217+0000", 2024-09-20T22:18:54.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8950000000000001e-06 2024-09-20T22:18:54.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:54.032 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.284222+0000", 2024-09-20T22:18:54.032 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1250000000000001e-06 2024-09-20T22:18:54.032 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.032 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.032 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:54.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.287492+0000", 2024-09-20T22:18:54.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0032701520000000001 2024-09-20T22:18:54.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:54.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.287511+0000", 2024-09-20T22:18:54.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8359000000000002e-05 2024-09-20T22:18:54.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.033 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:54.033 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.414476+0000", 2024-09-20T22:18:54.033 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.126965195 2024-09-20T22:18:54.033 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.033 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.034 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:54.034 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.414520+0000", 2024-09-20T22:18:54.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3776999999999998e-05 2024-09-20T22:18:54.034 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.034 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:54.034 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.034 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.034 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.034 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:240330 10.5 e446/370 10:a5626458:::benchmark_data_smithi060_167766_object115538:head v 446'15079, mlcod=446'15074)", 2024-09-20T22:18:54.034 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:26.286049+0000", 2024-09-20T22:18:54.034 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 27.646142952000002, 2024-09-20T22:18:54.034 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.13165033400000001, 2024-09-20T22:18:54.034 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:54.035 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:54.035 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:54.035 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:54.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.035 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:54.035 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.286049+0000", 2024-09-20T22:18:54.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.035 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:54.035 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.286049+0000", 2024-09-20T22:18:54.035 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.035 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.035 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:54.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.286049+0000", 2024-09-20T22:18:54.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6800000000000002e-07 2024-09-20T22:18:54.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:54.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.286053+0000", 2024-09-20T22:18:54.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.771e-06 2024-09-20T22:18:54.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.036 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:54.036 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.286059+0000", 2024-09-20T22:18:54.036 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0750000000000003e-06 2024-09-20T22:18:54.036 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.036 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:54.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.286065+0000", 2024-09-20T22:18:54.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0050000000000001e-06 2024-09-20T22:18:54.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:54.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.402975+0000", 2024-09-20T22:18:54.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11691032799999999 2024-09-20T22:18:54.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.037 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.037 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:54.037 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.403004+0000", 2024-09-20T22:18:54.037 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8470999999999999e-05 2024-09-20T22:18:54.037 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:54.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.417560+0000", 2024-09-20T22:18:54.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.01455613 2024-09-20T22:18:54.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.038 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:54.038 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.417699+0000", 2024-09-20T22:18:54.038 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013928599999999999 2024-09-20T22:18:54.038 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.038 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:54.038 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.038 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.038 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.039 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:240339 10.f e446/370 10:f9b6c144:::benchmark_data_smithi060_167766_object115547:head v 446'14827, mlcod=446'14824)", 2024-09-20T22:18:54.039 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:26.286083+0000", 2024-09-20T22:18:54.039 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 27.646108699999999, 2024-09-20T22:18:54.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11706145799999999, 2024-09-20T22:18:54.039 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:54.039 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:54.039 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:54.039 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:54.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.039 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:54.039 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.286083+0000", 2024-09-20T22:18:54.039 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.039 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.039 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:54.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.286083+0000", 2024-09-20T22:18:54.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:54.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.286083+0000", 2024-09-20T22:18:54.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5199999999999998e-07 2024-09-20T22:18:54.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.040 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.040 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:54.040 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.286087+0000", 2024-09-20T22:18:54.040 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.546e-06 2024-09-20T22:18:54.040 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:54.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.286093+0000", 2024-09-20T22:18:54.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3509999999999998e-06 2024-09-20T22:18:54.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:54.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.286098+0000", 2024-09-20T22:18:54.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4329999999999997e-06 2024-09-20T22:18:54.041 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.041 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.041 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:54.041 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.288273+0000", 2024-09-20T22:18:54.041 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021752210000000002 2024-09-20T22:18:54.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:54.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.288292+0000", 2024-09-20T22:18:54.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9199e-05 2024-09-20T22:18:54.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:54.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.403093+0000", 2024-09-20T22:18:54.042 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.11480077399999999 2024-09-20T22:18:54.042 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.042 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.042 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:54.042 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.403144+0000", 2024-09-20T22:18:54.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1582000000000002e-05 2024-09-20T22:18:54.043 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.043 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:54.043 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.043 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.043 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.043 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:240334 10.5 e446/370 10:ae695d26:::benchmark_data_smithi060_167766_object115542:head v 446'15080, mlcod=446'15075)", 2024-09-20T22:18:54.043 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:26.291035+0000", 2024-09-20T22:18:54.043 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 27.641157066000002, 2024-09-20T22:18:54.043 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.126838794, 2024-09-20T22:18:54.043 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:54.043 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:54.043 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:54.043 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:54.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:54.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.291035+0000", 2024-09-20T22:18:54.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:54.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.291035+0000", 2024-09-20T22:18:54.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.044 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.044 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:54.044 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.291036+0000", 2024-09-20T22:18:54.044 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.029e-06 2024-09-20T22:18:54.044 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:54.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.291043+0000", 2024-09-20T22:18:54.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1659999999999997e-06 2024-09-20T22:18:54.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:54.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.402916+0000", 2024-09-20T22:18:54.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.111873299 2024-09-20T22:18:54.045 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.045 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.045 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:54.045 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.402938+0000", 2024-09-20T22:18:54.045 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1481999999999999e-05 2024-09-20T22:18:54.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:54.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.405505+0000", 2024-09-20T22:18:54.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025670699999999999 2024-09-20T22:18:54.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:54.046 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.405525+0000", 2024-09-20T22:18:54.046 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0534999999999999e-05 2024-09-20T22:18:54.046 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.046 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.046 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:54.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.417704+0000", 2024-09-20T22:18:54.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.012178787999999999 2024-09-20T22:18:54.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.047 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:54.047 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.417873+0000", 2024-09-20T22:18:54.047 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016942500000000001 2024-09-20T22:18:54.047 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.047 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:54.047 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.047 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.047 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.047 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:240348 10.4 e446/370 10:2fdf786d:::benchmark_data_smithi060_167766_object115556:head v 446'14977, mlcod=446'14975)", 2024-09-20T22:18:54.047 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:26.295070+0000", 2024-09-20T22:18:54.047 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 27.637121238999999, 2024-09-20T22:18:54.048 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.113264304, 2024-09-20T22:18:54.048 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:18:54.048 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:18:54.048 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:18:54.048 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:18:54.048 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.048 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:18:54.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.295070+0000", 2024-09-20T22:18:54.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:18:54.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.295070+0000", 2024-09-20T22:18:54.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:18:54.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.049 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:18:54.049 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.295072+0000", 2024-09-20T22:18:54.049 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2279999999999999e-06 2024-09-20T22:18:54.049 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.049 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:18:54.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.295081+0000", 2024-09-20T22:18:54.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0240000000000003e-06 2024-09-20T22:18:54.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:18:54.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.402914+0000", 2024-09-20T22:18:54.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.10783359000000001 2024-09-20T22:18:54.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.050 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:18:54.050 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.402938+0000", 2024-09-20T22:18:54.050 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3485999999999998e-05 2024-09-20T22:18:54.050 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.050 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:18:54.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.403127+0000", 2024-09-20T22:18:54.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018931699999999999 2024-09-20T22:18:54.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:18:54.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.403150+0000", 2024-09-20T22:18:54.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2997e-05 2024-09-20T22:18:54.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.051 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:18:54.051 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.407760+0000", 2024-09-20T22:18:54.051 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0046095290000000002 2024-09-20T22:18:54.051 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:18:54.051 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:18:54.052 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:18:54.052 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:26.408335+0000", 2024-09-20T22:18:54.052 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00057513300000000002 2024-09-20T22:18:54.052 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.052 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:54.052 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.052 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:18:54.052 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:18:54.052 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:18:54.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:54.053 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:54.053 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:54.094 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 1 16 3327 3311 206.927 206.938 0.00472891 0.00481448 2024-09-20T22:18:54.153 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:54.192 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:54.193 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:54.193 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:54.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:53 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:18:54.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:53 smithi067 ceph-mon[33511]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T22:18:54.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:53 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:18:54.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:53 smithi060 ceph-mon[29694]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T22:18:54.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:53 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:18:54.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:53 smithi060 ceph-mon[40290]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-20T22:18:54.294 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:54.328 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:54.329 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:54.329 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:54.430 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:54.464 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:54.464 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:54.465 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:54.562 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T22:18:54.562 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-20T22:18:54.563 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.0 'trim stale osdmaps' 2024-09-20T22:18:54.565 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:54.600 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:54.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:54.600 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:54.701 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:54.735 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:54.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:54.736 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:54.837 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:54.869 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:54.870 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:54.870 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:54.971 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:55.004 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:55.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:55.005 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:55.094 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 2 16 6538 6522 203.799 200.688 0.0057289 0.00489556 2024-09-20T22:18:55.105 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:55.146 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:55.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:55.234 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:55.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:54 smithi067 ceph-mon[33511]: pgmap v857: 178 pgs: 178 active+clean; 21 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 764 B/s rd, 3.1 MiB/s wr, 50 op/s 2024-09-20T22:18:55.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:54 smithi060 ceph-mon[29694]: pgmap v857: 178 pgs: 178 active+clean; 21 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 764 B/s rd, 3.1 MiB/s wr, 50 op/s 2024-09-20T22:18:55.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:54 smithi060 ceph-mon[40290]: pgmap v857: 178 pgs: 178 active+clean; 21 MiB data, 5.2 GiB used, 710 GiB / 715 GiB avail; 764 B/s rd, 3.1 MiB/s wr, 50 op/s 2024-09-20T22:18:55.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:18:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:18:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:18:55.335 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:55.377 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:55.378 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:55.378 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:55.479 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:55.518 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:55.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:55.519 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:55.620 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:55.654 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:55.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:55.655 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:55.756 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:55.790 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:55.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:55.791 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:55.892 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:55.930 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:55.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:55.931 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:56.006 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:56.032 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:56.072 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:56.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:56.074 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:56.094 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 3 16 9536 9520 198.319 187.375 0.0054568 0.0050342 2024-09-20T22:18:56.174 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:56.210 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:56.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:56.211 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:56.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:55 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:18:56.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:55 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:18:56.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:55 smithi067 ceph-mon[33511]: osdmap e461: 8 total, 8 up, 8 in 2024-09-20T22:18:56.264 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:18:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=sqlstore.transactions t=2024-09-20T22:18:56.151476852Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-20T22:18:56.264 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:18:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=sqlstore.transactions t=2024-09-20T22:18:56.162016149Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-20T22:18:56.285 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:55 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:18:56.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:55 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:18:56.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:55 smithi060 ceph-mon[40290]: osdmap e461: 8 total, 8 up, 8 in 2024-09-20T22:18:56.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:55 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:18:56.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:55 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:18:56.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:55 smithi060 ceph-mon[29694]: osdmap e461: 8 total, 8 up, 8 in 2024-09-20T22:18:56.311 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:56.346 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:56.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:56.347 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:56.448 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:56.474 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T22:18:56.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:56.501 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:56.501 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:56.510 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:18:56.601 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:56.650 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:56.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:56.652 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:56.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:56.793 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:56.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:56.794 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:56.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:56.930 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:56.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:56.931 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:57.032 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:57.068 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:18:57.072 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:57.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:57.073 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:57.095 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 4 16 12347 12331 192.647 175.688 0.00434237 0.00518322 2024-09-20T22:18:57.174 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:57.213 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:57.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:57.215 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:57.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:56 smithi067 ceph-mon[33511]: pgmap v859: 178 pgs: 178 active+clean; 162 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 27 MiB/s wr, 431 op/s 2024-09-20T22:18:57.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:56 smithi060 ceph-mon[29694]: pgmap v859: 178 pgs: 178 active+clean; 162 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 27 MiB/s wr, 431 op/s 2024-09-20T22:18:57.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:56 smithi060 ceph-mon[40290]: pgmap v859: 178 pgs: 178 active+clean; 162 MiB data, 5.4 GiB used, 710 GiB / 715 GiB avail; 1.7 KiB/s rd, 27 MiB/s wr, 431 op/s 2024-09-20T22:18:57.296 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:18:57.296 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:18:57.315 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:57.345 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:18:57.345 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:18:57.348 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:57.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:57.349 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:57.384 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:18:57.385 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:18:57.420 INFO:teuthology.orchestra.run.smithi060.stderr:no valid command found; 10 closest matches: 2024-09-20T22:18:57.420 INFO:teuthology.orchestra.run.smithi060.stderr:0 2024-09-20T22:18:57.420 INFO:teuthology.orchestra.run.smithi060.stderr:1 2024-09-20T22:18:57.420 INFO:teuthology.orchestra.run.smithi060.stderr:2 2024-09-20T22:18:57.420 INFO:teuthology.orchestra.run.smithi060.stderr:abort 2024-09-20T22:18:57.421 INFO:teuthology.orchestra.run.smithi060.stderr:assert 2024-09-20T22:18:57.421 INFO:teuthology.orchestra.run.smithi060.stderr:bench [] [] [] [] 2024-09-20T22:18:57.421 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs debug_inject_read_zeros 2024-09-20T22:18:57.421 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs files list 2024-09-20T22:18:57.421 INFO:teuthology.orchestra.run.smithi060.stderr:bluefs stats 2024-09-20T22:18:57.421 INFO:teuthology.orchestra.run.smithi060.stderr:bluestore allocator dump block 2024-09-20T22:18:57.421 INFO:teuthology.orchestra.run.smithi060.stderr:Error EINVAL: invalid command 2024-09-20T22:18:57.424 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:18:57.424 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:18:57.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:57.464 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:18:57.464 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:18:57.484 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:57.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:57.485 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:57.504 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:18:57.504 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:18:57.543 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:18:57.543 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:18:57.577 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:18:57.577 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:18:57.586 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:57.620 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:57.621 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:57.621 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:57.722 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:57.745 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:57.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:57.746 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:57.794 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:18:57.795 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:18:57.846 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:57.891 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:57.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:57.893 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:57.986 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:18:57.993 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:18:58.075 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:58.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:58.076 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:18:58.094 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 5 16 14662 14646 183.053 144.688 0.00535176 0.00545489 2024-09-20T22:18:58.177 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:18:58.214 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:58.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:58.215 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:18:58.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:58 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2806179960' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:58.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:58 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2806179960' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:58.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:58 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2806179960' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:18:58.316 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:58.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:58.359 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:58.359 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:58.460 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:58.499 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:58.499 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:58.500 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:58.600 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:18:58.622 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:58.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:58.623 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:18:58.724 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:18:58.746 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:58.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:58.747 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:18:58.848 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:58.882 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:58.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:58.883 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:58.983 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:59.016 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:59.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:59.017 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:59.074 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T22:18:59.094 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 6 16 16694 16678 173.709 127 0.00485971 0.00575014 2024-09-20T22:18:59.118 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:59.156 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:59.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:59.157 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:59.258 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:59.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:18:59 smithi067 ceph-mon[33511]: pgmap v860: 178 pgs: 178 active+clean; 658 MiB data, 6.7 GiB used, 708 GiB / 715 GiB avail; 696 B/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-20T22:18:59.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:18:59 smithi060 ceph-mon[29694]: pgmap v860: 178 pgs: 178 active+clean; 658 MiB data, 6.7 GiB used, 708 GiB / 715 GiB avail; 696 B/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-20T22:18:59.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:18:59 smithi060 ceph-mon[40290]: pgmap v860: 178 pgs: 178 active+clean; 658 MiB data, 6.7 GiB used, 708 GiB / 715 GiB avail; 696 B/s rd, 89 MiB/s wr, 1.43k op/s 2024-09-20T22:18:59.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:59.289 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:59.289 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:59.390 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:18:59.427 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:59.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:59.428 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:18:59.455 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:18:59.529 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:18:59.562 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:59.562 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:59.563 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:18:59.664 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:18:59.688 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:18:59.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:59.689 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:18:59.789 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:59.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:59.827 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:59.828 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:18:59.928 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:18:59.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:18:59.969 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:18:59.970 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:00.070 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:00.096 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 7 16 18561 18545 165.533 116.688 0.00375345 0.00603421 2024-09-20T22:19:00.108 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:00.109 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:00.109 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:00.210 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:00.248 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:00.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:00.250 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:00.351 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:00.389 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:00.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:00.391 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:00.492 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:00.528 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:00.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:00.529 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:00.629 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:00.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:00.666 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:00.666 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:00.767 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:00.804 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:00.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:00.805 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:00.906 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:00.919 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T22:19:00.920 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-09-20T22:19:00.941 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:00.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:00.943 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:01.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:01.059 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:19:01.059 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:19:01.073 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:01.091 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:01.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:01.092 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:01.096 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 8 16 21040 21024 164.207 154.938 0.0168424 0.00608211 2024-09-20T22:19:01.193 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:01.237 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:01.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:01.239 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:01.340 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:01.375 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:01.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:01.375 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:01.476 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:01.511 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:01.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:01.512 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:01.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:01 smithi060 ceph-mon[29694]: pgmap v861: 178 pgs: 178 active+clean; 766 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-20T22:19:01.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:01 smithi060 ceph-mon[40290]: pgmap v861: 178 pgs: 178 active+clean; 766 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-20T22:19:01.613 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:01.653 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:01.654 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:01.654 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:01.754 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:01 smithi067 ceph-mon[33511]: pgmap v861: 178 pgs: 178 active+clean; 766 MiB data, 7.2 GiB used, 708 GiB / 715 GiB avail; 639 B/s rd, 96 MiB/s wr, 1.53k op/s 2024-09-20T22:19:01.785 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:01.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:01.786 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:01.886 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:01.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:01.928 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:01.928 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:02.029 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:02.060 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:02.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:02.061 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:02.096 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 9 16 23302 23286 161.669 141.375 0.00742535 0.00617902 2024-09-20T22:19:02.162 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:02.198 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:02.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:02.199 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:02.300 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:02.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:02.337 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:02.337 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:02.431 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:19:02.439 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:02.470 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:02.471 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:02.471 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:02.572 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:02.606 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:02.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:02.607 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:02.639 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:19:02.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:02.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:02.749 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:02.749 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:02.772 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T22:19:02.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:02 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2054075104' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:19:02.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:02 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2054075104' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:19:02.850 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:02.886 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:02.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:02.887 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:02.988 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:03.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:19:02 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:19:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:02 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2054075104' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:19:03.025 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:03.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:03.026 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:03.096 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 10 15 25449 25434 158.925 134.25 0.00885004 0.0062862 2024-09-20T22:19:03.127 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:03.167 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:03.168 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:03.168 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:03.201 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:19:03.269 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:03.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:03.294 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:03.294 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:03.395 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:03.430 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:03.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:03.431 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:03.532 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:03.576 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:03.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:03.577 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:03.678 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:03.719 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:03.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:03.720 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:03.795 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:19:03.796 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:19:03.821 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:03.855 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:03.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:03.857 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:03.958 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:03.994 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:03.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:03.995 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:03 smithi067 ceph-mon[33511]: pgmap v862: 178 pgs: 178 active+clean; 1.1 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.0 KiB/s rd, 120 MiB/s wr, 1.92k op/s 2024-09-20T22:19:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:03 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:19:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:03 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2054075104' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:19:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:03 smithi067 ceph-mon[33511]: osdmap e462: 8 total, 8 up, 8 in 2024-09-20T22:19:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:03 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:19:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:03 smithi067 ceph-mon[33511]: 5.13 scrub starts 2024-09-20T22:19:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:03 smithi067 ceph-mon[33511]: 5.13 scrub ok 2024-09-20T22:19:04.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:03 smithi060 ceph-mon[29694]: pgmap v862: 178 pgs: 178 active+clean; 1.1 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.0 KiB/s rd, 120 MiB/s wr, 1.92k op/s 2024-09-20T22:19:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:03 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:19:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:03 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2054075104' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:19:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:03 smithi060 ceph-mon[29694]: osdmap e462: 8 total, 8 up, 8 in 2024-09-20T22:19:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:03 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:19:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:03 smithi060 ceph-mon[29694]: 5.13 scrub starts 2024-09-20T22:19:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:03 smithi060 ceph-mon[29694]: 5.13 scrub ok 2024-09-20T22:19:04.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:03 smithi060 ceph-mon[40290]: pgmap v862: 178 pgs: 178 active+clean; 1.1 GiB data, 8.4 GiB used, 707 GiB / 715 GiB avail; 1.0 KiB/s rd, 120 MiB/s wr, 1.92k op/s 2024-09-20T22:19:04.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:03 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:19:04.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:03 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2054075104' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:19:04.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:03 smithi060 ceph-mon[40290]: osdmap e462: 8 total, 8 up, 8 in 2024-09-20T22:19:04.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:03 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:19:04.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:03 smithi060 ceph-mon[40290]: 5.13 scrub starts 2024-09-20T22:19:04.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:03 smithi060 ceph-mon[40290]: 5.13 scrub ok 2024-09-20T22:19:04.096 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:04.097 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 11 16 27457 27441 155.881 125.438 0.00615021 0.00640943 2024-09-20T22:19:04.135 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:04.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:04.136 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:04.237 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:04.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:04.268 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:04.268 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:04.369 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:04.401 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:04.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:04.402 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:04.503 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:04.538 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:04.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:04.539 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:04.640 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:04.678 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:04.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:04.679 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:04.679 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T22:19:04.779 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:04.824 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:04.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:04.824 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:04.925 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:04.961 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:04.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:04.962 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:04 smithi067 ceph-mon[33511]: 5.8 scrub starts 2024-09-20T22:19:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:04 smithi067 ceph-mon[33511]: 5.8 scrub ok 2024-09-20T22:19:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:04 smithi067 ceph-mon[33511]: 1.16 scrub starts 2024-09-20T22:19:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:04 smithi067 ceph-mon[33511]: 1.16 scrub ok 2024-09-20T22:19:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:04 smithi067 ceph-mon[33511]: pgmap v864: 178 pgs: 178 active+clean; 1.4 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.1 KiB/s rd, 154 MiB/s wr, 2.47k op/s 2024-09-20T22:19:05.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:04 smithi060 ceph-mon[29694]: 5.8 scrub starts 2024-09-20T22:19:05.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:04 smithi060 ceph-mon[29694]: 5.8 scrub ok 2024-09-20T22:19:05.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:04 smithi060 ceph-mon[29694]: 1.16 scrub starts 2024-09-20T22:19:05.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:04 smithi060 ceph-mon[29694]: 1.16 scrub ok 2024-09-20T22:19:05.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:04 smithi060 ceph-mon[29694]: pgmap v864: 178 pgs: 178 active+clean; 1.4 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.1 KiB/s rd, 154 MiB/s wr, 2.47k op/s 2024-09-20T22:19:05.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:04 smithi060 ceph-mon[40290]: 5.8 scrub starts 2024-09-20T22:19:05.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:04 smithi060 ceph-mon[40290]: 5.8 scrub ok 2024-09-20T22:19:05.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:04 smithi060 ceph-mon[40290]: 1.16 scrub starts 2024-09-20T22:19:05.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:04 smithi060 ceph-mon[40290]: 1.16 scrub ok 2024-09-20T22:19:05.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:04 smithi060 ceph-mon[40290]: pgmap v864: 178 pgs: 178 active+clean; 1.4 GiB data, 9.3 GiB used, 706 GiB / 715 GiB avail; 1.1 KiB/s rd, 154 MiB/s wr, 2.47k op/s 2024-09-20T22:19:05.063 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:05.096 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 12 16 29483 29467 153.442 126.625 0.0160561 0.00651066 2024-09-20T22:19:05.105 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:19:05.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:05.106 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:05.106 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:05.206 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:05.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:05.248 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:05.248 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:05.349 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:05.389 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:05.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:05.390 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:05.421 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:19:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:19:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:19:05.491 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:05.529 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:05.530 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:05.530 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:05.631 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:05.680 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:05.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:05.685 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:05.748 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:19:05.748 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:19:05.785 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:05.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:05 smithi060 ceph-mon[29694]: 5.f scrub starts 2024-09-20T22:19:05.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:05 smithi060 ceph-mon[29694]: 5.f scrub ok 2024-09-20T22:19:05.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:05 smithi060 ceph-mon[40290]: 5.f scrub starts 2024-09-20T22:19:05.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:05 smithi060 ceph-mon[40290]: 5.f scrub ok 2024-09-20T22:19:05.823 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:05.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:05.824 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:05.921 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:19:05.921 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:19:05.921 INFO:tasks.thrashosds.thrasher:fixing pg num pool unique_pool_4 2024-09-20T22:19:05.921 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:19:05.925 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:05.984 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:05 smithi067 ceph-mon[33511]: 5.f scrub starts 2024-09-20T22:19:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:05 smithi067 ceph-mon[33511]: 5.f scrub ok 2024-09-20T22:19:06.022 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:06.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:06.023 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:06.097 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 13 16 31508 31492 151.363 126.562 0.0108476 0.00660095 2024-09-20T22:19:06.124 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:06.152 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:06.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:06.154 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:06.254 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:06.313 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:06.313 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:06.314 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:06.415 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:06.448 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:06.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:06.449 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:06.549 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:06.582 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:06.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:06.583 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:06.672 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:06.684 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:06.720 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:19:06.720 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:19:06.724 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:06.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:06.726 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:06.757 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:06 smithi060 ceph-mon[40290]: 1.11 scrub starts 2024-09-20T22:19:06.757 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:06 smithi060 ceph-mon[40290]: 1.11 scrub ok 2024-09-20T22:19:06.757 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:06 smithi060 ceph-mon[40290]: 3.13 scrub starts 2024-09-20T22:19:06.757 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:06 smithi060 ceph-mon[40290]: 3.13 scrub ok 2024-09-20T22:19:06.757 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:19:06.757 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:19:06.757 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:06 smithi060 ceph-mon[40290]: pgmap v865: 178 pgs: 178 active+clean; 1.6 GiB data, 9.7 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 146 MiB/s wr, 2.34k op/s 2024-09-20T22:19:06.757 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:06 smithi060 ceph-mon[40290]: osdmap e463: 8 total, 8 up, 8 in 2024-09-20T22:19:06.758 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:19:06.758 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:19:06.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:06 smithi060 ceph-mon[29694]: 1.11 scrub starts 2024-09-20T22:19:06.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:06 smithi060 ceph-mon[29694]: 1.11 scrub ok 2024-09-20T22:19:06.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:06 smithi060 ceph-mon[29694]: 3.13 scrub starts 2024-09-20T22:19:06.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:06 smithi060 ceph-mon[29694]: 3.13 scrub ok 2024-09-20T22:19:06.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:19:06.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:19:06.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:06 smithi060 ceph-mon[29694]: pgmap v865: 178 pgs: 178 active+clean; 1.6 GiB data, 9.7 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 146 MiB/s wr, 2.34k op/s 2024-09-20T22:19:06.759 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:06 smithi060 ceph-mon[29694]: osdmap e463: 8 total, 8 up, 8 in 2024-09-20T22:19:06.796 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:19:06.796 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:19:06.826 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:06.833 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:19:06.833 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:19:06.868 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:19:06.868 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:19:06.871 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:06.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:06.872 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:06.893 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:19:06.893 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:19:06.929 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:19:06.929 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:19:06.967 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:19:06.967 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:19:06.973 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:07.012 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:07.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:07.013 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:06 smithi067 ceph-mon[33511]: 1.11 scrub starts 2024-09-20T22:19:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:06 smithi067 ceph-mon[33511]: 1.11 scrub ok 2024-09-20T22:19:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:06 smithi067 ceph-mon[33511]: 3.13 scrub starts 2024-09-20T22:19:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:06 smithi067 ceph-mon[33511]: 3.13 scrub ok 2024-09-20T22:19:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:19:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:19:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:06 smithi067 ceph-mon[33511]: pgmap v865: 178 pgs: 178 active+clean; 1.6 GiB data, 9.7 GiB used, 705 GiB / 715 GiB avail; 1023 B/s rd, 146 MiB/s wr, 2.34k op/s 2024-09-20T22:19:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:06 smithi067 ceph-mon[33511]: osdmap e463: 8 total, 8 up, 8 in 2024-09-20T22:19:07.097 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 14 15 33229 33214 148.238 107.625 0.00553813 0.00674062 2024-09-20T22:19:07.114 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:07.137 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:07.138 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:07.138 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:07.239 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:07.275 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:07.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:07.276 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:07.377 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:07.398 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:07.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:07.399 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:07.500 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:07.536 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:07.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:07.537 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:07.638 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:07.670 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:07.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:07.671 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:07.772 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:07.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:07.814 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:07.814 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:07.895 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:19:07.915 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:07.930 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:07 smithi060 ceph-mon[29694]: 1.12 scrub starts 2024-09-20T22:19:07.930 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:07 smithi060 ceph-mon[29694]: 1.12 scrub ok 2024-09-20T22:19:07.931 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:07 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1068068353' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:07.931 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:07 smithi060 ceph-mon[40290]: 1.12 scrub starts 2024-09-20T22:19:07.931 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:07 smithi060 ceph-mon[40290]: 1.12 scrub ok 2024-09-20T22:19:07.931 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:07 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1068068353' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:07.947 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:07.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:07.948 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:08.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:07 smithi067 ceph-mon[33511]: 1.12 scrub starts 2024-09-20T22:19:08.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:07 smithi067 ceph-mon[33511]: 1.12 scrub ok 2024-09-20T22:19:08.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:07 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1068068353' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:08.049 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:08.082 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:08.082 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:08.083 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:08.098 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 15 16 34788 34772 144.847 97.375 0.0047053 0.00689856 2024-09-20T22:19:08.183 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:08.217 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:08.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:08.218 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:08.318 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:08.347 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:08.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:08.348 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:08.449 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:08.480 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:08.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:08.481 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:08.581 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:08.624 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:08.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:08.625 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:08.726 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:08.762 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:08.763 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:08.764 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:08.864 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:08.918 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:08.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:08.919 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:08.921 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:08 smithi060 ceph-mon[29694]: 4.2 scrub starts 2024-09-20T22:19:08.921 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:08 smithi060 ceph-mon[29694]: 4.2 scrub ok 2024-09-20T22:19:08.921 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:08 smithi060 ceph-mon[29694]: pgmap v867: 178 pgs: 178 active+clean; 2.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 155 MiB/s wr, 2.48k op/s 2024-09-20T22:19:08.921 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:08 smithi060 ceph-mon[40290]: 4.2 scrub starts 2024-09-20T22:19:08.921 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:08 smithi060 ceph-mon[40290]: 4.2 scrub ok 2024-09-20T22:19:08.921 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:08 smithi060 ceph-mon[40290]: pgmap v867: 178 pgs: 178 active+clean; 2.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 155 MiB/s wr, 2.48k op/s 2024-09-20T22:19:09.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:08 smithi067 ceph-mon[33511]: 4.2 scrub starts 2024-09-20T22:19:09.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:08 smithi067 ceph-mon[33511]: 4.2 scrub ok 2024-09-20T22:19:09.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:08 smithi067 ceph-mon[33511]: pgmap v867: 178 pgs: 178 active+clean; 2.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 1.2 KiB/s rd, 155 MiB/s wr, 2.48k op/s 2024-09-20T22:19:09.021 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:09.059 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:09.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:09.060 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:09.097 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 16 15 36399 36384 142.091 100.75 0.00477564 0.00703336 2024-09-20T22:19:09.161 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:09.193 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:09.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:09.194 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:09.294 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:09.326 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:09.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:09.327 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:09.427 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:09.465 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:09.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:09.466 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:09.567 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:09.600 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:09.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:09.601 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:09.702 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:09.723 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:09.724 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:09.724 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:09.825 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:09.856 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:09.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:09.857 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:09.920 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:09.938 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:09 smithi060 ceph-mon[29694]: 6.b scrub starts 2024-09-20T22:19:09.939 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:09 smithi060 ceph-mon[29694]: 6.b scrub ok 2024-09-20T22:19:09.939 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:09 smithi060 ceph-mon[29694]: 5.6 scrub starts 2024-09-20T22:19:09.939 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:09 smithi060 ceph-mon[29694]: 5.6 scrub ok 2024-09-20T22:19:09.939 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:09 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:19:09.939 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:09 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:19:09.939 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:09 smithi060 ceph-mon[40290]: 6.b scrub starts 2024-09-20T22:19:09.939 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:09 smithi060 ceph-mon[40290]: 6.b scrub ok 2024-09-20T22:19:09.939 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:09 smithi060 ceph-mon[40290]: 5.6 scrub starts 2024-09-20T22:19:09.939 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:09 smithi060 ceph-mon[40290]: 5.6 scrub ok 2024-09-20T22:19:09.940 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:09 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:19:09.940 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:09 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:19:09.958 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:09.992 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:09.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:09.993 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:10.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:09 smithi067 ceph-mon[33511]: 6.b scrub starts 2024-09-20T22:19:10.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:09 smithi067 ceph-mon[33511]: 6.b scrub ok 2024-09-20T22:19:10.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:09 smithi067 ceph-mon[33511]: 5.6 scrub starts 2024-09-20T22:19:10.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:09 smithi067 ceph-mon[33511]: 5.6 scrub ok 2024-09-20T22:19:10.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:09 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:19:10.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:09 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:19:10.094 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:10.098 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 17 16 37826 37810 138.976 89.125 0.00783129 0.00718903 2024-09-20T22:19:10.126 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:10.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:10.127 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:10.228 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:10.273 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:10.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:10.274 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:10.276 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:19:10.282 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:19:10.375 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:10.402 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:10.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:10.404 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:10.504 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:10.535 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:10.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:10.536 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:10.636 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:10.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:10.679 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:10.679 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:10.780 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:10.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:10 smithi060 ceph-mon[40290]: 6.a scrub starts 2024-09-20T22:19:10.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:10 smithi060 ceph-mon[40290]: 6.a scrub ok 2024-09-20T22:19:10.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:10 smithi060 ceph-mon[40290]: osdmap e464: 8 total, 8 up, 8 in 2024-09-20T22:19:10.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:10 smithi060 ceph-mon[40290]: 1.14 scrub starts 2024-09-20T22:19:10.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:10 smithi060 ceph-mon[40290]: 1.14 scrub ok 2024-09-20T22:19:10.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:10 smithi060 ceph-mon[40290]: from='client.26327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:19:10.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:10 smithi060 ceph-mon[40290]: pgmap v869: 178 pgs: 178 active+clean; 2.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 666 B/s rd, 113 MiB/s wr, 1.81k op/s 2024-09-20T22:19:10.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:10 smithi060 ceph-mon[40290]: 5.18 scrub starts 2024-09-20T22:19:10.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:10 smithi060 ceph-mon[40290]: 5.18 scrub ok 2024-09-20T22:19:10.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:10 smithi060 ceph-mon[29694]: 6.a scrub starts 2024-09-20T22:19:10.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:10 smithi060 ceph-mon[29694]: 6.a scrub ok 2024-09-20T22:19:10.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:10 smithi060 ceph-mon[29694]: osdmap e464: 8 total, 8 up, 8 in 2024-09-20T22:19:10.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:10 smithi060 ceph-mon[29694]: 1.14 scrub starts 2024-09-20T22:19:10.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:10 smithi060 ceph-mon[29694]: 1.14 scrub ok 2024-09-20T22:19:10.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:10 smithi060 ceph-mon[29694]: from='client.26327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:19:10.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:10 smithi060 ceph-mon[29694]: pgmap v869: 178 pgs: 178 active+clean; 2.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 666 B/s rd, 113 MiB/s wr, 1.81k op/s 2024-09-20T22:19:10.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:10 smithi060 ceph-mon[29694]: 5.18 scrub starts 2024-09-20T22:19:10.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:10 smithi060 ceph-mon[29694]: 5.18 scrub ok 2024-09-20T22:19:10.821 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:10.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:10.822 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:10.923 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:10.950 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:10.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:10.951 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:10 smithi067 ceph-mon[33511]: 6.a scrub starts 2024-09-20T22:19:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:10 smithi067 ceph-mon[33511]: 6.a scrub ok 2024-09-20T22:19:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:10 smithi067 ceph-mon[33511]: osdmap e464: 8 total, 8 up, 8 in 2024-09-20T22:19:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:10 smithi067 ceph-mon[33511]: 1.14 scrub starts 2024-09-20T22:19:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:10 smithi067 ceph-mon[33511]: 1.14 scrub ok 2024-09-20T22:19:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:10 smithi067 ceph-mon[33511]: from='client.26327 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:19:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:10 smithi067 ceph-mon[33511]: pgmap v869: 178 pgs: 178 active+clean; 2.0 GiB data, 11 GiB used, 704 GiB / 715 GiB avail; 666 B/s rd, 113 MiB/s wr, 1.81k op/s 2024-09-20T22:19:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:10 smithi067 ceph-mon[33511]: 5.18 scrub starts 2024-09-20T22:19:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:10 smithi067 ceph-mon[33511]: 5.18 scrub ok 2024-09-20T22:19:11.052 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:11.086 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:11.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:11.087 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:11.098 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 18 16 39417 39401 136.779 99.4375 0.00856427 0.00730608 2024-09-20T22:19:11.187 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:11.224 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:11.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:11.225 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:11.325 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:11.327 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":868,"stamp":"2024-09-20T22:19:09.673698+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2176260635,"num_objects":33397,"num_object_clones":0,"num_object_copies":100191,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33397,"num_whiteouts":0,"num_read":2084,"num_read_kb":1836,"num_write":33920,"num_write_kb":2124717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":5985,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38026,"ondisk_log_size":38026,"up":534,"acting":534,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":534,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":11763816,"kb_used_data":6183392,"kb_used_omap":15,"kb_used_meta":5580272,"kb_avail":738197400,"statfs":{"total":767960285184,"available":755914137600,"internally_reserved":0,"allocated":6331793408,"data_stored":6328158974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":15597,"internal_metadata":5714199315},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":954728448,"num_objects":14568,"num_object_clones":0,"num_object_copies":43704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14568,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14568,"num_write_kb":932352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.355246"},"pg_stats":[{"pgid":"1.1d","version":"75'128","reported_seq":378,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.632127+0000","last_change":"2024-09-20T22:17:57.129002+0000","last_active":"2024-09-20T22:19:06.632127+0000","last_peered":"2024-09-20T22:19:06.632127+0000","last_clean":"2024-09-20T22:19:06.632127+0000","last_became_active":"2024-09-20T22:17:44.720436+0000","last_became_peered":"2024-09-20T22:17:44.720436+0000","last_unstale":"2024-09-20T22:19:06.632127+0000","last_undegraded":"2024-09-20T22:19:06.632127+0000","last_fullsized":"2024-09-20T22:19:06.632127+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:57.128894+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.128894+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:12.360328+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":316,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.643608+0000","last_change":"2024-09-20T22:18:43.512180+0000","last_active":"2024-09-20T22:19:06.643608+0000","last_peered":"2024-09-20T22:19:06.643608+0000","last_clean":"2024-09-20T22:19:06.643608+0000","last_became_active":"2024-09-20T22:12:52.515921+0000","last_became_peered":"2024-09-20T22:12:52.515921+0000","last_unstale":"2024-09-20T22:19:06.643608+0000","last_undegraded":"2024-09-20T22:19:06.643608+0000","last_fullsized":"2024-09-20T22:19:06.643608+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:43.512066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:43.512066+0000","last_clean_scrub_stamp":"2024-09-20T22:18:43.512066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:07.524087+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"327'19","reported_seq":895,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.635946+0000","last_change":"2024-09-20T22:16:29.508726+0000","last_active":"2024-09-20T22:19:02.635946+0000","last_peered":"2024-09-20T22:19:02.635946+0000","last_clean":"2024-09-20T22:19:02.635946+0000","last_became_active":"2024-09-20T22:06:54.831411+0000","last_became_peered":"2024-09-20T22:06:54.831411+0000","last_unstale":"2024-09-20T22:19:02.635946+0000","last_undegraded":"2024-09-20T22:19:02.635946+0000","last_fullsized":"2024-09-20T22:19:02.635946+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"327'19","last_scrub_stamp":"2024-09-20T22:16:29.508607+0000","last_deep_scrub":"252'13","last_deep_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_clean_scrub_stamp":"2024-09-20T22:16:29.508607+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.323538+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":743,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.630940+0000","last_change":"2024-09-20T22:17:04.319541+0000","last_active":"2024-09-20T22:19:06.630940+0000","last_peered":"2024-09-20T22:19:06.630940+0000","last_clean":"2024-09-20T22:19:06.630940+0000","last_became_active":"2024-09-20T22:06:54.842721+0000","last_became_peered":"2024-09-20T22:06:54.842721+0000","last_unstale":"2024-09-20T22:19:06.630940+0000","last_undegraded":"2024-09-20T22:19:06.630940+0000","last_fullsized":"2024-09-20T22:19:06.630940+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:04.319429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:04.319429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":870,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.636255+0000","last_change":"2024-09-20T22:18:44.321354+0000","last_active":"2024-09-20T22:19:02.636255+0000","last_peered":"2024-09-20T22:19:02.636255+0000","last_clean":"2024-09-20T22:19:02.636255+0000","last_became_active":"2024-09-20T22:11:30.485735+0000","last_became_peered":"2024-09-20T22:11:30.485735+0000","last_unstale":"2024-09-20T22:19:02.636255+0000","last_undegraded":"2024-09-20T22:19:02.636255+0000","last_fullsized":"2024-09-20T22:19:02.636255+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.321275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.321275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:45.653900+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1c","version":"75'128","reported_seq":816,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.651137+0000","last_change":"2024-09-20T22:17:57.852321+0000","last_active":"2024-09-20T22:19:02.651137+0000","last_peered":"2024-09-20T22:19:02.651137+0000","last_clean":"2024-09-20T22:19:02.651137+0000","last_became_active":"2024-09-20T22:17:44.724322+0000","last_became_peered":"2024-09-20T22:17:44.724322+0000","last_unstale":"2024-09-20T22:19:02.651137+0000","last_undegraded":"2024-09-20T22:19:02.651137+0000","last_fullsized":"2024-09-20T22:19:02.651137+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:57.852213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.852213+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":859,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.627238+0000","last_change":"2024-09-20T22:18:01.667508+0000","last_active":"2024-09-20T22:19:06.627238+0000","last_peered":"2024-09-20T22:19:06.627238+0000","last_clean":"2024-09-20T22:19:06.627238+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:19:06.627238+0000","last_undegraded":"2024-09-20T22:19:06.627238+0000","last_fullsized":"2024-09-20T22:19:06.627238+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:18:01.667298+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:01.667298+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:25.121768+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":343,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.630761+0000","last_change":"2024-09-20T22:18:31.987311+0000","last_active":"2024-09-20T22:19:06.630761+0000","last_peered":"2024-09-20T22:19:06.630761+0000","last_clean":"2024-09-20T22:19:06.630761+0000","last_became_active":"2024-09-20T22:12:33.545100+0000","last_became_peered":"2024-09-20T22:12:33.545100+0000","last_unstale":"2024-09-20T22:19:06.630761+0000","last_undegraded":"2024-09-20T22:19:06.630761+0000","last_fullsized":"2024-09-20T22:19:06.630761+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:31.987191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:31.987191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:00.716345+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"406'34","reported_seq":463,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.645898+0000","last_change":"2024-09-20T22:17:55.941665+0000","last_active":"2024-09-20T22:19:02.645898+0000","last_peered":"2024-09-20T22:19:02.645898+0000","last_clean":"2024-09-20T22:19:02.645898+0000","last_became_active":"2024-09-20T22:12:52.517296+0000","last_became_peered":"2024-09-20T22:12:52.517296+0000","last_unstale":"2024-09-20T22:19:02.645898+0000","last_undegraded":"2024-09-20T22:19:02.645898+0000","last_fullsized":"2024-09-20T22:19:02.645898+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'34","last_scrub_stamp":"2024-09-20T22:17:55.940057+0000","last_deep_scrub":"406'34","last_deep_scrub_stamp":"2024-09-20T22:16:31.033354+0000","last_clean_scrub_stamp":"2024-09-20T22:17:55.940057+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:04.916246+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":854,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.635942+0000","last_change":"2024-09-20T22:16:36.461743+0000","last_active":"2024-09-20T22:19:02.635942+0000","last_peered":"2024-09-20T22:19:02.635942+0000","last_clean":"2024-09-20T22:19:02.635942+0000","last_became_active":"2024-09-20T22:06:54.831818+0000","last_became_peered":"2024-09-20T22:06:54.831818+0000","last_unstale":"2024-09-20T22:19:02.635942+0000","last_undegraded":"2024-09-20T22:19:02.635942+0000","last_fullsized":"2024-09-20T22:19:02.635942+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:36.461517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:36.461517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.323401+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":876,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.652663+0000","last_change":"2024-09-20T22:17:41.350439+0000","last_active":"2024-09-20T22:19:02.652663+0000","last_peered":"2024-09-20T22:19:02.652663+0000","last_clean":"2024-09-20T22:19:02.652663+0000","last_became_active":"2024-09-20T22:02:50.107955+0000","last_became_peered":"2024-09-20T22:02:50.107955+0000","last_unstale":"2024-09-20T22:19:02.652663+0000","last_undegraded":"2024-09-20T22:19:02.652663+0000","last_fullsized":"2024-09-20T22:19:02.652663+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:41.350325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:17:41.350325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:05.591013+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"328'15","reported_seq":750,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.639664+0000","last_change":"2024-09-20T22:18:35.404979+0000","last_active":"2024-09-20T22:19:06.639664+0000","last_peered":"2024-09-20T22:19:06.639664+0000","last_clean":"2024-09-20T22:19:06.639664+0000","last_became_active":"2024-09-20T22:17:03.663697+0000","last_became_peered":"2024-09-20T22:17:03.663697+0000","last_unstale":"2024-09-20T22:19:06.639664+0000","last_undegraded":"2024-09-20T22:19:06.639664+0000","last_fullsized":"2024-09-20T22:19:06.639664+0000","mapping_epoch":415,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":416,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:18:35.404892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:35.404892+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:39.528072+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":408,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.651964+0000","last_change":"2024-09-20T22:17:30.235268+0000","last_active":"2024-09-20T22:19:02.651964+0000","last_peered":"2024-09-20T22:19:02.651964+0000","last_clean":"2024-09-20T22:19:02.651964+0000","last_became_active":"2024-09-20T22:12:52.518549+0000","last_became_peered":"2024-09-20T22:12:52.518549+0000","last_unstale":"2024-09-20T22:19:02.651964+0000","last_undegraded":"2024-09-20T22:19:02.651964+0000","last_fullsized":"2024-09-20T22:19:02.651964+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:30.235059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:30.235059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:05.591037+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":286,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.630570+0000","last_change":"2024-09-20T22:17:03.665051+0000","last_active":"2024-09-20T22:19:06.630570+0000","last_peered":"2024-09-20T22:19:06.630570+0000","last_clean":"2024-09-20T22:19:06.630570+0000","last_became_active":"2024-09-20T22:17:03.664619+0000","last_became_peered":"2024-09-20T22:17:03.664619+0000","last_unstale":"2024-09-20T22:19:06.630570+0000","last_undegraded":"2024-09-20T22:19:06.630570+0000","last_fullsized":"2024-09-20T22:19:06.630570+0000","mapping_epoch":415,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":416,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:37.934760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:37.934760+0000","last_clean_scrub_stamp":"2024-09-20T22:16:37.934760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,2],"acting":[1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1e","version":"75'128","reported_seq":377,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.632081+0000","last_change":"2024-09-20T22:17:56.699899+0000","last_active":"2024-09-20T22:19:06.632081+0000","last_peered":"2024-09-20T22:19:06.632081+0000","last_clean":"2024-09-20T22:19:06.632081+0000","last_became_active":"2024-09-20T22:17:44.708330+0000","last_became_peered":"2024-09-20T22:17:44.708330+0000","last_unstale":"2024-09-20T22:19:06.632081+0000","last_undegraded":"2024-09-20T22:19:06.632081+0000","last_fullsized":"2024-09-20T22:19:06.632081+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.699777+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:17:40.367940+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.699777+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:11.112545+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":881,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.652091+0000","last_change":"2024-09-20T22:18:03.470422+0000","last_active":"2024-09-20T22:19:02.652091+0000","last_peered":"2024-09-20T22:19:02.652091+0000","last_clean":"2024-09-20T22:19:02.652091+0000","last_became_active":"2024-09-20T22:06:55.815583+0000","last_became_peered":"2024-09-20T22:06:55.815583+0000","last_unstale":"2024-09-20T22:19:02.652091+0000","last_undegraded":"2024-09-20T22:19:02.652091+0000","last_fullsized":"2024-09-20T22:19:02.652091+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:03.470297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:30.215263+0000","last_clean_scrub_stamp":"2024-09-20T22:18:03.470297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:14.814301+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,1],"acting":[5,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"328'7","reported_seq":891,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.636436+0000","last_change":"2024-09-20T22:17:30.369631+0000","last_active":"2024-09-20T22:19:02.636436+0000","last_peered":"2024-09-20T22:19:02.636436+0000","last_clean":"2024-09-20T22:19:02.636436+0000","last_became_active":"2024-09-20T22:08:22.144364+0000","last_became_peered":"2024-09-20T22:08:22.144364+0000","last_unstale":"2024-09-20T22:19:02.636436+0000","last_undegraded":"2024-09-20T22:19:02.636436+0000","last_fullsized":"2024-09-20T22:19:02.636436+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'7","last_scrub_stamp":"2024-09-20T22:17:30.369518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:30.369518+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.323705+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":10,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,5],"acting":[4,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":862,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.649935+0000","last_change":"2024-09-20T22:17:03.783046+0000","last_active":"2024-09-20T22:19:02.649935+0000","last_peered":"2024-09-20T22:19:02.649935+0000","last_clean":"2024-09-20T22:19:02.649935+0000","last_became_active":"2024-09-20T22:06:54.829822+0000","last_became_peered":"2024-09-20T22:06:54.829822+0000","last_unstale":"2024-09-20T22:19:02.649935+0000","last_undegraded":"2024-09-20T22:19:02.649935+0000","last_fullsized":"2024-09-20T22:19:02.649935+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:03.782934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:03.782934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":396,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.651689+0000","last_change":"2024-09-20T22:17:33.243978+0000","last_active":"2024-09-20T22:19:02.651689+0000","last_peered":"2024-09-20T22:19:02.651689+0000","last_clean":"2024-09-20T22:19:02.651689+0000","last_became_active":"2024-09-20T22:11:30.487137+0000","last_became_peered":"2024-09-20T22:11:30.487137+0000","last_unstale":"2024-09-20T22:19:02.651689+0000","last_undegraded":"2024-09-20T22:19:02.651689+0000","last_fullsized":"2024-09-20T22:19:02.651689+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:33.243552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:17:33.243552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:05.591061+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.19","version":"75'128","reported_seq":643,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.639627+0000","last_change":"2024-09-20T22:17:56.103865+0000","last_active":"2024-09-20T22:19:06.639627+0000","last_peered":"2024-09-20T22:19:06.639627+0000","last_clean":"2024-09-20T22:19:06.639627+0000","last_became_active":"2024-09-20T22:17:44.716139+0000","last_became_peered":"2024-09-20T22:17:44.716139+0000","last_unstale":"2024-09-20T22:19:06.639627+0000","last_undegraded":"2024-09-20T22:19:06.639627+0000","last_fullsized":"2024-09-20T22:19:06.639627+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.103744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.103744+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:23.437505+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":851,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.636853+0000","last_change":"2024-09-20T22:16:34.472967+0000","last_active":"2024-09-20T22:19:02.636853+0000","last_peered":"2024-09-20T22:19:02.636853+0000","last_clean":"2024-09-20T22:19:02.636853+0000","last_became_active":"2024-09-20T22:02:56.346991+0000","last_became_peered":"2024-09-20T22:02:56.346991+0000","last_unstale":"2024-09-20T22:19:02.636853+0000","last_undegraded":"2024-09-20T22:19:02.636853+0000","last_fullsized":"2024-09-20T22:19:02.636853+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:34.472700+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:47.667180+0000","last_clean_scrub_stamp":"2024-09-20T22:16:34.472700+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.323602+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":129,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.636643+0000","last_change":"2024-09-20T22:17:07.404010+0000","last_active":"2024-09-20T22:19:02.636643+0000","last_peered":"2024-09-20T22:19:02.636643+0000","last_clean":"2024-09-20T22:19:02.636643+0000","last_became_active":"2024-09-20T22:17:03.652857+0000","last_became_peered":"2024-09-20T22:17:03.652857+0000","last_unstale":"2024-09-20T22:19:02.636643+0000","last_undegraded":"2024-09-20T22:19:02.636643+0000","last_fullsized":"2024-09-20T22:19:02.636643+0000","mapping_epoch":415,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":416,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:07.403912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:17:07.403912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.323779+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"328'34","reported_seq":460,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.647018+0000","last_change":"2024-09-20T22:17:41.808122+0000","last_active":"2024-09-20T22:19:02.647018+0000","last_peered":"2024-09-20T22:19:02.647018+0000","last_clean":"2024-09-20T22:19:02.647018+0000","last_became_active":"2024-09-20T22:11:12.507311+0000","last_became_peered":"2024-09-20T22:11:12.507311+0000","last_unstale":"2024-09-20T22:19:02.647018+0000","last_undegraded":"2024-09-20T22:19:02.647018+0000","last_fullsized":"2024-09-20T22:19:02.647018+0000","mapping_epoch":275,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":276,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'34","last_scrub_stamp":"2024-09-20T22:17:41.807990+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:17:41.807990+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":712,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.632325+0000","last_change":"2024-09-20T22:18:44.295132+0000","last_active":"2024-09-20T22:19:06.632325+0000","last_peered":"2024-09-20T22:19:06.632325+0000","last_clean":"2024-09-20T22:19:06.632325+0000","last_became_active":"2024-09-20T22:12:52.520985+0000","last_became_peered":"2024-09-20T22:12:52.520985+0000","last_unstale":"2024-09-20T22:19:06.632325+0000","last_undegraded":"2024-09-20T22:19:06.632325+0000","last_fullsized":"2024-09-20T22:19:06.632325+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.295028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:44.295028+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.295028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:13.854261+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.18","version":"75'128","reported_seq":481,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.651491+0000","last_change":"2024-09-20T22:17:56.357255+0000","last_active":"2024-09-20T22:19:02.651491+0000","last_peered":"2024-09-20T22:19:02.651491+0000","last_clean":"2024-09-20T22:19:02.651491+0000","last_became_active":"2024-09-20T22:17:44.717551+0000","last_became_peered":"2024-09-20T22:17:44.717551+0000","last_unstale":"2024-09-20T22:19:02.651491+0000","last_undegraded":"2024-09-20T22:19:02.651491+0000","last_fullsized":"2024-09-20T22:19:02.651491+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.357167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.357167+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.867832+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":812,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.630988+0000","last_change":"2024-09-20T22:18:31.015563+0000","last_active":"2024-09-20T22:19:06.630988+0000","last_peered":"2024-09-20T22:19:06.630988+0000","last_clean":"2024-09-20T22:19:06.630988+0000","last_became_active":"2024-09-20T22:12:21.521433+0000","last_became_peered":"2024-09-20T22:12:21.521433+0000","last_unstale":"2024-09-20T22:19:06.630988+0000","last_undegraded":"2024-09-20T22:19:06.630988+0000","last_fullsized":"2024-09-20T22:19:06.630988+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:31.015449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:31.015449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:51.877680+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":810,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.652348+0000","last_change":"2024-09-20T22:18:44.456530+0000","last_active":"2024-09-20T22:19:02.652348+0000","last_peered":"2024-09-20T22:19:02.652348+0000","last_clean":"2024-09-20T22:19:02.652348+0000","last_became_active":"2024-09-20T22:06:55.815529+0000","last_became_peered":"2024-09-20T22:06:55.815529+0000","last_unstale":"2024-09-20T22:19:02.652348+0000","last_undegraded":"2024-09-20T22:19:02.652348+0000","last_fullsized":"2024-09-20T22:19:02.652348+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.456415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:44.456415+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.456415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:51.490609+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,2],"acting":[5,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"328'23","reported_seq":777,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.632575+0000","last_change":"2024-09-20T22:17:38.263995+0000","last_active":"2024-09-20T22:19:06.632575+0000","last_peered":"2024-09-20T22:19:06.632575+0000","last_clean":"2024-09-20T22:19:06.632575+0000","last_became_active":"2024-09-20T22:12:21.517147+0000","last_became_peered":"2024-09-20T22:12:21.517147+0000","last_unstale":"2024-09-20T22:19:06.632575+0000","last_undegraded":"2024-09-20T22:19:06.632575+0000","last_fullsized":"2024-09-20T22:19:06.632575+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'23","last_scrub_stamp":"2024-09-20T22:17:38.263865+0000","last_deep_scrub":"250'17","last_deep_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_clean_scrub_stamp":"2024-09-20T22:17:38.263865+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":853,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.636392+0000","last_change":"2024-09-20T22:17:09.392111+0000","last_active":"2024-09-20T22:19:02.636392+0000","last_peered":"2024-09-20T22:19:02.636392+0000","last_clean":"2024-09-20T22:19:02.636392+0000","last_became_active":"2024-09-20T22:02:54.292865+0000","last_became_peered":"2024-09-20T22:02:54.292865+0000","last_unstale":"2024-09-20T22:19:02.636392+0000","last_undegraded":"2024-09-20T22:19:02.636392+0000","last_fullsized":"2024-09-20T22:19:02.636392+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:09.391995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:09.391995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.323730+0000","scrub_duration":39,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1b","version":"75'128","reported_seq":785,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.651730+0000","last_change":"2024-09-20T22:17:58.426160+0000","last_active":"2024-09-20T22:19:02.651730+0000","last_peered":"2024-09-20T22:19:02.651730+0000","last_clean":"2024-09-20T22:19:02.651730+0000","last_became_active":"2024-09-20T22:17:44.725866+0000","last_became_peered":"2024-09-20T22:17:44.725866+0000","last_unstale":"2024-09-20T22:19:02.651730+0000","last_undegraded":"2024-09-20T22:19:02.651730+0000","last_fullsized":"2024-09-20T22:19:02.651730+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:58.426044+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:17:58.426044+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:20.577220+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":851,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.650944+0000","last_change":"2024-09-20T22:18:09.700077+0000","last_active":"2024-09-20T22:19:02.650944+0000","last_peered":"2024-09-20T22:19:02.650944+0000","last_clean":"2024-09-20T22:19:02.650944+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:19:02.650944+0000","last_undegraded":"2024-09-20T22:19:02.650944+0000","last_fullsized":"2024-09-20T22:19:02.650944+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:18:09.699725+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:18:09.699725+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:19.270482+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"289'2","reported_seq":703,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.630987+0000","last_change":"2024-09-20T22:17:07.356345+0000","last_active":"2024-09-20T22:19:06.630987+0000","last_peered":"2024-09-20T22:19:06.630987+0000","last_clean":"2024-09-20T22:19:06.630987+0000","last_became_active":"2024-09-20T22:06:54.853666+0000","last_became_peered":"2024-09-20T22:06:54.853666+0000","last_unstale":"2024-09-20T22:19:06.630987+0000","last_undegraded":"2024-09-20T22:19:06.630987+0000","last_fullsized":"2024-09-20T22:19:06.630987+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-20T22:17:07.356220+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_clean_scrub_stamp":"2024-09-20T22:17:07.356220+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"328'20","reported_seq":343,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.641028+0000","last_change":"2024-09-20T22:18:10.346398+0000","last_active":"2024-09-20T22:19:06.641028+0000","last_peered":"2024-09-20T22:19:06.641028+0000","last_clean":"2024-09-20T22:19:06.641028+0000","last_became_active":"2024-09-20T22:12:52.526803+0000","last_became_peered":"2024-09-20T22:12:52.526803+0000","last_unstale":"2024-09-20T22:19:06.641028+0000","last_undegraded":"2024-09-20T22:19:06.641028+0000","last_fullsized":"2024-09-20T22:19:06.641028+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:18:10.346320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:10.346320+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:24.774405+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"291'11","reported_seq":1107,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.631444+0000","last_change":"2024-09-20T22:18:33.978246+0000","last_active":"2024-09-20T22:19:06.631444+0000","last_peered":"2024-09-20T22:19:06.631444+0000","last_clean":"2024-09-20T22:19:06.631444+0000","last_became_active":"2024-09-20T22:11:40.601048+0000","last_became_peered":"2024-09-20T22:11:40.601048+0000","last_unstale":"2024-09-20T22:19:06.631444+0000","last_undegraded":"2024-09-20T22:19:06.631444+0000","last_fullsized":"2024-09-20T22:19:06.631444+0000","mapping_epoch":290,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":291,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'11","last_scrub_stamp":"2024-09-20T22:18:33.978168+0000","last_deep_scrub":"291'11","last_deep_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_clean_scrub_stamp":"2024-09-20T22:18:33.978168+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:51.119032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.0","version":"463'2138","reported_seq":2162,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:08.548707+0000","last_change":"2024-09-20T22:18:46.630676+0000","last_active":"2024-09-20T22:19:08.548707+0000","last_peered":"2024-09-20T22:19:08.548707+0000","last_clean":"2024-09-20T22:19:08.548707+0000","last_became_active":"2024-09-20T22:18:46.630384+0000","last_became_peered":"2024-09-20T22:18:46.630384+0000","last_unstale":"2024-09-20T22:19:08.548707+0000","last_undegraded":"2024-09-20T22:19:08.548707+0000","last_fullsized":"2024-09-20T22:19:08.548707+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":2138,"log_dups_size":0,"ondisk_log_size":2138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:02.162927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":140115968,"num_objects":2138,"num_object_clones":0,"num_object_copies":6414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2138,"num_write_kb":136832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":876,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.652173+0000","last_change":"2024-09-20T22:17:40.331506+0000","last_active":"2024-09-20T22:19:02.652173+0000","last_peered":"2024-09-20T22:19:02.652173+0000","last_clean":"2024-09-20T22:19:02.652173+0000","last_became_active":"2024-09-20T22:17:40.139526+0000","last_became_peered":"2024-09-20T22:17:40.139526+0000","last_unstale":"2024-09-20T22:19:02.652173+0000","last_undegraded":"2024-09-20T22:19:02.652173+0000","last_fullsized":"2024-09-20T22:19:02.652173+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:40.331395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:40.331395+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:05.591088+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"328'27","reported_seq":653,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.645687+0000","last_change":"2024-09-20T22:17:34.818518+0000","last_active":"2024-09-20T22:19:02.645687+0000","last_peered":"2024-09-20T22:19:02.645687+0000","last_clean":"2024-09-20T22:19:02.645687+0000","last_became_active":"2024-09-20T22:08:22.145408+0000","last_became_peered":"2024-09-20T22:08:22.145408+0000","last_unstale":"2024-09-20T22:19:02.645687+0000","last_undegraded":"2024-09-20T22:19:02.645687+0000","last_fullsized":"2024-09-20T22:19:02.645687+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:17:34.818375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:34.818375+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":880,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.631193+0000","last_change":"2024-09-20T22:17:57.726107+0000","last_active":"2024-09-20T22:19:06.631193+0000","last_peered":"2024-09-20T22:19:06.631193+0000","last_clean":"2024-09-20T22:19:06.631193+0000","last_became_active":"2024-09-20T22:06:54.848443+0000","last_became_peered":"2024-09-20T22:06:54.848443+0000","last_unstale":"2024-09-20T22:19:06.631193+0000","last_undegraded":"2024-09-20T22:19:06.631193+0000","last_fullsized":"2024-09-20T22:19:06.631193+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:57.726043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:15:28.020341+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.726043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:07.260365+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"326'13","reported_seq":1127,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.636981+0000","last_change":"2024-09-20T22:18:40.407944+0000","last_active":"2024-09-20T22:19:02.636981+0000","last_peered":"2024-09-20T22:19:02.636981+0000","last_clean":"2024-09-20T22:19:02.636981+0000","last_became_active":"2024-09-20T22:12:52.517588+0000","last_became_peered":"2024-09-20T22:12:52.517588+0000","last_unstale":"2024-09-20T22:19:02.636981+0000","last_undegraded":"2024-09-20T22:19:02.636981+0000","last_fullsized":"2024-09-20T22:19:02.636981+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'13","last_scrub_stamp":"2024-09-20T22:18:40.407802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:40.407802+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:05.535042+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":408,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.652356+0000","last_change":"2024-09-20T22:18:04.453544+0000","last_active":"2024-09-20T22:19:02.652356+0000","last_peered":"2024-09-20T22:19:02.652356+0000","last_clean":"2024-09-20T22:19:02.652356+0000","last_became_active":"2024-09-20T22:12:52.517826+0000","last_became_peered":"2024-09-20T22:12:52.517826+0000","last_unstale":"2024-09-20T22:19:02.652356+0000","last_undegraded":"2024-09-20T22:19:02.652356+0000","last_fullsized":"2024-09-20T22:19:02.652356+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:04.453445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:04.453445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:10.865173+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"462'1957","reported_seq":1979,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:05.888431+0000","last_change":"2024-09-20T22:18:46.634853+0000","last_active":"2024-09-20T22:19:05.888431+0000","last_peered":"2024-09-20T22:19:05.888431+0000","last_clean":"2024-09-20T22:19:05.888431+0000","last_became_active":"2024-09-20T22:18:46.634515+0000","last_became_peered":"2024-09-20T22:18:46.634515+0000","last_unstale":"2024-09-20T22:19:05.888431+0000","last_undegraded":"2024-09-20T22:19:05.888431+0000","last_fullsized":"2024-09-20T22:19:05.888431+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":1957,"log_dups_size":0,"ondisk_log_size":1957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:10.586261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":128253952,"num_objects":1957,"num_object_clones":0,"num_object_copies":5871,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1957,"num_write_kb":125248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":881,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.636475+0000","last_change":"2024-09-20T22:17:42.412312+0000","last_active":"2024-09-20T22:19:02.636475+0000","last_peered":"2024-09-20T22:19:02.636475+0000","last_clean":"2024-09-20T22:19:02.636475+0000","last_became_active":"2024-09-20T22:17:40.147065+0000","last_became_peered":"2024-09-20T22:17:40.147065+0000","last_unstale":"2024-09-20T22:19:02.636475+0000","last_undegraded":"2024-09-20T22:19:02.636475+0000","last_fullsized":"2024-09-20T22:19:02.636475+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:42.412207+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:17:42.412207+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.323876+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"328'30","reported_seq":874,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.649596+0000","last_change":"2024-09-20T22:18:43.923440+0000","last_active":"2024-09-20T22:19:02.649596+0000","last_peered":"2024-09-20T22:19:02.649596+0000","last_clean":"2024-09-20T22:19:02.649596+0000","last_became_active":"2024-09-20T22:12:52.526867+0000","last_became_peered":"2024-09-20T22:12:52.526867+0000","last_unstale":"2024-09-20T22:19:02.649596+0000","last_undegraded":"2024-09-20T22:19:02.649596+0000","last_fullsized":"2024-09-20T22:19:02.649596+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'30","last_scrub_stamp":"2024-09-20T22:18:43.923335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:43.923335+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:10.834418+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":355,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.650805+0000","last_change":"2024-09-20T22:18:45.931576+0000","last_active":"2024-09-20T22:19:02.650805+0000","last_peered":"2024-09-20T22:19:02.650805+0000","last_clean":"2024-09-20T22:19:02.650805+0000","last_became_active":"2024-09-20T22:12:52.528947+0000","last_became_peered":"2024-09-20T22:12:52.528947+0000","last_unstale":"2024-09-20T22:19:02.650805+0000","last_undegraded":"2024-09-20T22:19:02.650805+0000","last_fullsized":"2024-09-20T22:19:02.650805+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.931495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.931495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:53.803123+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":874,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.635957+0000","last_change":"2024-09-20T22:18:10.359530+0000","last_active":"2024-09-20T22:19:02.635957+0000","last_peered":"2024-09-20T22:19:02.635957+0000","last_clean":"2024-09-20T22:19:02.635957+0000","last_became_active":"2024-09-20T22:10:48.919949+0000","last_became_peered":"2024-09-20T22:10:48.919949+0000","last_unstale":"2024-09-20T22:19:02.635957+0000","last_undegraded":"2024-09-20T22:19:02.635957+0000","last_fullsized":"2024-09-20T22:19:02.635957+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:10.359427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:10.359427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:30.515155+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":858,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.636394+0000","last_change":"2024-09-20T22:18:04.327361+0000","last_active":"2024-09-20T22:19:02.636394+0000","last_peered":"2024-09-20T22:19:02.636394+0000","last_clean":"2024-09-20T22:19:02.636394+0000","last_became_active":"2024-09-20T22:02:56.302779+0000","last_became_peered":"2024-09-20T22:02:56.302779+0000","last_unstale":"2024-09-20T22:19:02.636394+0000","last_undegraded":"2024-09-20T22:19:02.636394+0000","last_fullsized":"2024-09-20T22:19:02.636394+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:04.327270+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:04.327270+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:33.395263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"462'1900","reported_seq":1922,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:05.946628+0000","last_change":"2024-09-20T22:18:46.631821+0000","last_active":"2024-09-20T22:19:05.946628+0000","last_peered":"2024-09-20T22:19:05.946628+0000","last_clean":"2024-09-20T22:19:05.946628+0000","last_became_active":"2024-09-20T22:18:46.631379+0000","last_became_peered":"2024-09-20T22:18:46.631379+0000","last_unstale":"2024-09-20T22:19:05.946628+0000","last_undegraded":"2024-09-20T22:19:05.946628+0000","last_fullsized":"2024-09-20T22:19:05.946628+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":1900,"log_dups_size":0,"ondisk_log_size":1900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:12.197092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":124518400,"num_objects":1900,"num_object_clones":0,"num_object_copies":5700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1900,"num_write_kb":121600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,7],"acting":[4,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":912,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.626957+0000","last_change":"2024-09-20T22:17:29.719066+0000","last_active":"2024-09-20T22:19:06.626957+0000","last_peered":"2024-09-20T22:19:06.626957+0000","last_clean":"2024-09-20T22:19:06.626957+0000","last_became_active":"2024-09-20T22:15:45.642671+0000","last_became_peered":"2024-09-20T22:15:45.642671+0000","last_unstale":"2024-09-20T22:19:06.626957+0000","last_undegraded":"2024-09-20T22:19:06.626957+0000","last_fullsized":"2024-09-20T22:19:06.626957+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:29.718808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:29.718808+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:07.544976+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"328'28","reported_seq":900,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.636211+0000","last_change":"2024-09-20T22:16:18.404188+0000","last_active":"2024-09-20T22:19:02.636211+0000","last_peered":"2024-09-20T22:19:02.636211+0000","last_clean":"2024-09-20T22:19:02.636211+0000","last_became_active":"2024-09-20T22:02:52.273136+0000","last_became_peered":"2024-09-20T22:02:52.273136+0000","last_unstale":"2024-09-20T22:19:02.636211+0000","last_undegraded":"2024-09-20T22:19:02.636211+0000","last_fullsized":"2024-09-20T22:19:02.636211+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:16:18.403972+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:18.403972+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.323506+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":42,"num_read_kb":35,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":319,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.641228+0000","last_change":"2024-09-20T22:16:35.952285+0000","last_active":"2024-09-20T22:19:06.641228+0000","last_peered":"2024-09-20T22:19:06.641228+0000","last_clean":"2024-09-20T22:19:06.641228+0000","last_became_active":"2024-09-20T22:12:52.528068+0000","last_became_peered":"2024-09-20T22:12:52.528068+0000","last_unstale":"2024-09-20T22:19:06.641228+0000","last_undegraded":"2024-09-20T22:19:06.641228+0000","last_fullsized":"2024-09-20T22:19:06.641228+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:16:35.952049+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:35.952049+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.753332+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":92,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"315'11","reported_seq":640,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.629908+0000","last_change":"2024-09-20T22:18:39.290989+0000","last_active":"2024-09-20T22:19:06.629908+0000","last_peered":"2024-09-20T22:19:06.629908+0000","last_clean":"2024-09-20T22:19:06.629908+0000","last_became_active":"2024-09-20T22:12:33.545942+0000","last_became_peered":"2024-09-20T22:12:33.545942+0000","last_unstale":"2024-09-20T22:19:06.629908+0000","last_undegraded":"2024-09-20T22:19:06.629908+0000","last_fullsized":"2024-09-20T22:19:06.629908+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:18:39.290911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:39.290911+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:08.799124+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":850,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.649828+0000","last_change":"2024-09-20T22:18:11.668610+0000","last_active":"2024-09-20T22:19:02.649828+0000","last_peered":"2024-09-20T22:19:02.649828+0000","last_clean":"2024-09-20T22:19:02.649828+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:19:02.649828+0000","last_undegraded":"2024-09-20T22:19:02.649828+0000","last_fullsized":"2024-09-20T22:19:02.649828+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:11.668512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:11.668512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:13.017084+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.d","version":"462'1969","reported_seq":1991,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.503764+0000","last_change":"2024-09-20T22:18:46.634195+0000","last_active":"2024-09-20T22:19:06.503764+0000","last_peered":"2024-09-20T22:19:06.503764+0000","last_clean":"2024-09-20T22:19:06.503764+0000","last_became_active":"2024-09-20T22:18:46.633775+0000","last_became_peered":"2024-09-20T22:18:46.633775+0000","last_unstale":"2024-09-20T22:19:06.503764+0000","last_undegraded":"2024-09-20T22:19:06.503764+0000","last_fullsized":"2024-09-20T22:19:06.503764+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":1969,"log_dups_size":0,"ondisk_log_size":1969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:53.088874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":129040384,"num_objects":1969,"num_object_clones":0,"num_object_copies":5907,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1969,"num_write_kb":126016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,6],"acting":[2,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":921,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.630897+0000","last_change":"2024-09-20T22:17:41.753132+0000","last_active":"2024-09-20T22:19:06.630897+0000","last_peered":"2024-09-20T22:19:06.630897+0000","last_clean":"2024-09-20T22:19:06.630897+0000","last_became_active":"2024-09-20T22:17:40.142028+0000","last_became_peered":"2024-09-20T22:17:40.142028+0000","last_unstale":"2024-09-20T22:19:06.630897+0000","last_undegraded":"2024-09-20T22:19:06.630897+0000","last_fullsized":"2024-09-20T22:19:06.630897+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:41.752931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:41.752931+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"328'20","reported_seq":778,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:08.320304+0000","last_change":"2024-09-20T22:19:08.320330+0000","last_active":"2024-09-20T22:19:08.320304+0000","last_peered":"2024-09-20T22:19:08.320304+0000","last_clean":"2024-09-20T22:19:08.320304+0000","last_became_active":"2024-09-20T22:11:30.486532+0000","last_became_peered":"2024-09-20T22:11:30.486532+0000","last_unstale":"2024-09-20T22:19:08.320304+0000","last_undegraded":"2024-09-20T22:19:08.320304+0000","last_fullsized":"2024-09-20T22:19:08.320304+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:19:08.320252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:08.320252+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:29.096390+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":877,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.652721+0000","last_change":"2024-09-20T22:17:09.988639+0000","last_active":"2024-09-20T22:19:02.652721+0000","last_peered":"2024-09-20T22:19:02.652721+0000","last_clean":"2024-09-20T22:19:02.652721+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:19:02.652721+0000","last_undegraded":"2024-09-20T22:19:02.652721+0000","last_fullsized":"2024-09-20T22:19:02.652721+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:09.988552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:17:09.988552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:05.590905+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"363'13","reported_seq":385,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.641146+0000","last_change":"2024-09-20T22:16:25.963851+0000","last_active":"2024-09-20T22:19:06.641146+0000","last_peered":"2024-09-20T22:19:06.641146+0000","last_clean":"2024-09-20T22:19:06.641146+0000","last_became_active":"2024-09-20T22:12:57.884415+0000","last_became_peered":"2024-09-20T22:12:57.884415+0000","last_unstale":"2024-09-20T22:19:06.641146+0000","last_undegraded":"2024-09-20T22:19:06.641146+0000","last_fullsized":"2024-09-20T22:19:06.641146+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"363'13","last_scrub_stamp":"2024-09-20T22:16:25.963742+0000","last_deep_scrub":"363'13","last_deep_scrub_stamp":"2024-09-20T22:14:52.612615+0000","last_clean_scrub_stamp":"2024-09-20T22:16:25.963742+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":308,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.639427+0000","last_change":"2024-09-20T22:16:31.044840+0000","last_active":"2024-09-20T22:19:06.639427+0000","last_peered":"2024-09-20T22:19:06.639427+0000","last_clean":"2024-09-20T22:19:06.639427+0000","last_became_active":"2024-09-20T22:12:52.518529+0000","last_became_peered":"2024-09-20T22:12:52.518529+0000","last_unstale":"2024-09-20T22:19:06.639427+0000","last_undegraded":"2024-09-20T22:19:06.639427+0000","last_fullsized":"2024-09-20T22:19:06.639427+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:31.044564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:31.044564+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.044564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.753414+0000","scrub_duration":24,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"463'2207","reported_seq":2231,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:08.546222+0000","last_change":"2024-09-20T22:18:46.629561+0000","last_active":"2024-09-20T22:19:08.546222+0000","last_peered":"2024-09-20T22:19:08.546222+0000","last_clean":"2024-09-20T22:19:08.546222+0000","last_became_active":"2024-09-20T22:18:46.629258+0000","last_became_peered":"2024-09-20T22:18:46.629258+0000","last_unstale":"2024-09-20T22:19:08.546222+0000","last_undegraded":"2024-09-20T22:19:08.546222+0000","last_fullsized":"2024-09-20T22:19:08.546222+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":2207,"log_dups_size":0,"ondisk_log_size":2207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:48.886077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":144637952,"num_objects":2207,"num_object_clones":0,"num_object_copies":6621,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2207,"num_write_kb":141248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,4],"acting":[1,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":922,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.630555+0000","last_change":"2024-09-20T22:17:40.744592+0000","last_active":"2024-09-20T22:19:06.630555+0000","last_peered":"2024-09-20T22:19:06.630555+0000","last_clean":"2024-09-20T22:19:06.630555+0000","last_became_active":"2024-09-20T22:17:40.139979+0000","last_became_peered":"2024-09-20T22:17:40.139979+0000","last_unstale":"2024-09-20T22:19:06.630555+0000","last_undegraded":"2024-09-20T22:19:06.630555+0000","last_fullsized":"2024-09-20T22:19:06.630555+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:40.744482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:40.744482+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"392'37","reported_seq":357,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.641431+0000","last_change":"2024-09-20T22:17:38.045665+0000","last_active":"2024-09-20T22:19:06.641431+0000","last_peered":"2024-09-20T22:19:06.641431+0000","last_clean":"2024-09-20T22:19:06.641431+0000","last_became_active":"2024-09-20T22:12:52.525538+0000","last_became_peered":"2024-09-20T22:12:52.525538+0000","last_unstale":"2024-09-20T22:19:06.641431+0000","last_undegraded":"2024-09-20T22:19:06.641431+0000","last_fullsized":"2024-09-20T22:19:06.641431+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"392'37","last_scrub_stamp":"2024-09-20T22:17:38.043643+0000","last_deep_scrub":"392'37","last_deep_scrub_stamp":"2024-09-20T22:15:56.127540+0000","last_clean_scrub_stamp":"2024-09-20T22:17:38.043643+0000","objects_scrubbed":8,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":714,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.632424+0000","last_change":"2024-09-20T22:18:43.293871+0000","last_active":"2024-09-20T22:19:06.632424+0000","last_peered":"2024-09-20T22:19:06.632424+0000","last_clean":"2024-09-20T22:19:06.632424+0000","last_became_active":"2024-09-20T22:11:30.486428+0000","last_became_peered":"2024-09-20T22:11:30.486428+0000","last_unstale":"2024-09-20T22:19:06.632424+0000","last_undegraded":"2024-09-20T22:19:06.632424+0000","last_fullsized":"2024-09-20T22:19:06.632424+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:43.293797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:43.293797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:49.167920+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":891,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.632774+0000","last_change":"2024-09-20T22:17:37.755759+0000","last_active":"2024-09-20T22:19:06.632774+0000","last_peered":"2024-09-20T22:19:06.632774+0000","last_clean":"2024-09-20T22:19:06.632774+0000","last_became_active":"2024-09-20T22:12:52.519826+0000","last_became_peered":"2024-09-20T22:12:52.519826+0000","last_unstale":"2024-09-20T22:19:06.632774+0000","last_undegraded":"2024-09-20T22:19:06.632774+0000","last_fullsized":"2024-09-20T22:19:06.632774+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:37.755521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:17:37.755521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":700,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.630843+0000","last_change":"2024-09-20T22:17:36.301819+0000","last_active":"2024-09-20T22:19:06.630843+0000","last_peered":"2024-09-20T22:19:06.630843+0000","last_clean":"2024-09-20T22:19:06.630843+0000","last_became_active":"2024-09-20T22:06:54.820063+0000","last_became_peered":"2024-09-20T22:06:54.820063+0000","last_unstale":"2024-09-20T22:19:06.630843+0000","last_undegraded":"2024-09-20T22:19:06.630843+0000","last_fullsized":"2024-09-20T22:19:06.630843+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:36.301648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:36.301648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.b","version":"463'2037","reported_seq":2061,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.824575+0000","last_change":"2024-09-20T22:18:46.628687+0000","last_active":"2024-09-20T22:19:06.824575+0000","last_peered":"2024-09-20T22:19:06.824575+0000","last_clean":"2024-09-20T22:19:06.824575+0000","last_became_active":"2024-09-20T22:18:46.628392+0000","last_became_peered":"2024-09-20T22:18:46.628392+0000","last_unstale":"2024-09-20T22:19:06.824575+0000","last_undegraded":"2024-09-20T22:19:06.824575+0000","last_fullsized":"2024-09-20T22:19:06.824575+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":2037,"log_dups_size":0,"ondisk_log_size":2037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:15.336078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":133496832,"num_objects":2037,"num_object_clones":0,"num_object_copies":6111,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2037,"num_write_kb":130368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,0],"acting":[3,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":894,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.651515+0000","last_change":"2024-09-20T22:18:41.457692+0000","last_active":"2024-09-20T22:19:02.651515+0000","last_peered":"2024-09-20T22:19:02.651515+0000","last_clean":"2024-09-20T22:19:02.651515+0000","last_became_active":"2024-09-20T22:15:45.627135+0000","last_became_peered":"2024-09-20T22:15:45.627135+0000","last_unstale":"2024-09-20T22:19:02.651515+0000","last_undegraded":"2024-09-20T22:19:02.651515+0000","last_fullsized":"2024-09-20T22:19:02.651515+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:18:41.457600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:18:41.457600+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:59.531766+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"328'52","reported_seq":938,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.632559+0000","last_change":"2024-09-20T22:17:08.395721+0000","last_active":"2024-09-20T22:19:06.632559+0000","last_peered":"2024-09-20T22:19:06.632559+0000","last_clean":"2024-09-20T22:19:06.632559+0000","last_became_active":"2024-09-20T22:06:54.853920+0000","last_became_peered":"2024-09-20T22:06:54.853920+0000","last_unstale":"2024-09-20T22:19:06.632559+0000","last_undegraded":"2024-09-20T22:19:06.632559+0000","last_fullsized":"2024-09-20T22:19:06.632559+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'52","last_scrub_stamp":"2024-09-20T22:17:08.395578+0000","last_deep_scrub":"328'52","last_deep_scrub_stamp":"2024-09-20T22:17:08.395578+0000","last_clean_scrub_stamp":"2024-09-20T22:17:08.395578+0000","objects_scrubbed":10,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":65,"num_read_kb":50,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":879,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.636762+0000","last_change":"2024-09-20T22:17:28.386631+0000","last_active":"2024-09-20T22:19:02.636762+0000","last_peered":"2024-09-20T22:19:02.636762+0000","last_clean":"2024-09-20T22:19:02.636762+0000","last_became_active":"2024-09-20T22:12:57.884257+0000","last_became_peered":"2024-09-20T22:12:57.884257+0000","last_unstale":"2024-09-20T22:19:02.636762+0000","last_undegraded":"2024-09-20T22:19:02.636762+0000","last_fullsized":"2024-09-20T22:19:02.636762+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:28.386523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:28.386523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.323817+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":873,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.652537+0000","last_change":"2024-09-20T22:17:06.944648+0000","last_active":"2024-09-20T22:19:02.652537+0000","last_peered":"2024-09-20T22:19:02.652537+0000","last_clean":"2024-09-20T22:19:02.652537+0000","last_became_active":"2024-09-20T22:06:54.843792+0000","last_became_peered":"2024-09-20T22:06:54.843792+0000","last_unstale":"2024-09-20T22:19:02.652537+0000","last_undegraded":"2024-09-20T22:19:02.652537+0000","last_fullsized":"2024-09-20T22:19:02.652537+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:17:06.944406+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_clean_scrub_stamp":"2024-09-20T22:17:06.944406+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:05.590988+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":316,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.640232+0000","last_change":"2024-09-20T22:17:57.210932+0000","last_active":"2024-09-20T22:19:06.640232+0000","last_peered":"2024-09-20T22:19:06.640232+0000","last_clean":"2024-09-20T22:19:06.640232+0000","last_became_active":"2024-09-20T22:12:52.527547+0000","last_became_peered":"2024-09-20T22:12:52.527547+0000","last_unstale":"2024-09-20T22:19:06.640232+0000","last_undegraded":"2024-09-20T22:19:06.640232+0000","last_fullsized":"2024-09-20T22:19:06.640232+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:57.210706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.210706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:15.145091+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":859,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.631919+0000","last_change":"2024-09-20T22:18:11.542095+0000","last_active":"2024-09-20T22:19:06.631919+0000","last_peered":"2024-09-20T22:19:06.631919+0000","last_clean":"2024-09-20T22:19:06.631919+0000","last_became_active":"2024-09-20T22:02:56.305691+0000","last_became_peered":"2024-09-20T22:02:56.305691+0000","last_unstale":"2024-09-20T22:19:06.631919+0000","last_undegraded":"2024-09-20T22:19:06.631919+0000","last_fullsized":"2024-09-20T22:19:06.631919+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:11.541987+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:18:11.541987+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:32.953037+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"463'2174","reported_seq":2196,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:08.248758+0000","last_change":"2024-09-20T22:18:46.633457+0000","last_active":"2024-09-20T22:19:08.248758+0000","last_peered":"2024-09-20T22:19:08.248758+0000","last_clean":"2024-09-20T22:19:08.248758+0000","last_became_active":"2024-09-20T22:18:46.631872+0000","last_became_peered":"2024-09-20T22:18:46.631872+0000","last_unstale":"2024-09-20T22:19:08.248758+0000","last_undegraded":"2024-09-20T22:19:08.248758+0000","last_fullsized":"2024-09-20T22:19:08.248758+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":2174,"log_dups_size":0,"ondisk_log_size":2174,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:52.269579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":142475264,"num_objects":2174,"num_object_clones":0,"num_object_copies":6522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2174,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2174,"num_write_kb":139136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":892,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.649365+0000","last_change":"2024-09-20T22:17:38.946417+0000","last_active":"2024-09-20T22:19:02.649365+0000","last_peered":"2024-09-20T22:19:02.649365+0000","last_clean":"2024-09-20T22:19:02.649365+0000","last_became_active":"2024-09-20T22:15:45.620376+0000","last_became_peered":"2024-09-20T22:15:45.620376+0000","last_unstale":"2024-09-20T22:19:02.649365+0000","last_undegraded":"2024-09-20T22:19:02.649365+0000","last_fullsized":"2024-09-20T22:19:02.649365+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:38.946300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:38.946300+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"328'17","reported_seq":773,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.633698+0000","last_change":"2024-09-20T22:17:01.307957+0000","last_active":"2024-09-20T22:19:06.633698+0000","last_peered":"2024-09-20T22:19:06.633698+0000","last_clean":"2024-09-20T22:19:06.633698+0000","last_became_active":"2024-09-20T22:12:52.521073+0000","last_became_peered":"2024-09-20T22:12:52.521073+0000","last_unstale":"2024-09-20T22:19:06.633698+0000","last_undegraded":"2024-09-20T22:19:06.633698+0000","last_fullsized":"2024-09-20T22:19:06.633698+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'17","last_scrub_stamp":"2024-09-20T22:17:01.307755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:01.307755+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":35,"num_read_kb":24,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":702,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.630252+0000","last_change":"2024-09-20T22:17:09.478913+0000","last_active":"2024-09-20T22:19:06.630252+0000","last_peered":"2024-09-20T22:19:06.630252+0000","last_clean":"2024-09-20T22:19:06.630252+0000","last_became_active":"2024-09-20T22:06:54.822493+0000","last_became_peered":"2024-09-20T22:06:54.822493+0000","last_unstale":"2024-09-20T22:19:06.630252+0000","last_undegraded":"2024-09-20T22:19:06.630252+0000","last_fullsized":"2024-09-20T22:19:06.630252+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:09.478390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:09.478390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":51,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":923,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.645985+0000","last_change":"2024-09-20T22:17:40.827298+0000","last_active":"2024-09-20T22:19:02.645985+0000","last_peered":"2024-09-20T22:19:02.645985+0000","last_clean":"2024-09-20T22:19:02.645985+0000","last_became_active":"2024-09-20T22:12:52.517255+0000","last_became_peered":"2024-09-20T22:12:52.517255+0000","last_unstale":"2024-09-20T22:19:02.645985+0000","last_undegraded":"2024-09-20T22:19:02.645985+0000","last_fullsized":"2024-09-20T22:19:02.645985+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:17:40.827184+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:17:40.827184+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":877,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.646258+0000","last_change":"2024-09-20T22:17:08.832699+0000","last_active":"2024-09-20T22:19:02.646258+0000","last_peered":"2024-09-20T22:19:02.646258+0000","last_clean":"2024-09-20T22:19:02.646258+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:19:02.646258+0000","last_undegraded":"2024-09-20T22:19:02.646258+0000","last_fullsized":"2024-09-20T22:19:02.646258+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:08.832589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:17:08.832589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":850,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.650503+0000","last_change":"2024-09-20T22:18:35.868334+0000","last_active":"2024-09-20T22:19:02.650503+0000","last_peered":"2024-09-20T22:19:02.650503+0000","last_clean":"2024-09-20T22:19:02.650503+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:19:02.650503+0000","last_undegraded":"2024-09-20T22:19:02.650503+0000","last_fullsized":"2024-09-20T22:19:02.650503+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:35.868264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:35.868264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:03.414620+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"463'2117","reported_seq":2141,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.812218+0000","last_change":"2024-09-20T22:18:46.630336+0000","last_active":"2024-09-20T22:19:06.812218+0000","last_peered":"2024-09-20T22:19:06.812218+0000","last_clean":"2024-09-20T22:19:06.812218+0000","last_became_active":"2024-09-20T22:18:46.630028+0000","last_became_peered":"2024-09-20T22:18:46.630028+0000","last_unstale":"2024-09-20T22:19:06.812218+0000","last_undegraded":"2024-09-20T22:19:06.812218+0000","last_fullsized":"2024-09-20T22:19:06.812218+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":2117,"log_dups_size":0,"ondisk_log_size":2117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:04.438671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":138739712,"num_objects":2117,"num_object_clones":0,"num_object_copies":6351,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2117,"num_write_kb":135488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":877,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.631642+0000","last_change":"2024-09-20T22:18:01.139847+0000","last_active":"2024-09-20T22:19:06.631642+0000","last_peered":"2024-09-20T22:19:06.631642+0000","last_clean":"2024-09-20T22:19:06.631642+0000","last_became_active":"2024-09-20T22:15:45.630804+0000","last_became_peered":"2024-09-20T22:15:45.630804+0000","last_unstale":"2024-09-20T22:19:06.631642+0000","last_undegraded":"2024-09-20T22:19:06.631642+0000","last_fullsized":"2024-09-20T22:19:06.631642+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:18:01.139707+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:18:01.139707+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:17.176581+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"328'24","reported_seq":347,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.641134+0000","last_change":"2024-09-20T22:17:05.822751+0000","last_active":"2024-09-20T22:19:06.641134+0000","last_peered":"2024-09-20T22:19:06.641134+0000","last_clean":"2024-09-20T22:19:06.641134+0000","last_became_active":"2024-09-20T22:12:52.527919+0000","last_became_peered":"2024-09-20T22:12:52.527919+0000","last_unstale":"2024-09-20T22:19:06.641134+0000","last_undegraded":"2024-09-20T22:19:06.641134+0000","last_fullsized":"2024-09-20T22:19:06.641134+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'24","last_scrub_stamp":"2024-09-20T22:17:05.822650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:05.822650+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.753530+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":322,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.640289+0000","last_change":"2024-09-20T22:17:11.814549+0000","last_active":"2024-09-20T22:19:06.640289+0000","last_peered":"2024-09-20T22:19:06.640289+0000","last_clean":"2024-09-20T22:19:06.640289+0000","last_became_active":"2024-09-20T22:17:03.663870+0000","last_became_peered":"2024-09-20T22:17:03.663870+0000","last_unstale":"2024-09-20T22:19:06.640289+0000","last_undegraded":"2024-09-20T22:19:06.640289+0000","last_fullsized":"2024-09-20T22:19:06.640289+0000","mapping_epoch":415,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":416,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:11.814439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:17:11.814439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":1861,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:04.314715+0000","last_change":"2024-09-20T22:17:59.765338+0000","last_active":"2024-09-20T22:19:04.314715+0000","last_peered":"2024-09-20T22:19:04.314715+0000","last_clean":"2024-09-20T22:19:04.314715+0000","last_became_active":"2024-09-20T22:12:52.526622+0000","last_became_peered":"2024-09-20T22:12:52.526622+0000","last_unstale":"2024-09-20T22:19:04.314715+0000","last_undegraded":"2024-09-20T22:19:04.314715+0000","last_fullsized":"2024-09-20T22:19:04.314715+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:17:59.765217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:17:59.765217+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:23.072429+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":951,"num_read_kb":946,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":408,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.652998+0000","last_change":"2024-09-20T22:17:08.955894+0000","last_active":"2024-09-20T22:19:02.652998+0000","last_peered":"2024-09-20T22:19:02.652998+0000","last_clean":"2024-09-20T22:19:02.652998+0000","last_became_active":"2024-09-20T22:12:52.518823+0000","last_became_peered":"2024-09-20T22:12:52.518823+0000","last_unstale":"2024-09-20T22:19:02.652998+0000","last_undegraded":"2024-09-20T22:19:02.652998+0000","last_fullsized":"2024-09-20T22:19:02.652998+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:08.955790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:17:08.955790+0000","last_clean_scrub_stamp":"2024-09-20T22:17:08.955790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:05.590882+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":701,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.632151+0000","last_change":"2024-09-20T22:18:40.325418+0000","last_active":"2024-09-20T22:19:06.632151+0000","last_peered":"2024-09-20T22:19:06.632151+0000","last_clean":"2024-09-20T22:19:06.632151+0000","last_became_active":"2024-09-20T22:06:54.853520+0000","last_became_peered":"2024-09-20T22:06:54.853520+0000","last_unstale":"2024-09-20T22:19:06.632151+0000","last_undegraded":"2024-09-20T22:19:06.632151+0000","last_fullsized":"2024-09-20T22:19:06.632151+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:40.325308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:40.325308+0000","last_clean_scrub_stamp":"2024-09-20T22:18:40.325308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:40.858238+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.a","version":"463'2280","reported_seq":2304,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:08.251885+0000","last_change":"2024-09-20T22:18:46.631467+0000","last_active":"2024-09-20T22:19:08.251885+0000","last_peered":"2024-09-20T22:19:08.251885+0000","last_clean":"2024-09-20T22:19:08.251885+0000","last_became_active":"2024-09-20T22:18:46.630294+0000","last_became_peered":"2024-09-20T22:18:46.630294+0000","last_unstale":"2024-09-20T22:19:08.251885+0000","last_undegraded":"2024-09-20T22:19:08.251885+0000","last_fullsized":"2024-09-20T22:19:08.251885+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":2280,"log_dups_size":0,"ondisk_log_size":2280,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:52.214985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":149422080,"num_objects":2280,"num_object_clones":0,"num_object_copies":6840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2280,"num_write_kb":145920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"328'28","reported_seq":1111,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.634484+0000","last_change":"2024-09-20T22:17:57.685174+0000","last_active":"2024-09-20T22:19:06.634484+0000","last_peered":"2024-09-20T22:19:06.634484+0000","last_clean":"2024-09-20T22:19:06.634484+0000","last_became_active":"2024-09-20T22:02:52.482284+0000","last_became_peered":"2024-09-20T22:02:52.482284+0000","last_unstale":"2024-09-20T22:19:06.634484+0000","last_undegraded":"2024-09-20T22:19:06.634484+0000","last_fullsized":"2024-09-20T22:19:06.634484+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:17:57.685068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.685068+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:23.761739+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":814,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.651894+0000","last_change":"2024-09-20T22:16:36.729250+0000","last_active":"2024-09-20T22:19:02.651894+0000","last_peered":"2024-09-20T22:19:02.651894+0000","last_clean":"2024-09-20T22:19:02.651894+0000","last_became_active":"2024-09-20T22:12:21.519717+0000","last_became_peered":"2024-09-20T22:12:21.519717+0000","last_unstale":"2024-09-20T22:19:02.651894+0000","last_undegraded":"2024-09-20T22:19:02.651894+0000","last_fullsized":"2024-09-20T22:19:02.651894+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:36.729158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:36.729158+0000","last_clean_scrub_stamp":"2024-09-20T22:16:36.729158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:05.590833+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":892,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.649233+0000","last_change":"2024-09-20T22:18:06.776685+0000","last_active":"2024-09-20T22:19:02.649233+0000","last_peered":"2024-09-20T22:19:02.649233+0000","last_clean":"2024-09-20T22:19:02.649233+0000","last_became_active":"2024-09-20T22:15:45.620972+0000","last_became_peered":"2024-09-20T22:15:45.620972+0000","last_unstale":"2024-09-20T22:19:02.649233+0000","last_undegraded":"2024-09-20T22:19:02.649233+0000","last_fullsized":"2024-09-20T22:19:02.649233+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:18:06.776546+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:18:06.776546+0000","last_clean_scrub_stamp":"2024-09-20T22:18:06.776546+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:25.281912+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":444,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.650452+0000","last_change":"2024-09-20T22:18:04.809849+0000","last_active":"2024-09-20T22:19:02.650452+0000","last_peered":"2024-09-20T22:19:02.650452+0000","last_clean":"2024-09-20T22:19:02.650452+0000","last_became_active":"2024-09-20T22:10:48.907271+0000","last_became_peered":"2024-09-20T22:10:48.907271+0000","last_unstale":"2024-09-20T22:19:02.650452+0000","last_undegraded":"2024-09-20T22:19:02.650452+0000","last_fullsized":"2024-09-20T22:19:02.650452+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:04.809565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_clean_scrub_stamp":"2024-09-20T22:18:04.809565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:30.143079+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":869,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.652904+0000","last_change":"2024-09-20T22:17:02.907969+0000","last_active":"2024-09-20T22:19:02.652904+0000","last_peered":"2024-09-20T22:19:02.652904+0000","last_clean":"2024-09-20T22:19:02.652904+0000","last_became_active":"2024-09-20T22:12:44.630044+0000","last_became_peered":"2024-09-20T22:12:44.630044+0000","last_unstale":"2024-09-20T22:19:02.652904+0000","last_undegraded":"2024-09-20T22:19:02.652904+0000","last_fullsized":"2024-09-20T22:19:02.652904+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:02.907851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:17:02.907851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:05.590856+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"463'2252","reported_seq":2276,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:08.236223+0000","last_change":"2024-09-20T22:18:46.630204+0000","last_active":"2024-09-20T22:19:08.236223+0000","last_peered":"2024-09-20T22:19:08.236223+0000","last_clean":"2024-09-20T22:19:08.236223+0000","last_became_active":"2024-09-20T22:18:46.629898+0000","last_became_peered":"2024-09-20T22:18:46.629898+0000","last_unstale":"2024-09-20T22:19:08.236223+0000","last_undegraded":"2024-09-20T22:19:08.236223+0000","last_fullsized":"2024-09-20T22:19:08.236223+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":2252,"log_dups_size":0,"ondisk_log_size":2252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:00.178009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":147587072,"num_objects":2252,"num_object_clones":0,"num_object_copies":6756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2252,"num_write_kb":144128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"75'128","reported_seq":890,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.631480+0000","last_change":"2024-09-20T22:17:40.268249+0000","last_active":"2024-09-20T22:19:06.631480+0000","last_peered":"2024-09-20T22:19:06.631480+0000","last_clean":"2024-09-20T22:19:06.631480+0000","last_became_active":"2024-09-20T22:17:40.147603+0000","last_became_peered":"2024-09-20T22:17:40.147603+0000","last_unstale":"2024-09-20T22:19:06.631480+0000","last_undegraded":"2024-09-20T22:19:06.631480+0000","last_fullsized":"2024-09-20T22:19:06.631480+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:40.268008+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:17:40.268008+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"328'19","reported_seq":898,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.637049+0000","last_change":"2024-09-20T22:18:36.321764+0000","last_active":"2024-09-20T22:19:02.637049+0000","last_peered":"2024-09-20T22:19:02.637049+0000","last_clean":"2024-09-20T22:19:02.637049+0000","last_became_active":"2024-09-20T22:06:54.824284+0000","last_became_peered":"2024-09-20T22:06:54.824284+0000","last_unstale":"2024-09-20T22:19:02.637049+0000","last_undegraded":"2024-09-20T22:19:02.637049+0000","last_fullsized":"2024-09-20T22:19:02.637049+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:18:36.321653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:36.321653+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:37.875133+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":862,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.636275+0000","last_change":"2024-09-20T22:17:36.374494+0000","last_active":"2024-09-20T22:19:02.636275+0000","last_peered":"2024-09-20T22:19:02.636275+0000","last_clean":"2024-09-20T22:19:02.636275+0000","last_became_active":"2024-09-20T22:02:50.109297+0000","last_became_peered":"2024-09-20T22:02:50.109297+0000","last_unstale":"2024-09-20T22:19:02.636275+0000","last_undegraded":"2024-09-20T22:19:02.636275+0000","last_fullsized":"2024-09-20T22:19:02.636275+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:36.374371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:36.374371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:05.201143+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":869,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:04.545219+0000","last_change":"2024-09-20T22:19:04.545255+0000","last_active":"2024-09-20T22:19:04.545219+0000","last_peered":"2024-09-20T22:19:04.545219+0000","last_clean":"2024-09-20T22:19:04.545219+0000","last_became_active":"2024-09-20T22:02:54.289854+0000","last_became_peered":"2024-09-20T22:02:54.289854+0000","last_unstale":"2024-09-20T22:19:04.545219+0000","last_undegraded":"2024-09-20T22:19:04.545219+0000","last_fullsized":"2024-09-20T22:19:04.545219+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:04.545147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:04.545147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:15.415981+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":819,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.631526+0000","last_change":"2024-09-20T22:17:55.630149+0000","last_active":"2024-09-20T22:19:06.631526+0000","last_peered":"2024-09-20T22:19:06.631526+0000","last_clean":"2024-09-20T22:19:06.631526+0000","last_became_active":"2024-09-20T22:12:21.521195+0000","last_became_peered":"2024-09-20T22:12:21.521195+0000","last_unstale":"2024-09-20T22:19:06.631526+0000","last_undegraded":"2024-09-20T22:19:06.631526+0000","last_fullsized":"2024-09-20T22:19:06.631526+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:55.630084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:55.630084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:15.908970+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"462'2054","reported_seq":2075,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.519676+0000","last_change":"2024-09-20T22:18:46.631808+0000","last_active":"2024-09-20T22:19:06.519676+0000","last_peered":"2024-09-20T22:19:06.519676+0000","last_clean":"2024-09-20T22:19:06.519676+0000","last_became_active":"2024-09-20T22:18:46.631505+0000","last_became_peered":"2024-09-20T22:18:46.631505+0000","last_unstale":"2024-09-20T22:19:06.519676+0000","last_undegraded":"2024-09-20T22:19:06.519676+0000","last_fullsized":"2024-09-20T22:19:06.519676+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":2054,"log_dups_size":0,"ondisk_log_size":2054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:07.494908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":134610944,"num_objects":2054,"num_object_clones":0,"num_object_copies":6162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2054,"num_write_kb":131456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":645,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.630140+0000","last_change":"2024-09-20T22:17:54.588617+0000","last_active":"2024-09-20T22:19:06.630140+0000","last_peered":"2024-09-20T22:19:06.630140+0000","last_clean":"2024-09-20T22:19:06.630140+0000","last_became_active":"2024-09-20T22:17:40.139875+0000","last_became_peered":"2024-09-20T22:17:40.139875+0000","last_unstale":"2024-09-20T22:19:06.630140+0000","last_undegraded":"2024-09-20T22:19:06.630140+0000","last_fullsized":"2024-09-20T22:19:06.630140+0000","mapping_epoch":358,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:54.588504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:54.588504+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:10.848347+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"328'31","reported_seq":916,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.636939+0000","last_change":"2024-09-20T22:16:17.388362+0000","last_active":"2024-09-20T22:19:02.636939+0000","last_peered":"2024-09-20T22:19:02.636939+0000","last_clean":"2024-09-20T22:19:02.636939+0000","last_became_active":"2024-09-20T22:06:54.823849+0000","last_became_peered":"2024-09-20T22:06:54.823849+0000","last_unstale":"2024-09-20T22:19:02.636939+0000","last_undegraded":"2024-09-20T22:19:02.636939+0000","last_fullsized":"2024-09-20T22:19:02.636939+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'31","last_scrub_stamp":"2024-09-20T22:16:17.387895+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:17.387895+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.323474+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":50,"num_read_kb":40,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":880,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.633229+0000","last_change":"2024-09-20T22:17:08.847011+0000","last_active":"2024-09-20T22:19:06.633229+0000","last_peered":"2024-09-20T22:19:06.633229+0000","last_clean":"2024-09-20T22:19:06.633229+0000","last_became_active":"2024-09-20T22:06:54.828837+0000","last_became_peered":"2024-09-20T22:06:54.828837+0000","last_unstale":"2024-09-20T22:19:06.633229+0000","last_undegraded":"2024-09-20T22:19:06.633229+0000","last_fullsized":"2024-09-20T22:19:06.633229+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:08.846904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:17:08.846904+0000","last_clean_scrub_stamp":"2024-09-20T22:17:08.846904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":713,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.632243+0000","last_change":"2024-09-20T22:18:10.180063+0000","last_active":"2024-09-20T22:19:06.632243+0000","last_peered":"2024-09-20T22:19:06.632243+0000","last_clean":"2024-09-20T22:19:06.632243+0000","last_became_active":"2024-09-20T22:12:52.520612+0000","last_became_peered":"2024-09-20T22:12:52.520612+0000","last_unstale":"2024-09-20T22:19:06.632243+0000","last_undegraded":"2024-09-20T22:19:06.632243+0000","last_fullsized":"2024-09-20T22:19:06.632243+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:10.179939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:10.179939+0000","last_clean_scrub_stamp":"2024-09-20T22:18:10.179939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:22.393592+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":873,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.646356+0000","last_change":"2024-09-20T22:17:39.808194+0000","last_active":"2024-09-20T22:19:02.646356+0000","last_peered":"2024-09-20T22:19:02.646356+0000","last_clean":"2024-09-20T22:19:02.646356+0000","last_became_active":"2024-09-20T22:02:56.307217+0000","last_became_peered":"2024-09-20T22:02:56.307217+0000","last_unstale":"2024-09-20T22:19:02.646356+0000","last_undegraded":"2024-09-20T22:19:02.646356+0000","last_fullsized":"2024-09-20T22:19:02.646356+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:39.807978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:39.807978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:04.536384+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"463'2215","reported_seq":2239,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:08.243345+0000","last_change":"2024-09-20T22:18:46.632567+0000","last_active":"2024-09-20T22:19:08.243345+0000","last_peered":"2024-09-20T22:19:08.243345+0000","last_clean":"2024-09-20T22:19:08.243345+0000","last_became_active":"2024-09-20T22:18:46.632219+0000","last_became_peered":"2024-09-20T22:18:46.632219+0000","last_unstale":"2024-09-20T22:19:08.243345+0000","last_undegraded":"2024-09-20T22:19:08.243345+0000","last_fullsized":"2024-09-20T22:19:08.243345+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":2215,"log_dups_size":0,"ondisk_log_size":2215,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:14.629742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":145162240,"num_objects":2215,"num_object_clones":0,"num_object_copies":6645,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2215,"num_write_kb":141760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1],"acting":[6,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":1153,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.649430+0000","last_change":"2024-09-20T22:17:42.025688+0000","last_active":"2024-09-20T22:19:02.649430+0000","last_peered":"2024-09-20T22:19:02.649430+0000","last_clean":"2024-09-20T22:19:02.649430+0000","last_became_active":"2024-09-20T22:17:40.146794+0000","last_became_peered":"2024-09-20T22:17:40.146794+0000","last_unstale":"2024-09-20T22:19:02.649430+0000","last_undegraded":"2024-09-20T22:19:02.649430+0000","last_fullsized":"2024-09-20T22:19:02.649430+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:42.025576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:42.025576+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"328'25","reported_seq":893,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.636606+0000","last_change":"2024-09-20T22:18:05.300953+0000","last_active":"2024-09-20T22:19:02.636606+0000","last_peered":"2024-09-20T22:19:02.636606+0000","last_clean":"2024-09-20T22:19:02.636606+0000","last_became_active":"2024-09-20T22:02:52.283700+0000","last_became_peered":"2024-09-20T22:02:52.283700+0000","last_unstale":"2024-09-20T22:19:02.636606+0000","last_undegraded":"2024-09-20T22:19:02.636606+0000","last_fullsized":"2024-09-20T22:19:02.636606+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'25","last_scrub_stamp":"2024-09-20T22:18:05.300862+0000","last_deep_scrub":"328'25","last_deep_scrub_stamp":"2024-09-20T22:18:05.300862+0000","last_clean_scrub_stamp":"2024-09-20T22:18:05.300862+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:35.276486+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":319,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.641295+0000","last_change":"2024-09-20T22:17:35.054489+0000","last_active":"2024-09-20T22:19:06.641295+0000","last_peered":"2024-09-20T22:19:06.641295+0000","last_clean":"2024-09-20T22:19:06.641295+0000","last_became_active":"2024-09-20T22:12:52.528310+0000","last_became_peered":"2024-09-20T22:12:52.528310+0000","last_unstale":"2024-09-20T22:19:06.641295+0000","last_undegraded":"2024-09-20T22:19:06.641295+0000","last_fullsized":"2024-09-20T22:19:06.641295+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:17:35.054116+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:17:35.054116+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.753642+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"283'11","reported_seq":1110,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.647700+0000","last_change":"2024-09-20T22:17:31.773328+0000","last_active":"2024-09-20T22:19:02.647700+0000","last_peered":"2024-09-20T22:19:02.647700+0000","last_clean":"2024-09-20T22:19:02.647700+0000","last_became_active":"2024-09-20T22:11:30.485356+0000","last_became_peered":"2024-09-20T22:11:30.485356+0000","last_unstale":"2024-09-20T22:19:02.647700+0000","last_undegraded":"2024-09-20T22:19:02.647700+0000","last_fullsized":"2024-09-20T22:19:02.647700+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'11","last_scrub_stamp":"2024-09-20T22:17:31.773028+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:17:31.773028+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":851,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.636829+0000","last_change":"2024-09-20T22:16:27.486633+0000","last_active":"2024-09-20T22:19:02.636829+0000","last_peered":"2024-09-20T22:19:02.636829+0000","last_clean":"2024-09-20T22:19:02.636829+0000","last_became_active":"2024-09-20T22:06:54.823644+0000","last_became_peered":"2024-09-20T22:06:54.823644+0000","last_unstale":"2024-09-20T22:19:02.636829+0000","last_undegraded":"2024-09-20T22:19:02.636829+0000","last_fullsized":"2024-09-20T22:19:02.636829+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:27.486520+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:16:27.486520+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.323436+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.4","version":"462'1913","reported_seq":1935,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:05.892060+0000","last_change":"2024-09-20T22:18:46.634428+0000","last_active":"2024-09-20T22:19:05.892060+0000","last_peered":"2024-09-20T22:19:05.892060+0000","last_clean":"2024-09-20T22:19:05.892060+0000","last_became_active":"2024-09-20T22:18:46.633985+0000","last_became_peered":"2024-09-20T22:18:46.633985+0000","last_unstale":"2024-09-20T22:19:05.892060+0000","last_undegraded":"2024-09-20T22:19:05.892060+0000","last_fullsized":"2024-09-20T22:19:05.892060+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":1913,"log_dups_size":0,"ondisk_log_size":1913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:56.825909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125370368,"num_objects":1913,"num_object_clones":0,"num_object_copies":5739,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1913,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1913,"num_write_kb":122432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.e","version":"75'128","reported_seq":759,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.636175+0000","last_change":"2024-09-20T22:17:41.370648+0000","last_active":"2024-09-20T22:19:02.636175+0000","last_peered":"2024-09-20T22:19:02.636175+0000","last_clean":"2024-09-20T22:19:02.636175+0000","last_became_active":"2024-09-20T22:17:40.140779+0000","last_became_peered":"2024-09-20T22:17:40.140779+0000","last_unstale":"2024-09-20T22:19:02.636175+0000","last_undegraded":"2024-09-20T22:19:02.636175+0000","last_fullsized":"2024-09-20T22:19:02.636175+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:41.370226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:41.370226+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.323846+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"328'15","reported_seq":334,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.641482+0000","last_change":"2024-09-20T22:16:59.741092+0000","last_active":"2024-09-20T22:19:06.641482+0000","last_peered":"2024-09-20T22:19:06.641482+0000","last_clean":"2024-09-20T22:19:06.641482+0000","last_became_active":"2024-09-20T22:12:52.527531+0000","last_became_peered":"2024-09-20T22:12:52.527531+0000","last_unstale":"2024-09-20T22:19:06.641482+0000","last_undegraded":"2024-09-20T22:19:06.641482+0000","last_fullsized":"2024-09-20T22:19:06.641482+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:16:59.740962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:59.740962+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.753471+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":499,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.633927+0000","last_change":"2024-09-20T22:17:59.659397+0000","last_active":"2024-09-20T22:19:06.633927+0000","last_peered":"2024-09-20T22:19:06.633927+0000","last_clean":"2024-09-20T22:19:06.633927+0000","last_became_active":"2024-09-20T22:10:14.536155+0000","last_became_peered":"2024-09-20T22:10:14.536155+0000","last_unstale":"2024-09-20T22:19:06.633927+0000","last_undegraded":"2024-09-20T22:19:06.633927+0000","last_fullsized":"2024-09-20T22:19:06.633927+0000","mapping_epoch":246,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":247,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:17:59.659279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:59.659279+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:21.111403+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":885,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.646691+0000","last_change":"2024-09-20T22:18:40.088266+0000","last_active":"2024-09-20T22:19:02.646691+0000","last_peered":"2024-09-20T22:19:02.646691+0000","last_clean":"2024-09-20T22:19:02.646691+0000","last_became_active":"2024-09-20T22:02:54.288783+0000","last_became_peered":"2024-09-20T22:02:54.288783+0000","last_unstale":"2024-09-20T22:19:02.646691+0000","last_undegraded":"2024-09-20T22:19:02.646691+0000","last_fullsized":"2024-09-20T22:19:02.646691+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:40.088194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:18:40.088194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:55.347953+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":307,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.640571+0000","last_change":"2024-09-20T22:16:24.020680+0000","last_active":"2024-09-20T22:19:06.640571+0000","last_peered":"2024-09-20T22:19:06.640571+0000","last_clean":"2024-09-20T22:19:06.640571+0000","last_became_active":"2024-09-20T22:12:52.512924+0000","last_became_peered":"2024-09-20T22:12:52.512924+0000","last_unstale":"2024-09-20T22:19:06.640571+0000","last_undegraded":"2024-09-20T22:19:06.640571+0000","last_fullsized":"2024-09-20T22:19:06.640571+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:24.020573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:24.020573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.753271+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.5","version":"462'1996","reported_seq":2018,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:05.907750+0000","last_change":"2024-09-20T22:18:46.633877+0000","last_active":"2024-09-20T22:19:05.907750+0000","last_peered":"2024-09-20T22:19:05.907750+0000","last_clean":"2024-09-20T22:19:05.907750+0000","last_became_active":"2024-09-20T22:18:46.633490+0000","last_became_peered":"2024-09-20T22:18:46.633490+0000","last_unstale":"2024-09-20T22:19:05.907750+0000","last_undegraded":"2024-09-20T22:19:05.907750+0000","last_fullsized":"2024-09-20T22:19:05.907750+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":1996,"log_dups_size":0,"ondisk_log_size":1996,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:03.847391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":130809856,"num_objects":1996,"num_object_clones":0,"num_object_copies":5988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1996,"num_write_kb":127744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.f","version":"75'128","reported_seq":916,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.631552+0000","last_change":"2024-09-20T22:17:07.885447+0000","last_active":"2024-09-20T22:19:06.631552+0000","last_peered":"2024-09-20T22:19:06.631552+0000","last_clean":"2024-09-20T22:19:06.631552+0000","last_became_active":"2024-09-20T22:15:50.576540+0000","last_became_peered":"2024-09-20T22:15:50.576540+0000","last_unstale":"2024-09-20T22:19:06.631552+0000","last_undegraded":"2024-09-20T22:19:06.631552+0000","last_fullsized":"2024-09-20T22:19:06.631552+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:07.885346+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:17:07.885346+0000","last_clean_scrub_stamp":"2024-09-20T22:17:07.885346+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"328'35","reported_seq":975,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.632962+0000","last_change":"2024-09-20T22:17:03.808976+0000","last_active":"2024-09-20T22:19:06.632962+0000","last_peered":"2024-09-20T22:19:06.632962+0000","last_clean":"2024-09-20T22:19:06.632962+0000","last_became_active":"2024-09-20T22:10:30.652967+0000","last_became_peered":"2024-09-20T22:10:30.652967+0000","last_unstale":"2024-09-20T22:19:06.632962+0000","last_undegraded":"2024-09-20T22:19:06.632962+0000","last_fullsized":"2024-09-20T22:19:06.632962+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'35","last_scrub_stamp":"2024-09-20T22:17:03.808887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:03.808887+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":874,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.650860+0000","last_change":"2024-09-20T22:17:37.950739+0000","last_active":"2024-09-20T22:19:02.650860+0000","last_peered":"2024-09-20T22:19:02.650860+0000","last_clean":"2024-09-20T22:19:02.650860+0000","last_became_active":"2024-09-20T22:11:30.488650+0000","last_became_peered":"2024-09-20T22:11:30.488650+0000","last_unstale":"2024-09-20T22:19:02.650860+0000","last_undegraded":"2024-09-20T22:19:02.650860+0000","last_fullsized":"2024-09-20T22:19:02.650860+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:37.950674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:12:46.602035+0000","last_clean_scrub_stamp":"2024-09-20T22:17:37.950674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":909,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.646203+0000","last_change":"2024-09-20T22:18:11.941197+0000","last_active":"2024-09-20T22:19:02.646203+0000","last_peered":"2024-09-20T22:19:02.646203+0000","last_clean":"2024-09-20T22:19:02.646203+0000","last_became_active":"2024-09-20T22:12:52.515295+0000","last_became_peered":"2024-09-20T22:12:52.515295+0000","last_unstale":"2024-09-20T22:19:02.646203+0000","last_undegraded":"2024-09-20T22:19:02.646203+0000","last_fullsized":"2024-09-20T22:19:02.646203+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:11.941075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_clean_scrub_stamp":"2024-09-20T22:18:11.941075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:41.699594+0000","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":466,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.649351+0000","last_change":"2024-09-20T22:18:44.881893+0000","last_active":"2024-09-20T22:19:02.649351+0000","last_peered":"2024-09-20T22:19:02.649351+0000","last_clean":"2024-09-20T22:19:02.649351+0000","last_became_active":"2024-09-20T22:10:30.657116+0000","last_became_peered":"2024-09-20T22:10:30.657116+0000","last_unstale":"2024-09-20T22:19:02.649351+0000","last_undegraded":"2024-09-20T22:19:02.649351+0000","last_fullsized":"2024-09-20T22:19:02.649351+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.881819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.881819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:47.303693+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"463'2080","reported_seq":2104,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:07.371437+0000","last_change":"2024-09-20T22:18:46.630310+0000","last_active":"2024-09-20T22:19:07.371437+0000","last_peered":"2024-09-20T22:19:07.371437+0000","last_clean":"2024-09-20T22:19:07.371437+0000","last_became_active":"2024-09-20T22:18:46.630002+0000","last_became_peered":"2024-09-20T22:18:46.630002+0000","last_unstale":"2024-09-20T22:19:07.371437+0000","last_undegraded":"2024-09-20T22:19:07.371437+0000","last_fullsized":"2024-09-20T22:19:07.371437+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":2080,"log_dups_size":0,"ondisk_log_size":2080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:08.066821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":136314880,"num_objects":2080,"num_object_clones":0,"num_object_copies":6240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2080,"num_write_kb":133120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.c","version":"75'128","reported_seq":940,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.633507+0000","last_change":"2024-09-20T22:17:40.737208+0000","last_active":"2024-09-20T22:19:06.633507+0000","last_peered":"2024-09-20T22:19:06.633507+0000","last_clean":"2024-09-20T22:19:06.633507+0000","last_became_active":"2024-09-20T22:17:40.152364+0000","last_became_peered":"2024-09-20T22:17:40.152364+0000","last_unstale":"2024-09-20T22:19:06.633507+0000","last_undegraded":"2024-09-20T22:19:06.633507+0000","last_fullsized":"2024-09-20T22:19:06.633507+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:40.737088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:40.737088+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:07.545003+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"418'23","reported_seq":896,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.636047+0000","last_change":"2024-09-20T22:17:11.339243+0000","last_active":"2024-09-20T22:19:02.636047+0000","last_peered":"2024-09-20T22:19:02.636047+0000","last_clean":"2024-09-20T22:19:02.636047+0000","last_became_active":"2024-09-20T22:06:54.831957+0000","last_became_peered":"2024-09-20T22:06:54.831957+0000","last_unstale":"2024-09-20T22:19:02.636047+0000","last_undegraded":"2024-09-20T22:19:02.636047+0000","last_fullsized":"2024-09-20T22:19:02.636047+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"418'23","last_scrub_stamp":"2024-09-20T22:17:11.338821+0000","last_deep_scrub":"418'23","last_deep_scrub_stamp":"2024-09-20T22:17:11.338821+0000","last_clean_scrub_stamp":"2024-09-20T22:17:11.338821+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.323658+0000","scrub_duration":35,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":871,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.650405+0000","last_change":"2024-09-20T22:18:38.817211+0000","last_active":"2024-09-20T22:19:02.650405+0000","last_peered":"2024-09-20T22:19:02.650405+0000","last_clean":"2024-09-20T22:19:02.650405+0000","last_became_active":"2024-09-20T22:06:54.828266+0000","last_became_peered":"2024-09-20T22:06:54.828266+0000","last_unstale":"2024-09-20T22:19:02.650405+0000","last_undegraded":"2024-09-20T22:19:02.650405+0000","last_fullsized":"2024-09-20T22:19:02.650405+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:38.817089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:38.817089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:08.182041+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":906,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:03.959958+0000","last_change":"2024-09-20T22:19:03.959990+0000","last_active":"2024-09-20T22:19:03.959958+0000","last_peered":"2024-09-20T22:19:03.959958+0000","last_clean":"2024-09-20T22:19:03.959958+0000","last_became_active":"2024-09-20T22:12:52.514202+0000","last_became_peered":"2024-09-20T22:12:52.514202+0000","last_unstale":"2024-09-20T22:19:03.959958+0000","last_undegraded":"2024-09-20T22:19:03.959958+0000","last_fullsized":"2024-09-20T22:19:03.959958+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:03.959875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:03.959875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:24.857126+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":325,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.627143+0000","last_change":"2024-09-20T22:16:57.795001+0000","last_active":"2024-09-20T22:19:06.627143+0000","last_peered":"2024-09-20T22:19:06.627143+0000","last_clean":"2024-09-20T22:19:06.627143+0000","last_became_active":"2024-09-20T22:12:44.638546+0000","last_became_peered":"2024-09-20T22:12:44.638546+0000","last_unstale":"2024-09-20T22:19:06.627143+0000","last_undegraded":"2024-09-20T22:19:06.627143+0000","last_fullsized":"2024-09-20T22:19:06.627143+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:57.794891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:57.794891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:07.544947+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"462'1909","reported_seq":1931,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.322566+0000","last_change":"2024-09-20T22:18:46.630279+0000","last_active":"2024-09-20T22:19:06.322566+0000","last_peered":"2024-09-20T22:19:06.322566+0000","last_clean":"2024-09-20T22:19:06.322566+0000","last_became_active":"2024-09-20T22:18:46.629889+0000","last_became_peered":"2024-09-20T22:18:46.629889+0000","last_unstale":"2024-09-20T22:19:06.322566+0000","last_undegraded":"2024-09-20T22:19:06.322566+0000","last_fullsized":"2024-09-20T22:19:06.322566+0000","mapping_epoch":456,"log_start":"0'0","ondisk_log_start":"0'0","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":1909,"log_dups_size":0,"ondisk_log_size":1909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:14.258937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":125108224,"num_objects":1909,"num_object_clones":0,"num_object_copies":5727,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1909,"num_write_kb":122176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,4],"acting":[5,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"75'128","reported_seq":475,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.650045+0000","last_change":"2024-09-20T22:17:40.996913+0000","last_active":"2024-09-20T22:19:02.650045+0000","last_peered":"2024-09-20T22:19:02.650045+0000","last_clean":"2024-09-20T22:19:02.650045+0000","last_became_active":"2024-09-20T22:17:40.136632+0000","last_became_peered":"2024-09-20T22:17:40.136632+0000","last_unstale":"2024-09-20T22:19:02.650045+0000","last_undegraded":"2024-09-20T22:19:02.650045+0000","last_fullsized":"2024-09-20T22:19:02.650045+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:40.996791+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:17:40.996791+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:10.524269+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"416'34","reported_seq":429,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.651498+0000","last_change":"2024-09-20T22:17:04.960544+0000","last_active":"2024-09-20T22:19:02.651498+0000","last_peered":"2024-09-20T22:19:02.651498+0000","last_clean":"2024-09-20T22:19:02.651498+0000","last_became_active":"2024-09-20T22:11:30.493275+0000","last_became_peered":"2024-09-20T22:11:30.493275+0000","last_unstale":"2024-09-20T22:19:02.651498+0000","last_undegraded":"2024-09-20T22:19:02.651498+0000","last_fullsized":"2024-09-20T22:19:02.651498+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"416'34","last_scrub_stamp":"2024-09-20T22:17:04.960380+0000","last_deep_scrub":"416'34","last_deep_scrub_stamp":"2024-09-20T22:17:04.960380+0000","last_clean_scrub_stamp":"2024-09-20T22:17:04.960380+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:05.590960+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":488,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.650177+0000","last_change":"2024-09-20T22:16:37.713822+0000","last_active":"2024-09-20T22:19:02.650177+0000","last_peered":"2024-09-20T22:19:02.650177+0000","last_clean":"2024-09-20T22:19:02.650177+0000","last_became_active":"2024-09-20T22:12:52.526551+0000","last_became_peered":"2024-09-20T22:12:52.526551+0000","last_unstale":"2024-09-20T22:19:02.650177+0000","last_undegraded":"2024-09-20T22:19:02.650177+0000","last_fullsized":"2024-09-20T22:19:02.650177+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:16:37.713713+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:16:37.713713+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"71'8","reported_seq":1050,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.649359+0000","last_change":"2024-09-20T22:16:32.744150+0000","last_active":"2024-09-20T22:19:02.649359+0000","last_peered":"2024-09-20T22:19:02.649359+0000","last_clean":"2024-09-20T22:19:02.649359+0000","last_became_active":"2024-09-20T22:02:54.294416+0000","last_became_peered":"2024-09-20T22:02:54.294416+0000","last_unstale":"2024-09-20T22:19:02.649359+0000","last_undegraded":"2024-09-20T22:19:02.649359+0000","last_fullsized":"2024-09-20T22:19:02.649359+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:16:32.744031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:32.744031+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":400,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.652296+0000","last_change":"2024-09-20T22:16:21.904789+0000","last_active":"2024-09-20T22:19:02.652296+0000","last_peered":"2024-09-20T22:19:02.652296+0000","last_clean":"2024-09-20T22:19:02.652296+0000","last_became_active":"2024-09-20T22:12:52.517758+0000","last_became_peered":"2024-09-20T22:12:52.517758+0000","last_unstale":"2024-09-20T22:19:02.652296+0000","last_undegraded":"2024-09-20T22:19:02.652296+0000","last_fullsized":"2024-09-20T22:19:02.652296+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:21.904548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:21.904548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:05.590808+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"75'128","reported_seq":739,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.651988+0000","last_change":"2024-09-20T22:15:52.087940+0000","last_active":"2024-09-20T22:19:02.651988+0000","last_peered":"2024-09-20T22:19:02.651988+0000","last_clean":"2024-09-20T22:19:02.651988+0000","last_became_active":"2024-09-20T22:15:50.622954+0000","last_became_peered":"2024-09-20T22:15:50.622954+0000","last_unstale":"2024-09-20T22:19:02.651988+0000","last_undegraded":"2024-09-20T22:19:02.651988+0000","last_fullsized":"2024-09-20T22:19:02.651988+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:15:52.087707+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:15:52.087707+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:05.590782+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":911,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.630741+0000","last_change":"2024-09-20T22:18:00.786710+0000","last_active":"2024-09-20T22:19:06.630741+0000","last_peered":"2024-09-20T22:19:06.630741+0000","last_clean":"2024-09-20T22:19:06.630741+0000","last_became_active":"2024-09-20T22:12:52.516989+0000","last_became_peered":"2024-09-20T22:12:52.516989+0000","last_unstale":"2024-09-20T22:19:06.630741+0000","last_undegraded":"2024-09-20T22:19:06.630741+0000","last_fullsized":"2024-09-20T22:19:06.630741+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:00.786604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:00.786604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:16.514903+0000","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"328'12","reported_seq":912,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.633172+0000","last_change":"2024-09-20T22:18:05.655205+0000","last_active":"2024-09-20T22:19:06.633172+0000","last_peered":"2024-09-20T22:19:06.633172+0000","last_clean":"2024-09-20T22:19:06.633172+0000","last_became_active":"2024-09-20T22:10:48.922561+0000","last_became_peered":"2024-09-20T22:10:48.922561+0000","last_unstale":"2024-09-20T22:19:06.633172+0000","last_undegraded":"2024-09-20T22:19:06.633172+0000","last_fullsized":"2024-09-20T22:19:06.633172+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'12","last_scrub_stamp":"2024-09-20T22:18:05.654704+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:05.654704+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:26.318994+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":396,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.652192+0000","last_change":"2024-09-20T22:18:36.439122+0000","last_active":"2024-09-20T22:19:02.652192+0000","last_peered":"2024-09-20T22:19:02.652192+0000","last_clean":"2024-09-20T22:19:02.652192+0000","last_became_active":"2024-09-20T22:11:30.487322+0000","last_became_peered":"2024-09-20T22:11:30.487322+0000","last_unstale":"2024-09-20T22:19:02.652192+0000","last_undegraded":"2024-09-20T22:19:02.652192+0000","last_fullsized":"2024-09-20T22:19:02.652192+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:36.439030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:36.439030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:49.748224+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":841,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.650461+0000","last_change":"2024-09-20T22:17:02.792431+0000","last_active":"2024-09-20T22:19:02.650461+0000","last_peered":"2024-09-20T22:19:02.650461+0000","last_clean":"2024-09-20T22:19:02.650461+0000","last_became_active":"2024-09-20T22:02:56.349538+0000","last_became_peered":"2024-09-20T22:02:56.349538+0000","last_unstale":"2024-09-20T22:19:02.650461+0000","last_undegraded":"2024-09-20T22:19:02.650461+0000","last_fullsized":"2024-09-20T22:19:02.650461+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:02.792356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:54.359586+0000","last_clean_scrub_stamp":"2024-09-20T22:17:02.792356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"75'128","reported_seq":885,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.631414+0000","last_change":"2024-09-20T22:18:07.153362+0000","last_active":"2024-09-20T22:19:06.631414+0000","last_peered":"2024-09-20T22:19:06.631414+0000","last_clean":"2024-09-20T22:19:06.631414+0000","last_became_active":"2024-09-20T22:15:50.615607+0000","last_became_peered":"2024-09-20T22:15:50.615607+0000","last_unstale":"2024-09-20T22:19:06.631414+0000","last_undegraded":"2024-09-20T22:19:06.631414+0000","last_fullsized":"2024-09-20T22:19:06.631414+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:18:07.153103+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:18:07.153103+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:31.527749+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"328'15","reported_seq":334,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.641338+0000","last_change":"2024-09-20T22:17:39.086388+0000","last_active":"2024-09-20T22:19:06.641338+0000","last_peered":"2024-09-20T22:19:06.641338+0000","last_clean":"2024-09-20T22:19:06.641338+0000","last_became_active":"2024-09-20T22:12:52.527106+0000","last_became_peered":"2024-09-20T22:12:52.527106+0000","last_unstale":"2024-09-20T22:19:06.641338+0000","last_undegraded":"2024-09-20T22:19:06.641338+0000","last_fullsized":"2024-09-20T22:19:06.641338+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:17:39.086307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:39.086307+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.753613+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":861,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.649561+0000","last_change":"2024-09-20T22:17:10.768641+0000","last_active":"2024-09-20T22:19:02.649561+0000","last_peered":"2024-09-20T22:19:02.649561+0000","last_clean":"2024-09-20T22:19:02.649561+0000","last_became_active":"2024-09-20T22:02:50.108566+0000","last_became_peered":"2024-09-20T22:02:50.108566+0000","last_unstale":"2024-09-20T22:19:02.649561+0000","last_undegraded":"2024-09-20T22:19:02.649561+0000","last_fullsized":"2024-09-20T22:19:02.649561+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:10.768513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:10.768513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":501,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.633417+0000","last_change":"2024-09-20T22:18:36.302230+0000","last_active":"2024-09-20T22:19:06.633417+0000","last_peered":"2024-09-20T22:19:06.633417+0000","last_clean":"2024-09-20T22:19:06.633417+0000","last_became_active":"2024-09-20T22:12:33.545506+0000","last_became_peered":"2024-09-20T22:12:33.545506+0000","last_unstale":"2024-09-20T22:19:06.633417+0000","last_undegraded":"2024-09-20T22:19:06.633417+0000","last_fullsized":"2024-09-20T22:19:06.633417+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:36.302138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_clean_scrub_stamp":"2024-09-20T22:18:36.302138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:51.450300+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":880,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.647167+0000","last_change":"2024-09-20T22:18:03.970869+0000","last_active":"2024-09-20T22:19:02.647167+0000","last_peered":"2024-09-20T22:19:02.647167+0000","last_clean":"2024-09-20T22:19:02.647167+0000","last_became_active":"2024-09-20T22:02:56.298998+0000","last_became_peered":"2024-09-20T22:02:56.298998+0000","last_unstale":"2024-09-20T22:19:02.647167+0000","last_undegraded":"2024-09-20T22:19:02.647167+0000","last_fullsized":"2024-09-20T22:19:02.647167+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:03.970735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:26.862279+0000","last_clean_scrub_stamp":"2024-09-20T22:18:03.970735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:26.723590+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"75'128","reported_seq":747,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.651358+0000","last_change":"2024-09-20T22:18:07.509064+0000","last_active":"2024-09-20T22:19:02.651358+0000","last_peered":"2024-09-20T22:19:02.651358+0000","last_clean":"2024-09-20T22:19:02.651358+0000","last_became_active":"2024-09-20T22:15:50.574865+0000","last_became_peered":"2024-09-20T22:15:50.574865+0000","last_unstale":"2024-09-20T22:19:02.651358+0000","last_undegraded":"2024-09-20T22:19:02.651358+0000","last_fullsized":"2024-09-20T22:19:02.651358+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:18:07.508939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:18:07.508939+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:33.379356+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"328'15","reported_seq":415,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.652613+0000","last_change":"2024-09-20T22:18:39.362251+0000","last_active":"2024-09-20T22:19:02.652613+0000","last_peered":"2024-09-20T22:19:02.652613+0000","last_clean":"2024-09-20T22:19:02.652613+0000","last_became_active":"2024-09-20T22:11:30.493335+0000","last_became_peered":"2024-09-20T22:11:30.493335+0000","last_unstale":"2024-09-20T22:19:02.652613+0000","last_undegraded":"2024-09-20T22:19:02.652613+0000","last_fullsized":"2024-09-20T22:19:02.652613+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:18:39.362109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:39.362109+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:48.789111+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":315,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.640419+0000","last_change":"2024-09-20T22:16:27.938805+0000","last_active":"2024-09-20T22:19:06.640419+0000","last_peered":"2024-09-20T22:19:06.640419+0000","last_clean":"2024-09-20T22:19:06.640419+0000","last_became_active":"2024-09-20T22:12:52.524149+0000","last_became_peered":"2024-09-20T22:12:52.524149+0000","last_unstale":"2024-09-20T22:19:06.640419+0000","last_undegraded":"2024-09-20T22:19:06.640419+0000","last_fullsized":"2024-09-20T22:19:06.640419+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:27.938677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:56.538825+0000","last_clean_scrub_stamp":"2024-09-20T22:16:27.938677+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.753301+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"315'11","reported_seq":436,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.651085+0000","last_change":"2024-09-20T22:17:05.784312+0000","last_active":"2024-09-20T22:19:02.651085+0000","last_peered":"2024-09-20T22:19:02.651085+0000","last_clean":"2024-09-20T22:19:02.651085+0000","last_became_active":"2024-09-20T22:12:33.544098+0000","last_became_peered":"2024-09-20T22:12:33.544098+0000","last_unstale":"2024-09-20T22:19:02.651085+0000","last_undegraded":"2024-09-20T22:19:02.651085+0000","last_fullsized":"2024-09-20T22:19:02.651085+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:17:05.784212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:05.784212+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":861,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.637169+0000","last_change":"2024-09-20T22:18:03.323718+0000","last_active":"2024-09-20T22:19:02.637169+0000","last_peered":"2024-09-20T22:19:02.637169+0000","last_clean":"2024-09-20T22:19:02.637169+0000","last_became_active":"2024-09-20T22:11:30.484731+0000","last_became_peered":"2024-09-20T22:11:30.484731+0000","last_unstale":"2024-09-20T22:19:02.637169+0000","last_undegraded":"2024-09-20T22:19:02.637169+0000","last_fullsized":"2024-09-20T22:19:02.637169+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:03.323607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:03.323607+0000","last_clean_scrub_stamp":"2024-09-20T22:18:03.323607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:15.835890+0000","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"75'128","reported_seq":773,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.639823+0000","last_change":"2024-09-20T22:19:05.735170+0000","last_active":"2024-09-20T22:19:06.639823+0000","last_peered":"2024-09-20T22:19:06.639823+0000","last_clean":"2024-09-20T22:19:06.639823+0000","last_became_active":"2024-09-20T22:15:50.612544+0000","last_became_peered":"2024-09-20T22:15:50.612544+0000","last_unstale":"2024-09-20T22:19:06.639823+0000","last_undegraded":"2024-09-20T22:19:06.639823+0000","last_fullsized":"2024-09-20T22:19:06.639823+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:05.735058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:19:05.735058+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:27.569785+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"328'20","reported_seq":344,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.631992+0000","last_change":"2024-09-20T22:16:37.280708+0000","last_active":"2024-09-20T22:19:06.631992+0000","last_peered":"2024-09-20T22:19:06.631992+0000","last_clean":"2024-09-20T22:19:06.631992+0000","last_became_active":"2024-09-20T22:12:44.631986+0000","last_became_peered":"2024-09-20T22:12:44.631986+0000","last_unstale":"2024-09-20T22:19:06.631992+0000","last_undegraded":"2024-09-20T22:19:06.631992+0000","last_fullsized":"2024-09-20T22:19:06.631992+0000","mapping_epoch":318,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":319,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:16:37.280566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:37.280566+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":853,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.650071+0000","last_change":"2024-09-20T22:16:31.755794+0000","last_active":"2024-09-20T22:19:02.650071+0000","last_peered":"2024-09-20T22:19:02.650071+0000","last_clean":"2024-09-20T22:19:02.650071+0000","last_became_active":"2024-09-20T22:02:50.112602+0000","last_became_peered":"2024-09-20T22:02:50.112602+0000","last_unstale":"2024-09-20T22:19:02.650071+0000","last_undegraded":"2024-09-20T22:19:02.650071+0000","last_fullsized":"2024-09-20T22:19:02.650071+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:31.755589+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:31.755589+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"326'14","reported_seq":512,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.652421+0000","last_change":"2024-09-20T22:16:55.917269+0000","last_active":"2024-09-20T22:19:02.652421+0000","last_peered":"2024-09-20T22:19:02.652421+0000","last_clean":"2024-09-20T22:19:02.652421+0000","last_became_active":"2024-09-20T22:12:52.518770+0000","last_became_peered":"2024-09-20T22:12:52.518770+0000","last_unstale":"2024-09-20T22:19:02.652421+0000","last_undegraded":"2024-09-20T22:19:02.652421+0000","last_fullsized":"2024-09-20T22:19:02.652421+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'14","last_scrub_stamp":"2024-09-20T22:16:55.917159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:55.917159+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:05.590933+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":315,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.640819+0000","last_change":"2024-09-20T22:17:06.828671+0000","last_active":"2024-09-20T22:19:06.640819+0000","last_peered":"2024-09-20T22:19:06.640819+0000","last_clean":"2024-09-20T22:19:06.640819+0000","last_became_active":"2024-09-20T22:12:52.523781+0000","last_became_peered":"2024-09-20T22:12:52.523781+0000","last_unstale":"2024-09-20T22:19:06.640819+0000","last_undegraded":"2024-09-20T22:19:06.640819+0000","last_fullsized":"2024-09-20T22:19:06.640819+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:06.828504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:06.828504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.753439+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.16","version":"75'128","reported_seq":663,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.633263+0000","last_change":"2024-09-20T22:19:04.305743+0000","last_active":"2024-09-20T22:19:06.633263+0000","last_peered":"2024-09-20T22:19:06.633263+0000","last_clean":"2024-09-20T22:19:06.633263+0000","last_became_active":"2024-09-20T22:17:44.621596+0000","last_became_peered":"2024-09-20T22:17:44.621596+0000","last_unstale":"2024-09-20T22:19:06.633263+0000","last_undegraded":"2024-09-20T22:19:06.633263+0000","last_fullsized":"2024-09-20T22:19:06.633263+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:04.305501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:19:04.305501+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:06.675327+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"328'19","reported_seq":909,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.636543+0000","last_change":"2024-09-20T22:16:28.533330+0000","last_active":"2024-09-20T22:19:02.636543+0000","last_peered":"2024-09-20T22:19:02.636543+0000","last_clean":"2024-09-20T22:19:02.636543+0000","last_became_active":"2024-09-20T22:08:22.139581+0000","last_became_peered":"2024-09-20T22:08:22.139581+0000","last_unstale":"2024-09-20T22:19:02.636543+0000","last_undegraded":"2024-09-20T22:19:02.636543+0000","last_fullsized":"2024-09-20T22:19:02.636543+0000","mapping_epoch":187,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":188,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:16:28.533201+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:28.533201+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.323573+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":861,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.637102+0000","last_change":"2024-09-20T22:16:56.523856+0000","last_active":"2024-09-20T22:19:02.637102+0000","last_peered":"2024-09-20T22:19:02.637102+0000","last_clean":"2024-09-20T22:19:02.637102+0000","last_became_active":"2024-09-20T22:02:50.108661+0000","last_became_peered":"2024-09-20T22:02:50.108661+0000","last_unstale":"2024-09-20T22:19:02.637102+0000","last_undegraded":"2024-09-20T22:19:02.637102+0000","last_fullsized":"2024-09-20T22:19:02.637102+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:56.523743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:56.523743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.323629+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":57,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.639254+0000","last_change":"2024-09-20T22:18:40.528611+0000","last_active":"2024-09-20T22:19:06.639254+0000","last_peered":"2024-09-20T22:19:06.639254+0000","last_clean":"2024-09-20T22:19:06.639254+0000","last_became_active":"2024-09-20T22:18:13.637189+0000","last_became_peered":"2024-09-20T22:18:13.637189+0000","last_unstale":"2024-09-20T22:19:06.639254+0000","last_undegraded":"2024-09-20T22:19:06.639254+0000","last_fullsized":"2024-09-20T22:19:06.639254+0000","mapping_epoch":443,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":444,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:40.528539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:40.528539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:01.495994+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":703,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.630742+0000","last_change":"2024-09-20T22:16:59.256702+0000","last_active":"2024-09-20T22:19:06.630742+0000","last_peered":"2024-09-20T22:19:06.630742+0000","last_clean":"2024-09-20T22:19:06.630742+0000","last_became_active":"2024-09-20T22:12:52.521497+0000","last_became_peered":"2024-09-20T22:12:52.521497+0000","last_unstale":"2024-09-20T22:19:06.630742+0000","last_undegraded":"2024-09-20T22:19:06.630742+0000","last_fullsized":"2024-09-20T22:19:06.630742+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:59.256587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:59.256587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.17","version":"75'128","reported_seq":369,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.652089+0000","last_change":"2024-09-20T22:17:57.421052+0000","last_active":"2024-09-20T22:19:02.652089+0000","last_peered":"2024-09-20T22:19:02.652089+0000","last_clean":"2024-09-20T22:19:02.652089+0000","last_became_active":"2024-09-20T22:17:44.721755+0000","last_became_peered":"2024-09-20T22:17:44.721755+0000","last_unstale":"2024-09-20T22:19:02.652089+0000","last_undegraded":"2024-09-20T22:19:02.652089+0000","last_fullsized":"2024-09-20T22:19:02.652089+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:57.420935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.420935+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:14.038998+0000","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"328'19","reported_seq":777,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.631563+0000","last_change":"2024-09-20T22:17:03.305508+0000","last_active":"2024-09-20T22:19:06.631563+0000","last_peered":"2024-09-20T22:19:06.631563+0000","last_clean":"2024-09-20T22:19:06.631563+0000","last_became_active":"2024-09-20T22:06:54.843973+0000","last_became_peered":"2024-09-20T22:06:54.843973+0000","last_unstale":"2024-09-20T22:19:06.631563+0000","last_undegraded":"2024-09-20T22:19:06.631563+0000","last_fullsized":"2024-09-20T22:19:06.631563+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:17:03.305256+0000","last_deep_scrub":"291'13","last_deep_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_clean_scrub_stamp":"2024-09-20T22:17:03.305256+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":458,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.650108+0000","last_change":"2024-09-20T22:17:54.900226+0000","last_active":"2024-09-20T22:19:02.650108+0000","last_peered":"2024-09-20T22:19:02.650108+0000","last_clean":"2024-09-20T22:19:02.650108+0000","last_became_active":"2024-09-20T22:10:30.655502+0000","last_became_peered":"2024-09-20T22:10:30.655502+0000","last_unstale":"2024-09-20T22:19:02.650108+0000","last_undegraded":"2024-09-20T22:19:02.650108+0000","last_fullsized":"2024-09-20T22:19:02.650108+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:54.900125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:54.900125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:14.560078+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":323,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.640993+0000","last_change":"2024-09-20T22:19:02.799135+0000","last_active":"2024-09-20T22:19:06.640993+0000","last_peered":"2024-09-20T22:19:06.640993+0000","last_clean":"2024-09-20T22:19:06.640993+0000","last_became_active":"2024-09-20T22:12:52.516953+0000","last_became_peered":"2024-09-20T22:12:52.516953+0000","last_unstale":"2024-09-20T22:19:06.640993+0000","last_undegraded":"2024-09-20T22:19:06.640993+0000","last_fullsized":"2024-09-20T22:19:06.640993+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:02.798998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:02.798998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:28.834506+0000","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":315,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.639259+0000","last_change":"2024-09-20T22:17:09.839341+0000","last_active":"2024-09-20T22:19:06.639259+0000","last_peered":"2024-09-20T22:19:06.639259+0000","last_clean":"2024-09-20T22:19:06.639259+0000","last_became_active":"2024-09-20T22:12:52.514247+0000","last_became_peered":"2024-09-20T22:12:52.514247+0000","last_unstale":"2024-09-20T22:19:06.639259+0000","last_undegraded":"2024-09-20T22:19:06.639259+0000","last_fullsized":"2024-09-20T22:19:06.639259+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:09.839241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:09.839241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.753557+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"75'128","reported_seq":912,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.646866+0000","last_change":"2024-09-20T22:17:04.909914+0000","last_active":"2024-09-20T22:19:02.646866+0000","last_peered":"2024-09-20T22:19:02.646866+0000","last_clean":"2024-09-20T22:19:02.646866+0000","last_became_active":"2024-09-20T22:15:50.611421+0000","last_became_peered":"2024-09-20T22:15:50.611421+0000","last_unstale":"2024-09-20T22:19:02.646866+0000","last_undegraded":"2024-09-20T22:19:02.646866+0000","last_fullsized":"2024-09-20T22:19:02.646866+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:04.909798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:04.909798+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"328'19","reported_seq":778,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.649424+0000","last_change":"2024-09-20T22:16:21.625991+0000","last_active":"2024-09-20T22:19:02.649424+0000","last_peered":"2024-09-20T22:19:02.649424+0000","last_clean":"2024-09-20T22:19:02.649424+0000","last_became_active":"2024-09-20T22:12:33.545199+0000","last_became_peered":"2024-09-20T22:12:33.545199+0000","last_unstale":"2024-09-20T22:19:02.649424+0000","last_undegraded":"2024-09-20T22:19:02.649424+0000","last_fullsized":"2024-09-20T22:19:02.649424+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:16:21.625915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:16:21.625915+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":399,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.652086+0000","last_change":"2024-09-20T22:17:55.398271+0000","last_active":"2024-09-20T22:19:02.652086+0000","last_peered":"2024-09-20T22:19:02.652086+0000","last_clean":"2024-09-20T22:19:02.652086+0000","last_became_active":"2024-09-20T22:11:30.493598+0000","last_became_peered":"2024-09-20T22:11:30.493598+0000","last_unstale":"2024-09-20T22:19:02.652086+0000","last_undegraded":"2024-09-20T22:19:02.652086+0000","last_fullsized":"2024-09-20T22:19:02.652086+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:55.398175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_clean_scrub_stamp":"2024-09-20T22:17:55.398175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:13.951899+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":853,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.649449+0000","last_change":"2024-09-20T22:17:08.799282+0000","last_active":"2024-09-20T22:19:02.649449+0000","last_peered":"2024-09-20T22:19:02.649449+0000","last_clean":"2024-09-20T22:19:02.649449+0000","last_became_active":"2024-09-20T22:02:54.292708+0000","last_became_peered":"2024-09-20T22:02:54.292708+0000","last_unstale":"2024-09-20T22:19:02.649449+0000","last_undegraded":"2024-09-20T22:19:02.649449+0000","last_fullsized":"2024-09-20T22:19:02.649449+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:08.798598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:08.798598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":312,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.641201+0000","last_change":"2024-09-20T22:18:32.424564+0000","last_active":"2024-09-20T22:19:06.641201+0000","last_peered":"2024-09-20T22:19:06.641201+0000","last_clean":"2024-09-20T22:19:06.641201+0000","last_became_active":"2024-09-20T22:12:57.883998+0000","last_became_peered":"2024-09-20T22:12:57.883998+0000","last_unstale":"2024-09-20T22:19:06.641201+0000","last_undegraded":"2024-09-20T22:19:06.641201+0000","last_fullsized":"2024-09-20T22:19:06.641201+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:32.424321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:32.424321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:54.741463+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.15","version":"75'128","reported_seq":368,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.649921+0000","last_change":"2024-09-20T22:17:56.859515+0000","last_active":"2024-09-20T22:19:02.649921+0000","last_peered":"2024-09-20T22:19:02.649921+0000","last_clean":"2024-09-20T22:19:02.649921+0000","last_became_active":"2024-09-20T22:17:44.719101+0000","last_became_peered":"2024-09-20T22:17:44.719101+0000","last_unstale":"2024-09-20T22:19:02.649921+0000","last_undegraded":"2024-09-20T22:19:02.649921+0000","last_fullsized":"2024-09-20T22:19:02.649921+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.859373+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.859373+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:07.658533+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"453'10","reported_seq":889,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.627193+0000","last_change":"2024-09-20T22:18:39.589028+0000","last_active":"2024-09-20T22:19:06.627193+0000","last_peered":"2024-09-20T22:19:06.627193+0000","last_clean":"2024-09-20T22:19:06.627193+0000","last_became_active":"2024-09-20T22:06:54.847633+0000","last_became_peered":"2024-09-20T22:06:54.847633+0000","last_unstale":"2024-09-20T22:19:06.627193+0000","last_undegraded":"2024-09-20T22:19:06.627193+0000","last_fullsized":"2024-09-20T22:19:06.627193+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"453'10","last_scrub_stamp":"2024-09-20T22:18:39.588954+0000","last_deep_scrub":"453'10","last_deep_scrub_stamp":"2024-09-20T22:18:39.588954+0000","last_clean_scrub_stamp":"2024-09-20T22:18:39.588954+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:20:03.024197+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":315,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.643498+0000","last_change":"2024-09-20T22:18:09.367123+0000","last_active":"2024-09-20T22:19:06.643498+0000","last_peered":"2024-09-20T22:19:06.643498+0000","last_clean":"2024-09-20T22:19:06.643498+0000","last_became_active":"2024-09-20T22:12:52.517510+0000","last_became_peered":"2024-09-20T22:12:52.517510+0000","last_unstale":"2024-09-20T22:19:06.643498+0000","last_undegraded":"2024-09-20T22:19:06.643498+0000","last_fullsized":"2024-09-20T22:19:06.643498+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:09.366999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:09.366999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:15.480279+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":886,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.631223+0000","last_change":"2024-09-20T22:18:44.105994+0000","last_active":"2024-09-20T22:19:06.631223+0000","last_peered":"2024-09-20T22:19:06.631223+0000","last_clean":"2024-09-20T22:19:06.631223+0000","last_became_active":"2024-09-20T22:02:54.289517+0000","last_became_peered":"2024-09-20T22:02:54.289517+0000","last_unstale":"2024-09-20T22:19:06.631223+0000","last_undegraded":"2024-09-20T22:19:06.631223+0000","last_fullsized":"2024-09-20T22:19:06.631223+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.105874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.105874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:49.805315+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":849,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.649832+0000","last_change":"2024-09-20T22:17:33.935280+0000","last_active":"2024-09-20T22:19:02.649832+0000","last_peered":"2024-09-20T22:19:02.649832+0000","last_clean":"2024-09-20T22:19:02.649832+0000","last_became_active":"2024-09-20T22:02:56.300286+0000","last_became_peered":"2024-09-20T22:02:56.300286+0000","last_unstale":"2024-09-20T22:19:02.649832+0000","last_undegraded":"2024-09-20T22:19:02.649832+0000","last_fullsized":"2024-09-20T22:19:02.649832+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:33.935154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_clean_scrub_stamp":"2024-09-20T22:17:33.935154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1a","version":"75'128","reported_seq":368,"reported_epoch":462,"state":"active+clean","last_fresh":"2024-09-20T22:19:02.635981+0000","last_change":"2024-09-20T22:17:56.396410+0000","last_active":"2024-09-20T22:19:02.635981+0000","last_peered":"2024-09-20T22:19:02.635981+0000","last_clean":"2024-09-20T22:19:02.635981+0000","last_became_active":"2024-09-20T22:17:44.723076+0000","last_became_peered":"2024-09-20T22:17:44.723076+0000","last_unstale":"2024-09-20T22:19:02.635981+0000","last_undegraded":"2024-09-20T22:19:02.635981+0000","last_fullsized":"2024-09-20T22:19:02.635981+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.396173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.396173+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:16.999042+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":748,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.632233+0000","last_change":"2024-09-20T22:18:37.293672+0000","last_active":"2024-09-20T22:19:06.632233+0000","last_peered":"2024-09-20T22:19:06.632233+0000","last_clean":"2024-09-20T22:19:06.632233+0000","last_became_active":"2024-09-20T22:06:54.828469+0000","last_became_peered":"2024-09-20T22:06:54.828469+0000","last_unstale":"2024-09-20T22:19:06.632233+0000","last_undegraded":"2024-09-20T22:19:06.632233+0000","last_fullsized":"2024-09-20T22:19:06.632233+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:37.293600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:37.293600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:44.682563+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":315,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.639633+0000","last_change":"2024-09-20T22:16:56.775739+0000","last_active":"2024-09-20T22:19:06.639633+0000","last_peered":"2024-09-20T22:19:06.639633+0000","last_clean":"2024-09-20T22:19:06.639633+0000","last_became_active":"2024-09-20T22:12:52.518076+0000","last_became_peered":"2024-09-20T22:12:52.518076+0000","last_unstale":"2024-09-20T22:19:06.639633+0000","last_undegraded":"2024-09-20T22:19:06.639633+0000","last_fullsized":"2024-09-20T22:19:06.639633+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:56.775388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:56.775388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.753389+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,1],"acting":[0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"328'19","reported_seq":907,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.631659+0000","last_change":"2024-09-20T22:17:29.804302+0000","last_active":"2024-09-20T22:19:06.631659+0000","last_peered":"2024-09-20T22:19:06.631659+0000","last_clean":"2024-09-20T22:19:06.631659+0000","last_became_active":"2024-09-20T22:06:54.828652+0000","last_became_peered":"2024-09-20T22:06:54.828652+0000","last_unstale":"2024-09-20T22:19:06.631659+0000","last_undegraded":"2024-09-20T22:19:06.631659+0000","last_fullsized":"2024-09-20T22:19:06.631659+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:17:29.804188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:29.804188+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":316,"reported_epoch":463,"state":"active+clean","last_fresh":"2024-09-20T22:19:06.641425+0000","last_change":"2024-09-20T22:17:10.840330+0000","last_active":"2024-09-20T22:19:06.641425+0000","last_peered":"2024-09-20T22:19:06.641425+0000","last_clean":"2024-09-20T22:19:06.641425+0000","last_became_active":"2024-09-20T22:12:52.524425+0000","last_became_peered":"2024-09-20T22:12:52.524425+0000","last_unstale":"2024-09-20T22:19:06.641425+0000","last_undegraded":"2024-09-20T22:19:06.641425+0000","last_fullsized":"2024-09-20T22:19:06.641425+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:10.839899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:10.839899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:19:06.753583+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":2175664128,"num_objects":33198,"num_object_clones":0,"num_object_copies":99594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33198,"num_write_kb":2124672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":6540558336,"data_stored":6540558336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":33198,"ondisk_log_size":33198,"up":48,"acting":48,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1116,"num_read_kb":873,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":3410,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":745,"ondisk_log_size":745,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1797,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":951,"num_read_kb":946,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":778,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":31,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3968,"ondisk_log_size":3968,"up":93,"acting":93,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813136071,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1387588,"kb_used_data":702516,"kb_used_omap":1,"kb_used_meta":685054,"kb_avail":92357564,"statfs":{"total":95995035648,"available":94574145536,"internally_reserved":0,"allocated":719376384,"data_stored":718919226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":701495754},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430409,"num_pgs":66,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1724796,"kb_used_data":1093864,"kb_used_omap":1,"kb_used_meta":630910,"kb_avail":92020356,"statfs":{"total":95995035648,"available":94228844544,"internally_reserved":0,"allocated":1120116736,"data_stored":1119679527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":646052294},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724751,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1209264,"kb_used_data":488096,"kb_used_omap":1,"kb_used_meta":721150,"kb_avail":92535888,"statfs":{"total":95995035648,"available":94756749312,"internally_reserved":0,"allocated":499810304,"data_stored":499372887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":738458053},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986391,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1084416,"kb_used_data":471536,"kb_used_omap":1,"kb_used_meta":612862,"kb_avail":92660736,"statfs":{"total":95995035648,"available":94884593664,"internally_reserved":0,"allocated":482852864,"data_stored":482398981,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":627571141},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770258068,"num_pgs":74,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1718308,"kb_used_data":1111124,"kb_used_omap":4,"kb_used_meta":607163,"kb_avail":92026844,"statfs":{"total":95995035648,"available":94235488256,"internally_reserved":0,"allocated":1137790976,"data_stored":1137310043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":621735592},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149916,"num_pgs":72,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1734868,"kb_used_data":909572,"kb_used_omap":1,"kb_used_meta":825278,"kb_avail":92010284,"statfs":{"total":95995035648,"available":94218530816,"internally_reserved":0,"allocated":931401728,"data_stored":930928767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":845085127},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411553,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1404848,"kb_used_data":746144,"kb_used_omap":1,"kb_used_meta":658686,"kb_avail":92340304,"statfs":{"total":95995035648,"available":94556471296,"internally_reserved":0,"allocated":764051456,"data_stored":763614048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":674494917},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738604,"num_pgs":60,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1499728,"kb_used_data":660540,"kb_used_omap":1,"kb_used_meta":839166,"kb_avail":92245424,"statfs":{"total":95995035648,"available":94459314176,"internally_reserved":0,"allocated":676392960,"data_stored":675935495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":859306437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":700448768,"data_stored":700448768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1147863040,"data_stored":1147863040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":788791296,"data_stored":788791296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":949747712,"data_stored":949747712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":501940224,"data_stored":501940224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":523763712,"data_stored":523763712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1165950976,"data_stored":1165950976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":762052608,"data_stored":762052608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:19:11.334 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool set unique_pool_4 pgp_num 16 2024-09-20T22:19:11.407 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:11.407 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:11.408 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:11.426 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T22:19:11.508 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:11.547 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:11.547 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:11.548 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:11.638 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:19:11.648 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:11.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:11.702 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:11.702 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:11.803 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:11.833 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:11.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:11.834 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:11.849 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:19:11.934 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:11.942 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:19:11.974 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:11.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:11.976 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:11.980 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:11 smithi067 ceph-mon[33511]: 1.4 scrub starts 2024-09-20T22:19:11.980 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:11 smithi067 ceph-mon[33511]: 1.4 scrub ok 2024-09-20T22:19:11.980 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:11 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1943561983' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:11.981 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:11 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:11.981 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:11 smithi067 ceph-mon[33511]: 6.e scrub starts 2024-09-20T22:19:11.981 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:11 smithi067 ceph-mon[33511]: 6.e scrub ok 2024-09-20T22:19:11.981 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:11 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:19:11.981 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:11 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:19:11.981 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:11 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:19:11.981 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:11 smithi067 ceph-mon[33511]: osdmap e465: 8 total, 8 up, 8 in 2024-09-20T22:19:11.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[29694]: 1.4 scrub starts 2024-09-20T22:19:11.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[29694]: 1.4 scrub ok 2024-09-20T22:19:11.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1943561983' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:11.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:11.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[29694]: 6.e scrub starts 2024-09-20T22:19:11.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[29694]: 6.e scrub ok 2024-09-20T22:19:11.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:19:11.982 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:19:11.982 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:19:11.982 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[29694]: osdmap e465: 8 total, 8 up, 8 in 2024-09-20T22:19:11.982 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[40290]: 1.4 scrub starts 2024-09-20T22:19:11.982 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[40290]: 1.4 scrub ok 2024-09-20T22:19:11.982 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1943561983' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:11.982 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:11.982 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[40290]: 6.e scrub starts 2024-09-20T22:19:11.982 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[40290]: 6.e scrub ok 2024-09-20T22:19:11.982 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:19:11.982 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:19:11.983 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:19:11.983 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:11 smithi060 ceph-mon[40290]: osdmap e465: 8 total, 8 up, 8 in 2024-09-20T22:19:12.077 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:12.098 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 19 16 41064 41048 134.998 102.938 0.00261854 0.00739599 2024-09-20T22:19:12.103 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:12.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:12.104 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:12.205 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:12.228 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:12.228 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:12.228 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:12.329 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:12.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:12.373 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:12.373 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:12.474 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:12.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:12.509 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:12.509 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:12.609 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:12.652 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:12.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:12.653 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:12.753 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:12.763 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:19:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:19:12.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:12 smithi067 ceph-mon[33511]: 1.f scrub starts 2024-09-20T22:19:12.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:12 smithi067 ceph-mon[33511]: 1.f scrub ok 2024-09-20T22:19:12.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:12 smithi067 ceph-mon[33511]: 3.2 scrub starts 2024-09-20T22:19:12.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:12 smithi067 ceph-mon[33511]: 3.2 scrub ok 2024-09-20T22:19:12.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:12 smithi067 ceph-mon[33511]: 5.d scrub starts 2024-09-20T22:19:12.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:12 smithi067 ceph-mon[33511]: 5.d scrub ok 2024-09-20T22:19:12.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:12 smithi067 ceph-mon[33511]: 4.d scrub starts 2024-09-20T22:19:12.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:12 smithi067 ceph-mon[33511]: 4.d scrub ok 2024-09-20T22:19:12.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:12 smithi067 ceph-mon[33511]: pgmap v871: 178 pgs: 178 active+clean; 2.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 122 MiB/s wr, 1.95k op/s 2024-09-20T22:19:12.782 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:12.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:12.783 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:12.883 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:12.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:12.919 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:12.919 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:12.980 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:19:12.981 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:19:12.981 INFO:teuthology.orchestra.run.smithi067.stdout: "complaint_time": 30, 2024-09-20T22:19:12.981 INFO:teuthology.orchestra.run.smithi067.stdout: "num_blocked_ops": 0 2024-09-20T22:19:12.981 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:19:13.020 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:13.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:12 smithi060 ceph-mon[40290]: 1.f scrub starts 2024-09-20T22:19:13.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:12 smithi060 ceph-mon[40290]: 1.f scrub ok 2024-09-20T22:19:13.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:12 smithi060 ceph-mon[40290]: 3.2 scrub starts 2024-09-20T22:19:13.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:12 smithi060 ceph-mon[40290]: 3.2 scrub ok 2024-09-20T22:19:13.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:12 smithi060 ceph-mon[40290]: 5.d scrub starts 2024-09-20T22:19:13.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:12 smithi060 ceph-mon[40290]: 5.d scrub ok 2024-09-20T22:19:13.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:12 smithi060 ceph-mon[40290]: 4.d scrub starts 2024-09-20T22:19:13.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:12 smithi060 ceph-mon[40290]: 4.d scrub ok 2024-09-20T22:19:13.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:12 smithi060 ceph-mon[40290]: pgmap v871: 178 pgs: 178 active+clean; 2.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 122 MiB/s wr, 1.95k op/s 2024-09-20T22:19:13.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:12 smithi060 ceph-mon[29694]: 1.f scrub starts 2024-09-20T22:19:13.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:12 smithi060 ceph-mon[29694]: 1.f scrub ok 2024-09-20T22:19:13.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:12 smithi060 ceph-mon[29694]: 3.2 scrub starts 2024-09-20T22:19:13.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:12 smithi060 ceph-mon[29694]: 3.2 scrub ok 2024-09-20T22:19:13.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:12 smithi060 ceph-mon[29694]: 5.d scrub starts 2024-09-20T22:19:13.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:12 smithi060 ceph-mon[29694]: 5.d scrub ok 2024-09-20T22:19:13.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:12 smithi060 ceph-mon[29694]: 4.d scrub starts 2024-09-20T22:19:13.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:12 smithi060 ceph-mon[29694]: 4.d scrub ok 2024-09-20T22:19:13.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:12 smithi060 ceph-mon[29694]: pgmap v871: 178 pgs: 178 active+clean; 2.3 GiB data, 12 GiB used, 703 GiB / 715 GiB avail; 853 B/s rd, 122 MiB/s wr, 1.95k op/s 2024-09-20T22:19:13.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:13.058 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:13.058 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:13.098 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:19:13.097993+0000 min lat: 0.00118665 max lat: 0.266731 avg lat: 0.00756306 2024-09-20T22:19:13.098 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:19:13.098 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 20 16 42306 42290 132.129 77.625 0.00383643 0.00756306 2024-09-20T22:19:13.159 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:13.199 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:13.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:13.201 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:13.301 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:13.332 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:13.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:13.333 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:13.363 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:13.410 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:13.434 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:13.464 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:13.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:13.465 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:13.566 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:13.597 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:13.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:13.598 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:13.699 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:13.728 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:13.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:13.729 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:13.829 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:13.831 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T22:19:13.891 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:13 smithi060 ceph-mon[29694]: 3.a scrub starts 2024-09-20T22:19:13.891 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:13 smithi060 ceph-mon[29694]: 3.a scrub ok 2024-09-20T22:19:13.891 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:13 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:19:13.891 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:13 smithi060 ceph-mon[29694]: 6.9 scrub starts 2024-09-20T22:19:13.891 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:13 smithi060 ceph-mon[29694]: 6.9 scrub ok 2024-09-20T22:19:13.892 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:13 smithi060 ceph-mon[29694]: 6.18 scrub starts 2024-09-20T22:19:13.892 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:13 smithi060 ceph-mon[29694]: 6.18 scrub ok 2024-09-20T22:19:13.892 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:13 smithi060 ceph-mon[40290]: 3.a scrub starts 2024-09-20T22:19:13.892 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:13 smithi060 ceph-mon[40290]: 3.a scrub ok 2024-09-20T22:19:13.892 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:13 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:19:13.892 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:13 smithi060 ceph-mon[40290]: 6.9 scrub starts 2024-09-20T22:19:13.892 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:13 smithi060 ceph-mon[40290]: 6.9 scrub ok 2024-09-20T22:19:13.892 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:13 smithi060 ceph-mon[40290]: 6.18 scrub starts 2024-09-20T22:19:13.892 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:13 smithi060 ceph-mon[40290]: 6.18 scrub ok 2024-09-20T22:19:13.906 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:13.906 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:13.907 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:14.007 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:13 smithi067 ceph-mon[33511]: 3.a scrub starts 2024-09-20T22:19:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:13 smithi067 ceph-mon[33511]: 3.a scrub ok 2024-09-20T22:19:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:13 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:19:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:13 smithi067 ceph-mon[33511]: 6.9 scrub starts 2024-09-20T22:19:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:13 smithi067 ceph-mon[33511]: 6.9 scrub ok 2024-09-20T22:19:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:13 smithi067 ceph-mon[33511]: 6.18 scrub starts 2024-09-20T22:19:14.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:13 smithi067 ceph-mon[33511]: 6.18 scrub ok 2024-09-20T22:19:14.052 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:14.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:14.053 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:14.099 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 21 16 44000 43984 130.872 105.875 0.0254721 0.00763399 2024-09-20T22:19:14.155 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:14.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:14.193 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:14.193 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:14.280 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:19:14.294 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:14.336 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:14.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:14.337 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:14.438 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:14.477 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:14.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:14.479 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:14.579 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:14.623 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:14.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:14.625 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:14.723 INFO:teuthology.orchestra.run.smithi060.stderr:set pool 11 pgp_num to 16 2024-09-20T22:19:14.726 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:14.739 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:19:14.739 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:19:14.763 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:14.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:14.764 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:14.784 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:19:14.784 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:19:14.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[40290]: 6.7 scrub starts 2024-09-20T22:19:14.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[40290]: 6.7 scrub ok 2024-09-20T22:19:14.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[40290]: 2.1 scrub starts 2024-09-20T22:19:14.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[40290]: 2.1 scrub ok 2024-09-20T22:19:14.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[40290]: 3.12 scrub starts 2024-09-20T22:19:14.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[40290]: 3.12 scrub ok 2024-09-20T22:19:14.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[40290]: pgmap v872: 178 pgs: 178 active+clean; 2.5 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 665 B/s rd, 107 MiB/s wr, 1.71k op/s 2024-09-20T22:19:14.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[40290]: 4.1 scrub starts 2024-09-20T22:19:14.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[40290]: 4.1 scrub ok 2024-09-20T22:19:14.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1480723543' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-20T22:19:14.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-20T22:19:14.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/328412255' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[29694]: 6.7 scrub starts 2024-09-20T22:19:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[29694]: 6.7 scrub ok 2024-09-20T22:19:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[29694]: 2.1 scrub starts 2024-09-20T22:19:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[29694]: 2.1 scrub ok 2024-09-20T22:19:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[29694]: 3.12 scrub starts 2024-09-20T22:19:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[29694]: 3.12 scrub ok 2024-09-20T22:19:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[29694]: pgmap v872: 178 pgs: 178 active+clean; 2.5 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 665 B/s rd, 107 MiB/s wr, 1.71k op/s 2024-09-20T22:19:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[29694]: 4.1 scrub starts 2024-09-20T22:19:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[29694]: 4.1 scrub ok 2024-09-20T22:19:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1480723543' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-20T22:19:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-20T22:19:14.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:14 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/328412255' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:14.823 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:19:14.823 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:19:14.863 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:19:14.863 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:19:14.865 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:14.894 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:19:14.894 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:19:14.898 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:19:14.898 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:19:14.898 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:19:14.899 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:19:14.899 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.899 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163158 8.11 8:8e0bce89:::benchmark_data_smithi060_111051_object43159:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:19:14.899 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:08.839869+0000", 2024-09-20T22:19:14.899 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 546.05300021799997, 2024-09-20T22:19:14.899 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.65193807500000001, 2024-09-20T22:19:14.899 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:14.899 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:14.899 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:14.899 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:14.899 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:19:14.899 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:19:14.899 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163158 2024-09-20T22:19:14.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.900 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:14.900 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.900 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:14.900 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839869+0000", 2024-09-20T22:19:14.900 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.900 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.900 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:14.900 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839869+0000", 2024-09-20T22:19:14.900 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.900 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.900 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.900 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:14.900 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839870+0000", 2024-09-20T22:19:14.901 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5710000000000001e-06 2024-09-20T22:19:14.901 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.901 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.901 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:14.901 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839876+0000", 2024-09-20T22:19:14.901 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3820000000000003e-06 2024-09-20T22:19:14.901 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.901 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.901 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:14.901 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839890+0000", 2024-09-20T22:19:14.901 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3952e-05 2024-09-20T22:19:14.901 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.901 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.901 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:14.901 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839897+0000", 2024-09-20T22:19:14.902 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4150000000000002e-06 2024-09-20T22:19:14.902 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.902 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.902 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.902 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839956+0000", 2024-09-20T22:19:14.902 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9157999999999998e-05 2024-09-20T22:19:14.902 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.902 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.902 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:14.902 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.839967+0000", 2024-09-20T22:19:14.902 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1019000000000001e-05 2024-09-20T22:19:14.902 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.902 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.902 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.902 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477635+0000", 2024-09-20T22:19:14.903 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.63766765700000005 2024-09-20T22:19:14.903 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.903 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.903 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:14.903 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477737+0000", 2024-09-20T22:19:14.903 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010265899999999999 2024-09-20T22:19:14.903 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.903 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.903 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:19:14.903 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.477939+0000", 2024-09-20T22:19:14.903 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020106900000000001 2024-09-20T22:19:14.903 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.903 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.903 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:14.903 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.482557+0000", 2024-09-20T22:19:14.904 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046186830000000002 2024-09-20T22:19:14.904 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.904 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.904 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.904 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491495+0000", 2024-09-20T22:19:14.904 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0089377699999999994 2024-09-20T22:19:14.904 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.904 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.904 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.904 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491673+0000", 2024-09-20T22:19:14.904 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017837500000000001 2024-09-20T22:19:14.904 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.904 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.904 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:14.905 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491762+0000", 2024-09-20T22:19:14.905 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8555999999999997e-05 2024-09-20T22:19:14.905 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.905 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.905 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:14.905 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491807+0000", 2024-09-20T22:19:14.905 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4808999999999997e-05 2024-09-20T22:19:14.905 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.905 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:14.905 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.905 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.905 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.905 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163289 8.11 8:8c47e2c8:::benchmark_data_smithi060_111051_object43290:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:19:14.905 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:08.954133+0000", 2024-09-20T22:19:14.906 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 545.93873558400003, 2024-09-20T22:19:14.906 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.53808307099999997, 2024-09-20T22:19:14.906 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:14.906 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:14.906 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:14.906 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:14.906 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:19:14.906 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:19:14.906 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163289 2024-09-20T22:19:14.906 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.906 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:14.906 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.906 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:14.907 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954133+0000", 2024-09-20T22:19:14.907 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.907 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.907 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:14.907 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954133+0000", 2024-09-20T22:19:14.907 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.907 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.907 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:14.907 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954134+0000", 2024-09-20T22:19:14.907 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2900000000000002e-07 2024-09-20T22:19:14.907 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.907 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.908 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:14.908 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954138+0000", 2024-09-20T22:19:14.908 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5260000000000002e-06 2024-09-20T22:19:14.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.908 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.908 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:14.908 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954147+0000", 2024-09-20T22:19:14.908 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2010000000000005e-06 2024-09-20T22:19:14.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.908 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.908 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:14.908 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954153+0000", 2024-09-20T22:19:14.908 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0610000000000001e-06 2024-09-20T22:19:14.908 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.908 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.909 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.909 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954209+0000", 2024-09-20T22:19:14.909 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5767999999999999e-05 2024-09-20T22:19:14.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.909 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.909 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:14.909 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.954216+0000", 2024-09-20T22:19:14.909 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8099999999999998e-06 2024-09-20T22:19:14.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.909 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.909 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.909 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.478998+0000", 2024-09-20T22:19:14.909 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.52478124900000001 2024-09-20T22:19:14.909 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.909 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.909 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:14.910 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.479095+0000", 2024-09-20T22:19:14.910 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7125000000000003e-05 2024-09-20T22:19:14.910 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.910 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.910 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:19:14.910 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.479291+0000", 2024-09-20T22:19:14.910 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019642100000000001 2024-09-20T22:19:14.910 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.910 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.910 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:14.911 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.482759+0000", 2024-09-20T22:19:14.911 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0034672689999999998 2024-09-20T22:19:14.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.911 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.911 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.911 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491864+0000", 2024-09-20T22:19:14.911 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0091058630000000005 2024-09-20T22:19:14.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.911 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.911 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.911 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491961+0000", 2024-09-20T22:19:14.911 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6717000000000003e-05 2024-09-20T22:19:14.911 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.912 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.912 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:14.912 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492014+0000", 2024-09-20T22:19:14.912 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2877000000000002e-05 2024-09-20T22:19:14.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.912 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.912 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:14.912 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492216+0000", 2024-09-20T22:19:14.912 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020235500000000001 2024-09-20T22:19:14.912 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.912 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:14.912 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.912 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.913 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.913 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163324 8.11 8:8e312e9e:::benchmark_data_smithi060_111051_object43325:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:19:14.913 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:08.983848+0000", 2024-09-20T22:19:14.913 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 545.90902066199999, 2024-09-20T22:19:14.913 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.50861305999999995, 2024-09-20T22:19:14.913 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:14.913 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:14.913 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:14.913 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:14.913 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:19:14.913 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:19:14.913 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163324 2024-09-20T22:19:14.913 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.913 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:14.913 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.913 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:14.914 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983848+0000", 2024-09-20T22:19:14.914 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.914 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.914 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.914 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:14.914 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983848+0000", 2024-09-20T22:19:14.914 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.914 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.914 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.914 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:14.914 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983849+0000", 2024-09-20T22:19:14.914 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.11e-06 2024-09-20T22:19:14.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.915 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.915 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:14.915 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983853+0000", 2024-09-20T22:19:14.915 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0439999999999997e-06 2024-09-20T22:19:14.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.915 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.915 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:14.915 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983863+0000", 2024-09-20T22:19:14.915 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6430000000000007e-06 2024-09-20T22:19:14.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.915 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.915 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:14.915 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983868+0000", 2024-09-20T22:19:14.915 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1000000000000003e-06 2024-09-20T22:19:14.915 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.916 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.916 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.916 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983919+0000", 2024-09-20T22:19:14.916 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0433999999999999e-05 2024-09-20T22:19:14.916 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.916 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.916 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:14.916 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:08.983928+0000", 2024-09-20T22:19:14.916 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3659999999999998e-06 2024-09-20T22:19:14.916 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.916 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.916 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.916 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.480646+0000", 2024-09-20T22:19:14.916 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.49671833999999998 2024-09-20T22:19:14.916 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.916 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.917 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:14.917 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.480768+0000", 2024-09-20T22:19:14.917 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012145 2024-09-20T22:19:14.917 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.917 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.917 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:19:14.917 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.481025+0000", 2024-09-20T22:19:14.917 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025713299999999999 2024-09-20T22:19:14.917 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.917 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.917 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:14.917 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.488496+0000", 2024-09-20T22:19:14.917 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0074709540000000001 2024-09-20T22:19:14.917 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.917 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.917 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.918 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492276+0000", 2024-09-20T22:19:14.918 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0037803670000000002 2024-09-20T22:19:14.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.918 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.918 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.918 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492350+0000", 2024-09-20T22:19:14.918 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3436999999999995e-05 2024-09-20T22:19:14.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.918 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.918 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:14.918 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492421+0000", 2024-09-20T22:19:14.918 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1303000000000005e-05 2024-09-20T22:19:14.918 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.918 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.918 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:14.919 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492461+0000", 2024-09-20T22:19:14.919 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0379000000000001e-05 2024-09-20T22:19:14.919 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.919 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:14.919 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.919 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.919 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.919 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163358 8.11 8:8b7ace29:::benchmark_data_smithi060_111051_object43359:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:19:14.919 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.014358+0000", 2024-09-20T22:19:14.919 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 545.87851067700001, 2024-09-20T22:19:14.919 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.47828052599999998, 2024-09-20T22:19:14.919 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:14.919 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:14.919 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:14.920 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:14.920 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:19:14.920 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:19:14.920 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163358 2024-09-20T22:19:14.920 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.920 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:14.920 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.920 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:14.920 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014358+0000", 2024-09-20T22:19:14.920 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.920 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.920 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.920 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:14.920 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014358+0000", 2024-09-20T22:19:14.920 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.921 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.921 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.921 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:14.921 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014359+0000", 2024-09-20T22:19:14.921 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0669999999999999e-06 2024-09-20T22:19:14.921 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.921 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.921 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:14.921 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014364+0000", 2024-09-20T22:19:14.921 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0220000000000004e-06 2024-09-20T22:19:14.921 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.921 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.921 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:14.921 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014374+0000", 2024-09-20T22:19:14.922 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9599999999999995e-06 2024-09-20T22:19:14.922 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.922 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.922 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:14.922 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014388+0000", 2024-09-20T22:19:14.922 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3413e-05 2024-09-20T22:19:14.922 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.922 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.922 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.922 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014447+0000", 2024-09-20T22:19:14.922 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9698999999999999e-05 2024-09-20T22:19:14.922 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.922 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.922 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:14.923 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.014455+0000", 2024-09-20T22:19:14.923 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.072e-06 2024-09-20T22:19:14.923 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.923 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.923 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.923 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.482921+0000", 2024-09-20T22:19:14.923 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.46846513200000001 2024-09-20T22:19:14.923 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.923 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.923 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:14.923 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483020+0000", 2024-09-20T22:19:14.923 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9319000000000002e-05 2024-09-20T22:19:14.923 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.923 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.923 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:19:14.924 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.483251+0000", 2024-09-20T22:19:14.924 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00023124300000000001 2024-09-20T22:19:14.924 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.924 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.924 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:14.924 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491608+0000", 2024-09-20T22:19:14.924 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0083564190000000003 2024-09-20T22:19:14.924 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.924 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.924 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.924 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492507+0000", 2024-09-20T22:19:14.924 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00089972900000000005 2024-09-20T22:19:14.924 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.924 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.925 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.925 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492546+0000", 2024-09-20T22:19:14.925 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9156999999999998e-05 2024-09-20T22:19:14.925 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.925 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.925 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:14.925 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492602+0000", 2024-09-20T22:19:14.925 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5114999999999997e-05 2024-09-20T22:19:14.925 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.925 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.926 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:14.926 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492639+0000", 2024-09-20T22:19:14.926 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7178999999999998e-05 2024-09-20T22:19:14.926 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.926 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:14.926 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.926 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.926 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.926 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163372 8.11 8:8b6ec81f:::benchmark_data_smithi060_111051_object43373:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:19:14.926 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.027195+0000", 2024-09-20T22:19:14.926 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 545.86567386900003, 2024-09-20T22:19:14.926 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.46568983899999999, 2024-09-20T22:19:14.926 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:14.926 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:19:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:19:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163372 2024-09-20T22:19:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027195+0000", 2024-09-20T22:19:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027195+0000", 2024-09-20T22:19:14.927 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027196+0000", 2024-09-20T22:19:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0599999999999999e-07 2024-09-20T22:19:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027200+0000", 2024-09-20T22:19:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9280000000000003e-06 2024-09-20T22:19:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027210+0000", 2024-09-20T22:19:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0035e-05 2024-09-20T22:19:14.928 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027217+0000", 2024-09-20T22:19:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6450000000000005e-06 2024-09-20T22:19:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027281+0000", 2024-09-20T22:19:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3281000000000006e-05 2024-09-20T22:19:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027291+0000", 2024-09-20T22:19:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0268999999999999e-05 2024-09-20T22:19:14.929 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.484647+0000", 2024-09-20T22:19:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.45735564499999998 2024-09-20T22:19:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.484745+0000", 2024-09-20T22:19:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8712e-05 2024-09-20T22:19:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:19:14.930 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.484940+0000", 2024-09-20T22:19:14.931 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019504900000000001 2024-09-20T22:19:14.931 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.931 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.931 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:14.931 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491619+0000", 2024-09-20T22:19:14.931 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0066782090000000001 2024-09-20T22:19:14.931 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.931 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.931 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.931 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492716+0000", 2024-09-20T22:19:14.931 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001097701 2024-09-20T22:19:14.931 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.931 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.932 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.932 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492778+0000", 2024-09-20T22:19:14.932 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1538000000000006e-05 2024-09-20T22:19:14.932 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.932 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.932 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:14.932 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492830+0000", 2024-09-20T22:19:14.932 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1997999999999999e-05 2024-09-20T22:19:14.932 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.932 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.932 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:14.932 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492885+0000", 2024-09-20T22:19:14.932 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5022999999999998e-05 2024-09-20T22:19:14.932 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.933 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:14.933 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.933 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.933 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.933 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163373 8.11 8:8d07a686:::benchmark_data_smithi060_111051_object43374:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:19:14.933 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.027847+0000", 2024-09-20T22:19:14.933 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 545.86502146099997, 2024-09-20T22:19:14.933 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.465200326, 2024-09-20T22:19:14.933 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:14.933 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:14.933 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:14.933 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:14.933 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:19:14.934 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:19:14.934 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163373 2024-09-20T22:19:14.934 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.934 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:14.934 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.934 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:14.934 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027847+0000", 2024-09-20T22:19:14.934 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.934 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.934 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.934 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:14.934 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027847+0000", 2024-09-20T22:19:14.934 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.934 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.935 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.935 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:14.935 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027848+0000", 2024-09-20T22:19:14.935 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3600000000000003e-07 2024-09-20T22:19:14.935 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.935 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.935 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:14.935 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027852+0000", 2024-09-20T22:19:14.935 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6330000000000002e-06 2024-09-20T22:19:14.935 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.935 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.935 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:14.935 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027860+0000", 2024-09-20T22:19:14.935 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2789999999999998e-06 2024-09-20T22:19:14.936 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.936 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.936 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:14.936 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027866+0000", 2024-09-20T22:19:14.936 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1070000000000001e-06 2024-09-20T22:19:14.936 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.936 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.936 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.936 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027897+0000", 2024-09-20T22:19:14.936 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0722000000000002e-05 2024-09-20T22:19:14.936 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.936 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.936 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:14.936 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.027901+0000", 2024-09-20T22:19:14.936 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1649999999999999e-06 2024-09-20T22:19:14.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.937 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.937 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.937 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.486658+0000", 2024-09-20T22:19:14.937 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.45875733499999999 2024-09-20T22:19:14.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.937 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.937 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:14.937 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.486767+0000", 2024-09-20T22:19:14.937 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000108385 2024-09-20T22:19:14.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.937 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.937 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:19:14.937 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.487120+0000", 2024-09-20T22:19:14.937 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00035312499999999998 2024-09-20T22:19:14.937 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.938 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.938 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:14.938 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491877+0000", 2024-09-20T22:19:14.938 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0047569170000000003 2024-09-20T22:19:14.938 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.938 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.938 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.938 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492928+0000", 2024-09-20T22:19:14.938 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001051633 2024-09-20T22:19:14.938 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.938 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.939 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:14.939 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:14.940 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:14.940 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.940 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.492967+0000", 2024-09-20T22:19:14.940 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.8781e-05 2024-09-20T22:19:14.940 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.940 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.940 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:14.940 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493013+0000", 2024-09-20T22:19:14.940 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5555e-05 2024-09-20T22:19:14.940 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.940 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.941 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:14.941 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493048+0000", 2024-09-20T22:19:14.941 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5052999999999998e-05 2024-09-20T22:19:14.941 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.941 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:14.941 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.941 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.941 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.941 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163388 8.11 8:893f1034:::benchmark_data_smithi060_111051_object43389:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:19:14.941 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.050114+0000", 2024-09-20T22:19:14.941 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 545.84275475100003, 2024-09-20T22:19:14.941 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.44308652799999998, 2024-09-20T22:19:14.941 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:14.942 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:14.942 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:14.942 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:14.942 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:19:14.942 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:19:14.942 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163388 2024-09-20T22:19:14.942 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.942 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:14.942 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.942 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:14.942 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050114+0000", 2024-09-20T22:19:14.942 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.942 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.942 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.943 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:14.943 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050114+0000", 2024-09-20T22:19:14.943 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.943 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.943 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.943 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:14.943 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050115+0000", 2024-09-20T22:19:14.943 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2500000000000005e-07 2024-09-20T22:19:14.943 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.943 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.943 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:14.943 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050119+0000", 2024-09-20T22:19:14.943 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.698e-06 2024-09-20T22:19:14.943 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.944 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.944 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:14.944 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050128+0000", 2024-09-20T22:19:14.944 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5820000000000008e-06 2024-09-20T22:19:14.944 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.944 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.944 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:14.944 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050133+0000", 2024-09-20T22:19:14.944 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9060000000000001e-06 2024-09-20T22:19:14.944 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.944 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.944 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.944 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050169+0000", 2024-09-20T22:19:14.944 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5984e-05 2024-09-20T22:19:14.944 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.945 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.945 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:14.945 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.050174+0000", 2024-09-20T22:19:14.945 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0180000000000001e-06 2024-09-20T22:19:14.945 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.945 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.945 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.945 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.488568+0000", 2024-09-20T22:19:14.945 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.43839391 2024-09-20T22:19:14.945 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.945 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.945 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:14.945 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.488685+0000", 2024-09-20T22:19:14.945 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000117163 2024-09-20T22:19:14.945 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.946 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.946 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:19:14.946 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.489162+0000", 2024-09-20T22:19:14.946 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00047712600000000003 2024-09-20T22:19:14.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.946 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.946 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:14.946 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.491884+0000", 2024-09-20T22:19:14.946 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0027215329999999999 2024-09-20T22:19:14.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.946 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.946 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.946 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493086+0000", 2024-09-20T22:19:14.946 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001201765 2024-09-20T22:19:14.946 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.947 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.947 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.947 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493122+0000", 2024-09-20T22:19:14.947 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6622999999999998e-05 2024-09-20T22:19:14.947 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.947 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.947 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:14.947 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493168+0000", 2024-09-20T22:19:14.947 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5757999999999998e-05 2024-09-20T22:19:14.947 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.947 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.947 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:14.947 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493201+0000", 2024-09-20T22:19:14.947 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2737000000000002e-05 2024-09-20T22:19:14.947 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.948 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:14.948 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.948 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.948 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.948 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:163393 8.11 8:8bb06d42:::benchmark_data_smithi060_111051_object43394:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e242)", 2024-09-20T22:19:14.948 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:09.064776+0000", 2024-09-20T22:19:14.948 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 545.82809282799997, 2024-09-20T22:19:14.948 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.43944195600000002, 2024-09-20T22:19:14.948 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:14.948 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:14.948 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:14.948 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:14.948 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:19:14.948 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:19:14.948 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 163393 2024-09-20T22:19:14.948 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.949 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:14.949 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.949 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:14.949 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064776+0000", 2024-09-20T22:19:14.949 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.949 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.949 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.949 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:14.949 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064776+0000", 2024-09-20T22:19:14.949 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.949 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.949 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.949 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:14.949 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064777+0000", 2024-09-20T22:19:14.949 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9500000000000001e-07 2024-09-20T22:19:14.949 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.949 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.950 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:14.950 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064781+0000", 2024-09-20T22:19:14.950 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9870000000000001e-06 2024-09-20T22:19:14.950 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.950 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.950 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:14.950 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064791+0000", 2024-09-20T22:19:14.950 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0417999999999999e-05 2024-09-20T22:19:14.950 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.950 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.950 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:14.950 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064797+0000", 2024-09-20T22:19:14.950 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4040000000000003e-06 2024-09-20T22:19:14.950 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.951 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.951 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.951 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064838+0000", 2024-09-20T22:19:14.951 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1591000000000002e-05 2024-09-20T22:19:14.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.951 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.951 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:14.951 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.064843+0000", 2024-09-20T22:19:14.951 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9470000000000002e-06 2024-09-20T22:19:14.951 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.951 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.951 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.951 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493235+0000", 2024-09-20T22:19:14.951 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.42839186899999998 2024-09-20T22:19:14.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.952 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.952 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:14.952 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493336+0000", 2024-09-20T22:19:14.952 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100923 2024-09-20T22:19:14.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.952 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.952 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 6,7", 2024-09-20T22:19:14.952 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.493659+0000", 2024-09-20T22:19:14.952 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00032341599999999998 2024-09-20T22:19:14.952 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.952 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.952 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:14.952 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.499928+0000", 2024-09-20T22:19:14.952 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0062687730000000001 2024-09-20T22:19:14.953 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.953 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.953 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.953 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.503797+0000", 2024-09-20T22:19:14.953 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0038686020000000001 2024-09-20T22:19:14.953 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.953 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.953 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.953 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.504160+0000", 2024-09-20T22:19:14.953 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00036300900000000002 2024-09-20T22:19:14.953 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.953 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.953 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:14.953 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.504192+0000", 2024-09-20T22:19:14.954 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2592000000000002e-05 2024-09-20T22:19:14.954 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.954 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.954 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:14.954 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:09.504218+0000", 2024-09-20T22:19:14.954 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.563e-05 2024-09-20T22:19:14.954 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.954 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:14.954 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.954 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.954 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.954 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:125848 9.6 9:604ae09f:::benchmark_data_smithi060_140132_object15040:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:19:14.954 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:13:30.323134+0000", 2024-09-20T22:19:14.954 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 344.56973455799999, 2024-09-20T22:19:14.955 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.39324260900000002, 2024-09-20T22:19:14.955 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:14.955 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:14.955 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:14.955 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:14.955 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:19:14.955 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:19:14.955 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 125848 2024-09-20T22:19:14.955 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.955 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:14.955 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.955 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:14.955 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323134+0000", 2024-09-20T22:19:14.955 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.955 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.956 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.956 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:14.956 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323134+0000", 2024-09-20T22:19:14.956 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.956 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.956 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.956 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:14.956 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323135+0000", 2024-09-20T22:19:14.956 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1399999999999997e-07 2024-09-20T22:19:14.956 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.956 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.956 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:14.956 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323137+0000", 2024-09-20T22:19:14.956 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1789999999999998e-06 2024-09-20T22:19:14.956 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.957 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.957 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:14.957 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323143+0000", 2024-09-20T22:19:14.957 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1469999999999998e-06 2024-09-20T22:19:14.957 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.957 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.957 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:14.957 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323149+0000", 2024-09-20T22:19:14.957 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2639999999999997e-06 2024-09-20T22:19:14.957 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.957 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.957 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.957 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323210+0000", 2024-09-20T22:19:14.958 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0537e-05 2024-09-20T22:19:14.958 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.958 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.958 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:14.958 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323333+0000", 2024-09-20T22:19:14.958 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012330500000000001 2024-09-20T22:19:14.958 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.958 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.958 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:19:14.958 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323444+0000", 2024-09-20T22:19:14.958 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000110562 2024-09-20T22:19:14.958 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.958 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.958 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:14.959 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.326284+0000", 2024-09-20T22:19:14.959 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028407409999999999 2024-09-20T22:19:14.959 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.959 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.959 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.959 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.328793+0000", 2024-09-20T22:19:14.959 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0025082680000000001 2024-09-20T22:19:14.959 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.959 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.959 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.959 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.716294+0000", 2024-09-20T22:19:14.959 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.38750077100000002 2024-09-20T22:19:14.959 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.959 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.960 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:14.960 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.716331+0000", 2024-09-20T22:19:14.960 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7419999999999997e-05 2024-09-20T22:19:14.960 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.960 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.960 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:14.960 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.716377+0000", 2024-09-20T22:19:14.960 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6001000000000002e-05 2024-09-20T22:19:14.960 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.960 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:14.960 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.960 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.960 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.960 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:125853 9.6 9:65bf9cb5:::benchmark_data_smithi060_140132_object15045:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:19:14.960 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:13:30.329387+0000", 2024-09-20T22:19:14.960 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 344.563481895, 2024-09-20T22:19:14.961 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.39097777700000003, 2024-09-20T22:19:14.961 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:14.961 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:14.961 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:14.961 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:14.961 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:19:14.961 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:19:14.961 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 125853 2024-09-20T22:19:14.961 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.961 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:14.961 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.961 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:14.961 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329387+0000", 2024-09-20T22:19:14.961 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.961 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.961 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.962 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:14.962 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329387+0000", 2024-09-20T22:19:14.962 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.962 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.962 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.962 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:14.962 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329388+0000", 2024-09-20T22:19:14.962 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3990000000000001e-06 2024-09-20T22:19:14.962 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.962 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.962 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:14.962 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329395+0000", 2024-09-20T22:19:14.962 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.161e-06 2024-09-20T22:19:14.962 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.962 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.963 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:14.963 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329407+0000", 2024-09-20T22:19:14.963 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2323999999999999e-05 2024-09-20T22:19:14.963 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.963 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.963 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:14.963 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329414+0000", 2024-09-20T22:19:14.963 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.289e-06 2024-09-20T22:19:14.963 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.963 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.963 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.963 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329463+0000", 2024-09-20T22:19:14.963 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8658000000000001e-05 2024-09-20T22:19:14.963 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.963 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.964 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:14.964 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329589+0000", 2024-09-20T22:19:14.964 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012579900000000001 2024-09-20T22:19:14.964 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.964 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.964 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:19:14.964 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329695+0000", 2024-09-20T22:19:14.964 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000105933 2024-09-20T22:19:14.964 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.964 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.964 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.964 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.335795+0000", 2024-09-20T22:19:14.964 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061008720000000002 2024-09-20T22:19:14.965 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.965 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.965 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:14.965 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.335858+0000", 2024-09-20T22:19:14.965 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2285000000000004e-05 2024-09-20T22:19:14.965 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.965 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.965 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.965 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720262+0000", 2024-09-20T22:19:14.965 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.38440416300000002 2024-09-20T22:19:14.965 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.965 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.965 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:14.965 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720315+0000", 2024-09-20T22:19:14.965 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2985000000000002e-05 2024-09-20T22:19:14.965 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.966 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.966 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:14.966 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720365+0000", 2024-09-20T22:19:14.966 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:19:14.967 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9908999999999999e-05 2024-09-20T22:19:14.967 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.967 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:14.967 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.967 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.967 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.967 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:125877 9.6 9:618bafd1:::benchmark_data_smithi060_140132_object15069:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:19:14.967 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:19:14.967 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 344.54778031900003, 2024-09-20T22:19:14.967 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.37546249399999998, 2024-09-20T22:19:14.967 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:14.967 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:14.968 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:14.968 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:14.968 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:19:14.968 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:19:14.968 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 125877 2024-09-20T22:19:14.968 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.968 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:14.968 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.968 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:14.968 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:19:14.968 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.968 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.968 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.968 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:14.968 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:19:14.969 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.969 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.969 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.969 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:14.969 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:19:14.969 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1999999999999999e-07 2024-09-20T22:19:14.969 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.969 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.969 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:14.969 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345092+0000", 2024-09-20T22:19:14.969 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2899999999999998e-06 2024-09-20T22:19:14.969 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.969 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.969 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:14.970 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345101+0000", 2024-09-20T22:19:14.970 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1550000000000007e-06 2024-09-20T22:19:14.970 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.970 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.970 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:14.970 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345107+0000", 2024-09-20T22:19:14.970 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2609999999999999e-06 2024-09-20T22:19:14.970 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.970 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.970 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.970 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345160+0000", 2024-09-20T22:19:14.970 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.287e-05 2024-09-20T22:19:14.970 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.971 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.971 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:14.971 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345269+0000", 2024-09-20T22:19:14.971 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000108795 2024-09-20T22:19:14.971 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.971 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.971 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:19:14.971 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345395+0000", 2024-09-20T22:19:14.971 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012654800000000001 2024-09-20T22:19:14.971 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.971 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.971 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:14.971 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.348668+0000", 2024-09-20T22:19:14.971 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0032731729999999999 2024-09-20T22:19:14.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.972 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.972 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.972 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.351790+0000", 2024-09-20T22:19:14.972 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003121567 2024-09-20T22:19:14.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.972 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.972 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.972 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720464+0000", 2024-09-20T22:19:14.972 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.36867422 2024-09-20T22:19:14.972 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.972 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.972 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:14.972 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720511+0000", 2024-09-20T22:19:14.972 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7076999999999997e-05 2024-09-20T22:19:14.973 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.973 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.973 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:14.973 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720551+0000", 2024-09-20T22:19:14.973 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9918e-05 2024-09-20T22:19:14.973 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.973 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:14.973 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.973 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.973 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.973 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:77013 10.9 10:9a8ed282:::benchmark_data_smithi060_167766_object77012:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e395)", 2024-09-20T22:19:14.973 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:16:04.243329+0000", 2024-09-20T22:19:14.973 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 190.64953949100001, 2024-09-20T22:19:14.973 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.24326910600000001, 2024-09-20T22:19:14.973 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:14.974 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:14.974 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:14.974 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:14.974 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:19:14.974 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:19:14.974 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 77013 2024-09-20T22:19:14.974 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.974 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:14.974 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.974 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:14.974 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.243329+0000", 2024-09-20T22:19:14.974 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.974 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.974 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.975 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:14.975 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.243329+0000", 2024-09-20T22:19:14.975 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.975 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.975 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.975 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:14.975 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.243331+0000", 2024-09-20T22:19:14.975 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4479999999999999e-06 2024-09-20T22:19:14.975 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.975 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.975 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:14.975 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.243402+0000", 2024-09-20T22:19:14.975 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1604000000000001e-05 2024-09-20T22:19:14.976 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.976 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.976 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:14.976 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.243418+0000", 2024-09-20T22:19:14.976 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5934e-05 2024-09-20T22:19:14.976 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.976 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.976 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:14.976 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.243426+0000", 2024-09-20T22:19:14.976 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.165e-06 2024-09-20T22:19:14.976 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.976 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.976 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.976 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.247494+0000", 2024-09-20T22:19:14.976 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0040683840000000004 2024-09-20T22:19:14.976 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.977 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.977 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:14.977 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.247586+0000", 2024-09-20T22:19:14.977 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2409000000000002e-05 2024-09-20T22:19:14.977 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.977 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.977 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:19:14.977 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.247708+0000", 2024-09-20T22:19:14.977 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000121508 2024-09-20T22:19:14.977 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.977 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.977 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:14.977 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.250076+0000", 2024-09-20T22:19:14.977 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0023685999999999998 2024-09-20T22:19:14.977 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.978 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.978 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.978 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.259850+0000", 2024-09-20T22:19:14.978 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0097730779999999993 2024-09-20T22:19:14.978 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.978 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.978 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.978 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.486491+0000", 2024-09-20T22:19:14.978 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.22664168800000001 2024-09-20T22:19:14.978 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.978 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.978 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:14.978 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.486548+0000", 2024-09-20T22:19:14.979 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7215e-05 2024-09-20T22:19:14.979 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.979 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.979 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:14.979 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:16:04.486599+0000", 2024-09-20T22:19:14.979 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0073e-05 2024-09-20T22:19:14.979 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.979 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:14.979 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.979 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.979 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.979 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:207380 10.9 10:94c19669:::benchmark_data_smithi060_167766_object82588:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e437)", 2024-09-20T22:19:14.979 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:17:57.503744+0000", 2024-09-20T22:19:14.979 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 77.389124619, 2024-09-20T22:19:14.979 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.25047445699999998, 2024-09-20T22:19:14.980 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:14.980 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:14.980 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:14.980 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:14.980 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:19:14.980 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:19:14.980 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 207380 2024-09-20T22:19:14.980 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.980 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:14.980 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.980 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:14.980 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:57.503744+0000", 2024-09-20T22:19:14.980 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.980 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.980 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.980 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:14.981 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:57.503744+0000", 2024-09-20T22:19:14.981 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.981 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.981 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.981 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:14.981 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:57.503745+0000", 2024-09-20T22:19:14.981 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9400000000000004e-07 2024-09-20T22:19:14.981 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.981 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.981 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:14.981 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:57.503749+0000", 2024-09-20T22:19:14.981 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1919999999999998e-06 2024-09-20T22:19:14.981 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.981 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.981 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:14.982 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:57.503760+0000", 2024-09-20T22:19:14.982 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0587e-05 2024-09-20T22:19:14.982 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.982 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.982 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:14.982 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:57.503767+0000", 2024-09-20T22:19:14.982 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5980000000000002e-06 2024-09-20T22:19:14.982 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.982 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.982 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.982 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:57.503815+0000", 2024-09-20T22:19:14.982 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8831000000000001e-05 2024-09-20T22:19:14.982 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.983 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.983 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:14.983 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:57.503915+0000", 2024-09-20T22:19:14.983 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9970000000000007e-05 2024-09-20T22:19:14.983 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.983 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.983 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,7", 2024-09-20T22:19:14.983 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:57.504319+0000", 2024-09-20T22:19:14.983 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000403328 2024-09-20T22:19:14.983 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.983 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.983 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.983 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:57.509418+0000", 2024-09-20T22:19:14.983 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0050989 2024-09-20T22:19:14.983 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.984 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.984 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:14.984 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:57.509432+0000", 2024-09-20T22:19:14.984 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4096e-05 2024-09-20T22:19:14.984 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.984 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.984 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.984 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:57.754154+0000", 2024-09-20T22:19:14.984 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.24472242 2024-09-20T22:19:14.984 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.984 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.984 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:14.984 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:57.754189+0000", 2024-09-20T22:19:14.984 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5111000000000003e-05 2024-09-20T22:19:14.985 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.985 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.985 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:14.985 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:17:57.754219+0000", 2024-09-20T22:19:14.985 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.953e-05 2024-09-20T22:19:14.985 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.985 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:14.985 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.985 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.985 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.985 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:233503 10.6 10:6fc58bf5:::benchmark_data_smithi060_167766_object108711:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e445)", 2024-09-20T22:19:14.985 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:18:20.606346+0000", 2024-09-20T22:19:14.985 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 54.286523322999997, 2024-09-20T22:19:14.985 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.29572936999999999, 2024-09-20T22:19:14.985 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:14.985 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:14.986 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:14.986 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:14.986 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:19:14.986 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:19:14.986 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 233503 2024-09-20T22:19:14.986 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.986 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:14.986 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.986 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:14.986 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.606346+0000", 2024-09-20T22:19:14.986 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.986 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.986 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.986 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:14.986 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.606346+0000", 2024-09-20T22:19:14.986 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.987 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.987 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:14.987 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.606347+0000", 2024-09-20T22:19:14.987 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0589999999999999e-06 2024-09-20T22:19:14.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.987 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.987 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:14.987 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.606351+0000", 2024-09-20T22:19:14.987 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6709999999999998e-06 2024-09-20T22:19:14.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.987 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.987 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:14.987 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.606362+0000", 2024-09-20T22:19:14.987 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0873000000000001e-05 2024-09-20T22:19:14.987 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.988 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.988 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:14.988 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.606367+0000", 2024-09-20T22:19:14.988 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2889999999999997e-06 2024-09-20T22:19:14.988 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.988 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.988 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.988 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.606429+0000", 2024-09-20T22:19:14.988 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1105999999999994e-05 2024-09-20T22:19:14.988 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.988 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.988 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:14.988 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.606517+0000", 2024-09-20T22:19:14.988 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8004999999999995e-05 2024-09-20T22:19:14.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.989 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.989 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:19:14.989 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.606582+0000", 2024-09-20T22:19:14.989 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5461000000000002e-05 2024-09-20T22:19:14.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.989 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.989 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:14.989 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.611900+0000", 2024-09-20T22:19:14.989 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0053178720000000004 2024-09-20T22:19:14.989 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.989 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.989 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.990 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.612499+0000", 2024-09-20T22:19:14.990 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00059942700000000003 2024-09-20T22:19:14.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.990 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.990 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.990 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.902014+0000", 2024-09-20T22:19:14.990 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.289514575 2024-09-20T22:19:14.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.990 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.990 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:14.990 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.902047+0000", 2024-09-20T22:19:14.990 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.3392999999999997e-05 2024-09-20T22:19:14.990 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.990 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.991 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:14.991 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.902075+0000", 2024-09-20T22:19:14.991 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7639000000000001e-05 2024-09-20T22:19:14.991 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.991 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:14.991 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.991 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.991 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.991 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:233507 10.6 10:66b48a90:::benchmark_data_smithi060_167766_object108715:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e445)", 2024-09-20T22:19:14.991 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:18:20.607017+0000", 2024-09-20T22:19:14.991 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 54.285852128000002, 2024-09-20T22:19:14.991 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.29609884600000003, 2024-09-20T22:19:14.991 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:14.991 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:14.991 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:14.992 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:14.992 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:19:14.992 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:19:14.992 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 233507 2024-09-20T22:19:14.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.992 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:14.992 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.992 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:14.992 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.607017+0000", 2024-09-20T22:19:14.992 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.992 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.992 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:14.992 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.607017+0000", 2024-09-20T22:19:14.992 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:14.992 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.993 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.993 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:14.993 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.607018+0000", 2024-09-20T22:19:14.993 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1999999999999998e-07 2024-09-20T22:19:14.993 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:19:14.993 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:19:14.993 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:19:14.994 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:14 smithi067 ceph-mon[33511]: 6.7 scrub starts 2024-09-20T22:19:14.994 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:14 smithi067 ceph-mon[33511]: 6.7 scrub ok 2024-09-20T22:19:14.994 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:14 smithi067 ceph-mon[33511]: 2.1 scrub starts 2024-09-20T22:19:14.994 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:14 smithi067 ceph-mon[33511]: 2.1 scrub ok 2024-09-20T22:19:14.994 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:14 smithi067 ceph-mon[33511]: 3.12 scrub starts 2024-09-20T22:19:14.994 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:14 smithi067 ceph-mon[33511]: 3.12 scrub ok 2024-09-20T22:19:14.994 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:14 smithi067 ceph-mon[33511]: pgmap v872: 178 pgs: 178 active+clean; 2.5 GiB data, 13 GiB used, 703 GiB / 715 GiB avail; 665 B/s rd, 107 MiB/s wr, 1.71k op/s 2024-09-20T22:19:14.994 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:14 smithi067 ceph-mon[33511]: 4.1 scrub starts 2024-09-20T22:19:14.994 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:14 smithi067 ceph-mon[33511]: 4.1 scrub ok 2024-09-20T22:19:14.995 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:14 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1480723543' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-20T22:19:14.995 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:14 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]: dispatch 2024-09-20T22:19:14.995 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:14 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/328412255' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:14.995 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.995 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.995 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:14.995 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.607022+0000", 2024-09-20T22:19:14.995 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3000000000000003e-06 2024-09-20T22:19:14.996 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.996 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.996 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:14.996 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.607031+0000", 2024-09-20T22:19:14.996 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.0890000000000006e-06 2024-09-20T22:19:14.996 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.996 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.996 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:14.996 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.607038+0000", 2024-09-20T22:19:14.996 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7170000000000002e-06 2024-09-20T22:19:14.996 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.996 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.996 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:14.996 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.609062+0000", 2024-09-20T22:19:14.997 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0020244199999999999 2024-09-20T22:19:14.997 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.997 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.997 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:14.997 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.609166+0000", 2024-09-20T22:19:14.997 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010374099999999999 2024-09-20T22:19:14.997 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.997 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.997 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:19:14.997 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.609271+0000", 2024-09-20T22:19:14.997 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010552699999999999 2024-09-20T22:19:14.997 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.997 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.997 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:14.997 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.612012+0000", 2024-09-20T22:19:14.998 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0027403499999999999 2024-09-20T22:19:14.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.998 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.998 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.998 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.612630+0000", 2024-09-20T22:19:14.998 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00061808299999999996 2024-09-20T22:19:14.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.998 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.998 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:14.998 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.902994+0000", 2024-09-20T22:19:14.998 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.29036447700000001 2024-09-20T22:19:14.998 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.998 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.998 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:14.998 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.903072+0000", 2024-09-20T22:19:14.998 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7293999999999996e-05 2024-09-20T22:19:14.999 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.999 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.999 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:14.999 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.903116+0000", 2024-09-20T22:19:14.999 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4027999999999999e-05 2024-09-20T22:19:14.999 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.999 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:14.999 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:14.999 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:14.999 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:14.999 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:41032 11.7 11:eba560c0:::benchmark_data_smithi060_196073_object41031:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e465)", 2024-09-20T22:19:14.999 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:19:12.065782+0000", 2024-09-20T22:19:14.999 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 2.8270866950000002, 2024-09-20T22:19:14.999 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.26421668999999998, 2024-09-20T22:19:14.999 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:14.999 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:15.000 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:15.000 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:15.000 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:19:15.000 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:19:15.000 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 41032 2024-09-20T22:19:15.000 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.000 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:15.000 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.000 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:15.000 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.065782+0000", 2024-09-20T22:19:15.000 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:15.000 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.000 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.000 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:15.000 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.065782+0000", 2024-09-20T22:19:15.000 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:15.001 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.001 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.001 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:15.001 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.065784+0000", 2024-09-20T22:19:15.001 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7370000000000001e-06 2024-09-20T22:19:15.001 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.001 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.001 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:15.001 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.065916+0000", 2024-09-20T22:19:15.001 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013219999999999999 2024-09-20T22:19:15.001 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.001 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.001 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:15.001 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.065930+0000", 2024-09-20T22:19:15.001 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.377e-05 2024-09-20T22:19:15.001 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.002 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.002 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:15.002 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.065938+0000", 2024-09-20T22:19:15.002 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9689999999999994e-06 2024-09-20T22:19:15.002 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.002 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.002 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:15.002 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.066176+0000", 2024-09-20T22:19:15.002 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000237865 2024-09-20T22:19:15.002 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.002 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.002 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:15.002 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.066309+0000", 2024-09-20T22:19:15.002 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000133354 2024-09-20T22:19:15.003 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.003 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.003 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,4", 2024-09-20T22:19:15.003 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.066451+0000", 2024-09-20T22:19:15.003 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000142187 2024-09-20T22:19:15.003 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.003 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.003 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:15.003 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.328343+0000", 2024-09-20T22:19:15.003 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.26189169600000001 2024-09-20T22:19:15.003 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.003 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.003 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:15.003 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.328395+0000", 2024-09-20T22:19:15.003 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1583999999999999e-05 2024-09-20T22:19:15.004 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.004 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.004 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:15.004 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.329892+0000", 2024-09-20T22:19:15.004 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001497324 2024-09-20T22:19:15.004 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.004 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.004 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:15.004 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.329955+0000", 2024-09-20T22:19:15.004 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2707999999999997e-05 2024-09-20T22:19:15.004 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.004 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.004 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:15.004 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.329999+0000", 2024-09-20T22:19:15.004 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4295999999999997e-05 2024-09-20T22:19:15.004 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:15.004 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:15.005 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:15.005 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.005 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.005 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.16440.0:41029 11.5 e465/456 11:a6fd84dc:::benchmark_data_smithi060_196073_object41028:head v 465'2610, mlcod=465'2608)", 2024-09-20T22:19:15.005 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:19:12.066489+0000", 2024-09-20T22:19:15.005 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 2.8263798250000001, 2024-09-20T22:19:15.005 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.26468454800000002, 2024-09-20T22:19:15.005 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:15.005 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:15.005 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:15.005 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:15.005 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.005 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:15.005 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.066489+0000", 2024-09-20T22:19:15.006 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:15.006 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.006 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.006 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:15.006 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.066489+0000", 2024-09-20T22:19:15.006 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:15.006 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.006 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.006 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:15.006 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.066490+0000", 2024-09-20T22:19:15.006 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3600000000000003e-07 2024-09-20T22:19:15.006 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.006 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.006 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:15.006 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.066533+0000", 2024-09-20T22:19:15.006 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2713999999999998e-05 2024-09-20T22:19:15.006 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.007 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.007 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:15.007 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.066543+0000", 2024-09-20T22:19:15.007 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0661e-05 2024-09-20T22:19:15.007 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.007 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.007 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:15.007 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.066550+0000", 2024-09-20T22:19:15.007 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.443e-06 2024-09-20T22:19:15.007 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.007 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.007 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:15.007 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.327024+0000", 2024-09-20T22:19:15.007 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.26047456699999999 2024-09-20T22:19:15.008 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.008 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.008 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:15.008 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.327050+0000", 2024-09-20T22:19:15.008 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.5389e-05 2024-09-20T22:19:15.008 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.008 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.008 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:15.008 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.331079+0000", 2024-09-20T22:19:15.008 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0040290500000000002 2024-09-20T22:19:15.008 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.008 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.008 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:15.009 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.331174+0000", 2024-09-20T22:19:15.009 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4988000000000006e-05 2024-09-20T22:19:15.009 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:15.009 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:15.009 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:15.009 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.009 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.009 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.16440.0:41030 11.a e465/456 11:5e4f411e:::benchmark_data_smithi060_196073_object41029:head v 465'2626, mlcod=465'2624)", 2024-09-20T22:19:15.009 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:19:12.066698+0000", 2024-09-20T22:19:15.009 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 2.8261708049999998, 2024-09-20T22:19:15.009 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.26442670200000001, 2024-09-20T22:19:15.009 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:15.009 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:15.009 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:15.010 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:15.010 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.010 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:15.010 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.066698+0000", 2024-09-20T22:19:15.010 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:15.010 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.010 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.010 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:15.010 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.066698+0000", 2024-09-20T22:19:15.010 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:15.010 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.010 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.010 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:15.010 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.066699+0000", 2024-09-20T22:19:15.011 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7100000000000002e-07 2024-09-20T22:19:15.011 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.011 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.011 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:15.011 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.066720+0000", 2024-09-20T22:19:15.011 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.181e-05 2024-09-20T22:19:15.011 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.011 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.011 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:15.011 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.326971+0000", 2024-09-20T22:19:15.011 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.26025028900000002 2024-09-20T22:19:15.011 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.011 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.011 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:15.011 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.326993+0000", 2024-09-20T22:19:15.012 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1831000000000002e-05 2024-09-20T22:19:15.012 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.012 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.012 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:15.012 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.327235+0000", 2024-09-20T22:19:15.012 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00024265800000000001 2024-09-20T22:19:15.012 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.012 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.012 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:15.012 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.327257+0000", 2024-09-20T22:19:15.012 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1515999999999999e-05 2024-09-20T22:19:15.012 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.012 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.013 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:15.013 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.331067+0000", 2024-09-20T22:19:15.013 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0038099029999999999 2024-09-20T22:19:15.013 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.013 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.013 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:15.013 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.331125+0000", 2024-09-20T22:19:15.013 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8124000000000001e-05 2024-09-20T22:19:15.013 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:15.013 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:15.013 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:15.013 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.013 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.013 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop_reply(client.16440.0:41032 11.7 e465/456 ondisk, result = 0)", 2024-09-20T22:19:15.014 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:19:12.068299+0000", 2024-09-20T22:19:15.014 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 2.8245698360000002, 2024-09-20T22:19:15.014 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.26005035900000001, 2024-09-20T22:19:15.014 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:15.014 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:15.014 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:19:15.014 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:15.014 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.014 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:15.014 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.068299+0000", 2024-09-20T22:19:15.014 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:15.014 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.014 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.014 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:15.014 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.068299+0000", 2024-09-20T22:19:15.015 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:15.015 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.015 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.015 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:15.015 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.068300+0000", 2024-09-20T22:19:15.015 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4099999999999998e-07 2024-09-20T22:19:15.015 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.015 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.015 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:15.015 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.068303+0000", 2024-09-20T22:19:15.015 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5039999999999998e-06 2024-09-20T22:19:15.015 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.015 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.015 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:15.015 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.326972+0000", 2024-09-20T22:19:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.25866898900000002 2024-09-20T22:19:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.326997+0000", 2024-09-20T22:19:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.4459000000000001e-05 2024-09-20T22:19:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.328334+0000", 2024-09-20T22:19:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001336926 2024-09-20T22:19:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:15.016 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.328339+0000", 2024-09-20T22:19:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2510000000000002e-06 2024-09-20T22:19:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.328349+0000", 2024-09-20T22:19:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0589e-05 2024-09-20T22:19:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.16440.0:41046 11.5 e465/456 11:ac0e3353:::benchmark_data_smithi060_196073_object41045:head v 465'2611, mlcod=465'2608)", 2024-09-20T22:19:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:19:12.074380+0000", 2024-09-20T22:19:15.017 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 2.8184891109999999, 2024-09-20T22:19:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.25686124399999999, 2024-09-20T22:19:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.074380+0000", 2024-09-20T22:19:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.074380+0000", 2024-09-20T22:19:15.018 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.074381+0000", 2024-09-20T22:19:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0270000000000001e-06 2024-09-20T22:19:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.074417+0000", 2024-09-20T22:19:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6362999999999997e-05 2024-09-20T22:19:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.326975+0000", 2024-09-20T22:19:15.019 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.25255802500000002 2024-09-20T22:19:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.326992+0000", 2024-09-20T22:19:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6354999999999998e-05 2024-09-20T22:19:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.328564+0000", 2024-09-20T22:19:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0015728840000000001 2024-09-20T22:19:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.328581+0000", 2024-09-20T22:19:15.020 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6616000000000002e-05 2024-09-20T22:19:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.331177+0000", 2024-09-20T22:19:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0025958969999999998 2024-09-20T22:19:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.331241+0000", 2024-09-20T22:19:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4077000000000003e-05 2024-09-20T22:19:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:15.021 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:15.022 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:15.022 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:19:15.022 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:19:15.022 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:19:15.040 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:15.082 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:15.083 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:15.083 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:15.099 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 22 16 45556 45540 129.344 97.25 0.00653209 0.00772586 2024-09-20T22:19:15.184 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:15.215 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:15.216 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:15.216 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:15.317 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:15.331 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:19:15.354 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:15.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:15.355 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:15.397 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:19:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:19:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:19:15.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:15.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:15.526 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:15.526 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:15.627 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:15.655 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:15.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:15.657 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:15.757 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:15.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:15.793 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:15.793 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:15.894 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:15.932 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:15.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:15.933 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:15.987 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:15 smithi067 ceph-mon[33511]: 1.c scrub starts 2024-09-20T22:19:15.987 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:15 smithi067 ceph-mon[33511]: 1.c scrub ok 2024-09-20T22:19:15.987 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:15 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]': finished 2024-09-20T22:19:15.987 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:15 smithi067 ceph-mon[33511]: osdmap e466: 8 total, 8 up, 8 in 2024-09-20T22:19:15.987 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:15 smithi067 ceph-mon[33511]: 4.18 scrub starts 2024-09-20T22:19:15.987 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:15 smithi067 ceph-mon[33511]: 4.18 scrub ok 2024-09-20T22:19:16.034 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:16.036 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T22:19:16.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:15 smithi060 ceph-mon[29694]: 1.c scrub starts 2024-09-20T22:19:16.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:15 smithi060 ceph-mon[29694]: 1.c scrub ok 2024-09-20T22:19:16.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:15 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]': finished 2024-09-20T22:19:16.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:15 smithi060 ceph-mon[29694]: osdmap e466: 8 total, 8 up, 8 in 2024-09-20T22:19:16.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:15 smithi060 ceph-mon[29694]: 4.18 scrub starts 2024-09-20T22:19:16.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:15 smithi060 ceph-mon[29694]: 4.18 scrub ok 2024-09-20T22:19:16.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:15 smithi060 ceph-mon[40290]: 1.c scrub starts 2024-09-20T22:19:16.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:15 smithi060 ceph-mon[40290]: 1.c scrub ok 2024-09-20T22:19:16.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:15 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "unique_pool_4", "var": "pgp_num", "val": "16"}]': finished 2024-09-20T22:19:16.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:15 smithi060 ceph-mon[40290]: osdmap e466: 8 total, 8 up, 8 in 2024-09-20T22:19:16.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:15 smithi060 ceph-mon[40290]: 4.18 scrub starts 2024-09-20T22:19:16.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:15 smithi060 ceph-mon[40290]: 4.18 scrub ok 2024-09-20T22:19:16.072 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:16.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:16.073 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:16.099 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 23 16 47166 47150 128.095 100.625 0.00529387 0.00780103 2024-09-20T22:19:16.174 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:16.211 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:16.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:16.212 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:16.312 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:16.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:16.358 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:16.358 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:16.435 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:19:16.456 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:16.459 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:16.499 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:16.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:16.500 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:16.601 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:16.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:16.633 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:16.634 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:16.734 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:16.765 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:16.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:16.766 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:16.835 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:16 smithi067 ceph-mon[33511]: 4.14 deep-scrub starts 2024-09-20T22:19:16.835 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:16 smithi067 ceph-mon[33511]: 4.14 deep-scrub ok 2024-09-20T22:19:16.835 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:16 smithi067 ceph-mon[33511]: pgmap v874: 178 pgs: 178 active+clean; 2.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.5 KiB/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-20T22:19:16.835 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:16 smithi067 ceph-mon[33511]: osdmap e467: 8 total, 8 up, 8 in 2024-09-20T22:19:16.867 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:16.890 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:16.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:16.891 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:16.991 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:17.024 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:17.025 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:17.026 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:17.032 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:16 smithi060 ceph-mon[29694]: 4.14 deep-scrub starts 2024-09-20T22:19:17.032 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:16 smithi060 ceph-mon[29694]: 4.14 deep-scrub ok 2024-09-20T22:19:17.032 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:16 smithi060 ceph-mon[29694]: pgmap v874: 178 pgs: 178 active+clean; 2.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.5 KiB/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-20T22:19:17.032 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:16 smithi060 ceph-mon[29694]: osdmap e467: 8 total, 8 up, 8 in 2024-09-20T22:19:17.032 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:16 smithi060 ceph-mon[40290]: 4.14 deep-scrub starts 2024-09-20T22:19:17.032 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:16 smithi060 ceph-mon[40290]: 4.14 deep-scrub ok 2024-09-20T22:19:17.032 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:16 smithi060 ceph-mon[40290]: pgmap v874: 178 pgs: 178 active+clean; 2.6 GiB data, 13 GiB used, 702 GiB / 715 GiB avail; 1.5 KiB/s rd, 92 MiB/s wr, 1.47k op/s 2024-09-20T22:19:17.032 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:16 smithi060 ceph-mon[40290]: osdmap e467: 8 total, 8 up, 8 in 2024-09-20T22:19:17.099 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 24 16 48756 48740 126.898 99.375 0.00506502 0.00787522 2024-09-20T22:19:17.127 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:17.162 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:17.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:17.163 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:17.264 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:17.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:17.287 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:17.288 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:17.388 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:17.420 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:17.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:17.422 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:17.523 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:17.558 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:17.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:17.559 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:17.660 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:17.691 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:17.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:17.692 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:17.793 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:17.824 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:17.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:17.825 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:17.926 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:17.948 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:17.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:17.949 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:17 smithi067 ceph-mon[33511]: 4.a scrub starts 2024-09-20T22:19:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:17 smithi067 ceph-mon[33511]: 4.a scrub ok 2024-09-20T22:19:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:17 smithi067 ceph-mon[33511]: 5.9 scrub starts 2024-09-20T22:19:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:17 smithi067 ceph-mon[33511]: 5.9 scrub ok 2024-09-20T22:19:18.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:17 smithi060 ceph-mon[29694]: 4.a scrub starts 2024-09-20T22:19:18.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:17 smithi060 ceph-mon[29694]: 4.a scrub ok 2024-09-20T22:19:18.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:17 smithi060 ceph-mon[29694]: 5.9 scrub starts 2024-09-20T22:19:18.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:17 smithi060 ceph-mon[29694]: 5.9 scrub ok 2024-09-20T22:19:18.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:17 smithi060 ceph-mon[40290]: 4.a scrub starts 2024-09-20T22:19:18.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:17 smithi060 ceph-mon[40290]: 4.a scrub ok 2024-09-20T22:19:18.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:17 smithi060 ceph-mon[40290]: 5.9 scrub starts 2024-09-20T22:19:18.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:17 smithi060 ceph-mon[40290]: 5.9 scrub ok 2024-09-20T22:19:18.049 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:18.089 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:18.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:18.090 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:18.099 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 25 16 50073 50057 125.114 82.3125 0.00820496 0.00798751 2024-09-20T22:19:18.190 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:18.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:18.212 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:18.212 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:18.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:18.350 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:18.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:18.352 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:18.452 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:18.474 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:18.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:18.474 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:18.575 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:18.612 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:18.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:18.613 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:18.714 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:18.737 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:19:18.739 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:18.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:18.740 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:18.841 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:18.877 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:18.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:18.878 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:18.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:19:18.979 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:18.993 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[29694]: 3.8 deep-scrub starts 2024-09-20T22:19:18.993 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[29694]: 3.8 deep-scrub ok 2024-09-20T22:19:18.993 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[29694]: 3.6 scrub starts 2024-09-20T22:19:18.993 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[29694]: 3.6 scrub ok 2024-09-20T22:19:18.993 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1406739080' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:19:18.993 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[29694]: 4.11 scrub starts 2024-09-20T22:19:18.993 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[29694]: 4.11 scrub ok 2024-09-20T22:19:18.993 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[29694]: pgmap v876: 178 pgs: 178 active+clean; 2.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 765 B/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-20T22:19:18.993 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[29694]: 4.13 scrub starts 2024-09-20T22:19:18.994 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[29694]: 4.13 scrub ok 2024-09-20T22:19:18.994 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[40290]: 3.8 deep-scrub starts 2024-09-20T22:19:18.994 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[40290]: 3.8 deep-scrub ok 2024-09-20T22:19:18.994 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[40290]: 3.6 scrub starts 2024-09-20T22:19:18.994 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[40290]: 3.6 scrub ok 2024-09-20T22:19:18.994 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1406739080' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:19:18.994 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[40290]: 4.11 scrub starts 2024-09-20T22:19:18.994 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[40290]: 4.11 scrub ok 2024-09-20T22:19:18.994 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[40290]: pgmap v876: 178 pgs: 178 active+clean; 2.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 765 B/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-20T22:19:18.994 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[40290]: 4.13 scrub starts 2024-09-20T22:19:18.994 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:18 smithi060 ceph-mon[40290]: 4.13 scrub ok 2024-09-20T22:19:19.001 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:19.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:19.002 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:19.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:18 smithi067 ceph-mon[33511]: 3.8 deep-scrub starts 2024-09-20T22:19:19.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:18 smithi067 ceph-mon[33511]: 3.8 deep-scrub ok 2024-09-20T22:19:19.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:18 smithi067 ceph-mon[33511]: 3.6 scrub starts 2024-09-20T22:19:19.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:18 smithi067 ceph-mon[33511]: 3.6 scrub ok 2024-09-20T22:19:19.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:18 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1406739080' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:19:19.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:18 smithi067 ceph-mon[33511]: 4.11 scrub starts 2024-09-20T22:19:19.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:18 smithi067 ceph-mon[33511]: 4.11 scrub ok 2024-09-20T22:19:19.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:18 smithi067 ceph-mon[33511]: pgmap v876: 178 pgs: 178 active+clean; 2.9 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 765 B/s rd, 103 MiB/s wr, 1.65k op/s 2024-09-20T22:19:19.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:18 smithi067 ceph-mon[33511]: 4.13 scrub starts 2024-09-20T22:19:19.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:18 smithi067 ceph-mon[33511]: 4.13 scrub ok 2024-09-20T22:19:19.092 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:19:19.092 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:19:19.099 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 26 16 51100 51084 122.771 64.1875 0.00902325 0.00813942 2024-09-20T22:19:19.103 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:19.129 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:19.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:19.130 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:19.231 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:19.275 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:19.276 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:19.276 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:19.377 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:19.422 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:19.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:19.423 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:19.478 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:19.523 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:19.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:19.567 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:19.567 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:19.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:19.701 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:19.702 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:19.702 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:19.802 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:19.841 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:19.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:19.842 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:19.942 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:19.946 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T22:19:19.965 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:19 smithi060 ceph-mon[29694]: 5.2 scrub starts 2024-09-20T22:19:19.966 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:19 smithi060 ceph-mon[29694]: 5.2 scrub ok 2024-09-20T22:19:19.966 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:19 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:19:19.966 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:19 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1406739080' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:19:19.966 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:19 smithi060 ceph-mon[29694]: osdmap e468: 8 total, 8 up, 8 in 2024-09-20T22:19:19.966 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:19 smithi060 ceph-mon[40290]: 5.2 scrub starts 2024-09-20T22:19:19.966 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:19 smithi060 ceph-mon[40290]: 5.2 scrub ok 2024-09-20T22:19:19.966 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:19 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:19:19.966 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:19 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1406739080' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:19:19.966 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:19 smithi060 ceph-mon[40290]: osdmap e468: 8 total, 8 up, 8 in 2024-09-20T22:19:19.977 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:19.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:19.978 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:20.011 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:19 smithi067 ceph-mon[33511]: 5.2 scrub starts 2024-09-20T22:19:20.011 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:19 smithi067 ceph-mon[33511]: 5.2 scrub ok 2024-09-20T22:19:20.011 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:19 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:19:20.011 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:19 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1406739080' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:19:20.011 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:19 smithi067 ceph-mon[33511]: osdmap e468: 8 total, 8 up, 8 in 2024-09-20T22:19:20.079 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:20.100 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 27 16 52587 52571 121.666 92.9375 0.00645924 0.00821354 2024-09-20T22:19:20.120 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:20.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:20.121 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:20.144 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:19:20.145 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:19:20.186 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:19:20.186 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:19:20.220 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:19:20.220 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:19:20.222 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:20.259 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:20.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:20.260 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:20.263 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:19:20.263 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:19:20.295 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:19:20.295 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:19:20.329 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:19:20.330 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:19:20.361 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:20.370 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:19:20.371 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:19:20.398 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:19:20.399 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:20.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:20.400 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:20.407 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:19:20.407 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:19:20.502 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:20.534 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:20.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:20.535 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:20.636 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:20.674 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:20.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:20.675 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:20.776 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:20.833 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:20.834 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:20.834 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:20.935 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:20.975 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:20.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:20.976 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:20 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/930738238' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:20 smithi067 ceph-mon[33511]: pgmap v878: 178 pgs: 178 active+clean; 3.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-20T22:19:21.035 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:20 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/930738238' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:21.035 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:20 smithi060 ceph-mon[40290]: pgmap v878: 178 pgs: 178 active+clean; 3.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-20T22:19:21.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:20 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/930738238' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:21.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:20 smithi060 ceph-mon[29694]: pgmap v878: 178 pgs: 178 active+clean; 3.0 GiB data, 14 GiB used, 701 GiB / 715 GiB avail; 852 B/s rd, 80 MiB/s wr, 1.27k op/s 2024-09-20T22:19:21.077 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:21.099 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:21.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:21.100 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:21.100 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 28 16 54033 54017 120.548 90.375 0.00281942 0.00826968 2024-09-20T22:19:21.201 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:21.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:21.232 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:21.233 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:21.333 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:21.383 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:21.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:21.384 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:21.434 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:19:21.434 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:19:21.434 INFO:tasks.thrashosds.thrasher:Removing osd 4, in_osds are: [1, 2, 3, 4, 5, 6, 7, 0] 2024-09-20T22:19:21.434 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd out 4 2024-09-20T22:19:21.437 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:19:21.484 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:21.500 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:19:21.500 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:19:21.500 INFO:teuthology.orchestra.run.smithi067.stdout: "complaint_time": 30, 2024-09-20T22:19:21.500 INFO:teuthology.orchestra.run.smithi067.stdout: "num_blocked_ops": 0 2024-09-20T22:19:21.500 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:19:21.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:21.574 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:21.574 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:21.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:21.748 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:21.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:21.750 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:21.850 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:21.896 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:21.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:21.899 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:22.000 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:19:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:21 smithi067 ceph-mon[33511]: osdmap e469: 8 total, 8 up, 8 in 2024-09-20T22:19:22.026 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:22.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:22.028 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:22.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:19:22.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:21 smithi060 ceph-mon[29694]: osdmap e469: 8 total, 8 up, 8 in 2024-09-20T22:19:22.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:19:22.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:21 smithi060 ceph-mon[40290]: osdmap e469: 8 total, 8 up, 8 in 2024-09-20T22:19:22.100 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 29 16 55100 55084 118.691 66.6875 0.00604058 0.00841945 2024-09-20T22:19:22.128 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:22.162 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:22.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:22.163 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:22.263 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:22.296 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:22.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:22.298 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:22.369 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T22:19:22.399 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:22.434 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:22.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:22.435 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:22.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:22.577 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:22.578 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:22.578 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:22.679 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:22.713 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:22.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:22.714 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:22.733 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:19:22.816 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:22.848 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:22.849 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:22.849 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:22.950 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:22.990 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:22.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:22.991 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:23.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:22 smithi067 ceph-mon[33511]: pgmap v880: 178 pgs: 178 active+clean; 3.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-20T22:19:23.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:19:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:19:23.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:22 smithi060 ceph-mon[29694]: pgmap v880: 178 pgs: 178 active+clean; 3.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-20T22:19:23.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:22 smithi060 ceph-mon[40290]: pgmap v880: 178 pgs: 178 active+clean; 3.2 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 853 B/s rd, 99 MiB/s wr, 1.58k op/s 2024-09-20T22:19:23.092 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:23.100 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 30 16 56522 56506 117.697 88.875 0.0317616 0.0084847 2024-09-20T22:19:23.128 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:23.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:23.129 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:23.230 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:23.267 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:23.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:23.268 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:23.369 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:23.391 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:23.391 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:23.392 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:23.406 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:19:23.407 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:19:23.407 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:19:23.407 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:19:23.407 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.407 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:187857 8.9 8:977b7bdd:::benchmark_data_smithi060_111051_object67858:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e250)", 2024-09-20T22:19:23.407 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:29.640677+0000", 2024-09-20T22:19:23.407 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 533.736544794, 2024-09-20T22:19:23.407 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.102806486, 2024-09-20T22:19:23.407 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.407 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.407 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.407 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.407 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:19:23.407 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:19:23.408 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 187857 2024-09-20T22:19:23.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.408 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.408 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.408 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.640677+0000", 2024-09-20T22:19:23.408 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.408 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.408 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.640677+0000", 2024-09-20T22:19:23.408 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.408 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.408 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.408 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.409 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.640679+0000", 2024-09-20T22:19:23.409 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3430000000000001e-06 2024-09-20T22:19:23.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.409 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.409 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.640682+0000", 2024-09-20T22:19:23.409 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.557e-06 2024-09-20T22:19:23.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.409 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.409 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.640691+0000", 2024-09-20T22:19:23.409 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1710000000000008e-06 2024-09-20T22:19:23.409 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.409 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.410 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.410 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.640699+0000", 2024-09-20T22:19:23.410 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5059999999999998e-06 2024-09-20T22:19:23.410 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.410 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.410 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.410 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.643753+0000", 2024-09-20T22:19:23.410 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0030544809999999999 2024-09-20T22:19:23.410 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.410 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.410 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:23.410 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.643766+0000", 2024-09-20T22:19:23.410 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2227e-05 2024-09-20T22:19:23.410 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.410 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.411 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.411 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:30.635192+0000", 2024-09-20T22:19:23.411 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.991425899 2024-09-20T22:19:23.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.411 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.411 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:23.411 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:30.635203+0000", 2024-09-20T22:19:23.411 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0957e-05 2024-09-20T22:19:23.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.411 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.411 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.411 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.735750+0000", 2024-09-20T22:19:23.411 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.10054739 2024-09-20T22:19:23.411 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.411 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.412 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.412 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.735843+0000", 2024-09-20T22:19:23.412 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2980999999999996e-05 2024-09-20T22:19:23.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.412 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.412 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3,5", 2024-09-20T22:19:23.412 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.736139+0000", 2024-09-20T22:19:23.412 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000296104 2024-09-20T22:19:23.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.412 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.412 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.412 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.742910+0000", 2024-09-20T22:19:23.412 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0067707760000000001 2024-09-20T22:19:23.412 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.412 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.413 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.413 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.742932+0000", 2024-09-20T22:19:23.413 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2181e-05 2024-09-20T22:19:23.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.413 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.413 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.413 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743233+0000", 2024-09-20T22:19:23.413 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00030141400000000001 2024-09-20T22:19:23.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.413 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.413 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.413 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743293+0000", 2024-09-20T22:19:23.413 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9304e-05 2024-09-20T22:19:23.413 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.413 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.413 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.414 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743350+0000", 2024-09-20T22:19:23.414 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7015999999999998e-05 2024-09-20T22:19:23.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.414 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.414 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.414 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743484+0000", 2024-09-20T22:19:23.414 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000134179 2024-09-20T22:19:23.414 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.414 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.414 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.414 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.414 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.414 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:187866 8.9 8:90a8fa22:::benchmark_data_smithi060_111051_object67867:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:19:23.414 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:29.665081+0000", 2024-09-20T22:19:23.414 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 533.71214100899999, 2024-09-20T22:19:23.414 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0789447289999998, 2024-09-20T22:19:23.414 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.415 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.415 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.415 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.415 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:19:23.415 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:19:23.415 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 187866 2024-09-20T22:19:23.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.415 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.415 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.415 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.415 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665081+0000", 2024-09-20T22:19:23.415 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.415 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.415 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.415 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.415 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665081+0000", 2024-09-20T22:19:23.416 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.416 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.416 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.416 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665082+0000", 2024-09-20T22:19:23.416 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9800000000000002e-07 2024-09-20T22:19:23.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.416 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.416 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.416 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665086+0000", 2024-09-20T22:19:23.416 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.36e-06 2024-09-20T22:19:23.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.416 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.416 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.416 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665096+0000", 2024-09-20T22:19:23.416 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0755e-05 2024-09-20T22:19:23.416 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.417 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.417 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.417 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665104+0000", 2024-09-20T22:19:23.417 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6029999999999999e-06 2024-09-20T22:19:23.417 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.417 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.417 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.417 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665150+0000", 2024-09-20T22:19:23.417 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6409000000000002e-05 2024-09-20T22:19:23.417 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.417 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.417 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:23.417 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.665158+0000", 2024-09-20T22:19:23.417 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9039999999999992e-06 2024-09-20T22:19:23.417 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.417 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.417 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.418 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:30.635228+0000", 2024-09-20T22:19:23.418 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.97006970000000003 2024-09-20T22:19:23.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.418 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.418 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:23.418 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:30.635238+0000", 2024-09-20T22:19:23.418 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0108999999999999e-05 2024-09-20T22:19:23.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.418 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.418 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.418 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.739124+0000", 2024-09-20T22:19:23.418 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1038857790000001 2024-09-20T22:19:23.418 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.418 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.418 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.419 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.739264+0000", 2024-09-20T22:19:23.419 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014064700000000001 2024-09-20T22:19:23.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.419 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.419 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3,5", 2024-09-20T22:19:23.419 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.739542+0000", 2024-09-20T22:19:23.419 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00027733800000000002 2024-09-20T22:19:23.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.419 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.419 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.419 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.742938+0000", 2024-09-20T22:19:23.419 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0033961389999999998 2024-09-20T22:19:23.419 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.419 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743151+0000", 2024-09-20T22:19:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000212592 2024-09-20T22:19:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743850+0000", 2024-09-20T22:19:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00069932800000000004 2024-09-20T22:19:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743933+0000", 2024-09-20T22:19:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2720999999999996e-05 2024-09-20T22:19:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.420 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743965+0000", 2024-09-20T22:19:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2230000000000001e-05 2024-09-20T22:19:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.744026+0000", 2024-09-20T22:19:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1117000000000005e-05 2024-09-20T22:19:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:187902 8.9 8:95e80f7a:::benchmark_data_smithi060_111051_object67903:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e251)", 2024-09-20T22:19:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:10:29.720026+0000", 2024-09-20T22:19:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 533.65719646800005, 2024-09-20T22:19:23.421 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.024196764, 2024-09-20T22:19:23.422 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.422 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.422 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.422 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.422 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:19:23.422 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:19:23.422 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 187902 2024-09-20T22:19:23.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.422 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.422 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.422 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720026+0000", 2024-09-20T22:19:23.422 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.422 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.422 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.423 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.423 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720026+0000", 2024-09-20T22:19:23.423 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.423 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.423 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.423 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720027+0000", 2024-09-20T22:19:23.423 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.093e-06 2024-09-20T22:19:23.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.423 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.423 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.423 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720031+0000", 2024-09-20T22:19:23.423 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0720000000000001e-06 2024-09-20T22:19:23.423 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.424 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.424 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.424 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720042+0000", 2024-09-20T22:19:23.424 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1280999999999999e-05 2024-09-20T22:19:23.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.424 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.424 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.424 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720048+0000", 2024-09-20T22:19:23.424 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3870000000000003e-06 2024-09-20T22:19:23.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.424 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.424 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.424 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720098+0000", 2024-09-20T22:19:23.424 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0597999999999999e-05 2024-09-20T22:19:23.424 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.424 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.425 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:23.425 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:29.720107+0000", 2024-09-20T22:19:23.425 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3510000000000002e-06 2024-09-20T22:19:23.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.425 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.425 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:30.635262+0000", 2024-09-20T22:19:23.425 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.91515579300000005 2024-09-20T22:19:23.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.425 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:23.425 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:30.635270+0000", 2024-09-20T22:19:23.425 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.994e-06 2024-09-20T22:19:23.425 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.425 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.425 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.426 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.740932+0000", 2024-09-20T22:19:23.426 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.105662157 2024-09-20T22:19:23.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.426 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.426 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.426 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.741060+0000", 2024-09-20T22:19:23.426 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012800800000000001 2024-09-20T22:19:23.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.426 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.426 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3,5", 2024-09-20T22:19:23.426 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.741319+0000", 2024-09-20T22:19:23.426 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00025847700000000001 2024-09-20T22:19:23.426 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.426 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.426 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.427 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743179+0000", 2024-09-20T22:19:23.427 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0018601119999999999 2024-09-20T22:19:23.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.427 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.427 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.427 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743556+0000", 2024-09-20T22:19:23.427 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00037664000000000003 2024-09-20T22:19:23.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.427 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.427 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.427 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.743893+0000", 2024-09-20T22:19:23.427 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00033714699999999999 2024-09-20T22:19:23.427 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.427 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.427 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.428 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.744064+0000", 2024-09-20T22:19:23.428 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017144400000000001 2024-09-20T22:19:23.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.428 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.428 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.428 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.744116+0000", 2024-09-20T22:19:23.428 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1873999999999998e-05 2024-09-20T22:19:23.428 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.428 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.428 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.428 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:10:31.744223+0000", 2024-09-20T22:19:23.428 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010633600000000001 2024-09-20T22:19:23.428 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.428 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.428 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.429 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.429 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223308 8.9 8:97e7b109:::benchmark_data_smithi060_111051_object103309:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:19:23.429 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.506871+0000", 2024-09-20T22:19:23.429 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 496.870350938, 2024-09-20T22:19:23.429 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.99735096099999998, 2024-09-20T22:19:23.429 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.429 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.429 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.429 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.429 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:19:23.429 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:19:23.429 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223308 2024-09-20T22:19:23.429 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.429 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.429 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.429 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.430 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506871+0000", 2024-09-20T22:19:23.430 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.430 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.430 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.430 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506871+0000", 2024-09-20T22:19:23.430 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.430 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.430 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.430 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506873+0000", 2024-09-20T22:19:23.430 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0470000000000001e-06 2024-09-20T22:19:23.430 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.430 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.430 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.430 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506882+0000", 2024-09-20T22:19:23.431 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3310000000000004e-06 2024-09-20T22:19:23.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.431 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.431 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506904+0000", 2024-09-20T22:19:23.431 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.283e-05 2024-09-20T22:19:23.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.431 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.431 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506913+0000", 2024-09-20T22:19:23.431 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9530000000000005e-06 2024-09-20T22:19:23.431 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.431 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.431 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.431 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506975+0000", 2024-09-20T22:19:23.432 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1773000000000006e-05 2024-09-20T22:19:23.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.432 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:23.432 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.506982+0000", 2024-09-20T22:19:23.432 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2869999999999999e-06 2024-09-20T22:19:23.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.432 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.432 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.494954+0000", 2024-09-20T22:19:23.432 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98797138799999995 2024-09-20T22:19:23.432 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.432 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.432 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.432 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.495095+0000", 2024-09-20T22:19:23.433 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014108 2024-09-20T22:19:23.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.433 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.433 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:19:23.433 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.495506+0000", 2024-09-20T22:19:23.433 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00041153700000000001 2024-09-20T22:19:23.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.433 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.433 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.433 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.500203+0000", 2024-09-20T22:19:23.433 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046968380000000001 2024-09-20T22:19:23.433 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.433 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.433 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.433 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.500238+0000", 2024-09-20T22:19:23.434 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4195e-05 2024-09-20T22:19:23.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.434 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.434 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.504167+0000", 2024-09-20T22:19:23.434 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0039294819999999998 2024-09-20T22:19:23.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.434 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.434 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.504199+0000", 2024-09-20T22:19:23.434 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1853999999999997e-05 2024-09-20T22:19:23.434 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.434 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.434 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.434 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.504222+0000", 2024-09-20T22:19:23.434 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.3366e-05 2024-09-20T22:19:23.435 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.435 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.435 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.435 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.435 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.435 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223310 8.9 8:94027251:::benchmark_data_smithi060_111051_object103311:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:19:23.435 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.507858+0000", 2024-09-20T22:19:23.435 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 496.86936434500001, 2024-09-20T22:19:23.435 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0231674930000001, 2024-09-20T22:19:23.435 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.435 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.435 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.435 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.435 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:19:23.435 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:19:23.436 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223310 2024-09-20T22:19:23.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.436 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.436 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.436 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507858+0000", 2024-09-20T22:19:23.436 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.436 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.436 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507858+0000", 2024-09-20T22:19:23.436 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.436 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.436 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.436 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.436 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507859+0000", 2024-09-20T22:19:23.437 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.78e-07 2024-09-20T22:19:23.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.437 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.437 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.437 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507864+0000", 2024-09-20T22:19:23.437 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8969999999999999e-06 2024-09-20T22:19:23.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.437 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.437 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.437 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507879+0000", 2024-09-20T22:19:23.437 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4919999999999999e-05 2024-09-20T22:19:23.437 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.437 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.437 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.437 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507886+0000", 2024-09-20T22:19:23.438 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2119999999999997e-06 2024-09-20T22:19:23.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.438 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.438 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.438 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507948+0000", 2024-09-20T22:19:23.438 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1764e-05 2024-09-20T22:19:23.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.438 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.438 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:23.438 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.507959+0000", 2024-09-20T22:19:23.438 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1613999999999999e-05 2024-09-20T22:19:23.438 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.438 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.438 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.438 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.497481+0000", 2024-09-20T22:19:23.439 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.98952213899999997 2024-09-20T22:19:23.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.439 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.439 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.497555+0000", 2024-09-20T22:19:23.439 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4202000000000005e-05 2024-09-20T22:19:23.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.439 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:19:23.439 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.497785+0000", 2024-09-20T22:19:23.439 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00022984599999999999 2024-09-20T22:19:23.439 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.439 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.439 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.501438+0000", 2024-09-20T22:19:23.439 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0036526850000000001 2024-09-20T22:19:23.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.440 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.440 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.504265+0000", 2024-09-20T22:19:23.440 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002827319 2024-09-20T22:19:23.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.440 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.440 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.530930+0000", 2024-09-20T22:19:23.440 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.026664383999999999 2024-09-20T22:19:23.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.440 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.440 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.530981+0000", 2024-09-20T22:19:23.440 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1109999999999997e-05 2024-09-20T22:19:23.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.440 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.441 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531025+0000", 2024-09-20T22:19:23.441 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4523e-05 2024-09-20T22:19:23.441 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.441 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.441 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.441 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.441 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223355 8.9 8:94b7fbf7:::benchmark_data_smithi060_111051_object103356:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:19:23.441 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.561400+0000", 2024-09-20T22:19:23.441 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 496.81582256000002, 2024-09-20T22:19:23.441 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.96979188400000005, 2024-09-20T22:19:23.441 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.441 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.441 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.441 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.441 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:19:23.442 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:19:23.442 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223355 2024-09-20T22:19:23.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.442 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.442 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.442 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561400+0000", 2024-09-20T22:19:23.442 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.442 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.442 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561400+0000", 2024-09-20T22:19:23.442 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.442 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.443 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561401+0000", 2024-09-20T22:19:23.443 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3790000000000001e-06 2024-09-20T22:19:23.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.443 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.443 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561406+0000", 2024-09-20T22:19:23.443 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4559999999999999e-06 2024-09-20T22:19:23.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.443 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.443 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561419+0000", 2024-09-20T22:19:23.443 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2712e-05 2024-09-20T22:19:23.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.443 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.444 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561426+0000", 2024-09-20T22:19:23.444 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6080000000000001e-06 2024-09-20T22:19:23.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.444 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.444 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561475+0000", 2024-09-20T22:19:23.444 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9412000000000001e-05 2024-09-20T22:19:23.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.444 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:23.444 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.561482+0000", 2024-09-20T22:19:23.444 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7789999999999998e-06 2024-09-20T22:19:23.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.444 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.445 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.499040+0000", 2024-09-20T22:19:23.445 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93755812199999999 2024-09-20T22:19:23.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.445 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.445 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.499105+0000", 2024-09-20T22:19:23.445 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5328999999999997e-05 2024-09-20T22:19:23.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.445 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:19:23.445 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.499270+0000", 2024-09-20T22:19:23.445 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000164678 2024-09-20T22:19:23.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.445 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.446 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.504133+0000", 2024-09-20T22:19:23.446 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0048630009999999996 2024-09-20T22:19:23.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.446 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.446 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.511137+0000", 2024-09-20T22:19:23.446 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.007004316 2024-09-20T22:19:23.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.446 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.446 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531092+0000", 2024-09-20T22:19:23.446 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.019954896999999999 2024-09-20T22:19:23.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.446 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.446 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531148+0000", 2024-09-20T22:19:23.447 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5828999999999997e-05 2024-09-20T22:19:23.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.447 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.447 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.447 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531192+0000", 2024-09-20T22:19:23.447 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3365999999999998e-05 2024-09-20T22:19:23.447 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.447 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.447 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.447 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.447 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15192.0:223360 8.9 8:963e2bee:::benchmark_data_smithi060_111051_object103361:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e272)", 2024-09-20T22:19:23.447 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:11:06.566084+0000", 2024-09-20T22:19:23.447 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 496.81113788200003, 2024-09-20T22:19:23.448 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.965237028, 2024-09-20T22:19:23.448 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.448 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.448 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.448 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.448 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15192", 2024-09-20T22:19:23.448 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:19:23.448 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 223360 2024-09-20T22:19:23.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.448 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.448 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.448 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566084+0000", 2024-09-20T22:19:23.448 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.449 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.449 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566084+0000", 2024-09-20T22:19:23.449 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.449 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.449 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566086+0000", 2024-09-20T22:19:23.449 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.175e-06 2024-09-20T22:19:23.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.449 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.449 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566090+0000", 2024-09-20T22:19:23.449 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2080000000000002e-06 2024-09-20T22:19:23.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.450 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566099+0000", 2024-09-20T22:19:23.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3559999999999999e-06 2024-09-20T22:19:23.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.450 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566105+0000", 2024-09-20T22:19:23.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8649999999999998e-06 2024-09-20T22:19:23.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.450 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566150+0000", 2024-09-20T22:19:23.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5252e-05 2024-09-20T22:19:23.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.451 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:23.451 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:06.566156+0000", 2024-09-20T22:19:23.451 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7370000000000003e-06 2024-09-20T22:19:23.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.451 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.451 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.500259+0000", 2024-09-20T22:19:23.451 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.93410321200000002 2024-09-20T22:19:23.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.451 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.451 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.500327+0000", 2024-09-20T22:19:23.451 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8270999999999997e-05 2024-09-20T22:19:23.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.451 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:19:23.451 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.500502+0000", 2024-09-20T22:19:23.452 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000174797 2024-09-20T22:19:23.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.452 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.452 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.504273+0000", 2024-09-20T22:19:23.452 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003771019 2024-09-20T22:19:23.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.452 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.452 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.511209+0000", 2024-09-20T22:19:23.452 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0069359080000000002 2024-09-20T22:19:23.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.452 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.453 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531241+0000", 2024-09-20T22:19:23.453 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.020031694999999999 2024-09-20T22:19:23.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.453 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.453 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.453 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531287+0000", 2024-09-20T22:19:23.453 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6060000000000003e-05 2024-09-20T22:19:23.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.453 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.453 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.453 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:11:07.531321+0000", 2024-09-20T22:19:23.453 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4473e-05 2024-09-20T22:19:23.453 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.453 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.453 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.454 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.454 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:62123 9.1 9:879a37c5:::benchmark_data_smithi060_140132_object62122:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e309)", 2024-09-20T22:19:23.454 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:12:20.517780+0000", 2024-09-20T22:19:23.454 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 422.85944251799998, 2024-09-20T22:19:23.454 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0093993530000001, 2024-09-20T22:19:23.454 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.454 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.454 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:19:23.454 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.454 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:19:23.454 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:19:23.454 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 62123 2024-09-20T22:19:23.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.454 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.454 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.454 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.455 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.517780+0000", 2024-09-20T22:19:23.455 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.455 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.455 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.517780+0000", 2024-09-20T22:19:23.455 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.455 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.455 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.517781+0000", 2024-09-20T22:19:23.455 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.486e-06 2024-09-20T22:19:23.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.455 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.455 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.517964+0000", 2024-09-20T22:19:23.456 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000182542 2024-09-20T22:19:23.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.456 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.456 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.517976+0000", 2024-09-20T22:19:23.456 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2031999999999999e-05 2024-09-20T22:19:23.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.456 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.456 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.517984+0000", 2024-09-20T22:19:23.456 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2859999999999999e-06 2024-09-20T22:19:23.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.456 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.456 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.518698+0000", 2024-09-20T22:19:23.456 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00071389999999999995 2024-09-20T22:19:23.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.457 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:23.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.518708+0000", 2024-09-20T22:19:23.457 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0176999999999999e-05 2024-09-20T22:19:23.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.457 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.526085+0000", 2024-09-20T22:19:23.457 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.007377027 2024-09-20T22:19:23.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.457 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.527179+0000", 2024-09-20T22:19:23.457 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001093903 2024-09-20T22:19:23.457 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.457 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.458 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.458 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:62141 9.1 9:8c9cf8d8:::benchmark_data_smithi060_140132_object62140:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e309)", 2024-09-20T22:19:23.458 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:12:20.531998+0000", 2024-09-20T22:19:23.458 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 422.84522419199999, 2024-09-20T22:19:23.458 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0013003810000001, 2024-09-20T22:19:23.458 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.458 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.458 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.458 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.458 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:19:23.458 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:19:23.458 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 62141 2024-09-20T22:19:23.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.458 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.459 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.459 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.531998+0000", 2024-09-20T22:19:23.459 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.459 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.459 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.531998+0000", 2024-09-20T22:19:23.459 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.459 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.459 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.532000+0000", 2024-09-20T22:19:23.459 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5719999999999999e-06 2024-09-20T22:19:23.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.460 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.460 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.532075+0000", 2024-09-20T22:19:23.460 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5104999999999993e-05 2024-09-20T22:19:23.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.460 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.460 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.532087+0000", 2024-09-20T22:19:23.460 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1990999999999999e-05 2024-09-20T22:19:23.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.460 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.460 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.532096+0000", 2024-09-20T22:19:23.460 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4110000000000002e-06 2024-09-20T22:19:23.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.460 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.461 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.532156+0000", 2024-09-20T22:19:23.461 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0000999999999997e-05 2024-09-20T22:19:23.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.461 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:23.461 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.532168+0000", 2024-09-20T22:19:23.461 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2194e-05 2024-09-20T22:19:23.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.461 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.461 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.527535+0000", 2024-09-20T22:19:23.461 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.99536664699999999 2024-09-20T22:19:23.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.461 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.461 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.527885+0000", 2024-09-20T22:19:23.462 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00034985700000000002 2024-09-20T22:19:23.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.462 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:19:23.462 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.527994+0000", 2024-09-20T22:19:23.462 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000109628 2024-09-20T22:19:23.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.462 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.462 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533024+0000", 2024-09-20T22:19:23.462 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.005029342 2024-09-20T22:19:23.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.462 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.462 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533086+0000", 2024-09-20T22:19:23.462 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2079e-05 2024-09-20T22:19:23.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.463 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.463 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.463 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533178+0000", 2024-09-20T22:19:23.463 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.2559000000000006e-05 2024-09-20T22:19:23.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.463 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.463 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.463 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533243+0000", 2024-09-20T22:19:23.463 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4882999999999995e-05 2024-09-20T22:19:23.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.463 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.463 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.463 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533298+0000", 2024-09-20T22:19:23.463 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5111999999999997e-05 2024-09-20T22:19:23.464 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.464 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.464 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.464 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:62181 9.1 9:818888ea:::benchmark_data_smithi060_140132_object62180:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e309)", 2024-09-20T22:19:23.464 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:12:20.561567+0000", 2024-09-20T22:19:23.464 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 422.81565505100002, 2024-09-20T22:19:23.464 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.97555155000000005, 2024-09-20T22:19:23.464 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.464 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.464 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.464 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.464 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:19:23.464 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:19:23.464 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 62181 2024-09-20T22:19:23.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.465 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.465 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561567+0000", 2024-09-20T22:19:23.465 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.465 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561567+0000", 2024-09-20T22:19:23.465 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.465 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561569+0000", 2024-09-20T22:19:23.465 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9209999999999999e-06 2024-09-20T22:19:23.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.466 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.466 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561613+0000", 2024-09-20T22:19:23.466 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4295000000000002e-05 2024-09-20T22:19:23.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.466 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.466 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561639+0000", 2024-09-20T22:19:23.466 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.5177e-05 2024-09-20T22:19:23.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.466 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.466 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561648+0000", 2024-09-20T22:19:23.466 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8020000000000003e-06 2024-09-20T22:19:23.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.467 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.467 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.467 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561730+0000", 2024-09-20T22:19:23.467 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2032000000000003e-05 2024-09-20T22:19:23.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.467 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.467 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:23.467 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.561743+0000", 2024-09-20T22:19:23.467 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2466999999999999e-05 2024-09-20T22:19:23.467 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.467 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.467 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.467 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.529714+0000", 2024-09-20T22:19:23.467 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.96797125799999995 2024-09-20T22:19:23.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.468 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.468 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.468 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.529848+0000", 2024-09-20T22:19:23.468 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013433100000000001 2024-09-20T22:19:23.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.468 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.468 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:19:23.468 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.530029+0000", 2024-09-20T22:19:23.468 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000180693 2024-09-20T22:19:23.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.468 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.468 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.468 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533139+0000", 2024-09-20T22:19:23.468 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0031102299999999999 2024-09-20T22:19:23.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.469 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.469 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.533147+0000", 2024-09-20T22:19:23.469 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7330000000000003e-06 2024-09-20T22:19:23.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.469 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.469 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537004+0000", 2024-09-20T22:19:23.469 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0038571980000000001 2024-09-20T22:19:23.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.469 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.469 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537067+0000", 2024-09-20T22:19:23.469 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2385999999999995e-05 2024-09-20T22:19:23.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.470 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.470 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537119+0000", 2024-09-20T22:19:23.470 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2027000000000002e-05 2024-09-20T22:19:23.470 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.470 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.470 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.470 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.470 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:62188 9.1 9:83e4bb26:::benchmark_data_smithi060_140132_object62187:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e309)", 2024-09-20T22:19:23.470 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:12:20.566992+0000", 2024-09-20T22:19:23.470 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 422.81022976999998, 2024-09-20T22:19:23.470 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.97032768800000002, 2024-09-20T22:19:23.470 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.470 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.471 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.471 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.471 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:19:23.471 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:19:23.471 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 62188 2024-09-20T22:19:23.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.471 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.471 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.471 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.471 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.566992+0000", 2024-09-20T22:19:23.471 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.471 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.471 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.471 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.566992+0000", 2024-09-20T22:19:23.472 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.472 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.472 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.566994+0000", 2024-09-20T22:19:23.472 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9410000000000001e-06 2024-09-20T22:19:23.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.472 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.472 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.567080+0000", 2024-09-20T22:19:23.472 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5228999999999993e-05 2024-09-20T22:19:23.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.472 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.473 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.567104+0000", 2024-09-20T22:19:23.473 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.4309999999999999e-05 2024-09-20T22:19:23.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.473 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.473 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.567114+0000", 2024-09-20T22:19:23.473 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0487e-05 2024-09-20T22:19:23.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.473 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.473 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.567203+0000", 2024-09-20T22:19:23.473 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8109000000000006e-05 2024-09-20T22:19:23.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.473 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:19:23.474 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:20.567216+0000", 2024-09-20T22:19:23.474 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3040000000000001e-05 2024-09-20T22:19:23.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.474 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.474 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.474 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.531707+0000", 2024-09-20T22:19:23.474 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.96449152100000002 2024-09-20T22:19:23.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.474 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.474 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.474 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.531800+0000", 2024-09-20T22:19:23.474 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3015000000000003e-05 2024-09-20T22:19:23.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.474 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.475 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,7", 2024-09-20T22:19:23.475 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.531912+0000", 2024-09-20T22:19:23.475 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000111776 2024-09-20T22:19:23.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.475 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.475 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537181+0000", 2024-09-20T22:19:23.475 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0052689709999999999 2024-09-20T22:19:23.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.475 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.475 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537198+0000", 2024-09-20T22:19:23.475 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7142999999999998e-05 2024-09-20T22:19:23.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.475 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.475 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537256+0000", 2024-09-20T22:19:23.476 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8425999999999999e-05 2024-09-20T22:19:23.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.476 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.476 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537293+0000", 2024-09-20T22:19:23.476 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6313000000000003e-05 2024-09-20T22:19:23.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.476 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.476 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:12:21.537320+0000", 2024-09-20T22:19:23.476 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7407e-05 2024-09-20T22:19:23.476 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.476 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.476 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.476 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:215005 10.d 10:b25cdf3e:::benchmark_data_smithi060_167766_object90213:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e439)", 2024-09-20T22:19:23.477 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:18:04.023328+0000", 2024-09-20T22:19:23.477 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 79.353894138000001, 2024-09-20T22:19:23.477 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.36190466199999999, 2024-09-20T22:19:23.477 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.477 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.477 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.477 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.477 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:19:23.477 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:19:23.477 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 215005 2024-09-20T22:19:23.477 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.477 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.477 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.477 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.477 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.023328+0000", 2024-09-20T22:19:23.477 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.478 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.478 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.023328+0000", 2024-09-20T22:19:23.478 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.478 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.478 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.023329+0000", 2024-09-20T22:19:23.478 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7400000000000003e-07 2024-09-20T22:19:23.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.478 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.478 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.023332+0000", 2024-09-20T22:19:23.479 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5090000000000002e-06 2024-09-20T22:19:23.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.479 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.479 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.023342+0000", 2024-09-20T22:19:23.479 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4520000000000003e-06 2024-09-20T22:19:23.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.479 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.479 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.023349+0000", 2024-09-20T22:19:23.479 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3289999999999996e-06 2024-09-20T22:19:23.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.479 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.479 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.023408+0000", 2024-09-20T22:19:23.479 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8627e-05 2024-09-20T22:19:23.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.480 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.480 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.023513+0000", 2024-09-20T22:19:23.480 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000104861 2024-09-20T22:19:23.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.480 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:19:23.480 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.023617+0000", 2024-09-20T22:19:23.480 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000104743 2024-09-20T22:19:23.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.480 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.480 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.028852+0000", 2024-09-20T22:19:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0052352070000000004 2024-09-20T22:19:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.029360+0000", 2024-09-20T22:19:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00050801399999999999 2024-09-20T22:19:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.385120+0000", 2024-09-20T22:19:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.35575973399999999 2024-09-20T22:19:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.385183+0000", 2024-09-20T22:19:23.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2890000000000003e-05 2024-09-20T22:19:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.385233+0000", 2024-09-20T22:19:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9622e-05 2024-09-20T22:19:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:215008 10.d 10:b219a629:::benchmark_data_smithi060_167766_object90216:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e439)", 2024-09-20T22:19:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:18:04.024010+0000", 2024-09-20T22:19:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 79.353211819999999, 2024-09-20T22:19:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.36139602700000001, 2024-09-20T22:19:23.482 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:19:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:19:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 215008 2024-09-20T22:19:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.024010+0000", 2024-09-20T22:19:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.024010+0000", 2024-09-20T22:19:23.483 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.024011+0000", 2024-09-20T22:19:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5099999999999998e-07 2024-09-20T22:19:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.024016+0000", 2024-09-20T22:19:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9570000000000001e-06 2024-09-20T22:19:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.024029+0000", 2024-09-20T22:19:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.307e-05 2024-09-20T22:19:23.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.024036+0000", 2024-09-20T22:19:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2289999999999997e-06 2024-09-20T22:19:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.026106+0000", 2024-09-20T22:19:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0020691809999999998 2024-09-20T22:19:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.026213+0000", 2024-09-20T22:19:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000106848 2024-09-20T22:19:23.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:19:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.026336+0000", 2024-09-20T22:19:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012317099999999999 2024-09-20T22:19:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.029064+0000", 2024-09-20T22:19:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0027285830000000001 2024-09-20T22:19:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.029403+0000", 2024-09-20T22:19:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00033887700000000002 2024-09-20T22:19:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.385294+0000", 2024-09-20T22:19:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.35589074300000001 2024-09-20T22:19:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.385352+0000", 2024-09-20T22:19:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8134999999999998e-05 2024-09-20T22:19:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.385406+0000", 2024-09-20T22:19:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4382000000000002e-05 2024-09-20T22:19:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:215018 10.a 10:58fdeef1:::benchmark_data_smithi060_167766_object90226:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e439)", 2024-09-20T22:19:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:18:04.033316+0000", 2024-09-20T22:19:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 79.343906301999994, 2024-09-20T22:19:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.36139374200000002, 2024-09-20T22:19:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:19:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:19:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 215018 2024-09-20T22:19:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.488 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.033316+0000", 2024-09-20T22:19:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.033316+0000", 2024-09-20T22:19:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.033317+0000", 2024-09-20T22:19:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0049999999999999e-06 2024-09-20T22:19:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.489 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.033321+0000", 2024-09-20T22:19:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5549999999999996e-06 2024-09-20T22:19:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.033332+0000", 2024-09-20T22:19:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.03e-05 2024-09-20T22:19:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.033339+0000", 2024-09-20T22:19:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8839999999999997e-06 2024-09-20T22:19:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.490 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.033402+0000", 2024-09-20T22:19:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3108000000000006e-05 2024-09-20T22:19:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.033514+0000", 2024-09-20T22:19:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000112138 2024-09-20T22:19:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,5", 2024-09-20T22:19:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.033641+0000", 2024-09-20T22:19:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012684099999999999 2024-09-20T22:19:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.035770+0000", 2024-09-20T22:19:23.491 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0021291690000000002 2024-09-20T22:19:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.036261+0000", 2024-09-20T22:19:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00049101799999999997 2024-09-20T22:19:23.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.493 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.493 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.493 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.394597+0000", 2024-09-20T22:19:23.493 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.35833644799999997 2024-09-20T22:19:23.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.493 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.493 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.493 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.394660+0000", 2024-09-20T22:19:23.493 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2509999999999996e-05 2024-09-20T22:19:23.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.493 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.493 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.493 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.394710+0000", 2024-09-20T22:19:23.493 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9765999999999998e-05 2024-09-20T22:19:23.494 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.494 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.494 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.494 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.494 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.494 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:215025 10.a 10:5f13f8a2:::benchmark_data_smithi060_167766_object90233:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e439)", 2024-09-20T22:19:23.494 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:18:04.036442+0000", 2024-09-20T22:19:23.494 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 79.340780588000001, 2024-09-20T22:19:23.494 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.358419239, 2024-09-20T22:19:23.494 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.494 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.494 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.494 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.494 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:19:23.495 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:19:23.495 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 215025 2024-09-20T22:19:23.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.495 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.495 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.495 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.495 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.036442+0000", 2024-09-20T22:19:23.495 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.495 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.495 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.495 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.036442+0000", 2024-09-20T22:19:23.495 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.495 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.496 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.496 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.036442+0000", 2024-09-20T22:19:23.496 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3200000000000004e-07 2024-09-20T22:19:23.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.496 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.496 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.496 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.036447+0000", 2024-09-20T22:19:23.496 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4150000000000001e-06 2024-09-20T22:19:23.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.496 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.496 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.496 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.036457+0000", 2024-09-20T22:19:23.496 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0424999999999999e-05 2024-09-20T22:19:23.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.497 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.497 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.497 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.036464+0000", 2024-09-20T22:19:23.497 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2690000000000002e-06 2024-09-20T22:19:23.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.497 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.497 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.497 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.036536+0000", 2024-09-20T22:19:23.497 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1120999999999999e-05 2024-09-20T22:19:23.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.497 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.497 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.497 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.036636+0000", 2024-09-20T22:19:23.497 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9932999999999993e-05 2024-09-20T22:19:23.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.498 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.498 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,5", 2024-09-20T22:19:23.498 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.036737+0000", 2024-09-20T22:19:23.498 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010153300000000001 2024-09-20T22:19:23.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.498 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.498 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.498 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.039146+0000", 2024-09-20T22:19:23.498 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0024091070000000002 2024-09-20T22:19:23.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.498 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.498 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.498 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.039503+0000", 2024-09-20T22:19:23.499 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00035699700000000001 2024-09-20T22:19:23.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.499 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.499 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.499 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.394758+0000", 2024-09-20T22:19:23.499 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.355254809 2024-09-20T22:19:23.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.499 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.499 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.499 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.394814+0000", 2024-09-20T22:19:23.499 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5615999999999997e-05 2024-09-20T22:19:23.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.499 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.499 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.499 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.394861+0000", 2024-09-20T22:19:23.499 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7281999999999999e-05 2024-09-20T22:19:23.500 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.500 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.500 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.500 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.500 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:215028 10.d 10:b0594cd8:::benchmark_data_smithi060_167766_object90236:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e439)", 2024-09-20T22:19:23.500 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:18:04.039746+0000", 2024-09-20T22:19:23.500 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 79.337476554999995, 2024-09-20T22:19:23.500 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.35934350700000001, 2024-09-20T22:19:23.500 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.500 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.500 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.500 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.500 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:19:23.500 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:19:23.501 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 215028 2024-09-20T22:19:23.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.501 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.501 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.501 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.039746+0000", 2024-09-20T22:19:23.501 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.501 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.501 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.039746+0000", 2024-09-20T22:19:23.501 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.501 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.501 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.039747+0000", 2024-09-20T22:19:23.502 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1909999999999999e-06 2024-09-20T22:19:23.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.502 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.502 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.039752+0000", 2024-09-20T22:19:23.502 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2009999999999998e-06 2024-09-20T22:19:23.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.502 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.502 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.039763+0000", 2024-09-20T22:19:23.502 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0566e-05 2024-09-20T22:19:23.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.502 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.502 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.039770+0000", 2024-09-20T22:19:23.502 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.294e-06 2024-09-20T22:19:23.503 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.503 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.503 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.503 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.039832+0000", 2024-09-20T22:19:23.503 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2589999999999995e-05 2024-09-20T22:19:23.503 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.503 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.503 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.503 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.039949+0000", 2024-09-20T22:19:23.503 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000116626 2024-09-20T22:19:23.503 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.503 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.503 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:19:23.503 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.040075+0000", 2024-09-20T22:19:23.504 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000125554 2024-09-20T22:19:23.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.504 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.504 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.504 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.042691+0000", 2024-09-20T22:19:23.504 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0026165339999999998 2024-09-20T22:19:23.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.504 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.504 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.504 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.043199+0000", 2024-09-20T22:19:23.504 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000507911 2024-09-20T22:19:23.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.504 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.504 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.504 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.399010+0000", 2024-09-20T22:19:23.504 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.35581109500000002 2024-09-20T22:19:23.505 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.505 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.505 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.505 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.399058+0000", 2024-09-20T22:19:23.505 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8056000000000002e-05 2024-09-20T22:19:23.505 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.505 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.505 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.505 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.399089+0000", 2024-09-20T22:19:23.505 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0889000000000002e-05 2024-09-20T22:19:23.505 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.505 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.505 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.505 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.505 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.505 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:215033 10.d 10:bca61bbb:::benchmark_data_smithi060_167766_object90241:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e439)", 2024-09-20T22:19:23.506 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:18:04.047516+0000", 2024-09-20T22:19:23.506 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 79.329706486000006, 2024-09-20T22:19:23.506 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.35166720600000001, 2024-09-20T22:19:23.506 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.506 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.506 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.506 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.506 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:19:23.506 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:19:23.506 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 215033 2024-09-20T22:19:23.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.506 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.506 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.506 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.506 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.047516+0000", 2024-09-20T22:19:23.507 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.507 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.507 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.507 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.507 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.047516+0000", 2024-09-20T22:19:23.507 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.507 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.507 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.507 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.507 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.047516+0000", 2024-09-20T22:19:23.507 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7700000000000004e-07 2024-09-20T22:19:23.507 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.507 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.507 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.508 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.047520+0000", 2024-09-20T22:19:23.508 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5540000000000002e-06 2024-09-20T22:19:23.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.508 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.508 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.508 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.047530+0000", 2024-09-20T22:19:23.508 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6139999999999998e-06 2024-09-20T22:19:23.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.508 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.508 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.508 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.047535+0000", 2024-09-20T22:19:23.508 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0699999999999997e-06 2024-09-20T22:19:23.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.508 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.508 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.508 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.047573+0000", 2024-09-20T22:19:23.509 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.8473000000000002e-05 2024-09-20T22:19:23.509 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.509 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.509 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.509 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.047660+0000", 2024-09-20T22:19:23.509 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6644999999999995e-05 2024-09-20T22:19:23.509 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.509 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.509 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:19:23.509 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.047768+0000", 2024-09-20T22:19:23.509 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000108053 2024-09-20T22:19:23.509 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.509 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.509 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.509 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.050314+0000", 2024-09-20T22:19:23.510 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0025464490000000001 2024-09-20T22:19:23.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.510 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.510 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.510 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.050333+0000", 2024-09-20T22:19:23.510 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9236999999999999e-05 2024-09-20T22:19:23.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.510 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.510 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.510 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.399132+0000", 2024-09-20T22:19:23.510 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.34879811599999999 2024-09-20T22:19:23.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.510 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.510 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.511 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.399160+0000", 2024-09-20T22:19:23.511 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.8653000000000002e-05 2024-09-20T22:19:23.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.511 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.511 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.511 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:04.399183+0000", 2024-09-20T22:19:23.511 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.2665000000000001e-05 2024-09-20T22:19:23.511 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.511 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.511 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.511 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.511 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:233495 10.d 10:bc8bfc89:::benchmark_data_smithi060_167766_object108703:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e445)", 2024-09-20T22:19:23.511 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:18:20.596201+0000", 2024-09-20T22:19:23.511 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 62.781021375999998, 2024-09-20T22:19:23.512 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.30189378100000003, 2024-09-20T22:19:23.512 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.512 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.512 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.512 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.512 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:19:23.512 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:19:23.512 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 233495 2024-09-20T22:19:23.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.512 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.512 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.512 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.512 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.596201+0000", 2024-09-20T22:19:23.512 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.513 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.513 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.513 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.596201+0000", 2024-09-20T22:19:23.513 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.513 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.513 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.513 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.513 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.596202+0000", 2024-09-20T22:19:23.513 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6000000000000003e-07 2024-09-20T22:19:23.513 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.513 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.513 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.513 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.596206+0000", 2024-09-20T22:19:23.513 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1570000000000002e-06 2024-09-20T22:19:23.513 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.514 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.514 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.514 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.596219+0000", 2024-09-20T22:19:23.514 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3611999999999999e-05 2024-09-20T22:19:23.514 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.514 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.514 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.514 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.596226+0000", 2024-09-20T22:19:23.514 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9280000000000004e-06 2024-09-20T22:19:23.514 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.514 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.514 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.514 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.596281+0000", 2024-09-20T22:19:23.514 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4218999999999997e-05 2024-09-20T22:19:23.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.515 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.515 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.515 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.596393+0000", 2024-09-20T22:19:23.515 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000112608 2024-09-20T22:19:23.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.515 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.515 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:19:23.515 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.596498+0000", 2024-09-20T22:19:23.515 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000104848 2024-09-20T22:19:23.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.515 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.515 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.515 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.599981+0000", 2024-09-20T22:19:23.515 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0034829280000000002 2024-09-20T22:19:23.516 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.516 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.516 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.516 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.600154+0000", 2024-09-20T22:19:23.516 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000173263 2024-09-20T22:19:23.516 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.516 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.516 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.516 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.897965+0000", 2024-09-20T22:19:23.516 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.29781107899999998 2024-09-20T22:19:23.516 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.516 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.516 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.516 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.898033+0000", 2024-09-20T22:19:23.516 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8077999999999994e-05 2024-09-20T22:19:23.516 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.517 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.517 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.517 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.898095+0000", 2024-09-20T22:19:23.517 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1301000000000001e-05 2024-09-20T22:19:23.517 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.517 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.517 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.517 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.517 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.517 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:41030 11.a 11:5e4f411e:::benchmark_data_smithi060_196073_object41029:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e465)", 2024-09-20T22:19:23.517 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:19:12.065710+0000", 2024-09-20T22:19:23.517 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 11.311512378, 2024-09-20T22:19:23.517 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.26583985999999998, 2024-09-20T22:19:23.517 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.517 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.517 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.518 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.518 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:19:23.518 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:19:23.518 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 41030 2024-09-20T22:19:23.518 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.518 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.518 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.518 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.518 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.065710+0000", 2024-09-20T22:19:23.518 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.518 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.518 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.518 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.518 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.065710+0000", 2024-09-20T22:19:23.518 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.519 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.519 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.519 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.519 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.065711+0000", 2024-09-20T22:19:23.519 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0470000000000001e-06 2024-09-20T22:19:23.519 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.519 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.519 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.519 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.065899+0000", 2024-09-20T22:19:23.519 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00018846800000000001 2024-09-20T22:19:23.519 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.519 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.519 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.519 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.065918+0000", 2024-09-20T22:19:23.519 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8162e-05 2024-09-20T22:19:23.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.520 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.520 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.520 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.065926+0000", 2024-09-20T22:19:23.520 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4800000000000001e-06 2024-09-20T22:19:23.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.520 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.520 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.520 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.066021+0000", 2024-09-20T22:19:23.520 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4925000000000003e-05 2024-09-20T22:19:23.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.520 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.520 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.520 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.066191+0000", 2024-09-20T22:19:23.520 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017051600000000001 2024-09-20T22:19:23.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.521 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:19:23.521 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.066323+0000", 2024-09-20T22:19:23.521 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013137000000000001 2024-09-20T22:19:23.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.521 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.521 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.068176+0000", 2024-09-20T22:19:23.521 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001853078 2024-09-20T22:19:23.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.521 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.521 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.071349+0000", 2024-09-20T22:19:23.521 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0031732179999999998 2024-09-20T22:19:23.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.522 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.522 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.522 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.331325+0000", 2024-09-20T22:19:23.522 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.25997580999999997 2024-09-20T22:19:23.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.522 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.522 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.522 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.331504+0000", 2024-09-20T22:19:23.522 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017910100000000001 2024-09-20T22:19:23.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.522 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.522 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.522 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.331550+0000", 2024-09-20T22:19:23.522 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5685000000000001e-05 2024-09-20T22:19:23.523 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.523 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.523 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.523 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.523 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:41034 11.a 11:59b5ae97:::benchmark_data_smithi060_196073_object41033:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e465)", 2024-09-20T22:19:23.523 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:19:12.069474+0000", 2024-09-20T22:19:23.523 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 11.307748644, 2024-09-20T22:19:23.523 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.26528238300000001, 2024-09-20T22:19:23.523 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:23.523 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:23.523 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:19:23.523 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:23.523 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:19:23.524 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:19:23.524 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 41034 2024-09-20T22:19:23.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.524 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:23.524 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.524 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:23.524 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.069474+0000", 2024-09-20T22:19:23.524 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.524 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.524 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:23.524 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.069474+0000", 2024-09-20T22:19:23.524 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:23.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.524 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.524 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:23.525 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.069475+0000", 2024-09-20T22:19:23.525 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.187e-06 2024-09-20T22:19:23.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.525 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.525 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:23.525 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.069593+0000", 2024-09-20T22:19:23.525 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011814199999999999 2024-09-20T22:19:23.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.525 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.525 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:23.525 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.069606+0000", 2024-09-20T22:19:23.525 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3380000000000001e-05 2024-09-20T22:19:23.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.525 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.526 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:23.526 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.069614+0000", 2024-09-20T22:19:23.526 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3329999999999999e-06 2024-09-20T22:19:23.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.526 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.526 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:23.526 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.069663+0000", 2024-09-20T22:19:23.526 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9188999999999999e-05 2024-09-20T22:19:23.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.526 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.526 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:23.526 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.069751+0000", 2024-09-20T22:19:23.526 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8490000000000001e-05 2024-09-20T22:19:23.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.526 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.527 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 3,5", 2024-09-20T22:19:23.527 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.069837+0000", 2024-09-20T22:19:23.527 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5762999999999999e-05 2024-09-20T22:19:23.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.527 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:23.527 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.071020+0000", 2024-09-20T22:19:23.527 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0011834320000000001 2024-09-20T22:19:23.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.527 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.527 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.078372+0000", 2024-09-20T22:19:23.527 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0073511649999999998 2024-09-20T22:19:23.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.528 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:23.528 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.334638+0000", 2024-09-20T22:19:23.528 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.25626611599999999 2024-09-20T22:19:23.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.528 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.528 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:19:23.528 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.334714+0000", 2024-09-20T22:19:23.528 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6075000000000006e-05 2024-09-20T22:19:23.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:23.528 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:23.528 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:19:23.528 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:12.334756+0000", 2024-09-20T22:19:23.528 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2110999999999997e-05 2024-09-20T22:19:23.528 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.528 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.529 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.529 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:23.529 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:23.529 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:19:23.529 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:23.551 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:23.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:23.552 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:23.652 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:23.684 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:23.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:23.685 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:23.786 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:23.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:23.819 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:23.820 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:23.920 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:23.942 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:23.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:23.943 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:23 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:19:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:23 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:19:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:23 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:19:24.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:23 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:19:24.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:23 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:19:24.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:23 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:19:24.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:23 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:19:24.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:23 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:19:24.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:23 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:19:24.043 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:24.079 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:24.080 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:24.080 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:24.100 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 31 16 57721 57705 116.317 74.9375 0.00850851 0.00859197 2024-09-20T22:19:24.181 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:24.217 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:24.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:24.218 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:24.318 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:24.358 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:24.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:24.359 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:24.460 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:24.490 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:24.491 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:24.491 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:24.592 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:24.614 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:24.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:24.615 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:24.716 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:24.749 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:24.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:24.750 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:24.851 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:24.883 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:24.883 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:24.884 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:24.926 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:24.938 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:24.984 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:25.021 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:25.022 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:25.022 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:25.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:24 smithi060 ceph-mon[29694]: osdmap e470: 8 total, 8 up, 8 in 2024-09-20T22:19:25.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:24 smithi060 ceph-mon[29694]: pgmap v882: 178 pgs: 178 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 79 MiB/s wr, 1.26k op/s 2024-09-20T22:19:25.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:24 smithi060 ceph-mon[40290]: osdmap e470: 8 total, 8 up, 8 in 2024-09-20T22:19:25.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:24 smithi060 ceph-mon[40290]: pgmap v882: 178 pgs: 178 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 79 MiB/s wr, 1.26k op/s 2024-09-20T22:19:25.101 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 32 16 58989 58973 115.155 79.25 0.00934946 0.00867838 2024-09-20T22:19:25.123 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:25.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:25.166 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:25.166 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:25.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:24 smithi067 ceph-mon[33511]: osdmap e470: 8 total, 8 up, 8 in 2024-09-20T22:19:25.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:24 smithi067 ceph-mon[33511]: pgmap v882: 178 pgs: 178 active+clean; 3.4 GiB data, 15 GiB used, 700 GiB / 715 GiB avail; 852 B/s rd, 79 MiB/s wr, 1.26k op/s 2024-09-20T22:19:25.267 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:25.300 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:25.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:25.301 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:25.358 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:19:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:19:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:19:25.402 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:25.443 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:25.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:25.444 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:25.545 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:25.586 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:25.586 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:25.586 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:25.687 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:25.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:25.728 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:25.728 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:25.776 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T22:19:25.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:25.866 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:25.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:25.868 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:25.968 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:26.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:26.011 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:26.012 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:26.101 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 33 16 60299 60283 114.147 81.875 0.0263427 0.00875467 2024-09-20T22:19:26.113 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:26.157 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:26.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:26.158 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:26.159 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:19:26.215 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:19:26.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:26.301 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:26.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:26.302 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:26.402 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:26.438 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:26.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:26.439 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:26.475 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:26 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2926715474' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:26.476 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:26 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:26.535 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:26 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2926715474' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:26.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:26 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:26 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2926715474' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:26 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:26.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:26 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a[29671]: 2024-09-20T22:19:26.268+0000 7f01adcc6640 -1 mon.a@0(leader).osd e471 definitely_dead 0 2024-09-20T22:19:26.540 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:26.580 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:26.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:26.582 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:26.682 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:26.723 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:26.723 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:26.724 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:26.801 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:19:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:19:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:61233 11.5 11:ad1a59e7:::benchmark_data_smithi060_196073_object61232:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e471)", 2024-09-20T22:19:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:19:26.796211+0000", 2024-09-20T22:19:26.801 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0028544360000000001, 2024-09-20T22:19:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002887396, 2024-09-20T22:19:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:19:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:19:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:19:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 61233 2024-09-20T22:19:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:26.802 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796211+0000", 2024-09-20T22:19:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796211+0000", 2024-09-20T22:19:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796212+0000", 2024-09-20T22:19:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7999999999999996e-07 2024-09-20T22:19:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796240+0000", 2024-09-20T22:19:26.803 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.8243e-05 2024-09-20T22:19:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796247+0000", 2024-09-20T22:19:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8870000000000003e-06 2024-09-20T22:19:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796253+0000", 2024-09-20T22:19:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.5350000000000004e-06 2024-09-20T22:19:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:26.804 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796337+0000", 2024-09-20T22:19:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4868000000000001e-05 2024-09-20T22:19:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796477+0000", 2024-09-20T22:19:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013978300000000001 2024-09-20T22:19:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 5,6", 2024-09-20T22:19:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796605+0000", 2024-09-20T22:19:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012754300000000001 2024-09-20T22:19:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:26.805 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.798919+0000", 2024-09-20T22:19:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0023144559999999999 2024-09-20T22:19:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop_reply(client.16440.0:61224 11.5 e471/456 ondisk, result = 0)", 2024-09-20T22:19:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:19:26.797437+0000", 2024-09-20T22:19:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.001629248, 2024-09-20T22:19:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0017432089999999999, 2024-09-20T22:19:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:19:26.806 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.797437+0000", 2024-09-20T22:19:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.797437+0000", 2024-09-20T22:19:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.797438+0000", 2024-09-20T22:19:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5000000000000001e-07 2024-09-20T22:19:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.807 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.797441+0000", 2024-09-20T22:19:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9500000000000003e-06 2024-09-20T22:19:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.797451+0000", 2024-09-20T22:19:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6010000000000001e-06 2024-09-20T22:19:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.797457+0000", 2024-09-20T22:19:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2990000000000002e-06 2024-09-20T22:19:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.808 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.799095+0000", 2024-09-20T22:19:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0016378149999999999 2024-09-20T22:19:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.799102+0000", 2024-09-20T22:19:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.781e-06 2024-09-20T22:19:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop_reply(client.16440.0:61226 11.4 e471/456)", 2024-09-20T22:19:26.809 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:19:26.797483+0000", 2024-09-20T22:19:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0015831980000000001, 2024-09-20T22:19:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0017514640000000001, 2024-09-20T22:19:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "queued for pg", 2024-09-20T22:19:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.797483+0000", 2024-09-20T22:19:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:26.810 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.797483+0000", 2024-09-20T22:19:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.797483+0000", 2024-09-20T22:19:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1699999999999999e-07 2024-09-20T22:19:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.797486+0000", 2024-09-20T22:19:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7549999999999999e-06 2024-09-20T22:19:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:26.811 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.797492+0000", 2024-09-20T22:19:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4740000000000002e-06 2024-09-20T22:19:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.797497+0000", 2024-09-20T22:19:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6160000000000004e-06 2024-09-20T22:19:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:61226 11.4 11:2c5ed592:::benchmark_data_smithi060_196073_object61225:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e471)", 2024-09-20T22:19:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:19:26.790601+0000", 2024-09-20T22:19:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0084646240000000005, 2024-09-20T22:19:26.812 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0086842659999999995, 2024-09-20T22:19:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:19:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:19:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:19:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 61226 2024-09-20T22:19:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.790601+0000", 2024-09-20T22:19:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.813 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.790601+0000", 2024-09-20T22:19:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.790602+0000", 2024-09-20T22:19:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6600000000000005e-07 2024-09-20T22:19:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.814 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:19:26.815 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:26.815 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.790636+0000", 2024-09-20T22:19:26.815 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.3419e-05 2024-09-20T22:19:26.815 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.815 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.815 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:26.815 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.790648+0000", 2024-09-20T22:19:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1959e-05 2024-09-20T22:19:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.790655+0000", 2024-09-20T22:19:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0960000000000003e-06 2024-09-20T22:19:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.791688+0000", 2024-09-20T22:19:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0010329149999999999 2024-09-20T22:19:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:26.816 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.791809+0000", 2024-09-20T22:19:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000121718 2024-09-20T22:19:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,6", 2024-09-20T22:19:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.791930+0000", 2024-09-20T22:19:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000121193 2024-09-20T22:19:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.794532+0000", 2024-09-20T22:19:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0026016099999999999 2024-09-20T22:19:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.817 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:19:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796141+0000", 2024-09-20T22:19:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0016086729999999999 2024-09-20T22:19:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:61231 11.4 11:2cca6b46:::benchmark_data_smithi060_196073_object61230:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e471)", 2024-09-20T22:19:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:19:26.794682+0000", 2024-09-20T22:19:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0043840010000000002, 2024-09-20T22:19:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046883109999999997, 2024-09-20T22:19:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:26.818 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:19:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:19:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:19:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 61231 2024-09-20T22:19:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.794682+0000", 2024-09-20T22:19:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.794682+0000", 2024-09-20T22:19:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:26.819 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.794683+0000", 2024-09-20T22:19:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.319e-06 2024-09-20T22:19:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796076+0000", 2024-09-20T22:19:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001392438 2024-09-20T22:19:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796087+0000", 2024-09-20T22:19:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1012999999999999e-05 2024-09-20T22:19:26.820 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796094+0000", 2024-09-20T22:19:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9820000000000002e-06 2024-09-20T22:19:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796170+0000", 2024-09-20T22:19:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6287999999999999e-05 2024-09-20T22:19:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796298+0000", 2024-09-20T22:19:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000128474 2024-09-20T22:19:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.821 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 2,6", 2024-09-20T22:19:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796445+0000", 2024-09-20T22:19:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000146926 2024-09-20T22:19:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:61232 11.f 11:fd5339b9:::benchmark_data_smithi060_196073_object61231:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e471)", 2024-09-20T22:19:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:19:26.796149+0000", 2024-09-20T22:19:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0029171919999999999, 2024-09-20T22:19:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0033009020000000001, 2024-09-20T22:19:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:19:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:19:26.822 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:19:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:19:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:19:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:19:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 61232 2024-09-20T22:19:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:19:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:19:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796149+0000", 2024-09-20T22:19:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:19:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796149+0000", 2024-09-20T22:19:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:19:26.823 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:19:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796150+0000", 2024-09-20T22:19:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.7899999999999997e-07 2024-09-20T22:19:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:19:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796181+0000", 2024-09-20T22:19:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.1288999999999998e-05 2024-09-20T22:19:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:19:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796189+0000", 2024-09-20T22:19:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.5890000000000003e-06 2024-09-20T22:19:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.824 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:19:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796196+0000", 2024-09-20T22:19:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0850000000000002e-06 2024-09-20T22:19:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:19:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796245+0000", 2024-09-20T22:19:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9914999999999999e-05 2024-09-20T22:19:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:19:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796368+0000", 2024-09-20T22:19:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000122817 2024-09-20T22:19:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.825 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:19:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.796497+0000", 2024-09-20T22:19:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012915600000000001 2024-09-20T22:19:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:19:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:19:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:19:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:26.799024+0000", 2024-09-20T22:19:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0025266830000000001 2024-09-20T22:19:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:19:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:19:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-20T22:19:26.826 INFO:teuthology.orchestra.run.smithi067.stdout: "num_ops": 6 2024-09-20T22:19:26.826 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:19:26.826 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:26.866 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:26.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:26.867 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:26.968 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:27.002 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:27.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:27.003 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:27.101 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 34 16 61637 61621 113.249 83.625 0.0056459 0.00882539 2024-09-20T22:19:27.104 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:27.135 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:27.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:27.137 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:27.230 INFO:teuthology.orchestra.run.smithi060.stderr:marked out osd.4. 2024-09-20T22:19:27.237 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:27.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:27.297 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:27.297 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:27.398 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:27.420 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:27.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:27.421 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:27.458 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:27.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:27 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:19:27.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:27 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:19:27.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:27 smithi067 ceph-mon[33511]: osdmap e471: 8 total, 8 up, 8 in 2024-09-20T22:19:27.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:27 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3119961304' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-09-20T22:19:27.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:27 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-09-20T22:19:27.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:27 smithi067 ceph-mon[33511]: Client client.admin marked osd.4 out, while it was still marked up 2024-09-20T22:19:27.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:27 smithi067 ceph-mon[33511]: pgmap v884: 178 pgs: 178 active+clean; 3.5 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.7 KiB/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-20T22:19:27.522 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:27.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:27 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:19:27.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:27 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:19:27.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:27 smithi060 ceph-mon[29694]: osdmap e471: 8 total, 8 up, 8 in 2024-09-20T22:19:27.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:27 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3119961304' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-09-20T22:19:27.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:27 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-09-20T22:19:27.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:27 smithi060 ceph-mon[29694]: Client client.admin marked osd.4 out, while it was still marked up 2024-09-20T22:19:27.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:27 smithi060 ceph-mon[29694]: pgmap v884: 178 pgs: 178 active+clean; 3.5 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.7 KiB/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-20T22:19:27.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:27 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:19:27.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:27 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:19:27.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:27 smithi060 ceph-mon[40290]: osdmap e471: 8 total, 8 up, 8 in 2024-09-20T22:19:27.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:27 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3119961304' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-09-20T22:19:27.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:27 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["4"]}]: dispatch 2024-09-20T22:19:27.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:27 smithi060 ceph-mon[40290]: Client client.admin marked osd.4 out, while it was still marked up 2024-09-20T22:19:27.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:27 smithi060 ceph-mon[40290]: pgmap v884: 178 pgs: 178 active+clean; 3.5 GiB data, 16 GiB used, 700 GiB / 715 GiB avail; 1.7 KiB/s rd, 86 MiB/s wr, 1.37k op/s 2024-09-20T22:19:27.544 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:27.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:27.545 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:27.645 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:27.669 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:27.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:27.669 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:27.674 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T22:19:27.771 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:27.793 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:27.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:27.795 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:27.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:27.918 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:27.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:27.919 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:27.968 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:19:28.020 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:28.042 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:28.043 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:28.043 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:28.101 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 35 16 61871 61855 110.431 14.625 0.00301559 0.00883788 2024-09-20T22:19:28.144 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:28.166 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:28.167 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:28.167 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:28.268 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:28.293 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:28.294 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:28.294 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:28.368 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:19:28.368 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:19:28.385 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:19:28.385 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:19:28.395 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:28.401 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:19:28.402 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:19:28.416 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:28.417 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:28.417 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:28.418 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:19:28.418 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:19:28.419 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:28 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["4"]}]': finished 2024-09-20T22:19:28.419 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:28 smithi060 ceph-mon[29694]: osdmap e472: 8 total, 8 up, 7 in 2024-09-20T22:19:28.419 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:28 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["4"]}]': finished 2024-09-20T22:19:28.419 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:28 smithi060 ceph-mon[40290]: osdmap e472: 8 total, 8 up, 7 in 2024-09-20T22:19:28.435 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:19:28.436 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:19:28.439 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:28 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["4"]}]': finished 2024-09-20T22:19:28.439 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:28 smithi067 ceph-mon[33511]: osdmap e472: 8 total, 8 up, 7 in 2024-09-20T22:19:28.452 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:19:28.452 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:19:28.469 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:19:28.469 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:19:28.486 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:19:28.486 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:19:28.518 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:28.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:28.541 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:28.542 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:28.547 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:19:28.547 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:19:28.642 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:28.664 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:28.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:28.665 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:28.765 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:28.788 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:28.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:28.789 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:28.890 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:28.912 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:28.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:28.912 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:29.013 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:29.034 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:29.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:29.035 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:29.093 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:19:29.102 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 36 16 61871 61855 107.364 0 - 0.00883788 2024-09-20T22:19:29.136 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:29.158 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:29.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:29.159 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:29.246 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T22:19:29.259 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:29.341 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:29.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:29.342 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:29.442 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:29.468 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:29.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:29.469 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:29.498 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:29.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:29 smithi067 ceph-mon[33511]: osdmap e473: 8 total, 8 up, 7 in 2024-09-20T22:19:29.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:29 smithi067 ceph-mon[33511]: pgmap v887: 178 pgs: 1 remapped+peering, 5 peering, 172 active+clean; 3.7 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 225 B/s rd, 72 MiB/s wr, 1.16k op/s; 106 B/s, 6 objects/s recovering 2024-09-20T22:19:29.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:29 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2913605602' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:29.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:29 smithi067 ceph-mon[33511]: osdmap e474: 8 total, 8 up, 7 in 2024-09-20T22:19:29.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:29 smithi060 ceph-mon[29694]: osdmap e473: 8 total, 8 up, 7 in 2024-09-20T22:19:29.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:29 smithi060 ceph-mon[29694]: pgmap v887: 178 pgs: 1 remapped+peering, 5 peering, 172 active+clean; 3.7 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 225 B/s rd, 72 MiB/s wr, 1.16k op/s; 106 B/s, 6 objects/s recovering 2024-09-20T22:19:29.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:29 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2913605602' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:29.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:29 smithi060 ceph-mon[29694]: osdmap e474: 8 total, 8 up, 7 in 2024-09-20T22:19:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:29 smithi060 ceph-mon[40290]: osdmap e473: 8 total, 8 up, 7 in 2024-09-20T22:19:29.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:29 smithi060 ceph-mon[40290]: pgmap v887: 178 pgs: 1 remapped+peering, 5 peering, 172 active+clean; 3.7 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 225 B/s rd, 72 MiB/s wr, 1.16k op/s; 106 B/s, 6 objects/s recovering 2024-09-20T22:19:29.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:29 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2913605602' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:29.538 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:29 smithi060 ceph-mon[40290]: osdmap e474: 8 total, 8 up, 7 in 2024-09-20T22:19:29.570 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:29.588 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:19:29.607 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:29.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:29.608 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:29.709 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:29.767 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:29.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:29.768 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:29.869 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:29.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:29.905 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:29.905 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:30.006 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:30.034 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:30.035 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:30.035 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:30.102 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 37 16 62372 62356 105.309 15.6562 0.00725726 0.00948234 2024-09-20T22:19:30.136 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:30.165 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:30.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:30.167 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:30.212 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:19:30.212 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:19:30.267 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:30.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:30.347 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:30.348 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:30.448 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:30.501 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:30.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:30.502 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:30 smithi067 ceph-mon[33511]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:19:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:30 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:19:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:30 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:19:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:30 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:19:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:30 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:19:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:30 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:19:30.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:30 smithi067 ceph-mon[33511]: osdmap e475: 8 total, 8 up, 7 in 2024-09-20T22:19:30.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:30 smithi060 ceph-mon[29694]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:19:30.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:30 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:19:30.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:30 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:19:30.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:30 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:19:30.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:30 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:19:30.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:30 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:19:30.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:30 smithi060 ceph-mon[29694]: osdmap e475: 8 total, 8 up, 7 in 2024-09-20T22:19:30.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:30 smithi060 ceph-mon[40290]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:19:30.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:30 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:19:30.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:30 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:19:30.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:30 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:19:30.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:30 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:19:30.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:30 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:19:30.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:30 smithi060 ceph-mon[40290]: osdmap e475: 8 total, 8 up, 7 in 2024-09-20T22:19:30.603 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:30.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:30.651 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:30.651 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:30.752 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:30.790 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:30.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:30.791 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:30.891 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:30.930 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:30.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:30.931 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:31.032 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:31.068 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:31.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:31.069 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:31.102 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 38 16 63101 63085 103.736 45.5625 0.0190353 0.0096316 2024-09-20T22:19:31.170 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:31.196 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:31.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:31.197 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:31.239 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:19:31.261 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:19:31.298 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:31.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:31.376 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:31.376 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:31.473 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:31 smithi067 ceph-mon[33511]: pgmap v890: 178 pgs: 6 remapped+peering, 5 peering, 167 active+clean; 3.7 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 68 MiB/s wr, 1.09k op/s; 242 B/s, 11 objects/s recovering 2024-09-20T22:19:31.473 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:31 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2866647830' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:19:31.473 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:31 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:19:31.473 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:31 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:19:31.473 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:31 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:19:31.473 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:31 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:19:31.474 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:31 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2866647830' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:19:31.474 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:31 smithi067 ceph-mon[33511]: osdmap e476: 8 total, 8 up, 7 in 2024-09-20T22:19:31.477 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:31.512 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T22:19:31.524 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:31.525 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:31.525 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:31.535 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:31 smithi060 ceph-mon[40290]: pgmap v890: 178 pgs: 6 remapped+peering, 5 peering, 167 active+clean; 3.7 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 68 MiB/s wr, 1.09k op/s; 242 B/s, 11 objects/s recovering 2024-09-20T22:19:31.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:31 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2866647830' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:19:31.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:31 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:19:31.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:31 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:19:31.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:31 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:19:31.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:31 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:19:31.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:31 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2866647830' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:19:31.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:31 smithi060 ceph-mon[40290]: osdmap e476: 8 total, 8 up, 7 in 2024-09-20T22:19:31.542 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:31 smithi060 ceph-mon[29694]: pgmap v890: 178 pgs: 6 remapped+peering, 5 peering, 167 active+clean; 3.7 GiB data, 15 GiB used, 611 GiB / 626 GiB avail; 68 MiB/s wr, 1.09k op/s; 242 B/s, 11 objects/s recovering 2024-09-20T22:19:31.542 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:31 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2866647830' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:19:31.542 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:31 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:19:31.542 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:31 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:19:31.542 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:31 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:19:31.542 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:31 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:19:31.542 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:31 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2866647830' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:19:31.542 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:31 smithi060 ceph-mon[29694]: osdmap e476: 8 total, 8 up, 7 in 2024-09-20T22:19:31.626 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:31.652 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:31.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:31.653 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:31.753 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:31.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:31.820 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:31.820 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:31.921 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:31.948 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:31.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:31.949 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:32.049 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:32.074 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:32.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:32.075 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:32.102 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 39 15 63832 63817 102.25 45.75 0.0176336 0.00977408 2024-09-20T22:19:32.175 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:32.201 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:32.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:32.202 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:32.289 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:32.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:32.336 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:32.388 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:32.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:32.389 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:32.490 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:32.516 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:32.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:32.517 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:32.618 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:32.676 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:32.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:32.677 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:32.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:32 smithi060 ceph-mon[29694]: 5.1c scrub starts 2024-09-20T22:19:32.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:32 smithi060 ceph-mon[29694]: osdmap e477: 8 total, 8 up, 7 in 2024-09-20T22:19:32.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:32 smithi060 ceph-mon[29694]: 5.1c scrub ok 2024-09-20T22:19:32.745 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:32 smithi060 ceph-mon[29694]: pgmap v893: 178 pgs: 2 active+recovering+undersized+remapped, 6 remapped+peering, 9 peering, 161 active+clean; 3.8 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 5.2 KiB/s rd, 30 MiB/s wr, 490 op/s; 7485/188145 objects misplaced (3.978%); 2.0 MiB/s, 41 objects/s recovering 2024-09-20T22:19:32.745 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:32 smithi060 ceph-mon[40290]: 5.1c scrub starts 2024-09-20T22:19:32.746 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:32 smithi060 ceph-mon[40290]: osdmap e477: 8 total, 8 up, 7 in 2024-09-20T22:19:32.746 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:32 smithi060 ceph-mon[40290]: 5.1c scrub ok 2024-09-20T22:19:32.746 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:32 smithi060 ceph-mon[40290]: pgmap v893: 178 pgs: 2 active+recovering+undersized+remapped, 6 remapped+peering, 9 peering, 161 active+clean; 3.8 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 5.2 KiB/s rd, 30 MiB/s wr, 490 op/s; 7485/188145 objects misplaced (3.978%); 2.0 MiB/s, 41 objects/s recovering 2024-09-20T22:19:32.778 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:32.831 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:32.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:32.832 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:32.931 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 5, 6, 7, 0] out_osds: [4] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:19:32.931 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:19:32.931 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [1, 2, 3, 5, 6, 7, 0] 2024-09-20T22:19:32.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd out 1 2024-09-20T22:19:32.933 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:32.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:32.961 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:32.961 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:33.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:19:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:19:33.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:32 smithi067 ceph-mon[33511]: 5.1c scrub starts 2024-09-20T22:19:33.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:32 smithi067 ceph-mon[33511]: osdmap e477: 8 total, 8 up, 7 in 2024-09-20T22:19:33.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:32 smithi067 ceph-mon[33511]: 5.1c scrub ok 2024-09-20T22:19:33.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:32 smithi067 ceph-mon[33511]: pgmap v893: 178 pgs: 2 active+recovering+undersized+remapped, 6 remapped+peering, 9 peering, 161 active+clean; 3.8 GiB data, 15 GiB used, 610 GiB / 626 GiB avail; 5.2 KiB/s rd, 30 MiB/s wr, 490 op/s; 7485/188145 objects misplaced (3.978%); 2.0 MiB/s, 41 objects/s recovering 2024-09-20T22:19:33.046 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:19:33.046 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:19:33.062 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:33.103 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:19:33.103333+0000 min lat: 0.00118665 max lat: 2.19582 avg lat: 0.00990419 2024-09-20T22:19:33.103 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:19:33.104 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 40 16 64605 64589 100.896 48.25 0.0328439 0.00990419 2024-09-20T22:19:33.132 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:33.133 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:33.133 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:33.234 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:33.256 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:33.256 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:33.257 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:33.357 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:33.403 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:33.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:33.404 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:33.505 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:33.552 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:33.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:33.554 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:33.655 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:33.677 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:33.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:33.678 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:33.779 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:33.805 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:33.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:33.806 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:33.907 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:33.953 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:33.954 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:33.954 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:34.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:33 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:19:34.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:33 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:19:34.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:33 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:19:34.055 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:34.084 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:34.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:34.085 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:34.103 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 41 16 65307 65291 99.5057 43.875 0.0109259 0.0100439 2024-09-20T22:19:34.144 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:19:34.186 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:34.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:34.247 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:34.247 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:34.347 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:34.372 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:34.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:34.373 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:34.474 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:34.548 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:34.549 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:34.549 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:34.628 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:19:34.628 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:19:34.650 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:34.671 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:19:34.671 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:19:34.677 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:34.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:34.678 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:34.709 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:19:34.709 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:19:34.742 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:19:34.742 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:19:34.778 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:34.779 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:19:34.779 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:19:34.780 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:34 smithi060 ceph-mon[29694]: pgmap v894: 178 pgs: 4 active+recovering+undersized+remapped, 4 peering, 170 active+clean; 3.9 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 4.1 KiB/s rd, 34 MiB/s wr, 542 op/s; 14989/192579 objects misplaced (7.783%); 3.8 MiB/s, 66 objects/s recovering 2024-09-20T22:19:34.780 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:34 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1338311783' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:34.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:34 smithi060 ceph-mon[40290]: pgmap v894: 178 pgs: 4 active+recovering+undersized+remapped, 4 peering, 170 active+clean; 3.9 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 4.1 KiB/s rd, 34 MiB/s wr, 542 op/s; 14989/192579 objects misplaced (7.783%); 3.8 MiB/s, 66 objects/s recovering 2024-09-20T22:19:34.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:34 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1338311783' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:34.804 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:34.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:34.805 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:34.814 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:19:34.814 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:19:34.849 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:19:34.849 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:19:34.895 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:19:34.895 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:19:34.906 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:34.946 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:34.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:34.947 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:34 smithi067 ceph-mon[33511]: pgmap v894: 178 pgs: 4 active+recovering+undersized+remapped, 4 peering, 170 active+clean; 3.9 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 4.1 KiB/s rd, 34 MiB/s wr, 542 op/s; 14989/192579 objects misplaced (7.783%); 3.8 MiB/s, 66 objects/s recovering 2024-09-20T22:19:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:34 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1338311783' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:35.047 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:35.092 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:35.093 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:35.093 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:35.103 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 42 16 66073 66057 98.2765 47.875 0.00756895 0.0101691 2024-09-20T22:19:35.194 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:35.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:35.233 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:35.233 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:35.269 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T22:19:35.334 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:35.347 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:19:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:19:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:19:35.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:35.397 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:35.397 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:35.498 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:35.524 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:35.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:35.525 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:35.626 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:35.664 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:35 smithi060 ceph-mon[29694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-20T22:19:35.691 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:35.691 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:35.691 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:35.792 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:35.819 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:35.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:35.820 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:35.921 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:35.956 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:35.958 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:35.958 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:35 smithi067 ceph-mon[33511]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-20T22:19:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:35 smithi067 ceph-mon[33511]: 1.1e scrub starts 2024-09-20T22:19:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:35 smithi067 ceph-mon[33511]: 1.1e scrub ok 2024-09-20T22:19:36.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:35 smithi060 ceph-mon[29694]: 1.1e scrub starts 2024-09-20T22:19:36.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:35 smithi060 ceph-mon[29694]: 1.1e scrub ok 2024-09-20T22:19:36.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:35 smithi060 ceph-mon[40290]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-20T22:19:36.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:35 smithi060 ceph-mon[40290]: 1.1e scrub starts 2024-09-20T22:19:36.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:35 smithi060 ceph-mon[40290]: 1.1e scrub ok 2024-09-20T22:19:36.059 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:36.084 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:36.085 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:36.085 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:36.103 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 43 16 66910 66894 97.2076 52.3125 0.0048747 0.01028 2024-09-20T22:19:36.186 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:36.209 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:36.209 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:36.210 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:36.310 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:36.336 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:36.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:36.337 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:36.438 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:36.474 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:36.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:36.475 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:36.575 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:36.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:36.618 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:36.619 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:36.719 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:36.749 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:36.750 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:36.750 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:36.851 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:36.884 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:36.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:36.885 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:36.985 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:37.010 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:37.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:37.011 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:36 smithi067 ceph-mon[33511]: 5.1 scrub starts 2024-09-20T22:19:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:36 smithi067 ceph-mon[33511]: 5.1 scrub ok 2024-09-20T22:19:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:36 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:19:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:36 smithi067 ceph-mon[33511]: pgmap v895: 178 pgs: 4 active+recovering+undersized+remapped, 174 active+clean; 4.0 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 4.3 KiB/s rd, 36 MiB/s wr, 573 op/s; 14989/194739 objects misplaced (7.697%); 3.2 MiB/s, 55 objects/s recovering 2024-09-20T22:19:37.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:36 smithi060 ceph-mon[29694]: 5.1 scrub starts 2024-09-20T22:19:37.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:36 smithi060 ceph-mon[29694]: 5.1 scrub ok 2024-09-20T22:19:37.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:36 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:19:37.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:36 smithi060 ceph-mon[29694]: pgmap v895: 178 pgs: 4 active+recovering+undersized+remapped, 174 active+clean; 4.0 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 4.3 KiB/s rd, 36 MiB/s wr, 573 op/s; 14989/194739 objects misplaced (7.697%); 3.2 MiB/s, 55 objects/s recovering 2024-09-20T22:19:37.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:36 smithi060 ceph-mon[40290]: 5.1 scrub starts 2024-09-20T22:19:37.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:36 smithi060 ceph-mon[40290]: 5.1 scrub ok 2024-09-20T22:19:37.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:36 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:19:37.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:36 smithi060 ceph-mon[40290]: pgmap v895: 178 pgs: 4 active+recovering+undersized+remapped, 174 active+clean; 4.0 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 4.3 KiB/s rd, 36 MiB/s wr, 573 op/s; 14989/194739 objects misplaced (7.697%); 3.2 MiB/s, 55 objects/s recovering 2024-09-20T22:19:37.103 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 44 16 67566 67550 95.9302 41 0.0179925 0.0103876 2024-09-20T22:19:37.112 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:37.135 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:37.136 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:37.136 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:37.237 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:37.260 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:37.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:37.262 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:37.362 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:37.395 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:37.434 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:37.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:37.435 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:37.438 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:37.458 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:37.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:37.589 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:37.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:37.590 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:37.691 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:37.713 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:37.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:37.714 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:37.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:37.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:37.858 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:37.858 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:37.959 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:37.984 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:37.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:37.984 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:37 smithi067 ceph-mon[33511]: 6.2 scrub starts 2024-09-20T22:19:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:37 smithi067 ceph-mon[33511]: 6.2 scrub ok 2024-09-20T22:19:38.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:37 smithi060 ceph-mon[29694]: 6.2 scrub starts 2024-09-20T22:19:38.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:37 smithi060 ceph-mon[29694]: 6.2 scrub ok 2024-09-20T22:19:38.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:37 smithi060 ceph-mon[40290]: 6.2 scrub starts 2024-09-20T22:19:38.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:37 smithi060 ceph-mon[40290]: 6.2 scrub ok 2024-09-20T22:19:38.085 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:38.104 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 45 16 68156 68140 94.618 36.875 0.0345646 0.0105606 2024-09-20T22:19:38.113 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:38.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:38.114 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:38.215 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:38.240 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:38.240 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:38.240 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:38.341 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:38.365 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:38.365 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:38.366 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:38.466 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:38.511 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:38.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:38.512 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:38.613 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:38.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:38.661 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:38.661 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:38.761 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:38.790 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:38.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:38.791 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:38.892 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:38.922 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:38.922 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:38.923 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:38 smithi067 ceph-mon[33511]: 3.c scrub starts 2024-09-20T22:19:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:38 smithi067 ceph-mon[33511]: 3.c scrub ok 2024-09-20T22:19:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:38 smithi067 ceph-mon[33511]: pgmap v896: 178 pgs: 4 active+recovering+undersized+remapped, 174 active+clean; 4.1 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 3.2 KiB/s rd, 45 MiB/s wr, 724 op/s; 14645/201660 objects misplaced (7.262%); 5.1 MiB/s, 85 objects/s recovering 2024-09-20T22:19:39.023 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:38 smithi060 ceph-mon[29694]: 3.c scrub starts 2024-09-20T22:19:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:38 smithi060 ceph-mon[29694]: 3.c scrub ok 2024-09-20T22:19:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:38 smithi060 ceph-mon[29694]: pgmap v896: 178 pgs: 4 active+recovering+undersized+remapped, 174 active+clean; 4.1 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 3.2 KiB/s rd, 45 MiB/s wr, 724 op/s; 14645/201660 objects misplaced (7.262%); 5.1 MiB/s, 85 objects/s recovering 2024-09-20T22:19:39.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:38 smithi060 ceph-mon[40290]: 3.c scrub starts 2024-09-20T22:19:39.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:38 smithi060 ceph-mon[40290]: 3.c scrub ok 2024-09-20T22:19:39.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:38 smithi060 ceph-mon[40290]: pgmap v896: 178 pgs: 4 active+recovering+undersized+remapped, 174 active+clean; 4.1 GiB data, 16 GiB used, 610 GiB / 626 GiB avail; 3.2 KiB/s rd, 45 MiB/s wr, 724 op/s; 14645/201660 objects misplaced (7.262%); 5.1 MiB/s, 85 objects/s recovering 2024-09-20T22:19:39.051 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:39.051 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:39.051 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:39.104 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 46 16 68946 68930 93.6345 49.375 0.0324253 0.0106734 2024-09-20T22:19:39.152 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:39.183 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:39.184 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:39.185 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:39.285 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:39.307 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:39.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:39.308 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:39.354 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:19:39.409 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:39.439 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:39.440 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:39.440 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:39.541 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:39.580 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:39.580 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:39.581 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:39.681 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:39.700 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:39 smithi060 ceph-mon[29694]: 6.1b scrub starts 2024-09-20T22:19:39.700 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:39 smithi060 ceph-mon[29694]: 6.1b scrub ok 2024-09-20T22:19:39.700 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:39 smithi060 ceph-mon[40290]: 6.1b scrub starts 2024-09-20T22:19:39.700 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:39 smithi060 ceph-mon[40290]: 6.1b scrub ok 2024-09-20T22:19:39.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:39.739 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:39.739 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:39.840 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:39.863 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:39.864 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:39.864 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:39.965 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:40.009 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:40.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:40.010 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:39 smithi067 ceph-mon[33511]: 6.1b scrub starts 2024-09-20T22:19:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:39 smithi067 ceph-mon[33511]: 6.1b scrub ok 2024-09-20T22:19:40.104 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 47 16 69727 69711 92.6808 48.8125 0.00579819 0.0107832 2024-09-20T22:19:40.111 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:40.140 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:40.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:40.141 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:40.241 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:40.271 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:40.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:40.272 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:40.372 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:40.397 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:40.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:40.398 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:40.499 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:40.525 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:40.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:40.526 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:40.627 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:40.674 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:40.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:40.675 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:40.776 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:40.821 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:40.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:40.823 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:40.923 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:40.950 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:40.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:40.951 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:40 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:19:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:40 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:19:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:40 smithi067 ceph-mon[33511]: osdmap e478: 8 total, 8 up, 7 in 2024-09-20T22:19:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:40 smithi067 ceph-mon[33511]: pgmap v898: 178 pgs: 4 active+recovering+undersized+remapped, 174 active+clean; 4.1 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 588 B/s rd, 39 MiB/s wr, 625 op/s; 14411/203670 objects misplaced (7.076%); 5.5 MiB/s, 88 objects/s recovering 2024-09-20T22:19:41.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:40 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:19:41.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:40 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:19:41.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:40 smithi060 ceph-mon[29694]: osdmap e478: 8 total, 8 up, 7 in 2024-09-20T22:19:41.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:40 smithi060 ceph-mon[29694]: pgmap v898: 178 pgs: 4 active+recovering+undersized+remapped, 174 active+clean; 4.1 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 588 B/s rd, 39 MiB/s wr, 625 op/s; 14411/203670 objects misplaced (7.076%); 5.5 MiB/s, 88 objects/s recovering 2024-09-20T22:19:41.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:40 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:19:41.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:40 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:19:41.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:40 smithi060 ceph-mon[40290]: osdmap e478: 8 total, 8 up, 7 in 2024-09-20T22:19:41.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:40 smithi060 ceph-mon[40290]: pgmap v898: 178 pgs: 4 active+recovering+undersized+remapped, 174 active+clean; 4.1 GiB data, 16 GiB used, 609 GiB / 626 GiB avail; 588 B/s rd, 39 MiB/s wr, 625 op/s; 14411/203670 objects misplaced (7.076%); 5.5 MiB/s, 88 objects/s recovering 2024-09-20T22:19:41.052 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:41.076 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:41.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:41.077 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:41.104 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 48 16 70582 70566 91.8634 53.4375 0.0262725 0.0108787 2024-09-20T22:19:41.178 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:41.219 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:41.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:41.220 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:41.321 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:41.370 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:41.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:41.371 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:41.472 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:41.519 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:41.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:41.520 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:41.621 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:41.663 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:41.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:41.665 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:41.765 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:41.806 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:41.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:41.807 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:41.908 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:41.946 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:41.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:41.947 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:42.002 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:19:42.002 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:41 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/686534560' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:42.002 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:41 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:42.002 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:41 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:19:42.002 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:41 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:19:42.002 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:41 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:19:42.002 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:41 smithi060 ceph-mon[40290]: osdmap e479: 8 total, 8 up, 7 in 2024-09-20T22:19:42.003 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:41 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/686534560' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:42.003 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:41 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:42.003 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:41 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:19:42.003 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:41 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:19:42.003 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:41 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:19:42.003 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:41 smithi060 ceph-mon[29694]: osdmap e479: 8 total, 8 up, 7 in 2024-09-20T22:19:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:41 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/686534560' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:41 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:41 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:19:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:41 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:19:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:41 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:19:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:41 smithi067 ceph-mon[33511]: osdmap e479: 8 total, 8 up, 7 in 2024-09-20T22:19:42.047 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:42.077 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:42.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:42.078 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:42.104 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 49 16 71283 71267 90.8828 43.8125 0.0138707 0.0109878 2024-09-20T22:19:42.179 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:42.207 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:42.208 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:42.208 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:42.309 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:42.338 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:42.339 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:42.339 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:42.440 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:42.468 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:42.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:42.470 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:42.570 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:42.615 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:42.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:42.615 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:42.716 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:42.743 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:42.744 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:42.744 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:42.845 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:42.937 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:42.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:42.938 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:43.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:19:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:19:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:42 smithi067 ceph-mon[33511]: 4.17 scrub starts 2024-09-20T22:19:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:42 smithi067 ceph-mon[33511]: 4.17 scrub ok 2024-09-20T22:19:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:42 smithi067 ceph-mon[33511]: pgmap v900: 178 pgs: 4 active+recovering+undersized+remapped, 174 active+clean; 4.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 44 MiB/s wr, 701 op/s; 14055/209397 objects misplaced (6.712%); 7.3 MiB/s, 116 objects/s recovering 2024-09-20T22:19:43.035 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:42 smithi060 ceph-mon[40290]: 4.17 scrub starts 2024-09-20T22:19:43.035 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:42 smithi060 ceph-mon[40290]: 4.17 scrub ok 2024-09-20T22:19:43.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:42 smithi060 ceph-mon[40290]: pgmap v900: 178 pgs: 4 active+recovering+undersized+remapped, 174 active+clean; 4.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 44 MiB/s wr, 701 op/s; 14055/209397 objects misplaced (6.712%); 7.3 MiB/s, 116 objects/s recovering 2024-09-20T22:19:43.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:42 smithi060 ceph-mon[29694]: 4.17 scrub starts 2024-09-20T22:19:43.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:42 smithi060 ceph-mon[29694]: 4.17 scrub ok 2024-09-20T22:19:43.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:42 smithi060 ceph-mon[29694]: pgmap v900: 178 pgs: 4 active+recovering+undersized+remapped, 174 active+clean; 4.2 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 1.2 KiB/s rd, 44 MiB/s wr, 701 op/s; 14055/209397 objects misplaced (6.712%); 7.3 MiB/s, 116 objects/s recovering 2024-09-20T22:19:43.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:43.087 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:43.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:43.090 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:43.104 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 50 16 72120 72104 90.111 52.3125 0.0199832 0.0110896 2024-09-20T22:19:43.192 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:43.219 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:43.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:43.220 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:43.321 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:43.359 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:43.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:43.361 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:43.461 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:43.508 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:43.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:43.509 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:43.610 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:43.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:43.662 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:43.663 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:43.763 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:43.787 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:43.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:43.788 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:43.889 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:43.911 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:19:43.911 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:19:43.911 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:19:43.911 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:19:43.911 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:19:43.917 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:43.918 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:43.918 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:44.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:43 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:19:44.019 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:44.035 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:43 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:19:44.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:43 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:19:44.060 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:44.061 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:44.062 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:44.104 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 51 16 72882 72866 89.278 47.625 0.0442483 0.0111936 2024-09-20T22:19:44.163 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:44.212 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:44.213 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:44.213 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:44.314 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:44.341 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:44.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:44.342 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:44.442 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:44.472 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:44.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:44.473 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:44.535 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:44 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a[29671]: 2024-09-20T22:19:44.139+0000 7f01adcc6640 -1 mon.a@0(leader).osd e479 definitely_dead 0 2024-09-20T22:19:44.574 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:44.618 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:44.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:44.619 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:44.719 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:44.728 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:44.768 INFO:teuthology.orchestra.run.smithi060.stderr:marked out osd.1. 2024-09-20T22:19:44.788 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T22:19:44.798 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:44.803 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:44.803 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:44.846 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:19:44.904 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:44.926 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:44.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:44.927 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:44 smithi067 ceph-mon[33511]: 6.6 scrub starts 2024-09-20T22:19:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:44 smithi067 ceph-mon[33511]: 6.6 scrub ok 2024-09-20T22:19:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:44 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3460541550' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["1"]}]: dispatch 2024-09-20T22:19:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:44 smithi067 ceph-mon[33511]: Client client.admin marked osd.1 out, while it was still marked up 2024-09-20T22:19:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:44 smithi067 ceph-mon[33511]: pgmap v901: 178 pgs: 4 active+recovering+undersized+remapped, 174 active+clean; 4.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 53 MiB/s wr, 849 op/s; 13784/215118 objects misplaced (6.408%); 9.4 MiB/s, 150 objects/s recovering 2024-09-20T22:19:45.028 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:45.035 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:44 smithi060 ceph-mon[40290]: 6.6 scrub starts 2024-09-20T22:19:45.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:44 smithi060 ceph-mon[40290]: 6.6 scrub ok 2024-09-20T22:19:45.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:44 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3460541550' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["1"]}]: dispatch 2024-09-20T22:19:45.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:44 smithi060 ceph-mon[40290]: Client client.admin marked osd.1 out, while it was still marked up 2024-09-20T22:19:45.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:44 smithi060 ceph-mon[40290]: pgmap v901: 178 pgs: 4 active+recovering+undersized+remapped, 174 active+clean; 4.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 53 MiB/s wr, 849 op/s; 13784/215118 objects misplaced (6.408%); 9.4 MiB/s, 150 objects/s recovering 2024-09-20T22:19:45.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:44 smithi060 ceph-mon[29694]: 6.6 scrub starts 2024-09-20T22:19:45.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:44 smithi060 ceph-mon[29694]: 6.6 scrub ok 2024-09-20T22:19:45.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:44 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3460541550' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["1"]}]: dispatch 2024-09-20T22:19:45.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:44 smithi060 ceph-mon[29694]: Client client.admin marked osd.1 out, while it was still marked up 2024-09-20T22:19:45.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:44 smithi060 ceph-mon[29694]: pgmap v901: 178 pgs: 4 active+recovering+undersized+remapped, 174 active+clean; 4.4 GiB data, 17 GiB used, 609 GiB / 626 GiB avail; 639 B/s rd, 53 MiB/s wr, 849 op/s; 13784/215118 objects misplaced (6.408%); 9.4 MiB/s, 150 objects/s recovering 2024-09-20T22:19:45.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:45.051 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:45.051 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:45.104 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 52 16 73466 73450 88.2631 36.5 0.00306839 0.0112546 2024-09-20T22:19:45.152 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:45.174 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:45.175 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:45.175 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:45.276 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:45.298 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:45.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:45.299 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:45.357 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:19:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:19:45.399 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:45.421 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:45.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:45.422 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:45.474 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:19:45.474 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:19:45.480 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:19:45.480 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:19:45.498 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:19:45.498 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:19:45.520 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:19:45.520 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:19:45.523 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:45.537 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:19:45.538 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:19:45.545 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:45.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:45.546 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:45.555 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:19:45.556 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:19:45.572 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:19:45.572 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:19:45.588 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:19:45.589 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:19:45.647 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:45.669 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:45.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:45.670 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:45.771 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:45.820 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:45.821 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:45.821 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:45.911 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3460541550' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["1"]}]': finished 2024-09-20T22:19:45.912 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-mon[29694]: osdmap e480: 8 total, 8 up, 6 in 2024-09-20T22:19:45.912 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-mon[29694]: 6.1e deep-scrub starts 2024-09-20T22:19:45.912 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-mon[29694]: 6.1e deep-scrub ok 2024-09-20T22:19:45.912 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-mon[29694]: 3.5 scrub starts 2024-09-20T22:19:45.912 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-mon[29694]: 3.5 scrub ok 2024-09-20T22:19:45.912 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-mon[29694]: 4.19 scrub starts 2024-09-20T22:19:45.912 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-mon[29694]: 4.19 scrub ok 2024-09-20T22:19:45.912 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1121094626' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:45.912 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3460541550' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["1"]}]': finished 2024-09-20T22:19:45.912 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-mon[40290]: osdmap e480: 8 total, 8 up, 6 in 2024-09-20T22:19:45.912 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-mon[40290]: 6.1e deep-scrub starts 2024-09-20T22:19:45.912 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-mon[40290]: 6.1e deep-scrub ok 2024-09-20T22:19:45.912 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-mon[40290]: 3.5 scrub starts 2024-09-20T22:19:45.913 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-mon[40290]: 3.5 scrub ok 2024-09-20T22:19:45.913 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-mon[40290]: 4.19 scrub starts 2024-09-20T22:19:45.913 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-mon[40290]: 4.19 scrub ok 2024-09-20T22:19:45.913 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:45 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1121094626' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:45.922 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:45.947 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:45.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:45.948 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:46.049 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:46.071 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:46.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:46.072 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:46.105 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 53 16 73477 73461 86.6109 0.6875 0.00543927 0.0113204 2024-09-20T22:19:46.148 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:46.172 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:46.194 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:46.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:46.195 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:46.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:45 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3460541550' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["1"]}]': finished 2024-09-20T22:19:46.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:45 smithi067 ceph-mon[33511]: osdmap e480: 8 total, 8 up, 6 in 2024-09-20T22:19:46.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:45 smithi067 ceph-mon[33511]: 6.1e deep-scrub starts 2024-09-20T22:19:46.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:45 smithi067 ceph-mon[33511]: 6.1e deep-scrub ok 2024-09-20T22:19:46.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:45 smithi067 ceph-mon[33511]: 3.5 scrub starts 2024-09-20T22:19:46.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:45 smithi067 ceph-mon[33511]: 3.5 scrub ok 2024-09-20T22:19:46.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:45 smithi067 ceph-mon[33511]: 4.19 scrub starts 2024-09-20T22:19:46.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:45 smithi067 ceph-mon[33511]: 4.19 scrub ok 2024-09-20T22:19:46.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:45 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1121094626' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:46.296 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:46.319 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:46.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:46.320 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:46.421 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:46.443 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:46.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:46.444 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:46.544 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:46.566 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:46.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:46.567 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:46.615 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:46.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:46.690 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:46.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:46.690 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:46.791 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:46.832 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:46.832 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:46.832 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:46.876 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:46 smithi060 ceph-mon[29694]: osdmap e481: 8 total, 8 up, 6 in 2024-09-20T22:19:46.876 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:46 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:19:46.876 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:46 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:19:46.876 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:46 smithi060 ceph-mon[29694]: pgmap v904: 178 pgs: 7 remapped+peering, 3 active+recovering+undersized+remapped, 168 active+clean; 4.4 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1.2 KiB/s rd, 47 MiB/s wr, 751 op/s; 10327/217182 objects misplaced (4.755%); 6.8 MiB/s, 110 objects/s recovering 2024-09-20T22:19:46.876 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:46 smithi060 ceph-mon[29694]: osdmap e482: 8 total, 8 up, 6 in 2024-09-20T22:19:46.876 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:46 smithi060 ceph-mon[40290]: osdmap e481: 8 total, 8 up, 6 in 2024-09-20T22:19:46.876 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:46 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:19:46.876 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:46 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:19:46.876 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:46 smithi060 ceph-mon[40290]: pgmap v904: 178 pgs: 7 remapped+peering, 3 active+recovering+undersized+remapped, 168 active+clean; 4.4 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1.2 KiB/s rd, 47 MiB/s wr, 751 op/s; 10327/217182 objects misplaced (4.755%); 6.8 MiB/s, 110 objects/s recovering 2024-09-20T22:19:46.877 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:46 smithi060 ceph-mon[40290]: osdmap e482: 8 total, 8 up, 6 in 2024-09-20T22:19:46.933 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:46.966 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:46.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:46.967 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:47.067 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:47.104 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:47.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:47.105 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:47.105 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 54 16 73594 73578 85.1426 7.3125 0.0223046 0.011736 2024-09-20T22:19:47.206 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:47.252 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:47.253 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:47.253 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:47.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:46 smithi067 ceph-mon[33511]: osdmap e481: 8 total, 8 up, 6 in 2024-09-20T22:19:47.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:46 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:19:47.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:46 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:19:47.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:46 smithi067 ceph-mon[33511]: pgmap v904: 178 pgs: 7 remapped+peering, 3 active+recovering+undersized+remapped, 168 active+clean; 4.4 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1.2 KiB/s rd, 47 MiB/s wr, 751 op/s; 10327/217182 objects misplaced (4.755%); 6.8 MiB/s, 110 objects/s recovering 2024-09-20T22:19:47.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:46 smithi067 ceph-mon[33511]: osdmap e482: 8 total, 8 up, 6 in 2024-09-20T22:19:47.354 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:47.384 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:47.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:47.385 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:47.486 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:47.516 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:47.517 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:47.517 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:47.618 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:47.651 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:47.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:47.652 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:47.753 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:47.781 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:47.782 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:47.782 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:47.823 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:19:47.882 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:47.926 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:47.927 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:47.927 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:48.028 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:48.065 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:48.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:48.066 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:48.105 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 55 16 74332 74316 84.4332 46.125 0.0246817 0.0118379 2024-09-20T22:19:48.152 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:19:48.152 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:19:48.152 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:19:48.152 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:19:48.152 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[40290]: 3.19 scrub starts 2024-09-20T22:19:48.152 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[40290]: 3.19 scrub ok 2024-09-20T22:19:48.152 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[40290]: 11.c starting backfill to osd.5 from (0'0,0'0] MAX to 479'4601 2024-09-20T22:19:48.152 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[40290]: 11.3 starting backfill to osd.5 from (0'0,0'0] MAX to 479'4519 2024-09-20T22:19:48.153 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[40290]: 11.0 starting backfill to osd.6 from (0'0,0'0] MAX to 479'4513 2024-09-20T22:19:48.153 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[40290]: 11.b starting backfill to osd.5 from (0'0,0'0] MAX to 479'4621 2024-09-20T22:19:48.153 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[40290]: 11.f starting backfill to osd.0 from (0'0,0'0] MAX to 479'4665 2024-09-20T22:19:48.153 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[40290]: 11.e starting backfill to osd.3 from (0'0,0'0] MAX to 479'4505 2024-09-20T22:19:48.153 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[40290]: 11.7 starting backfill to osd.7 from (0'0,0'0] MAX to 479'4498 2024-09-20T22:19:48.153 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[40290]: 11.6 starting backfill to osd.5 from (0'0,0'0] MAX to 479'4456 2024-09-20T22:19:48.153 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2037237620' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:19:48.153 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:19:48.153 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:19:48.153 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:19:48.153 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[29694]: 3.19 scrub starts 2024-09-20T22:19:48.153 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[29694]: 3.19 scrub ok 2024-09-20T22:19:48.154 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[29694]: 11.c starting backfill to osd.5 from (0'0,0'0] MAX to 479'4601 2024-09-20T22:19:48.154 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[29694]: 11.3 starting backfill to osd.5 from (0'0,0'0] MAX to 479'4519 2024-09-20T22:19:48.154 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[29694]: 11.0 starting backfill to osd.6 from (0'0,0'0] MAX to 479'4513 2024-09-20T22:19:48.154 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[29694]: 11.b starting backfill to osd.5 from (0'0,0'0] MAX to 479'4621 2024-09-20T22:19:48.154 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[29694]: 11.f starting backfill to osd.0 from (0'0,0'0] MAX to 479'4665 2024-09-20T22:19:48.154 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[29694]: 11.e starting backfill to osd.3 from (0'0,0'0] MAX to 479'4505 2024-09-20T22:19:48.154 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[29694]: 11.7 starting backfill to osd.7 from (0'0,0'0] MAX to 479'4498 2024-09-20T22:19:48.154 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[29694]: 11.6 starting backfill to osd.5 from (0'0,0'0] MAX to 479'4456 2024-09-20T22:19:48.154 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2037237620' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:19:48.154 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:47 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:19:48.167 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:48.203 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:48.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:48.204 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:48.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:47 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:19:48.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:47 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:19:48.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:47 smithi067 ceph-mon[33511]: 3.19 scrub starts 2024-09-20T22:19:48.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:47 smithi067 ceph-mon[33511]: 3.19 scrub ok 2024-09-20T22:19:48.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:47 smithi067 ceph-mon[33511]: 11.c starting backfill to osd.5 from (0'0,0'0] MAX to 479'4601 2024-09-20T22:19:48.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:47 smithi067 ceph-mon[33511]: 11.3 starting backfill to osd.5 from (0'0,0'0] MAX to 479'4519 2024-09-20T22:19:48.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:47 smithi067 ceph-mon[33511]: 11.0 starting backfill to osd.6 from (0'0,0'0] MAX to 479'4513 2024-09-20T22:19:48.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:47 smithi067 ceph-mon[33511]: 11.b starting backfill to osd.5 from (0'0,0'0] MAX to 479'4621 2024-09-20T22:19:48.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:47 smithi067 ceph-mon[33511]: 11.f starting backfill to osd.0 from (0'0,0'0] MAX to 479'4665 2024-09-20T22:19:48.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:47 smithi067 ceph-mon[33511]: 11.e starting backfill to osd.3 from (0'0,0'0] MAX to 479'4505 2024-09-20T22:19:48.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:47 smithi067 ceph-mon[33511]: 11.7 starting backfill to osd.7 from (0'0,0'0] MAX to 479'4498 2024-09-20T22:19:48.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:47 smithi067 ceph-mon[33511]: 11.6 starting backfill to osd.5 from (0'0,0'0] MAX to 479'4456 2024-09-20T22:19:48.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:47 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2037237620' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:19:48.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:47 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:19:48.305 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:48.346 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:48.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:48.347 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:48.448 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:48.491 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:48.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:48.492 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:48.593 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:48.610 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:19:48.632 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:48.633 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:48.633 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:48.734 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:48.771 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:48.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:48.772 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:48.874 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:19:48.909 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:48.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:48.910 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:19:48.943 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T22:19:49.011 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:49.059 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:49.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:49.060 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:49.105 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 56 16 75197 75181 83.8901 54.0625 0.0184382 0.0119135 2024-09-20T22:19:49.161 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:49.204 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:49.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:49.206 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:49.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:48 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:19:49.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:48 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:19:49.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:48 smithi067 ceph-mon[33511]: osdmap e483: 8 total, 8 up, 6 in 2024-09-20T22:19:49.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:48 smithi067 ceph-mon[33511]: pgmap v907: 178 pgs: 1 active+remapped+backfill_wait, 1 peering, 1 active+remapped+backfilling, 3 active+undersized+remapped, 1 active+recovering+undersized+remapped, 1 activating+remapped, 12 remapped+peering, 158 active+clean; 4.5 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 1.7 KiB/s rd, 35 MiB/s wr, 562 op/s; 18151/221847 objects misplaced (8.182%); 7.1 MiB/s, 135 objects/s recovering 2024-09-20T22:19:49.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:48 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:19:49.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:48 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:19:49.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:48 smithi060 ceph-mon[29694]: osdmap e483: 8 total, 8 up, 6 in 2024-09-20T22:19:49.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:48 smithi060 ceph-mon[29694]: pgmap v907: 178 pgs: 1 active+remapped+backfill_wait, 1 peering, 1 active+remapped+backfilling, 3 active+undersized+remapped, 1 active+recovering+undersized+remapped, 1 activating+remapped, 12 remapped+peering, 158 active+clean; 4.5 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 1.7 KiB/s rd, 35 MiB/s wr, 562 op/s; 18151/221847 objects misplaced (8.182%); 7.1 MiB/s, 135 objects/s recovering 2024-09-20T22:19:49.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:48 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:19:49.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:48 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:19:49.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:48 smithi060 ceph-mon[40290]: osdmap e483: 8 total, 8 up, 6 in 2024-09-20T22:19:49.287 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:48 smithi060 ceph-mon[40290]: pgmap v907: 178 pgs: 1 active+remapped+backfill_wait, 1 peering, 1 active+remapped+backfilling, 3 active+undersized+remapped, 1 active+recovering+undersized+remapped, 1 activating+remapped, 12 remapped+peering, 158 active+clean; 4.5 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 1.7 KiB/s rd, 35 MiB/s wr, 562 op/s; 18151/221847 objects misplaced (8.182%); 7.1 MiB/s, 135 objects/s recovering 2024-09-20T22:19:49.306 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:49.365 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:49.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:49.366 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:49.467 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:49.520 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:49.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:49.521 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:49.621 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:49.656 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:49.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:49.657 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:49.757 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:49.758 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:49.816 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:49.830 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:49.831 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:49.831 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:49.899 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:49 smithi060 ceph-mon[29694]: osdmap e484: 8 total, 8 up, 6 in 2024-09-20T22:19:49.899 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:49 smithi060 ceph-mon[40290]: osdmap e484: 8 total, 8 up, 6 in 2024-09-20T22:19:49.933 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:49.960 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:49.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:49.961 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:50.061 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:50.100 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:50.101 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:50.101 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:50.105 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 57 16 75986 75970 83.2834 49.3125 0.0407892 0.0120003 2024-09-20T22:19:50.202 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:50.241 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:50.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:50.242 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:50.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:49 smithi067 ceph-mon[33511]: osdmap e484: 8 total, 8 up, 6 in 2024-09-20T22:19:50.343 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:50.376 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:50.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:50.378 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:50.478 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:50.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:50.522 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:50.522 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:50.623 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:50.660 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:50.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:50.661 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:50.681 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:19:50.682 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:19:50.733 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:19:50.733 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:19:50.762 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:50.771 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:19:50.771 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:19:50.791 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:19:50.791 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:19:50.791 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.791 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:76554 11.6 11:6d3d2787:::benchmark_data_smithi060_196073_object76553:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536 in=65536b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e484)", 2024-09-20T22:19:50.792 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:19:50.738934+0000", 2024-09-20T22:19:50.792 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0081848719999999993, 2024-09-20T22:19:50.792 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0082198689999999994, 2024-09-20T22:19:50.792 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:19:50.792 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:19:50.792 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:19:50.792 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:19:50.792 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:19:50.792 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:19:50.792 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 76554 2024-09-20T22:19:50.792 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.792 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:19:50.792 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.792 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:19:50.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.738934+0000", 2024-09-20T22:19:50.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:19:50.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:19:50.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.738934+0000", 2024-09-20T22:19:50.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:19:50.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:19:50.793 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.738936+0000", 2024-09-20T22:19:50.793 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6500000000000001e-06 2024-09-20T22:19:50.793 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.793 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.793 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:19:50.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.738958+0000", 2024-09-20T22:19:50.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1835e-05 2024-09-20T22:19:50.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:19:50.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.738971+0000", 2024-09-20T22:19:50.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3739e-05 2024-09-20T22:19:50.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:19:50.794 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.738980+0000", 2024-09-20T22:19:50.794 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2910000000000007e-06 2024-09-20T22:19:50.794 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.794 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.794 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:19:50.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.740212+0000", 2024-09-20T22:19:50.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012317599999999999 2024-09-20T22:19:50.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:19:50.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.740401+0000", 2024-09-20T22:19:50.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000189264 2024-09-20T22:19:50.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.795 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.795 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 1,2,5", 2024-09-20T22:19:50.795 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.740608+0000", 2024-09-20T22:19:50.795 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020758800000000001 2024-09-20T22:19:50.795 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:19:50.795 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:19:50.795 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:19:50.795 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.796 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "MOSDPGPush(11.f 484/481 [PushOp(11:f14ae7e4:::benchmark_data_smithi060_196073_object68201:head, version: 477'4327, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(11:f14ae7e4:::benchmark_data_smithi060_196073_object68201:head@477'4327, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-20T22:19:50.796 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:19:50.743216+0000", 2024-09-20T22:19:50.796 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.003903559, 2024-09-20T22:19:50.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0040258050000000004, 2024-09-20T22:19:50.796 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:19:50.796 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:19:50.796 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:19:50.796 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:19:50.796 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.796 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:19:50.796 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.743216+0000", 2024-09-20T22:19:50.796 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:19:50.796 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:19:50.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.743216+0000", 2024-09-20T22:19:50.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:19:50.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:19:50.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.743217+0000", 2024-09-20T22:19:50.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0049999999999999e-06 2024-09-20T22:19:50.797 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.797 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.797 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:19:50.797 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.743419+0000", 2024-09-20T22:19:50.797 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000202246 2024-09-20T22:19:50.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.798 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:19:50.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.743441+0000", 2024-09-20T22:19:50.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1806000000000001e-05 2024-09-20T22:19:50.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.798 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:19:50.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.743470+0000", 2024-09-20T22:19:50.798 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.9295e-05 2024-09-20T22:19:50.798 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.798 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.798 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:19:50.798 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.743990+0000", 2024-09-20T22:19:50.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00052000699999999996 2024-09-20T22:19:50.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.799 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:19:50.799 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.744622+0000", 2024-09-20T22:19:50.799 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00063182099999999997 2024-09-20T22:19:50.799 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:19:50.799 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:19:50.799 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:19:50.799 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.799 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.799 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16440.0:76557 11.1 e484/456 11:88f535e8:::benchmark_data_smithi060_196073_object76556:head v 484'4827, mlcod=484'4824)", 2024-09-20T22:19:50.799 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:19:50.744590+0000", 2024-09-20T22:19:50.800 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0025292549999999998, 2024-09-20T22:19:50.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002700956, 2024-09-20T22:19:50.800 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:19:50.800 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:19:50.800 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:19:50.800 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:19:50.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:19:50.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.744590+0000", 2024-09-20T22:19:50.800 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:19:50.800 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.800 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.800 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:19:50.800 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.744590+0000", 2024-09-20T22:19:50.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:19:50.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:19:50.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.744591+0000", 2024-09-20T22:19:50.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6599999999999996e-07 2024-09-20T22:19:50.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:19:50.801 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.744617+0000", 2024-09-20T22:19:50.801 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6038e-05 2024-09-20T22:19:50.801 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.801 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.801 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:19:50.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.744628+0000", 2024-09-20T22:19:50.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0943e-05 2024-09-20T22:19:50.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:19:50.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.744634+0000", 2024-09-20T22:19:50.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4819999999999999e-06 2024-09-20T22:19:50.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:19:50.802 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.744683+0000", 2024-09-20T22:19:50.802 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8690000000000003e-05 2024-09-20T22:19:50.802 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.802 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.802 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:19:50.803 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.744702+0000", 2024-09-20T22:19:50.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9667e-05 2024-09-20T22:19:50.803 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:19:50.803 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:19:50.803 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:19:50.803 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.803 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.803 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "MOSDPGPushReply(11.6 484/481 [PushReplyOp(11:61b0e8b5:::benchmark_data_smithi060_196073_object62300:head)])", 2024-09-20T22:19:50.803 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:19:50.746577+0000", 2024-09-20T22:19:50.803 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00054195100000000004, 2024-09-20T22:19:50.803 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00075886599999999997, 2024-09-20T22:19:50.803 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:19:50.803 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:19:50.803 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "reached pg", 2024-09-20T22:19:50.803 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:19:50.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:19:50.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746577+0000", 2024-09-20T22:19:50.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:19:50.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:19:50.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746577+0000", 2024-09-20T22:19:50.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:19:50.804 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.804 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.804 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:19:50.804 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746577+0000", 2024-09-20T22:19:50.804 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2000000000000001e-07 2024-09-20T22:19:50.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:19:50.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746583+0000", 2024-09-20T22:19:50.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4e-06 2024-09-20T22:19:50.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:19:50.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746590+0000", 2024-09-20T22:19:50.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1919999999999999e-06 2024-09-20T22:19:50.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.805 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.805 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:19:50.805 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746606+0000", 2024-09-20T22:19:50.805 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5777000000000002e-05 2024-09-20T22:19:50.805 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.806 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:19:50.806 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.747113+0000", 2024-09-20T22:19:50.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00050752399999999997 2024-09-20T22:19:50.806 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:19:50.806 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:19:50.806 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:19:50.806 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.806 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.806 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16440.0:76541 11.7 e484/481 11:edd9cabd:::benchmark_data_smithi060_196073_object76540:head v 484'4691, mlcod=461'706)", 2024-09-20T22:19:50.806 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:19:50.746619+0000", 2024-09-20T22:19:50.806 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.00050020499999999999, 2024-09-20T22:19:50.806 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00076200300000000003, 2024-09-20T22:19:50.806 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:19:50.806 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:19:50.806 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:19:50.807 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:19:50.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:19:50.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746619+0000", 2024-09-20T22:19:50.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:19:50.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:19:50.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746619+0000", 2024-09-20T22:19:50.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:19:50.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.807 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.807 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:19:50.807 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746619+0000", 2024-09-20T22:19:50.807 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.7599999999999998e-07 2024-09-20T22:19:50.807 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:19:50.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746660+0000", 2024-09-20T22:19:50.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.0973000000000002e-05 2024-09-20T22:19:50.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:19:50.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746667+0000", 2024-09-20T22:19:50.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2729999999999997e-06 2024-09-20T22:19:50.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.808 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:19:50.808 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746673+0000", 2024-09-20T22:19:50.808 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1320000000000002e-06 2024-09-20T22:19:50.808 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.808 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:19:50.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746978+0000", 2024-09-20T22:19:50.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00030503699999999997 2024-09-20T22:19:50.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.809 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.809 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:19:50.809 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.747002+0000", 2024-09-20T22:19:50.809 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3968000000000002e-05 2024-09-20T22:19:50.809 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:19:50.809 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:19:50.809 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:19:50.809 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.810 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16440.0:76548 11.7 e484/481)", 2024-09-20T22:19:50.810 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:19:50.746687+0000", 2024-09-20T22:19:50.810 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.000432242, 2024-09-20T22:19:50.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00073698999999999998, 2024-09-20T22:19:50.810 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:19:50.810 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:19:50.810 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:19:50.810 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:19:50.810 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.810 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:19:50.810 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746687+0000", 2024-09-20T22:19:50.810 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:19:50.810 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:19:50.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746687+0000", 2024-09-20T22:19:50.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:19:50.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:19:50.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746687+0000", 2024-09-20T22:19:50.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8000000000000002e-07 2024-09-20T22:19:50.811 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.811 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.811 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:19:50.811 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746731+0000", 2024-09-20T22:19:50.811 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3949999999999998e-05 2024-09-20T22:19:50.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:19:50.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746738+0000", 2024-09-20T22:19:50.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2880000000000004e-06 2024-09-20T22:19:50.812 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:19:50.812 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:19:50.812 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:19:50.812 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:50.746743+0000", 2024-09-20T22:19:50.812 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8339999999999998e-06 2024-09-20T22:19:50.812 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:19:50.812 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:19:50.812 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:19:50.812 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:19:50.812 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:19:50.812 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 6 2024-09-20T22:19:50.813 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:19:50.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:50.813 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:50.813 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:50.814 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:19:50.815 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:19:50.854 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:19:50.854 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:19:50.894 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:19:50.894 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:19:50.914 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:50.927 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:19:50.927 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:19:50.944 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 5, 6, 7, 0] out_osds: [4, 1] dead_osds: [] live_osds: [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:19:50.944 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:19:50.944 INFO:tasks.thrashosds.thrasher:Killing osd 4, live_osds are [0, 2, 3, 4, 5, 6, 7, 1] 2024-09-20T22:19:50.944 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-20T22:19:50.945 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.4 2024-09-20T22:19:50.955 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:50.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:50.956 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:50.990 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:19:50.991 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:19:50.991 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:50 smithi060 ceph-mon[29694]: pgmap v909: 178 pgs: 1 active+remapped+backfill_wait, 1 peering, 1 active+remapped+backfilling, 3 active+undersized+remapped, 1 active+recovering+undersized+remapped, 1 activating+remapped, 12 remapped+peering, 158 active+clean; 4.5 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 1.5 KiB/s rd, 31 MiB/s wr, 492 op/s; 18151/221847 objects misplaced (8.182%); 6.2 MiB/s, 118 objects/s recovering 2024-09-20T22:19:50.991 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:50 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2958469009' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:50.991 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:50 smithi060 ceph-mon[40290]: pgmap v909: 178 pgs: 1 active+remapped+backfill_wait, 1 peering, 1 active+remapped+backfilling, 3 active+undersized+remapped, 1 active+recovering+undersized+remapped, 1 activating+remapped, 12 remapped+peering, 158 active+clean; 4.5 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 1.5 KiB/s rd, 31 MiB/s wr, 492 op/s; 18151/221847 objects misplaced (8.182%); 6.2 MiB/s, 118 objects/s recovering 2024-09-20T22:19:50.991 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:50 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2958469009' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:51.057 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:51.093 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:51.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:51.094 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:51.105 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 58 16 76860 76844 82.7892 54.625 0.0297501 0.0120724 2024-09-20T22:19:51.195 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:51.236 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:51.237 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:51.237 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:51.263 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:19:51 smithi067 systemd[1]: Stopping Ceph osd.4 for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:19:51.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:50 smithi067 ceph-mon[33511]: pgmap v909: 178 pgs: 1 active+remapped+backfill_wait, 1 peering, 1 active+remapped+backfilling, 3 active+undersized+remapped, 1 active+recovering+undersized+remapped, 1 activating+remapped, 12 remapped+peering, 158 active+clean; 4.5 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 1.5 KiB/s rd, 31 MiB/s wr, 492 op/s; 18151/221847 objects misplaced (8.182%); 6.2 MiB/s, 118 objects/s recovering 2024-09-20T22:19:51.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:50 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2958469009' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:19:51.338 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:51.371 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:51.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:51.372 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:51.473 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:51.541 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:51.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:51.542 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:51.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:51.680 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:51.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:51.681 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:51.707 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:19:51.782 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:51.814 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:51.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:51.815 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:51.916 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:51.950 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:51.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:51.951 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:52.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:19:52.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:51 smithi060 ceph-mon[40290]: osdmap e485: 8 total, 8 up, 6 in 2024-09-20T22:19:52.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:19:52.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:51 smithi060 ceph-mon[29694]: osdmap e485: 8 total, 8 up, 6 in 2024-09-20T22:19:52.052 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:52.088 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:52.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:52.089 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:52.105 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 59 15 77664 77649 82.2388 50.3125 0.0219469 0.0121527 2024-09-20T22:19:52.190 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:52.226 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:52.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:52.227 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:52.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:19:52.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:51 smithi067 ceph-mon[33511]: osdmap e485: 8 total, 8 up, 6 in 2024-09-20T22:19:52.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:52.367 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:52.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:52.369 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:52.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T22:19:52.470 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:52.502 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:52.502 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:52.503 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:52.550 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:52.603 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:52.624 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:52.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:52.625 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:52.726 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:52.760 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:52.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:52.761 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:52.861 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:52.901 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:52.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:52.902 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:53.003 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:53.035 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:52 smithi060 ceph-mon[40290]: pgmap v911: 178 pgs: 1 active+recovering+remapped, 1 active+remapped+backfill_wait, 1 peering, 3 active+remapped+backfilling, 2 active+undersized+remapped, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 166 active+clean; 4.7 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 2.3 KiB/s rd, 40 MiB/s wr, 636 op/s; 45537/229347 objects misplaced (19.855%); 24 MiB/s, 392 objects/s recovering 2024-09-20T22:19:53.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:52 smithi060 ceph-mon[40290]: osd.4 marked itself down and dead 2024-09-20T22:19:53.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:52 smithi060 ceph-mon[29694]: pgmap v911: 178 pgs: 1 active+recovering+remapped, 1 active+remapped+backfill_wait, 1 peering, 3 active+remapped+backfilling, 2 active+undersized+remapped, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 166 active+clean; 4.7 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 2.3 KiB/s rd, 40 MiB/s wr, 636 op/s; 45537/229347 objects misplaced (19.855%); 24 MiB/s, 392 objects/s recovering 2024-09-20T22:19:53.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:52 smithi060 ceph-mon[29694]: osd.4 marked itself down and dead 2024-09-20T22:19:53.044 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:53.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:53.045 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:53.106 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:19:53.105803+0000 min lat: 0.00118665 max lat: 2.25818 avg lat: 0.0122353 2024-09-20T22:19:53.106 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:19:53.106 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 60 16 78455 78439 81.691 49.375 0.0154315 0.0122353 2024-09-20T22:19:53.146 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:53.178 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:53.179 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:53.179 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:53.250 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:19:53.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:52 smithi067 ceph-mon[33511]: pgmap v911: 178 pgs: 1 active+recovering+remapped, 1 active+remapped+backfill_wait, 1 peering, 3 active+remapped+backfilling, 2 active+undersized+remapped, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 166 active+clean; 4.7 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 2.3 KiB/s rd, 40 MiB/s wr, 636 op/s; 45537/229347 objects misplaced (19.855%); 24 MiB/s, 392 objects/s recovering 2024-09-20T22:19:53.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:52 smithi067 ceph-mon[33511]: osd.4 marked itself down and dead 2024-09-20T22:19:53.263 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:19:52 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[38685]: 2024-09-20T22:19:52.778+0000 7f1afc3da640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T22:19:53.264 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:19:52 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[38685]: 2024-09-20T22:19:52.779+0000 7f1afc3da640 -1 osd.4 485 *** Got signal Terminated *** 2024-09-20T22:19:53.264 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:19:52 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[38685]: 2024-09-20T22:19:52.779+0000 7f1afc3da640 -1 osd.4 485 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T22:19:53.264 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:19:52 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:19:53.280 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:53.314 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:53.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:53.315 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:53.415 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:53.452 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:53.452 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:53.453 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:53.553 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:53.588 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:53.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:53.589 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:53.690 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:53.714 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:53.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:53.715 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:53.816 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:53.843 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:19:53.860 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:53.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:53.862 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:53.863 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:19:53 smithi067 podman[145075]: 2024-09-20 22:19:53.525581435 +0000 UTC m=+2.153067057 container died d4b8d55522e8eecf4fa28c7cc86bfd78102b07acfc19ca19d75ad62cd6c324b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-9e2ac70, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-20T22:19:53.885 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:19:53.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:54.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:54.015 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:54.015 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:54.106 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 61 16 79079 79063 80.9912 39 0.0376469 0.0123407 2024-09-20T22:19:54.116 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:54.140 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:54.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:54.141 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:54.242 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:54.262 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:53 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:19:54.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:53 smithi067 ceph-mon[33511]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:19:54.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:53 smithi067 ceph-mon[33511]: osdmap e486: 8 total, 7 up, 6 in 2024-09-20T22:19:54.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:53 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1000022029' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:54.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:53 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:54.263 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:19:53 smithi067 podman[145075]: 2024-09-20 22:19:53.976449163 +0000 UTC m=+2.603934774 container cleanup d4b8d55522e8eecf4fa28c7cc86bfd78102b07acfc19ca19d75ad62cd6c324b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-9e2ac70, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-20T22:19:54.263 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:19:53 smithi067 bash[145075]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4 2024-09-20T22:19:54.279 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:54.279 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:54.280 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:54.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:53 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:19:54.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:53 smithi060 ceph-mon[29694]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:19:54.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:53 smithi060 ceph-mon[29694]: osdmap e486: 8 total, 7 up, 6 in 2024-09-20T22:19:54.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:53 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1000022029' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:54.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:53 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:54.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:53 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:19:54.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:53 smithi060 ceph-mon[40290]: Health check failed: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:19:54.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:53 smithi060 ceph-mon[40290]: osdmap e486: 8 total, 7 up, 6 in 2024-09-20T22:19:54.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:53 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1000022029' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:54.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:53 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:19:54.381 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:54.414 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:54.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:54.416 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:54.513 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:19:54 smithi067 podman[145315]: 2024-09-20 22:19:54.261839363 +0000 UTC m=+0.722060432 container remove d4b8d55522e8eecf4fa28c7cc86bfd78102b07acfc19ca19d75ad62cd6c324b0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4, RELEASE=squid-9e2ac70, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T22:19:54.516 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:54.560 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:54.561 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:54.561 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:54.662 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:54.699 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:54.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:54.700 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:54.801 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:54.823 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:54.823 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:54.823 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:54.924 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:54.962 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:54.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:54.963 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:55.064 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:55.107 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 62 16 79860 79844 80.4722 48.8125 0.0139471 0.0124206 2024-09-20T22:19:55.109 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:55.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:55.111 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:55.212 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:55.242 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:54 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:19:55.242 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:54 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:19:55.242 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:54 smithi067 ceph-mon[33511]: osdmap e487: 8 total, 7 up, 6 in 2024-09-20T22:19:55.242 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:54 smithi067 ceph-mon[33511]: pgmap v914: 178 pgs: 1 active+remapped+backfill_wait, 1 active+recovering+remapped, 3 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 169 active+clean; 4.7 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 928 B/s rd, 42 MiB/s wr, 677 op/s; 45389/233049 objects misplaced (19.476%); 31 MiB/s, 489 objects/s recovering 2024-09-20T22:19:55.242 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:19:54 smithi067 podman[145502]: 2024-09-20 22:19:54.804725549 +0000 UTC m=+0.019411576 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5 2024-09-20T22:19:55.242 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:19:54 smithi067 podman[145502]: 2024-09-20 22:19:54.938012975 +0000 UTC m=+0.152698983 container create 50fd75fb3cb27b0712af061a0125fd9e1fc0b20ec1167b19bc00b1edb76fe39c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-9e2ac70, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-20T22:19:55.243 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:55.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:55.244 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:55.285 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:19:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:19:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:19:55.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:54 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:19:55.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:54 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:19:55.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:54 smithi060 ceph-mon[29694]: osdmap e487: 8 total, 7 up, 6 in 2024-09-20T22:19:55.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:54 smithi060 ceph-mon[29694]: pgmap v914: 178 pgs: 1 active+remapped+backfill_wait, 1 active+recovering+remapped, 3 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 169 active+clean; 4.7 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 928 B/s rd, 42 MiB/s wr, 677 op/s; 45389/233049 objects misplaced (19.476%); 31 MiB/s, 489 objects/s recovering 2024-09-20T22:19:55.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:54 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:19:55.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:54 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:19:55.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:54 smithi060 ceph-mon[40290]: osdmap e487: 8 total, 7 up, 6 in 2024-09-20T22:19:55.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:54 smithi060 ceph-mon[40290]: pgmap v914: 178 pgs: 1 active+remapped+backfill_wait, 1 active+recovering+remapped, 3 active+remapped+backfilling, 3 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 169 active+clean; 4.7 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 928 B/s rd, 42 MiB/s wr, 677 op/s; 45389/233049 objects misplaced (19.476%); 31 MiB/s, 489 objects/s recovering 2024-09-20T22:19:55.345 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:55.392 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:55.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:55.393 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:55.494 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:55.513 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:19:55 smithi067 podman[145502]: 2024-09-20 22:19:55.241318189 +0000 UTC m=+0.456004213 container init 50fd75fb3cb27b0712af061a0125fd9e1fc0b20ec1167b19bc00b1edb76fe39c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-9e2ac70, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-20T22:19:55.513 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:19:55 smithi067 podman[145502]: 2024-09-20 22:19:55.250084536 +0000 UTC m=+0.464770547 container start 50fd75fb3cb27b0712af061a0125fd9e1fc0b20ec1167b19bc00b1edb76fe39c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, RELEASE=squid-9e2ac70, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-20T22:19:55.513 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:19:55 smithi067 podman[145502]: 2024-09-20 22:19:55.349498839 +0000 UTC m=+0.564184853 container attach 50fd75fb3cb27b0712af061a0125fd9e1fc0b20ec1167b19bc00b1edb76fe39c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-9e2ac70, org.label-schema.build-date=20240731) 2024-09-20T22:19:55.531 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:55.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:55.532 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:55.633 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:55.674 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:55.675 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:55.675 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:55.776 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:55.797 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:55.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:55.798 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:55.899 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:55.929 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:55.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:55.930 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:55.938 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:19:55 smithi067 podman[145502]: 2024-09-20 22:19:55.610598701 +0000 UTC m=+0.825284714 container died 50fd75fb3cb27b0712af061a0125fd9e1fc0b20ec1167b19bc00b1edb76fe39c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=squid-9e2ac70, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-20T22:19:56.030 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:56.070 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:56.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:56.071 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:56.106 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 63 16 80478 80462 79.8079 38.625 0.0156025 0.0125229 2024-09-20T22:19:56.171 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:56.211 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:56.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:56.212 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:56.295 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.4.service' 2024-09-20T22:19:56.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:56.346 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:56.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:56.347 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:56.374 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T22:19:56.374 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-20T22:19:56.374 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.4 2024-09-20T22:19:56.374 INFO:tasks.thrashosds.thrasher:remote for osd 4 is ubuntu@smithi067.front.sepia.ceph.com 2024-09-20T22:19:56.374 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -n osd.4 --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op list-pgs 2024-09-20T22:19:56.447 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:56.493 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:56.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:56.494 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:56.594 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:56.619 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:56.619 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:56.619 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:56.671 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/osd.4/config 2024-09-20T22:19:56.721 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:56.756 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:56.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:56.757 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:56.758 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:56 smithi060 ceph-mon[29694]: Health check failed: Degraded data redundancy: 3187/236565 objects degraded (1.347%), 1 pg degraded (PG_DEGRADED) 2024-09-20T22:19:56.758 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:56 smithi060 ceph-mon[40290]: Health check failed: Degraded data redundancy: 3187/236565 objects degraded (1.347%), 1 pg degraded (PG_DEGRADED) 2024-09-20T22:19:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:56 smithi067 ceph-mon[33511]: Health check failed: Degraded data redundancy: 3187/236565 objects degraded (1.347%), 1 pg degraded (PG_DEGRADED) 2024-09-20T22:19:56.858 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:56.896 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:56.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:56.897 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:56.997 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:57.037 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:57.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:57.037 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:57.106 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 64 16 81187 81171 79.2533 44.3125 0.00743878 0.0126114 2024-09-20T22:19:57.138 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:57.177 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:57.177 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:57.178 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:57.278 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:57.321 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:57.322 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:57.323 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:57.423 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:19:57.458 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:57.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:57.459 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:19:57.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:57 smithi067 ceph-mon[33511]: pgmap v915: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 1 active+recovering+remapped, 3 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 169 active+clean; 4.8 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 1.7 KiB/s rd, 51 MiB/s wr, 818 op/s; 3187/236565 objects degraded (1.347%); 40579/236565 objects misplaced (17.153%); 47 MiB/s, 757 objects/s recovering 2024-09-20T22:19:57.559 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:57.600 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:57.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:57.601 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:57.702 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:57.729 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:57.730 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:57.730 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:57.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:57 smithi060 ceph-mon[40290]: pgmap v915: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 1 active+recovering+remapped, 3 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 169 active+clean; 4.8 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 1.7 KiB/s rd, 51 MiB/s wr, 818 op/s; 3187/236565 objects degraded (1.347%); 40579/236565 objects misplaced (17.153%); 47 MiB/s, 757 objects/s recovering 2024-09-20T22:19:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:57 smithi060 ceph-mon[29694]: pgmap v915: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_wait, 1 active+recovering+remapped, 3 active+remapped+backfilling, 2 active+recovering+undersized+remapped, 1 active+remapped+backfill_toofull, 169 active+clean; 4.8 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 1.7 KiB/s rd, 51 MiB/s wr, 818 op/s; 3187/236565 objects degraded (1.347%); 40579/236565 objects misplaced (17.153%); 47 MiB/s, 757 objects/s recovering 2024-09-20T22:19:57.831 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:57.869 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:57.870 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:57.870 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:57.970 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:19:58.003 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:58.004 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:58.005 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:19:58.105 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:58.106 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 65 16 82056 82040 78.8696 54.3125 0.0178998 0.0126729 2024-09-20T22:19:58.128 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:58.129 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:58.129 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:58.230 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:58.255 INFO:teuthology.orchestra.run.smithi067.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-20T22:19:58.265 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:58.266 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:58.266 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:58.367 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:58.405 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:58.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:58.406 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:58.507 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:58.551 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:58.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:58.552 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:58.653 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:58.693 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:58.694 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:58.694 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:58.795 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:58.839 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:58.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:58.840 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:58.940 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:58.972 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:58.972 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:58.973 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:59.073 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:19:59.106 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 66 15 82924 82909 78.4975 54.3125 0.0223136 0.0127338 2024-09-20T22:19:59.114 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:59.114 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:19:59.115 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:19:59.215 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:59.256 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:59.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:59.257 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:59.358 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:19:59.382 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:59.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:59.383 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:19:59.484 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:59.527 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:59.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:59.528 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:19:59.629 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:19:59.670 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:59.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:59.671 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:19:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:19:59 smithi067 ceph-mon[33511]: pgmap v916: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 4.9 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 763 B/s rd, 56 MiB/s wr, 903 op/s; 12634/243837 objects degraded (5.181%); 30411/243837 objects misplaced (12.472%); 35 MiB/s, 559 objects/s recovering 2024-09-20T22:19:59.771 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:19:59.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:19:59 smithi060 ceph-mon[40290]: pgmap v916: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 4.9 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 763 B/s rd, 56 MiB/s wr, 903 op/s; 12634/243837 objects degraded (5.181%); 30411/243837 objects misplaced (12.472%); 35 MiB/s, 559 objects/s recovering 2024-09-20T22:19:59.788 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:19:59 smithi060 ceph-mon[29694]: pgmap v916: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 4.9 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 763 B/s rd, 56 MiB/s wr, 903 op/s; 12634/243837 objects degraded (5.181%); 30411/243837 objects misplaced (12.472%); 35 MiB/s, 559 objects/s recovering 2024-09-20T22:19:59.809 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:59.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:59.810 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:19:59.911 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:19:59.952 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:19:59.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:19:59.953 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:00.054 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:00.076 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:00.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:00.077 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:00.106 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 67 16 83787 83771 78.1299 53.875 0.00990818 0.0127937 2024-09-20T22:20:00.178 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:00.216 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:00.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:00.217 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:00.318 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:00.353 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:00.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:00.354 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:00.454 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:00.497 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:00.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:00.498 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:00.599 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:00.640 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:00.640 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:00.641 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:00.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:00 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:20:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:00 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:20:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:00 smithi067 ceph-mon[33511]: Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set; Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull; Degraded data redundancy: 12634/243837 objects degraded (5.181%), 4 pgs degraded 2024-09-20T22:20:00.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:00 smithi067 ceph-mon[33511]: [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2024-09-20T22:20:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:00 smithi067 ceph-mon[33511]: [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull 2024-09-20T22:20:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:00 smithi067 ceph-mon[33511]: pg 11.b is active+remapped+backfill_toofull, acting [3,0,1] 2024-09-20T22:20:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:00 smithi067 ceph-mon[33511]: [WRN] PG_DEGRADED: Degraded data redundancy: 12634/243837 objects degraded (5.181%), 4 pgs degraded 2024-09-20T22:20:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:00 smithi067 ceph-mon[33511]: pg 11.7 is active+recovering+degraded+remapped, acting [5,0,1] 2024-09-20T22:20:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:00 smithi067 ceph-mon[33511]: pg 11.c is active+recovering+undersized+degraded+remapped, acting [2,1] 2024-09-20T22:20:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:00 smithi067 ceph-mon[33511]: pg 11.d is active+recovering+undersized+degraded+remapped, acting [2,6] 2024-09-20T22:20:00.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:00 smithi067 ceph-mon[33511]: pg 11.e is active+recovering+undersized+degraded+remapped, acting [7,1] 2024-09-20T22:20:00.779 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:00.779 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:00.779 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:20:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:20:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[40290]: Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set; Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull; Degraded data redundancy: 12634/243837 objects degraded (5.181%), 4 pgs degraded 2024-09-20T22:20:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[40290]: [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2024-09-20T22:20:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[40290]: [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull 2024-09-20T22:20:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[40290]: pg 11.b is active+remapped+backfill_toofull, acting [3,0,1] 2024-09-20T22:20:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[40290]: [WRN] PG_DEGRADED: Degraded data redundancy: 12634/243837 objects degraded (5.181%), 4 pgs degraded 2024-09-20T22:20:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[40290]: pg 11.7 is active+recovering+degraded+remapped, acting [5,0,1] 2024-09-20T22:20:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[40290]: pg 11.c is active+recovering+undersized+degraded+remapped, acting [2,1] 2024-09-20T22:20:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[40290]: pg 11.d is active+recovering+undersized+degraded+remapped, acting [2,6] 2024-09-20T22:20:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[40290]: pg 11.e is active+recovering+undersized+degraded+remapped, acting [7,1] 2024-09-20T22:20:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:20:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:20:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[29694]: Health detail: HEALTH_WARN noscrub,nodeep-scrub flag(s) set; Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull; Degraded data redundancy: 12634/243837 objects degraded (5.181%), 4 pgs degraded 2024-09-20T22:20:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[29694]: [WRN] OSDMAP_FLAGS: noscrub,nodeep-scrub flag(s) set 2024-09-20T22:20:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[29694]: [WRN] PG_BACKFILL_FULL: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull 2024-09-20T22:20:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[29694]: pg 11.b is active+remapped+backfill_toofull, acting [3,0,1] 2024-09-20T22:20:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[29694]: [WRN] PG_DEGRADED: Degraded data redundancy: 12634/243837 objects degraded (5.181%), 4 pgs degraded 2024-09-20T22:20:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[29694]: pg 11.7 is active+recovering+degraded+remapped, acting [5,0,1] 2024-09-20T22:20:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[29694]: pg 11.c is active+recovering+undersized+degraded+remapped, acting [2,1] 2024-09-20T22:20:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[29694]: pg 11.d is active+recovering+undersized+degraded+remapped, acting [2,6] 2024-09-20T22:20:00.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:00 smithi060 ceph-mon[29694]: pg 11.e is active+recovering+undersized+degraded+remapped, acting [7,1] 2024-09-20T22:20:00.880 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:00.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:00.920 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:00.920 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:01.021 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:01.062 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:01.062 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:01.062 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:01.106 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 68 16 84606 84590 77.7337 51.1875 0.0172635 0.0128589 2024-09-20T22:20:01.163 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:01.190 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:01.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:01.191 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:01.292 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:01.325 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:01.326 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:01.326 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:01.427 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:01.471 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:01.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:01.472 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:01.573 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:01.608 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:01.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:01.609 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:01.710 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:01.747 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:01.750 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:01.751 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:01 smithi067 ceph-mon[33511]: pgmap v917: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 4.9 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 639 B/s rd, 38 MiB/s wr, 603 op/s; 12634/243837 objects degraded (5.181%); 30411/243837 objects misplaced (12.472%); 27 MiB/s, 436 objects/s recovering 2024-09-20T22:20:01.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:01 smithi067 ceph-mon[33511]: osdmap e488: 8 total, 7 up, 6 in 2024-09-20T22:20:01.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:01 smithi060 ceph-mon[29694]: pgmap v917: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 4.9 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 639 B/s rd, 38 MiB/s wr, 603 op/s; 12634/243837 objects degraded (5.181%); 30411/243837 objects misplaced (12.472%); 27 MiB/s, 436 objects/s recovering 2024-09-20T22:20:01.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:01 smithi060 ceph-mon[29694]: osdmap e488: 8 total, 7 up, 6 in 2024-09-20T22:20:01.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:01 smithi060 ceph-mon[40290]: pgmap v917: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 4.9 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 639 B/s rd, 38 MiB/s wr, 603 op/s; 12634/243837 objects degraded (5.181%); 30411/243837 objects misplaced (12.472%); 27 MiB/s, 436 objects/s recovering 2024-09-20T22:20:01.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:01 smithi060 ceph-mon[40290]: osdmap e488: 8 total, 7 up, 6 in 2024-09-20T22:20:01.851 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:01.879 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:01.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:01.880 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:01.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:02.021 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:02.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:02.022 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:02.106 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 69 16 85267 85251 77.2058 41.3125 0.0155621 0.0129462 2024-09-20T22:20:02.123 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:02.164 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:02.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:02.165 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:02.265 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:02.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:02.311 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:02.312 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:02.412 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:02.434 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:02.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:02.435 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:02.535 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:02.572 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:02.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:02.573 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:02.673 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:02.704 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:02.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:02.705 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:02.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:02 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 12634/243837 objects degraded (5.181%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:02.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:02 smithi067 ceph-mon[33511]: osdmap e489: 8 total, 7 up, 6 in 2024-09-20T22:20:02.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:02 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 12634/243837 objects degraded (5.181%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:02.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:02 smithi060 ceph-mon[29694]: osdmap e489: 8 total, 7 up, 6 in 2024-09-20T22:20:02.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:02 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 12634/243837 objects degraded (5.181%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:02.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:02 smithi060 ceph-mon[40290]: osdmap e489: 8 total, 7 up, 6 in 2024-09-20T22:20:02.805 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:02.829 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:02.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:02.830 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:02.931 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:02.975 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:02.975 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:02.976 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:03.076 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:03.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:03.105 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:03.105 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:03.106 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 70 16 85947 85931 76.71 42.5 0.00339316 0.0130265 2024-09-20T22:20:03.206 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:03.245 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:03.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:03.247 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:03.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:20:02 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:20:03.347 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:03.391 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:03.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:03.392 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:03.492 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:03.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:03.540 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:03.540 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:03.641 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:03.668 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:03.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:03.669 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:03.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:03 smithi067 ceph-mon[33511]: pgmap v920: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 5.1 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 1.2 KiB/s rd, 51 MiB/s wr, 810 op/s; 12124/252492 objects degraded (4.802%); 28648/252492 objects misplaced (11.346%); 41 MiB/s, 652 objects/s recovering 2024-09-20T22:20:03.770 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:03.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:03 smithi060 ceph-mon[29694]: pgmap v920: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 5.1 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 1.2 KiB/s rd, 51 MiB/s wr, 810 op/s; 12124/252492 objects degraded (4.802%); 28648/252492 objects misplaced (11.346%); 41 MiB/s, 652 objects/s recovering 2024-09-20T22:20:03.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:03 smithi060 ceph-mon[40290]: pgmap v920: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 5.1 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 1.2 KiB/s rd, 51 MiB/s wr, 810 op/s; 12124/252492 objects degraded (4.802%); 28648/252492 objects misplaced (11.346%); 41 MiB/s, 652 objects/s recovering 2024-09-20T22:20:03.810 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:03.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:03.811 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:03.912 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:03.956 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:03.957 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:03.957 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:04.058 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:04.089 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:04.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:04.090 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:04.107 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 71 16 86796 86780 76.3769 53.0625 0.00885455 0.0130834 2024-09-20T22:20:04.191 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:04.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:04.231 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:04.231 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:04.331 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:04.380 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:04.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:04.381 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:04.482 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:04.522 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:04.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:04.523 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:04.624 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:04.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:04.670 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:04.671 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:04.686 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:04 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:20:04.686 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:04 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:20:04.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:04 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:20:04.772 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:04.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:04.820 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:04.820 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:04.858 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:04.868 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:20:04.868 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:20:04.868 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:20:04.868 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:20:04.868 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:20:04.922 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:04.966 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:04.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:04.967 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:05.068 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:05.107 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:05.107 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:05.107 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:05.108 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 72 16 87655 87639 76.0617 53.6875 0.0204169 0.0131406 2024-09-20T22:20:05.208 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:05.249 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:05.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:05.250 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:05.351 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:05.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:05.394 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:05.394 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:05.495 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:05.499 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:20:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:20:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:20:05.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:05.545 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:05.545 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:05.646 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:05.689 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:05.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:05.690 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:05.721 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T22:20:05.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:05 smithi067 ceph-mon[33511]: pgmap v921: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 5.2 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 639 B/s rd, 51 MiB/s wr, 818 op/s; 12124/256197 objects degraded (4.732%); 28409/256197 objects misplaced (11.089%); 31 MiB/s, 501 objects/s recovering 2024-09-20T22:20:05.764 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:05 smithi060 ceph-mon[29694]: pgmap v921: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 5.2 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 639 B/s rd, 51 MiB/s wr, 818 op/s; 12124/256197 objects degraded (4.732%); 28409/256197 objects misplaced (11.089%); 31 MiB/s, 501 objects/s recovering 2024-09-20T22:20:05.764 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:05 smithi060 ceph-mon[40290]: pgmap v921: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 5.2 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 639 B/s rd, 51 MiB/s wr, 818 op/s; 12124/256197 objects degraded (4.732%); 28409/256197 objects misplaced (11.089%); 31 MiB/s, 501 objects/s recovering 2024-09-20T22:20:05.791 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:05.832 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:05.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:05.833 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:05.934 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:06.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:06.000 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:06.001 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:06.101 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:06.112 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 73 16 88560 88544 75.7945 56.5625 0.0234636 0.0131868 2024-09-20T22:20:06.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:06.161 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:06.162 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:06.263 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:06.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:20:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:20:06.228+0000 7fa5382c7640 -1 mgr get_metadata_python Requested missing service osd.4 2024-09-20T22:20:06.314 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:06.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:06.315 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:06.416 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:06.446 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:20:06.446 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:20:06.447 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:06.448 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:06.449 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:06.449 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:06.484 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:20:06.490 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:20:06.490 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:20:06.530 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:20:06.530 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:20:06.550 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:06.566 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:20:06.567 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:20:06.568 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-20T22:20:06.612 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:20:06.612 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:20:06.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:06.624 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:06.624 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:06.655 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:20:06.656 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:20:06.693 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:20:06.693 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:20:06.725 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:06.763 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:06.764 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:06.764 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:06.865 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:06.895 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:06.895 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:06.896 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:06.996 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:07.030 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:07.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:07.031 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:07.107 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 74 16 89166 89150 75.2821 37.875 0.00773554 0.0132212 2024-09-20T22:20:07.132 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:07.162 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:07.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:07.163 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:07.264 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:07.286 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:07.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:07.287 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:07.388 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:07.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:07.416 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:07.417 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:07.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:07 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:20:07.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:07 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:20:07.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:07 smithi067 ceph-mon[33511]: pgmap v922: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 5.3 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.2 KiB/s rd, 41 MiB/s wr, 665 op/s; 11959/259779 objects degraded (4.604%); 26632/259779 objects misplaced (10.252%); 39 MiB/s, 620 objects/s recovering 2024-09-20T22:20:07.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:07 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1485792243' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:20:07.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:07 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 12124/256197 objects degraded (4.732%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:07.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:07 smithi067 ceph-mon[33511]: osdmap e490: 8 total, 7 up, 6 in 2024-09-20T22:20:07.517 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:07.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:07 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:20:07.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:07 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:20:07.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:07 smithi060 ceph-mon[29694]: pgmap v922: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 5.3 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.2 KiB/s rd, 41 MiB/s wr, 665 op/s; 11959/259779 objects degraded (4.604%); 26632/259779 objects misplaced (10.252%); 39 MiB/s, 620 objects/s recovering 2024-09-20T22:20:07.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:07 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1485792243' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:20:07.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:07 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 12124/256197 objects degraded (4.732%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:07.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:07 smithi060 ceph-mon[29694]: osdmap e490: 8 total, 7 up, 6 in 2024-09-20T22:20:07.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:07 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:20:07.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:07 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:20:07.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:07 smithi060 ceph-mon[40290]: pgmap v922: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 5.3 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.2 KiB/s rd, 41 MiB/s wr, 665 op/s; 11959/259779 objects degraded (4.604%); 26632/259779 objects misplaced (10.252%); 39 MiB/s, 620 objects/s recovering 2024-09-20T22:20:07.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:07 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1485792243' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:20:07.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:07 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 12124/256197 objects degraded (4.732%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:07.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:07 smithi060 ceph-mon[40290]: osdmap e490: 8 total, 7 up, 6 in 2024-09-20T22:20:07.547 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:07.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:07.548 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:07.649 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:07.679 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:07.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:07.680 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:07.781 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:07.812 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:07.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:07.813 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:07.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:07.958 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:07.959 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:07.959 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:08.060 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:08.102 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:08.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:08.103 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:08.107 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 75 16 89403 89387 74.4759 14.8125 0.0243368 0.0134211 2024-09-20T22:20:08.204 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:08.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:08.242 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:08.242 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:08.343 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:08.394 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:08.395 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:08.395 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:08.496 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:08.536 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:08.537 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:08.537 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:08.638 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:08.685 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:08.686 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:08.686 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:08.787 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:08.821 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:08.822 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:08.823 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:08.923 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:08.947 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:08.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:08.948 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:09.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:08 smithi067 ceph-mon[33511]: osdmap e491: 8 total, 7 up, 6 in 2024-09-20T22:20:09.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:08 smithi067 ceph-mon[33511]: pgmap v925: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 5.4 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 763 B/s rd, 59 MiB/s wr, 938 op/s; 11461/267546 objects degraded (4.284%); 26042/267546 objects misplaced (9.734%); 39 MiB/s, 625 objects/s recovering 2024-09-20T22:20:09.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:08 smithi060 ceph-mon[29694]: osdmap e491: 8 total, 7 up, 6 in 2024-09-20T22:20:09.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:08 smithi060 ceph-mon[29694]: pgmap v925: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 5.4 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 763 B/s rd, 59 MiB/s wr, 938 op/s; 11461/267546 objects degraded (4.284%); 26042/267546 objects misplaced (9.734%); 39 MiB/s, 625 objects/s recovering 2024-09-20T22:20:09.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:08 smithi060 ceph-mon[40290]: osdmap e491: 8 total, 7 up, 6 in 2024-09-20T22:20:09.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:08 smithi060 ceph-mon[40290]: pgmap v925: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 5.4 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 763 B/s rd, 59 MiB/s wr, 938 op/s; 11461/267546 objects degraded (4.284%); 26042/267546 objects misplaced (9.734%); 39 MiB/s, 625 objects/s recovering 2024-09-20T22:20:09.048 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:09.086 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:09.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:09.087 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:09.107 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 76 16 90258 90242 74.199 53.4375 0.0108091 0.0134721 2024-09-20T22:20:09.188 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:09.223 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:09.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:09.224 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:09.325 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:09.347 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:09.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:09.348 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:09.449 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:09.489 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:09.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:09.490 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:09.591 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:09.634 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:09.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:09.635 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:09.736 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:09.748 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:20:09.749 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:20:09.768 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:09.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:09.770 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:09.870 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:09.917 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:09.917 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:09.917 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:10.018 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:10.064 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:10.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:10.065 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:10.107 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 77 16 91110 91094 73.9269 53.25 0.0154155 0.0135207 2024-09-20T22:20:10.166 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:10.203 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:10.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:10.204 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:10.242 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:20:10.242 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:20:10.258 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:10.304 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:10.363 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:10.363 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:10.363 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:10.464 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:10.500 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:10.501 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:10.501 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:10.602 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:10.634 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:10.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:10.635 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:10.694 INFO:teuthology.orchestra.run.smithi067.stdout:11.d 2024-09-20T22:20:10.694 INFO:teuthology.orchestra.run.smithi067.stdout:11.e 2024-09-20T22:20:10.694 INFO:teuthology.orchestra.run.smithi067.stdout:11.c 2024-09-20T22:20:10.694 INFO:teuthology.orchestra.run.smithi067.stdout:11.7 2024-09-20T22:20:10.736 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:10.782 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:10.783 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:10.783 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:10.884 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:10.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:10.915 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:10.916 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:11.016 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:11.063 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:11.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:11.064 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:11.107 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 78 16 92016 92000 73.7051 56.625 0.0340539 0.0135619 2024-09-20T22:20:11.164 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:11.203 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:11.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:11.204 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:11.304 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:11.356 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:11.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:11.357 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:11.458 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:11.498 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:11.498 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:11.498 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:11.513 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:11 smithi067 ceph-mon[33511]: pgmap v926: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 5.4 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 639 B/s rd, 39 MiB/s wr, 627 op/s; 11461/267546 objects degraded (4.284%); 26042/267546 objects misplaced (9.734%); 31 MiB/s, 488 objects/s recovering 2024-09-20T22:20:11.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T22:20:11.600 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:11.645 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:11.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:11.646 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:11.747 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:11.750 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:20:11.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:11 smithi060 ceph-mon[29694]: pgmap v926: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 5.4 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 639 B/s rd, 39 MiB/s wr, 627 op/s; 11461/267546 objects degraded (4.284%); 26042/267546 objects misplaced (9.734%); 31 MiB/s, 488 objects/s recovering 2024-09-20T22:20:11.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:11 smithi060 ceph-mon[40290]: pgmap v926: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 169 active+clean; 5.4 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 639 B/s rd, 39 MiB/s wr, 627 op/s; 11461/267546 objects degraded (4.284%); 26042/267546 objects misplaced (9.734%); 31 MiB/s, 488 objects/s recovering 2024-09-20T22:20:11.789 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:11.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:11.792 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:11.893 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:11.920 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:11.921 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:11.921 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:12.022 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:12.023 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:12.093 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:12.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:12.094 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:12.108 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 79 16 92867 92851 73.4453 53.1875 0.00402378 0.0136092 2024-09-20T22:20:12.195 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:12.228 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:12.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:12.229 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:12.330 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:12.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:12.367 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:12.367 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:12.433 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:20:12.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:12 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/682007114' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:20:12.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:12 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:20:12.436 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:12 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 11461/267546 objects degraded (4.284%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:12.436 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:12 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/682007114' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:20:12.436 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:12 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:20:12.436 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:12 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 11461/267546 objects degraded (4.284%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:12.468 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:12.499 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:12.500 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:12.501 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:12.601 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:12.635 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:12.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:12.636 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:12.736 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:12.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:12 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/682007114' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:20:12.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:12 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:20:12.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:12 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 11461/267546 objects degraded (4.284%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:12.771 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:12.772 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:12.772 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:12.873 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:12.904 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:12.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:12.905 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:13.006 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:13.051 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:13.052 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:13.052 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:13.108 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:2024-09-20T22:20:13.107745+0000 min lat: 0.00118665 max lat: 2.25818 avg lat: 0.0136522 2024-09-20T22:20:13.108 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-20T22:20:13.108 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 80 16 93733 93717 73.2037 54.125 0.0215307 0.0136522 2024-09-20T22:20:13.153 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:13.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:13.191 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:13.191 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:13.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:20:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:20:13.292 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:13.335 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:13.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:13.336 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:13.436 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:13.458 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:13.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:13.459 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:13.560 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:13.604 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:13.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:13.605 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:13.706 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:13.727 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:13.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:13.728 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:13.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:13 smithi067 ceph-mon[33511]: pgmap v927: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 170 active+clean; 5.6 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 1.2 KiB/s rd, 49 MiB/s wr, 777 op/s; 10580/274839 objects degraded (3.850%); 24920/274839 objects misplaced (9.067%); 39 MiB/s, 617 objects/s recovering 2024-09-20T22:20:13.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:13 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:20:13.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:13 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:20:13.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:13 smithi067 ceph-mon[33511]: osdmap e492: 8 total, 7 up, 6 in 2024-09-20T22:20:13.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:13 smithi060 ceph-mon[29694]: pgmap v927: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 170 active+clean; 5.6 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 1.2 KiB/s rd, 49 MiB/s wr, 777 op/s; 10580/274839 objects degraded (3.850%); 24920/274839 objects misplaced (9.067%); 39 MiB/s, 617 objects/s recovering 2024-09-20T22:20:13.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:13 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:20:13.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:13 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:20:13.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:13 smithi060 ceph-mon[29694]: osdmap e492: 8 total, 7 up, 6 in 2024-09-20T22:20:13.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:13 smithi060 ceph-mon[40290]: pgmap v927: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 170 active+clean; 5.6 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 1.2 KiB/s rd, 49 MiB/s wr, 777 op/s; 10580/274839 objects degraded (3.850%); 24920/274839 objects misplaced (9.067%); 39 MiB/s, 617 objects/s recovering 2024-09-20T22:20:13.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:13 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:20:13.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:13 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:20:13.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:13 smithi060 ceph-mon[40290]: osdmap e492: 8 total, 7 up, 6 in 2024-09-20T22:20:13.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:13.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:13.878 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:13.879 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:13.979 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:14.001 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:14.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:14.002 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:14.102 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:14.108 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 81 16 94570 94554 72.9457 52.3125 0.00452403 0.0137023 2024-09-20T22:20:14.138 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:14.139 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:14.139 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:14.240 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:14.262 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:14.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:14.263 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:14.364 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:14.393 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:14.394 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:14.394 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:14.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:14.528 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:14.528 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:14.529 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:14.629 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:14.665 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:14.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:14.667 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:14.694 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:14 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:20:14.767 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:14.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:14 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:20:14.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:14 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:20:14.809 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:14.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:14.811 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:14.911 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:14.947 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:14.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:14.948 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:15.049 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:15.088 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:15.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:15.089 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:15.108 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 82 16 95446 95430 72.7238 54.75 0.00484103 0.0137448 2024-09-20T22:20:15.189 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:15.233 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:15.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:15.234 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:15.334 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:20:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:20:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:20:15.334 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:15.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:15.374 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:15.374 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:15.411 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:20:15.411 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:20:15.475 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:15.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:15.514 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:15.514 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:15.615 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:15.653 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:15.654 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:15.654 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:15.743 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:15 smithi060 ceph-mon[29694]: pgmap v929: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 170 active+clean; 5.7 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 664 B/s rd, 49 MiB/s wr, 791 op/s; 10580/278787 objects degraded (3.795%); 24591/278787 objects misplaced (8.821%); 28 MiB/s, 450 objects/s recovering 2024-09-20T22:20:15.743 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:15 smithi060 ceph-mon[40290]: pgmap v929: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 170 active+clean; 5.7 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 664 B/s rd, 49 MiB/s wr, 791 op/s; 10580/278787 objects degraded (3.795%); 24591/278787 objects misplaced (8.821%); 28 MiB/s, 450 objects/s recovering 2024-09-20T22:20:15.756 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:15 smithi067 ceph-mon[33511]: pgmap v929: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 170 active+clean; 5.7 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 664 B/s rd, 49 MiB/s wr, 791 op/s; 10580/278787 objects degraded (3.795%); 24591/278787 objects misplaced (8.821%); 28 MiB/s, 450 objects/s recovering 2024-09-20T22:20:15.795 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:15.795 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:15.796 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:15.896 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:15.933 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:15.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:15.934 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:16.034 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:16.067 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:16.068 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:16.068 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:16.108 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 83 16 96357 96341 72.5336 56.9375 0.0537791 0.0137787 2024-09-20T22:20:16.170 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:16.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:16.226 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:16.226 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:16.327 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:16.361 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:16.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:16.362 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:16.463 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:16.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:16.485 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:16.485 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:16.586 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:16.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:16.622 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:16.623 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:16.723 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:16.759 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:16.760 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:16.760 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:16 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:20:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:16 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:20:16.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:16 smithi067 ceph-mon[33511]: osdmap e493: 8 total, 7 up, 6 in 2024-09-20T22:20:16.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:16 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:20:16.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:16 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:20:16.765 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:16 smithi060 ceph-mon[29694]: osdmap e493: 8 total, 7 up, 6 in 2024-09-20T22:20:16.765 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:16 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:20:16.766 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:16 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:20:16.766 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:16 smithi060 ceph-mon[40290]: osdmap e493: 8 total, 7 up, 6 in 2024-09-20T22:20:16.783 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:16.834 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T22:20:16.861 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:16.883 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:16.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:16.884 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:16.985 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:17.036 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:17.037 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:17.037 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:17.108 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 84 16 97141 97125 72.2534 49 0.0280909 0.0138344 2024-09-20T22:20:17.138 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:17.197 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:17.198 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:17.198 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:17.299 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:17.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:17.347 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:17.347 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:17.448 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:17.481 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:17.482 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:17.482 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:17.583 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:17.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:17.609 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:17.609 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:17.710 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:17.715 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -n osd.4 --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --op export --pgid 11.d --file /var/log/ceph/exp.11.d.4 2024-09-20T22:20:17.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:17.748 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:17.748 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:17.849 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:17.891 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:17.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:17.892 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:17.993 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:18.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:17 smithi067 ceph-mon[33511]: pgmap v931: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 170 active+clean; 5.7 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 1.2 KiB/s rd, 40 MiB/s wr, 638 op/s; 10425/282846 objects degraded (3.686%); 24103/282846 objects misplaced (8.522%); 23 MiB/s, 369 objects/s recovering 2024-09-20T22:20:18.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:17 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 10425/282846 objects degraded (3.686%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:18.018 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/osd.4/config 2024-09-20T22:20:18.026 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:18.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:18.027 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:18.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:17 smithi060 ceph-mon[29694]: pgmap v931: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 170 active+clean; 5.7 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 1.2 KiB/s rd, 40 MiB/s wr, 638 op/s; 10425/282846 objects degraded (3.686%); 24103/282846 objects misplaced (8.522%); 23 MiB/s, 369 objects/s recovering 2024-09-20T22:20:18.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:17 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 10425/282846 objects degraded (3.686%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:18.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:17 smithi060 ceph-mon[40290]: pgmap v931: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 170 active+clean; 5.7 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 1.2 KiB/s rd, 40 MiB/s wr, 638 op/s; 10425/282846 objects degraded (3.686%); 24103/282846 objects misplaced (8.522%); 23 MiB/s, 369 objects/s recovering 2024-09-20T22:20:18.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:17 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 10425/282846 objects degraded (3.686%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:18.108 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 85 16 97882 97866 71.9482 46.3125 0.00383281 0.0138567 2024-09-20T22:20:18.128 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:18.152 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:18.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:18.153 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:18.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:20:18.253 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:18.288 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:18.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:18.289 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:18.389 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:18.460 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:18.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:18.461 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:18.562 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:18.591 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:18.592 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:18.592 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:18.693 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:18.738 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:18.738 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:18.738 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:18.808 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:18 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2305327451' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:20:18.808 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:18 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2305327451' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:20:18.840 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:18.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:18.863 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:18.863 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:18.964 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:18.999 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:18.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:19.000 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:19.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:18 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2305327451' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:20:19.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:19.109 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 86 16 98632 98616 71.6563 46.875 0.00985468 0.013948 2024-09-20T22:20:19.140 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:19.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:19.141 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:19.242 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:19.286 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:19.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:19.287 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:19.388 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:19.425 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:19.425 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:19.425 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:19.526 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:19.563 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:19.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:19.564 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:19.611 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:20:19.611 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:20:19.612 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:20:19.612 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:20:19.612 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:20:19.612 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:20:19.612 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:20:19.612 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:20:19.613 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:20:19.613 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:20:19.613 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:20:19.613 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-20T22:20:19.613 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:20:19.613 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:20:19.613 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:20:19.665 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:19.698 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:19.699 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:19.699 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:19.800 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:19.835 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:19.835 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:19.836 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:19.937 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:19.985 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:19.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:19.986 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:20.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:19 smithi067 ceph-mon[33511]: pgmap v932: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 170 active+clean; 5.9 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 1.2 KiB/s rd, 63 MiB/s wr, 1.01k op/s; 9938/291744 objects degraded (3.406%); 23815/291744 objects misplaced (8.163%); 31 MiB/s, 498 objects/s recovering 2024-09-20T22:20:20.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:19 smithi060 ceph-mon[29694]: pgmap v932: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 170 active+clean; 5.9 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 1.2 KiB/s rd, 63 MiB/s wr, 1.01k op/s; 9938/291744 objects degraded (3.406%); 23815/291744 objects misplaced (8.163%); 31 MiB/s, 498 objects/s recovering 2024-09-20T22:20:20.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:19 smithi060 ceph-mon[40290]: pgmap v932: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 170 active+clean; 5.9 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 1.2 KiB/s rd, 63 MiB/s wr, 1.01k op/s; 9938/291744 objects degraded (3.406%); 23815/291744 objects misplaced (8.163%); 31 MiB/s, 498 objects/s recovering 2024-09-20T22:20:20.086 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:20.108 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 87 16 99451 99435 71.421 51.1875 0.0484123 0.0139876 2024-09-20T22:20:20.109 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:20.110 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:20.110 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:20.211 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:20.243 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:20.244 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:20.244 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:20.345 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:20.369 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:20.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:20.370 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:20.471 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:20.508 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:20.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:20.509 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:20.610 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:20.644 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:20.645 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:20.646 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:20.746 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:20.787 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:20.788 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:20.788 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:20.889 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:20.930 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:20.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:20.931 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:21.032 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:21.072 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:21.072 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:21.073 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:21.109 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 88 16 100366 100350 71.2592 57.1875 0.00797689 0.0140277 2024-09-20T22:20:21.173 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:21.204 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:21.205 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:21.205 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:21.306 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:21.350 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:21.351 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:21.351 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:21.451 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:21.483 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:21.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:21.484 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:21.535 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:20:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:20:21.248+0000 7fa5382c7640 -1 mgr get_metadata_python Requested missing service osd.4 2024-09-20T22:20:21.585 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:21.607 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:21.608 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:21.608 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:21.630 INFO:teuthology.orchestra.run.smithi067.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-20T22:20:21.708 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:21.732 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:21.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:21.733 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:21.813 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:21 smithi067 ceph-mon[33511]: pgmap v933: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 170 active+clean; 5.9 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 639 B/s rd, 44 MiB/s wr, 704 op/s; 9938/291744 objects degraded (3.406%); 23815/291744 objects misplaced (8.163%); 18 MiB/s, 290 objects/s recovering 2024-09-20T22:20:21.813 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:20:21.834 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:21.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:21.880 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:21.880 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:21.981 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:22.031 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:22.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:22.032 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:22.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:21 smithi060 ceph-mon[29694]: pgmap v933: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 170 active+clean; 5.9 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 639 B/s rd, 44 MiB/s wr, 704 op/s; 9938/291744 objects degraded (3.406%); 23815/291744 objects misplaced (8.163%); 18 MiB/s, 290 objects/s recovering 2024-09-20T22:20:22.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:20:22.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:21 smithi060 ceph-mon[40290]: pgmap v933: 178 pgs: 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 170 active+clean; 5.9 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 639 B/s rd, 44 MiB/s wr, 704 op/s; 9938/291744 objects degraded (3.406%); 23815/291744 objects misplaced (8.163%); 18 MiB/s, 290 objects/s recovering 2024-09-20T22:20:22.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:20:22.109 INFO:tasks.radosbench.radosbench.0.smithi060.stdout: 89 16 101035 101019 70.9283 41.8125 0.0107211 0.0140924 2024-09-20T22:20:22.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:22.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:22.255 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:22.256 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:22.356 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:22.400 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:22.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:22.401 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:22.502 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:22.540 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:22.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:22.541 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:22.641 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:22.681 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:22.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:22.683 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:22.729 INFO:teuthology.orchestra.run.smithi067.stderr:Exporting 11.d info 11.d( v 471'3847 (0'0,471'3847] local-lis/les=456/457 n=3847 ec=456/456 lis/c=473/456 les/c/f=474/457/0 sis=480) 2024-09-20T22:20:22.776 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0008cae:::benchmark_data_smithi060_196073_object45401:head# 2024-09-20T22:20:22.776 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0028b4f:::benchmark_data_smithi060_196073_object6636:head# 2024-09-20T22:20:22.777 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b002da64:::benchmark_data_smithi060_196073_object33538:head# 2024-09-20T22:20:22.777 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b005916d:::benchmark_data_smithi060_196073_object21370:head# 2024-09-20T22:20:22.778 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0061903:::benchmark_data_smithi060_196073_object7962:head# 2024-09-20T22:20:22.782 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b00654a9:::benchmark_data_smithi060_196073_object19172:head# 2024-09-20T22:20:22.783 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:22.784 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b00816bd:::benchmark_data_smithi060_196073_object20728:head# 2024-09-20T22:20:22.785 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0086d3a:::benchmark_data_smithi060_196073_object27194:head# 2024-09-20T22:20:22.785 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b008e665:::benchmark_data_smithi060_196073_object51056:head# 2024-09-20T22:20:22.788 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b00c49ab:::benchmark_data_smithi060_196073_object34666:head# 2024-09-20T22:20:22.792 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b00c5fff:::benchmark_data_smithi060_196073_object42440:head# 2024-09-20T22:20:22.795 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b00cc4a9:::benchmark_data_smithi060_196073_object54410:head# 2024-09-20T22:20:22.796 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b00eb298:::benchmark_data_smithi060_196073_object4306:head# 2024-09-20T22:20:22.797 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b00f55a7:::benchmark_data_smithi060_196073_object58567:head# 2024-09-20T22:20:22.798 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b00f9c3c:::benchmark_data_smithi060_196073_object25802:head# 2024-09-20T22:20:22.801 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0113c5b:::benchmark_data_smithi060_196073_object47631:head# 2024-09-20T22:20:22.802 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b01334da:::benchmark_data_smithi060_196073_object42837:head# 2024-09-20T22:20:22.803 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b01440a9:::benchmark_data_smithi060_196073_object21105:head# 2024-09-20T22:20:22.804 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b014abf5:::benchmark_data_smithi060_196073_object5104:head# 2024-09-20T22:20:22.808 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b014c92f:::benchmark_data_smithi060_196073_object37410:head# 2024-09-20T22:20:22.809 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0160b3d:::benchmark_data_smithi060_196073_object57293:head# 2024-09-20T22:20:22.811 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b01634c9:::benchmark_data_smithi060_196073_object36953:head# 2024-09-20T22:20:22.812 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:22.812 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b017b5d0:::benchmark_data_smithi060_196073_object9895:head# 2024-09-20T22:20:22.812 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:22.812 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:22.814 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b01ad0c5:::benchmark_data_smithi060_196073_object52924:head# 2024-09-20T22:20:22.814 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:20:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:20:22.814 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:22 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 9938/291744 objects degraded (3.406%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:22.815 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b01bff2e:::benchmark_data_smithi060_196073_object25016:head# 2024-09-20T22:20:22.816 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b01c8006:::benchmark_data_smithi060_196073_object11799:head# 2024-09-20T22:20:22.818 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b01e1bc9:::benchmark_data_smithi060_196073_object8450:head# 2024-09-20T22:20:22.820 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b01e6ef2:::benchmark_data_smithi060_196073_object44124:head# 2024-09-20T22:20:22.820 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b01ed1f9:::benchmark_data_smithi060_196073_object60023:head# 2024-09-20T22:20:22.821 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b01f4d04:::benchmark_data_smithi060_196073_object55077:head# 2024-09-20T22:20:22.822 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0207395:::benchmark_data_smithi060_196073_object19792:head# 2024-09-20T22:20:22.826 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b02104b2:::benchmark_data_smithi060_196073_object15905:head# 2024-09-20T22:20:22.829 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0227ac4:::benchmark_data_smithi060_196073_object25785:head# 2024-09-20T22:20:22.834 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0227e35:::benchmark_data_smithi060_196073_object16151:head# 2024-09-20T22:20:22.836 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0231296:::benchmark_data_smithi060_196073_object20651:head# 2024-09-20T22:20:22.837 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0236a02:::benchmark_data_smithi060_196073_object10745:head# 2024-09-20T22:20:22.838 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0261197:::benchmark_data_smithi060_196073_object18392:head# 2024-09-20T22:20:22.840 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b026fe91:::benchmark_data_smithi060_196073_object57481:head# 2024-09-20T22:20:22.840 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b02720a4:::benchmark_data_smithi060_196073_object17823:head# 2024-09-20T22:20:22.841 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0280b4f:::benchmark_data_smithi060_196073_object50341:head# 2024-09-20T22:20:22.844 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b02bfe22:::benchmark_data_smithi060_196073_object43504:head# 2024-09-20T22:20:22.845 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b02c78d9:::benchmark_data_smithi060_196073_object1106:head# 2024-09-20T22:20:22.846 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b02cc502:::benchmark_data_smithi060_196073_object56359:head# 2024-09-20T22:20:22.847 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b02ce76d:::benchmark_data_smithi060_196073_object22902:head# 2024-09-20T22:20:22.848 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b02db9fa:::benchmark_data_smithi060_196073_object45387:head# 2024-09-20T22:20:22.848 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b02e3280:::benchmark_data_smithi060_196073_object41181:head# 2024-09-20T22:20:22.850 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b02e7393:::benchmark_data_smithi060_196073_object52039:head# 2024-09-20T22:20:22.850 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b032fe6c:::benchmark_data_smithi060_196073_object17679:head# 2024-09-20T22:20:22.851 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b033e32f:::benchmark_data_smithi060_196073_object33004:head# 2024-09-20T22:20:22.851 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b034e87c:::benchmark_data_smithi060_196073_object30708:head# 2024-09-20T22:20:22.853 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b035d8d5:::benchmark_data_smithi060_196073_object25494:head# 2024-09-20T22:20:22.853 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b036a672:::benchmark_data_smithi060_196073_object32279:head# 2024-09-20T22:20:22.856 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b036bb2d:::benchmark_data_smithi060_196073_object28005:head# 2024-09-20T22:20:22.856 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b037a3e8:::benchmark_data_smithi060_196073_object26019:head# 2024-09-20T22:20:22.860 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b037dfb6:::benchmark_data_smithi060_196073_object24012:head# 2024-09-20T22:20:22.860 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b039355a:::benchmark_data_smithi060_196073_object5511:head# 2024-09-20T22:20:22.860 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b03a876d:::benchmark_data_smithi060_196073_object8415:head# 2024-09-20T22:20:22.860 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b03aa41c:::benchmark_data_smithi060_196073_object7789:head# 2024-09-20T22:20:22.860 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b03f240c:::benchmark_data_smithi060_196073_object26275:head# 2024-09-20T22:20:22.861 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b03f7ce7:::benchmark_data_smithi060_196073_object57961:head# 2024-09-20T22:20:22.861 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b03f8369:::benchmark_data_smithi060_196073_object61765:head# 2024-09-20T22:20:22.862 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b03ff313:::benchmark_data_smithi060_196073_object8844:head# 2024-09-20T22:20:22.862 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b040a6c3:::benchmark_data_smithi060_196073_object55588:head# 2024-09-20T22:20:22.863 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b041f568:::benchmark_data_smithi060_196073_object41161:head# 2024-09-20T22:20:22.864 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b04230cb:::benchmark_data_smithi060_196073_object54535:head# 2024-09-20T22:20:22.865 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0483c9f:::benchmark_data_smithi060_196073_object37011:head# 2024-09-20T22:20:22.865 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0487370:::benchmark_data_smithi060_196073_object61681:head# 2024-09-20T22:20:22.866 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b048d472:::benchmark_data_smithi060_196073_object17457:head# 2024-09-20T22:20:22.867 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b04adef1:::benchmark_data_smithi060_196073_object718:head# 2024-09-20T22:20:22.867 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b04cdc31:::benchmark_data_smithi060_196073_object35034:head# 2024-09-20T22:20:22.868 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b04ef815:::benchmark_data_smithi060_196073_object43278:head# 2024-09-20T22:20:22.869 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b05067cc:::benchmark_data_smithi060_196073_object8026:head# 2024-09-20T22:20:22.870 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b050a324:::benchmark_data_smithi060_196073_object20720:head# 2024-09-20T22:20:22.870 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b051f73e:::benchmark_data_smithi060_196073_object56637:head# 2024-09-20T22:20:22.871 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b051f98c:::benchmark_data_smithi060_196073_object48887:head# 2024-09-20T22:20:22.871 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b05275ec:::benchmark_data_smithi060_196073_object10596:head# 2024-09-20T22:20:22.872 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0530ce5:::benchmark_data_smithi060_196073_object24009:head# 2024-09-20T22:20:22.872 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b05630fc:::benchmark_data_smithi060_196073_object52874:head# 2024-09-20T22:20:22.873 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b05631e3:::benchmark_data_smithi060_196073_object51060:head# 2024-09-20T22:20:22.873 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0567c80:::benchmark_data_smithi060_196073_object265:head# 2024-09-20T22:20:22.874 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b056d98c:::benchmark_data_smithi060_196073_object29240:head# 2024-09-20T22:20:22.875 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b05797f3:::benchmark_data_smithi060_196073_object25631:head# 2024-09-20T22:20:22.875 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b05798f3:::benchmark_data_smithi060_196073_object40693:head# 2024-09-20T22:20:22.876 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b057c37c:::benchmark_data_smithi060_196073_object18575:head# 2024-09-20T22:20:22.877 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b059d9ce:::benchmark_data_smithi060_196073_object56032:head# 2024-09-20T22:20:22.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b05bbc3f:::benchmark_data_smithi060_196073_object9593:head# 2024-09-20T22:20:22.879 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b05d107a:::benchmark_data_smithi060_196073_object23845:head# 2024-09-20T22:20:22.879 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b05e6ec1:::benchmark_data_smithi060_196073_object37482:head# 2024-09-20T22:20:22.880 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b05ffd6c:::benchmark_data_smithi060_196073_object4441:head# 2024-09-20T22:20:22.881 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b060296f:::benchmark_data_smithi060_196073_object27032:head# 2024-09-20T22:20:22.882 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0607667:::benchmark_data_smithi060_196073_object20271:head# 2024-09-20T22:20:22.883 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0630582:::benchmark_data_smithi060_196073_object8852:head# 2024-09-20T22:20:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0639796:::benchmark_data_smithi060_196073_object6640:head# 2024-09-20T22:20:22.886 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b063bd40:::benchmark_data_smithi060_196073_object30511:head# 2024-09-20T22:20:22.887 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b065c65f:::benchmark_data_smithi060_196073_object22874:head# 2024-09-20T22:20:22.888 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b06669a4:::benchmark_data_smithi060_196073_object20344:head# 2024-09-20T22:20:22.888 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0674ff0:::benchmark_data_smithi060_196073_object52340:head# 2024-09-20T22:20:22.889 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b06a8253:::benchmark_data_smithi060_196073_object10026:head# 2024-09-20T22:20:22.892 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b06c5b03:::benchmark_data_smithi060_196073_object40935:head# 2024-09-20T22:20:22.893 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b06e9f6d:::benchmark_data_smithi060_196073_object51923:head# 2024-09-20T22:20:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b06f798a:::benchmark_data_smithi060_196073_object21005:head# 2024-09-20T22:20:22.894 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b06fc440:::benchmark_data_smithi060_196073_object6742:head# 2024-09-20T22:20:22.895 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b07035b5:::benchmark_data_smithi060_196073_object23935:head# 2024-09-20T22:20:22.895 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b07048cc:::benchmark_data_smithi060_196073_object54438:head# 2024-09-20T22:20:22.895 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b070566c:::benchmark_data_smithi060_196073_object47180:head# 2024-09-20T22:20:22.896 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b070db02:::benchmark_data_smithi060_196073_object3112:head# 2024-09-20T22:20:22.896 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0713c0b:::benchmark_data_smithi060_196073_object60838:head# 2024-09-20T22:20:22.897 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0732e07:::benchmark_data_smithi060_196073_object17687:head# 2024-09-20T22:20:22.897 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b07547db:::benchmark_data_smithi060_196073_object26530:head# 2024-09-20T22:20:22.898 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b07552d6:::benchmark_data_smithi060_196073_object15277:head# 2024-09-20T22:20:22.899 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b07594b8:::benchmark_data_smithi060_196073_object9769:head# 2024-09-20T22:20:22.900 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b075ed06:::benchmark_data_smithi060_196073_object5291:head# 2024-09-20T22:20:22.900 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0764457:::benchmark_data_smithi060_196073_object40741:head# 2024-09-20T22:20:22.901 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0767013:::benchmark_data_smithi060_196073_object3349:head# 2024-09-20T22:20:22.902 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0767491:::benchmark_data_smithi060_196073_object25145:head# 2024-09-20T22:20:22.903 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b076824a:::benchmark_data_smithi060_196073_object52981:head# 2024-09-20T22:20:22.906 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b076fde9:::benchmark_data_smithi060_196073_object27628:head# 2024-09-20T22:20:22.906 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0778e39:::benchmark_data_smithi060_196073_object52417:head# 2024-09-20T22:20:22.907 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b07905eb:::benchmark_data_smithi060_196073_object3959:head# 2024-09-20T22:20:22.908 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b07a2f96:::benchmark_data_smithi060_196073_object25281:head# 2024-09-20T22:20:22.909 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b07b4c92:::benchmark_data_smithi060_196073_object6088:head# 2024-09-20T22:20:22.909 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b07bb6a3:::benchmark_data_smithi060_196073_object32241:head# 2024-09-20T22:20:22.910 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b07c1c47:::benchmark_data_smithi060_196073_object3238:head# 2024-09-20T22:20:22.911 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b07ffab3:::benchmark_data_smithi060_196073_object51935:head# 2024-09-20T22:20:22.912 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0806d35:::benchmark_data_smithi060_196073_object15559:head# 2024-09-20T22:20:22.913 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0810acd:::benchmark_data_smithi060_196073_object57450:head# 2024-09-20T22:20:22.913 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:22.914 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0823837:::benchmark_data_smithi060_196073_object38736:head# 2024-09-20T22:20:22.914 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0859150:::benchmark_data_smithi060_196073_object9875:head# 2024-09-20T22:20:22.916 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0863723:::benchmark_data_smithi060_196073_object36472:head# 2024-09-20T22:20:22.916 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b086ca37:::benchmark_data_smithi060_196073_object56968:head# 2024-09-20T22:20:22.918 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b08801ea:::benchmark_data_smithi060_196073_object17681:head# 2024-09-20T22:20:22.919 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b088d617:::benchmark_data_smithi060_196073_object49197:head# 2024-09-20T22:20:22.920 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0891889:::benchmark_data_smithi060_196073_object9477:head# 2024-09-20T22:20:22.921 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0892563:::benchmark_data_smithi060_196073_object41452:head# 2024-09-20T22:20:22.923 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b089c3a7:::benchmark_data_smithi060_196073_object27371:head# 2024-09-20T22:20:22.923 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b089fa3e:::benchmark_data_smithi060_196073_object41682:head# 2024-09-20T22:20:22.925 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b08a04a6:::benchmark_data_smithi060_196073_object6757:head# 2024-09-20T22:20:22.925 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b08ae27b:::benchmark_data_smithi060_196073_object10973:head# 2024-09-20T22:20:22.926 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b08cc219:::benchmark_data_smithi060_196073_object28287:head# 2024-09-20T22:20:22.926 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b08ce801:::benchmark_data_smithi060_196073_object26198:head# 2024-09-20T22:20:22.927 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b08e9951:::benchmark_data_smithi060_196073_object47031:head# 2024-09-20T22:20:22.929 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b08eb7f3:::benchmark_data_smithi060_196073_object5377:head# 2024-09-20T22:20:22.929 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b09017db:::benchmark_data_smithi060_196073_object21415:head# 2024-09-20T22:20:22.931 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b09110a8:::benchmark_data_smithi060_196073_object15986:head# 2024-09-20T22:20:22.931 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0923b15:::benchmark_data_smithi060_196073_object58858:head# 2024-09-20T22:20:22.932 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0939f3d:::benchmark_data_smithi060_196073_object48519:head# 2024-09-20T22:20:22.932 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b09456af:::benchmark_data_smithi060_196073_object29234:head# 2024-09-20T22:20:22.932 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0968366:::benchmark_data_smithi060_196073_object35238:head# 2024-09-20T22:20:22.933 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0971e17:::benchmark_data_smithi060_196073_object47809:head# 2024-09-20T22:20:22.933 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b097742a:::benchmark_data_smithi060_196073_object25479:head# 2024-09-20T22:20:22.935 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b09aa1eb:::benchmark_data_smithi060_196073_object32343:head# 2024-09-20T22:20:22.935 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b09b26a0:::benchmark_data_smithi060_196073_object34323:head# 2024-09-20T22:20:22.936 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b09bce36:::benchmark_data_smithi060_196073_object61514:head# 2024-09-20T22:20:22.936 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b09d0cb2:::benchmark_data_smithi060_196073_object532:head# 2024-09-20T22:20:22.936 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b09e3ad8:::benchmark_data_smithi060_196073_object34535:head# 2024-09-20T22:20:22.937 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b09e7020:::benchmark_data_smithi060_196073_object52637:head# 2024-09-20T22:20:22.939 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b09f32a3:::benchmark_data_smithi060_196073_object41650:head# 2024-09-20T22:20:22.939 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0a1250f:::benchmark_data_smithi060_196073_object19596:head# 2024-09-20T22:20:22.940 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0a35f39:::benchmark_data_smithi060_196073_object22189:head# 2024-09-20T22:20:22.941 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0a3ac73:::benchmark_data_smithi060_196073_object51096:head# 2024-09-20T22:20:22.948 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0a3cad0:::benchmark_data_smithi060_196073_object54364:head# 2024-09-20T22:20:22.948 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0a495b9:::benchmark_data_smithi060_196073_object39684:head# 2024-09-20T22:20:22.948 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0a59103:::benchmark_data_smithi060_196073_object35915:head# 2024-09-20T22:20:22.948 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0a898fb:::benchmark_data_smithi060_196073_object18285:head# 2024-09-20T22:20:22.948 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0a8d1ee:::benchmark_data_smithi060_196073_object9079:head# 2024-09-20T22:20:22.948 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0ab7cdd:::benchmark_data_smithi060_196073_object28348:head# 2024-09-20T22:20:22.949 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0ac225a:::benchmark_data_smithi060_196073_object59363:head# 2024-09-20T22:20:22.949 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0ae2f17:::benchmark_data_smithi060_196073_object3332:head# 2024-09-20T22:20:22.949 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0ae41ec:::benchmark_data_smithi060_196073_object9966:head# 2024-09-20T22:20:22.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:22.957 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:22.957 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:22.958 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0b3b4ce:::benchmark_data_smithi060_196073_object39442:head# 2024-09-20T22:20:22.958 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0b45897:::benchmark_data_smithi060_196073_object8236:head# 2024-09-20T22:20:22.958 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0b464cc:::benchmark_data_smithi060_196073_object7048:head# 2024-09-20T22:20:22.958 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0b567a0:::benchmark_data_smithi060_196073_object13647:head# 2024-09-20T22:20:22.958 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0ba4e27:::benchmark_data_smithi060_196073_object18189:head# 2024-09-20T22:20:22.958 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0baed22:::benchmark_data_smithi060_196073_object55252:head# 2024-09-20T22:20:22.959 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0bcbb63:::benchmark_data_smithi060_196073_object23314:head# 2024-09-20T22:20:22.959 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0bde123:::benchmark_data_smithi060_196073_object40682:head# 2024-09-20T22:20:22.959 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0beaa9a:::benchmark_data_smithi060_196073_object26864:head# 2024-09-20T22:20:22.959 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0c17288:::benchmark_data_smithi060_196073_object52827:head# 2024-09-20T22:20:22.959 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0c4debd:::benchmark_data_smithi060_196073_object32372:head# 2024-09-20T22:20:22.960 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0c6a001:::benchmark_data_smithi060_196073_object9634:head# 2024-09-20T22:20:22.961 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0c86962:::benchmark_data_smithi060_196073_object52385:head# 2024-09-20T22:20:22.962 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0c8862d:::benchmark_data_smithi060_196073_object37135:head# 2024-09-20T22:20:22.962 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0c8dfde:::benchmark_data_smithi060_196073_object27140:head# 2024-09-20T22:20:22.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0cb6d49:::benchmark_data_smithi060_196073_object22288:head# 2024-09-20T22:20:22.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0cda6ab:::benchmark_data_smithi060_196073_object37585:head# 2024-09-20T22:20:22.965 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0cdab77:::benchmark_data_smithi060_196073_object38565:head# 2024-09-20T22:20:22.965 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0d06a37:::benchmark_data_smithi060_196073_object55018:head# 2024-09-20T22:20:22.968 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0d0d458:::benchmark_data_smithi060_196073_object34659:head# 2024-09-20T22:20:22.970 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0d15e27:::benchmark_data_smithi060_196073_object47147:head# 2024-09-20T22:20:22.970 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0d20b6d:::benchmark_data_smithi060_196073_object26695:head# 2024-09-20T22:20:22.972 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0d235b0:::benchmark_data_smithi060_196073_object43425:head# 2024-09-20T22:20:22.975 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0d31b13:::benchmark_data_smithi060_196073_object17223:head# 2024-09-20T22:20:22.975 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0d3352a:::benchmark_data_smithi060_196073_object29404:head# 2024-09-20T22:20:22.975 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0d39aba:::benchmark_data_smithi060_196073_object19192:head# 2024-09-20T22:20:22.975 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0d4474f:::benchmark_data_smithi060_196073_object59456:head# 2024-09-20T22:20:22.977 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0d4e53c:::benchmark_data_smithi060_196073_object25040:head# 2024-09-20T22:20:22.977 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0d66072:::benchmark_data_smithi060_196073_object50354:head# 2024-09-20T22:20:22.980 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0d7db4b:::benchmark_data_smithi060_196073_object47642:head# 2024-09-20T22:20:22.980 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0db2528:::benchmark_data_smithi060_196073_object59116:head# 2024-09-20T22:20:22.981 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0dba1a1:::benchmark_data_smithi060_196073_object41489:head# 2024-09-20T22:20:22.982 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0dc6e88:::benchmark_data_smithi060_196073_object41622:head# 2024-09-20T22:20:22.983 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0dd6f80:::benchmark_data_smithi060_196073_object29548:head# 2024-09-20T22:20:22.983 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0e0e7b3:::benchmark_data_smithi060_196073_object1012:head# 2024-09-20T22:20:22.984 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0e153b2:::benchmark_data_smithi060_196073_object24083:head# 2024-09-20T22:20:22.985 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0e165e5:::benchmark_data_smithi060_196073_object25633:head# 2024-09-20T22:20:22.985 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0e1f12e:::benchmark_data_smithi060_196073_object11000:head# 2024-09-20T22:20:22.986 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0e22439:::benchmark_data_smithi060_196073_object30966:head# 2024-09-20T22:20:22.987 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0e31d8f:::benchmark_data_smithi060_196073_object46151:head# 2024-09-20T22:20:22.989 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0e325fa:::benchmark_data_smithi060_196073_object60717:head# 2024-09-20T22:20:22.991 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0e5865f:::benchmark_data_smithi060_196073_object21920:head# 2024-09-20T22:20:22.992 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0e58ac5:::benchmark_data_smithi060_196073_object17667:head# 2024-09-20T22:20:22.995 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0e73d7b:::benchmark_data_smithi060_196073_object36188:head# 2024-09-20T22:20:22.999 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0e7ba07:::benchmark_data_smithi060_196073_object42039:head# 2024-09-20T22:20:22.999 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0e8c599:::benchmark_data_smithi060_196073_object19947:head# 2024-09-20T22:20:22.999 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0ea6925:::benchmark_data_smithi060_196073_object23393:head# 2024-09-20T22:20:22.999 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0ede2eb:::benchmark_data_smithi060_196073_object22931:head# 2024-09-20T22:20:22.999 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0edeceb:::benchmark_data_smithi060_196073_object47445:head# 2024-09-20T22:20:23.005 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0ee3fff:::benchmark_data_smithi060_196073_object39894:head# 2024-09-20T22:20:23.005 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0efa1fa:::benchmark_data_smithi060_196073_object56887:head# 2024-09-20T22:20:23.005 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0f2db78:::benchmark_data_smithi060_196073_object54278:head# 2024-09-20T22:20:23.006 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0f585ac:::benchmark_data_smithi060_196073_object10683:head# 2024-09-20T22:20:23.006 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0f872eb:::benchmark_data_smithi060_196073_object53104:head# 2024-09-20T22:20:23.006 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0faff0d:::benchmark_data_smithi060_196073_object40291:head# 2024-09-20T22:20:23.006 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0fcb5de:::benchmark_data_smithi060_196073_object47231:head# 2024-09-20T22:20:23.006 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0fe322a:::benchmark_data_smithi060_196073_object19133:head# 2024-09-20T22:20:23.010 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0feaba1:::benchmark_data_smithi060_196073_object28156:head# 2024-09-20T22:20:23.010 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0ff1ae2:::benchmark_data_smithi060_196073_object33648:head# 2024-09-20T22:20:23.010 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0ff1c5a:::benchmark_data_smithi060_196073_object4009:head# 2024-09-20T22:20:23.010 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b0ff5ba4:::benchmark_data_smithi060_196073_object40582:head# 2024-09-20T22:20:23.011 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b100472d:::benchmark_data_smithi060_196073_object53569:head# 2024-09-20T22:20:23.014 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1010683:::benchmark_data_smithi060_196073_object11247:head# 2024-09-20T22:20:23.014 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b101c32b:::benchmark_data_smithi060_196073_object45548:head# 2024-09-20T22:20:23.014 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b104ee61:::benchmark_data_smithi060_196073_object49883:head# 2024-09-20T22:20:23.014 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b106c2fa:::benchmark_data_smithi060_196073_object12322:head# 2024-09-20T22:20:23.016 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b107a0a9:::benchmark_data_smithi060_196073_object25345:head# 2024-09-20T22:20:23.016 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b108273b:::benchmark_data_smithi060_196073_object36141:head# 2024-09-20T22:20:23.017 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b10887fd:::benchmark_data_smithi060_196073_object51280:head# 2024-09-20T22:20:23.020 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b108a653:::benchmark_data_smithi060_196073_object11846:head# 2024-09-20T22:20:23.020 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1090eef:::benchmark_data_smithi060_196073_object24964:head# 2024-09-20T22:20:23.020 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b10aa44e:::benchmark_data_smithi060_196073_object47968:head# 2024-09-20T22:20:23.020 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b110e90b:::benchmark_data_smithi060_196073_object878:head# 2024-09-20T22:20:23.021 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1110f6e:::benchmark_data_smithi060_196073_object1026:head# 2024-09-20T22:20:23.022 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b111af73:::benchmark_data_smithi060_196073_object42542:head# 2024-09-20T22:20:23.024 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b111e1f9:::benchmark_data_smithi060_196073_object26066:head# 2024-09-20T22:20:23.024 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1124668:::benchmark_data_smithi060_196073_object15523:head# 2024-09-20T22:20:23.027 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b11573ca:::benchmark_data_smithi060_196073_object22922:head# 2024-09-20T22:20:23.028 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1160d6e:::benchmark_data_smithi060_196073_object31460:head# 2024-09-20T22:20:23.030 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1160fc3:::benchmark_data_smithi060_196073_object61184:head# 2024-09-20T22:20:23.030 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b11622c7:::benchmark_data_smithi060_196073_object7023:head# 2024-09-20T22:20:23.031 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1184cb1:::benchmark_data_smithi060_196073_object18970:head# 2024-09-20T22:20:23.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b119b825:::benchmark_data_smithi060_196073_object54373:head# 2024-09-20T22:20:23.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b11b31c6:::benchmark_data_smithi060_196073_object59545:head# 2024-09-20T22:20:23.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b11bf3ba:::benchmark_data_smithi060_196073_object51482:head# 2024-09-20T22:20:23.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b11c04a6:::benchmark_data_smithi060_196073_object35963:head# 2024-09-20T22:20:23.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b11e3f9b:::benchmark_data_smithi060_196073_object60770:head# 2024-09-20T22:20:23.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b11f10a4:::benchmark_data_smithi060_196073_object46272:head# 2024-09-20T22:20:23.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1254199:::benchmark_data_smithi060_196073_object51984:head# 2024-09-20T22:20:23.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b126b6c3:::benchmark_data_smithi060_196073_object14650:head# 2024-09-20T22:20:23.039 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:22 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 9938/291744 objects degraded (3.406%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:23.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:22 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 9938/291744 objects degraded (3.406%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:23.044 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b127a0a2:::benchmark_data_smithi060_196073_object25854:head# 2024-09-20T22:20:23.044 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b12c95ba:::benchmark_data_smithi060_196073_object59990:head# 2024-09-20T22:20:23.044 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1305039:::benchmark_data_smithi060_196073_object43136:head# 2024-09-20T22:20:23.044 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1306b70:::benchmark_data_smithi060_196073_object36231:head# 2024-09-20T22:20:23.044 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1327604:::benchmark_data_smithi060_196073_object12016:head# 2024-09-20T22:20:23.044 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b132c597:::benchmark_data_smithi060_196073_object12076:head# 2024-09-20T22:20:23.045 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b13402a2:::benchmark_data_smithi060_196073_object35009:head# 2024-09-20T22:20:23.046 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1348b69:::benchmark_data_smithi060_196073_object40473:head# 2024-09-20T22:20:23.049 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b134b1a2:::benchmark_data_smithi060_196073_object51484:head# 2024-09-20T22:20:23.050 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b134d217:::benchmark_data_smithi060_196073_object53306:head# 2024-09-20T22:20:23.052 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b136d7fb:::benchmark_data_smithi060_196073_object40531:head# 2024-09-20T22:20:23.055 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1392b64:::benchmark_data_smithi060_196073_object48207:head# 2024-09-20T22:20:23.055 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b13963d3:::benchmark_data_smithi060_196073_object61127:head# 2024-09-20T22:20:23.055 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b139f2f3:::benchmark_data_smithi060_196073_object34340:head# 2024-09-20T22:20:23.058 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b13bbb99:::benchmark_data_smithi060_196073_object1972:head# 2024-09-20T22:20:23.058 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b13c729d:::benchmark_data_smithi060_196073_object54958:head# 2024-09-20T22:20:23.058 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b13c8a4f:::benchmark_data_smithi060_196073_object48535:head# 2024-09-20T22:20:23.058 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b13fe383:::benchmark_data_smithi060_196073_object36265:head# 2024-09-20T22:20:23.058 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:23.063 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b13ffde4:::benchmark_data_smithi060_196073_object24285:head# 2024-09-20T22:20:23.064 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b140c696:::benchmark_data_smithi060_196073_object22776:head# 2024-09-20T22:20:23.064 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b14334f6:::benchmark_data_smithi060_196073_object57748:head# 2024-09-20T22:20:23.065 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1434cb3:::benchmark_data_smithi060_196073_object41363:head# 2024-09-20T22:20:23.065 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b147f949:::benchmark_data_smithi060_196073_object572:head# 2024-09-20T22:20:23.065 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1496c2c:::benchmark_data_smithi060_196073_object13638:head# 2024-09-20T22:20:23.066 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b14a1c0b:::benchmark_data_smithi060_196073_object10868:head# 2024-09-20T22:20:23.070 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b14a5603:::benchmark_data_smithi060_196073_object17653:head# 2024-09-20T22:20:23.071 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b14a7fb8:::benchmark_data_smithi060_196073_object57751:head# 2024-09-20T22:20:23.071 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b14a8220:::benchmark_data_smithi060_196073_object5455:head# 2024-09-20T22:20:23.071 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b14a985f:::benchmark_data_smithi060_196073_object9344:head# 2024-09-20T22:20:23.072 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b14b13f7:::benchmark_data_smithi060_196073_object2783:head# 2024-09-20T22:20:23.072 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b14b5208:::benchmark_data_smithi060_196073_object39123:head# 2024-09-20T22:20:23.073 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b14c2d08:::benchmark_data_smithi060_196073_object23809:head# 2024-09-20T22:20:23.074 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b14c4d5f:::benchmark_data_smithi060_196073_object41980:head# 2024-09-20T22:20:23.075 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b14da5cd:::benchmark_data_smithi060_196073_object39614:head# 2024-09-20T22:20:23.079 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b14e3d77:::benchmark_data_smithi060_196073_object30892:head# 2024-09-20T22:20:23.079 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b14e487c:::benchmark_data_smithi060_196073_object42723:head# 2024-09-20T22:20:23.079 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b14ecc31:::benchmark_data_smithi060_196073_object49362:head# 2024-09-20T22:20:23.079 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b14f0caf:::benchmark_data_smithi060_196073_object17359:head# 2024-09-20T22:20:23.079 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b14fa24a:::benchmark_data_smithi060_196073_object35102:head# 2024-09-20T22:20:23.081 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b14fed2e:::benchmark_data_smithi060_196073_object4207:head# 2024-09-20T22:20:23.082 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b150658b:::benchmark_data_smithi060_196073_object8061:head# 2024-09-20T22:20:23.082 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b15178e3:::benchmark_data_smithi060_196073_object38195:head# 2024-09-20T22:20:23.083 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b153176a:::benchmark_data_smithi060_196073_object31396:head# 2024-09-20T22:20:23.084 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b15933e6:::benchmark_data_smithi060_196073_object17298:head# 2024-09-20T22:20:23.085 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b15a91aa:::benchmark_data_smithi060_196073_object45751:head# 2024-09-20T22:20:23.086 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b15ab66a:::benchmark_data_smithi060_196073_object7429:head# 2024-09-20T22:20:23.086 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b15b2805:::benchmark_data_smithi060_196073_object49689:head# 2024-09-20T22:20:23.090 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b15c370d:::benchmark_data_smithi060_196073_object27366:head# 2024-09-20T22:20:23.090 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b15c3d3f:::benchmark_data_smithi060_196073_object39144:head# 2024-09-20T22:20:23.090 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b15d5543:::benchmark_data_smithi060_196073_object37633:head# 2024-09-20T22:20:23.092 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b15d65ac:::benchmark_data_smithi060_196073_object18434:head# 2024-09-20T22:20:23.092 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b15ddda3:::benchmark_data_smithi060_196073_object21715:head# 2024-09-20T22:20:23.092 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b15df093:::benchmark_data_smithi060_196073_object1946:head# 2024-09-20T22:20:23.093 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b15e15af:::benchmark_data_smithi060_196073_object52012:head# 2024-09-20T22:20:23.093 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b15f29de:::benchmark_data_smithi060_196073_object27742:head# 2024-09-20T22:20:23.094 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b15f71f1:::benchmark_data_smithi060_196073_object40642:head# 2024-09-20T22:20:23.095 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b15fe9b9:::benchmark_data_smithi060_196073_object52157:head# 2024-09-20T22:20:23.095 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1600319:::benchmark_data_smithi060_196073_object26737:head# 2024-09-20T22:20:23.096 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1616c96:::benchmark_data_smithi060_196073_object52965:head# 2024-09-20T22:20:23.096 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b16393cb:::benchmark_data_smithi060_196073_object50023:head# 2024-09-20T22:20:23.097 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b163cea2:::benchmark_data_smithi060_196073_object19401:head# 2024-09-20T22:20:23.097 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b16455e8:::benchmark_data_smithi060_196073_object31502:head# 2024-09-20T22:20:23.098 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1648b65:::benchmark_data_smithi060_196073_object9297:head# 2024-09-20T22:20:23.099 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b164ef76:::benchmark_data_smithi060_196073_object7090:head# 2024-09-20T22:20:23.099 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1653ccf:::benchmark_data_smithi060_196073_object15713:head# 2024-09-20T22:20:23.100 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1654535:::benchmark_data_smithi060_196073_object51012:head# 2024-09-20T22:20:23.100 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b16630d8:::benchmark_data_smithi060_196073_object44360:head# 2024-09-20T22:20:23.102 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:23.102 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b167353f:::benchmark_data_smithi060_196073_object19081:head# 2024-09-20T22:20:23.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:23.103 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:23.105 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b167db41:::benchmark_data_smithi060_196073_object41129:head# 2024-09-20T22:20:23.106 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b167f240:::benchmark_data_smithi060_196073_object38269:head# 2024-09-20T22:20:23.106 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b16984a8:::benchmark_data_smithi060_196073_object47325:head# 2024-09-20T22:20:23.107 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b16a9979:::benchmark_data_smithi060_196073_object59103:head# 2024-09-20T22:20:23.107 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b16bf061:::benchmark_data_smithi060_196073_object3708:head# 2024-09-20T22:20:23.109 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b16c9371:::benchmark_data_smithi060_196073_object29226:head# 2024-09-20T22:20:23.109 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b16cf1e5:::benchmark_data_smithi060_196073_object15823:head# 2024-09-20T22:20:23.109 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Total time run: 90.0104 2024-09-20T22:20:23.110 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Total writes made: 101984 2024-09-20T22:20:23.110 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Write size: 65536 2024-09-20T22:20:23.110 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Object size: 65536 2024-09-20T22:20:23.110 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Bandwidth (MB/sec): 70.8141 2024-09-20T22:20:23.110 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Stddev Bandwidth: 41.9803 2024-09-20T22:20:23.110 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Max bandwidth (MB/sec): 206.938 2024-09-20T22:20:23.110 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Min bandwidth (MB/sec): 0 2024-09-20T22:20:23.110 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Average IOPS: 1133 2024-09-20T22:20:23.110 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Stddev IOPS: 671.693 2024-09-20T22:20:23.110 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Max IOPS: 3311 2024-09-20T22:20:23.110 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Min IOPS: 0 2024-09-20T22:20:23.110 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Average Latency(s): 0.0141165 2024-09-20T22:20:23.110 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Stddev Latency(s): 0.0424103 2024-09-20T22:20:23.111 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Max latency(s): 2.25818 2024-09-20T22:20:23.111 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Min latency(s): 0.00118665 2024-09-20T22:20:23.111 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b16efe1d:::benchmark_data_smithi060_196073_object16018:head# 2024-09-20T22:20:23.112 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1700a63:::benchmark_data_smithi060_196073_object31402:head# 2024-09-20T22:20:23.112 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1701562:::benchmark_data_smithi060_196073_object24350:head# 2024-09-20T22:20:23.113 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b17069e9:::benchmark_data_smithi060_196073_object172:head# 2024-09-20T22:20:23.113 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b171f41d:::benchmark_data_smithi060_196073_object3345:head# 2024-09-20T22:20:23.113 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1721c86:::benchmark_data_smithi060_196073_object984:head# 2024-09-20T22:20:23.114 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b175837a:::benchmark_data_smithi060_196073_object18169:head# 2024-09-20T22:20:23.114 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b17611ee:::benchmark_data_smithi060_196073_object29612:head# 2024-09-20T22:20:23.115 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b177fd6a:::benchmark_data_smithi060_196073_object19486:head# 2024-09-20T22:20:23.116 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1786850:::benchmark_data_smithi060_196073_object57158:head# 2024-09-20T22:20:23.122 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b178b1ab:::benchmark_data_smithi060_196073_object8699:head# 2024-09-20T22:20:23.122 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1790c3c:::benchmark_data_smithi060_196073_object17249:head# 2024-09-20T22:20:23.122 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b17976aa:::benchmark_data_smithi060_196073_object1741:head# 2024-09-20T22:20:23.122 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b17a0c69:::benchmark_data_smithi060_196073_object34700:head# 2024-09-20T22:20:23.122 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b17a5ccc:::benchmark_data_smithi060_196073_object7348:head# 2024-09-20T22:20:23.122 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b17c3352:::benchmark_data_smithi060_196073_object55256:head# 2024-09-20T22:20:23.122 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b17c3424:::benchmark_data_smithi060_196073_object7166:head# 2024-09-20T22:20:23.125 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b17dfd3c:::benchmark_data_smithi060_196073_object44881:head# 2024-09-20T22:20:23.125 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b17e37e2:::benchmark_data_smithi060_196073_object28690:head# 2024-09-20T22:20:23.125 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b17f6f78:::benchmark_data_smithi060_196073_object30674:head# 2024-09-20T22:20:23.126 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1814e18:::benchmark_data_smithi060_196073_object61310:head# 2024-09-20T22:20:23.126 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b18166ad:::benchmark_data_smithi060_196073_object9972:head# 2024-09-20T22:20:23.126 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1820f7f:::benchmark_data_smithi060_196073_object41739:head# 2024-09-20T22:20:23.127 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b18339d0:::benchmark_data_smithi060_196073_object14394:head# 2024-09-20T22:20:23.128 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1833be7:::benchmark_data_smithi060_196073_object9168:head# 2024-09-20T22:20:23.131 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b184a8d4:::benchmark_data_smithi060_196073_object45331:head# 2024-09-20T22:20:23.132 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Cleaning up (deleting benchmark objects) 2024-09-20T22:20:23.134 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b18599e4:::benchmark_data_smithi060_196073_object59007:head# 2024-09-20T22:20:23.134 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b186bd67:::benchmark_data_smithi060_196073_object39468:head# 2024-09-20T22:20:23.135 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1885582:::benchmark_data_smithi060_196073_object31116:head# 2024-09-20T22:20:23.137 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b188d1ca:::benchmark_data_smithi060_196073_object9925:head# 2024-09-20T22:20:23.138 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1891c7f:::benchmark_data_smithi060_196073_object54192:head# 2024-09-20T22:20:23.139 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b189a5f3:::benchmark_data_smithi060_196073_object53520:head# 2024-09-20T22:20:23.139 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b189f82e:::benchmark_data_smithi060_196073_object37862:head# 2024-09-20T22:20:23.140 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b18ac185:::benchmark_data_smithi060_196073_object44799:head# 2024-09-20T22:20:23.142 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b18add0d:::benchmark_data_smithi060_196073_object35555:head# 2024-09-20T22:20:23.143 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b18ce425:::benchmark_data_smithi060_196073_object1223:head# 2024-09-20T22:20:23.143 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b18d539a:::benchmark_data_smithi060_196073_object28064:head# 2024-09-20T22:20:23.144 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b18f622f:::benchmark_data_smithi060_196073_object6777:head# 2024-09-20T22:20:23.145 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1903d9f:::benchmark_data_smithi060_196073_object3219:head# 2024-09-20T22:20:23.145 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b190539a:::benchmark_data_smithi060_196073_object35611:head# 2024-09-20T22:20:23.146 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b192a831:::benchmark_data_smithi060_196073_object11287:head# 2024-09-20T22:20:23.147 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b19432c6:::benchmark_data_smithi060_196073_object6855:head# 2024-09-20T22:20:23.148 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b19614be:::benchmark_data_smithi060_196073_object43347:head# 2024-09-20T22:20:23.148 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1968de8:::benchmark_data_smithi060_196073_object55470:head# 2024-09-20T22:20:23.149 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b198e6bb:::benchmark_data_smithi060_196073_object52355:head# 2024-09-20T22:20:23.150 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b19967ad:::benchmark_data_smithi060_196073_object11291:head# 2024-09-20T22:20:23.150 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b19a5d7b:::benchmark_data_smithi060_196073_object54770:head# 2024-09-20T22:20:23.151 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b19aa1c6:::benchmark_data_smithi060_196073_object23160:head# 2024-09-20T22:20:23.152 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b19c61c6:::benchmark_data_smithi060_196073_object23493:head# 2024-09-20T22:20:23.152 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b19c85cc:::benchmark_data_smithi060_196073_object3455:head# 2024-09-20T22:20:23.153 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b19de50b:::benchmark_data_smithi060_196073_object8191:head# 2024-09-20T22:20:23.154 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b19e238a:::benchmark_data_smithi060_196073_object27878:head# 2024-09-20T22:20:23.157 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1a03887:::benchmark_data_smithi060_196073_object13283:head# 2024-09-20T22:20:23.158 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1a03bb4:::benchmark_data_smithi060_196073_object11605:head# 2024-09-20T22:20:23.158 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1a118e4:::benchmark_data_smithi060_196073_object49878:head# 2024-09-20T22:20:23.159 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1a1caac:::benchmark_data_smithi060_196073_object31750:head# 2024-09-20T22:20:23.159 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1a227eb:::benchmark_data_smithi060_196073_object7064:head# 2024-09-20T22:20:23.160 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1a23260:::benchmark_data_smithi060_196073_object25868:head# 2024-09-20T22:20:23.161 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1a43f70:::benchmark_data_smithi060_196073_object51972:head# 2024-09-20T22:20:23.162 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1a44009:::benchmark_data_smithi060_196073_object59233:head# 2024-09-20T22:20:23.163 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1a459ea:::benchmark_data_smithi060_196073_object4214:head# 2024-09-20T22:20:23.164 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1a61082:::benchmark_data_smithi060_196073_object31147:head# 2024-09-20T22:20:23.165 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1a67f7b:::benchmark_data_smithi060_196073_object53154:head# 2024-09-20T22:20:23.165 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1a721d0:::benchmark_data_smithi060_196073_object42958:head# 2024-09-20T22:20:23.166 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1a788fc:::benchmark_data_smithi060_196073_object43478:head# 2024-09-20T22:20:23.167 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1a7e102:::benchmark_data_smithi060_196073_object42705:head# 2024-09-20T22:20:23.168 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1aa9c1c:::benchmark_data_smithi060_196073_object55570:head# 2024-09-20T22:20:23.168 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1aab4f9:::benchmark_data_smithi060_196073_object32181:head# 2024-09-20T22:20:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1adaad9:::benchmark_data_smithi060_196073_object13472:head# 2024-09-20T22:20:23.169 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1ae4663:::benchmark_data_smithi060_196073_object9570:head# 2024-09-20T22:20:23.171 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1aee09f:::benchmark_data_smithi060_196073_object53362:head# 2024-09-20T22:20:23.171 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1b0522a:::benchmark_data_smithi060_196073_object41427:head# 2024-09-20T22:20:23.172 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1b0a5b5:::benchmark_data_smithi060_196073_object42259:head# 2024-09-20T22:20:23.173 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1b0f6bc:::benchmark_data_smithi060_196073_object29401:head# 2024-09-20T22:20:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1b207fc:::benchmark_data_smithi060_196073_object34257:head# 2024-09-20T22:20:23.174 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1b532a1:::benchmark_data_smithi060_196073_object48477:head# 2024-09-20T22:20:23.175 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1b54768:::benchmark_data_smithi060_196073_object2564:head# 2024-09-20T22:20:23.176 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1b5d7e8:::benchmark_data_smithi060_196073_object52331:head# 2024-09-20T22:20:23.177 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1b6c68c:::benchmark_data_smithi060_196073_object52017:head# 2024-09-20T22:20:23.177 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1b6f222:::benchmark_data_smithi060_196073_object18597:head# 2024-09-20T22:20:23.178 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1b7f6bd:::benchmark_data_smithi060_196073_object30207:head# 2024-09-20T22:20:23.179 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1b80d52:::benchmark_data_smithi060_196073_object33795:head# 2024-09-20T22:20:23.179 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1b87cfe:::benchmark_data_smithi060_196073_object22447:head# 2024-09-20T22:20:23.180 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1b919bf:::benchmark_data_smithi060_196073_object1335:head# 2024-09-20T22:20:23.181 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1b95026:::benchmark_data_smithi060_196073_object56461:head# 2024-09-20T22:20:23.182 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1baaf1a:::benchmark_data_smithi060_196073_object60645:head# 2024-09-20T22:20:23.182 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1bea4e3:::benchmark_data_smithi060_196073_object34397:head# 2024-09-20T22:20:23.183 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1bf4351:::benchmark_data_smithi060_196073_object25097:head# 2024-09-20T22:20:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1c0045e:::benchmark_data_smithi060_196073_object11237:head# 2024-09-20T22:20:23.184 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1c137df:::benchmark_data_smithi060_196073_object22682:head# 2024-09-20T22:20:23.185 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1c1be94:::benchmark_data_smithi060_196073_object6615:head# 2024-09-20T22:20:23.186 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1c1d598:::benchmark_data_smithi060_196073_object30428:head# 2024-09-20T22:20:23.186 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1c5a945:::benchmark_data_smithi060_196073_object46141:head# 2024-09-20T22:20:23.187 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1c5d57a:::benchmark_data_smithi060_196073_object46063:head# 2024-09-20T22:20:23.188 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1c66ab2:::benchmark_data_smithi060_196073_object6252:head# 2024-09-20T22:20:23.189 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1c7399b:::benchmark_data_smithi060_196073_object9444:head# 2024-09-20T22:20:23.190 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1cc22f2:::benchmark_data_smithi060_196073_object38386:head# 2024-09-20T22:20:23.191 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1ccb637:::benchmark_data_smithi060_196073_object45236:head# 2024-09-20T22:20:23.191 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1cd8473:::benchmark_data_smithi060_196073_object59325:head# 2024-09-20T22:20:23.192 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1cddb43:::benchmark_data_smithi060_196073_object7980:head# 2024-09-20T22:20:23.193 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1cfbf83:::benchmark_data_smithi060_196073_object22920:head# 2024-09-20T22:20:23.193 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1d129eb:::benchmark_data_smithi060_196073_object13887:head# 2024-09-20T22:20:23.194 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1d173b3:::benchmark_data_smithi060_196073_object20016:head# 2024-09-20T22:20:23.194 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1d17f30:::benchmark_data_smithi060_196073_object4416:head# 2024-09-20T22:20:23.195 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1d19742:::benchmark_data_smithi060_196073_object24941:head# 2024-09-20T22:20:23.196 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1d1d8e3:::benchmark_data_smithi060_196073_object14596:head# 2024-09-20T22:20:23.197 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1d200a9:::benchmark_data_smithi060_196073_object59419:head# 2024-09-20T22:20:23.197 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1d44be9:::benchmark_data_smithi060_196073_object35648:head# 2024-09-20T22:20:23.200 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1d56bd0:::benchmark_data_smithi060_196073_object49131:head# 2024-09-20T22:20:23.200 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1d5eec2:::benchmark_data_smithi060_196073_object49591:head# 2024-09-20T22:20:23.200 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1d6a857:::benchmark_data_smithi060_196073_object23104:head# 2024-09-20T22:20:23.205 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1d7e05a:::benchmark_data_smithi060_196073_object51217:head# 2024-09-20T22:20:23.205 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1d7f90e:::benchmark_data_smithi060_196073_object40552:head# 2024-09-20T22:20:23.205 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1d84b70:::benchmark_data_smithi060_196073_object57038:head# 2024-09-20T22:20:23.205 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1d891ec:::benchmark_data_smithi060_196073_object15846:head# 2024-09-20T22:20:23.206 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1d9a595:::benchmark_data_smithi060_196073_object26678:head# 2024-09-20T22:20:23.206 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1d9ec61:::benchmark_data_smithi060_196073_object29331:head# 2024-09-20T22:20:23.206 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1da8d74:::benchmark_data_smithi060_196073_object37325:head# 2024-09-20T22:20:23.206 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1daca41:::benchmark_data_smithi060_196073_object54544:head# 2024-09-20T22:20:23.206 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1db0435:::benchmark_data_smithi060_196073_object26473:head# 2024-09-20T22:20:23.206 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1dc5dda:::benchmark_data_smithi060_196073_object12538:head# 2024-09-20T22:20:23.206 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:23.207 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1de9179:::benchmark_data_smithi060_196073_object33246:head# 2024-09-20T22:20:23.207 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1df0a95:::benchmark_data_smithi060_196073_object6389:head# 2024-09-20T22:20:23.207 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1dfd7a6:::benchmark_data_smithi060_196073_object5486:head# 2024-09-20T22:20:23.207 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1e1fabd:::benchmark_data_smithi060_196073_object29662:head# 2024-09-20T22:20:23.209 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1e65ad1:::benchmark_data_smithi060_196073_object60405:head# 2024-09-20T22:20:23.209 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1e6952c:::benchmark_data_smithi060_196073_object34924:head# 2024-09-20T22:20:23.209 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1e72a45:::benchmark_data_smithi060_196073_object45973:head# 2024-09-20T22:20:23.210 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1e8634a:::benchmark_data_smithi060_196073_object12591:head# 2024-09-20T22:20:23.210 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1ea119f:::benchmark_data_smithi060_196073_object52590:head# 2024-09-20T22:20:23.211 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1ec91eb:::benchmark_data_smithi060_196073_object24222:head# 2024-09-20T22:20:23.212 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1ecf8ae:::benchmark_data_smithi060_196073_object2354:head# 2024-09-20T22:20:23.212 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1ed4ad4:::benchmark_data_smithi060_196073_object119:head# 2024-09-20T22:20:23.213 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1ed9bfb:::benchmark_data_smithi060_196073_object60666:head# 2024-09-20T22:20:23.213 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1edbdbb:::benchmark_data_smithi060_196073_object28412:head# 2024-09-20T22:20:23.214 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1f06df8:::benchmark_data_smithi060_196073_object31446:head# 2024-09-20T22:20:23.215 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1f17865:::benchmark_data_smithi060_196073_object11361:head# 2024-09-20T22:20:23.216 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1f22190:::benchmark_data_smithi060_196073_object34572:head# 2024-09-20T22:20:23.216 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1f2fae9:::benchmark_data_smithi060_196073_object50113:head# 2024-09-20T22:20:23.217 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1f36a04:::benchmark_data_smithi060_196073_object13079:head# 2024-09-20T22:20:23.220 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1f3f8ba:::benchmark_data_smithi060_196073_object30183:head# 2024-09-20T22:20:23.221 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1f41e7f:::benchmark_data_smithi060_196073_object15184:head# 2024-09-20T22:20:23.221 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1f4d0fb:::benchmark_data_smithi060_196073_object7869:head# 2024-09-20T22:20:23.222 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1f6ab04:::benchmark_data_smithi060_196073_object44776:head# 2024-09-20T22:20:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1f781be:::benchmark_data_smithi060_196073_object14328:head# 2024-09-20T22:20:23.223 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1f9793e:::benchmark_data_smithi060_196073_object11339:head# 2024-09-20T22:20:23.224 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1fae681:::benchmark_data_smithi060_196073_object3377:head# 2024-09-20T22:20:23.227 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1fcb4e2:::benchmark_data_smithi060_196073_object45153:head# 2024-09-20T22:20:23.227 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1fcf4eb:::benchmark_data_smithi060_196073_object916:head# 2024-09-20T22:20:23.228 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1fe8778:::benchmark_data_smithi060_196073_object36232:head# 2024-09-20T22:20:23.228 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b1ff6e93:::benchmark_data_smithi060_196073_object55783:head# 2024-09-20T22:20:23.229 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b200413a:::benchmark_data_smithi060_196073_object7741:head# 2024-09-20T22:20:23.229 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2010394:::benchmark_data_smithi060_196073_object17992:head# 2024-09-20T22:20:23.230 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b20171b0:::benchmark_data_smithi060_196073_object803:head# 2024-09-20T22:20:23.230 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b207fb42:::benchmark_data_smithi060_196073_object29677:head# 2024-09-20T22:20:23.231 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b20868ce:::benchmark_data_smithi060_196073_object58754:head# 2024-09-20T22:20:23.231 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2098f46:::benchmark_data_smithi060_196073_object25424:head# 2024-09-20T22:20:23.233 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b20b4713:::benchmark_data_smithi060_196073_object25550:head# 2024-09-20T22:20:23.233 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b20bcdad:::benchmark_data_smithi060_196073_object19672:head# 2024-09-20T22:20:23.233 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b20c7f2e:::benchmark_data_smithi060_196073_object45777:head# 2024-09-20T22:20:23.234 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b20c95a2:::benchmark_data_smithi060_196073_object28775:head# 2024-09-20T22:20:23.235 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b20c9f3b:::benchmark_data_smithi060_196073_object54987:head# 2024-09-20T22:20:23.235 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b20d401b:::benchmark_data_smithi060_196073_object5917:head# 2024-09-20T22:20:23.236 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b20e0e74:::benchmark_data_smithi060_196073_object4828:head# 2024-09-20T22:20:23.237 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b20ef432:::benchmark_data_smithi060_196073_object10130:head# 2024-09-20T22:20:23.237 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2132318:::benchmark_data_smithi060_196073_object9695:head# 2024-09-20T22:20:23.238 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2156e07:::benchmark_data_smithi060_196073_object1018:head# 2024-09-20T22:20:23.239 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2179054:::benchmark_data_smithi060_196073_object37982:head# 2024-09-20T22:20:23.240 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b21a2524:::benchmark_data_smithi060_196073_object9008:head# 2024-09-20T22:20:23.241 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:23.242 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:23.243 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:23.244 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b21bfcd7:::benchmark_data_smithi060_196073_object14962:head# 2024-09-20T22:20:23.244 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b21dbc57:::benchmark_data_smithi060_196073_object45162:head# 2024-09-20T22:20:23.244 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b21dff35:::benchmark_data_smithi060_196073_object16919:head# 2024-09-20T22:20:23.246 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b21ed9a2:::benchmark_data_smithi060_196073_object31720:head# 2024-09-20T22:20:23.246 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b21ff9c8:::benchmark_data_smithi060_196073_object17922:head# 2024-09-20T22:20:23.246 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b220086b:::benchmark_data_smithi060_196073_object45263:head# 2024-09-20T22:20:23.246 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b22062a5:::benchmark_data_smithi060_196073_object34989:head# 2024-09-20T22:20:23.246 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b222f5d3:::benchmark_data_smithi060_196073_object30170:head# 2024-09-20T22:20:23.247 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b223bb9b:::benchmark_data_smithi060_196073_object2558:head# 2024-09-20T22:20:23.248 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b224d544:::benchmark_data_smithi060_196073_object8114:head# 2024-09-20T22:20:23.248 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b225549a:::benchmark_data_smithi060_196073_object44824:head# 2024-09-20T22:20:23.248 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b225a093:::benchmark_data_smithi060_196073_object25583:head# 2024-09-20T22:20:23.249 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b225ee06:::benchmark_data_smithi060_196073_object30095:head# 2024-09-20T22:20:23.251 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b225f674:::benchmark_data_smithi060_196073_object26602:head# 2024-09-20T22:20:23.251 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2299a88:::benchmark_data_smithi060_196073_object11487:head# 2024-09-20T22:20:23.254 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b22a7d27:::benchmark_data_smithi060_196073_object59889:head# 2024-09-20T22:20:23.254 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b22b3de0:::benchmark_data_smithi060_196073_object1177:head# 2024-09-20T22:20:23.255 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b22bdeb9:::benchmark_data_smithi060_196073_object18346:head# 2024-09-20T22:20:23.256 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b22c151e:::benchmark_data_smithi060_196073_object41038:head# 2024-09-20T22:20:23.256 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b22c3275:::benchmark_data_smithi060_196073_object24413:head# 2024-09-20T22:20:23.257 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b22fe516:::benchmark_data_smithi060_196073_object44411:head# 2024-09-20T22:20:23.257 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b230139c:::benchmark_data_smithi060_196073_object22443:head# 2024-09-20T22:20:23.258 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b230a86b:::benchmark_data_smithi060_196073_object61258:head# 2024-09-20T22:20:23.258 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2321333:::benchmark_data_smithi060_196073_object21113:head# 2024-09-20T22:20:23.261 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b23349a8:::benchmark_data_smithi060_196073_object14120:head# 2024-09-20T22:20:23.261 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b233b2dd:::benchmark_data_smithi060_196073_object38385:head# 2024-09-20T22:20:23.262 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b234fea3:::benchmark_data_smithi060_196073_object33439:head# 2024-09-20T22:20:23.263 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b237c20c:::benchmark_data_smithi060_196073_object61111:head# 2024-09-20T22:20:23.264 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:20:23.264 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:20:23.265 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b237c8d9:::benchmark_data_smithi060_196073_object31549:head# 2024-09-20T22:20:23.265 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b238da49:::benchmark_data_smithi060_196073_object44865:head# 2024-09-20T22:20:23.265 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2390d8c:::benchmark_data_smithi060_196073_object44901:head# 2024-09-20T22:20:23.266 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b23932c2:::benchmark_data_smithi060_196073_object58324:head# 2024-09-20T22:20:23.266 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b23b32ba:::benchmark_data_smithi060_196073_object12283:head# 2024-09-20T22:20:23.267 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b23c1223:::benchmark_data_smithi060_196073_object35928:head# 2024-09-20T22:20:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b23c7fac:::benchmark_data_smithi060_196073_object44485:head# 2024-09-20T22:20:23.268 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b23eb2fb:::benchmark_data_smithi060_196073_object7829:head# 2024-09-20T22:20:23.269 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b23f96dd:::benchmark_data_smithi060_196073_object28366:head# 2024-09-20T22:20:23.269 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2401156:::benchmark_data_smithi060_196073_object10706:head# 2024-09-20T22:20:23.270 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2459f38:::benchmark_data_smithi060_196073_object17407:head# 2024-09-20T22:20:23.270 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b248aad0:::benchmark_data_smithi060_196073_object38353:head# 2024-09-20T22:20:23.271 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b24922a8:::benchmark_data_smithi060_196073_object44140:head# 2024-09-20T22:20:23.271 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b24a3add:::benchmark_data_smithi060_196073_object3364:head# 2024-09-20T22:20:23.272 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b24c8a43:::benchmark_data_smithi060_196073_object10113:head# 2024-09-20T22:20:23.272 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b24d09ea:::benchmark_data_smithi060_196073_object51238:head# 2024-09-20T22:20:23.273 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b24e1568:::benchmark_data_smithi060_196073_object57688:head# 2024-09-20T22:20:23.273 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2522356:::benchmark_data_smithi060_196073_object35283:head# 2024-09-20T22:20:23.274 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b25310ac:::benchmark_data_smithi060_196073_object34918:head# 2024-09-20T22:20:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2532cd7:::benchmark_data_smithi060_196073_object5862:head# 2024-09-20T22:20:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2569327:::benchmark_data_smithi060_196073_object22146:head# 2024-09-20T22:20:23.275 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2580b34:::benchmark_data_smithi060_196073_object34937:head# 2024-09-20T22:20:23.281 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2587369:::benchmark_data_smithi060_196073_object78:head# 2024-09-20T22:20:23.281 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b259934e:::benchmark_data_smithi060_196073_object10574:head# 2024-09-20T22:20:23.281 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b25c358c:::benchmark_data_smithi060_196073_object8554:head# 2024-09-20T22:20:23.281 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b25c7b0c:::benchmark_data_smithi060_196073_object27741:head# 2024-09-20T22:20:23.281 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b25d957b:::benchmark_data_smithi060_196073_object34863:head# 2024-09-20T22:20:23.281 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b25edecc:::benchmark_data_smithi060_196073_object32105:head# 2024-09-20T22:20:23.281 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b260d0ba:::benchmark_data_smithi060_196073_object33848:head# 2024-09-20T22:20:23.282 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:23.283 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2612666:::benchmark_data_smithi060_196073_object9513:head# 2024-09-20T22:20:23.285 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b262cdbe:::benchmark_data_smithi060_196073_object24916:head# 2024-09-20T22:20:23.285 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b263a882:::benchmark_data_smithi060_196073_object50653:head# 2024-09-20T22:20:23.285 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b26504d7:::benchmark_data_smithi060_196073_object60348:head# 2024-09-20T22:20:23.286 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b265fbd1:::benchmark_data_smithi060_196073_object2357:head# 2024-09-20T22:20:23.286 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2691af5:::benchmark_data_smithi060_196073_object24751:head# 2024-09-20T22:20:23.287 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b26a791a:::benchmark_data_smithi060_196073_object43281:head# 2024-09-20T22:20:23.288 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b26b3bfe:::benchmark_data_smithi060_196073_object37248:head# 2024-09-20T22:20:23.289 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b26be0f0:::benchmark_data_smithi060_196073_object23798:head# 2024-09-20T22:20:23.290 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b26c82d2:::benchmark_data_smithi060_196073_object464:head# 2024-09-20T22:20:23.290 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b26ebdae:::benchmark_data_smithi060_196073_object1195:head# 2024-09-20T22:20:23.291 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b27076b2:::benchmark_data_smithi060_196073_object43811:head# 2024-09-20T22:20:23.291 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2736723:::benchmark_data_smithi060_196073_object27017:head# 2024-09-20T22:20:23.292 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2736ee6:::benchmark_data_smithi060_196073_object59635:head# 2024-09-20T22:20:23.292 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2737a43:::benchmark_data_smithi060_196073_object7062:head# 2024-09-20T22:20:23.293 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2747890:::benchmark_data_smithi060_196073_object14948:head# 2024-09-20T22:20:23.293 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b27542fc:::benchmark_data_smithi060_196073_object57302:head# 2024-09-20T22:20:23.294 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b275a8b6:::benchmark_data_smithi060_196073_object983:head# 2024-09-20T22:20:23.294 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b275fedb:::benchmark_data_smithi060_196073_object30078:head# 2024-09-20T22:20:23.295 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b279d775:::benchmark_data_smithi060_196073_object21517:head# 2024-09-20T22:20:23.295 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b279fde3:::benchmark_data_smithi060_196073_object52207:head# 2024-09-20T22:20:23.295 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b27a4b29:::benchmark_data_smithi060_196073_object40748:head# 2024-09-20T22:20:23.296 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b27ceb1e:::benchmark_data_smithi060_196073_object22989:head# 2024-09-20T22:20:23.297 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b27d1d5c:::benchmark_data_smithi060_196073_object46112:head# 2024-09-20T22:20:23.297 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b27d5eec:::benchmark_data_smithi060_196073_object44646:head# 2024-09-20T22:20:23.298 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b27dd5e1:::benchmark_data_smithi060_196073_object48318:head# 2024-09-20T22:20:23.299 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b27dd759:::benchmark_data_smithi060_196073_object33520:head# 2024-09-20T22:20:23.299 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b27f06f2:::benchmark_data_smithi060_196073_object41955:head# 2024-09-20T22:20:23.300 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:23.300 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b281377f:::benchmark_data_smithi060_196073_object32477:head# 2024-09-20T22:20:23.300 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b283d2ca:::benchmark_data_smithi060_196073_object17304:head# 2024-09-20T22:20:23.301 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b283de1f:::benchmark_data_smithi060_196073_object27980:head# 2024-09-20T22:20:23.301 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b284544b:::benchmark_data_smithi060_196073_object27029:head# 2024-09-20T22:20:23.301 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b28556f2:::benchmark_data_smithi060_196073_object14766:head# 2024-09-20T22:20:23.302 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b28625a9:::benchmark_data_smithi060_196073_object51427:head# 2024-09-20T22:20:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b287f370:::benchmark_data_smithi060_196073_object35216:head# 2024-09-20T22:20:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b28871f2:::benchmark_data_smithi060_196073_object4116:head# 2024-09-20T22:20:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b289146d:::benchmark_data_smithi060_196073_object38785:head# 2024-09-20T22:20:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b289805d:::benchmark_data_smithi060_196073_object25023:head# 2024-09-20T22:20:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b28bc710:::benchmark_data_smithi060_196073_object12811:head# 2024-09-20T22:20:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b290fa6b:::benchmark_data_smithi060_196073_object30851:head# 2024-09-20T22:20:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b293de27:::benchmark_data_smithi060_196073_object47335:head# 2024-09-20T22:20:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b294b9a3:::benchmark_data_smithi060_196073_object1080:head# 2024-09-20T22:20:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b294f445:::benchmark_data_smithi060_196073_object40876:head# 2024-09-20T22:20:23.311 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2953b6f:::benchmark_data_smithi060_196073_object14783:head# 2024-09-20T22:20:23.323 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2966440:::benchmark_data_smithi060_196073_object1396:head# 2024-09-20T22:20:23.323 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b29a103e:::benchmark_data_smithi060_196073_object9789:head# 2024-09-20T22:20:23.323 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b29aba81:::benchmark_data_smithi060_196073_object42573:head# 2024-09-20T22:20:23.323 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b29b0ad8:::benchmark_data_smithi060_196073_object29509:head# 2024-09-20T22:20:23.323 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b29c7432:::benchmark_data_smithi060_196073_object13899:head# 2024-09-20T22:20:23.323 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b29ddc72:::benchmark_data_smithi060_196073_object52185:head# 2024-09-20T22:20:23.323 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b29ea259:::benchmark_data_smithi060_196073_object4409:head# 2024-09-20T22:20:23.323 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b29f8b3f:::benchmark_data_smithi060_196073_object30608:head# 2024-09-20T22:20:23.324 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2a089fc:::benchmark_data_smithi060_196073_object24317:head# 2024-09-20T22:20:23.324 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2a1f2f9:::benchmark_data_smithi060_196073_object15571:head# 2024-09-20T22:20:23.324 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2a22189:::benchmark_data_smithi060_196073_object10863:head# 2024-09-20T22:20:23.324 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2a7166a:::benchmark_data_smithi060_196073_object60470:head# 2024-09-20T22:20:23.324 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2a731be:::benchmark_data_smithi060_196073_object46176:head# 2024-09-20T22:20:23.324 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2a9fac7:::benchmark_data_smithi060_196073_object48489:head# 2024-09-20T22:20:23.324 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2abe63e:::benchmark_data_smithi060_196073_object32904:head# 2024-09-20T22:20:23.324 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2ad61d1:::benchmark_data_smithi060_196073_object25363:head# 2024-09-20T22:20:23.326 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2b136c1:::benchmark_data_smithi060_196073_object20411:head# 2024-09-20T22:20:23.326 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2b13bb5:::benchmark_data_smithi060_196073_object19734:head# 2024-09-20T22:20:23.328 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2b1e9b0:::benchmark_data_smithi060_196073_object7787:head# 2024-09-20T22:20:23.329 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2b231be:::benchmark_data_smithi060_196073_object59830:head# 2024-09-20T22:20:23.330 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2b394cd:::benchmark_data_smithi060_196073_object14622:head# 2024-09-20T22:20:23.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2b4200e:::benchmark_data_smithi060_196073_object29482:head# 2024-09-20T22:20:23.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2b74017:::benchmark_data_smithi060_196073_object40068:head# 2024-09-20T22:20:23.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2b7676f:::benchmark_data_smithi060_196073_object44213:head# 2024-09-20T22:20:23.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2b8a694:::benchmark_data_smithi060_196073_object40914:head# 2024-09-20T22:20:23.335 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2bb27df:::benchmark_data_smithi060_196073_object33569:head# 2024-09-20T22:20:23.335 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2bc9749:::benchmark_data_smithi060_196073_object16627:head# 2024-09-20T22:20:23.338 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2be446d:::benchmark_data_smithi060_196073_object48798:head# 2024-09-20T22:20:23.338 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2bebd98:::benchmark_data_smithi060_196073_object22035:head# 2024-09-20T22:20:23.338 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2bf57bf:::benchmark_data_smithi060_196073_object1593:head# 2024-09-20T22:20:23.338 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2bfea11:::benchmark_data_smithi060_196073_object5435:head# 2024-09-20T22:20:23.339 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2c3c382:::benchmark_data_smithi060_196073_object51426:head# 2024-09-20T22:20:23.339 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2c3c544:::benchmark_data_smithi060_196073_object38470:head# 2024-09-20T22:20:23.339 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2c6c4c6:::benchmark_data_smithi060_196073_object57825:head# 2024-09-20T22:20:23.340 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2c74787:::benchmark_data_smithi060_196073_object61660:head# 2024-09-20T22:20:23.342 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2ca8fcf:::benchmark_data_smithi060_196073_object61342:head# 2024-09-20T22:20:23.342 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2cf3f90:::benchmark_data_smithi060_196073_object4555:head# 2024-09-20T22:20:23.343 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:23.344 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2cf8170:::benchmark_data_smithi060_196073_object32357:head# 2024-09-20T22:20:23.344 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2cf8936:::benchmark_data_smithi060_196073_object56358:head# 2024-09-20T22:20:23.346 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2cff83b:::benchmark_data_smithi060_196073_object58465:head# 2024-09-20T22:20:23.347 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2d13555:::benchmark_data_smithi060_196073_object46010:head# 2024-09-20T22:20:23.347 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2d2e433:::benchmark_data_smithi060_196073_object53674:head# 2024-09-20T22:20:23.348 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2d6a0d3:::benchmark_data_smithi060_196073_object6875:head# 2024-09-20T22:20:23.349 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2d6fd2b:::benchmark_data_smithi060_196073_object50022:head# 2024-09-20T22:20:23.350 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2d76032:::benchmark_data_smithi060_196073_object7084:head# 2024-09-20T22:20:23.350 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2d76146:::benchmark_data_smithi060_196073_object30547:head# 2024-09-20T22:20:23.351 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2d80060:::benchmark_data_smithi060_196073_object3728:head# 2024-09-20T22:20:23.352 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2d81877:::benchmark_data_smithi060_196073_object41032:head# 2024-09-20T22:20:23.354 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2d927df:::benchmark_data_smithi060_196073_object51718:head# 2024-09-20T22:20:23.354 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2da01b9:::benchmark_data_smithi060_196073_object56938:head# 2024-09-20T22:20:23.354 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2dcff72:::benchmark_data_smithi060_196073_object50714:head# 2024-09-20T22:20:23.355 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2dd7f4c:::benchmark_data_smithi060_196073_object54541:head# 2024-09-20T22:20:23.355 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2dfbee6:::benchmark_data_smithi060_196073_object34761:head# 2024-09-20T22:20:23.358 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2e04bef:::benchmark_data_smithi060_196073_object2705:head# 2024-09-20T22:20:23.358 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2e2939c:::benchmark_data_smithi060_196073_object16590:head# 2024-09-20T22:20:23.361 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2e43085:::benchmark_data_smithi060_196073_object38922:head# 2024-09-20T22:20:23.361 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2e65743:::benchmark_data_smithi060_196073_object35108:head# 2024-09-20T22:20:23.361 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2e6dcee:::benchmark_data_smithi060_196073_object46786:head# 2024-09-20T22:20:23.361 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2e6f39e:::benchmark_data_smithi060_196073_object36720:head# 2024-09-20T22:20:23.362 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2e71ad0:::benchmark_data_smithi060_196073_object14191:head# 2024-09-20T22:20:23.363 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2e78826:::benchmark_data_smithi060_196073_object26452:head# 2024-09-20T22:20:23.363 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2e7a4fb:::benchmark_data_smithi060_196073_object1092:head# 2024-09-20T22:20:23.365 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2e830f9:::benchmark_data_smithi060_196073_object21673:head# 2024-09-20T22:20:23.366 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2e96146:::benchmark_data_smithi060_196073_object5810:head# 2024-09-20T22:20:23.370 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2ead0f3:::benchmark_data_smithi060_196073_object34774:head# 2024-09-20T22:20:23.371 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2eb30a4:::benchmark_data_smithi060_196073_object7006:head# 2024-09-20T22:20:23.371 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2ed08d6:::benchmark_data_smithi060_196073_object9366:head# 2024-09-20T22:20:23.371 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2ed8dee:::benchmark_data_smithi060_196073_object25401:head# 2024-09-20T22:20:23.371 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2edc70b:::benchmark_data_smithi060_196073_object22861:head# 2024-09-20T22:20:23.375 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2efc947:::benchmark_data_smithi060_196073_object43470:head# 2024-09-20T22:20:23.375 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2efd5ec:::benchmark_data_smithi060_196073_object61753:head# 2024-09-20T22:20:23.375 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2f15d03:::benchmark_data_smithi060_196073_object61574:head# 2024-09-20T22:20:23.376 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2f34cad:::benchmark_data_smithi060_196073_object3320:head# 2024-09-20T22:20:23.376 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2f3c148:::benchmark_data_smithi060_196073_object40116:head# 2024-09-20T22:20:23.376 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2f3c30b:::benchmark_data_smithi060_196073_object61451:head# 2024-09-20T22:20:23.381 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2f4444a:::benchmark_data_smithi060_196073_object35173:head# 2024-09-20T22:20:23.381 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2f4619d:::benchmark_data_smithi060_196073_object30271:head# 2024-09-20T22:20:23.381 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2f4c574:::benchmark_data_smithi060_196073_object57434:head# 2024-09-20T22:20:23.381 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2f5ace5:::benchmark_data_smithi060_196073_object39279:head# 2024-09-20T22:20:23.381 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2f66d73:::benchmark_data_smithi060_196073_object25464:head# 2024-09-20T22:20:23.381 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2f69575:::benchmark_data_smithi060_196073_object45225:head# 2024-09-20T22:20:23.381 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2fbbfee:::benchmark_data_smithi060_196073_object35180:head# 2024-09-20T22:20:23.382 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2fc293a:::benchmark_data_smithi060_196073_object60239:head# 2024-09-20T22:20:23.382 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2fd36b9:::benchmark_data_smithi060_196073_object61408:head# 2024-09-20T22:20:23.382 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2fe22e4:::benchmark_data_smithi060_196073_object45259:head# 2024-09-20T22:20:23.384 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b2fe45f5:::benchmark_data_smithi060_196073_object53493:head# 2024-09-20T22:20:23.384 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b301fde5:::benchmark_data_smithi060_196073_object23020:head# 2024-09-20T22:20:23.384 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b302feec:::benchmark_data_smithi060_196073_object58771:head# 2024-09-20T22:20:23.384 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:23.385 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3036db4:::benchmark_data_smithi060_196073_object30063:head# 2024-09-20T22:20:23.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:23.386 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:23.386 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b304a5f0:::benchmark_data_smithi060_196073_object16348:head# 2024-09-20T22:20:23.392 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b306da09:::benchmark_data_smithi060_196073_object14517:head# 2024-09-20T22:20:23.392 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b308f454:::benchmark_data_smithi060_196073_object21195:head# 2024-09-20T22:20:23.392 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b30abb8f:::benchmark_data_smithi060_196073_object47523:head# 2024-09-20T22:20:23.392 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b30b8ca3:::benchmark_data_smithi060_196073_object22670:head# 2024-09-20T22:20:23.392 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b30c024d:::benchmark_data_smithi060_196073_object32192:head# 2024-09-20T22:20:23.392 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b30d0e01:::benchmark_data_smithi060_196073_object9251:head# 2024-09-20T22:20:23.392 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b30d8dfd:::benchmark_data_smithi060_196073_object47774:head# 2024-09-20T22:20:23.394 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b30e87d1:::benchmark_data_smithi060_196073_object35464:head# 2024-09-20T22:20:23.395 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b30f555a:::benchmark_data_smithi060_196073_object2735:head# 2024-09-20T22:20:23.395 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3114470:::benchmark_data_smithi060_196073_object12978:head# 2024-09-20T22:20:23.398 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b311f842:::benchmark_data_smithi060_196073_object37879:head# 2024-09-20T22:20:23.398 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3145174:::benchmark_data_smithi060_196073_object1023:head# 2024-09-20T22:20:23.398 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3193ccc:::benchmark_data_smithi060_196073_object48667:head# 2024-09-20T22:20:23.401 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b319b9b5:::benchmark_data_smithi060_196073_object24536:head# 2024-09-20T22:20:23.401 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b31a76a6:::benchmark_data_smithi060_196073_object2820:head# 2024-09-20T22:20:23.401 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b31b1ace:::benchmark_data_smithi060_196073_object20744:head# 2024-09-20T22:20:23.401 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b31b2c55:::benchmark_data_smithi060_196073_object52329:head# 2024-09-20T22:20:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b31b73b2:::benchmark_data_smithi060_196073_object46474:head# 2024-09-20T22:20:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b31ba861:::benchmark_data_smithi060_196073_object48966:head# 2024-09-20T22:20:23.402 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b31bb44b:::benchmark_data_smithi060_196073_object33444:head# 2024-09-20T22:20:23.403 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b31c3faa:::benchmark_data_smithi060_196073_object49070:head# 2024-09-20T22:20:23.403 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b31dc28b:::benchmark_data_smithi060_196073_object45623:head# 2024-09-20T22:20:23.404 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3211147:::benchmark_data_smithi060_196073_object40324:head# 2024-09-20T22:20:23.404 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3222107:::benchmark_data_smithi060_196073_object56490:head# 2024-09-20T22:20:23.405 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b322b370:::benchmark_data_smithi060_196073_object44192:head# 2024-09-20T22:20:23.405 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b322ce36:::benchmark_data_smithi060_196073_object35843:head# 2024-09-20T22:20:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b324dd74:::benchmark_data_smithi060_196073_object8369:head# 2024-09-20T22:20:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b32513ee:::benchmark_data_smithi060_196073_object20617:head# 2024-09-20T22:20:23.407 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3254927:::benchmark_data_smithi060_196073_object44572:head# 2024-09-20T22:20:23.410 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b325539d:::benchmark_data_smithi060_196073_object18165:head# 2024-09-20T22:20:23.411 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3272c75:::benchmark_data_smithi060_196073_object21367:head# 2024-09-20T22:20:23.412 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b329f168:::benchmark_data_smithi060_196073_object60297:head# 2024-09-20T22:20:23.413 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b32a69e1:::benchmark_data_smithi060_196073_object49464:head# 2024-09-20T22:20:23.414 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b32e39ff:::benchmark_data_smithi060_196073_object6496:head# 2024-09-20T22:20:23.416 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b32e831a:::benchmark_data_smithi060_196073_object46954:head# 2024-09-20T22:20:23.416 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b33182c9:::benchmark_data_smithi060_196073_object54476:head# 2024-09-20T22:20:23.420 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3328a55:::benchmark_data_smithi060_196073_object22827:head# 2024-09-20T22:20:23.420 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3333991:::benchmark_data_smithi060_196073_object408:head# 2024-09-20T22:20:23.420 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b333b1e7:::benchmark_data_smithi060_196073_object16259:head# 2024-09-20T22:20:23.420 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b33539d5:::benchmark_data_smithi060_196073_object13929:head# 2024-09-20T22:20:23.420 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3364479:::benchmark_data_smithi060_196073_object46658:head# 2024-09-20T22:20:23.425 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b33816a3:::benchmark_data_smithi060_196073_object50470:head# 2024-09-20T22:20:23.426 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b33dbae6:::benchmark_data_smithi060_196073_object38684:head# 2024-09-20T22:20:23.426 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b33e6740:::benchmark_data_smithi060_196073_object32485:head# 2024-09-20T22:20:23.426 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b33ed73d:::benchmark_data_smithi060_196073_object26847:head# 2024-09-20T22:20:23.426 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b33eea94:::benchmark_data_smithi060_196073_object54118:head# 2024-09-20T22:20:23.426 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b33f4ffc:::benchmark_data_smithi060_196073_object19744:head# 2024-09-20T22:20:23.426 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b33fb089:::benchmark_data_smithi060_196073_object59600:head# 2024-09-20T22:20:23.433 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b342e5d4:::benchmark_data_smithi060_196073_object18993:head# 2024-09-20T22:20:23.433 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b34397d5:::benchmark_data_smithi060_196073_object7549:head# 2024-09-20T22:20:23.433 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b343ec2b:::benchmark_data_smithi060_196073_object47723:head# 2024-09-20T22:20:23.433 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3457b61:::benchmark_data_smithi060_196073_object11280:head# 2024-09-20T22:20:23.433 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b34609f1:::benchmark_data_smithi060_196073_object45538:head# 2024-09-20T22:20:23.433 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b347f191:::benchmark_data_smithi060_196073_object43205:head# 2024-09-20T22:20:23.433 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3490fc6:::benchmark_data_smithi060_196073_object16685:head# 2024-09-20T22:20:23.433 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b349974b:::benchmark_data_smithi060_196073_object51292:head# 2024-09-20T22:20:23.434 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b34a1c30:::benchmark_data_smithi060_196073_object42670:head# 2024-09-20T22:20:23.439 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b34a6c2d:::benchmark_data_smithi060_196073_object15969:head# 2024-09-20T22:20:23.439 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b34c405f:::benchmark_data_smithi060_196073_object6209:head# 2024-09-20T22:20:23.439 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b34dba92:::benchmark_data_smithi060_196073_object30440:head# 2024-09-20T22:20:23.440 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b34dc7f1:::benchmark_data_smithi060_196073_object39801:head# 2024-09-20T22:20:23.440 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3505665:::benchmark_data_smithi060_196073_object23502:head# 2024-09-20T22:20:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b353dbce:::benchmark_data_smithi060_196073_object11532:head# 2024-09-20T22:20:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3566e26:::benchmark_data_smithi060_196073_object42202:head# 2024-09-20T22:20:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b356900e:::benchmark_data_smithi060_196073_object15044:head# 2024-09-20T22:20:23.448 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b356a0b0:::benchmark_data_smithi060_196073_object10826:head# 2024-09-20T22:20:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3587f08:::benchmark_data_smithi060_196073_object59208:head# 2024-09-20T22:20:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b35a2854:::benchmark_data_smithi060_196073_object29205:head# 2024-09-20T22:20:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b35a7f93:::benchmark_data_smithi060_196073_object20376:head# 2024-09-20T22:20:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b35bf006:::benchmark_data_smithi060_196073_object37099:head# 2024-09-20T22:20:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b35bfd74:::benchmark_data_smithi060_196073_object11623:head# 2024-09-20T22:20:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b35cd92f:::benchmark_data_smithi060_196073_object56214:head# 2024-09-20T22:20:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b35d5537:::benchmark_data_smithi060_196073_object11195:head# 2024-09-20T22:20:23.449 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b35fb759:::benchmark_data_smithi060_196073_object13261:head# 2024-09-20T22:20:23.454 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3600bc1:::benchmark_data_smithi060_196073_object51606:head# 2024-09-20T22:20:23.454 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b364b645:::benchmark_data_smithi060_196073_object54490:head# 2024-09-20T22:20:23.454 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b365c737:::benchmark_data_smithi060_196073_object59811:head# 2024-09-20T22:20:23.454 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3676304:::benchmark_data_smithi060_196073_object15586:head# 2024-09-20T22:20:23.454 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3677b1e:::benchmark_data_smithi060_196073_object40398:head# 2024-09-20T22:20:23.454 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3678862:::benchmark_data_smithi060_196073_object45343:head# 2024-09-20T22:20:23.454 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3679b55:::benchmark_data_smithi060_196073_object56832:head# 2024-09-20T22:20:23.457 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b368782f:::benchmark_data_smithi060_196073_object23257:head# 2024-09-20T22:20:23.460 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3692db2:::benchmark_data_smithi060_196073_object17747:head# 2024-09-20T22:20:23.461 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3695d58:::benchmark_data_smithi060_196073_object54341:head# 2024-09-20T22:20:23.461 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b36b4fa1:::benchmark_data_smithi060_196073_object5531:head# 2024-09-20T22:20:23.462 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b36bdc38:::benchmark_data_smithi060_196073_object48967:head# 2024-09-20T22:20:23.464 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b36cdb64:::benchmark_data_smithi060_196073_object12704:head# 2024-09-20T22:20:23.464 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b36f0f4c:::benchmark_data_smithi060_196073_object8171:head# 2024-09-20T22:20:23.464 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b370a175:::benchmark_data_smithi060_196073_object13045:head# 2024-09-20T22:20:23.467 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b371d383:::benchmark_data_smithi060_196073_object7011:head# 2024-09-20T22:20:23.467 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3721e98:::benchmark_data_smithi060_196073_object14484:head# 2024-09-20T22:20:23.468 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b373225e:::benchmark_data_smithi060_196073_object43566:head# 2024-09-20T22:20:23.485 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b37391d1:::benchmark_data_smithi060_196073_object42116:head# 2024-09-20T22:20:23.485 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b373f613:::benchmark_data_smithi060_196073_object13605:head# 2024-09-20T22:20:23.485 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b374fc81:::benchmark_data_smithi060_196073_object25335:head# 2024-09-20T22:20:23.485 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b376c2c4:::benchmark_data_smithi060_196073_object24154:head# 2024-09-20T22:20:23.485 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3774e2b:::benchmark_data_smithi060_196073_object49835:head# 2024-09-20T22:20:23.486 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3778523:::benchmark_data_smithi060_196073_object38319:head# 2024-09-20T22:20:23.486 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b377a09e:::benchmark_data_smithi060_196073_object23883:head# 2024-09-20T22:20:23.486 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3788946:::benchmark_data_smithi060_196073_object18214:head# 2024-09-20T22:20:23.486 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b379e91f:::benchmark_data_smithi060_196073_object38394:head# 2024-09-20T22:20:23.486 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b37a3a99:::benchmark_data_smithi060_196073_object4820:head# 2024-09-20T22:20:23.486 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b37b2b5f:::benchmark_data_smithi060_196073_object57870:head# 2024-09-20T22:20:23.486 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b37c9b68:::benchmark_data_smithi060_196073_object15463:head# 2024-09-20T22:20:23.486 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b37d433a:::benchmark_data_smithi060_196073_object3818:head# 2024-09-20T22:20:23.486 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b37e4a9f:::benchmark_data_smithi060_196073_object35702:head# 2024-09-20T22:20:23.486 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b37e8e59:::benchmark_data_smithi060_196073_object7247:head# 2024-09-20T22:20:23.486 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b37ed8e4:::benchmark_data_smithi060_196073_object8605:head# 2024-09-20T22:20:23.486 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b37fcb2b:::benchmark_data_smithi060_196073_object60373:head# 2024-09-20T22:20:23.486 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b37fdb7e:::benchmark_data_smithi060_196073_object18532:head# 2024-09-20T22:20:23.486 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b380a946:::benchmark_data_smithi060_196073_object14708:head# 2024-09-20T22:20:23.487 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3813513:::benchmark_data_smithi060_196073_object23944:head# 2024-09-20T22:20:23.487 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3820d15:::benchmark_data_smithi060_196073_object54521:head# 2024-09-20T22:20:23.487 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:23.488 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b382e64c:::benchmark_data_smithi060_196073_object58186:head# 2024-09-20T22:20:23.488 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b383521a:::benchmark_data_smithi060_196073_object25743:head# 2024-09-20T22:20:23.488 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b383527d:::benchmark_data_smithi060_196073_object13143:head# 2024-09-20T22:20:23.488 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b383b44b:::benchmark_data_smithi060_196073_object14348:head# 2024-09-20T22:20:23.489 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3841722:::benchmark_data_smithi060_196073_object8625:head# 2024-09-20T22:20:23.489 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b38664dd:::benchmark_data_smithi060_196073_object15661:head# 2024-09-20T22:20:23.489 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3883508:::benchmark_data_smithi060_196073_object26607:head# 2024-09-20T22:20:23.491 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3887809:::benchmark_data_smithi060_196073_object33881:head# 2024-09-20T22:20:23.491 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3889400:::benchmark_data_smithi060_196073_object18252:head# 2024-09-20T22:20:23.496 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b388c5b3:::benchmark_data_smithi060_196073_object23627:head# 2024-09-20T22:20:23.496 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b389ac97:::benchmark_data_smithi060_196073_object28506:head# 2024-09-20T22:20:23.496 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b389f602:::benchmark_data_smithi060_196073_object13258:head# 2024-09-20T22:20:23.496 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b38a3a89:::benchmark_data_smithi060_196073_object22211:head# 2024-09-20T22:20:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b38a5351:::benchmark_data_smithi060_196073_object36028:head# 2024-09-20T22:20:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b38c5b69:::benchmark_data_smithi060_196073_object59828:head# 2024-09-20T22:20:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b38f8b44:::benchmark_data_smithi060_196073_object17925:head# 2024-09-20T22:20:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b390b520:::benchmark_data_smithi060_196073_object17041:head# 2024-09-20T22:20:23.505 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3914467:::benchmark_data_smithi060_196073_object48739:head# 2024-09-20T22:20:23.506 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3918081:::benchmark_data_smithi060_196073_object46165:head# 2024-09-20T22:20:23.506 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b391b0c6:::benchmark_data_smithi060_196073_object28959:head# 2024-09-20T22:20:23.506 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b391f5db:::benchmark_data_smithi060_196073_object39985:head# 2024-09-20T22:20:23.506 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3924c9d:::benchmark_data_smithi060_196073_object3979:head# 2024-09-20T22:20:23.506 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3928a34:::benchmark_data_smithi060_196073_object2071:head# 2024-09-20T22:20:23.506 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b392b637:::benchmark_data_smithi060_196073_object39787:head# 2024-09-20T22:20:23.507 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3959f96:::benchmark_data_smithi060_196073_object43:head# 2024-09-20T22:20:23.507 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b396ba04:::benchmark_data_smithi060_196073_object638:head# 2024-09-20T22:20:23.508 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3972deb:::benchmark_data_smithi060_196073_object49371:head# 2024-09-20T22:20:23.509 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b397c237:::benchmark_data_smithi060_196073_object27782:head# 2024-09-20T22:20:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b39823f2:::benchmark_data_smithi060_196073_object657:head# 2024-09-20T22:20:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b399c574:::benchmark_data_smithi060_196073_object61207:head# 2024-09-20T22:20:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b39a6460:::benchmark_data_smithi060_196073_object42982:head# 2024-09-20T22:20:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b39bb063:::benchmark_data_smithi060_196073_object21581:head# 2024-09-20T22:20:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b39e8960:::benchmark_data_smithi060_196073_object2393:head# 2024-09-20T22:20:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b39f70d9:::benchmark_data_smithi060_196073_object34227:head# 2024-09-20T22:20:23.514 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b39f8ba0:::benchmark_data_smithi060_196073_object38173:head# 2024-09-20T22:20:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3a00005:::benchmark_data_smithi060_196073_object53980:head# 2024-09-20T22:20:23.515 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3a1dac5:::benchmark_data_smithi060_196073_object55062:head# 2024-09-20T22:20:23.516 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3a22afb:::benchmark_data_smithi060_196073_object29960:head# 2024-09-20T22:20:23.516 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3a24dcf:::benchmark_data_smithi060_196073_object46966:head# 2024-09-20T22:20:23.517 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3a2803c:::benchmark_data_smithi060_196073_object31224:head# 2024-09-20T22:20:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3a376a7:::benchmark_data_smithi060_196073_object22215:head# 2024-09-20T22:20:23.518 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3a47585:::benchmark_data_smithi060_196073_object50134:head# 2024-09-20T22:20:23.519 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3a5e10a:::benchmark_data_smithi060_196073_object31182:head# 2024-09-20T22:20:23.520 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3a60a81:::benchmark_data_smithi060_196073_object5364:head# 2024-09-20T22:20:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3a63705:::benchmark_data_smithi060_196073_object52418:head# 2024-09-20T22:20:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3a6fe06:::benchmark_data_smithi060_196073_object12219:head# 2024-09-20T22:20:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3a74e85:::benchmark_data_smithi060_196073_object52437:head# 2024-09-20T22:20:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3aa74c5:::benchmark_data_smithi060_196073_object22173:head# 2024-09-20T22:20:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3ae2cc0:::benchmark_data_smithi060_196073_object47825:head# 2024-09-20T22:20:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3af654f:::benchmark_data_smithi060_196073_object48655:head# 2024-09-20T22:20:23.525 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3b08303:::benchmark_data_smithi060_196073_object34351:head# 2024-09-20T22:20:23.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:23.527 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:23.528 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:23.534 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3b2bf6f:::benchmark_data_smithi060_196073_object36624:head# 2024-09-20T22:20:23.534 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3b36532:::benchmark_data_smithi060_196073_object38096:head# 2024-09-20T22:20:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3b478ad:::benchmark_data_smithi060_196073_object55442:head# 2024-09-20T22:20:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3b74619:::benchmark_data_smithi060_196073_object1937:head# 2024-09-20T22:20:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3b76747:::benchmark_data_smithi060_196073_object14214:head# 2024-09-20T22:20:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3b856c8:::benchmark_data_smithi060_196073_object19323:head# 2024-09-20T22:20:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3b93ce1:::benchmark_data_smithi060_196073_object49058:head# 2024-09-20T22:20:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3b95372:::benchmark_data_smithi060_196073_object12749:head# 2024-09-20T22:20:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3b95b52:::benchmark_data_smithi060_196073_object48647:head# 2024-09-20T22:20:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3b9f0c3:::benchmark_data_smithi060_196073_object20904:head# 2024-09-20T22:20:23.535 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3bb54fd:::benchmark_data_smithi060_196073_object23057:head# 2024-09-20T22:20:23.537 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3bd9154:::benchmark_data_smithi060_196073_object1944:head# 2024-09-20T22:20:23.537 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3be4271:::benchmark_data_smithi060_196073_object24030:head# 2024-09-20T22:20:23.547 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3bf5037:::benchmark_data_smithi060_196073_object61000:head# 2024-09-20T22:20:23.547 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3bfa141:::benchmark_data_smithi060_196073_object32245:head# 2024-09-20T22:20:23.547 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3c01b16:::benchmark_data_smithi060_196073_object45464:head# 2024-09-20T22:20:23.547 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3c271e6:::benchmark_data_smithi060_196073_object52291:head# 2024-09-20T22:20:23.547 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3c27d95:::benchmark_data_smithi060_196073_object55244:head# 2024-09-20T22:20:23.548 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3c3e680:::benchmark_data_smithi060_196073_object54149:head# 2024-09-20T22:20:23.548 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3c68ff7:::benchmark_data_smithi060_196073_object1440:head# 2024-09-20T22:20:23.548 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3c7449a:::benchmark_data_smithi060_196073_object21181:head# 2024-09-20T22:20:23.548 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3c764ed:::benchmark_data_smithi060_196073_object54815:head# 2024-09-20T22:20:23.548 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3c9f541:::benchmark_data_smithi060_196073_object52684:head# 2024-09-20T22:20:23.548 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3cab9bf:::benchmark_data_smithi060_196073_object27630:head# 2024-09-20T22:20:23.548 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3caccaf:::benchmark_data_smithi060_196073_object57735:head# 2024-09-20T22:20:23.548 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3cad36a:::benchmark_data_smithi060_196073_object26612:head# 2024-09-20T22:20:23.548 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3ccae71:::benchmark_data_smithi060_196073_object14821:head# 2024-09-20T22:20:23.548 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3ce1b8a:::benchmark_data_smithi060_196073_object20265:head# 2024-09-20T22:20:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3cea88d:::benchmark_data_smithi060_196073_object50107:head# 2024-09-20T22:20:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3cf0d2a:::benchmark_data_smithi060_196073_object61450:head# 2024-09-20T22:20:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3cf5424:::benchmark_data_smithi060_196073_object36838:head# 2024-09-20T22:20:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3cfd552:::benchmark_data_smithi060_196073_object21434:head# 2024-09-20T22:20:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3d10e0d:::benchmark_data_smithi060_196073_object44706:head# 2024-09-20T22:20:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3d17350:::benchmark_data_smithi060_196073_object37621:head# 2024-09-20T22:20:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3d20b18:::benchmark_data_smithi060_196073_object12329:head# 2024-09-20T22:20:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3d24c36:::benchmark_data_smithi060_196073_object8429:head# 2024-09-20T22:20:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3d50a52:::benchmark_data_smithi060_196073_object36356:head# 2024-09-20T22:20:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3d63d7c:::benchmark_data_smithi060_196073_object19944:head# 2024-09-20T22:20:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3d87478:::benchmark_data_smithi060_196073_object46554:head# 2024-09-20T22:20:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3d9b5df:::benchmark_data_smithi060_196073_object54837:head# 2024-09-20T22:20:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3da2142:::benchmark_data_smithi060_196073_object4729:head# 2024-09-20T22:20:23.559 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3dacfa7:::benchmark_data_smithi060_196073_object5619:head# 2024-09-20T22:20:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3ddba48:::benchmark_data_smithi060_196073_object12415:head# 2024-09-20T22:20:23.560 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3ddeb51:::benchmark_data_smithi060_196073_object40866:head# 2024-09-20T22:20:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3e06814:::benchmark_data_smithi060_196073_object48419:head# 2024-09-20T22:20:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3e29a3c:::benchmark_data_smithi060_196073_object43446:head# 2024-09-20T22:20:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3e2bd48:::benchmark_data_smithi060_196073_object60935:head# 2024-09-20T22:20:23.565 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3e5b1f9:::benchmark_data_smithi060_196073_object2064:head# 2024-09-20T22:20:23.566 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3e5cf7e:::benchmark_data_smithi060_196073_object47729:head# 2024-09-20T22:20:23.567 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:23 smithi067 ceph-mon[33511]: pgmap v934: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 170 active+clean; 6.1 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1.0 KiB/s rd, 53 MiB/s wr, 842 op/s; 9121/299907 objects degraded (3.041%); 23002/299907 objects misplaced (7.670%); 23 MiB/s, 368 objects/s recovering 2024-09-20T22:20:23.569 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3e76c95:::benchmark_data_smithi060_196073_object57663:head# 2024-09-20T22:20:23.569 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3ec7baf:::benchmark_data_smithi060_196073_object35461:head# 2024-09-20T22:20:23.571 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3ecb36d:::benchmark_data_smithi060_196073_object59644:head# 2024-09-20T22:20:23.571 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3ecd474:::benchmark_data_smithi060_196073_object8320:head# 2024-09-20T22:20:23.574 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3edd006:::benchmark_data_smithi060_196073_object37309:head# 2024-09-20T22:20:23.576 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3f0fec7:::benchmark_data_smithi060_196073_object5775:head# 2024-09-20T22:20:23.576 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3f1763e:::benchmark_data_smithi060_196073_object59263:head# 2024-09-20T22:20:23.577 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3f2a13c:::benchmark_data_smithi060_196073_object54069:head# 2024-09-20T22:20:23.587 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3f30210:::benchmark_data_smithi060_196073_object28297:head# 2024-09-20T22:20:23.587 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3f34092:::benchmark_data_smithi060_196073_object2657:head# 2024-09-20T22:20:23.587 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3f375af:::benchmark_data_smithi060_196073_object57254:head# 2024-09-20T22:20:23.587 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3f3ec9d:::benchmark_data_smithi060_196073_object16189:head# 2024-09-20T22:20:23.587 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3f46978:::benchmark_data_smithi060_196073_object52575:head# 2024-09-20T22:20:23.587 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3f66b17:::benchmark_data_smithi060_196073_object26628:head# 2024-09-20T22:20:23.588 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3f68623:::benchmark_data_smithi060_196073_object920:head# 2024-09-20T22:20:23.588 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3f7172b:::benchmark_data_smithi060_196073_object26665:head# 2024-09-20T22:20:23.588 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3f7abc8:::benchmark_data_smithi060_196073_object45649:head# 2024-09-20T22:20:23.588 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3f7f456:::benchmark_data_smithi060_196073_object47095:head# 2024-09-20T22:20:23.588 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3f876d3:::benchmark_data_smithi060_196073_object47019:head# 2024-09-20T22:20:23.588 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3f97ebe:::benchmark_data_smithi060_196073_object21218:head# 2024-09-20T22:20:23.588 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3fa2cff:::benchmark_data_smithi060_196073_object16017:head# 2024-09-20T22:20:23.594 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3fa3af9:::benchmark_data_smithi060_196073_object58498:head# 2024-09-20T22:20:23.594 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3fb2e71:::benchmark_data_smithi060_196073_object6670:head# 2024-09-20T22:20:23.594 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3fc3d91:::benchmark_data_smithi060_196073_object54374:head# 2024-09-20T22:20:23.594 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3fc956d:::benchmark_data_smithi060_196073_object48010:head# 2024-09-20T22:20:23.594 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3fe164f:::benchmark_data_smithi060_196073_object37779:head# 2024-09-20T22:20:23.594 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3fe8558:::benchmark_data_smithi060_196073_object9675:head# 2024-09-20T22:20:23.594 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b3ff4969:::benchmark_data_smithi060_196073_object28645:head# 2024-09-20T22:20:23.599 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4035c9e:::benchmark_data_smithi060_196073_object19774:head# 2024-09-20T22:20:23.599 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b40addcb:::benchmark_data_smithi060_196073_object17846:head# 2024-09-20T22:20:23.599 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b40df169:::benchmark_data_smithi060_196073_object8422:head# 2024-09-20T22:20:23.600 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b40e4cf8:::benchmark_data_smithi060_196073_object50683:head# 2024-09-20T22:20:23.600 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b40ec4a8:::benchmark_data_smithi060_196073_object43810:head# 2024-09-20T22:20:23.600 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b40f405f:::benchmark_data_smithi060_196073_object15687:head# 2024-09-20T22:20:23.600 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b41105d4:::benchmark_data_smithi060_196073_object58571:head# 2024-09-20T22:20:23.600 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b41285d9:::benchmark_data_smithi060_196073_object58725:head# 2024-09-20T22:20:23.600 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b413181b:::benchmark_data_smithi060_196073_object22170:head# 2024-09-20T22:20:23.602 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4138111:::benchmark_data_smithi060_196073_object41645:head# 2024-09-20T22:20:23.602 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b413a9ef:::benchmark_data_smithi060_196073_object51231:head# 2024-09-20T22:20:23.602 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b413b915:::benchmark_data_smithi060_196073_object29499:head# 2024-09-20T22:20:23.602 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b414189c:::benchmark_data_smithi060_196073_object53069:head# 2024-09-20T22:20:23.602 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4180775:::benchmark_data_smithi060_196073_object37837:head# 2024-09-20T22:20:23.602 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b41981e0:::benchmark_data_smithi060_196073_object53516:head# 2024-09-20T22:20:23.605 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b41b9d8e:::benchmark_data_smithi060_196073_object25461:head# 2024-09-20T22:20:23.605 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b41c194c:::benchmark_data_smithi060_196073_object24814:head# 2024-09-20T22:20:23.606 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b41d9a05:::benchmark_data_smithi060_196073_object57118:head# 2024-09-20T22:20:23.607 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4205ba6:::benchmark_data_smithi060_196073_object2065:head# 2024-09-20T22:20:23.608 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4209579:::benchmark_data_smithi060_196073_object59171:head# 2024-09-20T22:20:23.608 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b421a68d:::benchmark_data_smithi060_196073_object55237:head# 2024-09-20T22:20:23.609 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b421f2fa:::benchmark_data_smithi060_196073_object53192:head# 2024-09-20T22:20:23.610 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4245d80:::benchmark_data_smithi060_196073_object23836:head# 2024-09-20T22:20:23.612 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4252baf:::benchmark_data_smithi060_196073_object22662:head# 2024-09-20T22:20:23.612 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b428ead2:::benchmark_data_smithi060_196073_object61142:head# 2024-09-20T22:20:23.618 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b42a2006:::benchmark_data_smithi060_196073_object18076:head# 2024-09-20T22:20:23.618 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b42a6de8:::benchmark_data_smithi060_196073_object10149:head# 2024-09-20T22:20:23.618 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b42dac0e:::benchmark_data_smithi060_196073_object58637:head# 2024-09-20T22:20:23.618 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b42fc3cc:::benchmark_data_smithi060_196073_object49672:head# 2024-09-20T22:20:23.618 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b430a5de:::benchmark_data_smithi060_196073_object30056:head# 2024-09-20T22:20:23.618 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b43135a4:::benchmark_data_smithi060_196073_object20810:head# 2024-09-20T22:20:23.618 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b43288b2:::benchmark_data_smithi060_196073_object11885:head# 2024-09-20T22:20:23.618 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b432c92b:::benchmark_data_smithi060_196073_object46659:head# 2024-09-20T22:20:23.619 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4338bf3:::benchmark_data_smithi060_196073_object15620:head# 2024-09-20T22:20:23.624 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b434ea84:::benchmark_data_smithi060_196073_object49484:head# 2024-09-20T22:20:23.624 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b43513db:::benchmark_data_smithi060_196073_object52286:head# 2024-09-20T22:20:23.624 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4358778:::benchmark_data_smithi060_196073_object59039:head# 2024-09-20T22:20:23.624 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b43597d0:::benchmark_data_smithi060_196073_object60762:head# 2024-09-20T22:20:23.625 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b435c606:::benchmark_data_smithi060_196073_object1674:head# 2024-09-20T22:20:23.625 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b436d045:::benchmark_data_smithi060_196073_object18706:head# 2024-09-20T22:20:23.625 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4394c5f:::benchmark_data_smithi060_196073_object16909:head# 2024-09-20T22:20:23.625 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b43aa4f4:::benchmark_data_smithi060_196073_object36216:head# 2024-09-20T22:20:23.626 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b43aa553:::benchmark_data_smithi060_196073_object40738:head# 2024-09-20T22:20:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b43ac953:::benchmark_data_smithi060_196073_object39657:head# 2024-09-20T22:20:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b43be871:::benchmark_data_smithi060_196073_object40199:head# 2024-09-20T22:20:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b43da762:::benchmark_data_smithi060_196073_object39957:head# 2024-09-20T22:20:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b43daa9d:::benchmark_data_smithi060_196073_object60372:head# 2024-09-20T22:20:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b43e3050:::benchmark_data_smithi060_196073_object26853:head# 2024-09-20T22:20:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b43ecfda:::benchmark_data_smithi060_196073_object53916:head# 2024-09-20T22:20:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b43ef5e7:::benchmark_data_smithi060_196073_object1876:head# 2024-09-20T22:20:23.632 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b440a0d9:::benchmark_data_smithi060_196073_object45125:head# 2024-09-20T22:20:23.632 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:23.643 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4411057:::benchmark_data_smithi060_196073_object3675:head# 2024-09-20T22:20:23.643 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b442bd62:::benchmark_data_smithi060_196073_object44900:head# 2024-09-20T22:20:23.643 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b443ead0:::benchmark_data_smithi060_196073_object47054:head# 2024-09-20T22:20:23.643 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b443f4b1:::benchmark_data_smithi060_196073_object30725:head# 2024-09-20T22:20:23.644 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b444edbb:::benchmark_data_smithi060_196073_object55180:head# 2024-09-20T22:20:23.644 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4453ac3:::benchmark_data_smithi060_196073_object36378:head# 2024-09-20T22:20:23.644 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4488b89:::benchmark_data_smithi060_196073_object8769:head# 2024-09-20T22:20:23.644 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b448bcf3:::benchmark_data_smithi060_196073_object26948:head# 2024-09-20T22:20:23.644 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b448d214:::benchmark_data_smithi060_196073_object40412:head# 2024-09-20T22:20:23.644 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b44966d7:::benchmark_data_smithi060_196073_object24518:head# 2024-09-20T22:20:23.644 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b44a6018:::benchmark_data_smithi060_196073_object29208:head# 2024-09-20T22:20:23.644 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b44ab390:::benchmark_data_smithi060_196073_object37822:head# 2024-09-20T22:20:23.644 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b44ae9f9:::benchmark_data_smithi060_196073_object6460:head# 2024-09-20T22:20:23.644 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b44d3b0c:::benchmark_data_smithi060_196073_object32268:head# 2024-09-20T22:20:23.644 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b44dbc46:::benchmark_data_smithi060_196073_object20979:head# 2024-09-20T22:20:23.650 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b44de48a:::benchmark_data_smithi060_196073_object27779:head# 2024-09-20T22:20:23.650 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b44e8fab:::benchmark_data_smithi060_196073_object51119:head# 2024-09-20T22:20:23.650 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b44ef4cc:::benchmark_data_smithi060_196073_object44653:head# 2024-09-20T22:20:23.650 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b44fed18:::benchmark_data_smithi060_196073_object39213:head# 2024-09-20T22:20:23.650 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4506b9d:::benchmark_data_smithi060_196073_object6828:head# 2024-09-20T22:20:23.650 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b45101bc:::benchmark_data_smithi060_196073_object1406:head# 2024-09-20T22:20:23.650 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4531313:::benchmark_data_smithi060_196073_object28899:head# 2024-09-20T22:20:23.650 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b455bbfa:::benchmark_data_smithi060_196073_object5025:head# 2024-09-20T22:20:23.652 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b455c40e:::benchmark_data_smithi060_196073_object35628:head# 2024-09-20T22:20:23.654 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4566595:::benchmark_data_smithi060_196073_object28322:head# 2024-09-20T22:20:23.656 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b456b463:::benchmark_data_smithi060_196073_object27792:head# 2024-09-20T22:20:23.656 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b456c47c:::benchmark_data_smithi060_196073_object14728:head# 2024-09-20T22:20:23.656 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b458275b:::benchmark_data_smithi060_196073_object2496:head# 2024-09-20T22:20:23.657 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4583d2e:::benchmark_data_smithi060_196073_object17900:head# 2024-09-20T22:20:23.657 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b45955a8:::benchmark_data_smithi060_196073_object60576:head# 2024-09-20T22:20:23.658 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b45f192f:::benchmark_data_smithi060_196073_object40974:head# 2024-09-20T22:20:23.658 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b45fa64f:::benchmark_data_smithi060_196073_object33811:head# 2024-09-20T22:20:23.659 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b45ff681:::benchmark_data_smithi060_196073_object49309:head# 2024-09-20T22:20:23.662 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4608d82:::benchmark_data_smithi060_196073_object7669:head# 2024-09-20T22:20:23.662 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b460a72a:::benchmark_data_smithi060_196073_object56059:head# 2024-09-20T22:20:23.663 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b461e67d:::benchmark_data_smithi060_196073_object23762:head# 2024-09-20T22:20:23.663 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4620d6f:::benchmark_data_smithi060_196073_object22583:head# 2024-09-20T22:20:23.663 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b463af24:::benchmark_data_smithi060_196073_object21485:head# 2024-09-20T22:20:23.667 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b46440b9:::benchmark_data_smithi060_196073_object18136:head# 2024-09-20T22:20:23.667 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b465761b:::benchmark_data_smithi060_196073_object34731:head# 2024-09-20T22:20:23.667 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4659782:::benchmark_data_smithi060_196073_object33943:head# 2024-09-20T22:20:23.667 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b46787ab:::benchmark_data_smithi060_196073_object7194:head# 2024-09-20T22:20:23.667 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b468f66a:::benchmark_data_smithi060_196073_object2466:head# 2024-09-20T22:20:23.667 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b46979cf:::benchmark_data_smithi060_196073_object20686:head# 2024-09-20T22:20:23.667 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b46997a6:::benchmark_data_smithi060_196073_object15011:head# 2024-09-20T22:20:23.668 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:23.668 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b46a006c:::benchmark_data_smithi060_196073_object36280:head# 2024-09-20T22:20:23.668 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b46a1a6a:::benchmark_data_smithi060_196073_object50152:head# 2024-09-20T22:20:23.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:23.669 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:23.669 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b46c1387:::benchmark_data_smithi060_196073_object54907:head# 2024-09-20T22:20:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b46dfd30:::benchmark_data_smithi060_196073_object29929:head# 2024-09-20T22:20:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b46e1098:::benchmark_data_smithi060_196073_object19017:head# 2024-09-20T22:20:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b47013b5:::benchmark_data_smithi060_196073_object39585:head# 2024-09-20T22:20:23.670 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4724d02:::benchmark_data_smithi060_196073_object35079:head# 2024-09-20T22:20:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b47310aa:::benchmark_data_smithi060_196073_object61589:head# 2024-09-20T22:20:23.678 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b474d95d:::benchmark_data_smithi060_196073_object53974:head# 2024-09-20T22:20:23.679 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b474e93b:::benchmark_data_smithi060_196073_object59696:head# 2024-09-20T22:20:23.679 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b47951b0:::benchmark_data_smithi060_196073_object10858:head# 2024-09-20T22:20:23.679 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b47adc56:::benchmark_data_smithi060_196073_object3852:head# 2024-09-20T22:20:23.679 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b47c6b27:::benchmark_data_smithi060_196073_object9579:head# 2024-09-20T22:20:23.679 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b47dca2d:::benchmark_data_smithi060_196073_object54488:head# 2024-09-20T22:20:23.679 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b47e512f:::benchmark_data_smithi060_196073_object48039:head# 2024-09-20T22:20:23.679 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b47f373f:::benchmark_data_smithi060_196073_object39874:head# 2024-09-20T22:20:23.679 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b47fbd40:::benchmark_data_smithi060_196073_object45199:head# 2024-09-20T22:20:23.679 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4859679:::benchmark_data_smithi060_196073_object56037:head# 2024-09-20T22:20:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b485c997:::benchmark_data_smithi060_196073_object44344:head# 2024-09-20T22:20:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b48a895d:::benchmark_data_smithi060_196073_object13372:head# 2024-09-20T22:20:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b48c324e:::benchmark_data_smithi060_196073_object58720:head# 2024-09-20T22:20:23.681 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b48d3468:::benchmark_data_smithi060_196073_object11899:head# 2024-09-20T22:20:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b48f468c:::benchmark_data_smithi060_196073_object53023:head# 2024-09-20T22:20:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4902e9f:::benchmark_data_smithi060_196073_object43046:head# 2024-09-20T22:20:23.682 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b490572d:::benchmark_data_smithi060_196073_object45697:head# 2024-09-20T22:20:23.683 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b491b6b7:::benchmark_data_smithi060_196073_object7211:head# 2024-09-20T22:20:23.684 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b491bfad:::benchmark_data_smithi060_196073_object46394:head# 2024-09-20T22:20:23.684 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4929c02:::benchmark_data_smithi060_196073_object32894:head# 2024-09-20T22:20:23.685 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b492b41e:::benchmark_data_smithi060_196073_object16979:head# 2024-09-20T22:20:23.685 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b49450fa:::benchmark_data_smithi060_196073_object58087:head# 2024-09-20T22:20:23.686 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4959d2f:::benchmark_data_smithi060_196073_object6763:head# 2024-09-20T22:20:23.686 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b495c85c:::benchmark_data_smithi060_196073_object44928:head# 2024-09-20T22:20:23.687 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4962902:::benchmark_data_smithi060_196073_object21216:head# 2024-09-20T22:20:23.687 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4971397:::benchmark_data_smithi060_196073_object5199:head# 2024-09-20T22:20:23.690 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b49a5660:::benchmark_data_smithi060_196073_object32581:head# 2024-09-20T22:20:23.696 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b49c3a38:::benchmark_data_smithi060_196073_object1319:head# 2024-09-20T22:20:23.696 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b49e1762:::benchmark_data_smithi060_196073_object40197:head# 2024-09-20T22:20:23.696 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b49f828b:::benchmark_data_smithi060_196073_object43158:head# 2024-09-20T22:20:23.697 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b49f91e1:::benchmark_data_smithi060_196073_object61603:head# 2024-09-20T22:20:23.697 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b49f9c13:::benchmark_data_smithi060_196073_object23920:head# 2024-09-20T22:20:23.697 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4a17749:::benchmark_data_smithi060_196073_object11091:head# 2024-09-20T22:20:23.697 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4a37512:::benchmark_data_smithi060_196073_object35199:head# 2024-09-20T22:20:23.697 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4a4191f:::benchmark_data_smithi060_196073_object32284:head# 2024-09-20T22:20:23.700 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4a4ac2d:::benchmark_data_smithi060_196073_object31183:head# 2024-09-20T22:20:23.700 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4a4ca59:::benchmark_data_smithi060_196073_object45843:head# 2024-09-20T22:20:23.705 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4a68535:::benchmark_data_smithi060_196073_object16012:head# 2024-09-20T22:20:23.705 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4a6911d:::benchmark_data_smithi060_196073_object40547:head# 2024-09-20T22:20:23.705 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4a6b508:::benchmark_data_smithi060_196073_object37593:head# 2024-09-20T22:20:23.706 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4a81f50:::benchmark_data_smithi060_196073_object31464:head# 2024-09-20T22:20:23.714 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4a8c29c:::benchmark_data_smithi060_196073_object16478:head# 2024-09-20T22:20:23.714 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4a93006:::benchmark_data_smithi060_196073_object47292:head# 2024-09-20T22:20:23.714 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4a9e113:::benchmark_data_smithi060_196073_object18017:head# 2024-09-20T22:20:23.714 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4ac0213:::benchmark_data_smithi060_196073_object18639:head# 2024-09-20T22:20:23.714 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4ac7167:::benchmark_data_smithi060_196073_object49287:head# 2024-09-20T22:20:23.714 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4ace764:::benchmark_data_smithi060_196073_object16457:head# 2024-09-20T22:20:23.714 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4b06d57:::benchmark_data_smithi060_196073_object39141:head# 2024-09-20T22:20:23.715 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4b0cdaa:::benchmark_data_smithi060_196073_object9924:head# 2024-09-20T22:20:23.718 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4b39976:::benchmark_data_smithi060_196073_object16325:head# 2024-09-20T22:20:23.718 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4b3abfc:::benchmark_data_smithi060_196073_object49649:head# 2024-09-20T22:20:23.718 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4b3b27e:::benchmark_data_smithi060_196073_object49727:head# 2024-09-20T22:20:23.725 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:23 smithi060 ceph-mon[29694]: pgmap v934: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 170 active+clean; 6.1 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1.0 KiB/s rd, 53 MiB/s wr, 842 op/s; 9121/299907 objects degraded (3.041%); 23002/299907 objects misplaced (7.670%); 23 MiB/s, 368 objects/s recovering 2024-09-20T22:20:23.725 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:23 smithi060 ceph-mon[40290]: pgmap v934: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 170 active+clean; 6.1 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1.0 KiB/s rd, 53 MiB/s wr, 842 op/s; 9121/299907 objects degraded (3.041%); 23002/299907 objects misplaced (7.670%); 23 MiB/s, 368 objects/s recovering 2024-09-20T22:20:23.728 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4b4e0cc:::benchmark_data_smithi060_196073_object2706:head# 2024-09-20T22:20:23.728 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4b8edc9:::benchmark_data_smithi060_196073_object36054:head# 2024-09-20T22:20:23.728 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4b98c87:::benchmark_data_smithi060_196073_object29237:head# 2024-09-20T22:20:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4ba89da:::benchmark_data_smithi060_196073_object48299:head# 2024-09-20T22:20:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4bafe87:::benchmark_data_smithi060_196073_object53127:head# 2024-09-20T22:20:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4bbcbf0:::benchmark_data_smithi060_196073_object36416:head# 2024-09-20T22:20:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4bdbcc3:::benchmark_data_smithi060_196073_object18506:head# 2024-09-20T22:20:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4be0c6e:::benchmark_data_smithi060_196073_object34582:head# 2024-09-20T22:20:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4c066ed:::benchmark_data_smithi060_196073_object15799:head# 2024-09-20T22:20:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4c1918e:::benchmark_data_smithi060_196073_object40481:head# 2024-09-20T22:20:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4c2692b:::benchmark_data_smithi060_196073_object26523:head# 2024-09-20T22:20:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4c279c4:::benchmark_data_smithi060_196073_object7999:head# 2024-09-20T22:20:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4c29a67:::benchmark_data_smithi060_196073_object61245:head# 2024-09-20T22:20:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4c4be6e:::benchmark_data_smithi060_196073_object39483:head# 2024-09-20T22:20:23.729 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4c56d06:::benchmark_data_smithi060_196073_object20193:head# 2024-09-20T22:20:23.731 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4c6c0a4:::benchmark_data_smithi060_196073_object27405:head# 2024-09-20T22:20:23.731 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4c76bf8:::benchmark_data_smithi060_196073_object53246:head# 2024-09-20T22:20:23.731 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4c80a79:::benchmark_data_smithi060_196073_object33972:head# 2024-09-20T22:20:23.731 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4c948c5:::benchmark_data_smithi060_196073_object57942:head# 2024-09-20T22:20:23.732 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4c9a0dd:::benchmark_data_smithi060_196073_object27883:head# 2024-09-20T22:20:23.732 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4ca6ae5:::benchmark_data_smithi060_196073_object208:head# 2024-09-20T22:20:23.732 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4cb2827:::benchmark_data_smithi060_196073_object44049:head# 2024-09-20T22:20:23.733 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4cd78d9:::benchmark_data_smithi060_196073_object32238:head# 2024-09-20T22:20:23.736 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4cead35:::benchmark_data_smithi060_196073_object54791:head# 2024-09-20T22:20:23.736 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4cfde61:::benchmark_data_smithi060_196073_object9878:head# 2024-09-20T22:20:23.736 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4d234b0:::benchmark_data_smithi060_196073_object33097:head# 2024-09-20T22:20:23.736 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4d24881:::benchmark_data_smithi060_196073_object21892:head# 2024-09-20T22:20:23.736 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4d39b45:::benchmark_data_smithi060_196073_object25798:head# 2024-09-20T22:20:23.742 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4d690de:::benchmark_data_smithi060_196073_object40358:head# 2024-09-20T22:20:23.743 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4d6bb3c:::benchmark_data_smithi060_196073_object14287:head# 2024-09-20T22:20:23.743 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4d92608:::benchmark_data_smithi060_196073_object4268:head# 2024-09-20T22:20:23.743 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4d98dae:::benchmark_data_smithi060_196073_object48086:head# 2024-09-20T22:20:23.743 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4da5a7a:::benchmark_data_smithi060_196073_object8978:head# 2024-09-20T22:20:23.743 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4db0680:::benchmark_data_smithi060_196073_object9167:head# 2024-09-20T22:20:23.743 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4dbe494:::benchmark_data_smithi060_196073_object3016:head# 2024-09-20T22:20:23.743 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4dc0151:::benchmark_data_smithi060_196073_object8452:head# 2024-09-20T22:20:23.747 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4dcae66:::benchmark_data_smithi060_196073_object53751:head# 2024-09-20T22:20:23.747 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4ddb205:::benchmark_data_smithi060_196073_object32505:head# 2024-09-20T22:20:23.747 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4e01048:::benchmark_data_smithi060_196073_object31056:head# 2024-09-20T22:20:23.747 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4e1ea08:::benchmark_data_smithi060_196073_object18558:head# 2024-09-20T22:20:23.747 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4e1eae5:::benchmark_data_smithi060_196073_object33985:head# 2024-09-20T22:20:23.747 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4e2b3b8:::benchmark_data_smithi060_196073_object57328:head# 2024-09-20T22:20:23.748 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4e5526d:::benchmark_data_smithi060_196073_object55510:head# 2024-09-20T22:20:23.748 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4e896a1:::benchmark_data_smithi060_196073_object5059:head# 2024-09-20T22:20:23.749 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4e8a6cb:::benchmark_data_smithi060_196073_object38616:head# 2024-09-20T22:20:23.751 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4eaa75b:::benchmark_data_smithi060_196073_object11424:head# 2024-09-20T22:20:23.752 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4ee0350:::benchmark_data_smithi060_196073_object56729:head# 2024-09-20T22:20:23.752 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4ef1232:::benchmark_data_smithi060_196073_object456:head# 2024-09-20T22:20:23.753 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4ef34dc:::benchmark_data_smithi060_196073_object6628:head# 2024-09-20T22:20:23.756 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4f162a7:::benchmark_data_smithi060_196073_object55160:head# 2024-09-20T22:20:23.756 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4f4a3a7:::benchmark_data_smithi060_196073_object54665:head# 2024-09-20T22:20:23.766 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4f632e1:::benchmark_data_smithi060_196073_object14300:head# 2024-09-20T22:20:23.766 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4f65e77:::benchmark_data_smithi060_196073_object47671:head# 2024-09-20T22:20:23.766 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4f684f7:::benchmark_data_smithi060_196073_object43159:head# 2024-09-20T22:20:23.766 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4fabc3d:::benchmark_data_smithi060_196073_object61587:head# 2024-09-20T22:20:23.766 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4fbcc1b:::benchmark_data_smithi060_196073_object6492:head# 2024-09-20T22:20:23.766 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4fc5591:::benchmark_data_smithi060_196073_object55224:head# 2024-09-20T22:20:23.766 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4fea836:::benchmark_data_smithi060_196073_object55289:head# 2024-09-20T22:20:23.767 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4ff20e9:::benchmark_data_smithi060_196073_object2716:head# 2024-09-20T22:20:23.767 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b4ffb4e6:::benchmark_data_smithi060_196073_object7266:head# 2024-09-20T22:20:23.767 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b502b818:::benchmark_data_smithi060_196073_object56796:head# 2024-09-20T22:20:23.767 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b50448bd:::benchmark_data_smithi060_196073_object31770:head# 2024-09-20T22:20:23.767 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b50512ef:::benchmark_data_smithi060_196073_object21675:head# 2024-09-20T22:20:23.768 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b50518de:::benchmark_data_smithi060_196073_object49415:head# 2024-09-20T22:20:23.769 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5054b67:::benchmark_data_smithi060_196073_object31639:head# 2024-09-20T22:20:23.770 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:23.771 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5060808:::benchmark_data_smithi060_196073_object4967:head# 2024-09-20T22:20:23.774 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5066a84:::benchmark_data_smithi060_196073_object30709:head# 2024-09-20T22:20:23.775 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5068418:::benchmark_data_smithi060_196073_object24778:head# 2024-09-20T22:20:23.777 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b50708f0:::benchmark_data_smithi060_196073_object23017:head# 2024-09-20T22:20:23.778 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5081641:::benchmark_data_smithi060_196073_object36482:head# 2024-09-20T22:20:23.780 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b50c088b:::benchmark_data_smithi060_196073_object194:head# 2024-09-20T22:20:23.780 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b50c5a80:::benchmark_data_smithi060_196073_object14784:head# 2024-09-20T22:20:23.781 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b50e7b6e:::benchmark_data_smithi060_196073_object51966:head# 2024-09-20T22:20:23.781 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b50ea899:::benchmark_data_smithi060_196073_object39102:head# 2024-09-20T22:20:23.781 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b510d62c:::benchmark_data_smithi060_196073_object50007:head# 2024-09-20T22:20:23.782 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b51188ba:::benchmark_data_smithi060_196073_object21811:head# 2024-09-20T22:20:23.782 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b511dfb4:::benchmark_data_smithi060_196073_object32768:head# 2024-09-20T22:20:23.785 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b512176a:::benchmark_data_smithi060_196073_object12674:head# 2024-09-20T22:20:23.785 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b51490ab:::benchmark_data_smithi060_196073_object3304:head# 2024-09-20T22:20:23.786 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b515010e:::benchmark_data_smithi060_196073_object26849:head# 2024-09-20T22:20:23.789 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5164324:::benchmark_data_smithi060_196073_object13754:head# 2024-09-20T22:20:23.791 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b51892f9:::benchmark_data_smithi060_196073_object45222:head# 2024-09-20T22:20:23.791 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5195a07:::benchmark_data_smithi060_196073_object476:head# 2024-09-20T22:20:23.792 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b51ab719:::benchmark_data_smithi060_196073_object18337:head# 2024-09-20T22:20:23.793 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b51dc147:::benchmark_data_smithi060_196073_object28540:head# 2024-09-20T22:20:23.793 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b51e5e10:::benchmark_data_smithi060_196073_object21998:head# 2024-09-20T22:20:23.794 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b51e8cea:::benchmark_data_smithi060_196073_object60303:head# 2024-09-20T22:20:23.795 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5222578:::benchmark_data_smithi060_196073_object61701:head# 2024-09-20T22:20:23.796 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5228581:::benchmark_data_smithi060_196073_object32953:head# 2024-09-20T22:20:23.796 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b52302d8:::benchmark_data_smithi060_196073_object48827:head# 2024-09-20T22:20:23.797 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b525a928:::benchmark_data_smithi060_196073_object51745:head# 2024-09-20T22:20:23.797 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b525baa7:::benchmark_data_smithi060_196073_object48935:head# 2024-09-20T22:20:23.798 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b526b6f2:::benchmark_data_smithi060_196073_object9987:head# 2024-09-20T22:20:23.799 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b52743e8:::benchmark_data_smithi060_196073_object2293:head# 2024-09-20T22:20:23.799 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5282dd2:::benchmark_data_smithi060_196073_object18149:head# 2024-09-20T22:20:23.800 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5283060:::benchmark_data_smithi060_196073_object9158:head# 2024-09-20T22:20:23.800 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5287171:::benchmark_data_smithi060_196073_object56667:head# 2024-09-20T22:20:23.800 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5287e92:::benchmark_data_smithi060_196073_object18527:head# 2024-09-20T22:20:23.801 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b52977ab:::benchmark_data_smithi060_196073_object46227:head# 2024-09-20T22:20:23.801 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b52c4e96:::benchmark_data_smithi060_196073_object19504:head# 2024-09-20T22:20:23.802 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b52e0ddc:::benchmark_data_smithi060_196073_object57884:head# 2024-09-20T22:20:23.806 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b52eccf0:::benchmark_data_smithi060_196073_object25428:head# 2024-09-20T22:20:23.806 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b52f7a07:::benchmark_data_smithi060_196073_object3065:head# 2024-09-20T22:20:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5300b63:::benchmark_data_smithi060_196073_object11258:head# 2024-09-20T22:20:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b530460e:::benchmark_data_smithi060_196073_object34824:head# 2024-09-20T22:20:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5316bd1:::benchmark_data_smithi060_196073_object13951:head# 2024-09-20T22:20:23.807 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5320275:::benchmark_data_smithi060_196073_object10236:head# 2024-09-20T22:20:23.808 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b533af25:::benchmark_data_smithi060_196073_object52205:head# 2024-09-20T22:20:23.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:23.810 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:23.810 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:23.814 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5344f87:::benchmark_data_smithi060_196073_object14988:head# 2024-09-20T22:20:23.815 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b537b1f2:::benchmark_data_smithi060_196073_object27722:head# 2024-09-20T22:20:23.815 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b538fa66:::benchmark_data_smithi060_196073_object11940:head# 2024-09-20T22:20:23.815 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b53a2859:::benchmark_data_smithi060_196073_object58770:head# 2024-09-20T22:20:23.815 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b53a5d62:::benchmark_data_smithi060_196073_object58085:head# 2024-09-20T22:20:23.815 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b53b89d1:::benchmark_data_smithi060_196073_object45245:head# 2024-09-20T22:20:23.815 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b53d24d2:::benchmark_data_smithi060_196073_object56141:head# 2024-09-20T22:20:23.815 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b53d6290:::benchmark_data_smithi060_196073_object30156:head# 2024-09-20T22:20:23.815 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b53df96d:::benchmark_data_smithi060_196073_object13125:head# 2024-09-20T22:20:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b53fd5e9:::benchmark_data_smithi060_196073_object24658:head# 2024-09-20T22:20:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b542c473:::benchmark_data_smithi060_196073_object36600:head# 2024-09-20T22:20:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b544ab79:::benchmark_data_smithi060_196073_object42925:head# 2024-09-20T22:20:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5459a4c:::benchmark_data_smithi060_196073_object2380:head# 2024-09-20T22:20:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5490082:::benchmark_data_smithi060_196073_object37877:head# 2024-09-20T22:20:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b549f605:::benchmark_data_smithi060_196073_object12387:head# 2024-09-20T22:20:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b54a2588:::benchmark_data_smithi060_196073_object15412:head# 2024-09-20T22:20:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b54b073b:::benchmark_data_smithi060_196073_object13844:head# 2024-09-20T22:20:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b54b3e8d:::benchmark_data_smithi060_196073_object6424:head# 2024-09-20T22:20:23.825 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b54c2ab6:::benchmark_data_smithi060_196073_object53115:head# 2024-09-20T22:20:23.826 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b54c5ad4:::benchmark_data_smithi060_196073_object32451:head# 2024-09-20T22:20:23.826 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b54ced16:::benchmark_data_smithi060_196073_object43787:head# 2024-09-20T22:20:23.826 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b54e5e09:::benchmark_data_smithi060_196073_object144:head# 2024-09-20T22:20:23.826 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b54f145d:::benchmark_data_smithi060_196073_object9292:head# 2024-09-20T22:20:23.827 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5501e84:::benchmark_data_smithi060_196073_object4298:head# 2024-09-20T22:20:23.829 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b550f139:::benchmark_data_smithi060_196073_object14331:head# 2024-09-20T22:20:23.829 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5524eea:::benchmark_data_smithi060_196073_object42506:head# 2024-09-20T22:20:23.829 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5549b5c:::benchmark_data_smithi060_196073_object3483:head# 2024-09-20T22:20:23.831 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b558af68:::benchmark_data_smithi060_196073_object44322:head# 2024-09-20T22:20:23.833 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5592d52:::benchmark_data_smithi060_196073_object28828:head# 2024-09-20T22:20:23.838 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b559c2cc:::benchmark_data_smithi060_196073_object28330:head# 2024-09-20T22:20:23.838 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b559ff67:::benchmark_data_smithi060_196073_object33205:head# 2024-09-20T22:20:23.841 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b55b9778:::benchmark_data_smithi060_196073_object44283:head# 2024-09-20T22:20:23.841 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5600abe:::benchmark_data_smithi060_196073_object54163:head# 2024-09-20T22:20:23.841 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b565354e:::benchmark_data_smithi060_196073_object35071:head# 2024-09-20T22:20:23.841 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b565addc:::benchmark_data_smithi060_196073_object31201:head# 2024-09-20T22:20:23.842 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b565b4a0:::benchmark_data_smithi060_196073_object42469:head# 2024-09-20T22:20:23.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b567710b:::benchmark_data_smithi060_196073_object28716:head# 2024-09-20T22:20:23.845 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5677a6c:::benchmark_data_smithi060_196073_object12769:head# 2024-09-20T22:20:23.846 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b568944f:::benchmark_data_smithi060_196073_object40779:head# 2024-09-20T22:20:23.847 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b56a29ce:::benchmark_data_smithi060_196073_object26850:head# 2024-09-20T22:20:23.847 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b56bd718:::benchmark_data_smithi060_196073_object50136:head# 2024-09-20T22:20:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b56d3307:::benchmark_data_smithi060_196073_object4873:head# 2024-09-20T22:20:23.848 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b56da2ea:::benchmark_data_smithi060_196073_object34248:head# 2024-09-20T22:20:23.849 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b56e222e:::benchmark_data_smithi060_196073_object23633:head# 2024-09-20T22:20:23.850 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5703e0b:::benchmark_data_smithi060_196073_object3457:head# 2024-09-20T22:20:23.853 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b570a489:::benchmark_data_smithi060_196073_object61130:head# 2024-09-20T22:20:23.860 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b57129f7:::benchmark_data_smithi060_196073_object48271:head# 2024-09-20T22:20:23.860 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b571bdde:::benchmark_data_smithi060_196073_object2793:head# 2024-09-20T22:20:23.860 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b572934e:::benchmark_data_smithi060_196073_object13930:head# 2024-09-20T22:20:23.860 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b572fe81:::benchmark_data_smithi060_196073_object9416:head# 2024-09-20T22:20:23.860 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5783607:::benchmark_data_smithi060_196073_object37514:head# 2024-09-20T22:20:23.860 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5795fed:::benchmark_data_smithi060_196073_object34132:head# 2024-09-20T22:20:23.860 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b579fc84:::benchmark_data_smithi060_196073_object40975:head# 2024-09-20T22:20:23.860 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b57a2ac2:::benchmark_data_smithi060_196073_object20639:head# 2024-09-20T22:20:23.862 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b57a65c7:::benchmark_data_smithi060_196073_object28520:head# 2024-09-20T22:20:23.868 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b57a7430:::benchmark_data_smithi060_196073_object5741:head# 2024-09-20T22:20:23.868 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b57c8de3:::benchmark_data_smithi060_196073_object60107:head# 2024-09-20T22:20:23.868 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b57dd46d:::benchmark_data_smithi060_196073_object15231:head# 2024-09-20T22:20:23.868 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b57ea6db:::benchmark_data_smithi060_196073_object8462:head# 2024-09-20T22:20:23.868 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b57f8096:::benchmark_data_smithi060_196073_object57975:head# 2024-09-20T22:20:23.868 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b57fb179:::benchmark_data_smithi060_196073_object54519:head# 2024-09-20T22:20:23.868 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b58572ce:::benchmark_data_smithi060_196073_object37776:head# 2024-09-20T22:20:23.868 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5882c60:::benchmark_data_smithi060_196073_object23947:head# 2024-09-20T22:20:23.872 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5896224:::benchmark_data_smithi060_196073_object19526:head# 2024-09-20T22:20:23.872 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b58a8757:::benchmark_data_smithi060_196073_object42179:head# 2024-09-20T22:20:23.872 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b58b85ec:::benchmark_data_smithi060_196073_object48051:head# 2024-09-20T22:20:23.872 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b58d472d:::benchmark_data_smithi060_196073_object933:head# 2024-09-20T22:20:23.872 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b58d98d2:::benchmark_data_smithi060_196073_object26021:head# 2024-09-20T22:20:23.874 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b58de940:::benchmark_data_smithi060_196073_object49721:head# 2024-09-20T22:20:23.874 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b58e606a:::benchmark_data_smithi060_196073_object18626:head# 2024-09-20T22:20:23.875 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b58e917b:::benchmark_data_smithi060_196073_object61219:head# 2024-09-20T22:20:23.877 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b58f063a:::benchmark_data_smithi060_196073_object11942:head# 2024-09-20T22:20:23.877 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b58f1758:::benchmark_data_smithi060_196073_object33186:head# 2024-09-20T22:20:23.877 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b58fb1c6:::benchmark_data_smithi060_196073_object23260:head# 2024-09-20T22:20:23.877 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5908840:::benchmark_data_smithi060_196073_object57290:head# 2024-09-20T22:20:23.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b59098ad:::benchmark_data_smithi060_196073_object61227:head# 2024-09-20T22:20:23.884 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5922a84:::benchmark_data_smithi060_196073_object18188:head# 2024-09-20T22:20:23.884 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5924afa:::benchmark_data_smithi060_196073_object37271:head# 2024-09-20T22:20:23.884 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b596baba:::benchmark_data_smithi060_196073_object28850:head# 2024-09-20T22:20:23.884 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b59705aa:::benchmark_data_smithi060_196073_object36558:head# 2024-09-20T22:20:23.884 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b598a307:::benchmark_data_smithi060_196073_object58197:head# 2024-09-20T22:20:23.884 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b598f031:::benchmark_data_smithi060_196073_object10740:head# 2024-09-20T22:20:23.884 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5991d1c:::benchmark_data_smithi060_196073_object15538:head# 2024-09-20T22:20:23.884 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b599b96f:::benchmark_data_smithi060_196073_object45272:head# 2024-09-20T22:20:23.888 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b59b270e:::benchmark_data_smithi060_196073_object18785:head# 2024-09-20T22:20:23.888 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b59c4dd6:::benchmark_data_smithi060_196073_object1627:head# 2024-09-20T22:20:23.888 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b59cb565:::benchmark_data_smithi060_196073_object54915:head# 2024-09-20T22:20:23.888 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5a05b2a:::benchmark_data_smithi060_196073_object39432:head# 2024-09-20T22:20:23.888 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5a168e4:::benchmark_data_smithi060_196073_object30276:head# 2024-09-20T22:20:23.888 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5a217c8:::benchmark_data_smithi060_196073_object34840:head# 2024-09-20T22:20:23.889 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5a22a2d:::benchmark_data_smithi060_196073_object29169:head# 2024-09-20T22:20:23.889 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5a29521:::benchmark_data_smithi060_196073_object44320:head# 2024-09-20T22:20:23.889 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5a3ddfd:::benchmark_data_smithi060_196073_object5540:head# 2024-09-20T22:20:23.889 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5a516d7:::benchmark_data_smithi060_196073_object37567:head# 2024-09-20T22:20:23.890 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5a53cf1:::benchmark_data_smithi060_196073_object16468:head# 2024-09-20T22:20:23.891 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5a550cb:::benchmark_data_smithi060_196073_object58563:head# 2024-09-20T22:20:23.891 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5a669ae:::benchmark_data_smithi060_196073_object21714:head# 2024-09-20T22:20:23.891 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5a670f2:::benchmark_data_smithi060_196073_object40952:head# 2024-09-20T22:20:23.893 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5a72d65:::benchmark_data_smithi060_196073_object10370:head# 2024-09-20T22:20:23.895 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5a8124a:::benchmark_data_smithi060_196073_object55167:head# 2024-09-20T22:20:23.895 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5a8d4f1:::benchmark_data_smithi060_196073_object36810:head# 2024-09-20T22:20:23.895 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5a9eefc:::benchmark_data_smithi060_196073_object19762:head# 2024-09-20T22:20:23.895 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5aad28f:::benchmark_data_smithi060_196073_object36537:head# 2024-09-20T22:20:23.897 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5ab5004:::benchmark_data_smithi060_196073_object41503:head# 2024-09-20T22:20:23.897 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5ac820c:::benchmark_data_smithi060_196073_object23448:head# 2024-09-20T22:20:23.897 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5ad1723:::benchmark_data_smithi060_196073_object44499:head# 2024-09-20T22:20:23.902 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5ad9b3d:::benchmark_data_smithi060_196073_object52560:head# 2024-09-20T22:20:23.902 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5adbc46:::benchmark_data_smithi060_196073_object20737:head# 2024-09-20T22:20:23.902 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5adfd98:::benchmark_data_smithi060_196073_object4653:head# 2024-09-20T22:20:23.902 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5af240e:::benchmark_data_smithi060_196073_object51726:head# 2024-09-20T22:20:23.903 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5b08909:::benchmark_data_smithi060_196073_object40272:head# 2024-09-20T22:20:23.903 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5b146a5:::benchmark_data_smithi060_196073_object6565:head# 2024-09-20T22:20:23.903 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5b156b5:::benchmark_data_smithi060_196073_object44629:head# 2024-09-20T22:20:23.904 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5b1fff0:::benchmark_data_smithi060_196073_object37071:head# 2024-09-20T22:20:23.904 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5b265e1:::benchmark_data_smithi060_196073_object55824:head# 2024-09-20T22:20:23.906 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5b3b717:::benchmark_data_smithi060_196073_object25161:head# 2024-09-20T22:20:23.907 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5b48717:::benchmark_data_smithi060_196073_object58271:head# 2024-09-20T22:20:23.907 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5b580ae:::benchmark_data_smithi060_196073_object8553:head# 2024-09-20T22:20:23.908 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5b6eaf3:::benchmark_data_smithi060_196073_object6134:head# 2024-09-20T22:20:23.909 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5b75908:::benchmark_data_smithi060_196073_object16859:head# 2024-09-20T22:20:23.909 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5bb46ee:::benchmark_data_smithi060_196073_object48657:head# 2024-09-20T22:20:23.910 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5bdac34:::benchmark_data_smithi060_196073_object9165:head# 2024-09-20T22:20:23.910 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5bdae67:::benchmark_data_smithi060_196073_object56031:head# 2024-09-20T22:20:23.911 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:23.911 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5be6d54:::benchmark_data_smithi060_196073_object38961:head# 2024-09-20T22:20:23.912 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5be848a:::benchmark_data_smithi060_196073_object32878:head# 2024-09-20T22:20:23.912 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5bf3754:::benchmark_data_smithi060_196073_object17420:head# 2024-09-20T22:20:23.914 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5bfa013:::benchmark_data_smithi060_196073_object29797:head# 2024-09-20T22:20:23.914 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5c0ddd9:::benchmark_data_smithi060_196073_object39021:head# 2024-09-20T22:20:23.914 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5c176f8:::benchmark_data_smithi060_196073_object348:head# 2024-09-20T22:20:23.917 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5c3595d:::benchmark_data_smithi060_196073_object30702:head# 2024-09-20T22:20:23.917 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5c4a05b:::benchmark_data_smithi060_196073_object533:head# 2024-09-20T22:20:23.917 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5c5156e:::benchmark_data_smithi060_196073_object49436:head# 2024-09-20T22:20:23.918 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5c5dfcf:::benchmark_data_smithi060_196073_object51033:head# 2024-09-20T22:20:23.919 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5c5ea87:::benchmark_data_smithi060_196073_object15899:head# 2024-09-20T22:20:23.919 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5c9fb9b:::benchmark_data_smithi060_196073_object11138:head# 2024-09-20T22:20:23.920 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5cc04f1:::benchmark_data_smithi060_196073_object3544:head# 2024-09-20T22:20:23.922 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5cc0993:::benchmark_data_smithi060_196073_object18273:head# 2024-09-20T22:20:23.925 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5cdf5f3:::benchmark_data_smithi060_196073_object43238:head# 2024-09-20T22:20:23.931 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5cef808:::benchmark_data_smithi060_196073_object48183:head# 2024-09-20T22:20:23.931 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5cf23c2:::benchmark_data_smithi060_196073_object45131:head# 2024-09-20T22:20:23.931 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5d24e92:::benchmark_data_smithi060_196073_object11685:head# 2024-09-20T22:20:23.931 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5d2db6f:::benchmark_data_smithi060_196073_object52307:head# 2024-09-20T22:20:23.931 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5d3104d:::benchmark_data_smithi060_196073_object44177:head# 2024-09-20T22:20:23.931 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5d41b93:::benchmark_data_smithi060_196073_object54068:head# 2024-09-20T22:20:23.931 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5d6bac9:::benchmark_data_smithi060_196073_object58453:head# 2024-09-20T22:20:23.934 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5d704cb:::benchmark_data_smithi060_196073_object35005:head# 2024-09-20T22:20:23.934 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5d85196:::benchmark_data_smithi060_196073_object15025:head# 2024-09-20T22:20:23.934 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5d9da0c:::benchmark_data_smithi060_196073_object40367:head# 2024-09-20T22:20:23.935 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5dc1e2e:::benchmark_data_smithi060_196073_object10469:head# 2024-09-20T22:20:23.935 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5dda797:::benchmark_data_smithi060_196073_object57691:head# 2024-09-20T22:20:23.935 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5ddff8b:::benchmark_data_smithi060_196073_object28127:head# 2024-09-20T22:20:23.936 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5de6fc3:::benchmark_data_smithi060_196073_object27159:head# 2024-09-20T22:20:23.937 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5dec37f:::benchmark_data_smithi060_196073_object46995:head# 2024-09-20T22:20:23.938 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5defa5c:::benchmark_data_smithi060_196073_object19740:head# 2024-09-20T22:20:23.939 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5df1d4e:::benchmark_data_smithi060_196073_object57566:head# 2024-09-20T22:20:23.939 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5e053bf:::benchmark_data_smithi060_196073_object31650:head# 2024-09-20T22:20:23.940 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5e09db5:::benchmark_data_smithi060_196073_object56627:head# 2024-09-20T22:20:23.940 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5e0d0a8:::benchmark_data_smithi060_196073_object18500:head# 2024-09-20T22:20:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5e18c04:::benchmark_data_smithi060_196073_object52402:head# 2024-09-20T22:20:23.941 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5e239e1:::benchmark_data_smithi060_196073_object4609:head# 2024-09-20T22:20:23.942 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5e26834:::benchmark_data_smithi060_196073_object24348:head# 2024-09-20T22:20:23.942 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5e3dc55:::benchmark_data_smithi060_196073_object50240:head# 2024-09-20T22:20:23.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:23.951 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:23.951 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:23.963 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5e4ce7f:::benchmark_data_smithi060_196073_object4717:head# 2024-09-20T22:20:23.963 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5e578b3:::benchmark_data_smithi060_196073_object49072:head# 2024-09-20T22:20:23.963 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5e5c3a1:::benchmark_data_smithi060_196073_object60722:head# 2024-09-20T22:20:23.963 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5e626fa:::benchmark_data_smithi060_196073_object34332:head# 2024-09-20T22:20:23.963 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5e81a23:::benchmark_data_smithi060_196073_object32368:head# 2024-09-20T22:20:23.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5e84ce4:::benchmark_data_smithi060_196073_object48358:head# 2024-09-20T22:20:23.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5e975ca:::benchmark_data_smithi060_196073_object13016:head# 2024-09-20T22:20:23.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5ea75b2:::benchmark_data_smithi060_196073_object15727:head# 2024-09-20T22:20:23.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5eb6783:::benchmark_data_smithi060_196073_object52348:head# 2024-09-20T22:20:23.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5eb91f1:::benchmark_data_smithi060_196073_object20777:head# 2024-09-20T22:20:23.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5ec8cf1:::benchmark_data_smithi060_196073_object25221:head# 2024-09-20T22:20:23.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5eca7bb:::benchmark_data_smithi060_196073_object56725:head# 2024-09-20T22:20:23.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5ecb292:::benchmark_data_smithi060_196073_object7146:head# 2024-09-20T22:20:23.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5eedc00:::benchmark_data_smithi060_196073_object36318:head# 2024-09-20T22:20:23.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5eee940:::benchmark_data_smithi060_196073_object50832:head# 2024-09-20T22:20:23.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5efc6a3:::benchmark_data_smithi060_196073_object44727:head# 2024-09-20T22:20:23.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5f0e2c7:::benchmark_data_smithi060_196073_object2339:head# 2024-09-20T22:20:23.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5f23c5a:::benchmark_data_smithi060_196073_object57614:head# 2024-09-20T22:20:23.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5f27214:::benchmark_data_smithi060_196073_object24573:head# 2024-09-20T22:20:23.965 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5f2dac8:::benchmark_data_smithi060_196073_object35817:head# 2024-09-20T22:20:23.965 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5f34382:::benchmark_data_smithi060_196073_object43853:head# 2024-09-20T22:20:23.965 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5f75cd5:::benchmark_data_smithi060_196073_object23892:head# 2024-09-20T22:20:23.965 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5f8a008:::benchmark_data_smithi060_196073_object5895:head# 2024-09-20T22:20:23.966 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5f91bf1:::benchmark_data_smithi060_196073_object39436:head# 2024-09-20T22:20:23.969 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5fb9e3b:::benchmark_data_smithi060_196073_object50687:head# 2024-09-20T22:20:23.969 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5fc8aac:::benchmark_data_smithi060_196073_object51938:head# 2024-09-20T22:20:23.969 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5fd3619:::benchmark_data_smithi060_196073_object26224:head# 2024-09-20T22:20:23.969 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5fd6ae1:::benchmark_data_smithi060_196073_object10217:head# 2024-09-20T22:20:23.970 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5fe59bb:::benchmark_data_smithi060_196073_object44458:head# 2024-09-20T22:20:23.970 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b5feb9a5:::benchmark_data_smithi060_196073_object8410:head# 2024-09-20T22:20:23.970 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b600115c:::benchmark_data_smithi060_196073_object41931:head# 2024-09-20T22:20:23.971 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6002146:::benchmark_data_smithi060_196073_object55798:head# 2024-09-20T22:20:23.971 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b601c54f:::benchmark_data_smithi060_196073_object18781:head# 2024-09-20T22:20:23.972 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b60323a9:::benchmark_data_smithi060_196073_object9298:head# 2024-09-20T22:20:23.973 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6063d65:::benchmark_data_smithi060_196073_object14774:head# 2024-09-20T22:20:23.973 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6081ad6:::benchmark_data_smithi060_196073_object15190:head# 2024-09-20T22:20:23.975 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b609851a:::benchmark_data_smithi060_196073_object56985:head# 2024-09-20T22:20:23.980 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b609f449:::benchmark_data_smithi060_196073_object54014:head# 2024-09-20T22:20:23.980 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b60ae46e:::benchmark_data_smithi060_196073_object24744:head# 2024-09-20T22:20:23.980 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b60aed19:::benchmark_data_smithi060_196073_object20656:head# 2024-09-20T22:20:23.980 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b60be756:::benchmark_data_smithi060_196073_object15194:head# 2024-09-20T22:20:23.981 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b60d4cc3:::benchmark_data_smithi060_196073_object25642:head# 2024-09-20T22:20:23.981 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b60d7dfe:::benchmark_data_smithi060_196073_object15779:head# 2024-09-20T22:20:23.982 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b60ea40b:::benchmark_data_smithi060_196073_object41478:head# 2024-09-20T22:20:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b60fa58e:::benchmark_data_smithi060_196073_object50105:head# 2024-09-20T22:20:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6106d8b:::benchmark_data_smithi060_196073_object9638:head# 2024-09-20T22:20:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b612db1d:::benchmark_data_smithi060_196073_object59517:head# 2024-09-20T22:20:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b613389f:::benchmark_data_smithi060_196073_object38080:head# 2024-09-20T22:20:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b614723a:::benchmark_data_smithi060_196073_object42266:head# 2024-09-20T22:20:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b615016e:::benchmark_data_smithi060_196073_object55153:head# 2024-09-20T22:20:23.987 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b615a57d:::benchmark_data_smithi060_196073_object494:head# 2024-09-20T22:20:23.990 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b616069c:::benchmark_data_smithi060_196073_object13491:head# 2024-09-20T22:20:23.990 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6162045:::benchmark_data_smithi060_196073_object55116:head# 2024-09-20T22:20:23.990 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b616e425:::benchmark_data_smithi060_196073_object54678:head# 2024-09-20T22:20:23.990 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b617779e:::benchmark_data_smithi060_196073_object56148:head# 2024-09-20T22:20:23.992 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b61a57fa:::benchmark_data_smithi060_196073_object42157:head# 2024-09-20T22:20:23.992 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b61dadf5:::benchmark_data_smithi060_196073_object61069:head# 2024-09-20T22:20:23.997 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b61ea752:::benchmark_data_smithi060_196073_object52751:head# 2024-09-20T22:20:23.997 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b61fb5c5:::benchmark_data_smithi060_196073_object45402:head# 2024-09-20T22:20:24.001 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b620efb9:::benchmark_data_smithi060_196073_object15763:head# 2024-09-20T22:20:24.001 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6214bba:::benchmark_data_smithi060_196073_object38678:head# 2024-09-20T22:20:24.002 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6223849:::benchmark_data_smithi060_196073_object44345:head# 2024-09-20T22:20:24.002 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b62269e9:::benchmark_data_smithi060_196073_object57948:head# 2024-09-20T22:20:24.005 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6246fe2:::benchmark_data_smithi060_196073_object9354:head# 2024-09-20T22:20:24.006 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b624c89e:::benchmark_data_smithi060_196073_object40009:head# 2024-09-20T22:20:24.010 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6260f94:::benchmark_data_smithi060_196073_object12100:head# 2024-09-20T22:20:24.010 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6261160:::benchmark_data_smithi060_196073_object58918:head# 2024-09-20T22:20:24.010 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6265d4b:::benchmark_data_smithi060_196073_object26784:head# 2024-09-20T22:20:24.010 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6270012:::benchmark_data_smithi060_196073_object4015:head# 2024-09-20T22:20:24.010 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b62708ac:::benchmark_data_smithi060_196073_object6180:head# 2024-09-20T22:20:24.015 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6277450:::benchmark_data_smithi060_196073_object51036:head# 2024-09-20T22:20:24.016 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b627e6e2:::benchmark_data_smithi060_196073_object4840:head# 2024-09-20T22:20:24.016 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b627fa42:::benchmark_data_smithi060_196073_object43219:head# 2024-09-20T22:20:24.016 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b628e68d:::benchmark_data_smithi060_196073_object41206:head# 2024-09-20T22:20:24.016 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b62973e8:::benchmark_data_smithi060_196073_object55678:head# 2024-09-20T22:20:24.016 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b629bbf0:::benchmark_data_smithi060_196073_object26288:head# 2024-09-20T22:20:24.016 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b62cded8:::benchmark_data_smithi060_196073_object41366:head# 2024-09-20T22:20:24.019 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b62ce934:::benchmark_data_smithi060_196073_object61193:head# 2024-09-20T22:20:24.019 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b62d9172:::benchmark_data_smithi060_196073_object21389:head# 2024-09-20T22:20:24.019 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b62deb41:::benchmark_data_smithi060_196073_object49337:head# 2024-09-20T22:20:24.019 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b62eabfc:::benchmark_data_smithi060_196073_object23386:head# 2024-09-20T22:20:24.021 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b62ed327:::benchmark_data_smithi060_196073_object3879:head# 2024-09-20T22:20:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b62fa2c5:::benchmark_data_smithi060_196073_object40013:head# 2024-09-20T22:20:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6314864:::benchmark_data_smithi060_196073_object12913:head# 2024-09-20T22:20:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b631e799:::benchmark_data_smithi060_196073_object48510:head# 2024-09-20T22:20:24.025 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b633103b:::benchmark_data_smithi060_196073_object52999:head# 2024-09-20T22:20:24.029 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6336f6b:::benchmark_data_smithi060_196073_object34023:head# 2024-09-20T22:20:24.029 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6347fe1:::benchmark_data_smithi060_196073_object11974:head# 2024-09-20T22:20:24.029 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b636175c:::benchmark_data_smithi060_196073_object21706:head# 2024-09-20T22:20:24.029 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6363d95:::benchmark_data_smithi060_196073_object17752:head# 2024-09-20T22:20:24.029 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6378a2f:::benchmark_data_smithi060_196073_object17902:head# 2024-09-20T22:20:24.030 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b63a741a:::benchmark_data_smithi060_196073_object21454:head# 2024-09-20T22:20:24.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b63ad6e2:::benchmark_data_smithi060_196073_object56169:head# 2024-09-20T22:20:24.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b63ae6be:::benchmark_data_smithi060_196073_object31739:head# 2024-09-20T22:20:24.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b63b19f4:::benchmark_data_smithi060_196073_object35971:head# 2024-09-20T22:20:24.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b63b22cb:::benchmark_data_smithi060_196073_object33914:head# 2024-09-20T22:20:24.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b63d4bea:::benchmark_data_smithi060_196073_object10849:head# 2024-09-20T22:20:24.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b63df063:::benchmark_data_smithi060_196073_object26177:head# 2024-09-20T22:20:24.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b641dc61:::benchmark_data_smithi060_196073_object8041:head# 2024-09-20T22:20:24.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6420dfb:::benchmark_data_smithi060_196073_object22684:head# 2024-09-20T22:20:24.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6425ef4:::benchmark_data_smithi060_196073_object53590:head# 2024-09-20T22:20:24.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b642b2c5:::benchmark_data_smithi060_196073_object45537:head# 2024-09-20T22:20:24.039 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b643e031:::benchmark_data_smithi060_196073_object33672:head# 2024-09-20T22:20:24.039 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6480738:::benchmark_data_smithi060_196073_object56809:head# 2024-09-20T22:20:24.040 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6482040:::benchmark_data_smithi060_196073_object34579:head# 2024-09-20T22:20:24.041 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b649edae:::benchmark_data_smithi060_196073_object39947:head# 2024-09-20T22:20:24.042 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b64aca13:::benchmark_data_smithi060_196073_object61559:head# 2024-09-20T22:20:24.044 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b64c2692:::benchmark_data_smithi060_196073_object9041:head# 2024-09-20T22:20:24.047 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b64c5332:::benchmark_data_smithi060_196073_object40228:head# 2024-09-20T22:20:24.047 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b64d67b0:::benchmark_data_smithi060_196073_object36259:head# 2024-09-20T22:20:24.047 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b64f7b79:::benchmark_data_smithi060_196073_object50020:head# 2024-09-20T22:20:24.047 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b650c4de:::benchmark_data_smithi060_196073_object48907:head# 2024-09-20T22:20:24.047 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6514175:::benchmark_data_smithi060_196073_object21261:head# 2024-09-20T22:20:24.050 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b653f1a1:::benchmark_data_smithi060_196073_object61031:head# 2024-09-20T22:20:24.050 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b655db0d:::benchmark_data_smithi060_196073_object52646:head# 2024-09-20T22:20:24.050 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6566278:::benchmark_data_smithi060_196073_object29917:head# 2024-09-20T22:20:24.050 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b658618a:::benchmark_data_smithi060_196073_object27884:head# 2024-09-20T22:20:24.050 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6590ca8:::benchmark_data_smithi060_196073_object56411:head# 2024-09-20T22:20:24.051 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b660202f:::benchmark_data_smithi060_196073_object24993:head# 2024-09-20T22:20:24.051 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:24.054 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b66159c4:::benchmark_data_smithi060_196073_object58867:head# 2024-09-20T22:20:24.054 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6637e73:::benchmark_data_smithi060_196073_object27755:head# 2024-09-20T22:20:24.055 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b663c5a3:::benchmark_data_smithi060_196073_object13204:head# 2024-09-20T22:20:24.055 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b664897e:::benchmark_data_smithi060_196073_object20891:head# 2024-09-20T22:20:24.056 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6649e05:::benchmark_data_smithi060_196073_object14381:head# 2024-09-20T22:20:24.056 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6652f05:::benchmark_data_smithi060_196073_object34714:head# 2024-09-20T22:20:24.057 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6672325:::benchmark_data_smithi060_196073_object58412:head# 2024-09-20T22:20:24.057 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b66864cc:::benchmark_data_smithi060_196073_object22021:head# 2024-09-20T22:20:24.059 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b668a7b2:::benchmark_data_smithi060_196073_object10737:head# 2024-09-20T22:20:24.059 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b66a29ec:::benchmark_data_smithi060_196073_object30000:head# 2024-09-20T22:20:24.060 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b66b1b54:::benchmark_data_smithi060_196073_object49598:head# 2024-09-20T22:20:24.061 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b66b4e1b:::benchmark_data_smithi060_196073_object28031:head# 2024-09-20T22:20:24.061 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b66b8557:::benchmark_data_smithi060_196073_object61668:head# 2024-09-20T22:20:24.062 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b66baa54:::benchmark_data_smithi060_196073_object37673:head# 2024-09-20T22:20:24.063 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b66cb9b5:::benchmark_data_smithi060_196073_object33506:head# 2024-09-20T22:20:24.064 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b66e37a2:::benchmark_data_smithi060_196073_object25746:head# 2024-09-20T22:20:24.064 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b66f7788:::benchmark_data_smithi060_196073_object924:head# 2024-09-20T22:20:24.068 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b671667b:::benchmark_data_smithi060_196073_object39692:head# 2024-09-20T22:20:24.068 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b671beec:::benchmark_data_smithi060_196073_object10081:head# 2024-09-20T22:20:24.072 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6744ea8:::benchmark_data_smithi060_196073_object41275:head# 2024-09-20T22:20:24.072 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b675fa5e:::benchmark_data_smithi060_196073_object419:head# 2024-09-20T22:20:24.072 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b676963c:::benchmark_data_smithi060_196073_object21543:head# 2024-09-20T22:20:24.072 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6775f15:::benchmark_data_smithi060_196073_object44582:head# 2024-09-20T22:20:24.073 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b677e6c1:::benchmark_data_smithi060_196073_object58509:head# 2024-09-20T22:20:24.074 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b678d26d:::benchmark_data_smithi060_196073_object26167:head# 2024-09-20T22:20:24.076 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6792d52:::benchmark_data_smithi060_196073_object43792:head# 2024-09-20T22:20:24.076 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6793103:::benchmark_data_smithi060_196073_object9172:head# 2024-09-20T22:20:24.076 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b679d196:::benchmark_data_smithi060_196073_object34644:head# 2024-09-20T22:20:24.077 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b67ac480:::benchmark_data_smithi060_196073_object30644:head# 2024-09-20T22:20:24.077 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b67c581e:::benchmark_data_smithi060_196073_object6144:head# 2024-09-20T22:20:24.078 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b67dfb9d:::benchmark_data_smithi060_196073_object11497:head# 2024-09-20T22:20:24.078 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b67e417c:::benchmark_data_smithi060_196073_object32450:head# 2024-09-20T22:20:24.080 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6802453:::benchmark_data_smithi060_196073_object45957:head# 2024-09-20T22:20:24.080 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6802771:::benchmark_data_smithi060_196073_object23693:head# 2024-09-20T22:20:24.081 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b680daa6:::benchmark_data_smithi060_196073_object12096:head# 2024-09-20T22:20:24.081 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b680f03c:::benchmark_data_smithi060_196073_object20019:head# 2024-09-20T22:20:24.082 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b682013c:::benchmark_data_smithi060_196073_object23738:head# 2024-09-20T22:20:24.083 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6849ab3:::benchmark_data_smithi060_196073_object22353:head# 2024-09-20T22:20:24.083 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b68689be:::benchmark_data_smithi060_196073_object30965:head# 2024-09-20T22:20:24.084 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b686df15:::benchmark_data_smithi060_196073_object53220:head# 2024-09-20T22:20:24.085 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b688df34:::benchmark_data_smithi060_196073_object36508:head# 2024-09-20T22:20:24.087 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:24.088 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b68bfd99:::benchmark_data_smithi060_196073_object17114:head# 2024-09-20T22:20:24.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:24.089 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:24.100 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b68c5c09:::benchmark_data_smithi060_196073_object29120:head# 2024-09-20T22:20:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b68f1664:::benchmark_data_smithi060_196073_object21334:head# 2024-09-20T22:20:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b68ff422:::benchmark_data_smithi060_196073_object33437:head# 2024-09-20T22:20:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6905bca:::benchmark_data_smithi060_196073_object23710:head# 2024-09-20T22:20:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b69079ba:::benchmark_data_smithi060_196073_object58178:head# 2024-09-20T22:20:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6911e4d:::benchmark_data_smithi060_196073_object50919:head# 2024-09-20T22:20:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6912c94:::benchmark_data_smithi060_196073_object61693:head# 2024-09-20T22:20:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b692d566:::benchmark_data_smithi060_196073_object23918:head# 2024-09-20T22:20:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6946582:::benchmark_data_smithi060_196073_object49980:head# 2024-09-20T22:20:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b69772b8:::benchmark_data_smithi060_196073_object37685:head# 2024-09-20T22:20:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b699193b:::benchmark_data_smithi060_196073_object46920:head# 2024-09-20T22:20:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b69939d2:::benchmark_data_smithi060_196073_object37555:head# 2024-09-20T22:20:24.101 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b69a03f5:::benchmark_data_smithi060_196073_object48725:head# 2024-09-20T22:20:24.102 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b69a69a5:::benchmark_data_smithi060_196073_object47872:head# 2024-09-20T22:20:24.102 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b69abd79:::benchmark_data_smithi060_196073_object34916:head# 2024-09-20T22:20:24.102 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b69aec79:::benchmark_data_smithi060_196073_object43063:head# 2024-09-20T22:20:24.102 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b69ec570:::benchmark_data_smithi060_196073_object57062:head# 2024-09-20T22:20:24.138 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b69f34bb:::benchmark_data_smithi060_196073_object33242:head# 2024-09-20T22:20:24.138 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b69f97e9:::benchmark_data_smithi060_196073_object42303:head# 2024-09-20T22:20:24.138 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6a2b185:::benchmark_data_smithi060_196073_object32508:head# 2024-09-20T22:20:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6a607c9:::benchmark_data_smithi060_196073_object20992:head# 2024-09-20T22:20:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6a6e488:::benchmark_data_smithi060_196073_object6023:head# 2024-09-20T22:20:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6a73cb0:::benchmark_data_smithi060_196073_object57769:head# 2024-09-20T22:20:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6a7d932:::benchmark_data_smithi060_196073_object40382:head# 2024-09-20T22:20:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6a817f0:::benchmark_data_smithi060_196073_object8161:head# 2024-09-20T22:20:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6ab414d:::benchmark_data_smithi060_196073_object34377:head# 2024-09-20T22:20:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6acdce8:::benchmark_data_smithi060_196073_object34259:head# 2024-09-20T22:20:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6ad30e9:::benchmark_data_smithi060_196073_object44920:head# 2024-09-20T22:20:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6ae3fdc:::benchmark_data_smithi060_196073_object18210:head# 2024-09-20T22:20:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6af43a1:::benchmark_data_smithi060_196073_object8235:head# 2024-09-20T22:20:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6afa7d0:::benchmark_data_smithi060_196073_object34777:head# 2024-09-20T22:20:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6affc9b:::benchmark_data_smithi060_196073_object60537:head# 2024-09-20T22:20:24.139 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6b20135:::benchmark_data_smithi060_196073_object52335:head# 2024-09-20T22:20:24.140 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6b30927:::benchmark_data_smithi060_196073_object52202:head# 2024-09-20T22:20:24.140 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6b4b914:::benchmark_data_smithi060_196073_object42701:head# 2024-09-20T22:20:24.140 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6b8249f:::benchmark_data_smithi060_196073_object9174:head# 2024-09-20T22:20:24.140 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6bf7931:::benchmark_data_smithi060_196073_object61486:head# 2024-09-20T22:20:24.140 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6c01aca:::benchmark_data_smithi060_196073_object12883:head# 2024-09-20T22:20:24.140 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6c031d4:::benchmark_data_smithi060_196073_object57172:head# 2024-09-20T22:20:24.140 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6c06bd4:::benchmark_data_smithi060_196073_object31163:head# 2024-09-20T22:20:24.140 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6c28583:::benchmark_data_smithi060_196073_object53424:head# 2024-09-20T22:20:24.140 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6c2fe50:::benchmark_data_smithi060_196073_object29099:head# 2024-09-20T22:20:24.140 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6c35f28:::benchmark_data_smithi060_196073_object51050:head# 2024-09-20T22:20:24.140 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6c3c71f:::benchmark_data_smithi060_196073_object36625:head# 2024-09-20T22:20:24.140 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6c6f720:::benchmark_data_smithi060_196073_object34748:head# 2024-09-20T22:20:24.140 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6c7caa1:::benchmark_data_smithi060_196073_object13404:head# 2024-09-20T22:20:24.140 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6c7da78:::benchmark_data_smithi060_196073_object32269:head# 2024-09-20T22:20:24.140 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6c81516:::benchmark_data_smithi060_196073_object47777:head# 2024-09-20T22:20:24.141 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6c91b5c:::benchmark_data_smithi060_196073_object30278:head# 2024-09-20T22:20:24.141 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6c99c1a:::benchmark_data_smithi060_196073_object46542:head# 2024-09-20T22:20:24.141 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6cb4d32:::benchmark_data_smithi060_196073_object21136:head# 2024-09-20T22:20:24.141 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6cb95a6:::benchmark_data_smithi060_196073_object35031:head# 2024-09-20T22:20:24.141 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6ccd50e:::benchmark_data_smithi060_196073_object15710:head# 2024-09-20T22:20:24.141 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6cda1a2:::benchmark_data_smithi060_196073_object12395:head# 2024-09-20T22:20:24.141 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6cee230:::benchmark_data_smithi060_196073_object1889:head# 2024-09-20T22:20:24.141 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6cf3b95:::benchmark_data_smithi060_196073_object24829:head# 2024-09-20T22:20:24.141 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6d00eef:::benchmark_data_smithi060_196073_object9515:head# 2024-09-20T22:20:24.141 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6d14d50:::benchmark_data_smithi060_196073_object52247:head# 2024-09-20T22:20:24.141 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6d14f26:::benchmark_data_smithi060_196073_object34216:head# 2024-09-20T22:20:24.141 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6d444b2:::benchmark_data_smithi060_196073_object5310:head# 2024-09-20T22:20:24.141 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6d67379:::benchmark_data_smithi060_196073_object28259:head# 2024-09-20T22:20:24.141 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6d79867:::benchmark_data_smithi060_196073_object60748:head# 2024-09-20T22:20:24.141 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6d7afbc:::benchmark_data_smithi060_196073_object33498:head# 2024-09-20T22:20:24.142 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6d8bbd4:::benchmark_data_smithi060_196073_object32047:head# 2024-09-20T22:20:24.142 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6d9dd09:::benchmark_data_smithi060_196073_object10213:head# 2024-09-20T22:20:24.142 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6dab435:::benchmark_data_smithi060_196073_object43241:head# 2024-09-20T22:20:24.142 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6dbd3c8:::benchmark_data_smithi060_196073_object6120:head# 2024-09-20T22:20:24.142 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6dd27a3:::benchmark_data_smithi060_196073_object40447:head# 2024-09-20T22:20:24.142 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6dd5fdb:::benchmark_data_smithi060_196073_object27456:head# 2024-09-20T22:20:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6de8cdc:::benchmark_data_smithi060_196073_object46831:head# 2024-09-20T22:20:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6df3c34:::benchmark_data_smithi060_196073_object39507:head# 2024-09-20T22:20:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6dfbd94:::benchmark_data_smithi060_196073_object29052:head# 2024-09-20T22:20:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6dfd4dd:::benchmark_data_smithi060_196073_object46220:head# 2024-09-20T22:20:24.145 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6e047be:::benchmark_data_smithi060_196073_object48895:head# 2024-09-20T22:20:24.146 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6e0512c:::benchmark_data_smithi060_196073_object27813:head# 2024-09-20T22:20:24.146 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6e15c38:::benchmark_data_smithi060_196073_object50050:head# 2024-09-20T22:20:24.149 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6e32ffe:::benchmark_data_smithi060_196073_object48925:head# 2024-09-20T22:20:24.149 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6e4913b:::benchmark_data_smithi060_196073_object44251:head# 2024-09-20T22:20:24.150 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6e641bf:::benchmark_data_smithi060_196073_object54276:head# 2024-09-20T22:20:24.150 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6e6875e:::benchmark_data_smithi060_196073_object42120:head# 2024-09-20T22:20:24.150 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6e6e8d6:::benchmark_data_smithi060_196073_object9902:head# 2024-09-20T22:20:24.154 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6e7c84a:::benchmark_data_smithi060_196073_object24055:head# 2024-09-20T22:20:24.154 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6e891e8:::benchmark_data_smithi060_196073_object27565:head# 2024-09-20T22:20:24.154 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6ea0b68:::benchmark_data_smithi060_196073_object32560:head# 2024-09-20T22:20:24.155 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6ed46b2:::benchmark_data_smithi060_196073_object41534:head# 2024-09-20T22:20:24.156 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6ed7ffa:::benchmark_data_smithi060_196073_object29515:head# 2024-09-20T22:20:24.156 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6eddcf6:::benchmark_data_smithi060_196073_object5730:head# 2024-09-20T22:20:24.157 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6ee289c:::benchmark_data_smithi060_196073_object24521:head# 2024-09-20T22:20:24.159 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6ee5975:::benchmark_data_smithi060_196073_object24729:head# 2024-09-20T22:20:24.160 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6eed129:::benchmark_data_smithi060_196073_object26669:head# 2024-09-20T22:20:24.160 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6f1d67f:::benchmark_data_smithi060_196073_object29478:head# 2024-09-20T22:20:24.161 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6f2e239:::benchmark_data_smithi060_196073_object5838:head# 2024-09-20T22:20:24.161 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6f33834:::benchmark_data_smithi060_196073_object37686:head# 2024-09-20T22:20:24.162 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6f34bd2:::benchmark_data_smithi060_196073_object1185:head# 2024-09-20T22:20:24.165 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6f42673:::benchmark_data_smithi060_196073_object61566:head# 2024-09-20T22:20:24.165 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6f53d69:::benchmark_data_smithi060_196073_object18171:head# 2024-09-20T22:20:24.165 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6f8153d:::benchmark_data_smithi060_196073_object13284:head# 2024-09-20T22:20:24.165 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6f91d59:::benchmark_data_smithi060_196073_object3521:head# 2024-09-20T22:20:24.167 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6f98f01:::benchmark_data_smithi060_196073_object34787:head# 2024-09-20T22:20:24.167 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6fd338f:::benchmark_data_smithi060_196073_object36160:head# 2024-09-20T22:20:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6fdc62d:::benchmark_data_smithi060_196073_object15758:head# 2024-09-20T22:20:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6fe8611:::benchmark_data_smithi060_196073_object25189:head# 2024-09-20T22:20:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6fe8708:::benchmark_data_smithi060_196073_object19178:head# 2024-09-20T22:20:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b6feabba:::benchmark_data_smithi060_196073_object53269:head# 2024-09-20T22:20:24.173 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7004225:::benchmark_data_smithi060_196073_object49141:head# 2024-09-20T22:20:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b70090f9:::benchmark_data_smithi060_196073_object45631:head# 2024-09-20T22:20:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7015e63:::benchmark_data_smithi060_196073_object12996:head# 2024-09-20T22:20:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b701b87f:::benchmark_data_smithi060_196073_object46938:head# 2024-09-20T22:20:24.174 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b70442c0:::benchmark_data_smithi060_196073_object58863:head# 2024-09-20T22:20:24.175 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b70618f2:::benchmark_data_smithi060_196073_object31755:head# 2024-09-20T22:20:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7068fdf:::benchmark_data_smithi060_196073_object28190:head# 2024-09-20T22:20:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b707bd27:::benchmark_data_smithi060_196073_object3975:head# 2024-09-20T22:20:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b707e01f:::benchmark_data_smithi060_196073_object41386:head# 2024-09-20T22:20:24.179 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b707fba2:::benchmark_data_smithi060_196073_object39298:head# 2024-09-20T22:20:24.180 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7093571:::benchmark_data_smithi060_196073_object25761:head# 2024-09-20T22:20:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b709c1eb:::benchmark_data_smithi060_196073_object2714:head# 2024-09-20T22:20:24.181 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b70baaba:::benchmark_data_smithi060_196073_object34112:head# 2024-09-20T22:20:24.183 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b70c6d9d:::benchmark_data_smithi060_196073_object43840:head# 2024-09-20T22:20:24.186 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b70c8cdf:::benchmark_data_smithi060_196073_object49608:head# 2024-09-20T22:20:24.188 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b70d8431:::benchmark_data_smithi060_196073_object52553:head# 2024-09-20T22:20:24.188 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b710bb53:::benchmark_data_smithi060_196073_object29524:head# 2024-09-20T22:20:24.188 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7136ea9:::benchmark_data_smithi060_196073_object6892:head# 2024-09-20T22:20:24.190 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b713bd8c:::benchmark_data_smithi060_196073_object59460:head# 2024-09-20T22:20:24.190 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b71502c5:::benchmark_data_smithi060_196073_object11226:head# 2024-09-20T22:20:24.190 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:24.192 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7152744:::benchmark_data_smithi060_196073_object46471:head# 2024-09-20T22:20:24.193 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b716fe6d:::benchmark_data_smithi060_196073_object29236:head# 2024-09-20T22:20:24.194 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b718d0e3:::benchmark_data_smithi060_196073_object60342:head# 2024-09-20T22:20:24.194 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b718fca2:::benchmark_data_smithi060_196073_object25024:head# 2024-09-20T22:20:24.195 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b71936d7:::benchmark_data_smithi060_196073_object11517:head# 2024-09-20T22:20:24.195 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b719979d:::benchmark_data_smithi060_196073_object51344:head# 2024-09-20T22:20:24.195 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b71a1053:::benchmark_data_smithi060_196073_object37052:head# 2024-09-20T22:20:24.196 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b71a1560:::benchmark_data_smithi060_196073_object45780:head# 2024-09-20T22:20:24.197 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b71b9828:::benchmark_data_smithi060_196073_object45526:head# 2024-09-20T22:20:24.211 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b71ba462:::benchmark_data_smithi060_196073_object14657:head# 2024-09-20T22:20:24.211 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b71c31a9:::benchmark_data_smithi060_196073_object22316:head# 2024-09-20T22:20:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b71d97e7:::benchmark_data_smithi060_196073_object34917:head# 2024-09-20T22:20:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b71ea708:::benchmark_data_smithi060_196073_object57600:head# 2024-09-20T22:20:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b71ed22e:::benchmark_data_smithi060_196073_object48580:head# 2024-09-20T22:20:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b71f5836:::benchmark_data_smithi060_196073_object60660:head# 2024-09-20T22:20:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b724807f:::benchmark_data_smithi060_196073_object55905:head# 2024-09-20T22:20:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b72499e3:::benchmark_data_smithi060_196073_object18288:head# 2024-09-20T22:20:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b724dd4c:::benchmark_data_smithi060_196073_object21210:head# 2024-09-20T22:20:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b724f6d0:::benchmark_data_smithi060_196073_object18915:head# 2024-09-20T22:20:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b725281c:::benchmark_data_smithi060_196073_object53243:head# 2024-09-20T22:20:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b72910cc:::benchmark_data_smithi060_196073_object14621:head# 2024-09-20T22:20:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7292cfc:::benchmark_data_smithi060_196073_object27676:head# 2024-09-20T22:20:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b729a4d8:::benchmark_data_smithi060_196073_object21015:head# 2024-09-20T22:20:24.212 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b729fa3d:::benchmark_data_smithi060_196073_object38887:head# 2024-09-20T22:20:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b72accde:::benchmark_data_smithi060_196073_object27589:head# 2024-09-20T22:20:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b72b2250:::benchmark_data_smithi060_196073_object49693:head# 2024-09-20T22:20:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b72b7730:::benchmark_data_smithi060_196073_object13207:head# 2024-09-20T22:20:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b72cdb5f:::benchmark_data_smithi060_196073_object58659:head# 2024-09-20T22:20:24.213 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b72f2453:::benchmark_data_smithi060_196073_object3209:head# 2024-09-20T22:20:24.214 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b730285f:::benchmark_data_smithi060_196073_object3640:head# 2024-09-20T22:20:24.214 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b731560d:::benchmark_data_smithi060_196073_object13579:head# 2024-09-20T22:20:24.214 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b732d059:::benchmark_data_smithi060_196073_object34342:head# 2024-09-20T22:20:24.214 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7335226:::benchmark_data_smithi060_196073_object49051:head# 2024-09-20T22:20:24.214 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b734c8f7:::benchmark_data_smithi060_196073_object42672:head# 2024-09-20T22:20:24.218 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b735a211:::benchmark_data_smithi060_196073_object60876:head# 2024-09-20T22:20:24.218 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b735bb94:::benchmark_data_smithi060_196073_object8341:head# 2024-09-20T22:20:24.218 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b735f776:::benchmark_data_smithi060_196073_object55131:head# 2024-09-20T22:20:24.218 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7381328:::benchmark_data_smithi060_196073_object43527:head# 2024-09-20T22:20:24.218 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b73cf7ad:::benchmark_data_smithi060_196073_object16315:head# 2024-09-20T22:20:24.224 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b73db84c:::benchmark_data_smithi060_196073_object61164:head# 2024-09-20T22:20:24.224 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b73e7fff:::benchmark_data_smithi060_196073_object8720:head# 2024-09-20T22:20:24.224 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b73eaccd:::benchmark_data_smithi060_196073_object35492:head# 2024-09-20T22:20:24.224 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b741c70c:::benchmark_data_smithi060_196073_object46399:head# 2024-09-20T22:20:24.224 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7431984:::benchmark_data_smithi060_196073_object24485:head# 2024-09-20T22:20:24.224 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b74345f1:::benchmark_data_smithi060_196073_object42649:head# 2024-09-20T22:20:24.225 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7438d08:::benchmark_data_smithi060_196073_object48917:head# 2024-09-20T22:20:24.225 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b743b082:::benchmark_data_smithi060_196073_object49725:head# 2024-09-20T22:20:24.225 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b74633de:::benchmark_data_smithi060_196073_object18324:head# 2024-09-20T22:20:24.231 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b747a5f8:::benchmark_data_smithi060_196073_object6231:head# 2024-09-20T22:20:24.231 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b747c18c:::benchmark_data_smithi060_196073_object14170:head# 2024-09-20T22:20:24.231 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b747c6d3:::benchmark_data_smithi060_196073_object35608:head# 2024-09-20T22:20:24.231 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b747efb7:::benchmark_data_smithi060_196073_object41496:head# 2024-09-20T22:20:24.231 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7489932:::benchmark_data_smithi060_196073_object59385:head# 2024-09-20T22:20:24.231 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b748d786:::benchmark_data_smithi060_196073_object8666:head# 2024-09-20T22:20:24.231 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b74a8097:::benchmark_data_smithi060_196073_object29988:head# 2024-09-20T22:20:24.231 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b74ad72a:::benchmark_data_smithi060_196073_object38846:head# 2024-09-20T22:20:24.231 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b74c45f8:::benchmark_data_smithi060_196073_object58572:head# 2024-09-20T22:20:24.231 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b74cb81c:::benchmark_data_smithi060_196073_object3918:head# 2024-09-20T22:20:24.238 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:24.239 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:24.239 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:24.257 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b74d22d8:::benchmark_data_smithi060_196073_object46758:head# 2024-09-20T22:20:24.257 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b74d8f75:::benchmark_data_smithi060_196073_object26103:head# 2024-09-20T22:20:24.257 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b74d9215:::benchmark_data_smithi060_196073_object29183:head# 2024-09-20T22:20:24.257 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b74d9de2:::benchmark_data_smithi060_196073_object877:head# 2024-09-20T22:20:24.257 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b74da0a1:::benchmark_data_smithi060_196073_object13872:head# 2024-09-20T22:20:24.257 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b74e7bc0:::benchmark_data_smithi060_196073_object51985:head# 2024-09-20T22:20:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b74f301a:::benchmark_data_smithi060_196073_object5217:head# 2024-09-20T22:20:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b74f7d56:::benchmark_data_smithi060_196073_object39587:head# 2024-09-20T22:20:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b74fd607:::benchmark_data_smithi060_196073_object47022:head# 2024-09-20T22:20:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b75056cd:::benchmark_data_smithi060_196073_object40947:head# 2024-09-20T22:20:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b750753a:::benchmark_data_smithi060_196073_object46587:head# 2024-09-20T22:20:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b750e300:::benchmark_data_smithi060_196073_object60273:head# 2024-09-20T22:20:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7538f3d:::benchmark_data_smithi060_196073_object22280:head# 2024-09-20T22:20:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b753f284:::benchmark_data_smithi060_196073_object18806:head# 2024-09-20T22:20:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7562618:::benchmark_data_smithi060_196073_object11824:head# 2024-09-20T22:20:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7570998:::benchmark_data_smithi060_196073_object1202:head# 2024-09-20T22:20:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b757a387:::benchmark_data_smithi060_196073_object19646:head# 2024-09-20T22:20:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b758f605:::benchmark_data_smithi060_196073_object7518:head# 2024-09-20T22:20:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7598f49:::benchmark_data_smithi060_196073_object2695:head# 2024-09-20T22:20:24.258 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b759e390:::benchmark_data_smithi060_196073_object58899:head# 2024-09-20T22:20:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b75ae525:::benchmark_data_smithi060_196073_object52500:head# 2024-09-20T22:20:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b75b24be:::benchmark_data_smithi060_196073_object9069:head# 2024-09-20T22:20:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b75cce73:::benchmark_data_smithi060_196073_object56168:head# 2024-09-20T22:20:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b75d5c65:::benchmark_data_smithi060_196073_object27351:head# 2024-09-20T22:20:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b75f64eb:::benchmark_data_smithi060_196073_object3235:head# 2024-09-20T22:20:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b75fcba6:::benchmark_data_smithi060_196073_object32547:head# 2024-09-20T22:20:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b75ff4b6:::benchmark_data_smithi060_196073_object24836:head# 2024-09-20T22:20:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b760f897:::benchmark_data_smithi060_196073_object2667:head# 2024-09-20T22:20:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7623bd4:::benchmark_data_smithi060_196073_object48585:head# 2024-09-20T22:20:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b762610b:::benchmark_data_smithi060_196073_object44441:head# 2024-09-20T22:20:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7629076:::benchmark_data_smithi060_196073_object2857:head# 2024-09-20T22:20:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b762d804:::benchmark_data_smithi060_196073_object3962:head# 2024-09-20T22:20:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7631c1c:::benchmark_data_smithi060_196073_object19340:head# 2024-09-20T22:20:24.259 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b763bcc4:::benchmark_data_smithi060_196073_object56784:head# 2024-09-20T22:20:24.260 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b766cb88:::benchmark_data_smithi060_196073_object49218:head# 2024-09-20T22:20:24.260 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7676c3f:::benchmark_data_smithi060_196073_object30558:head# 2024-09-20T22:20:24.262 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7677d49:::benchmark_data_smithi060_196073_object12297:head# 2024-09-20T22:20:24.262 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b769aa93:::benchmark_data_smithi060_196073_object57922:head# 2024-09-20T22:20:24.262 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b76ae293:::benchmark_data_smithi060_196073_object19396:head# 2024-09-20T22:20:24.263 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b76b3457:::benchmark_data_smithi060_196073_object5384:head# 2024-09-20T22:20:24.268 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b76b683a:::benchmark_data_smithi060_196073_object14702:head# 2024-09-20T22:20:24.268 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b76bb6f6:::benchmark_data_smithi060_196073_object4947:head# 2024-09-20T22:20:24.268 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b771c4d7:::benchmark_data_smithi060_196073_object1875:head# 2024-09-20T22:20:24.268 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b771d673:::benchmark_data_smithi060_196073_object12579:head# 2024-09-20T22:20:24.268 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b772c219:::benchmark_data_smithi060_196073_object34041:head# 2024-09-20T22:20:24.268 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b77681d1:::benchmark_data_smithi060_196073_object2565:head# 2024-09-20T22:20:24.269 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b777e572:::benchmark_data_smithi060_196073_object58147:head# 2024-09-20T22:20:24.269 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b777ec62:::benchmark_data_smithi060_196073_object4395:head# 2024-09-20T22:20:24.269 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b778555e:::benchmark_data_smithi060_196073_object35920:head# 2024-09-20T22:20:24.270 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b77970f4:::benchmark_data_smithi060_196073_object12414:head# 2024-09-20T22:20:24.271 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b77d544c:::benchmark_data_smithi060_196073_object4611:head# 2024-09-20T22:20:24.271 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b77ed01c:::benchmark_data_smithi060_196073_object25122:head# 2024-09-20T22:20:24.272 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b77f2f59:::benchmark_data_smithi060_196073_object45068:head# 2024-09-20T22:20:24.272 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b77f9e80:::benchmark_data_smithi060_196073_object35103:head# 2024-09-20T22:20:24.273 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b77fc5d3:::benchmark_data_smithi060_196073_object53612:head# 2024-09-20T22:20:24.274 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b78065df:::benchmark_data_smithi060_196073_object48425:head# 2024-09-20T22:20:24.278 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b781413e:::benchmark_data_smithi060_196073_object28686:head# 2024-09-20T22:20:24.278 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7818bbc:::benchmark_data_smithi060_196073_object43820:head# 2024-09-20T22:20:24.278 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7821904:::benchmark_data_smithi060_196073_object17682:head# 2024-09-20T22:20:24.278 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b782abd6:::benchmark_data_smithi060_196073_object38243:head# 2024-09-20T22:20:24.278 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b783d93a:::benchmark_data_smithi060_196073_object8881:head# 2024-09-20T22:20:24.278 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7841794:::benchmark_data_smithi060_196073_object6872:head# 2024-09-20T22:20:24.279 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b784b3c8:::benchmark_data_smithi060_196073_object30663:head# 2024-09-20T22:20:24.280 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7865804:::benchmark_data_smithi060_196073_object11373:head# 2024-09-20T22:20:24.283 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b787aa30:::benchmark_data_smithi060_196073_object19520:head# 2024-09-20T22:20:24.285 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b787d09e:::benchmark_data_smithi060_196073_object45635:head# 2024-09-20T22:20:24.286 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b787d4b0:::benchmark_data_smithi060_196073_object20701:head# 2024-09-20T22:20:24.287 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7882b22:::benchmark_data_smithi060_196073_object48538:head# 2024-09-20T22:20:24.290 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7885ab2:::benchmark_data_smithi060_196073_object41170:head# 2024-09-20T22:20:24.290 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b78b9936:::benchmark_data_smithi060_196073_object5915:head# 2024-09-20T22:20:24.291 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b78e9227:::benchmark_data_smithi060_196073_object24571:head# 2024-09-20T22:20:24.298 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b78f21e1:::benchmark_data_smithi060_196073_object19108:head# 2024-09-20T22:20:24.298 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b78f326e:::benchmark_data_smithi060_196073_object27663:head# 2024-09-20T22:20:24.298 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b78f9d0a:::benchmark_data_smithi060_196073_object46993:head# 2024-09-20T22:20:24.298 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b78ffb98:::benchmark_data_smithi060_196073_object6611:head# 2024-09-20T22:20:24.298 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7901093:::benchmark_data_smithi060_196073_object31624:head# 2024-09-20T22:20:24.298 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b790362d:::benchmark_data_smithi060_196073_object57041:head# 2024-09-20T22:20:24.299 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7915424:::benchmark_data_smithi060_196073_object29807:head# 2024-09-20T22:20:24.299 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b793765a:::benchmark_data_smithi060_196073_object523:head# 2024-09-20T22:20:24.299 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b793b6a2:::benchmark_data_smithi060_196073_object20129:head# 2024-09-20T22:20:24.299 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b79433e6:::benchmark_data_smithi060_196073_object48934:head# 2024-09-20T22:20:24.308 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7954711:::benchmark_data_smithi060_196073_object8548:head# 2024-09-20T22:20:24.308 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b79547e9:::benchmark_data_smithi060_196073_object37228:head# 2024-09-20T22:20:24.308 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7955f03:::benchmark_data_smithi060_196073_object28979:head# 2024-09-20T22:20:24.308 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b797a699:::benchmark_data_smithi060_196073_object38227:head# 2024-09-20T22:20:24.308 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7990e67:::benchmark_data_smithi060_196073_object5689:head# 2024-09-20T22:20:24.308 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7998895:::benchmark_data_smithi060_196073_object7193:head# 2024-09-20T22:20:24.308 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b79bb126:::benchmark_data_smithi060_196073_object29105:head# 2024-09-20T22:20:24.308 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b79c3574:::benchmark_data_smithi060_196073_object23330:head# 2024-09-20T22:20:24.308 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b79ddd10:::benchmark_data_smithi060_196073_object58083:head# 2024-09-20T22:20:24.308 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b79e49f1:::benchmark_data_smithi060_196073_object56893:head# 2024-09-20T22:20:24.309 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b79f4742:::benchmark_data_smithi060_196073_object20998:head# 2024-09-20T22:20:24.311 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7a01f0e:::benchmark_data_smithi060_196073_object20056:head# 2024-09-20T22:20:24.311 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7a2e2be:::benchmark_data_smithi060_196073_object5538:head# 2024-09-20T22:20:24.314 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7a30274:::benchmark_data_smithi060_196073_object27344:head# 2024-09-20T22:20:24.316 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7a48602:::benchmark_data_smithi060_196073_object59:head# 2024-09-20T22:20:24.323 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7a58d58:::benchmark_data_smithi060_196073_object2948:head# 2024-09-20T22:20:24.324 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7a64207:::benchmark_data_smithi060_196073_object2767:head# 2024-09-20T22:20:24.324 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7a7ffc0:::benchmark_data_smithi060_196073_object37384:head# 2024-09-20T22:20:24.324 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7a88317:::benchmark_data_smithi060_196073_object36115:head# 2024-09-20T22:20:24.324 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7a8aaa5:::benchmark_data_smithi060_196073_object56315:head# 2024-09-20T22:20:24.324 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7a8df21:::benchmark_data_smithi060_196073_object32869:head# 2024-09-20T22:20:24.324 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7a913ae:::benchmark_data_smithi060_196073_object3375:head# 2024-09-20T22:20:24.324 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7a97078:::benchmark_data_smithi060_196073_object48598:head# 2024-09-20T22:20:24.324 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7aa1788:::benchmark_data_smithi060_196073_object12588:head# 2024-09-20T22:20:24.325 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7ab32da:::benchmark_data_smithi060_196073_object55304:head# 2024-09-20T22:20:24.325 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7ab368b:::benchmark_data_smithi060_196073_object38217:head# 2024-09-20T22:20:24.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7addde9:::benchmark_data_smithi060_196073_object34584:head# 2024-09-20T22:20:24.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7aed7ab:::benchmark_data_smithi060_196073_object10374:head# 2024-09-20T22:20:24.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7b079ee:::benchmark_data_smithi060_196073_object22464:head# 2024-09-20T22:20:24.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7b15ce1:::benchmark_data_smithi060_196073_object46594:head# 2024-09-20T22:20:24.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7b1a8e3:::benchmark_data_smithi060_196073_object228:head# 2024-09-20T22:20:24.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7b1cc06:::benchmark_data_smithi060_196073_object51276:head# 2024-09-20T22:20:24.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7b6730b:::benchmark_data_smithi060_196073_object13810:head# 2024-09-20T22:20:24.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7b9f223:::benchmark_data_smithi060_196073_object46878:head# 2024-09-20T22:20:24.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7bb31df:::benchmark_data_smithi060_196073_object59948:head# 2024-09-20T22:20:24.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7bb892d:::benchmark_data_smithi060_196073_object47989:head# 2024-09-20T22:20:24.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7bb8a11:::benchmark_data_smithi060_196073_object30588:head# 2024-09-20T22:20:24.337 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7bc3184:::benchmark_data_smithi060_196073_object10330:head# 2024-09-20T22:20:24.337 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7bdef7a:::benchmark_data_smithi060_196073_object52005:head# 2024-09-20T22:20:24.337 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7bf8c7a:::benchmark_data_smithi060_196073_object18469:head# 2024-09-20T22:20:24.339 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7c01089:::benchmark_data_smithi060_196073_object32701:head# 2024-09-20T22:20:24.341 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7c0d458:::benchmark_data_smithi060_196073_object19845:head# 2024-09-20T22:20:24.342 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7c1c5c4:::benchmark_data_smithi060_196073_object2687:head# 2024-09-20T22:20:24.342 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7c287cc:::benchmark_data_smithi060_196073_object44664:head# 2024-09-20T22:20:24.342 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7c3f305:::benchmark_data_smithi060_196073_object2269:head# 2024-09-20T22:20:24.342 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:24.343 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7c44b38:::benchmark_data_smithi060_196073_object52733:head# 2024-09-20T22:20:24.343 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7c50d82:::benchmark_data_smithi060_196073_object24300:head# 2024-09-20T22:20:24.343 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7caffb2:::benchmark_data_smithi060_196073_object24446:head# 2024-09-20T22:20:24.345 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7cb15d2:::benchmark_data_smithi060_196073_object52966:head# 2024-09-20T22:20:24.345 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7cb35dd:::benchmark_data_smithi060_196073_object3254:head# 2024-09-20T22:20:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7cd2e57:::benchmark_data_smithi060_196073_object43530:head# 2024-09-20T22:20:24.346 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7cfba7f:::benchmark_data_smithi060_196073_object30249:head# 2024-09-20T22:20:24.347 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7d0a0a1:::benchmark_data_smithi060_196073_object8559:head# 2024-09-20T22:20:24.347 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7d1f913:::benchmark_data_smithi060_196073_object15240:head# 2024-09-20T22:20:24.348 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7d58dfb:::benchmark_data_smithi060_196073_object54379:head# 2024-09-20T22:20:24.349 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7d72274:::benchmark_data_smithi060_196073_object60963:head# 2024-09-20T22:20:24.349 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7d85cf4:::benchmark_data_smithi060_196073_object30795:head# 2024-09-20T22:20:24.350 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7d985cc:::benchmark_data_smithi060_196073_object42881:head# 2024-09-20T22:20:24.350 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7da76dd:::benchmark_data_smithi060_196073_object56267:head# 2024-09-20T22:20:24.353 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7daa8cc:::benchmark_data_smithi060_196073_object53180:head# 2024-09-20T22:20:24.353 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7dbae02:::benchmark_data_smithi060_196073_object7145:head# 2024-09-20T22:20:24.353 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7dc2026:::benchmark_data_smithi060_196073_object37805:head# 2024-09-20T22:20:24.353 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7dc569b:::benchmark_data_smithi060_196073_object14967:head# 2024-09-20T22:20:24.353 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7dc600a:::benchmark_data_smithi060_196073_object3430:head# 2024-09-20T22:20:24.354 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7dd9129:::benchmark_data_smithi060_196073_object13114:head# 2024-09-20T22:20:24.354 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7deebe0:::benchmark_data_smithi060_196073_object38537:head# 2024-09-20T22:20:24.355 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7dfa7d6:::benchmark_data_smithi060_196073_object45338:head# 2024-09-20T22:20:24.357 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7dfcc53:::benchmark_data_smithi060_196073_object49717:head# 2024-09-20T22:20:24.358 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7e12ecb:::benchmark_data_smithi060_196073_object22587:head# 2024-09-20T22:20:24.358 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7e1d295:::benchmark_data_smithi060_196073_object23432:head# 2024-09-20T22:20:24.359 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7e24465:::benchmark_data_smithi060_196073_object1934:head# 2024-09-20T22:20:24.359 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7e2cd0c:::benchmark_data_smithi060_196073_object41437:head# 2024-09-20T22:20:24.359 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7e2eb15:::benchmark_data_smithi060_196073_object11886:head# 2024-09-20T22:20:24.360 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7e57467:::benchmark_data_smithi060_196073_object42304:head# 2024-09-20T22:20:24.361 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7e58a37:::benchmark_data_smithi060_196073_object47665:head# 2024-09-20T22:20:24.361 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7e5cdd4:::benchmark_data_smithi060_196073_object17767:head# 2024-09-20T22:20:24.363 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7e7bdac:::benchmark_data_smithi060_196073_object6653:head# 2024-09-20T22:20:24.363 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7e819be:::benchmark_data_smithi060_196073_object58380:head# 2024-09-20T22:20:24.364 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7e97d55:::benchmark_data_smithi060_196073_object16917:head# 2024-09-20T22:20:24.368 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7e9c1af:::benchmark_data_smithi060_196073_object60635:head# 2024-09-20T22:20:24.368 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7ed8564:::benchmark_data_smithi060_196073_object39628:head# 2024-09-20T22:20:24.368 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7ede956:::benchmark_data_smithi060_196073_object2625:head# 2024-09-20T22:20:24.371 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7ee554f:::benchmark_data_smithi060_196073_object27235:head# 2024-09-20T22:20:24.371 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7ee91ea:::benchmark_data_smithi060_196073_object61229:head# 2024-09-20T22:20:24.372 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7eea2da:::benchmark_data_smithi060_196073_object55658:head# 2024-09-20T22:20:24.372 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7eeea3b:::benchmark_data_smithi060_196073_object28659:head# 2024-09-20T22:20:24.372 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7ef8461:::benchmark_data_smithi060_196073_object3796:head# 2024-09-20T22:20:24.372 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7efbfd2:::benchmark_data_smithi060_196073_object45584:head# 2024-09-20T22:20:24.372 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7f0c72a:::benchmark_data_smithi060_196073_object38070:head# 2024-09-20T22:20:24.373 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7f0ecd7:::benchmark_data_smithi060_196073_object16599:head# 2024-09-20T22:20:24.373 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:24.373 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7f1a3e8:::benchmark_data_smithi060_196073_object15711:head# 2024-09-20T22:20:24.374 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7f2d649:::benchmark_data_smithi060_196073_object44013:head# 2024-09-20T22:20:24.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:24.374 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:24.381 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7f3bf85:::benchmark_data_smithi060_196073_object17791:head# 2024-09-20T22:20:24.381 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7f4a920:::benchmark_data_smithi060_196073_object43751:head# 2024-09-20T22:20:24.381 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7f517be:::benchmark_data_smithi060_196073_object35809:head# 2024-09-20T22:20:24.381 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7f5e90d:::benchmark_data_smithi060_196073_object55353:head# 2024-09-20T22:20:24.381 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7f6640c:::benchmark_data_smithi060_196073_object9876:head# 2024-09-20T22:20:24.381 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7f6b6a5:::benchmark_data_smithi060_196073_object7046:head# 2024-09-20T22:20:24.382 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7f6ecf2:::benchmark_data_smithi060_196073_object8979:head# 2024-09-20T22:20:24.385 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7f7a8d8:::benchmark_data_smithi060_196073_object51304:head# 2024-09-20T22:20:24.385 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7f7b31b:::benchmark_data_smithi060_196073_object50213:head# 2024-09-20T22:20:24.385 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7f8542b:::benchmark_data_smithi060_196073_object7207:head# 2024-09-20T22:20:24.385 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7f90733:::benchmark_data_smithi060_196073_object55711:head# 2024-09-20T22:20:24.388 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7fbdd5a:::benchmark_data_smithi060_196073_object59334:head# 2024-09-20T22:20:24.388 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7fcff3c:::benchmark_data_smithi060_196073_object19725:head# 2024-09-20T22:20:24.388 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7fe2f52:::benchmark_data_smithi060_196073_object27700:head# 2024-09-20T22:20:24.388 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7fe3468:::benchmark_data_smithi060_196073_object34756:head# 2024-09-20T22:20:24.389 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7fe564b:::benchmark_data_smithi060_196073_object3204:head# 2024-09-20T22:20:24.391 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7feaf2e:::benchmark_data_smithi060_196073_object8337:head# 2024-09-20T22:20:24.392 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b7ff66ea:::benchmark_data_smithi060_196073_object8077:head# 2024-09-20T22:20:24.393 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8002c84:::benchmark_data_smithi060_196073_object38931:head# 2024-09-20T22:20:24.393 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8008f9c:::benchmark_data_smithi060_196073_object25270:head# 2024-09-20T22:20:24.394 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8013544:::benchmark_data_smithi060_196073_object53557:head# 2024-09-20T22:20:24.395 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b80146cc:::benchmark_data_smithi060_196073_object59429:head# 2024-09-20T22:20:24.395 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b802a085:::benchmark_data_smithi060_196073_object29959:head# 2024-09-20T22:20:24.398 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b804b375:::benchmark_data_smithi060_196073_object15406:head# 2024-09-20T22:20:24.398 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b805c9d4:::benchmark_data_smithi060_196073_object53826:head# 2024-09-20T22:20:24.400 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b805e93f:::benchmark_data_smithi060_196073_object3280:head# 2024-09-20T22:20:24.402 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b806d707:::benchmark_data_smithi060_196073_object32333:head# 2024-09-20T22:20:24.402 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8099a98:::benchmark_data_smithi060_196073_object3159:head# 2024-09-20T22:20:24.405 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b80de8ba:::benchmark_data_smithi060_196073_object13988:head# 2024-09-20T22:20:24.406 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b80e0acf:::benchmark_data_smithi060_196073_object42399:head# 2024-09-20T22:20:24.408 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b80fbc33:::benchmark_data_smithi060_196073_object55388:head# 2024-09-20T22:20:24.409 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b80fc443:::benchmark_data_smithi060_196073_object53213:head# 2024-09-20T22:20:24.409 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b81685de:::benchmark_data_smithi060_196073_object28406:head# 2024-09-20T22:20:24.409 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b816900a:::benchmark_data_smithi060_196073_object31586:head# 2024-09-20T22:20:24.410 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b81757e5:::benchmark_data_smithi060_196073_object47719:head# 2024-09-20T22:20:24.411 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8181180:::benchmark_data_smithi060_196073_object51446:head# 2024-09-20T22:20:24.412 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8183fb3:::benchmark_data_smithi060_196073_object2955:head# 2024-09-20T22:20:24.413 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8184065:::benchmark_data_smithi060_196073_object17951:head# 2024-09-20T22:20:24.414 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8185402:::benchmark_data_smithi060_196073_object44868:head# 2024-09-20T22:20:24.415 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b819ebc1:::benchmark_data_smithi060_196073_object60475:head# 2024-09-20T22:20:24.415 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b81a7c23:::benchmark_data_smithi060_196073_object40036:head# 2024-09-20T22:20:24.416 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b81a8d37:::benchmark_data_smithi060_196073_object26879:head# 2024-09-20T22:20:24.416 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b81a9128:::benchmark_data_smithi060_196073_object58329:head# 2024-09-20T22:20:24.417 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b81bbb36:::benchmark_data_smithi060_196073_object20655:head# 2024-09-20T22:20:24.417 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b81c3b8e:::benchmark_data_smithi060_196073_object13376:head# 2024-09-20T22:20:24.419 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b81cdebc:::benchmark_data_smithi060_196073_object29441:head# 2024-09-20T22:20:24.419 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b81d6ba3:::benchmark_data_smithi060_196073_object42872:head# 2024-09-20T22:20:24.419 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b81db480:::benchmark_data_smithi060_196073_object60998:head# 2024-09-20T22:20:24.420 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b821779c:::benchmark_data_smithi060_196073_object11722:head# 2024-09-20T22:20:24.421 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b821d802:::benchmark_data_smithi060_196073_object27473:head# 2024-09-20T22:20:24.421 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b822e65c:::benchmark_data_smithi060_196073_object37122:head# 2024-09-20T22:20:24.422 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8232b34:::benchmark_data_smithi060_196073_object342:head# 2024-09-20T22:20:24.422 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b82394fb:::benchmark_data_smithi060_196073_object53765:head# 2024-09-20T22:20:24.423 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8249aa1:::benchmark_data_smithi060_196073_object41715:head# 2024-09-20T22:20:24.424 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b824e41d:::benchmark_data_smithi060_196073_object27270:head# 2024-09-20T22:20:24.424 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b824e547:::benchmark_data_smithi060_196073_object5443:head# 2024-09-20T22:20:24.424 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8270a9a:::benchmark_data_smithi060_196073_object50766:head# 2024-09-20T22:20:24.425 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b827a8a6:::benchmark_data_smithi060_196073_object5587:head# 2024-09-20T22:20:24.426 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b82868e0:::benchmark_data_smithi060_196073_object59973:head# 2024-09-20T22:20:24.426 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b829414d:::benchmark_data_smithi060_196073_object21794:head# 2024-09-20T22:20:24.426 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b82aefa6:::benchmark_data_smithi060_196073_object49202:head# 2024-09-20T22:20:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b82b432f:::benchmark_data_smithi060_196073_object27979:head# 2024-09-20T22:20:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b82cab8b:::benchmark_data_smithi060_196073_object11684:head# 2024-09-20T22:20:24.429 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b83056f3:::benchmark_data_smithi060_196073_object30457:head# 2024-09-20T22:20:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8314d17:::benchmark_data_smithi060_196073_object33774:head# 2024-09-20T22:20:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b832e86a:::benchmark_data_smithi060_196073_object33652:head# 2024-09-20T22:20:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b833583d:::benchmark_data_smithi060_196073_object9052:head# 2024-09-20T22:20:24.437 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b833c675:::benchmark_data_smithi060_196073_object27968:head# 2024-09-20T22:20:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b834b023:::benchmark_data_smithi060_196073_object34540:head# 2024-09-20T22:20:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8357e78:::benchmark_data_smithi060_196073_object35050:head# 2024-09-20T22:20:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b835d7d2:::benchmark_data_smithi060_196073_object4472:head# 2024-09-20T22:20:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b83661b3:::benchmark_data_smithi060_196073_object19367:head# 2024-09-20T22:20:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b836b36b:::benchmark_data_smithi060_196073_object21311:head# 2024-09-20T22:20:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b836d054:::benchmark_data_smithi060_196073_object45806:head# 2024-09-20T22:20:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8382dde:::benchmark_data_smithi060_196073_object52396:head# 2024-09-20T22:20:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8387557:::benchmark_data_smithi060_196073_object31022:head# 2024-09-20T22:20:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b838c051:::benchmark_data_smithi060_196073_object31336:head# 2024-09-20T22:20:24.438 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b83b02bd:::benchmark_data_smithi060_196073_object56080:head# 2024-09-20T22:20:24.439 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b83b2dee:::benchmark_data_smithi060_196073_object27304:head# 2024-09-20T22:20:24.439 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b83b8768:::benchmark_data_smithi060_196073_object29890:head# 2024-09-20T22:20:24.439 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b83c7f05:::benchmark_data_smithi060_196073_object42182:head# 2024-09-20T22:20:24.439 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b83d6cd3:::benchmark_data_smithi060_196073_object38131:head# 2024-09-20T22:20:24.440 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b83e6552:::benchmark_data_smithi060_196073_object10028:head# 2024-09-20T22:20:24.442 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b83fd279:::benchmark_data_smithi060_196073_object17711:head# 2024-09-20T22:20:24.442 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8409b85:::benchmark_data_smithi060_196073_object42195:head# 2024-09-20T22:20:24.442 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b84228aa:::benchmark_data_smithi060_196073_object29289:head# 2024-09-20T22:20:24.442 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b843262e:::benchmark_data_smithi060_196073_object6934:head# 2024-09-20T22:20:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b84824f7:::benchmark_data_smithi060_196073_object24365:head# 2024-09-20T22:20:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b84ab0a8:::benchmark_data_smithi060_196073_object35859:head# 2024-09-20T22:20:24.443 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b84aef72:::benchmark_data_smithi060_196073_object17235:head# 2024-09-20T22:20:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b84bb492:::benchmark_data_smithi060_196073_object21750:head# 2024-09-20T22:20:24.444 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b84c790f:::benchmark_data_smithi060_196073_object40508:head# 2024-09-20T22:20:24.445 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b84d017f:::benchmark_data_smithi060_196073_object40964:head# 2024-09-20T22:20:24.445 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b84dfefe:::benchmark_data_smithi060_196073_object41239:head# 2024-09-20T22:20:24.446 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b84fd434:::benchmark_data_smithi060_196073_object6782:head# 2024-09-20T22:20:24.446 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b85003bb:::benchmark_data_smithi060_196073_object47032:head# 2024-09-20T22:20:24.447 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8501b07:::benchmark_data_smithi060_196073_object856:head# 2024-09-20T22:20:24.447 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8514cf8:::benchmark_data_smithi060_196073_object59852:head# 2024-09-20T22:20:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b851b2a9:::benchmark_data_smithi060_196073_object54287:head# 2024-09-20T22:20:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8522d2f:::benchmark_data_smithi060_196073_object28217:head# 2024-09-20T22:20:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8539eff:::benchmark_data_smithi060_196073_object6581:head# 2024-09-20T22:20:24.449 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b854a639:::benchmark_data_smithi060_196073_object18424:head# 2024-09-20T22:20:24.450 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b854ace5:::benchmark_data_smithi060_196073_object18068:head# 2024-09-20T22:20:24.453 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b854be28:::benchmark_data_smithi060_196073_object9564:head# 2024-09-20T22:20:24.453 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b85545c6:::benchmark_data_smithi060_196073_object38059:head# 2024-09-20T22:20:24.453 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8554fab:::benchmark_data_smithi060_196073_object35594:head# 2024-09-20T22:20:24.456 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b855cf18:::benchmark_data_smithi060_196073_object12528:head# 2024-09-20T22:20:24.456 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b85720d9:::benchmark_data_smithi060_196073_object37667:head# 2024-09-20T22:20:24.456 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b85ac0d7:::benchmark_data_smithi060_196073_object57160:head# 2024-09-20T22:20:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b85c82d1:::benchmark_data_smithi060_196073_object50260:head# 2024-09-20T22:20:24.457 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b86137c1:::benchmark_data_smithi060_196073_object37760:head# 2024-09-20T22:20:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8634874:::benchmark_data_smithi060_196073_object49387:head# 2024-09-20T22:20:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b864f932:::benchmark_data_smithi060_196073_object21076:head# 2024-09-20T22:20:24.458 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8667f65:::benchmark_data_smithi060_196073_object38186:head# 2024-09-20T22:20:24.459 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b866a1f3:::benchmark_data_smithi060_196073_object13727:head# 2024-09-20T22:20:24.460 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b867052c:::benchmark_data_smithi060_196073_object37520:head# 2024-09-20T22:20:24.460 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8671a8c:::benchmark_data_smithi060_196073_object27048:head# 2024-09-20T22:20:24.460 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8681a93:::benchmark_data_smithi060_196073_object12106:head# 2024-09-20T22:20:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b86a8719:::benchmark_data_smithi060_196073_object35496:head# 2024-09-20T22:20:24.461 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b86aac2a:::benchmark_data_smithi060_196073_object7639:head# 2024-09-20T22:20:24.462 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b86caa10:::benchmark_data_smithi060_196073_object18244:head# 2024-09-20T22:20:24.462 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b86ce0b5:::benchmark_data_smithi060_196073_object28122:head# 2024-09-20T22:20:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b86eebb4:::benchmark_data_smithi060_196073_object3081:head# 2024-09-20T22:20:24.463 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b86f86a1:::benchmark_data_smithi060_196073_object59306:head# 2024-09-20T22:20:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b872faf4:::benchmark_data_smithi060_196073_object24697:head# 2024-09-20T22:20:24.464 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b874120b:::benchmark_data_smithi060_196073_object6414:head# 2024-09-20T22:20:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8754692:::benchmark_data_smithi060_196073_object52193:head# 2024-09-20T22:20:24.465 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8757ad9:::benchmark_data_smithi060_196073_object61513:head# 2024-09-20T22:20:24.466 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b875eeba:::benchmark_data_smithi060_196073_object18134:head# 2024-09-20T22:20:24.466 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8766d4e:::benchmark_data_smithi060_196073_object2166:head# 2024-09-20T22:20:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b876fa17:::benchmark_data_smithi060_196073_object28237:head# 2024-09-20T22:20:24.467 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8774c5b:::benchmark_data_smithi060_196073_object36090:head# 2024-09-20T22:20:24.468 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8789fd9:::benchmark_data_smithi060_196073_object28453:head# 2024-09-20T22:20:24.469 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b879fe24:::benchmark_data_smithi060_196073_object4322:head# 2024-09-20T22:20:24.469 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b87cc3b3:::benchmark_data_smithi060_196073_object41172:head# 2024-09-20T22:20:24.469 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b87e7a2e:::benchmark_data_smithi060_196073_object24673:head# 2024-09-20T22:20:24.472 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b87fc62e:::benchmark_data_smithi060_196073_object85:head# 2024-09-20T22:20:24.472 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b882ee44:::benchmark_data_smithi060_196073_object9237:head# 2024-09-20T22:20:24.473 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8844e33:::benchmark_data_smithi060_196073_object27975:head# 2024-09-20T22:20:24.473 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b884db60:::benchmark_data_smithi060_196073_object60248:head# 2024-09-20T22:20:24.474 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b88629b8:::benchmark_data_smithi060_196073_object33556:head# 2024-09-20T22:20:24.474 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b886cd10:::benchmark_data_smithi060_196073_object21725:head# 2024-09-20T22:20:24.475 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:24.476 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b88a013e:::benchmark_data_smithi060_196073_object10661:head# 2024-09-20T22:20:24.476 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b88a3c81:::benchmark_data_smithi060_196073_object24381:head# 2024-09-20T22:20:24.477 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b88a72d7:::benchmark_data_smithi060_196073_object18161:head# 2024-09-20T22:20:24.479 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b88b3cf5:::benchmark_data_smithi060_196073_object7135:head# 2024-09-20T22:20:24.479 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b88b5d9e:::benchmark_data_smithi060_196073_object52482:head# 2024-09-20T22:20:24.479 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b88c8983:::benchmark_data_smithi060_196073_object39798:head# 2024-09-20T22:20:24.479 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b88cf720:::benchmark_data_smithi060_196073_object23326:head# 2024-09-20T22:20:24.480 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b88e1faf:::benchmark_data_smithi060_196073_object9688:head# 2024-09-20T22:20:24.480 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b88e89a4:::benchmark_data_smithi060_196073_object716:head# 2024-09-20T22:20:24.481 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8909feb:::benchmark_data_smithi060_196073_object55104:head# 2024-09-20T22:20:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b89149da:::benchmark_data_smithi060_196073_object25136:head# 2024-09-20T22:20:24.483 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b892693d:::benchmark_data_smithi060_196073_object21977:head# 2024-09-20T22:20:24.484 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8928305:::benchmark_data_smithi060_196073_object57536:head# 2024-09-20T22:20:24.485 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b893e074:::benchmark_data_smithi060_196073_object54660:head# 2024-09-20T22:20:24.485 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8950aa2:::benchmark_data_smithi060_196073_object53813:head# 2024-09-20T22:20:24.485 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8974397:::benchmark_data_smithi060_196073_object20770:head# 2024-09-20T22:20:24.486 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b89aa745:::benchmark_data_smithi060_196073_object46953:head# 2024-09-20T22:20:24.487 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b89acb06:::benchmark_data_smithi060_196073_object20695:head# 2024-09-20T22:20:24.487 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b89f71ad:::benchmark_data_smithi060_196073_object37393:head# 2024-09-20T22:20:24.487 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b89f7f68:::benchmark_data_smithi060_196073_object33831:head# 2024-09-20T22:20:24.488 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b89ff576:::benchmark_data_smithi060_196073_object32785:head# 2024-09-20T22:20:24.489 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8a01fa6:::benchmark_data_smithi060_196073_object25745:head# 2024-09-20T22:20:24.489 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8a0604c:::benchmark_data_smithi060_196073_object12497:head# 2024-09-20T22:20:24.490 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8a11961:::benchmark_data_smithi060_196073_object54743:head# 2024-09-20T22:20:24.491 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8a15664:::benchmark_data_smithi060_196073_object13455:head# 2024-09-20T22:20:24.491 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8a37d8f:::benchmark_data_smithi060_196073_object11523:head# 2024-09-20T22:20:24.492 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8a399a2:::benchmark_data_smithi060_196073_object32613:head# 2024-09-20T22:20:24.492 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8a41fad:::benchmark_data_smithi060_196073_object37029:head# 2024-09-20T22:20:24.493 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8a4a42c:::benchmark_data_smithi060_196073_object47229:head# 2024-09-20T22:20:24.494 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8a689ca:::benchmark_data_smithi060_196073_object7548:head# 2024-09-20T22:20:24.494 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8a8dca5:::benchmark_data_smithi060_196073_object34601:head# 2024-09-20T22:20:24.495 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8aa3b9f:::benchmark_data_smithi060_196073_object21187:head# 2024-09-20T22:20:24.496 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8ab352b:::benchmark_data_smithi060_196073_object40592:head# 2024-09-20T22:20:24.496 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8ab4c61:::benchmark_data_smithi060_196073_object23371:head# 2024-09-20T22:20:24.497 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8ab9d13:::benchmark_data_smithi060_196073_object20232:head# 2024-09-20T22:20:24.497 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8ae06a0:::benchmark_data_smithi060_196073_object3794:head# 2024-09-20T22:20:24.498 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8ae6af9:::benchmark_data_smithi060_196073_object23392:head# 2024-09-20T22:20:24.498 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8af5de5:::benchmark_data_smithi060_196073_object473:head# 2024-09-20T22:20:24.499 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8b09772:::benchmark_data_smithi060_196073_object6248:head# 2024-09-20T22:20:24.500 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8b0b039:::benchmark_data_smithi060_196073_object51793:head# 2024-09-20T22:20:24.500 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8b15667:::benchmark_data_smithi060_196073_object23761:head# 2024-09-20T22:20:24.501 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8b1a2e3:::benchmark_data_smithi060_196073_object45307:head# 2024-09-20T22:20:24.501 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8b1dc12:::benchmark_data_smithi060_196073_object56782:head# 2024-09-20T22:20:24.502 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8b2fb09:::benchmark_data_smithi060_196073_object25132:head# 2024-09-20T22:20:24.502 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8b8af3b:::benchmark_data_smithi060_196073_object20809:head# 2024-09-20T22:20:24.503 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8baacc5:::benchmark_data_smithi060_196073_object6550:head# 2024-09-20T22:20:24.503 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8beca70:::benchmark_data_smithi060_196073_object27646:head# 2024-09-20T22:20:24.504 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8bee21c:::benchmark_data_smithi060_196073_object2642:head# 2024-09-20T22:20:24.504 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8bf11a7:::benchmark_data_smithi060_196073_object13447:head# 2024-09-20T22:20:24.505 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8bf8c2b:::benchmark_data_smithi060_196073_object5858:head# 2024-09-20T22:20:24.506 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:24.507 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:24.507 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:24.507 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:20:24.507 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:20:24.507 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:20:24.508 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:20:24.508 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:20:24.508 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8c09fbc:::benchmark_data_smithi060_196073_object33739:head# 2024-09-20T22:20:24.508 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8c0fc83:::benchmark_data_smithi060_196073_object14250:head# 2024-09-20T22:20:24.509 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8c255ab:::benchmark_data_smithi060_196073_object55844:head# 2024-09-20T22:20:24.509 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8c3dfc3:::benchmark_data_smithi060_196073_object11803:head# 2024-09-20T22:20:24.510 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8c4395e:::benchmark_data_smithi060_196073_object27144:head# 2024-09-20T22:20:24.511 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8c61d0a:::benchmark_data_smithi060_196073_object3838:head# 2024-09-20T22:20:24.511 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8c92385:::benchmark_data_smithi060_196073_object33935:head# 2024-09-20T22:20:24.512 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8c99acf:::benchmark_data_smithi060_196073_object2724:head# 2024-09-20T22:20:24.513 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8c9e5fb:::benchmark_data_smithi060_196073_object29368:head# 2024-09-20T22:20:24.513 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8cbe328:::benchmark_data_smithi060_196073_object46086:head# 2024-09-20T22:20:24.514 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8cd270b:::benchmark_data_smithi060_196073_object19713:head# 2024-09-20T22:20:24.515 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8cd4f2e:::benchmark_data_smithi060_196073_object54708:head# 2024-09-20T22:20:24.515 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8cdce40:::benchmark_data_smithi060_196073_object59046:head# 2024-09-20T22:20:24.516 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8cf01db:::benchmark_data_smithi060_196073_object29555:head# 2024-09-20T22:20:24.517 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8cf3b88:::benchmark_data_smithi060_196073_object15127:head# 2024-09-20T22:20:24.518 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8d12ba2:::benchmark_data_smithi060_196073_object15572:head# 2024-09-20T22:20:24.519 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8d349fb:::benchmark_data_smithi060_196073_object49928:head# 2024-09-20T22:20:24.519 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8d3adc6:::benchmark_data_smithi060_196073_object53423:head# 2024-09-20T22:20:24.520 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8d5aa1a:::benchmark_data_smithi060_196073_object47160:head# 2024-09-20T22:20:24.521 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8d7719a:::benchmark_data_smithi060_196073_object58758:head# 2024-09-20T22:20:24.521 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8d8b6ca:::benchmark_data_smithi060_196073_object21197:head# 2024-09-20T22:20:24.521 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8da08e5:::benchmark_data_smithi060_196073_object58415:head# 2024-09-20T22:20:24.523 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8da999a:::benchmark_data_smithi060_196073_object40370:head# 2024-09-20T22:20:24.523 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8db763c:::benchmark_data_smithi060_196073_object40472:head# 2024-09-20T22:20:24.528 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8dbbbdb:::benchmark_data_smithi060_196073_object6803:head# 2024-09-20T22:20:24.528 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8dcaad3:::benchmark_data_smithi060_196073_object27005:head# 2024-09-20T22:20:24.528 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8dcbc22:::benchmark_data_smithi060_196073_object1683:head# 2024-09-20T22:20:24.528 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8de1580:::benchmark_data_smithi060_196073_object14520:head# 2024-09-20T22:20:24.528 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8df1b20:::benchmark_data_smithi060_196073_object48306:head# 2024-09-20T22:20:24.528 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8df713e:::benchmark_data_smithi060_196073_object47856:head# 2024-09-20T22:20:24.531 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8e0a113:::benchmark_data_smithi060_196073_object47276:head# 2024-09-20T22:20:24.531 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8e27aee:::benchmark_data_smithi060_196073_object34757:head# 2024-09-20T22:20:24.533 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8e3b683:::benchmark_data_smithi060_196073_object20570:head# 2024-09-20T22:20:24.534 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8e482f1:::benchmark_data_smithi060_196073_object15230:head# 2024-09-20T22:20:24.534 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8e4baf2:::benchmark_data_smithi060_196073_object25544:head# 2024-09-20T22:20:24.535 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8e5b868:::benchmark_data_smithi060_196073_object1178:head# 2024-09-20T22:20:24.536 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8e7263c:::benchmark_data_smithi060_196073_object20050:head# 2024-09-20T22:20:24.536 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8e78436:::benchmark_data_smithi060_196073_object57629:head# 2024-09-20T22:20:24.538 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8eaa534:::benchmark_data_smithi060_196073_object22424:head# 2024-09-20T22:20:24.541 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8ec9063:::benchmark_data_smithi060_196073_object48374:head# 2024-09-20T22:20:24.541 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8ed6347:::benchmark_data_smithi060_196073_object35886:head# 2024-09-20T22:20:24.542 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8ef724b:::benchmark_data_smithi060_196073_object23624:head# 2024-09-20T22:20:24.543 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8f077ee:::benchmark_data_smithi060_196073_object34808:head# 2024-09-20T22:20:24.543 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8f2895f:::benchmark_data_smithi060_196073_object37211:head# 2024-09-20T22:20:24.544 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8f2da2b:::benchmark_data_smithi060_196073_object34758:head# 2024-09-20T22:20:24.545 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8f4544a:::benchmark_data_smithi060_196073_object17480:head# 2024-09-20T22:20:24.546 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8f4673a:::benchmark_data_smithi060_196073_object47698:head# 2024-09-20T22:20:24.547 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8f476db:::benchmark_data_smithi060_196073_object3984:head# 2024-09-20T22:20:24.548 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8f48364:::benchmark_data_smithi060_196073_object57947:head# 2024-09-20T22:20:24.549 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8f59a83:::benchmark_data_smithi060_196073_object45786:head# 2024-09-20T22:20:24.553 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8f7c32d:::benchmark_data_smithi060_196073_object45489:head# 2024-09-20T22:20:24.553 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8faef79:::benchmark_data_smithi060_196073_object45213:head# 2024-09-20T22:20:24.553 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8fc542f:::benchmark_data_smithi060_196073_object42278:head# 2024-09-20T22:20:24.553 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b8fcd953:::benchmark_data_smithi060_196073_object28561:head# 2024-09-20T22:20:24.554 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b90006d0:::benchmark_data_smithi060_196073_object9260:head# 2024-09-20T22:20:24.554 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9028474:::benchmark_data_smithi060_196073_object54350:head# 2024-09-20T22:20:24.557 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b907442c:::benchmark_data_smithi060_196073_object3626:head# 2024-09-20T22:20:24.559 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b908fa98:::benchmark_data_smithi060_196073_object43667:head# 2024-09-20T22:20:24.559 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b90af5c2:::benchmark_data_smithi060_196073_object8397:head# 2024-09-20T22:20:24.560 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b90e42df:::benchmark_data_smithi060_196073_object55733:head# 2024-09-20T22:20:24.561 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b91690f2:::benchmark_data_smithi060_196073_object51088:head# 2024-09-20T22:20:24.562 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9178296:::benchmark_data_smithi060_196073_object16715:head# 2024-09-20T22:20:24.566 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b91b8a7e:::benchmark_data_smithi060_196073_object37506:head# 2024-09-20T22:20:24.566 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b91c3e1f:::benchmark_data_smithi060_196073_object58991:head# 2024-09-20T22:20:24.566 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b91e6df8:::benchmark_data_smithi060_196073_object11402:head# 2024-09-20T22:20:24.566 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b91f9cb7:::benchmark_data_smithi060_196073_object5801:head# 2024-09-20T22:20:24.575 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9201e38:::benchmark_data_smithi060_196073_object54493:head# 2024-09-20T22:20:24.576 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9202a90:::benchmark_data_smithi060_196073_object37974:head# 2024-09-20T22:20:24.576 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9206974:::benchmark_data_smithi060_196073_object12573:head# 2024-09-20T22:20:24.576 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b92069a9:::benchmark_data_smithi060_196073_object47346:head# 2024-09-20T22:20:24.576 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b921a2b4:::benchmark_data_smithi060_196073_object34096:head# 2024-09-20T22:20:24.576 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9228740:::benchmark_data_smithi060_196073_object17834:head# 2024-09-20T22:20:24.576 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9238a21:::benchmark_data_smithi060_196073_object30412:head# 2024-09-20T22:20:24.576 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b924ddce:::benchmark_data_smithi060_196073_object28258:head# 2024-09-20T22:20:24.576 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b924e4e5:::benchmark_data_smithi060_196073_object5352:head# 2024-09-20T22:20:24.576 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b925985a:::benchmark_data_smithi060_196073_object26733:head# 2024-09-20T22:20:24.576 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9269ead:::benchmark_data_smithi060_196073_object56001:head# 2024-09-20T22:20:24.577 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b926f73e:::benchmark_data_smithi060_196073_object53266:head# 2024-09-20T22:20:24.578 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:24 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:20:24.578 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:24 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2030588030' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:20:24.582 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b929c54d:::benchmark_data_smithi060_196073_object45183:head# 2024-09-20T22:20:24.582 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b92a78d6:::benchmark_data_smithi060_196073_object18493:head# 2024-09-20T22:20:24.582 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b92c5cae:::benchmark_data_smithi060_196073_object10958:head# 2024-09-20T22:20:24.583 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b92ce3ac:::benchmark_data_smithi060_196073_object51300:head# 2024-09-20T22:20:24.583 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b92dada0:::benchmark_data_smithi060_196073_object44488:head# 2024-09-20T22:20:24.583 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b92e17a9:::benchmark_data_smithi060_196073_object17590:head# 2024-09-20T22:20:24.583 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b92ed993:::benchmark_data_smithi060_196073_object61745:head# 2024-09-20T22:20:24.583 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:20:24.584 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b92fd031:::benchmark_data_smithi060_196073_object801:head# 2024-09-20T22:20:24.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:24 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:20:24.584 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:24 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2030588030' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:20:24.584 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:24 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:20:24.584 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:24 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2030588030' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:20:24.590 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9310774:::benchmark_data_smithi060_196073_object47277:head# 2024-09-20T22:20:24.590 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b931cd6f:::benchmark_data_smithi060_196073_object54748:head# 2024-09-20T22:20:24.590 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9341f99:::benchmark_data_smithi060_196073_object13379:head# 2024-09-20T22:20:24.590 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b935429f:::benchmark_data_smithi060_196073_object59690:head# 2024-09-20T22:20:24.591 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9363330:::benchmark_data_smithi060_196073_object6067:head# 2024-09-20T22:20:24.591 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9369037:::benchmark_data_smithi060_196073_object8284:head# 2024-09-20T22:20:24.591 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9375cab:::benchmark_data_smithi060_196073_object36852:head# 2024-09-20T22:20:24.591 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b93805a7:::benchmark_data_smithi060_196073_object35033:head# 2024-09-20T22:20:24.591 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b938313e:::benchmark_data_smithi060_196073_object15761:head# 2024-09-20T22:20:24.591 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9384062:::benchmark_data_smithi060_196073_object8206:head# 2024-09-20T22:20:24.604 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b938e9c0:::benchmark_data_smithi060_196073_object25612:head# 2024-09-20T22:20:24.604 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b938ef00:::benchmark_data_smithi060_196073_object28851:head# 2024-09-20T22:20:24.604 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9395c31:::benchmark_data_smithi060_196073_object21200:head# 2024-09-20T22:20:24.604 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b93b3297:::benchmark_data_smithi060_196073_object46221:head# 2024-09-20T22:20:24.604 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b93c06dc:::benchmark_data_smithi060_196073_object51685:head# 2024-09-20T22:20:24.604 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b93c8762:::benchmark_data_smithi060_196073_object43514:head# 2024-09-20T22:20:24.604 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b93c922c:::benchmark_data_smithi060_196073_object23667:head# 2024-09-20T22:20:24.605 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b93e28bd:::benchmark_data_smithi060_196073_object1122:head# 2024-09-20T22:20:24.605 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b93e4c74:::benchmark_data_smithi060_196073_object45097:head# 2024-09-20T22:20:24.605 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b93ebcb2:::benchmark_data_smithi060_196073_object33152:head# 2024-09-20T22:20:24.605 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b93f9396:::benchmark_data_smithi060_196073_object50744:head# 2024-09-20T22:20:24.605 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b941c73e:::benchmark_data_smithi060_196073_object34649:head# 2024-09-20T22:20:24.605 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b941d5ef:::benchmark_data_smithi060_196073_object23469:head# 2024-09-20T22:20:24.605 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9424781:::benchmark_data_smithi060_196073_object3446:head# 2024-09-20T22:20:24.605 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b942b68d:::benchmark_data_smithi060_196073_object13241:head# 2024-09-20T22:20:24.605 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b942bc07:::benchmark_data_smithi060_196073_object48067:head# 2024-09-20T22:20:24.605 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9434817:::benchmark_data_smithi060_196073_object42014:head# 2024-09-20T22:20:24.609 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b945ec97:::benchmark_data_smithi060_196073_object15549:head# 2024-09-20T22:20:24.609 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b94630b0:::benchmark_data_smithi060_196073_object41352:head# 2024-09-20T22:20:24.609 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9472883:::benchmark_data_smithi060_196073_object25356:head# 2024-09-20T22:20:24.609 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9476456:::benchmark_data_smithi060_196073_object27073:head# 2024-09-20T22:20:24.609 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:24.610 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b94c611a:::benchmark_data_smithi060_196073_object9058:head# 2024-09-20T22:20:24.612 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9505516:::benchmark_data_smithi060_196073_object15393:head# 2024-09-20T22:20:24.618 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b950b947:::benchmark_data_smithi060_196073_object22746:head# 2024-09-20T22:20:24.618 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9518a07:::benchmark_data_smithi060_196073_object41994:head# 2024-09-20T22:20:24.618 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9531af1:::benchmark_data_smithi060_196073_object16734:head# 2024-09-20T22:20:24.618 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b95363f4:::benchmark_data_smithi060_196073_object14555:head# 2024-09-20T22:20:24.618 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b95366bd:::benchmark_data_smithi060_196073_object15304:head# 2024-09-20T22:20:24.618 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b95408da:::benchmark_data_smithi060_196073_object25432:head# 2024-09-20T22:20:24.618 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9551768:::benchmark_data_smithi060_196073_object34269:head# 2024-09-20T22:20:24.619 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b955cb28:::benchmark_data_smithi060_196073_object51981:head# 2024-09-20T22:20:24.622 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9563ad9:::benchmark_data_smithi060_196073_object58727:head# 2024-09-20T22:20:24.624 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9563f53:::benchmark_data_smithi060_196073_object7415:head# 2024-09-20T22:20:24.624 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b957928c:::benchmark_data_smithi060_196073_object23474:head# 2024-09-20T22:20:24.624 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9582dcf:::benchmark_data_smithi060_196073_object40341:head# 2024-09-20T22:20:24.624 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b959a9f3:::benchmark_data_smithi060_196073_object60534:head# 2024-09-20T22:20:24.625 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b95a9c3b:::benchmark_data_smithi060_196073_object37021:head# 2024-09-20T22:20:24.625 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b95abc44:::benchmark_data_smithi060_196073_object13449:head# 2024-09-20T22:20:24.626 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b95bebf3:::benchmark_data_smithi060_196073_object53061:head# 2024-09-20T22:20:24.626 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b95d18ac:::benchmark_data_smithi060_196073_object9791:head# 2024-09-20T22:20:24.627 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b95d573a:::benchmark_data_smithi060_196073_object19946:head# 2024-09-20T22:20:24.627 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b95e1611:::benchmark_data_smithi060_196073_object7196:head# 2024-09-20T22:20:24.628 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b95e16b8:::benchmark_data_smithi060_196073_object53851:head# 2024-09-20T22:20:24.629 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b95fcdc4:::benchmark_data_smithi060_196073_object57612:head# 2024-09-20T22:20:24.629 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b96070f0:::benchmark_data_smithi060_196073_object21957:head# 2024-09-20T22:20:24.629 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b960eb96:::benchmark_data_smithi060_196073_object45800:head# 2024-09-20T22:20:24.630 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9618b82:::benchmark_data_smithi060_196073_object12589:head# 2024-09-20T22:20:24.630 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b961fb2e:::benchmark_data_smithi060_196073_object38526:head# 2024-09-20T22:20:24.631 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b96390a0:::benchmark_data_smithi060_196073_object59347:head# 2024-09-20T22:20:24.632 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9662a3e:::benchmark_data_smithi060_196073_object27455:head# 2024-09-20T22:20:24.633 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9687beb:::benchmark_data_smithi060_196073_object40392:head# 2024-09-20T22:20:24.633 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b96c46e0:::benchmark_data_smithi060_196073_object58914:head# 2024-09-20T22:20:24.638 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b96cbe1d:::benchmark_data_smithi060_196073_object19535:head# 2024-09-20T22:20:24.638 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b96e6663:::benchmark_data_smithi060_196073_object16195:head# 2024-09-20T22:20:24.638 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b96ed098:::benchmark_data_smithi060_196073_object34847:head# 2024-09-20T22:20:24.640 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9715197:::benchmark_data_smithi060_196073_object37545:head# 2024-09-20T22:20:24.640 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b971f7d9:::benchmark_data_smithi060_196073_object45931:head# 2024-09-20T22:20:24.640 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9744e27:::benchmark_data_smithi060_196073_object6328:head# 2024-09-20T22:20:24.641 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9758c7e:::benchmark_data_smithi060_196073_object50951:head# 2024-09-20T22:20:24.641 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b975922f:::benchmark_data_smithi060_196073_object22927:head# 2024-09-20T22:20:24.642 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b975b286:::benchmark_data_smithi060_196073_object15130:head# 2024-09-20T22:20:24.644 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9763765:::benchmark_data_smithi060_196073_object4872:head# 2024-09-20T22:20:24.645 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b97718c8:::benchmark_data_smithi060_196073_object58546:head# 2024-09-20T22:20:24.648 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b978c6ec:::benchmark_data_smithi060_196073_object35709:head# 2024-09-20T22:20:24.648 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9790c29:::benchmark_data_smithi060_196073_object35142:head# 2024-09-20T22:20:24.648 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b97a6853:::benchmark_data_smithi060_196073_object58468:head# 2024-09-20T22:20:24.648 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b97bbe85:::benchmark_data_smithi060_196073_object7687:head# 2024-09-20T22:20:24.652 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b97c2b8c:::benchmark_data_smithi060_196073_object698:head# 2024-09-20T22:20:24.652 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b97c930d:::benchmark_data_smithi060_196073_object40550:head# 2024-09-20T22:20:24.652 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b97dde85:::benchmark_data_smithi060_196073_object41346:head# 2024-09-20T22:20:24.652 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b97e1fb5:::benchmark_data_smithi060_196073_object49211:head# 2024-09-20T22:20:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b97f0f59:::benchmark_data_smithi060_196073_object14252:head# 2024-09-20T22:20:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b982160a:::benchmark_data_smithi060_196073_object14970:head# 2024-09-20T22:20:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b982cda0:::benchmark_data_smithi060_196073_object27120:head# 2024-09-20T22:20:24.653 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9842bbd:::benchmark_data_smithi060_196073_object40181:head# 2024-09-20T22:20:24.654 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b984acb2:::benchmark_data_smithi060_196073_object57722:head# 2024-09-20T22:20:24.655 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b984e9c0:::benchmark_data_smithi060_196073_object60014:head# 2024-09-20T22:20:24.656 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:24.657 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:24.657 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:24.658 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9858a86:::benchmark_data_smithi060_196073_object13610:head# 2024-09-20T22:20:24.658 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b98690fc:::benchmark_data_smithi060_196073_object61552:head# 2024-09-20T22:20:24.658 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b986ddf4:::benchmark_data_smithi060_196073_object27103:head# 2024-09-20T22:20:24.658 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b98793c4:::benchmark_data_smithi060_196073_object16934:head# 2024-09-20T22:20:24.661 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b98ab4d0:::benchmark_data_smithi060_196073_object57219:head# 2024-09-20T22:20:24.661 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b98abab0:::benchmark_data_smithi060_196073_object28765:head# 2024-09-20T22:20:24.662 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b98ca8f7:::benchmark_data_smithi060_196073_object14386:head# 2024-09-20T22:20:24.663 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b98d1eab:::benchmark_data_smithi060_196073_object16436:head# 2024-09-20T22:20:24.664 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b98deebb:::benchmark_data_smithi060_196073_object33133:head# 2024-09-20T22:20:24.664 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b98e4b56:::benchmark_data_smithi060_196073_object26681:head# 2024-09-20T22:20:24.665 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b98e5002:::benchmark_data_smithi060_196073_object21098:head# 2024-09-20T22:20:24.666 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b98e644a:::benchmark_data_smithi060_196073_object17207:head# 2024-09-20T22:20:24.666 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b98f3909:::benchmark_data_smithi060_196073_object56306:head# 2024-09-20T22:20:24.666 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b991a0c2:::benchmark_data_smithi060_196073_object55394:head# 2024-09-20T22:20:24.667 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b991bb5a:::benchmark_data_smithi060_196073_object1957:head# 2024-09-20T22:20:24.671 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b991e77a:::benchmark_data_smithi060_196073_object7015:head# 2024-09-20T22:20:24.671 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b992fd9f:::benchmark_data_smithi060_196073_object34899:head# 2024-09-20T22:20:24.671 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b993797c:::benchmark_data_smithi060_196073_object9112:head# 2024-09-20T22:20:24.671 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b999148a:::benchmark_data_smithi060_196073_object42618:head# 2024-09-20T22:20:24.671 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9996be9:::benchmark_data_smithi060_196073_object15193:head# 2024-09-20T22:20:24.675 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b999e6b1:::benchmark_data_smithi060_196073_object31478:head# 2024-09-20T22:20:24.675 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b99a8bb5:::benchmark_data_smithi060_196073_object60147:head# 2024-09-20T22:20:24.675 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b99a9c52:::benchmark_data_smithi060_196073_object12925:head# 2024-09-20T22:20:24.675 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b99b20c6:::benchmark_data_smithi060_196073_object26799:head# 2024-09-20T22:20:24.675 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b99b5aac:::benchmark_data_smithi060_196073_object41557:head# 2024-09-20T22:20:24.691 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b99b7cfc:::benchmark_data_smithi060_196073_object19065:head# 2024-09-20T22:20:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b99d50fe:::benchmark_data_smithi060_196073_object30282:head# 2024-09-20T22:20:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b99dd65f:::benchmark_data_smithi060_196073_object61014:head# 2024-09-20T22:20:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b99e1fbb:::benchmark_data_smithi060_196073_object54254:head# 2024-09-20T22:20:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b99e642d:::benchmark_data_smithi060_196073_object39752:head# 2024-09-20T22:20:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b99e9569:::benchmark_data_smithi060_196073_object4730:head# 2024-09-20T22:20:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b99e9bc6:::benchmark_data_smithi060_196073_object8021:head# 2024-09-20T22:20:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b99ef744:::benchmark_data_smithi060_196073_object18595:head# 2024-09-20T22:20:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b99f9c4a:::benchmark_data_smithi060_196073_object22404:head# 2024-09-20T22:20:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b99fec63:::benchmark_data_smithi060_196073_object53869:head# 2024-09-20T22:20:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9a070c1:::benchmark_data_smithi060_196073_object26356:head# 2024-09-20T22:20:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9a08a4e:::benchmark_data_smithi060_196073_object19:head# 2024-09-20T22:20:24.692 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9a32e26:::benchmark_data_smithi060_196073_object7095:head# 2024-09-20T22:20:24.693 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9a37ab7:::benchmark_data_smithi060_196073_object46789:head# 2024-09-20T22:20:24.693 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9a468de:::benchmark_data_smithi060_196073_object59840:head# 2024-09-20T22:20:24.693 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9a4b0fc:::benchmark_data_smithi060_196073_object45869:head# 2024-09-20T22:20:24.693 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9a75ed5:::benchmark_data_smithi060_196073_object15833:head# 2024-09-20T22:20:24.693 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9a83d57:::benchmark_data_smithi060_196073_object29429:head# 2024-09-20T22:20:24.693 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9a863a9:::benchmark_data_smithi060_196073_object37769:head# 2024-09-20T22:20:24.693 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9a8f8a1:::benchmark_data_smithi060_196073_object6847:head# 2024-09-20T22:20:24.693 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9ab414a:::benchmark_data_smithi060_196073_object10757:head# 2024-09-20T22:20:24.693 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9abe9a9:::benchmark_data_smithi060_196073_object34900:head# 2024-09-20T22:20:24.698 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9ac5581:::benchmark_data_smithi060_196073_object59741:head# 2024-09-20T22:20:24.698 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9ad187d:::benchmark_data_smithi060_196073_object9747:head# 2024-09-20T22:20:24.698 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9aeb08e:::benchmark_data_smithi060_196073_object55501:head# 2024-09-20T22:20:24.698 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9aef59a:::benchmark_data_smithi060_196073_object6083:head# 2024-09-20T22:20:24.698 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9b04f86:::benchmark_data_smithi060_196073_object39541:head# 2024-09-20T22:20:24.699 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9b0d357:::benchmark_data_smithi060_196073_object37428:head# 2024-09-20T22:20:24.699 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9b16060:::benchmark_data_smithi060_196073_object31349:head# 2024-09-20T22:20:24.701 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9b3ed24:::benchmark_data_smithi060_196073_object49677:head# 2024-09-20T22:20:24.701 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9b4bc71:::benchmark_data_smithi060_196073_object17273:head# 2024-09-20T22:20:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9b64346:::benchmark_data_smithi060_196073_object42061:head# 2024-09-20T22:20:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9b7769f:::benchmark_data_smithi060_196073_object794:head# 2024-09-20T22:20:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9b7bd50:::benchmark_data_smithi060_196073_object32418:head# 2024-09-20T22:20:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9b92a52:::benchmark_data_smithi060_196073_object55663:head# 2024-09-20T22:20:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9bc7ea4:::benchmark_data_smithi060_196073_object35174:head# 2024-09-20T22:20:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9bc890a:::benchmark_data_smithi060_196073_object26959:head# 2024-09-20T22:20:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9bde130:::benchmark_data_smithi060_196073_object4134:head# 2024-09-20T22:20:24.707 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9c0d4bb:::benchmark_data_smithi060_196073_object30432:head# 2024-09-20T22:20:24.708 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9c39ee0:::benchmark_data_smithi060_196073_object6715:head# 2024-09-20T22:20:24.708 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9c3e469:::benchmark_data_smithi060_196073_object30298:head# 2024-09-20T22:20:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9c5cf2d:::benchmark_data_smithi060_196073_object57411:head# 2024-09-20T22:20:24.709 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9c829f5:::benchmark_data_smithi060_196073_object30245:head# 2024-09-20T22:20:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9c83f42:::benchmark_data_smithi060_196073_object38775:head# 2024-09-20T22:20:24.710 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9c88a83:::benchmark_data_smithi060_196073_object20670:head# 2024-09-20T22:20:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9c9e7d6:::benchmark_data_smithi060_196073_object38402:head# 2024-09-20T22:20:24.711 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9ca43a4:::benchmark_data_smithi060_196073_object42641:head# 2024-09-20T22:20:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9ca8a06:::benchmark_data_smithi060_196073_object6265:head# 2024-09-20T22:20:24.713 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9cbb8a6:::benchmark_data_smithi060_196073_object32260:head# 2024-09-20T22:20:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9cc9686:::benchmark_data_smithi060_196073_object24888:head# 2024-09-20T22:20:24.714 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9cde085:::benchmark_data_smithi060_196073_object46055:head# 2024-09-20T22:20:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9cebf68:::benchmark_data_smithi060_196073_object42417:head# 2024-09-20T22:20:24.715 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9cf3f73:::benchmark_data_smithi060_196073_object56908:head# 2024-09-20T22:20:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9cfa750:::benchmark_data_smithi060_196073_object42664:head# 2024-09-20T22:20:24.716 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9d433c8:::benchmark_data_smithi060_196073_object14892:head# 2024-09-20T22:20:24.717 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9d5bd8b:::benchmark_data_smithi060_196073_object14421:head# 2024-09-20T22:20:24.717 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9d6dbd3:::benchmark_data_smithi060_196073_object19339:head# 2024-09-20T22:20:24.718 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9d79b32:::benchmark_data_smithi060_196073_object12576:head# 2024-09-20T22:20:24.718 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9da01f9:::benchmark_data_smithi060_196073_object18583:head# 2024-09-20T22:20:24.719 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9db9252:::benchmark_data_smithi060_196073_object12392:head# 2024-09-20T22:20:24.719 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9dc8392:::benchmark_data_smithi060_196073_object33542:head# 2024-09-20T22:20:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9dcb291:::benchmark_data_smithi060_196073_object1041:head# 2024-09-20T22:20:24.720 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9df0608:::benchmark_data_smithi060_196073_object26202:head# 2024-09-20T22:20:24.721 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9dff7dc:::benchmark_data_smithi060_196073_object25182:head# 2024-09-20T22:20:24.721 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9e054c8:::benchmark_data_smithi060_196073_object8318:head# 2024-09-20T22:20:24.722 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9e06774:::benchmark_data_smithi060_196073_object10942:head# 2024-09-20T22:20:24.722 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9e1a1a9:::benchmark_data_smithi060_196073_object17195:head# 2024-09-20T22:20:24.723 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9e1f026:::benchmark_data_smithi060_196073_object38666:head# 2024-09-20T22:20:24.723 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9e2235b:::benchmark_data_smithi060_196073_object10952:head# 2024-09-20T22:20:24.724 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9e2b85b:::benchmark_data_smithi060_196073_object1890:head# 2024-09-20T22:20:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9e39b81:::benchmark_data_smithi060_196073_object33109:head# 2024-09-20T22:20:24.725 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9e3e2a5:::benchmark_data_smithi060_196073_object21124:head# 2024-09-20T22:20:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9e44fae:::benchmark_data_smithi060_196073_object35700:head# 2024-09-20T22:20:24.726 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9e58954:::benchmark_data_smithi060_196073_object41655:head# 2024-09-20T22:20:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9e75555:::benchmark_data_smithi060_196073_object56500:head# 2024-09-20T22:20:24.727 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9e82ef8:::benchmark_data_smithi060_196073_object7811:head# 2024-09-20T22:20:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9e8488b:::benchmark_data_smithi060_196073_object53473:head# 2024-09-20T22:20:24.728 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9e96723:::benchmark_data_smithi060_196073_object1987:head# 2024-09-20T22:20:24.729 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9e9cf2a:::benchmark_data_smithi060_196073_object48749:head# 2024-09-20T22:20:24.729 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9e9d366:::benchmark_data_smithi060_196073_object30700:head# 2024-09-20T22:20:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9ea5fe2:::benchmark_data_smithi060_196073_object31710:head# 2024-09-20T22:20:24.730 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9ea85c5:::benchmark_data_smithi060_196073_object59727:head# 2024-09-20T22:20:24.731 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9eba851:::benchmark_data_smithi060_196073_object14851:head# 2024-09-20T22:20:24.732 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9ed6556:::benchmark_data_smithi060_196073_object49490:head# 2024-09-20T22:20:24.732 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9ed7389:::benchmark_data_smithi060_196073_object14396:head# 2024-09-20T22:20:24.733 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9edf506:::benchmark_data_smithi060_196073_object37781:head# 2024-09-20T22:20:24.734 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9efdbc6:::benchmark_data_smithi060_196073_object42027:head# 2024-09-20T22:20:24.737 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9effd33:::benchmark_data_smithi060_196073_object48150:head# 2024-09-20T22:20:24.738 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9f0719b:::benchmark_data_smithi060_196073_object33346:head# 2024-09-20T22:20:24.738 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9f0a0fc:::benchmark_data_smithi060_196073_object21119:head# 2024-09-20T22:20:24.738 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9f3d752:::benchmark_data_smithi060_196073_object51418:head# 2024-09-20T22:20:24.746 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9f532e4:::benchmark_data_smithi060_196073_object21687:head# 2024-09-20T22:20:24.746 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9f53d44:::benchmark_data_smithi060_196073_object61636:head# 2024-09-20T22:20:24.746 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9f6c193:::benchmark_data_smithi060_196073_object19240:head# 2024-09-20T22:20:24.747 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9f7a6fd:::benchmark_data_smithi060_196073_object26542:head# 2024-09-20T22:20:24.747 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9f903d6:::benchmark_data_smithi060_196073_object52374:head# 2024-09-20T22:20:24.747 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9f9160d:::benchmark_data_smithi060_196073_object2551:head# 2024-09-20T22:20:24.747 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9f9f017:::benchmark_data_smithi060_196073_object1642:head# 2024-09-20T22:20:24.747 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9f9f05b:::benchmark_data_smithi060_196073_object47805:head# 2024-09-20T22:20:24.747 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9fa44f4:::benchmark_data_smithi060_196073_object2523:head# 2024-09-20T22:20:24.747 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9fae11f:::benchmark_data_smithi060_196073_object39953:head# 2024-09-20T22:20:24.747 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9fae60a:::benchmark_data_smithi060_196073_object53072:head# 2024-09-20T22:20:24.753 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9fb0f71:::benchmark_data_smithi060_196073_object24628:head# 2024-09-20T22:20:24.754 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9fb4c11:::benchmark_data_smithi060_196073_object58198:head# 2024-09-20T22:20:24.754 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9fc088c:::benchmark_data_smithi060_196073_object12931:head# 2024-09-20T22:20:24.754 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9fc478d:::benchmark_data_smithi060_196073_object41264:head# 2024-09-20T22:20:24.754 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9fca3a5:::benchmark_data_smithi060_196073_object3534:head# 2024-09-20T22:20:24.754 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9fd71aa:::benchmark_data_smithi060_196073_object24040:head# 2024-09-20T22:20:24.754 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9fddc0f:::benchmark_data_smithi060_196073_object42011:head# 2024-09-20T22:20:24.756 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9fddfd5:::benchmark_data_smithi060_196073_object21304:head# 2024-09-20T22:20:24.760 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9ff1b7d:::benchmark_data_smithi060_196073_object57494:head# 2024-09-20T22:20:24.760 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:b9ffa50f:::benchmark_data_smithi060_196073_object30251:head# 2024-09-20T22:20:24.760 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba03928a:::benchmark_data_smithi060_196073_object869:head# 2024-09-20T22:20:24.760 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba0449de:::benchmark_data_smithi060_196073_object28017:head# 2024-09-20T22:20:24.760 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:24.766 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba04a5dc:::benchmark_data_smithi060_196073_object50394:head# 2024-09-20T22:20:24.766 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba04ac3c:::benchmark_data_smithi060_196073_object3200:head# 2024-09-20T22:20:24.766 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba064320:::benchmark_data_smithi060_196073_object61189:head# 2024-09-20T22:20:24.766 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba077cb6:::benchmark_data_smithi060_196073_object35317:head# 2024-09-20T22:20:24.766 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba0be4ef:::benchmark_data_smithi060_196073_object13093:head# 2024-09-20T22:20:24.772 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba0c2ae8:::benchmark_data_smithi060_196073_object60002:head# 2024-09-20T22:20:24.772 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba0d244e:::benchmark_data_smithi060_196073_object34702:head# 2024-09-20T22:20:24.772 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba0ed990:::benchmark_data_smithi060_196073_object12679:head# 2024-09-20T22:20:24.772 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba0ee1a5:::benchmark_data_smithi060_196073_object27553:head# 2024-09-20T22:20:24.772 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba13bbc7:::benchmark_data_smithi060_196073_object51085:head# 2024-09-20T22:20:24.773 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba13f695:::benchmark_data_smithi060_196073_object36114:head# 2024-09-20T22:20:24.773 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba14ab5a:::benchmark_data_smithi060_196073_object28483:head# 2024-09-20T22:20:24.774 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba1707fa:::benchmark_data_smithi060_196073_object41098:head# 2024-09-20T22:20:24.774 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba18fbef:::benchmark_data_smithi060_196073_object7716:head# 2024-09-20T22:20:24.777 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba19018b:::benchmark_data_smithi060_196073_object12747:head# 2024-09-20T22:20:24.778 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba19db4f:::benchmark_data_smithi060_196073_object54101:head# 2024-09-20T22:20:24.778 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba1ad63c:::benchmark_data_smithi060_196073_object59105:head# 2024-09-20T22:20:24.779 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba1b6b72:::benchmark_data_smithi060_196073_object33787:head# 2024-09-20T22:20:24.782 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba1bcb81:::benchmark_data_smithi060_196073_object14523:head# 2024-09-20T22:20:24.782 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba1f5c2e:::benchmark_data_smithi060_196073_object48688:head# 2024-09-20T22:20:24.782 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba1fcf1a:::benchmark_data_smithi060_196073_object29803:head# 2024-09-20T22:20:24.783 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba2054c0:::benchmark_data_smithi060_196073_object52584:head# 2024-09-20T22:20:24.783 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba219c6a:::benchmark_data_smithi060_196073_object8031:head# 2024-09-20T22:20:24.783 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba21a4c6:::benchmark_data_smithi060_196073_object53437:head# 2024-09-20T22:20:24.784 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba2323dc:::benchmark_data_smithi060_196073_object16564:head# 2024-09-20T22:20:24.787 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba23e3c6:::benchmark_data_smithi060_196073_object46558:head# 2024-09-20T22:20:24.788 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba29cb3e:::benchmark_data_smithi060_196073_object7945:head# 2024-09-20T22:20:24.788 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba29e74a:::benchmark_data_smithi060_196073_object2111:head# 2024-09-20T22:20:24.788 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba2be85b:::benchmark_data_smithi060_196073_object51223:head# 2024-09-20T22:20:24.789 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba2c259a:::benchmark_data_smithi060_196073_object56084:head# 2024-09-20T22:20:24.789 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba2c9024:::benchmark_data_smithi060_196073_object23792:head# 2024-09-20T22:20:24.789 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba2cf814:::benchmark_data_smithi060_196073_object39749:head# 2024-09-20T22:20:24.794 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba2ddad3:::benchmark_data_smithi060_196073_object44146:head# 2024-09-20T22:20:24.794 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba2e210e:::benchmark_data_smithi060_196073_object22145:head# 2024-09-20T22:20:24.794 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba2fc604:::benchmark_data_smithi060_196073_object58282:head# 2024-09-20T22:20:24.795 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba300e42:::benchmark_data_smithi060_196073_object29818:head# 2024-09-20T22:20:24.795 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba3074ef:::benchmark_data_smithi060_196073_object35140:head# 2024-09-20T22:20:24.795 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba308aaf:::benchmark_data_smithi060_196073_object7918:head# 2024-09-20T22:20:24.796 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba30c1cf:::benchmark_data_smithi060_196073_object52548:head# 2024-09-20T22:20:24.796 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba30caa8:::benchmark_data_smithi060_196073_object6392:head# 2024-09-20T22:20:24.797 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba310c95:::benchmark_data_smithi060_196073_object34069:head# 2024-09-20T22:20:24.797 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba313f01:::benchmark_data_smithi060_196073_object18545:head# 2024-09-20T22:20:24.798 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:24.805 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:24.806 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:24.811 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba327a61:::benchmark_data_smithi060_196073_object25820:head# 2024-09-20T22:20:24.811 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba3330ac:::benchmark_data_smithi060_196073_object30623:head# 2024-09-20T22:20:24.811 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba360998:::benchmark_data_smithi060_196073_object35327:head# 2024-09-20T22:20:24.811 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba3b67b3:::benchmark_data_smithi060_196073_object10239:head# 2024-09-20T22:20:24.811 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba3d69ce:::benchmark_data_smithi060_196073_object20267:head# 2024-09-20T22:20:24.811 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba3e9d60:::benchmark_data_smithi060_196073_object53728:head# 2024-09-20T22:20:24.811 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba40193a:::benchmark_data_smithi060_196073_object7995:head# 2024-09-20T22:20:24.811 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba406f8b:::benchmark_data_smithi060_196073_object2876:head# 2024-09-20T22:20:24.811 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba40a12b:::benchmark_data_smithi060_196073_object33585:head# 2024-09-20T22:20:24.811 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba416d6f:::benchmark_data_smithi060_196073_object45430:head# 2024-09-20T22:20:24.811 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba41901a:::benchmark_data_smithi060_196073_object48366:head# 2024-09-20T22:20:24.812 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba419d95:::benchmark_data_smithi060_196073_object15772:head# 2024-09-20T22:20:24.812 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba41bee4:::benchmark_data_smithi060_196073_object6633:head# 2024-09-20T22:20:24.812 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba421772:::benchmark_data_smithi060_196073_object50960:head# 2024-09-20T22:20:24.812 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba444959:::benchmark_data_smithi060_196073_object39419:head# 2024-09-20T22:20:24.812 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba453ba5:::benchmark_data_smithi060_196073_object60019:head# 2024-09-20T22:20:24.812 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba472b1d:::benchmark_data_smithi060_196073_object557:head# 2024-09-20T22:20:24.812 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba473413:::benchmark_data_smithi060_196073_object7137:head# 2024-09-20T22:20:24.812 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba4bad9d:::benchmark_data_smithi060_196073_object2217:head# 2024-09-20T22:20:24.813 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba4c5b30:::benchmark_data_smithi060_196073_object51586:head# 2024-09-20T22:20:24.813 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba4da803:::benchmark_data_smithi060_196073_object44196:head# 2024-09-20T22:20:24.813 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba4e98b8:::benchmark_data_smithi060_196073_object17251:head# 2024-09-20T22:20:24.814 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba4ee986:::benchmark_data_smithi060_196073_object55559:head# 2024-09-20T22:20:24.814 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba523e78:::benchmark_data_smithi060_196073_object16173:head# 2024-09-20T22:20:24.815 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba5268b2:::benchmark_data_smithi060_196073_object50391:head# 2024-09-20T22:20:24.817 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba52fa5e:::benchmark_data_smithi060_196073_object18909:head# 2024-09-20T22:20:24.818 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba5357fd:::benchmark_data_smithi060_196073_object5080:head# 2024-09-20T22:20:24.819 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba569746:::benchmark_data_smithi060_196073_object43091:head# 2024-09-20T22:20:24.820 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba56f53e:::benchmark_data_smithi060_196073_object28723:head# 2024-09-20T22:20:24.821 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba577d55:::benchmark_data_smithi060_196073_object8465:head# 2024-09-20T22:20:24.822 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba592a95:::benchmark_data_smithi060_196073_object29055:head# 2024-09-20T22:20:24.822 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba5ab51b:::benchmark_data_smithi060_196073_object58875:head# 2024-09-20T22:20:24.823 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba5b18fb:::benchmark_data_smithi060_196073_object34100:head# 2024-09-20T22:20:24.826 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba5cfbe9:::benchmark_data_smithi060_196073_object50639:head# 2024-09-20T22:20:24.826 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba5d02de:::benchmark_data_smithi060_196073_object4455:head# 2024-09-20T22:20:24.827 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba5e9bfc:::benchmark_data_smithi060_196073_object52970:head# 2024-09-20T22:20:24.827 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba5ef08b:::benchmark_data_smithi060_196073_object17715:head# 2024-09-20T22:20:24.838 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba60595f:::benchmark_data_smithi060_196073_object9059:head# 2024-09-20T22:20:24.838 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba61dd67:::benchmark_data_smithi060_196073_object44670:head# 2024-09-20T22:20:24.838 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba626234:::benchmark_data_smithi060_196073_object56544:head# 2024-09-20T22:20:24.838 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba62c537:::benchmark_data_smithi060_196073_object29397:head# 2024-09-20T22:20:24.838 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba631589:::benchmark_data_smithi060_196073_object43710:head# 2024-09-20T22:20:24.838 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba658cc9:::benchmark_data_smithi060_196073_object16574:head# 2024-09-20T22:20:24.839 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba68088e:::benchmark_data_smithi060_196073_object54265:head# 2024-09-20T22:20:24.839 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba697d5e:::benchmark_data_smithi060_196073_object17216:head# 2024-09-20T22:20:24.839 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba6bbb6c:::benchmark_data_smithi060_196073_object16932:head# 2024-09-20T22:20:24.839 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba6bc824:::benchmark_data_smithi060_196073_object7612:head# 2024-09-20T22:20:24.839 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba6d11f5:::benchmark_data_smithi060_196073_object17379:head# 2024-09-20T22:20:24.839 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba6fdfa2:::benchmark_data_smithi060_196073_object33160:head# 2024-09-20T22:20:24.840 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba70dbbd:::benchmark_data_smithi060_196073_object55009:head# 2024-09-20T22:20:24.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba7156d1:::benchmark_data_smithi060_196073_object17704:head# 2024-09-20T22:20:24.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba71a71c:::benchmark_data_smithi060_196073_object54115:head# 2024-09-20T22:20:24.844 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba722ca6:::benchmark_data_smithi060_196073_object49529:head# 2024-09-20T22:20:24.845 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba741589:::benchmark_data_smithi060_196073_object14666:head# 2024-09-20T22:20:24.845 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba744924:::benchmark_data_smithi060_196073_object16584:head# 2024-09-20T22:20:24.846 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba748733:::benchmark_data_smithi060_196073_object41422:head# 2024-09-20T22:20:24.846 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba75d323:::benchmark_data_smithi060_196073_object38508:head# 2024-09-20T22:20:24.847 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba7745df:::benchmark_data_smithi060_196073_object38157:head# 2024-09-20T22:20:24.847 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba779426:::benchmark_data_smithi060_196073_object40553:head# 2024-09-20T22:20:24.849 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba77a5be:::benchmark_data_smithi060_196073_object12786:head# 2024-09-20T22:20:24.852 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba793676:::benchmark_data_smithi060_196073_object31282:head# 2024-09-20T22:20:24.852 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba7b99ce:::benchmark_data_smithi060_196073_object34902:head# 2024-09-20T22:20:24.853 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba7bbd2c:::benchmark_data_smithi060_196073_object29157:head# 2024-09-20T22:20:24.853 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba7c5da0:::benchmark_data_smithi060_196073_object61588:head# 2024-09-20T22:20:24.853 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba7da75b:::benchmark_data_smithi060_196073_object26064:head# 2024-09-20T22:20:24.853 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba7de81e:::benchmark_data_smithi060_196073_object38265:head# 2024-09-20T22:20:24.854 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba7e0287:::benchmark_data_smithi060_196073_object36262:head# 2024-09-20T22:20:24.854 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba7e1cdf:::benchmark_data_smithi060_196073_object44136:head# 2024-09-20T22:20:24.855 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba7fc97e:::benchmark_data_smithi060_196073_object36269:head# 2024-09-20T22:20:24.855 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba80163c:::benchmark_data_smithi060_196073_object57356:head# 2024-09-20T22:20:24.856 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba83747a:::benchmark_data_smithi060_196073_object24302:head# 2024-09-20T22:20:24.869 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba83bd5b:::benchmark_data_smithi060_196073_object5769:head# 2024-09-20T22:20:24.869 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba843aad:::benchmark_data_smithi060_196073_object37437:head# 2024-09-20T22:20:24.869 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba863dca:::benchmark_data_smithi060_196073_object58078:head# 2024-09-20T22:20:24.869 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba866b0b:::benchmark_data_smithi060_196073_object39073:head# 2024-09-20T22:20:24.869 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba868645:::benchmark_data_smithi060_196073_object15117:head# 2024-09-20T22:20:24.869 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba86c4ce:::benchmark_data_smithi060_196073_object54881:head# 2024-09-20T22:20:24.869 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba883fac:::benchmark_data_smithi060_196073_object2751:head# 2024-09-20T22:20:24.870 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba884c53:::benchmark_data_smithi060_196073_object3328:head# 2024-09-20T22:20:24.870 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba888cf8:::benchmark_data_smithi060_196073_object28489:head# 2024-09-20T22:20:24.870 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba89bfc6:::benchmark_data_smithi060_196073_object23445:head# 2024-09-20T22:20:24.870 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba8a091a:::benchmark_data_smithi060_196073_object58718:head# 2024-09-20T22:20:24.870 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba8a0f51:::benchmark_data_smithi060_196073_object49149:head# 2024-09-20T22:20:24.870 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba8a7d30:::benchmark_data_smithi060_196073_object61039:head# 2024-09-20T22:20:24.870 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba8aa5e1:::benchmark_data_smithi060_196073_object33966:head# 2024-09-20T22:20:24.870 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba8ab291:::benchmark_data_smithi060_196073_object14221:head# 2024-09-20T22:20:24.870 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba8ae651:::benchmark_data_smithi060_196073_object23438:head# 2024-09-20T22:20:24.870 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba8aefee:::benchmark_data_smithi060_196073_object39407:head# 2024-09-20T22:20:24.870 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba8c84b9:::benchmark_data_smithi060_196073_object53312:head# 2024-09-20T22:20:24.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba8ccd9a:::benchmark_data_smithi060_196073_object44813:head# 2024-09-20T22:20:24.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba8d836a:::benchmark_data_smithi060_196073_object47341:head# 2024-09-20T22:20:24.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba8e2021:::benchmark_data_smithi060_196073_object40287:head# 2024-09-20T22:20:24.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba8f819c:::benchmark_data_smithi060_196073_object17312:head# 2024-09-20T22:20:24.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba8fa256:::benchmark_data_smithi060_196073_object52388:head# 2024-09-20T22:20:24.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba9181ba:::benchmark_data_smithi060_196073_object55366:head# 2024-09-20T22:20:24.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba94a586:::benchmark_data_smithi060_196073_object21785:head# 2024-09-20T22:20:24.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba967006:::benchmark_data_smithi060_196073_object19175:head# 2024-09-20T22:20:24.879 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba97fbfc:::benchmark_data_smithi060_196073_object35399:head# 2024-09-20T22:20:24.879 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba984eaf:::benchmark_data_smithi060_196073_object5152:head# 2024-09-20T22:20:24.886 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba985a3a:::benchmark_data_smithi060_196073_object35870:head# 2024-09-20T22:20:24.886 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba99878d:::benchmark_data_smithi060_196073_object26278:head# 2024-09-20T22:20:24.886 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba9b15fb:::benchmark_data_smithi060_196073_object52948:head# 2024-09-20T22:20:24.886 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba9b6405:::benchmark_data_smithi060_196073_object25884:head# 2024-09-20T22:20:24.886 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba9b7b0c:::benchmark_data_smithi060_196073_object55764:head# 2024-09-20T22:20:24.887 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba9c0d04:::benchmark_data_smithi060_196073_object11301:head# 2024-09-20T22:20:24.887 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba9c26a9:::benchmark_data_smithi060_196073_object18423:head# 2024-09-20T22:20:24.887 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba9c3c11:::benchmark_data_smithi060_196073_object31612:head# 2024-09-20T22:20:24.887 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:ba9f513e:::benchmark_data_smithi060_196073_object19696:head# 2024-09-20T22:20:24.887 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baa0e0a6:::benchmark_data_smithi060_196073_object32351:head# 2024-09-20T22:20:24.892 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baa16a0d:::benchmark_data_smithi060_196073_object49569:head# 2024-09-20T22:20:24.892 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baa1dc27:::benchmark_data_smithi060_196073_object56049:head# 2024-09-20T22:20:24.892 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baa24fe2:::benchmark_data_smithi060_196073_object14648:head# 2024-09-20T22:20:24.892 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baa254ce:::benchmark_data_smithi060_196073_object15541:head# 2024-09-20T22:20:24.892 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baa268e0:::benchmark_data_smithi060_196073_object49975:head# 2024-09-20T22:20:24.893 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baa28ce9:::benchmark_data_smithi060_196073_object34325:head# 2024-09-20T22:20:24.893 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baa29e01:::benchmark_data_smithi060_196073_object58247:head# 2024-09-20T22:20:24.895 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baa67a56:::benchmark_data_smithi060_196073_object43520:head# 2024-09-20T22:20:24.899 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baa67f64:::benchmark_data_smithi060_196073_object872:head# 2024-09-20T22:20:24.899 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baa7c0ac:::benchmark_data_smithi060_196073_object44206:head# 2024-09-20T22:20:24.899 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baaa3fef:::benchmark_data_smithi060_196073_object56523:head# 2024-09-20T22:20:24.905 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baab2368:::benchmark_data_smithi060_196073_object1299:head# 2024-09-20T22:20:24.905 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baab4daa:::benchmark_data_smithi060_196073_object24108:head# 2024-09-20T22:20:24.905 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baac32e0:::benchmark_data_smithi060_196073_object10711:head# 2024-09-20T22:20:24.905 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baac7934:::benchmark_data_smithi060_196073_object17691:head# 2024-09-20T22:20:24.905 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baacd67b:::benchmark_data_smithi060_196073_object12897:head# 2024-09-20T22:20:24.905 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baad40d4:::benchmark_data_smithi060_196073_object23751:head# 2024-09-20T22:20:24.906 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baae478a:::benchmark_data_smithi060_196073_object47204:head# 2024-09-20T22:20:24.910 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baaf7379:::benchmark_data_smithi060_196073_object31749:head# 2024-09-20T22:20:24.911 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baaf8ea6:::benchmark_data_smithi060_196073_object30815:head# 2024-09-20T22:20:24.911 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bab0424e:::benchmark_data_smithi060_196073_object44545:head# 2024-09-20T22:20:24.911 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bab0874b:::benchmark_data_smithi060_196073_object47703:head# 2024-09-20T22:20:24.911 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bab179d7:::benchmark_data_smithi060_196073_object57988:head# 2024-09-20T22:20:24.911 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bab20397:::benchmark_data_smithi060_196073_object60684:head# 2024-09-20T22:20:24.911 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bab4ccd4:::benchmark_data_smithi060_196073_object31746:head# 2024-09-20T22:20:24.911 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:24.921 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bab4eee8:::benchmark_data_smithi060_196073_object1505:head# 2024-09-20T22:20:24.921 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bab4fc01:::benchmark_data_smithi060_196073_object4428:head# 2024-09-20T22:20:24.921 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bab75272:::benchmark_data_smithi060_196073_object16660:head# 2024-09-20T22:20:24.921 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bab7e36e:::benchmark_data_smithi060_196073_object22740:head# 2024-09-20T22:20:24.921 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bab84810:::benchmark_data_smithi060_196073_object11052:head# 2024-09-20T22:20:24.921 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baba74bc:::benchmark_data_smithi060_196073_object10936:head# 2024-09-20T22:20:24.921 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:babc44eb:::benchmark_data_smithi060_196073_object11435:head# 2024-09-20T22:20:24.921 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:babd59e0:::benchmark_data_smithi060_196073_object47560:head# 2024-09-20T22:20:24.922 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:babda761:::benchmark_data_smithi060_196073_object3359:head# 2024-09-20T22:20:24.922 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:babea306:::benchmark_data_smithi060_196073_object47779:head# 2024-09-20T22:20:24.922 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:babf1091:::benchmark_data_smithi060_196073_object55465:head# 2024-09-20T22:20:24.922 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:babf9880:::benchmark_data_smithi060_196073_object36874:head# 2024-09-20T22:20:24.922 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:babfa2dd:::benchmark_data_smithi060_196073_object39588:head# 2024-09-20T22:20:24.922 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bac164f8:::benchmark_data_smithi060_196073_object39306:head# 2024-09-20T22:20:24.922 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bac16c70:::benchmark_data_smithi060_196073_object37043:head# 2024-09-20T22:20:24.923 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bac2b047:::benchmark_data_smithi060_196073_object11967:head# 2024-09-20T22:20:24.924 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bac383f6:::benchmark_data_smithi060_196073_object21841:head# 2024-09-20T22:20:24.926 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bac416b1:::benchmark_data_smithi060_196073_object26926:head# 2024-09-20T22:20:24.930 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bac605fa:::benchmark_data_smithi060_196073_object37367:head# 2024-09-20T22:20:24.930 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bac6be54:::benchmark_data_smithi060_196073_object12034:head# 2024-09-20T22:20:24.930 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bac7d41b:::benchmark_data_smithi060_196073_object12462:head# 2024-09-20T22:20:24.930 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bac8bea0:::benchmark_data_smithi060_196073_object33393:head# 2024-09-20T22:20:24.930 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bac9efe3:::benchmark_data_smithi060_196073_object3969:head# 2024-09-20T22:20:24.931 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baca7eae:::benchmark_data_smithi060_196073_object34994:head# 2024-09-20T22:20:24.931 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bacaa223:::benchmark_data_smithi060_196073_object15653:head# 2024-09-20T22:20:24.932 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bacd31d9:::benchmark_data_smithi060_196073_object40088:head# 2024-09-20T22:20:24.934 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bace0f84:::benchmark_data_smithi060_196073_object55833:head# 2024-09-20T22:20:24.934 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bace93fd:::benchmark_data_smithi060_196073_object44420:head# 2024-09-20T22:20:24.934 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bacebdba:::benchmark_data_smithi060_196073_object27542:head# 2024-09-20T22:20:24.935 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bad16378:::benchmark_data_smithi060_196073_object26544:head# 2024-09-20T22:20:24.936 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bad2463b:::benchmark_data_smithi060_196073_object29753:head# 2024-09-20T22:20:24.936 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bad272e7:::benchmark_data_smithi060_196073_object57515:head# 2024-09-20T22:20:24.937 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bad3738d:::benchmark_data_smithi060_196073_object18229:head# 2024-09-20T22:20:24.938 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bad3b104:::benchmark_data_smithi060_196073_object31499:head# 2024-09-20T22:20:24.939 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bad63051:::benchmark_data_smithi060_196073_object34739:head# 2024-09-20T22:20:24.940 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bad6bf0c:::benchmark_data_smithi060_196073_object28119:head# 2024-09-20T22:20:24.942 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bad88581:::benchmark_data_smithi060_196073_object34681:head# 2024-09-20T22:20:24.942 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bada9855:::benchmark_data_smithi060_196073_object46473:head# 2024-09-20T22:20:24.943 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:badeacd1:::benchmark_data_smithi060_196073_object40471:head# 2024-09-20T22:20:24.943 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:badf0b13:::benchmark_data_smithi060_196073_object7604:head# 2024-09-20T22:20:24.944 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:badf2ddb:::benchmark_data_smithi060_196073_object39640:head# 2024-09-20T22:20:24.945 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bae07119:::benchmark_data_smithi060_196073_object32796:head# 2024-09-20T22:20:24.945 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bae33c74:::benchmark_data_smithi060_196073_object23795:head# 2024-09-20T22:20:24.946 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bae40d2e:::benchmark_data_smithi060_196073_object15124:head# 2024-09-20T22:20:24.946 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:24.947 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bae477d6:::benchmark_data_smithi060_196073_object30534:head# 2024-09-20T22:20:24.947 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bae490a0:::benchmark_data_smithi060_196073_object8915:head# 2024-09-20T22:20:24.948 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:24.948 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:24.950 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bae6ddc9:::benchmark_data_smithi060_196073_object43998:head# 2024-09-20T22:20:24.950 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bae7cb02:::benchmark_data_smithi060_196073_object19785:head# 2024-09-20T22:20:24.950 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bae8b617:::benchmark_data_smithi060_196073_object22986:head# 2024-09-20T22:20:24.950 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bae9b9ed:::benchmark_data_smithi060_196073_object8620:head# 2024-09-20T22:20:24.951 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baea4e20:::benchmark_data_smithi060_196073_object52054:head# 2024-09-20T22:20:24.952 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baeb6f1d:::benchmark_data_smithi060_196073_object25769:head# 2024-09-20T22:20:24.952 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baece6fb:::benchmark_data_smithi060_196073_object30625:head# 2024-09-20T22:20:24.953 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baf340d4:::benchmark_data_smithi060_196073_object5919:head# 2024-09-20T22:20:24.954 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baf57b83:::benchmark_data_smithi060_196073_object6936:head# 2024-09-20T22:20:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baf72c64:::benchmark_data_smithi060_196073_object28956:head# 2024-09-20T22:20:24.955 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baf76cde:::benchmark_data_smithi060_196073_object59878:head# 2024-09-20T22:20:24.956 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baf94f60:::benchmark_data_smithi060_196073_object37839:head# 2024-09-20T22:20:24.957 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:baf99a4d:::benchmark_data_smithi060_196073_object45724:head# 2024-09-20T22:20:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bafa2b11:::benchmark_data_smithi060_196073_object19064:head# 2024-09-20T22:20:24.958 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bafb36da:::benchmark_data_smithi060_196073_object31562:head# 2024-09-20T22:20:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bafc3618:::benchmark_data_smithi060_196073_object33105:head# 2024-09-20T22:20:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bafcd38b:::benchmark_data_smithi060_196073_object42523:head# 2024-09-20T22:20:24.961 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb0279dc:::benchmark_data_smithi060_196073_object34392:head# 2024-09-20T22:20:24.962 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb02e7a6:::benchmark_data_smithi060_196073_object50392:head# 2024-09-20T22:20:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb04be83:::benchmark_data_smithi060_196073_object1785:head# 2024-09-20T22:20:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb04cc95:::benchmark_data_smithi060_196073_object28670:head# 2024-09-20T22:20:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb09dd39:::benchmark_data_smithi060_196073_object8047:head# 2024-09-20T22:20:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb0a6db5:::benchmark_data_smithi060_196073_object17836:head# 2024-09-20T22:20:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb0a9911:::benchmark_data_smithi060_196073_object30803:head# 2024-09-20T22:20:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb0c2c88:::benchmark_data_smithi060_196073_object37957:head# 2024-09-20T22:20:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb0c62be:::benchmark_data_smithi060_196073_object31895:head# 2024-09-20T22:20:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb0cda61:::benchmark_data_smithi060_196073_object55380:head# 2024-09-20T22:20:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb0d9a04:::benchmark_data_smithi060_196073_object15953:head# 2024-09-20T22:20:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb0e25c3:::benchmark_data_smithi060_196073_object1252:head# 2024-09-20T22:20:24.972 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb0e8a81:::benchmark_data_smithi060_196073_object12509:head# 2024-09-20T22:20:24.976 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb0ef24c:::benchmark_data_smithi060_196073_object17169:head# 2024-09-20T22:20:24.977 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb0f0c8d:::benchmark_data_smithi060_196073_object79:head# 2024-09-20T22:20:24.977 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb0f5721:::benchmark_data_smithi060_196073_object34698:head# 2024-09-20T22:20:24.977 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb0ffc56:::benchmark_data_smithi060_196073_object12372:head# 2024-09-20T22:20:24.977 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb108cbd:::benchmark_data_smithi060_196073_object8328:head# 2024-09-20T22:20:24.978 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb10f295:::benchmark_data_smithi060_196073_object47643:head# 2024-09-20T22:20:24.981 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb132b63:::benchmark_data_smithi060_196073_object6745:head# 2024-09-20T22:20:24.981 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb133bab:::benchmark_data_smithi060_196073_object16489:head# 2024-09-20T22:20:24.982 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb14a501:::benchmark_data_smithi060_196073_object2664:head# 2024-09-20T22:20:24.983 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb150b51:::benchmark_data_smithi060_196073_object10828:head# 2024-09-20T22:20:24.984 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb15a23e:::benchmark_data_smithi060_196073_object21495:head# 2024-09-20T22:20:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb163528:::benchmark_data_smithi060_196073_object48046:head# 2024-09-20T22:20:24.985 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb164380:::benchmark_data_smithi060_196073_object31766:head# 2024-09-20T22:20:24.986 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb1a1197:::benchmark_data_smithi060_196073_object57659:head# 2024-09-20T22:20:24.987 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb1b87ef:::benchmark_data_smithi060_196073_object13483:head# 2024-09-20T22:20:24.988 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb1c6d62:::benchmark_data_smithi060_196073_object55141:head# 2024-09-20T22:20:24.989 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb1d34e8:::benchmark_data_smithi060_196073_object22233:head# 2024-09-20T22:20:24.990 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb1d6f4f:::benchmark_data_smithi060_196073_object41467:head# 2024-09-20T22:20:24.990 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb1df91b:::benchmark_data_smithi060_196073_object13828:head# 2024-09-20T22:20:24.991 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb1fb2b2:::benchmark_data_smithi060_196073_object45945:head# 2024-09-20T22:20:24.992 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb1fc7aa:::benchmark_data_smithi060_196073_object48441:head# 2024-09-20T22:20:24.993 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb213cdb:::benchmark_data_smithi060_196073_object37661:head# 2024-09-20T22:20:24.993 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb2178e2:::benchmark_data_smithi060_196073_object12181:head# 2024-09-20T22:20:24.993 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb21a8c0:::benchmark_data_smithi060_196073_object33395:head# 2024-09-20T22:20:24.994 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb21e115:::benchmark_data_smithi060_196073_object46082:head# 2024-09-20T22:20:24.994 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb239abc:::benchmark_data_smithi060_196073_object2815:head# 2024-09-20T22:20:24.995 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb24749c:::benchmark_data_smithi060_196073_object56707:head# 2024-09-20T22:20:24.995 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb24dd80:::benchmark_data_smithi060_196073_object3055:head# 2024-09-20T22:20:24.996 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb2754d7:::benchmark_data_smithi060_196073_object33730:head# 2024-09-20T22:20:24.998 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb2760d8:::benchmark_data_smithi060_196073_object32684:head# 2024-09-20T22:20:24.999 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb2960b7:::benchmark_data_smithi060_196073_object49937:head# 2024-09-20T22:20:25.002 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb29ffb4:::benchmark_data_smithi060_196073_object34337:head# 2024-09-20T22:20:25.002 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb2abae0:::benchmark_data_smithi060_196073_object39906:head# 2024-09-20T22:20:25.002 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb2b47c7:::benchmark_data_smithi060_196073_object46147:head# 2024-09-20T22:20:25.002 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb2cb30b:::benchmark_data_smithi060_196073_object29974:head# 2024-09-20T22:20:25.003 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb2e7178:::benchmark_data_smithi060_196073_object37033:head# 2024-09-20T22:20:25.003 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb2ebce3:::benchmark_data_smithi060_196073_object12439:head# 2024-09-20T22:20:25.004 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb315ff7:::benchmark_data_smithi060_196073_object17202:head# 2024-09-20T22:20:25.005 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb330bdd:::benchmark_data_smithi060_196073_object59161:head# 2024-09-20T22:20:25.006 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb333cd4:::benchmark_data_smithi060_196073_object33939:head# 2024-09-20T22:20:25.009 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb34052f:::benchmark_data_smithi060_196073_object18864:head# 2024-09-20T22:20:25.010 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb39ff7a:::benchmark_data_smithi060_196073_object4683:head# 2024-09-20T22:20:25.011 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb3cb8ac:::benchmark_data_smithi060_196073_object18128:head# 2024-09-20T22:20:25.011 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb3d4036:::benchmark_data_smithi060_196073_object43885:head# 2024-09-20T22:20:25.016 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb3f7e11:::benchmark_data_smithi060_196073_object58713:head# 2024-09-20T22:20:25.016 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb3f8b71:::benchmark_data_smithi060_196073_object30159:head# 2024-09-20T22:20:25.016 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb3ff54b:::benchmark_data_smithi060_196073_object14506:head# 2024-09-20T22:20:25.016 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb40f1b7:::benchmark_data_smithi060_196073_object7761:head# 2024-09-20T22:20:25.016 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb4204ad:::benchmark_data_smithi060_196073_object46075:head# 2024-09-20T22:20:25.020 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb420ec3:::benchmark_data_smithi060_196073_object38641:head# 2024-09-20T22:20:25.021 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb441ca6:::benchmark_data_smithi060_196073_object13776:head# 2024-09-20T22:20:25.021 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb45216b:::benchmark_data_smithi060_196073_object36692:head# 2024-09-20T22:20:25.021 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb4570a2:::benchmark_data_smithi060_196073_object17630:head# 2024-09-20T22:20:25.026 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb45c9c9:::benchmark_data_smithi060_196073_object60595:head# 2024-09-20T22:20:25.026 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb47e7ce:::benchmark_data_smithi060_196073_object35287:head# 2024-09-20T22:20:25.026 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb496a00:::benchmark_data_smithi060_196073_object59659:head# 2024-09-20T22:20:25.026 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb49ce7b:::benchmark_data_smithi060_196073_object28396:head# 2024-09-20T22:20:25.026 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb4a4682:::benchmark_data_smithi060_196073_object32945:head# 2024-09-20T22:20:25.026 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb4a8a1a:::benchmark_data_smithi060_196073_object30631:head# 2024-09-20T22:20:25.026 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb4b469c:::benchmark_data_smithi060_196073_object23165:head# 2024-09-20T22:20:25.029 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb4c4087:::benchmark_data_smithi060_196073_object8792:head# 2024-09-20T22:20:25.029 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb4da3ca:::benchmark_data_smithi060_196073_object6916:head# 2024-09-20T22:20:25.029 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb4f9f8a:::benchmark_data_smithi060_196073_object25711:head# 2024-09-20T22:20:25.029 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb4fa7ee:::benchmark_data_smithi060_196073_object9980:head# 2024-09-20T22:20:25.031 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb4fa871:::benchmark_data_smithi060_196073_object12066:head# 2024-09-20T22:20:25.031 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb517d06:::benchmark_data_smithi060_196073_object7900:head# 2024-09-20T22:20:25.032 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb51c69f:::benchmark_data_smithi060_196073_object32080:head# 2024-09-20T22:20:25.036 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb52dd92:::benchmark_data_smithi060_196073_object59108:head# 2024-09-20T22:20:25.036 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb531ae1:::benchmark_data_smithi060_196073_object16333:head# 2024-09-20T22:20:25.037 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb54322a:::benchmark_data_smithi060_196073_object6630:head# 2024-09-20T22:20:25.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb563288:::benchmark_data_smithi060_196073_object17075:head# 2024-09-20T22:20:25.038 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb583611:::benchmark_data_smithi060_196073_object12792:head# 2024-09-20T22:20:25.040 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb588bf5:::benchmark_data_smithi060_196073_object42371:head# 2024-09-20T22:20:25.040 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb5c2898:::benchmark_data_smithi060_196073_object4336:head# 2024-09-20T22:20:25.046 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb5d8761:::benchmark_data_smithi060_196073_object53366:head# 2024-09-20T22:20:25.046 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb5d90ab:::benchmark_data_smithi060_196073_object34025:head# 2024-09-20T22:20:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb5e13fb:::benchmark_data_smithi060_196073_object33272:head# 2024-09-20T22:20:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb60b240:::benchmark_data_smithi060_196073_object49234:head# 2024-09-20T22:20:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb60c140:::benchmark_data_smithi060_196073_object59940:head# 2024-09-20T22:20:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb61953b:::benchmark_data_smithi060_196073_object18848:head# 2024-09-20T22:20:25.047 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb62d8fd:::benchmark_data_smithi060_196073_object25206:head# 2024-09-20T22:20:25.049 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb639687:::benchmark_data_smithi060_196073_object59526:head# 2024-09-20T22:20:25.049 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb646aa5:::benchmark_data_smithi060_196073_object44604:head# 2024-09-20T22:20:25.049 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb65e2f0:::benchmark_data_smithi060_196073_object495:head# 2024-09-20T22:20:25.049 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb6656d5:::benchmark_data_smithi060_196073_object46937:head# 2024-09-20T22:20:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb68744f:::benchmark_data_smithi060_196073_object12531:head# 2024-09-20T22:20:25.054 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb6a3877:::benchmark_data_smithi060_196073_object2593:head# 2024-09-20T22:20:25.055 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb6ae4a2:::benchmark_data_smithi060_196073_object38780:head# 2024-09-20T22:20:25.055 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb6b6d19:::benchmark_data_smithi060_196073_object36149:head# 2024-09-20T22:20:25.056 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb6b8073:::benchmark_data_smithi060_196073_object22652:head# 2024-09-20T22:20:25.057 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb6b9bca:::benchmark_data_smithi060_196073_object50174:head# 2024-09-20T22:20:25.057 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb6d4117:::benchmark_data_smithi060_196073_object44440:head# 2024-09-20T22:20:25.058 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb6d43cb:::benchmark_data_smithi060_196073_object33913:head# 2024-09-20T22:20:25.059 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb6e6a3a:::benchmark_data_smithi060_196073_object49274:head# 2024-09-20T22:20:25.059 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb6e9980:::benchmark_data_smithi060_196073_object40766:head# 2024-09-20T22:20:25.060 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb6f4445:::benchmark_data_smithi060_196073_object30385:head# 2024-09-20T22:20:25.061 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb6fbf1b:::benchmark_data_smithi060_196073_object7929:head# 2024-09-20T22:20:25.064 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb702bab:::benchmark_data_smithi060_196073_object41526:head# 2024-09-20T22:20:25.064 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb725ec8:::benchmark_data_smithi060_196073_object59236:head# 2024-09-20T22:20:25.067 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb729b38:::benchmark_data_smithi060_196073_object25716:head# 2024-09-20T22:20:25.067 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb7317bc:::benchmark_data_smithi060_196073_object56866:head# 2024-09-20T22:20:25.068 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb747293:::benchmark_data_smithi060_196073_object3379:head# 2024-09-20T22:20:25.070 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb74c8a4:::benchmark_data_smithi060_196073_object37263:head# 2024-09-20T22:20:25.070 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb77bc5b:::benchmark_data_smithi060_196073_object12196:head# 2024-09-20T22:20:25.070 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb7972b4:::benchmark_data_smithi060_196073_object34135:head# 2024-09-20T22:20:25.072 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb79e732:::benchmark_data_smithi060_196073_object24266:head# 2024-09-20T22:20:25.072 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb7fe6d3:::benchmark_data_smithi060_196073_object36233:head# 2024-09-20T22:20:25.073 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb8051e9:::benchmark_data_smithi060_196073_object1374:head# 2024-09-20T22:20:25.073 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb80a544:::benchmark_data_smithi060_196073_object59458:head# 2024-09-20T22:20:25.077 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb80dcbd:::benchmark_data_smithi060_196073_object14263:head# 2024-09-20T22:20:25.077 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb80f7a4:::benchmark_data_smithi060_196073_object35229:head# 2024-09-20T22:20:25.077 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb82cc44:::benchmark_data_smithi060_196073_object2389:head# 2024-09-20T22:20:25.077 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb84150b:::benchmark_data_smithi060_196073_object16272:head# 2024-09-20T22:20:25.077 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb841d1d:::benchmark_data_smithi060_196073_object24647:head# 2024-09-20T22:20:25.078 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb861ca8:::benchmark_data_smithi060_196073_object38911:head# 2024-09-20T22:20:25.080 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb864169:::benchmark_data_smithi060_196073_object21737:head# 2024-09-20T22:20:25.082 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb86617e:::benchmark_data_smithi060_196073_object9821:head# 2024-09-20T22:20:25.082 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb868dee:::benchmark_data_smithi060_196073_object44253:head# 2024-09-20T22:20:25.083 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb8c9d28:::benchmark_data_smithi060_196073_object12546:head# 2024-09-20T22:20:25.084 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb8cc1f2:::benchmark_data_smithi060_196073_object60290:head# 2024-09-20T22:20:25.087 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb8d3d5a:::benchmark_data_smithi060_196073_object32283:head# 2024-09-20T22:20:25.087 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb8f0144:::benchmark_data_smithi060_196073_object45178:head# 2024-09-20T22:20:25.087 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb8fca97:::benchmark_data_smithi060_196073_object57596:head# 2024-09-20T22:20:25.087 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb9224d8:::benchmark_data_smithi060_196073_object3439:head# 2024-09-20T22:20:25.087 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb92e120:::benchmark_data_smithi060_196073_object58657:head# 2024-09-20T22:20:25.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:25.088 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:25.088 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:25.091 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb986cec:::benchmark_data_smithi060_196073_object53642:head# 2024-09-20T22:20:25.091 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb9b2e5c:::benchmark_data_smithi060_196073_object33876:head# 2024-09-20T22:20:25.091 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bb9b6a12:::benchmark_data_smithi060_196073_object55290:head# 2024-09-20T22:20:25.092 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bba06870:::benchmark_data_smithi060_196073_object36468:head# 2024-09-20T22:20:25.094 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bba0f957:::benchmark_data_smithi060_196073_object13972:head# 2024-09-20T22:20:25.095 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bba30c17:::benchmark_data_smithi060_196073_object1956:head# 2024-09-20T22:20:25.095 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bba378b1:::benchmark_data_smithi060_196073_object36298:head# 2024-09-20T22:20:25.095 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bba68543:::benchmark_data_smithi060_196073_object13822:head# 2024-09-20T22:20:25.098 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bba76a5f:::benchmark_data_smithi060_196073_object37126:head# 2024-09-20T22:20:25.098 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bba85e2b:::benchmark_data_smithi060_196073_object47404:head# 2024-09-20T22:20:25.098 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bba893a0:::benchmark_data_smithi060_196073_object2323:head# 2024-09-20T22:20:25.098 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bba8b435:::benchmark_data_smithi060_196073_object38274:head# 2024-09-20T22:20:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbab058c:::benchmark_data_smithi060_196073_object51540:head# 2024-09-20T22:20:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbac00b0:::benchmark_data_smithi060_196073_object58813:head# 2024-09-20T22:20:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbad5e5f:::benchmark_data_smithi060_196073_object57708:head# 2024-09-20T22:20:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbafb1f2:::benchmark_data_smithi060_196073_object30699:head# 2024-09-20T22:20:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbb31d9b:::benchmark_data_smithi060_196073_object12221:head# 2024-09-20T22:20:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbb33e47:::benchmark_data_smithi060_196073_object3537:head# 2024-09-20T22:20:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbb36f74:::benchmark_data_smithi060_196073_object37596:head# 2024-09-20T22:20:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbb42474:::benchmark_data_smithi060_196073_object11446:head# 2024-09-20T22:20:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbb4f47e:::benchmark_data_smithi060_196073_object13135:head# 2024-09-20T22:20:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbb6bc63:::benchmark_data_smithi060_196073_object45563:head# 2024-09-20T22:20:25.107 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbb7f065:::benchmark_data_smithi060_196073_object53383:head# 2024-09-20T22:20:25.108 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbb8790c:::benchmark_data_smithi060_196073_object55807:head# 2024-09-20T22:20:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbb8da08:::benchmark_data_smithi060_196073_object29586:head# 2024-09-20T22:20:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbb9536e:::benchmark_data_smithi060_196073_object53038:head# 2024-09-20T22:20:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbba6610:::benchmark_data_smithi060_196073_object44175:head# 2024-09-20T22:20:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbbd06f8:::benchmark_data_smithi060_196073_object35356:head# 2024-09-20T22:20:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbbd7192:::benchmark_data_smithi060_196073_object14433:head# 2024-09-20T22:20:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbbdb850:::benchmark_data_smithi060_196073_object46606:head# 2024-09-20T22:20:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbbdd59b:::benchmark_data_smithi060_196073_object11633:head# 2024-09-20T22:20:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbbde8ea:::benchmark_data_smithi060_196073_object19440:head# 2024-09-20T22:20:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbc027e9:::benchmark_data_smithi060_196073_object17917:head# 2024-09-20T22:20:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbc10cb3:::benchmark_data_smithi060_196073_object47866:head# 2024-09-20T22:20:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbc2028d:::benchmark_data_smithi060_196073_object5968:head# 2024-09-20T22:20:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbc2a008:::benchmark_data_smithi060_196073_object40075:head# 2024-09-20T22:20:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbc2c2b9:::benchmark_data_smithi060_196073_object33185:head# 2024-09-20T22:20:25.120 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbc2d4b1:::benchmark_data_smithi060_196073_object37561:head# 2024-09-20T22:20:25.121 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbc323af:::benchmark_data_smithi060_196073_object24898:head# 2024-09-20T22:20:25.121 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbc66ecb:::benchmark_data_smithi060_196073_object3671:head# 2024-09-20T22:20:25.121 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbc69f70:::benchmark_data_smithi060_196073_object61415:head# 2024-09-20T22:20:25.124 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbc746ba:::benchmark_data_smithi060_196073_object19607:head# 2024-09-20T22:20:25.124 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbc75e1c:::benchmark_data_smithi060_196073_object59959:head# 2024-09-20T22:20:25.124 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbc81daf:::benchmark_data_smithi060_196073_object57697:head# 2024-09-20T22:20:25.124 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbc95fb7:::benchmark_data_smithi060_196073_object10388:head# 2024-09-20T22:20:25.124 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbc9bc9f:::benchmark_data_smithi060_196073_object8092:head# 2024-09-20T22:20:25.125 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbca8540:::benchmark_data_smithi060_196073_object25961:head# 2024-09-20T22:20:25.126 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbcda8d0:::benchmark_data_smithi060_196073_object26327:head# 2024-09-20T22:20:25.126 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbce7e46:::benchmark_data_smithi060_196073_object91:head# 2024-09-20T22:20:25.127 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbd1ac89:::benchmark_data_smithi060_196073_object15422:head# 2024-09-20T22:20:25.129 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbd29803:::benchmark_data_smithi060_196073_object10159:head# 2024-09-20T22:20:25.129 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbd332cc:::benchmark_data_smithi060_196073_object61346:head# 2024-09-20T22:20:25.130 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbd456e4:::benchmark_data_smithi060_196073_object54033:head# 2024-09-20T22:20:25.131 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbd48e99:::benchmark_data_smithi060_196073_object60338:head# 2024-09-20T22:20:25.132 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbd4ec1d:::benchmark_data_smithi060_196073_object8481:head# 2024-09-20T22:20:25.132 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbd676cc:::benchmark_data_smithi060_196073_object23150:head# 2024-09-20T22:20:25.133 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbd69fe7:::benchmark_data_smithi060_196073_object9204:head# 2024-09-20T22:20:25.134 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbd6a8a4:::benchmark_data_smithi060_196073_object48280:head# 2024-09-20T22:20:25.134 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbd6e0d1:::benchmark_data_smithi060_196073_object38960:head# 2024-09-20T22:20:25.135 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbd7246e:::benchmark_data_smithi060_196073_object13641:head# 2024-09-20T22:20:25.136 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbd83065:::benchmark_data_smithi060_196073_object22361:head# 2024-09-20T22:20:25.136 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbd86d6a:::benchmark_data_smithi060_196073_object55178:head# 2024-09-20T22:20:25.137 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbd967a0:::benchmark_data_smithi060_196073_object17490:head# 2024-09-20T22:20:25.138 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbdb1102:::benchmark_data_smithi060_196073_object13874:head# 2024-09-20T22:20:25.139 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbdb2332:::benchmark_data_smithi060_196073_object18537:head# 2024-09-20T22:20:25.140 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbdcc80d:::benchmark_data_smithi060_196073_object44186:head# 2024-09-20T22:20:25.140 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbdd71ff:::benchmark_data_smithi060_196073_object47549:head# 2024-09-20T22:20:25.141 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbddf8dc:::benchmark_data_smithi060_196073_object60496:head# 2024-09-20T22:20:25.142 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbe0d441:::benchmark_data_smithi060_196073_object1522:head# 2024-09-20T22:20:25.142 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbe0ec45:::benchmark_data_smithi060_196073_object23893:head# 2024-09-20T22:20:25.143 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbe111e4:::benchmark_data_smithi060_196073_object47555:head# 2024-09-20T22:20:25.144 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbe3d295:::benchmark_data_smithi060_196073_object41105:head# 2024-09-20T22:20:25.145 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbe52e62:::benchmark_data_smithi060_196073_object40945:head# 2024-09-20T22:20:25.146 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbe736fd:::benchmark_data_smithi060_196073_object26367:head# 2024-09-20T22:20:25.146 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbe75eed:::benchmark_data_smithi060_196073_object7302:head# 2024-09-20T22:20:25.147 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbe9ff7d:::benchmark_data_smithi060_196073_object22650:head# 2024-09-20T22:20:25.148 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbeaa8eb:::benchmark_data_smithi060_196073_object6232:head# 2024-09-20T22:20:25.149 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbeac489:::benchmark_data_smithi060_196073_object36526:head# 2024-09-20T22:20:25.149 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbebb690:::benchmark_data_smithi060_196073_object31435:head# 2024-09-20T22:20:25.150 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbec0606:::benchmark_data_smithi060_196073_object54331:head# 2024-09-20T22:20:25.151 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbecfdaf:::benchmark_data_smithi060_196073_object12861:head# 2024-09-20T22:20:25.154 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbed1524:::benchmark_data_smithi060_196073_object42285:head# 2024-09-20T22:20:25.155 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbedde31:::benchmark_data_smithi060_196073_object39065:head# 2024-09-20T22:20:25.156 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbee1270:::benchmark_data_smithi060_196073_object59065:head# 2024-09-20T22:20:25.156 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbef3ad5:::benchmark_data_smithi060_196073_object6396:head# 2024-09-20T22:20:25.157 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbf05d4f:::benchmark_data_smithi060_196073_object14113:head# 2024-09-20T22:20:25.159 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbf10dd0:::benchmark_data_smithi060_196073_object35761:head# 2024-09-20T22:20:25.159 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbf2deb7:::benchmark_data_smithi060_196073_object29541:head# 2024-09-20T22:20:25.160 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbf2e272:::benchmark_data_smithi060_196073_object2677:head# 2024-09-20T22:20:25.160 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbf3c6db:::benchmark_data_smithi060_196073_object17211:head# 2024-09-20T22:20:25.161 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbf44b0e:::benchmark_data_smithi060_196073_object3148:head# 2024-09-20T22:20:25.162 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbf50c8a:::benchmark_data_smithi060_196073_object36143:head# 2024-09-20T22:20:25.162 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbf67010:::benchmark_data_smithi060_196073_object3293:head# 2024-09-20T22:20:25.163 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbf6a8bf:::benchmark_data_smithi060_196073_object42554:head# 2024-09-20T22:20:25.164 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbf7bc23:::benchmark_data_smithi060_196073_object2536:head# 2024-09-20T22:20:25.165 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbf7f065:::benchmark_data_smithi060_196073_object5592:head# 2024-09-20T22:20:25.165 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbf87f49:::benchmark_data_smithi060_196073_object19366:head# 2024-09-20T22:20:25.166 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbf89d67:::benchmark_data_smithi060_196073_object48610:head# 2024-09-20T22:20:25.168 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbf922a6:::benchmark_data_smithi060_196073_object49622:head# 2024-09-20T22:20:25.168 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbf97ad0:::benchmark_data_smithi060_196073_object31627:head# 2024-09-20T22:20:25.169 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbf9fb4b:::benchmark_data_smithi060_196073_object46537:head# 2024-09-20T22:20:25.170 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbfac2fd:::benchmark_data_smithi060_196073_object28420:head# 2024-09-20T22:20:25.171 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbfadd75:::benchmark_data_smithi060_196073_object57830:head# 2024-09-20T22:20:25.171 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbfb41f2:::benchmark_data_smithi060_196073_object60353:head# 2024-09-20T22:20:25.174 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbfb4a2d:::benchmark_data_smithi060_196073_object31866:head# 2024-09-20T22:20:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbfbb95e:::benchmark_data_smithi060_196073_object59900:head# 2024-09-20T22:20:25.175 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bbfde137:::benchmark_data_smithi060_196073_object30956:head# 2024-09-20T22:20:25.176 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc021f26:::benchmark_data_smithi060_196073_object25350:head# 2024-09-20T22:20:25.176 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc023271:::benchmark_data_smithi060_196073_object9427:head# 2024-09-20T22:20:25.177 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc02e9e1:::benchmark_data_smithi060_196073_object50099:head# 2024-09-20T22:20:25.178 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc0333d9:::benchmark_data_smithi060_196073_object14585:head# 2024-09-20T22:20:25.179 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc04a90f:::benchmark_data_smithi060_196073_object39192:head# 2024-09-20T22:20:25.180 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc050c1c:::benchmark_data_smithi060_196073_object13439:head# 2024-09-20T22:20:25.181 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc05158a:::benchmark_data_smithi060_196073_object56440:head# 2024-09-20T22:20:25.181 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc0638e7:::benchmark_data_smithi060_196073_object14804:head# 2024-09-20T22:20:25.182 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc0863d6:::benchmark_data_smithi060_196073_object35394:head# 2024-09-20T22:20:25.183 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc089e65:::benchmark_data_smithi060_196073_object17592:head# 2024-09-20T22:20:25.183 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc08fd22:::benchmark_data_smithi060_196073_object33219:head# 2024-09-20T22:20:25.184 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc098180:::benchmark_data_smithi060_196073_object27439:head# 2024-09-20T22:20:25.185 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc0a7d68:::benchmark_data_smithi060_196073_object52852:head# 2024-09-20T22:20:25.185 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc0a8449:::benchmark_data_smithi060_196073_object21980:head# 2024-09-20T22:20:25.186 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc0b2637:::benchmark_data_smithi060_196073_object16204:head# 2024-09-20T22:20:25.187 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc0b55e4:::benchmark_data_smithi060_196073_object40431:head# 2024-09-20T22:20:25.187 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc0d9548:::benchmark_data_smithi060_196073_object14629:head# 2024-09-20T22:20:25.188 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc0dfaf7:::benchmark_data_smithi060_196073_object18580:head# 2024-09-20T22:20:25.189 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc0fc093:::benchmark_data_smithi060_196073_object50946:head# 2024-09-20T22:20:25.189 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:25.191 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc10a788:::benchmark_data_smithi060_196073_object54378:head# 2024-09-20T22:20:25.192 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc11606e:::benchmark_data_smithi060_196073_object35441:head# 2024-09-20T22:20:25.193 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc11b00f:::benchmark_data_smithi060_196073_object49025:head# 2024-09-20T22:20:25.193 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc142077:::benchmark_data_smithi060_196073_object52838:head# 2024-09-20T22:20:25.194 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc15d3a8:::benchmark_data_smithi060_196073_object2348:head# 2024-09-20T22:20:25.195 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc173338:::benchmark_data_smithi060_196073_object30981:head# 2024-09-20T22:20:25.195 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc17cff6:::benchmark_data_smithi060_196073_object13510:head# 2024-09-20T22:20:25.196 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc17d24d:::benchmark_data_smithi060_196073_object18510:head# 2024-09-20T22:20:25.196 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc184813:::benchmark_data_smithi060_196073_object21301:head# 2024-09-20T22:20:25.197 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc189651:::benchmark_data_smithi060_196073_object34614:head# 2024-09-20T22:20:25.198 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc19bf57:::benchmark_data_smithi060_196073_object37381:head# 2024-09-20T22:20:25.198 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc1afed4:::benchmark_data_smithi060_196073_object53812:head# 2024-09-20T22:20:25.199 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc1ba603:::benchmark_data_smithi060_196073_object50479:head# 2024-09-20T22:20:25.200 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc1dd81d:::benchmark_data_smithi060_196073_object56726:head# 2024-09-20T22:20:25.200 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc1e6261:::benchmark_data_smithi060_196073_object58163:head# 2024-09-20T22:20:25.201 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc1fb221:::benchmark_data_smithi060_196073_object57307:head# 2024-09-20T22:20:25.202 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc21a96a:::benchmark_data_smithi060_196073_object5140:head# 2024-09-20T22:20:25.203 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc21af91:::benchmark_data_smithi060_196073_object33923:head# 2024-09-20T22:20:25.203 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc220cd3:::benchmark_data_smithi060_196073_object55554:head# 2024-09-20T22:20:25.204 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc25cd35:::benchmark_data_smithi060_196073_object19970:head# 2024-09-20T22:20:25.205 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc261fa5:::benchmark_data_smithi060_196073_object44048:head# 2024-09-20T22:20:25.206 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc267ee6:::benchmark_data_smithi060_196073_object22081:head# 2024-09-20T22:20:25.207 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc2865e9:::benchmark_data_smithi060_196073_object33422:head# 2024-09-20T22:20:25.207 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc29dfe9:::benchmark_data_smithi060_196073_object24624:head# 2024-09-20T22:20:25.221 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc29e24f:::benchmark_data_smithi060_196073_object36379:head# 2024-09-20T22:20:25.221 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc2af348:::benchmark_data_smithi060_196073_object50773:head# 2024-09-20T22:20:25.221 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc2bbafa:::benchmark_data_smithi060_196073_object53379:head# 2024-09-20T22:20:25.221 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc2c1579:::benchmark_data_smithi060_196073_object34:head# 2024-09-20T22:20:25.221 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc2c1b5f:::benchmark_data_smithi060_196073_object1661:head# 2024-09-20T22:20:25.221 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc2d8b3e:::benchmark_data_smithi060_196073_object34050:head# 2024-09-20T22:20:25.221 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc2fdb09:::benchmark_data_smithi060_196073_object34002:head# 2024-09-20T22:20:25.221 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc30a2e4:::benchmark_data_smithi060_196073_object5853:head# 2024-09-20T22:20:25.221 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc30b273:::benchmark_data_smithi060_196073_object14492:head# 2024-09-20T22:20:25.221 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc31d526:::benchmark_data_smithi060_196073_object55526:head# 2024-09-20T22:20:25.222 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc32f1a5:::benchmark_data_smithi060_196073_object49335:head# 2024-09-20T22:20:25.222 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc344d57:::benchmark_data_smithi060_196073_object9605:head# 2024-09-20T22:20:25.222 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc347aee:::benchmark_data_smithi060_196073_object42421:head# 2024-09-20T22:20:25.222 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc35aaad:::benchmark_data_smithi060_196073_object40678:head# 2024-09-20T22:20:25.222 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc36729f:::benchmark_data_smithi060_196073_object3910:head# 2024-09-20T22:20:25.222 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc37b92a:::benchmark_data_smithi060_196073_object58103:head# 2024-09-20T22:20:25.222 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc3808e1:::benchmark_data_smithi060_196073_object6317:head# 2024-09-20T22:20:25.222 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc38b58b:::benchmark_data_smithi060_196073_object15742:head# 2024-09-20T22:20:25.222 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc38fcb2:::benchmark_data_smithi060_196073_object15020:head# 2024-09-20T22:20:25.223 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc3a34ba:::benchmark_data_smithi060_196073_object53937:head# 2024-09-20T22:20:25.224 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc3a928c:::benchmark_data_smithi060_196073_object26845:head# 2024-09-20T22:20:25.225 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc41d9e5:::benchmark_data_smithi060_196073_object5257:head# 2024-09-20T22:20:25.225 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc42795c:::benchmark_data_smithi060_196073_object2868:head# 2024-09-20T22:20:25.226 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc44e8cc:::benchmark_data_smithi060_196073_object24670:head# 2024-09-20T22:20:25.226 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc461148:::benchmark_data_smithi060_196073_object45066:head# 2024-09-20T22:20:25.226 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc4a2867:::benchmark_data_smithi060_196073_object19487:head# 2024-09-20T22:20:25.227 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc4a7025:::benchmark_data_smithi060_196073_object32364:head# 2024-09-20T22:20:25.227 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc51df5f:::benchmark_data_smithi060_196073_object54610:head# 2024-09-20T22:20:25.228 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:25.228 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc522ec7:::benchmark_data_smithi060_196073_object27033:head# 2024-09-20T22:20:25.228 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc526f9f:::benchmark_data_smithi060_196073_object30040:head# 2024-09-20T22:20:25.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:25.229 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:25.234 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc5576c8:::benchmark_data_smithi060_196073_object14957:head# 2024-09-20T22:20:25.234 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc558c8c:::benchmark_data_smithi060_196073_object41266:head# 2024-09-20T22:20:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc55cbda:::benchmark_data_smithi060_196073_object30112:head# 2024-09-20T22:20:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc5734e6:::benchmark_data_smithi060_196073_object26589:head# 2024-09-20T22:20:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc576a73:::benchmark_data_smithi060_196073_object32400:head# 2024-09-20T22:20:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc598f36:::benchmark_data_smithi060_196073_object23443:head# 2024-09-20T22:20:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc5a6900:::benchmark_data_smithi060_196073_object17930:head# 2024-09-20T22:20:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc5ad5df:::benchmark_data_smithi060_196073_object51200:head# 2024-09-20T22:20:25.235 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc5baf30:::benchmark_data_smithi060_196073_object34206:head# 2024-09-20T22:20:25.236 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc5c00ca:::benchmark_data_smithi060_196073_object46292:head# 2024-09-20T22:20:25.237 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc5cbfce:::benchmark_data_smithi060_196073_object46269:head# 2024-09-20T22:20:25.238 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc5efba5:::benchmark_data_smithi060_196073_object42822:head# 2024-09-20T22:20:25.239 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc5f4f88:::benchmark_data_smithi060_196073_object2704:head# 2024-09-20T22:20:25.239 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc62b5ae:::benchmark_data_smithi060_196073_object28653:head# 2024-09-20T22:20:25.243 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc62b929:::benchmark_data_smithi060_196073_object27397:head# 2024-09-20T22:20:25.243 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc646c40:::benchmark_data_smithi060_196073_object9889:head# 2024-09-20T22:20:25.243 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc65ba63:::benchmark_data_smithi060_196073_object31051:head# 2024-09-20T22:20:25.243 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc663a37:::benchmark_data_smithi060_196073_object41882:head# 2024-09-20T22:20:25.244 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc684ffa:::benchmark_data_smithi060_196073_object13223:head# 2024-09-20T22:20:25.245 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc688a93:::benchmark_data_smithi060_196073_object4913:head# 2024-09-20T22:20:25.245 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc69882e:::benchmark_data_smithi060_196073_object25595:head# 2024-09-20T22:20:25.248 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc6a0d1f:::benchmark_data_smithi060_196073_object33660:head# 2024-09-20T22:20:25.248 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc6a141b:::benchmark_data_smithi060_196073_object51097:head# 2024-09-20T22:20:25.248 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc6a195b:::benchmark_data_smithi060_196073_object9832:head# 2024-09-20T22:20:25.249 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc6b50b0:::benchmark_data_smithi060_196073_object19508:head# 2024-09-20T22:20:25.251 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc6bad46:::benchmark_data_smithi060_196073_object21412:head# 2024-09-20T22:20:25.252 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc6c3d14:::benchmark_data_smithi060_196073_object2073:head# 2024-09-20T22:20:25.254 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc6e4a40:::benchmark_data_smithi060_196073_object30132:head# 2024-09-20T22:20:25.254 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc6f9d07:::benchmark_data_smithi060_196073_object47385:head# 2024-09-20T22:20:25.255 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc710bdc:::benchmark_data_smithi060_196073_object32920:head# 2024-09-20T22:20:25.255 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc720f0a:::benchmark_data_smithi060_196073_object55064:head# 2024-09-20T22:20:25.258 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc72c378:::benchmark_data_smithi060_196073_object10959:head# 2024-09-20T22:20:25.259 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc72c50e:::benchmark_data_smithi060_196073_object48078:head# 2024-09-20T22:20:25.261 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc74e02b:::benchmark_data_smithi060_196073_object21227:head# 2024-09-20T22:20:25.261 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc763740:::benchmark_data_smithi060_196073_object36773:head# 2024-09-20T22:20:25.262 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc7aa13c:::benchmark_data_smithi060_196073_object31913:head# 2024-09-20T22:20:25.263 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc7c8bb7:::benchmark_data_smithi060_196073_object45230:head# 2024-09-20T22:20:25.265 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:20:25.266 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc7d6d93:::benchmark_data_smithi060_196073_object35292:head# 2024-09-20T22:20:25.266 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc7d7867:::benchmark_data_smithi060_196073_object42084:head# 2024-09-20T22:20:25.267 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc7ea880:::benchmark_data_smithi060_196073_object51618:head# 2024-09-20T22:20:25.269 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc83b6fb:::benchmark_data_smithi060_196073_object24357:head# 2024-09-20T22:20:25.270 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc8401d8:::benchmark_data_smithi060_196073_object11686:head# 2024-09-20T22:20:25.271 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc85c4c3:::benchmark_data_smithi060_196073_object43442:head# 2024-09-20T22:20:25.272 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc877429:::benchmark_data_smithi060_196073_object14931:head# 2024-09-20T22:20:25.273 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc884db8:::benchmark_data_smithi060_196073_object31774:head# 2024-09-20T22:20:25.273 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc8a9747:::benchmark_data_smithi060_196073_object39719:head# 2024-09-20T22:20:25.274 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc8aa174:::benchmark_data_smithi060_196073_object4315:head# 2024-09-20T22:20:25.275 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc8ba328:::benchmark_data_smithi060_196073_object41816:head# 2024-09-20T22:20:25.276 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc8beaa6:::benchmark_data_smithi060_196073_object18955:head# 2024-09-20T22:20:25.276 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc8cfff1:::benchmark_data_smithi060_196073_object18257:head# 2024-09-20T22:20:25.277 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc8e7083:::benchmark_data_smithi060_196073_object28676:head# 2024-09-20T22:20:25.278 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc8f9251:::benchmark_data_smithi060_196073_object34064:head# 2024-09-20T22:20:25.278 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc922a7c:::benchmark_data_smithi060_196073_object1843:head# 2024-09-20T22:20:25.282 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc92a8a3:::benchmark_data_smithi060_196073_object61740:head# 2024-09-20T22:20:25.282 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc946ed7:::benchmark_data_smithi060_196073_object58742:head# 2024-09-20T22:20:25.282 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc9513c7:::benchmark_data_smithi060_196073_object46150:head# 2024-09-20T22:20:25.282 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc952e36:::benchmark_data_smithi060_196073_object43157:head# 2024-09-20T22:20:25.285 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc965a88:::benchmark_data_smithi060_196073_object989:head# 2024-09-20T22:20:25.285 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc97fb27:::benchmark_data_smithi060_196073_object36763:head# 2024-09-20T22:20:25.285 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc9b1622:::benchmark_data_smithi060_196073_object55564:head# 2024-09-20T22:20:25.288 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc9ba979:::benchmark_data_smithi060_196073_object14295:head# 2024-09-20T22:20:25.295 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc9c4861:::benchmark_data_smithi060_196073_object26338:head# 2024-09-20T22:20:25.295 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc9c74f0:::benchmark_data_smithi060_196073_object46173:head# 2024-09-20T22:20:25.295 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc9cf4ed:::benchmark_data_smithi060_196073_object53299:head# 2024-09-20T22:20:25.295 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc9d0a35:::benchmark_data_smithi060_196073_object34065:head# 2024-09-20T22:20:25.295 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc9d37a2:::benchmark_data_smithi060_196073_object48960:head# 2024-09-20T22:20:25.295 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bc9de400:::benchmark_data_smithi060_196073_object49521:head# 2024-09-20T22:20:25.295 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bca08d2c:::benchmark_data_smithi060_196073_object47010:head# 2024-09-20T22:20:25.296 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bca1014c:::benchmark_data_smithi060_196073_object28772:head# 2024-09-20T22:20:25.296 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bca104da:::benchmark_data_smithi060_196073_object43287:head# 2024-09-20T22:20:25.300 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bca1ad03:::benchmark_data_smithi060_196073_object59396:head# 2024-09-20T22:20:25.300 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bca1fcbe:::benchmark_data_smithi060_196073_object31005:head# 2024-09-20T22:20:25.300 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bca2f760:::benchmark_data_smithi060_196073_object41730:head# 2024-09-20T22:20:25.300 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bca34a15:::benchmark_data_smithi060_196073_object33707:head# 2024-09-20T22:20:25.300 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bca5d84c:::benchmark_data_smithi060_196073_object48154:head# 2024-09-20T22:20:25.300 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bca6894f:::benchmark_data_smithi060_196073_object4490:head# 2024-09-20T22:20:25.300 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bca7e26d:::benchmark_data_smithi060_196073_object27588:head# 2024-09-20T22:20:25.301 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bca7e591:::benchmark_data_smithi060_196073_object54179:head# 2024-09-20T22:20:25.301 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bca8c906:::benchmark_data_smithi060_196073_object4794:head# 2024-09-20T22:20:25.302 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcaa332e:::benchmark_data_smithi060_196073_object45217:head# 2024-09-20T22:20:25.302 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcaac595:::benchmark_data_smithi060_196073_object22944:head# 2024-09-20T22:20:25.303 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcabdaab:::benchmark_data_smithi060_196073_object35230:head# 2024-09-20T22:20:25.304 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcac0f7d:::benchmark_data_smithi060_196073_object47492:head# 2024-09-20T22:20:25.305 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcaf8cfc:::benchmark_data_smithi060_196073_object33467:head# 2024-09-20T22:20:25.306 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcb26583:::benchmark_data_smithi060_196073_object46038:head# 2024-09-20T22:20:25.306 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcb2de7c:::benchmark_data_smithi060_196073_object29985:head# 2024-09-20T22:20:25.308 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcb407ff:::benchmark_data_smithi060_196073_object10409:head# 2024-09-20T22:20:25.308 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcb7f02c:::benchmark_data_smithi060_196073_object6802:head# 2024-09-20T22:20:25.308 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:20:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:20:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:20:25.308 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcb7f3da:::benchmark_data_smithi060_196073_object37724:head# 2024-09-20T22:20:25.309 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcb81327:::benchmark_data_smithi060_196073_object1419:head# 2024-09-20T22:20:25.310 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcb8e950:::benchmark_data_smithi060_196073_object54092:head# 2024-09-20T22:20:25.311 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcbd9fc3:::benchmark_data_smithi060_196073_object44817:head# 2024-09-20T22:20:25.311 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcbdccd6:::benchmark_data_smithi060_196073_object27028:head# 2024-09-20T22:20:25.312 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcbe4e1c:::benchmark_data_smithi060_196073_object22300:head# 2024-09-20T22:20:25.313 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcbe7ab3:::benchmark_data_smithi060_196073_object57514:head# 2024-09-20T22:20:25.314 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcbf37f7:::benchmark_data_smithi060_196073_object53388:head# 2024-09-20T22:20:25.314 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcc009b2:::benchmark_data_smithi060_196073_object61364:head# 2024-09-20T22:20:25.315 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcc1286e:::benchmark_data_smithi060_196073_object47294:head# 2024-09-20T22:20:25.315 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcc213f4:::benchmark_data_smithi060_196073_object50601:head# 2024-09-20T22:20:25.316 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcc21c92:::benchmark_data_smithi060_196073_object15487:head# 2024-09-20T22:20:25.316 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcc281e0:::benchmark_data_smithi060_196073_object52864:head# 2024-09-20T22:20:25.317 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcc3c762:::benchmark_data_smithi060_196073_object2608:head# 2024-09-20T22:20:25.318 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcc5b1f5:::benchmark_data_smithi060_196073_object42459:head# 2024-09-20T22:20:25.318 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcc6ea35:::benchmark_data_smithi060_196073_object15205:head# 2024-09-20T22:20:25.319 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcc7362e:::benchmark_data_smithi060_196073_object31831:head# 2024-09-20T22:20:25.319 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcca0dca:::benchmark_data_smithi060_196073_object9746:head# 2024-09-20T22:20:25.320 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bccb5a04:::benchmark_data_smithi060_196073_object19613:head# 2024-09-20T22:20:25.321 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bccd76d8:::benchmark_data_smithi060_196073_object30648:head# 2024-09-20T22:20:25.322 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcd08e8c:::benchmark_data_smithi060_196073_object19042:head# 2024-09-20T22:20:25.322 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcd1c73d:::benchmark_data_smithi060_196073_object33170:head# 2024-09-20T22:20:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcd36c79:::benchmark_data_smithi060_196073_object19575:head# 2024-09-20T22:20:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcd36f5f:::benchmark_data_smithi060_196073_object7419:head# 2024-09-20T22:20:25.323 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcd4a71f:::benchmark_data_smithi060_196073_object26201:head# 2024-09-20T22:20:25.324 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcd53d47:::benchmark_data_smithi060_196073_object46454:head# 2024-09-20T22:20:25.324 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcd60f2b:::benchmark_data_smithi060_196073_object35380:head# 2024-09-20T22:20:25.325 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcd64f3e:::benchmark_data_smithi060_196073_object4115:head# 2024-09-20T22:20:25.326 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcd6a8f6:::benchmark_data_smithi060_196073_object21201:head# 2024-09-20T22:20:25.326 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcd77b90:::benchmark_data_smithi060_196073_object17101:head# 2024-09-20T22:20:25.327 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcd89ab5:::benchmark_data_smithi060_196073_object45369:head# 2024-09-20T22:20:25.329 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcd89b4e:::benchmark_data_smithi060_196073_object745:head# 2024-09-20T22:20:25.329 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcd95268:::benchmark_data_smithi060_196073_object16653:head# 2024-09-20T22:20:25.330 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:25.331 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcdac086:::benchmark_data_smithi060_196073_object536:head# 2024-09-20T22:20:25.331 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcdc542e:::benchmark_data_smithi060_196073_object48266:head# 2024-09-20T22:20:25.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcdce05b:::benchmark_data_smithi060_196073_object18512:head# 2024-09-20T22:20:25.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcdd4c06:::benchmark_data_smithi060_196073_object46169:head# 2024-09-20T22:20:25.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcde3794:::benchmark_data_smithi060_196073_object2903:head# 2024-09-20T22:20:25.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcdf4a20:::benchmark_data_smithi060_196073_object58216:head# 2024-09-20T22:20:25.334 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bce05fda:::benchmark_data_smithi060_196073_object22966:head# 2024-09-20T22:20:25.335 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bce0cb62:::benchmark_data_smithi060_196073_object25521:head# 2024-09-20T22:20:25.335 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bce1c775:::benchmark_data_smithi060_196073_object35816:head# 2024-09-20T22:20:25.336 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bce3ba7c:::benchmark_data_smithi060_196073_object627:head# 2024-09-20T22:20:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bce435c1:::benchmark_data_smithi060_196073_object620:head# 2024-09-20T22:20:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bce4db98:::benchmark_data_smithi060_196073_object8351:head# 2024-09-20T22:20:25.337 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bce56f6b:::benchmark_data_smithi060_196073_object33872:head# 2024-09-20T22:20:25.340 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bce5bcc1:::benchmark_data_smithi060_196073_object32795:head# 2024-09-20T22:20:25.341 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bce84bd4:::benchmark_data_smithi060_196073_object32627:head# 2024-09-20T22:20:25.342 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bce8af21:::benchmark_data_smithi060_196073_object47017:head# 2024-09-20T22:20:25.342 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bce904d0:::benchmark_data_smithi060_196073_object34948:head# 2024-09-20T22:20:25.344 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcebe04a:::benchmark_data_smithi060_196073_object59833:head# 2024-09-20T22:20:25.344 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcedeb23:::benchmark_data_smithi060_196073_object50061:head# 2024-09-20T22:20:25.345 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcee86b8:::benchmark_data_smithi060_196073_object59785:head# 2024-09-20T22:20:25.345 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bceed4de:::benchmark_data_smithi060_196073_object47050:head# 2024-09-20T22:20:25.345 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcefdc36:::benchmark_data_smithi060_196073_object15396:head# 2024-09-20T22:20:25.346 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcf1a06d:::benchmark_data_smithi060_196073_object35167:head# 2024-09-20T22:20:25.349 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcf219fa:::benchmark_data_smithi060_196073_object45358:head# 2024-09-20T22:20:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcf314a2:::benchmark_data_smithi060_196073_object6577:head# 2024-09-20T22:20:25.350 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcf3fd86:::benchmark_data_smithi060_196073_object50767:head# 2024-09-20T22:20:25.353 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcf429cc:::benchmark_data_smithi060_196073_object4103:head# 2024-09-20T22:20:25.354 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcf56cb1:::benchmark_data_smithi060_196073_object46909:head# 2024-09-20T22:20:25.354 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcf66696:::benchmark_data_smithi060_196073_object61504:head# 2024-09-20T22:20:25.355 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcfa6494:::benchmark_data_smithi060_196073_object33134:head# 2024-09-20T22:20:25.355 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcfbf76e:::benchmark_data_smithi060_196073_object51420:head# 2024-09-20T22:20:25.358 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcfc27ed:::benchmark_data_smithi060_196073_object5766:head# 2024-09-20T22:20:25.359 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcfdaeb6:::benchmark_data_smithi060_196073_object4479:head# 2024-09-20T22:20:25.361 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bcfe172e:::benchmark_data_smithi060_196073_object52666:head# 2024-09-20T22:20:25.361 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:25.362 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:25.363 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:25.363 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd002493:::benchmark_data_smithi060_196073_object25693:head# 2024-09-20T22:20:25.363 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd021dce:::benchmark_data_smithi060_196073_object60880:head# 2024-09-20T22:20:25.364 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd02ef0f:::benchmark_data_smithi060_196073_object49261:head# 2024-09-20T22:20:25.364 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd037b8a:::benchmark_data_smithi060_196073_object865:head# 2024-09-20T22:20:25.365 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd046a15:::benchmark_data_smithi060_196073_object42485:head# 2024-09-20T22:20:25.365 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd061fb3:::benchmark_data_smithi060_196073_object34754:head# 2024-09-20T22:20:25.366 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd0663c1:::benchmark_data_smithi060_196073_object13049:head# 2024-09-20T22:20:25.366 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd06a458:::benchmark_data_smithi060_196073_object28350:head# 2024-09-20T22:20:25.367 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd095101:::benchmark_data_smithi060_196073_object36311:head# 2024-09-20T22:20:25.367 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd09c45f:::benchmark_data_smithi060_196073_object25053:head# 2024-09-20T22:20:25.368 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd0a456e:::benchmark_data_smithi060_196073_object26348:head# 2024-09-20T22:20:25.369 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd0af5f5:::benchmark_data_smithi060_196073_object15451:head# 2024-09-20T22:20:25.370 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd0c349b:::benchmark_data_smithi060_196073_object40098:head# 2024-09-20T22:20:25.370 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd0e6234:::benchmark_data_smithi060_196073_object9731:head# 2024-09-20T22:20:25.371 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd0e7613:::benchmark_data_smithi060_196073_object46278:head# 2024-09-20T22:20:25.372 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd0ec051:::benchmark_data_smithi060_196073_object10870:head# 2024-09-20T22:20:25.373 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd0ec72c:::benchmark_data_smithi060_196073_object29525:head# 2024-09-20T22:20:25.374 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd10459a:::benchmark_data_smithi060_196073_object33620:head# 2024-09-20T22:20:25.375 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd10ca88:::benchmark_data_smithi060_196073_object52515:head# 2024-09-20T22:20:25.376 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd112e79:::benchmark_data_smithi060_196073_object37662:head# 2024-09-20T22:20:25.377 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd129a8f:::benchmark_data_smithi060_196073_object49774:head# 2024-09-20T22:20:25.380 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd14fd5d:::benchmark_data_smithi060_196073_object13865:head# 2024-09-20T22:20:25.381 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd156953:::benchmark_data_smithi060_196073_object22708:head# 2024-09-20T22:20:25.381 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd15f90f:::benchmark_data_smithi060_196073_object10240:head# 2024-09-20T22:20:25.382 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd19b42b:::benchmark_data_smithi060_196073_object59776:head# 2024-09-20T22:20:25.383 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd1c614c:::benchmark_data_smithi060_196073_object44489:head# 2024-09-20T22:20:25.384 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd1ca953:::benchmark_data_smithi060_196073_object10713:head# 2024-09-20T22:20:25.385 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd1d0715:::benchmark_data_smithi060_196073_object25068:head# 2024-09-20T22:20:25.386 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd1e719a:::benchmark_data_smithi060_196073_object22572:head# 2024-09-20T22:20:25.386 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd211ce0:::benchmark_data_smithi060_196073_object28372:head# 2024-09-20T22:20:25.387 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd226a4b:::benchmark_data_smithi060_196073_object10758:head# 2024-09-20T22:20:25.388 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd22cfc9:::benchmark_data_smithi060_196073_object12783:head# 2024-09-20T22:20:25.389 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd2423ab:::benchmark_data_smithi060_196073_object48281:head# 2024-09-20T22:20:25.390 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd242b03:::benchmark_data_smithi060_196073_object2932:head# 2024-09-20T22:20:25.390 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd25b89c:::benchmark_data_smithi060_196073_object43587:head# 2024-09-20T22:20:25.391 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd25d85e:::benchmark_data_smithi060_196073_object45582:head# 2024-09-20T22:20:25.392 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd26017e:::benchmark_data_smithi060_196073_object48243:head# 2024-09-20T22:20:25.393 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd268973:::benchmark_data_smithi060_196073_object61515:head# 2024-09-20T22:20:25.393 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd2787cb:::benchmark_data_smithi060_196073_object29819:head# 2024-09-20T22:20:25.394 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd284308:::benchmark_data_smithi060_196073_object34775:head# 2024-09-20T22:20:25.395 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd299e96:::benchmark_data_smithi060_196073_object29605:head# 2024-09-20T22:20:25.396 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd2bdda2:::benchmark_data_smithi060_196073_object31259:head# 2024-09-20T22:20:25.397 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd2dae66:::benchmark_data_smithi060_196073_object19041:head# 2024-09-20T22:20:25.397 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd2eb386:::benchmark_data_smithi060_196073_object14961:head# 2024-09-20T22:20:25.398 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd2ec6ae:::benchmark_data_smithi060_196073_object20560:head# 2024-09-20T22:20:25.399 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd329f73:::benchmark_data_smithi060_196073_object11987:head# 2024-09-20T22:20:25.400 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd33edb7:::benchmark_data_smithi060_196073_object8179:head# 2024-09-20T22:20:25.400 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd347336:::benchmark_data_smithi060_196073_object17699:head# 2024-09-20T22:20:25.401 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd357313:::benchmark_data_smithi060_196073_object9753:head# 2024-09-20T22:20:25.401 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd361864:::benchmark_data_smithi060_196073_object33535:head# 2024-09-20T22:20:25.402 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd382f2f:::benchmark_data_smithi060_196073_object22562:head# 2024-09-20T22:20:25.403 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd3a738c:::benchmark_data_smithi060_196073_object19224:head# 2024-09-20T22:20:25.403 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd3b73e2:::benchmark_data_smithi060_196073_object9276:head# 2024-09-20T22:20:25.404 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd3c1cf5:::benchmark_data_smithi060_196073_object19651:head# 2024-09-20T22:20:25.404 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd3c9e1d:::benchmark_data_smithi060_196073_object35772:head# 2024-09-20T22:20:25.405 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd3cfcc1:::benchmark_data_smithi060_196073_object38854:head# 2024-09-20T22:20:25.405 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd3dc06e:::benchmark_data_smithi060_196073_object14196:head# 2024-09-20T22:20:25.406 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd3e481a:::benchmark_data_smithi060_196073_object19544:head# 2024-09-20T22:20:25.407 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd3f3c66:::benchmark_data_smithi060_196073_object31899:head# 2024-09-20T22:20:25.407 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd40d363:::benchmark_data_smithi060_196073_object7233:head# 2024-09-20T22:20:25.408 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd428d2a:::benchmark_data_smithi060_196073_object49869:head# 2024-09-20T22:20:25.409 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd436b72:::benchmark_data_smithi060_196073_object25975:head# 2024-09-20T22:20:25.409 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd45f6b1:::benchmark_data_smithi060_196073_object21364:head# 2024-09-20T22:20:25.409 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd460150:::benchmark_data_smithi060_196073_object32705:head# 2024-09-20T22:20:25.410 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd479a1a:::benchmark_data_smithi060_196073_object53794:head# 2024-09-20T22:20:25.410 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd484573:::benchmark_data_smithi060_196073_object27701:head# 2024-09-20T22:20:25.411 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd484ca8:::benchmark_data_smithi060_196073_object54921:head# 2024-09-20T22:20:25.412 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd4a9887:::benchmark_data_smithi060_196073_object53801:head# 2024-09-20T22:20:25.412 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd4bbbe3:::benchmark_data_smithi060_196073_object59031:head# 2024-09-20T22:20:25.413 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd4bdd65:::benchmark_data_smithi060_196073_object39521:head# 2024-09-20T22:20:25.414 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd4c8ec9:::benchmark_data_smithi060_196073_object31809:head# 2024-09-20T22:20:25.414 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd4d390a:::benchmark_data_smithi060_196073_object43294:head# 2024-09-20T22:20:25.415 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd4d6a76:::benchmark_data_smithi060_196073_object20341:head# 2024-09-20T22:20:25.415 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd4ef28a:::benchmark_data_smithi060_196073_object43469:head# 2024-09-20T22:20:25.416 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd50dabd:::benchmark_data_smithi060_196073_object22689:head# 2024-09-20T22:20:25.417 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd5362a2:::benchmark_data_smithi060_196073_object23208:head# 2024-09-20T22:20:25.418 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd53b976:::benchmark_data_smithi060_196073_object50140:head# 2024-09-20T22:20:25.419 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd540ec4:::benchmark_data_smithi060_196073_object20534:head# 2024-09-20T22:20:25.420 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd54f8fd:::benchmark_data_smithi060_196073_object37906:head# 2024-09-20T22:20:25.420 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd58fece:::benchmark_data_smithi060_196073_object44476:head# 2024-09-20T22:20:25.420 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd5adb95:::benchmark_data_smithi060_196073_object25588:head# 2024-09-20T22:20:25.421 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd5b5b84:::benchmark_data_smithi060_196073_object10534:head# 2024-09-20T22:20:25.422 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd5f25e6:::benchmark_data_smithi060_196073_object7845:head# 2024-09-20T22:20:25.424 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd6104d3:::benchmark_data_smithi060_196073_object37389:head# 2024-09-20T22:20:25.424 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd615b46:::benchmark_data_smithi060_196073_object32077:head# 2024-09-20T22:20:25.424 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd632715:::benchmark_data_smithi060_196073_object31243:head# 2024-09-20T22:20:25.425 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd638b27:::benchmark_data_smithi060_196073_object27314:head# 2024-09-20T22:20:25.426 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd6395dd:::benchmark_data_smithi060_196073_object37695:head# 2024-09-20T22:20:25.427 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd63de4d:::benchmark_data_smithi060_196073_object16683:head# 2024-09-20T22:20:25.427 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd6477fa:::benchmark_data_smithi060_196073_object2222:head# 2024-09-20T22:20:25.428 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd64f177:::benchmark_data_smithi060_196073_object52306:head# 2024-09-20T22:20:25.430 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd66518a:::benchmark_data_smithi060_196073_object44456:head# 2024-09-20T22:20:25.430 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd66bc95:::benchmark_data_smithi060_196073_object31616:head# 2024-09-20T22:20:25.430 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd676d56:::benchmark_data_smithi060_196073_object14941:head# 2024-09-20T22:20:25.431 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd679481:::benchmark_data_smithi060_196073_object51683:head# 2024-09-20T22:20:25.431 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd6826ca:::benchmark_data_smithi060_196073_object21961:head# 2024-09-20T22:20:25.432 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd696e4a:::benchmark_data_smithi060_196073_object2515:head# 2024-09-20T22:20:25.433 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd69ada6:::benchmark_data_smithi060_196073_object44665:head# 2024-09-20T22:20:25.434 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd6a3c4c:::benchmark_data_smithi060_196073_object40528:head# 2024-09-20T22:20:25.435 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd6aa71a:::benchmark_data_smithi060_196073_object14015:head# 2024-09-20T22:20:25.435 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd6bceee:::benchmark_data_smithi060_196073_object140:head# 2024-09-20T22:20:25.436 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd6c0026:::benchmark_data_smithi060_196073_object47662:head# 2024-09-20T22:20:25.437 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd6d4630:::benchmark_data_smithi060_196073_object3000:head# 2024-09-20T22:20:25.438 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd6f2d49:::benchmark_data_smithi060_196073_object29043:head# 2024-09-20T22:20:25.439 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd6f37b1:::benchmark_data_smithi060_196073_object3094:head# 2024-09-20T22:20:25.439 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd6f5508:::benchmark_data_smithi060_196073_object52206:head# 2024-09-20T22:20:25.440 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd6f6557:::benchmark_data_smithi060_196073_object3418:head# 2024-09-20T22:20:25.441 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd70d9ca:::benchmark_data_smithi060_196073_object48953:head# 2024-09-20T22:20:25.441 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd716e59:::benchmark_data_smithi060_196073_object19115:head# 2024-09-20T22:20:25.442 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd720c5a:::benchmark_data_smithi060_196073_object31287:head# 2024-09-20T22:20:25.443 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd73055d:::benchmark_data_smithi060_196073_object54767:head# 2024-09-20T22:20:25.444 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd745c5c:::benchmark_data_smithi060_196073_object61416:head# 2024-09-20T22:20:25.445 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd7748ec:::benchmark_data_smithi060_196073_object10621:head# 2024-09-20T22:20:25.445 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd779c32:::benchmark_data_smithi060_196073_object39623:head# 2024-09-20T22:20:25.446 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd77d924:::benchmark_data_smithi060_196073_object2644:head# 2024-09-20T22:20:25.447 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd77d932:::benchmark_data_smithi060_196073_object44773:head# 2024-09-20T22:20:25.448 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd78a723:::benchmark_data_smithi060_196073_object46114:head# 2024-09-20T22:20:25.448 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd7a01b7:::benchmark_data_smithi060_196073_object2826:head# 2024-09-20T22:20:25.449 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd7aa845:::benchmark_data_smithi060_196073_object31324:head# 2024-09-20T22:20:25.453 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd7ab34b:::benchmark_data_smithi060_196073_object10403:head# 2024-09-20T22:20:25.453 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd7b11fe:::benchmark_data_smithi060_196073_object53174:head# 2024-09-20T22:20:25.453 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd7b4682:::benchmark_data_smithi060_196073_object15948:head# 2024-09-20T22:20:25.453 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd7b5d4e:::benchmark_data_smithi060_196073_object24613:head# 2024-09-20T22:20:25.453 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd7d29cc:::benchmark_data_smithi060_196073_object31751:head# 2024-09-20T22:20:25.454 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd7dd3b6:::benchmark_data_smithi060_196073_object51766:head# 2024-09-20T22:20:25.455 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd86fbcb:::benchmark_data_smithi060_196073_object5149:head# 2024-09-20T22:20:25.459 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd8784dd:::benchmark_data_smithi060_196073_object1293:head# 2024-09-20T22:20:25.459 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd88fffe:::benchmark_data_smithi060_196073_object35672:head# 2024-09-20T22:20:25.459 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd8913bf:::benchmark_data_smithi060_196073_object58698:head# 2024-09-20T22:20:25.459 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd8b094d:::benchmark_data_smithi060_196073_object32683:head# 2024-09-20T22:20:25.459 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd8d5c0b:::benchmark_data_smithi060_196073_object10568:head# 2024-09-20T22:20:25.459 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd8e0b28:::benchmark_data_smithi060_196073_object16362:head# 2024-09-20T22:20:25.460 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd8ea7c3:::benchmark_data_smithi060_196073_object44840:head# 2024-09-20T22:20:25.461 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd9220fd:::benchmark_data_smithi060_196073_object43640:head# 2024-09-20T22:20:25.461 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd9259b5:::benchmark_data_smithi060_196073_object4421:head# 2024-09-20T22:20:25.462 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd929453:::benchmark_data_smithi060_196073_object8634:head# 2024-09-20T22:20:25.462 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd92dbe3:::benchmark_data_smithi060_196073_object59268:head# 2024-09-20T22:20:25.463 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:25.464 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd931252:::benchmark_data_smithi060_196073_object58117:head# 2024-09-20T22:20:25.464 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd93a4b7:::benchmark_data_smithi060_196073_object39256:head# 2024-09-20T22:20:25.467 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd944f4d:::benchmark_data_smithi060_196073_object40891:head# 2024-09-20T22:20:25.467 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd96293f:::benchmark_data_smithi060_196073_object47925:head# 2024-09-20T22:20:25.467 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd990619:::benchmark_data_smithi060_196073_object772:head# 2024-09-20T22:20:25.470 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd994699:::benchmark_data_smithi060_196073_object56635:head# 2024-09-20T22:20:25.472 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd995692:::benchmark_data_smithi060_196073_object22546:head# 2024-09-20T22:20:25.474 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd9aa562:::benchmark_data_smithi060_196073_object37924:head# 2024-09-20T22:20:25.474 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd9c4ff6:::benchmark_data_smithi060_196073_object58281:head# 2024-09-20T22:20:25.475 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd9cb624:::benchmark_data_smithi060_196073_object57513:head# 2024-09-20T22:20:25.477 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd9e6177:::benchmark_data_smithi060_196073_object61262:head# 2024-09-20T22:20:25.477 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd9e9a3e:::benchmark_data_smithi060_196073_object53396:head# 2024-09-20T22:20:25.478 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd9ec2ea:::benchmark_data_smithi060_196073_object22271:head# 2024-09-20T22:20:25.480 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bd9f44d2:::benchmark_data_smithi060_196073_object42222:head# 2024-09-20T22:20:25.481 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bda42ee4:::benchmark_data_smithi060_196073_object58716:head# 2024-09-20T22:20:25.482 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bda451a9:::benchmark_data_smithi060_196073_object35000:head# 2024-09-20T22:20:25.483 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bda504c2:::benchmark_data_smithi060_196073_object1516:head# 2024-09-20T22:20:25.486 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bda58ea9:::benchmark_data_smithi060_196073_object30376:head# 2024-09-20T22:20:25.487 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bda59806:::benchmark_data_smithi060_196073_object44523:head# 2024-09-20T22:20:25.487 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bda5e4ab:::benchmark_data_smithi060_196073_object16507:head# 2024-09-20T22:20:25.487 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bda5fd88:::benchmark_data_smithi060_196073_object33832:head# 2024-09-20T22:20:25.488 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bda9bb00:::benchmark_data_smithi060_196073_object29982:head# 2024-09-20T22:20:25.488 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bda9e1cb:::benchmark_data_smithi060_196073_object31742:head# 2024-09-20T22:20:25.489 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdab5bec:::benchmark_data_smithi060_196073_object39026:head# 2024-09-20T22:20:25.490 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdabeb69:::benchmark_data_smithi060_196073_object8490:head# 2024-09-20T22:20:25.490 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdadc62a:::benchmark_data_smithi060_196073_object48656:head# 2024-09-20T22:20:25.490 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdae472e:::benchmark_data_smithi060_196073_object41896:head# 2024-09-20T22:20:25.491 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdaee6a3:::benchmark_data_smithi060_196073_object38673:head# 2024-09-20T22:20:25.493 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdaee935:::benchmark_data_smithi060_196073_object49661:head# 2024-09-20T22:20:25.493 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdb0070a:::benchmark_data_smithi060_196073_object22549:head# 2024-09-20T22:20:25.495 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdb07e23:::benchmark_data_smithi060_196073_object12060:head# 2024-09-20T22:20:25.498 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdb3171f:::benchmark_data_smithi060_196073_object12894:head# 2024-09-20T22:20:25.498 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdb340fc:::benchmark_data_smithi060_196073_object36296:head# 2024-09-20T22:20:25.499 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdb49396:::benchmark_data_smithi060_196073_object55463:head# 2024-09-20T22:20:25.500 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdb7d142:::benchmark_data_smithi060_196073_object17183:head# 2024-09-20T22:20:25.501 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdbad48e:::benchmark_data_smithi060_196073_object25907:head# 2024-09-20T22:20:25.501 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdbb0062:::benchmark_data_smithi060_196073_object41933:head# 2024-09-20T22:20:25.502 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdbd0d97:::benchmark_data_smithi060_196073_object56453:head# 2024-09-20T22:20:25.505 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdbf8a7c:::benchmark_data_smithi060_196073_object55553:head# 2024-09-20T22:20:25.508 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdbfa309:::benchmark_data_smithi060_196073_object25232:head# 2024-09-20T22:20:25.508 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdc23d50:::benchmark_data_smithi060_196073_object56272:head# 2024-09-20T22:20:25.509 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdc4a961:::benchmark_data_smithi060_196073_object38926:head# 2024-09-20T22:20:25.510 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdc4b875:::benchmark_data_smithi060_196073_object55251:head# 2024-09-20T22:20:25.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:25.511 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:25.511 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:25.515 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdc61e43:::benchmark_data_smithi060_196073_object26414:head# 2024-09-20T22:20:25.515 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdc71fce:::benchmark_data_smithi060_196073_object6817:head# 2024-09-20T22:20:25.515 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdc869db:::benchmark_data_smithi060_196073_object30711:head# 2024-09-20T22:20:25.515 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdc8ccc7:::benchmark_data_smithi060_196073_object54211:head# 2024-09-20T22:20:25.515 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdca7309:::benchmark_data_smithi060_196073_object42739:head# 2024-09-20T22:20:25.515 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdcab6c2:::benchmark_data_smithi060_196073_object15374:head# 2024-09-20T22:20:25.517 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdcb48d8:::benchmark_data_smithi060_196073_object30402:head# 2024-09-20T22:20:25.518 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdcdd5bb:::benchmark_data_smithi060_196073_object16003:head# 2024-09-20T22:20:25.518 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdcddce6:::benchmark_data_smithi060_196073_object37670:head# 2024-09-20T22:20:25.519 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdceb11d:::benchmark_data_smithi060_196073_object42347:head# 2024-09-20T22:20:25.519 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdceed79:::benchmark_data_smithi060_196073_object22941:head# 2024-09-20T22:20:25.520 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdcf1b77:::benchmark_data_smithi060_196073_object8193:head# 2024-09-20T22:20:25.521 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdcfc366:::benchmark_data_smithi060_196073_object33886:head# 2024-09-20T22:20:25.521 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdcfd970:::benchmark_data_smithi060_196073_object56510:head# 2024-09-20T22:20:25.522 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdd02d4a:::benchmark_data_smithi060_196073_object34031:head# 2024-09-20T22:20:25.522 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdd0807b:::benchmark_data_smithi060_196073_object50628:head# 2024-09-20T22:20:25.523 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdd0f835:::benchmark_data_smithi060_196073_object3239:head# 2024-09-20T22:20:25.524 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdd2e0b5:::benchmark_data_smithi060_196073_object11563:head# 2024-09-20T22:20:25.524 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdd35b27:::benchmark_data_smithi060_196073_object44138:head# 2024-09-20T22:20:25.525 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdd416b8:::benchmark_data_smithi060_196073_object60934:head# 2024-09-20T22:20:25.525 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdd70877:::benchmark_data_smithi060_196073_object60483:head# 2024-09-20T22:20:25.526 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdd954ed:::benchmark_data_smithi060_196073_object13178:head# 2024-09-20T22:20:25.527 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdd98d4f:::benchmark_data_smithi060_196073_object39656:head# 2024-09-20T22:20:25.527 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdd9b01e:::benchmark_data_smithi060_196073_object36771:head# 2024-09-20T22:20:25.528 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdd9da18:::benchmark_data_smithi060_196073_object48850:head# 2024-09-20T22:20:25.529 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdda05d0:::benchmark_data_smithi060_196073_object32527:head# 2024-09-20T22:20:25.531 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdda701f:::benchmark_data_smithi060_196073_object16270:head# 2024-09-20T22:20:25.532 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bddca708:::benchmark_data_smithi060_196073_object5938:head# 2024-09-20T22:20:25.532 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdddae5f:::benchmark_data_smithi060_196073_object5013:head# 2024-09-20T22:20:25.534 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bddddd22:::benchmark_data_smithi060_196073_object37224:head# 2024-09-20T22:20:25.534 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdde8196:::benchmark_data_smithi060_196073_object55329:head# 2024-09-20T22:20:25.537 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdded7df:::benchmark_data_smithi060_196073_object28087:head# 2024-09-20T22:20:25.538 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bde000c6:::benchmark_data_smithi060_196073_object19296:head# 2024-09-20T22:20:25.541 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bde2ed49:::benchmark_data_smithi060_196073_object24706:head# 2024-09-20T22:20:25.541 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bde34ec5:::benchmark_data_smithi060_196073_object48745:head# 2024-09-20T22:20:25.541 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bde46db3:::benchmark_data_smithi060_196073_object6887:head# 2024-09-20T22:20:25.541 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bde689a5:::benchmark_data_smithi060_196073_object58734:head# 2024-09-20T22:20:25.541 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bde7588a:::benchmark_data_smithi060_196073_object12571:head# 2024-09-20T22:20:25.542 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bde7c2b5:::benchmark_data_smithi060_196073_object39099:head# 2024-09-20T22:20:25.542 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bde87c01:::benchmark_data_smithi060_196073_object60675:head# 2024-09-20T22:20:25.543 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bde9d5fd:::benchmark_data_smithi060_196073_object39142:head# 2024-09-20T22:20:25.543 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdea5c9f:::benchmark_data_smithi060_196073_object22385:head# 2024-09-20T22:20:25.544 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdeb0fdb:::benchmark_data_smithi060_196073_object49660:head# 2024-09-20T22:20:25.545 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdebeaa5:::benchmark_data_smithi060_196073_object38727:head# 2024-09-20T22:20:25.545 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdebed55:::benchmark_data_smithi060_196073_object16623:head# 2024-09-20T22:20:25.545 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdefadf9:::benchmark_data_smithi060_196073_object41604:head# 2024-09-20T22:20:25.546 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdf04a9a:::benchmark_data_smithi060_196073_object9597:head# 2024-09-20T22:20:25.547 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdf0b340:::benchmark_data_smithi060_196073_object44636:head# 2024-09-20T22:20:25.547 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdf23047:::benchmark_data_smithi060_196073_object24259:head# 2024-09-20T22:20:25.548 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdf26d81:::benchmark_data_smithi060_196073_object4997:head# 2024-09-20T22:20:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdf797c1:::benchmark_data_smithi060_196073_object37068:head# 2024-09-20T22:20:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdf7a79d:::benchmark_data_smithi060_196073_object60037:head# 2024-09-20T22:20:25.549 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdf95a32:::benchmark_data_smithi060_196073_object49914:head# 2024-09-20T22:20:25.552 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdfdec3e:::benchmark_data_smithi060_196073_object57400:head# 2024-09-20T22:20:25.552 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdfe5738:::benchmark_data_smithi060_196073_object8964:head# 2024-09-20T22:20:25.552 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdfefddc:::benchmark_data_smithi060_196073_object49847:head# 2024-09-20T22:20:25.554 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bdffea0a:::benchmark_data_smithi060_196073_object26455:head# 2024-09-20T22:20:25.554 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be004767:::benchmark_data_smithi060_196073_object61662:head# 2024-09-20T22:20:25.555 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be01590c:::benchmark_data_smithi060_196073_object60524:head# 2024-09-20T22:20:25.555 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be041687:::benchmark_data_smithi060_196073_object18078:head# 2024-09-20T22:20:25.555 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be067520:::benchmark_data_smithi060_196073_object54160:head# 2024-09-20T22:20:25.555 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be070508:::benchmark_data_smithi060_196073_object9035:head# 2024-09-20T22:20:25.555 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be07aaa4:::benchmark_data_smithi060_196073_object53461:head# 2024-09-20T22:20:25.556 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be07aae6:::benchmark_data_smithi060_196073_object22177:head# 2024-09-20T22:20:25.556 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be0bf805:::benchmark_data_smithi060_196073_object46298:head# 2024-09-20T22:20:25.557 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be0c64d4:::benchmark_data_smithi060_196073_object23746:head# 2024-09-20T22:20:25.558 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be0e72c3:::benchmark_data_smithi060_196073_object11414:head# 2024-09-20T22:20:25.558 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be0eb517:::benchmark_data_smithi060_196073_object42326:head# 2024-09-20T22:20:25.559 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be107ad0:::benchmark_data_smithi060_196073_object13219:head# 2024-09-20T22:20:25.559 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be10bc8f:::benchmark_data_smithi060_196073_object11105:head# 2024-09-20T22:20:25.560 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be10f57b:::benchmark_data_smithi060_196073_object23241:head# 2024-09-20T22:20:25.561 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be113470:::benchmark_data_smithi060_196073_object49440:head# 2024-09-20T22:20:25.563 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be1190f5:::benchmark_data_smithi060_196073_object45950:head# 2024-09-20T22:20:25.563 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be11aada:::benchmark_data_smithi060_196073_object31736:head# 2024-09-20T22:20:25.568 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be13fb2b:::benchmark_data_smithi060_196073_object32550:head# 2024-09-20T22:20:25.568 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be152065:::benchmark_data_smithi060_196073_object55843:head# 2024-09-20T22:20:25.568 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be1599cf:::benchmark_data_smithi060_196073_object23718:head# 2024-09-20T22:20:25.568 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be18a531:::benchmark_data_smithi060_196073_object23841:head# 2024-09-20T22:20:25.568 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be18a835:::benchmark_data_smithi060_196073_object23817:head# 2024-09-20T22:20:25.569 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be195d0b:::benchmark_data_smithi060_196073_object30480:head# 2024-09-20T22:20:25.569 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be197797:::benchmark_data_smithi060_196073_object42580:head# 2024-09-20T22:20:25.569 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be19aa03:::benchmark_data_smithi060_196073_object18641:head# 2024-09-20T22:20:25.569 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be1a04fb:::benchmark_data_smithi060_196073_object28103:head# 2024-09-20T22:20:25.569 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be1a0b07:::benchmark_data_smithi060_196073_object16638:head# 2024-09-20T22:20:25.570 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be1a980f:::benchmark_data_smithi060_196073_object7154:head# 2024-09-20T22:20:25.570 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be1aeb81:::benchmark_data_smithi060_196073_object15356:head# 2024-09-20T22:20:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be1c0054:::benchmark_data_smithi060_196073_object46250:head# 2024-09-20T22:20:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be1c5e96:::benchmark_data_smithi060_196073_object9205:head# 2024-09-20T22:20:25.572 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be1cba5c:::benchmark_data_smithi060_196073_object17479:head# 2024-09-20T22:20:25.579 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be1dadcf:::benchmark_data_smithi060_196073_object34500:head# 2024-09-20T22:20:25.579 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be1e9a0b:::benchmark_data_smithi060_196073_object34507:head# 2024-09-20T22:20:25.579 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be1f6b23:::benchmark_data_smithi060_196073_object48543:head# 2024-09-20T22:20:25.580 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be204be9:::benchmark_data_smithi060_196073_object32332:head# 2024-09-20T22:20:25.580 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be214e25:::benchmark_data_smithi060_196073_object41461:head# 2024-09-20T22:20:25.580 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be236804:::benchmark_data_smithi060_196073_object60716:head# 2024-09-20T22:20:25.580 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be242f2b:::benchmark_data_smithi060_196073_object23678:head# 2024-09-20T22:20:25.580 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be24ad02:::benchmark_data_smithi060_196073_object5411:head# 2024-09-20T22:20:25.580 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be276441:::benchmark_data_smithi060_196073_object57719:head# 2024-09-20T22:20:25.580 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be28ec54:::benchmark_data_smithi060_196073_object52510:head# 2024-09-20T22:20:25.580 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:25 smithi067 ceph-mon[33511]: pgmap v935: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 170 active+clean; 6.2 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1023 B/s rd, 51 MiB/s wr, 830 op/s; 9121/303336 objects degraded (3.007%); 22581/303336 objects misplaced (7.444%); 23 MiB/s, 366 objects/s recovering 2024-09-20T22:20:25.580 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:25 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:20:25.580 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:25 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2030588030' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:20:25.580 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:25 smithi067 ceph-mon[33511]: osdmap e494: 8 total, 7 up, 6 in 2024-09-20T22:20:25.582 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be292ddc:::benchmark_data_smithi060_196073_object45154:head# 2024-09-20T22:20:25.582 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be29ffa3:::benchmark_data_smithi060_196073_object49237:head# 2024-09-20T22:20:25.582 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be2c4c89:::benchmark_data_smithi060_196073_object55243:head# 2024-09-20T22:20:25.582 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be2d32f7:::benchmark_data_smithi060_196073_object47255:head# 2024-09-20T22:20:25.582 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be2d3e92:::benchmark_data_smithi060_196073_object10169:head# 2024-09-20T22:20:25.583 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be2efcf1:::benchmark_data_smithi060_196073_object48133:head# 2024-09-20T22:20:25.584 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be2fa4b3:::benchmark_data_smithi060_196073_object16501:head# 2024-09-20T22:20:25.585 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be31c9b5:::benchmark_data_smithi060_196073_object29399:head# 2024-09-20T22:20:25.585 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be32e1e1:::benchmark_data_smithi060_196073_object57165:head# 2024-09-20T22:20:25.586 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be332ebe:::benchmark_data_smithi060_196073_object60900:head# 2024-09-20T22:20:25.587 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be33c15c:::benchmark_data_smithi060_196073_object57585:head# 2024-09-20T22:20:25.588 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be344d1a:::benchmark_data_smithi060_196073_object42606:head# 2024-09-20T22:20:25.588 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be35f1ed:::benchmark_data_smithi060_196073_object43775:head# 2024-09-20T22:20:25.589 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be35fa1e:::benchmark_data_smithi060_196073_object15729:head# 2024-09-20T22:20:25.590 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be36ad18:::benchmark_data_smithi060_196073_object26362:head# 2024-09-20T22:20:25.592 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be36bf8a:::benchmark_data_smithi060_196073_object35032:head# 2024-09-20T22:20:25.593 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be380a1c:::benchmark_data_smithi060_196073_object37095:head# 2024-09-20T22:20:25.593 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be3a268f:::benchmark_data_smithi060_196073_object9224:head# 2024-09-20T22:20:25.594 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be3acb11:::benchmark_data_smithi060_196073_object52436:head# 2024-09-20T22:20:25.595 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be3b76b6:::benchmark_data_smithi060_196073_object17062:head# 2024-09-20T22:20:25.596 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be3bb13c:::benchmark_data_smithi060_196073_object37421:head# 2024-09-20T22:20:25.597 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be3f03dc:::benchmark_data_smithi060_196073_object53334:head# 2024-09-20T22:20:25.599 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be408886:::benchmark_data_smithi060_196073_object24206:head# 2024-09-20T22:20:25.599 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be40cc2f:::benchmark_data_smithi060_196073_object2800:head# 2024-09-20T22:20:25.599 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be411bf4:::benchmark_data_smithi060_196073_object56998:head# 2024-09-20T22:20:25.600 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be41bc93:::benchmark_data_smithi060_196073_object24584:head# 2024-09-20T22:20:25.601 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be425abc:::benchmark_data_smithi060_196073_object41995:head# 2024-09-20T22:20:25.603 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be42816d:::benchmark_data_smithi060_196073_object1480:head# 2024-09-20T22:20:25.605 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be442848:::benchmark_data_smithi060_196073_object5395:head# 2024-09-20T22:20:25.606 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be44ab80:::benchmark_data_smithi060_196073_object36079:head# 2024-09-20T22:20:25.607 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be454f8c:::benchmark_data_smithi060_196073_object43845:head# 2024-09-20T22:20:25.607 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be457cfe:::benchmark_data_smithi060_196073_object48024:head# 2024-09-20T22:20:25.608 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be47df79:::benchmark_data_smithi060_196073_object11973:head# 2024-09-20T22:20:25.609 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be4ab74b:::benchmark_data_smithi060_196073_object16407:head# 2024-09-20T22:20:25.609 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be4f1613:::benchmark_data_smithi060_196073_object60994:head# 2024-09-20T22:20:25.612 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:25.613 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be4fdc4e:::benchmark_data_smithi060_196073_object56995:head# 2024-09-20T22:20:25.613 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be506461:::benchmark_data_smithi060_196073_object53972:head# 2024-09-20T22:20:25.614 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be50b151:::benchmark_data_smithi060_196073_object24074:head# 2024-09-20T22:20:25.614 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be512950:::benchmark_data_smithi060_196073_object51953:head# 2024-09-20T22:20:25.615 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be5313a9:::benchmark_data_smithi060_196073_object18736:head# 2024-09-20T22:20:25.616 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be53403f:::benchmark_data_smithi060_196073_object44093:head# 2024-09-20T22:20:25.618 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be56c0d8:::benchmark_data_smithi060_196073_object35766:head# 2024-09-20T22:20:25.619 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be573652:::benchmark_data_smithi060_196073_object54304:head# 2024-09-20T22:20:25.621 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be573a33:::benchmark_data_smithi060_196073_object1157:head# 2024-09-20T22:20:25.622 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be57fad5:::benchmark_data_smithi060_196073_object15395:head# 2024-09-20T22:20:25.622 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be58ff0d:::benchmark_data_smithi060_196073_object37000:head# 2024-09-20T22:20:25.623 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be5b449c:::benchmark_data_smithi060_196073_object36580:head# 2024-09-20T22:20:25.624 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be5c180c:::benchmark_data_smithi060_196073_object53135:head# 2024-09-20T22:20:25.627 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be5c64c1:::benchmark_data_smithi060_196073_object57714:head# 2024-09-20T22:20:25.628 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be5ded44:::benchmark_data_smithi060_196073_object8293:head# 2024-09-20T22:20:25.628 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be5e9bc5:::benchmark_data_smithi060_196073_object44929:head# 2024-09-20T22:20:25.629 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be5f1fff:::benchmark_data_smithi060_196073_object25848:head# 2024-09-20T22:20:25.630 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be613b9b:::benchmark_data_smithi060_196073_object56449:head# 2024-09-20T22:20:25.633 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be618f62:::benchmark_data_smithi060_196073_object4194:head# 2024-09-20T22:20:25.634 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be626a90:::benchmark_data_smithi060_196073_object17915:head# 2024-09-20T22:20:25.635 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be627406:::benchmark_data_smithi060_196073_object40707:head# 2024-09-20T22:20:25.635 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be633d9e:::benchmark_data_smithi060_196073_object46695:head# 2024-09-20T22:20:25.636 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be63e820:::benchmark_data_smithi060_196073_object49498:head# 2024-09-20T22:20:25.636 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be6410f7:::benchmark_data_smithi060_196073_object16113:head# 2024-09-20T22:20:25.637 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be6438ac:::benchmark_data_smithi060_196073_object45629:head# 2024-09-20T22:20:25.639 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be6639ba:::benchmark_data_smithi060_196073_object20832:head# 2024-09-20T22:20:25.639 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be678962:::benchmark_data_smithi060_196073_object18004:head# 2024-09-20T22:20:25.640 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be6866f2:::benchmark_data_smithi060_196073_object47340:head# 2024-09-20T22:20:25.640 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be69496d:::benchmark_data_smithi060_196073_object565:head# 2024-09-20T22:20:25.640 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be696027:::benchmark_data_smithi060_196073_object32155:head# 2024-09-20T22:20:25.640 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be6988f0:::benchmark_data_smithi060_196073_object3129:head# 2024-09-20T22:20:25.641 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be6ae7c0:::benchmark_data_smithi060_196073_object18183:head# 2024-09-20T22:20:25.642 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be6b4994:::benchmark_data_smithi060_196073_object20364:head# 2024-09-20T22:20:25.642 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be6cbe66:::benchmark_data_smithi060_196073_object34205:head# 2024-09-20T22:20:25.643 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be70a7c2:::benchmark_data_smithi060_196073_object29068:head# 2024-09-20T22:20:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be70c76a:::benchmark_data_smithi060_196073_object58082:head# 2024-09-20T22:20:25.644 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be713d49:::benchmark_data_smithi060_196073_object40135:head# 2024-09-20T22:20:25.645 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be72837b:::benchmark_data_smithi060_196073_object29750:head# 2024-09-20T22:20:25.647 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be73c203:::benchmark_data_smithi060_196073_object44184:head# 2024-09-20T22:20:25.647 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be7402e6:::benchmark_data_smithi060_196073_object21204:head# 2024-09-20T22:20:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be75a4cd:::benchmark_data_smithi060_196073_object1567:head# 2024-09-20T22:20:25.648 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be7650e6:::benchmark_data_smithi060_196073_object355:head# 2024-09-20T22:20:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be79ae1d:::benchmark_data_smithi060_196073_object37382:head# 2024-09-20T22:20:25.649 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be7c7199:::benchmark_data_smithi060_196073_object36604:head# 2024-09-20T22:20:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be7ce226:::benchmark_data_smithi060_196073_object13976:head# 2024-09-20T22:20:25.650 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be7d23de:::benchmark_data_smithi060_196073_object20897:head# 2024-09-20T22:20:25.651 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be7dcef4:::benchmark_data_smithi060_196073_object8678:head# 2024-09-20T22:20:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be7e0d0b:::benchmark_data_smithi060_196073_object56330:head# 2024-09-20T22:20:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be7f541a:::benchmark_data_smithi060_196073_object29097:head# 2024-09-20T22:20:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be80c8da:::benchmark_data_smithi060_196073_object48245:head# 2024-09-20T22:20:25.653 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be8195c0:::benchmark_data_smithi060_196073_object14676:head# 2024-09-20T22:20:25.654 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be82a0f8:::benchmark_data_smithi060_196073_object33688:head# 2024-09-20T22:20:25.654 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be8526d1:::benchmark_data_smithi060_196073_object10583:head# 2024-09-20T22:20:25.655 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be852901:::benchmark_data_smithi060_196073_object42968:head# 2024-09-20T22:20:25.655 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be853394:::benchmark_data_smithi060_196073_object56464:head# 2024-09-20T22:20:25.656 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be85fa69:::benchmark_data_smithi060_196073_object8265:head# 2024-09-20T22:20:25.657 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be8606f1:::benchmark_data_smithi060_196073_object46235:head# 2024-09-20T22:20:25.657 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be879d81:::benchmark_data_smithi060_196073_object42764:head# 2024-09-20T22:20:25.658 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be886fc5:::benchmark_data_smithi060_196073_object25643:head# 2024-09-20T22:20:25.659 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be8ad4a7:::benchmark_data_smithi060_196073_object7191:head# 2024-09-20T22:20:25.659 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be8b5304:::benchmark_data_smithi060_196073_object29703:head# 2024-09-20T22:20:25.659 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be8e5b9c:::benchmark_data_smithi060_196073_object47763:head# 2024-09-20T22:20:25.661 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be8f0310:::benchmark_data_smithi060_196073_object42327:head# 2024-09-20T22:20:25.661 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be8f450f:::benchmark_data_smithi060_196073_object51731:head# 2024-09-20T22:20:25.661 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be8f6327:::benchmark_data_smithi060_196073_object6914:head# 2024-09-20T22:20:25.662 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be918bd8:::benchmark_data_smithi060_196073_object34978:head# 2024-09-20T22:20:25.663 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be9206ba:::benchmark_data_smithi060_196073_object13029:head# 2024-09-20T22:20:25.664 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be92bb18:::benchmark_data_smithi060_196073_object11472:head# 2024-09-20T22:20:25.664 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be959dda:::benchmark_data_smithi060_196073_object47538:head# 2024-09-20T22:20:25.665 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be96cc5a:::benchmark_data_smithi060_196073_object41214:head# 2024-09-20T22:20:25.666 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:25.669 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be9772b2:::benchmark_data_smithi060_196073_object34795:head# 2024-09-20T22:20:25.669 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be9846a9:::benchmark_data_smithi060_196073_object2742:head# 2024-09-20T22:20:25.669 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be987cf6:::benchmark_data_smithi060_196073_object53743:head# 2024-09-20T22:20:25.670 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be98e603:::benchmark_data_smithi060_196073_object21851:head# 2024-09-20T22:20:25.670 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be9cd1ae:::benchmark_data_smithi060_196073_object31669:head# 2024-09-20T22:20:25.670 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be9d44f8:::benchmark_data_smithi060_196073_object25385:head# 2024-09-20T22:20:25.670 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:25.670 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:25.671 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:be9d7fd0:::benchmark_data_smithi060_196073_object57715:head# 2024-09-20T22:20:25.671 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bea1ecbf:::benchmark_data_smithi060_196073_object21416:head# 2024-09-20T22:20:25.672 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bea32f36:::benchmark_data_smithi060_196073_object2827:head# 2024-09-20T22:20:25.673 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bea46bfe:::benchmark_data_smithi060_196073_object19689:head# 2024-09-20T22:20:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bea4e2a9:::benchmark_data_smithi060_196073_object41470:head# 2024-09-20T22:20:25.674 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bea7f86b:::benchmark_data_smithi060_196073_object22790:head# 2024-09-20T22:20:25.675 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bea8006e:::benchmark_data_smithi060_196073_object35406:head# 2024-09-20T22:20:25.676 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bea9840e:::benchmark_data_smithi060_196073_object28915:head# 2024-09-20T22:20:25.676 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bea986fe:::benchmark_data_smithi060_196073_object30447:head# 2024-09-20T22:20:25.677 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beaa03f2:::benchmark_data_smithi060_196073_object35169:head# 2024-09-20T22:20:25.678 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beac048a:::benchmark_data_smithi060_196073_object19604:head# 2024-09-20T22:20:25.679 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beac4a1a:::benchmark_data_smithi060_196073_object45273:head# 2024-09-20T22:20:25.679 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beac9877:::benchmark_data_smithi060_196073_object38017:head# 2024-09-20T22:20:25.680 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beacb3b8:::benchmark_data_smithi060_196073_object19408:head# 2024-09-20T22:20:25.681 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beacbea1:::benchmark_data_smithi060_196073_object47484:head# 2024-09-20T22:20:25.682 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beae72ea:::benchmark_data_smithi060_196073_object48415:head# 2024-09-20T22:20:25.683 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beaedcf6:::benchmark_data_smithi060_196073_object19902:head# 2024-09-20T22:20:25.683 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beb00a29:::benchmark_data_smithi060_196073_object60848:head# 2024-09-20T22:20:25.684 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beb03bb0:::benchmark_data_smithi060_196073_object29712:head# 2024-09-20T22:20:25.685 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beb105b1:::benchmark_data_smithi060_196073_object45510:head# 2024-09-20T22:20:25.686 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beb1969b:::benchmark_data_smithi060_196073_object1713:head# 2024-09-20T22:20:25.686 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beb2268f:::benchmark_data_smithi060_196073_object21979:head# 2024-09-20T22:20:25.687 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beb2a764:::benchmark_data_smithi060_196073_object51856:head# 2024-09-20T22:20:25.687 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beb2a89b:::benchmark_data_smithi060_196073_object34864:head# 2024-09-20T22:20:25.691 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beb409ba:::benchmark_data_smithi060_196073_object13595:head# 2024-09-20T22:20:25.691 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beb4e744:::benchmark_data_smithi060_196073_object4177:head# 2024-09-20T22:20:25.693 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beb658d6:::benchmark_data_smithi060_196073_object27586:head# 2024-09-20T22:20:25.693 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beb74a5d:::benchmark_data_smithi060_196073_object42388:head# 2024-09-20T22:20:25.693 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beb95c92:::benchmark_data_smithi060_196073_object39772:head# 2024-09-20T22:20:25.694 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beb967c5:::benchmark_data_smithi060_196073_object54306:head# 2024-09-20T22:20:25.694 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bebcd17f:::benchmark_data_smithi060_196073_object25873:head# 2024-09-20T22:20:25.695 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bebcfad6:::benchmark_data_smithi060_196073_object23717:head# 2024-09-20T22:20:25.695 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bebd4533:::benchmark_data_smithi060_196073_object18998:head# 2024-09-20T22:20:25.696 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bebdb1eb:::benchmark_data_smithi060_196073_object47531:head# 2024-09-20T22:20:25.698 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bebe504a:::benchmark_data_smithi060_196073_object19491:head# 2024-09-20T22:20:25.698 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bebfa0b4:::benchmark_data_smithi060_196073_object56804:head# 2024-09-20T22:20:25.701 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bebfa435:::benchmark_data_smithi060_196073_object23916:head# 2024-09-20T22:20:25.701 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bec245b3:::benchmark_data_smithi060_196073_object32727:head# 2024-09-20T22:20:25.701 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bec4ceb9:::benchmark_data_smithi060_196073_object9098:head# 2024-09-20T22:20:25.702 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bec517fa:::benchmark_data_smithi060_196073_object33829:head# 2024-09-20T22:20:25.702 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bec5b789:::benchmark_data_smithi060_196073_object15242:head# 2024-09-20T22:20:25.703 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bec958b1:::benchmark_data_smithi060_196073_object15764:head# 2024-09-20T22:20:25.703 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:becc6417:::benchmark_data_smithi060_196073_object43484:head# 2024-09-20T22:20:25.704 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:becc8f64:::benchmark_data_smithi060_196073_object57696:head# 2024-09-20T22:20:25.704 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beccae52:::benchmark_data_smithi060_196073_object20531:head# 2024-09-20T22:20:25.705 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:becd15ac:::benchmark_data_smithi060_196073_object32347:head# 2024-09-20T22:20:25.705 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:becdb643:::benchmark_data_smithi060_196073_object370:head# 2024-09-20T22:20:25.706 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bece44ad:::benchmark_data_smithi060_196073_object46300:head# 2024-09-20T22:20:25.706 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bece6697:::benchmark_data_smithi060_196073_object4592:head# 2024-09-20T22:20:25.707 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bed0153c:::benchmark_data_smithi060_196073_object44366:head# 2024-09-20T22:20:25.708 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bed05f1d:::benchmark_data_smithi060_196073_object7280:head# 2024-09-20T22:20:25.708 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bed09857:::benchmark_data_smithi060_196073_object52460:head# 2024-09-20T22:20:25.709 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bed30272:::benchmark_data_smithi060_196073_object37556:head# 2024-09-20T22:20:25.711 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bed6108f:::benchmark_data_smithi060_196073_object7858:head# 2024-09-20T22:20:25.713 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bed6ca18:::benchmark_data_smithi060_196073_object17972:head# 2024-09-20T22:20:25.713 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bed8b7b1:::benchmark_data_smithi060_196073_object53931:head# 2024-09-20T22:20:25.714 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bedb04ad:::benchmark_data_smithi060_196073_object5312:head# 2024-09-20T22:20:25.714 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bedb115c:::benchmark_data_smithi060_196073_object48088:head# 2024-09-20T22:20:25.715 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bedbfafa:::benchmark_data_smithi060_196073_object51626:head# 2024-09-20T22:20:25.715 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bedc8edd:::benchmark_data_smithi060_196073_object8895:head# 2024-09-20T22:20:25.716 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bedd8d3e:::benchmark_data_smithi060_196073_object29479:head# 2024-09-20T22:20:25.716 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bedef3d0:::benchmark_data_smithi060_196073_object33497:head# 2024-09-20T22:20:25.717 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bee18ebb:::benchmark_data_smithi060_196073_object37408:head# 2024-09-20T22:20:25.717 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bee2babb:::benchmark_data_smithi060_196073_object60817:head# 2024-09-20T22:20:25.719 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bee540d6:::benchmark_data_smithi060_196073_object8135:head# 2024-09-20T22:20:25.719 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bee5d685:::benchmark_data_smithi060_196073_object26107:head# 2024-09-20T22:20:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bee7f822:::benchmark_data_smithi060_196073_object48169:head# 2024-09-20T22:20:25.720 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bee9c5c5:::benchmark_data_smithi060_196073_object27845:head# 2024-09-20T22:20:25.721 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bee9f912:::benchmark_data_smithi060_196073_object17680:head# 2024-09-20T22:20:25.721 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beeacc0a:::benchmark_data_smithi060_196073_object55712:head# 2024-09-20T22:20:25.722 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beead88b:::benchmark_data_smithi060_196073_object28272:head# 2024-09-20T22:20:25.722 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beeae637:::benchmark_data_smithi060_196073_object54023:head# 2024-09-20T22:20:25.723 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beee4a94:::benchmark_data_smithi060_196073_object39002:head# 2024-09-20T22:20:25.723 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beef33ff:::benchmark_data_smithi060_196073_object45816:head# 2024-09-20T22:20:25.724 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bef007b1:::benchmark_data_smithi060_196073_object25237:head# 2024-09-20T22:20:25.725 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bef103c4:::benchmark_data_smithi060_196073_object39863:head# 2024-09-20T22:20:25.726 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bef1420a:::benchmark_data_smithi060_196073_object26783:head# 2024-09-20T22:20:25.726 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bef1ff0f:::benchmark_data_smithi060_196073_object13832:head# 2024-09-20T22:20:25.727 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bef29835:::benchmark_data_smithi060_196073_object40479:head# 2024-09-20T22:20:25.727 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bef322ef:::benchmark_data_smithi060_196073_object36775:head# 2024-09-20T22:20:25.727 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bef5b39a:::benchmark_data_smithi060_196073_object22340:head# 2024-09-20T22:20:25.728 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bef6565f:::benchmark_data_smithi060_196073_object38168:head# 2024-09-20T22:20:25.728 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bef66184:::benchmark_data_smithi060_196073_object7519:head# 2024-09-20T22:20:25.729 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bef6d1fb:::benchmark_data_smithi060_196073_object38704:head# 2024-09-20T22:20:25.729 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bef97dea:::benchmark_data_smithi060_196073_object46783:head# 2024-09-20T22:20:25.730 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bef98044:::benchmark_data_smithi060_196073_object35992:head# 2024-09-20T22:20:25.730 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bef9a2b6:::benchmark_data_smithi060_196073_object48418:head# 2024-09-20T22:20:25.731 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bef9d0c6:::benchmark_data_smithi060_196073_object41294:head# 2024-09-20T22:20:25.732 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:befa6387:::benchmark_data_smithi060_196073_object51904:head# 2024-09-20T22:20:25.734 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:befc97f2:::benchmark_data_smithi060_196073_object44403:head# 2024-09-20T22:20:25.734 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:befcc443:::benchmark_data_smithi060_196073_object20352:head# 2024-09-20T22:20:25.734 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:befcdbbd:::benchmark_data_smithi060_196073_object5641:head# 2024-09-20T22:20:25.735 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:befe8ee7:::benchmark_data_smithi060_196073_object27486:head# 2024-09-20T22:20:25.735 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beff6cf0:::benchmark_data_smithi060_196073_object47639:head# 2024-09-20T22:20:25.736 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:beff76b1:::benchmark_data_smithi060_196073_object39574:head# 2024-09-20T22:20:25.737 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf010cf7:::benchmark_data_smithi060_196073_object36947:head# 2024-09-20T22:20:25.737 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf020781:::benchmark_data_smithi060_196073_object966:head# 2024-09-20T22:20:25.738 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf020d5f:::benchmark_data_smithi060_196073_object59444:head# 2024-09-20T22:20:25.740 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf025c69:::benchmark_data_smithi060_196073_object15181:head# 2024-09-20T22:20:25.740 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf02d8b9:::benchmark_data_smithi060_196073_object33348:head# 2024-09-20T22:20:25.741 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf02fbf0:::benchmark_data_smithi060_196073_object11285:head# 2024-09-20T22:20:25.742 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf044926:::benchmark_data_smithi060_196073_object16493:head# 2024-09-20T22:20:25.742 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf0498e6:::benchmark_data_smithi060_196073_object44185:head# 2024-09-20T22:20:25.744 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf05a9b4:::benchmark_data_smithi060_196073_object53748:head# 2024-09-20T22:20:25.744 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf05dfa3:::benchmark_data_smithi060_196073_object18170:head# 2024-09-20T22:20:25.744 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf07b6e4:::benchmark_data_smithi060_196073_object53815:head# 2024-09-20T22:20:25.746 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf095d0d:::benchmark_data_smithi060_196073_object24148:head# 2024-09-20T22:20:25.748 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf0a2324:::benchmark_data_smithi060_196073_object61094:head# 2024-09-20T22:20:25.748 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf0aff8b:::benchmark_data_smithi060_196073_object1525:head# 2024-09-20T22:20:25.748 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf0b3574:::benchmark_data_smithi060_196073_object30497:head# 2024-09-20T22:20:25.752 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf0b9ef2:::benchmark_data_smithi060_196073_object45654:head# 2024-09-20T22:20:25.752 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf0bcde6:::benchmark_data_smithi060_196073_object11316:head# 2024-09-20T22:20:25.753 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf0fff43:::benchmark_data_smithi060_196073_object44829:head# 2024-09-20T22:20:25.753 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf10b093:::benchmark_data_smithi060_196073_object38756:head# 2024-09-20T22:20:25.753 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf1462f4:::benchmark_data_smithi060_196073_object41234:head# 2024-09-20T22:20:25.753 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf15a9b9:::benchmark_data_smithi060_196073_object51220:head# 2024-09-20T22:20:25.756 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf15af25:::benchmark_data_smithi060_196073_object10590:head# 2024-09-20T22:20:25.756 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf176946:::benchmark_data_smithi060_196073_object57318:head# 2024-09-20T22:20:25.757 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf184d89:::benchmark_data_smithi060_196073_object38507:head# 2024-09-20T22:20:25.757 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf190470:::benchmark_data_smithi060_196073_object1641:head# 2024-09-20T22:20:25.757 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf194eb3:::benchmark_data_smithi060_196073_object31304:head# 2024-09-20T22:20:25.764 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf19953b:::benchmark_data_smithi060_196073_object40243:head# 2024-09-20T22:20:25.764 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf1a30fc:::benchmark_data_smithi060_196073_object55696:head# 2024-09-20T22:20:25.764 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf1a388a:::benchmark_data_smithi060_196073_object11511:head# 2024-09-20T22:20:25.764 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf1a50d3:::benchmark_data_smithi060_196073_object289:head# 2024-09-20T22:20:25.764 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf1c8755:::benchmark_data_smithi060_196073_object13422:head# 2024-09-20T22:20:25.764 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf1d3326:::benchmark_data_smithi060_196073_object8095:head# 2024-09-20T22:20:25.764 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf1d9493:::benchmark_data_smithi060_196073_object38707:head# 2024-09-20T22:20:25.764 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf1eb36e:::benchmark_data_smithi060_196073_object6085:head# 2024-09-20T22:20:25.765 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf1f0ade:::benchmark_data_smithi060_196073_object46085:head# 2024-09-20T22:20:25.765 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf202acb:::benchmark_data_smithi060_196073_object35980:head# 2024-09-20T22:20:25.765 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf20d650:::benchmark_data_smithi060_196073_object51226:head# 2024-09-20T22:20:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf213518:::benchmark_data_smithi060_196073_object28394:head# 2024-09-20T22:20:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf21b02c:::benchmark_data_smithi060_196073_object55067:head# 2024-09-20T22:20:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf2294b6:::benchmark_data_smithi060_196073_object21523:head# 2024-09-20T22:20:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf22ed3c:::benchmark_data_smithi060_196073_object6163:head# 2024-09-20T22:20:25.767 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf23a22e:::benchmark_data_smithi060_196073_object25718:head# 2024-09-20T22:20:25.768 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf246aea:::benchmark_data_smithi060_196073_object3365:head# 2024-09-20T22:20:25.769 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf26eb27:::benchmark_data_smithi060_196073_object30996:head# 2024-09-20T22:20:25.769 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf274fca:::benchmark_data_smithi060_196073_object52598:head# 2024-09-20T22:20:25.770 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf281dc1:::benchmark_data_smithi060_196073_object15851:head# 2024-09-20T22:20:25.771 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf296769:::benchmark_data_smithi060_196073_object61679:head# 2024-09-20T22:20:25.771 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:25.772 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf2b2577:::benchmark_data_smithi060_196073_object61053:head# 2024-09-20T22:20:25.772 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf2b8d59:::benchmark_data_smithi060_196073_object5179:head# 2024-09-20T22:20:25.775 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf2cdc14:::benchmark_data_smithi060_196073_object44195:head# 2024-09-20T22:20:25.776 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf301638:::benchmark_data_smithi060_196073_object35807:head# 2024-09-20T22:20:25.776 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf31b9c9:::benchmark_data_smithi060_196073_object59627:head# 2024-09-20T22:20:25.777 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf31e4e9:::benchmark_data_smithi060_196073_object61449:head# 2024-09-20T22:20:25.778 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf32c7fb:::benchmark_data_smithi060_196073_object17281:head# 2024-09-20T22:20:25.778 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf3437b3:::benchmark_data_smithi060_196073_object38297:head# 2024-09-20T22:20:25.779 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf34ad9e:::benchmark_data_smithi060_196073_object27027:head# 2024-09-20T22:20:25.780 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf34bff8:::benchmark_data_smithi060_196073_object331:head# 2024-09-20T22:20:25.780 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf3759d8:::benchmark_data_smithi060_196073_object38380:head# 2024-09-20T22:20:25.781 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf3878dd:::benchmark_data_smithi060_196073_object25847:head# 2024-09-20T22:20:25.782 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf3a1b2e:::benchmark_data_smithi060_196073_object25369:head# 2024-09-20T22:20:25.782 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf3a71fc:::benchmark_data_smithi060_196073_object40155:head# 2024-09-20T22:20:25.783 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf3c79a9:::benchmark_data_smithi060_196073_object31806:head# 2024-09-20T22:20:25.784 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf3c8c5a:::benchmark_data_smithi060_196073_object54763:head# 2024-09-20T22:20:25.787 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf3dfa98:::benchmark_data_smithi060_196073_object5737:head# 2024-09-20T22:20:25.787 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf3ec385:::benchmark_data_smithi060_196073_object58127:head# 2024-09-20T22:20:25.790 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf3f5586:::benchmark_data_smithi060_196073_object48891:head# 2024-09-20T22:20:25.790 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf401040:::benchmark_data_smithi060_196073_object16575:head# 2024-09-20T22:20:25.791 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf4096f1:::benchmark_data_smithi060_196073_object60706:head# 2024-09-20T22:20:25.792 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf416624:::benchmark_data_smithi060_196073_object20878:head# 2024-09-20T22:20:25.793 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf440774:::benchmark_data_smithi060_196073_object46624:head# 2024-09-20T22:20:25.793 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf4542ae:::benchmark_data_smithi060_196073_object59293:head# 2024-09-20T22:20:25.794 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf45c4f3:::benchmark_data_smithi060_196073_object49571:head# 2024-09-20T22:20:25.794 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf4662b4:::benchmark_data_smithi060_196073_object17323:head# 2024-09-20T22:20:25.795 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf472cf8:::benchmark_data_smithi060_196073_object55421:head# 2024-09-20T22:20:25.796 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf4771fb:::benchmark_data_smithi060_196073_object19925:head# 2024-09-20T22:20:25.796 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf478130:::benchmark_data_smithi060_196073_object5041:head# 2024-09-20T22:20:25.797 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf48810d:::benchmark_data_smithi060_196073_object3667:head# 2024-09-20T22:20:25.798 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf48db2f:::benchmark_data_smithi060_196073_object27204:head# 2024-09-20T22:20:25.799 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf48ea05:::benchmark_data_smithi060_196073_object14590:head# 2024-09-20T22:20:25.799 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf49a485:::benchmark_data_smithi060_196073_object32875:head# 2024-09-20T22:20:25.800 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf4a27aa:::benchmark_data_smithi060_196073_object47521:head# 2024-09-20T22:20:25.801 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf4c73c6:::benchmark_data_smithi060_196073_object46961:head# 2024-09-20T22:20:25.802 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf4d7081:::benchmark_data_smithi060_196073_object58387:head# 2024-09-20T22:20:25.803 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf4edd8f:::benchmark_data_smithi060_196073_object39416:head# 2024-09-20T22:20:25.804 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf4f2712:::benchmark_data_smithi060_196073_object8682:head# 2024-09-20T22:20:25.807 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf4f8aa3:::benchmark_data_smithi060_196073_object5117:head# 2024-09-20T22:20:25.808 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf4f90a1:::benchmark_data_smithi060_196073_object26251:head# 2024-09-20T22:20:25.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:25.810 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:25.810 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:25.841 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf50a2fb:::benchmark_data_smithi060_196073_object25010:head# 2024-09-20T22:20:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf5100f0:::benchmark_data_smithi060_196073_object28105:head# 2024-09-20T22:20:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf5433a0:::benchmark_data_smithi060_196073_object27960:head# 2024-09-20T22:20:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf554989:::benchmark_data_smithi060_196073_object44327:head# 2024-09-20T22:20:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf5613d1:::benchmark_data_smithi060_196073_object36258:head# 2024-09-20T22:20:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf56c048:::benchmark_data_smithi060_196073_object46044:head# 2024-09-20T22:20:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf57064f:::benchmark_data_smithi060_196073_object12249:head# 2024-09-20T22:20:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf572251:::benchmark_data_smithi060_196073_object24548:head# 2024-09-20T22:20:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf59962c:::benchmark_data_smithi060_196073_object18196:head# 2024-09-20T22:20:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf59b242:::benchmark_data_smithi060_196073_object38698:head# 2024-09-20T22:20:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf59e55d:::benchmark_data_smithi060_196073_object12799:head# 2024-09-20T22:20:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf5a6553:::benchmark_data_smithi060_196073_object40178:head# 2024-09-20T22:20:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf5afea2:::benchmark_data_smithi060_196073_object32062:head# 2024-09-20T22:20:25.842 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf5b5485:::benchmark_data_smithi060_196073_object55619:head# 2024-09-20T22:20:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf5c2216:::benchmark_data_smithi060_196073_object38293:head# 2024-09-20T22:20:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf5e9f93:::benchmark_data_smithi060_196073_object52979:head# 2024-09-20T22:20:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf5eedd9:::benchmark_data_smithi060_196073_object52255:head# 2024-09-20T22:20:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf63668f:::benchmark_data_smithi060_196073_object5802:head# 2024-09-20T22:20:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf63869c:::benchmark_data_smithi060_196073_object53348:head# 2024-09-20T22:20:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf660b0a:::benchmark_data_smithi060_196073_object16520:head# 2024-09-20T22:20:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf678344:::benchmark_data_smithi060_196073_object19539:head# 2024-09-20T22:20:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf679fd3:::benchmark_data_smithi060_196073_object18798:head# 2024-09-20T22:20:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf68016f:::benchmark_data_smithi060_196073_object53434:head# 2024-09-20T22:20:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf683e6b:::benchmark_data_smithi060_196073_object49395:head# 2024-09-20T22:20:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf6c2e0b:::benchmark_data_smithi060_196073_object21791:head# 2024-09-20T22:20:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf6d822f:::benchmark_data_smithi060_196073_object19176:head# 2024-09-20T22:20:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf73196a:::benchmark_data_smithi060_196073_object21452:head# 2024-09-20T22:20:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf73ac9c:::benchmark_data_smithi060_196073_object25719:head# 2024-09-20T22:20:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf7480ec:::benchmark_data_smithi060_196073_object45793:head# 2024-09-20T22:20:25.843 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf751b63:::benchmark_data_smithi060_196073_object40646:head# 2024-09-20T22:20:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf7647e9:::benchmark_data_smithi060_196073_object30719:head# 2024-09-20T22:20:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf76a800:::benchmark_data_smithi060_196073_object56525:head# 2024-09-20T22:20:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf78b14a:::benchmark_data_smithi060_196073_object47000:head# 2024-09-20T22:20:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf78d4af:::benchmark_data_smithi060_196073_object42098:head# 2024-09-20T22:20:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf79bfee:::benchmark_data_smithi060_196073_object40265:head# 2024-09-20T22:20:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf7b42f5:::benchmark_data_smithi060_196073_object56884:head# 2024-09-20T22:20:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf7bf830:::benchmark_data_smithi060_196073_object40648:head# 2024-09-20T22:20:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf7ca3fc:::benchmark_data_smithi060_196073_object14141:head# 2024-09-20T22:20:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf7d2d39:::benchmark_data_smithi060_196073_object59994:head# 2024-09-20T22:20:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf7d6c93:::benchmark_data_smithi060_196073_object59050:head# 2024-09-20T22:20:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf7ef371:::benchmark_data_smithi060_196073_object16756:head# 2024-09-20T22:20:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf806a40:::benchmark_data_smithi060_196073_object45036:head# 2024-09-20T22:20:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf853c32:::benchmark_data_smithi060_196073_object37513:head# 2024-09-20T22:20:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf857c77:::benchmark_data_smithi060_196073_object45071:head# 2024-09-20T22:20:25.844 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf859aa2:::benchmark_data_smithi060_196073_object42113:head# 2024-09-20T22:20:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf874434:::benchmark_data_smithi060_196073_object30543:head# 2024-09-20T22:20:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf8779cf:::benchmark_data_smithi060_196073_object45558:head# 2024-09-20T22:20:25.845 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf887612:::benchmark_data_smithi060_196073_object47558:head# 2024-09-20T22:20:25.849 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf889676:::benchmark_data_smithi060_196073_object4944:head# 2024-09-20T22:20:25.850 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf88b4ef:::benchmark_data_smithi060_196073_object22897:head# 2024-09-20T22:20:25.850 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf8906d9:::benchmark_data_smithi060_196073_object21044:head# 2024-09-20T22:20:25.850 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf8a678e:::benchmark_data_smithi060_196073_object7642:head# 2024-09-20T22:20:25.850 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf8a9b6b:::benchmark_data_smithi060_196073_object11950:head# 2024-09-20T22:20:25.850 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf8b461f:::benchmark_data_smithi060_196073_object10685:head# 2024-09-20T22:20:25.850 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf8c37d2:::benchmark_data_smithi060_196073_object56814:head# 2024-09-20T22:20:25.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf8f29da:::benchmark_data_smithi060_196073_object25507:head# 2024-09-20T22:20:25.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf907269:::benchmark_data_smithi060_196073_object34429:head# 2024-09-20T22:20:25.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf9092c4:::benchmark_data_smithi060_196073_object34464:head# 2024-09-20T22:20:25.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf9113a8:::benchmark_data_smithi060_196073_object26619:head# 2024-09-20T22:20:25.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf91bbc5:::benchmark_data_smithi060_196073_object1055:head# 2024-09-20T22:20:25.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf91f4da:::benchmark_data_smithi060_196073_object7149:head# 2024-09-20T22:20:25.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf929820:::benchmark_data_smithi060_196073_object44442:head# 2024-09-20T22:20:25.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf939aeb:::benchmark_data_smithi060_196073_object11056:head# 2024-09-20T22:20:25.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf93ab47:::benchmark_data_smithi060_196073_object12780:head# 2024-09-20T22:20:25.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf9433f2:::benchmark_data_smithi060_196073_object22309:head# 2024-09-20T22:20:25.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf94d18a:::benchmark_data_smithi060_196073_object42286:head# 2024-09-20T22:20:25.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf96f16c:::benchmark_data_smithi060_196073_object61156:head# 2024-09-20T22:20:25.878 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf976e8e:::benchmark_data_smithi060_196073_object2094:head# 2024-09-20T22:20:25.879 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf98e3a5:::benchmark_data_smithi060_196073_object32831:head# 2024-09-20T22:20:25.879 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf9987f4:::benchmark_data_smithi060_196073_object18325:head# 2024-09-20T22:20:25.879 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf9b8a42:::benchmark_data_smithi060_196073_object54942:head# 2024-09-20T22:20:25.879 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf9c8ee1:::benchmark_data_smithi060_196073_object54650:head# 2024-09-20T22:20:25.879 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf9cde30:::benchmark_data_smithi060_196073_object40583:head# 2024-09-20T22:20:25.879 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf9e602c:::benchmark_data_smithi060_196073_object17205:head# 2024-09-20T22:20:25.879 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf9e849e:::benchmark_data_smithi060_196073_object8225:head# 2024-09-20T22:20:25.879 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf9f68ad:::benchmark_data_smithi060_196073_object10981:head# 2024-09-20T22:20:25.879 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bf9fe015:::benchmark_data_smithi060_196073_object59578:head# 2024-09-20T22:20:25.879 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfa16d36:::benchmark_data_smithi060_196073_object25293:head# 2024-09-20T22:20:25.879 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfa3c8fa:::benchmark_data_smithi060_196073_object17181:head# 2024-09-20T22:20:25.879 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfa3cb7f:::benchmark_data_smithi060_196073_object57423:head# 2024-09-20T22:20:25.879 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfa3d5f7:::benchmark_data_smithi060_196073_object57633:head# 2024-09-20T22:20:25.879 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfa5f2e2:::benchmark_data_smithi060_196073_object31010:head# 2024-09-20T22:20:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfa6c414:::benchmark_data_smithi060_196073_object7535:head# 2024-09-20T22:20:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfa6fd8a:::benchmark_data_smithi060_196073_object38912:head# 2024-09-20T22:20:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfab6db4:::benchmark_data_smithi060_196073_object18952:head# 2024-09-20T22:20:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfabe7ba:::benchmark_data_smithi060_196073_object20638:head# 2024-09-20T22:20:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfacf6cc:::benchmark_data_smithi060_196073_object50461:head# 2024-09-20T22:20:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfad85cd:::benchmark_data_smithi060_196073_object21077:head# 2024-09-20T22:20:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfadabfd:::benchmark_data_smithi060_196073_object21294:head# 2024-09-20T22:20:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfadd9a6:::benchmark_data_smithi060_196073_object37943:head# 2024-09-20T22:20:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfade1a7:::benchmark_data_smithi060_196073_object59362:head# 2024-09-20T22:20:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfadf077:::benchmark_data_smithi060_196073_object7871:head# 2024-09-20T22:20:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfadfcb1:::benchmark_data_smithi060_196073_object9641:head# 2024-09-20T22:20:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfae0328:::benchmark_data_smithi060_196073_object32924:head# 2024-09-20T22:20:25.880 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfae54aa:::benchmark_data_smithi060_196073_object10793:head# 2024-09-20T22:20:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfaf0398:::benchmark_data_smithi060_196073_object3297:head# 2024-09-20T22:20:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfaf7bb1:::benchmark_data_smithi060_196073_object30972:head# 2024-09-20T22:20:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfb215a1:::benchmark_data_smithi060_196073_object37346:head# 2024-09-20T22:20:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfb3d92e:::benchmark_data_smithi060_196073_object25969:head# 2024-09-20T22:20:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfb4d85e:::benchmark_data_smithi060_196073_object50528:head# 2024-09-20T22:20:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfb5da1b:::benchmark_data_smithi060_196073_object55518:head# 2024-09-20T22:20:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfb5e50d:::benchmark_data_smithi060_196073_object19764:head# 2024-09-20T22:20:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfb72842:::benchmark_data_smithi060_196073_object10413:head# 2024-09-20T22:20:25.892 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfb76b3b:::benchmark_data_smithi060_196073_object43602:head# 2024-09-20T22:20:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfb9223d:::benchmark_data_smithi060_196073_object44173:head# 2024-09-20T22:20:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfb95d1d:::benchmark_data_smithi060_196073_object50164:head# 2024-09-20T22:20:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfb985ac:::benchmark_data_smithi060_196073_object45423:head# 2024-09-20T22:20:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfb99ad4:::benchmark_data_smithi060_196073_object60462:head# 2024-09-20T22:20:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfb9ba20:::benchmark_data_smithi060_196073_object10823:head# 2024-09-20T22:20:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfbb326b:::benchmark_data_smithi060_196073_object24900:head# 2024-09-20T22:20:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfbdacae:::benchmark_data_smithi060_196073_object48239:head# 2024-09-20T22:20:25.893 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfbebdd3:::benchmark_data_smithi060_196073_object4438:head# 2024-09-20T22:20:25.896 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfc29be8:::benchmark_data_smithi060_196073_object38951:head# 2024-09-20T22:20:25.896 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfc49771:::benchmark_data_smithi060_196073_object41612:head# 2024-09-20T22:20:25.896 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfc4af2f:::benchmark_data_smithi060_196073_object13298:head# 2024-09-20T22:20:25.896 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfc4d5dc:::benchmark_data_smithi060_196073_object51887:head# 2024-09-20T22:20:25.896 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfc4eac8:::benchmark_data_smithi060_196073_object14677:head# 2024-09-20T22:20:25.897 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfc5050a:::benchmark_data_smithi060_196073_object5:head# 2024-09-20T22:20:25.897 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfc53d38:::benchmark_data_smithi060_196073_object49288:head# 2024-09-20T22:20:25.899 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfc5f746:::benchmark_data_smithi060_196073_object36854:head# 2024-09-20T22:20:25.899 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfc797de:::benchmark_data_smithi060_196073_object22222:head# 2024-09-20T22:20:25.900 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfc7bbd5:::benchmark_data_smithi060_196073_object58046:head# 2024-09-20T22:20:25.901 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfc942a2:::benchmark_data_smithi060_196073_object19198:head# 2024-09-20T22:20:25.903 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfc972dc:::benchmark_data_smithi060_196073_object55213:head# 2024-09-20T22:20:25.904 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfc9b3de:::benchmark_data_smithi060_196073_object28955:head# 2024-09-20T22:20:25.905 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfcaad17:::benchmark_data_smithi060_196073_object11694:head# 2024-09-20T22:20:25.906 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfcae164:::benchmark_data_smithi060_196073_object35450:head# 2024-09-20T22:20:25.906 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfcb13a4:::benchmark_data_smithi060_196073_object53433:head# 2024-09-20T22:20:25.907 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfce7840:::benchmark_data_smithi060_196073_object9458:head# 2024-09-20T22:20:25.910 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfcf2adc:::benchmark_data_smithi060_196073_object57016:head# 2024-09-20T22:20:25.911 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:25.912 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfcffeab:::benchmark_data_smithi060_196073_object27522:head# 2024-09-20T22:20:25.913 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfd01b1c:::benchmark_data_smithi060_196073_object60588:head# 2024-09-20T22:20:25.913 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfd2505a:::benchmark_data_smithi060_196073_object28054:head# 2024-09-20T22:20:25.915 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfd293fb:::benchmark_data_smithi060_196073_object32468:head# 2024-09-20T22:20:25.917 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfd44f6a:::benchmark_data_smithi060_196073_object11400:head# 2024-09-20T22:20:25.917 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfd574d6:::benchmark_data_smithi060_196073_object21333:head# 2024-09-20T22:20:25.920 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfd60573:::benchmark_data_smithi060_196073_object17965:head# 2024-09-20T22:20:25.921 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfd6c2cd:::benchmark_data_smithi060_196073_object1044:head# 2024-09-20T22:20:25.922 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfd788a0:::benchmark_data_smithi060_196073_object40052:head# 2024-09-20T22:20:25.923 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfd7bab6:::benchmark_data_smithi060_196073_object44161:head# 2024-09-20T22:20:25.923 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfd808e8:::benchmark_data_smithi060_196073_object14972:head# 2024-09-20T22:20:25.926 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfd81573:::benchmark_data_smithi060_196073_object28098:head# 2024-09-20T22:20:25.927 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfd88be0:::benchmark_data_smithi060_196073_object36813:head# 2024-09-20T22:20:25.928 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfd9464c:::benchmark_data_smithi060_196073_object23949:head# 2024-09-20T22:20:25.931 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfd9f1d1:::benchmark_data_smithi060_196073_object43827:head# 2024-09-20T22:20:25.933 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfda6590:::benchmark_data_smithi060_196073_object43024:head# 2024-09-20T22:20:25.933 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfdacee9:::benchmark_data_smithi060_196073_object53193:head# 2024-09-20T22:20:25.934 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfdd98fe:::benchmark_data_smithi060_196073_object23952:head# 2024-09-20T22:20:25.935 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfdeebac:::benchmark_data_smithi060_196073_object16170:head# 2024-09-20T22:20:25.936 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfe12d8b:::benchmark_data_smithi060_196073_object19008:head# 2024-09-20T22:20:25.936 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfe16d65:::benchmark_data_smithi060_196073_object44870:head# 2024-09-20T22:20:25.937 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfe16f5b:::benchmark_data_smithi060_196073_object40055:head# 2024-09-20T22:20:25.938 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfe1788f:::benchmark_data_smithi060_196073_object23011:head# 2024-09-20T22:20:25.938 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfe336da:::benchmark_data_smithi060_196073_object37967:head# 2024-09-20T22:20:25.941 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfe3a11d:::benchmark_data_smithi060_196073_object2332:head# 2024-09-20T22:20:25.941 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfe4d6ca:::benchmark_data_smithi060_196073_object60435:head# 2024-09-20T22:20:25.941 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfe71b44:::benchmark_data_smithi060_196073_object17763:head# 2024-09-20T22:20:25.943 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfe7a66e:::benchmark_data_smithi060_196073_object31785:head# 2024-09-20T22:20:25.943 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfe7e69d:::benchmark_data_smithi060_196073_object147:head# 2024-09-20T22:20:25.944 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfe99375:::benchmark_data_smithi060_196073_object5300:head# 2024-09-20T22:20:25.944 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfea4e66:::benchmark_data_smithi060_196073_object17567:head# 2024-09-20T22:20:25.944 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfea5d59:::benchmark_data_smithi060_196073_object33411:head# 2024-09-20T22:20:25.945 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfeab294:::benchmark_data_smithi060_196073_object4549:head# 2024-09-20T22:20:25.946 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfeb0b38:::benchmark_data_smithi060_196073_object28080:head# 2024-09-20T22:20:25.946 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfebda12:::benchmark_data_smithi060_196073_object61060:head# 2024-09-20T22:20:25.947 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfedc629:::benchmark_data_smithi060_196073_object23969:head# 2024-09-20T22:20:25.947 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:25.948 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bfee1d25:::benchmark_data_smithi060_196073_object39079:head# 2024-09-20T22:20:25.948 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bff01c03:::benchmark_data_smithi060_196073_object43426:head# 2024-09-20T22:20:25.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:25.950 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:25.951 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bff071e2:::benchmark_data_smithi060_196073_object32909:head# 2024-09-20T22:20:25.951 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bff08449:::benchmark_data_smithi060_196073_object45408:head# 2024-09-20T22:20:25.951 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bff0a283:::benchmark_data_smithi060_196073_object15180:head# 2024-09-20T22:20:25.952 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bff30711:::benchmark_data_smithi060_196073_object51487:head# 2024-09-20T22:20:25.952 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bff5ee78:::benchmark_data_smithi060_196073_object53552:head# 2024-09-20T22:20:25.952 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bff96581:::benchmark_data_smithi060_196073_object34767:head# 2024-09-20T22:20:25.953 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bffa27f1:::benchmark_data_smithi060_196073_object57897:head# 2024-09-20T22:20:25.954 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bffa5258:::benchmark_data_smithi060_196073_object25931:head# 2024-09-20T22:20:25.954 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bffbc242:::benchmark_data_smithi060_196073_object9958:head# 2024-09-20T22:20:25.955 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bffc5a40:::benchmark_data_smithi060_196073_object18821:head# 2024-09-20T22:20:25.956 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bffc9b79:::benchmark_data_smithi060_196073_object24617:head# 2024-09-20T22:20:25.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bffd98bf:::benchmark_data_smithi060_196073_object22775:head# 2024-09-20T22:20:25.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bffef3c1:::benchmark_data_smithi060_196073_object50464:head# 2024-09-20T22:20:25.964 INFO:teuthology.orchestra.run.smithi067.stderr:Read #11:bffff9cb:::benchmark_data_smithi060_196073_object53164:head# 2024-09-20T22:20:25.964 INFO:teuthology.orchestra.run.smithi067.stderr:Export successful 2024-09-20T22:20:25.987 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:25 smithi060 ceph-mon[40290]: pgmap v935: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 170 active+clean; 6.2 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1023 B/s rd, 51 MiB/s wr, 830 op/s; 9121/303336 objects degraded (3.007%); 22581/303336 objects misplaced (7.444%); 23 MiB/s, 366 objects/s recovering 2024-09-20T22:20:25.987 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:25 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:20:25.987 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:25 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2030588030' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:20:25.987 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:25 smithi060 ceph-mon[40290]: osdmap e494: 8 total, 7 up, 6 in 2024-09-20T22:20:25.987 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:25 smithi060 ceph-mon[29694]: pgmap v935: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 170 active+clean; 6.2 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1023 B/s rd, 51 MiB/s wr, 830 op/s; 9121/303336 objects degraded (3.007%); 22581/303336 objects misplaced (7.444%); 23 MiB/s, 366 objects/s recovering 2024-09-20T22:20:25.987 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:25 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:20:25.987 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:25 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2030588030' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:20:25.987 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:25 smithi060 ceph-mon[29694]: osdmap e494: 8 total, 7 up, 6 in 2024-09-20T22:20:26.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T22:20:26.050 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:26.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:26.145 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:26.145 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:26.191 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:26.247 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:26.304 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:26.304 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:26.305 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:26.406 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:26.438 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:26.439 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:26.439 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:26.540 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:26.572 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:26.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:26.573 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:26.673 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:26.709 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:26.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:26.711 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:26.811 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:26.854 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:26.854 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:26.854 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:26.901 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:26.911 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:20:26.911 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:20:26.956 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:26.962 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:20:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:20:27.001 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:20:27.001 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:20:27.013 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:27.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:27.014 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:27.038 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:20:27.039 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:20:27.040 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-20T22:20:27.076 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:20:27.077 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:20:27.112 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:20:27.112 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:20:27.115 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:27.147 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:20:27.147 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:20:27.151 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:27.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:27.152 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:27.216 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -n osd.4 --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal --force --op remove --pgid 11.d 2024-09-20T22:20:27.252 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:27.284 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:27.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:27.285 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:27.385 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:27.422 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:27.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:27.423 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:27.487 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/osd.4/config 2024-09-20T22:20:27.523 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:27.561 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:27.562 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:27.562 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:27.574 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:20:27.663 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:27.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:27.713 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:27.713 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:27.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:27.860 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:27.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:27.861 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:27.942 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:27 smithi067 ceph-mon[33511]: pgmap v937: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 170 active+clean; 6.1 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1023 B/s rd, 46 MiB/s wr, 837 op/s; 8964/302271 objects degraded (2.966%); 21931/302271 objects misplaced (7.255%); 23 MiB/s, 369 objects/s recovering 2024-09-20T22:20:27.942 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:27 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 8964/302271 objects degraded (2.966%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:27.942 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:27 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2228176365' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:20:27.961 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:28.010 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:28.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:28.011 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:28.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:27 smithi060 ceph-mon[29694]: pgmap v937: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 170 active+clean; 6.1 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1023 B/s rd, 46 MiB/s wr, 837 op/s; 8964/302271 objects degraded (2.966%); 21931/302271 objects misplaced (7.255%); 23 MiB/s, 369 objects/s recovering 2024-09-20T22:20:28.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:27 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 8964/302271 objects degraded (2.966%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:28.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:27 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2228176365' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:20:28.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:27 smithi060 ceph-mon[40290]: pgmap v937: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 1 active+remapped+backfill_wait, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 170 active+clean; 6.1 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1023 B/s rd, 46 MiB/s wr, 837 op/s; 8964/302271 objects degraded (2.966%); 21931/302271 objects misplaced (7.255%); 23 MiB/s, 369 objects/s recovering 2024-09-20T22:20:28.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:27 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 8964/302271 objects degraded (2.966%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:28.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:27 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2228176365' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:20:28.112 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:28.146 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:28.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:28.147 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:28.248 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:28.288 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:28.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:28.289 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:28.389 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:28.436 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:28.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:28.437 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:28.538 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:28.575 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:28.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:28.576 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:28.676 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:28.709 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:28.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:28.710 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:28.811 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:28.840 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:28.841 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:28.842 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:28.854 INFO:teuthology.orchestra.run.smithi067.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-20T22:20:28.942 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:28.972 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:28.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:28.973 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:29.074 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:29.103 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:29.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:29.104 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:29.204 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:29.229 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:29.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:29.231 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:29.331 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:29.360 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:29.360 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:29.361 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:29.461 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:29.487 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:29.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:29.488 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:29.588 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:29.614 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:29.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:29.615 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:29.716 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:29.756 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:29.757 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:29.758 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:29.858 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:29.885 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:29.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:29.886 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:29.887 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:29 smithi067 ceph-mon[33511]: pgmap v938: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 170 active+clean; 6.0 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1.1 KiB/s rd, 31 MiB/s wr, 869 op/s; 8375/295284 objects degraded (2.836%); 20214/295284 objects misplaced (6.846%); 25 MiB/s, 405 objects/s recovering 2024-09-20T22:20:29.888 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:29 smithi067 ceph-mon[33511]: osdmap e495: 8 total, 7 up, 6 in 2024-09-20T22:20:29.888 INFO:teuthology.orchestra.run.smithi067.stdout: marking collection for removal 2024-09-20T22:20:29.888 INFO:teuthology.orchestra.run.smithi067.stdout:setting '_remove' omap key 2024-09-20T22:20:29.888 INFO:teuthology.orchestra.run.smithi067.stdout:finish_remove_pgs 11.d_head removing 11.d 2024-09-20T22:20:29.988 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:30.000 INFO:teuthology.orchestra.run.smithi067.stdout:Remove successful 2024-09-20T22:20:30.026 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:30.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:30.027 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:30.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:29 smithi060 ceph-mon[29694]: pgmap v938: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 170 active+clean; 6.0 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1.1 KiB/s rd, 31 MiB/s wr, 869 op/s; 8375/295284 objects degraded (2.836%); 20214/295284 objects misplaced (6.846%); 25 MiB/s, 405 objects/s recovering 2024-09-20T22:20:30.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:29 smithi060 ceph-mon[29694]: osdmap e495: 8 total, 7 up, 6 in 2024-09-20T22:20:30.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:29 smithi060 ceph-mon[40290]: pgmap v938: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 170 active+clean; 6.0 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1.1 KiB/s rd, 31 MiB/s wr, 869 op/s; 8375/295284 objects degraded (2.836%); 20214/295284 objects misplaced (6.846%); 25 MiB/s, 405 objects/s recovering 2024-09-20T22:20:30.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:29 smithi060 ceph-mon[40290]: osdmap e495: 8 total, 7 up, 6 in 2024-09-20T22:20:30.128 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:30.165 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:30.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:30.166 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:30.267 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:30.300 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:30.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:30.301 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:30.401 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:30.447 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:30.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:30.448 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:30.549 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:30.587 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:30.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:30.588 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:30.689 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:30.725 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:30.725 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:30.725 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:30 smithi067 ceph-mon[33511]: osdmap e496: 8 total, 7 up, 6 in 2024-09-20T22:20:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:30 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:20:30.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:30 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:20:30.826 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:30.859 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:30.860 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:30.860 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:30.961 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:30.998 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:30.998 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:30.999 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:31.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:30 smithi060 ceph-mon[29694]: osdmap e496: 8 total, 7 up, 6 in 2024-09-20T22:20:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:30 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:20:31.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:30 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:20:31.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:30 smithi060 ceph-mon[40290]: osdmap e496: 8 total, 7 up, 6 in 2024-09-20T22:20:31.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:30 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:20:31.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:30 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:20:31.099 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:31.136 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:31.137 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:31.137 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:31.222 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:20:31.222 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:20:31.239 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:31.271 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:31.272 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:31.272 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:31.373 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:31.418 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:31.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:31.419 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:31.519 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:31.541 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:31.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:31.542 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:31.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:31.687 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:31.688 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:31.688 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:31.789 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:31.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:31.828 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:31.828 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:31.929 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:31.951 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:31.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:31.952 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:31 smithi067 ceph-mon[33511]: pgmap v941: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 170 active+clean; 6.0 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1023 B/s rd, 11 MiB/s wr, 786 op/s; 8375/295284 objects degraded (2.836%); 20214/295284 objects misplaced (6.846%); 21 MiB/s, 337 objects/s recovering 2024-09-20T22:20:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:31 smithi067 ceph-mon[33511]: osdmap e497: 8 total, 7 up, 6 in 2024-09-20T22:20:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:31 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:20:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:31 smithi060 ceph-mon[29694]: pgmap v941: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 170 active+clean; 6.0 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1023 B/s rd, 11 MiB/s wr, 786 op/s; 8375/295284 objects degraded (2.836%); 20214/295284 objects misplaced (6.846%); 21 MiB/s, 337 objects/s recovering 2024-09-20T22:20:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:31 smithi060 ceph-mon[29694]: osdmap e497: 8 total, 7 up, 6 in 2024-09-20T22:20:32.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:31 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:20:32.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:31 smithi060 ceph-mon[40290]: pgmap v941: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 170 active+clean; 6.0 GiB data, 20 GiB used, 516 GiB / 536 GiB avail; 1023 B/s rd, 11 MiB/s wr, 786 op/s; 8375/295284 objects degraded (2.836%); 20214/295284 objects misplaced (6.846%); 21 MiB/s, 337 objects/s recovering 2024-09-20T22:20:32.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:31 smithi060 ceph-mon[40290]: osdmap e497: 8 total, 7 up, 6 in 2024-09-20T22:20:32.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:31 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:20:32.052 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:32.075 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:32.076 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:32.076 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:32.177 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:32.212 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:32.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:32.213 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:32.313 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:32.347 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:32.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:32.348 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:32.449 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:32.480 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:32.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:32.481 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:32.582 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:32.621 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:32.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:32.622 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:32.723 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:32.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:32.759 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:32.759 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:32.860 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:32.904 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:32 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 8375/295284 objects degraded (2.836%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:32.907 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:32 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 8375/295284 objects degraded (2.836%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:32.911 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:20:32.911 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:20:32.951 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:32.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:32.952 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:33.014 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:20:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:20:33.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:32 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 8375/295284 objects degraded (2.836%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:33.052 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:33.093 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:33.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:33.094 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:33.195 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:33.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:33.218 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:33.218 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:33.319 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:33.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:33.355 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:33.356 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:33.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:33.496 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:33.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:33.497 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:33.597 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:33.640 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:33.641 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:33.641 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:33.742 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:33.777 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:33.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:33.778 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:33.879 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:33.902 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:33.902 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:33.903 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:34.003 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:34.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:33 smithi067 ceph-mon[33511]: pgmap v943: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 170 active+clean; 5.8 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 1023 B/s rd, 5.0 MiB/s wr, 982 op/s; 7221/287475 objects degraded (2.512%); 18141/287475 objects misplaced (6.310%); 35 MiB/s, 558 objects/s recovering 2024-09-20T22:20:34.023 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:33 smithi060 ceph-mon[29694]: pgmap v943: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 170 active+clean; 5.8 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 1023 B/s rd, 5.0 MiB/s wr, 982 op/s; 7221/287475 objects degraded (2.512%); 18141/287475 objects misplaced (6.310%); 35 MiB/s, 558 objects/s recovering 2024-09-20T22:20:34.023 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:33 smithi060 ceph-mon[40290]: pgmap v943: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 170 active+clean; 5.8 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 1023 B/s rd, 5.0 MiB/s wr, 982 op/s; 7221/287475 objects degraded (2.512%); 18141/287475 objects misplaced (6.310%); 35 MiB/s, 558 objects/s recovering 2024-09-20T22:20:34.045 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:34.046 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:34.046 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:34.076 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T22:20:34.109 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:34.147 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:34.174 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:34.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:34.175 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:34.275 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:34.320 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:34.320 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:34.321 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:34.421 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:34.479 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:34.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:34.481 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:34.581 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:34.627 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:34.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:34.628 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:34.729 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:34.761 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:34.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:34.762 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:34.803 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:34.863 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:34.891 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:34.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:34.892 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:34.912 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:20:34.951 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:34 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:20:34.952 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:34 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:20:34.993 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:35.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:34 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:20:35.028 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:35.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:35.029 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:35.130 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:35.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:35.163 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:35.163 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:35.264 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:35.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:20:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:20:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:20:35.302 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:35.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:35.303 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:35.403 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:35.425 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:35.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:35.426 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:35.527 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:35.565 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:35.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:35.566 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:35.664 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:20:35.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:35.700 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:35.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:35.701 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:35.802 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:35.846 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:35.847 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:35.847 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:35.917 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:35 smithi060 ceph-mon[29694]: pgmap v944: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 171 active+clean; 5.8 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 623 op/s; 7221/284070 objects degraded (2.542%); 16577/284070 objects misplaced (5.836%); 35 MiB/s, 556 objects/s recovering 2024-09-20T22:20:35.917 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:35 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3673024122' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:20:35.917 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:35 smithi060 ceph-mon[40290]: pgmap v944: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 171 active+clean; 5.8 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 623 op/s; 7221/284070 objects degraded (2.542%); 16577/284070 objects misplaced (5.836%); 35 MiB/s, 556 objects/s recovering 2024-09-20T22:20:35.917 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:35 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3673024122' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:20:35.948 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:35.982 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:35.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:35.983 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:35 smithi067 ceph-mon[33511]: pgmap v944: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 171 active+clean; 5.8 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 623 op/s; 7221/284070 objects degraded (2.542%); 16577/284070 objects misplaced (5.836%); 35 MiB/s, 556 objects/s recovering 2024-09-20T22:20:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:35 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3673024122' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:20:36.084 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:36.121 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:36.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:36.122 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:36.223 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:36.260 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:36.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:36.261 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:36.361 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:36.399 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:36.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:36.400 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:36.501 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:36.535 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:20:36 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:20:36.268+0000 7fa5382c7640 -1 mgr get_metadata_python Requested missing service osd.4 2024-09-20T22:20:36.538 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:36.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:36.539 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:36.639 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:36.677 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:36.677 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:36.678 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:36.778 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:36.811 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:36.811 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:36.812 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:36.912 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:36.950 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:36.951 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:36.951 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:36 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:20:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:36 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3673024122' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:20:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:36 smithi067 ceph-mon[33511]: osdmap e498: 8 total, 7 up, 6 in 2024-09-20T22:20:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:36 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:20:37.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:36 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:20:37.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:36 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3673024122' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:20:37.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:36 smithi060 ceph-mon[29694]: osdmap e498: 8 total, 7 up, 6 in 2024-09-20T22:20:37.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:36 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:20:37.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:36 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:20:37.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:36 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3673024122' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:20:37.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:36 smithi060 ceph-mon[40290]: osdmap e498: 8 total, 7 up, 6 in 2024-09-20T22:20:37.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:36 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:20:37.052 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:37.085 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:37.086 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:37.086 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:37.187 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:37.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:37.234 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:37.234 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:37.335 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:37.368 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:37.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:37.369 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:37.470 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:37.505 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:37.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:37.506 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:37.606 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:37.642 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:37.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:37.643 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:37.744 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:37.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:37 smithi060 ceph-mon[29694]: pgmap v946: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 171 active+clean; 5.7 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 761 op/s; 7039/279894 objects degraded (2.515%); 15910/279894 objects misplaced (5.684%); 37 MiB/s, 589 objects/s recovering 2024-09-20T22:20:37.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:37 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 7221/284070 objects degraded (2.542%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:37.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:37 smithi060 ceph-mon[40290]: pgmap v946: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 171 active+clean; 5.7 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 761 op/s; 7039/279894 objects degraded (2.515%); 15910/279894 objects misplaced (5.684%); 37 MiB/s, 589 objects/s recovering 2024-09-20T22:20:37.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:37 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 7221/284070 objects degraded (2.542%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:37.800 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:20:37.800 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:20:37.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:37.801 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:37.801 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:37.902 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:37.952 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:37.953 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:37.953 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:37 smithi067 ceph-mon[33511]: pgmap v946: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 171 active+clean; 5.7 GiB data, 20 GiB used, 517 GiB / 536 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 761 op/s; 7039/279894 objects degraded (2.515%); 15910/279894 objects misplaced (5.684%); 37 MiB/s, 589 objects/s recovering 2024-09-20T22:20:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:37 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 7221/284070 objects degraded (2.542%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:38.023 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:38.054 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:38.088 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:38.089 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:38.089 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:38.189 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:38.229 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:38.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:38.230 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:38.331 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:38.373 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:38.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:38.374 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:38.474 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:38.512 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:38.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:38.513 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:38.614 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:38.670 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T22:20:38.671 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:38.672 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:38.672 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:38.773 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:38.799 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:38.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:38.799 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:38.900 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:38.935 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:38.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:38.936 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:38.986 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:38 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:20:38.986 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:38 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:20:38.986 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:38 smithi060 ceph-mon[40290]: pgmap v947: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 171 active+clean; 5.5 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.05k op/s; 6380/270069 objects degraded (2.362%); 13873/270069 objects misplaced (5.137%); 44 MiB/s, 705 objects/s recovering 2024-09-20T22:20:38.986 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:38 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:20:38.987 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:38 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:20:38.987 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:38 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:20:38.987 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:38 smithi060 ceph-mon[29694]: pgmap v947: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 171 active+clean; 5.5 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.05k op/s; 6380/270069 objects degraded (2.362%); 13873/270069 objects misplaced (5.137%); 44 MiB/s, 705 objects/s recovering 2024-09-20T22:20:38.987 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:38 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:20:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:38 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:20:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:38 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:20:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:38 smithi067 ceph-mon[33511]: pgmap v947: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 171 active+clean; 5.5 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.05k op/s; 6380/270069 objects degraded (2.362%); 13873/270069 objects misplaced (5.137%); 44 MiB/s, 705 objects/s recovering 2024-09-20T22:20:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:38 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:20:39.038 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:39.083 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:39.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:39.084 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:39.185 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:39.222 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:39.223 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:39.223 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:39.323 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:39.327 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell -n osd.4 --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-4 --journal-path /var/lib/ceph/osd/ceph-4/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.11.d.4 2024-09-20T22:20:39.359 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:39.359 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:39.359 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:39.360 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:39.382 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:20:39.383 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:20:39.421 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:20:39.422 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:20:39.453 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:20:39.453 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:20:39.460 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:39.486 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:20:39.486 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:20:39.487 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-20T22:20:39.505 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:39.506 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:39.506 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:39.523 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:20:39.524 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:20:39.547 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:20:39.551 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:20:39.551 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:20:39.573 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:20:39.573 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:20:39.607 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:39.657 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/osd.4/config 2024-09-20T22:20:39.658 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:39.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:39.661 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:39.761 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:39.804 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:39.804 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:39.804 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:39.905 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:39.934 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:39.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:39.935 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:39 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/380021827' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:20:40.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:39 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/380021827' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:20:40.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:39 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/380021827' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:20:40.036 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:40.062 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:40.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:40.063 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:40.164 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:40.189 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:40.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:40.190 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:40.291 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:40.315 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:40.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:40.316 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:40.417 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:40.443 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:40.444 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:40.444 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:40.545 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:40.570 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:40.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:40.570 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:40.671 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:40.702 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:40.703 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:40.703 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:40.803 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:40.835 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:40.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:40.836 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:40.937 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:40.940 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:40 smithi060 ceph-mon[29694]: osdmap e499: 8 total, 7 up, 6 in 2024-09-20T22:20:40.940 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:40 smithi060 ceph-mon[29694]: pgmap v949: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 171 active+clean; 5.5 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 725 op/s; 6380/270069 objects degraded (2.362%); 13873/270069 objects misplaced (5.137%); 29 MiB/s, 460 objects/s recovering 2024-09-20T22:20:40.940 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:40 smithi060 ceph-mon[40290]: osdmap e499: 8 total, 7 up, 6 in 2024-09-20T22:20:40.941 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:40 smithi060 ceph-mon[40290]: pgmap v949: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 171 active+clean; 5.5 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 725 op/s; 6380/270069 objects degraded (2.362%); 13873/270069 objects misplaced (5.137%); 29 MiB/s, 460 objects/s recovering 2024-09-20T22:20:40.984 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:40.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:40.985 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:40 smithi067 ceph-mon[33511]: osdmap e499: 8 total, 7 up, 6 in 2024-09-20T22:20:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:40 smithi067 ceph-mon[33511]: pgmap v949: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 2 active+remapped+backfilling, 171 active+clean; 5.5 GiB data, 19 GiB used, 517 GiB / 536 GiB avail; 639 B/s rd, 0 B/s wr, 725 op/s; 6380/270069 objects degraded (2.362%); 13873/270069 objects misplaced (5.137%); 29 MiB/s, 460 objects/s recovering 2024-09-20T22:20:41.086 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:41.123 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:41.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:41.125 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:41.225 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:41.259 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:41.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:41.260 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:41.360 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:41.404 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:41.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:41.405 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:41.506 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:41.537 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:41.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:41.538 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:41.639 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:41.697 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:20:41.698 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:20:41.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:41.714 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:41.714 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:41.798 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:41 smithi060 ceph-mon[29694]: osdmap e500: 8 total, 7 up, 6 in 2024-09-20T22:20:41.798 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:41 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:20:41.798 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:41 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:20:41.798 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:41 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 6380/270069 objects degraded (2.362%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:41.798 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:41 smithi060 ceph-mon[40290]: osdmap e500: 8 total, 7 up, 6 in 2024-09-20T22:20:41.798 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:41 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:20:41.798 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:41 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:20:41.798 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:41 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 6380/270069 objects degraded (2.362%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:41.815 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:41.845 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:41.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:41.846 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:41.947 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:41.965 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:20:41.965 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:20:41.965 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:20:41.965 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:20:41.965 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:20:41.978 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:41.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:41.979 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:41 smithi067 ceph-mon[33511]: osdmap e500: 8 total, 7 up, 6 in 2024-09-20T22:20:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:41 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:20:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:41 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:20:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:41 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 6380/270069 objects degraded (2.362%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:42.080 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:42.120 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:42.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:42.121 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:42.222 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:42.258 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:42.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:42.259 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:42.360 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:42.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:42.396 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:42.397 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:42.497 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:42.540 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:42.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:42.541 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:42.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:42.677 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:42.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:42.678 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:42.779 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:42.815 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:42.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:42.816 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:42.849 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:42.897 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T22:20:42.917 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:42.955 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:42.955 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:42.956 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:42.984 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:42 smithi060 ceph-mon[29694]: osdmap e501: 8 total, 7 up, 6 in 2024-09-20T22:20:42.984 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:42 smithi060 ceph-mon[29694]: pgmap v952: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfilling, 172 active+clean; 5.3 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.02k op/s; 5014/261555 objects degraded (1.917%); 12079/261555 objects misplaced (4.618%); 35 MiB/s, 564 objects/s recovering 2024-09-20T22:20:42.984 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:42 smithi060 ceph-mon[40290]: osdmap e501: 8 total, 7 up, 6 in 2024-09-20T22:20:42.984 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:42 smithi060 ceph-mon[40290]: pgmap v952: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfilling, 172 active+clean; 5.3 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.02k op/s; 5014/261555 objects degraded (1.917%); 12079/261555 objects misplaced (4.618%); 35 MiB/s, 564 objects/s recovering 2024-09-20T22:20:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:42 smithi067 ceph-mon[33511]: osdmap e501: 8 total, 7 up, 6 in 2024-09-20T22:20:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:42 smithi067 ceph-mon[33511]: pgmap v952: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfilling, 172 active+clean; 5.3 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.02k op/s; 5014/261555 objects degraded (1.917%); 12079/261555 objects misplaced (4.618%); 35 MiB/s, 564 objects/s recovering 2024-09-20T22:20:43.014 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:20:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:20:43.056 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:43.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:43.105 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:43.106 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:43.206 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:43.236 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:43.237 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:43.237 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:43.337 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:43.374 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:43.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:43.375 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:43.476 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:43.521 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:43.521 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:43.521 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:43.622 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:43.640 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:43.657 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:43.658 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:43.658 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:43.699 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:20:43.759 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:43.791 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:43.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:43.792 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:43.893 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:43.930 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:43.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:43.931 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:43.990 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:43 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:20:43.990 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:43 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:20:43.995 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:43 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:20:44.032 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:44.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:44.066 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:44.067 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:44.167 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:44.200 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:44.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:44.202 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:44.300 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:20:44.300 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:20:44.300 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:20:44.300 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:20:44.300 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.301 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214320 8.8 8:125eda0b:::benchmark_data_smithi060_111051_object94321:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:20:44.301 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:20:44.301 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 586.79496484900005, 2024-09-20T22:20:44.301 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0054397049999999, 2024-09-20T22:20:44.301 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.301 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.301 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.301 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.301 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:20:44.301 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:20:44.301 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214320 2024-09-20T22:20:44.301 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.302 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:20:44.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:20:44.302 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.302 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.302 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.302 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.302 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496382+0000", 2024-09-20T22:20:44.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0999999999999999e-07 2024-09-20T22:20:44.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496384+0000", 2024-09-20T22:20:44.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1859999999999999e-06 2024-09-20T22:20:44.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.303 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496390+0000", 2024-09-20T22:20:44.303 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9920000000000004e-06 2024-09-20T22:20:44.303 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.303 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.303 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.304 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496395+0000", 2024-09-20T22:20:44.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.741e-06 2024-09-20T22:20:44.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.304 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.304 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496429+0000", 2024-09-20T22:20:44.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3736999999999999e-05 2024-09-20T22:20:44.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.304 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:20:44.304 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.496436+0000", 2024-09-20T22:20:44.304 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2529999999999998e-06 2024-09-20T22:20:44.304 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.304 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.304 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.497957+0000", 2024-09-20T22:20:44.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.001521109 2024-09-20T22:20:44.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.498097+0000", 2024-09-20T22:20:44.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000139562 2024-09-20T22:20:44.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:20:44.305 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.498422+0000", 2024-09-20T22:20:44.305 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00032548199999999998 2024-09-20T22:20:44.305 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.305 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.305 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501661+0000", 2024-09-20T22:20:44.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0032386580000000002 2024-09-20T22:20:44.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501669+0000", 2024-09-20T22:20:44.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2500000000000006e-06 2024-09-20T22:20:44.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.306 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.306 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501709+0000", 2024-09-20T22:20:44.306 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9304999999999997e-05 2024-09-20T22:20:44.306 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.306 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501777+0000", 2024-09-20T22:20:44.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8285000000000001e-05 2024-09-20T22:20:44.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.307 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.307 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501821+0000", 2024-09-20T22:20:44.307 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4635000000000002e-05 2024-09-20T22:20:44.307 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.307 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.307 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.307 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.307 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.307 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214327 8.8 8:149a99e2:::benchmark_data_smithi060_111051_object94328:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:20:44.307 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.502392+0000", 2024-09-20T22:20:44.308 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 586.78895442700002, 2024-09-20T22:20:44.308 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.001539808, 2024-09-20T22:20:44.308 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.308 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.308 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.308 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.308 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:20:44.308 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:20:44.308 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214327 2024-09-20T22:20:44.308 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.308 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.308 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.308 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.308 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502392+0000", 2024-09-20T22:20:44.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502392+0000", 2024-09-20T22:20:44.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502393+0000", 2024-09-20T22:20:44.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2440000000000001e-06 2024-09-20T22:20:44.309 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.309 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.309 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.309 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502398+0000", 2024-09-20T22:20:44.309 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1749999999999998e-06 2024-09-20T22:20:44.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502406+0000", 2024-09-20T22:20:44.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4999999999999999e-06 2024-09-20T22:20:44.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502411+0000", 2024-09-20T22:20:44.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7119999999999999e-06 2024-09-20T22:20:44.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.310 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.310 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.310 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502465+0000", 2024-09-20T22:20:44.310 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4206000000000002e-05 2024-09-20T22:20:44.310 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:20:44.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.502470+0000", 2024-09-20T22:20:44.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8069999999999999e-06 2024-09-20T22:20:44.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.500319+0000", 2024-09-20T22:20:44.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99784936499999999 2024-09-20T22:20:44.311 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.311 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.311 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.311 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.500435+0000", 2024-09-20T22:20:44.311 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000115439 2024-09-20T22:20:44.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:20:44.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.500670+0000", 2024-09-20T22:20:44.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00023555700000000001 2024-09-20T22:20:44.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503764+0000", 2024-09-20T22:20:44.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0030938839999999999 2024-09-20T22:20:44.312 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.312 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.312 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.312 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503827+0000", 2024-09-20T22:20:44.312 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2669999999999995e-05 2024-09-20T22:20:44.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503839+0000", 2024-09-20T22:20:44.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2301e-05 2024-09-20T22:20:44.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.313 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.313 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.313 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503887+0000", 2024-09-20T22:20:44.313 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8319999999999998e-05 2024-09-20T22:20:44.313 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.314 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:44.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.315 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.315 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503932+0000", 2024-09-20T22:20:44.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4628e-05 2024-09-20T22:20:44.315 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.315 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.315 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.315 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.315 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.315 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214339 8.8 8:166843c0:::benchmark_data_smithi060_111051_object94340:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:20:44.315 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.507919+0000", 2024-09-20T22:20:44.315 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 586.78342733099998, 2024-09-20T22:20:44.315 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99621294400000004, 2024-09-20T22:20:44.316 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.316 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.316 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.316 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.316 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:20:44.316 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:20:44.316 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214339 2024-09-20T22:20:44.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.316 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.316 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.316 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507919+0000", 2024-09-20T22:20:44.316 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.316 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.316 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507919+0000", 2024-09-20T22:20:44.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507921+0000", 2024-09-20T22:20:44.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.316e-06 2024-09-20T22:20:44.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.317 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.317 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507926+0000", 2024-09-20T22:20:44.317 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0350000000000001e-06 2024-09-20T22:20:44.317 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.317 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507937+0000", 2024-09-20T22:20:44.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1379999999999999e-05 2024-09-20T22:20:44.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.507945+0000", 2024-09-20T22:20:44.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5780000000000001e-06 2024-09-20T22:20:44.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.318 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.318 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.508028+0000", 2024-09-20T22:20:44.318 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.3641999999999996e-05 2024-09-20T22:20:44.318 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.318 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:20:44.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.508036+0000", 2024-09-20T22:20:44.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1780000000000003e-06 2024-09-20T22:20:44.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501861+0000", 2024-09-20T22:20:44.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.99382438200000001 2024-09-20T22:20:44.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.319 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.319 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.319 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.501935+0000", 2024-09-20T22:20:44.319 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4151000000000002e-05 2024-09-20T22:20:44.319 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:20:44.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.502143+0000", 2024-09-20T22:20:44.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020823700000000001 2024-09-20T22:20:44.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.503984+0000", 2024-09-20T22:20:44.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001840602 2024-09-20T22:20:44.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.320 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.320 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504043+0000", 2024-09-20T22:20:44.320 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9061000000000003e-05 2024-09-20T22:20:44.320 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.320 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504055+0000", 2024-09-20T22:20:44.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1826e-05 2024-09-20T22:20:44.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504099+0000", 2024-09-20T22:20:44.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4801e-05 2024-09-20T22:20:44.321 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.321 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.321 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.321 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504132+0000", 2024-09-20T22:20:44.321 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2755000000000001e-05 2024-09-20T22:20:44.321 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.321 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.322 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.322 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.322 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214363 8.8 8:14aba4d5:::benchmark_data_smithi060_111051_object94364:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:20:44.322 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.530169+0000", 2024-09-20T22:20:44.322 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 586.76117739599999, 2024-09-20T22:20:44.322 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98026972499999998, 2024-09-20T22:20:44.322 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.322 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.322 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.322 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.322 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:20:44.322 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:20:44.322 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214363 2024-09-20T22:20:44.322 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.323 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530169+0000", 2024-09-20T22:20:44.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530169+0000", 2024-09-20T22:20:44.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.323 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.323 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.323 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530171+0000", 2024-09-20T22:20:44.323 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4300000000000001e-06 2024-09-20T22:20:44.323 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530177+0000", 2024-09-20T22:20:44.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8499999999999999e-06 2024-09-20T22:20:44.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530188+0000", 2024-09-20T22:20:44.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1669e-05 2024-09-20T22:20:44.324 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.324 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.324 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.324 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530196+0000", 2024-09-20T22:20:44.324 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0760000000000003e-06 2024-09-20T22:20:44.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530270+0000", 2024-09-20T22:20:44.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3709999999999997e-05 2024-09-20T22:20:44.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:20:44.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530278+0000", 2024-09-20T22:20:44.325 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.5320000000000006e-06 2024-09-20T22:20:44.325 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.325 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.325 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.325 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504165+0000", 2024-09-20T22:20:44.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97388663600000003 2024-09-20T22:20:44.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504263+0000", 2024-09-20T22:20:44.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7412999999999997e-05 2024-09-20T22:20:44.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:20:44.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.504540+0000", 2024-09-20T22:20:44.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00027761399999999997 2024-09-20T22:20:44.326 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.326 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.326 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.326 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510219+0000", 2024-09-20T22:20:44.326 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005678476 2024-09-20T22:20:44.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510260+0000", 2024-09-20T22:20:44.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1452000000000002e-05 2024-09-20T22:20:44.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510339+0000", 2024-09-20T22:20:44.327 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8669000000000002e-05 2024-09-20T22:20:44.327 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.327 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.327 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.327 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510393+0000", 2024-09-20T22:20:44.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4160999999999999e-05 2024-09-20T22:20:44.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.328 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.328 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510439+0000", 2024-09-20T22:20:44.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6037e-05 2024-09-20T22:20:44.328 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.328 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.328 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.328 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.328 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.328 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15192.0:214367 8.8 8:15e42233:::benchmark_data_smithi060_111051_object94368:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e268)", 2024-09-20T22:20:44.328 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:10:57.530758+0000", 2024-09-20T22:20:44.328 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 586.76058821200002, 2024-09-20T22:20:44.328 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.98471673000000004, 2024-09-20T22:20:44.328 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.329 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.329 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.329 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.329 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15192", 2024-09-20T22:20:44.329 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/3205427574", 2024-09-20T22:20:44.329 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 214367 2024-09-20T22:20:44.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.329 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.329 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530758+0000", 2024-09-20T22:20:44.329 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.329 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.329 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.329 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530758+0000", 2024-09-20T22:20:44.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530759+0000", 2024-09-20T22:20:44.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1200000000000002e-07 2024-09-20T22:20:44.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.330 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.330 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530764+0000", 2024-09-20T22:20:44.330 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5390000000000001e-06 2024-09-20T22:20:44.330 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.330 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.331 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530773+0000", 2024-09-20T22:20:44.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.986e-06 2024-09-20T22:20:44.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.331 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530780+0000", 2024-09-20T22:20:44.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8689999999999998e-06 2024-09-20T22:20:44.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.331 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.331 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.331 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530827+0000", 2024-09-20T22:20:44.331 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7314000000000001e-05 2024-09-20T22:20:44.331 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for peered", 2024-09-20T22:20:44.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:57.530832+0000", 2024-09-20T22:20:44.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.758e-06 2024-09-20T22:20:44.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.505913+0000", 2024-09-20T22:20:44.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.97508115200000001 2024-09-20T22:20:44.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.332 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.332 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.332 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.506026+0000", 2024-09-20T22:20:44.332 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011298000000000001 2024-09-20T22:20:44.332 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:20:44.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.506308+0000", 2024-09-20T22:20:44.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000282477 2024-09-20T22:20:44.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510274+0000", 2024-09-20T22:20:44.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0039657160000000002 2024-09-20T22:20:44.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.333 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.333 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.333 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.510478+0000", 2024-09-20T22:20:44.333 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000204002 2024-09-20T22:20:44.333 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.515367+0000", 2024-09-20T22:20:44.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0048889099999999998 2024-09-20T22:20:44.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.515428+0000", 2024-09-20T22:20:44.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.0826999999999999e-05 2024-09-20T22:20:44.334 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.334 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.334 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.334 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:10:58.515475+0000", 2024-09-20T22:20:44.334 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7388000000000001e-05 2024-09-20T22:20:44.335 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.335 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.335 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.335 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.335 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.335 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:67549 11.b 11:d97c1fa4:::benchmark_data_smithi060_196073_object67548:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e477)", 2024-09-20T22:20:44.335 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:19:36.964055+0000", 2024-09-20T22:20:44.335 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 67.327291209999999, 2024-09-20T22:20:44.335 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.287332594, 2024-09-20T22:20:44.335 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.335 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.335 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.335 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.335 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:20:44.335 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:20:44.336 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 67549 2024-09-20T22:20:44.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.336 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.964055+0000", 2024-09-20T22:20:44.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.964055+0000", 2024-09-20T22:20:44.336 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.336 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.336 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.336 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.336 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.964057+0000", 2024-09-20T22:20:44.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.482e-06 2024-09-20T22:20:44.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.964083+0000", 2024-09-20T22:20:44.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.5616999999999999e-05 2024-09-20T22:20:44.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.964101+0000", 2024-09-20T22:20:44.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8397e-05 2024-09-20T22:20:44.337 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.337 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.337 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.337 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.964109+0000", 2024-09-20T22:20:44.337 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0860000000000002e-06 2024-09-20T22:20:44.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.964165+0000", 2024-09-20T22:20:44.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5636000000000001e-05 2024-09-20T22:20:44.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.964325+0000", 2024-09-20T22:20:44.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016039900000000001 2024-09-20T22:20:44.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.338 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.338 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,1", 2024-09-20T22:20:44.338 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.964470+0000", 2024-09-20T22:20:44.338 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000144847 2024-09-20T22:20:44.338 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.967192+0000", 2024-09-20T22:20:44.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0027218839999999999 2024-09-20T22:20:44.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.972942+0000", 2024-09-20T22:20:44.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0057499090000000001 2024-09-20T22:20:44.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.339 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.339 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.339 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:37.251282+0000", 2024-09-20T22:20:44.339 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.27834039100000002 2024-09-20T22:20:44.339 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:37.251344+0000", 2024-09-20T22:20:44.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1484999999999998e-05 2024-09-20T22:20:44.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.340 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.340 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:37.251388+0000", 2024-09-20T22:20:44.340 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.4461e-05 2024-09-20T22:20:44.340 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.340 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.340 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.340 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.340 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.341 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:67566 11.9 11:9fd2b909:::benchmark_data_smithi060_196073_object67565:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e477)", 2024-09-20T22:20:44.341 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:19:36.996184+0000", 2024-09-20T22:20:44.341 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 67.295162961000003, 2024-09-20T22:20:44.341 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.26163831999999998, 2024-09-20T22:20:44.341 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.341 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.341 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.341 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.341 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:20:44.341 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:20:44.341 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 67566 2024-09-20T22:20:44.341 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.341 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.341 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.341 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.996184+0000", 2024-09-20T22:20:44.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.996184+0000", 2024-09-20T22:20:44.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.342 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.342 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.996186+0000", 2024-09-20T22:20:44.342 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.401e-06 2024-09-20T22:20:44.342 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.342 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.996217+0000", 2024-09-20T22:20:44.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0482999999999999e-05 2024-09-20T22:20:44.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.996238+0000", 2024-09-20T22:20:44.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1263999999999998e-05 2024-09-20T22:20:44.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.343 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.343 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.996250+0000", 2024-09-20T22:20:44.343 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2038999999999999e-05 2024-09-20T22:20:44.343 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.343 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.996329+0000", 2024-09-20T22:20:44.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.962e-05 2024-09-20T22:20:44.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.996513+0000", 2024-09-20T22:20:44.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018376199999999999 2024-09-20T22:20:44.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.344 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:20:44.344 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.996660+0000", 2024-09-20T22:20:44.344 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014685800000000001 2024-09-20T22:20:44.344 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.344 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.998821+0000", 2024-09-20T22:20:44.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021613769999999999 2024-09-20T22:20:44.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.998934+0000", 2024-09-20T22:20:44.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000112388 2024-09-20T22:20:44.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:37.257656+0000", 2024-09-20T22:20:44.345 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.25872220899999998 2024-09-20T22:20:44.345 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.345 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.345 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.345 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:37.257765+0000", 2024-09-20T22:20:44.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108436 2024-09-20T22:20:44.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.346 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.346 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:37.257822+0000", 2024-09-20T22:20:44.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7482999999999998e-05 2024-09-20T22:20:44.346 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.346 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.346 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.346 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.346 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.346 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:85135 11.b 11:db385f6e:::benchmark_data_smithi060_196073_object85134:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e489)", 2024-09-20T22:20:44.346 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:20:01.721746+0000", 2024-09-20T22:20:44.346 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 42.569600872000002, 2024-09-20T22:20:44.346 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.26968920099999999, 2024-09-20T22:20:44.347 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.347 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.347 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.347 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.347 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:20:44.347 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:20:44.347 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 85135 2024-09-20T22:20:44.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.347 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.721746+0000", 2024-09-20T22:20:44.347 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.347 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.347 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.347 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.347 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.721746+0000", 2024-09-20T22:20:44.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.721748+0000", 2024-09-20T22:20:44.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9489999999999999e-06 2024-09-20T22:20:44.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.721771+0000", 2024-09-20T22:20:44.348 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3235999999999999e-05 2024-09-20T22:20:44.348 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.348 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.348 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.348 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.721789+0000", 2024-09-20T22:20:44.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8550999999999999e-05 2024-09-20T22:20:44.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.721798+0000", 2024-09-20T22:20:44.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.038e-06 2024-09-20T22:20:44.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.721906+0000", 2024-09-20T22:20:44.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000108779 2024-09-20T22:20:44.349 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.349 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.349 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.349 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.722049+0000", 2024-09-20T22:20:44.349 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014278699999999999 2024-09-20T22:20:44.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,1,5", 2024-09-20T22:20:44.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.722214+0000", 2024-09-20T22:20:44.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016443600000000001 2024-09-20T22:20:44.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.724799+0000", 2024-09-20T22:20:44.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025855589999999999 2024-09-20T22:20:44.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.350 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.350 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.350 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.726123+0000", 2024-09-20T22:20:44.350 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0013234480000000001 2024-09-20T22:20:44.350 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.727077+0000", 2024-09-20T22:20:44.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00095432500000000001 2024-09-20T22:20:44.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.991048+0000", 2024-09-20T22:20:44.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.26397111600000001 2024-09-20T22:20:44.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.351 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.351 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.991248+0000", 2024-09-20T22:20:44.351 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00020048900000000001 2024-09-20T22:20:44.351 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.351 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.352 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.352 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.991435+0000", 2024-09-20T22:20:44.352 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018648800000000001 2024-09-20T22:20:44.352 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.352 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.352 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.352 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.352 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.353 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:85144 11.b 11:da68ff6e:::benchmark_data_smithi060_196073_object85143:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e489)", 2024-09-20T22:20:44.353 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:20:01.731703+0000", 2024-09-20T22:20:44.353 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 42.559643680999997, 2024-09-20T22:20:44.353 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.26950739400000001, 2024-09-20T22:20:44.353 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.353 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.353 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.353 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.353 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:20:44.353 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:20:44.353 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 85144 2024-09-20T22:20:44.353 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.353 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.353 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.353 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.731703+0000", 2024-09-20T22:20:44.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.731703+0000", 2024-09-20T22:20:44.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.354 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.731705+0000", 2024-09-20T22:20:44.354 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9139999999999998e-06 2024-09-20T22:20:44.354 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.354 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.354 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.731741+0000", 2024-09-20T22:20:44.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5873e-05 2024-09-20T22:20:44.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.731755+0000", 2024-09-20T22:20:44.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3922e-05 2024-09-20T22:20:44.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.355 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.355 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.731762+0000", 2024-09-20T22:20:44.355 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8150000000000006e-06 2024-09-20T22:20:44.355 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.355 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.731832+0000", 2024-09-20T22:20:44.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9536999999999995e-05 2024-09-20T22:20:44.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.731988+0000", 2024-09-20T22:20:44.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000155995 2024-09-20T22:20:44.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.356 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.356 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,1,5", 2024-09-20T22:20:44.356 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.732144+0000", 2024-09-20T22:20:44.356 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000156019 2024-09-20T22:20:44.356 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.734910+0000", 2024-09-20T22:20:44.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0027660409999999999 2024-09-20T22:20:44.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.735489+0000", 2024-09-20T22:20:44.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00057849400000000004 2024-09-20T22:20:44.357 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.357 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.357 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.357 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.736165+0000", 2024-09-20T22:20:44.357 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00067601100000000002 2024-09-20T22:20:44.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.000984+0000", 2024-09-20T22:20:44.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.26481959900000002 2024-09-20T22:20:44.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.358 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.358 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.001094+0000", 2024-09-20T22:20:44.358 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000109534 2024-09-20T22:20:44.358 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.358 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.359 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.359 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.001210+0000", 2024-09-20T22:20:44.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011663999999999999 2024-09-20T22:20:44.359 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.359 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.359 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.359 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.359 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.359 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:85657 11.b 11:dfeafebb:::benchmark_data_smithi060_196073_object85656:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e489)", 2024-09-20T22:20:44.359 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:20:02.573078+0000", 2024-09-20T22:20:44.359 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 41.718268807999998, 2024-09-20T22:20:44.359 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.28101744400000001, 2024-09-20T22:20:44.359 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.360 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.360 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.360 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.360 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:20:44.360 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:20:44.360 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 85657 2024-09-20T22:20:44.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.360 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.360 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.573078+0000", 2024-09-20T22:20:44.360 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.360 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.360 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.360 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.573078+0000", 2024-09-20T22:20:44.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.573080+0000", 2024-09-20T22:20:44.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9630000000000001e-06 2024-09-20T22:20:44.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.361 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.361 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.573108+0000", 2024-09-20T22:20:44.361 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8000999999999998e-05 2024-09-20T22:20:44.361 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.361 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.652557+0000", 2024-09-20T22:20:44.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.079448764000000005 2024-09-20T22:20:44.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.652570+0000", 2024-09-20T22:20:44.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.365e-05 2024-09-20T22:20:44.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.362 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.362 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.655518+0000", 2024-09-20T22:20:44.362 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0029473479999999998 2024-09-20T22:20:44.362 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.362 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.655683+0000", 2024-09-20T22:20:44.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000165926 2024-09-20T22:20:44.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,1,5", 2024-09-20T22:20:44.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.655860+0000", 2024-09-20T22:20:44.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000176272 2024-09-20T22:20:44.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.363 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.363 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.363 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.658415+0000", 2024-09-20T22:20:44.363 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025549909999999999 2024-09-20T22:20:44.363 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.661824+0000", 2024-09-20T22:20:44.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0034088809999999999 2024-09-20T22:20:44.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.661855+0000", 2024-09-20T22:20:44.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0920999999999998e-05 2024-09-20T22:20:44.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.364 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.364 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.364 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.853821+0000", 2024-09-20T22:20:44.364 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.19196670800000001 2024-09-20T22:20:44.364 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.853959+0000", 2024-09-20T22:20:44.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013786399999999999 2024-09-20T22:20:44.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.365 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.365 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.854095+0000", 2024-09-20T22:20:44.365 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000136155 2024-09-20T22:20:44.365 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.365 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.365 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.365 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.365 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.365 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:100662 11.9 11:9da2d835:::benchmark_data_smithi060_196073_object100661:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-20T22:20:44.366 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:20:21.445442+0000", 2024-09-20T22:20:44.366 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 22.845904907000001, 2024-09-20T22:20:44.366 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.30283800500000002, 2024-09-20T22:20:44.366 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.366 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.366 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.366 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.366 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:20:44.366 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:20:44.366 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 100662 2024-09-20T22:20:44.366 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.366 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.366 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.445442+0000", 2024-09-20T22:20:44.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.445442+0000", 2024-09-20T22:20:44.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.367 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.367 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.445443+0000", 2024-09-20T22:20:44.367 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2640000000000001e-06 2024-09-20T22:20:44.367 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.367 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.445553+0000", 2024-09-20T22:20:44.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0001105 2024-09-20T22:20:44.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.445566+0000", 2024-09-20T22:20:44.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2177e-05 2024-09-20T22:20:44.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.368 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.368 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.445573+0000", 2024-09-20T22:20:44.368 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.939e-06 2024-09-20T22:20:44.368 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.368 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.448431+0000", 2024-09-20T22:20:44.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0028579959999999998 2024-09-20T22:20:44.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.448580+0000", 2024-09-20T22:20:44.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000149801 2024-09-20T22:20:44.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.369 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:20:44.369 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.448747+0000", 2024-09-20T22:20:44.369 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016704099999999999 2024-09-20T22:20:44.369 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.369 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.458059+0000", 2024-09-20T22:20:44.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0093113970000000008 2024-09-20T22:20:44.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.458102+0000", 2024-09-20T22:20:44.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3439000000000002e-05 2024-09-20T22:20:44.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.370 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.370 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.370 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.747923+0000", 2024-09-20T22:20:44.370 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.28982080300000002 2024-09-20T22:20:44.370 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.748227+0000", 2024-09-20T22:20:44.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000304254 2024-09-20T22:20:44.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.371 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.371 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.748280+0000", 2024-09-20T22:20:44.371 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2394e-05 2024-09-20T22:20:44.371 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.371 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.371 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.371 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.371 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.371 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:100663 11.9 11:925108c2:::benchmark_data_smithi060_196073_object100662:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-20T22:20:44.372 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:20:21.447907+0000", 2024-09-20T22:20:44.372 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 22.843439376999999, 2024-09-20T22:20:44.372 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.31025863799999998, 2024-09-20T22:20:44.372 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.372 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.372 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.372 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.372 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:20:44.372 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:20:44.372 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 100663 2024-09-20T22:20:44.372 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.372 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.372 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.372 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.372 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.447907+0000", 2024-09-20T22:20:44.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.447907+0000", 2024-09-20T22:20:44.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.373 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.447909+0000", 2024-09-20T22:20:44.373 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2950000000000001e-06 2024-09-20T22:20:44.373 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.373 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.373 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.447962+0000", 2024-09-20T22:20:44.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3897000000000003e-05 2024-09-20T22:20:44.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.447976+0000", 2024-09-20T22:20:44.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3472000000000001e-05 2024-09-20T22:20:44.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.374 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.374 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.374 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.447984+0000", 2024-09-20T22:20:44.374 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7279999999999995e-06 2024-09-20T22:20:44.374 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.451140+0000", 2024-09-20T22:20:44.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0031561039999999999 2024-09-20T22:20:44.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.451290+0000", 2024-09-20T22:20:44.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000149841 2024-09-20T22:20:44.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.375 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.375 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:20:44.375 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.451436+0000", 2024-09-20T22:20:44.375 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000146157 2024-09-20T22:20:44.375 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.458114+0000", 2024-09-20T22:20:44.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0066781439999999996 2024-09-20T22:20:44.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.458150+0000", 2024-09-20T22:20:44.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5639999999999998e-05 2024-09-20T22:20:44.376 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.376 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.376 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.376 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.757624+0000", 2024-09-20T22:20:44.376 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.29947492199999998 2024-09-20T22:20:44.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.758120+0000", 2024-09-20T22:20:44.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00049519899999999999 2024-09-20T22:20:44.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.377 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.377 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.377 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.758166+0000", 2024-09-20T22:20:44.377 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6239000000000002e-05 2024-09-20T22:20:44.377 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.377 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.377 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.377 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.378 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.378 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:100665 11.9 11:9b492d66:::benchmark_data_smithi060_196073_object100664:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-20T22:20:44.378 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:20:21.448002+0000", 2024-09-20T22:20:44.378 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 22.843344661, 2024-09-20T22:20:44.378 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.31457281300000001, 2024-09-20T22:20:44.378 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.378 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.378 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.378 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.378 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:20:44.378 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:20:44.378 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 100665 2024-09-20T22:20:44.378 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.378 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.379 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.379 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.448002+0000", 2024-09-20T22:20:44.379 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.379 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.379 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.380 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.448002+0000", 2024-09-20T22:20:44.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.380 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.448003+0000", 2024-09-20T22:20:44.380 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7000000000000005e-07 2024-09-20T22:20:44.380 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.380 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.380 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.448053+0000", 2024-09-20T22:20:44.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0453e-05 2024-09-20T22:20:44.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.448061+0000", 2024-09-20T22:20:44.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0469999999999994e-06 2024-09-20T22:20:44.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.381 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.448067+0000", 2024-09-20T22:20:44.381 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4189999999999999e-06 2024-09-20T22:20:44.381 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.381 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.381 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.382 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.453713+0000", 2024-09-20T22:20:44.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005645153 2024-09-20T22:20:44.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.382 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.382 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.453858+0000", 2024-09-20T22:20:44.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00014567900000000001 2024-09-20T22:20:44.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.382 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:20:44.382 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.454008+0000", 2024-09-20T22:20:44.382 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000149825 2024-09-20T22:20:44.382 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.382 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.382 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.382 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.458196+0000", 2024-09-20T22:20:44.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0041878879999999999 2024-09-20T22:20:44.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.458239+0000", 2024-09-20T22:20:44.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.3108000000000001e-05 2024-09-20T22:20:44.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.762478+0000", 2024-09-20T22:20:44.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.30423903699999999 2024-09-20T22:20:44.383 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.383 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.383 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.383 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.762536+0000", 2024-09-20T22:20:44.383 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7855000000000001e-05 2024-09-20T22:20:44.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.384 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.384 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.762575+0000", 2024-09-20T22:20:44.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8779000000000003e-05 2024-09-20T22:20:44.384 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.384 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.384 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.384 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.384 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.384 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:100670 11.b 11:d66c09dc:::benchmark_data_smithi060_196073_object100669:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-20T22:20:44.384 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:20:21.451987+0000", 2024-09-20T22:20:44.384 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 22.839359425000001, 2024-09-20T22:20:44.384 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.31040408899999999, 2024-09-20T22:20:44.385 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.385 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.385 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.385 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.385 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:20:44.385 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:20:44.385 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 100670 2024-09-20T22:20:44.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.385 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.385 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.385 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.385 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.451987+0000", 2024-09-20T22:20:44.385 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.385 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.451987+0000", 2024-09-20T22:20:44.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.451989+0000", 2024-09-20T22:20:44.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.412e-06 2024-09-20T22:20:44.386 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.386 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.386 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.386 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.452039+0000", 2024-09-20T22:20:44.386 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0107e-05 2024-09-20T22:20:44.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.452051+0000", 2024-09-20T22:20:44.387 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2697999999999999e-05 2024-09-20T22:20:44.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.452059+0000", 2024-09-20T22:20:44.387 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.447e-06 2024-09-20T22:20:44.387 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.387 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.387 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.387 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.452122+0000", 2024-09-20T22:20:44.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3332999999999998e-05 2024-09-20T22:20:44.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.452262+0000", 2024-09-20T22:20:44.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013972400000000001 2024-09-20T22:20:44.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,1,5", 2024-09-20T22:20:44.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.452411+0000", 2024-09-20T22:20:44.388 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000149204 2024-09-20T22:20:44.388 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.388 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.388 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.388 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.456110+0000", 2024-09-20T22:20:44.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0036991519999999998 2024-09-20T22:20:44.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.389 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.389 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.456795+0000", 2024-09-20T22:20:44.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00068477200000000005 2024-09-20T22:20:44.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.389 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.389 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.460295+0000", 2024-09-20T22:20:44.389 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.003500202 2024-09-20T22:20:44.389 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.389 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.762185+0000", 2024-09-20T22:20:44.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.30189017899999998 2024-09-20T22:20:44.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.762261+0000", 2024-09-20T22:20:44.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5167999999999996e-05 2024-09-20T22:20:44.390 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.390 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.390 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.390 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.762391+0000", 2024-09-20T22:20:44.390 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000130691 2024-09-20T22:20:44.390 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.390 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.391 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.391 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.391 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.391 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:100674 11.b 11:d5a3d362:::benchmark_data_smithi060_196073_object100673:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-20T22:20:44.391 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:20:21.457002+0000", 2024-09-20T22:20:44.391 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 22.834344811000001, 2024-09-20T22:20:44.391 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.306114948, 2024-09-20T22:20:44.391 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.391 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.391 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.391 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.391 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:20:44.391 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:20:44.392 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 100674 2024-09-20T22:20:44.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.392 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.392 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.392 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.457002+0000", 2024-09-20T22:20:44.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.392 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.392 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.457002+0000", 2024-09-20T22:20:44.392 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.392 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.392 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.392 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.457003+0000", 2024-09-20T22:20:44.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6500000000000001e-06 2024-09-20T22:20:44.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.457059+0000", 2024-09-20T22:20:44.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5272000000000002e-05 2024-09-20T22:20:44.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.393 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.457071+0000", 2024-09-20T22:20:44.393 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2058e-05 2024-09-20T22:20:44.393 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.393 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.393 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.457077+0000", 2024-09-20T22:20:44.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6579999999999996e-06 2024-09-20T22:20:44.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.457132+0000", 2024-09-20T22:20:44.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4401999999999999e-05 2024-09-20T22:20:44.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.394 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.394 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.394 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.457283+0000", 2024-09-20T22:20:44.394 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015089899999999999 2024-09-20T22:20:44.394 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,1,5", 2024-09-20T22:20:44.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.457475+0000", 2024-09-20T22:20:44.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019242599999999999 2024-09-20T22:20:44.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.460337+0000", 2024-09-20T22:20:44.395 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0028621520000000002 2024-09-20T22:20:44.395 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.395 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.395 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.395 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.460428+0000", 2024-09-20T22:20:44.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0884000000000006e-05 2024-09-20T22:20:44.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.396 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.396 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.462142+0000", 2024-09-20T22:20:44.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0017133820000000001 2024-09-20T22:20:44.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.396 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.396 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.762938+0000", 2024-09-20T22:20:44.396 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.30079655700000002 2024-09-20T22:20:44.396 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.396 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.396 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.763004+0000", 2024-09-20T22:20:44.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5536000000000004e-05 2024-09-20T22:20:44.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.397 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.397 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.763117+0000", 2024-09-20T22:20:44.397 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113072 2024-09-20T22:20:44.397 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.397 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.397 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.397 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.397 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.397 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:100686 11.b 11:d990d61b:::benchmark_data_smithi060_196073_object100685:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-20T22:20:44.398 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:20:21.467500+0000", 2024-09-20T22:20:44.398 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 22.823846738, 2024-09-20T22:20:44.398 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.29619720500000002, 2024-09-20T22:20:44.398 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.398 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.398 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.398 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.398 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:20:44.398 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:20:44.398 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 100686 2024-09-20T22:20:44.398 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.398 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.398 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.467500+0000", 2024-09-20T22:20:44.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.467500+0000", 2024-09-20T22:20:44.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.399 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.399 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.399 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.399 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.467502+0000", 2024-09-20T22:20:44.399 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7209999999999999e-06 2024-09-20T22:20:44.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.400 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.400 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.467530+0000", 2024-09-20T22:20:44.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.8067000000000001e-05 2024-09-20T22:20:44.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.400 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.400 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.467544+0000", 2024-09-20T22:20:44.400 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4052e-05 2024-09-20T22:20:44.400 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.400 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.400 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.400 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.467556+0000", 2024-09-20T22:20:44.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2587e-05 2024-09-20T22:20:44.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.401 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.401 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.467621+0000", 2024-09-20T22:20:44.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4486999999999994e-05 2024-09-20T22:20:44.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.401 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.401 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.467820+0000", 2024-09-20T22:20:44.401 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019913 2024-09-20T22:20:44.401 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.401 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.401 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,1,5", 2024-09-20T22:20:44.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.468016+0000", 2024-09-20T22:20:44.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000196052 2024-09-20T22:20:44.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.470099+0000", 2024-09-20T22:20:44.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020832810000000002 2024-09-20T22:20:44.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.402 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.402 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.470454+0000", 2024-09-20T22:20:44.402 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00035455600000000001 2024-09-20T22:20:44.402 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.402 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.470610+0000", 2024-09-20T22:20:44.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015572900000000001 2024-09-20T22:20:44.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.763348+0000", 2024-09-20T22:20:44.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.29273834599999998 2024-09-20T22:20:44.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.403 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.403 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.403 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.763518+0000", 2024-09-20T22:20:44.403 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017043900000000001 2024-09-20T22:20:44.403 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.404 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.404 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.763697+0000", 2024-09-20T22:20:44.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000178758 2024-09-20T22:20:44.404 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.404 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.404 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.404 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.404 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.404 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:100692 11.9 11:9aa05a11:::benchmark_data_smithi060_196073_object100691:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-20T22:20:44.404 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:20:21.476905+0000", 2024-09-20T22:20:44.404 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 22.814441810000002, 2024-09-20T22:20:44.404 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.285824515, 2024-09-20T22:20:44.404 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.405 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.405 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.405 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.405 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:20:44.405 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:20:44.405 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 100692 2024-09-20T22:20:44.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.405 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.405 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.476905+0000", 2024-09-20T22:20:44.405 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.405 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.405 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.405 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.405 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.476905+0000", 2024-09-20T22:20:44.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.406 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.476906+0000", 2024-09-20T22:20:44.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.542e-06 2024-09-20T22:20:44.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.406 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.406 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.406 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.476930+0000", 2024-09-20T22:20:44.406 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3736000000000001e-05 2024-09-20T22:20:44.406 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.476948+0000", 2024-09-20T22:20:44.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8043999999999999e-05 2024-09-20T22:20:44.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.407 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.407 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.476957+0000", 2024-09-20T22:20:44.407 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.4349999999999997e-06 2024-09-20T22:20:44.407 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.407 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.408 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.477019+0000", 2024-09-20T22:20:44.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.2583999999999996e-05 2024-09-20T22:20:44.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.408 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.477120+0000", 2024-09-20T22:20:44.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010129 2024-09-20T22:20:44.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:20:44.408 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.477208+0000", 2024-09-20T22:20:44.408 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8047000000000006e-05 2024-09-20T22:20:44.408 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.408 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.408 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.479126+0000", 2024-09-20T22:20:44.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019172379999999999 2024-09-20T22:20:44.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.479251+0000", 2024-09-20T22:20:44.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012516 2024-09-20T22:20:44.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.762618+0000", 2024-09-20T22:20:44.409 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.28336661000000002 2024-09-20T22:20:44.409 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.409 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.409 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.409 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.762669+0000", 2024-09-20T22:20:44.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1629000000000003e-05 2024-09-20T22:20:44.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.410 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.410 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:21.762729+0000", 2024-09-20T22:20:44.410 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.02e-05 2024-09-20T22:20:44.410 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.410 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.410 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.410 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.410 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.410 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:119405 11.9 11:9c6beafa:::benchmark_data_smithi060_196073_object17418:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e501)", 2024-09-20T22:20:44.410 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:20:43.103693+0000", 2024-09-20T22:20:44.410 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.1876532609999999, 2024-09-20T22:20:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.064771087000000005, 2024-09-20T22:20:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:20:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:20:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 119405 2024-09-20T22:20:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.103693+0000", 2024-09-20T22:20:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.411 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.103693+0000", 2024-09-20T22:20:44.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.103696+0000", 2024-09-20T22:20:44.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0240000000000002e-06 2024-09-20T22:20:44.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.412 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.412 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.412 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.103706+0000", 2024-09-20T22:20:44.412 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.7659999999999993e-06 2024-09-20T22:20:44.412 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.103733+0000", 2024-09-20T22:20:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6497000000000001e-05 2024-09-20T22:20:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.103742+0000", 2024-09-20T22:20:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.8130000000000007e-06 2024-09-20T22:20:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.113879+0000", 2024-09-20T22:20:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.010136103 2024-09-20T22:20:44.413 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.114033+0000", 2024-09-20T22:20:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015400799999999999 2024-09-20T22:20:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:20:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.114184+0000", 2024-09-20T22:20:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015137000000000001 2024-09-20T22:20:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.121693+0000", 2024-09-20T22:20:44.414 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0075090620000000004 2024-09-20T22:20:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.121865+0000", 2024-09-20T22:20:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00017169799999999999 2024-09-20T22:20:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.168329+0000", 2024-09-20T22:20:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.046464002999999997 2024-09-20T22:20:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.168418+0000", 2024-09-20T22:20:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9222999999999996e-05 2024-09-20T22:20:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.415 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.168464+0000", 2024-09-20T22:20:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.6520000000000002e-05 2024-09-20T22:20:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:119414 11.9 11:944fe7cb:::benchmark_data_smithi060_196073_object17427:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e501)", 2024-09-20T22:20:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:20:43.109153+0000", 2024-09-20T22:20:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 1.1821933279999999, 2024-09-20T22:20:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.059423505000000001, 2024-09-20T22:20:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.416 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:20:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:20:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 119414 2024-09-20T22:20:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.109153+0000", 2024-09-20T22:20:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.109153+0000", 2024-09-20T22:20:44.417 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.109155+0000", 2024-09-20T22:20:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.0360000000000001e-06 2024-09-20T22:20:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.109162+0000", 2024-09-20T22:20:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.3010000000000003e-06 2024-09-20T22:20:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.418 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.109176+0000", 2024-09-20T22:20:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4338000000000001e-05 2024-09-20T22:20:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.109185+0000", 2024-09-20T22:20:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1619999999999997e-06 2024-09-20T22:20:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.117978+0000", 2024-09-20T22:20:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0087930639999999997 2024-09-20T22:20:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.118461+0000", 2024-09-20T22:20:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00048301300000000002 2024-09-20T22:20:44.419 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,6", 2024-09-20T22:20:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.118586+0000", 2024-09-20T22:20:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000124669 2024-09-20T22:20:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.121724+0000", 2024-09-20T22:20:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0031378700000000001 2024-09-20T22:20:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.122960+0000", 2024-09-20T22:20:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012361970000000001 2024-09-20T22:20:44.420 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.168502+0000", 2024-09-20T22:20:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.045541804999999998 2024-09-20T22:20:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.168544+0000", 2024-09-20T22:20:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1807999999999997e-05 2024-09-20T22:20:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:43.168577+0000", 2024-09-20T22:20:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3241999999999998e-05 2024-09-20T22:20:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.421 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:120238 11.b 11:daaf4e26:::benchmark_data_smithi060_196073_object18251:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e501)", 2024-09-20T22:20:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:20:44.068355+0000", 2024-09-20T22:20:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.22299171200000001, 2024-09-20T22:20:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.062026887000000003, 2024-09-20T22:20:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:20:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:20:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:20:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:20:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:20:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:20:44.422 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 120238 2024-09-20T22:20:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:20:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:20:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:44.068355+0000", 2024-09-20T22:20:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:20:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:44.068355+0000", 2024-09-20T22:20:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:20:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:20:44.423 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:44.068356+0000", 2024-09-20T22:20:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5829999999999999e-06 2024-09-20T22:20:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:20:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:44.068364+0000", 2024-09-20T22:20:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5190000000000003e-06 2024-09-20T22:20:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:20:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:44.068381+0000", 2024-09-20T22:20:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6799e-05 2024-09-20T22:20:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.424 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:20:44.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:44.068388+0000", 2024-09-20T22:20:44.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6229999999999997e-06 2024-09-20T22:20:44.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:20:44.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:44.068460+0000", 2024-09-20T22:20:44.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1785999999999993e-05 2024-09-20T22:20:44.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:20:44.425 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:44.068613+0000", 2024-09-20T22:20:44.425 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015255800000000001 2024-09-20T22:20:44.425 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.425 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.425 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,1,5", 2024-09-20T22:20:44.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:44.068782+0000", 2024-09-20T22:20:44.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00016967999999999999 2024-09-20T22:20:44.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:20:44.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:44.070742+0000", 2024-09-20T22:20:44.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.001959424 2024-09-20T22:20:44.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:44.071198+0000", 2024-09-20T22:20:44.426 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000455661 2024-09-20T22:20:44.426 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.426 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.426 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.426 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:44.071259+0000", 2024-09-20T22:20:44.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.1268999999999999e-05 2024-09-20T22:20:44.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:20:44.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:44.130186+0000", 2024-09-20T22:20:44.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.058926988999999999 2024-09-20T22:20:44.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:20:44.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:44.130261+0000", 2024-09-20T22:20:44.427 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5598000000000004e-05 2024-09-20T22:20:44.427 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:20:44.427 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:20:44.427 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:20:44.427 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:44.130382+0000", 2024-09-20T22:20:44.428 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00012039799999999999 2024-09-20T22:20:44.428 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.428 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.428 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.428 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:20:44.428 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:20:44.428 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:20:44.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:44.429 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:44.429 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:44.530 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:44.564 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:44.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:44.565 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:44.666 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:44.700 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:44.701 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:44.701 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:44.735 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:44.753 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:20:44.802 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:44.836 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:44.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:44.837 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:44.938 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:44.977 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:44.979 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:44.979 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:44 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3863607578' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:20:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:44 smithi067 ceph-mon[33511]: pgmap v953: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 172 active+clean; 5.2 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 691 op/s; 5014/257631 objects degraded (1.946%); 10691/257631 objects misplaced (4.150%); 31 MiB/s, 498 objects/s recovering 2024-09-20T22:20:45.035 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:44 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3863607578' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:20:45.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:44 smithi060 ceph-mon[40290]: pgmap v953: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 172 active+clean; 5.2 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 691 op/s; 5014/257631 objects degraded (1.946%); 10691/257631 objects misplaced (4.150%); 31 MiB/s, 498 objects/s recovering 2024-09-20T22:20:45.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:44 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3863607578' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:20:45.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:44 smithi060 ceph-mon[29694]: pgmap v953: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 172 active+clean; 5.2 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 691 op/s; 5014/257631 objects degraded (1.946%); 10691/257631 objects misplaced (4.150%); 31 MiB/s, 498 objects/s recovering 2024-09-20T22:20:45.080 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:45.112 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:45.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:45.113 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:45.214 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:45.256 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:45.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:45.258 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:45.358 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:45.395 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:45.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:45.396 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:45.496 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:45.533 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:45.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:45.534 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:45.536 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:20:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:20:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:20:45.635 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:45.661 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:45.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:45.662 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:45.763 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:45.791 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:45.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:45.792 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:45.893 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:45.915 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:45.915 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:45.916 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:45 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:20:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:45 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3863607578' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:20:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:45 smithi067 ceph-mon[33511]: osdmap e502: 8 total, 7 up, 6 in 2024-09-20T22:20:46.017 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:46.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:45 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:20:46.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:45 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3863607578' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:20:46.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:45 smithi060 ceph-mon[29694]: osdmap e502: 8 total, 7 up, 6 in 2024-09-20T22:20:46.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:45 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:20:46.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:45 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3863607578' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:20:46.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:45 smithi060 ceph-mon[40290]: osdmap e502: 8 total, 7 up, 6 in 2024-09-20T22:20:46.041 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:46.042 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:46.042 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:46.143 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:46.173 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:46.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:46.175 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:46.275 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:46.315 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:46.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:46.316 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:46.417 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:46.457 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:46.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:46.458 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:46.559 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:46.594 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:46.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:46.595 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:46.696 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:46.737 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:46.737 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:46.738 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:46.838 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:46.873 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:46.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:46.875 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:46.975 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:47.013 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:47.014 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:46 smithi060 ceph-mon[29694]: pgmap v955: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 172 active+clean; 5.1 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 946 op/s; 4761/253053 objects degraded (1.881%); 10438/253053 objects misplaced (4.125%); 34 MiB/s, 540 objects/s recovering 2024-09-20T22:20:47.014 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:46 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 5014/257631 objects degraded (1.946%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:47.014 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:46 smithi060 ceph-mon[40290]: pgmap v955: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 172 active+clean; 5.1 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 946 op/s; 4761/253053 objects degraded (1.881%); 10438/253053 objects misplaced (4.125%); 34 MiB/s, 540 objects/s recovering 2024-09-20T22:20:47.014 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:46 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 5014/257631 objects degraded (1.946%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:47.015 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:47.015 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:47.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:46 smithi067 ceph-mon[33511]: pgmap v955: 178 pgs: 1 active+remapped+backfilling, 1 active+remapped+backfill_wait+backfill_toofull, 1 active+recovering+degraded+remapped, 3 active+recovering+undersized+degraded+remapped, 172 active+clean; 5.1 GiB data, 19 GiB used, 518 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 946 op/s; 4761/253053 objects degraded (1.881%); 10438/253053 objects misplaced (4.125%); 34 MiB/s, 540 objects/s recovering 2024-09-20T22:20:47.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:46 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 5014/257631 objects degraded (1.946%), 4 pgs degraded (PG_DEGRADED) 2024-09-20T22:20:47.116 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:47.147 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:47.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:47.148 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:47.249 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:47.281 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:47.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:47.282 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:47.383 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:47.423 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:47.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:47.424 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:47.525 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:47.562 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:47.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:47.563 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:47.664 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:47.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:47.704 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:47.704 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:47.805 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:47.850 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:47.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:47.851 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:47.952 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:47.986 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:47.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:47.987 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:48.088 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:48.128 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:48.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:48.129 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:48.230 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:48.264 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:48.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:48.265 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:48.340 INFO:teuthology.orchestra.run.smithi067.stderr:Creating an OSD daemon form without an OSD FSID value 2024-09-20T22:20:48.366 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:48.416 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:48.416 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:48.417 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:48.517 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:48.551 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:48.552 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:48.552 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:48.561 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:48 smithi067 ceph-mon[33511]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:20:48.653 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:48.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:48.683 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:48.683 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:48.784 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:48.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:48 smithi060 ceph-mon[29694]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:20:48.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:48 smithi060 ceph-mon[40290]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 2 pgs backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:20:48.817 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:48.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:48.818 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:48.919 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:48.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:48.957 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:48.957 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:49.058 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:49.097 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:49.098 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:49.098 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:49.199 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:49.249 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:49.250 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:49.250 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:49.351 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:49.386 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:49.387 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:49.387 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:49.488 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:49.532 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:49.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:49.534 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:49.551 INFO:teuthology.orchestra.run.smithi067.stdout:get_pg_num_history pg_num_history pg_num_history(e485 pg_nums {1={197=11,383=21,428=31},2={62=3},3={63=32},4={65=32},5={67=32},6={69=32},10={370=16},11={456=16}} deleted_pools 455,10) 2024-09-20T22:20:49.551 INFO:teuthology.orchestra.run.smithi067.stdout:Importing pgid 11.d 2024-09-20T22:20:49.577 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0008cae:::benchmark_data_smithi060_196073_object45401:head# 2024-09-20T22:20:49.578 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.578 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0028b4f:::benchmark_data_smithi060_196073_object6636:head# 2024-09-20T22:20:49.578 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.578 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b002da64:::benchmark_data_smithi060_196073_object33538:head# 2024-09-20T22:20:49.578 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.579 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b005916d:::benchmark_data_smithi060_196073_object21370:head# 2024-09-20T22:20:49.579 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.579 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0061903:::benchmark_data_smithi060_196073_object7962:head# 2024-09-20T22:20:49.579 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.579 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b00654a9:::benchmark_data_smithi060_196073_object19172:head# 2024-09-20T22:20:49.579 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.579 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b00816bd:::benchmark_data_smithi060_196073_object20728:head# 2024-09-20T22:20:49.579 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.580 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0086d3a:::benchmark_data_smithi060_196073_object27194:head# 2024-09-20T22:20:49.580 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.580 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b008e665:::benchmark_data_smithi060_196073_object51056:head# 2024-09-20T22:20:49.580 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.580 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b00c49ab:::benchmark_data_smithi060_196073_object34666:head# 2024-09-20T22:20:49.580 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.581 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b00c5fff:::benchmark_data_smithi060_196073_object42440:head# 2024-09-20T22:20:49.582 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.582 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b00cc4a9:::benchmark_data_smithi060_196073_object54410:head# 2024-09-20T22:20:49.583 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.583 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b00eb298:::benchmark_data_smithi060_196073_object4306:head# 2024-09-20T22:20:49.583 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.584 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b00f55a7:::benchmark_data_smithi060_196073_object58567:head# 2024-09-20T22:20:49.584 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.584 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b00f9c3c:::benchmark_data_smithi060_196073_object25802:head# 2024-09-20T22:20:49.584 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.584 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0113c5b:::benchmark_data_smithi060_196073_object47631:head# 2024-09-20T22:20:49.585 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.585 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b01334da:::benchmark_data_smithi060_196073_object42837:head# 2024-09-20T22:20:49.586 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.586 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b01440a9:::benchmark_data_smithi060_196073_object21105:head# 2024-09-20T22:20:49.586 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.587 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b014abf5:::benchmark_data_smithi060_196073_object5104:head# 2024-09-20T22:20:49.587 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.587 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b014c92f:::benchmark_data_smithi060_196073_object37410:head# 2024-09-20T22:20:49.587 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.588 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0160b3d:::benchmark_data_smithi060_196073_object57293:head# 2024-09-20T22:20:49.588 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.588 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b01634c9:::benchmark_data_smithi060_196073_object36953:head# 2024-09-20T22:20:49.588 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.588 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b017b5d0:::benchmark_data_smithi060_196073_object9895:head# 2024-09-20T22:20:49.588 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.589 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b01ad0c5:::benchmark_data_smithi060_196073_object52924:head# 2024-09-20T22:20:49.589 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.589 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b01bff2e:::benchmark_data_smithi060_196073_object25016:head# 2024-09-20T22:20:49.620 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.621 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b01c8006:::benchmark_data_smithi060_196073_object11799:head# 2024-09-20T22:20:49.621 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.621 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b01e1bc9:::benchmark_data_smithi060_196073_object8450:head# 2024-09-20T22:20:49.622 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.623 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b01e6ef2:::benchmark_data_smithi060_196073_object44124:head# 2024-09-20T22:20:49.623 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.623 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b01ed1f9:::benchmark_data_smithi060_196073_object60023:head# 2024-09-20T22:20:49.624 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.624 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b01f4d04:::benchmark_data_smithi060_196073_object55077:head# 2024-09-20T22:20:49.624 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.624 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0207395:::benchmark_data_smithi060_196073_object19792:head# 2024-09-20T22:20:49.624 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.625 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b02104b2:::benchmark_data_smithi060_196073_object15905:head# 2024-09-20T22:20:49.625 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.625 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0227ac4:::benchmark_data_smithi060_196073_object25785:head# 2024-09-20T22:20:49.625 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.626 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0227e35:::benchmark_data_smithi060_196073_object16151:head# 2024-09-20T22:20:49.626 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.626 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0231296:::benchmark_data_smithi060_196073_object20651:head# 2024-09-20T22:20:49.626 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.627 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0236a02:::benchmark_data_smithi060_196073_object10745:head# 2024-09-20T22:20:49.627 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.627 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0261197:::benchmark_data_smithi060_196073_object18392:head# 2024-09-20T22:20:49.627 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.628 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b026fe91:::benchmark_data_smithi060_196073_object57481:head# 2024-09-20T22:20:49.628 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.628 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b02720a4:::benchmark_data_smithi060_196073_object17823:head# 2024-09-20T22:20:49.628 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.629 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0280b4f:::benchmark_data_smithi060_196073_object50341:head# 2024-09-20T22:20:49.629 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.629 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b02bfe22:::benchmark_data_smithi060_196073_object43504:head# 2024-09-20T22:20:49.630 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.630 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b02c78d9:::benchmark_data_smithi060_196073_object1106:head# 2024-09-20T22:20:49.630 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.630 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b02cc502:::benchmark_data_smithi060_196073_object56359:head# 2024-09-20T22:20:49.631 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.631 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b02ce76d:::benchmark_data_smithi060_196073_object22902:head# 2024-09-20T22:20:49.631 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.631 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b02db9fa:::benchmark_data_smithi060_196073_object45387:head# 2024-09-20T22:20:49.632 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.632 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b02e3280:::benchmark_data_smithi060_196073_object41181:head# 2024-09-20T22:20:49.632 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.633 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b02e7393:::benchmark_data_smithi060_196073_object52039:head# 2024-09-20T22:20:49.633 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.633 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b032fe6c:::benchmark_data_smithi060_196073_object17679:head# 2024-09-20T22:20:49.633 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.634 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b033e32f:::benchmark_data_smithi060_196073_object33004:head# 2024-09-20T22:20:49.634 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.634 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b034e87c:::benchmark_data_smithi060_196073_object30708:head# 2024-09-20T22:20:49.634 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.634 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:49.636 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b035d8d5:::benchmark_data_smithi060_196073_object25494:head# 2024-09-20T22:20:49.636 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.636 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b036a672:::benchmark_data_smithi060_196073_object32279:head# 2024-09-20T22:20:49.636 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.636 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b036bb2d:::benchmark_data_smithi060_196073_object28005:head# 2024-09-20T22:20:49.636 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.636 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b037a3e8:::benchmark_data_smithi060_196073_object26019:head# 2024-09-20T22:20:49.636 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.638 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b037dfb6:::benchmark_data_smithi060_196073_object24012:head# 2024-09-20T22:20:49.638 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.638 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b039355a:::benchmark_data_smithi060_196073_object5511:head# 2024-09-20T22:20:49.638 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.638 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b03a876d:::benchmark_data_smithi060_196073_object8415:head# 2024-09-20T22:20:49.638 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.638 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b03aa41c:::benchmark_data_smithi060_196073_object7789:head# 2024-09-20T22:20:49.638 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.638 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b03f240c:::benchmark_data_smithi060_196073_object26275:head# 2024-09-20T22:20:49.638 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.639 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b03f7ce7:::benchmark_data_smithi060_196073_object57961:head# 2024-09-20T22:20:49.639 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.639 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b03f8369:::benchmark_data_smithi060_196073_object61765:head# 2024-09-20T22:20:49.639 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.639 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b03ff313:::benchmark_data_smithi060_196073_object8844:head# 2024-09-20T22:20:49.639 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.640 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b040a6c3:::benchmark_data_smithi060_196073_object55588:head# 2024-09-20T22:20:49.640 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.640 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b041f568:::benchmark_data_smithi060_196073_object41161:head# 2024-09-20T22:20:49.640 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.642 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b04230cb:::benchmark_data_smithi060_196073_object54535:head# 2024-09-20T22:20:49.642 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.642 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0483c9f:::benchmark_data_smithi060_196073_object37011:head# 2024-09-20T22:20:49.642 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.642 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0487370:::benchmark_data_smithi060_196073_object61681:head# 2024-09-20T22:20:49.643 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.643 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b048d472:::benchmark_data_smithi060_196073_object17457:head# 2024-09-20T22:20:49.645 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.645 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b04adef1:::benchmark_data_smithi060_196073_object718:head# 2024-09-20T22:20:49.645 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.645 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b04cdc31:::benchmark_data_smithi060_196073_object35034:head# 2024-09-20T22:20:49.645 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.645 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b04ef815:::benchmark_data_smithi060_196073_object43278:head# 2024-09-20T22:20:49.645 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.647 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b05067cc:::benchmark_data_smithi060_196073_object8026:head# 2024-09-20T22:20:49.647 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.647 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b050a324:::benchmark_data_smithi060_196073_object20720:head# 2024-09-20T22:20:49.647 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.647 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b051f73e:::benchmark_data_smithi060_196073_object56637:head# 2024-09-20T22:20:49.647 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.647 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b051f98c:::benchmark_data_smithi060_196073_object48887:head# 2024-09-20T22:20:49.647 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.650 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b05275ec:::benchmark_data_smithi060_196073_object10596:head# 2024-09-20T22:20:49.650 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.650 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0530ce5:::benchmark_data_smithi060_196073_object24009:head# 2024-09-20T22:20:49.650 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.650 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b05630fc:::benchmark_data_smithi060_196073_object52874:head# 2024-09-20T22:20:49.650 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.650 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b05631e3:::benchmark_data_smithi060_196073_object51060:head# 2024-09-20T22:20:49.650 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.650 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0567c80:::benchmark_data_smithi060_196073_object265:head# 2024-09-20T22:20:49.650 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.650 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b056d98c:::benchmark_data_smithi060_196073_object29240:head# 2024-09-20T22:20:49.650 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.651 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b05797f3:::benchmark_data_smithi060_196073_object25631:head# 2024-09-20T22:20:49.651 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.651 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b05798f3:::benchmark_data_smithi060_196073_object40693:head# 2024-09-20T22:20:49.651 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.651 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b057c37c:::benchmark_data_smithi060_196073_object18575:head# 2024-09-20T22:20:49.652 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.652 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b059d9ce:::benchmark_data_smithi060_196073_object56032:head# 2024-09-20T22:20:49.652 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.652 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b05bbc3f:::benchmark_data_smithi060_196073_object9593:head# 2024-09-20T22:20:49.652 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.652 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b05d107a:::benchmark_data_smithi060_196073_object23845:head# 2024-09-20T22:20:49.652 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.652 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b05e6ec1:::benchmark_data_smithi060_196073_object37482:head# 2024-09-20T22:20:49.652 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.653 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b05ffd6c:::benchmark_data_smithi060_196073_object4441:head# 2024-09-20T22:20:49.653 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.653 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b060296f:::benchmark_data_smithi060_196073_object27032:head# 2024-09-20T22:20:49.653 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.653 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0607667:::benchmark_data_smithi060_196073_object20271:head# 2024-09-20T22:20:49.654 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.654 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0630582:::benchmark_data_smithi060_196073_object8852:head# 2024-09-20T22:20:49.654 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.654 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0639796:::benchmark_data_smithi060_196073_object6640:head# 2024-09-20T22:20:49.655 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.655 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b063bd40:::benchmark_data_smithi060_196073_object30511:head# 2024-09-20T22:20:49.655 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.656 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b065c65f:::benchmark_data_smithi060_196073_object22874:head# 2024-09-20T22:20:49.656 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.656 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b06669a4:::benchmark_data_smithi060_196073_object20344:head# 2024-09-20T22:20:49.656 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.656 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0674ff0:::benchmark_data_smithi060_196073_object52340:head# 2024-09-20T22:20:49.656 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.657 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b06a8253:::benchmark_data_smithi060_196073_object10026:head# 2024-09-20T22:20:49.657 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.657 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b06c5b03:::benchmark_data_smithi060_196073_object40935:head# 2024-09-20T22:20:49.657 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.658 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b06e9f6d:::benchmark_data_smithi060_196073_object51923:head# 2024-09-20T22:20:49.658 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.658 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b06f798a:::benchmark_data_smithi060_196073_object21005:head# 2024-09-20T22:20:49.658 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.658 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b06fc440:::benchmark_data_smithi060_196073_object6742:head# 2024-09-20T22:20:49.659 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.659 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b07035b5:::benchmark_data_smithi060_196073_object23935:head# 2024-09-20T22:20:49.659 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.660 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b07048cc:::benchmark_data_smithi060_196073_object54438:head# 2024-09-20T22:20:49.660 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.660 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b070566c:::benchmark_data_smithi060_196073_object47180:head# 2024-09-20T22:20:49.660 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.661 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b070db02:::benchmark_data_smithi060_196073_object3112:head# 2024-09-20T22:20:49.661 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.661 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0713c0b:::benchmark_data_smithi060_196073_object60838:head# 2024-09-20T22:20:49.661 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.662 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0732e07:::benchmark_data_smithi060_196073_object17687:head# 2024-09-20T22:20:49.662 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.662 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b07547db:::benchmark_data_smithi060_196073_object26530:head# 2024-09-20T22:20:49.662 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.663 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b07552d6:::benchmark_data_smithi060_196073_object15277:head# 2024-09-20T22:20:49.663 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.663 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b07594b8:::benchmark_data_smithi060_196073_object9769:head# 2024-09-20T22:20:49.663 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.663 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b075ed06:::benchmark_data_smithi060_196073_object5291:head# 2024-09-20T22:20:49.664 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.664 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0764457:::benchmark_data_smithi060_196073_object40741:head# 2024-09-20T22:20:49.664 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.664 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0767013:::benchmark_data_smithi060_196073_object3349:head# 2024-09-20T22:20:49.681 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.681 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0767491:::benchmark_data_smithi060_196073_object25145:head# 2024-09-20T22:20:49.682 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.682 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b076824a:::benchmark_data_smithi060_196073_object52981:head# 2024-09-20T22:20:49.682 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.683 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b076fde9:::benchmark_data_smithi060_196073_object27628:head# 2024-09-20T22:20:49.683 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.683 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:49.684 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0778e39:::benchmark_data_smithi060_196073_object52417:head# 2024-09-20T22:20:49.684 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.684 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b07905eb:::benchmark_data_smithi060_196073_object3959:head# 2024-09-20T22:20:49.684 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.684 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b07a2f96:::benchmark_data_smithi060_196073_object25281:head# 2024-09-20T22:20:49.685 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:49.685 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:49.686 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b07b4c92:::benchmark_data_smithi060_196073_object6088:head# 2024-09-20T22:20:49.686 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.686 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b07bb6a3:::benchmark_data_smithi060_196073_object32241:head# 2024-09-20T22:20:49.686 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.686 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b07c1c47:::benchmark_data_smithi060_196073_object3238:head# 2024-09-20T22:20:49.686 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.686 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b07ffab3:::benchmark_data_smithi060_196073_object51935:head# 2024-09-20T22:20:49.686 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.687 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0806d35:::benchmark_data_smithi060_196073_object15559:head# 2024-09-20T22:20:49.687 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.687 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0810acd:::benchmark_data_smithi060_196073_object57450:head# 2024-09-20T22:20:49.687 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.687 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0823837:::benchmark_data_smithi060_196073_object38736:head# 2024-09-20T22:20:49.687 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.687 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0859150:::benchmark_data_smithi060_196073_object9875:head# 2024-09-20T22:20:49.688 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.688 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0863723:::benchmark_data_smithi060_196073_object36472:head# 2024-09-20T22:20:49.688 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.688 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b086ca37:::benchmark_data_smithi060_196073_object56968:head# 2024-09-20T22:20:49.688 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.689 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b08801ea:::benchmark_data_smithi060_196073_object17681:head# 2024-09-20T22:20:49.689 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.689 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b088d617:::benchmark_data_smithi060_196073_object49197:head# 2024-09-20T22:20:49.690 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.690 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0891889:::benchmark_data_smithi060_196073_object9477:head# 2024-09-20T22:20:49.690 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.691 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0892563:::benchmark_data_smithi060_196073_object41452:head# 2024-09-20T22:20:49.691 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.691 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b089c3a7:::benchmark_data_smithi060_196073_object27371:head# 2024-09-20T22:20:49.691 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.692 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b089fa3e:::benchmark_data_smithi060_196073_object41682:head# 2024-09-20T22:20:49.692 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.692 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b08a04a6:::benchmark_data_smithi060_196073_object6757:head# 2024-09-20T22:20:49.692 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.692 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b08ae27b:::benchmark_data_smithi060_196073_object10973:head# 2024-09-20T22:20:49.706 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.706 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b08cc219:::benchmark_data_smithi060_196073_object28287:head# 2024-09-20T22:20:49.706 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.706 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b08ce801:::benchmark_data_smithi060_196073_object26198:head# 2024-09-20T22:20:49.707 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.707 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b08e9951:::benchmark_data_smithi060_196073_object47031:head# 2024-09-20T22:20:49.707 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.707 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b08eb7f3:::benchmark_data_smithi060_196073_object5377:head# 2024-09-20T22:20:49.707 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.708 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b09017db:::benchmark_data_smithi060_196073_object21415:head# 2024-09-20T22:20:49.708 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.708 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b09110a8:::benchmark_data_smithi060_196073_object15986:head# 2024-09-20T22:20:49.709 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.709 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0923b15:::benchmark_data_smithi060_196073_object58858:head# 2024-09-20T22:20:49.709 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.709 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0939f3d:::benchmark_data_smithi060_196073_object48519:head# 2024-09-20T22:20:49.709 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.709 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b09456af:::benchmark_data_smithi060_196073_object29234:head# 2024-09-20T22:20:49.710 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.710 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0968366:::benchmark_data_smithi060_196073_object35238:head# 2024-09-20T22:20:49.710 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.711 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0971e17:::benchmark_data_smithi060_196073_object47809:head# 2024-09-20T22:20:49.711 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.711 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b097742a:::benchmark_data_smithi060_196073_object25479:head# 2024-09-20T22:20:49.711 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.711 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b09aa1eb:::benchmark_data_smithi060_196073_object32343:head# 2024-09-20T22:20:49.713 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.713 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b09b26a0:::benchmark_data_smithi060_196073_object34323:head# 2024-09-20T22:20:49.713 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.713 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b09bce36:::benchmark_data_smithi060_196073_object61514:head# 2024-09-20T22:20:49.736 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.737 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b09d0cb2:::benchmark_data_smithi060_196073_object532:head# 2024-09-20T22:20:49.737 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.737 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b09e3ad8:::benchmark_data_smithi060_196073_object34535:head# 2024-09-20T22:20:49.737 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.737 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b09e7020:::benchmark_data_smithi060_196073_object52637:head# 2024-09-20T22:20:49.737 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.738 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b09f32a3:::benchmark_data_smithi060_196073_object41650:head# 2024-09-20T22:20:49.738 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.738 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0a1250f:::benchmark_data_smithi060_196073_object19596:head# 2024-09-20T22:20:49.738 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.739 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0a35f39:::benchmark_data_smithi060_196073_object22189:head# 2024-09-20T22:20:49.739 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.739 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0a3ac73:::benchmark_data_smithi060_196073_object51096:head# 2024-09-20T22:20:49.739 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.739 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0a3cad0:::benchmark_data_smithi060_196073_object54364:head# 2024-09-20T22:20:49.740 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.740 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0a495b9:::benchmark_data_smithi060_196073_object39684:head# 2024-09-20T22:20:49.740 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.740 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0a59103:::benchmark_data_smithi060_196073_object35915:head# 2024-09-20T22:20:49.740 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.741 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0a898fb:::benchmark_data_smithi060_196073_object18285:head# 2024-09-20T22:20:49.741 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.741 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0a8d1ee:::benchmark_data_smithi060_196073_object9079:head# 2024-09-20T22:20:49.743 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.743 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:49 smithi067 ceph-mon[33511]: pgmap v956: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfilling, 3 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.9 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.18k op/s; 4067/242787 objects degraded (1.675%); 13432/242787 objects misplaced (5.532%); 42 MiB/s, 668 objects/s recovering 2024-09-20T22:20:49.743 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0ab7cdd:::benchmark_data_smithi060_196073_object28348:head# 2024-09-20T22:20:49.744 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.744 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0ac225a:::benchmark_data_smithi060_196073_object59363:head# 2024-09-20T22:20:49.758 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.759 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0ae2f17:::benchmark_data_smithi060_196073_object3332:head# 2024-09-20T22:20:49.759 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.759 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0ae41ec:::benchmark_data_smithi060_196073_object9966:head# 2024-09-20T22:20:49.759 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.760 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0b3b4ce:::benchmark_data_smithi060_196073_object39442:head# 2024-09-20T22:20:49.760 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.760 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0b45897:::benchmark_data_smithi060_196073_object8236:head# 2024-09-20T22:20:49.760 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.760 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0b464cc:::benchmark_data_smithi060_196073_object7048:head# 2024-09-20T22:20:49.760 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.760 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0b567a0:::benchmark_data_smithi060_196073_object13647:head# 2024-09-20T22:20:49.760 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.760 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0ba4e27:::benchmark_data_smithi060_196073_object18189:head# 2024-09-20T22:20:49.761 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.764 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0baed22:::benchmark_data_smithi060_196073_object55252:head# 2024-09-20T22:20:49.764 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.764 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0bcbb63:::benchmark_data_smithi060_196073_object23314:head# 2024-09-20T22:20:49.764 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.764 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0bde123:::benchmark_data_smithi060_196073_object40682:head# 2024-09-20T22:20:49.764 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.764 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0beaa9a:::benchmark_data_smithi060_196073_object26864:head# 2024-09-20T22:20:49.764 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.764 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0c17288:::benchmark_data_smithi060_196073_object52827:head# 2024-09-20T22:20:49.764 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.764 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0c4debd:::benchmark_data_smithi060_196073_object32372:head# 2024-09-20T22:20:49.765 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.765 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0c6a001:::benchmark_data_smithi060_196073_object9634:head# 2024-09-20T22:20:49.765 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.765 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0c86962:::benchmark_data_smithi060_196073_object52385:head# 2024-09-20T22:20:49.765 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.765 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0c8862d:::benchmark_data_smithi060_196073_object37135:head# 2024-09-20T22:20:49.765 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.766 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0c8dfde:::benchmark_data_smithi060_196073_object27140:head# 2024-09-20T22:20:49.766 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.766 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0cb6d49:::benchmark_data_smithi060_196073_object22288:head# 2024-09-20T22:20:49.766 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.767 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0cda6ab:::benchmark_data_smithi060_196073_object37585:head# 2024-09-20T22:20:49.767 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.767 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0cdab77:::benchmark_data_smithi060_196073_object38565:head# 2024-09-20T22:20:49.767 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.767 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0d06a37:::benchmark_data_smithi060_196073_object55018:head# 2024-09-20T22:20:49.767 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.767 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0d0d458:::benchmark_data_smithi060_196073_object34659:head# 2024-09-20T22:20:49.767 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.768 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0d15e27:::benchmark_data_smithi060_196073_object47147:head# 2024-09-20T22:20:49.768 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.768 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0d20b6d:::benchmark_data_smithi060_196073_object26695:head# 2024-09-20T22:20:49.768 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.768 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0d235b0:::benchmark_data_smithi060_196073_object43425:head# 2024-09-20T22:20:49.768 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.769 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0d31b13:::benchmark_data_smithi060_196073_object17223:head# 2024-09-20T22:20:49.769 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.769 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0d3352a:::benchmark_data_smithi060_196073_object29404:head# 2024-09-20T22:20:49.769 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.770 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0d39aba:::benchmark_data_smithi060_196073_object19192:head# 2024-09-20T22:20:49.770 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.770 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0d4474f:::benchmark_data_smithi060_196073_object59456:head# 2024-09-20T22:20:49.770 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.770 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0d4e53c:::benchmark_data_smithi060_196073_object25040:head# 2024-09-20T22:20:49.770 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.770 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0d66072:::benchmark_data_smithi060_196073_object50354:head# 2024-09-20T22:20:49.771 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.771 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0d7db4b:::benchmark_data_smithi060_196073_object47642:head# 2024-09-20T22:20:49.771 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.774 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0db2528:::benchmark_data_smithi060_196073_object59116:head# 2024-09-20T22:20:49.774 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.775 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0dba1a1:::benchmark_data_smithi060_196073_object41489:head# 2024-09-20T22:20:49.775 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.775 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0dc6e88:::benchmark_data_smithi060_196073_object41622:head# 2024-09-20T22:20:49.775 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.775 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0dd6f80:::benchmark_data_smithi060_196073_object29548:head# 2024-09-20T22:20:49.775 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.775 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0e0e7b3:::benchmark_data_smithi060_196073_object1012:head# 2024-09-20T22:20:49.775 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.775 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0e153b2:::benchmark_data_smithi060_196073_object24083:head# 2024-09-20T22:20:49.775 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.775 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0e165e5:::benchmark_data_smithi060_196073_object25633:head# 2024-09-20T22:20:49.775 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.775 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0e1f12e:::benchmark_data_smithi060_196073_object11000:head# 2024-09-20T22:20:49.775 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.775 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0e22439:::benchmark_data_smithi060_196073_object30966:head# 2024-09-20T22:20:49.776 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.776 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0e31d8f:::benchmark_data_smithi060_196073_object46151:head# 2024-09-20T22:20:49.776 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.776 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0e325fa:::benchmark_data_smithi060_196073_object60717:head# 2024-09-20T22:20:49.776 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.778 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0e5865f:::benchmark_data_smithi060_196073_object21920:head# 2024-09-20T22:20:49.778 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.778 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0e58ac5:::benchmark_data_smithi060_196073_object17667:head# 2024-09-20T22:20:49.778 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.778 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0e73d7b:::benchmark_data_smithi060_196073_object36188:head# 2024-09-20T22:20:49.778 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.778 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0e7ba07:::benchmark_data_smithi060_196073_object42039:head# 2024-09-20T22:20:49.778 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.778 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0e8c599:::benchmark_data_smithi060_196073_object19947:head# 2024-09-20T22:20:49.778 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.778 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0ea6925:::benchmark_data_smithi060_196073_object23393:head# 2024-09-20T22:20:49.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:49 smithi060 ceph-mon[29694]: pgmap v956: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfilling, 3 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.9 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.18k op/s; 4067/242787 objects degraded (1.675%); 13432/242787 objects misplaced (5.532%); 42 MiB/s, 668 objects/s recovering 2024-09-20T22:20:49.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:49 smithi060 ceph-mon[40290]: pgmap v956: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfilling, 3 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.9 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.18k op/s; 4067/242787 objects degraded (1.675%); 13432/242787 objects misplaced (5.532%); 42 MiB/s, 668 objects/s recovering 2024-09-20T22:20:49.786 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:49.808 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.808 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0ede2eb:::benchmark_data_smithi060_196073_object22931:head# 2024-09-20T22:20:49.808 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.809 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0edeceb:::benchmark_data_smithi060_196073_object47445:head# 2024-09-20T22:20:49.809 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.811 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0ee3fff:::benchmark_data_smithi060_196073_object39894:head# 2024-09-20T22:20:49.811 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.811 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0efa1fa:::benchmark_data_smithi060_196073_object56887:head# 2024-09-20T22:20:49.824 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:49.825 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:49.825 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:49.840 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.841 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0f2db78:::benchmark_data_smithi060_196073_object54278:head# 2024-09-20T22:20:49.841 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.842 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0f585ac:::benchmark_data_smithi060_196073_object10683:head# 2024-09-20T22:20:49.842 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.842 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0f872eb:::benchmark_data_smithi060_196073_object53104:head# 2024-09-20T22:20:49.842 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.842 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0faff0d:::benchmark_data_smithi060_196073_object40291:head# 2024-09-20T22:20:49.852 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.852 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0fcb5de:::benchmark_data_smithi060_196073_object47231:head# 2024-09-20T22:20:49.853 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.853 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0fe322a:::benchmark_data_smithi060_196073_object19133:head# 2024-09-20T22:20:49.853 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.853 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0feaba1:::benchmark_data_smithi060_196073_object28156:head# 2024-09-20T22:20:49.853 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.854 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0ff1ae2:::benchmark_data_smithi060_196073_object33648:head# 2024-09-20T22:20:49.854 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.854 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0ff1c5a:::benchmark_data_smithi060_196073_object4009:head# 2024-09-20T22:20:49.854 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.855 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b0ff5ba4:::benchmark_data_smithi060_196073_object40582:head# 2024-09-20T22:20:49.855 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.855 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b100472d:::benchmark_data_smithi060_196073_object53569:head# 2024-09-20T22:20:49.855 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.856 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1010683:::benchmark_data_smithi060_196073_object11247:head# 2024-09-20T22:20:49.856 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.856 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b101c32b:::benchmark_data_smithi060_196073_object45548:head# 2024-09-20T22:20:49.856 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.856 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b104ee61:::benchmark_data_smithi060_196073_object49883:head# 2024-09-20T22:20:49.856 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.857 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b106c2fa:::benchmark_data_smithi060_196073_object12322:head# 2024-09-20T22:20:49.857 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.857 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b107a0a9:::benchmark_data_smithi060_196073_object25345:head# 2024-09-20T22:20:49.857 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.858 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b108273b:::benchmark_data_smithi060_196073_object36141:head# 2024-09-20T22:20:49.858 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.858 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b10887fd:::benchmark_data_smithi060_196073_object51280:head# 2024-09-20T22:20:49.858 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.859 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b108a653:::benchmark_data_smithi060_196073_object11846:head# 2024-09-20T22:20:49.859 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.859 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1090eef:::benchmark_data_smithi060_196073_object24964:head# 2024-09-20T22:20:49.867 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.868 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b10aa44e:::benchmark_data_smithi060_196073_object47968:head# 2024-09-20T22:20:49.868 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.868 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b110e90b:::benchmark_data_smithi060_196073_object878:head# 2024-09-20T22:20:49.868 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.869 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1110f6e:::benchmark_data_smithi060_196073_object1026:head# 2024-09-20T22:20:49.869 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.869 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b111af73:::benchmark_data_smithi060_196073_object42542:head# 2024-09-20T22:20:49.869 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.870 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b111e1f9:::benchmark_data_smithi060_196073_object26066:head# 2024-09-20T22:20:49.870 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.870 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1124668:::benchmark_data_smithi060_196073_object15523:head# 2024-09-20T22:20:49.870 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.870 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b11573ca:::benchmark_data_smithi060_196073_object22922:head# 2024-09-20T22:20:49.870 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.871 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1160d6e:::benchmark_data_smithi060_196073_object31460:head# 2024-09-20T22:20:49.871 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.871 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1160fc3:::benchmark_data_smithi060_196073_object61184:head# 2024-09-20T22:20:49.871 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.872 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b11622c7:::benchmark_data_smithi060_196073_object7023:head# 2024-09-20T22:20:49.872 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.872 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1184cb1:::benchmark_data_smithi060_196073_object18970:head# 2024-09-20T22:20:49.872 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.872 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b119b825:::benchmark_data_smithi060_196073_object54373:head# 2024-09-20T22:20:49.873 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.873 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b11b31c6:::benchmark_data_smithi060_196073_object59545:head# 2024-09-20T22:20:49.873 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.873 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b11bf3ba:::benchmark_data_smithi060_196073_object51482:head# 2024-09-20T22:20:49.874 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.874 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b11c04a6:::benchmark_data_smithi060_196073_object35963:head# 2024-09-20T22:20:49.874 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.874 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b11e3f9b:::benchmark_data_smithi060_196073_object60770:head# 2024-09-20T22:20:49.874 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.875 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b11f10a4:::benchmark_data_smithi060_196073_object46272:head# 2024-09-20T22:20:49.875 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.875 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1254199:::benchmark_data_smithi060_196073_object51984:head# 2024-09-20T22:20:49.875 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.876 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b126b6c3:::benchmark_data_smithi060_196073_object14650:head# 2024-09-20T22:20:49.876 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.876 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b127a0a2:::benchmark_data_smithi060_196073_object25854:head# 2024-09-20T22:20:49.876 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.876 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b12c95ba:::benchmark_data_smithi060_196073_object59990:head# 2024-09-20T22:20:49.876 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.877 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1305039:::benchmark_data_smithi060_196073_object43136:head# 2024-09-20T22:20:49.877 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.877 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1306b70:::benchmark_data_smithi060_196073_object36231:head# 2024-09-20T22:20:49.877 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.879 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1327604:::benchmark_data_smithi060_196073_object12016:head# 2024-09-20T22:20:49.879 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.879 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b132c597:::benchmark_data_smithi060_196073_object12076:head# 2024-09-20T22:20:49.879 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.879 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b13402a2:::benchmark_data_smithi060_196073_object35009:head# 2024-09-20T22:20:49.879 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.880 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1348b69:::benchmark_data_smithi060_196073_object40473:head# 2024-09-20T22:20:49.881 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.881 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b134b1a2:::benchmark_data_smithi060_196073_object51484:head# 2024-09-20T22:20:49.881 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.881 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b134d217:::benchmark_data_smithi060_196073_object53306:head# 2024-09-20T22:20:49.881 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.881 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b136d7fb:::benchmark_data_smithi060_196073_object40531:head# 2024-09-20T22:20:49.881 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.881 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1392b64:::benchmark_data_smithi060_196073_object48207:head# 2024-09-20T22:20:49.881 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.882 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b13963d3:::benchmark_data_smithi060_196073_object61127:head# 2024-09-20T22:20:49.882 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.882 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b139f2f3:::benchmark_data_smithi060_196073_object34340:head# 2024-09-20T22:20:49.882 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.882 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b13bbb99:::benchmark_data_smithi060_196073_object1972:head# 2024-09-20T22:20:49.882 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.883 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b13c729d:::benchmark_data_smithi060_196073_object54958:head# 2024-09-20T22:20:49.883 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.883 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b13c8a4f:::benchmark_data_smithi060_196073_object48535:head# 2024-09-20T22:20:49.883 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.883 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b13fe383:::benchmark_data_smithi060_196073_object36265:head# 2024-09-20T22:20:49.884 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.884 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b13ffde4:::benchmark_data_smithi060_196073_object24285:head# 2024-09-20T22:20:49.884 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.885 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b140c696:::benchmark_data_smithi060_196073_object22776:head# 2024-09-20T22:20:49.885 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.885 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b14334f6:::benchmark_data_smithi060_196073_object57748:head# 2024-09-20T22:20:49.885 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.885 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1434cb3:::benchmark_data_smithi060_196073_object41363:head# 2024-09-20T22:20:49.886 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.886 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b147f949:::benchmark_data_smithi060_196073_object572:head# 2024-09-20T22:20:49.886 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.886 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1496c2c:::benchmark_data_smithi060_196073_object13638:head# 2024-09-20T22:20:49.886 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.887 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b14a1c0b:::benchmark_data_smithi060_196073_object10868:head# 2024-09-20T22:20:49.887 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.887 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b14a5603:::benchmark_data_smithi060_196073_object17653:head# 2024-09-20T22:20:49.887 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.887 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b14a7fb8:::benchmark_data_smithi060_196073_object57751:head# 2024-09-20T22:20:49.887 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.888 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b14a8220:::benchmark_data_smithi060_196073_object5455:head# 2024-09-20T22:20:49.888 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.888 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b14a985f:::benchmark_data_smithi060_196073_object9344:head# 2024-09-20T22:20:49.888 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.888 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b14b13f7:::benchmark_data_smithi060_196073_object2783:head# 2024-09-20T22:20:49.889 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.889 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b14b5208:::benchmark_data_smithi060_196073_object39123:head# 2024-09-20T22:20:49.889 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.889 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b14c2d08:::benchmark_data_smithi060_196073_object23809:head# 2024-09-20T22:20:49.889 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.890 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b14c4d5f:::benchmark_data_smithi060_196073_object41980:head# 2024-09-20T22:20:49.890 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.890 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b14da5cd:::benchmark_data_smithi060_196073_object39614:head# 2024-09-20T22:20:49.890 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.891 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b14e3d77:::benchmark_data_smithi060_196073_object30892:head# 2024-09-20T22:20:49.891 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.891 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b14e487c:::benchmark_data_smithi060_196073_object42723:head# 2024-09-20T22:20:49.891 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.892 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b14ecc31:::benchmark_data_smithi060_196073_object49362:head# 2024-09-20T22:20:49.892 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.892 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b14f0caf:::benchmark_data_smithi060_196073_object17359:head# 2024-09-20T22:20:49.892 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.892 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b14fa24a:::benchmark_data_smithi060_196073_object35102:head# 2024-09-20T22:20:49.893 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.893 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b14fed2e:::benchmark_data_smithi060_196073_object4207:head# 2024-09-20T22:20:49.893 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.893 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b150658b:::benchmark_data_smithi060_196073_object8061:head# 2024-09-20T22:20:49.894 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.894 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b15178e3:::benchmark_data_smithi060_196073_object38195:head# 2024-09-20T22:20:49.898 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.898 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b153176a:::benchmark_data_smithi060_196073_object31396:head# 2024-09-20T22:20:49.898 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.899 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b15933e6:::benchmark_data_smithi060_196073_object17298:head# 2024-09-20T22:20:49.899 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.899 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b15a91aa:::benchmark_data_smithi060_196073_object45751:head# 2024-09-20T22:20:49.899 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.900 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b15ab66a:::benchmark_data_smithi060_196073_object7429:head# 2024-09-20T22:20:49.900 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.900 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b15b2805:::benchmark_data_smithi060_196073_object49689:head# 2024-09-20T22:20:49.900 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.901 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b15c370d:::benchmark_data_smithi060_196073_object27366:head# 2024-09-20T22:20:49.901 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.901 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b15c3d3f:::benchmark_data_smithi060_196073_object39144:head# 2024-09-20T22:20:49.901 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.901 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b15d5543:::benchmark_data_smithi060_196073_object37633:head# 2024-09-20T22:20:49.902 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.902 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b15d65ac:::benchmark_data_smithi060_196073_object18434:head# 2024-09-20T22:20:49.902 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.904 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b15ddda3:::benchmark_data_smithi060_196073_object21715:head# 2024-09-20T22:20:49.904 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.904 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b15df093:::benchmark_data_smithi060_196073_object1946:head# 2024-09-20T22:20:49.904 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.904 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b15e15af:::benchmark_data_smithi060_196073_object52012:head# 2024-09-20T22:20:49.905 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.905 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b15f29de:::benchmark_data_smithi060_196073_object27742:head# 2024-09-20T22:20:49.905 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.905 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b15f71f1:::benchmark_data_smithi060_196073_object40642:head# 2024-09-20T22:20:49.905 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.905 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b15fe9b9:::benchmark_data_smithi060_196073_object52157:head# 2024-09-20T22:20:49.905 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.905 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1600319:::benchmark_data_smithi060_196073_object26737:head# 2024-09-20T22:20:49.905 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.907 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1616c96:::benchmark_data_smithi060_196073_object52965:head# 2024-09-20T22:20:49.907 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.907 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b16393cb:::benchmark_data_smithi060_196073_object50023:head# 2024-09-20T22:20:49.907 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.907 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b163cea2:::benchmark_data_smithi060_196073_object19401:head# 2024-09-20T22:20:49.907 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.907 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b16455e8:::benchmark_data_smithi060_196073_object31502:head# 2024-09-20T22:20:49.907 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.908 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1648b65:::benchmark_data_smithi060_196073_object9297:head# 2024-09-20T22:20:49.908 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.908 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b164ef76:::benchmark_data_smithi060_196073_object7090:head# 2024-09-20T22:20:49.908 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.908 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1653ccf:::benchmark_data_smithi060_196073_object15713:head# 2024-09-20T22:20:49.908 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.909 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1654535:::benchmark_data_smithi060_196073_object51012:head# 2024-09-20T22:20:49.909 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.909 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b16630d8:::benchmark_data_smithi060_196073_object44360:head# 2024-09-20T22:20:49.909 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.909 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b167353f:::benchmark_data_smithi060_196073_object19081:head# 2024-09-20T22:20:49.910 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.910 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b167db41:::benchmark_data_smithi060_196073_object41129:head# 2024-09-20T22:20:49.910 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.910 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b167f240:::benchmark_data_smithi060_196073_object38269:head# 2024-09-20T22:20:49.910 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.910 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b16984a8:::benchmark_data_smithi060_196073_object47325:head# 2024-09-20T22:20:49.910 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.911 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b16a9979:::benchmark_data_smithi060_196073_object59103:head# 2024-09-20T22:20:49.911 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.911 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b16bf061:::benchmark_data_smithi060_196073_object3708:head# 2024-09-20T22:20:49.911 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.911 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b16c9371:::benchmark_data_smithi060_196073_object29226:head# 2024-09-20T22:20:49.911 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.912 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b16cf1e5:::benchmark_data_smithi060_196073_object15823:head# 2024-09-20T22:20:49.912 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.912 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b16efe1d:::benchmark_data_smithi060_196073_object16018:head# 2024-09-20T22:20:49.912 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.913 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1700a63:::benchmark_data_smithi060_196073_object31402:head# 2024-09-20T22:20:49.913 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.913 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1701562:::benchmark_data_smithi060_196073_object24350:head# 2024-09-20T22:20:49.913 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.913 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b17069e9:::benchmark_data_smithi060_196073_object172:head# 2024-09-20T22:20:49.913 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.913 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b171f41d:::benchmark_data_smithi060_196073_object3345:head# 2024-09-20T22:20:49.913 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.913 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1721c86:::benchmark_data_smithi060_196073_object984:head# 2024-09-20T22:20:49.913 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.914 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b175837a:::benchmark_data_smithi060_196073_object18169:head# 2024-09-20T22:20:49.914 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.914 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b17611ee:::benchmark_data_smithi060_196073_object29612:head# 2024-09-20T22:20:49.914 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.915 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b177fd6a:::benchmark_data_smithi060_196073_object19486:head# 2024-09-20T22:20:49.915 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.915 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1786850:::benchmark_data_smithi060_196073_object57158:head# 2024-09-20T22:20:49.916 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.916 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b178b1ab:::benchmark_data_smithi060_196073_object8699:head# 2024-09-20T22:20:49.916 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.916 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1790c3c:::benchmark_data_smithi060_196073_object17249:head# 2024-09-20T22:20:49.916 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.916 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b17976aa:::benchmark_data_smithi060_196073_object1741:head# 2024-09-20T22:20:49.916 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.917 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b17a0c69:::benchmark_data_smithi060_196073_object34700:head# 2024-09-20T22:20:49.917 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.917 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b17a5ccc:::benchmark_data_smithi060_196073_object7348:head# 2024-09-20T22:20:49.917 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.917 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b17c3352:::benchmark_data_smithi060_196073_object55256:head# 2024-09-20T22:20:49.917 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.917 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b17c3424:::benchmark_data_smithi060_196073_object7166:head# 2024-09-20T22:20:49.918 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.918 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b17dfd3c:::benchmark_data_smithi060_196073_object44881:head# 2024-09-20T22:20:49.918 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.919 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b17e37e2:::benchmark_data_smithi060_196073_object28690:head# 2024-09-20T22:20:49.919 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.919 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b17f6f78:::benchmark_data_smithi060_196073_object30674:head# 2024-09-20T22:20:49.919 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.919 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1814e18:::benchmark_data_smithi060_196073_object61310:head# 2024-09-20T22:20:49.919 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.920 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b18166ad:::benchmark_data_smithi060_196073_object9972:head# 2024-09-20T22:20:49.920 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.920 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1820f7f:::benchmark_data_smithi060_196073_object41739:head# 2024-09-20T22:20:49.920 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.920 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b18339d0:::benchmark_data_smithi060_196073_object14394:head# 2024-09-20T22:20:49.920 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.921 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1833be7:::benchmark_data_smithi060_196073_object9168:head# 2024-09-20T22:20:49.921 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.921 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b184a8d4:::benchmark_data_smithi060_196073_object45331:head# 2024-09-20T22:20:49.921 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.922 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b18599e4:::benchmark_data_smithi060_196073_object59007:head# 2024-09-20T22:20:49.922 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.922 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b186bd67:::benchmark_data_smithi060_196073_object39468:head# 2024-09-20T22:20:49.922 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.922 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1885582:::benchmark_data_smithi060_196073_object31116:head# 2024-09-20T22:20:49.922 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.923 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b188d1ca:::benchmark_data_smithi060_196073_object9925:head# 2024-09-20T22:20:49.923 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.923 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1891c7f:::benchmark_data_smithi060_196073_object54192:head# 2024-09-20T22:20:49.923 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.923 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b189a5f3:::benchmark_data_smithi060_196073_object53520:head# 2024-09-20T22:20:49.923 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.923 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b189f82e:::benchmark_data_smithi060_196073_object37862:head# 2024-09-20T22:20:49.926 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:49.926 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.927 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b18ac185:::benchmark_data_smithi060_196073_object44799:head# 2024-09-20T22:20:49.929 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.929 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b18add0d:::benchmark_data_smithi060_196073_object35555:head# 2024-09-20T22:20:49.929 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.930 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b18ce425:::benchmark_data_smithi060_196073_object1223:head# 2024-09-20T22:20:49.930 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.930 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b18d539a:::benchmark_data_smithi060_196073_object28064:head# 2024-09-20T22:20:49.930 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.930 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b18f622f:::benchmark_data_smithi060_196073_object6777:head# 2024-09-20T22:20:49.931 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.931 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1903d9f:::benchmark_data_smithi060_196073_object3219:head# 2024-09-20T22:20:49.932 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.932 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b190539a:::benchmark_data_smithi060_196073_object35611:head# 2024-09-20T22:20:49.932 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.932 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b192a831:::benchmark_data_smithi060_196073_object11287:head# 2024-09-20T22:20:49.932 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.932 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b19432c6:::benchmark_data_smithi060_196073_object6855:head# 2024-09-20T22:20:49.933 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T22:20:49.934 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.935 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b19614be:::benchmark_data_smithi060_196073_object43347:head# 2024-09-20T22:20:49.935 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.935 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1968de8:::benchmark_data_smithi060_196073_object55470:head# 2024-09-20T22:20:49.935 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.936 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b198e6bb:::benchmark_data_smithi060_196073_object52355:head# 2024-09-20T22:20:49.936 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.936 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b19967ad:::benchmark_data_smithi060_196073_object11291:head# 2024-09-20T22:20:49.939 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.939 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b19a5d7b:::benchmark_data_smithi060_196073_object54770:head# 2024-09-20T22:20:49.939 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.940 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b19aa1c6:::benchmark_data_smithi060_196073_object23160:head# 2024-09-20T22:20:49.940 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.941 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b19c61c6:::benchmark_data_smithi060_196073_object23493:head# 2024-09-20T22:20:49.941 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.941 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b19c85cc:::benchmark_data_smithi060_196073_object3455:head# 2024-09-20T22:20:49.943 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.945 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b19de50b:::benchmark_data_smithi060_196073_object8191:head# 2024-09-20T22:20:49.945 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.945 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b19e238a:::benchmark_data_smithi060_196073_object27878:head# 2024-09-20T22:20:49.945 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.945 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1a03887:::benchmark_data_smithi060_196073_object13283:head# 2024-09-20T22:20:49.945 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.945 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1a03bb4:::benchmark_data_smithi060_196073_object11605:head# 2024-09-20T22:20:49.950 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.951 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1a118e4:::benchmark_data_smithi060_196073_object49878:head# 2024-09-20T22:20:49.951 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.951 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1a1caac:::benchmark_data_smithi060_196073_object31750:head# 2024-09-20T22:20:49.951 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.951 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1a227eb:::benchmark_data_smithi060_196073_object7064:head# 2024-09-20T22:20:49.951 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.952 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1a23260:::benchmark_data_smithi060_196073_object25868:head# 2024-09-20T22:20:49.958 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.958 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1a43f70:::benchmark_data_smithi060_196073_object51972:head# 2024-09-20T22:20:49.958 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.959 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:49.960 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1a44009:::benchmark_data_smithi060_196073_object59233:head# 2024-09-20T22:20:49.960 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.960 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1a459ea:::benchmark_data_smithi060_196073_object4214:head# 2024-09-20T22:20:49.960 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.960 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1a61082:::benchmark_data_smithi060_196073_object31147:head# 2024-09-20T22:20:49.960 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:49.961 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:49.968 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.968 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1a67f7b:::benchmark_data_smithi060_196073_object53154:head# 2024-09-20T22:20:49.968 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.968 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1a721d0:::benchmark_data_smithi060_196073_object42958:head# 2024-09-20T22:20:49.968 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.969 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1a788fc:::benchmark_data_smithi060_196073_object43478:head# 2024-09-20T22:20:49.969 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.969 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1a7e102:::benchmark_data_smithi060_196073_object42705:head# 2024-09-20T22:20:49.973 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.973 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1aa9c1c:::benchmark_data_smithi060_196073_object55570:head# 2024-09-20T22:20:49.973 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.974 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1aab4f9:::benchmark_data_smithi060_196073_object32181:head# 2024-09-20T22:20:49.974 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.974 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1adaad9:::benchmark_data_smithi060_196073_object13472:head# 2024-09-20T22:20:49.974 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.975 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1ae4663:::benchmark_data_smithi060_196073_object9570:head# 2024-09-20T22:20:49.976 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.976 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1aee09f:::benchmark_data_smithi060_196073_object53362:head# 2024-09-20T22:20:49.977 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.977 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1b0522a:::benchmark_data_smithi060_196073_object41427:head# 2024-09-20T22:20:49.977 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.978 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1b0a5b5:::benchmark_data_smithi060_196073_object42259:head# 2024-09-20T22:20:49.978 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.978 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1b0f6bc:::benchmark_data_smithi060_196073_object29401:head# 2024-09-20T22:20:49.983 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.984 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1b207fc:::benchmark_data_smithi060_196073_object34257:head# 2024-09-20T22:20:49.984 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.984 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1b532a1:::benchmark_data_smithi060_196073_object48477:head# 2024-09-20T22:20:49.985 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.985 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1b54768:::benchmark_data_smithi060_196073_object2564:head# 2024-09-20T22:20:49.985 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.985 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1b5d7e8:::benchmark_data_smithi060_196073_object52331:head# 2024-09-20T22:20:49.991 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.992 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1b6c68c:::benchmark_data_smithi060_196073_object52017:head# 2024-09-20T22:20:49.992 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.993 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1b6f222:::benchmark_data_smithi060_196073_object18597:head# 2024-09-20T22:20:49.993 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.993 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1b7f6bd:::benchmark_data_smithi060_196073_object30207:head# 2024-09-20T22:20:49.993 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:49.993 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1b80d52:::benchmark_data_smithi060_196073_object33795:head# 2024-09-20T22:20:49.999 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.000 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1b87cfe:::benchmark_data_smithi060_196073_object22447:head# 2024-09-20T22:20:50.000 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.000 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1b919bf:::benchmark_data_smithi060_196073_object1335:head# 2024-09-20T22:20:50.000 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.001 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1b95026:::benchmark_data_smithi060_196073_object56461:head# 2024-09-20T22:20:50.001 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.001 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1baaf1a:::benchmark_data_smithi060_196073_object60645:head# 2024-09-20T22:20:50.005 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.006 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1bea4e3:::benchmark_data_smithi060_196073_object34397:head# 2024-09-20T22:20:50.006 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.006 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1bf4351:::benchmark_data_smithi060_196073_object25097:head# 2024-09-20T22:20:50.007 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.007 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1c0045e:::benchmark_data_smithi060_196073_object11237:head# 2024-09-20T22:20:50.012 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.013 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1c137df:::benchmark_data_smithi060_196073_object22682:head# 2024-09-20T22:20:50.014 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.014 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1c1be94:::benchmark_data_smithi060_196073_object6615:head# 2024-09-20T22:20:50.014 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.014 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1c1d598:::benchmark_data_smithi060_196073_object30428:head# 2024-09-20T22:20:50.014 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.014 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1c5a945:::benchmark_data_smithi060_196073_object46141:head# 2024-09-20T22:20:50.025 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.025 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1c5d57a:::benchmark_data_smithi060_196073_object46063:head# 2024-09-20T22:20:50.025 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.025 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1c66ab2:::benchmark_data_smithi060_196073_object6252:head# 2024-09-20T22:20:50.026 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.026 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1c7399b:::benchmark_data_smithi060_196073_object9444:head# 2024-09-20T22:20:50.026 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.026 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1cc22f2:::benchmark_data_smithi060_196073_object38386:head# 2024-09-20T22:20:50.026 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.027 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1ccb637:::benchmark_data_smithi060_196073_object45236:head# 2024-09-20T22:20:50.027 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.028 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1cd8473:::benchmark_data_smithi060_196073_object59325:head# 2024-09-20T22:20:50.028 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.028 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1cddb43:::benchmark_data_smithi060_196073_object7980:head# 2024-09-20T22:20:50.028 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.028 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1cfbf83:::benchmark_data_smithi060_196073_object22920:head# 2024-09-20T22:20:50.034 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.034 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1d129eb:::benchmark_data_smithi060_196073_object13887:head# 2024-09-20T22:20:50.035 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.035 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1d173b3:::benchmark_data_smithi060_196073_object20016:head# 2024-09-20T22:20:50.035 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.036 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1d17f30:::benchmark_data_smithi060_196073_object4416:head# 2024-09-20T22:20:50.036 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.036 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1d19742:::benchmark_data_smithi060_196073_object24941:head# 2024-09-20T22:20:50.059 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.059 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1d1d8e3:::benchmark_data_smithi060_196073_object14596:head# 2024-09-20T22:20:50.059 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.059 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1d200a9:::benchmark_data_smithi060_196073_object59419:head# 2024-09-20T22:20:50.060 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.060 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1d44be9:::benchmark_data_smithi060_196073_object35648:head# 2024-09-20T22:20:50.061 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.061 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1d56bd0:::benchmark_data_smithi060_196073_object49131:head# 2024-09-20T22:20:50.062 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.062 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1d5eec2:::benchmark_data_smithi060_196073_object49591:head# 2024-09-20T22:20:50.062 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.062 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1d6a857:::benchmark_data_smithi060_196073_object23104:head# 2024-09-20T22:20:50.063 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:50.071 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.071 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1d7e05a:::benchmark_data_smithi060_196073_object51217:head# 2024-09-20T22:20:50.071 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.071 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1d7f90e:::benchmark_data_smithi060_196073_object40552:head# 2024-09-20T22:20:50.072 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.072 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1d84b70:::benchmark_data_smithi060_196073_object57038:head# 2024-09-20T22:20:50.072 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.072 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1d891ec:::benchmark_data_smithi060_196073_object15846:head# 2024-09-20T22:20:50.072 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.072 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1d9a595:::benchmark_data_smithi060_196073_object26678:head# 2024-09-20T22:20:50.073 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.073 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1d9ec61:::benchmark_data_smithi060_196073_object29331:head# 2024-09-20T22:20:50.073 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.073 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1da8d74:::benchmark_data_smithi060_196073_object37325:head# 2024-09-20T22:20:50.073 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.073 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1daca41:::benchmark_data_smithi060_196073_object54544:head# 2024-09-20T22:20:50.073 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.073 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1db0435:::benchmark_data_smithi060_196073_object26473:head# 2024-09-20T22:20:50.073 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.073 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1dc5dda:::benchmark_data_smithi060_196073_object12538:head# 2024-09-20T22:20:50.107 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.107 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1de9179:::benchmark_data_smithi060_196073_object33246:head# 2024-09-20T22:20:50.108 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.108 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1df0a95:::benchmark_data_smithi060_196073_object6389:head# 2024-09-20T22:20:50.108 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.108 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1dfd7a6:::benchmark_data_smithi060_196073_object5486:head# 2024-09-20T22:20:50.108 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.108 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1e1fabd:::benchmark_data_smithi060_196073_object29662:head# 2024-09-20T22:20:50.111 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:50.112 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:50.112 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:50.133 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.133 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1e65ad1:::benchmark_data_smithi060_196073_object60405:head# 2024-09-20T22:20:50.133 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.133 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1e6952c:::benchmark_data_smithi060_196073_object34924:head# 2024-09-20T22:20:50.134 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.134 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1e72a45:::benchmark_data_smithi060_196073_object45973:head# 2024-09-20T22:20:50.134 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.134 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1e8634a:::benchmark_data_smithi060_196073_object12591:head# 2024-09-20T22:20:50.138 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.139 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1ea119f:::benchmark_data_smithi060_196073_object52590:head# 2024-09-20T22:20:50.139 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.139 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1ec91eb:::benchmark_data_smithi060_196073_object24222:head# 2024-09-20T22:20:50.139 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.140 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1ecf8ae:::benchmark_data_smithi060_196073_object2354:head# 2024-09-20T22:20:50.140 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.140 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1ed4ad4:::benchmark_data_smithi060_196073_object119:head# 2024-09-20T22:20:50.147 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.147 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1ed9bfb:::benchmark_data_smithi060_196073_object60666:head# 2024-09-20T22:20:50.147 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.147 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1edbdbb:::benchmark_data_smithi060_196073_object28412:head# 2024-09-20T22:20:50.148 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.148 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1f06df8:::benchmark_data_smithi060_196073_object31446:head# 2024-09-20T22:20:50.148 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.148 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1f17865:::benchmark_data_smithi060_196073_object11361:head# 2024-09-20T22:20:50.157 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.157 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1f22190:::benchmark_data_smithi060_196073_object34572:head# 2024-09-20T22:20:50.157 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.157 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1f2fae9:::benchmark_data_smithi060_196073_object50113:head# 2024-09-20T22:20:50.158 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.158 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1f36a04:::benchmark_data_smithi060_196073_object13079:head# 2024-09-20T22:20:50.158 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.158 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1f3f8ba:::benchmark_data_smithi060_196073_object30183:head# 2024-09-20T22:20:50.160 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.160 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1f41e7f:::benchmark_data_smithi060_196073_object15184:head# 2024-09-20T22:20:50.160 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.161 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1f4d0fb:::benchmark_data_smithi060_196073_object7869:head# 2024-09-20T22:20:50.172 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.172 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1f6ab04:::benchmark_data_smithi060_196073_object44776:head# 2024-09-20T22:20:50.172 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.173 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1f781be:::benchmark_data_smithi060_196073_object14328:head# 2024-09-20T22:20:50.173 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.173 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1f9793e:::benchmark_data_smithi060_196073_object11339:head# 2024-09-20T22:20:50.173 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.173 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1fae681:::benchmark_data_smithi060_196073_object3377:head# 2024-09-20T22:20:50.175 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.175 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1fcb4e2:::benchmark_data_smithi060_196073_object45153:head# 2024-09-20T22:20:50.175 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.176 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1fcf4eb:::benchmark_data_smithi060_196073_object916:head# 2024-09-20T22:20:50.176 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.176 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1fe8778:::benchmark_data_smithi060_196073_object36232:head# 2024-09-20T22:20:50.176 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.177 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b1ff6e93:::benchmark_data_smithi060_196073_object55783:head# 2024-09-20T22:20:50.177 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.177 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b200413a:::benchmark_data_smithi060_196073_object7741:head# 2024-09-20T22:20:50.177 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.178 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2010394:::benchmark_data_smithi060_196073_object17992:head# 2024-09-20T22:20:50.213 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:50.249 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:50.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:50.250 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:50.285 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.285 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b20171b0:::benchmark_data_smithi060_196073_object803:head# 2024-09-20T22:20:50.285 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.286 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b207fb42:::benchmark_data_smithi060_196073_object29677:head# 2024-09-20T22:20:50.286 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.287 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b20868ce:::benchmark_data_smithi060_196073_object58754:head# 2024-09-20T22:20:50.287 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.287 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2098f46:::benchmark_data_smithi060_196073_object25424:head# 2024-09-20T22:20:50.293 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.294 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b20b4713:::benchmark_data_smithi060_196073_object25550:head# 2024-09-20T22:20:50.294 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.294 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b20bcdad:::benchmark_data_smithi060_196073_object19672:head# 2024-09-20T22:20:50.295 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.296 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b20c7f2e:::benchmark_data_smithi060_196073_object45777:head# 2024-09-20T22:20:50.299 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.299 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b20c95a2:::benchmark_data_smithi060_196073_object28775:head# 2024-09-20T22:20:50.299 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.300 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b20c9f3b:::benchmark_data_smithi060_196073_object54987:head# 2024-09-20T22:20:50.346 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.347 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b20d401b:::benchmark_data_smithi060_196073_object5917:head# 2024-09-20T22:20:50.347 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.347 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b20e0e74:::benchmark_data_smithi060_196073_object4828:head# 2024-09-20T22:20:50.347 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.348 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b20ef432:::benchmark_data_smithi060_196073_object10130:head# 2024-09-20T22:20:50.348 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.348 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2132318:::benchmark_data_smithi060_196073_object9695:head# 2024-09-20T22:20:50.351 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:50.355 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.356 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2156e07:::benchmark_data_smithi060_196073_object1018:head# 2024-09-20T22:20:50.356 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.356 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2179054:::benchmark_data_smithi060_196073_object37982:head# 2024-09-20T22:20:50.391 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:50.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:50.392 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:50.398 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:20:50.493 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:50.536 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:50 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:20:50.536 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:50 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:20:50.538 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:50.539 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:50.539 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:50.557 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.557 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b21a2524:::benchmark_data_smithi060_196073_object9008:head# 2024-09-20T22:20:50.557 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.557 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b21bfcd7:::benchmark_data_smithi060_196073_object14962:head# 2024-09-20T22:20:50.558 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.558 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b21dbc57:::benchmark_data_smithi060_196073_object45162:head# 2024-09-20T22:20:50.558 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.558 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b21dff35:::benchmark_data_smithi060_196073_object16919:head# 2024-09-20T22:20:50.599 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.600 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b21ed9a2:::benchmark_data_smithi060_196073_object31720:head# 2024-09-20T22:20:50.600 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.600 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b21ff9c8:::benchmark_data_smithi060_196073_object17922:head# 2024-09-20T22:20:50.600 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.601 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b220086b:::benchmark_data_smithi060_196073_object45263:head# 2024-09-20T22:20:50.601 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.601 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b22062a5:::benchmark_data_smithi060_196073_object34989:head# 2024-09-20T22:20:50.606 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.607 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b222f5d3:::benchmark_data_smithi060_196073_object30170:head# 2024-09-20T22:20:50.607 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.607 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b223bb9b:::benchmark_data_smithi060_196073_object2558:head# 2024-09-20T22:20:50.607 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.607 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b224d544:::benchmark_data_smithi060_196073_object8114:head# 2024-09-20T22:20:50.608 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.608 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b225549a:::benchmark_data_smithi060_196073_object44824:head# 2024-09-20T22:20:50.613 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.613 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b225a093:::benchmark_data_smithi060_196073_object25583:head# 2024-09-20T22:20:50.613 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.613 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b225ee06:::benchmark_data_smithi060_196073_object30095:head# 2024-09-20T22:20:50.614 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.614 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b225f674:::benchmark_data_smithi060_196073_object26602:head# 2024-09-20T22:20:50.614 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:50.614 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2299a88:::benchmark_data_smithi060_196073_object11487:head# 2024-09-20T22:20:50.641 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:50.675 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:50.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:50.677 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:50.719 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:20:50.720 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:20:50.761 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:20:50.762 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:20:50.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:50 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:20:50.762 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:50 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:20:50.762 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:50 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:20:50.762 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:50 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:20:50.778 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:50.794 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:20:50.794 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:20:50.817 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:50.818 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:50.818 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:50.834 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:20:50.834 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:20:50.834 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-20T22:20:50.869 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:20:50.869 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:20:50.904 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:20:50.905 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:20:50.919 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:50.942 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:20:50.942 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:20:50.961 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:50.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:50.962 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:51.062 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:51.097 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.097 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b22a7d27:::benchmark_data_smithi060_196073_object59889:head# 2024-09-20T22:20:51.097 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.098 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b22b3de0:::benchmark_data_smithi060_196073_object1177:head# 2024-09-20T22:20:51.098 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.098 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b22bdeb9:::benchmark_data_smithi060_196073_object18346:head# 2024-09-20T22:20:51.098 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.098 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b22c151e:::benchmark_data_smithi060_196073_object41038:head# 2024-09-20T22:20:51.104 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:51.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:51.105 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:51.135 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.136 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b22c3275:::benchmark_data_smithi060_196073_object24413:head# 2024-09-20T22:20:51.136 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.136 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b22fe516:::benchmark_data_smithi060_196073_object44411:head# 2024-09-20T22:20:51.136 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.137 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b230139c:::benchmark_data_smithi060_196073_object22443:head# 2024-09-20T22:20:51.137 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.137 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b230a86b:::benchmark_data_smithi060_196073_object61258:head# 2024-09-20T22:20:51.165 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.165 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2321333:::benchmark_data_smithi060_196073_object21113:head# 2024-09-20T22:20:51.165 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.166 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b23349a8:::benchmark_data_smithi060_196073_object14120:head# 2024-09-20T22:20:51.166 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.166 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b233b2dd:::benchmark_data_smithi060_196073_object38385:head# 2024-09-20T22:20:51.166 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.166 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b234fea3:::benchmark_data_smithi060_196073_object33439:head# 2024-09-20T22:20:51.191 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.191 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b237c20c:::benchmark_data_smithi060_196073_object61111:head# 2024-09-20T22:20:51.191 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.191 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b237c8d9:::benchmark_data_smithi060_196073_object31549:head# 2024-09-20T22:20:51.191 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.192 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b238da49:::benchmark_data_smithi060_196073_object44865:head# 2024-09-20T22:20:51.192 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.192 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2390d8c:::benchmark_data_smithi060_196073_object44901:head# 2024-09-20T22:20:51.206 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:51.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:51.229 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:51.229 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:51.238 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.238 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b23932c2:::benchmark_data_smithi060_196073_object58324:head# 2024-09-20T22:20:51.238 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.239 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b23b32ba:::benchmark_data_smithi060_196073_object12283:head# 2024-09-20T22:20:51.239 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.239 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b23c1223:::benchmark_data_smithi060_196073_object35928:head# 2024-09-20T22:20:51.239 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.239 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b23c7fac:::benchmark_data_smithi060_196073_object44485:head# 2024-09-20T22:20:51.244 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.244 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b23eb2fb:::benchmark_data_smithi060_196073_object7829:head# 2024-09-20T22:20:51.244 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.245 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b23f96dd:::benchmark_data_smithi060_196073_object28366:head# 2024-09-20T22:20:51.246 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.246 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2401156:::benchmark_data_smithi060_196073_object10706:head# 2024-09-20T22:20:51.246 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.247 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2459f38:::benchmark_data_smithi060_196073_object17407:head# 2024-09-20T22:20:51.291 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.292 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b248aad0:::benchmark_data_smithi060_196073_object38353:head# 2024-09-20T22:20:51.292 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.292 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b24922a8:::benchmark_data_smithi060_196073_object44140:head# 2024-09-20T22:20:51.296 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.296 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b24a3add:::benchmark_data_smithi060_196073_object3364:head# 2024-09-20T22:20:51.296 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.296 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b24c8a43:::benchmark_data_smithi060_196073_object10113:head# 2024-09-20T22:20:51.302 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.303 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b24d09ea:::benchmark_data_smithi060_196073_object51238:head# 2024-09-20T22:20:51.303 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.303 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b24e1568:::benchmark_data_smithi060_196073_object57688:head# 2024-09-20T22:20:51.331 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:51.341 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.342 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2522356:::benchmark_data_smithi060_196073_object35283:head# 2024-09-20T22:20:51.342 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.342 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b25310ac:::benchmark_data_smithi060_196073_object34918:head# 2024-09-20T22:20:51.347 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.348 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2532cd7:::benchmark_data_smithi060_196073_object5862:head# 2024-09-20T22:20:51.348 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.348 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2569327:::benchmark_data_smithi060_196073_object22146:head# 2024-09-20T22:20:51.348 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.349 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2580b34:::benchmark_data_smithi060_196073_object34937:head# 2024-09-20T22:20:51.349 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.349 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2587369:::benchmark_data_smithi060_196073_object78:head# 2024-09-20T22:20:51.355 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:51.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:51.356 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:51.385 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.385 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b259934e:::benchmark_data_smithi060_196073_object10574:head# 2024-09-20T22:20:51.385 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.386 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b25c358c:::benchmark_data_smithi060_196073_object8554:head# 2024-09-20T22:20:51.408 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.408 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b25c7b0c:::benchmark_data_smithi060_196073_object27741:head# 2024-09-20T22:20:51.409 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.409 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b25d957b:::benchmark_data_smithi060_196073_object34863:head# 2024-09-20T22:20:51.409 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.409 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b25edecc:::benchmark_data_smithi060_196073_object32105:head# 2024-09-20T22:20:51.410 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.410 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b260d0ba:::benchmark_data_smithi060_196073_object33848:head# 2024-09-20T22:20:51.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:51.480 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.480 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2612666:::benchmark_data_smithi060_196073_object9513:head# 2024-09-20T22:20:51.480 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.481 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b262cdbe:::benchmark_data_smithi060_196073_object24916:head# 2024-09-20T22:20:51.481 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.481 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b263a882:::benchmark_data_smithi060_196073_object50653:head# 2024-09-20T22:20:51.482 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.482 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b26504d7:::benchmark_data_smithi060_196073_object60348:head# 2024-09-20T22:20:51.494 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:51.494 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:51.495 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:51.503 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.503 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b265fbd1:::benchmark_data_smithi060_196073_object2357:head# 2024-09-20T22:20:51.503 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.504 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2691af5:::benchmark_data_smithi060_196073_object24751:head# 2024-09-20T22:20:51.504 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.504 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b26a791a:::benchmark_data_smithi060_196073_object43281:head# 2024-09-20T22:20:51.504 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.505 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b26b3bfe:::benchmark_data_smithi060_196073_object37248:head# 2024-09-20T22:20:51.505 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.505 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b26be0f0:::benchmark_data_smithi060_196073_object23798:head# 2024-09-20T22:20:51.505 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.505 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b26c82d2:::benchmark_data_smithi060_196073_object464:head# 2024-09-20T22:20:51.506 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.506 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b26ebdae:::benchmark_data_smithi060_196073_object1195:head# 2024-09-20T22:20:51.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:51 smithi060 ceph-mon[29694]: pgmap v957: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfilling, 3 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.9 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 590 B/s rd, 0 B/s wr, 877 op/s; 4067/242787 objects degraded (1.675%); 13432/242787 objects misplaced (5.532%); 29 MiB/s, 456 objects/s recovering 2024-09-20T22:20:51.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:51 smithi060 ceph-mon[29694]: osdmap e503: 8 total, 7 up, 6 in 2024-09-20T22:20:51.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:51 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2811828921' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:20:51.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:20:51.536 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:20:51 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:20:51.288+0000 7fa5382c7640 -1 mgr get_metadata_python Requested missing service osd.4 2024-09-20T22:20:51.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:51 smithi060 ceph-mon[40290]: pgmap v957: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfilling, 3 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.9 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 590 B/s rd, 0 B/s wr, 877 op/s; 4067/242787 objects degraded (1.675%); 13432/242787 objects misplaced (5.532%); 29 MiB/s, 456 objects/s recovering 2024-09-20T22:20:51.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:51 smithi060 ceph-mon[40290]: osdmap e503: 8 total, 7 up, 6 in 2024-09-20T22:20:51.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:51 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2811828921' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:20:51.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:20:51.538 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.538 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b27076b2:::benchmark_data_smithi060_196073_object43811:head# 2024-09-20T22:20:51.539 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.539 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2736723:::benchmark_data_smithi060_196073_object27017:head# 2024-09-20T22:20:51.556 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.556 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2736ee6:::benchmark_data_smithi060_196073_object59635:head# 2024-09-20T22:20:51.556 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.557 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2737a43:::benchmark_data_smithi060_196073_object7062:head# 2024-09-20T22:20:51.568 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.568 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2747890:::benchmark_data_smithi060_196073_object14948:head# 2024-09-20T22:20:51.569 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.569 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b27542fc:::benchmark_data_smithi060_196073_object57302:head# 2024-09-20T22:20:51.569 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.570 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b275a8b6:::benchmark_data_smithi060_196073_object983:head# 2024-09-20T22:20:51.570 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.570 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b275fedb:::benchmark_data_smithi060_196073_object30078:head# 2024-09-20T22:20:51.575 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.575 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b279d775:::benchmark_data_smithi060_196073_object21517:head# 2024-09-20T22:20:51.575 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.575 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b279fde3:::benchmark_data_smithi060_196073_object52207:head# 2024-09-20T22:20:51.576 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.576 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b27a4b29:::benchmark_data_smithi060_196073_object40748:head# 2024-09-20T22:20:51.576 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.576 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b27ceb1e:::benchmark_data_smithi060_196073_object22989:head# 2024-09-20T22:20:51.578 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.578 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b27d1d5c:::benchmark_data_smithi060_196073_object46112:head# 2024-09-20T22:20:51.578 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.579 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b27d5eec:::benchmark_data_smithi060_196073_object44646:head# 2024-09-20T22:20:51.579 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.579 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b27dd5e1:::benchmark_data_smithi060_196073_object48318:head# 2024-09-20T22:20:51.579 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.580 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b27dd759:::benchmark_data_smithi060_196073_object33520:head# 2024-09-20T22:20:51.596 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:51.602 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.602 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:51 smithi067 ceph-mon[33511]: pgmap v957: 178 pgs: 1 active+remapped+backfill_wait+backfill_toofull, 1 active+remapped+backfilling, 3 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.9 GiB data, 18 GiB used, 518 GiB / 536 GiB avail; 590 B/s rd, 0 B/s wr, 877 op/s; 4067/242787 objects degraded (1.675%); 13432/242787 objects misplaced (5.532%); 29 MiB/s, 456 objects/s recovering 2024-09-20T22:20:51.603 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:51 smithi067 ceph-mon[33511]: osdmap e503: 8 total, 7 up, 6 in 2024-09-20T22:20:51.603 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:51 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2811828921' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:20:51.603 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:20:51.603 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b27f06f2:::benchmark_data_smithi060_196073_object41955:head# 2024-09-20T22:20:51.603 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.604 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b281377f:::benchmark_data_smithi060_196073_object32477:head# 2024-09-20T22:20:51.604 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.604 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b283d2ca:::benchmark_data_smithi060_196073_object17304:head# 2024-09-20T22:20:51.604 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.604 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b283de1f:::benchmark_data_smithi060_196073_object27980:head# 2024-09-20T22:20:51.637 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:51.637 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:51.638 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:51.639 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.640 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b284544b:::benchmark_data_smithi060_196073_object27029:head# 2024-09-20T22:20:51.640 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.640 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b28556f2:::benchmark_data_smithi060_196073_object14766:head# 2024-09-20T22:20:51.641 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.641 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b28625a9:::benchmark_data_smithi060_196073_object51427:head# 2024-09-20T22:20:51.641 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.641 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b287f370:::benchmark_data_smithi060_196073_object35216:head# 2024-09-20T22:20:51.651 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:20:51.651 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:20:51.658 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.658 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b28871f2:::benchmark_data_smithi060_196073_object4116:head# 2024-09-20T22:20:51.658 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.659 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b289146d:::benchmark_data_smithi060_196073_object38785:head# 2024-09-20T22:20:51.659 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.659 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b289805d:::benchmark_data_smithi060_196073_object25023:head# 2024-09-20T22:20:51.659 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.659 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b28bc710:::benchmark_data_smithi060_196073_object12811:head# 2024-09-20T22:20:51.681 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.681 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b290fa6b:::benchmark_data_smithi060_196073_object30851:head# 2024-09-20T22:20:51.681 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.682 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b293de27:::benchmark_data_smithi060_196073_object47335:head# 2024-09-20T22:20:51.682 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.682 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b294b9a3:::benchmark_data_smithi060_196073_object1080:head# 2024-09-20T22:20:51.682 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.683 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b294f445:::benchmark_data_smithi060_196073_object40876:head# 2024-09-20T22:20:51.711 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.711 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2953b6f:::benchmark_data_smithi060_196073_object14783:head# 2024-09-20T22:20:51.711 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.711 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2966440:::benchmark_data_smithi060_196073_object1396:head# 2024-09-20T22:20:51.711 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.712 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b29a103e:::benchmark_data_smithi060_196073_object9789:head# 2024-09-20T22:20:51.712 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.712 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b29aba81:::benchmark_data_smithi060_196073_object42573:head# 2024-09-20T22:20:51.737 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.737 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b29b0ad8:::benchmark_data_smithi060_196073_object29509:head# 2024-09-20T22:20:51.737 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.738 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b29c7432:::benchmark_data_smithi060_196073_object13899:head# 2024-09-20T22:20:51.738 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.738 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b29ddc72:::benchmark_data_smithi060_196073_object52185:head# 2024-09-20T22:20:51.738 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.738 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:51.739 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b29ea259:::benchmark_data_smithi060_196073_object4409:head# 2024-09-20T22:20:51.764 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.764 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b29f8b3f:::benchmark_data_smithi060_196073_object30608:head# 2024-09-20T22:20:51.764 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.765 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2a089fc:::benchmark_data_smithi060_196073_object24317:head# 2024-09-20T22:20:51.765 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.765 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2a1f2f9:::benchmark_data_smithi060_196073_object15571:head# 2024-09-20T22:20:51.765 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.766 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2a22189:::benchmark_data_smithi060_196073_object10863:head# 2024-09-20T22:20:51.770 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:51.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:51.771 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:51.781 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.781 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2a7166a:::benchmark_data_smithi060_196073_object60470:head# 2024-09-20T22:20:51.782 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.782 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2a731be:::benchmark_data_smithi060_196073_object46176:head# 2024-09-20T22:20:51.782 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.783 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2a9fac7:::benchmark_data_smithi060_196073_object48489:head# 2024-09-20T22:20:51.783 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.783 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2abe63e:::benchmark_data_smithi060_196073_object32904:head# 2024-09-20T22:20:51.807 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.807 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2ad61d1:::benchmark_data_smithi060_196073_object25363:head# 2024-09-20T22:20:51.807 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.808 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2b136c1:::benchmark_data_smithi060_196073_object20411:head# 2024-09-20T22:20:51.808 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.808 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2b13bb5:::benchmark_data_smithi060_196073_object19734:head# 2024-09-20T22:20:51.808 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.808 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2b1e9b0:::benchmark_data_smithi060_196073_object7787:head# 2024-09-20T22:20:51.808 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.809 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2b231be:::benchmark_data_smithi060_196073_object59830:head# 2024-09-20T22:20:51.809 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.810 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2b394cd:::benchmark_data_smithi060_196073_object14622:head# 2024-09-20T22:20:51.810 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.810 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2b4200e:::benchmark_data_smithi060_196073_object29482:head# 2024-09-20T22:20:51.810 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.810 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2b74017:::benchmark_data_smithi060_196073_object40068:head# 2024-09-20T22:20:51.810 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.811 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2b7676f:::benchmark_data_smithi060_196073_object44213:head# 2024-09-20T22:20:51.812 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.812 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2b8a694:::benchmark_data_smithi060_196073_object40914:head# 2024-09-20T22:20:51.812 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.812 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2bb27df:::benchmark_data_smithi060_196073_object33569:head# 2024-09-20T22:20:51.812 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.812 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2bc9749:::benchmark_data_smithi060_196073_object16627:head# 2024-09-20T22:20:51.812 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.812 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2be446d:::benchmark_data_smithi060_196073_object48798:head# 2024-09-20T22:20:51.812 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.812 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2bebd98:::benchmark_data_smithi060_196073_object22035:head# 2024-09-20T22:20:51.813 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.814 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2bf57bf:::benchmark_data_smithi060_196073_object1593:head# 2024-09-20T22:20:51.814 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.814 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2bfea11:::benchmark_data_smithi060_196073_object5435:head# 2024-09-20T22:20:51.814 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.814 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2c3c382:::benchmark_data_smithi060_196073_object51426:head# 2024-09-20T22:20:51.814 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.814 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2c3c544:::benchmark_data_smithi060_196073_object38470:head# 2024-09-20T22:20:51.814 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.814 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2c6c4c6:::benchmark_data_smithi060_196073_object57825:head# 2024-09-20T22:20:51.814 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.815 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2c74787:::benchmark_data_smithi060_196073_object61660:head# 2024-09-20T22:20:51.815 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.815 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2ca8fcf:::benchmark_data_smithi060_196073_object61342:head# 2024-09-20T22:20:51.815 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.815 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2cf3f90:::benchmark_data_smithi060_196073_object4555:head# 2024-09-20T22:20:51.816 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.816 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2cf8170:::benchmark_data_smithi060_196073_object32357:head# 2024-09-20T22:20:51.816 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.818 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2cf8936:::benchmark_data_smithi060_196073_object56358:head# 2024-09-20T22:20:51.818 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.818 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2cff83b:::benchmark_data_smithi060_196073_object58465:head# 2024-09-20T22:20:51.818 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.819 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2d13555:::benchmark_data_smithi060_196073_object46010:head# 2024-09-20T22:20:51.819 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.819 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2d2e433:::benchmark_data_smithi060_196073_object53674:head# 2024-09-20T22:20:51.819 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.819 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2d6a0d3:::benchmark_data_smithi060_196073_object6875:head# 2024-09-20T22:20:51.819 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.819 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2d6fd2b:::benchmark_data_smithi060_196073_object50022:head# 2024-09-20T22:20:51.819 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.820 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2d76032:::benchmark_data_smithi060_196073_object7084:head# 2024-09-20T22:20:51.820 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.820 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2d76146:::benchmark_data_smithi060_196073_object30547:head# 2024-09-20T22:20:51.820 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.821 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2d80060:::benchmark_data_smithi060_196073_object3728:head# 2024-09-20T22:20:51.821 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.821 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2d81877:::benchmark_data_smithi060_196073_object41032:head# 2024-09-20T22:20:51.821 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.821 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2d927df:::benchmark_data_smithi060_196073_object51718:head# 2024-09-20T22:20:51.822 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.822 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2da01b9:::benchmark_data_smithi060_196073_object56938:head# 2024-09-20T22:20:51.822 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.822 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2dcff72:::benchmark_data_smithi060_196073_object50714:head# 2024-09-20T22:20:51.822 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.823 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2dd7f4c:::benchmark_data_smithi060_196073_object54541:head# 2024-09-20T22:20:51.823 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.823 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2dfbee6:::benchmark_data_smithi060_196073_object34761:head# 2024-09-20T22:20:51.823 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.823 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2e04bef:::benchmark_data_smithi060_196073_object2705:head# 2024-09-20T22:20:51.823 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.823 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2e2939c:::benchmark_data_smithi060_196073_object16590:head# 2024-09-20T22:20:51.823 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.824 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2e43085:::benchmark_data_smithi060_196073_object38922:head# 2024-09-20T22:20:51.824 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.824 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2e65743:::benchmark_data_smithi060_196073_object35108:head# 2024-09-20T22:20:51.824 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.824 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2e6dcee:::benchmark_data_smithi060_196073_object46786:head# 2024-09-20T22:20:51.825 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.825 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2e6f39e:::benchmark_data_smithi060_196073_object36720:head# 2024-09-20T22:20:51.825 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.826 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2e71ad0:::benchmark_data_smithi060_196073_object14191:head# 2024-09-20T22:20:51.826 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.826 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2e78826:::benchmark_data_smithi060_196073_object26452:head# 2024-09-20T22:20:51.826 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.827 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2e7a4fb:::benchmark_data_smithi060_196073_object1092:head# 2024-09-20T22:20:51.827 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.827 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2e830f9:::benchmark_data_smithi060_196073_object21673:head# 2024-09-20T22:20:51.827 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.829 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2e96146:::benchmark_data_smithi060_196073_object5810:head# 2024-09-20T22:20:51.829 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.829 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2ead0f3:::benchmark_data_smithi060_196073_object34774:head# 2024-09-20T22:20:51.829 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.829 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2eb30a4:::benchmark_data_smithi060_196073_object7006:head# 2024-09-20T22:20:51.829 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.829 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2ed08d6:::benchmark_data_smithi060_196073_object9366:head# 2024-09-20T22:20:51.829 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.830 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2ed8dee:::benchmark_data_smithi060_196073_object25401:head# 2024-09-20T22:20:51.830 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.830 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2edc70b:::benchmark_data_smithi060_196073_object22861:head# 2024-09-20T22:20:51.830 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.831 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2efc947:::benchmark_data_smithi060_196073_object43470:head# 2024-09-20T22:20:51.831 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.831 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2efd5ec:::benchmark_data_smithi060_196073_object61753:head# 2024-09-20T22:20:51.831 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.832 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2f15d03:::benchmark_data_smithi060_196073_object61574:head# 2024-09-20T22:20:51.832 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.833 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2f34cad:::benchmark_data_smithi060_196073_object3320:head# 2024-09-20T22:20:51.833 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.833 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2f3c148:::benchmark_data_smithi060_196073_object40116:head# 2024-09-20T22:20:51.833 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.834 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2f3c30b:::benchmark_data_smithi060_196073_object61451:head# 2024-09-20T22:20:51.834 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.834 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2f4444a:::benchmark_data_smithi060_196073_object35173:head# 2024-09-20T22:20:51.835 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.835 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2f4619d:::benchmark_data_smithi060_196073_object30271:head# 2024-09-20T22:20:51.835 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.835 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2f4c574:::benchmark_data_smithi060_196073_object57434:head# 2024-09-20T22:20:51.835 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.836 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2f5ace5:::benchmark_data_smithi060_196073_object39279:head# 2024-09-20T22:20:51.836 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.836 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2f66d73:::benchmark_data_smithi060_196073_object25464:head# 2024-09-20T22:20:51.836 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.837 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2f69575:::benchmark_data_smithi060_196073_object45225:head# 2024-09-20T22:20:51.837 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.837 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2fbbfee:::benchmark_data_smithi060_196073_object35180:head# 2024-09-20T22:20:51.837 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.838 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2fc293a:::benchmark_data_smithi060_196073_object60239:head# 2024-09-20T22:20:51.838 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.838 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2fd36b9:::benchmark_data_smithi060_196073_object61408:head# 2024-09-20T22:20:51.838 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.839 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2fe22e4:::benchmark_data_smithi060_196073_object45259:head# 2024-09-20T22:20:51.839 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.839 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b2fe45f5:::benchmark_data_smithi060_196073_object53493:head# 2024-09-20T22:20:51.839 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.839 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b301fde5:::benchmark_data_smithi060_196073_object23020:head# 2024-09-20T22:20:51.839 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.839 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b302feec:::benchmark_data_smithi060_196073_object58771:head# 2024-09-20T22:20:51.856 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.857 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3036db4:::benchmark_data_smithi060_196073_object30063:head# 2024-09-20T22:20:51.857 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.857 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b304a5f0:::benchmark_data_smithi060_196073_object16348:head# 2024-09-20T22:20:51.857 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.858 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b306da09:::benchmark_data_smithi060_196073_object14517:head# 2024-09-20T22:20:51.858 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.858 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b308f454:::benchmark_data_smithi060_196073_object21195:head# 2024-09-20T22:20:51.872 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:51.875 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.875 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b30abb8f:::benchmark_data_smithi060_196073_object47523:head# 2024-09-20T22:20:51.875 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.876 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b30b8ca3:::benchmark_data_smithi060_196073_object22670:head# 2024-09-20T22:20:51.876 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.876 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b30c024d:::benchmark_data_smithi060_196073_object32192:head# 2024-09-20T22:20:51.876 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.876 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b30d0e01:::benchmark_data_smithi060_196073_object9251:head# 2024-09-20T22:20:51.892 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.893 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b30d8dfd:::benchmark_data_smithi060_196073_object47774:head# 2024-09-20T22:20:51.893 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.893 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b30e87d1:::benchmark_data_smithi060_196073_object35464:head# 2024-09-20T22:20:51.893 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.894 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b30f555a:::benchmark_data_smithi060_196073_object2735:head# 2024-09-20T22:20:51.894 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.894 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3114470:::benchmark_data_smithi060_196073_object12978:head# 2024-09-20T22:20:51.894 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.895 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b311f842:::benchmark_data_smithi060_196073_object37879:head# 2024-09-20T22:20:51.895 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.895 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3145174:::benchmark_data_smithi060_196073_object1023:head# 2024-09-20T22:20:51.895 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.895 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3193ccc:::benchmark_data_smithi060_196073_object48667:head# 2024-09-20T22:20:51.897 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:51.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:51.898 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:51.932 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.932 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b319b9b5:::benchmark_data_smithi060_196073_object24536:head# 2024-09-20T22:20:51.932 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.933 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b31a76a6:::benchmark_data_smithi060_196073_object2820:head# 2024-09-20T22:20:51.933 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.933 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b31b1ace:::benchmark_data_smithi060_196073_object20744:head# 2024-09-20T22:20:51.933 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.933 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b31b2c55:::benchmark_data_smithi060_196073_object52329:head# 2024-09-20T22:20:51.945 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.945 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b31b73b2:::benchmark_data_smithi060_196073_object46474:head# 2024-09-20T22:20:51.945 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.945 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b31ba861:::benchmark_data_smithi060_196073_object48966:head# 2024-09-20T22:20:51.955 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.956 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b31bb44b:::benchmark_data_smithi060_196073_object33444:head# 2024-09-20T22:20:51.956 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.956 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b31c3faa:::benchmark_data_smithi060_196073_object49070:head# 2024-09-20T22:20:51.971 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.971 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b31dc28b:::benchmark_data_smithi060_196073_object45623:head# 2024-09-20T22:20:51.971 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.971 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3211147:::benchmark_data_smithi060_196073_object40324:head# 2024-09-20T22:20:51.971 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.971 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3222107:::benchmark_data_smithi060_196073_object56490:head# 2024-09-20T22:20:51.972 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:51.972 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b322b370:::benchmark_data_smithi060_196073_object44192:head# 2024-09-20T22:20:51.999 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:52.008 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.010 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b322ce36:::benchmark_data_smithi060_196073_object35843:head# 2024-09-20T22:20:52.010 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.010 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b324dd74:::benchmark_data_smithi060_196073_object8369:head# 2024-09-20T22:20:52.010 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.010 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b32513ee:::benchmark_data_smithi060_196073_object20617:head# 2024-09-20T22:20:52.011 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.011 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3254927:::benchmark_data_smithi060_196073_object44572:head# 2024-09-20T22:20:52.026 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.027 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b325539d:::benchmark_data_smithi060_196073_object18165:head# 2024-09-20T22:20:52.027 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.027 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3272c75:::benchmark_data_smithi060_196073_object21367:head# 2024-09-20T22:20:52.027 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:52.028 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.028 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b329f168:::benchmark_data_smithi060_196073_object60297:head# 2024-09-20T22:20:52.028 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.028 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b32a69e1:::benchmark_data_smithi060_196073_object49464:head# 2024-09-20T22:20:52.028 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:52.029 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:52.038 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.039 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b32e39ff:::benchmark_data_smithi060_196073_object6496:head# 2024-09-20T22:20:52.039 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.039 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b32e831a:::benchmark_data_smithi060_196073_object46954:head# 2024-09-20T22:20:52.039 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.039 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b33182c9:::benchmark_data_smithi060_196073_object54476:head# 2024-09-20T22:20:52.040 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.040 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3328a55:::benchmark_data_smithi060_196073_object22827:head# 2024-09-20T22:20:52.063 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:20:52.079 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.079 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3333991:::benchmark_data_smithi060_196073_object408:head# 2024-09-20T22:20:52.079 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.079 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b333b1e7:::benchmark_data_smithi060_196073_object16259:head# 2024-09-20T22:20:52.079 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.080 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b33539d5:::benchmark_data_smithi060_196073_object13929:head# 2024-09-20T22:20:52.080 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.080 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3364479:::benchmark_data_smithi060_196073_object46658:head# 2024-09-20T22:20:52.088 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.089 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b33816a3:::benchmark_data_smithi060_196073_object50470:head# 2024-09-20T22:20:52.089 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.089 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b33dbae6:::benchmark_data_smithi060_196073_object38684:head# 2024-09-20T22:20:52.089 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.089 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b33e6740:::benchmark_data_smithi060_196073_object32485:head# 2024-09-20T22:20:52.089 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.090 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b33ed73d:::benchmark_data_smithi060_196073_object26847:head# 2024-09-20T22:20:52.098 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.098 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b33eea94:::benchmark_data_smithi060_196073_object54118:head# 2024-09-20T22:20:52.098 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.099 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b33f4ffc:::benchmark_data_smithi060_196073_object19744:head# 2024-09-20T22:20:52.099 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.099 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b33fb089:::benchmark_data_smithi060_196073_object59600:head# 2024-09-20T22:20:52.099 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.099 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b342e5d4:::benchmark_data_smithi060_196073_object18993:head# 2024-09-20T22:20:52.100 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.100 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b34397d5:::benchmark_data_smithi060_196073_object7549:head# 2024-09-20T22:20:52.100 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.100 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b343ec2b:::benchmark_data_smithi060_196073_object47723:head# 2024-09-20T22:20:52.127 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.127 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3457b61:::benchmark_data_smithi060_196073_object11280:head# 2024-09-20T22:20:52.127 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.128 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b34609f1:::benchmark_data_smithi060_196073_object45538:head# 2024-09-20T22:20:52.128 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.128 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b347f191:::benchmark_data_smithi060_196073_object43205:head# 2024-09-20T22:20:52.128 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.128 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3490fc6:::benchmark_data_smithi060_196073_object16685:head# 2024-09-20T22:20:52.130 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:52.138 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.138 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b349974b:::benchmark_data_smithi060_196073_object51292:head# 2024-09-20T22:20:52.139 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.139 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b34a1c30:::benchmark_data_smithi060_196073_object42670:head# 2024-09-20T22:20:52.139 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.139 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b34a6c2d:::benchmark_data_smithi060_196073_object15969:head# 2024-09-20T22:20:52.139 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.139 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b34c405f:::benchmark_data_smithi060_196073_object6209:head# 2024-09-20T22:20:52.158 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:52.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:52.159 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:52.163 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.163 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b34dba92:::benchmark_data_smithi060_196073_object30440:head# 2024-09-20T22:20:52.163 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.164 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b34dc7f1:::benchmark_data_smithi060_196073_object39801:head# 2024-09-20T22:20:52.164 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.164 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3505665:::benchmark_data_smithi060_196073_object23502:head# 2024-09-20T22:20:52.164 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.164 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b353dbce:::benchmark_data_smithi060_196073_object11532:head# 2024-09-20T22:20:52.198 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.198 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3566e26:::benchmark_data_smithi060_196073_object42202:head# 2024-09-20T22:20:52.198 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.198 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b356900e:::benchmark_data_smithi060_196073_object15044:head# 2024-09-20T22:20:52.198 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.199 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b356a0b0:::benchmark_data_smithi060_196073_object10826:head# 2024-09-20T22:20:52.199 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.199 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3587f08:::benchmark_data_smithi060_196073_object59208:head# 2024-09-20T22:20:52.218 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.218 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b35a2854:::benchmark_data_smithi060_196073_object29205:head# 2024-09-20T22:20:52.218 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.219 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b35a7f93:::benchmark_data_smithi060_196073_object20376:head# 2024-09-20T22:20:52.219 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.219 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b35bf006:::benchmark_data_smithi060_196073_object37099:head# 2024-09-20T22:20:52.219 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.219 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b35bfd74:::benchmark_data_smithi060_196073_object11623:head# 2024-09-20T22:20:52.251 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.251 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b35cd92f:::benchmark_data_smithi060_196073_object56214:head# 2024-09-20T22:20:52.251 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.251 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b35d5537:::benchmark_data_smithi060_196073_object11195:head# 2024-09-20T22:20:52.251 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.251 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b35fb759:::benchmark_data_smithi060_196073_object13261:head# 2024-09-20T22:20:52.252 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.252 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3600bc1:::benchmark_data_smithi060_196073_object51606:head# 2024-09-20T22:20:52.259 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:52.285 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:52.285 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.285 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:52.286 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:52.286 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b364b645:::benchmark_data_smithi060_196073_object54490:head# 2024-09-20T22:20:52.287 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.287 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b365c737:::benchmark_data_smithi060_196073_object59811:head# 2024-09-20T22:20:52.287 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.287 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3676304:::benchmark_data_smithi060_196073_object15586:head# 2024-09-20T22:20:52.287 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.287 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3677b1e:::benchmark_data_smithi060_196073_object40398:head# 2024-09-20T22:20:52.305 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.306 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:20:52.306 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:20:52.307 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3678862:::benchmark_data_smithi060_196073_object45343:head# 2024-09-20T22:20:52.307 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.307 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3679b55:::benchmark_data_smithi060_196073_object56832:head# 2024-09-20T22:20:52.307 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.307 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b368782f:::benchmark_data_smithi060_196073_object23257:head# 2024-09-20T22:20:52.307 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.321 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3692db2:::benchmark_data_smithi060_196073_object17747:head# 2024-09-20T22:20:52.321 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.322 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3695d58:::benchmark_data_smithi060_196073_object54341:head# 2024-09-20T22:20:52.322 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.323 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b36b4fa1:::benchmark_data_smithi060_196073_object5531:head# 2024-09-20T22:20:52.323 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.323 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b36bdc38:::benchmark_data_smithi060_196073_object48967:head# 2024-09-20T22:20:52.339 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.340 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b36cdb64:::benchmark_data_smithi060_196073_object12704:head# 2024-09-20T22:20:52.340 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.340 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b36f0f4c:::benchmark_data_smithi060_196073_object8171:head# 2024-09-20T22:20:52.340 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.340 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b370a175:::benchmark_data_smithi060_196073_object13045:head# 2024-09-20T22:20:52.340 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.340 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b371d383:::benchmark_data_smithi060_196073_object7011:head# 2024-09-20T22:20:52.347 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.347 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3721e98:::benchmark_data_smithi060_196073_object14484:head# 2024-09-20T22:20:52.347 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.348 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b373225e:::benchmark_data_smithi060_196073_object43566:head# 2024-09-20T22:20:52.348 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.348 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b37391d1:::benchmark_data_smithi060_196073_object42116:head# 2024-09-20T22:20:52.348 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.348 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b373f613:::benchmark_data_smithi060_196073_object13605:head# 2024-09-20T22:20:52.356 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.356 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b374fc81:::benchmark_data_smithi060_196073_object25335:head# 2024-09-20T22:20:52.356 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.356 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b376c2c4:::benchmark_data_smithi060_196073_object24154:head# 2024-09-20T22:20:52.356 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.356 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3774e2b:::benchmark_data_smithi060_196073_object49835:head# 2024-09-20T22:20:52.357 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.357 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3778523:::benchmark_data_smithi060_196073_object38319:head# 2024-09-20T22:20:52.376 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.376 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b377a09e:::benchmark_data_smithi060_196073_object23883:head# 2024-09-20T22:20:52.376 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.376 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3788946:::benchmark_data_smithi060_196073_object18214:head# 2024-09-20T22:20:52.377 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.377 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b379e91f:::benchmark_data_smithi060_196073_object38394:head# 2024-09-20T22:20:52.377 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.377 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b37a3a99:::benchmark_data_smithi060_196073_object4820:head# 2024-09-20T22:20:52.386 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:52.389 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.390 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b37b2b5f:::benchmark_data_smithi060_196073_object57870:head# 2024-09-20T22:20:52.390 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.390 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b37c9b68:::benchmark_data_smithi060_196073_object15463:head# 2024-09-20T22:20:52.390 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.391 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b37d433a:::benchmark_data_smithi060_196073_object3818:head# 2024-09-20T22:20:52.391 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.391 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b37e4a9f:::benchmark_data_smithi060_196073_object35702:head# 2024-09-20T22:20:52.402 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:52.411 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.412 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:52.412 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b37e8e59:::benchmark_data_smithi060_196073_object7247:head# 2024-09-20T22:20:52.413 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.413 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b37ed8e4:::benchmark_data_smithi060_196073_object8605:head# 2024-09-20T22:20:52.413 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.413 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:52.413 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:52.414 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b37fcb2b:::benchmark_data_smithi060_196073_object60373:head# 2024-09-20T22:20:52.414 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.414 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b37fdb7e:::benchmark_data_smithi060_196073_object18532:head# 2024-09-20T22:20:52.431 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.432 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b380a946:::benchmark_data_smithi060_196073_object14708:head# 2024-09-20T22:20:52.432 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.432 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3813513:::benchmark_data_smithi060_196073_object23944:head# 2024-09-20T22:20:52.432 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.432 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3820d15:::benchmark_data_smithi060_196073_object54521:head# 2024-09-20T22:20:52.432 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.432 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b382e64c:::benchmark_data_smithi060_196073_object58186:head# 2024-09-20T22:20:52.440 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.440 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b383521a:::benchmark_data_smithi060_196073_object25743:head# 2024-09-20T22:20:52.440 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.440 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b383527d:::benchmark_data_smithi060_196073_object13143:head# 2024-09-20T22:20:52.457 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.457 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b383b44b:::benchmark_data_smithi060_196073_object14348:head# 2024-09-20T22:20:52.458 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.458 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3841722:::benchmark_data_smithi060_196073_object8625:head# 2024-09-20T22:20:52.458 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.458 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b38664dd:::benchmark_data_smithi060_196073_object15661:head# 2024-09-20T22:20:52.459 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.459 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3883508:::benchmark_data_smithi060_196073_object26607:head# 2024-09-20T22:20:52.475 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.475 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3887809:::benchmark_data_smithi060_196073_object33881:head# 2024-09-20T22:20:52.475 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.475 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3889400:::benchmark_data_smithi060_196073_object18252:head# 2024-09-20T22:20:52.475 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.476 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b388c5b3:::benchmark_data_smithi060_196073_object23627:head# 2024-09-20T22:20:52.476 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.476 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b389ac97:::benchmark_data_smithi060_196073_object28506:head# 2024-09-20T22:20:52.476 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.476 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b389f602:::benchmark_data_smithi060_196073_object13258:head# 2024-09-20T22:20:52.477 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.477 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b38a3a89:::benchmark_data_smithi060_196073_object22211:head# 2024-09-20T22:20:52.477 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.477 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b38a5351:::benchmark_data_smithi060_196073_object36028:head# 2024-09-20T22:20:52.477 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.477 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b38c5b69:::benchmark_data_smithi060_196073_object59828:head# 2024-09-20T22:20:52.508 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.508 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b38f8b44:::benchmark_data_smithi060_196073_object17925:head# 2024-09-20T22:20:52.509 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.509 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b390b520:::benchmark_data_smithi060_196073_object17041:head# 2024-09-20T22:20:52.509 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.509 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3914467:::benchmark_data_smithi060_196073_object48739:head# 2024-09-20T22:20:52.509 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.510 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3918081:::benchmark_data_smithi060_196073_object46165:head# 2024-09-20T22:20:52.514 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:52.524 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.524 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b391b0c6:::benchmark_data_smithi060_196073_object28959:head# 2024-09-20T22:20:52.524 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.525 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b391f5db:::benchmark_data_smithi060_196073_object39985:head# 2024-09-20T22:20:52.525 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.525 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3924c9d:::benchmark_data_smithi060_196073_object3979:head# 2024-09-20T22:20:52.525 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.526 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3928a34:::benchmark_data_smithi060_196073_object2071:head# 2024-09-20T22:20:52.541 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.542 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b392b637:::benchmark_data_smithi060_196073_object39787:head# 2024-09-20T22:20:52.542 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.542 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3959f96:::benchmark_data_smithi060_196073_object43:head# 2024-09-20T22:20:52.543 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.543 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b396ba04:::benchmark_data_smithi060_196073_object638:head# 2024-09-20T22:20:52.543 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.543 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3972deb:::benchmark_data_smithi060_196073_object49371:head# 2024-09-20T22:20:52.543 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:52.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:52.544 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:52.565 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.566 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b397c237:::benchmark_data_smithi060_196073_object27782:head# 2024-09-20T22:20:52.566 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.566 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b39823f2:::benchmark_data_smithi060_196073_object657:head# 2024-09-20T22:20:52.566 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.567 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b399c574:::benchmark_data_smithi060_196073_object61207:head# 2024-09-20T22:20:52.567 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.567 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b39a6460:::benchmark_data_smithi060_196073_object42982:head# 2024-09-20T22:20:52.582 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.583 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b39bb063:::benchmark_data_smithi060_196073_object21581:head# 2024-09-20T22:20:52.583 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.583 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b39e8960:::benchmark_data_smithi060_196073_object2393:head# 2024-09-20T22:20:52.583 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.584 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b39f70d9:::benchmark_data_smithi060_196073_object34227:head# 2024-09-20T22:20:52.584 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.584 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b39f8ba0:::benchmark_data_smithi060_196073_object38173:head# 2024-09-20T22:20:52.601 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.602 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3a00005:::benchmark_data_smithi060_196073_object53980:head# 2024-09-20T22:20:52.602 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.602 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3a1dac5:::benchmark_data_smithi060_196073_object55062:head# 2024-09-20T22:20:52.602 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.603 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3a22afb:::benchmark_data_smithi060_196073_object29960:head# 2024-09-20T22:20:52.603 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.603 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3a24dcf:::benchmark_data_smithi060_196073_object46966:head# 2024-09-20T22:20:52.641 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.642 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:52 smithi067 ceph-mon[33511]: osdmap e504: 8 total, 7 up, 6 in 2024-09-20T22:20:52.642 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:52 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 4067/242787 objects degraded (1.675%), 3 pgs degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T22:20:52.643 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3a2803c:::benchmark_data_smithi060_196073_object31224:head# 2024-09-20T22:20:52.643 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.643 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3a376a7:::benchmark_data_smithi060_196073_object22215:head# 2024-09-20T22:20:52.643 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.643 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3a47585:::benchmark_data_smithi060_196073_object50134:head# 2024-09-20T22:20:52.645 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:52.662 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.663 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3a5e10a:::benchmark_data_smithi060_196073_object31182:head# 2024-09-20T22:20:52.663 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.663 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3a60a81:::benchmark_data_smithi060_196073_object5364:head# 2024-09-20T22:20:52.663 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.664 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3a63705:::benchmark_data_smithi060_196073_object52418:head# 2024-09-20T22:20:52.664 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.664 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3a6fe06:::benchmark_data_smithi060_196073_object12219:head# 2024-09-20T22:20:52.664 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.665 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3a74e85:::benchmark_data_smithi060_196073_object52437:head# 2024-09-20T22:20:52.665 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.665 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3aa74c5:::benchmark_data_smithi060_196073_object22173:head# 2024-09-20T22:20:52.665 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.665 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3ae2cc0:::benchmark_data_smithi060_196073_object47825:head# 2024-09-20T22:20:52.665 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.666 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3af654f:::benchmark_data_smithi060_196073_object48655:head# 2024-09-20T22:20:52.666 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.666 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3b08303:::benchmark_data_smithi060_196073_object34351:head# 2024-09-20T22:20:52.666 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.667 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3b2bf6f:::benchmark_data_smithi060_196073_object36624:head# 2024-09-20T22:20:52.667 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.667 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3b36532:::benchmark_data_smithi060_196073_object38096:head# 2024-09-20T22:20:52.667 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.667 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3b478ad:::benchmark_data_smithi060_196073_object55442:head# 2024-09-20T22:20:52.668 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.668 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3b74619:::benchmark_data_smithi060_196073_object1937:head# 2024-09-20T22:20:52.668 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.668 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3b76747:::benchmark_data_smithi060_196073_object14214:head# 2024-09-20T22:20:52.668 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.669 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3b856c8:::benchmark_data_smithi060_196073_object19323:head# 2024-09-20T22:20:52.669 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.669 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3b93ce1:::benchmark_data_smithi060_196073_object49058:head# 2024-09-20T22:20:52.669 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.670 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3b95372:::benchmark_data_smithi060_196073_object12749:head# 2024-09-20T22:20:52.670 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.670 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3b95b52:::benchmark_data_smithi060_196073_object48647:head# 2024-09-20T22:20:52.670 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.671 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3b9f0c3:::benchmark_data_smithi060_196073_object20904:head# 2024-09-20T22:20:52.671 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.671 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3bb54fd:::benchmark_data_smithi060_196073_object23057:head# 2024-09-20T22:20:52.671 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.672 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3bd9154:::benchmark_data_smithi060_196073_object1944:head# 2024-09-20T22:20:52.672 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.672 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3be4271:::benchmark_data_smithi060_196073_object24030:head# 2024-09-20T22:20:52.679 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.679 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3bf5037:::benchmark_data_smithi060_196073_object61000:head# 2024-09-20T22:20:52.680 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.680 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3bfa141:::benchmark_data_smithi060_196073_object32245:head# 2024-09-20T22:20:52.680 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.680 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3c01b16:::benchmark_data_smithi060_196073_object45464:head# 2024-09-20T22:20:52.680 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.681 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3c271e6:::benchmark_data_smithi060_196073_object52291:head# 2024-09-20T22:20:52.681 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.681 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3c27d95:::benchmark_data_smithi060_196073_object55244:head# 2024-09-20T22:20:52.681 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.681 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3c3e680:::benchmark_data_smithi060_196073_object54149:head# 2024-09-20T22:20:52.681 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.682 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3c68ff7:::benchmark_data_smithi060_196073_object1440:head# 2024-09-20T22:20:52.682 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.682 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3c7449a:::benchmark_data_smithi060_196073_object21181:head# 2024-09-20T22:20:52.682 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.682 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3c764ed:::benchmark_data_smithi060_196073_object54815:head# 2024-09-20T22:20:52.682 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.683 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3c9f541:::benchmark_data_smithi060_196073_object52684:head# 2024-09-20T22:20:52.683 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.683 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3cab9bf:::benchmark_data_smithi060_196073_object27630:head# 2024-09-20T22:20:52.683 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.684 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3caccaf:::benchmark_data_smithi060_196073_object57735:head# 2024-09-20T22:20:52.684 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.684 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3cad36a:::benchmark_data_smithi060_196073_object26612:head# 2024-09-20T22:20:52.684 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.684 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3ccae71:::benchmark_data_smithi060_196073_object14821:head# 2024-09-20T22:20:52.684 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.685 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3ce1b8a:::benchmark_data_smithi060_196073_object20265:head# 2024-09-20T22:20:52.685 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.685 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3cea88d:::benchmark_data_smithi060_196073_object50107:head# 2024-09-20T22:20:52.685 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.686 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3cf0d2a:::benchmark_data_smithi060_196073_object61450:head# 2024-09-20T22:20:52.686 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.686 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3cf5424:::benchmark_data_smithi060_196073_object36838:head# 2024-09-20T22:20:52.686 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.687 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3cfd552:::benchmark_data_smithi060_196073_object21434:head# 2024-09-20T22:20:52.687 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.687 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3d10e0d:::benchmark_data_smithi060_196073_object44706:head# 2024-09-20T22:20:52.687 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.688 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3d17350:::benchmark_data_smithi060_196073_object37621:head# 2024-09-20T22:20:52.688 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.688 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3d20b18:::benchmark_data_smithi060_196073_object12329:head# 2024-09-20T22:20:52.688 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.689 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:52.690 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:52.690 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:52.691 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3d24c36:::benchmark_data_smithi060_196073_object8429:head# 2024-09-20T22:20:52.691 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.691 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3d50a52:::benchmark_data_smithi060_196073_object36356:head# 2024-09-20T22:20:52.691 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.691 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3d63d7c:::benchmark_data_smithi060_196073_object19944:head# 2024-09-20T22:20:52.691 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.691 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3d87478:::benchmark_data_smithi060_196073_object46554:head# 2024-09-20T22:20:52.692 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.692 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3d9b5df:::benchmark_data_smithi060_196073_object54837:head# 2024-09-20T22:20:52.692 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.692 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3da2142:::benchmark_data_smithi060_196073_object4729:head# 2024-09-20T22:20:52.692 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.692 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3dacfa7:::benchmark_data_smithi060_196073_object5619:head# 2024-09-20T22:20:52.692 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.692 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3ddba48:::benchmark_data_smithi060_196073_object12415:head# 2024-09-20T22:20:52.692 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.693 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3ddeb51:::benchmark_data_smithi060_196073_object40866:head# 2024-09-20T22:20:52.693 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.693 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3e06814:::benchmark_data_smithi060_196073_object48419:head# 2024-09-20T22:20:52.693 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.694 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3e29a3c:::benchmark_data_smithi060_196073_object43446:head# 2024-09-20T22:20:52.694 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.694 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3e2bd48:::benchmark_data_smithi060_196073_object60935:head# 2024-09-20T22:20:52.694 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.694 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3e5b1f9:::benchmark_data_smithi060_196073_object2064:head# 2024-09-20T22:20:52.694 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.695 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3e5cf7e:::benchmark_data_smithi060_196073_object47729:head# 2024-09-20T22:20:52.695 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.695 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3e76c95:::benchmark_data_smithi060_196073_object57663:head# 2024-09-20T22:20:52.695 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.695 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3ec7baf:::benchmark_data_smithi060_196073_object35461:head# 2024-09-20T22:20:52.695 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.695 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3ecb36d:::benchmark_data_smithi060_196073_object59644:head# 2024-09-20T22:20:52.695 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.696 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3ecd474:::benchmark_data_smithi060_196073_object8320:head# 2024-09-20T22:20:52.696 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.696 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3edd006:::benchmark_data_smithi060_196073_object37309:head# 2024-09-20T22:20:52.696 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.696 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3f0fec7:::benchmark_data_smithi060_196073_object5775:head# 2024-09-20T22:20:52.696 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.697 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3f1763e:::benchmark_data_smithi060_196073_object59263:head# 2024-09-20T22:20:52.697 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.697 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3f2a13c:::benchmark_data_smithi060_196073_object54069:head# 2024-09-20T22:20:52.697 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.698 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3f30210:::benchmark_data_smithi060_196073_object28297:head# 2024-09-20T22:20:52.698 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.698 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3f34092:::benchmark_data_smithi060_196073_object2657:head# 2024-09-20T22:20:52.698 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.699 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3f375af:::benchmark_data_smithi060_196073_object57254:head# 2024-09-20T22:20:52.699 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.700 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3f3ec9d:::benchmark_data_smithi060_196073_object16189:head# 2024-09-20T22:20:52.700 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.700 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3f46978:::benchmark_data_smithi060_196073_object52575:head# 2024-09-20T22:20:52.700 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.701 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3f66b17:::benchmark_data_smithi060_196073_object26628:head# 2024-09-20T22:20:52.701 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.701 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3f68623:::benchmark_data_smithi060_196073_object920:head# 2024-09-20T22:20:52.701 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.702 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3f7172b:::benchmark_data_smithi060_196073_object26665:head# 2024-09-20T22:20:52.702 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.702 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3f7abc8:::benchmark_data_smithi060_196073_object45649:head# 2024-09-20T22:20:52.702 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.703 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3f7f456:::benchmark_data_smithi060_196073_object47095:head# 2024-09-20T22:20:52.703 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.704 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3f876d3:::benchmark_data_smithi060_196073_object47019:head# 2024-09-20T22:20:52.704 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.705 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3f97ebe:::benchmark_data_smithi060_196073_object21218:head# 2024-09-20T22:20:52.705 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.705 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3fa2cff:::benchmark_data_smithi060_196073_object16017:head# 2024-09-20T22:20:52.705 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.705 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3fa3af9:::benchmark_data_smithi060_196073_object58498:head# 2024-09-20T22:20:52.706 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.708 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3fb2e71:::benchmark_data_smithi060_196073_object6670:head# 2024-09-20T22:20:52.708 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.708 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3fc3d91:::benchmark_data_smithi060_196073_object54374:head# 2024-09-20T22:20:52.708 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.708 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3fc956d:::benchmark_data_smithi060_196073_object48010:head# 2024-09-20T22:20:52.709 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.709 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3fe164f:::benchmark_data_smithi060_196073_object37779:head# 2024-09-20T22:20:52.709 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.709 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3fe8558:::benchmark_data_smithi060_196073_object9675:head# 2024-09-20T22:20:52.709 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.710 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b3ff4969:::benchmark_data_smithi060_196073_object28645:head# 2024-09-20T22:20:52.710 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.710 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4035c9e:::benchmark_data_smithi060_196073_object19774:head# 2024-09-20T22:20:52.710 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.711 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b40addcb:::benchmark_data_smithi060_196073_object17846:head# 2024-09-20T22:20:52.711 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.711 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b40df169:::benchmark_data_smithi060_196073_object8422:head# 2024-09-20T22:20:52.712 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.713 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b40e4cf8:::benchmark_data_smithi060_196073_object50683:head# 2024-09-20T22:20:52.713 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.713 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b40ec4a8:::benchmark_data_smithi060_196073_object43810:head# 2024-09-20T22:20:52.713 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.714 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b40f405f:::benchmark_data_smithi060_196073_object15687:head# 2024-09-20T22:20:52.714 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.714 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b41105d4:::benchmark_data_smithi060_196073_object58571:head# 2024-09-20T22:20:52.737 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.737 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b41285d9:::benchmark_data_smithi060_196073_object58725:head# 2024-09-20T22:20:52.737 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.738 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b413181b:::benchmark_data_smithi060_196073_object22170:head# 2024-09-20T22:20:52.738 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.738 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4138111:::benchmark_data_smithi060_196073_object41645:head# 2024-09-20T22:20:52.738 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.739 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b413a9ef:::benchmark_data_smithi060_196073_object51231:head# 2024-09-20T22:20:52.750 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.750 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b413b915:::benchmark_data_smithi060_196073_object29499:head# 2024-09-20T22:20:52.750 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.751 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b414189c:::benchmark_data_smithi060_196073_object53069:head# 2024-09-20T22:20:52.751 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.751 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4180775:::benchmark_data_smithi060_196073_object37837:head# 2024-09-20T22:20:52.751 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.751 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b41981e0:::benchmark_data_smithi060_196073_object53516:head# 2024-09-20T22:20:52.764 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.765 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b41b9d8e:::benchmark_data_smithi060_196073_object25461:head# 2024-09-20T22:20:52.765 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.765 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b41c194c:::benchmark_data_smithi060_196073_object24814:head# 2024-09-20T22:20:52.765 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.765 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b41d9a05:::benchmark_data_smithi060_196073_object57118:head# 2024-09-20T22:20:52.766 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.766 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4205ba6:::benchmark_data_smithi060_196073_object2065:head# 2024-09-20T22:20:52.775 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.775 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4209579:::benchmark_data_smithi060_196073_object59171:head# 2024-09-20T22:20:52.776 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.776 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b421a68d:::benchmark_data_smithi060_196073_object55237:head# 2024-09-20T22:20:52.777 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.777 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b421f2fa:::benchmark_data_smithi060_196073_object53192:head# 2024-09-20T22:20:52.777 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.777 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4245d80:::benchmark_data_smithi060_196073_object23836:head# 2024-09-20T22:20:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:52 smithi060 ceph-mon[29694]: osdmap e504: 8 total, 7 up, 6 in 2024-09-20T22:20:52.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:52 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 4067/242787 objects degraded (1.675%), 3 pgs degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T22:20:52.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:52 smithi060 ceph-mon[40290]: osdmap e504: 8 total, 7 up, 6 in 2024-09-20T22:20:52.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:52 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 4067/242787 objects degraded (1.675%), 3 pgs degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T22:20:52.791 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:52.801 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.801 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4252baf:::benchmark_data_smithi060_196073_object22662:head# 2024-09-20T22:20:52.801 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.801 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b428ead2:::benchmark_data_smithi060_196073_object61142:head# 2024-09-20T22:20:52.801 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.802 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b42a2006:::benchmark_data_smithi060_196073_object18076:head# 2024-09-20T22:20:52.802 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.802 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b42a6de8:::benchmark_data_smithi060_196073_object10149:head# 2024-09-20T22:20:52.818 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.819 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b42dac0e:::benchmark_data_smithi060_196073_object58637:head# 2024-09-20T22:20:52.819 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.819 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b42fc3cc:::benchmark_data_smithi060_196073_object49672:head# 2024-09-20T22:20:52.819 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.820 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b430a5de:::benchmark_data_smithi060_196073_object30056:head# 2024-09-20T22:20:52.820 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.820 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b43135a4:::benchmark_data_smithi060_196073_object20810:head# 2024-09-20T22:20:52.820 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.820 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b43288b2:::benchmark_data_smithi060_196073_object11885:head# 2024-09-20T22:20:52.820 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.821 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b432c92b:::benchmark_data_smithi060_196073_object46659:head# 2024-09-20T22:20:52.827 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:52.827 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:52.827 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:52.842 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.842 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4338bf3:::benchmark_data_smithi060_196073_object15620:head# 2024-09-20T22:20:52.842 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.843 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b434ea84:::benchmark_data_smithi060_196073_object49484:head# 2024-09-20T22:20:52.843 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.843 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b43513db:::benchmark_data_smithi060_196073_object52286:head# 2024-09-20T22:20:52.843 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.843 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4358778:::benchmark_data_smithi060_196073_object59039:head# 2024-09-20T22:20:52.855 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.921 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b43597d0:::benchmark_data_smithi060_196073_object60762:head# 2024-09-20T22:20:52.921 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.921 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b435c606:::benchmark_data_smithi060_196073_object1674:head# 2024-09-20T22:20:52.921 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.921 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b436d045:::benchmark_data_smithi060_196073_object18706:head# 2024-09-20T22:20:52.921 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.921 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4394c5f:::benchmark_data_smithi060_196073_object16909:head# 2024-09-20T22:20:52.921 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.921 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b43aa4f4:::benchmark_data_smithi060_196073_object36216:head# 2024-09-20T22:20:52.921 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.921 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b43aa553:::benchmark_data_smithi060_196073_object40738:head# 2024-09-20T22:20:52.921 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.921 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b43ac953:::benchmark_data_smithi060_196073_object39657:head# 2024-09-20T22:20:52.921 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.922 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b43be871:::benchmark_data_smithi060_196073_object40199:head# 2024-09-20T22:20:52.922 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.922 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b43da762:::benchmark_data_smithi060_196073_object39957:head# 2024-09-20T22:20:52.922 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.922 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b43daa9d:::benchmark_data_smithi060_196073_object60372:head# 2024-09-20T22:20:52.922 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.922 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b43e3050:::benchmark_data_smithi060_196073_object26853:head# 2024-09-20T22:20:52.922 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.922 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b43ecfda:::benchmark_data_smithi060_196073_object53916:head# 2024-09-20T22:20:52.922 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.922 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b43ef5e7:::benchmark_data_smithi060_196073_object1876:head# 2024-09-20T22:20:52.922 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.922 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b440a0d9:::benchmark_data_smithi060_196073_object45125:head# 2024-09-20T22:20:52.922 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.923 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4411057:::benchmark_data_smithi060_196073_object3675:head# 2024-09-20T22:20:52.923 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.923 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b442bd62:::benchmark_data_smithi060_196073_object44900:head# 2024-09-20T22:20:52.923 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.923 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b443ead0:::benchmark_data_smithi060_196073_object47054:head# 2024-09-20T22:20:52.923 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.923 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b443f4b1:::benchmark_data_smithi060_196073_object30725:head# 2024-09-20T22:20:52.923 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.923 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b444edbb:::benchmark_data_smithi060_196073_object55180:head# 2024-09-20T22:20:52.923 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.923 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4453ac3:::benchmark_data_smithi060_196073_object36378:head# 2024-09-20T22:20:52.923 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.923 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4488b89:::benchmark_data_smithi060_196073_object8769:head# 2024-09-20T22:20:52.923 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.923 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b448bcf3:::benchmark_data_smithi060_196073_object26948:head# 2024-09-20T22:20:52.924 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.924 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b448d214:::benchmark_data_smithi060_196073_object40412:head# 2024-09-20T22:20:52.924 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.924 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b44966d7:::benchmark_data_smithi060_196073_object24518:head# 2024-09-20T22:20:52.924 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.924 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b44a6018:::benchmark_data_smithi060_196073_object29208:head# 2024-09-20T22:20:52.924 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:20:52 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:20:52.925 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b44ab390:::benchmark_data_smithi060_196073_object37822:head# 2024-09-20T22:20:52.925 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.925 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b44ae9f9:::benchmark_data_smithi060_196073_object6460:head# 2024-09-20T22:20:52.926 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.926 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b44d3b0c:::benchmark_data_smithi060_196073_object32268:head# 2024-09-20T22:20:52.926 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.926 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b44dbc46:::benchmark_data_smithi060_196073_object20979:head# 2024-09-20T22:20:52.926 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.926 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b44de48a:::benchmark_data_smithi060_196073_object27779:head# 2024-09-20T22:20:52.926 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.926 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b44e8fab:::benchmark_data_smithi060_196073_object51119:head# 2024-09-20T22:20:52.926 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.926 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b44ef4cc:::benchmark_data_smithi060_196073_object44653:head# 2024-09-20T22:20:52.926 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.926 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b44fed18:::benchmark_data_smithi060_196073_object39213:head# 2024-09-20T22:20:52.928 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:52.943 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.943 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4506b9d:::benchmark_data_smithi060_196073_object6828:head# 2024-09-20T22:20:52.943 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.943 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b45101bc:::benchmark_data_smithi060_196073_object1406:head# 2024-09-20T22:20:52.943 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.944 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4531313:::benchmark_data_smithi060_196073_object28899:head# 2024-09-20T22:20:52.944 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.944 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b455bbfa:::benchmark_data_smithi060_196073_object5025:head# 2024-09-20T22:20:52.969 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:52.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:52.970 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:52.973 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.974 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b455c40e:::benchmark_data_smithi060_196073_object35628:head# 2024-09-20T22:20:52.974 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.975 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4566595:::benchmark_data_smithi060_196073_object28322:head# 2024-09-20T22:20:52.975 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.975 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b456b463:::benchmark_data_smithi060_196073_object27792:head# 2024-09-20T22:20:52.975 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.975 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b456c47c:::benchmark_data_smithi060_196073_object14728:head# 2024-09-20T22:20:52.993 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.993 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b458275b:::benchmark_data_smithi060_196073_object2496:head# 2024-09-20T22:20:52.993 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.993 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4583d2e:::benchmark_data_smithi060_196073_object17900:head# 2024-09-20T22:20:52.994 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.994 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b45955a8:::benchmark_data_smithi060_196073_object60576:head# 2024-09-20T22:20:52.994 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:52.994 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b45f192f:::benchmark_data_smithi060_196073_object40974:head# 2024-09-20T22:20:53.025 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.026 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b45fa64f:::benchmark_data_smithi060_196073_object33811:head# 2024-09-20T22:20:53.026 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.026 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b45ff681:::benchmark_data_smithi060_196073_object49309:head# 2024-09-20T22:20:53.026 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.027 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4608d82:::benchmark_data_smithi060_196073_object7669:head# 2024-09-20T22:20:53.027 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.027 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b460a72a:::benchmark_data_smithi060_196073_object56059:head# 2024-09-20T22:20:53.038 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.039 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b461e67d:::benchmark_data_smithi060_196073_object23762:head# 2024-09-20T22:20:53.039 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.039 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4620d6f:::benchmark_data_smithi060_196073_object22583:head# 2024-09-20T22:20:53.039 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.040 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b463af24:::benchmark_data_smithi060_196073_object21485:head# 2024-09-20T22:20:53.040 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.040 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b46440b9:::benchmark_data_smithi060_196073_object18136:head# 2024-09-20T22:20:53.045 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.045 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b465761b:::benchmark_data_smithi060_196073_object34731:head# 2024-09-20T22:20:53.045 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.045 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4659782:::benchmark_data_smithi060_196073_object33943:head# 2024-09-20T22:20:53.060 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.060 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b46787ab:::benchmark_data_smithi060_196073_object7194:head# 2024-09-20T22:20:53.061 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.061 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b468f66a:::benchmark_data_smithi060_196073_object2466:head# 2024-09-20T22:20:53.061 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.062 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b46979cf:::benchmark_data_smithi060_196073_object20686:head# 2024-09-20T22:20:53.062 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.062 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b46997a6:::benchmark_data_smithi060_196073_object15011:head# 2024-09-20T22:20:53.070 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:53.089 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.090 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b46a006c:::benchmark_data_smithi060_196073_object36280:head# 2024-09-20T22:20:53.090 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.090 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b46a1a6a:::benchmark_data_smithi060_196073_object50152:head# 2024-09-20T22:20:53.090 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.090 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b46c1387:::benchmark_data_smithi060_196073_object54907:head# 2024-09-20T22:20:53.091 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.092 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b46dfd30:::benchmark_data_smithi060_196073_object29929:head# 2024-09-20T22:20:53.104 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:53.105 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:53.105 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:53.112 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.112 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b46e1098:::benchmark_data_smithi060_196073_object19017:head# 2024-09-20T22:20:53.112 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.112 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b47013b5:::benchmark_data_smithi060_196073_object39585:head# 2024-09-20T22:20:53.112 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.113 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4724d02:::benchmark_data_smithi060_196073_object35079:head# 2024-09-20T22:20:53.113 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.113 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b47310aa:::benchmark_data_smithi060_196073_object61589:head# 2024-09-20T22:20:53.142 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.142 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b474d95d:::benchmark_data_smithi060_196073_object53974:head# 2024-09-20T22:20:53.142 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.142 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b474e93b:::benchmark_data_smithi060_196073_object59696:head# 2024-09-20T22:20:53.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b47951b0:::benchmark_data_smithi060_196073_object10858:head# 2024-09-20T22:20:53.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b47adc56:::benchmark_data_smithi060_196073_object3852:head# 2024-09-20T22:20:53.169 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.170 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b47c6b27:::benchmark_data_smithi060_196073_object9579:head# 2024-09-20T22:20:53.170 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.170 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b47dca2d:::benchmark_data_smithi060_196073_object54488:head# 2024-09-20T22:20:53.170 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.171 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b47e512f:::benchmark_data_smithi060_196073_object48039:head# 2024-09-20T22:20:53.171 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.171 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b47f373f:::benchmark_data_smithi060_196073_object39874:head# 2024-09-20T22:20:53.194 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.195 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b47fbd40:::benchmark_data_smithi060_196073_object45199:head# 2024-09-20T22:20:53.195 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.196 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4859679:::benchmark_data_smithi060_196073_object56037:head# 2024-09-20T22:20:53.196 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.196 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b485c997:::benchmark_data_smithi060_196073_object44344:head# 2024-09-20T22:20:53.196 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.196 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b48a895d:::benchmark_data_smithi060_196073_object13372:head# 2024-09-20T22:20:53.206 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:53.215 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.216 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b48c324e:::benchmark_data_smithi060_196073_object58720:head# 2024-09-20T22:20:53.216 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.216 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b48d3468:::benchmark_data_smithi060_196073_object11899:head# 2024-09-20T22:20:53.217 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.217 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b48f468c:::benchmark_data_smithi060_196073_object53023:head# 2024-09-20T22:20:53.217 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.217 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4902e9f:::benchmark_data_smithi060_196073_object43046:head# 2024-09-20T22:20:53.247 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.247 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b490572d:::benchmark_data_smithi060_196073_object45697:head# 2024-09-20T22:20:53.247 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.247 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b491b6b7:::benchmark_data_smithi060_196073_object7211:head# 2024-09-20T22:20:53.248 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.248 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b491bfad:::benchmark_data_smithi060_196073_object46394:head# 2024-09-20T22:20:53.248 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.248 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4929c02:::benchmark_data_smithi060_196073_object32894:head# 2024-09-20T22:20:53.251 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:53.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:53.255 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:53.278 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.279 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b492b41e:::benchmark_data_smithi060_196073_object16979:head# 2024-09-20T22:20:53.279 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.279 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b49450fa:::benchmark_data_smithi060_196073_object58087:head# 2024-09-20T22:20:53.279 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.279 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4959d2f:::benchmark_data_smithi060_196073_object6763:head# 2024-09-20T22:20:53.308 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.308 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b495c85c:::benchmark_data_smithi060_196073_object44928:head# 2024-09-20T22:20:53.309 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.310 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4962902:::benchmark_data_smithi060_196073_object21216:head# 2024-09-20T22:20:53.310 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.310 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4971397:::benchmark_data_smithi060_196073_object5199:head# 2024-09-20T22:20:53.310 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.310 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b49a5660:::benchmark_data_smithi060_196073_object32581:head# 2024-09-20T22:20:53.328 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.329 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b49c3a38:::benchmark_data_smithi060_196073_object1319:head# 2024-09-20T22:20:53.329 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.330 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b49e1762:::benchmark_data_smithi060_196073_object40197:head# 2024-09-20T22:20:53.330 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.330 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b49f828b:::benchmark_data_smithi060_196073_object43158:head# 2024-09-20T22:20:53.330 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.330 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b49f91e1:::benchmark_data_smithi060_196073_object61603:head# 2024-09-20T22:20:53.355 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:53.361 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.361 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b49f9c13:::benchmark_data_smithi060_196073_object23920:head# 2024-09-20T22:20:53.361 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.361 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4a17749:::benchmark_data_smithi060_196073_object11091:head# 2024-09-20T22:20:53.362 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.362 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4a37512:::benchmark_data_smithi060_196073_object35199:head# 2024-09-20T22:20:53.362 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.362 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4a4191f:::benchmark_data_smithi060_196073_object32284:head# 2024-09-20T22:20:53.380 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.381 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4a4ac2d:::benchmark_data_smithi060_196073_object31183:head# 2024-09-20T22:20:53.381 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.381 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4a4ca59:::benchmark_data_smithi060_196073_object45843:head# 2024-09-20T22:20:53.381 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.382 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4a68535:::benchmark_data_smithi060_196073_object16012:head# 2024-09-20T22:20:53.382 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.382 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4a6911d:::benchmark_data_smithi060_196073_object40547:head# 2024-09-20T22:20:53.400 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:53.401 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:53.401 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:53.413 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.414 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4a6b508:::benchmark_data_smithi060_196073_object37593:head# 2024-09-20T22:20:53.414 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.414 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4a81f50:::benchmark_data_smithi060_196073_object31464:head# 2024-09-20T22:20:53.414 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.415 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4a8c29c:::benchmark_data_smithi060_196073_object16478:head# 2024-09-20T22:20:53.415 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.415 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4a93006:::benchmark_data_smithi060_196073_object47292:head# 2024-09-20T22:20:53.448 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.449 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4a9e113:::benchmark_data_smithi060_196073_object18017:head# 2024-09-20T22:20:53.450 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.450 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4ac0213:::benchmark_data_smithi060_196073_object18639:head# 2024-09-20T22:20:53.450 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.450 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4ac7167:::benchmark_data_smithi060_196073_object49287:head# 2024-09-20T22:20:53.450 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.450 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4ace764:::benchmark_data_smithi060_196073_object16457:head# 2024-09-20T22:20:53.483 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.484 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4b06d57:::benchmark_data_smithi060_196073_object39141:head# 2024-09-20T22:20:53.484 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.484 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4b0cdaa:::benchmark_data_smithi060_196073_object9924:head# 2024-09-20T22:20:53.484 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.484 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4b39976:::benchmark_data_smithi060_196073_object16325:head# 2024-09-20T22:20:53.484 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.485 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4b3abfc:::benchmark_data_smithi060_196073_object49649:head# 2024-09-20T22:20:53.502 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:53.505 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.505 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4b3b27e:::benchmark_data_smithi060_196073_object49727:head# 2024-09-20T22:20:53.506 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.506 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4b4e0cc:::benchmark_data_smithi060_196073_object2706:head# 2024-09-20T22:20:53.506 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.506 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4b8edc9:::benchmark_data_smithi060_196073_object36054:head# 2024-09-20T22:20:53.506 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.507 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4b98c87:::benchmark_data_smithi060_196073_object29237:head# 2024-09-20T22:20:53.535 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.536 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4ba89da:::benchmark_data_smithi060_196073_object48299:head# 2024-09-20T22:20:53.536 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.536 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4bafe87:::benchmark_data_smithi060_196073_object53127:head# 2024-09-20T22:20:53.536 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.537 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4bbcbf0:::benchmark_data_smithi060_196073_object36416:head# 2024-09-20T22:20:53.537 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.537 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4bdbcc3:::benchmark_data_smithi060_196073_object18506:head# 2024-09-20T22:20:53.539 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:53.540 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:53.540 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:53.547 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.547 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4be0c6e:::benchmark_data_smithi060_196073_object34582:head# 2024-09-20T22:20:53.547 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.548 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4c066ed:::benchmark_data_smithi060_196073_object15799:head# 2024-09-20T22:20:53.548 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.548 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4c1918e:::benchmark_data_smithi060_196073_object40481:head# 2024-09-20T22:20:53.548 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.548 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4c2692b:::benchmark_data_smithi060_196073_object26523:head# 2024-09-20T22:20:53.572 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.572 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4c279c4:::benchmark_data_smithi060_196073_object7999:head# 2024-09-20T22:20:53.572 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.573 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4c29a67:::benchmark_data_smithi060_196073_object61245:head# 2024-09-20T22:20:53.573 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.573 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4c4be6e:::benchmark_data_smithi060_196073_object39483:head# 2024-09-20T22:20:53.573 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.573 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4c56d06:::benchmark_data_smithi060_196073_object20193:head# 2024-09-20T22:20:53.591 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.592 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4c6c0a4:::benchmark_data_smithi060_196073_object27405:head# 2024-09-20T22:20:53.592 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.592 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4c76bf8:::benchmark_data_smithi060_196073_object53246:head# 2024-09-20T22:20:53.592 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.593 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4c80a79:::benchmark_data_smithi060_196073_object33972:head# 2024-09-20T22:20:53.593 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.593 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4c948c5:::benchmark_data_smithi060_196073_object57942:head# 2024-09-20T22:20:53.615 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.616 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4c9a0dd:::benchmark_data_smithi060_196073_object27883:head# 2024-09-20T22:20:53.616 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.616 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4ca6ae5:::benchmark_data_smithi060_196073_object208:head# 2024-09-20T22:20:53.616 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.617 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4cb2827:::benchmark_data_smithi060_196073_object44049:head# 2024-09-20T22:20:53.617 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.617 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4cd78d9:::benchmark_data_smithi060_196073_object32238:head# 2024-09-20T22:20:53.617 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.617 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4cead35:::benchmark_data_smithi060_196073_object54791:head# 2024-09-20T22:20:53.618 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.618 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4cfde61:::benchmark_data_smithi060_196073_object9878:head# 2024-09-20T22:20:53.641 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:53.652 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:20:53.655 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.656 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4d234b0:::benchmark_data_smithi060_196073_object33097:head# 2024-09-20T22:20:53.656 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.656 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4d24881:::benchmark_data_smithi060_196073_object21892:head# 2024-09-20T22:20:53.657 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.657 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4d39b45:::benchmark_data_smithi060_196073_object25798:head# 2024-09-20T22:20:53.657 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.657 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4d690de:::benchmark_data_smithi060_196073_object40358:head# 2024-09-20T22:20:53.667 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:53 smithi067 ceph-mon[33511]: pgmap v960: 178 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.7 GiB data, 18 GiB used, 519 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.16k op/s; 3185/231144 objects degraded (1.378%); 11830/231144 objects misplaced (5.118%); 26 MiB/s, 423 objects/s recovering 2024-09-20T22:20:53.667 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:53 smithi067 ceph-mon[33511]: osdmap e505: 8 total, 7 up, 6 in 2024-09-20T22:20:53.667 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:53 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3186488298' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:20:53.669 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.669 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4d6bb3c:::benchmark_data_smithi060_196073_object14287:head# 2024-09-20T22:20:53.669 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.670 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4d92608:::benchmark_data_smithi060_196073_object4268:head# 2024-09-20T22:20:53.670 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.670 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4d98dae:::benchmark_data_smithi060_196073_object48086:head# 2024-09-20T22:20:53.670 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.671 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4da5a7a:::benchmark_data_smithi060_196073_object8978:head# 2024-09-20T22:20:53.673 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.673 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4db0680:::benchmark_data_smithi060_196073_object9167:head# 2024-09-20T22:20:53.673 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.674 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4dbe494:::benchmark_data_smithi060_196073_object3016:head# 2024-09-20T22:20:53.674 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.674 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4dc0151:::benchmark_data_smithi060_196073_object8452:head# 2024-09-20T22:20:53.674 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.675 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:20:53.675 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:53 smithi060 ceph-mon[29694]: pgmap v960: 178 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.7 GiB data, 18 GiB used, 519 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.16k op/s; 3185/231144 objects degraded (1.378%); 11830/231144 objects misplaced (5.118%); 26 MiB/s, 423 objects/s recovering 2024-09-20T22:20:53.675 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:53 smithi060 ceph-mon[29694]: osdmap e505: 8 total, 7 up, 6 in 2024-09-20T22:20:53.675 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:53 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3186488298' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:20:53.675 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:53 smithi060 ceph-mon[40290]: pgmap v960: 178 pgs: 2 active+remapped+backfilling, 3 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 172 active+clean; 4.7 GiB data, 18 GiB used, 519 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1.16k op/s; 3185/231144 objects degraded (1.378%); 11830/231144 objects misplaced (5.118%); 26 MiB/s, 423 objects/s recovering 2024-09-20T22:20:53.676 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:53 smithi060 ceph-mon[40290]: osdmap e505: 8 total, 7 up, 6 in 2024-09-20T22:20:53.676 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:53 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3186488298' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:20:53.677 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:53.677 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4dcae66:::benchmark_data_smithi060_196073_object53751:head# 2024-09-20T22:20:53.677 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.677 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4ddb205:::benchmark_data_smithi060_196073_object32505:head# 2024-09-20T22:20:53.677 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.678 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:53.678 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:53.678 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4e01048:::benchmark_data_smithi060_196073_object31056:head# 2024-09-20T22:20:53.678 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.679 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4e1ea08:::benchmark_data_smithi060_196073_object18558:head# 2024-09-20T22:20:53.679 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.679 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4e1eae5:::benchmark_data_smithi060_196073_object33985:head# 2024-09-20T22:20:53.679 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.679 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4e2b3b8:::benchmark_data_smithi060_196073_object57328:head# 2024-09-20T22:20:53.680 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.680 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4e5526d:::benchmark_data_smithi060_196073_object55510:head# 2024-09-20T22:20:53.680 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.680 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4e896a1:::benchmark_data_smithi060_196073_object5059:head# 2024-09-20T22:20:53.681 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.681 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4e8a6cb:::benchmark_data_smithi060_196073_object38616:head# 2024-09-20T22:20:53.681 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.681 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4eaa75b:::benchmark_data_smithi060_196073_object11424:head# 2024-09-20T22:20:53.682 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.682 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4ee0350:::benchmark_data_smithi060_196073_object56729:head# 2024-09-20T22:20:53.682 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.682 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4ef1232:::benchmark_data_smithi060_196073_object456:head# 2024-09-20T22:20:53.682 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.683 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4ef34dc:::benchmark_data_smithi060_196073_object6628:head# 2024-09-20T22:20:53.683 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.683 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4f162a7:::benchmark_data_smithi060_196073_object55160:head# 2024-09-20T22:20:53.683 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.684 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4f4a3a7:::benchmark_data_smithi060_196073_object54665:head# 2024-09-20T22:20:53.684 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.684 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4f632e1:::benchmark_data_smithi060_196073_object14300:head# 2024-09-20T22:20:53.684 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.685 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4f65e77:::benchmark_data_smithi060_196073_object47671:head# 2024-09-20T22:20:53.685 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.685 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4f684f7:::benchmark_data_smithi060_196073_object43159:head# 2024-09-20T22:20:53.685 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.686 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4fabc3d:::benchmark_data_smithi060_196073_object61587:head# 2024-09-20T22:20:53.686 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.686 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4fbcc1b:::benchmark_data_smithi060_196073_object6492:head# 2024-09-20T22:20:53.686 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.687 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4fc5591:::benchmark_data_smithi060_196073_object55224:head# 2024-09-20T22:20:53.687 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.687 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4fea836:::benchmark_data_smithi060_196073_object55289:head# 2024-09-20T22:20:53.687 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.688 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4ff20e9:::benchmark_data_smithi060_196073_object2716:head# 2024-09-20T22:20:53.688 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.688 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b4ffb4e6:::benchmark_data_smithi060_196073_object7266:head# 2024-09-20T22:20:53.688 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.689 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b502b818:::benchmark_data_smithi060_196073_object56796:head# 2024-09-20T22:20:53.689 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.689 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b50448bd:::benchmark_data_smithi060_196073_object31770:head# 2024-09-20T22:20:53.689 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.690 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b50512ef:::benchmark_data_smithi060_196073_object21675:head# 2024-09-20T22:20:53.690 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.690 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b50518de:::benchmark_data_smithi060_196073_object49415:head# 2024-09-20T22:20:53.690 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.691 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5054b67:::benchmark_data_smithi060_196073_object31639:head# 2024-09-20T22:20:53.691 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.691 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5060808:::benchmark_data_smithi060_196073_object4967:head# 2024-09-20T22:20:53.691 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.692 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5066a84:::benchmark_data_smithi060_196073_object30709:head# 2024-09-20T22:20:53.692 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.692 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5068418:::benchmark_data_smithi060_196073_object24778:head# 2024-09-20T22:20:53.692 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.693 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b50708f0:::benchmark_data_smithi060_196073_object23017:head# 2024-09-20T22:20:53.693 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.693 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5081641:::benchmark_data_smithi060_196073_object36482:head# 2024-09-20T22:20:53.693 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.694 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b50c088b:::benchmark_data_smithi060_196073_object194:head# 2024-09-20T22:20:53.694 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.694 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b50c5a80:::benchmark_data_smithi060_196073_object14784:head# 2024-09-20T22:20:53.694 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.695 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b50e7b6e:::benchmark_data_smithi060_196073_object51966:head# 2024-09-20T22:20:53.695 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.695 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b50ea899:::benchmark_data_smithi060_196073_object39102:head# 2024-09-20T22:20:53.695 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.696 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b510d62c:::benchmark_data_smithi060_196073_object50007:head# 2024-09-20T22:20:53.696 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.696 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b51188ba:::benchmark_data_smithi060_196073_object21811:head# 2024-09-20T22:20:53.696 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.697 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b511dfb4:::benchmark_data_smithi060_196073_object32768:head# 2024-09-20T22:20:53.697 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.697 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b512176a:::benchmark_data_smithi060_196073_object12674:head# 2024-09-20T22:20:53.697 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.698 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b51490ab:::benchmark_data_smithi060_196073_object3304:head# 2024-09-20T22:20:53.698 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.698 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b515010e:::benchmark_data_smithi060_196073_object26849:head# 2024-09-20T22:20:53.698 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.699 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5164324:::benchmark_data_smithi060_196073_object13754:head# 2024-09-20T22:20:53.699 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.699 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b51892f9:::benchmark_data_smithi060_196073_object45222:head# 2024-09-20T22:20:53.699 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.700 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5195a07:::benchmark_data_smithi060_196073_object476:head# 2024-09-20T22:20:53.700 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.700 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b51ab719:::benchmark_data_smithi060_196073_object18337:head# 2024-09-20T22:20:53.700 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.701 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b51dc147:::benchmark_data_smithi060_196073_object28540:head# 2024-09-20T22:20:53.701 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.701 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b51e5e10:::benchmark_data_smithi060_196073_object21998:head# 2024-09-20T22:20:53.701 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.702 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b51e8cea:::benchmark_data_smithi060_196073_object60303:head# 2024-09-20T22:20:53.702 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.702 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5222578:::benchmark_data_smithi060_196073_object61701:head# 2024-09-20T22:20:53.702 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.703 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5228581:::benchmark_data_smithi060_196073_object32953:head# 2024-09-20T22:20:53.703 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.703 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b52302d8:::benchmark_data_smithi060_196073_object48827:head# 2024-09-20T22:20:53.703 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.704 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b525a928:::benchmark_data_smithi060_196073_object51745:head# 2024-09-20T22:20:53.704 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.704 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b525baa7:::benchmark_data_smithi060_196073_object48935:head# 2024-09-20T22:20:53.704 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.705 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b526b6f2:::benchmark_data_smithi060_196073_object9987:head# 2024-09-20T22:20:53.705 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.705 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b52743e8:::benchmark_data_smithi060_196073_object2293:head# 2024-09-20T22:20:53.705 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.705 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5282dd2:::benchmark_data_smithi060_196073_object18149:head# 2024-09-20T22:20:53.706 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.706 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5283060:::benchmark_data_smithi060_196073_object9158:head# 2024-09-20T22:20:53.706 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.707 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5287171:::benchmark_data_smithi060_196073_object56667:head# 2024-09-20T22:20:53.707 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.707 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5287e92:::benchmark_data_smithi060_196073_object18527:head# 2024-09-20T22:20:53.707 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.707 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b52977ab:::benchmark_data_smithi060_196073_object46227:head# 2024-09-20T22:20:53.708 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.708 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b52c4e96:::benchmark_data_smithi060_196073_object19504:head# 2024-09-20T22:20:53.708 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.708 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b52e0ddc:::benchmark_data_smithi060_196073_object57884:head# 2024-09-20T22:20:53.708 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.709 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b52eccf0:::benchmark_data_smithi060_196073_object25428:head# 2024-09-20T22:20:53.709 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.709 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b52f7a07:::benchmark_data_smithi060_196073_object3065:head# 2024-09-20T22:20:53.709 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.709 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5300b63:::benchmark_data_smithi060_196073_object11258:head# 2024-09-20T22:20:53.709 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.710 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b530460e:::benchmark_data_smithi060_196073_object34824:head# 2024-09-20T22:20:53.710 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.710 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5316bd1:::benchmark_data_smithi060_196073_object13951:head# 2024-09-20T22:20:53.710 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.711 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5320275:::benchmark_data_smithi060_196073_object10236:head# 2024-09-20T22:20:53.711 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.711 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b533af25:::benchmark_data_smithi060_196073_object52205:head# 2024-09-20T22:20:53.711 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.712 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5344f87:::benchmark_data_smithi060_196073_object14988:head# 2024-09-20T22:20:53.712 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.712 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b537b1f2:::benchmark_data_smithi060_196073_object27722:head# 2024-09-20T22:20:53.712 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.713 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b538fa66:::benchmark_data_smithi060_196073_object11940:head# 2024-09-20T22:20:53.713 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.713 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b53a2859:::benchmark_data_smithi060_196073_object58770:head# 2024-09-20T22:20:53.713 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.713 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b53a5d62:::benchmark_data_smithi060_196073_object58085:head# 2024-09-20T22:20:53.713 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.714 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b53b89d1:::benchmark_data_smithi060_196073_object45245:head# 2024-09-20T22:20:53.714 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.714 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b53d24d2:::benchmark_data_smithi060_196073_object56141:head# 2024-09-20T22:20:53.714 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.714 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b53d6290:::benchmark_data_smithi060_196073_object30156:head# 2024-09-20T22:20:53.714 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.715 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b53df96d:::benchmark_data_smithi060_196073_object13125:head# 2024-09-20T22:20:53.715 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.715 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b53fd5e9:::benchmark_data_smithi060_196073_object24658:head# 2024-09-20T22:20:53.715 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.716 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b542c473:::benchmark_data_smithi060_196073_object36600:head# 2024-09-20T22:20:53.716 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.716 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b544ab79:::benchmark_data_smithi060_196073_object42925:head# 2024-09-20T22:20:53.716 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.717 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5459a4c:::benchmark_data_smithi060_196073_object2380:head# 2024-09-20T22:20:53.717 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.717 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5490082:::benchmark_data_smithi060_196073_object37877:head# 2024-09-20T22:20:53.717 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.718 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b549f605:::benchmark_data_smithi060_196073_object12387:head# 2024-09-20T22:20:53.718 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.718 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b54a2588:::benchmark_data_smithi060_196073_object15412:head# 2024-09-20T22:20:53.718 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.718 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b54b073b:::benchmark_data_smithi060_196073_object13844:head# 2024-09-20T22:20:53.718 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.719 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b54b3e8d:::benchmark_data_smithi060_196073_object6424:head# 2024-09-20T22:20:53.719 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.719 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b54c2ab6:::benchmark_data_smithi060_196073_object53115:head# 2024-09-20T22:20:53.719 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.719 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b54c5ad4:::benchmark_data_smithi060_196073_object32451:head# 2024-09-20T22:20:53.720 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.720 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T22:20:53.722 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b54ced16:::benchmark_data_smithi060_196073_object43787:head# 2024-09-20T22:20:53.722 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.722 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b54e5e09:::benchmark_data_smithi060_196073_object144:head# 2024-09-20T22:20:53.722 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.722 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b54f145d:::benchmark_data_smithi060_196073_object9292:head# 2024-09-20T22:20:53.722 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.722 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5501e84:::benchmark_data_smithi060_196073_object4298:head# 2024-09-20T22:20:53.722 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.722 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b550f139:::benchmark_data_smithi060_196073_object14331:head# 2024-09-20T22:20:53.722 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.723 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5524eea:::benchmark_data_smithi060_196073_object42506:head# 2024-09-20T22:20:53.723 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.723 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5549b5c:::benchmark_data_smithi060_196073_object3483:head# 2024-09-20T22:20:53.723 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.723 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b558af68:::benchmark_data_smithi060_196073_object44322:head# 2024-09-20T22:20:53.724 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.724 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5592d52:::benchmark_data_smithi060_196073_object28828:head# 2024-09-20T22:20:53.725 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.725 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b559c2cc:::benchmark_data_smithi060_196073_object28330:head# 2024-09-20T22:20:53.725 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.726 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b559ff67:::benchmark_data_smithi060_196073_object33205:head# 2024-09-20T22:20:53.726 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.726 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b55b9778:::benchmark_data_smithi060_196073_object44283:head# 2024-09-20T22:20:53.727 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.727 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5600abe:::benchmark_data_smithi060_196073_object54163:head# 2024-09-20T22:20:53.727 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.727 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b565354e:::benchmark_data_smithi060_196073_object35071:head# 2024-09-20T22:20:53.727 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.728 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b565addc:::benchmark_data_smithi060_196073_object31201:head# 2024-09-20T22:20:53.728 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.728 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b565b4a0:::benchmark_data_smithi060_196073_object42469:head# 2024-09-20T22:20:53.728 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.729 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b567710b:::benchmark_data_smithi060_196073_object28716:head# 2024-09-20T22:20:53.729 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.729 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5677a6c:::benchmark_data_smithi060_196073_object12769:head# 2024-09-20T22:20:53.729 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.730 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b568944f:::benchmark_data_smithi060_196073_object40779:head# 2024-09-20T22:20:53.730 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.730 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b56a29ce:::benchmark_data_smithi060_196073_object26850:head# 2024-09-20T22:20:53.730 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.730 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b56bd718:::benchmark_data_smithi060_196073_object50136:head# 2024-09-20T22:20:53.731 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.731 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b56d3307:::benchmark_data_smithi060_196073_object4873:head# 2024-09-20T22:20:53.731 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.731 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b56da2ea:::benchmark_data_smithi060_196073_object34248:head# 2024-09-20T22:20:53.731 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.732 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b56e222e:::benchmark_data_smithi060_196073_object23633:head# 2024-09-20T22:20:53.732 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.732 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5703e0b:::benchmark_data_smithi060_196073_object3457:head# 2024-09-20T22:20:53.732 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.732 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b570a489:::benchmark_data_smithi060_196073_object61130:head# 2024-09-20T22:20:53.732 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.733 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b57129f7:::benchmark_data_smithi060_196073_object48271:head# 2024-09-20T22:20:53.733 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.733 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b571bdde:::benchmark_data_smithi060_196073_object2793:head# 2024-09-20T22:20:53.733 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.734 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b572934e:::benchmark_data_smithi060_196073_object13930:head# 2024-09-20T22:20:53.734 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.734 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b572fe81:::benchmark_data_smithi060_196073_object9416:head# 2024-09-20T22:20:53.734 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.735 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5783607:::benchmark_data_smithi060_196073_object37514:head# 2024-09-20T22:20:53.740 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.740 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5795fed:::benchmark_data_smithi060_196073_object34132:head# 2024-09-20T22:20:53.740 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.741 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b579fc84:::benchmark_data_smithi060_196073_object40975:head# 2024-09-20T22:20:53.741 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.741 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b57a2ac2:::benchmark_data_smithi060_196073_object20639:head# 2024-09-20T22:20:53.741 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.742 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b57a65c7:::benchmark_data_smithi060_196073_object28520:head# 2024-09-20T22:20:53.742 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.742 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b57a7430:::benchmark_data_smithi060_196073_object5741:head# 2024-09-20T22:20:53.742 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.743 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b57c8de3:::benchmark_data_smithi060_196073_object60107:head# 2024-09-20T22:20:53.743 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.743 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b57dd46d:::benchmark_data_smithi060_196073_object15231:head# 2024-09-20T22:20:53.743 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.744 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b57ea6db:::benchmark_data_smithi060_196073_object8462:head# 2024-09-20T22:20:53.744 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.744 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b57f8096:::benchmark_data_smithi060_196073_object57975:head# 2024-09-20T22:20:53.744 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.745 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b57fb179:::benchmark_data_smithi060_196073_object54519:head# 2024-09-20T22:20:53.745 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.745 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b58572ce:::benchmark_data_smithi060_196073_object37776:head# 2024-09-20T22:20:53.745 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.746 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5882c60:::benchmark_data_smithi060_196073_object23947:head# 2024-09-20T22:20:53.746 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.746 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5896224:::benchmark_data_smithi060_196073_object19526:head# 2024-09-20T22:20:53.746 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.746 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b58a8757:::benchmark_data_smithi060_196073_object42179:head# 2024-09-20T22:20:53.747 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.747 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b58b85ec:::benchmark_data_smithi060_196073_object48051:head# 2024-09-20T22:20:53.747 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.747 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b58d472d:::benchmark_data_smithi060_196073_object933:head# 2024-09-20T22:20:53.748 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.748 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b58d98d2:::benchmark_data_smithi060_196073_object26021:head# 2024-09-20T22:20:53.748 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.748 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b58de940:::benchmark_data_smithi060_196073_object49721:head# 2024-09-20T22:20:53.749 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.749 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b58e606a:::benchmark_data_smithi060_196073_object18626:head# 2024-09-20T22:20:53.749 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.749 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b58e917b:::benchmark_data_smithi060_196073_object61219:head# 2024-09-20T22:20:53.749 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.750 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b58f063a:::benchmark_data_smithi060_196073_object11942:head# 2024-09-20T22:20:53.750 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.750 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b58f1758:::benchmark_data_smithi060_196073_object33186:head# 2024-09-20T22:20:53.751 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.751 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b58fb1c6:::benchmark_data_smithi060_196073_object23260:head# 2024-09-20T22:20:53.751 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.751 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5908840:::benchmark_data_smithi060_196073_object57290:head# 2024-09-20T22:20:53.752 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.752 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b59098ad:::benchmark_data_smithi060_196073_object61227:head# 2024-09-20T22:20:53.752 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.752 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5922a84:::benchmark_data_smithi060_196073_object18188:head# 2024-09-20T22:20:53.753 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.753 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5924afa:::benchmark_data_smithi060_196073_object37271:head# 2024-09-20T22:20:53.753 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.753 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b596baba:::benchmark_data_smithi060_196073_object28850:head# 2024-09-20T22:20:53.753 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.754 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b59705aa:::benchmark_data_smithi060_196073_object36558:head# 2024-09-20T22:20:53.754 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.754 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b598a307:::benchmark_data_smithi060_196073_object58197:head# 2024-09-20T22:20:53.754 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.755 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b598f031:::benchmark_data_smithi060_196073_object10740:head# 2024-09-20T22:20:53.755 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.755 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5991d1c:::benchmark_data_smithi060_196073_object15538:head# 2024-09-20T22:20:53.755 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.756 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b599b96f:::benchmark_data_smithi060_196073_object45272:head# 2024-09-20T22:20:53.756 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.756 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b59b270e:::benchmark_data_smithi060_196073_object18785:head# 2024-09-20T22:20:53.756 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.757 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b59c4dd6:::benchmark_data_smithi060_196073_object1627:head# 2024-09-20T22:20:53.757 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.757 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b59cb565:::benchmark_data_smithi060_196073_object54915:head# 2024-09-20T22:20:53.757 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.758 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5a05b2a:::benchmark_data_smithi060_196073_object39432:head# 2024-09-20T22:20:53.758 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.758 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5a168e4:::benchmark_data_smithi060_196073_object30276:head# 2024-09-20T22:20:53.758 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.759 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5a217c8:::benchmark_data_smithi060_196073_object34840:head# 2024-09-20T22:20:53.759 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.759 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5a22a2d:::benchmark_data_smithi060_196073_object29169:head# 2024-09-20T22:20:53.759 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.760 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5a29521:::benchmark_data_smithi060_196073_object44320:head# 2024-09-20T22:20:53.760 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.760 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5a3ddfd:::benchmark_data_smithi060_196073_object5540:head# 2024-09-20T22:20:53.760 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.760 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5a516d7:::benchmark_data_smithi060_196073_object37567:head# 2024-09-20T22:20:53.760 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.761 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5a53cf1:::benchmark_data_smithi060_196073_object16468:head# 2024-09-20T22:20:53.761 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.762 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5a550cb:::benchmark_data_smithi060_196073_object58563:head# 2024-09-20T22:20:53.762 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.763 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5a669ae:::benchmark_data_smithi060_196073_object21714:head# 2024-09-20T22:20:53.763 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.763 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5a670f2:::benchmark_data_smithi060_196073_object40952:head# 2024-09-20T22:20:53.763 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.763 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5a72d65:::benchmark_data_smithi060_196073_object10370:head# 2024-09-20T22:20:53.763 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.763 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5a8124a:::benchmark_data_smithi060_196073_object55167:head# 2024-09-20T22:20:53.763 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.764 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5a8d4f1:::benchmark_data_smithi060_196073_object36810:head# 2024-09-20T22:20:53.764 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.764 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5a9eefc:::benchmark_data_smithi060_196073_object19762:head# 2024-09-20T22:20:53.764 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.764 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5aad28f:::benchmark_data_smithi060_196073_object36537:head# 2024-09-20T22:20:53.764 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.765 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5ab5004:::benchmark_data_smithi060_196073_object41503:head# 2024-09-20T22:20:53.765 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.765 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5ac820c:::benchmark_data_smithi060_196073_object23448:head# 2024-09-20T22:20:53.765 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.766 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5ad1723:::benchmark_data_smithi060_196073_object44499:head# 2024-09-20T22:20:53.767 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.767 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5ad9b3d:::benchmark_data_smithi060_196073_object52560:head# 2024-09-20T22:20:53.767 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.768 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5adbc46:::benchmark_data_smithi060_196073_object20737:head# 2024-09-20T22:20:53.768 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.768 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5adfd98:::benchmark_data_smithi060_196073_object4653:head# 2024-09-20T22:20:53.768 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.769 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5af240e:::benchmark_data_smithi060_196073_object51726:head# 2024-09-20T22:20:53.769 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.769 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5b08909:::benchmark_data_smithi060_196073_object40272:head# 2024-09-20T22:20:53.769 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.770 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5b146a5:::benchmark_data_smithi060_196073_object6565:head# 2024-09-20T22:20:53.770 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.770 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5b156b5:::benchmark_data_smithi060_196073_object44629:head# 2024-09-20T22:20:53.770 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.771 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5b1fff0:::benchmark_data_smithi060_196073_object37071:head# 2024-09-20T22:20:53.771 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.771 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5b265e1:::benchmark_data_smithi060_196073_object55824:head# 2024-09-20T22:20:53.771 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.771 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5b3b717:::benchmark_data_smithi060_196073_object25161:head# 2024-09-20T22:20:53.771 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.772 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5b48717:::benchmark_data_smithi060_196073_object58271:head# 2024-09-20T22:20:53.772 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.772 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5b580ae:::benchmark_data_smithi060_196073_object8553:head# 2024-09-20T22:20:53.772 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.773 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5b6eaf3:::benchmark_data_smithi060_196073_object6134:head# 2024-09-20T22:20:53.773 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.773 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5b75908:::benchmark_data_smithi060_196073_object16859:head# 2024-09-20T22:20:53.773 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.774 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5bb46ee:::benchmark_data_smithi060_196073_object48657:head# 2024-09-20T22:20:53.774 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.774 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5bdac34:::benchmark_data_smithi060_196073_object9165:head# 2024-09-20T22:20:53.774 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.774 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5bdae67:::benchmark_data_smithi060_196073_object56031:head# 2024-09-20T22:20:53.774 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.775 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5be6d54:::benchmark_data_smithi060_196073_object38961:head# 2024-09-20T22:20:53.775 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.775 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5be848a:::benchmark_data_smithi060_196073_object32878:head# 2024-09-20T22:20:53.775 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.776 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5bf3754:::benchmark_data_smithi060_196073_object17420:head# 2024-09-20T22:20:53.776 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.776 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5bfa013:::benchmark_data_smithi060_196073_object29797:head# 2024-09-20T22:20:53.776 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.777 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5c0ddd9:::benchmark_data_smithi060_196073_object39021:head# 2024-09-20T22:20:53.777 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.777 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5c176f8:::benchmark_data_smithi060_196073_object348:head# 2024-09-20T22:20:53.777 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.777 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5c3595d:::benchmark_data_smithi060_196073_object30702:head# 2024-09-20T22:20:53.778 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.778 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5c4a05b:::benchmark_data_smithi060_196073_object533:head# 2024-09-20T22:20:53.778 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.778 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5c5156e:::benchmark_data_smithi060_196073_object49436:head# 2024-09-20T22:20:53.778 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.778 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:53.779 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5c5dfcf:::benchmark_data_smithi060_196073_object51033:head# 2024-09-20T22:20:53.779 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.779 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5c5ea87:::benchmark_data_smithi060_196073_object15899:head# 2024-09-20T22:20:53.779 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.780 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5c9fb9b:::benchmark_data_smithi060_196073_object11138:head# 2024-09-20T22:20:53.780 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.781 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5cc04f1:::benchmark_data_smithi060_196073_object3544:head# 2024-09-20T22:20:53.781 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.781 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5cc0993:::benchmark_data_smithi060_196073_object18273:head# 2024-09-20T22:20:53.781 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.781 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5cdf5f3:::benchmark_data_smithi060_196073_object43238:head# 2024-09-20T22:20:53.781 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.781 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5cef808:::benchmark_data_smithi060_196073_object48183:head# 2024-09-20T22:20:53.781 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.783 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5cf23c2:::benchmark_data_smithi060_196073_object45131:head# 2024-09-20T22:20:53.783 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.783 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5d24e92:::benchmark_data_smithi060_196073_object11685:head# 2024-09-20T22:20:53.783 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.783 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5d2db6f:::benchmark_data_smithi060_196073_object52307:head# 2024-09-20T22:20:53.783 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.783 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5d3104d:::benchmark_data_smithi060_196073_object44177:head# 2024-09-20T22:20:53.783 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.783 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5d41b93:::benchmark_data_smithi060_196073_object54068:head# 2024-09-20T22:20:53.783 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.784 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5d6bac9:::benchmark_data_smithi060_196073_object58453:head# 2024-09-20T22:20:53.784 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.784 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5d704cb:::benchmark_data_smithi060_196073_object35005:head# 2024-09-20T22:20:53.784 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.784 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5d85196:::benchmark_data_smithi060_196073_object15025:head# 2024-09-20T22:20:53.784 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.785 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5d9da0c:::benchmark_data_smithi060_196073_object40367:head# 2024-09-20T22:20:53.785 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.785 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5dc1e2e:::benchmark_data_smithi060_196073_object10469:head# 2024-09-20T22:20:53.785 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.786 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5dda797:::benchmark_data_smithi060_196073_object57691:head# 2024-09-20T22:20:53.786 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.786 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5ddff8b:::benchmark_data_smithi060_196073_object28127:head# 2024-09-20T22:20:53.786 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.787 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5de6fc3:::benchmark_data_smithi060_196073_object27159:head# 2024-09-20T22:20:53.787 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.787 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5dec37f:::benchmark_data_smithi060_196073_object46995:head# 2024-09-20T22:20:53.787 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.787 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5defa5c:::benchmark_data_smithi060_196073_object19740:head# 2024-09-20T22:20:53.787 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.788 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5df1d4e:::benchmark_data_smithi060_196073_object57566:head# 2024-09-20T22:20:53.788 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.788 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5e053bf:::benchmark_data_smithi060_196073_object31650:head# 2024-09-20T22:20:53.788 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.788 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5e09db5:::benchmark_data_smithi060_196073_object56627:head# 2024-09-20T22:20:53.789 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.789 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5e0d0a8:::benchmark_data_smithi060_196073_object18500:head# 2024-09-20T22:20:53.789 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.789 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5e18c04:::benchmark_data_smithi060_196073_object52402:head# 2024-09-20T22:20:53.790 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.790 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5e239e1:::benchmark_data_smithi060_196073_object4609:head# 2024-09-20T22:20:53.790 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.790 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5e26834:::benchmark_data_smithi060_196073_object24348:head# 2024-09-20T22:20:53.790 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.791 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5e3dc55:::benchmark_data_smithi060_196073_object50240:head# 2024-09-20T22:20:53.791 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.791 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5e4ce7f:::benchmark_data_smithi060_196073_object4717:head# 2024-09-20T22:20:53.791 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.791 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5e578b3:::benchmark_data_smithi060_196073_object49072:head# 2024-09-20T22:20:53.791 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.791 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5e5c3a1:::benchmark_data_smithi060_196073_object60722:head# 2024-09-20T22:20:53.792 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.792 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5e626fa:::benchmark_data_smithi060_196073_object34332:head# 2024-09-20T22:20:53.792 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.792 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5e81a23:::benchmark_data_smithi060_196073_object32368:head# 2024-09-20T22:20:53.792 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.793 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5e84ce4:::benchmark_data_smithi060_196073_object48358:head# 2024-09-20T22:20:53.793 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.793 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5e975ca:::benchmark_data_smithi060_196073_object13016:head# 2024-09-20T22:20:53.793 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.793 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5ea75b2:::benchmark_data_smithi060_196073_object15727:head# 2024-09-20T22:20:53.793 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.793 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5eb6783:::benchmark_data_smithi060_196073_object52348:head# 2024-09-20T22:20:53.793 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.794 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5eb91f1:::benchmark_data_smithi060_196073_object20777:head# 2024-09-20T22:20:53.794 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.794 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5ec8cf1:::benchmark_data_smithi060_196073_object25221:head# 2024-09-20T22:20:53.794 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.794 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5eca7bb:::benchmark_data_smithi060_196073_object56725:head# 2024-09-20T22:20:53.794 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.795 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5ecb292:::benchmark_data_smithi060_196073_object7146:head# 2024-09-20T22:20:53.795 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.795 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5eedc00:::benchmark_data_smithi060_196073_object36318:head# 2024-09-20T22:20:53.795 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.795 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5eee940:::benchmark_data_smithi060_196073_object50832:head# 2024-09-20T22:20:53.803 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.803 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5efc6a3:::benchmark_data_smithi060_196073_object44727:head# 2024-09-20T22:20:53.804 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.804 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5f0e2c7:::benchmark_data_smithi060_196073_object2339:head# 2024-09-20T22:20:53.804 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.805 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5f23c5a:::benchmark_data_smithi060_196073_object57614:head# 2024-09-20T22:20:53.805 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.805 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5f27214:::benchmark_data_smithi060_196073_object24573:head# 2024-09-20T22:20:53.805 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.806 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5f2dac8:::benchmark_data_smithi060_196073_object35817:head# 2024-09-20T22:20:53.806 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.806 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5f34382:::benchmark_data_smithi060_196073_object43853:head# 2024-09-20T22:20:53.806 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.806 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5f75cd5:::benchmark_data_smithi060_196073_object23892:head# 2024-09-20T22:20:53.807 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.807 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5f8a008:::benchmark_data_smithi060_196073_object5895:head# 2024-09-20T22:20:53.807 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.807 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5f91bf1:::benchmark_data_smithi060_196073_object39436:head# 2024-09-20T22:20:53.807 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.808 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5fb9e3b:::benchmark_data_smithi060_196073_object50687:head# 2024-09-20T22:20:53.808 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.808 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5fc8aac:::benchmark_data_smithi060_196073_object51938:head# 2024-09-20T22:20:53.808 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.808 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5fd3619:::benchmark_data_smithi060_196073_object26224:head# 2024-09-20T22:20:53.808 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.809 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5fd6ae1:::benchmark_data_smithi060_196073_object10217:head# 2024-09-20T22:20:53.809 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.809 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5fe59bb:::benchmark_data_smithi060_196073_object44458:head# 2024-09-20T22:20:53.809 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.810 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b5feb9a5:::benchmark_data_smithi060_196073_object8410:head# 2024-09-20T22:20:53.810 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.810 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b600115c:::benchmark_data_smithi060_196073_object41931:head# 2024-09-20T22:20:53.810 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.811 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6002146:::benchmark_data_smithi060_196073_object55798:head# 2024-09-20T22:20:53.811 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.811 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b601c54f:::benchmark_data_smithi060_196073_object18781:head# 2024-09-20T22:20:53.811 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.811 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b60323a9:::benchmark_data_smithi060_196073_object9298:head# 2024-09-20T22:20:53.812 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.812 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6063d65:::benchmark_data_smithi060_196073_object14774:head# 2024-09-20T22:20:53.812 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.813 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:53.814 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:53.814 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:53.815 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6081ad6:::benchmark_data_smithi060_196073_object15190:head# 2024-09-20T22:20:53.815 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.815 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b609851a:::benchmark_data_smithi060_196073_object56985:head# 2024-09-20T22:20:53.815 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.815 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b609f449:::benchmark_data_smithi060_196073_object54014:head# 2024-09-20T22:20:53.815 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.815 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b60ae46e:::benchmark_data_smithi060_196073_object24744:head# 2024-09-20T22:20:53.815 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.815 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b60aed19:::benchmark_data_smithi060_196073_object20656:head# 2024-09-20T22:20:53.815 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.815 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b60be756:::benchmark_data_smithi060_196073_object15194:head# 2024-09-20T22:20:53.815 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.815 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b60d4cc3:::benchmark_data_smithi060_196073_object25642:head# 2024-09-20T22:20:53.815 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.816 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b60d7dfe:::benchmark_data_smithi060_196073_object15779:head# 2024-09-20T22:20:53.816 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.816 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b60ea40b:::benchmark_data_smithi060_196073_object41478:head# 2024-09-20T22:20:53.817 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.817 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b60fa58e:::benchmark_data_smithi060_196073_object50105:head# 2024-09-20T22:20:53.817 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.817 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6106d8b:::benchmark_data_smithi060_196073_object9638:head# 2024-09-20T22:20:53.817 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.818 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b612db1d:::benchmark_data_smithi060_196073_object59517:head# 2024-09-20T22:20:53.818 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.818 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b613389f:::benchmark_data_smithi060_196073_object38080:head# 2024-09-20T22:20:53.818 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.818 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b614723a:::benchmark_data_smithi060_196073_object42266:head# 2024-09-20T22:20:53.818 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.819 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b615016e:::benchmark_data_smithi060_196073_object55153:head# 2024-09-20T22:20:53.819 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.819 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b615a57d:::benchmark_data_smithi060_196073_object494:head# 2024-09-20T22:20:53.819 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.819 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b616069c:::benchmark_data_smithi060_196073_object13491:head# 2024-09-20T22:20:53.819 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.819 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6162045:::benchmark_data_smithi060_196073_object55116:head# 2024-09-20T22:20:53.820 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.820 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b616e425:::benchmark_data_smithi060_196073_object54678:head# 2024-09-20T22:20:53.820 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.820 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b617779e:::benchmark_data_smithi060_196073_object56148:head# 2024-09-20T22:20:53.820 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.821 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b61a57fa:::benchmark_data_smithi060_196073_object42157:head# 2024-09-20T22:20:53.821 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.821 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b61dadf5:::benchmark_data_smithi060_196073_object61069:head# 2024-09-20T22:20:53.821 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.821 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b61ea752:::benchmark_data_smithi060_196073_object52751:head# 2024-09-20T22:20:53.822 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.822 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b61fb5c5:::benchmark_data_smithi060_196073_object45402:head# 2024-09-20T22:20:53.822 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.822 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b620efb9:::benchmark_data_smithi060_196073_object15763:head# 2024-09-20T22:20:53.822 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.823 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6214bba:::benchmark_data_smithi060_196073_object38678:head# 2024-09-20T22:20:53.823 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.823 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6223849:::benchmark_data_smithi060_196073_object44345:head# 2024-09-20T22:20:53.823 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.824 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b62269e9:::benchmark_data_smithi060_196073_object57948:head# 2024-09-20T22:20:53.824 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.824 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6246fe2:::benchmark_data_smithi060_196073_object9354:head# 2024-09-20T22:20:53.825 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.825 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b624c89e:::benchmark_data_smithi060_196073_object40009:head# 2024-09-20T22:20:53.825 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.825 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6260f94:::benchmark_data_smithi060_196073_object12100:head# 2024-09-20T22:20:53.825 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.825 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6261160:::benchmark_data_smithi060_196073_object58918:head# 2024-09-20T22:20:53.825 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.826 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6265d4b:::benchmark_data_smithi060_196073_object26784:head# 2024-09-20T22:20:53.826 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.827 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6270012:::benchmark_data_smithi060_196073_object4015:head# 2024-09-20T22:20:53.827 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.827 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b62708ac:::benchmark_data_smithi060_196073_object6180:head# 2024-09-20T22:20:53.827 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.827 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6277450:::benchmark_data_smithi060_196073_object51036:head# 2024-09-20T22:20:53.827 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.828 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b627e6e2:::benchmark_data_smithi060_196073_object4840:head# 2024-09-20T22:20:53.828 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.828 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b627fa42:::benchmark_data_smithi060_196073_object43219:head# 2024-09-20T22:20:53.828 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.828 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b628e68d:::benchmark_data_smithi060_196073_object41206:head# 2024-09-20T22:20:53.828 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.829 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b62973e8:::benchmark_data_smithi060_196073_object55678:head# 2024-09-20T22:20:53.829 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.829 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b629bbf0:::benchmark_data_smithi060_196073_object26288:head# 2024-09-20T22:20:53.829 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.830 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b62cded8:::benchmark_data_smithi060_196073_object41366:head# 2024-09-20T22:20:53.830 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.830 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b62ce934:::benchmark_data_smithi060_196073_object61193:head# 2024-09-20T22:20:53.830 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.831 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b62d9172:::benchmark_data_smithi060_196073_object21389:head# 2024-09-20T22:20:53.831 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.831 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b62deb41:::benchmark_data_smithi060_196073_object49337:head# 2024-09-20T22:20:53.831 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.832 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b62eabfc:::benchmark_data_smithi060_196073_object23386:head# 2024-09-20T22:20:53.832 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.832 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b62ed327:::benchmark_data_smithi060_196073_object3879:head# 2024-09-20T22:20:53.832 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.832 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b62fa2c5:::benchmark_data_smithi060_196073_object40013:head# 2024-09-20T22:20:53.833 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.833 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6314864:::benchmark_data_smithi060_196073_object12913:head# 2024-09-20T22:20:53.833 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.833 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b631e799:::benchmark_data_smithi060_196073_object48510:head# 2024-09-20T22:20:53.833 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.833 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b633103b:::benchmark_data_smithi060_196073_object52999:head# 2024-09-20T22:20:53.833 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.834 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6336f6b:::benchmark_data_smithi060_196073_object34023:head# 2024-09-20T22:20:53.834 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.834 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6347fe1:::benchmark_data_smithi060_196073_object11974:head# 2024-09-20T22:20:53.834 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.835 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b636175c:::benchmark_data_smithi060_196073_object21706:head# 2024-09-20T22:20:53.835 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.835 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6363d95:::benchmark_data_smithi060_196073_object17752:head# 2024-09-20T22:20:53.835 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.835 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6378a2f:::benchmark_data_smithi060_196073_object17902:head# 2024-09-20T22:20:53.835 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.836 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b63a741a:::benchmark_data_smithi060_196073_object21454:head# 2024-09-20T22:20:53.836 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.836 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b63ad6e2:::benchmark_data_smithi060_196073_object56169:head# 2024-09-20T22:20:53.837 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.837 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b63ae6be:::benchmark_data_smithi060_196073_object31739:head# 2024-09-20T22:20:53.837 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.837 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b63b19f4:::benchmark_data_smithi060_196073_object35971:head# 2024-09-20T22:20:53.837 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.838 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b63b22cb:::benchmark_data_smithi060_196073_object33914:head# 2024-09-20T22:20:53.838 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.838 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b63d4bea:::benchmark_data_smithi060_196073_object10849:head# 2024-09-20T22:20:53.838 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.839 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b63df063:::benchmark_data_smithi060_196073_object26177:head# 2024-09-20T22:20:53.839 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.839 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b641dc61:::benchmark_data_smithi060_196073_object8041:head# 2024-09-20T22:20:53.839 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.840 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6420dfb:::benchmark_data_smithi060_196073_object22684:head# 2024-09-20T22:20:53.840 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.840 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6425ef4:::benchmark_data_smithi060_196073_object53590:head# 2024-09-20T22:20:53.840 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.840 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b642b2c5:::benchmark_data_smithi060_196073_object45537:head# 2024-09-20T22:20:53.840 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.841 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b643e031:::benchmark_data_smithi060_196073_object33672:head# 2024-09-20T22:20:53.841 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.841 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6480738:::benchmark_data_smithi060_196073_object56809:head# 2024-09-20T22:20:53.841 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.842 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6482040:::benchmark_data_smithi060_196073_object34579:head# 2024-09-20T22:20:53.842 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.842 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b649edae:::benchmark_data_smithi060_196073_object39947:head# 2024-09-20T22:20:53.842 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.842 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b64aca13:::benchmark_data_smithi060_196073_object61559:head# 2024-09-20T22:20:53.843 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.843 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b64c2692:::benchmark_data_smithi060_196073_object9041:head# 2024-09-20T22:20:53.843 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.843 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b64c5332:::benchmark_data_smithi060_196073_object40228:head# 2024-09-20T22:20:53.843 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.843 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b64d67b0:::benchmark_data_smithi060_196073_object36259:head# 2024-09-20T22:20:53.844 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.844 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b64f7b79:::benchmark_data_smithi060_196073_object50020:head# 2024-09-20T22:20:53.844 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.844 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b650c4de:::benchmark_data_smithi060_196073_object48907:head# 2024-09-20T22:20:53.844 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.845 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6514175:::benchmark_data_smithi060_196073_object21261:head# 2024-09-20T22:20:53.845 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.845 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b653f1a1:::benchmark_data_smithi060_196073_object61031:head# 2024-09-20T22:20:53.845 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.846 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b655db0d:::benchmark_data_smithi060_196073_object52646:head# 2024-09-20T22:20:53.846 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.846 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6566278:::benchmark_data_smithi060_196073_object29917:head# 2024-09-20T22:20:53.847 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.848 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b658618a:::benchmark_data_smithi060_196073_object27884:head# 2024-09-20T22:20:53.848 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.848 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6590ca8:::benchmark_data_smithi060_196073_object56411:head# 2024-09-20T22:20:53.848 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.849 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b660202f:::benchmark_data_smithi060_196073_object24993:head# 2024-09-20T22:20:53.849 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.849 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b66159c4:::benchmark_data_smithi060_196073_object58867:head# 2024-09-20T22:20:53.849 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.850 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6637e73:::benchmark_data_smithi060_196073_object27755:head# 2024-09-20T22:20:53.850 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.850 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b663c5a3:::benchmark_data_smithi060_196073_object13204:head# 2024-09-20T22:20:53.850 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.851 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b664897e:::benchmark_data_smithi060_196073_object20891:head# 2024-09-20T22:20:53.851 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.851 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6649e05:::benchmark_data_smithi060_196073_object14381:head# 2024-09-20T22:20:53.851 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.852 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6652f05:::benchmark_data_smithi060_196073_object34714:head# 2024-09-20T22:20:53.852 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.852 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6672325:::benchmark_data_smithi060_196073_object58412:head# 2024-09-20T22:20:53.852 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.852 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b66864cc:::benchmark_data_smithi060_196073_object22021:head# 2024-09-20T22:20:53.852 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.853 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b668a7b2:::benchmark_data_smithi060_196073_object10737:head# 2024-09-20T22:20:53.853 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.853 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b66a29ec:::benchmark_data_smithi060_196073_object30000:head# 2024-09-20T22:20:53.853 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.854 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b66b1b54:::benchmark_data_smithi060_196073_object49598:head# 2024-09-20T22:20:53.854 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.854 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b66b4e1b:::benchmark_data_smithi060_196073_object28031:head# 2024-09-20T22:20:53.854 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.855 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b66b8557:::benchmark_data_smithi060_196073_object61668:head# 2024-09-20T22:20:53.855 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.855 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b66baa54:::benchmark_data_smithi060_196073_object37673:head# 2024-09-20T22:20:53.855 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.856 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b66cb9b5:::benchmark_data_smithi060_196073_object33506:head# 2024-09-20T22:20:53.856 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.856 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b66e37a2:::benchmark_data_smithi060_196073_object25746:head# 2024-09-20T22:20:53.856 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.856 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b66f7788:::benchmark_data_smithi060_196073_object924:head# 2024-09-20T22:20:53.857 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.857 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b671667b:::benchmark_data_smithi060_196073_object39692:head# 2024-09-20T22:20:53.857 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.857 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b671beec:::benchmark_data_smithi060_196073_object10081:head# 2024-09-20T22:20:53.857 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.857 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6744ea8:::benchmark_data_smithi060_196073_object41275:head# 2024-09-20T22:20:53.858 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.858 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b675fa5e:::benchmark_data_smithi060_196073_object419:head# 2024-09-20T22:20:53.858 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.858 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b676963c:::benchmark_data_smithi060_196073_object21543:head# 2024-09-20T22:20:53.866 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.866 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6775f15:::benchmark_data_smithi060_196073_object44582:head# 2024-09-20T22:20:53.866 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.867 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b677e6c1:::benchmark_data_smithi060_196073_object58509:head# 2024-09-20T22:20:53.867 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.867 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b678d26d:::benchmark_data_smithi060_196073_object26167:head# 2024-09-20T22:20:53.867 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.869 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6792d52:::benchmark_data_smithi060_196073_object43792:head# 2024-09-20T22:20:53.869 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.869 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6793103:::benchmark_data_smithi060_196073_object9172:head# 2024-09-20T22:20:53.869 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.871 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b679d196:::benchmark_data_smithi060_196073_object34644:head# 2024-09-20T22:20:53.871 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.871 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b67ac480:::benchmark_data_smithi060_196073_object30644:head# 2024-09-20T22:20:53.871 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.871 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b67c581e:::benchmark_data_smithi060_196073_object6144:head# 2024-09-20T22:20:53.871 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.871 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b67dfb9d:::benchmark_data_smithi060_196073_object11497:head# 2024-09-20T22:20:53.871 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.871 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b67e417c:::benchmark_data_smithi060_196073_object32450:head# 2024-09-20T22:20:53.871 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.871 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6802453:::benchmark_data_smithi060_196073_object45957:head# 2024-09-20T22:20:53.871 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.871 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6802771:::benchmark_data_smithi060_196073_object23693:head# 2024-09-20T22:20:53.872 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.873 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b680daa6:::benchmark_data_smithi060_196073_object12096:head# 2024-09-20T22:20:53.873 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.873 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b680f03c:::benchmark_data_smithi060_196073_object20019:head# 2024-09-20T22:20:53.873 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.873 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b682013c:::benchmark_data_smithi060_196073_object23738:head# 2024-09-20T22:20:53.873 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.873 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6849ab3:::benchmark_data_smithi060_196073_object22353:head# 2024-09-20T22:20:53.873 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.874 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b68689be:::benchmark_data_smithi060_196073_object30965:head# 2024-09-20T22:20:53.874 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.874 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b686df15:::benchmark_data_smithi060_196073_object53220:head# 2024-09-20T22:20:53.874 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.874 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b688df34:::benchmark_data_smithi060_196073_object36508:head# 2024-09-20T22:20:53.874 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.875 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b68bfd99:::benchmark_data_smithi060_196073_object17114:head# 2024-09-20T22:20:53.875 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.875 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b68c5c09:::benchmark_data_smithi060_196073_object29120:head# 2024-09-20T22:20:53.875 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.876 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b68f1664:::benchmark_data_smithi060_196073_object21334:head# 2024-09-20T22:20:53.876 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.876 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b68ff422:::benchmark_data_smithi060_196073_object33437:head# 2024-09-20T22:20:53.876 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.876 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6905bca:::benchmark_data_smithi060_196073_object23710:head# 2024-09-20T22:20:53.877 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.877 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b69079ba:::benchmark_data_smithi060_196073_object58178:head# 2024-09-20T22:20:53.877 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.877 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6911e4d:::benchmark_data_smithi060_196073_object50919:head# 2024-09-20T22:20:53.877 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.878 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6912c94:::benchmark_data_smithi060_196073_object61693:head# 2024-09-20T22:20:53.878 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.878 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b692d566:::benchmark_data_smithi060_196073_object23918:head# 2024-09-20T22:20:53.878 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.878 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6946582:::benchmark_data_smithi060_196073_object49980:head# 2024-09-20T22:20:53.878 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.879 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b69772b8:::benchmark_data_smithi060_196073_object37685:head# 2024-09-20T22:20:53.879 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.879 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b699193b:::benchmark_data_smithi060_196073_object46920:head# 2024-09-20T22:20:53.879 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.880 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b69939d2:::benchmark_data_smithi060_196073_object37555:head# 2024-09-20T22:20:53.880 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.880 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b69a03f5:::benchmark_data_smithi060_196073_object48725:head# 2024-09-20T22:20:53.880 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.881 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b69a69a5:::benchmark_data_smithi060_196073_object47872:head# 2024-09-20T22:20:53.881 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.881 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b69abd79:::benchmark_data_smithi060_196073_object34916:head# 2024-09-20T22:20:53.881 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.881 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b69aec79:::benchmark_data_smithi060_196073_object43063:head# 2024-09-20T22:20:53.881 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.882 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b69ec570:::benchmark_data_smithi060_196073_object57062:head# 2024-09-20T22:20:53.882 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.882 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b69f34bb:::benchmark_data_smithi060_196073_object33242:head# 2024-09-20T22:20:53.882 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.882 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b69f97e9:::benchmark_data_smithi060_196073_object42303:head# 2024-09-20T22:20:53.882 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.883 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6a2b185:::benchmark_data_smithi060_196073_object32508:head# 2024-09-20T22:20:53.883 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.883 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6a607c9:::benchmark_data_smithi060_196073_object20992:head# 2024-09-20T22:20:53.883 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.884 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6a6e488:::benchmark_data_smithi060_196073_object6023:head# 2024-09-20T22:20:53.884 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.884 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6a73cb0:::benchmark_data_smithi060_196073_object57769:head# 2024-09-20T22:20:53.884 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.885 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6a7d932:::benchmark_data_smithi060_196073_object40382:head# 2024-09-20T22:20:53.885 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.885 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6a817f0:::benchmark_data_smithi060_196073_object8161:head# 2024-09-20T22:20:53.885 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.886 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6ab414d:::benchmark_data_smithi060_196073_object34377:head# 2024-09-20T22:20:53.886 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.886 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6acdce8:::benchmark_data_smithi060_196073_object34259:head# 2024-09-20T22:20:53.886 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.886 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6ad30e9:::benchmark_data_smithi060_196073_object44920:head# 2024-09-20T22:20:53.887 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.887 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6ae3fdc:::benchmark_data_smithi060_196073_object18210:head# 2024-09-20T22:20:53.887 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.887 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6af43a1:::benchmark_data_smithi060_196073_object8235:head# 2024-09-20T22:20:53.887 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.888 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6afa7d0:::benchmark_data_smithi060_196073_object34777:head# 2024-09-20T22:20:53.888 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.888 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6affc9b:::benchmark_data_smithi060_196073_object60537:head# 2024-09-20T22:20:53.888 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.888 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6b20135:::benchmark_data_smithi060_196073_object52335:head# 2024-09-20T22:20:53.888 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.889 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6b30927:::benchmark_data_smithi060_196073_object52202:head# 2024-09-20T22:20:53.889 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.889 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6b4b914:::benchmark_data_smithi060_196073_object42701:head# 2024-09-20T22:20:53.889 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.890 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6b8249f:::benchmark_data_smithi060_196073_object9174:head# 2024-09-20T22:20:53.890 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.890 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6bf7931:::benchmark_data_smithi060_196073_object61486:head# 2024-09-20T22:20:53.890 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.891 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6c01aca:::benchmark_data_smithi060_196073_object12883:head# 2024-09-20T22:20:53.891 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.891 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6c031d4:::benchmark_data_smithi060_196073_object57172:head# 2024-09-20T22:20:53.891 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.891 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6c06bd4:::benchmark_data_smithi060_196073_object31163:head# 2024-09-20T22:20:53.891 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.892 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6c28583:::benchmark_data_smithi060_196073_object53424:head# 2024-09-20T22:20:53.892 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.892 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6c2fe50:::benchmark_data_smithi060_196073_object29099:head# 2024-09-20T22:20:53.892 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.892 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6c35f28:::benchmark_data_smithi060_196073_object51050:head# 2024-09-20T22:20:53.892 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.893 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6c3c71f:::benchmark_data_smithi060_196073_object36625:head# 2024-09-20T22:20:53.893 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.893 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6c6f720:::benchmark_data_smithi060_196073_object34748:head# 2024-09-20T22:20:53.893 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.894 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6c7caa1:::benchmark_data_smithi060_196073_object13404:head# 2024-09-20T22:20:53.894 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.894 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6c7da78:::benchmark_data_smithi060_196073_object32269:head# 2024-09-20T22:20:53.894 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.895 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6c81516:::benchmark_data_smithi060_196073_object47777:head# 2024-09-20T22:20:53.895 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.895 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6c91b5c:::benchmark_data_smithi060_196073_object30278:head# 2024-09-20T22:20:53.897 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.897 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6c99c1a:::benchmark_data_smithi060_196073_object46542:head# 2024-09-20T22:20:53.897 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.898 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6cb4d32:::benchmark_data_smithi060_196073_object21136:head# 2024-09-20T22:20:53.898 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.898 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6cb95a6:::benchmark_data_smithi060_196073_object35031:head# 2024-09-20T22:20:53.898 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.899 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6ccd50e:::benchmark_data_smithi060_196073_object15710:head# 2024-09-20T22:20:53.899 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.899 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6cda1a2:::benchmark_data_smithi060_196073_object12395:head# 2024-09-20T22:20:53.899 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.900 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6cee230:::benchmark_data_smithi060_196073_object1889:head# 2024-09-20T22:20:53.900 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.900 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6cf3b95:::benchmark_data_smithi060_196073_object24829:head# 2024-09-20T22:20:53.900 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.901 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6d00eef:::benchmark_data_smithi060_196073_object9515:head# 2024-09-20T22:20:53.901 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.901 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6d14d50:::benchmark_data_smithi060_196073_object52247:head# 2024-09-20T22:20:53.901 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.902 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6d14f26:::benchmark_data_smithi060_196073_object34216:head# 2024-09-20T22:20:53.902 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.902 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6d444b2:::benchmark_data_smithi060_196073_object5310:head# 2024-09-20T22:20:53.902 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.902 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6d67379:::benchmark_data_smithi060_196073_object28259:head# 2024-09-20T22:20:53.902 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.903 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6d79867:::benchmark_data_smithi060_196073_object60748:head# 2024-09-20T22:20:53.903 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.903 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6d7afbc:::benchmark_data_smithi060_196073_object33498:head# 2024-09-20T22:20:53.903 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.903 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6d8bbd4:::benchmark_data_smithi060_196073_object32047:head# 2024-09-20T22:20:53.903 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.904 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6d9dd09:::benchmark_data_smithi060_196073_object10213:head# 2024-09-20T22:20:53.904 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.904 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6dab435:::benchmark_data_smithi060_196073_object43241:head# 2024-09-20T22:20:53.904 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.905 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6dbd3c8:::benchmark_data_smithi060_196073_object6120:head# 2024-09-20T22:20:53.905 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.905 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6dd27a3:::benchmark_data_smithi060_196073_object40447:head# 2024-09-20T22:20:53.905 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.905 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6dd5fdb:::benchmark_data_smithi060_196073_object27456:head# 2024-09-20T22:20:53.906 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.906 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6de8cdc:::benchmark_data_smithi060_196073_object46831:head# 2024-09-20T22:20:53.906 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.906 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6df3c34:::benchmark_data_smithi060_196073_object39507:head# 2024-09-20T22:20:53.906 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.907 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6dfbd94:::benchmark_data_smithi060_196073_object29052:head# 2024-09-20T22:20:53.907 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.907 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6dfd4dd:::benchmark_data_smithi060_196073_object46220:head# 2024-09-20T22:20:53.907 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.908 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6e047be:::benchmark_data_smithi060_196073_object48895:head# 2024-09-20T22:20:53.908 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.908 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6e0512c:::benchmark_data_smithi060_196073_object27813:head# 2024-09-20T22:20:53.908 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.909 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6e15c38:::benchmark_data_smithi060_196073_object50050:head# 2024-09-20T22:20:53.909 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.909 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6e32ffe:::benchmark_data_smithi060_196073_object48925:head# 2024-09-20T22:20:53.909 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.909 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6e4913b:::benchmark_data_smithi060_196073_object44251:head# 2024-09-20T22:20:53.910 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.910 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6e641bf:::benchmark_data_smithi060_196073_object54276:head# 2024-09-20T22:20:53.910 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.910 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6e6875e:::benchmark_data_smithi060_196073_object42120:head# 2024-09-20T22:20:53.910 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.910 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6e6e8d6:::benchmark_data_smithi060_196073_object9902:head# 2024-09-20T22:20:53.910 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.911 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6e7c84a:::benchmark_data_smithi060_196073_object24055:head# 2024-09-20T22:20:53.911 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.911 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6e891e8:::benchmark_data_smithi060_196073_object27565:head# 2024-09-20T22:20:53.911 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.912 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6ea0b68:::benchmark_data_smithi060_196073_object32560:head# 2024-09-20T22:20:53.912 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.912 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6ed46b2:::benchmark_data_smithi060_196073_object41534:head# 2024-09-20T22:20:53.912 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.913 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6ed7ffa:::benchmark_data_smithi060_196073_object29515:head# 2024-09-20T22:20:53.913 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.913 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6eddcf6:::benchmark_data_smithi060_196073_object5730:head# 2024-09-20T22:20:53.913 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.913 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6ee289c:::benchmark_data_smithi060_196073_object24521:head# 2024-09-20T22:20:53.914 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.914 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6ee5975:::benchmark_data_smithi060_196073_object24729:head# 2024-09-20T22:20:53.914 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.914 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6eed129:::benchmark_data_smithi060_196073_object26669:head# 2024-09-20T22:20:53.914 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.915 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6f1d67f:::benchmark_data_smithi060_196073_object29478:head# 2024-09-20T22:20:53.915 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.915 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:53.916 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6f2e239:::benchmark_data_smithi060_196073_object5838:head# 2024-09-20T22:20:53.916 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.916 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6f33834:::benchmark_data_smithi060_196073_object37686:head# 2024-09-20T22:20:53.916 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.916 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6f34bd2:::benchmark_data_smithi060_196073_object1185:head# 2024-09-20T22:20:53.916 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.916 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6f42673:::benchmark_data_smithi060_196073_object61566:head# 2024-09-20T22:20:53.917 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.918 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6f53d69:::benchmark_data_smithi060_196073_object18171:head# 2024-09-20T22:20:53.918 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.918 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6f8153d:::benchmark_data_smithi060_196073_object13284:head# 2024-09-20T22:20:53.918 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.921 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6f91d59:::benchmark_data_smithi060_196073_object3521:head# 2024-09-20T22:20:53.921 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.921 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6f98f01:::benchmark_data_smithi060_196073_object34787:head# 2024-09-20T22:20:53.921 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.921 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6fd338f:::benchmark_data_smithi060_196073_object36160:head# 2024-09-20T22:20:53.921 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.921 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6fdc62d:::benchmark_data_smithi060_196073_object15758:head# 2024-09-20T22:20:53.921 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.921 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6fe8611:::benchmark_data_smithi060_196073_object25189:head# 2024-09-20T22:20:53.921 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.922 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6fe8708:::benchmark_data_smithi060_196073_object19178:head# 2024-09-20T22:20:53.922 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.922 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b6feabba:::benchmark_data_smithi060_196073_object53269:head# 2024-09-20T22:20:53.922 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.922 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7004225:::benchmark_data_smithi060_196073_object49141:head# 2024-09-20T22:20:53.922 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.922 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b70090f9:::benchmark_data_smithi060_196073_object45631:head# 2024-09-20T22:20:53.922 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.925 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7015e63:::benchmark_data_smithi060_196073_object12996:head# 2024-09-20T22:20:53.925 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.926 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b701b87f:::benchmark_data_smithi060_196073_object46938:head# 2024-09-20T22:20:53.926 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.927 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b70442c0:::benchmark_data_smithi060_196073_object58863:head# 2024-09-20T22:20:53.927 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.927 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b70618f2:::benchmark_data_smithi060_196073_object31755:head# 2024-09-20T22:20:53.927 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.927 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7068fdf:::benchmark_data_smithi060_196073_object28190:head# 2024-09-20T22:20:53.928 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.928 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b707bd27:::benchmark_data_smithi060_196073_object3975:head# 2024-09-20T22:20:53.928 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.928 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b707e01f:::benchmark_data_smithi060_196073_object41386:head# 2024-09-20T22:20:53.928 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.929 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b707fba2:::benchmark_data_smithi060_196073_object39298:head# 2024-09-20T22:20:53.929 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.929 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7093571:::benchmark_data_smithi060_196073_object25761:head# 2024-09-20T22:20:53.929 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.930 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b709c1eb:::benchmark_data_smithi060_196073_object2714:head# 2024-09-20T22:20:53.930 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.933 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b70baaba:::benchmark_data_smithi060_196073_object34112:head# 2024-09-20T22:20:53.933 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.937 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b70c6d9d:::benchmark_data_smithi060_196073_object43840:head# 2024-09-20T22:20:53.937 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.937 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b70c8cdf:::benchmark_data_smithi060_196073_object49608:head# 2024-09-20T22:20:53.937 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.938 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b70d8431:::benchmark_data_smithi060_196073_object52553:head# 2024-09-20T22:20:53.938 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.939 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b710bb53:::benchmark_data_smithi060_196073_object29524:head# 2024-09-20T22:20:53.939 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.939 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7136ea9:::benchmark_data_smithi060_196073_object6892:head# 2024-09-20T22:20:53.939 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.940 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b713bd8c:::benchmark_data_smithi060_196073_object59460:head# 2024-09-20T22:20:53.940 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.942 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b71502c5:::benchmark_data_smithi060_196073_object11226:head# 2024-09-20T22:20:53.942 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.942 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7152744:::benchmark_data_smithi060_196073_object46471:head# 2024-09-20T22:20:53.942 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.943 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b716fe6d:::benchmark_data_smithi060_196073_object29236:head# 2024-09-20T22:20:53.943 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.943 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b718d0e3:::benchmark_data_smithi060_196073_object60342:head# 2024-09-20T22:20:53.943 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.944 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b718fca2:::benchmark_data_smithi060_196073_object25024:head# 2024-09-20T22:20:53.944 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.944 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b71936d7:::benchmark_data_smithi060_196073_object11517:head# 2024-09-20T22:20:53.944 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.944 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b719979d:::benchmark_data_smithi060_196073_object51344:head# 2024-09-20T22:20:53.944 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.945 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b71a1053:::benchmark_data_smithi060_196073_object37052:head# 2024-09-20T22:20:53.945 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.946 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b71a1560:::benchmark_data_smithi060_196073_object45780:head# 2024-09-20T22:20:53.946 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.949 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b71b9828:::benchmark_data_smithi060_196073_object45526:head# 2024-09-20T22:20:53.949 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.949 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b71ba462:::benchmark_data_smithi060_196073_object14657:head# 2024-09-20T22:20:53.949 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.949 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b71c31a9:::benchmark_data_smithi060_196073_object22316:head# 2024-09-20T22:20:53.949 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.950 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b71d97e7:::benchmark_data_smithi060_196073_object34917:head# 2024-09-20T22:20:53.950 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.950 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b71ea708:::benchmark_data_smithi060_196073_object57600:head# 2024-09-20T22:20:53.950 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.953 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b71ed22e:::benchmark_data_smithi060_196073_object48580:head# 2024-09-20T22:20:53.953 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.953 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b71f5836:::benchmark_data_smithi060_196073_object60660:head# 2024-09-20T22:20:53.953 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.953 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b724807f:::benchmark_data_smithi060_196073_object55905:head# 2024-09-20T22:20:53.953 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.953 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b72499e3:::benchmark_data_smithi060_196073_object18288:head# 2024-09-20T22:20:53.954 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.954 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b724dd4c:::benchmark_data_smithi060_196073_object21210:head# 2024-09-20T22:20:53.954 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.954 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b724f6d0:::benchmark_data_smithi060_196073_object18915:head# 2024-09-20T22:20:53.954 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.955 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b725281c:::benchmark_data_smithi060_196073_object53243:head# 2024-09-20T22:20:53.955 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.955 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b72910cc:::benchmark_data_smithi060_196073_object14621:head# 2024-09-20T22:20:53.955 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.967 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7292cfc:::benchmark_data_smithi060_196073_object27676:head# 2024-09-20T22:20:53.968 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.968 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b729a4d8:::benchmark_data_smithi060_196073_object21015:head# 2024-09-20T22:20:53.968 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.968 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b729fa3d:::benchmark_data_smithi060_196073_object38887:head# 2024-09-20T22:20:53.968 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.968 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b72accde:::benchmark_data_smithi060_196073_object27589:head# 2024-09-20T22:20:53.968 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.968 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b72b2250:::benchmark_data_smithi060_196073_object49693:head# 2024-09-20T22:20:53.968 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.968 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b72b7730:::benchmark_data_smithi060_196073_object13207:head# 2024-09-20T22:20:53.968 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.968 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b72cdb5f:::benchmark_data_smithi060_196073_object58659:head# 2024-09-20T22:20:53.968 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.968 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b72f2453:::benchmark_data_smithi060_196073_object3209:head# 2024-09-20T22:20:53.969 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.969 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b730285f:::benchmark_data_smithi060_196073_object3640:head# 2024-09-20T22:20:53.969 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.969 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b731560d:::benchmark_data_smithi060_196073_object13579:head# 2024-09-20T22:20:53.969 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.969 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b732d059:::benchmark_data_smithi060_196073_object34342:head# 2024-09-20T22:20:53.969 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.969 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7335226:::benchmark_data_smithi060_196073_object49051:head# 2024-09-20T22:20:53.969 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.969 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b734c8f7:::benchmark_data_smithi060_196073_object42672:head# 2024-09-20T22:20:53.969 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.969 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b735a211:::benchmark_data_smithi060_196073_object60876:head# 2024-09-20T22:20:53.969 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.969 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b735bb94:::benchmark_data_smithi060_196073_object8341:head# 2024-09-20T22:20:53.969 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.969 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b735f776:::benchmark_data_smithi060_196073_object55131:head# 2024-09-20T22:20:53.970 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.970 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7381328:::benchmark_data_smithi060_196073_object43527:head# 2024-09-20T22:20:53.970 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.970 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b73cf7ad:::benchmark_data_smithi060_196073_object16315:head# 2024-09-20T22:20:53.970 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.970 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b73db84c:::benchmark_data_smithi060_196073_object61164:head# 2024-09-20T22:20:53.970 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.970 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b73e7fff:::benchmark_data_smithi060_196073_object8720:head# 2024-09-20T22:20:53.970 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.970 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b73eaccd:::benchmark_data_smithi060_196073_object35492:head# 2024-09-20T22:20:53.970 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.970 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b741c70c:::benchmark_data_smithi060_196073_object46399:head# 2024-09-20T22:20:53.970 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.970 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7431984:::benchmark_data_smithi060_196073_object24485:head# 2024-09-20T22:20:53.971 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.971 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b74345f1:::benchmark_data_smithi060_196073_object42649:head# 2024-09-20T22:20:53.971 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.971 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7438d08:::benchmark_data_smithi060_196073_object48917:head# 2024-09-20T22:20:53.971 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.971 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b743b082:::benchmark_data_smithi060_196073_object49725:head# 2024-09-20T22:20:53.971 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.971 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b74633de:::benchmark_data_smithi060_196073_object18324:head# 2024-09-20T22:20:53.971 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.972 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b747a5f8:::benchmark_data_smithi060_196073_object6231:head# 2024-09-20T22:20:53.972 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.972 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b747c18c:::benchmark_data_smithi060_196073_object14170:head# 2024-09-20T22:20:53.972 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.972 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b747c6d3:::benchmark_data_smithi060_196073_object35608:head# 2024-09-20T22:20:53.972 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.972 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b747efb7:::benchmark_data_smithi060_196073_object41496:head# 2024-09-20T22:20:53.972 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.973 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:53.973 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7489932:::benchmark_data_smithi060_196073_object59385:head# 2024-09-20T22:20:53.973 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.973 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b748d786:::benchmark_data_smithi060_196073_object8666:head# 2024-09-20T22:20:53.973 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.974 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b74a8097:::benchmark_data_smithi060_196073_object29988:head# 2024-09-20T22:20:53.974 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:53.974 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:53.976 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b74ad72a:::benchmark_data_smithi060_196073_object38846:head# 2024-09-20T22:20:53.976 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.977 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b74c45f8:::benchmark_data_smithi060_196073_object58572:head# 2024-09-20T22:20:53.977 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.977 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b74cb81c:::benchmark_data_smithi060_196073_object3918:head# 2024-09-20T22:20:53.977 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.977 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b74d22d8:::benchmark_data_smithi060_196073_object46758:head# 2024-09-20T22:20:53.977 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.977 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b74d8f75:::benchmark_data_smithi060_196073_object26103:head# 2024-09-20T22:20:53.977 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.977 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b74d9215:::benchmark_data_smithi060_196073_object29183:head# 2024-09-20T22:20:53.977 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.977 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b74d9de2:::benchmark_data_smithi060_196073_object877:head# 2024-09-20T22:20:53.977 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.977 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b74da0a1:::benchmark_data_smithi060_196073_object13872:head# 2024-09-20T22:20:53.978 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.978 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b74e7bc0:::benchmark_data_smithi060_196073_object51985:head# 2024-09-20T22:20:53.978 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.978 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b74f301a:::benchmark_data_smithi060_196073_object5217:head# 2024-09-20T22:20:53.978 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.978 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b74f7d56:::benchmark_data_smithi060_196073_object39587:head# 2024-09-20T22:20:53.978 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.978 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b74fd607:::benchmark_data_smithi060_196073_object47022:head# 2024-09-20T22:20:53.978 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.978 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b75056cd:::benchmark_data_smithi060_196073_object40947:head# 2024-09-20T22:20:53.978 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.978 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b750753a:::benchmark_data_smithi060_196073_object46587:head# 2024-09-20T22:20:53.978 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.980 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b750e300:::benchmark_data_smithi060_196073_object60273:head# 2024-09-20T22:20:53.980 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.980 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7538f3d:::benchmark_data_smithi060_196073_object22280:head# 2024-09-20T22:20:53.980 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.980 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b753f284:::benchmark_data_smithi060_196073_object18806:head# 2024-09-20T22:20:53.980 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.980 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7562618:::benchmark_data_smithi060_196073_object11824:head# 2024-09-20T22:20:53.980 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.981 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7570998:::benchmark_data_smithi060_196073_object1202:head# 2024-09-20T22:20:53.981 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.981 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b757a387:::benchmark_data_smithi060_196073_object19646:head# 2024-09-20T22:20:53.981 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.981 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b758f605:::benchmark_data_smithi060_196073_object7518:head# 2024-09-20T22:20:53.981 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.981 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7598f49:::benchmark_data_smithi060_196073_object2695:head# 2024-09-20T22:20:53.981 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.982 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b759e390:::benchmark_data_smithi060_196073_object58899:head# 2024-09-20T22:20:53.982 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.982 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b75ae525:::benchmark_data_smithi060_196073_object52500:head# 2024-09-20T22:20:53.982 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.982 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b75b24be:::benchmark_data_smithi060_196073_object9069:head# 2024-09-20T22:20:53.982 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.982 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b75cce73:::benchmark_data_smithi060_196073_object56168:head# 2024-09-20T22:20:53.983 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.983 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b75d5c65:::benchmark_data_smithi060_196073_object27351:head# 2024-09-20T22:20:53.983 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.984 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b75f64eb:::benchmark_data_smithi060_196073_object3235:head# 2024-09-20T22:20:53.984 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.984 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b75fcba6:::benchmark_data_smithi060_196073_object32547:head# 2024-09-20T22:20:53.984 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.984 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b75ff4b6:::benchmark_data_smithi060_196073_object24836:head# 2024-09-20T22:20:53.984 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.984 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b760f897:::benchmark_data_smithi060_196073_object2667:head# 2024-09-20T22:20:53.984 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.985 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7623bd4:::benchmark_data_smithi060_196073_object48585:head# 2024-09-20T22:20:53.985 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.985 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b762610b:::benchmark_data_smithi060_196073_object44441:head# 2024-09-20T22:20:53.985 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.986 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7629076:::benchmark_data_smithi060_196073_object2857:head# 2024-09-20T22:20:53.986 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.986 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b762d804:::benchmark_data_smithi060_196073_object3962:head# 2024-09-20T22:20:53.986 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.986 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7631c1c:::benchmark_data_smithi060_196073_object19340:head# 2024-09-20T22:20:53.986 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.987 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b763bcc4:::benchmark_data_smithi060_196073_object56784:head# 2024-09-20T22:20:53.987 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.987 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b766cb88:::benchmark_data_smithi060_196073_object49218:head# 2024-09-20T22:20:53.987 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.987 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7676c3f:::benchmark_data_smithi060_196073_object30558:head# 2024-09-20T22:20:53.987 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.988 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7677d49:::benchmark_data_smithi060_196073_object12297:head# 2024-09-20T22:20:53.988 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.988 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b769aa93:::benchmark_data_smithi060_196073_object57922:head# 2024-09-20T22:20:53.988 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.988 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b76ae293:::benchmark_data_smithi060_196073_object19396:head# 2024-09-20T22:20:53.988 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.989 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b76b3457:::benchmark_data_smithi060_196073_object5384:head# 2024-09-20T22:20:53.989 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.989 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b76b683a:::benchmark_data_smithi060_196073_object14702:head# 2024-09-20T22:20:53.989 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.990 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b76bb6f6:::benchmark_data_smithi060_196073_object4947:head# 2024-09-20T22:20:53.990 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.990 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b771c4d7:::benchmark_data_smithi060_196073_object1875:head# 2024-09-20T22:20:53.992 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.993 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b771d673:::benchmark_data_smithi060_196073_object12579:head# 2024-09-20T22:20:53.993 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.993 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b772c219:::benchmark_data_smithi060_196073_object34041:head# 2024-09-20T22:20:53.993 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.994 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b77681d1:::benchmark_data_smithi060_196073_object2565:head# 2024-09-20T22:20:53.994 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.994 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b777e572:::benchmark_data_smithi060_196073_object58147:head# 2024-09-20T22:20:53.994 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.995 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b777ec62:::benchmark_data_smithi060_196073_object4395:head# 2024-09-20T22:20:53.995 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.995 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b778555e:::benchmark_data_smithi060_196073_object35920:head# 2024-09-20T22:20:53.995 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.996 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b77970f4:::benchmark_data_smithi060_196073_object12414:head# 2024-09-20T22:20:53.996 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:53.996 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b77d544c:::benchmark_data_smithi060_196073_object4611:head# 2024-09-20T22:20:54.004 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.005 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b77ed01c:::benchmark_data_smithi060_196073_object25122:head# 2024-09-20T22:20:54.005 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.005 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b77f2f59:::benchmark_data_smithi060_196073_object45068:head# 2024-09-20T22:20:54.005 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.006 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b77f9e80:::benchmark_data_smithi060_196073_object35103:head# 2024-09-20T22:20:54.006 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.006 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b77fc5d3:::benchmark_data_smithi060_196073_object53612:head# 2024-09-20T22:20:54.006 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.007 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b78065df:::benchmark_data_smithi060_196073_object48425:head# 2024-09-20T22:20:54.007 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.007 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b781413e:::benchmark_data_smithi060_196073_object28686:head# 2024-09-20T22:20:54.007 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.007 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7818bbc:::benchmark_data_smithi060_196073_object43820:head# 2024-09-20T22:20:54.007 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.008 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7821904:::benchmark_data_smithi060_196073_object17682:head# 2024-09-20T22:20:54.008 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.008 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b782abd6:::benchmark_data_smithi060_196073_object38243:head# 2024-09-20T22:20:54.008 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.009 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b783d93a:::benchmark_data_smithi060_196073_object8881:head# 2024-09-20T22:20:54.009 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.009 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7841794:::benchmark_data_smithi060_196073_object6872:head# 2024-09-20T22:20:54.009 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.010 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b784b3c8:::benchmark_data_smithi060_196073_object30663:head# 2024-09-20T22:20:54.010 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.010 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7865804:::benchmark_data_smithi060_196073_object11373:head# 2024-09-20T22:20:54.010 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.010 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b787aa30:::benchmark_data_smithi060_196073_object19520:head# 2024-09-20T22:20:54.011 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.011 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b787d09e:::benchmark_data_smithi060_196073_object45635:head# 2024-09-20T22:20:54.011 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.011 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b787d4b0:::benchmark_data_smithi060_196073_object20701:head# 2024-09-20T22:20:54.011 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.011 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7882b22:::benchmark_data_smithi060_196073_object48538:head# 2024-09-20T22:20:54.011 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.012 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7885ab2:::benchmark_data_smithi060_196073_object41170:head# 2024-09-20T22:20:54.012 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.013 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b78b9936:::benchmark_data_smithi060_196073_object5915:head# 2024-09-20T22:20:54.013 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.013 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b78e9227:::benchmark_data_smithi060_196073_object24571:head# 2024-09-20T22:20:54.013 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.013 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b78f21e1:::benchmark_data_smithi060_196073_object19108:head# 2024-09-20T22:20:54.013 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.014 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b78f326e:::benchmark_data_smithi060_196073_object27663:head# 2024-09-20T22:20:54.014 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.014 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b78f9d0a:::benchmark_data_smithi060_196073_object46993:head# 2024-09-20T22:20:54.014 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.015 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b78ffb98:::benchmark_data_smithi060_196073_object6611:head# 2024-09-20T22:20:54.015 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.015 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7901093:::benchmark_data_smithi060_196073_object31624:head# 2024-09-20T22:20:54.015 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.015 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b790362d:::benchmark_data_smithi060_196073_object57041:head# 2024-09-20T22:20:54.016 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.016 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7915424:::benchmark_data_smithi060_196073_object29807:head# 2024-09-20T22:20:54.016 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.016 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b793765a:::benchmark_data_smithi060_196073_object523:head# 2024-09-20T22:20:54.016 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.017 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b793b6a2:::benchmark_data_smithi060_196073_object20129:head# 2024-09-20T22:20:54.017 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.017 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b79433e6:::benchmark_data_smithi060_196073_object48934:head# 2024-09-20T22:20:54.017 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.018 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7954711:::benchmark_data_smithi060_196073_object8548:head# 2024-09-20T22:20:54.018 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.018 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b79547e9:::benchmark_data_smithi060_196073_object37228:head# 2024-09-20T22:20:54.018 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.019 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7955f03:::benchmark_data_smithi060_196073_object28979:head# 2024-09-20T22:20:54.019 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.019 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b797a699:::benchmark_data_smithi060_196073_object38227:head# 2024-09-20T22:20:54.019 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.020 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7990e67:::benchmark_data_smithi060_196073_object5689:head# 2024-09-20T22:20:54.020 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.020 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7998895:::benchmark_data_smithi060_196073_object7193:head# 2024-09-20T22:20:54.020 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.020 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b79bb126:::benchmark_data_smithi060_196073_object29105:head# 2024-09-20T22:20:54.020 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.020 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b79c3574:::benchmark_data_smithi060_196073_object23330:head# 2024-09-20T22:20:54.021 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.021 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b79ddd10:::benchmark_data_smithi060_196073_object58083:head# 2024-09-20T22:20:54.021 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.021 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b79e49f1:::benchmark_data_smithi060_196073_object56893:head# 2024-09-20T22:20:54.022 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.022 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b79f4742:::benchmark_data_smithi060_196073_object20998:head# 2024-09-20T22:20:54.022 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.022 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7a01f0e:::benchmark_data_smithi060_196073_object20056:head# 2024-09-20T22:20:54.022 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.023 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7a2e2be:::benchmark_data_smithi060_196073_object5538:head# 2024-09-20T22:20:54.023 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.023 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7a30274:::benchmark_data_smithi060_196073_object27344:head# 2024-09-20T22:20:54.023 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.023 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7a48602:::benchmark_data_smithi060_196073_object59:head# 2024-09-20T22:20:54.023 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.024 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7a58d58:::benchmark_data_smithi060_196073_object2948:head# 2024-09-20T22:20:54.024 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.024 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7a64207:::benchmark_data_smithi060_196073_object2767:head# 2024-09-20T22:20:54.024 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.025 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7a7ffc0:::benchmark_data_smithi060_196073_object37384:head# 2024-09-20T22:20:54.025 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.025 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7a88317:::benchmark_data_smithi060_196073_object36115:head# 2024-09-20T22:20:54.025 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.025 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7a8aaa5:::benchmark_data_smithi060_196073_object56315:head# 2024-09-20T22:20:54.026 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.026 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7a8df21:::benchmark_data_smithi060_196073_object32869:head# 2024-09-20T22:20:54.026 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.026 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7a913ae:::benchmark_data_smithi060_196073_object3375:head# 2024-09-20T22:20:54.026 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.027 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7a97078:::benchmark_data_smithi060_196073_object48598:head# 2024-09-20T22:20:54.027 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.027 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7aa1788:::benchmark_data_smithi060_196073_object12588:head# 2024-09-20T22:20:54.027 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.028 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7ab32da:::benchmark_data_smithi060_196073_object55304:head# 2024-09-20T22:20:54.028 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.028 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7ab368b:::benchmark_data_smithi060_196073_object38217:head# 2024-09-20T22:20:54.028 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.028 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7addde9:::benchmark_data_smithi060_196073_object34584:head# 2024-09-20T22:20:54.029 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.029 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7aed7ab:::benchmark_data_smithi060_196073_object10374:head# 2024-09-20T22:20:54.029 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.029 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7b079ee:::benchmark_data_smithi060_196073_object22464:head# 2024-09-20T22:20:54.029 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.030 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7b15ce1:::benchmark_data_smithi060_196073_object46594:head# 2024-09-20T22:20:54.030 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.031 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7b1a8e3:::benchmark_data_smithi060_196073_object228:head# 2024-09-20T22:20:54.031 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.031 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7b1cc06:::benchmark_data_smithi060_196073_object51276:head# 2024-09-20T22:20:54.031 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.031 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7b6730b:::benchmark_data_smithi060_196073_object13810:head# 2024-09-20T22:20:54.032 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.034 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7b9f223:::benchmark_data_smithi060_196073_object46878:head# 2024-09-20T22:20:54.034 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.034 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7bb31df:::benchmark_data_smithi060_196073_object59948:head# 2024-09-20T22:20:54.034 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.034 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7bb892d:::benchmark_data_smithi060_196073_object47989:head# 2024-09-20T22:20:54.034 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.034 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7bb8a11:::benchmark_data_smithi060_196073_object30588:head# 2024-09-20T22:20:54.034 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.034 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7bc3184:::benchmark_data_smithi060_196073_object10330:head# 2024-09-20T22:20:54.035 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.035 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7bdef7a:::benchmark_data_smithi060_196073_object52005:head# 2024-09-20T22:20:54.035 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.036 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7bf8c7a:::benchmark_data_smithi060_196073_object18469:head# 2024-09-20T22:20:54.036 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.036 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7c01089:::benchmark_data_smithi060_196073_object32701:head# 2024-09-20T22:20:54.037 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.037 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7c0d458:::benchmark_data_smithi060_196073_object19845:head# 2024-09-20T22:20:54.037 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.037 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7c1c5c4:::benchmark_data_smithi060_196073_object2687:head# 2024-09-20T22:20:54.038 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.038 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7c287cc:::benchmark_data_smithi060_196073_object44664:head# 2024-09-20T22:20:54.039 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.039 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7c3f305:::benchmark_data_smithi060_196073_object2269:head# 2024-09-20T22:20:54.039 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.039 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7c44b38:::benchmark_data_smithi060_196073_object52733:head# 2024-09-20T22:20:54.039 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.039 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7c50d82:::benchmark_data_smithi060_196073_object24300:head# 2024-09-20T22:20:54.040 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.040 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7caffb2:::benchmark_data_smithi060_196073_object24446:head# 2024-09-20T22:20:54.040 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.040 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7cb15d2:::benchmark_data_smithi060_196073_object52966:head# 2024-09-20T22:20:54.040 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.040 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7cb35dd:::benchmark_data_smithi060_196073_object3254:head# 2024-09-20T22:20:54.040 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.041 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7cd2e57:::benchmark_data_smithi060_196073_object43530:head# 2024-09-20T22:20:54.041 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.041 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7cfba7f:::benchmark_data_smithi060_196073_object30249:head# 2024-09-20T22:20:54.041 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.041 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7d0a0a1:::benchmark_data_smithi060_196073_object8559:head# 2024-09-20T22:20:54.042 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.042 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7d1f913:::benchmark_data_smithi060_196073_object15240:head# 2024-09-20T22:20:54.042 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.042 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7d58dfb:::benchmark_data_smithi060_196073_object54379:head# 2024-09-20T22:20:54.042 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.043 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7d72274:::benchmark_data_smithi060_196073_object60963:head# 2024-09-20T22:20:54.043 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.043 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7d85cf4:::benchmark_data_smithi060_196073_object30795:head# 2024-09-20T22:20:54.043 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.043 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7d985cc:::benchmark_data_smithi060_196073_object42881:head# 2024-09-20T22:20:54.044 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.044 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7da76dd:::benchmark_data_smithi060_196073_object56267:head# 2024-09-20T22:20:54.044 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.044 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7daa8cc:::benchmark_data_smithi060_196073_object53180:head# 2024-09-20T22:20:54.044 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.045 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7dbae02:::benchmark_data_smithi060_196073_object7145:head# 2024-09-20T22:20:54.045 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.045 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7dc2026:::benchmark_data_smithi060_196073_object37805:head# 2024-09-20T22:20:54.045 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.046 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7dc569b:::benchmark_data_smithi060_196073_object14967:head# 2024-09-20T22:20:54.046 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.046 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7dc600a:::benchmark_data_smithi060_196073_object3430:head# 2024-09-20T22:20:54.046 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.046 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7dd9129:::benchmark_data_smithi060_196073_object13114:head# 2024-09-20T22:20:54.046 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.047 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7deebe0:::benchmark_data_smithi060_196073_object38537:head# 2024-09-20T22:20:54.047 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.047 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7dfa7d6:::benchmark_data_smithi060_196073_object45338:head# 2024-09-20T22:20:54.047 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.048 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7dfcc53:::benchmark_data_smithi060_196073_object49717:head# 2024-09-20T22:20:54.048 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.048 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7e12ecb:::benchmark_data_smithi060_196073_object22587:head# 2024-09-20T22:20:54.048 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.048 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7e1d295:::benchmark_data_smithi060_196073_object23432:head# 2024-09-20T22:20:54.049 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.049 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7e24465:::benchmark_data_smithi060_196073_object1934:head# 2024-09-20T22:20:54.049 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.051 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7e2cd0c:::benchmark_data_smithi060_196073_object41437:head# 2024-09-20T22:20:54.051 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.051 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7e2eb15:::benchmark_data_smithi060_196073_object11886:head# 2024-09-20T22:20:54.051 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.051 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7e57467:::benchmark_data_smithi060_196073_object42304:head# 2024-09-20T22:20:54.051 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.051 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7e58a37:::benchmark_data_smithi060_196073_object47665:head# 2024-09-20T22:20:54.051 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.052 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7e5cdd4:::benchmark_data_smithi060_196073_object17767:head# 2024-09-20T22:20:54.052 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.052 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7e7bdac:::benchmark_data_smithi060_196073_object6653:head# 2024-09-20T22:20:54.052 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.052 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7e819be:::benchmark_data_smithi060_196073_object58380:head# 2024-09-20T22:20:54.052 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.052 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7e97d55:::benchmark_data_smithi060_196073_object16917:head# 2024-09-20T22:20:54.052 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.053 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7e9c1af:::benchmark_data_smithi060_196073_object60635:head# 2024-09-20T22:20:54.053 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.054 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7ed8564:::benchmark_data_smithi060_196073_object39628:head# 2024-09-20T22:20:54.054 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.054 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7ede956:::benchmark_data_smithi060_196073_object2625:head# 2024-09-20T22:20:54.054 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.056 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7ee554f:::benchmark_data_smithi060_196073_object27235:head# 2024-09-20T22:20:54.056 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.056 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7ee91ea:::benchmark_data_smithi060_196073_object61229:head# 2024-09-20T22:20:54.056 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.056 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7eea2da:::benchmark_data_smithi060_196073_object55658:head# 2024-09-20T22:20:54.056 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.056 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7eeea3b:::benchmark_data_smithi060_196073_object28659:head# 2024-09-20T22:20:54.056 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.056 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7ef8461:::benchmark_data_smithi060_196073_object3796:head# 2024-09-20T22:20:54.056 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.057 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7efbfd2:::benchmark_data_smithi060_196073_object45584:head# 2024-09-20T22:20:54.057 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.057 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7f0c72a:::benchmark_data_smithi060_196073_object38070:head# 2024-09-20T22:20:54.057 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.058 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7f0ecd7:::benchmark_data_smithi060_196073_object16599:head# 2024-09-20T22:20:54.058 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.058 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7f1a3e8:::benchmark_data_smithi060_196073_object15711:head# 2024-09-20T22:20:54.058 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.058 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7f2d649:::benchmark_data_smithi060_196073_object44013:head# 2024-09-20T22:20:54.058 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.059 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7f3bf85:::benchmark_data_smithi060_196073_object17791:head# 2024-09-20T22:20:54.059 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.059 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7f4a920:::benchmark_data_smithi060_196073_object43751:head# 2024-09-20T22:20:54.059 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.059 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7f517be:::benchmark_data_smithi060_196073_object35809:head# 2024-09-20T22:20:54.059 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.060 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7f5e90d:::benchmark_data_smithi060_196073_object55353:head# 2024-09-20T22:20:54.063 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.064 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7f6640c:::benchmark_data_smithi060_196073_object9876:head# 2024-09-20T22:20:54.064 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.066 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7f6b6a5:::benchmark_data_smithi060_196073_object7046:head# 2024-09-20T22:20:54.066 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.066 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7f6ecf2:::benchmark_data_smithi060_196073_object8979:head# 2024-09-20T22:20:54.066 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.066 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7f7a8d8:::benchmark_data_smithi060_196073_object51304:head# 2024-09-20T22:20:54.066 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.069 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7f7b31b:::benchmark_data_smithi060_196073_object50213:head# 2024-09-20T22:20:54.069 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.069 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7f8542b:::benchmark_data_smithi060_196073_object7207:head# 2024-09-20T22:20:54.070 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.070 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7f90733:::benchmark_data_smithi060_196073_object55711:head# 2024-09-20T22:20:54.070 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.070 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7fbdd5a:::benchmark_data_smithi060_196073_object59334:head# 2024-09-20T22:20:54.070 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.070 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7fcff3c:::benchmark_data_smithi060_196073_object19725:head# 2024-09-20T22:20:54.070 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.071 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7fe2f52:::benchmark_data_smithi060_196073_object27700:head# 2024-09-20T22:20:54.071 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.071 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7fe3468:::benchmark_data_smithi060_196073_object34756:head# 2024-09-20T22:20:54.071 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.072 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7fe564b:::benchmark_data_smithi060_196073_object3204:head# 2024-09-20T22:20:54.072 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.072 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7feaf2e:::benchmark_data_smithi060_196073_object8337:head# 2024-09-20T22:20:54.072 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.073 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b7ff66ea:::benchmark_data_smithi060_196073_object8077:head# 2024-09-20T22:20:54.073 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.073 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8002c84:::benchmark_data_smithi060_196073_object38931:head# 2024-09-20T22:20:54.073 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.073 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8008f9c:::benchmark_data_smithi060_196073_object25270:head# 2024-09-20T22:20:54.073 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.074 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8013544:::benchmark_data_smithi060_196073_object53557:head# 2024-09-20T22:20:54.074 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.074 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b80146cc:::benchmark_data_smithi060_196073_object59429:head# 2024-09-20T22:20:54.074 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.075 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b802a085:::benchmark_data_smithi060_196073_object29959:head# 2024-09-20T22:20:54.075 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.075 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:54.075 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b804b375:::benchmark_data_smithi060_196073_object15406:head# 2024-09-20T22:20:54.076 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.076 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b805c9d4:::benchmark_data_smithi060_196073_object53826:head# 2024-09-20T22:20:54.076 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.076 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b805e93f:::benchmark_data_smithi060_196073_object3280:head# 2024-09-20T22:20:54.076 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.076 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b806d707:::benchmark_data_smithi060_196073_object32333:head# 2024-09-20T22:20:54.077 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.079 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8099a98:::benchmark_data_smithi060_196073_object3159:head# 2024-09-20T22:20:54.079 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.079 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b80de8ba:::benchmark_data_smithi060_196073_object13988:head# 2024-09-20T22:20:54.079 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.079 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b80e0acf:::benchmark_data_smithi060_196073_object42399:head# 2024-09-20T22:20:54.079 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.079 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b80fbc33:::benchmark_data_smithi060_196073_object55388:head# 2024-09-20T22:20:54.079 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.081 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b80fc443:::benchmark_data_smithi060_196073_object53213:head# 2024-09-20T22:20:54.081 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.081 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b81685de:::benchmark_data_smithi060_196073_object28406:head# 2024-09-20T22:20:54.081 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.081 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b816900a:::benchmark_data_smithi060_196073_object31586:head# 2024-09-20T22:20:54.081 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.081 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b81757e5:::benchmark_data_smithi060_196073_object47719:head# 2024-09-20T22:20:54.081 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.081 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8181180:::benchmark_data_smithi060_196073_object51446:head# 2024-09-20T22:20:54.081 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.081 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8183fb3:::benchmark_data_smithi060_196073_object2955:head# 2024-09-20T22:20:54.081 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.082 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8184065:::benchmark_data_smithi060_196073_object17951:head# 2024-09-20T22:20:54.082 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.082 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8185402:::benchmark_data_smithi060_196073_object44868:head# 2024-09-20T22:20:54.082 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.084 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b819ebc1:::benchmark_data_smithi060_196073_object60475:head# 2024-09-20T22:20:54.084 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.084 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b81a7c23:::benchmark_data_smithi060_196073_object40036:head# 2024-09-20T22:20:54.084 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.084 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b81a8d37:::benchmark_data_smithi060_196073_object26879:head# 2024-09-20T22:20:54.084 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.084 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b81a9128:::benchmark_data_smithi060_196073_object58329:head# 2024-09-20T22:20:54.084 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.084 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b81bbb36:::benchmark_data_smithi060_196073_object20655:head# 2024-09-20T22:20:54.084 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.085 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b81c3b8e:::benchmark_data_smithi060_196073_object13376:head# 2024-09-20T22:20:54.085 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.085 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b81cdebc:::benchmark_data_smithi060_196073_object29441:head# 2024-09-20T22:20:54.086 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.086 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b81d6ba3:::benchmark_data_smithi060_196073_object42872:head# 2024-09-20T22:20:54.086 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.086 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b81db480:::benchmark_data_smithi060_196073_object60998:head# 2024-09-20T22:20:54.087 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.087 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b821779c:::benchmark_data_smithi060_196073_object11722:head# 2024-09-20T22:20:54.088 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.088 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b821d802:::benchmark_data_smithi060_196073_object27473:head# 2024-09-20T22:20:54.088 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.088 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b822e65c:::benchmark_data_smithi060_196073_object37122:head# 2024-09-20T22:20:54.089 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.089 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8232b34:::benchmark_data_smithi060_196073_object342:head# 2024-09-20T22:20:54.089 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.089 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b82394fb:::benchmark_data_smithi060_196073_object53765:head# 2024-09-20T22:20:54.089 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.093 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8249aa1:::benchmark_data_smithi060_196073_object41715:head# 2024-09-20T22:20:54.093 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.093 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b824e41d:::benchmark_data_smithi060_196073_object27270:head# 2024-09-20T22:20:54.094 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.094 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b824e547:::benchmark_data_smithi060_196073_object5443:head# 2024-09-20T22:20:54.094 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.094 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8270a9a:::benchmark_data_smithi060_196073_object50766:head# 2024-09-20T22:20:54.094 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.094 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b827a8a6:::benchmark_data_smithi060_196073_object5587:head# 2024-09-20T22:20:54.094 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.094 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b82868e0:::benchmark_data_smithi060_196073_object59973:head# 2024-09-20T22:20:54.094 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.094 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b829414d:::benchmark_data_smithi060_196073_object21794:head# 2024-09-20T22:20:54.094 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.094 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b82aefa6:::benchmark_data_smithi060_196073_object49202:head# 2024-09-20T22:20:54.094 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.095 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b82b432f:::benchmark_data_smithi060_196073_object27979:head# 2024-09-20T22:20:54.095 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.095 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b82cab8b:::benchmark_data_smithi060_196073_object11684:head# 2024-09-20T22:20:54.095 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.095 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b83056f3:::benchmark_data_smithi060_196073_object30457:head# 2024-09-20T22:20:54.095 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.095 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8314d17:::benchmark_data_smithi060_196073_object33774:head# 2024-09-20T22:20:54.096 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.098 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b832e86a:::benchmark_data_smithi060_196073_object33652:head# 2024-09-20T22:20:54.098 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.098 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b833583d:::benchmark_data_smithi060_196073_object9052:head# 2024-09-20T22:20:54.098 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.098 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b833c675:::benchmark_data_smithi060_196073_object27968:head# 2024-09-20T22:20:54.098 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.098 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b834b023:::benchmark_data_smithi060_196073_object34540:head# 2024-09-20T22:20:54.098 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.099 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8357e78:::benchmark_data_smithi060_196073_object35050:head# 2024-09-20T22:20:54.099 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.099 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b835d7d2:::benchmark_data_smithi060_196073_object4472:head# 2024-09-20T22:20:54.099 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.099 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b83661b3:::benchmark_data_smithi060_196073_object19367:head# 2024-09-20T22:20:54.099 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.100 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b836b36b:::benchmark_data_smithi060_196073_object21311:head# 2024-09-20T22:20:54.100 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.100 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b836d054:::benchmark_data_smithi060_196073_object45806:head# 2024-09-20T22:20:54.100 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.100 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8382dde:::benchmark_data_smithi060_196073_object52396:head# 2024-09-20T22:20:54.100 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.100 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8387557:::benchmark_data_smithi060_196073_object31022:head# 2024-09-20T22:20:54.100 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.101 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b838c051:::benchmark_data_smithi060_196073_object31336:head# 2024-09-20T22:20:54.101 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.101 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b83b02bd:::benchmark_data_smithi060_196073_object56080:head# 2024-09-20T22:20:54.101 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.104 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b83b2dee:::benchmark_data_smithi060_196073_object27304:head# 2024-09-20T22:20:54.104 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.104 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b83b8768:::benchmark_data_smithi060_196073_object29890:head# 2024-09-20T22:20:54.104 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.105 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b83c7f05:::benchmark_data_smithi060_196073_object42182:head# 2024-09-20T22:20:54.105 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.105 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b83d6cd3:::benchmark_data_smithi060_196073_object38131:head# 2024-09-20T22:20:54.105 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.105 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b83e6552:::benchmark_data_smithi060_196073_object10028:head# 2024-09-20T22:20:54.105 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.105 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b83fd279:::benchmark_data_smithi060_196073_object17711:head# 2024-09-20T22:20:54.105 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.105 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8409b85:::benchmark_data_smithi060_196073_object42195:head# 2024-09-20T22:20:54.105 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.105 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b84228aa:::benchmark_data_smithi060_196073_object29289:head# 2024-09-20T22:20:54.105 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.105 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b843262e:::benchmark_data_smithi060_196073_object6934:head# 2024-09-20T22:20:54.105 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.106 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b84824f7:::benchmark_data_smithi060_196073_object24365:head# 2024-09-20T22:20:54.106 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.106 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b84ab0a8:::benchmark_data_smithi060_196073_object35859:head# 2024-09-20T22:20:54.106 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.106 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b84aef72:::benchmark_data_smithi060_196073_object17235:head# 2024-09-20T22:20:54.106 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.106 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b84bb492:::benchmark_data_smithi060_196073_object21750:head# 2024-09-20T22:20:54.107 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.107 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b84c790f:::benchmark_data_smithi060_196073_object40508:head# 2024-09-20T22:20:54.107 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.107 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b84d017f:::benchmark_data_smithi060_196073_object40964:head# 2024-09-20T22:20:54.107 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.108 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b84dfefe:::benchmark_data_smithi060_196073_object41239:head# 2024-09-20T22:20:54.108 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.108 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b84fd434:::benchmark_data_smithi060_196073_object6782:head# 2024-09-20T22:20:54.108 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.108 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b85003bb:::benchmark_data_smithi060_196073_object47032:head# 2024-09-20T22:20:54.108 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.109 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8501b07:::benchmark_data_smithi060_196073_object856:head# 2024-09-20T22:20:54.109 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.109 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8514cf8:::benchmark_data_smithi060_196073_object59852:head# 2024-09-20T22:20:54.109 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.110 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b851b2a9:::benchmark_data_smithi060_196073_object54287:head# 2024-09-20T22:20:54.110 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.111 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8522d2f:::benchmark_data_smithi060_196073_object28217:head# 2024-09-20T22:20:54.111 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.111 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8539eff:::benchmark_data_smithi060_196073_object6581:head# 2024-09-20T22:20:54.111 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.111 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b854a639:::benchmark_data_smithi060_196073_object18424:head# 2024-09-20T22:20:54.112 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.112 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b854ace5:::benchmark_data_smithi060_196073_object18068:head# 2024-09-20T22:20:54.112 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.112 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b854be28:::benchmark_data_smithi060_196073_object9564:head# 2024-09-20T22:20:54.112 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.113 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b85545c6:::benchmark_data_smithi060_196073_object38059:head# 2024-09-20T22:20:54.113 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.113 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8554fab:::benchmark_data_smithi060_196073_object35594:head# 2024-09-20T22:20:54.113 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.114 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b855cf18:::benchmark_data_smithi060_196073_object12528:head# 2024-09-20T22:20:54.114 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.114 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b85720d9:::benchmark_data_smithi060_196073_object37667:head# 2024-09-20T22:20:54.114 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.114 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b85ac0d7:::benchmark_data_smithi060_196073_object57160:head# 2024-09-20T22:20:54.115 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.115 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b85c82d1:::benchmark_data_smithi060_196073_object50260:head# 2024-09-20T22:20:54.115 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.116 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b86137c1:::benchmark_data_smithi060_196073_object37760:head# 2024-09-20T22:20:54.116 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.116 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8634874:::benchmark_data_smithi060_196073_object49387:head# 2024-09-20T22:20:54.116 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.117 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b864f932:::benchmark_data_smithi060_196073_object21076:head# 2024-09-20T22:20:54.117 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.117 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8667f65:::benchmark_data_smithi060_196073_object38186:head# 2024-09-20T22:20:54.117 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.117 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b866a1f3:::benchmark_data_smithi060_196073_object13727:head# 2024-09-20T22:20:54.117 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.117 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:54.118 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b867052c:::benchmark_data_smithi060_196073_object37520:head# 2024-09-20T22:20:54.118 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.118 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8671a8c:::benchmark_data_smithi060_196073_object27048:head# 2024-09-20T22:20:54.119 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.119 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:54.119 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:54.119 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8681a93:::benchmark_data_smithi060_196073_object12106:head# 2024-09-20T22:20:54.119 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.119 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b86a8719:::benchmark_data_smithi060_196073_object35496:head# 2024-09-20T22:20:54.120 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.120 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b86aac2a:::benchmark_data_smithi060_196073_object7639:head# 2024-09-20T22:20:54.120 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.120 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b86caa10:::benchmark_data_smithi060_196073_object18244:head# 2024-09-20T22:20:54.120 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.121 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b86ce0b5:::benchmark_data_smithi060_196073_object28122:head# 2024-09-20T22:20:54.121 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.122 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b86eebb4:::benchmark_data_smithi060_196073_object3081:head# 2024-09-20T22:20:54.122 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.122 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b86f86a1:::benchmark_data_smithi060_196073_object59306:head# 2024-09-20T22:20:54.122 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.122 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b872faf4:::benchmark_data_smithi060_196073_object24697:head# 2024-09-20T22:20:54.122 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.124 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b874120b:::benchmark_data_smithi060_196073_object6414:head# 2024-09-20T22:20:54.124 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.124 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8754692:::benchmark_data_smithi060_196073_object52193:head# 2024-09-20T22:20:54.124 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.124 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8757ad9:::benchmark_data_smithi060_196073_object61513:head# 2024-09-20T22:20:54.124 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.125 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b875eeba:::benchmark_data_smithi060_196073_object18134:head# 2024-09-20T22:20:54.125 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.125 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8766d4e:::benchmark_data_smithi060_196073_object2166:head# 2024-09-20T22:20:54.133 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.134 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b876fa17:::benchmark_data_smithi060_196073_object28237:head# 2024-09-20T22:20:54.134 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.134 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8774c5b:::benchmark_data_smithi060_196073_object36090:head# 2024-09-20T22:20:54.134 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.134 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8789fd9:::benchmark_data_smithi060_196073_object28453:head# 2024-09-20T22:20:54.134 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.134 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b879fe24:::benchmark_data_smithi060_196073_object4322:head# 2024-09-20T22:20:54.134 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.135 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b87cc3b3:::benchmark_data_smithi060_196073_object41172:head# 2024-09-20T22:20:54.135 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.135 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b87e7a2e:::benchmark_data_smithi060_196073_object24673:head# 2024-09-20T22:20:54.135 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.135 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b87fc62e:::benchmark_data_smithi060_196073_object85:head# 2024-09-20T22:20:54.135 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.135 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b882ee44:::benchmark_data_smithi060_196073_object9237:head# 2024-09-20T22:20:54.135 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.136 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8844e33:::benchmark_data_smithi060_196073_object27975:head# 2024-09-20T22:20:54.136 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.137 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b884db60:::benchmark_data_smithi060_196073_object60248:head# 2024-09-20T22:20:54.137 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.137 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b88629b8:::benchmark_data_smithi060_196073_object33556:head# 2024-09-20T22:20:54.138 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.138 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b886cd10:::benchmark_data_smithi060_196073_object21725:head# 2024-09-20T22:20:54.138 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.138 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b88a013e:::benchmark_data_smithi060_196073_object10661:head# 2024-09-20T22:20:54.138 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.138 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b88a3c81:::benchmark_data_smithi060_196073_object24381:head# 2024-09-20T22:20:54.138 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.138 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b88a72d7:::benchmark_data_smithi060_196073_object18161:head# 2024-09-20T22:20:54.139 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.139 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b88b3cf5:::benchmark_data_smithi060_196073_object7135:head# 2024-09-20T22:20:54.139 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.139 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b88b5d9e:::benchmark_data_smithi060_196073_object52482:head# 2024-09-20T22:20:54.139 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.140 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b88c8983:::benchmark_data_smithi060_196073_object39798:head# 2024-09-20T22:20:54.140 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.140 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b88cf720:::benchmark_data_smithi060_196073_object23326:head# 2024-09-20T22:20:54.140 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b88e1faf:::benchmark_data_smithi060_196073_object9688:head# 2024-09-20T22:20:54.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b88e89a4:::benchmark_data_smithi060_196073_object716:head# 2024-09-20T22:20:54.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8909feb:::benchmark_data_smithi060_196073_object55104:head# 2024-09-20T22:20:54.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b89149da:::benchmark_data_smithi060_196073_object25136:head# 2024-09-20T22:20:54.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b892693d:::benchmark_data_smithi060_196073_object21977:head# 2024-09-20T22:20:54.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8928305:::benchmark_data_smithi060_196073_object57536:head# 2024-09-20T22:20:54.144 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.144 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b893e074:::benchmark_data_smithi060_196073_object54660:head# 2024-09-20T22:20:54.144 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.144 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8950aa2:::benchmark_data_smithi060_196073_object53813:head# 2024-09-20T22:20:54.144 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.144 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8974397:::benchmark_data_smithi060_196073_object20770:head# 2024-09-20T22:20:54.145 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.145 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b89aa745:::benchmark_data_smithi060_196073_object46953:head# 2024-09-20T22:20:54.145 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.145 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b89acb06:::benchmark_data_smithi060_196073_object20695:head# 2024-09-20T22:20:54.145 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.146 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b89f71ad:::benchmark_data_smithi060_196073_object37393:head# 2024-09-20T22:20:54.146 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.146 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b89f7f68:::benchmark_data_smithi060_196073_object33831:head# 2024-09-20T22:20:54.146 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.146 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b89ff576:::benchmark_data_smithi060_196073_object32785:head# 2024-09-20T22:20:54.146 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.147 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8a01fa6:::benchmark_data_smithi060_196073_object25745:head# 2024-09-20T22:20:54.147 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.147 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8a0604c:::benchmark_data_smithi060_196073_object12497:head# 2024-09-20T22:20:54.147 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.148 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8a11961:::benchmark_data_smithi060_196073_object54743:head# 2024-09-20T22:20:54.148 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.149 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8a15664:::benchmark_data_smithi060_196073_object13455:head# 2024-09-20T22:20:54.149 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.149 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8a37d8f:::benchmark_data_smithi060_196073_object11523:head# 2024-09-20T22:20:54.149 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.149 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8a399a2:::benchmark_data_smithi060_196073_object32613:head# 2024-09-20T22:20:54.149 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.150 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8a41fad:::benchmark_data_smithi060_196073_object37029:head# 2024-09-20T22:20:54.150 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.150 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8a4a42c:::benchmark_data_smithi060_196073_object47229:head# 2024-09-20T22:20:54.150 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.150 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8a689ca:::benchmark_data_smithi060_196073_object7548:head# 2024-09-20T22:20:54.150 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.151 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8a8dca5:::benchmark_data_smithi060_196073_object34601:head# 2024-09-20T22:20:54.151 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.151 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8aa3b9f:::benchmark_data_smithi060_196073_object21187:head# 2024-09-20T22:20:54.152 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.152 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8ab352b:::benchmark_data_smithi060_196073_object40592:head# 2024-09-20T22:20:54.152 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.152 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8ab4c61:::benchmark_data_smithi060_196073_object23371:head# 2024-09-20T22:20:54.152 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.152 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8ab9d13:::benchmark_data_smithi060_196073_object20232:head# 2024-09-20T22:20:54.152 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.153 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8ae06a0:::benchmark_data_smithi060_196073_object3794:head# 2024-09-20T22:20:54.153 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.153 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8ae6af9:::benchmark_data_smithi060_196073_object23392:head# 2024-09-20T22:20:54.153 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.154 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8af5de5:::benchmark_data_smithi060_196073_object473:head# 2024-09-20T22:20:54.154 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.154 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8b09772:::benchmark_data_smithi060_196073_object6248:head# 2024-09-20T22:20:54.154 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.154 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8b0b039:::benchmark_data_smithi060_196073_object51793:head# 2024-09-20T22:20:54.154 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.155 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8b15667:::benchmark_data_smithi060_196073_object23761:head# 2024-09-20T22:20:54.155 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.157 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8b1a2e3:::benchmark_data_smithi060_196073_object45307:head# 2024-09-20T22:20:54.157 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.158 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8b1dc12:::benchmark_data_smithi060_196073_object56782:head# 2024-09-20T22:20:54.158 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.161 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8b2fb09:::benchmark_data_smithi060_196073_object25132:head# 2024-09-20T22:20:54.161 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.161 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8b8af3b:::benchmark_data_smithi060_196073_object20809:head# 2024-09-20T22:20:54.161 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.162 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8baacc5:::benchmark_data_smithi060_196073_object6550:head# 2024-09-20T22:20:54.162 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.162 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8beca70:::benchmark_data_smithi060_196073_object27646:head# 2024-09-20T22:20:54.162 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.162 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8bee21c:::benchmark_data_smithi060_196073_object2642:head# 2024-09-20T22:20:54.162 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.162 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8bf11a7:::benchmark_data_smithi060_196073_object13447:head# 2024-09-20T22:20:54.162 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.162 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8bf8c2b:::benchmark_data_smithi060_196073_object5858:head# 2024-09-20T22:20:54.162 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.163 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8c09fbc:::benchmark_data_smithi060_196073_object33739:head# 2024-09-20T22:20:54.163 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.163 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8c0fc83:::benchmark_data_smithi060_196073_object14250:head# 2024-09-20T22:20:54.163 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.163 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8c255ab:::benchmark_data_smithi060_196073_object55844:head# 2024-09-20T22:20:54.163 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.163 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8c3dfc3:::benchmark_data_smithi060_196073_object11803:head# 2024-09-20T22:20:54.163 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.164 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8c4395e:::benchmark_data_smithi060_196073_object27144:head# 2024-09-20T22:20:54.164 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.164 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8c61d0a:::benchmark_data_smithi060_196073_object3838:head# 2024-09-20T22:20:54.164 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.164 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8c92385:::benchmark_data_smithi060_196073_object33935:head# 2024-09-20T22:20:54.164 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.165 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8c99acf:::benchmark_data_smithi060_196073_object2724:head# 2024-09-20T22:20:54.165 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.165 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8c9e5fb:::benchmark_data_smithi060_196073_object29368:head# 2024-09-20T22:20:54.165 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.166 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8cbe328:::benchmark_data_smithi060_196073_object46086:head# 2024-09-20T22:20:54.166 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.166 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8cd270b:::benchmark_data_smithi060_196073_object19713:head# 2024-09-20T22:20:54.166 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.167 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8cd4f2e:::benchmark_data_smithi060_196073_object54708:head# 2024-09-20T22:20:54.167 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.167 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8cdce40:::benchmark_data_smithi060_196073_object59046:head# 2024-09-20T22:20:54.167 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.167 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8cf01db:::benchmark_data_smithi060_196073_object29555:head# 2024-09-20T22:20:54.167 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.168 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8cf3b88:::benchmark_data_smithi060_196073_object15127:head# 2024-09-20T22:20:54.168 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.168 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8d12ba2:::benchmark_data_smithi060_196073_object15572:head# 2024-09-20T22:20:54.168 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.169 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8d349fb:::benchmark_data_smithi060_196073_object49928:head# 2024-09-20T22:20:54.169 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.172 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8d3adc6:::benchmark_data_smithi060_196073_object53423:head# 2024-09-20T22:20:54.172 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.172 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8d5aa1a:::benchmark_data_smithi060_196073_object47160:head# 2024-09-20T22:20:54.172 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.173 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8d7719a:::benchmark_data_smithi060_196073_object58758:head# 2024-09-20T22:20:54.173 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.174 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8d8b6ca:::benchmark_data_smithi060_196073_object21197:head# 2024-09-20T22:20:54.174 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.174 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8da08e5:::benchmark_data_smithi060_196073_object58415:head# 2024-09-20T22:20:54.174 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.175 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8da999a:::benchmark_data_smithi060_196073_object40370:head# 2024-09-20T22:20:54.175 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.175 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8db763c:::benchmark_data_smithi060_196073_object40472:head# 2024-09-20T22:20:54.175 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.176 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8dbbbdb:::benchmark_data_smithi060_196073_object6803:head# 2024-09-20T22:20:54.176 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.176 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8dcaad3:::benchmark_data_smithi060_196073_object27005:head# 2024-09-20T22:20:54.176 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.176 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8dcbc22:::benchmark_data_smithi060_196073_object1683:head# 2024-09-20T22:20:54.176 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.177 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8de1580:::benchmark_data_smithi060_196073_object14520:head# 2024-09-20T22:20:54.177 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.177 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8df1b20:::benchmark_data_smithi060_196073_object48306:head# 2024-09-20T22:20:54.177 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.178 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8df713e:::benchmark_data_smithi060_196073_object47856:head# 2024-09-20T22:20:54.178 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.179 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8e0a113:::benchmark_data_smithi060_196073_object47276:head# 2024-09-20T22:20:54.179 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.179 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8e27aee:::benchmark_data_smithi060_196073_object34757:head# 2024-09-20T22:20:54.179 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.180 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8e3b683:::benchmark_data_smithi060_196073_object20570:head# 2024-09-20T22:20:54.189 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:20:54.189 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.190 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8e482f1:::benchmark_data_smithi060_196073_object15230:head# 2024-09-20T22:20:54.190 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.191 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8e4baf2:::benchmark_data_smithi060_196073_object25544:head# 2024-09-20T22:20:54.191 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.191 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8e5b868:::benchmark_data_smithi060_196073_object1178:head# 2024-09-20T22:20:54.191 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.191 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8e7263c:::benchmark_data_smithi060_196073_object20050:head# 2024-09-20T22:20:54.191 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.192 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8e78436:::benchmark_data_smithi060_196073_object57629:head# 2024-09-20T22:20:54.192 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.192 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8eaa534:::benchmark_data_smithi060_196073_object22424:head# 2024-09-20T22:20:54.192 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.193 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8ec9063:::benchmark_data_smithi060_196073_object48374:head# 2024-09-20T22:20:54.193 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.193 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8ed6347:::benchmark_data_smithi060_196073_object35886:head# 2024-09-20T22:20:54.193 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.194 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8ef724b:::benchmark_data_smithi060_196073_object23624:head# 2024-09-20T22:20:54.194 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.194 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8f077ee:::benchmark_data_smithi060_196073_object34808:head# 2024-09-20T22:20:54.194 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.195 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8f2895f:::benchmark_data_smithi060_196073_object37211:head# 2024-09-20T22:20:54.195 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.195 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8f2da2b:::benchmark_data_smithi060_196073_object34758:head# 2024-09-20T22:20:54.196 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.196 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8f4544a:::benchmark_data_smithi060_196073_object17480:head# 2024-09-20T22:20:54.196 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.196 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8f4673a:::benchmark_data_smithi060_196073_object47698:head# 2024-09-20T22:20:54.196 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.196 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8f476db:::benchmark_data_smithi060_196073_object3984:head# 2024-09-20T22:20:54.196 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.197 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8f48364:::benchmark_data_smithi060_196073_object57947:head# 2024-09-20T22:20:54.197 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.197 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8f59a83:::benchmark_data_smithi060_196073_object45786:head# 2024-09-20T22:20:54.197 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.198 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8f7c32d:::benchmark_data_smithi060_196073_object45489:head# 2024-09-20T22:20:54.198 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.198 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8faef79:::benchmark_data_smithi060_196073_object45213:head# 2024-09-20T22:20:54.198 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.199 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8fc542f:::benchmark_data_smithi060_196073_object42278:head# 2024-09-20T22:20:54.199 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.199 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b8fcd953:::benchmark_data_smithi060_196073_object28561:head# 2024-09-20T22:20:54.199 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.199 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b90006d0:::benchmark_data_smithi060_196073_object9260:head# 2024-09-20T22:20:54.199 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.199 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9028474:::benchmark_data_smithi060_196073_object54350:head# 2024-09-20T22:20:54.199 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.200 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b907442c:::benchmark_data_smithi060_196073_object3626:head# 2024-09-20T22:20:54.200 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.200 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b908fa98:::benchmark_data_smithi060_196073_object43667:head# 2024-09-20T22:20:54.200 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.200 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b90af5c2:::benchmark_data_smithi060_196073_object8397:head# 2024-09-20T22:20:54.201 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.201 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b90e42df:::benchmark_data_smithi060_196073_object55733:head# 2024-09-20T22:20:54.201 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.201 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b91690f2:::benchmark_data_smithi060_196073_object51088:head# 2024-09-20T22:20:54.201 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.201 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9178296:::benchmark_data_smithi060_196073_object16715:head# 2024-09-20T22:20:54.202 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.202 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b91b8a7e:::benchmark_data_smithi060_196073_object37506:head# 2024-09-20T22:20:54.202 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.202 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b91c3e1f:::benchmark_data_smithi060_196073_object58991:head# 2024-09-20T22:20:54.202 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.202 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b91e6df8:::benchmark_data_smithi060_196073_object11402:head# 2024-09-20T22:20:54.210 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.211 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b91f9cb7:::benchmark_data_smithi060_196073_object5801:head# 2024-09-20T22:20:54.211 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.212 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9201e38:::benchmark_data_smithi060_196073_object54493:head# 2024-09-20T22:20:54.212 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.212 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9202a90:::benchmark_data_smithi060_196073_object37974:head# 2024-09-20T22:20:54.213 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.213 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9206974:::benchmark_data_smithi060_196073_object12573:head# 2024-09-20T22:20:54.213 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.214 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b92069a9:::benchmark_data_smithi060_196073_object47346:head# 2024-09-20T22:20:54.214 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.214 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b921a2b4:::benchmark_data_smithi060_196073_object34096:head# 2024-09-20T22:20:54.214 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.214 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9228740:::benchmark_data_smithi060_196073_object17834:head# 2024-09-20T22:20:54.214 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.214 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9238a21:::benchmark_data_smithi060_196073_object30412:head# 2024-09-20T22:20:54.214 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.215 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b924ddce:::benchmark_data_smithi060_196073_object28258:head# 2024-09-20T22:20:54.215 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.215 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b924e4e5:::benchmark_data_smithi060_196073_object5352:head# 2024-09-20T22:20:54.215 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.215 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b925985a:::benchmark_data_smithi060_196073_object26733:head# 2024-09-20T22:20:54.215 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.215 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9269ead:::benchmark_data_smithi060_196073_object56001:head# 2024-09-20T22:20:54.216 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.217 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b926f73e:::benchmark_data_smithi060_196073_object53266:head# 2024-09-20T22:20:54.217 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.217 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b929c54d:::benchmark_data_smithi060_196073_object45183:head# 2024-09-20T22:20:54.217 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.217 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b92a78d6:::benchmark_data_smithi060_196073_object18493:head# 2024-09-20T22:20:54.217 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.217 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b92c5cae:::benchmark_data_smithi060_196073_object10958:head# 2024-09-20T22:20:54.217 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.217 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b92ce3ac:::benchmark_data_smithi060_196073_object51300:head# 2024-09-20T22:20:54.217 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.218 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b92dada0:::benchmark_data_smithi060_196073_object44488:head# 2024-09-20T22:20:54.218 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.218 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b92e17a9:::benchmark_data_smithi060_196073_object17590:head# 2024-09-20T22:20:54.218 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.218 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b92ed993:::benchmark_data_smithi060_196073_object61745:head# 2024-09-20T22:20:54.219 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.219 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:54.220 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b92fd031:::benchmark_data_smithi060_196073_object801:head# 2024-09-20T22:20:54.220 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.220 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9310774:::benchmark_data_smithi060_196073_object47277:head# 2024-09-20T22:20:54.220 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.221 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b931cd6f:::benchmark_data_smithi060_196073_object54748:head# 2024-09-20T22:20:54.221 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.222 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9341f99:::benchmark_data_smithi060_196073_object13379:head# 2024-09-20T22:20:54.222 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.223 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b935429f:::benchmark_data_smithi060_196073_object59690:head# 2024-09-20T22:20:54.223 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.223 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9363330:::benchmark_data_smithi060_196073_object6067:head# 2024-09-20T22:20:54.223 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.223 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9369037:::benchmark_data_smithi060_196073_object8284:head# 2024-09-20T22:20:54.223 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.223 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9375cab:::benchmark_data_smithi060_196073_object36852:head# 2024-09-20T22:20:54.223 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.226 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b93805a7:::benchmark_data_smithi060_196073_object35033:head# 2024-09-20T22:20:54.226 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.226 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b938313e:::benchmark_data_smithi060_196073_object15761:head# 2024-09-20T22:20:54.226 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.226 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9384062:::benchmark_data_smithi060_196073_object8206:head# 2024-09-20T22:20:54.226 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.226 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b938e9c0:::benchmark_data_smithi060_196073_object25612:head# 2024-09-20T22:20:54.226 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.226 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b938ef00:::benchmark_data_smithi060_196073_object28851:head# 2024-09-20T22:20:54.226 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.227 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9395c31:::benchmark_data_smithi060_196073_object21200:head# 2024-09-20T22:20:54.227 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.227 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b93b3297:::benchmark_data_smithi060_196073_object46221:head# 2024-09-20T22:20:54.227 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.227 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b93c06dc:::benchmark_data_smithi060_196073_object51685:head# 2024-09-20T22:20:54.227 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.228 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b93c8762:::benchmark_data_smithi060_196073_object43514:head# 2024-09-20T22:20:54.228 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.228 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b93c922c:::benchmark_data_smithi060_196073_object23667:head# 2024-09-20T22:20:54.228 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b93e28bd:::benchmark_data_smithi060_196073_object1122:head# 2024-09-20T22:20:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b93e4c74:::benchmark_data_smithi060_196073_object45097:head# 2024-09-20T22:20:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b93ebcb2:::benchmark_data_smithi060_196073_object33152:head# 2024-09-20T22:20:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b93f9396:::benchmark_data_smithi060_196073_object50744:head# 2024-09-20T22:20:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b941c73e:::benchmark_data_smithi060_196073_object34649:head# 2024-09-20T22:20:54.229 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.230 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b941d5ef:::benchmark_data_smithi060_196073_object23469:head# 2024-09-20T22:20:54.230 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.230 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9424781:::benchmark_data_smithi060_196073_object3446:head# 2024-09-20T22:20:54.230 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.230 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b942b68d:::benchmark_data_smithi060_196073_object13241:head# 2024-09-20T22:20:54.231 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.231 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b942bc07:::benchmark_data_smithi060_196073_object48067:head# 2024-09-20T22:20:54.231 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.231 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9434817:::benchmark_data_smithi060_196073_object42014:head# 2024-09-20T22:20:54.231 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.232 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b945ec97:::benchmark_data_smithi060_196073_object15549:head# 2024-09-20T22:20:54.232 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.232 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b94630b0:::benchmark_data_smithi060_196073_object41352:head# 2024-09-20T22:20:54.232 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.232 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9472883:::benchmark_data_smithi060_196073_object25356:head# 2024-09-20T22:20:54.232 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.232 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9476456:::benchmark_data_smithi060_196073_object27073:head# 2024-09-20T22:20:54.232 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.232 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b94c611a:::benchmark_data_smithi060_196073_object9058:head# 2024-09-20T22:20:54.233 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.233 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9505516:::benchmark_data_smithi060_196073_object15393:head# 2024-09-20T22:20:54.233 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.233 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b950b947:::benchmark_data_smithi060_196073_object22746:head# 2024-09-20T22:20:54.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:54.245 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:54.245 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:54.279 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.279 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9518a07:::benchmark_data_smithi060_196073_object41994:head# 2024-09-20T22:20:54.279 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.279 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9531af1:::benchmark_data_smithi060_196073_object16734:head# 2024-09-20T22:20:54.280 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.280 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b95363f4:::benchmark_data_smithi060_196073_object14555:head# 2024-09-20T22:20:54.280 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.280 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b95366bd:::benchmark_data_smithi060_196073_object15304:head# 2024-09-20T22:20:54.290 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.290 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b95408da:::benchmark_data_smithi060_196073_object25432:head# 2024-09-20T22:20:54.291 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.291 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9551768:::benchmark_data_smithi060_196073_object34269:head# 2024-09-20T22:20:54.291 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.291 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b955cb28:::benchmark_data_smithi060_196073_object51981:head# 2024-09-20T22:20:54.291 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.292 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9563ad9:::benchmark_data_smithi060_196073_object58727:head# 2024-09-20T22:20:54.309 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.309 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9563f53:::benchmark_data_smithi060_196073_object7415:head# 2024-09-20T22:20:54.309 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.310 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b957928c:::benchmark_data_smithi060_196073_object23474:head# 2024-09-20T22:20:54.310 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.310 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9582dcf:::benchmark_data_smithi060_196073_object40341:head# 2024-09-20T22:20:54.311 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.311 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b959a9f3:::benchmark_data_smithi060_196073_object60534:head# 2024-09-20T22:20:54.323 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.324 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b95a9c3b:::benchmark_data_smithi060_196073_object37021:head# 2024-09-20T22:20:54.324 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.324 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b95abc44:::benchmark_data_smithi060_196073_object13449:head# 2024-09-20T22:20:54.324 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.325 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b95bebf3:::benchmark_data_smithi060_196073_object53061:head# 2024-09-20T22:20:54.325 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.325 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b95d18ac:::benchmark_data_smithi060_196073_object9791:head# 2024-09-20T22:20:54.346 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.346 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b95d573a:::benchmark_data_smithi060_196073_object19946:head# 2024-09-20T22:20:54.346 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.346 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:54.347 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b95e1611:::benchmark_data_smithi060_196073_object7196:head# 2024-09-20T22:20:54.347 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.347 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b95e16b8:::benchmark_data_smithi060_196073_object53851:head# 2024-09-20T22:20:54.347 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.347 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b95fcdc4:::benchmark_data_smithi060_196073_object57612:head# 2024-09-20T22:20:54.358 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.359 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b96070f0:::benchmark_data_smithi060_196073_object21957:head# 2024-09-20T22:20:54.359 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.359 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b960eb96:::benchmark_data_smithi060_196073_object45800:head# 2024-09-20T22:20:54.359 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.359 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9618b82:::benchmark_data_smithi060_196073_object12589:head# 2024-09-20T22:20:54.359 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.359 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b961fb2e:::benchmark_data_smithi060_196073_object38526:head# 2024-09-20T22:20:54.365 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.365 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b96390a0:::benchmark_data_smithi060_196073_object59347:head# 2024-09-20T22:20:54.365 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.365 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9662a3e:::benchmark_data_smithi060_196073_object27455:head# 2024-09-20T22:20:54.384 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:54.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:54.385 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:54.415 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.416 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9687beb:::benchmark_data_smithi060_196073_object40392:head# 2024-09-20T22:20:54.416 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.416 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b96c46e0:::benchmark_data_smithi060_196073_object58914:head# 2024-09-20T22:20:54.416 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.416 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b96cbe1d:::benchmark_data_smithi060_196073_object19535:head# 2024-09-20T22:20:54.417 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.417 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b96e6663:::benchmark_data_smithi060_196073_object16195:head# 2024-09-20T22:20:54.442 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.442 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b96ed098:::benchmark_data_smithi060_196073_object34847:head# 2024-09-20T22:20:54.442 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.443 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9715197:::benchmark_data_smithi060_196073_object37545:head# 2024-09-20T22:20:54.443 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.443 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b971f7d9:::benchmark_data_smithi060_196073_object45931:head# 2024-09-20T22:20:54.444 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.444 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9744e27:::benchmark_data_smithi060_196073_object6328:head# 2024-09-20T22:20:54.470 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.471 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9758c7e:::benchmark_data_smithi060_196073_object50951:head# 2024-09-20T22:20:54.471 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.471 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b975922f:::benchmark_data_smithi060_196073_object22927:head# 2024-09-20T22:20:54.471 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.472 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b975b286:::benchmark_data_smithi060_196073_object15130:head# 2024-09-20T22:20:54.472 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.472 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9763765:::benchmark_data_smithi060_196073_object4872:head# 2024-09-20T22:20:54.487 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:54.502 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.503 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b97718c8:::benchmark_data_smithi060_196073_object58546:head# 2024-09-20T22:20:54.503 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.504 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b978c6ec:::benchmark_data_smithi060_196073_object35709:head# 2024-09-20T22:20:54.504 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.504 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9790c29:::benchmark_data_smithi060_196073_object35142:head# 2024-09-20T22:20:54.504 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.504 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b97a6853:::benchmark_data_smithi060_196073_object58468:head# 2024-09-20T22:20:54.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:54.521 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:54.521 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:54.521 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.522 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b97bbe85:::benchmark_data_smithi060_196073_object7687:head# 2024-09-20T22:20:54.522 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.522 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b97c2b8c:::benchmark_data_smithi060_196073_object698:head# 2024-09-20T22:20:54.522 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.522 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b97c930d:::benchmark_data_smithi060_196073_object40550:head# 2024-09-20T22:20:54.522 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.523 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b97dde85:::benchmark_data_smithi060_196073_object41346:head# 2024-09-20T22:20:54.567 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.568 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b97e1fb5:::benchmark_data_smithi060_196073_object49211:head# 2024-09-20T22:20:54.568 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.568 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b97f0f59:::benchmark_data_smithi060_196073_object14252:head# 2024-09-20T22:20:54.568 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.569 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b982160a:::benchmark_data_smithi060_196073_object14970:head# 2024-09-20T22:20:54.599 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.599 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b982cda0:::benchmark_data_smithi060_196073_object27120:head# 2024-09-20T22:20:54.599 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.600 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9842bbd:::benchmark_data_smithi060_196073_object40181:head# 2024-09-20T22:20:54.600 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.600 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b984acb2:::benchmark_data_smithi060_196073_object57722:head# 2024-09-20T22:20:54.600 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.600 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b984e9c0:::benchmark_data_smithi060_196073_object60014:head# 2024-09-20T22:20:54.622 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:54.634 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.634 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9858a86:::benchmark_data_smithi060_196073_object13610:head# 2024-09-20T22:20:54.634 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.634 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b98690fc:::benchmark_data_smithi060_196073_object61552:head# 2024-09-20T22:20:54.634 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.635 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b986ddf4:::benchmark_data_smithi060_196073_object27103:head# 2024-09-20T22:20:54.635 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.635 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b98793c4:::benchmark_data_smithi060_196073_object16934:head# 2024-09-20T22:20:54.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:54.656 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:54.656 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:54.667 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.667 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b98ab4d0:::benchmark_data_smithi060_196073_object57219:head# 2024-09-20T22:20:54.667 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.668 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b98abab0:::benchmark_data_smithi060_196073_object28765:head# 2024-09-20T22:20:54.668 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.668 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b98ca8f7:::benchmark_data_smithi060_196073_object14386:head# 2024-09-20T22:20:54.668 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.668 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b98d1eab:::benchmark_data_smithi060_196073_object16436:head# 2024-09-20T22:20:54.711 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.712 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:54 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:20:54.712 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:54 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:20:54.712 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:54 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3186488298' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:20:54.712 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:54 smithi067 ceph-mon[33511]: osdmap e506: 8 total, 7 up, 6 in 2024-09-20T22:20:54.713 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b98deebb:::benchmark_data_smithi060_196073_object33133:head# 2024-09-20T22:20:54.713 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.713 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b98e4b56:::benchmark_data_smithi060_196073_object26681:head# 2024-09-20T22:20:54.713 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.713 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b98e5002:::benchmark_data_smithi060_196073_object21098:head# 2024-09-20T22:20:54.713 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.713 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b98e644a:::benchmark_data_smithi060_196073_object17207:head# 2024-09-20T22:20:54.726 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.726 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b98f3909:::benchmark_data_smithi060_196073_object56306:head# 2024-09-20T22:20:54.726 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.727 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b991a0c2:::benchmark_data_smithi060_196073_object55394:head# 2024-09-20T22:20:54.727 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.727 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b991bb5a:::benchmark_data_smithi060_196073_object1957:head# 2024-09-20T22:20:54.727 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.727 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b991e77a:::benchmark_data_smithi060_196073_object7015:head# 2024-09-20T22:20:54.728 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.728 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b992fd9f:::benchmark_data_smithi060_196073_object34899:head# 2024-09-20T22:20:54.729 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.729 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b993797c:::benchmark_data_smithi060_196073_object9112:head# 2024-09-20T22:20:54.736 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.737 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b999148a:::benchmark_data_smithi060_196073_object42618:head# 2024-09-20T22:20:54.737 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.737 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9996be9:::benchmark_data_smithi060_196073_object15193:head# 2024-09-20T22:20:54.738 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.738 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b999e6b1:::benchmark_data_smithi060_196073_object31478:head# 2024-09-20T22:20:54.738 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.738 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b99a8bb5:::benchmark_data_smithi060_196073_object60147:head# 2024-09-20T22:20:54.739 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.739 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b99a9c52:::benchmark_data_smithi060_196073_object12925:head# 2024-09-20T22:20:54.740 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.740 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b99b20c6:::benchmark_data_smithi060_196073_object26799:head# 2024-09-20T22:20:54.747 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.747 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b99b5aac:::benchmark_data_smithi060_196073_object41557:head# 2024-09-20T22:20:54.747 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.748 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b99b7cfc:::benchmark_data_smithi060_196073_object19065:head# 2024-09-20T22:20:54.748 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.748 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b99d50fe:::benchmark_data_smithi060_196073_object30282:head# 2024-09-20T22:20:54.748 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.748 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b99dd65f:::benchmark_data_smithi060_196073_object61014:head# 2024-09-20T22:20:54.757 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:54.757 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.761 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b99e1fbb:::benchmark_data_smithi060_196073_object54254:head# 2024-09-20T22:20:54.762 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.769 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b99e642d:::benchmark_data_smithi060_196073_object39752:head# 2024-09-20T22:20:54.769 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.770 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b99e9569:::benchmark_data_smithi060_196073_object4730:head# 2024-09-20T22:20:54.770 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.770 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b99e9bc6:::benchmark_data_smithi060_196073_object8021:head# 2024-09-20T22:20:54.774 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.775 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b99ef744:::benchmark_data_smithi060_196073_object18595:head# 2024-09-20T22:20:54.775 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.776 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b99f9c4a:::benchmark_data_smithi060_196073_object22404:head# 2024-09-20T22:20:54.776 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.776 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b99fec63:::benchmark_data_smithi060_196073_object53869:head# 2024-09-20T22:20:54.776 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.776 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9a070c1:::benchmark_data_smithi060_196073_object26356:head# 2024-09-20T22:20:54.777 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.777 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9a08a4e:::benchmark_data_smithi060_196073_object19:head# 2024-09-20T22:20:54.777 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.777 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9a32e26:::benchmark_data_smithi060_196073_object7095:head# 2024-09-20T22:20:54.777 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.778 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9a37ab7:::benchmark_data_smithi060_196073_object46789:head# 2024-09-20T22:20:54.778 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.778 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9a468de:::benchmark_data_smithi060_196073_object59840:head# 2024-09-20T22:20:54.778 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.779 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9a4b0fc:::benchmark_data_smithi060_196073_object45869:head# 2024-09-20T22:20:54.779 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.779 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9a75ed5:::benchmark_data_smithi060_196073_object15833:head# 2024-09-20T22:20:54.779 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.779 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9a83d57:::benchmark_data_smithi060_196073_object29429:head# 2024-09-20T22:20:54.780 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.780 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9a863a9:::benchmark_data_smithi060_196073_object37769:head# 2024-09-20T22:20:54.780 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.780 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9a8f8a1:::benchmark_data_smithi060_196073_object6847:head# 2024-09-20T22:20:54.780 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.781 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9ab414a:::benchmark_data_smithi060_196073_object10757:head# 2024-09-20T22:20:54.781 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.781 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9abe9a9:::benchmark_data_smithi060_196073_object34900:head# 2024-09-20T22:20:54.781 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.782 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9ac5581:::benchmark_data_smithi060_196073_object59741:head# 2024-09-20T22:20:54.782 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.782 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9ad187d:::benchmark_data_smithi060_196073_object9747:head# 2024-09-20T22:20:54.782 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.783 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9aeb08e:::benchmark_data_smithi060_196073_object55501:head# 2024-09-20T22:20:54.783 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.783 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9aef59a:::benchmark_data_smithi060_196073_object6083:head# 2024-09-20T22:20:54.783 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.783 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9b04f86:::benchmark_data_smithi060_196073_object39541:head# 2024-09-20T22:20:54.783 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.784 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9b0d357:::benchmark_data_smithi060_196073_object37428:head# 2024-09-20T22:20:54.784 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.785 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9b16060:::benchmark_data_smithi060_196073_object31349:head# 2024-09-20T22:20:54.785 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.785 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9b3ed24:::benchmark_data_smithi060_196073_object49677:head# 2024-09-20T22:20:54.785 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.786 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9b4bc71:::benchmark_data_smithi060_196073_object17273:head# 2024-09-20T22:20:54.786 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.787 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9b64346:::benchmark_data_smithi060_196073_object42061:head# 2024-09-20T22:20:54.787 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.787 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9b7769f:::benchmark_data_smithi060_196073_object794:head# 2024-09-20T22:20:54.787 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.789 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9b7bd50:::benchmark_data_smithi060_196073_object32418:head# 2024-09-20T22:20:54.789 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:54.791 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:54.791 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:54.795 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9b92a52:::benchmark_data_smithi060_196073_object55663:head# 2024-09-20T22:20:54.796 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.796 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9bc7ea4:::benchmark_data_smithi060_196073_object35174:head# 2024-09-20T22:20:54.796 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.799 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9bc890a:::benchmark_data_smithi060_196073_object26959:head# 2024-09-20T22:20:54.799 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.800 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9bde130:::benchmark_data_smithi060_196073_object4134:head# 2024-09-20T22:20:54.800 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.800 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9c0d4bb:::benchmark_data_smithi060_196073_object30432:head# 2024-09-20T22:20:54.805 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.807 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9c39ee0:::benchmark_data_smithi060_196073_object6715:head# 2024-09-20T22:20:54.807 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.808 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9c3e469:::benchmark_data_smithi060_196073_object30298:head# 2024-09-20T22:20:54.808 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.809 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9c5cf2d:::benchmark_data_smithi060_196073_object57411:head# 2024-09-20T22:20:54.809 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.809 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9c829f5:::benchmark_data_smithi060_196073_object30245:head# 2024-09-20T22:20:54.809 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.810 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9c83f42:::benchmark_data_smithi060_196073_object38775:head# 2024-09-20T22:20:54.821 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.821 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9c88a83:::benchmark_data_smithi060_196073_object20670:head# 2024-09-20T22:20:54.821 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.821 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9c9e7d6:::benchmark_data_smithi060_196073_object38402:head# 2024-09-20T22:20:54.854 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.854 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9ca43a4:::benchmark_data_smithi060_196073_object42641:head# 2024-09-20T22:20:54.854 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.855 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9ca8a06:::benchmark_data_smithi060_196073_object6265:head# 2024-09-20T22:20:54.855 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.855 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9cbb8a6:::benchmark_data_smithi060_196073_object32260:head# 2024-09-20T22:20:54.855 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.855 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9cc9686:::benchmark_data_smithi060_196073_object24888:head# 2024-09-20T22:20:54.882 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.882 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9cde085:::benchmark_data_smithi060_196073_object46055:head# 2024-09-20T22:20:54.882 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.883 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9cebf68:::benchmark_data_smithi060_196073_object42417:head# 2024-09-20T22:20:54.883 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.883 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9cf3f73:::benchmark_data_smithi060_196073_object56908:head# 2024-09-20T22:20:54.883 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.883 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9cfa750:::benchmark_data_smithi060_196073_object42664:head# 2024-09-20T22:20:54.892 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:54.894 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.895 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9d433c8:::benchmark_data_smithi060_196073_object14892:head# 2024-09-20T22:20:54.895 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.895 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9d5bd8b:::benchmark_data_smithi060_196073_object14421:head# 2024-09-20T22:20:54.895 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.895 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9d6dbd3:::benchmark_data_smithi060_196073_object19339:head# 2024-09-20T22:20:54.895 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.895 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9d79b32:::benchmark_data_smithi060_196073_object12576:head# 2024-09-20T22:20:54.909 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.910 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9da01f9:::benchmark_data_smithi060_196073_object18583:head# 2024-09-20T22:20:54.910 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.910 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9db9252:::benchmark_data_smithi060_196073_object12392:head# 2024-09-20T22:20:54.910 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.911 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9dc8392:::benchmark_data_smithi060_196073_object33542:head# 2024-09-20T22:20:54.911 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.911 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9dcb291:::benchmark_data_smithi060_196073_object1041:head# 2024-09-20T22:20:54.918 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.918 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9df0608:::benchmark_data_smithi060_196073_object26202:head# 2024-09-20T22:20:54.918 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.919 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9dff7dc:::benchmark_data_smithi060_196073_object25182:head# 2024-09-20T22:20:54.919 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.919 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9e054c8:::benchmark_data_smithi060_196073_object8318:head# 2024-09-20T22:20:54.919 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.919 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9e06774:::benchmark_data_smithi060_196073_object10942:head# 2024-09-20T22:20:54.926 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.927 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9e1a1a9:::benchmark_data_smithi060_196073_object17195:head# 2024-09-20T22:20:54.927 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.927 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9e1f026:::benchmark_data_smithi060_196073_object38666:head# 2024-09-20T22:20:54.928 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:54.929 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:54.929 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:54.937 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.937 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9e2235b:::benchmark_data_smithi060_196073_object10952:head# 2024-09-20T22:20:54.938 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.938 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9e2b85b:::benchmark_data_smithi060_196073_object1890:head# 2024-09-20T22:20:54.960 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.960 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9e39b81:::benchmark_data_smithi060_196073_object33109:head# 2024-09-20T22:20:54.960 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.961 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9e3e2a5:::benchmark_data_smithi060_196073_object21124:head# 2024-09-20T22:20:54.961 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.961 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9e44fae:::benchmark_data_smithi060_196073_object35700:head# 2024-09-20T22:20:54.961 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.961 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9e58954:::benchmark_data_smithi060_196073_object41655:head# 2024-09-20T22:20:54.972 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.973 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9e75555:::benchmark_data_smithi060_196073_object56500:head# 2024-09-20T22:20:54.973 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.973 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9e82ef8:::benchmark_data_smithi060_196073_object7811:head# 2024-09-20T22:20:54.973 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.973 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9e8488b:::benchmark_data_smithi060_196073_object53473:head# 2024-09-20T22:20:54.974 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.974 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9e96723:::benchmark_data_smithi060_196073_object1987:head# 2024-09-20T22:20:54.974 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.974 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9e9cf2a:::benchmark_data_smithi060_196073_object48749:head# 2024-09-20T22:20:54.974 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:54.975 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9e9d366:::benchmark_data_smithi060_196073_object30700:head# 2024-09-20T22:20:55.030 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:55.034 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.034 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9ea5fe2:::benchmark_data_smithi060_196073_object31710:head# 2024-09-20T22:20:55.034 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.034 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9ea85c5:::benchmark_data_smithi060_196073_object59727:head# 2024-09-20T22:20:55.034 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.035 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9eba851:::benchmark_data_smithi060_196073_object14851:head# 2024-09-20T22:20:55.035 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.035 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9ed6556:::benchmark_data_smithi060_196073_object49490:head# 2024-09-20T22:20:55.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:54 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:20:55.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:54 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:20:55.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:54 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3186488298' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:20:55.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:54 smithi060 ceph-mon[29694]: osdmap e506: 8 total, 7 up, 6 in 2024-09-20T22:20:55.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:54 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:20:55.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:54 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:20:55.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:54 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3186488298' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:20:55.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:54 smithi060 ceph-mon[40290]: osdmap e506: 8 total, 7 up, 6 in 2024-09-20T22:20:55.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:55.068 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:55.068 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:55.101 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.102 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9ed7389:::benchmark_data_smithi060_196073_object14396:head# 2024-09-20T22:20:55.102 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.102 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9edf506:::benchmark_data_smithi060_196073_object37781:head# 2024-09-20T22:20:55.102 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.103 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9efdbc6:::benchmark_data_smithi060_196073_object42027:head# 2024-09-20T22:20:55.103 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.103 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9effd33:::benchmark_data_smithi060_196073_object48150:head# 2024-09-20T22:20:55.103 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.103 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9f0719b:::benchmark_data_smithi060_196073_object33346:head# 2024-09-20T22:20:55.103 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.104 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9f0a0fc:::benchmark_data_smithi060_196073_object21119:head# 2024-09-20T22:20:55.145 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.145 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9f3d752:::benchmark_data_smithi060_196073_object51418:head# 2024-09-20T22:20:55.145 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.146 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9f532e4:::benchmark_data_smithi060_196073_object21687:head# 2024-09-20T22:20:55.146 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.146 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9f53d44:::benchmark_data_smithi060_196073_object61636:head# 2024-09-20T22:20:55.147 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.147 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9f6c193:::benchmark_data_smithi060_196073_object19240:head# 2024-09-20T22:20:55.147 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.147 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9f7a6fd:::benchmark_data_smithi060_196073_object26542:head# 2024-09-20T22:20:55.147 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.147 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9f903d6:::benchmark_data_smithi060_196073_object52374:head# 2024-09-20T22:20:55.147 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.148 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9f9160d:::benchmark_data_smithi060_196073_object2551:head# 2024-09-20T22:20:55.148 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.148 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9f9f017:::benchmark_data_smithi060_196073_object1642:head# 2024-09-20T22:20:55.169 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:55.214 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:55.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:55.215 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:55.228 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.228 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9f9f05b:::benchmark_data_smithi060_196073_object47805:head# 2024-09-20T22:20:55.228 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.228 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9fa44f4:::benchmark_data_smithi060_196073_object2523:head# 2024-09-20T22:20:55.228 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.229 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9fae11f:::benchmark_data_smithi060_196073_object39953:head# 2024-09-20T22:20:55.229 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.229 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9fae60a:::benchmark_data_smithi060_196073_object53072:head# 2024-09-20T22:20:55.247 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.247 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9fb0f71:::benchmark_data_smithi060_196073_object24628:head# 2024-09-20T22:20:55.247 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.247 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9fb4c11:::benchmark_data_smithi060_196073_object58198:head# 2024-09-20T22:20:55.247 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.248 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9fc088c:::benchmark_data_smithi060_196073_object12931:head# 2024-09-20T22:20:55.248 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.248 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9fc478d:::benchmark_data_smithi060_196073_object41264:head# 2024-09-20T22:20:55.258 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.258 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9fca3a5:::benchmark_data_smithi060_196073_object3534:head# 2024-09-20T22:20:55.259 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.259 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9fd71aa:::benchmark_data_smithi060_196073_object24040:head# 2024-09-20T22:20:55.261 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.261 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9fddc0f:::benchmark_data_smithi060_196073_object42011:head# 2024-09-20T22:20:55.261 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.262 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9fddfd5:::benchmark_data_smithi060_196073_object21304:head# 2024-09-20T22:20:55.262 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.262 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9ff1b7d:::benchmark_data_smithi060_196073_object57494:head# 2024-09-20T22:20:55.262 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.262 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:b9ffa50f:::benchmark_data_smithi060_196073_object30251:head# 2024-09-20T22:20:55.300 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.300 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba03928a:::benchmark_data_smithi060_196073_object869:head# 2024-09-20T22:20:55.300 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.301 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba0449de:::benchmark_data_smithi060_196073_object28017:head# 2024-09-20T22:20:55.301 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.301 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba04a5dc:::benchmark_data_smithi060_196073_object50394:head# 2024-09-20T22:20:55.316 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:55.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:55.353 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:55.353 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:55.389 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.389 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba04ac3c:::benchmark_data_smithi060_196073_object3200:head# 2024-09-20T22:20:55.389 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.390 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba064320:::benchmark_data_smithi060_196073_object61189:head# 2024-09-20T22:20:55.390 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.390 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba077cb6:::benchmark_data_smithi060_196073_object35317:head# 2024-09-20T22:20:55.390 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.390 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba0be4ef:::benchmark_data_smithi060_196073_object13093:head# 2024-09-20T22:20:55.454 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:55.456 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.457 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba0c2ae8:::benchmark_data_smithi060_196073_object60002:head# 2024-09-20T22:20:55.457 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.458 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba0d244e:::benchmark_data_smithi060_196073_object34702:head# 2024-09-20T22:20:55.458 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.458 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba0ed990:::benchmark_data_smithi060_196073_object12679:head# 2024-09-20T22:20:55.458 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.459 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba0ee1a5:::benchmark_data_smithi060_196073_object27553:head# 2024-09-20T22:20:55.489 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:55.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:55.490 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:55.495 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.495 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba13bbc7:::benchmark_data_smithi060_196073_object51085:head# 2024-09-20T22:20:55.495 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.495 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba13f695:::benchmark_data_smithi060_196073_object36114:head# 2024-09-20T22:20:55.495 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.496 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba14ab5a:::benchmark_data_smithi060_196073_object28483:head# 2024-09-20T22:20:55.496 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.496 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba1707fa:::benchmark_data_smithi060_196073_object41098:head# 2024-09-20T22:20:55.496 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.497 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba18fbef:::benchmark_data_smithi060_196073_object7716:head# 2024-09-20T22:20:55.497 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.497 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba19018b:::benchmark_data_smithi060_196073_object12747:head# 2024-09-20T22:20:55.504 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:20:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:20:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:20:55.520 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.521 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba19db4f:::benchmark_data_smithi060_196073_object54101:head# 2024-09-20T22:20:55.521 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.521 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba1ad63c:::benchmark_data_smithi060_196073_object59105:head# 2024-09-20T22:20:55.521 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.522 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba1b6b72:::benchmark_data_smithi060_196073_object33787:head# 2024-09-20T22:20:55.522 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.522 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba1bcb81:::benchmark_data_smithi060_196073_object14523:head# 2024-09-20T22:20:55.572 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.572 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba1f5c2e:::benchmark_data_smithi060_196073_object48688:head# 2024-09-20T22:20:55.573 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.573 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba1fcf1a:::benchmark_data_smithi060_196073_object29803:head# 2024-09-20T22:20:55.573 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.573 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba2054c0:::benchmark_data_smithi060_196073_object52584:head# 2024-09-20T22:20:55.573 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.574 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba219c6a:::benchmark_data_smithi060_196073_object8031:head# 2024-09-20T22:20:55.591 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:55.622 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:55.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:55.623 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:55.633 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.633 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba21a4c6:::benchmark_data_smithi060_196073_object53437:head# 2024-09-20T22:20:55.633 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.634 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba2323dc:::benchmark_data_smithi060_196073_object16564:head# 2024-09-20T22:20:55.634 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.634 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba23e3c6:::benchmark_data_smithi060_196073_object46558:head# 2024-09-20T22:20:55.634 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.634 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba29cb3e:::benchmark_data_smithi060_196073_object7945:head# 2024-09-20T22:20:55.661 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:20:55.661 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:20:55.663 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.663 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba29e74a:::benchmark_data_smithi060_196073_object2111:head# 2024-09-20T22:20:55.664 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.664 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba2be85b:::benchmark_data_smithi060_196073_object51223:head# 2024-09-20T22:20:55.664 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.664 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba2c259a:::benchmark_data_smithi060_196073_object56084:head# 2024-09-20T22:20:55.664 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.665 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba2c9024:::benchmark_data_smithi060_196073_object23792:head# 2024-09-20T22:20:55.701 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.702 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba2cf814:::benchmark_data_smithi060_196073_object39749:head# 2024-09-20T22:20:55.702 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.702 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba2ddad3:::benchmark_data_smithi060_196073_object44146:head# 2024-09-20T22:20:55.702 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.702 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba2e210e:::benchmark_data_smithi060_196073_object22145:head# 2024-09-20T22:20:55.702 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.703 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba2fc604:::benchmark_data_smithi060_196073_object58282:head# 2024-09-20T22:20:55.720 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.720 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba300e42:::benchmark_data_smithi060_196073_object29818:head# 2024-09-20T22:20:55.720 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.720 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba3074ef:::benchmark_data_smithi060_196073_object35140:head# 2024-09-20T22:20:55.720 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.720 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba308aaf:::benchmark_data_smithi060_196073_object7918:head# 2024-09-20T22:20:55.720 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.721 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba30c1cf:::benchmark_data_smithi060_196073_object52548:head# 2024-09-20T22:20:55.724 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:55.727 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.727 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba30caa8:::benchmark_data_smithi060_196073_object6392:head# 2024-09-20T22:20:55.727 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.727 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba310c95:::benchmark_data_smithi060_196073_object34069:head# 2024-09-20T22:20:55.727 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.728 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba313f01:::benchmark_data_smithi060_196073_object18545:head# 2024-09-20T22:20:55.728 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.728 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba327a61:::benchmark_data_smithi060_196073_object25820:head# 2024-09-20T22:20:55.734 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.735 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba3330ac:::benchmark_data_smithi060_196073_object30623:head# 2024-09-20T22:20:55.735 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.736 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba360998:::benchmark_data_smithi060_196073_object35327:head# 2024-09-20T22:20:55.737 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.737 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba3b67b3:::benchmark_data_smithi060_196073_object10239:head# 2024-09-20T22:20:55.737 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.737 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba3d69ce:::benchmark_data_smithi060_196073_object20267:head# 2024-09-20T22:20:55.755 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:55.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:55.756 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:55 smithi067 ceph-mon[33511]: pgmap v963: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.6 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s; 3185/226665 objects degraded (1.405%); 11420/226665 objects misplaced (5.038%); 23 MiB/s, 372 objects/s recovering 2024-09-20T22:20:55.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:55 smithi060 ceph-mon[29694]: pgmap v963: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.6 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s; 3185/226665 objects degraded (1.405%); 11420/226665 objects misplaced (5.038%); 23 MiB/s, 372 objects/s recovering 2024-09-20T22:20:55.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:55 smithi060 ceph-mon[40290]: pgmap v963: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.6 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.34k op/s; 3185/226665 objects degraded (1.405%); 11420/226665 objects misplaced (5.038%); 23 MiB/s, 372 objects/s recovering 2024-09-20T22:20:55.857 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:55.874 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.874 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba3e9d60:::benchmark_data_smithi060_196073_object53728:head# 2024-09-20T22:20:55.874 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.875 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba40193a:::benchmark_data_smithi060_196073_object7995:head# 2024-09-20T22:20:55.875 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.875 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba406f8b:::benchmark_data_smithi060_196073_object2876:head# 2024-09-20T22:20:55.875 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.875 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba40a12b:::benchmark_data_smithi060_196073_object33585:head# 2024-09-20T22:20:55.896 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:55.897 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:55.897 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:55.906 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.906 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba416d6f:::benchmark_data_smithi060_196073_object45430:head# 2024-09-20T22:20:55.906 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.907 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba41901a:::benchmark_data_smithi060_196073_object48366:head# 2024-09-20T22:20:55.907 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.907 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba419d95:::benchmark_data_smithi060_196073_object15772:head# 2024-09-20T22:20:55.907 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.907 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba41bee4:::benchmark_data_smithi060_196073_object6633:head# 2024-09-20T22:20:55.911 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.911 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba421772:::benchmark_data_smithi060_196073_object50960:head# 2024-09-20T22:20:55.911 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.914 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba444959:::benchmark_data_smithi060_196073_object39419:head# 2024-09-20T22:20:55.914 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.914 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba453ba5:::benchmark_data_smithi060_196073_object60019:head# 2024-09-20T22:20:55.914 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.914 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba472b1d:::benchmark_data_smithi060_196073_object557:head# 2024-09-20T22:20:55.914 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.914 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba473413:::benchmark_data_smithi060_196073_object7137:head# 2024-09-20T22:20:55.915 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.915 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba4bad9d:::benchmark_data_smithi060_196073_object2217:head# 2024-09-20T22:20:55.962 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.962 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba4c5b30:::benchmark_data_smithi060_196073_object51586:head# 2024-09-20T22:20:55.962 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.962 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba4da803:::benchmark_data_smithi060_196073_object44196:head# 2024-09-20T22:20:55.962 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.963 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba4e98b8:::benchmark_data_smithi060_196073_object17251:head# 2024-09-20T22:20:55.963 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:55.963 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba4ee986:::benchmark_data_smithi060_196073_object55559:head# 2024-09-20T22:20:55.998 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:56.020 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:56.040 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.041 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba523e78:::benchmark_data_smithi060_196073_object16173:head# 2024-09-20T22:20:56.041 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.041 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba5268b2:::benchmark_data_smithi060_196073_object50391:head# 2024-09-20T22:20:56.041 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.041 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba52fa5e:::benchmark_data_smithi060_196073_object18909:head# 2024-09-20T22:20:56.041 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.042 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba5357fd:::benchmark_data_smithi060_196073_object5080:head# 2024-09-20T22:20:56.049 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:56.050 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:56.050 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:56.106 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.107 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba569746:::benchmark_data_smithi060_196073_object43091:head# 2024-09-20T22:20:56.107 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.107 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba56f53e:::benchmark_data_smithi060_196073_object28723:head# 2024-09-20T22:20:56.107 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.107 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba577d55:::benchmark_data_smithi060_196073_object8465:head# 2024-09-20T22:20:56.107 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.107 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba592a95:::benchmark_data_smithi060_196073_object29055:head# 2024-09-20T22:20:56.108 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.108 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba5ab51b:::benchmark_data_smithi060_196073_object58875:head# 2024-09-20T22:20:56.128 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.129 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba5b18fb:::benchmark_data_smithi060_196073_object34100:head# 2024-09-20T22:20:56.129 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.129 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba5cfbe9:::benchmark_data_smithi060_196073_object50639:head# 2024-09-20T22:20:56.129 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.129 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba5d02de:::benchmark_data_smithi060_196073_object4455:head# 2024-09-20T22:20:56.130 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.130 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba5e9bfc:::benchmark_data_smithi060_196073_object52970:head# 2024-09-20T22:20:56.133 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.133 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba5ef08b:::benchmark_data_smithi060_196073_object17715:head# 2024-09-20T22:20:56.133 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.133 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba60595f:::benchmark_data_smithi060_196073_object9059:head# 2024-09-20T22:20:56.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba61dd67:::benchmark_data_smithi060_196073_object44670:head# 2024-09-20T22:20:56.144 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.144 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba626234:::benchmark_data_smithi060_196073_object56544:head# 2024-09-20T22:20:56.144 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.144 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba62c537:::benchmark_data_smithi060_196073_object29397:head# 2024-09-20T22:20:56.144 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.145 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba631589:::benchmark_data_smithi060_196073_object43710:head# 2024-09-20T22:20:56.151 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:56.188 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:56.189 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:56.189 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:56.229 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.229 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba658cc9:::benchmark_data_smithi060_196073_object16574:head# 2024-09-20T22:20:56.229 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.229 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba68088e:::benchmark_data_smithi060_196073_object54265:head# 2024-09-20T22:20:56.229 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.230 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba697d5e:::benchmark_data_smithi060_196073_object17216:head# 2024-09-20T22:20:56.230 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.230 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba6bbb6c:::benchmark_data_smithi060_196073_object16932:head# 2024-09-20T22:20:56.272 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.272 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba6bc824:::benchmark_data_smithi060_196073_object7612:head# 2024-09-20T22:20:56.272 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.273 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba6d11f5:::benchmark_data_smithi060_196073_object17379:head# 2024-09-20T22:20:56.273 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.273 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba6fdfa2:::benchmark_data_smithi060_196073_object33160:head# 2024-09-20T22:20:56.273 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.273 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba70dbbd:::benchmark_data_smithi060_196073_object55009:head# 2024-09-20T22:20:56.273 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.273 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba7156d1:::benchmark_data_smithi060_196073_object17704:head# 2024-09-20T22:20:56.274 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.274 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba71a71c:::benchmark_data_smithi060_196073_object54115:head# 2024-09-20T22:20:56.280 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.281 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba722ca6:::benchmark_data_smithi060_196073_object49529:head# 2024-09-20T22:20:56.281 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.281 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba741589:::benchmark_data_smithi060_196073_object14666:head# 2024-09-20T22:20:56.281 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.281 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba744924:::benchmark_data_smithi060_196073_object16584:head# 2024-09-20T22:20:56.281 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.282 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba748733:::benchmark_data_smithi060_196073_object41422:head# 2024-09-20T22:20:56.287 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.288 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba75d323:::benchmark_data_smithi060_196073_object38508:head# 2024-09-20T22:20:56.288 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.288 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba7745df:::benchmark_data_smithi060_196073_object38157:head# 2024-09-20T22:20:56.288 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.289 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba779426:::benchmark_data_smithi060_196073_object40553:head# 2024-09-20T22:20:56.289 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.289 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba77a5be:::benchmark_data_smithi060_196073_object12786:head# 2024-09-20T22:20:56.290 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:56.293 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.294 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba793676:::benchmark_data_smithi060_196073_object31282:head# 2024-09-20T22:20:56.294 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.294 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba7b99ce:::benchmark_data_smithi060_196073_object34902:head# 2024-09-20T22:20:56.294 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.295 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba7bbd2c:::benchmark_data_smithi060_196073_object29157:head# 2024-09-20T22:20:56.295 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.295 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba7c5da0:::benchmark_data_smithi060_196073_object61588:head# 2024-09-20T22:20:56.327 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:56.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:56.328 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:56.412 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.413 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba7da75b:::benchmark_data_smithi060_196073_object26064:head# 2024-09-20T22:20:56.413 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.413 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba7de81e:::benchmark_data_smithi060_196073_object38265:head# 2024-09-20T22:20:56.413 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.413 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba7e0287:::benchmark_data_smithi060_196073_object36262:head# 2024-09-20T22:20:56.413 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.414 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba7e1cdf:::benchmark_data_smithi060_196073_object44136:head# 2024-09-20T22:20:56.429 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:56.456 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.457 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba7fc97e:::benchmark_data_smithi060_196073_object36269:head# 2024-09-20T22:20:56.457 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.457 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba80163c:::benchmark_data_smithi060_196073_object57356:head# 2024-09-20T22:20:56.457 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.458 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba83747a:::benchmark_data_smithi060_196073_object24302:head# 2024-09-20T22:20:56.458 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.458 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba83bd5b:::benchmark_data_smithi060_196073_object5769:head# 2024-09-20T22:20:56.464 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:56.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:56.465 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:56.488 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.488 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba843aad:::benchmark_data_smithi060_196073_object37437:head# 2024-09-20T22:20:56.488 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.488 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba863dca:::benchmark_data_smithi060_196073_object58078:head# 2024-09-20T22:20:56.488 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.489 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba866b0b:::benchmark_data_smithi060_196073_object39073:head# 2024-09-20T22:20:56.489 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.489 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba868645:::benchmark_data_smithi060_196073_object15117:head# 2024-09-20T22:20:56.499 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.499 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba86c4ce:::benchmark_data_smithi060_196073_object54881:head# 2024-09-20T22:20:56.499 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.499 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba883fac:::benchmark_data_smithi060_196073_object2751:head# 2024-09-20T22:20:56.515 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.515 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba884c53:::benchmark_data_smithi060_196073_object3328:head# 2024-09-20T22:20:56.515 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.516 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba888cf8:::benchmark_data_smithi060_196073_object28489:head# 2024-09-20T22:20:56.516 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.516 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba89bfc6:::benchmark_data_smithi060_196073_object23445:head# 2024-09-20T22:20:56.516 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.516 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba8a091a:::benchmark_data_smithi060_196073_object58718:head# 2024-09-20T22:20:56.566 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:56.592 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.592 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba8a0f51:::benchmark_data_smithi060_196073_object49149:head# 2024-09-20T22:20:56.592 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.593 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba8a7d30:::benchmark_data_smithi060_196073_object61039:head# 2024-09-20T22:20:56.593 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.593 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba8aa5e1:::benchmark_data_smithi060_196073_object33966:head# 2024-09-20T22:20:56.593 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.593 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba8ab291:::benchmark_data_smithi060_196073_object14221:head# 2024-09-20T22:20:56.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:56.612 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:56.612 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:56.631 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.631 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba8ae651:::benchmark_data_smithi060_196073_object23438:head# 2024-09-20T22:20:56.631 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.631 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba8aefee:::benchmark_data_smithi060_196073_object39407:head# 2024-09-20T22:20:56.634 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.634 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba8c84b9:::benchmark_data_smithi060_196073_object53312:head# 2024-09-20T22:20:56.635 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.635 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba8ccd9a:::benchmark_data_smithi060_196073_object44813:head# 2024-09-20T22:20:56.644 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.644 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba8d836a:::benchmark_data_smithi060_196073_object47341:head# 2024-09-20T22:20:56.644 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.644 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba8e2021:::benchmark_data_smithi060_196073_object40287:head# 2024-09-20T22:20:56.646 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.646 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba8f819c:::benchmark_data_smithi060_196073_object17312:head# 2024-09-20T22:20:56.646 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.647 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba8fa256:::benchmark_data_smithi060_196073_object52388:head# 2024-09-20T22:20:56.647 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.647 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba9181ba:::benchmark_data_smithi060_196073_object55366:head# 2024-09-20T22:20:56.647 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.647 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba94a586:::benchmark_data_smithi060_196073_object21785:head# 2024-09-20T22:20:56.652 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.653 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba967006:::benchmark_data_smithi060_196073_object19175:head# 2024-09-20T22:20:56.653 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.653 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba97fbfc:::benchmark_data_smithi060_196073_object35399:head# 2024-09-20T22:20:56.654 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.654 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba984eaf:::benchmark_data_smithi060_196073_object5152:head# 2024-09-20T22:20:56.660 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.660 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba985a3a:::benchmark_data_smithi060_196073_object35870:head# 2024-09-20T22:20:56.660 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.661 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba99878d:::benchmark_data_smithi060_196073_object26278:head# 2024-09-20T22:20:56.661 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.661 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba9b15fb:::benchmark_data_smithi060_196073_object52948:head# 2024-09-20T22:20:56.661 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.661 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba9b6405:::benchmark_data_smithi060_196073_object25884:head# 2024-09-20T22:20:56.667 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.668 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:56 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:20:56.668 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:56 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:20:56.668 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba9b7b0c:::benchmark_data_smithi060_196073_object55764:head# 2024-09-20T22:20:56.669 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.669 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba9c0d04:::benchmark_data_smithi060_196073_object11301:head# 2024-09-20T22:20:56.669 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.669 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba9c26a9:::benchmark_data_smithi060_196073_object18423:head# 2024-09-20T22:20:56.669 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.669 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba9c3c11:::benchmark_data_smithi060_196073_object31612:head# 2024-09-20T22:20:56.675 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.675 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:ba9f513e:::benchmark_data_smithi060_196073_object19696:head# 2024-09-20T22:20:56.675 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.676 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baa0e0a6:::benchmark_data_smithi060_196073_object32351:head# 2024-09-20T22:20:56.676 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.676 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baa16a0d:::benchmark_data_smithi060_196073_object49569:head# 2024-09-20T22:20:56.676 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.676 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:20:56.677 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baa1dc27:::benchmark_data_smithi060_196073_object56049:head# 2024-09-20T22:20:56.677 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:20:56.683 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.683 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baa24fe2:::benchmark_data_smithi060_196073_object14648:head# 2024-09-20T22:20:56.683 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.683 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baa254ce:::benchmark_data_smithi060_196073_object15541:head# 2024-09-20T22:20:56.684 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.684 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baa268e0:::benchmark_data_smithi060_196073_object49975:head# 2024-09-20T22:20:56.684 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.684 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baa28ce9:::benchmark_data_smithi060_196073_object34325:head# 2024-09-20T22:20:56.691 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.691 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baa29e01:::benchmark_data_smithi060_196073_object58247:head# 2024-09-20T22:20:56.691 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.692 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baa67a56:::benchmark_data_smithi060_196073_object43520:head# 2024-09-20T22:20:56.692 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.692 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baa67f64:::benchmark_data_smithi060_196073_object872:head# 2024-09-20T22:20:56.692 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.692 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baa7c0ac:::benchmark_data_smithi060_196073_object44206:head# 2024-09-20T22:20:56.693 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:20:56.693 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:20:56.700 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.700 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baaa3fef:::benchmark_data_smithi060_196073_object56523:head# 2024-09-20T22:20:56.700 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.701 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baab2368:::benchmark_data_smithi060_196073_object1299:head# 2024-09-20T22:20:56.701 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.701 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baab4daa:::benchmark_data_smithi060_196073_object24108:head# 2024-09-20T22:20:56.701 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.701 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baac32e0:::benchmark_data_smithi060_196073_object10711:head# 2024-09-20T22:20:56.705 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.707 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baac7934:::benchmark_data_smithi060_196073_object17691:head# 2024-09-20T22:20:56.707 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.707 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baacd67b:::benchmark_data_smithi060_196073_object12897:head# 2024-09-20T22:20:56.707 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.707 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baad40d4:::benchmark_data_smithi060_196073_object23751:head# 2024-09-20T22:20:56.707 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:56.707 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baae478a:::benchmark_data_smithi060_196073_object47204:head# 2024-09-20T22:20:56.714 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:56.717 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:20:56.717 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:20:56.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:56.746 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:56.746 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:56.750 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:20:56.750 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:20:56.750 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-20T22:20:56.785 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:20:56.785 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:20:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:56 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:20:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:56 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:20:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:56 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:20:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:56 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:20:56.819 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:20:56.819 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:20:56.847 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:20:56.847 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:20:56.847 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:56.879 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:56.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:56.881 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:56.981 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:57.005 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:57.005 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:57.006 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:57.037 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.038 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baaf7379:::benchmark_data_smithi060_196073_object31749:head# 2024-09-20T22:20:57.038 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.038 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baaf8ea6:::benchmark_data_smithi060_196073_object30815:head# 2024-09-20T22:20:57.038 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.038 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bab0424e:::benchmark_data_smithi060_196073_object44545:head# 2024-09-20T22:20:57.038 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.039 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bab0874b:::benchmark_data_smithi060_196073_object47703:head# 2024-09-20T22:20:57.063 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.063 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bab179d7:::benchmark_data_smithi060_196073_object57988:head# 2024-09-20T22:20:57.063 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.064 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bab20397:::benchmark_data_smithi060_196073_object60684:head# 2024-09-20T22:20:57.064 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.064 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bab4ccd4:::benchmark_data_smithi060_196073_object31746:head# 2024-09-20T22:20:57.064 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.064 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bab4eee8:::benchmark_data_smithi060_196073_object1505:head# 2024-09-20T22:20:57.069 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.070 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bab4fc01:::benchmark_data_smithi060_196073_object4428:head# 2024-09-20T22:20:57.070 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.070 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bab75272:::benchmark_data_smithi060_196073_object16660:head# 2024-09-20T22:20:57.070 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.070 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bab7e36e:::benchmark_data_smithi060_196073_object22740:head# 2024-09-20T22:20:57.071 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.071 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bab84810:::benchmark_data_smithi060_196073_object11052:head# 2024-09-20T22:20:57.074 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.074 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baba74bc:::benchmark_data_smithi060_196073_object10936:head# 2024-09-20T22:20:57.074 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.075 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:babc44eb:::benchmark_data_smithi060_196073_object11435:head# 2024-09-20T22:20:57.075 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.075 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:babd59e0:::benchmark_data_smithi060_196073_object47560:head# 2024-09-20T22:20:57.075 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.075 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:babda761:::benchmark_data_smithi060_196073_object3359:head# 2024-09-20T22:20:57.107 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:57.112 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.112 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:babea306:::benchmark_data_smithi060_196073_object47779:head# 2024-09-20T22:20:57.113 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.113 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:babf1091:::benchmark_data_smithi060_196073_object55465:head# 2024-09-20T22:20:57.113 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.113 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:babf9880:::benchmark_data_smithi060_196073_object36874:head# 2024-09-20T22:20:57.113 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.113 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:babfa2dd:::benchmark_data_smithi060_196073_object39588:head# 2024-09-20T22:20:57.121 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.121 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bac164f8:::benchmark_data_smithi060_196073_object39306:head# 2024-09-20T22:20:57.122 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.122 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bac16c70:::benchmark_data_smithi060_196073_object37043:head# 2024-09-20T22:20:57.123 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.123 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bac2b047:::benchmark_data_smithi060_196073_object11967:head# 2024-09-20T22:20:57.123 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.123 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bac383f6:::benchmark_data_smithi060_196073_object21841:head# 2024-09-20T22:20:57.129 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:57.129 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:57.130 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:57.131 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.131 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bac416b1:::benchmark_data_smithi060_196073_object26926:head# 2024-09-20T22:20:57.131 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.131 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bac605fa:::benchmark_data_smithi060_196073_object37367:head# 2024-09-20T22:20:57.131 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.131 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bac6be54:::benchmark_data_smithi060_196073_object12034:head# 2024-09-20T22:20:57.131 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.131 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bac7d41b:::benchmark_data_smithi060_196073_object12462:head# 2024-09-20T22:20:57.136 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.136 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bac8bea0:::benchmark_data_smithi060_196073_object33393:head# 2024-09-20T22:20:57.136 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.136 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bac9efe3:::benchmark_data_smithi060_196073_object3969:head# 2024-09-20T22:20:57.142 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.142 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baca7eae:::benchmark_data_smithi060_196073_object34994:head# 2024-09-20T22:20:57.142 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.142 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bacaa223:::benchmark_data_smithi060_196073_object15653:head# 2024-09-20T22:20:57.231 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:57.256 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:57.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:57.257 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:57.326 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.326 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bacd31d9:::benchmark_data_smithi060_196073_object40088:head# 2024-09-20T22:20:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bace0f84:::benchmark_data_smithi060_196073_object55833:head# 2024-09-20T22:20:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bace93fd:::benchmark_data_smithi060_196073_object44420:head# 2024-09-20T22:20:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.327 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bacebdba:::benchmark_data_smithi060_196073_object27542:head# 2024-09-20T22:20:57.358 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:57.384 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:57.384 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:57.385 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bad16378:::benchmark_data_smithi060_196073_object26544:head# 2024-09-20T22:20:57.392 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bad2463b:::benchmark_data_smithi060_196073_object29753:head# 2024-09-20T22:20:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bad272e7:::benchmark_data_smithi060_196073_object57515:head# 2024-09-20T22:20:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.393 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bad3738d:::benchmark_data_smithi060_196073_object18229:head# 2024-09-20T22:20:57.399 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.399 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bad3b104:::benchmark_data_smithi060_196073_object31499:head# 2024-09-20T22:20:57.399 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.403 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bad63051:::benchmark_data_smithi060_196073_object34739:head# 2024-09-20T22:20:57.403 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.403 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bad6bf0c:::benchmark_data_smithi060_196073_object28119:head# 2024-09-20T22:20:57.404 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.405 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bad88581:::benchmark_data_smithi060_196073_object34681:head# 2024-09-20T22:20:57.405 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.405 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bada9855:::benchmark_data_smithi060_196073_object46473:head# 2024-09-20T22:20:57.405 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:badeacd1:::benchmark_data_smithi060_196073_object40471:head# 2024-09-20T22:20:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:badf0b13:::benchmark_data_smithi060_196073_object7604:head# 2024-09-20T22:20:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:badf2ddb:::benchmark_data_smithi060_196073_object39640:head# 2024-09-20T22:20:57.406 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bae07119:::benchmark_data_smithi060_196073_object32796:head# 2024-09-20T22:20:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bae33c74:::benchmark_data_smithi060_196073_object23795:head# 2024-09-20T22:20:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.407 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bae40d2e:::benchmark_data_smithi060_196073_object15124:head# 2024-09-20T22:20:57.441 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.441 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bae477d6:::benchmark_data_smithi060_196073_object30534:head# 2024-09-20T22:20:57.441 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.441 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bae490a0:::benchmark_data_smithi060_196073_object8915:head# 2024-09-20T22:20:57.441 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.442 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bae6ddc9:::benchmark_data_smithi060_196073_object43998:head# 2024-09-20T22:20:57.442 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.442 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bae7cb02:::benchmark_data_smithi060_196073_object19785:head# 2024-09-20T22:20:57.476 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.476 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bae8b617:::benchmark_data_smithi060_196073_object22986:head# 2024-09-20T22:20:57.476 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.476 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bae9b9ed:::benchmark_data_smithi060_196073_object8620:head# 2024-09-20T22:20:57.476 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.477 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baea4e20:::benchmark_data_smithi060_196073_object52054:head# 2024-09-20T22:20:57.477 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.477 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baeb6f1d:::benchmark_data_smithi060_196073_object25769:head# 2024-09-20T22:20:57.485 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:57.488 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.488 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baece6fb:::benchmark_data_smithi060_196073_object30625:head# 2024-09-20T22:20:57.488 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.488 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baf340d4:::benchmark_data_smithi060_196073_object5919:head# 2024-09-20T22:20:57.488 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.489 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baf57b83:::benchmark_data_smithi060_196073_object6936:head# 2024-09-20T22:20:57.489 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.489 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baf72c64:::benchmark_data_smithi060_196073_object28956:head# 2024-09-20T22:20:57.510 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:57.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:57.511 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:57.610 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.610 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:57 smithi067 ceph-mon[33511]: pgmap v964: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.5 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.17k op/s; 2707/222264 objects degraded (1.218%); 10942/222264 objects misplaced (4.923%); 21 MiB/s, 335 objects/s recovering 2024-09-20T22:20:57.610 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:57 smithi067 ceph-mon[33511]: osdmap e507: 8 total, 7 up, 6 in 2024-09-20T22:20:57.610 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:57 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2995317394' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:20:57.610 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:57 smithi067 ceph-mon[33511]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:20:57.611 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:57 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 2707/222264 objects degraded (1.218%), 3 pgs degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T22:20:57.611 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baf76cde:::benchmark_data_smithi060_196073_object59878:head# 2024-09-20T22:20:57.611 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.611 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baf94f60:::benchmark_data_smithi060_196073_object37839:head# 2024-09-20T22:20:57.612 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.612 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:baf99a4d:::benchmark_data_smithi060_196073_object45724:head# 2024-09-20T22:20:57.612 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.612 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bafa2b11:::benchmark_data_smithi060_196073_object19064:head# 2024-09-20T22:20:57.612 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:57.641 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:57.642 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:57.642 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:57.671 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.672 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bafb36da:::benchmark_data_smithi060_196073_object31562:head# 2024-09-20T22:20:57.672 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.672 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bafc3618:::benchmark_data_smithi060_196073_object33105:head# 2024-09-20T22:20:57.672 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.672 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bafcd38b:::benchmark_data_smithi060_196073_object42523:head# 2024-09-20T22:20:57.672 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.672 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb0279dc:::benchmark_data_smithi060_196073_object34392:head# 2024-09-20T22:20:57.673 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:20:57.673 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:20:57.675 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.676 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb02e7a6:::benchmark_data_smithi060_196073_object50392:head# 2024-09-20T22:20:57.676 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.676 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb04be83:::benchmark_data_smithi060_196073_object1785:head# 2024-09-20T22:20:57.717 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.719 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb04cc95:::benchmark_data_smithi060_196073_object28670:head# 2024-09-20T22:20:57.719 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.720 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb09dd39:::benchmark_data_smithi060_196073_object8047:head# 2024-09-20T22:20:57.720 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.720 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb0a6db5:::benchmark_data_smithi060_196073_object17836:head# 2024-09-20T22:20:57.720 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.721 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb0a9911:::benchmark_data_smithi060_196073_object30803:head# 2024-09-20T22:20:57.723 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.724 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb0c2c88:::benchmark_data_smithi060_196073_object37957:head# 2024-09-20T22:20:57.724 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.724 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb0c62be:::benchmark_data_smithi060_196073_object31895:head# 2024-09-20T22:20:57.724 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.725 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb0cda61:::benchmark_data_smithi060_196073_object55380:head# 2024-09-20T22:20:57.725 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.725 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb0d9a04:::benchmark_data_smithi060_196073_object15953:head# 2024-09-20T22:20:57.729 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.730 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb0e25c3:::benchmark_data_smithi060_196073_object1252:head# 2024-09-20T22:20:57.730 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.732 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb0e8a81:::benchmark_data_smithi060_196073_object12509:head# 2024-09-20T22:20:57.732 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.732 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb0ef24c:::benchmark_data_smithi060_196073_object17169:head# 2024-09-20T22:20:57.732 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.733 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb0f0c8d:::benchmark_data_smithi060_196073_object79:head# 2024-09-20T22:20:57.736 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.736 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb0f5721:::benchmark_data_smithi060_196073_object34698:head# 2024-09-20T22:20:57.736 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.736 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb0ffc56:::benchmark_data_smithi060_196073_object12372:head# 2024-09-20T22:20:57.736 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.737 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb108cbd:::benchmark_data_smithi060_196073_object8328:head# 2024-09-20T22:20:57.743 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:57.746 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.746 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb10f295:::benchmark_data_smithi060_196073_object47643:head# 2024-09-20T22:20:57.752 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.753 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb132b63:::benchmark_data_smithi060_196073_object6745:head# 2024-09-20T22:20:57.753 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.753 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb133bab:::benchmark_data_smithi060_196073_object16489:head# 2024-09-20T22:20:57.753 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.754 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb14a501:::benchmark_data_smithi060_196073_object2664:head# 2024-09-20T22:20:57.754 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.754 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb150b51:::benchmark_data_smithi060_196073_object10828:head# 2024-09-20T22:20:57.783 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.783 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb15a23e:::benchmark_data_smithi060_196073_object21495:head# 2024-09-20T22:20:57.783 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.784 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb163528:::benchmark_data_smithi060_196073_object48046:head# 2024-09-20T22:20:57.784 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.784 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb164380:::benchmark_data_smithi060_196073_object31766:head# 2024-09-20T22:20:57.784 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.784 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb1a1197:::benchmark_data_smithi060_196073_object57659:head# 2024-09-20T22:20:57.785 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.785 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb1b87ef:::benchmark_data_smithi060_196073_object13483:head# 2024-09-20T22:20:57.785 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.785 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb1c6d62:::benchmark_data_smithi060_196073_object55141:head# 2024-09-20T22:20:57.785 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.785 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb1d34e8:::benchmark_data_smithi060_196073_object22233:head# 2024-09-20T22:20:57.785 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.786 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb1d6f4f:::benchmark_data_smithi060_196073_object41467:head# 2024-09-20T22:20:57.786 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.787 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb1df91b:::benchmark_data_smithi060_196073_object13828:head# 2024-09-20T22:20:57.787 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.787 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb1fb2b2:::benchmark_data_smithi060_196073_object45945:head# 2024-09-20T22:20:57.787 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.787 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb1fc7aa:::benchmark_data_smithi060_196073_object48441:head# 2024-09-20T22:20:57.787 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.787 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb213cdb:::benchmark_data_smithi060_196073_object37661:head# 2024-09-20T22:20:57.787 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.787 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb2178e2:::benchmark_data_smithi060_196073_object12181:head# 2024-09-20T22:20:57.787 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.788 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb21a8c0:::benchmark_data_smithi060_196073_object33395:head# 2024-09-20T22:20:57.788 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.788 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb21e115:::benchmark_data_smithi060_196073_object46082:head# 2024-09-20T22:20:57.788 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.788 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb239abc:::benchmark_data_smithi060_196073_object2815:head# 2024-09-20T22:20:57.789 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:57.789 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:57.790 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:57.829 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.829 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb24749c:::benchmark_data_smithi060_196073_object56707:head# 2024-09-20T22:20:57.829 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.830 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb24dd80:::benchmark_data_smithi060_196073_object3055:head# 2024-09-20T22:20:57.830 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:57.830 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb2754d7:::benchmark_data_smithi060_196073_object33730:head# 2024-09-20T22:20:57.890 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:57.925 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:57.926 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:57.926 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:57.949 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:20:57.991 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:57 smithi060 ceph-mon[29694]: pgmap v964: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.5 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.17k op/s; 2707/222264 objects degraded (1.218%); 10942/222264 objects misplaced (4.923%); 21 MiB/s, 335 objects/s recovering 2024-09-20T22:20:57.991 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:57 smithi060 ceph-mon[29694]: osdmap e507: 8 total, 7 up, 6 in 2024-09-20T22:20:57.991 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:57 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2995317394' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:20:57.991 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:57 smithi060 ceph-mon[29694]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:20:57.991 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:57 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 2707/222264 objects degraded (1.218%), 3 pgs degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T22:20:57.991 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:57 smithi060 ceph-mon[40290]: pgmap v964: 178 pgs: 1 active+remapped+backfilling, 3 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.5 GiB data, 17 GiB used, 519 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.17k op/s; 2707/222264 objects degraded (1.218%); 10942/222264 objects misplaced (4.923%); 21 MiB/s, 335 objects/s recovering 2024-09-20T22:20:57.991 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:57 smithi060 ceph-mon[40290]: osdmap e507: 8 total, 7 up, 6 in 2024-09-20T22:20:57.992 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:57 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2995317394' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:20:57.992 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:57 smithi060 ceph-mon[40290]: Health check update: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull (PG_BACKFILL_FULL) 2024-09-20T22:20:57.992 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:57 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 2707/222264 objects degraded (1.218%), 3 pgs degraded, 3 pgs undersized (PG_DEGRADED) 2024-09-20T22:20:58.027 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:58.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:58.057 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:58.057 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:58.064 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.065 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb2760d8:::benchmark_data_smithi060_196073_object32684:head# 2024-09-20T22:20:58.065 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.065 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb2960b7:::benchmark_data_smithi060_196073_object49937:head# 2024-09-20T22:20:58.065 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.065 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb29ffb4:::benchmark_data_smithi060_196073_object34337:head# 2024-09-20T22:20:58.065 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.065 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb2abae0:::benchmark_data_smithi060_196073_object39906:head# 2024-09-20T22:20:58.113 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.113 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb2b47c7:::benchmark_data_smithi060_196073_object46147:head# 2024-09-20T22:20:58.113 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.113 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb2cb30b:::benchmark_data_smithi060_196073_object29974:head# 2024-09-20T22:20:58.113 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.114 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb2e7178:::benchmark_data_smithi060_196073_object37033:head# 2024-09-20T22:20:58.114 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.114 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb2ebce3:::benchmark_data_smithi060_196073_object12439:head# 2024-09-20T22:20:58.117 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.117 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb315ff7:::benchmark_data_smithi060_196073_object17202:head# 2024-09-20T22:20:58.117 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.118 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb330bdd:::benchmark_data_smithi060_196073_object59161:head# 2024-09-20T22:20:58.118 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.118 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb333cd4:::benchmark_data_smithi060_196073_object33939:head# 2024-09-20T22:20:58.118 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.118 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb34052f:::benchmark_data_smithi060_196073_object18864:head# 2024-09-20T22:20:58.126 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.126 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb39ff7a:::benchmark_data_smithi060_196073_object4683:head# 2024-09-20T22:20:58.126 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.126 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb3cb8ac:::benchmark_data_smithi060_196073_object18128:head# 2024-09-20T22:20:58.127 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.127 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb3d4036:::benchmark_data_smithi060_196073_object43885:head# 2024-09-20T22:20:58.127 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.127 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb3f7e11:::benchmark_data_smithi060_196073_object58713:head# 2024-09-20T22:20:58.127 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.128 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb3f8b71:::benchmark_data_smithi060_196073_object30159:head# 2024-09-20T22:20:58.128 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.128 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb3ff54b:::benchmark_data_smithi060_196073_object14506:head# 2024-09-20T22:20:58.128 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.129 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb40f1b7:::benchmark_data_smithi060_196073_object7761:head# 2024-09-20T22:20:58.129 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.129 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb4204ad:::benchmark_data_smithi060_196073_object46075:head# 2024-09-20T22:20:58.129 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.129 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb420ec3:::benchmark_data_smithi060_196073_object38641:head# 2024-09-20T22:20:58.129 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.129 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb441ca6:::benchmark_data_smithi060_196073_object13776:head# 2024-09-20T22:20:58.130 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.130 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb45216b:::benchmark_data_smithi060_196073_object36692:head# 2024-09-20T22:20:58.130 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.130 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb4570a2:::benchmark_data_smithi060_196073_object17630:head# 2024-09-20T22:20:58.130 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.130 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb45c9c9:::benchmark_data_smithi060_196073_object60595:head# 2024-09-20T22:20:58.130 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.130 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb47e7ce:::benchmark_data_smithi060_196073_object35287:head# 2024-09-20T22:20:58.131 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.131 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb496a00:::benchmark_data_smithi060_196073_object59659:head# 2024-09-20T22:20:58.131 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.132 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb49ce7b:::benchmark_data_smithi060_196073_object28396:head# 2024-09-20T22:20:58.132 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.132 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb4a4682:::benchmark_data_smithi060_196073_object32945:head# 2024-09-20T22:20:58.132 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.132 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb4a8a1a:::benchmark_data_smithi060_196073_object30631:head# 2024-09-20T22:20:58.132 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.133 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb4b469c:::benchmark_data_smithi060_196073_object23165:head# 2024-09-20T22:20:58.133 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.133 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb4c4087:::benchmark_data_smithi060_196073_object8792:head# 2024-09-20T22:20:58.133 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.134 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb4da3ca:::benchmark_data_smithi060_196073_object6916:head# 2024-09-20T22:20:58.134 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.134 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb4f9f8a:::benchmark_data_smithi060_196073_object25711:head# 2024-09-20T22:20:58.134 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.134 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb4fa7ee:::benchmark_data_smithi060_196073_object9980:head# 2024-09-20T22:20:58.134 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.134 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb4fa871:::benchmark_data_smithi060_196073_object12066:head# 2024-09-20T22:20:58.134 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.135 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb517d06:::benchmark_data_smithi060_196073_object7900:head# 2024-09-20T22:20:58.135 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.136 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb51c69f:::benchmark_data_smithi060_196073_object32080:head# 2024-09-20T22:20:58.136 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.136 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb52dd92:::benchmark_data_smithi060_196073_object59108:head# 2024-09-20T22:20:58.136 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.136 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb531ae1:::benchmark_data_smithi060_196073_object16333:head# 2024-09-20T22:20:58.136 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.136 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb54322a:::benchmark_data_smithi060_196073_object6630:head# 2024-09-20T22:20:58.136 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.136 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb563288:::benchmark_data_smithi060_196073_object17075:head# 2024-09-20T22:20:58.137 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.138 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb583611:::benchmark_data_smithi060_196073_object12792:head# 2024-09-20T22:20:58.138 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.138 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb588bf5:::benchmark_data_smithi060_196073_object42371:head# 2024-09-20T22:20:58.138 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.138 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb5c2898:::benchmark_data_smithi060_196073_object4336:head# 2024-09-20T22:20:58.138 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.138 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb5d8761:::benchmark_data_smithi060_196073_object53366:head# 2024-09-20T22:20:58.138 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.138 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb5d90ab:::benchmark_data_smithi060_196073_object34025:head# 2024-09-20T22:20:58.138 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.138 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb5e13fb:::benchmark_data_smithi060_196073_object33272:head# 2024-09-20T22:20:58.138 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.140 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb60b240:::benchmark_data_smithi060_196073_object49234:head# 2024-09-20T22:20:58.140 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.140 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb60c140:::benchmark_data_smithi060_196073_object59940:head# 2024-09-20T22:20:58.140 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.140 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb61953b:::benchmark_data_smithi060_196073_object18848:head# 2024-09-20T22:20:58.140 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.140 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb62d8fd:::benchmark_data_smithi060_196073_object25206:head# 2024-09-20T22:20:58.140 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.140 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb639687:::benchmark_data_smithi060_196073_object59526:head# 2024-09-20T22:20:58.141 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.141 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb646aa5:::benchmark_data_smithi060_196073_object44604:head# 2024-09-20T22:20:58.141 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.141 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb65e2f0:::benchmark_data_smithi060_196073_object495:head# 2024-09-20T22:20:58.141 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.141 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb6656d5:::benchmark_data_smithi060_196073_object46937:head# 2024-09-20T22:20:58.142 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb68744f:::benchmark_data_smithi060_196073_object12531:head# 2024-09-20T22:20:58.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb6a3877:::benchmark_data_smithi060_196073_object2593:head# 2024-09-20T22:20:58.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb6ae4a2:::benchmark_data_smithi060_196073_object38780:head# 2024-09-20T22:20:58.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb6b6d19:::benchmark_data_smithi060_196073_object36149:head# 2024-09-20T22:20:58.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb6b8073:::benchmark_data_smithi060_196073_object22652:head# 2024-09-20T22:20:58.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb6b9bca:::benchmark_data_smithi060_196073_object50174:head# 2024-09-20T22:20:58.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb6d4117:::benchmark_data_smithi060_196073_object44440:head# 2024-09-20T22:20:58.144 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.144 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb6d43cb:::benchmark_data_smithi060_196073_object33913:head# 2024-09-20T22:20:58.145 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.145 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb6e6a3a:::benchmark_data_smithi060_196073_object49274:head# 2024-09-20T22:20:58.145 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.145 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb6e9980:::benchmark_data_smithi060_196073_object40766:head# 2024-09-20T22:20:58.145 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.145 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb6f4445:::benchmark_data_smithi060_196073_object30385:head# 2024-09-20T22:20:58.145 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.145 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb6fbf1b:::benchmark_data_smithi060_196073_object7929:head# 2024-09-20T22:20:58.145 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.145 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb702bab:::benchmark_data_smithi060_196073_object41526:head# 2024-09-20T22:20:58.146 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.146 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb725ec8:::benchmark_data_smithi060_196073_object59236:head# 2024-09-20T22:20:58.147 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.147 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb729b38:::benchmark_data_smithi060_196073_object25716:head# 2024-09-20T22:20:58.147 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.147 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb7317bc:::benchmark_data_smithi060_196073_object56866:head# 2024-09-20T22:20:58.147 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.147 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb747293:::benchmark_data_smithi060_196073_object3379:head# 2024-09-20T22:20:58.147 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.147 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb74c8a4:::benchmark_data_smithi060_196073_object37263:head# 2024-09-20T22:20:58.147 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.148 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb77bc5b:::benchmark_data_smithi060_196073_object12196:head# 2024-09-20T22:20:58.148 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.148 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb7972b4:::benchmark_data_smithi060_196073_object34135:head# 2024-09-20T22:20:58.148 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.149 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb79e732:::benchmark_data_smithi060_196073_object24266:head# 2024-09-20T22:20:58.149 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.149 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb7fe6d3:::benchmark_data_smithi060_196073_object36233:head# 2024-09-20T22:20:58.149 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.149 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb8051e9:::benchmark_data_smithi060_196073_object1374:head# 2024-09-20T22:20:58.149 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.150 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb80a544:::benchmark_data_smithi060_196073_object59458:head# 2024-09-20T22:20:58.150 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.150 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb80dcbd:::benchmark_data_smithi060_196073_object14263:head# 2024-09-20T22:20:58.150 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.150 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb80f7a4:::benchmark_data_smithi060_196073_object35229:head# 2024-09-20T22:20:58.150 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.151 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb82cc44:::benchmark_data_smithi060_196073_object2389:head# 2024-09-20T22:20:58.151 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.151 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb84150b:::benchmark_data_smithi060_196073_object16272:head# 2024-09-20T22:20:58.151 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.151 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb841d1d:::benchmark_data_smithi060_196073_object24647:head# 2024-09-20T22:20:58.152 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.152 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb861ca8:::benchmark_data_smithi060_196073_object38911:head# 2024-09-20T22:20:58.152 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.152 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb864169:::benchmark_data_smithi060_196073_object21737:head# 2024-09-20T22:20:58.152 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.152 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb86617e:::benchmark_data_smithi060_196073_object9821:head# 2024-09-20T22:20:58.152 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.152 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb868dee:::benchmark_data_smithi060_196073_object44253:head# 2024-09-20T22:20:58.152 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.154 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb8c9d28:::benchmark_data_smithi060_196073_object12546:head# 2024-09-20T22:20:58.154 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.154 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb8cc1f2:::benchmark_data_smithi060_196073_object60290:head# 2024-09-20T22:20:58.154 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.154 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb8d3d5a:::benchmark_data_smithi060_196073_object32283:head# 2024-09-20T22:20:58.154 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.154 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb8f0144:::benchmark_data_smithi060_196073_object45178:head# 2024-09-20T22:20:58.154 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.154 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb8fca97:::benchmark_data_smithi060_196073_object57596:head# 2024-09-20T22:20:58.154 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.154 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb9224d8:::benchmark_data_smithi060_196073_object3439:head# 2024-09-20T22:20:58.156 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.156 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb92e120:::benchmark_data_smithi060_196073_object58657:head# 2024-09-20T22:20:58.156 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.156 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb986cec:::benchmark_data_smithi060_196073_object53642:head# 2024-09-20T22:20:58.156 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.156 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb9b2e5c:::benchmark_data_smithi060_196073_object33876:head# 2024-09-20T22:20:58.156 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.156 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bb9b6a12:::benchmark_data_smithi060_196073_object55290:head# 2024-09-20T22:20:58.156 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.156 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bba06870:::benchmark_data_smithi060_196073_object36468:head# 2024-09-20T22:20:58.156 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.157 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bba0f957:::benchmark_data_smithi060_196073_object13972:head# 2024-09-20T22:20:58.157 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.157 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bba30c17:::benchmark_data_smithi060_196073_object1956:head# 2024-09-20T22:20:58.157 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.157 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bba378b1:::benchmark_data_smithi060_196073_object36298:head# 2024-09-20T22:20:58.158 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:58.180 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:58.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:58.181 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:58.200 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.200 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bba68543:::benchmark_data_smithi060_196073_object13822:head# 2024-09-20T22:20:58.201 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.201 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bba76a5f:::benchmark_data_smithi060_196073_object37126:head# 2024-09-20T22:20:58.281 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:58.285 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.285 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bba85e2b:::benchmark_data_smithi060_196073_object47404:head# 2024-09-20T22:20:58.285 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.286 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bba893a0:::benchmark_data_smithi060_196073_object2323:head# 2024-09-20T22:20:58.286 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.286 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bba8b435:::benchmark_data_smithi060_196073_object38274:head# 2024-09-20T22:20:58.286 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.287 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbab058c:::benchmark_data_smithi060_196073_object51540:head# 2024-09-20T22:20:58.345 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.345 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbac00b0:::benchmark_data_smithi060_196073_object58813:head# 2024-09-20T22:20:58.346 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.346 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbad5e5f:::benchmark_data_smithi060_196073_object57708:head# 2024-09-20T22:20:58.346 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.346 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbafb1f2:::benchmark_data_smithi060_196073_object30699:head# 2024-09-20T22:20:58.346 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.346 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbb31d9b:::benchmark_data_smithi060_196073_object12221:head# 2024-09-20T22:20:58.346 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.347 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbb33e47:::benchmark_data_smithi060_196073_object3537:head# 2024-09-20T22:20:58.347 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.347 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbb36f74:::benchmark_data_smithi060_196073_object37596:head# 2024-09-20T22:20:58.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:58.357 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:58.357 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:58.363 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.364 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbb42474:::benchmark_data_smithi060_196073_object11446:head# 2024-09-20T22:20:58.364 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.364 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbb4f47e:::benchmark_data_smithi060_196073_object13135:head# 2024-09-20T22:20:58.364 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.364 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbb6bc63:::benchmark_data_smithi060_196073_object45563:head# 2024-09-20T22:20:58.365 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.365 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbb7f065:::benchmark_data_smithi060_196073_object53383:head# 2024-09-20T22:20:58.372 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.372 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbb8790c:::benchmark_data_smithi060_196073_object55807:head# 2024-09-20T22:20:58.372 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.372 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbb8da08:::benchmark_data_smithi060_196073_object29586:head# 2024-09-20T22:20:58.373 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.373 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbb9536e:::benchmark_data_smithi060_196073_object53038:head# 2024-09-20T22:20:58.373 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.373 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbba6610:::benchmark_data_smithi060_196073_object44175:head# 2024-09-20T22:20:58.374 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:20:58.408 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.408 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbbd06f8:::benchmark_data_smithi060_196073_object35356:head# 2024-09-20T22:20:58.408 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.408 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbbd7192:::benchmark_data_smithi060_196073_object14433:head# 2024-09-20T22:20:58.408 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.409 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbbdb850:::benchmark_data_smithi060_196073_object46606:head# 2024-09-20T22:20:58.409 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.409 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbbdd59b:::benchmark_data_smithi060_196073_object11633:head# 2024-09-20T22:20:58.412 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.412 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbbde8ea:::benchmark_data_smithi060_196073_object19440:head# 2024-09-20T22:20:58.413 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.413 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbc027e9:::benchmark_data_smithi060_196073_object17917:head# 2024-09-20T22:20:58.413 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.413 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbc10cb3:::benchmark_data_smithi060_196073_object47866:head# 2024-09-20T22:20:58.414 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.414 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbc2028d:::benchmark_data_smithi060_196073_object5968:head# 2024-09-20T22:20:58.457 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:58.492 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:58.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:58.493 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:58.536 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.536 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbc2a008:::benchmark_data_smithi060_196073_object40075:head# 2024-09-20T22:20:58.536 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.536 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbc2c2b9:::benchmark_data_smithi060_196073_object33185:head# 2024-09-20T22:20:58.536 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.537 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbc2d4b1:::benchmark_data_smithi060_196073_object37561:head# 2024-09-20T22:20:58.537 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.537 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbc323af:::benchmark_data_smithi060_196073_object24898:head# 2024-09-20T22:20:58.595 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:20:58.630 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:58.631 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:58.631 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:20:58.654 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.654 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:58 smithi067 ceph-mon[33511]: 11.c continuing backfill to osd.5 from (477'4347,506'8354] MIN to 506'8354 2024-09-20T22:20:58.654 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:58 smithi067 ceph-mon[33511]: osdmap e508: 8 total, 7 up, 6 in 2024-09-20T22:20:58.655 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbc66ecb:::benchmark_data_smithi060_196073_object3671:head# 2024-09-20T22:20:58.656 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.656 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbc69f70:::benchmark_data_smithi060_196073_object61415:head# 2024-09-20T22:20:58.656 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.656 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbc746ba:::benchmark_data_smithi060_196073_object19607:head# 2024-09-20T22:20:58.656 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.656 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbc75e1c:::benchmark_data_smithi060_196073_object59959:head# 2024-09-20T22:20:58.656 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.656 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbc81daf:::benchmark_data_smithi060_196073_object57697:head# 2024-09-20T22:20:58.656 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.656 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbc95fb7:::benchmark_data_smithi060_196073_object10388:head# 2024-09-20T22:20:58.663 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.664 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbc9bc9f:::benchmark_data_smithi060_196073_object8092:head# 2024-09-20T22:20:58.664 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.664 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbca8540:::benchmark_data_smithi060_196073_object25961:head# 2024-09-20T22:20:58.664 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.664 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbcda8d0:::benchmark_data_smithi060_196073_object26327:head# 2024-09-20T22:20:58.670 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.670 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbce7e46:::benchmark_data_smithi060_196073_object91:head# 2024-09-20T22:20:58.671 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.671 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbd1ac89:::benchmark_data_smithi060_196073_object15422:head# 2024-09-20T22:20:58.672 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.672 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbd29803:::benchmark_data_smithi060_196073_object10159:head# 2024-09-20T22:20:58.672 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.672 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbd332cc:::benchmark_data_smithi060_196073_object61346:head# 2024-09-20T22:20:58.700 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.701 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbd456e4:::benchmark_data_smithi060_196073_object54033:head# 2024-09-20T22:20:58.701 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.701 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbd48e99:::benchmark_data_smithi060_196073_object60338:head# 2024-09-20T22:20:58.716 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.717 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbd4ec1d:::benchmark_data_smithi060_196073_object8481:head# 2024-09-20T22:20:58.717 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.717 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbd676cc:::benchmark_data_smithi060_196073_object23150:head# 2024-09-20T22:20:58.717 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.718 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbd69fe7:::benchmark_data_smithi060_196073_object9204:head# 2024-09-20T22:20:58.718 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.718 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbd6a8a4:::benchmark_data_smithi060_196073_object48280:head# 2024-09-20T22:20:58.718 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.718 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbd6e0d1:::benchmark_data_smithi060_196073_object38960:head# 2024-09-20T22:20:58.719 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.719 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbd7246e:::benchmark_data_smithi060_196073_object13641:head# 2024-09-20T22:20:58.731 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:58.764 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:58.765 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:58.765 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:58.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:58 smithi060 ceph-mon[29694]: 11.c continuing backfill to osd.5 from (477'4347,506'8354] MIN to 506'8354 2024-09-20T22:20:58.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:58 smithi060 ceph-mon[29694]: osdmap e508: 8 total, 7 up, 6 in 2024-09-20T22:20:58.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:58 smithi060 ceph-mon[40290]: 11.c continuing backfill to osd.5 from (477'4347,506'8354] MIN to 506'8354 2024-09-20T22:20:58.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:58 smithi060 ceph-mon[40290]: osdmap e508: 8 total, 7 up, 6 in 2024-09-20T22:20:58.819 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.820 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbd83065:::benchmark_data_smithi060_196073_object22361:head# 2024-09-20T22:20:58.820 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.820 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbd86d6a:::benchmark_data_smithi060_196073_object55178:head# 2024-09-20T22:20:58.820 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.821 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbd967a0:::benchmark_data_smithi060_196073_object17490:head# 2024-09-20T22:20:58.821 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.821 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbdb1102:::benchmark_data_smithi060_196073_object13874:head# 2024-09-20T22:20:58.865 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.865 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbdb2332:::benchmark_data_smithi060_196073_object18537:head# 2024-09-20T22:20:58.865 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.866 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:20:58.866 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbdcc80d:::benchmark_data_smithi060_196073_object44186:head# 2024-09-20T22:20:58.867 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.867 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbdd71ff:::benchmark_data_smithi060_196073_object47549:head# 2024-09-20T22:20:58.867 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.867 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbddf8dc:::benchmark_data_smithi060_196073_object60496:head# 2024-09-20T22:20:58.872 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.872 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbe0d441:::benchmark_data_smithi060_196073_object1522:head# 2024-09-20T22:20:58.872 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.872 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbe0ec45:::benchmark_data_smithi060_196073_object23893:head# 2024-09-20T22:20:58.872 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.873 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbe111e4:::benchmark_data_smithi060_196073_object47555:head# 2024-09-20T22:20:58.873 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.873 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbe3d295:::benchmark_data_smithi060_196073_object41105:head# 2024-09-20T22:20:58.882 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.882 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbe52e62:::benchmark_data_smithi060_196073_object40945:head# 2024-09-20T22:20:58.882 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.883 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbe736fd:::benchmark_data_smithi060_196073_object26367:head# 2024-09-20T22:20:58.883 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.883 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbe75eed:::benchmark_data_smithi060_196073_object7302:head# 2024-09-20T22:20:58.883 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.883 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbe9ff7d:::benchmark_data_smithi060_196073_object22650:head# 2024-09-20T22:20:58.890 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.890 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbeaa8eb:::benchmark_data_smithi060_196073_object6232:head# 2024-09-20T22:20:58.890 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.891 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbeac489:::benchmark_data_smithi060_196073_object36526:head# 2024-09-20T22:20:58.891 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.891 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbebb690:::benchmark_data_smithi060_196073_object31435:head# 2024-09-20T22:20:58.891 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.891 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbec0606:::benchmark_data_smithi060_196073_object54331:head# 2024-09-20T22:20:58.898 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.898 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:58.899 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:58.900 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbecfdaf:::benchmark_data_smithi060_196073_object12861:head# 2024-09-20T22:20:58.900 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.900 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbed1524:::benchmark_data_smithi060_196073_object42285:head# 2024-09-20T22:20:58.900 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.900 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbedde31:::benchmark_data_smithi060_196073_object39065:head# 2024-09-20T22:20:58.900 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:58.900 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbee1270:::benchmark_data_smithi060_196073_object59065:head# 2024-09-20T22:20:58.900 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:20:59.001 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:59.031 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:59.032 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:59.032 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:59.034 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.034 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbef3ad5:::benchmark_data_smithi060_196073_object6396:head# 2024-09-20T22:20:59.034 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.035 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbf05d4f:::benchmark_data_smithi060_196073_object14113:head# 2024-09-20T22:20:59.035 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.035 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbf10dd0:::benchmark_data_smithi060_196073_object35761:head# 2024-09-20T22:20:59.035 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.035 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbf2deb7:::benchmark_data_smithi060_196073_object29541:head# 2024-09-20T22:20:59.095 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.095 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbf2e272:::benchmark_data_smithi060_196073_object2677:head# 2024-09-20T22:20:59.095 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.095 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbf3c6db:::benchmark_data_smithi060_196073_object17211:head# 2024-09-20T22:20:59.095 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.096 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbf44b0e:::benchmark_data_smithi060_196073_object3148:head# 2024-09-20T22:20:59.096 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.096 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbf50c8a:::benchmark_data_smithi060_196073_object36143:head# 2024-09-20T22:20:59.099 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.100 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbf67010:::benchmark_data_smithi060_196073_object3293:head# 2024-09-20T22:20:59.100 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.100 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbf6a8bf:::benchmark_data_smithi060_196073_object42554:head# 2024-09-20T22:20:59.100 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.100 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbf7bc23:::benchmark_data_smithi060_196073_object2536:head# 2024-09-20T22:20:59.100 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.101 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbf7f065:::benchmark_data_smithi060_196073_object5592:head# 2024-09-20T22:20:59.101 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.102 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbf87f49:::benchmark_data_smithi060_196073_object19366:head# 2024-09-20T22:20:59.102 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.102 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbf89d67:::benchmark_data_smithi060_196073_object48610:head# 2024-09-20T22:20:59.102 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.102 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbf922a6:::benchmark_data_smithi060_196073_object49622:head# 2024-09-20T22:20:59.102 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.102 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbf97ad0:::benchmark_data_smithi060_196073_object31627:head# 2024-09-20T22:20:59.132 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:59.140 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.141 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbf9fb4b:::benchmark_data_smithi060_196073_object46537:head# 2024-09-20T22:20:59.141 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.141 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbfac2fd:::benchmark_data_smithi060_196073_object28420:head# 2024-09-20T22:20:59.141 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.141 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbfadd75:::benchmark_data_smithi060_196073_object57830:head# 2024-09-20T22:20:59.142 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.142 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbfb41f2:::benchmark_data_smithi060_196073_object60353:head# 2024-09-20T22:20:59.156 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:59.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:59.157 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:20:59.225 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.225 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbfb4a2d:::benchmark_data_smithi060_196073_object31866:head# 2024-09-20T22:20:59.225 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.226 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbfbb95e:::benchmark_data_smithi060_196073_object59900:head# 2024-09-20T22:20:59.226 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.226 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bbfde137:::benchmark_data_smithi060_196073_object30956:head# 2024-09-20T22:20:59.226 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.226 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc021f26:::benchmark_data_smithi060_196073_object25350:head# 2024-09-20T22:20:59.253 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.254 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc023271:::benchmark_data_smithi060_196073_object9427:head# 2024-09-20T22:20:59.254 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.254 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc02e9e1:::benchmark_data_smithi060_196073_object50099:head# 2024-09-20T22:20:59.254 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.255 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc0333d9:::benchmark_data_smithi060_196073_object14585:head# 2024-09-20T22:20:59.255 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.255 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc04a90f:::benchmark_data_smithi060_196073_object39192:head# 2024-09-20T22:20:59.258 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:20:59.262 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.262 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc050c1c:::benchmark_data_smithi060_196073_object13439:head# 2024-09-20T22:20:59.262 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.262 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc05158a:::benchmark_data_smithi060_196073_object56440:head# 2024-09-20T22:20:59.265 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.265 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc0638e7:::benchmark_data_smithi060_196073_object14804:head# 2024-09-20T22:20:59.265 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.266 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc0863d6:::benchmark_data_smithi060_196073_object35394:head# 2024-09-20T22:20:59.266 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.266 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc089e65:::benchmark_data_smithi060_196073_object17592:head# 2024-09-20T22:20:59.271 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.271 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc08fd22:::benchmark_data_smithi060_196073_object33219:head# 2024-09-20T22:20:59.271 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.272 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc098180:::benchmark_data_smithi060_196073_object27439:head# 2024-09-20T22:20:59.272 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.272 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc0a7d68:::benchmark_data_smithi060_196073_object52852:head# 2024-09-20T22:20:59.272 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.272 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc0a8449:::benchmark_data_smithi060_196073_object21980:head# 2024-09-20T22:20:59.281 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.281 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc0b2637:::benchmark_data_smithi060_196073_object16204:head# 2024-09-20T22:20:59.281 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.282 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc0b55e4:::benchmark_data_smithi060_196073_object40431:head# 2024-09-20T22:20:59.282 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.282 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc0d9548:::benchmark_data_smithi060_196073_object14629:head# 2024-09-20T22:20:59.282 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.283 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc0dfaf7:::benchmark_data_smithi060_196073_object18580:head# 2024-09-20T22:20:59.287 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:59.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:59.288 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:20:59.299 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.300 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc0fc093:::benchmark_data_smithi060_196073_object50946:head# 2024-09-20T22:20:59.300 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.300 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc10a788:::benchmark_data_smithi060_196073_object54378:head# 2024-09-20T22:20:59.300 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.300 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc11606e:::benchmark_data_smithi060_196073_object35441:head# 2024-09-20T22:20:59.300 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.300 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc11b00f:::benchmark_data_smithi060_196073_object49025:head# 2024-09-20T22:20:59.389 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:20:59.411 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:59.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:59.412 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.412 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc142077:::benchmark_data_smithi060_196073_object52838:head# 2024-09-20T22:20:59.412 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.412 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc15d3a8:::benchmark_data_smithi060_196073_object2348:head# 2024-09-20T22:20:59.413 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.413 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc173338:::benchmark_data_smithi060_196073_object30981:head# 2024-09-20T22:20:59.413 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.413 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc17cff6:::benchmark_data_smithi060_196073_object13510:head# 2024-09-20T22:20:59.413 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:20:59.463 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.464 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc17d24d:::benchmark_data_smithi060_196073_object18510:head# 2024-09-20T22:20:59.464 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.464 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc184813:::benchmark_data_smithi060_196073_object21301:head# 2024-09-20T22:20:59.464 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.464 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc189651:::benchmark_data_smithi060_196073_object34614:head# 2024-09-20T22:20:59.465 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.465 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc19bf57:::benchmark_data_smithi060_196073_object37381:head# 2024-09-20T22:20:59.506 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.506 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc1afed4:::benchmark_data_smithi060_196073_object53812:head# 2024-09-20T22:20:59.506 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.507 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc1ba603:::benchmark_data_smithi060_196073_object50479:head# 2024-09-20T22:20:59.507 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.507 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc1dd81d:::benchmark_data_smithi060_196073_object56726:head# 2024-09-20T22:20:59.507 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.507 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc1e6261:::benchmark_data_smithi060_196073_object58163:head# 2024-09-20T22:20:59.514 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:20:59.541 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:20:59.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:59.542 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:20:59.576 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:20:59.608 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.608 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc1fb221:::benchmark_data_smithi060_196073_object57307:head# 2024-09-20T22:20:59.608 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.609 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc21a96a:::benchmark_data_smithi060_196073_object5140:head# 2024-09-20T22:20:59.609 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.609 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc21af91:::benchmark_data_smithi060_196073_object33923:head# 2024-09-20T22:20:59.609 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.609 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc220cd3:::benchmark_data_smithi060_196073_object55554:head# 2024-09-20T22:20:59.641 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.641 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc25cd35:::benchmark_data_smithi060_196073_object19970:head# 2024-09-20T22:20:59.641 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.642 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc261fa5:::benchmark_data_smithi060_196073_object44048:head# 2024-09-20T22:20:59.642 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.642 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:59.643 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc267ee6:::benchmark_data_smithi060_196073_object22081:head# 2024-09-20T22:20:59.643 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.643 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc2865e9:::benchmark_data_smithi060_196073_object33422:head# 2024-09-20T22:20:59.669 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.670 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc29dfe9:::benchmark_data_smithi060_196073_object24624:head# 2024-09-20T22:20:59.670 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.673 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc29e24f:::benchmark_data_smithi060_196073_object36379:head# 2024-09-20T22:20:59.673 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.673 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc2af348:::benchmark_data_smithi060_196073_object50773:head# 2024-09-20T22:20:59.673 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.673 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc2bbafa:::benchmark_data_smithi060_196073_object53379:head# 2024-09-20T22:20:59.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:20:59.682 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:59.683 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:59.683 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:59.731 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.731 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc2c1579:::benchmark_data_smithi060_196073_object34:head# 2024-09-20T22:20:59.731 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.731 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc2c1b5f:::benchmark_data_smithi060_196073_object1661:head# 2024-09-20T22:20:59.732 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.732 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc2d8b3e:::benchmark_data_smithi060_196073_object34050:head# 2024-09-20T22:20:59.732 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.732 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc2fdb09:::benchmark_data_smithi060_196073_object34002:head# 2024-09-20T22:20:59.759 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.759 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc30a2e4:::benchmark_data_smithi060_196073_object5853:head# 2024-09-20T22:20:59.760 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.760 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc30b273:::benchmark_data_smithi060_196073_object14492:head# 2024-09-20T22:20:59.760 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.760 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc31d526:::benchmark_data_smithi060_196073_object55526:head# 2024-09-20T22:20:59.760 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.761 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc32f1a5:::benchmark_data_smithi060_196073_object49335:head# 2024-09-20T22:20:59.780 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.781 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc344d57:::benchmark_data_smithi060_196073_object9605:head# 2024-09-20T22:20:59.781 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.781 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc347aee:::benchmark_data_smithi060_196073_object42421:head# 2024-09-20T22:20:59.784 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:20:59.787 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.787 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc35aaad:::benchmark_data_smithi060_196073_object40678:head# 2024-09-20T22:20:59.787 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.787 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc36729f:::benchmark_data_smithi060_196073_object3910:head# 2024-09-20T22:20:59.787 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.787 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc37b92a:::benchmark_data_smithi060_196073_object58103:head# 2024-09-20T22:20:59.787 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.787 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc3808e1:::benchmark_data_smithi060_196073_object6317:head# 2024-09-20T22:20:59.819 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:59.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:59.820 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:20:59.845 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.845 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:59 smithi067 ceph-mon[33511]: pgmap v967: 178 pgs: 1 remapped+peering, 1 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.3 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 876 B/s rd, 0 B/s wr, 1.07k op/s; 2041/212352 objects degraded (0.961%); 8901/212352 objects misplaced (4.192%); 28 MiB/s, 445 objects/s recovering 2024-09-20T22:20:59.846 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:59 smithi067 ceph-mon[33511]: osdmap e509: 8 total, 7 up, 6 in 2024-09-20T22:20:59.846 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:59 smithi067 ceph-mon[33511]: 5.3 scrub starts 2024-09-20T22:20:59.846 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:59 smithi067 ceph-mon[33511]: 5.3 scrub ok 2024-09-20T22:20:59.846 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:20:59 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1367023137' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:20:59.846 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc38b58b:::benchmark_data_smithi060_196073_object15742:head# 2024-09-20T22:20:59.847 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.847 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc38fcb2:::benchmark_data_smithi060_196073_object15020:head# 2024-09-20T22:20:59.847 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.847 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc3a34ba:::benchmark_data_smithi060_196073_object53937:head# 2024-09-20T22:20:59.847 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.847 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc3a928c:::benchmark_data_smithi060_196073_object26845:head# 2024-09-20T22:20:59.885 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.885 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc41d9e5:::benchmark_data_smithi060_196073_object5257:head# 2024-09-20T22:20:59.885 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.885 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc42795c:::benchmark_data_smithi060_196073_object2868:head# 2024-09-20T22:20:59.885 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.886 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc44e8cc:::benchmark_data_smithi060_196073_object24670:head# 2024-09-20T22:20:59.886 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.886 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc461148:::benchmark_data_smithi060_196073_object45066:head# 2024-09-20T22:20:59.890 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.891 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc4a2867:::benchmark_data_smithi060_196073_object19487:head# 2024-09-20T22:20:59.891 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.891 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc4a7025:::benchmark_data_smithi060_196073_object32364:head# 2024-09-20T22:20:59.893 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.894 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc51df5f:::benchmark_data_smithi060_196073_object54610:head# 2024-09-20T22:20:59.894 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.894 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc522ec7:::benchmark_data_smithi060_196073_object27033:head# 2024-09-20T22:20:59.894 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.894 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc526f9f:::benchmark_data_smithi060_196073_object30040:head# 2024-09-20T22:20:59.894 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.895 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc5576c8:::benchmark_data_smithi060_196073_object14957:head# 2024-09-20T22:20:59.904 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.904 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc558c8c:::benchmark_data_smithi060_196073_object41266:head# 2024-09-20T22:20:59.904 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.905 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc55cbda:::benchmark_data_smithi060_196073_object30112:head# 2024-09-20T22:20:59.905 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.905 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc5734e6:::benchmark_data_smithi060_196073_object26589:head# 2024-09-20T22:20:59.905 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.905 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc576a73:::benchmark_data_smithi060_196073_object32400:head# 2024-09-20T22:20:59.912 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.913 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc598f36:::benchmark_data_smithi060_196073_object23443:head# 2024-09-20T22:20:59.913 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.913 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc5a6900:::benchmark_data_smithi060_196073_object17930:head# 2024-09-20T22:20:59.913 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:20:59.914 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc5ad5df:::benchmark_data_smithi060_196073_object51200:head# 2024-09-20T22:20:59.921 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:20:59.955 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:20:59.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:20:59.956 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:00.029 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.029 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc5baf30:::benchmark_data_smithi060_196073_object34206:head# 2024-09-20T22:21:00.029 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.030 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc5c00ca:::benchmark_data_smithi060_196073_object46292:head# 2024-09-20T22:21:00.030 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.030 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc5cbfce:::benchmark_data_smithi060_196073_object46269:head# 2024-09-20T22:21:00.030 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.031 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc5efba5:::benchmark_data_smithi060_196073_object42822:head# 2024-09-20T22:21:00.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:59 smithi060 ceph-mon[29694]: pgmap v967: 178 pgs: 1 remapped+peering, 1 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.3 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 876 B/s rd, 0 B/s wr, 1.07k op/s; 2041/212352 objects degraded (0.961%); 8901/212352 objects misplaced (4.192%); 28 MiB/s, 445 objects/s recovering 2024-09-20T22:21:00.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:59 smithi060 ceph-mon[29694]: osdmap e509: 8 total, 7 up, 6 in 2024-09-20T22:21:00.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:59 smithi060 ceph-mon[29694]: 5.3 scrub starts 2024-09-20T22:21:00.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:59 smithi060 ceph-mon[29694]: 5.3 scrub ok 2024-09-20T22:21:00.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:20:59 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1367023137' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:21:00.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:59 smithi060 ceph-mon[40290]: pgmap v967: 178 pgs: 1 remapped+peering, 1 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.3 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 876 B/s rd, 0 B/s wr, 1.07k op/s; 2041/212352 objects degraded (0.961%); 8901/212352 objects misplaced (4.192%); 28 MiB/s, 445 objects/s recovering 2024-09-20T22:21:00.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:59 smithi060 ceph-mon[40290]: osdmap e509: 8 total, 7 up, 6 in 2024-09-20T22:21:00.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:59 smithi060 ceph-mon[40290]: 5.3 scrub starts 2024-09-20T22:21:00.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:59 smithi060 ceph-mon[40290]: 5.3 scrub ok 2024-09-20T22:21:00.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:20:59 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1367023137' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:21:00.057 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:00.069 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.069 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc5f4f88:::benchmark_data_smithi060_196073_object2704:head# 2024-09-20T22:21:00.069 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.070 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc62b5ae:::benchmark_data_smithi060_196073_object28653:head# 2024-09-20T22:21:00.070 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.070 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc62b929:::benchmark_data_smithi060_196073_object27397:head# 2024-09-20T22:21:00.070 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.070 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc646c40:::benchmark_data_smithi060_196073_object9889:head# 2024-09-20T22:21:00.083 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:00.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:00.084 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:00.151 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.151 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc65ba63:::benchmark_data_smithi060_196073_object31051:head# 2024-09-20T22:21:00.151 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.152 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc663a37:::benchmark_data_smithi060_196073_object41882:head# 2024-09-20T22:21:00.152 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.152 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc684ffa:::benchmark_data_smithi060_196073_object13223:head# 2024-09-20T22:21:00.152 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.152 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc688a93:::benchmark_data_smithi060_196073_object4913:head# 2024-09-20T22:21:00.185 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:00.212 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.212 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc69882e:::benchmark_data_smithi060_196073_object25595:head# 2024-09-20T22:21:00.212 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.212 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc6a0d1f:::benchmark_data_smithi060_196073_object33660:head# 2024-09-20T22:21:00.212 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.213 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc6a141b:::benchmark_data_smithi060_196073_object51097:head# 2024-09-20T22:21:00.213 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.214 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc6a195b:::benchmark_data_smithi060_196073_object9832:head# 2024-09-20T22:21:00.233 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:00.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:00.234 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:00.246 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.246 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc6b50b0:::benchmark_data_smithi060_196073_object19508:head# 2024-09-20T22:21:00.246 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.247 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc6bad46:::benchmark_data_smithi060_196073_object21412:head# 2024-09-20T22:21:00.247 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.247 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc6c3d14:::benchmark_data_smithi060_196073_object2073:head# 2024-09-20T22:21:00.247 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.247 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc6e4a40:::benchmark_data_smithi060_196073_object30132:head# 2024-09-20T22:21:00.248 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.249 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc6f9d07:::benchmark_data_smithi060_196073_object47385:head# 2024-09-20T22:21:00.249 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.249 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc710bdc:::benchmark_data_smithi060_196073_object32920:head# 2024-09-20T22:21:00.249 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.249 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc720f0a:::benchmark_data_smithi060_196073_object55064:head# 2024-09-20T22:21:00.250 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.250 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc72c378:::benchmark_data_smithi060_196073_object10959:head# 2024-09-20T22:21:00.254 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.254 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc72c50e:::benchmark_data_smithi060_196073_object48078:head# 2024-09-20T22:21:00.254 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.255 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc74e02b:::benchmark_data_smithi060_196073_object21227:head# 2024-09-20T22:21:00.255 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.255 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc763740:::benchmark_data_smithi060_196073_object36773:head# 2024-09-20T22:21:00.255 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.256 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc7aa13c:::benchmark_data_smithi060_196073_object31913:head# 2024-09-20T22:21:00.257 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.257 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc7c8bb7:::benchmark_data_smithi060_196073_object45230:head# 2024-09-20T22:21:00.257 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.257 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc7d6d93:::benchmark_data_smithi060_196073_object35292:head# 2024-09-20T22:21:00.267 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.267 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc7d7867:::benchmark_data_smithi060_196073_object42084:head# 2024-09-20T22:21:00.267 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.267 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc7ea880:::benchmark_data_smithi060_196073_object51618:head# 2024-09-20T22:21:00.268 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.268 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc83b6fb:::benchmark_data_smithi060_196073_object24357:head# 2024-09-20T22:21:00.268 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.268 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc8401d8:::benchmark_data_smithi060_196073_object11686:head# 2024-09-20T22:21:00.270 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.271 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc85c4c3:::benchmark_data_smithi060_196073_object43442:head# 2024-09-20T22:21:00.271 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.271 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc877429:::benchmark_data_smithi060_196073_object14931:head# 2024-09-20T22:21:00.271 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.271 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc884db8:::benchmark_data_smithi060_196073_object31774:head# 2024-09-20T22:21:00.271 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.272 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc8a9747:::benchmark_data_smithi060_196073_object39719:head# 2024-09-20T22:21:00.279 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.279 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc8aa174:::benchmark_data_smithi060_196073_object4315:head# 2024-09-20T22:21:00.279 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.280 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc8ba328:::benchmark_data_smithi060_196073_object41816:head# 2024-09-20T22:21:00.280 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.280 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc8beaa6:::benchmark_data_smithi060_196073_object18955:head# 2024-09-20T22:21:00.280 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.280 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc8cfff1:::benchmark_data_smithi060_196073_object18257:head# 2024-09-20T22:21:00.285 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.286 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc8e7083:::benchmark_data_smithi060_196073_object28676:head# 2024-09-20T22:21:00.286 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.286 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc8f9251:::benchmark_data_smithi060_196073_object34064:head# 2024-09-20T22:21:00.286 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.287 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc922a7c:::benchmark_data_smithi060_196073_object1843:head# 2024-09-20T22:21:00.287 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.287 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc92a8a3:::benchmark_data_smithi060_196073_object61740:head# 2024-09-20T22:21:00.336 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:00.368 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:00.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:00.369 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:00.469 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.469 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc946ed7:::benchmark_data_smithi060_196073_object58742:head# 2024-09-20T22:21:00.469 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.469 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:00.470 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc9513c7:::benchmark_data_smithi060_196073_object46150:head# 2024-09-20T22:21:00.470 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.470 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc952e36:::benchmark_data_smithi060_196073_object43157:head# 2024-09-20T22:21:00.470 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.470 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc965a88:::benchmark_data_smithi060_196073_object989:head# 2024-09-20T22:21:00.508 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:00.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:00.509 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:00.515 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.515 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc97fb27:::benchmark_data_smithi060_196073_object36763:head# 2024-09-20T22:21:00.515 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.516 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc9b1622:::benchmark_data_smithi060_196073_object55564:head# 2024-09-20T22:21:00.516 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.516 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc9ba979:::benchmark_data_smithi060_196073_object14295:head# 2024-09-20T22:21:00.516 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.517 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc9c4861:::benchmark_data_smithi060_196073_object26338:head# 2024-09-20T22:21:00.521 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.522 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc9c74f0:::benchmark_data_smithi060_196073_object46173:head# 2024-09-20T22:21:00.522 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.522 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc9cf4ed:::benchmark_data_smithi060_196073_object53299:head# 2024-09-20T22:21:00.523 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.523 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc9d0a35:::benchmark_data_smithi060_196073_object34065:head# 2024-09-20T22:21:00.523 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.523 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc9d37a2:::benchmark_data_smithi060_196073_object48960:head# 2024-09-20T22:21:00.542 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.542 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bc9de400:::benchmark_data_smithi060_196073_object49521:head# 2024-09-20T22:21:00.542 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.543 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bca08d2c:::benchmark_data_smithi060_196073_object47010:head# 2024-09-20T22:21:00.543 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.543 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bca1014c:::benchmark_data_smithi060_196073_object28772:head# 2024-09-20T22:21:00.543 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.543 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bca104da:::benchmark_data_smithi060_196073_object43287:head# 2024-09-20T22:21:00.557 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.558 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bca1ad03:::benchmark_data_smithi060_196073_object59396:head# 2024-09-20T22:21:00.558 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.558 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bca1fcbe:::benchmark_data_smithi060_196073_object31005:head# 2024-09-20T22:21:00.558 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.559 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bca2f760:::benchmark_data_smithi060_196073_object41730:head# 2024-09-20T22:21:00.559 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.559 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bca34a15:::benchmark_data_smithi060_196073_object33707:head# 2024-09-20T22:21:00.573 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.574 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bca5d84c:::benchmark_data_smithi060_196073_object48154:head# 2024-09-20T22:21:00.574 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.574 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bca6894f:::benchmark_data_smithi060_196073_object4490:head# 2024-09-20T22:21:00.574 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.574 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bca7e26d:::benchmark_data_smithi060_196073_object27588:head# 2024-09-20T22:21:00.574 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.609 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:00.642 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:00.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:00.643 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:00.678 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bca7e591:::benchmark_data_smithi060_196073_object54179:head# 2024-09-20T22:21:00.679 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.679 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bca8c906:::benchmark_data_smithi060_196073_object4794:head# 2024-09-20T22:21:00.698 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.699 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcaa332e:::benchmark_data_smithi060_196073_object45217:head# 2024-09-20T22:21:00.699 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.699 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcaac595:::benchmark_data_smithi060_196073_object22944:head# 2024-09-20T22:21:00.744 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:00.758 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.758 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:00 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:21:00.758 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:00 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1367023137' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:21:00.758 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:00 smithi067 ceph-mon[33511]: 11.e continuing backfill to osd.3 from (487'5113,508'8298] MIN to 508'8298 2024-09-20T22:21:00.758 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:00 smithi067 ceph-mon[33511]: osdmap e510: 8 total, 7 up, 6 in 2024-09-20T22:21:00.758 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:00 smithi067 ceph-mon[33511]: 1.b scrub starts 2024-09-20T22:21:00.759 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:00 smithi067 ceph-mon[33511]: 1.b scrub ok 2024-09-20T22:21:00.759 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcabdaab:::benchmark_data_smithi060_196073_object35230:head# 2024-09-20T22:21:00.759 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.759 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcac0f7d:::benchmark_data_smithi060_196073_object47492:head# 2024-09-20T22:21:00.759 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.760 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcaf8cfc:::benchmark_data_smithi060_196073_object33467:head# 2024-09-20T22:21:00.760 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.760 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcb26583:::benchmark_data_smithi060_196073_object46038:head# 2024-09-20T22:21:00.764 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.765 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcb2de7c:::benchmark_data_smithi060_196073_object29985:head# 2024-09-20T22:21:00.765 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.765 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcb407ff:::benchmark_data_smithi060_196073_object10409:head# 2024-09-20T22:21:00.765 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.766 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcb7f02c:::benchmark_data_smithi060_196073_object6802:head# 2024-09-20T22:21:00.766 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.766 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcb7f3da:::benchmark_data_smithi060_196073_object37724:head# 2024-09-20T22:21:00.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:00.767 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:00.767 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:00.806 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.806 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcb81327:::benchmark_data_smithi060_196073_object1419:head# 2024-09-20T22:21:00.807 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.807 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcb8e950:::benchmark_data_smithi060_196073_object54092:head# 2024-09-20T22:21:00.807 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.807 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcbd9fc3:::benchmark_data_smithi060_196073_object44817:head# 2024-09-20T22:21:00.807 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.807 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcbdccd6:::benchmark_data_smithi060_196073_object27028:head# 2024-09-20T22:21:00.808 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.809 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcbe4e1c:::benchmark_data_smithi060_196073_object22300:head# 2024-09-20T22:21:00.809 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.809 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcbe7ab3:::benchmark_data_smithi060_196073_object57514:head# 2024-09-20T22:21:00.810 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.810 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcbf37f7:::benchmark_data_smithi060_196073_object53388:head# 2024-09-20T22:21:00.810 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.812 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcc009b2:::benchmark_data_smithi060_196073_object61364:head# 2024-09-20T22:21:00.868 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:00.895 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:00.896 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:00.897 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:00.915 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.916 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcc1286e:::benchmark_data_smithi060_196073_object47294:head# 2024-09-20T22:21:00.916 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.916 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcc213f4:::benchmark_data_smithi060_196073_object50601:head# 2024-09-20T22:21:00.916 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.917 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcc21c92:::benchmark_data_smithi060_196073_object15487:head# 2024-09-20T22:21:00.917 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.917 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcc281e0:::benchmark_data_smithi060_196073_object52864:head# 2024-09-20T22:21:00.954 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.954 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcc3c762:::benchmark_data_smithi060_196073_object2608:head# 2024-09-20T22:21:00.954 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.955 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcc5b1f5:::benchmark_data_smithi060_196073_object42459:head# 2024-09-20T22:21:00.955 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.955 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcc6ea35:::benchmark_data_smithi060_196073_object15205:head# 2024-09-20T22:21:00.955 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.955 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcc7362e:::benchmark_data_smithi060_196073_object31831:head# 2024-09-20T22:21:00.965 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.965 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcca0dca:::benchmark_data_smithi060_196073_object9746:head# 2024-09-20T22:21:00.965 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.966 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bccb5a04:::benchmark_data_smithi060_196073_object19613:head# 2024-09-20T22:21:00.969 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.969 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bccd76d8:::benchmark_data_smithi060_196073_object30648:head# 2024-09-20T22:21:00.969 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.970 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcd08e8c:::benchmark_data_smithi060_196073_object19042:head# 2024-09-20T22:21:00.970 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.970 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcd1c73d:::benchmark_data_smithi060_196073_object33170:head# 2024-09-20T22:21:00.970 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.970 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcd36c79:::benchmark_data_smithi060_196073_object19575:head# 2024-09-20T22:21:00.975 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.975 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcd36f5f:::benchmark_data_smithi060_196073_object7419:head# 2024-09-20T22:21:00.975 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.976 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcd4a71f:::benchmark_data_smithi060_196073_object26201:head# 2024-09-20T22:21:00.976 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.976 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcd53d47:::benchmark_data_smithi060_196073_object46454:head# 2024-09-20T22:21:00.976 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.976 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcd60f2b:::benchmark_data_smithi060_196073_object35380:head# 2024-09-20T22:21:00.983 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.983 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcd64f3e:::benchmark_data_smithi060_196073_object4115:head# 2024-09-20T22:21:00.983 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.984 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcd6a8f6:::benchmark_data_smithi060_196073_object21201:head# 2024-09-20T22:21:00.984 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.984 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcd77b90:::benchmark_data_smithi060_196073_object17101:head# 2024-09-20T22:21:00.984 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.984 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcd89ab5:::benchmark_data_smithi060_196073_object45369:head# 2024-09-20T22:21:00.987 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.987 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcd89b4e:::benchmark_data_smithi060_196073_object745:head# 2024-09-20T22:21:00.987 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.988 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcd95268:::benchmark_data_smithi060_196073_object16653:head# 2024-09-20T22:21:00.988 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.988 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcdac086:::benchmark_data_smithi060_196073_object536:head# 2024-09-20T22:21:00.988 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:00.988 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcdc542e:::benchmark_data_smithi060_196073_object48266:head# 2024-09-20T22:21:00.998 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:01.032 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:01.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:01.033 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:01.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:00 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:21:01.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:00 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1367023137' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:21:01.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:00 smithi060 ceph-mon[29694]: 11.e continuing backfill to osd.3 from (487'5113,508'8298] MIN to 508'8298 2024-09-20T22:21:01.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:00 smithi060 ceph-mon[29694]: osdmap e510: 8 total, 7 up, 6 in 2024-09-20T22:21:01.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:00 smithi060 ceph-mon[29694]: 1.b scrub starts 2024-09-20T22:21:01.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:00 smithi060 ceph-mon[29694]: 1.b scrub ok 2024-09-20T22:21:01.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:00 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:21:01.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:00 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1367023137' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:21:01.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:00 smithi060 ceph-mon[40290]: 11.e continuing backfill to osd.3 from (487'5113,508'8298] MIN to 508'8298 2024-09-20T22:21:01.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:00 smithi060 ceph-mon[40290]: osdmap e510: 8 total, 7 up, 6 in 2024-09-20T22:21:01.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:00 smithi060 ceph-mon[40290]: 1.b scrub starts 2024-09-20T22:21:01.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:00 smithi060 ceph-mon[40290]: 1.b scrub ok 2024-09-20T22:21:01.120 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.120 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcdce05b:::benchmark_data_smithi060_196073_object18512:head# 2024-09-20T22:21:01.120 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.121 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcdd4c06:::benchmark_data_smithi060_196073_object46169:head# 2024-09-20T22:21:01.121 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.121 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcde3794:::benchmark_data_smithi060_196073_object2903:head# 2024-09-20T22:21:01.121 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.122 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcdf4a20:::benchmark_data_smithi060_196073_object58216:head# 2024-09-20T22:21:01.134 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:01.156 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.157 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bce05fda:::benchmark_data_smithi060_196073_object22966:head# 2024-09-20T22:21:01.157 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.157 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bce0cb62:::benchmark_data_smithi060_196073_object25521:head# 2024-09-20T22:21:01.158 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.158 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bce1c775:::benchmark_data_smithi060_196073_object35816:head# 2024-09-20T22:21:01.158 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.158 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bce3ba7c:::benchmark_data_smithi060_196073_object627:head# 2024-09-20T22:21:01.166 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.166 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bce435c1:::benchmark_data_smithi060_196073_object620:head# 2024-09-20T22:21:01.167 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.167 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bce4db98:::benchmark_data_smithi060_196073_object8351:head# 2024-09-20T22:21:01.169 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:01.169 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:01.170 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:01.175 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.176 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bce56f6b:::benchmark_data_smithi060_196073_object33872:head# 2024-09-20T22:21:01.176 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.176 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bce5bcc1:::benchmark_data_smithi060_196073_object32795:head# 2024-09-20T22:21:01.176 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.177 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bce84bd4:::benchmark_data_smithi060_196073_object32627:head# 2024-09-20T22:21:01.177 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.177 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bce8af21:::benchmark_data_smithi060_196073_object47017:head# 2024-09-20T22:21:01.215 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.215 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bce904d0:::benchmark_data_smithi060_196073_object34948:head# 2024-09-20T22:21:01.215 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.215 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcebe04a:::benchmark_data_smithi060_196073_object59833:head# 2024-09-20T22:21:01.216 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.216 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcedeb23:::benchmark_data_smithi060_196073_object50061:head# 2024-09-20T22:21:01.216 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.216 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcee86b8:::benchmark_data_smithi060_196073_object59785:head# 2024-09-20T22:21:01.223 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.224 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bceed4de:::benchmark_data_smithi060_196073_object47050:head# 2024-09-20T22:21:01.224 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.224 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcefdc36:::benchmark_data_smithi060_196073_object15396:head# 2024-09-20T22:21:01.224 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.225 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcf1a06d:::benchmark_data_smithi060_196073_object35167:head# 2024-09-20T22:21:01.225 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.225 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcf219fa:::benchmark_data_smithi060_196073_object45358:head# 2024-09-20T22:21:01.271 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:01.307 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:01.308 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:01.308 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:01.345 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.346 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcf314a2:::benchmark_data_smithi060_196073_object6577:head# 2024-09-20T22:21:01.346 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.346 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcf3fd86:::benchmark_data_smithi060_196073_object50767:head# 2024-09-20T22:21:01.346 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.347 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcf429cc:::benchmark_data_smithi060_196073_object4103:head# 2024-09-20T22:21:01.405 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.406 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcf56cb1:::benchmark_data_smithi060_196073_object46909:head# 2024-09-20T22:21:01.406 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.407 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcf66696:::benchmark_data_smithi060_196073_object61504:head# 2024-09-20T22:21:01.407 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.407 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcfa6494:::benchmark_data_smithi060_196073_object33134:head# 2024-09-20T22:21:01.407 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.407 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcfbf76e:::benchmark_data_smithi060_196073_object51420:head# 2024-09-20T22:21:01.409 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:01.410 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.412 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcfc27ed:::benchmark_data_smithi060_196073_object5766:head# 2024-09-20T22:21:01.412 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.412 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcfdaeb6:::benchmark_data_smithi060_196073_object4479:head# 2024-09-20T22:21:01.412 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.412 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bcfe172e:::benchmark_data_smithi060_196073_object52666:head# 2024-09-20T22:21:01.412 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.412 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd002493:::benchmark_data_smithi060_196073_object25693:head# 2024-09-20T22:21:01.417 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.418 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd021dce:::benchmark_data_smithi060_196073_object60880:head# 2024-09-20T22:21:01.418 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.418 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd02ef0f:::benchmark_data_smithi060_196073_object49261:head# 2024-09-20T22:21:01.448 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:01.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:01.449 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:01.500 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.500 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd037b8a:::benchmark_data_smithi060_196073_object865:head# 2024-09-20T22:21:01.500 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.501 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd046a15:::benchmark_data_smithi060_196073_object42485:head# 2024-09-20T22:21:01.501 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.501 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd061fb3:::benchmark_data_smithi060_196073_object34754:head# 2024-09-20T22:21:01.501 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.502 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd0663c1:::benchmark_data_smithi060_196073_object13049:head# 2024-09-20T22:21:01.549 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:01.553 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.553 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd06a458:::benchmark_data_smithi060_196073_object28350:head# 2024-09-20T22:21:01.553 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.554 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd095101:::benchmark_data_smithi060_196073_object36311:head# 2024-09-20T22:21:01.554 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.554 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd09c45f:::benchmark_data_smithi060_196073_object25053:head# 2024-09-20T22:21:01.554 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.555 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd0a456e:::benchmark_data_smithi060_196073_object26348:head# 2024-09-20T22:21:01.558 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.558 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd0af5f5:::benchmark_data_smithi060_196073_object15451:head# 2024-09-20T22:21:01.558 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.559 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd0c349b:::benchmark_data_smithi060_196073_object40098:head# 2024-09-20T22:21:01.559 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.559 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd0e6234:::benchmark_data_smithi060_196073_object9731:head# 2024-09-20T22:21:01.559 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.560 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd0e7613:::benchmark_data_smithi060_196073_object46278:head# 2024-09-20T22:21:01.567 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.567 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd0ec051:::benchmark_data_smithi060_196073_object10870:head# 2024-09-20T22:21:01.567 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.567 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd0ec72c:::benchmark_data_smithi060_196073_object29525:head# 2024-09-20T22:21:01.567 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.568 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd10459a:::benchmark_data_smithi060_196073_object33620:head# 2024-09-20T22:21:01.568 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.568 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd10ca88:::benchmark_data_smithi060_196073_object52515:head# 2024-09-20T22:21:01.580 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.580 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd112e79:::benchmark_data_smithi060_196073_object37662:head# 2024-09-20T22:21:01.580 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.581 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd129a8f:::benchmark_data_smithi060_196073_object49774:head# 2024-09-20T22:21:01.581 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:01.581 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:01.582 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:01.582 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.582 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd14fd5d:::benchmark_data_smithi060_196073_object13865:head# 2024-09-20T22:21:01.582 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.583 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd156953:::benchmark_data_smithi060_196073_object22708:head# 2024-09-20T22:21:01.583 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.583 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd15f90f:::benchmark_data_smithi060_196073_object10240:head# 2024-09-20T22:21:01.583 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.583 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd19b42b:::benchmark_data_smithi060_196073_object59776:head# 2024-09-20T22:21:01.675 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.675 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:01 smithi067 ceph-mon[33511]: pgmap v970: 178 pgs: 1 remapped+peering, 1 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.3 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 0 B/s wr, 825 op/s; 2041/212352 objects degraded (0.961%); 8901/212352 objects misplaced (4.192%); 27 MiB/s, 430 objects/s recovering 2024-09-20T22:21:01.675 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:01 smithi067 ceph-mon[33511]: 1.16 scrub starts 2024-09-20T22:21:01.675 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:01 smithi067 ceph-mon[33511]: 1.16 scrub ok 2024-09-20T22:21:01.676 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd1c614c:::benchmark_data_smithi060_196073_object44489:head# 2024-09-20T22:21:01.676 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.676 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd1ca953:::benchmark_data_smithi060_196073_object10713:head# 2024-09-20T22:21:01.676 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.677 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd1d0715:::benchmark_data_smithi060_196073_object25068:head# 2024-09-20T22:21:01.677 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.677 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd1e719a:::benchmark_data_smithi060_196073_object22572:head# 2024-09-20T22:21:01.682 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:01.706 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.707 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd211ce0:::benchmark_data_smithi060_196073_object28372:head# 2024-09-20T22:21:01.707 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.711 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd226a4b:::benchmark_data_smithi060_196073_object10758:head# 2024-09-20T22:21:01.711 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.713 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd22cfc9:::benchmark_data_smithi060_196073_object12783:head# 2024-09-20T22:21:01.713 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.714 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd2423ab:::benchmark_data_smithi060_196073_object48281:head# 2024-09-20T22:21:01.714 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.714 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd242b03:::benchmark_data_smithi060_196073_object2932:head# 2024-09-20T22:21:01.714 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.714 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd25b89c:::benchmark_data_smithi060_196073_object43587:head# 2024-09-20T22:21:01.730 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:01.731 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:01.731 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:01.746 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.747 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd25d85e:::benchmark_data_smithi060_196073_object45582:head# 2024-09-20T22:21:01.747 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.747 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd26017e:::benchmark_data_smithi060_196073_object48243:head# 2024-09-20T22:21:01.747 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.748 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd268973:::benchmark_data_smithi060_196073_object61515:head# 2024-09-20T22:21:01.748 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.748 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd2787cb:::benchmark_data_smithi060_196073_object29819:head# 2024-09-20T22:21:01.778 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.778 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd284308:::benchmark_data_smithi060_196073_object34775:head# 2024-09-20T22:21:01.778 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.779 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd299e96:::benchmark_data_smithi060_196073_object29605:head# 2024-09-20T22:21:01.779 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.779 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd2bdda2:::benchmark_data_smithi060_196073_object31259:head# 2024-09-20T22:21:01.779 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.780 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd2dae66:::benchmark_data_smithi060_196073_object19041:head# 2024-09-20T22:21:01.832 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:01.863 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.864 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:01.864 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd2eb386:::benchmark_data_smithi060_196073_object14961:head# 2024-09-20T22:21:01.864 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.864 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd2ec6ae:::benchmark_data_smithi060_196073_object20560:head# 2024-09-20T22:21:01.864 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:01.865 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:01.865 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd329f73:::benchmark_data_smithi060_196073_object11987:head# 2024-09-20T22:21:01.866 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.866 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd33edb7:::benchmark_data_smithi060_196073_object8179:head# 2024-09-20T22:21:01.910 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.910 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd347336:::benchmark_data_smithi060_196073_object17699:head# 2024-09-20T22:21:01.910 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.911 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd357313:::benchmark_data_smithi060_196073_object9753:head# 2024-09-20T22:21:01.911 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.911 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd361864:::benchmark_data_smithi060_196073_object33535:head# 2024-09-20T22:21:01.926 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.926 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd382f2f:::benchmark_data_smithi060_196073_object22562:head# 2024-09-20T22:21:01.926 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.927 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd3a738c:::benchmark_data_smithi060_196073_object19224:head# 2024-09-20T22:21:01.927 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.927 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd3b73e2:::benchmark_data_smithi060_196073_object9276:head# 2024-09-20T22:21:01.947 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T22:21:01.966 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:01.971 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:01 smithi060 ceph-mon[29694]: pgmap v970: 178 pgs: 1 remapped+peering, 1 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.3 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 0 B/s wr, 825 op/s; 2041/212352 objects degraded (0.961%); 8901/212352 objects misplaced (4.192%); 27 MiB/s, 430 objects/s recovering 2024-09-20T22:21:01.971 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:01 smithi060 ceph-mon[29694]: 1.16 scrub starts 2024-09-20T22:21:01.971 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:01 smithi060 ceph-mon[29694]: 1.16 scrub ok 2024-09-20T22:21:01.971 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:01 smithi060 ceph-mon[40290]: pgmap v970: 178 pgs: 1 remapped+peering, 1 active+remapped+backfilling, 2 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfill_toofull, 173 active+clean; 4.3 GiB data, 17 GiB used, 520 GiB / 536 GiB avail; 0 B/s wr, 825 op/s; 2041/212352 objects degraded (0.961%); 8901/212352 objects misplaced (4.192%); 27 MiB/s, 430 objects/s recovering 2024-09-20T22:21:01.971 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:01 smithi060 ceph-mon[40290]: 1.16 scrub starts 2024-09-20T22:21:01.971 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:01 smithi060 ceph-mon[40290]: 1.16 scrub ok 2024-09-20T22:21:01.977 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.978 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd3c1cf5:::benchmark_data_smithi060_196073_object19651:head# 2024-09-20T22:21:01.978 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.979 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd3c9e1d:::benchmark_data_smithi060_196073_object35772:head# 2024-09-20T22:21:01.979 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.980 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd3cfcc1:::benchmark_data_smithi060_196073_object38854:head# 2024-09-20T22:21:01.980 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.980 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd3dc06e:::benchmark_data_smithi060_196073_object14196:head# 2024-09-20T22:21:01.983 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.983 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd3e481a:::benchmark_data_smithi060_196073_object19544:head# 2024-09-20T22:21:01.983 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.983 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd3f3c66:::benchmark_data_smithi060_196073_object31899:head# 2024-09-20T22:21:01.984 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.984 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd40d363:::benchmark_data_smithi060_196073_object7233:head# 2024-09-20T22:21:01.984 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.984 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd428d2a:::benchmark_data_smithi060_196073_object49869:head# 2024-09-20T22:21:01.989 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.990 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd436b72:::benchmark_data_smithi060_196073_object25975:head# 2024-09-20T22:21:01.990 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.990 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd45f6b1:::benchmark_data_smithi060_196073_object21364:head# 2024-09-20T22:21:01.990 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.991 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd460150:::benchmark_data_smithi060_196073_object32705:head# 2024-09-20T22:21:01.998 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.998 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd479a1a:::benchmark_data_smithi060_196073_object53794:head# 2024-09-20T22:21:01.998 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.998 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd484573:::benchmark_data_smithi060_196073_object27701:head# 2024-09-20T22:21:01.999 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.999 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd484ca8:::benchmark_data_smithi060_196073_object54921:head# 2024-09-20T22:21:01.999 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:01.999 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd4a9887:::benchmark_data_smithi060_196073_object53801:head# 2024-09-20T22:21:02.005 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.005 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd4bbbe3:::benchmark_data_smithi060_196073_object59031:head# 2024-09-20T22:21:02.006 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.006 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd4bdd65:::benchmark_data_smithi060_196073_object39521:head# 2024-09-20T22:21:02.006 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.006 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd4c8ec9:::benchmark_data_smithi060_196073_object31809:head# 2024-09-20T22:21:02.006 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.007 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd4d390a:::benchmark_data_smithi060_196073_object43294:head# 2024-09-20T22:21:02.007 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:02.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:02.008 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:02.013 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.013 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd4d6a76:::benchmark_data_smithi060_196073_object20341:head# 2024-09-20T22:21:02.014 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.014 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd4ef28a:::benchmark_data_smithi060_196073_object43469:head# 2024-09-20T22:21:02.014 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.015 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd50dabd:::benchmark_data_smithi060_196073_object22689:head# 2024-09-20T22:21:02.015 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.015 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd5362a2:::benchmark_data_smithi060_196073_object23208:head# 2024-09-20T22:21:02.020 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.020 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd53b976:::benchmark_data_smithi060_196073_object50140:head# 2024-09-20T22:21:02.020 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.021 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd540ec4:::benchmark_data_smithi060_196073_object20534:head# 2024-09-20T22:21:02.021 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.021 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd54f8fd:::benchmark_data_smithi060_196073_object37906:head# 2024-09-20T22:21:02.021 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.022 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd58fece:::benchmark_data_smithi060_196073_object44476:head# 2024-09-20T22:21:02.028 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.028 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd5adb95:::benchmark_data_smithi060_196073_object25588:head# 2024-09-20T22:21:02.028 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.029 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd5b5b84:::benchmark_data_smithi060_196073_object10534:head# 2024-09-20T22:21:02.029 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.029 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd5f25e6:::benchmark_data_smithi060_196073_object7845:head# 2024-09-20T22:21:02.029 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.029 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd6104d3:::benchmark_data_smithi060_196073_object37389:head# 2024-09-20T22:21:02.035 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.036 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd615b46:::benchmark_data_smithi060_196073_object32077:head# 2024-09-20T22:21:02.036 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.036 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd632715:::benchmark_data_smithi060_196073_object31243:head# 2024-09-20T22:21:02.036 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.037 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd638b27:::benchmark_data_smithi060_196073_object27314:head# 2024-09-20T22:21:02.037 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.037 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd6395dd:::benchmark_data_smithi060_196073_object37695:head# 2024-09-20T22:21:02.042 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.042 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd63de4d:::benchmark_data_smithi060_196073_object16683:head# 2024-09-20T22:21:02.042 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.042 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd6477fa:::benchmark_data_smithi060_196073_object2222:head# 2024-09-20T22:21:02.042 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.042 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd64f177:::benchmark_data_smithi060_196073_object52306:head# 2024-09-20T22:21:02.043 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.043 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd66518a:::benchmark_data_smithi060_196073_object44456:head# 2024-09-20T22:21:02.049 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.050 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd66bc95:::benchmark_data_smithi060_196073_object31616:head# 2024-09-20T22:21:02.050 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.050 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd676d56:::benchmark_data_smithi060_196073_object14941:head# 2024-09-20T22:21:02.050 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.050 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd679481:::benchmark_data_smithi060_196073_object51683:head# 2024-09-20T22:21:02.050 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.050 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd6826ca:::benchmark_data_smithi060_196073_object21961:head# 2024-09-20T22:21:02.058 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.058 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd696e4a:::benchmark_data_smithi060_196073_object2515:head# 2024-09-20T22:21:02.058 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.058 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd69ada6:::benchmark_data_smithi060_196073_object44665:head# 2024-09-20T22:21:02.059 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.059 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd6a3c4c:::benchmark_data_smithi060_196073_object40528:head# 2024-09-20T22:21:02.059 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.059 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd6aa71a:::benchmark_data_smithi060_196073_object14015:head# 2024-09-20T22:21:02.062 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.063 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd6bceee:::benchmark_data_smithi060_196073_object140:head# 2024-09-20T22:21:02.063 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.063 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd6c0026:::benchmark_data_smithi060_196073_object47662:head# 2024-09-20T22:21:02.063 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.063 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd6d4630:::benchmark_data_smithi060_196073_object3000:head# 2024-09-20T22:21:02.064 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.064 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd6f2d49:::benchmark_data_smithi060_196073_object29043:head# 2024-09-20T22:21:02.064 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.065 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd6f37b1:::benchmark_data_smithi060_196073_object3094:head# 2024-09-20T22:21:02.065 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.065 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd6f5508:::benchmark_data_smithi060_196073_object52206:head# 2024-09-20T22:21:02.065 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.066 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd6f6557:::benchmark_data_smithi060_196073_object3418:head# 2024-09-20T22:21:02.066 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.066 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd70d9ca:::benchmark_data_smithi060_196073_object48953:head# 2024-09-20T22:21:02.066 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.066 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd716e59:::benchmark_data_smithi060_196073_object19115:head# 2024-09-20T22:21:02.066 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.066 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd720c5a:::benchmark_data_smithi060_196073_object31287:head# 2024-09-20T22:21:02.066 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.067 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd73055d:::benchmark_data_smithi060_196073_object54767:head# 2024-09-20T22:21:02.067 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.067 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd745c5c:::benchmark_data_smithi060_196073_object61416:head# 2024-09-20T22:21:02.068 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.068 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd7748ec:::benchmark_data_smithi060_196073_object10621:head# 2024-09-20T22:21:02.068 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.068 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd779c32:::benchmark_data_smithi060_196073_object39623:head# 2024-09-20T22:21:02.068 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.068 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd77d924:::benchmark_data_smithi060_196073_object2644:head# 2024-09-20T22:21:02.068 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.069 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd77d932:::benchmark_data_smithi060_196073_object44773:head# 2024-09-20T22:21:02.069 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.069 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd78a723:::benchmark_data_smithi060_196073_object46114:head# 2024-09-20T22:21:02.069 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.070 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd7a01b7:::benchmark_data_smithi060_196073_object2826:head# 2024-09-20T22:21:02.070 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.070 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd7aa845:::benchmark_data_smithi060_196073_object31324:head# 2024-09-20T22:21:02.070 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.070 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd7ab34b:::benchmark_data_smithi060_196073_object10403:head# 2024-09-20T22:21:02.070 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.070 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd7b11fe:::benchmark_data_smithi060_196073_object53174:head# 2024-09-20T22:21:02.070 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.071 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd7b4682:::benchmark_data_smithi060_196073_object15948:head# 2024-09-20T22:21:02.071 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.071 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd7b5d4e:::benchmark_data_smithi060_196073_object24613:head# 2024-09-20T22:21:02.072 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.072 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd7d29cc:::benchmark_data_smithi060_196073_object31751:head# 2024-09-20T22:21:02.072 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.072 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd7dd3b6:::benchmark_data_smithi060_196073_object51766:head# 2024-09-20T22:21:02.072 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.072 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd86fbcb:::benchmark_data_smithi060_196073_object5149:head# 2024-09-20T22:21:02.072 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.072 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd8784dd:::benchmark_data_smithi060_196073_object1293:head# 2024-09-20T22:21:02.072 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.073 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd88fffe:::benchmark_data_smithi060_196073_object35672:head# 2024-09-20T22:21:02.073 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.074 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd8913bf:::benchmark_data_smithi060_196073_object58698:head# 2024-09-20T22:21:02.074 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.074 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd8b094d:::benchmark_data_smithi060_196073_object32683:head# 2024-09-20T22:21:02.074 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.074 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd8d5c0b:::benchmark_data_smithi060_196073_object10568:head# 2024-09-20T22:21:02.074 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.074 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd8e0b28:::benchmark_data_smithi060_196073_object16362:head# 2024-09-20T22:21:02.074 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.074 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd8ea7c3:::benchmark_data_smithi060_196073_object44840:head# 2024-09-20T22:21:02.074 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.075 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd9220fd:::benchmark_data_smithi060_196073_object43640:head# 2024-09-20T22:21:02.075 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.075 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd9259b5:::benchmark_data_smithi060_196073_object4421:head# 2024-09-20T22:21:02.075 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.075 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd929453:::benchmark_data_smithi060_196073_object8634:head# 2024-09-20T22:21:02.076 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.076 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd92dbe3:::benchmark_data_smithi060_196073_object59268:head# 2024-09-20T22:21:02.076 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.077 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd931252:::benchmark_data_smithi060_196073_object58117:head# 2024-09-20T22:21:02.077 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.077 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd93a4b7:::benchmark_data_smithi060_196073_object39256:head# 2024-09-20T22:21:02.077 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.077 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd944f4d:::benchmark_data_smithi060_196073_object40891:head# 2024-09-20T22:21:02.077 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.078 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd96293f:::benchmark_data_smithi060_196073_object47925:head# 2024-09-20T22:21:02.078 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.078 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd990619:::benchmark_data_smithi060_196073_object772:head# 2024-09-20T22:21:02.078 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.078 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd994699:::benchmark_data_smithi060_196073_object56635:head# 2024-09-20T22:21:02.078 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.079 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd995692:::benchmark_data_smithi060_196073_object22546:head# 2024-09-20T22:21:02.079 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.079 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd9aa562:::benchmark_data_smithi060_196073_object37924:head# 2024-09-20T22:21:02.079 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.079 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd9c4ff6:::benchmark_data_smithi060_196073_object58281:head# 2024-09-20T22:21:02.079 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.080 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd9cb624:::benchmark_data_smithi060_196073_object57513:head# 2024-09-20T22:21:02.080 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.080 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd9e6177:::benchmark_data_smithi060_196073_object61262:head# 2024-09-20T22:21:02.080 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.081 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd9e9a3e:::benchmark_data_smithi060_196073_object53396:head# 2024-09-20T22:21:02.081 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.081 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd9ec2ea:::benchmark_data_smithi060_196073_object22271:head# 2024-09-20T22:21:02.081 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.082 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bd9f44d2:::benchmark_data_smithi060_196073_object42222:head# 2024-09-20T22:21:02.082 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.082 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bda42ee4:::benchmark_data_smithi060_196073_object58716:head# 2024-09-20T22:21:02.082 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.083 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bda451a9:::benchmark_data_smithi060_196073_object35000:head# 2024-09-20T22:21:02.083 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.083 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bda504c2:::benchmark_data_smithi060_196073_object1516:head# 2024-09-20T22:21:02.083 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.083 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bda58ea9:::benchmark_data_smithi060_196073_object30376:head# 2024-09-20T22:21:02.084 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.084 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bda59806:::benchmark_data_smithi060_196073_object44523:head# 2024-09-20T22:21:02.085 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.085 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bda5e4ab:::benchmark_data_smithi060_196073_object16507:head# 2024-09-20T22:21:02.085 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.085 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bda5fd88:::benchmark_data_smithi060_196073_object33832:head# 2024-09-20T22:21:02.085 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.086 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bda9bb00:::benchmark_data_smithi060_196073_object29982:head# 2024-09-20T22:21:02.086 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.086 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bda9e1cb:::benchmark_data_smithi060_196073_object31742:head# 2024-09-20T22:21:02.086 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.086 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdab5bec:::benchmark_data_smithi060_196073_object39026:head# 2024-09-20T22:21:02.086 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.087 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdabeb69:::benchmark_data_smithi060_196073_object8490:head# 2024-09-20T22:21:02.087 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.087 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdadc62a:::benchmark_data_smithi060_196073_object48656:head# 2024-09-20T22:21:02.087 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.087 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdae472e:::benchmark_data_smithi060_196073_object41896:head# 2024-09-20T22:21:02.087 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.088 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdaee6a3:::benchmark_data_smithi060_196073_object38673:head# 2024-09-20T22:21:02.088 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.088 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdaee935:::benchmark_data_smithi060_196073_object49661:head# 2024-09-20T22:21:02.088 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.089 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdb0070a:::benchmark_data_smithi060_196073_object22549:head# 2024-09-20T22:21:02.089 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.089 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdb07e23:::benchmark_data_smithi060_196073_object12060:head# 2024-09-20T22:21:02.089 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.089 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdb3171f:::benchmark_data_smithi060_196073_object12894:head# 2024-09-20T22:21:02.089 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.090 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdb340fc:::benchmark_data_smithi060_196073_object36296:head# 2024-09-20T22:21:02.090 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.091 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdb49396:::benchmark_data_smithi060_196073_object55463:head# 2024-09-20T22:21:02.091 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.091 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdb7d142:::benchmark_data_smithi060_196073_object17183:head# 2024-09-20T22:21:02.091 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.091 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdbad48e:::benchmark_data_smithi060_196073_object25907:head# 2024-09-20T22:21:02.091 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.092 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdbb0062:::benchmark_data_smithi060_196073_object41933:head# 2024-09-20T22:21:02.092 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.092 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdbd0d97:::benchmark_data_smithi060_196073_object56453:head# 2024-09-20T22:21:02.093 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.094 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdbf8a7c:::benchmark_data_smithi060_196073_object55553:head# 2024-09-20T22:21:02.094 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.094 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdbfa309:::benchmark_data_smithi060_196073_object25232:head# 2024-09-20T22:21:02.094 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.095 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdc23d50:::benchmark_data_smithi060_196073_object56272:head# 2024-09-20T22:21:02.095 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.095 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdc4a961:::benchmark_data_smithi060_196073_object38926:head# 2024-09-20T22:21:02.096 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.096 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdc4b875:::benchmark_data_smithi060_196073_object55251:head# 2024-09-20T22:21:02.096 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.097 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdc61e43:::benchmark_data_smithi060_196073_object26414:head# 2024-09-20T22:21:02.097 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.097 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdc71fce:::benchmark_data_smithi060_196073_object6817:head# 2024-09-20T22:21:02.097 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.097 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdc869db:::benchmark_data_smithi060_196073_object30711:head# 2024-09-20T22:21:02.097 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.098 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdc8ccc7:::benchmark_data_smithi060_196073_object54211:head# 2024-09-20T22:21:02.098 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.098 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdca7309:::benchmark_data_smithi060_196073_object42739:head# 2024-09-20T22:21:02.098 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.099 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdcab6c2:::benchmark_data_smithi060_196073_object15374:head# 2024-09-20T22:21:02.099 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.099 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdcb48d8:::benchmark_data_smithi060_196073_object30402:head# 2024-09-20T22:21:02.099 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.099 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdcdd5bb:::benchmark_data_smithi060_196073_object16003:head# 2024-09-20T22:21:02.099 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.100 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdcddce6:::benchmark_data_smithi060_196073_object37670:head# 2024-09-20T22:21:02.100 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.100 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdceb11d:::benchmark_data_smithi060_196073_object42347:head# 2024-09-20T22:21:02.100 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.100 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdceed79:::benchmark_data_smithi060_196073_object22941:head# 2024-09-20T22:21:02.100 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.101 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdcf1b77:::benchmark_data_smithi060_196073_object8193:head# 2024-09-20T22:21:02.101 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.101 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdcfc366:::benchmark_data_smithi060_196073_object33886:head# 2024-09-20T22:21:02.102 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.102 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdcfd970:::benchmark_data_smithi060_196073_object56510:head# 2024-09-20T22:21:02.102 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.102 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdd02d4a:::benchmark_data_smithi060_196073_object34031:head# 2024-09-20T22:21:02.102 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.103 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdd0807b:::benchmark_data_smithi060_196073_object50628:head# 2024-09-20T22:21:02.103 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.103 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdd0f835:::benchmark_data_smithi060_196073_object3239:head# 2024-09-20T22:21:02.103 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.103 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdd2e0b5:::benchmark_data_smithi060_196073_object11563:head# 2024-09-20T22:21:02.103 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.103 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdd35b27:::benchmark_data_smithi060_196073_object44138:head# 2024-09-20T22:21:02.103 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.104 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdd416b8:::benchmark_data_smithi060_196073_object60934:head# 2024-09-20T22:21:02.104 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.104 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdd70877:::benchmark_data_smithi060_196073_object60483:head# 2024-09-20T22:21:02.104 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.104 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdd954ed:::benchmark_data_smithi060_196073_object13178:head# 2024-09-20T22:21:02.105 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.105 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdd98d4f:::benchmark_data_smithi060_196073_object39656:head# 2024-09-20T22:21:02.105 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.106 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdd9b01e:::benchmark_data_smithi060_196073_object36771:head# 2024-09-20T22:21:02.106 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.106 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdd9da18:::benchmark_data_smithi060_196073_object48850:head# 2024-09-20T22:21:02.106 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.107 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdda05d0:::benchmark_data_smithi060_196073_object32527:head# 2024-09-20T22:21:02.107 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.107 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdda701f:::benchmark_data_smithi060_196073_object16270:head# 2024-09-20T22:21:02.107 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.107 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bddca708:::benchmark_data_smithi060_196073_object5938:head# 2024-09-20T22:21:02.107 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.107 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdddae5f:::benchmark_data_smithi060_196073_object5013:head# 2024-09-20T22:21:02.108 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.109 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bddddd22:::benchmark_data_smithi060_196073_object37224:head# 2024-09-20T22:21:02.109 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.109 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdde8196:::benchmark_data_smithi060_196073_object55329:head# 2024-09-20T22:21:02.109 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.109 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdded7df:::benchmark_data_smithi060_196073_object28087:head# 2024-09-20T22:21:02.109 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.109 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bde000c6:::benchmark_data_smithi060_196073_object19296:head# 2024-09-20T22:21:02.109 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.109 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:02.111 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bde2ed49:::benchmark_data_smithi060_196073_object24706:head# 2024-09-20T22:21:02.111 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.111 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bde34ec5:::benchmark_data_smithi060_196073_object48745:head# 2024-09-20T22:21:02.111 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.111 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bde46db3:::benchmark_data_smithi060_196073_object6887:head# 2024-09-20T22:21:02.111 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.111 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bde689a5:::benchmark_data_smithi060_196073_object58734:head# 2024-09-20T22:21:02.111 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.112 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bde7588a:::benchmark_data_smithi060_196073_object12571:head# 2024-09-20T22:21:02.112 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.113 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bde7c2b5:::benchmark_data_smithi060_196073_object39099:head# 2024-09-20T22:21:02.113 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.113 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bde87c01:::benchmark_data_smithi060_196073_object60675:head# 2024-09-20T22:21:02.113 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.113 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bde9d5fd:::benchmark_data_smithi060_196073_object39142:head# 2024-09-20T22:21:02.114 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.114 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdea5c9f:::benchmark_data_smithi060_196073_object22385:head# 2024-09-20T22:21:02.114 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.114 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdeb0fdb:::benchmark_data_smithi060_196073_object49660:head# 2024-09-20T22:21:02.115 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.115 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdebeaa5:::benchmark_data_smithi060_196073_object38727:head# 2024-09-20T22:21:02.115 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.115 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdebed55:::benchmark_data_smithi060_196073_object16623:head# 2024-09-20T22:21:02.115 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.115 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdefadf9:::benchmark_data_smithi060_196073_object41604:head# 2024-09-20T22:21:02.115 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.115 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdf04a9a:::benchmark_data_smithi060_196073_object9597:head# 2024-09-20T22:21:02.115 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.116 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdf0b340:::benchmark_data_smithi060_196073_object44636:head# 2024-09-20T22:21:02.116 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.116 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdf23047:::benchmark_data_smithi060_196073_object24259:head# 2024-09-20T22:21:02.116 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.116 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdf26d81:::benchmark_data_smithi060_196073_object4997:head# 2024-09-20T22:21:02.116 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.117 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdf797c1:::benchmark_data_smithi060_196073_object37068:head# 2024-09-20T22:21:02.117 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.117 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdf7a79d:::benchmark_data_smithi060_196073_object60037:head# 2024-09-20T22:21:02.117 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.118 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdf95a32:::benchmark_data_smithi060_196073_object49914:head# 2024-09-20T22:21:02.118 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.118 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdfdec3e:::benchmark_data_smithi060_196073_object57400:head# 2024-09-20T22:21:02.118 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.118 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdfe5738:::benchmark_data_smithi060_196073_object8964:head# 2024-09-20T22:21:02.118 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.119 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdfefddc:::benchmark_data_smithi060_196073_object49847:head# 2024-09-20T22:21:02.119 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.124 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bdffea0a:::benchmark_data_smithi060_196073_object26455:head# 2024-09-20T22:21:02.124 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.125 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be004767:::benchmark_data_smithi060_196073_object61662:head# 2024-09-20T22:21:02.125 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.125 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be01590c:::benchmark_data_smithi060_196073_object60524:head# 2024-09-20T22:21:02.125 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.125 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be041687:::benchmark_data_smithi060_196073_object18078:head# 2024-09-20T22:21:02.125 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.125 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be067520:::benchmark_data_smithi060_196073_object54160:head# 2024-09-20T22:21:02.125 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.125 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be070508:::benchmark_data_smithi060_196073_object9035:head# 2024-09-20T22:21:02.125 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.125 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be07aaa4:::benchmark_data_smithi060_196073_object53461:head# 2024-09-20T22:21:02.125 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.126 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be07aae6:::benchmark_data_smithi060_196073_object22177:head# 2024-09-20T22:21:02.126 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.126 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be0bf805:::benchmark_data_smithi060_196073_object46298:head# 2024-09-20T22:21:02.126 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.126 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be0c64d4:::benchmark_data_smithi060_196073_object23746:head# 2024-09-20T22:21:02.126 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.126 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be0e72c3:::benchmark_data_smithi060_196073_object11414:head# 2024-09-20T22:21:02.126 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.128 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be0eb517:::benchmark_data_smithi060_196073_object42326:head# 2024-09-20T22:21:02.128 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.128 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be107ad0:::benchmark_data_smithi060_196073_object13219:head# 2024-09-20T22:21:02.128 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.128 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be10bc8f:::benchmark_data_smithi060_196073_object11105:head# 2024-09-20T22:21:02.128 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.128 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be10f57b:::benchmark_data_smithi060_196073_object23241:head# 2024-09-20T22:21:02.128 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.128 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be113470:::benchmark_data_smithi060_196073_object49440:head# 2024-09-20T22:21:02.128 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.128 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be1190f5:::benchmark_data_smithi060_196073_object45950:head# 2024-09-20T22:21:02.128 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.129 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be11aada:::benchmark_data_smithi060_196073_object31736:head# 2024-09-20T22:21:02.129 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.129 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be13fb2b:::benchmark_data_smithi060_196073_object32550:head# 2024-09-20T22:21:02.129 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.129 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be152065:::benchmark_data_smithi060_196073_object55843:head# 2024-09-20T22:21:02.131 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.131 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be1599cf:::benchmark_data_smithi060_196073_object23718:head# 2024-09-20T22:21:02.131 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.131 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be18a531:::benchmark_data_smithi060_196073_object23841:head# 2024-09-20T22:21:02.131 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.131 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be18a835:::benchmark_data_smithi060_196073_object23817:head# 2024-09-20T22:21:02.131 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.131 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be195d0b:::benchmark_data_smithi060_196073_object30480:head# 2024-09-20T22:21:02.131 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.132 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be197797:::benchmark_data_smithi060_196073_object42580:head# 2024-09-20T22:21:02.132 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.132 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be19aa03:::benchmark_data_smithi060_196073_object18641:head# 2024-09-20T22:21:02.132 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.132 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be1a04fb:::benchmark_data_smithi060_196073_object28103:head# 2024-09-20T22:21:02.132 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.132 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be1a0b07:::benchmark_data_smithi060_196073_object16638:head# 2024-09-20T22:21:02.133 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.133 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be1a980f:::benchmark_data_smithi060_196073_object7154:head# 2024-09-20T22:21:02.133 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.133 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be1aeb81:::benchmark_data_smithi060_196073_object15356:head# 2024-09-20T22:21:02.133 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.134 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be1c0054:::benchmark_data_smithi060_196073_object46250:head# 2024-09-20T22:21:02.134 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.134 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be1c5e96:::benchmark_data_smithi060_196073_object9205:head# 2024-09-20T22:21:02.134 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.134 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be1cba5c:::benchmark_data_smithi060_196073_object17479:head# 2024-09-20T22:21:02.134 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.135 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be1dadcf:::benchmark_data_smithi060_196073_object34500:head# 2024-09-20T22:21:02.135 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.135 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be1e9a0b:::benchmark_data_smithi060_196073_object34507:head# 2024-09-20T22:21:02.135 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.136 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be1f6b23:::benchmark_data_smithi060_196073_object48543:head# 2024-09-20T22:21:02.136 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.136 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be204be9:::benchmark_data_smithi060_196073_object32332:head# 2024-09-20T22:21:02.136 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.136 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be214e25:::benchmark_data_smithi060_196073_object41461:head# 2024-09-20T22:21:02.137 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.137 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be236804:::benchmark_data_smithi060_196073_object60716:head# 2024-09-20T22:21:02.137 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.137 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be242f2b:::benchmark_data_smithi060_196073_object23678:head# 2024-09-20T22:21:02.138 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.138 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be24ad02:::benchmark_data_smithi060_196073_object5411:head# 2024-09-20T22:21:02.138 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.139 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be276441:::benchmark_data_smithi060_196073_object57719:head# 2024-09-20T22:21:02.139 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.139 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be28ec54:::benchmark_data_smithi060_196073_object52510:head# 2024-09-20T22:21:02.139 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.140 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be292ddc:::benchmark_data_smithi060_196073_object45154:head# 2024-09-20T22:21:02.140 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.140 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be29ffa3:::benchmark_data_smithi060_196073_object49237:head# 2024-09-20T22:21:02.140 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.141 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be2c4c89:::benchmark_data_smithi060_196073_object55243:head# 2024-09-20T22:21:02.141 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.141 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be2d32f7:::benchmark_data_smithi060_196073_object47255:head# 2024-09-20T22:21:02.141 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be2d3e92:::benchmark_data_smithi060_196073_object10169:head# 2024-09-20T22:21:02.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be2efcf1:::benchmark_data_smithi060_196073_object48133:head# 2024-09-20T22:21:02.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be2fa4b3:::benchmark_data_smithi060_196073_object16501:head# 2024-09-20T22:21:02.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be31c9b5:::benchmark_data_smithi060_196073_object29399:head# 2024-09-20T22:21:02.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.143 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be32e1e1:::benchmark_data_smithi060_196073_object57165:head# 2024-09-20T22:21:02.143 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.144 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be332ebe:::benchmark_data_smithi060_196073_object60900:head# 2024-09-20T22:21:02.144 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.144 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be33c15c:::benchmark_data_smithi060_196073_object57585:head# 2024-09-20T22:21:02.144 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.145 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be344d1a:::benchmark_data_smithi060_196073_object42606:head# 2024-09-20T22:21:02.145 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.146 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be35f1ed:::benchmark_data_smithi060_196073_object43775:head# 2024-09-20T22:21:02.146 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.146 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be35fa1e:::benchmark_data_smithi060_196073_object15729:head# 2024-09-20T22:21:02.146 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.147 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be36ad18:::benchmark_data_smithi060_196073_object26362:head# 2024-09-20T22:21:02.147 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.147 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be36bf8a:::benchmark_data_smithi060_196073_object35032:head# 2024-09-20T22:21:02.147 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.147 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be380a1c:::benchmark_data_smithi060_196073_object37095:head# 2024-09-20T22:21:02.147 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.149 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be3a268f:::benchmark_data_smithi060_196073_object9224:head# 2024-09-20T22:21:02.149 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.149 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be3acb11:::benchmark_data_smithi060_196073_object52436:head# 2024-09-20T22:21:02.149 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.149 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be3b76b6:::benchmark_data_smithi060_196073_object17062:head# 2024-09-20T22:21:02.149 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.149 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be3bb13c:::benchmark_data_smithi060_196073_object37421:head# 2024-09-20T22:21:02.149 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.150 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be3f03dc:::benchmark_data_smithi060_196073_object53334:head# 2024-09-20T22:21:02.150 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.150 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be408886:::benchmark_data_smithi060_196073_object24206:head# 2024-09-20T22:21:02.150 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.150 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be40cc2f:::benchmark_data_smithi060_196073_object2800:head# 2024-09-20T22:21:02.153 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.153 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be411bf4:::benchmark_data_smithi060_196073_object56998:head# 2024-09-20T22:21:02.153 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.154 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be41bc93:::benchmark_data_smithi060_196073_object24584:head# 2024-09-20T22:21:02.154 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.154 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be425abc:::benchmark_data_smithi060_196073_object41995:head# 2024-09-20T22:21:02.154 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.154 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be42816d:::benchmark_data_smithi060_196073_object1480:head# 2024-09-20T22:21:02.154 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.154 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be442848:::benchmark_data_smithi060_196073_object5395:head# 2024-09-20T22:21:02.154 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.154 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be44ab80:::benchmark_data_smithi060_196073_object36079:head# 2024-09-20T22:21:02.154 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.154 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be454f8c:::benchmark_data_smithi060_196073_object43845:head# 2024-09-20T22:21:02.155 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.155 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be457cfe:::benchmark_data_smithi060_196073_object48024:head# 2024-09-20T22:21:02.155 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.155 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be47df79:::benchmark_data_smithi060_196073_object11973:head# 2024-09-20T22:21:02.155 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.156 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be4ab74b:::benchmark_data_smithi060_196073_object16407:head# 2024-09-20T22:21:02.156 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.156 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be4f1613:::benchmark_data_smithi060_196073_object60994:head# 2024-09-20T22:21:02.156 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.156 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be4fdc4e:::benchmark_data_smithi060_196073_object56995:head# 2024-09-20T22:21:02.156 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.157 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be506461:::benchmark_data_smithi060_196073_object53972:head# 2024-09-20T22:21:02.157 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.158 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be50b151:::benchmark_data_smithi060_196073_object24074:head# 2024-09-20T22:21:02.158 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.158 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be512950:::benchmark_data_smithi060_196073_object51953:head# 2024-09-20T22:21:02.158 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.158 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be5313a9:::benchmark_data_smithi060_196073_object18736:head# 2024-09-20T22:21:02.159 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.159 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be53403f:::benchmark_data_smithi060_196073_object44093:head# 2024-09-20T22:21:02.159 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.159 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be56c0d8:::benchmark_data_smithi060_196073_object35766:head# 2024-09-20T22:21:02.159 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.160 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be573652:::benchmark_data_smithi060_196073_object54304:head# 2024-09-20T22:21:02.160 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.160 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be573a33:::benchmark_data_smithi060_196073_object1157:head# 2024-09-20T22:21:02.160 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.160 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be57fad5:::benchmark_data_smithi060_196073_object15395:head# 2024-09-20T22:21:02.161 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.161 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be58ff0d:::benchmark_data_smithi060_196073_object37000:head# 2024-09-20T22:21:02.161 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.161 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be5b449c:::benchmark_data_smithi060_196073_object36580:head# 2024-09-20T22:21:02.161 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.162 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be5c180c:::benchmark_data_smithi060_196073_object53135:head# 2024-09-20T22:21:02.162 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.162 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be5c64c1:::benchmark_data_smithi060_196073_object57714:head# 2024-09-20T22:21:02.162 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.163 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be5ded44:::benchmark_data_smithi060_196073_object8293:head# 2024-09-20T22:21:02.163 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.163 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be5e9bc5:::benchmark_data_smithi060_196073_object44929:head# 2024-09-20T22:21:02.163 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.164 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be5f1fff:::benchmark_data_smithi060_196073_object25848:head# 2024-09-20T22:21:02.164 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.164 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be613b9b:::benchmark_data_smithi060_196073_object56449:head# 2024-09-20T22:21:02.164 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.165 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be618f62:::benchmark_data_smithi060_196073_object4194:head# 2024-09-20T22:21:02.165 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.166 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be626a90:::benchmark_data_smithi060_196073_object17915:head# 2024-09-20T22:21:02.166 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.166 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be627406:::benchmark_data_smithi060_196073_object40707:head# 2024-09-20T22:21:02.166 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.166 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be633d9e:::benchmark_data_smithi060_196073_object46695:head# 2024-09-20T22:21:02.166 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.167 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be63e820:::benchmark_data_smithi060_196073_object49498:head# 2024-09-20T22:21:02.167 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.167 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be6410f7:::benchmark_data_smithi060_196073_object16113:head# 2024-09-20T22:21:02.167 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.168 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be6438ac:::benchmark_data_smithi060_196073_object45629:head# 2024-09-20T22:21:02.168 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.168 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be6639ba:::benchmark_data_smithi060_196073_object20832:head# 2024-09-20T22:21:02.168 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.168 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be678962:::benchmark_data_smithi060_196073_object18004:head# 2024-09-20T22:21:02.168 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.169 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be6866f2:::benchmark_data_smithi060_196073_object47340:head# 2024-09-20T22:21:02.169 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.169 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be69496d:::benchmark_data_smithi060_196073_object565:head# 2024-09-20T22:21:02.169 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.170 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be696027:::benchmark_data_smithi060_196073_object32155:head# 2024-09-20T22:21:02.170 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.170 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be6988f0:::benchmark_data_smithi060_196073_object3129:head# 2024-09-20T22:21:02.170 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.171 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be6ae7c0:::benchmark_data_smithi060_196073_object18183:head# 2024-09-20T22:21:02.171 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.171 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be6b4994:::benchmark_data_smithi060_196073_object20364:head# 2024-09-20T22:21:02.171 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.172 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be6cbe66:::benchmark_data_smithi060_196073_object34205:head# 2024-09-20T22:21:02.172 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.172 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be70a7c2:::benchmark_data_smithi060_196073_object29068:head# 2024-09-20T22:21:02.172 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.173 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be70c76a:::benchmark_data_smithi060_196073_object58082:head# 2024-09-20T22:21:02.173 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.174 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be713d49:::benchmark_data_smithi060_196073_object40135:head# 2024-09-20T22:21:02.174 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.174 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be72837b:::benchmark_data_smithi060_196073_object29750:head# 2024-09-20T22:21:02.174 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.175 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be73c203:::benchmark_data_smithi060_196073_object44184:head# 2024-09-20T22:21:02.175 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.175 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be7402e6:::benchmark_data_smithi060_196073_object21204:head# 2024-09-20T22:21:02.175 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.176 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be75a4cd:::benchmark_data_smithi060_196073_object1567:head# 2024-09-20T22:21:02.176 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.176 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be7650e6:::benchmark_data_smithi060_196073_object355:head# 2024-09-20T22:21:02.176 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.176 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be79ae1d:::benchmark_data_smithi060_196073_object37382:head# 2024-09-20T22:21:02.176 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.177 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be7c7199:::benchmark_data_smithi060_196073_object36604:head# 2024-09-20T22:21:02.177 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.177 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be7ce226:::benchmark_data_smithi060_196073_object13976:head# 2024-09-20T22:21:02.177 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.177 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be7d23de:::benchmark_data_smithi060_196073_object20897:head# 2024-09-20T22:21:02.178 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.178 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be7dcef4:::benchmark_data_smithi060_196073_object8678:head# 2024-09-20T22:21:02.178 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.178 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be7e0d0b:::benchmark_data_smithi060_196073_object56330:head# 2024-09-20T22:21:02.178 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.179 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be7f541a:::benchmark_data_smithi060_196073_object29097:head# 2024-09-20T22:21:02.179 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.179 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be80c8da:::benchmark_data_smithi060_196073_object48245:head# 2024-09-20T22:21:02.179 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.179 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be8195c0:::benchmark_data_smithi060_196073_object14676:head# 2024-09-20T22:21:02.179 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.179 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be82a0f8:::benchmark_data_smithi060_196073_object33688:head# 2024-09-20T22:21:02.179 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.180 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be8526d1:::benchmark_data_smithi060_196073_object10583:head# 2024-09-20T22:21:02.180 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.180 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be852901:::benchmark_data_smithi060_196073_object42968:head# 2024-09-20T22:21:02.182 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.182 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be853394:::benchmark_data_smithi060_196073_object56464:head# 2024-09-20T22:21:02.182 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.183 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be85fa69:::benchmark_data_smithi060_196073_object8265:head# 2024-09-20T22:21:02.183 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.183 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be8606f1:::benchmark_data_smithi060_196073_object46235:head# 2024-09-20T22:21:02.183 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.184 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be879d81:::benchmark_data_smithi060_196073_object42764:head# 2024-09-20T22:21:02.184 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.184 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be886fc5:::benchmark_data_smithi060_196073_object25643:head# 2024-09-20T22:21:02.184 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.185 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be8ad4a7:::benchmark_data_smithi060_196073_object7191:head# 2024-09-20T22:21:02.185 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:02.186 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:02.187 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:02.187 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be8b5304:::benchmark_data_smithi060_196073_object29703:head# 2024-09-20T22:21:02.187 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.188 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be8e5b9c:::benchmark_data_smithi060_196073_object47763:head# 2024-09-20T22:21:02.188 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.188 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be8f0310:::benchmark_data_smithi060_196073_object42327:head# 2024-09-20T22:21:02.188 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.188 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be8f450f:::benchmark_data_smithi060_196073_object51731:head# 2024-09-20T22:21:02.189 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.189 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be8f6327:::benchmark_data_smithi060_196073_object6914:head# 2024-09-20T22:21:02.189 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.189 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be918bd8:::benchmark_data_smithi060_196073_object34978:head# 2024-09-20T22:21:02.189 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.189 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be9206ba:::benchmark_data_smithi060_196073_object13029:head# 2024-09-20T22:21:02.189 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.190 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be92bb18:::benchmark_data_smithi060_196073_object11472:head# 2024-09-20T22:21:02.190 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.190 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be959dda:::benchmark_data_smithi060_196073_object47538:head# 2024-09-20T22:21:02.190 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.190 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be96cc5a:::benchmark_data_smithi060_196073_object41214:head# 2024-09-20T22:21:02.288 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:02.316 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:02.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:02.317 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:02.348 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:21:02.417 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:02.455 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:02.455 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:02.456 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:02.460 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.460 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be9772b2:::benchmark_data_smithi060_196073_object34795:head# 2024-09-20T22:21:02.460 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.461 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be9846a9:::benchmark_data_smithi060_196073_object2742:head# 2024-09-20T22:21:02.461 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.461 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be987cf6:::benchmark_data_smithi060_196073_object53743:head# 2024-09-20T22:21:02.461 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.462 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be98e603:::benchmark_data_smithi060_196073_object21851:head# 2024-09-20T22:21:02.485 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.486 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be9cd1ae:::benchmark_data_smithi060_196073_object31669:head# 2024-09-20T22:21:02.486 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.486 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be9d44f8:::benchmark_data_smithi060_196073_object25385:head# 2024-09-20T22:21:02.486 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.487 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:be9d7fd0:::benchmark_data_smithi060_196073_object57715:head# 2024-09-20T22:21:02.487 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.487 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bea1ecbf:::benchmark_data_smithi060_196073_object21416:head# 2024-09-20T22:21:02.524 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.524 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bea32f36:::benchmark_data_smithi060_196073_object2827:head# 2024-09-20T22:21:02.524 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.524 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bea46bfe:::benchmark_data_smithi060_196073_object19689:head# 2024-09-20T22:21:02.524 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.525 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bea4e2a9:::benchmark_data_smithi060_196073_object41470:head# 2024-09-20T22:21:02.525 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.525 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bea7f86b:::benchmark_data_smithi060_196073_object22790:head# 2024-09-20T22:21:02.544 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.544 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bea8006e:::benchmark_data_smithi060_196073_object35406:head# 2024-09-20T22:21:02.544 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.545 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bea9840e:::benchmark_data_smithi060_196073_object28915:head# 2024-09-20T22:21:02.545 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.545 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bea986fe:::benchmark_data_smithi060_196073_object30447:head# 2024-09-20T22:21:02.545 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.545 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beaa03f2:::benchmark_data_smithi060_196073_object35169:head# 2024-09-20T22:21:02.557 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:02.568 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.568 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beac048a:::benchmark_data_smithi060_196073_object19604:head# 2024-09-20T22:21:02.568 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.569 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beac4a1a:::benchmark_data_smithi060_196073_object45273:head# 2024-09-20T22:21:02.569 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.569 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beac9877:::benchmark_data_smithi060_196073_object38017:head# 2024-09-20T22:21:02.569 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.569 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beacb3b8:::benchmark_data_smithi060_196073_object19408:head# 2024-09-20T22:21:02.586 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.586 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beacbea1:::benchmark_data_smithi060_196073_object47484:head# 2024-09-20T22:21:02.586 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.587 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beae72ea:::benchmark_data_smithi060_196073_object48415:head# 2024-09-20T22:21:02.587 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.588 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beaedcf6:::benchmark_data_smithi060_196073_object19902:head# 2024-09-20T22:21:02.588 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.588 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beb00a29:::benchmark_data_smithi060_196073_object60848:head# 2024-09-20T22:21:02.589 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:02.589 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:02.589 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:02.612 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.612 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beb03bb0:::benchmark_data_smithi060_196073_object29712:head# 2024-09-20T22:21:02.612 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.613 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beb105b1:::benchmark_data_smithi060_196073_object45510:head# 2024-09-20T22:21:02.613 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.613 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beb1969b:::benchmark_data_smithi060_196073_object1713:head# 2024-09-20T22:21:02.614 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.614 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beb2268f:::benchmark_data_smithi060_196073_object21979:head# 2024-09-20T22:21:02.630 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.631 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beb2a764:::benchmark_data_smithi060_196073_object51856:head# 2024-09-20T22:21:02.631 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.632 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beb2a89b:::benchmark_data_smithi060_196073_object34864:head# 2024-09-20T22:21:02.632 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.632 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beb409ba:::benchmark_data_smithi060_196073_object13595:head# 2024-09-20T22:21:02.632 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.632 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beb4e744:::benchmark_data_smithi060_196073_object4177:head# 2024-09-20T22:21:02.633 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.633 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beb658d6:::benchmark_data_smithi060_196073_object27586:head# 2024-09-20T22:21:02.670 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.670 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beb74a5d:::benchmark_data_smithi060_196073_object42388:head# 2024-09-20T22:21:02.671 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.671 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beb95c92:::benchmark_data_smithi060_196073_object39772:head# 2024-09-20T22:21:02.671 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.671 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beb967c5:::benchmark_data_smithi060_196073_object54306:head# 2024-09-20T22:21:02.672 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.672 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bebcd17f:::benchmark_data_smithi060_196073_object25873:head# 2024-09-20T22:21:02.683 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.684 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bebcfad6:::benchmark_data_smithi060_196073_object23717:head# 2024-09-20T22:21:02.684 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.684 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bebd4533:::benchmark_data_smithi060_196073_object18998:head# 2024-09-20T22:21:02.685 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.685 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bebdb1eb:::benchmark_data_smithi060_196073_object47531:head# 2024-09-20T22:21:02.685 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.685 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bebe504a:::benchmark_data_smithi060_196073_object19491:head# 2024-09-20T22:21:02.685 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.685 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bebfa0b4:::benchmark_data_smithi060_196073_object56804:head# 2024-09-20T22:21:02.685 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.686 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bebfa435:::benchmark_data_smithi060_196073_object23916:head# 2024-09-20T22:21:02.686 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.686 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bec245b3:::benchmark_data_smithi060_196073_object32727:head# 2024-09-20T22:21:02.686 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.687 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bec4ceb9:::benchmark_data_smithi060_196073_object9098:head# 2024-09-20T22:21:02.690 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:02.705 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.706 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bec517fa:::benchmark_data_smithi060_196073_object33829:head# 2024-09-20T22:21:02.706 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.706 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bec5b789:::benchmark_data_smithi060_196073_object15242:head# 2024-09-20T22:21:02.706 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.706 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bec958b1:::benchmark_data_smithi060_196073_object15764:head# 2024-09-20T22:21:02.707 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.707 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:becc6417:::benchmark_data_smithi060_196073_object43484:head# 2024-09-20T22:21:02.727 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:02.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:02.728 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:02.731 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:02 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:21:02.732 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:02 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:21:02.732 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:02 smithi067 ceph-mon[33511]: osdmap e511: 8 total, 7 up, 6 in 2024-09-20T22:21:02.732 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:02 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 2041/212352 objects degraded (0.961%), 2 pgs degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-20T22:21:02.732 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:02 smithi067 ceph-mon[33511]: osdmap e512: 8 total, 7 up, 6 in 2024-09-20T22:21:02.732 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.732 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:becc8f64:::benchmark_data_smithi060_196073_object57696:head# 2024-09-20T22:21:02.732 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.732 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beccae52:::benchmark_data_smithi060_196073_object20531:head# 2024-09-20T22:21:02.733 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.733 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:becd15ac:::benchmark_data_smithi060_196073_object32347:head# 2024-09-20T22:21:02.733 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.733 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:becdb643:::benchmark_data_smithi060_196073_object370:head# 2024-09-20T22:21:02.758 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.759 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bece44ad:::benchmark_data_smithi060_196073_object46300:head# 2024-09-20T22:21:02.759 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.759 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bece6697:::benchmark_data_smithi060_196073_object4592:head# 2024-09-20T22:21:02.759 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.760 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bed0153c:::benchmark_data_smithi060_196073_object44366:head# 2024-09-20T22:21:02.760 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.760 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bed05f1d:::benchmark_data_smithi060_196073_object7280:head# 2024-09-20T22:21:02.779 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.780 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bed09857:::benchmark_data_smithi060_196073_object52460:head# 2024-09-20T22:21:02.780 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.780 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bed30272:::benchmark_data_smithi060_196073_object37556:head# 2024-09-20T22:21:02.780 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.780 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bed6108f:::benchmark_data_smithi060_196073_object7858:head# 2024-09-20T22:21:02.780 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.780 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bed6ca18:::benchmark_data_smithi060_196073_object17972:head# 2024-09-20T22:21:02.811 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.811 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bed8b7b1:::benchmark_data_smithi060_196073_object53931:head# 2024-09-20T22:21:02.811 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.812 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bedb04ad:::benchmark_data_smithi060_196073_object5312:head# 2024-09-20T22:21:02.812 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.812 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bedb115c:::benchmark_data_smithi060_196073_object48088:head# 2024-09-20T22:21:02.812 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.812 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bedbfafa:::benchmark_data_smithi060_196073_object51626:head# 2024-09-20T22:21:02.829 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:02.830 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.830 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bedc8edd:::benchmark_data_smithi060_196073_object8895:head# 2024-09-20T22:21:02.830 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.831 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bedd8d3e:::benchmark_data_smithi060_196073_object29479:head# 2024-09-20T22:21:02.831 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.831 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bedef3d0:::benchmark_data_smithi060_196073_object33497:head# 2024-09-20T22:21:02.832 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.832 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bee18ebb:::benchmark_data_smithi060_196073_object37408:head# 2024-09-20T22:21:02.864 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:02.865 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:02.865 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:02.871 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.871 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bee2babb:::benchmark_data_smithi060_196073_object60817:head# 2024-09-20T22:21:02.872 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.872 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bee540d6:::benchmark_data_smithi060_196073_object8135:head# 2024-09-20T22:21:02.872 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.872 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bee5d685:::benchmark_data_smithi060_196073_object26107:head# 2024-09-20T22:21:02.872 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.873 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bee7f822:::benchmark_data_smithi060_196073_object48169:head# 2024-09-20T22:21:02.883 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.883 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bee9c5c5:::benchmark_data_smithi060_196073_object27845:head# 2024-09-20T22:21:02.883 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.884 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bee9f912:::benchmark_data_smithi060_196073_object17680:head# 2024-09-20T22:21:02.898 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.899 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beeacc0a:::benchmark_data_smithi060_196073_object55712:head# 2024-09-20T22:21:02.899 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.899 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beead88b:::benchmark_data_smithi060_196073_object28272:head# 2024-09-20T22:21:02.899 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.900 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beeae637:::benchmark_data_smithi060_196073_object54023:head# 2024-09-20T22:21:02.900 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.900 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beee4a94:::benchmark_data_smithi060_196073_object39002:head# 2024-09-20T22:21:02.900 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.901 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beef33ff:::benchmark_data_smithi060_196073_object45816:head# 2024-09-20T22:21:02.901 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.901 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bef007b1:::benchmark_data_smithi060_196073_object25237:head# 2024-09-20T22:21:02.901 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.901 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bef103c4:::benchmark_data_smithi060_196073_object39863:head# 2024-09-20T22:21:02.902 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.902 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bef1420a:::benchmark_data_smithi060_196073_object26783:head# 2024-09-20T22:21:02.902 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.902 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bef1ff0f:::benchmark_data_smithi060_196073_object13832:head# 2024-09-20T22:21:02.902 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.903 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bef29835:::benchmark_data_smithi060_196073_object40479:head# 2024-09-20T22:21:02.903 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.903 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bef322ef:::benchmark_data_smithi060_196073_object36775:head# 2024-09-20T22:21:02.903 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.904 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bef5b39a:::benchmark_data_smithi060_196073_object22340:head# 2024-09-20T22:21:02.904 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.904 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bef6565f:::benchmark_data_smithi060_196073_object38168:head# 2024-09-20T22:21:02.904 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.904 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bef66184:::benchmark_data_smithi060_196073_object7519:head# 2024-09-20T22:21:02.904 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.905 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bef6d1fb:::benchmark_data_smithi060_196073_object38704:head# 2024-09-20T22:21:02.905 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.905 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bef97dea:::benchmark_data_smithi060_196073_object46783:head# 2024-09-20T22:21:02.905 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.906 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bef98044:::benchmark_data_smithi060_196073_object35992:head# 2024-09-20T22:21:02.906 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.906 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bef9a2b6:::benchmark_data_smithi060_196073_object48418:head# 2024-09-20T22:21:02.906 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.907 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bef9d0c6:::benchmark_data_smithi060_196073_object41294:head# 2024-09-20T22:21:02.907 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.907 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:befa6387:::benchmark_data_smithi060_196073_object51904:head# 2024-09-20T22:21:02.907 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.907 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:befc97f2:::benchmark_data_smithi060_196073_object44403:head# 2024-09-20T22:21:02.908 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.908 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:befcc443:::benchmark_data_smithi060_196073_object20352:head# 2024-09-20T22:21:02.908 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.909 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:befcdbbd:::benchmark_data_smithi060_196073_object5641:head# 2024-09-20T22:21:02.909 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.910 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:befe8ee7:::benchmark_data_smithi060_196073_object27486:head# 2024-09-20T22:21:02.910 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.911 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beff6cf0:::benchmark_data_smithi060_196073_object47639:head# 2024-09-20T22:21:02.911 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.911 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:beff76b1:::benchmark_data_smithi060_196073_object39574:head# 2024-09-20T22:21:02.912 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.912 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf010cf7:::benchmark_data_smithi060_196073_object36947:head# 2024-09-20T22:21:02.912 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.912 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf020781:::benchmark_data_smithi060_196073_object966:head# 2024-09-20T22:21:02.913 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.913 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf020d5f:::benchmark_data_smithi060_196073_object59444:head# 2024-09-20T22:21:02.913 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.913 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf025c69:::benchmark_data_smithi060_196073_object15181:head# 2024-09-20T22:21:02.913 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.914 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf02d8b9:::benchmark_data_smithi060_196073_object33348:head# 2024-09-20T22:21:02.914 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.914 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf02fbf0:::benchmark_data_smithi060_196073_object11285:head# 2024-09-20T22:21:02.914 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.915 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf044926:::benchmark_data_smithi060_196073_object16493:head# 2024-09-20T22:21:02.915 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.915 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf0498e6:::benchmark_data_smithi060_196073_object44185:head# 2024-09-20T22:21:02.915 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.916 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf05a9b4:::benchmark_data_smithi060_196073_object53748:head# 2024-09-20T22:21:02.916 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.917 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf05dfa3:::benchmark_data_smithi060_196073_object18170:head# 2024-09-20T22:21:02.917 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.917 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf07b6e4:::benchmark_data_smithi060_196073_object53815:head# 2024-09-20T22:21:02.917 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.918 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf095d0d:::benchmark_data_smithi060_196073_object24148:head# 2024-09-20T22:21:02.918 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.918 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf0a2324:::benchmark_data_smithi060_196073_object61094:head# 2024-09-20T22:21:02.918 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.920 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf0aff8b:::benchmark_data_smithi060_196073_object1525:head# 2024-09-20T22:21:02.920 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.921 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf0b3574:::benchmark_data_smithi060_196073_object30497:head# 2024-09-20T22:21:02.921 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.921 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf0b9ef2:::benchmark_data_smithi060_196073_object45654:head# 2024-09-20T22:21:02.921 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.922 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf0bcde6:::benchmark_data_smithi060_196073_object11316:head# 2024-09-20T22:21:02.922 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.922 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf0fff43:::benchmark_data_smithi060_196073_object44829:head# 2024-09-20T22:21:02.923 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.925 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf10b093:::benchmark_data_smithi060_196073_object38756:head# 2024-09-20T22:21:02.925 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.925 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf1462f4:::benchmark_data_smithi060_196073_object41234:head# 2024-09-20T22:21:02.925 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.927 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf15a9b9:::benchmark_data_smithi060_196073_object51220:head# 2024-09-20T22:21:02.927 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.927 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf15af25:::benchmark_data_smithi060_196073_object10590:head# 2024-09-20T22:21:02.927 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.927 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf176946:::benchmark_data_smithi060_196073_object57318:head# 2024-09-20T22:21:02.928 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.928 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf184d89:::benchmark_data_smithi060_196073_object38507:head# 2024-09-20T22:21:02.928 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.928 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf190470:::benchmark_data_smithi060_196073_object1641:head# 2024-09-20T22:21:02.928 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.929 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf194eb3:::benchmark_data_smithi060_196073_object31304:head# 2024-09-20T22:21:02.929 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.929 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf19953b:::benchmark_data_smithi060_196073_object40243:head# 2024-09-20T22:21:02.929 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.930 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf1a30fc:::benchmark_data_smithi060_196073_object55696:head# 2024-09-20T22:21:02.930 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.930 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf1a388a:::benchmark_data_smithi060_196073_object11511:head# 2024-09-20T22:21:02.930 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.931 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf1a50d3:::benchmark_data_smithi060_196073_object289:head# 2024-09-20T22:21:02.931 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.931 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf1c8755:::benchmark_data_smithi060_196073_object13422:head# 2024-09-20T22:21:02.931 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.932 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf1d3326:::benchmark_data_smithi060_196073_object8095:head# 2024-09-20T22:21:02.932 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.932 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf1d9493:::benchmark_data_smithi060_196073_object38707:head# 2024-09-20T22:21:02.933 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.933 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf1eb36e:::benchmark_data_smithi060_196073_object6085:head# 2024-09-20T22:21:02.933 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.933 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf1f0ade:::benchmark_data_smithi060_196073_object46085:head# 2024-09-20T22:21:02.933 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.934 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf202acb:::benchmark_data_smithi060_196073_object35980:head# 2024-09-20T22:21:02.934 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.934 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf20d650:::benchmark_data_smithi060_196073_object51226:head# 2024-09-20T22:21:02.935 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.935 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf213518:::benchmark_data_smithi060_196073_object28394:head# 2024-09-20T22:21:02.935 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.935 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf21b02c:::benchmark_data_smithi060_196073_object55067:head# 2024-09-20T22:21:02.935 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.936 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf2294b6:::benchmark_data_smithi060_196073_object21523:head# 2024-09-20T22:21:02.936 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.936 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf22ed3c:::benchmark_data_smithi060_196073_object6163:head# 2024-09-20T22:21:02.936 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.936 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf23a22e:::benchmark_data_smithi060_196073_object25718:head# 2024-09-20T22:21:02.936 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.936 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf246aea:::benchmark_data_smithi060_196073_object3365:head# 2024-09-20T22:21:02.936 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.937 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf26eb27:::benchmark_data_smithi060_196073_object30996:head# 2024-09-20T22:21:02.938 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.938 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf274fca:::benchmark_data_smithi060_196073_object52598:head# 2024-09-20T22:21:02.938 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.938 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf281dc1:::benchmark_data_smithi060_196073_object15851:head# 2024-09-20T22:21:02.938 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.938 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf296769:::benchmark_data_smithi060_196073_object61679:head# 2024-09-20T22:21:02.938 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.939 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf2b2577:::benchmark_data_smithi060_196073_object61053:head# 2024-09-20T22:21:02.939 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.939 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf2b8d59:::benchmark_data_smithi060_196073_object5179:head# 2024-09-20T22:21:02.939 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.939 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf2cdc14:::benchmark_data_smithi060_196073_object44195:head# 2024-09-20T22:21:02.939 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.939 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf301638:::benchmark_data_smithi060_196073_object35807:head# 2024-09-20T22:21:02.941 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.942 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf31b9c9:::benchmark_data_smithi060_196073_object59627:head# 2024-09-20T22:21:02.942 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.942 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf31e4e9:::benchmark_data_smithi060_196073_object61449:head# 2024-09-20T22:21:02.942 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.943 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf32c7fb:::benchmark_data_smithi060_196073_object17281:head# 2024-09-20T22:21:02.943 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.943 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf3437b3:::benchmark_data_smithi060_196073_object38297:head# 2024-09-20T22:21:02.943 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.943 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf34ad9e:::benchmark_data_smithi060_196073_object27027:head# 2024-09-20T22:21:02.943 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.944 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf34bff8:::benchmark_data_smithi060_196073_object331:head# 2024-09-20T22:21:02.944 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.944 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf3759d8:::benchmark_data_smithi060_196073_object38380:head# 2024-09-20T22:21:02.944 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.944 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf3878dd:::benchmark_data_smithi060_196073_object25847:head# 2024-09-20T22:21:02.945 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.945 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf3a1b2e:::benchmark_data_smithi060_196073_object25369:head# 2024-09-20T22:21:02.945 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.946 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf3a71fc:::benchmark_data_smithi060_196073_object40155:head# 2024-09-20T22:21:02.946 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.946 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf3c79a9:::benchmark_data_smithi060_196073_object31806:head# 2024-09-20T22:21:02.946 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.946 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf3c8c5a:::benchmark_data_smithi060_196073_object54763:head# 2024-09-20T22:21:02.946 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.946 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf3dfa98:::benchmark_data_smithi060_196073_object5737:head# 2024-09-20T22:21:02.947 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.947 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf3ec385:::benchmark_data_smithi060_196073_object58127:head# 2024-09-20T22:21:02.947 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.947 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf3f5586:::benchmark_data_smithi060_196073_object48891:head# 2024-09-20T22:21:02.947 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.948 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf401040:::benchmark_data_smithi060_196073_object16575:head# 2024-09-20T22:21:02.948 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.948 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf4096f1:::benchmark_data_smithi060_196073_object60706:head# 2024-09-20T22:21:02.948 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.948 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf416624:::benchmark_data_smithi060_196073_object20878:head# 2024-09-20T22:21:02.948 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.948 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf440774:::benchmark_data_smithi060_196073_object46624:head# 2024-09-20T22:21:02.958 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.958 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf4542ae:::benchmark_data_smithi060_196073_object59293:head# 2024-09-20T22:21:02.958 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.959 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf45c4f3:::benchmark_data_smithi060_196073_object49571:head# 2024-09-20T22:21:02.959 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.959 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf4662b4:::benchmark_data_smithi060_196073_object17323:head# 2024-09-20T22:21:02.959 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.959 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf472cf8:::benchmark_data_smithi060_196073_object55421:head# 2024-09-20T22:21:02.959 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.960 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf4771fb:::benchmark_data_smithi060_196073_object19925:head# 2024-09-20T22:21:02.960 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.960 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf478130:::benchmark_data_smithi060_196073_object5041:head# 2024-09-20T22:21:02.960 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.961 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf48810d:::benchmark_data_smithi060_196073_object3667:head# 2024-09-20T22:21:02.961 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.962 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf48db2f:::benchmark_data_smithi060_196073_object27204:head# 2024-09-20T22:21:02.962 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.962 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf48ea05:::benchmark_data_smithi060_196073_object14590:head# 2024-09-20T22:21:02.962 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.963 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf49a485:::benchmark_data_smithi060_196073_object32875:head# 2024-09-20T22:21:02.963 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.963 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf4a27aa:::benchmark_data_smithi060_196073_object47521:head# 2024-09-20T22:21:02.963 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.963 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf4c73c6:::benchmark_data_smithi060_196073_object46961:head# 2024-09-20T22:21:02.964 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.964 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf4d7081:::benchmark_data_smithi060_196073_object58387:head# 2024-09-20T22:21:02.964 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.964 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf4edd8f:::benchmark_data_smithi060_196073_object39416:head# 2024-09-20T22:21:02.964 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.964 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf4f2712:::benchmark_data_smithi060_196073_object8682:head# 2024-09-20T22:21:02.964 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.965 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf4f8aa3:::benchmark_data_smithi060_196073_object5117:head# 2024-09-20T22:21:02.965 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.965 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf4f90a1:::benchmark_data_smithi060_196073_object26251:head# 2024-09-20T22:21:02.966 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.966 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf50a2fb:::benchmark_data_smithi060_196073_object25010:head# 2024-09-20T22:21:02.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:02.989 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.989 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:21:02 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:21:02.989 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf5100f0:::benchmark_data_smithi060_196073_object28105:head# 2024-09-20T22:21:02.989 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.990 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf5433a0:::benchmark_data_smithi060_196073_object27960:head# 2024-09-20T22:21:02.990 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.990 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf554989:::benchmark_data_smithi060_196073_object44327:head# 2024-09-20T22:21:02.991 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:02.991 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf5613d1:::benchmark_data_smithi060_196073_object36258:head# 2024-09-20T22:21:03.002 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:03.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:03.004 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:03.008 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.008 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf56c048:::benchmark_data_smithi060_196073_object46044:head# 2024-09-20T22:21:03.008 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.009 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf57064f:::benchmark_data_smithi060_196073_object12249:head# 2024-09-20T22:21:03.009 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.009 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf572251:::benchmark_data_smithi060_196073_object24548:head# 2024-09-20T22:21:03.009 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.009 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf59962c:::benchmark_data_smithi060_196073_object18196:head# 2024-09-20T22:21:03.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:02 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:21:03.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:02 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:21:03.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:02 smithi060 ceph-mon[29694]: osdmap e511: 8 total, 7 up, 6 in 2024-09-20T22:21:03.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:02 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 2041/212352 objects degraded (0.961%), 2 pgs degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-20T22:21:03.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:02 smithi060 ceph-mon[29694]: osdmap e512: 8 total, 7 up, 6 in 2024-09-20T22:21:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:02 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:21:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:02 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:21:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:02 smithi060 ceph-mon[40290]: osdmap e511: 8 total, 7 up, 6 in 2024-09-20T22:21:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:02 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 2041/212352 objects degraded (0.961%), 2 pgs degraded, 2 pgs undersized (PG_DEGRADED) 2024-09-20T22:21:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:02 smithi060 ceph-mon[40290]: osdmap e512: 8 total, 7 up, 6 in 2024-09-20T22:21:03.037 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.037 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf59b242:::benchmark_data_smithi060_196073_object38698:head# 2024-09-20T22:21:03.037 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.037 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf59e55d:::benchmark_data_smithi060_196073_object12799:head# 2024-09-20T22:21:03.037 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.038 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf5a6553:::benchmark_data_smithi060_196073_object40178:head# 2024-09-20T22:21:03.038 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.038 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf5afea2:::benchmark_data_smithi060_196073_object32062:head# 2024-09-20T22:21:03.058 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.058 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf5b5485:::benchmark_data_smithi060_196073_object55619:head# 2024-09-20T22:21:03.059 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.059 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf5c2216:::benchmark_data_smithi060_196073_object38293:head# 2024-09-20T22:21:03.061 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.061 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf5e9f93:::benchmark_data_smithi060_196073_object52979:head# 2024-09-20T22:21:03.061 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.062 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf5eedd9:::benchmark_data_smithi060_196073_object52255:head# 2024-09-20T22:21:03.105 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:03.106 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.106 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf63668f:::benchmark_data_smithi060_196073_object5802:head# 2024-09-20T22:21:03.106 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.106 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf63869c:::benchmark_data_smithi060_196073_object53348:head# 2024-09-20T22:21:03.107 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.108 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf660b0a:::benchmark_data_smithi060_196073_object16520:head# 2024-09-20T22:21:03.108 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.108 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf678344:::benchmark_data_smithi060_196073_object19539:head# 2024-09-20T22:21:03.119 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.120 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf679fd3:::benchmark_data_smithi060_196073_object18798:head# 2024-09-20T22:21:03.120 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.121 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf68016f:::benchmark_data_smithi060_196073_object53434:head# 2024-09-20T22:21:03.121 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.121 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf683e6b:::benchmark_data_smithi060_196073_object49395:head# 2024-09-20T22:21:03.121 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.121 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf6c2e0b:::benchmark_data_smithi060_196073_object21791:head# 2024-09-20T22:21:03.147 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:03.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:03.148 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:03.175 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.176 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf6d822f:::benchmark_data_smithi060_196073_object19176:head# 2024-09-20T22:21:03.176 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.176 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf73196a:::benchmark_data_smithi060_196073_object21452:head# 2024-09-20T22:21:03.176 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.177 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf73ac9c:::benchmark_data_smithi060_196073_object25719:head# 2024-09-20T22:21:03.177 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.177 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf7480ec:::benchmark_data_smithi060_196073_object45793:head# 2024-09-20T22:21:03.246 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.246 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf751b63:::benchmark_data_smithi060_196073_object40646:head# 2024-09-20T22:21:03.246 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.247 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf7647e9:::benchmark_data_smithi060_196073_object30719:head# 2024-09-20T22:21:03.247 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.249 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:03.249 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf76a800:::benchmark_data_smithi060_196073_object56525:head# 2024-09-20T22:21:03.249 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.249 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf78b14a:::benchmark_data_smithi060_196073_object47000:head# 2024-09-20T22:21:03.260 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.261 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf78d4af:::benchmark_data_smithi060_196073_object42098:head# 2024-09-20T22:21:03.261 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.261 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf79bfee:::benchmark_data_smithi060_196073_object40265:head# 2024-09-20T22:21:03.261 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.261 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf7b42f5:::benchmark_data_smithi060_196073_object56884:head# 2024-09-20T22:21:03.261 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.262 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf7bf830:::benchmark_data_smithi060_196073_object40648:head# 2024-09-20T22:21:03.267 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.267 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf7ca3fc:::benchmark_data_smithi060_196073_object14141:head# 2024-09-20T22:21:03.267 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.268 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf7d2d39:::benchmark_data_smithi060_196073_object59994:head# 2024-09-20T22:21:03.268 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.268 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf7d6c93:::benchmark_data_smithi060_196073_object59050:head# 2024-09-20T22:21:03.268 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.268 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf7ef371:::benchmark_data_smithi060_196073_object16756:head# 2024-09-20T22:21:03.270 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.270 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf806a40:::benchmark_data_smithi060_196073_object45036:head# 2024-09-20T22:21:03.270 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.271 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf853c32:::benchmark_data_smithi060_196073_object37513:head# 2024-09-20T22:21:03.271 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.271 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf857c77:::benchmark_data_smithi060_196073_object45071:head# 2024-09-20T22:21:03.271 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.271 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf859aa2:::benchmark_data_smithi060_196073_object42113:head# 2024-09-20T22:21:03.276 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.277 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf874434:::benchmark_data_smithi060_196073_object30543:head# 2024-09-20T22:21:03.277 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.277 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf8779cf:::benchmark_data_smithi060_196073_object45558:head# 2024-09-20T22:21:03.277 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.278 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf887612:::benchmark_data_smithi060_196073_object47558:head# 2024-09-20T22:21:03.278 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.278 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf889676:::benchmark_data_smithi060_196073_object4944:head# 2024-09-20T22:21:03.282 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.283 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf88b4ef:::benchmark_data_smithi060_196073_object22897:head# 2024-09-20T22:21:03.283 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.283 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf8906d9:::benchmark_data_smithi060_196073_object21044:head# 2024-09-20T22:21:03.283 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.283 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf8a678e:::benchmark_data_smithi060_196073_object7642:head# 2024-09-20T22:21:03.284 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.284 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf8a9b6b:::benchmark_data_smithi060_196073_object11950:head# 2024-09-20T22:21:03.287 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:03.288 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:03.288 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:03.292 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.292 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf8b461f:::benchmark_data_smithi060_196073_object10685:head# 2024-09-20T22:21:03.292 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.292 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf8c37d2:::benchmark_data_smithi060_196073_object56814:head# 2024-09-20T22:21:03.293 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.293 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf8f29da:::benchmark_data_smithi060_196073_object25507:head# 2024-09-20T22:21:03.293 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.293 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf907269:::benchmark_data_smithi060_196073_object34429:head# 2024-09-20T22:21:03.302 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.302 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf9092c4:::benchmark_data_smithi060_196073_object34464:head# 2024-09-20T22:21:03.302 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.302 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf9113a8:::benchmark_data_smithi060_196073_object26619:head# 2024-09-20T22:21:03.302 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.303 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf91bbc5:::benchmark_data_smithi060_196073_object1055:head# 2024-09-20T22:21:03.322 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.323 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf91f4da:::benchmark_data_smithi060_196073_object7149:head# 2024-09-20T22:21:03.323 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.323 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf929820:::benchmark_data_smithi060_196073_object44442:head# 2024-09-20T22:21:03.323 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.323 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf939aeb:::benchmark_data_smithi060_196073_object11056:head# 2024-09-20T22:21:03.323 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.323 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf93ab47:::benchmark_data_smithi060_196073_object12780:head# 2024-09-20T22:21:03.371 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.372 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf9433f2:::benchmark_data_smithi060_196073_object22309:head# 2024-09-20T22:21:03.372 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.372 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf94d18a:::benchmark_data_smithi060_196073_object42286:head# 2024-09-20T22:21:03.372 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.373 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf96f16c:::benchmark_data_smithi060_196073_object61156:head# 2024-09-20T22:21:03.373 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.373 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf976e8e:::benchmark_data_smithi060_196073_object2094:head# 2024-09-20T22:21:03.389 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:03.426 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:03.427 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:03.427 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:03.528 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:03.564 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:03.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:03.565 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:03.574 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.574 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf98e3a5:::benchmark_data_smithi060_196073_object32831:head# 2024-09-20T22:21:03.574 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.574 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf9987f4:::benchmark_data_smithi060_196073_object18325:head# 2024-09-20T22:21:03.574 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.575 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf9b8a42:::benchmark_data_smithi060_196073_object54942:head# 2024-09-20T22:21:03.575 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.575 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf9c8ee1:::benchmark_data_smithi060_196073_object54650:head# 2024-09-20T22:21:03.624 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.625 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf9cde30:::benchmark_data_smithi060_196073_object40583:head# 2024-09-20T22:21:03.625 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.625 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf9e602c:::benchmark_data_smithi060_196073_object17205:head# 2024-09-20T22:21:03.625 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.625 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf9e849e:::benchmark_data_smithi060_196073_object8225:head# 2024-09-20T22:21:03.625 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.626 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf9f68ad:::benchmark_data_smithi060_196073_object10981:head# 2024-09-20T22:21:03.631 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.631 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bf9fe015:::benchmark_data_smithi060_196073_object59578:head# 2024-09-20T22:21:03.631 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.632 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfa16d36:::benchmark_data_smithi060_196073_object25293:head# 2024-09-20T22:21:03.632 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.632 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfa3c8fa:::benchmark_data_smithi060_196073_object17181:head# 2024-09-20T22:21:03.632 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.632 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfa3cb7f:::benchmark_data_smithi060_196073_object57423:head# 2024-09-20T22:21:03.640 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.641 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfa3d5f7:::benchmark_data_smithi060_196073_object57633:head# 2024-09-20T22:21:03.641 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.641 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfa5f2e2:::benchmark_data_smithi060_196073_object31010:head# 2024-09-20T22:21:03.641 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.642 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfa6c414:::benchmark_data_smithi060_196073_object7535:head# 2024-09-20T22:21:03.642 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.642 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfa6fd8a:::benchmark_data_smithi060_196073_object38912:head# 2024-09-20T22:21:03.665 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:03.703 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:03.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:03.704 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:03.745 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.745 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:03 smithi067 ceph-mon[33511]: pgmap v973: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 4.2 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 645 op/s; 1463/204618 objects degraded (0.715%); 14118/204618 objects misplaced (6.900%); 49 MiB/s, 780 objects/s recovering 2024-09-20T22:21:03.746 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:03 smithi067 ceph-mon[33511]: 6.0 deep-scrub starts 2024-09-20T22:21:03.746 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:03 smithi067 ceph-mon[33511]: 6.0 deep-scrub ok 2024-09-20T22:21:03.746 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfab6db4:::benchmark_data_smithi060_196073_object18952:head# 2024-09-20T22:21:03.746 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.746 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfabe7ba:::benchmark_data_smithi060_196073_object20638:head# 2024-09-20T22:21:03.747 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.747 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfacf6cc:::benchmark_data_smithi060_196073_object50461:head# 2024-09-20T22:21:03.747 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.747 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfad85cd:::benchmark_data_smithi060_196073_object21077:head# 2024-09-20T22:21:03.780 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.780 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfadabfd:::benchmark_data_smithi060_196073_object21294:head# 2024-09-20T22:21:03.781 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.781 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfadd9a6:::benchmark_data_smithi060_196073_object37943:head# 2024-09-20T22:21:03.781 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.781 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfade1a7:::benchmark_data_smithi060_196073_object59362:head# 2024-09-20T22:21:03.781 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.782 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfadf077:::benchmark_data_smithi060_196073_object7871:head# 2024-09-20T22:21:03.786 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.787 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfadfcb1:::benchmark_data_smithi060_196073_object9641:head# 2024-09-20T22:21:03.787 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.787 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfae0328:::benchmark_data_smithi060_196073_object32924:head# 2024-09-20T22:21:03.787 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.788 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfae54aa:::benchmark_data_smithi060_196073_object10793:head# 2024-09-20T22:21:03.788 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.788 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfaf0398:::benchmark_data_smithi060_196073_object3297:head# 2024-09-20T22:21:03.790 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.790 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfaf7bb1:::benchmark_data_smithi060_196073_object30972:head# 2024-09-20T22:21:03.790 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.791 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfb215a1:::benchmark_data_smithi060_196073_object37346:head# 2024-09-20T22:21:03.791 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.791 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfb3d92e:::benchmark_data_smithi060_196073_object25969:head# 2024-09-20T22:21:03.791 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.791 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfb4d85e:::benchmark_data_smithi060_196073_object50528:head# 2024-09-20T22:21:03.805 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:03.851 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:03.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:03.852 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:03.873 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.873 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfb5da1b:::benchmark_data_smithi060_196073_object55518:head# 2024-09-20T22:21:03.873 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.873 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfb5e50d:::benchmark_data_smithi060_196073_object19764:head# 2024-09-20T22:21:03.873 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.874 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfb72842:::benchmark_data_smithi060_196073_object10413:head# 2024-09-20T22:21:03.874 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.874 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfb76b3b:::benchmark_data_smithi060_196073_object43602:head# 2024-09-20T22:21:03.916 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.917 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfb9223d:::benchmark_data_smithi060_196073_object44173:head# 2024-09-20T22:21:03.917 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.917 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfb95d1d:::benchmark_data_smithi060_196073_object50164:head# 2024-09-20T22:21:03.917 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.918 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfb985ac:::benchmark_data_smithi060_196073_object45423:head# 2024-09-20T22:21:03.918 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.918 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfb99ad4:::benchmark_data_smithi060_196073_object60462:head# 2024-09-20T22:21:03.925 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.925 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfb9ba20:::benchmark_data_smithi060_196073_object10823:head# 2024-09-20T22:21:03.925 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.926 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfbb326b:::benchmark_data_smithi060_196073_object24900:head# 2024-09-20T22:21:03.926 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.926 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfbdacae:::benchmark_data_smithi060_196073_object48239:head# 2024-09-20T22:21:03.926 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.927 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfbebdd3:::benchmark_data_smithi060_196073_object4438:head# 2024-09-20T22:21:03.931 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.931 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfc29be8:::benchmark_data_smithi060_196073_object38951:head# 2024-09-20T22:21:03.931 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.932 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfc49771:::benchmark_data_smithi060_196073_object41612:head# 2024-09-20T22:21:03.932 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.932 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfc4af2f:::benchmark_data_smithi060_196073_object13298:head# 2024-09-20T22:21:03.932 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:03.933 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfc4d5dc:::benchmark_data_smithi060_196073_object51887:head# 2024-09-20T22:21:03.952 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:03.986 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:03.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:03.987 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:04.007 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.007 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfc4eac8:::benchmark_data_smithi060_196073_object14677:head# 2024-09-20T22:21:04.008 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.008 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfc5050a:::benchmark_data_smithi060_196073_object5:head# 2024-09-20T22:21:04.008 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.008 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfc53d38:::benchmark_data_smithi060_196073_object49288:head# 2024-09-20T22:21:04.008 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.009 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfc5f746:::benchmark_data_smithi060_196073_object36854:head# 2024-09-20T22:21:04.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:03 smithi060 ceph-mon[29694]: pgmap v973: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 4.2 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 645 op/s; 1463/204618 objects degraded (0.715%); 14118/204618 objects misplaced (6.900%); 49 MiB/s, 780 objects/s recovering 2024-09-20T22:21:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:03 smithi060 ceph-mon[29694]: 6.0 deep-scrub starts 2024-09-20T22:21:04.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:03 smithi060 ceph-mon[29694]: 6.0 deep-scrub ok 2024-09-20T22:21:04.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:03 smithi060 ceph-mon[40290]: pgmap v973: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 4.2 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1023 B/s rd, 0 B/s wr, 645 op/s; 1463/204618 objects degraded (0.715%); 14118/204618 objects misplaced (6.900%); 49 MiB/s, 780 objects/s recovering 2024-09-20T22:21:04.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:03 smithi060 ceph-mon[40290]: 6.0 deep-scrub starts 2024-09-20T22:21:04.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:03 smithi060 ceph-mon[40290]: 6.0 deep-scrub ok 2024-09-20T22:21:04.050 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:21:04.050 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:21:04.050 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:21:04.050 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:21:04.050 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.050 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:125848 9.6 9:604ae09f:::benchmark_data_smithi060_140132_object15040:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:21:04.050 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:13:30.323134+0000", 2024-09-20T22:21:04.051 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 453.072696635, 2024-09-20T22:21:04.051 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.39324260900000002, 2024-09-20T22:21:04.051 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.051 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.051 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.051 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.051 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:21:04.051 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:21:04.051 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 125848 2024-09-20T22:21:04.051 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.051 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.051 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.051 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.051 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323134+0000", 2024-09-20T22:21:04.052 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.052 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.052 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323134+0000", 2024-09-20T22:21:04.052 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.052 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.052 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323135+0000", 2024-09-20T22:21:04.052 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1399999999999997e-07 2024-09-20T22:21:04.052 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.052 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.052 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.053 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323137+0000", 2024-09-20T22:21:04.053 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1789999999999998e-06 2024-09-20T22:21:04.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.053 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.053 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.053 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323143+0000", 2024-09-20T22:21:04.053 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1469999999999998e-06 2024-09-20T22:21:04.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.053 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.053 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.053 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323149+0000", 2024-09-20T22:21:04.053 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2639999999999997e-06 2024-09-20T22:21:04.053 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.053 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.053 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.053 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323210+0000", 2024-09-20T22:21:04.054 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0537e-05 2024-09-20T22:21:04.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.054 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.054 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.054 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323333+0000", 2024-09-20T22:21:04.054 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012330500000000001 2024-09-20T22:21:04.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.054 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.054 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:21:04.054 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.323444+0000", 2024-09-20T22:21:04.054 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000110562 2024-09-20T22:21:04.054 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.054 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.054 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.054 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.326284+0000", 2024-09-20T22:21:04.055 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0028407409999999999 2024-09-20T22:21:04.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.055 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.055 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.328793+0000", 2024-09-20T22:21:04.055 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0025082680000000001 2024-09-20T22:21:04.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.055 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.055 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.716294+0000", 2024-09-20T22:21:04.055 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.38750077100000002 2024-09-20T22:21:04.055 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.055 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.056 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.056 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.716331+0000", 2024-09-20T22:21:04.056 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7419999999999997e-05 2024-09-20T22:21:04.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.056 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.056 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.056 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.716377+0000", 2024-09-20T22:21:04.056 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6001000000000002e-05 2024-09-20T22:21:04.056 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.056 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.056 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.056 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.056 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.057 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:125853 9.6 9:65bf9cb5:::benchmark_data_smithi060_140132_object15045:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:21:04.057 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:13:30.329387+0000", 2024-09-20T22:21:04.057 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 453.066443972, 2024-09-20T22:21:04.057 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.39097777700000003, 2024-09-20T22:21:04.057 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.057 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.057 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.057 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.057 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:21:04.057 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:21:04.057 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 125853 2024-09-20T22:21:04.057 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.057 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.057 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.058 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.058 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329387+0000", 2024-09-20T22:21:04.058 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.058 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.058 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.058 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329387+0000", 2024-09-20T22:21:04.058 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.058 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.058 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.058 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329388+0000", 2024-09-20T22:21:04.058 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3990000000000001e-06 2024-09-20T22:21:04.058 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.058 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.059 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.059 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329395+0000", 2024-09-20T22:21:04.059 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.161e-06 2024-09-20T22:21:04.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.059 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.059 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.059 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329407+0000", 2024-09-20T22:21:04.059 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2323999999999999e-05 2024-09-20T22:21:04.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.059 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.059 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.059 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329414+0000", 2024-09-20T22:21:04.059 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.289e-06 2024-09-20T22:21:04.059 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.059 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.060 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.060 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329463+0000", 2024-09-20T22:21:04.060 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8658000000000001e-05 2024-09-20T22:21:04.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.060 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.060 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329589+0000", 2024-09-20T22:21:04.060 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012579900000000001 2024-09-20T22:21:04.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.060 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:21:04.060 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.329695+0000", 2024-09-20T22:21:04.060 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000105933 2024-09-20T22:21:04.060 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.060 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.060 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.061 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.335795+0000", 2024-09-20T22:21:04.061 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061008720000000002 2024-09-20T22:21:04.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.061 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.061 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.335858+0000", 2024-09-20T22:21:04.061 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2285000000000004e-05 2024-09-20T22:21:04.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.061 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.061 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720262+0000", 2024-09-20T22:21:04.061 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.38440416300000002 2024-09-20T22:21:04.061 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.061 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.061 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.062 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720315+0000", 2024-09-20T22:21:04.062 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2985000000000002e-05 2024-09-20T22:21:04.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.062 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.062 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.062 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720365+0000", 2024-09-20T22:21:04.062 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9908999999999999e-05 2024-09-20T22:21:04.062 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.062 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.062 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.062 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.062 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.062 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.15594.0:125877 9.6 9:618bafd1:::benchmark_data_smithi060_140132_object15069:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:21:04.062 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:21:04.062 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 453.05074239599998, 2024-09-20T22:21:04.062 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.37546249399999998, 2024-09-20T22:21:04.063 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.063 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.063 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.063 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.063 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.15594", 2024-09-20T22:21:04.063 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:21:04.063 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 125877 2024-09-20T22:21:04.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.063 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.063 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.063 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.063 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:21:04.063 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.063 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.063 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.064 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.064 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:21:04.064 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.064 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.064 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.064 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345089+0000", 2024-09-20T22:21:04.064 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1999999999999999e-07 2024-09-20T22:21:04.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.064 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.064 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.064 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345092+0000", 2024-09-20T22:21:04.064 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2899999999999998e-06 2024-09-20T22:21:04.064 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.065 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.065 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.065 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345101+0000", 2024-09-20T22:21:04.065 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1550000000000007e-06 2024-09-20T22:21:04.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.065 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.065 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.065 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345107+0000", 2024-09-20T22:21:04.065 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2609999999999999e-06 2024-09-20T22:21:04.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.065 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.065 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.065 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345160+0000", 2024-09-20T22:21:04.065 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.287e-05 2024-09-20T22:21:04.065 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.066 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.066 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.066 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345269+0000", 2024-09-20T22:21:04.066 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000108795 2024-09-20T22:21:04.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.066 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.066 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 1,2", 2024-09-20T22:21:04.066 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.345395+0000", 2024-09-20T22:21:04.066 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012654800000000001 2024-09-20T22:21:04.066 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.066 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.066 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.066 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.348668+0000", 2024-09-20T22:21:04.067 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0032731729999999999 2024-09-20T22:21:04.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.067 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.067 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.351790+0000", 2024-09-20T22:21:04.067 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003121567 2024-09-20T22:21:04.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.067 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.067 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720464+0000", 2024-09-20T22:21:04.067 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.36867422 2024-09-20T22:21:04.067 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.067 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.067 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.067 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720511+0000", 2024-09-20T22:21:04.068 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7076999999999997e-05 2024-09-20T22:21:04.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.068 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.068 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:13:30.720551+0000", 2024-09-20T22:21:04.068 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9918e-05 2024-09-20T22:21:04.068 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.068 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.068 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.068 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.068 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.068 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:233503 10.6 10:6fc58bf5:::benchmark_data_smithi060_167766_object108711:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e445)", 2024-09-20T22:21:04.068 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:18:20.606346+0000", 2024-09-20T22:21:04.068 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 162.78948539999999, 2024-09-20T22:21:04.068 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.29572936999999999, 2024-09-20T22:21:04.068 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.069 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.069 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.069 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.069 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:21:04.069 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:21:04.069 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 233503 2024-09-20T22:21:04.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.069 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.069 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.069 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.069 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.606346+0000", 2024-09-20T22:21:04.069 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.069 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.069 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.069 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.070 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.606346+0000", 2024-09-20T22:21:04.070 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.070 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.070 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.070 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.606347+0000", 2024-09-20T22:21:04.070 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0589999999999999e-06 2024-09-20T22:21:04.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.070 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.070 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.070 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.606351+0000", 2024-09-20T22:21:04.070 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6709999999999998e-06 2024-09-20T22:21:04.070 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.070 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.070 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.071 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.606362+0000", 2024-09-20T22:21:04.071 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0873000000000001e-05 2024-09-20T22:21:04.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.071 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.071 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.071 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.606367+0000", 2024-09-20T22:21:04.071 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2889999999999997e-06 2024-09-20T22:21:04.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.071 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.071 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.071 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.606429+0000", 2024-09-20T22:21:04.071 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1105999999999994e-05 2024-09-20T22:21:04.071 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.071 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.071 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.071 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.606517+0000", 2024-09-20T22:21:04.072 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8004999999999995e-05 2024-09-20T22:21:04.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.072 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.072 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:21:04.072 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.606582+0000", 2024-09-20T22:21:04.072 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5461000000000002e-05 2024-09-20T22:21:04.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.072 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.072 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.072 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.611900+0000", 2024-09-20T22:21:04.072 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0053178720000000004 2024-09-20T22:21:04.072 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.072 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.072 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.073 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.612499+0000", 2024-09-20T22:21:04.073 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00059942700000000003 2024-09-20T22:21:04.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.073 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.073 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.073 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.902014+0000", 2024-09-20T22:21:04.073 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.289514575 2024-09-20T22:21:04.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.073 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.073 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.073 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.902047+0000", 2024-09-20T22:21:04.073 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.3392999999999997e-05 2024-09-20T22:21:04.073 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.073 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.073 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.073 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.902075+0000", 2024-09-20T22:21:04.073 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.7639000000000001e-05 2024-09-20T22:21:04.074 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.074 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.074 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.074 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.074 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.074 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16005.0:233507 10.6 10:66b48a90:::benchmark_data_smithi060_167766_object108715:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e445)", 2024-09-20T22:21:04.074 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:18:20.607017+0000", 2024-09-20T22:21:04.074 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 162.78881420499999, 2024-09-20T22:21:04.074 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.29609884600000003, 2024-09-20T22:21:04.074 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.074 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.074 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.074 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.074 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16005", 2024-09-20T22:21:04.074 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:21:04.075 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 233507 2024-09-20T22:21:04.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.075 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.075 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.075 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.075 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.607017+0000", 2024-09-20T22:21:04.075 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.075 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.075 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.075 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.607017+0000", 2024-09-20T22:21:04.075 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.075 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.075 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.075 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.075 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.607018+0000", 2024-09-20T22:21:04.076 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1999999999999998e-07 2024-09-20T22:21:04.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.076 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.076 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.607022+0000", 2024-09-20T22:21:04.076 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3000000000000003e-06 2024-09-20T22:21:04.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.076 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.076 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.607031+0000", 2024-09-20T22:21:04.076 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.0890000000000006e-06 2024-09-20T22:21:04.076 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.076 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.076 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.076 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.607038+0000", 2024-09-20T22:21:04.076 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7170000000000002e-06 2024-09-20T22:21:04.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.077 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.077 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.077 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.609062+0000", 2024-09-20T22:21:04.077 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0020244199999999999 2024-09-20T22:21:04.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.077 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.077 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.077 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.609166+0000", 2024-09-20T22:21:04.077 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010374099999999999 2024-09-20T22:21:04.077 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.077 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.077 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:21:04.077 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.609271+0000", 2024-09-20T22:21:04.078 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010552699999999999 2024-09-20T22:21:04.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.078 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.078 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.078 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.612012+0000", 2024-09-20T22:21:04.078 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0027403499999999999 2024-09-20T22:21:04.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.078 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.078 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.078 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.612630+0000", 2024-09-20T22:21:04.078 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00061808299999999996 2024-09-20T22:21:04.078 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.078 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.078 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.078 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.902994+0000", 2024-09-20T22:21:04.079 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.29036447700000001 2024-09-20T22:21:04.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.079 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.079 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.079 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.903072+0000", 2024-09-20T22:21:04.079 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7293999999999996e-05 2024-09-20T22:21:04.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.079 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.079 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.079 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:18:20.903116+0000", 2024-09-20T22:21:04.079 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4027999999999999e-05 2024-09-20T22:21:04.079 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.079 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.079 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.079 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.080 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.080 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:67518 11.7 11:efc9e62d:::benchmark_data_smithi060_196073_object67517:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e477)", 2024-09-20T22:21:04.080 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:19:36.928344+0000", 2024-09-20T22:21:04.080 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 86.467487254999995, 2024-09-20T22:21:04.081 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.332924575, 2024-09-20T22:21:04.081 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.081 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.081 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.081 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.082 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:04.082 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:04.082 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 67518 2024-09-20T22:21:04.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.082 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.082 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.082 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.082 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.928344+0000", 2024-09-20T22:21:04.082 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.082 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.082 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.082 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.082 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.928344+0000", 2024-09-20T22:21:04.083 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.083 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.083 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.083 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.928345+0000", 2024-09-20T22:21:04.083 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.545e-06 2024-09-20T22:21:04.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.083 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.083 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.083 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.928404+0000", 2024-09-20T22:21:04.083 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8783999999999998e-05 2024-09-20T22:21:04.083 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.083 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.083 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.083 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.928415+0000", 2024-09-20T22:21:04.084 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1293e-05 2024-09-20T22:21:04.084 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.084 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.084 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.084 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.928421+0000", 2024-09-20T22:21:04.084 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4589999999999997e-06 2024-09-20T22:21:04.084 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.084 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.084 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.084 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.933263+0000", 2024-09-20T22:21:04.084 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0048422420000000001 2024-09-20T22:21:04.084 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.084 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.084 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.084 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.933500+0000", 2024-09-20T22:21:04.084 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000237187 2024-09-20T22:21:04.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.085 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1", 2024-09-20T22:21:04.085 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.933710+0000", 2024-09-20T22:21:04.085 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020951300000000001 2024-09-20T22:21:04.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.085 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.085 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.937797+0000", 2024-09-20T22:21:04.085 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0040871739999999998 2024-09-20T22:21:04.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.085 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.085 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.085 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.944024+0000", 2024-09-20T22:21:04.085 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0062266129999999998 2024-09-20T22:21:04.085 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.086 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.086 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:37.258545+0000", 2024-09-20T22:21:04.086 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.31452115600000002 2024-09-20T22:21:04.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.086 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.086 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:37.258614+0000", 2024-09-20T22:21:04.086 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9338000000000006e-05 2024-09-20T22:21:04.086 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.086 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.086 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.086 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:37.261268+0000", 2024-09-20T22:21:04.086 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0026542710000000001 2024-09-20T22:21:04.086 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.086 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.087 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.087 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.087 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:67526 11.7 11:e73efc58:::benchmark_data_smithi060_196073_object67525:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e477)", 2024-09-20T22:21:04.087 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:19:36.936462+0000", 2024-09-20T22:21:04.087 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 86.459369277999997, 2024-09-20T22:21:04.087 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.32737844300000002, 2024-09-20T22:21:04.087 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.087 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.087 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.087 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.087 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:04.087 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:04.087 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 67526 2024-09-20T22:21:04.087 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.087 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.088 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.088 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.088 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.936462+0000", 2024-09-20T22:21:04.088 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.088 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.088 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.088 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.936462+0000", 2024-09-20T22:21:04.088 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.088 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.088 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.088 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.936463+0000", 2024-09-20T22:21:04.088 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.322e-06 2024-09-20T22:21:04.088 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.088 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.088 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.089 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.936533+0000", 2024-09-20T22:21:04.089 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9552000000000001e-05 2024-09-20T22:21:04.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.089 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.089 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.089 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.936543+0000", 2024-09-20T22:21:04.089 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0457e-05 2024-09-20T22:21:04.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.089 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.089 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.089 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.936549+0000", 2024-09-20T22:21:04.089 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2389999999999999e-06 2024-09-20T22:21:04.089 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.089 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.089 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.090 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.937830+0000", 2024-09-20T22:21:04.090 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0012804019999999999 2024-09-20T22:21:04.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.090 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.090 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.090 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.937997+0000", 2024-09-20T22:21:04.090 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00016702700000000001 2024-09-20T22:21:04.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.090 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.090 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1", 2024-09-20T22:21:04.090 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.938201+0000", 2024-09-20T22:21:04.090 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020390800000000001 2024-09-20T22:21:04.090 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.090 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.090 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.091 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.940221+0000", 2024-09-20T22:21:04.091 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0020204540000000001 2024-09-20T22:21:04.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.091 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.091 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.091 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.944058+0000", 2024-09-20T22:21:04.091 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003837053 2024-09-20T22:21:04.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.091 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.091 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.091 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:37.261511+0000", 2024-09-20T22:21:04.091 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.31745284899999998 2024-09-20T22:21:04.091 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.091 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.091 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.091 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:37.261592+0000", 2024-09-20T22:21:04.092 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1188999999999998e-05 2024-09-20T22:21:04.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.092 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.092 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.092 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:37.263840+0000", 2024-09-20T22:21:04.092 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002247991 2024-09-20T22:21:04.092 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.092 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.092 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.092 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.092 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.092 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:67528 11.7 11:e8fedcb8:::benchmark_data_smithi060_196073_object67527:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e477)", 2024-09-20T22:21:04.092 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:19:36.940953+0000", 2024-09-20T22:21:04.092 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 86.454878171999994, 2024-09-20T22:21:04.092 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.324511734, 2024-09-20T22:21:04.092 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.093 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.093 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.093 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.093 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:04.093 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:04.093 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 67528 2024-09-20T22:21:04.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.093 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.093 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.093 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.093 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.940953+0000", 2024-09-20T22:21:04.093 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.093 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.093 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.093 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.094 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.940953+0000", 2024-09-20T22:21:04.094 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.094 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.094 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.094 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.940954+0000", 2024-09-20T22:21:04.094 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0550000000000001e-06 2024-09-20T22:21:04.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.094 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.094 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.094 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.941000+0000", 2024-09-20T22:21:04.094 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5711000000000003e-05 2024-09-20T22:21:04.094 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.094 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.095 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.095 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.941012+0000", 2024-09-20T22:21:04.095 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2306e-05 2024-09-20T22:21:04.095 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.095 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.095 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.095 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.941021+0000", 2024-09-20T22:21:04.095 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9560000000000003e-06 2024-09-20T22:21:04.095 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.095 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.095 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.095 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.941066+0000", 2024-09-20T22:21:04.095 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.545e-05 2024-09-20T22:21:04.095 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.095 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.095 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.096 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.941180+0000", 2024-09-20T22:21:04.096 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011374699999999999 2024-09-20T22:21:04.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.096 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.096 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1", 2024-09-20T22:21:04.096 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.941281+0000", 2024-09-20T22:21:04.096 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100925 2024-09-20T22:21:04.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.096 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.096 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.096 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.943189+0000", 2024-09-20T22:21:04.096 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001908009 2024-09-20T22:21:04.096 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.096 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.096 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.097 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.953270+0000", 2024-09-20T22:21:04.097 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.010081099 2024-09-20T22:21:04.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.097 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.097 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.097 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:37.263902+0000", 2024-09-20T22:21:04.097 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.31063216199999999 2024-09-20T22:21:04.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.097 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.097 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.097 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:37.263944+0000", 2024-09-20T22:21:04.097 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2163999999999998e-05 2024-09-20T22:21:04.097 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.097 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.097 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.097 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:37.265465+0000", 2024-09-20T22:21:04.098 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0015201500000000001 2024-09-20T22:21:04.098 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.098 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.098 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.098 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.098 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.098 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:67533 11.7 11:e09c8fea:::benchmark_data_smithi060_196073_object67532:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e477)", 2024-09-20T22:21:04.098 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:19:36.944510+0000", 2024-09-20T22:21:04.098 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 86.451320914999997, 2024-09-20T22:21:04.098 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.32241007500000002, 2024-09-20T22:21:04.098 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.098 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.098 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.098 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.099 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:04.099 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:04.099 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 67533 2024-09-20T22:21:04.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.099 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.099 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.099 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.099 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.944510+0000", 2024-09-20T22:21:04.099 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.099 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.099 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.099 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.944510+0000", 2024-09-20T22:21:04.099 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.099 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.100 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.100 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.100 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.944511+0000", 2024-09-20T22:21:04.100 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.446e-06 2024-09-20T22:21:04.100 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.100 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.100 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.100 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.944579+0000", 2024-09-20T22:21:04.100 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7633999999999997e-05 2024-09-20T22:21:04.100 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.100 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.100 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.100 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.944595+0000", 2024-09-20T22:21:04.100 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5784e-05 2024-09-20T22:21:04.100 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.100 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.101 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.101 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.944602+0000", 2024-09-20T22:21:04.101 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4930000000000001e-06 2024-09-20T22:21:04.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.101 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.101 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.101 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.944675+0000", 2024-09-20T22:21:04.101 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2640999999999998e-05 2024-09-20T22:21:04.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.101 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.101 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.101 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.944862+0000", 2024-09-20T22:21:04.101 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000186653 2024-09-20T22:21:04.101 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.101 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.101 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1", 2024-09-20T22:21:04.102 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.945012+0000", 2024-09-20T22:21:04.102 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015008099999999999 2024-09-20T22:21:04.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.102 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.102 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.102 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.947360+0000", 2024-09-20T22:21:04.102 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0023479500000000001 2024-09-20T22:21:04.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.102 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.102 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.102 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.953316+0000", 2024-09-20T22:21:04.102 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0059566089999999999 2024-09-20T22:21:04.102 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.102 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.102 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.103 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:37.265525+0000", 2024-09-20T22:21:04.103 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.31220835400000002 2024-09-20T22:21:04.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.103 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.103 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.103 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:37.265561+0000", 2024-09-20T22:21:04.103 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6739000000000002e-05 2024-09-20T22:21:04.103 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.103 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.103 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.103 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:37.266920+0000", 2024-09-20T22:21:04.103 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001358691 2024-09-20T22:21:04.103 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.103 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.104 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.104 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.104 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.104 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:67539 11.7 11:e8c624ac:::benchmark_data_smithi060_196073_object67538:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e477)", 2024-09-20T22:21:04.104 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:19:36.949798+0000", 2024-09-20T22:21:04.104 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 86.446033291999996, 2024-09-20T22:21:04.104 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.318132572, 2024-09-20T22:21:04.104 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.104 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.104 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.104 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.104 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:04.104 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:04.104 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 67539 2024-09-20T22:21:04.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.105 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.105 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.105 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.105 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.949798+0000", 2024-09-20T22:21:04.105 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.105 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.105 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.105 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.949798+0000", 2024-09-20T22:21:04.105 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.105 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.105 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.105 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.106 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.949799+0000", 2024-09-20T22:21:04.106 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7370000000000001e-06 2024-09-20T22:21:04.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.106 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.106 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.106 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.949857+0000", 2024-09-20T22:21:04.106 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7170000000000003e-05 2024-09-20T22:21:04.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.106 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.106 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.106 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.949872+0000", 2024-09-20T22:21:04.106 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5658000000000002e-05 2024-09-20T22:21:04.106 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.107 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.107 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.949880+0000", 2024-09-20T22:21:04.107 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5109999999999997e-06 2024-09-20T22:21:04.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.107 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.107 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.949978+0000", 2024-09-20T22:21:04.107 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8342000000000001e-05 2024-09-20T22:21:04.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.107 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.107 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.950141+0000", 2024-09-20T22:21:04.107 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000163392 2024-09-20T22:21:04.107 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.107 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.108 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1", 2024-09-20T22:21:04.108 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.950348+0000", 2024-09-20T22:21:04.108 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020641799999999999 2024-09-20T22:21:04.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.108 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.108 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.952389+0000", 2024-09-20T22:21:04.108 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0020406510000000001 2024-09-20T22:21:04.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.108 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.108 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:36.961474+0000", 2024-09-20T22:21:04.108 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0090851349999999994 2024-09-20T22:21:04.108 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.108 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.108 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.109 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:37.266978+0000", 2024-09-20T22:21:04.109 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.30550418699999998 2024-09-20T22:21:04.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.109 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.109 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:37.267015+0000", 2024-09-20T22:21:04.109 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7088000000000001e-05 2024-09-20T22:21:04.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.109 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.109 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.109 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:19:37.267930+0000", 2024-09-20T22:21:04.109 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00091528300000000003 2024-09-20T22:21:04.109 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.109 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.109 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.109 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.110 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:85124 11.7 11:ed9c27de:::benchmark_data_smithi060_196073_object85123:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e489)", 2024-09-20T22:21:04.110 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:20:01.709334+0000", 2024-09-20T22:21:04.110 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 61.686497398999997, 2024-09-20T22:21:04.110 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.28884689499999999, 2024-09-20T22:21:04.110 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.110 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.110 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.110 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.110 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:04.110 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:04.110 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 85124 2024-09-20T22:21:04.110 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.110 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.110 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.110 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.111 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.709334+0000", 2024-09-20T22:21:04.111 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.111 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.111 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.111 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.709334+0000", 2024-09-20T22:21:04.111 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.111 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.111 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.111 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.709335+0000", 2024-09-20T22:21:04.111 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8560000000000001e-06 2024-09-20T22:21:04.111 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.111 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.111 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.111 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.709368+0000", 2024-09-20T22:21:04.112 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2212999999999998e-05 2024-09-20T22:21:04.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.112 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.112 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.709388+0000", 2024-09-20T22:21:04.112 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.0676e-05 2024-09-20T22:21:04.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.112 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.112 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.709396+0000", 2024-09-20T22:21:04.112 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7230000000000004e-06 2024-09-20T22:21:04.112 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.112 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.112 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.112 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.721712+0000", 2024-09-20T22:21:04.112 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.012316341999999999 2024-09-20T22:21:04.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.113 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.113 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.113 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.721890+0000", 2024-09-20T22:21:04.113 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00017776399999999999 2024-09-20T22:21:04.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.113 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.113 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1,7", 2024-09-20T22:21:04.113 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.722101+0000", 2024-09-20T22:21:04.113 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000210545 2024-09-20T22:21:04.113 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.113 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.113 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.113 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.728737+0000", 2024-09-20T22:21:04.114 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0066360050000000004 2024-09-20T22:21:04.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.114 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.114 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.735147+0000", 2024-09-20T22:21:04.114 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.006410125 2024-09-20T22:21:04.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.114 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.114 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.748017+0000", 2024-09-20T22:21:04.114 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.012870402 2024-09-20T22:21:04.114 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.114 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.114 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.114 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.996085+0000", 2024-09-20T22:21:04.115 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.24806778600000001 2024-09-20T22:21:04.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.115 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.115 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.996162+0000", 2024-09-20T22:21:04.115 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6812999999999998e-05 2024-09-20T22:21:04.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.115 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.115 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.115 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.998180+0000", 2024-09-20T22:21:04.115 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0020186449999999999 2024-09-20T22:21:04.115 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.115 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.115 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.115 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.116 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.116 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:85132 11.7 11:ec64a5db:::benchmark_data_smithi060_196073_object85131:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e489)", 2024-09-20T22:21:04.116 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:20:01.717086+0000", 2024-09-20T22:21:04.116 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 61.678744879, 2024-09-20T22:21:04.116 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.28656044600000002, 2024-09-20T22:21:04.116 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.116 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.116 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.116 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.116 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:04.116 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:04.116 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 85132 2024-09-20T22:21:04.116 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.116 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.117 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.117 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.717086+0000", 2024-09-20T22:21:04.117 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.117 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.117 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.717086+0000", 2024-09-20T22:21:04.117 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.117 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.117 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.117 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.117 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.717088+0000", 2024-09-20T22:21:04.117 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6339999999999999e-06 2024-09-20T22:21:04.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.118 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.118 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.717156+0000", 2024-09-20T22:21:04.118 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7984999999999994e-05 2024-09-20T22:21:04.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.118 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.118 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.717173+0000", 2024-09-20T22:21:04.118 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7377e-05 2024-09-20T22:21:04.118 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.118 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.118 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.118 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.717180+0000", 2024-09-20T22:21:04.119 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3209999999999999e-06 2024-09-20T22:21:04.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.119 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.119 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.724018+0000", 2024-09-20T22:21:04.119 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0068374120000000002 2024-09-20T22:21:04.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.119 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.119 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.724184+0000", 2024-09-20T22:21:04.119 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00016653800000000001 2024-09-20T22:21:04.119 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.119 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.119 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1,7", 2024-09-20T22:21:04.119 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.724376+0000", 2024-09-20T22:21:04.119 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00019194000000000001 2024-09-20T22:21:04.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.120 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.120 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.120 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.731072+0000", 2024-09-20T22:21:04.120 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0066953350000000002 2024-09-20T22:21:04.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.120 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.120 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.120 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.735258+0000", 2024-09-20T22:21:04.120 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0041865339999999996 2024-09-20T22:21:04.120 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.120 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.120 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.120 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:01.748079+0000", 2024-09-20T22:21:04.120 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.012821034 2024-09-20T22:21:04.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.121 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.121 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:02.001442+0000", 2024-09-20T22:21:04.121 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.25336272199999998 2024-09-20T22:21:04.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.121 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.121 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:02.001522+0000", 2024-09-20T22:21:04.121 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9956999999999994e-05 2024-09-20T22:21:04.121 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.121 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.121 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.121 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:02.003647+0000", 2024-09-20T22:21:04.121 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0021246569999999998 2024-09-20T22:21:04.121 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.122 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.122 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.122 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.122 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.122 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:100667 11.7 11:ec8e8907:::benchmark_data_smithi060_196073_object100666:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-20T22:21:04.122 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:20:21.448880+0000", 2024-09-20T22:21:04.122 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 41.946951278999997, 2024-09-20T22:21:04.122 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.30306275999999999, 2024-09-20T22:21:04.122 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.122 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.122 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.122 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.122 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:04.122 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:04.122 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 100667 2024-09-20T22:21:04.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.123 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.123 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.123 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.448880+0000", 2024-09-20T22:21:04.123 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.123 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.123 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.448880+0000", 2024-09-20T22:21:04.123 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.123 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.123 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.123 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.123 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.448881+0000", 2024-09-20T22:21:04.123 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4869999999999999e-06 2024-09-20T22:21:04.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.124 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.448986+0000", 2024-09-20T22:21:04.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010488399999999999 2024-09-20T22:21:04.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.124 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.449000+0000", 2024-09-20T22:21:04.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4309e-05 2024-09-20T22:21:04.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.124 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.124 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.124 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.449008+0000", 2024-09-20T22:21:04.124 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3939999999999998e-06 2024-09-20T22:21:04.124 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.125 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.125 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.449087+0000", 2024-09-20T22:21:04.125 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9354e-05 2024-09-20T22:21:04.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.125 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.125 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.449240+0000", 2024-09-20T22:21:04.125 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015328799999999999 2024-09-20T22:21:04.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.125 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1,7", 2024-09-20T22:21:04.125 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.449427+0000", 2024-09-20T22:21:04.125 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000186975 2024-09-20T22:21:04.125 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.125 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.126 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.126 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.453298+0000", 2024-09-20T22:21:04.126 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00387033 2024-09-20T22:21:04.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.126 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.126 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.126 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.455541+0000", 2024-09-20T22:21:04.126 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0022428410000000002 2024-09-20T22:21:04.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.126 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.126 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.126 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.456514+0000", 2024-09-20T22:21:04.126 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00097377300000000004 2024-09-20T22:21:04.126 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.126 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.127 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.127 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.751199+0000", 2024-09-20T22:21:04.127 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.294684954 2024-09-20T22:21:04.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.127 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.127 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.751242+0000", 2024-09-20T22:21:04.127 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3220000000000003e-05 2024-09-20T22:21:04.127 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.127 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.127 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.127 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.751942+0000", 2024-09-20T22:21:04.127 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00069995099999999998 2024-09-20T22:21:04.127 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.128 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.128 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.128 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.128 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:100694 11.7 11:ecc9ebc0:::benchmark_data_smithi060_196073_object100693:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e493)", 2024-09-20T22:21:04.128 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:20:21.488768+0000", 2024-09-20T22:21:04.128 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 41.907062799000002, 2024-09-20T22:21:04.128 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.26639881700000001, 2024-09-20T22:21:04.128 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.128 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.128 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.128 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.128 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:04.128 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:04.128 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 100694 2024-09-20T22:21:04.128 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.129 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.129 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.129 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.129 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.488768+0000", 2024-09-20T22:21:04.129 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.129 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.129 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.129 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.488768+0000", 2024-09-20T22:21:04.129 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.129 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.129 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.129 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.488772+0000", 2024-09-20T22:21:04.129 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2259999999999999e-06 2024-09-20T22:21:04.129 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.130 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.130 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.488832+0000", 2024-09-20T22:21:04.130 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9534999999999998e-05 2024-09-20T22:21:04.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.130 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.130 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.488843+0000", 2024-09-20T22:21:04.130 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1567e-05 2024-09-20T22:21:04.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.130 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.130 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.130 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.488849+0000", 2024-09-20T22:21:04.130 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4090000000000002e-06 2024-09-20T22:21:04.130 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.131 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.131 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.488911+0000", 2024-09-20T22:21:04.131 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1779000000000006e-05 2024-09-20T22:21:04.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.131 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.131 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.489070+0000", 2024-09-20T22:21:04.131 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015966600000000001 2024-09-20T22:21:04.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.131 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.131 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1,7", 2024-09-20T22:21:04.131 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.489234+0000", 2024-09-20T22:21:04.131 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00016367000000000001 2024-09-20T22:21:04.131 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.132 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.132 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.491116+0000", 2024-09-20T22:21:04.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0018816200000000001 2024-09-20T22:21:04.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.132 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.132 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.491204+0000", 2024-09-20T22:21:04.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.844e-05 2024-09-20T22:21:04.132 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.132 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.132 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.132 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.491515+0000", 2024-09-20T22:21:04.132 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00031112099999999998 2024-09-20T22:21:04.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.133 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.133 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.133 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.754500+0000", 2024-09-20T22:21:04.133 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.26298439800000001 2024-09-20T22:21:04.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.133 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.133 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.133 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.754568+0000", 2024-09-20T22:21:04.133 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8571000000000005e-05 2024-09-20T22:21:04.133 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.133 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.133 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.133 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:21.755167+0000", 2024-09-20T22:21:04.133 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00059881500000000005 2024-09-20T22:21:04.133 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.134 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.134 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.134 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.134 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.134 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:122121 11.7 11:ea109812:::benchmark_data_smithi060_196073_object20134:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e502)", 2024-09-20T22:21:04.134 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:20:45.857573+0000", 2024-09-20T22:21:04.134 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 17.538257546000001, 2024-09-20T22:21:04.134 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.24906529899999999, 2024-09-20T22:21:04.134 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.134 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.134 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.134 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.134 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:04.134 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:04.134 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 122121 2024-09-20T22:21:04.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.135 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.135 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.135 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.857573+0000", 2024-09-20T22:21:04.135 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.135 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.135 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.857573+0000", 2024-09-20T22:21:04.135 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.135 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.135 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.135 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.135 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.857575+0000", 2024-09-20T22:21:04.135 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.404e-06 2024-09-20T22:21:04.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.136 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.857581+0000", 2024-09-20T22:21:04.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7599999999999999e-06 2024-09-20T22:21:04.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.136 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.857598+0000", 2024-09-20T22:21:04.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7299000000000002e-05 2024-09-20T22:21:04.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.136 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.136 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.136 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.857605+0000", 2024-09-20T22:21:04.136 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1579999999999999e-06 2024-09-20T22:21:04.136 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.137 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.137 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.137 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.857665+0000", 2024-09-20T22:21:04.137 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.0368000000000002e-05 2024-09-20T22:21:04.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.137 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.137 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.137 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.857820+0000", 2024-09-20T22:21:04.137 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000154986 2024-09-20T22:21:04.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.137 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.137 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1,7", 2024-09-20T22:21:04.137 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.857984+0000", 2024-09-20T22:21:04.137 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00016394200000000001 2024-09-20T22:21:04.137 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.138 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.138 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.138 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.859914+0000", 2024-09-20T22:21:04.138 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001929577 2024-09-20T22:21:04.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.138 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.138 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.138 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.860829+0000", 2024-09-20T22:21:04.138 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00091472099999999996 2024-09-20T22:21:04.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.138 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.138 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.138 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.864021+0000", 2024-09-20T22:21:04.138 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0031918659999999998 2024-09-20T22:21:04.138 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.139 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.139 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.139 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:46.106454+0000", 2024-09-20T22:21:04.139 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.242433495 2024-09-20T22:21:04.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.139 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.139 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.139 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:46.106520+0000", 2024-09-20T22:21:04.139 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5751000000000001e-05 2024-09-20T22:21:04.139 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.139 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.139 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.139 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:46.106639+0000", 2024-09-20T22:21:04.139 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000118972 2024-09-20T22:21:04.140 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.140 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.140 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.140 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.140 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.140 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:122132 11.7 11:e9537b25:::benchmark_data_smithi060_196073_object20145:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e502)", 2024-09-20T22:21:04.140 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:20:45.864769+0000", 2024-09-20T22:21:04.140 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 17.531061666999999, 2024-09-20T22:21:04.140 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.24211274799999999, 2024-09-20T22:21:04.140 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.140 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.140 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.140 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.140 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:04.140 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:04.140 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 122132 2024-09-20T22:21:04.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.141 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.141 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.141 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.864769+0000", 2024-09-20T22:21:04.141 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.141 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.141 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.864769+0000", 2024-09-20T22:21:04.141 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.141 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.141 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.141 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.141 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.864771+0000", 2024-09-20T22:21:04.141 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.562e-06 2024-09-20T22:21:04.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.142 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.142 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.864775+0000", 2024-09-20T22:21:04.142 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3200000000000001e-06 2024-09-20T22:21:04.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.142 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.142 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.864787+0000", 2024-09-20T22:21:04.142 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1919000000000001e-05 2024-09-20T22:21:04.142 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.142 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.142 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.142 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.864795+0000", 2024-09-20T22:21:04.143 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4440000000000002e-06 2024-09-20T22:21:04.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.143 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.143 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.864839+0000", 2024-09-20T22:21:04.143 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4851999999999997e-05 2024-09-20T22:21:04.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.143 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.143 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.864926+0000", 2024-09-20T22:21:04.143 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6396000000000004e-05 2024-09-20T22:21:04.143 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.143 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.143 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1,7", 2024-09-20T22:21:04.143 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.865070+0000", 2024-09-20T22:21:04.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014412800000000001 2024-09-20T22:21:04.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.866709+0000", 2024-09-20T22:21:04.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.001638886 2024-09-20T22:21:04.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.868937+0000", 2024-09-20T22:21:04.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0022280400000000001 2024-09-20T22:21:04.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.144 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.144 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.144 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.868975+0000", 2024-09-20T22:21:04.144 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.8133000000000002e-05 2024-09-20T22:21:04.144 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.145 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.145 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:46.106681+0000", 2024-09-20T22:21:04.145 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.23770587900000001 2024-09-20T22:21:04.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.145 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.145 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:46.106755+0000", 2024-09-20T22:21:04.145 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4468000000000006e-05 2024-09-20T22:21:04.145 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.145 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.145 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.145 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:46.106882+0000", 2024-09-20T22:21:04.145 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000126721 2024-09-20T22:21:04.145 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.146 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.146 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.146 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.146 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.146 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:122136 11.7 11:ea28ad37:::benchmark_data_smithi060_196073_object20149:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e502)", 2024-09-20T22:21:04.146 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:20:45.867129+0000", 2024-09-20T22:21:04.146 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 17.528701787999999, 2024-09-20T22:21:04.146 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.244067179, 2024-09-20T22:21:04.146 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.146 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.146 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.146 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.146 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:04.146 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:04.146 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 122136 2024-09-20T22:21:04.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.147 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.147 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.147 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.867129+0000", 2024-09-20T22:21:04.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.147 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.147 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.867129+0000", 2024-09-20T22:21:04.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.147 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.147 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.147 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.147 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.867130+0000", 2024-09-20T22:21:04.147 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.142e-06 2024-09-20T22:21:04.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.148 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.148 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.867136+0000", 2024-09-20T22:21:04.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4709999999999998e-06 2024-09-20T22:21:04.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.148 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.148 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.867147+0000", 2024-09-20T22:21:04.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0841e-05 2024-09-20T22:21:04.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.148 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.148 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.148 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.867154+0000", 2024-09-20T22:21:04.148 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2289999999999997e-06 2024-09-20T22:21:04.148 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.149 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.149 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.867219+0000", 2024-09-20T22:21:04.149 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5610000000000004e-05 2024-09-20T22:21:04.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.149 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.149 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.867339+0000", 2024-09-20T22:21:04.149 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000119815 2024-09-20T22:21:04.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.149 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1,7", 2024-09-20T22:21:04.149 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.867451+0000", 2024-09-20T22:21:04.149 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000111357 2024-09-20T22:21:04.149 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.149 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.150 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.150 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.869011+0000", 2024-09-20T22:21:04.150 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0015601020000000001 2024-09-20T22:21:04.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.150 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.150 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.869688+0000", 2024-09-20T22:21:04.150 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00067760100000000001 2024-09-20T22:21:04.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.150 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.150 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:45.869742+0000", 2024-09-20T22:21:04.150 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3862e-05 2024-09-20T22:21:04.150 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.150 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.150 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.151 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:46.111008+0000", 2024-09-20T22:21:04.151 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.24126614399999999 2024-09-20T22:21:04.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.151 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.151 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:46.111076+0000", 2024-09-20T22:21:04.151 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7241000000000002e-05 2024-09-20T22:21:04.151 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.151 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.151 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.151 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:46.111196+0000", 2024-09-20T22:21:04.151 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000120764 2024-09-20T22:21:04.151 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.152 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.152 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.152 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.152 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "MOSDPGPush(11.b 509/481 [PushOp(11:d760312d:::benchmark_data_smithi060_196073_object57539:head, version: 470'3575, data_included: [0~65536], data_size: 65536, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(11:d760312d:::benchmark_data_smithi060_196073_object57539:head@470'3575, size: 65536, copy_subset: [(0, 65536)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 65536, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-09-20T22:21:04.152 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:20:59.326345+0000", 2024-09-20T22:21:04.152 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 4.0694855949999997, 2024-09-20T22:21:04.152 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.170724349, 2024-09-20T22:21:04.152 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.152 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.152 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:21:04.152 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.152 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.152 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.152 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.326345+0000", 2024-09-20T22:21:04.153 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.153 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.153 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.153 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.326345+0000", 2024-09-20T22:21:04.153 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.153 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.153 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.153 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.326346+0000", 2024-09-20T22:21:04.153 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6800000000000001e-07 2024-09-20T22:21:04.153 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.153 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.153 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.153 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.326408+0000", 2024-09-20T22:21:04.154 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1859999999999994e-05 2024-09-20T22:21:04.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.154 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.154 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.154 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.326419+0000", 2024-09-20T22:21:04.154 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0907000000000001e-05 2024-09-20T22:21:04.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.154 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.154 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.154 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.326427+0000", 2024-09-20T22:21:04.154 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9419999999999995e-06 2024-09-20T22:21:04.154 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.154 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.154 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.154 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.326479+0000", 2024-09-20T22:21:04.154 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2398000000000002e-05 2024-09-20T22:21:04.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.155 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.155 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.155 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.326486+0000", 2024-09-20T22:21:04.155 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7429999999999996e-06 2024-09-20T22:21:04.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.155 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.155 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.155 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.497070+0000", 2024-09-20T22:21:04.155 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.170584031 2024-09-20T22:21:04.155 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.155 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.155 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.155 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.155 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.155 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:137140 11.7 11:eb946c5f:::benchmark_data_smithi060_196073_object35153:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e512)", 2024-09-20T22:21:04.156 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:01.821399+0000", 2024-09-20T22:21:04.156 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.5744315709999999, 2024-09-20T22:21:04.156 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.200976352, 2024-09-20T22:21:04.156 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.156 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.156 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.156 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.156 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:04.156 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:04.156 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 137140 2024-09-20T22:21:04.156 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.156 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.156 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.156 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.156 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.821399+0000", 2024-09-20T22:21:04.156 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.157 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.157 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.157 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.821399+0000", 2024-09-20T22:21:04.157 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.157 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.157 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.157 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.821402+0000", 2024-09-20T22:21:04.157 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 2.1339999999999998e-06 2024-09-20T22:21:04.157 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.157 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.157 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.157 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.821407+0000", 2024-09-20T22:21:04.157 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7679999999999997e-06 2024-09-20T22:21:04.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.158 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.158 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.158 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.821425+0000", 2024-09-20T22:21:04.158 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7765e-05 2024-09-20T22:21:04.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.158 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.158 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.158 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.821432+0000", 2024-09-20T22:21:04.158 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.922e-06 2024-09-20T22:21:04.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.158 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.158 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.158 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.821516+0000", 2024-09-20T22:21:04.158 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4147000000000006e-05 2024-09-20T22:21:04.158 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.159 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.159 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.821655+0000", 2024-09-20T22:21:04.159 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000139238 2024-09-20T22:21:04.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.159 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1,7", 2024-09-20T22:21:04.159 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.821798+0000", 2024-09-20T22:21:04.159 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014286100000000001 2024-09-20T22:21:04.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.159 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.159 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.828604+0000", 2024-09-20T22:21:04.159 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0068061930000000003 2024-09-20T22:21:04.159 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.159 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.160 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.160 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.828622+0000", 2024-09-20T22:21:04.160 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7557000000000001e-05 2024-09-20T22:21:04.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.160 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.160 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.160 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.829155+0000", 2024-09-20T22:21:04.160 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00053270900000000003 2024-09-20T22:21:04.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.160 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.160 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.160 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:02.022020+0000", 2024-09-20T22:21:04.160 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.19286492399999999 2024-09-20T22:21:04.160 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.160 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.161 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.161 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:02.022093+0000", 2024-09-20T22:21:04.161 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2909000000000003e-05 2024-09-20T22:21:04.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.161 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.161 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.161 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:02.022376+0000", 2024-09-20T22:21:04.161 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00028322500000000001 2024-09-20T22:21:04.161 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.161 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.161 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.161 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.161 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.161 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:137145 11.7 11:ed1a061f:::benchmark_data_smithi060_196073_object35158:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e512)", 2024-09-20T22:21:04.161 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:01.822874+0000", 2024-09-20T22:21:04.162 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.5729569730000001, 2024-09-20T22:21:04.162 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.199747692, 2024-09-20T22:21:04.162 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:04.162 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:04.162 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:04.162 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:04.162 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:04.162 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:04.162 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 137145 2024-09-20T22:21:04.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.162 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:04.162 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.162 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:04.162 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.822874+0000", 2024-09-20T22:21:04.162 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.162 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.163 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:04.163 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.822874+0000", 2024-09-20T22:21:04.163 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:04.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.163 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:04.163 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.822876+0000", 2024-09-20T22:21:04.163 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.722e-06 2024-09-20T22:21:04.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.163 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:04.163 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.822880+0000", 2024-09-20T22:21:04.163 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5000000000000001e-06 2024-09-20T22:21:04.163 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.163 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.164 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:04.164 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.822891+0000", 2024-09-20T22:21:04.164 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.111e-05 2024-09-20T22:21:04.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.164 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:04.164 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.822898+0000", 2024-09-20T22:21:04.164 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6080000000000001e-06 2024-09-20T22:21:04.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.164 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:04.164 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.826462+0000", 2024-09-20T22:21:04.164 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0035635910000000001 2024-09-20T22:21:04.164 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.164 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.164 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:04.165 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.826598+0000", 2024-09-20T22:21:04.165 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013672000000000001 2024-09-20T22:21:04.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.165 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.165 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1,7", 2024-09-20T22:21:04.165 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.826730+0000", 2024-09-20T22:21:04.165 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013212200000000001 2024-09-20T22:21:04.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.165 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.165 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.165 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.829755+0000", 2024-09-20T22:21:04.165 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0030242860000000002 2024-09-20T22:21:04.165 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.165 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.165 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.165 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.830122+0000", 2024-09-20T22:21:04.166 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00036771800000000001 2024-09-20T22:21:04.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.166 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.166 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:04.166 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:01.831015+0000", 2024-09-20T22:21:04.166 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00089259599999999995 2024-09-20T22:21:04.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.166 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.166 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:04.166 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:02.022447+0000", 2024-09-20T22:21:04.166 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.19143216900000001 2024-09-20T22:21:04.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.166 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.166 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:04.166 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:02.022507+0000", 2024-09-20T22:21:04.166 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9562000000000003e-05 2024-09-20T22:21:04.166 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:04.167 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:04.167 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:04.167 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:02.022622+0000", 2024-09-20T22:21:04.167 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000114988 2024-09-20T22:21:04.167 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.167 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.167 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.167 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:04.167 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:04.167 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:21:04.167 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:04.170 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.170 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfc797de:::benchmark_data_smithi060_196073_object22222:head# 2024-09-20T22:21:04.170 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.170 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfc7bbd5:::benchmark_data_smithi060_196073_object58046:head# 2024-09-20T22:21:04.170 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.170 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfc942a2:::benchmark_data_smithi060_196073_object19198:head# 2024-09-20T22:21:04.170 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.170 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfc972dc:::benchmark_data_smithi060_196073_object55213:head# 2024-09-20T22:21:04.170 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.171 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfc9b3de:::benchmark_data_smithi060_196073_object28955:head# 2024-09-20T22:21:04.171 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.171 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfcaad17:::benchmark_data_smithi060_196073_object11694:head# 2024-09-20T22:21:04.171 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.171 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfcae164:::benchmark_data_smithi060_196073_object35450:head# 2024-09-20T22:21:04.171 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.171 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfcb13a4:::benchmark_data_smithi060_196073_object53433:head# 2024-09-20T22:21:04.171 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.171 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfce7840:::benchmark_data_smithi060_196073_object9458:head# 2024-09-20T22:21:04.171 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.171 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfcf2adc:::benchmark_data_smithi060_196073_object57016:head# 2024-09-20T22:21:04.171 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.171 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfcffeab:::benchmark_data_smithi060_196073_object27522:head# 2024-09-20T22:21:04.171 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.171 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfd01b1c:::benchmark_data_smithi060_196073_object60588:head# 2024-09-20T22:21:04.172 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.172 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfd2505a:::benchmark_data_smithi060_196073_object28054:head# 2024-09-20T22:21:04.172 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.172 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfd293fb:::benchmark_data_smithi060_196073_object32468:head# 2024-09-20T22:21:04.172 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.172 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfd44f6a:::benchmark_data_smithi060_196073_object11400:head# 2024-09-20T22:21:04.187 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.188 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfd574d6:::benchmark_data_smithi060_196073_object21333:head# 2024-09-20T22:21:04.188 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.188 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfd60573:::benchmark_data_smithi060_196073_object17965:head# 2024-09-20T22:21:04.188 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.188 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfd6c2cd:::benchmark_data_smithi060_196073_object1044:head# 2024-09-20T22:21:04.189 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.189 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfd788a0:::benchmark_data_smithi060_196073_object40052:head# 2024-09-20T22:21:04.196 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.196 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfd7bab6:::benchmark_data_smithi060_196073_object44161:head# 2024-09-20T22:21:04.196 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.197 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfd808e8:::benchmark_data_smithi060_196073_object14972:head# 2024-09-20T22:21:04.197 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.197 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfd81573:::benchmark_data_smithi060_196073_object28098:head# 2024-09-20T22:21:04.197 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.197 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfd88be0:::benchmark_data_smithi060_196073_object36813:head# 2024-09-20T22:21:04.201 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:04.201 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.202 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:04.202 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:04.203 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfd9464c:::benchmark_data_smithi060_196073_object23949:head# 2024-09-20T22:21:04.203 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.203 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfd9f1d1:::benchmark_data_smithi060_196073_object43827:head# 2024-09-20T22:21:04.203 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.203 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfda6590:::benchmark_data_smithi060_196073_object43024:head# 2024-09-20T22:21:04.203 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.203 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfdacee9:::benchmark_data_smithi060_196073_object53193:head# 2024-09-20T22:21:04.244 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.244 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfdd98fe:::benchmark_data_smithi060_196073_object23952:head# 2024-09-20T22:21:04.244 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.245 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfdeebac:::benchmark_data_smithi060_196073_object16170:head# 2024-09-20T22:21:04.245 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.245 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfe12d8b:::benchmark_data_smithi060_196073_object19008:head# 2024-09-20T22:21:04.245 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.245 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfe16d65:::benchmark_data_smithi060_196073_object44870:head# 2024-09-20T22:21:04.303 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:04.308 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.308 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfe16f5b:::benchmark_data_smithi060_196073_object40055:head# 2024-09-20T22:21:04.308 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.309 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfe1788f:::benchmark_data_smithi060_196073_object23011:head# 2024-09-20T22:21:04.309 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.309 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfe336da:::benchmark_data_smithi060_196073_object37967:head# 2024-09-20T22:21:04.309 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.309 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfe3a11d:::benchmark_data_smithi060_196073_object2332:head# 2024-09-20T22:21:04.341 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:04.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:04.342 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:04.348 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.348 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfe4d6ca:::benchmark_data_smithi060_196073_object60435:head# 2024-09-20T22:21:04.348 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.349 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfe71b44:::benchmark_data_smithi060_196073_object17763:head# 2024-09-20T22:21:04.349 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.349 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfe7a66e:::benchmark_data_smithi060_196073_object31785:head# 2024-09-20T22:21:04.350 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.350 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfe7e69d:::benchmark_data_smithi060_196073_object147:head# 2024-09-20T22:21:04.354 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.355 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfe99375:::benchmark_data_smithi060_196073_object5300:head# 2024-09-20T22:21:04.355 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.356 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfea4e66:::benchmark_data_smithi060_196073_object17567:head# 2024-09-20T22:21:04.365 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.366 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfea5d59:::benchmark_data_smithi060_196073_object33411:head# 2024-09-20T22:21:04.366 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.366 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfeab294:::benchmark_data_smithi060_196073_object4549:head# 2024-09-20T22:21:04.442 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:04.457 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.458 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfeb0b38:::benchmark_data_smithi060_196073_object28080:head# 2024-09-20T22:21:04.458 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.458 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfebda12:::benchmark_data_smithi060_196073_object61060:head# 2024-09-20T22:21:04.458 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.459 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfedc629:::benchmark_data_smithi060_196073_object23969:head# 2024-09-20T22:21:04.459 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.459 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bfee1d25:::benchmark_data_smithi060_196073_object39079:head# 2024-09-20T22:21:04.480 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:04.480 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:04.481 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:04.493 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.494 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bff01c03:::benchmark_data_smithi060_196073_object43426:head# 2024-09-20T22:21:04.494 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.494 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bff071e2:::benchmark_data_smithi060_196073_object32909:head# 2024-09-20T22:21:04.495 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.495 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bff08449:::benchmark_data_smithi060_196073_object45408:head# 2024-09-20T22:21:04.495 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.495 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bff0a283:::benchmark_data_smithi060_196073_object15180:head# 2024-09-20T22:21:04.528 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.528 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bff30711:::benchmark_data_smithi060_196073_object51487:head# 2024-09-20T22:21:04.528 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.529 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bff5ee78:::benchmark_data_smithi060_196073_object53552:head# 2024-09-20T22:21:04.529 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.529 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bff96581:::benchmark_data_smithi060_196073_object34767:head# 2024-09-20T22:21:04.529 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.530 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bffa27f1:::benchmark_data_smithi060_196073_object57897:head# 2024-09-20T22:21:04.550 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.550 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bffa5258:::benchmark_data_smithi060_196073_object25931:head# 2024-09-20T22:21:04.550 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.551 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bffbc242:::benchmark_data_smithi060_196073_object9958:head# 2024-09-20T22:21:04.551 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.551 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bffc5a40:::benchmark_data_smithi060_196073_object18821:head# 2024-09-20T22:21:04.551 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.552 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bffc9b79:::benchmark_data_smithi060_196073_object24617:head# 2024-09-20T22:21:04.581 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:04.596 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.597 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bffd98bf:::benchmark_data_smithi060_196073_object22775:head# 2024-09-20T22:21:04.597 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.597 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bffef3c1:::benchmark_data_smithi060_196073_object50464:head# 2024-09-20T22:21:04.597 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.598 INFO:teuthology.orchestra.run.smithi067.stdout:Write #11:bffff9cb:::benchmark_data_smithi060_196073_object53164:head# 2024-09-20T22:21:04.598 INFO:teuthology.orchestra.run.smithi067.stdout:snapset 0=[]:{} 2024-09-20T22:21:04.605 INFO:teuthology.orchestra.run.smithi067.stdout:write_pg epoch 481 info 11.d( v 471'3847 (0'0,471'3847] local-lis/les=456/457 n=3847 ec=456/456 lis/c=473/456 les/c/f=474/457/0 sis=480) 2024-09-20T22:21:04.618 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:04.619 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:04.619 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:04.640 INFO:teuthology.orchestra.run.smithi067.stdout:Import successful 2024-09-20T22:21:04.720 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:04.759 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:04.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:04.760 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:04.860 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:04.893 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:04.893 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:04.894 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:04.969 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:04 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:21:04.994 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:05.022 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:05.023 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:05.023 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:05.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:04 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:21:05.035 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:04 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:21:05.124 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:05.165 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:05.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:05.166 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:05.267 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:05.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:05.301 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:05.301 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:05.402 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:05.438 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:05.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:05.439 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:05.535 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:21:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:21:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:21:05.540 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:05.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:05.588 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:05.588 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:05.689 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:05.711 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:05.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:05.714 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:05.735 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T22:21:05.814 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:05.856 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:05.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:05.857 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:05.958 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:05.999 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:06.000 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:06.000 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:05 smithi067 ceph-mon[33511]: pgmap v974: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 4.1 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 705 B/s rd, 0 B/s wr, 680 op/s; 1463/200514 objects degraded (0.730%); 14118/200514 objects misplaced (7.041%); 34 MiB/s, 537 objects/s recovering 2024-09-20T22:21:06.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:05 smithi060 ceph-mon[29694]: pgmap v974: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 4.1 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 705 B/s rd, 0 B/s wr, 680 op/s; 1463/200514 objects degraded (0.730%); 14118/200514 objects misplaced (7.041%); 34 MiB/s, 537 objects/s recovering 2024-09-20T22:21:06.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:05 smithi060 ceph-mon[40290]: pgmap v974: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 4.1 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 705 B/s rd, 0 B/s wr, 680 op/s; 1463/200514 objects degraded (0.730%); 14118/200514 objects misplaced (7.041%); 34 MiB/s, 537 objects/s recovering 2024-09-20T22:21:06.101 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:06.130 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:06.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:06.131 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:06.232 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:06.254 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:06.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:06.255 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:06.355 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:06.398 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:06.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:06.399 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:06.500 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:06.511 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/exp.11.d.4 2024-09-20T22:21:06.543 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:06.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:06.544 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:06.571 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:21:06.621 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:21:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:21:06.322+0000 7fa5382c7640 -1 mgr get_metadata_python Requested missing service osd.4 2024-09-20T22:21:06.646 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:06.689 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:06.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:06.690 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:06.716 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:06 smithi067 ceph-mon[33511]: 4.b deep-scrub starts 2024-09-20T22:21:06.716 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:06 smithi067 ceph-mon[33511]: 4.b deep-scrub ok 2024-09-20T22:21:06.716 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:21:06.716 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:21:06.792 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:06.828 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:06.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:06.829 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:06.916 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:06.929 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:06.998 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:06.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:06.999 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:07.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:06 smithi060 ceph-mon[29694]: 4.b deep-scrub starts 2024-09-20T22:21:07.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:06 smithi060 ceph-mon[29694]: 4.b deep-scrub ok 2024-09-20T22:21:07.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:21:07.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:21:07.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:06 smithi060 ceph-mon[40290]: 4.b deep-scrub starts 2024-09-20T22:21:07.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:06 smithi060 ceph-mon[40290]: 4.b deep-scrub ok 2024-09-20T22:21:07.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:21:07.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:21:07.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:07.139 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:07.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:07.140 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:07.236 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:21:07.237 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:21:07.237 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.237 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.16440.0:142751 11.9 e513/456 11:9d359c99:::benchmark_data_smithi060_196073_object40764:head v 513'9121, mlcod=513'9119)", 2024-09-20T22:21:07.237 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:07.229945+0000", 2024-09-20T22:21:07.237 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0046241750000000003, 2024-09-20T22:21:07.237 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0046566749999999999, 2024-09-20T22:21:07.237 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:07.237 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:07.237 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:21:07.237 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:07.237 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.237 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:07.237 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.229945+0000", 2024-09-20T22:21:07.237 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:07.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.238 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.238 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:07.238 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.229945+0000", 2024-09-20T22:21:07.238 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:07.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.238 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.238 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:07.238 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.229946+0000", 2024-09-20T22:21:07.238 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.4700000000000001e-07 2024-09-20T22:21:07.238 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.238 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.239 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:07.239 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.229953+0000", 2024-09-20T22:21:07.239 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.939e-06 2024-09-20T22:21:07.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.239 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.239 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:07.239 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.229971+0000", 2024-09-20T22:21:07.239 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7487999999999999e-05 2024-09-20T22:21:07.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.239 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.239 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:07.239 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.229978+0000", 2024-09-20T22:21:07.239 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6650000000000003e-06 2024-09-20T22:21:07.239 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.239 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.239 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:07.240 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.230070+0000", 2024-09-20T22:21:07.240 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1751999999999998e-05 2024-09-20T22:21:07.240 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.240 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.240 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:07.240 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.230090+0000", 2024-09-20T22:21:07.240 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.9962e-05 2024-09-20T22:21:07.240 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:07.240 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:07.240 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:07.240 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.240 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.240 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:142756 11.1 11:8079d995:::benchmark_data_smithi060_196073_object40769:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e513)", 2024-09-20T22:21:07.240 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:07.231355+0000", 2024-09-20T22:21:07.240 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0032142859999999998, 2024-09-20T22:21:07.241 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003321086, 2024-09-20T22:21:07.241 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:07.241 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:07.241 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:21:07.241 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:07.241 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:07.241 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:07.241 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 142756 2024-09-20T22:21:07.241 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.241 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:07.241 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.241 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:07.241 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.231355+0000", 2024-09-20T22:21:07.242 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:07.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.242 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.242 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:07.242 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.231355+0000", 2024-09-20T22:21:07.242 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:07.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.242 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.242 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:07.242 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.231357+0000", 2024-09-20T22:21:07.242 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2529999999999999e-06 2024-09-20T22:21:07.242 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.242 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.242 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:07.243 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.231361+0000", 2024-09-20T22:21:07.243 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7840000000000003e-06 2024-09-20T22:21:07.243 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.243 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.243 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:07.243 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.231377+0000", 2024-09-20T22:21:07.243 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5441999999999999e-05 2024-09-20T22:21:07.243 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.243 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.243 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:07.243 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.231391+0000", 2024-09-20T22:21:07.243 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4089e-05 2024-09-20T22:21:07.243 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.243 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.243 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:07.243 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.231493+0000", 2024-09-20T22:21:07.244 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010182499999999999 2024-09-20T22:21:07.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.244 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.244 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:07.244 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.231599+0000", 2024-09-20T22:21:07.244 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010656499999999999 2024-09-20T22:21:07.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.244 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.244 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:21:07.244 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.231704+0000", 2024-09-20T22:21:07.244 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000105091 2024-09-20T22:21:07.244 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:07.244 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:07.244 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:07.244 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.245 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.245 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:142757 11.3 11:cc360f3f:::benchmark_data_smithi060_196073_object40770:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e513)", 2024-09-20T22:21:07.245 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:07.231421+0000", 2024-09-20T22:21:07.245 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0031488480000000001, 2024-09-20T22:21:07.245 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003332734, 2024-09-20T22:21:07.245 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:07.245 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:07.245 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:21:07.245 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:07.245 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:07.245 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:07.245 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 142757 2024-09-20T22:21:07.245 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.245 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:07.246 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.246 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:07.246 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.231421+0000", 2024-09-20T22:21:07.246 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:07.246 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.246 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.246 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:07.246 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.231421+0000", 2024-09-20T22:21:07.246 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:07.246 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.246 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.246 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:07.246 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.231421+0000", 2024-09-20T22:21:07.247 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8100000000000003e-07 2024-09-20T22:21:07.247 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.247 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.247 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:07.247 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.231425+0000", 2024-09-20T22:21:07.247 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5659999999999998e-06 2024-09-20T22:21:07.247 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.247 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.247 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:07.247 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.231433+0000", 2024-09-20T22:21:07.247 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2800000000000003e-06 2024-09-20T22:21:07.247 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.247 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.247 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:07.248 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.231439+0000", 2024-09-20T22:21:07.248 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.3639999999999998e-06 2024-09-20T22:21:07.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.248 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.248 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:07.248 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.231496+0000", 2024-09-20T22:21:07.248 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7410000000000001e-05 2024-09-20T22:21:07.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.248 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.248 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:07.248 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.231600+0000", 2024-09-20T22:21:07.248 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010427199999999999 2024-09-20T22:21:07.248 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.248 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.249 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 5,7", 2024-09-20T22:21:07.249 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.231705+0000", 2024-09-20T22:21:07.249 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000104998 2024-09-20T22:21:07.249 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.249 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.249 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:07.249 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.234123+0000", 2024-09-20T22:21:07.249 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0024177270000000002 2024-09-20T22:21:07.249 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:07.249 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:07.249 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:07.249 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.249 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.250 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.16440.0:142755 11.5 e513/456 11:a73bfce1:::benchmark_data_smithi060_196073_object40768:head v 513'9174, mlcod=513'9172)", 2024-09-20T22:21:07.250 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:07.232426+0000", 2024-09-20T22:21:07.250 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0021431100000000002, 2024-09-20T22:21:07.250 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0024001339999999999, 2024-09-20T22:21:07.250 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:07.250 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:07.250 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:21:07.250 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:07.250 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.250 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:07.250 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.232426+0000", 2024-09-20T22:21:07.250 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:07.250 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.250 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.250 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:07.251 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.232426+0000", 2024-09-20T22:21:07.251 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:07.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.251 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.251 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:07.251 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.232427+0000", 2024-09-20T22:21:07.251 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.1900000000000002e-07 2024-09-20T22:21:07.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.251 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.251 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:07.251 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.232433+0000", 2024-09-20T22:21:07.251 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7450000000000001e-06 2024-09-20T22:21:07.251 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.251 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.251 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:07.252 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.232448+0000", 2024-09-20T22:21:07.252 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4674999999999999e-05 2024-09-20T22:21:07.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.252 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.252 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:07.252 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.232455+0000", 2024-09-20T22:21:07.252 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2250000000000003e-06 2024-09-20T22:21:07.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.252 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.252 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:07.252 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.232506+0000", 2024-09-20T22:21:07.252 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0846000000000001e-05 2024-09-20T22:21:07.252 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.252 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.252 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:07.253 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.232524+0000", 2024-09-20T22:21:07.253 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8698e-05 2024-09-20T22:21:07.253 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:07.253 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:07.253 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:07.253 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.253 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.253 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_repop(client.16440.0:142745 11.4 e513/456 11:20d52e34:::benchmark_data_smithi060_196073_object40758:head v 513'8766, mlcod=513'8762)", 2024-09-20T22:21:07.253 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:07.232477+0000", 2024-09-20T22:21:07.253 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 0.0020927570000000002, 2024-09-20T22:21:07.253 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002418688, 2024-09-20T22:21:07.253 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:07.253 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:07.254 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "started", 2024-09-20T22:21:07.254 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:07.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.254 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:07.254 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.232477+0000", 2024-09-20T22:21:07.254 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:07.254 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.254 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:07.254 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.232477+0000", 2024-09-20T22:21:07.254 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:07.254 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.254 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.254 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:07.254 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.232477+0000", 2024-09-20T22:21:07.255 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.3500000000000002e-07 2024-09-20T22:21:07.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.255 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.255 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:07.255 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.232481+0000", 2024-09-20T22:21:07.255 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.523e-06 2024-09-20T22:21:07.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.255 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.255 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:07.255 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.232488+0000", 2024-09-20T22:21:07.255 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3359999999999997e-06 2024-09-20T22:21:07.255 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.255 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.255 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:07.256 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.232493+0000", 2024-09-20T22:21:07.256 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4219999999999999e-06 2024-09-20T22:21:07.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.256 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:07.256 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.232561+0000", 2024-09-20T22:21:07.256 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.7353000000000004e-05 2024-09-20T22:21:07.256 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:07.256 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:07.256 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:07.256 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:07.232578+0000", 2024-09-20T22:21:07.256 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.7197000000000002e-05 2024-09-20T22:21:07.256 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:07.256 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:07.256 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:07.256 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:07.256 INFO:teuthology.orchestra.run.smithi067.stdout: ], 2024-09-20T22:21:07.257 INFO:teuthology.orchestra.run.smithi067.stdout: "num_ops": 5 2024-09-20T22:21:07.257 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:21:07.257 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:07.296 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:07.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:07.297 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:07.398 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:07.436 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:07.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:07.437 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:07.538 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:07.577 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:07.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:07.577 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:07.678 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:07.716 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:07.717 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:07.717 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:07.818 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:07.862 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:07.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:07.863 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:07.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:07.992 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:07 smithi067 ceph-mon[33511]: pgmap v975: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 4.0 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 813 op/s; 1463/195774 objects degraded (0.747%); 13075/195774 objects misplaced (6.679%); 36 MiB/s, 572 objects/s recovering 2024-09-20T22:21:07.992 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:07 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 1463/200514 objects degraded (0.730%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T22:21:07.992 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:07 smithi067 ceph-mon[33511]: osdmap e513: 8 total, 7 up, 6 in 2024-09-20T22:21:08.000 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:08.001 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:08.001 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:08.027 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:07 smithi060 ceph-mon[29694]: pgmap v975: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 4.0 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 813 op/s; 1463/195774 objects degraded (0.747%); 13075/195774 objects misplaced (6.679%); 36 MiB/s, 572 objects/s recovering 2024-09-20T22:21:08.027 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:07 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 1463/200514 objects degraded (0.730%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T22:21:08.027 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:07 smithi060 ceph-mon[29694]: osdmap e513: 8 total, 7 up, 6 in 2024-09-20T22:21:08.027 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:07 smithi060 ceph-mon[40290]: pgmap v975: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 4.0 GiB data, 16 GiB used, 520 GiB / 536 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 813 op/s; 1463/195774 objects degraded (0.747%); 13075/195774 objects misplaced (6.679%); 36 MiB/s, 572 objects/s recovering 2024-09-20T22:21:08.027 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:07 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 1463/200514 objects degraded (0.730%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T22:21:08.027 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:07 smithi060 ceph-mon[40290]: osdmap e513: 8 total, 7 up, 6 in 2024-09-20T22:21:08.042 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T22:21:08.102 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:08.129 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:08.130 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:08.130 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:08.231 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:08.324 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:08.325 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:08.325 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:08.426 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:08.455 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:08.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:08.456 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:08.462 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:21:08.557 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:08.598 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:08.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:08.599 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:08.700 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:08.740 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:08.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:08.741 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:08.789 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:21:08.842 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:08.880 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:08.881 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:08.881 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:08.982 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:09.013 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:09.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:09.014 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:09.115 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:09.165 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:09.166 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:09.166 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:09.180 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:21:09.181 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:21:09.223 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:21:09.223 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:21:09.249 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:09.257 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:21:09.257 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:21:09.266 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:09.300 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:21:09.300 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:21:09.304 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-20T22:21:09.314 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:09.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:09.315 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:09.343 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:21:09.343 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:21:09.383 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:21:09.383 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:21:09.416 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:09.422 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:21:09.422 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:21:09.447 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:09.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:09.448 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:09.549 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:09.595 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:09.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:09.597 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:09.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:09.734 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:09.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:09.735 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:09.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:09 smithi060 ceph-mon[29694]: pgmap v977: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 3.8 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 755 B/s rd, 0 B/s wr, 989 op/s; 1276/188748 objects degraded (0.676%); 10902/188748 objects misplaced (5.776%); 46 MiB/s, 743 objects/s recovering 2024-09-20T22:21:09.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:09 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1140553931' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:09.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:09 smithi060 ceph-mon[40290]: pgmap v977: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 3.8 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 755 B/s rd, 0 B/s wr, 989 op/s; 1276/188748 objects degraded (0.676%); 10902/188748 objects misplaced (5.776%); 46 MiB/s, 743 objects/s recovering 2024-09-20T22:21:09.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:09 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1140553931' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:09.836 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:09.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:09.874 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:09.874 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:09.908 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:09 smithi067 ceph-mon[33511]: pgmap v977: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 3.8 GiB data, 16 GiB used, 521 GiB / 536 GiB avail; 755 B/s rd, 0 B/s wr, 989 op/s; 1276/188748 objects degraded (0.676%); 10902/188748 objects misplaced (5.776%); 46 MiB/s, 743 objects/s recovering 2024-09-20T22:21:09.908 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:09 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1140553931' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:09.976 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:10.010 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:10.011 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:10.011 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:10.112 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:10.148 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:10.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:10.149 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:10.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:10.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:10.290 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:10.290 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:10.391 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:10.441 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:10.442 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:10.442 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:10.443 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:21:10.444 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:21:10.542 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:10.569 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:10.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:10.570 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:10.625 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:21:10.626 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:21:10.671 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:10.710 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:10.710 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:10.711 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:10.811 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:10.847 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:10.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:10.848 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:10.948 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:10.984 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:10.985 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:10.985 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:11.086 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:11.127 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:11.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:11.128 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:11.228 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:11.261 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:11.262 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:11.262 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:11.363 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:11.388 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:11.389 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:11.389 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:11.490 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:11.526 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:11.527 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:11.527 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:11.588 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 5, 6, 7, 0] out_osds: [4, 1] dead_osds: [4] live_osds: [0, 2, 3, 5, 6, 7, 1] 2024-09-20T22:21:11.589 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:21:11.589 INFO:tasks.thrashosds.thrasher:fixing pg num pool .mgr 2024-09-20T22:21:11.589 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:21:11.628 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:11.649 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:21:11.659 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:11.660 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:11.660 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:11.761 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:11.791 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:11.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:11.792 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:11.893 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:11.932 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:11.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:11.933 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:11.943 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:11 smithi060 ceph-mon[29694]: pgmap v978: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 3.7 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 587 B/s rd, 0 B/s wr, 899 op/s; 1276/184680 objects degraded (0.691%); 10902/184680 objects misplaced (5.903%); 36 MiB/s, 578 objects/s recovering 2024-09-20T22:21:11.943 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:11 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:21:11.943 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:11 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:21:11.943 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:11 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2034543070' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:21:11.943 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:11 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:21:11.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:11 smithi060 ceph-mon[40290]: pgmap v978: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 3.7 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 587 B/s rd, 0 B/s wr, 899 op/s; 1276/184680 objects degraded (0.691%); 10902/184680 objects misplaced (5.903%); 36 MiB/s, 578 objects/s recovering 2024-09-20T22:21:11.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:11 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:21:11.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:11 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:21:11.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:11 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2034543070' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:21:11.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:11 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:21:12.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:11 smithi067 ceph-mon[33511]: pgmap v978: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 3.7 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 587 B/s rd, 0 B/s wr, 899 op/s; 1276/184680 objects degraded (0.691%); 10902/184680 objects misplaced (5.903%); 36 MiB/s, 578 objects/s recovering 2024-09-20T22:21:12.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:11 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:21:12.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:11 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:21:12.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:11 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2034543070' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:21:12.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:11 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:21:12.034 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:12.074 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:12.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:12.075 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:12.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:12.216 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:12.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:12.217 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:12.259 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T22:21:12.318 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:12.348 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:12.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:12.349 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:12.444 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:21:12.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:12.483 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:12.483 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:12.484 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:12.584 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:12.630 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:12.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:12.632 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:12.654 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:21:12.733 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:12.757 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:12.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:12.758 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:12.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:12 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:21:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:12 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:21:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:12 smithi060 ceph-mon[29694]: osdmap e514: 8 total, 7 up, 6 in 2024-09-20T22:21:12.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:12 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 1276/184680 objects degraded (0.691%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T22:21:12.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:12 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:21:12.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:12 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:21:12.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:12 smithi060 ceph-mon[40290]: osdmap e514: 8 total, 7 up, 6 in 2024-09-20T22:21:12.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:12 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 1276/184680 objects degraded (0.691%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T22:21:12.792 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:12 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:21:12.792 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:12 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:21:12.792 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:12 smithi067 ceph-mon[33511]: osdmap e514: 8 total, 7 up, 6 in 2024-09-20T22:21:12.793 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:12 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 1276/184680 objects degraded (0.691%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T22:21:12.859 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:12.890 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:12.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:12.891 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:12.991 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:13.017 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:13.018 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:13.018 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:13.043 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:21:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:21:13.120 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:13.147 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:13.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:13.148 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:13.249 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:13.275 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:13.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:13.276 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:13.311 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:21:13.311 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:21:13.377 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:13.403 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:13.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:13.404 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:13.505 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:13.532 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:13.533 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:13.533 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:13.634 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:13.661 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:13.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:13.662 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:13.763 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:13.808 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:13.809 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:13.809 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:13.910 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:13.933 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:13.933 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:13.934 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:13 smithi067 ceph-mon[33511]: pgmap v980: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 3.5 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.18k op/s; 1075/172287 objects degraded (0.624%); 7675/172287 objects misplaced (4.455%); 39 MiB/s, 621 objects/s recovering 2024-09-20T22:21:14.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:13 smithi067 ceph-mon[33511]: osdmap e515: 8 total, 7 up, 6 in 2024-09-20T22:21:14.035 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:14.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:13 smithi060 ceph-mon[29694]: pgmap v980: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 3.5 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.18k op/s; 1075/172287 objects degraded (0.624%); 7675/172287 objects misplaced (4.455%); 39 MiB/s, 621 objects/s recovering 2024-09-20T22:21:14.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:13 smithi060 ceph-mon[29694]: osdmap e515: 8 total, 7 up, 6 in 2024-09-20T22:21:14.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:13 smithi060 ceph-mon[40290]: pgmap v980: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 3.5 GiB data, 15 GiB used, 521 GiB / 536 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1.18k op/s; 1075/172287 objects degraded (0.624%); 7675/172287 objects misplaced (4.455%); 39 MiB/s, 621 objects/s recovering 2024-09-20T22:21:14.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:13 smithi060 ceph-mon[40290]: osdmap e515: 8 total, 7 up, 6 in 2024-09-20T22:21:14.059 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:14.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:14.060 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:14.160 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:14.184 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:14.185 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:14.185 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:14.286 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:14.317 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:14.317 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:14.318 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:14.418 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:14.460 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:14.461 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:14.461 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:14.562 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:14.599 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:14.600 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:14.600 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:14.701 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:14.734 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:14.735 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:14.736 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:14.836 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:14.876 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:14.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:14.877 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:14.954 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:14 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:21:14.954 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:14 smithi060 ceph-mon[29694]: osdmap e516: 8 total, 7 up, 6 in 2024-09-20T22:21:14.954 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:14 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:21:14.954 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:14 smithi060 ceph-mon[40290]: osdmap e516: 8 total, 7 up, 6 in 2024-09-20T22:21:14.978 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:15.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:15.013 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:15.013 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:15.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:14 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:21:15.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:14 smithi067 ceph-mon[33511]: osdmap e516: 8 total, 7 up, 6 in 2024-09-20T22:21:15.114 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:15.148 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:15.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:15.149 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:15.250 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:15.285 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:21:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:21:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:21:15.290 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:15.290 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:15.291 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:15.318 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T22:21:15.392 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:15.419 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:15.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:15.420 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:15.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:15.555 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:15.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:15.556 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:15.656 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:15.698 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:15.699 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:15.699 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:15.797 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:21:15.800 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:15.860 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:15.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:15.861 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:15.937 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:15 smithi067 ceph-mon[33511]: pgmap v983: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 3.4 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.20k op/s; 1075/167184 objects degraded (0.643%); 7675/167184 objects misplaced (4.591%); 26 MiB/s, 422 objects/s recovering 2024-09-20T22:21:15.963 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:16.008 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:16.009 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:16.010 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:16.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:15 smithi060 ceph-mon[29694]: pgmap v983: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 3.4 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.20k op/s; 1075/167184 objects degraded (0.643%); 7675/167184 objects misplaced (4.591%); 26 MiB/s, 422 objects/s recovering 2024-09-20T22:21:16.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:15 smithi060 ceph-mon[40290]: pgmap v983: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 3.4 GiB data, 15 GiB used, 522 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.20k op/s; 1075/167184 objects degraded (0.643%); 7675/167184 objects misplaced (4.591%); 26 MiB/s, 422 objects/s recovering 2024-09-20T22:21:16.111 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:16.162 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:16.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:16.163 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:16.175 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:16.264 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:16.279 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:16.299 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:16.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:16.301 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:16.401 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:16.438 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:16.438 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:16.439 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:16.492 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:21:16.492 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:21:16.540 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:16.575 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:16.576 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:16.576 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:16.677 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:16.716 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:16.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:16.717 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:16.817 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:16.853 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:16.853 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:16.854 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:16.954 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:16.992 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:16.993 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:16.993 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:17.094 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:17.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:17.135 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:17.135 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:17.236 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:17.271 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:17.271 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:17.272 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:17.280 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T22:21:17.373 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:17.426 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:17.427 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:17.427 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:17.528 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:17.563 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:17.564 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:17.564 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:17.665 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:17.706 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:17.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:17.707 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:17.711 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:21:17.808 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:17.839 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:17.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:17.840 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:17.901 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:17 smithi067 ceph-mon[33511]: pgmap v984: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 3.3 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.21k op/s; 1075/162897 objects degraded (0.660%); 6584/162897 objects misplaced (4.042%); 37 MiB/s, 584 objects/s recovering 2024-09-20T22:21:17.901 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:17 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 1075/162897 objects degraded (0.660%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T22:21:17.940 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:17.965 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:17.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:17.966 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:18.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:17 smithi060 ceph-mon[29694]: pgmap v984: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 3.3 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.21k op/s; 1075/162897 objects degraded (0.660%); 6584/162897 objects misplaced (4.042%); 37 MiB/s, 584 objects/s recovering 2024-09-20T22:21:18.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:17 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 1075/162897 objects degraded (0.660%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T22:21:18.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:17 smithi060 ceph-mon[40290]: pgmap v984: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 1 active+remapped+backfill_toofull, 173 active+clean; 3.3 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.21k op/s; 1075/162897 objects degraded (0.660%); 6584/162897 objects misplaced (4.042%); 37 MiB/s, 584 objects/s recovering 2024-09-20T22:21:18.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:17 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 1075/162897 objects degraded (0.660%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T22:21:18.067 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:18.090 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:18.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:18.091 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:18.119 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:21:18.192 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:18.223 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:21:18.224 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:21:18.224 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:18.225 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:18.225 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:18.245 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:21:18.246 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:21:18.267 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:21:18.267 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:21:18.289 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:21:18.290 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:21:18.290 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-20T22:21:18.309 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:21:18.309 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:21:18.326 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:18.331 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:21:18.331 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:21:18.355 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:21:18.355 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:21:18.356 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:18.356 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:18.356 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:18.457 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:18.534 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:18.535 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:18.536 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:18.636 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:18.658 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:18.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:18.659 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:18.760 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:18.792 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:18.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:18.793 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:18.894 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:18.930 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:18.931 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:18.931 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:19.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:18 smithi067 ceph-mon[33511]: osdmap e517: 8 total, 7 up, 6 in 2024-09-20T22:21:19.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:18 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3941645026' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:19.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:18 smithi067 ceph-mon[33511]: pgmap v986: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 174 active+clean; 3.1 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.05k op/s; 737/153360 objects degraded (0.481%); 4647/153360 objects misplaced (3.030%); 31 MiB/s, 489 objects/s recovering 2024-09-20T22:21:19.032 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:19.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:18 smithi060 ceph-mon[29694]: osdmap e517: 8 total, 7 up, 6 in 2024-09-20T22:21:19.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:18 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3941645026' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:19.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:18 smithi060 ceph-mon[29694]: pgmap v986: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 174 active+clean; 3.1 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.05k op/s; 737/153360 objects degraded (0.481%); 4647/153360 objects misplaced (3.030%); 31 MiB/s, 489 objects/s recovering 2024-09-20T22:21:19.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:18 smithi060 ceph-mon[40290]: osdmap e517: 8 total, 7 up, 6 in 2024-09-20T22:21:19.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:18 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3941645026' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:19.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:18 smithi060 ceph-mon[40290]: pgmap v986: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 174 active+clean; 3.1 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.05k op/s; 737/153360 objects degraded (0.481%); 4647/153360 objects misplaced (3.030%); 31 MiB/s, 489 objects/s recovering 2024-09-20T22:21:19.055 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:21:19.056 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:21:19.073 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:19.075 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:19.075 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:19.176 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:19.213 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:19.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:19.215 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:19.315 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:19.354 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:19.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:19.355 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:19.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:19.492 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:19.493 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:19.493 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:19.594 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:19.625 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:19.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:19.627 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:19.727 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:19.760 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:19.761 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:19.761 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:19.862 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:19.892 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:19.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:19.893 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:19.993 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:19 smithi067 ceph-mon[33511]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-20T22:21:20.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:19 smithi067 ceph-mon[33511]: osdmap e518: 8 total, 7 up, 6 in 2024-09-20T22:21:20.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:20.026 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:20.026 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:19 smithi060 ceph-mon[29694]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-20T22:21:20.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:19 smithi060 ceph-mon[29694]: osdmap e518: 8 total, 7 up, 6 in 2024-09-20T22:21:20.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:19 smithi060 ceph-mon[40290]: Health check cleared: PG_BACKFILL_FULL (was: Low space hindering backfill (add storage if this doesn't resolve itself): 1 pg backfill_toofull) 2024-09-20T22:21:20.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:19 smithi060 ceph-mon[40290]: osdmap e518: 8 total, 7 up, 6 in 2024-09-20T22:21:20.127 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:20.137 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:21:20.140 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:21:20.175 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:20.176 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:20.176 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:20.277 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:20.318 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:20.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:20.319 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:20.362 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:21:20.363 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:21:20.420 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:20.459 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:20.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:20.461 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:20.561 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:20.597 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:20.597 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:20.597 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:20.698 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:20.720 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:20.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:20.721 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:20.821 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:20.846 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:20.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:20.847 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:20.948 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:20.977 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:20.978 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:20.978 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:20 smithi067 ceph-mon[33511]: from='client.16608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:21:21.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:20 smithi067 ceph-mon[33511]: pgmap v988: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 174 active+clean; 3.1 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 763 B/s rd, 0 B/s wr, 688 op/s; 737/153360 objects degraded (0.481%); 4647/153360 objects misplaced (3.030%); 27 MiB/s, 438 objects/s recovering 2024-09-20T22:21:21.030 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:20 smithi060 ceph-mon[40290]: from='client.16608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:21:21.030 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:20 smithi060 ceph-mon[40290]: pgmap v988: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 174 active+clean; 3.1 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 763 B/s rd, 0 B/s wr, 688 op/s; 737/153360 objects degraded (0.481%); 4647/153360 objects misplaced (3.030%); 27 MiB/s, 438 objects/s recovering 2024-09-20T22:21:21.030 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:20 smithi060 ceph-mon[29694]: from='client.16608 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:21:21.030 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:20 smithi060 ceph-mon[29694]: pgmap v988: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 3 active+remapped+backfilling, 174 active+clean; 3.1 GiB data, 14 GiB used, 522 GiB / 536 GiB avail; 763 B/s rd, 0 B/s wr, 688 op/s; 737/153360 objects degraded (0.481%); 4647/153360 objects misplaced (3.030%); 27 MiB/s, 438 objects/s recovering 2024-09-20T22:21:21.061 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":987,"stamp":"2024-09-20T22:21:18.684573+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3337747793,"num_objects":51120,"num_object_clones":0,"num_object_copies":153360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":737,"num_objects_misplaced":4647,"num_objects_unfound":0,"num_objects_dirty":51120,"num_whiteouts":0,"num_read":2219,"num_read_kb":1980,"num_write":153788,"num_write_kb":6527260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47721,"num_bytes_recovered":3103576690,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":60065,"ondisk_log_size":60065,"up":534,"acting":533,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":534,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":14681248,"kb_used_data":9540412,"kb_used_omap":9,"kb_used_meta":5140470,"kb_avail":547789664,"statfs":{"total":575970213888,"available":560936615936,"internally_reserved":0,"allocated":9769381888,"data_stored":9767123127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":5263841958},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-389283840,"num_objects":-5940,"num_object_clones":0,"num_object_copies":-17820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-338,"num_objects_misplaced":-3028,"num_objects_unfound":0,"num_objects_dirty":-5940,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5940,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2938,"num_bytes_recovered":192544768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.998742"},"pg_stats":[{"pgid":"1.1d","version":"75'128","reported_seq":481,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689210+0000","last_change":"2024-09-20T22:19:48.867479+0000","last_active":"2024-09-20T22:21:13.689210+0000","last_peered":"2024-09-20T22:21:13.689210+0000","last_clean":"2024-09-20T22:21:13.689210+0000","last_became_active":"2024-09-20T22:19:48.866938+0000","last_became_peered":"2024-09-20T22:19:48.866938+0000","last_unstale":"2024-09-20T22:21:13.689210+0000","last_undegraded":"2024-09-20T22:21:13.689210+0000","last_fullsized":"2024-09-20T22:21:13.689210+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:57.128894+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.128894+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":423,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.696370+0000","last_change":"2024-09-20T22:18:43.512180+0000","last_active":"2024-09-20T22:21:13.696370+0000","last_peered":"2024-09-20T22:21:13.696370+0000","last_clean":"2024-09-20T22:21:13.696370+0000","last_became_active":"2024-09-20T22:12:52.515921+0000","last_became_peered":"2024-09-20T22:12:52.515921+0000","last_unstale":"2024-09-20T22:21:13.696370+0000","last_undegraded":"2024-09-20T22:21:13.696370+0000","last_fullsized":"2024-09-20T22:21:13.696370+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:43.512066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:43.512066+0000","last_clean_scrub_stamp":"2024-09-20T22:18:43.512066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913517+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"327'19","reported_seq":723,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.696728+0000","last_change":"2024-09-20T22:19:45.861936+0000","last_active":"2024-09-20T22:21:17.696728+0000","last_peered":"2024-09-20T22:21:17.696728+0000","last_clean":"2024-09-20T22:21:17.696728+0000","last_became_active":"2024-09-20T22:19:45.798056+0000","last_became_peered":"2024-09-20T22:19:45.798056+0000","last_unstale":"2024-09-20T22:21:17.696728+0000","last_undegraded":"2024-09-20T22:21:17.696728+0000","last_fullsized":"2024-09-20T22:21:17.696728+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"327'19","last_scrub_stamp":"2024-09-20T22:19:15.299519+0000","last_deep_scrub":"252'13","last_deep_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_clean_scrub_stamp":"2024-09-20T22:19:15.299519+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":195,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.688458+0000","last_change":"2024-09-20T22:19:45.795725+0000","last_active":"2024-09-20T22:21:13.688458+0000","last_peered":"2024-09-20T22:21:13.688458+0000","last_clean":"2024-09-20T22:21:13.688458+0000","last_became_active":"2024-09-20T22:19:45.795374+0000","last_became_peered":"2024-09-20T22:19:45.795374+0000","last_unstale":"2024-09-20T22:21:13.688458+0000","last_undegraded":"2024-09-20T22:21:13.688458+0000","last_fullsized":"2024-09-20T22:21:13.688458+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:04.319429+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:04.319429+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":572,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.688853+0000","last_change":"2024-09-20T22:19:45.797905+0000","last_active":"2024-09-20T22:21:13.688853+0000","last_peered":"2024-09-20T22:21:13.688853+0000","last_clean":"2024-09-20T22:21:13.688853+0000","last_became_active":"2024-09-20T22:19:45.796589+0000","last_became_peered":"2024-09-20T22:19:45.796589+0000","last_unstale":"2024-09-20T22:21:13.688853+0000","last_undegraded":"2024-09-20T22:21:13.688853+0000","last_fullsized":"2024-09-20T22:21:13.688853+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.321275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.321275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1c","version":"75'128","reported_seq":934,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.698187+0000","last_change":"2024-09-20T22:19:48.838779+0000","last_active":"2024-09-20T22:21:13.698187+0000","last_peered":"2024-09-20T22:21:13.698187+0000","last_clean":"2024-09-20T22:21:13.698187+0000","last_became_active":"2024-09-20T22:19:48.838208+0000","last_became_peered":"2024-09-20T22:19:48.838208+0000","last_unstale":"2024-09-20T22:21:13.698187+0000","last_undegraded":"2024-09-20T22:21:13.698187+0000","last_fullsized":"2024-09-20T22:21:13.698187+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:57.852213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.852213+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":974,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.692188+0000","last_change":"2024-09-20T22:19:38.797027+0000","last_active":"2024-09-20T22:21:13.692188+0000","last_peered":"2024-09-20T22:21:13.692188+0000","last_clean":"2024-09-20T22:21:13.692188+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:21:13.692188+0000","last_undegraded":"2024-09-20T22:21:13.692188+0000","last_fullsized":"2024-09-20T22:21:13.692188+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:19:38.796920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:19:38.796920+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":452,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.695563+0000","last_change":"2024-09-20T22:18:31.987311+0000","last_active":"2024-09-20T22:21:17.695563+0000","last_peered":"2024-09-20T22:21:17.695563+0000","last_clean":"2024-09-20T22:21:17.695563+0000","last_became_active":"2024-09-20T22:12:33.545100+0000","last_became_peered":"2024-09-20T22:12:33.545100+0000","last_unstale":"2024-09-20T22:21:17.695563+0000","last_undegraded":"2024-09-20T22:21:17.695563+0000","last_fullsized":"2024-09-20T22:21:17.695563+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:31.987191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:31.987191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"406'34","reported_seq":580,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.692234+0000","last_change":"2024-09-20T22:19:45.100154+0000","last_active":"2024-09-20T22:21:13.692234+0000","last_peered":"2024-09-20T22:21:13.692234+0000","last_clean":"2024-09-20T22:21:13.692234+0000","last_became_active":"2024-09-20T22:12:52.517296+0000","last_became_peered":"2024-09-20T22:12:52.517296+0000","last_unstale":"2024-09-20T22:21:13.692234+0000","last_undegraded":"2024-09-20T22:21:13.692234+0000","last_fullsized":"2024-09-20T22:21:13.692234+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'34","last_scrub_stamp":"2024-09-20T22:19:45.100033+0000","last_deep_scrub":"406'34","last_deep_scrub_stamp":"2024-09-20T22:16:31.033354+0000","last_clean_scrub_stamp":"2024-09-20T22:19:45.100033+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":274,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.696830+0000","last_change":"2024-09-20T22:19:45.791468+0000","last_active":"2024-09-20T22:21:17.696830+0000","last_peered":"2024-09-20T22:21:17.696830+0000","last_clean":"2024-09-20T22:21:17.696830+0000","last_became_active":"2024-09-20T22:19:45.791138+0000","last_became_peered":"2024-09-20T22:19:45.791138+0000","last_unstale":"2024-09-20T22:21:17.696830+0000","last_undegraded":"2024-09-20T22:21:17.696830+0000","last_fullsized":"2024-09-20T22:21:17.696830+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:10.335527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:10.335527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":997,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689418+0000","last_change":"2024-09-20T22:19:45.790639+0000","last_active":"2024-09-20T22:21:13.689418+0000","last_peered":"2024-09-20T22:21:13.689418+0000","last_clean":"2024-09-20T22:21:13.689418+0000","last_became_active":"2024-09-20T22:19:45.790130+0000","last_became_peered":"2024-09-20T22:19:45.790130+0000","last_unstale":"2024-09-20T22:21:13.689418+0000","last_undegraded":"2024-09-20T22:21:13.689418+0000","last_fullsized":"2024-09-20T22:21:13.689418+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:41.350325+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:17:41.350325+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"328'15","reported_seq":857,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.695615+0000","last_change":"2024-09-20T22:18:35.404979+0000","last_active":"2024-09-20T22:21:13.695615+0000","last_peered":"2024-09-20T22:21:13.695615+0000","last_clean":"2024-09-20T22:21:13.695615+0000","last_became_active":"2024-09-20T22:17:03.663697+0000","last_became_peered":"2024-09-20T22:17:03.663697+0000","last_unstale":"2024-09-20T22:21:13.695615+0000","last_undegraded":"2024-09-20T22:21:13.695615+0000","last_fullsized":"2024-09-20T22:21:13.695615+0000","mapping_epoch":415,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":416,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:18:35.404892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:35.404892+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913382+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":516,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.688933+0000","last_change":"2024-09-20T22:17:30.235268+0000","last_active":"2024-09-20T22:21:13.688933+0000","last_peered":"2024-09-20T22:21:13.688933+0000","last_clean":"2024-09-20T22:21:13.688933+0000","last_became_active":"2024-09-20T22:12:52.518549+0000","last_became_peered":"2024-09-20T22:12:52.518549+0000","last_unstale":"2024-09-20T22:21:13.688933+0000","last_undegraded":"2024-09-20T22:21:13.688933+0000","last_fullsized":"2024-09-20T22:21:13.688933+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:30.235059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:30.235059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":36,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":265,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.697524+0000","last_change":"2024-09-20T22:19:45.801571+0000","last_active":"2024-09-20T22:21:13.697524+0000","last_peered":"2024-09-20T22:21:13.697524+0000","last_clean":"2024-09-20T22:21:13.697524+0000","last_became_active":"2024-09-20T22:19:45.801215+0000","last_became_peered":"2024-09-20T22:19:45.801215+0000","last_unstale":"2024-09-20T22:21:13.697524+0000","last_undegraded":"2024-09-20T22:21:13.697524+0000","last_fullsized":"2024-09-20T22:21:13.697524+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:13.392405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:37.934760+0000","last_clean_scrub_stamp":"2024-09-20T22:19:13.392405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1e","version":"75'128","reported_seq":492,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.694345+0000","last_change":"2024-09-20T22:19:34.669163+0000","last_active":"2024-09-20T22:21:13.694345+0000","last_peered":"2024-09-20T22:21:13.694345+0000","last_clean":"2024-09-20T22:21:13.694345+0000","last_became_active":"2024-09-20T22:17:44.708330+0000","last_became_peered":"2024-09-20T22:17:44.708330+0000","last_unstale":"2024-09-20T22:21:13.694345+0000","last_undegraded":"2024-09-20T22:21:13.694345+0000","last_fullsized":"2024-09-20T22:21:13.694345+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:34.669046+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:17:40.367940+0000","last_clean_scrub_stamp":"2024-09-20T22:19:34.669046+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1002,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689493+0000","last_change":"2024-09-20T22:19:45.796110+0000","last_active":"2024-09-20T22:21:13.689493+0000","last_peered":"2024-09-20T22:21:13.689493+0000","last_clean":"2024-09-20T22:21:13.689493+0000","last_became_active":"2024-09-20T22:19:45.795550+0000","last_became_peered":"2024-09-20T22:19:45.795550+0000","last_unstale":"2024-09-20T22:21:13.689493+0000","last_undegraded":"2024-09-20T22:21:13.689493+0000","last_fullsized":"2024-09-20T22:21:13.689493+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:03.470297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:30.215263+0000","last_clean_scrub_stamp":"2024-09-20T22:18:03.470297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"328'7","reported_seq":685,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.690859+0000","last_change":"2024-09-20T22:19:45.837552+0000","last_active":"2024-09-20T22:21:13.690859+0000","last_peered":"2024-09-20T22:21:13.690859+0000","last_clean":"2024-09-20T22:21:13.690859+0000","last_became_active":"2024-09-20T22:19:45.799278+0000","last_became_peered":"2024-09-20T22:19:45.799278+0000","last_unstale":"2024-09-20T22:21:13.690859+0000","last_undegraded":"2024-09-20T22:21:13.690859+0000","last_fullsized":"2024-09-20T22:21:13.690859+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'7","last_scrub_stamp":"2024-09-20T22:17:30.369518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:30.369518+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":983,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.697934+0000","last_change":"2024-09-20T22:19:45.817841+0000","last_active":"2024-09-20T22:21:13.697934+0000","last_peered":"2024-09-20T22:21:13.697934+0000","last_clean":"2024-09-20T22:21:13.697934+0000","last_became_active":"2024-09-20T22:19:45.817477+0000","last_became_peered":"2024-09-20T22:19:45.817477+0000","last_unstale":"2024-09-20T22:21:13.697934+0000","last_undegraded":"2024-09-20T22:21:13.697934+0000","last_fullsized":"2024-09-20T22:21:13.697934+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:03.782934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:03.782934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":511,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.688554+0000","last_change":"2024-09-20T22:19:45.789735+0000","last_active":"2024-09-20T22:21:13.688554+0000","last_peered":"2024-09-20T22:21:13.688554+0000","last_clean":"2024-09-20T22:21:13.688554+0000","last_became_active":"2024-09-20T22:19:45.789372+0000","last_became_peered":"2024-09-20T22:19:45.789372+0000","last_unstale":"2024-09-20T22:21:13.688554+0000","last_undegraded":"2024-09-20T22:21:13.688554+0000","last_fullsized":"2024-09-20T22:21:13.688554+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:33.243552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:17:33.243552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.19","version":"75'128","reported_seq":750,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.693970+0000","last_change":"2024-09-20T22:17:56.103865+0000","last_active":"2024-09-20T22:21:13.693970+0000","last_peered":"2024-09-20T22:21:13.693970+0000","last_clean":"2024-09-20T22:21:13.693970+0000","last_became_active":"2024-09-20T22:17:44.716139+0000","last_became_peered":"2024-09-20T22:17:44.716139+0000","last_unstale":"2024-09-20T22:21:13.693970+0000","last_undegraded":"2024-09-20T22:21:13.693970+0000","last_fullsized":"2024-09-20T22:21:13.693970+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.103744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.103744+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913294+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":108,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.695414+0000","last_change":"2024-09-20T22:19:44.809795+0000","last_active":"2024-09-20T22:21:17.695414+0000","last_peered":"2024-09-20T22:21:17.695414+0000","last_clean":"2024-09-20T22:21:17.695414+0000","last_became_active":"2024-09-20T22:19:28.245420+0000","last_became_peered":"2024-09-20T22:19:28.245420+0000","last_unstale":"2024-09-20T22:21:17.695414+0000","last_undegraded":"2024-09-20T22:21:17.695414+0000","last_fullsized":"2024-09-20T22:21:17.695414+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:44.809724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:19:44.809724+0000","last_clean_scrub_stamp":"2024-09-20T22:19:44.809724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":123,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.698117+0000","last_change":"2024-09-20T22:19:45.796330+0000","last_active":"2024-09-20T22:21:13.698117+0000","last_peered":"2024-09-20T22:21:13.698117+0000","last_clean":"2024-09-20T22:21:13.698117+0000","last_became_active":"2024-09-20T22:19:45.796028+0000","last_became_peered":"2024-09-20T22:19:45.796028+0000","last_unstale":"2024-09-20T22:21:13.698117+0000","last_undegraded":"2024-09-20T22:21:13.698117+0000","last_fullsized":"2024-09-20T22:21:13.698117+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:07.403912+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:17:07.403912+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"328'34","reported_seq":591,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.693604+0000","last_change":"2024-09-20T22:19:45.881363+0000","last_active":"2024-09-20T22:21:13.693604+0000","last_peered":"2024-09-20T22:21:13.693604+0000","last_clean":"2024-09-20T22:21:13.693604+0000","last_became_active":"2024-09-20T22:19:45.805481+0000","last_became_peered":"2024-09-20T22:19:45.805481+0000","last_unstale":"2024-09-20T22:21:13.693604+0000","last_undegraded":"2024-09-20T22:21:13.693604+0000","last_fullsized":"2024-09-20T22:21:13.693604+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'34","last_scrub_stamp":"2024-09-20T22:17:41.807990+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:17:41.807990+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":812,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.694094+0000","last_change":"2024-09-20T22:19:45.818815+0000","last_active":"2024-09-20T22:21:13.694094+0000","last_peered":"2024-09-20T22:21:13.694094+0000","last_clean":"2024-09-20T22:21:13.694094+0000","last_became_active":"2024-09-20T22:19:45.818397+0000","last_became_peered":"2024-09-20T22:19:45.818397+0000","last_unstale":"2024-09-20T22:21:13.694094+0000","last_undegraded":"2024-09-20T22:21:13.694094+0000","last_fullsized":"2024-09-20T22:21:13.694094+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.295028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:44.295028+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.295028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913492+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.18","version":"75'128","reported_seq":590,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.688319+0000","last_change":"2024-09-20T22:17:56.357255+0000","last_active":"2024-09-20T22:21:13.688319+0000","last_peered":"2024-09-20T22:21:13.688319+0000","last_clean":"2024-09-20T22:21:13.688319+0000","last_became_active":"2024-09-20T22:17:44.717551+0000","last_became_peered":"2024-09-20T22:17:44.717551+0000","last_unstale":"2024-09-20T22:21:13.688319+0000","last_undegraded":"2024-09-20T22:21:13.688319+0000","last_fullsized":"2024-09-20T22:21:13.688319+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.357167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.357167+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":921,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.695862+0000","last_change":"2024-09-20T22:18:31.015563+0000","last_active":"2024-09-20T22:21:17.695862+0000","last_peered":"2024-09-20T22:21:17.695862+0000","last_clean":"2024-09-20T22:21:17.695862+0000","last_became_active":"2024-09-20T22:12:21.521433+0000","last_became_peered":"2024-09-20T22:12:21.521433+0000","last_unstale":"2024-09-20T22:21:17.695862+0000","last_undegraded":"2024-09-20T22:21:17.695862+0000","last_fullsized":"2024-09-20T22:21:17.695862+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:31.015449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:31.015449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":211,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.690474+0000","last_change":"2024-09-20T22:19:45.802144+0000","last_active":"2024-09-20T22:21:13.690474+0000","last_peered":"2024-09-20T22:21:13.690474+0000","last_clean":"2024-09-20T22:21:13.690474+0000","last_became_active":"2024-09-20T22:19:45.801808+0000","last_became_peered":"2024-09-20T22:19:45.801808+0000","last_unstale":"2024-09-20T22:21:13.690474+0000","last_undegraded":"2024-09-20T22:21:13.690474+0000","last_fullsized":"2024-09-20T22:21:13.690474+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.456415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:44.456415+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.456415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"328'23","reported_seq":890,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.695286+0000","last_change":"2024-09-20T22:19:45.926740+0000","last_active":"2024-09-20T22:21:17.695286+0000","last_peered":"2024-09-20T22:21:17.695286+0000","last_clean":"2024-09-20T22:21:17.695286+0000","last_became_active":"2024-09-20T22:19:45.791170+0000","last_became_peered":"2024-09-20T22:19:45.791170+0000","last_unstale":"2024-09-20T22:21:17.695286+0000","last_undegraded":"2024-09-20T22:21:17.695286+0000","last_fullsized":"2024-09-20T22:21:17.695286+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'23","last_scrub_stamp":"2024-09-20T22:17:38.263865+0000","last_deep_scrub":"250'17","last_deep_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_clean_scrub_stamp":"2024-09-20T22:17:38.263865+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":106,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.692529+0000","last_change":"2024-09-20T22:19:31.646518+0000","last_active":"2024-09-20T22:21:13.692529+0000","last_peered":"2024-09-20T22:21:13.692529+0000","last_clean":"2024-09-20T22:21:13.692529+0000","last_became_active":"2024-09-20T22:19:28.238293+0000","last_became_peered":"2024-09-20T22:19:28.238293+0000","last_unstale":"2024-09-20T22:21:13.692529+0000","last_undegraded":"2024-09-20T22:21:13.692529+0000","last_fullsized":"2024-09-20T22:21:13.692529+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:31.646417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:31.646417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1b","version":"75'128","reported_seq":903,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689093+0000","last_change":"2024-09-20T22:19:31.252457+0000","last_active":"2024-09-20T22:21:13.689093+0000","last_peered":"2024-09-20T22:21:13.689093+0000","last_clean":"2024-09-20T22:21:13.689093+0000","last_became_active":"2024-09-20T22:19:31.252076+0000","last_became_peered":"2024-09-20T22:19:31.252076+0000","last_unstale":"2024-09-20T22:21:13.689093+0000","last_undegraded":"2024-09-20T22:21:13.689093+0000","last_fullsized":"2024-09-20T22:21:13.689093+0000","mapping_epoch":475,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":476,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:58.426044+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:17:58.426044+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":960,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.698442+0000","last_change":"2024-09-20T22:18:09.700077+0000","last_active":"2024-09-20T22:21:13.698442+0000","last_peered":"2024-09-20T22:21:13.698442+0000","last_clean":"2024-09-20T22:21:13.698442+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:21:13.698442+0000","last_undegraded":"2024-09-20T22:21:13.698442+0000","last_fullsized":"2024-09-20T22:21:13.698442+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:18:09.699725+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:18:09.699725+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"289'2","reported_seq":415,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.691374+0000","last_change":"2024-09-20T22:19:46.757972+0000","last_active":"2024-09-20T22:21:13.691374+0000","last_peered":"2024-09-20T22:21:13.691374+0000","last_clean":"2024-09-20T22:21:13.691374+0000","last_became_active":"2024-09-20T22:19:45.796939+0000","last_became_peered":"2024-09-20T22:19:45.796939+0000","last_unstale":"2024-09-20T22:21:13.691374+0000","last_undegraded":"2024-09-20T22:21:13.691374+0000","last_fullsized":"2024-09-20T22:21:13.691374+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-20T22:19:46.757900+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_clean_scrub_stamp":"2024-09-20T22:19:46.757900+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"328'20","reported_seq":450,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.695155+0000","last_change":"2024-09-20T22:18:10.346398+0000","last_active":"2024-09-20T22:21:13.695155+0000","last_peered":"2024-09-20T22:21:13.695155+0000","last_clean":"2024-09-20T22:21:13.695155+0000","last_became_active":"2024-09-20T22:12:52.526803+0000","last_became_peered":"2024-09-20T22:12:52.526803+0000","last_unstale":"2024-09-20T22:21:13.695155+0000","last_undegraded":"2024-09-20T22:21:13.695155+0000","last_fullsized":"2024-09-20T22:21:13.695155+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:18:10.346320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:10.346320+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913325+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"475'12","reported_seq":1252,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.696006+0000","last_change":"2024-09-20T22:19:28.274959+0000","last_active":"2024-09-20T22:21:17.696006+0000","last_peered":"2024-09-20T22:21:17.696006+0000","last_clean":"2024-09-20T22:21:17.696006+0000","last_became_active":"2024-09-20T22:19:28.256901+0000","last_became_peered":"2024-09-20T22:19:28.256901+0000","last_unstale":"2024-09-20T22:21:17.696006+0000","last_undegraded":"2024-09-20T22:21:17.696006+0000","last_fullsized":"2024-09-20T22:21:17.696006+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'11","last_scrub_stamp":"2024-09-20T22:18:33.978168+0000","last_deep_scrub":"291'11","last_deep_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_clean_scrub_stamp":"2024-09-20T22:18:33.978168+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.0","version":"516'9374","reported_seq":19506,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:16.152471+0000","last_change":"2024-09-20T22:20:07.810133+0000","last_active":"2024-09-20T22:21:16.152471+0000","last_peered":"2024-09-20T22:21:16.152471+0000","last_clean":"2024-09-20T22:21:16.152471+0000","last_became_active":"2024-09-20T22:20:07.804419+0000","last_became_peered":"2024-09-20T22:20:07.804419+0000","last_unstale":"2024-09-20T22:21:16.152471+0000","last_undegraded":"2024-09-20T22:21:16.152471+0000","last_fullsized":"2024-09-20T22:21:16.152471+0000","mapping_epoch":490,"log_start":"493'6221","ondisk_log_start":"493'6221","created":456,"last_epoch_clean":491,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":3153,"log_dups_size":0,"ondisk_log_size":3153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":214958080,"num_objects":3280,"num_object_clones":0,"num_object_copies":9840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9374,"num_write_kb":404928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4993,"num_bytes_recovered":327221248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":985,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.688822+0000","last_change":"2024-09-20T22:17:40.331506+0000","last_active":"2024-09-20T22:21:13.688822+0000","last_peered":"2024-09-20T22:21:13.688822+0000","last_clean":"2024-09-20T22:21:13.688822+0000","last_became_active":"2024-09-20T22:17:40.139526+0000","last_became_peered":"2024-09-20T22:17:40.139526+0000","last_unstale":"2024-09-20T22:21:13.688822+0000","last_undegraded":"2024-09-20T22:21:13.688822+0000","last_fullsized":"2024-09-20T22:21:13.688822+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:40.331395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:40.331395+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"328'27","reported_seq":770,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.698295+0000","last_change":"2024-09-20T22:19:45.821849+0000","last_active":"2024-09-20T22:21:13.698295+0000","last_peered":"2024-09-20T22:21:13.698295+0000","last_clean":"2024-09-20T22:21:13.698295+0000","last_became_active":"2024-09-20T22:19:45.801225+0000","last_became_peered":"2024-09-20T22:19:45.801225+0000","last_unstale":"2024-09-20T22:21:13.698295+0000","last_undegraded":"2024-09-20T22:21:13.698295+0000","last_fullsized":"2024-09-20T22:21:13.698295+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:17:34.818375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:34.818375+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1001,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689941+0000","last_change":"2024-09-20T22:19:45.799368+0000","last_active":"2024-09-20T22:21:13.689941+0000","last_peered":"2024-09-20T22:21:13.689941+0000","last_clean":"2024-09-20T22:21:13.689941+0000","last_became_active":"2024-09-20T22:19:45.797845+0000","last_became_peered":"2024-09-20T22:19:45.797845+0000","last_unstale":"2024-09-20T22:21:13.689941+0000","last_undegraded":"2024-09-20T22:21:13.689941+0000","last_fullsized":"2024-09-20T22:21:13.689941+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:17.513228+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:19:17.513228+0000","last_clean_scrub_stamp":"2024-09-20T22:19:17.513228+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"475'14","reported_seq":732,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689978+0000","last_change":"2024-09-20T22:19:28.252125+0000","last_active":"2024-09-20T22:21:13.689978+0000","last_peered":"2024-09-20T22:21:13.689978+0000","last_clean":"2024-09-20T22:21:13.689978+0000","last_became_active":"2024-09-20T22:19:28.242412+0000","last_became_peered":"2024-09-20T22:19:28.242412+0000","last_unstale":"2024-09-20T22:21:13.689978+0000","last_undegraded":"2024-09-20T22:21:13.689978+0000","last_fullsized":"2024-09-20T22:21:13.689978+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'13","last_scrub_stamp":"2024-09-20T22:18:40.407802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:40.407802+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":523,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689833+0000","last_change":"2024-09-20T22:19:45.840327+0000","last_active":"2024-09-20T22:21:13.689833+0000","last_peered":"2024-09-20T22:21:13.689833+0000","last_clean":"2024-09-20T22:21:13.689833+0000","last_became_active":"2024-09-20T22:19:45.839873+0000","last_became_peered":"2024-09-20T22:19:45.839873+0000","last_unstale":"2024-09-20T22:21:13.689833+0000","last_undegraded":"2024-09-20T22:21:13.689833+0000","last_fullsized":"2024-09-20T22:21:13.689833+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:04.453445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:04.453445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.f","version":"516'9648","reported_seq":21183,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:16.173587+0000","last_change":"2024-09-20T22:20:40.705764+0000","last_active":"2024-09-20T22:21:16.173587+0000","last_peered":"2024-09-20T22:21:16.173587+0000","last_clean":"2024-09-20T22:21:16.173587+0000","last_became_active":"2024-09-20T22:20:40.695504+0000","last_became_peered":"2024-09-20T22:20:40.695504+0000","last_unstale":"2024-09-20T22:21:16.173587+0000","last_undegraded":"2024-09-20T22:21:16.173587+0000","last_fullsized":"2024-09-20T22:21:16.173587+0000","mapping_epoch":499,"log_start":"493'6473","ondisk_log_start":"493'6473","created":456,"last_epoch_clean":500,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":3175,"log_dups_size":0,"ondisk_log_size":3175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":216399872,"num_objects":3302,"num_object_clones":0,"num_object_copies":9906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9648,"num_write_kb":414400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5692,"num_bytes_recovered":373030912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,0],"acting":[7,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":390,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689455+0000","last_change":"2024-09-20T22:19:31.288520+0000","last_active":"2024-09-20T22:21:13.689455+0000","last_peered":"2024-09-20T22:21:13.689455+0000","last_clean":"2024-09-20T22:21:13.689455+0000","last_became_active":"2024-09-20T22:19:31.280713+0000","last_became_peered":"2024-09-20T22:19:31.280713+0000","last_unstale":"2024-09-20T22:21:13.689455+0000","last_undegraded":"2024-09-20T22:21:13.689455+0000","last_fullsized":"2024-09-20T22:21:13.689455+0000","mapping_epoch":475,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":476,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:42.412207+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:17:42.412207+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.518349+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13417,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":4,"num_write_kb":55,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":40252,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"328'30","reported_seq":983,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.697491+0000","last_change":"2024-09-20T22:18:43.923440+0000","last_active":"2024-09-20T22:21:13.697491+0000","last_peered":"2024-09-20T22:21:13.697491+0000","last_clean":"2024-09-20T22:21:13.697491+0000","last_became_active":"2024-09-20T22:12:52.526867+0000","last_became_peered":"2024-09-20T22:12:52.526867+0000","last_unstale":"2024-09-20T22:21:13.697491+0000","last_undegraded":"2024-09-20T22:21:13.697491+0000","last_fullsized":"2024-09-20T22:21:13.697491+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'30","last_scrub_stamp":"2024-09-20T22:18:43.923335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:43.923335+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":464,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.698939+0000","last_change":"2024-09-20T22:18:45.931576+0000","last_active":"2024-09-20T22:21:13.698939+0000","last_peered":"2024-09-20T22:21:13.698939+0000","last_clean":"2024-09-20T22:21:13.698939+0000","last_became_active":"2024-09-20T22:12:52.528947+0000","last_became_peered":"2024-09-20T22:12:52.528947+0000","last_unstale":"2024-09-20T22:21:13.698939+0000","last_undegraded":"2024-09-20T22:21:13.698939+0000","last_fullsized":"2024-09-20T22:21:13.698939+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.931495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.931495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":537,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.691647+0000","last_change":"2024-09-20T22:19:35.720326+0000","last_active":"2024-09-20T22:21:13.691647+0000","last_peered":"2024-09-20T22:21:13.691647+0000","last_clean":"2024-09-20T22:21:13.691647+0000","last_became_active":"2024-09-20T22:19:28.241950+0000","last_became_peered":"2024-09-20T22:19:28.241950+0000","last_unstale":"2024-09-20T22:21:13.691647+0000","last_undegraded":"2024-09-20T22:21:13.691647+0000","last_fullsized":"2024-09-20T22:21:13.691647+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:35.719732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:35.719732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":106,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.691401+0000","last_change":"2024-09-20T22:19:36.736788+0000","last_active":"2024-09-20T22:21:13.691401+0000","last_peered":"2024-09-20T22:21:13.691401+0000","last_clean":"2024-09-20T22:21:13.691401+0000","last_became_active":"2024-09-20T22:19:28.241638+0000","last_became_peered":"2024-09-20T22:19:28.241638+0000","last_unstale":"2024-09-20T22:21:13.691401+0000","last_undegraded":"2024-09-20T22:21:13.691401+0000","last_fullsized":"2024-09-20T22:21:13.691401+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:36.736314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:19:36.736314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.e","version":"516'9383","reported_seq":22479,"reported_epoch":516,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T22:21:16.179721+0000","last_change":"2024-09-20T22:20:59.622989+0000","last_active":"2024-09-20T22:21:16.179721+0000","last_peered":"2024-09-20T22:21:16.179721+0000","last_clean":"2024-09-20T22:19:27.209252+0000","last_became_active":"2024-09-20T22:20:59.583228+0000","last_became_peered":"2024-09-20T22:20:59.583228+0000","last_unstale":"2024-09-20T22:21:16.179721+0000","last_undegraded":"2024-09-20T22:21:16.179721+0000","last_fullsized":"2024-09-20T22:21:16.179721+0000","mapping_epoch":509,"log_start":"493'6213","ondisk_log_start":"493'6213","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":3170,"log_dups_size":0,"ondisk_log_size":3170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":211615744,"num_objects":3229,"num_object_clones":0,"num_object_copies":9687,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":864,"num_objects_unfound":0,"num_objects_dirty":3229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9383,"num_write_kb":403584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6472,"num_bytes_recovered":424148992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,1],"avail_no_missing":["7","0","1"],"object_location_counts":[{"shards":"0,1,7","objects":3229}],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":1027,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.692370+0000","last_change":"2024-09-20T22:19:10.384471+0000","last_active":"2024-09-20T22:21:13.692370+0000","last_peered":"2024-09-20T22:21:13.692370+0000","last_clean":"2024-09-20T22:21:13.692370+0000","last_became_active":"2024-09-20T22:15:45.642671+0000","last_became_peered":"2024-09-20T22:15:45.642671+0000","last_unstale":"2024-09-20T22:21:13.692370+0000","last_undegraded":"2024-09-20T22:21:13.692370+0000","last_fullsized":"2024-09-20T22:21:13.692370+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:10.384343+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:19:10.384343+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"328'28","reported_seq":725,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.688665+0000","last_change":"2024-09-20T22:19:28.268499+0000","last_active":"2024-09-20T22:21:13.688665+0000","last_peered":"2024-09-20T22:21:13.688665+0000","last_clean":"2024-09-20T22:21:13.688665+0000","last_became_active":"2024-09-20T22:19:28.245672+0000","last_became_peered":"2024-09-20T22:19:28.245672+0000","last_unstale":"2024-09-20T22:21:13.688665+0000","last_undegraded":"2024-09-20T22:21:13.688665+0000","last_fullsized":"2024-09-20T22:21:13.688665+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:19:14.334175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:14.334175+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":454,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.695488+0000","last_change":"2024-09-20T22:19:45.873383+0000","last_active":"2024-09-20T22:21:13.695488+0000","last_peered":"2024-09-20T22:21:13.695488+0000","last_clean":"2024-09-20T22:21:13.695488+0000","last_became_active":"2024-09-20T22:19:45.824809+0000","last_became_peered":"2024-09-20T22:19:45.824809+0000","last_unstale":"2024-09-20T22:21:13.695488+0000","last_undegraded":"2024-09-20T22:21:13.695488+0000","last_fullsized":"2024-09-20T22:21:13.695488+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:19:17.876492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:19:17.876492+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913661+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"481'13","reported_seq":766,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.694756+0000","last_change":"2024-09-20T22:19:45.872368+0000","last_active":"2024-09-20T22:21:13.694756+0000","last_peered":"2024-09-20T22:21:13.694756+0000","last_clean":"2024-09-20T22:21:13.694756+0000","last_became_active":"2024-09-20T22:19:45.817931+0000","last_became_peered":"2024-09-20T22:19:45.817931+0000","last_unstale":"2024-09-20T22:21:13.694756+0000","last_undegraded":"2024-09-20T22:21:13.694756+0000","last_fullsized":"2024-09-20T22:21:13.694756+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:18:39.290911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:39.290911+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913467+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":959,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.697935+0000","last_change":"2024-09-20T22:18:11.668610+0000","last_active":"2024-09-20T22:21:13.697935+0000","last_peered":"2024-09-20T22:21:13.697935+0000","last_clean":"2024-09-20T22:21:13.697935+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:21:13.697935+0000","last_undegraded":"2024-09-20T22:21:13.697935+0000","last_fullsized":"2024-09-20T22:21:13.697935+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:11.668512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:11.668512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.d","version":"516'9562","reported_seq":16662,"reported_epoch":516,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-20T22:21:17.004505+0000","last_change":"2024-09-20T22:19:52.892719+0000","last_active":"2024-09-20T22:21:17.004505+0000","last_peered":"2024-09-20T22:21:17.004505+0000","last_clean":"2024-09-20T22:19:27.208215+0000","last_became_active":"2024-09-20T22:19:45.949771+0000","last_became_peered":"2024-09-20T22:19:45.949771+0000","last_unstale":"2024-09-20T22:21:17.004505+0000","last_undegraded":"2024-09-20T22:19:52.886751+0000","last_fullsized":"2024-09-20T22:19:45.774067+0000","mapping_epoch":480,"log_start":"489'5462","ondisk_log_start":"489'5462","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":4100,"log_dups_size":0,"ondisk_log_size":4100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":209256472,"num_objects":3194,"num_object_clones":0,"num_object_copies":9582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":737,"num_objects_misplaced":769,"num_objects_unfound":0,"num_objects_dirty":3194,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9562,"num_write_kb":408129,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3481,"num_bytes_recovered":228130816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6],"avail_no_missing":["2","6"],"object_location_counts":[{"shards":"1,2,6","objects":769},{"shards":"2,6","objects":1688}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":1030,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.695777+0000","last_change":"2024-09-20T22:17:41.753132+0000","last_active":"2024-09-20T22:21:17.695777+0000","last_peered":"2024-09-20T22:21:17.695777+0000","last_clean":"2024-09-20T22:21:17.695777+0000","last_became_active":"2024-09-20T22:17:40.142028+0000","last_became_peered":"2024-09-20T22:17:40.142028+0000","last_unstale":"2024-09-20T22:21:17.695777+0000","last_undegraded":"2024-09-20T22:21:17.695777+0000","last_fullsized":"2024-09-20T22:21:17.695777+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:41.752931+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:41.752931+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"328'20","reported_seq":573,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.688743+0000","last_change":"2024-09-20T22:19:45.875079+0000","last_active":"2024-09-20T22:21:13.688743+0000","last_peered":"2024-09-20T22:21:13.688743+0000","last_clean":"2024-09-20T22:21:13.688743+0000","last_became_active":"2024-09-20T22:19:45.808494+0000","last_became_peered":"2024-09-20T22:19:45.808494+0000","last_unstale":"2024-09-20T22:21:13.688743+0000","last_undegraded":"2024-09-20T22:21:13.688743+0000","last_fullsized":"2024-09-20T22:21:13.688743+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:19:08.320252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:08.320252+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":994,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689318+0000","last_change":"2024-09-20T22:19:44.874011+0000","last_active":"2024-09-20T22:21:13.689318+0000","last_peered":"2024-09-20T22:21:13.689318+0000","last_clean":"2024-09-20T22:21:13.689318+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:21:13.689318+0000","last_undegraded":"2024-09-20T22:21:13.689318+0000","last_fullsized":"2024-09-20T22:21:13.689318+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:44.873944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:19:44.873944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"363'13","reported_seq":526,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.695496+0000","last_change":"2024-09-20T22:20:58.946384+0000","last_active":"2024-09-20T22:21:13.695496+0000","last_peered":"2024-09-20T22:21:13.695496+0000","last_clean":"2024-09-20T22:21:13.695496+0000","last_became_active":"2024-09-20T22:12:57.884415+0000","last_became_peered":"2024-09-20T22:12:57.884415+0000","last_unstale":"2024-09-20T22:21:13.695496+0000","last_undegraded":"2024-09-20T22:21:13.695496+0000","last_fullsized":"2024-09-20T22:21:13.695496+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"363'13","last_scrub_stamp":"2024-09-20T22:20:58.946239+0000","last_deep_scrub":"363'13","last_deep_scrub_stamp":"2024-09-20T22:14:52.612615+0000","last_clean_scrub_stamp":"2024-09-20T22:20:58.946239+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:04.994436+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":423,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.695089+0000","last_change":"2024-09-20T22:21:02.972285+0000","last_active":"2024-09-20T22:21:13.695089+0000","last_peered":"2024-09-20T22:21:13.695089+0000","last_clean":"2024-09-20T22:21:13.695089+0000","last_became_active":"2024-09-20T22:12:52.518529+0000","last_became_peered":"2024-09-20T22:12:52.518529+0000","last_unstale":"2024-09-20T22:21:13.695089+0000","last_undegraded":"2024-09-20T22:21:13.695089+0000","last_fullsized":"2024-09-20T22:21:13.695089+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:02.972172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:21:02.972172+0000","last_clean_scrub_stamp":"2024-09-20T22:21:02.972172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:27.662631+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"516'9678","reported_seq":24916,"reported_epoch":516,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T22:21:17.016643+0000","last_change":"2024-09-20T22:20:57.595947+0000","last_active":"2024-09-20T22:21:17.016643+0000","last_peered":"2024-09-20T22:21:17.016643+0000","last_clean":"2024-09-20T22:19:27.213062+0000","last_became_active":"2024-09-20T22:20:57.570107+0000","last_became_peered":"2024-09-20T22:20:57.570107+0000","last_unstale":"2024-09-20T22:21:17.016643+0000","last_undegraded":"2024-09-20T22:21:17.016643+0000","last_fullsized":"2024-09-20T22:21:17.016643+0000","mapping_epoch":507,"log_start":"491'5647","ondisk_log_start":"491'5647","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":4031,"log_dups_size":0,"ondisk_log_size":4031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":204603392,"num_objects":3122,"num_object_clones":0,"num_object_copies":9366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":78,"num_objects_unfound":0,"num_objects_dirty":3122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9678,"num_write_kb":409600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7544,"num_bytes_recovered":494403584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,1],"avail_no_missing":["2","1","7"],"object_location_counts":[{"shards":"1,2,7","objects":3122}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":1031,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.695320+0000","last_change":"2024-09-20T22:17:40.744592+0000","last_active":"2024-09-20T22:21:17.695320+0000","last_peered":"2024-09-20T22:21:17.695320+0000","last_clean":"2024-09-20T22:21:17.695320+0000","last_became_active":"2024-09-20T22:17:40.139979+0000","last_became_peered":"2024-09-20T22:17:40.139979+0000","last_unstale":"2024-09-20T22:21:17.695320+0000","last_undegraded":"2024-09-20T22:21:17.695320+0000","last_fullsized":"2024-09-20T22:21:17.695320+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:40.744482+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:40.744482+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"392'37","reported_seq":464,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.694455+0000","last_change":"2024-09-20T22:17:38.045665+0000","last_active":"2024-09-20T22:21:13.694455+0000","last_peered":"2024-09-20T22:21:13.694455+0000","last_clean":"2024-09-20T22:21:13.694455+0000","last_became_active":"2024-09-20T22:12:52.525538+0000","last_became_peered":"2024-09-20T22:12:52.525538+0000","last_unstale":"2024-09-20T22:21:13.694455+0000","last_undegraded":"2024-09-20T22:21:13.694455+0000","last_fullsized":"2024-09-20T22:21:13.694455+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"392'37","last_scrub_stamp":"2024-09-20T22:17:38.043643+0000","last_deep_scrub":"392'37","last_deep_scrub_stamp":"2024-09-20T22:15:56.127540+0000","last_clean_scrub_stamp":"2024-09-20T22:17:38.043643+0000","objects_scrubbed":8,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913209+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":400,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.694048+0000","last_change":"2024-09-20T22:19:45.807161+0000","last_active":"2024-09-20T22:21:13.694048+0000","last_peered":"2024-09-20T22:21:13.694048+0000","last_clean":"2024-09-20T22:21:13.694048+0000","last_became_active":"2024-09-20T22:19:45.806472+0000","last_became_peered":"2024-09-20T22:19:45.806472+0000","last_unstale":"2024-09-20T22:21:13.694048+0000","last_undegraded":"2024-09-20T22:21:13.694048+0000","last_fullsized":"2024-09-20T22:21:13.694048+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:43.293797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:43.293797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1008,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.695375+0000","last_change":"2024-09-20T22:19:18.540268+0000","last_active":"2024-09-20T22:21:17.695375+0000","last_peered":"2024-09-20T22:21:17.695375+0000","last_clean":"2024-09-20T22:21:17.695375+0000","last_became_active":"2024-09-20T22:12:52.519826+0000","last_became_peered":"2024-09-20T22:12:52.519826+0000","last_unstale":"2024-09-20T22:21:17.695375+0000","last_undegraded":"2024-09-20T22:21:17.695375+0000","last_fullsized":"2024-09-20T22:21:17.695375+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:18.540166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:19:18.540166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":197,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.698540+0000","last_change":"2024-09-20T22:19:45.792151+0000","last_active":"2024-09-20T22:21:17.698540+0000","last_peered":"2024-09-20T22:21:17.698540+0000","last_clean":"2024-09-20T22:21:17.698540+0000","last_became_active":"2024-09-20T22:19:45.791792+0000","last_became_peered":"2024-09-20T22:19:45.791792+0000","last_unstale":"2024-09-20T22:21:17.698540+0000","last_undegraded":"2024-09-20T22:21:17.698540+0000","last_fullsized":"2024-09-20T22:21:17.698540+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:36.301648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:36.301648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.b","version":"516'9637","reported_seq":18424,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:16.931791+0000","last_change":"2024-09-20T22:21:13.715815+0000","last_active":"2024-09-20T22:21:16.931791+0000","last_peered":"2024-09-20T22:21:16.931791+0000","last_clean":"2024-09-20T22:21:16.931791+0000","last_became_active":"2024-09-20T22:21:13.696433+0000","last_became_peered":"2024-09-20T22:21:13.696433+0000","last_unstale":"2024-09-20T22:21:16.931791+0000","last_undegraded":"2024-09-20T22:21:16.931791+0000","last_fullsized":"2024-09-20T22:21:16.931791+0000","mapping_epoch":515,"log_start":"493'6251","ondisk_log_start":"493'6251","created":456,"last_epoch_clean":516,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":3386,"log_dups_size":0,"ondisk_log_size":3386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222101504,"num_objects":3389,"num_object_clones":0,"num_object_copies":10167,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9637,"num_write_kb":416832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4319,"num_bytes_recovered":283049984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":1003,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.688274+0000","last_change":"2024-09-20T22:18:41.457692+0000","last_active":"2024-09-20T22:21:13.688274+0000","last_peered":"2024-09-20T22:21:13.688274+0000","last_clean":"2024-09-20T22:21:13.688274+0000","last_became_active":"2024-09-20T22:15:45.627135+0000","last_became_peered":"2024-09-20T22:15:45.627135+0000","last_unstale":"2024-09-20T22:21:13.688274+0000","last_undegraded":"2024-09-20T22:21:13.688274+0000","last_fullsized":"2024-09-20T22:21:13.688274+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:18:41.457600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:18:41.457600+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"481'53","reported_seq":688,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.693989+0000","last_change":"2024-09-20T22:19:45.856316+0000","last_active":"2024-09-20T22:21:13.693989+0000","last_peered":"2024-09-20T22:21:13.693989+0000","last_clean":"2024-09-20T22:21:13.693989+0000","last_became_active":"2024-09-20T22:19:45.805924+0000","last_became_peered":"2024-09-20T22:19:45.805924+0000","last_unstale":"2024-09-20T22:21:13.693989+0000","last_undegraded":"2024-09-20T22:21:13.693989+0000","last_fullsized":"2024-09-20T22:21:13.693989+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'52","last_scrub_stamp":"2024-09-20T22:17:08.395578+0000","last_deep_scrub":"328'52","last_deep_scrub_stamp":"2024-09-20T22:17:08.395578+0000","last_clean_scrub_stamp":"2024-09-20T22:17:08.395578+0000","objects_scrubbed":10,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":32,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":64,"num_read_kb":49,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":677,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.695646+0000","last_change":"2024-09-20T22:19:45.811322+0000","last_active":"2024-09-20T22:21:13.695646+0000","last_peered":"2024-09-20T22:21:13.695646+0000","last_clean":"2024-09-20T22:21:13.695646+0000","last_became_active":"2024-09-20T22:19:45.810511+0000","last_became_peered":"2024-09-20T22:19:45.810511+0000","last_unstale":"2024-09-20T22:21:13.695646+0000","last_undegraded":"2024-09-20T22:21:13.695646+0000","last_fullsized":"2024-09-20T22:21:13.695646+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:28.386523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:28.386523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":992,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689026+0000","last_change":"2024-09-20T22:19:45.824354+0000","last_active":"2024-09-20T22:21:13.689026+0000","last_peered":"2024-09-20T22:21:13.689026+0000","last_clean":"2024-09-20T22:21:13.689026+0000","last_became_active":"2024-09-20T22:19:45.808162+0000","last_became_peered":"2024-09-20T22:19:45.808162+0000","last_unstale":"2024-09-20T22:21:13.689026+0000","last_undegraded":"2024-09-20T22:21:13.689026+0000","last_fullsized":"2024-09-20T22:21:13.689026+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:17:06.944406+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_clean_scrub_stamp":"2024-09-20T22:17:06.944406+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":435,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.694181+0000","last_change":"2024-09-20T22:19:45.815814+0000","last_active":"2024-09-20T22:21:13.694181+0000","last_peered":"2024-09-20T22:21:13.694181+0000","last_clean":"2024-09-20T22:21:13.694181+0000","last_became_active":"2024-09-20T22:19:45.815528+0000","last_became_peered":"2024-09-20T22:19:45.815528+0000","last_unstale":"2024-09-20T22:21:13.694181+0000","last_undegraded":"2024-09-20T22:21:13.694181+0000","last_fullsized":"2024-09-20T22:21:13.694181+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:57.210706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.210706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913351+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":980,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.694578+0000","last_change":"2024-09-20T22:19:43.806944+0000","last_active":"2024-09-20T22:21:13.694578+0000","last_peered":"2024-09-20T22:21:13.694578+0000","last_clean":"2024-09-20T22:21:13.694578+0000","last_became_active":"2024-09-20T22:19:28.238492+0000","last_became_peered":"2024-09-20T22:19:28.238492+0000","last_unstale":"2024-09-20T22:21:13.694578+0000","last_undegraded":"2024-09-20T22:21:13.694578+0000","last_fullsized":"2024-09-20T22:21:13.694578+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:43.806828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:19:43.806828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.8","version":"517'9624","reported_seq":9758,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.897464+0000","last_change":"2024-09-20T22:18:46.633457+0000","last_active":"2024-09-20T22:21:17.897464+0000","last_peered":"2024-09-20T22:21:17.897464+0000","last_clean":"2024-09-20T22:21:17.897464+0000","last_became_active":"2024-09-20T22:18:46.631872+0000","last_became_peered":"2024-09-20T22:18:46.631872+0000","last_unstale":"2024-09-20T22:21:17.897464+0000","last_undegraded":"2024-09-20T22:21:17.897464+0000","last_fullsized":"2024-09-20T22:21:17.897464+0000","mapping_epoch":456,"log_start":"493'6193","ondisk_log_start":"493'6193","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":3431,"log_dups_size":0,"ondisk_log_size":3431,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":203161600,"num_objects":3100,"num_object_clones":0,"num_object_copies":9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9624,"num_write_kb":407168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":1010,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.697509+0000","last_change":"2024-09-20T22:19:31.274033+0000","last_active":"2024-09-20T22:21:13.697509+0000","last_peered":"2024-09-20T22:21:13.697509+0000","last_clean":"2024-09-20T22:21:13.697509+0000","last_became_active":"2024-09-20T22:19:31.273654+0000","last_became_peered":"2024-09-20T22:19:31.273654+0000","last_unstale":"2024-09-20T22:21:13.697509+0000","last_undegraded":"2024-09-20T22:21:13.697509+0000","last_fullsized":"2024-09-20T22:21:13.697509+0000","mapping_epoch":475,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":476,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:38.946300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:38.946300+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"328'17","reported_seq":576,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.690321+0000","last_change":"2024-09-20T22:19:45.884804+0000","last_active":"2024-09-20T22:21:13.690321+0000","last_peered":"2024-09-20T22:21:13.690321+0000","last_clean":"2024-09-20T22:21:13.690321+0000","last_became_active":"2024-09-20T22:19:45.810715+0000","last_became_peered":"2024-09-20T22:19:45.810715+0000","last_unstale":"2024-09-20T22:21:13.690321+0000","last_undegraded":"2024-09-20T22:21:13.690321+0000","last_fullsized":"2024-09-20T22:21:13.690321+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'17","last_scrub_stamp":"2024-09-20T22:17:01.307755+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:01.307755+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":23,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":196,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.692633+0000","last_change":"2024-09-20T22:19:45.810047+0000","last_active":"2024-09-20T22:21:13.692633+0000","last_peered":"2024-09-20T22:21:13.692633+0000","last_clean":"2024-09-20T22:21:13.692633+0000","last_became_active":"2024-09-20T22:19:45.807340+0000","last_became_peered":"2024-09-20T22:19:45.807340+0000","last_unstale":"2024-09-20T22:21:13.692633+0000","last_undegraded":"2024-09-20T22:21:13.692633+0000","last_fullsized":"2024-09-20T22:21:13.692633+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:09.478390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:09.478390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":1040,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.692328+0000","last_change":"2024-09-20T22:19:13.810454+0000","last_active":"2024-09-20T22:21:13.692328+0000","last_peered":"2024-09-20T22:21:13.692328+0000","last_clean":"2024-09-20T22:21:13.692328+0000","last_became_active":"2024-09-20T22:12:52.517255+0000","last_became_peered":"2024-09-20T22:12:52.517255+0000","last_unstale":"2024-09-20T22:21:13.692328+0000","last_undegraded":"2024-09-20T22:21:13.692328+0000","last_fullsized":"2024-09-20T22:21:13.692328+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:19:13.810352+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:19:13.810352+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":994,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.691393+0000","last_change":"2024-09-20T22:19:08.895449+0000","last_active":"2024-09-20T22:21:13.691393+0000","last_peered":"2024-09-20T22:21:13.691393+0000","last_clean":"2024-09-20T22:21:13.691393+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:21:13.691393+0000","last_undegraded":"2024-09-20T22:21:13.691393+0000","last_fullsized":"2024-09-20T22:21:13.691393+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:08.894168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:19:08.894168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":959,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.698337+0000","last_change":"2024-09-20T22:18:35.868334+0000","last_active":"2024-09-20T22:21:13.698337+0000","last_peered":"2024-09-20T22:21:13.698337+0000","last_clean":"2024-09-20T22:21:13.698337+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:21:13.698337+0000","last_undegraded":"2024-09-20T22:21:13.698337+0000","last_fullsized":"2024-09-20T22:21:13.698337+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:35.868264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:35.868264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.9","version":"516'9785","reported_seq":9917,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:16.918100+0000","last_change":"2024-09-20T22:18:46.630336+0000","last_active":"2024-09-20T22:21:16.918100+0000","last_peered":"2024-09-20T22:21:16.918100+0000","last_clean":"2024-09-20T22:21:16.918100+0000","last_became_active":"2024-09-20T22:18:46.630028+0000","last_became_peered":"2024-09-20T22:18:46.630028+0000","last_unstale":"2024-09-20T22:21:16.918100+0000","last_undegraded":"2024-09-20T22:21:16.918100+0000","last_fullsized":"2024-09-20T22:21:16.918100+0000","mapping_epoch":456,"log_start":"493'6330","ondisk_log_start":"493'6330","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":3455,"log_dups_size":0,"ondisk_log_size":3455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":206372864,"num_objects":3149,"num_object_clones":0,"num_object_copies":9447,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9785,"num_write_kb":413888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":373,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.694716+0000","last_change":"2024-09-20T22:19:48.844171+0000","last_active":"2024-09-20T22:21:17.694716+0000","last_peered":"2024-09-20T22:21:17.694716+0000","last_clean":"2024-09-20T22:21:17.694716+0000","last_became_active":"2024-09-20T22:19:48.843794+0000","last_became_peered":"2024-09-20T22:19:48.843794+0000","last_unstale":"2024-09-20T22:21:17.694716+0000","last_undegraded":"2024-09-20T22:21:17.694716+0000","last_fullsized":"2024-09-20T22:21:17.694716+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:18:01.139707+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:18:01.139707+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:19.906534+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17890,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":9,"num_write":5,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":53670,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"328'24","reported_seq":474,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.695251+0000","last_change":"2024-09-20T22:19:45.884736+0000","last_active":"2024-09-20T22:21:13.695251+0000","last_peered":"2024-09-20T22:21:13.695251+0000","last_clean":"2024-09-20T22:21:13.695251+0000","last_became_active":"2024-09-20T22:19:45.819406+0000","last_became_peered":"2024-09-20T22:19:45.819406+0000","last_unstale":"2024-09-20T22:21:13.695251+0000","last_undegraded":"2024-09-20T22:21:13.695251+0000","last_fullsized":"2024-09-20T22:21:13.695251+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'24","last_scrub_stamp":"2024-09-20T22:17:05.822650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:05.822650+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913013+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":429,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.693483+0000","last_change":"2024-09-20T22:17:11.814549+0000","last_active":"2024-09-20T22:21:13.693483+0000","last_peered":"2024-09-20T22:21:13.693483+0000","last_clean":"2024-09-20T22:21:13.693483+0000","last_became_active":"2024-09-20T22:17:03.663870+0000","last_became_peered":"2024-09-20T22:17:03.663870+0000","last_unstale":"2024-09-20T22:21:13.693483+0000","last_undegraded":"2024-09-20T22:21:13.693483+0000","last_fullsized":"2024-09-20T22:21:13.693483+0000","mapping_epoch":415,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":416,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:11.814439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:17:11.814439+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913039+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":2109,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:14.246732+0000","last_change":"2024-09-20T22:19:45.875808+0000","last_active":"2024-09-20T22:21:14.246732+0000","last_peered":"2024-09-20T22:21:14.246732+0000","last_clean":"2024-09-20T22:21:14.246732+0000","last_became_active":"2024-09-20T22:19:45.817469+0000","last_became_peered":"2024-09-20T22:19:45.817469+0000","last_unstale":"2024-09-20T22:21:14.246732+0000","last_undegraded":"2024-09-20T22:21:14.246732+0000","last_fullsized":"2024-09-20T22:21:14.246732+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:17:59.765217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:17:59.765217+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1077,"num_read_kb":1072,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":523,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689561+0000","last_change":"2024-09-20T22:19:45.811058+0000","last_active":"2024-09-20T22:21:13.689561+0000","last_peered":"2024-09-20T22:21:13.689561+0000","last_clean":"2024-09-20T22:21:13.689561+0000","last_became_active":"2024-09-20T22:19:45.810608+0000","last_became_peered":"2024-09-20T22:19:45.810608+0000","last_unstale":"2024-09-20T22:21:13.689561+0000","last_undegraded":"2024-09-20T22:21:13.689561+0000","last_fullsized":"2024-09-20T22:21:13.689561+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:08.955790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:17:08.955790+0000","last_clean_scrub_stamp":"2024-09-20T22:17:08.955790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":195,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689699+0000","last_change":"2024-09-20T22:19:45.790117+0000","last_active":"2024-09-20T22:21:13.689699+0000","last_peered":"2024-09-20T22:21:13.689699+0000","last_clean":"2024-09-20T22:21:13.689699+0000","last_became_active":"2024-09-20T22:19:45.789135+0000","last_became_peered":"2024-09-20T22:19:45.789135+0000","last_unstale":"2024-09-20T22:21:13.689699+0000","last_undegraded":"2024-09-20T22:21:13.689699+0000","last_fullsized":"2024-09-20T22:21:13.689699+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:40.325308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:40.325308+0000","last_clean_scrub_stamp":"2024-09-20T22:18:40.325308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.a","version":"517'9769","reported_seq":9903,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.898145+0000","last_change":"2024-09-20T22:18:46.631467+0000","last_active":"2024-09-20T22:21:17.898145+0000","last_peered":"2024-09-20T22:21:17.898145+0000","last_clean":"2024-09-20T22:21:17.898145+0000","last_became_active":"2024-09-20T22:18:46.630294+0000","last_became_peered":"2024-09-20T22:18:46.630294+0000","last_unstale":"2024-09-20T22:21:17.898145+0000","last_undegraded":"2024-09-20T22:21:17.898145+0000","last_fullsized":"2024-09-20T22:21:17.898145+0000","mapping_epoch":456,"log_start":"493'6265","ondisk_log_start":"493'6265","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":3504,"log_dups_size":0,"ondisk_log_size":3504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":200736768,"num_objects":3063,"num_object_clones":0,"num_object_copies":9189,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9769,"num_write_kb":410624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"475'29","reported_seq":1270,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.695599+0000","last_change":"2024-09-20T22:19:28.285619+0000","last_active":"2024-09-20T22:21:17.695599+0000","last_peered":"2024-09-20T22:21:17.695599+0000","last_clean":"2024-09-20T22:21:17.695599+0000","last_became_active":"2024-09-20T22:19:28.259634+0000","last_became_peered":"2024-09-20T22:19:28.259634+0000","last_unstale":"2024-09-20T22:21:17.695599+0000","last_undegraded":"2024-09-20T22:21:17.695599+0000","last_fullsized":"2024-09-20T22:21:17.695599+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:17:57.685068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.685068+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":931,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.688633+0000","last_change":"2024-09-20T22:19:11.606709+0000","last_active":"2024-09-20T22:21:13.688633+0000","last_peered":"2024-09-20T22:21:13.688633+0000","last_clean":"2024-09-20T22:21:13.688633+0000","last_became_active":"2024-09-20T22:12:21.519717+0000","last_became_peered":"2024-09-20T22:12:21.519717+0000","last_unstale":"2024-09-20T22:21:13.688633+0000","last_undegraded":"2024-09-20T22:21:13.688633+0000","last_fullsized":"2024-09-20T22:21:13.688633+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:11.606601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:36.729158+0000","last_clean_scrub_stamp":"2024-09-20T22:19:11.606601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":1001,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.697284+0000","last_change":"2024-09-20T22:18:06.776685+0000","last_active":"2024-09-20T22:21:13.697284+0000","last_peered":"2024-09-20T22:21:13.697284+0000","last_clean":"2024-09-20T22:21:13.697284+0000","last_became_active":"2024-09-20T22:15:45.620972+0000","last_became_peered":"2024-09-20T22:15:45.620972+0000","last_unstale":"2024-09-20T22:21:13.697284+0000","last_undegraded":"2024-09-20T22:21:13.697284+0000","last_fullsized":"2024-09-20T22:21:13.697284+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:18:06.776546+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:18:06.776546+0000","last_clean_scrub_stamp":"2024-09-20T22:18:06.776546+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":554,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.698759+0000","last_change":"2024-09-20T22:18:04.809849+0000","last_active":"2024-09-20T22:21:13.698759+0000","last_peered":"2024-09-20T22:21:13.698759+0000","last_clean":"2024-09-20T22:21:13.698759+0000","last_became_active":"2024-09-20T22:10:48.907271+0000","last_became_peered":"2024-09-20T22:10:48.907271+0000","last_unstale":"2024-09-20T22:21:13.698759+0000","last_undegraded":"2024-09-20T22:21:13.698759+0000","last_fullsized":"2024-09-20T22:21:13.698759+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:04.809565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_clean_scrub_stamp":"2024-09-20T22:18:04.809565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":998,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.690232+0000","last_change":"2024-09-20T22:19:45.799513+0000","last_active":"2024-09-20T22:21:13.690232+0000","last_peered":"2024-09-20T22:21:13.690232+0000","last_clean":"2024-09-20T22:21:13.690232+0000","last_became_active":"2024-09-20T22:19:45.797853+0000","last_became_peered":"2024-09-20T22:19:45.797853+0000","last_unstale":"2024-09-20T22:21:13.690232+0000","last_undegraded":"2024-09-20T22:21:13.690232+0000","last_fullsized":"2024-09-20T22:21:13.690232+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:13.654339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:19:13.654339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1","version":"517'9709","reported_seq":9843,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.911803+0000","last_change":"2024-09-20T22:18:46.630204+0000","last_active":"2024-09-20T22:21:17.911803+0000","last_peered":"2024-09-20T22:21:17.911803+0000","last_clean":"2024-09-20T22:21:17.911803+0000","last_became_active":"2024-09-20T22:18:46.629898+0000","last_became_peered":"2024-09-20T22:18:46.629898+0000","last_unstale":"2024-09-20T22:21:17.911803+0000","last_undegraded":"2024-09-20T22:21:17.911803+0000","last_fullsized":"2024-09-20T22:21:17.911803+0000","mapping_epoch":456,"log_start":"493'6235","ondisk_log_start":"493'6235","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":3474,"log_dups_size":0,"ondisk_log_size":3474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199557120,"num_objects":3045,"num_object_clones":0,"num_object_copies":9135,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9709,"num_write_kb":408128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.b","version":"75'128","reported_seq":793,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.695607+0000","last_change":"2024-09-20T22:20:59.964237+0000","last_active":"2024-09-20T22:21:13.695607+0000","last_peered":"2024-09-20T22:21:13.695607+0000","last_clean":"2024-09-20T22:21:13.695607+0000","last_became_active":"2024-09-20T22:19:48.848609+0000","last_became_peered":"2024-09-20T22:19:48.848609+0000","last_unstale":"2024-09-20T22:21:13.695607+0000","last_undegraded":"2024-09-20T22:21:13.695607+0000","last_fullsized":"2024-09-20T22:21:13.695607+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:20:59.964107+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:20:59.964107+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:22.269663+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"328'19","reported_seq":731,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.695466+0000","last_change":"2024-09-20T22:19:45.806692+0000","last_active":"2024-09-20T22:21:17.695466+0000","last_peered":"2024-09-20T22:21:17.695466+0000","last_clean":"2024-09-20T22:21:17.695466+0000","last_became_active":"2024-09-20T22:19:45.786395+0000","last_became_peered":"2024-09-20T22:19:45.786395+0000","last_unstale":"2024-09-20T22:21:17.695466+0000","last_undegraded":"2024-09-20T22:21:17.695466+0000","last_fullsized":"2024-09-20T22:21:17.695466+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:18:36.321653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:36.321653+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":104,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.692141+0000","last_change":"2024-09-20T22:19:45.802221+0000","last_active":"2024-09-20T22:21:13.692141+0000","last_peered":"2024-09-20T22:21:13.692141+0000","last_clean":"2024-09-20T22:21:13.692141+0000","last_became_active":"2024-09-20T22:19:45.800297+0000","last_became_peered":"2024-09-20T22:19:45.800297+0000","last_unstale":"2024-09-20T22:21:13.692141+0000","last_undegraded":"2024-09-20T22:21:13.692141+0000","last_fullsized":"2024-09-20T22:21:13.692141+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:36.374371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:36.374371+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":984,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.690433+0000","last_change":"2024-09-20T22:19:28.241963+0000","last_active":"2024-09-20T22:21:13.690433+0000","last_peered":"2024-09-20T22:21:13.690433+0000","last_clean":"2024-09-20T22:21:13.690433+0000","last_became_active":"2024-09-20T22:19:28.241278+0000","last_became_peered":"2024-09-20T22:19:28.241278+0000","last_unstale":"2024-09-20T22:21:13.690433+0000","last_undegraded":"2024-09-20T22:21:13.690433+0000","last_fullsized":"2024-09-20T22:21:13.690433+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:04.545147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:04.545147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":928,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.695708+0000","last_change":"2024-09-20T22:17:55.630149+0000","last_active":"2024-09-20T22:21:17.695708+0000","last_peered":"2024-09-20T22:21:17.695708+0000","last_clean":"2024-09-20T22:21:17.695708+0000","last_became_active":"2024-09-20T22:12:21.521195+0000","last_became_peered":"2024-09-20T22:12:21.521195+0000","last_unstale":"2024-09-20T22:21:17.695708+0000","last_undegraded":"2024-09-20T22:21:17.695708+0000","last_fullsized":"2024-09-20T22:21:17.695708+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:55.630084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:55.630084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"516'9488","reported_seq":9620,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.012933+0000","last_change":"2024-09-20T22:18:46.631808+0000","last_active":"2024-09-20T22:21:17.012933+0000","last_peered":"2024-09-20T22:21:17.012933+0000","last_clean":"2024-09-20T22:21:17.012933+0000","last_became_active":"2024-09-20T22:18:46.631505+0000","last_became_peered":"2024-09-20T22:18:46.631505+0000","last_unstale":"2024-09-20T22:21:17.012933+0000","last_undegraded":"2024-09-20T22:21:17.012933+0000","last_fullsized":"2024-09-20T22:21:17.012933+0000","mapping_epoch":456,"log_start":"489'5458","ondisk_log_start":"489'5458","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":4030,"log_dups_size":0,"ondisk_log_size":4030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":206700544,"num_objects":3154,"num_object_clones":0,"num_object_copies":9462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9488,"num_write_kb":404544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":754,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.696438+0000","last_change":"2024-09-20T22:17:54.588617+0000","last_active":"2024-09-20T22:21:17.696438+0000","last_peered":"2024-09-20T22:21:17.696438+0000","last_clean":"2024-09-20T22:21:17.696438+0000","last_became_active":"2024-09-20T22:17:40.139875+0000","last_became_peered":"2024-09-20T22:17:40.139875+0000","last_unstale":"2024-09-20T22:21:17.696438+0000","last_undegraded":"2024-09-20T22:21:17.696438+0000","last_fullsized":"2024-09-20T22:21:17.696438+0000","mapping_epoch":358,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:54.588504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:54.588504+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"328'31","reported_seq":759,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689745+0000","last_change":"2024-09-20T22:19:45.976627+0000","last_active":"2024-09-20T22:21:13.689745+0000","last_peered":"2024-09-20T22:21:13.689745+0000","last_clean":"2024-09-20T22:21:13.689745+0000","last_became_active":"2024-09-20T22:19:45.833530+0000","last_became_peered":"2024-09-20T22:19:45.833530+0000","last_unstale":"2024-09-20T22:21:13.689745+0000","last_undegraded":"2024-09-20T22:21:13.689745+0000","last_fullsized":"2024-09-20T22:21:13.689745+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'31","last_scrub_stamp":"2024-09-20T22:19:12.269851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:12.269851+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1009,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.696366+0000","last_change":"2024-09-20T22:19:45.798910+0000","last_active":"2024-09-20T22:21:17.696366+0000","last_peered":"2024-09-20T22:21:17.696366+0000","last_clean":"2024-09-20T22:21:17.696366+0000","last_became_active":"2024-09-20T22:19:45.798363+0000","last_became_peered":"2024-09-20T22:19:45.798363+0000","last_unstale":"2024-09-20T22:21:17.696366+0000","last_undegraded":"2024-09-20T22:21:17.696366+0000","last_fullsized":"2024-09-20T22:21:17.696366+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:12.437986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:17:08.846904+0000","last_clean_scrub_stamp":"2024-09-20T22:19:12.437986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":812,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.693647+0000","last_change":"2024-09-20T22:19:45.809454+0000","last_active":"2024-09-20T22:21:13.693647+0000","last_peered":"2024-09-20T22:21:13.693647+0000","last_clean":"2024-09-20T22:21:13.693647+0000","last_became_active":"2024-09-20T22:19:45.808291+0000","last_became_peered":"2024-09-20T22:19:45.808291+0000","last_unstale":"2024-09-20T22:21:13.693647+0000","last_undegraded":"2024-09-20T22:21:13.693647+0000","last_fullsized":"2024-09-20T22:21:13.693647+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:10.179939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:10.179939+0000","last_clean_scrub_stamp":"2024-09-20T22:18:10.179939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913263+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":988,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.693303+0000","last_change":"2024-09-20T22:19:28.262035+0000","last_active":"2024-09-20T22:21:13.693303+0000","last_peered":"2024-09-20T22:21:13.693303+0000","last_clean":"2024-09-20T22:21:13.693303+0000","last_became_active":"2024-09-20T22:19:28.261628+0000","last_became_peered":"2024-09-20T22:19:28.261628+0000","last_unstale":"2024-09-20T22:21:13.693303+0000","last_undegraded":"2024-09-20T22:21:13.693303+0000","last_fullsized":"2024-09-20T22:21:13.693303+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:39.807978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:39.807978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"517'9545","reported_seq":20515,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.876428+0000","last_change":"2024-09-20T22:20:52.556884+0000","last_active":"2024-09-20T22:21:17.876428+0000","last_peered":"2024-09-20T22:21:17.876428+0000","last_clean":"2024-09-20T22:21:17.876428+0000","last_became_active":"2024-09-20T22:20:52.545710+0000","last_became_peered":"2024-09-20T22:20:52.545710+0000","last_unstale":"2024-09-20T22:21:17.876428+0000","last_undegraded":"2024-09-20T22:21:17.876428+0000","last_fullsized":"2024-09-20T22:21:17.876428+0000","mapping_epoch":504,"log_start":"493'6110","ondisk_log_start":"493'6110","created":456,"last_epoch_clean":505,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":3435,"log_dups_size":0,"ondisk_log_size":3435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":200998912,"num_objects":3067,"num_object_clones":0,"num_object_copies":9201,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3067,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9545,"num_write_kb":403584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5410,"num_bytes_recovered":354549760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":1262,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.697753+0000","last_change":"2024-09-20T22:17:42.025688+0000","last_active":"2024-09-20T22:21:13.697753+0000","last_peered":"2024-09-20T22:21:13.697753+0000","last_clean":"2024-09-20T22:21:13.697753+0000","last_became_active":"2024-09-20T22:17:40.146794+0000","last_became_peered":"2024-09-20T22:17:40.146794+0000","last_unstale":"2024-09-20T22:21:13.697753+0000","last_undegraded":"2024-09-20T22:21:13.697753+0000","last_fullsized":"2024-09-20T22:21:13.697753+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:42.025576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:42.025576+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"328'25","reported_seq":726,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.695342+0000","last_change":"2024-09-20T22:19:45.814785+0000","last_active":"2024-09-20T22:21:17.695342+0000","last_peered":"2024-09-20T22:21:17.695342+0000","last_clean":"2024-09-20T22:21:17.695342+0000","last_became_active":"2024-09-20T22:19:45.791116+0000","last_became_peered":"2024-09-20T22:19:45.791116+0000","last_unstale":"2024-09-20T22:21:17.695342+0000","last_undegraded":"2024-09-20T22:21:17.695342+0000","last_fullsized":"2024-09-20T22:21:17.695342+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'25","last_scrub_stamp":"2024-09-20T22:18:05.300862+0000","last_deep_scrub":"328'25","last_deep_scrub_stamp":"2024-09-20T22:18:05.300862+0000","last_clean_scrub_stamp":"2024-09-20T22:18:05.300862+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":446,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.694549+0000","last_change":"2024-09-20T22:19:45.864178+0000","last_active":"2024-09-20T22:21:13.694549+0000","last_peered":"2024-09-20T22:21:13.694549+0000","last_clean":"2024-09-20T22:21:13.694549+0000","last_became_active":"2024-09-20T22:19:45.810979+0000","last_became_peered":"2024-09-20T22:19:45.810979+0000","last_unstale":"2024-09-20T22:21:13.694549+0000","last_undegraded":"2024-09-20T22:21:13.694549+0000","last_fullsized":"2024-09-20T22:21:13.694549+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:17:35.054116+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:17:35.054116+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913178+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"283'11","reported_seq":1253,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689666+0000","last_change":"2024-09-20T22:19:11.823449+0000","last_active":"2024-09-20T22:21:13.689666+0000","last_peered":"2024-09-20T22:21:13.689666+0000","last_clean":"2024-09-20T22:21:13.689666+0000","last_became_active":"2024-09-20T22:11:30.485356+0000","last_became_peered":"2024-09-20T22:11:30.485356+0000","last_unstale":"2024-09-20T22:21:13.689666+0000","last_undegraded":"2024-09-20T22:21:13.689666+0000","last_fullsized":"2024-09-20T22:21:13.689666+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'11","last_scrub_stamp":"2024-09-20T22:19:11.823322+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:19:11.823322+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":248,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.690855+0000","last_change":"2024-09-20T22:19:45.804591+0000","last_active":"2024-09-20T22:21:13.690855+0000","last_peered":"2024-09-20T22:21:13.690855+0000","last_clean":"2024-09-20T22:21:13.690855+0000","last_became_active":"2024-09-20T22:19:45.799390+0000","last_became_peered":"2024-09-20T22:19:45.799390+0000","last_unstale":"2024-09-20T22:21:13.690855+0000","last_undegraded":"2024-09-20T22:21:13.690855+0000","last_fullsized":"2024-09-20T22:21:13.690855+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:11.319469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:19:11.319469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.4","version":"516'9349","reported_seq":9481,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:16.155752+0000","last_change":"2024-09-20T22:18:46.634428+0000","last_active":"2024-09-20T22:21:16.155752+0000","last_peered":"2024-09-20T22:21:16.155752+0000","last_clean":"2024-09-20T22:21:16.155752+0000","last_became_active":"2024-09-20T22:18:46.633985+0000","last_became_peered":"2024-09-20T22:18:46.633985+0000","last_unstale":"2024-09-20T22:21:16.155752+0000","last_undegraded":"2024-09-20T22:21:16.155752+0000","last_fullsized":"2024-09-20T22:21:16.155752+0000","mapping_epoch":456,"log_start":"493'6236","ondisk_log_start":"493'6236","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":3113,"log_dups_size":0,"ondisk_log_size":3113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":206635008,"num_objects":3153,"num_object_clones":0,"num_object_copies":9459,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3153,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9349,"num_write_kb":400064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.e","version":"75'128","reported_seq":676,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.699417+0000","last_change":"2024-09-20T22:19:31.259361+0000","last_active":"2024-09-20T22:21:13.699417+0000","last_peered":"2024-09-20T22:21:13.699417+0000","last_clean":"2024-09-20T22:21:13.699417+0000","last_became_active":"2024-09-20T22:19:31.258800+0000","last_became_peered":"2024-09-20T22:19:31.258800+0000","last_unstale":"2024-09-20T22:21:13.699417+0000","last_undegraded":"2024-09-20T22:21:13.699417+0000","last_fullsized":"2024-09-20T22:21:13.699417+0000","mapping_epoch":475,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":476,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:41.370226+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:41.370226+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:14.596125+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"512'19","reported_seq":485,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.694359+0000","last_change":"2024-09-20T22:21:05.960413+0000","last_active":"2024-09-20T22:21:13.694359+0000","last_peered":"2024-09-20T22:21:13.694359+0000","last_clean":"2024-09-20T22:21:13.694359+0000","last_became_active":"2024-09-20T22:19:45.820510+0000","last_became_peered":"2024-09-20T22:19:45.820510+0000","last_unstale":"2024-09-20T22:21:13.694359+0000","last_undegraded":"2024-09-20T22:21:13.694359+0000","last_fullsized":"2024-09-20T22:21:13.694359+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"512'19","last_scrub_stamp":"2024-09-20T22:21:05.960284+0000","last_deep_scrub":"512'19","last_deep_scrub_stamp":"2024-09-20T22:21:05.960284+0000","last_clean_scrub_stamp":"2024-09-20T22:21:05.960284+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:20.601201+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":85,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689586+0000","last_change":"2024-09-20T22:19:45.801745+0000","last_active":"2024-09-20T22:21:13.689586+0000","last_peered":"2024-09-20T22:21:13.689586+0000","last_clean":"2024-09-20T22:21:13.689586+0000","last_became_active":"2024-09-20T22:19:45.788623+0000","last_became_peered":"2024-09-20T22:19:45.788623+0000","last_unstale":"2024-09-20T22:21:13.689586+0000","last_undegraded":"2024-09-20T22:21:13.689586+0000","last_fullsized":"2024-09-20T22:21:13.689586+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:19:37.837883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:19:37.837883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1000,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.690228+0000","last_change":"2024-09-20T22:19:28.247348+0000","last_active":"2024-09-20T22:21:13.690228+0000","last_peered":"2024-09-20T22:21:13.690228+0000","last_clean":"2024-09-20T22:21:13.690228+0000","last_became_active":"2024-09-20T22:19:28.246512+0000","last_became_peered":"2024-09-20T22:19:28.246512+0000","last_unstale":"2024-09-20T22:21:13.690228+0000","last_undegraded":"2024-09-20T22:21:13.690228+0000","last_fullsized":"2024-09-20T22:21:13.690228+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:40.088194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:18:40.088194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":428,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.693387+0000","last_change":"2024-09-20T22:19:45.813316+0000","last_active":"2024-09-20T22:21:13.693387+0000","last_peered":"2024-09-20T22:21:13.693387+0000","last_clean":"2024-09-20T22:21:13.693387+0000","last_became_active":"2024-09-20T22:19:45.813003+0000","last_became_peered":"2024-09-20T22:19:45.813003+0000","last_unstale":"2024-09-20T22:21:13.693387+0000","last_undegraded":"2024-09-20T22:21:13.693387+0000","last_fullsized":"2024-09-20T22:21:13.693387+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:12.806158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:19:12.806158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913691+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.5","version":"516'9772","reported_seq":9900,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:16.156486+0000","last_change":"2024-09-20T22:18:46.633877+0000","last_active":"2024-09-20T22:21:16.156486+0000","last_peered":"2024-09-20T22:21:16.156486+0000","last_clean":"2024-09-20T22:21:16.156486+0000","last_became_active":"2024-09-20T22:18:46.633490+0000","last_became_peered":"2024-09-20T22:18:46.633490+0000","last_unstale":"2024-09-20T22:21:16.156486+0000","last_undegraded":"2024-09-20T22:21:16.156486+0000","last_fullsized":"2024-09-20T22:21:16.156486+0000","mapping_epoch":456,"log_start":"493'6591","ondisk_log_start":"493'6591","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":3181,"log_dups_size":0,"ondisk_log_size":3181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":222035968,"num_objects":3388,"num_object_clones":0,"num_object_copies":10164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9772,"num_write_kb":421120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.f","version":"75'128","reported_seq":1042,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.696106+0000","last_change":"2024-09-20T22:19:31.262868+0000","last_active":"2024-09-20T22:21:17.696106+0000","last_peered":"2024-09-20T22:21:17.696106+0000","last_clean":"2024-09-20T22:21:17.696106+0000","last_became_active":"2024-09-20T22:19:31.262445+0000","last_became_peered":"2024-09-20T22:19:31.262445+0000","last_unstale":"2024-09-20T22:21:17.696106+0000","last_undegraded":"2024-09-20T22:21:17.696106+0000","last_fullsized":"2024-09-20T22:21:17.696106+0000","mapping_epoch":475,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":476,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:11.475678+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:17:07.885346+0000","last_clean_scrub_stamp":"2024-09-20T22:19:11.475678+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"328'35","reported_seq":1118,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.696013+0000","last_change":"2024-09-20T22:19:45.813869+0000","last_active":"2024-09-20T22:21:17.696013+0000","last_peered":"2024-09-20T22:21:17.696013+0000","last_clean":"2024-09-20T22:21:17.696013+0000","last_became_active":"2024-09-20T22:19:45.790684+0000","last_became_peered":"2024-09-20T22:19:45.790684+0000","last_unstale":"2024-09-20T22:21:17.696013+0000","last_undegraded":"2024-09-20T22:21:17.696013+0000","last_fullsized":"2024-09-20T22:21:17.696013+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'35","last_scrub_stamp":"2024-09-20T22:19:16.424262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:16.424262+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":983,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.698775+0000","last_change":"2024-09-20T22:17:37.950739+0000","last_active":"2024-09-20T22:21:13.698775+0000","last_peered":"2024-09-20T22:21:13.698775+0000","last_clean":"2024-09-20T22:21:13.698775+0000","last_became_active":"2024-09-20T22:11:30.488650+0000","last_became_peered":"2024-09-20T22:11:30.488650+0000","last_unstale":"2024-09-20T22:21:13.698775+0000","last_undegraded":"2024-09-20T22:21:13.698775+0000","last_fullsized":"2024-09-20T22:21:13.698775+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:37.950674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:12:46.602035+0000","last_clean_scrub_stamp":"2024-09-20T22:17:37.950674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1018,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.691887+0000","last_change":"2024-09-20T22:18:11.941197+0000","last_active":"2024-09-20T22:21:13.691887+0000","last_peered":"2024-09-20T22:21:13.691887+0000","last_clean":"2024-09-20T22:21:13.691887+0000","last_became_active":"2024-09-20T22:12:52.515295+0000","last_became_peered":"2024-09-20T22:12:52.515295+0000","last_unstale":"2024-09-20T22:21:13.691887+0000","last_undegraded":"2024-09-20T22:21:13.691887+0000","last_fullsized":"2024-09-20T22:21:13.691887+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:11.941075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_clean_scrub_stamp":"2024-09-20T22:18:11.941075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":575,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.697289+0000","last_change":"2024-09-20T22:18:44.881893+0000","last_active":"2024-09-20T22:21:13.697289+0000","last_peered":"2024-09-20T22:21:13.697289+0000","last_clean":"2024-09-20T22:21:13.697289+0000","last_became_active":"2024-09-20T22:10:30.657116+0000","last_became_peered":"2024-09-20T22:10:30.657116+0000","last_unstale":"2024-09-20T22:21:13.697289+0000","last_undegraded":"2024-09-20T22:21:13.697289+0000","last_fullsized":"2024-09-20T22:21:13.697289+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.881819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.881819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.6","version":"516'9318","reported_seq":20154,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.617706+0000","last_change":"2024-09-20T22:20:29.639073+0000","last_active":"2024-09-20T22:21:17.617706+0000","last_peered":"2024-09-20T22:21:17.617706+0000","last_clean":"2024-09-20T22:21:17.617706+0000","last_became_active":"2024-09-20T22:20:29.633931+0000","last_became_peered":"2024-09-20T22:20:29.633931+0000","last_unstale":"2024-09-20T22:21:17.617706+0000","last_undegraded":"2024-09-20T22:21:17.617706+0000","last_fullsized":"2024-09-20T22:21:17.617706+0000","mapping_epoch":495,"log_start":"493'5808","ondisk_log_start":"493'5808","created":456,"last_epoch_clean":496,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":3510,"log_dups_size":0,"ondisk_log_size":3510,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":198311936,"num_objects":3026,"num_object_clones":0,"num_object_copies":9078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9318,"num_write_kb":395008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5343,"num_bytes_recovered":350158848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,5],"acting":[0,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.c","version":"75'128","reported_seq":1055,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.692671+0000","last_change":"2024-09-20T22:19:14.493452+0000","last_active":"2024-09-20T22:21:13.692671+0000","last_peered":"2024-09-20T22:21:13.692671+0000","last_clean":"2024-09-20T22:21:13.692671+0000","last_became_active":"2024-09-20T22:17:40.152364+0000","last_became_peered":"2024-09-20T22:17:40.152364+0000","last_unstale":"2024-09-20T22:21:13.692671+0000","last_undegraded":"2024-09-20T22:21:13.692671+0000","last_fullsized":"2024-09-20T22:21:13.692671+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:14.493305+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:19:14.493305+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"418'23","reported_seq":891,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.691479+0000","last_change":"2024-09-20T22:19:45.882797+0000","last_active":"2024-09-20T22:21:13.691479+0000","last_peered":"2024-09-20T22:21:13.691479+0000","last_clean":"2024-09-20T22:21:13.691479+0000","last_became_active":"2024-09-20T22:19:45.816632+0000","last_became_peered":"2024-09-20T22:19:45.816632+0000","last_unstale":"2024-09-20T22:21:13.691479+0000","last_undegraded":"2024-09-20T22:21:13.691479+0000","last_fullsized":"2024-09-20T22:21:13.691479+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"418'23","last_scrub_stamp":"2024-09-20T22:17:11.338821+0000","last_deep_scrub":"418'23","last_deep_scrub_stamp":"2024-09-20T22:17:11.338821+0000","last_clean_scrub_stamp":"2024-09-20T22:17:11.338821+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":992,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.698582+0000","last_change":"2024-09-20T22:19:45.858377+0000","last_active":"2024-09-20T22:21:13.698582+0000","last_peered":"2024-09-20T22:21:13.698582+0000","last_clean":"2024-09-20T22:21:13.698582+0000","last_became_active":"2024-09-20T22:19:45.858043+0000","last_became_peered":"2024-09-20T22:19:45.858043+0000","last_unstale":"2024-09-20T22:21:13.698582+0000","last_undegraded":"2024-09-20T22:21:13.698582+0000","last_fullsized":"2024-09-20T22:21:13.698582+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:38.817089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:38.817089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,5],"acting":[7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1021,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.692494+0000","last_change":"2024-09-20T22:19:45.806148+0000","last_active":"2024-09-20T22:21:13.692494+0000","last_peered":"2024-09-20T22:21:13.692494+0000","last_clean":"2024-09-20T22:21:13.692494+0000","last_became_active":"2024-09-20T22:19:45.802808+0000","last_became_peered":"2024-09-20T22:19:45.802808+0000","last_unstale":"2024-09-20T22:21:13.692494+0000","last_undegraded":"2024-09-20T22:21:13.692494+0000","last_fullsized":"2024-09-20T22:21:13.692494+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:03.959875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:03.959875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":446,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.692057+0000","last_change":"2024-09-20T22:19:45.799827+0000","last_active":"2024-09-20T22:21:13.692057+0000","last_peered":"2024-09-20T22:21:13.692057+0000","last_clean":"2024-09-20T22:21:13.692057+0000","last_became_active":"2024-09-20T22:19:45.798815+0000","last_became_peered":"2024-09-20T22:19:45.798815+0000","last_unstale":"2024-09-20T22:21:13.692057+0000","last_undegraded":"2024-09-20T22:21:13.692057+0000","last_fullsized":"2024-09-20T22:21:13.692057+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:08.435903+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:19:08.435903+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"516'9408","reported_seq":17756,"reported_epoch":516,"state":"active+remapped+backfilling","last_fresh":"2024-09-20T22:21:16.633429+0000","last_change":"2024-09-20T22:21:14.536972+0000","last_active":"2024-09-20T22:21:16.633429+0000","last_peered":"2024-09-20T22:21:16.633429+0000","last_clean":"2024-09-20T22:19:27.221135+0000","last_became_active":"2024-09-20T22:19:46.922302+0000","last_became_peered":"2024-09-20T22:19:46.922302+0000","last_unstale":"2024-09-20T22:21:16.633429+0000","last_undegraded":"2024-09-20T22:21:16.633429+0000","last_fullsized":"2024-09-20T22:21:16.633429+0000","mapping_epoch":481,"log_start":"493'6337","ondisk_log_start":"493'6337","created":456,"last_epoch_clean":457,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:45.997784+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.997784+0000","objects_scrubbed":0,"log_size":3071,"log_dups_size":0,"ondisk_log_size":3071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":213647360,"num_objects":3260,"num_object_clones":0,"num_object_copies":9780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2936,"num_objects_unfound":0,"num_objects_dirty":3260,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9408,"num_write_kb":405376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4100,"num_bytes_recovered":268697600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,1],"avail_no_missing":["5","0","1"],"object_location_counts":[{"shards":"0,1,5","objects":3260}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.d","version":"75'128","reported_seq":602,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.698326+0000","last_change":"2024-09-20T22:19:48.871040+0000","last_active":"2024-09-20T22:21:13.698326+0000","last_peered":"2024-09-20T22:21:13.698326+0000","last_clean":"2024-09-20T22:21:13.698326+0000","last_became_active":"2024-09-20T22:19:48.870633+0000","last_became_peered":"2024-09-20T22:19:48.870633+0000","last_unstale":"2024-09-20T22:21:13.698326+0000","last_undegraded":"2024-09-20T22:21:13.698326+0000","last_fullsized":"2024-09-20T22:21:13.698326+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:40.996791+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:17:40.996791+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"416'34","reported_seq":538,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.688405+0000","last_change":"2024-09-20T22:17:04.960544+0000","last_active":"2024-09-20T22:21:13.688405+0000","last_peered":"2024-09-20T22:21:13.688405+0000","last_clean":"2024-09-20T22:21:13.688405+0000","last_became_active":"2024-09-20T22:11:30.493275+0000","last_became_peered":"2024-09-20T22:11:30.493275+0000","last_unstale":"2024-09-20T22:21:13.688405+0000","last_undegraded":"2024-09-20T22:21:13.688405+0000","last_fullsized":"2024-09-20T22:21:13.688405+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"416'34","last_scrub_stamp":"2024-09-20T22:17:04.960380+0000","last_deep_scrub":"416'34","last_deep_scrub_stamp":"2024-09-20T22:17:04.960380+0000","last_clean_scrub_stamp":"2024-09-20T22:17:04.960380+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":621,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.698561+0000","last_change":"2024-09-20T22:19:45.878368+0000","last_active":"2024-09-20T22:21:13.698561+0000","last_peered":"2024-09-20T22:21:13.698561+0000","last_clean":"2024-09-20T22:21:13.698561+0000","last_became_active":"2024-09-20T22:19:45.818725+0000","last_became_peered":"2024-09-20T22:19:45.818725+0000","last_unstale":"2024-09-20T22:21:13.698561+0000","last_undegraded":"2024-09-20T22:21:13.698561+0000","last_fullsized":"2024-09-20T22:21:13.698561+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:16:37.713713+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:16:37.713713+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":460,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,5],"acting":[7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"481'10","reported_seq":1213,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.697471+0000","last_change":"2024-09-20T22:19:45.804947+0000","last_active":"2024-09-20T22:21:13.697471+0000","last_peered":"2024-09-20T22:21:13.697471+0000","last_clean":"2024-09-20T22:21:13.697471+0000","last_became_active":"2024-09-20T22:19:45.793150+0000","last_became_peered":"2024-09-20T22:19:45.793150+0000","last_unstale":"2024-09-20T22:21:13.697471+0000","last_undegraded":"2024-09-20T22:21:13.697471+0000","last_fullsized":"2024-09-20T22:21:13.697471+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:19:16.895874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:16.895874+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":517,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.688872+0000","last_change":"2024-09-20T22:19:09.620194+0000","last_active":"2024-09-20T22:21:13.688872+0000","last_peered":"2024-09-20T22:21:13.688872+0000","last_clean":"2024-09-20T22:21:13.688872+0000","last_became_active":"2024-09-20T22:12:52.517758+0000","last_became_peered":"2024-09-20T22:12:52.517758+0000","last_unstale":"2024-09-20T22:21:13.688872+0000","last_undegraded":"2024-09-20T22:21:13.688872+0000","last_fullsized":"2024-09-20T22:21:13.688872+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:09.620092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:19:09.620092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"75'128","reported_seq":865,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.688747+0000","last_change":"2024-09-20T22:19:48.841486+0000","last_active":"2024-09-20T22:21:13.688747+0000","last_peered":"2024-09-20T22:21:13.688747+0000","last_clean":"2024-09-20T22:21:13.688747+0000","last_became_active":"2024-09-20T22:19:48.841087+0000","last_became_peered":"2024-09-20T22:19:48.841087+0000","last_unstale":"2024-09-20T22:21:13.688747+0000","last_undegraded":"2024-09-20T22:21:13.688747+0000","last_fullsized":"2024-09-20T22:21:13.688747+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:06.524269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:19:06.524269+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1019,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.696914+0000","last_change":"2024-09-20T22:18:00.786710+0000","last_active":"2024-09-20T22:21:17.696914+0000","last_peered":"2024-09-20T22:21:17.696914+0000","last_clean":"2024-09-20T22:21:17.696914+0000","last_became_active":"2024-09-20T22:12:52.516989+0000","last_became_peered":"2024-09-20T22:12:52.516989+0000","last_unstale":"2024-09-20T22:21:17.696914+0000","last_undegraded":"2024-09-20T22:21:17.696914+0000","last_fullsized":"2024-09-20T22:21:17.696914+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:00.786604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:00.786604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"328'12","reported_seq":1041,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.693700+0000","last_change":"2024-09-20T22:19:45.839546+0000","last_active":"2024-09-20T22:21:13.693700+0000","last_peered":"2024-09-20T22:21:13.693700+0000","last_clean":"2024-09-20T22:21:13.693700+0000","last_became_active":"2024-09-20T22:19:45.798694+0000","last_became_peered":"2024-09-20T22:19:45.798694+0000","last_unstale":"2024-09-20T22:21:13.693700+0000","last_undegraded":"2024-09-20T22:21:13.693700+0000","last_fullsized":"2024-09-20T22:21:13.693700+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'12","last_scrub_stamp":"2024-09-20T22:19:41.849111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:41.849111+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":511,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689800+0000","last_change":"2024-09-20T22:19:45.789577+0000","last_active":"2024-09-20T22:21:13.689800+0000","last_peered":"2024-09-20T22:21:13.689800+0000","last_clean":"2024-09-20T22:21:13.689800+0000","last_became_active":"2024-09-20T22:19:45.789220+0000","last_became_peered":"2024-09-20T22:19:45.789220+0000","last_unstale":"2024-09-20T22:21:13.689800+0000","last_undegraded":"2024-09-20T22:21:13.689800+0000","last_fullsized":"2024-09-20T22:21:13.689800+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:36.439030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:36.439030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":956,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.698228+0000","last_change":"2024-09-20T22:19:28.250214+0000","last_active":"2024-09-20T22:21:13.698228+0000","last_peered":"2024-09-20T22:21:13.698228+0000","last_clean":"2024-09-20T22:21:13.698228+0000","last_became_active":"2024-09-20T22:19:28.249892+0000","last_became_peered":"2024-09-20T22:19:28.249892+0000","last_unstale":"2024-09-20T22:21:13.698228+0000","last_undegraded":"2024-09-20T22:21:13.698228+0000","last_fullsized":"2024-09-20T22:21:13.698228+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:02.792356+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:54.359586+0000","last_clean_scrub_stamp":"2024-09-20T22:17:02.792356+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"75'128","reported_seq":785,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.692373+0000","last_change":"2024-09-20T22:19:48.862335+0000","last_active":"2024-09-20T22:21:13.692373+0000","last_peered":"2024-09-20T22:21:13.692373+0000","last_clean":"2024-09-20T22:21:13.692373+0000","last_became_active":"2024-09-20T22:19:48.853347+0000","last_became_peered":"2024-09-20T22:19:48.853347+0000","last_unstale":"2024-09-20T22:21:13.692373+0000","last_undegraded":"2024-09-20T22:21:13.692373+0000","last_fullsized":"2024-09-20T22:21:13.692373+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:18:07.153103+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:18:07.153103+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:16.905212+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"328'15","reported_seq":441,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.695439+0000","last_change":"2024-09-20T22:17:39.086388+0000","last_active":"2024-09-20T22:21:13.695439+0000","last_peered":"2024-09-20T22:21:13.695439+0000","last_clean":"2024-09-20T22:21:13.695439+0000","last_became_active":"2024-09-20T22:12:52.527106+0000","last_became_peered":"2024-09-20T22:12:52.527106+0000","last_unstale":"2024-09-20T22:21:13.695439+0000","last_undegraded":"2024-09-20T22:21:13.695439+0000","last_fullsized":"2024-09-20T22:21:13.695439+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:17:39.086307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:39.086307+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913149+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":976,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.697840+0000","last_change":"2024-09-20T22:19:28.250709+0000","last_active":"2024-09-20T22:21:13.697840+0000","last_peered":"2024-09-20T22:21:13.697840+0000","last_clean":"2024-09-20T22:21:13.697840+0000","last_became_active":"2024-09-20T22:19:28.249922+0000","last_became_peered":"2024-09-20T22:19:28.249922+0000","last_unstale":"2024-09-20T22:21:13.697840+0000","last_undegraded":"2024-09-20T22:21:13.697840+0000","last_fullsized":"2024-09-20T22:21:13.697840+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:10.768513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:10.768513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":252,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.699041+0000","last_change":"2024-09-20T22:19:45.818793+0000","last_active":"2024-09-20T22:21:13.699041+0000","last_peered":"2024-09-20T22:21:13.699041+0000","last_clean":"2024-09-20T22:21:13.699041+0000","last_became_active":"2024-09-20T22:19:45.818298+0000","last_became_peered":"2024-09-20T22:19:45.818298+0000","last_unstale":"2024-09-20T22:21:13.699041+0000","last_undegraded":"2024-09-20T22:21:13.699041+0000","last_fullsized":"2024-09-20T22:21:13.699041+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:36.302138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_clean_scrub_stamp":"2024-09-20T22:18:36.302138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":995,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.693687+0000","last_change":"2024-09-20T22:19:28.241258+0000","last_active":"2024-09-20T22:21:13.693687+0000","last_peered":"2024-09-20T22:21:13.693687+0000","last_clean":"2024-09-20T22:21:13.693687+0000","last_became_active":"2024-09-20T22:19:28.240554+0000","last_became_peered":"2024-09-20T22:19:28.240554+0000","last_unstale":"2024-09-20T22:21:13.693687+0000","last_undegraded":"2024-09-20T22:21:13.693687+0000","last_fullsized":"2024-09-20T22:21:13.693687+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:03.970735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:26.862279+0000","last_clean_scrub_stamp":"2024-09-20T22:18:03.970735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"75'128","reported_seq":856,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.688193+0000","last_change":"2024-09-20T22:18:07.509064+0000","last_active":"2024-09-20T22:21:13.688193+0000","last_peered":"2024-09-20T22:21:13.688193+0000","last_clean":"2024-09-20T22:21:13.688193+0000","last_became_active":"2024-09-20T22:15:50.574865+0000","last_became_peered":"2024-09-20T22:15:50.574865+0000","last_unstale":"2024-09-20T22:21:13.688193+0000","last_undegraded":"2024-09-20T22:21:13.688193+0000","last_fullsized":"2024-09-20T22:21:13.688193+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:18:07.508939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:18:07.508939+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"328'15","reported_seq":524,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689462+0000","last_change":"2024-09-20T22:18:39.362251+0000","last_active":"2024-09-20T22:21:13.689462+0000","last_peered":"2024-09-20T22:21:13.689462+0000","last_clean":"2024-09-20T22:21:13.689462+0000","last_became_active":"2024-09-20T22:11:30.493335+0000","last_became_peered":"2024-09-20T22:11:30.493335+0000","last_unstale":"2024-09-20T22:21:13.689462+0000","last_undegraded":"2024-09-20T22:21:13.689462+0000","last_fullsized":"2024-09-20T22:21:13.689462+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:18:39.362109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:39.362109+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":430,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.695642+0000","last_change":"2024-09-20T22:19:13.833938+0000","last_active":"2024-09-20T22:21:13.695642+0000","last_peered":"2024-09-20T22:21:13.695642+0000","last_clean":"2024-09-20T22:21:13.695642+0000","last_became_active":"2024-09-20T22:12:52.524149+0000","last_became_peered":"2024-09-20T22:12:52.524149+0000","last_unstale":"2024-09-20T22:21:13.695642+0000","last_undegraded":"2024-09-20T22:21:13.695642+0000","last_fullsized":"2024-09-20T22:21:13.695642+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:13.833672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:56.538825+0000","last_clean_scrub_stamp":"2024-09-20T22:19:13.833672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913543+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"315'11","reported_seq":571,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.698372+0000","last_change":"2024-09-20T22:17:05.784312+0000","last_active":"2024-09-20T22:21:13.698372+0000","last_peered":"2024-09-20T22:21:13.698372+0000","last_clean":"2024-09-20T22:21:13.698372+0000","last_became_active":"2024-09-20T22:12:33.544098+0000","last_became_peered":"2024-09-20T22:12:33.544098+0000","last_unstale":"2024-09-20T22:21:13.698372+0000","last_undegraded":"2024-09-20T22:21:13.698372+0000","last_fullsized":"2024-09-20T22:21:13.698372+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:17:05.784212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:05.784212+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":565,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.693211+0000","last_change":"2024-09-20T22:19:28.247124+0000","last_active":"2024-09-20T22:21:13.693211+0000","last_peered":"2024-09-20T22:21:13.693211+0000","last_clean":"2024-09-20T22:21:13.693211+0000","last_became_active":"2024-09-20T22:19:28.245964+0000","last_became_peered":"2024-09-20T22:19:28.245964+0000","last_unstale":"2024-09-20T22:21:13.693211+0000","last_undegraded":"2024-09-20T22:21:13.693211+0000","last_fullsized":"2024-09-20T22:21:13.693211+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:03.323607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:03.323607+0000","last_clean_scrub_stamp":"2024-09-20T22:18:03.323607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.11","version":"75'128","reported_seq":880,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.693860+0000","last_change":"2024-09-20T22:19:05.735170+0000","last_active":"2024-09-20T22:21:13.693860+0000","last_peered":"2024-09-20T22:21:13.693860+0000","last_clean":"2024-09-20T22:21:13.693860+0000","last_became_active":"2024-09-20T22:15:50.612544+0000","last_became_peered":"2024-09-20T22:15:50.612544+0000","last_unstale":"2024-09-20T22:21:13.693860+0000","last_undegraded":"2024-09-20T22:21:13.693860+0000","last_fullsized":"2024-09-20T22:21:13.693860+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:05.735058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:19:05.735058+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913624+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"466'25","reported_seq":447,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.698076+0000","last_change":"2024-09-20T22:19:45.871562+0000","last_active":"2024-09-20T22:21:13.698076+0000","last_peered":"2024-09-20T22:21:13.698076+0000","last_clean":"2024-09-20T22:21:13.698076+0000","last_became_active":"2024-09-20T22:19:45.809071+0000","last_became_peered":"2024-09-20T22:19:45.809071+0000","last_unstale":"2024-09-20T22:21:13.698076+0000","last_undegraded":"2024-09-20T22:21:13.698076+0000","last_fullsized":"2024-09-20T22:21:13.698076+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'25","last_scrub_stamp":"2024-09-20T22:19:16.310142+0000","last_deep_scrub":"466'25","last_deep_scrub_stamp":"2024-09-20T22:19:16.310142+0000","last_clean_scrub_stamp":"2024-09-20T22:19:16.310142+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":976,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.698293+0000","last_change":"2024-09-20T22:19:28.251114+0000","last_active":"2024-09-20T22:21:13.698293+0000","last_peered":"2024-09-20T22:21:13.698293+0000","last_clean":"2024-09-20T22:21:13.698293+0000","last_became_active":"2024-09-20T22:19:28.250811+0000","last_became_peered":"2024-09-20T22:19:28.250811+0000","last_unstale":"2024-09-20T22:21:13.698293+0000","last_undegraded":"2024-09-20T22:21:13.698293+0000","last_fullsized":"2024-09-20T22:21:13.698293+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:06.001800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:19:06.001800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"481'15","reported_seq":657,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689773+0000","last_change":"2024-09-20T22:19:45.873516+0000","last_active":"2024-09-20T22:21:13.689773+0000","last_peered":"2024-09-20T22:21:13.689773+0000","last_clean":"2024-09-20T22:21:13.689773+0000","last_became_active":"2024-09-20T22:19:45.809247+0000","last_became_peered":"2024-09-20T22:19:45.809247+0000","last_unstale":"2024-09-20T22:21:13.689773+0000","last_undegraded":"2024-09-20T22:21:13.689773+0000","last_fullsized":"2024-09-20T22:21:13.689773+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'14","last_scrub_stamp":"2024-09-20T22:16:55.917159+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:16:55.917159+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":422,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.694992+0000","last_change":"2024-09-20T22:17:06.828671+0000","last_active":"2024-09-20T22:21:13.694992+0000","last_peered":"2024-09-20T22:21:13.694992+0000","last_clean":"2024-09-20T22:21:13.694992+0000","last_became_active":"2024-09-20T22:12:52.523781+0000","last_became_peered":"2024-09-20T22:12:52.523781+0000","last_unstale":"2024-09-20T22:21:13.694992+0000","last_undegraded":"2024-09-20T22:21:13.694992+0000","last_fullsized":"2024-09-20T22:21:13.694992+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:06.828504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:06.828504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.912980+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.16","version":"75'128","reported_seq":675,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.694606+0000","last_change":"2024-09-20T22:21:00.998420+0000","last_active":"2024-09-20T22:21:13.694606+0000","last_peered":"2024-09-20T22:21:13.694606+0000","last_clean":"2024-09-20T22:21:13.694606+0000","last_became_active":"2024-09-20T22:19:48.855437+0000","last_became_peered":"2024-09-20T22:19:48.855437+0000","last_unstale":"2024-09-20T22:21:13.694606+0000","last_undegraded":"2024-09-20T22:21:13.694606+0000","last_fullsized":"2024-09-20T22:21:13.694606+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:00.998286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:21:00.998286+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:26.540377+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"328'19","reported_seq":739,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.695085+0000","last_change":"2024-09-20T22:19:45.788839+0000","last_active":"2024-09-20T22:21:17.695085+0000","last_peered":"2024-09-20T22:21:17.695085+0000","last_clean":"2024-09-20T22:21:17.695085+0000","last_became_active":"2024-09-20T22:19:45.788204+0000","last_became_peered":"2024-09-20T22:19:45.788204+0000","last_unstale":"2024-09-20T22:21:17.695085+0000","last_undegraded":"2024-09-20T22:21:17.695085+0000","last_fullsized":"2024-09-20T22:21:17.695085+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:19:18.385865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:18.385865+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":98,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.697979+0000","last_change":"2024-09-20T22:19:28.261113+0000","last_active":"2024-09-20T22:21:13.697979+0000","last_peered":"2024-09-20T22:21:13.697979+0000","last_clean":"2024-09-20T22:21:13.697979+0000","last_became_active":"2024-09-20T22:19:28.260795+0000","last_became_peered":"2024-09-20T22:19:28.260795+0000","last_unstale":"2024-09-20T22:21:13.697979+0000","last_undegraded":"2024-09-20T22:21:13.697979+0000","last_fullsized":"2024-09-20T22:21:13.697979+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:56.523743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:56.523743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":195,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.688963+0000","last_change":"2024-09-20T22:19:45.798598+0000","last_active":"2024-09-20T22:21:13.688963+0000","last_peered":"2024-09-20T22:21:13.688963+0000","last_clean":"2024-09-20T22:21:13.688963+0000","last_became_active":"2024-09-20T22:19:45.797640+0000","last_became_peered":"2024-09-20T22:19:45.797640+0000","last_unstale":"2024-09-20T22:21:13.688963+0000","last_undegraded":"2024-09-20T22:21:13.688963+0000","last_fullsized":"2024-09-20T22:21:13.688963+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:40.528539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:40.528539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":479,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.693630+0000","last_change":"2024-09-20T22:19:45.812535+0000","last_active":"2024-09-20T22:21:13.693630+0000","last_peered":"2024-09-20T22:21:13.693630+0000","last_clean":"2024-09-20T22:21:13.693630+0000","last_became_active":"2024-09-20T22:19:45.810800+0000","last_became_peered":"2024-09-20T22:19:45.810800+0000","last_unstale":"2024-09-20T22:21:13.693630+0000","last_undegraded":"2024-09-20T22:21:13.693630+0000","last_fullsized":"2024-09-20T22:21:13.693630+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:59.256587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:16:59.256587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.912953+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.17","version":"75'128","reported_seq":478,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.690604+0000","last_change":"2024-09-20T22:17:57.421052+0000","last_active":"2024-09-20T22:21:13.690604+0000","last_peered":"2024-09-20T22:21:13.690604+0000","last_clean":"2024-09-20T22:21:13.690604+0000","last_became_active":"2024-09-20T22:17:44.721755+0000","last_became_peered":"2024-09-20T22:17:44.721755+0000","last_unstale":"2024-09-20T22:21:13.690604+0000","last_undegraded":"2024-09-20T22:21:13.690604+0000","last_fullsized":"2024-09-20T22:21:13.690604+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:57.420935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.420935+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"328'19","reported_seq":580,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.696262+0000","last_change":"2024-09-20T22:19:45.867565+0000","last_active":"2024-09-20T22:21:17.696262+0000","last_peered":"2024-09-20T22:21:17.696262+0000","last_clean":"2024-09-20T22:21:17.696262+0000","last_became_active":"2024-09-20T22:19:45.801107+0000","last_became_peered":"2024-09-20T22:19:45.801107+0000","last_unstale":"2024-09-20T22:21:17.696262+0000","last_undegraded":"2024-09-20T22:21:17.696262+0000","last_fullsized":"2024-09-20T22:21:17.696262+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:17:03.305256+0000","last_deep_scrub":"291'13","last_deep_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_clean_scrub_stamp":"2024-09-20T22:17:03.305256+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":573,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.698112+0000","last_change":"2024-09-20T22:19:28.243263+0000","last_active":"2024-09-20T22:21:13.698112+0000","last_peered":"2024-09-20T22:21:13.698112+0000","last_clean":"2024-09-20T22:21:13.698112+0000","last_became_active":"2024-09-20T22:19:28.242951+0000","last_became_peered":"2024-09-20T22:19:28.242951+0000","last_unstale":"2024-09-20T22:21:13.698112+0000","last_undegraded":"2024-09-20T22:21:13.698112+0000","last_fullsized":"2024-09-20T22:21:13.698112+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:54.900125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:54.900125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":436,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.695286+0000","last_change":"2024-09-20T22:19:45.820095+0000","last_active":"2024-09-20T22:21:13.695286+0000","last_peered":"2024-09-20T22:21:13.695286+0000","last_clean":"2024-09-20T22:21:13.695286+0000","last_became_active":"2024-09-20T22:19:45.817319+0000","last_became_peered":"2024-09-20T22:19:45.817319+0000","last_unstale":"2024-09-20T22:21:13.695286+0000","last_undegraded":"2024-09-20T22:21:13.695286+0000","last_fullsized":"2024-09-20T22:21:13.695286+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:02.798998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:02.798998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913576+0000","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":428,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.694984+0000","last_change":"2024-09-20T22:19:45.818923+0000","last_active":"2024-09-20T22:21:13.694984+0000","last_peered":"2024-09-20T22:21:13.694984+0000","last_clean":"2024-09-20T22:21:13.694984+0000","last_became_active":"2024-09-20T22:19:45.818444+0000","last_became_peered":"2024-09-20T22:19:45.818444+0000","last_unstale":"2024-09-20T22:21:13.694984+0000","last_undegraded":"2024-09-20T22:21:13.694984+0000","last_fullsized":"2024-09-20T22:21:13.694984+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:09.839241+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:09.839241+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913119+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"75'128","reported_seq":1047,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.690812+0000","last_change":"2024-09-20T22:19:48.880703+0000","last_active":"2024-09-20T22:21:13.690812+0000","last_peered":"2024-09-20T22:21:13.690812+0000","last_clean":"2024-09-20T22:21:13.690812+0000","last_became_active":"2024-09-20T22:19:48.880005+0000","last_became_peered":"2024-09-20T22:19:48.880005+0000","last_unstale":"2024-09-20T22:21:13.690812+0000","last_undegraded":"2024-09-20T22:21:13.690812+0000","last_fullsized":"2024-09-20T22:21:13.690812+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:09.890234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:19:09.890234+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"328'19","reported_seq":895,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.697671+0000","last_change":"2024-09-20T22:19:17.961328+0000","last_active":"2024-09-20T22:21:13.697671+0000","last_peered":"2024-09-20T22:21:13.697671+0000","last_clean":"2024-09-20T22:21:13.697671+0000","last_became_active":"2024-09-20T22:12:33.545199+0000","last_became_peered":"2024-09-20T22:12:33.545199+0000","last_unstale":"2024-09-20T22:21:13.697671+0000","last_undegraded":"2024-09-20T22:21:13.697671+0000","last_fullsized":"2024-09-20T22:21:13.697671+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:19:17.961211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:17.961211+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":514,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689780+0000","last_change":"2024-09-20T22:19:45.795979+0000","last_active":"2024-09-20T22:21:13.689780+0000","last_peered":"2024-09-20T22:21:13.689780+0000","last_clean":"2024-09-20T22:21:13.689780+0000","last_became_active":"2024-09-20T22:19:45.795652+0000","last_became_peered":"2024-09-20T22:19:45.795652+0000","last_unstale":"2024-09-20T22:21:13.689780+0000","last_undegraded":"2024-09-20T22:21:13.689780+0000","last_fullsized":"2024-09-20T22:21:13.689780+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:55.398175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_clean_scrub_stamp":"2024-09-20T22:17:55.398175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":968,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.697415+0000","last_change":"2024-09-20T22:19:28.260778+0000","last_active":"2024-09-20T22:21:13.697415+0000","last_peered":"2024-09-20T22:21:13.697415+0000","last_clean":"2024-09-20T22:21:13.697415+0000","last_became_active":"2024-09-20T22:19:28.260465+0000","last_became_peered":"2024-09-20T22:19:28.260465+0000","last_unstale":"2024-09-20T22:21:13.697415+0000","last_undegraded":"2024-09-20T22:21:13.697415+0000","last_fullsized":"2024-09-20T22:21:13.697415+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:08.798598+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:08.798598+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":419,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.694653+0000","last_change":"2024-09-20T22:18:32.424564+0000","last_active":"2024-09-20T22:21:13.694653+0000","last_peered":"2024-09-20T22:21:13.694653+0000","last_clean":"2024-09-20T22:21:13.694653+0000","last_became_active":"2024-09-20T22:12:57.883998+0000","last_became_peered":"2024-09-20T22:12:57.883998+0000","last_unstale":"2024-09-20T22:21:13.694653+0000","last_undegraded":"2024-09-20T22:21:13.694653+0000","last_fullsized":"2024-09-20T22:21:13.694653+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:32.424321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:32.424321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913408+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.15","version":"75'128","reported_seq":486,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.697980+0000","last_change":"2024-09-20T22:19:48.849798+0000","last_active":"2024-09-20T22:21:13.697980+0000","last_peered":"2024-09-20T22:21:13.697980+0000","last_clean":"2024-09-20T22:21:13.697980+0000","last_became_active":"2024-09-20T22:19:48.849447+0000","last_became_peered":"2024-09-20T22:19:48.849447+0000","last_unstale":"2024-09-20T22:21:13.697980+0000","last_undegraded":"2024-09-20T22:21:13.697980+0000","last_fullsized":"2024-09-20T22:21:13.697980+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.859373+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.859373+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"453'10","reported_seq":1008,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689607+0000","last_change":"2024-09-20T22:19:45.814298+0000","last_active":"2024-09-20T22:21:13.689607+0000","last_peered":"2024-09-20T22:21:13.689607+0000","last_clean":"2024-09-20T22:21:13.689607+0000","last_became_active":"2024-09-20T22:19:45.795544+0000","last_became_peered":"2024-09-20T22:19:45.795544+0000","last_unstale":"2024-09-20T22:21:13.689607+0000","last_undegraded":"2024-09-20T22:21:13.689607+0000","last_fullsized":"2024-09-20T22:21:13.689607+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"453'10","last_scrub_stamp":"2024-09-20T22:18:39.588954+0000","last_deep_scrub":"453'10","last_deep_scrub_stamp":"2024-09-20T22:18:39.588954+0000","last_clean_scrub_stamp":"2024-09-20T22:18:39.588954+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":422,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.696251+0000","last_change":"2024-09-20T22:18:09.367123+0000","last_active":"2024-09-20T22:21:13.696251+0000","last_peered":"2024-09-20T22:21:13.696251+0000","last_clean":"2024-09-20T22:21:13.696251+0000","last_became_active":"2024-09-20T22:12:52.517510+0000","last_became_peered":"2024-09-20T22:12:52.517510+0000","last_unstale":"2024-09-20T22:21:13.696251+0000","last_undegraded":"2024-09-20T22:21:13.696251+0000","last_fullsized":"2024-09-20T22:21:13.696251+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:09.366999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:09.366999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913236+0000","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1001,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.698696+0000","last_change":"2024-09-20T22:19:28.261147+0000","last_active":"2024-09-20T22:21:17.698696+0000","last_peered":"2024-09-20T22:21:17.698696+0000","last_clean":"2024-09-20T22:21:17.698696+0000","last_became_active":"2024-09-20T22:19:28.260576+0000","last_became_peered":"2024-09-20T22:19:28.260576+0000","last_unstale":"2024-09-20T22:21:17.698696+0000","last_undegraded":"2024-09-20T22:21:17.698696+0000","last_fullsized":"2024-09-20T22:21:17.698696+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.105874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.105874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":964,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.697883+0000","last_change":"2024-09-20T22:19:28.245693+0000","last_active":"2024-09-20T22:21:13.697883+0000","last_peered":"2024-09-20T22:21:13.697883+0000","last_clean":"2024-09-20T22:21:13.697883+0000","last_became_active":"2024-09-20T22:19:28.245306+0000","last_became_peered":"2024-09-20T22:19:28.245306+0000","last_unstale":"2024-09-20T22:21:13.697883+0000","last_undegraded":"2024-09-20T22:21:13.697883+0000","last_fullsized":"2024-09-20T22:21:13.697883+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:33.935154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_clean_scrub_stamp":"2024-09-20T22:17:33.935154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1a","version":"75'128","reported_seq":401,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.697735+0000","last_change":"2024-09-20T22:19:48.843826+0000","last_active":"2024-09-20T22:21:13.697735+0000","last_peered":"2024-09-20T22:21:13.697735+0000","last_clean":"2024-09-20T22:21:13.697735+0000","last_became_active":"2024-09-20T22:19:48.843466+0000","last_became_peered":"2024-09-20T22:19:48.843466+0000","last_unstale":"2024-09-20T22:21:13.697735+0000","last_undegraded":"2024-09-20T22:21:13.697735+0000","last_fullsized":"2024-09-20T22:21:13.697735+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.396173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.396173+0000","objects_scrubbed":2,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:14.596150+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26835,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":9,"num_read_kb":13,"num_write":8,"num_write_kb":110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":80505,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":848,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.689797+0000","last_change":"2024-09-20T22:19:45.840129+0000","last_active":"2024-09-20T22:21:13.689797+0000","last_peered":"2024-09-20T22:21:13.689797+0000","last_clean":"2024-09-20T22:21:13.689797+0000","last_became_active":"2024-09-20T22:19:45.839497+0000","last_became_peered":"2024-09-20T22:19:45.839497+0000","last_unstale":"2024-09-20T22:21:13.689797+0000","last_undegraded":"2024-09-20T22:21:13.689797+0000","last_fullsized":"2024-09-20T22:21:13.689797+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:37.293600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:37.293600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":428,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.695371+0000","last_change":"2024-09-20T22:19:45.818587+0000","last_active":"2024-09-20T22:21:13.695371+0000","last_peered":"2024-09-20T22:21:13.695371+0000","last_clean":"2024-09-20T22:21:13.695371+0000","last_became_active":"2024-09-20T22:19:45.817473+0000","last_became_peered":"2024-09-20T22:19:45.817473+0000","last_unstale":"2024-09-20T22:21:13.695371+0000","last_undegraded":"2024-09-20T22:21:13.695371+0000","last_fullsized":"2024-09-20T22:21:13.695371+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:16:56.775388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:16:56.775388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.912925+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"328'19","reported_seq":1034,"reported_epoch":517,"state":"active+clean","last_fresh":"2024-09-20T22:21:17.696247+0000","last_change":"2024-09-20T22:19:45.888614+0000","last_active":"2024-09-20T22:21:17.696247+0000","last_peered":"2024-09-20T22:21:17.696247+0000","last_clean":"2024-09-20T22:21:17.696247+0000","last_became_active":"2024-09-20T22:19:45.815971+0000","last_became_peered":"2024-09-20T22:19:45.815971+0000","last_unstale":"2024-09-20T22:21:17.696247+0000","last_undegraded":"2024-09-20T22:21:17.696247+0000","last_fullsized":"2024-09-20T22:21:17.696247+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:17:29.804188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:29.804188+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":423,"reported_epoch":516,"state":"active+clean","last_fresh":"2024-09-20T22:21:13.694942+0000","last_change":"2024-09-20T22:17:10.840330+0000","last_active":"2024-09-20T22:21:13.694942+0000","last_peered":"2024-09-20T22:21:13.694942+0000","last_clean":"2024-09-20T22:21:13.694942+0000","last_became_active":"2024-09-20T22:12:52.524425+0000","last_became_peered":"2024-09-20T22:12:52.524425+0000","last_unstale":"2024-09-20T22:21:13.694942+0000","last_undegraded":"2024-09-20T22:21:13.694942+0000","last_fullsized":"2024-09-20T22:21:13.694942+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:10.839899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:10.839899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:21:17.913093+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":16,"stat_sum":{"num_bytes":3337093144,"num_objects":50921,"num_object_clones":0,"num_object_copies":152763,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":737,"num_objects_misplaced":4647,"num_objects_unfound":0,"num_objects_dirty":50921,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":153049,"num_write_kb":6526977,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47354,"num_bytes_recovered":3103391744,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":11629047808,"data_stored":11629035592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55219,"ondisk_log_size":55219,"up":48,"acting":47,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1105,"num_read_kb":862,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":6342,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":14368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":756,"ondisk_log_size":756,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2991,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":7072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1077,"num_read_kb":1072,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1186,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":31,"stat_sum":{"num_bytes":648510,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":19,"num_read_kb":28,"num_write":17,"num_write_kb":238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":174427,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3968,"ondisk_log_size":3968,"up":93,"acting":93,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813136097,"num_pgs":97,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2314496,"kb_used_data":1489200,"kb_used_omap":1,"kb_used_meta":825278,"kb_avail":91430656,"statfs":{"total":95995035648,"available":93624991744,"internally_reserved":0,"allocated":1524940800,"data_stored":1524557237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":845085130},"hb_peers":[0,1,2,3,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430435,"num_pgs":88,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2523208,"kb_used_data":1788148,"kb_used_omap":1,"kb_used_meta":735038,"kb_avail":91221944,"statfs":{"total":95995035648,"available":93411270656,"internally_reserved":0,"allocated":1831063552,"data_stored":1830683427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":752679366},"hb_peers":[0,1,2,3,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724777,"num_pgs":98,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2736124,"kb_used_data":1788652,"kb_used_omap":1,"kb_used_meta":947454,"kb_avail":91009028,"statfs":{"total":95995035648,"available":93193244672,"internally_reserved":0,"allocated":1831579648,"data_stored":1831200289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":970193349},"hb_peers":[0,1,2,3,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":0,"seq":133143986400,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":622770258093,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149942,"num_pgs":89,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2535492,"kb_used_data":1615604,"kb_used_omap":1,"kb_used_meta":919678,"kb_avail":91209660,"statfs":{"total":95995035648,"available":93398691840,"internally_reserved":0,"allocated":1654378496,"data_stored":1653991169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":941750727},"hb_peers":[0,1,2,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411579,"num_pgs":75,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1964952,"kb_used_data":1221512,"kb_used_omap":1,"kb_used_meta":743358,"kb_avail":91780200,"statfs":{"total":95995035648,"available":93982924800,"internally_reserved":0,"allocated":1250828288,"data_stored":1250484121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":761199045},"hb_peers":[0,1,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738630,"num_pgs":87,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":2606976,"kb_used_data":1637296,"kb_used_omap":1,"kb_used_meta":969662,"kb_avail":91138176,"statfs":{"total":95995035648,"available":93325492224,"internally_reserved":0,"allocated":1676591104,"data_stored":1676206884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":992934341},"hb_peers":[1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1645215744,"data_stored":1645215744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":980418560,"data_stored":980418560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1235161088,"data_stored":1235157016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1624375296,"data_stored":1624375296,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1002569728,"data_stored":1002569728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1804341248,"data_stored":1804337176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1828261888,"data_stored":1828257816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1508704256,"data_stored":1508704256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:21:21.068 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool set .mgr pgp_num 31 2024-09-20T22:21:21.079 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:21.134 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:21.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:21.135 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:21.148 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:21.236 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:21.259 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:21.260 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:21.260 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:21.361 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:21.387 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:21.388 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:21.388 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:21.399 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T22:21:21.489 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:21.513 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:21.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:21.514 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:21.583 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:21:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:21:21.340+0000 7fa5382c7640 -1 mgr get_metadata_python Requested missing service osd.4 2024-09-20T22:21:21.615 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:21.638 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:21.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:21.639 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:21.707 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:21 smithi067 ceph-mon[33511]: osdmap e519: 8 total, 7 up, 6 in 2024-09-20T22:21:21.707 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:21:21.707 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:21:21.737 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:21.740 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:21.746 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:21:21.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:21.766 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:21.766 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:21.861 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:21 smithi060 ceph-mon[29694]: osdmap e519: 8 total, 7 up, 6 in 2024-09-20T22:21:21.861 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:21:21.861 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:21:21.861 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:21 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 737/153360 objects degraded (0.481%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T22:21:21.862 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:21 smithi060 ceph-mon[40290]: osdmap e519: 8 total, 7 up, 6 in 2024-09-20T22:21:21.862 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:21:21.862 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:21:21.862 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:21 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 737/153360 objects degraded (0.481%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T22:21:21.867 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:21.892 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:21.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:21.893 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:21.993 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:21 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 737/153360 objects degraded (0.481%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T22:21:22.019 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:22.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:22.020 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:22.121 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:22.159 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:22.160 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:22.160 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:22.261 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:22.304 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:22.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:22.305 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:22.364 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:21:22.406 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:22.439 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:21:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: "size": 20, 2024-09-20T22:21:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 600, 2024-09-20T22:21:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [ 2024-09-20T22:21:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.439 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:134474 11.e 11:7b0d185f:::benchmark_data_smithi060_196073_object32487:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e508)", 2024-09-20T22:21:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:20:58.583207+0000", 2024-09-20T22:21:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 23.851990272999998, 2024-09-20T22:21:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0326385789999999, 2024-09-20T22:21:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 134474 2024-09-20T22:21:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.440 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.583207+0000", 2024-09-20T22:21:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.583207+0000", 2024-09-20T22:21:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.583209+0000", 2024-09-20T22:21:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3430000000000001e-06 2024-09-20T22:21:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.441 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.583213+0000", 2024-09-20T22:21:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.4599999999999996e-06 2024-09-20T22:21:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.583222+0000", 2024-09-20T22:21:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3549999999999994e-06 2024-09-20T22:21:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.583229+0000", 2024-09-20T22:21:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8140000000000003e-06 2024-09-20T22:21:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.442 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.583317+0000", 2024-09-20T22:21:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.7834e-05 2024-09-20T22:21:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.583327+0000", 2024-09-20T22:21:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5019999999999998e-06 2024-09-20T22:21:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.601475+0000", 2024-09-20T22:21:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.01814808 2024-09-20T22:21:22.443 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.601595+0000", 2024-09-20T22:21:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000120078 2024-09-20T22:21:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1,3", 2024-09-20T22:21:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.601681+0000", 2024-09-20T22:21:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6008999999999996e-05 2024-09-20T22:21:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.444 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.605929+0000", 2024-09-20T22:21:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0042481819999999997 2024-09-20T22:21:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.605972+0000", 2024-09-20T22:21:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2775999999999998e-05 2024-09-20T22:21:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.606130+0000", 2024-09-20T22:21:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015871799999999999 2024-09-20T22:21:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.445 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.615740+0000", 2024-09-20T22:21:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0096100560000000005 2024-09-20T22:21:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.615776+0000", 2024-09-20T22:21:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.5948999999999998e-05 2024-09-20T22:21:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.615846+0000", 2024-09-20T22:21:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9423000000000003e-05 2024-09-20T22:21:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.446 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:134504 11.e 11:776a4633:::benchmark_data_smithi060_196073_object32517:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e509)", 2024-09-20T22:21:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:20:58.608024+0000", 2024-09-20T22:21:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 23.827173372000001, 2024-09-20T22:21:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0114155840000001, 2024-09-20T22:21:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.447 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 134504 2024-09-20T22:21:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.608024+0000", 2024-09-20T22:21:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.608024+0000", 2024-09-20T22:21:22.448 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.608025+0000", 2024-09-20T22:21:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3120000000000001e-06 2024-09-20T22:21:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.608030+0000", 2024-09-20T22:21:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0139999999999998e-06 2024-09-20T22:21:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.449 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.608042+0000", 2024-09-20T22:21:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1649e-05 2024-09-20T22:21:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.608049+0000", 2024-09-20T22:21:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9319999999999999e-06 2024-09-20T22:21:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.608101+0000", 2024-09-20T22:21:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2340999999999999e-05 2024-09-20T22:21:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.450 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.608110+0000", 2024-09-20T22:21:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9819999999999997e-06 2024-09-20T22:21:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.603395+0000", 2024-09-20T22:21:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.99528440500000004 2024-09-20T22:21:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.603470+0000", 2024-09-20T22:21:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5741000000000005e-05 2024-09-20T22:21:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.451 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1,3", 2024-09-20T22:21:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.603576+0000", 2024-09-20T22:21:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010543300000000001 2024-09-20T22:21:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.606092+0000", 2024-09-20T22:21:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.002515963 2024-09-20T22:21:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.609580+0000", 2024-09-20T22:21:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0034880279999999998 2024-09-20T22:21:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.452 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.609619+0000", 2024-09-20T22:21:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.8624000000000002e-05 2024-09-20T22:21:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.453 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.619308+0000", 2024-09-20T22:21:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0096890499999999994 2024-09-20T22:21:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.619355+0000", 2024-09-20T22:21:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.7608999999999998e-05 2024-09-20T22:21:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.619440+0000", 2024-09-20T22:21:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4500999999999996e-05 2024-09-20T22:21:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.454 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:134510 11.e 11:77b94e91:::benchmark_data_smithi060_196073_object32523:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e509)", 2024-09-20T22:21:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:20:58.613685+0000", 2024-09-20T22:21:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 23.821512638000002, 2024-09-20T22:21:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0059978570000001, 2024-09-20T22:21:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.455 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 134510 2024-09-20T22:21:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.613685+0000", 2024-09-20T22:21:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.613685+0000", 2024-09-20T22:21:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.456 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.613686+0000", 2024-09-20T22:21:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8299999999999996e-07 2024-09-20T22:21:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.613690+0000", 2024-09-20T22:21:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9750000000000001e-06 2024-09-20T22:21:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.613698+0000", 2024-09-20T22:21:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.6349999999999995e-06 2024-09-20T22:21:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.457 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.613704+0000", 2024-09-20T22:21:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.9059999999999999e-06 2024-09-20T22:21:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.613762+0000", 2024-09-20T22:21:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8264000000000003e-05 2024-09-20T22:21:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.613771+0000", 2024-09-20T22:21:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4700000000000002e-06 2024-09-20T22:21:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.458 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.604657+0000", 2024-09-20T22:21:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.99088584400000002 2024-09-20T22:21:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.604777+0000", 2024-09-20T22:21:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012040999999999999 2024-09-20T22:21:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1,3", 2024-09-20T22:21:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.604917+0000", 2024-09-20T22:21:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000139772 2024-09-20T22:21:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.459 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.607159+0000", 2024-09-20T22:21:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0022425660000000001 2024-09-20T22:21:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.609653+0000", 2024-09-20T22:21:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0024936429999999998 2024-09-20T22:21:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.613379+0000", 2024-09-20T22:21:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0037263029999999998 2024-09-20T22:21:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.619504+0000", 2024-09-20T22:21:22.460 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061242670000000001 2024-09-20T22:21:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.619570+0000", 2024-09-20T22:21:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6761000000000007e-05 2024-09-20T22:21:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.619683+0000", 2024-09-20T22:21:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000112358 2024-09-20T22:21:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.461 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:134524 11.e 11:70bbd77a:::benchmark_data_smithi060_196073_object32537:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e509)", 2024-09-20T22:21:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:20:58.626860+0000", 2024-09-20T22:21:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 23.808337404, 2024-09-20T22:21:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.99303049799999998, 2024-09-20T22:21:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 134524 2024-09-20T22:21:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.462 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.626860+0000", 2024-09-20T22:21:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.626860+0000", 2024-09-20T22:21:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.626861+0000", 2024-09-20T22:21:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2200000000000004e-07 2024-09-20T22:21:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.463 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.626864+0000", 2024-09-20T22:21:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.2880000000000001e-06 2024-09-20T22:21:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.626873+0000", 2024-09-20T22:21:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3889999999999994e-06 2024-09-20T22:21:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.626879+0000", 2024-09-20T22:21:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.66e-06 2024-09-20T22:21:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.464 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.626941+0000", 2024-09-20T22:21:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.1859000000000005e-05 2024-09-20T22:21:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:58.626959+0000", 2024-09-20T22:21:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.8501000000000001e-05 2024-09-20T22:21:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.606160+0000", 2024-09-20T22:21:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.97920044299999998 2024-09-20T22:21:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.465 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.606228+0000", 2024-09-20T22:21:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.8009999999999994e-05 2024-09-20T22:21:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1,3", 2024-09-20T22:21:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.606301+0000", 2024-09-20T22:21:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3468999999999997e-05 2024-09-20T22:21:22.466 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:22.467 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:22.467 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.609627+0000", 2024-09-20T22:21:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003325401 2024-09-20T22:21:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.613433+0000", 2024-09-20T22:21:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.003806537 2024-09-20T22:21:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.468 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.613474+0000", 2024-09-20T22:21:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.0488000000000003e-05 2024-09-20T22:21:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.619746+0000", 2024-09-20T22:21:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0062720079999999999 2024-09-20T22:21:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.619776+0000", 2024-09-20T22:21:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.0602000000000003e-05 2024-09-20T22:21:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.469 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:20:59.619891+0000", 2024-09-20T22:21:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000114221 2024-09-20T22:21:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:156790 11.e 11:75038bf2:::benchmark_data_smithi060_196073_object54803:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e518)", 2024-09-20T22:21:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:20.688820+0000", 2024-09-20T22:21:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.7463778059999999, 2024-09-20T22:21:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3613733400000001, 2024-09-20T22:21:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.470 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "reached pg", 2024-09-20T22:21:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 156790 2024-09-20T22:21:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.688820+0000", 2024-09-20T22:21:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.471 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.688820+0000", 2024-09-20T22:21:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.688821+0000", 2024-09-20T22:21:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1200000000000001e-06 2024-09-20T22:21:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.688824+0000", 2024-09-20T22:21:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6490000000000001e-06 2024-09-20T22:21:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.472 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.688833+0000", 2024-09-20T22:21:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.585e-06 2024-09-20T22:21:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.688839+0000", 2024-09-20T22:21:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.2700000000000001e-06 2024-09-20T22:21:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.688967+0000", 2024-09-20T22:21:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012784299999999999 2024-09-20T22:21:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.473 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.689088+0000", 2024-09-20T22:21:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00012036699999999999 2024-09-20T22:21:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,1,3", 2024-09-20T22:21:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.725432+0000", 2024-09-20T22:21:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.036344017999999999 2024-09-20T22:21:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.729816+0000", 2024-09-20T22:21:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0043840229999999999 2024-09-20T22:21:22.474 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.734757+0000", 2024-09-20T22:21:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0049417979999999999 2024-09-20T22:21:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.734769+0000", 2024-09-20T22:21:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1654999999999999e-05 2024-09-20T22:21:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.475 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.045061+0000", 2024-09-20T22:21:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3102924840000001 2024-09-20T22:21:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for missing object", 2024-09-20T22:21:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.045155+0000", 2024-09-20T22:21:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3752999999999995e-05 2024-09-20T22:21:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.050135+0000", 2024-09-20T22:21:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.004979738 2024-09-20T22:21:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.476 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.050193+0000", 2024-09-20T22:21:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8037e-05 2024-09-20T22:21:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.477 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:156795 11.e 11:78884445:::benchmark_data_smithi060_196073_object54808:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e518)", 2024-09-20T22:21:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:20.736012+0000", 2024-09-20T22:21:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.699185277, 2024-09-20T22:21:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3180955189999999, 2024-09-20T22:21:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.478 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 156795 2024-09-20T22:21:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.736012+0000", 2024-09-20T22:21:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.736012+0000", 2024-09-20T22:21:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.479 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.736014+0000", 2024-09-20T22:21:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3489999999999999e-06 2024-09-20T22:21:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.736017+0000", 2024-09-20T22:21:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.7160000000000001e-06 2024-09-20T22:21:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.736025+0000", 2024-09-20T22:21:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1990000000000006e-06 2024-09-20T22:21:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.480 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.736032+0000", 2024-09-20T22:21:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3810000000000004e-06 2024-09-20T22:21:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.736089+0000", 2024-09-20T22:21:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7343999999999998e-05 2024-09-20T22:21:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.736098+0000", 2024-09-20T22:21:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8550000000000008e-06 2024-09-20T22:21:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.481 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.045426+0000", 2024-09-20T22:21:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.30932775 2024-09-20T22:21:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.045509+0000", 2024-09-20T22:21:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.3473000000000005e-05 2024-09-20T22:21:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:21:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.045646+0000", 2024-09-20T22:21:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000136279 2024-09-20T22:21:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.482 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.049937+0000", 2024-09-20T22:21:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0042911590000000001 2024-09-20T22:21:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.053874+0000", 2024-09-20T22:21:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0039372139999999996 2024-09-20T22:21:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.053956+0000", 2024-09-20T22:21:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2358e-05 2024-09-20T22:21:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.483 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.054009+0000", 2024-09-20T22:21:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.2794000000000003e-05 2024-09-20T22:21:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.054108+0000", 2024-09-20T22:21:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8647999999999995e-05 2024-09-20T22:21:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.484 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:156831 11.e 11:7d98419c:::benchmark_data_smithi060_196073_object54844:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e519)", 2024-09-20T22:21:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:20.759335+0000", 2024-09-20T22:21:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.6758624559999999, 2024-09-20T22:21:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.296086539, 2024-09-20T22:21:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 156831 2024-09-20T22:21:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.485 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.759335+0000", 2024-09-20T22:21:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.759335+0000", 2024-09-20T22:21:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.759337+0000", 2024-09-20T22:21:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4839999999999999e-06 2024-09-20T22:21:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.486 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.759342+0000", 2024-09-20T22:21:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.305e-06 2024-09-20T22:21:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.759349+0000", 2024-09-20T22:21:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6170000000000001e-06 2024-09-20T22:21:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.759356+0000", 2024-09-20T22:21:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3879999999999997e-06 2024-09-20T22:21:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.487 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.759456+0000", 2024-09-20T22:21:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100184 2024-09-20T22:21:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.759467+0000", 2024-09-20T22:21:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0695e-05 2024-09-20T22:21:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.047229+0000", 2024-09-20T22:21:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2877622399999999 2024-09-20T22:21:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.488 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.047361+0000", 2024-09-20T22:21:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013204300000000001 2024-09-20T22:21:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:21:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.047514+0000", 2024-09-20T22:21:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015347700000000001 2024-09-20T22:21:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.049945+0000", 2024-09-20T22:21:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0024306890000000002 2024-09-20T22:21:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.489 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.053917+0000", 2024-09-20T22:21:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0039720550000000004 2024-09-20T22:21:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.055244+0000", 2024-09-20T22:21:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0013272010000000001 2024-09-20T22:21:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.055310+0000", 2024-09-20T22:21:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5247999999999996e-05 2024-09-20T22:21:22.490 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.055422+0000", 2024-09-20T22:21:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000111913 2024-09-20T22:21:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:156853 11.e 11:725a703f:::benchmark_data_smithi060_196073_object54866:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e519)", 2024-09-20T22:21:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:20.771240+0000", 2024-09-20T22:21:22.491 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.6639572249999999, 2024-09-20T22:21:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2845487279999999, 2024-09-20T22:21:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 156853 2024-09-20T22:21:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.771240+0000", 2024-09-20T22:21:22.492 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.771240+0000", 2024-09-20T22:21:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.771241+0000", 2024-09-20T22:21:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7400000000000002e-07 2024-09-20T22:21:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.771245+0000", 2024-09-20T22:21:22.493 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1350000000000002e-06 2024-09-20T22:21:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.771255+0000", 2024-09-20T22:21:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5540000000000003e-06 2024-09-20T22:21:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.771261+0000", 2024-09-20T22:21:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6499999999999999e-06 2024-09-20T22:21:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.494 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.771328+0000", 2024-09-20T22:21:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6709000000000001e-05 2024-09-20T22:21:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.771338+0000", 2024-09-20T22:21:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7059999999999999e-06 2024-09-20T22:21:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.048299+0000", 2024-09-20T22:21:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.276961112 2024-09-20T22:21:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.495 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.048402+0000", 2024-09-20T22:21:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000103552 2024-09-20T22:21:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:21:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.048498+0000", 2024-09-20T22:21:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5818999999999999e-05 2024-09-20T22:21:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.050191+0000", 2024-09-20T22:21:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00169236 2024-09-20T22:21:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.496 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.055465+0000", 2024-09-20T22:21:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0052740820000000002 2024-09-20T22:21:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.055636+0000", 2024-09-20T22:21:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000171244 2024-09-20T22:21:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.055690+0000", 2024-09-20T22:21:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4160000000000003e-05 2024-09-20T22:21:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.055789+0000", 2024-09-20T22:21:22.497 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8870999999999996e-05 2024-09-20T22:21:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:156855 11.e 11:72a56a46:::benchmark_data_smithi060_196073_object54868:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e519)", 2024-09-20T22:21:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:20.774269+0000", 2024-09-20T22:21:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.660928945, 2024-09-20T22:21:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.297297138, 2024-09-20T22:21:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.498 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 156855 2024-09-20T22:21:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.774269+0000", 2024-09-20T22:21:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.774269+0000", 2024-09-20T22:21:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.499 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.774270+0000", 2024-09-20T22:21:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2839999999999999e-06 2024-09-20T22:21:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.774275+0000", 2024-09-20T22:21:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1250000000000001e-06 2024-09-20T22:21:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.774285+0000", 2024-09-20T22:21:22.500 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9450000000000005e-06 2024-09-20T22:21:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.774292+0000", 2024-09-20T22:21:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0600000000000002e-06 2024-09-20T22:21:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.774351+0000", 2024-09-20T22:21:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8559e-05 2024-09-20T22:21:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.501 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.774360+0000", 2024-09-20T22:21:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.7909999999999999e-06 2024-09-20T22:21:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.050216+0000", 2024-09-20T22:21:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2758559840000001 2024-09-20T22:21:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.050283+0000", 2024-09-20T22:21:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.6779000000000006e-05 2024-09-20T22:21:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:21:22.502 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.050353+0000", 2024-09-20T22:21:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0149999999999998e-05 2024-09-20T22:21:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.055589+0000", 2024-09-20T22:21:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0052360729999999999 2024-09-20T22:21:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.055856+0000", 2024-09-20T22:21:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00026658600000000003 2024-09-20T22:21:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.503 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.071413+0000", 2024-09-20T22:21:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.015557447 2024-09-20T22:21:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.071465+0000", 2024-09-20T22:21:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1186000000000001e-05 2024-09-20T22:21:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.071566+0000", 2024-09-20T22:21:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010116900000000001 2024-09-20T22:21:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.504 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:156860 11.e 11:7e6922af:::benchmark_data_smithi060_196073_object54873:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e519)", 2024-09-20T22:21:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:20.777400+0000", 2024-09-20T22:21:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.6577972050000001, 2024-09-20T22:21:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.294315066, 2024-09-20T22:21:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 156860 2024-09-20T22:21:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.505 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.777400+0000", 2024-09-20T22:21:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.777400+0000", 2024-09-20T22:21:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.777402+0000", 2024-09-20T22:21:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4249999999999999e-06 2024-09-20T22:21:22.506 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.777407+0000", 2024-09-20T22:21:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4349999999999996e-06 2024-09-20T22:21:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.777425+0000", 2024-09-20T22:21:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.802e-05 2024-09-20T22:21:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.777433+0000", 2024-09-20T22:21:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.926e-06 2024-09-20T22:21:22.507 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.777507+0000", 2024-09-20T22:21:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3465000000000002e-05 2024-09-20T22:21:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.777521+0000", 2024-09-20T22:21:22.508 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3988999999999999e-05 2024-09-20T22:21:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.051905+0000", 2024-09-20T22:21:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.274384298 2024-09-20T22:21:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.509 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.052047+0000", 2024-09-20T22:21:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00014231199999999999 2024-09-20T22:21:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:21:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.052182+0000", 2024-09-20T22:21:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000134456 2024-09-20T22:21:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.055902+0000", 2024-09-20T22:21:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0037207759999999999 2024-09-20T22:21:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.510 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.060502+0000", 2024-09-20T22:21:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0045999930000000001 2024-09-20T22:21:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.071580+0000", 2024-09-20T22:21:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.011077391000000001 2024-09-20T22:21:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.071622+0000", 2024-09-20T22:21:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.1820999999999998e-05 2024-09-20T22:21:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.511 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.071715+0000", 2024-09-20T22:21:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3758999999999995e-05 2024-09-20T22:21:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:156882 11.e 11:74e0dd1d:::benchmark_data_smithi060_196073_object54895:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e519)", 2024-09-20T22:21:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:20.789314+0000", 2024-09-20T22:21:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.6458837609999999, 2024-09-20T22:21:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.28940867, 2024-09-20T22:21:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.512 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 156882 2024-09-20T22:21:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.789314+0000", 2024-09-20T22:21:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.513 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.789314+0000", 2024-09-20T22:21:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.789315+0000", 2024-09-20T22:21:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.221e-06 2024-09-20T22:21:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.789331+0000", 2024-09-20T22:21:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6130999999999999e-05 2024-09-20T22:21:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.514 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.789346+0000", 2024-09-20T22:21:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.4579999999999999e-05 2024-09-20T22:21:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.789353+0000", 2024-09-20T22:21:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7160000000000003e-06 2024-09-20T22:21:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.789443+0000", 2024-09-20T22:21:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.9790000000000006e-05 2024-09-20T22:21:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.515 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.789454+0000", 2024-09-20T22:21:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.129e-05 2024-09-20T22:21:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.055934+0000", 2024-09-20T22:21:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2664792730000001 2024-09-20T22:21:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.056046+0000", 2024-09-20T22:21:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000111915 2024-09-20T22:21:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:21:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.056221+0000", 2024-09-20T22:21:22.516 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000175219 2024-09-20T22:21:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.060670+0000", 2024-09-20T22:21:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0044495569999999998 2024-09-20T22:21:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.066819+0000", 2024-09-20T22:21:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061488690000000004 2024-09-20T22:21:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.078570+0000", 2024-09-20T22:21:22.517 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.011751216 2024-09-20T22:21:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.078616+0000", 2024-09-20T22:21:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.5368000000000003e-05 2024-09-20T22:21:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.078722+0000", 2024-09-20T22:21:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000106525 2024-09-20T22:21:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.518 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:156904 11.e 11:7c62c6fd:::benchmark_data_smithi060_196073_object54917:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e519)", 2024-09-20T22:21:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:20.806325+0000", 2024-09-20T22:21:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.6288726059999998, 2024-09-20T22:21:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2725259929999999, 2024-09-20T22:21:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 156904 2024-09-20T22:21:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.519 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.806325+0000", 2024-09-20T22:21:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.806325+0000", 2024-09-20T22:21:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.806325+0000", 2024-09-20T22:21:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.06e-07 2024-09-20T22:21:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.520 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.806329+0000", 2024-09-20T22:21:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.4740000000000001e-06 2024-09-20T22:21:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.806338+0000", 2024-09-20T22:21:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.3209999999999994e-06 2024-09-20T22:21:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.806345+0000", 2024-09-20T22:21:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.4080000000000003e-06 2024-09-20T22:21:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.521 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.806417+0000", 2024-09-20T22:21:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2143000000000004e-05 2024-09-20T22:21:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.806427+0000", 2024-09-20T22:21:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0271e-05 2024-09-20T22:21:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.057870+0000", 2024-09-20T22:21:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2514429030000001 2024-09-20T22:21:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.522 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.057970+0000", 2024-09-20T22:21:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.9778999999999995e-05 2024-09-20T22:21:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:21:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.058082+0000", 2024-09-20T22:21:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000112525 2024-09-20T22:21:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.060717+0000", 2024-09-20T22:21:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0026347219999999999 2024-09-20T22:21:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.523 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.066863+0000", 2024-09-20T22:21:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061463639999999996 2024-09-20T22:21:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.078729+0000", 2024-09-20T22:21:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.011866008000000001 2024-09-20T22:21:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.078769+0000", 2024-09-20T22:21:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9907999999999998e-05 2024-09-20T22:21:22.524 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.078851+0000", 2024-09-20T22:21:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.1561000000000001e-05 2024-09-20T22:21:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:156916 11.e 11:7d69ffad:::benchmark_data_smithi060_196073_object54929:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e519)", 2024-09-20T22:21:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:20.814930+0000", 2024-09-20T22:21:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.620267879, 2024-09-20T22:21:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.264129421, 2024-09-20T22:21:22.525 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 156916 2024-09-20T22:21:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.814930+0000", 2024-09-20T22:21:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.526 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.814930+0000", 2024-09-20T22:21:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.814931+0000", 2024-09-20T22:21:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.02e-07 2024-09-20T22:21:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.814935+0000", 2024-09-20T22:21:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3100000000000002e-06 2024-09-20T22:21:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.527 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.814947+0000", 2024-09-20T22:21:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2262999999999999e-05 2024-09-20T22:21:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.814955+0000", 2024-09-20T22:21:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.6459999999999993e-06 2024-09-20T22:21:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.815025+0000", 2024-09-20T22:21:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9826999999999994e-05 2024-09-20T22:21:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.528 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.815035+0000", 2024-09-20T22:21:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0002e-05 2024-09-20T22:21:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.060752+0000", 2024-09-20T22:21:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.245717312 2024-09-20T22:21:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.060843+0000", 2024-09-20T22:21:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.0719999999999999e-05 2024-09-20T22:21:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.529 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:21:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.060937+0000", 2024-09-20T22:21:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.4303999999999997e-05 2024-09-20T22:21:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.066732+0000", 2024-09-20T22:21:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0057951959999999999 2024-09-20T22:21:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.072692+0000", 2024-09-20T22:21:22.530 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0059602860000000004 2024-09-20T22:21:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.078863+0000", 2024-09-20T22:21:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0061704990000000003 2024-09-20T22:21:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.078902+0000", 2024-09-20T22:21:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9487000000000003e-05 2024-09-20T22:21:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079059+0000", 2024-09-20T22:21:22.531 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00015666700000000001 2024-09-20T22:21:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:156963 11.e 11:74952a29:::benchmark_data_smithi060_196073_object54976:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e519)", 2024-09-20T22:21:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:20.928803+0000", 2024-09-20T22:21:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.5063942340000001, 2024-09-20T22:21:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.150411673, 2024-09-20T22:21:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 156963 2024-09-20T22:21:22.532 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.928803+0000", 2024-09-20T22:21:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.928803+0000", 2024-09-20T22:21:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.928805+0000", 2024-09-20T22:21:22.533 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.277e-06 2024-09-20T22:21:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.928810+0000", 2024-09-20T22:21:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0420000000000002e-06 2024-09-20T22:21:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.928821+0000", 2024-09-20T22:21:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1521e-05 2024-09-20T22:21:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.534 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.928828+0000", 2024-09-20T22:21:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2840000000000001e-06 2024-09-20T22:21:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.928906+0000", 2024-09-20T22:21:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7401e-05 2024-09-20T22:21:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.928916+0000", 2024-09-20T22:21:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0312999999999999e-05 2024-09-20T22:21:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.535 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.062687+0000", 2024-09-20T22:21:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1337705769999999 2024-09-20T22:21:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.062782+0000", 2024-09-20T22:21:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.5538000000000006e-05 2024-09-20T22:21:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:21:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.062884+0000", 2024-09-20T22:21:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000101383 2024-09-20T22:21:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.536 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.066776+0000", 2024-09-20T22:21:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0038919470000000002 2024-09-20T22:21:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.072788+0000", 2024-09-20T22:21:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0060127269999999998 2024-09-20T22:21:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079069+0000", 2024-09-20T22:21:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0062809989999999998 2024-09-20T22:21:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079127+0000", 2024-09-20T22:21:22.537 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.7605999999999997e-05 2024-09-20T22:21:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079215+0000", 2024-09-20T22:21:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.8058000000000003e-05 2024-09-20T22:21:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:156966 11.e 11:74f2dfd9:::benchmark_data_smithi060_196073_object54979:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e519)", 2024-09-20T22:21:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:20.931650+0000", 2024-09-20T22:21:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.503547787, 2024-09-20T22:21:22.538 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1476938569999999, 2024-09-20T22:21:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 156966 2024-09-20T22:21:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.931650+0000", 2024-09-20T22:21:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.539 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.931650+0000", 2024-09-20T22:21:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.931651+0000", 2024-09-20T22:21:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.313e-06 2024-09-20T22:21:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.931656+0000", 2024-09-20T22:21:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.1189999999999996e-06 2024-09-20T22:21:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.540 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.931670+0000", 2024-09-20T22:21:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3747000000000001e-05 2024-09-20T22:21:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.931677+0000", 2024-09-20T22:21:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5610000000000001e-06 2024-09-20T22:21:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.931753+0000", 2024-09-20T22:21:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.5512999999999993e-05 2024-09-20T22:21:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.541 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.931763+0000", 2024-09-20T22:21:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0496000000000001e-05 2024-09-20T22:21:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.064873+0000", 2024-09-20T22:21:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1331094939999999 2024-09-20T22:21:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.064977+0000", 2024-09-20T22:21:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00010414199999999999 2024-09-20T22:21:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.542 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:21:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.065090+0000", 2024-09-20T22:21:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011257 2024-09-20T22:21:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.071394+0000", 2024-09-20T22:21:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0063039819999999996 2024-09-20T22:21:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.072830+0000", 2024-09-20T22:21:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0014364880000000001 2024-09-20T22:21:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.543 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079225+0000", 2024-09-20T22:21:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0063951709999999998 2024-09-20T22:21:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079264+0000", 2024-09-20T22:21:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.9159999999999998e-05 2024-09-20T22:21:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079344+0000", 2024-09-20T22:21:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.9101e-05 2024-09-20T22:21:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.544 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:156993 11.e 11:78efed92:::benchmark_data_smithi060_196073_object55006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e519)", 2024-09-20T22:21:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:20.958993+0000", 2024-09-20T22:21:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.476204112, 2024-09-20T22:21:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.120487155, 2024-09-20T22:21:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 156993 2024-09-20T22:21:22.545 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.958993+0000", 2024-09-20T22:21:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.958993+0000", 2024-09-20T22:21:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.546 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.958995+0000", 2024-09-20T22:21:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1990000000000001e-06 2024-09-20T22:21:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.959000+0000", 2024-09-20T22:21:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0300000000000001e-06 2024-09-20T22:21:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.959011+0000", 2024-09-20T22:21:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1664999999999999e-05 2024-09-20T22:21:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.959019+0000", 2024-09-20T22:21:22.547 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.7640000000000005e-06 2024-09-20T22:21:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.959069+0000", 2024-09-20T22:21:22.548 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.0470000000000003e-05 2024-09-20T22:21:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.959077+0000", 2024-09-20T22:21:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.289e-06 2024-09-20T22:21:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.066904+0000", 2024-09-20T22:21:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1078270610000001 2024-09-20T22:21:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.067017+0000", 2024-09-20T22:21:22.549 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011328 2024-09-20T22:21:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:21:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.067118+0000", 2024-09-20T22:21:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000100739 2024-09-20T22:21:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.072321+0000", 2024-09-20T22:21:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.005203526 2024-09-20T22:21:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.072865+0000", 2024-09-20T22:21:22.550 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00054349499999999996 2024-09-20T22:21:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079351+0000", 2024-09-20T22:21:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0064864730000000004 2024-09-20T22:21:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079400+0000", 2024-09-20T22:21:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8590000000000001e-05 2024-09-20T22:21:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.551 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079481+0000", 2024-09-20T22:21:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0574000000000005e-05 2024-09-20T22:21:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:157011 11.e 11:78d3545e:::benchmark_data_smithi060_196073_object55024:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e519)", 2024-09-20T22:21:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:20.977705+0000", 2024-09-20T22:21:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.457492233, 2024-09-20T22:21:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1019112179999999, 2024-09-20T22:21:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.552 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 157011 2024-09-20T22:21:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.977705+0000", 2024-09-20T22:21:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.977705+0000", 2024-09-20T22:21:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.553 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.977706+0000", 2024-09-20T22:21:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.012e-06 2024-09-20T22:21:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.977709+0000", 2024-09-20T22:21:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.202e-06 2024-09-20T22:21:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.977726+0000", 2024-09-20T22:21:22.554 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.6458999999999999e-05 2024-09-20T22:21:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.977733+0000", 2024-09-20T22:21:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.5899999999999996e-06 2024-09-20T22:21:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.977782+0000", 2024-09-20T22:21:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9724e-05 2024-09-20T22:21:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.977789+0000", 2024-09-20T22:21:22.555 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.9040000000000003e-06 2024-09-20T22:21:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.068729+0000", 2024-09-20T22:21:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.090939374 2024-09-20T22:21:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.068841+0000", 2024-09-20T22:21:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011259100000000001 2024-09-20T22:21:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:21:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.068959+0000", 2024-09-20T22:21:22.556 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011802399999999999 2024-09-20T22:21:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.072745+0000", 2024-09-20T22:21:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0037862939999999999 2024-09-20T22:21:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.072901+0000", 2024-09-20T22:21:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000156057 2024-09-20T22:21:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079491+0000", 2024-09-20T22:21:22.557 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.006589648 2024-09-20T22:21:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079534+0000", 2024-09-20T22:21:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3186000000000003e-05 2024-09-20T22:21:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079616+0000", 2024-09-20T22:21:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.2152999999999998e-05 2024-09-20T22:21:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.558 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:157021 11.e 11:7ea0cfde:::benchmark_data_smithi060_196073_object55034:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e519)", 2024-09-20T22:21:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:20.985885+0000", 2024-09-20T22:21:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.449312183, 2024-09-20T22:21:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.093959372, 2024-09-20T22:21:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 157021 2024-09-20T22:21:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.559 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.985885+0000", 2024-09-20T22:21:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.985885+0000", 2024-09-20T22:21:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.985886+0000", 2024-09-20T22:21:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.0999999999999998e-07 2024-09-20T22:21:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.560 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.985890+0000", 2024-09-20T22:21:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3760000000000001e-06 2024-09-20T22:21:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.985902+0000", 2024-09-20T22:21:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1364e-05 2024-09-20T22:21:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.985907+0000", 2024-09-20T22:21:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9830000000000004e-06 2024-09-20T22:21:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.561 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.985961+0000", 2024-09-20T22:21:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.4351000000000002e-05 2024-09-20T22:21:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:20.985974+0000", 2024-09-20T22:21:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.2614e-05 2024-09-20T22:21:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.072929+0000", 2024-09-20T22:21:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0869554429999999 2024-09-20T22:21:22.562 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.073026+0000", 2024-09-20T22:21:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.6745999999999998e-05 2024-09-20T22:21:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:21:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.073136+0000", 2024-09-20T22:21:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00011044600000000001 2024-09-20T22:21:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079659+0000", 2024-09-20T22:21:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0065231530000000003 2024-09-20T22:21:22.563 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079671+0000", 2024-09-20T22:21:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.1807000000000001e-05 2024-09-20T22:21:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079714+0000", 2024-09-20T22:21:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2976999999999999e-05 2024-09-20T22:21:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079760+0000", 2024-09-20T22:21:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.6010000000000002e-05 2024-09-20T22:21:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.564 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079845+0000", 2024-09-20T22:21:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.4392000000000001e-05 2024-09-20T22:21:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:157044 11.e 11:7d2b25f6:::benchmark_data_smithi060_196073_object55057:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e519)", 2024-09-20T22:21:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:21.020020+0000", 2024-09-20T22:21:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.4151772920000001, 2024-09-20T22:21:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0600218640000001, 2024-09-20T22:21:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.565 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 157044 2024-09-20T22:21:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:21.020020+0000", 2024-09-20T22:21:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.566 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:21.020020+0000", 2024-09-20T22:21:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:21.020022+0000", 2024-09-20T22:21:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.39e-06 2024-09-20T22:21:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:21.020026+0000", 2024-09-20T22:21:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.8029999999999996e-06 2024-09-20T22:21:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.567 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:21.020040+0000", 2024-09-20T22:21:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.3585e-05 2024-09-20T22:21:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:21.020049+0000", 2024-09-20T22:21:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.1099999999999992e-06 2024-09-20T22:21:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:21.020113+0000", 2024-09-20T22:21:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3536999999999998e-05 2024-09-20T22:21:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.568 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:21.020119+0000", 2024-09-20T22:21:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 6.3759999999999996e-06 2024-09-20T22:21:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.074963+0000", 2024-09-20T22:21:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.054844471 2024-09-20T22:21:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.075084+0000", 2024-09-20T22:21:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000120171 2024-09-20T22:21:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.569 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:21:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.075219+0000", 2024-09-20T22:21:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00013489900000000001 2024-09-20T22:21:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079676+0000", 2024-09-20T22:21:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0044574740000000003 2024-09-20T22:21:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079883+0000", 2024-09-20T22:21:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00020679100000000001 2024-09-20T22:21:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.570 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079919+0000", 2024-09-20T22:21:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 3.6041000000000003e-05 2024-09-20T22:21:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079962+0000", 2024-09-20T22:21:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.2899999999999999e-05 2024-09-20T22:21:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.080042+0000", 2024-09-20T22:21:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 8.0315999999999995e-05 2024-09-20T22:21:22.571 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "description": "osd_op(client.16440.0:157056 11.e 11:773a1f98:::benchmark_data_smithi060_196073_object55069:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e519)", 2024-09-20T22:21:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "initiated_at": "2024-09-20T22:21:21.064368+0000", 2024-09-20T22:21:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "age": 1.370829141, 2024-09-20T22:21:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.0197750059999999, 2024-09-20T22:21:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "continuous": false, 2024-09-20T22:21:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "type_data": { 2024-09-20T22:21:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "client_info": { 2024-09-20T22:21:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "client": "client.16440", 2024-09-20T22:21:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:22.572 INFO:teuthology.orchestra.run.smithi067.stdout: "tid": 157056 2024-09-20T22:21:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "events": [ 2024-09-20T22:21:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "initiated", 2024-09-20T22:21:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:21.064368+0000", 2024-09-20T22:21:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "header_read", 2024-09-20T22:21:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:21.064368+0000", 2024-09-20T22:21:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0 2024-09-20T22:21:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "throttled", 2024-09-20T22:21:22.573 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:21.064369+0000", 2024-09-20T22:21:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.2699999999999999e-07 2024-09-20T22:21:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "all_read", 2024-09-20T22:21:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:21.064373+0000", 2024-09-20T22:21:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.3490000000000002e-06 2024-09-20T22:21:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "dispatched", 2024-09-20T22:21:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:21.064389+0000", 2024-09-20T22:21:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.5478999999999999e-05 2024-09-20T22:21:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.574 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "queued_for_pg", 2024-09-20T22:21:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:21.064397+0000", 2024-09-20T22:21:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.8320000000000006e-06 2024-09-20T22:21:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:21.064456+0000", 2024-09-20T22:21:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 5.8835999999999997e-05 2024-09-20T22:21:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for peered", 2024-09-20T22:21:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:21.064463+0000", 2024-09-20T22:21:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 7.3919999999999997e-06 2024-09-20T22:21:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.575 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "reached_pg", 2024-09-20T22:21:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.076756+0000", 2024-09-20T22:21:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 1.012292677 2024-09-20T22:21:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "started", 2024-09-20T22:21:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.076859+0000", 2024-09-20T22:21:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000103714 2024-09-20T22:21:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "waiting for subops from 0,3", 2024-09-20T22:21:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.076978+0000", 2024-09-20T22:21:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.000118353 2024-09-20T22:21:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.576 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "op_commit", 2024-09-20T22:21:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.079681+0000", 2024-09-20T22:21:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0027033510000000001 2024-09-20T22:21:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.080081+0000", 2024-09-20T22:21:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.00039983599999999998 2024-09-20T22:21:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.083995+0000", 2024-09-20T22:21:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 0.0039144510000000002 2024-09-20T22:21:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "commit_sent", 2024-09-20T22:21:22.577 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.084045+0000", 2024-09-20T22:21:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 4.9935999999999998e-05 2024-09-20T22:21:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: }, 2024-09-20T22:21:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: { 2024-09-20T22:21:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: "event": "done", 2024-09-20T22:21:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: "time": "2024-09-20T22:21:22.084143+0000", 2024-09-20T22:21:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: "duration": 9.8072999999999994e-05 2024-09-20T22:21:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: } 2024-09-20T22:21:22.578 INFO:teuthology.orchestra.run.smithi067.stdout: ] 2024-09-20T22:21:22.578 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:21:22.579 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:22.617 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:22.617 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:22.618 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:22.718 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:22.719 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:21:22.728 INFO:teuthology.orchestra.run.smithi060.stderr:set pool 1 pgp_num to 31 2024-09-20T22:21:22.751 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:22.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:22.752 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:22.853 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:22.890 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:22.891 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:22.891 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:22.992 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:23.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:21:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:21:23.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:22 smithi067 ceph-mon[33511]: osdmap e520: 8 total, 7 up, 6 in 2024-09-20T22:21:23.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:22 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2956349579' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:21:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:22 smithi067 ceph-mon[33511]: pgmap v991: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 1 peering, 1 active+remapped+backfilling, 175 active+clean; 2.9 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 909 op/s; 165/144096 objects degraded (0.115%); 2023/144096 objects misplaced (1.404%); 39 MiB/s, 629 objects/s recovering 2024-09-20T22:21:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:22 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1168516828' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"}]: dispatch 2024-09-20T22:21:23.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:22 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"}]: dispatch 2024-09-20T22:21:23.028 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:23.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:23.029 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:23.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:22 smithi060 ceph-mon[29694]: osdmap e520: 8 total, 7 up, 6 in 2024-09-20T22:21:23.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:22 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2956349579' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:21:23.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:22 smithi060 ceph-mon[29694]: pgmap v991: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 1 peering, 1 active+remapped+backfilling, 175 active+clean; 2.9 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 909 op/s; 165/144096 objects degraded (0.115%); 2023/144096 objects misplaced (1.404%); 39 MiB/s, 629 objects/s recovering 2024-09-20T22:21:23.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:22 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1168516828' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"}]: dispatch 2024-09-20T22:21:23.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:22 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"}]: dispatch 2024-09-20T22:21:23.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:22 smithi060 ceph-mon[40290]: osdmap e520: 8 total, 7 up, 6 in 2024-09-20T22:21:23.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:22 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2956349579' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:21:23.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:22 smithi060 ceph-mon[40290]: pgmap v991: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 1 peering, 1 active+remapped+backfilling, 175 active+clean; 2.9 GiB data, 14 GiB used, 523 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 909 op/s; 165/144096 objects degraded (0.115%); 2023/144096 objects misplaced (1.404%); 39 MiB/s, 629 objects/s recovering 2024-09-20T22:21:23.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:22 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1168516828' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"}]: dispatch 2024-09-20T22:21:23.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:22 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"}]: dispatch 2024-09-20T22:21:23.130 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:23.172 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:23.173 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:23.173 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:23.274 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:23.307 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:23.307 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:23.308 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:23.408 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:23.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:23.456 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:23.457 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:23.464 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T22:21:23.558 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:23.602 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:23.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:23.603 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:23.704 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:23.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:23.744 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:23.744 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:23.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:23 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:21:23.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:23 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2956349579' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:21:23.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:23 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"}]': finished 2024-09-20T22:21:23.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:23 smithi060 ceph-mon[40290]: osdmap e521: 8 total, 7 up, 6 in 2024-09-20T22:21:23.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:23 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:21:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:23 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:21:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:23 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2956349579' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:21:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:23 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"}]': finished 2024-09-20T22:21:23.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:23 smithi060 ceph-mon[29694]: osdmap e521: 8 total, 7 up, 6 in 2024-09-20T22:21:23.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:23 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:21:23.845 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:23.876 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:23.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:23.878 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:23.978 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:23 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:21:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:23 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2956349579' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:21:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:23 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": ".mgr", "var": "pgp_num", "val": "31"}]': finished 2024-09-20T22:21:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:23 smithi067 ceph-mon[33511]: osdmap e521: 8 total, 7 up, 6 in 2024-09-20T22:21:24.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:23 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:21:24.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:24.014 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:24.014 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:24.115 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:24.148 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:24.148 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:24.149 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:24.250 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:24.286 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:24.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:24.287 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:24.388 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:24.427 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:24.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:24.429 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:24.530 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:24.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:24.573 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:24.573 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:24.674 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:24.700 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:24.701 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:24.701 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:24.802 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:24.838 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:24.839 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:24.839 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:24.940 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:24.973 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:24.973 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:24.974 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:24 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:21:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:24 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:21:25.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:24 smithi067 ceph-mon[33511]: pgmap v993: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 1 peering, 1 active+remapped+backfilling, 175 active+clean; 2.8 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 897 B/s rd, 0 B/s wr, 787 op/s; 165/139902 objects degraded (0.118%); 2023/139902 objects misplaced (1.446%); 32 MiB/s, 518 objects/s recovering 2024-09-20T22:21:25.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:24 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:21:25.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:24 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:21:25.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:24 smithi060 ceph-mon[29694]: pgmap v993: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 1 peering, 1 active+remapped+backfilling, 175 active+clean; 2.8 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 897 B/s rd, 0 B/s wr, 787 op/s; 165/139902 objects degraded (0.118%); 2023/139902 objects misplaced (1.446%); 32 MiB/s, 518 objects/s recovering 2024-09-20T22:21:25.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:24 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:21:25.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:24 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:21:25.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:24 smithi060 ceph-mon[40290]: pgmap v993: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 1 peering, 1 active+remapped+backfilling, 175 active+clean; 2.8 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 897 B/s rd, 0 B/s wr, 787 op/s; 165/139902 objects degraded (0.118%); 2023/139902 objects misplaced (1.446%); 32 MiB/s, 518 objects/s recovering 2024-09-20T22:21:25.075 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:25.117 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:25.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:25.118 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:25.219 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:25.254 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:25.254 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:25.255 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:25.355 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:25.370 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:21:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:21:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:21:25.389 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:25.390 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:25.391 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:25.491 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:25.525 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:25.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:25.526 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:25.626 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:25.660 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:25.661 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:25.661 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:25.762 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:25.793 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:25.800 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:25.801 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:25.801 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:25.865 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:25.902 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:25.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:25.940 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:25.941 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:26.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:25 smithi067 ceph-mon[33511]: osdmap e522: 8 total, 7 up, 6 in 2024-09-20T22:21:26.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:25 smithi060 ceph-mon[29694]: osdmap e522: 8 total, 7 up, 6 in 2024-09-20T22:21:26.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:25 smithi060 ceph-mon[40290]: osdmap e522: 8 total, 7 up, 6 in 2024-09-20T22:21:26.041 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:26.077 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:26.078 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:26.078 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:26.179 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:26.214 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:26.215 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:26.215 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:26.316 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:26.353 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:26.353 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:26.354 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:26.454 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:26.489 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:26.490 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:26.490 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:26.591 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:26.628 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:26.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:26.629 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:26.730 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:26.767 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:26.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:26.768 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:26.816 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:21:26.817 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:21:26.849 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:21:26.849 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:21:26.870 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:26.873 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:26 smithi060 ceph-mon[40290]: pgmap v995: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfilling, 176 active+clean; 2.7 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 1.14k op/s; 165/133974 objects degraded (0.123%); 2023/133974 objects misplaced (1.510%); 32 MiB/s, 519 objects/s recovering 2024-09-20T22:21:26.873 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:26 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 165/139902 objects degraded (0.118%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T22:21:26.873 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:26 smithi060 ceph-mon[40290]: osdmap e523: 8 total, 7 up, 6 in 2024-09-20T22:21:26.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:26 smithi060 ceph-mon[29694]: pgmap v995: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfilling, 176 active+clean; 2.7 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 1.14k op/s; 165/133974 objects degraded (0.123%); 2023/133974 objects misplaced (1.510%); 32 MiB/s, 519 objects/s recovering 2024-09-20T22:21:26.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:26 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 165/139902 objects degraded (0.118%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T22:21:26.874 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:26 smithi060 ceph-mon[29694]: osdmap e523: 8 total, 7 up, 6 in 2024-09-20T22:21:26.882 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:21:26.882 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:21:26.908 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:26.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:26.910 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:26.924 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:21:26.924 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:21:26.927 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-20T22:21:26.962 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:21:26.962 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:21:27.003 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:21:27.004 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:21:27.011 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:27.035 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:21:27.035 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:21:27.039 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:21:27.040 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:21:27.057 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:27.058 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:27.058 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:27.147 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:26 smithi067 ceph-mon[33511]: pgmap v995: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfilling, 176 active+clean; 2.7 GiB data, 13 GiB used, 523 GiB / 536 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 1.14k op/s; 165/133974 objects degraded (0.123%); 2023/133974 objects misplaced (1.510%); 32 MiB/s, 519 objects/s recovering 2024-09-20T22:21:27.147 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:26 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 165/139902 objects degraded (0.118%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T22:21:27.147 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:26 smithi067 ceph-mon[33511]: osdmap e523: 8 total, 7 up, 6 in 2024-09-20T22:21:27.158 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:27.195 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:27.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:27.196 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:27.224 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:21:27.297 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:27.335 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:27.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:27.336 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:27.437 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:27.483 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:27.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:27.484 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:27.585 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:27.609 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:27.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:27.609 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:27.710 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:27.745 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:27.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:27.747 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:27.847 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:27.881 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:27.882 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:27.882 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:27.983 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:28.007 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:28.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:28.008 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:28.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:27 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/786600161' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:28.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:27 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/786600161' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:28.109 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:28.131 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:28.132 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:28.132 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:28.233 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:28.257 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:28.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:28.258 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:28.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:27 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/786600161' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:28.358 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:28.380 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:28.381 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:28.381 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:28.482 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:28.505 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:28.505 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:28.506 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:28.606 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:28.629 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:28.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:28.630 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:28.731 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:28.754 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:28.754 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:28.755 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:28.856 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:28.886 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:28.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:28.887 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:28.987 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:29.019 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:29.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:29.020 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:29.121 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:29.158 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:29.159 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:29.159 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:29.260 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:28 smithi067 ceph-mon[33511]: osdmap e524: 8 total, 7 up, 6 in 2024-09-20T22:21:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:28 smithi067 ceph-mon[33511]: pgmap v998: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfilling, 176 active+clean; 2.5 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 901 B/s rd, 0 B/s wr, 1.10k op/s; 162/124653 objects degraded (0.130%); 611/124653 objects misplaced (0.490%); 15 MiB/s, 236 objects/s recovering 2024-09-20T22:21:29.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:28 smithi060 ceph-mon[29694]: osdmap e524: 8 total, 7 up, 6 in 2024-09-20T22:21:29.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:28 smithi060 ceph-mon[29694]: pgmap v998: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfilling, 176 active+clean; 2.5 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 901 B/s rd, 0 B/s wr, 1.10k op/s; 162/124653 objects degraded (0.130%); 611/124653 objects misplaced (0.490%); 15 MiB/s, 236 objects/s recovering 2024-09-20T22:21:29.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:28 smithi060 ceph-mon[40290]: osdmap e524: 8 total, 7 up, 6 in 2024-09-20T22:21:29.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:28 smithi060 ceph-mon[40290]: pgmap v998: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfilling, 176 active+clean; 2.5 GiB data, 13 GiB used, 524 GiB / 536 GiB avail; 901 B/s rd, 0 B/s wr, 1.10k op/s; 162/124653 objects degraded (0.130%); 611/124653 objects misplaced (0.490%); 15 MiB/s, 236 objects/s recovering 2024-09-20T22:21:29.297 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:29.298 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:29.298 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:29.399 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:29.421 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:29.421 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:29.421 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:29.522 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:29.557 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:29.558 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:29.558 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:29.659 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:29.691 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:29.692 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:29.692 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:29.793 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:29.829 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:29.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:29.830 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:29.931 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:29.968 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:29.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:29.969 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:30.070 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:30.095 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:30.095 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:30.096 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:30.196 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:30.229 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:30.230 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:30.230 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:30.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:29 smithi067 ceph-mon[33511]: osdmap e525: 8 total, 7 up, 6 in 2024-09-20T22:21:30.285 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:29 smithi060 ceph-mon[29694]: osdmap e525: 8 total, 7 up, 6 in 2024-09-20T22:21:30.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:29 smithi060 ceph-mon[40290]: osdmap e525: 8 total, 7 up, 6 in 2024-09-20T22:21:30.331 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:30.367 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:30.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:30.369 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:30.396 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:21:30.397 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:21:30.469 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:30.508 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:30.508 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:30.509 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:30.610 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:30.646 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:30.646 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:30.647 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:30.748 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:30.784 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:30.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:30.785 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:30.843 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T22:21:30.850 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 5, 6, 7, 0] out_osds: [4, 1] dead_osds: [4] live_osds: [0, 2, 3, 5, 6, 7, 1] 2024-09-20T22:21:30.850 INFO:tasks.thrashosds.thrasher:Reviving osd 4 2024-09-20T22:21:30.851 INFO:tasks.cephadm.osd.4:Restarting osd.4 (starting--it wasn't running)... 2024-09-20T22:21:30.851 DEBUG:teuthology.orchestra.run.smithi067:osd.4> sudo journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.4.service 2024-09-20T22:21:30.853 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl start ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.4 2024-09-20T22:21:30.886 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:30.928 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:30.934 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:30.935 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:30.935 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:31.035 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:31.071 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:30 smithi060 ceph-mon[29694]: pgmap v1000: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfilling, 176 active+clean; 2.4 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 908 B/s rd, 0 B/s wr, 1.21k op/s; 162/119436 objects degraded (0.136%); 611/119436 objects misplaced (0.512%); 15 MiB/s, 238 objects/s recovering 2024-09-20T22:21:31.071 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:30 smithi060 ceph-mon[40290]: pgmap v1000: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfilling, 176 active+clean; 2.4 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 908 B/s rd, 0 B/s wr, 1.21k op/s; 162/119436 objects degraded (0.136%); 611/119436 objects misplaced (0.512%); 15 MiB/s, 238 objects/s recovering 2024-09-20T22:21:31.087 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:31.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:31.088 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:31.188 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:31.225 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:31.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:31.226 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:31.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:30 smithi067 ceph-mon[33511]: pgmap v1000: 178 pgs: 1 active+recovering+undersized+degraded+remapped, 1 active+remapped+backfilling, 176 active+clean; 2.4 GiB data, 12 GiB used, 524 GiB / 536 GiB avail; 908 B/s rd, 0 B/s wr, 1.21k op/s; 162/119436 objects degraded (0.136%); 611/119436 objects misplaced (0.512%); 15 MiB/s, 238 objects/s recovering 2024-09-20T22:21:31.263 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:30 smithi067 systemd[1]: Starting Ceph osd.4 for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:21:31.327 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:31.357 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:31.358 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:31.358 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:31.459 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:31.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:31.497 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:31.497 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:31.557 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:31 smithi067 podman[154300]: 2024-09-20 22:21:31.177135036 +0000 UTC m=+0.027251813 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5 2024-09-20T22:21:31.598 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:31.607 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:31.627 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:31.628 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:31.628 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:31.729 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:31.762 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:31.762 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:31.763 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:31.817 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:31 smithi067 podman[154300]: 2024-09-20 22:21:31.556974518 +0000 UTC m=+0.407091304 container create 799e5c28aa12e5356ec28849854ea4008cc90b3785c162e9601efbb2a7516a4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-activate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-9e2ac70, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-20T22:21:31.863 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:31.885 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:31.886 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:31.886 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:31.987 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:32.009 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:32.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:32.010 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:32.028 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:31 smithi060 ceph-mon[29694]: Health check update: Degraded data redundancy: 162/119436 objects degraded (0.136%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T22:21:32.028 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:31 smithi060 ceph-mon[29694]: osdmap e526: 8 total, 7 up, 6 in 2024-09-20T22:21:32.028 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:31 smithi060 ceph-mon[40290]: Health check update: Degraded data redundancy: 162/119436 objects degraded (0.136%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T22:21:32.028 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:31 smithi060 ceph-mon[40290]: osdmap e526: 8 total, 7 up, 6 in 2024-09-20T22:21:32.111 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:32.133 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:32.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:32.134 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:32.144 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:21:32.144 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:21:32.235 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:32.258 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:32.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:32.259 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:32.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:31 smithi067 ceph-mon[33511]: Health check update: Degraded data redundancy: 162/119436 objects degraded (0.136%), 1 pg degraded, 1 pg undersized (PG_DEGRADED) 2024-09-20T22:21:32.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:31 smithi067 ceph-mon[33511]: osdmap e526: 8 total, 7 up, 6 in 2024-09-20T22:21:32.360 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:32.382 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:32.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:32.384 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:32.398 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:21:32.484 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:32.511 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:32.512 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:32.512 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:32.613 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:32.639 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:32.639 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:32.640 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:32.702 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:21:32.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:32.759 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T22:21:32.766 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:32.766 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:32.767 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:32.868 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:32.911 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:32.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:32.912 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:33.012 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:33.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:32 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1603849334' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:21:33.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:32 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:21:33.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:32 smithi067 ceph-mon[33511]: pgmap v1002: 178 pgs: 1 peering, 177 active+clean; 2.2 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 897 B/s rd, 0 B/s wr, 1.51k op/s; 10 MiB/s, 193 objects/s recovering 2024-09-20T22:21:33.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:32 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:21:33.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:32 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:21:33.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:32 smithi067 ceph-mon[33511]: osdmap e527: 8 total, 7 up, 6 in 2024-09-20T22:21:33.014 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:21:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:21:33.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:32 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1603849334' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:21:33.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:32 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:21:33.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:32 smithi060 ceph-mon[40290]: pgmap v1002: 178 pgs: 1 peering, 177 active+clean; 2.2 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 897 B/s rd, 0 B/s wr, 1.51k op/s; 10 MiB/s, 193 objects/s recovering 2024-09-20T22:21:33.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:32 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:21:33.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:32 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:21:33.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:32 smithi060 ceph-mon[40290]: osdmap e527: 8 total, 7 up, 6 in 2024-09-20T22:21:33.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:32 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1603849334' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:21:33.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:32 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:21:33.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:32 smithi060 ceph-mon[29694]: pgmap v1002: 178 pgs: 1 peering, 177 active+clean; 2.2 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 897 B/s rd, 0 B/s wr, 1.51k op/s; 10 MiB/s, 193 objects/s recovering 2024-09-20T22:21:33.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:32 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:21:33.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:32 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:21:33.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:32 smithi060 ceph-mon[29694]: osdmap e527: 8 total, 7 up, 6 in 2024-09-20T22:21:33.064 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:33.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:33.066 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:33.166 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:33.200 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:33.200 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:33.200 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:33.301 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:33.339 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:33.340 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:33.340 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:33.441 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:33.468 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:33.468 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:33.468 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:33.568 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:33.611 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:33.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:33.612 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:33.621 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:33.633 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:33.714 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:33.736 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:33.736 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:33.737 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:33.837 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:34.026 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:34.027 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:34.028 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:34.034 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:33 smithi060 ceph-mon[29694]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 162/119436 objects degraded (0.136%), 1 pg degraded, 1 pg undersized) 2024-09-20T22:21:34.034 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:33 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:21:34.034 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:33 smithi060 ceph-mon[40290]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 162/119436 objects degraded (0.136%), 1 pg degraded, 1 pg undersized) 2024-09-20T22:21:34.034 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:33 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:21:34.129 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:34.162 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:34.163 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:34.163 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:34.190 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:33 smithi067 ceph-mon[33511]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 162/119436 objects degraded (0.136%), 1 pg degraded, 1 pg undersized) 2024-09-20T22:21:34.190 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:33 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:21:34.264 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:34.301 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:34.302 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:34.302 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:34.403 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:34.434 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:34.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:34.435 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:34.459 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:34 smithi067 podman[154300]: 2024-09-20 22:21:34.189654218 +0000 UTC m=+3.039771014 container init 799e5c28aa12e5356ec28849854ea4008cc90b3785c162e9601efbb2a7516a4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-activate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-9e2ac70, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-20T22:21:34.460 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:34 smithi067 podman[154300]: 2024-09-20 22:21:34.198783755 +0000 UTC m=+3.048900537 container start 799e5c28aa12e5356ec28849854ea4008cc90b3785c162e9601efbb2a7516a4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-activate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=squid-9e2ac70, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-20T22:21:34.488 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:21:34.488 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:21:34.488 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:21:34.488 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:21:34.512 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:21:34.512 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:21:34.536 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:34.538 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:21:34.539 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:21:34.543 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-20T22:21:34.576 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:34.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:34.577 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:34.579 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:21:34.579 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:21:34.597 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:21:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "size": 20, 2024-09-20T22:21:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 600, 2024-09-20T22:21:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:21:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:125840 9.2 9:4a8ba3ac:::benchmark_data_smithi060_140132_object15032:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:21:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:30.317157+0000", 2024-09-20T22:21:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 484.25199069799999, 2024-09-20T22:21:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.39984702300000002, 2024-09-20T22:21:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:21:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:21:34.598 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:21:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 125840 2024-09-20T22:21:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317157+0000", 2024-09-20T22:21:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317157+0000", 2024-09-20T22:21:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.599 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317159+0000", 2024-09-20T22:21:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4929999999999999e-06 2024-09-20T22:21:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317165+0000", 2024-09-20T22:21:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8610000000000003e-06 2024-09-20T22:21:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317181+0000", 2024-09-20T22:21:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6133999999999998e-05 2024-09-20T22:21:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317188+0000", 2024-09-20T22:21:34.600 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2840000000000001e-06 2024-09-20T22:21:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317259+0000", 2024-09-20T22:21:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.0816999999999996e-05 2024-09-20T22:21:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317363+0000", 2024-09-20T22:21:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103702 2024-09-20T22:21:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3", 2024-09-20T22:21:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.317465+0000", 2024-09-20T22:21:34.601 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010202 2024-09-20T22:21:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:21:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.320561+0000", 2024-09-20T22:21:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0030965540000000001 2024-09-20T22:21:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.321130+0000", 2024-09-20T22:21:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00056882299999999998 2024-09-20T22:21:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.716883+0000", 2024-09-20T22:21:34.602 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.39575325 2024-09-20T22:21:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.716956+0000", 2024-09-20T22:21:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.2352000000000001e-05 2024-09-20T22:21:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.717004+0000", 2024-09-20T22:21:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.8733000000000002e-05 2024-09-20T22:21:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.603 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:125859 9.0 9:05dacd16:::benchmark_data_smithi060_140132_object15051:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:21:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:30.331372+0000", 2024-09-20T22:21:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 484.23777563499999, 2024-09-20T22:21:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.38946761499999999, 2024-09-20T22:21:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:21:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:21:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:21:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 125859 2024-09-20T22:21:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.604 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331372+0000", 2024-09-20T22:21:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331372+0000", 2024-09-20T22:21:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331374+0000", 2024-09-20T22:21:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5290000000000001e-06 2024-09-20T22:21:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.605 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331379+0000", 2024-09-20T22:21:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.502e-06 2024-09-20T22:21:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331397+0000", 2024-09-20T22:21:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.7249e-05 2024-09-20T22:21:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331406+0000", 2024-09-20T22:21:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0659999999999993e-06 2024-09-20T22:21:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.606 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331474+0000", 2024-09-20T22:21:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8396e-05 2024-09-20T22:21:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331609+0000", 2024-09-20T22:21:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013487800000000001 2024-09-20T22:21:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,6", 2024-09-20T22:21:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.331730+0000", 2024-09-20T22:21:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000121408 2024-09-20T22:21:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:21:34.607 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.334028+0000", 2024-09-20T22:21:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022976429999999998 2024-09-20T22:21:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.334978+0000", 2024-09-20T22:21:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00095049999999999996 2024-09-20T22:21:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720566+0000", 2024-09-20T22:21:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.38558782699999999 2024-09-20T22:21:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.608 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720619+0000", 2024-09-20T22:21:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2404000000000002e-05 2024-09-20T22:21:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720840+0000", 2024-09-20T22:21:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000221213 2024-09-20T22:21:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:125862 9.2 9:4de341df:::benchmark_data_smithi060_140132_object15054:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:21:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:30.336602+0000", 2024-09-20T22:21:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 484.23254608600001, 2024-09-20T22:21:34.609 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.38056002500000002, 2024-09-20T22:21:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:21:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:21:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:21:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 125862 2024-09-20T22:21:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336602+0000", 2024-09-20T22:21:34.610 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336602+0000", 2024-09-20T22:21:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336603+0000", 2024-09-20T22:21:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.3799999999999996e-07 2024-09-20T22:21:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.611 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336608+0000", 2024-09-20T22:21:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9350000000000002e-06 2024-09-20T22:21:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336624+0000", 2024-09-20T22:21:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5849e-05 2024-09-20T22:21:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336645+0000", 2024-09-20T22:21:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1067999999999999e-05 2024-09-20T22:21:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.612 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336704+0000", 2024-09-20T22:21:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9525999999999999e-05 2024-09-20T22:21:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336824+0000", 2024-09-20T22:21:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011957 2024-09-20T22:21:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,3", 2024-09-20T22:21:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.336942+0000", 2024-09-20T22:21:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117819 2024-09-20T22:21:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.613 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.339146+0000", 2024-09-20T22:21:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022047680000000002 2024-09-20T22:21:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:21:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.339234+0000", 2024-09-20T22:21:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7968999999999998e-05 2024-09-20T22:21:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.717065+0000", 2024-09-20T22:21:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.37783098799999998 2024-09-20T22:21:34.614 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.717120+0000", 2024-09-20T22:21:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.5133999999999998e-05 2024-09-20T22:21:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.717162+0000", 2024-09-20T22:21:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1461000000000002e-05 2024-09-20T22:21:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.615 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:125869 9.0 9:0c9f6e22:::benchmark_data_smithi060_140132_object15061:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:21:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:30.340895+0000", 2024-09-20T22:21:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 484.22825274799999, 2024-09-20T22:21:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.38006649199999998, 2024-09-20T22:21:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:21:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:21:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:21:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 125869 2024-09-20T22:21:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.616 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340895+0000", 2024-09-20T22:21:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340895+0000", 2024-09-20T22:21:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340896+0000", 2024-09-20T22:21:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.31e-06 2024-09-20T22:21:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.617 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340902+0000", 2024-09-20T22:21:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.9370000000000001e-06 2024-09-20T22:21:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340917+0000", 2024-09-20T22:21:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4181e-05 2024-09-20T22:21:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340924+0000", 2024-09-20T22:21:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.3499999999999999e-06 2024-09-20T22:21:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.618 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.340992+0000", 2024-09-20T22:21:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7887999999999998e-05 2024-09-20T22:21:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.341093+0000", 2024-09-20T22:21:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000101074 2024-09-20T22:21:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,6", 2024-09-20T22:21:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.341159+0000", 2024-09-20T22:21:34.619 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6167999999999994e-05 2024-09-20T22:21:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:21:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343748+0000", 2024-09-20T22:21:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0025892060000000001 2024-09-20T22:21:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.348961+0000", 2024-09-20T22:21:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.005213082 2024-09-20T22:21:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720878+0000", 2024-09-20T22:21:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.37191645499999998 2024-09-20T22:21:34.620 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720928+0000", 2024-09-20T22:21:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.9888000000000001e-05 2024-09-20T22:21:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720962+0000", 2024-09-20T22:21:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.3952999999999998e-05 2024-09-20T22:21:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.621 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:125872 9.0 9:0ae55481:::benchmark_data_smithi060_140132_object15064:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:21:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:30.343972+0000", 2024-09-20T22:21:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 484.22517595300002, 2024-09-20T22:21:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.37716829000000002, 2024-09-20T22:21:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:21:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:21:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:21:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 125872 2024-09-20T22:21:34.622 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343972+0000", 2024-09-20T22:21:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343972+0000", 2024-09-20T22:21:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.623 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343973+0000", 2024-09-20T22:21:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2640000000000001e-06 2024-09-20T22:21:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343978+0000", 2024-09-20T22:21:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.1569999999999999e-06 2024-09-20T22:21:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343991+0000", 2024-09-20T22:21:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2313000000000001e-05 2024-09-20T22:21:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.624 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.343998+0000", 2024-09-20T22:21:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7379999999999994e-06 2024-09-20T22:21:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344068+0000", 2024-09-20T22:21:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9962000000000005e-05 2024-09-20T22:21:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344184+0000", 2024-09-20T22:21:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000116063 2024-09-20T22:21:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.625 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,6", 2024-09-20T22:21:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344284+0000", 2024-09-20T22:21:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.9779999999999997e-05 2024-09-20T22:21:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:21:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.348919+0000", 2024-09-20T22:21:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0046344170000000001 2024-09-20T22:21:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.349022+0000", 2024-09-20T22:21:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000103521 2024-09-20T22:21:34.626 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.720993+0000", 2024-09-20T22:21:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.37197118600000001 2024-09-20T22:21:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.721105+0000", 2024-09-20T22:21:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000111161 2024-09-20T22:21:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.721140+0000", 2024-09-20T22:21:34.627 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.5728000000000001e-05 2024-09-20T22:21:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.15594.0:125874 9.0 9:01e41a72:::benchmark_data_smithi060_140132_object15066:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e338)", 2024-09-20T22:21:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:13:30.344246+0000", 2024-09-20T22:21:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 484.22490179200003, 2024-09-20T22:21:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.37701091199999998, 2024-09-20T22:21:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:21:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.15594", 2024-09-20T22:21:34.628 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2825583262", 2024-09-20T22:21:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 125874 2024-09-20T22:21:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344246+0000", 2024-09-20T22:21:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344246+0000", 2024-09-20T22:21:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.629 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344247+0000", 2024-09-20T22:21:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.6099999999999999e-07 2024-09-20T22:21:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344252+0000", 2024-09-20T22:21:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5129999999999998e-06 2024-09-20T22:21:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344263+0000", 2024-09-20T22:21:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1744e-05 2024-09-20T22:21:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.630 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.344271+0000", 2024-09-20T22:21:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8709999999999997e-06 2024-09-20T22:21:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.346350+0000", 2024-09-20T22:21:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002079072 2024-09-20T22:21:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.346467+0000", 2024-09-20T22:21:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000117207 2024-09-20T22:21:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.631 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,6", 2024-09-20T22:21:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.346581+0000", 2024-09-20T22:21:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000113241 2024-09-20T22:21:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:21:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.348991+0000", 2024-09-20T22:21:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0024100020000000001 2024-09-20T22:21:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.349689+0000", 2024-09-20T22:21:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00069810599999999999 2024-09-20T22:21:34.632 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.721177+0000", 2024-09-20T22:21:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.37148772800000002 2024-09-20T22:21:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.721218+0000", 2024-09-20T22:21:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1366999999999999e-05 2024-09-20T22:21:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:13:30.721257+0000", 2024-09-20T22:21:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.9100000000000002e-05 2024-09-20T22:21:34.633 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16005.0:215023 10.2 10:4760a86c:::benchmark_data_smithi060_167766_object90231:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e439)", 2024-09-20T22:21:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:04.035908+0000", 2024-09-20T22:21:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 210.533240377, 2024-09-20T22:21:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.35885745800000002, 2024-09-20T22:21:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:21:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16005", 2024-09-20T22:21:34.634 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/2400248125", 2024-09-20T22:21:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 215023 2024-09-20T22:21:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.035908+0000", 2024-09-20T22:21:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.035908+0000", 2024-09-20T22:21:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.635 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.035909+0000", 2024-09-20T22:21:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3880000000000001e-06 2024-09-20T22:21:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.035916+0000", 2024-09-20T22:21:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6619999999999999e-06 2024-09-20T22:21:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.035932+0000", 2024-09-20T22:21:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6495e-05 2024-09-20T22:21:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.636 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.035941+0000", 2024-09-20T22:21:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.0720000000000006e-06 2024-09-20T22:21:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.036009+0000", 2024-09-20T22:21:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.8192e-05 2024-09-20T22:21:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.036098+0000", 2024-09-20T22:21:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.8960000000000002e-05 2024-09-20T22:21:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.637 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,4", 2024-09-20T22:21:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.036450+0000", 2024-09-20T22:21:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00035144700000000001 2024-09-20T22:21:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:21:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.039224+0000", 2024-09-20T22:21:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0027737870000000002 2024-09-20T22:21:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.039810+0000", 2024-09-20T22:21:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00058632699999999999 2024-09-20T22:21:34.638 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.394602+0000", 2024-09-20T22:21:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.35479249099999999 2024-09-20T22:21:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.394708+0000", 2024-09-20T22:21:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010586699999999999 2024-09-20T22:21:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:04.394765+0000", 2024-09-20T22:21:34.639 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.677e-05 2024-09-20T22:21:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16005.0:230256 10.b e445/370 10:d5079f28:::benchmark_data_smithi060_167766_object105464:head v 445'14547, mlcod=445'14544)", 2024-09-20T22:21:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:18:17.914810+0000", 2024-09-20T22:21:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 196.654337424, 2024-09-20T22:21:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.28074447600000002, 2024-09-20T22:21:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.640 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:17.914810+0000", 2024-09-20T22:21:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:17.914810+0000", 2024-09-20T22:21:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.641 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:17.914812+0000", 2024-09-20T22:21:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0109999999999999e-06 2024-09-20T22:21:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:17.914819+0000", 2024-09-20T22:21:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.909e-06 2024-09-20T22:21:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:18.185825+0000", 2024-09-20T22:21:34.642 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.27100544799999998 2024-09-20T22:21:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:18.185839+0000", 2024-09-20T22:21:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.4042e-05 2024-09-20T22:21:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:18.185948+0000", 2024-09-20T22:21:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010950099999999999 2024-09-20T22:21:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.643 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:18.185975+0000", 2024-09-20T22:21:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6285999999999999e-05 2024-09-20T22:21:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:18.195518+0000", 2024-09-20T22:21:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0095431590000000007 2024-09-20T22:21:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:18:18.195555+0000", 2024-09-20T22:21:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7119999999999997e-05 2024-09-20T22:21:34.644 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:67552 11.e 11:7d44dee1:::benchmark_data_smithi060_196073_object67551:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e477)", 2024-09-20T22:21:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:19:36.969362+0000", 2024-09-20T22:21:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 117.599785938, 2024-09-20T22:21:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.307205169, 2024-09-20T22:21:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:21:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:21:34.645 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 67552 2024-09-20T22:21:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.969362+0000", 2024-09-20T22:21:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.969362+0000", 2024-09-20T22:21:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.646 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.969363+0000", 2024-09-20T22:21:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7199999999999999e-07 2024-09-20T22:21:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.969385+0000", 2024-09-20T22:21:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2498999999999999e-05 2024-09-20T22:21:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.969393+0000", 2024-09-20T22:21:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5290000000000003e-06 2024-09-20T22:21:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.647 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.969398+0000", 2024-09-20T22:21:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0100000000000003e-06 2024-09-20T22:21:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.969485+0000", 2024-09-20T22:21:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7681000000000003e-05 2024-09-20T22:21:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.969655+0000", 2024-09-20T22:21:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.000170228 2024-09-20T22:21:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:21:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.969846+0000", 2024-09-20T22:21:34.648 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019016399999999999 2024-09-20T22:21:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:21:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.974920+0000", 2024-09-20T22:21:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0050747320000000002 2024-09-20T22:21:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.974968+0000", 2024-09-20T22:21:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.7769000000000003e-05 2024-09-20T22:21:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.649 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:37.269959+0000", 2024-09-20T22:21:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.29499124599999998 2024-09-20T22:21:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:37.270159+0000", 2024-09-20T22:21:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00019975799999999999 2024-09-20T22:21:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:37.276567+0000", 2024-09-20T22:21:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0064079810000000001 2024-09-20T22:21:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.650 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:67554 11.e 11:7a22042b:::benchmark_data_smithi060_196073_object67553:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e477)", 2024-09-20T22:21:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:19:36.970608+0000", 2024-09-20T22:21:34.651 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 117.598539831, 2024-09-20T22:21:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.31237448899999998, 2024-09-20T22:21:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:21:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:21:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 67554 2024-09-20T22:21:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.652 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.970608+0000", 2024-09-20T22:21:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.970608+0000", 2024-09-20T22:21:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.970609+0000", 2024-09-20T22:21:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1230000000000001e-06 2024-09-20T22:21:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.653 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.970633+0000", 2024-09-20T22:21:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3340999999999999e-05 2024-09-20T22:21:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.970649+0000", 2024-09-20T22:21:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6629e-05 2024-09-20T22:21:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.970656+0000", 2024-09-20T22:21:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1269999999999997e-06 2024-09-20T22:21:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.654 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.972508+0000", 2024-09-20T22:21:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0018516920000000001 2024-09-20T22:21:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.972667+0000", 2024-09-20T22:21:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015891599999999999 2024-09-20T22:21:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:21:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.972857+0000", 2024-09-20T22:21:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018995000000000001 2024-09-20T22:21:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.655 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.975025+0000", 2024-09-20T22:21:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021676540000000002 2024-09-20T22:21:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:21:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.975040+0000", 2024-09-20T22:21:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5549999999999999e-05 2024-09-20T22:21:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:37.279616+0000", 2024-09-20T22:21:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.30457592500000003 2024-09-20T22:21:34.656 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:37.279675+0000", 2024-09-20T22:21:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8881000000000001e-05 2024-09-20T22:21:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:37.282983+0000", 2024-09-20T22:21:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0033077010000000001 2024-09-20T22:21:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.657 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:67562 11.e 11:79af1391:::benchmark_data_smithi060_196073_object67561:head [set-alloc-hint object_size 65536 write_size 65536,write 0~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e477)", 2024-09-20T22:21:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:19:36.986448+0000", 2024-09-20T22:21:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 117.58269971599999, 2024-09-20T22:21:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.299800705, 2024-09-20T22:21:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:21:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:21:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 67562 2024-09-20T22:21:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.658 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.986448+0000", 2024-09-20T22:21:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.986448+0000", 2024-09-20T22:21:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.986450+0000", 2024-09-20T22:21:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.353e-06 2024-09-20T22:21:34.659 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.986473+0000", 2024-09-20T22:21:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3753000000000001e-05 2024-09-20T22:21:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.986487+0000", 2024-09-20T22:21:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3416e-05 2024-09-20T22:21:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.986494+0000", 2024-09-20T22:21:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.9059999999999996e-06 2024-09-20T22:21:34.660 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.986555+0000", 2024-09-20T22:21:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.143e-05 2024-09-20T22:21:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.986663+0000", 2024-09-20T22:21:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00010813 2024-09-20T22:21:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 0,7", 2024-09-20T22:21:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.986820+0000", 2024-09-20T22:21:34.661 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00015657900000000001 2024-09-20T22:21:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:21:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.988713+0000", 2024-09-20T22:21:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0018935829999999999 2024-09-20T22:21:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:36.989407+0000", 2024-09-20T22:21:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00069353500000000001 2024-09-20T22:21:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "sub_op_commit_rec", 2024-09-20T22:21:34.662 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:37.283075+0000", 2024-09-20T22:21:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.29366799100000002 2024-09-20T22:21:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:37.283141+0000", 2024-09-20T22:21:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.5852000000000006e-05 2024-09-20T22:21:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:19:37.286249+0000", 2024-09-20T22:21:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0031081770000000002 2024-09-20T22:21:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.663 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16440.0:85135 11.b e489/481 11:db385f6e:::benchmark_data_smithi060_196073_object85134:head v 489'5371, mlcod=489'5369)", 2024-09-20T22:21:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:20:01.722378+0000", 2024-09-20T22:21:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 92.846769601999995, 2024-09-20T22:21:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.26851586, 2024-09-20T22:21:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.722378+0000", 2024-09-20T22:21:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.664 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.722378+0000", 2024-09-20T22:21:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.722379+0000", 2024-09-20T22:21:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.111e-06 2024-09-20T22:21:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.722410+0000", 2024-09-20T22:21:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.0914999999999998e-05 2024-09-20T22:21:34.665 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.722426+0000", 2024-09-20T22:21:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5948999999999999e-05 2024-09-20T22:21:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.722438+0000", 2024-09-20T22:21:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1807999999999999e-05 2024-09-20T22:21:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.723260+0000", 2024-09-20T22:21:34.666 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00082200100000000005 2024-09-20T22:21:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.723285+0000", 2024-09-20T22:21:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.4720999999999999e-05 2024-09-20T22:21:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.990669+0000", 2024-09-20T22:21:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.267384018 2024-09-20T22:21:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.990894+0000", 2024-09-20T22:21:34.667 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00022533699999999999 2024-09-20T22:21:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16440.0:85124 11.7 e489/481 11:ed9c27de:::benchmark_data_smithi060_196073_object85123:head v 489'5249, mlcod=461'706)", 2024-09-20T22:21:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:20:01.724467+0000", 2024-09-20T22:21:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 92.844681246999997, 2024-09-20T22:21:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.269957049, 2024-09-20T22:21:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.668 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.724467+0000", 2024-09-20T22:21:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.724467+0000", 2024-09-20T22:21:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.724468+0000", 2024-09-20T22:21:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.102e-06 2024-09-20T22:21:34.669 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.724510+0000", 2024-09-20T22:21:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.1770999999999997e-05 2024-09-20T22:21:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.724522+0000", 2024-09-20T22:21:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.2289e-05 2024-09-20T22:21:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.724529+0000", 2024-09-20T22:21:34.670 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5660000000000001e-06 2024-09-20T22:21:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.724597+0000", 2024-09-20T22:21:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.7401000000000001e-05 2024-09-20T22:21:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.986858+0000", 2024-09-20T22:21:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.26226096500000001 2024-09-20T22:21:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.671 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.994244+0000", 2024-09-20T22:21:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0073859290000000003 2024-09-20T22:21:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.994424+0000", 2024-09-20T22:21:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00018002599999999999 2024-09-20T22:21:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16440.0:85137 11.3 e489/481 11:caef5a62:::benchmark_data_smithi060_196073_object85136:head v 489'5221, mlcod=489'5219)", 2024-09-20T22:21:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:20:01.724561+0000", 2024-09-20T22:21:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 92.844586624000002, 2024-09-20T22:21:34.672 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.26911784399999999, 2024-09-20T22:21:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.724561+0000", 2024-09-20T22:21:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.724561+0000", 2024-09-20T22:21:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.673 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.724562+0000", 2024-09-20T22:21:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.5900000000000002e-07 2024-09-20T22:21:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.724594+0000", 2024-09-20T22:21:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.2157999999999999e-05 2024-09-20T22:21:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.724602+0000", 2024-09-20T22:21:34.674 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.661e-06 2024-09-20T22:21:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.986239+0000", 2024-09-20T22:21:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.26163731400000001 2024-09-20T22:21:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.988490+0000", 2024-09-20T22:21:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0022515069999999998 2024-09-20T22:21:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.675 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.988528+0000", 2024-09-20T22:21:34.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.7725999999999998e-05 2024-09-20T22:21:34.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.993622+0000", 2024-09-20T22:21:34.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0050936189999999998 2024-09-20T22:21:34.676 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.676 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.676 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.676 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.993679+0000", 2024-09-20T22:21:34.676 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.7399999999999999e-05 2024-09-20T22:21:34.676 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.676 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16440.0:85144 11.b e489/481 11:da68ff6e:::benchmark_data_smithi060_196073_object85143:head v 489'5372, mlcod=489'5369)", 2024-09-20T22:21:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:20:01.732401+0000", 2024-09-20T22:21:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 92.836747173999996, 2024-09-20T22:21:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.268858445, 2024-09-20T22:21:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.677 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.732401+0000", 2024-09-20T22:21:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.732401+0000", 2024-09-20T22:21:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.732402+0000", 2024-09-20T22:21:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3880000000000001e-06 2024-09-20T22:21:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.678 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.679 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:21:34.679 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:21:34.679 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.732440+0000", 2024-09-20T22:21:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8358000000000001e-05 2024-09-20T22:21:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.986195+0000", 2024-09-20T22:21:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.25375451300000001 2024-09-20T22:21:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.986216+0000", 2024-09-20T22:21:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.1335000000000002e-05 2024-09-20T22:21:34.680 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.991048+0000", 2024-09-20T22:21:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0048319749999999996 2024-09-20T22:21:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:01.991102+0000", 2024-09-20T22:21:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3919000000000003e-05 2024-09-20T22:21:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.000676+0000", 2024-09-20T22:21:34.681 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0095736299999999996 2024-09-20T22:21:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:20:02.001259+0000", 2024-09-20T22:21:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00058332700000000002 2024-09-20T22:21:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16440.0:137142 11.b e512/481 11:d48527d3:::benchmark_data_smithi060_196073_object35155:head v 512'8688, mlcod=512'8686)", 2024-09-20T22:21:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:21:01.822423+0000", 2024-09-20T22:21:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 32.746725062000003, 2024-09-20T22:21:34.682 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.19812216199999999, 2024-09-20T22:21:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:01.822423+0000", 2024-09-20T22:21:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:01.822423+0000", 2024-09-20T22:21:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.683 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:01.822424+0000", 2024-09-20T22:21:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.114e-06 2024-09-20T22:21:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:01.822430+0000", 2024-09-20T22:21:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4659999999999996e-06 2024-09-20T22:21:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:01.822442+0000", 2024-09-20T22:21:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1328000000000001e-05 2024-09-20T22:21:34.684 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:01.822449+0000", 2024-09-20T22:21:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.43e-06 2024-09-20T22:21:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:01.822539+0000", 2024-09-20T22:21:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9492000000000003e-05 2024-09-20T22:21:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:01.822562+0000", 2024-09-20T22:21:34.685 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3411999999999999e-05 2024-09-20T22:21:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:02.020468+0000", 2024-09-20T22:21:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.19790592600000001 2024-09-20T22:21:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:02.020545+0000", 2024-09-20T22:21:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6994000000000002e-05 2024-09-20T22:21:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.686 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16440.0:137140 11.7 e512/481 11:eb946c5f:::benchmark_data_smithi060_196073_object35153:head v 512'8443, mlcod=512'8441)", 2024-09-20T22:21:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:21:01.822454+0000", 2024-09-20T22:21:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 32.746694392999999, 2024-09-20T22:21:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.199380002, 2024-09-20T22:21:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:01.822454+0000", 2024-09-20T22:21:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.687 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:01.822454+0000", 2024-09-20T22:21:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:01.822454+0000", 2024-09-20T22:21:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.2699999999999998e-07 2024-09-20T22:21:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:01.822462+0000", 2024-09-20T22:21:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4869999999999996e-06 2024-09-20T22:21:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.688 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:01.822474+0000", 2024-09-20T22:21:34.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1684e-05 2024-09-20T22:21:34.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:01.822481+0000", 2024-09-20T22:21:34.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.238e-06 2024-09-20T22:21:34.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.689 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:01.822555+0000", 2024-09-20T22:21:34.689 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4422e-05 2024-09-20T22:21:34.689 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.689 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.689 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:01.822578+0000", 2024-09-20T22:21:34.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.2912e-05 2024-09-20T22:21:34.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:02.021795+0000", 2024-09-20T22:21:34.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.19921692699999999 2024-09-20T22:21:34.690 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.690 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.690 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.690 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:02.021834+0000", 2024-09-20T22:21:34.690 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 3.8405000000000002e-05 2024-09-20T22:21:34.690 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.690 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.691 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.691 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.691 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16440.0:140406 11.b e512/481 11:d383b498:::benchmark_data_smithi060_196073_object38419:head v 512'8902, mlcod=512'8898)", 2024-09-20T22:21:34.691 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:21:04.849647+0000", 2024-09-20T22:21:34.691 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 29.719500774, 2024-09-20T22:21:34.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.26377603399999999, 2024-09-20T22:21:34.691 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.691 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.691 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.691 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.691 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.691 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.691 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:04.849647+0000", 2024-09-20T22:21:34.691 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:04.849647+0000", 2024-09-20T22:21:34.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:04.849648+0000", 2024-09-20T22:21:34.692 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1299999999999998e-07 2024-09-20T22:21:34.692 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.692 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.692 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.692 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:04.849654+0000", 2024-09-20T22:21:34.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.8069999999999996e-06 2024-09-20T22:21:34.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:04.849664+0000", 2024-09-20T22:21:34.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0482000000000001e-05 2024-09-20T22:21:34.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:04.849671+0000", 2024-09-20T22:21:34.693 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.657e-06 2024-09-20T22:21:34.693 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.693 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.693 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.693 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:04.849747+0000", 2024-09-20T22:21:34.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.6279999999999995e-05 2024-09-20T22:21:34.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:04.849770+0000", 2024-09-20T22:21:34.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.228e-05 2024-09-20T22:21:34.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.694 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:05.113373+0000", 2024-09-20T22:21:34.694 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.26360361300000001 2024-09-20T22:21:34.694 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.694 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.694 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.695 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:05.113423+0000", 2024-09-20T22:21:34.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.0002e-05 2024-09-20T22:21:34.695 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.695 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.695 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.695 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.695 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.695 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16440.0:140411 11.e e512/509 11:7c8abfb9:::benchmark_data_smithi060_196073_object38424:head v 512'8661, mlcod=512'8659)", 2024-09-20T22:21:34.695 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:21:04.853101+0000", 2024-09-20T22:21:34.695 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 29.716047160999999, 2024-09-20T22:21:34.695 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.26837232900000002, 2024-09-20T22:21:34.695 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.695 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.695 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.696 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:04.853101+0000", 2024-09-20T22:21:34.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.696 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:04.853101+0000", 2024-09-20T22:21:34.696 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.696 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.696 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.696 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:04.853102+0000", 2024-09-20T22:21:34.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 9.1800000000000004e-07 2024-09-20T22:21:34.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:04.853110+0000", 2024-09-20T22:21:34.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.8450000000000003e-06 2024-09-20T22:21:34.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.697 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.697 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.697 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:04.853123+0000", 2024-09-20T22:21:34.697 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3169e-05 2024-09-20T22:21:34.697 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:05.113357+0000", 2024-09-20T22:21:34.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.260233979 2024-09-20T22:21:34.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:05.113431+0000", 2024-09-20T22:21:34.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.4129999999999997e-05 2024-09-20T22:21:34.698 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.698 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.698 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.698 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:05.113457+0000", 2024-09-20T22:21:34.698 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.6088000000000002e-05 2024-09-20T22:21:34.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:05.121431+0000", 2024-09-20T22:21:34.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.007973707 2024-09-20T22:21:34.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.699 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.699 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.699 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:05.121473+0000", 2024-09-20T22:21:34.699 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 4.2493000000000001e-05 2024-09-20T22:21:34.699 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.699 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.699 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.699 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.700 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16440.0:140413 11.b e512/481 11:d1b0a080:::benchmark_data_smithi060_196073_object38426:head v 512'8903, mlcod=512'8898)", 2024-09-20T22:21:34.700 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:21:04.853954+0000", 2024-09-20T22:21:34.700 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 29.715193620000001, 2024-09-20T22:21:34.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.26529228700000002, 2024-09-20T22:21:34.700 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:34.700 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:34.700 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:34.700 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:34.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.700 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:34.700 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:04.853954+0000", 2024-09-20T22:21:34.700 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.700 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.700 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:34.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:04.853954+0000", 2024-09-20T22:21:34.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:34.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:34.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:04.853955+0000", 2024-09-20T22:21:34.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.1699999999999997e-07 2024-09-20T22:21:34.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.701 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.701 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:34.701 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:04.853963+0000", 2024-09-20T22:21:34.701 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.5780000000000001e-06 2024-09-20T22:21:34.701 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:34.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:05.113356+0000", 2024-09-20T22:21:34.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.25939383700000002 2024-09-20T22:21:34.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:34.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:05.113370+0000", 2024-09-20T22:21:34.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3993e-05 2024-09-20T22:21:34.702 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.702 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.702 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:34.702 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:05.113453+0000", 2024-09-20T22:21:34.702 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.2126999999999995e-05 2024-09-20T22:21:34.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:34.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:05.113476+0000", 2024-09-20T22:21:34.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 2.3496999999999999e-05 2024-09-20T22:21:34.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:34.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:05.119180+0000", 2024-09-20T22:21:34.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0057042610000000004 2024-09-20T22:21:34.703 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:34.703 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:34.703 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "done", 2024-09-20T22:21:34.703 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:05.119247+0000", 2024-09-20T22:21:34.703 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.6277000000000003e-05 2024-09-20T22:21:34.704 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.704 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.704 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.704 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:34.704 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:34.704 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:21:34.704 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:21:34.705 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:21:34.705 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:34.740 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:34.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:34.742 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:34.763 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:34 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-activate[154434]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T22:21:34.763 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:34 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-activate[154434]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T22:21:34.843 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:34.867 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:34.867 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:34.868 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:34.969 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:35.000 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:34 smithi060 ceph-mon[29694]: 1.3 scrub starts 2024-09-20T22:21:35.000 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:34 smithi060 ceph-mon[29694]: 1.3 scrub ok 2024-09-20T22:21:35.000 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:34 smithi060 ceph-mon[29694]: 4.7 scrub starts 2024-09-20T22:21:35.000 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:34 smithi060 ceph-mon[29694]: 4.7 scrub ok 2024-09-20T22:21:35.000 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:34 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/218996336' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:35.000 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:34 smithi060 ceph-mon[29694]: pgmap v1004: 178 pgs: 1 peering, 177 active+clean; 2.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.26k op/s; 6.3 MiB/s, 128 objects/s recovering 2024-09-20T22:21:35.000 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:34 smithi060 ceph-mon[29694]: 1.1a scrub starts 2024-09-20T22:21:35.000 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:34 smithi060 ceph-mon[29694]: 1.1a scrub ok 2024-09-20T22:21:35.001 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:34 smithi060 ceph-mon[40290]: 1.3 scrub starts 2024-09-20T22:21:35.001 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:34 smithi060 ceph-mon[40290]: 1.3 scrub ok 2024-09-20T22:21:35.001 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:34 smithi060 ceph-mon[40290]: 4.7 scrub starts 2024-09-20T22:21:35.001 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:34 smithi060 ceph-mon[40290]: 4.7 scrub ok 2024-09-20T22:21:35.001 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:34 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/218996336' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:35.001 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:34 smithi060 ceph-mon[40290]: pgmap v1004: 178 pgs: 1 peering, 177 active+clean; 2.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.26k op/s; 6.3 MiB/s, 128 objects/s recovering 2024-09-20T22:21:35.001 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:34 smithi060 ceph-mon[40290]: 1.1a scrub starts 2024-09-20T22:21:35.001 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:34 smithi060 ceph-mon[40290]: 1.1a scrub ok 2024-09-20T22:21:35.005 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:35.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:35.007 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:35.107 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:35.116 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 bash[154300]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T22:21:35.116 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 bash[154300]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-20T22:21:35.116 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 podman[154300]: 2024-09-20 22:21:35.015597407 +0000 UTC m=+3.865714186 container attach 799e5c28aa12e5356ec28849854ea4008cc90b3785c162e9601efbb2a7516a4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-activate, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-9e2ac70, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True) 2024-09-20T22:21:35.116 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-activate[154434]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-20T22:21:35.116 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:34 smithi067 ceph-mon[33511]: 1.3 scrub starts 2024-09-20T22:21:35.117 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:34 smithi067 ceph-mon[33511]: 1.3 scrub ok 2024-09-20T22:21:35.117 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:34 smithi067 ceph-mon[33511]: 4.7 scrub starts 2024-09-20T22:21:35.117 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:34 smithi067 ceph-mon[33511]: 4.7 scrub ok 2024-09-20T22:21:35.117 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:34 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/218996336' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:35.117 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:34 smithi067 ceph-mon[33511]: pgmap v1004: 178 pgs: 1 peering, 177 active+clean; 2.1 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 853 B/s rd, 0 B/s wr, 1.26k op/s; 6.3 MiB/s, 128 objects/s recovering 2024-09-20T22:21:35.117 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:34 smithi067 ceph-mon[33511]: 1.1a scrub starts 2024-09-20T22:21:35.117 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:34 smithi067 ceph-mon[33511]: 1.1a scrub ok 2024-09-20T22:21:35.143 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:35.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:35.144 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:35.245 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:35.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:35.281 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:35.281 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:35.285 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:21:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:21:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:21:35.382 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:35.419 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:35.420 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:35.420 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:35.514 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 bash[154300]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-20T22:21:35.514 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-activate[154434]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-20T22:21:35.514 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 bash[154300]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-20T22:21:35.514 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-activate[154434]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-20T22:21:35.514 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 bash[154300]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-20T22:21:35.514 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-activate[154434]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-20T22:21:35.515 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 bash[154300]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-20T22:21:35.515 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-activate[154434]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-4/block 2024-09-20T22:21:35.515 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 bash[154300]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-4/block 2024-09-20T22:21:35.515 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-activate[154434]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-20T22:21:35.515 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 bash[154300]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-20T22:21:35.515 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-activate[154434]: --> ceph-volume raw activate successful for osd ID: 4 2024-09-20T22:21:35.515 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 bash[154300]: --> ceph-volume raw activate successful for osd ID: 4 2024-09-20T22:21:35.515 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 podman[154300]: 2024-09-20 22:21:35.211705759 +0000 UTC m=+4.061822543 container died 799e5c28aa12e5356ec28849854ea4008cc90b3785c162e9601efbb2a7516a4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-activate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-9e2ac70, ceph=True, org.label-schema.schema-version=1.0) 2024-09-20T22:21:35.521 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:35.543 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:21:35.557 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:35.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:35.558 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:35.660 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:35.708 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:35.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:35.709 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:35.810 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:35.858 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:35.859 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:35.859 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:35.960 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:36.006 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:36.006 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:36.007 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:36.009 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_ops_in_flight 2024-09-20T22:21:36.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:35 smithi060 ceph-mon[29694]: 1.5 scrub starts 2024-09-20T22:21:36.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:35 smithi060 ceph-mon[29694]: 1.5 scrub ok 2024-09-20T22:21:36.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:35 smithi060 ceph-mon[29694]: 4.12 scrub starts 2024-09-20T22:21:36.092 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:35 smithi060 ceph-mon[29694]: 4.12 scrub ok 2024-09-20T22:21:36.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:35 smithi060 ceph-mon[29694]: 1.13 scrub starts 2024-09-20T22:21:36.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:35 smithi060 ceph-mon[29694]: 1.13 scrub ok 2024-09-20T22:21:36.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:35 smithi060 ceph-mon[29694]: 1.e scrub starts 2024-09-20T22:21:36.093 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:35 smithi060 ceph-mon[29694]: 1.e scrub ok 2024-09-20T22:21:36.093 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:35 smithi060 ceph-mon[40290]: 1.5 scrub starts 2024-09-20T22:21:36.093 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:35 smithi060 ceph-mon[40290]: 1.5 scrub ok 2024-09-20T22:21:36.093 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:35 smithi060 ceph-mon[40290]: 4.12 scrub starts 2024-09-20T22:21:36.093 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:35 smithi060 ceph-mon[40290]: 4.12 scrub ok 2024-09-20T22:21:36.093 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:35 smithi060 ceph-mon[40290]: 1.13 scrub starts 2024-09-20T22:21:36.094 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:35 smithi060 ceph-mon[40290]: 1.13 scrub ok 2024-09-20T22:21:36.094 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:35 smithi060 ceph-mon[40290]: 1.e scrub starts 2024-09-20T22:21:36.094 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:35 smithi060 ceph-mon[40290]: 1.e scrub ok 2024-09-20T22:21:36.108 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:36.143 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:36.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:36.144 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:36.191 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:35 smithi067 podman[154699]: 2024-09-20 22:21:35.895659082 +0000 UTC m=+0.674151261 container remove 799e5c28aa12e5356ec28849854ea4008cc90b3785c162e9601efbb2a7516a4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-activate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-9e2ac70, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-20T22:21:36.191 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:35 smithi067 ceph-mon[33511]: 1.5 scrub starts 2024-09-20T22:21:36.191 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:35 smithi067 ceph-mon[33511]: 1.5 scrub ok 2024-09-20T22:21:36.191 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:35 smithi067 ceph-mon[33511]: 4.12 scrub starts 2024-09-20T22:21:36.191 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:35 smithi067 ceph-mon[33511]: 4.12 scrub ok 2024-09-20T22:21:36.191 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:35 smithi067 ceph-mon[33511]: 1.13 scrub starts 2024-09-20T22:21:36.191 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:35 smithi067 ceph-mon[33511]: 1.13 scrub ok 2024-09-20T22:21:36.191 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:35 smithi067 ceph-mon[33511]: 1.e scrub starts 2024-09-20T22:21:36.191 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:35 smithi067 ceph-mon[33511]: 1.e scrub ok 2024-09-20T22:21:36.245 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:36.288 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:36.289 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:36.289 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:36.390 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:36.427 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:36.428 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:36.428 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:36.513 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:36 smithi067 podman[154759]: 2024-09-20 22:21:36.095042044 +0000 UTC m=+0.028691586 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5 2024-09-20T22:21:36.513 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:36 smithi067 podman[154759]: 2024-09-20 22:21:36.2358906 +0000 UTC m=+0.169540140 container create 2781cb10f4db06d3503e21d969e095bef71d3cf615cf8a98d6106692eefa1212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-9e2ac70, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-20T22:21:36.529 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:36.564 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:36.565 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:36.565 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:36.588 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:21:36.589 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:21:36.646 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:36.666 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:36.667 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:36.704 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:36.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:36.705 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:36.762 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T22:21:36.785 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:21:36 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: 2024-09-20T22:21:36.363+0000 7fa5382c7640 -1 mgr get_metadata_python Requested missing service osd.4 2024-09-20T22:21:36.805 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:36.845 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:36.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:36.846 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:36.946 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:36.986 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:36.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:36.987 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:37.013 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:36 smithi067 podman[154759]: 2024-09-20 22:21:36.63355702 +0000 UTC m=+0.567206562 container init 2781cb10f4db06d3503e21d969e095bef71d3cf615cf8a98d6106692eefa1212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=squid-9e2ac70, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-20T22:21:37.013 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:36 smithi067 podman[154759]: 2024-09-20 22:21:36.644191615 +0000 UTC m=+0.577841150 container start 2781cb10f4db06d3503e21d969e095bef71d3cf615cf8a98d6106692eefa1212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-9e2ac70, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-20T22:21:37.013 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:36 smithi067 bash[154759]: 2781cb10f4db06d3503e21d969e095bef71d3cf615cf8a98d6106692eefa1212 2024-09-20T22:21:37.013 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:36 smithi067 systemd[1]: Started Ceph osd.4 for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T22:21:37.088 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:37.151 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:37.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:37.152 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:37.238 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:21:37.253 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:37.305 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:37.305 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:37.305 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:37.406 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:37.444 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:37.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:37.445 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:37.546 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:37.581 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:37.582 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:37.582 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:37.669 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:37 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:21:37.669 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:37 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:21:37.669 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:37 smithi060 ceph-mon[29694]: pgmap v1005: 178 pgs: 1 peering, 177 active+clean; 1.9 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 0 B/s rd, 0 B/s wr, 1.01k op/s; 5.0 MiB/s, 101 objects/s recovering 2024-09-20T22:21:37.669 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:37 smithi060 ceph-mon[29694]: osdmap e528: 8 total, 7 up, 6 in 2024-09-20T22:21:37.669 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:37 smithi060 ceph-mon[29694]: 5.15 scrub starts 2024-09-20T22:21:37.669 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:37 smithi060 ceph-mon[29694]: 5.15 scrub ok 2024-09-20T22:21:37.669 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:37 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:21:37.669 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:37 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:21:37.669 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:37 smithi060 ceph-mon[40290]: pgmap v1005: 178 pgs: 1 peering, 177 active+clean; 1.9 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 0 B/s rd, 0 B/s wr, 1.01k op/s; 5.0 MiB/s, 101 objects/s recovering 2024-09-20T22:21:37.669 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:37 smithi060 ceph-mon[40290]: osdmap e528: 8 total, 7 up, 6 in 2024-09-20T22:21:37.670 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:37 smithi060 ceph-mon[40290]: 5.15 scrub starts 2024-09-20T22:21:37.670 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:37 smithi060 ceph-mon[40290]: 5.15 scrub ok 2024-09-20T22:21:37.683 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:37.714 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:37.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:37.715 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:37.736 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:21:37.737 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:21:37.737 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.737 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:177648 11.6 11:6bed56a3:::benchmark_data_smithi060_196073_object75661:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e528)", 2024-09-20T22:21:37.737 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:21:37.730226+0000", 2024-09-20T22:21:37.737 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.004408688, 2024-09-20T22:21:37.737 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0044392030000000001, 2024-09-20T22:21:37.737 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:37.737 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:37.737 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:21:37.737 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:21:37.737 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:21:37.737 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:37.738 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 177648 2024-09-20T22:21:37.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.738 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:37.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:37.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.730226+0000", 2024-09-20T22:21:37.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:37.738 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.738 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.738 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:37.738 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.730226+0000", 2024-09-20T22:21:37.738 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:37.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:37.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.730228+0000", 2024-09-20T22:21:37.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6920000000000001e-06 2024-09-20T22:21:37.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:37.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.730233+0000", 2024-09-20T22:21:37.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2660000000000001e-06 2024-09-20T22:21:37.739 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.739 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.739 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:37.739 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.730249+0000", 2024-09-20T22:21:37.739 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.5678000000000001e-05 2024-09-20T22:21:37.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:37.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.730257+0000", 2024-09-20T22:21:37.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.0039999999999999e-06 2024-09-20T22:21:37.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:37.740 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.730343+0000", 2024-09-20T22:21:37.740 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.6246000000000001e-05 2024-09-20T22:21:37.740 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.740 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.740 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:37.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.730456+0000", 2024-09-20T22:21:37.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011332099999999999 2024-09-20T22:21:37.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 2,5", 2024-09-20T22:21:37.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.730571+0000", 2024-09-20T22:21:37.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00011503899999999999 2024-09-20T22:21:37.741 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.741 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.741 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "op_commit", 2024-09-20T22:21:37.741 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.733178+0000", 2024-09-20T22:21:37.741 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002607261 2024-09-20T22:21:37.741 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:37.741 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:37.742 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:37.742 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.742 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.742 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16440.0:177649 11.2 e528/456 11:43afbfde:::benchmark_data_smithi060_196073_object75662:head v 528'11027, mlcod=528'11022)", 2024-09-20T22:21:37.742 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:21:37.730775+0000", 2024-09-20T22:21:37.742 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0038599039999999999, 2024-09-20T22:21:37.742 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0039805420000000001, 2024-09-20T22:21:37.742 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:37.742 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:37.742 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "started", 2024-09-20T22:21:37.742 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:37.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:37.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.730775+0000", 2024-09-20T22:21:37.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:37.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:37.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.730775+0000", 2024-09-20T22:21:37.743 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:37.743 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.743 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.743 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:37.743 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.730775+0000", 2024-09-20T22:21:37.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.4899999999999995e-07 2024-09-20T22:21:37.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:37.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.730781+0000", 2024-09-20T22:21:37.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.3020000000000002e-06 2024-09-20T22:21:37.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:37.744 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.730794+0000", 2024-09-20T22:21:37.744 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.3050999999999999e-05 2024-09-20T22:21:37.744 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.744 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.744 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:37.745 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.730801+0000", 2024-09-20T22:21:37.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.939e-06 2024-09-20T22:21:37.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.745 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:37.745 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.732753+0000", 2024-09-20T22:21:37.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0019527749999999999 2024-09-20T22:21:37.745 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.745 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.745 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:37.745 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.732772+0000", 2024-09-20T22:21:37.745 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8182999999999999e-05 2024-09-20T22:21:37.745 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:37.746 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:37.746 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:37.746 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.746 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16440.0:177645 11.b e528/515)", 2024-09-20T22:21:37.746 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:21:37.732628+0000", 2024-09-20T22:21:37.746 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0020066849999999998, 2024-09-20T22:21:37.746 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0021800500000000002, 2024-09-20T22:21:37.746 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:37.746 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:37.746 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:21:37.746 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:37.746 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.746 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:37.746 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.732628+0000", 2024-09-20T22:21:37.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:37.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:37.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.732628+0000", 2024-09-20T22:21:37.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:37.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:37.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.732629+0000", 2024-09-20T22:21:37.747 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9899999999999999e-07 2024-09-20T22:21:37.747 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.747 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.747 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:37.747 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.732636+0000", 2024-09-20T22:21:37.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.849e-06 2024-09-20T22:21:37.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:37.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.732647+0000", 2024-09-20T22:21:37.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.1343e-05 2024-09-20T22:21:37.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.748 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.748 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:37.748 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.732653+0000", 2024-09-20T22:21:37.748 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4489999999999996e-06 2024-09-20T22:21:37.748 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:37.748 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:37.748 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:37.748 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.749 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16440.0:177650 11.1 e528/456 11:84aae127:::benchmark_data_smithi060_196073_object75663:head v 528'11157, mlcod=528'11155)", 2024-09-20T22:21:37.749 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:21:37.732652+0000", 2024-09-20T22:21:37.749 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.0019823129999999999, 2024-09-20T22:21:37.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.002199767, 2024-09-20T22:21:37.749 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:37.749 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:37.749 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "commit sent; apply or cleanup", 2024-09-20T22:21:37.749 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:37.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:37.749 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.732652+0000", 2024-09-20T22:21:37.749 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:37.749 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.749 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.749 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:37.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.732652+0000", 2024-09-20T22:21:37.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:37.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:37.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.732653+0000", 2024-09-20T22:21:37.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.99e-07 2024-09-20T22:21:37.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.750 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:37.750 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.732658+0000", 2024-09-20T22:21:37.750 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.2259999999999996e-06 2024-09-20T22:21:37.750 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.750 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:37.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.732668+0000", 2024-09-20T22:21:37.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.0234000000000001e-05 2024-09-20T22:21:37.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:37.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.732675+0000", 2024-09-20T22:21:37.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4500000000000001e-06 2024-09-20T22:21:37.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.751 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.751 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:37.751 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.732756+0000", 2024-09-20T22:21:37.751 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.1583999999999997e-05 2024-09-20T22:21:37.751 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:37.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.732776+0000", 2024-09-20T22:21:37.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.9182e-05 2024-09-20T22:21:37.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.752 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "commit_sent", 2024-09-20T22:21:37.752 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.734812+0000", 2024-09-20T22:21:37.752 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0020360280000000001 2024-09-20T22:21:37.752 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:37.752 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:37.752 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:37.752 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.752 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.752 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_repop(client.16440.0:177651 11.2 e528/456)", 2024-09-20T22:21:37.753 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:21:37.734531+0000", 2024-09-20T22:21:37.753 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.000103832, 2024-09-20T22:21:37.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00037832000000000002, 2024-09-20T22:21:37.753 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:37.753 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:37.753 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "queued for pg", 2024-09-20T22:21:37.753 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:37.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:37.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.734531+0000", 2024-09-20T22:21:37.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:37.753 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.753 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.753 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:37.753 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.734531+0000", 2024-09-20T22:21:37.753 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:37.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:37.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.734532+0000", 2024-09-20T22:21:37.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.7300000000000005e-07 2024-09-20T22:21:37.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:37.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.734538+0000", 2024-09-20T22:21:37.754 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 6.4579999999999998e-06 2024-09-20T22:21:37.754 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.754 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.754 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:37.754 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.734556+0000", 2024-09-20T22:21:37.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.8153999999999999e-05 2024-09-20T22:21:37.755 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:37.755 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:37.755 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:37.755 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:37.734564+0000", 2024-09-20T22:21:37.755 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 7.9449999999999993e-06 2024-09-20T22:21:37.755 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:37.755 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:37.755 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:37.755 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:37.755 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:21:37.755 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 5 2024-09-20T22:21:37.755 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:21:37.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:37 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:21:37.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:37 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:21:37.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:37 smithi067 ceph-mon[33511]: pgmap v1005: 178 pgs: 1 peering, 177 active+clean; 1.9 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 0 B/s rd, 0 B/s wr, 1.01k op/s; 5.0 MiB/s, 101 objects/s recovering 2024-09-20T22:21:37.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:37 smithi067 ceph-mon[33511]: osdmap e528: 8 total, 7 up, 6 in 2024-09-20T22:21:37.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:37 smithi067 ceph-mon[33511]: 5.15 scrub starts 2024-09-20T22:21:37.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:37 smithi067 ceph-mon[33511]: 5.15 scrub ok 2024-09-20T22:21:37.816 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:37.844 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:37.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:37.845 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:37.945 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:37.980 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:37.981 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:37.982 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:38.082 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:38.121 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:38.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:38.122 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:38.223 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:38.267 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:38.268 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:38.268 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:38.369 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:38.371 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:21:38.405 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:38.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:38.407 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:38.507 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:38.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:38 smithi060 ceph-mon[29694]: 4.9 scrub starts 2024-09-20T22:21:38.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:38 smithi060 ceph-mon[29694]: 4.9 scrub ok 2024-09-20T22:21:38.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:38 smithi060 ceph-mon[29694]: 3.f scrub starts 2024-09-20T22:21:38.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:38 smithi060 ceph-mon[29694]: 3.f scrub ok 2024-09-20T22:21:38.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:38 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/345719715' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:21:38.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:38 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:21:38.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:38 smithi060 ceph-mon[40290]: 4.9 scrub starts 2024-09-20T22:21:38.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:38 smithi060 ceph-mon[40290]: 4.9 scrub ok 2024-09-20T22:21:38.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:38 smithi060 ceph-mon[40290]: 3.f scrub starts 2024-09-20T22:21:38.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:38 smithi060 ceph-mon[40290]: 3.f scrub ok 2024-09-20T22:21:38.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:38 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/345719715' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:21:38.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:38 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:21:38.554 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:38.555 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:38.555 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:38.555 INFO:teuthology.orchestra.run.smithi067.stderr:no valid command found; 10 closest matches: 2024-09-20T22:21:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:0 2024-09-20T22:21:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:1 2024-09-20T22:21:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:2 2024-09-20T22:21:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:abort 2024-09-20T22:21:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:assert 2024-09-20T22:21:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:config diff 2024-09-20T22:21:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:config diff get 2024-09-20T22:21:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:config get 2024-09-20T22:21:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:config help [] 2024-09-20T22:21:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:config set ... 2024-09-20T22:21:38.556 INFO:teuthology.orchestra.run.smithi067.stderr:admin_socket: invalid command 2024-09-20T22:21:38.590 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:21:38.656 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:38.697 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:38.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:38.698 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:38.749 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_blocked_ops 2024-09-20T22:21:38.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:38 smithi067 ceph-mon[33511]: 4.9 scrub starts 2024-09-20T22:21:38.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:38 smithi067 ceph-mon[33511]: 4.9 scrub ok 2024-09-20T22:21:38.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:38 smithi067 ceph-mon[33511]: 3.f scrub starts 2024-09-20T22:21:38.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:38 smithi067 ceph-mon[33511]: 3.f scrub ok 2024-09-20T22:21:38.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:38 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/345719715' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:21:38.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:38 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:21:38.764 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:38 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[154790]: 2024-09-20T22:21:38.309+0000 7fd950307640 -1 Falling back to public interface 2024-09-20T22:21:38.798 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:38.836 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:38.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:38.837 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:38.938 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:38.983 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:38.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:38.984 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:39.085 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:39.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:39.120 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:39.120 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:39.221 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:39.258 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:39.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:39.260 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:39.360 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:39.396 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:39.397 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:39.397 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:39.498 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:39.531 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:39.532 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:39.532 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:39.633 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:39.655 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:39.655 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:39.656 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:39.756 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:39.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:39 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:21:39.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:39 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:21:39.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:39 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:21:39.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:39 smithi067 ceph-mon[33511]: osdmap e529: 8 total, 7 up, 6 in 2024-09-20T22:21:39.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:39 smithi067 ceph-mon[33511]: pgmap v1008: 178 pgs: 178 active+clean; 1.8 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 0 B/s wr, 1.05k op/s; 0 B/s, 0 objects/s recovering 2024-09-20T22:21:39.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:39 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:21:39.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:39 smithi067 ceph-mon[33511]: 5.19 scrub starts 2024-09-20T22:21:39.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:39 smithi067 ceph-mon[33511]: 5.19 scrub ok 2024-09-20T22:21:39.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:39 smithi067 ceph-mon[33511]: 3.18 deep-scrub starts 2024-09-20T22:21:39.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:39 smithi067 ceph-mon[33511]: 3.18 deep-scrub ok 2024-09-20T22:21:39.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:21:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:21:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:21:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[29694]: osdmap e529: 8 total, 7 up, 6 in 2024-09-20T22:21:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[29694]: pgmap v1008: 178 pgs: 178 active+clean; 1.8 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 0 B/s wr, 1.05k op/s; 0 B/s, 0 objects/s recovering 2024-09-20T22:21:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:21:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[29694]: 5.19 scrub starts 2024-09-20T22:21:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[29694]: 5.19 scrub ok 2024-09-20T22:21:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[29694]: 3.18 deep-scrub starts 2024-09-20T22:21:39.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[29694]: 3.18 deep-scrub ok 2024-09-20T22:21:39.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:21:39.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:21:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:21:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[40290]: osdmap e529: 8 total, 7 up, 6 in 2024-09-20T22:21:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[40290]: pgmap v1008: 178 pgs: 178 active+clean; 1.8 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 0 B/s wr, 1.05k op/s; 0 B/s, 0 objects/s recovering 2024-09-20T22:21:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:21:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[40290]: 5.19 scrub starts 2024-09-20T22:21:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[40290]: 5.19 scrub ok 2024-09-20T22:21:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[40290]: 3.18 deep-scrub starts 2024-09-20T22:21:39.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:39 smithi060 ceph-mon[40290]: 3.18 deep-scrub ok 2024-09-20T22:21:39.793 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:39.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:39.794 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:39.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:39.932 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:39.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:39.933 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:40.033 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:40.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:40.094 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:40.095 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:40.116 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:40.131 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:40.195 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:40.235 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:40.235 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:40.236 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:40.336 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:40.373 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:40.374 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:40.374 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:40.410 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-20T22:21:40.411 INFO:tasks.cephadm.ceph_manager.ceph:waiting on admin_socket for osd-4, ['dump_ops_in_flight'] 2024-09-20T22:21:40.474 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:40.513 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:40.514 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:40.514 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:40.518 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:40 smithi060 ceph-mon[29694]: 4.10 deep-scrub starts 2024-09-20T22:21:40.518 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:40 smithi060 ceph-mon[29694]: 4.10 deep-scrub ok 2024-09-20T22:21:40.518 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:40 smithi060 ceph-mon[29694]: 6.11 scrub starts 2024-09-20T22:21:40.518 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:40 smithi060 ceph-mon[29694]: 6.11 scrub ok 2024-09-20T22:21:40.518 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:40 smithi060 ceph-mon[40290]: 4.10 deep-scrub starts 2024-09-20T22:21:40.518 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:40 smithi060 ceph-mon[40290]: 4.10 deep-scrub ok 2024-09-20T22:21:40.519 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:40 smithi060 ceph-mon[40290]: 6.11 scrub starts 2024-09-20T22:21:40.519 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:40 smithi060 ceph-mon[40290]: 6.11 scrub ok 2024-09-20T22:21:40.615 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:40.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:40.659 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:40.660 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:40.760 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:40.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:40 smithi067 ceph-mon[33511]: 4.10 deep-scrub starts 2024-09-20T22:21:40.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:40 smithi067 ceph-mon[33511]: 4.10 deep-scrub ok 2024-09-20T22:21:40.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:40 smithi067 ceph-mon[33511]: 6.11 scrub starts 2024-09-20T22:21:40.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:40 smithi067 ceph-mon[33511]: 6.11 scrub ok 2024-09-20T22:21:40.798 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:40.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:40.799 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:40.900 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:40.940 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:40.941 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:40.941 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:40.957 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:21:40.958 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:21:40.990 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:21:40.990 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:21:41.027 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:21:41.027 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:21:41.042 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:41.068 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:21:41.069 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:21:41.071 INFO:teuthology.orchestra.run.smithi060.stderr:Error ENXIO: problem getting command descriptions from osd.4 2024-09-20T22:21:41.079 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:41.079 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:41.080 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:41.108 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:21:41.108 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:21:41.144 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:21:41.144 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:21:41.181 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:41.184 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:21:41.184 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:21:41.193 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:21:41.193 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:21:41.212 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:41.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:41.213 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:41.313 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:41.351 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:41.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:41.353 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:41.454 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:41.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:41.497 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:41.497 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:41.598 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:41.636 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:41.636 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:41.637 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:41.738 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:41.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:41 smithi067 ceph-mon[33511]: 1.7 scrub starts 2024-09-20T22:21:41.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:41 smithi067 ceph-mon[33511]: 1.7 scrub ok 2024-09-20T22:21:41.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:41 smithi067 ceph-mon[33511]: pgmap v1009: 178 pgs: 178 active+clean; 1.7 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 0 B/s wr, 1.01k op/s; 0 B/s, 0 objects/s recovering 2024-09-20T22:21:41.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:41 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:21:41.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:41 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:21:41.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:41 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:21:41.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:41 smithi067 ceph-mon[33511]: 2.2 scrub starts 2024-09-20T22:21:41.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:41 smithi067 ceph-mon[33511]: 2.2 scrub ok 2024-09-20T22:21:41.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:41 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2069194779' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:41.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:41 smithi067 ceph-mon[33511]: 6.16 scrub starts 2024-09-20T22:21:41.765 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:41 smithi067 ceph-mon[33511]: 6.16 scrub ok 2024-09-20T22:21:41.779 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:41.780 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:41.780 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[29694]: 1.7 scrub starts 2024-09-20T22:21:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[29694]: 1.7 scrub ok 2024-09-20T22:21:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[29694]: pgmap v1009: 178 pgs: 178 active+clean; 1.7 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 0 B/s wr, 1.01k op/s; 0 B/s, 0 objects/s recovering 2024-09-20T22:21:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:21:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:21:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:21:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[29694]: 2.2 scrub starts 2024-09-20T22:21:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[29694]: 2.2 scrub ok 2024-09-20T22:21:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2069194779' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[29694]: 6.16 scrub starts 2024-09-20T22:21:41.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[29694]: 6.16 scrub ok 2024-09-20T22:21:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[40290]: 1.7 scrub starts 2024-09-20T22:21:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[40290]: 1.7 scrub ok 2024-09-20T22:21:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[40290]: pgmap v1009: 178 pgs: 178 active+clean; 1.7 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 0 B/s wr, 1.01k op/s; 0 B/s, 0 objects/s recovering 2024-09-20T22:21:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:21:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:21:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:21:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[40290]: 2.2 scrub starts 2024-09-20T22:21:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[40290]: 2.2 scrub ok 2024-09-20T22:21:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2069194779' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[40290]: 6.16 scrub starts 2024-09-20T22:21:41.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:41 smithi060 ceph-mon[40290]: 6.16 scrub ok 2024-09-20T22:21:41.881 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:41.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:41.921 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:41.921 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:41.980 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set noscrub 2024-09-20T22:21:42.022 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:42.054 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:42.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:42.055 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:42.155 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:42.191 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:42.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:42.192 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:42.263 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:41 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[154790]: 2024-09-20T22:21:41.933+0000 7fd950307640 -1 osd.4 485 log_to_monitors true 2024-09-20T22:21:42.292 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:42.350 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:42.350 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:42.351 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:42.381 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.0 dump_historic_ops 2024-09-20T22:21:42.452 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:42.487 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:42.487 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:42.487 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:42.588 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:42.626 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:42.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:42.627 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:42.728 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:42.769 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:42.770 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:42.770 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:42.871 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:42.908 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:42.908 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:42.909 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:42.963 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:42 smithi060 ceph-mon[29694]: 5.1a scrub starts 2024-09-20T22:21:42.963 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:42 smithi060 ceph-mon[29694]: 5.1a scrub ok 2024-09-20T22:21:42.964 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:42 smithi060 ceph-mon[29694]: osdmap e530: 8 total, 7 up, 6 in 2024-09-20T22:21:42.964 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:42 smithi060 ceph-mon[29694]: 1.6 scrub starts 2024-09-20T22:21:42.964 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:42 smithi060 ceph-mon[29694]: 1.6 scrub ok 2024-09-20T22:21:42.964 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:42 smithi060 ceph-mon[29694]: from='osd.4 [v2:172.21.15.67:6800/3152108694,v1:172.21.15.67:6801/3152108694]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-20T22:21:42.964 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:42 smithi060 ceph-mon[29694]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-20T22:21:42.964 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:42 smithi060 ceph-mon[29694]: pgmap v1011: 178 pgs: 178 active+clean; 1.3 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 0 B/s wr, 1.64k op/s; 0 B/s, 0 objects/s recovering 2024-09-20T22:21:42.964 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:42 smithi060 ceph-mon[40290]: 5.1a scrub starts 2024-09-20T22:21:42.964 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:42 smithi060 ceph-mon[40290]: 5.1a scrub ok 2024-09-20T22:21:42.964 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:42 smithi060 ceph-mon[40290]: osdmap e530: 8 total, 7 up, 6 in 2024-09-20T22:21:42.964 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:42 smithi060 ceph-mon[40290]: 1.6 scrub starts 2024-09-20T22:21:42.964 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:42 smithi060 ceph-mon[40290]: 1.6 scrub ok 2024-09-20T22:21:42.965 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:42 smithi060 ceph-mon[40290]: from='osd.4 [v2:172.21.15.67:6800/3152108694,v1:172.21.15.67:6801/3152108694]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-20T22:21:42.965 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:42 smithi060 ceph-mon[40290]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-20T22:21:42.965 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:42 smithi060 ceph-mon[40290]: pgmap v1011: 178 pgs: 178 active+clean; 1.3 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 0 B/s wr, 1.64k op/s; 0 B/s, 0 objects/s recovering 2024-09-20T22:21:42.965 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-09-20T22:21:42.966 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-09-20T22:21:42.966 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:43.009 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:43.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:21:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:21:43.013 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:21:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[154790]: 2024-09-20T22:21:42.757+0000 7fd9440aa640 -1 osd.4 531 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-20T22:21:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:42 smithi067 ceph-mon[33511]: 5.1a scrub starts 2024-09-20T22:21:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:42 smithi067 ceph-mon[33511]: 5.1a scrub ok 2024-09-20T22:21:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:42 smithi067 ceph-mon[33511]: osdmap e530: 8 total, 7 up, 6 in 2024-09-20T22:21:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:42 smithi067 ceph-mon[33511]: 1.6 scrub starts 2024-09-20T22:21:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:42 smithi067 ceph-mon[33511]: 1.6 scrub ok 2024-09-20T22:21:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:42 smithi067 ceph-mon[33511]: from='osd.4 [v2:172.21.15.67:6800/3152108694,v1:172.21.15.67:6801/3152108694]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-20T22:21:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:42 smithi067 ceph-mon[33511]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-20T22:21:43.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:42 smithi067 ceph-mon[33511]: pgmap v1011: 178 pgs: 178 active+clean; 1.3 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 0 B/s wr, 1.64k op/s; 0 B/s, 0 objects/s recovering 2024-09-20T22:21:43.066 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:43.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:43.067 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:43.168 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:43.203 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:43.204 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:43.204 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:43.305 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:43.344 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:43.345 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:43.345 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:43.446 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:43.480 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:43.481 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:43.481 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:43.582 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:43.606 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:43.612 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:43.613 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:43.613 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:43.714 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:43.758 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:43.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:43.759 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:43.860 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:43.904 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:43.905 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:43.905 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:43.918 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[29694]: 6.1 scrub starts 2024-09-20T22:21:43.918 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[29694]: 3.0 scrub starts 2024-09-20T22:21:43.918 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[29694]: 6.1 scrub ok 2024-09-20T22:21:43.918 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[29694]: 3.0 scrub ok 2024-09-20T22:21:43.918 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[29694]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-20T22:21:43.918 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[29694]: osdmap e531: 8 total, 7 up, 6 in 2024-09-20T22:21:43.918 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[29694]: from='osd.4 [v2:172.21.15.67:6800/3152108694,v1:172.21.15.67:6801/3152108694]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:21:43.918 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[29694]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:21:43.918 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:21:43.918 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:21:43.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[40290]: 6.1 scrub starts 2024-09-20T22:21:43.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[40290]: 3.0 scrub starts 2024-09-20T22:21:43.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[40290]: 6.1 scrub ok 2024-09-20T22:21:43.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[40290]: 3.0 scrub ok 2024-09-20T22:21:43.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[40290]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-20T22:21:43.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[40290]: osdmap e531: 8 total, 7 up, 6 in 2024-09-20T22:21:43.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[40290]: from='osd.4 [v2:172.21.15.67:6800/3152108694,v1:172.21.15.67:6801/3152108694]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:21:43.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[40290]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:21:43.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:21:43.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:43 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:21:44.006 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:44.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:43 smithi067 ceph-mon[33511]: 6.1 scrub starts 2024-09-20T22:21:44.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:43 smithi067 ceph-mon[33511]: 3.0 scrub starts 2024-09-20T22:21:44.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:43 smithi067 ceph-mon[33511]: 6.1 scrub ok 2024-09-20T22:21:44.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:43 smithi067 ceph-mon[33511]: 3.0 scrub ok 2024-09-20T22:21:44.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:43 smithi067 ceph-mon[33511]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-20T22:21:44.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:43 smithi067 ceph-mon[33511]: osdmap e531: 8 total, 7 up, 6 in 2024-09-20T22:21:44.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:43 smithi067 ceph-mon[33511]: from='osd.4 [v2:172.21.15.67:6800/3152108694,v1:172.21.15.67:6801/3152108694]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:21:44.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:43 smithi067 ceph-mon[33511]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi067", "root=default"]}]: dispatch 2024-09-20T22:21:44.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:43 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:21:44.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:43 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:21:44.048 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:44.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:44.049 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:44.150 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:44.181 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:44.182 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:44.182 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:44.248 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:21:44.248 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:21:44.283 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:44.322 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:44.323 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:44.323 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:44.424 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:44.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:44.459 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:44.459 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:44.560 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:44.592 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:44.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:44.594 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:44.694 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:44.726 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:44.727 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:44.727 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:44.756 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is set 2024-09-20T22:21:44.828 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:44.855 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:44.855 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:44.856 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:44.956 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:44.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-09-20T22:21:44.990 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:44.992 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:44.992 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:44 smithi067 ceph-mon[33511]: 3.14 scrub starts 2024-09-20T22:21:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:44 smithi067 ceph-mon[33511]: 3.14 scrub ok 2024-09-20T22:21:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:44 smithi067 ceph-mon[33511]: 1.8 scrub starts 2024-09-20T22:21:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:44 smithi067 ceph-mon[33511]: 1.8 scrub ok 2024-09-20T22:21:45.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:44 smithi067 ceph-mon[33511]: 4.4 scrub starts 2024-09-20T22:21:45.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:44 smithi067 ceph-mon[33511]: osd.4 [v2:172.21.15.67:6800/3152108694,v1:172.21.15.67:6801/3152108694] boot 2024-09-20T22:21:45.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:44 smithi067 ceph-mon[33511]: osdmap e532: 8 total, 8 up, 6 in 2024-09-20T22:21:45.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:44 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:21:45.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:44 smithi067 ceph-mon[33511]: 4.4 scrub ok 2024-09-20T22:21:45.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:44 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/254087415' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:21:45.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:44 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:21:45.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:44 smithi067 ceph-mon[33511]: 11.b scrub starts 2024-09-20T22:21:45.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:44 smithi067 ceph-mon[33511]: pgmap v1014: 178 pgs: 178 active+clean; 1.2 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 0 B/s wr, 1.55k op/s 2024-09-20T22:21:45.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:44 smithi067 ceph-mon[33511]: 11.b scrub ok 2024-09-20T22:21:45.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[29694]: 3.14 scrub starts 2024-09-20T22:21:45.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[29694]: 3.14 scrub ok 2024-09-20T22:21:45.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[29694]: 1.8 scrub starts 2024-09-20T22:21:45.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[29694]: 1.8 scrub ok 2024-09-20T22:21:45.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[29694]: 4.4 scrub starts 2024-09-20T22:21:45.029 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[29694]: osd.4 [v2:172.21.15.67:6800/3152108694,v1:172.21.15.67:6801/3152108694] boot 2024-09-20T22:21:45.030 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[29694]: osdmap e532: 8 total, 8 up, 6 in 2024-09-20T22:21:45.030 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:21:45.030 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[29694]: 4.4 scrub ok 2024-09-20T22:21:45.030 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/254087415' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:21:45.030 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:21:45.030 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[29694]: 11.b scrub starts 2024-09-20T22:21:45.030 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[29694]: pgmap v1014: 178 pgs: 178 active+clean; 1.2 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 0 B/s wr, 1.55k op/s 2024-09-20T22:21:45.030 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[29694]: 11.b scrub ok 2024-09-20T22:21:45.030 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[40290]: 3.14 scrub starts 2024-09-20T22:21:45.030 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[40290]: 3.14 scrub ok 2024-09-20T22:21:45.030 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[40290]: 1.8 scrub starts 2024-09-20T22:21:45.031 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[40290]: 1.8 scrub ok 2024-09-20T22:21:45.031 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[40290]: 4.4 scrub starts 2024-09-20T22:21:45.031 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[40290]: osd.4 [v2:172.21.15.67:6800/3152108694,v1:172.21.15.67:6801/3152108694] boot 2024-09-20T22:21:45.031 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[40290]: osdmap e532: 8 total, 8 up, 6 in 2024-09-20T22:21:45.031 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-20T22:21:45.031 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[40290]: 4.4 scrub ok 2024-09-20T22:21:45.031 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/254087415' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:21:45.031 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "noscrub"}]: dispatch 2024-09-20T22:21:45.031 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[40290]: 11.b scrub starts 2024-09-20T22:21:45.031 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[40290]: pgmap v1014: 178 pgs: 178 active+clean; 1.2 GiB data, 8.8 GiB used, 528 GiB / 536 GiB avail; 0 B/s wr, 1.55k op/s 2024-09-20T22:21:45.031 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:44 smithi060 ceph-mon[40290]: 11.b scrub ok 2024-09-20T22:21:45.094 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:45.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:45.120 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:45.121 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:45.221 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:45.261 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:45.261 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:45.262 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:45.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:21:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:21:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:21:45.362 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:45.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:45.393 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:45.393 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:45.411 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T22:21:45.494 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:45.524 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:45.525 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:45.525 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:45.626 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:45.661 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:45.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:45.662 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:45.763 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:45.793 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:45.794 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:45.794 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:45.850 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:21:45.895 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:45.941 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:45.942 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:45.942 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:45 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:21:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:45 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:21:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:45 smithi067 ceph-mon[33511]: 5.10 scrub starts 2024-09-20T22:21:46.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:45 smithi067 ceph-mon[33511]: 5.10 scrub ok 2024-09-20T22:21:46.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:45 smithi067 ceph-mon[33511]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:21:46.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:45 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:21:46.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:45 smithi067 ceph-mon[33511]: osdmap e533: 8 total, 8 up, 6 in 2024-09-20T22:21:46.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:45 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:21:46.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:45 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:21:46.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:45 smithi060 ceph-mon[29694]: 5.10 scrub starts 2024-09-20T22:21:46.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:45 smithi060 ceph-mon[29694]: 5.10 scrub ok 2024-09-20T22:21:46.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:45 smithi060 ceph-mon[29694]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:21:46.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:45 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:21:46.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:45 smithi060 ceph-mon[29694]: osdmap e533: 8 total, 8 up, 6 in 2024-09-20T22:21:46.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:45 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:21:46.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:45 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:21:46.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:45 smithi060 ceph-mon[40290]: 5.10 scrub starts 2024-09-20T22:21:46.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:45 smithi060 ceph-mon[40290]: 5.10 scrub ok 2024-09-20T22:21:46.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:45 smithi060 ceph-mon[40290]: Health check failed: noscrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:21:46.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:45 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "noscrub"}]': finished 2024-09-20T22:21:46.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:45 smithi060 ceph-mon[40290]: osdmap e533: 8 total, 8 up, 6 in 2024-09-20T22:21:46.043 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:46.065 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:46.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:46.065 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:46.166 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:46.200 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:46.201 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:46.201 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:46.302 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:46.327 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:46.328 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:46.328 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:46.429 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:46.471 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:46.472 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:46.472 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:46.536 INFO:teuthology.orchestra.run.smithi067.stdout:{ 2024-09-20T22:21:46.536 INFO:teuthology.orchestra.run.smithi067.stdout: "ops": [], 2024-09-20T22:21:46.536 INFO:teuthology.orchestra.run.smithi067.stdout: "num_ops": 0 2024-09-20T22:21:46.537 INFO:teuthology.orchestra.run.smithi067.stdout:} 2024-09-20T22:21:46.573 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:46.595 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:46.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:46.596 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:46.697 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:46.732 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:46.732 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:46.733 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:46.833 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:46.860 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:46.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:46.862 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:46.962 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:46.999 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:46.999 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:47.000 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:46 smithi067 ceph-mon[33511]: pgmap v1016: 178 pgs: 178 active+clean; 1.1 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.46k op/s 2024-09-20T22:21:47.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:46 smithi067 ceph-mon[33511]: osdmap e534: 8 total, 8 up, 6 in 2024-09-20T22:21:47.035 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:46 smithi060 ceph-mon[29694]: pgmap v1016: 178 pgs: 178 active+clean; 1.1 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.46k op/s 2024-09-20T22:21:47.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:46 smithi060 ceph-mon[29694]: osdmap e534: 8 total, 8 up, 6 in 2024-09-20T22:21:47.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:46 smithi060 ceph-mon[40290]: pgmap v1016: 178 pgs: 178 active+clean; 1.1 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1.46k op/s 2024-09-20T22:21:47.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:46 smithi060 ceph-mon[40290]: osdmap e534: 8 total, 8 up, 6 in 2024-09-20T22:21:47.100 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:47.125 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:47.125 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:47.126 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:47.226 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:47.264 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:47.265 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:47.266 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:47.366 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:47.399 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:47.400 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:47.400 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:47.501 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:47.535 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:47.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:47.536 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:47.637 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:47.662 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:47.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:47.663 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:47.763 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:47.785 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:47.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:47.786 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:47.887 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:47.923 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:47.923 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:47.923 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:48.024 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:48.062 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:48.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:48.063 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:48.164 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:48.197 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:48.198 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:48.198 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:48.298 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:48.330 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:48.330 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:48.330 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:48.431 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:48.467 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:48.467 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:48.468 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:48.568 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:48.604 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:48.605 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:48.605 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:48.706 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:48.741 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:48.741 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:48.741 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:48.842 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:48.878 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:48.879 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:48.879 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:48.980 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:49.002 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:49.003 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:49.003 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:49.104 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:49.126 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:49.126 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:49.127 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:49.228 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:49.333 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:49.334 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:49.334 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:49.435 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:49.473 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:49.473 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:49.474 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:49.574 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:49.609 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:49.610 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:49.610 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:49.711 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:49.747 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:49.747 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:49.747 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:49.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:49 smithi067 ceph-mon[33511]: pgmap v1018: 178 pgs: 178 active+clean; 904 MiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 902 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-20T22:21:49.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:49 smithi060 ceph-mon[29694]: pgmap v1018: 178 pgs: 178 active+clean; 904 MiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 902 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-20T22:21:49.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:49 smithi060 ceph-mon[40290]: pgmap v1018: 178 pgs: 178 active+clean; 904 MiB data, 7.8 GiB used, 529 GiB / 536 GiB avail; 902 B/s rd, 0 B/s wr, 1.26k op/s 2024-09-20T22:21:49.848 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:49.880 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:49.880 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:49.881 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:49.982 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:49.983 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_ops_in_flight 2024-09-20T22:21:50.017 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:50.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:50.018 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:50.118 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:50.140 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:50.141 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:50.141 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:50.242 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:50.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:50.285 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:50.285 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:50.386 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:50.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:50.409 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:50.409 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:50.509 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:50.537 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:50.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:50.538 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:50.587 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:50.624 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:50.640 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:50.676 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:50.677 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:50.677 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:50.777 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:50.813 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:50.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:50.814 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:50.915 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:50.950 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:50.950 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:50.951 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:50.952 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 2 1 2024-09-20T22:21:51.052 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:21:51.090 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:51.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:51.092 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:21:51.192 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:51.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:51.225 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:51.225 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:51.326 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:51.372 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:51.373 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:51.373 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:51.474 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:21:51.509 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:51.510 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:51.510 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:21:51.548 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:51 smithi060 ceph-mon[29694]: pgmap v1019: 178 pgs: 178 active+clean; 797 MiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 767 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-20T22:21:51.549 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:51 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:21:51.549 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:51 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/399017732' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:51.549 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:51 smithi060 ceph-mon[40290]: pgmap v1019: 178 pgs: 178 active+clean; 797 MiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 767 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-20T22:21:51.549 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:51 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:21:51.549 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:51 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/399017732' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:51.611 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:51.647 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:51.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:51.648 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:51.748 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:51.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:51 smithi067 ceph-mon[33511]: pgmap v1019: 178 pgs: 178 active+clean; 797 MiB data, 7.5 GiB used, 529 GiB / 536 GiB avail; 767 B/s rd, 0 B/s wr, 1.05k op/s 2024-09-20T22:21:51.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:51 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:21:51.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:51 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/399017732' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:51.784 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:51.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:51.785 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:51.886 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:51.908 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:51.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:51.909 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:52.010 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:52.032 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:52.033 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:52.033 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:52.130 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:21:52.130 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:21:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:21:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:21:52.130 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:21:52.131 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:21:52.131 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:21:52.131 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:21:52.131 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:21:52.131 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:21:52.131 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-09-20T22:21:52.131 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:21:52.132 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:21:52.132 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:21:52.132 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:21:52.132 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:21:52.134 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:52.161 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:52.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:52.163 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:52.263 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:52.295 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:52.295 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:52.296 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:52.396 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:52.434 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:52.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:52.435 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:52.535 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:52.553 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd set nodeep-scrub 2024-09-20T22:21:52.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:52.568 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:52.569 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:52.670 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:52.705 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:52.705 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:52.705 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:52.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:52 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:21:52.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:52 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:21:52.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:52 smithi060 ceph-mon[29694]: osdmap e535: 8 total, 8 up, 6 in 2024-09-20T22:21:52.730 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:52 smithi060 ceph-mon[29694]: osdmap e536: 8 total, 8 up, 6 in 2024-09-20T22:21:52.730 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:52 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:21:52.730 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:52 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:21:52.730 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:52 smithi060 ceph-mon[40290]: osdmap e535: 8 total, 8 up, 6 in 2024-09-20T22:21:52.731 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:52 smithi060 ceph-mon[40290]: osdmap e536: 8 total, 8 up, 6 in 2024-09-20T22:21:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:52 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:21:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:52 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:21:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:52 smithi067 ceph-mon[33511]: osdmap e535: 8 total, 8 up, 6 in 2024-09-20T22:21:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:52 smithi067 ceph-mon[33511]: osdmap e536: 8 total, 8 up, 6 in 2024-09-20T22:21:52.807 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:52.847 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:52.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:52.848 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:52.949 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:52.986 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:52.987 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:52.987 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:53.088 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:53.134 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:53.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:53.135 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:53.236 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:53.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:21:52 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:21:53.277 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:53.278 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:53.278 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:53.379 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:53.422 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:53.423 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:53.423 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:53.446 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:21:53.446 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [ 2024-09-20T22:21:53.446 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:53.446 INFO:teuthology.orchestra.run.smithi060.stdout: "description": "osd_op(client.16440.0:200132 11.c 11:3eace709:::benchmark_data_smithi060_196073_object98145:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e536)", 2024-09-20T22:21:53.446 INFO:teuthology.orchestra.run.smithi060.stdout: "initiated_at": "2024-09-20T22:21:53.440386+0000", 2024-09-20T22:21:53.446 INFO:teuthology.orchestra.run.smithi060.stdout: "age": 0.001189591, 2024-09-20T22:21:53.446 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.0012203699999999999, 2024-09-20T22:21:53.446 INFO:teuthology.orchestra.run.smithi060.stdout: "continuous": false, 2024-09-20T22:21:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: "type_data": { 2024-09-20T22:21:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: "flag_point": "waiting for sub ops", 2024-09-20T22:21:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: "client_info": { 2024-09-20T22:21:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: "client": "client.16440", 2024-09-20T22:21:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: "client_addr": "172.21.15.60:0/1768496104", 2024-09-20T22:21:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: "tid": 200132 2024-09-20T22:21:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: "events": [ 2024-09-20T22:21:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "initiated", 2024-09-20T22:21:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:53.440386+0000", 2024-09-20T22:21:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "header_read", 2024-09-20T22:21:53.447 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:53.440386+0000", 2024-09-20T22:21:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0 2024-09-20T22:21:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "throttled", 2024-09-20T22:21:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:53.440388+0000", 2024-09-20T22:21:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.42e-06 2024-09-20T22:21:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "all_read", 2024-09-20T22:21:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:53.440393+0000", 2024-09-20T22:21:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 5.6289999999999998e-06 2024-09-20T22:21:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "dispatched", 2024-09-20T22:21:53.448 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:53.440410+0000", 2024-09-20T22:21:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 1.6971999999999999e-05 2024-09-20T22:21:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "queued_for_pg", 2024-09-20T22:21:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:53.440419+0000", 2024-09-20T22:21:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.9830000000000002e-06 2024-09-20T22:21:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "reached_pg", 2024-09-20T22:21:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:53.440506+0000", 2024-09-20T22:21:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 8.7199000000000003e-05 2024-09-20T22:21:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "started", 2024-09-20T22:21:53.449 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:53.440637+0000", 2024-09-20T22:21:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013066 2024-09-20T22:21:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: }, 2024-09-20T22:21:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: { 2024-09-20T22:21:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: "event": "waiting for subops from 5,7", 2024-09-20T22:21:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: "time": "2024-09-20T22:21:53.440770+0000", 2024-09-20T22:21:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: "duration": 0.00013284200000000001 2024-09-20T22:21:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: ] 2024-09-20T22:21:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: } 2024-09-20T22:21:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: ], 2024-09-20T22:21:53.450 INFO:teuthology.orchestra.run.smithi060.stdout: "num_ops": 1 2024-09-20T22:21:53.450 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:21:53.461 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:53.524 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:21:53.563 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:53.563 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:53.564 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:21:53.664 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:53.708 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:53.709 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:53.709 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:53 smithi067 ceph-mon[33511]: pgmap v1022: 178 pgs: 178 active+clean; 494 MiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.60k op/s 2024-09-20T22:21:53.776 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:53 smithi060 ceph-mon[29694]: pgmap v1022: 178 pgs: 178 active+clean; 494 MiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.60k op/s 2024-09-20T22:21:53.776 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:53 smithi060 ceph-mon[40290]: pgmap v1022: 178 pgs: 178 active+clean; 494 MiB data, 6.8 GiB used, 530 GiB / 536 GiB avail; 852 B/s rd, 0 B/s wr, 1.60k op/s 2024-09-20T22:21:53.810 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:53.847 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:53.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:53.848 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:53.948 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:53.984 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:53.984 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:53.985 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:54.085 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:54.122 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:54.122 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:54.122 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:54.207 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_blocked_ops 2024-09-20T22:21:54.223 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:54.232 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:54.282 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:54.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:54.284 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:54.384 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:54.429 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:54.431 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:54.431 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:54.532 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:54.566 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:54.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:54.567 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:54.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:54.704 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:54.704 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:54.704 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:54.746 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:54 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:21:54.747 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:54 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:21:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:54 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:21:54.805 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:54.843 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:54.844 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:54.844 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:54.945 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:54.986 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:54.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:54.987 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:55.087 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:55.114 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-09-20T22:21:55.121 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:55.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:55.122 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:55.222 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:55.249 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:55.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:55.250 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:55.316 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:55.350 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:55.407 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:55.408 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:55.408 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:55.467 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:21:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:21:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:21:55.506 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is set 2024-09-20T22:21:55.509 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:55.550 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:55.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:55.551 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:55.652 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:55.684 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:55.685 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:55.685 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:55.754 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:55 smithi060 ceph-mon[29694]: pgmap v1023: 178 pgs: 178 active+clean; 359 MiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 663 B/s rd, 0 B/s wr, 1.47k op/s 2024-09-20T22:21:55.754 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:55 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2506541780' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:21:55.754 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:55 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:21:55.755 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:55 smithi060 ceph-mon[40290]: pgmap v1023: 178 pgs: 178 active+clean; 359 MiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 663 B/s rd, 0 B/s wr, 1.47k op/s 2024-09-20T22:21:55.755 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:55 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2506541780' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:21:55.755 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:55 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:21:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:55 smithi067 ceph-mon[33511]: pgmap v1023: 178 pgs: 178 active+clean; 359 MiB data, 6.3 GiB used, 530 GiB / 536 GiB avail; 663 B/s rd, 0 B/s wr, 1.47k op/s 2024-09-20T22:21:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2506541780' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:21:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:55 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd set", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:21:55.786 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:21:55.823 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:55.824 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:55.824 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:21:55.886 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Removed 101984 objects 2024-09-20T22:21:55.887 INFO:tasks.radosbench.radosbench.0.smithi060.stdout:Clean up completed and total clean up time :92.7542 2024-09-20T22:21:55.925 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:21:55.946 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:55.947 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:55.947 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:21:56.048 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:56.071 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:56.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:56.071 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:56.073 INFO:tasks.cephadm.ceph_manager.ceph:removing pool_name unique_pool_4 2024-09-20T22:21:56.073 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd pool rm unique_pool_4 unique_pool_4 --yes-i-really-really-mean-it 2024-09-20T22:21:56.172 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:21:56.195 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:56.195 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:56.196 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:21:56.296 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:56.301 INFO:teuthology.orchestra.run.smithi060.stdout:{ 2024-09-20T22:21:56.301 INFO:teuthology.orchestra.run.smithi060.stdout: "ops": [], 2024-09-20T22:21:56.302 INFO:teuthology.orchestra.run.smithi060.stdout: "complaint_time": 30, 2024-09-20T22:21:56.302 INFO:teuthology.orchestra.run.smithi060.stdout: "num_blocked_ops": 0 2024-09-20T22:21:56.302 INFO:teuthology.orchestra.run.smithi060.stdout:} 2024-09-20T22:21:56.319 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:56.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:56.320 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:56.422 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:21:56.444 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:56.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:56.445 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:21:56.516 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.2 to 1 (10000) 2024-09-20T22:21:56.546 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:56.569 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:56.570 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:56.570 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:56.671 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:56.695 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:56.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:56.696 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:56 smithi067 ceph-mon[33511]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:21:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:56 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:21:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:56 smithi067 ceph-mon[33511]: osdmap e537: 8 total, 8 up, 6 in 2024-09-20T22:21:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:56 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/120393834' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:21:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:56 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:21:56.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:56 smithi060 ceph-mon[29694]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:21:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:56 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:21:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:56 smithi060 ceph-mon[29694]: osdmap e537: 8 total, 8 up, 6 in 2024-09-20T22:21:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:56 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/120393834' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:21:56.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:56 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:21:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:56 smithi060 ceph-mon[40290]: Health check update: noscrub,nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:21:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:56 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd set", "key": "nodeep-scrub"}]': finished 2024-09-20T22:21:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:56 smithi060 ceph-mon[40290]: osdmap e537: 8 total, 8 up, 6 in 2024-09-20T22:21:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:56 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/120393834' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:21:56.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:56 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:21:56.797 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:56.819 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:56.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:56.820 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:56.921 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:56.943 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:56.943 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:56.944 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:57.044 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:57.067 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:57.067 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:57.068 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:57.168 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:57.190 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:57.191 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:57.191 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:57.292 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:57.317 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:57.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:57.318 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:57.419 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:57.441 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:57.442 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:57.443 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:57.543 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:57.566 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:57.566 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:57.567 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:57.668 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:57.695 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:57.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:57.696 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:57 smithi067 ceph-mon[33511]: pgmap v1025: 178 pgs: 178 active+clean; 223 MiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.53k op/s 2024-09-20T22:21:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:57 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:21:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:57 smithi067 ceph-mon[33511]: osdmap e538: 8 total, 8 up, 6 in 2024-09-20T22:21:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:57 smithi060 ceph-mon[29694]: pgmap v1025: 178 pgs: 178 active+clean; 223 MiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.53k op/s 2024-09-20T22:21:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:57 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:21:57.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:57 smithi060 ceph-mon[29694]: osdmap e538: 8 total, 8 up, 6 in 2024-09-20T22:21:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:57 smithi060 ceph-mon[40290]: pgmap v1025: 178 pgs: 178 active+clean; 223 MiB data, 6.1 GiB used, 530 GiB / 536 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 1.53k op/s 2024-09-20T22:21:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:57 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:21:57.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:57 smithi060 ceph-mon[40290]: osdmap e538: 8 total, 8 up, 6 in 2024-09-20T22:21:57.796 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:57.819 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:57.820 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:57.821 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:57.917 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.2 dump_historic_ops 2024-09-20T22:21:57.921 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:57.948 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:57.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:57.949 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:58.050 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:21:58.076 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:58.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:58.077 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:21:58.178 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:58.203 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:58.203 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:58.204 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:58.304 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:58.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:58.328 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:58.328 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:58.400 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:58.429 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:58.432 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 3 1 2024-09-20T22:21:58.441 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:58.455 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:58.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:58.456 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:58.557 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:58.584 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:58.584 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:58.585 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:58.685 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:58.711 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:58.712 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:58.712 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:58.813 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:58.835 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:58.836 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:58.836 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:21:58.937 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:21:58.964 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:58.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:58.965 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:21:59.037 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:21:59.066 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:59.101 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:59.102 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:59.102 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:59.203 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:21:59.226 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:59.227 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:59.227 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:21:59.328 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:21:59.350 INFO:teuthology.orchestra.run.smithi060.stdout:osd.0: {} 2024-09-20T22:21:59.350 INFO:teuthology.orchestra.run.smithi060.stderr:osd.0: osd_enable_op_tracker = '' 2024-09-20T22:21:59.354 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:59.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:59.355 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:21:59.357 INFO:teuthology.orchestra.run.smithi060.stdout:osd.1: {} 2024-09-20T22:21:59.358 INFO:teuthology.orchestra.run.smithi060.stderr:osd.1: osd_enable_op_tracker = '' 2024-09-20T22:21:59.378 INFO:teuthology.orchestra.run.smithi060.stdout:osd.2: {} 2024-09-20T22:21:59.378 INFO:teuthology.orchestra.run.smithi060.stderr:osd.2: osd_enable_op_tracker = '' 2024-09-20T22:21:59.403 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:59.403 INFO:teuthology.orchestra.run.smithi060.stdout:osd.3: {} 2024-09-20T22:21:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:osd.3: osd_enable_op_tracker = '' 2024-09-20T22:21:59.404 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:59.424 INFO:teuthology.orchestra.run.smithi060.stdout:osd.4: {} 2024-09-20T22:21:59.424 INFO:teuthology.orchestra.run.smithi060.stderr:osd.4: osd_enable_op_tracker = '' 2024-09-20T22:21:59.425 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:21:59.448 INFO:teuthology.orchestra.run.smithi060.stdout:osd.5: {} 2024-09-20T22:21:59.448 INFO:teuthology.orchestra.run.smithi060.stderr:osd.5: osd_enable_op_tracker = '' 2024-09-20T22:21:59.456 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:21:59.465 INFO:teuthology.orchestra.run.smithi060.stdout:osd.6: {} 2024-09-20T22:21:59.466 INFO:teuthology.orchestra.run.smithi060.stderr:osd.6: osd_enable_op_tracker = '' 2024-09-20T22:21:59.478 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:59.478 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:59.478 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:21:59.483 INFO:teuthology.orchestra.run.smithi060.stdout:osd.7: {} 2024-09-20T22:21:59.483 INFO:teuthology.orchestra.run.smithi060.stderr:osd.7: osd_enable_op_tracker = '' 2024-09-20T22:21:59.516 INFO:teuthology.orchestra.run.smithi060.stderr:pool 'unique_pool_4' removed 2024-09-20T22:21:59.579 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:21:59.603 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:21:59.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:59.604 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:21:59.705 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:59.727 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:59.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:59.728 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:59.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:59 smithi060 ceph-mon[29694]: pgmap v1027: 178 pgs: 178 active+clean; 59 MiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 763 B/s rd, 0 B/s wr, 1.37k op/s 2024-09-20T22:21:59.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:59 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3046984846' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:21:59.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:21:59 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2305169437' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:59.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:59 smithi060 ceph-mon[40290]: pgmap v1027: 178 pgs: 178 active+clean; 59 MiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 763 B/s rd, 0 B/s wr, 1.37k op/s 2024-09-20T22:21:59.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:59 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3046984846' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:21:59.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:21:59 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2305169437' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:21:59.829 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:21:59.851 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:59.852 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:59.852 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:21:59.953 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:21:59.975 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:21:59.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:21:59.976 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:22:00.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:59 smithi067 ceph-mon[33511]: pgmap v1027: 178 pgs: 178 active+clean; 59 MiB data, 5.3 GiB used, 531 GiB / 536 GiB avail; 763 B/s rd, 0 B/s wr, 1.37k op/s 2024-09-20T22:22:00.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:59 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3046984846' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-20T22:22:00.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:21:59 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2305169437' entity='client.admin' cmd=[{"prefix": "osd ls"}]: dispatch 2024-09-20T22:22:00.077 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:22:00.099 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:22:00.099 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:00.100 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:22:00.200 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:22:00.224 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:22:00.224 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:00.224 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:22:00.325 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:22:00.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:00.349 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:22:00.349 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:22:00.450 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:22:00.473 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:22:00.474 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:00.474 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:22:00.575 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:22:00.597 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:22:00.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:00.598 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:22:00.699 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:22:00.721 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:22:00.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:00.722 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:22:00.785 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:00 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:22:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:00 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:22:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:00 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3046984846' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T22:22:00.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:00 smithi060 ceph-mon[29694]: osdmap e539: 8 total, 8 up, 6 in 2024-09-20T22:22:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:00 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:22:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:00 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:22:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:00 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3046984846' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T22:22:00.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:00 smithi060 ceph-mon[40290]: osdmap e539: 8 total, 8 up, 6 in 2024-09-20T22:22:00.823 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:22:00.845 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:22:00.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:00.846 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:22:00.946 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:22:00.968 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:22:00.969 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:00.969 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:22:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:00 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:22:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:00 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:22:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:00 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3046984846' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "unique_pool_4", "pool2": "unique_pool_4", "yes_i_really_really_mean_it": true}]': finished 2024-09-20T22:22:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:00 smithi067 ceph-mon[33511]: osdmap e539: 8 total, 8 up, 6 in 2024-09-20T22:22:01.070 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3 2024-09-20T22:22:01.092 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:01.093 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:22:01.093 INFO:tasks.cephadm.osd.3:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.3' 2024-09-20T22:22:01.194 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:22:01.216 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:22:01.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:01.217 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:22:01.318 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1 2024-09-20T22:22:01.340 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:22:01.341 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:01.341 INFO:tasks.cephadm.osd.1:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.1' 2024-09-20T22:22:01.442 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2 2024-09-20T22:22:01.465 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:22:01.466 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:01.466 INFO:tasks.cephadm.osd.2:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.2' 2024-09-20T22:22:01.529 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.3 to 1 (10000) 2024-09-20T22:22:01.566 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7 2024-09-20T22:22:01.593 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:22:01.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:01.594 INFO:tasks.cephadm.osd.7:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.7' 2024-09-20T22:22:01.694 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5 2024-09-20T22:22:01.722 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:22:01.722 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:01.723 INFO:tasks.cephadm.osd.5:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.5' 2024-09-20T22:22:01.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:01 smithi060 ceph-mon[29694]: pgmap v1029: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T22:22:01.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:01 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1471011102' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:22:01.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:01 smithi060 ceph-mon[40290]: pgmap v1029: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T22:22:01.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:01 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1471011102' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:22:01.824 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:22:01.849 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:22:01.850 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:01.850 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:22:01.951 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:22:01.973 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:22:01.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:01.974 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:22:02.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:01 smithi067 ceph-mon[33511]: pgmap v1029: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T22:22:02.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:01 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1471011102' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:22:02.076 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:22:02.102 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:22:02.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:02.103 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:22:02.204 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:22:02.231 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:22:02.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:02.232 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:22:02.333 DEBUG:teuthology.orchestra.run.smithi060:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0 2024-09-20T22:22:02.354 INFO:teuthology.orchestra.run.smithi060.stderr:sudo: docker: command not found 2024-09-20T22:22:02.355 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:02.355 INFO:tasks.cephadm.osd.0:Ignoring exception while sending signal: Command failed on smithi060 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.0' 2024-09-20T22:22:02.456 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:22:02.483 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:22:02.484 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:02.484 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:22:02.585 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4 2024-09-20T22:22:02.611 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:22:02.611 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:02.612 INFO:tasks.cephadm.osd.4:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.4' 2024-09-20T22:22:02.702 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:22:02.713 DEBUG:teuthology.orchestra.run.smithi067:> sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6 2024-09-20T22:22:02.739 INFO:teuthology.orchestra.run.smithi067.stderr:sudo: docker: command not found 2024-09-20T22:22:02.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:22:02.740 INFO:tasks.cephadm.osd.6:Ignoring exception while sending signal: Command failed on smithi067 with status 1: 'sudo docker kill -s 1 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd.6' 2024-09-20T22:22:02.784 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-09-20T22:22:02.799 INFO:tasks.thrashosds:joining thrashosds 2024-09-20T22:22:02.918 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:02 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1471011102' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:22:02.918 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:02 smithi067 ceph-mon[33511]: osdmap e540: 8 total, 8 up, 6 in 2024-09-20T22:22:02.918 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:02 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/713116622' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:22:02.918 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:02 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:22:03.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:02 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1471011102' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:22:03.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:02 smithi060 ceph-mon[29694]: osdmap e540: 8 total, 8 up, 6 in 2024-09-20T22:22:03.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:02 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/713116622' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:22:03.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:02 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:22:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:02 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1471011102' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:22:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:02 smithi060 ceph-mon[40290]: osdmap e540: 8 total, 8 up, 6 in 2024-09-20T22:22:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:02 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/713116622' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:22:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:02 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:22:03.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:22:02 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:22:03.425 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 5 1 2024-09-20T22:22:03.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:03 smithi060 ceph-mon[29694]: pgmap v1031: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T22:22:03.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:03 smithi060 ceph-mon[29694]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:22:03.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:03 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:22:03.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:03 smithi060 ceph-mon[29694]: osdmap e541: 8 total, 8 up, 6 in 2024-09-20T22:22:03.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:03 smithi060 ceph-mon[29694]: 6.b scrub starts 2024-09-20T22:22:03.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:03 smithi060 ceph-mon[29694]: 6.b scrub ok 2024-09-20T22:22:03.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:03 smithi060 ceph-mon[29694]: 4.6 scrub starts 2024-09-20T22:22:03.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:03 smithi060 ceph-mon[29694]: 4.6 scrub ok 2024-09-20T22:22:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:03 smithi060 ceph-mon[40290]: pgmap v1031: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T22:22:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:03 smithi060 ceph-mon[40290]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:22:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:03 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:22:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:03 smithi060 ceph-mon[40290]: osdmap e541: 8 total, 8 up, 6 in 2024-09-20T22:22:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:03 smithi060 ceph-mon[40290]: 6.b scrub starts 2024-09-20T22:22:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:03 smithi060 ceph-mon[40290]: 6.b scrub ok 2024-09-20T22:22:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:03 smithi060 ceph-mon[40290]: 4.6 scrub starts 2024-09-20T22:22:03.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:03 smithi060 ceph-mon[40290]: 4.6 scrub ok 2024-09-20T22:22:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:03 smithi067 ceph-mon[33511]: pgmap v1031: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T22:22:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:03 smithi067 ceph-mon[33511]: Health check update: nodeep-scrub flag(s) set (OSDMAP_FLAGS) 2024-09-20T22:22:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:03 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:22:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:03 smithi067 ceph-mon[33511]: osdmap e541: 8 total, 8 up, 6 in 2024-09-20T22:22:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:03 smithi067 ceph-mon[33511]: 6.b scrub starts 2024-09-20T22:22:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:03 smithi067 ceph-mon[33511]: 6.b scrub ok 2024-09-20T22:22:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:03 smithi067 ceph-mon[33511]: 4.6 scrub starts 2024-09-20T22:22:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:03 smithi067 ceph-mon[33511]: 4.6 scrub ok 2024-09-20T22:22:04.057 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:04.057 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:04.923 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:04 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:22:04.923 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:04 smithi060 ceph-mon[40290]: 1.4 scrub starts 2024-09-20T22:22:04.923 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:04 smithi060 ceph-mon[40290]: 1.4 scrub ok 2024-09-20T22:22:04.923 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:04 smithi060 ceph-mon[40290]: 4.8 scrub starts 2024-09-20T22:22:04.923 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:04 smithi060 ceph-mon[40290]: 4.8 scrub ok 2024-09-20T22:22:04.924 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:04 smithi060 ceph-mon[40290]: 3.1 scrub starts 2024-09-20T22:22:04.924 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:04 smithi060 ceph-mon[40290]: 3.1 scrub ok 2024-09-20T22:22:04.924 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:04 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:22:04.924 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:04 smithi060 ceph-mon[29694]: 1.4 scrub starts 2024-09-20T22:22:04.924 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:04 smithi060 ceph-mon[29694]: 1.4 scrub ok 2024-09-20T22:22:04.924 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:04 smithi060 ceph-mon[29694]: 4.8 scrub starts 2024-09-20T22:22:04.924 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:04 smithi060 ceph-mon[29694]: 4.8 scrub ok 2024-09-20T22:22:04.924 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:04 smithi060 ceph-mon[29694]: 3.1 scrub starts 2024-09-20T22:22:04.924 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:04 smithi060 ceph-mon[29694]: 3.1 scrub ok 2024-09-20T22:22:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:04 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:22:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:04 smithi067 ceph-mon[33511]: 1.4 scrub starts 2024-09-20T22:22:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:04 smithi067 ceph-mon[33511]: 1.4 scrub ok 2024-09-20T22:22:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:04 smithi067 ceph-mon[33511]: 4.8 scrub starts 2024-09-20T22:22:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:04 smithi067 ceph-mon[33511]: 4.8 scrub ok 2024-09-20T22:22:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:04 smithi067 ceph-mon[33511]: 3.1 scrub starts 2024-09-20T22:22:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:04 smithi067 ceph-mon[33511]: 3.1 scrub ok 2024-09-20T22:22:05.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:22:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:22:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:22:05.324 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:05.358 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_ops_in_flight 2024-09-20T22:22:05.580 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:05 smithi060 ceph-mon[40290]: pgmap v1033: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T22:22:05.581 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:05 smithi060 ceph-mon[40290]: 1.c scrub starts 2024-09-20T22:22:05.581 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:05 smithi060 ceph-mon[40290]: 1.c scrub ok 2024-09-20T22:22:05.581 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:05 smithi060 ceph-mon[29694]: pgmap v1033: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T22:22:05.581 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:05 smithi060 ceph-mon[29694]: 1.c scrub starts 2024-09-20T22:22:05.581 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:05 smithi060 ceph-mon[29694]: 1.c scrub ok 2024-09-20T22:22:05.852 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset noscrub 2024-09-20T22:22:05.877 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:05 smithi067 ceph-mon[33511]: pgmap v1033: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 682 B/s rd, 0 op/s 2024-09-20T22:22:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:05 smithi067 ceph-mon[33511]: 1.c scrub starts 2024-09-20T22:22:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:05 smithi067 ceph-mon[33511]: 1.c scrub ok 2024-09-20T22:22:06.369 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:06.369 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:06.550 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.5 to 1 (10000) 2024-09-20T22:22:06.815 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:06.928 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[29694]: 4.1b scrub starts 2024-09-20T22:22:06.928 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[29694]: 4.1b scrub ok 2024-09-20T22:22:06.929 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[29694]: 4.1f scrub starts 2024-09-20T22:22:06.929 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[29694]: 4.1f scrub ok 2024-09-20T22:22:06.929 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3322266130' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:22:06.929 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[29694]: 3.3 scrub starts 2024-09-20T22:22:06.929 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[29694]: 3.3 scrub ok 2024-09-20T22:22:06.929 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[29694]: 5.7 scrub starts 2024-09-20T22:22:06.929 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[29694]: 5.7 scrub ok 2024-09-20T22:22:06.929 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:22:06.929 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[40290]: 4.1b scrub starts 2024-09-20T22:22:06.929 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[40290]: 4.1b scrub ok 2024-09-20T22:22:06.929 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[40290]: 4.1f scrub starts 2024-09-20T22:22:06.930 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[40290]: 4.1f scrub ok 2024-09-20T22:22:06.930 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3322266130' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:22:06.930 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[40290]: 3.3 scrub starts 2024-09-20T22:22:06.930 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[40290]: 3.3 scrub ok 2024-09-20T22:22:06.930 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[40290]: 5.7 scrub starts 2024-09-20T22:22:06.930 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[40290]: 5.7 scrub ok 2024-09-20T22:22:06.930 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:22:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:06 smithi067 ceph-mon[33511]: 4.1b scrub starts 2024-09-20T22:22:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:06 smithi067 ceph-mon[33511]: 4.1b scrub ok 2024-09-20T22:22:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:06 smithi067 ceph-mon[33511]: 4.1f scrub starts 2024-09-20T22:22:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:06 smithi067 ceph-mon[33511]: 4.1f scrub ok 2024-09-20T22:22:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:06 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3322266130' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:22:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:06 smithi067 ceph-mon[33511]: 3.3 scrub starts 2024-09-20T22:22:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:06 smithi067 ceph-mon[33511]: 3.3 scrub ok 2024-09-20T22:22:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:06 smithi067 ceph-mon[33511]: 5.7 scrub starts 2024-09-20T22:22:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:06 smithi067 ceph-mon[33511]: 5.7 scrub ok 2024-09-20T22:22:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:22:07.307 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 6 1 2024-09-20T22:22:07.767 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_blocked_ops 2024-09-20T22:22:07.924 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:07 smithi060 ceph-mon[29694]: pgmap v1034: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-20T22:22:07.924 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:07 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3322266130' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:22:07.924 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:07 smithi060 ceph-mon[29694]: osdmap e542: 8 total, 8 up, 6 in 2024-09-20T22:22:07.925 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:07 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:22:07.925 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:07 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:22:07.925 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:07 smithi060 ceph-mon[29694]: 6.15 scrub starts 2024-09-20T22:22:07.925 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:07 smithi060 ceph-mon[29694]: 6.15 scrub ok 2024-09-20T22:22:07.925 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:07 smithi060 ceph-mon[40290]: pgmap v1034: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-20T22:22:07.925 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:07 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3322266130' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:22:07.925 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:07 smithi060 ceph-mon[40290]: osdmap e542: 8 total, 8 up, 6 in 2024-09-20T22:22:07.925 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:07 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:22:07.925 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:07 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:22:07.925 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:07 smithi060 ceph-mon[40290]: 6.15 scrub starts 2024-09-20T22:22:07.925 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:07 smithi060 ceph-mon[40290]: 6.15 scrub ok 2024-09-20T22:22:08.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:07 smithi067 ceph-mon[33511]: pgmap v1034: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-20T22:22:08.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:07 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3322266130' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:22:08.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:07 smithi067 ceph-mon[33511]: osdmap e542: 8 total, 8 up, 6 in 2024-09-20T22:22:08.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:07 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:22:08.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:07 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:22:08.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:07 smithi067 ceph-mon[33511]: 6.15 scrub starts 2024-09-20T22:22:08.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:07 smithi067 ceph-mon[33511]: 6.15 scrub ok 2024-09-20T22:22:08.310 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:08.311 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:08.563 INFO:teuthology.orchestra.run.smithi060.stderr:noscrub is unset 2024-09-20T22:22:08.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:08 smithi060 ceph-mon[40290]: osdmap e543: 8 total, 8 up, 6 in 2024-09-20T22:22:08.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:08 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/997172659' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:22:08.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:08 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:22:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:08 smithi060 ceph-mon[29694]: osdmap e543: 8 total, 8 up, 6 in 2024-09-20T22:22:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:08 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/997172659' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:22:08.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:08 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:22:09.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:08 smithi067 ceph-mon[33511]: osdmap e543: 8 total, 8 up, 6 in 2024-09-20T22:22:09.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:08 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/997172659' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:22:09.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:08 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "noscrub"}]: dispatch 2024-09-20T22:22:09.057 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:09.057 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:09.637 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:09 smithi060 ceph-mon[29694]: pgmap v1037: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:22:09.637 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:09 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:22:09.638 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:09 smithi060 ceph-mon[29694]: osdmap e544: 8 total, 8 up, 6 in 2024-09-20T22:22:09.638 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:09 smithi060 ceph-mon[29694]: 3.11 scrub starts 2024-09-20T22:22:09.638 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:09 smithi060 ceph-mon[29694]: 3.11 scrub ok 2024-09-20T22:22:09.638 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:09 smithi060 ceph-mon[40290]: pgmap v1037: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:22:09.638 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:09 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:22:09.638 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:09 smithi060 ceph-mon[40290]: osdmap e544: 8 total, 8 up, 6 in 2024-09-20T22:22:09.638 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:09 smithi060 ceph-mon[40290]: 3.11 scrub starts 2024-09-20T22:22:09.638 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:09 smithi060 ceph-mon[40290]: 3.11 scrub ok 2024-09-20T22:22:09.674 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.3 dump_historic_ops 2024-09-20T22:22:10.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:09 smithi067 ceph-mon[33511]: pgmap v1037: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:22:10.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:09 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "noscrub"}]': finished 2024-09-20T22:22:10.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:09 smithi067 ceph-mon[33511]: osdmap e544: 8 total, 8 up, 6 in 2024-09-20T22:22:10.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:09 smithi067 ceph-mon[33511]: 3.11 scrub starts 2024-09-20T22:22:10.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:09 smithi067 ceph-mon[33511]: 3.11 scrub ok 2024-09-20T22:22:10.562 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.6 to 1 (10000) 2024-09-20T22:22:10.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:10 smithi060 ceph-mon[29694]: 3.9 scrub starts 2024-09-20T22:22:10.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:10 smithi060 ceph-mon[29694]: 3.9 scrub ok 2024-09-20T22:22:10.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:10 smithi060 ceph-mon[29694]: 3.8 scrub starts 2024-09-20T22:22:10.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:10 smithi060 ceph-mon[29694]: 3.8 scrub ok 2024-09-20T22:22:10.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:10 smithi060 ceph-mon[29694]: 5.1e scrub starts 2024-09-20T22:22:10.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:10 smithi060 ceph-mon[29694]: 5.1e scrub ok 2024-09-20T22:22:10.703 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:10 smithi060 ceph-mon[29694]: 6.19 scrub starts 2024-09-20T22:22:10.703 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:10 smithi060 ceph-mon[29694]: 6.19 scrub ok 2024-09-20T22:22:10.703 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:10 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1896463824' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:22:10.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:10 smithi060 ceph-mon[40290]: 3.9 scrub starts 2024-09-20T22:22:10.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:10 smithi060 ceph-mon[40290]: 3.9 scrub ok 2024-09-20T22:22:10.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:10 smithi060 ceph-mon[40290]: 3.8 scrub starts 2024-09-20T22:22:10.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:10 smithi060 ceph-mon[40290]: 3.8 scrub ok 2024-09-20T22:22:10.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:10 smithi060 ceph-mon[40290]: 5.1e scrub starts 2024-09-20T22:22:10.704 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:10 smithi060 ceph-mon[40290]: 5.1e scrub ok 2024-09-20T22:22:10.704 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:10 smithi060 ceph-mon[40290]: 6.19 scrub starts 2024-09-20T22:22:10.704 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:10 smithi060 ceph-mon[40290]: 6.19 scrub ok 2024-09-20T22:22:10.704 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:10 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1896463824' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:22:10.729 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd unset nodeep-scrub 2024-09-20T22:22:11.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:10 smithi067 ceph-mon[33511]: 3.9 scrub starts 2024-09-20T22:22:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:10 smithi067 ceph-mon[33511]: 3.9 scrub ok 2024-09-20T22:22:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:10 smithi067 ceph-mon[33511]: 3.8 scrub starts 2024-09-20T22:22:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:10 smithi067 ceph-mon[33511]: 3.8 scrub ok 2024-09-20T22:22:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:10 smithi067 ceph-mon[33511]: 5.1e scrub starts 2024-09-20T22:22:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:10 smithi067 ceph-mon[33511]: 5.1e scrub ok 2024-09-20T22:22:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:10 smithi067 ceph-mon[33511]: 6.19 scrub starts 2024-09-20T22:22:11.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:10 smithi067 ceph-mon[33511]: 6.19 scrub ok 2024-09-20T22:22:11.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:10 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1896463824' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:22:11.384 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 7 1 2024-09-20T22:22:11.410 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:11.441 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:11.668 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[29694]: 4.1c scrub starts 2024-09-20T22:22:11.669 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[29694]: 4.1c scrub ok 2024-09-20T22:22:11.669 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[29694]: pgmap v1039: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:22:11.669 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1896463824' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:22:11.669 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[29694]: osdmap e545: 8 total, 8 up, 6 in 2024-09-20T22:22:11.669 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[29694]: 4.1d scrub starts 2024-09-20T22:22:11.669 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[29694]: 4.1d scrub ok 2024-09-20T22:22:11.669 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[29694]: 4.17 scrub starts 2024-09-20T22:22:11.669 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[29694]: 4.17 scrub ok 2024-09-20T22:22:11.669 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[29694]: 3.1d scrub starts 2024-09-20T22:22:11.669 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[29694]: 3.1d scrub ok 2024-09-20T22:22:11.669 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[29694]: 6.10 scrub starts 2024-09-20T22:22:11.669 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[29694]: 6.10 scrub ok 2024-09-20T22:22:11.670 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[40290]: 4.1c scrub starts 2024-09-20T22:22:11.670 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[40290]: 4.1c scrub ok 2024-09-20T22:22:11.670 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[40290]: pgmap v1039: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:22:11.670 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1896463824' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:22:11.670 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[40290]: osdmap e545: 8 total, 8 up, 6 in 2024-09-20T22:22:11.670 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[40290]: 4.1d scrub starts 2024-09-20T22:22:11.670 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[40290]: 4.1d scrub ok 2024-09-20T22:22:11.670 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[40290]: 4.17 scrub starts 2024-09-20T22:22:11.670 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[40290]: 4.17 scrub ok 2024-09-20T22:22:11.670 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[40290]: 3.1d scrub starts 2024-09-20T22:22:11.670 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[40290]: 3.1d scrub ok 2024-09-20T22:22:11.670 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[40290]: 6.10 scrub starts 2024-09-20T22:22:11.671 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:11 smithi060 ceph-mon[40290]: 6.10 scrub ok 2024-09-20T22:22:12.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:11 smithi067 ceph-mon[33511]: 4.1c scrub starts 2024-09-20T22:22:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:11 smithi067 ceph-mon[33511]: 4.1c scrub ok 2024-09-20T22:22:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:11 smithi067 ceph-mon[33511]: pgmap v1039: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:22:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:11 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1896463824' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:22:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:11 smithi067 ceph-mon[33511]: osdmap e545: 8 total, 8 up, 6 in 2024-09-20T22:22:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:11 smithi067 ceph-mon[33511]: 4.1d scrub starts 2024-09-20T22:22:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:11 smithi067 ceph-mon[33511]: 4.1d scrub ok 2024-09-20T22:22:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:11 smithi067 ceph-mon[33511]: 4.17 scrub starts 2024-09-20T22:22:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:11 smithi067 ceph-mon[33511]: 4.17 scrub ok 2024-09-20T22:22:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:11 smithi067 ceph-mon[33511]: 3.1d scrub starts 2024-09-20T22:22:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:11 smithi067 ceph-mon[33511]: 3.1d scrub ok 2024-09-20T22:22:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:11 smithi067 ceph-mon[33511]: 6.10 scrub starts 2024-09-20T22:22:12.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:11 smithi067 ceph-mon[33511]: 6.10 scrub ok 2024-09-20T22:22:12.129 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:12.129 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:12.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:12 smithi060 ceph-mon[29694]: 6.f scrub starts 2024-09-20T22:22:12.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:12 smithi060 ceph-mon[29694]: 6.f scrub ok 2024-09-20T22:22:12.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:12 smithi060 ceph-mon[29694]: osdmap e546: 8 total, 8 up, 6 in 2024-09-20T22:22:12.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:12 smithi060 ceph-mon[29694]: 3.1b scrub starts 2024-09-20T22:22:12.702 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:12 smithi060 ceph-mon[29694]: 3.1b scrub ok 2024-09-20T22:22:12.703 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:12 smithi060 ceph-mon[29694]: 4.16 scrub starts 2024-09-20T22:22:12.703 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:12 smithi060 ceph-mon[29694]: 4.16 scrub ok 2024-09-20T22:22:12.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:12 smithi060 ceph-mon[40290]: 6.f scrub starts 2024-09-20T22:22:12.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:12 smithi060 ceph-mon[40290]: 6.f scrub ok 2024-09-20T22:22:12.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:12 smithi060 ceph-mon[40290]: osdmap e546: 8 total, 8 up, 6 in 2024-09-20T22:22:12.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:12 smithi060 ceph-mon[40290]: 3.1b scrub starts 2024-09-20T22:22:12.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:12 smithi060 ceph-mon[40290]: 3.1b scrub ok 2024-09-20T22:22:12.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:12 smithi060 ceph-mon[40290]: 4.16 scrub starts 2024-09-20T22:22:12.703 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:12 smithi060 ceph-mon[40290]: 4.16 scrub ok 2024-09-20T22:22:12.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:12 smithi067 ceph-mon[33511]: 6.f scrub starts 2024-09-20T22:22:12.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:12 smithi067 ceph-mon[33511]: 6.f scrub ok 2024-09-20T22:22:12.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:12 smithi067 ceph-mon[33511]: osdmap e546: 8 total, 8 up, 6 in 2024-09-20T22:22:12.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:12 smithi067 ceph-mon[33511]: 3.1b scrub starts 2024-09-20T22:22:12.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:12 smithi067 ceph-mon[33511]: 3.1b scrub ok 2024-09-20T22:22:12.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:12 smithi067 ceph-mon[33511]: 4.16 scrub starts 2024-09-20T22:22:12.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:12 smithi067 ceph-mon[33511]: 4.16 scrub ok 2024-09-20T22:22:13.116 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_ops_in_flight 2024-09-20T22:22:13.184 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:13.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:22:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:22:13.416 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:22:13.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:13 smithi060 ceph-mon[29694]: pgmap v1042: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-20T22:22:13.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:13 smithi060 ceph-mon[29694]: 4.c scrub starts 2024-09-20T22:22:13.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:13 smithi060 ceph-mon[29694]: 4.c scrub ok 2024-09-20T22:22:13.649 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:13 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1368141769' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:22:13.649 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:13 smithi060 ceph-mon[29694]: 1.a scrub starts 2024-09-20T22:22:13.649 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:13 smithi060 ceph-mon[29694]: 1.a scrub ok 2024-09-20T22:22:13.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:13 smithi060 ceph-mon[40290]: pgmap v1042: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-20T22:22:13.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:13 smithi060 ceph-mon[40290]: 4.c scrub starts 2024-09-20T22:22:13.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:13 smithi060 ceph-mon[40290]: 4.c scrub ok 2024-09-20T22:22:13.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:13 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1368141769' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:22:13.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:13 smithi060 ceph-mon[40290]: 1.a scrub starts 2024-09-20T22:22:13.649 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:13 smithi060 ceph-mon[40290]: 1.a scrub ok 2024-09-20T22:22:13.699 INFO:teuthology.orchestra.run.smithi060.stderr:nodeep-scrub is unset 2024-09-20T22:22:13.832 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:13 smithi067 ceph-mon[33511]: pgmap v1042: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-20T22:22:13.832 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:13 smithi067 ceph-mon[33511]: 4.c scrub starts 2024-09-20T22:22:13.832 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:13 smithi067 ceph-mon[33511]: 4.c scrub ok 2024-09-20T22:22:13.832 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:13 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1368141769' entity='client.admin' cmd=[{"prefix": "osd unset", "key": "nodeep-scrub"}]: dispatch 2024-09-20T22:22:13.832 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:13 smithi067 ceph-mon[33511]: 1.a scrub starts 2024-09-20T22:22:13.832 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:13 smithi067 ceph-mon[33511]: 1.a scrub ok 2024-09-20T22:22:14.014 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:14.014 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:14.700 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.7 to 1 (10000) 2024-09-20T22:22:14.700 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:14 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:22:14.700 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:14 smithi060 ceph-mon[29694]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:22:14.700 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:14 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:22:14.700 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:14 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1368141769' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:22:14.701 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:14 smithi060 ceph-mon[29694]: osdmap e547: 8 total, 8 up, 6 in 2024-09-20T22:22:14.701 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:14 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2493471789' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:22:14.701 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:14 smithi060 ceph-mon[29694]: 5.1b scrub starts 2024-09-20T22:22:14.701 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:14 smithi060 ceph-mon[29694]: 5.1b scrub ok 2024-09-20T22:22:14.701 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:14 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:22:14.701 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:14 smithi060 ceph-mon[40290]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:22:14.701 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:14 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:22:14.701 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:14 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1368141769' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:22:14.701 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:14 smithi060 ceph-mon[40290]: osdmap e547: 8 total, 8 up, 6 in 2024-09-20T22:22:14.701 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:14 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2493471789' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:22:14.701 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:14 smithi060 ceph-mon[40290]: 5.1b scrub starts 2024-09-20T22:22:14.702 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:14 smithi060 ceph-mon[40290]: 5.1b scrub ok 2024-09-20T22:22:14.718 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:14 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:22:14.718 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:14 smithi067 ceph-mon[33511]: Health check cleared: OSDMAP_FLAGS (was: nodeep-scrub flag(s) set) 2024-09-20T22:22:14.719 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:14 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:22:14.719 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:14 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1368141769' entity='client.admin' cmd='[{"prefix": "osd unset", "key": "nodeep-scrub"}]': finished 2024-09-20T22:22:14.719 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:14 smithi067 ceph-mon[33511]: osdmap e547: 8 total, 8 up, 6 in 2024-09-20T22:22:14.719 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:14 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2493471789' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:22:14.719 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:14 smithi067 ceph-mon[33511]: 5.1b scrub starts 2024-09-20T22:22:14.719 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:14 smithi067 ceph-mon[33511]: 5.1b scrub ok 2024-09-20T22:22:14.751 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_blocked_ops 2024-09-20T22:22:15.010 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:22:15.340 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:22:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:22:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:22:15.377 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 0 1 2024-09-20T22:22:15.522 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:15.523 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:15.646 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:15 smithi067 ceph-mon[33511]: pgmap v1044: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 875 B/s rd, 0 op/s 2024-09-20T22:22:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:15 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2493471789' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:22:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:15 smithi067 ceph-mon[33511]: osdmap e548: 8 total, 8 up, 6 in 2024-09-20T22:22:15.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:15 smithi067 ceph-mon[33511]: 4.5 scrub starts 2024-09-20T22:22:15.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:15 smithi067 ceph-mon[33511]: 4.5 scrub ok 2024-09-20T22:22:15.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:15 smithi067 ceph-mon[33511]: 1.1b scrub starts 2024-09-20T22:22:15.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:15 smithi067 ceph-mon[33511]: 1.1b scrub ok 2024-09-20T22:22:15.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:15 smithi060 ceph-mon[29694]: pgmap v1044: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 875 B/s rd, 0 op/s 2024-09-20T22:22:15.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:15 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2493471789' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:22:15.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:15 smithi060 ceph-mon[29694]: osdmap e548: 8 total, 8 up, 6 in 2024-09-20T22:22:15.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:15 smithi060 ceph-mon[29694]: 4.5 scrub starts 2024-09-20T22:22:15.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:15 smithi060 ceph-mon[29694]: 4.5 scrub ok 2024-09-20T22:22:15.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:15 smithi060 ceph-mon[29694]: 1.1b scrub starts 2024-09-20T22:22:15.768 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:15 smithi060 ceph-mon[29694]: 1.1b scrub ok 2024-09-20T22:22:15.768 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:15 smithi060 ceph-mon[40290]: pgmap v1044: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 875 B/s rd, 0 op/s 2024-09-20T22:22:15.768 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:15 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2493471789' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:22:15.769 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:15 smithi060 ceph-mon[40290]: osdmap e548: 8 total, 8 up, 6 in 2024-09-20T22:22:15.769 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:15 smithi060 ceph-mon[40290]: 4.5 scrub starts 2024-09-20T22:22:15.769 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:15 smithi060 ceph-mon[40290]: 4.5 scrub ok 2024-09-20T22:22:15.769 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:15 smithi060 ceph-mon[40290]: 1.1b scrub starts 2024-09-20T22:22:15.769 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:15 smithi060 ceph-mon[40290]: 1.1b scrub ok 2024-09-20T22:22:16.157 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.5 dump_historic_ops 2024-09-20T22:22:16.414 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:22:16.590 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.0 to 1 (10000) 2024-09-20T22:22:16.847 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:16 smithi067 ceph-mon[33511]: 1.1d scrub starts 2024-09-20T22:22:16.847 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:16 smithi067 ceph-mon[33511]: 1.1d scrub ok 2024-09-20T22:22:16.848 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:16 smithi067 ceph-mon[33511]: 5.14 scrub starts 2024-09-20T22:22:16.848 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:16 smithi067 ceph-mon[33511]: 5.14 scrub ok 2024-09-20T22:22:16.848 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:16 smithi067 ceph-mon[33511]: 3.b scrub starts 2024-09-20T22:22:16.848 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:16 smithi067 ceph-mon[33511]: 3.b scrub ok 2024-09-20T22:22:16.848 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:16 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3446045846' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:22:16.848 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:16 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:22:16.897 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:16 smithi060 ceph-mon[29694]: 1.1d scrub starts 2024-09-20T22:22:16.898 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:16 smithi060 ceph-mon[29694]: 1.1d scrub ok 2024-09-20T22:22:16.898 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:16 smithi060 ceph-mon[29694]: 5.14 scrub starts 2024-09-20T22:22:16.898 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:16 smithi060 ceph-mon[29694]: 5.14 scrub ok 2024-09-20T22:22:16.898 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:16 smithi060 ceph-mon[29694]: 3.b scrub starts 2024-09-20T22:22:16.898 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:16 smithi060 ceph-mon[29694]: 3.b scrub ok 2024-09-20T22:22:16.898 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:16 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3446045846' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:22:16.898 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:16 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:22:16.898 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:16 smithi060 ceph-mon[40290]: 1.1d scrub starts 2024-09-20T22:22:16.898 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:16 smithi060 ceph-mon[40290]: 1.1d scrub ok 2024-09-20T22:22:16.898 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:16 smithi060 ceph-mon[40290]: 5.14 scrub starts 2024-09-20T22:22:16.898 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:16 smithi060 ceph-mon[40290]: 5.14 scrub ok 2024-09-20T22:22:16.899 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:16 smithi060 ceph-mon[40290]: 3.b scrub starts 2024-09-20T22:22:16.899 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:16 smithi060 ceph-mon[40290]: 3.b scrub ok 2024-09-20T22:22:16.899 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:16 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3446045846' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:22:16.899 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:16 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:22:16.955 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:16.956 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:17.331 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-20T22:22:17.332 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:22:17.604 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:17.727 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_ops_in_flight 2024-09-20T22:22:17.859 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[29694]: 6.14 scrub starts 2024-09-20T22:22:17.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[29694]: 6.14 scrub ok 2024-09-20T22:22:17.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[29694]: pgmap v1046: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-20T22:22:17.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[29694]: 1.9 scrub starts 2024-09-20T22:22:17.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[29694]: 1.9 scrub ok 2024-09-20T22:22:17.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:22:17.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[29694]: osdmap e549: 8 total, 8 up, 6 in 2024-09-20T22:22:17.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[29694]: osdmap e550: 8 total, 8 up, 6 in 2024-09-20T22:22:17.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[29694]: 5.1 scrub starts 2024-09-20T22:22:17.860 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[29694]: 5.1 scrub ok 2024-09-20T22:22:17.860 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[40290]: 6.14 scrub starts 2024-09-20T22:22:17.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[40290]: 6.14 scrub ok 2024-09-20T22:22:17.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[40290]: pgmap v1046: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-20T22:22:17.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[40290]: 1.9 scrub starts 2024-09-20T22:22:17.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[40290]: 1.9 scrub ok 2024-09-20T22:22:17.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:22:17.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[40290]: osdmap e549: 8 total, 8 up, 6 in 2024-09-20T22:22:17.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[40290]: osdmap e550: 8 total, 8 up, 6 in 2024-09-20T22:22:17.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[40290]: 5.1 scrub starts 2024-09-20T22:22:17.861 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:17 smithi060 ceph-mon[40290]: 5.1 scrub ok 2024-09-20T22:22:17.995 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:22:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:17 smithi067 ceph-mon[33511]: 6.14 scrub starts 2024-09-20T22:22:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:17 smithi067 ceph-mon[33511]: 6.14 scrub ok 2024-09-20T22:22:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:17 smithi067 ceph-mon[33511]: pgmap v1046: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-20T22:22:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:17 smithi067 ceph-mon[33511]: 1.9 scrub starts 2024-09-20T22:22:18.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:17 smithi067 ceph-mon[33511]: 1.9 scrub ok 2024-09-20T22:22:18.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:17 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:22:18.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:17 smithi067 ceph-mon[33511]: osdmap e549: 8 total, 8 up, 6 in 2024-09-20T22:22:18.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:17 smithi067 ceph-mon[33511]: osdmap e550: 8 total, 8 up, 6 in 2024-09-20T22:22:18.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:17 smithi067 ceph-mon[33511]: 5.1 scrub starts 2024-09-20T22:22:18.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:17 smithi067 ceph-mon[33511]: 5.1 scrub ok 2024-09-20T22:22:18.281 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:22:18.283 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:22:18.621 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:18.621 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:18.630 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:18 smithi067 ceph-mon[33511]: 6.17 scrub starts 2024-09-20T22:22:18.630 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:18 smithi067 ceph-mon[33511]: 6.17 scrub ok 2024-09-20T22:22:18.630 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:18 smithi067 ceph-mon[33511]: 4.f scrub starts 2024-09-20T22:22:18.630 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:18 smithi067 ceph-mon[33511]: 4.f scrub ok 2024-09-20T22:22:18.630 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:18 smithi067 ceph-mon[33511]: 6.c scrub starts 2024-09-20T22:22:18.630 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:18 smithi067 ceph-mon[33511]: 6.c scrub ok 2024-09-20T22:22:18.630 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:18 smithi067 ceph-mon[33511]: 3.16 scrub starts 2024-09-20T22:22:18.630 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:18 smithi067 ceph-mon[33511]: 3.16 scrub ok 2024-09-20T22:22:18.630 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:18 smithi067 ceph-mon[33511]: 5.1c scrub starts 2024-09-20T22:22:18.631 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:18 smithi067 ceph-mon[33511]: 5.1c scrub ok 2024-09-20T22:22:18.941 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[29694]: 6.17 scrub starts 2024-09-20T22:22:18.941 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[29694]: 6.17 scrub ok 2024-09-20T22:22:18.941 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[29694]: 4.f scrub starts 2024-09-20T22:22:18.941 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[29694]: 4.f scrub ok 2024-09-20T22:22:18.941 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[29694]: 6.c scrub starts 2024-09-20T22:22:18.941 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[29694]: 6.c scrub ok 2024-09-20T22:22:18.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[29694]: 3.16 scrub starts 2024-09-20T22:22:18.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[29694]: 3.16 scrub ok 2024-09-20T22:22:18.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[29694]: 5.1c scrub starts 2024-09-20T22:22:18.942 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[29694]: 5.1c scrub ok 2024-09-20T22:22:18.942 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[40290]: 6.17 scrub starts 2024-09-20T22:22:18.942 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[40290]: 6.17 scrub ok 2024-09-20T22:22:18.942 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[40290]: 4.f scrub starts 2024-09-20T22:22:18.942 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[40290]: 4.f scrub ok 2024-09-20T22:22:18.942 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[40290]: 6.c scrub starts 2024-09-20T22:22:18.942 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[40290]: 6.c scrub ok 2024-09-20T22:22:18.942 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[40290]: 3.16 scrub starts 2024-09-20T22:22:18.943 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[40290]: 3.16 scrub ok 2024-09-20T22:22:18.943 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[40290]: 5.1c scrub starts 2024-09-20T22:22:18.943 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:18 smithi060 ceph-mon[40290]: 5.1c scrub ok 2024-09-20T22:22:18.973 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":1048,"stamp":"2024-09-20T22:22:16.696317+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":2259,"num_read_kb":2011,"num_write":722,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":367,"num_bytes_recovered":10519,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4846,"ondisk_log_size":4846,"up":486,"acting":486,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":486,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":5326076,"kb_used_data":9144,"kb_used_omap":9,"kb_used_meta":5316790,"kb_avail":557144836,"statfs":{"total":575970213888,"available":570516312064,"internally_reserved":0,"allocated":9363456,"data_stored":6347241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":5444393638},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.291427"},"pg_stats":[{"pgid":"1.1d","version":"75'128","reported_seq":542,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111116+0000","last_change":"2024-09-20T22:19:48.867479+0000","last_active":"2024-09-20T22:22:12.111116+0000","last_peered":"2024-09-20T22:22:12.111116+0000","last_clean":"2024-09-20T22:22:12.111116+0000","last_became_active":"2024-09-20T22:19:48.866938+0000","last_became_peered":"2024-09-20T22:19:48.866938+0000","last_unstale":"2024-09-20T22:22:12.111116+0000","last_undegraded":"2024-09-20T22:22:12.111116+0000","last_fullsized":"2024-09-20T22:22:12.111116+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:57.128894+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.128894+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":483,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111886+0000","last_change":"2024-09-20T22:18:43.512180+0000","last_active":"2024-09-20T22:22:12.111886+0000","last_peered":"2024-09-20T22:22:12.111886+0000","last_clean":"2024-09-20T22:22:12.111886+0000","last_became_active":"2024-09-20T22:12:52.515921+0000","last_became_peered":"2024-09-20T22:12:52.515921+0000","last_unstale":"2024-09-20T22:22:12.111886+0000","last_undegraded":"2024-09-20T22:22:12.111886+0000","last_fullsized":"2024-09-20T22:22:12.111886+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:43.512066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:43.512066+0000","last_clean_scrub_stamp":"2024-09-20T22:18:43.512066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"327'19","reported_seq":782,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577539+0000","last_change":"2024-09-20T22:19:45.861936+0000","last_active":"2024-09-20T22:22:11.577539+0000","last_peered":"2024-09-20T22:22:11.577539+0000","last_clean":"2024-09-20T22:22:11.577539+0000","last_became_active":"2024-09-20T22:19:45.798056+0000","last_became_peered":"2024-09-20T22:19:45.798056+0000","last_unstale":"2024-09-20T22:22:11.577539+0000","last_undegraded":"2024-09-20T22:22:11.577539+0000","last_fullsized":"2024-09-20T22:22:11.577539+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"327'19","last_scrub_stamp":"2024-09-20T22:19:15.299519+0000","last_deep_scrub":"252'13","last_deep_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_clean_scrub_stamp":"2024-09-20T22:19:15.299519+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":270,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592643+0000","last_change":"2024-09-20T22:21:38.882602+0000","last_active":"2024-09-20T22:22:16.592643+0000","last_peered":"2024-09-20T22:22:16.592643+0000","last_clean":"2024-09-20T22:22:16.592643+0000","last_became_active":"2024-09-20T22:19:45.795374+0000","last_became_peered":"2024-09-20T22:19:45.795374+0000","last_unstale":"2024-09-20T22:22:16.592643+0000","last_undegraded":"2024-09-20T22:22:16.592643+0000","last_fullsized":"2024-09-20T22:22:16.592643+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:38.882495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:21:38.882495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:43.895865+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":639,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593339+0000","last_change":"2024-09-20T22:19:45.797905+0000","last_active":"2024-09-20T22:22:16.593339+0000","last_peered":"2024-09-20T22:22:16.593339+0000","last_clean":"2024-09-20T22:22:16.593339+0000","last_became_active":"2024-09-20T22:19:45.796589+0000","last_became_peered":"2024-09-20T22:19:45.796589+0000","last_unstale":"2024-09-20T22:22:16.593339+0000","last_undegraded":"2024-09-20T22:22:16.593339+0000","last_fullsized":"2024-09-20T22:22:16.593339+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.321275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.321275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1c","version":"75'128","reported_seq":999,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738379+0000","last_change":"2024-09-20T22:19:48.838779+0000","last_active":"2024-09-20T22:22:14.738379+0000","last_peered":"2024-09-20T22:22:14.738379+0000","last_clean":"2024-09-20T22:22:14.738379+0000","last_became_active":"2024-09-20T22:19:48.838208+0000","last_became_peered":"2024-09-20T22:19:48.838208+0000","last_unstale":"2024-09-20T22:22:14.738379+0000","last_undegraded":"2024-09-20T22:22:14.738379+0000","last_fullsized":"2024-09-20T22:22:14.738379+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:57.852213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.852213+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":1034,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577017+0000","last_change":"2024-09-20T22:19:38.797027+0000","last_active":"2024-09-20T22:22:11.577017+0000","last_peered":"2024-09-20T22:22:11.577017+0000","last_clean":"2024-09-20T22:22:11.577017+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:22:11.577017+0000","last_undegraded":"2024-09-20T22:22:11.577017+0000","last_fullsized":"2024-09-20T22:22:11.577017+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:19:38.796920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:19:38.796920+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":510,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578685+0000","last_change":"2024-09-20T22:18:31.987311+0000","last_active":"2024-09-20T22:22:11.578685+0000","last_peered":"2024-09-20T22:22:11.578685+0000","last_clean":"2024-09-20T22:22:11.578685+0000","last_became_active":"2024-09-20T22:12:33.545100+0000","last_became_peered":"2024-09-20T22:12:33.545100+0000","last_unstale":"2024-09-20T22:22:11.578685+0000","last_undegraded":"2024-09-20T22:22:11.578685+0000","last_fullsized":"2024-09-20T22:22:11.578685+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:31.987191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:31.987191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"406'34","reported_seq":641,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110131+0000","last_change":"2024-09-20T22:19:45.100154+0000","last_active":"2024-09-20T22:22:12.110131+0000","last_peered":"2024-09-20T22:22:12.110131+0000","last_clean":"2024-09-20T22:22:12.110131+0000","last_became_active":"2024-09-20T22:12:52.517296+0000","last_became_peered":"2024-09-20T22:12:52.517296+0000","last_unstale":"2024-09-20T22:22:12.110131+0000","last_undegraded":"2024-09-20T22:22:12.110131+0000","last_fullsized":"2024-09-20T22:22:12.110131+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'34","last_scrub_stamp":"2024-09-20T22:19:45.100033+0000","last_deep_scrub":"406'34","last_deep_scrub_stamp":"2024-09-20T22:16:31.033354+0000","last_clean_scrub_stamp":"2024-09-20T22:19:45.100033+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":333,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577441+0000","last_change":"2024-09-20T22:19:45.791468+0000","last_active":"2024-09-20T22:22:11.577441+0000","last_peered":"2024-09-20T22:22:11.577441+0000","last_clean":"2024-09-20T22:22:11.577441+0000","last_became_active":"2024-09-20T22:19:45.791138+0000","last_became_peered":"2024-09-20T22:19:45.791138+0000","last_unstale":"2024-09-20T22:22:11.577441+0000","last_undegraded":"2024-09-20T22:22:11.577441+0000","last_fullsized":"2024-09-20T22:22:11.577441+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:10.335527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:10.335527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1071,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593696+0000","last_change":"2024-09-20T22:22:11.073921+0000","last_active":"2024-09-20T22:22:16.593696+0000","last_peered":"2024-09-20T22:22:16.593696+0000","last_clean":"2024-09-20T22:22:16.593696+0000","last_became_active":"2024-09-20T22:19:45.790130+0000","last_became_peered":"2024-09-20T22:19:45.790130+0000","last_unstale":"2024-09-20T22:22:16.593696+0000","last_undegraded":"2024-09-20T22:22:16.593696+0000","last_fullsized":"2024-09-20T22:22:16.593696+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:11.073832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:22:11.073832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:32.257687+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"328'15","reported_seq":918,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111361+0000","last_change":"2024-09-20T22:18:35.404979+0000","last_active":"2024-09-20T22:22:12.111361+0000","last_peered":"2024-09-20T22:22:12.111361+0000","last_clean":"2024-09-20T22:22:12.111361+0000","last_became_active":"2024-09-20T22:17:03.663697+0000","last_became_peered":"2024-09-20T22:17:03.663697+0000","last_unstale":"2024-09-20T22:22:12.111361+0000","last_undegraded":"2024-09-20T22:22:12.111361+0000","last_fullsized":"2024-09-20T22:22:12.111361+0000","mapping_epoch":415,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":416,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:18:35.404892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:35.404892+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":591,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592994+0000","last_change":"2024-09-20T22:22:14.027262+0000","last_active":"2024-09-20T22:22:16.592994+0000","last_peered":"2024-09-20T22:22:16.592994+0000","last_clean":"2024-09-20T22:22:16.592994+0000","last_became_active":"2024-09-20T22:12:52.518549+0000","last_became_peered":"2024-09-20T22:12:52.518549+0000","last_unstale":"2024-09-20T22:22:16.592994+0000","last_undegraded":"2024-09-20T22:22:16.592994+0000","last_fullsized":"2024-09-20T22:22:16.592994+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:14.027140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:14.027140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:20.774583+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":330,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737460+0000","last_change":"2024-09-20T22:19:45.801571+0000","last_active":"2024-09-20T22:22:14.737460+0000","last_peered":"2024-09-20T22:22:14.737460+0000","last_clean":"2024-09-20T22:22:14.737460+0000","last_became_active":"2024-09-20T22:19:45.801215+0000","last_became_peered":"2024-09-20T22:19:45.801215+0000","last_unstale":"2024-09-20T22:22:14.737460+0000","last_undegraded":"2024-09-20T22:22:14.737460+0000","last_fullsized":"2024-09-20T22:22:14.737460+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:13.392405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:37.934760+0000","last_clean_scrub_stamp":"2024-09-20T22:19:13.392405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1e","version":"75'128","reported_seq":554,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578041+0000","last_change":"2024-09-20T22:19:34.669163+0000","last_active":"2024-09-20T22:22:11.578041+0000","last_peered":"2024-09-20T22:22:11.578041+0000","last_clean":"2024-09-20T22:22:11.578041+0000","last_became_active":"2024-09-20T22:17:44.708330+0000","last_became_peered":"2024-09-20T22:17:44.708330+0000","last_unstale":"2024-09-20T22:22:11.578041+0000","last_undegraded":"2024-09-20T22:22:11.578041+0000","last_fullsized":"2024-09-20T22:22:11.578041+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:34.669046+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:17:40.367940+0000","last_clean_scrub_stamp":"2024-09-20T22:19:34.669046+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1068,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593194+0000","last_change":"2024-09-20T22:19:45.796110+0000","last_active":"2024-09-20T22:22:16.593194+0000","last_peered":"2024-09-20T22:22:16.593194+0000","last_clean":"2024-09-20T22:22:16.593194+0000","last_became_active":"2024-09-20T22:19:45.795550+0000","last_became_peered":"2024-09-20T22:19:45.795550+0000","last_unstale":"2024-09-20T22:22:16.593194+0000","last_undegraded":"2024-09-20T22:22:16.593194+0000","last_fullsized":"2024-09-20T22:22:16.593194+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:03.470297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:30.215263+0000","last_clean_scrub_stamp":"2024-09-20T22:18:03.470297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"328'7","reported_seq":753,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110522+0000","last_change":"2024-09-20T22:22:04.903574+0000","last_active":"2024-09-20T22:22:12.110522+0000","last_peered":"2024-09-20T22:22:12.110522+0000","last_clean":"2024-09-20T22:22:12.110522+0000","last_became_active":"2024-09-20T22:19:45.799278+0000","last_became_peered":"2024-09-20T22:19:45.799278+0000","last_unstale":"2024-09-20T22:22:12.110522+0000","last_undegraded":"2024-09-20T22:22:12.110522+0000","last_fullsized":"2024-09-20T22:22:12.110522+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'7","last_scrub_stamp":"2024-09-20T22:22:04.903491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:04.903491+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:31.027625+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1055,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737759+0000","last_change":"2024-09-20T22:21:41.666202+0000","last_active":"2024-09-20T22:22:14.737759+0000","last_peered":"2024-09-20T22:22:14.737759+0000","last_clean":"2024-09-20T22:22:14.737759+0000","last_became_active":"2024-09-20T22:19:45.817477+0000","last_became_peered":"2024-09-20T22:19:45.817477+0000","last_unstale":"2024-09-20T22:22:14.737759+0000","last_undegraded":"2024-09-20T22:22:14.737759+0000","last_fullsized":"2024-09-20T22:22:14.737759+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:41.666098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:21:41.666098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:43.290262+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":585,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592642+0000","last_change":"2024-09-20T22:22:10.109040+0000","last_active":"2024-09-20T22:22:16.592642+0000","last_peered":"2024-09-20T22:22:16.592642+0000","last_clean":"2024-09-20T22:22:16.592642+0000","last_became_active":"2024-09-20T22:19:45.789372+0000","last_became_peered":"2024-09-20T22:19:45.789372+0000","last_unstale":"2024-09-20T22:22:16.592642+0000","last_undegraded":"2024-09-20T22:22:16.592642+0000","last_fullsized":"2024-09-20T22:22:16.592642+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:10.108947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:22:10.108947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:22.119266+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.19","version":"75'128","reported_seq":811,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110634+0000","last_change":"2024-09-20T22:17:56.103865+0000","last_active":"2024-09-20T22:22:12.110634+0000","last_peered":"2024-09-20T22:22:12.110634+0000","last_clean":"2024-09-20T22:22:12.110634+0000","last_became_active":"2024-09-20T22:17:44.716139+0000","last_became_peered":"2024-09-20T22:17:44.716139+0000","last_unstale":"2024-09-20T22:22:12.110634+0000","last_undegraded":"2024-09-20T22:22:12.110634+0000","last_fullsized":"2024-09-20T22:22:12.110634+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.103744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.103744+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":167,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578564+0000","last_change":"2024-09-20T22:19:44.809795+0000","last_active":"2024-09-20T22:22:11.578564+0000","last_peered":"2024-09-20T22:22:11.578564+0000","last_clean":"2024-09-20T22:22:11.578564+0000","last_became_active":"2024-09-20T22:19:28.245420+0000","last_became_peered":"2024-09-20T22:19:28.245420+0000","last_unstale":"2024-09-20T22:22:11.578564+0000","last_undegraded":"2024-09-20T22:22:11.578564+0000","last_fullsized":"2024-09-20T22:22:11.578564+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:44.809724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:19:44.809724+0000","last_clean_scrub_stamp":"2024-09-20T22:19:44.809724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":196,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738252+0000","last_change":"2024-09-20T22:22:11.655972+0000","last_active":"2024-09-20T22:22:14.738252+0000","last_peered":"2024-09-20T22:22:14.738252+0000","last_clean":"2024-09-20T22:22:14.738252+0000","last_became_active":"2024-09-20T22:19:45.796028+0000","last_became_peered":"2024-09-20T22:19:45.796028+0000","last_unstale":"2024-09-20T22:22:14.738252+0000","last_undegraded":"2024-09-20T22:22:14.738252+0000","last_fullsized":"2024-09-20T22:22:14.738252+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:11.655870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:22:11.655870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:24.567560+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1c","version":"328'34","reported_seq":659,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111110+0000","last_change":"2024-09-20T22:22:10.009868+0000","last_active":"2024-09-20T22:22:12.111110+0000","last_peered":"2024-09-20T22:22:12.111110+0000","last_clean":"2024-09-20T22:22:12.111110+0000","last_became_active":"2024-09-20T22:19:45.805481+0000","last_became_peered":"2024-09-20T22:19:45.805481+0000","last_unstale":"2024-09-20T22:22:12.111110+0000","last_undegraded":"2024-09-20T22:22:12.111110+0000","last_fullsized":"2024-09-20T22:22:12.111110+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'34","last_scrub_stamp":"2024-09-20T22:22:10.009795+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:22:10.009795+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:18.889671+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":872,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110693+0000","last_change":"2024-09-20T22:19:45.818815+0000","last_active":"2024-09-20T22:22:12.110693+0000","last_peered":"2024-09-20T22:22:12.110693+0000","last_clean":"2024-09-20T22:22:12.110693+0000","last_became_active":"2024-09-20T22:19:45.818397+0000","last_became_peered":"2024-09-20T22:19:45.818397+0000","last_unstale":"2024-09-20T22:22:12.110693+0000","last_undegraded":"2024-09-20T22:22:12.110693+0000","last_fullsized":"2024-09-20T22:22:12.110693+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.295028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:44.295028+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.295028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.18","version":"75'128","reported_seq":657,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592864+0000","last_change":"2024-09-20T22:17:56.357255+0000","last_active":"2024-09-20T22:22:16.592864+0000","last_peered":"2024-09-20T22:22:16.592864+0000","last_clean":"2024-09-20T22:22:16.592864+0000","last_became_active":"2024-09-20T22:17:44.717551+0000","last_became_peered":"2024-09-20T22:17:44.717551+0000","last_unstale":"2024-09-20T22:22:16.592864+0000","last_undegraded":"2024-09-20T22:22:16.592864+0000","last_fullsized":"2024-09-20T22:22:16.592864+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.357167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.357167+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":979,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578505+0000","last_change":"2024-09-20T22:18:31.015563+0000","last_active":"2024-09-20T22:22:11.578505+0000","last_peered":"2024-09-20T22:22:11.578505+0000","last_clean":"2024-09-20T22:22:11.578505+0000","last_became_active":"2024-09-20T22:12:21.521433+0000","last_became_peered":"2024-09-20T22:12:21.521433+0000","last_unstale":"2024-09-20T22:22:11.578505+0000","last_undegraded":"2024-09-20T22:22:11.578505+0000","last_fullsized":"2024-09-20T22:22:11.578505+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:31.015449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:31.015449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":271,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110973+0000","last_change":"2024-09-20T22:19:45.802144+0000","last_active":"2024-09-20T22:22:12.110973+0000","last_peered":"2024-09-20T22:22:12.110973+0000","last_clean":"2024-09-20T22:22:12.110973+0000","last_became_active":"2024-09-20T22:19:45.801808+0000","last_became_peered":"2024-09-20T22:19:45.801808+0000","last_unstale":"2024-09-20T22:22:12.110973+0000","last_undegraded":"2024-09-20T22:22:12.110973+0000","last_fullsized":"2024-09-20T22:22:12.110973+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.456415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:44.456415+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.456415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"328'23","reported_seq":956,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578442+0000","last_change":"2024-09-20T22:22:10.697410+0000","last_active":"2024-09-20T22:22:11.578442+0000","last_peered":"2024-09-20T22:22:11.578442+0000","last_clean":"2024-09-20T22:22:11.578442+0000","last_became_active":"2024-09-20T22:19:45.791170+0000","last_became_peered":"2024-09-20T22:19:45.791170+0000","last_unstale":"2024-09-20T22:22:11.578442+0000","last_undegraded":"2024-09-20T22:22:11.578442+0000","last_fullsized":"2024-09-20T22:22:11.578442+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'23","last_scrub_stamp":"2024-09-20T22:22:10.697295+0000","last_deep_scrub":"250'17","last_deep_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_clean_scrub_stamp":"2024-09-20T22:22:10.697295+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:20.186114+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":167,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577040+0000","last_change":"2024-09-20T22:19:31.646518+0000","last_active":"2024-09-20T22:22:11.577040+0000","last_peered":"2024-09-20T22:22:11.577040+0000","last_clean":"2024-09-20T22:22:11.577040+0000","last_became_active":"2024-09-20T22:19:28.238293+0000","last_became_peered":"2024-09-20T22:19:28.238293+0000","last_unstale":"2024-09-20T22:22:11.577040+0000","last_undegraded":"2024-09-20T22:22:11.577040+0000","last_fullsized":"2024-09-20T22:22:11.577040+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:31.646417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:31.646417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1b","version":"75'128","reported_seq":978,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593227+0000","last_change":"2024-09-20T22:22:15.018084+0000","last_active":"2024-09-20T22:22:16.593227+0000","last_peered":"2024-09-20T22:22:16.593227+0000","last_clean":"2024-09-20T22:22:16.593227+0000","last_became_active":"2024-09-20T22:19:31.252076+0000","last_became_peered":"2024-09-20T22:19:31.252076+0000","last_unstale":"2024-09-20T22:22:16.593227+0000","last_undegraded":"2024-09-20T22:22:16.593227+0000","last_fullsized":"2024-09-20T22:22:16.593227+0000","mapping_epoch":475,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":476,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:15.018011+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:22:15.018011+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:15.254394+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":1024,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738604+0000","last_change":"2024-09-20T22:18:09.700077+0000","last_active":"2024-09-20T22:22:14.738604+0000","last_peered":"2024-09-20T22:22:14.738604+0000","last_clean":"2024-09-20T22:22:14.738604+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:22:14.738604+0000","last_undegraded":"2024-09-20T22:22:14.738604+0000","last_fullsized":"2024-09-20T22:22:14.738604+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:18:09.699725+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:18:09.699725+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"289'2","reported_seq":476,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578572+0000","last_change":"2024-09-20T22:19:46.757972+0000","last_active":"2024-09-20T22:22:11.578572+0000","last_peered":"2024-09-20T22:22:11.578572+0000","last_clean":"2024-09-20T22:22:11.578572+0000","last_became_active":"2024-09-20T22:19:45.796939+0000","last_became_peered":"2024-09-20T22:19:45.796939+0000","last_unstale":"2024-09-20T22:22:11.578572+0000","last_undegraded":"2024-09-20T22:22:11.578572+0000","last_fullsized":"2024-09-20T22:22:11.578572+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-20T22:19:46.757900+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_clean_scrub_stamp":"2024-09-20T22:19:46.757900+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"328'20","reported_seq":510,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111857+0000","last_change":"2024-09-20T22:18:10.346398+0000","last_active":"2024-09-20T22:22:12.111857+0000","last_peered":"2024-09-20T22:22:12.111857+0000","last_clean":"2024-09-20T22:22:12.111857+0000","last_became_active":"2024-09-20T22:12:52.526803+0000","last_became_peered":"2024-09-20T22:12:52.526803+0000","last_unstale":"2024-09-20T22:22:12.111857+0000","last_undegraded":"2024-09-20T22:22:12.111857+0000","last_fullsized":"2024-09-20T22:22:12.111857+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:18:10.346320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:10.346320+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"475'12","reported_seq":1322,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578820+0000","last_change":"2024-09-20T22:19:28.274959+0000","last_active":"2024-09-20T22:22:11.578820+0000","last_peered":"2024-09-20T22:22:11.578820+0000","last_clean":"2024-09-20T22:22:11.578820+0000","last_became_active":"2024-09-20T22:19:28.256901+0000","last_became_peered":"2024-09-20T22:19:28.256901+0000","last_unstale":"2024-09-20T22:22:11.578820+0000","last_undegraded":"2024-09-20T22:22:11.578820+0000","last_fullsized":"2024-09-20T22:22:11.578820+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'11","last_scrub_stamp":"2024-09-20T22:18:33.978168+0000","last_deep_scrub":"291'11","last_deep_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_clean_scrub_stamp":"2024-09-20T22:18:33.978168+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":1061,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593059+0000","last_change":"2024-09-20T22:22:13.011302+0000","last_active":"2024-09-20T22:22:16.593059+0000","last_peered":"2024-09-20T22:22:16.593059+0000","last_clean":"2024-09-20T22:22:16.593059+0000","last_became_active":"2024-09-20T22:17:40.139526+0000","last_became_peered":"2024-09-20T22:17:40.139526+0000","last_unstale":"2024-09-20T22:22:16.593059+0000","last_undegraded":"2024-09-20T22:22:16.593059+0000","last_fullsized":"2024-09-20T22:22:16.593059+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:13.011208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:13.011208+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:28.826840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"328'27","reported_seq":835,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738474+0000","last_change":"2024-09-20T22:19:45.821849+0000","last_active":"2024-09-20T22:22:14.738474+0000","last_peered":"2024-09-20T22:22:14.738474+0000","last_clean":"2024-09-20T22:22:14.738474+0000","last_became_active":"2024-09-20T22:19:45.801225+0000","last_became_peered":"2024-09-20T22:19:45.801225+0000","last_unstale":"2024-09-20T22:22:14.738474+0000","last_undegraded":"2024-09-20T22:22:14.738474+0000","last_fullsized":"2024-09-20T22:22:14.738474+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:17:34.818375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:34.818375+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1069,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577571+0000","last_change":"2024-09-20T22:22:09.970276+0000","last_active":"2024-09-20T22:22:11.577571+0000","last_peered":"2024-09-20T22:22:11.577571+0000","last_clean":"2024-09-20T22:22:11.577571+0000","last_became_active":"2024-09-20T22:19:45.797845+0000","last_became_peered":"2024-09-20T22:19:45.797845+0000","last_unstale":"2024-09-20T22:22:11.577571+0000","last_undegraded":"2024-09-20T22:22:11.577571+0000","last_fullsized":"2024-09-20T22:22:11.577571+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:09.970183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:19:17.513228+0000","last_clean_scrub_stamp":"2024-09-20T22:22:09.970183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:38.994419+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"475'14","reported_seq":811,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593806+0000","last_change":"2024-09-20T22:19:28.252125+0000","last_active":"2024-09-20T22:22:16.593806+0000","last_peered":"2024-09-20T22:22:16.593806+0000","last_clean":"2024-09-20T22:22:16.593806+0000","last_became_active":"2024-09-20T22:19:28.242412+0000","last_became_peered":"2024-09-20T22:19:28.242412+0000","last_unstale":"2024-09-20T22:22:16.593806+0000","last_undegraded":"2024-09-20T22:22:16.593806+0000","last_fullsized":"2024-09-20T22:22:16.593806+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'13","last_scrub_stamp":"2024-09-20T22:18:40.407802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:40.407802+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":589,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593963+0000","last_change":"2024-09-20T22:19:45.840327+0000","last_active":"2024-09-20T22:22:16.593963+0000","last_peered":"2024-09-20T22:22:16.593963+0000","last_clean":"2024-09-20T22:22:16.593963+0000","last_became_active":"2024-09-20T22:19:45.839873+0000","last_became_peered":"2024-09-20T22:19:45.839873+0000","last_unstale":"2024-09-20T22:22:16.593963+0000","last_undegraded":"2024-09-20T22:22:16.593963+0000","last_fullsized":"2024-09-20T22:22:16.593963+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:04.453445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:04.453445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":460,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111327+0000","last_change":"2024-09-20T22:21:34.606254+0000","last_active":"2024-09-20T22:22:12.111327+0000","last_peered":"2024-09-20T22:22:12.111327+0000","last_clean":"2024-09-20T22:22:12.111327+0000","last_became_active":"2024-09-20T22:19:31.280713+0000","last_became_peered":"2024-09-20T22:19:31.280713+0000","last_unstale":"2024-09-20T22:22:12.111327+0000","last_undegraded":"2024-09-20T22:22:12.111327+0000","last_fullsized":"2024-09-20T22:22:12.111327+0000","mapping_epoch":475,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":476,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:34.606042+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:21:34.606042+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:45.282173+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"328'30","reported_seq":1047,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737436+0000","last_change":"2024-09-20T22:18:43.923440+0000","last_active":"2024-09-20T22:22:14.737436+0000","last_peered":"2024-09-20T22:22:14.737436+0000","last_clean":"2024-09-20T22:22:14.737436+0000","last_became_active":"2024-09-20T22:12:52.526867+0000","last_became_peered":"2024-09-20T22:12:52.526867+0000","last_unstale":"2024-09-20T22:22:14.737436+0000","last_undegraded":"2024-09-20T22:22:14.737436+0000","last_fullsized":"2024-09-20T22:22:14.737436+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'30","last_scrub_stamp":"2024-09-20T22:18:43.923335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:43.923335+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":528,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.739164+0000","last_change":"2024-09-20T22:18:45.931576+0000","last_active":"2024-09-20T22:22:14.739164+0000","last_peered":"2024-09-20T22:22:14.739164+0000","last_clean":"2024-09-20T22:22:14.739164+0000","last_became_active":"2024-09-20T22:12:52.528947+0000","last_became_peered":"2024-09-20T22:12:52.528947+0000","last_unstale":"2024-09-20T22:22:14.739164+0000","last_undegraded":"2024-09-20T22:22:14.739164+0000","last_fullsized":"2024-09-20T22:22:14.739164+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.931495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.931495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":598,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578701+0000","last_change":"2024-09-20T22:19:35.720326+0000","last_active":"2024-09-20T22:22:11.578701+0000","last_peered":"2024-09-20T22:22:11.578701+0000","last_clean":"2024-09-20T22:22:11.578701+0000","last_became_active":"2024-09-20T22:19:28.241950+0000","last_became_peered":"2024-09-20T22:19:28.241950+0000","last_unstale":"2024-09-20T22:22:11.578701+0000","last_undegraded":"2024-09-20T22:22:11.578701+0000","last_fullsized":"2024-09-20T22:22:11.578701+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:35.719732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:35.719732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":167,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577883+0000","last_change":"2024-09-20T22:19:36.736788+0000","last_active":"2024-09-20T22:22:11.577883+0000","last_peered":"2024-09-20T22:22:11.577883+0000","last_clean":"2024-09-20T22:22:11.577883+0000","last_became_active":"2024-09-20T22:19:28.241638+0000","last_became_peered":"2024-09-20T22:19:28.241638+0000","last_unstale":"2024-09-20T22:22:11.577883+0000","last_undegraded":"2024-09-20T22:22:11.577883+0000","last_fullsized":"2024-09-20T22:22:11.577883+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:36.736314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:19:36.736314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":1097,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.576846+0000","last_change":"2024-09-20T22:22:04.005288+0000","last_active":"2024-09-20T22:22:11.576846+0000","last_peered":"2024-09-20T22:22:11.576846+0000","last_clean":"2024-09-20T22:22:11.576846+0000","last_became_active":"2024-09-20T22:15:45.642671+0000","last_became_peered":"2024-09-20T22:15:45.642671+0000","last_unstale":"2024-09-20T22:22:11.576846+0000","last_undegraded":"2024-09-20T22:22:11.576846+0000","last_fullsized":"2024-09-20T22:22:11.576846+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:04.005213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:04.005213+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:25.775753+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"328'28","reported_seq":792,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592758+0000","last_change":"2024-09-20T22:19:28.268499+0000","last_active":"2024-09-20T22:22:16.592758+0000","last_peered":"2024-09-20T22:22:16.592758+0000","last_clean":"2024-09-20T22:22:16.592758+0000","last_became_active":"2024-09-20T22:19:28.245672+0000","last_became_peered":"2024-09-20T22:19:28.245672+0000","last_unstale":"2024-09-20T22:22:16.592758+0000","last_undegraded":"2024-09-20T22:22:16.592758+0000","last_fullsized":"2024-09-20T22:22:16.592758+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:19:14.334175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:14.334175+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":515,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111954+0000","last_change":"2024-09-20T22:19:45.873383+0000","last_active":"2024-09-20T22:22:12.111954+0000","last_peered":"2024-09-20T22:22:12.111954+0000","last_clean":"2024-09-20T22:22:12.111954+0000","last_became_active":"2024-09-20T22:19:45.824809+0000","last_became_peered":"2024-09-20T22:19:45.824809+0000","last_unstale":"2024-09-20T22:22:12.111954+0000","last_undegraded":"2024-09-20T22:22:12.111954+0000","last_fullsized":"2024-09-20T22:22:12.111954+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:19:17.876492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:19:17.876492+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"481'13","reported_seq":838,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.109574+0000","last_change":"2024-09-20T22:19:45.872368+0000","last_active":"2024-09-20T22:22:12.109574+0000","last_peered":"2024-09-20T22:22:12.109574+0000","last_clean":"2024-09-20T22:22:12.109574+0000","last_became_active":"2024-09-20T22:19:45.817931+0000","last_became_peered":"2024-09-20T22:19:45.817931+0000","last_unstale":"2024-09-20T22:22:12.109574+0000","last_undegraded":"2024-09-20T22:22:12.109574+0000","last_fullsized":"2024-09-20T22:22:12.109574+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:18:39.290911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:39.290911+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1024,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738098+0000","last_change":"2024-09-20T22:18:11.668610+0000","last_active":"2024-09-20T22:22:14.738098+0000","last_peered":"2024-09-20T22:22:14.738098+0000","last_clean":"2024-09-20T22:22:14.738098+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:22:14.738098+0000","last_undegraded":"2024-09-20T22:22:14.738098+0000","last_fullsized":"2024-09-20T22:22:14.738098+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:11.668512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:11.668512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":1098,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578609+0000","last_change":"2024-09-20T22:21:39.704522+0000","last_active":"2024-09-20T22:22:11.578609+0000","last_peered":"2024-09-20T22:22:11.578609+0000","last_clean":"2024-09-20T22:22:11.578609+0000","last_became_active":"2024-09-20T22:17:40.142028+0000","last_became_peered":"2024-09-20T22:17:40.142028+0000","last_unstale":"2024-09-20T22:22:11.578609+0000","last_undegraded":"2024-09-20T22:22:11.578609+0000","last_fullsized":"2024-09-20T22:22:11.578609+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:39.704410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:21:39.704410+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:46.432362+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"328'20","reported_seq":640,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593337+0000","last_change":"2024-09-20T22:19:45.875079+0000","last_active":"2024-09-20T22:22:16.593337+0000","last_peered":"2024-09-20T22:22:16.593337+0000","last_clean":"2024-09-20T22:22:16.593337+0000","last_became_active":"2024-09-20T22:19:45.808494+0000","last_became_peered":"2024-09-20T22:19:45.808494+0000","last_unstale":"2024-09-20T22:22:16.593337+0000","last_undegraded":"2024-09-20T22:22:16.593337+0000","last_fullsized":"2024-09-20T22:22:16.593337+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:19:08.320252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:08.320252+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1061,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593376+0000","last_change":"2024-09-20T22:19:44.874011+0000","last_active":"2024-09-20T22:22:16.593376+0000","last_peered":"2024-09-20T22:22:16.593376+0000","last_clean":"2024-09-20T22:22:16.593376+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:22:16.593376+0000","last_undegraded":"2024-09-20T22:22:16.593376+0000","last_fullsized":"2024-09-20T22:22:16.593376+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:44.873944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:19:44.873944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"363'13","reported_seq":597,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111205+0000","last_change":"2024-09-20T22:20:58.946384+0000","last_active":"2024-09-20T22:22:12.111205+0000","last_peered":"2024-09-20T22:22:12.111205+0000","last_clean":"2024-09-20T22:22:12.111205+0000","last_became_active":"2024-09-20T22:12:57.884415+0000","last_became_peered":"2024-09-20T22:12:57.884415+0000","last_unstale":"2024-09-20T22:22:12.111205+0000","last_undegraded":"2024-09-20T22:22:12.111205+0000","last_fullsized":"2024-09-20T22:22:12.111205+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"363'13","last_scrub_stamp":"2024-09-20T22:20:58.946239+0000","last_deep_scrub":"363'13","last_deep_scrub_stamp":"2024-09-20T22:14:52.612615+0000","last_clean_scrub_stamp":"2024-09-20T22:20:58.946239+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":483,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110035+0000","last_change":"2024-09-20T22:21:02.972285+0000","last_active":"2024-09-20T22:22:12.110035+0000","last_peered":"2024-09-20T22:22:12.110035+0000","last_clean":"2024-09-20T22:22:12.110035+0000","last_became_active":"2024-09-20T22:12:52.518529+0000","last_became_peered":"2024-09-20T22:12:52.518529+0000","last_unstale":"2024-09-20T22:22:12.110035+0000","last_undegraded":"2024-09-20T22:22:12.110035+0000","last_fullsized":"2024-09-20T22:22:12.110035+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:02.972172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:21:02.972172+0000","last_clean_scrub_stamp":"2024-09-20T22:21:02.972172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:27.662631+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":1099,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578278+0000","last_change":"2024-09-20T22:21:41.700453+0000","last_active":"2024-09-20T22:22:11.578278+0000","last_peered":"2024-09-20T22:22:11.578278+0000","last_clean":"2024-09-20T22:22:11.578278+0000","last_became_active":"2024-09-20T22:17:40.139979+0000","last_became_peered":"2024-09-20T22:17:40.139979+0000","last_unstale":"2024-09-20T22:22:11.578278+0000","last_undegraded":"2024-09-20T22:22:11.578278+0000","last_fullsized":"2024-09-20T22:22:11.578278+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:41.700323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:21:41.700323+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:00.974157+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"392'37","reported_seq":524,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111069+0000","last_change":"2024-09-20T22:17:38.045665+0000","last_active":"2024-09-20T22:22:12.111069+0000","last_peered":"2024-09-20T22:22:12.111069+0000","last_clean":"2024-09-20T22:22:12.111069+0000","last_became_active":"2024-09-20T22:12:52.525538+0000","last_became_peered":"2024-09-20T22:12:52.525538+0000","last_unstale":"2024-09-20T22:22:12.111069+0000","last_undegraded":"2024-09-20T22:22:12.111069+0000","last_fullsized":"2024-09-20T22:22:12.111069+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"392'37","last_scrub_stamp":"2024-09-20T22:17:38.043643+0000","last_deep_scrub":"392'37","last_deep_scrub_stamp":"2024-09-20T22:15:56.127540+0000","last_clean_scrub_stamp":"2024-09-20T22:17:38.043643+0000","objects_scrubbed":8,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":461,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111535+0000","last_change":"2024-09-20T22:19:45.807161+0000","last_active":"2024-09-20T22:22:12.111535+0000","last_peered":"2024-09-20T22:22:12.111535+0000","last_clean":"2024-09-20T22:22:12.111535+0000","last_became_active":"2024-09-20T22:19:45.806472+0000","last_became_peered":"2024-09-20T22:19:45.806472+0000","last_unstale":"2024-09-20T22:22:12.111535+0000","last_undegraded":"2024-09-20T22:22:12.111535+0000","last_fullsized":"2024-09-20T22:22:12.111535+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:43.293797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:43.293797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1067,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577984+0000","last_change":"2024-09-20T22:19:18.540268+0000","last_active":"2024-09-20T22:22:11.577984+0000","last_peered":"2024-09-20T22:22:11.577984+0000","last_clean":"2024-09-20T22:22:11.577984+0000","last_became_active":"2024-09-20T22:12:52.519826+0000","last_became_peered":"2024-09-20T22:12:52.519826+0000","last_unstale":"2024-09-20T22:22:11.577984+0000","last_undegraded":"2024-09-20T22:22:11.577984+0000","last_fullsized":"2024-09-20T22:22:11.577984+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:18.540166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:19:18.540166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":264,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577371+0000","last_change":"2024-09-20T22:21:42.695919+0000","last_active":"2024-09-20T22:22:11.577371+0000","last_peered":"2024-09-20T22:22:11.577371+0000","last_clean":"2024-09-20T22:22:11.577371+0000","last_became_active":"2024-09-20T22:19:45.791792+0000","last_became_peered":"2024-09-20T22:19:45.791792+0000","last_unstale":"2024-09-20T22:22:11.577371+0000","last_undegraded":"2024-09-20T22:22:11.577371+0000","last_fullsized":"2024-09-20T22:22:11.577371+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:42.694648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:21:42.694648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:49.660066+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":1071,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592519+0000","last_change":"2024-09-20T22:18:41.457692+0000","last_active":"2024-09-20T22:22:16.592519+0000","last_peered":"2024-09-20T22:22:16.592519+0000","last_clean":"2024-09-20T22:22:16.592519+0000","last_became_active":"2024-09-20T22:15:45.627135+0000","last_became_peered":"2024-09-20T22:15:45.627135+0000","last_unstale":"2024-09-20T22:22:16.592519+0000","last_undegraded":"2024-09-20T22:22:16.592519+0000","last_fullsized":"2024-09-20T22:22:16.592519+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:18:41.457600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:18:41.457600+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"481'53","reported_seq":768,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111443+0000","last_change":"2024-09-20T22:21:43.738362+0000","last_active":"2024-09-20T22:22:12.111443+0000","last_peered":"2024-09-20T22:22:12.111443+0000","last_clean":"2024-09-20T22:22:12.111443+0000","last_became_active":"2024-09-20T22:19:45.805924+0000","last_became_peered":"2024-09-20T22:19:45.805924+0000","last_unstale":"2024-09-20T22:22:12.111443+0000","last_undegraded":"2024-09-20T22:22:12.111443+0000","last_fullsized":"2024-09-20T22:22:12.111443+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"481'53","last_scrub_stamp":"2024-09-20T22:21:43.737956+0000","last_deep_scrub":"328'52","last_deep_scrub_stamp":"2024-09-20T22:17:08.395578+0000","last_clean_scrub_stamp":"2024-09-20T22:21:43.737956+0000","objects_scrubbed":10,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:44.477449+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":64,"num_read_kb":49,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":746,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111414+0000","last_change":"2024-09-20T22:22:06.131270+0000","last_active":"2024-09-20T22:22:12.111414+0000","last_peered":"2024-09-20T22:22:12.111414+0000","last_clean":"2024-09-20T22:22:12.111414+0000","last_became_active":"2024-09-20T22:19:45.810511+0000","last_became_peered":"2024-09-20T22:19:45.810511+0000","last_unstale":"2024-09-20T22:22:12.111414+0000","last_undegraded":"2024-09-20T22:22:12.111414+0000","last_fullsized":"2024-09-20T22:22:12.111414+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:06.131160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:06.131160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:27.704618+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":1067,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593585+0000","last_change":"2024-09-20T22:21:40.841813+0000","last_active":"2024-09-20T22:22:16.593585+0000","last_peered":"2024-09-20T22:22:16.593585+0000","last_clean":"2024-09-20T22:22:16.593585+0000","last_became_active":"2024-09-20T22:19:45.808162+0000","last_became_peered":"2024-09-20T22:19:45.808162+0000","last_unstale":"2024-09-20T22:22:16.593585+0000","last_undegraded":"2024-09-20T22:22:16.593585+0000","last_fullsized":"2024-09-20T22:22:16.593585+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:21:40.841702+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_clean_scrub_stamp":"2024-09-20T22:21:40.841702+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:56.405290+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":495,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110793+0000","last_change":"2024-09-20T22:19:45.815814+0000","last_active":"2024-09-20T22:22:12.110793+0000","last_peered":"2024-09-20T22:22:12.110793+0000","last_clean":"2024-09-20T22:22:12.110793+0000","last_became_active":"2024-09-20T22:19:45.815528+0000","last_became_peered":"2024-09-20T22:19:45.815528+0000","last_unstale":"2024-09-20T22:22:12.110793+0000","last_undegraded":"2024-09-20T22:22:12.110793+0000","last_fullsized":"2024-09-20T22:22:12.110793+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:57.210706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.210706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1046,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.655576+0000","last_change":"2024-09-20T22:22:11.655576+0000","last_active":"2024-09-20T22:22:11.655576+0000","last_peered":"2024-09-20T22:22:11.655576+0000","last_clean":"2024-09-20T22:22:11.655576+0000","last_became_active":"2024-09-20T22:22:11.655188+0000","last_became_peered":"2024-09-20T22:22:11.655188+0000","last_unstale":"2024-09-20T22:22:11.655576+0000","last_undegraded":"2024-09-20T22:22:11.655576+0000","last_fullsized":"2024-09-20T22:22:11.655576+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:43.806828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:19:43.806828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":1076,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737550+0000","last_change":"2024-09-20T22:19:31.274033+0000","last_active":"2024-09-20T22:22:14.737550+0000","last_peered":"2024-09-20T22:22:14.737550+0000","last_clean":"2024-09-20T22:22:14.737550+0000","last_became_active":"2024-09-20T22:19:31.273654+0000","last_became_peered":"2024-09-20T22:19:31.273654+0000","last_unstale":"2024-09-20T22:22:14.737550+0000","last_undegraded":"2024-09-20T22:22:14.737550+0000","last_fullsized":"2024-09-20T22:22:14.737550+0000","mapping_epoch":475,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":476,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:38.946300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:38.946300+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"328'17","reported_seq":651,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593975+0000","last_change":"2024-09-20T22:21:33.920405+0000","last_active":"2024-09-20T22:22:16.593975+0000","last_peered":"2024-09-20T22:22:16.593975+0000","last_clean":"2024-09-20T22:22:16.593975+0000","last_became_active":"2024-09-20T22:19:45.810715+0000","last_became_peered":"2024-09-20T22:19:45.810715+0000","last_unstale":"2024-09-20T22:22:16.593975+0000","last_undegraded":"2024-09-20T22:22:16.593975+0000","last_fullsized":"2024-09-20T22:22:16.593975+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'17","last_scrub_stamp":"2024-09-20T22:21:33.920310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:21:33.920310+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:41.502954+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":23,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":265,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.109853+0000","last_change":"2024-09-20T22:21:42.696816+0000","last_active":"2024-09-20T22:22:12.109853+0000","last_peered":"2024-09-20T22:22:12.109853+0000","last_clean":"2024-09-20T22:22:12.109853+0000","last_became_active":"2024-09-20T22:19:45.807340+0000","last_became_peered":"2024-09-20T22:19:45.807340+0000","last_unstale":"2024-09-20T22:22:12.109853+0000","last_undegraded":"2024-09-20T22:22:12.109853+0000","last_fullsized":"2024-09-20T22:22:12.109853+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:42.696690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:21:42.696690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:11.856661+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":1101,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110226+0000","last_change":"2024-09-20T22:19:13.810454+0000","last_active":"2024-09-20T22:22:12.110226+0000","last_peered":"2024-09-20T22:22:12.110226+0000","last_clean":"2024-09-20T22:22:12.110226+0000","last_became_active":"2024-09-20T22:12:52.517255+0000","last_became_peered":"2024-09-20T22:12:52.517255+0000","last_unstale":"2024-09-20T22:22:12.110226+0000","last_undegraded":"2024-09-20T22:22:12.110226+0000","last_fullsized":"2024-09-20T22:22:12.110226+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:19:13.810352+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:19:13.810352+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1054,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110492+0000","last_change":"2024-09-20T22:19:08.895449+0000","last_active":"2024-09-20T22:22:12.110492+0000","last_peered":"2024-09-20T22:22:12.110492+0000","last_clean":"2024-09-20T22:22:12.110492+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:22:12.110492+0000","last_undegraded":"2024-09-20T22:22:12.110492+0000","last_fullsized":"2024-09-20T22:22:12.110492+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:08.894168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:19:08.894168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1024,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738708+0000","last_change":"2024-09-20T22:18:35.868334+0000","last_active":"2024-09-20T22:22:14.738708+0000","last_peered":"2024-09-20T22:22:14.738708+0000","last_clean":"2024-09-20T22:22:14.738708+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:22:14.738708+0000","last_undegraded":"2024-09-20T22:22:14.738708+0000","last_fullsized":"2024-09-20T22:22:14.738708+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:35.868264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:35.868264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":441,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577617+0000","last_change":"2024-09-20T22:21:33.763221+0000","last_active":"2024-09-20T22:22:11.577617+0000","last_peered":"2024-09-20T22:22:11.577617+0000","last_clean":"2024-09-20T22:22:11.577617+0000","last_became_active":"2024-09-20T22:19:48.843794+0000","last_became_peered":"2024-09-20T22:19:48.843794+0000","last_unstale":"2024-09-20T22:22:11.577617+0000","last_undegraded":"2024-09-20T22:22:11.577617+0000","last_fullsized":"2024-09-20T22:22:11.577617+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:33.763116+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:21:33.763116+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:52.572140+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"328'24","reported_seq":543,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111879+0000","last_change":"2024-09-20T22:22:03.105751+0000","last_active":"2024-09-20T22:22:12.111879+0000","last_peered":"2024-09-20T22:22:12.111879+0000","last_clean":"2024-09-20T22:22:12.111879+0000","last_became_active":"2024-09-20T22:19:45.819406+0000","last_became_peered":"2024-09-20T22:19:45.819406+0000","last_unstale":"2024-09-20T22:22:12.111879+0000","last_undegraded":"2024-09-20T22:22:12.111879+0000","last_fullsized":"2024-09-20T22:22:12.111879+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'24","last_scrub_stamp":"2024-09-20T22:22:03.105662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:03.105662+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:10.038202+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":497,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110185+0000","last_change":"2024-09-20T22:22:04.127512+0000","last_active":"2024-09-20T22:22:12.110185+0000","last_peered":"2024-09-20T22:22:12.110185+0000","last_clean":"2024-09-20T22:22:12.110185+0000","last_became_active":"2024-09-20T22:17:03.663870+0000","last_became_peered":"2024-09-20T22:17:03.663870+0000","last_unstale":"2024-09-20T22:22:12.110185+0000","last_undegraded":"2024-09-20T22:22:12.110185+0000","last_fullsized":"2024-09-20T22:22:12.110185+0000","mapping_epoch":415,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":416,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:04.127443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:22:04.127443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:23.980359+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":2233,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737618+0000","last_change":"2024-09-20T22:19:45.875808+0000","last_active":"2024-09-20T22:22:14.737618+0000","last_peered":"2024-09-20T22:22:14.737618+0000","last_clean":"2024-09-20T22:22:14.737618+0000","last_became_active":"2024-09-20T22:19:45.817469+0000","last_became_peered":"2024-09-20T22:19:45.817469+0000","last_unstale":"2024-09-20T22:22:14.737618+0000","last_undegraded":"2024-09-20T22:22:14.737618+0000","last_fullsized":"2024-09-20T22:22:14.737618+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:17:59.765217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:17:59.765217+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1137,"num_read_kb":1132,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":597,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593767+0000","last_change":"2024-09-20T22:22:06.156934+0000","last_active":"2024-09-20T22:22:16.593767+0000","last_peered":"2024-09-20T22:22:16.593767+0000","last_clean":"2024-09-20T22:22:16.593767+0000","last_became_active":"2024-09-20T22:19:45.810608+0000","last_became_peered":"2024-09-20T22:19:45.810608+0000","last_unstale":"2024-09-20T22:22:16.593767+0000","last_undegraded":"2024-09-20T22:22:16.593767+0000","last_fullsized":"2024-09-20T22:22:16.593767+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:06.156848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:17:08.955790+0000","last_clean_scrub_stamp":"2024-09-20T22:22:06.156848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:22.327333+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":262,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593428+0000","last_change":"2024-09-20T22:19:45.790117+0000","last_active":"2024-09-20T22:22:16.593428+0000","last_peered":"2024-09-20T22:22:16.593428+0000","last_clean":"2024-09-20T22:22:16.593428+0000","last_became_active":"2024-09-20T22:19:45.789135+0000","last_became_peered":"2024-09-20T22:19:45.789135+0000","last_unstale":"2024-09-20T22:22:16.593428+0000","last_undegraded":"2024-09-20T22:22:16.593428+0000","last_fullsized":"2024-09-20T22:22:16.593428+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:40.325308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:40.325308+0000","last_clean_scrub_stamp":"2024-09-20T22:18:40.325308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"475'29","reported_seq":1340,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578723+0000","last_change":"2024-09-20T22:19:28.285619+0000","last_active":"2024-09-20T22:22:11.578723+0000","last_peered":"2024-09-20T22:22:11.578723+0000","last_clean":"2024-09-20T22:22:11.578723+0000","last_became_active":"2024-09-20T22:19:28.259634+0000","last_became_peered":"2024-09-20T22:19:28.259634+0000","last_unstale":"2024-09-20T22:22:11.578723+0000","last_undegraded":"2024-09-20T22:22:11.578723+0000","last_fullsized":"2024-09-20T22:22:11.578723+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:17:57.685068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.685068+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":997,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592596+0000","last_change":"2024-09-20T22:19:11.606709+0000","last_active":"2024-09-20T22:22:16.592596+0000","last_peered":"2024-09-20T22:22:16.592596+0000","last_clean":"2024-09-20T22:22:16.592596+0000","last_became_active":"2024-09-20T22:12:21.519717+0000","last_became_peered":"2024-09-20T22:12:21.519717+0000","last_unstale":"2024-09-20T22:22:16.592596+0000","last_undegraded":"2024-09-20T22:22:16.592596+0000","last_fullsized":"2024-09-20T22:22:16.592596+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:11.606601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:36.729158+0000","last_clean_scrub_stamp":"2024-09-20T22:19:11.606601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":1066,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737173+0000","last_change":"2024-09-20T22:18:06.776685+0000","last_active":"2024-09-20T22:22:14.737173+0000","last_peered":"2024-09-20T22:22:14.737173+0000","last_clean":"2024-09-20T22:22:14.737173+0000","last_became_active":"2024-09-20T22:15:45.620972+0000","last_became_peered":"2024-09-20T22:15:45.620972+0000","last_unstale":"2024-09-20T22:22:14.737173+0000","last_undegraded":"2024-09-20T22:22:14.737173+0000","last_fullsized":"2024-09-20T22:22:14.737173+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:18:06.776546+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:18:06.776546+0000","last_clean_scrub_stamp":"2024-09-20T22:18:06.776546+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":618,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738764+0000","last_change":"2024-09-20T22:18:04.809849+0000","last_active":"2024-09-20T22:22:14.738764+0000","last_peered":"2024-09-20T22:22:14.738764+0000","last_clean":"2024-09-20T22:22:14.738764+0000","last_became_active":"2024-09-20T22:10:48.907271+0000","last_became_peered":"2024-09-20T22:10:48.907271+0000","last_unstale":"2024-09-20T22:22:14.738764+0000","last_undegraded":"2024-09-20T22:22:14.738764+0000","last_fullsized":"2024-09-20T22:22:14.738764+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:04.809565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_clean_scrub_stamp":"2024-09-20T22:18:04.809565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1064,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593877+0000","last_change":"2024-09-20T22:19:45.799513+0000","last_active":"2024-09-20T22:22:16.593877+0000","last_peered":"2024-09-20T22:22:16.593877+0000","last_clean":"2024-09-20T22:22:16.593877+0000","last_became_active":"2024-09-20T22:19:45.797853+0000","last_became_peered":"2024-09-20T22:19:45.797853+0000","last_unstale":"2024-09-20T22:22:16.593877+0000","last_undegraded":"2024-09-20T22:22:16.593877+0000","last_fullsized":"2024-09-20T22:22:16.593877+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:13.654339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:19:13.654339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"75'128","reported_seq":855,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111504+0000","last_change":"2024-09-20T22:20:59.964237+0000","last_active":"2024-09-20T22:22:12.111504+0000","last_peered":"2024-09-20T22:22:12.111504+0000","last_clean":"2024-09-20T22:22:12.111504+0000","last_became_active":"2024-09-20T22:19:48.848609+0000","last_became_peered":"2024-09-20T22:19:48.848609+0000","last_unstale":"2024-09-20T22:22:12.111504+0000","last_undegraded":"2024-09-20T22:22:12.111504+0000","last_fullsized":"2024-09-20T22:22:12.111504+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:20:59.964107+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:20:59.964107+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"328'19","reported_seq":790,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578552+0000","last_change":"2024-09-20T22:19:45.806692+0000","last_active":"2024-09-20T22:22:11.578552+0000","last_peered":"2024-09-20T22:22:11.578552+0000","last_clean":"2024-09-20T22:22:11.578552+0000","last_became_active":"2024-09-20T22:19:45.786395+0000","last_became_peered":"2024-09-20T22:19:45.786395+0000","last_unstale":"2024-09-20T22:22:11.578552+0000","last_undegraded":"2024-09-20T22:22:11.578552+0000","last_fullsized":"2024-09-20T22:22:11.578552+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:18:36.321653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:36.321653+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":173,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110054+0000","last_change":"2024-09-20T22:22:08.966893+0000","last_active":"2024-09-20T22:22:12.110054+0000","last_peered":"2024-09-20T22:22:12.110054+0000","last_clean":"2024-09-20T22:22:12.110054+0000","last_became_active":"2024-09-20T22:19:45.800297+0000","last_became_peered":"2024-09-20T22:19:45.800297+0000","last_unstale":"2024-09-20T22:22:12.110054+0000","last_undegraded":"2024-09-20T22:22:12.110054+0000","last_fullsized":"2024-09-20T22:22:12.110054+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:08.966822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:08.966822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:23.677528+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1050,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593985+0000","last_change":"2024-09-20T22:19:28.241963+0000","last_active":"2024-09-20T22:22:16.593985+0000","last_peered":"2024-09-20T22:22:16.593985+0000","last_clean":"2024-09-20T22:22:16.593985+0000","last_became_active":"2024-09-20T22:19:28.241278+0000","last_became_peered":"2024-09-20T22:19:28.241278+0000","last_unstale":"2024-09-20T22:22:16.593985+0000","last_undegraded":"2024-09-20T22:22:16.593985+0000","last_fullsized":"2024-09-20T22:22:16.593985+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:04.545147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:04.545147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":986,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578750+0000","last_change":"2024-09-20T22:17:55.630149+0000","last_active":"2024-09-20T22:22:11.578750+0000","last_peered":"2024-09-20T22:22:11.578750+0000","last_clean":"2024-09-20T22:22:11.578750+0000","last_became_active":"2024-09-20T22:12:21.521195+0000","last_became_peered":"2024-09-20T22:12:21.521195+0000","last_unstale":"2024-09-20T22:22:11.578750+0000","last_undegraded":"2024-09-20T22:22:11.578750+0000","last_fullsized":"2024-09-20T22:22:11.578750+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:55.630084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:55.630084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":821,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577167+0000","last_change":"2024-09-20T22:21:43.651683+0000","last_active":"2024-09-20T22:22:11.577167+0000","last_peered":"2024-09-20T22:22:11.577167+0000","last_clean":"2024-09-20T22:22:11.577167+0000","last_became_active":"2024-09-20T22:17:40.139875+0000","last_became_peered":"2024-09-20T22:17:40.139875+0000","last_unstale":"2024-09-20T22:22:11.577167+0000","last_undegraded":"2024-09-20T22:22:11.577167+0000","last_fullsized":"2024-09-20T22:22:11.577167+0000","mapping_epoch":358,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:43.651464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:21:43.651464+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:52.773176+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"328'31","reported_seq":820,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111526+0000","last_change":"2024-09-20T22:19:45.976627+0000","last_active":"2024-09-20T22:22:12.111526+0000","last_peered":"2024-09-20T22:22:12.111526+0000","last_clean":"2024-09-20T22:22:12.111526+0000","last_became_active":"2024-09-20T22:19:45.833530+0000","last_became_peered":"2024-09-20T22:19:45.833530+0000","last_unstale":"2024-09-20T22:22:12.111526+0000","last_undegraded":"2024-09-20T22:22:12.111526+0000","last_fullsized":"2024-09-20T22:22:12.111526+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'31","last_scrub_stamp":"2024-09-20T22:19:12.269851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:12.269851+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1067,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578430+0000","last_change":"2024-09-20T22:19:45.798910+0000","last_active":"2024-09-20T22:22:11.578430+0000","last_peered":"2024-09-20T22:22:11.578430+0000","last_clean":"2024-09-20T22:22:11.578430+0000","last_became_active":"2024-09-20T22:19:45.798363+0000","last_became_peered":"2024-09-20T22:19:45.798363+0000","last_unstale":"2024-09-20T22:22:11.578430+0000","last_undegraded":"2024-09-20T22:22:11.578430+0000","last_fullsized":"2024-09-20T22:22:11.578430+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:12.437986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:17:08.846904+0000","last_clean_scrub_stamp":"2024-09-20T22:19:12.437986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":873,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110315+0000","last_change":"2024-09-20T22:19:45.809454+0000","last_active":"2024-09-20T22:22:12.110315+0000","last_peered":"2024-09-20T22:22:12.110315+0000","last_clean":"2024-09-20T22:22:12.110315+0000","last_became_active":"2024-09-20T22:19:45.808291+0000","last_became_peered":"2024-09-20T22:19:45.808291+0000","last_unstale":"2024-09-20T22:22:12.110315+0000","last_undegraded":"2024-09-20T22:22:12.110315+0000","last_fullsized":"2024-09-20T22:22:12.110315+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:10.179939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:10.179939+0000","last_clean_scrub_stamp":"2024-09-20T22:18:10.179939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1056,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110961+0000","last_change":"2024-09-20T22:22:10.991785+0000","last_active":"2024-09-20T22:22:12.110961+0000","last_peered":"2024-09-20T22:22:12.110961+0000","last_clean":"2024-09-20T22:22:12.110961+0000","last_became_active":"2024-09-20T22:19:28.261628+0000","last_became_peered":"2024-09-20T22:19:28.261628+0000","last_unstale":"2024-09-20T22:22:12.110961+0000","last_undegraded":"2024-09-20T22:22:12.110961+0000","last_fullsized":"2024-09-20T22:22:12.110961+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:10.991691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:10.991691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:15.249186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":1328,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737672+0000","last_change":"2024-09-20T22:17:42.025688+0000","last_active":"2024-09-20T22:22:14.737672+0000","last_peered":"2024-09-20T22:22:14.737672+0000","last_clean":"2024-09-20T22:22:14.737672+0000","last_became_active":"2024-09-20T22:17:40.146794+0000","last_became_peered":"2024-09-20T22:17:40.146794+0000","last_unstale":"2024-09-20T22:22:14.737672+0000","last_undegraded":"2024-09-20T22:22:14.737672+0000","last_fullsized":"2024-09-20T22:22:14.737672+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:42.025576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:42.025576+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"328'25","reported_seq":793,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.703568+0000","last_change":"2024-09-20T22:22:12.703600+0000","last_active":"2024-09-20T22:22:12.703568+0000","last_peered":"2024-09-20T22:22:12.703568+0000","last_clean":"2024-09-20T22:22:12.703568+0000","last_became_active":"2024-09-20T22:19:45.791116+0000","last_became_peered":"2024-09-20T22:19:45.791116+0000","last_unstale":"2024-09-20T22:22:12.703568+0000","last_undegraded":"2024-09-20T22:22:12.703568+0000","last_fullsized":"2024-09-20T22:22:12.703568+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'25","last_scrub_stamp":"2024-09-20T22:22:12.703492+0000","last_deep_scrub":"328'25","last_deep_scrub_stamp":"2024-09-20T22:18:05.300862+0000","last_clean_scrub_stamp":"2024-09-20T22:22:12.703492+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:34.704284+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":507,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111106+0000","last_change":"2024-09-20T22:19:45.864178+0000","last_active":"2024-09-20T22:22:12.111106+0000","last_peered":"2024-09-20T22:22:12.111106+0000","last_clean":"2024-09-20T22:22:12.111106+0000","last_became_active":"2024-09-20T22:19:45.810979+0000","last_became_peered":"2024-09-20T22:19:45.810979+0000","last_unstale":"2024-09-20T22:22:12.111106+0000","last_undegraded":"2024-09-20T22:22:12.111106+0000","last_fullsized":"2024-09-20T22:22:12.111106+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:17:35.054116+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:17:35.054116+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"283'11","reported_seq":1324,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111427+0000","last_change":"2024-09-20T22:19:11.823449+0000","last_active":"2024-09-20T22:22:12.111427+0000","last_peered":"2024-09-20T22:22:12.111427+0000","last_clean":"2024-09-20T22:22:12.111427+0000","last_became_active":"2024-09-20T22:11:30.485356+0000","last_became_peered":"2024-09-20T22:11:30.485356+0000","last_unstale":"2024-09-20T22:22:12.111427+0000","last_undegraded":"2024-09-20T22:22:12.111427+0000","last_fullsized":"2024-09-20T22:22:12.111427+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'11","last_scrub_stamp":"2024-09-20T22:19:11.823322+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:19:11.823322+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":309,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110283+0000","last_change":"2024-09-20T22:19:45.804591+0000","last_active":"2024-09-20T22:22:12.110283+0000","last_peered":"2024-09-20T22:22:12.110283+0000","last_clean":"2024-09-20T22:22:12.110283+0000","last_became_active":"2024-09-20T22:19:45.799390+0000","last_became_peered":"2024-09-20T22:19:45.799390+0000","last_unstale":"2024-09-20T22:22:12.110283+0000","last_undegraded":"2024-09-20T22:22:12.110283+0000","last_fullsized":"2024-09-20T22:22:12.110283+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:11.319469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:19:11.319469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.e","version":"75'128","reported_seq":750,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.739440+0000","last_change":"2024-09-20T22:21:35.700257+0000","last_active":"2024-09-20T22:22:14.739440+0000","last_peered":"2024-09-20T22:22:14.739440+0000","last_clean":"2024-09-20T22:22:14.739440+0000","last_became_active":"2024-09-20T22:19:31.258800+0000","last_became_peered":"2024-09-20T22:19:31.258800+0000","last_unstale":"2024-09-20T22:22:14.739440+0000","last_undegraded":"2024-09-20T22:22:14.739440+0000","last_fullsized":"2024-09-20T22:22:14.739440+0000","mapping_epoch":475,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":476,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:35.700020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:21:35.700020+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:50.597949+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"512'19","reported_seq":546,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110971+0000","last_change":"2024-09-20T22:21:05.960413+0000","last_active":"2024-09-20T22:22:12.110971+0000","last_peered":"2024-09-20T22:22:12.110971+0000","last_clean":"2024-09-20T22:22:12.110971+0000","last_became_active":"2024-09-20T22:19:45.820510+0000","last_became_peered":"2024-09-20T22:19:45.820510+0000","last_unstale":"2024-09-20T22:22:12.110971+0000","last_undegraded":"2024-09-20T22:22:12.110971+0000","last_fullsized":"2024-09-20T22:22:12.110971+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"512'19","last_scrub_stamp":"2024-09-20T22:21:05.960284+0000","last_deep_scrub":"512'19","last_deep_scrub_stamp":"2024-09-20T22:21:05.960284+0000","last_clean_scrub_stamp":"2024-09-20T22:21:05.960284+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:20.601201+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":152,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593297+0000","last_change":"2024-09-20T22:19:45.801745+0000","last_active":"2024-09-20T22:22:16.593297+0000","last_peered":"2024-09-20T22:22:16.593297+0000","last_clean":"2024-09-20T22:22:16.593297+0000","last_became_active":"2024-09-20T22:19:45.788623+0000","last_became_peered":"2024-09-20T22:19:45.788623+0000","last_unstale":"2024-09-20T22:22:16.593297+0000","last_undegraded":"2024-09-20T22:22:16.593297+0000","last_fullsized":"2024-09-20T22:22:16.593297+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:19:37.837883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:19:37.837883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1060,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110810+0000","last_change":"2024-09-20T22:19:28.247348+0000","last_active":"2024-09-20T22:22:12.110810+0000","last_peered":"2024-09-20T22:22:12.110810+0000","last_clean":"2024-09-20T22:22:12.110810+0000","last_became_active":"2024-09-20T22:19:28.246512+0000","last_became_peered":"2024-09-20T22:19:28.246512+0000","last_unstale":"2024-09-20T22:22:12.110810+0000","last_undegraded":"2024-09-20T22:22:12.110810+0000","last_fullsized":"2024-09-20T22:22:12.110810+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:40.088194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:18:40.088194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":489,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110096+0000","last_change":"2024-09-20T22:19:45.813316+0000","last_active":"2024-09-20T22:22:12.110096+0000","last_peered":"2024-09-20T22:22:12.110096+0000","last_clean":"2024-09-20T22:22:12.110096+0000","last_became_active":"2024-09-20T22:19:45.813003+0000","last_became_peered":"2024-09-20T22:19:45.813003+0000","last_unstale":"2024-09-20T22:22:12.110096+0000","last_undegraded":"2024-09-20T22:22:12.110096+0000","last_fullsized":"2024-09-20T22:22:12.110096+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:12.806158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:19:12.806158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"75'128","reported_seq":1101,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578893+0000","last_change":"2024-09-20T22:19:31.262868+0000","last_active":"2024-09-20T22:22:11.578893+0000","last_peered":"2024-09-20T22:22:11.578893+0000","last_clean":"2024-09-20T22:22:11.578893+0000","last_became_active":"2024-09-20T22:19:31.262445+0000","last_became_peered":"2024-09-20T22:19:31.262445+0000","last_unstale":"2024-09-20T22:22:11.578893+0000","last_undegraded":"2024-09-20T22:22:11.578893+0000","last_fullsized":"2024-09-20T22:22:11.578893+0000","mapping_epoch":475,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":476,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:11.475678+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:17:07.885346+0000","last_clean_scrub_stamp":"2024-09-20T22:19:11.475678+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"328'35","reported_seq":1176,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578146+0000","last_change":"2024-09-20T22:19:45.813869+0000","last_active":"2024-09-20T22:22:11.578146+0000","last_peered":"2024-09-20T22:22:11.578146+0000","last_clean":"2024-09-20T22:22:11.578146+0000","last_became_active":"2024-09-20T22:19:45.790684+0000","last_became_peered":"2024-09-20T22:19:45.790684+0000","last_unstale":"2024-09-20T22:22:11.578146+0000","last_undegraded":"2024-09-20T22:22:11.578146+0000","last_fullsized":"2024-09-20T22:22:11.578146+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'35","last_scrub_stamp":"2024-09-20T22:19:16.424262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:16.424262+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1047,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738865+0000","last_change":"2024-09-20T22:17:37.950739+0000","last_active":"2024-09-20T22:22:14.738865+0000","last_peered":"2024-09-20T22:22:14.738865+0000","last_clean":"2024-09-20T22:22:14.738865+0000","last_became_active":"2024-09-20T22:11:30.488650+0000","last_became_peered":"2024-09-20T22:11:30.488650+0000","last_unstale":"2024-09-20T22:22:14.738865+0000","last_undegraded":"2024-09-20T22:22:14.738865+0000","last_fullsized":"2024-09-20T22:22:14.738865+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:37.950674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:12:46.602035+0000","last_clean_scrub_stamp":"2024-09-20T22:17:37.950674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1078,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110673+0000","last_change":"2024-09-20T22:18:11.941197+0000","last_active":"2024-09-20T22:22:12.110673+0000","last_peered":"2024-09-20T22:22:12.110673+0000","last_clean":"2024-09-20T22:22:12.110673+0000","last_became_active":"2024-09-20T22:12:52.515295+0000","last_became_peered":"2024-09-20T22:12:52.515295+0000","last_unstale":"2024-09-20T22:22:12.110673+0000","last_undegraded":"2024-09-20T22:22:12.110673+0000","last_fullsized":"2024-09-20T22:22:12.110673+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:11.941075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_clean_scrub_stamp":"2024-09-20T22:18:11.941075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":640,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737214+0000","last_change":"2024-09-20T22:18:44.881893+0000","last_active":"2024-09-20T22:22:14.737214+0000","last_peered":"2024-09-20T22:22:14.737214+0000","last_clean":"2024-09-20T22:22:14.737214+0000","last_became_active":"2024-09-20T22:10:30.657116+0000","last_became_peered":"2024-09-20T22:10:30.657116+0000","last_unstale":"2024-09-20T22:22:14.737214+0000","last_undegraded":"2024-09-20T22:22:14.737214+0000","last_fullsized":"2024-09-20T22:22:14.737214+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.881819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.881819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"75'128","reported_seq":1124,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577194+0000","last_change":"2024-09-20T22:22:04.994705+0000","last_active":"2024-09-20T22:22:11.577194+0000","last_peered":"2024-09-20T22:22:11.577194+0000","last_clean":"2024-09-20T22:22:11.577194+0000","last_became_active":"2024-09-20T22:17:40.152364+0000","last_became_peered":"2024-09-20T22:17:40.152364+0000","last_unstale":"2024-09-20T22:22:11.577194+0000","last_undegraded":"2024-09-20T22:22:11.577194+0000","last_fullsized":"2024-09-20T22:22:11.577194+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:04.994620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:04.994620+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:20.725429+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"418'23","reported_seq":960,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578568+0000","last_change":"2024-09-20T22:21:36.956588+0000","last_active":"2024-09-20T22:22:11.578568+0000","last_peered":"2024-09-20T22:22:11.578568+0000","last_clean":"2024-09-20T22:22:11.578568+0000","last_became_active":"2024-09-20T22:19:45.816632+0000","last_became_peered":"2024-09-20T22:19:45.816632+0000","last_unstale":"2024-09-20T22:22:11.578568+0000","last_undegraded":"2024-09-20T22:22:11.578568+0000","last_fullsized":"2024-09-20T22:22:11.578568+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"418'23","last_scrub_stamp":"2024-09-20T22:21:36.956465+0000","last_deep_scrub":"418'23","last_deep_scrub_stamp":"2024-09-20T22:17:11.338821+0000","last_clean_scrub_stamp":"2024-09-20T22:21:36.956465+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:05.245014+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1056,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.739023+0000","last_change":"2024-09-20T22:19:45.858377+0000","last_active":"2024-09-20T22:22:14.739023+0000","last_peered":"2024-09-20T22:22:14.739023+0000","last_clean":"2024-09-20T22:22:14.739023+0000","last_became_active":"2024-09-20T22:19:45.858043+0000","last_became_peered":"2024-09-20T22:19:45.858043+0000","last_unstale":"2024-09-20T22:22:14.739023+0000","last_undegraded":"2024-09-20T22:22:14.739023+0000","last_fullsized":"2024-09-20T22:22:14.739023+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:38.817089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:38.817089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,5],"acting":[7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1081,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.109743+0000","last_change":"2024-09-20T22:19:45.806148+0000","last_active":"2024-09-20T22:22:12.109743+0000","last_peered":"2024-09-20T22:22:12.109743+0000","last_clean":"2024-09-20T22:22:12.109743+0000","last_became_active":"2024-09-20T22:19:45.802808+0000","last_became_peered":"2024-09-20T22:19:45.802808+0000","last_unstale":"2024-09-20T22:22:12.109743+0000","last_undegraded":"2024-09-20T22:22:12.109743+0000","last_fullsized":"2024-09-20T22:22:12.109743+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:03.959875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:03.959875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":515,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.576907+0000","last_change":"2024-09-20T22:22:03.002505+0000","last_active":"2024-09-20T22:22:11.576907+0000","last_peered":"2024-09-20T22:22:11.576907+0000","last_clean":"2024-09-20T22:22:11.576907+0000","last_became_active":"2024-09-20T22:19:45.798815+0000","last_became_peered":"2024-09-20T22:19:45.798815+0000","last_unstale":"2024-09-20T22:22:11.576907+0000","last_undegraded":"2024-09-20T22:22:11.576907+0000","last_fullsized":"2024-09-20T22:22:11.576907+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:03.002391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:03.002391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:25.958684+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'128","reported_seq":667,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738401+0000","last_change":"2024-09-20T22:19:48.871040+0000","last_active":"2024-09-20T22:22:14.738401+0000","last_peered":"2024-09-20T22:22:14.738401+0000","last_clean":"2024-09-20T22:22:14.738401+0000","last_became_active":"2024-09-20T22:19:48.870633+0000","last_became_peered":"2024-09-20T22:19:48.870633+0000","last_unstale":"2024-09-20T22:22:14.738401+0000","last_undegraded":"2024-09-20T22:22:14.738401+0000","last_fullsized":"2024-09-20T22:22:14.738401+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:40.996791+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:17:40.996791+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"416'34","reported_seq":612,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592868+0000","last_change":"2024-09-20T22:22:04.107105+0000","last_active":"2024-09-20T22:22:16.592868+0000","last_peered":"2024-09-20T22:22:16.592868+0000","last_clean":"2024-09-20T22:22:16.592868+0000","last_became_active":"2024-09-20T22:11:30.493275+0000","last_became_peered":"2024-09-20T22:11:30.493275+0000","last_unstale":"2024-09-20T22:22:16.592868+0000","last_undegraded":"2024-09-20T22:22:16.592868+0000","last_fullsized":"2024-09-20T22:22:16.592868+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"416'34","last_scrub_stamp":"2024-09-20T22:22:04.106984+0000","last_deep_scrub":"416'34","last_deep_scrub_stamp":"2024-09-20T22:17:04.960380+0000","last_clean_scrub_stamp":"2024-09-20T22:22:04.106984+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:18.483370+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":694,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738988+0000","last_change":"2024-09-20T22:21:37.623111+0000","last_active":"2024-09-20T22:22:14.738988+0000","last_peered":"2024-09-20T22:22:14.738988+0000","last_clean":"2024-09-20T22:22:14.738988+0000","last_became_active":"2024-09-20T22:19:45.818725+0000","last_became_peered":"2024-09-20T22:19:45.818725+0000","last_unstale":"2024-09-20T22:22:14.738988+0000","last_undegraded":"2024-09-20T22:22:14.738988+0000","last_fullsized":"2024-09-20T22:22:14.738988+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:21:37.623026+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:21:37.623026+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:06.964973+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":460,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,5],"acting":[7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"481'10","reported_seq":1289,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737394+0000","last_change":"2024-09-20T22:19:45.804947+0000","last_active":"2024-09-20T22:22:14.737394+0000","last_peered":"2024-09-20T22:22:14.737394+0000","last_clean":"2024-09-20T22:22:14.737394+0000","last_became_active":"2024-09-20T22:19:45.793150+0000","last_became_peered":"2024-09-20T22:19:45.793150+0000","last_unstale":"2024-09-20T22:22:14.737394+0000","last_undegraded":"2024-09-20T22:22:14.737394+0000","last_fullsized":"2024-09-20T22:22:14.737394+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:19:16.895874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:16.895874+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":583,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593439+0000","last_change":"2024-09-20T22:19:09.620194+0000","last_active":"2024-09-20T22:22:16.593439+0000","last_peered":"2024-09-20T22:22:16.593439+0000","last_clean":"2024-09-20T22:22:16.593439+0000","last_became_active":"2024-09-20T22:12:52.517758+0000","last_became_peered":"2024-09-20T22:12:52.517758+0000","last_unstale":"2024-09-20T22:22:16.593439+0000","last_undegraded":"2024-09-20T22:22:16.593439+0000","last_fullsized":"2024-09-20T22:22:16.593439+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:09.620092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:19:09.620092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"75'128","reported_seq":932,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592697+0000","last_change":"2024-09-20T22:19:48.841486+0000","last_active":"2024-09-20T22:22:16.592697+0000","last_peered":"2024-09-20T22:22:16.592697+0000","last_clean":"2024-09-20T22:22:16.592697+0000","last_became_active":"2024-09-20T22:19:48.841087+0000","last_became_peered":"2024-09-20T22:19:48.841087+0000","last_unstale":"2024-09-20T22:22:16.592697+0000","last_undegraded":"2024-09-20T22:22:16.592697+0000","last_fullsized":"2024-09-20T22:22:16.592697+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:06.524269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:19:06.524269+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1078,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577770+0000","last_change":"2024-09-20T22:18:00.786710+0000","last_active":"2024-09-20T22:22:11.577770+0000","last_peered":"2024-09-20T22:22:11.577770+0000","last_clean":"2024-09-20T22:22:11.577770+0000","last_became_active":"2024-09-20T22:12:52.516989+0000","last_became_peered":"2024-09-20T22:12:52.516989+0000","last_unstale":"2024-09-20T22:22:11.577770+0000","last_undegraded":"2024-09-20T22:22:11.577770+0000","last_fullsized":"2024-09-20T22:22:11.577770+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:00.786604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:00.786604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"328'12","reported_seq":1110,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577611+0000","last_change":"2024-09-20T22:22:11.002144+0000","last_active":"2024-09-20T22:22:11.577611+0000","last_peered":"2024-09-20T22:22:11.577611+0000","last_clean":"2024-09-20T22:22:11.577611+0000","last_became_active":"2024-09-20T22:19:45.798694+0000","last_became_peered":"2024-09-20T22:19:45.798694+0000","last_unstale":"2024-09-20T22:22:11.577611+0000","last_undegraded":"2024-09-20T22:22:11.577611+0000","last_fullsized":"2024-09-20T22:22:11.577611+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'12","last_scrub_stamp":"2024-09-20T22:22:11.002033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:11.002033+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:37.055926+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":577,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593750+0000","last_change":"2024-09-20T22:19:45.789577+0000","last_active":"2024-09-20T22:22:16.593750+0000","last_peered":"2024-09-20T22:22:16.593750+0000","last_clean":"2024-09-20T22:22:16.593750+0000","last_became_active":"2024-09-20T22:19:45.789220+0000","last_became_peered":"2024-09-20T22:19:45.789220+0000","last_unstale":"2024-09-20T22:22:16.593750+0000","last_undegraded":"2024-09-20T22:22:16.593750+0000","last_fullsized":"2024-09-20T22:22:16.593750+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:36.439030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:36.439030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1029,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738660+0000","last_change":"2024-09-20T22:22:06.626292+0000","last_active":"2024-09-20T22:22:14.738660+0000","last_peered":"2024-09-20T22:22:14.738660+0000","last_clean":"2024-09-20T22:22:14.738660+0000","last_became_active":"2024-09-20T22:19:28.249892+0000","last_became_peered":"2024-09-20T22:19:28.249892+0000","last_unstale":"2024-09-20T22:22:14.738660+0000","last_undegraded":"2024-09-20T22:22:14.738660+0000","last_fullsized":"2024-09-20T22:22:14.738660+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:06.626203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:54.359586+0000","last_clean_scrub_stamp":"2024-09-20T22:22:06.626203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:32.454372+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"75'128","reported_seq":855,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577189+0000","last_change":"2024-09-20T22:21:34.911974+0000","last_active":"2024-09-20T22:22:11.577189+0000","last_peered":"2024-09-20T22:22:11.577189+0000","last_clean":"2024-09-20T22:22:11.577189+0000","last_became_active":"2024-09-20T22:19:48.853347+0000","last_became_peered":"2024-09-20T22:19:48.853347+0000","last_unstale":"2024-09-20T22:22:11.577189+0000","last_undegraded":"2024-09-20T22:22:11.577189+0000","last_fullsized":"2024-09-20T22:22:11.577189+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:34.911846+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:21:34.911846+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:02.540378+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"328'15","reported_seq":509,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.130220+0000","last_change":"2024-09-20T22:22:12.130246+0000","last_active":"2024-09-20T22:22:12.130220+0000","last_peered":"2024-09-20T22:22:12.130220+0000","last_clean":"2024-09-20T22:22:12.130220+0000","last_became_active":"2024-09-20T22:12:52.527106+0000","last_became_peered":"2024-09-20T22:12:52.527106+0000","last_unstale":"2024-09-20T22:22:12.130220+0000","last_undegraded":"2024-09-20T22:22:12.130220+0000","last_fullsized":"2024-09-20T22:22:12.130220+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:22:12.130164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:12.130164+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:14.805756+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1049,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737769+0000","last_change":"2024-09-20T22:22:08.665641+0000","last_active":"2024-09-20T22:22:14.737769+0000","last_peered":"2024-09-20T22:22:14.737769+0000","last_clean":"2024-09-20T22:22:14.737769+0000","last_became_active":"2024-09-20T22:19:28.249922+0000","last_became_peered":"2024-09-20T22:19:28.249922+0000","last_unstale":"2024-09-20T22:22:14.737769+0000","last_undegraded":"2024-09-20T22:22:14.737769+0000","last_fullsized":"2024-09-20T22:22:14.737769+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:08.665517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:08.665517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:10.102382+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":317,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.739079+0000","last_change":"2024-09-20T22:19:45.818793+0000","last_active":"2024-09-20T22:22:14.739079+0000","last_peered":"2024-09-20T22:22:14.739079+0000","last_clean":"2024-09-20T22:22:14.739079+0000","last_became_active":"2024-09-20T22:19:45.818298+0000","last_became_peered":"2024-09-20T22:19:45.818298+0000","last_unstale":"2024-09-20T22:22:14.739079+0000","last_undegraded":"2024-09-20T22:22:14.739079+0000","last_fullsized":"2024-09-20T22:22:14.739079+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:36.302138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_clean_scrub_stamp":"2024-09-20T22:18:36.302138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1056,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111201+0000","last_change":"2024-09-20T22:19:28.241258+0000","last_active":"2024-09-20T22:22:12.111201+0000","last_peered":"2024-09-20T22:22:12.111201+0000","last_clean":"2024-09-20T22:22:12.111201+0000","last_became_active":"2024-09-20T22:19:28.240554+0000","last_became_peered":"2024-09-20T22:19:28.240554+0000","last_unstale":"2024-09-20T22:22:12.111201+0000","last_undegraded":"2024-09-20T22:22:12.111201+0000","last_fullsized":"2024-09-20T22:22:12.111201+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:03.970735+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:26.862279+0000","last_clean_scrub_stamp":"2024-09-20T22:18:03.970735+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"75'128","reported_seq":923,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592706+0000","last_change":"2024-09-20T22:18:07.509064+0000","last_active":"2024-09-20T22:22:16.592706+0000","last_peered":"2024-09-20T22:22:16.592706+0000","last_clean":"2024-09-20T22:22:16.592706+0000","last_became_active":"2024-09-20T22:15:50.574865+0000","last_became_peered":"2024-09-20T22:15:50.574865+0000","last_unstale":"2024-09-20T22:22:16.592706+0000","last_undegraded":"2024-09-20T22:22:16.592706+0000","last_fullsized":"2024-09-20T22:22:16.592706+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:18:07.508939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:18:07.508939+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"328'15","reported_seq":590,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593734+0000","last_change":"2024-09-20T22:18:39.362251+0000","last_active":"2024-09-20T22:22:16.593734+0000","last_peered":"2024-09-20T22:22:16.593734+0000","last_clean":"2024-09-20T22:22:16.593734+0000","last_became_active":"2024-09-20T22:11:30.493335+0000","last_became_peered":"2024-09-20T22:11:30.493335+0000","last_unstale":"2024-09-20T22:22:16.593734+0000","last_undegraded":"2024-09-20T22:22:16.593734+0000","last_fullsized":"2024-09-20T22:22:16.593734+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:18:39.362109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:39.362109+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":491,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111493+0000","last_change":"2024-09-20T22:19:13.833938+0000","last_active":"2024-09-20T22:22:12.111493+0000","last_peered":"2024-09-20T22:22:12.111493+0000","last_clean":"2024-09-20T22:22:12.111493+0000","last_became_active":"2024-09-20T22:12:52.524149+0000","last_became_peered":"2024-09-20T22:12:52.524149+0000","last_unstale":"2024-09-20T22:22:12.111493+0000","last_undegraded":"2024-09-20T22:22:12.111493+0000","last_fullsized":"2024-09-20T22:22:12.111493+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:13.833672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:56.538825+0000","last_clean_scrub_stamp":"2024-09-20T22:19:13.833672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"315'11","reported_seq":656,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:15.595465+0000","last_change":"2024-09-20T22:22:15.595487+0000","last_active":"2024-09-20T22:22:15.595465+0000","last_peered":"2024-09-20T22:22:15.595465+0000","last_clean":"2024-09-20T22:22:15.595465+0000","last_became_active":"2024-09-20T22:12:33.544098+0000","last_became_peered":"2024-09-20T22:12:33.544098+0000","last_unstale":"2024-09-20T22:22:15.595465+0000","last_undegraded":"2024-09-20T22:22:15.595465+0000","last_fullsized":"2024-09-20T22:22:15.595465+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:22:15.595411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:15.595411+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:21.969870+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":626,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110869+0000","last_change":"2024-09-20T22:19:28.247124+0000","last_active":"2024-09-20T22:22:12.110869+0000","last_peered":"2024-09-20T22:22:12.110869+0000","last_clean":"2024-09-20T22:22:12.110869+0000","last_became_active":"2024-09-20T22:19:28.245964+0000","last_became_peered":"2024-09-20T22:19:28.245964+0000","last_unstale":"2024-09-20T22:22:12.110869+0000","last_undegraded":"2024-09-20T22:22:12.110869+0000","last_fullsized":"2024-09-20T22:22:12.110869+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:03.323607+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:03.323607+0000","last_clean_scrub_stamp":"2024-09-20T22:18:03.323607+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.11","version":"75'128","reported_seq":941,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110536+0000","last_change":"2024-09-20T22:19:05.735170+0000","last_active":"2024-09-20T22:22:12.110536+0000","last_peered":"2024-09-20T22:22:12.110536+0000","last_clean":"2024-09-20T22:22:12.110536+0000","last_became_active":"2024-09-20T22:15:50.612544+0000","last_became_peered":"2024-09-20T22:15:50.612544+0000","last_unstale":"2024-09-20T22:22:12.110536+0000","last_undegraded":"2024-09-20T22:22:12.110536+0000","last_fullsized":"2024-09-20T22:22:12.110536+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:05.735058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:19:05.735058+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"466'25","reported_seq":512,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738253+0000","last_change":"2024-09-20T22:19:45.871562+0000","last_active":"2024-09-20T22:22:14.738253+0000","last_peered":"2024-09-20T22:22:14.738253+0000","last_clean":"2024-09-20T22:22:14.738253+0000","last_became_active":"2024-09-20T22:19:45.809071+0000","last_became_peered":"2024-09-20T22:19:45.809071+0000","last_unstale":"2024-09-20T22:22:14.738253+0000","last_undegraded":"2024-09-20T22:22:14.738253+0000","last_fullsized":"2024-09-20T22:22:14.738253+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'25","last_scrub_stamp":"2024-09-20T22:19:16.310142+0000","last_deep_scrub":"466'25","last_deep_scrub_stamp":"2024-09-20T22:19:16.310142+0000","last_clean_scrub_stamp":"2024-09-20T22:19:16.310142+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1041,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738448+0000","last_change":"2024-09-20T22:19:28.251114+0000","last_active":"2024-09-20T22:22:14.738448+0000","last_peered":"2024-09-20T22:22:14.738448+0000","last_clean":"2024-09-20T22:22:14.738448+0000","last_became_active":"2024-09-20T22:19:28.250811+0000","last_became_peered":"2024-09-20T22:19:28.250811+0000","last_unstale":"2024-09-20T22:22:14.738448+0000","last_undegraded":"2024-09-20T22:22:14.738448+0000","last_fullsized":"2024-09-20T22:22:14.738448+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:06.001800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:19:06.001800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"481'15","reported_seq":743,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593955+0000","last_change":"2024-09-20T22:21:36.865254+0000","last_active":"2024-09-20T22:22:16.593955+0000","last_peered":"2024-09-20T22:22:16.593955+0000","last_clean":"2024-09-20T22:22:16.593955+0000","last_became_active":"2024-09-20T22:19:45.809247+0000","last_became_peered":"2024-09-20T22:19:45.809247+0000","last_unstale":"2024-09-20T22:22:16.593955+0000","last_undegraded":"2024-09-20T22:22:16.593955+0000","last_fullsized":"2024-09-20T22:22:16.593955+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"481'15","last_scrub_stamp":"2024-09-20T22:21:36.865145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:21:36.865145+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:45.209874+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":490,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111768+0000","last_change":"2024-09-20T22:21:40.964557+0000","last_active":"2024-09-20T22:22:12.111768+0000","last_peered":"2024-09-20T22:22:12.111768+0000","last_clean":"2024-09-20T22:22:12.111768+0000","last_became_active":"2024-09-20T22:12:52.523781+0000","last_became_peered":"2024-09-20T22:12:52.523781+0000","last_unstale":"2024-09-20T22:22:12.111768+0000","last_undegraded":"2024-09-20T22:22:12.111768+0000","last_fullsized":"2024-09-20T22:22:12.111768+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:40.964227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:21:40.964227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:52.822485+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.16","version":"75'128","reported_seq":736,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111570+0000","last_change":"2024-09-20T22:21:00.998420+0000","last_active":"2024-09-20T22:22:12.111570+0000","last_peered":"2024-09-20T22:22:12.111570+0000","last_clean":"2024-09-20T22:22:12.111570+0000","last_became_active":"2024-09-20T22:19:48.855437+0000","last_became_peered":"2024-09-20T22:19:48.855437+0000","last_unstale":"2024-09-20T22:22:12.111570+0000","last_undegraded":"2024-09-20T22:22:12.111570+0000","last_fullsized":"2024-09-20T22:22:12.111570+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:00.998286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:21:00.998286+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:14.181658+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"328'19","reported_seq":798,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578106+0000","last_change":"2024-09-20T22:19:45.788839+0000","last_active":"2024-09-20T22:22:11.578106+0000","last_peered":"2024-09-20T22:22:11.578106+0000","last_clean":"2024-09-20T22:22:11.578106+0000","last_became_active":"2024-09-20T22:19:45.788204+0000","last_became_peered":"2024-09-20T22:19:45.788204+0000","last_unstale":"2024-09-20T22:22:11.578106+0000","last_undegraded":"2024-09-20T22:22:11.578106+0000","last_fullsized":"2024-09-20T22:22:11.578106+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:19:18.385865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:18.385865+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":171,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738159+0000","last_change":"2024-09-20T22:21:43.594958+0000","last_active":"2024-09-20T22:22:14.738159+0000","last_peered":"2024-09-20T22:22:14.738159+0000","last_clean":"2024-09-20T22:22:14.738159+0000","last_became_active":"2024-09-20T22:19:28.260795+0000","last_became_peered":"2024-09-20T22:19:28.260795+0000","last_unstale":"2024-09-20T22:22:14.738159+0000","last_undegraded":"2024-09-20T22:22:14.738159+0000","last_fullsized":"2024-09-20T22:22:14.738159+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:43.594884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:21:43.594884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:49.149766+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":262,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593556+0000","last_change":"2024-09-20T22:19:45.798598+0000","last_active":"2024-09-20T22:22:16.593556+0000","last_peered":"2024-09-20T22:22:16.593556+0000","last_clean":"2024-09-20T22:22:16.593556+0000","last_became_active":"2024-09-20T22:19:45.797640+0000","last_became_peered":"2024-09-20T22:19:45.797640+0000","last_unstale":"2024-09-20T22:22:16.593556+0000","last_undegraded":"2024-09-20T22:22:16.593556+0000","last_fullsized":"2024-09-20T22:22:16.593556+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:40.528539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:40.528539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":548,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110266+0000","last_change":"2024-09-20T22:21:39.989384+0000","last_active":"2024-09-20T22:22:12.110266+0000","last_peered":"2024-09-20T22:22:12.110266+0000","last_clean":"2024-09-20T22:22:12.110266+0000","last_became_active":"2024-09-20T22:19:45.810800+0000","last_became_peered":"2024-09-20T22:19:45.810800+0000","last_unstale":"2024-09-20T22:22:12.110266+0000","last_undegraded":"2024-09-20T22:22:12.110266+0000","last_fullsized":"2024-09-20T22:22:12.110266+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:39.989289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:21:39.989289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:58.373658+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.17","version":"75'128","reported_seq":545,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.594124+0000","last_change":"2024-09-20T22:17:57.421052+0000","last_active":"2024-09-20T22:22:16.594124+0000","last_peered":"2024-09-20T22:22:16.594124+0000","last_clean":"2024-09-20T22:22:16.594124+0000","last_became_active":"2024-09-20T22:17:44.721755+0000","last_became_peered":"2024-09-20T22:17:44.721755+0000","last_unstale":"2024-09-20T22:22:16.594124+0000","last_undegraded":"2024-09-20T22:22:16.594124+0000","last_fullsized":"2024-09-20T22:22:16.594124+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:57.420935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.420935+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"328'19","reported_seq":647,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578469+0000","last_change":"2024-09-20T22:21:34.778780+0000","last_active":"2024-09-20T22:22:11.578469+0000","last_peered":"2024-09-20T22:22:11.578469+0000","last_clean":"2024-09-20T22:22:11.578469+0000","last_became_active":"2024-09-20T22:19:45.801107+0000","last_became_peered":"2024-09-20T22:19:45.801107+0000","last_unstale":"2024-09-20T22:22:11.578469+0000","last_undegraded":"2024-09-20T22:22:11.578469+0000","last_fullsized":"2024-09-20T22:22:11.578469+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:21:34.778616+0000","last_deep_scrub":"291'13","last_deep_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_clean_scrub_stamp":"2024-09-20T22:21:34.778616+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:38.758924+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":637,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738519+0000","last_change":"2024-09-20T22:19:28.243263+0000","last_active":"2024-09-20T22:22:14.738519+0000","last_peered":"2024-09-20T22:22:14.738519+0000","last_clean":"2024-09-20T22:22:14.738519+0000","last_became_active":"2024-09-20T22:19:28.242951+0000","last_became_peered":"2024-09-20T22:19:28.242951+0000","last_unstale":"2024-09-20T22:22:14.738519+0000","last_undegraded":"2024-09-20T22:22:14.738519+0000","last_fullsized":"2024-09-20T22:22:14.738519+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:54.900125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:54.900125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":497,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111284+0000","last_change":"2024-09-20T22:19:45.820095+0000","last_active":"2024-09-20T22:22:12.111284+0000","last_peered":"2024-09-20T22:22:12.111284+0000","last_clean":"2024-09-20T22:22:12.111284+0000","last_became_active":"2024-09-20T22:19:45.817319+0000","last_became_peered":"2024-09-20T22:19:45.817319+0000","last_unstale":"2024-09-20T22:22:12.111284+0000","last_undegraded":"2024-09-20T22:22:12.111284+0000","last_fullsized":"2024-09-20T22:22:12.111284+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:02.798998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:02.798998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":497,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.109938+0000","last_change":"2024-09-20T22:22:11.135140+0000","last_active":"2024-09-20T22:22:12.109938+0000","last_peered":"2024-09-20T22:22:12.109938+0000","last_clean":"2024-09-20T22:22:12.109938+0000","last_became_active":"2024-09-20T22:19:45.818444+0000","last_became_peered":"2024-09-20T22:19:45.818444+0000","last_unstale":"2024-09-20T22:22:12.109938+0000","last_undegraded":"2024-09-20T22:22:12.109938+0000","last_fullsized":"2024-09-20T22:22:12.109938+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:11.135068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:11.135068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:15.398811+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"75'128","reported_seq":1108,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110958+0000","last_change":"2024-09-20T22:19:48.880703+0000","last_active":"2024-09-20T22:22:12.110958+0000","last_peered":"2024-09-20T22:22:12.110958+0000","last_clean":"2024-09-20T22:22:12.110958+0000","last_became_active":"2024-09-20T22:19:48.880005+0000","last_became_peered":"2024-09-20T22:19:48.880005+0000","last_unstale":"2024-09-20T22:22:12.110958+0000","last_undegraded":"2024-09-20T22:22:12.110958+0000","last_fullsized":"2024-09-20T22:22:12.110958+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:09.890234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:19:09.890234+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"328'19","reported_seq":960,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737568+0000","last_change":"2024-09-20T22:19:17.961328+0000","last_active":"2024-09-20T22:22:14.737568+0000","last_peered":"2024-09-20T22:22:14.737568+0000","last_clean":"2024-09-20T22:22:14.737568+0000","last_became_active":"2024-09-20T22:12:33.545199+0000","last_became_peered":"2024-09-20T22:12:33.545199+0000","last_unstale":"2024-09-20T22:22:14.737568+0000","last_undegraded":"2024-09-20T22:22:14.737568+0000","last_fullsized":"2024-09-20T22:22:14.737568+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:19:17.961211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:17.961211+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":580,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593687+0000","last_change":"2024-09-20T22:19:45.795979+0000","last_active":"2024-09-20T22:22:16.593687+0000","last_peered":"2024-09-20T22:22:16.593687+0000","last_clean":"2024-09-20T22:22:16.593687+0000","last_became_active":"2024-09-20T22:19:45.795652+0000","last_became_peered":"2024-09-20T22:19:45.795652+0000","last_unstale":"2024-09-20T22:22:16.593687+0000","last_undegraded":"2024-09-20T22:22:16.593687+0000","last_fullsized":"2024-09-20T22:22:16.593687+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:55.398175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_clean_scrub_stamp":"2024-09-20T22:17:55.398175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1041,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737279+0000","last_change":"2024-09-20T22:21:44.577623+0000","last_active":"2024-09-20T22:22:14.737279+0000","last_peered":"2024-09-20T22:22:14.737279+0000","last_clean":"2024-09-20T22:22:14.737279+0000","last_became_active":"2024-09-20T22:19:28.260465+0000","last_became_peered":"2024-09-20T22:19:28.260465+0000","last_unstale":"2024-09-20T22:22:14.737279+0000","last_undegraded":"2024-09-20T22:22:14.737279+0000","last_fullsized":"2024-09-20T22:22:14.737279+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:44.577510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:21:44.577510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:55.402165+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":480,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111203+0000","last_change":"2024-09-20T22:18:32.424564+0000","last_active":"2024-09-20T22:22:12.111203+0000","last_peered":"2024-09-20T22:22:12.111203+0000","last_clean":"2024-09-20T22:22:12.111203+0000","last_became_active":"2024-09-20T22:12:57.883998+0000","last_became_peered":"2024-09-20T22:12:57.883998+0000","last_unstale":"2024-09-20T22:22:12.111203+0000","last_undegraded":"2024-09-20T22:22:12.111203+0000","last_fullsized":"2024-09-20T22:22:12.111203+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:32.424321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:32.424321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.15","version":"75'128","reported_seq":551,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738278+0000","last_change":"2024-09-20T22:19:48.849798+0000","last_active":"2024-09-20T22:22:14.738278+0000","last_peered":"2024-09-20T22:22:14.738278+0000","last_clean":"2024-09-20T22:22:14.738278+0000","last_became_active":"2024-09-20T22:19:48.849447+0000","last_became_peered":"2024-09-20T22:19:48.849447+0000","last_unstale":"2024-09-20T22:22:14.738278+0000","last_undegraded":"2024-09-20T22:22:14.738278+0000","last_fullsized":"2024-09-20T22:22:14.738278+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.859373+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.859373+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"453'10","reported_seq":1077,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577296+0000","last_change":"2024-09-20T22:21:38.901104+0000","last_active":"2024-09-20T22:22:11.577296+0000","last_peered":"2024-09-20T22:22:11.577296+0000","last_clean":"2024-09-20T22:22:11.577296+0000","last_became_active":"2024-09-20T22:19:45.795544+0000","last_became_peered":"2024-09-20T22:19:45.795544+0000","last_unstale":"2024-09-20T22:22:11.577296+0000","last_undegraded":"2024-09-20T22:22:11.577296+0000","last_fullsized":"2024-09-20T22:22:11.577296+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"453'10","last_scrub_stamp":"2024-09-20T22:21:38.900946+0000","last_deep_scrub":"453'10","last_deep_scrub_stamp":"2024-09-20T22:21:38.900946+0000","last_clean_scrub_stamp":"2024-09-20T22:21:38.900946+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:51.759107+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":482,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111745+0000","last_change":"2024-09-20T22:18:09.367123+0000","last_active":"2024-09-20T22:22:12.111745+0000","last_peered":"2024-09-20T22:22:12.111745+0000","last_clean":"2024-09-20T22:22:12.111745+0000","last_became_active":"2024-09-20T22:12:52.517510+0000","last_became_peered":"2024-09-20T22:12:52.517510+0000","last_unstale":"2024-09-20T22:22:12.111745+0000","last_undegraded":"2024-09-20T22:22:12.111745+0000","last_fullsized":"2024-09-20T22:22:12.111745+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:09.366999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:09.366999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1060,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577380+0000","last_change":"2024-09-20T22:19:28.261147+0000","last_active":"2024-09-20T22:22:11.577380+0000","last_peered":"2024-09-20T22:22:11.577380+0000","last_clean":"2024-09-20T22:22:11.577380+0000","last_became_active":"2024-09-20T22:19:28.260576+0000","last_became_peered":"2024-09-20T22:19:28.260576+0000","last_unstale":"2024-09-20T22:22:11.577380+0000","last_undegraded":"2024-09-20T22:22:11.577380+0000","last_fullsized":"2024-09-20T22:22:11.577380+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.105874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.105874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1029,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737721+0000","last_change":"2024-09-20T22:19:28.245693+0000","last_active":"2024-09-20T22:22:14.737721+0000","last_peered":"2024-09-20T22:22:14.737721+0000","last_clean":"2024-09-20T22:22:14.737721+0000","last_became_active":"2024-09-20T22:19:28.245306+0000","last_became_peered":"2024-09-20T22:19:28.245306+0000","last_unstale":"2024-09-20T22:22:14.737721+0000","last_undegraded":"2024-09-20T22:22:14.737721+0000","last_fullsized":"2024-09-20T22:22:14.737721+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:33.935154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_clean_scrub_stamp":"2024-09-20T22:17:33.935154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1a","version":"75'128","reported_seq":475,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737652+0000","last_change":"2024-09-20T22:21:34.683687+0000","last_active":"2024-09-20T22:22:14.737652+0000","last_peered":"2024-09-20T22:22:14.737652+0000","last_clean":"2024-09-20T22:22:14.737652+0000","last_became_active":"2024-09-20T22:19:48.843466+0000","last_became_peered":"2024-09-20T22:19:48.843466+0000","last_unstale":"2024-09-20T22:22:14.737652+0000","last_undegraded":"2024-09-20T22:22:14.737652+0000","last_fullsized":"2024-09-20T22:22:14.737652+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:34.683575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:21:34.683575+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:03.901269+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":915,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593913+0000","last_change":"2024-09-20T22:19:45.840129+0000","last_active":"2024-09-20T22:22:16.593913+0000","last_peered":"2024-09-20T22:22:16.593913+0000","last_clean":"2024-09-20T22:22:16.593913+0000","last_became_active":"2024-09-20T22:19:45.839497+0000","last_became_peered":"2024-09-20T22:19:45.839497+0000","last_unstale":"2024-09-20T22:22:16.593913+0000","last_undegraded":"2024-09-20T22:22:16.593913+0000","last_fullsized":"2024-09-20T22:22:16.593913+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:37.293600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:37.293600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":496,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.110239+0000","last_change":"2024-09-20T22:21:39.004287+0000","last_active":"2024-09-20T22:22:12.110239+0000","last_peered":"2024-09-20T22:22:12.110239+0000","last_clean":"2024-09-20T22:22:12.110239+0000","last_became_active":"2024-09-20T22:19:45.817473+0000","last_became_peered":"2024-09-20T22:19:45.817473+0000","last_unstale":"2024-09-20T22:22:12.110239+0000","last_undegraded":"2024-09-20T22:22:12.110239+0000","last_fullsized":"2024-09-20T22:22:12.110239+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:39.004178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:21:39.004178+0000","last_clean_scrub_stamp":"2024-09-20T22:21:39.004178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:03.825432+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"328'19","reported_seq":1101,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578946+0000","last_change":"2024-09-20T22:22:05.674913+0000","last_active":"2024-09-20T22:22:11.578946+0000","last_peered":"2024-09-20T22:22:11.578946+0000","last_clean":"2024-09-20T22:22:11.578946+0000","last_became_active":"2024-09-20T22:19:45.815971+0000","last_became_peered":"2024-09-20T22:19:45.815971+0000","last_unstale":"2024-09-20T22:22:11.578946+0000","last_undegraded":"2024-09-20T22:22:11.578946+0000","last_fullsized":"2024-09-20T22:22:11.578946+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:22:05.674812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:05.674812+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:08.236842+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":491,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.111739+0000","last_change":"2024-09-20T22:22:10.095005+0000","last_active":"2024-09-20T22:22:12.111739+0000","last_peered":"2024-09-20T22:22:12.111739+0000","last_clean":"2024-09-20T22:22:12.111739+0000","last_became_active":"2024-09-20T22:12:52.524425+0000","last_became_peered":"2024-09-20T22:12:52.524425+0000","last_unstale":"2024-09-20T22:22:12.111739+0000","last_undegraded":"2024-09-20T22:22:12.111739+0000","last_fullsized":"2024-09-20T22:22:12.111739+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:10.094934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:10.094934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:29.788661+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1105,"num_read_kb":862,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":6342,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":14368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":756,"ondisk_log_size":756,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2991,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":7072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1137,"num_read_kb":1132,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1186,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":31,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3968,"ondisk_log_size":3968,"up":93,"acting":93,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813136109,"num_pgs":89,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":876812,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":875262,"kb_avail":92868340,"statfs":{"total":95995035648,"available":95097180160,"internally_reserved":0,"allocated":1560576,"data_stored":1039815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":896268746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430446,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":750504,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":748926,"kb_avail":92994648,"statfs":{"total":95995035648,"available":95226519552,"internally_reserved":0,"allocated":1589248,"data_stored":1089021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":766900678},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724789,"num_pgs":88,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":997572,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":996030,"kb_avail":92747580,"statfs":{"total":95995035648,"available":94973521920,"internally_reserved":0,"allocated":1552384,"data_stored":1039899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1019935173},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":0,"seq":2284922601479,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":622770258105,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149953,"num_pgs":81,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":943452,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":941886,"kb_avail":92801700,"statfs":{"total":95995035648,"available":95028940800,"internally_reserved":0,"allocated":1581056,"data_stored":1074514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":964491719},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411590,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":754552,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":753086,"kb_avail":92990600,"statfs":{"total":95995035648,"available":95222374400,"internally_reserved":0,"allocated":1478656,"data_stored":1014868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":771160517},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738641,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1003184,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":1001598,"kb_avail":92741968,"statfs":{"total":95995035648,"available":94967775232,"internally_reserved":0,"allocated":1601536,"data_stored":1089124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1025636805},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:22:18.979 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:22:19.239 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:19.415 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_blocked_ops 2024-09-20T22:22:19.670 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:22:19.694 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:19 smithi060 ceph-mon[29694]: from='client.26576 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:22:19.694 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:19 smithi060 ceph-mon[29694]: pgmap v1049: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T22:22:19.694 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:19 smithi060 ceph-mon[29694]: 1.1e scrub starts 2024-09-20T22:22:19.694 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:19 smithi060 ceph-mon[29694]: 1.1e scrub ok 2024-09-20T22:22:19.695 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:19 smithi060 ceph-mon[40290]: from='client.26576 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:22:19.695 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:19 smithi060 ceph-mon[40290]: pgmap v1049: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T22:22:19.695 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:19 smithi060 ceph-mon[40290]: 1.1e scrub starts 2024-09-20T22:22:19.695 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:19 smithi060 ceph-mon[40290]: 1.1e scrub ok 2024-09-20T22:22:19.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:19 smithi067 ceph-mon[33511]: from='client.26576 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:22:19.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:19 smithi067 ceph-mon[33511]: pgmap v1049: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T22:22:19.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:19 smithi067 ceph-mon[33511]: 1.1e scrub starts 2024-09-20T22:22:19.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:19 smithi067 ceph-mon[33511]: 1.1e scrub ok 2024-09-20T22:22:19.873 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:22:19.875 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:22:20.210 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:20.210 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:20.469 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":1049,"stamp":"2024-09-20T22:22:18.406374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":2259,"num_read_kb":2011,"num_write":722,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":367,"num_bytes_recovered":10519,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4846,"ondisk_log_size":4846,"up":486,"acting":486,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":486,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":6,"kb":562470912,"kb_used":5326240,"kb_used_data":9300,"kb_used_omap":9,"kb_used_meta":5316790,"kb_avail":557144672,"statfs":{"total":575970213888,"available":570516144128,"internally_reserved":0,"allocated":9523200,"data_stored":6448637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9562,"internal_metadata":5444393638},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.708895"},"pg_stats":[{"pgid":"1.1d","version":"75'128","reported_seq":559,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.701156+0000","last_change":"2024-09-20T22:22:15.002475+0000","last_active":"2024-09-20T22:22:16.701156+0000","last_peered":"2024-09-20T22:22:16.701156+0000","last_clean":"2024-09-20T22:22:16.701156+0000","last_became_active":"2024-09-20T22:19:48.866938+0000","last_became_peered":"2024-09-20T22:19:48.866938+0000","last_unstale":"2024-09-20T22:22:16.701156+0000","last_undegraded":"2024-09-20T22:22:16.701156+0000","last_fullsized":"2024-09-20T22:22:16.701156+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:15.002349+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:22:15.002349+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:41.199221+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,5],"acting":[2,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":491,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.699943+0000","last_change":"2024-09-20T22:18:43.512180+0000","last_active":"2024-09-20T22:22:16.699943+0000","last_peered":"2024-09-20T22:22:16.699943+0000","last_clean":"2024-09-20T22:22:16.699943+0000","last_became_active":"2024-09-20T22:12:52.515921+0000","last_became_peered":"2024-09-20T22:12:52.515921+0000","last_unstale":"2024-09-20T22:22:16.699943+0000","last_undegraded":"2024-09-20T22:22:16.699943+0000","last_fullsized":"2024-09-20T22:22:16.699943+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:43.512066+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:43.512066+0000","last_clean_scrub_stamp":"2024-09-20T22:18:43.512066+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"327'19","reported_seq":782,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577539+0000","last_change":"2024-09-20T22:19:45.861936+0000","last_active":"2024-09-20T22:22:11.577539+0000","last_peered":"2024-09-20T22:22:11.577539+0000","last_clean":"2024-09-20T22:22:11.577539+0000","last_became_active":"2024-09-20T22:19:45.798056+0000","last_became_peered":"2024-09-20T22:19:45.798056+0000","last_unstale":"2024-09-20T22:22:11.577539+0000","last_undegraded":"2024-09-20T22:22:11.577539+0000","last_fullsized":"2024-09-20T22:22:11.577539+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"327'19","last_scrub_stamp":"2024-09-20T22:19:15.299519+0000","last_deep_scrub":"252'13","last_deep_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_clean_scrub_stamp":"2024-09-20T22:19:15.299519+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":270,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592643+0000","last_change":"2024-09-20T22:21:38.882602+0000","last_active":"2024-09-20T22:22:16.592643+0000","last_peered":"2024-09-20T22:22:16.592643+0000","last_clean":"2024-09-20T22:22:16.592643+0000","last_became_active":"2024-09-20T22:19:45.795374+0000","last_became_peered":"2024-09-20T22:19:45.795374+0000","last_unstale":"2024-09-20T22:22:16.592643+0000","last_undegraded":"2024-09-20T22:22:16.592643+0000","last_fullsized":"2024-09-20T22:22:16.592643+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:38.882495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:21:38.882495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:43.895865+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,2],"acting":[5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":639,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593339+0000","last_change":"2024-09-20T22:19:45.797905+0000","last_active":"2024-09-20T22:22:16.593339+0000","last_peered":"2024-09-20T22:22:16.593339+0000","last_clean":"2024-09-20T22:22:16.593339+0000","last_became_active":"2024-09-20T22:19:45.796589+0000","last_became_peered":"2024-09-20T22:19:45.796589+0000","last_unstale":"2024-09-20T22:22:16.593339+0000","last_undegraded":"2024-09-20T22:22:16.593339+0000","last_fullsized":"2024-09-20T22:22:16.593339+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.321275+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.321275+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1c","version":"75'128","reported_seq":999,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738379+0000","last_change":"2024-09-20T22:19:48.838779+0000","last_active":"2024-09-20T22:22:14.738379+0000","last_peered":"2024-09-20T22:22:14.738379+0000","last_clean":"2024-09-20T22:22:14.738379+0000","last_became_active":"2024-09-20T22:19:48.838208+0000","last_became_peered":"2024-09-20T22:19:48.838208+0000","last_unstale":"2024-09-20T22:22:14.738379+0000","last_undegraded":"2024-09-20T22:22:14.738379+0000","last_fullsized":"2024-09-20T22:22:14.738379+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:57.852213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.852213+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":1041,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.927311+0000","last_change":"2024-09-20T22:19:38.797027+0000","last_active":"2024-09-20T22:22:16.927311+0000","last_peered":"2024-09-20T22:22:16.927311+0000","last_clean":"2024-09-20T22:22:16.927311+0000","last_became_active":"2024-09-20T22:02:56.349915+0000","last_became_peered":"2024-09-20T22:02:56.349915+0000","last_unstale":"2024-09-20T22:22:16.927311+0000","last_undegraded":"2024-09-20T22:22:16.927311+0000","last_fullsized":"2024-09-20T22:22:16.927311+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:19:38.796920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:19:38.796920+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":510,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578685+0000","last_change":"2024-09-20T22:18:31.987311+0000","last_active":"2024-09-20T22:22:11.578685+0000","last_peered":"2024-09-20T22:22:11.578685+0000","last_clean":"2024-09-20T22:22:11.578685+0000","last_became_active":"2024-09-20T22:12:33.545100+0000","last_became_peered":"2024-09-20T22:12:33.545100+0000","last_unstale":"2024-09-20T22:22:11.578685+0000","last_undegraded":"2024-09-20T22:22:11.578685+0000","last_fullsized":"2024-09-20T22:22:11.578685+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:31.987191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:31.987191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.19","version":"406'34","reported_seq":649,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.700627+0000","last_change":"2024-09-20T22:19:45.100154+0000","last_active":"2024-09-20T22:22:16.700627+0000","last_peered":"2024-09-20T22:22:16.700627+0000","last_clean":"2024-09-20T22:22:16.700627+0000","last_became_active":"2024-09-20T22:12:52.517296+0000","last_became_peered":"2024-09-20T22:12:52.517296+0000","last_unstale":"2024-09-20T22:22:16.700627+0000","last_undegraded":"2024-09-20T22:22:16.700627+0000","last_fullsized":"2024-09-20T22:22:16.700627+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'34","last_scrub_stamp":"2024-09-20T22:19:45.100033+0000","last_deep_scrub":"406'34","last_deep_scrub_stamp":"2024-09-20T22:16:31.033354+0000","last_clean_scrub_stamp":"2024-09-20T22:19:45.100033+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":333,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577441+0000","last_change":"2024-09-20T22:19:45.791468+0000","last_active":"2024-09-20T22:22:11.577441+0000","last_peered":"2024-09-20T22:22:11.577441+0000","last_clean":"2024-09-20T22:22:11.577441+0000","last_became_active":"2024-09-20T22:19:45.791138+0000","last_became_peered":"2024-09-20T22:19:45.791138+0000","last_unstale":"2024-09-20T22:22:11.577441+0000","last_undegraded":"2024-09-20T22:22:11.577441+0000","last_fullsized":"2024-09-20T22:22:11.577441+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:10.335527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:10.335527+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1071,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593696+0000","last_change":"2024-09-20T22:22:11.073921+0000","last_active":"2024-09-20T22:22:16.593696+0000","last_peered":"2024-09-20T22:22:16.593696+0000","last_clean":"2024-09-20T22:22:16.593696+0000","last_became_active":"2024-09-20T22:19:45.790130+0000","last_became_peered":"2024-09-20T22:19:45.790130+0000","last_unstale":"2024-09-20T22:22:16.593696+0000","last_undegraded":"2024-09-20T22:22:16.593696+0000","last_fullsized":"2024-09-20T22:22:16.593696+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:11.073832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:22:11.073832+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:32.257687+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,3],"acting":[5,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"328'15","reported_seq":926,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.698373+0000","last_change":"2024-09-20T22:18:35.404979+0000","last_active":"2024-09-20T22:22:16.698373+0000","last_peered":"2024-09-20T22:22:16.698373+0000","last_clean":"2024-09-20T22:22:16.698373+0000","last_became_active":"2024-09-20T22:17:03.663697+0000","last_became_peered":"2024-09-20T22:17:03.663697+0000","last_unstale":"2024-09-20T22:22:16.698373+0000","last_undegraded":"2024-09-20T22:22:16.698373+0000","last_fullsized":"2024-09-20T22:22:16.698373+0000","mapping_epoch":415,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":416,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:18:35.404892+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:35.404892+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":591,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592994+0000","last_change":"2024-09-20T22:22:14.027262+0000","last_active":"2024-09-20T22:22:16.592994+0000","last_peered":"2024-09-20T22:22:16.592994+0000","last_clean":"2024-09-20T22:22:16.592994+0000","last_became_active":"2024-09-20T22:12:52.518549+0000","last_became_peered":"2024-09-20T22:12:52.518549+0000","last_unstale":"2024-09-20T22:22:16.592994+0000","last_undegraded":"2024-09-20T22:22:16.592994+0000","last_fullsized":"2024-09-20T22:22:16.592994+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:14.027140+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:14.027140+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:20.774583+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:17.063591+0000","last_change":"2024-09-20T22:22:17.063191+0000","last_active":"2024-09-20T22:22:17.063591+0000","last_peered":"2024-09-20T22:22:17.063591+0000","last_clean":"2024-09-20T22:22:17.063591+0000","last_became_active":"2024-09-20T22:22:17.062842+0000","last_became_peered":"2024-09-20T22:22:17.062842+0000","last_unstale":"2024-09-20T22:22:17.063591+0000","last_undegraded":"2024-09-20T22:22:17.063591+0000","last_fullsized":"2024-09-20T22:22:17.063591+0000","mapping_epoch":549,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":550,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:13.392405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:37.934760+0000","last_clean_scrub_stamp":"2024-09-20T22:19:13.392405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1e","version":"75'128","reported_seq":561,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.927973+0000","last_change":"2024-09-20T22:19:34.669163+0000","last_active":"2024-09-20T22:22:16.927973+0000","last_peered":"2024-09-20T22:22:16.927973+0000","last_clean":"2024-09-20T22:22:16.927973+0000","last_became_active":"2024-09-20T22:17:44.708330+0000","last_became_peered":"2024-09-20T22:17:44.708330+0000","last_unstale":"2024-09-20T22:22:16.927973+0000","last_undegraded":"2024-09-20T22:22:16.927973+0000","last_fullsized":"2024-09-20T22:22:16.927973+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:34.669046+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:17:40.367940+0000","last_clean_scrub_stamp":"2024-09-20T22:19:34.669046+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1068,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593194+0000","last_change":"2024-09-20T22:19:45.796110+0000","last_active":"2024-09-20T22:22:16.593194+0000","last_peered":"2024-09-20T22:22:16.593194+0000","last_clean":"2024-09-20T22:22:16.593194+0000","last_became_active":"2024-09-20T22:19:45.795550+0000","last_became_peered":"2024-09-20T22:19:45.795550+0000","last_unstale":"2024-09-20T22:22:16.593194+0000","last_undegraded":"2024-09-20T22:22:16.593194+0000","last_fullsized":"2024-09-20T22:22:16.593194+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:03.470297+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:30.215263+0000","last_clean_scrub_stamp":"2024-09-20T22:18:03.470297+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"328'7","reported_seq":762,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.701102+0000","last_change":"2024-09-20T22:22:04.903574+0000","last_active":"2024-09-20T22:22:16.701102+0000","last_peered":"2024-09-20T22:22:16.701102+0000","last_clean":"2024-09-20T22:22:16.701102+0000","last_became_active":"2024-09-20T22:19:45.799278+0000","last_became_peered":"2024-09-20T22:19:45.799278+0000","last_unstale":"2024-09-20T22:22:16.701102+0000","last_undegraded":"2024-09-20T22:22:16.701102+0000","last_fullsized":"2024-09-20T22:22:16.701102+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'7","last_scrub_stamp":"2024-09-20T22:22:04.903491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:04.903491+0000","objects_scrubbed":2,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:31.027625+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":9,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1055,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737759+0000","last_change":"2024-09-20T22:21:41.666202+0000","last_active":"2024-09-20T22:22:14.737759+0000","last_peered":"2024-09-20T22:22:14.737759+0000","last_clean":"2024-09-20T22:22:14.737759+0000","last_became_active":"2024-09-20T22:19:45.817477+0000","last_became_peered":"2024-09-20T22:19:45.817477+0000","last_unstale":"2024-09-20T22:22:14.737759+0000","last_undegraded":"2024-09-20T22:22:14.737759+0000","last_fullsized":"2024-09-20T22:22:14.737759+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:41.666098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:21:41.666098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:43.290262+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":585,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592642+0000","last_change":"2024-09-20T22:22:10.109040+0000","last_active":"2024-09-20T22:22:16.592642+0000","last_peered":"2024-09-20T22:22:16.592642+0000","last_clean":"2024-09-20T22:22:16.592642+0000","last_became_active":"2024-09-20T22:19:45.789372+0000","last_became_peered":"2024-09-20T22:19:45.789372+0000","last_unstale":"2024-09-20T22:22:16.592642+0000","last_undegraded":"2024-09-20T22:22:16.592642+0000","last_fullsized":"2024-09-20T22:22:16.592642+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:10.108947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:22:10.108947+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:22.119266+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.19","version":"75'128","reported_seq":819,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.698249+0000","last_change":"2024-09-20T22:17:56.103865+0000","last_active":"2024-09-20T22:22:16.698249+0000","last_peered":"2024-09-20T22:22:16.698249+0000","last_clean":"2024-09-20T22:22:16.698249+0000","last_became_active":"2024-09-20T22:17:44.716139+0000","last_became_peered":"2024-09-20T22:17:44.716139+0000","last_unstale":"2024-09-20T22:22:16.698249+0000","last_undegraded":"2024-09-20T22:22:16.698249+0000","last_fullsized":"2024-09-20T22:22:16.698249+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.103744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.103744+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":167,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578564+0000","last_change":"2024-09-20T22:19:44.809795+0000","last_active":"2024-09-20T22:22:11.578564+0000","last_peered":"2024-09-20T22:22:11.578564+0000","last_clean":"2024-09-20T22:22:11.578564+0000","last_became_active":"2024-09-20T22:19:28.245420+0000","last_became_peered":"2024-09-20T22:19:28.245420+0000","last_unstale":"2024-09-20T22:22:11.578564+0000","last_undegraded":"2024-09-20T22:22:11.578564+0000","last_fullsized":"2024-09-20T22:22:11.578564+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:44.809724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:19:44.809724+0000","last_clean_scrub_stamp":"2024-09-20T22:19:44.809724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:17.063364+0000","last_change":"2024-09-20T22:22:17.062456+0000","last_active":"2024-09-20T22:22:17.063364+0000","last_peered":"2024-09-20T22:22:17.063364+0000","last_clean":"2024-09-20T22:22:17.063364+0000","last_became_active":"2024-09-20T22:22:17.062019+0000","last_became_peered":"2024-09-20T22:22:17.062019+0000","last_unstale":"2024-09-20T22:22:17.063364+0000","last_undegraded":"2024-09-20T22:22:17.063364+0000","last_fullsized":"2024-09-20T22:22:17.063364+0000","mapping_epoch":549,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":550,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:11.655870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:22:11.655870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:41.141276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"328'34","reported_seq":667,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.701139+0000","last_change":"2024-09-20T22:22:10.009868+0000","last_active":"2024-09-20T22:22:16.701139+0000","last_peered":"2024-09-20T22:22:16.701139+0000","last_clean":"2024-09-20T22:22:16.701139+0000","last_became_active":"2024-09-20T22:19:45.805481+0000","last_became_peered":"2024-09-20T22:19:45.805481+0000","last_unstale":"2024-09-20T22:22:16.701139+0000","last_undegraded":"2024-09-20T22:22:16.701139+0000","last_fullsized":"2024-09-20T22:22:16.701139+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'34","last_scrub_stamp":"2024-09-20T22:22:10.009795+0000","last_deep_scrub":"195'22","last_deep_scrub_stamp":"2024-09-20T22:08:39.355859+0000","last_clean_scrub_stamp":"2024-09-20T22:22:10.009795+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:18.889671+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":881,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.698825+0000","last_change":"2024-09-20T22:19:45.818815+0000","last_active":"2024-09-20T22:22:16.698825+0000","last_peered":"2024-09-20T22:22:16.698825+0000","last_clean":"2024-09-20T22:22:16.698825+0000","last_became_active":"2024-09-20T22:19:45.818397+0000","last_became_peered":"2024-09-20T22:19:45.818397+0000","last_unstale":"2024-09-20T22:22:16.698825+0000","last_undegraded":"2024-09-20T22:22:16.698825+0000","last_fullsized":"2024-09-20T22:22:16.698825+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.295028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:44.295028+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.295028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.18","version":"75'128","reported_seq":657,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592864+0000","last_change":"2024-09-20T22:17:56.357255+0000","last_active":"2024-09-20T22:22:16.592864+0000","last_peered":"2024-09-20T22:22:16.592864+0000","last_clean":"2024-09-20T22:22:16.592864+0000","last_became_active":"2024-09-20T22:17:44.717551+0000","last_became_peered":"2024-09-20T22:17:44.717551+0000","last_unstale":"2024-09-20T22:22:16.592864+0000","last_undegraded":"2024-09-20T22:22:16.592864+0000","last_fullsized":"2024-09-20T22:22:16.592864+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.357167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.357167+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":979,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578505+0000","last_change":"2024-09-20T22:18:31.015563+0000","last_active":"2024-09-20T22:22:11.578505+0000","last_peered":"2024-09-20T22:22:11.578505+0000","last_clean":"2024-09-20T22:22:11.578505+0000","last_became_active":"2024-09-20T22:12:21.521433+0000","last_became_peered":"2024-09-20T22:12:21.521433+0000","last_unstale":"2024-09-20T22:22:11.578505+0000","last_undegraded":"2024-09-20T22:22:11.578505+0000","last_fullsized":"2024-09-20T22:22:11.578505+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:31.015449+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:31.015449+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":280,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.700892+0000","last_change":"2024-09-20T22:19:45.802144+0000","last_active":"2024-09-20T22:22:16.700892+0000","last_peered":"2024-09-20T22:22:16.700892+0000","last_clean":"2024-09-20T22:22:16.700892+0000","last_became_active":"2024-09-20T22:19:45.801808+0000","last_became_peered":"2024-09-20T22:19:45.801808+0000","last_unstale":"2024-09-20T22:22:16.700892+0000","last_undegraded":"2024-09-20T22:22:16.700892+0000","last_fullsized":"2024-09-20T22:22:16.700892+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.456415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:44.456415+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.456415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,7],"acting":[2,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"328'23","reported_seq":956,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578442+0000","last_change":"2024-09-20T22:22:10.697410+0000","last_active":"2024-09-20T22:22:11.578442+0000","last_peered":"2024-09-20T22:22:11.578442+0000","last_clean":"2024-09-20T22:22:11.578442+0000","last_became_active":"2024-09-20T22:19:45.791170+0000","last_became_peered":"2024-09-20T22:19:45.791170+0000","last_unstale":"2024-09-20T22:22:11.578442+0000","last_undegraded":"2024-09-20T22:22:11.578442+0000","last_fullsized":"2024-09-20T22:22:11.578442+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'23","last_scrub_stamp":"2024-09-20T22:22:10.697295+0000","last_deep_scrub":"250'17","last_deep_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_clean_scrub_stamp":"2024-09-20T22:22:10.697295+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:20.186114+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":174,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.927474+0000","last_change":"2024-09-20T22:19:31.646518+0000","last_active":"2024-09-20T22:22:16.927474+0000","last_peered":"2024-09-20T22:22:16.927474+0000","last_clean":"2024-09-20T22:22:16.927474+0000","last_became_active":"2024-09-20T22:19:28.238293+0000","last_became_peered":"2024-09-20T22:19:28.238293+0000","last_unstale":"2024-09-20T22:22:16.927474+0000","last_undegraded":"2024-09-20T22:22:16.927474+0000","last_fullsized":"2024-09-20T22:22:16.927474+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:31.646417+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:31.646417+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7],"acting":[3,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1b","version":"75'128","reported_seq":978,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593227+0000","last_change":"2024-09-20T22:22:15.018084+0000","last_active":"2024-09-20T22:22:16.593227+0000","last_peered":"2024-09-20T22:22:16.593227+0000","last_clean":"2024-09-20T22:22:16.593227+0000","last_became_active":"2024-09-20T22:19:31.252076+0000","last_became_peered":"2024-09-20T22:19:31.252076+0000","last_unstale":"2024-09-20T22:22:16.593227+0000","last_undegraded":"2024-09-20T22:22:16.593227+0000","last_fullsized":"2024-09-20T22:22:16.593227+0000","mapping_epoch":475,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":476,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:15.018011+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:22:15.018011+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:15.254394+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":1024,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738604+0000","last_change":"2024-09-20T22:18:09.700077+0000","last_active":"2024-09-20T22:22:14.738604+0000","last_peered":"2024-09-20T22:22:14.738604+0000","last_clean":"2024-09-20T22:22:14.738604+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:22:14.738604+0000","last_undegraded":"2024-09-20T22:22:14.738604+0000","last_fullsized":"2024-09-20T22:22:14.738604+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:18:09.699725+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:18:09.699725+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"289'2","reported_seq":483,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.927517+0000","last_change":"2024-09-20T22:19:46.757972+0000","last_active":"2024-09-20T22:22:16.927517+0000","last_peered":"2024-09-20T22:22:16.927517+0000","last_clean":"2024-09-20T22:22:16.927517+0000","last_became_active":"2024-09-20T22:19:45.796939+0000","last_became_peered":"2024-09-20T22:19:45.796939+0000","last_unstale":"2024-09-20T22:22:16.927517+0000","last_undegraded":"2024-09-20T22:22:16.927517+0000","last_fullsized":"2024-09-20T22:22:16.927517+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-20T22:19:46.757900+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-20T22:11:37.188992+0000","last_clean_scrub_stamp":"2024-09-20T22:19:46.757900+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":46,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"328'20","reported_seq":518,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.699197+0000","last_change":"2024-09-20T22:18:10.346398+0000","last_active":"2024-09-20T22:22:16.699197+0000","last_peered":"2024-09-20T22:22:16.699197+0000","last_clean":"2024-09-20T22:22:16.699197+0000","last_became_active":"2024-09-20T22:12:52.526803+0000","last_became_peered":"2024-09-20T22:12:52.526803+0000","last_unstale":"2024-09-20T22:22:16.699197+0000","last_undegraded":"2024-09-20T22:22:16.699197+0000","last_fullsized":"2024-09-20T22:22:16.699197+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:18:10.346320+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:10.346320+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"475'12","reported_seq":1322,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578820+0000","last_change":"2024-09-20T22:19:28.274959+0000","last_active":"2024-09-20T22:22:11.578820+0000","last_peered":"2024-09-20T22:22:11.578820+0000","last_clean":"2024-09-20T22:22:11.578820+0000","last_became_active":"2024-09-20T22:19:28.256901+0000","last_became_peered":"2024-09-20T22:19:28.256901+0000","last_unstale":"2024-09-20T22:22:11.578820+0000","last_undegraded":"2024-09-20T22:22:11.578820+0000","last_fullsized":"2024-09-20T22:22:11.578820+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"291'11","last_scrub_stamp":"2024-09-20T22:18:33.978168+0000","last_deep_scrub":"291'11","last_deep_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_clean_scrub_stamp":"2024-09-20T22:18:33.978168+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":1061,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593059+0000","last_change":"2024-09-20T22:22:13.011302+0000","last_active":"2024-09-20T22:22:16.593059+0000","last_peered":"2024-09-20T22:22:16.593059+0000","last_clean":"2024-09-20T22:22:16.593059+0000","last_became_active":"2024-09-20T22:17:40.139526+0000","last_became_peered":"2024-09-20T22:17:40.139526+0000","last_unstale":"2024-09-20T22:22:16.593059+0000","last_undegraded":"2024-09-20T22:22:16.593059+0000","last_fullsized":"2024-09-20T22:22:16.593059+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:13.011208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:13.011208+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:28.826840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"328'27","reported_seq":835,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738474+0000","last_change":"2024-09-20T22:19:45.821849+0000","last_active":"2024-09-20T22:22:14.738474+0000","last_peered":"2024-09-20T22:22:14.738474+0000","last_clean":"2024-09-20T22:22:14.738474+0000","last_became_active":"2024-09-20T22:19:45.801225+0000","last_became_peered":"2024-09-20T22:19:45.801225+0000","last_unstale":"2024-09-20T22:22:14.738474+0000","last_undegraded":"2024-09-20T22:22:14.738474+0000","last_fullsized":"2024-09-20T22:22:14.738474+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:17:34.818375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:34.818375+0000","objects_scrubbed":7,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1076,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.927250+0000","last_change":"2024-09-20T22:22:09.970276+0000","last_active":"2024-09-20T22:22:16.927250+0000","last_peered":"2024-09-20T22:22:16.927250+0000","last_clean":"2024-09-20T22:22:16.927250+0000","last_became_active":"2024-09-20T22:19:45.797845+0000","last_became_peered":"2024-09-20T22:19:45.797845+0000","last_unstale":"2024-09-20T22:22:16.927250+0000","last_undegraded":"2024-09-20T22:22:16.927250+0000","last_fullsized":"2024-09-20T22:22:16.927250+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:09.970183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:19:17.513228+0000","last_clean_scrub_stamp":"2024-09-20T22:22:09.970183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:38.994419+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,5],"acting":[3,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"475'14","reported_seq":811,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593806+0000","last_change":"2024-09-20T22:19:28.252125+0000","last_active":"2024-09-20T22:22:16.593806+0000","last_peered":"2024-09-20T22:22:16.593806+0000","last_clean":"2024-09-20T22:22:16.593806+0000","last_became_active":"2024-09-20T22:19:28.242412+0000","last_became_peered":"2024-09-20T22:19:28.242412+0000","last_unstale":"2024-09-20T22:22:16.593806+0000","last_undegraded":"2024-09-20T22:22:16.593806+0000","last_fullsized":"2024-09-20T22:22:16.593806+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"326'13","last_scrub_stamp":"2024-09-20T22:18:40.407802+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:40.407802+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":589,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593963+0000","last_change":"2024-09-20T22:19:45.840327+0000","last_active":"2024-09-20T22:22:16.593963+0000","last_peered":"2024-09-20T22:22:16.593963+0000","last_clean":"2024-09-20T22:22:16.593963+0000","last_became_active":"2024-09-20T22:19:45.839873+0000","last_became_peered":"2024-09-20T22:19:45.839873+0000","last_unstale":"2024-09-20T22:22:16.593963+0000","last_undegraded":"2024-09-20T22:22:16.593963+0000","last_fullsized":"2024-09-20T22:22:16.593963+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:04.453445+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:04.453445+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":468,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.701217+0000","last_change":"2024-09-20T22:21:34.606254+0000","last_active":"2024-09-20T22:22:16.701217+0000","last_peered":"2024-09-20T22:22:16.701217+0000","last_clean":"2024-09-20T22:22:16.701217+0000","last_became_active":"2024-09-20T22:19:31.280713+0000","last_became_peered":"2024-09-20T22:19:31.280713+0000","last_unstale":"2024-09-20T22:22:16.701217+0000","last_undegraded":"2024-09-20T22:22:16.701217+0000","last_fullsized":"2024-09-20T22:22:16.701217+0000","mapping_epoch":475,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":476,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:34.606042+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:21:34.606042+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:45.282173+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,5],"acting":[2,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.0","version":"328'30","reported_seq":1047,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737436+0000","last_change":"2024-09-20T22:18:43.923440+0000","last_active":"2024-09-20T22:22:14.737436+0000","last_peered":"2024-09-20T22:22:14.737436+0000","last_clean":"2024-09-20T22:22:14.737436+0000","last_became_active":"2024-09-20T22:12:52.526867+0000","last_became_peered":"2024-09-20T22:12:52.526867+0000","last_unstale":"2024-09-20T22:22:14.737436+0000","last_undegraded":"2024-09-20T22:22:14.737436+0000","last_fullsized":"2024-09-20T22:22:14.737436+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'30","last_scrub_stamp":"2024-09-20T22:18:43.923335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:43.923335+0000","objects_scrubbed":8,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":528,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.739164+0000","last_change":"2024-09-20T22:18:45.931576+0000","last_active":"2024-09-20T22:22:14.739164+0000","last_peered":"2024-09-20T22:22:14.739164+0000","last_clean":"2024-09-20T22:22:14.739164+0000","last_became_active":"2024-09-20T22:12:52.528947+0000","last_became_peered":"2024-09-20T22:12:52.528947+0000","last_unstale":"2024-09-20T22:22:14.739164+0000","last_undegraded":"2024-09-20T22:22:14.739164+0000","last_fullsized":"2024-09-20T22:22:14.739164+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:45.931495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:45.931495+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":605,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.927801+0000","last_change":"2024-09-20T22:19:35.720326+0000","last_active":"2024-09-20T22:22:16.927801+0000","last_peered":"2024-09-20T22:22:16.927801+0000","last_clean":"2024-09-20T22:22:16.927801+0000","last_became_active":"2024-09-20T22:19:28.241950+0000","last_became_peered":"2024-09-20T22:19:28.241950+0000","last_unstale":"2024-09-20T22:22:16.927801+0000","last_undegraded":"2024-09-20T22:22:16.927801+0000","last_fullsized":"2024-09-20T22:22:16.927801+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:35.719732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:35.719732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":174,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.927362+0000","last_change":"2024-09-20T22:19:36.736788+0000","last_active":"2024-09-20T22:22:16.927362+0000","last_peered":"2024-09-20T22:22:16.927362+0000","last_clean":"2024-09-20T22:22:16.927362+0000","last_became_active":"2024-09-20T22:19:28.241638+0000","last_became_peered":"2024-09-20T22:19:28.241638+0000","last_unstale":"2024-09-20T22:22:16.927362+0000","last_undegraded":"2024-09-20T22:22:16.927362+0000","last_fullsized":"2024-09-20T22:22:16.927362+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:36.736314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:19:36.736314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,6],"acting":[3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":1104,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.927256+0000","last_change":"2024-09-20T22:22:04.005288+0000","last_active":"2024-09-20T22:22:16.927256+0000","last_peered":"2024-09-20T22:22:16.927256+0000","last_clean":"2024-09-20T22:22:16.927256+0000","last_became_active":"2024-09-20T22:15:45.642671+0000","last_became_peered":"2024-09-20T22:15:45.642671+0000","last_unstale":"2024-09-20T22:22:16.927256+0000","last_undegraded":"2024-09-20T22:22:16.927256+0000","last_fullsized":"2024-09-20T22:22:16.927256+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:04.005213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:04.005213+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:25.775753+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"328'28","reported_seq":792,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592758+0000","last_change":"2024-09-20T22:19:28.268499+0000","last_active":"2024-09-20T22:22:16.592758+0000","last_peered":"2024-09-20T22:22:16.592758+0000","last_clean":"2024-09-20T22:22:16.592758+0000","last_became_active":"2024-09-20T22:19:28.245672+0000","last_became_peered":"2024-09-20T22:19:28.245672+0000","last_unstale":"2024-09-20T22:22:16.592758+0000","last_undegraded":"2024-09-20T22:22:16.592758+0000","last_fullsized":"2024-09-20T22:22:16.592758+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:19:14.334175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:14.334175+0000","objects_scrubbed":7,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":41,"num_read_kb":34,"num_write":28,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":523,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.699450+0000","last_change":"2024-09-20T22:19:45.873383+0000","last_active":"2024-09-20T22:22:16.699450+0000","last_peered":"2024-09-20T22:22:16.699450+0000","last_clean":"2024-09-20T22:22:16.699450+0000","last_became_active":"2024-09-20T22:19:45.824809+0000","last_became_peered":"2024-09-20T22:19:45.824809+0000","last_unstale":"2024-09-20T22:22:16.699450+0000","last_undegraded":"2024-09-20T22:22:16.699450+0000","last_fullsized":"2024-09-20T22:22:16.699450+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:19:17.876492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:19:17.876492+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,5],"acting":[0,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"481'13","reported_seq":847,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.697556+0000","last_change":"2024-09-20T22:19:45.872368+0000","last_active":"2024-09-20T22:22:16.697556+0000","last_peered":"2024-09-20T22:22:16.697556+0000","last_clean":"2024-09-20T22:22:16.697556+0000","last_became_active":"2024-09-20T22:19:45.817931+0000","last_became_peered":"2024-09-20T22:19:45.817931+0000","last_unstale":"2024-09-20T22:22:16.697556+0000","last_undegraded":"2024-09-20T22:22:16.697556+0000","last_fullsized":"2024-09-20T22:22:16.697556+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:18:39.290911+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:39.290911+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1024,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738098+0000","last_change":"2024-09-20T22:18:11.668610+0000","last_active":"2024-09-20T22:22:14.738098+0000","last_peered":"2024-09-20T22:22:14.738098+0000","last_clean":"2024-09-20T22:22:14.738098+0000","last_became_active":"2024-09-20T22:02:56.349583+0000","last_became_peered":"2024-09-20T22:02:56.349583+0000","last_unstale":"2024-09-20T22:22:14.738098+0000","last_undegraded":"2024-09-20T22:22:14.738098+0000","last_fullsized":"2024-09-20T22:22:14.738098+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:11.668512+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:11.668512+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":1098,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578609+0000","last_change":"2024-09-20T22:21:39.704522+0000","last_active":"2024-09-20T22:22:11.578609+0000","last_peered":"2024-09-20T22:22:11.578609+0000","last_clean":"2024-09-20T22:22:11.578609+0000","last_became_active":"2024-09-20T22:17:40.142028+0000","last_became_peered":"2024-09-20T22:17:40.142028+0000","last_unstale":"2024-09-20T22:22:11.578609+0000","last_undegraded":"2024-09-20T22:22:11.578609+0000","last_fullsized":"2024-09-20T22:22:11.578609+0000","mapping_epoch":254,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:39.704410+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:21:39.704410+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:46.432362+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"328'20","reported_seq":640,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593337+0000","last_change":"2024-09-20T22:19:45.875079+0000","last_active":"2024-09-20T22:22:16.593337+0000","last_peered":"2024-09-20T22:22:16.593337+0000","last_clean":"2024-09-20T22:22:16.593337+0000","last_became_active":"2024-09-20T22:19:45.808494+0000","last_became_peered":"2024-09-20T22:19:45.808494+0000","last_unstale":"2024-09-20T22:22:16.593337+0000","last_undegraded":"2024-09-20T22:22:16.593337+0000","last_fullsized":"2024-09-20T22:22:16.593337+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:19:08.320252+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:08.320252+0000","objects_scrubbed":5,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":28,"num_read_kb":23,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1061,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593376+0000","last_change":"2024-09-20T22:19:44.874011+0000","last_active":"2024-09-20T22:22:16.593376+0000","last_peered":"2024-09-20T22:22:16.593376+0000","last_clean":"2024-09-20T22:22:16.593376+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:22:16.593376+0000","last_undegraded":"2024-09-20T22:22:16.593376+0000","last_fullsized":"2024-09-20T22:22:16.593376+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:44.873944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:19:44.873944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"363'13","reported_seq":606,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.699089+0000","last_change":"2024-09-20T22:20:58.946384+0000","last_active":"2024-09-20T22:22:16.699089+0000","last_peered":"2024-09-20T22:22:16.699089+0000","last_clean":"2024-09-20T22:22:16.699089+0000","last_became_active":"2024-09-20T22:12:57.884415+0000","last_became_peered":"2024-09-20T22:12:57.884415+0000","last_unstale":"2024-09-20T22:22:16.699089+0000","last_undegraded":"2024-09-20T22:22:16.699089+0000","last_fullsized":"2024-09-20T22:22:16.699089+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"363'13","last_scrub_stamp":"2024-09-20T22:20:58.946239+0000","last_deep_scrub":"363'13","last_deep_scrub_stamp":"2024-09-20T22:14:52.612615+0000","last_clean_scrub_stamp":"2024-09-20T22:20:58.946239+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":491,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.697839+0000","last_change":"2024-09-20T22:21:02.972285+0000","last_active":"2024-09-20T22:22:16.697839+0000","last_peered":"2024-09-20T22:22:16.697839+0000","last_clean":"2024-09-20T22:22:16.697839+0000","last_became_active":"2024-09-20T22:12:52.518529+0000","last_became_peered":"2024-09-20T22:12:52.518529+0000","last_unstale":"2024-09-20T22:22:16.697839+0000","last_undegraded":"2024-09-20T22:22:16.697839+0000","last_fullsized":"2024-09-20T22:22:16.697839+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:02.972172+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:21:02.972172+0000","last_clean_scrub_stamp":"2024-09-20T22:21:02.972172+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:27.662631+0000","scrub_duration":22,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":1099,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578278+0000","last_change":"2024-09-20T22:21:41.700453+0000","last_active":"2024-09-20T22:22:11.578278+0000","last_peered":"2024-09-20T22:22:11.578278+0000","last_clean":"2024-09-20T22:22:11.578278+0000","last_became_active":"2024-09-20T22:17:40.139979+0000","last_became_peered":"2024-09-20T22:17:40.139979+0000","last_unstale":"2024-09-20T22:22:11.578278+0000","last_undegraded":"2024-09-20T22:22:11.578278+0000","last_fullsized":"2024-09-20T22:22:11.578278+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:41.700323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:21:41.700323+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:00.974157+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"392'37","reported_seq":532,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.698828+0000","last_change":"2024-09-20T22:17:38.045665+0000","last_active":"2024-09-20T22:22:16.698828+0000","last_peered":"2024-09-20T22:22:16.698828+0000","last_clean":"2024-09-20T22:22:16.698828+0000","last_became_active":"2024-09-20T22:12:52.525538+0000","last_became_peered":"2024-09-20T22:12:52.525538+0000","last_unstale":"2024-09-20T22:22:16.698828+0000","last_undegraded":"2024-09-20T22:22:16.698828+0000","last_fullsized":"2024-09-20T22:22:16.698828+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"392'37","last_scrub_stamp":"2024-09-20T22:17:38.043643+0000","last_deep_scrub":"392'37","last_deep_scrub_stamp":"2024-09-20T22:15:56.127540+0000","last_clean_scrub_stamp":"2024-09-20T22:17:38.043643+0000","objects_scrubbed":8,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":53,"num_read_kb":39,"num_write":32,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":469,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.701730+0000","last_change":"2024-09-20T22:19:45.807161+0000","last_active":"2024-09-20T22:22:16.701730+0000","last_peered":"2024-09-20T22:22:16.701730+0000","last_clean":"2024-09-20T22:22:16.701730+0000","last_became_active":"2024-09-20T22:19:45.806472+0000","last_became_peered":"2024-09-20T22:19:45.806472+0000","last_unstale":"2024-09-20T22:22:16.701730+0000","last_undegraded":"2024-09-20T22:22:16.701730+0000","last_fullsized":"2024-09-20T22:22:16.701730+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:43.293797+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:43.293797+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1067,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577984+0000","last_change":"2024-09-20T22:19:18.540268+0000","last_active":"2024-09-20T22:22:11.577984+0000","last_peered":"2024-09-20T22:22:11.577984+0000","last_clean":"2024-09-20T22:22:11.577984+0000","last_became_active":"2024-09-20T22:12:52.519826+0000","last_became_peered":"2024-09-20T22:12:52.519826+0000","last_unstale":"2024-09-20T22:22:11.577984+0000","last_undegraded":"2024-09-20T22:22:11.577984+0000","last_fullsized":"2024-09-20T22:22:11.577984+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:18.540166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:19:18.540166+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":264,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577371+0000","last_change":"2024-09-20T22:21:42.695919+0000","last_active":"2024-09-20T22:22:11.577371+0000","last_peered":"2024-09-20T22:22:11.577371+0000","last_clean":"2024-09-20T22:22:11.577371+0000","last_became_active":"2024-09-20T22:19:45.791792+0000","last_became_peered":"2024-09-20T22:19:45.791792+0000","last_unstale":"2024-09-20T22:22:11.577371+0000","last_undegraded":"2024-09-20T22:22:11.577371+0000","last_fullsized":"2024-09-20T22:22:11.577371+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:42.694648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:21:42.694648+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:49.660066+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,7],"acting":[6,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":1071,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592519+0000","last_change":"2024-09-20T22:18:41.457692+0000","last_active":"2024-09-20T22:22:16.592519+0000","last_peered":"2024-09-20T22:22:16.592519+0000","last_clean":"2024-09-20T22:22:16.592519+0000","last_became_active":"2024-09-20T22:15:45.627135+0000","last_became_peered":"2024-09-20T22:15:45.627135+0000","last_unstale":"2024-09-20T22:22:16.592519+0000","last_undegraded":"2024-09-20T22:22:16.592519+0000","last_fullsized":"2024-09-20T22:22:16.592519+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:18:41.457600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:18:41.457600+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"481'53","reported_seq":777,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.701595+0000","last_change":"2024-09-20T22:21:43.738362+0000","last_active":"2024-09-20T22:22:16.701595+0000","last_peered":"2024-09-20T22:22:16.701595+0000","last_clean":"2024-09-20T22:22:16.701595+0000","last_became_active":"2024-09-20T22:19:45.805924+0000","last_became_peered":"2024-09-20T22:19:45.805924+0000","last_unstale":"2024-09-20T22:22:16.701595+0000","last_undegraded":"2024-09-20T22:22:16.701595+0000","last_fullsized":"2024-09-20T22:22:16.701595+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"481'53","last_scrub_stamp":"2024-09-20T22:21:43.737956+0000","last_deep_scrub":"328'52","last_deep_scrub_stamp":"2024-09-20T22:17:08.395578+0000","last_clean_scrub_stamp":"2024-09-20T22:21:43.737956+0000","objects_scrubbed":10,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:44.477449+0000","scrub_duration":16,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":64,"num_read_kb":49,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":358,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":754,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.699617+0000","last_change":"2024-09-20T22:22:06.131270+0000","last_active":"2024-09-20T22:22:16.699617+0000","last_peered":"2024-09-20T22:22:16.699617+0000","last_clean":"2024-09-20T22:22:16.699617+0000","last_became_active":"2024-09-20T22:19:45.810511+0000","last_became_peered":"2024-09-20T22:19:45.810511+0000","last_unstale":"2024-09-20T22:22:16.699617+0000","last_undegraded":"2024-09-20T22:22:16.699617+0000","last_fullsized":"2024-09-20T22:22:16.699617+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:06.131160+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:06.131160+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:27.704618+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":1067,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593585+0000","last_change":"2024-09-20T22:21:40.841813+0000","last_active":"2024-09-20T22:22:16.593585+0000","last_peered":"2024-09-20T22:22:16.593585+0000","last_clean":"2024-09-20T22:22:16.593585+0000","last_became_active":"2024-09-20T22:19:45.808162+0000","last_became_peered":"2024-09-20T22:19:45.808162+0000","last_unstale":"2024-09-20T22:22:16.593585+0000","last_undegraded":"2024-09-20T22:22:16.593585+0000","last_fullsized":"2024-09-20T22:22:16.593585+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:21:40.841702+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_clean_scrub_stamp":"2024-09-20T22:21:40.841702+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:56.405290+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":503,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.698828+0000","last_change":"2024-09-20T22:19:45.815814+0000","last_active":"2024-09-20T22:22:16.698828+0000","last_peered":"2024-09-20T22:22:16.698828+0000","last_clean":"2024-09-20T22:22:16.698828+0000","last_became_active":"2024-09-20T22:19:45.815528+0000","last_became_peered":"2024-09-20T22:19:45.815528+0000","last_unstale":"2024-09-20T22:22:16.698828+0000","last_undegraded":"2024-09-20T22:22:16.698828+0000","last_fullsized":"2024-09-20T22:22:16.698828+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:57.210706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.210706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1053,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.928138+0000","last_change":"2024-09-20T22:22:11.655576+0000","last_active":"2024-09-20T22:22:16.928138+0000","last_peered":"2024-09-20T22:22:16.928138+0000","last_clean":"2024-09-20T22:22:16.928138+0000","last_became_active":"2024-09-20T22:22:11.655188+0000","last_became_peered":"2024-09-20T22:22:11.655188+0000","last_unstale":"2024-09-20T22:22:16.928138+0000","last_undegraded":"2024-09-20T22:22:16.928138+0000","last_fullsized":"2024-09-20T22:22:16.928138+0000","mapping_epoch":545,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":546,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:43.806828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:34.953944+0000","last_clean_scrub_stamp":"2024-09-20T22:19:43.806828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":1076,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737550+0000","last_change":"2024-09-20T22:19:31.274033+0000","last_active":"2024-09-20T22:22:14.737550+0000","last_peered":"2024-09-20T22:22:14.737550+0000","last_clean":"2024-09-20T22:22:14.737550+0000","last_became_active":"2024-09-20T22:19:31.273654+0000","last_became_peered":"2024-09-20T22:19:31.273654+0000","last_unstale":"2024-09-20T22:22:14.737550+0000","last_undegraded":"2024-09-20T22:22:14.737550+0000","last_fullsized":"2024-09-20T22:22:14.737550+0000","mapping_epoch":475,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":476,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:38.946300+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:38.946300+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,2],"acting":[7,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"328'17","reported_seq":651,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593975+0000","last_change":"2024-09-20T22:21:33.920405+0000","last_active":"2024-09-20T22:22:16.593975+0000","last_peered":"2024-09-20T22:22:16.593975+0000","last_clean":"2024-09-20T22:22:16.593975+0000","last_became_active":"2024-09-20T22:19:45.810715+0000","last_became_peered":"2024-09-20T22:19:45.810715+0000","last_unstale":"2024-09-20T22:22:16.593975+0000","last_undegraded":"2024-09-20T22:22:16.593975+0000","last_fullsized":"2024-09-20T22:22:16.593975+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'17","last_scrub_stamp":"2024-09-20T22:21:33.920310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:21:33.920310+0000","objects_scrubbed":5,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:41.502954+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":23,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":990,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":273,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.700921+0000","last_change":"2024-09-20T22:21:42.696816+0000","last_active":"2024-09-20T22:22:16.700921+0000","last_peered":"2024-09-20T22:22:16.700921+0000","last_clean":"2024-09-20T22:22:16.700921+0000","last_became_active":"2024-09-20T22:19:45.807340+0000","last_became_peered":"2024-09-20T22:19:45.807340+0000","last_unstale":"2024-09-20T22:22:16.700921+0000","last_undegraded":"2024-09-20T22:22:16.700921+0000","last_fullsized":"2024-09-20T22:22:16.700921+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:42.696690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:21:42.696690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:11.856661+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":1109,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.700682+0000","last_change":"2024-09-20T22:19:13.810454+0000","last_active":"2024-09-20T22:22:16.700682+0000","last_peered":"2024-09-20T22:22:16.700682+0000","last_clean":"2024-09-20T22:22:16.700682+0000","last_became_active":"2024-09-20T22:12:52.517255+0000","last_became_peered":"2024-09-20T22:12:52.517255+0000","last_unstale":"2024-09-20T22:22:16.700682+0000","last_undegraded":"2024-09-20T22:22:16.700682+0000","last_fullsized":"2024-09-20T22:22:16.700682+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:19:13.810352+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:19:13.810352+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1062,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.701414+0000","last_change":"2024-09-20T22:19:08.895449+0000","last_active":"2024-09-20T22:22:16.701414+0000","last_peered":"2024-09-20T22:22:16.701414+0000","last_clean":"2024-09-20T22:22:16.701414+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:22:16.701414+0000","last_undegraded":"2024-09-20T22:22:16.701414+0000","last_fullsized":"2024-09-20T22:22:16.701414+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:08.894168+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:19:08.894168+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1024,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738708+0000","last_change":"2024-09-20T22:18:35.868334+0000","last_active":"2024-09-20T22:22:14.738708+0000","last_peered":"2024-09-20T22:22:14.738708+0000","last_clean":"2024-09-20T22:22:14.738708+0000","last_became_active":"2024-09-20T22:02:56.349484+0000","last_became_peered":"2024-09-20T22:02:56.349484+0000","last_unstale":"2024-09-20T22:22:14.738708+0000","last_undegraded":"2024-09-20T22:22:14.738708+0000","last_fullsized":"2024-09-20T22:22:14.738708+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:35.868264+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:35.868264+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":441,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577617+0000","last_change":"2024-09-20T22:21:33.763221+0000","last_active":"2024-09-20T22:22:11.577617+0000","last_peered":"2024-09-20T22:22:11.577617+0000","last_clean":"2024-09-20T22:22:11.577617+0000","last_became_active":"2024-09-20T22:19:48.843794+0000","last_became_peered":"2024-09-20T22:19:48.843794+0000","last_unstale":"2024-09-20T22:22:11.577617+0000","last_undegraded":"2024-09-20T22:22:11.577617+0000","last_fullsized":"2024-09-20T22:22:11.577617+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:33.763116+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:21:33.763116+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:52.572140+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"328'24","reported_seq":551,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.699345+0000","last_change":"2024-09-20T22:22:03.105751+0000","last_active":"2024-09-20T22:22:16.699345+0000","last_peered":"2024-09-20T22:22:16.699345+0000","last_clean":"2024-09-20T22:22:16.699345+0000","last_became_active":"2024-09-20T22:19:45.819406+0000","last_became_peered":"2024-09-20T22:19:45.819406+0000","last_unstale":"2024-09-20T22:22:16.699345+0000","last_undegraded":"2024-09-20T22:22:16.699345+0000","last_fullsized":"2024-09-20T22:22:16.699345+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'24","last_scrub_stamp":"2024-09-20T22:22:03.105662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:03.105662+0000","objects_scrubbed":6,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:10.038202+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":24,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,7],"acting":[0,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":505,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.697834+0000","last_change":"2024-09-20T22:22:04.127512+0000","last_active":"2024-09-20T22:22:16.697834+0000","last_peered":"2024-09-20T22:22:16.697834+0000","last_clean":"2024-09-20T22:22:16.697834+0000","last_became_active":"2024-09-20T22:17:03.663870+0000","last_became_peered":"2024-09-20T22:17:03.663870+0000","last_unstale":"2024-09-20T22:22:16.697834+0000","last_undegraded":"2024-09-20T22:22:16.697834+0000","last_fullsized":"2024-09-20T22:22:16.697834+0000","mapping_epoch":415,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":416,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:04.127443+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:22:04.127443+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:23.980359+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":2233,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737618+0000","last_change":"2024-09-20T22:19:45.875808+0000","last_active":"2024-09-20T22:22:14.737618+0000","last_peered":"2024-09-20T22:22:14.737618+0000","last_clean":"2024-09-20T22:22:14.737618+0000","last_became_active":"2024-09-20T22:19:45.817469+0000","last_became_peered":"2024-09-20T22:19:45.817469+0000","last_unstale":"2024-09-20T22:22:14.737618+0000","last_undegraded":"2024-09-20T22:22:14.737618+0000","last_fullsized":"2024-09-20T22:22:14.737618+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:17:59.765217+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:17:59.765217+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1137,"num_read_kb":1132,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1167,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,3],"acting":[7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":597,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593767+0000","last_change":"2024-09-20T22:22:06.156934+0000","last_active":"2024-09-20T22:22:16.593767+0000","last_peered":"2024-09-20T22:22:16.593767+0000","last_clean":"2024-09-20T22:22:16.593767+0000","last_became_active":"2024-09-20T22:19:45.810608+0000","last_became_peered":"2024-09-20T22:19:45.810608+0000","last_unstale":"2024-09-20T22:22:16.593767+0000","last_undegraded":"2024-09-20T22:22:16.593767+0000","last_fullsized":"2024-09-20T22:22:16.593767+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:06.156848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:17:08.955790+0000","last_clean_scrub_stamp":"2024-09-20T22:22:06.156848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:22.327333+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":262,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593428+0000","last_change":"2024-09-20T22:19:45.790117+0000","last_active":"2024-09-20T22:22:16.593428+0000","last_peered":"2024-09-20T22:22:16.593428+0000","last_clean":"2024-09-20T22:22:16.593428+0000","last_became_active":"2024-09-20T22:19:45.789135+0000","last_became_peered":"2024-09-20T22:19:45.789135+0000","last_unstale":"2024-09-20T22:22:16.593428+0000","last_undegraded":"2024-09-20T22:22:16.593428+0000","last_fullsized":"2024-09-20T22:22:16.593428+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:40.325308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:40.325308+0000","last_clean_scrub_stamp":"2024-09-20T22:18:40.325308+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.5","version":"475'29","reported_seq":1340,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578723+0000","last_change":"2024-09-20T22:19:28.285619+0000","last_active":"2024-09-20T22:22:11.578723+0000","last_peered":"2024-09-20T22:22:11.578723+0000","last_clean":"2024-09-20T22:22:11.578723+0000","last_became_active":"2024-09-20T22:19:28.259634+0000","last_became_peered":"2024-09-20T22:19:28.259634+0000","last_unstale":"2024-09-20T22:22:11.578723+0000","last_undegraded":"2024-09-20T22:22:11.578723+0000","last_fullsized":"2024-09-20T22:22:11.578723+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:17:57.685068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.685068+0000","objects_scrubbed":8,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":43,"num_read_kb":33,"num_write":25,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":154,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":997,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592596+0000","last_change":"2024-09-20T22:19:11.606709+0000","last_active":"2024-09-20T22:22:16.592596+0000","last_peered":"2024-09-20T22:22:16.592596+0000","last_clean":"2024-09-20T22:22:16.592596+0000","last_became_active":"2024-09-20T22:12:21.519717+0000","last_became_peered":"2024-09-20T22:12:21.519717+0000","last_unstale":"2024-09-20T22:22:16.592596+0000","last_undegraded":"2024-09-20T22:22:16.592596+0000","last_fullsized":"2024-09-20T22:22:16.592596+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:11.606601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:36.729158+0000","last_clean_scrub_stamp":"2024-09-20T22:19:11.606601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":1066,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737173+0000","last_change":"2024-09-20T22:18:06.776685+0000","last_active":"2024-09-20T22:22:14.737173+0000","last_peered":"2024-09-20T22:22:14.737173+0000","last_clean":"2024-09-20T22:22:14.737173+0000","last_became_active":"2024-09-20T22:15:45.620972+0000","last_became_peered":"2024-09-20T22:15:45.620972+0000","last_unstale":"2024-09-20T22:22:14.737173+0000","last_undegraded":"2024-09-20T22:22:14.737173+0000","last_fullsized":"2024-09-20T22:22:14.737173+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:18:06.776546+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:18:06.776546+0000","last_clean_scrub_stamp":"2024-09-20T22:18:06.776546+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":618,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738764+0000","last_change":"2024-09-20T22:18:04.809849+0000","last_active":"2024-09-20T22:22:14.738764+0000","last_peered":"2024-09-20T22:22:14.738764+0000","last_clean":"2024-09-20T22:22:14.738764+0000","last_became_active":"2024-09-20T22:10:48.907271+0000","last_became_peered":"2024-09-20T22:10:48.907271+0000","last_unstale":"2024-09-20T22:22:14.738764+0000","last_undegraded":"2024-09-20T22:22:14.738764+0000","last_fullsized":"2024-09-20T22:22:14.738764+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:04.809565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_clean_scrub_stamp":"2024-09-20T22:18:04.809565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1064,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593877+0000","last_change":"2024-09-20T22:19:45.799513+0000","last_active":"2024-09-20T22:22:16.593877+0000","last_peered":"2024-09-20T22:22:16.593877+0000","last_clean":"2024-09-20T22:22:16.593877+0000","last_became_active":"2024-09-20T22:19:45.797853+0000","last_became_peered":"2024-09-20T22:19:45.797853+0000","last_unstale":"2024-09-20T22:22:16.593877+0000","last_undegraded":"2024-09-20T22:22:16.593877+0000","last_fullsized":"2024-09-20T22:22:16.593877+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:13.654339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:19:13.654339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"75'128","reported_seq":863,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.699672+0000","last_change":"2024-09-20T22:20:59.964237+0000","last_active":"2024-09-20T22:22:16.699672+0000","last_peered":"2024-09-20T22:22:16.699672+0000","last_clean":"2024-09-20T22:22:16.699672+0000","last_became_active":"2024-09-20T22:19:48.848609+0000","last_became_peered":"2024-09-20T22:19:48.848609+0000","last_unstale":"2024-09-20T22:22:16.699672+0000","last_undegraded":"2024-09-20T22:22:16.699672+0000","last_fullsized":"2024-09-20T22:22:16.699672+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:20:59.964107+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:20:59.964107+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"328'19","reported_seq":790,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578552+0000","last_change":"2024-09-20T22:19:45.806692+0000","last_active":"2024-09-20T22:22:11.578552+0000","last_peered":"2024-09-20T22:22:11.578552+0000","last_clean":"2024-09-20T22:22:11.578552+0000","last_became_active":"2024-09-20T22:19:45.786395+0000","last_became_peered":"2024-09-20T22:19:45.786395+0000","last_unstale":"2024-09-20T22:22:11.578552+0000","last_undegraded":"2024-09-20T22:22:11.578552+0000","last_fullsized":"2024-09-20T22:22:11.578552+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:18:36.321653+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:36.321653+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":181,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.700574+0000","last_change":"2024-09-20T22:22:08.966893+0000","last_active":"2024-09-20T22:22:16.700574+0000","last_peered":"2024-09-20T22:22:16.700574+0000","last_clean":"2024-09-20T22:22:16.700574+0000","last_became_active":"2024-09-20T22:19:45.800297+0000","last_became_peered":"2024-09-20T22:19:45.800297+0000","last_unstale":"2024-09-20T22:22:16.700574+0000","last_undegraded":"2024-09-20T22:22:16.700574+0000","last_fullsized":"2024-09-20T22:22:16.700574+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:08.966822+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:08.966822+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:23.677528+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1050,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593985+0000","last_change":"2024-09-20T22:19:28.241963+0000","last_active":"2024-09-20T22:22:16.593985+0000","last_peered":"2024-09-20T22:22:16.593985+0000","last_clean":"2024-09-20T22:22:16.593985+0000","last_became_active":"2024-09-20T22:19:28.241278+0000","last_became_peered":"2024-09-20T22:19:28.241278+0000","last_unstale":"2024-09-20T22:22:16.593985+0000","last_undegraded":"2024-09-20T22:22:16.593985+0000","last_fullsized":"2024-09-20T22:22:16.593985+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:04.545147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:04.545147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":986,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578750+0000","last_change":"2024-09-20T22:17:55.630149+0000","last_active":"2024-09-20T22:22:11.578750+0000","last_peered":"2024-09-20T22:22:11.578750+0000","last_clean":"2024-09-20T22:22:11.578750+0000","last_became_active":"2024-09-20T22:12:21.521195+0000","last_became_peered":"2024-09-20T22:12:21.521195+0000","last_unstale":"2024-09-20T22:22:11.578750+0000","last_undegraded":"2024-09-20T22:22:11.578750+0000","last_fullsized":"2024-09-20T22:22:11.578750+0000","mapping_epoch":309,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":310,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:55.630084+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:17:55.630084+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":821,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577167+0000","last_change":"2024-09-20T22:21:43.651683+0000","last_active":"2024-09-20T22:22:11.577167+0000","last_peered":"2024-09-20T22:22:11.577167+0000","last_clean":"2024-09-20T22:22:11.577167+0000","last_became_active":"2024-09-20T22:17:40.139875+0000","last_became_peered":"2024-09-20T22:17:40.139875+0000","last_unstale":"2024-09-20T22:22:11.577167+0000","last_undegraded":"2024-09-20T22:22:11.577167+0000","last_fullsized":"2024-09-20T22:22:11.577167+0000","mapping_epoch":358,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:43.651464+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:21:43.651464+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:52.773176+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"328'31","reported_seq":828,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.701325+0000","last_change":"2024-09-20T22:19:45.976627+0000","last_active":"2024-09-20T22:22:16.701325+0000","last_peered":"2024-09-20T22:22:16.701325+0000","last_clean":"2024-09-20T22:22:16.701325+0000","last_became_active":"2024-09-20T22:19:45.833530+0000","last_became_peered":"2024-09-20T22:19:45.833530+0000","last_unstale":"2024-09-20T22:22:16.701325+0000","last_undegraded":"2024-09-20T22:22:16.701325+0000","last_fullsized":"2024-09-20T22:22:16.701325+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'31","last_scrub_stamp":"2024-09-20T22:19:12.269851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:12.269851+0000","objects_scrubbed":8,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":49,"num_read_kb":39,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,0],"acting":[2,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1067,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578430+0000","last_change":"2024-09-20T22:19:45.798910+0000","last_active":"2024-09-20T22:22:11.578430+0000","last_peered":"2024-09-20T22:22:11.578430+0000","last_clean":"2024-09-20T22:22:11.578430+0000","last_became_active":"2024-09-20T22:19:45.798363+0000","last_became_peered":"2024-09-20T22:19:45.798363+0000","last_unstale":"2024-09-20T22:22:11.578430+0000","last_undegraded":"2024-09-20T22:22:11.578430+0000","last_fullsized":"2024-09-20T22:22:11.578430+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:12.437986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:17:08.846904+0000","last_clean_scrub_stamp":"2024-09-20T22:19:12.437986+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":881,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.698389+0000","last_change":"2024-09-20T22:19:45.809454+0000","last_active":"2024-09-20T22:22:16.698389+0000","last_peered":"2024-09-20T22:22:16.698389+0000","last_clean":"2024-09-20T22:22:16.698389+0000","last_became_active":"2024-09-20T22:19:45.808291+0000","last_became_peered":"2024-09-20T22:19:45.808291+0000","last_unstale":"2024-09-20T22:22:16.698389+0000","last_undegraded":"2024-09-20T22:22:16.698389+0000","last_fullsized":"2024-09-20T22:22:16.698389+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:10.179939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:10.179939+0000","last_clean_scrub_stamp":"2024-09-20T22:18:10.179939+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,7],"acting":[0,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1064,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.701686+0000","last_change":"2024-09-20T22:22:10.991785+0000","last_active":"2024-09-20T22:22:16.701686+0000","last_peered":"2024-09-20T22:22:16.701686+0000","last_clean":"2024-09-20T22:22:16.701686+0000","last_became_active":"2024-09-20T22:19:28.261628+0000","last_became_peered":"2024-09-20T22:19:28.261628+0000","last_unstale":"2024-09-20T22:22:16.701686+0000","last_undegraded":"2024-09-20T22:22:16.701686+0000","last_fullsized":"2024-09-20T22:22:16.701686+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:10.991691+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:10.991691+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:15.249186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":1328,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737672+0000","last_change":"2024-09-20T22:17:42.025688+0000","last_active":"2024-09-20T22:22:14.737672+0000","last_peered":"2024-09-20T22:22:14.737672+0000","last_clean":"2024-09-20T22:22:14.737672+0000","last_became_active":"2024-09-20T22:17:40.146794+0000","last_became_peered":"2024-09-20T22:17:40.146794+0000","last_unstale":"2024-09-20T22:22:14.737672+0000","last_undegraded":"2024-09-20T22:22:14.737672+0000","last_fullsized":"2024-09-20T22:22:14.737672+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:42.025576+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:42.025576+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"328'25","reported_seq":793,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:12.703568+0000","last_change":"2024-09-20T22:22:12.703600+0000","last_active":"2024-09-20T22:22:12.703568+0000","last_peered":"2024-09-20T22:22:12.703568+0000","last_clean":"2024-09-20T22:22:12.703568+0000","last_became_active":"2024-09-20T22:19:45.791116+0000","last_became_peered":"2024-09-20T22:19:45.791116+0000","last_unstale":"2024-09-20T22:22:12.703568+0000","last_undegraded":"2024-09-20T22:22:12.703568+0000","last_fullsized":"2024-09-20T22:22:12.703568+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'25","last_scrub_stamp":"2024-09-20T22:22:12.703492+0000","last_deep_scrub":"328'25","last_deep_scrub_stamp":"2024-09-20T22:18:05.300862+0000","last_clean_scrub_stamp":"2024-09-20T22:22:12.703492+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:34.704284+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":24,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":523,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.698882+0000","last_change":"2024-09-20T22:22:16.089014+0000","last_active":"2024-09-20T22:22:16.698882+0000","last_peered":"2024-09-20T22:22:16.698882+0000","last_clean":"2024-09-20T22:22:16.698882+0000","last_became_active":"2024-09-20T22:19:45.810979+0000","last_became_peered":"2024-09-20T22:19:45.810979+0000","last_unstale":"2024-09-20T22:22:16.698882+0000","last_undegraded":"2024-09-20T22:22:16.698882+0000","last_fullsized":"2024-09-20T22:22:16.698882+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:22:16.088938+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:22:16.088938+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:19.140449+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"283'11","reported_seq":1333,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.701276+0000","last_change":"2024-09-20T22:19:11.823449+0000","last_active":"2024-09-20T22:22:16.701276+0000","last_peered":"2024-09-20T22:22:16.701276+0000","last_clean":"2024-09-20T22:22:16.701276+0000","last_became_active":"2024-09-20T22:11:30.485356+0000","last_became_peered":"2024-09-20T22:11:30.485356+0000","last_unstale":"2024-09-20T22:22:16.701276+0000","last_undegraded":"2024-09-20T22:22:16.701276+0000","last_fullsized":"2024-09-20T22:22:16.701276+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'11","last_scrub_stamp":"2024-09-20T22:19:11.823322+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:19:11.823322+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":317,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.701302+0000","last_change":"2024-09-20T22:19:45.804591+0000","last_active":"2024-09-20T22:22:16.701302+0000","last_peered":"2024-09-20T22:22:16.701302+0000","last_clean":"2024-09-20T22:22:16.701302+0000","last_became_active":"2024-09-20T22:19:45.799390+0000","last_became_peered":"2024-09-20T22:19:45.799390+0000","last_unstale":"2024-09-20T22:22:16.701302+0000","last_undegraded":"2024-09-20T22:22:16.701302+0000","last_fullsized":"2024-09-20T22:22:16.701302+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:11.319469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:07:23.482372+0000","last_clean_scrub_stamp":"2024-09-20T22:19:11.319469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.e","version":"75'128","reported_seq":750,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.739440+0000","last_change":"2024-09-20T22:21:35.700257+0000","last_active":"2024-09-20T22:22:14.739440+0000","last_peered":"2024-09-20T22:22:14.739440+0000","last_clean":"2024-09-20T22:22:14.739440+0000","last_became_active":"2024-09-20T22:19:31.258800+0000","last_became_peered":"2024-09-20T22:19:31.258800+0000","last_unstale":"2024-09-20T22:22:14.739440+0000","last_undegraded":"2024-09-20T22:22:14.739440+0000","last_fullsized":"2024-09-20T22:22:14.739440+0000","mapping_epoch":475,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":476,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:35.700020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:21:35.700020+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:50.597949+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"512'19","reported_seq":554,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.698763+0000","last_change":"2024-09-20T22:21:05.960413+0000","last_active":"2024-09-20T22:22:16.698763+0000","last_peered":"2024-09-20T22:22:16.698763+0000","last_clean":"2024-09-20T22:22:16.698763+0000","last_became_active":"2024-09-20T22:19:45.820510+0000","last_became_peered":"2024-09-20T22:19:45.820510+0000","last_unstale":"2024-09-20T22:22:16.698763+0000","last_undegraded":"2024-09-20T22:22:16.698763+0000","last_fullsized":"2024-09-20T22:22:16.698763+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"512'19","last_scrub_stamp":"2024-09-20T22:21:05.960284+0000","last_deep_scrub":"512'19","last_deep_scrub_stamp":"2024-09-20T22:21:05.960284+0000","last_clean_scrub_stamp":"2024-09-20T22:21:05.960284+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:20.601201+0000","scrub_duration":13,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":26,"num_read_kb":20,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":152,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593297+0000","last_change":"2024-09-20T22:19:45.801745+0000","last_active":"2024-09-20T22:22:16.593297+0000","last_peered":"2024-09-20T22:22:16.593297+0000","last_clean":"2024-09-20T22:22:16.593297+0000","last_became_active":"2024-09-20T22:19:45.788623+0000","last_became_peered":"2024-09-20T22:19:45.788623+0000","last_unstale":"2024-09-20T22:22:16.593297+0000","last_undegraded":"2024-09-20T22:22:16.593297+0000","last_fullsized":"2024-09-20T22:22:16.593297+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:19:37.837883+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:19:37.837883+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":993,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1068,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.700762+0000","last_change":"2024-09-20T22:19:28.247348+0000","last_active":"2024-09-20T22:22:16.700762+0000","last_peered":"2024-09-20T22:22:16.700762+0000","last_clean":"2024-09-20T22:22:16.700762+0000","last_became_active":"2024-09-20T22:19:28.246512+0000","last_became_peered":"2024-09-20T22:19:28.246512+0000","last_unstale":"2024-09-20T22:22:16.700762+0000","last_undegraded":"2024-09-20T22:22:16.700762+0000","last_fullsized":"2024-09-20T22:22:16.700762+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:40.088194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:18:40.088194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":497,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.697794+0000","last_change":"2024-09-20T22:19:45.813316+0000","last_active":"2024-09-20T22:22:16.697794+0000","last_peered":"2024-09-20T22:22:16.697794+0000","last_clean":"2024-09-20T22:22:16.697794+0000","last_became_active":"2024-09-20T22:19:45.813003+0000","last_became_peered":"2024-09-20T22:19:45.813003+0000","last_unstale":"2024-09-20T22:22:16.697794+0000","last_undegraded":"2024-09-20T22:22:16.697794+0000","last_fullsized":"2024-09-20T22:22:16.697794+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:12.806158+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:19:12.806158+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"75'128","reported_seq":1101,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578893+0000","last_change":"2024-09-20T22:19:31.262868+0000","last_active":"2024-09-20T22:22:11.578893+0000","last_peered":"2024-09-20T22:22:11.578893+0000","last_clean":"2024-09-20T22:22:11.578893+0000","last_became_active":"2024-09-20T22:19:31.262445+0000","last_became_peered":"2024-09-20T22:19:31.262445+0000","last_unstale":"2024-09-20T22:22:11.578893+0000","last_undegraded":"2024-09-20T22:22:11.578893+0000","last_fullsized":"2024-09-20T22:22:11.578893+0000","mapping_epoch":475,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":476,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:11.475678+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:17:07.885346+0000","last_clean_scrub_stamp":"2024-09-20T22:19:11.475678+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,3],"acting":[6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"328'35","reported_seq":1176,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578146+0000","last_change":"2024-09-20T22:19:45.813869+0000","last_active":"2024-09-20T22:22:11.578146+0000","last_peered":"2024-09-20T22:22:11.578146+0000","last_clean":"2024-09-20T22:22:11.578146+0000","last_became_active":"2024-09-20T22:19:45.790684+0000","last_became_peered":"2024-09-20T22:19:45.790684+0000","last_unstale":"2024-09-20T22:22:11.578146+0000","last_undegraded":"2024-09-20T22:22:11.578146+0000","last_fullsized":"2024-09-20T22:22:11.578146+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'35","last_scrub_stamp":"2024-09-20T22:19:16.424262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:16.424262+0000","objects_scrubbed":9,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":36,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,5],"acting":[6,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1047,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738865+0000","last_change":"2024-09-20T22:17:37.950739+0000","last_active":"2024-09-20T22:22:14.738865+0000","last_peered":"2024-09-20T22:22:14.738865+0000","last_clean":"2024-09-20T22:22:14.738865+0000","last_became_active":"2024-09-20T22:11:30.488650+0000","last_became_peered":"2024-09-20T22:11:30.488650+0000","last_unstale":"2024-09-20T22:22:14.738865+0000","last_undegraded":"2024-09-20T22:22:14.738865+0000","last_fullsized":"2024-09-20T22:22:14.738865+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:37.950674+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:12:46.602035+0000","last_clean_scrub_stamp":"2024-09-20T22:17:37.950674+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1086,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.700982+0000","last_change":"2024-09-20T22:18:11.941197+0000","last_active":"2024-09-20T22:22:16.700982+0000","last_peered":"2024-09-20T22:22:16.700982+0000","last_clean":"2024-09-20T22:22:16.700982+0000","last_became_active":"2024-09-20T22:12:52.515295+0000","last_became_peered":"2024-09-20T22:12:52.515295+0000","last_unstale":"2024-09-20T22:22:16.700982+0000","last_undegraded":"2024-09-20T22:22:16.700982+0000","last_fullsized":"2024-09-20T22:22:16.700982+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:11.941075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_clean_scrub_stamp":"2024-09-20T22:18:11.941075+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":43,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":640,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737214+0000","last_change":"2024-09-20T22:18:44.881893+0000","last_active":"2024-09-20T22:22:14.737214+0000","last_peered":"2024-09-20T22:22:14.737214+0000","last_clean":"2024-09-20T22:22:14.737214+0000","last_became_active":"2024-09-20T22:10:30.657116+0000","last_became_peered":"2024-09-20T22:10:30.657116+0000","last_unstale":"2024-09-20T22:22:14.737214+0000","last_undegraded":"2024-09-20T22:22:14.737214+0000","last_fullsized":"2024-09-20T22:22:14.737214+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.881819+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.881819+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"75'128","reported_seq":1131,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.927589+0000","last_change":"2024-09-20T22:22:04.994705+0000","last_active":"2024-09-20T22:22:16.927589+0000","last_peered":"2024-09-20T22:22:16.927589+0000","last_clean":"2024-09-20T22:22:16.927589+0000","last_became_active":"2024-09-20T22:17:40.152364+0000","last_became_peered":"2024-09-20T22:17:40.152364+0000","last_unstale":"2024-09-20T22:22:16.927589+0000","last_undegraded":"2024-09-20T22:22:16.927589+0000","last_fullsized":"2024-09-20T22:22:16.927589+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:04.994620+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:04.994620+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:20.725429+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"418'23","reported_seq":967,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.927650+0000","last_change":"2024-09-20T22:21:36.956588+0000","last_active":"2024-09-20T22:22:16.927650+0000","last_peered":"2024-09-20T22:22:16.927650+0000","last_clean":"2024-09-20T22:22:16.927650+0000","last_became_active":"2024-09-20T22:19:45.816632+0000","last_became_peered":"2024-09-20T22:19:45.816632+0000","last_unstale":"2024-09-20T22:22:16.927650+0000","last_undegraded":"2024-09-20T22:22:16.927650+0000","last_fullsized":"2024-09-20T22:22:16.927650+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"418'23","last_scrub_stamp":"2024-09-20T22:21:36.956465+0000","last_deep_scrub":"418'23","last_deep_scrub_stamp":"2024-09-20T22:17:11.338821+0000","last_clean_scrub_stamp":"2024-09-20T22:21:36.956465+0000","objects_scrubbed":5,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:05.245014+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":34,"num_read_kb":25,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1056,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.739023+0000","last_change":"2024-09-20T22:19:45.858377+0000","last_active":"2024-09-20T22:22:14.739023+0000","last_peered":"2024-09-20T22:22:14.739023+0000","last_clean":"2024-09-20T22:22:14.739023+0000","last_became_active":"2024-09-20T22:19:45.858043+0000","last_became_peered":"2024-09-20T22:19:45.858043+0000","last_unstale":"2024-09-20T22:22:14.739023+0000","last_undegraded":"2024-09-20T22:22:14.739023+0000","last_fullsized":"2024-09-20T22:22:14.739023+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:38.817089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:38.817089+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,5],"acting":[7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1089,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.700775+0000","last_change":"2024-09-20T22:19:45.806148+0000","last_active":"2024-09-20T22:22:16.700775+0000","last_peered":"2024-09-20T22:22:16.700775+0000","last_clean":"2024-09-20T22:22:16.700775+0000","last_became_active":"2024-09-20T22:19:45.802808+0000","last_became_peered":"2024-09-20T22:19:45.802808+0000","last_unstale":"2024-09-20T22:22:16.700775+0000","last_undegraded":"2024-09-20T22:22:16.700775+0000","last_fullsized":"2024-09-20T22:22:16.700775+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:03.959875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:03.959875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":522,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.927302+0000","last_change":"2024-09-20T22:22:03.002505+0000","last_active":"2024-09-20T22:22:16.927302+0000","last_peered":"2024-09-20T22:22:16.927302+0000","last_clean":"2024-09-20T22:22:16.927302+0000","last_became_active":"2024-09-20T22:19:45.798815+0000","last_became_peered":"2024-09-20T22:19:45.798815+0000","last_unstale":"2024-09-20T22:22:16.927302+0000","last_undegraded":"2024-09-20T22:22:16.927302+0000","last_fullsized":"2024-09-20T22:22:16.927302+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:03.002391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:03.002391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:25.958684+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'128","reported_seq":667,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738401+0000","last_change":"2024-09-20T22:19:48.871040+0000","last_active":"2024-09-20T22:22:14.738401+0000","last_peered":"2024-09-20T22:22:14.738401+0000","last_clean":"2024-09-20T22:22:14.738401+0000","last_became_active":"2024-09-20T22:19:48.870633+0000","last_became_peered":"2024-09-20T22:19:48.870633+0000","last_unstale":"2024-09-20T22:22:14.738401+0000","last_undegraded":"2024-09-20T22:22:14.738401+0000","last_fullsized":"2024-09-20T22:22:14.738401+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:40.996791+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:17:40.996791+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"416'34","reported_seq":612,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592868+0000","last_change":"2024-09-20T22:22:04.107105+0000","last_active":"2024-09-20T22:22:16.592868+0000","last_peered":"2024-09-20T22:22:16.592868+0000","last_clean":"2024-09-20T22:22:16.592868+0000","last_became_active":"2024-09-20T22:11:30.493275+0000","last_became_peered":"2024-09-20T22:11:30.493275+0000","last_unstale":"2024-09-20T22:22:16.592868+0000","last_undegraded":"2024-09-20T22:22:16.592868+0000","last_fullsized":"2024-09-20T22:22:16.592868+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"416'34","last_scrub_stamp":"2024-09-20T22:22:04.106984+0000","last_deep_scrub":"416'34","last_deep_scrub_stamp":"2024-09-20T22:17:04.960380+0000","last_clean_scrub_stamp":"2024-09-20T22:22:04.106984+0000","objects_scrubbed":7,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:18.483370+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":694,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738988+0000","last_change":"2024-09-20T22:21:37.623111+0000","last_active":"2024-09-20T22:22:14.738988+0000","last_peered":"2024-09-20T22:22:14.738988+0000","last_clean":"2024-09-20T22:22:14.738988+0000","last_became_active":"2024-09-20T22:19:45.818725+0000","last_became_peered":"2024-09-20T22:19:45.818725+0000","last_unstale":"2024-09-20T22:22:14.738988+0000","last_undegraded":"2024-09-20T22:22:14.738988+0000","last_fullsized":"2024-09-20T22:22:14.738988+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:21:37.623026+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:21:37.623026+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:06.964973+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":460,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,5],"acting":[7,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"481'10","reported_seq":1289,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737394+0000","last_change":"2024-09-20T22:19:45.804947+0000","last_active":"2024-09-20T22:22:14.737394+0000","last_peered":"2024-09-20T22:22:14.737394+0000","last_clean":"2024-09-20T22:22:14.737394+0000","last_became_active":"2024-09-20T22:19:45.793150+0000","last_became_peered":"2024-09-20T22:19:45.793150+0000","last_unstale":"2024-09-20T22:22:14.737394+0000","last_undegraded":"2024-09-20T22:22:14.737394+0000","last_fullsized":"2024-09-20T22:22:14.737394+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'8","last_scrub_stamp":"2024-09-20T22:19:16.895874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:16.895874+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":583,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593439+0000","last_change":"2024-09-20T22:19:09.620194+0000","last_active":"2024-09-20T22:22:16.593439+0000","last_peered":"2024-09-20T22:22:16.593439+0000","last_clean":"2024-09-20T22:22:16.593439+0000","last_became_active":"2024-09-20T22:12:52.517758+0000","last_became_peered":"2024-09-20T22:12:52.517758+0000","last_unstale":"2024-09-20T22:22:16.593439+0000","last_undegraded":"2024-09-20T22:22:16.593439+0000","last_fullsized":"2024-09-20T22:22:16.593439+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:09.620092+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:19:09.620092+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"75'128","reported_seq":932,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592697+0000","last_change":"2024-09-20T22:19:48.841486+0000","last_active":"2024-09-20T22:22:16.592697+0000","last_peered":"2024-09-20T22:22:16.592697+0000","last_clean":"2024-09-20T22:22:16.592697+0000","last_became_active":"2024-09-20T22:19:48.841087+0000","last_became_peered":"2024-09-20T22:19:48.841087+0000","last_unstale":"2024-09-20T22:22:16.592697+0000","last_undegraded":"2024-09-20T22:22:16.592697+0000","last_fullsized":"2024-09-20T22:22:16.592697+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:06.524269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:19:06.524269+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,7],"acting":[5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1078,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577770+0000","last_change":"2024-09-20T22:18:00.786710+0000","last_active":"2024-09-20T22:22:11.577770+0000","last_peered":"2024-09-20T22:22:11.577770+0000","last_clean":"2024-09-20T22:22:11.577770+0000","last_became_active":"2024-09-20T22:12:52.516989+0000","last_became_peered":"2024-09-20T22:12:52.516989+0000","last_unstale":"2024-09-20T22:22:11.577770+0000","last_undegraded":"2024-09-20T22:22:11.577770+0000","last_fullsized":"2024-09-20T22:22:11.577770+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:00.786604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:00.786604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":17,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"328'12","reported_seq":1117,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.928138+0000","last_change":"2024-09-20T22:22:11.002144+0000","last_active":"2024-09-20T22:22:16.928138+0000","last_peered":"2024-09-20T22:22:16.928138+0000","last_clean":"2024-09-20T22:22:16.928138+0000","last_became_active":"2024-09-20T22:19:45.798694+0000","last_became_peered":"2024-09-20T22:19:45.798694+0000","last_unstale":"2024-09-20T22:22:16.928138+0000","last_undegraded":"2024-09-20T22:22:16.928138+0000","last_fullsized":"2024-09-20T22:22:16.928138+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'12","last_scrub_stamp":"2024-09-20T22:22:11.002033+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:11.002033+0000","objects_scrubbed":3,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:37.055926+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":18,"num_read_kb":15,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":577,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593750+0000","last_change":"2024-09-20T22:19:45.789577+0000","last_active":"2024-09-20T22:22:16.593750+0000","last_peered":"2024-09-20T22:22:16.593750+0000","last_clean":"2024-09-20T22:22:16.593750+0000","last_became_active":"2024-09-20T22:19:45.789220+0000","last_became_peered":"2024-09-20T22:19:45.789220+0000","last_unstale":"2024-09-20T22:22:16.593750+0000","last_undegraded":"2024-09-20T22:22:16.593750+0000","last_fullsized":"2024-09-20T22:22:16.593750+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:36.439030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:36.439030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1029,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738660+0000","last_change":"2024-09-20T22:22:06.626292+0000","last_active":"2024-09-20T22:22:14.738660+0000","last_peered":"2024-09-20T22:22:14.738660+0000","last_clean":"2024-09-20T22:22:14.738660+0000","last_became_active":"2024-09-20T22:19:28.249892+0000","last_became_peered":"2024-09-20T22:19:28.249892+0000","last_unstale":"2024-09-20T22:22:14.738660+0000","last_undegraded":"2024-09-20T22:22:14.738660+0000","last_fullsized":"2024-09-20T22:22:14.738660+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:06.626203+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:54.359586+0000","last_clean_scrub_stamp":"2024-09-20T22:22:06.626203+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:32.454372+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,5],"acting":[7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"75'128","reported_seq":862,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.927439+0000","last_change":"2024-09-20T22:21:34.911974+0000","last_active":"2024-09-20T22:22:16.927439+0000","last_peered":"2024-09-20T22:22:16.927439+0000","last_clean":"2024-09-20T22:22:16.927439+0000","last_became_active":"2024-09-20T22:19:48.853347+0000","last_became_peered":"2024-09-20T22:19:48.853347+0000","last_unstale":"2024-09-20T22:22:16.927439+0000","last_undegraded":"2024-09-20T22:22:16.927439+0000","last_fullsized":"2024-09-20T22:22:16.927439+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:34.911846+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:21:34.911846+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:02.540378+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"328'15","reported_seq":517,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.699547+0000","last_change":"2024-09-20T22:22:12.130246+0000","last_active":"2024-09-20T22:22:16.699547+0000","last_peered":"2024-09-20T22:22:16.699547+0000","last_clean":"2024-09-20T22:22:16.699547+0000","last_became_active":"2024-09-20T22:12:52.527106+0000","last_became_peered":"2024-09-20T22:12:52.527106+0000","last_unstale":"2024-09-20T22:22:16.699547+0000","last_undegraded":"2024-09-20T22:22:16.699547+0000","last_fullsized":"2024-09-20T22:22:16.699547+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:22:12.130164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:12.130164+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:14.805756+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1049,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737769+0000","last_change":"2024-09-20T22:22:08.665641+0000","last_active":"2024-09-20T22:22:14.737769+0000","last_peered":"2024-09-20T22:22:14.737769+0000","last_clean":"2024-09-20T22:22:14.737769+0000","last_became_active":"2024-09-20T22:19:28.249922+0000","last_became_peered":"2024-09-20T22:19:28.249922+0000","last_unstale":"2024-09-20T22:22:14.737769+0000","last_undegraded":"2024-09-20T22:22:14.737769+0000","last_fullsized":"2024-09-20T22:22:14.737769+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:08.665517+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:08.665517+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:10.102382+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":317,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.739079+0000","last_change":"2024-09-20T22:19:45.818793+0000","last_active":"2024-09-20T22:22:14.739079+0000","last_peered":"2024-09-20T22:22:14.739079+0000","last_clean":"2024-09-20T22:22:14.739079+0000","last_became_active":"2024-09-20T22:19:45.818298+0000","last_became_peered":"2024-09-20T22:19:45.818298+0000","last_unstale":"2024-09-20T22:22:14.739079+0000","last_undegraded":"2024-09-20T22:22:14.739079+0000","last_fullsized":"2024-09-20T22:22:14.739079+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:36.302138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_clean_scrub_stamp":"2024-09-20T22:18:36.302138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1078,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:17.063208+0000","last_change":"2024-09-20T22:22:17.063208+0000","last_active":"2024-09-20T22:22:17.063208+0000","last_peered":"2024-09-20T22:22:17.063208+0000","last_clean":"2024-09-20T22:22:17.063208+0000","last_became_active":"2024-09-20T22:22:17.062775+0000","last_became_peered":"2024-09-20T22:22:17.062775+0000","last_unstale":"2024-09-20T22:22:17.063208+0000","last_undegraded":"2024-09-20T22:22:17.063208+0000","last_fullsized":"2024-09-20T22:22:17.063208+0000","mapping_epoch":549,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":550,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:16.043342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:26.862279+0000","last_clean_scrub_stamp":"2024-09-20T22:22:16.043342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:29.267440+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,0],"acting":[2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"75'128","reported_seq":923,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.592706+0000","last_change":"2024-09-20T22:18:07.509064+0000","last_active":"2024-09-20T22:22:16.592706+0000","last_peered":"2024-09-20T22:22:16.592706+0000","last_clean":"2024-09-20T22:22:16.592706+0000","last_became_active":"2024-09-20T22:15:50.574865+0000","last_became_peered":"2024-09-20T22:15:50.574865+0000","last_unstale":"2024-09-20T22:22:16.592706+0000","last_undegraded":"2024-09-20T22:22:16.592706+0000","last_fullsized":"2024-09-20T22:22:16.592706+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:18:07.508939+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:18:07.508939+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"328'15","reported_seq":590,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593734+0000","last_change":"2024-09-20T22:18:39.362251+0000","last_active":"2024-09-20T22:22:16.593734+0000","last_peered":"2024-09-20T22:22:16.593734+0000","last_clean":"2024-09-20T22:22:16.593734+0000","last_became_active":"2024-09-20T22:11:30.493335+0000","last_became_peered":"2024-09-20T22:11:30.493335+0000","last_unstale":"2024-09-20T22:22:16.593734+0000","last_undegraded":"2024-09-20T22:22:16.593734+0000","last_fullsized":"2024-09-20T22:22:16.593734+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:18:39.362109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:18:39.362109+0000","objects_scrubbed":4,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":25,"num_read_kb":19,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":499,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.698476+0000","last_change":"2024-09-20T22:19:13.833938+0000","last_active":"2024-09-20T22:22:16.698476+0000","last_peered":"2024-09-20T22:22:16.698476+0000","last_clean":"2024-09-20T22:22:16.698476+0000","last_became_active":"2024-09-20T22:12:52.524149+0000","last_became_peered":"2024-09-20T22:12:52.524149+0000","last_unstale":"2024-09-20T22:22:16.698476+0000","last_undegraded":"2024-09-20T22:22:16.698476+0000","last_fullsized":"2024-09-20T22:22:16.698476+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:13.833672+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:14:56.538825+0000","last_clean_scrub_stamp":"2024-09-20T22:19:13.833672+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"315'11","reported_seq":656,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:15.595465+0000","last_change":"2024-09-20T22:22:15.595487+0000","last_active":"2024-09-20T22:22:15.595465+0000","last_peered":"2024-09-20T22:22:15.595465+0000","last_clean":"2024-09-20T22:22:15.595465+0000","last_became_active":"2024-09-20T22:12:33.544098+0000","last_became_peered":"2024-09-20T22:12:33.544098+0000","last_unstale":"2024-09-20T22:22:15.595465+0000","last_undegraded":"2024-09-20T22:22:15.595465+0000","last_fullsized":"2024-09-20T22:22:15.595465+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"315'11","last_scrub_stamp":"2024-09-20T22:22:15.595411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:15.595411+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:21.969870+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":642,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:17.031840+0000","last_change":"2024-09-20T22:22:17.031863+0000","last_active":"2024-09-20T22:22:17.031840+0000","last_peered":"2024-09-20T22:22:17.031840+0000","last_clean":"2024-09-20T22:22:17.031840+0000","last_became_active":"2024-09-20T22:19:28.245964+0000","last_became_peered":"2024-09-20T22:19:28.245964+0000","last_unstale":"2024-09-20T22:22:17.031840+0000","last_undegraded":"2024-09-20T22:22:17.031840+0000","last_fullsized":"2024-09-20T22:22:17.031840+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:17.031182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:03.323607+0000","last_clean_scrub_stamp":"2024-09-20T22:22:17.031182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:24.773972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.11","version":"75'128","reported_seq":949,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.698121+0000","last_change":"2024-09-20T22:19:05.735170+0000","last_active":"2024-09-20T22:22:16.698121+0000","last_peered":"2024-09-20T22:22:16.698121+0000","last_clean":"2024-09-20T22:22:16.698121+0000","last_became_active":"2024-09-20T22:15:50.612544+0000","last_became_peered":"2024-09-20T22:15:50.612544+0000","last_unstale":"2024-09-20T22:22:16.698121+0000","last_undegraded":"2024-09-20T22:22:16.698121+0000","last_fullsized":"2024-09-20T22:22:16.698121+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:05.735058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:19:05.735058+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"466'25","reported_seq":512,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738253+0000","last_change":"2024-09-20T22:19:45.871562+0000","last_active":"2024-09-20T22:22:14.738253+0000","last_peered":"2024-09-20T22:22:14.738253+0000","last_clean":"2024-09-20T22:22:14.738253+0000","last_became_active":"2024-09-20T22:19:45.809071+0000","last_became_peered":"2024-09-20T22:19:45.809071+0000","last_unstale":"2024-09-20T22:22:14.738253+0000","last_undegraded":"2024-09-20T22:22:14.738253+0000","last_fullsized":"2024-09-20T22:22:14.738253+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"466'25","last_scrub_stamp":"2024-09-20T22:19:16.310142+0000","last_deep_scrub":"466'25","last_deep_scrub_stamp":"2024-09-20T22:19:16.310142+0000","last_clean_scrub_stamp":"2024-09-20T22:19:16.310142+0000","objects_scrubbed":5,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1041,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738448+0000","last_change":"2024-09-20T22:19:28.251114+0000","last_active":"2024-09-20T22:22:14.738448+0000","last_peered":"2024-09-20T22:22:14.738448+0000","last_clean":"2024-09-20T22:22:14.738448+0000","last_became_active":"2024-09-20T22:19:28.250811+0000","last_became_peered":"2024-09-20T22:19:28.250811+0000","last_unstale":"2024-09-20T22:22:14.738448+0000","last_undegraded":"2024-09-20T22:22:14.738448+0000","last_fullsized":"2024-09-20T22:22:14.738448+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:06.001800+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:19:06.001800+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,6],"acting":[7,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"481'15","reported_seq":743,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593955+0000","last_change":"2024-09-20T22:21:36.865254+0000","last_active":"2024-09-20T22:22:16.593955+0000","last_peered":"2024-09-20T22:22:16.593955+0000","last_clean":"2024-09-20T22:22:16.593955+0000","last_became_active":"2024-09-20T22:19:45.809247+0000","last_became_peered":"2024-09-20T22:19:45.809247+0000","last_unstale":"2024-09-20T22:22:16.593955+0000","last_undegraded":"2024-09-20T22:22:16.593955+0000","last_fullsized":"2024-09-20T22:22:16.593955+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"481'15","last_scrub_stamp":"2024-09-20T22:21:36.865145+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:21:36.865145+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:45.209874+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3],"acting":[5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":498,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.699091+0000","last_change":"2024-09-20T22:21:40.964557+0000","last_active":"2024-09-20T22:22:16.699091+0000","last_peered":"2024-09-20T22:22:16.699091+0000","last_clean":"2024-09-20T22:22:16.699091+0000","last_became_active":"2024-09-20T22:12:52.523781+0000","last_became_peered":"2024-09-20T22:12:52.523781+0000","last_unstale":"2024-09-20T22:22:16.699091+0000","last_undegraded":"2024-09-20T22:22:16.699091+0000","last_fullsized":"2024-09-20T22:22:16.699091+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:40.964227+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:21:40.964227+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:52.822485+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.16","version":"75'128","reported_seq":744,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.698965+0000","last_change":"2024-09-20T22:21:00.998420+0000","last_active":"2024-09-20T22:22:16.698965+0000","last_peered":"2024-09-20T22:22:16.698965+0000","last_clean":"2024-09-20T22:22:16.698965+0000","last_became_active":"2024-09-20T22:19:48.855437+0000","last_became_peered":"2024-09-20T22:19:48.855437+0000","last_unstale":"2024-09-20T22:22:16.698965+0000","last_undegraded":"2024-09-20T22:22:16.698965+0000","last_fullsized":"2024-09-20T22:22:16.698965+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:00.998286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:21:00.998286+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"328'19","reported_seq":798,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578106+0000","last_change":"2024-09-20T22:19:45.788839+0000","last_active":"2024-09-20T22:22:11.578106+0000","last_peered":"2024-09-20T22:22:11.578106+0000","last_clean":"2024-09-20T22:22:11.578106+0000","last_became_active":"2024-09-20T22:19:45.788204+0000","last_became_peered":"2024-09-20T22:19:45.788204+0000","last_unstale":"2024-09-20T22:22:11.578106+0000","last_undegraded":"2024-09-20T22:22:11.578106+0000","last_fullsized":"2024-09-20T22:22:11.578106+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:19:18.385865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:18.385865+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":31,"num_read_kb":24,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":171,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738159+0000","last_change":"2024-09-20T22:21:43.594958+0000","last_active":"2024-09-20T22:22:14.738159+0000","last_peered":"2024-09-20T22:22:14.738159+0000","last_clean":"2024-09-20T22:22:14.738159+0000","last_became_active":"2024-09-20T22:19:28.260795+0000","last_became_peered":"2024-09-20T22:19:28.260795+0000","last_unstale":"2024-09-20T22:22:14.738159+0000","last_undegraded":"2024-09-20T22:22:14.738159+0000","last_fullsized":"2024-09-20T22:22:14.738159+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:43.594884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:21:43.594884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:49.149766+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":262,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593556+0000","last_change":"2024-09-20T22:19:45.798598+0000","last_active":"2024-09-20T22:22:16.593556+0000","last_peered":"2024-09-20T22:22:16.593556+0000","last_clean":"2024-09-20T22:22:16.593556+0000","last_became_active":"2024-09-20T22:19:45.797640+0000","last_became_peered":"2024-09-20T22:19:45.797640+0000","last_unstale":"2024-09-20T22:22:16.593556+0000","last_undegraded":"2024-09-20T22:22:16.593556+0000","last_fullsized":"2024-09-20T22:22:16.593556+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:40.528539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:40.528539+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":556,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.697892+0000","last_change":"2024-09-20T22:21:39.989384+0000","last_active":"2024-09-20T22:22:16.697892+0000","last_peered":"2024-09-20T22:22:16.697892+0000","last_clean":"2024-09-20T22:22:16.697892+0000","last_became_active":"2024-09-20T22:19:45.810800+0000","last_became_peered":"2024-09-20T22:19:45.810800+0000","last_unstale":"2024-09-20T22:22:16.697892+0000","last_undegraded":"2024-09-20T22:22:16.697892+0000","last_fullsized":"2024-09-20T22:22:16.697892+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:39.989289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:21:39.989289+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:58.373658+0000","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.17","version":"75'128","reported_seq":545,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.594124+0000","last_change":"2024-09-20T22:17:57.421052+0000","last_active":"2024-09-20T22:22:16.594124+0000","last_peered":"2024-09-20T22:22:16.594124+0000","last_clean":"2024-09-20T22:22:16.594124+0000","last_became_active":"2024-09-20T22:17:44.721755+0000","last_became_peered":"2024-09-20T22:17:44.721755+0000","last_unstale":"2024-09-20T22:22:16.594124+0000","last_undegraded":"2024-09-20T22:22:16.594124+0000","last_fullsized":"2024-09-20T22:22:16.594124+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:57.420935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:17:57.420935+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":18,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"328'19","reported_seq":647,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578469+0000","last_change":"2024-09-20T22:21:34.778780+0000","last_active":"2024-09-20T22:22:11.578469+0000","last_peered":"2024-09-20T22:22:11.578469+0000","last_clean":"2024-09-20T22:22:11.578469+0000","last_became_active":"2024-09-20T22:19:45.801107+0000","last_became_peered":"2024-09-20T22:19:45.801107+0000","last_unstale":"2024-09-20T22:22:11.578469+0000","last_undegraded":"2024-09-20T22:22:11.578469+0000","last_fullsized":"2024-09-20T22:22:11.578469+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:21:34.778616+0000","last_deep_scrub":"291'13","last_deep_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_clean_scrub_stamp":"2024-09-20T22:21:34.778616+0000","objects_scrubbed":4,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:38.758924+0000","scrub_duration":14,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,0],"acting":[6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":637,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738519+0000","last_change":"2024-09-20T22:19:28.243263+0000","last_active":"2024-09-20T22:22:14.738519+0000","last_peered":"2024-09-20T22:22:14.738519+0000","last_clean":"2024-09-20T22:22:14.738519+0000","last_became_active":"2024-09-20T22:19:28.242951+0000","last_became_peered":"2024-09-20T22:19:28.242951+0000","last_unstale":"2024-09-20T22:22:14.738519+0000","last_undegraded":"2024-09-20T22:22:14.738519+0000","last_fullsized":"2024-09-20T22:22:14.738519+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:54.900125+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:17:54.900125+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,0],"acting":[7,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":505,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.699149+0000","last_change":"2024-09-20T22:19:45.820095+0000","last_active":"2024-09-20T22:22:16.699149+0000","last_peered":"2024-09-20T22:22:16.699149+0000","last_clean":"2024-09-20T22:22:16.699149+0000","last_became_active":"2024-09-20T22:19:45.817319+0000","last_became_peered":"2024-09-20T22:19:45.817319+0000","last_unstale":"2024-09-20T22:22:16.699149+0000","last_undegraded":"2024-09-20T22:22:16.699149+0000","last_fullsized":"2024-09-20T22:22:16.699149+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:19:02.798998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:19:02.798998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,5],"acting":[0,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":505,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.697781+0000","last_change":"2024-09-20T22:22:11.135140+0000","last_active":"2024-09-20T22:22:16.697781+0000","last_peered":"2024-09-20T22:22:16.697781+0000","last_clean":"2024-09-20T22:22:16.697781+0000","last_became_active":"2024-09-20T22:19:45.818444+0000","last_became_peered":"2024-09-20T22:19:45.818444+0000","last_unstale":"2024-09-20T22:22:16.697781+0000","last_undegraded":"2024-09-20T22:22:16.697781+0000","last_fullsized":"2024-09-20T22:22:16.697781+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:11.135068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:11.135068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:15.398811+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,6],"acting":[0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"75'128","reported_seq":1116,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.701069+0000","last_change":"2024-09-20T22:19:48.880703+0000","last_active":"2024-09-20T22:22:16.701069+0000","last_peered":"2024-09-20T22:22:16.701069+0000","last_clean":"2024-09-20T22:22:16.701069+0000","last_became_active":"2024-09-20T22:19:48.880005+0000","last_became_peered":"2024-09-20T22:19:48.880005+0000","last_unstale":"2024-09-20T22:22:16.701069+0000","last_undegraded":"2024-09-20T22:22:16.701069+0000","last_fullsized":"2024-09-20T22:22:16.701069+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:19:09.890234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:19:09.890234+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,6],"acting":[2,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"328'19","reported_seq":960,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737568+0000","last_change":"2024-09-20T22:19:17.961328+0000","last_active":"2024-09-20T22:22:14.737568+0000","last_peered":"2024-09-20T22:22:14.737568+0000","last_clean":"2024-09-20T22:22:14.737568+0000","last_became_active":"2024-09-20T22:12:33.545199+0000","last_became_peered":"2024-09-20T22:12:33.545199+0000","last_unstale":"2024-09-20T22:22:14.737568+0000","last_undegraded":"2024-09-20T22:22:14.737568+0000","last_fullsized":"2024-09-20T22:22:14.737568+0000","mapping_epoch":314,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":315,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:19:17.961211+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:19:17.961211+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,6],"acting":[7,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":580,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593687+0000","last_change":"2024-09-20T22:19:45.795979+0000","last_active":"2024-09-20T22:22:16.593687+0000","last_peered":"2024-09-20T22:22:16.593687+0000","last_clean":"2024-09-20T22:22:16.593687+0000","last_became_active":"2024-09-20T22:19:45.795652+0000","last_became_peered":"2024-09-20T22:19:45.795652+0000","last_unstale":"2024-09-20T22:22:16.593687+0000","last_undegraded":"2024-09-20T22:22:16.593687+0000","last_fullsized":"2024-09-20T22:22:16.593687+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:55.398175+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_clean_scrub_stamp":"2024-09-20T22:17:55.398175+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1041,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737279+0000","last_change":"2024-09-20T22:21:44.577623+0000","last_active":"2024-09-20T22:22:14.737279+0000","last_peered":"2024-09-20T22:22:14.737279+0000","last_clean":"2024-09-20T22:22:14.737279+0000","last_became_active":"2024-09-20T22:19:28.260465+0000","last_became_peered":"2024-09-20T22:19:28.260465+0000","last_unstale":"2024-09-20T22:22:14.737279+0000","last_undegraded":"2024-09-20T22:22:14.737279+0000","last_fullsized":"2024-09-20T22:22:14.737279+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:44.577510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:21:44.577510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:55.402165+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":488,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.698986+0000","last_change":"2024-09-20T22:18:32.424564+0000","last_active":"2024-09-20T22:22:16.698986+0000","last_peered":"2024-09-20T22:22:16.698986+0000","last_clean":"2024-09-20T22:22:16.698986+0000","last_became_active":"2024-09-20T22:12:57.883998+0000","last_became_peered":"2024-09-20T22:12:57.883998+0000","last_unstale":"2024-09-20T22:22:16.698986+0000","last_undegraded":"2024-09-20T22:22:16.698986+0000","last_fullsized":"2024-09-20T22:22:16.698986+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":327,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:32.424321+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:32.424321+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.15","version":"75'128","reported_seq":551,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.738278+0000","last_change":"2024-09-20T22:19:48.849798+0000","last_active":"2024-09-20T22:22:14.738278+0000","last_peered":"2024-09-20T22:22:14.738278+0000","last_clean":"2024-09-20T22:22:14.738278+0000","last_became_active":"2024-09-20T22:19:48.849447+0000","last_became_peered":"2024-09-20T22:19:48.849447+0000","last_unstale":"2024-09-20T22:22:14.738278+0000","last_undegraded":"2024-09-20T22:22:14.738278+0000","last_fullsized":"2024-09-20T22:22:14.738278+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:17:56.859373+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:17:56.859373+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,3],"acting":[7,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"453'10","reported_seq":1084,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.927103+0000","last_change":"2024-09-20T22:21:38.901104+0000","last_active":"2024-09-20T22:22:16.927103+0000","last_peered":"2024-09-20T22:22:16.927103+0000","last_clean":"2024-09-20T22:22:16.927103+0000","last_became_active":"2024-09-20T22:19:45.795544+0000","last_became_peered":"2024-09-20T22:19:45.795544+0000","last_unstale":"2024-09-20T22:22:16.927103+0000","last_undegraded":"2024-09-20T22:22:16.927103+0000","last_fullsized":"2024-09-20T22:22:16.927103+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"453'10","last_scrub_stamp":"2024-09-20T22:21:38.900946+0000","last_deep_scrub":"453'10","last_deep_scrub_stamp":"2024-09-20T22:21:38.900946+0000","last_clean_scrub_stamp":"2024-09-20T22:21:38.900946+0000","objects_scrubbed":2,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:22:51.759107+0000","scrub_duration":11,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":10,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":490,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.699857+0000","last_change":"2024-09-20T22:18:09.367123+0000","last_active":"2024-09-20T22:22:16.699857+0000","last_peered":"2024-09-20T22:22:16.699857+0000","last_clean":"2024-09-20T22:22:16.699857+0000","last_became_active":"2024-09-20T22:12:52.517510+0000","last_became_peered":"2024-09-20T22:12:52.517510+0000","last_unstale":"2024-09-20T22:22:16.699857+0000","last_undegraded":"2024-09-20T22:22:16.699857+0000","last_fullsized":"2024-09-20T22:22:16.699857+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:09.366999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:18:09.366999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":47,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1060,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.577380+0000","last_change":"2024-09-20T22:19:28.261147+0000","last_active":"2024-09-20T22:22:11.577380+0000","last_peered":"2024-09-20T22:22:11.577380+0000","last_clean":"2024-09-20T22:22:11.577380+0000","last_became_active":"2024-09-20T22:19:28.260576+0000","last_became_peered":"2024-09-20T22:19:28.260576+0000","last_unstale":"2024-09-20T22:22:11.577380+0000","last_undegraded":"2024-09-20T22:22:11.577380+0000","last_fullsized":"2024-09-20T22:22:11.577380+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:44.105874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:18:44.105874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1029,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737721+0000","last_change":"2024-09-20T22:19:28.245693+0000","last_active":"2024-09-20T22:22:14.737721+0000","last_peered":"2024-09-20T22:22:14.737721+0000","last_clean":"2024-09-20T22:22:14.737721+0000","last_became_active":"2024-09-20T22:19:28.245306+0000","last_became_peered":"2024-09-20T22:19:28.245306+0000","last_unstale":"2024-09-20T22:22:14.737721+0000","last_undegraded":"2024-09-20T22:22:14.737721+0000","last_fullsized":"2024-09-20T22:22:14.737721+0000","mapping_epoch":472,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":473,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:17:33.935154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_clean_scrub_stamp":"2024-09-20T22:17:33.935154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1a","version":"75'128","reported_seq":475,"reported_epoch":548,"state":"active+clean","last_fresh":"2024-09-20T22:22:14.737652+0000","last_change":"2024-09-20T22:21:34.683687+0000","last_active":"2024-09-20T22:22:14.737652+0000","last_peered":"2024-09-20T22:22:14.737652+0000","last_clean":"2024-09-20T22:22:14.737652+0000","last_became_active":"2024-09-20T22:19:48.843466+0000","last_became_peered":"2024-09-20T22:19:48.843466+0000","last_unstale":"2024-09-20T22:22:14.737652+0000","last_undegraded":"2024-09-20T22:22:14.737652+0000","last_fullsized":"2024-09-20T22:22:14.737652+0000","mapping_epoch":483,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":484,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:21:34.683575+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:21:34.683575+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:03.901269+0000","scrub_duration":10,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":915,"reported_epoch":549,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.593913+0000","last_change":"2024-09-20T22:19:45.840129+0000","last_active":"2024-09-20T22:22:16.593913+0000","last_peered":"2024-09-20T22:22:16.593913+0000","last_clean":"2024-09-20T22:22:16.593913+0000","last_became_active":"2024-09-20T22:19:45.839497+0000","last_became_peered":"2024-09-20T22:19:45.839497+0000","last_unstale":"2024-09-20T22:22:16.593913+0000","last_undegraded":"2024-09-20T22:22:16.593913+0000","last_fullsized":"2024-09-20T22:22:16.593913+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:18:37.293600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:18:37.293600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":504,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.698058+0000","last_change":"2024-09-20T22:21:39.004287+0000","last_active":"2024-09-20T22:22:16.698058+0000","last_peered":"2024-09-20T22:22:16.698058+0000","last_clean":"2024-09-20T22:22:16.698058+0000","last_became_active":"2024-09-20T22:19:45.817473+0000","last_became_peered":"2024-09-20T22:19:45.817473+0000","last_unstale":"2024-09-20T22:22:16.698058+0000","last_undegraded":"2024-09-20T22:22:16.698058+0000","last_fullsized":"2024-09-20T22:22:16.698058+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:21:39.004178+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:21:39.004178+0000","last_clean_scrub_stamp":"2024-09-20T22:21:39.004178+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:03.825432+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"328'19","reported_seq":1101,"reported_epoch":546,"state":"active+clean","last_fresh":"2024-09-20T22:22:11.578946+0000","last_change":"2024-09-20T22:22:05.674913+0000","last_active":"2024-09-20T22:22:11.578946+0000","last_peered":"2024-09-20T22:22:11.578946+0000","last_clean":"2024-09-20T22:22:11.578946+0000","last_became_active":"2024-09-20T22:19:45.815971+0000","last_became_peered":"2024-09-20T22:19:45.815971+0000","last_unstale":"2024-09-20T22:22:11.578946+0000","last_undegraded":"2024-09-20T22:22:11.578946+0000","last_fullsized":"2024-09-20T22:22:11.578946+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:22:05.674812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:05.674812+0000","objects_scrubbed":5,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:08.236842+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":32,"num_read_kb":25,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,0],"acting":[6,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":499,"reported_epoch":550,"state":"active+clean","last_fresh":"2024-09-20T22:22:16.699074+0000","last_change":"2024-09-20T22:22:10.095005+0000","last_active":"2024-09-20T22:22:16.699074+0000","last_peered":"2024-09-20T22:22:16.699074+0000","last_clean":"2024-09-20T22:22:16.699074+0000","last_became_active":"2024-09-20T22:12:52.524425+0000","last_became_peered":"2024-09-20T22:12:52.524425+0000","last_unstale":"2024-09-20T22:22:16.699074+0000","last_undegraded":"2024-09-20T22:22:16.699074+0000","last_fullsized":"2024-09-20T22:22:16.699074+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:10.094934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:10.094934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:29.788661+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1105,"num_read_kb":862,"num_write":703,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":326,"num_bytes_recovered":6342,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":14368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":756,"ondisk_log_size":756,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":2991,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":7072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1137,"num_read_kb":1132,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1186,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":31,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3968,"ondisk_log_size":3968,"up":93,"acting":93,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813136109,"num_pgs":89,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":876812,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":875262,"kb_avail":92868340,"statfs":{"total":95995035648,"available":95097180160,"internally_reserved":0,"allocated":1560576,"data_stored":1039815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":896268746},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430446,"num_pgs":80,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":750504,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":748926,"kb_avail":92994648,"statfs":{"total":95995035648,"available":95226519552,"internally_reserved":0,"allocated":1589248,"data_stored":1089021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":766900678},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724789,"num_pgs":88,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":997572,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":996030,"kb_avail":92747580,"statfs":{"total":95995035648,"available":94973521920,"internally_reserved":0,"allocated":1552384,"data_stored":1039899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1019935173},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":0,"seq":2284922601479,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":0,"seq":622770258105,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149954,"num_pgs":81,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":943500,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":941886,"kb_avail":92801652,"statfs":{"total":95995035648,"available":95028891648,"internally_reserved":0,"allocated":1630208,"data_stored":1105683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":964491719},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411591,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":754620,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":753086,"kb_avail":92990532,"statfs":{"total":95995035648,"available":95222304768,"internally_reserved":0,"allocated":1540096,"data_stored":1053876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":771160517},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738642,"num_pgs":79,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1003232,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":1001598,"kb_avail":92741920,"statfs":{"total":95995035648,"available":94967726080,"internally_reserved":0,"allocated":1650688,"data_stored":1120343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1025636805},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":2134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":1804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:22:20.475 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-20T22:22:20.476 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-09-20T22:22:20.476 INFO:tasks.thrashosds.thrasher:Removing osd 6, in_osds are: [2, 3, 5, 6, 7, 0] 2024-09-20T22:22:20.476 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd out 6 2024-09-20T22:22:20.738 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:20.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:20 smithi067 ceph-mon[33511]: 5.b scrub starts 2024-09-20T22:22:20.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:20 smithi067 ceph-mon[33511]: 5.b scrub ok 2024-09-20T22:22:20.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:20 smithi060 ceph-mon[29694]: 5.b scrub starts 2024-09-20T22:22:20.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:20 smithi060 ceph-mon[29694]: 5.b scrub ok 2024-09-20T22:22:20.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:20 smithi060 ceph-mon[40290]: 5.b scrub starts 2024-09-20T22:22:20.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:20 smithi060 ceph-mon[40290]: 5.b scrub ok 2024-09-20T22:22:20.914 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.6 dump_historic_ops 2024-09-20T22:22:21.174 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:22:21.602 INFO:teuthology.orchestra.run.smithi060.stderr:marked out osd.6. 2024-09-20T22:22:21.767 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:21.767 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:21.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-mon[40290]: from='client.16722 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:22:21.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-mon[40290]: pgmap v1050: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:22:21.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-mon[40290]: 4.3 scrub starts 2024-09-20T22:22:21.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-mon[40290]: 4.3 scrub ok 2024-09-20T22:22:21.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:22:21.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:22:21.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/354179985' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-09-20T22:22:21.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-09-20T22:22:21.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-mon[40290]: Client client.admin marked osd.6 out, while it was still marked up 2024-09-20T22:22:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a[29671]: 2024-09-20T22:22:21.420+0000 7f01adcc6640 -1 mon.a@0(leader).osd e550 definitely_dead 0 2024-09-20T22:22:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-mon[29694]: from='client.16722 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:22:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-mon[29694]: pgmap v1050: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:22:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-mon[29694]: 4.3 scrub starts 2024-09-20T22:22:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-mon[29694]: 4.3 scrub ok 2024-09-20T22:22:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:22:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:22:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/354179985' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-09-20T22:22:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-09-20T22:22:21.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:21 smithi060 ceph-mon[29694]: Client client.admin marked osd.6 out, while it was still marked up 2024-09-20T22:22:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:21 smithi067 ceph-mon[33511]: from='client.16722 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:22:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:21 smithi067 ceph-mon[33511]: pgmap v1050: 162 pgs: 162 active+clean; 583 KiB data, 5.1 GiB used, 531 GiB / 536 GiB avail; 853 B/s rd, 0 op/s 2024-09-20T22:22:22.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:21 smithi067 ceph-mon[33511]: 4.3 scrub starts 2024-09-20T22:22:22.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:21 smithi067 ceph-mon[33511]: 4.3 scrub ok 2024-09-20T22:22:22.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:22:22.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:22:22.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:21 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/354179985' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-09-20T22:22:22.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:21 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd out", "ids": ["6"]}]: dispatch 2024-09-20T22:22:22.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:21 smithi067 ceph-mon[33511]: Client client.admin marked osd.6 out, while it was still marked up 2024-09-20T22:22:22.510 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_ops_in_flight 2024-09-20T22:22:22.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:22 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:22:22.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:22 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:22:22.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:22 smithi067 ceph-mon[33511]: 6.12 scrub starts 2024-09-20T22:22:22.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:22 smithi067 ceph-mon[33511]: 6.12 scrub ok 2024-09-20T22:22:22.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:22 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["6"]}]': finished 2024-09-20T22:22:22.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:22 smithi067 ceph-mon[33511]: osdmap e551: 8 total, 8 up, 5 in 2024-09-20T22:22:22.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:22 smithi067 ceph-mon[33511]: osdmap e552: 8 total, 8 up, 5 in 2024-09-20T22:22:22.775 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:22:22.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:22 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:22:22.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:22 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:22:22.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:22 smithi060 ceph-mon[29694]: 6.12 scrub starts 2024-09-20T22:22:22.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:22 smithi060 ceph-mon[29694]: 6.12 scrub ok 2024-09-20T22:22:22.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:22 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["6"]}]': finished 2024-09-20T22:22:22.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:22 smithi060 ceph-mon[29694]: osdmap e551: 8 total, 8 up, 5 in 2024-09-20T22:22:22.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:22 smithi060 ceph-mon[29694]: osdmap e552: 8 total, 8 up, 5 in 2024-09-20T22:22:22.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:22 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:22:22.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:22 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:22:22.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:22 smithi060 ceph-mon[40290]: 6.12 scrub starts 2024-09-20T22:22:22.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:22 smithi060 ceph-mon[40290]: 6.12 scrub ok 2024-09-20T22:22:22.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:22 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd out", "ids": ["6"]}]': finished 2024-09-20T22:22:22.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:22 smithi060 ceph-mon[40290]: osdmap e551: 8 total, 8 up, 5 in 2024-09-20T22:22:22.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:22 smithi060 ceph-mon[40290]: osdmap e552: 8 total, 8 up, 5 in 2024-09-20T22:22:23.223 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:22:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:22:23.329 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:23.329 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:23.977 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:23 smithi067 ceph-mon[33511]: 3.1a scrub starts 2024-09-20T22:22:23.977 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:23 smithi067 ceph-mon[33511]: 3.1a scrub ok 2024-09-20T22:22:23.977 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:23 smithi067 ceph-mon[33511]: pgmap v1053: 162 pgs: 26 peering, 5 remapped+peering, 2 unknown, 129 active+clean; 711 KiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 703 B/s rd, 0 op/s; 44 KiB/s, 1 objects/s recovering 2024-09-20T22:22:23.977 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:23 smithi067 ceph-mon[33511]: 3.d scrub starts 2024-09-20T22:22:23.977 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:23 smithi067 ceph-mon[33511]: 3.d scrub ok 2024-09-20T22:22:23.977 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:23 smithi067 ceph-mon[33511]: osdmap e553: 8 total, 8 up, 5 in 2024-09-20T22:22:23.977 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:23 smithi067 ceph-mon[33511]: Health check failed: Reduced data availability: 9 pgs inactive, 9 pgs peering (PG_AVAILABILITY) 2024-09-20T22:22:23.978 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:23 smithi067 ceph-mon[33511]: 6.6 deep-scrub starts 2024-09-20T22:22:23.978 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:23 smithi067 ceph-mon[33511]: 6.6 deep-scrub ok 2024-09-20T22:22:23.978 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:23 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:22:23.978 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:23 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:22:24.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[40290]: 3.1a scrub starts 2024-09-20T22:22:24.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[40290]: 3.1a scrub ok 2024-09-20T22:22:24.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[40290]: pgmap v1053: 162 pgs: 26 peering, 5 remapped+peering, 2 unknown, 129 active+clean; 711 KiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 703 B/s rd, 0 op/s; 44 KiB/s, 1 objects/s recovering 2024-09-20T22:22:24.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[40290]: 3.d scrub starts 2024-09-20T22:22:24.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[40290]: 3.d scrub ok 2024-09-20T22:22:24.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[40290]: osdmap e553: 8 total, 8 up, 5 in 2024-09-20T22:22:24.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[40290]: Health check failed: Reduced data availability: 9 pgs inactive, 9 pgs peering (PG_AVAILABILITY) 2024-09-20T22:22:24.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[40290]: 6.6 deep-scrub starts 2024-09-20T22:22:24.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[40290]: 6.6 deep-scrub ok 2024-09-20T22:22:24.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:22:24.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:22:24.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[29694]: 3.1a scrub starts 2024-09-20T22:22:24.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[29694]: 3.1a scrub ok 2024-09-20T22:22:24.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[29694]: pgmap v1053: 162 pgs: 26 peering, 5 remapped+peering, 2 unknown, 129 active+clean; 711 KiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 703 B/s rd, 0 op/s; 44 KiB/s, 1 objects/s recovering 2024-09-20T22:22:24.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[29694]: 3.d scrub starts 2024-09-20T22:22:24.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[29694]: 3.d scrub ok 2024-09-20T22:22:24.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[29694]: osdmap e553: 8 total, 8 up, 5 in 2024-09-20T22:22:24.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[29694]: Health check failed: Reduced data availability: 9 pgs inactive, 9 pgs peering (PG_AVAILABILITY) 2024-09-20T22:22:24.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[29694]: 6.6 deep-scrub starts 2024-09-20T22:22:24.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[29694]: 6.6 deep-scrub ok 2024-09-20T22:22:24.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:22:24.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:23 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:22:24.303 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_blocked_ops 2024-09-20T22:22:24.563 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:22:24.871 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:24 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:22:24.871 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:24 smithi067 ceph-mon[33511]: osdmap e554: 8 total, 8 up, 5 in 2024-09-20T22:22:24.871 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:24 smithi067 ceph-mon[33511]: 6.d scrub starts 2024-09-20T22:22:24.872 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:24 smithi067 ceph-mon[33511]: 6.d scrub ok 2024-09-20T22:22:25.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:24 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:22:25.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:24 smithi060 ceph-mon[29694]: osdmap e554: 8 total, 8 up, 5 in 2024-09-20T22:22:25.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:24 smithi060 ceph-mon[29694]: 6.d scrub starts 2024-09-20T22:22:25.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:24 smithi060 ceph-mon[29694]: 6.d scrub ok 2024-09-20T22:22:25.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:24 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:22:25.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:24 smithi060 ceph-mon[40290]: osdmap e554: 8 total, 8 up, 5 in 2024-09-20T22:22:25.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:24 smithi060 ceph-mon[40290]: 6.d scrub starts 2024-09-20T22:22:25.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:24 smithi060 ceph-mon[40290]: 6.d scrub ok 2024-09-20T22:22:25.118 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:25.118 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:25.536 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:22:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:22:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:22:25.693 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:25 smithi067 ceph-mon[33511]: pgmap v1056: 162 pgs: 2 active+undersized+remapped, 26 peering, 5 remapped+peering, 2 unknown, 127 active+clean; 711 KiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s; 64 KiB/s, 2 objects/s recovering 2024-09-20T22:22:25.693 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:25 smithi067 ceph-mon[33511]: osdmap e555: 8 total, 8 up, 5 in 2024-09-20T22:22:25.694 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:25 smithi067 ceph-mon[33511]: 1.8 scrub starts 2024-09-20T22:22:25.694 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:25 smithi067 ceph-mon[33511]: 1.8 scrub ok 2024-09-20T22:22:25.694 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:25 smithi067 ceph-mon[33511]: 1.6 scrub starts 2024-09-20T22:22:25.694 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:25 smithi067 ceph-mon[33511]: 1.6 scrub ok 2024-09-20T22:22:25.722 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.7 dump_historic_ops 2024-09-20T22:22:25.982 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:22:26.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:25 smithi060 ceph-mon[29694]: pgmap v1056: 162 pgs: 2 active+undersized+remapped, 26 peering, 5 remapped+peering, 2 unknown, 127 active+clean; 711 KiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s; 64 KiB/s, 2 objects/s recovering 2024-09-20T22:22:26.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:25 smithi060 ceph-mon[29694]: osdmap e555: 8 total, 8 up, 5 in 2024-09-20T22:22:26.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:25 smithi060 ceph-mon[29694]: 1.8 scrub starts 2024-09-20T22:22:26.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:25 smithi060 ceph-mon[29694]: 1.8 scrub ok 2024-09-20T22:22:26.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:25 smithi060 ceph-mon[29694]: 1.6 scrub starts 2024-09-20T22:22:26.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:25 smithi060 ceph-mon[29694]: 1.6 scrub ok 2024-09-20T22:22:26.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:25 smithi060 ceph-mon[40290]: pgmap v1056: 162 pgs: 2 active+undersized+remapped, 26 peering, 5 remapped+peering, 2 unknown, 127 active+clean; 711 KiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 1023 B/s rd, 0 op/s; 64 KiB/s, 2 objects/s recovering 2024-09-20T22:22:26.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:25 smithi060 ceph-mon[40290]: osdmap e555: 8 total, 8 up, 5 in 2024-09-20T22:22:26.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:25 smithi060 ceph-mon[40290]: 1.8 scrub starts 2024-09-20T22:22:26.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:25 smithi060 ceph-mon[40290]: 1.8 scrub ok 2024-09-20T22:22:26.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:25 smithi060 ceph-mon[40290]: 1.6 scrub starts 2024-09-20T22:22:26.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:25 smithi060 ceph-mon[40290]: 1.6 scrub ok 2024-09-20T22:22:26.455 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:26.455 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:26 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:22:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:26 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:22:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:26 smithi067 ceph-mon[33511]: 1.7 scrub starts 2024-09-20T22:22:26.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:26 smithi067 ceph-mon[33511]: 1.7 scrub ok 2024-09-20T22:22:26.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:26 smithi067 ceph-mon[33511]: 1.f scrub starts 2024-09-20T22:22:26.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:26 smithi067 ceph-mon[33511]: 1.f scrub ok 2024-09-20T22:22:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:26 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:22:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:26 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:22:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:26 smithi060 ceph-mon[29694]: 1.7 scrub starts 2024-09-20T22:22:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:26 smithi060 ceph-mon[29694]: 1.7 scrub ok 2024-09-20T22:22:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:26 smithi060 ceph-mon[29694]: 1.f scrub starts 2024-09-20T22:22:27.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:26 smithi060 ceph-mon[29694]: 1.f scrub ok 2024-09-20T22:22:27.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:26 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:22:27.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:26 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:22:27.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:26 smithi060 ceph-mon[40290]: 1.7 scrub starts 2024-09-20T22:22:27.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:26 smithi060 ceph-mon[40290]: 1.7 scrub ok 2024-09-20T22:22:27.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:26 smithi060 ceph-mon[40290]: 1.f scrub starts 2024-09-20T22:22:27.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:26 smithi060 ceph-mon[40290]: 1.f scrub ok 2024-09-20T22:22:27.098 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_ops_in_flight 2024-09-20T22:22:27.370 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:27.472 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-20T22:22:27.473 INFO:tasks.thrashosds.thrasher:Adding osd 6 2024-09-20T22:22:27.473 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd in 6 2024-09-20T22:22:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:27 smithi060 ceph-mon[29694]: 5.a scrub starts 2024-09-20T22:22:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:27 smithi060 ceph-mon[29694]: 5.a scrub ok 2024-09-20T22:22:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:27 smithi060 ceph-mon[29694]: pgmap v1058: 162 pgs: 2 active+undersized+remapped, 26 peering, 5 remapped+peering, 2 unknown, 127 active+clean; 711 KiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 652 B/s rd, 0 op/s; 55 KiB/s, 7 objects/s recovering 2024-09-20T22:22:27.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:27 smithi060 ceph-mon[29694]: osdmap e556: 8 total, 8 up, 5 in 2024-09-20T22:22:27.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:27 smithi060 ceph-mon[29694]: 3.1e scrub starts 2024-09-20T22:22:27.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:27 smithi060 ceph-mon[29694]: 3.1e scrub ok 2024-09-20T22:22:27.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:27 smithi060 ceph-mon[29694]: 3.19 deep-scrub starts 2024-09-20T22:22:27.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:27 smithi060 ceph-mon[29694]: 3.19 deep-scrub ok 2024-09-20T22:22:27.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:27 smithi060 ceph-mon[40290]: 5.a scrub starts 2024-09-20T22:22:27.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:27 smithi060 ceph-mon[40290]: 5.a scrub ok 2024-09-20T22:22:27.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:27 smithi060 ceph-mon[40290]: pgmap v1058: 162 pgs: 2 active+undersized+remapped, 26 peering, 5 remapped+peering, 2 unknown, 127 active+clean; 711 KiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 652 B/s rd, 0 op/s; 55 KiB/s, 7 objects/s recovering 2024-09-20T22:22:27.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:27 smithi060 ceph-mon[40290]: osdmap e556: 8 total, 8 up, 5 in 2024-09-20T22:22:27.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:27 smithi060 ceph-mon[40290]: 3.1e scrub starts 2024-09-20T22:22:27.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:27 smithi060 ceph-mon[40290]: 3.1e scrub ok 2024-09-20T22:22:27.788 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:27 smithi060 ceph-mon[40290]: 3.19 deep-scrub starts 2024-09-20T22:22:27.789 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:27 smithi060 ceph-mon[40290]: 3.19 deep-scrub ok 2024-09-20T22:22:28.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:27 smithi067 ceph-mon[33511]: 5.a scrub starts 2024-09-20T22:22:28.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:27 smithi067 ceph-mon[33511]: 5.a scrub ok 2024-09-20T22:22:28.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:27 smithi067 ceph-mon[33511]: pgmap v1058: 162 pgs: 2 active+undersized+remapped, 26 peering, 5 remapped+peering, 2 unknown, 127 active+clean; 711 KiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 652 B/s rd, 0 op/s; 55 KiB/s, 7 objects/s recovering 2024-09-20T22:22:28.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:27 smithi067 ceph-mon[33511]: osdmap e556: 8 total, 8 up, 5 in 2024-09-20T22:22:28.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:27 smithi067 ceph-mon[33511]: 3.1e scrub starts 2024-09-20T22:22:28.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:27 smithi067 ceph-mon[33511]: 3.1e scrub ok 2024-09-20T22:22:28.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:27 smithi067 ceph-mon[33511]: 3.19 deep-scrub starts 2024-09-20T22:22:28.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:27 smithi067 ceph-mon[33511]: 3.19 deep-scrub ok 2024-09-20T22:22:28.102 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:28.102 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:28.364 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:28 smithi060 ceph-mon[29694]: 3.15 scrub starts 2024-09-20T22:22:28.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:28 smithi060 ceph-mon[29694]: 3.15 scrub ok 2024-09-20T22:22:28.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:28 smithi060 ceph-mon[40290]: 3.15 scrub starts 2024-09-20T22:22:28.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:28 smithi060 ceph-mon[40290]: 3.15 scrub ok 2024-09-20T22:22:28.854 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_blocked_ops 2024-09-20T22:22:29.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:28 smithi067 ceph-mon[33511]: 3.15 scrub starts 2024-09-20T22:22:29.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:28 smithi067 ceph-mon[33511]: 3.15 scrub ok 2024-09-20T22:22:29.421 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:29.635 INFO:teuthology.orchestra.run.smithi060.stderr:marked in osd.6. 2024-09-20T22:22:29.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:29 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a[29671]: 2024-09-20T22:22:29.559+0000 7f01adcc6640 -1 mon.a@0(leader).osd e556 definitely_dead 0 2024-09-20T22:22:29.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:29 smithi060 ceph-mon[29694]: pgmap v1060: 162 pgs: 162 active+clean; 583 KiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 537 B/s rd, 0 op/s; 11 KiB/s, 7 objects/s recovering 2024-09-20T22:22:29.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:29 smithi060 ceph-mon[29694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs inactive, 9 pgs peering) 2024-09-20T22:22:29.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:29 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:22:29.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:29 smithi060 ceph-mon[29694]: 3.4 scrub starts 2024-09-20T22:22:29.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:29 smithi060 ceph-mon[29694]: 3.4 scrub ok 2024-09-20T22:22:29.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:29 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1940113059' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["6"]}]: dispatch 2024-09-20T22:22:29.958 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:29 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["6"]}]: dispatch 2024-09-20T22:22:29.959 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:29 smithi060 ceph-mon[40290]: pgmap v1060: 162 pgs: 162 active+clean; 583 KiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 537 B/s rd, 0 op/s; 11 KiB/s, 7 objects/s recovering 2024-09-20T22:22:29.959 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:29 smithi060 ceph-mon[40290]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs inactive, 9 pgs peering) 2024-09-20T22:22:29.959 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:29 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:22:29.959 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:29 smithi060 ceph-mon[40290]: 3.4 scrub starts 2024-09-20T22:22:29.959 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:29 smithi060 ceph-mon[40290]: 3.4 scrub ok 2024-09-20T22:22:29.959 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:29 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1940113059' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["6"]}]: dispatch 2024-09-20T22:22:29.959 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:29 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["6"]}]: dispatch 2024-09-20T22:22:30.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:29 smithi067 ceph-mon[33511]: pgmap v1060: 162 pgs: 162 active+clean; 583 KiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 537 B/s rd, 0 op/s; 11 KiB/s, 7 objects/s recovering 2024-09-20T22:22:30.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:29 smithi067 ceph-mon[33511]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 9 pgs inactive, 9 pgs peering) 2024-09-20T22:22:30.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:29 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:22:30.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:29 smithi067 ceph-mon[33511]: 3.4 scrub starts 2024-09-20T22:22:30.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:29 smithi067 ceph-mon[33511]: 3.4 scrub ok 2024-09-20T22:22:30.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:29 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1940113059' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["6"]}]: dispatch 2024-09-20T22:22:30.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:29 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["6"]}]: dispatch 2024-09-20T22:22:30.066 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:30.066 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:30.792 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.1 dump_historic_ops 2024-09-20T22:22:30.932 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:30 smithi060 ceph-mon[29694]: 1.2 scrub starts 2024-09-20T22:22:30.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:30 smithi060 ceph-mon[29694]: 1.2 scrub ok 2024-09-20T22:22:30.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:30 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["6"]}]': finished 2024-09-20T22:22:30.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:30 smithi060 ceph-mon[29694]: osdmap e557: 8 total, 8 up, 6 in 2024-09-20T22:22:30.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:30 smithi060 ceph-mon[29694]: 5.8 scrub starts 2024-09-20T22:22:30.933 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:30 smithi060 ceph-mon[29694]: 5.8 scrub ok 2024-09-20T22:22:30.933 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:30 smithi060 ceph-mon[40290]: 1.2 scrub starts 2024-09-20T22:22:30.933 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:30 smithi060 ceph-mon[40290]: 1.2 scrub ok 2024-09-20T22:22:30.933 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:30 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["6"]}]': finished 2024-09-20T22:22:30.933 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:30 smithi060 ceph-mon[40290]: osdmap e557: 8 total, 8 up, 6 in 2024-09-20T22:22:30.933 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:30 smithi060 ceph-mon[40290]: 5.8 scrub starts 2024-09-20T22:22:30.933 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:30 smithi060 ceph-mon[40290]: 5.8 scrub ok 2024-09-20T22:22:31.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:30 smithi067 ceph-mon[33511]: 1.2 scrub starts 2024-09-20T22:22:31.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:30 smithi067 ceph-mon[33511]: 1.2 scrub ok 2024-09-20T22:22:31.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:30 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["6"]}]': finished 2024-09-20T22:22:31.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:30 smithi067 ceph-mon[33511]: osdmap e557: 8 total, 8 up, 6 in 2024-09-20T22:22:31.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:30 smithi067 ceph-mon[33511]: 5.8 scrub starts 2024-09-20T22:22:31.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:30 smithi067 ceph-mon[33511]: 5.8 scrub ok 2024-09-20T22:22:31.268 INFO:tasks.thrashosds.thrasher:Added osd 6 2024-09-20T22:22:31.268 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-20T22:22:31.268 INFO:tasks.thrashosds.thrasher:Adding osd 4 2024-09-20T22:22:31.269 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd in 4 2024-09-20T22:22:31.385 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:31.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:31 smithi060 ceph-mon[29694]: pgmap v1062: 162 pgs: 162 active+clean; 583 KiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 511 B/s rd, 0 op/s; 11 KiB/s, 6 objects/s recovering 2024-09-20T22:22:31.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:31 smithi060 ceph-mon[29694]: osdmap e558: 8 total, 8 up, 6 in 2024-09-20T22:22:31.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:31 smithi060 ceph-mon[29694]: 1.18 scrub starts 2024-09-20T22:22:31.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:31 smithi060 ceph-mon[29694]: 1.18 scrub ok 2024-09-20T22:22:31.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:31 smithi060 ceph-mon[29694]: 5.5 scrub starts 2024-09-20T22:22:31.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:31 smithi060 ceph-mon[29694]: 5.5 scrub ok 2024-09-20T22:22:31.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:31 smithi060 ceph-mon[40290]: pgmap v1062: 162 pgs: 162 active+clean; 583 KiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 511 B/s rd, 0 op/s; 11 KiB/s, 6 objects/s recovering 2024-09-20T22:22:31.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:31 smithi060 ceph-mon[40290]: osdmap e558: 8 total, 8 up, 6 in 2024-09-20T22:22:31.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:31 smithi060 ceph-mon[40290]: 1.18 scrub starts 2024-09-20T22:22:31.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:31 smithi060 ceph-mon[40290]: 1.18 scrub ok 2024-09-20T22:22:31.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:31 smithi060 ceph-mon[40290]: 5.5 scrub starts 2024-09-20T22:22:31.787 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:31 smithi060 ceph-mon[40290]: 5.5 scrub ok 2024-09-20T22:22:31.962 INFO:teuthology.orchestra.run.smithi060.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:31.963 INFO:teuthology.orchestra.run.smithi060.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:31 smithi067 ceph-mon[33511]: pgmap v1062: 162 pgs: 162 active+clean; 583 KiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 511 B/s rd, 0 op/s; 11 KiB/s, 6 objects/s recovering 2024-09-20T22:22:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:31 smithi067 ceph-mon[33511]: osdmap e558: 8 total, 8 up, 6 in 2024-09-20T22:22:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:31 smithi067 ceph-mon[33511]: 1.18 scrub starts 2024-09-20T22:22:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:31 smithi067 ceph-mon[33511]: 1.18 scrub ok 2024-09-20T22:22:32.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:31 smithi067 ceph-mon[33511]: 5.5 scrub starts 2024-09-20T22:22:32.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:31 smithi067 ceph-mon[33511]: 5.5 scrub ok 2024-09-20T22:22:32.666 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:32.697 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_ops_in_flight 2024-09-20T22:22:32.911 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:32 smithi060 ceph-mon[29694]: 1.15 deep-scrub starts 2024-09-20T22:22:32.911 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:32 smithi060 ceph-mon[29694]: 1.15 deep-scrub ok 2024-09-20T22:22:32.912 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:32 smithi060 ceph-mon[29694]: osdmap e559: 8 total, 8 up, 6 in 2024-09-20T22:22:32.912 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:32 smithi060 ceph-mon[29694]: osdmap e560: 8 total, 8 up, 6 in 2024-09-20T22:22:32.912 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:32 smithi060 ceph-mon[29694]: 3.1c scrub starts 2024-09-20T22:22:32.912 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:32 smithi060 ceph-mon[29694]: 3.1c scrub ok 2024-09-20T22:22:32.912 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:32 smithi060 ceph-mon[40290]: 1.15 deep-scrub starts 2024-09-20T22:22:32.912 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:32 smithi060 ceph-mon[40290]: 1.15 deep-scrub ok 2024-09-20T22:22:32.912 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:32 smithi060 ceph-mon[40290]: osdmap e559: 8 total, 8 up, 6 in 2024-09-20T22:22:32.912 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:32 smithi060 ceph-mon[40290]: osdmap e560: 8 total, 8 up, 6 in 2024-09-20T22:22:32.912 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:32 smithi060 ceph-mon[40290]: 3.1c scrub starts 2024-09-20T22:22:32.912 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:32 smithi060 ceph-mon[40290]: 3.1c scrub ok 2024-09-20T22:22:32.951 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:32 smithi067 ceph-mon[33511]: 1.15 deep-scrub starts 2024-09-20T22:22:32.951 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:32 smithi067 ceph-mon[33511]: 1.15 deep-scrub ok 2024-09-20T22:22:32.952 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:32 smithi067 ceph-mon[33511]: osdmap e559: 8 total, 8 up, 6 in 2024-09-20T22:22:32.952 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:32 smithi067 ceph-mon[33511]: osdmap e560: 8 total, 8 up, 6 in 2024-09-20T22:22:32.952 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:32 smithi067 ceph-mon[33511]: 3.1c scrub starts 2024-09-20T22:22:32.952 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:32 smithi067 ceph-mon[33511]: 3.1c scrub ok 2024-09-20T22:22:32.973 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:22:33.261 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:22:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:22:33.527 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:33.527 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:33.648 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:33 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a[29671]: 2024-09-20T22:22:33.329+0000 7f01adcc6640 -1 mon.a@0(leader).osd e561 definitely_dead 0 2024-09-20T22:22:33.711 INFO:teuthology.orchestra.run.smithi060.stderr:marked in osd.4. 2024-09-20T22:22:33.901 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:33 smithi067 ceph-mon[33511]: pgmap v1066: 162 pgs: 1 active+remapped, 6 active+undersized+remapped, 4 activating+undersized+remapped, 3 remapped+peering, 148 active+clean; 724 KiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 2.7 KiB/s rd, 14 KiB/s wr, 3 op/s; 154 KiB/s, 12 objects/s recovering 2024-09-20T22:22:33.901 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:33 smithi067 ceph-mon[33511]: osdmap e561: 8 total, 8 up, 6 in 2024-09-20T22:22:33.901 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:33 smithi067 ceph-mon[33511]: 3.17 scrub starts 2024-09-20T22:22:33.901 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:33 smithi067 ceph-mon[33511]: 2.1 scrub starts 2024-09-20T22:22:33.901 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:33 smithi067 ceph-mon[33511]: 3.17 scrub ok 2024-09-20T22:22:33.901 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:33 smithi067 ceph-mon[33511]: 2.1 scrub ok 2024-09-20T22:22:33.901 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:33 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/707674565' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["4"]}]: dispatch 2024-09-20T22:22:34.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:33 smithi060 ceph-mon[29694]: pgmap v1066: 162 pgs: 1 active+remapped, 6 active+undersized+remapped, 4 activating+undersized+remapped, 3 remapped+peering, 148 active+clean; 724 KiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 2.7 KiB/s rd, 14 KiB/s wr, 3 op/s; 154 KiB/s, 12 objects/s recovering 2024-09-20T22:22:34.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:33 smithi060 ceph-mon[29694]: osdmap e561: 8 total, 8 up, 6 in 2024-09-20T22:22:34.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:33 smithi060 ceph-mon[29694]: 3.17 scrub starts 2024-09-20T22:22:34.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:33 smithi060 ceph-mon[29694]: 2.1 scrub starts 2024-09-20T22:22:34.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:33 smithi060 ceph-mon[29694]: 3.17 scrub ok 2024-09-20T22:22:34.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:33 smithi060 ceph-mon[29694]: 2.1 scrub ok 2024-09-20T22:22:34.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:33 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/707674565' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["4"]}]: dispatch 2024-09-20T22:22:34.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:33 smithi060 ceph-mon[40290]: pgmap v1066: 162 pgs: 1 active+remapped, 6 active+undersized+remapped, 4 activating+undersized+remapped, 3 remapped+peering, 148 active+clean; 724 KiB data, 4.4 GiB used, 443 GiB / 447 GiB avail; 2.7 KiB/s rd, 14 KiB/s wr, 3 op/s; 154 KiB/s, 12 objects/s recovering 2024-09-20T22:22:34.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:33 smithi060 ceph-mon[40290]: osdmap e561: 8 total, 8 up, 6 in 2024-09-20T22:22:34.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:33 smithi060 ceph-mon[40290]: 3.17 scrub starts 2024-09-20T22:22:34.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:33 smithi060 ceph-mon[40290]: 2.1 scrub starts 2024-09-20T22:22:34.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:33 smithi060 ceph-mon[40290]: 3.17 scrub ok 2024-09-20T22:22:34.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:33 smithi060 ceph-mon[40290]: 2.1 scrub ok 2024-09-20T22:22:34.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:33 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/707674565' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["4"]}]: dispatch 2024-09-20T22:22:34.295 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_blocked_ops 2024-09-20T22:22:34.400 INFO:tasks.thrashosds.thrasher:Added osd 4 2024-09-20T22:22:34.400 INFO:tasks.thrashosds.thrasher:inning osd 2024-09-20T22:22:34.400 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-09-20T22:22:34.400 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd in 1 2024-09-20T22:22:34.548 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:22:34.661 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:34.948 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:22:34.949 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/707674565' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["4"]}]': finished 2024-09-20T22:22:34.949 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[29694]: osdmap e562: 8 total, 8 up, 7 in 2024-09-20T22:22:34.949 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[29694]: 1.7 scrub starts 2024-09-20T22:22:34.949 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[29694]: 1.7 scrub ok 2024-09-20T22:22:34.949 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[29694]: 6.1b scrub starts 2024-09-20T22:22:34.949 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[29694]: 6.1b scrub ok 2024-09-20T22:22:34.949 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[29694]: 1.17 scrub starts 2024-09-20T22:22:34.949 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[29694]: 1.17 scrub ok 2024-09-20T22:22:34.949 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[29694]: 5.d scrub starts 2024-09-20T22:22:34.949 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[29694]: 5.d scrub ok 2024-09-20T22:22:34.950 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:22:34.950 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/707674565' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["4"]}]': finished 2024-09-20T22:22:34.950 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[40290]: osdmap e562: 8 total, 8 up, 7 in 2024-09-20T22:22:34.950 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[40290]: 1.7 scrub starts 2024-09-20T22:22:34.950 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[40290]: 1.7 scrub ok 2024-09-20T22:22:34.950 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[40290]: 6.1b scrub starts 2024-09-20T22:22:34.950 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[40290]: 6.1b scrub ok 2024-09-20T22:22:34.950 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[40290]: 1.17 scrub starts 2024-09-20T22:22:34.950 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[40290]: 1.17 scrub ok 2024-09-20T22:22:34.950 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[40290]: 5.d scrub starts 2024-09-20T22:22:34.950 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:34 smithi060 ceph-mon[40290]: 5.d scrub ok 2024-09-20T22:22:34.971 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:34 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:22:34.972 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:34 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/707674565' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["4"]}]': finished 2024-09-20T22:22:34.972 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:34 smithi067 ceph-mon[33511]: osdmap e562: 8 total, 8 up, 7 in 2024-09-20T22:22:34.972 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:34 smithi067 ceph-mon[33511]: 1.7 scrub starts 2024-09-20T22:22:34.972 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:34 smithi067 ceph-mon[33511]: 1.7 scrub ok 2024-09-20T22:22:34.972 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:34 smithi067 ceph-mon[33511]: 6.1b scrub starts 2024-09-20T22:22:34.972 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:34 smithi067 ceph-mon[33511]: 6.1b scrub ok 2024-09-20T22:22:34.972 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:34 smithi067 ceph-mon[33511]: 1.17 scrub starts 2024-09-20T22:22:34.972 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:34 smithi067 ceph-mon[33511]: 1.17 scrub ok 2024-09-20T22:22:34.972 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:34 smithi067 ceph-mon[33511]: 5.d scrub starts 2024-09-20T22:22:34.972 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:34 smithi067 ceph-mon[33511]: 5.d scrub ok 2024-09-20T22:22:35.079 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:35.080 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:35.229 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:22:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:22:35.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a[29671]: 2024-09-20T22:22:35.399+0000 7f01adcc6640 -1 mon.a@0(leader).osd e563 definitely_dead 0 2024-09-20T22:22:35.724 INFO:teuthology.orchestra.run.smithi060.stderr:marked in osd.1. 2024-09-20T22:22:35.788 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph daemon osd.4 dump_historic_ops 2024-09-20T22:22:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:35 smithi067 ceph-mon[33511]: pgmap v1069: 162 pgs: 32 peering, 3 active+undersized+remapped, 4 activating+undersized+remapped, 4 unknown, 3 remapped+peering, 116 active+clean; 741 KiB data, 5.2 GiB used, 621 GiB / 626 GiB avail; 177 KiB/s, 0 objects/s recovering 2024-09-20T22:22:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:35 smithi067 ceph-mon[33511]: 1.1c scrub starts 2024-09-20T22:22:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:35 smithi067 ceph-mon[33511]: 1.1c scrub ok 2024-09-20T22:22:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:35 smithi067 ceph-mon[33511]: Health check failed: Reduced data availability: 2 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2024-09-20T22:22:36.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:35 smithi067 ceph-mon[33511]: osdmap e563: 8 total, 8 up, 7 in 2024-09-20T22:22:36.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:35 smithi067 ceph-mon[33511]: 1.8 scrub starts 2024-09-20T22:22:36.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:35 smithi067 ceph-mon[33511]: 1.8 scrub ok 2024-09-20T22:22:36.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:35 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1869057472' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2024-09-20T22:22:36.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:35 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2024-09-20T22:22:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-mon[29694]: pgmap v1069: 162 pgs: 32 peering, 3 active+undersized+remapped, 4 activating+undersized+remapped, 4 unknown, 3 remapped+peering, 116 active+clean; 741 KiB data, 5.2 GiB used, 621 GiB / 626 GiB avail; 177 KiB/s, 0 objects/s recovering 2024-09-20T22:22:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-mon[29694]: 1.1c scrub starts 2024-09-20T22:22:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-mon[29694]: 1.1c scrub ok 2024-09-20T22:22:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-mon[29694]: Health check failed: Reduced data availability: 2 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2024-09-20T22:22:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-mon[29694]: osdmap e563: 8 total, 8 up, 7 in 2024-09-20T22:22:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-mon[29694]: 1.8 scrub starts 2024-09-20T22:22:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-mon[29694]: 1.8 scrub ok 2024-09-20T22:22:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1869057472' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2024-09-20T22:22:36.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2024-09-20T22:22:36.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-mon[40290]: pgmap v1069: 162 pgs: 32 peering, 3 active+undersized+remapped, 4 activating+undersized+remapped, 4 unknown, 3 remapped+peering, 116 active+clean; 741 KiB data, 5.2 GiB used, 621 GiB / 626 GiB avail; 177 KiB/s, 0 objects/s recovering 2024-09-20T22:22:36.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-mon[40290]: 1.1c scrub starts 2024-09-20T22:22:36.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-mon[40290]: 1.1c scrub ok 2024-09-20T22:22:36.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-mon[40290]: Health check failed: Reduced data availability: 2 pgs inactive, 4 pgs peering (PG_AVAILABILITY) 2024-09-20T22:22:36.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-mon[40290]: osdmap e563: 8 total, 8 up, 7 in 2024-09-20T22:22:36.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-mon[40290]: 1.8 scrub starts 2024-09-20T22:22:36.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-mon[40290]: 1.8 scrub ok 2024-09-20T22:22:36.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1869057472' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2024-09-20T22:22:36.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:35 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd in", "ids": ["1"]}]: dispatch 2024-09-20T22:22:36.051 INFO:teuthology.orchestra.run.smithi067.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.b/config 2024-09-20T22:22:36.435 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-09-20T22:22:36.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-09-20T22:22:36.563 INFO:teuthology.orchestra.run.smithi067.stdout:ERROR: (22) Invalid argument 2024-09-20T22:22:36.563 INFO:teuthology.orchestra.run.smithi067.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-09-20T22:22:36.698 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:36.723 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:36 smithi060 ceph-mon[40290]: 6.1c scrub starts 2024-09-20T22:22:36.723 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:36 smithi060 ceph-mon[40290]: 6.1c scrub ok 2024-09-20T22:22:36.723 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:36 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["1"]}]': finished 2024-09-20T22:22:36.723 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:36 smithi060 ceph-mon[40290]: osdmap e564: 8 total, 8 up, 8 in 2024-09-20T22:22:36.723 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:36 smithi060 ceph-mon[40290]: pgmap v1072: 162 pgs: 1 remapped, 36 peering, 4 active+undersized+remapped, 4 activating+undersized+remapped, 5 unknown, 112 active+clean; 741 KiB data, 5.2 GiB used, 621 GiB / 626 GiB avail; 13 KiB/s, 0 objects/s recovering 2024-09-20T22:22:36.723 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:36 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:22:36.723 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:36 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:22:36.723 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:36 smithi060 ceph-mon[40290]: osdmap e565: 8 total, 8 up, 8 in 2024-09-20T22:22:37.006 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:36 smithi060 ceph-mon[29694]: 6.1c scrub starts 2024-09-20T22:22:37.006 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:36 smithi060 ceph-mon[29694]: 6.1c scrub ok 2024-09-20T22:22:37.006 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:36 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["1"]}]': finished 2024-09-20T22:22:37.007 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:36 smithi060 ceph-mon[29694]: osdmap e564: 8 total, 8 up, 8 in 2024-09-20T22:22:37.007 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:36 smithi060 ceph-mon[29694]: pgmap v1072: 162 pgs: 1 remapped, 36 peering, 4 active+undersized+remapped, 4 activating+undersized+remapped, 5 unknown, 112 active+clean; 741 KiB data, 5.2 GiB used, 621 GiB / 626 GiB avail; 13 KiB/s, 0 objects/s recovering 2024-09-20T22:22:37.007 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:36 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:22:37.007 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:36 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:22:37.007 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:36 smithi060 ceph-mon[29694]: osdmap e565: 8 total, 8 up, 8 in 2024-09-20T22:22:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:36 smithi067 ceph-mon[33511]: 6.1c scrub starts 2024-09-20T22:22:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:36 smithi067 ceph-mon[33511]: 6.1c scrub ok 2024-09-20T22:22:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:36 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd in", "ids": ["1"]}]': finished 2024-09-20T22:22:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:36 smithi067 ceph-mon[33511]: osdmap e564: 8 total, 8 up, 8 in 2024-09-20T22:22:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:36 smithi067 ceph-mon[33511]: pgmap v1072: 162 pgs: 1 remapped, 36 peering, 4 active+undersized+remapped, 4 activating+undersized+remapped, 5 unknown, 112 active+clean; 741 KiB data, 5.2 GiB used, 621 GiB / 626 GiB avail; 13 KiB/s, 0 objects/s recovering 2024-09-20T22:22:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:36 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:22:37.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:36 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:22:37.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:36 smithi067 ceph-mon[33511]: osdmap e565: 8 total, 8 up, 8 in 2024-09-20T22:22:37.231 INFO:teuthology.orchestra.run.smithi060.stdout:mon.a: {} 2024-09-20T22:22:37.232 INFO:teuthology.orchestra.run.smithi060.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-20T22:22:37.240 INFO:teuthology.orchestra.run.smithi060.stdout:mon.b: {} 2024-09-20T22:22:37.240 INFO:teuthology.orchestra.run.smithi060.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-20T22:22:37.249 INFO:teuthology.orchestra.run.smithi060.stdout:mon.c: {} 2024-09-20T22:22:37.249 INFO:teuthology.orchestra.run.smithi060.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-09-20T22:22:37.926 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:37 smithi060 ceph-mon[29694]: 5.e scrub starts 2024-09-20T22:22:37.926 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:37 smithi060 ceph-mon[29694]: 5.e scrub ok 2024-09-20T22:22:37.926 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:37 smithi060 ceph-mon[29694]: 5.6 scrub starts 2024-09-20T22:22:37.926 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:37 smithi060 ceph-mon[29694]: 5.6 scrub ok 2024-09-20T22:22:37.926 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:37 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1319281793' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-20T22:22:37.926 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:37 smithi060 ceph-mon[29694]: osdmap e566: 8 total, 8 up, 8 in 2024-09-20T22:22:37.927 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:37 smithi060 ceph-mon[40290]: 5.e scrub starts 2024-09-20T22:22:37.927 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:37 smithi060 ceph-mon[40290]: 5.e scrub ok 2024-09-20T22:22:37.927 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:37 smithi060 ceph-mon[40290]: 5.6 scrub starts 2024-09-20T22:22:37.927 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:37 smithi060 ceph-mon[40290]: 5.6 scrub ok 2024-09-20T22:22:37.927 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:37 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1319281793' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-20T22:22:37.927 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:37 smithi060 ceph-mon[40290]: osdmap e566: 8 total, 8 up, 8 in 2024-09-20T22:22:37.955 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 0 1 2024-09-20T22:22:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:37 smithi067 ceph-mon[33511]: 5.e scrub starts 2024-09-20T22:22:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:37 smithi067 ceph-mon[33511]: 5.e scrub ok 2024-09-20T22:22:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:37 smithi067 ceph-mon[33511]: 5.6 scrub starts 2024-09-20T22:22:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:37 smithi067 ceph-mon[33511]: 5.6 scrub ok 2024-09-20T22:22:38.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:37 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1319281793' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-20T22:22:38.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:37 smithi067 ceph-mon[33511]: osdmap e566: 8 total, 8 up, 8 in 2024-09-20T22:22:38.213 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:39.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:38 smithi067 ceph-mon[33511]: 4.e scrub starts 2024-09-20T22:22:39.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:38 smithi067 ceph-mon[33511]: 4.e scrub ok 2024-09-20T22:22:39.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:38 smithi067 ceph-mon[33511]: 1.f scrub starts 2024-09-20T22:22:39.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:38 smithi067 ceph-mon[33511]: 1.f scrub ok 2024-09-20T22:22:39.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:38 smithi067 ceph-mon[33511]: 3.c scrub starts 2024-09-20T22:22:39.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:38 smithi067 ceph-mon[33511]: 3.c scrub ok 2024-09-20T22:22:39.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:38 smithi067 ceph-mon[33511]: 1.14 scrub starts 2024-09-20T22:22:39.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:38 smithi067 ceph-mon[33511]: 1.14 scrub ok 2024-09-20T22:22:39.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:38 smithi067 ceph-mon[33511]: 1.19 scrub starts 2024-09-20T22:22:39.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:38 smithi067 ceph-mon[33511]: 1.19 scrub ok 2024-09-20T22:22:39.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:38 smithi067 ceph-mon[33511]: pgmap v1075: 162 pgs: 1 remapped, 45 peering, 4 active+undersized+remapped, 2 remapped+peering, 3 unknown, 107 active+clean; 767 KiB data, 5.2 GiB used, 621 GiB / 626 GiB avail; 4.2 KiB/s rd, 27 KiB/s wr, 5 op/s; 20 KiB/s, 8 objects/s recovering 2024-09-20T22:22:39.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:38 smithi067 ceph-mon[33511]: osdmap e567: 8 total, 8 up, 8 in 2024-09-20T22:22:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[29694]: 4.e scrub starts 2024-09-20T22:22:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[29694]: 4.e scrub ok 2024-09-20T22:22:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[29694]: 1.f scrub starts 2024-09-20T22:22:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[29694]: 1.f scrub ok 2024-09-20T22:22:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[29694]: 3.c scrub starts 2024-09-20T22:22:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[29694]: 3.c scrub ok 2024-09-20T22:22:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[29694]: 1.14 scrub starts 2024-09-20T22:22:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[29694]: 1.14 scrub ok 2024-09-20T22:22:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[29694]: 1.19 scrub starts 2024-09-20T22:22:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[29694]: 1.19 scrub ok 2024-09-20T22:22:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[29694]: pgmap v1075: 162 pgs: 1 remapped, 45 peering, 4 active+undersized+remapped, 2 remapped+peering, 3 unknown, 107 active+clean; 767 KiB data, 5.2 GiB used, 621 GiB / 626 GiB avail; 4.2 KiB/s rd, 27 KiB/s wr, 5 op/s; 20 KiB/s, 8 objects/s recovering 2024-09-20T22:22:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[29694]: osdmap e567: 8 total, 8 up, 8 in 2024-09-20T22:22:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[40290]: 4.e scrub starts 2024-09-20T22:22:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[40290]: 4.e scrub ok 2024-09-20T22:22:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[40290]: 1.f scrub starts 2024-09-20T22:22:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[40290]: 1.f scrub ok 2024-09-20T22:22:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[40290]: 3.c scrub starts 2024-09-20T22:22:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[40290]: 3.c scrub ok 2024-09-20T22:22:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[40290]: 1.14 scrub starts 2024-09-20T22:22:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[40290]: 1.14 scrub ok 2024-09-20T22:22:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[40290]: 1.19 scrub starts 2024-09-20T22:22:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[40290]: 1.19 scrub ok 2024-09-20T22:22:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[40290]: pgmap v1075: 162 pgs: 1 remapped, 45 peering, 4 active+undersized+remapped, 2 remapped+peering, 3 unknown, 107 active+clean; 767 KiB data, 5.2 GiB used, 621 GiB / 626 GiB avail; 4.2 KiB/s rd, 27 KiB/s wr, 5 op/s; 20 KiB/s, 8 objects/s recovering 2024-09-20T22:22:39.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:38 smithi060 ceph-mon[40290]: osdmap e567: 8 total, 8 up, 8 in 2024-09-20T22:22:39.712 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.0 to 1 (10000) 2024-09-20T22:22:40.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:39 smithi067 ceph-mon[33511]: 1.e scrub starts 2024-09-20T22:22:40.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:39 smithi067 ceph-mon[33511]: 1.e scrub ok 2024-09-20T22:22:40.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:39 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2291118005' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:22:40.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:39 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:22:40.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:39 smithi067 ceph-mon[33511]: 4.1e scrub starts 2024-09-20T22:22:40.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:39 smithi067 ceph-mon[33511]: 4.1e scrub ok 2024-09-20T22:22:40.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:39 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:22:40.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:39 smithi067 ceph-mon[33511]: osdmap e568: 8 total, 8 up, 8 in 2024-09-20T22:22:40.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:39 smithi060 ceph-mon[29694]: 1.e scrub starts 2024-09-20T22:22:40.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:39 smithi060 ceph-mon[29694]: 1.e scrub ok 2024-09-20T22:22:40.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:39 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2291118005' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:22:40.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:39 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:22:40.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:39 smithi060 ceph-mon[29694]: 4.1e scrub starts 2024-09-20T22:22:40.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:39 smithi060 ceph-mon[29694]: 4.1e scrub ok 2024-09-20T22:22:40.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:39 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:22:40.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:39 smithi060 ceph-mon[29694]: osdmap e568: 8 total, 8 up, 8 in 2024-09-20T22:22:40.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:39 smithi060 ceph-mon[40290]: 1.e scrub starts 2024-09-20T22:22:40.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:39 smithi060 ceph-mon[40290]: 1.e scrub ok 2024-09-20T22:22:40.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:39 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2291118005' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:22:40.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:39 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:22:40.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:39 smithi060 ceph-mon[40290]: 4.1e scrub starts 2024-09-20T22:22:40.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:39 smithi060 ceph-mon[40290]: 4.1e scrub ok 2024-09-20T22:22:40.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:39 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:22:40.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:39 smithi060 ceph-mon[40290]: osdmap e568: 8 total, 8 up, 8 in 2024-09-20T22:22:40.429 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 0 1 2024-09-20T22:22:40.693 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:40.987 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[29694]: 2.0 scrub starts 2024-09-20T22:22:40.987 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[29694]: 2.0 scrub ok 2024-09-20T22:22:40.987 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[29694]: 3.10 scrub starts 2024-09-20T22:22:40.987 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[29694]: 3.10 scrub ok 2024-09-20T22:22:40.987 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[29694]: 4.19 scrub starts 2024-09-20T22:22:40.987 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[29694]: 4.19 scrub ok 2024-09-20T22:22:40.987 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[29694]: 4.1a deep-scrub starts 2024-09-20T22:22:40.987 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[29694]: 4.1a deep-scrub ok 2024-09-20T22:22:40.987 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[29694]: 1.3 scrub starts 2024-09-20T22:22:40.988 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[29694]: 1.3 scrub ok 2024-09-20T22:22:40.988 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[29694]: pgmap v1078: 162 pgs: 1 remapped, 18 peering, 2 active+undersized+remapped, 1 unknown, 140 active+clean; 626 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 51 KiB/s rd, 16 KiB/s wr, 107 op/s; 0 B/s, 43 objects/s recovering 2024-09-20T22:22:40.988 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[40290]: 2.0 scrub starts 2024-09-20T22:22:40.988 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[40290]: 2.0 scrub ok 2024-09-20T22:22:40.988 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[40290]: 3.10 scrub starts 2024-09-20T22:22:40.988 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[40290]: 3.10 scrub ok 2024-09-20T22:22:40.988 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[40290]: 4.19 scrub starts 2024-09-20T22:22:40.988 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[40290]: 4.19 scrub ok 2024-09-20T22:22:40.988 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[40290]: 4.1a deep-scrub starts 2024-09-20T22:22:40.988 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[40290]: 4.1a deep-scrub ok 2024-09-20T22:22:40.988 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[40290]: 1.3 scrub starts 2024-09-20T22:22:40.989 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[40290]: 1.3 scrub ok 2024-09-20T22:22:40.989 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:40 smithi060 ceph-mon[40290]: pgmap v1078: 162 pgs: 1 remapped, 18 peering, 2 active+undersized+remapped, 1 unknown, 140 active+clean; 626 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 51 KiB/s rd, 16 KiB/s wr, 107 op/s; 0 B/s, 43 objects/s recovering 2024-09-20T22:22:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:40 smithi067 ceph-mon[33511]: 2.0 scrub starts 2024-09-20T22:22:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:40 smithi067 ceph-mon[33511]: 2.0 scrub ok 2024-09-20T22:22:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:40 smithi067 ceph-mon[33511]: 3.10 scrub starts 2024-09-20T22:22:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:40 smithi067 ceph-mon[33511]: 3.10 scrub ok 2024-09-20T22:22:41.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:40 smithi067 ceph-mon[33511]: 4.19 scrub starts 2024-09-20T22:22:41.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:40 smithi067 ceph-mon[33511]: 4.19 scrub ok 2024-09-20T22:22:41.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:40 smithi067 ceph-mon[33511]: 4.1a deep-scrub starts 2024-09-20T22:22:41.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:40 smithi067 ceph-mon[33511]: 4.1a deep-scrub ok 2024-09-20T22:22:41.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:40 smithi067 ceph-mon[33511]: 1.3 scrub starts 2024-09-20T22:22:41.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:40 smithi067 ceph-mon[33511]: 1.3 scrub ok 2024-09-20T22:22:41.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:40 smithi067 ceph-mon[33511]: pgmap v1078: 162 pgs: 1 remapped, 18 peering, 2 active+undersized+remapped, 1 unknown, 140 active+clean; 626 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 51 KiB/s rd, 16 KiB/s wr, 107 op/s; 0 B/s, 43 objects/s recovering 2024-09-20T22:22:42.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:41 smithi067 ceph-mon[33511]: 1.1a scrub starts 2024-09-20T22:22:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:41 smithi067 ceph-mon[33511]: 1.1a scrub ok 2024-09-20T22:22:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:41 smithi067 ceph-mon[33511]: Health check update: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:22:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:41 smithi067 ceph-mon[33511]: 1.10 scrub starts 2024-09-20T22:22:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:41 smithi067 ceph-mon[33511]: 1.10 scrub ok 2024-09-20T22:22:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:41 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:22:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:41 smithi067 ceph-mon[33511]: 5.c scrub starts 2024-09-20T22:22:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:41 smithi067 ceph-mon[33511]: 5.c scrub ok 2024-09-20T22:22:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:41 smithi067 ceph-mon[33511]: 6.13 scrub starts 2024-09-20T22:22:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:41 smithi067 ceph-mon[33511]: 6.13 scrub ok 2024-09-20T22:22:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:41 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:22:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:41 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:22:42.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:41 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:22:42.015 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:41 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/117017937' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:22:42.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[40290]: 1.1a scrub starts 2024-09-20T22:22:42.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[40290]: 1.1a scrub ok 2024-09-20T22:22:42.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[40290]: Health check update: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:22:42.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[40290]: 1.10 scrub starts 2024-09-20T22:22:42.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[40290]: 1.10 scrub ok 2024-09-20T22:22:42.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:22:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[40290]: 5.c scrub starts 2024-09-20T22:22:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[40290]: 5.c scrub ok 2024-09-20T22:22:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[40290]: 6.13 scrub starts 2024-09-20T22:22:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[40290]: 6.13 scrub ok 2024-09-20T22:22:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:22:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:22:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:22:42.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/117017937' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:22:42.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[29694]: 1.1a scrub starts 2024-09-20T22:22:42.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[29694]: 1.1a scrub ok 2024-09-20T22:22:42.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[29694]: Health check update: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-20T22:22:42.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[29694]: 1.10 scrub starts 2024-09-20T22:22:42.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[29694]: 1.10 scrub ok 2024-09-20T22:22:42.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:22:42.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[29694]: 5.c scrub starts 2024-09-20T22:22:42.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[29694]: 5.c scrub ok 2024-09-20T22:22:42.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[29694]: 6.13 scrub starts 2024-09-20T22:22:42.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[29694]: 6.13 scrub ok 2024-09-20T22:22:42.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:22:42.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:22:42.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:22:42.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:41 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/117017937' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]: dispatch 2024-09-20T22:22:42.274 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-09-20T22:22:42.971 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 2 1 2024-09-20T22:22:43.230 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:43.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:22:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:22:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:43 smithi060 ceph-mon[29694]: 3.1f scrub starts 2024-09-20T22:22:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:43 smithi060 ceph-mon[29694]: 3.1f scrub ok 2024-09-20T22:22:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:43 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/117017937' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:22:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:43 smithi060 ceph-mon[29694]: osdmap e569: 8 total, 8 up, 8 in 2024-09-20T22:22:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:43 smithi060 ceph-mon[29694]: pgmap v1080: 162 pgs: 4 peering, 158 active+clean; 626 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 88 op/s; 0 B/s, 35 objects/s recovering 2024-09-20T22:22:43.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:43 smithi060 ceph-mon[29694]: 6.1a scrub starts 2024-09-20T22:22:43.537 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:43 smithi060 ceph-mon[29694]: 6.1a scrub ok 2024-09-20T22:22:43.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:43 smithi060 ceph-mon[40290]: 3.1f scrub starts 2024-09-20T22:22:43.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:43 smithi060 ceph-mon[40290]: 3.1f scrub ok 2024-09-20T22:22:43.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:43 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/117017937' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:22:43.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:43 smithi060 ceph-mon[40290]: osdmap e569: 8 total, 8 up, 8 in 2024-09-20T22:22:43.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:43 smithi060 ceph-mon[40290]: pgmap v1080: 162 pgs: 4 peering, 158 active+clean; 626 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 88 op/s; 0 B/s, 35 objects/s recovering 2024-09-20T22:22:43.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:43 smithi060 ceph-mon[40290]: 6.1a scrub starts 2024-09-20T22:22:43.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:43 smithi060 ceph-mon[40290]: 6.1a scrub ok 2024-09-20T22:22:43.684 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:43 smithi067 ceph-mon[33511]: 3.1f scrub starts 2024-09-20T22:22:43.684 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:43 smithi067 ceph-mon[33511]: 3.1f scrub ok 2024-09-20T22:22:43.684 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:43 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/117017937' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 0, "weight": 1.0}]': finished 2024-09-20T22:22:43.684 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:43 smithi067 ceph-mon[33511]: osdmap e569: 8 total, 8 up, 8 in 2024-09-20T22:22:43.684 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:43 smithi067 ceph-mon[33511]: pgmap v1080: 162 pgs: 4 peering, 158 active+clean; 626 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 88 op/s; 0 B/s, 35 objects/s recovering 2024-09-20T22:22:43.684 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:43 smithi067 ceph-mon[33511]: 6.1a scrub starts 2024-09-20T22:22:43.684 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:43 smithi067 ceph-mon[33511]: 6.1a scrub ok 2024-09-20T22:22:44.282 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.2 to 1 (10000) 2024-09-20T22:22:44.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:44 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:22:44.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:44 smithi067 ceph-mon[33511]: 3.12 deep-scrub starts 2024-09-20T22:22:44.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:44 smithi067 ceph-mon[33511]: 3.12 deep-scrub ok 2024-09-20T22:22:44.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:44 smithi067 ceph-mon[33511]: 4.d scrub starts 2024-09-20T22:22:44.514 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:44 smithi067 ceph-mon[33511]: 4.d scrub ok 2024-09-20T22:22:44.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:44 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3723779084' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:22:44.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:44 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:22:44.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:44 smithi067 ceph-mon[33511]: 5.13 deep-scrub starts 2024-09-20T22:22:44.515 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:44 smithi067 ceph-mon[33511]: 5.13 deep-scrub ok 2024-09-20T22:22:44.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:44 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:22:44.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:44 smithi060 ceph-mon[29694]: 3.12 deep-scrub starts 2024-09-20T22:22:44.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:44 smithi060 ceph-mon[29694]: 3.12 deep-scrub ok 2024-09-20T22:22:44.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:44 smithi060 ceph-mon[29694]: 4.d scrub starts 2024-09-20T22:22:44.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:44 smithi060 ceph-mon[29694]: 4.d scrub ok 2024-09-20T22:22:44.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:44 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3723779084' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:22:44.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:44 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:22:44.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:44 smithi060 ceph-mon[29694]: 5.13 deep-scrub starts 2024-09-20T22:22:44.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:44 smithi060 ceph-mon[29694]: 5.13 deep-scrub ok 2024-09-20T22:22:44.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:44 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:22:44.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:44 smithi060 ceph-mon[40290]: 3.12 deep-scrub starts 2024-09-20T22:22:44.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:44 smithi060 ceph-mon[40290]: 3.12 deep-scrub ok 2024-09-20T22:22:44.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:44 smithi060 ceph-mon[40290]: 4.d scrub starts 2024-09-20T22:22:44.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:44 smithi060 ceph-mon[40290]: 4.d scrub ok 2024-09-20T22:22:44.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:44 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3723779084' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:22:44.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:44 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:22:44.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:44 smithi060 ceph-mon[40290]: 5.13 deep-scrub starts 2024-09-20T22:22:44.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:44 smithi060 ceph-mon[40290]: 5.13 deep-scrub ok 2024-09-20T22:22:44.915 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 2 1 2024-09-20T22:22:45.172 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:45.285 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:22:45 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:22:45] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:22:45.565 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:45 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:22:45.565 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:45 smithi060 ceph-mon[29694]: osdmap e570: 8 total, 8 up, 8 in 2024-09-20T22:22:45.565 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:45 smithi060 ceph-mon[29694]: pgmap v1082: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 32 KiB/s rd, 0 B/s wr, 69 op/s; 0 B/s, 22 objects/s recovering 2024-09-20T22:22:45.565 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:45 smithi060 ceph-mon[29694]: 1.d scrub starts 2024-09-20T22:22:45.565 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:45 smithi060 ceph-mon[29694]: 1.d scrub ok 2024-09-20T22:22:45.565 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:45 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:22:45.565 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:45 smithi060 ceph-mon[40290]: osdmap e570: 8 total, 8 up, 8 in 2024-09-20T22:22:45.565 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:45 smithi060 ceph-mon[40290]: pgmap v1082: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 32 KiB/s rd, 0 B/s wr, 69 op/s; 0 B/s, 22 objects/s recovering 2024-09-20T22:22:45.566 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:45 smithi060 ceph-mon[40290]: 1.d scrub starts 2024-09-20T22:22:45.566 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:45 smithi060 ceph-mon[40290]: 1.d scrub ok 2024-09-20T22:22:45.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:45 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:22:45.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:45 smithi067 ceph-mon[33511]: osdmap e570: 8 total, 8 up, 8 in 2024-09-20T22:22:45.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:45 smithi067 ceph-mon[33511]: pgmap v1082: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 32 KiB/s rd, 0 B/s wr, 69 op/s; 0 B/s, 22 objects/s recovering 2024-09-20T22:22:45.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:45 smithi067 ceph-mon[33511]: 1.d scrub starts 2024-09-20T22:22:45.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:45 smithi067 ceph-mon[33511]: 1.d scrub ok 2024-09-20T22:22:46.292 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-09-20T22:22:46.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:46 smithi060 ceph-mon[29694]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-20T22:22:46.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:46 smithi060 ceph-mon[29694]: Cluster is now healthy 2024-09-20T22:22:46.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:46 smithi060 ceph-mon[29694]: 1.0 scrub starts 2024-09-20T22:22:46.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:46 smithi060 ceph-mon[29694]: 1.0 scrub ok 2024-09-20T22:22:46.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:46 smithi060 ceph-mon[29694]: 6.e deep-scrub starts 2024-09-20T22:22:46.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:46 smithi060 ceph-mon[29694]: 6.e deep-scrub ok 2024-09-20T22:22:46.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:46 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1940969046' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:22:46.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:46 smithi060 ceph-mon[29694]: 6.18 scrub starts 2024-09-20T22:22:46.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:46 smithi060 ceph-mon[29694]: 6.18 scrub ok 2024-09-20T22:22:46.594 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:46 smithi060 ceph-mon[40290]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-20T22:22:46.594 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:46 smithi060 ceph-mon[40290]: Cluster is now healthy 2024-09-20T22:22:46.594 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:46 smithi060 ceph-mon[40290]: 1.0 scrub starts 2024-09-20T22:22:46.594 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:46 smithi060 ceph-mon[40290]: 1.0 scrub ok 2024-09-20T22:22:46.594 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:46 smithi060 ceph-mon[40290]: 6.e deep-scrub starts 2024-09-20T22:22:46.594 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:46 smithi060 ceph-mon[40290]: 6.e deep-scrub ok 2024-09-20T22:22:46.594 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:46 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1940969046' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:22:46.594 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:46 smithi060 ceph-mon[40290]: 6.18 scrub starts 2024-09-20T22:22:46.594 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:46 smithi060 ceph-mon[40290]: 6.18 scrub ok 2024-09-20T22:22:46.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:46 smithi067 ceph-mon[33511]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-20T22:22:46.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:46 smithi067 ceph-mon[33511]: Cluster is now healthy 2024-09-20T22:22:46.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:46 smithi067 ceph-mon[33511]: 1.0 scrub starts 2024-09-20T22:22:46.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:46 smithi067 ceph-mon[33511]: 1.0 scrub ok 2024-09-20T22:22:46.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:46 smithi067 ceph-mon[33511]: 6.e deep-scrub starts 2024-09-20T22:22:46.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:46 smithi067 ceph-mon[33511]: 6.e deep-scrub ok 2024-09-20T22:22:46.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:46 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1940969046' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]: dispatch 2024-09-20T22:22:46.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:46 smithi067 ceph-mon[33511]: 6.18 scrub starts 2024-09-20T22:22:46.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:46 smithi067 ceph-mon[33511]: 6.18 scrub ok 2024-09-20T22:22:46.992 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 3 1 2024-09-20T22:22:47.249 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:47.553 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:47 smithi060 ceph-mon[29694]: 6.1f scrub starts 2024-09-20T22:22:47.553 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:47 smithi060 ceph-mon[29694]: 6.1f scrub ok 2024-09-20T22:22:47.553 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:47 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1940969046' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:22:47.553 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:47 smithi060 ceph-mon[29694]: osdmap e571: 8 total, 8 up, 8 in 2024-09-20T22:22:47.553 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:47 smithi060 ceph-mon[29694]: pgmap v1084: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-09-20T22:22:47.553 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:47 smithi060 ceph-mon[29694]: osdmap e572: 8 total, 8 up, 8 in 2024-09-20T22:22:47.554 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:47 smithi060 ceph-mon[40290]: 6.1f scrub starts 2024-09-20T22:22:47.554 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:47 smithi060 ceph-mon[40290]: 6.1f scrub ok 2024-09-20T22:22:47.554 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:47 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1940969046' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:22:47.554 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:47 smithi060 ceph-mon[40290]: osdmap e571: 8 total, 8 up, 8 in 2024-09-20T22:22:47.554 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:47 smithi060 ceph-mon[40290]: pgmap v1084: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-09-20T22:22:47.554 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:47 smithi060 ceph-mon[40290]: osdmap e572: 8 total, 8 up, 8 in 2024-09-20T22:22:47.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:47 smithi067 ceph-mon[33511]: 6.1f scrub starts 2024-09-20T22:22:47.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:47 smithi067 ceph-mon[33511]: 6.1f scrub ok 2024-09-20T22:22:47.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:47 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1940969046' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 2, "weight": 1.0}]': finished 2024-09-20T22:22:47.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:47 smithi067 ceph-mon[33511]: osdmap e571: 8 total, 8 up, 8 in 2024-09-20T22:22:47.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:47 smithi067 ceph-mon[33511]: pgmap v1084: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 0 B/s, 0 objects/s recovering 2024-09-20T22:22:47.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:47 smithi067 ceph-mon[33511]: osdmap e572: 8 total, 8 up, 8 in 2024-09-20T22:22:48.308 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.3 to 1 (10000) 2024-09-20T22:22:48.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:48 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3252450420' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:22:48.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:48 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:22:48.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:48 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3252450420' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:22:48.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:48 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:22:48.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:48 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3252450420' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:22:48.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:48 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:22:49.132 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 3 1 2024-09-20T22:22:49.395 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:49.517 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:49 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:22:49.517 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:49 smithi060 ceph-mon[29694]: osdmap e573: 8 total, 8 up, 8 in 2024-09-20T22:22:49.517 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:49 smithi060 ceph-mon[29694]: pgmap v1087: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:22:49.517 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:49 smithi060 ceph-mon[29694]: 5.18 scrub starts 2024-09-20T22:22:49.517 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:49 smithi060 ceph-mon[29694]: 5.18 scrub ok 2024-09-20T22:22:49.518 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:49 smithi060 ceph-mon[29694]: 5.4 scrub starts 2024-09-20T22:22:49.518 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:49 smithi060 ceph-mon[29694]: 5.4 scrub ok 2024-09-20T22:22:49.518 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:49 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:22:49.518 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:49 smithi060 ceph-mon[40290]: osdmap e573: 8 total, 8 up, 8 in 2024-09-20T22:22:49.518 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:49 smithi060 ceph-mon[40290]: pgmap v1087: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:22:49.518 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:49 smithi060 ceph-mon[40290]: 5.18 scrub starts 2024-09-20T22:22:49.518 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:49 smithi060 ceph-mon[40290]: 5.18 scrub ok 2024-09-20T22:22:49.518 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:49 smithi060 ceph-mon[40290]: 5.4 scrub starts 2024-09-20T22:22:49.518 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:49 smithi060 ceph-mon[40290]: 5.4 scrub ok 2024-09-20T22:22:49.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:49 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:22:49.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:49 smithi067 ceph-mon[33511]: osdmap e573: 8 total, 8 up, 8 in 2024-09-20T22:22:49.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:49 smithi067 ceph-mon[33511]: pgmap v1087: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:22:49.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:49 smithi067 ceph-mon[33511]: 5.18 scrub starts 2024-09-20T22:22:49.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:49 smithi067 ceph-mon[33511]: 5.18 scrub ok 2024-09-20T22:22:49.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:49 smithi067 ceph-mon[33511]: 5.4 scrub starts 2024-09-20T22:22:49.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:49 smithi067 ceph-mon[33511]: 5.4 scrub ok 2024-09-20T22:22:50.326 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-09-20T22:22:50.662 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:50 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/787906088' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:22:50.662 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:50 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:22:50.662 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:50 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/787906088' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:22:50.662 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:50 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:22:50.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:50 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/787906088' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:22:50.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:50 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]: dispatch 2024-09-20T22:22:51.085 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 5 1 2024-09-20T22:22:51.356 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:51.593 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:51 smithi060 ceph-mon[29694]: 5.1f scrub starts 2024-09-20T22:22:51.594 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:51 smithi060 ceph-mon[29694]: 5.1f scrub ok 2024-09-20T22:22:51.594 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:51 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:22:51.594 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:51 smithi060 ceph-mon[29694]: osdmap e574: 8 total, 8 up, 8 in 2024-09-20T22:22:51.594 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:51 smithi060 ceph-mon[29694]: pgmap v1089: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail 2024-09-20T22:22:51.594 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:51 smithi060 ceph-mon[40290]: 5.1f scrub starts 2024-09-20T22:22:51.594 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:51 smithi060 ceph-mon[40290]: 5.1f scrub ok 2024-09-20T22:22:51.594 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:51 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:22:51.594 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:51 smithi060 ceph-mon[40290]: osdmap e574: 8 total, 8 up, 8 in 2024-09-20T22:22:51.594 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:51 smithi060 ceph-mon[40290]: pgmap v1089: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail 2024-09-20T22:22:51.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:51 smithi067 ceph-mon[33511]: 5.1f scrub starts 2024-09-20T22:22:51.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:51 smithi067 ceph-mon[33511]: 5.1f scrub ok 2024-09-20T22:22:51.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:51 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 3, "weight": 1.0}]': finished 2024-09-20T22:22:51.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:51 smithi067 ceph-mon[33511]: osdmap e574: 8 total, 8 up, 8 in 2024-09-20T22:22:51.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:51 smithi067 ceph-mon[33511]: pgmap v1089: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail 2024-09-20T22:22:52.655 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:52 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:22:52.655 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:52 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:22:52.655 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:52 smithi060 ceph-mon[29694]: 6.1d deep-scrub starts 2024-09-20T22:22:52.655 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:52 smithi060 ceph-mon[29694]: 6.1d deep-scrub ok 2024-09-20T22:22:52.655 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:52 smithi060 ceph-mon[29694]: osdmap e575: 8 total, 8 up, 8 in 2024-09-20T22:22:52.655 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:52 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:22:52.655 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:52 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:22:52.655 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:52 smithi060 ceph-mon[29694]: osdmap e576: 8 total, 8 up, 8 in 2024-09-20T22:22:52.656 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:52 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3900513462' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:22:52.656 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:52 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:22:52.656 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:52 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:22:52.656 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:52 smithi060 ceph-mon[40290]: 6.1d deep-scrub starts 2024-09-20T22:22:52.656 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:52 smithi060 ceph-mon[40290]: 6.1d deep-scrub ok 2024-09-20T22:22:52.656 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:52 smithi060 ceph-mon[40290]: osdmap e575: 8 total, 8 up, 8 in 2024-09-20T22:22:52.656 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:52 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:22:52.656 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:52 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:22:52.656 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:52 smithi060 ceph-mon[40290]: osdmap e576: 8 total, 8 up, 8 in 2024-09-20T22:22:52.656 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:52 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3900513462' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:22:52.709 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.5 to 1 (10000) 2024-09-20T22:22:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:52 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:22:52.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:52 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:22:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:52 smithi067 ceph-mon[33511]: 6.1d deep-scrub starts 2024-09-20T22:22:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:52 smithi067 ceph-mon[33511]: 6.1d deep-scrub ok 2024-09-20T22:22:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:52 smithi067 ceph-mon[33511]: osdmap e575: 8 total, 8 up, 8 in 2024-09-20T22:22:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:52 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:22:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:52 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:22:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:52 smithi067 ceph-mon[33511]: osdmap e576: 8 total, 8 up, 8 in 2024-09-20T22:22:52.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:52 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3900513462' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:22:53.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:22:52 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:22:53.504 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 5 1 2024-09-20T22:22:53.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:53 smithi067 ceph-mon[33511]: 1.19 scrub starts 2024-09-20T22:22:53.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:53 smithi067 ceph-mon[33511]: 1.19 scrub ok 2024-09-20T22:22:53.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:53 smithi067 ceph-mon[33511]: pgmap v1092: 162 pgs: 162 active+clean; 609 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 4.1 KiB/s rd, 27 KiB/s wr, 5 op/s; 19 KiB/s, 0 objects/s recovering 2024-09-20T22:22:53.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:53 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3900513462' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:22:53.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:53 smithi067 ceph-mon[33511]: osdmap e577: 8 total, 8 up, 8 in 2024-09-20T22:22:53.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:53 smithi067 ceph-mon[33511]: 1.11 scrub starts 2024-09-20T22:22:53.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:53 smithi067 ceph-mon[33511]: 1.11 scrub ok 2024-09-20T22:22:53.767 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:53.785 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:53 smithi060 ceph-mon[40290]: 1.19 scrub starts 2024-09-20T22:22:53.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:53 smithi060 ceph-mon[40290]: 1.19 scrub ok 2024-09-20T22:22:53.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:53 smithi060 ceph-mon[40290]: pgmap v1092: 162 pgs: 162 active+clean; 609 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 4.1 KiB/s rd, 27 KiB/s wr, 5 op/s; 19 KiB/s, 0 objects/s recovering 2024-09-20T22:22:53.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:53 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3900513462' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:22:53.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:53 smithi060 ceph-mon[40290]: osdmap e577: 8 total, 8 up, 8 in 2024-09-20T22:22:53.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:53 smithi060 ceph-mon[40290]: 1.11 scrub starts 2024-09-20T22:22:53.786 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:53 smithi060 ceph-mon[40290]: 1.11 scrub ok 2024-09-20T22:22:53.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:53 smithi060 ceph-mon[29694]: 1.19 scrub starts 2024-09-20T22:22:53.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:53 smithi060 ceph-mon[29694]: 1.19 scrub ok 2024-09-20T22:22:53.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:53 smithi060 ceph-mon[29694]: pgmap v1092: 162 pgs: 162 active+clean; 609 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 4.1 KiB/s rd, 27 KiB/s wr, 5 op/s; 19 KiB/s, 0 objects/s recovering 2024-09-20T22:22:53.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:53 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3900513462' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:22:53.786 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:53 smithi060 ceph-mon[29694]: osdmap e577: 8 total, 8 up, 8 in 2024-09-20T22:22:53.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:53 smithi060 ceph-mon[29694]: 1.11 scrub starts 2024-09-20T22:22:53.787 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:53 smithi060 ceph-mon[29694]: 1.11 scrub ok 2024-09-20T22:22:54.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:54 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:22:54.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:54 smithi060 ceph-mon[29694]: 5.11 deep-scrub starts 2024-09-20T22:22:54.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:54 smithi060 ceph-mon[29694]: 4.0 scrub starts 2024-09-20T22:22:54.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:54 smithi060 ceph-mon[29694]: 5.11 deep-scrub ok 2024-09-20T22:22:54.536 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:54 smithi060 ceph-mon[29694]: 4.0 scrub ok 2024-09-20T22:22:54.536 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:54 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:22:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:54 smithi060 ceph-mon[40290]: 5.11 deep-scrub starts 2024-09-20T22:22:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:54 smithi060 ceph-mon[40290]: 4.0 scrub starts 2024-09-20T22:22:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:54 smithi060 ceph-mon[40290]: 5.11 deep-scrub ok 2024-09-20T22:22:54.537 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:54 smithi060 ceph-mon[40290]: 4.0 scrub ok 2024-09-20T22:22:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:54 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:22:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:54 smithi067 ceph-mon[33511]: 5.11 deep-scrub starts 2024-09-20T22:22:54.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:54 smithi067 ceph-mon[33511]: 4.0 scrub starts 2024-09-20T22:22:54.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:54 smithi067 ceph-mon[33511]: 5.11 deep-scrub ok 2024-09-20T22:22:54.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:54 smithi067 ceph-mon[33511]: 4.0 scrub ok 2024-09-20T22:22:55.335 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-09-20T22:22:55.335 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:22:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:22:55] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:22:55.653 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:55 smithi060 ceph-mon[29694]: 3.7 deep-scrub starts 2024-09-20T22:22:55.653 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:55 smithi060 ceph-mon[29694]: 3.7 deep-scrub ok 2024-09-20T22:22:55.653 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:55 smithi060 ceph-mon[29694]: 5.12 scrub starts 2024-09-20T22:22:55.653 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:55 smithi060 ceph-mon[29694]: 5.12 scrub ok 2024-09-20T22:22:55.653 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:55 smithi060 ceph-mon[29694]: pgmap v1094: 162 pgs: 162 active+clean; 609 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 4.2 KiB/s rd, 27 KiB/s wr, 5 op/s; 19 KiB/s, 0 objects/s recovering 2024-09-20T22:22:55.653 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:55 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3475208641' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:22:55.653 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:55 smithi060 ceph-mon[29694]: 4.15 scrub starts 2024-09-20T22:22:55.653 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:55 smithi060 ceph-mon[29694]: 4.15 scrub ok 2024-09-20T22:22:55.654 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:55 smithi060 ceph-mon[40290]: 3.7 deep-scrub starts 2024-09-20T22:22:55.654 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:55 smithi060 ceph-mon[40290]: 3.7 deep-scrub ok 2024-09-20T22:22:55.654 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:55 smithi060 ceph-mon[40290]: 5.12 scrub starts 2024-09-20T22:22:55.654 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:55 smithi060 ceph-mon[40290]: 5.12 scrub ok 2024-09-20T22:22:55.654 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:55 smithi060 ceph-mon[40290]: pgmap v1094: 162 pgs: 162 active+clean; 609 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 4.2 KiB/s rd, 27 KiB/s wr, 5 op/s; 19 KiB/s, 0 objects/s recovering 2024-09-20T22:22:55.654 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:55 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3475208641' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:22:55.654 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:55 smithi060 ceph-mon[40290]: 4.15 scrub starts 2024-09-20T22:22:55.654 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:55 smithi060 ceph-mon[40290]: 4.15 scrub ok 2024-09-20T22:22:55.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:55 smithi067 ceph-mon[33511]: 3.7 deep-scrub starts 2024-09-20T22:22:55.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:55 smithi067 ceph-mon[33511]: 3.7 deep-scrub ok 2024-09-20T22:22:55.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:55 smithi067 ceph-mon[33511]: 5.12 scrub starts 2024-09-20T22:22:55.915 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:55 smithi067 ceph-mon[33511]: 5.12 scrub ok 2024-09-20T22:22:55.916 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:55 smithi067 ceph-mon[33511]: pgmap v1094: 162 pgs: 162 active+clean; 609 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 4.2 KiB/s rd, 27 KiB/s wr, 5 op/s; 19 KiB/s, 0 objects/s recovering 2024-09-20T22:22:55.916 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:55 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3475208641' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]: dispatch 2024-09-20T22:22:55.916 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:55 smithi067 ceph-mon[33511]: 4.15 scrub starts 2024-09-20T22:22:55.916 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:55 smithi067 ceph-mon[33511]: 4.15 scrub ok 2024-09-20T22:22:56.041 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 6 1 2024-09-20T22:22:56.306 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:56.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:56 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3475208641' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:22:56.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:56 smithi060 ceph-mon[29694]: osdmap e578: 8 total, 8 up, 8 in 2024-09-20T22:22:56.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:56 smithi060 ceph-mon[29694]: 4.1 scrub starts 2024-09-20T22:22:56.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:56 smithi060 ceph-mon[29694]: 4.1 scrub ok 2024-09-20T22:22:56.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:56 smithi060 ceph-mon[29694]: 3.6 scrub starts 2024-09-20T22:22:56.682 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:56 smithi060 ceph-mon[29694]: 3.6 scrub ok 2024-09-20T22:22:56.682 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:56 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3475208641' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:22:56.682 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:56 smithi060 ceph-mon[40290]: osdmap e578: 8 total, 8 up, 8 in 2024-09-20T22:22:56.683 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:56 smithi060 ceph-mon[40290]: 4.1 scrub starts 2024-09-20T22:22:56.683 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:56 smithi060 ceph-mon[40290]: 4.1 scrub ok 2024-09-20T22:22:56.683 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:56 smithi060 ceph-mon[40290]: 3.6 scrub starts 2024-09-20T22:22:56.683 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:56 smithi060 ceph-mon[40290]: 3.6 scrub ok 2024-09-20T22:22:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:56 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3475208641' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 5, "weight": 1.0}]': finished 2024-09-20T22:22:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:56 smithi067 ceph-mon[33511]: osdmap e578: 8 total, 8 up, 8 in 2024-09-20T22:22:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:56 smithi067 ceph-mon[33511]: 4.1 scrub starts 2024-09-20T22:22:56.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:56 smithi067 ceph-mon[33511]: 4.1 scrub ok 2024-09-20T22:22:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:56 smithi067 ceph-mon[33511]: 3.6 scrub starts 2024-09-20T22:22:56.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:56 smithi067 ceph-mon[33511]: 3.6 scrub ok 2024-09-20T22:22:57.710 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.6 to 1 (10000) 2024-09-20T22:22:57.710 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:57 smithi060 ceph-mon[29694]: 5.2 scrub starts 2024-09-20T22:22:57.711 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:57 smithi060 ceph-mon[29694]: 5.2 scrub ok 2024-09-20T22:22:57.711 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:57 smithi060 ceph-mon[29694]: pgmap v1096: 162 pgs: 162 active+clean; 609 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 4 op/s; 15 KiB/s, 0 objects/s recovering 2024-09-20T22:22:57.711 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:57 smithi060 ceph-mon[29694]: 1.1 scrub starts 2024-09-20T22:22:57.711 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:57 smithi060 ceph-mon[29694]: 1.1 scrub ok 2024-09-20T22:22:57.711 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:57 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1810933957' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:22:57.711 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:57 smithi060 ceph-mon[29694]: 6.9 deep-scrub starts 2024-09-20T22:22:57.711 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:57 smithi060 ceph-mon[29694]: 6.9 deep-scrub ok 2024-09-20T22:22:57.711 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:57 smithi060 ceph-mon[40290]: 5.2 scrub starts 2024-09-20T22:22:57.711 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:57 smithi060 ceph-mon[40290]: 5.2 scrub ok 2024-09-20T22:22:57.711 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:57 smithi060 ceph-mon[40290]: pgmap v1096: 162 pgs: 162 active+clean; 609 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 4 op/s; 15 KiB/s, 0 objects/s recovering 2024-09-20T22:22:57.712 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:57 smithi060 ceph-mon[40290]: 1.1 scrub starts 2024-09-20T22:22:57.712 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:57 smithi060 ceph-mon[40290]: 1.1 scrub ok 2024-09-20T22:22:57.712 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:57 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1810933957' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:22:57.712 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:57 smithi060 ceph-mon[40290]: 6.9 deep-scrub starts 2024-09-20T22:22:57.712 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:57 smithi060 ceph-mon[40290]: 6.9 deep-scrub ok 2024-09-20T22:22:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:57 smithi067 ceph-mon[33511]: 5.2 scrub starts 2024-09-20T22:22:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:57 smithi067 ceph-mon[33511]: 5.2 scrub ok 2024-09-20T22:22:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:57 smithi067 ceph-mon[33511]: pgmap v1096: 162 pgs: 162 active+clean; 609 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 3.3 KiB/s rd, 22 KiB/s wr, 4 op/s; 15 KiB/s, 0 objects/s recovering 2024-09-20T22:22:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:57 smithi067 ceph-mon[33511]: 1.1 scrub starts 2024-09-20T22:22:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:57 smithi067 ceph-mon[33511]: 1.1 scrub ok 2024-09-20T22:22:57.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:57 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1810933957' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:22:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:57 smithi067 ceph-mon[33511]: 6.9 deep-scrub starts 2024-09-20T22:22:57.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:57 smithi067 ceph-mon[33511]: 6.9 deep-scrub ok 2024-09-20T22:22:58.396 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 6 1 2024-09-20T22:22:58.657 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:22:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:58 smithi067 ceph-mon[33511]: 5.0 scrub starts 2024-09-20T22:22:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:58 smithi067 ceph-mon[33511]: 5.0 scrub ok 2024-09-20T22:22:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:58 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1810933957' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:22:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:58 smithi067 ceph-mon[33511]: osdmap e579: 8 total, 8 up, 8 in 2024-09-20T22:22:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:58 smithi067 ceph-mon[33511]: 4.13 scrub starts 2024-09-20T22:22:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:58 smithi067 ceph-mon[33511]: 4.13 scrub ok 2024-09-20T22:22:58.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:58 smithi067 ceph-mon[33511]: 5.3 scrub starts 2024-09-20T22:22:58.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:58 smithi067 ceph-mon[33511]: 5.3 scrub ok 2024-09-20T22:22:58.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:58 smithi060 ceph-mon[29694]: 5.0 scrub starts 2024-09-20T22:22:58.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:58 smithi060 ceph-mon[29694]: 5.0 scrub ok 2024-09-20T22:22:58.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:58 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1810933957' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:22:58.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:58 smithi060 ceph-mon[29694]: osdmap e579: 8 total, 8 up, 8 in 2024-09-20T22:22:58.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:58 smithi060 ceph-mon[29694]: 4.13 scrub starts 2024-09-20T22:22:58.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:58 smithi060 ceph-mon[29694]: 4.13 scrub ok 2024-09-20T22:22:58.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:58 smithi060 ceph-mon[29694]: 5.3 scrub starts 2024-09-20T22:22:58.779 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:58 smithi060 ceph-mon[29694]: 5.3 scrub ok 2024-09-20T22:22:58.779 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:58 smithi060 ceph-mon[40290]: 5.0 scrub starts 2024-09-20T22:22:58.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:58 smithi060 ceph-mon[40290]: 5.0 scrub ok 2024-09-20T22:22:58.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:58 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1810933957' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:22:58.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:58 smithi060 ceph-mon[40290]: osdmap e579: 8 total, 8 up, 8 in 2024-09-20T22:22:58.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:58 smithi060 ceph-mon[40290]: 4.13 scrub starts 2024-09-20T22:22:58.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:58 smithi060 ceph-mon[40290]: 4.13 scrub ok 2024-09-20T22:22:58.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:58 smithi060 ceph-mon[40290]: 5.3 scrub starts 2024-09-20T22:22:58.780 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:58 smithi060 ceph-mon[40290]: 5.3 scrub ok 2024-09-20T22:22:59.714 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-09-20T22:22:59.714 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:59 smithi060 ceph-mon[29694]: 5.17 scrub starts 2024-09-20T22:22:59.714 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:59 smithi060 ceph-mon[29694]: 5.17 scrub ok 2024-09-20T22:22:59.714 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:59 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:22:59.714 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:59 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:22:59.714 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:59 smithi060 ceph-mon[29694]: pgmap v1098: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail 2024-09-20T22:22:59.714 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:59 smithi060 ceph-mon[29694]: osdmap e580: 8 total, 8 up, 8 in 2024-09-20T22:22:59.715 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:59 smithi060 ceph-mon[29694]: 5.16 scrub starts 2024-09-20T22:22:59.715 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:59 smithi060 ceph-mon[29694]: 5.16 scrub ok 2024-09-20T22:22:59.715 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:22:59 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/616193276' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:22:59.715 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:59 smithi060 ceph-mon[40290]: 5.17 scrub starts 2024-09-20T22:22:59.715 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:59 smithi060 ceph-mon[40290]: 5.17 scrub ok 2024-09-20T22:22:59.715 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:59 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:22:59.715 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:59 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:22:59.715 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:59 smithi060 ceph-mon[40290]: pgmap v1098: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail 2024-09-20T22:22:59.715 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:59 smithi060 ceph-mon[40290]: osdmap e580: 8 total, 8 up, 8 in 2024-09-20T22:22:59.715 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:59 smithi060 ceph-mon[40290]: 5.16 scrub starts 2024-09-20T22:22:59.715 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:59 smithi060 ceph-mon[40290]: 5.16 scrub ok 2024-09-20T22:22:59.715 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:22:59 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/616193276' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:22:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:59 smithi067 ceph-mon[33511]: 5.17 scrub starts 2024-09-20T22:22:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:59 smithi067 ceph-mon[33511]: 5.17 scrub ok 2024-09-20T22:22:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:59 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:22:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:59 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:22:59.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:59 smithi067 ceph-mon[33511]: pgmap v1098: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail 2024-09-20T22:22:59.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:59 smithi067 ceph-mon[33511]: osdmap e580: 8 total, 8 up, 8 in 2024-09-20T22:22:59.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:59 smithi067 ceph-mon[33511]: 5.16 scrub starts 2024-09-20T22:22:59.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:59 smithi067 ceph-mon[33511]: 5.16 scrub ok 2024-09-20T22:22:59.764 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:22:59 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/616193276' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]: dispatch 2024-09-20T22:23:00.458 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 7 1 2024-09-20T22:23:00.724 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:00.973 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:00 smithi060 ceph-mon[29694]: 6.3 scrub starts 2024-09-20T22:23:00.973 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:00 smithi060 ceph-mon[29694]: 6.3 scrub ok 2024-09-20T22:23:00.974 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:00 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/616193276' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:23:00.974 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:00 smithi060 ceph-mon[29694]: osdmap e581: 8 total, 8 up, 8 in 2024-09-20T22:23:00.974 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:00 smithi060 ceph-mon[29694]: 3.2 deep-scrub starts 2024-09-20T22:23:00.974 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:00 smithi060 ceph-mon[29694]: 3.2 deep-scrub ok 2024-09-20T22:23:00.974 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:00 smithi060 ceph-mon[29694]: pgmap v1101: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 70 op/s 2024-09-20T22:23:00.974 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:00 smithi060 ceph-mon[40290]: 6.3 scrub starts 2024-09-20T22:23:00.974 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:00 smithi060 ceph-mon[40290]: 6.3 scrub ok 2024-09-20T22:23:00.974 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:00 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/616193276' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:23:00.974 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:00 smithi060 ceph-mon[40290]: osdmap e581: 8 total, 8 up, 8 in 2024-09-20T22:23:00.974 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:00 smithi060 ceph-mon[40290]: 3.2 deep-scrub starts 2024-09-20T22:23:00.974 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:00 smithi060 ceph-mon[40290]: 3.2 deep-scrub ok 2024-09-20T22:23:00.974 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:00 smithi060 ceph-mon[40290]: pgmap v1101: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 70 op/s 2024-09-20T22:23:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:00 smithi067 ceph-mon[33511]: 6.3 scrub starts 2024-09-20T22:23:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:00 smithi067 ceph-mon[33511]: 6.3 scrub ok 2024-09-20T22:23:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:00 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/616193276' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 6, "weight": 1.0}]': finished 2024-09-20T22:23:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:00 smithi067 ceph-mon[33511]: osdmap e581: 8 total, 8 up, 8 in 2024-09-20T22:23:01.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:00 smithi067 ceph-mon[33511]: 3.2 deep-scrub starts 2024-09-20T22:23:01.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:00 smithi067 ceph-mon[33511]: 3.2 deep-scrub ok 2024-09-20T22:23:01.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:00 smithi067 ceph-mon[33511]: pgmap v1101: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 42 KiB/s rd, 0 B/s wr, 70 op/s 2024-09-20T22:23:01.714 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.7 to 1 (10000) 2024-09-20T22:23:02.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:01 smithi067 ceph-mon[33511]: 4.18 scrub starts 2024-09-20T22:23:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:01 smithi067 ceph-mon[33511]: 4.18 scrub ok 2024-09-20T22:23:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:01 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:23:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:01 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:23:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:01 smithi067 ceph-mon[33511]: 4.11 scrub starts 2024-09-20T22:23:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:01 smithi067 ceph-mon[33511]: 4.11 scrub ok 2024-09-20T22:23:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:01 smithi067 ceph-mon[33511]: 3.a scrub starts 2024-09-20T22:23:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:01 smithi067 ceph-mon[33511]: 3.a scrub ok 2024-09-20T22:23:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:01 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3172760487' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:23:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:01 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:23:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:01 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:23:02.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:01 smithi067 ceph-mon[33511]: osdmap e582: 8 total, 8 up, 8 in 2024-09-20T22:23:02.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[29694]: 4.18 scrub starts 2024-09-20T22:23:02.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[29694]: 4.18 scrub ok 2024-09-20T22:23:02.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:23:02.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:23:02.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[29694]: 4.11 scrub starts 2024-09-20T22:23:02.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[29694]: 4.11 scrub ok 2024-09-20T22:23:02.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[29694]: 3.a scrub starts 2024-09-20T22:23:02.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[29694]: 3.a scrub ok 2024-09-20T22:23:02.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3172760487' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:23:02.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:23:02.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:23:02.038 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[29694]: osdmap e582: 8 total, 8 up, 8 in 2024-09-20T22:23:02.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[40290]: 4.18 scrub starts 2024-09-20T22:23:02.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[40290]: 4.18 scrub ok 2024-09-20T22:23:02.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:23:02.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:23:02.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[40290]: 4.11 scrub starts 2024-09-20T22:23:02.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[40290]: 4.11 scrub ok 2024-09-20T22:23:02.038 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[40290]: 3.a scrub starts 2024-09-20T22:23:02.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[40290]: 3.a scrub ok 2024-09-20T22:23:02.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3172760487' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:23:02.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:23:02.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:23:02.039 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:01 smithi060 ceph-mon[40290]: osdmap e582: 8 total, 8 up, 8 in 2024-09-20T22:23:02.675 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 7 1 2024-09-20T22:23:02.942 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:03.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:23:02 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:23:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:02 smithi067 ceph-mon[33511]: 6.5 scrub starts 2024-09-20T22:23:03.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:02 smithi067 ceph-mon[33511]: 6.5 scrub ok 2024-09-20T22:23:03.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:02 smithi067 ceph-mon[33511]: 5.f deep-scrub starts 2024-09-20T22:23:03.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:02 smithi067 ceph-mon[33511]: 5.f deep-scrub ok 2024-09-20T22:23:03.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:02 smithi067 ceph-mon[33511]: 6.0 scrub starts 2024-09-20T22:23:03.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:02 smithi067 ceph-mon[33511]: 6.0 scrub ok 2024-09-20T22:23:03.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:02 smithi067 ceph-mon[33511]: 6.4 scrub starts 2024-09-20T22:23:03.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:02 smithi067 ceph-mon[33511]: 6.4 scrub ok 2024-09-20T22:23:03.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:02 smithi067 ceph-mon[33511]: pgmap v1103: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 78 KiB/s rd, 0 B/s wr, 129 op/s 2024-09-20T22:23:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:02 smithi060 ceph-mon[40290]: 6.5 scrub starts 2024-09-20T22:23:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:02 smithi060 ceph-mon[40290]: 6.5 scrub ok 2024-09-20T22:23:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:02 smithi060 ceph-mon[40290]: 5.f deep-scrub starts 2024-09-20T22:23:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:02 smithi060 ceph-mon[40290]: 5.f deep-scrub ok 2024-09-20T22:23:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:02 smithi060 ceph-mon[40290]: 6.0 scrub starts 2024-09-20T22:23:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:02 smithi060 ceph-mon[40290]: 6.0 scrub ok 2024-09-20T22:23:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:02 smithi060 ceph-mon[40290]: 6.4 scrub starts 2024-09-20T22:23:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:02 smithi060 ceph-mon[40290]: 6.4 scrub ok 2024-09-20T22:23:03.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:02 smithi060 ceph-mon[40290]: pgmap v1103: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 78 KiB/s rd, 0 B/s wr, 129 op/s 2024-09-20T22:23:03.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:02 smithi060 ceph-mon[29694]: 6.5 scrub starts 2024-09-20T22:23:03.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:02 smithi060 ceph-mon[29694]: 6.5 scrub ok 2024-09-20T22:23:03.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:02 smithi060 ceph-mon[29694]: 5.f deep-scrub starts 2024-09-20T22:23:03.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:02 smithi060 ceph-mon[29694]: 5.f deep-scrub ok 2024-09-20T22:23:03.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:02 smithi060 ceph-mon[29694]: 6.0 scrub starts 2024-09-20T22:23:03.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:02 smithi060 ceph-mon[29694]: 6.0 scrub ok 2024-09-20T22:23:03.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:02 smithi060 ceph-mon[29694]: 6.4 scrub starts 2024-09-20T22:23:03.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:02 smithi060 ceph-mon[29694]: 6.4 scrub ok 2024-09-20T22:23:03.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:02 smithi060 ceph-mon[29694]: pgmap v1103: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 78 KiB/s rd, 0 B/s wr, 129 op/s 2024-09-20T22:23:03.730 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-09-20T22:23:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:03 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:23:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:03 smithi067 ceph-mon[33511]: 4.14 deep-scrub starts 2024-09-20T22:23:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:03 smithi067 ceph-mon[33511]: 4.14 deep-scrub ok 2024-09-20T22:23:04.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:03 smithi067 ceph-mon[33511]: 5.1d scrub starts 2024-09-20T22:23:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:03 smithi067 ceph-mon[33511]: 5.1d scrub ok 2024-09-20T22:23:04.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:03 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2999022178' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:23:04.014 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:03 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:23:04.014 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:03 smithi060 ceph-mon[29694]: 4.14 deep-scrub starts 2024-09-20T22:23:04.014 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:03 smithi060 ceph-mon[29694]: 4.14 deep-scrub ok 2024-09-20T22:23:04.014 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:03 smithi060 ceph-mon[29694]: 5.1d scrub starts 2024-09-20T22:23:04.015 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:03 smithi060 ceph-mon[29694]: 5.1d scrub ok 2024-09-20T22:23:04.015 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:03 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2999022178' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:23:04.015 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:03 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:23:04.015 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:03 smithi060 ceph-mon[40290]: 4.14 deep-scrub starts 2024-09-20T22:23:04.015 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:03 smithi060 ceph-mon[40290]: 4.14 deep-scrub ok 2024-09-20T22:23:04.015 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:03 smithi060 ceph-mon[40290]: 5.1d scrub starts 2024-09-20T22:23:04.015 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:03 smithi060 ceph-mon[40290]: 5.1d scrub ok 2024-09-20T22:23:04.015 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:03 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2999022178' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]: dispatch 2024-09-20T22:23:04.443 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 1 1 2024-09-20T22:23:04.707 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:05.010 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[40290]: 6.8 scrub starts 2024-09-20T22:23:05.011 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[40290]: 6.8 scrub ok 2024-09-20T22:23:05.011 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2999022178' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:23:05.011 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[40290]: osdmap e583: 8 total, 8 up, 8 in 2024-09-20T22:23:05.011 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[40290]: 6.2 scrub starts 2024-09-20T22:23:05.011 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[40290]: 6.2 scrub ok 2024-09-20T22:23:05.011 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[40290]: 1.12 scrub starts 2024-09-20T22:23:05.011 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[40290]: 1.12 scrub ok 2024-09-20T22:23:05.011 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[40290]: 6.11 scrub starts 2024-09-20T22:23:05.011 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[40290]: 6.11 scrub ok 2024-09-20T22:23:05.011 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[40290]: 4.b scrub starts 2024-09-20T22:23:05.011 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[40290]: 4.b scrub ok 2024-09-20T22:23:05.011 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[40290]: 4.2 deep-scrub starts 2024-09-20T22:23:05.011 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[40290]: 4.2 deep-scrub ok 2024-09-20T22:23:05.012 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[40290]: pgmap v1105: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 67 KiB/s rd, 0 B/s wr, 110 op/s 2024-09-20T22:23:05.012 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[29694]: 6.8 scrub starts 2024-09-20T22:23:05.012 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[29694]: 6.8 scrub ok 2024-09-20T22:23:05.012 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2999022178' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:23:05.012 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[29694]: osdmap e583: 8 total, 8 up, 8 in 2024-09-20T22:23:05.012 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[29694]: 6.2 scrub starts 2024-09-20T22:23:05.012 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[29694]: 6.2 scrub ok 2024-09-20T22:23:05.012 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[29694]: 1.12 scrub starts 2024-09-20T22:23:05.012 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[29694]: 1.12 scrub ok 2024-09-20T22:23:05.012 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[29694]: 6.11 scrub starts 2024-09-20T22:23:05.012 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[29694]: 6.11 scrub ok 2024-09-20T22:23:05.012 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[29694]: 4.b scrub starts 2024-09-20T22:23:05.013 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[29694]: 4.b scrub ok 2024-09-20T22:23:05.013 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[29694]: 4.2 deep-scrub starts 2024-09-20T22:23:05.013 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[29694]: 4.2 deep-scrub ok 2024-09-20T22:23:05.013 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:04 smithi060 ceph-mon[29694]: pgmap v1105: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 67 KiB/s rd, 0 B/s wr, 110 op/s 2024-09-20T22:23:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:04 smithi067 ceph-mon[33511]: 6.8 scrub starts 2024-09-20T22:23:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:04 smithi067 ceph-mon[33511]: 6.8 scrub ok 2024-09-20T22:23:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:04 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2999022178' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 7, "weight": 1.0}]': finished 2024-09-20T22:23:05.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:04 smithi067 ceph-mon[33511]: osdmap e583: 8 total, 8 up, 8 in 2024-09-20T22:23:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:04 smithi067 ceph-mon[33511]: 6.2 scrub starts 2024-09-20T22:23:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:04 smithi067 ceph-mon[33511]: 6.2 scrub ok 2024-09-20T22:23:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:04 smithi067 ceph-mon[33511]: 1.12 scrub starts 2024-09-20T22:23:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:04 smithi067 ceph-mon[33511]: 1.12 scrub ok 2024-09-20T22:23:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:04 smithi067 ceph-mon[33511]: 6.11 scrub starts 2024-09-20T22:23:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:04 smithi067 ceph-mon[33511]: 6.11 scrub ok 2024-09-20T22:23:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:04 smithi067 ceph-mon[33511]: 4.b scrub starts 2024-09-20T22:23:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:04 smithi067 ceph-mon[33511]: 4.b scrub ok 2024-09-20T22:23:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:04 smithi067 ceph-mon[33511]: 4.2 deep-scrub starts 2024-09-20T22:23:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:04 smithi067 ceph-mon[33511]: 4.2 deep-scrub ok 2024-09-20T22:23:05.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:04 smithi067 ceph-mon[33511]: pgmap v1105: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 67 KiB/s rd, 0 B/s wr, 110 op/s 2024-09-20T22:23:05.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:05 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:23:05] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:23:05.744 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.1 to 1 (10000) 2024-09-20T22:23:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:05 smithi067 ceph-mon[33511]: 6.1e scrub starts 2024-09-20T22:23:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:05 smithi067 ceph-mon[33511]: 6.1e scrub ok 2024-09-20T22:23:06.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:05 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1340030607' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:23:06.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:05 smithi060 ceph-mon[29694]: 6.1e scrub starts 2024-09-20T22:23:06.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:05 smithi060 ceph-mon[29694]: 6.1e scrub ok 2024-09-20T22:23:06.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:05 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1340030607' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:23:06.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:05 smithi060 ceph-mon[40290]: 6.1e scrub starts 2024-09-20T22:23:06.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:05 smithi060 ceph-mon[40290]: 6.1e scrub ok 2024-09-20T22:23:06.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:05 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1340030607' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:23:06.400 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 1 1 2024-09-20T22:23:06.664 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:07.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:06 smithi067 ceph-mon[33511]: 1.5 scrub starts 2024-09-20T22:23:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:06 smithi067 ceph-mon[33511]: 1.5 scrub ok 2024-09-20T22:23:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:06 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1340030607' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T22:23:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:06 smithi067 ceph-mon[33511]: osdmap e584: 8 total, 8 up, 8 in 2024-09-20T22:23:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:06 smithi067 ceph-mon[33511]: 6.a scrub starts 2024-09-20T22:23:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:06 smithi067 ceph-mon[33511]: 6.a scrub ok 2024-09-20T22:23:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:06 smithi067 ceph-mon[33511]: 1.b scrub starts 2024-09-20T22:23:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:06 smithi067 ceph-mon[33511]: 1.b scrub ok 2024-09-20T22:23:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:06 smithi067 ceph-mon[33511]: pgmap v1107: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 59 op/s 2024-09-20T22:23:07.014 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:06 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:23:07.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[29694]: 1.5 scrub starts 2024-09-20T22:23:07.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[29694]: 1.5 scrub ok 2024-09-20T22:23:07.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1340030607' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T22:23:07.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[29694]: osdmap e584: 8 total, 8 up, 8 in 2024-09-20T22:23:07.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[29694]: 6.a scrub starts 2024-09-20T22:23:07.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[29694]: 6.a scrub ok 2024-09-20T22:23:07.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[29694]: 1.b scrub starts 2024-09-20T22:23:07.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[29694]: 1.b scrub ok 2024-09-20T22:23:07.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[29694]: pgmap v1107: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 59 op/s 2024-09-20T22:23:07.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:23:07.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[40290]: 1.5 scrub starts 2024-09-20T22:23:07.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[40290]: 1.5 scrub ok 2024-09-20T22:23:07.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1340030607' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 1, "weight": 1.0}]': finished 2024-09-20T22:23:07.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[40290]: osdmap e584: 8 total, 8 up, 8 in 2024-09-20T22:23:07.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[40290]: 6.a scrub starts 2024-09-20T22:23:07.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[40290]: 6.a scrub ok 2024-09-20T22:23:07.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[40290]: 1.b scrub starts 2024-09-20T22:23:07.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[40290]: 1.b scrub ok 2024-09-20T22:23:07.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[40290]: pgmap v1107: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 59 op/s 2024-09-20T22:23:07.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:06 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:23:07.758 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-09-20T22:23:08.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:07 smithi060 ceph-mon[29694]: 6.7 scrub starts 2024-09-20T22:23:08.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:07 smithi060 ceph-mon[29694]: 6.7 scrub ok 2024-09-20T22:23:08.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:07 smithi060 ceph-mon[29694]: 4.a scrub starts 2024-09-20T22:23:08.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:07 smithi060 ceph-mon[29694]: 4.a scrub ok 2024-09-20T22:23:08.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:07 smithi060 ceph-mon[29694]: 1.16 scrub starts 2024-09-20T22:23:08.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:07 smithi060 ceph-mon[29694]: 1.16 scrub ok 2024-09-20T22:23:08.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:07 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3406958795' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:23:08.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:07 smithi060 ceph-mon[40290]: 6.7 scrub starts 2024-09-20T22:23:08.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:07 smithi060 ceph-mon[40290]: 6.7 scrub ok 2024-09-20T22:23:08.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:07 smithi060 ceph-mon[40290]: 4.a scrub starts 2024-09-20T22:23:08.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:07 smithi060 ceph-mon[40290]: 4.a scrub ok 2024-09-20T22:23:08.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:07 smithi060 ceph-mon[40290]: 1.16 scrub starts 2024-09-20T22:23:08.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:07 smithi060 ceph-mon[40290]: 1.16 scrub ok 2024-09-20T22:23:08.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:07 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3406958795' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:23:08.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:07 smithi067 ceph-mon[33511]: 6.7 scrub starts 2024-09-20T22:23:08.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:07 smithi067 ceph-mon[33511]: 6.7 scrub ok 2024-09-20T22:23:08.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:07 smithi067 ceph-mon[33511]: 4.a scrub starts 2024-09-20T22:23:08.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:07 smithi067 ceph-mon[33511]: 4.a scrub ok 2024-09-20T22:23:08.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:07 smithi067 ceph-mon[33511]: 1.16 scrub starts 2024-09-20T22:23:08.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:07 smithi067 ceph-mon[33511]: 1.16 scrub ok 2024-09-20T22:23:08.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:07 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3406958795' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]: dispatch 2024-09-20T22:23:08.518 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd reweight 4 1 2024-09-20T22:23:08.791 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:09.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[40290]: 3.14 scrub starts 2024-09-20T22:23:09.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[40290]: 3.14 scrub ok 2024-09-20T22:23:09.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/3406958795' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-20T22:23:09.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[40290]: osdmap e585: 8 total, 8 up, 8 in 2024-09-20T22:23:09.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[40290]: 3.5 scrub starts 2024-09-20T22:23:09.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[40290]: 3.5 scrub ok 2024-09-20T22:23:09.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[40290]: 4.10 scrub starts 2024-09-20T22:23:09.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[40290]: 4.10 scrub ok 2024-09-20T22:23:09.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[40290]: 4.7 scrub starts 2024-09-20T22:23:09.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[40290]: 4.7 scrub ok 2024-09-20T22:23:09.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[40290]: pgmap v1109: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 12 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-20T22:23:09.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[29694]: 3.14 scrub starts 2024-09-20T22:23:09.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[29694]: 3.14 scrub ok 2024-09-20T22:23:09.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/3406958795' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-20T22:23:09.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[29694]: osdmap e585: 8 total, 8 up, 8 in 2024-09-20T22:23:09.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[29694]: 3.5 scrub starts 2024-09-20T22:23:09.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[29694]: 3.5 scrub ok 2024-09-20T22:23:09.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[29694]: 4.10 scrub starts 2024-09-20T22:23:09.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[29694]: 4.10 scrub ok 2024-09-20T22:23:09.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[29694]: 4.7 scrub starts 2024-09-20T22:23:09.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[29694]: 4.7 scrub ok 2024-09-20T22:23:09.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:08 smithi060 ceph-mon[29694]: pgmap v1109: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 12 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-20T22:23:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:08 smithi067 ceph-mon[33511]: 3.14 scrub starts 2024-09-20T22:23:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:08 smithi067 ceph-mon[33511]: 3.14 scrub ok 2024-09-20T22:23:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:08 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/3406958795' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 1, "weight": 1.0}]': finished 2024-09-20T22:23:09.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:08 smithi067 ceph-mon[33511]: osdmap e585: 8 total, 8 up, 8 in 2024-09-20T22:23:09.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:08 smithi067 ceph-mon[33511]: 3.5 scrub starts 2024-09-20T22:23:09.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:08 smithi067 ceph-mon[33511]: 3.5 scrub ok 2024-09-20T22:23:09.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:08 smithi067 ceph-mon[33511]: 4.10 scrub starts 2024-09-20T22:23:09.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:08 smithi067 ceph-mon[33511]: 4.10 scrub ok 2024-09-20T22:23:09.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:08 smithi067 ceph-mon[33511]: 4.7 scrub starts 2024-09-20T22:23:09.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:08 smithi067 ceph-mon[33511]: 4.7 scrub ok 2024-09-20T22:23:09.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:08 smithi067 ceph-mon[33511]: pgmap v1109: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 12 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-20T22:23:09.773 INFO:teuthology.orchestra.run.smithi060.stderr:reweighted osd.4 to 1 (10000) 2024-09-20T22:23:10.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:09 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:23:10.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:09 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:23:10.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:09 smithi060 ceph-mon[29694]: osdmap e586: 8 total, 8 up, 8 in 2024-09-20T22:23:10.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:09 smithi060 ceph-mon[29694]: 4.1f scrub starts 2024-09-20T22:23:10.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:09 smithi060 ceph-mon[29694]: 4.1f scrub ok 2024-09-20T22:23:10.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:09 smithi060 ceph-mon[29694]: 6.16 scrub starts 2024-09-20T22:23:10.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:09 smithi060 ceph-mon[29694]: 6.16 scrub ok 2024-09-20T22:23:10.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:09 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2078338657' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:23:10.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:09 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:23:10.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:09 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:23:10.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:09 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:23:10.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:09 smithi060 ceph-mon[40290]: osdmap e586: 8 total, 8 up, 8 in 2024-09-20T22:23:10.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:09 smithi060 ceph-mon[40290]: 4.1f scrub starts 2024-09-20T22:23:10.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:09 smithi060 ceph-mon[40290]: 4.1f scrub ok 2024-09-20T22:23:10.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:09 smithi060 ceph-mon[40290]: 6.16 scrub starts 2024-09-20T22:23:10.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:09 smithi060 ceph-mon[40290]: 6.16 scrub ok 2024-09-20T22:23:10.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:09 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2078338657' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:23:10.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:09 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:23:10.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:09 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:23:10.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:09 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:23:10.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:09 smithi067 ceph-mon[33511]: osdmap e586: 8 total, 8 up, 8 in 2024-09-20T22:23:10.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:09 smithi067 ceph-mon[33511]: 4.1f scrub starts 2024-09-20T22:23:10.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:09 smithi067 ceph-mon[33511]: 4.1f scrub ok 2024-09-20T22:23:10.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:09 smithi067 ceph-mon[33511]: 6.16 scrub starts 2024-09-20T22:23:10.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:09 smithi067 ceph-mon[33511]: 6.16 scrub ok 2024-09-20T22:23:10.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:09 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2078338657' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:23:10.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:09 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:23:10.458 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd primary-affinity 4 1 2024-09-20T22:23:10.723 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:11.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:10 smithi060 ceph-mon[29694]: 3.e scrub starts 2024-09-20T22:23:11.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:10 smithi060 ceph-mon[29694]: 3.e scrub ok 2024-09-20T22:23:11.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:10 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T22:23:11.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:10 smithi060 ceph-mon[29694]: osdmap e587: 8 total, 8 up, 8 in 2024-09-20T22:23:11.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:10 smithi060 ceph-mon[29694]: pgmap v1112: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T22:23:11.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:10 smithi060 ceph-mon[29694]: osdmap e588: 8 total, 8 up, 8 in 2024-09-20T22:23:11.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:10 smithi060 ceph-mon[40290]: 3.e scrub starts 2024-09-20T22:23:11.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:10 smithi060 ceph-mon[40290]: 3.e scrub ok 2024-09-20T22:23:11.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:10 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T22:23:11.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:10 smithi060 ceph-mon[40290]: osdmap e587: 8 total, 8 up, 8 in 2024-09-20T22:23:11.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:10 smithi060 ceph-mon[40290]: pgmap v1112: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T22:23:11.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:10 smithi060 ceph-mon[40290]: osdmap e588: 8 total, 8 up, 8 in 2024-09-20T22:23:11.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:10 smithi067 ceph-mon[33511]: 3.e scrub starts 2024-09-20T22:23:11.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:10 smithi067 ceph-mon[33511]: 3.e scrub ok 2024-09-20T22:23:11.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:10 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd reweight", "id": 4, "weight": 1.0}]': finished 2024-09-20T22:23:11.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:10 smithi067 ceph-mon[33511]: osdmap e587: 8 total, 8 up, 8 in 2024-09-20T22:23:11.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:10 smithi067 ceph-mon[33511]: pgmap v1112: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-20T22:23:11.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:10 smithi067 ceph-mon[33511]: osdmap e588: 8 total, 8 up, 8 in 2024-09-20T22:23:11.786 INFO:teuthology.orchestra.run.smithi060.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-09-20T22:23:12.086 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:11 smithi060 ceph-mon[29694]: 2.2 scrub starts 2024-09-20T22:23:12.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:11 smithi060 ceph-mon[29694]: 2.2 scrub ok 2024-09-20T22:23:12.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:11 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2392399342' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:23:12.087 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:11 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:23:12.087 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:11 smithi060 ceph-mon[40290]: 2.2 scrub starts 2024-09-20T22:23:12.087 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:11 smithi060 ceph-mon[40290]: 2.2 scrub ok 2024-09-20T22:23:12.087 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:11 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2392399342' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:23:12.087 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:11 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:23:12.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:11 smithi067 ceph-mon[33511]: 2.2 scrub starts 2024-09-20T22:23:12.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:11 smithi067 ceph-mon[33511]: 2.2 scrub ok 2024-09-20T22:23:12.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:11 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2392399342' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:23:12.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:11 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]: dispatch 2024-09-20T22:23:12.544 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-09-20T22:23:12.544 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-09-20T22:23:12.544 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-09-20T22:23:12.544 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-09-20T22:23:12.544 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-20T22:23:12.544 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-20T22:23:12.806 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:13.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:12 smithi060 ceph-mon[29694]: 4.9 scrub starts 2024-09-20T22:23:13.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:12 smithi060 ceph-mon[29694]: 4.9 scrub ok 2024-09-20T22:23:13.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:12 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-20T22:23:13.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:12 smithi060 ceph-mon[29694]: osdmap e589: 8 total, 8 up, 8 in 2024-09-20T22:23:13.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:12 smithi060 ceph-mon[29694]: 4.12 scrub starts 2024-09-20T22:23:13.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:12 smithi060 ceph-mon[29694]: 4.12 scrub ok 2024-09-20T22:23:13.094 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:12 smithi060 ceph-mon[29694]: pgmap v1115: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:23:13.094 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:12 smithi060 ceph-mon[40290]: 4.9 scrub starts 2024-09-20T22:23:13.095 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:12 smithi060 ceph-mon[40290]: 4.9 scrub ok 2024-09-20T22:23:13.095 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:12 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-20T22:23:13.095 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:12 smithi060 ceph-mon[40290]: osdmap e589: 8 total, 8 up, 8 in 2024-09-20T22:23:13.095 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:12 smithi060 ceph-mon[40290]: 4.12 scrub starts 2024-09-20T22:23:13.095 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:12 smithi060 ceph-mon[40290]: 4.12 scrub ok 2024-09-20T22:23:13.095 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:12 smithi060 ceph-mon[40290]: pgmap v1115: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:23:13.165 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:23:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:23:13.165 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:12 smithi067 ceph-mon[33511]: 4.9 scrub starts 2024-09-20T22:23:13.165 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:12 smithi067 ceph-mon[33511]: 4.9 scrub ok 2024-09-20T22:23:13.165 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:12 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd primary-affinity", "id": 4, "weight": 1.0}]': finished 2024-09-20T22:23:13.165 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:12 smithi067 ceph-mon[33511]: osdmap e589: 8 total, 8 up, 8 in 2024-09-20T22:23:13.165 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:12 smithi067 ceph-mon[33511]: 4.12 scrub starts 2024-09-20T22:23:13.165 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:12 smithi067 ceph-mon[33511]: 4.12 scrub ok 2024-09-20T22:23:13.165 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:12 smithi067 ceph-mon[33511]: pgmap v1115: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:23:13.503 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:23:13.504 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":589,"fsid":"3df9647e-779b-11ef-baf5-efdc52797490","created":"2024-09-20T21:57:31.244934+0000","modified":"2024-09-20T22:23:11.779798+0000","last_up_change":"2024-09-20T22:21:43.718655+0000","last_in_change":"2024-09-20T22:22:35.715096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-20T22:00:35.131125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":31,"pg_placement_num":31,"pg_placement_num_target":31,"pg_num_target":31,"pg_num_pending":31,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"521","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"428","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.809999942779541,"score_stable":1.809999942779541,"optimal_score":1,"raw_score_acting":1.809999942779541,"raw_score_stable":1.809999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-20T22:02:47.974480+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-20T22:02:48.902465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-20T22:02:50.620057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-20T22:02:52.722912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-20T22:02:54.794248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0997c01a-a9bd-421e-abcc-fbcf7abe8512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":587,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6803","nonce":1395849550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6805","nonce":1395849550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6809","nonce":1395849550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6807","nonce":1395849550}]},"public_addr":"172.21.15.60:6803/1395849550","cluster_addr":"172.21.15.60:6805/1395849550","heartbeat_back_addr":"172.21.15.60:6809/1395849550","heartbeat_front_addr":"172.21.15.60:6807/1395849550","state":["exists","up"]},{"osd":1,"uuid":"c04bdcb1-d76b-47c8-a38b-c02043df6ba8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":90,"up_from":145,"up_thru":567,"down_at":91,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2970874921},{"type":"v1","addr":"172.21.15.60:6811","nonce":2970874921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2970874921},{"type":"v1","addr":"172.21.15.60:6813","nonce":2970874921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2970874921},{"type":"v1","addr":"172.21.15.60:6817","nonce":2970874921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2970874921},{"type":"v1","addr":"172.21.15.60:6815","nonce":2970874921}]},"public_addr":"172.21.15.60:6811/2970874921","cluster_addr":"172.21.15.60:6813/2970874921","heartbeat_back_addr":"172.21.15.60:6817/2970874921","heartbeat_front_addr":"172.21.15.60:6815/2970874921","state":["exists","up"]},{"osd":2,"uuid":"32c09d19-eb98-4926-a44e-e7ddee152956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":566,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6819","nonce":3698562274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6821","nonce":3698562274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6825","nonce":3698562274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6823","nonce":3698562274}]},"public_addr":"172.21.15.60:6819/3698562274","cluster_addr":"172.21.15.60:6821/3698562274","heartbeat_back_addr":"172.21.15.60:6825/3698562274","heartbeat_front_addr":"172.21.15.60:6823/3698562274","state":["exists","up"]},{"osd":3,"uuid":"b1c2cb69-8565-49ac-b854-7ea7615c6660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":574,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6827","nonce":2592322449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6829","nonce":2592322449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6833","nonce":2592322449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6831","nonce":2592322449}]},"public_addr":"172.21.15.60:6827/2592322449","cluster_addr":"172.21.15.60:6829/2592322449","heartbeat_back_addr":"172.21.15.60:6833/2592322449","heartbeat_front_addr":"172.21.15.60:6831/2592322449","state":["exists","up"]},{"osd":4,"uuid":"04a547d6-fc48-43b7-a8d7-18df8a83aa96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":31,"last_clean_end":485,"up_from":532,"up_thru":587,"down_at":486,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":3152108694},{"type":"v1","addr":"172.21.15.67:6801","nonce":3152108694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3152108694},{"type":"v1","addr":"172.21.15.67:6803","nonce":3152108694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3152108694},{"type":"v1","addr":"172.21.15.67:6807","nonce":3152108694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3152108694},{"type":"v1","addr":"172.21.15.67:6805","nonce":3152108694}]},"public_addr":"172.21.15.67:6801/3152108694","cluster_addr":"172.21.15.67:6803/3152108694","heartbeat_back_addr":"172.21.15.67:6807/3152108694","heartbeat_front_addr":"172.21.15.67:6805/3152108694","state":["exists","up"]},{"osd":5,"uuid":"4763aa10-c367-485b-b525-e5241e642b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":567,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6809","nonce":1622258887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6811","nonce":1622258887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6815","nonce":1622258887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6813","nonce":1622258887}]},"public_addr":"172.21.15.67:6809/1622258887","cluster_addr":"172.21.15.67:6811/1622258887","heartbeat_back_addr":"172.21.15.67:6815/1622258887","heartbeat_front_addr":"172.21.15.67:6813/1622258887","state":["exists","up"]},{"osd":6,"uuid":"a359116e-c013-4e2f-8b79-86219516ee16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":566,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6817","nonce":1187114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6819","nonce":1187114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6823","nonce":1187114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6821","nonce":1187114}]},"public_addr":"172.21.15.67:6817/1187114","cluster_addr":"172.21.15.67:6819/1187114","heartbeat_back_addr":"172.21.15.67:6823/1187114","heartbeat_front_addr":"172.21.15.67:6821/1187114","state":["exists","up"]},{"osd":7,"uuid":"a4bf4188-f133-4a31-8897-d7e8182248d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":567,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6825","nonce":3021842840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6827","nonce":3021842840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6831","nonce":3021842840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6829","nonce":3021842840}]},"public_addr":"172.21.15.67:6825/3021842840","cluster_addr":"172.21.15.67:6827/3021842840","heartbeat_back_addr":"172.21.15.67:6831/3021842840","heartbeat_front_addr":"172.21.15.67:6829/3021842840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:22:06.567255+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:23:00.946230+0000","dead_epoch":90},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:22:24.656555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:22:23.316008+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:22:58.194233+0000","dead_epoch":485},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:22:21.349067+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:23:08.524548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:22:50.932549+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"1.8","osds":[6,7,2]},{"pgid":"3.c","osds":[3,1,6]},{"pgid":"6.13","osds":[0,7,6]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/890154710":"2024-09-21T22:03:19.518405+0000","172.21.15.60:0/3386526350":"2024-09-21T22:03:19.518405+0000","172.21.15.60:6801/604714621":"2024-09-21T22:03:19.518405+0000","172.21.15.60:6800/604714621":"2024-09-21T22:03:19.518405+0000","172.21.15.60:6800/3151907776":"2024-09-21T21:58:02.105557+0000","172.21.15.60:0/956264518":"2024-09-21T21:58:02.105557+0000","172.21.15.60:0/4150585713":"2024-09-21T21:58:02.105557+0000","172.21.15.60:0/3311255314":"2024-09-21T22:03:19.518405+0000","172.21.15.60:0/3965541528":"2024-09-21T21:58:02.105557+0000","172.21.15.60:6800/3734574621":"2024-09-21T21:58:26.613414+0000","172.21.15.60:0/1729862661":"2024-09-21T21:58:26.613414+0000","172.21.15.60:6801/3734574621":"2024-09-21T21:58:26.613414+0000","172.21.15.60:0/3927294026":"2024-09-21T21:58:26.613414+0000","172.21.15.60:6801/3151907776":"2024-09-21T21:58:02.105557+0000","172.21.15.60:0/1871351181":"2024-09-21T21:58:26.613414+0000","172.21.15.60:0/1854455098":"2024-09-21T22:03:19.518405+0000","172.21.15.60:0/2221262203":"2024-09-21T22:03:19.518405+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-20T22:23:14.188 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:13 smithi067 ceph-mon[33511]: 5.9 scrub starts 2024-09-20T22:23:14.188 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:13 smithi067 ceph-mon[33511]: 5.9 scrub ok 2024-09-20T22:23:14.188 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:13 smithi067 ceph-mon[33511]: 4.1b scrub starts 2024-09-20T22:23:14.189 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:13 smithi067 ceph-mon[33511]: 4.1b scrub ok 2024-09-20T22:23:14.189 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:13 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:23:14.189 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:13 smithi067 ceph-mon[33511]: 5.15 scrub starts 2024-09-20T22:23:14.189 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:13 smithi067 ceph-mon[33511]: 5.15 scrub ok 2024-09-20T22:23:14.189 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:13 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/612000069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-20T22:23:14.239 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:13 smithi060 ceph-mon[29694]: 5.9 scrub starts 2024-09-20T22:23:14.239 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:13 smithi060 ceph-mon[29694]: 5.9 scrub ok 2024-09-20T22:23:14.239 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:13 smithi060 ceph-mon[29694]: 4.1b scrub starts 2024-09-20T22:23:14.239 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:13 smithi060 ceph-mon[29694]: 4.1b scrub ok 2024-09-20T22:23:14.239 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:13 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:23:14.240 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:13 smithi060 ceph-mon[29694]: 5.15 scrub starts 2024-09-20T22:23:14.240 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:13 smithi060 ceph-mon[29694]: 5.15 scrub ok 2024-09-20T22:23:14.240 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:13 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/612000069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-20T22:23:14.240 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:13 smithi060 ceph-mon[40290]: 5.9 scrub starts 2024-09-20T22:23:14.240 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:13 smithi060 ceph-mon[40290]: 5.9 scrub ok 2024-09-20T22:23:14.240 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:13 smithi060 ceph-mon[40290]: 4.1b scrub starts 2024-09-20T22:23:14.240 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:13 smithi060 ceph-mon[40290]: 4.1b scrub ok 2024-09-20T22:23:14.240 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:13 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:23:14.240 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:13 smithi060 ceph-mon[40290]: 5.15 scrub starts 2024-09-20T22:23:14.240 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:13 smithi060 ceph-mon[40290]: 5.15 scrub ok 2024-09-20T22:23:14.240 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:13 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/612000069' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-20T22:23:14.273 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-20T22:23:14.274 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-20T22:23:14.537 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:14.935 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:14 smithi060 ceph-mon[29694]: 3.3 deep-scrub starts 2024-09-20T22:23:14.935 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:14 smithi060 ceph-mon[29694]: 3.3 deep-scrub ok 2024-09-20T22:23:14.935 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:14 smithi060 ceph-mon[29694]: pgmap v1116: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 724 B/s rd, 0 op/s; 19 B/s, 0 objects/s recovering 2024-09-20T22:23:14.935 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:14 smithi060 ceph-mon[40290]: 3.3 deep-scrub starts 2024-09-20T22:23:14.935 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:14 smithi060 ceph-mon[40290]: 3.3 deep-scrub ok 2024-09-20T22:23:14.935 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:14 smithi060 ceph-mon[40290]: pgmap v1116: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 724 B/s rd, 0 op/s; 19 B/s, 0 objects/s recovering 2024-09-20T22:23:15.207 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:23:15.207 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:15 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:23:15] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:23:15.207 INFO:teuthology.orchestra.run.smithi060.stdout:{"epoch":589,"fsid":"3df9647e-779b-11ef-baf5-efdc52797490","created":"2024-09-20T21:57:31.244934+0000","modified":"2024-09-20T22:23:11.779798+0000","last_up_change":"2024-09-20T22:21:43.718655+0000","last_in_change":"2024-09-20T22:22:35.715096+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-20T22:00:35.131125+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":31,"pg_placement_num":31,"pg_placement_num_target":31,"pg_num_target":31,"pg_num_pending":31,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"521","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"428","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.809999942779541,"score_stable":1.809999942779541,"optimal_score":1,"raw_score_acting":1.809999942779541,"raw_score_stable":1.809999942779541,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"datapool","create_time":"2024-09-20T22:02:47.974480+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-09-20T22:02:48.902465+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-20T22:02:50.620057+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-20T22:02:52.722912+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-20T22:02:54.794248+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0997c01a-a9bd-421e-abcc-fbcf7abe8512","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":587,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6802","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6803","nonce":1395849550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6804","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6805","nonce":1395849550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6808","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6809","nonce":1395849550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6806","nonce":1395849550},{"type":"v1","addr":"172.21.15.60:6807","nonce":1395849550}]},"public_addr":"172.21.15.60:6803/1395849550","cluster_addr":"172.21.15.60:6805/1395849550","heartbeat_back_addr":"172.21.15.60:6809/1395849550","heartbeat_front_addr":"172.21.15.60:6807/1395849550","state":["exists","up"]},{"osd":1,"uuid":"c04bdcb1-d76b-47c8-a38b-c02043df6ba8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":90,"up_from":145,"up_thru":567,"down_at":91,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6810","nonce":2970874921},{"type":"v1","addr":"172.21.15.60:6811","nonce":2970874921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6812","nonce":2970874921},{"type":"v1","addr":"172.21.15.60:6813","nonce":2970874921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6816","nonce":2970874921},{"type":"v1","addr":"172.21.15.60:6817","nonce":2970874921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6814","nonce":2970874921},{"type":"v1","addr":"172.21.15.60:6815","nonce":2970874921}]},"public_addr":"172.21.15.60:6811/2970874921","cluster_addr":"172.21.15.60:6813/2970874921","heartbeat_back_addr":"172.21.15.60:6817/2970874921","heartbeat_front_addr":"172.21.15.60:6815/2970874921","state":["exists","up"]},{"osd":2,"uuid":"32c09d19-eb98-4926-a44e-e7ddee152956","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":566,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6818","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6819","nonce":3698562274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6820","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6821","nonce":3698562274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6824","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6825","nonce":3698562274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6822","nonce":3698562274},{"type":"v1","addr":"172.21.15.60:6823","nonce":3698562274}]},"public_addr":"172.21.15.60:6819/3698562274","cluster_addr":"172.21.15.60:6821/3698562274","heartbeat_back_addr":"172.21.15.60:6825/3698562274","heartbeat_front_addr":"172.21.15.60:6823/3698562274","state":["exists","up"]},{"osd":3,"uuid":"b1c2cb69-8565-49ac-b854-7ea7615c6660","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":574,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6826","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6827","nonce":2592322449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6828","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6829","nonce":2592322449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6832","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6833","nonce":2592322449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.60:6830","nonce":2592322449},{"type":"v1","addr":"172.21.15.60:6831","nonce":2592322449}]},"public_addr":"172.21.15.60:6827/2592322449","cluster_addr":"172.21.15.60:6829/2592322449","heartbeat_back_addr":"172.21.15.60:6833/2592322449","heartbeat_front_addr":"172.21.15.60:6831/2592322449","state":["exists","up"]},{"osd":4,"uuid":"04a547d6-fc48-43b7-a8d7-18df8a83aa96","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":31,"last_clean_end":485,"up_from":532,"up_thru":587,"down_at":486,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6800","nonce":3152108694},{"type":"v1","addr":"172.21.15.67:6801","nonce":3152108694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6802","nonce":3152108694},{"type":"v1","addr":"172.21.15.67:6803","nonce":3152108694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6806","nonce":3152108694},{"type":"v1","addr":"172.21.15.67:6807","nonce":3152108694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6804","nonce":3152108694},{"type":"v1","addr":"172.21.15.67:6805","nonce":3152108694}]},"public_addr":"172.21.15.67:6801/3152108694","cluster_addr":"172.21.15.67:6803/3152108694","heartbeat_back_addr":"172.21.15.67:6807/3152108694","heartbeat_front_addr":"172.21.15.67:6805/3152108694","state":["exists","up"]},{"osd":5,"uuid":"4763aa10-c367-485b-b525-e5241e642b86","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":567,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6808","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6809","nonce":1622258887}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6810","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6811","nonce":1622258887}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6814","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6815","nonce":1622258887}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6812","nonce":1622258887},{"type":"v1","addr":"172.21.15.67:6813","nonce":1622258887}]},"public_addr":"172.21.15.67:6809/1622258887","cluster_addr":"172.21.15.67:6811/1622258887","heartbeat_back_addr":"172.21.15.67:6815/1622258887","heartbeat_front_addr":"172.21.15.67:6813/1622258887","state":["exists","up"]},{"osd":6,"uuid":"a359116e-c013-4e2f-8b79-86219516ee16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":566,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6816","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6817","nonce":1187114}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6818","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6819","nonce":1187114}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6822","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6823","nonce":1187114}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6820","nonce":1187114},{"type":"v1","addr":"172.21.15.67:6821","nonce":1187114}]},"public_addr":"172.21.15.67:6817/1187114","cluster_addr":"172.21.15.67:6819/1187114","heartbeat_back_addr":"172.21.15.67:6823/1187114","heartbeat_front_addr":"172.21.15.67:6821/1187114","state":["exists","up"]},{"osd":7,"uuid":"a4bf4188-f133-4a31-8897-d7e8182248d7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":567,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6824","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6825","nonce":3021842840}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6826","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6827","nonce":3021842840}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6830","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6831","nonce":3021842840}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.67:6828","nonce":3021842840},{"type":"v1","addr":"172.21.15.67:6829","nonce":3021842840}]},"public_addr":"172.21.15.67:6825/3021842840","cluster_addr":"172.21.15.67:6827/3021842840","heartbeat_back_addr":"172.21.15.67:6831/3021842840","heartbeat_front_addr":"172.21.15.67:6829/3021842840","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:22:06.567255+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:23:00.946230+0000","dead_epoch":90},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:22:24.656555+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:22:23.316008+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:22:58.194233+0000","dead_epoch":485},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:22:21.349067+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:23:08.524548+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-20T22:22:50.932549+0000","dead_epoch":0}],"pg_upmap":[{"pgid":"1.8","osds":[6,7,2]},{"pgid":"3.c","osds":[3,1,6]},{"pgid":"6.13","osds":[0,7,6]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.60:0/890154710":"2024-09-21T22:03:19.518405+0000","172.21.15.60:0/3386526350":"2024-09-21T22:03:19.518405+0000","172.21.15.60:6801/604714621":"2024-09-21T22:03:19.518405+0000","172.21.15.60:6800/604714621":"2024-09-21T22:03:19.518405+0000","172.21.15.60:6800/3151907776":"2024-09-21T21:58:02.105557+0000","172.21.15.60:0/956264518":"2024-09-21T21:58:02.105557+0000","172.21.15.60:0/4150585713":"2024-09-21T21:58:02.105557+0000","172.21.15.60:0/3311255314":"2024-09-21T22:03:19.518405+0000","172.21.15.60:0/3965541528":"2024-09-21T21:58:02.105557+0000","172.21.15.60:6800/3734574621":"2024-09-21T21:58:26.613414+0000","172.21.15.60:0/1729862661":"2024-09-21T21:58:26.613414+0000","172.21.15.60:6801/3734574621":"2024-09-21T21:58:26.613414+0000","172.21.15.60:0/3927294026":"2024-09-21T21:58:26.613414+0000","172.21.15.60:6801/3151907776":"2024-09-21T21:58:02.105557+0000","172.21.15.60:0/1871351181":"2024-09-21T21:58:26.613414+0000","172.21.15.60:0/1854455098":"2024-09-21T22:03:19.518405+0000","172.21.15.60:0/2221262203":"2024-09-21T22:03:19.518405+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-20T22:23:15.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:14 smithi067 ceph-mon[33511]: 3.3 deep-scrub starts 2024-09-20T22:23:15.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:14 smithi067 ceph-mon[33511]: 3.3 deep-scrub ok 2024-09-20T22:23:15.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:14 smithi067 ceph-mon[33511]: pgmap v1116: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 724 B/s rd, 0 op/s; 19 B/s, 0 objects/s recovering 2024-09-20T22:23:15.853 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-20T22:23:15.853 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-20T22:23:15.854 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-20T22:23:15.854 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-20T22:23:15.855 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-20T22:23:15.855 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-20T22:23:15.855 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-20T22:23:15.856 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-20T22:23:16.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:15 smithi067 ceph-mon[33511]: 3.18 scrub starts 2024-09-20T22:23:16.319 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:15 smithi067 ceph-mon[33511]: 3.18 scrub ok 2024-09-20T22:23:16.320 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:15 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1724730700' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-20T22:23:16.320 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:15 smithi060 ceph-mon[29694]: 3.18 scrub starts 2024-09-20T22:23:16.320 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:15 smithi060 ceph-mon[29694]: 3.18 scrub ok 2024-09-20T22:23:16.320 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:15 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1724730700' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-20T22:23:16.320 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:15 smithi060 ceph-mon[40290]: 3.18 scrub starts 2024-09-20T22:23:16.320 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:15 smithi060 ceph-mon[40290]: 3.18 scrub ok 2024-09-20T22:23:16.320 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:15 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1724730700' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-20T22:23:16.413 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:16.415 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:16.419 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:16.419 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:16.420 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:16.420 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:16.429 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:16.429 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:16.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:16 smithi060 ceph-mon[40290]: 4.c scrub starts 2024-09-20T22:23:16.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:16 smithi060 ceph-mon[40290]: 4.c scrub ok 2024-09-20T22:23:16.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:16 smithi060 ceph-mon[40290]: 1.c scrub starts 2024-09-20T22:23:16.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:16 smithi060 ceph-mon[40290]: 1.c scrub ok 2024-09-20T22:23:16.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:16 smithi060 ceph-mon[40290]: pgmap v1117: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 16 B/s, 0 objects/s recovering 2024-09-20T22:23:16.919 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:16 smithi060 ceph-mon[40290]: osdmap e590: 8 total, 8 up, 8 in 2024-09-20T22:23:16.920 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:16 smithi060 ceph-mon[29694]: 4.c scrub starts 2024-09-20T22:23:16.920 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:16 smithi060 ceph-mon[29694]: 4.c scrub ok 2024-09-20T22:23:16.920 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:16 smithi060 ceph-mon[29694]: 1.c scrub starts 2024-09-20T22:23:16.920 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:16 smithi060 ceph-mon[29694]: 1.c scrub ok 2024-09-20T22:23:16.920 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:16 smithi060 ceph-mon[29694]: pgmap v1117: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 16 B/s, 0 objects/s recovering 2024-09-20T22:23:16.920 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:16 smithi060 ceph-mon[29694]: osdmap e590: 8 total, 8 up, 8 in 2024-09-20T22:23:17.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:16 smithi067 ceph-mon[33511]: 4.c scrub starts 2024-09-20T22:23:17.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:16 smithi067 ceph-mon[33511]: 4.c scrub ok 2024-09-20T22:23:17.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:16 smithi067 ceph-mon[33511]: 1.c scrub starts 2024-09-20T22:23:17.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:16 smithi067 ceph-mon[33511]: 1.c scrub ok 2024-09-20T22:23:17.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:16 smithi067 ceph-mon[33511]: pgmap v1117: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s; 16 B/s, 0 objects/s recovering 2024-09-20T22:23:17.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:16 smithi067 ceph-mon[33511]: osdmap e590: 8 total, 8 up, 8 in 2024-09-20T22:23:18.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:17 smithi060 ceph-mon[29694]: 4.17 scrub starts 2024-09-20T22:23:18.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:17 smithi060 ceph-mon[29694]: 4.17 scrub ok 2024-09-20T22:23:18.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:17 smithi060 ceph-mon[29694]: 4.4 scrub starts 2024-09-20T22:23:18.113 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:17 smithi060 ceph-mon[29694]: 4.4 scrub ok 2024-09-20T22:23:18.113 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:17 smithi060 ceph-mon[40290]: 4.17 scrub starts 2024-09-20T22:23:18.114 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:17 smithi060 ceph-mon[40290]: 4.17 scrub ok 2024-09-20T22:23:18.114 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:17 smithi060 ceph-mon[40290]: 4.4 scrub starts 2024-09-20T22:23:18.114 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:17 smithi060 ceph-mon[40290]: 4.4 scrub ok 2024-09-20T22:23:18.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:17 smithi067 ceph-mon[33511]: 4.17 scrub starts 2024-09-20T22:23:18.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:17 smithi067 ceph-mon[33511]: 4.17 scrub ok 2024-09-20T22:23:18.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:17 smithi067 ceph-mon[33511]: 4.4 scrub starts 2024-09-20T22:23:18.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:17 smithi067 ceph-mon[33511]: 4.4 scrub ok 2024-09-20T22:23:18.559 INFO:teuthology.orchestra.run.smithi060.stdout:111669149967 2024-09-20T22:23:18.559 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-20T22:23:19.078 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:18 smithi060 ceph-mon[29694]: pgmap v1119: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 14 B/s, 0 objects/s recovering 2024-09-20T22:23:19.079 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:18 smithi060 ceph-mon[40290]: pgmap v1119: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 14 B/s, 0 objects/s recovering 2024-09-20T22:23:19.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:18 smithi067 ceph-mon[33511]: pgmap v1119: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 14 B/s, 0 objects/s recovering 2024-09-20T22:23:19.925 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:19 smithi060 ceph-mon[40290]: 3.13 scrub starts 2024-09-20T22:23:19.925 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:19 smithi060 ceph-mon[40290]: 3.13 scrub ok 2024-09-20T22:23:19.926 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:19 smithi060 ceph-mon[29694]: 3.13 scrub starts 2024-09-20T22:23:19.926 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:19 smithi060 ceph-mon[29694]: 3.13 scrub ok 2024-09-20T22:23:19.966 INFO:teuthology.orchestra.run.smithi060.stdout:77309411603 2024-09-20T22:23:19.966 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-20T22:23:20.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:19 smithi067 ceph-mon[33511]: 3.13 scrub starts 2024-09-20T22:23:20.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:19 smithi067 ceph-mon[33511]: 3.13 scrub ok 2024-09-20T22:23:20.944 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:20 smithi060 ceph-mon[29694]: 5.1a scrub starts 2024-09-20T22:23:20.944 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:20 smithi060 ceph-mon[29694]: 5.1a scrub ok 2024-09-20T22:23:20.944 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:20 smithi060 ceph-mon[29694]: pgmap v1120: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 473 B/s rd, 0 op/s; 12 B/s, 0 objects/s recovering 2024-09-20T22:23:20.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:20 smithi060 ceph-mon[40290]: 5.1a scrub starts 2024-09-20T22:23:20.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:20 smithi060 ceph-mon[40290]: 5.1a scrub ok 2024-09-20T22:23:20.944 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:20 smithi060 ceph-mon[40290]: pgmap v1120: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 473 B/s rd, 0 op/s; 12 B/s, 0 objects/s recovering 2024-09-20T22:23:21.250 INFO:teuthology.orchestra.run.smithi060.stdout:2284922601493 2024-09-20T22:23:21.250 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-20T22:23:21.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:20 smithi067 ceph-mon[33511]: 5.1a scrub starts 2024-09-20T22:23:21.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:20 smithi067 ceph-mon[33511]: 5.1a scrub ok 2024-09-20T22:23:21.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:20 smithi067 ceph-mon[33511]: pgmap v1120: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 473 B/s rd, 0 op/s; 12 B/s, 0 objects/s recovering 2024-09-20T22:23:22.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:21 smithi060 ceph-mon[29694]: 3.f scrub starts 2024-09-20T22:23:22.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:21 smithi060 ceph-mon[29694]: 3.f scrub ok 2024-09-20T22:23:22.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:21 smithi060 ceph-mon[29694]: 6.19 scrub starts 2024-09-20T22:23:22.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:21 smithi060 ceph-mon[29694]: 6.19 scrub ok 2024-09-20T22:23:22.112 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:21 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:23:22.113 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:21 smithi060 ceph-mon[40290]: 3.f scrub starts 2024-09-20T22:23:22.113 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:21 smithi060 ceph-mon[40290]: 3.f scrub ok 2024-09-20T22:23:22.113 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:21 smithi060 ceph-mon[40290]: 6.19 scrub starts 2024-09-20T22:23:22.113 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:21 smithi060 ceph-mon[40290]: 6.19 scrub ok 2024-09-20T22:23:22.113 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:21 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:23:22.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:21 smithi067 ceph-mon[33511]: 3.f scrub starts 2024-09-20T22:23:22.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:21 smithi067 ceph-mon[33511]: 3.f scrub ok 2024-09-20T22:23:22.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:21 smithi067 ceph-mon[33511]: 6.19 scrub starts 2024-09-20T22:23:22.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:21 smithi067 ceph-mon[33511]: 6.19 scrub ok 2024-09-20T22:23:22.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:21 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:23:23.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[29694]: 5.10 scrub starts 2024-09-20T22:23:23.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[29694]: 5.10 scrub ok 2024-09-20T22:23:23.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[29694]: 3.9 scrub starts 2024-09-20T22:23:23.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[29694]: 3.9 scrub ok 2024-09-20T22:23:23.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-20T22:23:23.072 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-20T22:23:23.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-09-20T22:23:23.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[29694]: 3.8 scrub starts 2024-09-20T22:23:23.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[29694]: 3.8 scrub ok 2024-09-20T22:23:23.073 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[29694]: pgmap v1121: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 10 B/s, 0 objects/s recovering 2024-09-20T22:23:23.073 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[40290]: 5.10 scrub starts 2024-09-20T22:23:23.073 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[40290]: 5.10 scrub ok 2024-09-20T22:23:23.073 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[40290]: 3.9 scrub starts 2024-09-20T22:23:23.073 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[40290]: 3.9 scrub ok 2024-09-20T22:23:23.073 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-20T22:23:23.073 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-20T22:23:23.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-09-20T22:23:23.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[40290]: 3.8 scrub starts 2024-09-20T22:23:23.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[40290]: 3.8 scrub ok 2024-09-20T22:23:23.074 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:22 smithi060 ceph-mon[40290]: pgmap v1121: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 10 B/s, 0 objects/s recovering 2024-09-20T22:23:23.100 INFO:teuthology.orchestra.run.smithi060.stdout:244813136123 2024-09-20T22:23:23.100 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-20T22:23:23.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:23:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:23:23.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:22 smithi067 ceph-mon[33511]: 5.10 scrub starts 2024-09-20T22:23:23.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:22 smithi067 ceph-mon[33511]: 5.10 scrub ok 2024-09-20T22:23:23.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:22 smithi067 ceph-mon[33511]: 3.9 scrub starts 2024-09-20T22:23:23.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:22 smithi067 ceph-mon[33511]: 3.9 scrub ok 2024-09-20T22:23:23.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:22 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]: dispatch 2024-09-20T22:23:23.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:22 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]: dispatch 2024-09-20T22:23:23.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:22 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-09-20T22:23:23.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:22 smithi067 ceph-mon[33511]: 3.8 scrub starts 2024-09-20T22:23:23.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:22 smithi067 ceph-mon[33511]: 3.8 scrub ok 2024-09-20T22:23:23.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:22 smithi067 ceph-mon[33511]: pgmap v1121: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s; 10 B/s, 0 objects/s recovering 2024-09-20T22:23:23.630 INFO:teuthology.orchestra.run.smithi060.stdout:206158430460 2024-09-20T22:23:23.630 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-20T22:23:24.013 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:23 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-20T22:23:24.013 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:23 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-20T22:23:24.013 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:23 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-09-20T22:23:24.013 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:23 smithi060 ceph-mon[29694]: osdmap e591: 8 total, 8 up, 8 in 2024-09-20T22:23:24.013 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:23 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:23:24.013 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:23 smithi060 ceph-mon[29694]: 3.1d scrub starts 2024-09-20T22:23:24.013 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:23 smithi060 ceph-mon[29694]: 3.1d scrub ok 2024-09-20T22:23:24.014 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:23 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-20T22:23:24.014 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:23 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-20T22:23:24.014 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:23 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-09-20T22:23:24.014 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:23 smithi060 ceph-mon[40290]: osdmap e591: 8 total, 8 up, 8 in 2024-09-20T22:23:24.014 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:23 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:23:24.014 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:23 smithi060 ceph-mon[40290]: 3.1d scrub starts 2024-09-20T22:23:24.014 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:23 smithi060 ceph-mon[40290]: 3.1d scrub ok 2024-09-20T22:23:24.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:23 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.7", "id": [1, 2]}]': finished 2024-09-20T22:23:24.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:23 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.e", "id": [1, 2]}]': finished 2024-09-20T22:23:24.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:23 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-09-20T22:23:24.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:23 smithi067 ceph-mon[33511]: osdmap e591: 8 total, 8 up, 8 in 2024-09-20T22:23:24.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:23 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:23:24.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:23 smithi067 ceph-mon[33511]: 3.1d scrub starts 2024-09-20T22:23:24.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:23 smithi067 ceph-mon[33511]: 3.1d scrub ok 2024-09-20T22:23:24.435 INFO:teuthology.orchestra.run.smithi060.stdout:34359738655 2024-09-20T22:23:24.435 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-20T22:23:25.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:24 smithi067 ceph-mon[33511]: 6.15 deep-scrub starts 2024-09-20T22:23:25.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:24 smithi067 ceph-mon[33511]: 6.15 deep-scrub ok 2024-09-20T22:23:25.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:24 smithi067 ceph-mon[33511]: osdmap e592: 8 total, 8 up, 8 in 2024-09-20T22:23:25.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:24 smithi067 ceph-mon[33511]: 1.13 scrub starts 2024-09-20T22:23:25.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:24 smithi067 ceph-mon[33511]: 1.13 scrub ok 2024-09-20T22:23:25.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:24 smithi067 ceph-mon[33511]: 5.1b deep-scrub starts 2024-09-20T22:23:25.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:24 smithi067 ceph-mon[33511]: 5.1b deep-scrub ok 2024-09-20T22:23:25.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:24 smithi067 ceph-mon[33511]: 4.16 scrub starts 2024-09-20T22:23:25.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:24 smithi067 ceph-mon[33511]: 4.16 scrub ok 2024-09-20T22:23:25.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:24 smithi067 ceph-mon[33511]: pgmap v1124: 162 pgs: 1 peering, 161 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 796 B/s rd, 0 op/s 2024-09-20T22:23:25.285 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:25 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:23:25] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:23:25.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[40290]: 6.15 deep-scrub starts 2024-09-20T22:23:25.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[40290]: 6.15 deep-scrub ok 2024-09-20T22:23:25.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[40290]: osdmap e592: 8 total, 8 up, 8 in 2024-09-20T22:23:25.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[40290]: 1.13 scrub starts 2024-09-20T22:23:25.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[40290]: 1.13 scrub ok 2024-09-20T22:23:25.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[40290]: 5.1b deep-scrub starts 2024-09-20T22:23:25.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[40290]: 5.1b deep-scrub ok 2024-09-20T22:23:25.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[40290]: 4.16 scrub starts 2024-09-20T22:23:25.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[40290]: 4.16 scrub ok 2024-09-20T22:23:25.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[40290]: pgmap v1124: 162 pgs: 1 peering, 161 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 796 B/s rd, 0 op/s 2024-09-20T22:23:25.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[29694]: 6.15 deep-scrub starts 2024-09-20T22:23:25.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[29694]: 6.15 deep-scrub ok 2024-09-20T22:23:25.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[29694]: osdmap e592: 8 total, 8 up, 8 in 2024-09-20T22:23:25.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[29694]: 1.13 scrub starts 2024-09-20T22:23:25.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[29694]: 1.13 scrub ok 2024-09-20T22:23:25.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[29694]: 5.1b deep-scrub starts 2024-09-20T22:23:25.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[29694]: 5.1b deep-scrub ok 2024-09-20T22:23:25.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[29694]: 4.16 scrub starts 2024-09-20T22:23:25.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[29694]: 4.16 scrub ok 2024-09-20T22:23:25.287 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:24 smithi060 ceph-mon[29694]: pgmap v1124: 162 pgs: 1 peering, 161 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 796 B/s rd, 0 op/s 2024-09-20T22:23:25.339 INFO:teuthology.orchestra.run.smithi060.stdout:167503724802 2024-09-20T22:23:25.339 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-20T22:23:25.347 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:25.349 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:25.349 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:25.741 INFO:teuthology.orchestra.run.smithi060.stdout:622770258120 2024-09-20T22:23:25.741 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-20T22:23:25.980 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:25 smithi060 ceph-mon[40290]: 3.11 scrub starts 2024-09-20T22:23:25.981 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:25 smithi060 ceph-mon[40290]: 3.11 scrub ok 2024-09-20T22:23:25.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:25 smithi060 ceph-mon[29694]: 3.11 scrub starts 2024-09-20T22:23:25.981 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:25 smithi060 ceph-mon[29694]: 3.11 scrub ok 2024-09-20T22:23:26.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:25 smithi067 ceph-mon[33511]: 3.11 scrub starts 2024-09-20T22:23:26.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:25 smithi067 ceph-mon[33511]: 3.11 scrub ok 2024-09-20T22:23:26.798 INFO:teuthology.orchestra.run.smithi060.stdout:77309411605 2024-09-20T22:23:27.125 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:26 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:23:27.125 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:26 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:23:27.125 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:26 smithi060 ceph-mon[40290]: 3.b scrub starts 2024-09-20T22:23:27.125 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:26 smithi060 ceph-mon[40290]: 3.b scrub ok 2024-09-20T22:23:27.126 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:26 smithi060 ceph-mon[40290]: pgmap v1125: 162 pgs: 1 peering, 161 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:23:27.126 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:26 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1607031951' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-20T22:23:27.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:26 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:23:27.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:26 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:23:27.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:26 smithi060 ceph-mon[29694]: 3.b scrub starts 2024-09-20T22:23:27.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:26 smithi060 ceph-mon[29694]: 3.b scrub ok 2024-09-20T22:23:27.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:26 smithi060 ceph-mon[29694]: pgmap v1125: 162 pgs: 1 peering, 161 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:23:27.126 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:26 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1607031951' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-20T22:23:27.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:26 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:23:27.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:26 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:23:27.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:26 smithi067 ceph-mon[33511]: 3.b scrub starts 2024-09-20T22:23:27.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:26 smithi067 ceph-mon[33511]: 3.b scrub ok 2024-09-20T22:23:27.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:26 smithi067 ceph-mon[33511]: pgmap v1125: 162 pgs: 1 peering, 161 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:23:27.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:26 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1607031951' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-20T22:23:27.311 INFO:teuthology.orchestra.run.smithi060.stdout:2284922601494 2024-09-20T22:23:27.674 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411603 got 77309411605 for osd.2 2024-09-20T22:23:27.675 DEBUG:teuthology.parallel:result is None 2024-09-20T22:23:27.983 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:27 smithi060 ceph-mon[29694]: 1.9 deep-scrub starts 2024-09-20T22:23:27.983 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:27 smithi060 ceph-mon[29694]: 1.9 deep-scrub ok 2024-09-20T22:23:27.983 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:27 smithi060 ceph-mon[29694]: 4.1c deep-scrub starts 2024-09-20T22:23:27.983 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:27 smithi060 ceph-mon[29694]: 4.1c deep-scrub ok 2024-09-20T22:23:27.983 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:27 smithi060 ceph-mon[29694]: osdmap e593: 8 total, 8 up, 8 in 2024-09-20T22:23:27.983 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:27 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/2984355551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-20T22:23:27.983 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:27 smithi060 ceph-mon[40290]: 1.9 deep-scrub starts 2024-09-20T22:23:27.983 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:27 smithi060 ceph-mon[40290]: 1.9 deep-scrub ok 2024-09-20T22:23:27.983 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:27 smithi060 ceph-mon[40290]: 4.1c deep-scrub starts 2024-09-20T22:23:27.984 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:27 smithi060 ceph-mon[40290]: 4.1c deep-scrub ok 2024-09-20T22:23:27.984 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:27 smithi060 ceph-mon[40290]: osdmap e593: 8 total, 8 up, 8 in 2024-09-20T22:23:27.984 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:27 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/2984355551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-20T22:23:28.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:27 smithi067 ceph-mon[33511]: 1.9 deep-scrub starts 2024-09-20T22:23:28.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:27 smithi067 ceph-mon[33511]: 1.9 deep-scrub ok 2024-09-20T22:23:28.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:27 smithi067 ceph-mon[33511]: 4.1c deep-scrub starts 2024-09-20T22:23:28.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:27 smithi067 ceph-mon[33511]: 4.1c deep-scrub ok 2024-09-20T22:23:28.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:27 smithi067 ceph-mon[33511]: osdmap e593: 8 total, 8 up, 8 in 2024-09-20T22:23:28.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:27 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/2984355551' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-20T22:23:28.331 INFO:tasks.cephadm.ceph_manager.ceph:need seq 2284922601493 got 2284922601494 for osd.4 2024-09-20T22:23:28.331 DEBUG:teuthology.parallel:result is None 2024-09-20T22:23:28.560 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:28.560 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:28.561 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:28.634 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:28.636 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:28.778 INFO:teuthology.orchestra.run.smithi060.stdout:111669149968 2024-09-20T22:23:29.208 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:28 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:23:29.208 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:28 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:23:29.208 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:28 smithi060 ceph-mon[29694]: 5.19 scrub starts 2024-09-20T22:23:29.208 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:28 smithi060 ceph-mon[29694]: 5.19 scrub ok 2024-09-20T22:23:29.209 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:28 smithi060 ceph-mon[29694]: pgmap v1127: 162 pgs: 1 peering, 161 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 54 B/s, 1 objects/s recovering 2024-09-20T22:23:29.209 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:28 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/900480799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-20T22:23:29.209 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:28 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:23:29.209 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:28 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:23:29.209 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:28 smithi060 ceph-mon[40290]: 5.19 scrub starts 2024-09-20T22:23:29.209 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:28 smithi060 ceph-mon[40290]: 5.19 scrub ok 2024-09-20T22:23:29.209 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:28 smithi060 ceph-mon[40290]: pgmap v1127: 162 pgs: 1 peering, 161 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 54 B/s, 1 objects/s recovering 2024-09-20T22:23:29.209 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:28 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/900480799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-20T22:23:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:28 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:23:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:28 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:23:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:28 smithi067 ceph-mon[33511]: 5.19 scrub starts 2024-09-20T22:23:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:28 smithi067 ceph-mon[33511]: 5.19 scrub ok 2024-09-20T22:23:29.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:28 smithi067 ceph-mon[33511]: pgmap v1127: 162 pgs: 1 peering, 161 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 511 B/s rd, 0 op/s; 54 B/s, 1 objects/s recovering 2024-09-20T22:23:29.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:28 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/900480799' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-20T22:23:29.538 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149967 got 111669149968 for osd.3 2024-09-20T22:23:29.538 DEBUG:teuthology.parallel:result is None 2024-09-20T22:23:30.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:29 smithi060 ceph-mon[29694]: osdmap e594: 8 total, 8 up, 8 in 2024-09-20T22:23:30.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:29 smithi060 ceph-mon[29694]: 1.4 scrub starts 2024-09-20T22:23:30.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:29 smithi060 ceph-mon[29694]: 1.4 scrub ok 2024-09-20T22:23:30.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:29 smithi060 ceph-mon[29694]: 6.1 scrub starts 2024-09-20T22:23:30.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:29 smithi060 ceph-mon[29694]: 6.1 scrub ok 2024-09-20T22:23:30.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:29 smithi060 ceph-mon[40290]: osdmap e594: 8 total, 8 up, 8 in 2024-09-20T22:23:30.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:29 smithi060 ceph-mon[40290]: 1.4 scrub starts 2024-09-20T22:23:30.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:29 smithi060 ceph-mon[40290]: 1.4 scrub ok 2024-09-20T22:23:30.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:29 smithi060 ceph-mon[40290]: 6.1 scrub starts 2024-09-20T22:23:30.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:29 smithi060 ceph-mon[40290]: 6.1 scrub ok 2024-09-20T22:23:30.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:29 smithi067 ceph-mon[33511]: osdmap e594: 8 total, 8 up, 8 in 2024-09-20T22:23:30.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:29 smithi067 ceph-mon[33511]: 1.4 scrub starts 2024-09-20T22:23:30.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:29 smithi067 ceph-mon[33511]: 1.4 scrub ok 2024-09-20T22:23:30.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:29 smithi067 ceph-mon[33511]: 6.1 scrub starts 2024-09-20T22:23:30.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:29 smithi067 ceph-mon[33511]: 6.1 scrub ok 2024-09-20T22:23:30.356 INFO:teuthology.orchestra.run.smithi060.stdout:34359738657 2024-09-20T22:23:30.584 INFO:teuthology.orchestra.run.smithi060.stdout:622770258120 2024-09-20T22:23:31.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:30 smithi060 ceph-mon[29694]: 5.1 scrub starts 2024-09-20T22:23:31.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:30 smithi060 ceph-mon[29694]: 5.1 scrub ok 2024-09-20T22:23:31.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:30 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/867521151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-20T22:23:31.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:30 smithi060 ceph-mon[29694]: pgmap v1129: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 465 B/s rd, 0 op/s; 66 B/s, 1 objects/s recovering 2024-09-20T22:23:31.263 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:30 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1220401956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-20T22:23:31.263 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:30 smithi060 ceph-mon[40290]: 5.1 scrub starts 2024-09-20T22:23:31.263 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:30 smithi060 ceph-mon[40290]: 5.1 scrub ok 2024-09-20T22:23:31.263 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:30 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/867521151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-20T22:23:31.264 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:30 smithi060 ceph-mon[40290]: pgmap v1129: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 465 B/s rd, 0 op/s; 66 B/s, 1 objects/s recovering 2024-09-20T22:23:31.264 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:30 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1220401956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-20T22:23:31.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:30 smithi067 ceph-mon[33511]: 5.1 scrub starts 2024-09-20T22:23:31.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:30 smithi067 ceph-mon[33511]: 5.1 scrub ok 2024-09-20T22:23:31.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:30 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/867521151' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-20T22:23:31.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:30 smithi067 ceph-mon[33511]: pgmap v1129: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 465 B/s rd, 0 op/s; 66 B/s, 1 objects/s recovering 2024-09-20T22:23:31.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:30 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1220401956' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-20T22:23:31.292 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738655 got 34359738657 for osd.0 2024-09-20T22:23:31.292 DEBUG:teuthology.parallel:result is None 2024-09-20T22:23:31.934 INFO:teuthology.orchestra.run.smithi060.stdout:167503724804 2024-09-20T22:23:32.116 INFO:teuthology.orchestra.run.smithi060.stdout:244813136124 2024-09-20T22:23:32.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:23:32.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:23:32.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[29694]: 6.f scrub starts 2024-09-20T22:23:32.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[29694]: 6.f scrub ok 2024-09-20T22:23:32.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[29694]: osdmap e595: 8 total, 8 up, 8 in 2024-09-20T22:23:32.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[29694]: purged_snaps scrub starts 2024-09-20T22:23:32.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[29694]: purged_snaps scrub ok 2024-09-20T22:23:32.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[29694]: 5.14 scrub starts 2024-09-20T22:23:32.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[29694]: 5.14 scrub ok 2024-09-20T22:23:32.117 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[29694]: 3.1 scrub starts 2024-09-20T22:23:32.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[29694]: 3.1 scrub ok 2024-09-20T22:23:32.118 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[29694]: osdmap e596: 8 total, 8 up, 8 in 2024-09-20T22:23:32.118 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:23:32.118 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:23:32.118 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[40290]: 6.f scrub starts 2024-09-20T22:23:32.118 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[40290]: 6.f scrub ok 2024-09-20T22:23:32.118 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[40290]: osdmap e595: 8 total, 8 up, 8 in 2024-09-20T22:23:32.118 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[40290]: purged_snaps scrub starts 2024-09-20T22:23:32.118 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[40290]: purged_snaps scrub ok 2024-09-20T22:23:32.118 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[40290]: 5.14 scrub starts 2024-09-20T22:23:32.118 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[40290]: 5.14 scrub ok 2024-09-20T22:23:32.118 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[40290]: 3.1 scrub starts 2024-09-20T22:23:32.118 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[40290]: 3.1 scrub ok 2024-09-20T22:23:32.119 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:31 smithi060 ceph-mon[40290]: osdmap e596: 8 total, 8 up, 8 in 2024-09-20T22:23:32.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:31 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:23:32.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:31 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:23:32.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:31 smithi067 ceph-mon[33511]: 6.f scrub starts 2024-09-20T22:23:32.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:31 smithi067 ceph-mon[33511]: 6.f scrub ok 2024-09-20T22:23:32.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:31 smithi067 ceph-mon[33511]: osdmap e595: 8 total, 8 up, 8 in 2024-09-20T22:23:32.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:31 smithi067 ceph-mon[33511]: purged_snaps scrub starts 2024-09-20T22:23:32.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:31 smithi067 ceph-mon[33511]: purged_snaps scrub ok 2024-09-20T22:23:32.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:31 smithi067 ceph-mon[33511]: 5.14 scrub starts 2024-09-20T22:23:32.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:31 smithi067 ceph-mon[33511]: 5.14 scrub ok 2024-09-20T22:23:32.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:31 smithi067 ceph-mon[33511]: 3.1 scrub starts 2024-09-20T22:23:32.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:31 smithi067 ceph-mon[33511]: 3.1 scrub ok 2024-09-20T22:23:32.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:31 smithi067 ceph-mon[33511]: osdmap e596: 8 total, 8 up, 8 in 2024-09-20T22:23:32.929 INFO:tasks.cephadm.ceph_manager.ceph:need seq 622770258120 got 622770258120 for osd.1 2024-09-20T22:23:32.929 DEBUG:teuthology.parallel:result is None 2024-09-20T22:23:33.263 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:23:33 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:23:33.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:32 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/1261245914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-20T22:23:33.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:32 smithi067 ceph-mon[33511]: 3.0 scrub starts 2024-09-20T22:23:33.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:32 smithi067 ceph-mon[33511]: 3.0 scrub ok 2024-09-20T22:23:33.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:32 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/110254594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-20T22:23:33.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:32 smithi067 ceph-mon[33511]: pgmap v1132: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 733 B/s rd, 0 op/s; 78 B/s, 2 objects/s recovering 2024-09-20T22:23:33.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:32 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/1261245914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-20T22:23:33.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:32 smithi060 ceph-mon[29694]: 3.0 scrub starts 2024-09-20T22:23:33.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:32 smithi060 ceph-mon[29694]: 3.0 scrub ok 2024-09-20T22:23:33.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:32 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/110254594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-20T22:23:33.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:32 smithi060 ceph-mon[29694]: pgmap v1132: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 733 B/s rd, 0 op/s; 78 B/s, 2 objects/s recovering 2024-09-20T22:23:33.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:32 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/1261245914' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-20T22:23:33.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:32 smithi060 ceph-mon[40290]: 3.0 scrub starts 2024-09-20T22:23:33.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:32 smithi060 ceph-mon[40290]: 3.0 scrub ok 2024-09-20T22:23:33.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:32 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/110254594' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-20T22:23:33.286 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:32 smithi060 ceph-mon[40290]: pgmap v1132: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 733 B/s rd, 0 op/s; 78 B/s, 2 objects/s recovering 2024-09-20T22:23:33.392 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813136123 got 244813136124 for osd.7 2024-09-20T22:23:33.392 DEBUG:teuthology.parallel:result is None 2024-09-20T22:23:33.885 INFO:teuthology.orchestra.run.smithi060.stdout:206158430462 2024-09-20T22:23:34.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:33 smithi060 ceph-mon[29694]: 5.1c scrub starts 2024-09-20T22:23:34.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:33 smithi060 ceph-mon[29694]: 5.1c scrub ok 2024-09-20T22:23:34.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:33 smithi060 ceph-mon[29694]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:23:34.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:33 smithi060 ceph-mon[29694]: 3.d scrub starts 2024-09-20T22:23:34.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:33 smithi060 ceph-mon[29694]: 3.d scrub ok 2024-09-20T22:23:34.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:33 smithi060 ceph-mon[40290]: 5.1c scrub starts 2024-09-20T22:23:34.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:33 smithi060 ceph-mon[40290]: 5.1c scrub ok 2024-09-20T22:23:34.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:33 smithi060 ceph-mon[40290]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:23:34.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:33 smithi060 ceph-mon[40290]: 3.d scrub starts 2024-09-20T22:23:34.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:33 smithi060 ceph-mon[40290]: 3.d scrub ok 2024-09-20T22:23:34.177 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724802 got 167503724804 for osd.5 2024-09-20T22:23:34.177 DEBUG:teuthology.parallel:result is None 2024-09-20T22:23:34.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:33 smithi067 ceph-mon[33511]: 5.1c scrub starts 2024-09-20T22:23:34.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:33 smithi067 ceph-mon[33511]: 5.1c scrub ok 2024-09-20T22:23:34.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:33 smithi067 ceph-mon[33511]: from='client.24434 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-20T22:23:34.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:33 smithi067 ceph-mon[33511]: 3.d scrub starts 2024-09-20T22:23:34.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:33 smithi067 ceph-mon[33511]: 3.d scrub ok 2024-09-20T22:23:34.749 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430460 got 206158430462 for osd.6 2024-09-20T22:23:34.750 DEBUG:teuthology.parallel:result is None 2024-09-20T22:23:34.750 INFO:tasks.cephadm.ceph_manager.ceph:waiting for recovery to complete 2024-09-20T22:23:34.750 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:23:35.010 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:35.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:34 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/396843024' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-20T22:23:35.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:34 smithi060 ceph-mon[29694]: 4.8 scrub starts 2024-09-20T22:23:35.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:34 smithi060 ceph-mon[29694]: 4.8 scrub ok 2024-09-20T22:23:35.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:34 smithi060 ceph-mon[29694]: 4.6 scrub starts 2024-09-20T22:23:35.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:34 smithi060 ceph-mon[29694]: 4.6 scrub ok 2024-09-20T22:23:35.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:34 smithi060 ceph-mon[29694]: pgmap v1133: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 18 B/s, 0 objects/s recovering 2024-09-20T22:23:35.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:34 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/396843024' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-20T22:23:35.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:34 smithi060 ceph-mon[40290]: 4.8 scrub starts 2024-09-20T22:23:35.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:34 smithi060 ceph-mon[40290]: 4.8 scrub ok 2024-09-20T22:23:35.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:34 smithi060 ceph-mon[40290]: 4.6 scrub starts 2024-09-20T22:23:35.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:34 smithi060 ceph-mon[40290]: 4.6 scrub ok 2024-09-20T22:23:35.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:34 smithi060 ceph-mon[40290]: pgmap v1133: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 18 B/s, 0 objects/s recovering 2024-09-20T22:23:35.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:34 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/396843024' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-20T22:23:35.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:34 smithi067 ceph-mon[33511]: 4.8 scrub starts 2024-09-20T22:23:35.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:34 smithi067 ceph-mon[33511]: 4.8 scrub ok 2024-09-20T22:23:35.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:34 smithi067 ceph-mon[33511]: 4.6 scrub starts 2024-09-20T22:23:35.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:34 smithi067 ceph-mon[33511]: 4.6 scrub ok 2024-09-20T22:23:35.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:34 smithi067 ceph-mon[33511]: pgmap v1133: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 853 B/s rd, 0 op/s; 18 B/s, 0 objects/s recovering 2024-09-20T22:23:35.297 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:35 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: ::ffff:172.21.15.67 - - [20/Sep/2024:22:23:35] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-09-20T22:23:35.703 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:23:35.705 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:23:36.018 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:35 smithi060 ceph-mon[29694]: 4.f scrub starts 2024-09-20T22:23:36.018 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:35 smithi060 ceph-mon[29694]: 4.f scrub ok 2024-09-20T22:23:36.019 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:35 smithi060 ceph-mon[40290]: 4.f scrub starts 2024-09-20T22:23:36.019 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:35 smithi060 ceph-mon[40290]: 4.f scrub ok 2024-09-20T22:23:36.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:35 smithi067 ceph-mon[33511]: 4.f scrub starts 2024-09-20T22:23:36.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:35 smithi067 ceph-mon[33511]: 4.f scrub ok 2024-09-20T22:23:36.442 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":1133,"stamp":"2024-09-20T22:23:34.426947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":2756,"num_read_kb":2508,"num_write":1008,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":521,"num_bytes_recovered":669612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5156,"ondisk_log_size":5156,"up":486,"acting":486,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":486,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":6153340,"kb_used_data":16136,"kb_used_omap":131,"kb_used_meta":6136892,"kb_avail":743807876,"statfs":{"total":767960285184,"available":761659265024,"internally_reserved":0,"allocated":16523264,"data_stored":11021034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":134878,"internal_metadata":6284177698},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001580"},"pg_stats":[{"pgid":"1.1d","version":"75'128","reported_seq":654,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718936+0000","last_change":"2024-09-20T22:22:38.716841+0000","last_active":"2024-09-20T22:23:31.718936+0000","last_peered":"2024-09-20T22:23:31.718936+0000","last_clean":"2024-09-20T22:23:31.718936+0000","last_became_active":"2024-09-20T22:22:38.716205+0000","last_became_peered":"2024-09-20T22:22:38.716205+0000","last_unstale":"2024-09-20T22:23:31.718936+0000","last_undegraded":"2024-09-20T22:23:31.718936+0000","last_fullsized":"2024-09-20T22:23:31.718936+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:15.002349+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:22:15.002349+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:40.297605+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":592,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.720326+0000","last_change":"2024-09-20T22:22:42.153828+0000","last_active":"2024-09-20T22:23:31.720326+0000","last_peered":"2024-09-20T22:23:31.720326+0000","last_clean":"2024-09-20T22:23:31.720326+0000","last_became_active":"2024-09-20T22:12:52.515921+0000","last_became_peered":"2024-09-20T22:12:52.515921+0000","last_unstale":"2024-09-20T22:23:31.720326+0000","last_undegraded":"2024-09-20T22:23:31.720326+0000","last_fullsized":"2024-09-20T22:23:31.720326+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:42.153726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:43.512066+0000","last_clean_scrub_stamp":"2024-09-20T22:22:42.153726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:10.825665+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"579'25","reported_seq":119,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718090+0000","last_change":"2024-09-20T22:23:00.748734+0000","last_active":"2024-09-20T22:23:31.718090+0000","last_peered":"2024-09-20T22:23:31.718090+0000","last_clean":"2024-09-20T22:23:31.718090+0000","last_became_active":"2024-09-20T22:22:36.732025+0000","last_became_peered":"2024-09-20T22:22:36.732025+0000","last_unstale":"2024-09-20T22:23:31.718090+0000","last_undegraded":"2024-09-20T22:23:31.718090+0000","last_fullsized":"2024-09-20T22:23:31.718090+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'25","last_scrub_stamp":"2024-09-20T22:23:00.748614+0000","last_deep_scrub":"252'13","last_deep_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_clean_scrub_stamp":"2024-09-20T22:23:00.748614+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:24.603883+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":81,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718810+0000","last_change":"2024-09-20T22:23:28.170111+0000","last_active":"2024-09-20T22:23:31.718810+0000","last_peered":"2024-09-20T22:23:31.718810+0000","last_clean":"2024-09-20T22:23:31.718810+0000","last_became_active":"2024-09-20T22:22:36.741459+0000","last_became_peered":"2024-09-20T22:22:36.741459+0000","last_unstale":"2024-09-20T22:23:31.718810+0000","last_undegraded":"2024-09-20T22:23:31.718810+0000","last_fullsized":"2024-09-20T22:23:31.718810+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:28.170020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:23:28.170020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:57.705531+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":90,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718831+0000","last_change":"2024-09-20T22:22:42.640949+0000","last_active":"2024-09-20T22:23:31.718831+0000","last_peered":"2024-09-20T22:23:31.718831+0000","last_clean":"2024-09-20T22:23:31.718831+0000","last_became_active":"2024-09-20T22:22:36.732433+0000","last_became_peered":"2024-09-20T22:22:36.732433+0000","last_unstale":"2024-09-20T22:23:31.718831+0000","last_undegraded":"2024-09-20T22:23:31.718831+0000","last_fullsized":"2024-09-20T22:23:31.718831+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:42.640859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:42.640859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:48.484375+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1c","version":"75'128","reported_seq":1128,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.954238+0000","last_change":"2024-09-20T22:22:39.718800+0000","last_active":"2024-09-20T22:23:30.954238+0000","last_peered":"2024-09-20T22:23:30.954238+0000","last_clean":"2024-09-20T22:23:30.954238+0000","last_became_active":"2024-09-20T22:22:39.718360+0000","last_became_peered":"2024-09-20T22:22:39.718360+0000","last_unstale":"2024-09-20T22:23:30.954238+0000","last_undegraded":"2024-09-20T22:23:30.954238+0000","last_fullsized":"2024-09-20T22:23:30.954238+0000","mapping_epoch":567,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":568,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:34.574149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:34.574149+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:52.649725+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":1159,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952347+0000","last_change":"2024-09-20T22:22:33.952815+0000","last_active":"2024-09-20T22:23:30.952347+0000","last_peered":"2024-09-20T22:23:30.952347+0000","last_clean":"2024-09-20T22:23:30.952347+0000","last_became_active":"2024-09-20T22:22:30.667295+0000","last_became_peered":"2024-09-20T22:22:30.667295+0000","last_unstale":"2024-09-20T22:23:30.952347+0000","last_undegraded":"2024-09-20T22:23:30.952347+0000","last_fullsized":"2024-09-20T22:23:30.952347+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:22:33.952687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:33.952687+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:49.531816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":587,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952312+0000","last_change":"2024-09-20T22:22:51.340462+0000","last_active":"2024-09-20T22:23:30.952312+0000","last_peered":"2024-09-20T22:23:30.952312+0000","last_clean":"2024-09-20T22:23:30.952312+0000","last_became_active":"2024-09-20T22:22:51.340054+0000","last_became_peered":"2024-09-20T22:22:51.340054+0000","last_unstale":"2024-09-20T22:23:30.952312+0000","last_undegraded":"2024-09-20T22:23:30.952312+0000","last_fullsized":"2024-09-20T22:23:30.952312+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:26.924884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:26.924884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:56.275060+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"579'46","reported_seq":400,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.951787+0000","last_change":"2024-09-20T22:22:51.341873+0000","last_active":"2024-09-20T22:23:30.951787+0000","last_peered":"2024-09-20T22:23:30.951787+0000","last_clean":"2024-09-20T22:23:30.951787+0000","last_became_active":"2024-09-20T22:22:51.341154+0000","last_became_peered":"2024-09-20T22:22:51.341154+0000","last_unstale":"2024-09-20T22:23:30.951787+0000","last_undegraded":"2024-09-20T22:23:30.951787+0000","last_fullsized":"2024-09-20T22:23:30.951787+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'34","last_scrub_stamp":"2024-09-20T22:22:40.048191+0000","last_deep_scrub":"406'34","last_deep_scrub_stamp":"2024-09-20T22:16:31.033354+0000","last_clean_scrub_stamp":"2024-09-20T22:22:40.048191+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:40.973089+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":90,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718092+0000","last_change":"2024-09-20T22:22:48.688177+0000","last_active":"2024-09-20T22:23:31.718092+0000","last_peered":"2024-09-20T22:23:31.718092+0000","last_clean":"2024-09-20T22:23:31.718092+0000","last_became_active":"2024-09-20T22:22:36.732727+0000","last_became_peered":"2024-09-20T22:22:36.732727+0000","last_unstale":"2024-09-20T22:23:31.718092+0000","last_undegraded":"2024-09-20T22:23:31.718092+0000","last_fullsized":"2024-09-20T22:23:31.718092+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:48.688079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:48.688079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:02.631329+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1189,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062825+0000","last_change":"2024-09-20T22:23:23.138971+0000","last_active":"2024-09-20T22:23:31.062825+0000","last_peered":"2024-09-20T22:23:31.062825+0000","last_clean":"2024-09-20T22:23:31.062825+0000","last_became_active":"2024-09-20T22:22:34.743055+0000","last_became_peered":"2024-09-20T22:22:34.743055+0000","last_unstale":"2024-09-20T22:23:31.062825+0000","last_undegraded":"2024-09-20T22:23:31.062825+0000","last_fullsized":"2024-09-20T22:23:31.062825+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:23.138880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:23:23.138880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:45.175131+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"579'25","reported_seq":37,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718979+0000","last_change":"2024-09-20T22:23:10.785953+0000","last_active":"2024-09-20T22:23:31.718979+0000","last_peered":"2024-09-20T22:23:31.718979+0000","last_clean":"2024-09-20T22:23:31.718979+0000","last_became_active":"2024-09-20T22:23:10.777633+0000","last_became_peered":"2024-09-20T22:23:10.777633+0000","last_unstale":"2024-09-20T22:23:31.718979+0000","last_undegraded":"2024-09-20T22:23:31.718979+0000","last_fullsized":"2024-09-20T22:23:31.718979+0000","mapping_epoch":587,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":588,"parent":"0.0","parent_split_bits":0,"last_scrub":"568'19","last_scrub_stamp":"2024-09-20T22:22:40.110956+0000","last_deep_scrub":"568'19","last_deep_scrub_stamp":"2024-09-20T22:22:40.110956+0000","last_clean_scrub_stamp":"2024-09-20T22:22:40.110956+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:02.123233+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":690,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062620+0000","last_change":"2024-09-20T22:23:24.139950+0000","last_active":"2024-09-20T22:23:31.062620+0000","last_peered":"2024-09-20T22:23:31.062620+0000","last_clean":"2024-09-20T22:23:31.062620+0000","last_became_active":"2024-09-20T22:12:52.518549+0000","last_became_peered":"2024-09-20T22:12:52.518549+0000","last_unstale":"2024-09-20T22:23:31.062620+0000","last_undegraded":"2024-09-20T22:23:31.062620+0000","last_fullsized":"2024-09-20T22:23:31.062620+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:24.139844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:23:24.139844+0000","last_clean_scrub_stamp":"2024-09-20T22:23:24.139844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:48.864393+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":117,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.717736+0000","last_change":"2024-09-20T22:22:46.044116+0000","last_active":"2024-09-20T22:23:31.717736+0000","last_peered":"2024-09-20T22:23:31.717736+0000","last_clean":"2024-09-20T22:23:31.717736+0000","last_became_active":"2024-09-20T22:22:36.747092+0000","last_became_peered":"2024-09-20T22:22:36.747092+0000","last_unstale":"2024-09-20T22:23:31.717736+0000","last_undegraded":"2024-09-20T22:23:31.717736+0000","last_fullsized":"2024-09-20T22:23:31.717736+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:46.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:37.934760+0000","last_clean_scrub_stamp":"2024-09-20T22:22:46.044046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:56.375187+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1e","version":"75'128","reported_seq":676,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952206+0000","last_change":"2024-09-20T22:22:33.741991+0000","last_active":"2024-09-20T22:23:30.952206+0000","last_peered":"2024-09-20T22:23:30.952206+0000","last_clean":"2024-09-20T22:23:30.952206+0000","last_became_active":"2024-09-20T22:22:33.741612+0000","last_became_peered":"2024-09-20T22:22:33.741612+0000","last_unstale":"2024-09-20T22:23:30.952206+0000","last_undegraded":"2024-09-20T22:23:30.952206+0000","last_fullsized":"2024-09-20T22:23:30.952206+0000","mapping_epoch":561,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":562,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:19.051625+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:17:40.367940+0000","last_clean_scrub_stamp":"2024-09-20T22:22:19.051625+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:40.530410+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1192,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062869+0000","last_change":"2024-09-20T22:22:36.743983+0000","last_active":"2024-09-20T22:23:31.062869+0000","last_peered":"2024-09-20T22:23:31.062869+0000","last_clean":"2024-09-20T22:23:31.062869+0000","last_became_active":"2024-09-20T22:22:36.743656+0000","last_became_peered":"2024-09-20T22:22:36.743656+0000","last_unstale":"2024-09-20T22:23:31.062869+0000","last_undegraded":"2024-09-20T22:23:31.062869+0000","last_fullsized":"2024-09-20T22:23:31.062869+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:32.037246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:30.215263+0000","last_clean_scrub_stamp":"2024-09-20T22:22:32.037246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:41.815514+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"579'9","reported_seq":105,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718729+0000","last_change":"2024-09-20T22:23:12.747136+0000","last_active":"2024-09-20T22:23:31.718729+0000","last_peered":"2024-09-20T22:23:31.718729+0000","last_clean":"2024-09-20T22:23:31.718729+0000","last_became_active":"2024-09-20T22:22:36.732453+0000","last_became_peered":"2024-09-20T22:22:36.732453+0000","last_unstale":"2024-09-20T22:23:31.718729+0000","last_undegraded":"2024-09-20T22:23:31.718729+0000","last_fullsized":"2024-09-20T22:23:31.718729+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'9","last_scrub_stamp":"2024-09-20T22:23:12.747038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:23:12.747038+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:19.255811+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":12,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1181,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.954098+0000","last_change":"2024-09-20T22:23:19.597163+0000","last_active":"2024-09-20T22:23:30.954098+0000","last_peered":"2024-09-20T22:23:30.954098+0000","last_clean":"2024-09-20T22:23:30.954098+0000","last_became_active":"2024-09-20T22:22:36.744577+0000","last_became_peered":"2024-09-20T22:22:36.744577+0000","last_unstale":"2024-09-20T22:23:30.954098+0000","last_undegraded":"2024-09-20T22:23:30.954098+0000","last_fullsized":"2024-09-20T22:23:30.954098+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:19.597078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:23:19.597078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:35.017777+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":696,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062425+0000","last_change":"2024-09-20T22:23:21.059592+0000","last_active":"2024-09-20T22:23:31.062425+0000","last_peered":"2024-09-20T22:23:31.062425+0000","last_clean":"2024-09-20T22:23:31.062425+0000","last_became_active":"2024-09-20T22:22:36.723117+0000","last_became_peered":"2024-09-20T22:22:36.723117+0000","last_unstale":"2024-09-20T22:23:31.062425+0000","last_undegraded":"2024-09-20T22:23:31.062425+0000","last_fullsized":"2024-09-20T22:23:31.062425+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:21.059499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:23:21.059499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:43.096015+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.19","version":"75'128","reported_seq":622,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952474+0000","last_change":"2024-09-20T22:22:52.005610+0000","last_active":"2024-09-20T22:23:30.952474+0000","last_peered":"2024-09-20T22:23:30.952474+0000","last_clean":"2024-09-20T22:23:30.952474+0000","last_became_active":"2024-09-20T22:22:51.340072+0000","last_became_peered":"2024-09-20T22:22:51.340072+0000","last_unstale":"2024-09-20T22:23:30.952474+0000","last_undegraded":"2024-09-20T22:23:30.952474+0000","last_fullsized":"2024-09-20T22:23:30.952474+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:52.005539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:52.005539+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:02.049542+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":84,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719503+0000","last_change":"2024-09-20T22:23:04.752285+0000","last_active":"2024-09-20T22:23:31.719503+0000","last_peered":"2024-09-20T22:23:31.719503+0000","last_clean":"2024-09-20T22:23:31.719503+0000","last_became_active":"2024-09-20T22:22:34.743937+0000","last_became_peered":"2024-09-20T22:22:34.743937+0000","last_unstale":"2024-09-20T22:23:31.719503+0000","last_undegraded":"2024-09-20T22:23:31.719503+0000","last_fullsized":"2024-09-20T22:23:31.719503+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:04.752194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:19:44.809724+0000","last_clean_scrub_stamp":"2024-09-20T22:23:04.752194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:11.062720+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":109,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719396+0000","last_change":"2024-09-20T22:22:34.741813+0000","last_active":"2024-09-20T22:23:31.719396+0000","last_peered":"2024-09-20T22:23:31.719396+0000","last_clean":"2024-09-20T22:23:31.719396+0000","last_became_active":"2024-09-20T22:22:34.741335+0000","last_became_peered":"2024-09-20T22:22:34.741335+0000","last_unstale":"2024-09-20T22:23:31.719396+0000","last_undegraded":"2024-09-20T22:23:31.719396+0000","last_fullsized":"2024-09-20T22:23:31.719396+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:11.655870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:22:11.655870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:39.733864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"579'46","reported_seq":806,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959731+0000","last_change":"2024-09-20T22:23:26.769670+0000","last_active":"2024-09-20T22:23:31.959731+0000","last_peered":"2024-09-20T22:23:31.959731+0000","last_clean":"2024-09-20T22:23:31.959731+0000","last_became_active":"2024-09-20T22:22:36.735934+0000","last_became_peered":"2024-09-20T22:22:36.735934+0000","last_unstale":"2024-09-20T22:23:31.959731+0000","last_undegraded":"2024-09-20T22:23:31.959731+0000","last_fullsized":"2024-09-20T22:23:31.959731+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'46","last_scrub_stamp":"2024-09-20T22:23:26.769577+0000","last_deep_scrub":"579'46","last_deep_scrub_stamp":"2024-09-20T22:23:26.769577+0000","last_clean_scrub_stamp":"2024-09-20T22:23:26.769577+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:55.782563+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":81,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719156+0000","last_change":"2024-09-20T22:23:03.207880+0000","last_active":"2024-09-20T22:23:31.719156+0000","last_peered":"2024-09-20T22:23:31.719156+0000","last_clean":"2024-09-20T22:23:31.719156+0000","last_became_active":"2024-09-20T22:22:36.748783+0000","last_became_peered":"2024-09-20T22:22:36.748783+0000","last_unstale":"2024-09-20T22:23:31.719156+0000","last_undegraded":"2024-09-20T22:23:31.719156+0000","last_fullsized":"2024-09-20T22:23:31.719156+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:03.207810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:44.295028+0000","last_clean_scrub_stamp":"2024-09-20T22:23:03.207810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:17.801439+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.18","version":"75'128","reported_seq":757,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062197+0000","last_change":"2024-09-20T22:22:31.033583+0000","last_active":"2024-09-20T22:23:31.062197+0000","last_peered":"2024-09-20T22:23:31.062197+0000","last_clean":"2024-09-20T22:23:31.062197+0000","last_became_active":"2024-09-20T22:17:44.717551+0000","last_became_peered":"2024-09-20T22:17:44.717551+0000","last_unstale":"2024-09-20T22:23:31.062197+0000","last_undegraded":"2024-09-20T22:23:31.062197+0000","last_fullsized":"2024-09-20T22:23:31.062197+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:31.033455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:31.033455+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:52.517058+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":585,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952634+0000","last_change":"2024-09-20T22:22:51.735400+0000","last_active":"2024-09-20T22:23:30.952634+0000","last_peered":"2024-09-20T22:23:30.952634+0000","last_clean":"2024-09-20T22:23:30.952634+0000","last_became_active":"2024-09-20T22:22:51.735132+0000","last_became_peered":"2024-09-20T22:22:51.735132+0000","last_unstale":"2024-09-20T22:23:30.952634+0000","last_undegraded":"2024-09-20T22:23:30.952634+0000","last_fullsized":"2024-09-20T22:23:30.952634+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":576,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:46.003147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:46.003147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:10.763248+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":82,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718480+0000","last_change":"2024-09-20T22:22:36.732933+0000","last_active":"2024-09-20T22:23:31.718480+0000","last_peered":"2024-09-20T22:23:31.718480+0000","last_clean":"2024-09-20T22:23:31.718480+0000","last_became_active":"2024-09-20T22:22:36.732366+0000","last_became_peered":"2024-09-20T22:22:36.732366+0000","last_unstale":"2024-09-20T22:23:31.718480+0000","last_undegraded":"2024-09-20T22:23:31.718480+0000","last_fullsized":"2024-09-20T22:23:31.718480+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:21.993777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:44.456415+0000","last_clean_scrub_stamp":"2024-09-20T22:22:21.993777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:49.795468+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"579'29","reported_seq":540,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952145+0000","last_change":"2024-09-20T22:22:51.338089+0000","last_active":"2024-09-20T22:23:30.952145+0000","last_peered":"2024-09-20T22:23:30.952145+0000","last_clean":"2024-09-20T22:23:30.952145+0000","last_became_active":"2024-09-20T22:22:51.337739+0000","last_became_peered":"2024-09-20T22:22:51.337739+0000","last_unstale":"2024-09-20T22:23:30.952145+0000","last_undegraded":"2024-09-20T22:23:30.952145+0000","last_fullsized":"2024-09-20T22:23:30.952145+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'23","last_scrub_stamp":"2024-09-20T22:22:10.697295+0000","last_deep_scrub":"250'17","last_deep_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_clean_scrub_stamp":"2024-09-20T22:22:10.697295+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:36.409961+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":84,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:32.814440+0000","last_change":"2024-09-20T22:23:32.814464+0000","last_active":"2024-09-20T22:23:32.814440+0000","last_peered":"2024-09-20T22:23:32.814440+0000","last_clean":"2024-09-20T22:23:32.814440+0000","last_became_active":"2024-09-20T22:22:34.756273+0000","last_became_peered":"2024-09-20T22:22:34.756273+0000","last_unstale":"2024-09-20T22:23:32.814440+0000","last_undegraded":"2024-09-20T22:23:32.814440+0000","last_fullsized":"2024-09-20T22:23:32.814440+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:32.814357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:23:32.814357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:41.502219+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1b","version":"75'128","reported_seq":1079,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062774+0000","last_change":"2024-09-20T22:22:37.725081+0000","last_active":"2024-09-20T22:23:31.062774+0000","last_peered":"2024-09-20T22:23:31.062774+0000","last_clean":"2024-09-20T22:23:31.062774+0000","last_became_active":"2024-09-20T22:22:37.724684+0000","last_became_peered":"2024-09-20T22:22:37.724684+0000","last_unstale":"2024-09-20T22:23:31.062774+0000","last_undegraded":"2024-09-20T22:23:31.062774+0000","last_fullsized":"2024-09-20T22:23:31.062774+0000","mapping_epoch":565,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":566,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:15.018011+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:22:15.018011+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:35.067218+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":1126,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.954339+0000","last_change":"2024-09-20T22:22:35.623837+0000","last_active":"2024-09-20T22:23:30.954339+0000","last_peered":"2024-09-20T22:23:30.954339+0000","last_clean":"2024-09-20T22:23:30.954339+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:23:30.954339+0000","last_undegraded":"2024-09-20T22:23:30.954339+0000","last_fullsized":"2024-09-20T22:23:30.954339+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:22:35.623755+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:22:35.623755+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:43.896324+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"289'2","reported_seq":77,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718934+0000","last_change":"2024-09-20T22:22:36.771205+0000","last_active":"2024-09-20T22:23:31.718934+0000","last_peered":"2024-09-20T22:23:31.718934+0000","last_clean":"2024-09-20T22:23:31.718934+0000","last_became_active":"2024-09-20T22:22:36.746942+0000","last_became_peered":"2024-09-20T22:22:36.746942+0000","last_unstale":"2024-09-20T22:23:31.718934+0000","last_undegraded":"2024-09-20T22:23:31.718934+0000","last_fullsized":"2024-09-20T22:23:31.718934+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-20T22:22:26.927516+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-20T22:22:26.927516+0000","last_clean_scrub_stamp":"2024-09-20T22:22:26.927516+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:47.270285+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":230,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"579'30","reported_seq":634,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.720297+0000","last_change":"2024-09-20T22:22:39.067136+0000","last_active":"2024-09-20T22:23:31.720297+0000","last_peered":"2024-09-20T22:23:31.720297+0000","last_clean":"2024-09-20T22:23:31.720297+0000","last_became_active":"2024-09-20T22:12:52.526803+0000","last_became_peered":"2024-09-20T22:12:52.526803+0000","last_unstale":"2024-09-20T22:23:31.720297+0000","last_undegraded":"2024-09-20T22:23:31.720297+0000","last_fullsized":"2024-09-20T22:23:31.720297+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:22:39.067058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:39.067058+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:51.512889+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"563'15","reported_seq":121,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.961414+0000","last_change":"2024-09-20T22:22:49.948730+0000","last_active":"2024-09-20T22:23:31.961414+0000","last_peered":"2024-09-20T22:23:31.961414+0000","last_clean":"2024-09-20T22:23:31.961414+0000","last_became_active":"2024-09-20T22:22:34.751023+0000","last_became_peered":"2024-09-20T22:22:34.751023+0000","last_unstale":"2024-09-20T22:23:31.961414+0000","last_undegraded":"2024-09-20T22:23:31.961414+0000","last_fullsized":"2024-09-20T22:23:31.961414+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"563'15","last_scrub_stamp":"2024-09-20T22:22:49.948627+0000","last_deep_scrub":"291'11","last_deep_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_clean_scrub_stamp":"2024-09-20T22:22:49.948627+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:05.343938+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":1170,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062483+0000","last_change":"2024-09-20T22:22:33.745681+0000","last_active":"2024-09-20T22:23:31.062483+0000","last_peered":"2024-09-20T22:23:31.062483+0000","last_clean":"2024-09-20T22:23:31.062483+0000","last_became_active":"2024-09-20T22:22:33.745348+0000","last_became_peered":"2024-09-20T22:22:33.745348+0000","last_unstale":"2024-09-20T22:23:31.062483+0000","last_undegraded":"2024-09-20T22:23:31.062483+0000","last_fullsized":"2024-09-20T22:23:31.062483+0000","mapping_epoch":561,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":562,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:13.011208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:13.011208+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:32.864474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"579'39","reported_seq":108,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.720191+0000","last_change":"2024-09-20T22:22:36.781358+0000","last_active":"2024-09-20T22:23:31.720191+0000","last_peered":"2024-09-20T22:23:31.720191+0000","last_clean":"2024-09-20T22:23:31.720191+0000","last_became_active":"2024-09-20T22:22:36.750028+0000","last_became_peered":"2024-09-20T22:22:36.750028+0000","last_unstale":"2024-09-20T22:23:31.720191+0000","last_undegraded":"2024-09-20T22:23:31.720191+0000","last_fullsized":"2024-09-20T22:23:31.720191+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:22:17.506272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:17.506272+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":51,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1186,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952055+0000","last_change":"2024-09-20T22:23:22.126785+0000","last_active":"2024-09-20T22:23:30.952055+0000","last_peered":"2024-09-20T22:23:30.952055+0000","last_clean":"2024-09-20T22:23:30.952055+0000","last_became_active":"2024-09-20T22:22:36.735340+0000","last_became_peered":"2024-09-20T22:22:36.735340+0000","last_unstale":"2024-09-20T22:23:30.952055+0000","last_undegraded":"2024-09-20T22:23:30.952055+0000","last_fullsized":"2024-09-20T22:23:30.952055+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:22.126699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:19:17.513228+0000","last_clean_scrub_stamp":"2024-09-20T22:23:22.126699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:42.027891+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"563'15","reported_seq":100,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719350+0000","last_change":"2024-09-20T22:22:36.728747+0000","last_active":"2024-09-20T22:23:31.719350+0000","last_peered":"2024-09-20T22:23:31.719350+0000","last_clean":"2024-09-20T22:23:31.719350+0000","last_became_active":"2024-09-20T22:22:34.741247+0000","last_became_peered":"2024-09-20T22:22:34.741247+0000","last_unstale":"2024-09-20T22:23:31.719350+0000","last_undegraded":"2024-09-20T22:23:31.719350+0000","last_fullsized":"2024-09-20T22:23:31.719350+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"563'15","last_scrub_stamp":"2024-09-20T22:22:36.728677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:36.728677+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:44.364115+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":695,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.063281+0000","last_change":"2024-09-20T22:22:36.742379+0000","last_active":"2024-09-20T22:23:31.063281+0000","last_peered":"2024-09-20T22:23:31.063281+0000","last_clean":"2024-09-20T22:23:31.063281+0000","last_became_active":"2024-09-20T22:22:36.741883+0000","last_became_peered":"2024-09-20T22:22:36.741883+0000","last_unstale":"2024-09-20T22:23:31.063281+0000","last_undegraded":"2024-09-20T22:23:31.063281+0000","last_fullsized":"2024-09-20T22:23:31.063281+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:24.110670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:24.110670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:49.044230+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":581,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719425+0000","last_change":"2024-09-20T22:23:05.709713+0000","last_active":"2024-09-20T22:23:31.719425+0000","last_peered":"2024-09-20T22:23:31.719425+0000","last_clean":"2024-09-20T22:23:31.719425+0000","last_became_active":"2024-09-20T22:22:37.719212+0000","last_became_peered":"2024-09-20T22:22:37.719212+0000","last_unstale":"2024-09-20T22:23:31.719425+0000","last_undegraded":"2024-09-20T22:23:31.719425+0000","last_fullsized":"2024-09-20T22:23:31.719425+0000","mapping_epoch":565,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":566,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:23:05.709618+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:23:05.709618+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:32.710626+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"579'42","reported_seq":655,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.951756+0000","last_change":"2024-09-20T22:22:53.008769+0000","last_active":"2024-09-20T22:23:30.951756+0000","last_peered":"2024-09-20T22:23:30.951756+0000","last_clean":"2024-09-20T22:23:30.951756+0000","last_became_active":"2024-09-20T22:22:51.342407+0000","last_became_peered":"2024-09-20T22:22:51.342407+0000","last_unstale":"2024-09-20T22:23:30.951756+0000","last_undegraded":"2024-09-20T22:23:30.951756+0000","last_fullsized":"2024-09-20T22:23:30.951756+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'30","last_scrub_stamp":"2024-09-20T22:22:53.008206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:53.008206+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:56.046438+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":57,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":584,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952888+0000","last_change":"2024-09-20T22:22:53.961979+0000","last_active":"2024-09-20T22:23:30.952888+0000","last_peered":"2024-09-20T22:23:30.952888+0000","last_clean":"2024-09-20T22:23:30.952888+0000","last_became_active":"2024-09-20T22:22:51.342358+0000","last_became_peered":"2024-09-20T22:22:51.342358+0000","last_unstale":"2024-09-20T22:23:30.952888+0000","last_undegraded":"2024-09-20T22:23:30.952888+0000","last_fullsized":"2024-09-20T22:23:30.952888+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:53.961902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:22:53.961902+0000","last_clean_scrub_stamp":"2024-09-20T22:22:53.961902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:55.672484+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":84,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718126+0000","last_change":"2024-09-20T22:23:29.798841+0000","last_active":"2024-09-20T22:23:31.718126+0000","last_peered":"2024-09-20T22:23:31.718126+0000","last_clean":"2024-09-20T22:23:31.718126+0000","last_became_active":"2024-09-20T22:22:34.752645+0000","last_became_peered":"2024-09-20T22:22:34.752645+0000","last_unstale":"2024-09-20T22:23:31.718126+0000","last_undegraded":"2024-09-20T22:23:31.718126+0000","last_fullsized":"2024-09-20T22:23:31.718126+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:29.798753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:23:29.798753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:46.871379+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":84,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719103+0000","last_change":"2024-09-20T22:23:03.778957+0000","last_active":"2024-09-20T22:23:31.719103+0000","last_peered":"2024-09-20T22:23:31.719103+0000","last_clean":"2024-09-20T22:23:31.719103+0000","last_became_active":"2024-09-20T22:22:34.748356+0000","last_became_peered":"2024-09-20T22:22:34.748356+0000","last_unstale":"2024-09-20T22:23:31.719103+0000","last_undegraded":"2024-09-20T22:23:31.719103+0000","last_fullsized":"2024-09-20T22:23:31.719103+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:03.778866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:23:03.778866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:08.518592+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":1202,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952100+0000","last_change":"2024-09-20T22:23:29.019475+0000","last_active":"2024-09-20T22:23:30.952100+0000","last_peered":"2024-09-20T22:23:30.952100+0000","last_clean":"2024-09-20T22:23:30.952100+0000","last_became_active":"2024-09-20T22:15:45.642671+0000","last_became_peered":"2024-09-20T22:15:45.642671+0000","last_unstale":"2024-09-20T22:23:30.952100+0000","last_undegraded":"2024-09-20T22:23:30.952100+0000","last_fullsized":"2024-09-20T22:23:30.952100+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:23:29.019401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:23:29.019401+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:52.994019+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"579'42","reported_seq":121,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718538+0000","last_change":"2024-09-20T22:22:55.742297+0000","last_active":"2024-09-20T22:23:31.718538+0000","last_peered":"2024-09-20T22:23:31.718538+0000","last_clean":"2024-09-20T22:23:31.718538+0000","last_became_active":"2024-09-20T22:22:34.744774+0000","last_became_peered":"2024-09-20T22:22:34.744774+0000","last_unstale":"2024-09-20T22:23:31.718538+0000","last_undegraded":"2024-09-20T22:23:31.718538+0000","last_fullsized":"2024-09-20T22:23:31.718538+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:22:55.742181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:55.742181+0000","objects_scrubbed":7,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:04.491052+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":55,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":643,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.720397+0000","last_change":"2024-09-20T22:22:56.184568+0000","last_active":"2024-09-20T22:23:31.720397+0000","last_peered":"2024-09-20T22:23:31.720397+0000","last_clean":"2024-09-20T22:23:31.720397+0000","last_became_active":"2024-09-20T22:22:36.748251+0000","last_became_peered":"2024-09-20T22:22:36.748251+0000","last_unstale":"2024-09-20T22:23:31.720397+0000","last_undegraded":"2024-09-20T22:23:31.720397+0000","last_fullsized":"2024-09-20T22:23:31.720397+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:22:56.184492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:56.184492+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:23.711516+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"575'16","reported_seq":724,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.951627+0000","last_change":"2024-09-20T22:22:56.994140+0000","last_active":"2024-09-20T22:23:30.951627+0000","last_peered":"2024-09-20T22:23:30.951627+0000","last_clean":"2024-09-20T22:23:30.951627+0000","last_became_active":"2024-09-20T22:22:51.338769+0000","last_became_peered":"2024-09-20T22:22:51.338769+0000","last_unstale":"2024-09-20T22:23:30.951627+0000","last_undegraded":"2024-09-20T22:23:30.951627+0000","last_fullsized":"2024-09-20T22:23:30.951627+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"575'16","last_scrub_stamp":"2024-09-20T22:22:56.994064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:56.994064+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:00.016540+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1137,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.954097+0000","last_change":"2024-09-20T22:22:59.568901+0000","last_active":"2024-09-20T22:23:30.954097+0000","last_peered":"2024-09-20T22:23:30.954097+0000","last_clean":"2024-09-20T22:23:30.954097+0000","last_became_active":"2024-09-20T22:22:30.672451+0000","last_became_peered":"2024-09-20T22:22:30.672451+0000","last_unstale":"2024-09-20T22:23:30.954097+0000","last_undegraded":"2024-09-20T22:23:30.954097+0000","last_fullsized":"2024-09-20T22:23:30.954097+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:59.568808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:59.568808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:12.462194+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":381,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.961297+0000","last_change":"2024-09-20T22:22:33.815459+0000","last_active":"2024-09-20T22:23:31.961297+0000","last_peered":"2024-09-20T22:23:31.961297+0000","last_clean":"2024-09-20T22:23:31.961297+0000","last_became_active":"2024-09-20T22:22:33.730147+0000","last_became_peered":"2024-09-20T22:22:33.730147+0000","last_unstale":"2024-09-20T22:23:31.961297+0000","last_undegraded":"2024-09-20T22:23:31.961297+0000","last_fullsized":"2024-09-20T22:23:31.961297+0000","mapping_epoch":561,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":562,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:33.815352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:33.815352+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:58.920874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"579'30","reported_seq":108,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719919+0000","last_change":"2024-09-20T22:23:04.233663+0000","last_active":"2024-09-20T22:23:31.719919+0000","last_peered":"2024-09-20T22:23:31.719919+0000","last_clean":"2024-09-20T22:23:31.719919+0000","last_became_active":"2024-09-20T22:22:36.748048+0000","last_became_peered":"2024-09-20T22:22:36.748048+0000","last_unstale":"2024-09-20T22:23:31.719919+0000","last_undegraded":"2024-09-20T22:23:31.719919+0000","last_fullsized":"2024-09-20T22:23:31.719919+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'30","last_scrub_stamp":"2024-09-20T22:23:04.233577+0000","last_deep_scrub":"579'30","last_deep_scrub_stamp":"2024-09-20T22:23:04.233577+0000","last_clean_scrub_stamp":"2024-09-20T22:23:04.233577+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:27.841823+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1160,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062885+0000","last_change":"2024-09-20T22:23:08.015717+0000","last_active":"2024-09-20T22:23:31.062885+0000","last_peered":"2024-09-20T22:23:31.062885+0000","last_clean":"2024-09-20T22:23:31.062885+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:23:31.062885+0000","last_undegraded":"2024-09-20T22:23:31.062885+0000","last_fullsized":"2024-09-20T22:23:31.062885+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:08.015622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:23:08.015622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:21.120384+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"558'15","reported_seq":742,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719952+0000","last_change":"2024-09-20T22:22:58.192796+0000","last_active":"2024-09-20T22:23:31.719952+0000","last_peered":"2024-09-20T22:23:31.719952+0000","last_clean":"2024-09-20T22:23:31.719952+0000","last_became_active":"2024-09-20T22:22:30.665240+0000","last_became_peered":"2024-09-20T22:22:30.665240+0000","last_unstale":"2024-09-20T22:23:31.719952+0000","last_undegraded":"2024-09-20T22:23:31.719952+0000","last_fullsized":"2024-09-20T22:23:31.719952+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"558'15","last_scrub_stamp":"2024-09-20T22:22:58.192698+0000","last_deep_scrub":"363'13","last_deep_scrub_stamp":"2024-09-20T22:14:52.612615+0000","last_clean_scrub_stamp":"2024-09-20T22:22:58.192698+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:23.572479+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":592,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.717837+0000","last_change":"2024-09-20T22:23:02.192742+0000","last_active":"2024-09-20T22:23:31.717837+0000","last_peered":"2024-09-20T22:23:31.717837+0000","last_clean":"2024-09-20T22:23:31.717837+0000","last_became_active":"2024-09-20T22:12:52.518529+0000","last_became_peered":"2024-09-20T22:12:52.518529+0000","last_unstale":"2024-09-20T22:23:31.717837+0000","last_undegraded":"2024-09-20T22:23:31.717837+0000","last_fullsized":"2024-09-20T22:23:31.717837+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:02.192649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:21:02.972172+0000","last_clean_scrub_stamp":"2024-09-20T22:23:02.192649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:23.377281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":686,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.960738+0000","last_change":"2024-09-20T22:22:33.747123+0000","last_active":"2024-09-20T22:23:31.960738+0000","last_peered":"2024-09-20T22:23:31.960738+0000","last_clean":"2024-09-20T22:23:31.960738+0000","last_became_active":"2024-09-20T22:22:33.746751+0000","last_became_peered":"2024-09-20T22:22:33.746751+0000","last_unstale":"2024-09-20T22:23:31.960738+0000","last_undegraded":"2024-09-20T22:23:31.960738+0000","last_fullsized":"2024-09-20T22:23:31.960738+0000","mapping_epoch":561,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":562,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:25.085071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:25.085071+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:51.350420+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"579'47","reported_seq":648,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719664+0000","last_change":"2024-09-20T22:22:21.039662+0000","last_active":"2024-09-20T22:23:31.719664+0000","last_peered":"2024-09-20T22:23:31.719664+0000","last_clean":"2024-09-20T22:23:31.719664+0000","last_became_active":"2024-09-20T22:12:52.525538+0000","last_became_peered":"2024-09-20T22:12:52.525538+0000","last_unstale":"2024-09-20T22:23:31.719664+0000","last_undegraded":"2024-09-20T22:23:31.719664+0000","last_fullsized":"2024-09-20T22:23:31.719664+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"392'37","last_scrub_stamp":"2024-09-20T22:22:21.039584+0000","last_deep_scrub":"392'37","last_deep_scrub_stamp":"2024-09-20T22:15:56.127540+0000","last_clean_scrub_stamp":"2024-09-20T22:22:21.039584+0000","objects_scrubbed":8,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:45.784068+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":68,"num_read_kb":54,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":73,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719168+0000","last_change":"2024-09-20T22:22:36.737910+0000","last_active":"2024-09-20T22:23:31.719168+0000","last_peered":"2024-09-20T22:23:31.719168+0000","last_clean":"2024-09-20T22:23:31.719168+0000","last_became_active":"2024-09-20T22:22:36.737560+0000","last_became_peered":"2024-09-20T22:22:36.737560+0000","last_unstale":"2024-09-20T22:23:31.719168+0000","last_undegraded":"2024-09-20T22:23:31.719168+0000","last_fullsized":"2024-09-20T22:23:31.719168+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:28.920875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:28.920875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:37.001115+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":94,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.960881+0000","last_change":"2024-09-20T22:22:56.022918+0000","last_active":"2024-09-20T22:23:31.960881+0000","last_peered":"2024-09-20T22:23:31.960881+0000","last_clean":"2024-09-20T22:23:31.960881+0000","last_became_active":"2024-09-20T22:22:30.678085+0000","last_became_peered":"2024-09-20T22:22:30.678085+0000","last_unstale":"2024-09-20T22:23:31.960881+0000","last_undegraded":"2024-09-20T22:23:31.960881+0000","last_fullsized":"2024-09-20T22:23:31.960881+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:56.022829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:22:56.022829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:59.190286+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":81,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718784+0000","last_change":"2024-09-20T22:23:29.164346+0000","last_active":"2024-09-20T22:23:31.718784+0000","last_peered":"2024-09-20T22:23:31.718784+0000","last_clean":"2024-09-20T22:23:31.718784+0000","last_became_active":"2024-09-20T22:22:36.729183+0000","last_became_peered":"2024-09-20T22:22:36.729183+0000","last_unstale":"2024-09-20T22:23:31.718784+0000","last_undegraded":"2024-09-20T22:23:31.718784+0000","last_fullsized":"2024-09-20T22:23:31.718784+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:29.164246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:23:29.164246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:47.523982+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":1171,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062326+0000","last_change":"2024-09-20T22:22:56.865975+0000","last_active":"2024-09-20T22:23:31.062326+0000","last_peered":"2024-09-20T22:23:31.062326+0000","last_clean":"2024-09-20T22:23:31.062326+0000","last_became_active":"2024-09-20T22:15:45.627135+0000","last_became_peered":"2024-09-20T22:15:45.627135+0000","last_unstale":"2024-09-20T22:23:31.062326+0000","last_undegraded":"2024-09-20T22:23:31.062326+0000","last_fullsized":"2024-09-20T22:23:31.062326+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:56.865872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:56.865872+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:17.244735+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"579'68","reported_seq":133,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718815+0000","last_change":"2024-09-20T22:23:17.244337+0000","last_active":"2024-09-20T22:23:31.718815+0000","last_peered":"2024-09-20T22:23:31.718815+0000","last_clean":"2024-09-20T22:23:31.718815+0000","last_became_active":"2024-09-20T22:22:36.732949+0000","last_became_peered":"2024-09-20T22:22:36.732949+0000","last_unstale":"2024-09-20T22:23:31.718815+0000","last_undegraded":"2024-09-20T22:23:31.718815+0000","last_fullsized":"2024-09-20T22:23:31.718815+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'68","last_scrub_stamp":"2024-09-20T22:23:17.244238+0000","last_deep_scrub":"328'52","last_deep_scrub_stamp":"2024-09-20T22:17:08.395578+0000","last_clean_scrub_stamp":"2024-09-20T22:23:17.244238+0000","objects_scrubbed":10,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:19.448842+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":82,"num_read_kb":67,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":84,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719177+0000","last_change":"2024-09-20T22:23:13.750879+0000","last_active":"2024-09-20T22:23:31.719177+0000","last_peered":"2024-09-20T22:23:31.719177+0000","last_clean":"2024-09-20T22:23:31.719177+0000","last_became_active":"2024-09-20T22:22:34.745230+0000","last_became_peered":"2024-09-20T22:22:34.745230+0000","last_unstale":"2024-09-20T22:23:31.719177+0000","last_undegraded":"2024-09-20T22:23:31.719177+0000","last_fullsized":"2024-09-20T22:23:31.719177+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:13.750775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:23:13.750775+0000","last_clean_scrub_stamp":"2024-09-20T22:23:13.750775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:34.016719+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":1196,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062756+0000","last_change":"2024-09-20T22:23:11.052803+0000","last_active":"2024-09-20T22:23:31.062756+0000","last_peered":"2024-09-20T22:23:31.062756+0000","last_clean":"2024-09-20T22:23:31.062756+0000","last_became_active":"2024-09-20T22:22:36.744910+0000","last_became_peered":"2024-09-20T22:22:36.744910+0000","last_unstale":"2024-09-20T22:23:31.062756+0000","last_undegraded":"2024-09-20T22:23:31.062756+0000","last_fullsized":"2024-09-20T22:23:31.062756+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:23:11.052708+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_clean_scrub_stamp":"2024-09-20T22:23:11.052708+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:20.926715+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":628,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719839+0000","last_change":"2024-09-20T22:22:36.746533+0000","last_active":"2024-09-20T22:23:31.719839+0000","last_peered":"2024-09-20T22:23:31.719839+0000","last_clean":"2024-09-20T22:23:31.719839+0000","last_became_active":"2024-09-20T22:22:36.746280+0000","last_became_peered":"2024-09-20T22:22:36.746280+0000","last_unstale":"2024-09-20T22:23:31.719839+0000","last_undegraded":"2024-09-20T22:23:31.719839+0000","last_fullsized":"2024-09-20T22:23:31.719839+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:31.035693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:31.035693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:55.917496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1169,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952434+0000","last_change":"2024-09-20T22:22:34.756279+0000","last_active":"2024-09-20T22:23:30.952434+0000","last_peered":"2024-09-20T22:23:30.952434+0000","last_clean":"2024-09-20T22:23:30.952434+0000","last_became_active":"2024-09-20T22:22:34.755781+0000","last_became_peered":"2024-09-20T22:22:34.755781+0000","last_unstale":"2024-09-20T22:23:30.952434+0000","last_undegraded":"2024-09-20T22:23:30.952434+0000","last_fullsized":"2024-09-20T22:23:30.952434+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:23.018760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:22:23.018760+0000","last_clean_scrub_stamp":"2024-09-20T22:22:23.018760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:43.284624+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":1186,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.953659+0000","last_change":"2024-09-20T22:22:37.721200+0000","last_active":"2024-09-20T22:23:30.953659+0000","last_peered":"2024-09-20T22:23:30.953659+0000","last_clean":"2024-09-20T22:23:30.953659+0000","last_became_active":"2024-09-20T22:22:37.720715+0000","last_became_peered":"2024-09-20T22:22:37.720715+0000","last_unstale":"2024-09-20T22:23:30.953659+0000","last_undegraded":"2024-09-20T22:23:30.953659+0000","last_fullsized":"2024-09-20T22:23:30.953659+0000","mapping_epoch":565,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":566,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:29.535404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:29.535404+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:47.788568+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"579'21","reported_seq":31,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.957876+0000","last_change":"2024-09-20T22:23:23.838224+0000","last_active":"2024-09-20T22:23:31.957876+0000","last_peered":"2024-09-20T22:23:31.957876+0000","last_clean":"2024-09-20T22:23:31.957876+0000","last_became_active":"2024-09-20T22:23:23.829978+0000","last_became_peered":"2024-09-20T22:23:23.829978+0000","last_unstale":"2024-09-20T22:23:31.957876+0000","last_undegraded":"2024-09-20T22:23:31.957876+0000","last_fullsized":"2024-09-20T22:23:31.957876+0000","mapping_epoch":591,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":592,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'21","last_scrub_stamp":"2024-09-20T22:23:08.286952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:23:08.286952+0000","objects_scrubbed":5,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:23.970165+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":39,"num_read_kb":28,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":81,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:32.073109+0000","last_change":"2024-09-20T22:23:32.073127+0000","last_active":"2024-09-20T22:23:32.073109+0000","last_peered":"2024-09-20T22:23:32.073109+0000","last_clean":"2024-09-20T22:23:32.073109+0000","last_became_active":"2024-09-20T22:22:36.742021+0000","last_became_peered":"2024-09-20T22:22:36.742021+0000","last_unstale":"2024-09-20T22:23:32.073109+0000","last_undegraded":"2024-09-20T22:23:32.073109+0000","last_fullsized":"2024-09-20T22:23:32.073109+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:32.073057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:23:32.073057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:43.993378+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":1208,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959409+0000","last_change":"2024-09-20T22:22:33.052302+0000","last_active":"2024-09-20T22:23:31.959409+0000","last_peered":"2024-09-20T22:23:31.959409+0000","last_clean":"2024-09-20T22:23:31.959409+0000","last_became_active":"2024-09-20T22:12:52.517255+0000","last_became_peered":"2024-09-20T22:12:52.517255+0000","last_unstale":"2024-09-20T22:23:31.959409+0000","last_undegraded":"2024-09-20T22:23:31.959409+0000","last_fullsized":"2024-09-20T22:23:31.959409+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:22:33.052225+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:22:33.052225+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:57.127668+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1162,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959462+0000","last_change":"2024-09-20T22:22:37.055462+0000","last_active":"2024-09-20T22:23:31.959462+0000","last_peered":"2024-09-20T22:23:31.959462+0000","last_clean":"2024-09-20T22:23:31.959462+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:23:31.959462+0000","last_undegraded":"2024-09-20T22:23:31.959462+0000","last_fullsized":"2024-09-20T22:23:31.959462+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:37.055358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:22:37.055358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:03.817284+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1137,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.954609+0000","last_change":"2024-09-20T22:23:01.609407+0000","last_active":"2024-09-20T22:23:30.954609+0000","last_peered":"2024-09-20T22:23:30.954609+0000","last_clean":"2024-09-20T22:23:30.954609+0000","last_became_active":"2024-09-20T22:22:30.673317+0000","last_became_peered":"2024-09-20T22:22:30.673317+0000","last_unstale":"2024-09-20T22:23:30.954609+0000","last_undegraded":"2024-09-20T22:23:30.954609+0000","last_fullsized":"2024-09-20T22:23:30.954609+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:01.609306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:23:01.609306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:12.576775+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":375,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.717967+0000","last_change":"2024-09-20T22:22:40.124825+0000","last_active":"2024-09-20T22:23:31.717967+0000","last_peered":"2024-09-20T22:23:31.717967+0000","last_clean":"2024-09-20T22:23:31.717967+0000","last_became_active":"2024-09-20T22:22:39.716276+0000","last_became_peered":"2024-09-20T22:22:39.716276+0000","last_unstale":"2024-09-20T22:23:31.717967+0000","last_undegraded":"2024-09-20T22:23:31.717967+0000","last_fullsized":"2024-09-20T22:23:31.717967+0000","mapping_epoch":567,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":568,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:40.124745+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:22:40.124745+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:52.106218+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"579'36","reported_seq":681,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.720377+0000","last_change":"2024-09-20T22:22:36.784115+0000","last_active":"2024-09-20T22:23:31.720377+0000","last_peered":"2024-09-20T22:23:31.720377+0000","last_clean":"2024-09-20T22:23:31.720377+0000","last_became_active":"2024-09-20T22:22:36.748288+0000","last_became_peered":"2024-09-20T22:22:36.748288+0000","last_unstale":"2024-09-20T22:23:31.720377+0000","last_undegraded":"2024-09-20T22:23:31.720377+0000","last_fullsized":"2024-09-20T22:23:31.720377+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'24","last_scrub_stamp":"2024-09-20T22:22:03.105662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:03.105662+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":47,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":612,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718272+0000","last_change":"2024-09-20T22:23:31.213672+0000","last_active":"2024-09-20T22:23:31.718272+0000","last_peered":"2024-09-20T22:23:31.718272+0000","last_clean":"2024-09-20T22:23:31.718272+0000","last_became_active":"2024-09-20T22:23:10.778296+0000","last_became_peered":"2024-09-20T22:23:10.778296+0000","last_unstale":"2024-09-20T22:23:31.718272+0000","last_undegraded":"2024-09-20T22:23:31.718272+0000","last_fullsized":"2024-09-20T22:23:31.718272+0000","mapping_epoch":587,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":588,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:31.213596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:23:31.213596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:38.208077+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":2420,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.954061+0000","last_change":"2024-09-20T22:22:39.610305+0000","last_active":"2024-09-20T22:23:30.954061+0000","last_peered":"2024-09-20T22:23:30.954061+0000","last_clean":"2024-09-20T22:23:30.954061+0000","last_became_active":"2024-09-20T22:22:36.743951+0000","last_became_peered":"2024-09-20T22:22:36.743951+0000","last_unstale":"2024-09-20T22:23:30.954061+0000","last_undegraded":"2024-09-20T22:23:30.954061+0000","last_fullsized":"2024-09-20T22:23:30.954061+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:22:39.610209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:22:39.610209+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:41.985811+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1211,"num_read_kb":1206,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":701,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.063092+0000","last_change":"2024-09-20T22:22:36.744877+0000","last_active":"2024-09-20T22:23:31.063092+0000","last_peered":"2024-09-20T22:23:31.063092+0000","last_clean":"2024-09-20T22:23:31.063092+0000","last_became_active":"2024-09-20T22:22:36.744548+0000","last_became_peered":"2024-09-20T22:22:36.744548+0000","last_unstale":"2024-09-20T22:23:31.063092+0000","last_undegraded":"2024-09-20T22:23:31.063092+0000","last_fullsized":"2024-09-20T22:23:31.063092+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:06.156848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:17:08.955790+0000","last_clean_scrub_stamp":"2024-09-20T22:22:06.156848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:34.908139+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":81,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718971+0000","last_change":"2024-09-20T22:23:02.210754+0000","last_active":"2024-09-20T22:23:31.718971+0000","last_peered":"2024-09-20T22:23:31.718971+0000","last_clean":"2024-09-20T22:23:31.718971+0000","last_became_active":"2024-09-20T22:22:36.736380+0000","last_became_peered":"2024-09-20T22:22:36.736380+0000","last_unstale":"2024-09-20T22:23:31.718971+0000","last_undegraded":"2024-09-20T22:23:31.718971+0000","last_fullsized":"2024-09-20T22:23:31.718971+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:02.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:40.325308+0000","last_clean_scrub_stamp":"2024-09-20T22:23:02.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:25.771253+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"579'44","reported_seq":159,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.961041+0000","last_change":"2024-09-20T22:22:34.778254+0000","last_active":"2024-09-20T22:23:31.961041+0000","last_peered":"2024-09-20T22:23:31.961041+0000","last_clean":"2024-09-20T22:23:31.961041+0000","last_became_active":"2024-09-20T22:22:34.748021+0000","last_became_peered":"2024-09-20T22:22:34.748021+0000","last_unstale":"2024-09-20T22:23:31.961041+0000","last_undegraded":"2024-09-20T22:23:31.961041+0000","last_fullsized":"2024-09-20T22:23:31.961041+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"475'29","last_scrub_stamp":"2024-09-20T22:22:14.737957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:14.737957+0000","objects_scrubbed":8,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:41.499501+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":61,"num_read_kb":51,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1078,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952141+0000","last_change":"2024-09-20T22:23:00.051665+0000","last_active":"2024-09-20T22:23:30.952141+0000","last_peered":"2024-09-20T22:23:30.952141+0000","last_clean":"2024-09-20T22:23:30.952141+0000","last_became_active":"2024-09-20T22:22:51.734926+0000","last_became_peered":"2024-09-20T22:22:51.734926+0000","last_unstale":"2024-09-20T22:23:30.952141+0000","last_undegraded":"2024-09-20T22:23:30.952141+0000","last_fullsized":"2024-09-20T22:23:30.952141+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":576,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:00.051547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:23:00.051547+0000","last_clean_scrub_stamp":"2024-09-20T22:23:00.051547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:25.256549+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":1186,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.953545+0000","last_change":"2024-09-20T22:22:45.629159+0000","last_active":"2024-09-20T22:23:30.953545+0000","last_peered":"2024-09-20T22:23:30.953545+0000","last_clean":"2024-09-20T22:23:30.953545+0000","last_became_active":"2024-09-20T22:22:33.742983+0000","last_became_peered":"2024-09-20T22:22:33.742983+0000","last_unstale":"2024-09-20T22:23:30.953545+0000","last_undegraded":"2024-09-20T22:23:30.953545+0000","last_fullsized":"2024-09-20T22:23:30.953545+0000","mapping_epoch":561,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":562,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:45.629061+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:18:06.776546+0000","last_clean_scrub_stamp":"2024-09-20T22:22:45.629061+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:46.807860+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":720,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.954525+0000","last_change":"2024-09-20T22:22:48.698028+0000","last_active":"2024-09-20T22:23:30.954525+0000","last_peered":"2024-09-20T22:23:30.954525+0000","last_clean":"2024-09-20T22:23:30.954525+0000","last_became_active":"2024-09-20T22:10:48.907271+0000","last_became_peered":"2024-09-20T22:10:48.907271+0000","last_unstale":"2024-09-20T22:23:30.954525+0000","last_undegraded":"2024-09-20T22:23:30.954525+0000","last_fullsized":"2024-09-20T22:23:30.954525+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:48.697944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_clean_scrub_stamp":"2024-09-20T22:22:48.697944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:08.599761+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1170,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.063250+0000","last_change":"2024-09-20T22:23:07.023515+0000","last_active":"2024-09-20T22:23:31.063250+0000","last_peered":"2024-09-20T22:23:31.063250+0000","last_clean":"2024-09-20T22:23:31.063250+0000","last_became_active":"2024-09-20T22:22:34.748659+0000","last_became_peered":"2024-09-20T22:22:34.748659+0000","last_unstale":"2024-09-20T22:23:31.063250+0000","last_undegraded":"2024-09-20T22:23:31.063250+0000","last_fullsized":"2024-09-20T22:23:31.063250+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:07.023412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:23:07.023412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:35.787281+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"75'128","reported_seq":967,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718440+0000","last_change":"2024-09-20T22:23:06.300361+0000","last_active":"2024-09-20T22:23:31.718440+0000","last_peered":"2024-09-20T22:23:31.718440+0000","last_clean":"2024-09-20T22:23:31.718440+0000","last_became_active":"2024-09-20T22:22:39.717701+0000","last_became_peered":"2024-09-20T22:22:39.717701+0000","last_unstale":"2024-09-20T22:23:31.718440+0000","last_undegraded":"2024-09-20T22:23:31.718440+0000","last_fullsized":"2024-09-20T22:23:31.718440+0000","mapping_epoch":567,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":568,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:23:06.300224+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:23:06.300224+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:23.574749+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"579'27","reported_seq":144,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719423+0000","last_change":"2024-09-20T22:23:23.837874+0000","last_active":"2024-09-20T22:23:31.719423+0000","last_peered":"2024-09-20T22:23:31.719423+0000","last_clean":"2024-09-20T22:23:31.719423+0000","last_became_active":"2024-09-20T22:23:23.829700+0000","last_became_peered":"2024-09-20T22:23:23.829700+0000","last_unstale":"2024-09-20T22:23:31.719423+0000","last_undegraded":"2024-09-20T22:23:31.719423+0000","last_fullsized":"2024-09-20T22:23:31.719423+0000","mapping_epoch":591,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":592,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:22:37.724251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:37.724251+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:38.459394+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":84,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718717+0000","last_change":"2024-09-20T22:23:21.646272+0000","last_active":"2024-09-20T22:23:31.718717+0000","last_peered":"2024-09-20T22:23:31.718717+0000","last_clean":"2024-09-20T22:23:31.718717+0000","last_became_active":"2024-09-20T22:22:34.742470+0000","last_became_peered":"2024-09-20T22:22:34.742470+0000","last_unstale":"2024-09-20T22:23:31.718717+0000","last_undegraded":"2024-09-20T22:23:31.718717+0000","last_fullsized":"2024-09-20T22:23:31.718717+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:21.646189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:23:21.646189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:23.057280+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1168,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.063422+0000","last_change":"2024-09-20T22:23:02.000751+0000","last_active":"2024-09-20T22:23:31.063422+0000","last_peered":"2024-09-20T22:23:31.063422+0000","last_clean":"2024-09-20T22:23:31.063422+0000","last_became_active":"2024-09-20T22:22:34.748654+0000","last_became_peered":"2024-09-20T22:22:34.748654+0000","last_unstale":"2024-09-20T22:23:31.063422+0000","last_undegraded":"2024-09-20T22:23:31.063422+0000","last_fullsized":"2024-09-20T22:23:31.063422+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:02.000619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:23:02.000619+0000","last_clean_scrub_stamp":"2024-09-20T22:23:02.000619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:09.628446+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":577,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.951974+0000","last_change":"2024-09-20T22:22:51.735316+0000","last_active":"2024-09-20T22:23:30.951974+0000","last_peered":"2024-09-20T22:23:30.951974+0000","last_clean":"2024-09-20T22:23:30.951974+0000","last_became_active":"2024-09-20T22:22:51.734972+0000","last_became_peered":"2024-09-20T22:22:51.734972+0000","last_unstale":"2024-09-20T22:23:30.951974+0000","last_undegraded":"2024-09-20T22:23:30.951974+0000","last_fullsized":"2024-09-20T22:23:30.951974+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":576,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:17.690638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:17.690638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":498,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.960135+0000","last_change":"2024-09-20T22:22:34.836902+0000","last_active":"2024-09-20T22:23:31.960135+0000","last_peered":"2024-09-20T22:23:31.960135+0000","last_clean":"2024-09-20T22:23:31.960135+0000","last_became_active":"2024-09-20T22:22:33.732055+0000","last_became_peered":"2024-09-20T22:22:33.732055+0000","last_unstale":"2024-09-20T22:23:31.960135+0000","last_undegraded":"2024-09-20T22:23:31.960135+0000","last_fullsized":"2024-09-20T22:23:31.960135+0000","mapping_epoch":561,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":562,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:34.836783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:34.836783+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:51.889111+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"579'45","reported_seq":147,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719264+0000","last_change":"2024-09-20T22:22:43.690865+0000","last_active":"2024-09-20T22:23:31.719264+0000","last_peered":"2024-09-20T22:23:31.719264+0000","last_clean":"2024-09-20T22:23:31.719264+0000","last_became_active":"2024-09-20T22:22:36.733426+0000","last_became_peered":"2024-09-20T22:22:36.733426+0000","last_unstale":"2024-09-20T22:23:31.719264+0000","last_undegraded":"2024-09-20T22:23:31.719264+0000","last_fullsized":"2024-09-20T22:23:31.719264+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'31","last_scrub_stamp":"2024-09-20T22:22:43.690780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:43.690780+0000","objects_scrubbed":8,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:02.781256+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":60,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":106,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.961081+0000","last_change":"2024-09-20T22:23:01.036132+0000","last_active":"2024-09-20T22:23:31.961081+0000","last_peered":"2024-09-20T22:23:31.961081+0000","last_clean":"2024-09-20T22:23:31.961081+0000","last_became_active":"2024-09-20T22:22:36.741709+0000","last_became_peered":"2024-09-20T22:22:36.741709+0000","last_unstale":"2024-09-20T22:23:31.961081+0000","last_undegraded":"2024-09-20T22:23:31.961081+0000","last_fullsized":"2024-09-20T22:23:31.961081+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:01.036043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:17:08.846904+0000","last_clean_scrub_stamp":"2024-09-20T22:23:01.036043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:25.318457+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":81,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719067+0000","last_change":"2024-09-20T22:22:41.094622+0000","last_active":"2024-09-20T22:23:31.719067+0000","last_peered":"2024-09-20T22:23:31.719067+0000","last_clean":"2024-09-20T22:23:31.719067+0000","last_became_active":"2024-09-20T22:22:36.747619+0000","last_became_peered":"2024-09-20T22:22:36.747619+0000","last_unstale":"2024-09-20T22:23:31.719067+0000","last_undegraded":"2024-09-20T22:23:31.719067+0000","last_fullsized":"2024-09-20T22:23:31.719067+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:41.094548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:10.179939+0000","last_clean_scrub_stamp":"2024-09-20T22:22:41.094548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:48.330041+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1182,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959567+0000","last_change":"2024-09-20T22:23:30.843127+0000","last_active":"2024-09-20T22:23:31.959567+0000","last_peered":"2024-09-20T22:23:31.959567+0000","last_clean":"2024-09-20T22:23:31.959567+0000","last_became_active":"2024-09-20T22:22:34.737506+0000","last_became_peered":"2024-09-20T22:22:34.737506+0000","last_unstale":"2024-09-20T22:23:31.959567+0000","last_undegraded":"2024-09-20T22:23:31.959567+0000","last_fullsized":"2024-09-20T22:23:31.959567+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:30.843055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:23:30.843055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:56.617968+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":1437,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.953661+0000","last_change":"2024-09-20T22:23:26.502347+0000","last_active":"2024-09-20T22:23:30.953661+0000","last_peered":"2024-09-20T22:23:30.953661+0000","last_clean":"2024-09-20T22:23:30.953661+0000","last_became_active":"2024-09-20T22:17:40.146794+0000","last_became_peered":"2024-09-20T22:17:40.146794+0000","last_unstale":"2024-09-20T22:23:30.953661+0000","last_undegraded":"2024-09-20T22:23:30.953661+0000","last_fullsized":"2024-09-20T22:23:30.953661+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:23:26.502246+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:23:26.502246+0000","last_clean_scrub_stamp":"2024-09-20T22:23:26.502246+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:27.150554+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"579'35","reported_seq":111,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718576+0000","last_change":"2024-09-20T22:23:15.706930+0000","last_active":"2024-09-20T22:23:31.718576+0000","last_peered":"2024-09-20T22:23:31.718576+0000","last_clean":"2024-09-20T22:23:31.718576+0000","last_became_active":"2024-09-20T22:22:34.753666+0000","last_became_peered":"2024-09-20T22:22:34.753666+0000","last_unstale":"2024-09-20T22:23:31.718576+0000","last_undegraded":"2024-09-20T22:23:31.718576+0000","last_fullsized":"2024-09-20T22:23:31.718576+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'35","last_scrub_stamp":"2024-09-20T22:23:15.706837+0000","last_deep_scrub":"328'25","last_deep_scrub_stamp":"2024-09-20T22:18:05.300862+0000","last_clean_scrub_stamp":"2024-09-20T22:23:15.706837+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:29.869686+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":600,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952634+0000","last_change":"2024-09-20T22:23:26.140556+0000","last_active":"2024-09-20T22:23:30.952634+0000","last_peered":"2024-09-20T22:23:30.952634+0000","last_clean":"2024-09-20T22:23:30.952634+0000","last_became_active":"2024-09-20T22:22:51.337688+0000","last_became_peered":"2024-09-20T22:22:51.337688+0000","last_unstale":"2024-09-20T22:23:30.952634+0000","last_undegraded":"2024-09-20T22:23:30.952634+0000","last_fullsized":"2024-09-20T22:23:30.952634+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:23:26.140475+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:23:26.140475+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:49.059487+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"283'11","reported_seq":1448,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959018+0000","last_change":"2024-09-20T22:22:34.002553+0000","last_active":"2024-09-20T22:23:31.959018+0000","last_peered":"2024-09-20T22:23:31.959018+0000","last_clean":"2024-09-20T22:23:31.959018+0000","last_became_active":"2024-09-20T22:11:30.485356+0000","last_became_peered":"2024-09-20T22:11:30.485356+0000","last_unstale":"2024-09-20T22:23:31.959018+0000","last_undegraded":"2024-09-20T22:23:31.959018+0000","last_fullsized":"2024-09-20T22:23:31.959018+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'11","last_scrub_stamp":"2024-09-20T22:22:34.002442+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:22:34.002442+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:59.619774+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":90,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719021+0000","last_change":"2024-09-20T22:22:45.706731+0000","last_active":"2024-09-20T22:23:31.719021+0000","last_peered":"2024-09-20T22:23:31.719021+0000","last_clean":"2024-09-20T22:23:31.719021+0000","last_became_active":"2024-09-20T22:22:36.732067+0000","last_became_peered":"2024-09-20T22:22:36.732067+0000","last_unstale":"2024-09-20T22:23:31.719021+0000","last_undegraded":"2024-09-20T22:23:31.719021+0000","last_fullsized":"2024-09-20T22:23:31.719021+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:45.706636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:22:45.706636+0000","last_clean_scrub_stamp":"2024-09-20T22:22:45.706636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:10.120861+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"75'128","reported_seq":659,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719195+0000","last_change":"2024-09-20T22:22:38.702591+0000","last_active":"2024-09-20T22:23:31.719195+0000","last_peered":"2024-09-20T22:23:31.719195+0000","last_clean":"2024-09-20T22:23:31.719195+0000","last_became_active":"2024-09-20T22:22:37.717414+0000","last_became_peered":"2024-09-20T22:22:37.717414+0000","last_unstale":"2024-09-20T22:23:31.719195+0000","last_undegraded":"2024-09-20T22:23:31.719195+0000","last_fullsized":"2024-09-20T22:23:31.719195+0000","mapping_epoch":565,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":566,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:38.702498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:38.702498+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:54.154505+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"579'25","reported_seq":695,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719547+0000","last_change":"2024-09-20T22:23:04.200554+0000","last_active":"2024-09-20T22:23:31.719547+0000","last_peered":"2024-09-20T22:23:31.719547+0000","last_clean":"2024-09-20T22:23:31.719547+0000","last_became_active":"2024-09-20T22:22:36.750171+0000","last_became_peered":"2024-09-20T22:22:36.750171+0000","last_unstale":"2024-09-20T22:23:31.719547+0000","last_undegraded":"2024-09-20T22:23:31.719547+0000","last_fullsized":"2024-09-20T22:23:31.719547+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'25","last_scrub_stamp":"2024-09-20T22:23:04.200459+0000","last_deep_scrub":"512'19","last_deep_scrub_stamp":"2024-09-20T22:21:05.960284+0000","last_clean_scrub_stamp":"2024-09-20T22:23:04.200459+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:31.706993+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":622,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952772+0000","last_change":"2024-09-20T22:22:37.910548+0000","last_active":"2024-09-20T22:23:30.952772+0000","last_peered":"2024-09-20T22:23:30.952772+0000","last_clean":"2024-09-20T22:23:30.952772+0000","last_became_active":"2024-09-20T22:22:36.736413+0000","last_became_peered":"2024-09-20T22:22:36.736413+0000","last_unstale":"2024-09-20T22:23:30.952772+0000","last_undegraded":"2024-09-20T22:23:30.952772+0000","last_fullsized":"2024-09-20T22:23:30.952772+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:22:37.910471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:37.910471+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:05.238762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1174,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.958622+0000","last_change":"2024-09-20T22:22:34.738023+0000","last_active":"2024-09-20T22:23:31.958622+0000","last_peered":"2024-09-20T22:23:31.958622+0000","last_clean":"2024-09-20T22:23:31.958622+0000","last_became_active":"2024-09-20T22:22:34.737563+0000","last_became_peered":"2024-09-20T22:22:34.737563+0000","last_unstale":"2024-09-20T22:23:31.958622+0000","last_undegraded":"2024-09-20T22:23:31.958622+0000","last_fullsized":"2024-09-20T22:23:31.958622+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:25.949839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:22:25.949839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:40.478352+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":597,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718181+0000","last_change":"2024-09-20T22:22:57.193074+0000","last_active":"2024-09-20T22:23:31.718181+0000","last_peered":"2024-09-20T22:23:31.718181+0000","last_clean":"2024-09-20T22:23:31.718181+0000","last_became_active":"2024-09-20T22:19:45.813003+0000","last_became_peered":"2024-09-20T22:19:45.813003+0000","last_unstale":"2024-09-20T22:23:31.718181+0000","last_undegraded":"2024-09-20T22:23:31.718181+0000","last_fullsized":"2024-09-20T22:23:31.718181+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:57.192999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:22:57.192999+0000","last_clean_scrub_stamp":"2024-09-20T22:22:57.192999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:59.657894+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"75'128","reported_seq":828,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.961036+0000","last_change":"2024-09-20T22:22:37.907747+0000","last_active":"2024-09-20T22:23:31.961036+0000","last_peered":"2024-09-20T22:23:31.961036+0000","last_clean":"2024-09-20T22:23:31.961036+0000","last_became_active":"2024-09-20T22:22:37.719520+0000","last_became_peered":"2024-09-20T22:22:37.719520+0000","last_unstale":"2024-09-20T22:23:31.961036+0000","last_undegraded":"2024-09-20T22:23:31.961036+0000","last_fullsized":"2024-09-20T22:23:31.961036+0000","mapping_epoch":565,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":566,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:37.907659+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:17:07.885346+0000","last_clean_scrub_stamp":"2024-09-20T22:22:37.907659+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:58.920652+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"579'51","reported_seq":164,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.960881+0000","last_change":"2024-09-20T22:23:07.072101+0000","last_active":"2024-09-20T22:23:31.960881+0000","last_peered":"2024-09-20T22:23:31.960881+0000","last_clean":"2024-09-20T22:23:31.960881+0000","last_became_active":"2024-09-20T22:22:36.742973+0000","last_became_peered":"2024-09-20T22:22:36.742973+0000","last_unstale":"2024-09-20T22:23:31.960881+0000","last_undegraded":"2024-09-20T22:23:31.960881+0000","last_fullsized":"2024-09-20T22:23:31.960881+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'51","last_scrub_stamp":"2024-09-20T22:23:07.071970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:23:07.071970+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:21.049002+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":79,"num_read_kb":68,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1161,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.954786+0000","last_change":"2024-09-20T22:22:30.676058+0000","last_active":"2024-09-20T22:23:30.954786+0000","last_peered":"2024-09-20T22:23:30.954786+0000","last_clean":"2024-09-20T22:23:30.954786+0000","last_became_active":"2024-09-20T22:22:30.675715+0000","last_became_peered":"2024-09-20T22:22:30.675715+0000","last_unstale":"2024-09-20T22:23:30.954786+0000","last_undegraded":"2024-09-20T22:23:30.954786+0000","last_fullsized":"2024-09-20T22:23:30.954786+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:22.523926+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:12:46.602035+0000","last_clean_scrub_stamp":"2024-09-20T22:22:22.523926+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1186,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.958803+0000","last_change":"2024-09-20T22:22:19.052616+0000","last_active":"2024-09-20T22:23:31.958803+0000","last_peered":"2024-09-20T22:23:31.958803+0000","last_clean":"2024-09-20T22:23:31.958803+0000","last_became_active":"2024-09-20T22:12:52.515295+0000","last_became_peered":"2024-09-20T22:12:52.515295+0000","last_unstale":"2024-09-20T22:23:31.958803+0000","last_undegraded":"2024-09-20T22:23:31.958803+0000","last_fullsized":"2024-09-20T22:23:31.958803+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:19.052544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_clean_scrub_stamp":"2024-09-20T22:22:19.052544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:39.745768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":742,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.953699+0000","last_change":"2024-09-20T22:23:03.698199+0000","last_active":"2024-09-20T22:23:30.953699+0000","last_peered":"2024-09-20T22:23:30.953699+0000","last_clean":"2024-09-20T22:23:30.953699+0000","last_became_active":"2024-09-20T22:10:30.657116+0000","last_became_peered":"2024-09-20T22:10:30.657116+0000","last_unstale":"2024-09-20T22:23:30.953699+0000","last_undegraded":"2024-09-20T22:23:30.953699+0000","last_fullsized":"2024-09-20T22:23:30.953699+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:03.698074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:23:03.698074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:22.938112+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"75'128","reported_seq":1255,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952217+0000","last_change":"2024-09-20T22:23:16.175251+0000","last_active":"2024-09-20T22:23:30.952217+0000","last_peered":"2024-09-20T22:23:30.952217+0000","last_clean":"2024-09-20T22:23:30.952217+0000","last_became_active":"2024-09-20T22:22:33.734769+0000","last_became_peered":"2024-09-20T22:22:33.734769+0000","last_unstale":"2024-09-20T22:23:30.952217+0000","last_undegraded":"2024-09-20T22:23:30.952217+0000","last_fullsized":"2024-09-20T22:23:30.952217+0000","mapping_epoch":561,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":562,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:23:16.175099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:23:16.175099+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:20.873689+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"579'29","reported_seq":123,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718367+0000","last_change":"2024-09-20T22:23:11.705652+0000","last_active":"2024-09-20T22:23:31.718367+0000","last_peered":"2024-09-20T22:23:31.718367+0000","last_clean":"2024-09-20T22:23:31.718367+0000","last_became_active":"2024-09-20T22:22:36.732746+0000","last_became_peered":"2024-09-20T22:22:36.732746+0000","last_unstale":"2024-09-20T22:23:31.718367+0000","last_undegraded":"2024-09-20T22:23:31.718367+0000","last_fullsized":"2024-09-20T22:23:31.718367+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'29","last_scrub_stamp":"2024-09-20T22:23:11.705547+0000","last_deep_scrub":"418'23","last_deep_scrub_stamp":"2024-09-20T22:17:11.338821+0000","last_clean_scrub_stamp":"2024-09-20T22:23:11.705547+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:22.808300+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1170,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.954492+0000","last_change":"2024-09-20T22:23:09.655954+0000","last_active":"2024-09-20T22:23:30.954492+0000","last_peered":"2024-09-20T22:23:30.954492+0000","last_clean":"2024-09-20T22:23:30.954492+0000","last_became_active":"2024-09-20T22:22:36.742134+0000","last_became_peered":"2024-09-20T22:22:36.742134+0000","last_unstale":"2024-09-20T22:23:30.954492+0000","last_undegraded":"2024-09-20T22:23:30.954492+0000","last_fullsized":"2024-09-20T22:23:30.954492+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:09.655865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:23:09.655865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:13.182997+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1195,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.958110+0000","last_change":"2024-09-20T22:22:36.734670+0000","last_active":"2024-09-20T22:23:31.958110+0000","last_peered":"2024-09-20T22:23:31.958110+0000","last_clean":"2024-09-20T22:23:31.958110+0000","last_became_active":"2024-09-20T22:22:36.734292+0000","last_became_peered":"2024-09-20T22:22:36.734292+0000","last_unstale":"2024-09-20T22:23:31.958110+0000","last_undegraded":"2024-09-20T22:23:31.958110+0000","last_fullsized":"2024-09-20T22:23:31.958110+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:29.970946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:29.970946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:43.300720+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":629,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952273+0000","last_change":"2024-09-20T22:22:36.734889+0000","last_active":"2024-09-20T22:23:30.952273+0000","last_peered":"2024-09-20T22:23:30.952273+0000","last_clean":"2024-09-20T22:23:30.952273+0000","last_became_active":"2024-09-20T22:22:36.734590+0000","last_became_peered":"2024-09-20T22:22:36.734590+0000","last_unstale":"2024-09-20T22:23:30.952273+0000","last_undegraded":"2024-09-20T22:23:30.952273+0000","last_fullsized":"2024-09-20T22:23:30.952273+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:03.002391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:03.002391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:31.336479+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'128","reported_seq":787,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.954341+0000","last_change":"2024-09-20T22:22:44.673975+0000","last_active":"2024-09-20T22:23:30.954341+0000","last_peered":"2024-09-20T22:23:30.954341+0000","last_clean":"2024-09-20T22:23:30.954341+0000","last_became_active":"2024-09-20T22:22:39.718957+0000","last_became_peered":"2024-09-20T22:22:39.718957+0000","last_unstale":"2024-09-20T22:23:30.954341+0000","last_undegraded":"2024-09-20T22:23:30.954341+0000","last_fullsized":"2024-09-20T22:23:30.954341+0000","mapping_epoch":567,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":568,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:44.673852+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:22:44.673852+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:03.639902+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"579'46","reported_seq":766,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062310+0000","last_change":"2024-09-20T22:22:30.705915+0000","last_active":"2024-09-20T22:23:31.062310+0000","last_peered":"2024-09-20T22:23:31.062310+0000","last_clean":"2024-09-20T22:23:31.062310+0000","last_became_active":"2024-09-20T22:22:30.679068+0000","last_became_peered":"2024-09-20T22:22:30.679068+0000","last_unstale":"2024-09-20T22:23:31.062310+0000","last_undegraded":"2024-09-20T22:23:31.062310+0000","last_fullsized":"2024-09-20T22:23:31.062310+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"416'34","last_scrub_stamp":"2024-09-20T22:22:04.106984+0000","last_deep_scrub":"416'34","last_deep_scrub_stamp":"2024-09-20T22:17:04.960380+0000","last_clean_scrub_stamp":"2024-09-20T22:22:04.106984+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":807,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.954788+0000","last_change":"2024-09-20T22:23:20.552782+0000","last_active":"2024-09-20T22:23:30.954788+0000","last_peered":"2024-09-20T22:23:30.954788+0000","last_clean":"2024-09-20T22:23:30.954788+0000","last_became_active":"2024-09-20T22:22:34.738267+0000","last_became_peered":"2024-09-20T22:22:34.738267+0000","last_unstale":"2024-09-20T22:23:30.954788+0000","last_undegraded":"2024-09-20T22:23:30.954788+0000","last_fullsized":"2024-09-20T22:23:30.954788+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:23:20.552660+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:23:20.552660+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:44.224134+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"563'13","reported_seq":1436,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.953891+0000","last_change":"2024-09-20T22:23:12.613078+0000","last_active":"2024-09-20T22:23:30.953891+0000","last_peered":"2024-09-20T22:23:30.953891+0000","last_clean":"2024-09-20T22:23:30.953891+0000","last_became_active":"2024-09-20T22:22:34.740079+0000","last_became_peered":"2024-09-20T22:22:34.740079+0000","last_unstale":"2024-09-20T22:23:30.953891+0000","last_undegraded":"2024-09-20T22:23:30.953891+0000","last_fullsized":"2024-09-20T22:23:30.953891+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"563'13","last_scrub_stamp":"2024-09-20T22:23:12.612938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:23:12.612938+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:23.589739+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":695,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062594+0000","last_change":"2024-09-20T22:23:06.007858+0000","last_active":"2024-09-20T22:23:31.062594+0000","last_peered":"2024-09-20T22:23:31.062594+0000","last_clean":"2024-09-20T22:23:31.062594+0000","last_became_active":"2024-09-20T22:22:30.688569+0000","last_became_peered":"2024-09-20T22:22:30.688569+0000","last_unstale":"2024-09-20T22:23:31.062594+0000","last_undegraded":"2024-09-20T22:23:31.062594+0000","last_fullsized":"2024-09-20T22:23:31.062594+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:06.007552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:23:06.007552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:30.820960+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"75'128","reported_seq":1059,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062310+0000","last_change":"2024-09-20T22:23:04.007381+0000","last_active":"2024-09-20T22:23:31.062310+0000","last_peered":"2024-09-20T22:23:31.062310+0000","last_clean":"2024-09-20T22:23:31.062310+0000","last_became_active":"2024-09-20T22:22:39.718576+0000","last_became_peered":"2024-09-20T22:22:39.718576+0000","last_unstale":"2024-09-20T22:23:31.062310+0000","last_undegraded":"2024-09-20T22:23:31.062310+0000","last_fullsized":"2024-09-20T22:23:31.062310+0000","mapping_epoch":567,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":568,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:23:04.007289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:23:04.007289+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:08.625589+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":94,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959891+0000","last_change":"2024-09-20T22:22:39.953831+0000","last_active":"2024-09-20T22:23:31.959891+0000","last_peered":"2024-09-20T22:23:31.959891+0000","last_clean":"2024-09-20T22:23:31.959891+0000","last_became_active":"2024-09-20T22:22:30.674649+0000","last_became_peered":"2024-09-20T22:22:30.674649+0000","last_unstale":"2024-09-20T22:23:31.959891+0000","last_undegraded":"2024-09-20T22:23:31.959891+0000","last_fullsized":"2024-09-20T22:23:31.959891+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:39.953740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:39.953740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:08.818852+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"579'18","reported_seq":1251,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952838+0000","last_change":"2024-09-20T22:23:23.836786+0000","last_active":"2024-09-20T22:23:30.952838+0000","last_peered":"2024-09-20T22:23:30.952838+0000","last_clean":"2024-09-20T22:23:30.952838+0000","last_became_active":"2024-09-20T22:23:23.832325+0000","last_became_peered":"2024-09-20T22:23:23.832325+0000","last_unstale":"2024-09-20T22:23:30.952838+0000","last_undegraded":"2024-09-20T22:23:30.952838+0000","last_fullsized":"2024-09-20T22:23:30.952838+0000","mapping_epoch":591,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":592,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'18","last_scrub_stamp":"2024-09-20T22:23:17.214710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:23:17.214710+0000","objects_scrubbed":3,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:27.093767+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":694,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062978+0000","last_change":"2024-09-20T22:22:58.915746+0000","last_active":"2024-09-20T22:23:31.062978+0000","last_peered":"2024-09-20T22:23:31.062978+0000","last_clean":"2024-09-20T22:23:31.062978+0000","last_became_active":"2024-09-20T22:22:36.743711+0000","last_became_peered":"2024-09-20T22:22:36.743711+0000","last_unstale":"2024-09-20T22:23:31.062978+0000","last_undegraded":"2024-09-20T22:23:31.062978+0000","last_fullsized":"2024-09-20T22:23:31.062978+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:58.915662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:58.915662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:27.833030+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1148,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.954516+0000","last_change":"2024-09-20T22:23:23.584474+0000","last_active":"2024-09-20T22:23:30.954516+0000","last_peered":"2024-09-20T22:23:30.954516+0000","last_clean":"2024-09-20T22:23:30.954516+0000","last_became_active":"2024-09-20T22:22:34.756254+0000","last_became_peered":"2024-09-20T22:22:34.756254+0000","last_unstale":"2024-09-20T22:23:30.954516+0000","last_undegraded":"2024-09-20T22:23:30.954516+0000","last_fullsized":"2024-09-20T22:23:30.954516+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:23.584366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:23:23.584366+0000","last_clean_scrub_stamp":"2024-09-20T22:23:23.584366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:31.042510+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"75'128","reported_seq":964,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718205+0000","last_change":"2024-09-20T22:23:24.115165+0000","last_active":"2024-09-20T22:23:31.718205+0000","last_peered":"2024-09-20T22:23:31.718205+0000","last_clean":"2024-09-20T22:23:31.718205+0000","last_became_active":"2024-09-20T22:22:39.717148+0000","last_became_peered":"2024-09-20T22:22:39.717148+0000","last_unstale":"2024-09-20T22:23:31.718205+0000","last_undegraded":"2024-09-20T22:23:31.718205+0000","last_fullsized":"2024-09-20T22:23:31.718205+0000","mapping_epoch":567,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":568,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:23:24.115087+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:23:24.115087+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:40.063289+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"579'21","reported_seq":627,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.720487+0000","last_change":"2024-09-20T22:23:24.215479+0000","last_active":"2024-09-20T22:23:31.720487+0000","last_peered":"2024-09-20T22:23:31.720487+0000","last_clean":"2024-09-20T22:23:31.720487+0000","last_became_active":"2024-09-20T22:12:52.527106+0000","last_became_peered":"2024-09-20T22:12:52.527106+0000","last_unstale":"2024-09-20T22:23:31.720487+0000","last_undegraded":"2024-09-20T22:23:31.720487+0000","last_fullsized":"2024-09-20T22:23:31.720487+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'21","last_scrub_stamp":"2024-09-20T22:23:24.215395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:23:24.215395+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:36.126021+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1168,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.953544+0000","last_change":"2024-09-20T22:23:24.557886+0000","last_active":"2024-09-20T22:23:30.953544+0000","last_peered":"2024-09-20T22:23:30.953544+0000","last_clean":"2024-09-20T22:23:30.953544+0000","last_became_active":"2024-09-20T22:22:34.739214+0000","last_became_peered":"2024-09-20T22:22:34.739214+0000","last_unstale":"2024-09-20T22:23:30.953544+0000","last_undegraded":"2024-09-20T22:23:30.953544+0000","last_fullsized":"2024-09-20T22:23:30.953544+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:24.557758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:23:24.557758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:28.189434+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":587,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952685+0000","last_change":"2024-09-20T22:22:58.030950+0000","last_active":"2024-09-20T22:23:30.952685+0000","last_peered":"2024-09-20T22:23:30.952685+0000","last_clean":"2024-09-20T22:23:30.952685+0000","last_became_active":"2024-09-20T22:22:51.342516+0000","last_became_peered":"2024-09-20T22:22:51.342516+0000","last_unstale":"2024-09-20T22:23:30.952685+0000","last_undegraded":"2024-09-20T22:23:30.952685+0000","last_fullsized":"2024-09-20T22:23:30.952685+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:58.030861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_clean_scrub_stamp":"2024-09-20T22:22:58.030861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:26.761346+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1175,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959680+0000","last_change":"2024-09-20T22:22:34.738019+0000","last_active":"2024-09-20T22:23:31.959680+0000","last_peered":"2024-09-20T22:23:31.959680+0000","last_clean":"2024-09-20T22:23:31.959680+0000","last_became_active":"2024-09-20T22:22:34.737638+0000","last_became_peered":"2024-09-20T22:22:34.737638+0000","last_unstale":"2024-09-20T22:23:31.959680+0000","last_undegraded":"2024-09-20T22:23:31.959680+0000","last_fullsized":"2024-09-20T22:23:31.959680+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:16.043342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:26.862279+0000","last_clean_scrub_stamp":"2024-09-20T22:22:16.043342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:45.338584+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"75'128","reported_seq":1041,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062146+0000","last_change":"2024-09-20T22:22:40.842030+0000","last_active":"2024-09-20T22:23:31.062146+0000","last_peered":"2024-09-20T22:23:31.062146+0000","last_clean":"2024-09-20T22:23:31.062146+0000","last_became_active":"2024-09-20T22:22:33.745216+0000","last_became_peered":"2024-09-20T22:22:33.745216+0000","last_unstale":"2024-09-20T22:23:31.062146+0000","last_undegraded":"2024-09-20T22:23:31.062146+0000","last_fullsized":"2024-09-20T22:23:31.062146+0000","mapping_epoch":561,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":562,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:40.841933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:40.841933+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:52.374743+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"579'21","reported_seq":698,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.063025+0000","last_change":"2024-09-20T22:22:54.908266+0000","last_active":"2024-09-20T22:23:31.063025+0000","last_peered":"2024-09-20T22:23:31.063025+0000","last_clean":"2024-09-20T22:23:31.063025+0000","last_became_active":"2024-09-20T22:11:30.493335+0000","last_became_peered":"2024-09-20T22:11:30.493335+0000","last_unstale":"2024-09-20T22:23:31.063025+0000","last_undegraded":"2024-09-20T22:23:31.063025+0000","last_fullsized":"2024-09-20T22:23:31.063025+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:22:54.908165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:54.908165+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:18.473541+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":599,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718988+0000","last_change":"2024-09-20T22:22:43.111083+0000","last_active":"2024-09-20T22:23:31.718988+0000","last_peered":"2024-09-20T22:23:31.718988+0000","last_clean":"2024-09-20T22:23:31.718988+0000","last_became_active":"2024-09-20T22:12:52.524149+0000","last_became_peered":"2024-09-20T22:12:52.524149+0000","last_unstale":"2024-09-20T22:23:31.718988+0000","last_undegraded":"2024-09-20T22:23:31.718988+0000","last_fullsized":"2024-09-20T22:23:31.718988+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:43.111008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:22:43.111008+0000","last_clean_scrub_stamp":"2024-09-20T22:22:43.111008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:09.952898+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"575'12","reported_seq":741,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.953238+0000","last_change":"2024-09-20T22:23:30.953260+0000","last_active":"2024-09-20T22:23:30.953238+0000","last_peered":"2024-09-20T22:23:30.953238+0000","last_clean":"2024-09-20T22:23:30.953238+0000","last_became_active":"2024-09-20T22:22:51.344012+0000","last_became_peered":"2024-09-20T22:22:51.344012+0000","last_unstale":"2024-09-20T22:23:30.953238+0000","last_undegraded":"2024-09-20T22:23:30.953238+0000","last_fullsized":"2024-09-20T22:23:30.953238+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"575'12","last_scrub_stamp":"2024-09-20T22:23:30.953187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:23:30.953187+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:48.784625+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":76,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719340+0000","last_change":"2024-09-20T22:22:34.747354+0000","last_active":"2024-09-20T22:23:31.719340+0000","last_peered":"2024-09-20T22:23:31.719340+0000","last_clean":"2024-09-20T22:23:31.719340+0000","last_became_active":"2024-09-20T22:22:34.747053+0000","last_became_peered":"2024-09-20T22:22:34.747053+0000","last_unstale":"2024-09-20T22:23:31.719340+0000","last_undegraded":"2024-09-20T22:23:31.719340+0000","last_fullsized":"2024-09-20T22:23:31.719340+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:17.031182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:03.323607+0000","last_clean_scrub_stamp":"2024-09-20T22:22:17.031182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:34.770291+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"75'128","reported_seq":1050,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718558+0000","last_change":"2024-09-20T22:22:53.181146+0000","last_active":"2024-09-20T22:23:31.718558+0000","last_peered":"2024-09-20T22:23:31.718558+0000","last_clean":"2024-09-20T22:23:31.718558+0000","last_became_active":"2024-09-20T22:15:50.612544+0000","last_became_peered":"2024-09-20T22:15:50.612544+0000","last_unstale":"2024-09-20T22:23:31.718558+0000","last_undegraded":"2024-09-20T22:23:31.718558+0000","last_fullsized":"2024-09-20T22:23:31.718558+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:53.181058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:53.181058+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:02.922575+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"579'35","reported_seq":428,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952369+0000","last_change":"2024-09-20T22:23:03.099892+0000","last_active":"2024-09-20T22:23:30.952369+0000","last_peered":"2024-09-20T22:23:30.952369+0000","last_clean":"2024-09-20T22:23:30.952369+0000","last_became_active":"2024-09-20T22:22:51.343502+0000","last_became_peered":"2024-09-20T22:22:51.343502+0000","last_unstale":"2024-09-20T22:23:30.952369+0000","last_undegraded":"2024-09-20T22:23:30.952369+0000","last_fullsized":"2024-09-20T22:23:30.952369+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'35","last_scrub_stamp":"2024-09-20T22:23:03.099751+0000","last_deep_scrub":"579'35","last_deep_scrub_stamp":"2024-09-20T22:23:03.099751+0000","last_clean_scrub_stamp":"2024-09-20T22:23:03.099751+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:27.881468+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1160,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.954328+0000","last_change":"2024-09-20T22:23:18.625730+0000","last_active":"2024-09-20T22:23:30.954328+0000","last_peered":"2024-09-20T22:23:30.954328+0000","last_clean":"2024-09-20T22:23:30.954328+0000","last_became_active":"2024-09-20T22:22:34.738109+0000","last_became_peered":"2024-09-20T22:22:34.738109+0000","last_unstale":"2024-09-20T22:23:30.954328+0000","last_undegraded":"2024-09-20T22:23:30.954328+0000","last_fullsized":"2024-09-20T22:23:30.954328+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:18.625639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:23:18.625639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:40.294115+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"567'16","reported_seq":868,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.063263+0000","last_change":"2024-09-20T22:23:13.038668+0000","last_active":"2024-09-20T22:23:31.063263+0000","last_peered":"2024-09-20T22:23:31.063263+0000","last_clean":"2024-09-20T22:23:31.063263+0000","last_became_active":"2024-09-20T22:22:36.741754+0000","last_became_peered":"2024-09-20T22:22:36.741754+0000","last_unstale":"2024-09-20T22:23:31.063263+0000","last_undegraded":"2024-09-20T22:23:31.063263+0000","last_fullsized":"2024-09-20T22:23:31.063263+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"567'16","last_scrub_stamp":"2024-09-20T22:23:13.038571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:23:13.038571+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:34.445388+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":599,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.720160+0000","last_change":"2024-09-20T22:23:09.207287+0000","last_active":"2024-09-20T22:23:31.720160+0000","last_peered":"2024-09-20T22:23:31.720160+0000","last_clean":"2024-09-20T22:23:31.720160+0000","last_became_active":"2024-09-20T22:12:52.523781+0000","last_became_peered":"2024-09-20T22:12:52.523781+0000","last_unstale":"2024-09-20T22:23:31.720160+0000","last_undegraded":"2024-09-20T22:23:31.720160+0000","last_fullsized":"2024-09-20T22:23:31.720160+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:09.207188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:23:09.207188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:17.469298+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.16","version":"75'128","reported_seq":849,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719922+0000","last_change":"2024-09-20T22:23:07.318573+0000","last_active":"2024-09-20T22:23:31.719922+0000","last_peered":"2024-09-20T22:23:31.719922+0000","last_clean":"2024-09-20T22:23:31.719922+0000","last_became_active":"2024-09-20T22:22:38.715194+0000","last_became_peered":"2024-09-20T22:22:38.715194+0000","last_unstale":"2024-09-20T22:23:31.719922+0000","last_undegraded":"2024-09-20T22:23:31.719922+0000","last_fullsized":"2024-09-20T22:23:31.719922+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:23:07.318498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:23:07.318498+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:15.015805+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"579'27","reported_seq":126,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718601+0000","last_change":"2024-09-20T22:22:57.761230+0000","last_active":"2024-09-20T22:23:31.718601+0000","last_peered":"2024-09-20T22:23:31.718601+0000","last_clean":"2024-09-20T22:23:31.718601+0000","last_became_active":"2024-09-20T22:22:36.746508+0000","last_became_peered":"2024-09-20T22:22:36.746508+0000","last_unstale":"2024-09-20T22:23:31.718601+0000","last_undegraded":"2024-09-20T22:23:31.718601+0000","last_fullsized":"2024-09-20T22:23:31.718601+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:22:57.761101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:57.761101+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:10.300960+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":84,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718444+0000","last_change":"2024-09-20T22:23:07.660296+0000","last_active":"2024-09-20T22:23:31.718444+0000","last_peered":"2024-09-20T22:23:31.718444+0000","last_clean":"2024-09-20T22:23:31.718444+0000","last_became_active":"2024-09-20T22:22:34.745153+0000","last_became_peered":"2024-09-20T22:22:34.745153+0000","last_unstale":"2024-09-20T22:23:31.718444+0000","last_undegraded":"2024-09-20T22:23:31.718444+0000","last_fullsized":"2024-09-20T22:23:31.718444+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:07.660206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:23:07.660206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:34.264244+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":81,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719813+0000","last_change":"2024-09-20T22:22:54.204146+0000","last_active":"2024-09-20T22:23:31.719813+0000","last_peered":"2024-09-20T22:23:31.719813+0000","last_clean":"2024-09-20T22:23:31.719813+0000","last_became_active":"2024-09-20T22:22:36.735628+0000","last_became_peered":"2024-09-20T22:22:36.735628+0000","last_unstale":"2024-09-20T22:23:31.719813+0000","last_undegraded":"2024-09-20T22:23:31.719813+0000","last_fullsized":"2024-09-20T22:23:31.719813+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:54.204042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:54.204042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:07.791351+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":459,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.951696+0000","last_change":"2024-09-20T22:23:04.065863+0000","last_active":"2024-09-20T22:23:30.951696+0000","last_peered":"2024-09-20T22:23:30.951696+0000","last_clean":"2024-09-20T22:23:30.951696+0000","last_became_active":"2024-09-20T22:22:51.734410+0000","last_became_peered":"2024-09-20T22:22:51.734410+0000","last_unstale":"2024-09-20T22:23:30.951696+0000","last_undegraded":"2024-09-20T22:23:30.951696+0000","last_fullsized":"2024-09-20T22:23:30.951696+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":576,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:04.065790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:23:04.065790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:29.481511+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.17","version":"75'128","reported_seq":663,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.063542+0000","last_change":"2024-09-20T22:22:33.961540+0000","last_active":"2024-09-20T22:23:31.063542+0000","last_peered":"2024-09-20T22:23:31.063542+0000","last_clean":"2024-09-20T22:23:31.063542+0000","last_became_active":"2024-09-20T22:22:33.743368+0000","last_became_peered":"2024-09-20T22:22:33.743368+0000","last_unstale":"2024-09-20T22:23:31.063542+0000","last_undegraded":"2024-09-20T22:23:31.063542+0000","last_fullsized":"2024-09-20T22:23:31.063542+0000","mapping_epoch":561,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":562,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:33.961468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:33.961468+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:58.544200+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"579'25","reported_seq":100,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719784+0000","last_change":"2024-09-20T22:23:12.296665+0000","last_active":"2024-09-20T22:23:31.719784+0000","last_peered":"2024-09-20T22:23:31.719784+0000","last_clean":"2024-09-20T22:23:31.719784+0000","last_became_active":"2024-09-20T22:22:36.729181+0000","last_became_peered":"2024-09-20T22:22:36.729181+0000","last_unstale":"2024-09-20T22:23:31.719784+0000","last_undegraded":"2024-09-20T22:23:31.719784+0000","last_fullsized":"2024-09-20T22:23:31.719784+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'25","last_scrub_stamp":"2024-09-20T22:23:12.296574+0000","last_deep_scrub":"291'13","last_deep_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_clean_scrub_stamp":"2024-09-20T22:23:12.296574+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:32.519705+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":745,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.954460+0000","last_change":"2024-09-20T22:22:34.755241+0000","last_active":"2024-09-20T22:23:30.954460+0000","last_peered":"2024-09-20T22:23:30.954460+0000","last_clean":"2024-09-20T22:23:30.954460+0000","last_became_active":"2024-09-20T22:22:34.754881+0000","last_became_peered":"2024-09-20T22:22:34.754881+0000","last_unstale":"2024-09-20T22:23:30.954460+0000","last_undegraded":"2024-09-20T22:23:30.954460+0000","last_fullsized":"2024-09-20T22:23:30.954460+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:27.499518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:27.499518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:48.896639+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":584,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952362+0000","last_change":"2024-09-20T22:22:51.338655+0000","last_active":"2024-09-20T22:23:30.952362+0000","last_peered":"2024-09-20T22:23:30.952362+0000","last_clean":"2024-09-20T22:23:30.952362+0000","last_became_active":"2024-09-20T22:22:51.338112+0000","last_became_peered":"2024-09-20T22:22:51.338112+0000","last_unstale":"2024-09-20T22:23:30.952362+0000","last_undegraded":"2024-09-20T22:23:30.952362+0000","last_fullsized":"2024-09-20T22:23:30.952362+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:44.067640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:22:44.067640+0000","last_clean_scrub_stamp":"2024-09-20T22:22:44.067640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:48.150761+0000","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":615,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.717964+0000","last_change":"2024-09-20T22:22:36.748696+0000","last_active":"2024-09-20T22:23:31.717964+0000","last_peered":"2024-09-20T22:23:31.717964+0000","last_clean":"2024-09-20T22:23:31.717964+0000","last_became_active":"2024-09-20T22:22:36.747860+0000","last_became_peered":"2024-09-20T22:22:36.747860+0000","last_unstale":"2024-09-20T22:23:31.717964+0000","last_undegraded":"2024-09-20T22:23:31.717964+0000","last_fullsized":"2024-09-20T22:23:31.717964+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:11.135068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:11.135068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"75'128","reported_seq":1236,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959843+0000","last_change":"2024-09-20T22:22:38.047138+0000","last_active":"2024-09-20T22:23:31.959843+0000","last_peered":"2024-09-20T22:23:31.959843+0000","last_clean":"2024-09-20T22:23:31.959843+0000","last_became_active":"2024-09-20T22:22:37.719347+0000","last_became_peered":"2024-09-20T22:22:37.719347+0000","last_unstale":"2024-09-20T22:23:31.959843+0000","last_undegraded":"2024-09-20T22:23:31.959843+0000","last_fullsized":"2024-09-20T22:23:31.959843+0000","mapping_epoch":565,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":566,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:38.047061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:38.047061+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:39.387643+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"579'27","reported_seq":562,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952025+0000","last_change":"2024-09-20T22:23:01.029518+0000","last_active":"2024-09-20T22:23:30.952025+0000","last_peered":"2024-09-20T22:23:30.952025+0000","last_clean":"2024-09-20T22:23:30.952025+0000","last_became_active":"2024-09-20T22:22:51.342232+0000","last_became_peered":"2024-09-20T22:22:51.342232+0000","last_unstale":"2024-09-20T22:23:30.952025+0000","last_undegraded":"2024-09-20T22:23:30.952025+0000","last_fullsized":"2024-09-20T22:23:30.952025+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'27","last_scrub_stamp":"2024-09-20T22:23:01.029439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:23:01.029439+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:25.584617+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":685,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062970+0000","last_change":"2024-09-20T22:22:36.742839+0000","last_active":"2024-09-20T22:23:31.062970+0000","last_peered":"2024-09-20T22:23:31.062970+0000","last_clean":"2024-09-20T22:23:31.062970+0000","last_became_active":"2024-09-20T22:22:36.742517+0000","last_became_peered":"2024-09-20T22:22:36.742517+0000","last_unstale":"2024-09-20T22:23:31.062970+0000","last_undegraded":"2024-09-20T22:23:31.062970+0000","last_fullsized":"2024-09-20T22:23:31.062970+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:18.069137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_clean_scrub_stamp":"2024-09-20T22:22:18.069137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:34.524346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1160,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.953795+0000","last_change":"2024-09-20T22:23:21.517779+0000","last_active":"2024-09-20T22:23:30.953795+0000","last_peered":"2024-09-20T22:23:30.953795+0000","last_clean":"2024-09-20T22:23:30.953795+0000","last_became_active":"2024-09-20T22:22:34.756599+0000","last_became_peered":"2024-09-20T22:22:34.756599+0000","last_unstale":"2024-09-20T22:23:30.953795+0000","last_undegraded":"2024-09-20T22:23:30.953795+0000","last_fullsized":"2024-09-20T22:23:30.953795+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:21.517686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:23:21.517686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:46.367958+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":600,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719834+0000","last_change":"2024-09-20T22:22:41.121676+0000","last_active":"2024-09-20T22:23:31.719834+0000","last_peered":"2024-09-20T22:23:31.719834+0000","last_clean":"2024-09-20T22:23:31.719834+0000","last_became_active":"2024-09-20T22:22:30.665777+0000","last_became_peered":"2024-09-20T22:22:30.665777+0000","last_unstale":"2024-09-20T22:23:31.719834+0000","last_undegraded":"2024-09-20T22:23:31.719834+0000","last_fullsized":"2024-09-20T22:23:31.719834+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:41.121604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:41.121604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:03.413460+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.15","version":"75'128","reported_seq":662,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.954299+0000","last_change":"2024-09-20T22:22:39.719978+0000","last_active":"2024-09-20T22:23:30.954299+0000","last_peered":"2024-09-20T22:23:30.954299+0000","last_clean":"2024-09-20T22:23:30.954299+0000","last_became_active":"2024-09-20T22:22:39.719578+0000","last_became_peered":"2024-09-20T22:22:39.719578+0000","last_unstale":"2024-09-20T22:23:30.954299+0000","last_undegraded":"2024-09-20T22:23:30.954299+0000","last_fullsized":"2024-09-20T22:23:30.954299+0000","mapping_epoch":567,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":568,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:31.546728+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:22:31.546728+0000","last_clean_scrub_stamp":"2024-09-20T22:22:31.546728+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:37.843719+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"579'14","reported_seq":1235,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.951694+0000","last_change":"2024-09-20T22:23:08.171590+0000","last_active":"2024-09-20T22:23:30.951694+0000","last_peered":"2024-09-20T22:23:30.951694+0000","last_clean":"2024-09-20T22:23:30.951694+0000","last_became_active":"2024-09-20T22:22:36.737769+0000","last_became_peered":"2024-09-20T22:22:36.737769+0000","last_unstale":"2024-09-20T22:23:30.951694+0000","last_undegraded":"2024-09-20T22:23:30.951694+0000","last_fullsized":"2024-09-20T22:23:30.951694+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'14","last_scrub_stamp":"2024-09-20T22:23:08.171508+0000","last_deep_scrub":"453'10","last_deep_scrub_stamp":"2024-09-20T22:21:38.900946+0000","last_clean_scrub_stamp":"2024-09-20T22:23:08.171508+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:18.059049+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":591,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.720190+0000","last_change":"2024-09-20T22:22:33.051496+0000","last_active":"2024-09-20T22:23:31.720190+0000","last_peered":"2024-09-20T22:23:31.720190+0000","last_clean":"2024-09-20T22:23:31.720190+0000","last_became_active":"2024-09-20T22:12:52.517510+0000","last_became_peered":"2024-09-20T22:12:52.517510+0000","last_unstale":"2024-09-20T22:23:31.720190+0000","last_undegraded":"2024-09-20T22:23:31.720190+0000","last_fullsized":"2024-09-20T22:23:31.720190+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:33.051367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:33.051367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:01.221318+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":100,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.961092+0000","last_change":"2024-09-20T22:22:53.005692+0000","last_active":"2024-09-20T22:23:31.961092+0000","last_peered":"2024-09-20T22:23:31.961092+0000","last_clean":"2024-09-20T22:23:31.961092+0000","last_became_active":"2024-09-20T22:22:34.747685+0000","last_became_peered":"2024-09-20T22:22:34.747685+0000","last_unstale":"2024-09-20T22:23:31.961092+0000","last_undegraded":"2024-09-20T22:23:31.961092+0000","last_fullsized":"2024-09-20T22:23:31.961092+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:53.005597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:22:53.005597+0000","last_clean_scrub_stamp":"2024-09-20T22:22:53.005597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:16.098003+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1136,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.953944+0000","last_change":"2024-09-20T22:22:34.738847+0000","last_active":"2024-09-20T22:23:30.953944+0000","last_peered":"2024-09-20T22:23:30.953944+0000","last_clean":"2024-09-20T22:23:30.953944+0000","last_became_active":"2024-09-20T22:22:34.738379+0000","last_became_peered":"2024-09-20T22:22:34.738379+0000","last_unstale":"2024-09-20T22:23:30.953944+0000","last_undegraded":"2024-09-20T22:23:30.953944+0000","last_fullsized":"2024-09-20T22:23:30.953944+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:21.514519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_clean_scrub_stamp":"2024-09-20T22:22:21.514519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:48.892847+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1a","version":"75'128","reported_seq":400,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718608+0000","last_change":"2024-09-20T22:22:40.669892+0000","last_active":"2024-09-20T22:23:31.718608+0000","last_peered":"2024-09-20T22:23:31.718608+0000","last_clean":"2024-09-20T22:23:31.718608+0000","last_became_active":"2024-09-20T22:22:39.715570+0000","last_became_peered":"2024-09-20T22:22:39.715570+0000","last_unstale":"2024-09-20T22:23:31.718608+0000","last_undegraded":"2024-09-20T22:23:31.718608+0000","last_fullsized":"2024-09-20T22:23:31.718608+0000","mapping_epoch":567,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":568,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:40.669777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:40.669777+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:00.927551+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":81,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719242+0000","last_change":"2024-09-20T22:22:51.237996+0000","last_active":"2024-09-20T22:23:31.719242+0000","last_peered":"2024-09-20T22:23:31.719242+0000","last_clean":"2024-09-20T22:23:31.719242+0000","last_became_active":"2024-09-20T22:22:36.751845+0000","last_became_peered":"2024-09-20T22:22:36.751845+0000","last_unstale":"2024-09-20T22:23:31.719242+0000","last_undegraded":"2024-09-20T22:23:31.719242+0000","last_fullsized":"2024-09-20T22:23:31.719242+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:51.237919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:22:51.237919+0000","last_clean_scrub_stamp":"2024-09-20T22:22:51.237919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:55.984423+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":596,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.951954+0000","last_change":"2024-09-20T22:23:15.172847+0000","last_active":"2024-09-20T22:23:30.951954+0000","last_peered":"2024-09-20T22:23:30.951954+0000","last_clean":"2024-09-20T22:23:30.951954+0000","last_became_active":"2024-09-20T22:22:51.338363+0000","last_became_peered":"2024-09-20T22:22:51.338363+0000","last_unstale":"2024-09-20T22:23:30.951954+0000","last_undegraded":"2024-09-20T22:23:30.951954+0000","last_fullsized":"2024-09-20T22:23:30.951954+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:15.172777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:21:39.004178+0000","last_clean_scrub_stamp":"2024-09-20T22:23:15.172777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:16.913341+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"579'27","reported_seq":136,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.961174+0000","last_change":"2024-09-20T22:23:09.049171+0000","last_active":"2024-09-20T22:23:31.961174+0000","last_peered":"2024-09-20T22:23:31.961174+0000","last_clean":"2024-09-20T22:23:31.961174+0000","last_became_active":"2024-09-20T22:22:36.741967+0000","last_became_peered":"2024-09-20T22:22:36.741967+0000","last_unstale":"2024-09-20T22:23:31.961174+0000","last_undegraded":"2024-09-20T22:23:31.961174+0000","last_fullsized":"2024-09-20T22:23:31.961174+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'27","last_scrub_stamp":"2024-09-20T22:23:09.049009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:23:09.049009+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:21.052436+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":592,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.720119+0000","last_change":"2024-09-20T22:22:10.095005+0000","last_active":"2024-09-20T22:23:31.720119+0000","last_peered":"2024-09-20T22:23:31.720119+0000","last_clean":"2024-09-20T22:23:31.720119+0000","last_became_active":"2024-09-20T22:12:52.524425+0000","last_became_peered":"2024-09-20T22:12:52.524425+0000","last_unstale":"2024-09-20T22:23:31.720119+0000","last_undegraded":"2024-09-20T22:23:31.720119+0000","last_fullsized":"2024-09-20T22:23:31.720119+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:10.094934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:10.094934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1528,"num_read_kb":1285,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":456,"num_bytes_recovered":9550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1052,"ondisk_log_size":1052,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":3044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1211,"num_read_kb":1206,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":31,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3968,"ondisk_log_size":3968,"up":93,"acting":93,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813136125,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":881492,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":879422,"kb_avail":92863660,"statfs":{"total":95995035648,"available":95092387840,"internally_reserved":0,"allocated":2093056,"data_stored":1409903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":900528586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430463,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":755220,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":753086,"kb_avail":92989932,"statfs":{"total":95995035648,"available":95221690368,"internally_reserved":0,"allocated":2162688,"data_stored":1490581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":771160518},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724805,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1002236,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":1000190,"kb_avail":92742916,"statfs":{"total":95995035648,"available":94968745984,"internally_reserved":0,"allocated":2072576,"data_stored":1409164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1024195013},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":532,"seq":2284922601496,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":84400,"kb_used_data":1816,"kb_used_omap":118,"kb_used_meta":82441,"kb_avail":93660752,"statfs":{"total":95995035648,"available":95908610048,"internally_reserved":0,"allocated":1859584,"data_stored":1162936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":120876,"internal_metadata":84420564},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770258122,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":722884,"kb_used_data":1836,"kb_used_omap":4,"kb_used_meta":721019,"kb_avail":93022268,"statfs":{"total":95995035648,"available":95254802432,"internally_reserved":0,"allocated":1880064,"data_stored":1164233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":738324136},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149970,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":944032,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":941886,"kb_avail":92801120,"statfs":{"total":95995035648,"available":95028346880,"internally_reserved":0,"allocated":2179072,"data_stored":1476130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":964491719},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411607,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":755156,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":753086,"kb_avail":92989996,"statfs":{"total":95995035648,"available":95221755904,"internally_reserved":0,"allocated":2093056,"data_stored":1417282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":771160517},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738658,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1007920,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":1005758,"kb_avail":92737232,"statfs":{"total":95995035648,"available":94962925568,"internally_reserved":0,"allocated":2183168,"data_stored":1490805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1029896645},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:23:36.449 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-20T22:23:36.709 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:37.097 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:36 smithi060 ceph-mon[29694]: from='client.16968 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:23:37.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:36 smithi060 ceph-mon[29694]: 6.c scrub starts 2024-09-20T22:23:37.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:36 smithi060 ceph-mon[29694]: 6.c scrub ok 2024-09-20T22:23:37.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:36 smithi060 ceph-mon[29694]: 1.a scrub starts 2024-09-20T22:23:37.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:36 smithi060 ceph-mon[29694]: 1.a scrub ok 2024-09-20T22:23:37.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:36 smithi060 ceph-mon[29694]: pgmap v1134: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-20T22:23:37.098 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:36 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:23:37.098 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:36 smithi060 ceph-mon[40290]: from='client.16968 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:23:37.098 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:36 smithi060 ceph-mon[40290]: 6.c scrub starts 2024-09-20T22:23:37.098 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:36 smithi060 ceph-mon[40290]: 6.c scrub ok 2024-09-20T22:23:37.098 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:36 smithi060 ceph-mon[40290]: 1.a scrub starts 2024-09-20T22:23:37.098 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:36 smithi060 ceph-mon[40290]: 1.a scrub ok 2024-09-20T22:23:37.099 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:36 smithi060 ceph-mon[40290]: pgmap v1134: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-20T22:23:37.099 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:36 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:23:37.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:36 smithi067 ceph-mon[33511]: from='client.16968 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:23:37.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:36 smithi067 ceph-mon[33511]: 6.c scrub starts 2024-09-20T22:23:37.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:36 smithi067 ceph-mon[33511]: 6.c scrub ok 2024-09-20T22:23:37.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:36 smithi067 ceph-mon[33511]: 1.a scrub starts 2024-09-20T22:23:37.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:36 smithi067 ceph-mon[33511]: 1.a scrub ok 2024-09-20T22:23:37.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:36 smithi067 ceph-mon[33511]: pgmap v1134: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-20T22:23:37.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:36 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-20T22:23:37.470 INFO:teuthology.orchestra.run.smithi060.stdout: 2024-09-20T22:23:37.472 INFO:teuthology.orchestra.run.smithi060.stderr:dumped all 2024-09-20T22:23:38.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:37 smithi060 ceph-mon[29694]: 3.16 scrub starts 2024-09-20T22:23:38.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:37 smithi060 ceph-mon[29694]: 3.16 scrub ok 2024-09-20T22:23:38.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:37 smithi060 ceph-mon[29694]: 6.10 scrub starts 2024-09-20T22:23:38.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:37 smithi060 ceph-mon[29694]: 6.10 scrub ok 2024-09-20T22:23:38.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:37 smithi060 ceph-mon[29694]: from='client.16974 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:23:38.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:37 smithi060 ceph-mon[40290]: 3.16 scrub starts 2024-09-20T22:23:38.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:37 smithi060 ceph-mon[40290]: 3.16 scrub ok 2024-09-20T22:23:38.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:37 smithi060 ceph-mon[40290]: 6.10 scrub starts 2024-09-20T22:23:38.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:37 smithi060 ceph-mon[40290]: 6.10 scrub ok 2024-09-20T22:23:38.037 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:37 smithi060 ceph-mon[40290]: from='client.16974 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:23:38.137 INFO:teuthology.orchestra.run.smithi060.stdout:{"pg_ready":true,"pg_map":{"version":1134,"stamp":"2024-09-20T22:23:36.427568+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596507,"num_objects":199,"num_object_clones":0,"num_object_copies":597,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":199,"num_whiteouts":0,"num_read":2761,"num_read_kb":2513,"num_write":1008,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":521,"num_bytes_recovered":669612,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5156,"ondisk_log_size":5156,"up":486,"acting":486,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":486,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":6153356,"kb_used_data":16160,"kb_used_omap":131,"kb_used_meta":6136892,"kb_avail":743807860,"statfs":{"total":767960285184,"available":761659248640,"internally_reserved":0,"allocated":16547840,"data_stored":11036812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":134878,"internal_metadata":6284177698},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.580430"},"pg_stats":[{"pgid":"1.1d","version":"75'128","reported_seq":654,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718936+0000","last_change":"2024-09-20T22:22:38.716841+0000","last_active":"2024-09-20T22:23:31.718936+0000","last_peered":"2024-09-20T22:23:31.718936+0000","last_clean":"2024-09-20T22:23:31.718936+0000","last_became_active":"2024-09-20T22:22:38.716205+0000","last_became_peered":"2024-09-20T22:22:38.716205+0000","last_unstale":"2024-09-20T22:23:31.718936+0000","last_undegraded":"2024-09-20T22:23:31.718936+0000","last_fullsized":"2024-09-20T22:23:31.718936+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:15.002349+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:22:15.002349+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:40.297605+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":592,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.720326+0000","last_change":"2024-09-20T22:22:42.153828+0000","last_active":"2024-09-20T22:23:31.720326+0000","last_peered":"2024-09-20T22:23:31.720326+0000","last_clean":"2024-09-20T22:23:31.720326+0000","last_became_active":"2024-09-20T22:12:52.515921+0000","last_became_peered":"2024-09-20T22:12:52.515921+0000","last_unstale":"2024-09-20T22:23:31.720326+0000","last_undegraded":"2024-09-20T22:23:31.720326+0000","last_fullsized":"2024-09-20T22:23:31.720326+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:42.153726+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:43.512066+0000","last_clean_scrub_stamp":"2024-09-20T22:22:42.153726+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:10.825665+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"579'25","reported_seq":119,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718090+0000","last_change":"2024-09-20T22:23:00.748734+0000","last_active":"2024-09-20T22:23:31.718090+0000","last_peered":"2024-09-20T22:23:31.718090+0000","last_clean":"2024-09-20T22:23:31.718090+0000","last_became_active":"2024-09-20T22:22:36.732025+0000","last_became_peered":"2024-09-20T22:22:36.732025+0000","last_unstale":"2024-09-20T22:23:31.718090+0000","last_undegraded":"2024-09-20T22:23:31.718090+0000","last_fullsized":"2024-09-20T22:23:31.718090+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'25","last_scrub_stamp":"2024-09-20T22:23:00.748614+0000","last_deep_scrub":"252'13","last_deep_scrub_stamp":"2024-09-20T22:10:31.568098+0000","last_clean_scrub_stamp":"2024-09-20T22:23:00.748614+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:24.603883+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":81,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718810+0000","last_change":"2024-09-20T22:23:28.170111+0000","last_active":"2024-09-20T22:23:31.718810+0000","last_peered":"2024-09-20T22:23:31.718810+0000","last_clean":"2024-09-20T22:23:31.718810+0000","last_became_active":"2024-09-20T22:22:36.741459+0000","last_became_peered":"2024-09-20T22:22:36.741459+0000","last_unstale":"2024-09-20T22:23:31.718810+0000","last_undegraded":"2024-09-20T22:23:31.718810+0000","last_fullsized":"2024-09-20T22:23:31.718810+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:28.170020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:23:28.170020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:57.705531+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":90,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718831+0000","last_change":"2024-09-20T22:22:42.640949+0000","last_active":"2024-09-20T22:23:31.718831+0000","last_peered":"2024-09-20T22:23:31.718831+0000","last_clean":"2024-09-20T22:23:31.718831+0000","last_became_active":"2024-09-20T22:22:36.732433+0000","last_became_peered":"2024-09-20T22:22:36.732433+0000","last_unstale":"2024-09-20T22:23:31.718831+0000","last_undegraded":"2024-09-20T22:23:31.718831+0000","last_fullsized":"2024-09-20T22:23:31.718831+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:42.640859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:42.640859+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:48.484375+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1c","version":"75'128","reported_seq":1130,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.958638+0000","last_change":"2024-09-20T22:22:39.718800+0000","last_active":"2024-09-20T22:23:31.958638+0000","last_peered":"2024-09-20T22:23:31.958638+0000","last_clean":"2024-09-20T22:23:31.958638+0000","last_became_active":"2024-09-20T22:22:39.718360+0000","last_became_peered":"2024-09-20T22:22:39.718360+0000","last_unstale":"2024-09-20T22:23:31.958638+0000","last_undegraded":"2024-09-20T22:23:31.958638+0000","last_fullsized":"2024-09-20T22:23:31.958638+0000","mapping_epoch":567,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":568,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:34.574149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:34.574149+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:52.649725+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1b","version":"71'1","reported_seq":1159,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952347+0000","last_change":"2024-09-20T22:22:33.952815+0000","last_active":"2024-09-20T22:23:30.952347+0000","last_peered":"2024-09-20T22:23:30.952347+0000","last_clean":"2024-09-20T22:23:30.952347+0000","last_became_active":"2024-09-20T22:22:30.667295+0000","last_became_peered":"2024-09-20T22:22:30.667295+0000","last_unstale":"2024-09-20T22:23:30.952347+0000","last_undegraded":"2024-09-20T22:23:30.952347+0000","last_fullsized":"2024-09-20T22:23:30.952347+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'1","last_scrub_stamp":"2024-09-20T22:22:33.952687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:33.952687+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:49.531816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":587,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952312+0000","last_change":"2024-09-20T22:22:51.340462+0000","last_active":"2024-09-20T22:23:30.952312+0000","last_peered":"2024-09-20T22:23:30.952312+0000","last_clean":"2024-09-20T22:23:30.952312+0000","last_became_active":"2024-09-20T22:22:51.340054+0000","last_became_peered":"2024-09-20T22:22:51.340054+0000","last_unstale":"2024-09-20T22:23:30.952312+0000","last_undegraded":"2024-09-20T22:23:30.952312+0000","last_fullsized":"2024-09-20T22:23:30.952312+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:26.924884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:26.924884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:56.275060+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"579'46","reported_seq":400,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.951787+0000","last_change":"2024-09-20T22:22:51.341873+0000","last_active":"2024-09-20T22:23:30.951787+0000","last_peered":"2024-09-20T22:23:30.951787+0000","last_clean":"2024-09-20T22:23:30.951787+0000","last_became_active":"2024-09-20T22:22:51.341154+0000","last_became_peered":"2024-09-20T22:22:51.341154+0000","last_unstale":"2024-09-20T22:23:30.951787+0000","last_undegraded":"2024-09-20T22:23:30.951787+0000","last_fullsized":"2024-09-20T22:23:30.951787+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"406'34","last_scrub_stamp":"2024-09-20T22:22:40.048191+0000","last_deep_scrub":"406'34","last_deep_scrub_stamp":"2024-09-20T22:16:31.033354+0000","last_clean_scrub_stamp":"2024-09-20T22:22:40.048191+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:40.973089+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":61,"num_read_kb":52,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":90,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718092+0000","last_change":"2024-09-20T22:22:48.688177+0000","last_active":"2024-09-20T22:23:31.718092+0000","last_peered":"2024-09-20T22:23:31.718092+0000","last_clean":"2024-09-20T22:23:31.718092+0000","last_became_active":"2024-09-20T22:22:36.732727+0000","last_became_peered":"2024-09-20T22:22:36.732727+0000","last_unstale":"2024-09-20T22:23:31.718092+0000","last_undegraded":"2024-09-20T22:23:31.718092+0000","last_fullsized":"2024-09-20T22:23:31.718092+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:48.688079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:48.688079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:02.631329+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1189,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062825+0000","last_change":"2024-09-20T22:23:23.138971+0000","last_active":"2024-09-20T22:23:31.062825+0000","last_peered":"2024-09-20T22:23:31.062825+0000","last_clean":"2024-09-20T22:23:31.062825+0000","last_became_active":"2024-09-20T22:22:34.743055+0000","last_became_peered":"2024-09-20T22:22:34.743055+0000","last_unstale":"2024-09-20T22:23:31.062825+0000","last_undegraded":"2024-09-20T22:23:31.062825+0000","last_fullsized":"2024-09-20T22:23:31.062825+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:23.138880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:35.146554+0000","last_clean_scrub_stamp":"2024-09-20T22:23:23.138880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:45.175131+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"579'25","reported_seq":37,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718979+0000","last_change":"2024-09-20T22:23:10.785953+0000","last_active":"2024-09-20T22:23:31.718979+0000","last_peered":"2024-09-20T22:23:31.718979+0000","last_clean":"2024-09-20T22:23:31.718979+0000","last_became_active":"2024-09-20T22:23:10.777633+0000","last_became_peered":"2024-09-20T22:23:10.777633+0000","last_unstale":"2024-09-20T22:23:31.718979+0000","last_undegraded":"2024-09-20T22:23:31.718979+0000","last_fullsized":"2024-09-20T22:23:31.718979+0000","mapping_epoch":587,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":588,"parent":"0.0","parent_split_bits":0,"last_scrub":"568'19","last_scrub_stamp":"2024-09-20T22:22:40.110956+0000","last_deep_scrub":"568'19","last_deep_scrub_stamp":"2024-09-20T22:22:40.110956+0000","last_clean_scrub_stamp":"2024-09-20T22:22:40.110956+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:02.123233+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":690,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062620+0000","last_change":"2024-09-20T22:23:24.139950+0000","last_active":"2024-09-20T22:23:31.062620+0000","last_peered":"2024-09-20T22:23:31.062620+0000","last_clean":"2024-09-20T22:23:31.062620+0000","last_became_active":"2024-09-20T22:12:52.518549+0000","last_became_peered":"2024-09-20T22:12:52.518549+0000","last_unstale":"2024-09-20T22:23:31.062620+0000","last_undegraded":"2024-09-20T22:23:31.062620+0000","last_fullsized":"2024-09-20T22:23:31.062620+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:24.139844+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:23:24.139844+0000","last_clean_scrub_stamp":"2024-09-20T22:23:24.139844+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:48.864393+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":117,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.717736+0000","last_change":"2024-09-20T22:22:46.044116+0000","last_active":"2024-09-20T22:23:31.717736+0000","last_peered":"2024-09-20T22:23:31.717736+0000","last_clean":"2024-09-20T22:23:31.717736+0000","last_became_active":"2024-09-20T22:22:36.747092+0000","last_became_peered":"2024-09-20T22:22:36.747092+0000","last_unstale":"2024-09-20T22:23:31.717736+0000","last_undegraded":"2024-09-20T22:23:31.717736+0000","last_fullsized":"2024-09-20T22:23:31.717736+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:46.044046+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:37.934760+0000","last_clean_scrub_stamp":"2024-09-20T22:22:46.044046+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:56.375187+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1e","version":"75'128","reported_seq":676,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952206+0000","last_change":"2024-09-20T22:22:33.741991+0000","last_active":"2024-09-20T22:23:30.952206+0000","last_peered":"2024-09-20T22:23:30.952206+0000","last_clean":"2024-09-20T22:23:30.952206+0000","last_became_active":"2024-09-20T22:22:33.741612+0000","last_became_peered":"2024-09-20T22:22:33.741612+0000","last_unstale":"2024-09-20T22:23:30.952206+0000","last_undegraded":"2024-09-20T22:23:30.952206+0000","last_fullsized":"2024-09-20T22:23:30.952206+0000","mapping_epoch":561,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":562,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:19.051625+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:17:40.367940+0000","last_clean_scrub_stamp":"2024-09-20T22:22:19.051625+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:40.530410+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,7],"acting":[3,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1192,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062869+0000","last_change":"2024-09-20T22:22:36.743983+0000","last_active":"2024-09-20T22:23:31.062869+0000","last_peered":"2024-09-20T22:23:31.062869+0000","last_clean":"2024-09-20T22:23:31.062869+0000","last_became_active":"2024-09-20T22:22:36.743656+0000","last_became_peered":"2024-09-20T22:22:36.743656+0000","last_unstale":"2024-09-20T22:23:31.062869+0000","last_undegraded":"2024-09-20T22:23:31.062869+0000","last_fullsized":"2024-09-20T22:23:31.062869+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:32.037246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:30.215263+0000","last_clean_scrub_stamp":"2024-09-20T22:22:32.037246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:41.815514+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"579'9","reported_seq":105,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718729+0000","last_change":"2024-09-20T22:23:12.747136+0000","last_active":"2024-09-20T22:23:31.718729+0000","last_peered":"2024-09-20T22:23:31.718729+0000","last_clean":"2024-09-20T22:23:31.718729+0000","last_became_active":"2024-09-20T22:22:36.732453+0000","last_became_peered":"2024-09-20T22:22:36.732453+0000","last_unstale":"2024-09-20T22:23:31.718729+0000","last_undegraded":"2024-09-20T22:23:31.718729+0000","last_fullsized":"2024-09-20T22:23:31.718729+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'9","last_scrub_stamp":"2024-09-20T22:23:12.747038+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:23:12.747038+0000","objects_scrubbed":2,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:19.255811+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":12,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1183,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.958849+0000","last_change":"2024-09-20T22:23:19.597163+0000","last_active":"2024-09-20T22:23:31.958849+0000","last_peered":"2024-09-20T22:23:31.958849+0000","last_clean":"2024-09-20T22:23:31.958849+0000","last_became_active":"2024-09-20T22:22:36.744577+0000","last_became_peered":"2024-09-20T22:22:36.744577+0000","last_unstale":"2024-09-20T22:23:31.958849+0000","last_undegraded":"2024-09-20T22:23:31.958849+0000","last_fullsized":"2024-09-20T22:23:31.958849+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:19.597078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:23:19.597078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:35.017777+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":696,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062425+0000","last_change":"2024-09-20T22:23:21.059592+0000","last_active":"2024-09-20T22:23:31.062425+0000","last_peered":"2024-09-20T22:23:31.062425+0000","last_clean":"2024-09-20T22:23:31.062425+0000","last_became_active":"2024-09-20T22:22:36.723117+0000","last_became_peered":"2024-09-20T22:22:36.723117+0000","last_unstale":"2024-09-20T22:23:31.062425+0000","last_undegraded":"2024-09-20T22:23:31.062425+0000","last_fullsized":"2024-09-20T22:23:31.062425+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:21.059499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:29.551172+0000","last_clean_scrub_stamp":"2024-09-20T22:23:21.059499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:43.096015+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.19","version":"75'128","reported_seq":622,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952474+0000","last_change":"2024-09-20T22:22:52.005610+0000","last_active":"2024-09-20T22:23:30.952474+0000","last_peered":"2024-09-20T22:23:30.952474+0000","last_clean":"2024-09-20T22:23:30.952474+0000","last_became_active":"2024-09-20T22:22:51.340072+0000","last_became_peered":"2024-09-20T22:22:51.340072+0000","last_unstale":"2024-09-20T22:23:30.952474+0000","last_undegraded":"2024-09-20T22:23:30.952474+0000","last_fullsized":"2024-09-20T22:23:30.952474+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:52.005539+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:52.005539+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:02.049542+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":84,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719503+0000","last_change":"2024-09-20T22:23:04.752285+0000","last_active":"2024-09-20T22:23:31.719503+0000","last_peered":"2024-09-20T22:23:31.719503+0000","last_clean":"2024-09-20T22:23:31.719503+0000","last_became_active":"2024-09-20T22:22:34.743937+0000","last_became_peered":"2024-09-20T22:22:34.743937+0000","last_unstale":"2024-09-20T22:23:31.719503+0000","last_undegraded":"2024-09-20T22:23:31.719503+0000","last_fullsized":"2024-09-20T22:23:31.719503+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:04.752194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:19:44.809724+0000","last_clean_scrub_stamp":"2024-09-20T22:23:04.752194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:11.062720+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":109,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719396+0000","last_change":"2024-09-20T22:22:34.741813+0000","last_active":"2024-09-20T22:23:31.719396+0000","last_peered":"2024-09-20T22:23:31.719396+0000","last_clean":"2024-09-20T22:23:31.719396+0000","last_became_active":"2024-09-20T22:22:34.741335+0000","last_became_peered":"2024-09-20T22:22:34.741335+0000","last_unstale":"2024-09-20T22:23:31.719396+0000","last_undegraded":"2024-09-20T22:23:31.719396+0000","last_fullsized":"2024-09-20T22:23:31.719396+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:11.655870+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:34.814930+0000","last_clean_scrub_stamp":"2024-09-20T22:22:11.655870+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:39.733864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"579'46","reported_seq":806,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959731+0000","last_change":"2024-09-20T22:23:26.769670+0000","last_active":"2024-09-20T22:23:31.959731+0000","last_peered":"2024-09-20T22:23:31.959731+0000","last_clean":"2024-09-20T22:23:31.959731+0000","last_became_active":"2024-09-20T22:22:36.735934+0000","last_became_peered":"2024-09-20T22:22:36.735934+0000","last_unstale":"2024-09-20T22:23:31.959731+0000","last_undegraded":"2024-09-20T22:23:31.959731+0000","last_fullsized":"2024-09-20T22:23:31.959731+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'46","last_scrub_stamp":"2024-09-20T22:23:26.769577+0000","last_deep_scrub":"579'46","last_deep_scrub_stamp":"2024-09-20T22:23:26.769577+0000","last_clean_scrub_stamp":"2024-09-20T22:23:26.769577+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:55.782563+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":81,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719156+0000","last_change":"2024-09-20T22:23:03.207880+0000","last_active":"2024-09-20T22:23:31.719156+0000","last_peered":"2024-09-20T22:23:31.719156+0000","last_clean":"2024-09-20T22:23:31.719156+0000","last_became_active":"2024-09-20T22:22:36.748783+0000","last_became_peered":"2024-09-20T22:22:36.748783+0000","last_unstale":"2024-09-20T22:23:31.719156+0000","last_undegraded":"2024-09-20T22:23:31.719156+0000","last_fullsized":"2024-09-20T22:23:31.719156+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:03.207810+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:44.295028+0000","last_clean_scrub_stamp":"2024-09-20T22:23:03.207810+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:17.801439+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.18","version":"75'128","reported_seq":757,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062197+0000","last_change":"2024-09-20T22:22:31.033583+0000","last_active":"2024-09-20T22:23:31.062197+0000","last_peered":"2024-09-20T22:23:31.062197+0000","last_clean":"2024-09-20T22:23:31.062197+0000","last_became_active":"2024-09-20T22:17:44.717551+0000","last_became_peered":"2024-09-20T22:17:44.717551+0000","last_unstale":"2024-09-20T22:23:31.062197+0000","last_undegraded":"2024-09-20T22:23:31.062197+0000","last_fullsized":"2024-09-20T22:23:31.062197+0000","mapping_epoch":433,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":434,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:31.033455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:31.033455+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:52.517058+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":262688,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":585,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952634+0000","last_change":"2024-09-20T22:22:51.735400+0000","last_active":"2024-09-20T22:23:30.952634+0000","last_peered":"2024-09-20T22:23:30.952634+0000","last_clean":"2024-09-20T22:23:30.952634+0000","last_became_active":"2024-09-20T22:22:51.735132+0000","last_became_peered":"2024-09-20T22:22:51.735132+0000","last_unstale":"2024-09-20T22:23:30.952634+0000","last_undegraded":"2024-09-20T22:23:30.952634+0000","last_fullsized":"2024-09-20T22:23:30.952634+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":576,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:46.003147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:46.003147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:10.763248+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":82,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718480+0000","last_change":"2024-09-20T22:22:36.732933+0000","last_active":"2024-09-20T22:23:31.718480+0000","last_peered":"2024-09-20T22:23:31.718480+0000","last_clean":"2024-09-20T22:23:31.718480+0000","last_became_active":"2024-09-20T22:22:36.732366+0000","last_became_peered":"2024-09-20T22:22:36.732366+0000","last_unstale":"2024-09-20T22:23:31.718480+0000","last_undegraded":"2024-09-20T22:23:31.718480+0000","last_fullsized":"2024-09-20T22:23:31.718480+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:21.993777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:44.456415+0000","last_clean_scrub_stamp":"2024-09-20T22:22:21.993777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:49.795468+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"579'29","reported_seq":540,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952145+0000","last_change":"2024-09-20T22:22:51.338089+0000","last_active":"2024-09-20T22:23:30.952145+0000","last_peered":"2024-09-20T22:23:30.952145+0000","last_clean":"2024-09-20T22:23:30.952145+0000","last_became_active":"2024-09-20T22:22:51.337739+0000","last_became_peered":"2024-09-20T22:22:51.337739+0000","last_unstale":"2024-09-20T22:23:30.952145+0000","last_undegraded":"2024-09-20T22:23:30.952145+0000","last_fullsized":"2024-09-20T22:23:30.952145+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'23","last_scrub_stamp":"2024-09-20T22:22:10.697295+0000","last_deep_scrub":"250'17","last_deep_scrub_stamp":"2024-09-20T22:10:27.157501+0000","last_clean_scrub_stamp":"2024-09-20T22:22:10.697295+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:36.409961+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":42,"num_read_kb":33,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":84,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:32.814440+0000","last_change":"2024-09-20T22:23:32.814464+0000","last_active":"2024-09-20T22:23:32.814440+0000","last_peered":"2024-09-20T22:23:32.814440+0000","last_clean":"2024-09-20T22:23:32.814440+0000","last_became_active":"2024-09-20T22:22:34.756273+0000","last_became_peered":"2024-09-20T22:22:34.756273+0000","last_unstale":"2024-09-20T22:23:32.814440+0000","last_undegraded":"2024-09-20T22:23:32.814440+0000","last_fullsized":"2024-09-20T22:23:32.814440+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:32.814357+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:28.003183+0000","last_clean_scrub_stamp":"2024-09-20T22:23:32.814357+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:41.502219+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1b","version":"75'128","reported_seq":1079,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062774+0000","last_change":"2024-09-20T22:22:37.725081+0000","last_active":"2024-09-20T22:23:31.062774+0000","last_peered":"2024-09-20T22:23:31.062774+0000","last_clean":"2024-09-20T22:23:31.062774+0000","last_became_active":"2024-09-20T22:22:37.724684+0000","last_became_peered":"2024-09-20T22:22:37.724684+0000","last_unstale":"2024-09-20T22:23:31.062774+0000","last_undegraded":"2024-09-20T22:23:31.062774+0000","last_fullsized":"2024-09-20T22:23:31.062774+0000","mapping_epoch":565,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":566,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:15.018011+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:22:15.018011+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:35.067218+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,0],"acting":[5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"113'2","reported_seq":1128,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.958821+0000","last_change":"2024-09-20T22:22:35.623837+0000","last_active":"2024-09-20T22:23:31.958821+0000","last_peered":"2024-09-20T22:23:31.958821+0000","last_clean":"2024-09-20T22:23:31.958821+0000","last_became_active":"2024-09-20T22:02:56.301332+0000","last_became_peered":"2024-09-20T22:02:56.301332+0000","last_unstale":"2024-09-20T22:23:31.958821+0000","last_undegraded":"2024-09-20T22:23:31.958821+0000","last_fullsized":"2024-09-20T22:23:31.958821+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":70,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:22:35.623755+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:05:33.751883+0000","last_clean_scrub_stamp":"2024-09-20T22:22:35.623755+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:43.896324+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":403,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.19","version":"289'2","reported_seq":77,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718934+0000","last_change":"2024-09-20T22:22:36.771205+0000","last_active":"2024-09-20T22:23:31.718934+0000","last_peered":"2024-09-20T22:23:31.718934+0000","last_clean":"2024-09-20T22:23:31.718934+0000","last_became_active":"2024-09-20T22:22:36.746942+0000","last_became_peered":"2024-09-20T22:22:36.746942+0000","last_unstale":"2024-09-20T22:23:31.718934+0000","last_undegraded":"2024-09-20T22:23:31.718934+0000","last_fullsized":"2024-09-20T22:23:31.718934+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"289'2","last_scrub_stamp":"2024-09-20T22:22:26.927516+0000","last_deep_scrub":"289'2","last_deep_scrub_stamp":"2024-09-20T22:22:26.927516+0000","last_clean_scrub_stamp":"2024-09-20T22:22:26.927516+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:47.270285+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":230,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"579'30","reported_seq":634,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.720297+0000","last_change":"2024-09-20T22:22:39.067136+0000","last_active":"2024-09-20T22:23:31.720297+0000","last_peered":"2024-09-20T22:23:31.720297+0000","last_clean":"2024-09-20T22:23:31.720297+0000","last_became_active":"2024-09-20T22:12:52.526803+0000","last_became_peered":"2024-09-20T22:12:52.526803+0000","last_unstale":"2024-09-20T22:23:31.720297+0000","last_undegraded":"2024-09-20T22:23:31.720297+0000","last_fullsized":"2024-09-20T22:23:31.720297+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'20","last_scrub_stamp":"2024-09-20T22:22:39.067058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:39.067058+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:51.512889+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"563'15","reported_seq":121,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.961414+0000","last_change":"2024-09-20T22:22:49.948730+0000","last_active":"2024-09-20T22:23:31.961414+0000","last_peered":"2024-09-20T22:23:31.961414+0000","last_clean":"2024-09-20T22:23:31.961414+0000","last_became_active":"2024-09-20T22:22:34.751023+0000","last_became_peered":"2024-09-20T22:22:34.751023+0000","last_unstale":"2024-09-20T22:23:31.961414+0000","last_undegraded":"2024-09-20T22:23:31.961414+0000","last_fullsized":"2024-09-20T22:23:31.961414+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"563'15","last_scrub_stamp":"2024-09-20T22:22:49.948627+0000","last_deep_scrub":"291'11","last_deep_scrub_stamp":"2024-09-20T22:11:41.372195+0000","last_clean_scrub_stamp":"2024-09-20T22:22:49.948627+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:05.343938+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.a","version":"75'128","reported_seq":1170,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062483+0000","last_change":"2024-09-20T22:22:33.745681+0000","last_active":"2024-09-20T22:23:31.062483+0000","last_peered":"2024-09-20T22:23:31.062483+0000","last_clean":"2024-09-20T22:23:31.062483+0000","last_became_active":"2024-09-20T22:22:33.745348+0000","last_became_peered":"2024-09-20T22:22:33.745348+0000","last_unstale":"2024-09-20T22:23:31.062483+0000","last_undegraded":"2024-09-20T22:23:31.062483+0000","last_fullsized":"2024-09-20T22:23:31.062483+0000","mapping_epoch":561,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":562,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:13.011208+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:13.011208+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:32.864474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,6],"acting":[5,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"579'39","reported_seq":108,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.720191+0000","last_change":"2024-09-20T22:22:36.781358+0000","last_active":"2024-09-20T22:23:31.720191+0000","last_peered":"2024-09-20T22:23:31.720191+0000","last_clean":"2024-09-20T22:23:31.720191+0000","last_became_active":"2024-09-20T22:22:36.750028+0000","last_became_peered":"2024-09-20T22:22:36.750028+0000","last_unstale":"2024-09-20T22:23:31.720191+0000","last_undegraded":"2024-09-20T22:23:31.720191+0000","last_fullsized":"2024-09-20T22:23:31.720191+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'27","last_scrub_stamp":"2024-09-20T22:22:17.506272+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:17.506272+0000","objects_scrubbed":7,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":60,"num_read_kb":51,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1186,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952055+0000","last_change":"2024-09-20T22:23:22.126785+0000","last_active":"2024-09-20T22:23:30.952055+0000","last_peered":"2024-09-20T22:23:30.952055+0000","last_clean":"2024-09-20T22:23:30.952055+0000","last_became_active":"2024-09-20T22:22:36.735340+0000","last_became_peered":"2024-09-20T22:22:36.735340+0000","last_unstale":"2024-09-20T22:23:30.952055+0000","last_undegraded":"2024-09-20T22:23:30.952055+0000","last_fullsized":"2024-09-20T22:23:30.952055+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:22.126699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:19:17.513228+0000","last_clean_scrub_stamp":"2024-09-20T22:23:22.126699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:42.027891+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"563'15","reported_seq":100,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719350+0000","last_change":"2024-09-20T22:22:36.728747+0000","last_active":"2024-09-20T22:23:31.719350+0000","last_peered":"2024-09-20T22:23:31.719350+0000","last_clean":"2024-09-20T22:23:31.719350+0000","last_became_active":"2024-09-20T22:22:34.741247+0000","last_became_peered":"2024-09-20T22:22:34.741247+0000","last_unstale":"2024-09-20T22:23:31.719350+0000","last_undegraded":"2024-09-20T22:23:31.719350+0000","last_fullsized":"2024-09-20T22:23:31.719350+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"563'15","last_scrub_stamp":"2024-09-20T22:22:36.728677+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:36.728677+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:44.364115+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":695,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.063281+0000","last_change":"2024-09-20T22:22:36.742379+0000","last_active":"2024-09-20T22:23:31.063281+0000","last_peered":"2024-09-20T22:23:31.063281+0000","last_clean":"2024-09-20T22:23:31.063281+0000","last_became_active":"2024-09-20T22:22:36.741883+0000","last_became_peered":"2024-09-20T22:22:36.741883+0000","last_unstale":"2024-09-20T22:23:31.063281+0000","last_undegraded":"2024-09-20T22:23:31.063281+0000","last_fullsized":"2024-09-20T22:23:31.063281+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:24.110670+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:24.110670+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:49.044230+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.5","version":"75'128","reported_seq":581,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719425+0000","last_change":"2024-09-20T22:23:05.709713+0000","last_active":"2024-09-20T22:23:31.719425+0000","last_peered":"2024-09-20T22:23:31.719425+0000","last_clean":"2024-09-20T22:23:31.719425+0000","last_became_active":"2024-09-20T22:22:37.719212+0000","last_became_peered":"2024-09-20T22:22:37.719212+0000","last_unstale":"2024-09-20T22:23:31.719425+0000","last_undegraded":"2024-09-20T22:23:31.719425+0000","last_fullsized":"2024-09-20T22:23:31.719425+0000","mapping_epoch":565,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":566,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:23:05.709618+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:23:05.709618+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:32.710626+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,6],"acting":[4,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"579'42","reported_seq":655,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.951756+0000","last_change":"2024-09-20T22:22:53.008769+0000","last_active":"2024-09-20T22:23:30.951756+0000","last_peered":"2024-09-20T22:23:30.951756+0000","last_clean":"2024-09-20T22:23:30.951756+0000","last_became_active":"2024-09-20T22:22:51.342407+0000","last_became_peered":"2024-09-20T22:22:51.342407+0000","last_unstale":"2024-09-20T22:23:30.951756+0000","last_undegraded":"2024-09-20T22:23:30.951756+0000","last_fullsized":"2024-09-20T22:23:30.951756+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'30","last_scrub_stamp":"2024-09-20T22:22:53.008206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:53.008206+0000","objects_scrubbed":8,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:56.046438+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":69,"num_read_kb":57,"num_write":44,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":660,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":584,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952888+0000","last_change":"2024-09-20T22:22:53.961979+0000","last_active":"2024-09-20T22:23:30.952888+0000","last_peered":"2024-09-20T22:23:30.952888+0000","last_clean":"2024-09-20T22:23:30.952888+0000","last_became_active":"2024-09-20T22:22:51.342358+0000","last_became_peered":"2024-09-20T22:22:51.342358+0000","last_unstale":"2024-09-20T22:23:30.952888+0000","last_undegraded":"2024-09-20T22:23:30.952888+0000","last_fullsized":"2024-09-20T22:23:30.952888+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:53.961902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:22:53.961902+0000","last_clean_scrub_stamp":"2024-09-20T22:22:53.961902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:55.672484+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":84,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718126+0000","last_change":"2024-09-20T22:23:29.798841+0000","last_active":"2024-09-20T22:23:31.718126+0000","last_peered":"2024-09-20T22:23:31.718126+0000","last_clean":"2024-09-20T22:23:31.718126+0000","last_became_active":"2024-09-20T22:22:34.752645+0000","last_became_peered":"2024-09-20T22:22:34.752645+0000","last_unstale":"2024-09-20T22:23:31.718126+0000","last_undegraded":"2024-09-20T22:23:31.718126+0000","last_fullsized":"2024-09-20T22:23:31.718126+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:29.798753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:23:29.798753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:46.871379+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":84,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719103+0000","last_change":"2024-09-20T22:23:03.778957+0000","last_active":"2024-09-20T22:23:31.719103+0000","last_peered":"2024-09-20T22:23:31.719103+0000","last_clean":"2024-09-20T22:23:31.719103+0000","last_became_active":"2024-09-20T22:22:34.748356+0000","last_became_peered":"2024-09-20T22:22:34.748356+0000","last_unstale":"2024-09-20T22:23:31.719103+0000","last_undegraded":"2024-09-20T22:23:31.719103+0000","last_fullsized":"2024-09-20T22:23:31.719103+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:03.778866+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:23:03.778866+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:08.518592+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"75'128","reported_seq":1202,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952100+0000","last_change":"2024-09-20T22:23:29.019475+0000","last_active":"2024-09-20T22:23:30.952100+0000","last_peered":"2024-09-20T22:23:30.952100+0000","last_clean":"2024-09-20T22:23:30.952100+0000","last_became_active":"2024-09-20T22:15:45.642671+0000","last_became_peered":"2024-09-20T22:15:45.642671+0000","last_unstale":"2024-09-20T22:23:30.952100+0000","last_undegraded":"2024-09-20T22:23:30.952100+0000","last_fullsized":"2024-09-20T22:23:30.952100+0000","mapping_epoch":285,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:23:29.019401+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:23:29.019401+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:52.994019+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,7],"acting":[3,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"579'42","reported_seq":121,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718538+0000","last_change":"2024-09-20T22:22:55.742297+0000","last_active":"2024-09-20T22:23:31.718538+0000","last_peered":"2024-09-20T22:23:31.718538+0000","last_clean":"2024-09-20T22:23:31.718538+0000","last_became_active":"2024-09-20T22:22:34.744774+0000","last_became_peered":"2024-09-20T22:22:34.744774+0000","last_unstale":"2024-09-20T22:23:31.718538+0000","last_undegraded":"2024-09-20T22:23:31.718538+0000","last_fullsized":"2024-09-20T22:23:31.718538+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'28","last_scrub_stamp":"2024-09-20T22:22:55.742181+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:55.742181+0000","objects_scrubbed":7,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:04.491052+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":55,"num_write":42,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"64'1","reported_seq":643,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.720397+0000","last_change":"2024-09-20T22:22:56.184568+0000","last_active":"2024-09-20T22:23:31.720397+0000","last_peered":"2024-09-20T22:23:31.720397+0000","last_clean":"2024-09-20T22:23:31.720397+0000","last_became_active":"2024-09-20T22:22:36.748251+0000","last_became_peered":"2024-09-20T22:22:36.748251+0000","last_unstale":"2024-09-20T22:23:31.720397+0000","last_undegraded":"2024-09-20T22:23:31.720397+0000","last_fullsized":"2024-09-20T22:23:31.720397+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:22:56.184492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:56.184492+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:23.711516+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":276,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.0","version":"575'16","reported_seq":724,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.951627+0000","last_change":"2024-09-20T22:22:56.994140+0000","last_active":"2024-09-20T22:23:30.951627+0000","last_peered":"2024-09-20T22:23:30.951627+0000","last_clean":"2024-09-20T22:23:30.951627+0000","last_became_active":"2024-09-20T22:22:51.338769+0000","last_became_peered":"2024-09-20T22:22:51.338769+0000","last_unstale":"2024-09-20T22:23:30.951627+0000","last_undegraded":"2024-09-20T22:23:30.951627+0000","last_fullsized":"2024-09-20T22:23:30.951627+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"575'16","last_scrub_stamp":"2024-09-20T22:22:56.994064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:56.994064+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:00.016540+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1139,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959195+0000","last_change":"2024-09-20T22:22:59.568901+0000","last_active":"2024-09-20T22:23:31.959195+0000","last_peered":"2024-09-20T22:23:31.959195+0000","last_clean":"2024-09-20T22:23:31.959195+0000","last_became_active":"2024-09-20T22:22:30.672451+0000","last_became_peered":"2024-09-20T22:22:30.672451+0000","last_unstale":"2024-09-20T22:23:31.959195+0000","last_undegraded":"2024-09-20T22:23:31.959195+0000","last_fullsized":"2024-09-20T22:23:31.959195+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:59.568808+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:59.568808+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:12.462194+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"75'128","reported_seq":381,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.961297+0000","last_change":"2024-09-20T22:22:33.815459+0000","last_active":"2024-09-20T22:23:31.961297+0000","last_peered":"2024-09-20T22:23:31.961297+0000","last_clean":"2024-09-20T22:23:31.961297+0000","last_became_active":"2024-09-20T22:22:33.730147+0000","last_became_peered":"2024-09-20T22:22:33.730147+0000","last_unstale":"2024-09-20T22:23:31.961297+0000","last_undegraded":"2024-09-20T22:23:31.961297+0000","last_fullsized":"2024-09-20T22:23:31.961297+0000","mapping_epoch":561,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":562,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:33.815352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:33.815352+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:58.920874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,7],"acting":[6,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2","version":"579'30","reported_seq":108,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719919+0000","last_change":"2024-09-20T22:23:04.233663+0000","last_active":"2024-09-20T22:23:31.719919+0000","last_peered":"2024-09-20T22:23:31.719919+0000","last_clean":"2024-09-20T22:23:31.719919+0000","last_became_active":"2024-09-20T22:22:36.748048+0000","last_became_peered":"2024-09-20T22:22:36.748048+0000","last_unstale":"2024-09-20T22:23:31.719919+0000","last_undegraded":"2024-09-20T22:23:31.719919+0000","last_fullsized":"2024-09-20T22:23:31.719919+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'30","last_scrub_stamp":"2024-09-20T22:23:04.233577+0000","last_deep_scrub":"579'30","last_deep_scrub_stamp":"2024-09-20T22:23:04.233577+0000","last_clean_scrub_stamp":"2024-09-20T22:23:04.233577+0000","objects_scrubbed":5,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:27.841823+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1160,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062885+0000","last_change":"2024-09-20T22:23:08.015717+0000","last_active":"2024-09-20T22:23:31.062885+0000","last_peered":"2024-09-20T22:23:31.062885+0000","last_clean":"2024-09-20T22:23:31.062885+0000","last_became_active":"2024-09-20T22:02:50.114139+0000","last_became_peered":"2024-09-20T22:02:50.114139+0000","last_unstale":"2024-09-20T22:23:31.062885+0000","last_undegraded":"2024-09-20T22:23:31.062885+0000","last_fullsized":"2024-09-20T22:23:31.062885+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:08.015622+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:43.597924+0000","last_clean_scrub_stamp":"2024-09-20T22:23:08.015622+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:21.120384+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3","version":"558'15","reported_seq":742,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719952+0000","last_change":"2024-09-20T22:22:58.192796+0000","last_active":"2024-09-20T22:23:31.719952+0000","last_peered":"2024-09-20T22:23:31.719952+0000","last_clean":"2024-09-20T22:23:31.719952+0000","last_became_active":"2024-09-20T22:22:30.665240+0000","last_became_peered":"2024-09-20T22:22:30.665240+0000","last_unstale":"2024-09-20T22:23:31.719952+0000","last_undegraded":"2024-09-20T22:23:31.719952+0000","last_fullsized":"2024-09-20T22:23:31.719952+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"558'15","last_scrub_stamp":"2024-09-20T22:22:58.192698+0000","last_deep_scrub":"363'13","last_deep_scrub_stamp":"2024-09-20T22:14:52.612615+0000","last_clean_scrub_stamp":"2024-09-20T22:22:58.192698+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:23.572479+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":592,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.717837+0000","last_change":"2024-09-20T22:23:02.192742+0000","last_active":"2024-09-20T22:23:31.717837+0000","last_peered":"2024-09-20T22:23:31.717837+0000","last_clean":"2024-09-20T22:23:31.717837+0000","last_became_active":"2024-09-20T22:12:52.518529+0000","last_became_peered":"2024-09-20T22:12:52.518529+0000","last_unstale":"2024-09-20T22:23:31.717837+0000","last_undegraded":"2024-09-20T22:23:31.717837+0000","last_fullsized":"2024-09-20T22:23:31.717837+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:02.192649+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:21:02.972172+0000","last_clean_scrub_stamp":"2024-09-20T22:23:02.192649+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:23.377281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"75'128","reported_seq":686,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.960738+0000","last_change":"2024-09-20T22:22:33.747123+0000","last_active":"2024-09-20T22:23:31.960738+0000","last_peered":"2024-09-20T22:23:31.960738+0000","last_clean":"2024-09-20T22:23:31.960738+0000","last_became_active":"2024-09-20T22:22:33.746751+0000","last_became_peered":"2024-09-20T22:22:33.746751+0000","last_unstale":"2024-09-20T22:23:31.960738+0000","last_undegraded":"2024-09-20T22:23:31.960738+0000","last_fullsized":"2024-09-20T22:23:31.960738+0000","mapping_epoch":561,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":562,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:25.085071+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:25.085071+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:51.350420+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,3],"acting":[6,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"579'47","reported_seq":648,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719664+0000","last_change":"2024-09-20T22:22:21.039662+0000","last_active":"2024-09-20T22:23:31.719664+0000","last_peered":"2024-09-20T22:23:31.719664+0000","last_clean":"2024-09-20T22:23:31.719664+0000","last_became_active":"2024-09-20T22:12:52.525538+0000","last_became_peered":"2024-09-20T22:12:52.525538+0000","last_unstale":"2024-09-20T22:23:31.719664+0000","last_undegraded":"2024-09-20T22:23:31.719664+0000","last_fullsized":"2024-09-20T22:23:31.719664+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"392'37","last_scrub_stamp":"2024-09-20T22:22:21.039584+0000","last_deep_scrub":"392'37","last_deep_scrub_stamp":"2024-09-20T22:15:56.127540+0000","last_clean_scrub_stamp":"2024-09-20T22:22:21.039584+0000","objects_scrubbed":8,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:45.784068+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":68,"num_read_kb":54,"num_write":42,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":73,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719168+0000","last_change":"2024-09-20T22:22:36.737910+0000","last_active":"2024-09-20T22:23:31.719168+0000","last_peered":"2024-09-20T22:23:31.719168+0000","last_clean":"2024-09-20T22:23:31.719168+0000","last_became_active":"2024-09-20T22:22:36.737560+0000","last_became_peered":"2024-09-20T22:22:36.737560+0000","last_unstale":"2024-09-20T22:23:31.719168+0000","last_undegraded":"2024-09-20T22:23:31.719168+0000","last_fullsized":"2024-09-20T22:23:31.719168+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:28.920875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:28.920875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:37.001115+0000","scrub_duration":15,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":94,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.960881+0000","last_change":"2024-09-20T22:22:56.022918+0000","last_active":"2024-09-20T22:23:31.960881+0000","last_peered":"2024-09-20T22:23:31.960881+0000","last_clean":"2024-09-20T22:23:31.960881+0000","last_became_active":"2024-09-20T22:22:30.678085+0000","last_became_peered":"2024-09-20T22:22:30.678085+0000","last_unstale":"2024-09-20T22:23:31.960881+0000","last_undegraded":"2024-09-20T22:23:31.960881+0000","last_fullsized":"2024-09-20T22:23:31.960881+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:56.022829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:08.234520+0000","last_clean_scrub_stamp":"2024-09-20T22:22:56.022829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:59.190286+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":81,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718784+0000","last_change":"2024-09-20T22:23:29.164346+0000","last_active":"2024-09-20T22:23:31.718784+0000","last_peered":"2024-09-20T22:23:31.718784+0000","last_clean":"2024-09-20T22:23:31.718784+0000","last_became_active":"2024-09-20T22:22:36.729183+0000","last_became_peered":"2024-09-20T22:22:36.729183+0000","last_unstale":"2024-09-20T22:23:31.718784+0000","last_undegraded":"2024-09-20T22:23:31.718784+0000","last_fullsized":"2024-09-20T22:23:31.718784+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:29.164246+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:23:29.164246+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:47.523982+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"75'128","reported_seq":1171,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062326+0000","last_change":"2024-09-20T22:22:56.865975+0000","last_active":"2024-09-20T22:23:31.062326+0000","last_peered":"2024-09-20T22:23:31.062326+0000","last_clean":"2024-09-20T22:23:31.062326+0000","last_became_active":"2024-09-20T22:15:45.627135+0000","last_became_peered":"2024-09-20T22:15:45.627135+0000","last_unstale":"2024-09-20T22:23:31.062326+0000","last_undegraded":"2024-09-20T22:23:31.062326+0000","last_fullsized":"2024-09-20T22:23:31.062326+0000","mapping_epoch":326,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":384,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:56.865872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:56.865872+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:17.244735+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2],"acting":[5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"579'68","reported_seq":133,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718815+0000","last_change":"2024-09-20T22:23:17.244337+0000","last_active":"2024-09-20T22:23:31.718815+0000","last_peered":"2024-09-20T22:23:31.718815+0000","last_clean":"2024-09-20T22:23:31.718815+0000","last_became_active":"2024-09-20T22:22:36.732949+0000","last_became_peered":"2024-09-20T22:22:36.732949+0000","last_unstale":"2024-09-20T22:23:31.718815+0000","last_undegraded":"2024-09-20T22:23:31.718815+0000","last_fullsized":"2024-09-20T22:23:31.718815+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'68","last_scrub_stamp":"2024-09-20T22:23:17.244238+0000","last_deep_scrub":"328'52","last_deep_scrub_stamp":"2024-09-20T22:17:08.395578+0000","last_clean_scrub_stamp":"2024-09-20T22:23:17.244238+0000","objects_scrubbed":10,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:19.448842+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":82,"num_read_kb":67,"num_write":50,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40,"num_bytes_recovered":1432,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":84,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719177+0000","last_change":"2024-09-20T22:23:13.750879+0000","last_active":"2024-09-20T22:23:31.719177+0000","last_peered":"2024-09-20T22:23:31.719177+0000","last_clean":"2024-09-20T22:23:31.719177+0000","last_became_active":"2024-09-20T22:22:34.745230+0000","last_became_peered":"2024-09-20T22:22:34.745230+0000","last_unstale":"2024-09-20T22:23:31.719177+0000","last_undegraded":"2024-09-20T22:23:31.719177+0000","last_fullsized":"2024-09-20T22:23:31.719177+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:13.750775+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:23:13.750775+0000","last_clean_scrub_stamp":"2024-09-20T22:23:13.750775+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:34.016719+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"162'3","reported_seq":1196,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062756+0000","last_change":"2024-09-20T22:23:11.052803+0000","last_active":"2024-09-20T22:23:31.062756+0000","last_peered":"2024-09-20T22:23:31.062756+0000","last_clean":"2024-09-20T22:23:31.062756+0000","last_became_active":"2024-09-20T22:22:36.744910+0000","last_became_peered":"2024-09-20T22:22:36.744910+0000","last_unstale":"2024-09-20T22:23:31.062756+0000","last_undegraded":"2024-09-20T22:23:31.062756+0000","last_fullsized":"2024-09-20T22:23:31.062756+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"162'3","last_scrub_stamp":"2024-09-20T22:23:11.052708+0000","last_deep_scrub":"162'3","last_deep_scrub_stamp":"2024-09-20T22:13:51.454850+0000","last_clean_scrub_stamp":"2024-09-20T22:23:11.052708+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:20.926715+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.002896062,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":76,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":628,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719839+0000","last_change":"2024-09-20T22:22:36.746533+0000","last_active":"2024-09-20T22:23:31.719839+0000","last_peered":"2024-09-20T22:23:31.719839+0000","last_clean":"2024-09-20T22:23:31.719839+0000","last_became_active":"2024-09-20T22:22:36.746280+0000","last_became_peered":"2024-09-20T22:22:36.746280+0000","last_unstale":"2024-09-20T22:23:31.719839+0000","last_undegraded":"2024-09-20T22:23:31.719839+0000","last_fullsized":"2024-09-20T22:23:31.719839+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:31.035693+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:31.035693+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:55.917496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1169,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952434+0000","last_change":"2024-09-20T22:22:34.756279+0000","last_active":"2024-09-20T22:23:30.952434+0000","last_peered":"2024-09-20T22:23:30.952434+0000","last_clean":"2024-09-20T22:23:30.952434+0000","last_became_active":"2024-09-20T22:22:34.755781+0000","last_became_peered":"2024-09-20T22:22:34.755781+0000","last_unstale":"2024-09-20T22:23:30.952434+0000","last_undegraded":"2024-09-20T22:23:30.952434+0000","last_fullsized":"2024-09-20T22:23:30.952434+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:23.018760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:22:23.018760+0000","last_clean_scrub_stamp":"2024-09-20T22:22:23.018760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:43.284624+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"75'128","reported_seq":1188,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.958376+0000","last_change":"2024-09-20T22:22:37.721200+0000","last_active":"2024-09-20T22:23:31.958376+0000","last_peered":"2024-09-20T22:23:31.958376+0000","last_clean":"2024-09-20T22:23:31.958376+0000","last_became_active":"2024-09-20T22:22:37.720715+0000","last_became_peered":"2024-09-20T22:22:37.720715+0000","last_unstale":"2024-09-20T22:23:31.958376+0000","last_undegraded":"2024-09-20T22:23:31.958376+0000","last_fullsized":"2024-09-20T22:23:31.958376+0000","mapping_epoch":565,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":566,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:29.535404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:29.535404+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:47.788568+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"579'21","reported_seq":31,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.957876+0000","last_change":"2024-09-20T22:23:23.838224+0000","last_active":"2024-09-20T22:23:31.957876+0000","last_peered":"2024-09-20T22:23:31.957876+0000","last_clean":"2024-09-20T22:23:31.957876+0000","last_became_active":"2024-09-20T22:23:23.829978+0000","last_became_peered":"2024-09-20T22:23:23.829978+0000","last_unstale":"2024-09-20T22:23:31.957876+0000","last_undegraded":"2024-09-20T22:23:31.957876+0000","last_fullsized":"2024-09-20T22:23:31.957876+0000","mapping_epoch":591,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":592,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'21","last_scrub_stamp":"2024-09-20T22:23:08.286952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:23:08.286952+0000","objects_scrubbed":5,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:23.970165+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":39,"num_read_kb":28,"num_write":24,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":81,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:32.073109+0000","last_change":"2024-09-20T22:23:32.073127+0000","last_active":"2024-09-20T22:23:32.073109+0000","last_peered":"2024-09-20T22:23:32.073109+0000","last_clean":"2024-09-20T22:23:32.073109+0000","last_became_active":"2024-09-20T22:22:36.742021+0000","last_became_peered":"2024-09-20T22:22:36.742021+0000","last_unstale":"2024-09-20T22:23:32.073109+0000","last_undegraded":"2024-09-20T22:23:32.073109+0000","last_fullsized":"2024-09-20T22:23:32.073109+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:32.073057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:23:32.073057+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:43.993378+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"113'2","reported_seq":1208,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959409+0000","last_change":"2024-09-20T22:22:33.052302+0000","last_active":"2024-09-20T22:23:31.959409+0000","last_peered":"2024-09-20T22:23:31.959409+0000","last_clean":"2024-09-20T22:23:31.959409+0000","last_became_active":"2024-09-20T22:12:52.517255+0000","last_became_peered":"2024-09-20T22:12:52.517255+0000","last_unstale":"2024-09-20T22:23:31.959409+0000","last_undegraded":"2024-09-20T22:23:31.959409+0000","last_fullsized":"2024-09-20T22:23:31.959409+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"113'2","last_scrub_stamp":"2024-09-20T22:22:33.052225+0000","last_deep_scrub":"113'2","last_deep_scrub_stamp":"2024-09-20T22:08:36.250748+0000","last_clean_scrub_stamp":"2024-09-20T22:22:33.052225+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:57.127668+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00078794599999999998,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":1162,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959462+0000","last_change":"2024-09-20T22:22:37.055462+0000","last_active":"2024-09-20T22:23:31.959462+0000","last_peered":"2024-09-20T22:23:31.959462+0000","last_clean":"2024-09-20T22:23:31.959462+0000","last_became_active":"2024-09-20T22:02:54.292168+0000","last_became_peered":"2024-09-20T22:02:54.292168+0000","last_unstale":"2024-09-20T22:23:31.959462+0000","last_undegraded":"2024-09-20T22:23:31.959462+0000","last_fullsized":"2024-09-20T22:23:31.959462+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:37.055358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:08:02.167030+0000","last_clean_scrub_stamp":"2024-09-20T22:22:37.055358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:03.817284+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1139,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.960293+0000","last_change":"2024-09-20T22:23:01.609407+0000","last_active":"2024-09-20T22:23:31.960293+0000","last_peered":"2024-09-20T22:23:31.960293+0000","last_clean":"2024-09-20T22:23:31.960293+0000","last_became_active":"2024-09-20T22:22:30.673317+0000","last_became_peered":"2024-09-20T22:22:30.673317+0000","last_unstale":"2024-09-20T22:23:31.960293+0000","last_undegraded":"2024-09-20T22:23:31.960293+0000","last_fullsized":"2024-09-20T22:23:31.960293+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:01.609306+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:23:01.609306+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:12.576775+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"75'128","reported_seq":375,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.717967+0000","last_change":"2024-09-20T22:22:40.124825+0000","last_active":"2024-09-20T22:23:31.717967+0000","last_peered":"2024-09-20T22:23:31.717967+0000","last_clean":"2024-09-20T22:23:31.717967+0000","last_became_active":"2024-09-20T22:22:39.716276+0000","last_became_peered":"2024-09-20T22:22:39.716276+0000","last_unstale":"2024-09-20T22:23:31.717967+0000","last_undegraded":"2024-09-20T22:23:31.717967+0000","last_fullsized":"2024-09-20T22:23:31.717967+0000","mapping_epoch":567,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":568,"parent":"0.0","parent_split_bits":3,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:40.124745+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:22:40.124745+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:52.106218+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"579'36","reported_seq":681,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.720377+0000","last_change":"2024-09-20T22:22:36.784115+0000","last_active":"2024-09-20T22:23:31.720377+0000","last_peered":"2024-09-20T22:23:31.720377+0000","last_clean":"2024-09-20T22:23:31.720377+0000","last_became_active":"2024-09-20T22:22:36.748288+0000","last_became_peered":"2024-09-20T22:22:36.748288+0000","last_unstale":"2024-09-20T22:23:31.720377+0000","last_undegraded":"2024-09-20T22:23:31.720377+0000","last_fullsized":"2024-09-20T22:23:31.720377+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'24","last_scrub_stamp":"2024-09-20T22:22:03.105662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:03.105662+0000","objects_scrubbed":6,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":47,"num_write":36,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":612,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718272+0000","last_change":"2024-09-20T22:23:31.213672+0000","last_active":"2024-09-20T22:23:31.718272+0000","last_peered":"2024-09-20T22:23:31.718272+0000","last_clean":"2024-09-20T22:23:31.718272+0000","last_became_active":"2024-09-20T22:23:10.778296+0000","last_became_peered":"2024-09-20T22:23:10.778296+0000","last_unstale":"2024-09-20T22:23:31.718272+0000","last_undegraded":"2024-09-20T22:23:31.718272+0000","last_fullsized":"2024-09-20T22:23:31.718272+0000","mapping_epoch":587,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":588,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:31.213596+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:06:44.862338+0000","last_clean_scrub_stamp":"2024-09-20T22:23:31.213596+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:38.208077+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"72'5","reported_seq":2427,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:34.827000+0000","last_change":"2024-09-20T22:22:39.610305+0000","last_active":"2024-09-20T22:23:34.827000+0000","last_peered":"2024-09-20T22:23:34.827000+0000","last_clean":"2024-09-20T22:23:34.827000+0000","last_became_active":"2024-09-20T22:22:36.743951+0000","last_became_peered":"2024-09-20T22:22:36.743951+0000","last_unstale":"2024-09-20T22:23:34.827000+0000","last_undegraded":"2024-09-20T22:23:34.827000+0000","last_fullsized":"2024-09-20T22:23:34.827000+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"72'5","last_scrub_stamp":"2024-09-20T22:22:39.610209+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:48.075477+0000","last_clean_scrub_stamp":"2024-09-20T22:22:39.610209+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:41.985811+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0.00072923400000000005,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1216,"num_read_kb":1211,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1556,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":701,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.063092+0000","last_change":"2024-09-20T22:22:36.744877+0000","last_active":"2024-09-20T22:23:31.063092+0000","last_peered":"2024-09-20T22:23:31.063092+0000","last_clean":"2024-09-20T22:23:31.063092+0000","last_became_active":"2024-09-20T22:22:36.744548+0000","last_became_peered":"2024-09-20T22:22:36.744548+0000","last_unstale":"2024-09-20T22:23:31.063092+0000","last_undegraded":"2024-09-20T22:23:31.063092+0000","last_fullsized":"2024-09-20T22:23:31.063092+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:06.156848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:17:08.955790+0000","last_clean_scrub_stamp":"2024-09-20T22:22:06.156848+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:34.908139+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":81,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718971+0000","last_change":"2024-09-20T22:23:02.210754+0000","last_active":"2024-09-20T22:23:31.718971+0000","last_peered":"2024-09-20T22:23:31.718971+0000","last_clean":"2024-09-20T22:23:31.718971+0000","last_became_active":"2024-09-20T22:22:36.736380+0000","last_became_peered":"2024-09-20T22:22:36.736380+0000","last_unstale":"2024-09-20T22:23:31.718971+0000","last_undegraded":"2024-09-20T22:23:31.718971+0000","last_fullsized":"2024-09-20T22:23:31.718971+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:02.210668+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:40.325308+0000","last_clean_scrub_stamp":"2024-09-20T22:23:02.210668+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:25.771253+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"579'44","reported_seq":159,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.961041+0000","last_change":"2024-09-20T22:22:34.778254+0000","last_active":"2024-09-20T22:23:31.961041+0000","last_peered":"2024-09-20T22:23:31.961041+0000","last_clean":"2024-09-20T22:23:31.961041+0000","last_became_active":"2024-09-20T22:22:34.748021+0000","last_became_peered":"2024-09-20T22:22:34.748021+0000","last_unstale":"2024-09-20T22:23:31.961041+0000","last_undegraded":"2024-09-20T22:23:31.961041+0000","last_fullsized":"2024-09-20T22:23:31.961041+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"475'29","last_scrub_stamp":"2024-09-20T22:22:14.737957+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:14.737957+0000","objects_scrubbed":8,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:41.499501+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":61,"num_read_kb":51,"num_write":37,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16,"num_bytes_recovered":308,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1078,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952141+0000","last_change":"2024-09-20T22:23:00.051665+0000","last_active":"2024-09-20T22:23:30.952141+0000","last_peered":"2024-09-20T22:23:30.952141+0000","last_clean":"2024-09-20T22:23:30.952141+0000","last_became_active":"2024-09-20T22:22:51.734926+0000","last_became_peered":"2024-09-20T22:22:51.734926+0000","last_unstale":"2024-09-20T22:23:30.952141+0000","last_undegraded":"2024-09-20T22:23:30.952141+0000","last_fullsized":"2024-09-20T22:23:30.952141+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":576,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:00.051547+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:23:00.051547+0000","last_clean_scrub_stamp":"2024-09-20T22:23:00.051547+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:25.256549+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":1188,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.958293+0000","last_change":"2024-09-20T22:22:45.629159+0000","last_active":"2024-09-20T22:23:31.958293+0000","last_peered":"2024-09-20T22:23:31.958293+0000","last_clean":"2024-09-20T22:23:31.958293+0000","last_became_active":"2024-09-20T22:22:33.742983+0000","last_became_peered":"2024-09-20T22:22:33.742983+0000","last_unstale":"2024-09-20T22:23:31.958293+0000","last_undegraded":"2024-09-20T22:23:31.958293+0000","last_fullsized":"2024-09-20T22:23:31.958293+0000","mapping_epoch":561,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":562,"parent":"0.0","parent_split_bits":0,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:45.629061+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:18:06.776546+0000","last_clean_scrub_stamp":"2024-09-20T22:22:45.629061+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:46.807860+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":722,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959104+0000","last_change":"2024-09-20T22:22:48.698028+0000","last_active":"2024-09-20T22:23:31.959104+0000","last_peered":"2024-09-20T22:23:31.959104+0000","last_clean":"2024-09-20T22:23:31.959104+0000","last_became_active":"2024-09-20T22:10:48.907271+0000","last_became_peered":"2024-09-20T22:10:48.907271+0000","last_unstale":"2024-09-20T22:23:31.959104+0000","last_undegraded":"2024-09-20T22:23:31.959104+0000","last_fullsized":"2024-09-20T22:23:31.959104+0000","mapping_epoch":262,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":263,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:48.697944+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:13:51.674976+0000","last_clean_scrub_stamp":"2024-09-20T22:22:48.697944+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:08.599761+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1170,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.063250+0000","last_change":"2024-09-20T22:23:07.023515+0000","last_active":"2024-09-20T22:23:31.063250+0000","last_peered":"2024-09-20T22:23:31.063250+0000","last_clean":"2024-09-20T22:23:31.063250+0000","last_became_active":"2024-09-20T22:22:34.748659+0000","last_became_peered":"2024-09-20T22:22:34.748659+0000","last_unstale":"2024-09-20T22:23:31.063250+0000","last_undegraded":"2024-09-20T22:23:31.063250+0000","last_fullsized":"2024-09-20T22:23:31.063250+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:07.023412+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:05:27.659252+0000","last_clean_scrub_stamp":"2024-09-20T22:23:07.023412+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:35.787281+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.b","version":"75'128","reported_seq":967,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718440+0000","last_change":"2024-09-20T22:23:06.300361+0000","last_active":"2024-09-20T22:23:31.718440+0000","last_peered":"2024-09-20T22:23:31.718440+0000","last_clean":"2024-09-20T22:23:31.718440+0000","last_became_active":"2024-09-20T22:22:39.717701+0000","last_became_peered":"2024-09-20T22:22:39.717701+0000","last_unstale":"2024-09-20T22:23:31.718440+0000","last_undegraded":"2024-09-20T22:23:31.718440+0000","last_fullsized":"2024-09-20T22:23:31.718440+0000","mapping_epoch":567,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":568,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:23:06.300224+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:23:06.300224+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:23.574749+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,5],"acting":[1,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"579'27","reported_seq":144,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719423+0000","last_change":"2024-09-20T22:23:23.837874+0000","last_active":"2024-09-20T22:23:31.719423+0000","last_peered":"2024-09-20T22:23:31.719423+0000","last_clean":"2024-09-20T22:23:31.719423+0000","last_became_active":"2024-09-20T22:23:23.829700+0000","last_became_peered":"2024-09-20T22:23:23.829700+0000","last_unstale":"2024-09-20T22:23:31.719423+0000","last_undegraded":"2024-09-20T22:23:31.719423+0000","last_fullsized":"2024-09-20T22:23:31.719423+0000","mapping_epoch":591,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":592,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:22:37.724251+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:37.724251+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:38.459394+0000","scrub_duration":9,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":84,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718717+0000","last_change":"2024-09-20T22:23:21.646272+0000","last_active":"2024-09-20T22:23:31.718717+0000","last_peered":"2024-09-20T22:23:31.718717+0000","last_clean":"2024-09-20T22:23:31.718717+0000","last_became_active":"2024-09-20T22:22:34.742470+0000","last_became_peered":"2024-09-20T22:22:34.742470+0000","last_unstale":"2024-09-20T22:23:31.718717+0000","last_undegraded":"2024-09-20T22:23:31.718717+0000","last_fullsized":"2024-09-20T22:23:31.718717+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:21.646189+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:23:21.646189+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:23.057280+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1168,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.063422+0000","last_change":"2024-09-20T22:23:02.000751+0000","last_active":"2024-09-20T22:23:31.063422+0000","last_peered":"2024-09-20T22:23:31.063422+0000","last_clean":"2024-09-20T22:23:31.063422+0000","last_became_active":"2024-09-20T22:22:34.748654+0000","last_became_peered":"2024-09-20T22:22:34.748654+0000","last_unstale":"2024-09-20T22:23:31.063422+0000","last_undegraded":"2024-09-20T22:23:31.063422+0000","last_fullsized":"2024-09-20T22:23:31.063422+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:02.000619+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:23:02.000619+0000","last_clean_scrub_stamp":"2024-09-20T22:23:02.000619+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:09.628446+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":577,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.951974+0000","last_change":"2024-09-20T22:22:51.735316+0000","last_active":"2024-09-20T22:23:30.951974+0000","last_peered":"2024-09-20T22:23:30.951974+0000","last_clean":"2024-09-20T22:23:30.951974+0000","last_became_active":"2024-09-20T22:22:51.734972+0000","last_became_peered":"2024-09-20T22:22:51.734972+0000","last_unstale":"2024-09-20T22:23:30.951974+0000","last_undegraded":"2024-09-20T22:23:30.951974+0000","last_fullsized":"2024-09-20T22:23:30.951974+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":576,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:17.690638+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:17.690638+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":12,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.8","version":"75'128","reported_seq":498,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.960135+0000","last_change":"2024-09-20T22:22:34.836902+0000","last_active":"2024-09-20T22:23:31.960135+0000","last_peered":"2024-09-20T22:23:31.960135+0000","last_clean":"2024-09-20T22:23:31.960135+0000","last_became_active":"2024-09-20T22:22:33.732055+0000","last_became_peered":"2024-09-20T22:22:33.732055+0000","last_unstale":"2024-09-20T22:23:31.960135+0000","last_undegraded":"2024-09-20T22:23:31.960135+0000","last_fullsized":"2024-09-20T22:23:31.960135+0000","mapping_epoch":561,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":562,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:34.836783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:34.836783+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:51.889111+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"579'45","reported_seq":147,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719264+0000","last_change":"2024-09-20T22:22:43.690865+0000","last_active":"2024-09-20T22:23:31.719264+0000","last_peered":"2024-09-20T22:23:31.719264+0000","last_clean":"2024-09-20T22:23:31.719264+0000","last_became_active":"2024-09-20T22:22:36.733426+0000","last_became_peered":"2024-09-20T22:22:36.733426+0000","last_unstale":"2024-09-20T22:23:31.719264+0000","last_undegraded":"2024-09-20T22:23:31.719264+0000","last_fullsized":"2024-09-20T22:23:31.719264+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'31","last_scrub_stamp":"2024-09-20T22:22:43.690780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:43.690780+0000","objects_scrubbed":8,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:02.781256+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":70,"num_read_kb":60,"num_write":46,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":106,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.961081+0000","last_change":"2024-09-20T22:23:01.036132+0000","last_active":"2024-09-20T22:23:31.961081+0000","last_peered":"2024-09-20T22:23:31.961081+0000","last_clean":"2024-09-20T22:23:31.961081+0000","last_became_active":"2024-09-20T22:22:36.741709+0000","last_became_peered":"2024-09-20T22:22:36.741709+0000","last_unstale":"2024-09-20T22:23:31.961081+0000","last_undegraded":"2024-09-20T22:23:31.961081+0000","last_fullsized":"2024-09-20T22:23:31.961081+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:01.036043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:17:08.846904+0000","last_clean_scrub_stamp":"2024-09-20T22:23:01.036043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:25.318457+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":81,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719067+0000","last_change":"2024-09-20T22:22:41.094622+0000","last_active":"2024-09-20T22:23:31.719067+0000","last_peered":"2024-09-20T22:23:31.719067+0000","last_clean":"2024-09-20T22:23:31.719067+0000","last_became_active":"2024-09-20T22:22:36.747619+0000","last_became_peered":"2024-09-20T22:22:36.747619+0000","last_unstale":"2024-09-20T22:23:31.719067+0000","last_undegraded":"2024-09-20T22:23:31.719067+0000","last_fullsized":"2024-09-20T22:23:31.719067+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:41.094548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:10.179939+0000","last_clean_scrub_stamp":"2024-09-20T22:22:41.094548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:48.330041+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1182,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959567+0000","last_change":"2024-09-20T22:23:30.843127+0000","last_active":"2024-09-20T22:23:31.959567+0000","last_peered":"2024-09-20T22:23:31.959567+0000","last_clean":"2024-09-20T22:23:31.959567+0000","last_became_active":"2024-09-20T22:22:34.737506+0000","last_became_peered":"2024-09-20T22:22:34.737506+0000","last_unstale":"2024-09-20T22:23:31.959567+0000","last_undegraded":"2024-09-20T22:23:31.959567+0000","last_fullsized":"2024-09-20T22:23:31.959567+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:30.843055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:23:30.843055+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:56.617968+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.9","version":"75'128","reported_seq":1439,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959518+0000","last_change":"2024-09-20T22:23:26.502347+0000","last_active":"2024-09-20T22:23:31.959518+0000","last_peered":"2024-09-20T22:23:31.959518+0000","last_clean":"2024-09-20T22:23:31.959518+0000","last_became_active":"2024-09-20T22:17:40.146794+0000","last_became_peered":"2024-09-20T22:17:40.146794+0000","last_unstale":"2024-09-20T22:23:31.959518+0000","last_undegraded":"2024-09-20T22:23:31.959518+0000","last_fullsized":"2024-09-20T22:23:31.959518+0000","mapping_epoch":202,"log_start":"0'0","ondisk_log_start":"0'0","created":197,"last_epoch_clean":429,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:23:26.502246+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:23:26.502246+0000","last_clean_scrub_stamp":"2024-09-20T22:23:26.502246+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:27.150554+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,2],"acting":[7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"579'35","reported_seq":111,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718576+0000","last_change":"2024-09-20T22:23:15.706930+0000","last_active":"2024-09-20T22:23:31.718576+0000","last_peered":"2024-09-20T22:23:31.718576+0000","last_clean":"2024-09-20T22:23:31.718576+0000","last_became_active":"2024-09-20T22:22:34.753666+0000","last_became_peered":"2024-09-20T22:22:34.753666+0000","last_unstale":"2024-09-20T22:23:31.718576+0000","last_undegraded":"2024-09-20T22:23:31.718576+0000","last_fullsized":"2024-09-20T22:23:31.718576+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'35","last_scrub_stamp":"2024-09-20T22:23:15.706837+0000","last_deep_scrub":"328'25","last_deep_scrub_stamp":"2024-09-20T22:18:05.300862+0000","last_clean_scrub_stamp":"2024-09-20T22:23:15.706837+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:29.869686+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":39,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"237'2","reported_seq":600,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952634+0000","last_change":"2024-09-20T22:23:26.140556+0000","last_active":"2024-09-20T22:23:30.952634+0000","last_peered":"2024-09-20T22:23:30.952634+0000","last_clean":"2024-09-20T22:23:30.952634+0000","last_became_active":"2024-09-20T22:22:51.337688+0000","last_became_peered":"2024-09-20T22:22:51.337688+0000","last_unstale":"2024-09-20T22:23:30.952634+0000","last_undegraded":"2024-09-20T22:23:30.952634+0000","last_fullsized":"2024-09-20T22:23:30.952634+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"237'2","last_scrub_stamp":"2024-09-20T22:23:26.140475+0000","last_deep_scrub":"237'2","last_deep_scrub_stamp":"2024-09-20T22:09:53.395833+0000","last_clean_scrub_stamp":"2024-09-20T22:23:26.140475+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:49.059487+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"283'11","reported_seq":1448,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959018+0000","last_change":"2024-09-20T22:22:34.002553+0000","last_active":"2024-09-20T22:23:31.959018+0000","last_peered":"2024-09-20T22:23:31.959018+0000","last_clean":"2024-09-20T22:23:31.959018+0000","last_became_active":"2024-09-20T22:11:30.485356+0000","last_became_peered":"2024-09-20T22:11:30.485356+0000","last_unstale":"2024-09-20T22:23:31.959018+0000","last_undegraded":"2024-09-20T22:23:31.959018+0000","last_fullsized":"2024-09-20T22:23:31.959018+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"283'11","last_scrub_stamp":"2024-09-20T22:22:34.002442+0000","last_deep_scrub":"139'9","last_deep_scrub_stamp":"2024-09-20T22:09:50.486676+0000","last_clean_scrub_stamp":"2024-09-20T22:22:34.002442+0000","objects_scrubbed":1,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:59.619774+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":90,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719021+0000","last_change":"2024-09-20T22:22:45.706731+0000","last_active":"2024-09-20T22:23:31.719021+0000","last_peered":"2024-09-20T22:23:31.719021+0000","last_clean":"2024-09-20T22:23:31.719021+0000","last_became_active":"2024-09-20T22:22:36.732067+0000","last_became_peered":"2024-09-20T22:22:36.732067+0000","last_unstale":"2024-09-20T22:23:31.719021+0000","last_undegraded":"2024-09-20T22:23:31.719021+0000","last_fullsized":"2024-09-20T22:23:31.719021+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:45.706636+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:22:45.706636+0000","last_clean_scrub_stamp":"2024-09-20T22:22:45.706636+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:10.120861+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.e","version":"75'128","reported_seq":659,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719195+0000","last_change":"2024-09-20T22:22:38.702591+0000","last_active":"2024-09-20T22:23:31.719195+0000","last_peered":"2024-09-20T22:23:31.719195+0000","last_clean":"2024-09-20T22:23:31.719195+0000","last_became_active":"2024-09-20T22:22:37.717414+0000","last_became_peered":"2024-09-20T22:22:37.717414+0000","last_unstale":"2024-09-20T22:23:31.719195+0000","last_undegraded":"2024-09-20T22:23:31.719195+0000","last_fullsized":"2024-09-20T22:23:31.719195+0000","mapping_epoch":565,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":566,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:38.702498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:38.702498+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:54.154505+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,3],"acting":[4,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.b","version":"579'25","reported_seq":695,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719547+0000","last_change":"2024-09-20T22:23:04.200554+0000","last_active":"2024-09-20T22:23:31.719547+0000","last_peered":"2024-09-20T22:23:31.719547+0000","last_clean":"2024-09-20T22:23:31.719547+0000","last_became_active":"2024-09-20T22:22:36.750171+0000","last_became_peered":"2024-09-20T22:22:36.750171+0000","last_unstale":"2024-09-20T22:23:31.719547+0000","last_undegraded":"2024-09-20T22:23:31.719547+0000","last_fullsized":"2024-09-20T22:23:31.719547+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'25","last_scrub_stamp":"2024-09-20T22:23:04.200459+0000","last_deep_scrub":"512'19","last_deep_scrub_stamp":"2024-09-20T22:21:05.960284+0000","last_clean_scrub_stamp":"2024-09-20T22:23:04.200459+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:31.706993+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"64'1","reported_seq":622,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952772+0000","last_change":"2024-09-20T22:22:37.910548+0000","last_active":"2024-09-20T22:23:30.952772+0000","last_peered":"2024-09-20T22:23:30.952772+0000","last_clean":"2024-09-20T22:23:30.952772+0000","last_became_active":"2024-09-20T22:22:36.736413+0000","last_became_peered":"2024-09-20T22:22:36.736413+0000","last_unstale":"2024-09-20T22:23:30.952772+0000","last_undegraded":"2024-09-20T22:23:30.952772+0000","last_fullsized":"2024-09-20T22:23:30.952772+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"64'1","last_scrub_stamp":"2024-09-20T22:22:37.910471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:37.910471+0000","objects_scrubbed":1,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:05.238762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":1986,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":1174,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.958622+0000","last_change":"2024-09-20T22:22:34.738023+0000","last_active":"2024-09-20T22:23:31.958622+0000","last_peered":"2024-09-20T22:23:31.958622+0000","last_clean":"2024-09-20T22:23:31.958622+0000","last_became_active":"2024-09-20T22:22:34.737563+0000","last_became_peered":"2024-09-20T22:22:34.737563+0000","last_unstale":"2024-09-20T22:23:31.958622+0000","last_undegraded":"2024-09-20T22:23:31.958622+0000","last_fullsized":"2024-09-20T22:23:31.958622+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:25.949839+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:09:06.528588+0000","last_clean_scrub_stamp":"2024-09-20T22:22:25.949839+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:40.478352+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":597,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718181+0000","last_change":"2024-09-20T22:22:57.193074+0000","last_active":"2024-09-20T22:23:31.718181+0000","last_peered":"2024-09-20T22:23:31.718181+0000","last_clean":"2024-09-20T22:23:31.718181+0000","last_became_active":"2024-09-20T22:19:45.813003+0000","last_became_peered":"2024-09-20T22:19:45.813003+0000","last_unstale":"2024-09-20T22:23:31.718181+0000","last_undegraded":"2024-09-20T22:23:31.718181+0000","last_fullsized":"2024-09-20T22:23:31.718181+0000","mapping_epoch":480,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":481,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:57.192999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:22:57.192999+0000","last_clean_scrub_stamp":"2024-09-20T22:22:57.192999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:59.657894+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.f","version":"75'128","reported_seq":828,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.961036+0000","last_change":"2024-09-20T22:22:37.907747+0000","last_active":"2024-09-20T22:23:31.961036+0000","last_peered":"2024-09-20T22:23:31.961036+0000","last_clean":"2024-09-20T22:23:31.961036+0000","last_became_active":"2024-09-20T22:22:37.719520+0000","last_became_peered":"2024-09-20T22:22:37.719520+0000","last_unstale":"2024-09-20T22:23:31.961036+0000","last_undegraded":"2024-09-20T22:23:31.961036+0000","last_fullsized":"2024-09-20T22:23:31.961036+0000","mapping_epoch":565,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":566,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:37.907659+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:17:07.885346+0000","last_clean_scrub_stamp":"2024-09-20T22:22:37.907659+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:58.920652+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,4],"acting":[6,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.a","version":"579'51","reported_seq":164,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.960881+0000","last_change":"2024-09-20T22:23:07.072101+0000","last_active":"2024-09-20T22:23:31.960881+0000","last_peered":"2024-09-20T22:23:31.960881+0000","last_clean":"2024-09-20T22:23:31.960881+0000","last_became_active":"2024-09-20T22:22:36.742973+0000","last_became_peered":"2024-09-20T22:22:36.742973+0000","last_unstale":"2024-09-20T22:23:31.960881+0000","last_undegraded":"2024-09-20T22:23:31.960881+0000","last_fullsized":"2024-09-20T22:23:31.960881+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'51","last_scrub_stamp":"2024-09-20T22:23:07.071970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:23:07.071970+0000","objects_scrubbed":9,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:21.049002+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":79,"num_read_kb":68,"num_write":52,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1171,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:33.613622+0000","last_change":"2024-09-20T22:23:33.613646+0000","last_active":"2024-09-20T22:23:33.613622+0000","last_peered":"2024-09-20T22:23:33.613622+0000","last_clean":"2024-09-20T22:23:33.613622+0000","last_became_active":"2024-09-20T22:22:30.675715+0000","last_became_peered":"2024-09-20T22:22:30.675715+0000","last_unstale":"2024-09-20T22:23:33.613622+0000","last_undegraded":"2024-09-20T22:23:33.613622+0000","last_fullsized":"2024-09-20T22:23:33.613622+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:33.613550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:12:46.602035+0000","last_clean_scrub_stamp":"2024-09-20T22:23:33.613550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:45.066220+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":1186,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.958803+0000","last_change":"2024-09-20T22:22:19.052616+0000","last_active":"2024-09-20T22:23:31.958803+0000","last_peered":"2024-09-20T22:23:31.958803+0000","last_clean":"2024-09-20T22:23:31.958803+0000","last_became_active":"2024-09-20T22:12:52.515295+0000","last_became_peered":"2024-09-20T22:12:52.515295+0000","last_unstale":"2024-09-20T22:23:31.958803+0000","last_undegraded":"2024-09-20T22:23:31.958803+0000","last_fullsized":"2024-09-20T22:23:31.958803+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:19.052544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:26.501363+0000","last_clean_scrub_stamp":"2024-09-20T22:22:19.052544+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:39.745768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":744,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.960784+0000","last_change":"2024-09-20T22:23:03.698199+0000","last_active":"2024-09-20T22:23:31.960784+0000","last_peered":"2024-09-20T22:23:31.960784+0000","last_clean":"2024-09-20T22:23:31.960784+0000","last_became_active":"2024-09-20T22:10:30.657116+0000","last_became_peered":"2024-09-20T22:10:30.657116+0000","last_unstale":"2024-09-20T22:23:31.960784+0000","last_undegraded":"2024-09-20T22:23:31.960784+0000","last_fullsized":"2024-09-20T22:23:31.960784+0000","mapping_epoch":251,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":252,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:03.698074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:23:03.698074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:22.938112+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.c","version":"75'128","reported_seq":1255,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952217+0000","last_change":"2024-09-20T22:23:16.175251+0000","last_active":"2024-09-20T22:23:30.952217+0000","last_peered":"2024-09-20T22:23:30.952217+0000","last_clean":"2024-09-20T22:23:30.952217+0000","last_became_active":"2024-09-20T22:22:33.734769+0000","last_became_peered":"2024-09-20T22:22:33.734769+0000","last_unstale":"2024-09-20T22:23:30.952217+0000","last_undegraded":"2024-09-20T22:23:30.952217+0000","last_fullsized":"2024-09-20T22:23:30.952217+0000","mapping_epoch":561,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":562,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:23:16.175099+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:23:16.175099+0000","objects_scrubbed":1,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:20.873689+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":327680,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.9","version":"579'29","reported_seq":123,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718367+0000","last_change":"2024-09-20T22:23:11.705652+0000","last_active":"2024-09-20T22:23:31.718367+0000","last_peered":"2024-09-20T22:23:31.718367+0000","last_clean":"2024-09-20T22:23:31.718367+0000","last_became_active":"2024-09-20T22:22:36.732746+0000","last_became_peered":"2024-09-20T22:22:36.732746+0000","last_unstale":"2024-09-20T22:23:31.718367+0000","last_undegraded":"2024-09-20T22:23:31.718367+0000","last_fullsized":"2024-09-20T22:23:31.718367+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'29","last_scrub_stamp":"2024-09-20T22:23:11.705547+0000","last_deep_scrub":"418'23","last_deep_scrub_stamp":"2024-09-20T22:17:11.338821+0000","last_clean_scrub_stamp":"2024-09-20T22:23:11.705547+0000","objects_scrubbed":5,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:22.808300+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":34,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":440,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1172,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959213+0000","last_change":"2024-09-20T22:23:09.655954+0000","last_active":"2024-09-20T22:23:31.959213+0000","last_peered":"2024-09-20T22:23:31.959213+0000","last_clean":"2024-09-20T22:23:31.959213+0000","last_became_active":"2024-09-20T22:22:36.742134+0000","last_became_peered":"2024-09-20T22:22:36.742134+0000","last_unstale":"2024-09-20T22:23:31.959213+0000","last_undegraded":"2024-09-20T22:23:31.959213+0000","last_fullsized":"2024-09-20T22:23:31.959213+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:09.655865+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:23:09.655865+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:13.182997+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":1195,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.958110+0000","last_change":"2024-09-20T22:22:36.734670+0000","last_active":"2024-09-20T22:23:31.958110+0000","last_peered":"2024-09-20T22:23:31.958110+0000","last_clean":"2024-09-20T22:23:31.958110+0000","last_became_active":"2024-09-20T22:22:36.734292+0000","last_became_peered":"2024-09-20T22:22:36.734292+0000","last_unstale":"2024-09-20T22:23:31.958110+0000","last_undegraded":"2024-09-20T22:23:31.958110+0000","last_fullsized":"2024-09-20T22:23:31.958110+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:29.970946+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:29.970946+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:43.300720+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":629,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952273+0000","last_change":"2024-09-20T22:22:36.734889+0000","last_active":"2024-09-20T22:23:30.952273+0000","last_peered":"2024-09-20T22:23:30.952273+0000","last_clean":"2024-09-20T22:23:30.952273+0000","last_became_active":"2024-09-20T22:22:36.734590+0000","last_became_peered":"2024-09-20T22:22:36.734590+0000","last_unstale":"2024-09-20T22:23:30.952273+0000","last_undegraded":"2024-09-20T22:23:30.952273+0000","last_fullsized":"2024-09-20T22:23:30.952273+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:03.002391+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:03.002391+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:31.336479+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.d","version":"75'128","reported_seq":789,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959518+0000","last_change":"2024-09-20T22:22:44.673975+0000","last_active":"2024-09-20T22:23:31.959518+0000","last_peered":"2024-09-20T22:23:31.959518+0000","last_clean":"2024-09-20T22:23:31.959518+0000","last_became_active":"2024-09-20T22:22:39.718957+0000","last_became_peered":"2024-09-20T22:22:39.718957+0000","last_unstale":"2024-09-20T22:23:31.959518+0000","last_undegraded":"2024-09-20T22:23:31.959518+0000","last_fullsized":"2024-09-20T22:23:31.959518+0000","mapping_epoch":567,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":568,"parent":"0.0","parent_split_bits":4,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:44.673852+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:14:21.058895+0000","last_clean_scrub_stamp":"2024-09-20T22:22:44.673852+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:03.639902+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"579'46","reported_seq":766,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062310+0000","last_change":"2024-09-20T22:22:30.705915+0000","last_active":"2024-09-20T22:23:31.062310+0000","last_peered":"2024-09-20T22:23:31.062310+0000","last_clean":"2024-09-20T22:23:31.062310+0000","last_became_active":"2024-09-20T22:22:30.679068+0000","last_became_peered":"2024-09-20T22:22:30.679068+0000","last_unstale":"2024-09-20T22:23:31.062310+0000","last_undegraded":"2024-09-20T22:23:31.062310+0000","last_fullsized":"2024-09-20T22:23:31.062310+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"416'34","last_scrub_stamp":"2024-09-20T22:22:04.106984+0000","last_deep_scrub":"416'34","last_deep_scrub_stamp":"2024-09-20T22:17:04.960380+0000","last_clean_scrub_stamp":"2024-09-20T22:22:04.106984+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":62,"num_read_kb":53,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"139'4","reported_seq":809,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.958938+0000","last_change":"2024-09-20T22:23:20.552782+0000","last_active":"2024-09-20T22:23:31.958938+0000","last_peered":"2024-09-20T22:23:31.958938+0000","last_clean":"2024-09-20T22:23:31.958938+0000","last_became_active":"2024-09-20T22:22:34.738267+0000","last_became_peered":"2024-09-20T22:22:34.738267+0000","last_unstale":"2024-09-20T22:23:31.958938+0000","last_undegraded":"2024-09-20T22:23:31.958938+0000","last_fullsized":"2024-09-20T22:23:31.958938+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"139'4","last_scrub_stamp":"2024-09-20T22:23:20.552660+0000","last_deep_scrub":"139'4","last_deep_scrub_stamp":"2024-09-20T22:06:40.813590+0000","last_clean_scrub_stamp":"2024-09-20T22:23:20.552660+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:44.224134+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.9","version":"563'13","reported_seq":1439,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959937+0000","last_change":"2024-09-20T22:23:12.613078+0000","last_active":"2024-09-20T22:23:31.959937+0000","last_peered":"2024-09-20T22:23:31.959937+0000","last_clean":"2024-09-20T22:23:31.959937+0000","last_became_active":"2024-09-20T22:22:34.740079+0000","last_became_peered":"2024-09-20T22:22:34.740079+0000","last_unstale":"2024-09-20T22:23:31.959937+0000","last_undegraded":"2024-09-20T22:23:31.959937+0000","last_fullsized":"2024-09-20T22:23:31.959937+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"563'13","last_scrub_stamp":"2024-09-20T22:23:12.612938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:23:12.612938+0000","objects_scrubbed":1,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:23.589739+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":695,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062594+0000","last_change":"2024-09-20T22:23:06.007858+0000","last_active":"2024-09-20T22:23:31.062594+0000","last_peered":"2024-09-20T22:23:31.062594+0000","last_clean":"2024-09-20T22:23:31.062594+0000","last_became_active":"2024-09-20T22:22:30.688569+0000","last_became_peered":"2024-09-20T22:22:30.688569+0000","last_unstale":"2024-09-20T22:23:31.062594+0000","last_undegraded":"2024-09-20T22:23:31.062594+0000","last_fullsized":"2024-09-20T22:23:31.062594+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:06.007552+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:23:06.007552+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:30.820960+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.12","version":"75'128","reported_seq":1059,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062310+0000","last_change":"2024-09-20T22:23:04.007381+0000","last_active":"2024-09-20T22:23:31.062310+0000","last_peered":"2024-09-20T22:23:31.062310+0000","last_clean":"2024-09-20T22:23:31.062310+0000","last_became_active":"2024-09-20T22:22:39.718576+0000","last_became_peered":"2024-09-20T22:22:39.718576+0000","last_unstale":"2024-09-20T22:23:31.062310+0000","last_undegraded":"2024-09-20T22:23:31.062310+0000","last_fullsized":"2024-09-20T22:23:31.062310+0000","mapping_epoch":567,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":568,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:23:04.007289+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:23:04.007289+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:08.625589+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":94,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959891+0000","last_change":"2024-09-20T22:22:39.953831+0000","last_active":"2024-09-20T22:23:31.959891+0000","last_peered":"2024-09-20T22:23:31.959891+0000","last_clean":"2024-09-20T22:23:31.959891+0000","last_became_active":"2024-09-20T22:22:30.674649+0000","last_became_peered":"2024-09-20T22:22:30.674649+0000","last_unstale":"2024-09-20T22:23:31.959891+0000","last_undegraded":"2024-09-20T22:23:31.959891+0000","last_fullsized":"2024-09-20T22:23:31.959891+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:39.953740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:39.953740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:08.818852+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"579'18","reported_seq":1251,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952838+0000","last_change":"2024-09-20T22:23:23.836786+0000","last_active":"2024-09-20T22:23:30.952838+0000","last_peered":"2024-09-20T22:23:30.952838+0000","last_clean":"2024-09-20T22:23:30.952838+0000","last_became_active":"2024-09-20T22:23:23.832325+0000","last_became_peered":"2024-09-20T22:23:23.832325+0000","last_unstale":"2024-09-20T22:23:30.952838+0000","last_undegraded":"2024-09-20T22:23:30.952838+0000","last_fullsized":"2024-09-20T22:23:30.952838+0000","mapping_epoch":591,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":592,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'18","last_scrub_stamp":"2024-09-20T22:23:17.214710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:23:17.214710+0000","objects_scrubbed":3,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:27.093767+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":24,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":694,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062978+0000","last_change":"2024-09-20T22:22:58.915746+0000","last_active":"2024-09-20T22:23:31.062978+0000","last_peered":"2024-09-20T22:23:31.062978+0000","last_clean":"2024-09-20T22:23:31.062978+0000","last_became_active":"2024-09-20T22:22:36.743711+0000","last_became_peered":"2024-09-20T22:22:36.743711+0000","last_unstale":"2024-09-20T22:23:31.062978+0000","last_undegraded":"2024-09-20T22:23:31.062978+0000","last_fullsized":"2024-09-20T22:23:31.062978+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:58.915662+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:58.915662+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:27.833030+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1150,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.960190+0000","last_change":"2024-09-20T22:23:23.584474+0000","last_active":"2024-09-20T22:23:31.960190+0000","last_peered":"2024-09-20T22:23:31.960190+0000","last_clean":"2024-09-20T22:23:31.960190+0000","last_became_active":"2024-09-20T22:22:34.756254+0000","last_became_peered":"2024-09-20T22:22:34.756254+0000","last_unstale":"2024-09-20T22:23:31.960190+0000","last_undegraded":"2024-09-20T22:23:31.960190+0000","last_fullsized":"2024-09-20T22:23:31.960190+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:23.584366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:23:23.584366+0000","last_clean_scrub_stamp":"2024-09-20T22:23:23.584366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:31.042510+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.13","version":"75'128","reported_seq":964,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718205+0000","last_change":"2024-09-20T22:23:24.115165+0000","last_active":"2024-09-20T22:23:31.718205+0000","last_peered":"2024-09-20T22:23:31.718205+0000","last_clean":"2024-09-20T22:23:31.718205+0000","last_became_active":"2024-09-20T22:22:39.717148+0000","last_became_peered":"2024-09-20T22:22:39.717148+0000","last_unstale":"2024-09-20T22:23:31.718205+0000","last_undegraded":"2024-09-20T22:23:31.718205+0000","last_fullsized":"2024-09-20T22:23:31.718205+0000","mapping_epoch":567,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":568,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:23:24.115087+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:13:48.490464+0000","last_clean_scrub_stamp":"2024-09-20T22:23:24.115087+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:40.063289+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.16","version":"579'21","reported_seq":627,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.720487+0000","last_change":"2024-09-20T22:23:24.215479+0000","last_active":"2024-09-20T22:23:31.720487+0000","last_peered":"2024-09-20T22:23:31.720487+0000","last_clean":"2024-09-20T22:23:31.720487+0000","last_became_active":"2024-09-20T22:12:52.527106+0000","last_became_peered":"2024-09-20T22:12:52.527106+0000","last_unstale":"2024-09-20T22:23:31.720487+0000","last_undegraded":"2024-09-20T22:23:31.720487+0000","last_fullsized":"2024-09-20T22:23:31.720487+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'21","last_scrub_stamp":"2024-09-20T22:23:24.215395+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:23:24.215395+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:36.126021+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1170,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959331+0000","last_change":"2024-09-20T22:23:24.557886+0000","last_active":"2024-09-20T22:23:31.959331+0000","last_peered":"2024-09-20T22:23:31.959331+0000","last_clean":"2024-09-20T22:23:31.959331+0000","last_became_active":"2024-09-20T22:22:34.739214+0000","last_became_peered":"2024-09-20T22:22:34.739214+0000","last_unstale":"2024-09-20T22:23:31.959331+0000","last_undegraded":"2024-09-20T22:23:31.959331+0000","last_fullsized":"2024-09-20T22:23:31.959331+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:24.557758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:23:24.557758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:28.189434+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":587,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952685+0000","last_change":"2024-09-20T22:22:58.030950+0000","last_active":"2024-09-20T22:23:30.952685+0000","last_peered":"2024-09-20T22:23:30.952685+0000","last_clean":"2024-09-20T22:23:30.952685+0000","last_became_active":"2024-09-20T22:22:51.342516+0000","last_became_peered":"2024-09-20T22:22:51.342516+0000","last_unstale":"2024-09-20T22:23:30.952685+0000","last_undegraded":"2024-09-20T22:23:30.952685+0000","last_fullsized":"2024-09-20T22:23:30.952685+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:58.030861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:09.360439+0000","last_clean_scrub_stamp":"2024-09-20T22:22:58.030861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:26.761346+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1175,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959680+0000","last_change":"2024-09-20T22:22:34.738019+0000","last_active":"2024-09-20T22:23:31.959680+0000","last_peered":"2024-09-20T22:23:31.959680+0000","last_clean":"2024-09-20T22:23:31.959680+0000","last_became_active":"2024-09-20T22:22:34.737638+0000","last_became_peered":"2024-09-20T22:22:34.737638+0000","last_unstale":"2024-09-20T22:23:31.959680+0000","last_undegraded":"2024-09-20T22:23:31.959680+0000","last_fullsized":"2024-09-20T22:23:31.959680+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:16.043342+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:16:26.862279+0000","last_clean_scrub_stamp":"2024-09-20T22:22:16.043342+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:45.338584+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.10","version":"75'128","reported_seq":1041,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062146+0000","last_change":"2024-09-20T22:22:40.842030+0000","last_active":"2024-09-20T22:23:31.062146+0000","last_peered":"2024-09-20T22:23:31.062146+0000","last_clean":"2024-09-20T22:23:31.062146+0000","last_became_active":"2024-09-20T22:22:33.745216+0000","last_became_peered":"2024-09-20T22:22:33.745216+0000","last_unstale":"2024-09-20T22:23:31.062146+0000","last_undegraded":"2024-09-20T22:23:31.062146+0000","last_fullsized":"2024-09-20T22:23:31.062146+0000","mapping_epoch":561,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":562,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:40.841933+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:40.841933+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:52.374743+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.15","version":"579'21","reported_seq":698,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.063025+0000","last_change":"2024-09-20T22:22:54.908266+0000","last_active":"2024-09-20T22:23:31.063025+0000","last_peered":"2024-09-20T22:23:31.063025+0000","last_clean":"2024-09-20T22:23:31.063025+0000","last_became_active":"2024-09-20T22:11:30.493335+0000","last_became_peered":"2024-09-20T22:11:30.493335+0000","last_unstale":"2024-09-20T22:23:31.063025+0000","last_undegraded":"2024-09-20T22:23:31.063025+0000","last_fullsized":"2024-09-20T22:23:31.063025+0000","mapping_epoch":282,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":283,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'15","last_scrub_stamp":"2024-09-20T22:22:54.908165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:54.908165+0000","objects_scrubbed":4,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:18.473541+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":34,"num_read_kb":28,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":599,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718988+0000","last_change":"2024-09-20T22:22:43.111083+0000","last_active":"2024-09-20T22:23:31.718988+0000","last_peered":"2024-09-20T22:23:31.718988+0000","last_clean":"2024-09-20T22:23:31.718988+0000","last_became_active":"2024-09-20T22:12:52.524149+0000","last_became_peered":"2024-09-20T22:12:52.524149+0000","last_unstale":"2024-09-20T22:23:31.718988+0000","last_undegraded":"2024-09-20T22:23:31.718988+0000","last_fullsized":"2024-09-20T22:23:31.718988+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:43.111008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:22:43.111008+0000","last_clean_scrub_stamp":"2024-09-20T22:22:43.111008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:09.952898+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"575'12","reported_seq":741,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.953238+0000","last_change":"2024-09-20T22:23:30.953260+0000","last_active":"2024-09-20T22:23:30.953238+0000","last_peered":"2024-09-20T22:23:30.953238+0000","last_clean":"2024-09-20T22:23:30.953238+0000","last_became_active":"2024-09-20T22:22:51.344012+0000","last_became_peered":"2024-09-20T22:22:51.344012+0000","last_unstale":"2024-09-20T22:23:30.953238+0000","last_undegraded":"2024-09-20T22:23:30.953238+0000","last_fullsized":"2024-09-20T22:23:30.953238+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"575'12","last_scrub_stamp":"2024-09-20T22:23:30.953187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:23:30.953187+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:48.784625+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":76,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719340+0000","last_change":"2024-09-20T22:22:34.747354+0000","last_active":"2024-09-20T22:23:31.719340+0000","last_peered":"2024-09-20T22:23:31.719340+0000","last_clean":"2024-09-20T22:23:31.719340+0000","last_became_active":"2024-09-20T22:22:34.747053+0000","last_became_peered":"2024-09-20T22:22:34.747053+0000","last_unstale":"2024-09-20T22:23:31.719340+0000","last_undegraded":"2024-09-20T22:23:31.719340+0000","last_fullsized":"2024-09-20T22:23:31.719340+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:17.031182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:18:03.323607+0000","last_clean_scrub_stamp":"2024-09-20T22:22:17.031182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:34.770291+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.11","version":"75'128","reported_seq":1050,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718558+0000","last_change":"2024-09-20T22:22:53.181146+0000","last_active":"2024-09-20T22:23:31.718558+0000","last_peered":"2024-09-20T22:23:31.718558+0000","last_clean":"2024-09-20T22:23:31.718558+0000","last_became_active":"2024-09-20T22:15:50.612544+0000","last_became_peered":"2024-09-20T22:15:50.612544+0000","last_unstale":"2024-09-20T22:23:31.718558+0000","last_undegraded":"2024-09-20T22:23:31.718558+0000","last_fullsized":"2024-09-20T22:23:31.718558+0000","mapping_epoch":388,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":389,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:53.181058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:53.181058+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:02.922575+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"579'35","reported_seq":428,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952369+0000","last_change":"2024-09-20T22:23:03.099892+0000","last_active":"2024-09-20T22:23:30.952369+0000","last_peered":"2024-09-20T22:23:30.952369+0000","last_clean":"2024-09-20T22:23:30.952369+0000","last_became_active":"2024-09-20T22:22:51.343502+0000","last_became_peered":"2024-09-20T22:22:51.343502+0000","last_unstale":"2024-09-20T22:23:30.952369+0000","last_undegraded":"2024-09-20T22:23:30.952369+0000","last_fullsized":"2024-09-20T22:23:30.952369+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'35","last_scrub_stamp":"2024-09-20T22:23:03.099751+0000","last_deep_scrub":"579'35","last_deep_scrub_stamp":"2024-09-20T22:23:03.099751+0000","last_clean_scrub_stamp":"2024-09-20T22:23:03.099751+0000","objects_scrubbed":5,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:27.881468+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":45,"num_read_kb":40,"num_write":30,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1162,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959557+0000","last_change":"2024-09-20T22:23:18.625730+0000","last_active":"2024-09-20T22:23:31.959557+0000","last_peered":"2024-09-20T22:23:31.959557+0000","last_clean":"2024-09-20T22:23:31.959557+0000","last_became_active":"2024-09-20T22:22:34.738109+0000","last_became_peered":"2024-09-20T22:22:34.738109+0000","last_unstale":"2024-09-20T22:23:31.959557+0000","last_undegraded":"2024-09-20T22:23:31.959557+0000","last_fullsized":"2024-09-20T22:23:31.959557+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:18.625639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:23:18.625639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:40.294115+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.15","version":"567'16","reported_seq":868,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.063263+0000","last_change":"2024-09-20T22:23:13.038668+0000","last_active":"2024-09-20T22:23:31.063263+0000","last_peered":"2024-09-20T22:23:31.063263+0000","last_clean":"2024-09-20T22:23:31.063263+0000","last_became_active":"2024-09-20T22:22:36.741754+0000","last_became_peered":"2024-09-20T22:22:36.741754+0000","last_unstale":"2024-09-20T22:23:31.063263+0000","last_undegraded":"2024-09-20T22:23:31.063263+0000","last_fullsized":"2024-09-20T22:23:31.063263+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"567'16","last_scrub_stamp":"2024-09-20T22:23:13.038571+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:23:13.038571+0000","objects_scrubbed":1,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:34.445388+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":599,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.720160+0000","last_change":"2024-09-20T22:23:09.207287+0000","last_active":"2024-09-20T22:23:31.720160+0000","last_peered":"2024-09-20T22:23:31.720160+0000","last_clean":"2024-09-20T22:23:31.720160+0000","last_became_active":"2024-09-20T22:12:52.523781+0000","last_became_peered":"2024-09-20T22:12:52.523781+0000","last_unstale":"2024-09-20T22:23:31.720160+0000","last_undegraded":"2024-09-20T22:23:31.720160+0000","last_fullsized":"2024-09-20T22:23:31.720160+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:09.207188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:23:09.207188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:17.469298+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.16","version":"75'128","reported_seq":849,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719922+0000","last_change":"2024-09-20T22:23:07.318573+0000","last_active":"2024-09-20T22:23:31.719922+0000","last_peered":"2024-09-20T22:23:31.719922+0000","last_clean":"2024-09-20T22:23:31.719922+0000","last_became_active":"2024-09-20T22:22:38.715194+0000","last_became_peered":"2024-09-20T22:22:38.715194+0000","last_unstale":"2024-09-20T22:23:31.719922+0000","last_undegraded":"2024-09-20T22:23:31.719922+0000","last_fullsized":"2024-09-20T22:23:31.719922+0000","mapping_epoch":566,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":567,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:23:07.318498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:23:07.318498+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:15.015805+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,3],"acting":[1,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"579'27","reported_seq":126,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718601+0000","last_change":"2024-09-20T22:22:57.761230+0000","last_active":"2024-09-20T22:23:31.718601+0000","last_peered":"2024-09-20T22:23:31.718601+0000","last_clean":"2024-09-20T22:23:31.718601+0000","last_became_active":"2024-09-20T22:22:36.746508+0000","last_became_peered":"2024-09-20T22:22:36.746508+0000","last_unstale":"2024-09-20T22:23:31.718601+0000","last_undegraded":"2024-09-20T22:23:31.718601+0000","last_fullsized":"2024-09-20T22:23:31.718601+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"328'19","last_scrub_stamp":"2024-09-20T22:22:57.761101+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:22:57.761101+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:10.300960+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15,"num_bytes_recovered":330,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":84,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718444+0000","last_change":"2024-09-20T22:23:07.660296+0000","last_active":"2024-09-20T22:23:31.718444+0000","last_peered":"2024-09-20T22:23:31.718444+0000","last_clean":"2024-09-20T22:23:31.718444+0000","last_became_active":"2024-09-20T22:22:34.745153+0000","last_became_peered":"2024-09-20T22:22:34.745153+0000","last_unstale":"2024-09-20T22:23:31.718444+0000","last_undegraded":"2024-09-20T22:23:31.718444+0000","last_fullsized":"2024-09-20T22:23:31.718444+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:07.660206+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:23:07.660206+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:34.264244+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":81,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719813+0000","last_change":"2024-09-20T22:22:54.204146+0000","last_active":"2024-09-20T22:23:31.719813+0000","last_peered":"2024-09-20T22:23:31.719813+0000","last_clean":"2024-09-20T22:23:31.719813+0000","last_became_active":"2024-09-20T22:22:36.735628+0000","last_became_peered":"2024-09-20T22:22:36.735628+0000","last_unstale":"2024-09-20T22:23:31.719813+0000","last_undegraded":"2024-09-20T22:23:31.719813+0000","last_fullsized":"2024-09-20T22:23:31.719813+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:54.204042+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:54.204042+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:07.791351+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":459,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.951696+0000","last_change":"2024-09-20T22:23:04.065863+0000","last_active":"2024-09-20T22:23:30.951696+0000","last_peered":"2024-09-20T22:23:30.951696+0000","last_clean":"2024-09-20T22:23:30.951696+0000","last_became_active":"2024-09-20T22:22:51.734410+0000","last_became_peered":"2024-09-20T22:22:51.734410+0000","last_unstale":"2024-09-20T22:23:30.951696+0000","last_undegraded":"2024-09-20T22:23:30.951696+0000","last_fullsized":"2024-09-20T22:23:30.951696+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":576,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:04.065790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:23:04.065790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:29.481511+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.17","version":"75'128","reported_seq":663,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.063542+0000","last_change":"2024-09-20T22:22:33.961540+0000","last_active":"2024-09-20T22:23:31.063542+0000","last_peered":"2024-09-20T22:23:31.063542+0000","last_clean":"2024-09-20T22:23:31.063542+0000","last_became_active":"2024-09-20T22:22:33.743368+0000","last_became_peered":"2024-09-20T22:22:33.743368+0000","last_unstale":"2024-09-20T22:23:31.063542+0000","last_undegraded":"2024-09-20T22:23:31.063542+0000","last_fullsized":"2024-09-20T22:23:31.063542+0000","mapping_epoch":561,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":562,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:33.961468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:33.961468+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:58.544200+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,6],"acting":[5,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"579'25","reported_seq":100,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719784+0000","last_change":"2024-09-20T22:23:12.296665+0000","last_active":"2024-09-20T22:23:31.719784+0000","last_peered":"2024-09-20T22:23:31.719784+0000","last_clean":"2024-09-20T22:23:31.719784+0000","last_became_active":"2024-09-20T22:22:36.729181+0000","last_became_peered":"2024-09-20T22:22:36.729181+0000","last_unstale":"2024-09-20T22:23:31.719784+0000","last_undegraded":"2024-09-20T22:23:31.719784+0000","last_fullsized":"2024-09-20T22:23:31.719784+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'25","last_scrub_stamp":"2024-09-20T22:23:12.296574+0000","last_deep_scrub":"291'13","last_deep_scrub_stamp":"2024-09-20T22:11:41.362215+0000","last_clean_scrub_stamp":"2024-09-20T22:23:12.296574+0000","objects_scrubbed":4,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:32.519705+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":33,"num_read_kb":27,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":747,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.960096+0000","last_change":"2024-09-20T22:22:34.755241+0000","last_active":"2024-09-20T22:23:31.960096+0000","last_peered":"2024-09-20T22:23:31.960096+0000","last_clean":"2024-09-20T22:23:31.960096+0000","last_became_active":"2024-09-20T22:22:34.754881+0000","last_became_peered":"2024-09-20T22:22:34.754881+0000","last_unstale":"2024-09-20T22:23:31.960096+0000","last_undegraded":"2024-09-20T22:23:31.960096+0000","last_fullsized":"2024-09-20T22:23:31.960096+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:27.499518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:27.499518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:48.896639+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":584,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952362+0000","last_change":"2024-09-20T22:22:51.338655+0000","last_active":"2024-09-20T22:23:30.952362+0000","last_peered":"2024-09-20T22:23:30.952362+0000","last_clean":"2024-09-20T22:23:30.952362+0000","last_became_active":"2024-09-20T22:22:51.338112+0000","last_became_peered":"2024-09-20T22:22:51.338112+0000","last_unstale":"2024-09-20T22:23:30.952362+0000","last_undegraded":"2024-09-20T22:23:30.952362+0000","last_fullsized":"2024-09-20T22:23:30.952362+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:44.067640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:22:44.067640+0000","last_clean_scrub_stamp":"2024-09-20T22:22:44.067640+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:48.150761+0000","scrub_duration":58,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":615,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.717964+0000","last_change":"2024-09-20T22:22:36.748696+0000","last_active":"2024-09-20T22:23:31.717964+0000","last_peered":"2024-09-20T22:23:31.717964+0000","last_clean":"2024-09-20T22:23:31.717964+0000","last_became_active":"2024-09-20T22:22:36.747860+0000","last_became_peered":"2024-09-20T22:22:36.747860+0000","last_unstale":"2024-09-20T22:23:31.717964+0000","last_undegraded":"2024-09-20T22:23:31.717964+0000","last_fullsized":"2024-09-20T22:23:31.717964+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:11.135068+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:11.135068+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.14","version":"75'128","reported_seq":1236,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.959843+0000","last_change":"2024-09-20T22:22:38.047138+0000","last_active":"2024-09-20T22:23:31.959843+0000","last_peered":"2024-09-20T22:23:31.959843+0000","last_clean":"2024-09-20T22:23:31.959843+0000","last_became_active":"2024-09-20T22:22:37.719347+0000","last_became_peered":"2024-09-20T22:22:37.719347+0000","last_unstale":"2024-09-20T22:23:31.959843+0000","last_undegraded":"2024-09-20T22:23:31.959843+0000","last_fullsized":"2024-09-20T22:23:31.959843+0000","mapping_epoch":565,"log_start":"0'0","ondisk_log_start":"0'0","created":383,"last_epoch_clean":566,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:38.047061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:38.047061+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:39.387643+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,4],"acting":[2,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"579'27","reported_seq":562,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.952025+0000","last_change":"2024-09-20T22:23:01.029518+0000","last_active":"2024-09-20T22:23:30.952025+0000","last_peered":"2024-09-20T22:23:30.952025+0000","last_clean":"2024-09-20T22:23:30.952025+0000","last_became_active":"2024-09-20T22:22:51.342232+0000","last_became_peered":"2024-09-20T22:22:51.342232+0000","last_unstale":"2024-09-20T22:23:30.952025+0000","last_undegraded":"2024-09-20T22:23:30.952025+0000","last_fullsized":"2024-09-20T22:23:30.952025+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'27","last_scrub_stamp":"2024-09-20T22:23:01.029439+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:23:01.029439+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:25.584617+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":43,"num_read_kb":36,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":685,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.062970+0000","last_change":"2024-09-20T22:22:36.742839+0000","last_active":"2024-09-20T22:23:31.062970+0000","last_peered":"2024-09-20T22:23:31.062970+0000","last_clean":"2024-09-20T22:23:31.062970+0000","last_became_active":"2024-09-20T22:22:36.742517+0000","last_became_peered":"2024-09-20T22:22:36.742517+0000","last_unstale":"2024-09-20T22:23:31.062970+0000","last_undegraded":"2024-09-20T22:23:31.062970+0000","last_fullsized":"2024-09-20T22:23:31.062970+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:18.069137+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:10:36.098315+0000","last_clean_scrub_stamp":"2024-09-20T22:22:18.069137+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:34.524346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1162,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.960894+0000","last_change":"2024-09-20T22:23:21.517779+0000","last_active":"2024-09-20T22:23:31.960894+0000","last_peered":"2024-09-20T22:23:31.960894+0000","last_clean":"2024-09-20T22:23:31.960894+0000","last_became_active":"2024-09-20T22:22:34.756599+0000","last_became_peered":"2024-09-20T22:22:34.756599+0000","last_unstale":"2024-09-20T22:23:31.960894+0000","last_undegraded":"2024-09-20T22:23:31.960894+0000","last_fullsized":"2024-09-20T22:23:31.960894+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:21.517686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:23:21.517686+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:46.367958+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":600,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719834+0000","last_change":"2024-09-20T22:22:41.121676+0000","last_active":"2024-09-20T22:23:31.719834+0000","last_peered":"2024-09-20T22:23:31.719834+0000","last_clean":"2024-09-20T22:23:31.719834+0000","last_became_active":"2024-09-20T22:22:30.665777+0000","last_became_peered":"2024-09-20T22:22:30.665777+0000","last_unstale":"2024-09-20T22:23:31.719834+0000","last_undegraded":"2024-09-20T22:23:31.719834+0000","last_fullsized":"2024-09-20T22:23:31.719834+0000","mapping_epoch":557,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":558,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:41.121604+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:55.278800+0000","last_clean_scrub_stamp":"2024-09-20T22:22:41.121604+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:03.413460+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6],"acting":[0,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.15","version":"75'128","reported_seq":664,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.958374+0000","last_change":"2024-09-20T22:22:39.719978+0000","last_active":"2024-09-20T22:23:31.958374+0000","last_peered":"2024-09-20T22:23:31.958374+0000","last_clean":"2024-09-20T22:23:31.958374+0000","last_became_active":"2024-09-20T22:22:39.719578+0000","last_became_peered":"2024-09-20T22:22:39.719578+0000","last_unstale":"2024-09-20T22:23:31.958374+0000","last_undegraded":"2024-09-20T22:23:31.958374+0000","last_fullsized":"2024-09-20T22:23:31.958374+0000","mapping_epoch":567,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":568,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:31.546728+0000","last_deep_scrub":"75'128","last_deep_scrub_stamp":"2024-09-20T22:22:31.546728+0000","last_clean_scrub_stamp":"2024-09-20T22:22:31.546728+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:37.843719+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,1],"acting":[7,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"579'14","reported_seq":1235,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.951694+0000","last_change":"2024-09-20T22:23:08.171590+0000","last_active":"2024-09-20T22:23:30.951694+0000","last_peered":"2024-09-20T22:23:30.951694+0000","last_clean":"2024-09-20T22:23:30.951694+0000","last_became_active":"2024-09-20T22:22:36.737769+0000","last_became_peered":"2024-09-20T22:22:36.737769+0000","last_unstale":"2024-09-20T22:23:30.951694+0000","last_undegraded":"2024-09-20T22:23:30.951694+0000","last_fullsized":"2024-09-20T22:23:30.951694+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'14","last_scrub_stamp":"2024-09-20T22:23:08.171508+0000","last_deep_scrub":"453'10","last_deep_scrub_stamp":"2024-09-20T22:21:38.900946+0000","last_clean_scrub_stamp":"2024-09-20T22:23:08.171508+0000","objects_scrubbed":2,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:18.059049+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":16,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":591,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.720190+0000","last_change":"2024-09-20T22:22:33.051496+0000","last_active":"2024-09-20T22:23:31.720190+0000","last_peered":"2024-09-20T22:23:31.720190+0000","last_clean":"2024-09-20T22:23:31.720190+0000","last_became_active":"2024-09-20T22:12:52.517510+0000","last_became_peered":"2024-09-20T22:12:52.517510+0000","last_unstale":"2024-09-20T22:23:31.720190+0000","last_undegraded":"2024-09-20T22:23:31.720190+0000","last_fullsized":"2024-09-20T22:23:31.720190+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:33.051367+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:49.083871+0000","last_clean_scrub_stamp":"2024-09-20T22:22:33.051367+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:01.221318+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":100,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.961092+0000","last_change":"2024-09-20T22:22:53.005692+0000","last_active":"2024-09-20T22:23:31.961092+0000","last_peered":"2024-09-20T22:23:31.961092+0000","last_clean":"2024-09-20T22:23:31.961092+0000","last_became_active":"2024-09-20T22:22:34.747685+0000","last_became_peered":"2024-09-20T22:22:34.747685+0000","last_unstale":"2024-09-20T22:23:31.961092+0000","last_undegraded":"2024-09-20T22:23:31.961092+0000","last_fullsized":"2024-09-20T22:23:31.961092+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:53.005597+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:22:53.005597+0000","last_clean_scrub_stamp":"2024-09-20T22:22:53.005597+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:16.098003+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1138,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.958731+0000","last_change":"2024-09-20T22:22:34.738847+0000","last_active":"2024-09-20T22:23:31.958731+0000","last_peered":"2024-09-20T22:23:31.958731+0000","last_clean":"2024-09-20T22:23:31.958731+0000","last_became_active":"2024-09-20T22:22:34.738379+0000","last_became_peered":"2024-09-20T22:22:34.738379+0000","last_unstale":"2024-09-20T22:23:31.958731+0000","last_undegraded":"2024-09-20T22:23:31.958731+0000","last_fullsized":"2024-09-20T22:23:31.958731+0000","mapping_epoch":562,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":563,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:21.514519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:11:05.468863+0000","last_clean_scrub_stamp":"2024-09-20T22:22:21.514519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:48.892847+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1a","version":"75'128","reported_seq":400,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.718608+0000","last_change":"2024-09-20T22:22:40.669892+0000","last_active":"2024-09-20T22:23:31.718608+0000","last_peered":"2024-09-20T22:23:31.718608+0000","last_clean":"2024-09-20T22:23:31.718608+0000","last_became_active":"2024-09-20T22:22:39.715570+0000","last_became_peered":"2024-09-20T22:22:39.715570+0000","last_unstale":"2024-09-20T22:23:31.718608+0000","last_undegraded":"2024-09-20T22:23:31.718608+0000","last_fullsized":"2024-09-20T22:23:31.718608+0000","mapping_epoch":567,"log_start":"0'0","ondisk_log_start":"0'0","created":428,"last_epoch_clean":568,"parent":"0.0","parent_split_bits":5,"last_scrub":"75'128","last_scrub_stamp":"2024-09-20T22:22:40.669777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:00:35.131533+0000","last_clean_scrub_stamp":"2024-09-20T22:22:40.669777+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:00.927551+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,1],"acting":[4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":81,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.719242+0000","last_change":"2024-09-20T22:22:51.237996+0000","last_active":"2024-09-20T22:23:31.719242+0000","last_peered":"2024-09-20T22:23:31.719242+0000","last_clean":"2024-09-20T22:23:31.719242+0000","last_became_active":"2024-09-20T22:22:36.751845+0000","last_became_peered":"2024-09-20T22:22:36.751845+0000","last_unstale":"2024-09-20T22:23:31.719242+0000","last_undegraded":"2024-09-20T22:23:31.719242+0000","last_fullsized":"2024-09-20T22:23:31.719242+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":69,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:51.237919+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:22:51.237919+0000","last_clean_scrub_stamp":"2024-09-20T22:22:51.237919+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:23:55.984423+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":596,"reported_epoch":595,"state":"active+clean","last_fresh":"2024-09-20T22:23:30.951954+0000","last_change":"2024-09-20T22:23:15.172847+0000","last_active":"2024-09-20T22:23:30.951954+0000","last_peered":"2024-09-20T22:23:30.951954+0000","last_clean":"2024-09-20T22:23:30.951954+0000","last_became_active":"2024-09-20T22:22:51.338363+0000","last_became_peered":"2024-09-20T22:22:51.338363+0000","last_unstale":"2024-09-20T22:23:30.951954+0000","last_undegraded":"2024-09-20T22:23:30.951954+0000","last_fullsized":"2024-09-20T22:23:30.951954+0000","mapping_epoch":574,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":575,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:23:15.172777+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:21:39.004178+0000","last_clean_scrub_stamp":"2024-09-20T22:23:15.172777+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:16.913341+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"579'27","reported_seq":136,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.961174+0000","last_change":"2024-09-20T22:23:09.049171+0000","last_active":"2024-09-20T22:23:31.961174+0000","last_peered":"2024-09-20T22:23:31.961174+0000","last_clean":"2024-09-20T22:23:31.961174+0000","last_became_active":"2024-09-20T22:22:36.741967+0000","last_became_peered":"2024-09-20T22:22:36.741967+0000","last_unstale":"2024-09-20T22:23:31.961174+0000","last_undegraded":"2024-09-20T22:23:31.961174+0000","last_fullsized":"2024-09-20T22:23:31.961174+0000","mapping_epoch":564,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":565,"parent":"0.0","parent_split_bits":0,"last_scrub":"579'27","last_scrub_stamp":"2024-09-20T22:23:09.049009+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:51.256183+0000","last_clean_scrub_stamp":"2024-09-20T22:23:09.049009+0000","objects_scrubbed":5,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-20T22:24:21.052436+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":220,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":592,"reported_epoch":596,"state":"active+clean","last_fresh":"2024-09-20T22:23:31.720119+0000","last_change":"2024-09-20T22:22:10.095005+0000","last_active":"2024-09-20T22:23:31.720119+0000","last_peered":"2024-09-20T22:23:31.720119+0000","last_clean":"2024-09-20T22:23:31.720119+0000","last_became_active":"2024-09-20T22:12:52.524425+0000","last_became_peered":"2024-09-20T22:12:52.524425+0000","last_unstale":"2024-09-20T22:23:31.720119+0000","last_undegraded":"2024-09-20T22:23:31.720119+0000","last_fullsized":"2024-09-20T22:23:31.720119+0000","mapping_epoch":323,"log_start":"0'0","ondisk_log_start":"0'0","created":67,"last_epoch_clean":324,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-20T22:22:10.094934+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-20T22:02:53.269054+0000","last_clean_scrub_stamp":"2024-09-20T22:22:10.094934+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for scrub","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":416,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":26,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":113,"ondisk_log_size":113,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":1528,"num_read_kb":1285,"num_write":989,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":456,"num_bytes_recovered":9550,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1052,"ondisk_log_size":1052,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1613,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":17,"num_read_kb":17,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":3044,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":96,"acting":96,"num_store_stats":8},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1216,"num_read_kb":1211,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":10,"ondisk_log_size":10,"up":9,"acting":9,"num_store_stats":7},{"poolid":1,"num_pg":31,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":655360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3968,"ondisk_log_size":3968,"up":93,"acting":93,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":57,"seq":244813136126,"num_pgs":68,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":881508,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":879422,"kb_avail":92863644,"statfs":{"total":95995035648,"available":95092371456,"internally_reserved":0,"allocated":2117632,"data_stored":1425681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":900528586},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":48,"seq":206158430463,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":755220,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":753086,"kb_avail":92989932,"statfs":{"total":95995035648,"available":95221690368,"internally_reserved":0,"allocated":2162688,"data_stored":1490581,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":771160518},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724805,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1002236,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":1000190,"kb_avail":92742916,"statfs":{"total":95995035648,"available":94968745984,"internally_reserved":0,"allocated":2072576,"data_stored":1409164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1024195013},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":532,"seq":2284922601496,"num_pgs":69,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":84400,"kb_used_data":1816,"kb_used_omap":118,"kb_used_meta":82441,"kb_avail":93660752,"statfs":{"total":95995035648,"available":95908610048,"internally_reserved":0,"allocated":1859584,"data_stored":1162936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":120876,"internal_metadata":84420564},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":145,"seq":622770258122,"num_pgs":65,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":722884,"kb_used_data":1836,"kb_used_omap":4,"kb_used_meta":721019,"kb_avail":93022268,"statfs":{"total":95995035648,"available":95254802432,"internally_reserved":0,"allocated":1880064,"data_stored":1164233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4440,"internal_metadata":738324136},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149970,"num_pgs":67,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":944032,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":941886,"kb_avail":92801120,"statfs":{"total":95995035648,"available":95028346880,"internally_reserved":0,"allocated":2179072,"data_stored":1476130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":964491719},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411607,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":755156,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":753086,"kb_avail":92989996,"statfs":{"total":95995035648,"available":95221755904,"internally_reserved":0,"allocated":2093056,"data_stored":1417282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":771160517},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738658,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":1007920,"kb_used_data":2132,"kb_used_omap":1,"kb_used_meta":1005758,"kb_avail":92737232,"statfs":{"total":95995035648,"available":94962925568,"internally_reserved":0,"allocated":2183168,"data_stored":1490805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":1029896645},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":266240,"data_stored":262688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-20T22:23:38.143 INFO:tasks.cephadm.ceph_manager.ceph:recovered! 2024-09-20T22:23:38.143 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736 shell --fsid 3df9647e-779b-11ef-baf5-efdc52797490 -- ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-09-20T22:23:38.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:37 smithi067 ceph-mon[33511]: 3.16 scrub starts 2024-09-20T22:23:38.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:37 smithi067 ceph-mon[33511]: 3.16 scrub ok 2024-09-20T22:23:38.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:37 smithi067 ceph-mon[33511]: 6.10 scrub starts 2024-09-20T22:23:38.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:37 smithi067 ceph-mon[33511]: 6.10 scrub ok 2024-09-20T22:23:38.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:37 smithi067 ceph-mon[33511]: from='client.16974 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-20T22:23:38.405 INFO:teuthology.orchestra.run.smithi060.stderr:Inferring config /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/mon.a/config 2024-09-20T22:23:39.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:38 smithi060 ceph-mon[40290]: 6.b scrub starts 2024-09-20T22:23:39.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:38 smithi060 ceph-mon[40290]: 6.b scrub ok 2024-09-20T22:23:39.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:38 smithi060 ceph-mon[40290]: 3.4 deep-scrub starts 2024-09-20T22:23:39.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:38 smithi060 ceph-mon[40290]: 3.4 deep-scrub ok 2024-09-20T22:23:39.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:38 smithi060 ceph-mon[40290]: pgmap v1135: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:23:39.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:38 smithi060 ceph-mon[40290]: 1.15 scrub starts 2024-09-20T22:23:39.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:38 smithi060 ceph-mon[40290]: 1.15 scrub ok 2024-09-20T22:23:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:38 smithi060 ceph-mon[29694]: 6.b scrub starts 2024-09-20T22:23:39.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:38 smithi060 ceph-mon[29694]: 6.b scrub ok 2024-09-20T22:23:39.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:38 smithi060 ceph-mon[29694]: 3.4 deep-scrub starts 2024-09-20T22:23:39.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:38 smithi060 ceph-mon[29694]: 3.4 deep-scrub ok 2024-09-20T22:23:39.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:38 smithi060 ceph-mon[29694]: pgmap v1135: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:23:39.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:38 smithi060 ceph-mon[29694]: 1.15 scrub starts 2024-09-20T22:23:39.037 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:38 smithi060 ceph-mon[29694]: 1.15 scrub ok 2024-09-20T22:23:39.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:38 smithi067 ceph-mon[33511]: 6.b scrub starts 2024-09-20T22:23:39.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:38 smithi067 ceph-mon[33511]: 6.b scrub ok 2024-09-20T22:23:39.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:38 smithi067 ceph-mon[33511]: 3.4 deep-scrub starts 2024-09-20T22:23:39.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:38 smithi067 ceph-mon[33511]: 3.4 deep-scrub ok 2024-09-20T22:23:39.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:38 smithi067 ceph-mon[33511]: pgmap v1135: 162 pgs: 162 active+clean; 583 KiB data, 5.9 GiB used, 709 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-20T22:23:39.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:38 smithi067 ceph-mon[33511]: 1.15 scrub starts 2024-09-20T22:23:39.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:38 smithi067 ceph-mon[33511]: 1.15 scrub ok 2024-09-20T22:23:39.410 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:23:39] ENGINE Bus STOPPING 2024-09-20T22:23:39.745 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:23:39] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-20T22:23:39.745 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:23:39] ENGINE Bus STOPPED 2024-09-20T22:23:39.745 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:23:39] ENGINE Bus STARTING 2024-09-20T22:23:39.745 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:23:39] ENGINE Serving on http://:::9283 2024-09-20T22:23:39.745 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:23:39] ENGINE Bus STARTED 2024-09-20T22:23:39.781 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-20T22:23:39.789 INFO:tasks.cephadm:Teardown begin 2024-09-20T22:23:39.789 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-20T22:23:39.851 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-20T22:23:39.910 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-20T22:23:39.910 DEBUG:teuthology.orchestra.run.smithi060:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-20T22:23:39.944 DEBUG:teuthology.orchestra.run.smithi067:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-20T22:23:39.970 INFO:tasks.cephadm:Stopping all daemons... 2024-09-20T22:23:39.970 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-20T22:23:39.970 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-3df9647e-779b-11ef-baf5-efdc52797490@mon.a 2024-09-20T22:23:40.008 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[29694]: 6.17 scrub starts 2024-09-20T22:23:40.008 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[29694]: 6.17 scrub ok 2024-09-20T22:23:40.008 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[29694]: 5.7 scrub starts 2024-09-20T22:23:40.008 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[29694]: 5.7 scrub ok 2024-09-20T22:23:40.008 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[29694]: 4.1d scrub starts 2024-09-20T22:23:40.008 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[29694]: 4.1d scrub ok 2024-09-20T22:23:40.008 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[29694]: from='client.? 172.21.15.60:0/4125110332' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-20T22:23:40.008 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-20T22:23:40.008 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[29694]: from='client.? ' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-20T22:23:40.008 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:23:40.008 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:23:40.008 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:23:40.009 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[29694]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:23:40.009 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[40290]: 6.17 scrub starts 2024-09-20T22:23:40.009 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[40290]: 6.17 scrub ok 2024-09-20T22:23:40.009 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[40290]: 5.7 scrub starts 2024-09-20T22:23:40.009 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[40290]: 5.7 scrub ok 2024-09-20T22:23:40.009 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[40290]: 4.1d scrub starts 2024-09-20T22:23:40.009 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[40290]: 4.1d scrub ok 2024-09-20T22:23:40.009 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[40290]: from='client.? 172.21.15.60:0/4125110332' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-20T22:23:40.009 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-20T22:23:40.009 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[40290]: from='client.? ' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-20T22:23:40.009 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:23:40.009 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:23:40.010 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:23:40.010 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:39 smithi060 ceph-mon[40290]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:23:40.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:39 smithi067 ceph-mon[33511]: 6.17 scrub starts 2024-09-20T22:23:40.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:39 smithi067 ceph-mon[33511]: 6.17 scrub ok 2024-09-20T22:23:40.263 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:39 smithi067 ceph-mon[33511]: 5.7 scrub starts 2024-09-20T22:23:40.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:39 smithi067 ceph-mon[33511]: 5.7 scrub ok 2024-09-20T22:23:40.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:39 smithi067 ceph-mon[33511]: 4.1d scrub starts 2024-09-20T22:23:40.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:39 smithi067 ceph-mon[33511]: 4.1d scrub ok 2024-09-20T22:23:40.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:39 smithi067 ceph-mon[33511]: from='client.? 172.21.15.60:0/4125110332' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-20T22:23:40.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:39 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd=[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]: dispatch 2024-09-20T22:23:40.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:39 smithi067 ceph-mon[33511]: from='client.? ' entity='client.admin' cmd='[{"prefix": "config rm", "who": "mgr", "name": "mgr_debug_aggressive_pg_num_changes"}]': finished 2024-09-20T22:23:40.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:39 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-20T22:23:40.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:39 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-20T22:23:40.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:39 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-20T22:23:40.264 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:39 smithi067 ceph-mon[33511]: from='mgr.24494 172.21.15.60:0/59072180' entity='mgr.y' 2024-09-20T22:23:40.286 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:40 smithi060 systemd[1]: Stopping Ceph mon.a for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:23:40.624 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:40 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:23:40] ENGINE Bus STOPPING 2024-09-20T22:23:40.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:40 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a[29671]: 2024-09-20T22:23:40.316+0000 7f01b34d1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T22:23:40.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:40 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a[29671]: 2024-09-20T22:23:40.316+0000 7f01b34d1640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-20T22:23:40.624 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:40 smithi060 podman[226002]: 2024-09-20 22:23:40.347253533 +0000 UTC m=+0.153148977 container died e363e67817222ccb42015593a937673a46aae158883950c4035ffe4bc2de6be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-9e2ac70, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T22:23:41.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:40 smithi060 podman[226002]: 2024-09-20 22:23:40.823453208 +0000 UTC m=+0.629348641 container cleanup e363e67817222ccb42015593a937673a46aae158883950c4035ffe4bc2de6be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-9e2ac70, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0) 2024-09-20T22:23:41.036 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:40 smithi060 bash[226002]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a 2024-09-20T22:23:41.036 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:40 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:23:40] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-20T22:23:41.036 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:40 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:23:40] ENGINE Bus STOPPED 2024-09-20T22:23:41.036 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:40 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:23:40] ENGINE Bus STARTING 2024-09-20T22:23:41.036 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:40 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:23:40] ENGINE Serving on http://:::9283 2024-09-20T22:23:41.037 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:40 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y[29947]: [20/Sep/2024:22:23:40] ENGINE Bus STARTED 2024-09-20T22:23:41.292 INFO:journalctl@ceph.mon.a.smithi060.stdout:Sep 20 22:23:41 smithi060 podman[226019]: 2024-09-20 22:23:41.125201336 +0000 UTC m=+0.772187518 container remove e363e67817222ccb42015593a937673a46aae158883950c4035ffe4bc2de6be1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-a, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-9e2ac70, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-20T22:23:41.311 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@mon.a.service' 2024-09-20T22:23:41.395 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T22:23:41.395 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-20T22:23:41.395 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-20T22:23:41.395 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-3df9647e-779b-11ef-baf5-efdc52797490@mon.c 2024-09-20T22:23:41.739 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:41 smithi060 systemd[1]: Stopping Ceph mon.c for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:23:42.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:41 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-c[40267]: 2024-09-20T22:23:41.738+0000 7fd8b53eb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T22:23:42.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:41 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-c[40267]: 2024-09-20T22:23:41.738+0000 7fd8b53eb640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-20T22:23:42.036 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:41 smithi060 podman[226145]: 2024-09-20 22:23:41.769651471 +0000 UTC m=+0.146563811 container died a7ce8d1ab286d47dc7f3f55c0f76949af7bdadf8d6bba8a5eae3301d4d36d636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-c, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=squid-9e2ac70, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-20T22:23:42.514 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:42 smithi060 podman[226145]: 2024-09-20 22:23:42.253772112 +0000 UTC m=+0.630684452 container cleanup a7ce8d1ab286d47dc7f3f55c0f76949af7bdadf8d6bba8a5eae3301d4d36d636 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-c, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-9e2ac70, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T22:23:42.515 INFO:journalctl@ceph.mon.c.smithi060.stdout:Sep 20 22:23:42 smithi060 bash[226145]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-c 2024-09-20T22:23:42.691 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@mon.c.service' 2024-09-20T22:23:42.731 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T22:23:42.731 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-20T22:23:42.731 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-20T22:23:42.731 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-3df9647e-779b-11ef-baf5-efdc52797490@mon.b 2024-09-20T22:23:43.013 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:42 smithi067 systemd[1]: Stopping Ceph mon.b for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:23:43.014 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:23:43 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug there is no tcmu-runner data available 2024-09-20T22:23:43.506 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:43 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-b[33480]: 2024-09-20T22:23:43.062+0000 7fe6b4190640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T22:23:43.506 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:43 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-b[33480]: 2024-09-20T22:23:43.062+0000 7fe6b4190640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-20T22:23:43.506 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:43 smithi067 podman[159009]: 2024-09-20 22:23:43.227244612 +0000 UTC m=+0.287876938 container died 497f6b558d69936edda24dcbdf173b5a215e5bfb44eb5a3cb4f818dea5aa7ab5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-9e2ac70, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-20T22:23:43.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:43 smithi067 podman[159009]: 2024-09-20 22:23:43.678051924 +0000 UTC m=+0.738684292 container cleanup 497f6b558d69936edda24dcbdf173b5a215e5bfb44eb5a3cb4f818dea5aa7ab5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-9e2ac70, org.label-schema.license=GPLv2) 2024-09-20T22:23:43.763 INFO:journalctl@ceph.mon.b.smithi067.stdout:Sep 20 22:23:43 smithi067 bash[159009]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-mon-b 2024-09-20T22:23:44.124 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@mon.b.service' 2024-09-20T22:23:44.164 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T22:23:44.166 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-20T22:23:44.166 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-20T22:23:44.166 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-3df9647e-779b-11ef-baf5-efdc52797490@mgr.y 2024-09-20T22:23:44.286 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:44 smithi060 systemd[1]: Stopping Ceph mgr.y for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:23:44.786 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:44 smithi060 podman[226288]: 2024-09-20 22:23:44.508239122 +0000 UTC m=+0.148331171 container died 66443b29df2ff148d9da4279bf9b97516f42f8df2a03a597bef1dd678382db79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-9e2ac70, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-20T22:23:45.203 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:44 smithi060 podman[226288]: 2024-09-20 22:23:44.883746079 +0000 UTC m=+0.523838159 container cleanup 66443b29df2ff148d9da4279bf9b97516f42f8df2a03a597bef1dd678382db79 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9e2ac70bb7aea67d34f0d50002344abbe7803736, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-9e2ac70, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-20T22:23:45.203 INFO:journalctl@ceph.mgr.y.smithi060.stdout:Sep 20 22:23:44 smithi060 bash[226288]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-y 2024-09-20T22:23:45.377 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-20T22:23:45.414 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T22:23:45.414 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-20T22:23:45.414 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-20T22:23:45.414 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-3df9647e-779b-11ef-baf5-efdc52797490@mgr.x 2024-09-20T22:23:45.750 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:23:45 smithi067 systemd[1]: Stopping Ceph mgr.x for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:23:46.013 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:23:45 smithi067 podman[159141]: 2024-09-20 22:23:45.749568491 +0000 UTC m=+0.133395076 container died 6062092286ad3070224d6bb5a8af2c1371cb00ccbb62cff57195475007ba5181 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-9e2ac70, CEPH_POINT_RELEASE=) 2024-09-20T22:23:46.427 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:23:46 smithi067 podman[159141]: 2024-09-20 22:23:46.158341834 +0000 UTC m=+0.542168411 container cleanup 6062092286ad3070224d6bb5a8af2c1371cb00ccbb62cff57195475007ba5181 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, RELEASE=squid-9e2ac70, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-20T22:23:46.427 INFO:journalctl@ceph.mgr.x.smithi067.stdout:Sep 20 22:23:46 smithi067 bash[159141]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-mgr-x 2024-09-20T22:23:46.565 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-20T22:23:46.601 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T22:23:46.601 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-20T22:23:46.601 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-20T22:23:46.601 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.0 2024-09-20T22:23:46.900 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:23:46 smithi060 systemd[1]: Stopping Ceph osd.0 for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:23:47.286 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:23:46 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-0[44430]: 2024-09-20T22:23:46.898+0000 7f446a043640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T22:23:47.286 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:23:46 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-0[44430]: 2024-09-20T22:23:46.898+0000 7f446a043640 -1 osd.0 596 *** Got signal Terminated *** 2024-09-20T22:23:47.286 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:23:46 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-0[44430]: 2024-09-20T22:23:46.898+0000 7f446a043640 -1 osd.0 596 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T22:23:52.842 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:23:52 smithi060 podman[226420]: 2024-09-20 22:23:52.539939053 +0000 UTC m=+5.737101879 container died 06b4526d5bac07e8f20f777f5ef67cd79aa8f489de64dfeaec19db663c7716a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-0, org.label-schema.license=GPLv2, RELEASE=squid-9e2ac70, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-20T22:23:53.274 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:23:53 smithi060 podman[226420]: 2024-09-20 22:23:53.013707199 +0000 UTC m=+6.210870024 container cleanup 06b4526d5bac07e8f20f777f5ef67cd79aa8f489de64dfeaec19db663c7716a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-9e2ac70, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-20T22:23:53.274 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:23:53 smithi060 bash[226420]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-0 2024-09-20T22:23:53.536 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:23:53 smithi060 podman[226440]: 2024-09-20 22:23:53.273692507 +0000 UTC m=+0.735103518 container remove 06b4526d5bac07e8f20f777f5ef67cd79aa8f489de64dfeaec19db663c7716a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-9e2ac70, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-20T22:23:53.844 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:23:53 smithi060 podman[226535]: 2024-09-20 22:23:53.591161056 +0000 UTC m=+0.110726622 container create b38d6c3dbafdc2b15c7dc49d9f67964e2eaf427dc5afd3b2e52589f16809647a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=squid-9e2ac70, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-20T22:23:53.845 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:23:53 smithi060 podman[226535]: 2024-09-20 22:23:53.497030506 +0000 UTC m=+0.016596038 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5 2024-09-20T22:23:53.845 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:23:53 smithi060 podman[226535]: 2024-09-20 22:23:53.8436385 +0000 UTC m=+0.363204027 container init b38d6c3dbafdc2b15c7dc49d9f67964e2eaf427dc5afd3b2e52589f16809647a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=squid-9e2ac70, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-20T22:23:54.153 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:23:53 smithi060 podman[226535]: 2024-09-20 22:23:53.848730585 +0000 UTC m=+0.368296105 container start b38d6c3dbafdc2b15c7dc49d9f67964e2eaf427dc5afd3b2e52589f16809647a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-0-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-9e2ac70, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T22:23:54.153 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:23:53 smithi060 podman[226535]: 2024-09-20 22:23:53.960231102 +0000 UTC m=+0.479796629 container attach b38d6c3dbafdc2b15c7dc49d9f67964e2eaf427dc5afd3b2e52589f16809647a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-9e2ac70, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-20T22:23:54.448 INFO:journalctl@ceph.osd.0.smithi060.stdout:Sep 20 22:23:54 smithi060 podman[226535]: 2024-09-20 22:23:54.152344632 +0000 UTC m=+0.671910162 container died b38d6c3dbafdc2b15c7dc49d9f67964e2eaf427dc5afd3b2e52589f16809647a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-9e2ac70, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T22:23:54.873 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.0.service' 2024-09-20T22:23:54.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T22:23:54.917 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-20T22:23:54.917 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-20T22:23:54.917 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.1 2024-09-20T22:23:55.263 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:23:54 smithi060 systemd[1]: Stopping Ceph osd.1 for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:23:55.536 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:23:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1[101165]: 2024-09-20T22:23:55.261+0000 7f15a632f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T22:23:55.536 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:23:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1[101165]: 2024-09-20T22:23:55.261+0000 7f15a632f640 -1 osd.1 596 *** Got signal Terminated *** 2024-09-20T22:23:55.536 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:23:55 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1[101165]: 2024-09-20T22:23:55.261+0000 7f15a632f640 -1 osd.1 596 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T22:23:57.263 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:23:56 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=cleanup t=2024-09-20T22:23:56.804909062Z level=info msg="Completed cleanup jobs" duration=2.418627ms 2024-09-20T22:24:01.137 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:24:00 smithi060 podman[226664]: 2024-09-20 22:24:00.815040925 +0000 UTC m=+5.670716900 container died ec8a7318fda765e76c05c0474fc38573c84b1da012032b49870e2acca31a4a29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-9e2ac70, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-20T22:24:01.536 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:24:01 smithi060 podman[226664]: 2024-09-20 22:24:01.318185894 +0000 UTC m=+6.173861904 container cleanup ec8a7318fda765e76c05c0474fc38573c84b1da012032b49870e2acca31a4a29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1, org.label-schema.license=GPLv2, RELEASE=squid-9e2ac70, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T22:24:01.536 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:24:01 smithi060 bash[226664]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1 2024-09-20T22:24:01.888 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:24:01 smithi060 podman[226686]: 2024-09-20 22:24:01.569664057 +0000 UTC m=+0.738430585 container remove ec8a7318fda765e76c05c0474fc38573c84b1da012032b49870e2acca31a4a29 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-9e2ac70, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-20T22:24:02.215 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:24:01 smithi060 podman[226779]: 2024-09-20 22:24:01.790603372 +0000 UTC m=+0.016728626 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5 2024-09-20T22:24:02.216 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:24:01 smithi060 podman[226779]: 2024-09-20 22:24:01.937153885 +0000 UTC m=+0.163279181 container create c742c0e8def3abe9d5500dd5d71bffbc2bb52c493af5c81c6280b60faaf20ef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=squid-9e2ac70, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-20T22:24:02.216 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:24:02 smithi060 podman[226779]: 2024-09-20 22:24:02.131422798 +0000 UTC m=+0.357548050 container init c742c0e8def3abe9d5500dd5d71bffbc2bb52c493af5c81c6280b60faaf20ef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-9e2ac70, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-20T22:24:02.216 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:24:02 smithi060 podman[226779]: 2024-09-20 22:24:02.136191111 +0000 UTC m=+0.362316372 container start c742c0e8def3abe9d5500dd5d71bffbc2bb52c493af5c81c6280b60faaf20ef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-9e2ac70, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-20T22:24:02.536 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:24:02 smithi060 podman[226779]: 2024-09-20 22:24:02.214482976 +0000 UTC m=+0.440608228 container attach c742c0e8def3abe9d5500dd5d71bffbc2bb52c493af5c81c6280b60faaf20ef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-9e2ac70, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-20T22:24:02.536 INFO:journalctl@ceph.osd.1.smithi060.stdout:Sep 20 22:24:02 smithi060 podman[226779]: 2024-09-20 22:24:02.331948348 +0000 UTC m=+0.558073617 container died c742c0e8def3abe9d5500dd5d71bffbc2bb52c493af5c81c6280b60faaf20ef8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-1-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-9e2ac70, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-20T22:24:03.097 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.1.service' 2024-09-20T22:24:03.144 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T22:24:03.144 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-20T22:24:03.144 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-20T22:24:03.145 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.2 2024-09-20T22:24:03.442 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:24:03 smithi060 systemd[1]: Stopping Ceph osd.2 for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:24:03.786 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:24:03 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-2[56776]: 2024-09-20T22:24:03.440+0000 7f73226fc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T22:24:03.786 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:24:03 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-2[56776]: 2024-09-20T22:24:03.440+0000 7f73226fc640 -1 osd.2 596 *** Got signal Terminated *** 2024-09-20T22:24:03.786 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:24:03 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-2[56776]: 2024-09-20T22:24:03.440+0000 7f73226fc640 -1 osd.2 596 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T22:24:09.382 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:24:09 smithi060 podman[226904]: 2024-09-20 22:24:09.099677126 +0000 UTC m=+5.761017089 container died 6388d405e0fee4b26f35ca402f8b7980912ad90f3ece541dccdba64f4be44465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-9e2ac70, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-20T22:24:09.723 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:24:09 smithi060 podman[226904]: 2024-09-20 22:24:09.538230079 +0000 UTC m=+6.199570088 container cleanup 6388d405e0fee4b26f35ca402f8b7980912ad90f3ece541dccdba64f4be44465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-2, org.label-schema.build-date=20240731, RELEASE=squid-9e2ac70, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-20T22:24:09.724 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:24:09 smithi060 bash[226904]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-2 2024-09-20T22:24:10.035 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:24:09 smithi060 podman[226925]: 2024-09-20 22:24:09.722843105 +0000 UTC m=+0.623329141 container remove 6388d405e0fee4b26f35ca402f8b7980912ad90f3ece541dccdba64f4be44465 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-9e2ac70, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-20T22:24:10.518 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:24:10 smithi060 podman[227021]: 2024-09-20 22:24:10.156989954 +0000 UTC m=+0.106673628 container create 32501e984e6b291d798d8d3f76a18b94b69f0d22b6265db37912b37f51149116 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-9e2ac70, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T22:24:10.518 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:24:10 smithi060 podman[227021]: 2024-09-20 22:24:10.06676788 +0000 UTC m=+0.016451562 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5 2024-09-20T22:24:10.519 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:24:10 smithi060 podman[227021]: 2024-09-20 22:24:10.392919952 +0000 UTC m=+0.342603646 container init 32501e984e6b291d798d8d3f76a18b94b69f0d22b6265db37912b37f51149116 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-9e2ac70, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-20T22:24:10.519 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:24:10 smithi060 podman[227021]: 2024-09-20 22:24:10.39807554 +0000 UTC m=+0.347759213 container start 32501e984e6b291d798d8d3f76a18b94b69f0d22b6265db37912b37f51149116 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=squid-9e2ac70, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2) 2024-09-20T22:24:10.519 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:24:10 smithi060 podman[227021]: 2024-09-20 22:24:10.517657382 +0000 UTC m=+0.467341067 container attach 32501e984e6b291d798d8d3f76a18b94b69f0d22b6265db37912b37f51149116 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-9e2ac70, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-20T22:24:10.786 INFO:journalctl@ceph.osd.2.smithi060.stdout:Sep 20 22:24:10 smithi060 podman[227021]: 2024-09-20 22:24:10.591898014 +0000 UTC m=+0.541581690 container died 32501e984e6b291d798d8d3f76a18b94b69f0d22b6265db37912b37f51149116 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-9e2ac70, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-20T22:24:11.400 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.2.service' 2024-09-20T22:24:11.446 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T22:24:11.446 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-20T22:24:11.446 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-20T22:24:11.446 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.3 2024-09-20T22:24:11.771 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 20 22:24:11 smithi060 systemd[1]: Stopping Ceph osd.3 for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:24:12.036 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 20 22:24:11 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-3[63042]: 2024-09-20T22:24:11.769+0000 7fe71e324640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T22:24:12.036 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 20 22:24:11 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-3[63042]: 2024-09-20T22:24:11.769+0000 7fe71e324640 -1 osd.3 596 *** Got signal Terminated *** 2024-09-20T22:24:12.036 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 20 22:24:11 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-3[63042]: 2024-09-20T22:24:11.769+0000 7fe71e324640 -1 osd.3 596 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T22:24:13.013 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:24:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:24:12.584Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.60:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.60:8765: connect: connection refused" 2024-09-20T22:24:13.014 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:24:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:24:12.586Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.60:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.60:8765: connect: connection refused" 2024-09-20T22:24:13.014 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:24:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:24:12.587Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.60:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.60:8765: connect: connection refused" 2024-09-20T22:24:13.014 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:24:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:24:12.587Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.60:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.60:8765: connect: connection refused" 2024-09-20T22:24:13.014 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:24:12 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:24:12.587Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.60:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.60:8765: connect: connection refused" 2024-09-20T22:24:15.513 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:15 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:15.213+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:16.013 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:15 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[154790]: 2024-09-20T22:24:15.719+0000 7fd9458ad640 -1 osd.4 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.860718+0000 front 2024-09-20T22:23:50.860792+0000 (oldest deadline 2024-09-20T22:24:15.560758+0000) 2024-09-20T22:24:16.013 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:15 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:15.929+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.370657+0000 front 2024-09-20T22:23:50.370787+0000 (oldest deadline 2024-09-20T22:24:15.070546+0000) 2024-09-20T22:24:16.513 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:16 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:16.228+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:16.915 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:16 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[154790]: 2024-09-20T22:24:16.691+0000 7fd9458ad640 -1 osd.4 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.860718+0000 front 2024-09-20T22:23:50.860792+0000 (oldest deadline 2024-09-20T22:24:15.560758+0000) 2024-09-20T22:24:16.915 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:16 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:16.564+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:17.186 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:16 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:16.913+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.370657+0000 front 2024-09-20T22:23:50.370787+0000 (oldest deadline 2024-09-20T22:24:15.070546+0000) 2024-09-20T22:24:17.513 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:17 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:17.184+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:17.627 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 20 22:24:17 smithi060 podman[227147]: 2024-09-20 22:24:17.339670436 +0000 UTC m=+5.698970360 container died 551069d8d8846bb1324209b55ef9b48b8f30bad3ed24f741152b59f9e9b68473 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-9e2ac70, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-20T22:24:17.933 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:17 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:17.533+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:17.934 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:17 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[154790]: 2024-09-20T22:24:17.685+0000 7fd9458ad640 -1 osd.4 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.860718+0000 front 2024-09-20T22:23:50.860792+0000 (oldest deadline 2024-09-20T22:24:15.560758+0000) 2024-09-20T22:24:18.036 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 20 22:24:17 smithi060 podman[227147]: 2024-09-20 22:24:17.809396133 +0000 UTC m=+6.168696090 container cleanup 551069d8d8846bb1324209b55ef9b48b8f30bad3ed24f741152b59f9e9b68473 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-3, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-9e2ac70, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-20T22:24:18.036 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 20 22:24:17 smithi060 bash[227147]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-3 2024-09-20T22:24:18.204 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:17 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:17.931+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.370657+0000 front 2024-09-20T22:23:50.370787+0000 (oldest deadline 2024-09-20T22:24:15.070546+0000) 2024-09-20T22:24:18.437 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 20 22:24:18 smithi060 podman[227181]: 2024-09-20 22:24:18.127307981 +0000 UTC m=+0.778615356 container remove 551069d8d8846bb1324209b55ef9b48b8f30bad3ed24f741152b59f9e9b68473 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-3, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=squid-9e2ac70, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-20T22:24:18.513 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:18 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:18.202+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:18.765 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 20 22:24:18 smithi060 podman[227273]: 2024-09-20 22:24:18.436432243 +0000 UTC m=+0.108979129 container create 57a3398d10e414b87d6d9563dccf4ff793151fd91c4a4dc04902b9070cf11581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-9e2ac70, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-20T22:24:18.765 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 20 22:24:18 smithi060 podman[227273]: 2024-09-20 22:24:18.344606252 +0000 UTC m=+0.017153140 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5 2024-09-20T22:24:18.765 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 20 22:24:18 smithi060 podman[227273]: 2024-09-20 22:24:18.680616955 +0000 UTC m=+0.353163842 container init 57a3398d10e414b87d6d9563dccf4ff793151fd91c4a4dc04902b9070cf11581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=squid-9e2ac70, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-20T22:24:18.765 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 20 22:24:18 smithi060 podman[227273]: 2024-09-20 22:24:18.685888 +0000 UTC m=+0.358434891 container start 57a3398d10e414b87d6d9563dccf4ff793151fd91c4a4dc04902b9070cf11581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-9e2ac70, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-20T22:24:18.765 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 20 22:24:18 smithi060 podman[227273]: 2024-09-20 22:24:18.763908468 +0000 UTC m=+0.436455358 container attach 57a3398d10e414b87d6d9563dccf4ff793151fd91c4a4dc04902b9070cf11581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-3-deactivate, RELEASE=squid-9e2ac70, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-20T22:24:18.891 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:18 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[154790]: 2024-09-20T22:24:18.639+0000 7fd9458ad640 -1 osd.4 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.860718+0000 front 2024-09-20T22:23:50.860792+0000 (oldest deadline 2024-09-20T22:24:15.560758+0000) 2024-09-20T22:24:18.891 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:18 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:18.534+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:19.036 INFO:journalctl@ceph.osd.3.smithi060.stdout:Sep 20 22:24:18 smithi060 podman[227273]: 2024-09-20 22:24:18.885066701 +0000 UTC m=+0.557613588 container died 57a3398d10e414b87d6d9563dccf4ff793151fd91c4a4dc04902b9070cf11581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-9e2ac70, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, ceph=True) 2024-09-20T22:24:19.188 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:18 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:18.888+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.370657+0000 front 2024-09-20T22:23:50.370787+0000 (oldest deadline 2024-09-20T22:24:15.070546+0000) 2024-09-20T22:24:19.513 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:19.186+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:19.515 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.3.service' 2024-09-20T22:24:19.557 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T22:24:19.557 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-20T22:24:19.557 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-20T22:24:19.557 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.4 2024-09-20T22:24:19.872 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:19.514+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:19.872 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:19 smithi067 systemd[1]: Stopping Ceph osd.4 for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:24:19.873 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[154790]: 2024-09-20T22:24:19.675+0000 7fd9458ad640 -1 osd.4 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.860718+0000 front 2024-09-20T22:23:50.860792+0000 (oldest deadline 2024-09-20T22:24:15.560758+0000) 2024-09-20T22:24:19.873 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[154790]: 2024-09-20T22:24:19.870+0000 7fd94d29f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T22:24:19.873 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[154790]: 2024-09-20T22:24:19.870+0000 7fd94d29f640 -1 osd.4 596 *** Got signal Terminated *** 2024-09-20T22:24:20.179 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[154790]: 2024-09-20T22:24:19.870+0000 7fd94d29f640 -1 osd.4 596 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T22:24:20.180 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:19 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:19.886+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.370657+0000 front 2024-09-20T22:23:50.370787+0000 (oldest deadline 2024-09-20T22:24:15.070546+0000) 2024-09-20T22:24:20.513 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:20 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:20.177+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:20.763 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:20 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:20.511+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:20.764 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:20 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[154790]: 2024-09-20T22:24:20.656+0000 7fd9458ad640 -1 osd.4 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.860718+0000 front 2024-09-20T22:23:50.860792+0000 (oldest deadline 2024-09-20T22:24:15.560758+0000) 2024-09-20T22:24:21.175 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:20 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:20.849+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.370657+0000 front 2024-09-20T22:23:50.370787+0000 (oldest deadline 2024-09-20T22:24:15.070546+0000) 2024-09-20T22:24:21.485 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:21 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:21.173+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:21.763 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:21 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:21.483+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:21.763 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:21 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[154790]: 2024-09-20T22:24:21.627+0000 7fd9458ad640 -1 osd.4 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.860718+0000 front 2024-09-20T22:23:50.860792+0000 (oldest deadline 2024-09-20T22:24:15.560758+0000) 2024-09-20T22:24:22.221 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:21 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:21.861+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.370657+0000 front 2024-09-20T22:23:50.370787+0000 (oldest deadline 2024-09-20T22:24:15.070546+0000) 2024-09-20T22:24:22.513 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:22.218+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:22.899 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[154790]: 2024-09-20T22:24:22.610+0000 7fd9458ad640 -1 osd.4 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.860718+0000 front 2024-09-20T22:23:50.860792+0000 (oldest deadline 2024-09-20T22:24:15.560758+0000) 2024-09-20T22:24:22.899 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:22.523+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:23.191 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:22 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:22.896+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.370657+0000 front 2024-09-20T22:23:50.370787+0000 (oldest deadline 2024-09-20T22:24:15.070546+0000) 2024-09-20T22:24:23.510 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:23 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:23.188+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:23.510 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:23 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:23.188+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:23.763 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:23 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:23.508+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:23.763 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:23 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[154790]: 2024-09-20T22:24:23.659+0000 7fd9458ad640 -1 osd.4 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.860718+0000 front 2024-09-20T22:23:50.860792+0000 (oldest deadline 2024-09-20T22:24:15.560758+0000) 2024-09-20T22:24:24.222 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:23 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:23.878+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.370657+0000 front 2024-09-20T22:23:50.370787+0000 (oldest deadline 2024-09-20T22:24:15.070546+0000) 2024-09-20T22:24:24.512 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:24 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:24.220+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:24.512 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:24 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:24.220+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:24.763 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:24 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4[154790]: 2024-09-20T22:24:24.627+0000 7fd9458ad640 -1 osd.4 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.860718+0000 front 2024-09-20T22:23:50.860792+0000 (oldest deadline 2024-09-20T22:24:15.560758+0000) 2024-09-20T22:24:24.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:24 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:24.510+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:25.260 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:24 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:24.845+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.370657+0000 front 2024-09-20T22:23:50.370787+0000 (oldest deadline 2024-09-20T22:24:15.070546+0000) 2024-09-20T22:24:25.513 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:25 smithi067 podman[159277]: 2024-09-20 22:24:25.320182529 +0000 UTC m=+5.556136387 container died 2781cb10f4db06d3503e21d969e095bef71d3cf615cf8a98d6106692eefa1212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-9e2ac70, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True) 2024-09-20T22:24:25.513 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:25 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:25.257+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:25.514 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:25 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:25.257+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:25.780 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:25 smithi067 podman[159277]: 2024-09-20 22:24:25.76142035 +0000 UTC m=+5.997374195 container cleanup 2781cb10f4db06d3503e21d969e095bef71d3cf615cf8a98d6106692eefa1212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-9e2ac70, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T22:24:25.780 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:25 smithi067 bash[159277]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4 2024-09-20T22:24:25.780 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:25 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:25.513+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:25.781 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:25 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:25.513+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:26.263 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:25 smithi067 podman[159299]: 2024-09-20 22:24:25.996410338 +0000 UTC m=+0.668624523 container remove 2781cb10f4db06d3503e21d969e095bef71d3cf615cf8a98d6106692eefa1212 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-9e2ac70, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-20T22:24:26.263 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:25 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:25.883+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.370657+0000 front 2024-09-20T22:23:50.370787+0000 (oldest deadline 2024-09-20T22:24:15.070546+0000) 2024-09-20T22:24:26.264 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:25 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:25.883+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.472209+0000 front 2024-09-20T22:23:58.472231+0000 (oldest deadline 2024-09-20T22:24:24.872627+0000) 2024-09-20T22:24:26.572 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:26 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:26.274+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:26.572 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:26 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:26.274+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:26.573 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:26 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:26.506+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:26.573 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:26 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:26.506+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:26.573 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:26 smithi067 podman[159392]: 2024-09-20 22:24:26.319105917 +0000 UTC m=+0.016466237 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5 2024-09-20T22:24:26.573 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:26 smithi067 podman[159392]: 2024-09-20 22:24:26.430605664 +0000 UTC m=+0.127966018 container create b61b63e13b68b57be950a6f36e681fd9af6cfec6b2ffeabc50b5be8ac3b01543 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-9e2ac70) 2024-09-20T22:24:26.870 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:26 smithi067 podman[159392]: 2024-09-20 22:24:26.682933451 +0000 UTC m=+0.380293763 container init b61b63e13b68b57be950a6f36e681fd9af6cfec6b2ffeabc50b5be8ac3b01543 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-9e2ac70, GIT_BRANCH=HEAD) 2024-09-20T22:24:26.870 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:26 smithi067 podman[159392]: 2024-09-20 22:24:26.688212872 +0000 UTC m=+0.385573182 container start b61b63e13b68b57be950a6f36e681fd9af6cfec6b2ffeabc50b5be8ac3b01543 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=squid-9e2ac70) 2024-09-20T22:24:26.870 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:26 smithi067 podman[159392]: 2024-09-20 22:24:26.774541267 +0000 UTC m=+0.471901583 container attach b61b63e13b68b57be950a6f36e681fd9af6cfec6b2ffeabc50b5be8ac3b01543 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-9e2ac70, io.buildah.version=1.37.2) 2024-09-20T22:24:27.237 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:26 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:26.867+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.370657+0000 front 2024-09-20T22:23:50.370787+0000 (oldest deadline 2024-09-20T22:24:15.070546+0000) 2024-09-20T22:24:27.238 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:26 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:26.867+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.472209+0000 front 2024-09-20T22:23:58.472231+0000 (oldest deadline 2024-09-20T22:24:24.872627+0000) 2024-09-20T22:24:27.238 INFO:journalctl@ceph.osd.4.smithi067.stdout:Sep 20 22:24:26 smithi067 podman[159392]: 2024-09-20 22:24:26.939544882 +0000 UTC m=+0.636905194 container died b61b63e13b68b57be950a6f36e681fd9af6cfec6b2ffeabc50b5be8ac3b01543 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-9e2ac70, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-20T22:24:27.506 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:27 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:27.277+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:27.506 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:27 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:27.277+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:27.706 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.4.service' 2024-09-20T22:24:27.748 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T22:24:27.748 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-20T22:24:27.748 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-20T22:24:27.748 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.5 2024-09-20T22:24:27.763 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:27 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:27.503+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:27.763 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:27 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:27.503+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:28.052 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:27 smithi067 systemd[1]: Stopping Ceph osd.5 for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:24:28.052 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:27 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:27.907+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.370657+0000 front 2024-09-20T22:23:50.370787+0000 (oldest deadline 2024-09-20T22:24:15.070546+0000) 2024-09-20T22:24:28.052 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:27 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:27.907+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.472209+0000 front 2024-09-20T22:23:58.472231+0000 (oldest deadline 2024-09-20T22:24:24.872627+0000) 2024-09-20T22:24:28.052 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:28 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:28.049+0000 7f8f6ccea640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T22:24:28.052 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:28 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:28.049+0000 7f8f6ccea640 -1 osd.5 596 *** Got signal Terminated *** 2024-09-20T22:24:28.052 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:28 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:28.049+0000 7f8f6ccea640 -1 osd.5 596 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T22:24:28.513 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:28 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:28.294+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:28.513 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:28 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:28.294+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:28.934 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:28 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:28.533+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:28.934 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:28 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:28.533+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:29.263 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:28 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:28.932+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.370657+0000 front 2024-09-20T22:23:50.370787+0000 (oldest deadline 2024-09-20T22:24:15.070546+0000) 2024-09-20T22:24:29.263 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:28 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:28.932+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.472209+0000 front 2024-09-20T22:23:58.472231+0000 (oldest deadline 2024-09-20T22:24:24.872627+0000) 2024-09-20T22:24:29.763 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:29 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:29.344+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:29.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:29 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:29.344+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:29.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:29 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:29.574+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:29.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:29 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:29.574+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:30.263 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:29 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:29.897+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.370657+0000 front 2024-09-20T22:23:50.370787+0000 (oldest deadline 2024-09-20T22:24:15.070546+0000) 2024-09-20T22:24:30.263 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:29 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:29.897+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.472209+0000 front 2024-09-20T22:23:58.472231+0000 (oldest deadline 2024-09-20T22:24:24.872627+0000) 2024-09-20T22:24:30.763 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:30 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:30.385+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:30.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:30 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:30.385+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:30.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:30 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:30.560+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:30.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:30 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:30.560+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:31.263 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:30 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:30.867+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.370657+0000 front 2024-09-20T22:23:50.370787+0000 (oldest deadline 2024-09-20T22:24:15.070546+0000) 2024-09-20T22:24:31.263 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:30 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:30.867+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.472209+0000 front 2024-09-20T22:23:58.472231+0000 (oldest deadline 2024-09-20T22:24:24.872627+0000) 2024-09-20T22:24:31.264 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:30 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:30.867+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.873199+0000 front 2024-09-20T22:24:04.873163+0000 (oldest deadline 2024-09-20T22:24:30.172974+0000) 2024-09-20T22:24:31.604 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:31 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:31.337+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:31.604 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:31 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:31.337+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:31.875 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:31 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:31.602+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:31.875 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:31 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:31.602+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:32.263 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:31 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:31.872+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.370657+0000 front 2024-09-20T22:23:50.370787+0000 (oldest deadline 2024-09-20T22:24:15.070546+0000) 2024-09-20T22:24:32.263 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:31 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:31.872+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.472209+0000 front 2024-09-20T22:23:58.472231+0000 (oldest deadline 2024-09-20T22:24:24.872627+0000) 2024-09-20T22:24:32.263 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:31 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:31.872+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.873199+0000 front 2024-09-20T22:24:04.873163+0000 (oldest deadline 2024-09-20T22:24:30.172974+0000) 2024-09-20T22:24:32.763 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:32.555+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:32.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:32.555+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:32.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:32.327+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:32.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:32.327+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:32.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:32.327+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:05.904536+0000 front 2024-09-20T22:24:05.904434+0000 (oldest deadline 2024-09-20T22:24:31.804418+0000) 2024-09-20T22:24:33.263 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:32.856+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:50.370657+0000 front 2024-09-20T22:23:50.370787+0000 (oldest deadline 2024-09-20T22:24:15.070546+0000) 2024-09-20T22:24:33.264 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:32.856+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.472209+0000 front 2024-09-20T22:23:58.472231+0000 (oldest deadline 2024-09-20T22:24:24.872627+0000) 2024-09-20T22:24:33.264 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:32 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5[45050]: 2024-09-20T22:24:32.856+0000 7f8f69300640 -1 osd.5 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.873199+0000 front 2024-09-20T22:24:04.873163+0000 (oldest deadline 2024-09-20T22:24:30.172974+0000) 2024-09-20T22:24:33.631 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:33 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:33.306+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:33.631 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:33 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:33.306+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:33.631 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:33 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:33.306+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:05.904536+0000 front 2024-09-20T22:24:05.904434+0000 (oldest deadline 2024-09-20T22:24:31.804418+0000) 2024-09-20T22:24:33.632 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:33 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:33.519+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:33.632 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:33 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:33.519+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:33.632 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:33 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:33.519+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.662758+0000 front 2024-09-20T22:24:04.662697+0000 (oldest deadline 2024-09-20T22:24:33.463114+0000) 2024-09-20T22:24:33.932 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:33 smithi067 podman[159521]: 2024-09-20 22:24:33.630492595 +0000 UTC m=+5.684844772 container died 12fb61072ea948c0de09ae458beb8fb690c0e2ff6983258f7b59353bc98b39fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-9e2ac70) 2024-09-20T22:24:34.263 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:34 smithi067 podman[159521]: 2024-09-20 22:24:34.08183312 +0000 UTC m=+6.136185287 container cleanup 12fb61072ea948c0de09ae458beb8fb690c0e2ff6983258f7b59353bc98b39fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-9e2ac70, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-20T22:24:34.263 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:34 smithi067 bash[159521]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5 2024-09-20T22:24:34.731 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:34 smithi067 podman[159542]: 2024-09-20 22:24:34.375188192 +0000 UTC m=+0.737686179 container remove 12fb61072ea948c0de09ae458beb8fb690c0e2ff6983258f7b59353bc98b39fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5, RELEASE=squid-9e2ac70, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-20T22:24:34.731 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:34 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:34.318+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:34.732 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:34 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:34.318+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:34.732 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:34 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:34.318+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:05.904536+0000 front 2024-09-20T22:24:05.904434+0000 (oldest deadline 2024-09-20T22:24:31.804418+0000) 2024-09-20T22:24:34.732 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:34 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:34.502+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:34.732 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:34 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:34.502+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:34.732 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:34 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:34.502+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.662758+0000 front 2024-09-20T22:24:04.662697+0000 (oldest deadline 2024-09-20T22:24:33.463114+0000) 2024-09-20T22:24:34.985 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:34 smithi067 podman[159635]: 2024-09-20 22:24:34.633414112 +0000 UTC m=+0.016366213 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5 2024-09-20T22:24:34.985 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:34 smithi067 podman[159635]: 2024-09-20 22:24:34.750901677 +0000 UTC m=+0.133853775 container create 06e410ac87c62b0a76495eab7c4b24b0fe200dbcc77a82de94f40fd18015aa2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-9e2ac70, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=) 2024-09-20T22:24:34.985 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:34 smithi067 podman[159635]: 2024-09-20 22:24:34.978640234 +0000 UTC m=+0.361592342 container init 06e410ac87c62b0a76495eab7c4b24b0fe200dbcc77a82de94f40fd18015aa2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-9e2ac70, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-20T22:24:34.985 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:34 smithi067 podman[159635]: 2024-09-20 22:24:34.983883518 +0000 UTC m=+0.366835626 container start 06e410ac87c62b0a76495eab7c4b24b0fe200dbcc77a82de94f40fd18015aa2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-9e2ac70, maintainer=Guillaume Abrioux ) 2024-09-20T22:24:35.264 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:35 smithi067 podman[159635]: 2024-09-20 22:24:35.095109669 +0000 UTC m=+0.478061769 container attach 06e410ac87c62b0a76495eab7c4b24b0fe200dbcc77a82de94f40fd18015aa2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, RELEASE=squid-9e2ac70) 2024-09-20T22:24:35.264 INFO:journalctl@ceph.osd.5.smithi067.stdout:Sep 20 22:24:35 smithi067 podman[159635]: 2024-09-20 22:24:35.179987757 +0000 UTC m=+0.562939864 container died 06e410ac87c62b0a76495eab7c4b24b0fe200dbcc77a82de94f40fd18015aa2b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-9e2ac70, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-20T22:24:35.558 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:35 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:35.510+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:35.559 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:35 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:35.510+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:35.559 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:35 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:35.510+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.662758+0000 front 2024-09-20T22:24:04.662697+0000 (oldest deadline 2024-09-20T22:24:33.463114+0000) 2024-09-20T22:24:35.559 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:35 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:35.293+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:35.559 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:35 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:35.293+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:35.559 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:35 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:35.293+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:05.904536+0000 front 2024-09-20T22:24:05.904434+0000 (oldest deadline 2024-09-20T22:24:31.804418+0000) 2024-09-20T22:24:35.876 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.5.service' 2024-09-20T22:24:35.920 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T22:24:35.920 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-20T22:24:35.920 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-20T22:24:35.920 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.6 2024-09-20T22:24:36.263 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:36 smithi067 systemd[1]: Stopping Ceph osd.6 for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:24:36.763 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:36 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:36.507+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:36.763 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:36 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:36.507+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:36.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:36 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:36.507+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.662758+0000 front 2024-09-20T22:24:04.662697+0000 (oldest deadline 2024-09-20T22:24:33.463114+0000) 2024-09-20T22:24:36.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:36 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:36.261+0000 7fa9c8717640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T22:24:36.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:36 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:36.261+0000 7fa9c8717640 -1 osd.6 596 *** Got signal Terminated *** 2024-09-20T22:24:36.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:36 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:36.261+0000 7fa9c8717640 -1 osd.6 596 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T22:24:36.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:36 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:36.306+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:36.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:36 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:36.306+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:36.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:36 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:36.306+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:05.904536+0000 front 2024-09-20T22:24:05.904434+0000 (oldest deadline 2024-09-20T22:24:31.804418+0000) 2024-09-20T22:24:37.763 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:37 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:37.271+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:37.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:37 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:37.271+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:37.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:37 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:37.271+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:05.904536+0000 front 2024-09-20T22:24:05.904434+0000 (oldest deadline 2024-09-20T22:24:31.804418+0000) 2024-09-20T22:24:37.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:37 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:37.469+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:37.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:37 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:37.469+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:37.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:37 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:37.469+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.662758+0000 front 2024-09-20T22:24:04.662697+0000 (oldest deadline 2024-09-20T22:24:33.463114+0000) 2024-09-20T22:24:37.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:37 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:37.469+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-20T22:24:14.564028+0000 front 2024-09-20T22:24:14.564005+0000 (oldest deadline 2024-09-20T22:24:37.364517+0000) 2024-09-20T22:24:38.763 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:38 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:38.304+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:38.763 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:38 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:38.304+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:38.763 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:38 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:38.304+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:05.904536+0000 front 2024-09-20T22:24:05.904434+0000 (oldest deadline 2024-09-20T22:24:31.804418+0000) 2024-09-20T22:24:38.763 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:38 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:38.304+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-20T22:24:15.305306+0000 front 2024-09-20T22:24:15.305279+0000 (oldest deadline 2024-09-20T22:24:37.605192+0000) 2024-09-20T22:24:38.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:38 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:38.467+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:38.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:38 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:38.467+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:38.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:38 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:38.467+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.662758+0000 front 2024-09-20T22:24:04.662697+0000 (oldest deadline 2024-09-20T22:24:33.463114+0000) 2024-09-20T22:24:38.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:38 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:38.467+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-20T22:24:14.564028+0000 front 2024-09-20T22:24:14.564005+0000 (oldest deadline 2024-09-20T22:24:37.364517+0000) 2024-09-20T22:24:39.763 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:39 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:39.275+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:39.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:39 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:39.275+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:39.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:39 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:39.275+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:05.904536+0000 front 2024-09-20T22:24:05.904434+0000 (oldest deadline 2024-09-20T22:24:31.804418+0000) 2024-09-20T22:24:39.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:39 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:39.275+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-20T22:24:15.305306+0000 front 2024-09-20T22:24:15.305279+0000 (oldest deadline 2024-09-20T22:24:37.605192+0000) 2024-09-20T22:24:39.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:39 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:39.496+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:39.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:39 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:39.496+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:39.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:39 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:39.496+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.662758+0000 front 2024-09-20T22:24:04.662697+0000 (oldest deadline 2024-09-20T22:24:33.463114+0000) 2024-09-20T22:24:39.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:39 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:39.496+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-20T22:24:14.564028+0000 front 2024-09-20T22:24:14.564005+0000 (oldest deadline 2024-09-20T22:24:37.364517+0000) 2024-09-20T22:24:40.763 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:40 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:40.277+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:40.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:40 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:40.277+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:40.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:40 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:40.277+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:05.904536+0000 front 2024-09-20T22:24:05.904434+0000 (oldest deadline 2024-09-20T22:24:31.804418+0000) 2024-09-20T22:24:40.764 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:40 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:40.277+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-20T22:24:15.305306+0000 front 2024-09-20T22:24:15.305279+0000 (oldest deadline 2024-09-20T22:24:37.605192+0000) 2024-09-20T22:24:40.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:40 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:40.523+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:40.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:40 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:40.523+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:40.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:40 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:40.523+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.662758+0000 front 2024-09-20T22:24:04.662697+0000 (oldest deadline 2024-09-20T22:24:33.463114+0000) 2024-09-20T22:24:40.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:40 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:40.523+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-20T22:24:14.564028+0000 front 2024-09-20T22:24:14.564005+0000 (oldest deadline 2024-09-20T22:24:37.364517+0000) 2024-09-20T22:24:41.513 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:41 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:41.242+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.302619+0000 front 2024-09-20T22:23:51.302659+0000 (oldest deadline 2024-09-20T22:24:14.802574+0000) 2024-09-20T22:24:41.513 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:41 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:41.242+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:58.903334+0000 front 2024-09-20T22:23:58.903467+0000 (oldest deadline 2024-09-20T22:24:22.403355+0000) 2024-09-20T22:24:41.514 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:41 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:41.242+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:05.904536+0000 front 2024-09-20T22:24:05.904434+0000 (oldest deadline 2024-09-20T22:24:31.804418+0000) 2024-09-20T22:24:41.514 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:41 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6[50899]: 2024-09-20T22:24:41.242+0000 7fa9c0d25640 -1 osd.6 596 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-20T22:24:15.305306+0000 front 2024-09-20T22:24:15.305279+0000 (oldest deadline 2024-09-20T22:24:37.605192+0000) 2024-09-20T22:24:42.013 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:41 smithi067 podman[159762]: 2024-09-20 22:24:41.770957049 +0000 UTC m=+5.621439702 container died 73ea26d3c1216b5af93848522c0517e869cdc7c4dc8b692d2e5f4e1fd73e3aa5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-9e2ac70, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-20T22:24:42.014 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:41 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:41.525+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:42.014 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:41 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:41.525+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:42.014 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:41 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:41.525+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.662758+0000 front 2024-09-20T22:24:04.662697+0000 (oldest deadline 2024-09-20T22:24:33.463114+0000) 2024-09-20T22:24:42.014 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:41 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:41.525+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-20T22:24:14.564028+0000 front 2024-09-20T22:24:14.564005+0000 (oldest deadline 2024-09-20T22:24:37.364517+0000) 2024-09-20T22:24:42.437 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:42 smithi067 podman[159762]: 2024-09-20 22:24:42.160009301 +0000 UTC m=+6.010491938 container cleanup 73ea26d3c1216b5af93848522c0517e869cdc7c4dc8b692d2e5f4e1fd73e3aa5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6, RELEASE=squid-9e2ac70, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-20T22:24:42.437 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:42 smithi067 bash[159762]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6 2024-09-20T22:24:42.753 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:42 smithi067 podman[159783]: 2024-09-20 22:24:42.436256103 +0000 UTC m=+0.662776232 container remove 73ea26d3c1216b5af93848522c0517e869cdc7c4dc8b692d2e5f4e1fd73e3aa5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6, RELEASE=squid-9e2ac70, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-20T22:24:42.753 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:42.482+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:42.753 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:42.482+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:42.753 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:42.482+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.662758+0000 front 2024-09-20T22:24:04.662697+0000 (oldest deadline 2024-09-20T22:24:33.463114+0000) 2024-09-20T22:24:42.753 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:42 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:42.482+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-20T22:24:14.564028+0000 front 2024-09-20T22:24:14.564005+0000 (oldest deadline 2024-09-20T22:24:37.364517+0000) 2024-09-20T22:24:43.004 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:42 smithi067 podman[159876]: 2024-09-20 22:24:42.654774162 +0000 UTC m=+0.016506925 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5 2024-09-20T22:24:43.004 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:42 smithi067 podman[159876]: 2024-09-20 22:24:42.77487124 +0000 UTC m=+0.136604036 container create 32ce0d6344e9e86cbc29166cae1b9017fce72772fde5d897074edb07a31afaf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-9e2ac70, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True) 2024-09-20T22:24:43.004 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:42 smithi067 podman[159876]: 2024-09-20 22:24:42.997976974 +0000 UTC m=+0.359709734 container init 32ce0d6344e9e86cbc29166cae1b9017fce72772fde5d897074edb07a31afaf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-9e2ac70, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-20T22:24:43.004 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:43 smithi067 podman[159876]: 2024-09-20 22:24:43.002946244 +0000 UTC m=+0.364678998 container start 32ce0d6344e9e86cbc29166cae1b9017fce72772fde5d897074edb07a31afaf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-9e2ac70, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-20T22:24:43.264 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:43 smithi067 podman[159876]: 2024-09-20 22:24:43.106096261 +0000 UTC m=+0.467829026 container attach 32ce0d6344e9e86cbc29166cae1b9017fce72772fde5d897074edb07a31afaf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-9e2ac70, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-20T22:24:43.396 INFO:journalctl@ceph.osd.6.smithi067.stdout:Sep 20 22:24:43 smithi067 podman[159876]: 2024-09-20 22:24:43.200164427 +0000 UTC m=+0.561897180 container died 32ce0d6344e9e86cbc29166cae1b9017fce72772fde5d897074edb07a31afaf7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-9e2ac70) 2024-09-20T22:24:43.763 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:43 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:43.512+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:43.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:43 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:43.512+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:43.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:43 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:43.512+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.662758+0000 front 2024-09-20T22:24:04.662697+0000 (oldest deadline 2024-09-20T22:24:33.463114+0000) 2024-09-20T22:24:43.764 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:43 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:43.512+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-20T22:24:14.564028+0000 front 2024-09-20T22:24:14.564005+0000 (oldest deadline 2024-09-20T22:24:37.364517+0000) 2024-09-20T22:24:43.820 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.6.service' 2024-09-20T22:24:43.862 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T22:24:43.862 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-20T22:24:43.862 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-20T22:24:43.862 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.7 2024-09-20T22:24:44.199 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:43 smithi067 systemd[1]: Stopping Ceph osd.7 for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:24:44.200 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:44 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:44.198+0000 7fc6f4c00640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T22:24:44.200 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:44 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:44.198+0000 7fc6f4c00640 -1 osd.7 596 *** Got signal Terminated *** 2024-09-20T22:24:44.200 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:44 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:44.198+0000 7fc6f4c00640 -1 osd.7 596 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-20T22:24:44.763 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:44 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:44.525+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:44.763 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:44 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:44.525+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:44.763 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:44 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:44.525+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.662758+0000 front 2024-09-20T22:24:04.662697+0000 (oldest deadline 2024-09-20T22:24:33.463114+0000) 2024-09-20T22:24:44.763 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:44 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:44.525+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-20T22:24:14.564028+0000 front 2024-09-20T22:24:14.564005+0000 (oldest deadline 2024-09-20T22:24:37.364517+0000) 2024-09-20T22:24:46.013 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:45 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:45.545+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:46.013 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:45 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:45.545+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:46.013 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:45 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:45.545+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.662758+0000 front 2024-09-20T22:24:04.662697+0000 (oldest deadline 2024-09-20T22:24:33.463114+0000) 2024-09-20T22:24:46.013 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:45 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:45.545+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-20T22:24:14.564028+0000 front 2024-09-20T22:24:14.564005+0000 (oldest deadline 2024-09-20T22:24:37.364517+0000) 2024-09-20T22:24:47.013 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:46 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:46.520+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:47.014 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:46 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:46.520+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:47.014 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:46 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:46.520+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.662758+0000 front 2024-09-20T22:24:04.662697+0000 (oldest deadline 2024-09-20T22:24:33.463114+0000) 2024-09-20T22:24:47.014 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:46 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:46.520+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-20T22:24:14.564028+0000 front 2024-09-20T22:24:14.564005+0000 (oldest deadline 2024-09-20T22:24:37.364517+0000) 2024-09-20T22:24:47.014 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:46 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:46.520+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.67:6804 osd.4 since back 2024-09-20T22:24:20.265200+0000 front 2024-09-20T22:24:20.265151+0000 (oldest deadline 2024-09-20T22:24:45.565178+0000) 2024-09-20T22:24:48.014 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:47 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:47.520+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:48.014 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:47 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:47.520+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:48.014 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:47 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:47.520+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.662758+0000 front 2024-09-20T22:24:04.662697+0000 (oldest deadline 2024-09-20T22:24:33.463114+0000) 2024-09-20T22:24:48.014 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:47 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:47.520+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-20T22:24:14.564028+0000 front 2024-09-20T22:24:14.564005+0000 (oldest deadline 2024-09-20T22:24:37.364517+0000) 2024-09-20T22:24:48.014 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:47 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:47.520+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.67:6804 osd.4 since back 2024-09-20T22:24:20.265200+0000 front 2024-09-20T22:24:20.265151+0000 (oldest deadline 2024-09-20T22:24:45.565178+0000) 2024-09-20T22:24:49.013 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:48 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:48.538+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6806 osd.0 since back 2024-09-20T22:23:51.761596+0000 front 2024-09-20T22:23:51.761358+0000 (oldest deadline 2024-09-20T22:24:15.861280+0000) 2024-09-20T22:24:49.013 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:48 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:48.538+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6814 osd.1 since back 2024-09-20T22:23:59.962281+0000 front 2024-09-20T22:23:59.962355+0000 (oldest deadline 2024-09-20T22:24:24.662155+0000) 2024-09-20T22:24:49.013 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:48 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:48.538+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6822 osd.2 since back 2024-09-20T22:24:04.662758+0000 front 2024-09-20T22:24:04.662697+0000 (oldest deadline 2024-09-20T22:24:33.463114+0000) 2024-09-20T22:24:49.013 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:48 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:48.538+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.60:6830 osd.3 since back 2024-09-20T22:24:14.564028+0000 front 2024-09-20T22:24:14.564005+0000 (oldest deadline 2024-09-20T22:24:37.364517+0000) 2024-09-20T22:24:49.014 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:48 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7[57262]: 2024-09-20T22:24:48.538+0000 7fc6ed20e640 -1 osd.7 596 heartbeat_check: no reply from 172.21.15.67:6804 osd.4 since back 2024-09-20T22:24:20.265200+0000 front 2024-09-20T22:24:20.265151+0000 (oldest deadline 2024-09-20T22:24:45.565178+0000) 2024-09-20T22:24:50.114 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:49 smithi067 podman[160012]: 2024-09-20 22:24:49.844205038 +0000 UTC m=+5.756469321 container died 1937a1e0b87cb406cc9af88318472e4a8a4e4fe01fd8d6b15b4310894429e727 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-9e2ac70, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-20T22:24:50.372 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:50 smithi067 podman[160012]: 2024-09-20 22:24:50.187303182 +0000 UTC m=+6.099567464 container cleanup 1937a1e0b87cb406cc9af88318472e4a8a4e4fe01fd8d6b15b4310894429e727 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=squid-9e2ac70, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-20T22:24:50.372 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:50 smithi067 bash[160012]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7 2024-09-20T22:24:50.763 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:50 smithi067 podman[160033]: 2024-09-20 22:24:50.371673822 +0000 UTC m=+0.521447184 container remove 1937a1e0b87cb406cc9af88318472e4a8a4e4fe01fd8d6b15b4310894429e727 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-9e2ac70, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-20T22:24:51.134 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:50 smithi067 podman[160129]: 2024-09-20 22:24:50.700861938 +0000 UTC m=+0.016214211 image pull cc9251b744a3faafca7c3095f79bfff81873afe823f31a03a11e015a34abcdbd quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5 2024-09-20T22:24:51.135 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:50 smithi067 podman[160129]: 2024-09-20 22:24:50.805897042 +0000 UTC m=+0.121249347 container create 1d562b6ebb060bfcdb2c644d806e7fde1fdec717213405fbbc1f2856571b2e7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-9e2ac70, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, ceph=True) 2024-09-20T22:24:51.135 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:51 smithi067 podman[160129]: 2024-09-20 22:24:51.042051182 +0000 UTC m=+0.357403451 container init 1d562b6ebb060bfcdb2c644d806e7fde1fdec717213405fbbc1f2856571b2e7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-9e2ac70, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-20T22:24:51.135 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:51 smithi067 podman[160129]: 2024-09-20 22:24:51.047429475 +0000 UTC m=+0.362781741 container start 1d562b6ebb060bfcdb2c644d806e7fde1fdec717213405fbbc1f2856571b2e7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-9e2ac70, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-20T22:24:51.135 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:51 smithi067 podman[160129]: 2024-09-20 22:24:51.133377823 +0000 UTC m=+0.448730097 container attach 1d562b6ebb060bfcdb2c644d806e7fde1fdec717213405fbbc1f2856571b2e7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, RELEASE=squid-9e2ac70, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True) 2024-09-20T22:24:51.513 INFO:journalctl@ceph.osd.7.smithi067.stdout:Sep 20 22:24:51 smithi067 podman[160129]: 2024-09-20 22:24:51.248028082 +0000 UTC m=+0.563380345 container died 1d562b6ebb060bfcdb2c644d806e7fde1fdec717213405fbbc1f2856571b2e7a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=squid-9e2ac70, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-09-20T22:24:51.834 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@osd.7.service' 2024-09-20T22:24:51.875 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T22:24:51.875 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-20T22:24:51.875 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-20T22:24:51.876 DEBUG:teuthology.orchestra.run.smithi060:> sudo systemctl stop ceph-3df9647e-779b-11ef-baf5-efdc52797490@rgw.foo.a 2024-09-20T22:24:52.187 INFO:journalctl@ceph.rgw.foo.a.smithi060.stdout:Sep 20 22:24:51 smithi060 systemd[1]: Stopping Ceph rgw.foo.a for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:24:52.457 INFO:journalctl@ceph.rgw.foo.a.smithi060.stdout:Sep 20 22:24:52 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-rgw-foo-a[67948]: 2024-09-20T22:24:52.184+0000 7f0e6b35c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-20T22:24:52.457 INFO:journalctl@ceph.rgw.foo.a.smithi060.stdout:Sep 20 22:24:52 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-rgw-foo-a[67948]: 2024-09-20T22:24:52.184+0000 7f0e6ebc8880 -1 shutting down 2024-09-20T22:25:02.536 INFO:journalctl@ceph.rgw.foo.a.smithi060.stdout:Sep 20 22:25:02 smithi060 bash[227398]: time="2024-09-20T22:25:02Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-3df9647e-779b-11ef-baf5-efdc52797490-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-20T22:25:02.536 INFO:journalctl@ceph.rgw.foo.a.smithi060.stdout:Sep 20 22:25:02 smithi060 podman[227398]: 2024-09-20 22:25:02.225496684 +0000 UTC m=+10.142861956 container died e3bab278a660b7ae9c8cd3bf24dc50009dc9e130df1830a8de9b4481c6a47676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-rgw-foo-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, RELEASE=squid-9e2ac70, GIT_CLEAN=True) 2024-09-20T22:25:02.948 INFO:journalctl@ceph.rgw.foo.a.smithi060.stdout:Sep 20 22:25:02 smithi060 podman[227398]: 2024-09-20 22:25:02.682710679 +0000 UTC m=+10.600075936 container cleanup e3bab278a660b7ae9c8cd3bf24dc50009dc9e130df1830a8de9b4481c6a47676 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-rgw-foo-a, org.label-schema.build-date=20240731, RELEASE=squid-9e2ac70, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-20T22:25:02.948 INFO:journalctl@ceph.rgw.foo.a.smithi060.stdout:Sep 20 22:25:02 smithi060 bash[227398]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-rgw-foo-a 2024-09-20T22:25:03.111 DEBUG:teuthology.orchestra.run.smithi060:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@rgw.foo.a.service' 2024-09-20T22:25:03.149 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T22:25:03.149 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-20T22:25:03.149 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-20T22:25:03.149 DEBUG:teuthology.orchestra.run.smithi067:> sudo systemctl stop ceph-3df9647e-779b-11ef-baf5-efdc52797490@prometheus.a 2024-09-20T22:25:03.450 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:25:03 smithi067 systemd[1]: Stopping Ceph prometheus.a for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:25:03.732 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:25:03 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:25:03.449Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-20T22:25:03.733 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:25:03 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:25:03.449Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-20T22:25:03.733 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:25:03 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:25:03.449Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-20T22:25:03.733 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:25:03 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:25:03.449Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-20T22:25:03.733 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:25:03 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:25:03.449Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-20T22:25:03.733 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:25:03 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:25:03.449Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-20T22:25:03.733 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:25:03 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:25:03.449Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-20T22:25:03.733 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:25:03 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:25:03.449Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-20T22:25:03.733 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:25:03 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:25:03.459Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-20T22:25:03.733 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:25:03 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:25:03.459Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-20T22:25:03.733 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:25:03 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:25:03.459Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-20T22:25:03.733 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:25:03 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a[73774]: ts=2024-09-20T22:25:03.459Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-20T22:25:03.733 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:25:03 smithi067 podman[160256]: 2024-09-20 22:25:03.470009762 +0000 UTC m=+0.123436409 container died 8e494b97ac052a6072fc03dfde220506c002c2d00da0544b7ac8aec3ceb27739 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T22:25:04.013 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:25:03 smithi067 podman[160256]: 2024-09-20 22:25:03.93056846 +0000 UTC m=+0.583995145 container cleanup 8e494b97ac052a6072fc03dfde220506c002c2d00da0544b7ac8aec3ceb27739 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-20T22:25:04.013 INFO:journalctl@ceph.prometheus.a.smithi067.stdout:Sep 20 22:25:03 smithi067 bash[160256]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-prometheus-a 2024-09-20T22:25:04.410 DEBUG:teuthology.orchestra.run.smithi067:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3df9647e-779b-11ef-baf5-efdc52797490@prometheus.a.service' 2024-09-20T22:25:04.445 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-20T22:25:04.445 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-20T22:25:04.445 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3df9647e-779b-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-20T22:25:04.657 INFO:teuthology.orchestra.run.smithi060.stdout:Deleting cluster with fsid: 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T22:25:06.625 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:25:06 smithi060 systemd[1]: Stopping Ceph alertmanager.a for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:25:06.940 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:25:06 smithi060 ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a[87854]: ts=2024-09-20T22:25:06.623Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-20T22:25:06.941 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:25:06 smithi060 podman[227914]: 2024-09-20 22:25:06.664672033 +0000 UTC m=+0.149333281 container died f4474f3f5bac57806d0cea5d47af51fdc93a4ff3202a3ce4ec1225ab09a02a7b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T22:25:07.286 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:25:07 smithi060 podman[227914]: 2024-09-20 22:25:07.130172786 +0000 UTC m=+0.614834033 container cleanup f4474f3f5bac57806d0cea5d47af51fdc93a4ff3202a3ce4ec1225ab09a02a7b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T22:25:07.286 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:25:07 smithi060 bash[227914]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a 2024-09-20T22:25:07.786 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:25:07 smithi060 podman[227933]: 2024-09-20 22:25:07.389953467 +0000 UTC m=+0.721602494 container remove f4474f3f5bac57806d0cea5d47af51fdc93a4ff3202a3ce4ec1225ab09a02a7b (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-20T22:25:07.786 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:25:07 smithi060 podman[227933]: 2024-09-20 22:25:07.474766412 +0000 UTC m=+0.806415427 volume remove b4f84d023d10072378df1d05ab06b8345e94b0707f83df6bcdec453eb47b66e0 2024-09-20T22:25:07.786 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:25:07 smithi060 systemd[1]: ceph-3df9647e-779b-11ef-baf5-efdc52797490@alertmanager.a.service: Deactivated successfully. 2024-09-20T22:25:07.786 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:25:07 smithi060 systemd[1]: Stopped Ceph alertmanager.a for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T22:25:07.786 INFO:journalctl@ceph.alertmanager.a.smithi060.stdout:Sep 20 22:25:07 smithi060 systemd[1]: ceph-3df9647e-779b-11ef-baf5-efdc52797490@alertmanager.a.service: Consumed 1.839s CPU time. 2024-09-20T22:25:08.082 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:25:07 smithi060 systemd[1]: Stopping Ceph node-exporter.a for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:25:08.345 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:25:08 smithi060 podman[228052]: 2024-09-20 22:25:08.081827887 +0000 UTC m=+0.111420930 container died f9f089f39405c767f6aa092f807245d65fb1882128788d470b8bbc25c6ca7583 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-20T22:25:08.712 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:25:08 smithi060 podman[228052]: 2024-09-20 22:25:08.434955541 +0000 UTC m=+0.464548571 container cleanup f9f089f39405c767f6aa092f807245d65fb1882128788d470b8bbc25c6ca7583 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-20T22:25:08.712 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:25:08 smithi060 bash[228052]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a 2024-09-20T22:25:09.013 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:25:08 smithi060 podman[228070]: 2024-09-20 22:25:08.711310162 +0000 UTC m=+0.629546403 container remove f9f089f39405c767f6aa092f807245d65fb1882128788d470b8bbc25c6ca7583 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-20T22:25:09.014 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:25:08 smithi060 systemd[1]: ceph-3df9647e-779b-11ef-baf5-efdc52797490@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-20T22:25:09.014 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:25:08 smithi060 systemd[1]: ceph-3df9647e-779b-11ef-baf5-efdc52797490@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-20T22:25:09.014 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:25:08 smithi060 systemd[1]: Stopped Ceph node-exporter.a for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T22:25:09.014 INFO:journalctl@ceph.node-exporter.a.smithi060.stdout:Sep 20 22:25:08 smithi060 systemd[1]: ceph-3df9647e-779b-11ef-baf5-efdc52797490@node-exporter.a.service: Consumed 5.144s CPU time. 2024-09-20T22:25:09.546 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3df9647e-779b-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-20T22:25:09.757 INFO:teuthology.orchestra.run.smithi067.stdout:Deleting cluster with fsid: 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T22:25:11.322 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:25:11 smithi067 systemd[1]: Stopping Ceph iscsi.iscsi.a for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:25:11.763 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:25:11 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a[62370]: debug Shutdown received 2024-09-20T22:25:21.670 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:25:21 smithi067 bash[160670]: time="2024-09-20T22:25:21Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-20T22:25:21.670 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:25:21 smithi067 podman[160670]: 2024-09-20 22:25:21.342938352 +0000 UTC m=+10.129335332 container died 0a4eea18e1996852a18043cf3e4e52dd3a507abd1b940adbf85fbf5af5f768cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, RELEASE=squid-9e2ac70, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-20T22:25:22.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:25:21 smithi067 podman[160670]: 2024-09-20 22:25:21.867435188 +0000 UTC m=+10.653832153 container cleanup 0a4eea18e1996852a18043cf3e4e52dd3a507abd1b940adbf85fbf5af5f768cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-9e2ac70, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2) 2024-09-20T22:25:22.013 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:25:21 smithi067 bash[160670]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a 2024-09-20T22:25:22.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:25:22 smithi067 podman[160692]: 2024-09-20 22:25:22.060532455 +0000 UTC m=+0.711980262 container remove 0a4eea18e1996852a18043cf3e4e52dd3a507abd1b940adbf85fbf5af5f768cc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:07c2c7a2cb9ca9b41c609b2ddfb8539abdc0283fbb4aebf87c70eeaf670bffe5, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-iscsi-iscsi-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=squid-9e2ac70, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-20T22:25:22.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:25:22 smithi067 systemd[1]: ceph-3df9647e-779b-11ef-baf5-efdc52797490@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-20T22:25:22.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:25:22 smithi067 systemd[1]: ceph-3df9647e-779b-11ef-baf5-efdc52797490@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-20T22:25:22.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:25:22 smithi067 systemd[1]: Stopped Ceph iscsi.iscsi.a for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T22:25:22.323 INFO:journalctl@ceph.iscsi.iscsi.a.smithi067.stdout:Sep 20 22:25:22 smithi067 systemd[1]: ceph-3df9647e-779b-11ef-baf5-efdc52797490@iscsi.iscsi.a.service: Consumed 2.798s CPU time. 2024-09-20T22:25:23.925 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:25:23 smithi067 systemd[1]: Stopping Ceph grafana.a for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:25:24.263 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:25:24 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=server t=2024-09-20T22:25:24.135881734Z level=info msg="Shutdown started" reason="System signal: terminated" 2024-09-20T22:25:24.263 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:25:24 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=ticker t=2024-09-20T22:25:24.136395681Z level=info msg=stopped last_tick=2024-09-20T22:25:20Z 2024-09-20T22:25:24.263 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:25:24 smithi067 ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a[65764]: logger=tracing t=2024-09-20T22:25:24.136434151Z level=info msg="Closing tracing" 2024-09-20T22:25:24.263 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:25:24 smithi067 podman[160982]: 2024-09-20 22:25:24.156405255 +0000 UTC m=+0.160174934 container died 04a820c2471a0b105c5efcfd677177fea3faf83a3dca118e276754adbc848501 (image=quay.io/ceph/grafana:9.4.12, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-20T22:25:24.730 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:25:24 smithi067 podman[160982]: 2024-09-20 22:25:24.499654762 +0000 UTC m=+0.503424432 container cleanup 04a820c2471a0b105c5efcfd677177fea3faf83a3dca118e276754adbc848501 (image=quay.io/ceph/grafana:9.4.12, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-20T22:25:24.730 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:25:24 smithi067 bash[160982]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a 2024-09-20T22:25:25.013 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:25:24 smithi067 podman[161000]: 2024-09-20 22:25:24.826227878 +0000 UTC m=+0.674004886 container remove 04a820c2471a0b105c5efcfd677177fea3faf83a3dca118e276754adbc848501 (image=quay.io/ceph/grafana:9.4.12, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-grafana-a, maintainer=Grafana Labs ) 2024-09-20T22:25:25.013 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:25:24 smithi067 systemd[1]: ceph-3df9647e-779b-11ef-baf5-efdc52797490@grafana.a.service: Deactivated successfully. 2024-09-20T22:25:25.013 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:25:24 smithi067 systemd[1]: Stopped Ceph grafana.a for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T22:25:25.014 INFO:journalctl@ceph.grafana.a.smithi067.stdout:Sep 20 22:25:24 smithi067 systemd[1]: ceph-3df9647e-779b-11ef-baf5-efdc52797490@grafana.a.service: Consumed 15.477s CPU time. 2024-09-20T22:25:25.418 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:25:25 smithi067 systemd[1]: Stopping Ceph node-exporter.b for 3df9647e-779b-11ef-baf5-efdc52797490... 2024-09-20T22:25:25.740 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:25:25 smithi067 podman[161121]: 2024-09-20 22:25:25.417370207 +0000 UTC m=+0.115847513 container died f3ca125f7fc6e953697d711936758d1eb27bbda33bcb15fdf6c5db591d67b348 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-20T22:25:25.740 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:25:25 smithi067 podman[161121]: 2024-09-20 22:25:25.739380728 +0000 UTC m=+0.437858038 container cleanup f3ca125f7fc6e953697d711936758d1eb27bbda33bcb15fdf6c5db591d67b348 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-20T22:25:26.013 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:25:25 smithi067 bash[161121]: ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b 2024-09-20T22:25:26.013 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:25:25 smithi067 podman[161141]: 2024-09-20 22:25:25.940544518 +0000 UTC m=+0.522019420 container remove f3ca125f7fc6e953697d711936758d1eb27bbda33bcb15fdf6c5db591d67b348 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-3df9647e-779b-11ef-baf5-efdc52797490-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-20T22:25:26.013 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:25:25 smithi067 systemd[1]: ceph-3df9647e-779b-11ef-baf5-efdc52797490@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-20T22:25:26.451 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:25:26 smithi067 systemd[1]: ceph-3df9647e-779b-11ef-baf5-efdc52797490@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-20T22:25:26.451 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:25:26 smithi067 systemd[1]: Stopped Ceph node-exporter.b for 3df9647e-779b-11ef-baf5-efdc52797490. 2024-09-20T22:25:26.451 INFO:journalctl@ceph.node-exporter.b.smithi067.stdout:Sep 20 22:25:26 smithi067 systemd[1]: ceph-3df9647e-779b-11ef-baf5-efdc52797490@node-exporter.b.service: Consumed 4.793s CPU time. 2024-09-20T22:25:26.858 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-20T22:25:26.892 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-20T22:25:26.921 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-20T22:25:26.922 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/teuthology-2024-09-20_21:08:02-orch-squid-distro-default-smithi/7912980/remote/smithi060/crash 2024-09-20T22:25:26.923 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/crash -- . 2024-09-20T22:25:26.974 INFO:teuthology.orchestra.run.smithi060.stderr:tar: /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-20T22:25:26.974 INFO:teuthology.orchestra.run.smithi060.stderr:tar: Error is not recoverable: exiting now 2024-09-20T22:25:26.976 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/teuthology-2024-09-20_21:08:02-orch-squid-distro-default-smithi/7912980/remote/smithi067/crash 2024-09-20T22:25:26.977 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/crash -- . 2024-09-20T22:25:27.013 INFO:teuthology.orchestra.run.smithi067.stderr:tar: /var/lib/ceph/3df9647e-779b-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-20T22:25:27.013 INFO:teuthology.orchestra.run.smithi067.stderr:tar: Error is not recoverable: exiting now 2024-09-20T22:25:27.014 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-20T22:25:27.014 DEBUG:teuthology.orchestra.run.smithi060:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.log | egrep CEPHADM_ | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v CEPHADM_STRAY_DAEMON | egrep -v CEPHADM_FAILED_DAEMON | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2024-09-20T22:25:27.097 INFO:tasks.cephadm:Compressing logs... 2024-09-20T22:25:27.097 DEBUG:teuthology.orchestra.run.smithi060:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-20T22:25:27.100 DEBUG:teuthology.orchestra.run.smithi067:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-20T22:25:27.136 INFO:teuthology.orchestra.run.smithi060.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-09-20T22:25:27.137 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-20T22:25:27.138 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-20T22:25:27.138 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-20T22:25:27.138 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.0.log: /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-20T22:25:27.138 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-20T22:25:27.144 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-20T22:25:27.144 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-20T22:25:27.144 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/objectstore_tool.6.log 2024-09-20T22:25:27.144 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-20T22:25:27.144 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/objectstore_tool.6.log: gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-20T22:25:27.144 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.log 2024-09-20T22:25:27.144 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-client.rgw.foo.a.log 2024-09-20T22:25:27.144 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.log: gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-20T22:25:27.145 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/objectstore_tool.7.log 2024-09-20T22:25:27.145 INFO:teuthology.orchestra.run.smithi060.stderr: 59.0% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-client.rgw.foo.a.log.gz 2024-09-20T22:25:27.145 INFO:teuthology.orchestra.run.smithi060.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-20T22:25:27.145 INFO:teuthology.orchestra.run.smithi067.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/tcmu-runner.log 2024-09-20T22:25:27.145 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-09-20T22:25:27.145 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-20T22:25:27.145 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-20T22:25:27.145 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/tcmu-runner.log: /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.7.log: 63.3% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/tcmu-runner.log.gz 2024-09-20T22:25:27.145 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-20T22:25:27.145 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-20T22:25:27.145 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-20T22:25:27.145 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.audit.log: --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/objectstore_tool.6.log 2024-09-20T22:25:27.145 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-20T22:25:27.146 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/objectstore_tool.6.log: gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-20T22:25:27.146 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.log 2024-09-20T22:25:27.146 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-20T22:25:27.146 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.5.log: gzip -5/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.log: --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-20T22:25:27.146 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/objectstore_tool.7.log 2024-09-20T22:25:27.146 INFO:teuthology.orchestra.run.smithi067.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-20T22:25:27.152 INFO:teuthology.orchestra.run.smithi060.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.3.log: /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/objectstore_tool.7.log: /var/log/ceph/cephadm.log: 91.7% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-20T22:25:27.242 INFO:teuthology.orchestra.run.smithi067.stderr:/var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-mon.b.log: /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/objectstore_tool.7.log: /var/log/ceph/cephadm.log: 79.8% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-20T22:25:27.245 INFO:teuthology.orchestra.run.smithi060.stderr: 89.2% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-20T22:25:27.281 INFO:teuthology.orchestra.run.smithi060.stderr: 94.4% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/objectstore_tool.6.log.gz 2024-09-20T22:25:27.306 INFO:teuthology.orchestra.run.smithi060.stderr: 94.0% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-20T22:25:27.316 INFO:teuthology.orchestra.run.smithi067.stderr: 85.4% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-20T22:25:27.320 INFO:teuthology.orchestra.run.smithi060.stderr: 94.6% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/objectstore_tool.7.log.gz 2024-09-20T22:25:27.332 INFO:teuthology.orchestra.run.smithi067.stderr: 92.1% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-20T22:25:27.332 INFO:teuthology.orchestra.run.smithi067.stderr: 90.0% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-20T22:25:27.354 INFO:teuthology.orchestra.run.smithi060.stderr: 95.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-20T22:25:27.360 INFO:teuthology.orchestra.run.smithi067.stderr: 94.0% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/objectstore_tool.6.log.gz 2024-09-20T22:25:27.379 INFO:teuthology.orchestra.run.smithi060.stderr: 91.0% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-20T22:25:27.506 INFO:teuthology.orchestra.run.smithi067.stderr: 94.0% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/objectstore_tool.7.log.gz 2024-09-20T22:25:27.560 INFO:teuthology.orchestra.run.smithi067.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-20T22:25:27.681 INFO:teuthology.orchestra.run.smithi067.stderr: 91.2% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-20T22:25:30.059 INFO:teuthology.orchestra.run.smithi067.stderr: 91.1% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-20T22:25:33.821 INFO:teuthology.orchestra.run.smithi060.stderr: 90.2% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-20T22:25:34.573 INFO:teuthology.orchestra.run.smithi060.stderr: 91.0% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-20T22:25:45.676 INFO:teuthology.orchestra.run.smithi060.stderr: 90.3% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-20T22:35:52.304 INFO:teuthology.orchestra.run.smithi067.stderr: 93.7% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-20T22:39:14.960 INFO:teuthology.orchestra.run.smithi060.stderr: 93.1% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-20T22:41:31.296 INFO:teuthology.orchestra.run.smithi060.stderr: 93.3% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-20T22:42:03.649 INFO:teuthology.orchestra.run.smithi067.stderr: 93.0% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-20T22:42:12.878 INFO:teuthology.orchestra.run.smithi067.stderr: 93.0% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-20T22:42:26.126 INFO:teuthology.orchestra.run.smithi067.stderr: 93.5% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-20T22:42:26.128 INFO:teuthology.orchestra.run.smithi067.stderr: 2024-09-20T22:42:26.128 INFO:teuthology.orchestra.run.smithi067.stderr:real 16m59.013s 2024-09-20T22:42:26.128 INFO:teuthology.orchestra.run.smithi067.stderr:user 6m11.466s 2024-09-20T22:42:26.128 INFO:teuthology.orchestra.run.smithi067.stderr:sys 0m18.964s 2024-09-20T22:43:05.321 INFO:teuthology.orchestra.run.smithi060.stderr: 92.7% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-20T22:43:35.599 INFO:teuthology.orchestra.run.smithi060.stderr: 93.2% -- replaced with /var/log/ceph/3df9647e-779b-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-20T22:43:35.601 INFO:teuthology.orchestra.run.smithi060.stderr: 2024-09-20T22:43:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:real 18m8.488s 2024-09-20T22:43:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:user 6m40.581s 2024-09-20T22:43:35.601 INFO:teuthology.orchestra.run.smithi060.stderr:sys 0m19.635s 2024-09-20T22:43:35.602 INFO:tasks.cephadm:Archiving logs... 2024-09-20T22:43:35.603 DEBUG:teuthology.misc:Transferring archived files from smithi060:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-20_21:08:02-orch-squid-distro-default-smithi/7912980/remote/smithi060/log 2024-09-20T22:43:35.603 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-20T22:44:30.441 DEBUG:teuthology.misc:Transferring archived files from smithi067:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-20_21:08:02-orch-squid-distro-default-smithi/7912980/remote/smithi067/log 2024-09-20T22:44:30.442 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-20T22:45:20.230 INFO:tasks.cephadm:Removing cluster... 2024-09-20T22:45:20.231 DEBUG:teuthology.orchestra.run.smithi060:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3df9647e-779b-11ef-baf5-efdc52797490 --force 2024-09-20T22:45:20.491 INFO:teuthology.orchestra.run.smithi060.stdout:Deleting cluster with fsid: 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T22:45:21.180 DEBUG:teuthology.orchestra.run.smithi067:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3df9647e-779b-11ef-baf5-efdc52797490 --force 2024-09-20T22:45:21.445 INFO:teuthology.orchestra.run.smithi067.stdout:Deleting cluster with fsid: 3df9647e-779b-11ef-baf5-efdc52797490 2024-09-20T22:45:22.094 INFO:tasks.cephadm:Removing cephadm ... 2024-09-20T22:45:22.094 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-20T22:45:22.113 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-20T22:45:22.130 INFO:tasks.cephadm:Teardown complete 2024-09-20T22:45:22.131 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-20T22:45:22.145 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-20T22:45:22.145 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-20T22:45:22.156 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-20T22:45:22.271 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-20T22:45:22.279 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-20T22:45:22.280 DEBUG:teuthology.orchestra.run.smithi060:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-20T22:45:22.282 DEBUG:teuthology.orchestra.run.smithi067:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-20T22:45:22.297 INFO:teuthology.orchestra.run.smithi067.stderr:bash: line 1: ntpq: command not found 2024-09-20T22:45:22.297 INFO:teuthology.orchestra.run.smithi060.stderr:bash: line 1: ntpq: command not found 2024-09-20T22:45:22.317 INFO:teuthology.orchestra.run.smithi067.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-20T22:45:22.317 INFO:teuthology.orchestra.run.smithi067.stdout:=============================================================================== 2024-09-20T22:45:22.317 INFO:teuthology.orchestra.run.smithi067.stdout:^* hv01.front.sepia.ceph.com 3 6 377 4 -333us[ -345us] +/- 47ms 2024-09-20T22:45:22.317 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 6 -176us[ -188us] +/- 68ms 2024-09-20T22:45:22.317 INFO:teuthology.orchestra.run.smithi067.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +751us[ +739us] +/- 72ms 2024-09-20T22:45:22.317 INFO:teuthology.orchestra.run.smithi067.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T22:45:22.318 INFO:teuthology.orchestra.run.smithi060.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-20T22:45:22.318 INFO:teuthology.orchestra.run.smithi060.stdout:=============================================================================== 2024-09-20T22:45:22.318 INFO:teuthology.orchestra.run.smithi060.stdout:^* hv01.front.sepia.ceph.com 3 6 377 5 -340us[ -336us] +/- 47ms 2024-09-20T22:45:22.318 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 5 -178us[ -178us] +/- 68ms 2024-09-20T22:45:22.319 INFO:teuthology.orchestra.run.smithi060.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 5 +743us[ +747us] +/- 72ms 2024-09-20T22:45:22.319 INFO:teuthology.orchestra.run.smithi060.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-20T22:45:22.319 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-20T22:45:22.328 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-20T22:45:22.328 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-20T22:45:22.336 DEBUG:teuthology.orchestra.run.smithi060:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-20T22:45:22.475 DEBUG:teuthology.orchestra.run.smithi067:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-20T22:45:22.561 DEBUG:teuthology.orchestra.run.smithi060:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-20T22:45:22.593 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:45:22.593 DEBUG:teuthology.orchestra.run.smithi067:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-20T22:45:22.620 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:45:22.620 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-20T22:45:22.629 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-20T22:45:22.636 INFO:teuthology.task.internal:Duration was 3283.955244 seconds 2024-09-20T22:45:22.636 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-20T22:45:22.644 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-20T22:45:22.644 DEBUG:teuthology.orchestra.run.smithi060:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-20T22:45:22.647 DEBUG:teuthology.orchestra.run.smithi067:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-20T22:45:22.718 INFO:teuthology.orchestra.run.smithi060.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-20T22:45:22.727 INFO:teuthology.orchestra.run.smithi067.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-20T22:45:23.089 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-20T22:45:23.089 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi060.front.sepia.ceph.com 2024-09-20T22:45:23.089 DEBUG:teuthology.orchestra.run.smithi060:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-20T22:45:23.120 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi067.front.sepia.ceph.com 2024-09-20T22:45:23.120 DEBUG:teuthology.orchestra.run.smithi067:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-20T22:45:23.146 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-20T22:45:23.146 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-20T22:45:23.163 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-20T22:45:23.213 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-20T22:45:23.213 DEBUG:teuthology.orchestra.run.smithi060:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-20T22:45:23.233 DEBUG:teuthology.orchestra.run.smithi067:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-20T22:45:25.274 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-20T22:45:25.291 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-20T22:45:25.291 DEBUG:teuthology.orchestra.run.smithi060:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-20T22:45:25.323 DEBUG:teuthology.orchestra.run.smithi067:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-20T22:45:25.352 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-20T22:45:25.362 DEBUG:teuthology.orchestra.run.smithi060:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-20T22:45:25.366 DEBUG:teuthology.orchestra.run.smithi067:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-20T22:45:25.394 INFO:teuthology.orchestra.run.smithi060.stdout:kernel.core_pattern = core 2024-09-20T22:45:25.429 INFO:teuthology.orchestra.run.smithi067.stdout:kernel.core_pattern = core 2024-09-20T22:45:25.447 DEBUG:teuthology.orchestra.run.smithi060:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-20T22:45:25.469 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:45:25.470 DEBUG:teuthology.orchestra.run.smithi067:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-20T22:45:25.503 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-20T22:45:25.504 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-20T22:45:25.514 INFO:teuthology.task.internal:Transferring archived files... 2024-09-20T22:45:25.515 DEBUG:teuthology.misc:Transferring archived files from smithi060:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-20_21:08:02-orch-squid-distro-default-smithi/7912980/remote/smithi060 2024-09-20T22:45:25.515 DEBUG:teuthology.orchestra.run.smithi060:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-20T22:45:25.584 DEBUG:teuthology.misc:Transferring archived files from smithi067:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-20_21:08:02-orch-squid-distro-default-smithi/7912980/remote/smithi067 2024-09-20T22:45:25.584 DEBUG:teuthology.orchestra.run.smithi067:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-20T22:45:25.626 INFO:teuthology.task.internal:Removing archive directory... 2024-09-20T22:45:25.627 DEBUG:teuthology.orchestra.run.smithi060:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-20T22:45:25.629 DEBUG:teuthology.orchestra.run.smithi067:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-20T22:45:25.675 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-20T22:45:25.686 INFO:teuthology.task.internal:Not uploading archives. 2024-09-20T22:45:25.686 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-20T22:45:25.696 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-20T22:45:25.696 DEBUG:teuthology.orchestra.run.smithi060:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-20T22:45:25.699 DEBUG:teuthology.orchestra.run.smithi067:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-20T22:45:25.712 INFO:teuthology.orchestra.run.smithi060.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 20 22:45 /home/ubuntu/cephtest 2024-09-20T22:45:25.732 INFO:teuthology.orchestra.run.smithi067.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 20 22:45 /home/ubuntu/cephtest 2024-09-20T22:45:25.733 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-20T22:45:25.743 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-20T22:45:25.765 INFO:teuthology.run:Summary data: description: orch/cephadm/thrash/{0-distro/centos_9.stream_runc 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} duration: 3283.955244064331 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-20T22:45:25.765 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-20T22:45:25.857 INFO:teuthology.run:pass